FCM Communication Buffers Considerations

The FCM memory requirements are allocated either from the Database Manager Shared Memory (DMSM) or from both the DMSM and the FCM Buffer Shared Memory, depending on whether the partitioned database system uses multiple database partitions on the same server.

If you have a partitioned database system that does not have multiple database partitions on the same server, then for each database partition or each server, the FCM memory requirements are allocated from the DMSM:
```
DMSM = (FCM Connection Retries +
        FCM Message Anchors +
        FCM Request Blocks +
        FCM Buffers)
```
If you have a partitioned database system that does have multiple database partitions on the same server, then for each database partition, the FCM memory requirements are allocated from the DMSM:
```
DMSM = (FCM Connection Retries +
        FCM Message Anchors +
        FCM Request Blocks)
```

For each server or for all database partitions on the same server, one pool of FCM buffers is allocated from the FCM Buffer Shared Memory.

NOTE

By default, the FCM Request Blocks is set to 2048; the FCM Connection Retries and the FCM Message Anchors are 0.75 x FCM Request Blocks.

For example, using the default values, the FCM memory required per database partition is:

(2048 + 1536 + 1536) * 4096 = 5120 * 4096 = 20 MB

The DB2_FORCE_FCM_BP registry variable can be used to allow DB2 to communicate between database partitions on the same server entirely through shared memory, instead of using the high-speed interconnect. If the DB2_FORCE_FCM_BP registry variable is set to YES, the FCM communication buffers are created in a separate memory segment so that communication between the FCM daemons for the different database partitions on the same server will occur through shared memory. Otherwise, FCM daemons on the same server must communicate through UNIX sockets using the high-speed interconnect, even if they are on the same server. Communicating through shared memory is faster, but if DB2 is installed in 32-bit mode, there will be less shared memory available for other uses, particularly for database buffer pools.

To analyze the number of FCM buffers (FCM_NUM_BUFFERS), it is important to look at the current allocation of the number of FCM buffers, as well as the maximum number of FCM buffers that have been allocated, i.e., the low water mark for the free FCM buffers. These numbers can then be compared with the configured number of FCM buffers from the database manager configuration. If the percentage of free FCM buffers drops below 10%, there is a potential that DB2 may run out of available buffers, and this would indicate that the number of FCM buffers should be increased.

Example of Overallocated FCM Communication Buffers

The system is running four database partitions on a single DB2 server with a total of 2 GB of available memory. The DBA decided to add another four database partitions to this server (a total of eight database partitions), and db2 failed to start. Assume that the FCM_NUM_BUFFERS was set to 30,000, due to the high number of concurrent applications.

Observation

For this particular DB2 server, the FCM memory is taken up by FCM communication buffers on a server that has a 256-MB shared segment on AIX and a registry variable DB2_FORCE_FCM_BP set to YES, which means the FCM communication buffer is shared among the database partitions on this server.

When there were only four database partitions, DB2 started successfully. Because the FCM_NUM_BUFFERS was set to 30,000, the FCM memory requirements reserved a total of only 30,000 x 4 KB = 117 MB on this server.

After the DBA added another four database partitions, DB2 failed to start or would hang. The DBA had to reduce the FCM_NUM_BUFFERS down to 24,000 in order to get all eight database partitions to start.

Resolution

Based on the FCM memory structures, the FCM Connection Entries, FCM Message Anchors, and FCM Request Blocks are allocated on a per-partition basis in the same shared memory segment as the FCM communication buffers.

The total of FCM memory for four database partitions on the same server when the FCM_NUM_BUFFERS is set at 30,000 is: 117 + (20 x 4) = 197 MB, and DB2 started successfully.

With the same value for the FCM_NUM_BUFFERS, four additional database partitions were added, requiring the FCM memory to handle a total of eight database partitions on the same server, 117 + (20 x 8) = 277 MB, which is more than the shared memory segment (256 MB), resulting in a failure of DB2 to start.

Therefore, when FCM_NUM_BUFFERS was reduced to 24,000, all eight database partitions started successfully. Because the total of the FCM memory for eight database partitions on the same server is less than the shared memory segment (256 MB):