Recommendations for PostgreSQL


This topic provides recommendations for the appropriate database configuration for PostgreSQL in highly distributed environments. BMC recommends PostgreSQL on Linux server for better performance.

Disk layout

When you run the Create_TS.pgsql.sql script, which is provided for tablespace creation on your database, BMC recommends that you segregate the tablespaces on different disks, as shown in the following table:

Tablespaces for tables

Small

Large

Static

BCM_DATA

BCM_EL

BCM_VMCKB

BCM_VM

BCM_INV

BCM_ESIDKB

BCM_WQ

BCM_DELTAINV

BCM_PATCH

BCM_VMINV

BCM_ESIDDATA

BCM_SCAP

Tablespaces for indexes

Small

Large

Static

BCM_INDEX

BCM_ELINDEX

BCM_VMCKBINDEX

BCM_VMINDEX

BCM_INVINDEX

BCM_ESIDKBINDEX

BCM_WQINDEX

BCM_DELTAINDEX

BCM_PATCHINDEX

BCM_VMINVINDEX

BCM_ESIDDATAINDEX

BCM_SCAPINDEX

Minimum requirement disk layout

Disk

Purpose

Disk 1

/ OS-SYSTEM / POSTGRESQL

Disk 2

/ PG_XLOG - WALL

Disk 3

/ TABLESPACES FOR TABLES

Disk 4

/ TABLESPACES FOR INDEXES

Disk 5

/ TEMPORARY TABLESPACES

Recommended best disk layout

Disk

Purpose

Disk 1

/ OS-SYSTEM / POSTGRESQL

Disk 2

/ PG_XLOG - WALL

Disk 3

/ TABLESPACES FOR SMALL TABLES

Disk 4

/ TABLESPACES FOR LARGE TABLES

Disk 5

/ TABLESPACES FOR STATIC TABLES

Disk 6

/ TABLESPACES FOR SMALL INDEXES

Disk 7

/ TABLESPACES FOR LARGE INDEXES

Disk 8

/ TABLESPACES FOR STATIC INDEXES

Disk 9

/ TEMPORARY TABLESPACES

Storage capacity sizing

When sizing storage capacity for a PostgreSQL system, you need to consider the following PostgreSQL database components:

  • Database files
  • Transaction log files
  • Backup

Database element

5,000 nodes

10,000

nodes

20,000 nodes

50,000 nodes

100,000 nodes

150,000 nodes

200,000 nodes

DB size (1)

5 GB

10 GB

20 GB

50 GB

100 GB

150 GB

200 GB

Tablespaces for tables
DB size * 0.4

2 GB

4 GB

8 GB

20 GB

40 GB

60 GB

80 GB

Tablespaces for index
DB size * 0.6

3 GB

6 GB

12 GB

30 GB

60 GB

90 GB

120 GB

Transaction log

DB size * 0.1

500 MB

1 GB

2 GB

5 GB

10 GB

15 GB

20 GB

Backup

DB size * 0.2

1 GB

2 GB

4 GB

10 GB

20 GB

30 GB

40 GB

Total (GB)

6,500

13

26

65

130

145

260

  1. As a rule of thumb, the size of the database is: number of nodes * 1 MB

System tuning recommendations for PostgreSQL

File access times

Applying the noatime attribute can significantly improve the file I/O performance of your server. Edit the /etc/fstab file and add the noatime attribute as follows:

/dev/sdb1 on / PostgreSQL ext3   rw,noatime
/dev/sdc1 on / Pg_xlog ext3   rw,noatime
/dev/sdd1 on / Tablespaces for tables ext3   rw,noatime
/dev/sde1 on / Tablespaces for indexes ext3 rw,noatime
# repeat for each additional disk

Read ahead

Modifying the Read-Ahead parameter can improve the read performance of a disk.

Type # blockdev --getra /dev/sda to check the Read-Ahead value of an individual disk. The default value is generally 256.

To permanently increase this value to 4096, edit the /etc/rc.local file by adding the following block to the end of the file:

blockdev --setra 4096 /dev/sda
blockdev --setra 4096 /dev/sdb
blockdev --setra 4096 /dev/sdc  
# repeat for each additional disks

Read caching and swapping

Modifying the following parameters increases database performance:

Parameter

Recommendation

vm.dirty_background_ratio

Decrease this value to 5 to make flushes more frequent but result in fewer I/O spikes.

vm.dirty_ratio

Decrease this value to 10 to make flushes more frequent but result in fewer I/O spikes.

vm.swappiness

Deactivate swapping to eliminate the tendency of the kernel to move processes out of physical memory and onto the swap disk.

To permanently change these parameter values, edit the /etc/sysctl.conf file as follows:

vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.swappiness = 0

To set the values without rebooting, run the sysctl -p command after saving and exiting the sysctl.conf file.

Shared memory and semaphores

A large PostgreSQL installation can quickly exhaust various operating system resource limits. The following example shows how to increase the shared memory to 16 GB of RAM:

# sysctl -w kernel.shmmax=8420048896
# sysctl -w kernel.shmall=2055676
# sysctl -w kernel.sem = 250 32000 32 128

Configure PostgreSQL database

  1. Go to  PGTune   and provide basic information about the hardware configuration.
    image2022-1-5_15-31-10.png
  2. Select your DB version from the DB version list.
  3. Select Linux from the OS Type list.
  4. Select Mixed type of applications from the DB Type list.
  5. For Total Memory (RAM) and Number of CPUs see Verify-the-database-server-requirements.
  6. Select 100 from the Number of Connections list.
  7. Select the type of hard drive which PostgreSQL uses as storage from the Data Storage list.
  8. Click Generate and go to the ALTER SYSTEM tab. The configuration for PostgreSQL is calculated based on the provided hardware configuration.
    image2022-1-5_15-31-30.png
  9. Click Copy configuration.
  10. Connect to the PostgreSQL database.

    sudo -u postgresql psql
    postgres=#
    paste the copy configuration
    ALTER SYSTEM SET max_connections = '100';
    ALTER SYSTEM SET shared_buffers = '4GB';
    ALTER SYSTEM SET effective_cache_size = '12GB';
    ALTER SYSTEM SET maintenance_work_mem = '1GB';
    ALTER SYSTEM SET checkpoint_completion_target = '0.9';
    ALTER SYSTEM SET wal_buffers = '16MB';
    ALTER SYSTEM SET default_statistics_target = '100';
    ALTER SYSTEM SET random_page_cost = '4';
    ALTER SYSTEM SET effective_io_concurrency = '2';
    ALTER SYSTEM SET work_mem = '5242kB';
    ALTER SYSTEM SET min_wal_size = '1GB';
    ALTER SYSTEM SET max_wal_size = '4GB';
    ALTER SYSTEM SET max_worker_processes = '8';
    ALTER SYSTEM SET max_parallel_workers_per_gather = '4';
    ALTER SYSTEM SET max_parallel_workers = '8';
    ALTER SYSTEM SET max_parallel_maintenance_workers = '4';
  11. Add the following settings to perform autovacuum.

    ALTER SYSTEM SET bgwriter_lru_maxpages = '1000';
    ALTER SYSTEM SET bgwriter_lru_multiplier = '4.0';
    ALTER SYSTEM SET random_page_cost = '2.0';
    ALTER SYSTEM SET cpu_tuple_cost = '0.03';
    ALTER SYSTEM SET log_autovacuum_min_duration = '0';
    ALTER SYSTEM SET autovacuum_max_workers = '5';
    ALTER SYSTEM SET autovacuum_vacuum_cost_delay = '10ms'
  12. Enter # \q to quit the PostgreSQL database.
  13. Restart the database.

    sudo systemctl restart postgresql-13
  14. To check whether these settings are running, execute the following command.

    sudo -u postgresql psql
    postgres=# show shared_buffers;
    shared_buffers
    ----------------
    4GB

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*