# Amazon RDS for PostgreSQL
Amazon RDS supports DB instances running several versions of PostgreSQL. For a list of available versions, see [Available PostgreSQL database versions](PostgreSQL.Concepts.General.DBVersions.md).
You can create DB instances and DB snapshots, point-in-time restores and backups. DB instances running PostgreSQL support Multi-AZ deployments, read replicas, Provisioned IOPS, and can be created inside a virtual private cloud (VPC). You can also use Secure Socket Layer (SSL) to connect to a DB instance running PostgreSQL.
Before creating a DB instance, make sure to complete the steps in [Setting up your Amazon RDS environment](CHAP_SettingUp.md).
You can use any standard SQL client application to run commands for the instance from your client computer. Such applications include pgAdmin, a popular Open Source administration and development tool for PostgreSQL, or psql, a command line utility that is part of a PostgreSQL installation. To deliver a managed service experience, Amazon RDS doesn't provide host access to DB instances. Also, it restricts access to certain system procedures and tables that require advanced privileges. Amazon RDS supports access to databases on a DB instance using any standard SQL client application. Amazon RDS doesn't allow direct host access to a DB instance by using Telnet or Secure Shell (SSH).
Amazon RDS for PostgreSQL is compliant with many industry standards. For example, you can use Amazon RDS for PostgreSQL databases to build HIPAA-compliant applications and to store healthcare-related information. This includes storage for protected health information (PHI) under a completed Business Associate Agreement (BAA) with AWS. Amazon RDS for PostgreSQL also meets Federal Risk and Authorization Management Program (FedRAMP) security requirements. Amazon RDS for PostgreSQL has received a FedRAMP Joint Authorization Board (JAB) Provisional Authority to Operate (P-ATO) at the FedRAMP HIGH Baseline within the AWS GovCloud (US) Regions. For more information on supported compliance standards, see [AWS cloud compliance](https://aws.amazon.com/compliance/).
To import PostgreSQL data into a DB instance, follow the information in the [Importing data into PostgreSQL on Amazon RDS](PostgreSQL.Procedural.Importing.md) section.
**Important**
If you encounter an issue with your RDS for PostgreSQL DB instance, your AWS support agent might need more information about the health of your databases. The goal is to ensure that AWS Support gets the required information as soon as possible.
You can use PG Collector to help gather valuable database information in a consolidated HTML file. For more information on PG Collector, how to run it, and how to download the HTML report, see [PG Collector](https://github.com/awslabs/pg-collector).
Upon successful completion, and unless otherwise noted, the script returns output in a readable HTML format. The script is designed to exclude any data or security details from the HTML that might compromise your business. It also makes no modifications to your database or its environment. However, if you find any information in the HTML that you are uncomfortable sharing, feel free to remove the problematic information before uploading the HTML. When the HTML is acceptable, upload it using the attachments section in the case details of your support case.
**Topics**
+ [
# Common management tasks for Amazon RDS for PostgreSQL
](CHAP_PostgreSQL.CommonTasks.md)
+ [
# Working with the Database Preview environment
](working-with-the-database-preview-environment.md)
+ [
# Available PostgreSQL database versions
](PostgreSQL.Concepts.General.DBVersions.md)
+ [
# Understanding the RDS for PostgreSQL incremental release process
](PostgreSQL.Concepts.General.ReleaseProcess.md)
+ [
# Supported PostgreSQL extension versions
](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md)
+ [
# Working with PostgreSQL features supported by Amazon RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.md)
+ [
# Connecting to a DB instance running the PostgreSQL database engine
](USER_ConnectToPostgreSQLInstance.md)
+ [
# Securing connections to RDS for PostgreSQL with SSL/TLS
](PostgreSQL.Concepts.General.Security.md)
+ [
# Using Kerberos authentication with Amazon RDS for PostgreSQL
](postgresql-kerberos.md)
+ [
# Using a custom DNS server for outbound network access
](Appendix.PostgreSQL.CommonDBATasks.CustomDNS.md)
+ [
# Upgrades of the RDS for PostgreSQL DB engine
](USER_UpgradeDBInstance.PostgreSQL.md)
+ [
# Upgrading a PostgreSQL DB snapshot engine version
](USER_UpgradeDBSnapshot.PostgreSQL.md)
+ [
# Working with read replicas for Amazon RDS for PostgreSQL
](USER_PostgreSQL.Replication.ReadReplicas.md)
+ [
# Improving query performance for RDS for PostgreSQL with Amazon RDS Optimized Reads
](USER_PostgreSQL.optimizedreads.md)
+ [
# Importing data into PostgreSQL on Amazon RDS
](PostgreSQL.Procedural.Importing.md)
+ [
# Exporting data from an RDS for PostgreSQL DB instance to Amazon S3
](postgresql-s3-export.md)
+ [
# Invoking an AWS Lambda function from an RDS for PostgreSQL DB instance
](PostgreSQL-Lambda.md)
+ [
# Common DBA tasks for Amazon RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.md)
+ [
# Tuning with wait events for RDS for PostgreSQL
](PostgreSQL.Tuning.md)
+ [
# Tuning RDS for PostgreSQL with Amazon DevOps Guru proactive insights
](PostgreSQL.Tuning_proactive_insights.md)
+ [
# Using PostgreSQL extensions with Amazon RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Extensions.md)
+ [
# Working with the supported foreign data wrappers for Amazon RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Extensions.foreign-data-wrappers.md)
+ [
# Working with Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension.md)
# Common management tasks for Amazon RDS for PostgreSQL
The following are the common management tasks you perform with an Amazon RDS for PostgreSQL DB instance, with links to relevant documentation for each task.
| Task area | Relevant documentation |
| --- | --- |
| **Setting up Amazon RDS for first-time use** Before you can create your DB instance, make sure to complete a few prerequisites. For example, DB instances are created by default with a firewall that prevents access to it. So you need to create a security group with the correct IP addresses and network configuration to access the DB instance. | [Setting up your Amazon RDS environment](CHAP_SettingUp.md) |
| **Understanding Amazon RDS DB instances** If you are creating a DB instance for production purposes, you should understand how instance classes, storage types, and Provisioned IOPS work in Amazon RDS. | [DB instance classes](Concepts.DBInstanceClass.md) [Amazon RDS storage types](CHAP_Storage.md#Concepts.Storage) [Provisioned IOPS SSD storage](CHAP_Storage.md#USER_PIOPS) |
| **Finding available PostgreSQL versions** Amazon RDS supports several versions of PostgreSQL. | [Available PostgreSQL database versions](PostgreSQL.Concepts.General.DBVersions.md) |
| **Setting up high availability and failover support** A production DB instance should use Multi-AZ deployments. Multi-AZ deployments provide increased availability, data durability, and fault tolerance for DB instances. | [Configuring and managing a Multi-AZ deployment for Amazon RDS](Concepts.MultiAZ.md) |
| **Understanding the Amazon Virtual Private Cloud (VPC) network** If your AWS account has a default VPC, then your DB instance is automatically created inside the default VPC. In some cases, your account might not have a default VPC, and you might want the DB instance in a VPC. In these cases, create the VPC and subnet groups before you create the DB instance. | [Working with a DB instance in a VPC](USER_VPC.WorkingWithRDSInstanceinaVPC.md) |
| **Importing data into Amazon RDS PostgreSQL** You can use several different tools to import data into your PostgreSQL DB instance on Amazon RDS. | [Importing data into PostgreSQL on Amazon RDS](PostgreSQL.Procedural.Importing.md) |
| **Setting up read-only read replicas (primary and standbys)** RDS for PostgreSQL supports read replicas in both the same AWS Region and in a different AWS Region from the primary instance. | [Working with DB instance read replicas](USER_ReadRepl.md) [Working with read replicas for Amazon RDS for PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.md) [Creating a read replica in a different AWS Region](USER_ReadRepl.XRgn.md) |
| **Understanding security groups** By default, DB instances are created with a firewall that prevents access to them. To provide access through that firewall, you edit the inbound rules for the VPC security group associated with the VPC hosting the DB instance. | [Controlling access with security groups](Overview.RDSSecurityGroups.md) |
| **Setting up parameter groups and features** To change the default parameters for your DB instance, create a custom DB parameter group and change settings to that. If you do this before creating your DB instance, you can choose your custom DB parameter group when you create the instance. | [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md) |
| **Connecting to your PostgreSQL DB instance** After creating a security group and associating it to a DB instance, you can connect to the DB instance using any standard SQL client application such as `psql` or `pgAdmin`. | [Connecting to a DB instance running the PostgreSQL database engine](USER_ConnectToPostgreSQLInstance.md) [Using SSL with a PostgreSQL DB instance](PostgreSQL.Concepts.General.SSL.md) |
| **Backing up and restoring your DB instance** You can configure your DB instance to take automated backups, or take manual snapshots, and then restore instances from the backups or snapshots. | [Backing up, restoring, and exporting data](CHAP_CommonTasks.BackupRestore.md) |
| **Monitoring the activity and performance of your DB instance** You can monitor a PostgreSQL DB instance by using CloudWatch Amazon RDS metrics, events, and enhanced monitoring. | [Viewing metrics in the Amazon RDS console](USER_Monitoring.md) [Viewing Amazon RDS events](USER_ListEvents.md) |
| **Upgrading the PostgreSQL database version** You can do both major and minor version upgrades for your PostgreSQL DB instance. | [Upgrades of the RDS for PostgreSQL DB engine](USER_UpgradeDBInstance.PostgreSQL.md) [Choosing a major version for an RDS for PostgreSQL upgrade](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.md) |
| **Working with log files** You can access the log files for your PostgreSQL DB instance. | [ RDS for PostgreSQL database log files](USER_LogAccess.Concepts.PostgreSQL.md) |
| **Understanding the best practices for PostgreSQL DB instances** Find some of the best practices for working with PostgreSQL on Amazon RDS. | [Best practices for working with PostgreSQL](CHAP_BestPractices.md#CHAP_BestPractices.PostgreSQL) |
Following is a list of other sections in this guide that can help you understand and use important features of RDS for PostgreSQL:
+ [Understanding PostgreSQL roles and permissions](Appendix.PostgreSQL.CommonDBATasks.Roles.md)
+ [Controlling user access to the PostgreSQL databaseControlling user access to PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Access.md)
+ [Working with parameters on your RDS for PostgreSQL DB instance](Appendix.PostgreSQL.CommonDBATasks.Parameters.md)
+ [Understanding logging mechanisms supported by RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.md#Appendix.PostgreSQL.CommonDBATasks.Auditing)
+ [Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md)
+ [Using a custom DNS server for outbound network access](Appendix.PostgreSQL.CommonDBATasks.CustomDNS.md)
# Working with the Database Preview environment
The PostgreSQL community continuously releases new PostgreSQL version and extensions, including beta versions. This gives PostgreSQL users the opportunity to try out a new PostgreSQL version early. To learn more about the PostgreSQL community beta release process, see [Beta Information](https://www.postgresql.org/developer/beta/) in the PostgreSQL documentation. Similarly, Amazon RDS makes certain PostgreSQL beta versions available as Preview releases. This allows you to create DB instances using the Preview version and test out its features in the Database Preview Environment.
RDS for PostgreSQL DB instances in the Database Preview Environment are functionally similar to other RDS for PostgreSQL instances. However, you can't use a Preview version for production.
Keep in mind the following important limitations:
+ All DB instances are deleted 60 days after you create them, along with any backups and snapshots.
+ You can only create a DB instance in a virtual private cloud (VPC) based on the Amazon VPC service.
+ You can only use General Purpose SSD and Provisioned IOPS SSD storage.
+ You can't get help from AWS Support with DB instances. Instead, you can post your questions to the AWS‐managed Q&A community, [AWS re:Post](https://repost.aws/tags/TAsibBK6ZeQYihN9as4S_psg/amazon-relational-database-service).
+ You can't copy a snapshot of a DB instance to a production environment.
The following options are supported by the Preview.
+ You can create DB instances using M6i, R6i, M6g, M5, T3, R6g, and R5 instance types only. For more information about RDS instance classes, see [DB instance classes](Concepts.DBInstanceClass.md).
+ You can use both single-AZ and multi-AZ deployments.
+ You can use standard PostgreSQL dump and load functions to export databases from or import databases to the Database Preview Environment.
**Topics**
+ [
## Features not supported in the Database Preview environment
](#preview-environment-exclusions)
+ [
## PostgreSQL version 17 in the Database Preview environment
](#PostgreSQL.Concepts.General.version17)
+ [
# Creating a new DB instance in the Database Preview environment
](create-db-instance-in-preview-environment.md)
## Features not supported in the Database Preview environment
The following features aren't available in the Database Preview environment:
+ Cross-Region snapshot copy
+ Cross-Region read replicas
## PostgreSQL version 17 in the Database Preview environment
**Note**
This is preview documentation for Amazon RDS PostgreSQL version 17. It is subject to change.
PostgreSQL version 17.0 is now available in the Amazon RDS Database Preview environment. PostgreSQL version 17.0 contains several improvements that are described in the following PostgreSQL documentation, [PostgreSQL 17 Released\$1](https://www.postgresql.org/docs/17/release-17.html)
For information on the Database Preview Environment, see [Working with the Database Preview environment](#working-with-the-database-preview-environment). To access the Preview Environment from the console, select [https://console.aws.amazon.com/rds-preview/](https://console.aws.amazon.com/rds-preview/).
# Creating a new DB instance in the Database Preview environment
Use the following procedure to create a DB instance in the preview environment.
**To create a DB instance in the Database Preview environment**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose **Dashboard** from the navigation pane.
1. In the Dashboard page, locate the **Database Preview Environment** section on the Dashboard page, as shown in the following image.
![\[Preview environment section with link displayed in RDS Console, Dashboard\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/preview-environment-dashboard.png)
You can navigate directly to the [Database Preview environment](https://us-east-2.console.aws.amazon.com/rds-preview/home?region=us-east-2#). Before you can proceed, you must acknowledge and accept the limitations.
![\[Preview environment limitations dialog\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/preview-environment-console.png)
1. To create the RDS for PostgreSQL DB instance, follow the same process as that for creating any Amazon RDS DB instance. For more information, see the [Console](USER_CreateDBInstance.md#USER_CreateDBInstance.CON) procedure in [Creating a DB instance](USER_CreateDBInstance.md#USER_CreateDBInstance.Creating).
To create an instance in the Database Preview Environment using the RDS API or the AWS CLI, use the following endpoint.
```
rds-preview.us-east-2.amazonaws.com
```
# Available PostgreSQL database versions
Amazon RDS supports DB instances running several editions of PostgreSQL. You can specify any currently available PostgreSQL version when creating a new DB instance. You can specify the major version (such as PostgreSQL 14), and any available minor version for the specified major version. If no version is specified, Amazon RDS defaults to an available version, typically the most recent version. If a major version is specified but a minor version is not, Amazon RDS defaults to a recent release of the major version you have specified.
To see a list of available versions, as well as defaults for newly created DB instances, use the [https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-engine-versions.html](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-engine-versions.html) AWS CLI command. For example, to display the default PostgreSQL engine version, use the following command:
```
aws rds describe-db-engine-versions --default-only --engine postgres
```
For details about the PostgreSQL versions that are supported on Amazon RDS, see the [https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/Welcome.html](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/Welcome.html). You can also view information about support dates for major engine versions by running the [describe-db-major-engine-versions](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-major-engine-versions.html) AWS CLI command or by using the [DescribeDBMajorEngineVersions](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBMajorEngineVersions.html) RDS API operation.
If you aren't ready to manually upgrade to a new major engine version before the RDS end of standard support date, Amazon RDS will automatically enroll your databases in Amazon RDS Extended Support after the RDS end of standard support date. Then, you can continue to run RDS for PostgreSQL version 11 and higher. For more information, see [Amazon RDS Extended Support with Amazon RDS](extended-support.md) and [Amazon RDS pricing](https://aws.amazon.com/rds/pricing/).
## Deprecated versions for Amazon RDS for PostgreSQL
Note the following deprecated versions:
+ RDS for PostgreSQL 10 was deprecated in February 2023.
+ RDS for PostgreSQL 9.6 was deprecated in March 2022.
+ RDS for PostgreSQL 9.5 was deprecated in March 2021.
To learn more about deprecation policy for RDS for PostgreSQL, see [Amazon RDS FAQs](https://aws.amazon.com/rds/faqs/). For more information about PostgreSQL versions, see [Versioning Policy](https://www.postgresql.org/support/versioning/) in the PostgreSQL documentation.
# Understanding the RDS for PostgreSQL incremental release process
RDS for PostgreSQL delivers security fixes, performance improvements, and new features through incremental releases while maintaining minor version compatibility. These releases are labeled as R1, R2, R3, and so on.
**Release version naming convention**
+ R1 is the initial release of a minor version. It occasionally includes new features, extensions, or upgrades to existing extensions.
+ Subsequent release versions (R2, R3, and later) include:
+ Security updates
+ Performance improvements
+ Bug fixes
+ Extension updates
## Advantages of RDS for PostgreSQL incremental release process
The incremental release process provides the following advantages:
+ Quick adoption of new PostgreSQL community releases while separately managing RDS-specific enhancements through subsequent releases. This streamlines the release process and ensures faster delivery of critical updates.
+ Access to bug fixes, new features, security updates, and extension updates while maintaining compatibility with the PostgreSQL minor version.
## Managing release updates
Amazon RDS notifies you about new incremental releases through pending maintenance actions in the AWS Management Console. You can update your database using one of these methods:
+ Enable automatic updates during scheduled maintenance windows.
+ Apply updates manually through pending maintenance actions.
+ Use Blue/Green deployments with physical replication to minimize downtime. For more information, see [Blue/Green Deployments support minor version upgrade for RDS for PostgreSQL](https://aws.amazon.com/about-aws/whats-new/2024/11/rds-blue-green-deployments-upgrade-rds-postgresql/).
Before updating your database, consider the following key points:
+ Database reboots are required for updates unless you use Blue/Green deployments with physical replication.
+ Some incremental releases are mandatory, particularly those that include security fixes.
For more information about updating your Amazon RDS DB instance instance, see [PostgreSQL trusted extensions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md#PostgreSQL.Concepts.General.Extensions.Trusted) and [apply-pending-maintenance-action](https://docs.aws.amazon.com/cli/latest/reference/rds/apply-pending-maintenance-action.html).
# Supported PostgreSQL extension versions
RDS for PostgreSQL supports many PostgreSQL extensions. The PostgreSQL community sometimes refers to these as modules. Extensions expand on the functionality provided by the PostgreSQL engine. You can find a list of extensions supported by Amazon RDS in the default DB parameter group for that PostgreSQL version. You can also see the current extensions list using `psql` by showing the `rds.extensions` parameter as in the following example.
```
SHOW rds.extensions;
```
**Note**
Parameters added in a minor version release might display inaccurately when using the `rds.extensions` parameter in `psql`.
As of RDS for PostgreSQL 13, certain extensions can be installed by database users other than the `rds_superuser`. These are known as *trusted extensions*. To learn more, see [PostgreSQL trusted extensions](#PostgreSQL.Concepts.General.Extensions.Trusted).
Certain versions of RDS for PostgreSQL support the `rds.allowed_extensions` parameter. This parameter lets an `rds_superuser` limit the extensions that can be installed in the RDS for PostgreSQL DB instance. For more information, see [Restricting installation of PostgreSQL extensions](#PostgreSQL.Concepts.General.FeatureSupport.Extensions.Restriction).
For lists of PostgreSQL extensions and versions that are supported by each available RDS for PostgreSQL version, see [PostgreSQL extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html) in *Amazon RDS for PostgreSQL Release Notes*.
## Restricting installation of PostgreSQL extensions
You can restrict which extensions can be installed on a PostgreSQL DB instance. By default, this parameter isn't set, so any supported extension can be added if the user has permissions to do so. To do so, set the `rds.allowed_extensions` parameter to a string of comma-separated extension names. By adding a list of extensions to this parameter, you explicitly identify the extensions that your RDS for PostgreSQL DB instance can use. Only these extensions can then be installed in the PostgreSQL DB instance.
The default string for the `rds.allowed_extensions` parameter is '\$1', which means that any extension available for the engine version can be installed. Changing the `rds.allowed_extensions` parameter does not require a database restart because it's a dynamic parameter.
The PostgreSQL DB instance engine must be one of the following versions for you to use the `rds.allowed_extensions` parameter:
+ All PostgreSQL 16 versions
+ PostgreSQL 15 and all higher versions
+ PostgreSQL 14 and all higher versions
+ PostgreSQL 13.3 and higher minor versions
+ PostgreSQL 12.7 and higher minor versions
To see which extension installations are allowed, use the following psql command.
```
postgres=> SHOW rds.allowed_extensions;
rds.allowed_extensions
------------------------
*
```
If an extension was installed prior to it being left out of the list in the `rds.allowed_extensions` parameter, the extension can still be used normally, and commands such as `ALTER EXTENSION` and `DROP EXTENSION` will continue to work. However, after an extension is restricted, `CREATE EXTENSION` commands for the restricted extension will fail.
Installation of extension dependencies with `CREATE EXTENSION CASCADE` are also restricted. The extension and its dependencies must be specified in `rds.allowed_extensions`. If an extension dependency installation fails, the entire `CREATE EXTENSION CASCADE` statement will fail.
If an extension is not included with the `rds.allowed_extensions` parameter, you will see an error such as the following if you try to install it.
```
ERROR: permission denied to create extension "extension-name"
HINT: This extension is not specified in "rds.allowed_extensions".
```
## PostgreSQL trusted extensions
To install most PostgreSQL extensions requires `rds_superuser` privileges. PostgreSQL 13 introduced trusted extensions, which reduce the need to grant `rds_superuser` privileges to regular users. With this feature, users can install many extensions if they have the `CREATE` privilege on the current database instead of requiring the `rds_superuser` role. For more information, see the SQL [CREATE EXTENSION](https://www.postgresql.org/docs/current/sql-createextension.html) command in the PostgreSQL documentation.
The following lists the extensions that can be installed by a user who has the `CREATE` privilege on the current database and do not require the `rds_superuser` role:
+ bool\$1plperl
+ [btree\$1gin](http://www.postgresql.org/docs/current/btree-gin.html)
+ [btree\$1gist](http://www.postgresql.org/docs/current/btree-gist.html)
+ [citext ](http://www.postgresql.org/docs/current/citext.html)
+ [cube ](http://www.postgresql.org/docs/current/cube.html)
+ [ dict\$1int ](http://www.postgresql.org/docs/current/dict-int.html)
+ [fuzzystrmatch](http://www.postgresql.org/docs/current/fuzzystrmatch.html)
+ [hstore](http://www.postgresql.org/docs/current/hstore.html)
+ [ intarray](http://www.postgresql.org/docs/current/intarray.html)
+ [isn](http://www.postgresql.org/docs/current/isn.html)
+ jsonb\$1plperl
+ [ltree ](http://www.postgresql.org/docs/current/ltree.html)
+ [pg\$1trgm](http://www.postgresql.org/docs/current/pgtrgm.html)
+ [pgcrypto](http://www.postgresql.org/docs/current/pgcrypto.html)
+ [plperl](https://www.postgresql.org/docs/current/plperl.html)
+ [plpgsql](https://www.postgresql.org/docs/current/plpgsql.html)
+ [pltcl](https://www.postgresql.org/docs/current/pltcl-overview.html)
+ [tablefunc](http://www.postgresql.org/docs/current/tablefunc.html)
+ [tsm\$1system\$1rows](https://www.postgresql.org/docs/current/tsm-system-rows.html)
+ [tsm\$1system\$1time](https://www.postgresql.org/docs/current/tsm-system-time.html)
+ [unaccent](http://www.postgresql.org/docs/current/unaccent.html)
+ [uuid-ossp](http://www.postgresql.org/docs/current/uuid-ossp.html)
For lists of PostgreSQL extensions and versions that are supported by each available RDS for PostgreSQL version, see [PostgreSQL extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html) in *Amazon RDS for PostgreSQL Release Notes*.
# Working with PostgreSQL features supported by Amazon RDS for PostgreSQL
Amazon RDS for PostgreSQL supports many of the most common PostgreSQL features. For example, PostgreSQL has an autovacuum feature that performs routine maintenance on the database. The autovacuum feature is active by default. Although you can turn off this feature, we highly recommend that you keep it on. Understanding this feature and what you can do to make sure it works as it should is a basic task of any DBA. For more information about the autovacuum, see [Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md). To learn more about other common DBA tasks, [Common DBA tasks for Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.md).
RDS for PostgreSQL also supports extensions that add important functionality to the DB instance. For example, you can use the PostGIS extension to work with spatial data, or use the pg\$1cron extension to schedule maintenance from within the instance. For more information about PostgreSQL extensions, see [Using PostgreSQL extensions with Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Extensions.md).
Foreign data wrappers are a specific type of extension designed to let your RDS for PostgreSQL DB instance work with other commercial databases or data types. For more information about foreign data wrappers supported by RDS for PostgreSQL, see [Working with the supported foreign data wrappers for Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Extensions.foreign-data-wrappers.md).
Following, you can find information about some other features supported by RDS for PostgreSQL.
**Topics**
+ [
# Custom data types and enumerations with RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.AlterEnum.md)
+ [
# Event triggers for RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.EventTriggers.md)
+ [
# Huge pages for RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.HugePages.md)
+ [
# Performing logical replication for Amazon RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.LogicalReplication.md)
+ [
# Configuring IAM authentication for logical replication connections
](PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.md)
+ [
# RAM disk for the stats\$1temp\$1directory
](PostgreSQL.Concepts.General.FeatureSupport.RamDisk.md)
+ [
# Tablespaces for RDS for PostgreSQL
](PostgreSQL.Concepts.General.FeatureSupport.Tablespaces.md)
+ [
# RDS for PostgreSQL collations for EBCDIC and other mainframe migrations
](PostgreSQL.Collations.mainframe.migration.md)
+ [
# Managing logical slot synchronization for RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.pglogical.slot.synchronization.md)
# Custom data types and enumerations with RDS for PostgreSQL
PostgreSQL supports creating custom data types and working with enumerations. For more information about creating and working with enumerations and other data types, see [Enumerated types](https://www.postgresql.org/docs/14/datatype-enum.html) in the PostgreSQL documentation.
The following is an example of creating a type as an enumeration and then inserting values into a table.
```
CREATE TYPE rainbow AS ENUM ('red', 'orange', 'yellow', 'green', 'blue', 'purple');
CREATE TYPE
CREATE TABLE t1 (colors rainbow);
CREATE TABLE
INSERT INTO t1 VALUES ('red'), ( 'orange');
INSERT 0 2
SELECT * from t1;
colors
--------
red
orange
(2 rows)
postgres=> ALTER TYPE rainbow RENAME VALUE 'red' TO 'crimson';
ALTER TYPE
postgres=> SELECT * from t1;
colors
---------
crimson
orange
(2 rows)
```
# Event triggers for RDS for PostgreSQL
All current PostgreSQL versions support event triggers, and so do all available versions of RDS for PostgreSQL. You can use the main user account (default, `postgres`) to create, modify, rename, and delete event triggers. Event triggers are at the DB instance level, so they can apply to all databases on an instance.
For example, the following code creates an event trigger that prints the current user at the end of every data definition language (DDL) command.
```
CREATE OR REPLACE FUNCTION raise_notice_func()
RETURNS event_trigger
LANGUAGE plpgsql AS
$$
BEGIN
RAISE NOTICE 'In trigger function: %', current_user;
END;
$$;
CREATE EVENT TRIGGER event_trigger_1
ON ddl_command_end
EXECUTE PROCEDURE raise_notice_func();
```
For more information about PostgreSQL event triggers, see [Event triggers](https://www.postgresql.org/docs/current/static/event-triggers.html) in the PostgreSQL documentation.
There are several limitations to using PostgreSQL event triggers on Amazon RDS. These include the following:
+ You can't create event triggers on read replicas. You can, however, create event triggers on a read replica source. The event triggers are then copied to the read replica. The event triggers on the read replica don't fire on the read replica when changes are pushed from the source. However, if the read replica is promoted, the existing event triggers fire when database operations occur.
+ To perform a major version upgrade to a PostgreSQL DB instance that uses event triggers, make sure to delete the event triggers before you upgrade the instance.
# Huge pages for RDS for PostgreSQL
*Huge pages* are a memory management feature that reduces overhead when a DB instance is working with large contiguous chunks of memory, such as that used by shared buffers. This PostgreSQL feature is supported by all currently available RDS for PostgreSQL versions. You allocate huge pages for your application by using calls to `mmap` or `SYSV` shared memory. RDS for PostgreSQL supports both 4-KB and 2-MB page sizes.
You can turn huge pages on or off by changing the value of the `huge_pages` parameter. The feature is turned on by default for all the DB instance classes other than micro, small, and medium DB instance classes.
RDS for PostgreSQL uses huge pages based on the available shared memory. If the DB instance can't use huge pages due to shared memory constraints, Amazon RDS prevents the DB instance from starting. In this case, Amazon RDS sets the status of the DB instance to an incompatible parameters state. If this occurs, you can set the `huge_pages` parameter to `off` to allow Amazon RDS to start the DB instance.
The `shared_buffers` parameter is key to setting the shared memory pool that is required for using huge pages. The default value for the `shared_buffers` parameter uses a database parameters macro. This macro sets a percentage of the total 8 KB pages available for the DB instance's memory. When you use huge pages, those pages are located with the huge pages. Amazon RDS puts a DB instance into an incompatible parameters state if the shared memory parameters are set to require more than 90 percent of the DB instance memory.
To learn more about PostgreSQL memory management, see [Resource Consumption](https://www.postgresql.org/docs/current/static/runtime-config-resource.html) in the PostgreSQL documentation.
# Performing logical replication for Amazon RDS for PostgreSQL
Starting with version 10.4, RDS for PostgreSQL supports the publication and subscription SQL syntax that was introduced in PostgreSQL 10. To learn more, see [Logical replication](https://www.postgresql.org/docs/current/logical-replication.html) in the PostgreSQL documentation.
**Note**
In addition to the native PostgreSQL logical replication feature introduced in PostgreSQL 10, RDS for PostgreSQL also supports the `pglogical` extension. For more information, see [Using pglogical to synchronize data across instances](Appendix.PostgreSQL.CommonDBATasks.pglogical.md).
Following, you can find information about setting up logical replication for an RDS for PostgreSQL DB instance.
**Topics**
+ [
## Understanding logical replication and logical decoding
](#PostgreSQL.Concepts.General.FeatureSupport.LogicalDecoding)
+ [
## Working with logical replication slots
](#PostgreSQL.Concepts.General.FeatureSupport.LogicalReplicationSlots)
+ [
## Replicating table level data using logical replication
](#PostgreSQL.Concepts.LogicalReplication.Tables)
## Understanding logical replication and logical decoding
RDS for PostgreSQL supports the streaming of write-ahead log (WAL) changes using PostgreSQL's logical replication slots. It also supports using logical decoding. You can set up logical replication slots on your instance and stream database changes through these slots to a client such as `pg_recvlogical`. You create logical replication slots at the database level, and they support replication connections to a single database.
The most common clients for PostgreSQL logical replication are AWS Database Migration Service or a custom-managed host on an Amazon EC2 instance. The logical replication slot has no information about the receiver of the stream. Also, there's no requirement that the target be a replica database. If you set up a logical replication slot and don't read from the slot, data can be written and quickly fill up your DB instance's storage.
You turn on PostgreSQL logical replication and logical decoding for Amazon RDS with a parameter, a replication connection type, and a security role. The client for logical decoding can be any client that can establish a replication connection to a database on a PostgreSQL DB instance.
**To turn on logical decoding for an RDS for PostgreSQL DB instance**
1. Make sure that the user account that you're using has these roles:
+ The `rds_superuser` role so you can turn on logical replication
+ The `rds_replication` role to grant permissions to manage logical slots and to stream data using logical slots
1. Set the `rds.logical_replication` static parameter to 1. As part of applying this parameter, also set the parameters `wal_level`, `max_wal_senders`, `max_replication_slots`, and `max_connections`. These parameter changes can increase WAL generation, so set the `rds.logical_replication` parameter only when you are using logical slots.
1. Reboot the DB instance for the static `rds.logical_replication` parameter to take effect.
1. Create a logical replication slot as explained in the next section. This process requires that you specify a decoding plugin. Currently, RDS for PostgreSQL supports the test\$1decoding and wal2json output plugins that ship with PostgreSQL.
For more information on PostgreSQL logical decoding, see the [ PostgreSQL documentation](https://www.postgresql.org/docs/current/static/logicaldecoding-explanation.html).
## Working with logical replication slots
You can use SQL commands to work with logical slots. For example, the following command creates a logical slot named `test_slot` using the default PostgreSQL output plugin `test_decoding`.
```
SELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding');
slot_name | xlog_position
-----------------+---------------
regression_slot | 0/16B1970
(1 row)
```
To list logical slots, use the following command.
```
SELECT * FROM pg_replication_slots;
```
To drop a logical slot, use the following command.
```
SELECT pg_drop_replication_slot('test_slot');
pg_drop_replication_slot
-----------------------
(1 row)
```
For more examples on working with logical replication slots, see [ Logical decoding examples](https://www.postgresql.org/docs/9.5/static/logicaldecoding-example.html) in the PostgreSQL documentation.
After you create the logical replication slot, you can start streaming. The following example shows how logical decoding is controlled over the streaming replication protocol. This example uses the program pg\$1recvlogical, which is included in the PostgreSQL distribution. Doing this requires that client authentication is set up to allow replication connections.
```
pg_recvlogical -d postgres --slot test_slot -U postgres
--host -instance-name.111122223333.aws-region.rds.amazonaws.com
-f - --start
```
To see the contents of the `pg_replication_origin_status` view, query the `pg_show_replication_origin_status` function.
```
SELECT * FROM pg_show_replication_origin_status();
local_id | external_id | remote_lsn | local_lsn
----------+-------------+------------+-----------
(0 rows)
```
## Replicating table level data using logical replication
You can use logical replication to replicate data from source tables to target tables in RDS for PostgreSQL. Logical replication first performs an initial load of existing data from the source tables and then continues to replicate ongoing changes.
1.
**Create the source tables**
Connect to the source database in your RDS for PostgreSQL DB instance:
```
source=> CREATE TABLE testtab (slno int primary key);
CREATE TABLE
```
1.
**Insert data into the source tables**
```
source=> INSERT INTO testtab VALUES (generate_series(1,1000));
INSERT 0 1000
```
1.
**Create a publication for source tables**
+ Create a publication for the source tables:
```
source=> CREATE PUBLICATION testpub FOR TABLE testtab;
CREATE PUBLICATION
```
+ Use a SELECT query to verify the details of the publication that was created:
```
source=> SELECT * FROM pg_publication;
oid | pubname | pubowner | puballtables | pubinsert | pubupdate | pubdelete | pubtruncate | pubviaroot
--------+---------+----------+--------------+-----------+-----------+-----------+-------------+------------
115069 | testpub | 16395 | f | t | t | t | t | f
(1 row)
```
+ Verify that the source tables are added to the publication:
```
source=> SELECT * FROM pg_publication_tables;
pubname | schemaname | tablename
---------+------------+-----------
testpub | public | testtab
(1 rows)
```
+ To replicate all tables in a database, use:
```
CREATE PUBLICATION testpub FOR ALL TABLES;
```
+ If the publication is already created for individual table and you need to add new table, you can run below query to add any new tables into the existing publication:
```
ALTER PUBLICATION add table ;
```
1.
**Connect to target database and create target tables**
+ Connect to the target database in the target DB instance. Create the target tables with the same names as the source tables:
```
target=> CREATE TABLE testtab (slno int primary key);
CREATE TABLE
```
+ Make sure that there's no data present in the target tables by running a SELECT query on the target tables:
```
target=> SELECT count(*) FROM testtab;
count
-------
0
(1 row)
```
1.
**Create and verify subscription in target database**
+ Create the subscription in the target database:
```
target=> CREATE SUBSCRIPTION testsub
CONNECTION 'host= port=5432 dbname= user= password='
PUBLICATION testpub;
NOTICE: Created replication slot "testsub" on publisher
CREATE SUBSCRIPTION
```
+ Use a SELECT query to verify that the subscription is enabled:
```
target=> SELECT oid, subname, subenabled, subslotname, subpublications FROM pg_subscription;
oid | subname | subenabled | subslotname | subpublications
-------+---------+------------+-------------+-----------------
16434 | testsub | t | testsub | {testpub}
(1 row)
```
+ When the subscription is created, it loads all data from the source tables to the target tables. Run a SELECT query on the target tables to verify that the initial data loads:
```
target=> SELECT count(*) FROM testtab;
count
-------
1000
(1 row)
```
1.
**Verify replication slot in source database**
The creation of a subscription in the target database creates a replication slot in the source database. Verify the replication slot details by running the following SELECT query on the source database:
```
source=> SELECT * FROM pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn | wal_status | safe_wal_size
----------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------+------------+---------------
testsub | pgoutput | logical | 115048 | source | f | t | 846 | | 6945 | 58/B4000568 | 58/B40005A0 | reserved |
(1 row)
```
1.
**Testing replication**
+ Test whether data changes in the source tables are being replicated to the target tables by inserting rows into the source tables:
```
source=> INSERT INTO testtab VALUES(generate_series(1001,2000));
INSERT 0 1000
source=> SELECT count(*) FROM testtab;
count
-------
2000
(1 row)
```
+ Verify the number of rows in the target tables to confirm that new inserts are being replicated:
```
target=> SELECT count(*) FROM testtab;
count
-------
2000
(1 row)
```
1.
**Refreshing the subscription after adding tables**
+ When you add new tables to an existing publication, it is mandatory to refresh the subscription for the changes to take effect:
```
ALTER SUBSCRIPTION REFRESH PUBLICATION;
```
+ This command fetches missing table information from the publisher and starts replication for tables that were added to the subscribed-to publications since the subscription was created or last refreshed.
# Configuring IAM authentication for logical replication connections
Starting with RDS for PostgreSQL versions 11 and higher, you can use AWS Identity and Access Management (IAM) authentication for replication connections. This feature enhances security by allowing you to manage database access using IAM roles instead of passwords. It works both at the cluster and instance granularity and follows the same security model as standard IAM authentication.
IAM authentication for replication connections is an opt-in feature. To enable it, set the `rds.iam_auth_for_replication` parameter to 1 in your DB cluster or DB parameter group. As this is a dynamic parameter, your DB cluster or instance doesn't need to restart, enabling you to leverage IAM authentication with existing workloads without downtime. Before enabling this feature, you must meet the Prerequisites listed below.
**Topics**
+ [
## Prerequisites
](#PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.Prerequisites)
+ [
## Enabling IAM authentication for replication connections
](#PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.Enabling)
+ [
## Disabling IAM authentication for replication connections
](#PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.Disabling)
+ [
## Limitations and considerations
](#PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.Limitations)
## Prerequisites
To use IAM authentication for replication connections, you need to meet all of the following requirements:
+ Your RDS for PostgreSQL DB instance must be version 11 or later.
+ On your publisher RDS for PostgreSQL DB instance:
+ Enable IAM database authentication. For more information, see [Enabling and disabling IAM database authentication](UsingWithRDS.IAMDBAuth.Enabling.md).
+ Enable logical replication by setting the `rds.logical_replication` parameter to 1.
In logical replication, the publisher is the source RDS for PostgreSQL database that sends data to subscriber database. For more information, see [Performing logical replication for Amazon RDS for PostgreSQL](PostgreSQL.Concepts.General.FeatureSupport.LogicalReplication.md).
**Note**
Both IAM authentication and logical replication must be enabled on your publisher RDS for PostgreSQL DB instance. If either one isn't enabled, you can't use IAM authentication for replication connections.
## Enabling IAM authentication for replication connections
Complete the following steps to enable IAM authentication for replication connection.
**To enable IAM authentication for replication connections**
1. Verify that your RDS for PostgreSQL DB cluster or instance meets all prerequisites for IAM authentication with replication connections. For details, see [Prerequisites](#PostgreSQL.Concepts.General.FeatureSupport.IAMLogicalReplication.Prerequisites).
1. Configure the `rds.iam_auth_for_replication` parameter based on your RDS for PostgreSQL setup:
+ For RDS for PostgreSQL DB instances: Modify your DB parameter group.
+ For Multi-AZ clusters: Modify your DB cluster parameter group.
Set `rds.iam_auth_for_replication` to 1. This is a dynamic parameter that takes effect immediately without requiring a reboot.
**Note**
Multi-AZ clusters use only DB cluster parameter groups. Individual instance parameter groups cannot be modified in Multi-AZ clusters.
1. Connect to your database and grant the necessary roles to your replication user:
The following SQL commands grant the necessary roles to enable IAM authentication for replication connections:
```
-- Grant IAM authentication role
GRANT rds_iam TO replication_user_name;
-- Grant replication privileges
ALTER USER replication_user_name WITH REPLICATION;
```
After you complete these steps, the specified user must use IAM authentication for replication connections.
**Important**
When you enable the feature, users with both `rds_iam` and `rds_replication` roles must use IAM authentication for replication connections. This applies whether the roles are assigned directly to the user or inherited through other roles.
## Disabling IAM authentication for replication connections
You can disable IAM authentication for replication connections by using any of the following methods:
+ Set the `rds.iam_auth_for_replication` parameter to 0 in your DB parameter group for DB instances or DB cluster parameter group for Multi-AZ clusters.
+ Alternatively, you can disable either of these features on your RDS for PostgreSQL DB cluster or instance:
+ Disable logical replication by setting the `rds.logical_replication` parameter to 0
+ Disable IAM authentication
When you disable the feature, replication connections can use database passwords for authentication.
**Note**
Replication connections for users without the `rds_iam` role can use password authentication even when the feature is enabled.
## Limitations and considerations
Consider the following limitations and considerations when using IAM authentication for logical replication connections:
+ This feature is available only for RDS for PostgreSQL versions 11 and higher.
+ The publisher must support IAM authentication for replication connections.
+ The IAM authentication token expires after 15 minutes by default. You might need to refresh long-running replication connections before the token expires.
# RAM disk for the stats\$1temp\$1directory
You can use the RDS for PostgreSQL parameter `rds.pg_stat_ramdisk_size` to specify the system memory allocated to a RAM disk for storing the PostgreSQL `stats_temp_directory`. The RAM disk parameter is only available in RDS for PostgreSQL version 14 and lower versions.
Under certain workloads, setting this parameter can improve performance and decrease I/O requirements. For more information about the `stats_temp_directory`, see [ the PostgreSQL documentation.](https://www.postgresql.org/docs/current/static/runtime-config-statistics.html#GUC-STATS-TEMP-DIRECTORY).
To set up a RAM disk for your `stats_temp_directory`, set the `rds.pg_stat_ramdisk_size` parameter to an integer literal value in the parameter group used by your DB instance. This parameter denotes MB, so you must use an integer value. Expressions, formulas, and functions aren't valid for the `rds.pg_stat_ramdisk_size` parameter. Be sure to reboot the DB instance so that the change takes effect. For information about setting parameters, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
For example, the following AWS CLI command sets the RAM disk parameter to 256 MB.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name pg-95-ramdisk-testing \
--parameters "ParameterName=rds.pg_stat_ramdisk_size, ParameterValue=256, ApplyMethod=pending-reboot"
```
After you reboot, run the following command to see the status of the `stats_temp_directory`.
```
postgres=> SHOW stats_temp_directory;
```
The command should return the following.
```
stats_temp_directory
---------------------------
/rdsdbramdisk/pg_stat_tmp
(1 row)
```
# Tablespaces for RDS for PostgreSQL
RDS for PostgreSQL supports tablespaces for compatibility. Because all storage is on a single logical volume, you can't use tablespaces for I/O splitting or isolation. Our benchmarks and experience indicate that a single logical volume is the best setup for most use cases.
To create and use tablespaces with your RDS for PostgreSQL DB instance requires the `rds_superuser` role. Your RDS for PostgreSQL DB instance's main user account (default name, `postgres`) is a member of this role. For more information, see [Understanding PostgreSQL roles and permissions](Appendix.PostgreSQL.CommonDBATasks.Roles.md).
If you specify a file name when you create a tablespace, the path prefix is `/rdsdbdata/db/base/tablespace`. The following example places tablespace files in `/rdsdbdata/db/base/tablespace/data`. This example assumes that a `dbadmin` user (role) exists and that it's been granted the `rds_superuser` role needed to work with tablespaces.
```
postgres=> CREATE TABLESPACE act_data
OWNER dbadmin
LOCATION '/data';
CREATE TABLESPACE
```
To learn more about PostgreSQL tablespaces, see [Tablespaces](https://www.postgresql.org/docs/current/manage-ag-tablespaces.html) in the PostgreSQL documentation.
# RDS for PostgreSQL collations for EBCDIC and other mainframe migrations
RDS for PostgreSQL versions 10 and higher include ICU version 60.2, which is based on Unicode 10.0 and includes collations from the Unicode Common Locale Data Repository, CLDR 32. These software internationalization libraries ensure that character encodings are presented in a consistent way, regardless of operating system or platform. For more information about Unicode CLDR-32, see the [CLDR 32 Release Note](https://cldr.unicode.org/index/downloads/cldr-32) on the Unicode CLDR website. You can learn more about the internationalization components for Unicode (ICU) at the [ICU Technical Committee (ICU-TC)](https://icu.unicode.org/home) website. For information about ICU-60, see [Download ICU 60](https://icu.unicode.org/download/60).
Starting with version 14.3, RDS for PostgreSQL also includes collations that help with data integration and conversion from EBCDIC-based systems. The extended binary coded decimal interchange code or *EBCDIC* encoding is commonly used by mainframe operating systems. These Amazon RDS-provided collations are narrowly defined to sort only those Unicode characters that directly map to EBCDIC code pages. The characters are sorted in EBCDIC code-point order to allow for data validation after conversion. These collations don't include denormalized forms, nor do they include Unicode characters that don't directly map to a character on the source EBCDIC code page.
The character mappings between EBCDIC code pages and Unicode code points are based on tables published by IBM. The complete set is available from IBM as a [compressed file](http://download.boulder.ibm.com/ibmdl/pub/software/dw/java/cdctables.zip) for download. RDS for PostgreSQL used these mappings with tools provided by the ICU to create the collations listed in the tables in this section. The collation names include a language and country as required by the ICU. However, EBCDIC code pages don't specify languages, and some EBCDIC code pages cover multiple countries. That means that the language and country portion of the collation names in the table are arbitrary, and they don't need to match the current locale. In other words, the code page number is the most important part of the collation name in this table. You can use any of the collations listed in the following tables in any RDS for PostgreSQL database.
+ [Unicode to EBCDIC collations table](#ebcdic-table) – Some mainframe data migration tools internally use LATIN1 or LATIN9 to encode and process data. Such tools use round-trip schemes to preserve data integrity and support reverse conversion. The collations in this table can be used by tools that process data using LATIN1 encoding, which doesn't require special handling.
+ [Unicode to LATIN9 collations table](#latin9-table) – You can use these collations in any RDS for PostgreSQL database.
In the following table, you find collations available in RDS for PostgreSQL that map EBCDIC code pages to Unicode code points. We recommend that you use the collations in this table for application development that requires sorting based on the ordering of IBM code pages.
| PostgreSQL collation name | Description of code-page mapping and sort order |
| --- | --- |
| da-DK-cp277-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 277 (per conversion tables) are sorted in IBM CP 277 code point order |
| de-DE-cp273-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 273 (per conversion tables) are sorted in IBM CP 273 code point order |
| en-GB-cp285-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 285 (per conversion tables) are sorted in IBM CP 285 code point order |
| en-US-cp037-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 037 (per conversion tables) are sorted in IBM CP 37 code point order |
| es-ES-cp284-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 284 (per conversion tables) are sorted in IBM CP 284 code point order |
| fi-FI-cp278-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 278 (per conversion tables) are sorted in IBM CP 278 code point order |
| fr-FR-cp297-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 297 (per conversion tables) are sorted in IBM CP 297 code point order |
| it-IT-cp280-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 280 (per conversion tables) are sorted in IBM CP 280 code point order |
| nl-BE-cp500-x-icu | Unicode characters that directly map to IBM EBCDIC Code Page 500 (per conversion tables) are sorted in IBM CP 500 code point order |
Amazon RDS provides a set of additional collations that sort Unicode code points that map to LATIN9 characters using the tables published by IBM, in the order of the original code points according to the EBCDIC code page of the source data.
| PostgreSQL collation name | Description of code-page mapping and sort order |
| --- | --- |
| da-DK-cp1142m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1142 (per conversion tables) are sorted in IBM CP 1142 code point order |
| de-DE-cp1141m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1141 (per conversion tables) are sorted in IBM CP 1141 code point order |
| en-GB-cp1146m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1146 (per conversion tables) are sorted in IBM CP 1146 code point order |
| en-US-cp1140m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1140 (per conversion tables) are sorted in IBM CP 1140 code point order |
| es-ES-cp1145m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1145 (per conversion tables) are sorted in IBM CP 1145 code point order |
| fi-FI-cp1143m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1143 (per conversion tables) are sorted in IBM CP 1143 code point order |
| fr-FR-cp1147m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1147 (per conversion tables) are sorted in IBM CP 1147 code point order |
| it-IT-cp1144m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1144 (per conversion tables) are sorted in IBM CP 1144 code point order |
| nl-BE-cp1148m-x-icu | Unicode characters that map to LATIN9 characters originally converted from IBM EBCDIC Code Page 1148 (per conversion tables) are sorted in IBM CP 1148 code point order |
In the following, you can find an example of using an RDS for PostgreSQL collation.
```
db1=> SELECT pg_import_system_collations('pg_catalog');
pg_import_system_collations
-----------------------------
36
db1=> SELECT '¤' < 'a' col1;
col1
------
t
db1=> SELECT '¤' < 'a' COLLATE "da-DK-cp277-x-icu" col1;
col1
------
f
```
We recommend that you use the collations in the [Unicode to EBCDIC collations table](#ebcdic-table) and in the [Unicode to LATIN9 collations table](#latin9-table) for application development that requires sorting based on the ordering of IBM code pages. The following collations (suffixed with the letter “b”) are also visible in `pg_collation`, but are intended for use by mainframe data integration and migration tools at AWS that map code pages with specific code point shifts and require special handling in collation. In other words, the following collations aren't recommended for use.
+ da-DK-277b-x-icu
+ da-DK-1142b-x-icu
+ de-DE-cp273b-x-icu
+ de-DE-cp1141b-x-icu
+ en-GB-cp1146b-x-icu
+ en-GB-cp285b-x-icu
+ en-US-cp037b-x-icu
+ en-US-cp1140b-x-icu
+ es-ES-cp1145b-x-icu
+ es-ES-cp284b-x-icu
+ fi-FI-cp1143b-x-icu
+ fr-FR-cp1147b-x-icu
+ fr-FR-cp297b-x-icu
+ it-IT-cp1144b-x-icu
+ it-IT-cp280b-x-icu
+ nl-BE-cp1148b-x-icu
+ nl-BE-cp500b-x-icu
To learn more about migrating applications from mainframe environments to AWS, see [What is AWS Mainframe Modernization?](https://docs.aws.amazon.com/m2/latest/userguide/what-is-m2.html).
For more information about managing collations in PostgreSQL, see [Collation Support](https://www.postgresql.org/docs/current/collation.html) in the PostgreSQL documentation.
# Managing logical slot synchronization for RDS for PostgreSQL
Starting in community PostgreSQL 17, a new feature to automatically synchronize logical replication slots from primary to standby servers has been introduced through the parameter `sync_replication_slots` or the related function `pg_sync_replication_slots()`, which manually synchronizes slots on execution.
These features are available starting with RDS for PostgreSQL 17. A typical setup will have a primary instance and its [read replica](USER_PostgreSQL.Replication.ReadReplicas.md), as well as a logical replication subscriber to the primary.
Ensure the subscription is created with the failover option set to true:
```
CREATE SUBSCRIPTION subname CONNECTION 'host=...' PUBLICATION pubname WITH (failover = true);
```
This creates a logical slot on the publisher with failover enabled.
```
postgres=> SELECT slot_name, slot_type, failover FROM pg_catalog.pg_replication_slots;
slot_name | slot_type | failover
-----------+-----------+----------
subname | logical | t
(1 row)
```
By enabling slot synchronization, all of the failover logical replication slots on the primary are automatically created on the physical standbys and are synced periodically. Ensure the following values have been set through [parameter groups](USER_WorkingWithParamGroups.Associating.md):
+ `rds.logical_replication` must be `1` to enable logical replication
+ `hot_standby_feedback` must be `1` on the standby
+ `rds.logical_slot_sync_dbname` on the standby must be set to a valid database name
The parameter's default value is `postgres`. If the logical publishing instance has the `postgres` database, the default parameter does not need to be changed.
+ `synchronized_standby_slots` on the primary must be set to the physical replication slot of the standby intended to be in-sync
+ `sync_replication_slots` must be `1` to enable automatic synchronization
With a failover-enabled subscription slot and the above parameter values, when a standby is promoted, the subscriber can alter its subscription to this newly promoted instance and continue logical replication seamlessly.
# Connecting to a DB instance running the PostgreSQL database engine
After Amazon RDS provisions your DB instance, you can use any standard SQL client application to connect to the instance. Before you can connect, the DB instance must be available and accessible. Whether you can connect to the instance from outside the VPC depends on how you created the Amazon RDS DB instance:
+ If you created your DB instance as *public*, devices and Amazon EC2 instances outside the VPC can connect to your database.
+ If you created your DB instance as *private*, only Amazon EC2 instances and devices inside the Amazon VPC can connect to your database.
To check whether your DB instance is public or private, use the AWS Management Console to view the **Connectivity & security** tab for your instance. Under **Security**, you can find the "Publicly accessible" value, with No for private, Yes for public.
To learn more about different Amazon RDS and Amazon VPC configurations and how they affect accessibility, see [Scenarios for accessing a DB instance in a VPC](USER_VPC.Scenarios.md).
**Contents**
+ [
## Installing the psql client
](#install-psql)
+ [
## Finding the connection information for an RDS for PostgreSQL DB instance
](#postgresql-endpoint)
+ [
# Using pgAdmin to connect to a RDS for PostgreSQL DB instance
](USER_ConnectToPostgreSQLInstance.pgAdmin.md)
+ [
# Using psql to connect to your RDS for PostgreSQL DB instance
](USER_ConnectToPostgreSQLInstance.psql.md)
+ [
# Connecting to RDS for PostgreSQL with the Amazon Web Services (AWS) JDBC Driver
](PostgreSQL.Connecting.JDBCDriver.md)
+ [
# Connecting to RDS for PostgreSQL with the Amazon Web Services (AWS) Python Driver
](PostgreSQL.Connecting.PythonDriver.md)
+ [
# Troubleshooting connections to your RDS for PostgreSQL instance
](USER_ConnectToPostgreSQLInstance.Troubleshooting.md)
+ [
## Error – FATAL: database *name* does not exist
](USER_ConnectToPostgreSQLInstance.Troubleshooting.md#USER_ConnectToPostgreSQLInstance.Troubleshooting-DBname)
+ [
## Error – Could not connect to server: Connection timed out
](USER_ConnectToPostgreSQLInstance.Troubleshooting.md#USER_ConnectToPostgreSQLInstance.Troubleshooting-timeout)
+ [
## Errors with security group access rules
](USER_ConnectToPostgreSQLInstance.Troubleshooting.md#USER_ConnectToPostgreSQLInstance.Troubleshooting-AccessRules)
## Installing the psql client
To connect to your DB instance from an EC2 instance, you can install a PostgreSQL client on the EC2 instance. To install the latest version of the psql client on Amazon Linux 2023, run the following command:
```
sudo dnf install postgresql
```
To install the latest version of the psql client on Amazon Linux 2, run the following command:
```
sudo yum install -y postgresql
```
To install the latest version of the psql client on Ubuntu, run the following command:
```
sudo apt install -y postgresql-client
```
## Finding the connection information for an RDS for PostgreSQL DB instance
If the DB instance is available and accessible, you can connect by providing the following information to the SQL client application:
+ The DB instance endpoint, which serves as the host name (DNS name) for the instance.
+ The port on which the DB instance is listening. For PostgreSQL, the default port is 5432.
+ The user name and password for the DB instance. The default 'master username' for PostgreSQL is `postgres`.
+ The name and password of the database (DB name).
You can obtain these details by using the AWS Management Console, the AWS CLI [describe-db-instances](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-instances.html) command, or the Amazon RDS API [DescribeDBInstances](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBInstances.html) operation.
**To find the endpoint, port number, and DB name using the AWS Management Console**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Open the RDS console and then choose **Databases** to display a list of your DB instances.
1. Choose the PostgreSQL DB instance name to display its details.
1. On the **Connectivity & security** tab, copy the endpoint. Also, note the port number. You need both the endpoint and the port number to connect to the DB instance.
![\[Obtain the endpoint from the RDS Console\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/PostgreSQL-endpoint.png)
1. On the **Configuration** tab, note the DB name. If you created a database when you created the RDS for PostgreSQL instance, you see the name listed under DB name. If you didn't create a database, the DB name displays a dash (‐).
![\[Obtain the DB name from the RDS Console\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/PostgreSQL-db-name.png)
Following are two ways to connect to a PostgreSQL DB instance. The first example uses pgAdmin, a popular open-source administration and development tool for PostgreSQL. The second example uses psql, a command line utility that is part of a PostgreSQL installation.
# Using pgAdmin to connect to a RDS for PostgreSQL DB instance
You can use the open-source tool pgAdmin to connect to your RDS for PostgreSQL DB instance. You can download and install pgAdmin from [http://www.pgadmin.org/](http://www.pgadmin.org/) without having a local instance of PostgreSQL on your client computer.
**To connect to your RDS for PostgreSQL DB instance using pgAdmin**
1. Launch the pgAdmin application on your client computer.
1. On the **Dashboard** tab, choose **Add New Server**.
1. In the **Create - Server** dialog box, type a name on the **General** tab to identify the server in pgAdmin.
1. On the **Connection** tab, type the following information from your DB instance:
+ For **Host**, type the endpoint, for example `mypostgresql.c6c8dntfzzhgv0.us-east-2.rds.amazonaws.com`.
+ For **Port**, type the assigned port.
+ For **Username**, type the user name that you entered when you created the DB instance (if you changed the 'master username' from the default, `postgres`).
+ For **Password**, type the password that you entered when you created the DB instance.
![\[Type the password that you entered when creating the DB instance\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/Postgres-Connect01.png)
1. Choose **Save**.
If you have any problems connecting, see [Troubleshooting connections to your RDS for PostgreSQL instance](USER_ConnectToPostgreSQLInstance.Troubleshooting.md).
1. To access a database in the pgAdmin browser, expand **Servers**, the DB instance, and **Databases**. Choose the DB instance's database name.
![\[Choose the DB instance's database name in the pgAdmin browser\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/Postgres-Connect02.png)
1. To open a panel where you can enter SQL commands, choose **Tools**, **Query Tool**.
# Using psql to connect to your RDS for PostgreSQL DB instance
You can use a local instance of the psql command line utility to connect to a RDS for PostgreSQL DB instance. You need either PostgreSQL or the psql client installed on your client computer.
You can download the PostgreSQL client from the [PostgreSQL](https://www.postgresql.org/download/) website. Follow the instructions specific to your operating system version to install psql.
To connect to your RDS for PostgreSQL DB instance using psql, you need to provide host (DNS) information, access credentials, and the name of the database.
Use one of the following formats to connect to your RDS for PostgreSQL DB instance. When you connect, you're prompted for a password. For batch jobs or scripts, use the `--no-password` option. This option is set for the entire session.
**Note**
A connection attempt with `--no-password` fails when the server requires password authentication and a password is not available from other sources. For more information, see the [psql documentation](https://www.postgresql.org/docs/13/app-psql.html).
If this is the first time you are connecting to this DB instance, or if you didn't yet create a database for this RDS for PostgreSQL instance, you can connect to the **postgres** database using the 'master username' and password.
For Unix, use the following format.
```
psql \
--host= \
--port= \
--username= \
--password \
--dbname=
```
For Windows, use the following format.
```
psql ^
--host= ^
--port= ^
--username= ^
--password ^
--dbname=
```
For example, the following command connects to a database called `mypgdb` on a PostgreSQL DB instance called `mypostgresql` using fictitious credentials.
```
psql --host=mypostgresql.c6c8mwvfdgv0.us-west-2.rds.amazonaws.com --port=5432 --username=awsuser --password --dbname=mypgdb
```
# Connecting to RDS for PostgreSQL with the Amazon Web Services (AWS) JDBC Driver
The Amazon Web Services (AWS) JDBC Driver is designed as an advanced JDBC wrapper. This wrapper is complementary to and extends the functionality of an existing JDBC driver. The driver is drop-in compatible with the community pgJDBC driver.
To install the AWS JDBC Driver, append the AWS JDBC Driver .jar file (located in the application `CLASSPATH`), and keep references to the respective community driver. Update the respective connection URL prefix as follows:
+ `jdbc:postgresql://` to `jdbc:aws-wrapper:postgresql://`
For more information about the AWS JDBC Driver and complete instructions for using it, see the [Amazon Web Services (AWS) JDBC Driver GitHub repository](https://github.com/awslabs/aws-advanced-jdbc-wrapper).
# Connecting to RDS for PostgreSQL with the Amazon Web Services (AWS) Python Driver
The Amazon Web Services (AWS) Python Driver is designed as an advanced Python wrapper. This wrapper is complementary to and extends the functionality of the open-source Psycopg driver. The AWS Python Driver supports Python versions 3.8 and higher. You can install the `aws-advanced-python-wrapper` package using the `pip` command, along with the `psycopg` open-source packages.
For more information about the AWS Python Driver and complete instructions for using it, see the [Amazon Web Services (AWS) Python Driver GitHub repository](https://github.com/awslabs/aws-advanced-python-wrapper).
# Troubleshooting connections to your RDS for PostgreSQL instance
**Topics**
+ [
## Error – FATAL: database *name* does not exist
](#USER_ConnectToPostgreSQLInstance.Troubleshooting-DBname)
+ [
## Error – Could not connect to server: Connection timed out
](#USER_ConnectToPostgreSQLInstance.Troubleshooting-timeout)
+ [
## Errors with security group access rules
](#USER_ConnectToPostgreSQLInstance.Troubleshooting-AccessRules)
## Error – FATAL: database *name* does not exist
If when trying to connect you receive an error like `FATAL: database name does not exist`, try using the default database name **postgres** for the `--dbname` option.
## Error – Could not connect to server: Connection timed out
If you can't connect to the DB instance, the most common error is `Could not connect to server: Connection timed out.` If you receive this error, check the following:
+ Check that the host name used is the DB instance endpoint and that the port number used is correct.
+ Make sure that the DB instance's public accessibility is set to **Yes** to allow external connections. To modify the **Public access** setting, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md).
+ Make sure that the user connecting to the database has CONNECT access to it. You can use the following query to provide connect access to the database.
```
GRANT CONNECT ON DATABASE database name TO username;
```
+ Check that the security group assigned to the DB instance has rules to allow access through any firewall your connection might go through. For example, if the DB instance was created using the default port of 5432, your company might have firewall rules blocking connections to that port from external company devices.
To fix this, modify the DB instance to use a different port. Also, make sure that the security group applied to the DB instance allows connections to the new port. To modify the **Database port** setting, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md).
+ Check whether the port you're attempting to use is already occupied by a local instance of PostgreSQL or another service running on your computer. For example, if you have a local PostgreSQL database running on the same port (default is 5432), it might prevent a successful connection to the RDS for PostgreSQL DB instance. Make sure that the port is free, or try connecting with a different port number if possible.
+ See also [Errors with security group access rules](#USER_ConnectToPostgreSQLInstance.Troubleshooting-AccessRules).
## Errors with security group access rules
By far the most common connection problem is with the security group's access rules assigned to the DB instance. If you used the default security group when you created the DB instance, the security group likely didn't have access rules that allow you to access the instance.
For the connection to work, the security group you assigned to the DB instance at its creation must allow access to the DB instance. For example, if the DB instance was created in a VPC, it must have a VPC security group that authorizes connections. Check if the DB instance was created using a security group that doesn't authorize connections from the device or Amazon EC2 instance where the application is running.
You can add or edit an inbound rule in the security group. For **Source**, choosing **My IP** allows access to the DB instance from the IP address detected in your browser. For more information, see [Provide access to your DB instance in your VPC by creating a security group](CHAP_SettingUp.md#CHAP_SettingUp.SecurityGroup).
Alternatively, if the DB instance was created outside of a VPC, it must have a database security group that authorizes those connections.
For more information about Amazon RDS security groups, see [Controlling access with security groups](Overview.RDSSecurityGroups.md).
# Securing connections to RDS for PostgreSQL with SSL/TLS
RDS for PostgreSQL supports Secure Socket Layer (SSL) encryption for PostgreSQL DB instances. Using SSL, you can encrypt a PostgreSQL connection between your applications and your PostgreSQL DB instances. You can also force all connections to your PostgreSQL DB instance to use SSL. RDS for PostgreSQL also supports Transport Layer Security (TLS), the successor protocol to SSL.
To learn more about Amazon RDS and data protection, including encrypting connections using SSL/TLS, see [Data protection in Amazon RDS](DataDurability.md).
**Topics**
+ [
# Using SSL with a PostgreSQL DB instance
](PostgreSQL.Concepts.General.SSL.md)
+ [
# Updating applications to connect to PostgreSQL DB instances using new SSL/TLS certificates
](ssl-certificate-rotation-postgresql.md)
# Using SSL with a PostgreSQL DB instance
Amazon RDS supports Secure Socket Layer (SSL) encryption for PostgreSQL DB instances. Using SSL, you can encrypt a PostgreSQL connection between your applications and your PostgreSQL DB instances. By default, RDS for PostgreSQL uses and expects all clients to connect using SSL/TLS, but you can also require it. RDS for PostgreSQL supports Transport Layer Security (TLS) versions 1.1, 1.2, and 1.3.
For general information about SSL support and PostgreSQL databases, see [SSL support](https://www.postgresql.org/docs/11/libpq-ssl.html) in the PostgreSQL documentation. For information about using an SSL connection over JDBC, see [Configuring the client](https://jdbc.postgresql.org/documentation/head/ssl-client.html) in the PostgreSQL documentation.
SSL support is available in all AWS Regions for PostgreSQL. Amazon RDS creates an SSL certificate for your PostgreSQL DB instance when the instance is created. If you enable SSL certificate verification, then the SSL certificate includes the DB instance endpoint as the Common Name (CN) for the SSL certificate to guard against spoofing attacks.
**Topics**
+ [
## Connecting to a PostgreSQL DB instance over SSL
](#PostgreSQL.Concepts.General.SSL.Connecting)
+ [
## Requiring an SSL connection to a PostgreSQL DB instance
](#PostgreSQL.Concepts.General.SSL.Requiring)
+ [
## Determining the SSL connection status
](#PostgreSQL.Concepts.General.SSL.Status)
+ [
## SSL cipher suites in RDS for PostgreSQL
](#PostgreSQL.Concepts.General.SSL.Ciphers)
## Connecting to a PostgreSQL DB instance over SSL
**To connect to a PostgreSQL DB instance over SSL**
1. Download the certificate.
For information about downloading certificates, see [Using SSL/TLS to encrypt a connection to a DB instance or cluster ](UsingWithRDS.SSL.md).
1. Connect to your PostgreSQL DB instance over SSL.
When you connect using SSL, your client can choose whether to verify the certificate chain. If your connection parameters specify `sslmode=verify-ca` or `sslmode=verify-full`, then your client requires the RDS CA certificates to be in their trust store or referenced in the connection URL. This requirement is to verify the certificate chain that signs your database certificate.
When a client, such as psql or JDBC, is configured with SSL support, the client first tries to connect to the database with SSL by default. If the client can't connect with SSL, it reverts to connecting without SSL. The default `sslmode` mode used is different between libpq-based clients (such as psql) and JDBC. The libpq-based and JDBC clients default to `prefer`.
Use the `sslrootcert` parameter to reference the certificate, for example `sslrootcert=rds-ssl-ca-cert.pem`.
The following is an example of using `psql` to connect to a PostgreSQL DB instance using SSL with certificate verification.
```
$ psql "host=db-name.555555555555.ap-southeast-1.rds.amazonaws.com
port=5432 dbname=testDB user=testuser sslrootcert=rds-ca-rsa2048-g1.pem sslmode=verify-full"
```
## Requiring an SSL connection to a PostgreSQL DB instance
You can require that connections to your PostgreSQL DB instance use SSL by using the `rds.force_ssl` parameter. The `rds.force_ssl` parameter default value is 1 (on) for RDS for PostgreSQL version 15 and later. For all other RDS for PostgreSQL major versions 14 and older, the default value of this parameter is 0 (off). You can set the `rds.force_ssl` parameter to 1 (on) to require SSL/TLS for connections to your DB cluster. You can set the `rds.force_ssl` parameter to 1 (on) to require SSL for connections to your DB instance.
To change the value of this parameter, you need to create a custom DB parameter group. You then change the value for `rds.force_ssl` in your custom DB parameter group to `1` to turn on this feature. If you prepare the custom DB parameter group before creating your RDS for PostgreSQL DB instance you can choose it (instead of a default parameter group) during the creation process. If you do this after your RDS for PostgreSQL DB instance is already running, you need to reboot the instance so that your instance uses the custom parameter group. For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
When the `rds.force_ssl` feature is active on your DB instance, connection attempts that aren't using SSL are rejected with the following message:
```
$ psql -h db-name.555555555555.ap-southeast-1.rds.amazonaws.com port=5432 dbname=testDB user=testuser
psql: error: FATAL: no pg_hba.conf entry for host "w.x.y.z", user "testuser", database "testDB", SSL off
```
## Determining the SSL connection status
The encrypted status of your connection is shown in the logon banner when you connect to the DB instance:
```
Password for user master:
psql (10.3)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.
postgres=>
```
You can also load the `sslinfo` extension and then call the `ssl_is_used()` function to determine if SSL is being used. The function returns `t` if the connection is using SSL, otherwise it returns `f`.
```
postgres=> CREATE EXTENSION sslinfo;
CREATE EXTENSION
postgres=> SELECT ssl_is_used();
ssl_is_used
---------
t
(1 row)
```
For more detailed information, you can use the following query to get information from `pg_settings`:
```
SELECT name as "Parameter name", setting as value, short_desc FROM pg_settings WHERE name LIKE '%ssl%';
Parameter name | value | short_desc
----------------------------------------+-----------------------------------------+-------------------------------------------------------
ssl | on | Enables SSL connections.
ssl_ca_file | /rdsdbdata/rds-metadata/ca-cert.pem | Location of the SSL certificate authority file.
ssl_cert_file | /rdsdbdata/rds-metadata/server-cert.pem | Location of the SSL server certificate file.
ssl_ciphers | HIGH:!aNULL:!3DES | Sets the list of allowed SSL ciphers.
ssl_crl_file | | Location of the SSL certificate revocation list file.
ssl_dh_params_file | | Location of the SSL DH parameters file.
ssl_ecdh_curve | prime256v1 | Sets the curve to use for ECDH.
ssl_key_file | /rdsdbdata/rds-metadata/server-key.pem | Location of the SSL server private key file.
ssl_library | OpenSSL | Name of the SSL library.
ssl_max_protocol_version | | Sets the maximum SSL/TLS protocol version to use.
ssl_min_protocol_version | TLSv1.2 | Sets the minimum SSL/TLS protocol version to use.
ssl_passphrase_command | | Command to obtain passphrases for SSL.
ssl_passphrase_command_supports_reload | off | Also use ssl_passphrase_command during server reload.
ssl_prefer_server_ciphers | on | Give priority to server ciphersuite order.
(14 rows)
```
You can also collect all the information about your RDS for PostgreSQL DB instance's SSL usage by process, client, and application by using the following query:
```
SELECT datname as "Database name", usename as "User name", ssl, client_addr, application_name, backend_type
FROM pg_stat_ssl
JOIN pg_stat_activity
ON pg_stat_ssl.pid = pg_stat_activity.pid
ORDER BY ssl;
Database name | User name | ssl | client_addr | application_name | backend_type
---------------+-----------+-----+----------------+------------------------+------------------------------
| | f | | | autovacuum launcher
| rdsadmin | f | | | logical replication launcher
| | f | | | background writer
| | f | | | checkpointer
| | f | | | walwriter
rdsadmin | rdsadmin | t | 127.0.0.1 | | client backend
rdsadmin | rdsadmin | t | 127.0.0.1 | PostgreSQL JDBC Driver | client backend
postgres | postgres | t | 204.246.162.36 | psql | client backend
(8 rows)
```
To identify the cipher used for your SSL connection, you can query as follows:
```
postgres=> SELECT ssl_cipher();
ssl_cipher
--------------------
DHE-RSA-AES256-SHA
(1 row)
```
To learn more about the `sslmode` option, see [Database connection control functions](https://www.postgresql.org/docs/11/libpq-connect.html#LIBPQ-CONNECT-SSLMODE) in the *PostgreSQL documentation*.
## SSL cipher suites in RDS for PostgreSQL
The PostgreSQL configuration parameter [ssl\$1ciphers](https://www.postgresql.org/docs/current/runtime-config-connection.html#RUNTIME-CONFIG-CONNECTION-SSL) specifies the categories of cipher suites that are allowed for SSL connections to the database when using TLS 1.2 and lower.
In RDS for PostgreSQL 16 and later, you can modify the `ssl_ciphers` parameter to use specific values from the allowlisted cipher suites. This is a dynamic parameter that doesn't require a database instance reboot. To view the allowlisted cipher suites, use either the Amazon RDS console or the following AWS CLI command:
```
aws rds describe-db-parameters --db-parameter-group-name --region --endpoint-url --output json | jq '.Parameters[] | select(.ParameterName == "ssl_ciphers")'
```
The following table lists both the default cipher suites and the allowed cipher suites for versions that support custom configurations.
| PostgreSQL engine version | Default ssl\$1cipher suite values | Allowlisted custom ssl\$1cipher suite values |
| --- | --- | --- |
| 18 | HIGH:\$1aNULL:\$13DES | `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256` `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` |
| 17 | HIGH:\$1aNULL:\$13DES | `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256` `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` |
| 16 | HIGH:\$1aNULL:\$13DES | `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256` `TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256` `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` `TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256` |
| 15 | HIGH:\$1aNULL:\$13DES | Custom ssl\$1ciphers isn't supported |
| 14 | HIGH:\$1aNULL:\$13DES | Custom ssl\$1ciphers isn't supported |
| 13 | HIGH:\$1aNULL:\$13DES | Custom ssl\$1ciphers isn't supported |
| 12 | HIGH:\$1aNULL:\$13DES | Custom ssl\$1ciphers isn't supported |
| 11.4 and higher minor versions | HIGH:MEDIUM:\$13DES:\$1aNULL:\$1RC4 | Custom ssl\$1ciphers isn't supported |
| 11.1, 11.2 | HIGH:MEDIUM:\$13DES:\$1aNULL | Custom ssl\$1ciphers isn't supported |
| 10.9 and higher minor versions | HIGH:MEDIUM:\$13DES:\$1aNULL:\$1RC4 | Custom ssl\$1ciphers isn't supported |
| 10.7 and lower minor versions | HIGH:MEDIUM:\$13DES:\$1aNULL | Custom ssl\$1ciphers isn't supported |
To configure all instance connections to use the `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` cipher suite, modify your parameter group as shown in the following example:
```
aws rds modify-db-parameter-group --db-parameter-group-name --parameters "ParameterName='ssl_ciphers',ParameterValue='TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384',ApplyMethod=immediate"
```
This example uses an ECDSA cipher, which requires your instance to use a certificate authority with elliptic curve cryptography (ECC) to establish a connection. For information about certificate authorities provided by Amazon RDS, see [Certificate authorities](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/singWithRDS.SSL.html#UsingWithRDS.SSL.RegionCertificateAuthorities).
You can verify the ciphers in use through the methods described in [Determining the SSL connection status](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Concepts.General.SSL.html#PostgreSQL.Concepts.General.SSL.Status).
Ciphers may have different names depending on the context:
+ The allowlisted ciphers that you can configure in your parameter group are referred to with their IANA names.
+ The `sslinfo` and `psql` logon banner refer to ciphers using their OpenSSL names.
By default, the value of `ssl_max_protocol_version` in RDS for PostgreSQL 16 and later is TLS v1.3. You must set the value of this parameter to TLS v1.2 as TLS v1.3 doesn't use the cipher configurations specified in the `ssl_ciphers` parameter. When you set the value as TLS v1.2, connections use only the ciphers that you define in `ssl_ciphers`.
```
aws rds modify-db-parameter-group --db-parameter-group-name --parameters "ParameterName='ssl_max_protocol_version',ParameterValue='TLSv1.2',ApplyMethod=immediate"
```
To ensure database connections use SSL, set the `rds.force_ssl parameter` to 1 in your parameter group. For more information about parameters and parameter groups, see [Parameter groups for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html).
# Updating applications to connect to PostgreSQL DB instances using new SSL/TLS certificates
Certificates used for Secure Socket Layer or Transport Layer Security (SSL/TLS) typically have a set lifetime. When service providers update their Certificate Authority (CA) certificates, clients must update their applications to use the new certificates. Following, you can find information about how to determine if your client applications use SSL/TLS to connect to your Amazon RDS for PostgreSQL DB instance. You also find information about how to check if those applications verify the server certificate when they connect.
**Note**
A client application that's configured to verify the server certificate before SSL/TLS connection must have a valid CA certificate in the client's trust store. Update the client trust store when necessary for new certificates.
After you update your CA certificates in the client application trust stores, you can rotate the certificates on your DB instances. We strongly recommend testing these procedures in a nonproduction environment before implementing them in your production environments.
For more information about certificate rotation, see [Rotating your SSL/TLS certificate](UsingWithRDS.SSL-certificate-rotation.md). For more information about downloading certificates, see [Using SSL/TLS to encrypt a connection to a DB instance or cluster ](UsingWithRDS.SSL.md). For information about using SSL/TLS with PostgreSQL DB instances, see [Using SSL with a PostgreSQL DB instance](PostgreSQL.Concepts.General.SSL.md).
**Topics**
+ [
## Determining whether applications are connecting to PostgreSQL DB instances using SSL
](#ssl-certificate-rotation-postgresql.determining-server)
+ [
## Determining whether a client requires certificate verification in order to connect
](#ssl-certificate-rotation-postgresql.determining-client)
+ [
## Updating your application trust store
](#ssl-certificate-rotation-postgresql.updating-trust-store)
+ [
## Using SSL/TLS connections for different types of applications
](#ssl-certificate-rotation-postgresql.applications)
## Determining whether applications are connecting to PostgreSQL DB instances using SSL
Check the DB instance configuration for the value of the `rds.force_ssl` parameter. By default, the `rds.force_ssl` parameter is set to `0` (off) for DB instances using PostgreSQL versions before version 15. By default, `rds.force_ssl` is set to `1` (on) for DB instances using PostgreSQL version 15 and later major versions. If the `rds.force_ssl` parameter is set to `1` (on), clients are required to use SSL/TLS for connections. For more information about parameter groups, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
If you are using RDS PostgreSQL version 9.5 or later major version and `rds.force_ssl` is not set to `1` (on), query the `pg_stat_ssl` view to check connections using SSL. For example, the following query returns only SSL connections and information about the clients using SSL.
```
SELECT datname, usename, ssl, client_addr
FROM pg_stat_ssl INNER JOIN pg_stat_activity ON pg_stat_ssl.pid = pg_stat_activity.pid
WHERE ssl is true and usename<>'rdsadmin';
```
Only rows using SSL/TLS connections are displayed with information about the connection. The following is sample output.
```
datname | usename | ssl | client_addr
----------+---------+-----+-------------
benchdb | pgadmin | t | 53.95.6.13
postgres | pgadmin | t | 53.95.6.13
(2 rows)
```
This query displays only the current connections at the time of the query. The absence of results doesn't indicate that no applications are using SSL connections. Other SSL connections might be established at a different time.
## Determining whether a client requires certificate verification in order to connect
When a client, such as psql or JDBC, is configured with SSL support, the client first tries to connect to the database with SSL by default. If the client can't connect with SSL, it reverts to connecting without SSL. The default `sslmode` mode used for both libpq-based clients (such as psql) and JDBC is set to `prefer`. The certificate on the server is verified only when `sslrootcert` is provided with `sslmode` set to `verify-ca` or `verify-full`. An error is thrown if the certificate is invalid.
Use `PGSSLROOTCERT` to verify the certificate with the `PGSSLMODE` environment variable, with `PGSSLMODE` set to `verify-ca` or `verify-full`.
```
PGSSLMODE=verify-full PGSSLROOTCERT=/fullpath/ssl-cert.pem psql -h pgdbidentifier.cxxxxxxxx.us-east-2.rds.amazonaws.com -U masteruser -d postgres
```
Use the `sslrootcert` argument to verify the certificate with `sslmode` in connection string format, with `sslmode` set to `verify-ca` or `verify-full` to verify the certificate.
```
psql "host=pgdbidentifier.cxxxxxxxx.us-east-2.rds.amazonaws.com sslmode=verify-full sslrootcert=/full/path/ssl-cert.pem user=masteruser dbname=postgres"
```
For example, in the preceding case, if you are using an invalid root certificate, then you see an error similar to the following on your client.
```
psql: SSL error: certificate verify failed
```
## Updating your application trust store
For information about updating the trust store for PostgreSQL applications, see [Secure TCP/IP connections with SSL](https://www.postgresql.org/docs/current/ssl-tcp.html) in the PostgreSQL documentation.
For information about downloading the root certificate, see [Using SSL/TLS to encrypt a connection to a DB instance or cluster ](UsingWithRDS.SSL.md).
For sample scripts that import certificates, see [Sample script for importing certificates into your trust store](UsingWithRDS.SSL-certificate-rotation.md#UsingWithRDS.SSL-certificate-rotation-sample-script).
**Note**
When you update the trust store, you can retain older certificates in addition to adding the new certificates.
## Using SSL/TLS connections for different types of applications
The following provides information about using SSL/TLS connections for different types of applications:
+ **psql**
The client is invoked from the command line by specifying options either as a connection string or as environment variables. For SSL/TLS connections, the relevant options are `sslmode` (environment variable `PGSSLMODE`), `sslrootcert` (environment variable `PGSSLROOTCERT`).
For the complete list of options, see [Parameter key words](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS) in the PostgreSQL documentation. For the complete list of environment variables, see [Environment variables](https://www.postgresql.org/docs/current/libpq-envars.html) in the PostgreSQL documentation.
+ **pgAdmin**
This browser-based client is a more user-friendly interface for connecting to a PostgreSQL database.
For information about configuring connections, see the [pgAdmin documentation](https://www.pgadmin.org/docs/pgadmin4/latest/server_dialog.html).
+ **JDBC**
JDBC enables database connections with Java applications.
For general information about connecting to a PostgreSQL database with JDBC, see [Connecting to the database](https://jdbc.postgresql.org/documentation/use/#connecting-to-the-database) in the PostgreSQL JDBC driver documentation. For information about connecting with SSL/TLS, see [Configuring the client](https://jdbc.postgresql.org/documentation/ssl/#configuring-the-client) in the PostgreSQL JDBC driver documentation.
+ **Python**
A popular Python library for connecting to PostgreSQL databases is `psycopg2`.
For information about using `psycopg2`, see the [psycopg2 documentation](https://pypi.org/project/psycopg2/). For a short tutorial on how to connect to a PostgreSQL database, see [Psycopg2 tutorial](https://wiki.postgresql.org/wiki/Psycopg2_Tutorial). You can find information about the options the connect command accepts in [The psycopg2 module content](http://initd.org/psycopg/docs/module.html#module-psycopg2).
**Important**
After you have determined that your database connections use SSL/TLS and have updated your application trust store, you can update your database to use the rds-ca-rsa2048-g1 certificates. For instructions, see step 3 in [Updating your CA certificate by modifying your DB instance or cluster](UsingWithRDS.SSL-certificate-rotation.md#UsingWithRDS.SSL-certificate-rotation-updating).
# Using Kerberos authentication with Amazon RDS for PostgreSQL
You can use Kerberos to authenticate users when they connect to your DB instance running PostgreSQL. To do so, configure your DB instance to use AWS Directory Service for Microsoft Active Directory for Kerberos authentication. AWS Directory Service for Microsoft Active Directory is also called AWS Managed Microsoft AD. It's a feature available with Directory Service. To learn more, see [What is Directory Service?](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/what_is.html) in the *AWS Directory Service Administration Guide*.
To start, create an AWS Managed Microsoft AD directory to store user credentials. Then, provide to your PostgreSQL DB instance the Active Directory's domain and other information. When users authenticate with the PostgreSQL DB instance, authentication requests are forwarded to the AWS Managed Microsoft AD directory.
Keeping all of your credentials in the same directory can save you time and effort. You have a centralized location for storing and managing credentials for multiple DB instances. Using a directory can also improve your overall security profile.
In addition, you can access credentials from your own on-premises Microsoft Active Directory. To do so, create a trusting domain relationship so that the AWS Managed Microsoft AD directory trusts your on-premises Microsoft Active Directory. In this way, your users can access your PostgreSQL instances with the same Windows single sign-on (SSO) experience as when they access workloads in your on-premises network.
A database can use password authentication or password authentication with either Kerberos or AWS Identity and Access Management (IAM) authentication. For more information about IAM authentication, see [IAM database authentication for MariaDB, MySQL, and PostgreSQL](UsingWithRDS.IAMDBAuth.md).
**Note**
RDS for PostgreSQL doesn't support Kerberos authentication for Active Directory groups.
**Topics**
+ [
## Region and version availability
](#postgresql-kerberos.RegionVersionAvailability)
+ [
## Overview of Kerberos authentication for PostgreSQL DB instances
](#postgresql-kerberos-overview)
+ [
# Setting up Kerberos authentication for PostgreSQL DB instances
](postgresql-kerberos-setting-up.md)
+ [
# Managing an RDS for PostgreSQL DB instance in an Active Directory domain
](postgresql-kerberos-managing.md)
+ [
# Connecting to PostgreSQL with Kerberos authentication
](postgresql-kerberos-connecting.md)
## Region and version availability
Feature availability and support varies across specific versions of each database engine, and across AWS Regions. For more information on version and Region availability of RDS for PostgreSQL with Kerberos authentication, see [Supported Regions and DB engines for Kerberos authentication in Amazon RDS](Concepts.RDS_Fea_Regions_DB-eng.Feature.KerberosAuthentication.md).
## Overview of Kerberos authentication for PostgreSQL DB instances
To set up Kerberos authentication for a PostgreSQL DB instance, take the following steps, described in more detail later:
1. Use AWS Managed Microsoft AD to create an AWS Managed Microsoft AD directory. You can use the AWS Management Console, the AWS CLI, or the Directory Service API to create the directory. Make sure to open the relevant outbound ports on the directory security group so that the directory can communicate with the instance.
1. Create a role that provides Amazon RDS access to make calls to your AWS Managed Microsoft AD directory. To do so, create an AWS Identity and Access Management (IAM) role that uses the managed IAM policy `AmazonRDSDirectoryServiceAccess`.
For the IAM role to allow access, the AWS Security Token Service (AWS STS) endpoint must be activated in the correct AWS Region for your AWS account. AWS STS endpoints are active by default in all AWS Regions, and you can use them without any further actions. For more information, see [Activating and deactivating AWS STS in an AWS Region](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#sts-regions-activate-deactivate) in the *IAM User Guide*.
1. Create and configure users in the AWS Managed Microsoft AD directory using the Microsoft Active Directory tools. For more information about creating users in your Active Directory, see [Manage users and groups in AWS Managed Microsoft AD](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_manage_users_groups.html) in the *Directory Service Administration Guide*.
1. If you plan to locate the directory and the DB instance in different AWS accounts or virtual private clouds (VPCs), configure VPC peering. For more information, see [What is VPC peering?](https://docs.aws.amazon.com/vpc/latest/peering/Welcome.html) in the *Amazon VPC Peering Guide*.
1. Create or modify a PostgreSQL DB instance either from the console, CLI, or RDS API using one of the following methods:
+ [Creating an Amazon RDS DB instance](USER_CreateDBInstance.md)
+ [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md)
+ [Restoring to a DB instance](USER_RestoreFromSnapshot.md)
+ [Restoring a DB instance to a specified time for Amazon RDS](USER_PIT.md)
You can locate the instance in the same Amazon Virtual Private Cloud (VPC) as the directory or in a different AWS account or VPC. When you create or modify the PostgreSQL DB instance, do the following:
+ Provide the domain identifier (`d-*` identifier) that was generated when you created your directory.
+ Provide the name of the IAM role that you created.
+ Ensure that the DB instance security group can receive inbound traffic from the directory security group.
1. Use the RDS master user credentials to connect to the PostgreSQL DB instance. Create the user in PostgreSQL to be identified externally. Externally identified users can log in to the PostgreSQL DB instance using Kerberos authentication.
# Setting up Kerberos authentication for PostgreSQL DB instances
You use AWS Directory Service for Microsoft Active Directory (AWS Managed Microsoft AD) to set up Kerberos authentication for a PostgreSQL DB instance. To set up Kerberos authentication, take the following steps.
**Topics**
+ [
## Step 1: Create a directory using AWS Managed Microsoft AD
](#postgresql-kerberos-setting-up.create-directory)
+ [
## Step 2: (Optional) Create a trust relationship between your on-premises Active Directory and Directory Service
](#postgresql-kerberos-setting-up.create-trust)
+ [
## Step 3: Create an IAM role for Amazon RDS to access the Directory Service
](#postgresql-kerberos-setting-up.CreateIAMRole)
+ [
## Step 4: Create and configure users
](#postgresql-kerberos-setting-up.create-users)
+ [
## Step 5: Enable cross-VPC traffic between the directory and the DB instance
](#postgresql-kerberos-setting-up.vpc-peering)
+ [
## Step 6: Create or modify a PostgreSQL DB instance
](#postgresql-kerberos-setting-up.create-modify)
+ [
## Step 7: Create PostgreSQL users for your Kerberos principals
](#postgresql-kerberos-setting-up.create-logins)
+ [
## Step 8: Configure a PostgreSQL client
](#postgresql-kerberos-setting-up.configure-client)
## Step 1: Create a directory using AWS Managed Microsoft AD
Directory Service creates a fully managed Active Directory in the AWS Cloud. When you create an AWS Managed Microsoft AD directory, Directory Service creates two domain controllers and DNS servers for you. The directory servers are created in different subnets in a VPC. This redundancy helps make sure that your directory remains accessible even if a failure occurs.
When you create an AWS Managed Microsoft AD directory, AWS Directory Service performs the following tasks on your behalf:
+ Sets up an Active Directory within your VPC.
+ Creates a directory administrator account with the user name `Admin` and the specified password. You use this account to manage your directory.
**Important**
Make sure to save this password. Directory Service doesn't store this password, and it can't be retrieved or reset.
+ Creates a security group for the directory controllers. The security group must permit communication with the PostgreSQL DB instance.
When you launch AWS Directory Service for Microsoft Active Directory, AWS creates an Organizational Unit (OU) that contains all of your directory's objects. This OU, which has the NetBIOS name that you entered when you created your directory, is located in the domain root. The domain root is owned and managed by AWS.
The `Admin` account that was created with your AWS Managed Microsoft AD directory has permissions for the most common administrative activities for your OU:
+ Create, update, or delete users
+ Add resources to your domain such as file or print servers, and then assign permissions for those resources to users in your OU
+ Create additional OUs and containers
+ Delegate authority
+ Restore deleted objects from the Active Directory Recycle Bin
+ Run Active Directory and Domain Name Service (DNS) modules for Windows PowerShell on the Active Directory Web Service
The `Admin` account also has rights to perform the following domain-wide activities:
+ Manage DNS configurations (add, remove, or update records, zones, and forwarders)
+ View DNS event logs
+ View security event logs
**To create a directory with AWS Managed Microsoft AD**
1. In the [Directory Service console](https://console.aws.amazon.com/directoryservicev2/) navigation pane, choose **Directories**, and then choose **Set up directory**.
1. Choose **AWS Managed Microsoft AD**. AWS Managed Microsoft AD is the only option currently supported for use with Amazon RDS.
1. Choose **Next**.
1. On the **Enter directory information** page, provide the following information:
**Edition**
Choose the edition that meets your requirements.
**Directory DNS name**
The fully qualified name for the directory, such as **corp.example.com**.
**Directory NetBIOS name**
An optional short name for the directory, such as `CORP`.
**Directory description**
An optional description for the directory.
**Admin password**
The password for the directory administrator. The directory creation process creates an administrator account with the user name `Admin` and this password.
The directory administrator password can't include the word "admin." The password is case-sensitive and must be 8–64 characters in length. It must also contain at least one character from three of the following four categories:
+ Lowercase letters (a–z)
+ Uppercase letters (A–Z)
+ Numbers (0–9)
+ Nonalphanumeric characters (\$1\$1@\$1\$1%^&\$1\$1-\$1=`\$1\$1()\$1\$1[]:;"'<>,.?/)
**Confirm password**
Retype the administrator password.
Make sure that you save this password. Directory Service doesn't store this password, and it can't be retrieved or reset.
1. Choose **Next**.
1. On the **Choose VPC and subnets** page, provide the following information:
**VPC**
Choose the VPC for the directory. You can create the PostgreSQL DB instance in this same VPC or in a different VPC.
**Subnets**
Choose the subnets for the directory servers. The two subnets must be in different Availability Zones.
1. Choose **Next**.
1. Review the directory information. If changes are needed, choose **Previous** and make the changes. When the information is correct, choose **Create directory**.
![\[Directory details page\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/WinAuth2.png)
It takes several minutes for the directory to be created. When it has been successfully created, the **Status** value changes to **Active**.
To see information about your directory, choose the directory ID in the directory listing. Make a note of the **Directory ID** value. You need this value when you create or modify your PostgreSQL DB instance.
![\[Image of details page\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/WinAuth3.png)
## Step 2: (Optional) Create a trust relationship between your on-premises Active Directory and Directory Service
If you don't plan to use your own on-premises Microsoft Active Directory, skip to [Step 3: Create an IAM role for Amazon RDS to access the Directory Service](#postgresql-kerberos-setting-up.CreateIAMRole).
To get Kerberos authentication using your on-premises Active Directory, you need to create a trusting domain relationship using a forest trust between your on-premises Microsoft Active Directory and the AWS Managed Microsoft AD directory (created in [Step 1: Create a directory using AWS Managed Microsoft AD](#postgresql-kerberos-setting-up.create-directory)). The trust can be one-way, where the AWS Managed Microsoft AD directory trusts the on-premises Microsoft Active Directory. The trust can also be two-way, where both Active Directories trust each other. For more information about setting up trusts using Directory Service, see [When to create a trust relationship](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_setup_trust.html) in the *AWS Directory Service Administration Guide*.
**Note**
If you use an on-premises Microsoft Active Directory, Windows clients connect using the domain name of the Directory Service in the endpoint rather than rds.amazonaws.com. To learn more, see [Connecting to PostgreSQL with Kerberos authentication](postgresql-kerberos-connecting.md).
Make sure that your on-premises Microsoft Active Directory domain name includes a DNS suffix routing that corresponds to the newly created trust relationship. The following screenshot shows an example.
![\[DNS routing corresponds to the created trust\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/kerberos-auth-trust.png)
## Step 3: Create an IAM role for Amazon RDS to access the Directory Service
For Amazon RDS to call Directory Service for you, your AWS account needs an IAM role that uses the managed IAM policy `AmazonRDSDirectoryServiceAccess`. This role allows Amazon RDS to make calls to Directory Service.
When you create a DB instance using the AWS Management Console and your console user account has the `iam:CreateRole` permission, the console creates the needed IAM role automatically. In this case, the role name is `rds-directoryservice-kerberos-access-role`. Otherwise, you must create the IAM role manually. When you create this IAM role, choose `Directory Service`, and attach the AWS managed policy `AmazonRDSDirectoryServiceAccess` to it.
For more information about creating IAM roles for a service, see [Creating a role to delegate permissions to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) in the *IAM User Guide*.
**Note**
The IAM role used for Windows Authentication for RDS for Microsoft SQL Server can't be used for Amazon RDS for PostgreSQL.
As an alternative to using the `AmazonRDSDirectoryServiceAccess` managed policy, you can create policies with the required permissions. In this case, the IAM role must have the following IAM trust policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": [
"directoryservice.rds.amazonaws.com",
"rds.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
```
------
The role must also have the following IAM role policy.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"ds:DescribeDirectories",
"ds:AuthorizeApplication",
"ds:UnauthorizeApplication",
"ds:GetAuthorizedApplicationDetails"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
```
------
For opt-in AWS Regions, use Region-specific service principals in IAM role trust policies. When you create a trust policy for services in these Regions, specify the Region code in the service principal.
The following example shows a trust policy that includes Region-specific service principals:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"directoryservice.rds.REGION-CODE.amazonaws.com",
"rds.REGION-CODE.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
```
------
Replace REGION-CODE with the code for your specific Region. For example, use the following service principals for the Asia Pacific (Melbourne) Region:
```
"Service": [
"directoryservice.rds.ap-southeast-4.amazonaws.com",
"rds.ap-southeast-4.amazonaws.com"
]
```
## Step 4: Create and configure users
You can create users by using the Active Directory Users and Computers tool. This is one of the Active Directory Domain Services and Active Directory Lightweight Directory Services tools. For more information, see [Add Users and Computers to the Active Directory domain](https://learn.microsoft.com/en-us/troubleshoot/windows-server/identity/create-an-active-directory-server#add-users-and-computers-to-the-active-directory-domain) in the Microsoft documentation. In this case, users are individuals or other entities, such as their computers that are part of the domain and whose identities are being maintained in the directory.
To create users in an Directory Service directory, you must be connected to a Windows-based Amazon EC2 instance that's a member of the Directory Service directory. At the same time, you must be logged in as a user that has privileges to create users. For more information, see [Create a user](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_manage_users_groups_create_user.html) in the *AWS Directory Service Administration Guide*.
## Step 5: Enable cross-VPC traffic between the directory and the DB instance
If you plan to locate the directory and the DB instance in the same VPC, skip this step and move on to [Step 6: Create or modify a PostgreSQL DB instance](#postgresql-kerberos-setting-up.create-modify).
If you plan to locate the directory and the DB instance in different VPCs, configure cross-VPC traffic using VPC peering or [AWS Transit Gateway](https://docs.aws.amazon.com/vpc/latest/tgw/what-is-transit-gateway.html).
The following procedure enables traffic between VPCs using VPC peering. Follow the instructions in [What is VPC peering?](https://docs.aws.amazon.com/vpc/latest/peering/Welcome.html) in the *Amazon Virtual Private Cloud Peering Guide*.
**To enable cross-VPC traffic using VPC peering**
1. Set up appropriate VPC routing rules to ensure that network traffic can flow both ways.
1. Ensure that the DB instance security group can receive inbound traffic from the directory security group.
1. Ensure that there is no network access control list (ACL) rule to block traffic.
If a different AWS account owns the directory, you must share the directory.
**To share the directory between AWS accounts**
1. Start sharing the directory with the AWS account that the DB instance will be created in by following the instructions in [Tutorial: Sharing your AWS Managed Microsoft AD directory for seamless EC2 Domain-join](https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_tutorial_directory_sharing.html) in the *Directory Service Administration Guide*.
1. Sign in to the Directory Service console using the account for the DB instance, and ensure that the domain has the `SHARED` status before proceeding.
1. While signed into the Directory Service console using the account for the DB instance, note the **Directory ID** value. You use this directory ID to join the DB instance to the domain.
## Step 6: Create or modify a PostgreSQL DB instance
Create or modify a PostgreSQL DB instance for use with your directory. You can use the console, CLI, or RDS API to associate a DB instance with a directory. You can do this in one of the following ways:
+ Create a new PostgreSQL DB instance using the console, the [ create-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-instance.html) CLI command, or the [CreateDBInstance](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstance.html) RDS API operation. For instructions, see [Creating an Amazon RDS DB instance](USER_CreateDBInstance.md).
+ Modify an existing PostgreSQL DB instance using the console, the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) CLI command, or the [ModifyDBInstance](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ModifyDBInstance.html) RDS API operation. For instructions, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md).
+ Restore a PostgreSQL DB instance from a DB snapshot using the console, the [restore-db-instance-from-db-snapshot](https://docs.aws.amazon.com/cli/latest/reference/rds/restore-db-instance-from-db-snapshot.html) CLI command, or the [ RestoreDBInstanceFromDBSnapshot](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_RestoreDBInstanceFromDBSnapshot.html) RDS API operation. For instructions, see [Restoring to a DB instance](USER_RestoreFromSnapshot.md).
+ Restore a PostgreSQL DB instance to a point-in-time using the console, the [ restore-db-instance-to-point-in-time](https://docs.aws.amazon.com/cli/latest/reference/rds/restore-db-instance-to-point-in-time.html) CLI command, or the [ RestoreDBInstanceToPointInTime](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_RestoreDBInstanceToPointInTime.html) RDS API operation. For instructions, see [Restoring a DB instance to a specified time for Amazon RDS](USER_PIT.md).
Kerberos authentication is only supported for PostgreSQL DB instances in a VPC. The DB instance can be in the same VPC as the directory, or in a different VPC. The DB instance must use a security group that allows ingress and egress within the directory's VPC so the DB instance can communicate with the directory.
### Console
When you use the console to create, modify, or restore a DB instance, choose **Password and Kerberos authentication** in the **Database authentication** section. Then choose **Browse Directory**. Select the directory or choose **Create a new directory** to use the Directory Service.
![\[Choosing Kerberos for authentication and identifying the directory to use.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/rpg-authentication-use-kerberos.png)
### AWS CLI
When you use the AWS CLI, the following parameters are required for the DB instance to be able to use the directory that you created:
+ For the `--domain` parameter, use the domain identifier ("d-\$1" identifier) generated when you created the directory.
+ For the `--domain-iam-role-name` parameter, use the role you created that uses the managed IAM policy `AmazonRDSDirectoryServiceAccess`.
For example, the following CLI command modifies a DB instance to use a directory.
```
aws rds modify-db-instance --db-instance-identifier mydbinstance --domain d-Directory-ID --domain-iam-role-name role-name
```
**Important**
If you modify a DB instance to enable Kerberos authentication, reboot the DB instance after making the change.
## Step 7: Create PostgreSQL users for your Kerberos principals
At this point, your RDS for PostgreSQL DB instance is joined to the AWS Managed Microsoft AD domain. The users that you created in the directory in [Step 4: Create and configure users](#postgresql-kerberos-setting-up.create-users) need to be set up as PostgreSQL database users and granted privileges to login to the database. You do that by signing in as the database user with `rds_superuser` privileges. For example, if you accepted the defaults when you created your RDS for PostgreSQL DB instance, you use `postgres`, as shown in the following steps.
**To create PostgreSQL database users for Kerberos principals**
1. Use `psql` to connect to your RDS for PostgreSQL DB instance endpoint using `psql`. The following example uses the default `postgres` account for the `rds_superuser` role.
```
psql --host=cluster-instance-1.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. Create a database user name for each Kerberos principal (Active Directory username) that you want to have access to the database. Use the canonical username (identity) as defined in the Active Directory instance, that is, a lower-case `alias` (username in Active Directory) and the upper-case name of the Active Directory domain for that user name. The Active Directory user name is an externally authenticated user, so use quotes around the name as shown following.
```
postgres=> CREATE USER "username@CORP.EXAMPLE.COM" WITH LOGIN;
CREATE ROLE
```
1. Grant the `rds_ad` role to the database user.
```
postgres=> GRANT rds_ad TO "username@CORP.EXAMPLE.COM";
GRANT ROLE
```
After you finish creating all the PostgreSQL users for your Active Directory user identities, users can access the RDS for PostgreSQL DB instance by using their Kerberos credentials.
It's required that the database users who authenticate using Kerberos are doing so from client machines that are members of the Active Directory domain.
Database users that have been granted the `rds_ad` role can't also have the `rds_iam` role. This also applies to nested memberships. For more information, see [IAM database authentication for MariaDB, MySQL, and PostgreSQL](UsingWithRDS.IAMDBAuth.md).
## Step 8: Configure a PostgreSQL client
To configure a PostgreSQL client, take the following steps:
+ Create a krb5.conf file (or equivalent) to point to the domain.
+ Verify that traffic can flow between the client host and Directory Service. Use a network utility such as Netcat for the following:
+ Verify traffic over DNS for port 53.
+ Verify traffic over TCP/UDP for port 53 and for Kerberos, which includes ports 88 and 464 for Directory Service.
+ Verify that traffic can flow between the client host and the DB instance over the database port. For example, use psql to connect and access the database.
The following is sample krb5.conf content for AWS Managed Microsoft AD.
```
[libdefaults]
default_realm = EXAMPLE.COM
[realms]
EXAMPLE.COM = {
kdc = example.com
admin_server = example.com
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
```
The following is sample krb5.conf content for an on-premises Microsoft Active Directory.
```
[libdefaults]
default_realm = EXAMPLE.COM
[realms]
EXAMPLE.COM = {
kdc = example.com
admin_server = example.com
}
ONPREM.COM = {
kdc = onprem.com
admin_server = onprem.com
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
.onprem.com = ONPREM.COM
onprem.com = ONPREM.COM
.rds.amazonaws.com = EXAMPLE.COM
.amazonaws.com.rproxy.goskope.com.cn = EXAMPLE.COM
.amazon.com = EXAMPLE.COM
```
# Managing an RDS for PostgreSQL DB instance in an Active Directory domain
You can use the console, the CLI, or the RDS API to manage your DB instance and its relationship with your Microsoft Active Directory. For example, you can associate an Active Directory to enable Kerberos authentication. You can also remove the association for an Active Directory to disable Kerberos authentication. You can also move a DB instance to be externally authenticated by one Microsoft Active Directory to another.
For example, using the CLI, you can do the following:
+ To reattempt enabling Kerberos authentication for a failed membership, use the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) CLI command. Specify the current membership's directory ID for the `--domain` option.
+ To disable Kerberos authentication on a DB instance, use the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) CLI command. Specify `none` for the `--domain` option.
+ To move a DB instance from one domain to another, use the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) CLI command. Specify the domain identifier of the new domain for the `--domain` option.
## Understanding Domain membership
After you create or modify your DB instance, it becomes a member of the domain. You can view the status of the domain membership in the console or by running the [describe-db-instances](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-instances.html) CLI command. The status of the DB instance can be one of the following:
+ `kerberos-enabled` – The DB instance has Kerberos authentication enabled.
+ `enabling-kerberos` – AWS is in the process of enabling Kerberos authentication on this DB instance.
+ `pending-enable-kerberos` – Enabling Kerberos authentication is pending on this DB instance.
+ `pending-maintenance-enable-kerberos` – AWS will attempt to enable Kerberos authentication on the DB instance during the next scheduled maintenance window.
+ `pending-disable-kerberos` – Disabling Kerberos authentication is pending on this DB instance.
+ `pending-maintenance-disable-kerberos` – AWS will attempt to disable Kerberos authentication on the DB instance during the next scheduled maintenance window.
+ `enable-kerberos-failed` – A configuration problem prevented AWS from enabling Kerberos authentication on the DB instance. Correct the configuration problem before reissuing the command to modify the DB instance.
+ `disabling-kerberos` – AWS is in the process of disabling Kerberos authentication on this DB instance.
A request to enable Kerberos authentication can fail because of a network connectivity issue or an incorrect IAM role. In some cases, the attempt to enable Kerberos authentication might fail when you create or modify a DB instance. If so, make sure that you are using the correct IAM role, then modify the DB instance to join the domain.
**Note**
Only Kerberos authentication with RDS for PostgreSQL sends traffic to the domain's DNS servers. All other DNS requests are treated as outbound network access on your DB instances running PostgreSQL. For more information about outbound network access with RDS for PostgreSQL, see [Using a custom DNS server for outbound network access](Appendix.PostgreSQL.CommonDBATasks.CustomDNS.md).
# Connecting to PostgreSQL with Kerberos authentication
You can connect to PostgreSQL with Kerberos authentication with the pgAdmin interface or with a command-line interface such as psql. For more information about connecting, see [Connecting to a DB instance running the PostgreSQL database engine](USER_ConnectToPostgreSQLInstance.md) . For information about obtaining the endpoint, port number, and other details needed for connection, see [Connect to a PostgreSQL DB instance](CHAP_GettingStarted.CreatingConnecting.PostgreSQL.md#CHAP_GettingStarted.Connecting.PostgreSQL).
**Note**
GSSAPI authentication and encryption in PostgreSQL are implemented by the Kerberos library `libkrb5.so`. Features such as `postgres_fdw` and `dblink` also rely on this same library for outbound connections with Kerberos authentication or encryption.
## pgAdmin
To use pgAdmin to connect to PostgreSQL with Kerberos authentication, take the following steps:
1. Launch the pgAdmin application on your client computer.
1. On the **Dashboard** tab, choose **Add New Server**.
1. In the **Create - Server** dialog box, enter a name on the **General** tab to identify the server in pgAdmin.
1. On the **Connection** tab, enter the following information from your RDS for PostgreSQL database.
+ For **Host**, enter the endpoint for the RDS for PostgreSQL DB instance. An endpoint looks similar to the following:
```
RDS-DB-instance.111122223333.aws-region.rds.amazonaws.com
```
To connect to an on-premises Microsoft Active Directory from a Windows client, you use the domain name of the AWS Managed Active Directory instead of `rds.amazonaws.com` in the host endpoint. For example, suppose that the domain name for the AWS Managed Active Directory is `corp.example.com`. Then for **Host**, the endpoint would be specified as follows:
```
RDS-DB-instance.111122223333.aws-region.corp.example.com
```
+ For **Port**, enter the assigned port.
+ For **Maintenance database**, enter the name of the initial database to which the client will connect.
+ For **Username**, enter the user name that you entered for Kerberos authentication in [Step 7: Create PostgreSQL users for your Kerberos principals](postgresql-kerberos-setting-up.md#postgresql-kerberos-setting-up.create-logins).
1. Choose **Save**.
## Psql
To use psql to connect to PostgreSQL with Kerberos authentication, take the following steps:
1. At a command prompt, run the following command.
```
kinit username
```
Replace *`username`* with the user name. At the prompt, enter the password stored in the Microsoft Active Directory for the user.
1. If the PostgreSQL DB instance is using a publicly accessible VPC, put IP address for your DB instance endpoint in your `/etc/hosts` file on the EC2 client. For example, the following commands obtain the IP address and then put it in the `/etc/hosts` file.
```
% dig +short PostgreSQL-endpoint.AWS-Region.rds.amazonaws.com
;; Truncated, retrying in TCP mode.
ec2-34-210-197-118.AWS-Region.compute.amazonaws.com.
34.210.197.118
% echo " 34.210.197.118 PostgreSQL-endpoint.AWS-Region.rds.amazonaws.com" >> /etc/hosts
```
If you're using an on-premises Microsoft Active Directory from a Windows client, then you need to connect using a specialized endpoint. Instead of using the Amazon domain `rds.amazonaws.com` in the host endpoint, use the domain name of the AWS Managed Active Directory.
For example, suppose that the domain name for your AWS Managed Active Directory is `corp.example.com`. Then use the format `PostgreSQL-endpoint.AWS-Region.corp.example.com` for the endpoint and put it in the `/etc/hosts` file.
```
% echo " 34.210.197.118 PostgreSQL-endpoint.AWS-Region.corp.example.com" >> /etc/hosts
```
1. Use the following psql command to log in to a PostgreSQL DB instance that is integrated with Active Directory.
```
psql -U username@CORP.EXAMPLE.COM -p 5432 -h PostgreSQL-endpoint.AWS-Region.rds.amazonaws.com postgres
```
To log in to the PostgreSQL DB cluster from a Windows client using an on-premises Active Directory, use the following psql command with the domain name from the previous step (`corp.example.com`):
```
psql -U username@CORP.EXAMPLE.COM -p 5432 -h PostgreSQL-endpoint.AWS-Region.corp.example.com postgres
```
# Using a custom DNS server for outbound network access
RDS for PostgreSQL supports outbound network access on your DB instances and allows Domain Name Service (DNS) resolution from a custom DNS server owned by the customer. You can resolve only fully qualified domain names from your RDS for PostgreSQL DB instance through your custom DNS server.
**Topics**
+ [
## Turning on custom DNS resolution
](#Appendix.PostgreSQL.CommonDBATasks.CustomDNS.Enable)
+ [
## Turning off custom DNS resolution
](#Appendix.PostgreSQL.CommonDBATasks.CustomDNS.Disable)
+ [
## Setting up a custom DNS server
](#Appendix.Oracle.CommonDBATasks.CustomDNS.Setup)
## Turning on custom DNS resolution
To turn on DNS resolution in your customer VPC, first associate a custom DB parameter group to your RDS for PostgreSQL instance. Then turn on the `rds.custom_dns_resolution` parameter by setting it to 1, and then restart the DB instance for the changes to take place.
## Turning off custom DNS resolution
To turn off DNS resolution in your customer VPC, first turn off the `rds.custom_dns_resolution` parameter of your custom DB parameter group by setting it to 0. Then restart the DB instance for the changes to take place.
## Setting up a custom DNS server
After you set up your custom DNS name server, it takes up to 30 minutes to propagate the changes to your DB instance. After the changes are propagated to your DB instance, all outbound network traffic requiring a DNS lookup queries your DNS server over port 53.
**Note**
If you don't set up a custom DNS server and `rds.custom_dns_resolution` is set to 1, hosts are resolved using an Amazon Route 53 private zone. For more information, see [Working with private hosted zones](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/hosted-zones-private.html).
**To set up a custom DNS server for your RDS for PostgreSQL DB instance**
1. From the Dynamic Host Configuration Protocol (DHCP) options set attached to your VPC, set the `domain-name-servers` option to the IP address of your DNS name server. For more information, see [DHCP options sets](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_DHCP_Options.html).
**Note**
The `domain-name-servers` option accepts up to four values, but your Amazon RDS DB instance uses only the first value.
1. Ensure that your DNS server can resolve all lookup queries, including public DNS names, Amazon EC2 private DNS names, and customer-specific DNS names. If the outbound network traffic contains any DNS lookups that your DNS server can't handle, your DNS server must have appropriate upstream DNS providers configured.
1. Configure your DNS server to produce User Datagram Protocol (UDP) responses of 512 bytes or less.
1. Configure your DNS server to produce Transmission Control Protocol (TCP) responses of 1,024 bytes or less.
1. Configure your DNS server to allow inbound traffic from your Amazon RDS DB instances over port 53. If your DNS server is in an Amazon VPC, the VPC must have a security group that contains inbound rules that allow UDP and TCP traffic on port 53. If your DNS server is not in an Amazon VPC, it must have appropriate firewall settings to allow UDP and TCP inbound traffic on port 53.
For more information, see [Security groups for your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) and [Adding and removing rules](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#AddRemoveRules).
1. Configure the VPC of your Amazon RDS DB instance to allow outbound traffic over port 53. Your VPC must have a security group that contains outbound rules that allow UDP and TCP traffic on port 53.
For more information, see [Security groups for your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html) and [Adding and removing rules](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html#AddRemoveRules) in the *Amazon VPC User Guide*.
1. Make sure that the routing path between the Amazon RDS DB instance and the DNS server is configured correctly to allow DNS traffic.
Also, if the Amazon RDS DB instance and the DNS server are not in the same VPC, make sure that a peering connection is set up between them. For more information, see [What is VPC peering?](https://docs.aws.amazon.com/vpc/latest/peering/Welcome.html) in *Amazon VPC Peering Guide*.
# Upgrades of the RDS for PostgreSQL DB engine
There are two types of upgrades that you can manage for your PostgreSQL database:
+ Operating system updates – Occasionally, Amazon RDS might need to update the underlying operating system of your database to apply security fixes or OS changes. You can decide when Amazon RDS applies OS updates by using the RDS console, AWS Command Line Interface (AWS CLI), or RDS API. For more information about OS updates, see [Applying updates to a DB instance ](USER_UpgradeDBInstance.Maintenance.md#USER_UpgradeDBInstance.OSUpgrades).
+ Database engine upgrades – When Amazon RDS supports a new version of a database engine, you can upgrade your databases to the new version.
A *database* in this context is an RDS for PostgreSQL DB instance or Multi-AZ DB cluster.
There are two kinds of engine upgrades for PostgreSQL databases: major version upgrades and minor version upgrades.
**Major version upgrades**
*Major version upgrades* can contain database changes that are not backward-compatible with existing applications. As a result, you must manually perform major version upgrades of your databases. You can initiate a major version upgrade by modifying your DB instance or Multi-AZ DB cluster. Before you perform a major version upgrade, we recommend that you follow the steps described in [Choosing a major version for an RDS for PostgreSQL upgrade](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.md).
Amazon RDS handles Multi-AZ major version upgrades in the following ways:
+ **Multi-AZ DB instance deployment** – Amazon RDS simultaneously upgrades the primary and any standby instances. Your database might not be available for several minutes while the upgrade completes.
+ **Multi-AZ DB cluster deployment** – Amazon RDS simultaneously upgrades the reader and writer instances. Your database might not be available for several minutes while the upgrade completes.
If you upgrade a DB instance that has in-Region read replicas, Amazon RDS upgrades the replicas along with the primary DB instance.
Amazon RDS doesn't upgrade Multi-AZ DB cluster read replicas. If you perform a major version upgrade of a Multi-AZ DB cluster, then the replication state of its read replicas changes to **terminated**. You must manually delete and recreate the read replicas after the upgrade completes.
You can minimize the downtime required for a major version upgrade by using a blue/green deployment. For more information, see [Using Amazon RDS Blue/Green Deployments for database updates](blue-green-deployments.md).
**Minor version upgrades**
In contrast, *minor version upgrades* include only changes that are backward-compatible with existing applications. You can initiate a minor version upgrade manually by modifying your database. Or, you can enable the **Auto minor version upgrade** option when you create or modify a database. Doing so means that Amazon RDS automatically upgrades your database after testing and approving the new version.
Amazon RDS handles Multi-AZ minor version upgrades in the following ways:
+ **Multi-AZ DB instance deployment** – Amazon RDS simultaneously upgrades the primary and any standby instances. Your database might not be available for several minutes while the upgrade completes.
+ **Multi-AZ DB cluster deployment** – Amazon RDS upgrades the reader DB instances one at a time. Then, one of the reader DB instances switches to be the new writer DB instance. Amazon RDS then upgrades the old writer instance (which is now a reader instance). Multi-AZ DB clusters typically reduce the downtime of minor version upgrades to approximately 35 seconds. When used with RDS Proxy, they can further reduce downtime to one second or less. For more information, see [Amazon RDS Proxy](rds-proxy.md). Alternately, you can use an open source database proxy such as [ProxySQL](https://aws.amazon.com/blogs/database/achieve-one-second-or-less-of-downtime-with-proxysql-when-upgrading-amazon-rds-multi-az-deployments-with-two-readable-standbys/), [PgBouncer](https://aws.amazon.com/blogs/database/fast-switchovers-with-pgbouncer-on-amazon-rds-multi-az-deployments-with-two-readable-standbys-for-postgresql/), or the [AWS Advanced JDBC Wrapper Driver](https://aws.amazon.com/blogs/database/achieve-one-second-or-less-downtime-with-the-advanced-jdbc-wrapper-driver-when-upgrading-amazon-rds-multi-az-db-clusters/).
If your database has read replicas, you must first upgrade all of the read replicas before you upgrade the source instance or cluster.
For more information, see [Automatic minor version upgrades for RDS for PostgreSQL](USER_UpgradeDBInstance.PostgreSQL.Minor.md). For information about manually performing a minor version upgrade, see [Manually upgrading the engine version](USER_UpgradeDBInstance.Upgrading.md#USER_UpgradeDBInstance.Upgrading.Manual).
For more information about database engine versions and the policy for deprecating database engine versions, see [Database Engine Versions](https://aws.amazon.com/rds/faqs/#Database_Engine_Versions) in the Amazon RDS FAQs.
**Topics**
+ [
## Considerations for PostgreSQL upgrades
](#USER_UpgradeDBInstance.PostgreSQL.Considerations)
+ [
## Finding valid upgrade targets
](#USER_UpgradeDBInstance.PostgreSQL.FindingTargets)
+ [
# PostgreSQL version numbers
](USER_UpgradeDBInstance.PostgreSQL.VersionID.md)
+ [
# RDS version numbers in RDS for PostgreSQL
](USER_UpgradeDBInstance.PostgreSQL.rds.version.md)
+ [
# Choosing a major version for an RDS for PostgreSQL upgrade
](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.md)
+ [
# How to perform a major version upgrade for RDS for PostgreSQL
](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.Process.md)
+ [
# Automatic minor version upgrades for RDS for PostgreSQL
](USER_UpgradeDBInstance.PostgreSQL.Minor.md)
+ [
# Upgrading PostgreSQL extensions in RDS for PostgreSQL databases
](USER_UpgradeDBInstance.PostgreSQL.ExtensionUpgrades.md)
+ [
# Monitoring RDS for PostgreSQL engine upgrades with events
](USER_UpgradeDBInstance.PostgreSQL.Monitoring.md)
## Considerations for PostgreSQL upgrades
To safely upgrade your databases, Amazon RDS uses the `pg_upgrade` utility described in the [PostgreSQL documentation](https://www.postgresql.org/docs/current/pgupgrade.html)
If your backup retention period is greater than 0, Amazon RDS takes two DB snapshots during the upgrade process. The first DB snapshot is of the database before any upgrade changes have been made. If the upgrade fails for your databases, you can restore this snapshot to create a database running the old version. The second DB snapshot is taken after the upgrade completes. These DB snapshots are deleted automatically once the backup retention period expires.
**Note**
Amazon RDS takes DB snapshots during the upgrade process only if you have set the backup retention period for your database to a number greater than 0. To change the backup retention period for a DB instance, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md). You can't configure a custom backup retention period for a Multi-AZ DB cluster.
When you perform a major version upgrade of a DB instance, any in-Region read replicas are also automatically upgraded. After the upgrade workflow starts, the read replicas wait for the `pg_upgrade` to complete successfully on the primary DB instance. Then the primary DB instance upgrade waits for the read replica upgrades to complete. You experience an outage until the upgrade is complete. When you perform a major version upgrade of a Multi-AZ DB cluster, the replication state of its read replicas changes to **terminated**.
After an upgrade is complete, you can't revert to the previous version of the DB engine. If you want to return to the previous version, restore the DB snapshot that was taken before the upgrade to create a new database.
## Finding valid upgrade targets
When you use the AWS Management Console to upgrade a database, it shows the valid upgrade targets for the database. You can also use the following AWS CLI command to identify the valid upgrade targets for a database:
For Linux, macOS, or Unix:
```
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version version-number \
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
```
For Windows:
```
aws rds describe-db-engine-versions ^
--engine postgres ^
--engine-version version-number ^
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
```
For example, to identify the valid upgrade targets for a PostgreSQL version 16.1 database, run the following AWS CLI command:
For Linux, macOS, or Unix:
```
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version 16.1 \
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
```
For Windows:
```
aws rds describe-db-engine-versions ^
--engine postgres ^
--engine-version 16.1 ^
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
```
# PostgreSQL version numbers
The version numbering sequence for the PostgreSQL database engine is as follows:
+ For PostgreSQL versions 10 and higher, the engine version number is in the form *major.minor*. The major version number is the integer part of the version number. The minor version number is the fractional part of the version number.
A major version upgrade increases the integer part of the version number, such as upgrading from 10.*minor* to 11.*minor*.
+ For PostgreSQL versions lower than 10, the engine version number is in the form *major.major.minor*. The major engine version number is both the integer and the first fractional part of the version number. For example, 9.6 is a major version. The minor version number is the third part of the version number. For example, for version 9.6.12, the 12 is the minor version number.
A major version upgrade increases the major part of the version number. For example, an upgrade from *9.6*.12 to 11.14 is a major version upgrade, where *9.6* and *11* are the major version numbers.
For information about RDS Extended Support version numbering, see [Amazon RDS Extended Support version naming](extended-support-versions.md#extended-support-naming).
# RDS version numbers in RDS for PostgreSQL
RDS version numbers use the `major.minor.patch` naming scheme. An RDS patch version includes important bug fixes added to a minor version after its release. For information about RDS Extended Support version numbering, see [Amazon RDS Extended Support version naming](extended-support-versions.md#extended-support-naming).
To identify the Amazon RDS version number of your database, you must first create the `rds_tools` extension by using the following command:
```
CREATE EXTENSION rds_tools;
```
Starting with the release of PostgreSQL version 15.2-R2, you can find out the RDS version number of your RDS for PostgreSQL database with the following SQL query:
```
postgres=> SELECT rds_tools.rds_version();
```
For example, querying an RDS for PostgreSQL 15.2 database returns the following:
```
rds_version
----------------
15.2.R2
(1 row)
```
# Choosing a major version for an RDS for PostgreSQL upgrade
Major version upgrades can contain changes that are not backward-compatible with previous versions of the database. New functionality can cause your existing applications to stop working correctly. For this reason, Amazon RDS doesn't apply major version upgrades automatically. To perform a major version upgrade, you modify your database manually. Make sure that you thoroughly test any upgrade to verify that your applications work correctly before applying the upgrade to your production databases. When you do a PostgreSQL major version upgrade, we recommend that you follow the steps described in [How to perform a major version upgrade for RDS for PostgreSQL](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.Process.md).
When you upgrade a PostgreSQL Single-AZ DB instance or Multi-AZ DB instance deployment to its next major version, any read replicas associated with the database are also upgraded to that next major version. In some cases, you can skip to a higher major version when upgrading. If your upgrade skips a major version, the read replicas are also upgraded to that target major version. Upgrades to version 11 that skip other major versions have certain limitations. You can find the details in the steps described in [How to perform a major version upgrade for RDS for PostgreSQL](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.Process.md).
Most PostgreSQL extensions aren't upgraded during a PostgreSQL engine upgrade. These must be upgraded separately. For more information, see [Upgrading PostgreSQL extensions in RDS for PostgreSQL databases](USER_UpgradeDBInstance.PostgreSQL.ExtensionUpgrades.md).
You can find out which major versions are available for your RDS for PostgreSQL database by running the following AWS CLI query:
```
aws rds describe-db-engine-versions --engine postgres --engine-version your-version --query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text
```
The following table summarizes the results of this query for all available versions. An asterisk (\$1) on the version number means that version is no longer supported. If your current version is unsupported, we recommend that you upgrade to the newest minor version upgrade target or to one of the other available upgrade targets for that version.
| Current source version | Upgrade targets |
| --- | --- |
| 17.6 | None |
| 17.5 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176) |
| 17.4 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) |
| 17.3\$1, 17.2 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) |
| 17.1\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) |
| 16.10 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176) |
| 16.9 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610) |
| 16.8 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) |
| 16.7\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) |
| 16.7 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) |
| 16.6 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) |
| 16.5\$1, 16.4 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) |
| 16.3 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164) |
| 16.2\$1, 16.1\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) |
| 15.14 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610) |
| 15.13 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514) |
| 15.12, 15.11\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513) |
| 15.10 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512) |
| 15.9\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) |
| 15.8 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) |
| 15.7 | [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version167), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version165), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.11](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1511), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version159), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158) |
| 15.6\$1, 15.5\$1, 15.4\$1, 15.3\$1, 15.2\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) |
| 14.19 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514) |
| 14.18 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419) |
| 14.17, 14.16\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418) |
| 14.15 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417) |
| 14.14\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415) |
| 14.13 | [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164) [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.11](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1511), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version159), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158) [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1416), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1414) |
| 14.12 | [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.11](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1511), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version159), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1416), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1414), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413) |
| 14.11\$1, 14.10\$1, 14.9\$1, 14.8\$1, 14.7\$1, 14.6\$1, 14.5\$1, 14.4\$1, 14.3\$1, 14.2\$1, 14.1\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) |
| 13.22 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419) |
| 13.21 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322) |
| 13.20, 13.19\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321) |
| 13.18, 13.17\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320) |
| 13.16 | [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164) [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158) [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1414), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413) [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1319), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1317) |
| 13.15 | [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1414), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1319), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1317), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316) |
| 13.14\$1, 13.13\$1, 13.12\$1, 13.11\$1, 13.10\$1, 13.9\$1, 13.8\$1, 13.7\$1, 13.6\$1, 13.5\$1, 13.4\$1, 13.3\$1, 13.2\$1, 13.1\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316), [13.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1315) |
| 12.22-rds.20250508 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321) |
| 12.22-rds.20250220 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320) [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) |
| 12.22, 12.21\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318) [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) |
| 12.20\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316) [12.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222), [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) |
| 12.19\$1, 12.18\$1, 12.17\$1, 12.16\$1, 12.15\$1, 12.14\$1, 12.13\$1, 12.12\$1, 12.11\$1, 12.10\$1, 12.9\$1, 12.8\$1, 12.7\$1, 12.6\$1, 12.5\$1, 12.4\$1, 12.3\$1, 12.2\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316), [13.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1315) [12.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222), [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) |
| 11.22-rds.20250508 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321) [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) |
| 11.22-rds.20250220 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320) [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) [11.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20250508) |
| 11.22-rds.20240509 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316), [13.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1315) [12.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222), [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) [11.22-rds.20240808](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20240808), [11.22-rds.20241121](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20241121), [11.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20250220), [11.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20250508) |
| 11.22, 11.21\$1, 11.20\$1, 11.19\$1, 11.18\$1, 11.17\$1, 11.16\$1, 11.15\$1, 11.14\$1, 11.13\$1, 11.12\$1, 11.11\$1, 11.10\$1, 11.9\$1, 11.8\$1, 11.7\$1, 11.6\$1, 11.5\$1, 11.4\$1, 11.2\$1, 11.1\$1 | [17.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version176), [17.5](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version175), [17.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version174), [17.2](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version172) [16.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1610), [16.9](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version169), [16.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version168), [16.6](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version166), [16.4](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version164), [16.3](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version163) [15.14](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1514), [15.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1513), [15.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1512), [15.10](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1510), [15.8](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version158), [15.7](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version157) [14.19](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1419), [14.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1418), [14.17](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1417), [14.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1415), [14.13](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1413), [14.12](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1412) [13.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1322), [13.21](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1321), [13.20](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1320), [13.18](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1318), [13.16](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1316), [13.15](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1315) [12.22](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222), [12.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250220), [12.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1222rds20250508) [11.22-rds.20240418](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20240418), [11.22-rds.20240509](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20240509), [11.22-rds.20240808](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20240808), [11.22-rds.20241121](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20241121), [11.22-rds.20250220](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20250220), [11.22-rds.20250508](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html#postgresql-versions-version1122rds20250508) |
\$1 This version is no longer supported.
# How to perform a major version upgrade for RDS for PostgreSQL
We recommend the following process when performing a major version upgrade on an Amazon RDS for PostgreSQL database:
1. **Have a version-compatible parameter group ready** – If you are using a custom parameter group, you have two options. You can specify a default parameter group for the new DB engine version. Or you can create your own custom parameter group for the new DB engine version. For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md) and [Working with DB cluster parameter groups for Multi-AZ DB clusters](USER_WorkingWithDBClusterParamGroups.md).
1. **Check for unsupported database classes** – Check that your database's instance class is compatible with the PostgreSQL version you are upgrading to. For more information, see [Supported DB engines for DB instance classes](Concepts.DBInstanceClass.Support.md).
1. **Check for unsupported usage:**
+ **Prepared transactions** – Commit or roll back all open prepared transactions before attempting an upgrade.
You can use the following query to verify that there are no open prepared transactions on your database.
```
SELECT count(*) FROM pg_catalog.pg_prepared_xacts;
```
+ **Reg\$1 data types** – Remove all uses of the *reg\$1* data types before attempting an upgrade. Except for `regtype` and `regclass`, you can't upgrade the *reg\$1* data types. The `pg_upgrade` utility can't persist this data type, which is used by Amazon RDS to do the upgrade.
To verify that there are no uses of unsupported *reg\$1* data types, use the following query for each database.
```
SELECT count(*) FROM pg_catalog.pg_class c, pg_catalog.pg_namespace n, pg_catalog.pg_attribute a
WHERE c.oid = a.attrelid
AND NOT a.attisdropped
AND a.atttypid IN ('pg_catalog.regproc'::pg_catalog.regtype,
'pg_catalog.regprocedure'::pg_catalog.regtype,
'pg_catalog.regoper'::pg_catalog.regtype,
'pg_catalog.regoperator'::pg_catalog.regtype,
'pg_catalog.regconfig'::pg_catalog.regtype,
'pg_catalog.regdictionary'::pg_catalog.regtype)
AND c.relnamespace = n.oid
AND n.nspname NOT IN ('pg_catalog', 'information_schema');
```
1. **Check for invalid databases:**
+ Ensure there are no invalid databases. The `datconnlimit` column in the `pg_database` catalog includes a value of `-2` to mark databases as invalid that were interrupted during a `DROP DATABASE` operation.
Use the following query to check for invalid databases:
```
SELECT datname FROM pg_database WHERE datconnlimit = - 2;
```
+ The previous query returns invalid database names. You can use `DROP DATABASE invalid_db_name;` to drop invalid databases. You can also use the following command to drop invalid databases:
```
SELECT 'DROP DATABASE ' || quote_ident(datname) || ';' FROM pg_database WHERE datconnlimit = -2 \gexec
```
For more information about invalid databases, see [Understanding the behavior of autovacuum with invalid databases](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/appendix.postgresql.commondbatasks.autovacuumbehavior.html).
1. **Handle logical replication slots** – An upgrade can't occur if the database has any logical replication slots. Logical replication slots are typically used for AWS DMS migration and for replicating tables from the database to data lakes, BI tools, and other targets. Before upgrading, make sure that you know the purpose of any logical replication slots that are in use, and confirm that it's okay to delete them. If the logical replication slots are still being used, you shouldn't delete them, and you can't proceed with the upgrade.
If the logical replication slots aren't needed, you can delete them using the following SQL:
```
SELECT * FROM pg_replication_slots WHERE slot_type NOT LIKE 'physical';
SELECT pg_drop_replication_slot(slot_name);
```
Logical replication setups that use the `pglogical` extension also need to have slots dropped for a successful major version upgrade. For information about how to identify and drop slots created using the `pglogical` extension, see [Managing logical replication slots for RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.pglogical.handle-slots.md).
On source version 17 and later, logical replication slots on non-read-replicas can be retained through upgrades. Logical replication slots created on read replicas are not retained through upgrades.
Ensure that all transactions and logical decoding messages have been consumed from the slot before initiating the upgrade. If there are unconsumed write-ahead log files (WAL) held by logical replication slots, the upgrade will fail with a message identifying the problem slots. See the [PostgreSQL documentation](https://www.postgresql.org/docs/current/logical-replication-upgrade.html) for further details.
On Multi-AZ clusters with source versions earlier than 17.8 or 18.2, ensure that `flow_control` is disabled. For more information, see [Turning on and turning off flow control for Multi-AZ DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/multi-az-db-clusters-concepts.html#multi-az-db-clusters-concepts-replica-lag). You can turn off flow control by removing the extension from the `shared_preload_libraries` and rebooting your DB instance.
1. **Handle read replicas** – An upgrade of a Single-AZ DB instance or Multi-AZ DB instance deployment also upgrades the in-Region read replicas along with the primary DB instance. Amazon RDS doesn't upgrade Multi-AZ DB cluster read replicas.
You can't upgrade read replicas separately. If you could, it could lead to situations where the primary and replica databases have different PostgreSQL major versions. However, read replica upgrades might increase downtime on the primary DB instance. To prevent a read replica upgrade, promote the replica to a standalone instance or delete it before starting the upgrade process.
The upgrade process recreates the read replica's parameter group based on the read replica's current parameter group. You can apply a custom parameter group to a read replica only after the upgrade completes by modifying the read replica. For more information about read replicas, see [Working with read replicas for Amazon RDS for PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.md).
1. **Handle large objects** – In PostgreSQL, large objects (also known as BLOBs) are used to store and manage large binary objects (like files, images, videos, etc.) that are larger than the maximum size allowed for regular column data types. For more information see [PostgreSQL Large Objects documentation](https://www.postgresql.org/docs/current/largeobjects.html).
An upgrade can run out of memory and fail if there are millions of large objects and the instance cannot handle them during an upgrade. The PostgreSQL major version upgrade process comprises of two broad phases: dumping the schema via pg\$1dump and restoring it through pg\$1restore. If your database has millions of large objects you need to ensure your instance has sufficient memory to handle the pg\$1dump and pg\$1restore during an upgrade and scale it to a larger instance type.
Before you begin an upgrade, check if your database is having any large objects. The catalog `pg_largeobject_metadata` holds metadata associated with large objects. The actual large object data is stored in `pg_largeobject`. Use the following query to check for the number of large objects:
```
SELECT count(*) FROM pg_largeobject_metadata;
```
To cleanup existing large objects or orphaned large objects, see [Managing large objects with the lo module](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/PostgreSQL_large_objects_lo_extension.html).
When planning a major version upgrade, we recommend using an instance type with at least 32 GB of memory if your database contains 25 to 30 million large objects. This recommendation is based on our tests and can vary depending on your specific workload and database configuration. If your database includes additional objects (such as tables, indexes, or materialized views), we recommend selecting a larger instance type to ensure optimal performance during the upgrade process.
1. **Handle zero-ETL integrations** – If you have an existing [zero-ETL integration](zero-etl.md), [delete it](zero-etl.deleting.md) before performing a major version upgrade. Then, after completing the upgrade, recreate the integration.
On source versions majors 17 and up, the zero-ETL integration can be retained through the upgrade.
1. **Perform a backup** – We recommend that you perform a backup before performing the major version upgrade so that you have a known restore point for your database. If your backup retention period is greater than 0, the upgrade process creates DB snapshots of your database before and after upgrading. To change your backup retention period, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md) and [Modifying a Multi-AZ DB cluster for Amazon RDS](modify-multi-az-db-cluster.md).
To perform a backup manually, see [Creating a DB snapshot for a Single-AZ DB instance for Amazon RDS](USER_CreateSnapshot.md) and [Creating a Multi-AZ DB cluster snapshot for Amazon RDS](USER_CreateMultiAZDBClusterSnapshot.md).
1. **Upgrade certain extensions before a major version upgrade** – If you plan to skip a major version with the upgrade, you need to update certain extensions *before* performing the major version upgrade. For example, upgrading from versions 9.5.x or 9.6.x to version 11.x skips a major version. The extensions to update include PostGIS and related extensions for processing spatial data.
+ `address_standardizer`
+ `address_standardizer_data_us`
+ `postgis_raster`
+ `postgis_tiger_geocoder`
+ `postgis_topology`
You cannot directly upgrade to PostgreSQL version 17 if you are using `rdkit` version 4.6.0 and lower, and PostgreSQL version 16 and lower, due to `rdkit` incompatibility. Below are the upgrade options:
+ If you are on PostgreSQL version 13 and lower, you need to perform a major version upgrade to version 14.14 and higher 14 versions, 15.9 and higher 15 versions, or 16.5 and higher 16 versions first, and then perform the version upgrade to PostgreSQL 17.
+ If you are on PostgreSQL version 14, 15, or 16, you need to perform a minor version upgrade to 14.14 and higher 14 versions, 15.9 and higher 15 versions, or 16.5 and higher 16 versions, and then upgrade to PostgreSQL version 17.
Run the following command for each extension that you're using:
```
ALTER EXTENSION PostgreSQL-extension UPDATE TO 'new-version';
```
For more information, see [Upgrading PostgreSQL extensions in RDS for PostgreSQL databases](USER_UpgradeDBInstance.PostgreSQL.ExtensionUpgrades.md). To learn more about upgrading PostGIS, see [Step 6: Upgrade the PostGIS extension](Appendix.PostgreSQL.CommonDBATasks.PostGIS.md#Appendix.PostgreSQL.CommonDBATasks.PostGIS.Update).
1. **Drop certain extensions before the major version upgrade** – Extensions that are not supported on the target version must be dropped, or else the upgrade will fail.
The `plrust` extension is removed starting in RDS PostgreSQL 18. The `postgis_topology` extension is unavailable on RDS PostgreSQL versions 18.1 and 18.2 due to known issues [[1](https://trac.osgeo.org/postgis/ticket/5983)], [[2](https://trac.osgeo.org/postgis/ticket/6016)]. These extensions must be removed prior to upgrading.
An upgrade that skips a major version to version 11.x doesn't support updating the `pgRouting` extension. Upgrading from versions 9.4.x, 9.5.x, or 9.6.x to versions 11.x skips a major version. It's safe to drop the `pgRouting` extension and then reinstall it to a compatible version after the upgrade. For the extension versions you can update to, see [Supported PostgreSQL extension versions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md).
The `tsearch2` and `chkpass` extensions are no longer supported for PostgreSQL versions 11 or later.
You can check if an extension is installed with the following query:
```
SELECT * FROM pg_extension WHERE extname in ('extension_name');
```
1. **Drop unknown data types** – Drop `unknown` data types depending on the target version.
PostgreSQL version 10 stopped supporting the `unknown` data type. If a version 9.6 database uses the `unknown` data type, an upgrade to a version 10 shows an error message such as the following:
```
Database instance is in a state that cannot be upgraded: PreUpgrade checks failed:
The instance could not be upgraded because the 'unknown' data type is used in user tables.
Please remove all usages of the 'unknown' data type and try again."
```
To find the `unknown` data type in your database so you can remove the offending column or change it to a supported data type, use the following SQL:
```
SELECT DISTINCT data_type FROM information_schema.columns WHERE data_type ILIKE 'unknown';
```
1. **Perform an upgrade dry run** – We highly recommend testing a major version upgrade on a duplicate of your production database before attempting the upgrade on your production database. You can monitor the execution plans on the duplicate test database for any possible execution plan regressions and to evaluate its performance. To create a duplicate test instance, you can either restore your database from a recent snapshot or do a point-in-time restore of your database to its latest restorable time.
For more information, see [Restoring from a snapshot](USER_RestoreFromSnapshot.md#USER_RestoreFromSnapshot.Restoring) or [Restoring a DB instance to a specified time for Amazon RDS](USER_PIT.md). For Multi-AZ DB clusters, see [Restoring from a snapshot to a Multi-AZ DB cluster](USER_RestoreFromMultiAZDBClusterSnapshot.Restoring.md) or [Restoring a Multi-AZ DB cluster to a specified time](USER_PIT.MultiAZDBCluster.md).
For details on performing the upgrade, see [Manually upgrading the engine version](USER_UpgradeDBInstance.Upgrading.md#USER_UpgradeDBInstance.Upgrading.Manual).
In upgrading a version 9.6 database to version 10, be aware that PostgreSQL 10 enables parallel queries by default. You can test the impact of parallelism *before* the upgrade by changing the `max_parallel_workers_per_gather` parameter on your test database to 2.
**Note**
The default value for `max_parallel_workers_per_gather` parameter in the `default.postgresql10` DB parameter group is 2.
For more information, see [Parallel Query](https://www.postgresql.org/docs/10/parallel-query.html) in the PostgreSQL documentation. To disable parallelism on version 10, set the `max_parallel_workers_per_gather` parameter to 0.
During the major version upgrade, the `public` and `template1` databases and the `public` schema in every database are temporarily renamed. These objects appear in the logs with their original name and a random string appended. The string is appended so that custom settings such as `locale` and `owner` are preserved during the major version upgrade. After the upgrade completes, the objects are renamed back to their original names.
**Note**
During the major version upgrade process, you can't do a point-in-time restore of your DB instance or Multi-AZ DB cluster. After Amazon RDS performs the upgrade, it takes an automatic backup of the database. You can perform a point-in-time restore to times before the upgrade began and after the automatic backup of your database has completed.
1. **If an upgrade fails with precheck procedure errors, resolve the issues** – During the major version upgrade process, Amazon RDS for PostgreSQL first runs a precheck procedure to identify any issues that might cause the upgrade to fail. The precheck procedure checks all potential incompatible conditions across all databases in the instance.
If the precheck encounters an issue, it creates a log event indicating the upgrade precheck failed. The precheck process details are in an upgrade log named `pg_upgrade_precheck.log` for all the databases of a database. Amazon RDS appends a timestamp to the file name. For more information about viewing logs, see [Monitoring Amazon RDS log files](USER_LogAccess.md).
If a read replica upgrade fails at precheck, replication on the failed read replica is broken and the read replica is put in the terminated state. Delete the read replica and recreate a new read replica based on the upgraded primary DB instance.
Resolve all of the issues identified in the precheck log and then retry the major version upgrade. The following is an example of a precheck log.
```
------------------------------------------------------------------------
Upgrade could not be run on Wed Apr 4 18:30:52 2018
-------------------------------------------------------------------------
The instance could not be upgraded from 9.6.11 to 10.6 for the following reasons.
Please take appropriate action on databases that have usage incompatible with the requested major engine version upgrade and try the upgrade again.
* There are uncommitted prepared transactions. Please commit or rollback all prepared transactions.* One or more role names start with 'pg_'. Rename all role names that start with 'pg_'.
* The following issues in the database 'my"million$"db' need to be corrected before upgrading:** The ["line","reg*"] data types are used in user tables. Remove all usage of these data types.
** The database name contains characters that are not supported by RDS for PostgreSQL. Rename the database.
** The database has extensions installed that are not supported on the target database version. Drop the following extensions from your database: ["tsearch2"].
* The following issues in the database 'mydb' need to be corrected before upgrading:** The database has views or materialized views that depend on 'pg_stat_activity'. Drop the views.
```
1. **If a read replica upgrade fails while upgrading the database, resolve the issue** – A failed read replica is placed in the `incompatible-restore` state and replication is terminated on the database. Delete the read replica and recreate a new read replica based on the upgraded primary DB instance.
**Note**
Amazon RDS doesn't upgrade read replicas for Multi-AZ DB clusters. If you perform a major version upgrade on a Multi-AZ DB cluster, then the replication state of its read replicas changes to **terminated**.
A read replica upgrade might fail for the following reasons:
+ It was unable to catch up with the primary DB instance even after a wait time.
+ It was in a terminal or incompatible lifecycle state such as storage-full, incompatible-restore, and so on.
+ When the primary DB instance upgrade started, there was a separate minor version upgrade running on the read replica.
+ The read replica used incompatible parameters.
+ The read replica was unable to communicate with the primary DB instance to synchronize the data folder.
1. **Upgrade your production database** – When the dry-run major version upgrade is successful, you should be able to upgrade your production database with confidence. For more information, see [Manually upgrading the engine version](USER_UpgradeDBInstance.Upgrading.md#USER_UpgradeDBInstance.Upgrading.Manual).
1. Run the `ANALYZE` operation to refresh the `pg_statistic` table. You should do this for every database on all your PostgreSQL databases. Optimizer statistics aren't transferred during a major version upgrade, so you need to regenerate all statistics to avoid performance issues. Run the command without any parameters to generate statistics for all regular tables in the current database, as follows:
```
ANALYZE VERBOSE;
```
The `VERBOSE` flag is optional, but using it shows you the progress. For more information, see [ANALYZE](https://www.postgresql.org/docs/10/sql-analyze.html) in the PostgreSQL documentation.
When analyzing specific tables instead of using ANALYZE VERBOSE, run the ANALYZE command for each table as follows:
```
ANALYZE table_name;
```
For partitioned tables, always analyze the parent table. This process:
+ Automatically samples rows across all partitions
+ Updates statistics for each partition recursively
+ Maintains essential planning statistics at the parent level
While parent tables store no actual data, analyzing them is vital for query optimization. Running ANALYZE only on individual partitions can lead to poor query performance since the optimizer won't have the comprehensive statistics needed for efficient cross-partition planning.
**Note**
Run ANALYZE on your system after the upgrade to avoid performance issues.
After the major version upgrade is complete, we recommend the following:
+ A PostgreSQL upgrade doesn't upgrade any PostgreSQL extensions. To upgrade extensions, see [Upgrading PostgreSQL extensions in RDS for PostgreSQL databases](USER_UpgradeDBInstance.PostgreSQL.ExtensionUpgrades.md).
+ Optionally, use Amazon RDS to view two logs that the `pg_upgrade` utility produces. These are `pg_upgrade_internal.log` and `pg_upgrade_server.log`. Amazon RDS appends a timestamp to the file name for these logs. You can view these logs as you can any other log. For more information, see [Monitoring Amazon RDS log files](USER_LogAccess.md).
You can also upload the upgrade logs to Amazon CloudWatch Logs. For more information, see [Publishing PostgreSQL logs to Amazon CloudWatch Logs](USER_LogAccess.Concepts.PostgreSQL.md#USER_LogAccess.Concepts.PostgreSQL.PublishtoCloudWatchLogs).
+ To verify that everything works as expected, test your application on the upgraded database with a similar workload. After the upgrade is verified, you can delete this test instance.
# Automatic minor version upgrades for RDS for PostgreSQL
If you enable the **Auto minor version upgrade** option when creating or modifying a DB instance or Multi-AZ DB cluster, you can have your database automatically upgraded.
Amazon RDS also supports upgrade rollout policy to manage automatic minor version upgrades across multiple database resources and AWS accounts. For more information, see [Using AWS Organizations upgrade rollout policy for automatic minor version upgrades](RDS.Maintenance.AMVU.UpgradeRollout.md).
For each RDS for PostgreSQL major version, one minor version is designated by RDS as the automatic upgrade version. After a minor version has been tested and approved by Amazon RDS, the minor version upgrade occurs automatically during your maintenance window. RDS doesn't automatically set newer released minor versions as the automatic upgrade version. Before RDS designates a newer automatic upgrade version, several criteria are considered, such as the following:
+ Known security issues
+ Bugs in the PostgreSQL community version
+ Overall fleet stability since the minor version was released
You can use the following AWS CLI command to determine the current automatic minor upgrade target version for a specified PostgreSQL minor version in a specific AWS Region.
For Linux, macOS, or Unix:
```
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version minor-version \
--region region \
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}" \
--output text
```
For Windows:
```
aws rds describe-db-engine-versions ^
--engine postgres ^
--engine-version minor-version ^
--region region ^
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}" ^
--output text
```
For example, the following AWS CLI command determines the automatic minor upgrade target for PostgreSQL minor version 16.1 in the US East (Ohio) AWS Region (us-east-2).
For Linux, macOS, or Unix:
```
aws rds describe-db-engine-versions \
--engine postgres \
--engine-version 16.1 \
--region us-east-2 \
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}" \
--output table
```
For Windows:
```
aws rds describe-db-engine-versions ^
--engine postgres ^
--engine-version 16.1 ^
--region us-east-2 ^
--query "DBEngineVersions[*].ValidUpgradeTarget[*].{AutoUpgrade:AutoUpgrade,EngineVersion:EngineVersion}" ^
--output table
```
Your output is similar to the following.
```
----------------------------------
| DescribeDBEngineVersions |
+--------------+-----------------+
| AutoUpgrade | EngineVersion |
+--------------+-----------------+
| False | 16.2 |
| True | 16.3 |
| False | 16.4 |
| False | 16.5 |
| False | 16.6 |
| False | 17.1 |
| False | 17.2 |
+--------------+-----------------+
```
In this example, the `AutoUpgrade` value is `True` for PostgreSQL version 16.3. So, the automatic minor upgrade target is PostgreSQL version 16.3, which is highlighted in the output.
A PostgreSQL database is automatically upgraded during your maintenance window if the following criteria are met:
+ The database has the **Auto minor version upgrade** option enabled.
+ The database is running a minor DB engine version that is less than the current automatic upgrade minor version.
For more information, see [Automatically upgrading the minor engine version](USER_UpgradeDBInstance.Upgrading.md#USER_UpgradeDBInstance.Upgrading.AutoMinorVersionUpgrades).
**Note**
A PostgreSQL upgrade doesn't upgrade PostgreSQL extensions. To upgrade extensions, see [Upgrading PostgreSQL extensions in RDS for PostgreSQL databases](USER_UpgradeDBInstance.PostgreSQL.ExtensionUpgrades.md).
# Upgrading PostgreSQL extensions in RDS for PostgreSQL databases
A PostgreSQL engine upgrade doesn't upgrade most PostgreSQL extensions. To update an extension after a version upgrade, use the `ALTER EXTENSION UPDATE` command.
**Note**
For information about updating the PostGIS extension, see [Managing spatial data with the PostGIS extension](Appendix.PostgreSQL.CommonDBATasks.PostGIS.md) ([Step 6: Upgrade the PostGIS extension](Appendix.PostgreSQL.CommonDBATasks.PostGIS.md#Appendix.PostgreSQL.CommonDBATasks.PostGIS.Update)).
To update the `pg_repack` extension, drop the extension and then create the new version in the upgraded database. For more information, see [pg\$1repack installation](https://reorg.github.io/pg_repack/) in the `pg_repack` documentation.
To upgrade an extension, use the following command.
```
ALTER EXTENSION extension_name UPDATE TO 'new_version';
```
For the list of supported versions of PostgreSQL extensions, see [Supported PostgreSQL extension versions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md).
To list your currently installed extensions, use the PostgreSQL [pg\$1extension](https://www.postgresql.org/docs/current/catalog-pg-extension.html) catalog in the following command.
```
SELECT * FROM pg_extension;
```
To view a list of the specific extension versions that are available for your installation, use the PostgreSQL [ pg\$1available\$1extension\$1versions](https://www.postgresql.org/docs/current/view-pg-available-extension-versions.html) view in the following command.
```
SELECT * FROM pg_available_extension_versions;
```
# Monitoring RDS for PostgreSQL engine upgrades with events
When you upgrade the engine version of a RDS for PostgreSQL database, Amazon RDS emits a specific event during each phase of the process. To track the progress of an upgrade, you can view or subscribe to these events.
For more information about RDS events, see [Monitoring Amazon RDS events](working-with-events.md).
For detailed information about a specific Amazon RDS event that occurs during your engine upgrade, see [Amazon RDS event categories and event messages](USER_Events.Messages.md).
# Upgrading a PostgreSQL DB snapshot engine version
With Amazon RDS, you can create a storage volume DB snapshot of your PostgreSQL DB instance. When you create a DB snapshot, the snapshot is based on the engine version used by your Amazon RDS instance. You can upgrade the engine version for your DB snapshots.
After restoring a DB snapshot upgraded to a new engine version, make sure to test that the upgrade was successful. For more information about a major version upgrade, see [Upgrades of the RDS for PostgreSQL DB engine](USER_UpgradeDBInstance.PostgreSQL.md). To learn how to restore a DB snapshot, see [Restoring to a DB instance](USER_RestoreFromSnapshot.md).
You can upgrade manual DB snapshots that are either encrypted or not encrypted.
To view the available engine versions for your RDS for PostgreSQL DB snapshot, use the following AWS CLI example.
```
aws rds describe-db-engine-versions --engine postgres --engine-version example-engine-version --query "DBEngineVersions[*].ValidUpgradeTarget[*].{EngineVersion:EngineVersion}" --output text --include-all
```
For more information about available engine versions for RDS for PostgreSQL DB snapshots, see [Choosing a major version for an RDS for PostgreSQL upgrade](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.md).
**Note**
You can't upgrade automated DB snapshots that are created during the automated backup process.
## Console
**To upgrade a DB snapshot**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose **Snapshots**.
1. Choose the snapshot that you want to upgrade.
1. For **Actions**, choose **Upgrade snapshot**. The **Upgrade snapshot** page appears.
1. Choose the **New engine version** to upgrade to.
1. Choose **Save changes** to upgrade the snapshot.
During the upgrade process, all snapshot actions are disabled for this DB snapshot. Also, the DB snapshot status changes from **available** to **upgrading**, and then changes to **active** upon completion. If the DB snapshot can't be upgraded because of snapshot corruption issues, the status changes to **unavailable**. You can't recover the snapshot from this state.
**Note**
If the DB snapshot upgrade fails, the snapshot is rolled back to the original state with the original version.
## AWS CLI
To upgrade a DB snapshot to a new database engine version, use the AWS CLI [modify-db-snapshot](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-snapshot.html) command.
**Parameters**
+ `--db-snapshot-identifier` – The identifier of the DB snapshot to upgrade. The identifier must be a unique Amazon Resource Name (ARN). For more information, see [Amazon Resource Names (ARNs) in Amazon RDS](USER_Tagging.ARN.md).
+ `--engine-version` – The engine version to upgrade the DB snapshot to.
**Example**
For Linux, macOS, or Unix:
```
1. aws rds modify-db-snapshot \
2. --db-snapshot-identifier my_db_snapshot \
3. --engine-version new_version
```
For Windows:
```
1. aws rds modify-db-snapshot ^
2. --db-snapshot-identifier my_db_snapshot ^
3. --engine-version new_version
```
## RDS API
To upgrade a DB snapshot to a new database engine version, call the Amazon RDS API [ ModifyDBSnapshot](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ModifyDBSnapshot.html) operation.
+ `DBSnapshotIdentifier` – The identifier of the DB snapshot to upgrade. The identifier must be a unique Amazon Resource Name (ARN). For more information, see [Amazon Resource Names (ARNs) in Amazon RDS](USER_Tagging.ARN.md).
+ `EngineVersion` – The engine version to upgrade the DB snapshot to.
# Working with read replicas for Amazon RDS for PostgreSQL
You can scale reads for your Amazon RDS for PostgreSQL DB instances by adding read replicas to the instances. As with other Amazon RDS database engines, RDS for PostgreSQL uses native replication mechanisms of PostgreSQL to keep read replicas up to date with changes on the source DB. For general information about read replicas and Amazon RDS, see [Working with DB instance read replicas](USER_ReadRepl.md).
Following, you can find information specific to working with read replicas with RDS for PostgreSQL.
## Read replica limitations with PostgreSQL
The following are limitations for PostgreSQL read replicas:
+ PostgreSQL read replicas are read-only. Although a read replica isn't a writeable DB instance, you can promote it to become a standalone RDS for PostgreSQL DB instance. However, the process isn't reversible.
+ You can't create a read replica from another read replica if your RDS for PostgreSQL DB instance is running a PostgreSQL version earlier than 14.1. RDS for PostgreSQL supports cascading read replicas on RDS for PostgreSQL version 14.1 and higher releases only. For more information, see [Using cascading read replicas with RDS for PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.Cascading.md).
+ If you promote a PostgreSQL read replica, it becomes a writable DB instance. It stops receiving write-ahead log (WAL) files from a source DB instance, and it's no longer a read-only instance. You can create new read replicas from the promoted DB instance as you do for any RDS for PostgreSQL DB instance. For more information, see [Promoting a read replica to be a standalone DB instance](USER_ReadRepl.Promote.md).
+ If you promote a PostgreSQL read replica from within a replication chain (a series of cascading read replicas), any existing downstream read replicas continue receiving WAL files from the promoted instance automatically. For more information, see [Using cascading read replicas with RDS for PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.Cascading.md).
+ If no user transactions are running on the source DB instance, the associated PostgreSQL read replica reports a replication lag of up to five minutes. The replica lag is calculated as `currentTime - lastCommitedTransactionTimestamp`, which means that when no transactions are being processed, the value of replica lag increases for a period of time until the write-ahead log (WAL) segment switches. By default RDS for PostgreSQL switches the WAL segment every 5 minutes, which results in a transaction record and a decrease in the reported lag.
+ You can't turn on automated backups for PostgreSQL read replicas for RDS for PostgreSQL versions earlier than 14.1. Automated backups for read replicas are supported for RDS for PostgreSQL 14.1 and higher versions only. For RDS for PostgreSQL 13 and earlier versions, create a snapshot from a read replica if you want a backup of it.
+ Point-in-time recovery (PITR) isn't supported for read replicas. You can use PITR with a primary (writer) instance only, not a read replica. To learn more, see [Restoring a DB instance to a specified time for Amazon RDS](USER_PIT.md).
+ Read replicas for PostgreSQL versions 12 and lower automatically reboot during the 60-90 day maintenance window to apply password rotation. If the replica loses connection to the source before the scheduled reboot, it still reboots to resume replication. For PostgreSQL versions 13 and higher, read replicas might experience brief replication disconnections and reconnections during the password rotation process.
# Read replica configuration with PostgreSQL
RDS for PostgreSQL uses PostgreSQL native streaming replication to create a read-only copy of a source DB instance. This read replica DB instance is an asynchronously created physical replica of the source DB instance. It's created by a special connection that transmits write ahead log (WAL) data from the source DB instance to the read replica. For more information, see [Streaming Replication](https://www.postgresql.org/docs/14/warm-standby.html#STREAMING-REPLICATION) in the PostgreSQL documentation.
PostgreSQL asynchronously streams database changes to this secure connection as they're made on the source DB instance. You can encrypt communications from your client applications to the source DB instance or any read replicas by setting the `ssl` parameter to `1`. For more information, see [Using SSL with a PostgreSQL DB instance](PostgreSQL.Concepts.General.SSL.md) .
PostgreSQL uses a *replication* role to perform streaming replication. The role is privileged, but you can't use it to modify any data. PostgreSQL uses a single process for handling replication.
You can create a PostgreSQL read replica without affecting operations or users of the source DB instance. Amazon RDS sets the necessary parameters and permissions for you, on the source DB instance and the read replica, without affecting the service. A snapshot is taken of the source DB instance, and this snapshot is used to create the read replica. If you delete the read replica at some point in the future, no outage occurs.
You can create up to 15 read replicas from one source DB instance within the same Region. As of RDS for PostgreSQL 14.1, you can also create up to three levels of read replica in a chain (cascade) from a source DB instance. For more information, see [Using cascading read replicas with RDS for PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.Cascading.md). In all cases, the source DB instance needs to have automated backups configured. You do this by setting the backup retention period on your DB instance to any value other than 0. For more information, see [Creating a read replica](USER_ReadRepl.Create.md).
You can create read replicas for your RDS for PostgreSQL DB instance in the same AWS Region as your source DB instance. This is known as *in-Region* replication. You can also create read replicas in different AWS Regions than the source DB instance. This is known as *cross-Region* replication. For more information about setting up cross-Region read replicas, see [Creating a read replica in a different AWS Region](USER_ReadRepl.XRgn.md). The various mechanisms supporting the replication process for in-Region and cross-Region differ slightly depending on the RDS for PostgreSQL version as explained in [How streaming replication works for different RDS for PostgreSQL versions](USER_PostgreSQL.Replication.ReadReplicas.Mechanisms-versions.md).
For replication to operate effectively, each read replica should have the same amount of compute and storage resources as the source DB instance. If you scale the source DB instance, be sure to also scale the read replicas.
Amazon RDS overrides any incompatible parameters on a read replica if they prevent the read replica from starting. For example, suppose that the `max_connections` parameter value is higher on the source DB instance than on the read replica. In that case, Amazon RDS updates the parameter on the read replica to be the same value as that on the source DB instance.
RDS for PostgreSQL read replicas have access to external databases that are available through foreign data wrappers (FDWs) on the source DB instance. For example, suppose that your RDS for PostgreSQL DB instance is using the `mysql_fdw` wrapper to access data from RDS for MySQL. If so, your read replicas can also access that data. Other supported FDWs include `oracle_fdw`, `postgres_fdw`, and `tds_fdw`. For more information, see [Working with the supported foreign data wrappers for Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Extensions.foreign-data-wrappers.md).
## Using RDS for PostgreSQL read replicas with Multi-AZ configurations
You can create a read replica from a single-AZ or Multi-AZ DB instance. You can use Multi-AZ deployments to improve the durability and availability of critical data, with a standby replica. A *standby replica* is a dedicated read replica that can assume the workload if the source DB fails over. You can't use your standby replica to serve read traffic. However, you can create read replicas from high-traffic Multi-AZ DB instances to offload read-only queries. To learn more about Multi-AZ deployments, see [Multi-AZ DB instance deployments for Amazon RDS](Concepts.MultiAZSingleStandby.md).
If the source DB instance of a Multi-AZ deployment fails over to a standby, the associated read replicas switch to using the standby (now primary) as their replication source. The read replicas might need to restart, depending on the RDS for PostgreSQL version, as follows:
+ **PostgreSQL 13 and higher versions** – Restarting isn't required. The read replicas are automatically synchronized with the new primary. However, in some cases your client application might cache Domain Name Service (DNS) details for your read replicas. If so, set the time-to-live (TTL) value to less than 30 seconds. Doing this prevents the read replica from holding on to a stale IP address (and thus, prevents it from synchronizing with the new primary). To learn more about this and other best practices, see [Amazon RDS basic operational guidelines](CHAP_BestPractices.md#CHAP_BestPractices.DiskPerformance).
+ **PostgreSQL 12 and all earlier versions** – The read replicas restart automatically after a fail over to the standby replica because the standby (now primary) has a different IP address and a different instance name. Restarting synchronizes the read replica with the new primary.
To learn more about failover, see [Failing over a Multi-AZ DB instance for Amazon RDS](Concepts.MultiAZ.Failover.md). To learn more about how read replicas work in a Multi-AZ deployment, see [Working with DB instance read replicas](USER_ReadRepl.md).
To provide failover support for a read replica, you can create the read replica as a Multi-AZ DB instance so that Amazon RDS creates a standby of your replica in another Availability Zone (AZ). Creating your read replica as a Multi-AZ DB instance is independent of whether the source database is a Multi-AZ DB instance.
# Logical decoding on a read replica
RDS for PostgreSQL supports logical replication from standbys with PostgreSQL 16.1. This allows you to create logical decoding from a read-only standby that reduces the load on the primary DB instance. You can achieve higher-availability for your applications that need to synchronize data across multiple systems. This feature boosts the performance of your data warehouse and data analytics.
Also, replication slots on a given standby persist the promotion of that standby to a primary. This means that in the event of a primary DB instance failover or the promotion of a standby to be the new primary, the replication slots will persist and the former standby subscribers will not be affected.
**To create logical decoding on a read replica**
1. **Turn on logical replication** – To create logical decoding on a standby, you must turn on logical replication on your source DB instance and its physical replica. For more information, see [Read replica configuration with PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.Configuration.md).
+ **To turn on logical replication for a newly created RDS for PostgreSQL DB instance** – Create a new DB custom parameter group and set the static parameter `rds.logical_replication` to `1`. Then, associate this DB parameter group with the Source DB instance and its physical read replica. For more information, see [Associating a DB parameter group with a DB instance in Amazon RDS](USER_WorkingWithParamGroups.Associating.md).
+ **To turn on logical replication for an existing RDS for PostgreSQL DB instance** – Modify the DB custom parameter group of the source DB instance and its physical read replica to set the static parameter `rds.logical_replication` to `1`. For more information, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md).
**Note**
You must reboot the DB instance to apply these parameter changes.
You can use the following query to verify the values for `wal_level` and `rds.logical_replication` on the source DB instance and its physical read replica.
```
Postgres=>SELECT name,setting FROM pg_settings WHERE name IN ('wal_level','rds.logical_replication');
name | setting
-------------------------+---------
rds.logical_replication | on
wal_level | logical
(2 rows)
```
1. **Create a table in the source database** – Connect to the database in your source DB instance. For more information, see [Connecting to a DB instance running the PostgreSQL database engine](USER_ConnectToPostgreSQLInstance.md).
Use the following queries to create table in your source database and to insert values:
```
Postgres=>CREATE TABLE LR_test (a int PRIMARY KEY);
CREATE TABLE
```
```
Postgres=>INSERT INTO LR_test VALUES (generate_series(1,10000));
INSERT 0 10000
```
1. **Create a publication for the source table** – Use the following query to create a publication for the table on the source DB instance.
```
Postgres=>CREATE PUBLICATION testpub FOR TABLE LR_test;
CREATE PUBLICATION
```
Use a SELECT query to verify the details of the publication that was created on both the source DB instance and the physical read replica instance.
```
Postgres=>SELECT * from pg_publication;
oid | pubname | pubowner | puballtables | pubinsert | pubupdate | pubdelete | pubtruncate | pubviaroot
-------+---------+----------+--------------+-----------+-----------+-----------+-------------+------------
16429 | testpub | 16413 | f | t | t | t | t | f
(1 row)
```
1. **Create a subscription from logical replica instance** – Create another RDS for PostgreSQL DB instance as the logical replica instance. Make sure that VPC is setup correctly to ensure that this logical replica instance can access the physical read replica instance. For more information, see [Amazon VPC and Amazon RDS](USER_VPC.md). If your source DB instance is idle, connectivity issues might occur and the primary doesn't send the data to standby.
```
Postgres=>CREATE SUBSCRIPTION testsub CONNECTION 'host=Physical replica host name port=port
dbname=source_db_name user=user password=password' PUBLICATION testpub;
NOTICE: created replication slot "testsub" on publisher
CREATE SUBSCRIPTION
```
```
Postgres=>CREATE TABLE LR_test (a int PRIMARY KEY);
CREATE TABLE
```
Use a SELECT query to verify the details of the subscription on the logical replica instance.
```
Postgres=>SELECT oid,subname,subenabled,subslotname,subpublications FROM pg_subscription;
oid | subname | subenabled | subslotname | subpublications
-------+---------+------------+-------------+-----------------
16429 | testsub | t | testsub | {testpub}
(1 row)
postgres=> select count(*) from LR_test;
count
-------
10000
(1 row)
```
1. **Inspect logical replication slot state** – You can only see the physical replication slot on your source DB instance.
```
Postgres=>select slot_name, slot_type, confirmed_flush_lsn from pg_replication_slots;
slot_name | slot_type | confirmed_flush_lsn
---------------------------------------------+-----------+---------------------
rds_us_west_2_db_dhqfsmo5wbbjqrn3m6b6ivdhu4 | physical |
(1 row)
```
However, on your read replica instance, you can see the logical replication slot and the `confirmed_flush_lsn` value changes as the application actively consumes logical changes.
```
Postgres=>select slot_name, slot_type, confirmed_flush_lsn from pg_replication_slots;
slot_name | slot_type | confirmed_flush_lsn
-----------+-----------+---------------------
testsub | logical | 0/500002F0
(1 row)
```
```
Postgres=>select slot_name, slot_type, confirmed_flush_lsn from pg_replication_slots;
slot_name | slot_type | confirmed_flush_lsn
-----------+-----------+---------------------
testsub | logical | 0/5413F5C0
(1 row)
```
# Using cascading read replicas with RDS for PostgreSQL
As of version 14.1, RDS for PostgreSQL supports cascading read replicas. With *cascading read replicas*, you can scale reads without adding overhead to your source RDS for PostgreSQL DB instance. Updates to the WAL log aren't sent by the source DB instance to each read replica. Instead, each read replica in a cascading series sends WAL log updates to the next read replica in the series. This reduces the burden on the source DB instance.
With cascading read replicas, your RDS for PostgreSQL DB instance sends WAL data to the first read replica in the chain. That read replica then sends WAL data to the second replica in the chain, and so on. The end result is that all read replicas in the chain have the changes from the RDS for PostgreSQL DB instance, but without the overhead solely on the source DB instance.
You can create a series of up to three read replicas in a chain from a source RDS for PostgreSQL DB instance. For example, suppose that you have an RDS for PostgreSQL 14.1 DB instance, `rpg-db-main`. You can do the following:
+ Starting with `rpg-db-main`, create the first read replica in the chain, `read-replica-1`.
+ Next, from `read-replica-1`, create the next read replica in the chain, `read-replica-2`.
+ Finally, from `read-replica-2`, create the third read replica in the chain, `read-replica-3`.
You can't create another read replica beyond this third cascading read replica in the series for `rpg-db-main`. A complete series of instances from an RDS for PostgreSQL source DB instance through to the end of a series of cascading read replicas can consist of at most four DB instances.
For cascading read replicas to work, turn on automatic backups on your RDS for PostgreSQL. Create the read replica first and then turn on automatic backups on the RDS for PostgreSQL DB instance. The process is the same as for other Amazon RDS DB engines. For more information, see [Creating a read replica](USER_ReadRepl.Create.md).
As with any read replica, you can promote a read replica that's part of a cascade. Promoting a read replica from within a chain of read replicas removes that replica from the chain. For example, suppose that you want to move some of the workload off of your `rpg-db-main` DB instance to a new instance for use by the accounting department only. Assuming the chain of three read replicas from the example, you decide to promote `read-replica-2`. The chain is affected as follows:
+ Promoting `read-replica-2` removes it from the replication chain.
+ It is now a full read/write DB instance.
+ It continues replicating to `read-replica-3`, just as it was doing before promotion.
+ Your `rpg-db-main` continues replicating to `read-replica-1`.
For more information about promoting read replicas, see [Promoting a read replica to be a standalone DB instance](USER_ReadRepl.Promote.md).
**Note**
RDS for PostgreSQL doesn't support major version upgrades for cascading replicas. Before performing a major version upgrade, you need to remove cascading replicas. You can recreate them after completing the upgrade on your source DB instance and first-level replicas.
For cascading read replicas, RDS for PostgreSQL supports 15 read replicas for each source DB instance at first level of replication, and 5 read replicas for each source DB instance at the second and third level of replication.
# Creating cross-Region cascading read replicas with RDS for PostgreSQL
RDS for PostgreSQL supports cross-Region cascading read replicas. You can create a cross-Region replica from the source DB instance, and then create same-Region replicas from it. You can also create a same-Region replica from the source DB instance, and then create cross-Region replicas from it.
**Create a cross-Region replica and then create same-Region replicas**
You can use an RDS for PostgreSQL DB instance with version 14.1 or higher, `rpg-db-main`, to do the following:
1. Start with `rpg-db-main` (US-EAST-1), create the first cross-Region read replica in the chain, `read-replica-1` (US-WEST-2).
1. Using the first cross-Region `read-replica-1` (US-WEST-2), create the second read replica in the chain, `read-replica-2` (US-WEST-2).
1. Using `read-replica-2`, create the third read replica in the chain, `read-replica-3` (US-WEST-2).
**Create a same-Region replica and then create cross-Region replicas**
You can use an RDS for PostgreSQL DB instance with version 14.1 or higher, `rpg-db-main`, to do the following:
1. Starting with `rpg-db-main` (US-EAST-1), create the first read replica in the chain, `read-replica-1` (US-EAST-1).
1. Using `read-replica-1` (US-EAST-1), create the first cross-Region read replica in the chain, `read-replica-2` (US-WEST-2).
1. Using `read-replica-2` (US-WEST-2), create the third read replica in the chain, `read-replica-3` (US-WEST-2).
**Limitations in creating cross-Region read replicas**
+ A cross-Region cascading chain of database replicas can span a maximum of two Regions, with a maximum of four levels. The four levels include the database source and three read replicas.
**Advantages of using cascading read replicas**
+ Improved read scalability – By distributing read queries across multiple replicas, cascading replication helps balance the load. This improves performance, especially in read-heavy applications, by reducing the strain on the writer database.
+ Geographical distribution – Cascading replicas can be located in different geographic locations. This reduces latency for users located far from the primary database and provides a local read replica, enhancing performance and user experience.
+ High availability and disaster recovery – In the event of a primary server failure, replicas can be promoted to primary, ensuring continuity. cascading replication further enhances this by providing multiple layers of failover options, improving the overall resilience of the system.
+ Flexibility and modular growth – As the system grows, new replicas can be added at different levels without major reconfiguration of the primary database. This modular approach allows for scalable and manageable growth of the replication setup.
**Best practice for using cross-Region read replicas**
+ Before promoting a replica, create additional replicas. This will save time, and provide efficient handling of the workload.
# How streaming replication works for different RDS for PostgreSQL versions
As discussed in [Read replica configuration with PostgreSQL](USER_PostgreSQL.Replication.ReadReplicas.Configuration.md), RDS for PostgreSQL uses PostgreSQL's native streaming replication protocol to send WAL data from the source DB instance. It sends source WAL data to read replicas for both in-Region and cross-Region read replicas. With version 9.4, PostgreSQL introduced physical replication slots as a supporting mechanism for the replication process.
A *physical replication slot* prevents a source DB instance from removing WAL data before it's consumed by all read replicas. Each read replica has its own physical slot on the source DB instance. The slot keeps track of the oldest WAL (by logical sequence number, LSN) that might be needed by the replica. After all slots and DB connections have progressed beyond a given WAL (LSN), that LSN becomes a candidate for removal at the next checkpoint.
Amazon RDS uses Amazon S3 to archive WAL data. For in-Region read replicas, you can use this archived data to recover the read replica when necessary. An example of when you might do so is if the connection between source DB and read replica is interrupted for any reason.
In the following table, you can find a summary of differences between PostgreSQL versions and the supporting mechanisms for in-Region and cross-Region used by RDS for PostgreSQL.
| Version | In-Region | Cross-Region |
| --- | --- | --- |
| PostgreSQL 14.1 and higher versions | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.Replication.ReadReplicas.Mechanisms-versions.html) | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.Replication.ReadReplicas.Mechanisms-versions.html) |
| PostgreSQL 13 and lower versions | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.Replication.ReadReplicas.Mechanisms-versions.html) | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.Replication.ReadReplicas.Mechanisms-versions.html) |
For more information, see [Monitoring and tuning the replication process](USER_PostgreSQL.Replication.ReadReplicas.Monitor.md).
## Understanding the parameters that control PostgreSQL replication
The following parameters affect the replication process and determine how well read replicas stay up to date with the source DB instance:
**max\$1wal\$1senders**
The `max_wal_senders` parameter specifies the maximum number of connections that the source DB instance can support at the same time over the streaming replication protocol.
The default value varies for RDS for PostgreSQL versions:
+ For versions 13, 14, and 15, the default value is 20.
+ For versions 16 and above, the default value is 35.
This parameter should be set to slightly higher than the actual number of read replicas. If this parameter is set too low for the number of read replicas, replication stops.
For more information, see [max\$1wal\$1senders](https://www.postgresql.org/docs/devel/runtime-config-replication.html#GUC-MAX-WAL-SENDERS) in the PostgreSQL documentation.
`max_wal_senders` is a static parameter that requires a DB instance reboot for changes to take effect.
**wal\$1keep\$1segments**
The `wal_keep_segments` parameter specifies the number of write-ahead log (WAL) files that the source DB instance keeps in the `pg_wal` directory. The default setting is 32.
If `wal_keep_segments` isn't set to a large enough value for your deployment, a read replica can fall so far behind that streaming replication stops. If that happens, Amazon RDS generates a replication error and begins recovery on the read replica. It does so by replaying the source DB instance's archived WAL data from Amazon S3. This recovery process continues until the read replica has caught up enough to continue streaming replication. You can see this process in action as captured by the PostgreSQL log in [Example: How a read replica recovers from replication interruptionsExample: Read replica recovery from replication interruptions](#USER_PostgreSQL.Replication.example-how-it-works).
In PostgreSQL version 13, the `wal_keep_segments` parameter is named `wal_keep_size`. It serves the same purpose as `wal_keep_segments`, but its default value is in megabytes (MB) (2048 MB) rather than the number of files. For more information, see [wal\$1keep\$1segments](https://www.postgresql.org/docs/12/runtime-config-replication.html#GUC-WAL-KEEP-SEGMENTS) and [wal\$1keep\$1size](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-WAL-KEEP-SIZE) in the PostgreSQL documentation.
**max\$1slot\$1wal\$1keep\$1size**
The `max_slot_wal_keep_size` parameter controls the quantity of WAL data that the RDS for PostgreSQL DB instance retains in the `pg_wal` directory to serve slots. This parameter is used for configurations that use replication slots. The default value for this parameter is `-1`, meaning that there's no limit to how much WAL data is kept on the source DB instance. For information about monitoring your replication slots, see [Monitoring replication slots for your RDS for PostgreSQL DB instance](USER_PostgreSQL.Replication.ReadReplicas.Monitor.md#USER_PostgreSQL.Replication.ReadReplicas.Monitor-monitor-replication-slots).
For more information about this parameter, see [max\$1slot\$1wal\$1keep\$1size](https://www.postgresql.org/docs/devel/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE) in the PostgreSQL documentation.
Whenever the stream that provides WAL data to a read replica is interrupted, PostgreSQL switches into recovery mode. It restores the read replica by using archived WAL data from Amazon S3 or by using the WAL data associated with the replication slot. When this process is complete, PostgreSQL re-establishes streaming replication.
### Example: How a read replica recovers from replication interruptions
In the following example, you find the log details that demonstrate the recovery process for a read replica. The example is from an RDS for PostgreSQL DB instance running PostgreSQL version 12.9 in the same AWS Region as the source DB, so replication slots aren't used. The recovery process is the same for other RDS for PostgreSQL DB instances running PostgreSQL earlier than version 14.1 with in-Region read replicas.
When the read replica lost contact with the source DB instance, Amazon RDS records the issue in the log as `FATAL: could not receive data from WAL stream` message, along with the `ERROR: requested WAL segment ... has already been removed`. As shown in the bold line, Amazon RDS recovers the replica by replaying an archived WAL file.
```
2014-11-07 19:01:10 UTC::@:[23180]:DEBUG: switched WAL source from archive to stream after failure
2014-11-07 19:01:10 UTC::@:[11575]:LOG: started streaming WAL from primary at 1A/D3000000 on timeline 1
2014-11-07 19:01:10 UTC::@:[11575]:FATAL: could not receive data from WAL stream:
ERROR: requested WAL segment 000000010000001A000000D3 has already been removed
2014-11-07 19:01:10 UTC::@:[23180]:DEBUG: could not restore file "00000002.history" from archive: return code 0
2014-11-07 19:01:15 UTC::@:[23180]:DEBUG: switched WAL source from stream to archive after failure recovering 000000010000001A000000D3
2014-11-07 19:01:16 UTC::@:[23180]:LOG: restored log file "000000010000001A000000D3" from archive
```
When Amazon RDS replays enough archived WAL data on the replica to catch up, streaming to the read replica begins again. When streaming resumes, Amazon RDS writes an entry to the log file similar to the following.
```
2014-11-07 19:41:36 UTC::@:[24714]:LOG:started streaming WAL from primary at 1B/B6000000 on timeline 1
```
## Setting the parameters that control shared memory
The parameters you set determine the size of shared memory for tracking transaction IDs, locks, and prepared transactions. **The shared memory structure of a standby instance must be equal or greater than that of a primary instance.** This ensures that the former doesn't run out of shared memory during recovery. If the parameter values on the replica are less than the parameter values on the primary, Amazon RDS will automatically adjust the replica parameters and restart the engine.
The parameters affected are:
+ max\$1connections
+ max\$1worker\$1processes
+ max\$1wal\$1senders
+ max\$1prepared\$1transactions
+ max\$1locks\$1per\$1transaction
To avoid RDS reboots of replicas due to insufficient memory, we recommend applying the parameter changes as a rolling reboot to each replica. You must apply the following rules, when you set the parameters:
+ **Increasing the parameter values:**
+ You should always increase the parameter values of all the read replicas first, and perform a rolling reboot of all replicas. Then, apply the parameter changes on the primary instance and reboot.
+ **Decreasing the parameter values:**
+ You should first decrease the parameter values of the primary instance and perform a reboot. Then, apply the parameter changes to all the associated read replicas and perform a rolling reboot.
# Monitoring and tuning the replication process
We strongly recommend that you routinely monitor your RDS for PostgreSQL DB instance and read replicas. You need to ensure that your read replicas are keeping up with changes on the source DB instance. Amazon RDS transparently recovers your read replicas when interruptions to the replication process occur. However, it's best to avoid needing to recover at all. Recovering using replication slots is faster than using the Amazon S3 archive, but any recovery process can affect read performance.
To determine how well your read replicas are keeping up with the source DB instance, you can do the following:
+ **Check the amount of `ReplicaLag` between source DB instance and replicas.** *Replica lag* is the amount of time, in seconds, that a read replica lags behind its source DB instance. This metric reports the result of the following query.
```
SELECT extract(epoch from now() - pg_last_xact_replay_timestamp()) AS "ReplicaLag";
```
Replica lag is an indication of how well a read replica is keeping up with the source DB instance. It's the amount of latency between the source DB instance and a specific read instance. A high value for replica lag can indicate a mismatch between the DB instance classes or storage types (or both) used by the source DB instance and its read replicas. The DB instance class and storage types for DB source instance and all read replicas should be the same.
Replica lag can also be the result of intermittent connection issues. You can monitor replication lag in Amazon CloudWatch by viewing the Amazon RDS `ReplicaLag` metric. To learn more about `ReplicaLag` and other metrics for Amazon RDS, see [Amazon CloudWatch metrics for Amazon RDS](rds-metrics.md).
+ **Check the PostgreSQL log for information you can use to adjust your settings.** At every checkpoint, the PostgreSQL log captures the number of recycled transaction log files, as shown in the following example.
```
2014-11-07 19:59:35 UTC::@:[26820]:LOG: checkpoint complete: wrote 376 buffers (0.2%);
0 transaction log file(s) added, 0 removed, 1 recycled; write=35.681 s, sync=0.013 s, total=35.703 s;
sync files=10, longest=0.013 s, average=0.001 s
```
You can use this information to figure out how many transaction files are being recycled in a given time period. You can then change the setting for `wal_keep_segments` if necessary. For example, suppose that the PostgreSQL log at `checkpoint complete` shows `35 recycled` for a 5-minute interval. In this case, the `wal_keep_segments` default value of 32 isn't sufficient to keep pace with the streaming activity, so you should increase the value of this parameter.
+ **Use Amazon CloudWatch to monitor metrics that can predict replication issues.** Rather than analyzing the PostgreSQL log directly, you can use Amazon CloudWatch to check metrics that have been collected. For example, you can check the value of the `TransactionLogsGeneration` metric to see how much WAL data is being generated by the source DB instance. In some cases, the workload on your DB instance might generate a large amount of WAL data. If so, you might need to change the DB instance class for your source DB instance and read replicas. Using an instance class with high (10 Gbps) network performance can reduce replica lag.
## Monitoring replication slots for your RDS for PostgreSQL DB instance
All versions of RDS for PostgreSQL use replication slots for cross-Region read replicas. RDS for PostgreSQL 14.1 and higher versions use replication slots for in-Region read replicas. In-region read replicas also use Amazon S3 to archive WAL data. In other words, if your DB instance and read replicas are running PostgreSQL 14.1 or higher, replication slots and Amazon S3 archives are both available for recovering the read replica. Recovering a read replica using its replication slot is faster than recovering from Amazon S3 archive. So, we recommend that you monitor the replication slots and related metrics.
You can view the replication slots on your RDS for PostgreSQL DB instances by querying the `pg_replication_slots` view, as follows.
```
postgres=> SELECT * FROM pg_replication_slots;
slot_name | plugin | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn | wal_status | safe_wal_size | two_phase
---------------------------+--------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------+------------+---------------+-----------
rds_us_west_1_db_555555555 | | physical | | | f | t | 13194 | | | 23/D8000060 | | reserved | | f
(1 row)
```
The `wal_status` of `reserved` value means that the amount of WAL data held by the slot is within the bounds of the `max_wal_size` parameter. In other words, the replication slot is properly sized. Other possible status values are as follows:
+ `extended` – The slot exceeds the `max_wal_size` setting, but the WAL data is retained.
+ `unreserved` – The slot no longer has the all required WAL data. Some of it will be removed at the next checkpoint.
+ `lost` – Some required WAL data has been removed. The slot is no longer usable.
The `unreserved` and `lost` states of the `wal_status` are seen only when `max_slot_wal_keep_size` is non-negative.
The `pg_replication_slots` view shows you the current state of your replication slots. To assess the performance of your replication slots, you can use Amazon CloudWatch and monitor the following metrics:
+ **`OldestReplicationSlotLag`** – Shows the amount of Write-Ahead Log (WAL) data on the source that hasn't been consumed by the most lagging replica.
+ **`TransactionLogsDiskUsage`** – Shows how much storage is being used for WAL data. When a read replica lags significantly, the value of this metric can increase substantially.
To learn more about using Amazon CloudWatch and its metrics for RDS for PostgreSQL, see [Monitoring Amazon RDS metrics with Amazon CloudWatch](monitoring-cloudwatch.md). For more information about monitoring streaming replication on your RDS for PostgreSQL DB instances, see [Best practices for Amazon RDS PostgreSQL replication](https://aws.amazon.com/blogs/database/best-practices-for-amazon-rds-postgresql-replication/) on the *AWS Database Blog*.
# Configuring delayed replication with RDS for PostgreSQL
## Overview and Benefits
The delayed replication feature in RDS for PostgreSQL allows you to intentionally delay the replication of data changes from your primary database to one or more standby (read replica) servers. This provides valuable protection against data corruption, accidental data loss, or erroneous transactions that could otherwise be immediately propagated to all replicas.
Delayed replication is supported in the following RDS for PostgreSQL versions:
+ 14.19 and higher 14 versions
+ 15.14 and higher 15 versions
+ 16.10 and higher 16 versions
+ 17.6 and higher 17 versions
By introducing a time lag in the replication process, you gain a window of opportunity to detect and respond to data-related incidents before they affect your entire DB cluster. Key benefits of delayed replication include the following:
+ Allows you to recover from accidental deletions, updates, or other logical mistakes.
+ Provides a buffer against the spread of corrupted data across your DB cluster.
+ Offers an additional recovery point option to complement your traditional backup strategies.
+ Allows you to configure the delay period based on your organization's specific needs and risk tolerance.
## Enabling and Configuring Delayed Replication
To enable delayed replication on an RDS for PostgreSQL read replica, follow these steps:
**Note**
For cascaded read replicas, use the same `recovery_min_apply_delay` parameter and steps described below.
**To enable delayed replication**
1. Create a new custom parameter group or modify an existing one. For more information, see [DB parameter groups for Amazon RDS DB instances](USER_WorkingWithDBInstanceParamGroups.md).
1. In the parameter group, configure the `recovery_min_apply_delay` parameter:
+ Set the value to the desired delay in milliseconds. For example, 3600000 for a 1-hour delay.
+ Allowed range: 0 to 86400000 ms (0 to 24 hours)
+ Default: 0
1. Apply the parameter group to the read replica instance you want to configure for delayed replication.
1. Reboot the read replica instance for the changes to take effect.
**Note**
The `recovery_min_apply_delay` parameter is dynamic. If you modify an existing parameter group that's already attached to the instance, the changes take effect immediately without requiring a reboot. However, when applying a new parameter group to the instance, you must reboot for the changes to take effect.
## Managing Delayed Replication Recovery
Delayed replication is particularly useful in scenarios where traditional point-in-time recovery methods may be insufficient or too time-consuming.
During the delayed replication period, you can use the following PostgreSQL functions to manage the recovery process:
+ `pg_wal_replay_pause()`: Request to pause the recovery process on the delayed replica.
+ `pg_wal_replay_resume()`: Restart the recovery process if it was previously paused.
+ `pg_is_wal_replay_paused()`: Check if the recovery process is currently paused.
+ `pg_get_wal_replay_pause_state()`: Get the current state of the recovery process (not paused, pause requested, or paused).
Users with the `rds_superuser` role have EXECUTE privileges on `pg_wal_replay_pause()` and `pg_wal_replay_resume()`. If other database users need access to these functions, you must grant them the `rds_superuser` role. For more information about the `rds_superuser` role, see [Understanding the rds\$1superuser role](Appendix.PostgreSQL.CommonDBATasks.Roles.rds_superuser.md).
Access to other functions like `pg_is_wal_replay_paused()` and `pg_get_wal_replay_pause_state()` doesn't require the `rds_superuser` role.
You can use the following recovery target parameters to precisely control the point in time to which the delayed replica is recovered. These parameters are static and require a database reboot to apply changes:
+ recovery\$1target
+ recovery\$1target\$1lsn
+ recovery\$1target\$1name
+ recovery\$1target\$1time
+ recovery\$1target\$1xid
+ recovery\$1target\$1inclusive
**Important**
You can specify only one recovery target parameter at a time. Configuring multiple recovery target parameters in the configuration file results in an error.
## Planning considerations
Consider the following when planning delayed replication with RDS for PostgreSQL:
+ During the automatic rotation of `rdsrepladmin` credentials (which occurs every 90 days), delayed read replicas may temporarily enter a `REPLICATION_ERROR` state. If the delayed replica has sufficient WAL logs to maintain the configured delay, it may pause WAL receiver process, causing WAL accumulation on the source. You should monitor the replication status on the replica and the storage consumption on the source to avoid hitting storage-full.
+ When delayed read replicas encounter system events (such as reboot or restart), they enter a `REPLICATION_ERROR` state where the WAL receiver process remains inactive until the configured delay period expires. This behavior can cause WAL accumulation on the source instance, potentially leading to storage exhaustion. Consider the following preventive measures:
+ Configure CloudWatch alarms to monitor storage utilization on source instances.
+ Enable storage auto-scaling to handle unexpected WAL growth.
+ Set the `max_slot_wal_keep_size` parameter on the source instance to limit WAL retention per replication slot.
+ Monitor replication lag and slot status regularly.
+ Longer delays increase WAL logs on replicas, consuming more storage. Monitor storage space using CloudWatch alarms, enable auto-scaling, or catch up replicas when needed.
+ When promoting a delayed read replica, the `recovery_min_apply_delay` parameter is not honored, and all pending WAL records are immediately applied.
+ The `recovery_min_apply_delay` parameter is independent on each level of a cascading replication setup. Setting a delay on a replica does not add to the delay of any cascaded replicas.
For more information, see the [RDS for PostgreSQL Read Replicas documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html) and the [RDS for PostgreSQL Disaster Recovery documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PostgreSQL.Disaster-Recovery.html).
## Understanding limitations
The delayed replication feature for Amazon RDS for PostgreSQL has the following limitations:
+ Blue/Green deployments have the following limitations when configuring delayed replication:
+ **Green source instance** — The `recovery_min_apply_delay parameter` is disregarded, even if configured in the parameter group. Any delay settings on the green source instance do not take effect.
+ **Green replica instance** — The `recovery_min_apply_delay parameter` is fully supported and applied to the PostgreSQL configuration file. Delay settings function as expected during the switchover workflow.
+ RDS Blue/Green deployments for major version upgrades
+ During major version upgrades, any delayed read replicas will be automatically terminated to allow the source instance to proceed with the upgrade process in order to ensure minimal downtime. After the source instance completed the upgrade, you must manually recreate the delayed replicas.
+ Delayed replication is not compatible with the following features.
+ RDS for PostgreSQL Logical Replication
+ RDS for PostgreSQL Multi-AZ Clusters (including both inbound and outbound replication)
+ Aurora PostgreSQL
# Troubleshooting for RDS for PostgreSQL read replica
Following, you can find troubleshooting ideas for some common RDS for PostgreSQL read replica issues.
**Terminate the query that causes the read replica lag**
Transactions either in active or idle in transaction state that are running for a long time in the database might interfere with the WAL replication process, thereby increasing the replication lag. Therefore, be sure to monitor the runtime of these transactions with the PostgreSQL `pg_stat_activity` view.
Run a query on the primary instance similar to the following to find the process ID (PID) of the query that's running for a long time:
```
SELECT datname, pid,usename, client_addr, backend_start,
xact_start, current_timestamp - xact_start AS xact_runtime, state,
backend_xmin FROM pg_stat_activity WHERE state='active';
```
```
SELECT now() - state_change as idle_in_transaction_duration, now() - xact_start as xact_duration,*
FROM pg_stat_activity
WHERE state = 'idle in transaction'
AND xact_start is not null
ORDER BY 1 DESC;
```
After identifying the PID of the query, you can choose to end the query.
Run a query on the primary instance similar to the following to terminate the query that's running for a long time:
```
SELECT pg_terminate_backend(PID);
```
# Improving query performance for RDS for PostgreSQL with Amazon RDS Optimized Reads
You can achieve faster query processing for RDS for PostgreSQL with Amazon RDS Optimized Reads. An RDS for PostgreSQL DB instance or Multi-AZ DB cluster that uses RDS Optimized Reads can achieve up to 50% faster query processing compared to one that doesn't use it.
**Topics**
+ [
## Overview of RDS Optimized Reads in PostgreSQL
](#USER_PostgreSQL.optimizedreads-overview)
+ [
## Use cases for RDS Optimized Reads
](#USER_PostgreSQL.optimizedreads-use-cases)
+ [
## Best practices for RDS Optimized Reads
](#USER_PostgreSQL.optimizedreads-best-practices)
+ [
## Using RDS Optimized Reads
](#USER_PostgreSQL.optimizedreads-using)
+ [
## Monitoring DB instances that use RDS Optimized Reads
](#USER_PostgreSQL.optimizedreads-monitoring)
+ [
## Limitations for RDS Optimized Reads in PostgreSQL
](#USER_PostgreSQL.optimizedreads-limitations)
## Overview of RDS Optimized Reads in PostgreSQL
Optimized Reads is available by default on RDS for PostgreSQL versions 15.2 and higher, 14.7 and higher, and 13.10 and higher when using NVMe-based DB instance classes. For hardware specifications that indicate which instances use NVMe, see [Hardware specifications for DB instance classes](Concepts.DBInstanceClass.Summary.md).
When you use an RDS for PostgreSQL DB instance or Multi-AZ DB cluster that has RDS Optimized Reads turned on, it achieves up to 50% faster query performance using the local Non-Volatile Memory Express (NVMe) based solid state drive (SSD) block-level storage. You can achieve faster query processing by placing the temporary tables that are generated by PostgreSQL on the local storage, which reduces the traffic to Elastic Block Storage (EBS) over the network.
In PostgreSQL, temporary objects are assigned to a temporary namespace that drops automatically at the end of the session. The temporary namespace while dropping removes any objects that are session-dependent, including schema-qualified objects, such as tables, functions, operators, or even extensions.
In RDS for PostgreSQL, the `temp_tablespaces` parameter is configured for this temporary work area where the temporary objects are stored.
The following queries return the name of the tablespace and its location.
```
postgres=> show temp_tablespaces;
temp_tablespaces
---------------------
rds_temp_tablespace
(1 row)
```
The `rds_temp_tablespace` is a tablespace configured by RDS that points to the NVMe local storage. You can always switch back to Amazon EBS storage by modifying this parameter in the `Parameter group` using the AWS Management Console to point to any tablespace other than `rds_temp_tablespace`. For more information, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md). You can also use the SET command to modify the value of the `temp_tablespaces` parameter to `pg_default` at the session level using SET command. Modifying the parameter redirects the temporary work area to Amazon EBS. Switching back to Amazon EBS helps when the local storage for your RDS instance or cluster isn't sufficient to perform a specific SQL operation.
```
postgres=> SET temp_tablespaces TO 'pg_default';
SET
```
```
postgres=> show temp_tablespaces;
temp_tablespaces
------------------
pg_default
```
## Use cases for RDS Optimized Reads
The following are some use cases that can benefit from Optimized Reads:
+ Analytical queries that include Common Table Expressions (CTEs), derived tables, and grouping operations.
+ Read replicas that handle the unoptimized queries for an application.
+ On-demand or dynamic reporting queries with complex operations such as GROUP BY and ORDER BY that can't always use appropriate indexes.
+ Other workloads that use internal temporary tables.
+ `CREATE INDEX` or `REINDEX` operations for sorting.
## Best practices for RDS Optimized Reads
Use the following best practices for RDS Optimized Reads:
+ Add retry logic for read-only queries in case they fail because the instance store is full during the execution.
+ Monitor the storage space available on the instance store with the CloudWatch metric `FreeLocalStorage`. If the instance store is reaching its limit because of the workload on the DB instance or Multi-AZ DB cluster, modify it to use a larger DB instance class.
## Using RDS Optimized Reads
When you provision an RDS for PostgreSQL DB instance with one of the NVMe based DB instance classes in a Single-AZ DB instance deployment, Multi-AZ DB instance deployment, or Multi-AZ DB cluster deployment, the DB instance automatically uses RDS Optimized Reads.
For more information about Multi-AZ deployment, see [Configuring and managing a Multi-AZ deployment for Amazon RDS](Concepts.MultiAZ.md).
To turn on RDS Optimized Reads, do one of the following:
+ Create an RDS for PostgreSQL DB instance or Multi-AZ DB cluster using one of the NVMe based DB instance classes. For more information, see [Creating an Amazon RDS DB instance](USER_CreateDBInstance.md).
+ Modify an existing RDS for PostgreSQL DB instance or Multi-AZ DB cluster to use one of the NVMe based DB instance classes. For more information, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md).
RDS Optimized Reads is available in all AWS Regions where one or more of the DB instance classes with local NVMe SSD storage are supported. For more information, see [DB instance classes](Concepts.DBInstanceClass.md).
To switch back to a non-optimized reads RDS instance, modify the DB instance class of your RDS instance or cluster to the similar instance class that only supports EBS storage for your database workloads. For example, if the current DB instance class is db.r6gd.4xlarge, choose db.r6g.4xlarge to switch back. For more information, see [Modifying an Amazon RDS DB instance](Overview.DBInstance.Modifying.md).
## Monitoring DB instances that use RDS Optimized Reads
You can monitor DB instances that use RDS Optimized Reads using the following CloudWatch metrics:
+ `FreeLocalStorage`
+ `ReadIOPSLocalStorage`
+ `ReadLatencyLocalStorage`
+ `ReadThroughputLocalStorage`
+ `WriteIOPSLocalStorage`
+ `WriteLatencyLocalStorage`
+ `WriteThroughputLocalStorage`
These metrics provide data about available instance store storage, IOPS, and throughput. For more information about these metrics, see [Amazon CloudWatch instance-level metrics for Amazon RDS](rds-metrics.md#rds-cw-metrics-instance).
To monitor current usage of your local storage, lo in to your database and run the following query:
```
SELECT
spcname AS "Name",
pg_catalog.pg_size_pretty(pg_catalog.pg_tablespace_size(oid)) AS "size"
FROM
pg_catalog.pg_tablespace
WHERE
spcname IN ('rds_temp_tablespace');
```
For more information about the temporary files and their usage, see [Managing temporary files with PostgreSQL](PostgreSQL.ManagingTempFiles.md).
## Limitations for RDS Optimized Reads in PostgreSQL
The following limitation apply to RDS Optimized Reads in PostgreSQL:
+ Transactions can fail when the instance store is full.
# Importing data into PostgreSQL on Amazon RDS
Suppose that you have an existing PostgreSQL deployment that you want to move to Amazon RDS. The complexity of your task depends on the size of your database and the types of database objects that you're transferring. For example, consider a database that contains datasets on the order of gigabytes, along with stored procedures and triggers. Such a database is going to be more complicated than a simple database with only a few megabytes of test data and no triggers or stored procedures.
We recommend that you use native PostgreSQL database migration tools under the following conditions:
+ You have a homogeneous migration, where you are migrating from a database with the same database engine as the target database.
+ You are migrating an entire database.
+ The native tools allow you to migrate your system with minimal downtime.
In most other cases, performing a database migration using AWS Database Migration Service (AWS DMS) is the best approach. AWS DMS can migrate databases without downtime and, for many database engines, continue ongoing replication until you are ready to switch over to the target database. You can migrate to either the same database engine or a different database engine using AWS DMS. If you are migrating to a different database engine than your source database, you can use the AWS Schema Conversion Tool (AWS SCT). You use AWS SCT to migrate schema objects that are not migrated by AWS DMS. For more information about AWS DMS, see [ What is AWS Database Migration Service?](https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html)
Modify your DB parameter group to include the following settings *for your import only*. You should test the parameter settings to find the most efficient settings for your DB instance size. You also need to revert back to production values for these parameters after your import completes.
Modify your DB instance settings to the following:
+ Disable DB instance backups (set backup\$1retention to 0).
+ Disable Multi-AZ.
Modify your DB parameter group to include the following settings. You should only use these settings when importing data. You should test the parameter settings to find the most efficient settings for your DB instance size. You also need to revert back to production values for these parameters after your import completes.
| Parameter | Recommended value when importing | Description |
| --- | --- | --- |
| `maintenance_work_mem` | 524288, 1048576, 2097152, or 4194304 (in KB). These settings are comparable to 512 MB, 1 GB, 2 GB, and 4 GB. | The value for this setting depends on the size of your host. This parameter is used during CREATE INDEX statements and each parallel command can use this much memory. Calculate the best value so that you don't set this value so high that you run out of memory. |
| `max_wal_size` | 256 (for version 9.6), 4096 (for versions 10 and higher) | Maximum size to let the WAL grow during automatic checkpoints. Increasing this parameter can increase the amount of time needed for crash recovery. This parameter replaces `checkpoint_segments` for PostgreSQL 9.6 and later. For PostgreSQL version 9.6, this value is in 16 MB units. For later versions, the value is in 1 MB units. For example, in version 9.6, 128 means 128 chunks that are each 16 MB in size. In version 12.4, 2048 means 2048 chunks that are each 1 MB in size. |
| `checkpoint_timeout` | 1800 | The value for this setting allows for less frequent WAL rotation. |
| `synchronous_commit` | Off | Disable this setting to speed up writes. Turning this parameter off can increase the risk of data loss in the event of a server crash (do not turn off FSYNC). |
| `wal_buffers` | 8192 | This is value is in 8 KB units. This again helps your WAL generation speed |
| `autovacuum` | 0 | Disable the PostgreSQL auto vacuum parameter while you are loading data so that it doesn't use resources |
Use the `pg_dump -Fc` (compressed) or `pg_restore -j` (parallel) commands with these settings.
**Note**
The PostgreSQL command `pg_dumpall` requires super\$1user permissions that are not granted when you create a DB instance, so it cannot be used for importing data.
**Topics**
+ [
# Importing a PostgreSQL database from an Amazon EC2 instance
](PostgreSQL.Procedural.Importing.EC2.md)
+ [
# Using the \$1copy command to import data to a table on a PostgreSQL DB instance
](PostgreSQL.Procedural.Importing.Copy.md)
+ [
# Importing data from Amazon S3 into an RDS for PostgreSQL DB instance
](USER_PostgreSQL.S3Import.md)
+ [
# Transporting PostgreSQL databases between DB instances
](PostgreSQL.TransportableDB.md)
# Importing a PostgreSQL database from an Amazon EC2 instance
If you have data in a PostgreSQL server on an Amazon EC2 instance and want to move it to a PostgreSQL DB instance, you can follow this process to migrate the data.
1. Create a file using pg\$1dump that contains the data to be loaded
1. Create the target DB instance
1. Use *psql* to create the database on the DB instance and load the data
1. Create a DB snapshot of the DB instance
The following sections provide more details on each step listed above.
## Step 1: Create a file using pg\$1dump that contains the data to load
The `pg_dump` utility uses the COPY command to create a schema and data dump of a PostgreSQL database. The dump script generated by `pg_dump` loads data into a database with the same name and recreates the tables, indexes, and foreign keys. You can use the `pg_restore` command and the `-d` parameter to restore the data to a database with a different name.
Before you create the data dump, you should query the tables to be dumped to get a row count so you can confirm the count on the target DB instance.
The following command creates a dump file called mydb2dump.sql for a database called mydb2.
```
prompt>pg_dump dbname=mydb2 -f mydb2dump.sql
```
## Step 2: Create the target DB instance
Create the target PostgreSQL DB instance using either the Amazon RDS console, AWS CLI, or API. Create the instance with the backup retention setting set to 0 and disable Multi-AZ. Doing so allows faster data import. You must create a database on the instance before you can dump the data. The database can have the same name as the database that is contained the dumped data. Alternatively, you can create a database with a different name. In this case, you use the `pg_restore` command and the `-d` parameter to restore the data into the newly named database.
For example, the following commands can be used to dump, restore, and rename a database.
```
pg_dump -Fc -v -h [endpoint of instance] -U [master username] [database] > [database].dump
createdb [new database name]
pg_restore -v -h [endpoint of instance] -U [master username] -d [new database name] [database].dump
```
## Step 3: Use psql to create the database on the DB instance and load data
You can use the same connection you used to run the pg\$1dump command to connect to the target DB instance and recreate the database. Using *psql*, you can use the master user name and master password to create the database on the DB instance
The following example uses *psql* and a dump file named mydb2dump.sql to create a database called mydb2 on a PostgreSQL DB instance called mypginstance:
For Linux, macOS, or Unix:
```
psql \
-f mydb2dump.sql \
--host mypginstance.555555555555.aws-region.rds.amazonaws.com \
--port 8199 \
--username myawsuser \
--password password \
--dbname mydb2
```
For Windows:
```
psql ^
-f mydb2dump.sql ^
--host mypginstance.555555555555.aws-region.rds.amazonaws.com ^
--port 8199 ^
--username myawsuser ^
--password password ^
--dbname mydb2
```
**Note**
Specify a password other than the prompt shown here as a security best practice.
## Step 4: Create a DB snapshot of the DB instance
Once you have verified that the data was loaded into your DB instance, we recommend that you create a DB snapshot of the target PostgreSQL DB instance. DB snapshots are complete backups of your DB instance that can be used to restore your DB instance to a known state. A DB snapshot taken immediately after the load protects you from having to load the data again in case of a mishap. You can also use such a snapshot to seed new DB instances. For information about creating a DB snapshot, see [Creating a DB snapshot for a Single-AZ DB instance for Amazon RDS](USER_CreateSnapshot.md).
# Using the \$1copy command to import data to a table on a PostgreSQL DB instance
The PostgreSQL `\copy` command is a meta-command available from the `psql` interactive client tool. You can use `\copy` to import data into a table on your RDS for PostgreSQL DB instance. To use the `\copy` command, you need to first create the table structure on the target DB instance so that `\copy` has a destination for the data being copied.
You can use `\copy` to load data from a comma-separated values (CSV) file, such as one that's been exported and saved to your client workstation.
To import the CSV data to the target RDS for PostgreSQL DB instance, first connect to the target DB instance using `psql`.
```
psql --host=db-instance.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=target-db
```
You then run `\copy` command with the following parameters to identify the target for the data and its format.
+ `target_table` – The name of the table that should receive the data being copied from the CSV file.
+ `column_list` – Column specifications for the table.
+ `'filename'` – The complete path to the CSV file on your local workstation.
```
\copy target_table from '/path/to/local/filename.csv' WITH DELIMITER ',' CSV;
```
If your CSV file has column heading information, you can use this version of the command and parameters.
```
\copy target_table (column-1, column-2, column-3, ...)
from '/path/to/local/filename.csv' WITH DELIMITER ',' CSV HEADER;
```
If the `\copy` command fails, PostgreSQL outputs error messages.
Creating a new DB instance in the Database Preview environment using `psql` command with the `\copy` meta-command as shown in the following examples. This example uses *source-table* as the source table name, *source-table.csv* as the .csv file, and *target-db* as the target database:
For Linux, macOS, or Unix:
```
$psql target-db \
-U \
-p \
-h \
-c "\copy source-table from 'source-table.csv' with DELIMITER ','"
```
For Windows:
```
$psql target-db ^
-U ^
-p ^
-h ^
-c "\copy source-table from 'source-table.csv' with DELIMITER ','"
```
For complete details about the `\copy` command, see the [psql](http://www.postgresql.org/docs/current/static/app-psql.html) page in the PostgreSQL documentation, in the *Meta-Commands* section.
# Importing data from Amazon S3 into an RDS for PostgreSQL DB instance
You can import data that's been stored using Amazon Simple Storage Service into a table on an RDS for PostgreSQL DB instance. To do this, you first install the RDS for PostgreSQL `aws_s3` extension. This extension provides the functions that you use to import data from an Amazon S3 bucket. A *bucket* is an Amazon S3 container for objects and files. The data can be in a comma-separate value (CSV) file, a text file, or a compressed (gzip) file. Following, you can learn how to install the extension and how to import data from Amazon S3 into a table.
Your database must be running PostgreSQL version 10.7 or higher to import from Amazon S3 into RDS for PostgreSQL.
If you don't have data stored on Amazon S3, you need to first create a bucket and store the data. For more information, see the following topics in the *Amazon Simple Storage Service User Guide*.
+ [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/GetStartedWithS3.html#creating-bucket)
+ [Add an object to a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/GetStartedWithS3.html#uploading-an-object-bucket)
Cross-account import from Amazon S3 is supported. For more information, see [ Granting cross-account permissions](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-walkthroughs-managing-access-example2.html) in the *Amazon Simple Storage Service User Guide*.
You can use the customer managed key for encryption while importing data from S3. For more information, see [ KMS keys stored in AWS KMS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html) in the *Amazon Simple Storage Service User Guide*.
**Topics**
+ [
# Installing the aws\$1s3 extension
](USER_PostgreSQL.S3Import.InstallExtension.md)
+ [
# Overview of importing data from Amazon S3 data
](USER_PostgreSQL.S3Import.Overview.md)
+ [
# Setting up access to an Amazon S3 bucket
](USER_PostgreSQL.S3Import.AccessPermission.md)
+ [
# Importing data from Amazon S3 to your RDS for PostgreSQL DB instance
](USER_PostgreSQL.S3Import.FileFormats.md)
+ [
# Function reference
](USER_PostgreSQL.S3Import.Reference.md)
# Installing the aws\$1s3 extension
Before you can use Amazon S3 with your RDS for PostgreSQL DB instance, you need to install the `aws_s3` extension. This extension provides functions for importing data from an Amazon S3. It also provides functions for exporting data from an RDS for PostgreSQL DB instance to an Amazon S3 bucket. For more information, see [Exporting data from an RDS for PostgreSQL DB instance to Amazon S3](postgresql-s3-export.md). The `aws_s3` extension depends on some of the helper functions in the `aws_commons` extension, which is installed automatically when needed.
**To install the `aws_s3` extension**
1. Use psql (or pgAdmin) to connect to the RDS for PostgreSQL DB instance as a user that has `rds_superuser` privileges. If you kept the default name during the setup process, you connect as `postgres`.
```
psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. To install the extension, run the following command.
```
postgres=> CREATE EXTENSION aws_s3 CASCADE;
NOTICE: installing required extension "aws_commons"
CREATE EXTENSION
```
1. To verify that the extension is installed, you can use the psql `\dx` metacommand.
```
postgres=> \dx
List of installed extensions
Name | Version | Schema | Description
-------------+---------+------------+---------------------------------------------
aws_commons | 1.2 | public | Common data types across AWS services
aws_s3 | 1.1 | public | AWS S3 extension for importing data from S3
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(3 rows)
```
The functions for importing data from Amazon S3 and exporting data to Amazon S3 are now available to use.
# Overview of importing data from Amazon S3 data
**To import S3 data into Amazon RDS**
First, gather the details that you need to supply to the function. These include the name of the table on your RDS for PostgreSQL DB instance, and the bucket name, file path, file type, and AWS Region where the Amazon S3 data is stored. For more information, see [View an object](https://docs.aws.amazon.com/AmazonS3/latest/userguide/OpeningAnObject.html) in the *Amazon Simple Storage Service User Guide*.
**Note**
Multi part data import from Amazon S3 isn't currently supported.
1. Get the name of the table into which the `aws_s3.table_import_from_s3` function is to import the data. As an example, the following command creates a table `t1` that can be used in later steps.
```
postgres=> CREATE TABLE t1
(col1 varchar(80),
col2 varchar(80),
col3 varchar(80));
```
1. Get the details about the Amazon S3 bucket and the data to import. To do this, open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/), and choose **Buckets**. Find the bucket containing your data in the list. Choose the bucket, open its Object overview page, and then choose Properties.
Make a note of the bucket name, path, the AWS Region, and file type. You need the Amazon Resource Name (ARN) later, to set up access to Amazon S3 through an IAM role. For more more information, see [Setting up access to an Amazon S3 bucket](USER_PostgreSQL.S3Import.AccessPermission.md). The image following shows an example.
![\[Image of a file object in an Amazon S3 bucket.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/aws_s3_import-export_s3_bucket-info.png)
1. You can verify the path to the data on the Amazon S3 bucket by using the AWS CLI command `aws s3 cp`. If the information is correct, this command downloads a copy of the Amazon S3 file.
```
aws s3 cp s3://amzn-s3-demo-bucket/sample_file_path ./
```
1. Set up permissions on your RDS for PostgreSQL DB instance to allow access to the file on the Amazon S3 bucket. To do so, you use either an AWS Identity and Access Management (IAM) role or security credentials. For more information, see [Setting up access to an Amazon S3 bucket](USER_PostgreSQL.S3Import.AccessPermission.md).
1. Supply the path and other Amazon S3 object details gathered (see step 2) to the `create_s3_uri` function to construct an Amazon S3 URI object. To learn more about this function, see [aws\$1commons.create\$1s3\$1uri](USER_PostgreSQL.S3Import.Reference.md#USER_PostgreSQL.S3Import.create_s3_uri). The following is an example of constructing this object during a psql session.
```
postgres=> SELECT aws_commons.create_s3_uri(
'docs-lab-store-for-rpg',
'versions_and_jdks_listing.csv',
'us-west-1'
) AS s3_uri \gset
```
In the next step, you pass this object (`aws_commons._s3_uri_1`) to the `aws_s3.table_import_from_s3` function to import the data to the table.
1. Invoke the `aws_s3.table_import_from_s3` function to import the data from Amazon S3 into your table. For reference information, see [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3). For examples, see [Importing data from Amazon S3 to your RDS for PostgreSQL DB instance](USER_PostgreSQL.S3Import.FileFormats.md).
# Setting up access to an Amazon S3 bucket
To import data from an Amazon S3 file, give the RDS for PostgreSQL DB instance permission to access the Amazon S3 bucket containing the file. You provide access to an Amazon S3 bucket in one of two ways, as described in the following topics.
**Topics**
+ [
## Using an IAM role to access an Amazon S3 bucket
](#USER_PostgreSQL.S3Import.ARNRole)
+ [
## Using security credentials to access an Amazon S3 bucket
](#USER_PostgreSQL.S3Import.Credentials)
+ [
## Troubleshooting access to Amazon S3
](#USER_PostgreSQL.S3Import.troubleshooting)
## Using an IAM role to access an Amazon S3 bucket
Before you load data from an Amazon S3 file, give your RDS for PostgreSQL DB instance permission to access the Amazon S3 bucket the file is in. This way, you don't have to manage additional credential information or provide it in the [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3) function call.
To do this, create an IAM policy that provides access to the Amazon S3 bucket. Create an IAM role and attach the policy to the role. Then assign the IAM role to your DB instance.
**To give an RDS for PostgreSQL DB instance access to Amazon S3 through an IAM role**
1. Create an IAM policy.
This policy provides the bucket and object permissions that allow your RDS for PostgreSQL DB instance to access Amazon S3.
Include in the policy the following required actions to allow the transfer of files from an Amazon S3 bucket to Amazon RDS:
+ `s3:GetObject`
+ `s3:ListBucket`
Include in the policy the following resources to identify the Amazon S3 bucket and objects in the bucket. This shows the Amazon Resource Name (ARN) format for accessing Amazon S3.
+ arn:aws:s3:::*amzn-s3-demo-bucket*
+ arn:aws:s3:::*amzn-s3-demo-bucket*/\$1
For more information on creating an IAM policy for RDS for PostgreSQL, see [Creating and using an IAM policy for IAM database access](UsingWithRDS.IAMDBAuth.IAMPolicy.md). See also [Tutorial: Create and attach your first customer managed policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_managed-policies.html) in the *IAM User Guide*.
The following AWS CLI command creates an IAM policy named `rds-s3-import-policy` with these options. It grants access to a bucket named *amzn-s3-demo-bucket*.
**Note**
Make a note of the Amazon Resource Name (ARN) of the policy returned by this command. You need the ARN in a subsequent step when you attach the policy to an IAM role.
**Example**
For Linux, macOS, or Unix:
```
aws iam create-policy \
--policy-name rds-s3-import-policy \
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "s3import",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::amzn-s3-demo-bucket",
"arn:aws:s3:::amzn-s3-demo-bucket/*"
]
}
]
}'
```
For Windows:
```
aws iam create-policy ^
--policy-name rds-s3-import-policy ^
--policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "s3import",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::amzn-s3-demo-bucket",
"arn:aws:s3:::amzn-s3-demo-bucket/*"
]
}
]
}'
```
1. Create an IAM role.
You do this so Amazon RDS can assume this IAM role to access your Amazon S3 buckets. For more information, see [Creating a role to delegate permissions to an IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html) in the *IAM User Guide*.
We recommend using the `[aws:SourceArn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn)` and `[aws:SourceAccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount)` global condition context keys in resource-based policies to limit the service's permissions to a specific resource. This is the most effective way to protect against the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
If you use both global condition context keys and the `aws:SourceArn` value contains the account ID, the `aws:SourceAccount` value and the account in the `aws:SourceArn` value must use the same account ID when used in the same policy statement.
+ Use `aws:SourceArn` if you want cross-service access for a single resource.
+ Use `aws:SourceAccount` if you want to allow any resource in that account to be associated with the cross-service use.
In the policy, be sure to use the `aws:SourceArn` global condition context key with the full ARN of the resource. The following example shows how to do so using the AWS CLI command to create a role named `rds-s3-import-role`.
**Example**
For Linux, macOS, or Unix:
```
aws iam create-role \
--role-name rds-s3-import-role \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "111122223333",
"aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:db:dbname"
}
}
}
]
}'
```
For Windows:
```
aws iam create-role ^
--role-name rds-s3-import-role ^
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "111122223333",
"aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:db:dbname"
}
}
}
]
}'
```
1. Attach the IAM policy that you created to the IAM role that you created.
The following AWS CLI command attaches the policy created in the previous step to the role named `rds-s3-import-role` Replace `your-policy-arn` with the policy ARN that you noted in an earlier step.
**Example**
For Linux, macOS, or Unix:
```
aws iam attach-role-policy \
--policy-arn your-policy-arn \
--role-name rds-s3-import-role
```
For Windows:
```
aws iam attach-role-policy ^
--policy-arn your-policy-arn ^
--role-name rds-s3-import-role
```
1. Add the IAM role to the DB instance.
You do so by using the AWS Management Console or AWS CLI, as described following.
### Console
**To add an IAM role for a PostgreSQL DB instance using the console**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose the PostgreSQL DB instance name to display its details.
1. On the **Connectivity & security** tab, in the **Manage IAM roles **section, choose the role to add under **Add IAM roles to this instance **.
1. Under **Feature**, choose **s3Import**.
1. Choose **Add role**.
### AWS CLI
**To add an IAM role for a PostgreSQL DB instance using the CLI**
+ Use the following command to add the role to the PostgreSQL DB instance named `my-db-instance`. Replace *`your-role-arn`* with the role ARN that you noted in a previous step. Use `s3Import` for the value of the `--feature-name` option.
**Example**
For Linux, macOS, or Unix:
```
aws rds add-role-to-db-instance \
--db-instance-identifier my-db-instance \
--feature-name s3Import \
--role-arn your-role-arn \
--region your-region
```
For Windows:
```
aws rds add-role-to-db-instance ^
--db-instance-identifier my-db-instance ^
--feature-name s3Import ^
--role-arn your-role-arn ^
--region your-region
```
### RDS API
To add an IAM role for a PostgreSQL DB instance using the Amazon RDS API, call the [ AddRoleToDBInstance](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_AddRoleToDBInstance.html) operation.
## Using security credentials to access an Amazon S3 bucket
If you prefer, you can use security credentials to provide access to an Amazon S3 bucket instead of providing access with an IAM role. You do so by specifying the `credentials` parameter in the [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3) function call.
The `credentials` parameter is a structure of type `aws_commons._aws_credentials_1`, which contains AWS credentials. Use the [aws\$1commons.create\$1aws\$1credentials](USER_PostgreSQL.S3Import.Reference.md#USER_PostgreSQL.S3Import.create_aws_credentials) function to set the access key and secret key in an `aws_commons._aws_credentials_1` structure, as shown following.
```
postgres=> SELECT aws_commons.create_aws_credentials(
'sample_access_key', 'sample_secret_key', '')
AS creds \gset
```
After creating the `aws_commons._aws_credentials_1 `structure, use the [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3) function with the `credentials` parameter to import the data, as shown following.
```
postgres=> SELECT aws_s3.table_import_from_s3(
't', '', '(format csv)',
:'s3_uri',
:'creds'
);
```
Or you can include the [aws\$1commons.create\$1aws\$1credentials](USER_PostgreSQL.S3Import.Reference.md#USER_PostgreSQL.S3Import.create_aws_credentials) function call inline within the `aws_s3.table_import_from_s3` function call.
```
postgres=> SELECT aws_s3.table_import_from_s3(
't', '', '(format csv)',
:'s3_uri',
aws_commons.create_aws_credentials('sample_access_key', 'sample_secret_key', '')
);
```
## Troubleshooting access to Amazon S3
If you encounter connection problems when attempting to import data from Amazon S3, see the following for recommendations:
+ [Troubleshooting Amazon RDS identity and access](security_iam_troubleshoot.md)
+ [Troubleshooting Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/troubleshooting.html) in the *Amazon Simple Storage Service User Guide*
+ [Troubleshooting Amazon S3 and IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_iam-s3.html) in the *IAM User Guide*
# Importing data from Amazon S3 to your RDS for PostgreSQL DB instance
You import data from your Amazon S3 bucket by using the `table_import_from_s3` function of the aws\$1s3 extension. For reference information, see [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3).
**Note**
The following examples use the IAM role method to allow access to the Amazon S3 bucket. Thus, the `aws_s3.table_import_from_s3` function calls don't include credential parameters.
The following shows a typical example.
```
postgres=> SELECT aws_s3.table_import_from_s3(
't1',
'',
'(format csv)',
:'s3_uri'
);
```
The parameters are the following:
+ `t1` – The name for the table in the PostgreSQL DB instance to copy the data into.
+ `''` – An optional list of columns in the database table. You can use this parameter to indicate which columns of the S3 data go in which table columns. If no columns are specified, all the columns are copied to the table. For an example of using a column list, see [Importing an Amazon S3 file that uses a custom delimiter](#USER_PostgreSQL.S3Import.FileFormats.CustomDelimiter).
+ `(format csv)` – PostgreSQL COPY arguments. The copy process uses the arguments and format of the [PostgreSQL COPY](https://www.postgresql.org/docs/current/sql-copy.html) command to import the data. Choices for format include comma-separated value (CSV) as shown in this example, text, and binary. The default is text.
+ `s3_uri` – A structure that contains the information identifying the Amazon S3 file. For an example of using the [aws\$1commons.create\$1s3\$1uri](USER_PostgreSQL.S3Import.Reference.md#USER_PostgreSQL.S3Import.create_s3_uri) function to create an `s3_uri` structure, see [Overview of importing data from Amazon S3 data](USER_PostgreSQL.S3Import.Overview.md).
For more information about this function, see [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3).
The `aws_s3.table_import_from_s3` function returns text. To specify other kinds of files for import from an Amazon S3 bucket, see one of the following examples.
**Note**
Importing 0 bytes file will cause an error.
**Topics**
+ [
## Importing an Amazon S3 file that uses a custom delimiter
](#USER_PostgreSQL.S3Import.FileFormats.CustomDelimiter)
+ [
## Importing an Amazon S3 compressed (gzip) file
](#USER_PostgreSQL.S3Import.FileFormats.gzip)
+ [
## Importing an encoded Amazon S3 file
](#USER_PostgreSQL.S3Import.FileFormats.Encoded)
## Importing an Amazon S3 file that uses a custom delimiter
The following example shows how to import a file that uses a custom delimiter. It also shows how to control where to put the data in the database table using the `column_list` parameter of the [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3) function.
For this example, assume that the following information is organized into pipe-delimited columns in the Amazon S3 file.
```
1|foo1|bar1|elephant1
2|foo2|bar2|elephant2
3|foo3|bar3|elephant3
4|foo4|bar4|elephant4
...
```
**To import a file that uses a custom delimiter**
1. Create a table in the database for the imported data.
```
postgres=> CREATE TABLE test (a text, b text, c text, d text, e text);
```
1. Use the following form of the [aws\$1s3.table\$1import\$1from\$1s3](USER_PostgreSQL.S3Import.Reference.md#aws_s3.table_import_from_s3) function to import data from the Amazon S3 file.
You can include the [aws\$1commons.create\$1s3\$1uri](USER_PostgreSQL.S3Import.Reference.md#USER_PostgreSQL.S3Import.create_s3_uri) function call inline within the `aws_s3.table_import_from_s3` function call to specify the file.
```
postgres=> SELECT aws_s3.table_import_from_s3(
'test',
'a,b,d,e',
'DELIMITER ''|''',
aws_commons.create_s3_uri('amzn-s3-demo-bucket', 'pipeDelimitedSampleFile', 'us-east-2')
);
```
The data is now in the table in the following columns.
```
postgres=> SELECT * FROM test;
a | b | c | d | e
---+------+---+---+------+-----------
1 | foo1 | | bar1 | elephant1
2 | foo2 | | bar2 | elephant2
3 | foo3 | | bar3 | elephant3
4 | foo4 | | bar4 | elephant4
```
## Importing an Amazon S3 compressed (gzip) file
The following example shows how to import a file from Amazon S3 that is compressed with gzip. The file that you import needs to have the following Amazon S3 metadata:
+ Key: `Content-Encoding`
+ Value: `gzip`
If you upload the file using the AWS Management Console, the metadata is typically applied by the system. For information about uploading files to Amazon S3 using the AWS Management Console, the AWS CLI, or the API, see [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) in the *Amazon Simple Storage Service User Guide*.
For more information about Amazon S3 metadata and details about system-provided metadata, see [Editing object metadata in the Amazon S3 console](https://docs.aws.amazon.com/AmazonS3/latest/userguide/add-object-metadata.html) in the *Amazon Simple Storage Service User Guide*.
Import the gzip file into your RDS for PostgreSQL DB instance as shown following.
```
postgres=> CREATE TABLE test_gzip(id int, a text, b text, c text, d text);
postgres=> SELECT aws_s3.table_import_from_s3(
'test_gzip', '', '(format csv)',
'amzn-s3-demo-bucket', 'test-data.gz', 'us-east-2'
);
```
## Importing an encoded Amazon S3 file
The following example shows how to import a file from Amazon S3 that has Windows-1252 encoding.
```
postgres=> SELECT aws_s3.table_import_from_s3(
'test_table', '', 'encoding ''WIN1252''',
aws_commons.create_s3_uri('amzn-s3-demo-bucket', 'SampleFile', 'us-east-2')
);
```
# Function reference
**Topics**
+ [
## aws\$1s3.table\$1import\$1from\$1s3
](#aws_s3.table_import_from_s3)
+ [
## aws\$1commons.create\$1s3\$1uri
](#USER_PostgreSQL.S3Import.create_s3_uri)
+ [
## aws\$1commons.create\$1aws\$1credentials
](#USER_PostgreSQL.S3Import.create_aws_credentials)
## aws\$1s3.table\$1import\$1from\$1s3
Imports Amazon S3 data into an Amazon RDS table. The `aws_s3` extension provides the `aws_s3.table_import_from_s3` function. The return value is text.
### Syntax
The required parameters are `table_name`, `column_list` and `options`. These identify the database table and specify how the data is copied into the table.
You can also use the following parameters:
+ The `s3_info` parameter specifies the Amazon S3 file to import. When you use this parameter, access to Amazon S3 is provided by an IAM role for the PostgreSQL DB instance.
```
aws_s3.table_import_from_s3 (
table_name text,
column_list text,
options text,
s3_info aws_commons._s3_uri_1
)
```
+ The `credentials` parameter specifies the credentials to access Amazon S3. When you use this parameter, you don't use an IAM role.
```
aws_s3.table_import_from_s3 (
table_name text,
column_list text,
options text,
s3_info aws_commons._s3_uri_1,
credentials aws_commons._aws_credentials_1
)
```
### Parameters
*table\$1name*
A required text string containing the name of the PostgreSQL database table to import the data into.
*column\$1list*
A required text string containing an optional list of the PostgreSQL database table columns in which to copy the data. If the string is empty, all columns of the table are used. For an example, see [Importing an Amazon S3 file that uses a custom delimiter](USER_PostgreSQL.S3Import.FileFormats.md#USER_PostgreSQL.S3Import.FileFormats.CustomDelimiter).
*options*
A required text string containing arguments for the PostgreSQL `COPY` command. These arguments specify how the data is to be copied into the PostgreSQL table. For more details, see the [PostgreSQL COPY documentation](https://www.postgresql.org/docs/current/sql-copy.html).
*s3\$1info*
An `aws_commons._s3_uri_1` composite type containing the following information about the S3 object:
+ `bucket` – The name of the Amazon S3 bucket containing the file.
+ `file_path` – The Amazon S3 file name including the path of the file.
+ `region` – The AWS Region that the file is in. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
*credentials*
An `aws_commons._aws_credentials_1` composite type containing the following credentials to use for the import operation:
+ Access key
+ Secret key
+ Session token
For information about creating an `aws_commons._aws_credentials_1` composite structure, see [aws\$1commons.create\$1aws\$1credentials](#USER_PostgreSQL.S3Import.create_aws_credentials).
### Alternate syntax
To help with testing, you can use an expanded set of parameters instead of the `s3_info` and `credentials` parameters. Following are additional syntax variations for the `aws_s3.table_import_from_s3` function:
+ Instead of using the `s3_info` parameter to identify an Amazon S3 file, use the combination of the `bucket`, `file_path`, and `region` parameters. With this form of the function, access to Amazon S3 is provided by an IAM role on the PostgreSQL DB instance.
```
aws_s3.table_import_from_s3 (
table_name text,
column_list text,
options text,
bucket text,
file_path text,
region text
)
```
+ Instead of using the `credentials` parameter to specify Amazon S3 access, use the combination of the `access_key`, `session_key`, and `session_token` parameters.
```
aws_s3.table_import_from_s3 (
table_name text,
column_list text,
options text,
bucket text,
file_path text,
region text,
access_key text,
secret_key text,
session_token text
)
```
### Alternate parameters
*bucket*
A text string containing the name of the Amazon S3 bucket that contains the file.
*file\$1path*
A text string containing the Amazon S3 file name including the path of the file.
*region*
A text string identifying the AWS Region location of the file. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
*access\$1key*
A text string containing the access key to use for the import operation. The default is NULL.
*secret\$1key*
A text string containing the secret key to use for the import operation. The default is NULL.
*session\$1token*
(Optional) A text string containing the session key to use for the import operation. The default is NULL.
## aws\$1commons.create\$1s3\$1uri
Creates an `aws_commons._s3_uri_1` structure to hold Amazon S3 file information. Use the results of the `aws_commons.create_s3_uri` function in the `s3_info` parameter of the [aws\$1s3.table\$1import\$1from\$1s3](#aws_s3.table_import_from_s3) function.
### Syntax
```
aws_commons.create_s3_uri(
bucket text,
file_path text,
region text
)
```
### Parameters
*bucket*
A required text string containing the Amazon S3 bucket name for the file.
*file\$1path*
A required text string containing the Amazon S3 file name including the path of the file.
*region*
A required text string containing the AWS Region that the file is in. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
## aws\$1commons.create\$1aws\$1credentials
Sets an access key and secret key in an `aws_commons._aws_credentials_1` structure. Use the results of the `aws_commons.create_aws_credentials` function in the `credentials` parameter of the [aws\$1s3.table\$1import\$1from\$1s3](#aws_s3.table_import_from_s3) function.
### Syntax
```
aws_commons.create_aws_credentials(
access_key text,
secret_key text,
session_token text
)
```
### Parameters
*access\$1key*
A required text string containing the access key to use for importing an Amazon S3 file. The default is NULL.
*secret\$1key*
A required text string containing the secret key to use for importing an Amazon S3 file. The default is NULL.
*session\$1token*
An optional text string containing the session token to use for importing an Amazon S3 file. The default is NULL. If you provide an optional `session_token`, you can use temporary credentials.
# Transporting PostgreSQL databases between DB instances
By using PostgreSQL transportable databases for Amazon RDS, you can move a PostgreSQL database between two DB instances. This is a very fast way to migrate large databases between different DB instances. To use this approach, your DB instances must both run the same major version of PostgreSQL.
This capability requires that you install the `pg_transport` extension on both the source and the destination DB instance. The `pg_transport` extension provides a physical transport mechanism that moves the database files with minimal processing. This mechanism moves data much faster than traditional dump and load processes, with less downtime.
**Note**
PostgreSQL transportable databases are available in RDS for PostgreSQL 11.5 and higher, and RDS for PostgreSQL version 10.10 and higher.
To transport a PostgreSQL DB instance from one RDS for PostgreSQL DB instance to another, you first set up the source and destination instances as detailed in [ Setting up a DB instance for transport](PostgreSQL.TransportableDB.Setup.md). You can then transport the database by using the function described in [ Transporting a PostgreSQL database](PostgreSQL.TransportableDB.Transporting.md).
**Topics**
+ [
## What happens during database transport
](#PostgreSQL.TransportableDB.DuringTransport)
+ [
## Limitations for using PostgreSQL transportable databases
](#PostgreSQL.TransportableDB.Limits)
+ [
# Setting up to transport a PostgreSQL database
](PostgreSQL.TransportableDB.Setup.md)
+ [
# Transporting a PostgreSQL database to the destination from the source
](PostgreSQL.TransportableDB.Transporting.md)
+ [
# Transportable databases function reference
](PostgreSQL.TransportableDB.transport.import_from_server.md)
+ [
# Transportable databases parameter reference
](PostgreSQL.TransportableDB.Parameters.md)
## What happens during database transport
The PostgreSQL transportable databases feature uses a pull model to import the database from the source DB instance to the destination. The `transport.import_from_server` function creates the in-transit database on the destination DB instance. The in-transit database is inaccessible on the destination DB instance for the duration of the transport.
When transport begins, all current sessions on the source database are ended. Any databases other than the source database on the source DB instance aren't affected by the transport.
The source database is put into a special read-only mode. While it's in this mode, you can connect to the source database and run read-only queries. However, write-enabled queries and some other types of commands are blocked. Only the specific source database that is being transported is affected by these restrictions.
During transport, you can't restore the destination DB instance to a point in time. This is because the transport isn't transactional and doesn't use the PostgreSQL write-ahead log to record changes. If the destination DB instance has automatic backups enabled, a backup is automatically taken after transport completes. Point-in-time restores are available for times *after* the backup finishes.
If the transport fails, the `pg_transport` extension attempts to undo all changes to the source and destination DB instances. This includes removing the destination's partially transported database. Depending on the type of failure, the source database might continue to reject write-enabled queries. If this happens, use the following command to allow write-enabled queries.
```
ALTER DATABASE db-name SET default_transaction_read_only = false;
```
## Limitations for using PostgreSQL transportable databases
Transportable databases have the following limitations:
+ **Read replicas ** – You can't use transportable databases on read replicas or parent instances of read replicas.
+ **Unsupported column types** – You can't use the `reg` data types in any database tables that you plan to transport with this method. These types depend on system catalog object IDs (OIDs), which often change during transport.
+ **Tablespaces** – All source database objects must be in the default `pg_default` tablespace.
+ **Compatibility** – Both the source and destination DB instances must run the same major version of PostgreSQL.
+ **Extensions** – The source DB instance can have only the `pg_transport` installed.
+ **Roles and ACLs** – The source database's access privileges and ownership information aren't carried over to the destination database. All database objects are created and owned by the local destination user of the transport.
+ **Concurrent transports** – A single DB instance can support up to 32 concurrent transports, including both imports and exports, if worker processes have been configured properly.
+ **RDS for PostgreSQL DB instances only** – PostgreSQL transportable databases are supported on RDS for PostgreSQL DB instances only. You can't use it with on-premises databases or databases running on Amazon EC2.
# Setting up to transport a PostgreSQL database
Before you begin, make sure that your RDS for PostgreSQL DB instances meet the following requirements:
+ The RDS for PostgreSQL DB instances for source and destination must run the same version of PostgreSQL.
+ The destination DB can't have a database of the same name as the source DB that you want to transport.
+ The account you use to run the transport needs `rds_superuser` privileges on both the source DB and the destination DB.
+ The security group for the source DB instance must allow inbound access from the destination DB instance. This might already be the case if your source and destination DB instances are located in the VPC. For more information about security groups, see [Controlling access with security groups](Overview.RDSSecurityGroups.md).
Transporting databases from a source DB instance to a destination DB instance requires several changes to the DB parameter group associated with each instance. That means that you must create a custom DB parameter group for the source DB instance and create a custom DB parameter group for the destination DB instance.
**Note**
If your DB instances are already configured using custom DB parameter groups, you can start with step 2 in the following procedure.
**To configure the custom DB group parameters for transporting databases**
For the following steps, use an account that has `rds_superuser` privileges.
1. If the source and destination DB instances use a default DB parameter group, you need to create a custom DB parameter group using the appropriate version for your instances. You do this so you can change values for several parameters. For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
1. In the custom DB parameter group, change values for the following parameters:
+ `shared_preload_libraries` – Add `pg_transport` to the list of libraries.
+ `pg_transport.num_workers` – The default value is 3. Increase or reduce this value as needed for your database. For a 200 GB database, we recommend no larger than 8. Keep in mind that if you increase the default value for this parameter, you should also increase the value of `max_worker_processes`.
+ `pg_transport.work_mem` – The default value is either 128 MB or 256 MB, depending on the PostgreSQL version. The default setting can typically be left unchanged.
+ `max_worker_processes` – The value of this parameter needs to be set using the following calculation:
```
(3 * pg_transport.num_workers) + 9
```
This value is required on the destination to handle various background worker processes involved in the transport. To learn more about `max_worker_processes,` see [Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html) in the PostgreSQL documentation.
For more information about `pg_transport` parameters, see [Transportable databases parameter reference](PostgreSQL.TransportableDB.Parameters.md).
1. Reboot the source RDS for PostgreSQL DB instance and the destination instance so that the settings for the parameters take effect.
1. Connect to your RDS for PostgreSQL source DB instance.
```
psql --host=source-instance.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. Remove extraneous extensions from the public schema of the DB instance. Only the `pg_transport` extension is allowed during the actual transport operation.
1. Install the `pg_transport` extension as follows:
```
postgres=> CREATE EXTENSION pg_transport;
CREATE EXTENSION
```
1. Connect to your RDS for PostgreSQL destination DB instance. Remove any extraneous extensions, and then install the `pg_transport` extension.
```
postgres=> CREATE EXTENSION pg_transport;
CREATE EXTENSION
```
# Transporting a PostgreSQL database to the destination from the source
After you complete the process described in [Setting up to transport a PostgreSQL database](PostgreSQL.TransportableDB.Setup.md), you can start the transport. To do so, run the `transport.import_from_server` function on the destination DB instance. In the syntax following you can find the function parameters.
```
SELECT transport.import_from_server(
'source-db-instance-endpoint',
source-db-instance-port,
'source-db-instance-user',
'source-user-password',
'source-database-name',
'destination-user-password',
false);
```
The `false` value shown in the example tells the function that this is not a dry run. To test your transport setup, you can specify `true` for the `dry_run` option when you call the function, as shown following:
```
postgres=> SELECT transport.import_from_server(
'docs-lab-source-db.666666666666aws-region.rds.amazonaws.com', 5432,
'postgres', '********', 'labdb', '******', true);
INFO: Starting dry-run of import of database "labdb".
INFO: Created connections to remote database (took 0.03 seconds).
INFO: Checked remote cluster compatibility (took 0.05 seconds).
INFO: Dry-run complete (took 0.08 seconds total).
import_from_server
--------------------
(1 row)
```
The INFO lines are output because the `pg_transport.timing` parameter is set to its default value, `true`. Set the `dry_run` to `false` when you run the command and the source database is imported to the destination, as shown following:
```
INFO: Starting import of database "labdb".
INFO: Created connections to remote database (took 0.02 seconds).
INFO: Marked remote database as read only (took 0.13 seconds).
INFO: Checked remote cluster compatibility (took 0.03 seconds).
INFO: Signaled creation of PITR blackout window (took 2.01 seconds).
INFO: Applied remote database schema pre-data (took 0.50 seconds).
INFO: Created connections to local cluster (took 0.01 seconds).
INFO: Locked down destination database (took 0.00 seconds).
INFO: Completed transfer of database files (took 0.24 seconds).
INFO: Completed clean up (took 1.02 seconds).
INFO: Physical transport complete (took 3.97 seconds total).
import_from_server
--------------------
(1 row)
```
This function requires that you provide database user passwords. Thus, we recommend that you change the passwords of the user roles you used after transport is complete. Or, you can use SQL bind variables to create temporary user roles. Use these temporary roles for the transport and then discard the roles afterwards.
If your transport isn't successful, you might see an error message similar to the following:
```
pg_transport.num_workers=8 25% of files transported failed to download file data
```
The "failed to download file data" error message indicates that the number of worker processes isn't set correctly for the size of the database. You might need to increase or decrease the value set for `pg_transport.num_workers`. Each failure reports the percentage of completion, so you can see the impact of your changes. For example, changing the setting from 8 to 4 in one case resulted in the following:
```
pg_transport.num_workers=4 75% of files transported failed to download file data
```
Keep in mind that the `max_worker_processes` parameter is also taken into account during the transport process. In other words, you might need to modify both `pg_transport.num_workers` and `max_worker_processes` to successfully transport the database. The example shown finally worked when the `pg_transport.num_workers` was set to 2:
```
pg_transport.num_workers=2 100% of files transported
```
For more information about the `transport.import_from_server` function and its parameters, see [Transportable databases function reference](PostgreSQL.TransportableDB.transport.import_from_server.md).
# Transportable databases function reference
The `transport.import_from_server` function transports a PostgreSQL database by importing it from a source DB instance to a destination DB instance. It does this by using a physical database connection transport mechanism.
Before starting the transport, this function verifies that the source and the destination DB instances are the same version and are compatible for the migration. It also confirms that the destination DB instance has enough space for the source.
**Syntax**
```
transport.import_from_server(
host text,
port int,
username text,
password text,
database text,
local_password text,
dry_run bool
)
```
**Return Value**
None.
**Parameters**
You can find descriptions of the `transport.import_from_server` function parameters in the following table.
****
| Parameter | Description |
| --- | --- |
| host | The endpoint of the source DB instance. |
| port | An integer representing the port of the source DB instance. PostgreSQL DB instances often use port 5432. |
| username | The user of the source DB instance. This user must be a member of the `rds_superuser` role. |
| password | The user password of the source DB instance. |
| database | The name of the database in the source DB instance to transport. |
| local\$1password | The local password of the current user for the destination DB instance. This user must be a member of the `rds_superuser` role. |
| dry\$1run | An optional Boolean value specifying whether to perform a dry run. The default is `false`, which means the transport proceeds.To confirm compatibility between the source and destination DB instances without performing the actual transport, set dry\$1run to true. |
**Example**
For an example, see [Transporting a PostgreSQL database to the destination from the source](PostgreSQL.TransportableDB.Transporting.md).
# Transportable databases parameter reference
Several parameters control the behavior of the `pg_transport` extension. Following, you can find descriptions of these parameters.
**`pg_transport.num_workers`**
The number of workers to use for the transport process. The default is 3. Valid values are 1–32. Even the largest database transports typically require fewer than 8 workers. The value of this setting on the destination DB instance is used by both destination and source during transport.
**`pg_transport.timing` **
Specifies whether to report timing information during the transport. The default is `true`, meaning that timing information is reported. We recommend that you leave this parameter set to `true` so you can monitor progress. For example output, see [Transporting a PostgreSQL database to the destination from the source](PostgreSQL.TransportableDB.Transporting.md).
**`pg_transport.work_mem`**
The maximum amount of memory to allocate for each worker. The default is 131072 kilobytes (KB) or 262144 KB (256 MB), depending on the PostgreSQL version. The minimum value is 64 megabytes (65536 KB). Valid values are in kilobytes (KBs) as binary base-2 units, where 1 KB = 1024 bytes.
The transport might use less memory than is specified in this parameter. Even large database transports typically require less than 256 MB (262144 KB) of memory per worker.
# Exporting data from an RDS for PostgreSQL DB instance to Amazon S3
You can query data from an RDS for PostgreSQL DB instance and export it directly into files stored in an Amazon S3 bucket. To do this, you first install the RDS for PostgreSQL `aws_s3` extension. This extension provides you with the functions that you use to export the results of queries to Amazon S3. Following, you can find out how to install the extension and how to export data to Amazon S3.
**Note**
Cross-account export to Amazon S3 isn't supported.
All currently available versions of RDS for PostgreSQL support exporting data to Amazon Simple Storage Service. For detailed version information, see [Amazon RDS for PostgreSQL updates](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-versions.html) in the *Amazon RDS for PostgreSQL Release Notes*.
If you don't have a bucket set up for your export, see the following topics the *Amazon Simple Storage Service User Guide*.
+ [Setting up Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/setting-up-s3.html)
+ [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html)
By default, the data exported from RDS for PostgreSQL to Amazon S3 uses server-side encryption with an AWS managed key. If you are using bucket encryption, the Amazon S3 bucket must be encrypted with an AWS Key Management Service (AWS KMS) key (SSE-KMS). Currently, buckets encrypted with Amazon S3 managed keys (SSE-S3) are not supported.
**Note**
You can save DB snapshot data to Amazon S3 using the AWS Management Console, AWS CLI, or Amazon RDS API. For more information, see [Exporting DB snapshot data to Amazon S3 for Amazon RDS](USER_ExportSnapshot.md).
**Topics**
+ [
## Installing the aws\$1s3 extension
](#USER_PostgreSQL.S3Export.InstallExtension)
+ [
## Overview of exporting data to Amazon S3
](#postgresql-s3-export-overview)
+ [
## Specifying the Amazon S3 file path to export to
](#postgresql-s3-export-file)
+ [
# Setting up access to an Amazon S3 bucket
](postgresql-s3-export-access-bucket.md)
+ [
# Exporting query data using the aws\$1s3.query\$1export\$1to\$1s3 function
](postgresql-s3-export-examples.md)
+ [
# Function reference
](postgresql-s3-export-functions.md)
+ [
# Troubleshooting access to Amazon S3
](postgresql-s3-export-troubleshoot.md)
## Installing the aws\$1s3 extension
Before you can use Amazon Simple Storage Service with your RDS for PostgreSQL DB instance, you need to install the `aws_s3` extension. This extension provides functions for exporting data from an RDS for PostgreSQL DB instance to an Amazon S3 bucket. It also provides functions for importing data from an Amazon S3. For more information, see [Importing data from Amazon S3 into an RDS for PostgreSQL DB instance](USER_PostgreSQL.S3Import.md). The `aws_s3` extension depends on some of the helper functions in the `aws_commons` extension, which is installed automatically when needed.
**To install the `aws_s3` extension**
1. Use psql (or pgAdmin) to connect to the RDS for PostgreSQL DB instance as a user that has `rds_superuser` privileges. If you kept the default name during the setup process, you connect as `postgres`.
```
psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. To install the extension, run the following command.
```
postgres=> CREATE EXTENSION aws_s3 CASCADE;
NOTICE: installing required extension "aws_commons"
CREATE EXTENSION
```
1. To verify that the extension is installed, you can use the psql `\dx` metacommand.
```
postgres=> \dx
List of installed extensions
Name | Version | Schema | Description
-------------+---------+------------+---------------------------------------------
aws_commons | 1.2 | public | Common data types across AWS services
aws_s3 | 1.1 | public | AWS S3 extension for importing data from S3
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(3 rows)
```
The functions for importing data from Amazon S3 and exporting data to Amazon S3 are now available to use.
### Verify that your RDS for PostgreSQL version supports exports to Amazon S3
You can verify that your RDS for PostgreSQL version supports export to Amazon S3 by using the `describe-db-engine-versions` command. The following example verifies support for version 10.14.
```
aws rds describe-db-engine-versions --region us-east-1
--engine postgres --engine-version 10.14 | grep s3Export
```
If the output includes the string `"s3Export"`, then the engine supports Amazon S3 exports. Otherwise, the engine doesn't support them.
## Overview of exporting data to Amazon S3
To export data stored in an RDS for PostgreSQL database to an Amazon S3 bucket, use the following procedure.
**To export RDS for PostgreSQL data to S3**
1. Identify an Amazon S3 file path to use for exporting data. For details about this process, see [Specifying the Amazon S3 file path to export to](#postgresql-s3-export-file).
1. Provide permission to access the Amazon S3 bucket.
To export data to an Amazon S3 file, give the RDS for PostgreSQL DB instance permission to access the Amazon S3 bucket that the export will use for storage. Doing this includes the following steps:
1. Create an IAM policy that provides access to an Amazon S3 bucket that you want to export to.
1. Create an IAM role.
1. Attach the policy you created to the role you created.
1. Add this IAM role to your DB instance.
For details about this process, see [Setting up access to an Amazon S3 bucket](postgresql-s3-export-access-bucket.md).
1. Identify a database query to get the data. Export the query data by calling the `aws_s3.query_export_to_s3` function.
After you complete the preceding preparation tasks, use the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function to export query results to Amazon S3. For details about this process, see [Exporting query data using the aws\$1s3.query\$1export\$1to\$1s3 function](postgresql-s3-export-examples.md).
## Specifying the Amazon S3 file path to export to
Specify the following information to identify the location in Amazon S3 where you want to export data to:
+ Bucket name – A *bucket* is a container for Amazon S3 objects or files.
For more information on storing data with Amazon S3, see [ Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) and [Working with objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/uploading-downloading-objects.html) in the *Amazon Simple Storage Service User Guide*.
+ File path – The file path identifies where the export is stored in the Amazon S3 bucket. The file path consists of the following:
+ An optional path prefix that identifies a virtual folder path.
+ A file prefix that identifies one or more files to be stored. Larger exports are stored in multiple files, each with a maximum size of approximately 6 GB. The additional file names have the same file prefix but with `_partXX` appended. The `XX` represents 2, then 3, and so on.
For example, a file path with an `exports` folder and a `query-1-export` file prefix is `/exports/query-1-export`.
+ AWS Region (optional) – The AWS Region where the Amazon S3 bucket is located. If you don't specify an AWS Region value, then Amazon RDS saves your files into Amazon S3 in the same AWS Region as the exporting DB instance.
**Note**
Currently, the AWS Region must be the same as the region of the exporting DB instance.
For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
To hold the Amazon S3 file information about where the export is to be stored, you can use the [aws\$1commons.create\$1s3\$1uri](postgresql-s3-export-functions.md#aws_commons.create_s3_uri) function to create an `aws_commons._s3_uri_1` composite structure as follows.
```
psql=> SELECT aws_commons.create_s3_uri(
'amzn-s3-demo-bucket',
'sample-filepath',
'us-west-2'
) AS s3_uri_1 \gset
```
You later provide this `s3_uri_1` value as a parameter in the call to the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function. For examples, see [Exporting query data using the aws\$1s3.query\$1export\$1to\$1s3 function](postgresql-s3-export-examples.md).
# Setting up access to an Amazon S3 bucket
To export data to Amazon S3, give your PostgreSQL DB instance permission to access the Amazon S3 bucket that the files are to go in.
To do this, use the following procedure.
**To give a PostgreSQL DB instance access to Amazon S3 through an IAM role**
1. Create an IAM policy.
This policy provides the bucket and object permissions that allow your PostgreSQL DB instance to access Amazon S3.
As part of creating this policy, take the following steps:
1. Include in the policy the following required actions to allow the transfer of files from your PostgreSQL DB instance to an Amazon S3 bucket:
+ `s3:PutObject`
+ `s3:AbortMultipartUpload`
1. Include the Amazon Resource Name (ARN) that identifies the Amazon S3 bucket and objects in the bucket. The ARN format for accessing Amazon S3 is: `arn:aws:s3:::amzn-s3-demo-bucket/*`
For more information on creating an IAM policy for Amazon RDS for PostgreSQL, see [Creating and using an IAM policy for IAM database access](UsingWithRDS.IAMDBAuth.IAMPolicy.md). See also [Tutorial: Create and attach your first customer managed policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_managed-policies.html) in the *IAM User Guide*.
The following AWS CLI command creates an IAM policy named `rds-s3-export-policy` with these options. It grants access to a bucket named *amzn-s3-demo-bucket*.
**Warning**
We recommend that you set up your database within a private VPC that has endpoint policies configured for accessing specific buckets. For more information, see [ Using endpoint policies for Amazon S3](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-s3.html#vpc-endpoints-policies-s3) in the Amazon VPC User Guide.
We strongly recommend that you do not create a policy with all-resource access. This access can pose a threat for data security. If you create a policy that gives `S3:PutObject` access to all resources using `"Resource":"*"`, then a user with export privileges can export data to all buckets in your account. In addition, the user can export data to *any publicly writable bucket within your AWS Region*.
After you create the policy, note the Amazon Resource Name (ARN) of the policy. You need the ARN for a subsequent step when you attach the policy to an IAM role.
```
aws iam create-policy --policy-name rds-s3-export-policy --policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "s3export",
"Action": [
"s3:PutObject*",
"s3:ListBucket",
"s3:GetObject*",
"s3:DeleteObject*",
"s3:GetBucketLocation",
"s3:AbortMultipartUpload"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::amzn-s3-demo-bucket/*"
]
}
]
}'
```
1. Create an IAM role.
You do this so Amazon RDS can assume this IAM role on your behalf to access your Amazon S3 buckets. For more information, see [Creating a role to delegate permissions to an IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html) in the *IAM User Guide*.
We recommend using the `[aws:SourceArn](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourcearn)` and `[aws:SourceAccount](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html#condition-keys-sourceaccount)` global condition context keys in resource-based policies to limit the service's permissions to a specific resource. This is the most effective way to protect against the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
If you use both global condition context keys and the `aws:SourceArn` value contains the account ID, the `aws:SourceAccount` value and the account in the `aws:SourceArn` value must use the same account ID when used in the same policy statement.
+ Use `aws:SourceArn` if you want cross-service access for a single resource.
+ Use `aws:SourceAccount` if you want to allow any resource in that account to be associated with the cross-service use.
In the policy, be sure to use the `aws:SourceArn` global condition context key with the full ARN of the resource. The following example shows how to do so using the AWS CLI command to create a role named `rds-s3-export-role`.
**Example**
For Linux, macOS, or Unix:
```
aws iam create-role \
--role-name rds-s3-export-role \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "111122223333",
"aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:db:dbname"
}
}
}
]
}'
```
For Windows:
```
aws iam create-role ^
--role-name rds-s3-export-role ^
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"aws:SourceAccount": "111122223333",
"aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:db:dbname"
}
}
}
]
}'
```
1. Attach the IAM policy that you created to the IAM role that you created.
The following AWS CLI command attaches the policy created earlier to the role named `rds-s3-export-role.` Replace `your-policy-arn` with the policy ARN that you noted in an earlier step.
```
aws iam attach-role-policy --policy-arn your-policy-arn --role-name rds-s3-export-role
```
1. Add the IAM role to the DB instance. You do so by using the AWS Management Console or AWS CLI, as described following.
## Console
**To add an IAM role for a PostgreSQL DB instance using the console**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose the PostgreSQL DB instance name to display its details.
1. On the **Connectivity & security** tab, in the **Manage IAM roles **section, choose the role to add under **Add IAM roles to this instance**.
1. Under **Feature**, choose **s3Export**.
1. Choose **Add role**.
## AWS CLI
**To add an IAM role for a PostgreSQL DB instance using the CLI**
+ Use the following command to add the role to the PostgreSQL DB instance named `my-db-instance`. Replace *`your-role-arn`* with the role ARN that you noted in a previous step. Use `s3Export` for the value of the `--feature-name` option.
**Example**
For Linux, macOS, or Unix:
```
aws rds add-role-to-db-instance \
--db-instance-identifier my-db-instance \
--feature-name s3Export \
--role-arn your-role-arn \
--region your-region
```
For Windows:
```
aws rds add-role-to-db-instance ^
--db-instance-identifier my-db-instance ^
--feature-name s3Export ^
--role-arn your-role-arn ^
--region your-region
```
# Exporting query data using the aws\$1s3.query\$1export\$1to\$1s3 function
Export your PostgreSQL data to Amazon S3 by calling the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function.
**Topics**
+ [
## Prerequisites
](#postgresql-s3-export-examples-prerequisites)
+ [
## Calling aws\$1s3.query\$1export\$1to\$1s3
](#postgresql-s3-export-examples-basic)
+ [
## Exporting to a CSV file that uses a custom delimiter
](#postgresql-s3-export-examples-custom-delimiter)
+ [
## Exporting to a binary file with encoding
](#postgresql-s3-export-examples-encoded)
## Prerequisites
Before you use the `aws_s3.query_export_to_s3` function, be sure to complete the following prerequisites:
+ Install the required PostgreSQL extensions as described in [Overview of exporting data to Amazon S3](postgresql-s3-export.md#postgresql-s3-export-overview).
+ Determine where to export your data to Amazon S3 as described in [Specifying the Amazon S3 file path to export to](postgresql-s3-export.md#postgresql-s3-export-file).
+ Make sure that the DB instance has export access to Amazon S3 as described in [Setting up access to an Amazon S3 bucket](postgresql-s3-export-access-bucket.md).
The examples following use a database table called `sample_table`. These examples export the data into a bucket called *amzn-s3-demo-bucket*. The example table and data are created with the following SQL statements in psql.
```
psql=> CREATE TABLE sample_table (bid bigint PRIMARY KEY, name varchar(80));
psql=> INSERT INTO sample_table (bid,name) VALUES (1, 'Monday'), (2,'Tuesday'), (3, 'Wednesday');
```
## Calling aws\$1s3.query\$1export\$1to\$1s3
The following shows the basic ways of calling the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function.
These examples use the variable `s3_uri_1` to identify a structure that contains the information identifying the Amazon S3 file. Use the [aws\$1commons.create\$1s3\$1uri](postgresql-s3-export-functions.md#aws_commons.create_s3_uri) function to create the structure.
```
psql=> SELECT aws_commons.create_s3_uri(
'amzn-s3-demo-bucket',
'sample-filepath',
'us-west-2'
) AS s3_uri_1 \gset
```
Although the parameters vary for the following two `aws_s3.query_export_to_s3` function calls, the results are the same for these examples. All rows of the `sample_table` table are exported into a bucket called *amzn-s3-demo-bucket*.
```
psql=> SELECT * FROM aws_s3.query_export_to_s3('SELECT * FROM sample_table', :'s3_uri_1');
psql=> SELECT * FROM aws_s3.query_export_to_s3('SELECT * FROM sample_table', :'s3_uri_1', options :='format text');
```
The parameters are described as follows:
+ `'SELECT * FROM sample_table'` – The first parameter is a required text string containing an SQL query. The PostgreSQL engine runs this query. The results of the query are copied to the S3 bucket identified in other parameters.
+ `:'s3_uri_1'` – This parameter is a structure that identifies the Amazon S3 file. This example uses a variable to identify the previously created structure. You can instead create the structure by including the `aws_commons.create_s3_uri` function call inline within the `aws_s3.query_export_to_s3` function call as follows.
```
SELECT * from aws_s3.query_export_to_s3('select * from sample_table',
aws_commons.create_s3_uri('amzn-s3-demo-bucket', 'sample-filepath', 'us-west-2')
);
```
+ `options :='format text'` – The `options` parameter is an optional text string containing PostgreSQL `COPY` arguments. The copy process uses the arguments and format of the [PostgreSQL COPY](https://www.postgresql.org/docs/current/sql-copy.html) command.
If the file specified doesn't exist in the Amazon S3 bucket, it's created. If the file already exists, it's overwritten. The syntax for accessing the exported data in Amazon S3 is the following.
```
s3-region://bucket-name[/path-prefix]/file-prefix
```
Larger exports are stored in multiple files, each with a maximum size of approximately 6 GB. The additional file names have the same file prefix but with `_partXX` appended. The `XX` represents 2, then 3, and so on. For example, suppose that you specify the path where you store data files as the following.
```
s3-us-west-2://amzn-s3-demo-bucket/my-prefix
```
If the export has to create three data files, the Amazon S3 bucket contains the following data files.
```
s3-us-west-2://amzn-s3-demo-bucket/my-prefix
s3-us-west-2://amzn-s3-demo-bucket/my-prefix_part2
s3-us-west-2://amzn-s3-demo-bucket/my-prefix_part3
```
For the full reference for this function and additional ways to call it, see [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3). For more about accessing files in Amazon S3, see [View an object](https://docs.aws.amazon.com/AmazonS3/latest/userguide/OpeningAnObject.html) in the *Amazon Simple Storage Service User Guide*.
## Exporting to a CSV file that uses a custom delimiter
The following example shows how to call the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function to export data to a file that uses a custom delimiter. The example uses arguments of the [PostgreSQL COPY](https://www.postgresql.org/docs/current/sql-copy.html) command to specify the comma-separated value (CSV) format and a colon (:) delimiter.
```
SELECT * from aws_s3.query_export_to_s3('select * from basic_test', :'s3_uri_1', options :='format csv, delimiter $$:$$');
```
## Exporting to a binary file with encoding
The following example shows how to call the [aws\$1s3.query\$1export\$1to\$1s3](postgresql-s3-export-functions.md#aws_s3.export_query_to_s3) function to export data to a binary file that has Windows-1253 encoding.
```
SELECT * from aws_s3.query_export_to_s3('select * from basic_test', :'s3_uri_1', options :='format binary, encoding WIN1253');
```
# Function reference
**Topics**
+ [
## aws\$1s3.query\$1export\$1to\$1s3
](#aws_s3.export_query_to_s3)
+ [
## aws\$1commons.create\$1s3\$1uri
](#aws_commons.create_s3_uri)
## aws\$1s3.query\$1export\$1to\$1s3
Exports a PostgreSQL query result to an Amazon S3 bucket. The `aws_s3` extension provides the `aws_s3.query_export_to_s3` function.
The two required parameters are `query` and `s3_info`. These define the query to be exported and identify the Amazon S3 bucket to export to. An optional parameter called `options` provides for defining various export parameters. For examples of using the `aws_s3.query_export_to_s3` function, see [Exporting query data using the aws\$1s3.query\$1export\$1to\$1s3 function](postgresql-s3-export-examples.md).
**Syntax**
```
aws_s3.query_export_to_s3(
query text,
s3_info aws_commons._s3_uri_1,
options text,
kms_key text
)
```Input parameters
*query*
A required text string containing an SQL query that the PostgreSQL engine runs. The results of this query are copied to an S3 bucket identified in the `s3_info` parameter.
*s3\$1info*
An `aws_commons._s3_uri_1` composite type containing the following information about the S3 object:
+ `bucket` – The name of the Amazon S3 bucket to contain the file.
+ `file_path` – The Amazon S3 file name and path.
+ `region` – The AWS Region that the bucket is in. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
Currently, this value must be the same AWS Region as that of the exporting DB instance. The default is the AWS Region of the exporting DB instance.
To create an `aws_commons._s3_uri_1` composite structure, see the [aws\$1commons.create\$1s3\$1uri](#aws_commons.create_s3_uri) function.
*options*
An optional text string containing arguments for the PostgreSQL `COPY` command. These arguments specify how the data is to be copied when exported. For more details, see the [PostgreSQL COPY documentation](https://www.postgresql.org/docs/current/sql-copy.html).
*kms\$1key text*
An optional text string containing the customer managed KMS key of the S3 bucket to export the data to.
### Alternate input parameters
To help with testing, you can use an expanded set of parameters instead of the `s3_info` parameter. Following are additional syntax variations for the `aws_s3.query_export_to_s3` function.
Instead of using the `s3_info` parameter to identify an Amazon S3 file, use the combination of the `bucket`, `file_path`, and `region` parameters.
```
aws_s3.query_export_to_s3(
query text,
bucket text,
file_path text,
region text,
options text,
kms_key text
)
```
*query*
A required text string containing an SQL query that the PostgreSQL engine runs. The results of this query are copied to an S3 bucket identified in the `s3_info` parameter.
*bucket*
A required text string containing the name of the Amazon S3 bucket that contains the file.
*file\$1path*
A required text string containing the Amazon S3 file name including the path of the file.
*region*
An optional text string containing the AWS Region that the bucket is in. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
Currently, this value must be the same AWS Region as that of the exporting DB instance. The default is the AWS Region of the exporting DB instance.
*options*
An optional text string containing arguments for the PostgreSQL `COPY` command. These arguments specify how the data is to be copied when exported. For more details, see the [PostgreSQL COPY documentation](https://www.postgresql.org/docs/current/sql-copy.html).
*kms\$1key text*
An optional text string containing the customer managed KMS key of the S3 bucket to export the data to.
### Output parameters
```
aws_s3.query_export_to_s3(
OUT rows_uploaded bigint,
OUT files_uploaded bigint,
OUT bytes_uploaded bigint
)
```
*rows\$1uploaded*
The number of table rows that were successfully uploaded to Amazon S3 for the given query.
*files\$1uploaded*
The number of files uploaded to Amazon S3. Files are created in sizes of approximately 6 GB. Each additional file created has `_partXX` appended to the name. The `XX` represents 2, then 3, and so on as needed.
*bytes\$1uploaded*
The total number of bytes uploaded to Amazon S3.
### Examples
```
psql=> SELECT * from aws_s3.query_export_to_s3('select * from sample_table', 'amzn-s3-demo-bucket', 'sample-filepath');
psql=> SELECT * from aws_s3.query_export_to_s3('select * from sample_table', 'amzn-s3-demo-bucket', 'sample-filepath','us-west-2');
psql=> SELECT * from aws_s3.query_export_to_s3('select * from sample_table', 'amzn-s3-demo-bucket', 'sample-filepath','us-west-2','format text');
```
## aws\$1commons.create\$1s3\$1uri
Creates an `aws_commons._s3_uri_1` structure to hold Amazon S3 file information. You use the results of the `aws_commons.create_s3_uri` function in the `s3_info` parameter of the [aws\$1s3.query\$1export\$1to\$1s3](#aws_s3.export_query_to_s3) function. For an example of using the `aws_commons.create_s3_uri` function, see [Specifying the Amazon S3 file path to export to](postgresql-s3-export.md#postgresql-s3-export-file).
**Syntax**
```
aws_commons.create_s3_uri(
bucket text,
file_path text,
region text
)
```Input parameters
*bucket*
A required text string containing the Amazon S3 bucket name for the file.
*file\$1path*
A required text string containing the Amazon S3 file name including the path of the file.
*region*
A required text string containing the AWS Region that the file is in. For a listing of AWS Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
# Troubleshooting access to Amazon S3
If you encounter connection problems when attempting to export data to Amazon S3, first confirm that the outbound access rules for the VPC security group associated with your DB instance permit network connectivity. Specifically, the security group must have a rule that allows the DB instance to send TCP traffic to port 443 and to any IPv4 addresses (0.0.0.0/0). For more information, see [Provide access to your DB instance in your VPC by creating a security group](CHAP_SettingUp.md#CHAP_SettingUp.SecurityGroup).
See also the following for recommendations:
+ [Troubleshooting Amazon RDS identity and access](security_iam_troubleshoot.md)
+ [Troubleshooting Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/troubleshooting.html) in the *Amazon Simple Storage Service User Guide*
+ [Troubleshooting Amazon S3 and IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_iam-s3.html) in the *IAM User Guide*
# Invoking an AWS Lambda function from an RDS for PostgreSQL DB instance
AWS Lambda is an event-driven compute service that lets you run code without provisioning or managing servers. It's available for use with many AWS services, including RDS for PostgreSQL. For example, you can use Lambda functions to process event notifications from a database, or to load data from files whenever a new file is uploaded to Amazon S3. To learn more about Lambda, see [What is AWS Lambda?](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) in the *AWS Lambda Developer Guide.*
**Note**
Invoking an AWS Lambda function is supported in these RDS for PostgreSQL versions:
All PostgreSQL 18 versions
All PostgreSQL 17 versions
All PostgreSQL 16 versions
All PostgreSQL 15 versions
PostgreSQL 14.1 and higher minor versions
PostgreSQL 13.2 and higher minor versions
PostgreSQL 12.6 and higher minor versions
Setting up RDS for PostgreSQL to work with Lambda functions is a multi-step process involving AWS Lambda, IAM, your VPC, and your RDS for PostgreSQL DB instance. Following, you can find summaries of the necessary steps.
For more information about Lambda functions, see [Getting started with Lambda](https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html) and [AWS Lambda foundations](https://docs.aws.amazon.com/lambda/latest/dg/lambda-foundation.html) in the *AWS Lambda Developer Guide*.
**Topics**
+ [
## Step 1: Configure your RDS for PostgreSQL DB instance for outbound connections to AWS Lambda
](#PostgreSQL-Lambda-network)
+ [
## Step 2: Configure IAM for your RDS for PostgreSQL DB instance and AWS Lambda
](#PostgreSQL-Lambda-access)
+ [
## Step 3: Install the `aws_lambda` extension for an RDS for PostgreSQL DB instance
](#PostgreSQL-Lambda-install-extension)
+ [
## Step 4: Use Lambda helper functions with your RDS for PostgreSQL DB instance (Optional)
](#PostgreSQL-Lambda-specify-function)
+ [
## Step 5: Invoke a Lambda function from your RDS for PostgreSQL DB instance
](#PostgreSQL-Lambda-invoke)
+ [
## Step 6: Grant other users permission to invoke Lambda functions
](#PostgreSQL-Lambda-grant-users-permissions)
+ [
# Examples: Invoking Lambda functions from your RDS for PostgreSQL DB instance
](PostgreSQL-Lambda-examples.md)
+ [
# Lambda function error messages
](PostgreSQL-Lambda-errors.md)
+ [
# AWS Lambda function and parameter reference
](PostgreSQL-Lambda-functions.md)
## Step 1: Configure your RDS for PostgreSQL DB instance for outbound connections to AWS Lambda
Lambda functions always run inside an Amazon VPC that's owned by the AWS Lambda service. Lambda applies network access and security rules to this VPC and it maintains and monitors the VPC automatically. Your RDS for PostgreSQL DB instance sends network traffic to the Lambda service's VPC. How you configure this depends on whether your DB instance is public or private.
+ **Public RDS for PostgreSQL DB instance** – A DB instance is public if it's located in a public subnet on your VPC, and if the instance's "PubliclyAccessible" property is `true`. To find the value of this property, you can use the [describe-db-instances](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-instances.html) AWS CLI command. Or, you can use the AWS Management Console to open the **Connectivity & security** tab and check that **Publicly accessible** is **Yes**. To verify that the instance is in the public subnet of your VPC, you can use the AWS Management Console or the AWS CLI.
To set up access to Lambda, you use the AWS Management Console or the AWS CLI to create an outbound rule on your VPC's security group. The outbound rule specifies that TCP can use port 443 to send packets to any IPv4 addresses (0.0.0.0/0).
+ **Private RDS for PostgreSQL DB instance** – In this case, the instance's "PubliclyAccessible" property is `false` or it's in a private subnet. To allow the instance to work with Lambda, you can use a Network Address Translation) NAT gateway. For more information, see [NAT gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html). Or, you can configure your VPC with a VPC endpoint for Lambda. For more information, see [VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html) in the *Amazon VPC User Guide*. The endpoint responds to calls made by your RDS for PostgreSQL DB instance to your Lambda functions. The VPC endpoint uses its own private DNS resolution. RDS for PostgreSQL can't use the Lambda VPC endpoint until you change the value of the `rds.custom_dns_resolution` from its default value of 0 (not enabled) to 1. To do so:
+ Create a custom DB parameter group.
+ Change the value of the `rds.custom_dns_resolution` parameter from its default of `0` to `1`.
+ Modify your DB instance to use your custom DB parameter group.
+ Reboot the instance to have the modified parameter take effect.
Your VPC can now interact with the AWS Lambda VPC at the network level. Next, you configure the permissions using IAM.
## Step 2: Configure IAM for your RDS for PostgreSQL DB instance and AWS Lambda
Invoking Lambda functions from your RDS for PostgreSQL DB instance requires certain privileges. To configure the necessary privileges, we recommend that you create an IAM policy that allows invoking Lambda functions, assign that policy to a role, and then apply the role to your DB instance. This approach gives the DB instance privileges to invoke the specified Lambda function on your behalf. The following steps show you how to do this using the AWS CLI.
**To configure IAM permissions for using your Amazon RDS instance with Lambda**
1. Use the [create-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/create-policy.html) AWS CLI command to create an IAM policy that allows your RDS for PostgreSQL DB instance to invoke the specified Lambda function. (The statement ID (Sid) is an optional description for your policy statement and has no effect on usage.) This policy gives your DB instance the minimum permissions needed to invoke the specified Lambda function.
```
aws iam create-policy --policy-name rds-lambda-policy --policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAccessToExampleFunction",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:aws-region:444455556666:function:my-function"
}
]
}'
```
Alternatively, you can use the predefined `AWSLambdaRole` policy that allows you to invoke any of your Lambda functions. For more information, see [Identity-based IAM policies for Lambda](https://docs.aws.amazon.com/lambda/latest/dg/access-control-identity-based.html#access-policy-examples-aws-managed)
1. Use the [create-role](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/create-role.html) AWS CLI command to create an IAM role that the policy can assume at runtime.
```
aws iam create-role --role-name rds-lambda-role --assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "rds.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}'
```
1. Apply the policy to the role by using the [attach-role-policy](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iam/attach-role-policy.html) AWS CLI command.
```
aws iam attach-role-policy \
--policy-arn arn:aws:iam::444455556666:policy/rds-lambda-policy \
--role-name rds-lambda-role --region aws-region
```
1. Apply the role to your RDS for PostgreSQL DB instance by using the [add-role-to-db-instance](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/rds/add-role-to-db-instance.html) AWS CLI command. This last step allows your DB instance's database users to invoke Lambda functions.
```
aws rds add-role-to-db-instance \
--db-instance-identifier my-instance-name \
--feature-name Lambda \
--role-arn arn:aws:iam::444455556666:role/rds-lambda-role \
--region aws-region
```
With the VPC and the IAM configurations complete, you can now install the `aws_lambda` extension. (Note that you can install the extension at any time, but until you set up the correct VPC support and IAM privileges, the `aws_lambda` extension adds nothing to your RDS for PostgreSQL DB instance's capabilities.)
## Step 3: Install the `aws_lambda` extension for an RDS for PostgreSQL DB instance
To use AWS Lambda with your RDS for PostgreSQL DB instance, add the `aws_lambda` PostgreSQL extension to your RDS for PostgreSQL DB instance. This extension provides your RDS for PostgreSQL DB instance with the ability to call Lambda functions from PostgreSQL.
**To install the `aws_lambda` extension in your RDS for PostgreSQL DB instance**
Use the PostgreSQL `psql` command-line or the pgAdmin tool to connect to your RDS for PostgreSQL DB instance.
1. Connect to your RDS for PostgreSQL DB instance as a user with `rds_superuser` privileges. The default `postgres` user is shown in the example.
```
psql -h instance.444455556666.aws-region.rds.amazonaws.com -U postgres -p 5432
```
1. Install the `aws_lambda` extension. The `aws_commons` extension is also required. It provides helper functions to `aws_lambda` and many other Aurora extensions for PostgreSQL. If it's not already on your RDS for PostgreSQLDB instance, it's installed with `aws_lambda` as shown following.
```
CREATE EXTENSION IF NOT EXISTS aws_lambda CASCADE;
NOTICE: installing required extension "aws_commons"
CREATE EXTENSION
```
The `aws_lambda` extension is installed in your DB instance. You can now create convenience structures for invoking your Lambda functions.
## Step 4: Use Lambda helper functions with your RDS for PostgreSQL DB instance (Optional)
You can use the helper functions in the `aws_commons` extension to prepare entities that you can more easily invoke from PostgreSQL. To do this, you need to have the following information about your Lambda functions:
+ **Function name** – The name, Amazon Resource Name (ARN), version, or alias of the Lambda function. The IAM policy created in [Step 2: Configure IAM for your instance and Lambda](#PostgreSQL-Lambda-access) requires the ARN, so we recommend that you use your function's ARN.
+ **AWS Region** – (Optional) The AWS Region where the Lambda function is located if it's not in the same Region as your RDS for PostgreSQL DB instance.
To hold the Lambda function name information, you use the [aws\$1commons.create\$1lambda\$1function\$1arn](PostgreSQL-Lambda-functions.md#aws_commons.create_lambda_function_arn) function. This helper function creates an `aws_commons._lambda_function_arn_1` composite structure with the details needed by the invoke function. Following, you can find three alternative approaches to setting up this composite structure.
```
SELECT aws_commons.create_lambda_function_arn(
'my-function',
'aws-region'
) AS aws_lambda_arn_1 \gset
```
```
SELECT aws_commons.create_lambda_function_arn(
'111122223333:function:my-function',
'aws-region'
) AS lambda_partial_arn_1 \gset
```
```
SELECT aws_commons.create_lambda_function_arn(
'arn:aws:lambda:aws-region:111122223333:function:my-function'
) AS lambda_arn_1 \gset
```
Any of these values can be used in calls to the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) function. For examples, see [Step 5: Invoke a Lambda function from your RDS for PostgreSQL DB instance](#PostgreSQL-Lambda-invoke).
## Step 5: Invoke a Lambda function from your RDS for PostgreSQL DB instance
The `aws_lambda.invoke` function behaves synchronously or asynchronously, depending on the `invocation_type`. The two alternatives for this parameter are `RequestResponse` (the default) and `Event`, as follows.
+ **`RequestResponse`** – This invocation type is *synchronous*. It's the default behavior when the call is made without specifying an invocation type. The response payload includes the results of the `aws_lambda.invoke` function. Use this invocation type when your workflow requires receiving results from the Lambda function before proceeding.
+ **`Event`** – This invocation type is *asynchronous*. The response doesn't include a payload containing results. Use this invocation type when your workflow doesn't need a result from the Lambda function to continue processing.
As a simple test of your setup, you can connect to your DB instance using `psql` and invoke an example function from the command line. Suppose that you have one of the basic functions set up on your Lambda service, such as the simple Python function shown in the following screenshot.
![\[Example Lambda function shown in the AWS CLI for AWS Lambda\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/lambda_simple_function.png)
**To invoke an example function**
1. Connect to your DB instance using `psql` or pgAdmin.
```
psql -h instance.444455556666.aws-region.rds.amazonaws.com -U postgres -p 5432
```
1. Invoke the function using its ARN.
```
SELECT * from aws_lambda.invoke(aws_commons.create_lambda_function_arn('arn:aws:lambda:aws-region:444455556666:function:simple', 'us-west-1'), '{"body": "Hello from Postgres!"}'::json );
```
The response looks as follows.
```
status_code | payload | executed_version | log_result
-------------+-------------------------------------------------------+------------------+------------
200 | {"statusCode": 200, "body": "\"Hello from Lambda!\""} | $LATEST |
(1 row)
```
If your invocation attempt doesn't succeed, see [Lambda function error messages](PostgreSQL-Lambda-errors.md).
## Step 6: Grant other users permission to invoke Lambda functions
At this point in the procedures, only you as `rds_superuser` can invoke your Lambda functions. To allow other users to invoke any functions that you create, you need to grant them permission.
**To grant others permission to invoke Lambda functions**
1. Connect to your DB instance using `psql` or pgAdmin.
```
psql -h instance.444455556666.aws-region.rds.amazonaws.com -U postgres -p 5432
```
1. Run the following SQL commands:
```
postgres=> GRANT USAGE ON SCHEMA aws_lambda TO db_username;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA aws_lambda TO db_username;
```
# Examples: Invoking Lambda functions from your RDS for PostgreSQL DB instance
Following, you can find several examples of calling the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) function. Most of the examples use the composite structure `aws_lambda_arn_1` that you create in [Step 4: Use Lambda helper functions with your RDS for PostgreSQL DB instance (Optional)](PostgreSQL-Lambda.md#PostgreSQL-Lambda-specify-function) to simplify passing the function details. For an example of asynchronous invocation, see [Example: Asynchronous (Event) invocation of Lambda functions](#PostgreSQL-Lambda-Event). All the other examples listed use synchronous invocation.
To learn more about Lambda invocation types, see [Invoking Lambda functions](https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html) in the *AWS Lambda Developer Guide*. For more information about `aws_lambda_arn_1`, see [aws\$1commons.create\$1lambda\$1function\$1arn](PostgreSQL-Lambda-functions.md#aws_commons.create_lambda_function_arn).
**Topics**
+ [
## Example: Synchronous (RequestResponse) invocation of Lambda functions
](#PostgreSQL-Lambda-RequestResponse)
+ [
## Example: Asynchronous (Event) invocation of Lambda functions
](#PostgreSQL-Lambda-Event)
+ [
## Example: Capturing the Lambda execution log in a function response
](#PostgreSQL-Lambda-log-response)
+ [
## Example: Including client context in a Lambda function
](#PostgreSQL-Lambda-client-context)
+ [
## Example: Invoking a specific version of a Lambda function
](#PostgreSQL-Lambda-function-version)
## Example: Synchronous (RequestResponse) invocation of Lambda functions
Following are two examples of a synchronous Lambda function invocation. The results of these `aws_lambda.invoke` function calls are the same.
```
SELECT * FROM aws_lambda.invoke('aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json);
```
```
SELECT * FROM aws_lambda.invoke('aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json, 'RequestResponse');
```
The parameters are described as follows:
+ `:'aws_lambda_arn_1'` – This parameter identifies the composite structure created in [Step 4: Use Lambda helper functions with your RDS for PostgreSQL DB instance (Optional)](PostgreSQL-Lambda.md#PostgreSQL-Lambda-specify-function), with the `aws_commons.create_lambda_function_arn` helper function. You can also create this structure inline within your `aws_lambda.invoke` call as follows.
```
SELECT * FROM aws_lambda.invoke(aws_commons.create_lambda_function_arn('my-function', 'aws-region'),
'{"body": "Hello from Postgres!"}'::json
);
```
+ `'{"body": "Hello from PostgreSQL!"}'::json` – The JSON payload to pass to the Lambda function.
+ `'RequestResponse'` – The Lambda invocation type.
## Example: Asynchronous (Event) invocation of Lambda functions
Following is an example of an asynchronous Lambda function invocation. The `Event` invocation type schedules the Lambda function invocation with the specified input payload and returns immediately. Use the `Event` invocation type in certain workflows that don't depend on the results of the Lambda function.
```
SELECT * FROM aws_lambda.invoke('aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json, 'Event');
```
## Example: Capturing the Lambda execution log in a function response
You can include the last 4 KB of the execution log in the function response by using the `log_type` parameter in your `aws_lambda.invoke` function call. By default, this parameter is set to `None`, but you can specify `Tail` to capture the results of the Lambda execution log in the response, as shown following.
```
SELECT *, select convert_from(decode(log_result, 'base64'), 'utf-8') as log FROM aws_lambda.invoke(:'aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json, 'RequestResponse', 'Tail');
```
Set the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) function's `log_type` parameter to `Tail` to include the execution log in the response. The default value for the `log_type` parameter is `None`.
The `log_result` that's returned is a `base64` encoded string. You can decode the contents using a combination of the `decode` and `convert_from` PostgreSQL functions.
For more information about `log_type`, see [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke).
## Example: Including client context in a Lambda function
The `aws_lambda.invoke` function has a `context` parameter that you can use to pass information separate from the payload, as shown following.
```
SELECT *, convert_from(decode(log_result, 'base64'), 'utf-8') as log FROM aws_lambda.invoke(:'aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json, 'RequestResponse', 'Tail');
```
To include client context, use a JSON object for the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) function's `context` parameter.
For more information about the `context` parameter, see the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) reference.
## Example: Invoking a specific version of a Lambda function
You can specify a particular version of a Lambda function by including the `qualifier` parameter with the `aws_lambda.invoke` call. Following, you can find an example that does this using `'custom_version'` as an alias for the version.
```
SELECT * FROM aws_lambda.invoke('aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json, 'RequestResponse', 'None', NULL, 'custom_version');
```
You can also supply a Lambda function qualifier with the function name details instead, as follows.
```
SELECT * FROM aws_lambda.invoke(aws_commons.create_lambda_function_arn('my-function:custom_version', 'us-west-2'),
'{"body": "Hello from Postgres!"}'::json);
```
For more information about `qualifier` and other parameters, see the [aws\$1lambda.invoke](PostgreSQL-Lambda-functions.md#aws_lambda.invoke) reference.
# Lambda function error messages
In the following list you can find information about error messages, with possible causes and solutions.
+ **VPC configuration issues**
VPC configuration issues can raise the following error messages when trying to connect:
```
ERROR: invoke API failed
DETAIL: AWS Lambda client returned 'Unable to connect to endpoint'.
CONTEXT: SQL function "invoke" statement 1
```
A common cause for this error is improperly configured VPC security group. Make sure you have an outbound rule for TCP open on port 443 of your VPC security group so that your VPC can connect to the Lambda VPC.
If your DB instance is private, check the private DNS setup for your VPC. Make sure that you set the `rds.custom_dns_resolution` parameter to 1 and setup AWS PrivateLink as outlined in [Step 1: Configure your RDS for PostgreSQL DB instance for outbound connections to AWS Lambda](PostgreSQL-Lambda.md#PostgreSQL-Lambda-network). For more information, see [Interface VPC endpoints (AWS PrivateLink)](https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-interface.html#vpce-private-dns).
+ **Lack of permissions needed to invoke Lambda functions**
If you see either of the following error messages, the user (role) invoking the function doesn't have proper permissions.
```
ERROR: permission denied for schema aws_lambda
```
```
ERROR: permission denied for function invoke
```
A user (role) must be given specific grants to invoke Lambda functions. For more information, see [Step 6: Grant other users permission to invoke Lambda functions](PostgreSQL-Lambda.md#PostgreSQL-Lambda-grant-users-permissions).
+ **Improper handling of errors in your Lambda functions**
If a Lambda function throws an exception during request processing, `aws_lambda.invoke` fails with a PostgreSQL error such as the following.
```
SELECT * FROM aws_lambda.invoke('aws_lambda_arn_1', '{"body": "Hello from Postgres!"}'::json);
ERROR: lambda invocation failed
DETAIL: "arn:aws:lambda:us-west-2:555555555555:function:my-function" returned error "Unhandled", details: "".
```
Be sure to handle errors in your Lambda functions or in your PostgreSQL application.
# AWS Lambda function and parameter reference
Following is the reference for the functions and parameters to use for invoking Lambda with RDS for PostgreSQL.
**Topics**
+ [
## aws\$1lambda.invoke
](#aws_lambda.invoke)
+ [
## aws\$1commons.create\$1lambda\$1function\$1arn
](#aws_commons.create_lambda_function_arn)
+ [
## aws\$1lambda parameters
](#aws_lambda.parameters)
## aws\$1lambda.invoke
Runs a Lambda function for an RDS for PostgreSQL DB instance.
For more details about invoking Lambda functions, see also [Invoke](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html) in the *AWS Lambda Developer Guide.*
**Syntax**
------
#### [ JSON ]
```
aws_lambda.invoke(
IN function_name TEXT,
IN payload JSON,
IN region TEXT DEFAULT NULL,
IN invocation_type TEXT DEFAULT 'RequestResponse',
IN log_type TEXT DEFAULT 'None',
IN context JSON DEFAULT NULL,
IN qualifier VARCHAR(128) DEFAULT NULL,
OUT status_code INT,
OUT payload JSON,
OUT executed_version TEXT,
OUT log_result TEXT)
```
```
aws_lambda.invoke(
IN function_name aws_commons._lambda_function_arn_1,
IN payload JSON,
IN invocation_type TEXT DEFAULT 'RequestResponse',
IN log_type TEXT DEFAULT 'None',
IN context JSON DEFAULT NULL,
IN qualifier VARCHAR(128) DEFAULT NULL,
OUT status_code INT,
OUT payload JSON,
OUT executed_version TEXT,
OUT log_result TEXT)
```
------
#### [ JSONB ]
```
aws_lambda.invoke(
IN function_name TEXT,
IN payload JSONB,
IN region TEXT DEFAULT NULL,
IN invocation_type TEXT DEFAULT 'RequestResponse',
IN log_type TEXT DEFAULT 'None',
IN context JSONB DEFAULT NULL,
IN qualifier VARCHAR(128) DEFAULT NULL,
OUT status_code INT,
OUT payload JSONB,
OUT executed_version TEXT,
OUT log_result TEXT)
```
```
aws_lambda.invoke(
IN function_name aws_commons._lambda_function_arn_1,
IN payload JSONB,
IN invocation_type TEXT DEFAULT 'RequestResponse',
IN log_type TEXT DEFAULT 'None',
IN context JSONB DEFAULT NULL,
IN qualifier VARCHAR(128) DEFAULT NULL,
OUT status_code INT,
OUT payload JSONB,
OUT executed_version TEXT,
OUT log_result TEXT
)
```
------Input parameters
**function\$1name**
The identifying name of the Lambda function. The value can be the function name, an ARN, or a partial ARN. For a listing of possible formats, see [ Lambda function name formats](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html#API_Invoke_RequestParameters) in the *AWS Lambda Developer Guide.*
*payload*
The input for the Lambda function. The format can be JSON or JSONB. For more information, see [JSON Types](https://www.postgresql.org/docs/current/datatype-json.html) in the PostgreSQL documentation.
*region*
(Optional) The Lambda Region for the function. By default, RDS resolves the AWS Region from the full ARN in the `function_name` or it uses the RDS for PostgreSQL DB instance Region. If this Region value conflicts with the one provided in the `function_name` ARN, an error is raised.
*invocation\$1type*
The invocation type of the Lambda function. The value is case-sensitive. Possible values include the following:
+ `RequestResponse` – The default. This type of invocation for a Lambda function is synchronous and returns a response payload in the result. Use the `RequestResponse` invocation type when your workflow depends on receiving the Lambda function result immediately.
+ `Event` – This type of invocation for a Lambda function is asynchronous and returns immediately without a returned payload. Use the `Event` invocation type when you don't need results of the Lambda function before your workflow moves on.
+ `DryRun` – This type of invocation tests access without running the Lambda function.
*log\$1type*
The type of Lambda log to return in the `log_result` output parameter. The value is case-sensitive. Possible values include the following:
+ Tail – The returned `log_result` output parameter will include the last 4 KB of the execution log.
+ None – No Lambda log information is returned.
*context*
Client context in JSON or JSONB format. Fields to use include than `custom` and `env`.
*qualifier*
A qualifier that identifies a Lambda function's version to be invoked. If this value conflicts with one provided in the `function_name` ARN, an error is raised.Output parameters
*status\$1code*
An HTTP status response code. For more information, see [Lambda Invoke response elements](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html#API_Invoke_ResponseElements) in the *AWS Lambda Developer Guide.*
*payload*
The information returned from the Lambda function that ran. The format is in JSON or JSONB.
*executed\$1version*
The version of the Lambda function that ran.
*log\$1result*
The execution log information returned if the `log_type` value is `Tail` when the Lambda function was invoked. The result contains the last 4 KB of the execution log encoded in Base64.
## aws\$1commons.create\$1lambda\$1function\$1arn
Creates an `aws_commons._lambda_function_arn_1` structure to hold Lambda function name information. You can use the results of the `aws_commons.create_lambda_function_arn` function in the `function_name` parameter of the aws\$1lambda.invoke [aws\$1lambda.invoke](#aws_lambda.invoke) function.
**Syntax**
```
aws_commons.create_lambda_function_arn(
function_name TEXT,
region TEXT DEFAULT NULL
)
RETURNS aws_commons._lambda_function_arn_1
```Input parameters
*function\$1name*
A required text string containing the Lambda function name. The value can be a function name, a partial ARN, or a full ARN.
*region*
An optional text string containing the AWS Region that the Lambda function is in. For a listing of Region names and associated values, see [Regions, Availability Zones, and Local Zones](Concepts.RegionsAndAvailabilityZones.md).
## aws\$1lambda parameters
In this table, you can find parameters associated with the `aws_lambda` function.
| Parameter | Description |
| --- | --- |
| `aws_lambda.connect_timeout_ms` | This is a dynamic parameter and it sets the maximum wait time while connecting to AWS Lambda. The default values is `1000`. Allowed values for this parameter are 1 - 900000. |
| `aws_lambda.request_timeout_ms` | This is a dynamic parameter and it sets the maximum wait time while waiting for response from AWS Lambda. The default values is `3000`. Allowed values for this parameter are 1 - 900000. |
| `aws_lambda.endpoint_override` | Specifies the endpoint that can be used to connect to AWS Lambda. An empty string selects the default AWS Lambda endpoint for the region. You must restart the database for this static parameter change to take effect. |
# Common DBA tasks for Amazon RDS for PostgreSQL
Database administrators (DBAs) perform a variety of tasks when administering an Amazon RDS for PostgreSQL DB instance. If you're a DBA already familiar with PostgreSQL, you need to be aware of some of the important differences between running PostgreSQL on your hardware and RDS for PostgreSQL. For example, because it's a managed service, Amazon RDS doesn't allow shell access to your DB instances. That means that you don't have direct access to `pg_hba.conf` and other configuration files. For RDS for PostgreSQL, changes that are typically made to the PostgreSQL configuration file of an on-premises instance are made to a custom DB parameter group associated with the RDS for PostgreSQL DB instance. For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
You also can't access log files in the same way that you do with an on-premises PostgreSQL instance. To learn more about logging, see [ RDS for PostgreSQL database log files](USER_LogAccess.Concepts.PostgreSQL.md).
As another example, you don't have access to the PostgreSQL `superuser` account. On RDS for PostgreSQL, the `rds_superuser` role is the most highly privileged role, and it's granted to `postgres` at set up time. Whether you're familiar with using PostgreSQL on-premises or completely new to RDS for PostgreSQL, we recommend that you understand the `rds_superuser` role, and how to work with roles, users, groups, and permissions. For more information, see [Understanding PostgreSQL roles and permissions](Appendix.PostgreSQL.CommonDBATasks.Roles.md).
Following are some common DBA tasks for RDS for PostgreSQL.
**Topics**
+ [
# Collations supported in RDS for PostgreSQL
](PostgreSQL-Collations.md)
+ [
# Understanding PostgreSQL roles and permissions
](Appendix.PostgreSQL.CommonDBATasks.Roles.md)
+ [
# Dead connection handling in PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.DeadConnectionHandling.md)
+ [
# Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md)
+ [
# Managing high object counts in Amazon RDS for PostgreSQL
](PostgreSQL.HighObjectCount.md)
+ [
# Managing TOAST OID contention in Amazon RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.TOAST_OID.md)
+ [
## Working with logging mechanisms supported by RDS for PostgreSQL
](#Appendix.PostgreSQL.CommonDBATasks.Auditing)
+ [
# Managing temporary files with PostgreSQL
](PostgreSQL.ManagingTempFiles.md)
+ [
## Using pgBadger for log analysis with PostgreSQL
](#Appendix.PostgreSQL.CommonDBATasks.Badger)
+ [
## Using PGSnapper for monitoring PostgreSQL
](#Appendix.PostgreSQL.CommonDBATasks.Snapper)
+ [
# Managing custom casts in RDS for PostgreSQL
](PostgreSQL.CustomCasts.md)
+ [
# Best Practices for Parallel Queries in RDS for PostgreSQL
](PostgreSQL.ParallelQueries.md)
+ [
# Working with parameters on your RDS for PostgreSQL DB instance
](Appendix.PostgreSQL.CommonDBATasks.Parameters.md)
# Collations supported in RDS for PostgreSQL
Collations are set of rules that determine how character strings stored in the database are sorted and compared. Collations play a fundamental role in the computer system and are included as part of the operating system. Collations change over time when new characters are added to languages or when ordering rules change.
Collation libraries define specific rules and algorithms for a collation. The most popular collation libraries used within PostgreSQL are GNU C (glibc) and Internationalization components for Unicode (ICU). By default, RDS for PostgreSQL uses the glibc collation that includes unicode character sort orders for multi-byte character sequences.
When you create a new DB instance in RDS for PostgreSQL , it checks the operating system for the available collation. The PostgreSQL parameters of the `CREATE DATABASE` command `LC_COLLATE` and `LC_CTYPE` are used to specify a collation, which stands as the default collation in that database. Alternatively, you can also use the `LOCALE` parameter in `CREATE DATABASE` to set these parameters. This determines the default collation for character strings in the database and the rules for classifying characters as letters, numbers, or symbols. You can also choose a collation to use on a column, index, or on a query.
RDS for PostgreSQL depends on the glibc library in the operating system for collation support. RDS for PostgreSQL instance is periodically updated with the latest versions of the operating system. These updates sometimes include a newer version of the glibc library. Rarely, newer versions of glibc change the sort order or collation of some characters, which can cause the data to sort differently or produce invalid index entries. If you discover sort order issues for collation during an update, you might need to rebuild the indexes.
To reduce the possible impacts of the glibc updates, RDS for PostgreSQL now includes an independent default collation library. This collation library is available in RDS for PostgreSQL 14.6, 13.9, 12.13, 11.18, 10.23 and newer minor version releases. It is compatible with glibc 2.26-59.amzn2, and provides sort order stability to prevent incorrect query results.
# Understanding PostgreSQL roles and permissions
When you create an RDS for PostgreSQL DB instance using the AWS Management Console, an administrator account is created at the same time. By default, its name is `postgres`, as shown in the following screenshot:
![\[The default login identity for Credentials in the Create database page is postgres.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/default-login-identity-apg-rpg.png)
You can choose another name rather than accept the default (`postgres`). If you do, the name you choose must start with a letter and be between 1 and 16 alphanumeric characters. For simplicity's sake, we refer to this main user account by its default value (`postgres`) throughout this guide.
If you use the `create-db-instance` AWS CLI rather than the AWS Management Console, you create the name by passing it with the `master-username` parameter in the command. For more information, see [Creating an Amazon RDS DB instance](USER_CreateDBInstance.md).
Whether you use the AWS Management Console, the AWS CLI, or the Amazon RDS API, and whether you use the default `postgres` name or choose a different name, this first database user account is a member of the `rds_superuser` group and has `rds_superuser` privileges.
**Topics**
+ [
# Understanding the rds\$1superuser role
](Appendix.PostgreSQL.CommonDBATasks.Roles.rds_superuser.md)
+ [
# Controlling user access to the PostgreSQL database
](Appendix.PostgreSQL.CommonDBATasks.Access.md)
+ [
# Delegating and controlling user password management
](Appendix.PostgreSQL.CommonDBATasks.RestrictPasswordMgmt.md)
+ [
# Using SCRAM for PostgreSQL password encryption
](PostgreSQL_Password_Encryption_configuration.md)
# Understanding the rds\$1superuser role
In PostgreSQL, a *role* can define a user, a group, or a set of specific permissions granted to a group or user for various objects in the database. PostgreSQL commands to `CREATE USER` and `CREATE GROUP` have been replaced by the more general, `CREATE ROLE` with specific properties to distinguish database users. A database user can be thought of as a role with the LOGIN privilege.
**Note**
The `CREATE USER` and `CREATE GROUP` commands can still be used. For more information, see [Database Roles](https://www.postgresql.org/docs/current/user-manag.html) in the PostgreSQL documentation.
The `postgres` user is the most highly privileged database user on your RDS for PostgreSQL DB instance. It has the characteristics defined by the following `CREATE ROLE` statement.
```
CREATE ROLE postgres WITH LOGIN NOSUPERUSER INHERIT CREATEDB CREATEROLE NOREPLICATION VALID UNTIL 'infinity'
```
The properties `NOSUPERUSER`, `NOREPLICATION`, `INHERIT`, and `VALID UNTIL 'infinity'` are the default options for CREATE ROLE, unless otherwise specified.
By default, `postgres` has privileges granted to the `rds_superuser` role, and permissions to create roles and databases. The `rds_superuser` role allows the `postgres` user to do the following:
+ Add extensions that are available for use with Amazon RDS. For more information, see [Working with PostgreSQL features supported by Amazon RDS for PostgreSQL](PostgreSQL.Concepts.General.FeatureSupport.md)
+ Create roles for users and grant privileges to users. For more information, see [CREATE ROLE](https://www.postgresql.org/docs/current/sql-createrole.html) and [GRANT](https://www.postgresql.org/docs/14/sql-grant.html) in the PostgreSQL documentation.
+ Create databases. For more information, see [CREATE DATABASE](https://www.postgresql.org/docs/14/sql-createdatabase.html) in the PostgreSQL documentation.
+ Grant `rds_superuser` privileges to user roles that don't have these privileges, and revoke privileges as needed. We recommend that you grant this role only to those users who perform superuser tasks. In other words, you can grant this role to database administrators (DBAs) or system administrators.
+ Grant (and revoke) the `rds_replication` role to database users that don't have the `rds_superuser` role.
+ Grant (and revoke) the `rds_password` role to database users that don't have the `rds_superuser` role.
+ Obtain status information about all database connections by using the `pg_stat_activity` view. When needed, `rds_superuser` can stop any connections by using `pg_terminate_backend` or `pg_cancel_backend`.
In the `CREATE ROLE postgres...` statement, you can see that the `postgres` user role specifically disallows PostgreSQL `superuser` permissions. RDS for PostgreSQL is a managed service, so you can't access the host OS, and you can't connect using the PostgreSQL `superuser` account. Many of the tasks that require `superuser` access on a stand-alone PostgreSQL are managed automatically by Amazon RDS.
For more information about granting privileges, see [GRANT](http://www.postgresql.org/docs/current/sql-grant.html) in the PostgreSQL documentation.
The `rds_superuser` role is one of several *predefined* roles in an RDS for PostgreSQL DB instance.
**Note**
In PostgreSQL 13 and earlier releases, *predefined* roles are known as *default* roles.
In the following list, you find some of the other predefined roles that are created automatically for a new RDS for PostgreSQL DB instance. Predefined roles and their privileges can't be changed. You can't drop, rename, or modify privileges for these predefined roles. Attempting to do so results in an error.
+ **rds\$1password** – A role that can change passwords and set up password constraints for database users. The `rds_superuser` role is granted with this role by default, and can grant the role to database users. For more information, see [Controlling user access to the PostgreSQL databaseControlling user access to PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Access.md).
+ For RDS for PostgreSQL versions older than 14, `rds_password` role can change passwords and set up password constraints for database users and users with `rds_superuser` role. From RDS for PostgreSQL version 14 and later, `rds_password` role can change passwords and set up password constraints only for database users. Only users with `rds_superuser` role can perform these actions on other users with `rds_superuser` role.
+ **rdsadmin** – A role that's created to handle many of the management tasks that the administrator with `superuser` privileges would perform on a standalone PostgreSQL database. This role is used internally by RDS for PostgreSQL for many management tasks.
+ **rdstopmgr** – A role that's used internally by Amazon RDS to support Multi-AZ deployments.
+ **rds\$1reserved** – A role that's used internally by Amazon RDS to reserve database connections.
# Viewing roles and their privileges
You can view predefined roles and their privileges in your RDS for PostgreSQL DB instance using different commands depending on your PostgreSQL version. To see all predefined roles, you can connect to your RDS for PostgreSQL DB instance and run following commands using the `psql`.
**For `psql` 15 and earlier versions**
Connect to your RDS for PostgreSQL DB instance and use the `\du` command in psql:
```
postgres=> \du
List of roles
Role name | Attributes | Member of
-----------------+------------------------------------------------------------+------------------------------------------------------
postgres | Create role, Create DB +| {rds_superuser}
| Password valid until infinity |
rds_ad | Cannot login | {}
rds_iam | Cannot login | {}
rds_password | Cannot login | {}
rds_replication | Cannot login | {}
rds_superuser | Cannot login | {pg_monitor,pg_signal_backend,rds_password,rds_replication}
rdsadmin | Superuser, Create role, Create DB, Replication, Bypass RLS+| {}
| Password valid until infinity |
```
**For `psql` 16 and later versions**
```
postgres=> \drg+
List of role grants
Role name | Member of | Options | Grantor
---------------+-----------------------------+---------------------+----------
postgres | rds_superuser | INHERIT, SET | rdsadmin
rds_superuser | pg_checkpoint | ADMIN, INHERIT, SET | rdsadmin
rds_superuser | pg_monitor | ADMIN, INHERIT, SET | rdsadmin
rds_superuser | pg_signal_backend | ADMIN, INHERIT, SET | rdsadmin
rds_superuser | pg_use_reserved_connections | ADMIN, INHERIT, SET | rdsadmin
rds_superuser | rds_password | ADMIN, INHERIT, SET | rdsadmin
rds_superuser | rds_replication | ADMIN, INHERIT, SET | rdsadmin
```
To check role membership without version dependency, you can use the following SQL query:
```
SELECT m.rolname AS "Role name", r.rolname AS "Member of"
FROM pg_catalog.pg_roles m
JOIN pg_catalog.pg_auth_members pam ON (pam.member = m.oid)
LEFT JOIN pg_catalog.pg_roles r ON (pam.roleid = r.oid)
LEFT JOIN pg_catalog.pg_roles g ON (pam.grantor = g.oid)
WHERE m.rolname !~ '^pg_'
ORDER BY 1, 2;
```
In the output, you can see that `rds_superuser` isn't a database user role (it can't login), but it has the privileges of many other roles. You can also see that database user `postgres` is a member of the `rds_superuser` role. As mentioned previously, `postgres` is the default value in the Amazon RDS console's **Create database** page. If you chose another name, that name is shown in the list of roles instead.
# Controlling user access to the PostgreSQL database
New databases in PostgreSQL are always created with a default set of privileges in the database's `public` schema that allow all database users and roles to create objects. These privileges allow database users to connect to the database, for example, and create temporary tables while connected.
To better control user access to the databases instances that you create on your RDS for PostgreSQL DB instance, we recommend that you revoke these default `public` privileges. After doing so, you then grant specific privileges for database users on a more granular basis, as shown in the following procedure.
**To set up roles and privileges for a new database instance**
Suppose you're setting up a database on a newly created RDS for PostgreSQL DB instance for use by several researchers, all of whom need read-write access to the database.
1. Use `psql` (or pgAdmin) to connect to your RDS for PostgreSQL DB instance:
```
psql --host=your-db-instance.666666666666.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
When prompted, enter your password. The `psql` client connects and displays the default administrative connection database, `postgres=>`, as the prompt.
1. To prevent database users from creating objects in the `public` schema, do the following:
```
postgres=> REVOKE CREATE ON SCHEMA public FROM PUBLIC;
REVOKE
```
1. Next, you create a new database instance:
```
postgres=> CREATE DATABASE lab_db;
CREATE DATABASE
```
1. Revoke all privileges from the `PUBLIC` schema on this new database.
```
postgres=> REVOKE ALL ON DATABASE lab_db FROM public;
REVOKE
```
1. Create a role for database users.
```
postgres=> CREATE ROLE lab_tech;
CREATE ROLE
```
1. Give database users that have this role the ability to connect to the database.
```
postgres=> GRANT CONNECT ON DATABASE lab_db TO lab_tech;
GRANT
```
1. Grant all users with the `lab_tech` role all privileges on this database.
```
postgres=> GRANT ALL PRIVILEGES ON DATABASE lab_db TO lab_tech;
GRANT
```
1. Create database users, as follows:
```
postgres=> CREATE ROLE lab_user1 LOGIN PASSWORD 'change_me';
CREATE ROLE
postgres=> CREATE ROLE lab_user2 LOGIN PASSWORD 'change_me';
CREATE ROLE
```
1. Grant these two users the privileges associated with the lab\$1tech role:
```
postgres=> GRANT lab_tech TO lab_user1;
GRANT ROLE
postgres=> GRANT lab_tech TO lab_user2;
GRANT ROLE
```
At this point, `lab_user1` and `lab_user2` can connect to the `lab_db` database. This example doesn't follow best practices for enterprise usage, which might include creating multiple database instances, different schemas, and granting limited permissions. For more complete information and additional scenarios, see [Managing PostgreSQL Users and Roles](https://aws.amazon.com/blogs//database/managing-postgresql-users-and-roles/).
For more information about privileges in PostgreSQL databases, see the [GRANT](https://www.postgresql.org/docs/current/static/sql-grant.html) command in the PostgreSQL documentation.
# Delegating and controlling user password management
As a DBA, you might want to delegate the management of user passwords. Or, you might want to prevent database users from changing their passwords or reconfiguring password constraints, such as password lifetime. To ensure that only the database users that you choose can change password settings, you can turn on the restricted password management feature. When you activate this feature, only those database users that have been granted the `rds_password` role can manage passwords.
**Note**
To use restricted password management, your RDS for PostgreSQL DB instance must be running PostgreSQL 10.6 or higher.
By default, this feature is `off`, as shown in the following:
```
postgres=> SHOW rds.restrict_password_commands;
rds.restrict_password_commands
--------------------------------
off
(1 row)
```
To turn on this feature, you use a custom parameter group and change the setting for `rds.restrict_password_commands` to 1. Be sure to reboot your RDS for PostgreSQL DB instance so that the setting takes effect.
With this feature active, `rds_password` privileges are needed for the following SQL commands:
```
CREATE ROLE myrole WITH PASSWORD 'mypassword';
CREATE ROLE myrole WITH PASSWORD 'mypassword' VALID UNTIL '2023-01-01';
ALTER ROLE myrole WITH PASSWORD 'mypassword' VALID UNTIL '2023-01-01';
ALTER ROLE myrole WITH PASSWORD 'mypassword';
ALTER ROLE myrole VALID UNTIL '2023-01-01';
ALTER ROLE myrole RENAME TO myrole2;
```
Renaming a role (`ALTER ROLE myrole RENAME TO newname`) is also restricted if the password uses the MD5 hashing algorithm.
With this feature active, attempting any of these SQL commands without the `rds_password` role permissions generates the following error:
```
ERROR: must be a member of rds_password to alter passwords
```
We recommend that you grant the `rds_password` to only a few roles that you use solely for password management. If you grant `rds_password` privileges to database users that don't have `rds_superuser` privileges, you need to also grant them the `CREATEROLE` attribute.
Make sure that you verify password requirements such as expiration and needed complexity on the client side. If you use your own client-side utility for password related changes, the utility needs to be a member of `rds_password` and have `CREATE ROLE` privileges.
# Using SCRAM for PostgreSQL password encryption
The *Salted Challenge Response Authentication Mechanism (SCRAM)* is an alternative to PostgreSQL's default message digest (MD5) algorithm for encrypting passwords. The SCRAM authentication mechanism is considered more secure than MD5. To learn more about these two different approaches to securing passwords, see [Password Authentication](https://www.postgresql.org/docs/14/auth-password.html) in the PostgreSQL documentation.
We recommend that you use SCRAM rather than MD5 as the password encryption scheme for your RDS for PostgreSQL DB instance. It's a cryptographic challenge-response mechanism that uses the scram-sha-256 algorithm for password authentication and encryption.
You might need to update libraries for your client applications to support SCRAM. For example, JDBC versions before 42.2.0 don't support SCRAM. For more information, see [PostgreSQL JDBC Driver](https://jdbc.postgresql.org/changelogs/2018-01-17-42.2.0-release/) in the PostgreSQL JDBC Driver documentation. For a list of other PostgreSQL drivers and SCRAM support, see [List of drivers](https://wiki.postgresql.org/wiki/List_of_drivers) in the PostgreSQL documentation.
RDS for PostgreSQL version 13.1 and higher support scram-sha-256. These versions also let you configure your DB instance to require SCRAM, as discussed in the following procedures.
## Setting up RDS for PostgreSQL DB instance to require SCRAM
you can require the RDS for PostgreSQL DB instance to accept only passwords that use the scram-sha-256 algorithm.
**Important**
For existing RDS Proxies with PostgreSQL databases, if you modify the database authentication to use `SCRAM` only, the proxy becomes unavailable for up to 60 seconds. To avoid the issue, do one of the following:
Ensure that the database allows both `SCRAM` and `MD5` authentication.
To use only `SCRAM` authentication, create a new proxy, migrate your application traffic to the new proxy, then delete the proxy previously associated with the database.
Before making changes to your system, be sure you understand the complete process, as follows:
+ Get information about all roles and password encryption for all database users.
+ Double-check the parameter settings for your RDS for PostgreSQL DB instance for the parameters that control password encryption.
+ If your RDS for PostgreSQL DB instance uses a default parameter group, you need to create a custom DB parameter group and apply it to your RDS for PostgreSQL DB instance so that you can modify parameters when needed. If your RDS for PostgreSQL DB instance uses a custom parameter group, you can modify the necessary parameters later in the process, as needed.
+ Change the `password_encryption` parameter to `scram-sha-256`.
+ Notify all database users that they need to update their passwords. Do the same for your `postgres` account. The new passwords are encrypted and stored using the scram-sha-256 algorithm.
+ Verify that all passwords are encrypted using as the type of encryption.
+ If all passwords use scram-sha-256, you can change the `rds.accepted_password_auth_method` parameter from `md5+scram` to `scram-sha-256`.
**Warning**
After you change `rds.accepted_password_auth_method` to scram-sha-256 alone, any users (roles) with `md5`–encrypted passwords can't connect.
### Getting ready to require SCRAM for your RDS for PostgreSQL DB instance
Before making any changes to your RDS for PostgreSQL DB instance, check all existing database user accounts. Also, check the type of encryption used for passwords. You can do these tasks by using the `rds_tools` extension. To see which PostgreSQL versions support `rds_tools`, see [Extension versions for Amazon RDS for PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html).
**To get a list of database users (roles) and password encryption methods**
1. Use `psql` to connect to your RDS for PostgreSQL DB instance, as shown in the following.
```
psql --host=db-name.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. Install the `rds_tools` extension.
```
postgres=> CREATE EXTENSION rds_tools;
CREATE EXTENSION
```
1. Get a listing of roles and encryption.
```
postgres=> SELECT * FROM
rds_tools.role_password_encryption_type();
```
You see output similar to the following.
```
rolname | encryption_type
----------------------+-----------------
pg_monitor |
pg_read_all_settings |
pg_read_all_stats |
pg_stat_scan_tables |
pg_signal_backend |
lab_tester | md5
user_465 | md5
postgres | md5
(8 rows)
```
### Creating a custom DB parameter group
**Note**
If your RDS for PostgreSQL DB instance already uses a custom parameter group, you don't need to create a new one.
For an overview of parameter groups for Amazon RDS, see [Working with parameters on your RDS for PostgreSQL DB instance](Appendix.PostgreSQL.CommonDBATasks.Parameters.md).
The password encryption type used for passwords is set in one parameter, `password_encryption`. The encryption that the RDS for PostgreSQL DB instance allows is set in another parameter, `rds.accepted_password_auth_method`. Changing either of these from the default values requires that you create a custom DB parameter group and apply it to your instance.
You can also use the AWS Management Console or the RDS API to create a custom DB parameter group. For more information, see
You can now associate the custom parameter group with your DB instance.
**To create a custom DB parameter group**
1. Use the `[create-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-parameter-group.html) ` CLI command to create the custom DB parameter group. This example uses `postgres13` as the source for this custom parameter group.
For Linux, macOS, or Unix:
```
aws rds create-db-parameter-group --db-parameter-group-name 'docs-lab-scram-passwords' \
--db-parameter-group-family postgres13 --description 'Custom parameter group for SCRAM'
```
For Windows:
```
aws rds create-db-parameter-group --db-parameter-group-name "docs-lab-scram-passwords" ^
--db-parameter-group-family postgres13 --description "Custom DB parameter group for SCRAM"
```
1. Use the `[modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html)` CLI command to apply this custom parameter group to your RDS for PostgreSQL DB cluster.
For Linux, macOS, or Unix:
```
aws rds modify-db-instance --db-instance-identifier 'your-instance-name' \
--db-parameter-group-name "docs-lab-scram-passwords
```
For Windows:
```
aws rds modify-db-instance --db-instance-identifier "your-instance-name" ^
--db-parameter-group-name "docs-lab-scram-passwords
```
To resynchronize your RDS for PostgreSQL DB instance with your custom DB parameter group, you need to reboot the primary and all other instances of the cluster. To minimize impact to your users, schedule this to occur during your regular maintenance window.
### Configuring password encryption to use SCRAM
The password encryption mechanism used by an RDS for PostgreSQL DB instance is set in the DB parameter group in the `password_encryption` parameter. Allowed values are unset, `md5`, or `scram-sha-256`. The default value depends on the RDS for PostgreSQL version, as follows:
+ RDS for PostgreSQL 14 and above – Default is `scram-sha-256`
+ RDS for PostgreSQL 13 – Default is `md5`
With a custom DB parameter group attached to your RDS for PostgreSQL DB instance, you can modify values for the password encryption parameter.
![\[Following, the RDS console shows the default values for the password_encryption parameters for RDS for PostgreSQL.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/rpg-pwd-encryption-md5-scram-1.png)
**To change password encryption setting to scram-sha-256**
+ Change the value of password encryption to scram-sha-256, as shown following. The change can be applied immediately because the parameter is dynamic, so a restart isn't required for the change to take effect.
For Linux, macOS, or Unix:
```
aws rds modify-db-parameter-group --db-parameter-group-name \
'docs-lab-scram-passwords' --parameters 'ParameterName=password_encryption,ParameterValue=scram-sha-256,ApplyMethod=immediate'
```
For Windows:
```
aws rds modify-db-parameter-group --db-parameter-group-name ^
"docs-lab-scram-passwords" --parameters "ParameterName=password_encryption,ParameterValue=scram-sha-256,ApplyMethod=immediate"
```
### Migrating passwords for user roles to SCRAM
You can migrate passwords for user roles to SCRAM as described following.
**To migrate database user (role) passwords from MD5 to SCRAM**
1. Log in as the administrator user (default user name, `postgres`) as shown following.
```
psql --host=db-name.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
1. Check the setting of the `password_encryption` parameter on your RDS for PostgreSQL DB instance by using the following command.
```
postgres=> SHOW password_encryption;
password_encryption
---------------------
md5
(1 row)
```
1. Change the value of this parameter to scram-sha-256. For more information, see [Configuring password encryption to use SCRAM](#PostgreSQL_Password_Encryption_configuration.configure-password-encryption).
1. Check the value again to make sure that it's now set to `scram-sha-256`, as follows.
```
postgres=> SHOW password_encryption;
password_encryption
---------------------
scram-sha-256
(1 row)
```
1. Notify all database users to change their passwords. Be sure to also change your own password for account `postgres` (the database user with `rds_superuser` privileges).
```
labdb=> ALTER ROLE postgres WITH LOGIN PASSWORD 'change_me';
ALTER ROLE
```
1. Repeat the process for all databases on your RDS for PostgreSQL DB instance.
### Changing parameter to require SCRAM
This is the final step in the process. After you make the change in the following procedure, any user accounts (roles) that still use `md5` encryption for passwords can't log in to the RDS for PostgreSQL DB instance.
The `rds.accepted_password_auth_method` specifies the encryption method that the RDS for PostgreSQL DB instance accepts for a user password during the login process. The default value is `md5+scram`, meaning that either method is accepted. In the following image, you can find the default setting for this parameter.
![\[The RDS console showing the default and allowed values for the rds.accepted_password_auth_method parameters.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/pwd-encryption-md5-scram-2.png)
The allowed values for this parameter are `md5+scram` or `scram` alone. Changing this parameter value to `scram` makes this a requirement.
**To change the parameter value to require SCRAM authentication for passwords**
1. Verify that all database user passwords for all databases on your RDS for PostgreSQL DB instance use `scram-sha-256` for password encryption. To do so, query `rds_tools` for the role (user) and encryption type, as follows.
```
postgres=> SELECT * FROM rds_tools.role_password_encryption_type();
rolname | encryption_type
----------------------+-----------------
pg_monitor |
pg_read_all_settings |
pg_read_all_stats |
pg_stat_scan_tables |
pg_signal_backend |
lab_tester | scram-sha-256
user_465 | scram-sha-256
postgres | scram-sha-256
( rows)
```
1. Repeat the query across all DB instances in your RDS for PostgreSQL DB instance.
If all passwords use scram-sha-256, you can proceed.
1. Change the value of the accepted password authentication to scram-sha-256, as follows.
For Linux, macOS, or Unix:
```
aws rds modify-db-parameter-group --db-parameter-group-name 'docs-lab-scram-passwords' \
--parameters 'ParameterName=rds.accepted_password_auth_method,ParameterValue=scram,ApplyMethod=immediate'
```
For Windows:
```
aws rds modify-db-parameter-group --db-parameter-group-name "docs-lab-scram-passwords" ^
--parameters "ParameterName=rds.accepted_password_auth_method,ParameterValue=scram,ApplyMethod=immediate"
```
# Dead connection handling in PostgreSQL
Dead connections occur when a database session remains active on the server despite the client application having abandoned or terminated abnormally. This situation typically arises when client processes crash or terminate unexpectedly without properly closing their database connections or canceling ongoing requests.
PostgreSQL efficiently identifies and cleans up dead connections when server processes are idle or attempt to send data to clients. However, detection is challenging for sessions that are idle, waiting for client input, or actively running queries. To handle these scenarios, PostgreSQL provides `tcp_keepalives_*`, `tcp_user_timeout`, and `client_connection_check_interval` parameters.
**Topics**
+ [
## Understanding TCP keepalive
](#Appendix.PostgreSQL.CommonDBATasks.DeadConnectionHandling.Understanding)
+ [
## Key TCP keepalive parameters in RDS for PostgreSQL
](#Appendix.PostgreSQL.CommonDBATasks.DeadConnectionHandling.Parameters)
+ [
## Use cases for TCP keepalive settings
](#Appendix.PostgreSQL.CommonDBATasks.DeadConnectionHandling.UseCases)
+ [
## Best practices
](#Appendix.PostgreSQL.CommonDBATasks.DeadConnectionHandling.BestPractices)
## Understanding TCP keepalive
TCP Keepalive is a protocol-level mechanism that helps maintain and verify connection integrity. Each TCP connection maintains kernel-level settings that govern keepalive behavior. When the keepalive timer expires, the system does the following:
+ Sends a probe packet with no data and the ACK flag set.
+ Expects a response from the remote endpoint according to TCP/IP specifications.
+ Manages connection state based on the response or lack thereof.
## Key TCP keepalive parameters in RDS for PostgreSQL
| Parameter | Description | Default values |
| --- |--- |--- |
| tcp\$1keepalives\$1idle | Specifies number of seconds of inactivity before sending keepalive message. | 300 |
| tcp\$1keepalives\$1interval | Specifies number of seconds between retransmissions of unacknowledged keepalive messages. | 30 |
| tcp\$1keepalives\$1count | Maximum lost keepalive messages before declaring connection dead | 2 |
| tcp\$1user\$1timeout | Specifies how long (in Milliseconds) unacknowledged data can remain before forcibly closing the connection. | 0 |
| client\$1connection\$1check\$1interval | Sets the interval (in Milliseconds) for checking client connection status during long-running queries. This ensures quicker detection of closed connections. | 0 |
## Use cases for TCP keepalive settings
### Keeping idle sessions alive
To prevent idle connections from being terminated by firewalls or routers due to inactivity:
+ Configure `tcp_keepalives_idle` to send keepalive packets at regular intervals.
### Detecting dead connections
To detect dead connections promptly:
+ Adjust `tcp_keepalives_idle`, `tcp_keepalives_interval`, and `tcp_keepalives_count`. For example, with Aurora PostgreSQL defaults, it takes about a minute (2 probes × 30 seconds) to detect a dead connection. Lowering these values can speed up detection.
+ Use `tcp_user_timeout` to specify the maximum wait time for an acknowledgment.
TCP keepalive settings help the kernel detect dead connections, but PostgreSQL may not act until the socket is used. If a session is running a long query, dead connections might only be detected after query completion. In PostgreSQL 14 and higher versions, `client_connection_check_interval` can expedite dead connection detection by periodically polling the socket during query execution.
## Best practices
+ **Set reasonable keepalive intervals:** Tune `tcp_user_timeout`, `tcp_keepalives_idle`, `tcp_keepalives_count` and `tcp_keepalives_interval` to balance detection speed and resource use.
+ **Optimize for your environment:** Align settings with network behavior, firewall policies, and session needs.
+ **Leverage PostgreSQL features:** Use `client_connection_check_interval` in PostgreSQL 14 and higher versions for efficient connection checks.
# Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL
We strongly recommend that you use the autovacuum feature to maintain the health of your PostgreSQL DB instance. Autovacuum automates the start of the VACUUM and the ANALYZE commands. It checks for tables with a large number of inserted, updated, or deleted tuples. After this check, it reclaims storage by removing obsolete data or tuples from the PostgreSQL database.
By default, autovacuum is turned on for the RDS for PostgreSQL DB instances that you create using any of the default PostgreSQL DB parameter groups. Other configuration parameters associated with the autovacuum feature are also set by default. Because these defaults are somewhat generic, you can benefit from tuning some of the parameters associated with the autovacuum feature for your specific workload.
Following, you can find more information about the autovacuum and how to tune some of its parameters on your RDS for PostgreSQL DB instance. For high-level information, see [Best practices for working with PostgreSQL](CHAP_BestPractices.md#CHAP_BestPractices.PostgreSQL).
**Topics**
+ [
## Allocating memory for autovacuum
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum.WorkMemory)
+ [
## Reducing the likelihood of transaction ID wraparound
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum.AdaptiveAutoVacuuming)
+ [
# Determining if the tables in your database need vacuuming
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.NeedVacuuming.md)
+ [
# Determining which tables are currently eligible for autovacuum
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.EligibleTables.md)
+ [
# Determining if autovacuum is currently running and for how long
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.AutovacuumRunning.md)
+ [
# Performing a manual vacuum freeze
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.VacuumFreeze.md)
+ [
# Reindexing a table when autovacuum is running
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.Reindexing.md)
+ [
# Managing autovacuum with large indexes
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.LargeIndexes.md)
+ [
# Other parameters that affect autovacuum
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.OtherParms.md)
+ [
# Setting table-level autovacuum parameters
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.TableParameters.md)
+ [
# Logging autovacuum and vacuum activities
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.Logging.md)
+ [
# Understanding the behavior of autovacuum with invalid databases
](appendix.postgresql.commondbatasks.autovacuumbehavior.md)
+ [
# Identify and resolve aggressive vacuum blockers in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.md)
## Allocating memory for autovacuum
One of the most important parameters influencing autovacuum performance is the [https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-AUTOVACUUM-WORK-MEM](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-AUTOVACUUM-WORK-MEM) parameter. In RDS for PostgreSQL versions 14 and prior, the `autovacuum_work_mem` parameter is set to -1, indicating that the setting of `maintenance_work_mem` is used instead. For all other versions, `autovacuum_work_mem` is determined by GREATEST(\$1DBInstanceClassMemory/32768\$1, 65536).
Manual vacuum operations always use the `maintenance_work_mem` setting, with a default setting of GREATEST(\$1DBInstanceClassMemory/63963136\$11024\$1, 65536), and it can also be adjusted at the session level using the `SET` command for more targeted manual `VACUUM` operations.
The `autovacuum_work_mem` determines memory for autovacuum to hold identifiers of dead tuples (`pg_stat_all_tables.n_dead_tup`) for vacuuming indexes.
When doing calculations to determine the `autovacuum_work_mem` parameter's value, be aware of the following:
+ If you set the parameter too low, the vacuum process might have to scan the table multiple times to complete its work. Such multiple scans can have a negative impact on performance. For larger instances, setting `maintenance_work_mem` or `autovacuum_work_mem` to at least 1 GB can improve the performance of vacuuming tables with a high number of dead tuples. However, in PostgreSQL versions 16 and prior, vacuum’s memory usage is capped at 1 GB, which is sufficient to process approximately 179 million dead tuples in a single pass. If a table has more dead tuples than this, vacuum will need to make multiple passes through the table's indexes, significantly increasing the time required. Starting with PostgreSQL version 17, there isn't a limit of 1 GB, and autovacuum can process more than 179 million tuples by using radix trees.
A tuple identifier is 6 bytes in size. To estimate the memory needed for vacuuming an index of a table, query `pg_stat_all_tables.n_dead_tup` to find the number of dead tuples, then multiply this number by 6 to determine the memory required for vacuuming the index in a single pass. You may use the following query:
```
SELECT
relname AS table_name,
n_dead_tup,
pg_size_pretty(n_dead_tup * 6) AS estimated_memory
FROM
pg_stat_all_tables
WHERE
relname = 'name_of_the_table';
```
+ The `autovacuum_work_mem` parameter works in conjunction with the `autovacuum_max_workers` parameter. Each worker among `autovacuum_max_workers` can use the memory that you allocate. If you have many small tables, allocate more `autovacuum_max_workers` and less `autovacuum_work_mem`. If you have large tables (larger than 100 GB), allocate more memory and fewer worker processes. You need to have enough memory allocated to succeed on your biggest table. Thus, make sure that the combination of worker processes and memory equals the total memory that you want to allocate.
## Reducing the likelihood of transaction ID wraparound
In some cases, parameter group settings related to autovacuum might not be aggressive enough to prevent transaction ID wraparound. To address this, RDS for PostgreSQL provides a mechanism that adapts the autovacuum parameter values automatically. *Adaptive autovacuum* is a feature for RDS for PostgreSQL. A detailed explanation of [TransactionID wraparound](https://www.postgresql.org/docs/current/static/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND) is found in the PostgreSQL documentation.
Adaptive autovacuum is turned on by default for RDS for PostgreSQL instances with the dynamic parameter `rds.adaptive_autovacuum` set to ON. We strongly recommend that you keep this turned on. However, to turn off adaptive autovacuum parameter tuning, set the `rds.adaptive_autovacuum` parameter to 0 or OFF.
Transaction ID wraparound is still possible even when Amazon RDS Amazon RDS tunes the autovacuum parameters. We encourage you to implement an Amazon CloudWatch alarm for transaction ID wraparound. For more information, see the post [Implement an early warning system for transaction ID wraparound in RDS for PostgreSQL](https://aws.amazon.com/blogs/database/implement-an-early-warning-system-for-transaction-id-wraparound-in-amazon-rds-for-postgresql/) on the AWS Database Blog.
With adaptive autovacuum parameter tuning turned on, Amazon RDS begins adjusting autovacuum parameters when the CloudWatch metric `MaximumUsedTransactionIDs` reaches the value of the `autovacuum_freeze_max_age` parameter or 500,000,000, whichever is greater.
Amazon RDS continues to adjust parameters for autovacuum if a table continues to trend toward transaction ID wraparound. Each of these adjustments dedicates more resources to autovacuum to avoid wraparound. Amazon RDS updates the following autovacuum-related parameters:
+ [autovacuum\$1vacuum\$1cost\$1delay](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-DELAY)
+ [ autovacuum\$1vacuum\$1cost\$1limit](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-LIMIT)
+ [https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-AUTOVACUUM-WORK-MEM](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-AUTOVACUUM-WORK-MEM)
+ [autovacuum\$1naptime](https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-NAPTIME)
RDS modifies these parameters only if the new value makes autovacuum more aggressive. The parameters are modified in memory on the DB instance. The values in the parameter group aren't changed. To view the current in-memory settings, use the PostgreSQL [SHOW](https://www.postgresql.org/docs/current/sql-show.html) SQL command.
When Amazon RDS modifies any of these autovacuum parameters, it generates an event for the affected DB instance. This event is visible on the AWS Management Console and through the Amazon RDS API. After the `MaximumUsedTransactionIDs` CloudWatch metric returns below the threshold, Amazon RDS resets the autovacuum-related parameters in memory back to the values specified in the parameter group. It then generates another event corresponding to this change.
# Determining if the tables in your database need vacuuming
You can use the following query to show the number of unfrozen transactions in a database. The `datfrozenxid` column of a database's `pg_database` row is a lower bound on the normal transaction IDs appearing in that database. This column is the minimum of the per-table `relfrozenxid` values within the database.
```
SELECT datname, age(datfrozenxid) FROM pg_database ORDER BY age(datfrozenxid) desc limit 20;
```
For example, the results of running the preceding query might be the following.
```
datname | age
mydb | 1771757888
template0 | 1721757888
template1 | 1721757888
rdsadmin | 1694008527
postgres | 1693881061
(5 rows)
```
When the age of a database reaches 2 billion transaction IDs, transaction ID (XID) wraparound occurs and the database becomes read-only. You can use this query to produce a metric and run a few times a day. By default, autovacuum is set to keep the age of transactions to no more than 200,000,000 ([https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-FREEZE-MAX-AGE](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-FREEZE-MAX-AGE)).
A sample monitoring strategy might look like this:
+ Set the `autovacuum_freeze_max_age` value to 200 million transactions.
+ If a table reaches 500 million unfrozen transactions, that triggers a low-severity alarm. This isn't an unreasonable value, but it can indicate that autovacuum isn't keeping up.
+ If a table ages to 1 billion, this should be treated as an alarm to take action on. In general, you want to keep ages closer to `autovacuum_freeze_max_age` for performance reasons. We recommend that you investigate using the recommendations that follow.
+ If a table reaches 1.5 billion unvacuumed transactions, that triggers a high-severity alarm. Depending on how quickly your database uses transaction IDs, this alarm can indicate that the system is running out of time to run autovacuum. In this case, we recommend that you resolve this immediately.
If a table is constantly breaching these thresholds, modify your autovacuum parameters further. By default, using VACUUM manually (which has cost-based delays disabled) is more aggressive than using the default autovacuum, but it is also more intrusive to the system as a whole.
We recommend the following:
+ Be aware and turn on a monitoring mechanism so that you are aware of the age of your oldest transactions.
For information on creating a process that warns you about transaction ID wraparound, see the AWS Database Blog post [Implement an early warning system for transaction ID wraparound in Amazon RDS for PostgreSQL](https://aws.amazon.com/blogs/database/implement-an-early-warning-system-for-transaction-id-wraparound-in-amazon-rds-for-postgresql/).
+ For busier tables, perform a manual vacuum freeze regularly during a maintenance window, in addition to relying on autovacuum. For information on performing a manual vacuum freeze, see [Performing a manual vacuum freeze](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.VacuumFreeze.md).
# Determining which tables are currently eligible for autovacuum
Often, it is one or two tables in need of vacuuming. Tables whose `relfrozenxid` value is greater than the number of transactions in `autovacuum_freeze_max_age` are always targeted by autovacuum. Otherwise, if the number of tuples made obsolete since the last VACUUM exceeds the vacuum threshold, the table is vacuumed.
The [autovacuum threshold](https://www.postgresql.org/docs/current/static/routine-vacuuming.html#AUTOVACUUM) is defined as:
```
Vacuum-threshold = vacuum-base-threshold + vacuum-scale-factor * number-of-tuples
```
where the `vacuum base threshold` is `autovacuum_vacuum_threshold`, the `vacuum scale factor` is `autovacuum_vacuum_scale_factor`, and the `number of tuples` is `pg_class.reltuples`.
While you are connected to your database, run the following query to see a list of tables that autovacuum sees as eligible for vacuuming.
```
WITH vbt AS (SELECT setting AS autovacuum_vacuum_threshold FROM
pg_settings WHERE name = 'autovacuum_vacuum_threshold'),
vsf AS (SELECT setting AS autovacuum_vacuum_scale_factor FROM
pg_settings WHERE name = 'autovacuum_vacuum_scale_factor'),
fma AS (SELECT setting AS autovacuum_freeze_max_age FROM pg_settings WHERE name = 'autovacuum_freeze_max_age'),
sto AS (select opt_oid, split_part(setting, '=', 1) as param,
split_part(setting, '=', 2) as value from (select oid opt_oid, unnest(reloptions) setting from pg_class) opt)
SELECT '"'||ns.nspname||'"."'||c.relname||'"' as relation,
pg_size_pretty(pg_table_size(c.oid)) as table_size,
age(relfrozenxid) as xid_age,
coalesce(cfma.value::float, autovacuum_freeze_max_age::float) autovacuum_freeze_max_age,
(coalesce(cvbt.value::float, autovacuum_vacuum_threshold::float) +
coalesce(cvsf.value::float,autovacuum_vacuum_scale_factor::float) * c.reltuples)
AS autovacuum_vacuum_tuples, n_dead_tup as dead_tuples FROM
pg_class c join pg_namespace ns on ns.oid = c.relnamespace
join pg_stat_all_tables stat on stat.relid = c.oid join vbt on (1=1) join vsf on (1=1) join fma on (1=1)
left join sto cvbt on cvbt.param = 'autovacuum_vacuum_threshold' and c.oid = cvbt.opt_oid
left join sto cvsf on cvsf.param = 'autovacuum_vacuum_scale_factor' and c.oid = cvsf.opt_oid
left join sto cfma on cfma.param = 'autovacuum_freeze_max_age' and c.oid = cfma.opt_oid
WHERE c.relkind = 'r' and nspname <> 'pg_catalog'
AND (age(relfrozenxid) >= coalesce(cfma.value::float, autovacuum_freeze_max_age::float)
OR coalesce(cvbt.value::float, autovacuum_vacuum_threshold::float) +
coalesce(cvsf.value::float,autovacuum_vacuum_scale_factor::float) *
c.reltuples <= n_dead_tup)
ORDER BY age(relfrozenxid) DESC LIMIT 50;
```
# Determining if autovacuum is currently running and for how long
If you need to manually vacuum a table, make sure to determine if autovacuum is currently running. If it is, you might need to adjust parameters to make it run more efficiently, or turn off autovacuum temporarily so that you can manually run VACUUM.
Use the following query to determine if autovacuum is running, how long it has been running, and if it is waiting on another session.
```
SELECT datname, usename, pid, state, wait_event, current_timestamp - xact_start AS xact_runtime, query
FROM pg_stat_activity
WHERE upper(query) LIKE '%VACUUM%'
ORDER BY xact_start;
```
After running the query, you should see output similar to the following.
```
datname | usename | pid | state | wait_event | xact_runtime | query
--------+----------+-------+--------+------------+-------------------------+--------------------------------------------------------------------------------------------------------
mydb | rdsadmin | 16473 | active | | 33 days 16:32:11.600656 | autovacuum: VACUUM ANALYZE public.mytable1 (to prevent wraparound)
mydb | rdsadmin | 22553 | active | | 14 days 09:15:34.073141 | autovacuum: VACUUM ANALYZE public.mytable2 (to prevent wraparound)
mydb | rdsadmin | 41909 | active | | 3 days 02:43:54.203349 | autovacuum: VACUUM ANALYZE public.mytable3
mydb | rdsadmin | 618 | active | | 00:00:00 | SELECT datname, usename, pid, state, wait_event, current_timestamp - xact_start AS xact_runtime, query+
| | | | | | FROM pg_stat_activity +
| | | | | | WHERE query like '%VACUUM%' +
| | | | | | ORDER BY xact_start; +
```
Several issues can cause a long-running autovacuum session (that is, multiple days long). The most common issue is that your [https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM](https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM) parameter value is set too low for the size of the table or rate of updates.
We recommend that you use the following formula to set the `maintenance_work_mem` parameter value.
```
GREATEST({DBInstanceClassMemory/63963136*1024},65536)
```
Short running autovacuum sessions can also indicate problems:
+ It can indicate that there aren't enough `autovacuum_max_workers` for your workload. In this case, you need to indicate the number of workers.
+ It can indicate that there is an index corruption (autovacuum crashes and restarts on the same relation but makes no progress). In this case, run a manual `vacuum freeze verbose table` to see the exact cause.
# Performing a manual vacuum freeze
You might want to perform a manual vacuum on a table that has a vacuum process already running. This is useful if you have identified a table with an age approaching 2 billion transactions (or above any threshold you are monitoring).
The following steps are guidelines, with several variations to the process. For example, during testing, suppose that you find that the [https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM](https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM) parameter value is set too small and that you need to take immediate action on a table. However, perhaps you don't want to bounce the instance at the moment. Using the queries in previous sections, you determine which table is the problem and notice a long running autovacuum session. You know that you need to change the `maintenance_work_mem` parameter setting, but you also need to take immediate action and vacuum the table in question. The following procedure shows what to do in this situation.
**To manually perform a vacuum freeze**
1. Open two sessions to the database containing the table you want to vacuum. For the second session, use "screen" or another utility that maintains the session if your connection is dropped.
1. In session one, get the process ID (PID) of the autovacuum session running on the table.
Run the following query to get the PID of the autovacuum session.
```
SELECT datname, usename, pid, current_timestamp - xact_start
AS xact_runtime, query
FROM pg_stat_activity WHERE upper(query) LIKE '%VACUUM%' ORDER BY
xact_start;
```
1. In session two, calculate the amount of memory that you need for this operation. In this example, we determine that we can afford to use up to 2 GB of memory for this operation, so we set [https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM](https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE-WORK-MEM) for the current session to 2 GB.
```
SET maintenance_work_mem='2 GB';
SET
```
1. In session two, issue a `vacuum freeze verbose` command for the table. The verbose setting is useful because, although there is no progress report for this in PostgreSQL currently, you can see activity.
```
\timing on
Timing is on.
vacuum freeze verbose pgbench_branches;
```
```
INFO: vacuuming "public.pgbench_branches"
INFO: index "pgbench_branches_pkey" now contains 50 row versions in 2 pages
DETAIL: 0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: index "pgbench_branches_test_index" now contains 50 row versions in 2 pages
DETAIL: 0 index row versions were removed.
0 index pages have been deleted, 0 are currently reusable.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
INFO: "pgbench_branches": found 0 removable, 50 nonremovable row versions
in 43 out of 43 pages
DETAIL: 0 dead row versions cannot be removed yet.
There were 9347 unused item pointers.
0 pages are entirely empty.
CPU 0.00s/0.00u sec elapsed 0.00 sec.
VACUUM
Time: 2.765 ms
```
1. In session one, if autovacuum was blocking the vacuum session, `pg_stat_activity` shows that waiting is `T` for your vacuum session. In this case, end the autovacuum process as follows.
```
SELECT pg_terminate_backend('the_pid');
```
**Note**
Some lower versions of Amazon RDS can't terminate an autovacuum process using the preceding command and fail with the following error: `ERROR: 42501: must be a superuser to terminate superuser process LOCATION: pg_terminate_backend, signalfuncs.c:227`.
At this point, your session begins. Autovacuum restarts immediately because this table is probably the highest on its list of work.
1. Initiate your `vacuum freeze verbose` command in session two, and then end the autovacuum process in session one.
# Reindexing a table when autovacuum is running
If an index has become corrupt, autovacuum continues to process the table and fails. If you attempt a manual vacuum in this situation, you receive an error message like the following.
```
postgres=> vacuum freeze pgbench_branches;
ERROR: index "pgbench_branches_test_index" contains unexpected
zero page at block 30521
HINT: Please REINDEX it.
```
When the index is corrupted and autovacuum is attempting to run on the table, you contend with an already running autovacuum session. When you issue a [REINDEX](https://www.postgresql.org/docs/current/static/sql-reindex.html) command, you take out an exclusive lock on the table. Write operations are blocked, and also read operations that use that specific index.
**To reindex a table when autovacuum is running on the table**
1. Open two sessions to the database containing the table that you want to vacuum. For the second session, use "screen" or another utility that maintains the session if your connection is dropped.
1. In session one, get the PID of the autovacuum session running on the table.
Run the following query to get the PID of the autovacuum session.
```
SELECT datname, usename, pid, current_timestamp - xact_start
AS xact_runtime, query
FROM pg_stat_activity WHERE upper(query) like '%VACUUM%' ORDER BY
xact_start;
```
1. In session two, issue the reindex command.
```
\timing on
Timing is on.
reindex index pgbench_branches_test_index;
REINDEX
Time: 9.966 ms
```
1. In session one, if autovacuum was blocking the process, you see in `pg_stat_activity` that waiting is "T" for your vacuum session. In this case, you end the autovacuum process.
```
SELECT pg_terminate_backend('the_pid');
```
At this point, your session begins. It's important to note that autovacuum restarts immediately because this table is probably the highest on its list of work.
1. Initiate your command in session two, and then end the autovacuum process in session 1.
# Managing autovacuum with large indexes
As part of its operation, *autovacuum* performs several [ vacuum phases](https://www.postgresql.org/docs/current/progress-reporting.html#VACUUM-PHASES) while running on a table. Before the table is cleaned up, all of its indexes are first vacuumed. When removing multiple large indexes, this phase consumes a significant amount of time and resources. Therefore, as a best practice, be sure to control the number of indexes on a table and eliminate unused indexes.
For this process, first check the overall index size. Then, determine if there are potentially unused indexes that can be removed as shown in the following examples.
**To check the size of the table and its indexes**
```
postgres=> select pg_size_pretty(pg_relation_size('pgbench_accounts'));
pg_size_pretty
6404 MB
(1 row)
```
```
postgres=> select pg_size_pretty(pg_indexes_size('pgbench_accounts'));
pg_size_pretty
11 GB
(1 row)
```
In this example, the size of indexes is larger than the table. This difference can cause performance issues as the indexes are bloated or unused, which impacts the autovacuum as well as insert operations.
**To check for unused indexes**
Using the [https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ALL-INDEXES-VIEW](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ALL-INDEXES-VIEW) view, you can check how frequently an index is used with the `idx_scan` column. In the following example, the unused indexes have the `idx_scan` value of `0`.
```
postgres=> select * from pg_stat_user_indexes where relname = 'pgbench_accounts' order by idx_scan desc;
relid | indexrelid | schemaname | relname | indexrelname | idx_scan | idx_tup_read | idx_tup_fetch
-------+------------+------------+------------------+-----------------------+----------+--------------+---------------
16433 | 16454 | public | pgbench_accounts | index_f | 6 | 6 | 0
16433 | 16450 | public | pgbench_accounts | index_b | 3 | 199999 | 0
16433 | 16447 | public | pgbench_accounts | pgbench_accounts_pkey | 0 | 0 | 0
16433 | 16452 | public | pgbench_accounts | index_d | 0 | 0 | 0
16433 | 16453 | public | pgbench_accounts | index_e | 0 | 0 | 0
16433 | 16451 | public | pgbench_accounts | index_c | 0 | 0 | 0
16433 | 16449 | public | pgbench_accounts | index_a | 0 | 0 | 0
(7 rows)
```
```
postgres=> select schemaname, relname, indexrelname, idx_scan from pg_stat_user_indexes where relname = 'pgbench_accounts' order by idx_scan desc;
schemaname | relname | indexrelname | idx_scan
------------+------------------+-----------------------+----------
public | pgbench_accounts | index_f | 6
public | pgbench_accounts | index_b | 3
public | pgbench_accounts | pgbench_accounts_pkey | 0
public | pgbench_accounts | index_d | 0
public | pgbench_accounts | index_e | 0
public | pgbench_accounts | index_c | 0
public | pgbench_accounts | index_a | 0
(7 rows)
```
**Note**
These statistics are incremental from the time that the statistics are reset. Suppose you have an index that is only used at the end of a business quarter or just for a specific report. It's possible that this index hasn't been used since the statistics were reset. For more information, see [Statistics Functions](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-STATS-FUNCTIONS). Indexes that are used to enforce uniqueness won't have scans performed and shouldn't be identified as unused indexes. To identify the unused indexes, you should have in-depth knowledge of the application and its queries.
To check when the stats were last reset for a database, use [ https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-DATABASE-VIEW]( https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-DATABASE-VIEW)
```
postgres=> select datname, stats_reset from pg_stat_database where datname = 'postgres';
datname | stats_reset
----------+-------------------------------
postgres | 2022-11-17 08:58:11.427224+00
(1 row)
```
## Vacuuming a table as quickly as possible
**RDS for PostgreSQL 12 and higher**
If you have too many indexes in a large table, your DB instance could be nearing transaction ID wraparound (XID), which is when the XID counter wraps around to zero. Left unchecked, this situation could result in data loss. However, you can quickly vacuum the table without cleaning up the indexes. In RDS for PostgreSQL 12 and higher, you can use VACUUM with the [https://www.postgresql.org/docs/current/sql-vacuum.html](https://www.postgresql.org/docs/current/sql-vacuum.html) clause.
```
postgres=> VACUUM (INDEX_CLEANUP FALSE, VERBOSE TRUE) pgbench_accounts;
INFO: vacuuming "public.pgbench_accounts"
INFO: table "pgbench_accounts": found 0 removable, 8 nonremovable row versions in 1 out of 819673 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 7517
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.01 s, system: 0.00 s, elapsed: 0.01 s.
```
If an autovacuum session is already running, you must terminate it to begin the manual VACUUM. For information on performing a manual vacuum freeze, see [Performing a manual vacuum freeze](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.VacuumFreeze.md)
**Note**
Skipping index cleanup regularly causes index bloat, which degrades scan performance. The index retains dead rows, and the table retains dead line pointers. As a result, `pg_stat_all_tables.n_dead_tup` increases until autovacuum or a manual VACUUM with index cleanup runs. As a best practice, use this procedure only to prevent transaction ID wraparound.
**RDS for PostgreSQL 11 and older**
However, in RDS for PostgreSQL 11 and lower versions, the only way to allow vacuum to complete faster is to reduce the number of indexes on a table. Dropping an index can affect query plans. We recommend that you drop unused indexes first, then drop the indexes when XID wraparound is very near. After the vacuum process completes, you can recreate these indexes.
# Other parameters that affect autovacuum
The following query shows the values of some of the parameters that directly affect autovacuum and its behavior. The [autovacuum parameters](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html) are described fully in the PostgreSQL documentation.
```
SELECT name, setting, unit, short_desc
FROM pg_settings
WHERE name IN (
'autovacuum_max_workers',
'autovacuum_analyze_scale_factor',
'autovacuum_naptime',
'autovacuum_analyze_threshold',
'autovacuum_analyze_scale_factor',
'autovacuum_vacuum_threshold',
'autovacuum_vacuum_scale_factor',
'autovacuum_vacuum_threshold',
'autovacuum_vacuum_cost_delay',
'autovacuum_vacuum_cost_limit',
'vacuum_cost_limit',
'autovacuum_freeze_max_age',
'maintenance_work_mem',
'vacuum_freeze_min_age');
```
While these all affect autovacuum, some of the most important ones are:
+ [maintenance\$1work\$1mem](https://www.postgresql.org/docs/current/static/runtime-config-resource.html#GUC-MAINTENANCE_WORK_MEM)
+ [autovacuum\$1freeze\$1max\$1age](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-FREEZE-MAX-AGE)
+ [autovacuum\$1max\$1workers](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-MAX-WORKERS)
+ [autovacuum\$1vacuum\$1cost\$1delay](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-DELAY)
+ [ autovacuum\$1vacuum\$1cost\$1limit](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-LIMIT)
# Setting table-level autovacuum parameters
You can set autovacuum-related [storage parameters](https://www.postgresql.org/docs/current/static/sql-createtable.html#SQL-CREATETABLE-STORAGE-PARAMETERS) at a table level, which can be better than altering the behavior of the entire database. For large tables, you might need to set aggressive settings and you might not want to make autovacuum behave that way for all tables.
The following query shows which tables currently have table-level options in place.
```
SELECT relname, reloptions
FROM pg_class
WHERE reloptions IS NOT null;
```
An example where this might be useful is on tables that are much larger than the rest of your tables. Suppose that you have one 300-GB table and 30 other tables less than 1 GB. In this case, you might set some specific parameters for your large table so you don't alter the behavior of your entire system.
```
ALTER TABLE mytable set (autovacuum_vacuum_cost_delay=0);
```
Doing this turns off the cost-based autovacuum delay for this table at the expense of more resource usage on your system. Normally, autovacuum pauses for `autovacuum_vacuum_cost_delay` each time `autovacuum_cost_limit` is reached. For more details, see the PostgreSQL documentation about [cost-based vacuuming](https://www.postgresql.org/docs/current/static/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-VACUUM-COST).
# Logging autovacuum and vacuum activities
Information about autovacuum activities is sent to the `postgresql.log` based on the level specified in the `rds.force_autovacuum_logging_level` parameter. Following are the values allowed for this parameter and the PostgreSQL versions for which that value is the default setting:
+ `disabled` (PostgreSQL 10, PostgreSQL 9.6)
+ `debug5`, `debug4`, `debug3`, `debug2`, `debug1`
+ `info` (PostgreSQL 12, PostgreSQL 11)
+ `notice`
+ `warning` (PostgreSQL 13 and above)
+ `error`, log, `fatal`, `panic`
The `rds.force_autovacuum_logging_level` works with the `log_autovacuum_min_duration` parameter. The `log_autovacuum_min_duration` parameter's value is the threshold (in milliseconds) above which autovacuum actions get logged. A setting of `-1` logs nothing, while a setting of 0 logs all actions. As with `rds.force_autovacuum_logging_level`, default values for `log_autovacuum_min_duration` are version dependent, as follows:
+ `10000 ms` – PostgreSQL 14, PostgreSQL 13, PostgreSQL 12, and PostgreSQL 11
+ `(empty)` – No default value for PostgreSQL 10 and PostgreSQL 9.6
We recommend that you set `rds.force_autovacuum_logging_level` to `WARNING`. We also recommend that you set `log_autovacuum_min_duration` to a value from 1000 to 5000. A setting of 5000 logs activity that takes longer than 5,000 milliseconds. Any setting other than -1 also logs messages if the autovacuum action is skipped because of a conflicting lock or concurrently dropped relations. For more information, see [Automatic Vacuuming](https://www.postgresql.org/docs/current/runtime-config-autovacuum.html) in the PostgreSQL documentation.
To troubleshoot issues, you can change the `rds.force_autovacuum_logging_level` parameter to one of the debug levels, from `debug1` up to `debug5` for the most verbose information. We recommend that you use debug settings for short periods of time and for troubleshooting purposes only. To learn more, see [When to log](https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-WHEN) in the PostgreSQL documentation.
**Note**
PostgreSQL allows the `rds_superuser` account to view autovacuum sessions in `pg_stat_activity`. For example, you can identify and end an autovacuum session that is blocking a command from running, or running slower than a manually issued vacuum command.
# Understanding the behavior of autovacuum with invalid databases
A new value `-2` is introduced into the `datconnlimit` column in the `pg_database` catalog to indicate databases that have been interrupted in the middle of the DROP DATABASE operation as invalid.
This new value is available from the following RDS for PostgreSQL versions:
+ 15.4 and all higher versions
+ 14.9 and higher versions
+ 13.12 and higher versions
+ 12.16 and higher versions
+ 11.21 and higher versions
Invalid databases do not affect autovacuum's ability to freeze functionality for valid databases. Autovacuum ignores invalid databases. Consequently, regular vacuum operations will continue to function properly and efficiently for all valid databases in your PostgreSQL environment.
**Topics**
+ [
## Monitoring transaction ID
](#appendix.postgresql.commondbatasks.autovacuum.monitorxid)
+ [
## Adjusting the monitoring query
](#appendix.postgresql.commondbatasks.autovacuum.monitoradjust)
+ [
## Resolving invalid database issue
](#appendix.postgresql.commondbatasks.autovacuum.connissue)
## Monitoring transaction ID
The `age(datfrozenxid)` function is commonly used to monitor the transaction ID (XID) age of databases to prevent transaction ID wraparound.
Since invalid databases are excluded from autovacuum, their transaction ID (XID) counter can reach the maximum value of `2 billion`, wrap around to `- 2 billion`, and continue this cycle indefinitely. A typical query to monitor Transaction ID wraparound might look like:
```
SELECT max(age(datfrozenxid)) FROM pg_database;
```
However, with the introduction of the -2 value for `datconnlimit`, invalid databases can skew the results of this query. Since these databases are not valid and should not be part of regular maintenance checks, they can cause false positives, leading you to believe that the `age(datfrozenxid)` is higher than it actually is.
## Adjusting the monitoring query
To ensure accurate monitoring, you should adjust your monitoring query to exclude invalid databases. Follow this recommended query:
```
SELECT
max(age(datfrozenxid))
FROM
pg_database
WHERE
datconnlimit <> -2;
```
This query ensures that only valid databases are considered in the `age(datfrozenxid)` calculation, providing a true reflection of the transaction ID age across your PostgreSQL environment.
## Resolving invalid database issue
When attempting to connect to an invalid database, you may encounter an error message similar to the following:
```
postgres=> \c db1
connection to server at "mydb.xxxxxxxxxx.us-west-2.rds.amazonaws.com" (xx.xx.xx.xxx), port xxxx failed: FATAL: cannot connect to invalid database "db1"
HINT: Use DROP DATABASE to drop invalid databases.
Previous connection kept
```
Additionally, if the `log_min_messages` parameter is set to `DEBUG2` or higher, you may notice the following log entries indicating that the autovacuum process is skipping the invalid database:
```
2024-07-30 05:59:00 UTC::@:[32000]:DEBUG: autovacuum: skipping invalid database "db6"
2024-07-30 05:59:00 UTC::@:[32000]:DEBUG: autovacuum: skipping invalid database "db1"
```
To resolve the issue, follow the `HINT` provided during the connection attempt. Connect to any valid database using your RDS master account or a database account with the `rds_superuser` role, and drop invalid database(s).
```
SELECT
'DROP DATABASE ' || quote_ident(datname) || ';'
FROM
pg_database
WHERE
datconnlimit = -2 \gexec
```
# Identify and resolve aggressive vacuum blockers in RDS for PostgreSQL
In PostgreSQL, vacuuming is vital for ensuring database health as it reclaims storage and prevents [transaction ID wraparound](https://www.postgresql.org/docs/current/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND) issues. However, there are times when vacuuming can be prevented from operating as desired, which can result in performance degradation, storage bloat, and even impact availability of your DB instance by transaction ID wraparound. Therefore, identifying and resolving these issues are essential for optimal database performance and availability. Read [Understanding autovacuum in Amazon RDS for PostgreSQL environments](https://aws.amazon.com/blogs/database/understanding-autovacuum-in-amazon-rds-for-postgresql-environments/) to learn more about autovacuum.
The `postgres_get_av_diag()` function helps identify issues that either prevent or delay the aggressive vacuum progress. Suggestions are provided, which may include commands to resolve the issue where it is identifiable or guidance for further diagnostics where the issue is not identifiable. Aggressive vacuum blockers are reported when the age exceeds RDS' [adaptive autovacuum](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum.AdaptiveAutoVacuuming) threshold of 500 million transaction IDs.
**What is the age of the transaction ID?**
The `age()` function for transaction IDs calculates the number of transactions that have occurred since the oldest unfrozen transaction ID for a database (`pg_database.datfrozenxid`) or table (`pg_class.relfrozenxid`). This value indicates database activity since the last aggressive vacuum operation and highlights the likely workload for upcoming VACUUM processes.
**What is an aggressive vacuum?**
An aggressive VACUUM operation conducts a comprehensive scan of all pages within a table, including those typically skipped during regular VACUUMs. This thorough scan aims to "freeze" transaction IDs approaching their maximum age, effectively preventing a situation known as [transaction ID wraparound](https://www.postgresql.org/docs/current/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND).
For `postgres_get_av_diag()` to report blockers, the blocker must be at least 500 million transactions old.
**Topics**
+ [
# Installing autovacuum monitoring and diagnostic tools in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Installation.md)
+ [
# Functions of postgres\$1get\$1av\$1diag() in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Functions.md)
+ [
# Resolving identifiable vacuum blockers in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md)
+ [
# Resolving unidentifiable vacuum blockers in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Unidentifiable_blockers.md)
+ [
# Resolving vacuum performance issues in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Performance.md)
+ [
# Explanation of the NOTICE messages in RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.NOTICE.md)
# Installing autovacuum monitoring and diagnostic tools in RDS for PostgreSQL
The `postgres_get_av_diag()` function is currently available in the following RDS for PostgreSQL versions:
+ 17.2 and higher 17 versions
+ 16.7 and higher 16 versions
+ 15.11 and higher 15 versions
+ 14.16 and higher 14 versions
+ 13.19 and higher 13 versions
In order to use `postgres_get_av_diag()`, create the `rds_tools` extension.
```
postgres=> CREATE EXTENSION rds_tools ;
CREATE EXTENSION
```
Verify that the extension is installed.
```
postgres=> \dx rds_tools
List of installed extensions
Name | Version | Schema | Description
----------+---------+-----------+----------------------------------------------------------
rds_tools | 1.8 | rds_tools | miscellaneous administrative functions for RDS PostgreSQL
1 row
```
Verify that the function is created.
```
postgres=> SELECT
proname function_name,
pronamespace::regnamespace function_schema,
proowner::regrole function_owner
FROM
pg_proc
WHERE
proname = 'postgres_get_av_diag';
function_name | function_schema | function_owner
----------------------+-----------------+----------------
postgres_get_av_diag | rds_tools | rds_superuser
(1 row)
```
# Functions of postgres\$1get\$1av\$1diag() in RDS for PostgreSQL
The `postgres_get_av_diag()` function retrieves diagnostic information about autovacuum processes that are blocking or lagging behind in a RDS for PostgreSQL database. The query needs to be executed in the database with the oldest transaction ID for accurate results. For more information about using the database with the oldest transaction ID, see [Not connected to the database with the age of oldest transaction ID](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.NOTICE.md)
```
SELECT
blocker,
DATABASE,
blocker_identifier,
wait_event,
TO_CHAR(autovacuum_lagging_by, 'FM9,999,999,999') AS autovacuum_lagging_by,
suggestion,
suggested_action
FROM (
SELECT
*
FROM
rds_tools.postgres_get_av_diag ()
ORDER BY
autovacuum_lagging_by DESC) q;
```
The `postgres_get_av_diag()` function returns a table with the following information:
**blocker**
Specifies the category of database activity that is blocking the vacuum.
+ [Active statement](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Active_statement)
+ [Idle in transaction](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Idle_in_transaction)
+ [Prepared transaction](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Prepared_transaction)
+ [Logical replication slot](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Logical_replication_slot)
+ [Read replica with physical replication slot](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Read_replicas)
+ [Read replica with streaming replication](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Read_replicas)
+ [Temporary tables](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Identifiableblockers.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Temporary_tables)
**database**
Specifies the name of the database where applicable and supported. This is the database in which the activity is ongoing and blocking or will block the autovacuum. This is the database you are required to connect to and take action.
**blocker\$1identifier**
Specifies the identifier of the activity that is blocking or will block the autovacuum. The identifier can be a process ID along with a SQL statement, a prepared transaction, an IP address of a read replica, and the name of the replication slot, either logical or physical.
**wait\$1event**
Specifies the [wait event](PostgreSQL.Tuning.md) of the blocking session and is applicable for the following blockers:
+ Active statement
+ Idle in transaction
**autovacum\$1lagging\$1by**
Specifies the number of transactions that autovacuum is lagging behind in its backlog work per category.
**suggestion**
Specifies suggestions to resolve the blocker. These instructions include the name of the database in which the activity exists where applicable, the Process ID (PID) of the session where applicable, and the action to be taken.
**suggested\$1action**
Suggests the action that needs to be taken to resolve the blocker.
# Resolving identifiable vacuum blockers in RDS for PostgreSQL
Autovacuum performs aggressive vacuums and lowers the age of transaction IDs to below the threshold specified by the `autovacuum_freeze_max_age` parameter of your RDS instance. You can track this age using the Amazon CloudWatch metric `MaximumUsedTransactionIDs`.
To find the setting of `autovacuum_freeze_max_age` (which has a default of 200 million transaction IDs) for your Amazon RDS instance, you can use the following query:
```
SELECT
TO_CHAR(setting::bigint, 'FM9,999,999,999') autovacuum_freeze_max_age
FROM
pg_settings
WHERE
name = 'autovacuum_freeze_max_age';
```
Note that `postgres_get_av_diag()` only checks for aggressive vacuum blockers when the age exceeds Amazon RDS’ [adaptive autovacuum](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum.AdaptiveAutoVacuuming) threshold of 500 million transaction IDs. For `postgres_get_av_diag()` to detect blockers, the blocker must be at least 500 million transactions old.
The `postgres_get_av_diag()` function identifies the following types of blockers:
**Topics**
+ [
## Active statement
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Active_statement)
+ [
## Idle in transaction
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Idle_in_transaction)
+ [
## Prepared transaction
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Prepared_transaction)
+ [
## Logical replication slot
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Logical_replication_slot)
+ [
## Read replicas
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Read_replicas)
+ [
## Temporary tables
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Temporary_tables)
## Active statement
In PostgreSQL, an active statement is an SQL statement that is currently being executed by the database. This includes queries, transactions, or any operations in progress. When monitoring via `pg_stat_activity`, the state column indicates that the process with the corresponding PID is active.
The `postgres_get_av_diag()` function displays output similar to the following when it identifies a statement that is an active statement.
```
blocker | Active statement
database | my_database
blocker_identifier | SELECT pg_sleep(20000);
wait_event | Timeout:PgSleep
autovacuum_lagging_by | 568,600,871
suggestion | Connect to database "my_database", review carefully and you may consider terminating the process using suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide.
suggested_action | {"SELECT pg_terminate_backend (29621);"}
```
**Suggested action**
Following the guidance in the `suggestion` column, the user can connect to the database where the active statement is present and, as specified in the `suggested_action` column, it's advisable to carefully review the option to terminate the session. If termination is safe, you may use the `pg_terminate_backend()` function to terminate the session. This action can be performed by an administrator (such as the RDS master account) or a user with the required `pg_terminate_backend()` privilege.
**Warning**
A terminated session will undo (`ROLLBACK`) changes it made. Depending on your requirements, you may want to rerun the statement. However, it is recommended to do so only after the autovacuum process has finished its aggressive vacuum operation.
## Idle in transaction
An idle in transaction statement refers to any session that has opened an explicit transaction (such as by issuing a `BEGIN` statement), performed some work, and is now waiting for the client to either pass more work or signal the end of the transaction by issuing a `COMMIT`, `ROLLBACK`, or `END` (which would result in an implicit `COMMIT`).
The `postgres_get_av_diag()` function displays output similar to the following when it identifies an `idle in transaction` statement as a blocker.
```
blocker | idle in transaction
database | my_database
blocker_identifier | INSERT INTO tt SELECT * FROM tt;
wait_event | Client:ClientRead
autovacuum_lagging_by | 1,237,201,759
suggestion | Connect to database "my_database", review carefully and you may consider terminating the process using suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide.
suggested_action | {"SELECT pg_terminate_backend (28438);"}
```
**Suggested action**
As indicated in the `suggestion` column, you can connect to the database where the idle in transaction session is present and terminate the session using the `pg_terminate_backend()` function. The user can be your admin (RDS master account) user or a user with the `pg_terminate_backend()` privilege.
**Warning**
A terminated session will undo (`ROLLBACK`) changes it made. Depending on your requirements, you may want to rerun the statement. However, it is recommended to do so only after the autovacuum process has finished its aggressive vacuum operation.
## Prepared transaction
PostgreSQL allows transactions that are part of a two-phase commit strategy called [prepared transactions](https://www.postgresql.org/docs/current/sql-prepare-transaction.html). These are enabled by setting the `max_prepared_transactions` parameter to a non-zero value. Prepared transactions are designed to ensure that a transaction is durable and remains available even after database crashes, restarts, or client disconnections. Like regular transactions, they are assigned a transaction ID and can affect the autovacuum. If left in a prepared state, autovacuum cannot perform freeezing and it can lead to transaction ID wraparound.
When transactions are left prepared indefinitely without being resolved by a transaction manager, they become orphaned prepared transactions. The only way to fix this is to either commit or rollback the transaction using the `COMMIT PREPARED` or `ROLLBACK PREPARED` commands, respectively.
**Note**
Be aware that a backup taken during a prepared transaction will still contain that transaction after restoration. Refer to the following information about how to locate and close such transactions.
The `postgres_get_av_diag()` function displays the following output when it identifies a blocker that is a prepared transaction.
```
blocker | Prepared transaction
database | my_database
blocker_identifier | myptx
wait_event | Not applicable
autovacuum_lagging_by | 1,805,802,632
suggestion | Connect to database "my_database" and consider either COMMIT or ROLLBACK the prepared transaction using suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide.
suggested_action | {"COMMIT PREPARED 'myptx';",[OR],"ROLLBACK PREPARED 'myptx';"}
```
**Suggested action**
As mentioned in the suggestion column, connect to the database where the prepared transaction is located. Based on the `suggested_action` column, carefully review whether to perform either `COMMIT` or `ROLLBACK`, and the the appropiate the action.
To monitor prepared transactions in general, PostgreSQL offers a catalog view called `pg_prepared_xacts`. You can use the following query to find prepared transactions.
```
SELECT
gid,
prepared,
owner,
database,
transaction AS oldest_xmin
FROM
pg_prepared_xacts
ORDER BY
age(transaction) DESC;
```
## Logical replication slot
The purpose of a replication slot is to hold unconsumed changes until they are replicated to a target server. For more information, see PostgreSQL's [Logical replication](https://www.postgresql.org/docs/current/logical-replication.html).
There are two types of logical replication slots.
**Inactive logical replication slots**
When replication is terminated, unconsumed transaction logs can't be removed, and the replication slot becomes inactive. Although an inactive logical replication slot isn't currently used by a subscriber, it remains on the server, leading to the retention of WAL files and preventing the removal of old transaction logs. This can increase disk usage and specifically block autovacuum from cleaning up internal catalog tables, as the system must preserve LSN information from being overwritten. If not addressed, this can result in catalog bloat, performance degradation, and an increased risk of wraparound vacuum, potentially causing transaction downtime.
**Active but slow logical replication slots**
Sometimes removal of dead tuples of catalog is delayed due to the performance degradation of logical replication. This delay in replication slows down updating the `catalog_xmin` and can lead to catalog bloat and wraparound vacuum.
The `postgres_get_av_diag()` function displays output similar to the following when it finds a logical replication slot as a blocker.
```
blocker | Logical replication slot
database | my_database
blocker_identifier | slot1
wait_event | Not applicable
autovacuum_lagging_by | 1,940,103,068
suggestion | Ensure replication is active and resolve any lag for the slot if active. If inactive, consider dropping it using the command in suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide.
suggested_action | {"SELECT pg_drop_replication_slot('slot1') FROM pg_replication_slots WHERE active = 'f';"}
```
**Suggested action**
To resolve this problem, check the replication configuration for issues with the target schema or data that might be terminating the apply process. The most common reasons are the following:
+ Missing columns
+ Incompatible data type
+ Data mismatch
+ Missing table
If the problem is related to infrastructure issues:
+ Network issues - [How do I resolve issues with an Amazon RDS DB in an incompatible network state?](https://repost.aws/knowledge-center/rds-incompatible-network).
+ Database or DB instance is not available due to the following reasons:
+ Replica instance is out of storage - Review [Amazon RDS DB instances run out of storage](https://repost.aws/knowledge-center/rds-out-of-storage) for information about adding storage.
+ Incompatible-parameters - Review [How can I fix an Amazon RDS DB instance that is stuck in the incompatible-parameters status?](https://repost.aws/knowledge-center/rds-incompatible-parameters) for more information about how you can resolve the issue.
If your instance is outside the AWS network or on AWS EC2, consult your administrator on how to resolve the availability or infrastructure-related issues.
**Dropping the inactive slot**
**Warning**
Caution: Before dropping a replication slot, carefully ensure that it has no ongoing replication, is inactive, and is in an unrecoverable state. Dropping a slot prematurely could disrupt replication or cause data loss.
After confirming that the replication slot is no longer needed, drop it to allow autovacuum to continue. The condition `active = 'f'` ensures that only an inactive slot is dropped.
```
SELECT pg_drop_replication_slot('slot1') WHERE active ='f'
```
## Read replicas
When the `hot_standby_feedback` setting is enabled for [Amazon RDS read replicas](USER_PostgreSQL.Replication.ReadReplicas.md), it prevents autovacuum on the primary database from removing dead rows that might still be needed by queries running on the read replica. This affects all types of physical read replicas including those managed with or without replication slots. This behavior is necessary because queries running on the standby replica require those rows to remain available on the primary preventing [query conflicts](https://www.postgresql.org/docs/current/hot-standby.html#HOT-STANDBY-CONFLICT) and cancellations.
**Read replica with physical replication slot**
Read replicas with physical replication slots significantly enhance the reliability and stability of replication in RDS for PostgreSQL. These slots ensure the primary database retains essential Write-Ahead Log files until the replica processes them, maintaining data consistency even during network disruptions.
Beginning with RDS for PostgreSQL version 14, all replicas utilize replication slots. In earlier versions, only cross-Region replicas used replication slots.
The `postgres_get_av_diag()` function displays output similar to the following when it finds a read replica with physical replication slot as the blocker.
```
blocker | Read replica with physical replication slot
database |
blocker_identifier | rds_us_west_2_db_xxxxxxxxxxxxxxxxxxxxx
wait_event | Not applicable
autovacuum_lagging_by | 554,080,689
suggestion | Run the following query on the replica "rds_us_west_2_db_xxxxxxxxxxxxxxxxxxxx" to find the long running query:
| SELECT * FROM pg_catalog.pg_stat_activity WHERE backend_xmin::text::bigint = 757989377;
| Review carefully and you may consdier terminating the query on read replica using suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide. + |
suggested_action | {"SELECT pg_terminate_backend(pid) FROM pg_catalog.pg_stat_activity WHERE backend_xmin::text::bigint = 757989377;"," +
| [OR] +
| ","Disable hot_standby_feedback"," +
| [OR] +
| ","Delete the read replica if not needed"}
```
**Read replica with streaming replication**
Amazon RDS allows setting up read replicas without a physical replication slot in older versions, up to version 13. This approach reduces overhead by allowing the primary to recycle WAL files more aggressively, which is advantageous in environments with limited disk space and can tolerate occasional ReplicaLag. However, without a slot, the standby must remain in sync to avoid missing WAL files. Amazon RDS uses archived WAL files to help the replica catch up if it falls behind, but this process requires careful monitoring and can be slow.
The `postgres_get_av_diag()` function displays output similar to the following when it finds a streaming read replica as the blocker.
```
blocker | Read replica with streaming replication slot
database | Not applicable
blocker_identifier | xx.x.x.xxx/xx
wait_event | Not applicable
autovacuum_lagging_by | 610,146,760
suggestion | Run the following query on the replica "xx.x.x.xxx" to find the long running query: +
| SELECT * FROM pg_catalog.pg_stat_activity WHERE backend_xmin::text::bigint = 348319343; +
| Review carefully and you may consdier terminating the query on read replica using suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide. +
|
suggested_action | {"SELECT pg_terminate_backend(pid) FROM pg_catalog.pg_stat_activity WHERE backend_xmin::text::bigint = 348319343;"," +
| [OR] +
| ","Disable hot_standby_feedback"," +
| [OR] +
| ","Delete the read replica if not needed"}
```
**Suggested action**
As recommended in the `suggested_action` column, carefully review these options to unblock autovacuum.
+ **Terminate the query** – Following the guidance in the suggestion column, you can connect to the read replica, as specified in the suggested\$1action column, it's advisable to carefully review the option to terminate the session. If termination is deemed safe, you may use the `pg_terminate_backend()` function to terminate the session. This action can be performed by an administrator (such as the RDS master account) or a user with the required pg\$1terminate\$1backend() privilege.
You may run the following SQL command on the read replica to terminate the query that is preventing the vacuum on the primary from cleaning up old rows. The value of `backend_xmin` is reported in the function’s output:
```
SELECT
pg_terminate_backend(pid)
FROM
pg_catalog.pg_stat_activity
WHERE
backend_xmin::text::bigint = backend_xmin;
```
+ **Disable hot standby feedback** – Consider disabling the `hot_standby_feedback` parameter if it's causing significant vacuum delays.
The `hot_standby_feedback` parameter allows a read replica to inform the primary about its query activity, preventing the primary from vacuuming tables or rows that are in use on the standby. While this ensures query stability on the standby, it can significantly delay vacuuming on the primary. Disabling this feature allows the primary to proceed with vacuuming without waiting for the standby to catch up. However, this can lead to query cancellations or failures on the standby if it attempts to access rows that have been vacuumed by the primary.
+ **Delete the read replica if not needed** – If the read replica is no longer necessary, you can delete it. This will remove the associated replication overhead and allow the primary to recycle transaction logs without being held back by the replica.
## Temporary tables
[Temporary tables](https://www.postgresql.org/docs/current/sql-createtable.html), created using the `TEMPORARY` keyword, reside in the temp schema, for example pg\$1temp\$1xxx, and are only accessible to the session that created them. Temporary tables are dropped when the session ends. However, these tables are invisible to PostgreSQL's autovacuum process, and must be manually vacuumed by the session that created them. Trying to vacuum the temp table from another session has no effect.
In unusual circumstances, a temporary table exists without an active session owning it. If the owning session ends unexpectedly due to a fatal crash, network issue, or similar event, the temporary table might not be cleaned up, leaving it behind as an "orphaned" table. When the PostgreSQL autovacuum process detects an orphaned temporary table, it logs the following message:
```
LOG: autovacuum: found orphan temp table \"%s\".\"%s\" in database \"%s\"
```
The `postgres_get_av_diag()` function displays output similar to the following when it identifies a temporary table as a blocker. For the function to correctly show the output related to temporary tables, it needs to be executed within the same database where those tables exist.
```
blocker | Temporary table
database | my_database
blocker_identifier | pg_temp_14.ttemp
wait_event | Not applicable
autovacuum_lagging_by | 1,805,802,632
suggestion | Connect to database "my_database". Review carefully, you may consider dropping temporary table using command in suggested_action. For more information, see Working with PostgreSQL autovacuum in the Amazon RDS User Guide.
suggested_action | {"DROP TABLE ttemp;"}
```
**Suggested action**
Follow the instructions provided in the `suggestion` column of the output to identify and remove the temporary table that is preventing autovacuum from running. Use the following command to drop the temporary table reported by `postgres_get_av_diag()`. Replace the table name based on the output provided by the `postgres_get_av_diag()` function.
```
DROP TABLE my_temp_schema.my_temp_table;
```
The following query can be used to identify temporary tables:
```
SELECT
oid,
relname,
relnamespace::regnamespace,
age(relfrozenxid)
FROM
pg_class
WHERE
relpersistence = 't'
ORDER BY
age(relfrozenxid) DESC;
```
# Resolving unidentifiable vacuum blockers in RDS for PostgreSQL
This section explores additional reasons that can prevent vacuuming from making progress. These issues are currently not directly identifiable by the `postgres_get_av_diag()` function.
**Topics**
+ [
## Invalid pages
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Invalid_pages)
+ [
## Index inconsistency
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Index_inconsistency)
+ [
## Exceptionally high transaction rate
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.High_transaction_rate)
## Invalid pages
An invalid page error occurs when PostgreSQL detects a mismatch in a page’s checksum while accessing that page. The contents are unreadable, preventing autovacuum from freezing tuples. This effectively stops the cleanup process. The following error is written into PostgreSQL’s log:
```
WARNING: page verification failed, calculated checksum YYYYY but expected XXXX
ERROR: invalid page in block ZZZZZ of relation base/XXXXX/XXXXX
CONTEXT: automatic vacuum of table myschema.mytable
```
**Determine the object type**
```
ERROR: invalid page in block 4305910 of relation base/16403/186752608
WARNING: page verification failed, calculated checksum 50065 but expected 60033
```
From the error message, the path `base/16403/186752608` provides the following information:
+ "base" is the directory name under the PostgreSQL data directory.
+ "16403" is the database OID, which you can look up in the `pg_database` system catalog.
+ "186752608" is the `relfilenode`, which you can use to look up the schema and object name in the `pg_class` system catalog.
By checking the output of the following query in the impacted database, you can determine the object type. The following query retrieves object information for oid: 186752608. Replace the OID with the one relevant to the error you encountered.
```
SELECT
relname AS object_name,
relkind AS object_type,
nspname AS schema_name
FROM
pg_class c
JOIN pg_namespace n ON c.relnamespace = n.oid
WHERE
c.oid = 186752608;
```
For more information, see the PostgreSQL documentation [https://www.postgresql.org/docs/current/catalog-pg-class.html](https://www.postgresql.org/docs/current/catalog-pg-class.html) for all the supported object types, noted by the `relkind` column in `pg_class`.
**Guidance**
The most effective solution for this issue depends on the configuration of your specific Amazon RDS instance and the type of data impacted by the inconsistent page.
**If the object type is an index:**
Rebuilding the index is recommended.
+ **Using the `CONCURRENTLY` option** – Prior to PostgreSQL version 12, rebuilding an index required an exclusive table lock, restricting access to the table. With PostgreSQL version 12, and later versions, the `CONCURRENTLY` option allows for row-level locking, significantly improving the table's availability. Following is the command:
```
REINDEX INDEX ix_name CONCURRENTLY;
```
While `CONCURRENTLY` is less disruptive, it can be slower on busy tables. Consider building the index during low-traffic periods if possible.
For more information, see the PostgreSQL [REINDEX](https://www.postgresql.org/docs/current/sql-reindex.html) documentation.
+ **Using the `INDEX_CLEANUP FALSE` option** – If the indexes are large and estimated to require a significant amount of time to finish, you can unblock autovacuum by executing a manual `VACUUM FREEZE` while excluding indexes. This functionality is available in PostgreSQL version 12 and later versions.
Bypassing indexes will allow you to skip the vacuum process of the inconsistent index and mitigate the wraparound issue. However, this will not resolve the underlying invalid page problem. To fully address and resolve the invalid page issue, you will still need to rebuild the index.
**If the object type is a materialized view:**
If an invalid page error occurs on a materialized view, login to the impacted database and refresh it to resolve the invalid page:
Refresh the materialized view:
```
REFRESH MATERIALIZED VIEW schema_name.materialized_view_name;
```
If refreshing fails, try recreating:
```
DROP MATERIALIZED VIEW schema_name.materialized_view_name;
CREATE MATERIALIZED VIEW schema_name.materialized_view_name AS query;
```
Refreshing or recreating the materialized view restores it without impacting the underlying table data.
**For all other object types:**
For all other object types, reach out to AWS support.
## Index inconsistency
A logically inconsistent index can prevent autovacuum from making progress. The following errors or similar errors are logged during either the vacuum phase of the index or when the index is accessed by SQL statements.
```
ERROR: right sibling's left-link doesn't match:block 5 links to 10 instead of expected 2 in index ix_name
```
```
ERROR: failed to re-find parent key in index "XXXXXXXXXX" for deletion target page XXX
CONTEXT: while vacuuming index index_name of relation schema.table
```
**Guidance**
Rebuild the index or skip indexes using `INDEX_CLEANUP` on manual `VACUUM FREEZE`. For information about how to rebuild the index, see [If the object type is an index](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Invalid_pages).
+ **Using the CONCURRENTLY option** – Prior to PostgreSQL version 12, rebuilding an index required an exclusive table lock, restricting access to the table. With PostgreSQL version 12, and later versions, the CONCURRENTLY option allows for row-level locking, significantly improving the table's availability. Following is the command:
```
REINDEX INDEX ix_name CONCURRENTLY;
```
While CONCURRENTLY is less disruptive, it can be slower on busy tables. Consider building the index during low-traffic periods if possible. For more information, see [REINDEX](https://www.postgresql.org/docs/current/sql-reindex.html) in *PostgreSQL* documentation.
+ **Using the INDEX\$1CLEANUP FALSE option** – If the indexes are large and estimated to require a significant amount of time to finish, you can unblock autovacuum by executing a manual VACUUM FREEZE while excluding indexes. This functionality is available in PostgreSQL version 12 and later versions.
Bypassing indexes will allow you to skip the vacuum process of the inconsistent index and mitigate the wraparound issue. However, this will not resolve the underlying invalid page problem. To fully address and resolve the invalid page issue, you will still need to rebuild the index.
## Exceptionally high transaction rate
In PostgreSQL, high transaction rates can significantly impact autovacuum's performance, leading to slower cleanup of dead tuples and increased risk of transaction ID wraparound. You can monitor the transaction rate by measuring the difference in `max(age(datfrozenxid))` between two time periods, typically per second. Additionally, you can use the following counter metrics from RDS Performance Insights to measure the transaction rate (the sum of xact\$1commit and xact\$1rollback) which is the total number of transactions.
| Counter | Type | Unit | Metric |
| --- | --- | --- | --- |
| xact\$1commit | Transactions | Commits per second | db.Transactions.xact\$1commit |
| xact\$1rollback | Transactions | Rollbacks per second | db.Transactions.xact\$1rollback |
A rapid increase indicates a high transaction load, which can overwhelm autovacuum, causing bloat, lock contention, and potential performance issues. This can negatively impact the autovacuum process in a couple of ways:
+ **Table Activity:** The specific table being vacuumed could be experiencing a high volume of transactions, causing delays.
+ **System Resources** The overall system might be overloaded, making it difficult for autovacuum to access the necessary resources to function efficiently.
Consider the following strategies for allowing autovacuum to operate more effectively and keep up with its tasks:
1. Reduce the transaction rate if possible. Consider to batch or group similar transactions where feasible.
1. Target frequently updated tables with manual `VACUUM FREEZE` operation nightly, weekly, or biweekly during off-peak hours.
1. Consider scaling up your instance class to allocate more system resources to handle the high transaction volume and autovacuum.
# Resolving vacuum performance issues in RDS for PostgreSQL
This section discusses factors that often contribute to slower vacuum performance and how to address these issues.
**Topics**
+ [
## Vacuum large indexes
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Large_indexes)
+ [
## Too many tables or databases to vacuum
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Multiple_tables)
+ [
## Aggressive vacuum (to prevent wraparound) is running
](#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Aggressive_vacuum)
## Vacuum large indexes
VACUUM operates through sequential phases: initialization, heap scanning, index and heap vacuuming, index cleanup, heap truncation, and final cleanup. During the heap scan, the process prunes pages, defragments and freezes them. After completing the heap scan, VACUUM cleans indexes, returns empty pages to the operating system, and performs final cleanup tasks like vacuuming the free space map and updating statistics.
Index vacuuming may require multiple passes when `maintenance_work_mem` (or `autovacuum_work_mem`) is insufficient to process the index. In PostgreSQL 16 and earlier, a 1 GB memory limit for storing dead tuple IDs often forced multiple passes on large indexes. PostgreSQL 17 introduces `TidStore`, which dynamically allocates memory instead of using a single-allocation array. This removes the 1 GB constraint, uses memory more efficiently, and reduces the need for multiple index scans per each index.
Large indexes may still require multiple passes in PostgreSQL 17 if available memory can't accommodate the entire index processing at once. Typically, larger indexes contain more dead tuples that require multiple passes.
**Detecting slow vacuum operations**
The `postgres_get_av_diag()` function can detect when vacuum operations are running slowly due to insufficient memory. For more information on this function, see [Installing autovacuum monitoring and diagnostic tools in RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Installation.md).
The `postgres_get_av_diag()` function issues the following notices when the available memory is not enough to complete the index vacuuming in a single pass.
**`rds_tools` 1.8**
```
NOTICE: Your database is currently running aggressive vacuum to prevent wraparound and it might be slow.
```
```
NOTICE: The current setting of autovacuum_work_mem is "XXX" and might not be sufficient. Consider increasing the setting, and if necessary, scaling up the Amazon RDS instance class for more memory.
Additionally, review the possibility of manual vacuum with exclusion of indexes using (VACUUM (INDEX_CLEANUP FALSE, VERBOSE TRUE) table_name;).
```
**`rds_tools` 1.9**
```
NOTICE: Your database is currently running aggressive vacuum to prevent wraparound and it might be slow.
```
```
NOTICE: The current setting of autovacuum_work_mem is XX might not be sufficient. Consider increasing the setting to XXX, and if necessary, scaling up the RDS instance class for more
memory. The suggested value is an estimate based on the current number of dead tuples for the table being vacuumed, which might not fully reflect the latest state. Additionally, review the possibility of manual
vacuum with exclusion of indexes using (VACUUM (INDEX_CLEANUP FALSE, VERBOSE TRUE) table_name;). For more information, see
[Working with PostgreSQL autovacuum in the Amazon Amazon RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Autovacuum.html)
.
```
**Note**
The `postgres_get_av_diag()` function relies on `pg_stat_all_tables.n_dead_tup` for estimating the amount of memory required for index vacuuming.
When the `postgres_get_av_diag()` function identifies a slow vacuum operation that requires multiple index scans due to insufficient `autovacuum_work_mem`, it will generate the following message:
```
NOTICE: Your vacuum is performing multiple index scans due to insufficient autovacuum_work_mem:XXX for index vacuuming.
For more information, see [Working with PostgreSQL autovacuum in the Amazon Amazon RDS User Guide](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Autovacuum.html).
```
**Guidance**
You can apply the following workarounds using manual `VACUUM FREEZE` to speed up freezing the table.
**Increase the memory for vacuuming**
As suggested by the `postgres_get_av_diag()` function, it's advisable to increase the `autovacuum_work_mem` parameter to address potential memory constraints at the instance level. While `autovacuum_work_mem` is a dynamic parameter, it's important to note that for the new memory setting to take effect, the autovacuum daemon needs to restart its workers. To accomplish this:
1. Confirm that the new setting is in place.
1. Terminate the processes currently running autovacuum.
This approach ensures that the adjusted memory allocation is applied to new autovacuum operations.
For more immediate results, consider manually performing a `VACUUM FREEZE` operation with an increased `maintenance_work_mem` setting within your session:
```
SET maintenance_work_mem TO '1GB';
VACUUM FREEZE VERBOSE table_name;
```
If you're using Amazon RDS and find that you need additional memory to support higher values for `maintenance_work_mem` or `autovacuum_work_mem`, consider upgrading to an instance class with more memory. This can provide the necessary resources to enhance both manual and automatic vacuum operations, leading to improved overall vacuum and database performance.
**Disable INDEX\$1CLEANUP**
Manual `VACUUM` in PostgreSQL version 12 and later allows skipping the index cleanup phase, while emergency autovacuum in PostgreSQL version 14 and later does this automatically based on the [https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-VACUUM-FAILSAFE-AGE](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-VACUUM-FAILSAFE-AGE) parameter.
**Warning**
Skipping index cleanup can lead to index bloat and negatively impact query performance. To mitigate this, consider reindexing or vacuuming affected indexes during a maintenance window.
For additional guidance on handling large indexes, refer to the documentation on [Managing autovacuum with large indexes](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.LargeIndexes.md).
**Parallel index vacuuming**
Starting with PostgreSQL 13, indexes can be vacuumed and cleaned in parallel by default using manual `VACUUM`, with one vacuum worker process assigned to each index. However, for PostgreSQL to determine if a vacuum operation qualifies for parallel execution, specific criteria must be met:
+ There must be at least two indexes.
+ The `max_parallel_maintenance_workers` parameter should be set to at least 2.
+ The index size must exceed the `min_parallel_index_scan_size` limit, which defaults to 512KB.
You can adjust the `max_parallel_maintenance_workers` setting based on the number of vCPUs available on your Amazon RDS instance and the number of indexes on the table to optimize vacuuming turnaround time.
For more information, see [Parallel vacuuming in Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL](https://aws.amazon.com/blogs/database/parallel-vacuuming-in-amazon-rds-for-postgresql-and-amazon-aurora-postgresql/).
## Too many tables or databases to vacuum
As mentioned in PostgreSQL's [The Autovacuum Daemon](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM') documentation, the autovacuum daemon operates through multiple processes. This includes a persistent autovacuum launcher responsible for starting autovacuum worker processes for each database within the system. The launcher schedules these workers to initiate approximately every `autovacuum_naptime` seconds per database.
With 'N' databases, a new worker begins roughly every [`autovacuum_naptime`/N seconds]. However, the total number of concurrent workers is limited by the `autovacuum_max_workers` setting. If the number of databases or tables requiring vacuuming exceeds this limit, the next database or table will be processed as soon as a worker becomes available.
When many large tables or databases require vacuuming concurrently, all available autovacuum workers can become occupied for an extended duration, delaying maintenance on other tables and databases. In environments with high transaction rates, this bottleneck can quickly escalate and potentially lead to wraparound vacuum issues within your Amazon RDS instance.
When `postgres_get_av_diag()` detects a high number of tables or databases, it provides the following recommendation:
```
NOTICE: Your database is currently running aggressive vacuum to prevent wraparound and it might be slow.
```
```
NOTICE: The current setting of autovacuum_max_workers:3 might not be sufficient. Consider increasing the setting and, if necessary, consider scaling up the Amazon RDS instance class for more workers.
```
**Guidance**
**Increase autovacuum\$1max\$1workers**
To expedite the vacuuming, we recommend adjusting the `autovacuum_max_workers` parameter to allow more concurrent autovacuum workers. If performance bottlenecks persist, consider scaling up your Amazon RDS instance to a class with more vCPUs, which can further improve the parallel processing capabilities.
## Aggressive vacuum (to prevent wraparound) is running
The age of the database (MaximumUsedTransactionIDs) in PostgreSQL only decreases when an aggressive vacuum (to prevent wraparound) is successfully completed. Until this vacuum finishes, the age will continue to increase depending on the transaction rate.
The `postgres_get_av_diag()` function generates the following `NOTICE` when it detects an aggressive vacuum. However, it only triggers this output after the vacuum has been active for at least two minutes.
```
NOTICE: Your database is currently running aggressive vacuum to prevent wraparound, monitor autovacuum performance.
```
For more information about aggressive vacuum, see [When an aggressive vacuum is already running](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.NOTICE.md).
You can verify if an aggressive vacuum is in progress with the following query:
```
SELECT
a.xact_start AS start_time,
v.datname "database",
a.query,
a.wait_event,
v.pid,
v.phase,
v.relid::regclass,
pg_size_pretty(pg_relation_size(v.relid)) AS heap_size,
(
SELECT
string_agg(pg_size_pretty(pg_relation_size(i.indexrelid)) || ':' || i.indexrelid::regclass || chr(10), ', ')
FROM
pg_index i
WHERE
i.indrelid = v.relid
) AS index_sizes,
trunc(v.heap_blks_scanned * 100 / NULLIF(v.heap_blks_total, 0)) AS step1_scan_pct,
v.index_vacuum_count || '/' || (
SELECT
count(*)
FROM
pg_index i
WHERE
i.indrelid = v.relid
) AS step2_vacuum_indexes,
trunc(v.heap_blks_vacuumed * 100 / NULLIF(v.heap_blks_total, 0)) AS step3_vacuum_pct,
age(CURRENT_TIMESTAMP, a.xact_start) AS total_time_spent_sofar
FROM
pg_stat_activity a
INNER JOIN pg_stat_progress_vacuum v ON v.pid = a.pid;
```
You can determine if it's an aggressive vacuum (to prevent wraparound) by checking the query column in the output. The phrase "to prevent wraparound" indicates that it is an aggressive vacuum.
```
query | autovacuum: VACUUM public.t3 (to prevent wraparound)
```
For example, suppose you have a blocker at transaction age 1 billion and a table requiring an aggressive vacuum to prevent wraparound at the same transaction age. Additionally, there's another blocker at transaction age 750 million. After clearing the blocker at transaction age 1 billion, the transaction age won't immediately drop to 750 million. It will remain high until the table needing the aggressive vacuum or any transaction with an age over 750 million is completed. During this period, the transaction age of your PostgreSQL cluster will continue to rise. Once the vacuum process is completed, the transaction age will drop to 750 million but will start increasing again until further vacuuming is finished. This cycle will continue as long as these conditions persist, until the transaction age eventually drops to the level configured for your Amazon RDS instance, specified by `autovacuum_freeze_max_age`.
# Explanation of the NOTICE messages in RDS for PostgreSQL
The `postgres_get_av_diag()` function provides the following NOTICE messages:
**When the age has not reached the monitoring threshold yet**
The monitoring threshold for `postgres_get_av_diag()` to identify blockers is 500 million transactions by default. If `postgres_get_av_diag()` generates the following NOTICE, it indicates that the transaction age has not yet reached this threshold.
```
NOTICE: postgres_get_av_diag() checks for blockers that prevent aggressive vacuums only, it does so only after exceeding dvb_threshold which is 500,000,000 and age of this PostgreSQL cluster is currently at 2.
```
**Not connected to the database with the age of oldest transaction ID**
The `postgres_get_av_diag()` function provides the most accurate output when connected to the database with the oldest transaction ID age. The database with the oldest transaction ID age reported by `postgres_get_av_diag()` will be different than “my\$1database” in your case. If you are not connected to the correct database, the following NOTICE is generated:
```
NOTICE: You are not connected to the database with the age of oldest transaction ID. Connect to my_database database and run postgres_get_av_diag() for accurate reporting.
```
Connecting to the database with the oldest transaction age is important for the following reasons:
+ **Identifying temporary table blockers:** Because the metadata for temporary tables is specific to each database, they are typically found in the database where they are created. However, if a temporary table happens to be the top blocker and resides in the database with the oldest transaction, this could be misleading. Connecting to the correct database ensures the accurate identification of the temporary table blocker.
+ **Diagnosing slow vacuums:** The index metadata and table count information are database-specific and necessary for diagnosing slow vacuum issues.
**Database with oldest transaction by age is on an rdsadmin or template0 database**
In certain cases, the `rdsadmin` or `template0` databases may be identified as the database with the oldest transaction ID age. If this happens, `postgres_get_av_diag()` will issue the following NOTICE:
```
NOTICE: The database with the age of oldest transaction ID is rdsadmin or template0, reach out to support if the reported blocker is in rdsadmin or template0.
```
Verify that the listed blocker is not originating from either of these two databases. If the blocker is reported to be present in either `rdsadmin` or `template0`, contact support as these databases are not user-accessible and require intervention.
It is highly unlikely for either the `rdsadmin` or `template0` database to contain a top blocker.
**When an aggressive vacuum is already running**
The `postgres_get_av_diag()` function is designed to report when an aggressive vacuum process is running, but it only triggers this output after the vacuum has been active for at least 1 minute. This intentional delay helps reduce the chances of false positives. By waiting, the function ensures that only effective, significant vacuums are reported, leading to more accurate and reliable monitoring of vacuum activity.
The `postgres_get_av_diag()` function generates the following NOTICE when it detects one or more aggressive vacuums in progress.
```
NOTICE: Your database is currently running aggressive vacuum to prevent wraparound, monitor autovacuum performance.
```
As indicated in the NOTICE, continue to monitor the performance of vacuum. For more information about aggressive vacuum see [Aggressive vacuum (to prevent wraparound) is running](Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Resolving_Performance.md#Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Aggressive_vacuum)
**When autovacuum is off**
The `postgres_get_av_diag()` function generates the following NOTICE if autovacuum is disabled on your database instance:
```
NOTICE: Autovacuum is OFF, we strongly recommend to enable it, no restart is necessary.
```
Autovacuum is a critical feature of your RDS for PostgreSQL DB instance that ensures smooth database operation. It automatically removes old row versions, reclaims storage space, and prevents table bloat, helping to keep tables and indexes efficient for optimal performance. Additionally, it protects against transaction ID wraparound, which can halt transactions on your Amazon RDS instance. Disabling autovacuum can lead to long-term declines in database performance and stability. We suggest you to keep it on all the times. For more information, see [Understanding autovacuum in RDS for PostgreSQL environments](https://aws.amazon.com/blogs/database/understanding-autovacuum-in-amazon-rds-for-postgresql-environments/).
Turning off autovacuum doesn't stop aggressive vacuums. These will still occur once your tables hit the `autovacuum_freeze_max_age` threshold.
**The number of transactions remaining is critically low**
The `postgres_get_av_diag()` function generates the following NOTICE when a wraparound vacuum is imminent. This NOTICE is issued when your Amazon RDS instance is 100 million transactions away from potentially rejecting new transactions.
```
WARNING: Number of transactions remaining is critically low, resolve issues with autovacuum or perform manual VACUUM FREEZE before your instance stops accepting transactions.
```
Your immediate action is required to avoid database downtime. You should closely monitor your vacuuming operations and consider manually initiating a `VACUUM FREEZE` on the affected database to prevent transaction failures.
# Managing high object counts in Amazon RDS for PostgreSQL
While PostgreSQL limitations are theoretical, having extremely high object counts in a database will cause noticeable performance impact to various operations. This documentation covers several common object types that, when having a high total count can lead to several possible impacts.
The following table provides a summary of object types and their potential impacts:
**Object types and potential impacts**
| Type of Object | Autovacuum | Logical Replication | Major Version Upgrade | pg\$1dump / pg\$1restore | General Performance | Instance Restart |
| --- | --- | --- | --- | --- | --- | --- |
| [Relations](#PostgreSQL.HighObjectCount.Relations) | x | | x | x | x | |
| [Temporary tables](#PostgreSQL.HighObjectCount.TempTables) | x | | | | x | |
| [Unlogged tables](#PostgreSQL.HighObjectCount.UnloggedTables) | | x | | | | x |
| [Partitions](#PostgreSQL.HighObjectCount.Partitions) | | | | | x | |
| [Temporary files](#PostgreSQL.HighObjectCount.TempFiles) | | | | | x | |
| [Sequences](#PostgreSQL.HighObjectCount.Sequences) | | x | | | | |
| [Large objects](#PostgreSQL.HighObjectCount.LargeObjects) | | x | x | | | |
## Relations
There is not a specific hard limit regarding the number of tables in a PostgreSQL database. The theoretical limit is extremely high, but there are other practical limits that need to be kept in mind during database design.
**Impact: Autovacuum falling behind**
Autovacuum can struggle to keep up with transaction ID growth or table bloat due to lack of workers compared to amount of work.
**Recommended action:** There are several factors for tuning autovacuum to keep up properly with a given number of tables and given workload. See [Best practices for working with PostgreSQL autovacuum](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Autovacuum.html) for suggestions on how to determine appropriate autovacuum settings. Use the [postgres\$1get\$1av\$1diag utility](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Autovacuum_Monitoring.Functions.html) to monitor problems with transaction ID growth.
**Impact: Major version upgrade / pg\$1dump and restore**
Amazon RDS uses the "--link" option during pg\$1upgrade execution to avoid having to make copies of datafiles, the schema metadata is still required to be restored into the new version of the database. Even with parallel pg\$1restore, if there are a significant number of relations this will increase the amount of downtime.
**Impact: General performance degradation**
General performance degradation due to catalog size. Each table and its associated columns will add to `pg_attribute`, `pg_class` and `pg_depend` tables which are frequently used in normal database operations. There won't be a specific wait event visible, but shared buffer efficiency will be impacted.
**Recommended action:** Regularly check table bloat for these specific tables and occasionally perform a `VACUUM FULL` on these specific tables. Be aware that `VACUUM FULL` on catalog tables requires an `ACCESS EXCLUSIVE` lock which means no other queries will be able to access them until the operation completes.
**Impact: File descriptor exhaustion**
Error: "out of file descriptors: Too many open files in system; release and retry". The PostgreSQL parameter `max_files_per_process` determines how many files each process can open. If there are a high number of connections joining a high number of tables, it is possible to hit this limit.
**Recommended action:**
+ Lowering the value of the parameter `max_files_per_process` may help alleviate this error. Each process and subprocess (for example, parallel query) can open this number of files, and if the queries are joining several tables, this limit can be exhausted.
+ Reduce the overall number of connections and use a connection pooler such as [Amazon RDS Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html) or other solutions such as PgBouncer. To learn more, see the [PgBouncer website](https://www.pgbouncer.org/).
**Impact: Inode exhaustion**
Error: "No space left on device". If this is observed when there is plenty of storage free space, this is caused by running out of inodes. [Amazon RDS Enhanced Monitoring](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Monitoring.OS.html) provides visibility for inodes in use and the maximum number available for your host.
**Approximate threshold:** [Millions](#PostgreSQL.HighObjectCount.Note)
## Temporary tables
Using temporary tables is useful for test data or intermediate results and is a common pattern seen in many database engines. The implications of heavy use in PostgreSQL must be understood to avoid some of the pitfalls. Each temporary table create and drop will add rows to system catalog tables, which when they become bloated, will cause general performance issues.
**Impact: Autovacuum falling behind**
Temporary tables aren't vacuumed by autovacuum but will hold on to transaction IDs during their existence and can lead to wraparound if not removed.
**Recommended action:** Temporary tables will live for the duration of the session that created them or can be dropped manually. A best practice of avoiding long-running transactions with temporary tables will prevent these tables from contributing to maximum used transaction ID growth.
**Impact: General performance degradation**
General performance degradation due to catalog size. When sessions continuously create and drop temporary tables, it will add to `pg_attribute`, `pg_class` and `pg_depend` tables which are frequently used in normal database operations. There won't be a specific wait event visible, but shared buffer efficiency will be impacted.
**Recommended action:**
+ Regularly check table bloat for these specific tables and occasionally perform a `VACUUM FULL` on these specific tables. Be aware that `VACUUM FULL` on catalog tables requires an `ACCESS EXCLUSIVE` lock which means no other queries will be able to access them until the operation completes.
+ If temporary tables are heavily used, prior to a major version upgrade, a `VACUUM FULL` of these specific catalog tables is highly recommended to reduce downtime.
**General best practices:**
+ Reduce the use of temporary tables by using common table expressions to produce intermediate results. These can sometimes complicate the queries needed, but will eliminate the impacts listed above.
+ Reuse temporary tables by using the `TRUNCATE` command to clear the contents instead of doing drop/create steps. This will also eliminate the problem of transaction ID growth caused by temporary tables.
**Approximate threshold:** [Tens of thousands](#PostgreSQL.HighObjectCount.Note)
## Unlogged tables
Unlogged tables can offer performance gains as they won't generate any WAL information. They must be used carefully as they offer no durability during database crash recovery as they will be truncated. This is an expensive operation in PostgreSQL as each unlogged table is truncated serially. While this operation is fast for a low number of unlogged tables, when they number in the thousands it can start to add notable delay during startup.
**Impact: Logical replication**
Unlogged tables are generally not included in logical replication, including [Blue/Green Deployments](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments.html), because logical replication relies on the WAL to capture and transfer changes.
**Impact: Extended downtime during recovery**
During any database state that involves database crash recovery such as Multi-AZ reboot with failover, Amazon RDS point-in-time recovery, and Amazon RDS major version upgrade, the serialized operation of truncating the unlogged tables will occur. This can lead to a much higher downtime experience than expected.
**Recommended action:**
+ Minimize the use of unlogged tables only to data which is acceptable to lose during database crash recovery operations.
+ Minimize the use of unlogged tables as the current behavior of serial truncation can cause startup of a database to take a significant amount of time.
**General best practices:**
+ Unlogged tables are not crash safe. Initiating a point-in-time recovery, which involves crash recovery, takes a significant time in PostgreSQL because this is a serial process that truncates each table.
**Approximate threshold:** [Thousands](#PostgreSQL.HighObjectCount.Note)
## Partitions
Partitioning can increase query performance and provide a logical organization of data. In ideal scenarios, partitioning is organized so that partition pruning can be used during query planning and execution. Using too many partitions can have negative impacts on query performance and database maintenance. The choice of how to partition a table should be made carefully, as the performance of query planning and execution can be negatively affected by poor design. See [PostgreSQL documentation](https://www.postgresql.org/docs/current/ddl-partitioning.html) for details about partitioning.
**Impact: General performance degradation**
Sometimes planning time overhead will increase and explain plans for your queries will become more complicated, making it difficult to identify tuning opportunities. For PostgreSQL versions earlier than 18, many partitions with high workload can lead to `LWLock:LockManager` waits.
**Recommended action:** Determine a minimum number of partitions that will allow you to complete both the organization of your data while at the same time providing performant query execution.
**Impact: Maintenance complexity**
Very high number of partitions will introduce maintenance difficulties like pre-creation and removal. Autovacuum will treat partitions as normal relations and have to perform regular cleanup, therefore requiring enough workers to complete the task.
**Recommended action:**
+ Ensure you precreate partitions so that workload isn't blocked when a new partition is needed (for example, monthly based partitions) and old partitions are rolled off.
+ Ensure you have enough autovacuum workers to perform normal cleanup maintenance of all partitions.
**Approximate threshold:** [Hundreds](#PostgreSQL.HighObjectCount.Note)
## Temporary files
Different than temporary tables mentioned above, temporary files are created by PostgreSQL when a complex query might perform several sort or hash operations at the same time, with each operation using instance memory to store results up to the value specified in the `work_mem` parameter. When the instance memory is not sufficient, temporary files are created to store the results. See [Managing temporary files](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.ManagingTempFiles.html) for more details on temporary files. If your workload generates high numbers of these files, there can be several impacts.
**Impact: File descriptor exhaustion**
Error: "out of file descriptors: Too many open files in system; release and retry". The PostgreSQL parameter `max_files_per_process` determines how many files each process can open. If there are a high number of connections joining a high number of tables, it is possible to hit this limit.
**Recommended action:**
+ Lowering the value of the parameter `max_files_per_process` may help alleviate this error. Each process and subprocess (for example, parallel query) can open this number of files, and if the queries are joining several tables, this limit can be exhausted.
+ Reduce the overall number of connections and use a connection pooler such as [Amazon RDS Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html) or other solutions such as PgBouncer. To learn more, see the [PgBouncer website](https://www.pgbouncer.org/).
**Impact: Inode exhaustion**
Error: "No space left on device". If this is observed when there is plenty of storage free space, this is caused by running out of inodes. [Amazon RDS Enhanced Monitoring](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Monitoring.OS.html) provides visibility for inodes in use and the maximum number available for your host.
**General best practices:**
+ Monitor your temp file usage with [Performance Insights](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PerfInsights.html).
+ Tune queries that are generating significant temporary files to see if it's possible to reduce the total number of temp files.
**Approximate threshold:** [Thousands](#PostgreSQL.HighObjectCount.Note)
## Sequences
Sequences are the underlying object used for auto-incrementing columns in PostgreSQL and they provide uniqueness and a key for the data. These can be used on individual tables with no consequence during normal operations with one exception of logical replication.
In PostgreSQL, logical replication does not currently replicate a sequence's current value to any subscriber. To learn more, see the [Restrictions page in PostgreSQL documentation](https://www.postgresql.org/docs/current/logical-replication-restrictions.html).
**Impact: Extended switchover time**
If you plan to use [Amazon RDS Blue/Green Deployments](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments.html) for any type of configuration change or upgrade, it is important to understand the impact of a high number of sequences on switchover. One of the last phases of a switchover will synchronize the current value of sequences, and if there are several thousand, this will increase the overall switchover time.
**Recommended action:** If your database workload would allow for the use of a shared UUID instead of a sequence-per-table approach, this would cut down on the synchronization step during a switchover.
**Approximate threshold:** [Thousands](#PostgreSQL.HighObjectCount.Note)
## Large objects
Large objects are stored in a single system table named pg\$1largeobject. Each large object also has an entry in the system table pg\$1largeobject\$1metadata. These objects are created, modified and cleaned up much differently than standard relations. Large objects are not handled by autovacuum and must be periodically cleaned up via a separate process called vacuumlo. See managing large objects with the lo module for examples on managing large objects.
**Impact: Logical replication**
Large objects are not currently replicated in PostgreSQL during logical replication. To learn more, see the [Restrictions page in PostgreSQL documentation](https://www.postgresql.org/docs/current/logical-replication-restrictions.html). In a [Blue/Green](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments.html) configuration, this means large objects in the blue environment aren't replicated to the green environment.
**Impact: Major version upgrade**
An upgrade can run out of memory and fail if there are millions of large objects and the instance cannot handle them during an upgrade. The PostgreSQL major version upgrade process comprises of two broad phases: dumping the schema via pg\$1dump and restoring it through pg\$1restore. If your database has millions of large objects you need to ensure your instance has sufficient memory to handle the pg\$1dump and pg\$1restore during an upgrade and scale it to a larger instance type.
**General best practices:**
+ Regularly use the vacuumlo utility to remove any orphaned large objects you may have.
+ Consider using the BYTEA datatype for storing your large objects in the database.
**Approximate threshold:** [Millions](#PostgreSQL.HighObjectCount.Note)
## Approximate thresholds
The approximate thresholds mentioned in this topic are only used to provide an estimate of how far a particular resource can scale. They represent the general range where the described impacts become more likely, but actual behavior depends on your specific workload, instance size, and configuration. While it may be possible to exceed these estimates, care and maintenance must be adhered to so as to avoid the impacts listed.
# Managing TOAST OID contention in Amazon RDS for PostgreSQL
TOAST (The Oversized-Attribute Storage Technique) is a PostgreSQL feature designed to handle large data values that exceed the typical 8KB database block size. PostgreSQL doesn't allow physical rows to span multiple blocks. The block size acts as an upper limit on row size. TOAST overcomes this restriction by splitting large field values into smaller chunks. It stores them separately in a dedicated TOAST table linked to the main table. For more information, see the [PostgreSQL TOAST storage mechanism and implementation documentation](https://www.postgresql.org/docs/current/storage-toast.html).
**Topics**
+ [
## Understanding TOAST operations
](#Appendix.PostgreSQL.CommonDBATasks.TOAST_OID.HowWorks)
+ [
## Identifying performance challenges
](#Appendix.PostgreSQL.CommonDBATasks.TOAST_OID.PerformanceChallenges)
+ [
## Recommendations
](#Appendix.PostgreSQL.CommonDBATasks.TOAST_OID.Recommendations)
+ [
## Monitoring
](#Appendix.PostgreSQL.CommonDBATasks.TOAST_OID.Monitoring)
## Understanding TOAST operations
TOAST performs compression and stores large field values out of line. TOAST assigns a unique OID (Object Identifier) to each chunk of oversized data stored in the TOAST table. The main table stores the TOAST value ID and relation ID on the page to reference the corresponding row in the TOAST table. This allows PostgreSQL to efficiently locate and manage these TOAST chunks. However, as the TOAST table grows, the system risks exhausting available OIDs, leading to both performance degradation and potential downtime due to OID depletion.
### Object identifiers in TOAST
An Object Identifier (OID) is a system-wide unique identifier used by PostgreSQL to reference database objects like tables, indexes, and functions. These identifiers play a vital role in PostgreSQL's internal operations, allowing the database to efficiently locate and manage objects.
For tables with eligible data sets for toasting, PostgreSQL assigns OIDs to uniquely identify each chunk of oversized data stored in the associated TOAST table. The system associates each chunk with a `chunk_id`, which helps PostgreSQL organize and locate these chunks efficiently within the TOAST table.
## Identifying performance challenges
PostgreSQL's OID management relies on a global 32-bit counter so that it wraps around after generating 4 billion unique values. While the database cluster shares this counter, OID allocation involves two steps during TOAST operations:
+ **Global counter for allocation** – The global counter assigns a new OID across the cluster.
+ **Local search for conflicts** – The TOAST table ensures the new OID does not conflict with existing OIDs already used in that specific table.
Performance degradation can occur when:
+ The TOAST table has high fragmentation or dense OID usage, leading to delays in assigning the OID.
+ The system frequently allocates and reuses OIDs in environments with high data churn or wide tables that use TOAST extensively.
For more information, see the [PostgreSQL TOAST table size limits and OID allocation documentation](https://wiki.postgresql.org/wiki/TOAST#Total_table_size_limit):
A global counter generates the OIDs and wraps around every 4 billion values, so that from time to time, the system generates an already-used value again. PostgreSQL detects that and tries again with the next OID. A slow INSERT could occur if there is a very long run of used OID values with no gaps in the TOAST table. These challenges become more pronounced as the OID space fills, leading to slower inserts and updates.
### Identifying the problem
+ Simple `INSERT` statements take significantly longer than usual in an inconsistent and random manner.
+ Delays occur only for `INSERT` and `UPDATE` statements involving TOAST operations.
+ The following log entries appear in PostgreSQL logs when the system struggles to find available OIDs in TOAST tables:
```
LOG: still searching for an unused OID in relation "pg_toast_20815"
DETAIL: OID candidates have been checked 1000000 times, but no unused OID has been found yet.
```
+ Performance Insights indicates a high number of average active sessions (AAS) associated with `LWLock:buffer_io` and `LWLock:OidGenLock` wait events.
You can run the following SQL query to identify long-running INSERT transactions with wait events:
```
SELECT
datname AS database_name,
usename AS database_user,
pid,
now() - pg_stat_activity.xact_start AS transaction_duration,
concat(wait_event_type, ':', wait_event) AS wait_event,
substr(query, 1, 30) AS TRANSACTION,
state
FROM
pg_stat_activity
WHERE (now() - pg_stat_activity.xact_start) > INTERVAL '60 seconds'
AND state IN ('active', 'idle in transaction', 'idle in transaction (aborted)', 'fastpath function call', 'disabled')
AND pid <> pg_backend_pid()
AND lower(query) LIKE '%insert%'
ORDER BY
transaction_duration DESC;
```
Example query results displaying INSERT operations with extended wait times:
```
database_name | database_user | pid | transaction_duration | wait_event | transaction | state
---------------+-----------------+-------+----------------------+---------------------+--------------------------------+--------
postgres | db_admin_user| 70965 | 00:10:19.484061 | LWLock:buffer_io | INSERT INTO "products" (......... | active
postgres | db_admin_user| 69878 | 00:06:14.976037 | LWLock:buffer_io | INSERT INTO "products" (......... | active
postgres | db_admin_user| 68937 | 00:05:13.942847 | : | INSERT INTO "products" (......... | active
```
### Isolating the problem
+ **Test small insert** – Insert a record smaller than the `toast_tuple_target` threshold. Remember that compression is applied before TOAST storage. If this operates without performance issues, the problem is related to TOAST operations.
+ **Test new table** – Create a new table with the same structure and insert a record larger than `toast_tuple_target`. If this works without issues, the problem is localized to the original table's OID allocation.
## Recommendations
The following approaches can help resolve TOAST OID contention issues.
+ **Data cleanup and archive** – Review and delete any obsolete or unnecessary data to free up OIDs for future use, or archive the data. Consider the following limitations:
+ Limited scalability, as future cleanup might not always be possible.
+ Possible long-running VACUUM operation to remove the resulting dead tuples.
+ **Write to a new table** – Create a new table for future inserts and use a `UNION ALL` view to combine old and new data for queries. This view presents the combined data from both old and new tables, allowing queries to access them as a single table. Consider the following limitations:
+ Updates on the old table might still cause OID exhaustion.
+ **Partition or Shard** – Partition the table or shard data for better scalability and performance. Consider the following limitations:
+ Increased complexity in query logic and maintenance, potential need for application changes to handle partitioned data correctly.
## Monitoring
### Using system tables
You can use PostgreSQL's system tables to monitor growth of OID usage.
**Warning**
Depending on the number of OIDs in the TOAST table, it may take time to complete. We recommend that you schedule monitoring during off-business hours to minimize impact.
The following anonymous block counts the number of distinct OIDs used in each TOAST table and displays the parent table information:
```
DO $$
DECLARE
r record;
o bigint;
parent_table text;
parent_schema text;
BEGIN
SET LOCAL client_min_messages TO notice;
FOR r IN
SELECT
c.oid,
c.oid::regclass AS toast_table
FROM
pg_class c
WHERE
c.relkind = 't'
AND c.relowner != 10 LOOP
-- Fetch the number of distinct used OIDs (chunk IDs) from the TOAST table
EXECUTE 'SELECT COUNT(DISTINCT chunk_id) FROM ' || r.toast_table INTO o;
-- If there are used OIDs, find the associated parent table and its schema
IF o <> 0 THEN
SELECT
n.nspname,
c.relname INTO parent_schema,
parent_table
FROM
pg_class c
JOIN pg_namespace n ON c.relnamespace = n.oid
WHERE
c.reltoastrelid = r.oid;
-- Raise a concise NOTICE message
RAISE NOTICE 'Parent schema: % | Parent table: % | Toast table: % | Number of used OIDs: %', parent_schema, parent_table, r.toast_table, TO_CHAR(o, 'FM9,999,999,999,999');
END IF;
END LOOP;
END
$$;
```
Example output displaying OID usage statistics by TOAST table:
```
NOTICE: Parent schema: public | Parent table: my_table | Toast table: pg_toast.pg_toast_16559 | Number of used OIDs: 45,623,317
NOTICE: Parent schema: public | Parent table: my_table1 | Toast table: pg_toast.pg_toast_45639925 | Number of used OIDs: 10,000
NOTICE: Parent schema: public | Parent table: my_table2 | Toast table: pg_toast.pg_toast_45649931 | Number of used OIDs: 1,000,000
DO
```
The following anonymous block retrieves the maximum assigned OID for each non-empty TOAST table:
```
DO $$
DECLARE
r record;
o bigint;
parent_table text;
parent_schema text;
BEGIN
SET LOCAL client_min_messages TO notice;
FOR r IN
SELECT
c.oid,
c.oid::regclass AS toast_table
FROM
pg_class c
WHERE
c.relkind = 't'
AND c.relowner != 10 LOOP
-- Fetch the max(chunk_id) from the TOAST table
EXECUTE 'SELECT max(chunk_id) FROM ' || r.toast_table INTO o;
-- If there's at least one TOASTed chunk, find the associated parent table and its schema
IF o IS NOT NULL THEN
SELECT
n.nspname,
c.relname INTO parent_schema,
parent_table
FROM
pg_class c
JOIN pg_namespace n ON c.relnamespace = n.oid
WHERE
c.reltoastrelid = r.oid;
-- Raise a concise NOTICE message
RAISE NOTICE 'Parent schema: % | Parent table: % | Toast table: % | Max chunk_id: %', parent_schema, parent_table, r.toast_table, TO_CHAR(o, 'FM9,999,999,999,999');
END IF;
END LOOP;
END
$$;
```
Example output displaying maximum chunk IDs for TOAST tables:
```
NOTICE: Parent schema: public | Parent table: my_table | Toast table: pg_toast.pg_toast_16559 | Max chunk_id: 45,639,907
NOTICE: Parent schema: public | Parent table: my_table1 | Toast table: pg_toast.pg_toast_45639925 | Max chunk_id: 45,649,929
NOTICE: Parent schema: public | Parent table: my_table2 | Toast table: pg_toast.pg_toast_45649931 | Max chunk_id: 46,649,935
DO
```
### Using Performance Insights
The wait events `LWLock:buffer_io` and `LWLock:OidGenLock` appear in Performance Insights during operations that require assigning new Object Identifiers (OIDs). High Average Active Sessions (AAS) for these events typically point to contention during OID assignment and resource management. This is particularly common in environments with high data churn, extensive large data usage, or frequent object creation.
#### LWLock:buffer\$1io
`LWLock:buffer_io` is a wait event that occurs when a PostgreSQL session is waiting for I/O operations on a shared buffer to complete. This typically happens when the database reads data from disk into memory or writes modified pages from memory to disk. The `BufferIO` wait event ensures consistency by preventing multiple processes from accessing or modifying the same buffer while I/O operations are in progress. High occurrences of this wait event may indicate disk bottlenecks or excessive I/O activity in the database workload.
During TOAST operations:
+ PostgreSQL allocates OIDs for large objects and ensures their uniqueness by scanning the TOAST table's index.
+ Large TOAST indexes may require accessing multiple pages to verify OID uniqueness. This results in increased disk I/O, especially when the buffer pool cannot cache all required pages.
The size of the index directly affects the number of buffer pages that need to be accessed during these operations. Even if the index is not bloated, its sheer size can increase buffer I/O, particularly in high-concurrency or high-churn environments. For more information, see [LWLock:BufferIO wait event troubleshooting guide](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/apg-waits.lwlockbufferio.html).
#### LWLock:OidGenLock
`OidGenLock` is a wait event that occurs when a PostgreSQL session is waiting to allocate a new object identifier (OID). This lock ensures that OIDs are generated sequentially and safely, allowing only one process to generate OIDs at a time.
During TOAST operations:
+ **OID allocation for chunks in TOAST table** – PostgreSQL assigns OIDs to chunks in TOAST tables when managing large data records. Each OID must be unique to prevent conflicts in the system catalog.
+ **High concurrency** – Since access to OID generator is sequential, when multiple sessions are concurrently creating objects that require OIDs, contention for `OidGenLock` can occur. This increases the likelihood of sessions waiting for OID allocation to complete.
+ **Dependency on system catalog access** – Allocating OIDs requires updates to shared system catalog tables like `pg_class` and `pg_type`. If these tables experience heavy activity (due to frequent DDL operations), it can increase lock contention for `OidGenLock`.
+ **High OID allocation demand** – TOAST heavy workloads with large data records require constant OID allocation, increasing contention.
Additional factors that increase OID contention:
+ **Frequent object creation** – Workloads that frequently create and drop objects, such as temporary tables, amplify contention on the global OID counter.
+ **Global counter locking** – The global OID counter is accessed serially to ensure uniqueness, creating a single point of contention in high-concurrency environments.
## Working with logging mechanisms supported by RDS for PostgreSQL
There are several parameters, extensions, and other configurable items that you can set to log activities that occur on your PostgreSQL DB instance. These include the following:
+ The `log_statement` parameter can be used to log user activity in your PostgreSQL database. To learn more about RDS for PostgreSQL logging and how to monitor the logs, see [ RDS for PostgreSQL database log files](USER_LogAccess.Concepts.PostgreSQL.md).
+ The `rds.force_admin_logging_level` parameter logs actions by the Amazon RDS internal user (rdsadmin) in the databases on the DB instance. It writes the output to the PostgreSQL error log. Allowed values are `disabled`, `debug5`, `debug4`, `debug3`, `debug2`, `debug1`, `info`, `notice`, `warning`, `error`, log, `fatal`, and `panic`. The default value is `disabled`.
+ The `rds.force_autovacuum_logging_level` parameter can be set to capture various autovacuum operations in the PostgreSQL error log. For more information, see [Logging autovacuum and vacuum activities](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.Logging.md).
+ The PostgreSQL Audit (pgAudit) extension can be installed and configured to capture activities at the session level or at the object level. For more information, see [Using pgAudit to log database activity](Appendix.PostgreSQL.CommonDBATasks.pgaudit.md).
+ The `log_fdw` extension makes it possible for you to access the database engine log using SQL. For more information, see [Using the log\$1fdw extension to access the DB log using SQL](CHAP_PostgreSQL.Extensions.log_fdw.md).
+ The `pg_stat_statements` library is specified as the default for the `shared_preload_libraries` parameter in RDS for PostgreSQL version 10 and higher. It's this library that you can use to analyze running queries. Be sure that `pg_stat_statements` is set in your DB parameter group. For more information about monitoring your RDS for PostgreSQL DB instance using the information that this library provides, see [SQL statistics for RDS PostgreSQL](USER_PerfInsights.UsingDashboard.AnalyzeDBLoad.AdditionalMetrics.PostgreSQL.md).
+ The `log_hostname` parameter captures to the log the hostname of each client connection. For RDS for PostgreSQL version 12 and higher versions, this parameter is set to `off` by default. If you turn it on, be sure to monitor session connection times. When turned on, the service uses the domain name system (DNS) reverse lookup request to get the hostname of the client that's making the connection and add it to the PostgreSQL log. This has a noticeable impact during session connection. We recommend that you turn on this parameter for troubleshooting purposes only.
In general terms, the point of logging is so that the DBA can monitor, tune performance, and troubleshoot. Many of the logs are uploaded automatically to Amazon CloudWatch or Performance Insights. Here, they're sorted and grouped to provide complete metrics for your DB instance. To learn more about Amazon RDS monitoring and metrics, see [Monitoring metrics in an Amazon RDS instance](CHAP_Monitoring.md).
# Managing temporary files with PostgreSQL
In PostgreSQL, a complex query might perform several sort or hash operations at the same time, with each operation using instance memory to store results up to the value specified in the [https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM) parameter. When the instance memory is not sufficient, temporary files are created to store the results. These are written to disk to complete the query execution. Later, these files are automatically removed after the query completes. In RDS for PostgreSQL, these files are stored in Amazon EBS on the data volume. For more information, see [Amazon RDS DB instance storage](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html). You can monitor the `FreeStorageSpace` metric published in CloudWatch to make sure that your DB instance has enough free storage space. For more information, see [https://repost.aws/knowledge-center/storage-full-rds-cloudwatch-alarm](https://repost.aws/knowledge-center/storage-full-rds-cloudwatch-alarm).
We recommend using Amazon RDS Optimized Read instances for workloads involving multiple concurrent queries that increase the usage of temporary files. These instances use local Non-Volatile Memory Express (NVMe) based solid state drive (SSD) block-level storage to place the temporary files. For more information, see [Improving query performance for RDS for PostgreSQL with Amazon RDS Optimized Reads](USER_PostgreSQL.optimizedreads.md).
You can use the following parameters and functions to manage the temporary files in your instance.
+ **[https://www.postgresql.org/docs/current/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-DISK](https://www.postgresql.org/docs/current/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-DISK)** – This parameter cancels any query exceeding the size of temp\$1files in KB. This limit prevents any query from running endlessly and consuming disk space with temporary files. You can estimate the value using the results from the `log_temp_files` parameter. As a best practice, examine the workload behavior and set the limit according to the estimation. The following example shows how a query is canceled when it exceeds the limit.
```
postgres=>select * from pgbench_accounts, pg_class, big_table;
```
```
ERROR: temporary file size exceeds temp_file_limit (64kB)
```
+ **[https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-TEMP-FILES](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-TEMP-FILES)** – This parameter sends messages to the postgresql.log when the temporary files of a session are removed. This parameter produces logs after a query successfully completes. Therefore, it might not help in troubleshooting active, long-running queries.
The following example shows that when the query successfully completes, the entries are logged in the postgresql.log file while the temporary files are cleaned up.
```
2023-02-06 23:48:35 UTC:205.251.233.182(12456):adminuser@postgres:[31236]:LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp31236.5", size 140353536
2023-02-06 23:48:35 UTC:205.251.233.182(12456):adminuser@postgres:[31236]:STATEMENT: select a.aid from pgbench_accounts a, pgbench_accounts b where a.bid=b.bid order by a.bid limit 10;
2023-02-06 23:48:35 UTC:205.251.233.182(12456):adminuser@postgres:[31236]:LOG: temporary file: path "base/pgsql_tmp/pgsql_tmp31236.4", size 180428800
2023-02-06 23:48:35 UTC:205.251.233.182(12456):adminuser@postgres:[31236]:STATEMENT: select a.aid from pgbench_accounts a, pgbench_accounts b where a.bid=b.bid order by a.bid limit 10;
```
+ **[https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-GENFILE](https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-GENFILE)** – This function that is available from RDS for PostgreSQL 13 and above provides visibility into the current temporary file usage. The completed query doesn't appear in the results of the function. In the following example, you can view the results of this function.
```
postgres=>select * from pg_ls_tmpdir();
```
```
name | size | modification
-----------------+------------+------------------------
pgsql_tmp8355.1 | 1072250880 | 2023-02-06 22:54:56+00
pgsql_tmp8351.0 | 1072250880 | 2023-02-06 22:54:43+00
pgsql_tmp8327.0 | 1072250880 | 2023-02-06 22:54:56+00
pgsql_tmp8351.1 | 703168512 | 2023-02-06 22:54:56+00
pgsql_tmp8355.0 | 1072250880 | 2023-02-06 22:54:00+00
pgsql_tmp8328.1 | 835031040 | 2023-02-06 22:54:56+00
pgsql_tmp8328.0 | 1072250880 | 2023-02-06 22:54:40+00
(7 rows)
```
```
postgres=>select query from pg_stat_activity where pid = 8355;
query
----------------------------------------------------------------------------------------
select a.aid from pgbench_accounts a, pgbench_accounts b where a.bid=b.bid order by a.bid
(1 row)
```
The file name includes the processing ID (PID) of the session that generated the temporary file. A more advanced query, such as in the following example, performs a sum of the temporary files for each PID.
```
postgres=>select replace(left(name, strpos(name, '.')-1),'pgsql_tmp','') as pid, count(*), sum(size) from pg_ls_tmpdir() group by pid;
```
```
pid | count | sum
------+-------------------
8355 | 2 | 2144501760
8351 | 2 | 2090770432
8327 | 1 | 1072250880
8328 | 2 | 2144501760
(4 rows)
```
+ **`[ pg\$1stat\$1statements](https://www.postgresql.org/docs/current/pgstatstatements.html)`** – If you activate the pg\$1stat\$1statements parameter, then you can view the average temporary file usage per call. You can identify the query\$1id of the query and use it to examine the temporary file usage as shown in the following example.
```
postgres=>select queryid from pg_stat_statements where query like 'select a.aid from pgbench%';
```
```
queryid
----------------------
-7170349228837045701
(1 row)
```
```
postgres=>select queryid, substr(query,1,25), calls, temp_blks_read/calls temp_blks_read_per_call, temp_blks_written/calls temp_blks_written_per_call from pg_stat_statements where queryid = -7170349228837045701;
```
```
queryid | substr | calls | temp_blks_read_per_call | temp_blks_written_per_call
----------------------+---------------------------+-------+-------------------------+----------------------------
-7170349228837045701 | select a.aid from pgbench | 50 | 239226 | 388678
(1 row)
```
+ **`[Performance Insights](https://aws.amazon.com/rds/performance-insights/)`** – In the Performance Insights dashboard, you can view temporary file usage by turning on the metrics **temp\$1bytes** and **temp\$1files**. Then, you can see the average of both of these metrics and see how they correspond to the query workload. The view within Performance Insights doesn't show specifically the queries that are generating the temporary files. However, when you combine Performance Insights with the query shown for `pg_ls_tmpdir`, you can troubleshoot, analyze, and determine the changes in your query workload.
For more information about how to analyze metrics and queries with Performance Insights, see [Analyzing metrics with the Performance Insights dashboard](USER_PerfInsights.UsingDashboard.md).
For an example of viewing temporary file usage with Performance Insights, see [Viewing temporary file usage with Performance Insights](PostgreSQL.ManagingTempFiles.Example.md)
# Viewing temporary file usage with Performance Insights
You can use Performance Insights to view temporary file usage by turning on the metrics **temp\$1bytes** and **temp\$1files**. The view in Performance Insights doesn't show the specific queries that generate temporary files, however, when you combine Performance Insights with the query shown for `pg_ls_tmpdir`, you can troubleshoot, analyze, and determine the changes in your query workload.
1. In the Performance Insights dashboard, choose **Manage Metrics**.
1. Choose **Database metrics**, and select the **temp\$1bytes** and **temp\$1files** metrics as shown in the following image.
![\[Metrics displayed in the graph.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/rpg_mantempfiles_metrics.png)
1. In the **Top SQL** tab, choose the **Preferences** icon.
1. In the **Preferences** window, turn on the following statistics to appear in the **Top SQL**tab and choose **Continue**.
+ Temp writes/sec
+ Temp reads/sec
+ Tmp blk write/call
+ Tmp blk read/call
1. The temporary file is broken out when combined with the query shown for `pg_ls_tmpdir`, as shown in the following example.
![\[Query that displays the temporary file usage.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/rpg_mantempfiles_query.png)
The `IO:BufFileRead` and `IO:BufFileWrite` events occur when the top queries in your workload often create temporary files. You can use Performance Insights to identify top queries waiting on `IO:BufFileRead` and `IO:BufFileWrite` by reviewing Average Active Session (AAS) in Database Load and Top SQL sections.
![\[IO:BufFileRead and IO:BufFileWrite in the graph.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/perfinsights_IOBufFile.png)
For more information on how to analyze top queries and load by wait event with Performance Insights, see [Overview of the Top SQL tab](USER_PerfInsights.UsingDashboard.AnalyzeDBLoad.AdditionalMetrics.md#USER_PerfInsights.UsingDashboard.Components.AvgActiveSessions.TopLoadItemsTable.TopSQL). You should identify and tune the queries that cause increase in temporary file usage and related wait events. For more information on these wait events and remediation, see [ IO:BufFileRead and IO:BufFileWrite](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/wait-event.iobuffile.html).
**Note**
The [https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM) parameter controls when the sort operation runs out of memory and results are written into temporary files. We recommend that you don't change the setting of this parameter higher than the default value because it would permit every database session to consume more memory. Also, a single session that performs complex joins and sorts can perform parallel operations in which each operation consumes memory.
As a best practice, when you have a large report with multiple joins and sorts, set this parameter at the session level by using the `SET work_mem` command. Then the change is only applied to the current session and doesn't change the value globally.
## Using pgBadger for log analysis with PostgreSQL
You can use a log analyzer such as [pgBadger](http://dalibo.github.io/pgbadger/) to analyze PostgreSQL logs. The pgBadger documentation states that the %l pattern (the log line for the session or process) should be a part of the prefix. However, if you provide the current RDS `log_line_prefix` as a parameter to pgBadger it should still produce a report.
For example, the following command correctly formats an Amazon RDS for PostgreSQL log file dated 2014-02-04 using pgBadger.
```
./pgbadger -f stderr -p '%t:%r:%u@%d:[%p]:' postgresql.log.2014-02-04-00
```
## Using PGSnapper for monitoring PostgreSQL
You can use PGSnapper to assist with periodic collection of Amazon RDS for PostgreSQL performance-related statistics and metrics. For more information, see[ Monitor Amazon RDS for PostgreSQL performance using PGSnapper](https://aws.amazon.com/blogs/database/monitor-amazon-rds-for-postgresql-and-amazon-aurora-postgresql-performance-using-pgsnapper/).
# Managing custom casts in RDS for PostgreSQL
**Type casting** in PostgreSQL is the process of converting a value from one data type to another. PostgreSQL provides built-in casts for many common conversions, but you can also create custom casts to define how specific type conversions should behave.
A cast specifies how to perform a conversion from one data type to another. For example, converting text `'123'` to integer `123`, or numeric `45.67` to text `'45.67'`.
For comprehensive information about PostgreSQL casting concepts and syntax, refer to the [PostgreSQL CREATE CAST Documentation](https://www.postgresql.org/docs/current/sql-createcast.html).
Starting with RDS for PostgreSQL versions 13.23, 14.20, 15.15, 16.11, 17.7, and 18.1, you can use the rds\$1casts extension to install additional casts for built-in types, while still being able to create your own casts for custom types.
**Topics**
+ [
## Installing and using the rds\$1casts extension
](#PostgreSQL.CustomCasts.Installing)
+ [
## Supported casts
](#PostgreSQL.CustomCasts.Supported)
+ [
## Creating or dropping casts
](#PostgreSQL.CustomCasts.Creating)
+ [
## Creating custom casts with proper context strategy
](#PostgreSQL.CustomCasts.BestPractices)
## Installing and using the rds\$1casts extension
To create the `rds_casts` extension, connect to your RDS for PostgreSQL DB instance as an `rds_superuser` and run the following command:
```
CREATE EXTENSION IF NOT EXISTS rds_casts;
```
## Supported casts
Create the extension in each database where you want to use custom casts. After creating the extension, use the following command to view all available casts:
```
SELECT * FROM rds_casts.list_supported_casts();
```
This function lists the available cast combinations (source type, target type, coercion context, and cast function). For example, if you want to create `text` to `numeric` as an `implicit` cast. You can use the following query to find if the cast is available to create:
```
SELECT * FROM rds_casts.list_supported_casts()
WHERE source_type = 'text' AND target_type = 'numeric';
id | source_type | target_type | qualified_function | coercion_context
----+-------------+-------------+--------------------------------------+------------------
10 | text | numeric | rds_casts.rds_text_to_numeric_custom | implicit
11 | text | numeric | rds_casts.rds_text_to_numeric_custom | assignment
13 | text | numeric | rds_casts.rds_text_to_numeric_custom | explicit
20 | text | numeric | rds_casts.rds_text_to_numeric_inout | implicit
21 | text | numeric | rds_casts.rds_text_to_numeric_inout | assignment
23 | text | numeric | rds_casts.rds_text_to_numeric_inout | explicit
```
The rds\$1casts extension provides two types of conversion functions for each cast:
+ *\$1inout functions* - Use PostgreSQL's standard I/O conversion mechanism, behaving identically to casts created with the INOUT method
+ *\$1custom functions* - Provide enhanced conversion logic that handles edge cases, such as converting empty strings to NULL values to avoid conversion errors
The `inout` functions replicate PostgreSQL's native casting behavior, while `custom` functions extend this functionality by handling scenarios that standard INOUT casts cannot accommodate, such as converting empty strings to integers.
## Creating or dropping casts
You can create and drop supported casts using two methods:
### Cast creation
**Method 1: Using native CREATE CAST command**
```
CREATE CAST (text AS numeric)
WITH FUNCTION rds_casts.rds_text_to_numeric_custom
AS IMPLICIT;
```
**Method 2: Using the rds\$1casts.create\$1cast function**
```
SELECT rds_casts.create_cast(10);
```
The `create_cast` function takes the ID from the `list_supported_casts()` output. This method is simpler and ensures you're using the correct function and context combination. This id is guaranteed to remain the same across different postgres versions.
To verify the cast was created successfully, query the pg\$1cast system catalog:
```
SELECT oid, castsource::regtype, casttarget::regtype, castfunc::regproc, castcontext, castmethod
FROM pg_cast
WHERE castsource = 'text'::regtype AND casttarget = 'numeric'::regtype;
oid | castsource | casttarget | castfunc | castcontext | castmethod
--------+------------+------------+--------------------------------------+-------------+------------
356372 | text | numeric | rds_casts.rds_text_to_numeric_custom | i | f
```
The `castcontext` column shows: `e` for EXPLICIT, `a` for ASSIGNMENT, or `i` for IMPLICIT.
### Dropping casts
**Method 1: Using DROP CAST command**
```
DROP CAST IF EXISTS (text AS numeric);
```
**Method 2: Using the rds\$1casts.drop\$1cast function**
```
SELECT rds_casts.drop_cast(10);
```
The `drop_cast` function takes the same ID used when creating the cast. This method ensures you're dropping the exact cast that was created with the corresponding ID.
## Creating custom casts with proper context strategy
When creating multiple casts for integer types, operator ambiguity errors can occur if all casts are created as IMPLICIT. The following example demonstrates this issue by creating two implicit casts from text to different integer widths:
```
-- Creating multiple IMPLICIT casts causes ambiguity
postgres=> CREATE CAST (text AS int4) WITH FUNCTION rds_casts.rds_text_to_int4_custom(text) AS IMPLICIT;
CREATE CAST
postgres=> CREATE CAST (text AS int8) WITH FUNCTION rds_casts.rds_text_to_int8_custom(text) AS IMPLICIT;
CREATE CAST
postgres=> CREATE TABLE test_cast(col int);
CREATE TABLE
postgres=> INSERT INTO test_cast VALUES ('123'::text);
INSERT 0 1
postgres=> SELECT * FROM test_cast WHERE col='123'::text;
ERROR: operator is not unique: integer = text
LINE 1: SELECT * FROM test_cast WHERE col='123'::text;
^
HINT: Could not choose a best candidate operator. You might need to add explicit type casts.
```
The error occurs because PostgreSQL cannot determine which implicit cast to use when comparing an integer column with a text value. Both the int4 and int8 implicit casts are valid candidates, creating ambiguity.
To avoid this operator ambiguity, use ASSIGNMENT context for smaller integer widths and IMPLICIT context for larger integer widths:
```
-- Use ASSIGNMENT for smaller integer widths
CREATE CAST (text AS int2)
WITH FUNCTION rds_casts.rds_text_to_int2_custom(text)
AS ASSIGNMENT;
CREATE CAST (text AS int4)
WITH FUNCTION rds_casts.rds_text_to_int4_custom(text)
AS ASSIGNMENT;
-- Use IMPLICIT for larger integer widths
CREATE CAST (text AS int8)
WITH FUNCTION rds_casts.rds_text_to_int8_custom(text)
AS IMPLICIT;
postgres=> INSERT INTO test_cast VALUES ('123'::text);
INSERT 0 1
postgres=> SELECT * FROM test_cast WHERE col='123'::text;
col
-----
123
(1 row)
```
With this strategy, only the int8 cast is implicit, so PostgreSQL can unambiguously determine which cast to use.
# Best Practices for Parallel Queries in RDS for PostgreSQL
Parallel query execution is a feature in PostgreSQL that allows a single SQL query to be broken into smaller tasks that are processed simultaneously by multiple background worker processes. Instead of executing a query entirely in a single backend process, PostgreSQL can distribute parts of the query, such as scans, joins, aggregations, or sorting, across multiple CPU cores. The *leader process* coordinates this execution and gathers the results from the *parallel workers*.
However, for most production workloads, especially high-concurrency OLTP systems, we recommend disabling automatic parallel query execution. While parallelism can accelerate queries on large datasets in analytics or reporting workloads, it introduces significant risks that often outweigh the benefits in busy production environments.
Parallel execution also introduces significant overhead. Each parallel worker is a full PostgreSQL backend process, which requires process forking (copying memory structures and initializing process state) and authentication (consuming connection slots from your `max_connections` limit). Each worker also consumes its own memory, including `work_mem` for sorting and hashing operations, with multiple workers per query, memory usage multiplies quickly (e.g., 4 workers × 64MB `work_mem` = 256MB per query). As a result, parallel queries can consume considerably more system resources than single-process queries. If not tuned properly, they may lead to CPU saturation (multiple workers overwhelming available processing capacity), increased context switching (the operating system frequently switching between numerous worker processes, adding overhead and reducing throughput), or connection exhaustion (since each parallel worker consumes a connection slot, a single query with 4 workers uses 5 connections total, 1 leader \$1 4 workers, which can quickly exhaust your connection pool under high concurrency, preventing new client connections and causing application failures). These issues are particularly severe under high-concurrency workloads where multiple queries may attempt parallel execution simultaneously.
PostgreSQL decides whether to use parallelism based on cost estimates. In some cases, the planner may automatically switch to a parallel plan if it appears cheaper even when it's not ideal in practice. This can happen if index statistics are outdated or if bloat makes sequential scans appear more attractive than index lookups. Because of this behavior, automatic parallel plans can sometimes introduce regressions in query performance or system stability.
To get the most benefit from parallel queries in RDS for PostgreSQL, it's important to test and tune them based on your workload, monitor system impact, and disable automatic parallel plan selection in favor of query-level control.
## Configuration Parameters
PostgreSQL uses several parameters to control the behavior and availability of parallel queries. Understanding and tuning these is critical to achieving predictable performance:
| Parameter | Description | Default |
| --- | --- | --- |
| max\$1parallel\$1workers | Maximum number of background worker processes that can run in total | GREATEST(\$1DBInstanceVCPU/2,8) |
| max\$1parallel\$1workers\$1per\$1gather | Maximum number of workers per query plan node (e.g., per Gather) | 2 |
| parallel\$1setup\$1cost | Planner cost added for initiating parallel query infrastructure | 1000 |
| parallel\$1tuple\$1cost | Cost per tuple processed in parallel mode (impacts planner decision) | 0.1 |
| force\$1parallel\$1mode | Forces planner to test parallel plans (off, on, regress) | off |
### Key Considerations
+ `max_parallel_workers` controls the total pool of parallel workers. If set too low, some queries may fall back to serial execution.
+ `max_parallel_workers_per_gather` affects how many workers a single query can use. A higher value increases concurrency, but also resource usage.
+ `parallel_setup_cost` and `parallel_tuple_cost` affect the planner's cost model. Lowering these can make parallel plans more likely to be chosen.
+ `force_parallel_mode` is useful for testing but should not be used in production unless necessary.
**Note**
The default value of the `max_parallel_workers` parameter is dynamically calculated based on instance size using the formula `GREATEST($DBInstanceVCPU/2, 8)`. This means that when you scale your DB instance to a larger compute size with more vCPUs, the maximum number of available parallel workers will automatically increase. As a result, queries that previously executed serially or with limited parallelism may suddenly utilize more parallel workers after a scale-up operation, potentially leading to unexpected increases in connection usage, CPU utilization, and memory consumption. It's important to monitor parallel query behavior after any compute scaling event and adjust `max_parallel_workers_per_gather` if necessary to maintain predictable resource usage.
## Identify Parallel Queries Usage
Queries may flip to parallel plans based on data distribution or statistics. For example:
```
SELECT count(*) FROM customers WHERE last_login < now() - interval '6 months';
```
This query might use an index for recent data, but switch to a parallel sequential scan for historical data.
You can log query execution plans by loading the `auto_explain` module. To learn more, see [Logging execution plans of queries](https://aws.amazon.com/premiumsupport/knowledge-center/rds-postgresql-tune-query-performance/#) in the AWS knowledge center.
You can monitor [CloudWatch Database Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Database-Insights-Database-Instance-Dashboard.html) for Parallel Query related wait events. To learn more about Parallel Query related wait events, go through [IPC:parallel wait events](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/apg-ipc-parallel.html)
From PostgreSQL version 18, you can monitor parallel worker activity using new columns in [https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-DATABASE-VIEW](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-DATABASE-VIEW) and [https://www.postgresql.org/docs/current/pgstatstatements.html](https://www.postgresql.org/docs/current/pgstatstatements.html):
+ `parallel_workers_to_launch`: Number of parallel workers planned to be launched
+ `parallel_workers_launched`: Number of parallel workers actually launched
These metrics help identify discrepancies between planned and actual parallelism, which can indicate resource constraints or configuration issues. Use the following queries to monitor parallel execution:
For Database-level parallel worker metrics:
```
SELECT datname, parallel_workers_to_launch, parallel_workers_launched
FROM pg_stat_database
WHERE datname = current_database();
```
For Query-level parallel worker metrics
```
SELECT query, parallel_workers_to_launch, parallel_workers_launched
FROM pg_stat_statements
ORDER BY parallel_workers_launched;
```
## How to Control Parallelism
There are several ways to control query parallelism, each designed for different scenarios and requirements.
To disable automatic parallelism globally, [modify your parameter group](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.Modifying.html) to set:
```
max_parallel_workers_per_gather = 0;
```
For persistent, user-specific settings, the ALTER ROLE command provides a way to set parameters that will apply to all future sessions for a particular user.
For example:
`ALTER ROLE username SET max_parallel_workers_per_gather = 4;` ensures that every time this user connects to the database, their sessions will use this parallel worker setting when required.
Session-level control can be achieved using the SET command, which modifies parameters for the duration of the current database session. This is particularly useful when you need to temporarily adjust settings without affecting other users or future sessions. Once set, these parameters remain in effect until explicitly reset or until the session ends. The commands are straightforward:
```
SET max_parallel_workers_per_gather = 4;
-- Run your queries
RESET max_parallel_workers_per_gather;
```
For even more granular control, SET LOCAL allows you to modify parameters for a single transaction. This is ideal when you need to adjust settings for a specific set of queries within a transaction, after which the settings automatically revert to their previous values. This approach helps prevent unintended effects on other operations within the same session.
## Diagnosing Parallel Query Behavior
Use `EXPLAIN (ANALYZE, VERBOSE)` to confirm whether a query used parallel execution:
+ Look for nodes such as `Gather`, `Gather Merge`, or `Parallel Seq Scan`.
+ Compare plans with and without parallelism.
To disable parallelism temporarily for comparison:
```
SET max_parallel_workers_per_gather = 0;
EXPLAIN ANALYZE ;
RESET max_parallel_workers_per_gather;
```
# Working with parameters on your RDS for PostgreSQL DB instance
In some cases, you might create an RDS for PostgreSQL DB instance without specifying a custom parameter group. If so, your DB instance is created using the default parameter group for the version of PostgreSQL that you choose. For example, suppose that you create an RDS for PostgreSQL DB instance using PostgreSQL 13.3. In this case, the DB instance is created using the values in the parameter group for PostgreSQL 13 releases, `default.postgres13`.
You can also create your own custom DB parameter group. You need to do this if you want to modify any settings for the RDS for PostgreSQL DB instance from their default values. To learn how, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
You can track the settings on your RDS for PostgreSQL DB instance in several different ways. You can use the AWS Management Console, the AWS CLI, or the Amazon RDS API. You can also query the values from the PostgreSQL `pg_settings` table of your instance, as shown following.
```
SELECT name, setting, boot_val, reset_val, unit
FROM pg_settings
ORDER BY name;
```
To learn more about the values returned from this query, see [https://www.postgresql.org/docs/current/view-pg-settings.html](https://www.postgresql.org/docs/current/view-pg-settings.html) in the PostgreSQL documentation.
Be especially careful when changing the settings for `max_connections` and `shared_buffers` on your RDS for PostgreSQL DB instance. For example, suppose that you modify settings for `max_connections` or `shared_buffers` and you use values that are too high for your actual workload. In this case, your RDS for PostgreSQL DB instance won't start. If this happens, you see an error such as the following in the `postgres.log`.
```
2018-09-18 21:13:15 UTC::@:[8097]:FATAL: could not map anonymous shared memory: Cannot allocate memory
2018-09-18 21:13:15 UTC::@:[8097]:HINT: This error usually means that PostgreSQL's request for a shared memory segment
exceeded available memory or swap space. To reduce the request size (currently 3514134274048 bytes), reduce
PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
```
However, you can't change any values of the settings contained in the default RDS for PostgreSQL DB parameter groups. To change settings for any parameters, first create a custom DB parameter group. Then change the settings in that custom group, and then apply the custom parameter group to your RDS for PostgreSQL DB instance. To learn more, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
There are two types of parameters in RDS for PostgreSQL.
+ **Static parameters** – Static parameters require that the RDS for PostgreSQL DB instance be rebooted after a change so that the new value can take effect.
+ **Dynamic parameters** – Dynamic parameters don't require a reboot after changing their settings.
**Note**
If your RDS for PostgreSQL DB instance is using your own custom DB parameter group, you can change the values of dynamic parameters on the running DB instance. You can do this by using the AWS Management Console, the AWS CLI, or the Amazon RDS API.
If you have privileges to do so, you can also change parameter values by using the `ALTER DATABASE`, `ALTER ROLE`, and `SET` commands.
## RDS for PostgreSQL DB instance parameter list
The following table lists some (but not all) parameters available in an RDS for PostgreSQL DB instance. To view all available parameters, you use the [describe-db-parameters](https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-parameters.html) AWS CLI command. For example, to get the list of all parameters available in the default parameter group for RDS for PostgreSQL version 13, run the following.
```
aws rds describe-db-parameters --db-parameter-group-name default.postgres13
```
You can also use the Console. Choose **Parameter groups** from the Amazon RDS menu, and then choose the parameter group from those available in your AWS Region.
| Parameter name | Apply\$1Type | Description |
| --- | --- | --- |
| `application_name` | Dynamic | Sets the application name to be reported in statistics and logs. |
| `archive_command` | Dynamic | Sets the shell command that will be called to archive a WAL file. |
| `array_nulls` | Dynamic | Enables input of NULL elements in arrays. |
| `authentication_timeout` | Dynamic | Sets the maximum allowed time to complete client authentication. |
| `autovacuum` | Dynamic | Starts the autovacuum subprocess. |
| `autovacuum_analyze_scale_factor` | Dynamic | Number of tuple inserts, updates, or deletes before analyze as a fraction of reltuples. |
| `autovacuum_analyze_threshold` | Dynamic | Minimum number of tuple inserts, updates, or deletes before analyze. |
| `autovacuum_freeze_max_age` | Static | Age at which to autovacuum a table to prevent transaction ID wraparound. |
| `autovacuum_naptime` | Dynamic | Time to sleep between autovacuum runs. |
| `autovacuum_max_workers` | Static | Sets the maximum number of simultaneously running autovacuum worker processes. |
| `autovacuum_vacuum_cost_delay` | Dynamic | Vacuum cost delay, in milliseconds, for autovacuum. |
| `autovacuum_vacuum_cost_limit` | Dynamic | Vacuum cost amount available before napping, for autovacuum. |
| `autovacuum_vacuum_scale_factor` | Dynamic | Number of tuple updates or deletes before vacuum as a fraction of reltuples. |
| `autovacuum_vacuum_threshold` | Dynamic | Minimum number of tuple updates or deletes before vacuum. |
| `backslash_quote` | Dynamic | Sets whether a backslash (\$1) is allowed in string literals. |
| `bgwriter_delay` | Dynamic | Background writer sleep time between rounds. |
| `bgwriter_lru_maxpages` | Dynamic | Background writer maximum number of LRU pages to flush per round. |
| `bgwriter_lru_multiplier` | Dynamic | Multiple of the average buffer usage to free per round. |
| `bytea_output` | Dynamic | Sets the output format for bytes. |
| `check_function_bodies` | Dynamic | Checks function bodies during CREATE FUNCTION. |
| `checkpoint_completion_target` | Dynamic | Time spent flushing dirty buffers during checkpoint, as a fraction of the checkpoint interval. |
| `checkpoint_segments` | Dynamic | Sets the maximum distance in log segments between automatic write-ahead log (WAL) checkpoints. |
| `checkpoint_timeout` | Dynamic | Sets the maximum time between automatic WAL checkpoints. |
| `checkpoint_warning` | Dynamic | Enables warnings if checkpoint segments are filled more frequently than this. |
| `client_connection_check_interval` | Dynamic | Sets the time interval between checks for disconnection while running queries. |
| `client_encoding` | Dynamic | Sets the client's character set encoding. |
| `client_min_messages` | Dynamic | Sets the message levels that are sent to the client. |
| `commit_delay` | Dynamic | Sets the delay in microseconds between transaction commit and flushing WAL to disk. |
| `commit_siblings` | Dynamic | Sets the minimum concurrent open transactions before performing commit\$1delay. |
| `constraint_exclusion` | Dynamic | Enables the planner to use constraints to optimize queries. |
| `cpu_index_tuple_cost` | Dynamic | Sets the planner's estimate of the cost of processing each index entry during an index scan. |
| `cpu_operator_cost` | Dynamic | Sets the planner's estimate of the cost of processing each operator or function call. |
| `cpu_tuple_cost` | Dynamic | Sets the planner's estimate of the cost of processing each tuple (row). |
| `cursor_tuple_fraction` | Dynamic | Sets the planner's estimate of the fraction of a cursor's rows that will be retrieved. |
| `datestyle` | Dynamic | Sets the display format for date and time values. |
| `deadlock_timeout` | Dynamic | Sets the time to wait on a lock before checking for deadlock. |
| `debug_pretty_print` | Dynamic | Indents parse and plan tree displays. |
| `debug_print_parse` | Dynamic | Logs each query's parse tree. |
| `debug_print_plan` | Dynamic | Logs each query's execution plan. |
| `debug_print_rewritten` | Dynamic | Logs each query's rewritten parse tree. |
| `default_statistics_target` | Dynamic | Sets the default statistics target. |
| `default_tablespace` | Dynamic | Sets the default tablespace to create tables and indexes in. |
| `default_transaction_deferrable` | Dynamic | Sets the default deferrable status of new transactions. |
| `default_transaction_isolation` | Dynamic | Sets the transaction isolation level of each new transaction. |
| `default_transaction_read_only` | Dynamic | Sets the default read-only status of new transactions. |
| `default_with_oids` | Dynamic | Creates new tables with object IDs (OIDs) by default. |
| `effective_cache_size` | Dynamic | Sets the planner's assumption about the size of the disk cache. |
| `effective_io_concurrency` | Dynamic | Number of simultaneous requests that can be handled efficiently by the disk subsystem. |
| `enable_bitmapscan` | Dynamic | Enables the planner's use of bitmap-scan plans. |
| `enable_hashagg` | Dynamic | Enables the planner's use of hashed aggregation plans. |
| `enable_hashjoin` | Dynamic | Enables the planner's use of hash join plans. |
| `enable_indexscan` | Dynamic | Enables the planner's use of index-scan plans. |
| `enable_material` | Dynamic | Enables the planner's use of materialization. |
| `enable_mergejoin` | Dynamic | Enables the planner's use of merge join plans. |
| `enable_nestloop` | Dynamic | Enables the planner's use of nested-loop join plans. |
| `enable_seqscan` | Dynamic | Enables the planner's use of sequential-scan plans. |
| `enable_sort` | Dynamic | Enables the planner's use of explicit sort steps. |
| `enable_tidscan` | Dynamic | Enables the planner's use of TID scan plans. |
| `escape_string_warning` | Dynamic | Warns about backslash (\$1) escapes in ordinary string literals. |
| `extra_float_digits` | Dynamic | Sets the number of digits displayed for floating-point values. |
| `from_collapse_limit` | Dynamic | Sets the FROM-list size beyond which subqueries are not collapsed. |
| `fsync` | Dynamic | Forces synchronization of updates to disk. |
| `full_page_writes` | Dynamic | Writes full pages to WAL when first modified after a checkpoint. |
| `geqo` | Dynamic | Enables genetic query optimization. |
| `geqo_effort` | Dynamic | GEQO: effort is used to set the default for other GEQO parameters. |
| `geqo_generations` | Dynamic | GEQO: number of iterations of the algorithm. |
| `geqo_pool_size` | Dynamic | GEQO: number of individuals in the population. |
| `geqo_seed` | Dynamic | GEQO: seed for random path selection. |
| `geqo_selection_bias` | Dynamic | GEQO: selective pressure within the population. |
| `geqo_threshold` | Dynamic | Sets the threshold of FROM items beyond which GEQO is used. |
| `gin_fuzzy_search_limit` | Dynamic | Sets the maximum allowed result for exact search by GIN. |
| `hot_standby_feedback` | Dynamic | Determines whether a hot standby sends feedback messages to the primary or upstream standby. |
| `intervalstyle` | Dynamic | Sets the display format for interval values. |
| `join_collapse_limit` | Dynamic | Sets the FROM-list size beyond which JOIN constructs are not flattened. |
| `lc_messages` | Dynamic | Sets the language in which messages are displayed. |
| `lc_monetary` | Dynamic | Sets the locale for formatting monetary amounts. |
| `lc_numeric` | Dynamic | Sets the locale for formatting numbers. |
| `lc_time` | Dynamic | Sets the locale for formatting date and time values. |
| `log_autovacuum_min_duration` | Dynamic | Sets the minimum running time above which autovacuum actions will be logged. |
| `log_checkpoints` | Dynamic | Logs each checkpoint. |
| `log_connections` | Dynamic | Logs each successful connection. |
| `log_disconnections` | Dynamic | Logs end of a session, including duration. |
| `log_duration` | Dynamic | Logs the duration of each completed SQL statement. |
| `log_error_verbosity` | Dynamic | Sets the verbosity of logged messages. |
| `log_executor_stats` | Dynamic | Writes executor performance statistics to the server log. |
| `log_filename` | Dynamic | Sets the file name pattern for log files. |
| `log_file_mode` | Dynamic | Sets file permissions for log files. Default value is 0644. |
| `log_hostname` | Dynamic | Logs the host name in the connection logs. As of PostgreSQL 12 and later versions, this parameter is 'off' by default. When turned on, the connection uses DNS reverse-lookup to get the hostname that gets captured to the connection logs. If you turn on this parameter, you should monitor the impact that it has on the time it takes to establish connections. |
| `log_line_prefix ` | Dynamic | Controls information prefixed to each log line. |
| `log_lock_waits` | Dynamic | Logs long lock waits. |
| `log_min_duration_statement` | Dynamic | Sets the minimum running time above which statements will be logged. |
| `log_min_error_statement` | Dynamic | Causes all statements generating an error at or above this level to be logged. |
| `log_min_messages` | Dynamic | Sets the message levels that are logged. |
| `log_parser_stats` | Dynamic | Writes parser performance statistics to the server log. |
| `log_planner_stats` | Dynamic | Writes planner performance statistics to the server log. |
| `log_rotation_age` | Dynamic | Automatic log file rotation will occur after N minutes. |
| `log_rotation_size` | Dynamic | Automatic log file rotation will occur after N kilobytes. |
| `log_statement` | Dynamic | Sets the type of statements logged. |
| `log_statement_stats` | Dynamic | Writes cumulative performance statistics to the server log. |
| `log_temp_files` | Dynamic | Logs the use of temporary files larger than this number of kilobytes. |
| `log_timezone` | Dynamic | Sets the time zone to use in log messages. |
| `log_truncate_on_rotation` | Dynamic | Truncate existing log files of same name during log rotation. |
| `logging_collector` | Static | Start a subprocess to capture stderr output and/or csvlogs into log files. |
| `maintenance_work_mem` | Dynamic | Sets the maximum memory to be used for maintenance operations. |
| `max_connections` | Static | Sets the maximum number of concurrent connections. |
| `max_files_per_process` | Static | Sets the maximum number of simultaneously open files for each server process. |
| `max_locks_per_transaction` | Static | Sets the maximum number of locks per transaction. |
| `max_pred_locks_per_transaction` | Static | Sets the maximum number of predicate locks per transaction. |
| `max_prepared_transactions` | Static | Sets the maximum number of simultaneously prepared transactions. |
| `max_stack_depth` | Dynamic | Sets the maximum stack depth, in kilobytes. |
| `max_standby_archive_delay` | Dynamic | Sets the maximum delay before canceling queries when a hot standby server is processing archived WAL data. |
| `max_standby_streaming_delay` | Dynamic | Sets the maximum delay before canceling queries when a hot standby server is processing streamed WAL data. |
| max\$1wal\$1size | Dynamic | Sets the WAL size (MB) that triggers a checkpoint. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Parameters.html) Use the following command on your Amazon RDS for PostgreSQL DB instance to see its current value:
SHOW max_wal_size;
|
| min\$1wal\$1size | Dynamic | Sets the minimum size to shrink the WAL to. For PostgreSQL version 9.6 and earlier, min\$1wal\$1size is in units of 16 MB. For PostgreSQL version 10 and later, min\$1wal\$1size is in units of 1 MB. |
| `quote_all_identifiers` | Dynamic | Adds quotes (") to all identifiers when generating SQL fragments. |
| `random_page_cost` | Dynamic | Sets the planner's estimate of the cost of a non-sequentially fetched disk page. This parameter has no value unless query plan management (QPM) is turned on. When QPM is on, the default value for this parameter 4. |
| rds.adaptive\$1autovacuum | Dynamic | Automatically tunes the autovacuum parameters whenever the transaction ID thresholds are exceeded. |
| rds.force\$1ssl | Dynamic | Requires the use of SSL connections. The default value is set to 1 (on) for RDS for PostgreSQL version 15. All other RDS for PostgreSQL major version 14 and older have the default value set to 0 (off). |
| `rds.local_volume_spill_enabled` | Static | Enables writing logical spill files to the local volume. |
| `rds.log_retention_period` | Dynamic | Sets log retention such that Amazon RDS deletes PostgreSQL logs that are older than n minutes. |
| rds.rds\$1superuser\$1reserved\$1connections | Static | Sets the number of connection slots reserved for rds\$1superusers. This parameter is only available in versions 15 and earlier. For more information, see the PostgreSQL documentation [reserved\$1connections](https://www.postgresql.org/docs/current/runtime-config-connection.html#GUC-RESERVED-CONNECTIONS). |
| `rds.replica_identity_full` | Dynamic | When you set this parameter to `on`, it overrides the replica identity setting to `FULL` for all database tables. This means all column values are written to the write ahead log (WAL), regardless of your `REPLICA IDENTITY FULL` settings. Turning on this parameter may increase your database instance IOPS due to the additional WAL logging. |
| rds.restrict\$1password\$1commands | Static | Restricts who can manage passwords to users with the rds\$1password role. Set this parameter to 1 to enable password restriction. The default is 0. |
| `search_path` | Dynamic | Sets the schema search order for names that are not schema-qualified. |
| `seq_page_cost` | Dynamic | Sets the planner's estimate of the cost of a sequentially fetched disk page. |
| `session_replication_role` | Dynamic | Sets the sessions behavior for triggers and rewrite rules. |
| `shared_buffers` | Static | Sets the number of shared memory buffers used by the server. |
| `shared_preload_libraries ` | Static | Lists the shared libraries to preload into the RDS for PostgreSQL DB instance. Supported values include auto\$1explain, orafce, pgaudit, pglogical, pg\$1bigm, pg\$1cron, pg\$1hint\$1plan, pg\$1prewarm, pg\$1similarity, pg\$1stat\$1statements, pg\$1tle, pg\$1transport, plprofiler, and plrust. |
| `ssl` | Dynamic | Enables SSL connections. |
| `sql_inheritance` | Dynamic | Causes subtables to be included by default in various commands. |
| `ssl_renegotiation_limit` | Dynamic | Sets the amount of traffic to send and receive before renegotiating the encryption keys. |
| `standard_conforming_strings` | Dynamic | Causes ... strings to treat backslashes literally. |
| `statement_timeout` | Dynamic | Sets the maximum allowed duration of any statement. |
| `synchronize_seqscans` | Dynamic | Enables synchronized sequential scans. |
| `synchronous_commit` | Dynamic | Sets the current transactions synchronization level. |
| `tcp_keepalives_count` | Dynamic | Maximum number of TCP keepalive retransmits. |
| `tcp_keepalives_idle` | Dynamic | Time between issuing TCP keepalives. |
| `tcp_keepalives_interval` | Dynamic | Time between TCP keepalive retransmits. |
| `temp_buffers` | Dynamic | Sets the maximum number of temporary buffers used by each session. |
| temp\$1file\$1limit | Dynamic | Sets the maximum size in KB to which the temporary files can grow. |
| `temp_tablespaces` | Dynamic | Sets the tablespaces to use for temporary tables and sort files. |
| `timezone` | Dynamic | Sets the time zone for displaying and interpreting time stamps. The Internet Assigned Numbers Authority (IANA) publishes new time zones at [ https://www.iana.org/time-zones](https://www.iana.org/time-zones) several times a year. Every time RDS releases a new minor maintenance release of PostgreSQL, it ships with the latest time zone data at the time of the release. When you use the latest RDS for PostgreSQL versions, you have recent time zone data from RDS. To ensure that your DB instance has recent time zone data, we recommend upgrading to a higher DB engine version. You can't modify the time zone tables in PostgreSQL DB instances manually. RDS doesn't modify or reset the time zone data of running DB instances. New time zone data is installed only when you perform a database engine version upgrade. |
| `track_activities` | Dynamic | Collects information about running commands. |
| `track_activity_query_size` | Static | Sets the size reserved for pg\$1stat\$1activity.current\$1query, in bytes. |
| `track_counts` | Dynamic | Collects statistics on database activity. |
| `track_functions` | Dynamic | Collects function-level statistics on database activity. |
| `track_io_timing` | Dynamic | Collects timing statistics on database I/O activity. |
| `transaction_deferrable` | Dynamic | Indicates whether to defer a read-only serializable transaction until it can be started with no possible serialization failures. |
| `transaction_isolation` | Dynamic | Sets the current transactions isolation level. |
| `transaction_read_only` | Dynamic | Sets the current transactions read-only status. |
| `transform_null_equals` | Dynamic | Treats expr=NULL as expr IS NULL. |
| `update_process_title` | Dynamic | Updates the process title to show the active SQL command. |
| `vacuum_cost_delay` | Dynamic | Vacuum cost delay in milliseconds. |
| `vacuum_cost_limit` | Dynamic | Vacuum cost amount available before napping. |
| `vacuum_cost_page_dirty` | Dynamic | Vacuum cost for a page dirtied by vacuum. |
| `vacuum_cost_page_hit` | Dynamic | Vacuum cost for a page found in the buffer cache. |
| `vacuum_cost_page_miss` | Dynamic | Vacuum cost for a page not found in the buffer cache. |
| `vacuum_defer_cleanup_age` | Dynamic | Number of transactions by which vacuum and hot cleanup should be deferred, if any. |
| `vacuum_freeze_min_age` | Dynamic | Minimum age at which vacuum should freeze a table row. |
| `vacuum_freeze_table_age` | Dynamic | Age at which vacuum should scan a whole table to freeze tuples. |
| `wal_buffers` | Static | Sets the number of disk-page buffers in shared memory for WAL. |
| `wal_writer_delay` | Dynamic | WAL writer sleep time between WAL flushes. |
| `work_mem` | Dynamic | Sets the maximum memory to be used for query workspaces. |
| `xmlbinary` | Dynamic | Sets how binary values are to be encoded in XML. |
| `xmloption` | Dynamic | Sets whether XML data in implicit parsing and serialization operations is to be considered as documents or content fragments. |
Amazon RDS uses the default PostgreSQL units for all parameters. The following table shows the PostgreSQL default unit for each parameter.
| Parameter name | Unit |
| --- | --- |
| `archive_timeout` | s |
| `authentication_timeout` | s |
| `autovacuum_naptime` | s |
| `autovacuum_vacuum_cost_delay` | ms |
| `bgwriter_delay` | ms |
| `checkpoint_timeout` | s |
| `checkpoint_warning` | s |
| `deadlock_timeout` | ms |
| `effective_cache_size` | 8 KB |
| `lock_timeout` | ms |
| `log_autovacuum_min_duration` | ms |
| `log_min_duration_statement` | ms |
| `log_rotation_age` | minutes |
| `log_rotation_size` | KB |
| `log_temp_files` | KB |
| `maintenance_work_mem` | KB |
| `max_stack_depth` | KB |
| `max_standby_archive_delay` | ms |
| `max_standby_streaming_delay` | ms |
| `post_auth_delay` | s |
| `pre_auth_delay` | s |
| `segment_size` | 8 KB |
| `shared_buffers` | 8 KB |
| `statement_timeout` | ms |
| `ssl_renegotiation_limit` | KB |
| `tcp_keepalives_idle` | s |
| `tcp_keepalives_interval` | s |
| `temp_file_limit` | KB |
| `work_mem` | KB |
| `temp_buffers` | 8 KB |
| `vacuum_cost_delay` | ms |
| `wal_buffers` | 8 KB |
| `wal_receiver_timeout` | ms |
| `wal_segment_size` | B |
| `wal_sender_timeout` | ms |
| `wal_writer_delay` | ms |
| `wal_receiver_status_interval` | s |
# Tuning with wait events for RDS for PostgreSQL
Wait events are an important tuning tool for RDS for PostgreSQL. When you can find out why sessions are waiting for resources and what they are doing, you're better able to reduce bottlenecks. You can use the information in this section to find possible causes and corrective actions. This section also discusses basic PostgreSQL tuning concepts.
The wait events in this section are specific to RDS for PostgreSQL.
**Topics**
+ [
# Essential concepts for RDS for PostgreSQL tuning
](PostgreSQL.Tuning.concepts.md)
+ [
# RDS for PostgreSQL wait events
](PostgreSQL.Tuning.concepts.summary.md)
+ [
# Client:ClientRead
](wait-event.clientread.md)
+ [
# Client:ClientWrite
](wait-event.clientwrite.md)
+ [
# CPU
](wait-event.cpu.md)
+ [
# IO:BufFileRead and IO:BufFileWrite
](wait-event.iobuffile.md)
+ [
# IO:DataFileRead
](wait-event.iodatafileread.md)
+ [
# IO:WALWrite
](wait-event.iowalwrite.md)
+ [
# IPC:parallel wait events
](rpg-ipc-parallel.md)
+ [
# IPC:ProcArrayGroupUpdate
](apg-rpg-ipcprocarraygroup.md)
+ [
# Lock:advisory
](wait-event.lockadvisory.md)
+ [
# Lock:extend
](wait-event.lockextend.md)
+ [
# Lock:Relation
](wait-event.lockrelation.md)
+ [
# Lock:transactionid
](wait-event.locktransactionid.md)
+ [
# Lock:tuple
](wait-event.locktuple.md)
+ [
# LWLock:BufferMapping (LWLock:buffer\$1mapping)
](wait-event.lwl-buffer-mapping.md)
+ [
# LWLock:BufferIO (IPC:BufferIO)
](wait-event.lwlockbufferio.md)
+ [
# LWLock:buffer\$1content (BufferContent)
](wait-event.lwlockbuffercontent.md)
+ [
# LWLock:lock\$1manager (LWLock:lockmanager)
](wait-event.lw-lock-manager.md)
+ [
# LWLock:pg\$1stat\$1statements
](apg-rpg-lwlockpgstat.md)
+ [
# LWLock:SubtransSLRU (LWLock:SubtransControlLock)
](wait-event.lwlocksubtransslru.md)
+ [
# Timeout:PgSleep
](wait-event.timeoutpgsleep.md)
+ [
# Timeout:VacuumDelay
](wait-event.timeoutvacuumdelay.md)
# Essential concepts for RDS for PostgreSQL tuning
Before you tune your RDS for PostgreSQL database, make sure to learn what wait events are and why they occur. Also review the basic memory and disk architecture of RDS for PostgreSQL. For a helpful architecture diagram, see the [PostgreSQL](https://en.wikibooks.org/wiki/PostgreSQL/Architecture) wikibook.
**Topics**
+ [
# RDS for PostgreSQL wait events
](PostgreSQL.Tuning.concepts.waits.md)
+ [
# RDS for PostgreSQL memory
](PostgreSQL.Tuning.concepts.memory.md)
+ [
# RDS for PostgreSQL processes
](PostgreSQL.Tuning.concepts.processes.md)
# RDS for PostgreSQL wait events
A *wait event* is an indication that the session is waiting for a resource. For example, the wait event `Client:ClientRead` occurs when RDS for PostgreSQL is waiting to receive data from the client. Sessions typically wait for resources such as the following.
+ Single-threaded access to a buffer, for example, when a session is attempting to modify a buffer
+ A row that is currently locked by another session
+ A data file read
+ A log file write
For example, to satisfy a query, the session might perform a full table scan. If the data isn't already in memory, the session waits for the disk I/O to complete. When the buffers are read into memory, the session might need to wait because other sessions are accessing the same buffers. The database records the waits by using a predefined wait event. These events are grouped into categories.
By itself, a single wait event doesn't indicate a performance problem. For example, if requested data isn't in memory, reading data from disk is necessary. If one session locks a row for an update, another session waits for the row to be unlocked so that it can update it. A commit requires waiting for the write to a log file to complete. Waits are integral to the normal functioning of a database.
On the other hand, large numbers of wait events typically show a performance problem. In such cases, you can use wait event data to determine where sessions are spending time. For example, if a report that typically runs in minutes now takes hours to run, you can identify the wait events that contribute the most to total wait time. If you can determine the causes of the top wait events, you can sometimes make changes that improve performance. For example, if your session is waiting on a row that has been locked by another session, you can end the locking session.
# RDS for PostgreSQL memory
RDS for PostgreSQL memory is divided into shared and local.
**Topics**
+ [
## Shared memory in RDS for PostgreSQL
](#PostgreSQL.Tuning.concepts.shared)
+ [
## Local memory in RDS for PostgreSQL
](#PostgreSQL.Tuning.concepts.local)
## Shared memory in RDS for PostgreSQL
RDS for PostgreSQL allocates shared memory when the instance starts. Shared memory is divided into multiple subareas. The following sections provide descriptions of the most important ones.
**Topics**
+ [
### Shared buffers
](#PostgreSQL.Tuning.concepts.buffer-pool)
+ [
### Write ahead log (WAL) buffers
](#PostgreSQL.Tuning.concepts.WAL)
### Shared buffers
The *shared buffer pool* is an RDS for PostgreSQL memory area that holds all pages that are or were being used by application connections. A *page* is the memory version of a disk block. The shared buffer pool caches the data blocks read from disk. The pool reduces the need to reread data from disk, making the database operate more efficiently.
Every table and index is stored as an array of pages of a fixed size. Each block contains multiple tuples, which correspond to rows. A tuple can be stored in any page.
The shared buffer pool has finite memory. If a new request requires a page that isn't in memory, and no more memory exists, RDS for PostgreSQL evicts a less frequently used page to accommodate the request. The eviction policy is implemented by a clock sweep algorithm.
The `shared_buffers` parameter determines how much memory the server dedicates to caching data. The default value is set to `{DBInstanceClassMemory/32768}` bytes, based on the available memory for the DB instance.
### Write ahead log (WAL) buffers
A *write-ahead log (WAL) buffer* holds transaction data that RDS for PostgreSQL later writes to persistent storage. Using the WAL mechanism, RDS for PostgreSQL can do the following:
+ Recover data after a failure
+ Reduce disk I/O by avoiding frequent writes to disk
When a client changes data, RDS for PostgreSQL writes the changes to the WAL buffer. When the client issues a `COMMIT`, the WAL writer process writes transaction data to the WAL file.
The `wal_level` parameter determines how much information is written to the WAL, with possible values such as `minimal`, `replica`, and `logical`.
## Local memory in RDS for PostgreSQL
Every backend process allocates local memory for query processing.
**Topics**
+ [
### Work memory area
](#PostgreSQL.Tuning.concepts.local.work_mem)
+ [
### Maintenance work memory area
](#PostgreSQL.Tuning.concepts.local.maintenance_work_mem)
+ [
### Temporary buffer area
](#PostgreSQL.Tuning.concepts.temp)
### Work memory area
The *work memory area* holds temporary data for queries that performs sorts and hashes. For example, a query with an `ORDER BY` clause performs a sort. Queries use hash tables in hash joins and aggregations.
The `work_mem` parameter the amount of memory to be used by internal sort operations and hash tables before writing to temporary disk files, measured in megabytes. The default value is 4 MB. Multiple sessions can run simultaneously, and each session can run maintenance operations in parallel. For this reason, the total work memory used can be multiples of the `work_mem` setting.
### Maintenance work memory area
The *maintenance work memory area* caches data for maintenance operations. These operations include vacuuming, creating an index, and adding foreign keys.
The `maintenance_work_mem` parameter specifies the maximum amount of memory to be used by maintenance operations, measured in megabytes. The default value is 64 MB. A database session can only run one maintenance operation at a time.
### Temporary buffer area
The *temporary buffer area* caches temporary tables for each database session.
Each session allocates temporary buffers as needed up to the limit you specify. When the session ends, the server clears the buffers.
The `temp_buffers` parameter sets the maximum number of temporary buffers used by each session, measured in megabytes. The default value is 8 MB. Before the first use of temporary tables within a session, you can change the `temp_buffers` value.
# RDS for PostgreSQL processes
RDS for PostgreSQL uses multiple processes.
**Topics**
+ [
## Postmaster process
](#PostgreSQL.Tuning.concepts.postmaster)
+ [
## Backend processes
](#PostgreSQL.Tuning.concepts.backend)
+ [
## Background processes
](#PostgreSQL.Tuning.concepts.vacuum)
## Postmaster process
The *postmaster process* is the first process started when you start RDS for PostgreSQL. The postmaster process has the following primary responsibilities:
+ Fork and monitor background processes
+ Receive authentication requests from client processes, and authenticate them before allowing the database to service requests
## Backend processes
If the postmaster authenticates a client request, the postmaster forks a new backend process, also called a postgres process. One client process connects to exactly one backend process. The client process and the backend process communicate directly without intervention by the postmaster process.
## Background processes
The postmaster process forks several processes that perform different backend tasks. Some of the more important include the following:
+ WAL writer
RDS for PostgreSQL writes data in the WAL (write ahead logging) buffer to the log files. The principle of write ahead logging is that the database can't write changes to the data files until after the database writes log records describing those changes to disk. The WAL mechanism reduces disk I/O, and allows RDS for PostgreSQL to use the logs to recover the database after a failure.
+ Background writer
This process periodically write dirty (modified) pages from the memory buffers to the data files. A page becomes dirty when a backend process modifies it in memory.
+ Autovacuum daemon
The daemon consists of the following:
+ The autovacuum launcher
+ The autovacuum worker processes
When autovacuum is turned on, it checks for tables that have had a large number of inserted, updated, or deleted tuples. The daemon has the following responsibilities:
+ Recover or reuse disk space occupied by updated or deleted rows
+ Update statistics used by the planner
+ Protect against loss of old data because of transaction ID wraparound
The autovacuum feature automates the execution of `VACUUM` and `ANALYZE` commands. `VACUUM` has the following variants: standard and full. Standard vacuum runs in parallel with other database operations. `VACUUM FULL` requires an exclusive lock on the table it is working on. Thus, it can't run in parallel with operations that access the same table. `VACUUM` creates a substantial amount of I/O traffic, which can cause poor performance for other active sessions.
# RDS for PostgreSQL wait events
The following table lists the wait events for RDS for PostgreSQL that most commonly indicate performance problems, and summarizes the most common causes and corrective actions..
| Wait event | Definition |
| --- | --- |
| [Client:ClientRead](wait-event.clientread.md) | This event occurs when RDS for PostgreSQL is waiting to receive data from the client. |
| [Client:ClientWrite](wait-event.clientwrite.md) | This event occurs when RDS for PostgreSQL is waiting to write data to the client. |
| [CPU](wait-event.cpu.md) | This event occurs when a thread is active in CPU or is waiting for CPU. |
| [IO:BufFileRead and IO:BufFileWrite](wait-event.iobuffile.md) | These events occur when RDS for PostgreSQL creates temporary files. |
| [IO:DataFileRead](wait-event.iodatafileread.md) | This event occurs when a connection waits on a backend process to read a required page from storage because the page isn't available in shared memory. |
| [IO:WALWrite](wait-event.iowalwrite.md) | This event occurs when RDS for PostgreSQL is waiting for the write-ahead log (WAL) buffers to be written to a WAL file. |
| [IPC:parallel wait events](rpg-ipc-parallel.md) | These wait events indicate that a session is waiting for inter-process communication related to parallel query execution operations. |
| [IPC:ProcArrayGroupUpdate](apg-rpg-ipcprocarraygroup.md) | This event occurs when a session is waiting for the group leader to update the transaction status at the end of the transaction. |
| [Lock:advisory](wait-event.lockadvisory.md) | This event occurs when a PostgreSQL application uses a lock to coordinate activity across multiple sessions. |
| [Lock:extend](wait-event.lockextend.md) | This event occurs when a backend process is waiting to lock a relation to extend it while another process has a lock on that relation for the same purpose. |
| [Lock:Relation](wait-event.lockrelation.md) | This event occurs when a query is waiting to acquire a lock on a table or view that's currently locked by another transaction. |
| [Lock:transactionid](wait-event.locktransactionid.md) | This event occurs when a transaction is waiting for a row-level lock. |
| [Lock:tuple](wait-event.locktuple.md) | This event occurs when a backend process is waiting to acquire a lock on a tuple. |
| [LWLock:BufferMapping (LWLock:buffer\$1mapping)](wait-event.lwl-buffer-mapping.md) | This event occurs when a session is waiting to associate a data block with a buffer in the shared buffer pool. |
| [LWLock:BufferIO (IPC:BufferIO)](wait-event.lwlockbufferio.md) | This event occurs when RDS for PostgreSQL is waiting for other processes to finish their input/output (I/O) operations when concurrently trying to access a page. |
| [LWLock:buffer\$1content (BufferContent)](wait-event.lwlockbuffercontent.md) | This event occurs when a session is waiting to read or write a data page in memory while another session has that page locked for writing. |
| [LWLock:lock\$1manager (LWLock:lockmanager)](wait-event.lw-lock-manager.md) | This event occurs when the RDS for PostgreSQL engine maintains the shared lock's memory area to allocate, check, and deallocate a lock when a fast path lock isn't possible. |
| [LWLock:SubtransSLRU (LWLock:SubtransControlLock)](wait-event.lwlocksubtransslru.md) | This event occurs when a process is waiting to access the simple least-recently used (SLRU) cache for a subtransaction. |
| [LWLock:pg\$1stat\$1statements](apg-rpg-lwlockpgstat.md) | This event occurs when the `pg_stat_statements` extension takes an exclusive lock on the hash table that tracks SQL statements. |
| [Timeout:PgSleep](wait-event.timeoutpgsleep.md) | This event occurs when a server process has called the `pg_sleep` function and is waiting for the sleep timeout to expire. |
| [Timeout:VacuumDelay](wait-event.timeoutvacuumdelay.md) | This event indicates that the vacuum process is sleeping because the estimated cost limit has been reached. |
# Client:ClientRead
The `Client:ClientRead` event occurs when RDS for PostgreSQL is waiting to receive data from the client.
**Topics**
+ [
## Supported engine versions
](#wait-event.clientread.context.supported)
+ [
## Context
](#wait-event.clientread.context)
+ [
## Likely causes of increased waits
](#wait-event.clientread.causes)
+ [
## Actions
](#wait-event.clientread.actions)
## Supported engine versions
This wait event information is supported for RDS for PostgreSQL version 10 and higher.
## Context
An RDS for PostgreSQL DB instance is waiting to receive data from the client. The RDS for PostgreSQL DB instance must receive the data from the client before it can send more data to the client. The time that the instance waits before receiving data from the client is a `Client:ClientRead` event.
## Likely causes of increased waits
Common causes for the `Client:ClientRead` event to appear in top waits include the following:
**Increased network latency**
There might be increased network latency between the RDS for PostgreSQL DB instance and client. Higher network latency increases the time required for DB instance to receive data from the client.
**Increased load on the client**
There might be CPU pressure or network saturation on the client. An increase in load on the client can delay transmission of data from the client to the RDS for PostgreSQL DB instance.
**Excessive network round trips**
A large number of network round trips between the RDS for PostgreSQL DB instance and the client can delay transmission of data from the client to the RDS for PostgreSQL DB instance.
**Large copy operation**
During a copy operation, the data is transferred from the client's file system to the RDS for PostgreSQL DB instance. Sending a large amount of data to the DB instance can delay transmission of data from the client to the DB instance.
**Idle client connection**
When a client connects to the RDS for PostgreSQL DB instance in an `idle in transaction` state, the DB instance might wait for the client to send more data or issue a command. A connection in this state can lead to an increase in `Client:ClientRead` events.
**PgBouncer used for connection pooling**
PgBouncer has a low-level network configuration setting called `pkt_buf`, which is set to 4,096 by default. If the workload is sending query packets larger than 4,096 bytes through PgBouncer, we recommend increasing the `pkt_buf` setting to 8,192. If the new setting doesn't decrease the number of `Client:ClientRead` events, we recommend increasing the `pkt_buf` setting to larger values, such as 16,384 or 32,768. If the query text is large, the larger setting can be particularly helpful.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Place the clients in the same Availability Zone and VPC subnet as the instance
](#wait-event.clientread.actions.az-vpc-subnet)
+ [
### Scale your client
](#wait-event.clientread.actions.scale-client)
+ [
### Use current generation instances
](#wait-event.clientread.actions.db-instance-class)
+ [
### Increase network bandwidth
](#wait-event.clientread.actions.increase-network-bandwidth)
+ [
### Monitor maximums for network performance
](#wait-event.clientread.actions.monitor-network-performance)
+ [
### Monitor for transactions in the "idle in transaction" state
](#wait-event.clientread.actions.check-idle-in-transaction)
### Place the clients in the same Availability Zone and VPC subnet as the instance
To reduce network latency and increase network throughput, place clients in the same Availability Zone and virtual private cloud (VPC) subnet as the RDS for PostgreSQL DB instance. Make sure that the clients are as geographically close to the DB instance as possible.
### Scale your client
Using Amazon CloudWatch or other host metrics, determine if your client is currently constrained by CPU or network bandwidth, or both. If the client is constrained, scale your client accordingly.
### Use current generation instances
In some cases, you might not be using a DB instance class that supports jumbo frames. If you're running your application on Amazon EC2, consider using a current generation instance for the client. Also, configure the maximum transmission unit (MTU) on the client operating system. This technique might reduce the number of network round trips and increase network throughput. For more information, see [ Jumbo frames (9001 MTU)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) in the *Amazon EC2 User Guide*.
For information about DB instance classes, see [DB instance classes](Concepts.DBInstanceClass.md). To determine the DB instance class that is equivalent to an Amazon EC2 instance type, place `db.` before the Amazon EC2 instance type name. For example, the `r5.8xlarge` Amazon EC2 instance is equivalent to the `db.r5.8xlarge` DB instance class.
### Increase network bandwidth
Use `NetworkReceiveThroughput` and `NetworkTransmitThroughput` Amazon CloudWatch metrics to monitor incoming and outgoing network traffic on the DB instance. These metrics can help you to determine if network bandwidth is sufficient for your workload.
If your network bandwidth isn't enough, increase it. If the AWS client or your DB instance is reaching the network bandwidth limits, the only way to increase the bandwidth is to increase your DB instance size. For more information, see [DB instance class types](Concepts.DBInstanceClass.Types.md).
For more information about CloudWatch metrics, see [Amazon CloudWatch metrics for Amazon RDS](rds-metrics.md).
### Monitor maximums for network performance
If you are using Amazon EC2 clients, Amazon EC2 provides maximums for network performance metrics, including aggregate inbound and outbound network bandwidth. It also provides connection tracking to ensure that packets are returned as expected and link-local services access for services such as the Domain Name System (DNS). To monitor these maximums, use a current enhanced networking driver and monitor network performance for your client.
For more information, see [ Monitor network performance for your Amazon EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-network-performance-ena.html) in the *Amazon EC2 User Guide* and [Monitor network performance for your Amazon EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/monitoring-network-performance-ena.html) in the *Amazon EC2 User Guide*.
### Monitor for transactions in the "idle in transaction" state
Check whether you have an increasing number of `idle in transaction` connections. To do this, monitor the `state` column in the `pg_stat_activity` table. You might be able to identify the connection source by running a query similar to the following.
```
select client_addr, state, count(1) from pg_stat_activity
where state like 'idle in transaction%'
group by 1,2
order by 3 desc
```
# Client:ClientWrite
The `Client:ClientWrite` event occurs when RDS for PostgreSQL is waiting to write data to the client.
**Topics**
+ [
## Supported engine versions
](#wait-event.clientwrite.context.supported)
+ [
## Context
](#wait-event.clientwrite.context)
+ [
## Likely causes of increased waits
](#wait-event.clientwrite.causes)
+ [
## Actions
](#wait-event.clientwrite.actions)
## Supported engine versions
This wait event information is supported for RDS for PostgreSQL version 10 and higher.
## Context
A client process must read all of the data received from an RDS for PostgreSQL DB cluster before the cluster can send more data. The time that the cluster waits before sending more data to the client is a `Client:ClientWrite` event.
Reduced network throughput between the RDS for PostgreSQL DB instance and the client can cause this event. CPU pressure and network saturation on the client can also cause this event. *CPU pressure* is when the CPU is fully utilized and there are tasks waiting for CPU time. *Network saturation* is when the network between the database and client is carrying more data than it can handle.
## Likely causes of increased waits
Common causes for the `Client:ClientWrite` event to appear in top waits include the following:
**Increased network latency**
There might be increased network latency between the RDS for PostgreSQL DB instance and client. Higher network latency increases the time required for the client to receive the data.
**Increased load on the client**
There might be CPU pressure or network saturation on the client. An increase in load on the client delays the reception of data from the RDS for PostgreSQL DB instance.
**Large volume of data sent to the client**
The RDS for PostgreSQL DB instance might be sending a large amount of data to the client. A client might not be able to receive the data as fast as the cluster is sending it. Activities such as a copy of a large table can result in an increase in `Client:ClientWrite` events.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Place the clients in the same Availability Zone and VPC subnet as the cluster
](#wait-event.clientwrite.actions.az-vpc-subnet)
+ [
### Use current generation instances
](#wait-event.clientwrite.actions.db-instance-class)
+ [
### Reduce the amount of data sent to the client
](#wait-event.clientwrite.actions.reduce-data)
+ [
### Scale your client
](#wait-event.clientwrite.actions.scale-client)
### Place the clients in the same Availability Zone and VPC subnet as the cluster
To reduce network latency and increase network throughput, place clients in the same Availability Zone and virtual private cloud (VPC) subnet as the RDS for PostgreSQL DB instance.
### Use current generation instances
In some cases, you might not be using a DB instance class that supports jumbo frames. If you're running your application on Amazon EC2, consider using a current generation instance for the client. Also, configure the maximum transmission unit (MTU) on the client operating system. This technique might reduce the number of network round trips and increase network throughput. For more information, see [ Jumbo frames (9001 MTU)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) in the *Amazon EC2 User Guide*.
For information about DB instance classes, see [DB instance classes](Concepts.DBInstanceClass.md). To determine the DB instance class that is equivalent to an Amazon EC2 instance type, place `db.` before the Amazon EC2 instance type name. For example, the `r5.8xlarge` Amazon EC2 instance is equivalent to the `db.r5.8xlarge` DB instance class.
### Reduce the amount of data sent to the client
When possible, adjust your application to reduce the amount of data that the RDS for PostgreSQL DB instance sends to the client. Making such adjustments relieves CPU and network contention on the client.
### Scale your client
Using Amazon CloudWatch or other host metrics, determine if your client is currently constrained by CPU or network bandwidth, or both. If the client is constrained, scale your client accordingly.
# CPU
This event occurs when a thread is active in CPU or is waiting for CPU.
**Topics**
+ [
## Supported engine versions
](#wait-event.cpu.context.supported)
+ [
## Context
](#wait-event.cpu.context)
+ [
## Likely causes of increased waits
](#wait-event.cpu.causes)
+ [
## Actions
](#wait-event.cpu.actions)
## Supported engine versions
This wait event information is relevant for all all versions of RDS for PostgreSQL.
## Context
The *central processing unit (CPU)* is the component of a computer that runs instructions. For example, CPU instructions perform arithmetic operations and exchange data in memory. If a query increases the number of instructions that it performs through the database engine, the time spent running the query increases. *CPU scheduling* is giving CPU time to a process. Scheduling is orchestrated by the kernel of the operating system.
**Topics**
+ [
### How to tell when this wait occurs
](#wait-event.cpu.when-it-occurs)
+ [
### DBLoadCPU metric
](#wait-event.cpu.context.dbloadcpu)
+ [
### os.cpuUtilization metrics
](#wait-event.cpu.context.osmetrics)
+ [
### Likely cause of CPU scheduling
](#wait-event.cpu.context.scheduling)
### How to tell when this wait occurs
This `CPU` wait event indicates that a backend process is active in CPU or is waiting for CPU. You know that it's occurring when a query shows the following information:
+ The `pg_stat_activity.state` column has the value `active`.
+ The `wait_event_type` and `wait_event` columns in `pg_stat_activity` are both `null`.
To see the backend processes that are using or waiting on CPU, run the following query.
```
SELECT *
FROM pg_stat_activity
WHERE state = 'active'
AND wait_event_type IS NULL
AND wait_event IS NULL;
```
### DBLoadCPU metric
The Performance Insights metric for CPU is `DBLoadCPU`. The value for `DBLoadCPU` can differ from the value for the Amazon CloudWatch metric `CPUUtilization`. The latter metric is collected from the HyperVisor for a database instance.
### os.cpuUtilization metrics
Performance Insights operating-system metrics provide detailed information about CPU utilization. For example, you can display the following metrics:
+ `os.cpuUtilization.nice.avg`
+ `os.cpuUtilization.total.avg`
+ `os.cpuUtilization.wait.avg`
+ `os.cpuUtilization.idle.avg`
Performance Insights reports the CPU usage by the database engine as `os.cpuUtilization.nice.avg`.
### Likely cause of CPU scheduling
The operating system (OS) kernel handles scheduling for the CPU. When the CPU is *active*, a process might need to wait to get scheduled. The CPU is active while it's performing computations. It's also active while it has an idle thread that it's not running, that is, an idle thread that's waiting on memory I/O. This type of I/O dominates the typical database workload.
Processes are likely to wait to get scheduled on a CPU when the following conditions are met:
+ The CloudWatch `CPUUtilization` metric is near 100 percent.
+ The average load is greater than the number of vCPUs, indicating a heavy load. You can find the `loadAverageMinute` metric in the OS metrics section in Performance Insights.
## Likely causes of increased waits
When the CPU wait event occurs more than normal, possibly indicating a performance problem, typical causes include the following.
**Topics**
+ [
### Likely causes of sudden spikes
](#wait-event.cpu.causes.spikes)
+ [
### Likely causes of long-term high frequency
](#wait-event.cpu.causes.long-term)
+ [
### Corner cases
](#wait-event.cpu.causes.corner-cases)
### Likely causes of sudden spikes
The most likely causes of sudden spikes are as follows:
+ Your application has opened too many simultaneous connections to the database. This scenario is known as a "connection storm."
+ Your application workload changed in any of the following ways:
+ New queries
+ An increase in the size of your dataset
+ Index maintenance or creation
+ New functions
+ New operators
+ An increase in parallel query execution
+ Your query execution plans have changed. In some cases, a change can cause an increase in buffers. For example, the query is now using a sequential scan when it previously used an index. In this case, the queries need more CPU to accomplish the same goal.
### Likely causes of long-term high frequency
The most likely causes of events that recur over a long period:
+ Too many backend processes are running concurrently on CPU. These processes can be parallel workers.
+ Queries are performing suboptimally because they need a large number of buffers.
### Corner cases
If none of the likely causes turn out to be actual causes, the following situations might be occurring:
+ The CPU is swapping processes in and out.
+ The CPU might be managing page table entries if the *huge pages* feature has been turned off. This memory management feature is turned on by default for all DB instance classes other than micro, small, and medium DB instance classes. For more information, see [Huge pages for RDS for PostgreSQL](PostgreSQL.Concepts.General.FeatureSupport.HugePages.md).
## Actions
If the `CPU` wait event dominates database activity, it doesn't necessarily indicate a performance problem. Respond to this event only when performance degrades.
**Topics**
+ [
### Investigate whether the database is causing the CPU increase
](#wait-event.cpu.actions.db-CPU)
+ [
### Determine whether the number of connections increased
](#wait-event.cpu.actions.connections)
+ [
### Respond to workload changes
](#wait-event.cpu.actions.workload)
### Investigate whether the database is causing the CPU increase
Examine the `os.cpuUtilization.nice.avg` metric in Performance Insights. If this value is far less than the CPU usage, nondatabase processes are the main contributor to CPU.
### Determine whether the number of connections increased
Examine the `DatabaseConnections` metric in Amazon CloudWatch. Your action depends on whether the number increased or decreased during the period of increased CPU wait events.
#### The connections increased
If the number of connections went up, compare the number of backend processes consuming CPU to the number of vCPUs. The following scenarios are possible:
+ The number of backend processes consuming CPU is less than the number of vCPUs.
In this case, the number of connections isn't an issue. However, you might still try to reduce CPU utilization.
+ The number of backend processes consuming CPU is greater than the number of vCPUs.
In this case, consider the following options:
+ Decrease the number of backend processes connected to your database. For example, implement a connection pooling solution such as RDS Proxy. To learn more, see [Amazon RDS Proxy](rds-proxy.md).
+ Upgrade your instance size to get a higher number of vCPUs.
+ Redirect some read-only workloads to reader nodes, if applicable.
#### The connections didn't increase
Examine the `blks_hit` metrics in Performance Insights. Look for a correlation between an increase in `blks_hit` and CPU usage. The following scenarios are possible:
+ CPU usage and `blks_hit` are correlated.
In this case, find the top SQL statements that are linked to the CPU usage, and look for plan changes. You can use either of the following techniques:
+ Explain the plans manually and compare them to the expected execution plan.
+ Look for an increase in block hits per second and local block hits per second. In the **Top SQL** section of Performance Insights dashboard, choose **Preferences**.
+ CPU usage and `blks_hit` aren't correlated.
In this case, determine whether any of the following occurs:
+ The application is rapidly connecting to and disconnecting from the database.
Diagnose this behavior by turning on `log_connections` and `log_disconnections`, then analyzing the PostgreSQL logs. Consider using the `pgbadger` log analyzer. For more information, see [https://github.com/darold/pgbadger](https://github.com/darold/pgbadger).
+ The OS is overloaded.
In this case, Performance Insights shows that backend processes are consuming CPU for a longer time than usual. Look for evidence in the Performance Insights `os.cpuUtilization` metrics or the CloudWatch `CPUUtilization` metric. If the operating system is overloaded, look at Enhanced Monitoring metrics to diagnose further. Specifically, look at the process list and the percentage of CPU consumed by each process.
+ Top SQL statements are consuming too much CPU.
Examine statements that are linked to the CPU usage to see whether they can use less CPU. Run an `EXPLAIN` command, and focus on the plan nodes that have the most impact. Consider using a PostgreSQL execution plan visualizer. To try out this tool, see [http://explain.dalibo.com/](http://explain.dalibo.com/).
### Respond to workload changes
If your workload has changed, look for the following types of changes:
New queries
Check whether the new queries are expected. If so, ensure that their execution plans and the number of executions per second are expected.
An increase in the size of the data set
Determine whether partitioning, if it's not already implemented, might help. This strategy might reduce the number of pages that a query needs to retrieve.
Index maintenance or creation
Check whether the schedule for the maintenance is expected. A best practice is to schedule maintenance activities outside of peak activities.
New functions
Check whether these functions perform as expected during testing. Specifically, check whether the number of executions per second is expected.
New operators
Check whether they perform as expected during the testing.
An increase in running parallel queries
Determine whether any of the following situations has occurred:
+ The relations or indexes involved have suddenly grown in size so that they differ significantly from `min_parallel_table_scan_size` or `min_parallel_index_scan_size`.
+ Recent changes have been made to `parallel_setup_cost` or `parallel_tuple_cost`.
+ Recent changes have been made to `max_parallel_workers` or `max_parallel_workers_per_gather`.
# IO:BufFileRead and IO:BufFileWrite
The `IO:BufFileRead` and `IO:BufFileWrite` events occur when RDS for PostgreSQL creates temporary files. When operations require more memory than the working memory parameters currently define, they write temporary data to persistent storage. This operation is sometimes called *spilling to disk*. For more information about the temporary files and their usage, see [Managing temporary files with PostgreSQL](PostgreSQL.ManagingTempFiles.md).
**Topics**
+ [
## Supported engine versions
](#wait-event.iobuffile.context.supported)
+ [
## Context
](#wait-event.iobuffile.context)
+ [
## Likely causes of increased waits
](#wait-event.iobuffile.causes)
+ [
## Actions
](#wait-event.iobuffile.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
`IO:BufFileRead` and `IO:BufFileWrite` relate to the work memory area and maintenance work memory area. For more information about these local memory areas, see [Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html) in the PostgreSQL documentation.
The default value for `work_mem` is 4 MB. If one session performs operations in parallel, each worker handling the parallelism uses 4 MB of memory. For this reason, set `work_mem` carefully. If you increase the value too much, a database running many sessions might consume too much memory. If you set the value too low, RDS for PostgreSQL creates temporary files in local storage. The disk I/O for these temporary files can reduce performance.
If you observe the following sequence of events, your database might be generating temporary files:
1. Sudden and sharp decreases in availability
1. Fast recovery for the free space
You might also see a "chainsaw" pattern. This pattern can indicate that your database is creating small files constantly.
## Likely causes of increased waits
In general, these wait events are caused by operations that consume more memory than the `work_mem` or `maintenance_work_mem` parameters allocate. To compensate, the operations write to temporary files. Common causes for the `IO:BufFileRead` and `IO:BufFileWrite` events include the following:
**Queries that need more memory than exists in the work memory area**
Queries with the following characteristics use the work memory area:
+ Hash joins
+ `ORDER BY` clause
+ `GROUP BY` clause
+ `DISTINCT`
+ Window functions
+ `CREATE TABLE AS SELECT`
+ Materialized view refresh
**Statements that need more memory than exists in the maintenance work memory area**
The following statements use the maintenance work memory area:
+ `CREATE INDEX`
+ `CLUSTER`
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Identify the problem
](#wait-event.iobuffile.actions.problem)
+ [
### Examine your join queries
](#wait-event.iobuffile.actions.joins)
+ [
### Examine your ORDER BY and GROUP BY queries
](#wait-event.iobuffile.actions.order-by)
+ [
### Avoid using the DISTINCT operation
](#wait-event.iobuffile.actions.distinct)
+ [
### Consider using window functions instead of GROUP BY functions
](#wait-event.iobuffile.actions.window)
+ [
### Investigate materialized views and CTAS statements
](#wait-event.iobuffile.actions.mv-refresh)
+ [
### Use pg\$1repack when you rebuild indexes
](#wait-event.iobuffile.actions.pg_repack)
+ [
### Increase maintenance\$1work\$1mem when you cluster tables
](#wait-event.iobuffile.actions.cluster)
+ [
### Tune memory to prevent IO:BufFileRead and IO:BufFileWrite
](#wait-event.iobuffile.actions.tuning-memory)
### Identify the problem
You can view temporary file usage directly in Performance Insights. For more information, see [Viewing temporary file usage with Performance Insights](PostgreSQL.ManagingTempFiles.Example.md). When Performance Insights is disabled, you might notice increased `IO:BufFileRead` and `IO:BufFileWrite` operations.
To identify the source of the problem, you can set the `log_temp_files` parameter to log all queries that generate more than your specified threshold KB of temporary files. By default, `log_temp_files` is set to `-1`, which turns off this logging feature. If you set this parameter to `0`, RDS for PostgreSQL logs all temporary files. If you set it to is `1024`, RDS for PostgreSQL logs all queries that produce temporary files larger than 1 MB. For more information about `log_temp_files`, see [Error Reporting and Logging](https://www.postgresql.org/docs/current/runtime-config-logging.html) in the PostgreSQL documentation.
### Examine your join queries
It's likely that your query uses joins. For example, the following query joins four tables.
```
SELECT *
FROM "order"
INNER JOIN order_item
ON (order.id = order_item.order_id)
INNER JOIN customer
ON (customer.id = order.customer_id)
INNER JOIN customer_address
ON (customer_address.customer_id = customer.id AND
order.customer_address_id = customer_address.id)
WHERE customer.id = 1234567890;
```
A possible cause of spikes in temporary file usage is a problem in the query itself. For example, a broken clause might not filter the joins properly. Consider the second inner join in the following example.
```
SELECT *
FROM "order"
INNER JOIN order_item
ON (order.id = order_item.order_id)
INNER JOIN customer
ON (customer.id = customer.id)
INNER JOIN customer_address
ON (customer_address.customer_id = customer.id AND
order.customer_address_id = customer_address.id)
WHERE customer.id = 1234567890;
```
The preceding query mistakenly joins `customer.id` to `customer.id`, generating a Cartesian product between every customer and every order. This type of accidental join generates large temporary files. Depending on the size of the tables, a Cartesian query can even fill up storage. Your application might have Cartesian joins when the following conditions are met:
+ You see large, sharp decreases in storage availability, followed by fast recovery.
+ No indexes are being created.
+ No `CREATE TABLE FROM SELECT` statements are being issued.
+ No materialized views are being refreshed.
To see whether the tables are being joined using the proper keys, inspect your query and object-relational mapping directives. Bear in mind that certain queries of your application are not called all the time, and some queries are dynamically generated.
### Examine your ORDER BY and GROUP BY queries
In some cases, an `ORDER BY` clause can result in excessive temporary files. Consider the following guidelines:
+ Only include columns in an `ORDER BY` clause when they need to be ordered. This guideline is especially important for queries that return thousands of rows and specify many columns in the `ORDER BY` clause.
+ Considering creating indexes to accelerate `ORDER BY` clauses when they match columns that have the same ascending or descending order. Partial indexes are preferable because they are smaller. Smaller indexes are read and traversed more quickly.
+ If you create indexes for columns that can accept null values, consider whether you want the null values stored at the end or at the beginning of the indexes.
If possible, reduce the number of rows that need to be ordered by filtering the result set. If you use `WITH` clause statements or subqueries, remember that an inner query generates a result set and passes it to the outside query. The more rows that a query can filter out, the less ordering the query needs to do.
+ If you don't need to obtain the full result set, use the `LIMIT` clause. For example, if you only want the top five rows, a query using the `LIMIT` clause doesn't keep generating results. In this way, the query requires less memory and temporary files.
A query that uses a `GROUP BY` clause can also require temporary files. `GROUP BY` queries summarize values by using functions such as the following:
+ `COUNT`
+ `AVG`
+ `MIN`
+ `MAX`
+ `SUM`
+ `STDDEV`
To tune `GROUP BY` queries, follow the recommendations for `ORDER BY` queries.
### Avoid using the DISTINCT operation
If possible, avoid using the `DISTINCT` operation to remove duplicated rows. The more unnecessary and duplicated rows that your query returns, the more expensive the `DISTINCT` operation becomes. If possible, add filters in the `WHERE` clause even if you use the same filters for different tables. Filtering the query and joining correctly improves your performance and reduces resource use. It also prevents incorrect reports and results.
If you need to use `DISTINCT` for multiple rows of a same table, consider creating a composite index. Grouping multiple columns in an index can improve the time to evaluate distinct rows. Also, if you use RDS for PostgreSQL version 10 or higher, you can correlate statistics among multiple columns by using the `CREATE STATISTICS` command.
### Consider using window functions instead of GROUP BY functions
Using `GROUP BY`, you change the result set, and then retrieve the aggregated result. Using window functions, you aggregate data without changing the result set. A window function uses the `OVER` clause to perform calculations across the sets defined by the query, correlating one row with another. You can use all the `GROUP BY` functions in window functions, but also use functions such as the following:
+ `RANK`
+ `ARRAY_AGG`
+ `ROW_NUMBER`
+ `LAG`
+ `LEAD`
To minimize the number of temporary files generated by a window function, remove duplications for the same result set when you need two distinct aggregations. Consider the following query.
```
SELECT sum(salary) OVER (PARTITION BY dept ORDER BY salary DESC) as sum_salary
, avg(salary) OVER (PARTITION BY dept ORDER BY salary ASC) as avg_salary
FROM empsalary;
```
You can rewrite the query with the `WINDOW` clause as follows.
```
SELECT sum(salary) OVER w as sum_salary
, avg(salary) OVER w as_avg_salary
FROM empsalary
WINDOW w AS (PARTITION BY dept ORDER BY salary DESC);
```
By default, the RDS for PostgreSQL execution planner consolidates similar nodes so that it doesn't duplicate operations. However, by using an explicit declaration for the window block, you can maintain the query more easily. You might also improve performance by preventing duplication.
### Investigate materialized views and CTAS statements
When a materialized view refreshes, it runs a query. This query can contain an operation such as `GROUP BY`, `ORDER BY`, or `DISTINCT`. During a refresh, you might observe large numbers of temporary files and the wait events `IO:BufFileWrite` and `IO:BufFileRead`. Similarly, when you create a table based on a `SELECT` statement, the `CREATE TABLE` statement runs a query. To reduce the temporary files needed, optimize the query.
### Use pg\$1repack when you rebuild indexes
When you create an index, the engine orders the result set. As tables grow in size, and as values in the indexed column become more diverse, the temporary files require more space. In most cases, you can't prevent the creation of temporary files for large tables without modifying the maintenance work memory area. For more information about `maintenance_work_mem`, see [https://www.postgresql.org/docs/current/runtime-config-resource.html](https://www.postgresql.org/docs/current/runtime-config-resource.html) in the PostgreSQL documentation.
A possible workaround when recreating a large index is to use the pg\$1repack extension. For more information, see [Reorganize tables in PostgreSQL databases with minimal locks](https://reorg.github.io/pg_repack/) in the pg\$1repack documentation. For information about setting up the extension in your RDS for PostgreSQL DB instance, see [Reducing bloat in tables and indexes with the pg\$1repack extension](Appendix.PostgreSQL.CommonDBATasks.pg_repack.md).
### Increase maintenance\$1work\$1mem when you cluster tables
The `CLUSTER` command clusters the table specified by *table\$1name* based on an existing index specified by *index\$1name*. RDS for PostgreSQL physically recreates the table to match the order of a given index.
When magnetic storage was prevalent, clustering was common because storage throughput was limited. Now that SSD-based storage is common, clustering is less popular. However, if you cluster tables, you can still increase performance slightly depending on the table size, index, query, and so on.
If you run the `CLUSTER` command and observe the wait events `IO:BufFileWrite` and `IO:BufFileRead`, tune `maintenance_work_mem`. Increase the memory size to a fairly large amount. A high value means that the engine can use more memory for the clustering operation.
### Tune memory to prevent IO:BufFileRead and IO:BufFileWrite
In some situations, you need to tune memory. Your goal is to balance memory across the following areas of consumption using the appropriate parameters, as follows.
+ The `work_mem` value
+ The memory remaining after discounting the `shared_buffers` value
+ The maximum connections opened and in use, which is limited by `max_connections`
For more information about tuning memory, see [Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html) in the PostgreSQL documentation.
#### Increase the size of the work memory area
In some situations, your only option is to increase the memory used by your session. If your queries are correctly written and are using the correct keys for joins, consider increasing the `work_mem` value.
To find out how many temporary files a query generates, set `log_temp_files` to `0`. If you increase the `work_mem` value to the maximum value identified in the logs, you prevent the query from generating temporary files. However, `work_mem` sets the maximum per plan node for each connection or parallel worker. If the database has 5,000 connections, and if each one uses 256 MiB memory, the engine needs 1.2 TiB of RAM. Thus, your instance might run out of memory.
#### Reserve sufficient memory for the shared buffer pool
Your database uses memory areas such as the shared buffer pool, not just the work memory area. Consider the requirements of these additional memory areas before you increase `work_mem`.
For example, assume that your RDS for PostgreSQL instance class is db.r5.2xlarge. This class has 64 GiB of memory. By default, 25 percent of the memory is reserved for the shared buffer pool. After you subtract the amount allocated to the shared memory area, 16,384 MB remains. Don't allocate the remaining memory exclusively to the work memory area because the operating system and the engine also require memory.
The memory that you can allocate to `work_mem` depends on the instance class. If you use a larger instance class, more memory is available. However, in the preceding example, you can't use more than 16 GiB. Otherwise, your instance becomes unavailable when it runs out of memory. To recover the instance from the unavailable state, the RDS for PostgreSQL automation services automatically restart.
#### Manage the number of connections
Suppose that your database instance has 5,000 simultaneous connections. Each connection uses at least 4 MiB of `work_mem`. The high memory consumption of the connections is likely to degrade performance. In response, you have the following options:
+ Upgrade to a larger instance class.
+ Decrease the number of simultaneous database connections by using a connection proxy or pooler.
For proxies, consider Amazon RDS Proxy, pgBouncer, or a connection pooler based on your application. This solution alleviates the CPU load. It also reduces the risk when all connections require the work memory area. When fewer database connections exist, you can increase the value of `work_mem`. In this way, you reduce the occurrence of the `IO:BufFileRead` and `IO:BufFileWrite` wait events. Also, the queries waiting for the work memory area speed up significantly.
# IO:DataFileRead
The `IO:DataFileRead` event occurs when a connection waits on a backend process to read a required page from storage because the page isn't available in shared memory.
**Topics**
+ [
## Supported engine versions
](#wait-event.iodatafileread.context.supported)
+ [
## Context
](#wait-event.iodatafileread.context)
+ [
## Likely causes of increased waits
](#wait-event.iodatafileread.causes)
+ [
## Actions
](#wait-event.iodatafileread.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
All queries and data manipulation (DML) operations access pages in the buffer pool. Statements that can induce reads include `SELECT`, `UPDATE`, and `DELETE`. For example, an `UPDATE` can read pages from tables or indexes. If the page being requested or updated isn't in the shared buffer pool, this read can lead to the `IO:DataFileRead` event.
Because the shared buffer pool is finite, it can fill up. In this case, requests for pages that aren't in memory force the database to read blocks from disk. If the `IO:DataFileRead` event occurs frequently, your shared buffer pool might be too small to accommodate your workload. This problem is acute for `SELECT` queries that read a large number of rows that don't fit in the buffer pool. For more information about the buffer pool, see [Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html) in the PostgreSQL documentation.
## Likely causes of increased waits
Common causes for the `IO:DataFileRead` event include the following:
**Connection spikes**
You might find multiple connections generating the same number of IO:DataFileRead wait events. In this case, a spike (sudden and large increase) in `IO:DataFileRead` events can occur.
**SELECT and DML statements performing sequential scans**
Your application might be performing a new operation. Or an existing operation might change because of a new execution plan. In such cases, look for tables (particularly large tables) that have a greater `seq_scan` value. Find them by querying `pg_stat_user_tables`. To track queries that are generating more read operations, use the extension `pg_stat_statements`.
**CTAS and CREATE INDEX for large data sets**
A *CTAS* is a `CREATE TABLE AS SELECT` statement. If you run a CTAS using a large data set as a source, or create an index on a large table, the `IO:DataFileRead` event can occur. When you create an index, the database might need to read the entire object using a sequential scan. A CTAS generates `IO:DataFile` reads when pages aren't in memory.
**Multiple vacuum workers running at the same time**
Vacuum workers can be triggered manually or automatically. We recommend adopting an aggressive vacuum strategy. However, when a table has many updated or deleted rows, the `IO:DataFileRead` waits increase. After space is reclaimed, the vacuum time spent on `IO:DataFileRead` decreases.
**Ingesting large amounts of data**
When your application ingests large amounts of data, `ANALYZE` operations might occur more often. The `ANALYZE` process can be triggered by an autovacuum launcher or invoked manually.
The `ANALYZE` operation reads a subset of the table. The number of pages that must be scanned is calculated by multiplying 30 by the `default_statistics_target` value. For more information, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-DEFAULT-STATISTICS-TARGET). The `default_statistics_target` parameter accepts values between 1 and 10,000, where the default is 100.
**Resource starvation**
If instance network bandwidth or CPU are consumed, the `IO:DataFileRead` event might occur more frequently.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Check predicate filters for queries that generate waits
](#wait-event.iodatafileread.actions.filters)
+ [
### Minimize the effect of maintenance operations
](#wait-event.iodatafileread.actions.maintenance)
+ [
### Respond to high numbers of connections
](#wait-event.iodatafileread.actions.connections)
### Check predicate filters for queries that generate waits
Assume that you identify specific queries that are generating `IO:DataFileRead` wait events. You might identify them using the following techniques:
+ Performance Insights
+ Catalog views such as the one provided by the extension `pg_stat_statements`
+ The catalog view `pg_stat_all_tables`, if it periodically shows an increased number of physical reads
+ The `pg_statio_all_tables` view, if it shows that `_read` counters are increasing
We recommend that you determine which filters are used in the predicate (`WHERE` clause) of these queries. Follow these guidelines:
+ Run the `EXPLAIN` command. In the output, identify which types of scans are used. A sequential scan doesn't necessarily indicate a problem. Queries that use sequential scans naturally produce more `IO:DataFileRead` events when compared to queries that use filters.
Find out whether the column listed in the `WHERE` clause is indexed. If not, consider creating an index for this column. This approach avoids the sequential scans and reduces the `IO:DataFileRead` events. If a query has restrictive filters and still produces sequential scans, evaluate whether the proper indexes are being used.
+ Find out whether the query is accessing a very large table. In some cases, partitioning a table can improve performance, allowing the query to only read necessary partitions.
+ Examine the cardinality (total number of rows) from your join operations. Note how restrictive the values are that you're passing in the filters for your `WHERE` clause. If possible, tune your query to reduce the number of rows that are passed in each step of the plan.
### Minimize the effect of maintenance operations
Maintenance operations such as `VACUUM` and `ANALYZE` are important. We recommend that you don't turn them off because you find `IO:DataFileRead` wait events related to these maintenance operations. The following approaches can minimize the effect of these operations:
+ Run maintenance operations manually during off-peak hours. This technique prevents the database from reaching the threshold for automatic operations.
+ For very large tables, consider partitioning the table. This technique reduces the overhead of maintenance operations. The database only accesses the partitions that require maintenance.
+ When you ingest large amounts of data, consider disabling the autoanalyze feature.
The autovacuum feature is automatically triggered for a table when the following formula is true.
```
pg_stat_user_tables.n_dead_tup > (pg_class.reltuples x autovacuum_vacuum_scale_factor) + autovacuum_vacuum_threshold
```
The view `pg_stat_user_tables` and catalog `pg_class` have multiple rows. One row can correspond to one row in your table. This formula assumes that the `reltuples` are for a specific table. The parameters `autovacuum_vacuum_scale_factor` (0.20 by default) and `autovacuum_vacuum_threshold` (50 tuples by default) are usually set globally for the whole instance. However, you can set different values for a specific table.
**Topics**
+ [
#### Find tables consuming space unnecessarily
](#wait-event.iodatafileread.actions.maintenance.tables)
+ [
#### Find indexes consuming space unnecessarily
](#wait-event.iodatafileread.actions.maintenance.indexes)
+ [
#### Find tables that are eligible to be autovacuumed
](#wait-event.iodatafileread.actions.maintenance.autovacuumed)
#### Find tables consuming space unnecessarily
To find tables consuming space unnecessarily, you can use functions from the PostgreSQL `pgstattuple` extension. This extension (module) is available by default on all RDS for PostgreSQL DB instances and can be instantiated on the instance with the following command.
```
CREATE EXTENSION pgstattuple;
```
For more information about this extension, see [pgstattuple](https://www.postgresql.org/docs/current/pgstattuple.html) in the PostgreSQL documentation.
You can check for table and index bloat in your application. For more information, see [Diagnosing table and index bloat](https://docs.aws.amazon.com//AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.diag-table-ind-bloat.html).
#### Find indexes consuming space unnecessarily
To find bloated indexes and estimate the amount of space consumed unnecessarily on the tables for which you have read privileges, you can run the following query.
```
-- WARNING: rows with is_na = 't' are known to have bad statistics ("name" type is not supported).
-- This query is compatible with PostgreSQL 8.2 and later.
SELECT current_database(), nspname AS schemaname, tblname, idxname, bs*(relpages)::bigint AS real_size,
bs*(relpages-est_pages)::bigint AS extra_size,
100 * (relpages-est_pages)::float / relpages AS extra_ratio,
fillfactor, bs*(relpages-est_pages_ff) AS bloat_size,
100 * (relpages-est_pages_ff)::float / relpages AS bloat_ratio,
is_na
-- , 100-(sub.pst).avg_leaf_density, est_pages, index_tuple_hdr_bm,
-- maxalign, pagehdr, nulldatawidth, nulldatahdrwidth, sub.reltuples, sub.relpages
-- (DEBUG INFO)
FROM (
SELECT coalesce(1 +
ceil(reltuples/floor((bs-pageopqdata-pagehdr)/(4+nulldatahdrwidth)::float)), 0
-- ItemIdData size + computed avg size of a tuple (nulldatahdrwidth)
) AS est_pages,
coalesce(1 +
ceil(reltuples/floor((bs-pageopqdata-pagehdr)*fillfactor/(100*(4+nulldatahdrwidth)::float))), 0
) AS est_pages_ff,
bs, nspname, table_oid, tblname, idxname, relpages, fillfactor, is_na
-- , stattuple.pgstatindex(quote_ident(nspname)||'.'||quote_ident(idxname)) AS pst,
-- index_tuple_hdr_bm, maxalign, pagehdr, nulldatawidth, nulldatahdrwidth, reltuples
-- (DEBUG INFO)
FROM (
SELECT maxalign, bs, nspname, tblname, idxname, reltuples, relpages, relam, table_oid, fillfactor,
( index_tuple_hdr_bm +
maxalign - CASE -- Add padding to the index tuple header to align on MAXALIGN
WHEN index_tuple_hdr_bm%maxalign = 0 THEN maxalign
ELSE index_tuple_hdr_bm%maxalign
END
+ nulldatawidth + maxalign - CASE -- Add padding to the data to align on MAXALIGN
WHEN nulldatawidth = 0 THEN 0
WHEN nulldatawidth::integer%maxalign = 0 THEN maxalign
ELSE nulldatawidth::integer%maxalign
END
)::numeric AS nulldatahdrwidth, pagehdr, pageopqdata, is_na
-- , index_tuple_hdr_bm, nulldatawidth -- (DEBUG INFO)
FROM (
SELECT
i.nspname, i.tblname, i.idxname, i.reltuples, i.relpages, i.relam, a.attrelid AS table_oid,
current_setting('block_size')::numeric AS bs, fillfactor,
CASE -- MAXALIGN: 4 on 32bits, 8 on 64bits (and mingw32 ?)
WHEN version() ~ 'mingw32' OR version() ~ '64-bit|x86_64|ppc64|ia64|amd64' THEN 8
ELSE 4
END AS maxalign,
/* per page header, fixed size: 20 for 7.X, 24 for others */
24 AS pagehdr,
/* per page btree opaque data */
16 AS pageopqdata,
/* per tuple header: add IndexAttributeBitMapData if some cols are null-able */
CASE WHEN max(coalesce(s.null_frac,0)) = 0
THEN 2 -- IndexTupleData size
ELSE 2 + (( 32 + 8 - 1 ) / 8)
-- IndexTupleData size + IndexAttributeBitMapData size ( max num filed per index + 8 - 1 /8)
END AS index_tuple_hdr_bm,
/* data len: we remove null values save space using it fractionnal part from stats */
sum( (1-coalesce(s.null_frac, 0)) * coalesce(s.avg_width, 1024)) AS nulldatawidth,
max( CASE WHEN a.atttypid = 'pg_catalog.name'::regtype THEN 1 ELSE 0 END ) > 0 AS is_na
FROM pg_attribute AS a
JOIN (
SELECT nspname, tbl.relname AS tblname, idx.relname AS idxname,
idx.reltuples, idx.relpages, idx.relam,
indrelid, indexrelid, indkey::smallint[] AS attnum,
coalesce(substring(
array_to_string(idx.reloptions, ' ')
from 'fillfactor=([0-9]+)')::smallint, 90) AS fillfactor
FROM pg_index
JOIN pg_class idx ON idx.oid=pg_index.indexrelid
JOIN pg_class tbl ON tbl.oid=pg_index.indrelid
JOIN pg_namespace ON pg_namespace.oid = idx.relnamespace
WHERE pg_index.indisvalid AND tbl.relkind = 'r' AND idx.relpages > 0
) AS i ON a.attrelid = i.indexrelid
JOIN pg_stats AS s ON s.schemaname = i.nspname
AND ((s.tablename = i.tblname AND s.attname = pg_catalog.pg_get_indexdef(a.attrelid, a.attnum, TRUE))
-- stats from tbl
OR (s.tablename = i.idxname AND s.attname = a.attname))
-- stats from functional cols
JOIN pg_type AS t ON a.atttypid = t.oid
WHERE a.attnum > 0
GROUP BY 1, 2, 3, 4, 5, 6, 7, 8, 9
) AS s1
) AS s2
JOIN pg_am am ON s2.relam = am.oid WHERE am.amname = 'btree'
) AS sub
-- WHERE NOT is_na
ORDER BY 2,3,4;
```
#### Find tables that are eligible to be autovacuumed
To find tables that are eligible to be autovacuumed, run the following query.
```
--This query shows tables that need vacuuming and are eligible candidates.
--The following query lists all tables that are due to be processed by autovacuum.
-- During normal operation, this query should return very little.
WITH vbt AS (SELECT setting AS autovacuum_vacuum_threshold
FROM pg_settings WHERE name = 'autovacuum_vacuum_threshold')
, vsf AS (SELECT setting AS autovacuum_vacuum_scale_factor
FROM pg_settings WHERE name = 'autovacuum_vacuum_scale_factor')
, fma AS (SELECT setting AS autovacuum_freeze_max_age
FROM pg_settings WHERE name = 'autovacuum_freeze_max_age')
, sto AS (SELECT opt_oid, split_part(setting, '=', 1) as param,
split_part(setting, '=', 2) as value
FROM (SELECT oid opt_oid, unnest(reloptions) setting FROM pg_class) opt)
SELECT
'"'||ns.nspname||'"."'||c.relname||'"' as relation
, pg_size_pretty(pg_table_size(c.oid)) as table_size
, age(relfrozenxid) as xid_age
, coalesce(cfma.value::float, autovacuum_freeze_max_age::float) autovacuum_freeze_max_age
, (coalesce(cvbt.value::float, autovacuum_vacuum_threshold::float) +
coalesce(cvsf.value::float,autovacuum_vacuum_scale_factor::float) * c.reltuples)
as autovacuum_vacuum_tuples
, n_dead_tup as dead_tuples
FROM pg_class c
JOIN pg_namespace ns ON ns.oid = c.relnamespace
JOIN pg_stat_all_tables stat ON stat.relid = c.oid
JOIN vbt on (1=1)
JOIN vsf ON (1=1)
JOIN fma on (1=1)
LEFT JOIN sto cvbt ON cvbt.param = 'autovacuum_vacuum_threshold' AND c.oid = cvbt.opt_oid
LEFT JOIN sto cvsf ON cvsf.param = 'autovacuum_vacuum_scale_factor' AND c.oid = cvsf.opt_oid
LEFT JOIN sto cfma ON cfma.param = 'autovacuum_freeze_max_age' AND c.oid = cfma.opt_oid
WHERE c.relkind = 'r'
AND nspname <> 'pg_catalog'
AND (
age(relfrozenxid) >= coalesce(cfma.value::float, autovacuum_freeze_max_age::float)
or
coalesce(cvbt.value::float, autovacuum_vacuum_threshold::float) +
coalesce(cvsf.value::float,autovacuum_vacuum_scale_factor::float) * c.reltuples <= n_dead_tup
-- or 1 = 1
)
ORDER BY age(relfrozenxid) DESC;
```
### Respond to high numbers of connections
When you monitor Amazon CloudWatch, you might find that the `DatabaseConnections` metric spikes. This increase indicates an increased number of connections to your database. We recommend the following approach:
+ Limit the number of connections that the application can open with each instance. If your application has an embedded connection pool feature, set a reasonable number of connections. Base the number on what the vCPUs in your instance can parallelize effectively.
If your application doesn't use a connection pool feature, considering using Amazon RDS Proxy or an alternative. This approach lets your application open multiple connections with the load balancer. The balancer can then open a restricted number of connections with the database. As fewer connections are running in parallel, your DB instance performs less context switching in the kernel. Queries should progress faster, leading to fewer wait events. For more information, see [Amazon RDS Proxy](rds-proxy.md).
+ Whenever possible, take advantage of read replicas for RDS for PostgreSQL. When your application runs a read-only operation, send these requests to the read replica(s). This technique reduces the I/O pressure on the primary (writer) node.
+ Consider scaling up your DB instance. A higher-capacity instance class gives more memory, which gives RDS for PostgreSQL a larger shared buffer pool to hold pages. The larger size also gives the DB instance more vCPUs to handle connections. More vCPUs are particularly helpful when the operations that are generating `IO:DataFileRead` wait events are writes.
# IO:WALWrite
**Topics**
+ [
## Supported engine versions
](#wait-event.iowalwrite.context.supported)
+ [
## Context
](#wait-event.iowalwrite.context)
+ [
## Likely causes of increased waits
](#wait-event.iowalwrite.causes)
+ [
## Actions
](#wait-event.iowalwrite.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL 10 and higher.
## Context
Activity in the database that's generating write-ahead log data fills up the WAL buffers first and then writes to disk, asynchronously. The wait event `IO:WALWrite` is generated when the SQL session is waiting for the WAL data to complete writing to disk so that it can release the transaction's COMMIT call.
## Likely causes of increased waits
If this wait event occurs often, you should review your workload and the type of updates that your workload performs and their frequency. In particular, look for the following types of activity.
**Heavy DML activity**
Changing data in database tables doesn't happen instantaneously. An insert to one table might need to wait for an insert or an update to the same table from another client. The data manipulation language (DML) statements for changing data values (INSERT, UPDATE, DELETE, COMMIT, ROLLBACK TRANSACTION) can result in contention that causes the write-ahead logfile to be waiting for the buffers to be flushed. This situation is captured in the following Amazon RDS Performance Insights metrics that indicate heavy DML activity.
+ `tup_inserted`
+ `tup_updated`
+ `tup_deleted`
+ `xact_rollback`
+ `xact_commit`
For more information about these metrics, see [Performance Insights counters for Amazon RDS for PostgreSQL](USER_PerfInsights_Counters.md#USER_PerfInsights_Counters.PostgreSQL).
**Frequent checkpoint activity**
Frequent checkpoints contribute to a higher number of WAL files. In RDS for PostgreSQL, full page writes are always "on." Full page writes help protect against data loss. However, when checkpointing occurs too frequently, the system can suffer overall performance issues. This is especially true on systems with heavy DML activity. In some cases, you might find error messages in your `postgresql.log` stating that “checkpoints are occurring too frequently."
We recommend that when tuning checkpoints, you carefully balance performance against expected time need to recover in the event of an abnormal shutdown.
## Actions
We recommend the following actions to reduce the numbers of this wait event.
**Topics**
+ [
### Reduce the number of commits
](#wait-event.iowalwrite.actions.problem)
+ [
### Monitor your checkpoints
](#wait-event.iowalwrite.actions.monitor)
+ [
### Scale up IO
](#wait-event.iowalwrite.actions.scale-io)
+ [
### Dedicated log volume (DLV)
](#wait-event.iowalwrite.actions.dlv)
### Reduce the number of commits
To reduce the number of commits, you can combine statements into transaction blocks. Use Amazon RDS Performance Insights to examine the type of queries being run. You can also move large maintenance operations to off-peak hours. For example, create indexes or use `pg_repack` operations during non-production hours.
### Monitor your checkpoints
There are two parameters that you can monitor to see how frequently your RDS for PostgreSQL DB instance is writing to the WAL file for checkpoints.
+ `log_checkpoints` – This parameter is set to "on" by default. It causes a message to get sent to the PostgreSQL log for each checkpoint. These log messages include the number of buffers written, the time spent writing them, and the number of WAL files added, removed, or recycled for the given checkpoint.
For more information about this parameter, see [Error Reporting and Logging](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-CHECKPOINTS) in the PostgreSQL documentation.
+ `checkpoint_warning` – This parameter sets a threshold value (in seconds) for checkpoint frequency above which a warning is generated. By default, this parameter isn't set in RDS for PostgreSQL. You can set the value of this parameter to get a warning when the database changes in your RDS for PostgreSQL DB instance are written at a rate for which the WAL files are not sized to handle. For example, say you set this parameter to 30. If your RDS for PostgreSQL instance needs to write changes more often than every 30 seconds, the warning that "checkpoints are occurring too frequently" is sent to the PostgreSQL log. This can indicate that your `max_wal_size` value should be increased.
For more information, see [Write Ahead Log](https://www.postgresql.org/docs/current/runtime-config-wal.html#RUNTIME-CONFIG-WAL-CHECKPOINTS) in the PostgreSQL documentation.
### Scale up IO
This type of input/output (IO) wait event can remediated by scaling the input/output operations per second (IOPs) to provide faster IO. Scaling IO is preferable to scaling CPU, because scaling CPU can result in even more IO contention because the increased CPU can handle more work and thus make the IO bottleneck even worse. In general, we recommend that you consider tuning your workload before performing scaling operations.
### Dedicated log volume (DLV)
You can use a dedicated log volume (DLV) for a DB instance that uses Provisioned IOPS (PIOPS) storage by using the Amazon RDS console, AWS CLI, or Amazon RDS API. A DLV moves PostgreSQL database transaction logs to a storage volume that's separate from the volume containing the database tables. For more information, see [Dedicated log volume (DLV)](CHAP_Storage.md#CHAP_Storage.dlv).
# IPC:parallel wait events
The following `IPC:parallel wait events` indicate that a session is waiting for inter-process communication related to parallel query execution operations.
+ `IPC:BgWorkerStartup` - A process is waiting for a parallel worker process to complete its startup sequence. This happens when initializing workers for parallel query execution.
+ `IPC:BgWorkerShutdown` - A process is waiting for a parallel worker process to complete its shutdown sequence. This occurs during the cleanup phase of parallel query execution.
+ `IPC:ExecuteGather` - A process is waiting to receive data from parallel worker processes during query execution. This occurs when the leader process needs to gather results from its workers.
+ `IPC:ParallelFinish` - A process is waiting for parallel workers to finish their execution and report their final results. This happens during the completion phase of parallel query execution.
**Topics**
+ [
## Supported engine versions
](#rpg-ipc-parallel-context-supported)
+ [
## Context
](#rpg-ipc-parallel-context)
+ [
## Likely causes of increased waits
](#rpg-ipc-parallel-causes)
+ [
## Actions
](#rpg-ipc-parallel-actions)
## Supported engine versions
This wait event information is supported for all versions of Aurora PostgreSQL.
## Context
Parallel query execution in PostgreSQL involves multiple processes working together to process a single query. When a query is determined to be suitable for parallelization, a leader process coordinates with one or more parallel worker processes based on the `max_parallel_workers_per_gather` parameter setting. The leader process divides the work among workers, each worker processes its portion of data independently, and results are gathered back to the leader process.
**Note**
Each parallel worker operates as a separate process with resource requirements similar to a full user session. This means a parallel query with 4 workers can consume up to 5 times the resources (CPU, memory, I/O bandwidth) compared to a non-parallel query, as both the leader process and each worker process maintain their own resource allocations. For instance, settings like `work_mem` are applied individually to each worker, potentially multiplying the total memory usage across all processes.
The parallel query architecture consists of three main components:
+ Leader process: The main process that initiates the parallel operation, divides the workload, and coordinates with worker processes.
+ Worker processes: Background processes that execute portions of the query in parallel.
+ Gather/Gather merge: Operations that combine results from multiple worker processes back to the leader
During parallel execution, processes need to communicate with each other through Inter-Process Communication (IPC) mechanisms. These IPC wait events occur during different phases:
+ Worker startup: When parallel workers are being initialized
+ Data exchange: When workers are processing data and sending results to the leader
+ Worker shutdown: When parallel execution completes and workers are being terminated
+ Synchronization points: When processes need to coordinate or wait for other processes to complete their tasks
Understanding these wait events is crucial for diagnosing performance issues related to parallel query execution, especially in high-concurrency environments where multiple parallel queries may be executing simultaneously.
## Likely causes of increased waits
Several factors can contribute to an increase in parallel-related IPC wait events:
**High concurrency of parallel queries**
When many parallel queries are running simultaneously, it can lead to resource contention and increased waiting times for IPC operations. This is particularly common in systems with high transaction volumes or analytical workloads.
**Suboptimal parallel query plans**
If the query planner chooses inefficient parallel plans, it may result in unnecessary parallelization or poor work distribution among workers. This can lead to increased IPC waits, especially for `IPC:ExecuteGather` and `IPC:ParallelFinish` events. These planning issues often stem from outdated statistics and table/index bloat.
**Frequent startup and shutdown of parallel workers**
Short-lived queries that frequently initiate and terminate parallel workers can cause an increase in `IPC:BgWorkerStartup` and `IPC:BgWorkerShutdown` events. This is often seen in OLTP workloads with many small, parallelizable queries.
**Resource constraints**
Limited CPU, memory, or I/O capacity can cause bottlenecks in parallel execution, leading to increased wait times across all IPC events. For example, if CPU is saturated, worker processes may take longer to start up or process their portion of work.
**Complex query structures**
Queries with multiple levels of parallelism (e.g., parallel joins followed by parallel aggregations) can lead to more complex IPC patterns and potentially increased wait times, especially for `IPC:ExecuteGather` events.
**Large result sets**
Queries that produce large result sets may cause increased `IPC:ExecuteGather` wait times as the leader process spends more time collecting and processing results from worker processes.
Understanding these factors can help in diagnosing and addressing performance issues related to parallel query execution in Aurora PostgreSQL.
## Actions
When you see waits related to parallel query, it typically means that a backend process is coordinating or waiting on parallel worker processes. These waits are common during execution of parallel plans. You can investigate and mitigate the impact of these waits by monitoring parallel worker usage, reviewing the parameter settings, and tuning query execution and resource allocation.
**Topics**
+ [
### Analyze query plans for inefficient parallelism
](#rpg-ipc-parallel-analyze-plans)
+ [
### Monitor parallel query usage
](#rpg-ipc-parallel-monitor)
+ [
### Review and adjust parallel query settings
](#rpg-ipc-parallel-adjust-settings)
+ [
### Optimize resource allocation
](#rpg-ipc-parallel-optimize-resources)
+ [
### Investigate connection management
](#rpg-ipc-parallel-connection-management)
+ [
### Review and optimize maintenance operations
](#rpg-ipc-parallel-maintenance)
### Analyze query plans for inefficient parallelism
Parallel query execution can often lead to system instability, CPU spikes, and unpredictable query performance variance. It's crucial to thoroughly analyze whether parallelism actually improves your specific workload. Use EXPLAIN ANALYZE to review parallel query execution plans.
Temporarily disable parallelism at the session level to compare plan efficiency:
```
SET max_parallel_workers_per_gather = 0;
EXPLAIN ANALYZE ;
```
Re-enable parallelism and compare:
```
RESET max_parallel_workers_per_gather;
EXPLAIN ANALYZE ;
```
If disabling parallelism yields better or more consistent results, consider disabling it for specific queries at the session level using SET commands. For a broader impact, you might want to disable parallelism at the instance level by adjusting the relevant parameters in your DB parameter group. For more information, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md).
### Monitor parallel query usage
Use the following queries to gain visibility into parallel query activity and capacity:
Check active parallel worker processes:
```
SELECT
COUNT(*)
FROM
pg_stat_activity
WHERE
backend_type = 'parallel worker';
```
This query shows the number of active parallel worker processes. A high value may indicate that your `max\$1parallel\$1workers` is configured with a high value and you might want to consider reducing it.
Check concurrent parallel queries:
```
SELECT
COUNT(DISTINCT leader_pid)
FROM
pg_stat_activity
WHERE
leader_pid IS NOT NULL;
```
This query returns the number of distinct leader processes that have launched parallel queries. A high number here indicates that multiple sessions are running parallel queries concurrently, which can increase demand on CPU and memory.
### Review and adjust parallel query settings
Review the following parameters to ensure they align with your workload:
+ `max_parallel_workers`: Total number of parallel workers across all sessions.
+ `max_parallel_workers_per_gather`: Max workers per query.
For OLAP workloads, increasing these values can improve performance. For OLTP workloads, lower values are generally preferred.
```
SHOW max_parallel_workers;
SHOW max_parallel_workers_per_gather;
```
### Optimize resource allocation
Monitor CPU utilization and consider adjusting the number of vCPUs if consistently high and if your application benefits from parallel queries. Ensure adequate memory is available for parallel operations.
+ Use Performance Insights metrics to determine if the system is CPU-bound.
+ Each parallel worker uses its own `work_mem`. Ensure total memory usage is within instance limits.
The parallel queries may consume very substantially more resources than non-parallel queries, because each worker process is a completely separate process which has roughly the same impact on the system as an additional user session. This should be taken into account when choosing a value for this setting, as well as when configuring other settings that control resource utilization, such as `work_mem`. For more information, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-WORK-MEM). Resource limits such as `work_mem` are applied individually to each worker, which means the total utilization may be much higher across all processes than it would normally be for any single process.
Consider increasing vCPUs or tuning memory parameters if your workload is heavily parallelized.
### Investigate connection management
If experiencing connection exhaustion, review application connection pooling strategies. Consider implementing connection pooling at the application level if not already in use.
### Review and optimize maintenance operations
Coordinate index creation and other maintenance tasks to prevent resource contention. Consider scheduling these operations during off-peak hours. Avoid scheduling heavy maintenance (e.g., parallel index builds) during periods of high user query load. These operations can consume parallel workers and impact performance for regular queries.
# IPC:ProcArrayGroupUpdate
The `IPC:ProcArrayGroupUpdate` event occurs when a session is waiting for the group leader to update the transaction status at the end of that operation. While PostgreSQL generally associates IPC type wait events with parallel query operations, this particular wait event is not specific to parallel queries.
**Topics**
+ [
## Supported engine versions
](#apg-rpg-ipcprocarraygroup.supported)
+ [
## Context
](#apg-rpg-ipcprocarraygroup.context)
+ [
## Likely causes of increased waits
](#apg-rpg-ipcprocarraygroup.causes)
+ [
## Actions
](#apg-rpg-ipcprocarraygroup.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
**Understanding the process array** – The process (proc) array is a shared memory structure in PostgreSQL. It holds information about all running processes, including transaction details. During transaction completion (`COMMIT` or `ROLLBACK`), the ProcArray needs to be updated to reflect the change and clear the transactionID from the array. The session attempting to finish its transaction must acquire an exclusive lock on the ProcArray. This prevents other processes from obtaining shared or exclusive locks on it.
**Group update mechanism** – While performing a COMMIT or ROLLBACK, if a backend process cannot obtain a ProcArrayLock in exclusive mode, it updates a special field called ProcArrayGroupMember. This adds the transaction to the list of sessions that intend to end. This backend process then sleeps and the time it sleeps is instrumented as the ProcArrayGroupUpdate wait event. The first process in the ProcArray with procArrayGroupMember, referred to as the leader process, acquires the ProcArrayLock in exclusive mode. It then clears the list of processes waiting for group transactionID clearing. Once this completes, the leader releases the ProcArrayLock and then wakes up all processes in this list, notifying them that their transaction is completed.
## Likely causes of increased waits
The more processes that are running, the longer a leader will hold on to a procArrayLock in exclusive mode. Consequently, the more write transactions end up in a group update scenario causing a potential pile up of processes waiting on the `ProcArrayGroupUpdate` wait event. In Database Insights' Top SQL view, you will see that COMMIT is the statement with the majority of this wait event. This is expected but will require deeper investigation into the specific write SQL being run to determine what appropriate action to take.
## Actions
We recommend different actions depending on the causes of your wait event. Identify `IPC:ProcArrayGroupUpdate` events by using Amazon RDS Performance Insights or by querying the PostgreSQL system view `pg_stat_activity`.
**Topics**
+ [
### Monitoring transaction commit and rollback operations
](#apg-rpg-ipcprocarraygroup.actions.monitor)
+ [
### Reducing concurrency
](#apg-rpg-ipcprocarraygroup.actions.concurrency)
+ [
### Implementing connection pooling
](#apg-rpg-ipcprocarraygroup.actions.pooling)
+ [
### Using faster storage
](#apg-rpg-ipcprocarraygroup.actions.storage)
### Monitoring transaction commit and rollback operations
**Monitor commits and rollbacks** – An increased number of commits and rollbacks can lead to increased pressure on the ProcArray. For example, if a SQL statement begins to fail due to increased duplicate key violations, you may see an increase in rollbacks which can increase ProcArray contention and table bloat.
Amazon RDS Database Insights provides the PostgreSQL metrics `xact_commit` and `xact_rollback` to report the number of commits and rollbacks per second.
### Reducing concurrency
**Batching transactions** – Where possible, batch operations in single transactions to reduce commit/rollback operations.
**Limit concurrency** – Reduce the number of concurrently active transactions to alleviate lock contention on the ProcArray. While it will require some testing, reducing the total number of concurrent connections can reduce contention and maintain throughput.
### Implementing connection pooling
**Connection pooling solutions** – Use connection pooling to manage database connections efficiently, reducing the total number of backends and therefore the workload on the ProcArray. While it will require some testing, reducing the total number of concurrent connections can reduce contention and maintain throughput.
**Reduce connection storms** – Similarly, a pattern of frequently creating and terminating connections causes additional pressure on the ProcArray. By reducing this pattern, overall contention is reduced.
### Using faster storage
**Dedicated log volume** – If the `IPC:ProcArrayGroupUpdate` wait event is accompanied with high `IO:WALWrite` wait events, setting up a dedicated log volume can reduce the bottleneck writing to WAL. In turn, this improves the performance of commits.
For more information, see [Dedicated log volume](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIOPS.dlv.html).
# Lock:advisory
The `Lock:advisory` event occurs when a PostgreSQL application uses a lock to coordinate activity across multiple sessions.
**Topics**
+ [
## Relevant engine versions
](#wait-event.lockadvisory.context.supported)
+ [
## Context
](#wait-event.lockadvisory.context)
+ [
## Causes
](#wait-event.lockadvisory.causes)
+ [
## Actions
](#wait-event.lockadvisory.actions)
## Relevant engine versions
This wait event information is relevant for RDS for PostgreSQL versions 9.6 and higher.
## Context
PostgreSQL advisory locks are application-level, cooperative locks explicitly locked and unlocked by the user's application code. An application can use PostgreSQL advisory locks to coordinate activity across multiple sessions. Unlike regular, object- or row-level locks, the application has full control over the lifetime of the lock. For more information, see [Advisory Locks](https://www.postgresql.org/docs/12/explicit-locking.html#ADVISORY-LOCKS) in the PostgreSQL documentation.
Advisory locks can be released before a transaction ends or be held by a session across transactions. This isn't true for implicit, system-enforced locks, such as an access-exclusive lock on a table acquired by a `CREATE INDEX` statement.
For a description of the functions used to acquire (lock) and release (unlock) advisory locks, see [Advisory Lock Functions](https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADVISORY-LOCKS) in the PostgreSQL documentation.
Advisory locks are implemented on top of the regular PostgreSQL locking system and are visible in the `pg_locks` system view.
## Causes
This lock type is exclusively controlled by an application explicitly using it. Advisory locks that are acquired for each row as part of a query can cause a spike in locks or a long-term buildup.
These effects happen when the query is run in a way that acquires locks on more rows than are returned by the query. The application must eventually release every lock, but if locks are acquired on rows that aren't returned, the application can't find all of the locks.
The following example is from [Advisory Locks](https://www.postgresql.org/docs/12/explicit-locking.html#ADVISORY-LOCKS) in the PostgreSQL documentation.
```
SELECT pg_advisory_lock(id) FROM foo WHERE id > 12345 LIMIT 100;
```
In this example, the `LIMIT` clause can only stop the query's output after the rows have already been internally selected and their ID values locked. This can happen suddenly when a growing data volume causes the planner to choose a different execution plan that wasn't tested during development. The buildup in this case happens because the application explicitly calls `pg_advisory_unlock` for every ID value that was locked. However, in this case it can't find the set of locks acquired on rows that weren't returned. Because the locks are acquired on the session level, they aren't released automatically at the end of the transaction.
Another possible cause for spikes in blocked lock attempts is unintended conflicts. In these conflicts, unrelated parts of the application share the same lock ID space by mistake.
## Actions
Review application usage of advisory locks and detail where and when in the application flow each type of advisory lock is acquired and released.
Determine whether a session is acquiring too many locks or a long-running session isn't releasing locks early enough, leading to a slow buildup of locks. You can correct a slow buildup of session-level locks by ending the session using `pg_terminate_backend(pid)`.
A client waiting for an advisory lock appears in `pg_stat_activity` with `wait_event_type=Lock` and `wait_event=advisory`. You can obtain specific lock values by querying the `pg_locks` system view for the same `pid`, looking for `locktype=advisory` and `granted=f`.
You can then identify the blocking session by querying `pg_locks` for the same advisory lock having `granted=t`, as shown in the following example.
```
SELECT blocked_locks.pid AS blocked_pid,
blocking_locks.pid AS blocking_pid,
blocked_activity.usename AS blocked_user,
blocking_activity.usename AS blocking_user,
now() - blocked_activity.xact_start AS blocked_transaction_duration,
now() - blocking_activity.xact_start AS blocking_transaction_duration,
concat(blocked_activity.wait_event_type,':',blocked_activity.wait_event) AS blocked_wait_event,
concat(blocking_activity.wait_event_type,':',blocking_activity.wait_event) AS blocking_wait_event,
blocked_activity.state AS blocked_state,
blocking_activity.state AS blocking_state,
blocked_locks.locktype AS blocked_locktype,
blocking_locks.locktype AS blocking_locktype,
blocked_activity.query AS blocked_statement,
blocking_activity.query AS blocking_statement
FROM pg_catalog.pg_locks blocked_locks
JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid
JOIN pg_catalog.pg_locks blocking_locks
ON blocking_locks.locktype = blocked_locks.locktype
AND blocking_locks.DATABASE IS NOT DISTINCT FROM blocked_locks.DATABASE
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
AND blocking_locks.pid != blocked_locks.pid
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
WHERE NOT blocked_locks.GRANTED;
```
All of the advisory lock API functions have two sets of arguments, either one `bigint` argument or two `integer` arguments:
+ For the API functions with one `bigint` argument, the upper 32 bits are in `pg_locks.classid` and the lower 32 bits are in `pg_locks.objid`.
+ For the API functions with two `integer` arguments, the first argument is `pg_locks.classid` and the second argument is `pg_locks.objid`.
The `pg_locks.objsubid` value indicates which API form was used: `1` means one `bigint` argument; `2` means two `integer` arguments.
# Lock:extend
The `Lock:extend` event occurs when a backend process is waiting to lock a relation to extend it while another process has a lock on that relation for the same purpose.
**Topics**
+ [
## Supported engine versions
](#wait-event.lockextend.context.supported)
+ [
## Context
](#wait-event.lockextend.context)
+ [
## Likely causes of increased waits
](#wait-event.lockextend.causes)
+ [
## Actions
](#wait-event.lockextend.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
The event `Lock:extend` indicates that a backend process is waiting to extend a relation that another backend process holds a lock on while it's extending that relation. Because only one process at a time can extend a relation, the system generates a `Lock:extend` wait event. `INSERT`, `COPY`, and `UPDATE` operations can generate this event.
## Likely causes of increased waits
When the `Lock:extend` event appears more than normal, possibly indicating a performance problem, typical causes include the following:
**Surge in concurrent inserts or updates to the same table **
There might be an increase in the number of concurrent sessions with queries that insert into or update the same table.
**Insufficient network bandwidth**
The network bandwidth on the DB instance might be insufficient for the storage communication needs of the current workload. This can contribute to storage latency that causes an increase in `Lock:extend` events.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Reduce concurrent inserts and updates to the same relation
](#wait-event.lockextend.actions.action1)
+ [
### Increase network bandwidth
](#wait-event.lockextend.actions.increase-network-bandwidth)
### Reduce concurrent inserts and updates to the same relation
First, determine whether there's an increase in `tup_inserted` and `tup_updated` metrics and an accompanying increase in this wait event. If so, check which relations are in high contention for insert and update operations. To determine this, query the `pg_stat_all_tables` view for the values in `n_tup_ins` and `n_tup_upd` fields. For information about the `pg_stat_all_tables` view, see [pg\$1stat\$1all\$1tables](https://www.postgresql.org/docs/13/monitoring-stats.html#MONITORING-PG-STAT-ALL-TABLES-VIEW) in the PostgreSQL documentation.
To get more information about blocking and blocked queries, query `pg_stat_activity` as in the following example:
```
SELECT
blocked.pid,
blocked.usename,
blocked.query,
blocking.pid AS blocking_id,
blocking.query AS blocking_query,
blocking.wait_event AS blocking_wait_event,
blocking.wait_event_type AS blocking_wait_event_type
FROM pg_stat_activity AS blocked
JOIN pg_stat_activity AS blocking ON blocking.pid = ANY(pg_blocking_pids(blocked.pid))
where
blocked.wait_event = 'extend'
and blocked.wait_event_type = 'Lock';
pid | usename | query | blocking_id | blocking_query | blocking_wait_event | blocking_wait_event_type
------+----------+------------------------------+-------------+------------------------------------------------------------------+---------------------+--------------------------
7143 | myuser | insert into tab1 values (1); | 4600 | INSERT INTO tab1 (a) SELECT s FROM generate_series(1,1000000) s; | DataFileExtend | IO
```
After you identify relations that contribute to increase `Lock:extend` events, use the following techniques to reduce the contention:
+ Find out whether you can use partitioning to reduce contention for the same table. Separating inserted or updated tuples into different partitions can reduce contention. For information about partitioning, see [Managing PostgreSQL partitions with the pg\$1partman extension](PostgreSQL_Partitions.md).
+ If the wait event is mainly due to update activity, consider reducing the relation's fillfactor value. This can reduce requests for new blocks during the update. The fillfactor is a storage parameter for a table that determines the maximum amount of space for packing a table page. It's expressed as a percentage of the total space for a page. For more information about the fillfactor parameter, see [CREATE TABLE](https://www.postgresql.org/docs/13/sql-createtable.html) in the PostgreSQL documentation.
**Important**
We highly recommend that you test your system if you change the fillfactor because changing this value can negatively impact performance, depending on your workload.
### Increase network bandwidth
To see whether there's an increase in write latency, check the `WriteLatency` metric in CloudWatch. If there is, use the `WriteThroughput` and `ReadThroughput` Amazon CloudWatch metrics to monitor the storage related traffic on the DB instance. These metrics can help you to determine if network bandwidth is sufficient for the storage activity of your workload.
If your network bandwidth isn't enough, increase it. If your DB instance is reaching the network bandwidth limits, the only way to increase the bandwidth is to increase your DB instance size.
For more information about CloudWatch metrics, see [Amazon CloudWatch instance-level metrics for Amazon RDS](rds-metrics.md#rds-cw-metrics-instance). For information about network performance for each DB instance class, see [Amazon CloudWatch instance-level metrics for Amazon RDS](rds-metrics.md#rds-cw-metrics-instance).
# Lock:Relation
The `Lock:Relation` event occurs when a query is waiting to acquire a lock on a table or view (relation) that's currently locked by another transaction.
**Topics**
+ [
## Supported engine versions
](#wait-event.lockrelation.context.supported)
+ [
## Context
](#wait-event.lockrelation.context)
+ [
## Likely causes of increased waits
](#wait-event.lockrelation.causes)
+ [
## Actions
](#wait-event.lockrelation.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
Most PostgreSQL commands implicitly use locks to control concurrent access to data in tables. You can also use these locks explicitly in your application code with the `LOCK` command. Many lock modes aren't compatible with each other, and they can block transactions when they're trying to access the same object. When this happens, RDS for PostgreSQL generates a `Lock:Relation` event. Some common examples are the following:
+ Exclusive locks such as `ACCESS EXCLUSIVE` can block all concurrent access. Data definition language (DDL) operations such as `DROP TABLE`, `TRUNCATE`, `VACUUM FULL`, and `CLUSTER` acquire `ACCESS EXCLUSIVE` locks implicitly. `ACCESS EXCLUSIVE` is also the default lock mode for `LOCK TABLE` statements that don't specify a mode explicitly.
+ Using `CREATE INDEX (without CONCURRENT)` on a table conflicts with data manipulation language (DML) statements `UPDATE`, `DELETE`, and `INSERT`, which acquire `ROW EXCLUSIVE` locks.
For more information about table-level locks and conflicting lock modes, see [Explicit Locking](https://www.postgresql.org/docs/13/explicit-locking.html) in the PostgreSQL documentation.
Blocking queries and transactions typically unblock in one of the following ways:
+ Blocking query – The application can cancel the query or the user can end the process. The engine can also force the query to end because of a session's statement-timeout or a deadlock detection mechanism.
+ Blocking transaction – A transaction stops blocking when it runs a `ROLLBACK` or `COMMIT` statement. Rollbacks also happen automatically when sessions are disconnected by a client or by network issues, or are ended. Sessions can be ended when the database engine is shut down, when the system is out of memory, and so forth.
## Likely causes of increased waits
When the `Lock:Relation` event occurs more frequently than normal, it can indicate a performance issue. Typical causes include the following:
**Increased concurrent sessions with conflicting table locks**
There might be an increase in the number of concurrent sessions with queries that lock the same table with conflicting locking modes.
**Maintenance operations**
Health maintenance operations such as `VACUUM` and `ANALYZE` can significantly increase the number of conflicting locks. `VACUUM FULL` acquires an `ACCESS EXCLUSIVE` lock, and `ANALYSE` acquires a `SHARE UPDATE EXCLUSIVE` lock. Both types of locks can cause a `Lock:Relation` wait event. Application data maintenance operations such as refreshing a materialized view can also increase blocked queries and transactions.
**Locks on reader instances**
There might be a conflict between the relation locks held by the writer and readers. Currently, only `ACCESS EXCLUSIVE` relation locks are replicated to reader instances. However, the `ACCESS EXCLUSIVE` relation lock will conflict with any `ACCESS SHARE` relation locks held by the reader. This can cause an increase in lock relation wait events on the reader.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Reduce the impact of blocking SQL statements
](#wait-event.lockrelation.actions.reduce-blocks)
+ [
### Minimize the effect of maintenance operations
](#wait-event.lockrelation.actions.maintenance)
### Reduce the impact of blocking SQL statements
To reduce the impact of blocking SQL statements, modify your application code where possible. Following are two common techniques for reducing blocks:
+ Use the `NOWAIT` option – Some SQL commands, such as `SELECT` and `LOCK` statements, support this option. The `NOWAIT` directive cancels the lock-requesting query if the lock can't be acquired immediately. This technique can help prevent a blocking session from causing a pile-up of blocked sessions behind it.
For example: Assume that transaction A is waiting on a lock held by transaction B. Now, if B requests a lock on a table that’s locked by transaction C, transaction A might be blocked until transaction C completes. But if transaction B uses a `NOWAIT` when it requests the lock on C, it can fail fast and ensure that transaction A doesn't have to wait indefinitely.
+ Use `SET lock_timeout` – Set a `lock_timeout` value to limit the time a SQL statement waits to acquire a lock on a relation. If the lock isn't acquired within the timeout specified, the transaction requesting the lock is cancelled. Set this value at the session level.
### Minimize the effect of maintenance operations
Maintenance operations such as `VACUUM` and `ANALYZE` are important. We recommend that you don't turn them off because you find `Lock:Relation` wait events related to these maintenance operations. The following approaches can minimize the effect of these operations:
+ Run maintenance operations manually during off-peak hours.
+ To reduce `Lock:Relation` waits caused by autovacuum tasks, perform any needed autovacuum tuning. For information about tuning autovacuum, see [ Working with PostgreSQL autovacuum on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Autovacuum.html) in the * Amazon RDS User Guide*.
# Lock:transactionid
The `Lock:transactionid` event occurs when a transaction is waiting for a row-level lock.
**Topics**
+ [
## Supported engine versions
](#wait-event.locktransactionid.context.supported)
+ [
## Context
](#wait-event.locktransactionid.context)
+ [
## Likely causes of increased waits
](#wait-event.locktransactionid.causes)
+ [
## Actions
](#wait-event.locktransactionid.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
The event `Lock:transactionid` occurs when a transaction is trying to acquire a row-level lock that has already been granted to a transaction that is running at the same time. The session that shows the `Lock:transactionid` wait event is blocked because of this lock. After the blocking transaction ends in either a `COMMIT` or `ROLLBACK` statement, the blocked transaction can proceed.
The multiversion concurrency control semantics of RDS for PostgreSQL guarantee that readers don't block writers and writers don't block readers. For row-level conflicts to occur, blocking and blocked transactions must issue conflicting statements of the following types:
+ `UPDATE`
+ `SELECT … FOR UPDATE`
+ `SELECT … FOR KEY SHARE`
The statement `SELECT … FOR KEY SHARE` is a special case. The database uses the clause `FOR KEY SHARE` to optimize the performance of referential integrity. A row-level lock on a row can block `INSERT`, `UPDATE`, and `DELETE` commands on other tables that reference the row.
## Likely causes of increased waits
When this event appears more than normal, the cause is typically `UPDATE`, `SELECT … FOR UPDATE`, or `SELECT … FOR KEY SHARE` statements combined with the following conditions.
**Topics**
+ [
### High concurrency
](#wait-event.locktransactionid.concurrency)
+ [
### Idle in transaction
](#wait-event.locktransactionid.idle)
+ [
### Long-running transactions
](#wait-event.locktransactionid.long-running)
### High concurrency
RDS for PostgreSQL can use granular row-level locking semantics. The probability of row-level conflicts increases when the following conditions are met:
+ A highly concurrent workload contends for the same rows.
+ Concurrency increases.
### Idle in transaction
Sometimes the `pg_stat_activity.state` column shows the value `idle in transaction`. This value appears for sessions that have started a transaction, but haven't yet issued a `COMMIT` or `ROLLBACK`. If the `pg_stat_activity.state` value isn't `active`, the query shown in `pg_stat_activity` is the most recent one to finish running. The blocking session isn't actively processing a query because an open transaction is holding a lock.
If an idle transaction acquired a row-level lock, it might be preventing other sessions from acquiring it. This condition leads to frequent occurrence of the wait event `Lock:transactionid`. To diagnose the issue, examine the output from `pg_stat_activity` and `pg_locks`.
### Long-running transactions
Transactions that run for a long time get locks for a long time. These long-held locks can block other transactions from running.
## Actions
Row-locking is a conflict among `UPDATE`, `SELECT … FOR UPDATE`, or `SELECT … FOR KEY SHARE` statements. Before attempting a solution, find out when these statements are running on the same row. Use this information to choose a strategy described in the following sections.
**Topics**
+ [
### Respond to high concurrency
](#wait-event.locktransactionid.actions.problem)
+ [
### Respond to idle transactions
](#wait-event.locktransactionid.actions.find-blocker)
+ [
### Respond to long-running transactions
](#wait-event.locktransactionid.actions.concurrency)
### Respond to high concurrency
If concurrency is the issue, try one of the following techniques:
+ Lower the concurrency in the application. For example, decrease the number of active sessions.
+ Implement a connection pool. To learn how to pool connections with RDS Proxy, see [Amazon RDS Proxy](rds-proxy.md).
+ Design the application or data model to avoid contending `UPDATE` and `SELECT … FOR UPDATE` statements. You can also decrease the number of foreign keys accessed by `SELECT … FOR KEY SHARE` statements.
### Respond to idle transactions
If `pg_stat_activity.state` shows `idle in transaction`, use the following strategies:
+ Turn on autocommit wherever possible. This approach prevents transactions from blocking other transactions while waiting for a `COMMIT` or `ROLLBACK`.
+ Search for code paths that are missing `COMMIT`, `ROLLBACK`, or `END`.
+ Make sure that the exception handling logic in your application always has a path to a valid `end of transaction`.
+ Make sure that your application processes query results after ending the transaction with `COMMIT` or `ROLLBACK`.
### Respond to long-running transactions
If long-running transactions are causing the frequent occurrence of `Lock:transactionid`, try the following strategies:
+ Keep row locks out of long-running transactions.
+ Limit the length of queries by implementing autocommit whenever possible.
# Lock:tuple
The `Lock:tuple` event occurs when a backend process is waiting to acquire a lock on a tuple.
**Topics**
+ [
## Supported engine versions
](#wait-event.locktuple.context.supported)
+ [
## Context
](#wait-event.locktuple.context)
+ [
## Likely causes of increased waits
](#wait-event.locktuple.causes)
+ [
## Actions
](#wait-event.locktuple.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
The event `Lock:tuple` indicates that a backend is waiting to acquire a lock on a tuple while another backend holds a conflicting lock on the same tuple. The following table illustrates a scenario in which sessions generate the `Lock:tuple` event.
| Time | Session 1 | Session 2 | Session 3 |
| --- | --- | --- | --- |
| t1 | Starts a transaction. | | |
| t2 | Updates row 1. | | |
| t3 | | Updates row 1. The session acquires an exclusive lock on the tuple and then waits for session 1 to release the lock by committing or rolling back. | |
| t4 | | | Updates row 1. The session waits for session 2 to release the exclusive lock on the tuple. |
Or you can simulate this wait event by using the benchmarking tool `pgbench`. Configure a high number of concurrent sessions to update the same row in a table with a custom SQL file.
To learn more about conflicting lock modes, see [Explicit Locking](https://www.postgresql.org/docs/current/explicit-locking.html) in the PostgreSQL documentation. To learn more about `pgbench`, see [pgbench](https://www.postgresql.org/docs/current/pgbench.html) in the PostgreSQL documentation.
## Likely causes of increased waits
When this event appears more than normal, possibly indicating a performance problem, typical causes include the following:
+ A high number of concurrent sessions are trying to acquire a conflicting lock for the same tuple by running `UPDATE` or `DELETE` statements.
+ Highly concurrent sessions are running a `SELECT` statement using the `FOR UPDATE` or `FOR NO KEY UPDATE` lock modes.
+ Various factors drive application or connection pools to open more sessions to execute the same operations. As new sessions are trying to modify the same rows, DB load can spike, and `Lock:tuple` can appear.
For more information, see [Row-Level Locks](https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-ROWS) in the PostgreSQL documentation.
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Investigate your application logic
](#wait-event.locktuple.actions.problem)
+ [
### Find the blocker session
](#wait-event.locktuple.actions.find-blocker)
+ [
### Reduce concurrency when it is high
](#wait-event.locktuple.actions.concurrency)
+ [
### Troubleshoot bottlenecks
](#wait-event.locktuple.actions.bottlenecks)
### Investigate your application logic
Find out whether a blocker session has been in the `idle in transaction` state for long time. If so, consider ending the blocker session as a short-term solution. You can use the `pg_terminate_backend` function. For more information about this function, see [Server Signaling Functions](https://www.postgresql.org/docs/13/functions-admin.html#FUNCTIONS-ADMIN-SIGNAL) in the PostgreSQL documentation.
For a long-term solution, do the following:
+ Adjust the application logic.
+ Use the `idle_in_transaction_session_timeout` parameter. This parameter ends any session with an open transaction that has been idle for longer than the specified amount of time. For more information, see [Client Connection Defaults](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-IDLE-IN-TRANSACTION-SESSION-TIMEOUT) in the PostgreSQL documentation.
+ Use autocommit as much as possible. For more information, see [SET AUTOCOMMIT](https://www.postgresql.org/docs/current/ecpg-sql-set-autocommit.html) in the PostgreSQL documentation.
### Find the blocker session
While the `Lock:tuple` wait event is occurring, identify the blocker and blocked session by finding out which locks depend on one another. For more information, see [Lock dependency information](https://wiki.postgresql.org/wiki/Lock_dependency_information) in the PostgreSQL wiki.
The following example queries all sessions, filtering on `tuple` and ordering by `wait_time`.
```
SELECT blocked_locks.pid AS blocked_pid,
blocking_locks.pid AS blocking_pid,
blocked_activity.usename AS blocked_user,
blocking_activity.usename AS blocking_user,
now() - blocked_activity.xact_start AS blocked_transaction_duration,
now() - blocking_activity.xact_start AS blocking_transaction_duration,
concat(blocked_activity.wait_event_type,':',blocked_activity.wait_event) AS blocked_wait_event,
concat(blocking_activity.wait_event_type,':',blocking_activity.wait_event) AS blocking_wait_event,
blocked_activity.state AS blocked_state,
blocking_activity.state AS blocking_state,
blocked_locks.locktype AS blocked_locktype,
blocking_locks.locktype AS blocking_locktype,
blocked_activity.query AS blocked_statement,
blocking_activity.query AS blocking_statement
FROM pg_catalog.pg_locks blocked_locks
JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid
JOIN pg_catalog.pg_locks blocking_locks
ON blocking_locks.locktype = blocked_locks.locktype
AND blocking_locks.DATABASE IS NOT DISTINCT FROM blocked_locks.DATABASE
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
AND blocking_locks.pid != blocked_locks.pid
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
WHERE NOT blocked_locks.GRANTED;
```
### Reduce concurrency when it is high
The `Lock:tuple` event might occur constantly, especially in a busy workload time. In this situation, consider reducing the high concurrency for very busy rows. Often, just a few rows control a queue or the Boolean logic, which makes these rows very busy.
You can reduce concurrency by using different approaches based in the business requirement, application logic, and workload type. For example, you can do the following:
+ Redesign your table and data logic to reduce high concurrency.
+ Change the application logic to reduce high concurrency at the row level.
+ Leverage and redesign queries with row-level locks.
+ Use the `NOWAIT` clause with retry operations.
+ Consider using optimistic and hybrid-locking logic concurrency control.
+ Consider changing the database isolation level.
### Troubleshoot bottlenecks
The `Lock:tuple` can occur with bottlenecks such as CPU starvation or maximum usage of Amazon EBS bandwidth. To reduce bottlenecks, consider the following approaches:
+ Scale up your instance class type.
+ Optimize resource-intensive queries.
+ Change the application logic.
+ Archive data that is rarely accessed.
# LWLock:BufferMapping (LWLock:buffer\$1mapping)
This event occurs when a session is waiting to associate a data block with a buffer in the shared buffer pool.
**Note**
This event is named `LWLock:BufferMapping` for RDS for PostgreSQL version 13 and higher versions. For RDS for PostgreSQL version 12 and older versions, this event is named `LWLock:buffer_mapping`.
**Topics**
+ [
## Supported engine versions
](#wait-event.lwl-buffer-mapping.context.supported)
+ [
## Context
](#wait-event.lwl-buffer-mapping.context)
+ [
## Causes
](#wait-event.lwl-buffer-mapping.causes)
+ [
## Actions
](#wait-event.lwl-buffer-mapping.actions)
## Supported engine versions
This wait event information is relevant for RDS for PostgreSQL version 9.6 and higher.
## Context
The *shared buffer pool* is a PostgreSQL memory area that holds all pages that are or were being used by processes. When a process needs a page, it reads the page into the shared buffer pool. The `shared_buffers` parameter sets the shared buffer size and reserves a memory area to store the table and index pages. If you change this parameter, make sure to restart the database. .
The `LWLock:buffer_mapping` wait event occurs in the following scenarios:
+ A process searches the buffer table for a page and acquires a shared buffer mapping lock.
+ A process loads a page into the buffer pool and acquires an exclusive buffer mapping lock.
+ A process removes a page from the pool and acquires an exclusive buffer mapping lock.
## Causes
When this event appears more than normal, possibly indicating a performance problem, the database is paging in and out of the shared buffer pool. Typical causes include the following:
+ Large queries
+ Bloated indexes and tables
+ Full table scans
+ A shared pool size that is smaller than the working set
## Actions
We recommend different actions depending on the causes of your wait event.
**Topics**
+ [
### Monitor buffer-related metrics
](#wait-event.lwl-buffer-mapping.actions.monitor-metrics)
+ [
### Assess your indexing strategy
](#wait-event.lwl-buffer-mapping.actions.indexes)
+ [
### Reduce the number of buffers that must be allocated quickly
](#wait-event.lwl-buffer-mapping.actions.buffers)
### Monitor buffer-related metrics
When `LWLock:buffer_mapping` waits spike, investigate the buffer hit ratio. You can use these metrics to get a better understanding of what is happening in the buffer cache. Examine the following metrics:
`blks_hit`
This Performance Insights counter metric indicates the number of blocks that were retrieved from the shared buffer pool. After the `LWLock:buffer_mapping` wait event appears, you might observe a spike in `blks_hit`.
`blks_read`
This Performance Insights counter metric indicates the number of blocks that required I/O to be read into the shared buffer pool. You might observe a spike in `blks_read` in the lead-up to the `LWLock:buffer_mapping` wait event.
### Assess your indexing strategy
To confirm that your indexing strategy is not degrading performance, check the following:
Index bloat
Ensure that index and table bloat aren't leading to unnecessary pages being read into the shared buffer. If your tables contain unused rows, consider archiving the data and removing the rows from the tables. You can then rebuild the indexes for the resized tables.
Indexes for frequently used queries
To determine whether you have the optimal indexes, monitor DB engine metrics in Performance Insights. The `tup_returned` metric shows the number of rows read. The `tup_fetched` metric shows the number of rows returned to the client. If `tup_returned` is significantly larger than `tup_fetched`, the data might not be properly indexed. Also, your table statistics might not be current.
### Reduce the number of buffers that must be allocated quickly
To reduce the `LWLock:buffer_mapping` wait events, try to reduce the number of buffers that must be allocated quickly. One strategy is to perform smaller batch operations. You might be able to achieve smaller batches by partitioning your tables.
# LWLock:BufferIO (IPC:BufferIO)
The `LWLock:BufferIO` event occurs when RDS for PostgreSQL is waiting for other processes to finish their input/output (I/O) operations when concurrently trying to access a page. Its purpose is for the same page to be read into the shared buffer.
**Topics**
+ [
## Relevant engine versions
](#wait-event.lwlockbufferio.context.supported)
+ [
## Context
](#wait-event.lwlockbufferio.context)
+ [
## Causes
](#wait-event.lwlockbufferio.causes)
+ [
## Actions
](#wait-event.lwlockbufferio.actions)
## Relevant engine versions
This wait event information is relevant for all RDS for PostgreSQL versions. For RDS for PostgreSQL 12 and earlier versions this wait event is named as lwlock:buffer\$1io whereas in RDS for PostgreSQL 13 version it is named as lwlock:bufferio. From RDS for PostgreSQL 14 version BufferIO wait event moved from `LWLock` to `IPC` wait event type (IPC:BufferIO).
## Context
Each shared buffer has an I/O lock that is associated with the `LWLock:BufferIO` wait event, each time a block (or a page) has to be retrieved outside the shared buffer pool.
This lock is used to handle multiple sessions that all require access to the same block. This block has to be read from outside the shared buffer pool, defined by the `shared_buffers` parameter.
As soon as the page is read inside the shared buffer pool, the `LWLock:BufferIO` lock is released.
**Note**
The `LWLock:BufferIO` wait event precedes the [IO:DataFileRead](wait-event.iodatafileread.md) wait event. The `IO:DataFileRead` wait event occurs while data is being read from storage.
For more information on lightweight locks, see [Locking Overview](https://github.com/postgres/postgres/blob/65dc30ced64cd17f3800ff1b73ab1d358e92efd8/src/backend/storage/lmgr/README#L20).
## Causes
Common causes for the `LWLock:BufferIO` event to appear in top waits include the following:
+ Multiple backends or connections trying to access the same page that's also pending an I/O operation
+ The ratio between the size of the shared buffer pool (defined by the `shared_buffers` parameter) and the number of buffers needed by the current workload
+ The size of the shared buffer pool not being well balanced with the number of pages being evicted by other operations
+ Large or bloated indexes that require the engine to read more pages than necessary into the shared buffer pool
+ Lack of indexes that forces the DB engine to read more pages from the tables than necessary
+ Checkpoints occurring too frequently or needing to flush too many modified pages
+ Sudden spikes for database connections trying to perform operations on the same page
## Actions
We recommend different actions depending on the causes of your wait event:
+ Tune `max_wal_size` and `checkpoint_timeout` based on your workload peak time if you see `LWLock:BufferIO` coinciding with `BufferCacheHitRatio` metric dips. Then identify which query might be causing it.
+ Verify whether you have unused indexes, then remove them.
+ Use partitioned tables (which also have partitioned indexes). Doing this helps to keep index reordering low and reduces its impact.
+ Avoid indexing columns unnecessarily.
+ Prevent sudden database connection spikes by using a connection pool.
+ Restrict the maximum number of connections to the database as a best practice.
# LWLock:buffer\$1content (BufferContent)
The `LWLock:buffer_content` event occurs when a session is waiting to read or write a data page in memory while another session has that page locked for writing. In RDS for PostgreSQL 13 and higher, this wait event is called `BufferContent`.
**Topics**
+ [
## Supported engine versions
](#wait-event.lwlockbuffercontent.context.supported)
+ [
## Context
](#wait-event.lwlockbuffercontent.context)
+ [
## Likely causes of increased waits
](#wait-event.lwlockbuffercontent.causes)
+ [
## Actions
](#wait-event.lwlockbuffercontent.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
To read or manipulate data, PostgreSQL accesses it through shared memory buffers. To read from the buffer, a process gets a lightweight lock (LWLock) on the buffer content in shared mode. To write to the buffer, it gets that lock in exclusive mode. Shared locks allow other processes to concurrently acquire shared locks on that content. Exclusive locks prevent other processes from getting any type of lock on it.
The `LWLock:buffer_content` (`BufferContent`) event indicates that multiple processes are attempting to get a lock on contents of a specific buffer.
## Likely causes of increased waits
When the `LWLock:buffer_content` (`BufferContent`) event appears more than normal, possibly indicating a performance problem, typical causes include the following:
**Increased concurrent updates to the same data**
There might be an increase in the number of concurrent sessions with queries that update the same buffer content. This contention can be more pronounced on tables with a lot of indexes.
**Workload data is not in memory**
When data that the active workload is processing is not in memory, these wait events can increase. This effect is because processes holding locks can keep them longer while they perform disk I/O operations.
**Excessive use of foreign key constraints**
Foreign key constraints can increase the amount of time a process holds onto a buffer content lock. This effect is because read operations require a shared buffer content lock on the referenced key while that key is being updated.
## Actions
We recommend different actions depending on the causes of your wait event. You might identify `LWLock:buffer_content` (`BufferContent`) events by using Amazon RDS Performance Insights or by querying the view `pg_stat_activity`.
**Topics**
+ [
### Improve in-memory efficiency
](#wait-event.lwlockbuffercontent.actions.in-memory)
+ [
### Reduce usage of foreign key constraints
](#wait-event.lwlockbuffercontent.actions.foreignkey)
+ [
### Remove unused indexes
](#wait-event.lwlockbuffercontent.actions.indexes)
+ [
### Increase the cache size when using sequences
](#wait-event.lwlockbuffercontent.actions.sequences)
### Improve in-memory efficiency
To increase the chance that active workload data is in memory, partition tables or scale up your instance class. For information about DB instance classes, see [DB instance classes](Concepts.DBInstanceClass.md).
### Reduce usage of foreign key constraints
Investigate workloads experiencing high numbers of `LWLock:buffer_content` (`BufferContent`) wait events for usage of foreign key constraints. Remove unnecessary foreign key constraints.
### Remove unused indexes
For workloads experiencing high numbers of `LWLock:buffer_content` (`BufferContent`) wait events, identify unused indexes and remove them.
### Increase the cache size when using sequences
If your tables uses sequences, increase the cache size to remove contention on sequence pages and index pages. Each sequence is a single page in shared memory. The pre-defined cache is per connection. This might not be enough to handle the workload when many concurrent sessions are getting a sequence value.
# LWLock:lock\$1manager (LWLock:lockmanager)
This event occurs when the RDS for PostgreSQL engine maintains the shared lock's memory area to allocate, check, and deallocate a lock when a fast path lock isn't possible.
**Topics**
+ [
## Supported engine versions
](#wait-event.lw-lock-manager.context.supported)
+ [
## Context
](#wait-event.lw-lock-manager.context)
+ [
## Likely causes of increased waits
](#wait-event.lw-lock-manager.causes)
+ [
## Actions
](#wait-event.lw-lock-manager.actions)
## Supported engine versions
This wait event information is relevant for RDS for PostgreSQL version 9.6 and higher. For RDS for PostgreSQL releases older than version 13, the name of this wait event is `LWLock:lock_manager`. For RDS for PostgreSQL version 13 and higher, the name of this wait event is `LWLock:lockmanager`.
## Context
When you issue a SQL statement, RDS for PostgreSQL records locks to protect the structure, data, and integrity of your database during concurrent operations. The engine can achieve this goal using a fast path lock or a path lock that isn't fast. A path lock that isn't fast is more expensive and creates more overhead than a fast path lock.
### Fast path locking
To reduce the overhead of locks that are taken and released frequently, but that rarely conflict, backend processes can use fast path locking. The database uses this mechanism for locks that meet the following criteria:
+ They use the DEFAULT lock method.
+ They represent a lock on a database relation rather than a shared relation.
+ They are weak locks that are unlikely to conflict.
+ The engine can quickly verify that no conflicting locks can possibly exist.
The engine can't use fast path locking when either of the following conditions is true:
+ The lock doesn't meet the preceding criteria.
+ No more slots are available for the backend process.
To tune your queries for fast-path locking, you can use the following query.
```
SELECT count(*), pid, mode, fastpath
FROM pg_locks
WHERE fastpath IS NOT NULL
GROUP BY 4,3,2
ORDER BY pid, mode;
count | pid | mode | fastpath
-------+------+-----------------+----------
16 | 9185 | AccessShareLock | t
336 | 9185 | AccessShareLock | f
1 | 9185 | ExclusiveLock | t
```
The following query shows only the total across the database.
```
SELECT count(*), mode, fastpath
FROM pg_locks
WHERE fastpath IS NOT NULL
GROUP BY 3,2
ORDER BY mode,1;
count | mode | fastpath
-------+-----------------+----------
16 | AccessShareLock | t
337 | AccessShareLock | f
1 | ExclusiveLock | t
(3 rows)
```
For more information about fast path locking, see [fast path](https://github.com/postgres/postgres/blob/master/src/backend/storage/lmgr/README#L70-L76) in the PostgreSQL lock manager README and [pg-locks](https://www.postgresql.org/docs/9.3/view-pg-locks.html#AEN98195) in the PostgreSQL documentation.
### Example of a scaling problem for the lock manager
In this example, a table named `purchases` stores five years of data, partitioned by day. Each partition has two indexes. The following sequence of events occurs:
1. You query many days worth of data, which requires the database to read many partitions.
1. The database creates a lock entry for each partition. If partition indexes are part of the optimizer access path, the database creates a lock entry for them, too.
1. When the number of requested locks entries for the same backend process is higher than 16, which is the value of `FP_LOCK_SLOTS_PER_BACKEND`, the lock manager uses the non–fast path lock method.
Modern applications might have hundreds of sessions. If concurrent sessions are querying the parent without proper partition pruning, the database might create hundreds or even thousands of non–fast path locks. Typically, when this concurrency is higher than the number of vCPUs, the `LWLock:lock_manager` wait event appears.
**Note**
The `LWLock:lock_manager` wait event isn't related to the number of partitions or indexes in a database schema. Instead, it's related to the number of non–fast path locks that the database must control.
## Likely causes of increased waits
When the `LWLock:lock_manager` wait event occurs more than normal, possibly indicating a performance problem, the most likely causes of sudden spikes are as follows:
+ Concurrent active sessions are running queries that don't use fast path locks. These sessions also exceed the maximum vCPU.
+ A large number of concurrent active sessions are accessing a heavily partitioned table. Each partition has multiple indexes.
+ The database is experiencing a connection storm. By default, some applications and connection pool software create more connections when the database is slow. This practice makes the problem worse. Tune your connection pool software so that connection storms don't occur.
+ A large number of sessions query a parent table without pruning partitions.
+ A data definition language (DDL), data manipulation language (DML), or a maintenance command exclusively locks either a busy relation or tuples that are frequently accessed or modified.
## Actions
If the `CPU` wait event occurs, it doesn't necessarily indicate a performance problem. Respond to this event only when performance degrades and this wait event is dominating DB load.
**Topics**
+ [
### Use partition pruning
](#wait-event.lw-lock-manager.actions.pruning)
+ [
### Remove unnecessary indexes
](#wait-event.lw-lock-manager.actions.indexes)
+ [
### Tune your queries for fast path locking
](#wait-event.lw-lock-manager.actions.tuning)
+ [
### Tune for other wait events
](#wait-event.lw-lock-manager.actions.other-waits)
+ [
### Reduce hardware bottlenecks
](#wait-event.lw-lock-manager.actions.hw-bottlenecks)
+ [
### Use a connection pooler
](#wait-event.lw-lock-manager.actions.pooler)
+ [
### Upgrade your RDS for PostgreSQL version
](#wait-event.lw-lock-manager.actions.pg-version)
### Use partition pruning
*Partition pruning* is a query optimization strategy for declaratively partitioned tables that excludes unneeded partitions from table scans, thereby improving performance. Partition pruning is turned on by default. If it is turned off, turn it on as follows.
```
SET enable_partition_pruning = on;
```
Queries can take advantage of partition pruning when their `WHERE` clause contains the column used for the partitioning. For more information, see [Partition Pruning](https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITION-PRUNING) in the PostgreSQL documentation.
### Remove unnecessary indexes
Your database might contain unused or rarely used indexes. If so, consider deleting them. Do either of the following:
+ Learn how to find unnecessary indexes by reading [Unused Indexes](https://wiki.postgresql.org/wiki/Index_Maintenance#Unused_Indexes) in the PostgreSQL wiki.
+ Run PG Collector. This SQL script gathers database information and presents it in a consolidated HTML report. Check the "Unused indexes" section. For more information, see [pg-collector](https://github.com/awslabs/pg-collector) in the AWS Labs GitHub repository.
### Tune your queries for fast path locking
To find out whether your queries use fast path locking, query the `fastpath` column in the `pg_locks` table. If your queries aren't using fast path locking, try to reduce number of relations per query to fewer than 16.
### Tune for other wait events
If `LWLock:lock_manager` is first or second in the list of top waits, check whether the following wait events also appear in the list:
+ `Lock:Relation`
+ `Lock:transactionid`
+ `Lock:tuple`
If the preceding events appear high in the list, consider tuning these wait events first. These events can be a driver for `LWLock:lock_manager`.
### Reduce hardware bottlenecks
You might have a hardware bottleneck, such as CPU starvation or maximum usage of your Amazon EBS bandwidth. In these cases, consider reducing the hardware bottlenecks. Consider the following actions:
+ Scale up your instance class.
+ Optimize queries that consume large amounts of CPU and memory.
+ Change your application logic.
+ Archive your data.
For more information about CPU, memory, and EBS network bandwidth, see [Amazon RDS Instance Types](https://aws.amazon.com/rds/instance-types/).
### Use a connection pooler
If your total number of active connections exceeds the maximum vCPU, more OS processes require CPU than your instance type can support. In this case, consider using or tuning a connection pool. For more information about the vCPUs for your instance type, see [Amazon RDS Instance Types](https://aws.amazon.com/rds/instance-types/).
For more information about connection pooling, see the following resources:
+ [Amazon RDS Proxy](rds-proxy.md)
+ [pgbouncer](http://www.pgbouncer.org/usage.html)
+ [Connection Pools and Data Sources](https://www.postgresql.org/docs/7.4/jdbc-datasource.html) in the *PostgreSQL Documentation*
### Upgrade your RDS for PostgreSQL version
If your current version of RDS for PostgreSQL is lower than 12, upgrade to version 12 or higher. PostgreSQL versions 12 and later have an improved partition mechanism. For more information about version 12, see [PostgreSQL 12.0 Release Notes]( https://www.postgresql.org/docs/release/12.0/). For more information about upgrading RDS for PostgreSQL, see [Upgrades of the RDS for PostgreSQL DB engine](USER_UpgradeDBInstance.PostgreSQL.md).
# LWLock:pg\$1stat\$1statements
The LWLock:pg\$1stat\$1statements wait event occurs when the `pg_stat_statements` extension takes an exclusive lock on the hash table that tracks SQL statements. This happens in the following scenarios:
+ When the number of tracked statements reaches the configured `pg_stat_statements.max` parameter value and there is a need to make room for more entries, the extension performs a sort on the number of calls, removes the 5% of the least-executed statements, and re-populates the hash with the remaining entries.
+ When `pg_stat_statements` performs a `garbage collection` operation to the `pgss_query_texts.stat` file on disk and rewrites the file.
**Topics**
+ [
## Supported engine versions
](#apg-rpg-lwlockpgstat.supported)
+ [
## Context
](#apg-rpg-lwlockpgstat.context)
+ [
## Likely causes of increased waits
](#apg-rpg-lwlockpgstat.causes)
+ [
## Actions
](#apg-rpg-lwlockpgstat.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
**Understanding the pg\$1stat\$1statements extension** – The pg\$1stat\$1statements extension tracks SQL statement execution statistics in a hash table. The extension tracks SQL statements up to the limit defined by the `pg_stat_statements.max` parameter. This parameter determines the maximum number of statements that can be tracked which corresponds to the maximum number of rows in the pg\$1stat\$1statements view.
**Statement statistics persistence** – The extension persists statement statistics across instance restarts by:
+ Writing data to a file named pg\$1stat\$1statements.stat
+ Using the pg\$1stat\$1statements.save parameter to control persistence behavior
When pg\$1stat\$1statements.save is set to:
+ on (default): Statistics are saved at shutdown and reloaded at server start
+ off: Statistics are neither saved at shutdown nor reloaded at server start
**Query text storage** – The extension stores the text of tracked queries in a file named `pgss_query_texts.stat`. This file can grow to double the average size of all tracked SQL statements before garbage collection occurs. The extension requires an exclusive lock on the hash table during cleanup operations and rewrite `pgss_query_texts.stat` file.
**Statement deallocation process** – When the number of tracked statements reaches the `pg_stat_statements.max` limit and new statements need to be tracked, the extension:
+ Takes an exclusive lock (LWLock:pg\$1stat\$1statements) on the hash table.
+ Loads existing data into local memory.
+ Performs a quicksort based on the number of calls.
+ Removes the least-called statements (bottom 5%).
+ Re-populates the hash table with the remaining entries.
**Monitoring statement deallocation** – In PostgreSQL 14 and later, you can monitor statement deallocation using the pg\$1stat\$1statements\$1info view. This view includes a dealloc column that shows how many times statements were deallocated to make room for new ones
If the deallocation of statements occurs frequently, it will lead to more frequent garbage collection of the `pgss_query_texts.stat` file on disk.
## Likely causes of increased waits
The typical causes of increased `LWLock:pg_stat_statements` waits include:
+ An increase in the number of unique queries used by the application.
+ The `pg_stat_statements.max` parameter value being small compared to the number of unique queries being used.
## Actions
We recommend different actions depending on the causes of your wait event. You might identify `LWLock:pg_stat_statements` events by using Amazon RDS Performance Insights or by querying the view `pg_stat_activity`.
Adjust the following `pg_stat_statements` parameters to control tracking behavior and reduce LWLock:pg\$1stat\$1 statements wait events.
**Topics**
+ [
### Disable pg\$1stat\$1statements.track parameter
](#apg-rpg-lwlockpgstat.actions.disabletrack)
+ [
### Increase pg\$1stat\$1statements.max parameter
](#apg-rpg-lwlockpgstat.actions.increasemax)
+ [
### Disable pg\$1stat\$1statements.track\$1utility parameter
](#apg-rpg-lwlockpgstat.actions.disableutility)
### Disable pg\$1stat\$1statements.track parameter
If the LWLock:pg\$1stat\$1statements wait event is adversely impacting database performance, and a rapid solution is required before further analysis of the `pg_stat_statements` view to identify the root cause, the `pg_stat_statements.track` parameter can be disabled by setting it to `none`. This will disable the collection of statement statistics.
### Increase pg\$1stat\$1statements.max parameter
To reduce deallocation and minimize garbage collection of the `pgss_query_texts.stat` file on disk, increase the value of the `pg_stat_statements.max` parameter. The default value is `5,000`.
**Note**
The `pg_stat_statements.max` parameter is static. You must restart your DB instance to apply any changes to this parameter.
### Disable pg\$1stat\$1statements.track\$1utility parameter
You can analyze the pg\$1stat\$1statements view to determine which utility commands are consuming the most resources tracked by `pg_stat_statements`.
The `pg_stat_statements.track_utility` parameter controls whether the module tracks utility commands, which include all commands except SELECT, INSERT, UPDATE, DELETE, and MERGE. By default, this parameter is set to `on`.
For example, when your application uses many savepoint queries, which are inherently unique, it can increase statement deallocation. To address this, you can disable the `pg_stat_statements.track_utility` parameter to stop `pg_stat_statements` from tracking savepoint queries.
**Note**
The `pg_stat_statements.track_utility` parameter is a dynamic parameter. You can change its value without restarting your database instance.
**Example of unique save point queries in pg\$1stat\$1statements**
```
query | queryid
-------------------------------------------------+---------------------
SAVEPOINT JDBC_SAVEPOINT_495701 | -7249565344517699703
SAVEPOINT JDBC_SAVEPOINT_1320 | -1572997038849006629
SAVEPOINT JDBC_SAVEPOINT_26739 | 54791337410474486
SAVEPOINT JDBC_SAVEPOINT_1294466 | 8170064357463507593
ROLLBACK TO SAVEPOINT JDBC_SAVEPOINT_65016 | -33608214779996400
SAVEPOINT JDBC_SAVEPOINT_14185 | -2175035613806809562
SAVEPOINT JDBC_SAVEPOINT_45837 | -6201592986750645383
SAVEPOINT JDBC_SAVEPOINT_1324 | 6388797791882029332
```
PostgreSQL 17 introduces several enhancements for utility command tracking:
+ Savepoint names are now displayed as constants.
+ Global Transaction IDs (GIDs) of two-phase commit commands are now displayed as constants.
+ Names of DEALLOCATE statements are shown as constants.
+ CALL parameters are now displayed as constants.
# LWLock:SubtransSLRU (LWLock:SubtransControlLock)
The `LWLock:SubtransSLRU` and `LWLock:SubtransBuffer` wait events indicate that a session is waiting to access the simple least-recently used (SLRU) cache for subtransaction information. This occurs when determining transaction visibility and parent-child relationships.
+ `LWLock:SubtransSLRU`: A process is waiting to access the simple least-recently used (SLRU) cache for a subtransaction. In RDS for PostgreSQL prior to version 13, this wait event is called `SubtransControlLock`.
+ `LWLock:SubtransBuffer`: A process is waiting for I/O on a simple least-recently used (SLRU) buffer for a subtransaction. In RDS for PostgreSQL prior to version 13, this wait event is called `subtrans`.
**Topics**
+ [
## Supported engine versions
](#wait-event.lwlocksubtransslru.supported)
+ [
## Context
](#wait-event.lwlocksubtransslru.context)
+ [
## Likely causes of increased waits
](#wait-event.lwlocksubtransslru.causes)
+ [
## Actions
](#wait-event.lwlocksubtransslru.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
**Understanding subtransactions** – A subtransaction is a transaction within a transaction in PostgreSQL. It's also known as a nested transaction.
Subtransactions are typically created when you use:
+ `SAVEPOINT` commands
+ Exception blocks (`BEGIN/EXCEPTION/END`)
Subtransactions let you roll back parts of a transaction without affecting the entire transaction. This gives you fine-grained control over transaction management.
**Implementation details** – PostgreSQL implements subtransactions as nested structures within main transactions. Each subtransaction gets its own transaction ID.
Key implementation aspects:
+ Transaction IDs are tracked in `pg_xact`
+ Parent-child relationships are stored in `pg_subtrans` subdirectory under `PGDATA`
+ Each database session can maintain up to `64` active subtransactions
+ Exceeding this limit causes subtransaction overflow, which requires accessing the simple least-recently used (SLRU) cache for subtransaction information
## Likely causes of increased waits
Common causes of subtransaction SLRU contention include:
+ **Excessive use of SAVEPOINT and EXCEPTION handling** – PL/pgSQL procedures with `EXCEPTION` handlers automatically create implicit savepoints, regardless of whether exceptions occur. Each `SAVEPOINT` initiates a new subtransaction. When a single transaction accumulates more than 64 subtransactions, it triggers a subtransaction SLRU overflow.
+ **Driver and ORM configurations** – `SAVEPOINT` usage can be explicit in application code or implicit through driver configurations. Many commonly used ORM tools and application frameworks support nested transactions natively. Here are some common examples:
+ The JDBC driver parameter `autosave`, if set to `always` or `conservative`, generates savepoints before each query.
+ Spring Framework transaction definitions when set to `propagation_nested`.
+ Rails when `requires_new: true` is set.
+ SQLAlchemy when `session.begin_nested` is used.
+ Django when nested `atomic()` blocks are used.
+ GORM when `Savepoint` is used.
+ psqlODBC when rollback level setting is set to statement-level rollback (for example, `PROTOCOL=7.4-2`).
+ **High concurrent workloads with long-running transactions and subtransactions** – When subtransaction SLRU overflow occurs during high concurrent workloads and long-running transactions and subtransactions, PostgreSQL experiences increased contention. This manifests as elevated wait events for `LWLock:SubtransBuffer` and `LWLock:SubtransSLRU` locks.
## Actions
We recommend different actions depending on the causes of your wait event. Some actions provide immediate relief, while others require investigation and long-term correction.
**Topics**
+ [
### Monitoring subtransaction usage
](#wait-event.lwlocksubtransslru.actions.monitor)
+ [
### Configuring memory parameters
](#wait-event.lwlocksubtransslru.actions.memory)
+ [
### Long-term actions
](#wait-event.lwlocksubtransslru.actions.longterm)
### Monitoring subtransaction usage
For PostgreSQL versions 16.1 and later, use the following query to monitor subtransaction counts and overflow status per backend. This query joins backend statistics with activity information to show which processes are using subtransactions:
```
SELECT a.pid, usename, query, state, wait_event_type,
wait_event, subxact_count, subxact_overflowed
FROM (SELECT id, pg_stat_get_backend_pid(id) pid, subxact_count, subxact_overflowed
FROM pg_stat_get_backend_idset() id
JOIN LATERAL pg_stat_get_backend_subxact(id) AS s ON true
) a
JOIN pg_stat_activity b ON a.pid = b.pid;
```
For PostgreSQL versions 13.3 and later, monitor the `pg_stat_slru` view for subtransaction cache pressure. The following SQL query retrieves SLRU cache statistics for the Subtrans component:
```
SELECT * FROM pg_stat_slru WHERE name = 'Subtrans';
```
A consistently increasing `blks_read` value indicates frequent disk access for uncached subtransactions, signaling potential SLRU cache pressure.
### Configuring memory parameters
For PostgreSQL 17.1 and later, you can configure the subtransaction SLRU cache size using the `subtransaction_buffers` parameter. The following configuration example shows how to set the subtransaction buffer parameter:
```
subtransaction_buffers = 128
```
This parameter specifies the amount of shared memory used to cache subtransaction contents (`pg_subtrans`). When specified without units, the value represents blocks of `BLCKSZ` bytes, typically 8KB each. For example, setting the value to 128 allocates 1MB (128 \$1 8kB) of memory for the subtransaction cache.
**Note**
You can set this parameter at the cluster level so that all instances remain consistent. Test and adjust the value to suit your specific workload requirements and instance class. You must reboot the writer instance for parameter changes to take effect.
### Long-term actions
+ **Examine application code and configurations** – Review your application code and database driver configurations for both explicit and implicit `SAVEPOINT` usage and subtransactions usage in general. Identify transactions potentially generating over 64 subtransactions.
+ **Reduce savepoint usage** – Minimize the use of savepoints in your transactions:
+ Review PL/pgSQL procedures and functions with EXCEPTION blocks. EXCEPTION blocks automatically create implicit savepoints, which can contribute to subtransaction overflow. Each EXCEPTION clause creates a subtransaction, regardless of whether an exception actually occurs during execution.
**Example**
Example 1: Problematic EXCEPTION block usage
The following code example shows problematic EXCEPTION block usage that creates multiple subtransactions:
```
CREATE OR REPLACE FUNCTION process_user_data()
RETURNS void AS $$
DECLARE
user_record RECORD;
BEGIN
FOR user_record IN SELECT * FROM users LOOP
BEGIN
-- This creates a subtransaction for each iteration
INSERT INTO user_audit (user_id, action, timestamp)
VALUES (user_record.id, 'processed', NOW());
UPDATE users
SET last_processed = NOW()
WHERE id = user_record.id;
EXCEPTION
WHEN unique_violation THEN
-- Handle duplicate audit entries
UPDATE user_audit
SET timestamp = NOW()
WHERE user_id = user_record.id AND action = 'processed';
END;
END LOOP;
END;
$$ LANGUAGE plpgsql;
```
The following improved code example reduces subtransaction usage by using UPSERT instead of exception handling:
```
CREATE OR REPLACE FUNCTION process_user_data()
RETURNS void AS $$
DECLARE
user_record RECORD;
BEGIN
FOR user_record IN SELECT * FROM users LOOP
-- Use UPSERT to avoid exception handling
INSERT INTO user_audit (user_id, action, timestamp)
VALUES (user_record.id, 'processed', NOW())
ON CONFLICT (user_id, action)
DO UPDATE SET timestamp = NOW();
UPDATE users
SET last_processed = NOW()
WHERE id = user_record.id;
END LOOP;
END;
$$ LANGUAGE plpgsql;
```
**Example**
Example 2: STRICT exception handler
The following code example shows problematic EXCEPTION handling with NO\$1DATA\$1FOUND:
```
CREATE OR REPLACE FUNCTION get_user_email(p_user_id INTEGER)
RETURNS TEXT AS $$
DECLARE
user_email TEXT;
BEGIN
BEGIN
-- STRICT causes an exception if no rows or multiple rows found
SELECT email INTO STRICT user_email
FROM users
WHERE id = p_user_id;
RETURN user_email;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RETURN 'Email not found';
END;
END;
$$ LANGUAGE plpgsql;
```
The following improved code example avoids subtransactions by using IF NOT FOUND instead of exception handling:
```
CREATE OR REPLACE FUNCTION get_user_email(p_user_id INTEGER)
RETURNS TEXT AS $$
DECLARE
user_email TEXT;
BEGIN
SELECT email INTO user_email
FROM users
WHERE id = p_user_id;
IF NOT FOUND THEN
RETURN 'Email not found';
ELSE
RETURN user_email;
END IF;
END;
$$ LANGUAGE plpgsql;
```
+ JDBC driver – The `autosave` parameter, if set to `always` or `conservative`, generates savepoints before each query. Evaluate whether the `never` setting would be acceptable for your application.
+ PostgreSQL ODBC driver (psqlODBC) — The rollback level setting (for statement-level rollback) creates implicit savepoints to enable statement rollback functionality. Evaluate whether transaction-level or no rollback would be acceptable for your application.
+ Examine ORM transaction configurations
+ Consider alternative error handling strategies that don't require savepoints
+ **Optimize transaction design** – Restructure transactions to avoid excessive nesting and reduce the likelihood of subtransaction overflow conditions.
+ **Reduce long-running transactions** – Long-running transactions can exacerbate subtransaction issues by holding onto subtransaction information longer. Monitor Performance Insights metrics and configure the `idle_in_transaction_session_timeout` parameter to automatically terminate idle transactions.
+ Monitor Performance Insights metrics – Track metrics including `idle_in_transaction_count` (number of sessions in idle in transaction state) and `idle_in_transaction_max_time` (duration of the longest running idle transaction) to detect long-running transactions.
+ Configure `idle_in_transaction_session_timeout` – Set this parameter in your parameter group to automatically terminate idle transactions after a specified duration.
+ Proactive monitoring – Monitor for high occurrences of `LWLock:SubtransBuffer` and `LWLock:SubtransSLRU` wait events to detect subtransaction-related contention before it becomes critical.
# Timeout:PgSleep
The `Timeout:PgSleep` event occurs when a server process has called the `pg_sleep` function and is waiting for the sleep timeout to expire.
**Topics**
+ [
## Supported engine versions
](#wait-event.timeoutpgsleep.context.supported)
+ [
## Likely causes of increased waits
](#wait-event.timeoutpgsleep.causes)
+ [
## Actions
](#wait-event.timeoutpgsleep.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Likely causes of increased waits
This wait event occurs when an application, stored function, or user issues a SQL statement that calls one of the following functions:
+ `pg_sleep`
+ `pg_sleep_for`
+ `pg_sleep_until`
The preceding functions delay execution until the specified number of seconds have elapsed. For example, `SELECT pg_sleep(1)` pauses for 1 second. For more information, see [Delaying Execution](https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-DELAY) in the PostgreSQL documentation.
## Actions
Identify the statement that was running the `pg_sleep` function. Determine if the use of the function is appropriate.
# Timeout:VacuumDelay
The `Timeout:VacuumDelay` event indicates that the cost limit for vacuum I/O has been exceeded and that the vacuum process has been put to sleep. Vacuum operations stop for the duration specified in the respective cost delay parameter and then it resumes its work. For the manual vacuum command, the delay is specified in the `vacuum_cost_delay` parameter. For the autovacuum daemon, the delay is specified in the `autovacuum_vacuum_cost_delay parameter.`
**Topics**
+ [
## Supported engine versions
](#wait-event.timeoutvacuumdelay.context.supported)
+ [
## Context
](#wait-event.timeoutvacuumdelay.context)
+ [
## Likely causes of increased waits
](#wait-event.timeoutvacuumdelay.causes)
+ [
## Actions
](#wait-event.timeoutvacuumdelay.actions)
## Supported engine versions
This wait event information is supported for all versions of RDS for PostgreSQL.
## Context
PostgreSQL has both an autovacuum daemon and a manual vacuum command. The autovacuum process is "on" by default for RDS for PostgreSQL DB instances. The manual vacuum command is used on an as-needed basis, for example, to purge tables of dead tuples or generate new statistics.
When vacuuming is underway, PostgreSQL uses an internal counter to keep track of estimated costs as the system performs various I/O operations. When the counter reaches the value specified by the cost limit parameter, the process performing the operation sleeps for the brief duration specified in the cost delay parameter. It then resets the counter and continues operations.
The vacuum process has parameters that can be used to regulate resource consumption. The autovacuum and the manual vacuum command have their own parameters for setting the cost limit value. They also have their own parameters to specify a cost delay, an amount of time to put the vacuum to sleep when the limit is reached. In this way, the cost delay parameter works as a throttling mechanism for resource consumption. In the following lists, you can find description of these parameters.
**Parameters that affect throttling of the autovacuum daemon**
+ `[autovacuum\$1vacuum\$1cost\$1limit](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-LIMIT)` – Specifies the cost limit value to use in automatic vacuum operations. Increasing the setting for this parameter allows the vacuum process to use more resources and decreases the `Timeout:VacuumDelay` wait event.
+ `[autovacuum\$1vacuum\$1cost\$1delay](https://www.postgresql.org/docs/current/static/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-DELAY)` – Specifies the cost delay value to use in automatic vacuum operations. The default value is 2 milliseconds. Setting the delay parameter to 0 turns off the throttling mechanism and thus, the `Timeout:VacuumDelay` wait event won't appear.
For more information, see [Automatic Vacuuming](https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-VACUUM-COST-DELAY) in the PostgreSQL documentation.
**Parameters that affect throttling of the manual vacuum process**
+ `vacuum_cost_limit` – The threshold at which the vacuuming process is put to sleep. By default, the limit is 200. This number represents the accumulated cost estimates for extra I/O needed by various resources. Increasing this value reduces the number of the `Timeout:VacuumDelay` wait event.
+ `vacuum_cost_delay` – The amount of time that the vacuum process sleeps when the vacuum cost limit has been reached. The default setting is 0, which means that this feature is off. You can set this to an integer value to specify the number of milliseconds to turn on this feature, but we recommend that you leave it at its default setting.
For more information about the `vacuum_cost_delay` parameter, see [Resource Consumption](https://www.postgresql.org/docs/current/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-VACUUM-COST) in the PostgreSQL documentation.
To learn more about how to configure and use the autovacuum with RDS for PostgreSQL, see [Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md).
## Likely causes of increased waits
The `Timeout:VacuumDelay` is affected by the balance between the cost limit parameter settings (`vacuum_cost_limit`, `autovacuum_vacuum_cost_limit`) and the cost delay parameters (`vacuum_cost_delay`, `autovacuum_vacuum_cost_delay`) that control the vacuum's sleep duration. Raising a cost limit parameter value allows more resources to be used by the vacuum before being put to sleep. That results in fewer `Timeout:VacuumDelay` wait events. Increasing either of the delay parameters causes the `Timeout:VacuumDelay` wait event to occur more frequently and for longer periods of time.
The `autovacuum_max_workers` parameter setting can also increase numbers of the `Timeout:VacuumDelay`. Each additional autovacuum worker process contributes to the internal counter mechanism, and thus the limit can be reached more quickly than with a single autovacuum worker process. As the cost limit is reached more quickly, the cost delay is put to effect more frequently, resulting in more `Timeout:VacuumDelay` wait events. For more information, see [autovacuum\$1max\$1workers](https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-MAX-WORKERS) in the PostgreSQL documentation.
Large objects, such as 500GB or larger, also raise this wait event because it can take some time for the vacuum to complete processing large objects.
## Actions
If the vacuum operations complete as expected, no remediation is needed. In other words, this wait event doesn't necessarily indicate a problem. It indicates that the vacuum is being put to sleep for the period of time specified in the delay parameter so that resources can be applied to other processes that need to complete.
If you want vacuum operations to complete faster, you can lower the delay parameters. This shortens the time that the vacuum sleeps.
# Tuning RDS for PostgreSQL with Amazon DevOps Guru proactive insights
DevOps Guru proactive insights detects conditions on your RDS for PostgreSQL DB instances that can cause problems, and lets you know about them before they occur. Proactive insights can alert you to a long running idle in transaction connection. For more information about troubleshooting long running idle in transaction connections, see [Database has long running idle in transaction connection](#proactive-insights.idle-txn)
DevOps Guru can do the following:
+ Prevent many common database issues by cross-checking your database configuration against common recommended settings.
+ Alert you to critical issues in your fleet that, if left unchecked, can lead to larger problems later.
+ Alert you to newly discovered problems.
Every proactive insight contains an analysis of the cause of the problem and recommendations for corrective actions.
For more information about Amazon DevOps Guru for Amazon RDS, see [Analyzing performance anomalies with Amazon DevOps Guru for Amazon RDS](devops-guru-for-rds.md).
## Database has long running idle in transaction connection
A connection to the database has been in the `idle in transaction` state for more than 1800 seconds.
**Topics**
+ [
### Supported engine versions
](#proactive-insights.idle-txn.context.supported)
+ [
### Context
](#proactive-insights.idle-txn.context)
+ [
### Likely causes for this issue
](#proactive-insights.idle-txn.causes)
+ [
### Actions
](#proactive-insights.idle-txn.actions)
+ [
### Relevant metrics
](#proactive-insights.idle-txn.metrics)
### Supported engine versions
This insight information is supported for all versions of RDS for PostgreSQL.
### Context
A transaction in the `idle in transaction` state can hold locks that block other queries. It can also prevent `VACUUM` (including autovacuum) from cleaning up dead rows, leading to index or table bloat or transaction ID wraparound.
### Likely causes for this issue
A transaction initiated in an interactive session with BEGIN or START TRANSACTION hasn't ended by using a COMMIT, ROLLBACK, or END command. This causes the transaction to move to `idle in transaction` state.
### Actions
You can find idle transactions by querying `pg_stat_activity`.
In your SQL client, run the following query to list all connections in `idle in transaction` state and to order them by duration:
```
SELECT now() - state_change as idle_in_transaction_duration, now() - xact_start as xact_duration,*
FROM pg_stat_activity
WHERE state = 'idle in transaction'
AND xact_start is not null
ORDER BY 1 DESC;
```
We recommend different actions depending on the causes of your insight.
**Topics**
+ [
#### End transaction
](#proactive-insights.idle-txn.actions.end-txn)
+ [
#### Terminate the connection
](#proactive-insights.idle-txn.actions.end-connection)
+ [
#### Configure the idle\$1in\$1transaction\$1session\$1timeout parameter
](#proactive-insights.idle-txn.actions.parameter)
+ [
#### Check the AUTOCOMMIT status
](#proactive-insights.idle-txn.actions.autocommit)
+ [
#### Check the transaction logic in your application code
](#proactive-insights.idle-txn.actions.app-logic)
#### End transaction
When you initiate a transaction in an interactive session with BEGIN or START TRANSACTION, it moves to `idle in transaction` state. It remains in this state until you end the transaction by issuing a COMMIT, ROLLBACK, END command or disconnect the connection completely to roll back the transaction.
#### Terminate the connection
Terminate the connection with an idle transaction using the following query:
```
SELECT pg_terminate_backend(pid);
```
pid is the process ID of the connection.
#### Configure the idle\$1in\$1transaction\$1session\$1timeout parameter
Configure the `idle_in_transaction_session_timeout` parameter in the parameter group. The advantage of configuring this parameter is that it does not require a manual intervention to terminate the long idle in transaction. For more information on this parameter, see [the PostgreSQL documentation](https://www.postgresql.org/docs/current/runtime-config-client.html).
The following message will be reported in the PostgreSQL log file after the connection is terminated, when a transaction is in the idle\$1in\$1transaction state for longer than the specified time.
```
FATAL: terminating connection due to idle in transaction timeout
```
#### Check the AUTOCOMMIT status
AUTOCOMMIT is turned on by default. But if it is accidentally turned off in the client ensure that you turn it back on.
+ In your psql client, run the following command:
```
postgres=> \set AUTOCOMMIT on
```
+ In pgadmin, turn it on by choosing the AUTOCOMMIT option from the down arrow.
![\[In pgadmin, choose AUTOCOMMIT to turn it on.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/apg-insight-pgadmin-autocommit.png)
#### Check the transaction logic in your application code
Investigate your application logic for possible problems. Consider the following actions:
+ Check if the JDBC auto commit is set true in your application. Also, consider using explicit `COMMIT` commands in your code.
+ Check your error handling logic to see whether it closes a transaction after errors.
+ Check whether your application is taking long to process the rows returned by a query while the transaction is open. If so, consider coding the application to close the transaction before processing the rows.
+ Check whether a transaction contains many long-running operations. If so, divide a single transaction into multiple transactions.
### Relevant metrics
The following PI metrics are related to this insight:
+ idle\$1in\$1transaction\$1count - Number of sessions in `idle in transaction` state.
+ idle\$1in\$1transaction\$1max\$1time - The duration of the longest running transaction in the `idle in transaction` state.
# Using PostgreSQL extensions with Amazon RDS for PostgreSQL
You can extend the functionality of PostgreSQL by installing a variety of extensions and modules. For example, to work with spatial data you can install and use the PostGIS extension. For more information, see [Managing spatial data with the PostGIS extension](Appendix.PostgreSQL.CommonDBATasks.PostGIS.md). As another example, if you want to improve data entry for very large tables, you can consider partitioning your data by using the `pg_partman` extension. To learn more, see [Managing PostgreSQL partitions with the pg\$1partman extension](PostgreSQL_Partitions.md).
**Note**
RDS for PostgreSQL supports Trusted Language Extensions for PostgreSQL through the `pg_tle` extension, which you can add to your DB instance. By using this extension, developers can create their own PostgreSQL extensions in a safe environment that simplifies the setup and configuration requirements. To learn about RDS for PostgreSQL versions supporting `pg_tle` extension and for more information, see [Working with Trusted Language Extensions for PostgreSQL](PostgreSQL_trusted_language_extension.md).
In some cases, rather than installing an extension, you might add a specific module to the list of `shared_preload_libraries` in your RDS for PostgreSQL DB instance's custom DB parameter group. Typically, the default DB cluster parameter group loads only the `pg_stat_statements`, but several other modules are available to add to the list. For example, you can add scheduling capability by adding the `pg_cron` module, as detailed in [Scheduling maintenance with the PostgreSQL pg\$1cron extension](PostgreSQL_pg_cron.md). As another example, you can log query execution plans by loading the `auto_explain` module. To learn more, see [Logging execution plans of queries](https://aws.amazon.com/premiumsupport/knowledge-center/rds-postgresql-tune-query-performance/#) in the AWS knowledge center.
Depending on your version of RDS for PostgreSQL, installing an extension might require `rds_superuser` permissions, as follows:
+ For RDS for PostgreSQL versions 12 and earlier versions, installing extensions requires `rds_superuser` privileges.
+ For RDS for PostgreSQL version 13 and higher versions, users (roles) with create permissions on a given database instance can install and use any *trusted extensions*. For a list of trusted extensions, see [PostgreSQL trusted extensions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md#PostgreSQL.Concepts.General.Extensions.Trusted).
You can also specify precisely which extensions can be installed on your RDS for PostgreSQL DB instance, by listing them in the `rds.allowed_extensions` parameter. For more information, see [Restricting installation of PostgreSQL extensions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md#PostgreSQL.Concepts.General.FeatureSupport.Extensions.Restriction).
To learn more about the `rds_superuser` role, see [Understanding PostgreSQL roles and permissions](Appendix.PostgreSQL.CommonDBATasks.Roles.md).
**Topics**
+ [
# Using functions from the orafce extension
](Appendix.PostgreSQL.CommonDBATasks.orafce.md)
+ [
# Using Amazon RDS delegated extension support for PostgreSQL
](RDS_delegated_ext.md)
+ [
# Managing PostgreSQL partitions with the pg\$1partman extension
](PostgreSQL_Partitions.md)
+ [
# Using pgAudit to log database activity
](Appendix.PostgreSQL.CommonDBATasks.pgaudit.md)
+ [
# Scheduling maintenance with the PostgreSQL pg\$1cron extension
](PostgreSQL_pg_cron.md)
+ [
# Using pglogical to synchronize data across instances
](Appendix.PostgreSQL.CommonDBATasks.pglogical.md)
+ [
# Using pgactive to support active-active replication
](Appendix.PostgreSQL.CommonDBATasks.pgactive.md)
+ [
# Reducing bloat in tables and indexes with the pg\$1repack extension
](Appendix.PostgreSQL.CommonDBATasks.pg_repack.md)
+ [
# Upgrading and using the PLV8 extension
](PostgreSQL.Concepts.General.UpgradingPLv8.md)
+ [
# Using PL/Rust to write PostgreSQL functions in the Rust language
](PostgreSQL.Concepts.General.Using.PL_Rust.md)
+ [
# Managing spatial data with the PostGIS extension
](Appendix.PostgreSQL.CommonDBATasks.PostGIS.md)
# Using functions from the orafce extension
The orafce extension provides functions and operators that emulate a subset of functions and packages from an Oracle database. The orafce extension makes it easier for you to port an Oracle application to PostgreSQL. RDS for PostgreSQL versions 9.6.6 and higher support this extension. For more information about orafce, see [orafce](https://github.com/orafce/orafce) on GitHub.
**Note**
RDS for PostgreSQL doesn't support the `utl_file` package that is part of the orafce extension. This is because the `utl_file` schema functions provide read and write operations on operating-system text files, which requires superuser access to the underlying host. As a managed service, RDS for PostgreSQL doesn't provide host access.
**To use the orafce extension**
1. Connect to the DB instance with the primary user name that you used to create the DB instance.
If you want to turn on orafce for a different database in the same DB instance, use the `/c dbname` psql command. Using this command, you change from the primary database after initiating the connection.
1. Turn on the orafce extension with the `CREATE EXTENSION` statement.
```
CREATE EXTENSION orafce;
```
1. Transfer ownership of the oracle schema to the rds\$1superuser role with the `ALTER SCHEMA` statement.
```
ALTER SCHEMA oracle OWNER TO rds_superuser;
```
If you want to see the list of owners for the oracle schema, use the `\dn` psql command.
# Using Amazon RDS delegated extension support for PostgreSQL
Using Amazon RDS delegated extension support for PostgreSQL, you can delegate the extension management to a user who need not be an `rds_superuser`. With this delegated extension support, a new role called `rds_extension` is created and you must assign this to a user to manage other extensions. This role can create, update, and drop extensions.
You can specify the extensions that can be installed on your RDS DB instance, by listing them in the `rds.allowed_extensions` parameter. For more information, see [Using PostgreSQL extensions with Amazon RDS for PostgreSQL](https://docs.aws.amazon.com//AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Extensions.html).
You can restrict the list of extensions available that can be managed by the user with the `rds_extension` role using `rds.allowed_delegated_extensions` parameter.
The delegated extension support is available in the following versions:
+ All higher versions
+ 16.4 and higher 16 versions
+ 15.8 and higher 15 versions
+ 14.13 and higher 14 versions
+ 13.16 and higher 13 versions
+ 12.20 and higher 12 versions
**Topics**
+ [
## Turning on delegate extension support to a user
](#RDSPostgreSQL.delegated_ext_mgmt)
+ [
## Configuration used in RDS delegated extension support for PostgreSQL
](#RDSPostgreSQL.delegated_ext_config)
+ [
## Turning off the support for the delegated extension
](#RDSPostgreSQL.delegated_ext_disable)
+ [
## Benefits of using Amazon RDS delegated extension support
](#RDSPostgreSQL.delegated_ext_benefits)
+ [
## Limitation of Amazon RDS delegated extension support for PostgreSQL
](#RDSPostgreSQL.delegated_ext_limit)
+ [
## Permissions required for certain extensions
](#RDSPostgreSQL.delegated_ext_perm)
+ [
## Security Considerations
](#RDSPostgreSQL.delegated_ext_sec)
+ [
## Drop extension cascade disabled
](#RDSPostgreSQL.delegated_ext_drop)
+ [
## Example extensions that can be added using delegated extension support
](#RDSPostgreSQL.delegated_ext_support)
## Turning on delegate extension support to a user
You must perform the following to enable delegate extension support to a user:
1. **Grant `rds_extension` role to a user** – Connect to the database as `rds_superuser` and execute the following command:
```
Postgres => grant rds_extension to user_name;
```
1. **Set the list of extensions available for delegated users to manage** – The `rds.allowed_delegated_extensions` allows you to specify a subset of the available extensions using `rds.allowed_extensions` in the DB cluster parameter. You can perform this at one of the following levels:
+ In the cluster or the instance parameter group, through the AWS Management Console or API. For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
+ Use the following command at the database level:
```
alter database database_name set rds.allowed_delegated_extensions = 'extension_name_1,
extension_name_2,...extension_name_n';
```
+ Use the following command at the user level:
```
alter user user_name set rds.allowed_delegated_extensions = 'extension_name_1,
extension_name_2,...extension_name_n';
```
**Note**
You need not restart the database after changing the `rds.allowed_delegated_extensions` dynamic parameter.
1. **Allow access to the delegated user to objects created during the extension creation process** – Certain extensions create objects that require additional permissions to be granted before the user with `rds_extension` role can access them. The `rds_superuser` must grant the delegated user access to those objects. One of the options is to use an event trigger to automatically grant permission to the delegated user.
**Example of event trigger**
If you want to allow a delegated user with `rds_extension` to use extensions that require setting permissions on their objects created by the extension creation, you can customize the below example of an event trigger and add only the extensions for which you want the delegated users to have access to the full functionality. This event trigger can be created on template1 (the default template), therefore all database created from template1 will have that event trigger. When a delegated user installs the extension, this trigger will automatically grant ownership on the objects created by the extension.
```
CREATE OR REPLACE FUNCTION create_ext()
RETURNS event_trigger AS $$
DECLARE
schemaname TEXT;
databaseowner TEXT;
r RECORD;
BEGIN
IF tg_tag = 'CREATE EXTENSION' and current_user != 'rds_superuser' THEN
RAISE NOTICE 'SECURITY INVOKER';
RAISE NOTICE 'user: %', current_user;
FOR r IN SELECT * FROM pg_catalog.pg_event_trigger_ddl_commands()
LOOP
CONTINUE WHEN r.command_tag != 'CREATE EXTENSION' OR r.object_type != 'extension';
schemaname = (
SELECT n.nspname
FROM pg_catalog.pg_extension AS e
INNER JOIN pg_catalog.pg_namespace AS n
ON e.extnamespace = n.oid
WHERE e.oid = r.objid
);
databaseowner = (
SELECT pg_catalog.pg_get_userbyid(d.datdba)
FROM pg_catalog.pg_database d
WHERE d.datname = current_database()
);
RAISE NOTICE 'Record for event trigger %, objid: %,tag: %, current_user: %, schema: %, database_owenr: %', r.object_identity, r.objid, tg_tag, current_user, schemaname, databaseowner;
IF r.object_identity = 'address_standardizer_data_us' THEN
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.us_gaz TO %I WITH GRANT OPTION;', schemaname, databaseowner);
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.us_lex TO %I WITH GRANT OPTION;', schemaname, databaseowner);
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.us_rules TO %I WITH GRANT OPTION;', schemaname, databaseowner);
ELSIF r.object_identity = 'dict_int' THEN
EXECUTE pg_catalog.format('ALTER TEXT SEARCH DICTIONARY %I.intdict OWNER TO %I;', schemaname, databaseowner);
ELSIF r.object_identity = 'pg_partman' THEN
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.part_config TO %I WITH GRANT OPTION;', schemaname, databaseowner);
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.part_config_sub TO %I WITH GRANT OPTION;', schemaname, databaseowner);
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON TABLE %I.custom_time_partitions TO %I WITH GRANT OPTION;', schemaname, databaseowner);
ELSIF r.object_identity = 'postgis_topology' THEN
EXECUTE pg_catalog.format('GRANT SELECT, UPDATE, INSERT, DELETE ON ALL TABLES IN SCHEMA topology TO %I WITH GRANT OPTION;', databaseowner);
EXECUTE pg_catalog.format('GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA topology TO %I WITH GRANT OPTION;', databaseowner);
EXECUTE pg_catalog.format('GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA topology TO %I WITH GRANT OPTION;', databaseowner);
EXECUTE pg_catalog.format('GRANT USAGE ON SCHEMA topology TO %I WITH GRANT OPTION;', databaseowner);
END IF;
END LOOP;
END IF;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
CREATE EVENT TRIGGER log_create_ext ON ddl_command_end EXECUTE PROCEDURE create_ext();
```
## Configuration used in RDS delegated extension support for PostgreSQL
| Configuration Name | Description | Default Value | Notes | Who can modify or grant permission |
| --- | --- | --- | --- | --- |
| `rds.allowed_delegated_extensions` | This parameter limits the extensions a rds\$1extension role can manage in a database. It must be a subset of rds.allowed\$1extensions. | empty string | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/RDS_delegated_ext.html) To learn more about setting up this parameter, see [Turning on delegate extension support to a user](#RDSPostgreSQL.delegated_ext_mgmt). | rds\$1superuser |
| `rds.allowed_extensions` | This parameter lets the customer limit the extensions that can be installed in the RDS DB instance. For more information, see [Restricting installation of PostgreSQL extensions ](https://docs.aws.amazon.com//AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.Extensions.Restriction) | "\$1" | By default, this parameter is set to "\$1", which means that all extensions supported on RDS for PostgreSQL and Aurora PostgreSQL are allowed to be created by users with necessary privileges. Empty means no extensions can be installed in the RDS DB instance. | administrator |
| `rds-delegated_extension_allow_drop_cascade` | This parameter controls the ability for user with `rds_extension` to drop the extension using a cascade option. | off | By default, `rds-delegated_extension_allow_drop_cascade` is set to `off`. This means that users with `rds_extension` are not allowed to drop an extension using the cascade option. To grant that ability, the `rds.delegated_extension_allow_drop_cascade` parameter should be set to `on`. | rds\$1superuser |
## Turning off the support for the delegated extension
**Turning off partially**
The delegated users can’t create new extensions but can still update existing extensions.
+ Reset `rds.allowed_delegated_extensions` to the default value in the DB cluster parameter group.
+ Use the following command at the database level:
```
alter database database_name reset rds.allowed_delegated_extensions;
```
+ Use the following command at the user level:
```
alter user user_name reset rds.allowed_delegated_extensions;
```
**Turning off fully**
Revoking `rds_extension` role from a user will revert the user to standard permissions. The user can no longer create, update, or drop extensions.
```
postgres => revoke rds_extension from user_name;
```
## Benefits of using Amazon RDS delegated extension support
By using Amazon RDS delegated extension support for PostgreSQL, you securely delegate the extension management to users who do not have the `rds_superuser` role. This feature provides the following benefits:
+ You can easily delegate extension management to users of your choice.
+ This doesn’t require `rds_superuser` role.
+ Provides ability to support different set of extensions for different databases in the same DB cluster.
## Limitation of Amazon RDS delegated extension support for PostgreSQL
+ Objects created during the extension creation process may require additional privileges for the extension to function properly.
+ Some extensions can't be managed by the delegated extension user by default, including the following: `log_fdw`, `pg_cron`, `pg_tle`, `pgactive`, `pglogical`, `postgis_raster`, `postgis_tiger_geocoder`, `postgis_topology`.
## Permissions required for certain extensions
In order to create, use, or update the following extensions, the delegated user should have the necessary privileges on the following functions, tables, and schema.
| Extensions that need ownership or permissions | Function | Tables | Schema | Text Search Dictionary | Comment |
| --- | --- | --- | --- | --- | --- |
| address\$1standardizer\$1data\$1us | none | us\$1gaz, us\$1lex, us\$1lex, I.us\$1rules | none | none | none |
| amcheck | bt\$1index\$1check, bt\$1index\$1parent\$1check | none | none | none | none |
| dict\$1int | none | none | none | intdict | none |
| pg\$1partman | none | custom\$1time\$1partitions, part\$1config, part\$1config\$1sub | none | none | none |
| pg\$1stat\$1statements | none | none | none | none | none |
| PostGIS | st\$1tileenvelope | spatial\$1ref\$1sys | none | none | none |
| postgis\$1raster | none | none | none | none | none |
| postgis\$1topology | none | topology, layer | topology | none | the delegated user Must be the database owner |
| log\$1fdw | create\$1foreign\$1table\$1for\$1log\$1file | none | none | none | none |
| rds\$1tools | role\$1password\$1encryption\$1type | none | none | none | none |
| postgis\$1tiger\$1geocoder | none | geocode\$1settings\$1default, geocode\$1settings | tiger | none | none |
| pg\$1freespacemap | pg\$1freespace | none | none | none | none |
| pg\$1visibility | pg\$1visibility | none | none | none | none |
## Security Considerations
Keep in mind that a user with `rds_extension` role will be able to manage extensions on all databases they have the connect privilege on. If the intention is to have a delegated user manage extension on a single database, a good practice is to revoke all privileges from public on each database, then explicitly grant the connect privilege for that specific database to the delegate user.
There are several extensions that can allow a user to access information from multiple database. Ensure the users you grant `rds_extension` has cross database capabilities before adding these extensions to `rds.allowed_delegated_extensions`. For example, `postgres_fdw` and `dblink` provide functionality to query across databases on the same instance or remote instances. `log_fdw` reads the postgres engine log files, which are for all databases in the instance, potentially containing slow queries or error messages from multiple databases. `pg_cron` enables running scheduled background jobs on the DB instance and can configure jobs to run in a different database.
## Drop extension cascade disabled
The ability to drop the extension with cascade option by a user with the `rds_extension` role is controlled by `rds.delegated_extension_allow_drop_cascade` parameter. By default, `rds-delegated_extension_allow_drop_cascade` is set to `off`. This means that users with the `rds_extension` role are not allowed to drop an extension using the cascade option as shown in the below query.
```
DROP EXTENSION CASCADE;
```
As this will automatically drop objects that depend on the extension, and in turn all objects that depend on those objects. Attempting to use the cascade option will result in an error.
To grant that ability, the `rds.delegated_extension_allow_drop_cascade` parameter should be set to `on`.
Changing the `rds.delegated_extension_allow_drop_cascade` dynamic parameter doesn't require a database restart. You can do this at one of the following levels:
+ In the cluster or the instance parameter group, through the AWS Management Console or API.
+ Using the following command at the database level:
```
alter database database_name set rds.delegated_extension_allow_drop_cascade = 'on';
```
+ Using the following command at the user level:
```
alter role tenant_user set rds.delegated_extension_allow_drop_cascade = 'on';
```
## Example extensions that can be added using delegated extension support
+ `rds_tools`
```
extension_test_db=> create extension rds_tools;
CREATE EXTENSION
extension_test_db=> SELECT * from rds_tools.role_password_encryption_type() where rolname = 'pg_read_server_files';
ERROR: permission denied for function role_password_encryption_type
```
+ `amcheck`
```
extension_test_db=> CREATE TABLE amcheck_test (id int);
CREATE TABLE
extension_test_db=> INSERT INTO amcheck_test VALUES (generate_series(1,100000));
INSERT 0 100000
extension_test_db=> CREATE INDEX amcheck_test_btree_idx ON amcheck_test USING btree (id);
CREATE INDEX
extension_test_db=> create extension amcheck;
CREATE EXTENSION
extension_test_db=> SELECT bt_index_check('amcheck_test_btree_idx'::regclass);
ERROR: permission denied for function bt_index_check
extension_test_db=> SELECT bt_index_parent_check('amcheck_test_btree_idx'::regclass);
ERROR: permission denied for function bt_index_parent_check
```
+ `pg_freespacemap`
```
extension_test_db=> create extension pg_freespacemap;
CREATE EXTENSION
extension_test_db=> SELECT * FROM pg_freespace('pg_authid');
ERROR: permission denied for function pg_freespace
extension_test_db=> SELECT * FROM pg_freespace('pg_authid',0);
ERROR: permission denied for function pg_freespace
```
+ `pg_visibility`
```
extension_test_db=> create extension pg_visibility;
CREATE EXTENSION
extension_test_db=> select * from pg_visibility('pg_database'::regclass);
ERROR: permission denied for function pg_visibility
```
+ `postgres_fdw`
```
extension_test_db=> create extension postgres_fdw;
CREATE EXTENSION
extension_test_db=> create server myserver foreign data wrapper postgres_fdw options (host 'foo', dbname 'foodb', port '5432');
ERROR: permission denied for foreign-data wrapper postgres_fdw
```
# Managing PostgreSQL partitions with the pg\$1partman extension
PostgreSQL table partitioning provides a framework for high-performance handling of data input and reporting. Use partitioning for databases that require very fast input of large amounts of data. Partitioning also provides for faster queries of large tables. Partitioning helps maintain data without impacting the database instance because it requires less I/O resources.
By using partitioning, you can split data into custom-sized chunks for processing. For example, you can partition time-series data for ranges such as hourly, daily, weekly, monthly, quarterly, yearly, custom, or any combination of these. For a time-series data example, if you partition the table by hour, each partition contains one hour of data. If you partition the time-series table by day, the partitions holds one day's worth of data, and so on. The partition key controls the size of a partition.
When you use an `INSERT` or `UPDATE` SQL command on a partitioned table, the database engine routes the data to the appropriate partition. PostgreSQL table partitions that store the data are child tables of the main table.
During database query reads, the PostgreSQL optimizer examines the `WHERE` clause of the query and, if possible, directs the database scan to only the relevant partitions.
Starting with version 10, PostgreSQL uses declarative partitioning to implement table partitioning. This is also known as native PostgreSQL partitioning. Before PostgreSQL version 10, you used triggers to implement partitions.
PostgreSQL table partitioning provides the following features:
+ Creation of new partitions at any time.
+ Variable partition ranges.
+ Detachable and reattachable partitions using data definition language (DDL) statements.
For example, detachable partitions are useful for removing historical data from the main partition but keeping historical data for analysis.
+ New partitions inherit the parent database table properties, including the following:
+ Indexes
+ Primary keys, which must include the partition key column
+ Foreign keys
+ Check constraints
+ References
+ Creating indexes for the full table or each specific partition.
You can't alter the schema for an individual partition. However, you can alter the parent table (such as adding a new column), which propagates to partitions.
**Topics**
+ [
## Overview of the PostgreSQL pg\$1partman extension
](#PostgreSQL_Partitions.pg_partman)
+ [
## Enabling the pg\$1partman extension
](#PostgreSQL_Partitions.enable)
+ [
## Configuring partitions using the create\$1parent function
](#PostgreSQL_Partitions.create_parent)
+ [
## Configuring partition maintenance using the run\$1maintenance\$1proc function
](#PostgreSQL_Partitions.run_maintenance_proc)
## Overview of the PostgreSQL pg\$1partman extension
You can use the PostgreSQL `pg_partman` extension to automate the creation and maintenance of table partitions. For more general information, see [PG Partition Manager](https://github.com/pgpartman/pg_partman) in the `pg_partman` documentation.
**Note**
The `pg_partman` extension is supported on RDS for PostgreSQL versions 12.5 and higher.
Instead of having to manually create each partition, you configure `pg_partman` with the following settings:
+ Table to be partitioned
+ Partition type
+ Partition key
+ Partition granularity
+ Partition precreation and management options
After you create a PostgreSQL partitioned table, you register it with `pg_partman` by calling the `create_parent` function. Doing this creates the necessary partitions based on the parameters you pass to the function.
The `pg_partman` extension also provides the `run_maintenance_proc` function, which you can call on a scheduled basis to automatically manage partitions. To ensure that the proper partitions are created as needed, schedule this function to run periodically (such as hourly). You can also ensure that partitions are automatically dropped.
## Enabling the pg\$1partman extension
If you have multiple databases inside the same PostgreSQL DB instance for which you want to manage partitions, enable the `pg_partman` extension separately for each database. To enable the `pg_partman` extension for a specific database, create the partition maintenance schema and then create the `pg_partman` extension as follows.
```
CREATE SCHEMA partman;
CREATE EXTENSION pg_partman WITH SCHEMA partman;
```
**Note**
To create the `pg_partman` extension, make sure that you have `rds_superuser` privileges.
If you receive an error such as the following, grant the `rds_superuser` privileges to the account or use your superuser account.
```
ERROR: permission denied to create extension "pg_partman"
HINT: Must be superuser to create this extension.
```
To grant `rds_superuser` privileges, connect with your superuser account and run the following command.
```
GRANT rds_superuser TO user-or-role;
```
For the examples that show using the pg\$1partman extension, we use the following sample database table and partition. This database uses a partitioned table based on a timestamp. A schema `data_mart` contains a table named `events` with a column named `created_at`. The following settings are included in the `events` table:
+ Primary keys `event_id` and `created_at`, which must have the column used to guide the partition.
+ A check constraint `ck_valid_operation` to enforce values for an `operation` table column.
+ Two foreign keys, where one (`fk_orga_membership)` points to the external table `organization` and the other (`fk_parent_event_id`) is a self-referenced foreign key.
+ Two indexes, where one (`idx_org_id`) is for the foreign key and the other (`idx_event_type`) is for the event type.
The following DDL statements create these objects, which are automatically included on each partition.
```
CREATE SCHEMA data_mart;
CREATE TABLE data_mart.organization ( org_id BIGSERIAL,
org_name TEXT,
CONSTRAINT pk_organization PRIMARY KEY (org_id)
);
CREATE TABLE data_mart.events(
event_id BIGSERIAL,
operation CHAR(1),
value FLOAT(24),
parent_event_id BIGINT,
event_type VARCHAR(25),
org_id BIGSERIAL,
created_at timestamp,
CONSTRAINT pk_data_mart_event PRIMARY KEY (event_id, created_at),
CONSTRAINT ck_valid_operation CHECK (operation = 'C' OR operation = 'D'),
CONSTRAINT fk_orga_membership
FOREIGN KEY(org_id)
REFERENCES data_mart.organization (org_id),
CONSTRAINT fk_parent_event_id
FOREIGN KEY(parent_event_id, created_at)
REFERENCES data_mart.events (event_id,created_at)
) PARTITION BY RANGE (created_at);
CREATE INDEX idx_org_id ON data_mart.events(org_id);
CREATE INDEX idx_event_type ON data_mart.events(event_type);
```
## Configuring partitions using the create\$1parent function
After you enable the `pg_partman` extension, use the `create_parent` function to configure partitions inside the partition maintenance schema. The following example uses the `events` table example created in [Enabling the pg\$1partman extensionConfiguring partition maintenance using the run\$1maintenance\$1proc function](#PostgreSQL_Partitions.enable). Call the `create_parent` function as follows.
```
SELECT partman.create_parent(
p_parent_table => 'data_mart.events',
p_control => 'created_at',
p_type => 'range',
p_interval => '1 day',
p_premake => 30);
```
The parameters are as follows:
+ `p_parent_table` – The parent partitioned table. This table must already exist and be fully qualified, including the schema.
+ `p_control` – The column on which the partitioning is to be based. The data type must be an integer or time-based.
+ `p_type` – The type is either `'range'` or `'list'`.
+ `p_interval` – The time interval or integer range for each partition. Example values include `1 day`, `1 hour`, and so on.
+ `p_premake` – The number of partitions to create in advance to support new inserts.
For a complete description of the `create_parent` function, see [Creation Functions](https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#user-content-creation-functions) in the `pg_partman` documentation.
## Configuring partition maintenance using the run\$1maintenance\$1proc function
You can run partition maintenance operations to automatically create new partitions, detach partitions, or remove old partitions. Partition maintenance relies on the `run_maintenance_proc` function of the `pg_partman` extension and the `pg_cron` extension, which initiates an internal scheduler. The `pg_cron` scheduler automatically executes SQL statements, functions, and procedures defined in your databases.
The following example uses the `events` table example created in [Enabling the pg\$1partman extensionConfiguring partition maintenance using the run\$1maintenance\$1proc function](#PostgreSQL_Partitions.enable) to set partition maintenance operations to run automatically. As a prerequisite, add `pg_cron` to the `shared_preload_libraries` parameter in the DB instance's parameter group.
```
CREATE EXTENSION pg_cron;
UPDATE partman.part_config
SET infinite_time_partitions = true,
retention = '3 months',
retention_keep_table=true
WHERE parent_table = 'data_mart.events';
SELECT cron.schedule('@hourly', $$CALL partman.run_maintenance_proc()$$);
```
Following, you can find a step-by-step explanation of the preceding example:
1. Modify the parameter group associated with your DB instance and add `pg_cron` to the `shared_preload_libraries` parameter value. This change requires a DB instance restart for it to take effect. For more information, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md).
1. Run the command `CREATE EXTENSION pg_cron;` using an account that has the `rds_superuser` permissions. Doing this enables the `pg_cron` extension. For more information, see [Scheduling maintenance with the PostgreSQL pg\$1cron extension](PostgreSQL_pg_cron.md).
1. Run the command `UPDATE partman.part_config` to adjust the `pg_partman` settings for the `data_mart.events` table.
1. Run the command `SET` . . . to configure the `data_mart.events` table, with these clauses:
1. `infinite_time_partitions = true,` – Configures the table to be able to automatically create new partitions without any limit.
1. `retention = '3 months',` – Configures the table to have a maximum retention of three months.
1. `retention_keep_table=true `– Configures the table so that when the retention period is due, the table isn't deleted automatically. Instead, partitions that are older than the retention period are only detached from the parent table.
1. Run the command `SELECT cron.schedule` . . . to make a `pg_cron` function call. This call defines how often the scheduler runs the `pg_partman` maintenance procedure, `partman.run_maintenance_proc`. For this example, the procedure runs every hour.
For a complete description of the `run_maintenance_proc` function, see [Maintenance Functions](https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#maintenance-functions) in the `pg_partman` documentation.
# Using pgAudit to log database activity
Financial institutions, government agencies, and many industries need to keep *audit logs* to meet regulatory requirements. By using the PostgreSQL Audit extension (pgAudit) with your RDS for PostgreSQL DB instance, you can capture the detailed records that are typically needed by auditors or to meet regulatory requirements. For example, you can set up the pgAudit extension to track changes made to specific databases and tables, to record the user who made the change, and many other details.
The pgAudit extension builds on the functionality of the native PostgreSQL logging infrastructure by extending the log messages with more detail. In other words, you use the same approach to view your audit log as you do to view any log messages. For more information about PostgreSQL logging, see [ RDS for PostgreSQL database log files](USER_LogAccess.Concepts.PostgreSQL.md).
The pgAudit extension redacts sensitive data such as cleartext passwords from the logs. If your RDS for PostgreSQL DB instance is configured to log data manipulation language (DML) statements as detailed in [Turning on query logging for your RDS for PostgreSQL DB instance](USER_LogAccess.Concepts.PostgreSQL.Query_Logging.md), you can avoid the cleartext password issue by using the PostgreSQL Audit extension.
You can configure auditing on your database instances with a great degree of specificity. You can audit all databases and all users. Or, you can choose to audit only certain databases, users, and other objects. You can also explicitly exclude certain users and databases from being audited. For more information, see [Excluding users or databases from audit logging](Appendix.PostgreSQL.CommonDBATasks.pgaudit.exclude-user-db.md).
Given the amount of detail that can be captured, we recommend that if you do use pgAudit, you monitor your storage consumption.
The pgAudit extension is supported on all available RDS for PostgreSQL versions. For a list of pgAudit versions supported by available RDS for PostgreSQL versions, see [Extension versions for Amazon RDS for PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html) in the *Amazon RDS for PostgreSQL Release Notes.*
**Topics**
+ [
# Setting up the pgAudit extension
](Appendix.PostgreSQL.CommonDBATasks.pgaudit.basic-setup.md)
+ [
# Auditing database objects
](Appendix.PostgreSQL.CommonDBATasks.pgaudit.auditing.md)
+ [
# Excluding users or databases from audit logging
](Appendix.PostgreSQL.CommonDBATasks.pgaudit.exclude-user-db.md)
+ [
# Reference for the pgAudit extension
](Appendix.PostgreSQL.CommonDBATasks.pgaudit.reference.md)
# Setting up the pgAudit extension
To set up the pgAudit extension on your RDS for PostgreSQL DB instance , you first add pgAudit to the shared libraries on the custom DB parameter group for your RDS for PostgreSQL DB instance. For information about creating a custom DB parameter group, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md). Next, you install the pgAudit extension. Finally, you specify the databases and objects that you want to audit. The procedures in this section show you how. You can use the AWS Management Console or the AWS CLI.
You must have permissions as the `rds_superuser` role to perform all these tasks.
The steps following assume that your RDS for PostgreSQL DB instance is associated with a custom DB parameter group.
## Console
**To set up the pgAudit extension**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose your RDS for PostgreSQL DB instance.
1. Open the **Configuration** tab for your RDS for PostgreSQL DB instance. Among the Instance details, find the **Parameter group** link.
1. Choose the link to open the custom parameters associated with your RDS for PostgreSQL DB instance.
1. In the **Parameters** search field, type `shared_pre` to find the `shared_preload_libraries` parameter.
1. Choose **Edit parameters** to access the property values.
1. Add `pgaudit` to the list in the **Values** field. Use a comma to separate items in the list of values.
![\[Image of the shared_preload_libaries parameter with pgAudit added.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/apg_rpg_shared_preload_pgaudit.png)
1. Reboot the RDS for PostgreSQL DB instance so that your change to the `shared_preload_libraries` parameter takes effect.
1. When the instance is available, verify that pgAudit has been initialized. Use `psql` to connect to the RDS for PostgreSQL DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
shared_preload_libraries
--------------------------
rdsutils,pgaudit
(1 row)
```
1. With pgAudit initialized, you can now create the extension. You need to create the extension after initializing the library because the `pgaudit` extension installs event triggers for auditing data definition language (DDL) statements.
```
CREATE EXTENSION pgaudit;
```
1. Close the `psql` session.
```
labdb=> \q
```
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Find the `pgaudit.log` parameter in the list and set to the appropriate value for your use case. For example, setting the `pgaudit.log` parameter to `write` as shown in the following image captures inserts, updates, deletes, and some other types changes to the log.
![\[Image of the pgaudit.log parameter with setting.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/rpg_set_pgaudit-log-level.png)
You can also choose one of the following values for the `pgaudit.log` parameter.
+ none – This is the default. No database changes are logged.
+ all – Logs everything (read, write, function, role, ddl, misc).
+ ddl – Logs all data definition language (DDL) statements that aren't included in the `ROLE` class.
+ function – Logs function calls and `DO` blocks.
+ misc – Logs miscellaneous commands, such as `DISCARD`, `FETCH`, `CHECKPOINT`, `VACUUM`, and `SET`.
+ read – Logs `SELECT` and `COPY` when the source is a relation (such as a table) or a query.
+ role – Logs statements related to roles and privileges, such as `GRANT`, `REVOKE`, `CREATE ROLE`, `ALTER ROLE`, and `DROP ROLE`.
+ write – Logs `INSERT`, `UPDATE`, `DELETE`, `TRUNCATE`, and `COPY` when the destination is a relation (table).
1. Choose **Save changes**.
1. Open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose your RDS for PostgreSQL DB instance from the Databases list.
## AWS CLI
**To setup pgAudit**
To setup pgAudit using the AWS CLI, you call the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) operation to modify the audit log parameters in your custom parameter group, as shown in the following procedure.
1. Use the following AWS CLI command to add `pgaudit` to the `shared_preload_libraries` parameter.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=shared_preload_libraries,ParameterValue=pgaudit,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the following AWS CLI command to reboot the RDS for PostgreSQL DB instance so that the pgaudit library is initialized.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. When the instance is available, you can verify that `pgaudit` has been initialized. Use `psql` to connect to the RDS for PostgreSQL DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
shared_preload_libraries
--------------------------
rdsutils,pgaudit
(1 row)
```
With pgAudit initialized, you can now create the extension.
```
CREATE EXTENSION pgaudit;
```
1. Close the `psql` session so that you can use the AWS CLI.
```
labdb=> \q
```
1. Use the following AWS CLI command to specify the classes of statement that want logged by session audit logging. The example sets the `pgaudit.log` parameter to `write`, which captures inserts, updates, and deletes to the log.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=pgaudit.log,ParameterValue=write,ApplyMethod=pending-reboot" \
--region aws-region
```
You can also choose one of the following values for the `pgaudit.log` parameter.
+ none – This is the default. No database changes are logged.
+ all – Logs everything (read, write, function, role, ddl, misc).
+ ddl – Logs all data definition language (DDL) statements that aren't included in the `ROLE` class.
+ function – Logs function calls and `DO` blocks.
+ misc – Logs miscellaneous commands, such as `DISCARD`, `FETCH`, `CHECKPOINT`, `VACUUM`, and `SET`.
+ read – Logs `SELECT` and `COPY` when the source is a relation (such as a table) or a query.
+ role – Logs statements related to roles and privileges, such as `GRANT`, `REVOKE`, `CREATE ROLE`, `ALTER ROLE`, and `DROP ROLE`.
+ write – Logs `INSERT`, `UPDATE`, `DELETE`, `TRUNCATE`, and `COPY` when the destination is a relation (table).
Reboot the RDS for PostgreSQL DB instance using the following AWS CLI command.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
# Auditing database objects
With pgAudit set up on your RDS for PostgreSQL DB instance and configured for your requirements, more detailed information is captured in the PostgreSQL log. For example, while the default PostgreSQL logging configuration identifies the date and time that a change was made in a database table, with the pgAudit extension the log entry can include the schema, user who made the change, and other details depending on how the extension parameters are configured. You can set up auditing to track changes in the following ways.
+ For each session, by user. For the session level, you can capture the fully qualified command text.
+ For each object, by user and by database.
The object auditing capability is activated when you create the `rds_pgaudit` role on your system and then add this role to the `pgaudit.role` parameter in your custom parameter parameter group. By default, the `pgaudit.role` parameter is unset and the only allowable value is `rds_pgaudit`. The following steps assume that `pgaudit` has been initialized and that you have created the `pgaudit` extension by following the procedure in [Setting up the pgAudit extension](Appendix.PostgreSQL.CommonDBATasks.pgaudit.basic-setup.md).
![\[Image of the PostgreSQL log file after setting up pgAudit.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/pgaudit-log-example.png)
As shown in this example, the "LOG: AUDIT: SESSION" line provides information about the table and its schema, among other details.
**To set up object auditing**
1. Use `psql` to connect to the RDS for PostgreSQL DB instance.
```
psql --host=your-instance-name.aws-region.rds.amazonaws.com --port=5432 --username=postgrespostgres --password --dbname=labdb
```
1. Create a database role named `rds_pgaudit` using the following command.
```
labdb=> CREATE ROLE rds_pgaudit;
CREATE ROLE
labdb=>
```
1. Close the `psql` session.
```
labdb=> \q
```
In the next few steps, use the AWS CLI to modify the audit log parameters in your custom parameter group.
1. Use the following AWS CLI command to set the `pgaudit.role` parameter to `rds_pgaudit`. By default, this parameter is empty, and `rds_pgaudit` is the only allowable value.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=pgaudit.role,ParameterValue=rds_pgaudit,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the following AWS CLI command to reboot the RDS for PostgreSQL DB instance so that your changes to the parameters take effect.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. Run the following command to confirm that the `pgaudit.role` is set to `rds_pgaudit`.
```
SHOW pgaudit.role;
pgaudit.role
------------------
rds_pgaudit
```
To test pgAudit logging, you can run several example commands that you want to audit. For example, you might run the following commands.
```
CREATE TABLE t1 (id int);
GRANT SELECT ON t1 TO rds_pgaudit;
SELECT * FROM t1;
id
----
(0 rows)
```
The database logs should contain an entry similar to the following.
```
...
2017-06-12 19:09:49 UTC:...:rds_test@postgres:[11701]:LOG: AUDIT:
OBJECT,1,1,READ,SELECT,TABLE,public.t1,select * from t1;
...
```
For information on viewing the logs, see [Monitoring Amazon RDS log files](USER_LogAccess.md).
To learn more about the pgAudit extension, see [pgAudit](https://github.com/pgaudit/pgaudit/blob/master/README.md) on GitHub.
# Excluding users or databases from audit logging
As discussed in [ RDS for PostgreSQL database log files](USER_LogAccess.Concepts.PostgreSQL.md), PostgreSQL logs consume storage space. Using the pgAudit extension adds to the volume of data gathered in your logs to varying degrees, depending on the changes that you track. You might not need to audit every user or database in your RDS for PostgreSQL DB instance.
To minimize impacts to your storage and to avoid needlessly capturing audit records, you can exclude users and databases from being audited. You can also change logging within a given session. The following examples show you how.
**Note**
Parameter settings at the session level take precedence over the settings in the custom DB parameter group for the RDS for PostgreSQL DB instance. If you don't want database users to bypass your audit logging configuration settings, be sure to change their permissions.
Suppose that your RDS for PostgreSQL DB instance is configured to audit the same level of activity for all users and databases. You then decide that you don't want to audit the user `myuser`. You can turn off auditing for `myuser` with the following SQL command.
```
ALTER USER myuser SET pgaudit.log TO 'NONE';
```
Then, you can use the following query to check the `user_specific_settings` column for `pgaudit.log` to confirm that the parameter is set to `NONE`.
```
SELECT
usename AS user_name,
useconfig AS user_specific_settings
FROM
pg_user
WHERE
usename = 'myuser';
```
You see output such as the following.
```
user_name | user_specific_settings
-----------+------------------------
myuser | {pgaudit.log=NONE}
(1 row)
```
You can turn off logging for a given user in the midst of their session with the database with the following command.
```
ALTER USER myuser IN DATABASE mydatabase SET pgaudit.log TO 'none';
```
Use the following query to check the settings column for pgaudit.log for a specific user and database combination.
```
SELECT
usename AS "user_name",
datname AS "database_name",
pg_catalog.array_to_string(setconfig, E'\n') AS "settings"
FROM
pg_catalog.pg_db_role_setting s
LEFT JOIN pg_catalog.pg_database d ON d.oid = setdatabase
LEFT JOIN pg_catalog.pg_user r ON r.usesysid = setrole
WHERE
usename = 'myuser'
AND datname = 'mydatabase'
ORDER BY
1,
2;
```
You see output similar to the following.
```
user_name | database_name | settings
-----------+---------------+------------------
myuser | mydatabase | pgaudit.log=none
(1 row)
```
After turning off auditing for `myuser`, you decide that you don't want to track changes to `mydatabase`. You turn off auditing for that specific database by using the following command.
```
ALTER DATABASE mydatabase SET pgaudit.log to 'NONE';
```
Then, use the following query to check the database\$1specific\$1settings column to confirm that pgaudit.log is set to NONE.
```
SELECT
a.datname AS database_name,
b.setconfig AS database_specific_settings
FROM
pg_database a
FULL JOIN pg_db_role_setting b ON a.oid = b.setdatabase
WHERE
a.datname = 'mydatabase';
```
You see output such as the following.
```
database_name | database_specific_settings
---------------+----------------------------
mydatabase | {pgaudit.log=NONE}
(1 row)
```
To return settings to the default setting for myuser, use the following command:
```
ALTER USER myuser RESET pgaudit.log;
```
To return settings to their default setting for a database, use the following command.
```
ALTER DATABASE mydatabase RESET pgaudit.log;
```
To reset user and database to the default setting, use the following command.
```
ALTER USER myuser IN DATABASE mydatabase RESET pgaudit.log;
```
You can also capture specific events to the log by setting the `pgaudit.log` to one of the other allowed values for the `pgaudit.log` parameter. For more information, see [List of allowable settings for the `pgaudit.log` parameter](Appendix.PostgreSQL.CommonDBATasks.pgaudit.reference.md#Appendix.PostgreSQL.CommonDBATasks.pgaudit.reference.pgaudit-log-settings).
```
ALTER USER myuser SET pgaudit.log TO 'read';
ALTER DATABASE mydatabase SET pgaudit.log TO 'function';
ALTER USER myuser IN DATABASE mydatabase SET pgaudit.log TO 'read,function'
```
# Reference for the pgAudit extension
You can specify the level of detail that you want for your audit log by changing one or more of the parameters listed in this section.
## Controlling pgAudit behavior
You can control the audit logging by changing one or more of the parameters listed in the following table.
| Parameter | Description |
| --- | --- |
| `pgaudit.log` | Specifies the statement classes that will be logged by session audit logging. Allowable values include ddl, function, misc, read, role, write, none, all. For more information, see [List of allowable settings for the `pgaudit.log` parameter](#Appendix.PostgreSQL.CommonDBATasks.pgaudit.reference.pgaudit-log-settings). |
| `pgaudit.log_catalog` | When turned on (set to 1), adds statements to audit trail if all relations in a statement are in pg\$1catalog. |
| `pgaudit.log_level` | Specifies the log level to use for log entries. Allowed values: debug5, debug4, debug3, debug2, debug1, info, notice, warning, log |
| `pgaudit.log_parameter` | When turned on (set to 1), parameters passed with the statement are captured in the audit log. |
| `pgaudit.log_relation` | When turned on (set to 1), the audit log for the session creates a separate log entry for each relation (TABLE, VIEW, and so on) referenced in a SELECT or DML statement. |
| `pgaudit.log_statement_once` | Specifies whether logging will include the statement text and parameters with the first log entry for a statement/substatement combination or with every entry. |
| `pgaudit.role` | Specifies the master role to use for object audit logging. The only allowable entry is `rds_pgaudit`. |
## List of allowable settings for the `pgaudit.log` parameter
| Value | Description |
| --- | --- |
| none | This is the default. No database changes are logged. |
| all | Logs everything (read, write, function, role, ddl, misc). |
| ddl | Logs all data definition language (DDL) statements that aren't included in the `ROLE` class. |
| function | Logs function calls and `DO` blocks. |
| misc | Logs miscellaneous commands, such as `DISCARD`, `FETCH`, `CHECKPOINT`, `VACUUM`, and `SET`. |
| read | Logs `SELECT` and `COPY` when the source is a relation (such as a table) or a query. |
| role | Logs statements related to roles and privileges, such as `GRANT`, `REVOKE`, `CREATE ROLE`, `ALTER ROLE`, and `DROP ROLE`. |
| write | Logs `INSERT`, `UPDATE`, `DELETE`, `TRUNCATE`, and `COPY` when the destination is a relation (table). |
To log multiple event types with session auditing, use a comma-separated list. To log all event types, set `pgaudit.log` to `ALL`. Reboot your DB instance to apply the changes.
With object auditing, you can refine audit logging to work with specific relations. For example, you can specify that you want audit logging for `READ` operations on one or more tables.
# Scheduling maintenance with the PostgreSQL pg\$1cron extension
You can use the PostgreSQL `pg_cron` extension to schedule maintenance commands within a PostgreSQL database. For more information about the extension, see [What is pg\$1cron?](https://github.com/citusdata/pg_cron) in the pg\$1cron documentation.
The `pg_cron` extension is supported on RDS for PostgreSQL engine versions 12.5 and higher.
To learn more about using `pg_cron`, see [Schedule jobs with pg\$1cron on your RDS for PostgreSQL or your Aurora PostgreSQL-Compatible Edition databases](https://aws.amazon.com/blogs/database/schedule-jobs-with-pg_cron-on-your-amazon-rds-for-postgresql-or-amazon-aurora-for-postgresql-databases/).
**Note**
The `pg_cron` extension version is displayed as a two digit version, for example, 1.6, in the pg\$1available\$1extensions view. While you might see three digit versions, for example, 1.6.4 or 1.6.5, listed in some contexts, you must specify the two digit version when performing an extension upgrade.
**Topics**
+ [
## Setting up the pg\$1cron extension
](#PostgreSQL_pg_cron.enable)
+ [
## Granting database users permissions to use pg\$1cron
](#PostgreSQL_pg_cron.permissions)
+ [
## Scheduling pg\$1cron jobs
](#PostgreSQL_pg_cron.examples)
+ [
## Reference for the pg\$1cron extension
](#PostgreSQL_pg_cron.reference)
## Setting up the pg\$1cron extension
Set up the `pg_cron` extension as follows:
1. Modify the custom parameter group associated with your PostgreSQL DB instance by adding `pg_cron` to the `shared_preload_libraries` parameter value.
+ If your RDS for PostgreSQL DB instance uses the `rds.allowed_extensions` parameter to explicitly list extensions that can be installed, you need to add the `pg_cron` extension to the list. Only certain versions of RDS for PostgreSQL support the `rds.allowed_extensions` parameter. By default, all available extensions are allowed. For more information, see [Restricting installation of PostgreSQL extensions](PostgreSQL.Concepts.General.FeatureSupport.Extensions.md#PostgreSQL.Concepts.General.FeatureSupport.Extensions.Restriction).
Restart the PostgreSQL DB instance to have changes to the parameter group take effect. To learn more about working with parameter groups, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md).
1. After the PostgreSQL DB instance has restarted, run the following command using an account that has `rds_superuser` permissions. For example, if you used the default settings when you created your RDS for PostgreSQL DB instance, connect as user `postgres` and create the extension.
```
CREATE EXTENSION pg_cron;
```
The `pg_cron` scheduler is set in the default PostgreSQL database named `postgres`. The `pg_cron` objects are created in this `postgres` database and all scheduling actions run in this database.
1. You can use the default settings, or you can schedule jobs to run in other databases within your PostgreSQL DB instance. To schedule jobs for other databases within your PostgreSQL DB instance, see the example in [Scheduling a cron job for a database other than the default database](#PostgreSQL_pg_cron.otherDB).
## Granting database users permissions to use pg\$1cron
Installing the `pg_cron` extension requires the `rds_superuser` privileges. However, permissions to use the `pg_cron` can be granted (by a member of the `rds_superuser` group/role) to other database users, so that they can schedule their own jobs. We recommend that you grant permissions to the `cron` schema only as needed if it improves operations in your production environment.
To grant a database user permission in the `cron` schema, run the following command:
```
postgres=> GRANT USAGE ON SCHEMA cron TO db-user;
```
This gives *db-user* permission to access the `cron` schema to schedule cron jobs for the objects that they have permissions to access. If the database user doesn't have permissions, the job fails after posting the error message to the `postgresql.log` file, as shown in the following:
```
2020-12-08 16:41:00 UTC::@:[30647]:ERROR: permission denied for table table-name
2020-12-08 16:41:00 UTC::@:[27071]:LOG: background worker "pg_cron" (PID 30647) exited with exit code 1
```
In other words, make sure that database users that are granted permissions on the `cron` schema also have permissions on the objects (tables, schemas, and so on) that they plan to schedule.
The details of the cron job and its success or failure are also captured in the `cron.job_run_details` table. For more information, see [Tables for scheduling jobs and capturing status](#PostgreSQL_pg_cron.tables).
## Scheduling pg\$1cron jobs
The following sections show how you can schedule various management tasks using `pg_cron` jobs.
**Note**
When you create `pg_cron` jobs, check that the `max_worker_processes` setting is larger than the number of `cron.max_running_jobs`. A `pg_cron` job fails if it runs out of background worker processes. The default number of `pg_cron` jobs is `5`. For more information, see [Parameters for managing the pg\$1cron extension](#PostgreSQL_pg_cron.parameters).
**Topics**
+ [
### Vacuuming a table
](#PostgreSQL_pg_cron.vacuum)
+ [
### Purging the pg\$1cron history table
](#PostgreSQL_pg_cron.job_run_details)
+ [
### Logging errors to the postgresql.log file only
](#PostgreSQL_pg_cron.log_run)
+ [
### Scheduling a cron job for a database other than the default database
](#PostgreSQL_pg_cron.otherDB)
### Vacuuming a table
Autovacuum handles vacuum maintenance for most cases. However, you might want to schedule a vacuum of a specific table at a time of your choosing.
See also, [Working with PostgreSQL autovacuum on Amazon RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.Autovacuum.md).
Following is an example of using the `cron.schedule` function to set up a job to use `VACUUM FREEZE` on a specific table every day at 22:00 (GMT).
```
SELECT cron.schedule('manual vacuum', '0 22 * * *', 'VACUUM FREEZE pgbench_accounts');
schedule
----------
1
(1 row)
```
After the preceding example runs, you can check the history in the `cron.job_run_details` table as follows.
```
postgres=> SELECT * FROM cron.job_run_details;
jobid | runid | job_pid | database | username | command | status | return_message | start_time | end_time
-------+-------+---------+----------+----------+--------------------------------+-----------+----------------+-------------------------------+-------------------------------
1 | 1 | 3395 | postgres | adminuser| vacuum freeze pgbench_accounts | succeeded | VACUUM | 2020-12-04 21:10:00.050386+00 | 2020-12-04 21:10:00.072028+00
(1 row)
```
Following is a query of the `cron.job_run_details` table to see the failed jobs.
```
postgres=> SELECT * FROM cron.job_run_details WHERE status = 'failed';
jobid | runid | job_pid | database | username | command | status | return_message | start_time | end_time
------+-------+---------+----------+----------+-------------------------------+--------+--------------------------------------------------+-------------------------------+------------------------------
5 | 4 | 30339 | postgres | adminuser| vacuum freeze pgbench_account | failed | ERROR: relation "pgbench_account" does not exist | 2020-12-04 21:48:00.015145+00 | 2020-12-04 21:48:00.029567+00
(1 row)
```
For more information, see [Tables for scheduling jobs and capturing status](#PostgreSQL_pg_cron.tables).
### Purging the pg\$1cron history table
The `cron.job_run_details` table contains a history of cron jobs that can become very large over time. We recommend that you schedule a job that purges this table. For example, keeping a week's worth of entries might be sufficient for troubleshooting purposes.
The following example uses the [cron.schedule](#PostgreSQL_pg_cron.schedule) function to schedule a job that runs every day at midnight to purge the `cron.job_run_details` table. The job keeps only the last seven days. Use your `rds_superuser` account to schedule the job such as the following.
```
SELECT cron.schedule('0 0 * * *', $$DELETE
FROM cron.job_run_details
WHERE end_time < now() - interval '7 days'$$);
```
For more information, see [Tables for scheduling jobs and capturing status](#PostgreSQL_pg_cron.tables).
### Logging errors to the postgresql.log file only
To prevent writing to the `cron.job_run_details` table, modify the parameter group associated with the PostgreSQL DB instance and set the `cron.log_run` parameter to off. The `pg_cron` extension no longer writes to the table and captures errors to the `postgresql.log` file only. For more information, see [Modifying parameters in a DB parameter group in Amazon RDS](USER_WorkingWithParamGroups.Modifying.md).
Use the following command to check the value of the `cron.log_run` parameter.
```
postgres=> SHOW cron.log_run;
```
For more information, see [Parameters for managing the pg\$1cron extension](#PostgreSQL_pg_cron.parameters).
### Scheduling a cron job for a database other than the default database
The metadata for `pg_cron` is all held in the PostgreSQL default database named `postgres`. Because background workers are used for running the maintenance cron jobs, you can schedule a job in any of your databases within the PostgreSQL DB instance:
**Note**
Only users with `rds_superuser` role or `rds_superuser` privileges can list all cron jobs in the database. Other users can view only their own jobs in the `cron.job` table.
1. In the cron database, schedule the job as you normally do using the [cron.schedule](#PostgreSQL_pg_cron.schedule).
```
postgres=> SELECT cron.schedule('database1 manual vacuum', '29 03 * * *', 'vacuum freeze test_table');
```
1. As a user with the `rds_superuser` role, update the database column for the job that you just created so that it runs in another database within your PostgreSQL DB instance.
```
postgres=> UPDATE cron.job SET database = 'database1' WHERE jobid = 106;
```
1. Verify by querying the `cron.job` table.
```
postgres=> SELECT * FROM cron.job;
jobid | schedule | command | nodename | nodeport | database | username | active | jobname
------+-------------+--------------------------------+-----------+----------+----------+-----------+--------+-------------------------
106 | 29 03 * * * | vacuum freeze test_table | localhost | 8192 | database1| adminuser | t | database1 manual vacuum
1 | 59 23 * * * | vacuum freeze pgbench_accounts | localhost | 8192 | postgres | adminuser | t | manual vacuum
(2 rows)
```
**Note**
In some situations, you might add a cron job that you intend to run on a different database. In such cases, the job might try to run in the default database (`postgres`) before you update the correct database column. If the user name has permissions, the job successfully runs in the default database.
## Reference for the pg\$1cron extension
You can use the following parameters, functions, and tables with the `pg_cron` extension. For more information, see [What is pg\$1cron?](https://github.com/citusdata/pg_cron) in the pg\$1cron documentation.
**Topics**
+ [
### Parameters for managing the pg\$1cron extension
](#PostgreSQL_pg_cron.parameters)
+ [
### Function reference: cron.schedule
](#PostgreSQL_pg_cron.schedule)
+ [
### Function reference: cron.unschedule
](#PostgreSQL_pg_cron.unschedule)
+ [
### Tables for scheduling jobs and capturing status
](#PostgreSQL_pg_cron.tables)
### Parameters for managing the pg\$1cron extension
Following is a list of parameters that control the `pg_cron` extension behavior.
| Parameter | Description |
| --- | --- |
| cron.database\$1name | The database in which `pg_cron` metadata is kept. |
| cron.host | The hostname to connect to PostgreSQL. You can't modify this value. |
| cron.log\$1run | Log every job that runs in the `job_run_details` table. Values are `on` or `off`. For more information, see [Tables for scheduling jobs and capturing status](#PostgreSQL_pg_cron.tables). |
| cron.log\$1statement | Log all cron statements before running them. Values are `on` or `off`. |
| cron.max\$1running\$1jobs | The maximum number of jobs that can run concurrently. |
| cron.use\$1background\$1workers | Use background workers instead of client sessions. You can't modify this value. |
Use the following SQL command to display these parameters and their values.
```
postgres=> SELECT name, setting, short_desc FROM pg_settings WHERE name LIKE 'cron.%' ORDER BY name;
```
### Function reference: cron.schedule
This function schedules a cron job. The job is initially scheduled in the default `postgres` database. The function returns a `bigint` value representing the job identifier. To schedule jobs to run in other databases within your PostgreSQL DB instance, see the example in [Scheduling a cron job for a database other than the default database](#PostgreSQL_pg_cron.otherDB).
The function has two syntax formats.
**Syntax**
```
cron.schedule (job_name,
schedule,
command
);
cron.schedule (schedule,
command
);
```
**Parameters**
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_pg_cron.html)
**Examples**
```
postgres=> SELECT cron.schedule ('test','0 10 * * *', 'VACUUM pgbench_history');
schedule
----------
145
(1 row)
postgres=> SELECT cron.schedule ('0 15 * * *', 'VACUUM pgbench_accounts');
schedule
----------
146
(1 row)
```
### Function reference: cron.unschedule
This function deletes a cron job. You can specify either the `job_name` or the `job_id`. A policy makes sure that you are the owner to remove the schedule for the job. The function returns a Boolean indicating success or failure.
The function has the following syntax formats.
**Syntax**
```
cron.unschedule (job_id);
cron.unschedule (job_name);
```
**Parameters**
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_pg_cron.html)
**Examples**
```
postgres=> SELECT cron.unschedule(108);
unschedule
------------
t
(1 row)
postgres=> SELECT cron.unschedule('test');
unschedule
------------
t
(1 row)
```
### Tables for scheduling jobs and capturing status
The following tables are used to schedule the cron jobs and record how the jobs completed.
| Table | Description |
| --- | --- |
| cron.job | Contains the metadata about each scheduled job. Most interactions with this table should be done by using the `cron.schedule` and `cron.unschedule` functions. We recommend that you don't give update or insert privileges directly to this table. Doing so would allow the user to update the `username` column to run as `rds-superuser`. |
| cron.job\$1run\$1details | Contains historic information about past scheduled jobs that ran. This is useful to investigate the status, return messages, and start and end time from the job that ran. To prevent this table from growing indefinitely, purge it on a regular basis. For an example, see [Purging the pg\$1cron history table](#PostgreSQL_pg_cron.job_run_details). |
# Using pglogical to synchronize data across instances
All currently available RDS for PostgreSQL versions support the `pglogical` extension. The pglogical extension predates the functionally similar logical replication feature that was introduced by PostgreSQL in version 10. For more information, see [Performing logical replication for Amazon RDS for PostgreSQL](PostgreSQL.Concepts.General.FeatureSupport.LogicalReplication.md).
The `pglogical` extension supports logical replication between two or more RDS for PostgreSQL DB instances. It also supports replication between different PostgreSQL versions, and between databases running on RDS for PostgreSQL DB instances and Aurora PostgreSQL DB clusters. The `pglogical` extension uses a publish-subscribe model to replicate changes to tables and other objects, such as sequences, from a publisher to a subscriber. It relies on a replication slot to ensure that changes are synchronized from a publisher node to a subscriber node, defined as follows.
+ The *publisher node* is the RDS for PostgreSQL DB instance that's the source of data to be replicated to other nodes. The publisher node defines the tables to be replicated in a publication set.
+ The *subscriber node* is the RDS for PostgreSQL DB instance that receives WAL updates from the publisher. The subscriber creates a subscription to connect to the publisher and get the decoded WAL data. When the subscriber creates the subscription, the replication slot is created on the publisher node.
Following, you can find information about setting up the `pglogical` extension.
**Topics**
+ [
## Requirements and limitations for the pglogical extension
](#Appendix.PostgreSQL.CommonDBATasks.pglogical.requirements-limitations)
+ [
# Setting up the pglogical extension
](Appendix.PostgreSQL.CommonDBATasks.pglogical.basic-setup.md)
+ [
# Setting up logical replication for RDS for PostgreSQL DB instance
](Appendix.PostgreSQL.CommonDBATasks.pglogical.setup-replication.md)
+ [
# Reestablishing logical replication after a major upgrade
](Appendix.PostgreSQL.CommonDBATasks.pglogical.recover-replication-after-upgrade.md)
+ [
# Managing logical replication slots for RDS for PostgreSQL
](Appendix.PostgreSQL.CommonDBATasks.pglogical.handle-slots.md)
+ [
# Parameter reference for the pglogical extension
](Appendix.PostgreSQL.CommonDBATasks.pglogical.reference.md)
## Requirements and limitations for the pglogical extension
All currently available releases of RDS for PostgreSQL support the `pglogical` extension.
Both the publisher node and the subscriber node must be set up for logical replication.
The tables that you want to replicate from a publisher to a subscriber must have the same names and the same schema. These tables must also contain the same columns, and the columns must use the same data types. Both publisher and subscriber tables must have the same primary keys. We recommend that you use only the PRIMARY KEY as the unique constraint.
The tables on the subscriber node can have more permissive constraints than those on the publisher node for CHECK constraints and NOT NULL constraints.
The `pglogical` extension provides features such as two-way replication that aren't supported by the logical replication feature built into PostgreSQL (version 10 and higher). For more information, see [PostgreSQL bi-directional replication using pglogical](https://aws.amazon.com/blogs/database/postgresql-bi-directional-replication-using-pglogical/).
# Setting up the pglogical extension
To set up the `pglogical` extension on your RDS for PostgreSQL DB instance , you add `pglogical` to the shared libraries on the custom DB parameter group for your RDS for PostgreSQL DB instance. You also need to set the value of the `rds.logical_replication` parameter to `1`, to turn on logical decoding. Finally, you create the extension in the database. You can use the AWS Management Console or the AWS CLI for these tasks.
You must have permissions as the `rds_superuser` role to perform these tasks.
The steps following assume that your RDS for PostgreSQL DB instance is associated with a custom DB parameter group. For information about creating a custom DB parameter group, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
## Console
**To set up the pglogical extension**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose your RDS for PostgreSQL DB instance.
1. Open the **Configuration** tab for your RDS for PostgreSQL DB instance. Among the Instance details, find the **Parameter group** link.
1. Choose the link to open the custom parameters associated with your RDS for PostgreSQL DB instance.
1. In the **Parameters** search field, type `shared_pre` to find the `shared_preload_libraries` parameter.
1. Choose **Edit parameters** to access the property values.
1. Add `pglogical` to the list in the **Values** field. Use a comma to separate items in the list of values.
![\[Image of the shared_preload_libraries parameter with pglogical added.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/apg_rpg_shared_preload_pglogical.png)
1. Find the `rds.logical_replication` parameter and set it to `1`, to turn on logical replication.
1. Reboot the RDS for PostgreSQL DB instance so that your changes take effect.
1. When the instance is available, you can use `psql` (or pgAdmin) to connect to the RDS for PostgreSQL DB instance.
```
psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=labdb
```
1. To verify that pglogical is initialized, run the following command.
```
SHOW shared_preload_libraries;
shared_preload_libraries
--------------------------
rdsutils,pglogical
(1 row)
```
1. Verify the setting that enables logical decoding, as follows.
```
SHOW wal_level;
wal_level
-----------
logical
(1 row)
```
1. Create the extension, as follows.
```
CREATE EXTENSION pglogical;
EXTENSION CREATED
```
1. Choose **Save changes**.
1. Open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose your RDS for PostgreSQL DB instance from the Databases list to select it, and then choose **Reboot** from the Actions menu.
## AWS CLI
**To setup the pglogical extension**
To setup pglogical using the AWS CLI, you call the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) operation to modify certain parameters in your custom parameter group as shown in the following procedure.
1. Use the following AWS CLI command to add `pglogical` to the `shared_preload_libraries` parameter.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=shared_preload_libraries,ParameterValue=pglogical,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the following AWS CLI command to set `rds.logical_replication` to `1` to turn on the logical decoding capability for the RDS for PostgreSQL DB instance.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=rds.logical_replication,ParameterValue=1,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the following AWS CLI command to reboot the RDS for PostgreSQL DB instance so that the pglogical library is initialized.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. When the instance is available, use `psql` to connect to the RDS for PostgreSQL DB instance.
```
psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=labdb
```
1. Create the extension, as follows.
```
CREATE EXTENSION pglogical;
EXTENSION CREATED
```
1. Reboot the RDS for PostgreSQL DB instance using the following AWS CLI command.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
# Setting up logical replication for RDS for PostgreSQL DB instance
The following procedure shows you how to start logical replication between two RDS for PostgreSQL DB instances. The steps assume that both the source (publisher) and the target (subscriber) have the `pglogical` extension set up as detailed in [Setting up the pglogical extension](Appendix.PostgreSQL.CommonDBATasks.pglogical.basic-setup.md).
**Note**
The `node_name` of a subscriber node can't start with `rds`.
**To create the publisher node and define the tables to replicate**
These steps assume that your RDS for PostgreSQL DB instance has a database that has one or more tables that you want to replicate to another node. You need to recreate the table structure from the publisher on the subscriber, so first, get the table structure if necessary. You can do that by using the `psql` metacommand `\d tablename` and then creating the same table on the subscriber instance. The following procedure creates an example table on the publisher (source) for demonstration purposes.
1. Use `psql` to connect to the instance that has the table you want to use as a source for subscribers.
```
psql --host=source-instance.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=labdb
```
If you don't have an existing table that you want to replicate, you can create a sample table as follows.
1. Create an example table using the following SQL statement.
```
CREATE TABLE docs_lab_table (a int PRIMARY KEY);
```
1. Populate the table with generated data by using the following SQL statement.
```
INSERT INTO docs_lab_table VALUES (generate_series(1,5000));
INSERT 0 5000
```
1. Verify that data exists in the table by using the following SQL statement.
```
SELECT count(*) FROM docs_lab_table;
```
1. Identify this RDS for PostgreSQL DB instance as the publisher node, as follows.
```
SELECT pglogical.create_node(
node_name := 'docs_lab_provider',
dsn := 'host=source-instance.aws-region.rds.amazonaws.com port=5432 dbname=labdb');
create_node
-------------
3410995529
(1 row)
```
1. Add the table that you want to replicate to the default replication set. For more information about replication sets, see [Replication sets](https://github.com/2ndQuadrant/pglogical/tree/REL2_x_STABLE/docs#replication-sets) in the pglogical documentation.
```
SELECT pglogical.replication_set_add_table('default', 'docs_lab_table', 'true', NULL, NULL);
replication_set_add_table
---------------------------
t
(1 row)
```
The publisher node setup is complete. You can now set up the subscriber node to receive the updates from the publisher.
**To set up the subscriber node and create a subscription to receive updates**
These steps assume that the RDS for PostgreSQL DB instance has been set up with the `pglogical` extension. For more information, see [Setting up the pglogical extension](Appendix.PostgreSQL.CommonDBATasks.pglogical.basic-setup.md).
1. Use `psql` to connect to the instance that you want to receive updates from the publisher.
```
psql --host=target-instance.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=labdb
```
1. On the subscriber RDS for PostgreSQL DB instance,create the same table that exists on the publisher. For this example, the table is `docs_lab_table`. You can create the table as follows.
```
CREATE TABLE docs_lab_table (a int PRIMARY KEY);
```
1. Verify that this table is empty.
```
SELECT count(*) FROM docs_lab_table;
count
-------
0
(1 row)
```
1. Identify this RDS for PostgreSQL DB instance as the subscriber node, as follows.
```
SELECT pglogical.create_node(
node_name := 'docs_lab_target',
dsn := 'host=target-instance.aws-region.rds.amazonaws.com port=5432 sslmode=require dbname=labdb user=postgres password=********');
create_node
-------------
2182738256
(1 row)
```
1. Create the subscription.
```
SELECT pglogical.create_subscription(
subscription_name := 'docs_lab_subscription',
provider_dsn := 'host=source-instance.aws-region.rds.amazonaws.com port=5432 sslmode=require dbname=labdb user=postgres password=*******',
replication_sets := ARRAY['default'],
synchronize_data := true,
forward_origins := '{}' );
create_subscription
---------------------
1038357190
(1 row)
```
When you complete this step, the data from the table on the publisher is created in the table on the subscriber. You can verify that this has occurred by using the following SQL query.
```
SELECT count(*) FROM docs_lab_table;
count
-------
5000
(1 row)
```
From this point forward, changes made to the table on the publisher are replicated to the table on the subscriber.
# Reestablishing logical replication after a major upgrade
Before you can perform a major version upgrade of an RDS for PostgreSQL DB instance that's set up as a publisher node for logical replication, you must drop all replication slots, even those that aren't active. We recommend that you temporarily divert database transactions from the publisher node, drop the replication slots, upgrade the RDS for PostgreSQL DB instance, and then re-establish and restart replication.
The replication slots are hosted on the publisher node only. The RDS for PostgreSQL subscriber node in a logical replication scenario has no slots to drops, but it can't be upgraded to a major version while it's designated as a subscriber node with a subscription to the publisher. Before upgrading the RDS for PostgreSQL subscriber node, drop the subscription and the node. For more information, see [Managing logical replication slots for RDS for PostgreSQL](Appendix.PostgreSQL.CommonDBATasks.pglogical.handle-slots.md).
## Determining that logical replication has been disrupted
You can determine that the replication process has been disrupted by querying either the publisher node or the subscriber node, as follows.
**To check the publisher node**
+ Use `psql` to connect to the publisher node, and then query the `pg_replication_slots` function. Note the value in the active column. Normally, this will return `t` (true), showing that replication is active. If the query returns `f` (false), it's an indication that replication to the subscriber has stopped.
```
SELECT slot_name,plugin,slot_type,active FROM pg_replication_slots;
slot_name | plugin | slot_type | active
-------------------------------------------+------------------+-----------+--------
pgl_labdb_docs_labcb4fa94_docs_lab3de412c | pglogical_output | logical | f
(1 row)
```
**To check the subscriber node**
On the subscriber node, you can check the status of replication in three different ways.
+ Look through the PostgreSQL logs on the subscriber node to find failure messages. The log identifies failure with messages that include exit code 1, as shown following.
```
2022-07-06 16:17:03 UTC::@:[7361]:LOG: background worker "pglogical apply 16404:2880255011" (PID 14610) exited with exit code 1
2022-07-06 16:19:44 UTC::@:[7361]:LOG: background worker "pglogical apply 16404:2880255011" (PID 21783) exited with exit code 1
```
+ Query the `pg_replication_origin` function. Connect to the database on the subscriber node using `psql` and query the `pg_replication_origin` function, as follows.
```
SELECT * FROM pg_replication_origin;
roident | roname
---------+--------
(0 rows)
```
The empty result set means that replication has been disrupted. Normally, you see output such as the following.
```
roident | roname
---------+----------------------------------------------------
1 | pgl_labdb_docs_labcb4fa94_docs_lab3de412c
(1 row)
```
+ Query the `pglogical.show_subscription_status` function as shown in the following example.
```
SELECT subscription_name,status,slot_name FROM pglogical.show_subscription_status();
subscription_name | status | slot_name
---====----------------+--------+-------------------------------------
docs_lab_subscription | down | pgl_labdb_docs_labcb4fa94_docs_lab3de412c
(1 row)
```
This output shows that replication has been disrupted. Its status is `down`. Normally, the output shows the status as `replicating`.
If your logical replication process has been disrupted, you can re-establish replication by following these steps.
**To reestablish logical replication between publisher and subscriber nodes**
To re-establish replication, you first disconnect the subscriber from the publisher node and then re-establish the subscription, as outlined in these steps.
1. Connect to the subscriber node using `psql` as follows.
```
psql --host=222222222222.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=labdb
```
1. Deactivate the subscription by using the `pglogical.alter_subscription_disable` function.
```
SELECT pglogical.alter_subscription_disable('docs_lab_subscription',true);
alter_subscription_disable
----------------------------
t
(1 row)
```
1. Get the publisher node's identifier by querying the `pg_replication_origin`, as follows.
```
SELECT * FROM pg_replication_origin;
roident | roname
---------+-------------------------------------
1 | pgl_labdb_docs_labcb4fa94_docs_lab3de412c
(1 row)
```
1. Use the response from the previous step with the `pg_replication_origin_create` command to assign the identifier that can be used by the subscription when re-established.
```
SELECT pg_replication_origin_create('pgl_labdb_docs_labcb4fa94_docs_lab3de412c');
pg_replication_origin_create
------------------------------
1
(1 row)
```
1. Turn on the subscription by passing its name with a status of `true`, as shown in the following example.
```
SELECT pglogical.alter_subscription_enable('docs_lab_subscription',true);
alter_subscription_enable
---------------------------
t
(1 row)
```
Check the status of the node. Its status should be `replicating` as shown in this example.
```
SELECT subscription_name,status,slot_name
FROM pglogical.show_subscription_status();
subscription_name | status | slot_name
-------------------------------+-------------+-------------------------------------
docs_lab_subscription | replicating | pgl_labdb_docs_lab98f517b_docs_lab3de412c
(1 row)
```
Check the status of the subscriber's replication slot on the publisher node. The slot's `active` column should return `t` (true), indicating that replication has been re-established.
```
SELECT slot_name,plugin,slot_type,active
FROM pg_replication_slots;
slot_name | plugin | slot_type | active
-------------------------------------------+------------------+-----------+--------
pgl_labdb_docs_lab98f517b_docs_lab3de412c | pglogical_output | logical | t
(1 row)
```
# Managing logical replication slots for RDS for PostgreSQL
Before you can perform a major version upgrade on an RDS for PostgreSQL DB instance that's serving as a publisher node in a logical replication scenario, you must drop the replication slots on the instance. The major version upgrade pre-check process notifies you that the upgrade can't proceed until the slots are dropped.
To drop slots from your RDS for PostgreSQL DB instance, first drop the subscription and then drop the slot.
To identify replication slots that were created using the `pglogical` extension, log in to each database and get the name of the nodes. When you query the subscriber node, you get both the publisher and the subscriber nodes in the output, as shown in this example.
```
SELECT * FROM pglogical.node;
node_id | node_name
------------+-------------------
2182738256 | docs_lab_target
3410995529 | docs_lab_provider
(2 rows)
```
You can get the details about the subscription with the following query.
```
SELECT sub_name,sub_slot_name,sub_target
FROM pglogical.subscription;
sub_name | sub_slot_name | sub_target
----------+--------------------------------+------------
docs_lab_subscription | pgl_labdb_docs_labcb4fa94_docs_lab3de412c | 2182738256
(1 row)
```
You can now drop the subscription, as follows.
```
SELECT pglogical.drop_subscription(subscription_name := 'docs_lab_subscription');
drop_subscription
-------------------
1
(1 row)
```
After dropping the subscription, you can delete the node.
```
SELECT pglogical.drop_node(node_name := 'docs-lab-subscriber');
drop_node
-----------
t
(1 row)
```
You can verify that the node no longer exists, as follows.
```
SELECT * FROM pglogical.node;
node_id | node_name
---------+-----------
(0 rows)
```
# Parameter reference for the pglogical extension
In the table you can find parameters associated with the `pglogical` extension. Parameters such as `pglogical.conflict_log_level` and `pglogical.conflict_resolution` are used to handle update conflicts. Conflicts can emerge when changes are made locally to the same tables that are subscribed to changes from the publisher. Conflicts can also occur during various scenarios, such as two-way replication or when multiple subscribers are replicating from the same publisher. For more information, see [ PostgreSQL bi-directional replication using pglogical](https://aws.amazon.com/blogs/database/postgresql-bi-directional-replication-using-pglogical/).
| Parameter | Description |
| --- | --- |
| pglogical.batch\$1inserts | Batch inserts if possible. Not set by default. Change to '1' to turn on, '0' to turn off. |
| pglogical.conflict\$1log\$1level | Sets the log level to use for logging resolved conflicts. Supported string values are debug5, debug4, debug3, debug2, debug1, info, notice, warning, error, log, fatal, panic. |
| pglogical.conflict\$1resolution | Sets method to use to resolve conflicts when conflicts are resolvable. Supported string values are error, apply\$1remote, keep\$1local, last\$1update\$1wins, first\$1update\$1wins. |
| pglogical.extra\$1connection\$1options | Connection options to add to all peer node connections. |
| pglogical.synchronous\$1commit | pglogical specific synchronous commit value |
| pglogical.use\$1spi | Use SPI (server programming interface) instead of low-level API to apply changes. Set to '1' to turn on, '0' to turn off. For more information about SPI, see [Server Programming Interface](https://www.postgresql.org/docs/current/spi.html) in the PostgreSQL documentation. |
# Using pgactive to support active-active replication
The `pgactive` extension uses active-active replication to support and coordinate write operations on multiple RDS for PostgreSQL databases. Amazon RDS for PostgreSQL supports the `pgactive` extension on the following versions:
+ RDS for PostgreSQL 17.0 and all higher versions
+ RDS for PostgreSQL 16.1 and higher 16 versions
+ RDS for PostgreSQL 15.4-R2 and higher 15 versions
+ RDS for PostgreSQL 14.10 and higher 14 versions
+ RDS for PostgreSQL 13.13 and higher 13 versions
+ RDS for PostgreSQL 12.17 and higher 12 versions
+ RDS for PostgreSQL 11.22
**Note**
When there are write operations on more than one database in a replication configuration, conflicts are possible. For more information, see [Handling conflicts in active-active replication](Appendix.PostgreSQL.CommonDBATasks.pgactive.handle-conflicts.md)
**Topics**
+ [
## Limitations for the pgactive extension
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.requirements-limitations)
+ [
# Initializing the pgactive extension capability
](Appendix.PostgreSQL.CommonDBATasks.pgactive.basic-setup.md)
+ [
# Setting up active-active replication for RDS for PostgreSQL DB instances
](Appendix.PostgreSQL.CommonDBATasks.pgactive.setup-replication.md)
+ [
# Measuring replication lag among pgactive members
](Appendix.PostgreSQL.CommonDBATasks.pgactive.replicationlag.md)
+ [
# Configuring parameter settings for the pgactive extension
](Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.md)
+ [
# Understanding active-active conflicts
](Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.replication.md)
+ [
# Understanding the pgactive schema
](Appendix.PostgreSQL.CommonDBATasks.pgactive.schema.md)
+ [
# pgactive functions reference
](pgactive-functions-reference.md)
+ [
# Handling conflicts in active-active replication
](Appendix.PostgreSQL.CommonDBATasks.pgactive.handle-conflicts.md)
+ [
# Handling sequences in active-active replication
](Appendix.PostgreSQL.CommonDBATasks.pgactive.handle-sequences.md)
## Limitations for the pgactive extension
+ All tables require a Primary Key, otherwise Update's and Delete's aren't allowed. The values in the Primary Key column shouldn't be updated.
+ Sequences may have gaps and sometimes might not follow an order. Sequences are not replicated. For more information, see [Handling sequences in active-active replication](Appendix.PostgreSQL.CommonDBATasks.pgactive.handle-sequences.md).
+ DDL and large objects are not replicated.
+ Secondary unique indexes can cause data divergence.
+ Collation needs to be identical on all node in the group.
+ Load balancing across nodes is an anti-pattern.
+ Large transactions can cause replication lag.
# Initializing the pgactive extension capability
To initialize the `pgactive` extension capability on your RDS for PostgreSQL DB instance, set the value of the `rds.enable_pgactive` parameter to `1` and then create the extension in the database. Doing so automatically turns on the parameters `rds.logical_replication` and `track_commit_timestamp` and sets the value of `wal_level` to `logical`.
You must have permissions as the `rds_superuser` role to perform these tasks.
You can use the AWS Management Console or the AWS CLI to create the required RDS for PostgreSQL DB instances. The steps following assume that your RDS for PostgreSQL DB instance is associated with a custom DB parameter group. For information about creating a custom DB parameter group, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
## Console
**To initialize the pgactive extension capability**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose your RDS for PostgreSQL DB instance.
1. Open the **Configuration** tab for your RDS for PostgreSQL DB instance. In the instance details, find the **DB instance parameter group** link.
1. Choose the link to open the custom parameters associated with your RDS for PostgreSQL DB instance.
1. Find the `rds.enable_pgactive` parameter, and set it to `1` to initialize the `pgactive` capability.
1. Choose **Save changes**.
1. In the navigation pane of the Amazon RDS console, choose **Databases**.
1. Select your RDS for PostgreSQL DB instance, and then choose **Reboot** from the **Actions** menu.
1. Confirm the DB instance reboot so that your changes take effect.
1. When the DB instance is available, you can use `psql` or any other PostgreSQL client to connect to the RDS for PostgreSQL DB instance.
The following example assumes that your RDS for PostgreSQL DB instance has a default database named *postgres*.
```
psql --host=mydb.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password=PASSWORD --dbname=postgres
```
1. To verify that pgactive is initialized, run the following command.
```
postgres=>SELECT setting ~ 'pgactive'
FROM pg_catalog.pg_settings
WHERE name = 'shared_preload_libraries';
```
If `pgactive` is in `shared_preload_libraries`, the preceding command will return the following:
```
?column?
----------
t
```
## AWS CLI
**To initialize the pgactive extension capability**
To initialize the `pgactive` using the AWS CLI, call the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) operation to modify certain parameters in your custom parameter group as shown in the following procedure.
1. Use the following AWS CLI command to set `rds.enable_pgactive` to `1` to initialize the `pgactive` capability for the RDS for PostgreSQL DB instance.
```
postgres=>aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=rds.enable_pgactive,ParameterValue=1,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the following AWS CLI command to reboot the RDS for PostgreSQL DB instance so that the `pgactive` library is initialized.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. When the instance is available, use `psql` to connect to the RDS for PostgreSQL DB instance.
```
psql --host=mydb.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=master user --password=PASSWORD --dbname=postgres
```
1. To verify that pgactive is initialized, run the following command.
```
postgres=>SELECT setting ~ 'pgactive'
FROM pg_catalog.pg_settings
WHERE name = 'shared_preload_libraries';
```
If `pgactive` is in `shared_preload_libraries`, the preceding command will return the following:
```
?column?
----------
t
```
# Setting up active-active replication for RDS for PostgreSQL DB instances
The following procedure shows you how to start active-active replication between two RDS for PostgreSQL DB instances where `pgactive` is available. To run the multi-region high availability example, you need to deploy Amazon RDS for PostgreSQL instances in two different regions and set up VPC Peering. For more information, see [VPC peering](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html).
**Note**
Sending traffic between multiple regions may incur additional costs.
These steps assume that the RDS for PostgreSQL DB instance has been enabled with the `pgactive` extension. For more information, see [Initializing the pgactive extension capability](Appendix.PostgreSQL.CommonDBATasks.pgactive.basic-setup.md).
**To configure the first RDS for PostgreSQL DB instance with the `pgactive` extension**
The following example illustrates how the `pgactive` group is created, along with other steps required to create the `pgactive` extension on the RDS for PostgreSQL DB instance.
1. Use `psql` or another client tool to connect to your first RDS for PostgreSQL DB instance.
```
psql --host=firstinstance.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password=PASSWORD --dbname=postgres
```
1. Create a database on the RDS for PostgreSQL instance using the following command:
```
postgres=> CREATE DATABASE app;
```
1. Switch connection to the new database using the following command:
```
\c app
```
1. Create and populate a sample table using the following SQL statements:
1. Create an example table using the following SQL statement.
```
app=> CREATE SCHEMA inventory;
CREATE TABLE inventory.products (
id int PRIMARY KEY, product_name text NOT NULL,
created_at timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP);
```
1. Populate the table with some sample data by using the following SQL statement.
```
app=> INSERT INTO inventory.products (id, product_name)
VALUES (1, 'soap'), (2, 'shampoo'), (3, 'conditioner');
```
1. Verify that data exists in the table by using the following SQL statement.
```
app=>SELECT count(*) FROM inventory.products;
count
-------
3
```
1. Create `pgactive` extension on the existing database.
```
app=> CREATE EXTENSION pgactive;
```
1. To securely create and initialize the pgactive group use the following commands:
```
app=>
-- connection info for endpoint1
CREATE SERVER pgactive_server_endpoint1
FOREIGN DATA WRAPPER pgactive_fdw
OPTIONS (host '', dbname 'app');
CREATE USER MAPPING FOR postgres
SERVER pgactive_server_endpoint1
OPTIONS (user 'postgres', password '');
-- connection info for endpoint2
CREATE SERVER pgactive_server_endpoint2
FOREIGN DATA WRAPPER pgactive_fdw
OPTIONS (host '', dbname 'app');
CREATE USER MAPPING FOR postgres
SERVER pgactive_server_endpoint2
OPTIONS (user 'postgres', password '');
```
Now you can initialize the replication group and add this first instance:
```
SELECT pgactive.pgactive_create_group(
node_name := 'endpoint1-app',
node_dsn := 'user_mapping=postgres pgactive_foreign_server=pgactive_server_endpoint1'
);
```
Use the following commands as an alternate but less secure method to create and initialize the pgactive group:
```
app=> SELECT pgactive.pgactive_create_group(
node_name := 'node1-app',
node_dsn := 'dbname=app host=firstinstance.111122223333.aws-region.rds.amazonaws.com user=postgres password=PASSWORD');
```
node1-app is the name that you assign to uniquely identify a node in the `pgactive` group.
**Note**
To perform this step successfully on a DB instance that is publicly accessible, you must turn on the `rds.custom_dns_resolution` parameter by setting it to `1`.
1. To check if the DB instance is ready, use the following command:
```
app=> SELECT pgactive.pgactive_wait_for_node_ready();
```
If the command succeeds, you can see the following output:
```
pgactive_wait_for_node_ready
------------------------------
(1 row)
```
**To configure the second RDS for PostgreSQL instance and join it to the `pgactive` group**
The following example illustrates how you can join an RDS for PostgreSQL DB instance to the `pgactive` group, along with other steps that are required to create the `pgactive` extension on the DB instance.
These steps assume that another RDS for PostgreSQL DB instances has been set up with the `pgactive` extension. For more information, see [Initializing the pgactive extension capability](Appendix.PostgreSQL.CommonDBATasks.pgactive.basic-setup.md).
1. Use `psql` to connect to the instance that you want to receive updates from the publisher.
```
psql --host=secondinstance.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password=PASSWORD --dbname=postgres
```
1. Create a database on the second RDS for PostgreSQL DB instance using the following command:
```
postgres=> CREATE DATABASE app;
```
1. Switch connection to the new database using the following command:
```
\c app
```
1. Create the `pgactive` extension on the existing database.
```
app=> CREATE EXTENSION pgactive;
```
1. Join the RDS for PostgreSQL second DB instance to the `pgactive` group in a more secure way using the following commands:
```
-- connection info for endpoint1
CREATE SERVER pgactive_server_endpoint1
FOREIGN DATA WRAPPER pgactive_fdw
OPTIONS (host '', dbname 'app');
CREATE USER MAPPING FOR postgres
SERVER pgactive_server_endpoint1
OPTIONS (user 'postgres', password '');
-- connection info for endpoint2
CREATE SERVER pgactive_server_endpoint2
FOREIGN DATA WRAPPER pgactive_fdw
OPTIONS (host '', dbname 'app');
CREATE USER MAPPING FOR postgres
SERVER pgactive_server_endpoint2
OPTIONS (user 'postgres', password '');
```
```
SELECT pgactive.pgactive_join_group(
node_name := 'endpoint2-app',
node_dsn := 'user_mapping=postgres pgactive_foreign_server=pgactive_server_endpoint2',
join_using_dsn := 'user_mapping=postgres pgactive_foreign_server=pgactive_server_endpoint1'
);
```
Use the following commands as an alternate but less secure method to join the RDS for PostgreSQL second DB instance to the `pgactive` group
```
app=> SELECT pgactive.pgactive_join_group(
node_name := 'node2-app',
node_dsn := 'dbname=app host=secondinstance.111122223333.aws-region.rds.amazonaws.com user=postgres password=PASSWORD',
join_using_dsn := 'dbname=app host=firstinstance.111122223333.aws-region.rds.amazonaws.com user=postgres password=PASSWORD');
```
node2-app is the name that you assign to uniquely identify a node in the `pgactive` group.
1. To check if the DB instance is ready, use the following command:
```
app=> SELECT pgactive.pgactive_wait_for_node_ready();
```
If the command succeeds, you can see the following output:
```
pgactive_wait_for_node_ready
------------------------------
(1 row)
```
If the first RDS for PostgreSQL database is relatively large, you can see `pgactive.pgactive_wait_for_node_ready()` emitting the progress report of the restore operation. The output looks similar to the following:
```
NOTICE: restoring database 'app', 6% of 7483 MB complete
NOTICE: restoring database 'app', 42% of 7483 MB complete
NOTICE: restoring database 'app', 77% of 7483 MB complete
NOTICE: restoring database 'app', 98% of 7483 MB complete
NOTICE: successfully restored database 'app' from node node1-app in 00:04:12.274956
pgactive_wait_for_node_ready
------------------------------
(1 row)
```
From this point forward, `pgactive` synchronizes the data between the two DB instances.
1. You can use the following command to verify if the database of the second DB instance has the data:
```
app=> SELECT count(*) FROM inventory.products;
```
If the data is successfully synchronized, you’ll see the following output:
```
count
-------
3
```
1. Run the following command to insert new values:
```
app=> INSERT INTO inventory.products (id, product_name) VALUES (4, 'lotion');
```
1. Connect to the database of the first DB instance and run the following query:
```
app=> SELECT count(*) FROM inventory.products;
```
If the active-active replication is initialized, the output is similar to the following:
```
count
-------
4
```
**To detach and remove a DB instance from the `pgactive` group**
You can detach and remove a DB instance from the `pgactive` group using these steps:
1. You can detach the second DB instance from the first DB instance using the following command:
```
app=> SELECT * FROM pgactive.pgactive_detach_nodes(ARRAY[‘node2-app']);
```
1. Remove the `pgactive` extension from the second DB instance using the following command:
```
app=> SELECT * FROM pgactive.pgactive_remove();
```
To forcefully remove the extension:
```
app=> SELECT * FROM pgactive.pgactive_remove(true);
```
1. Drop the extension using the following command:
```
app=> DROP EXTENSION pgactive;
```
# Measuring replication lag among pgactive members
You can use the following query to view the replication lag among the `pgactive` members. Run this query on every `pgactive` node to get the full picture.
```
app=> SELECT * FROM pgactive.pgactive_get_replication_lag_info();
│-[ RECORD 1 ]--------+---------------------------------------------
│node_name | node2-app
│node_sysid | 7481018224801653637
│application_name | pgactive:7481018224801653637:send
│slot_name | pgactive_16385_7481018224801653637_0_16385__
│active | t
│active_pid | 783486
│pending_wal_decoding | 0
│pending_wal_to_apply | 0
│restart_lsn | 0/2108150
│confirmed_flush_lsn | 0/2154690
│sent_lsn | 0/2154690
│write_lsn | 0/2154690
│flush_lsn | 0/2154690
│replay_lsn | 0/2154690
│-[ RECORD 2 ]--------+---------------------------------------------
│node_name | node1-app
│node_sysid | 7481018033434600853
│application_name | pgactive:7481018033434600853:send
│slot_name | pgactive_16385_7481018033434600853_0_16385__
│active | t
│active_pid | 783488
│pending_wal_decoding | 0
│pending_wal_to_apply | 0
│restart_lsn | 0/20F5AD0
│confirmed_flush_lsn | 0/214EF68
│sent_lsn | 0/214EF68
│write_lsn | 0/214EF68
│flush_lsn | 0/214EF68
│replay_lsn | 0/214EF68
```
Monitor the following diagnostics at a minimum:
active
Set up alerts when active is false, which indicates that the slot isn't currently in use (the subscriber instance has disconnected from the publisher).
pending\$1wal\$1decoding
In PostgreSQL's logical replication, WAL files are stored in binary format. The publisher must decode these WAL changes and convert them into logical changes (such as insert, update, or delete operations).
The metric pending\$1wal\$1decoding shows the number of WAL files waiting to be decoded on the publisher side.
This number can increase due to these factors:
+ When the subscriber isn't connected, active status will be false and pending\$1wal\$1decoding will increase
+ The slot is active, but the publisher can't keep up with the volume of WAL changes
pending\$1wal\$1to\$1apply
The metric pending\$1wal\$1apply indicates the number of WAL files waiting to be applied on the subscriber side.
Several factors can prevent the subscriber from applying changes and potentially cause a disk full scenario:
+ Schema differences - for example, when you have changes in the WAL stream for a table named sample, but that table doesn't exist on the subscriber side
+ Values in the primary key columns were updated
+ Secondary unique indexes can cause data divergence
# Configuring parameter settings for the pgactive extension
You can use the following query to view all the parameters associated with `pgactive` extension.
```
app=> SELECT * FROM pg_settings WHERE name LIKE 'pgactive.%';
```
You can configure the `pgactive` extension using various parameters. These parameters can be set through either the AWS Management Console or the AWS CLI interface.
## Main pgactive extension parameters
The following table provides a reference for the main parameters of the `pgactive` extension:
| Parameter | Unit | Default | Description |
| --- | --- | --- | --- |
| pgactive.conflict\$1logging\$1include\$1tuples | `boolean` | – | Logs complete tuple information for the `pgactive` extension. A server restart is required for changes to take effect. |
| pgactive.log\$1conflicts\$1to\$1table | `boolean` | – | Determines whether the `pgactive` extension logs the detected conflicts to the `pgactive.pgactive_conflict_history` table. For more information, see Conflict logging for details. A server restart is required for changes to take effect. |
| pgactive.log\$1conflicts\$1to\$1logfile | `boolean` | – | Determines whether the `pgactive` extension logs the detected conflicts to the PostgreSQL log file. For more information, see Conflict logging for details. A server restart is required for changes to take effect. |
| pgactive.synchronous\$1commit | `boolean` | off | Determines the commit behavior for pgactive apply workers. When disabled(off), apply workers perform asynchronous commits, which improves PostgreSQL throughput during apply operations but delays replay confirmations to the upstream. Setting it to `off` is always safe and won't cause transaction loss or skipping. This setting only affects the timing of disk flushes on the downstream node and when confirmations are sent upstream. The system delays sending replay flush confirmations until commits are flushed to disk through unrelated operations like checkpoints or periodic work. However, if the upstream has the downstream listed in `synchronous_standby_names`, setting it to `off` causes synchronous commits on the upstream to take longer to report success to the client. In this case, set the parameter to `on`. Even when this parameter is set to `on` with nodes listed in `synchronous_standby_names`, replication conflicts can still occur in active-active configurations. This is because the system lacks inter-node locking and global snapshot management, allowing concurrent transactions on different nodes to modify the same tuple. Additionally, transactions only begin replication after committing on the upstream node. Enabling synchronous commit doesn't transform the pgactive extension into an always-consistent system. |
| pgactive.temp\$1dump\$1directory | `string` | – | Defines the temporary storage path required for database cloning operations during initial setup. This directory must be writable by the postgres user and have sufficient storage space to contain a complete database dump. The system uses this location only during initial database setup with logical copy operations. This parameter isn't used by the `pgactive_init_copy command`. |
| pgactive.max\$1ddl\$1lock\$1delay | `milliseconds` | `-1` | Specifies the maximum wait time for DDL lock before forcibly aborting concurrent write transactions. The default value is `-1`, which adopts the value set in `max_standby_streaming_delay`. This parameter accepts time units. For example, you can set it to 10s for 10 seconds. During this wait period, the system attempts to acquire DDL locks while waiting for ongoing write transactions to either commit or roll back. For more information, see the DDL Locking. |
| pgactive.ddl\$1lock\$1timeout | `milliseconds` | `-1` | Specifies how long a DDL lock attempt waits to obtain the lock. The default value is `-1`, which uses the value specified in lock\$1timeout. You can set this parameter using time units such as 10s for 10 seconds. This timer only controls the waiting period for obtaining a DDL lock. Once the system obtains the lock and begins the DDL operation, the timer stops. This parameter doesn't limit the total duration a DDL lock can be held or the overall DDL operation time. To control the total duration of the operation, use `statement_timeout` instead. For more information, see DDL Locking. |
| pgactive.debug\$1trace\$1ddl\$1locks\$1level | `boolean` | – | Overrides the default debug log level for DDL locking operations in the `pgactive` extension. When configured, this setting causes DDL lock-related messages to be emitted at the LOG debug level instead of their default level. Use this parameter to monitor DDL locking activity without enabling the verbose `DEBUG1` or `DEBUG2` log levels across your entire server. Available log levels, in increasing order of verbosity: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) For more information about monitoring options, see Monitoring global DDL locks. Changes to this setting take effect when you reload the configuration. You don't need to restart the server. |
## Additional pgactive extension parameters
The following table presents less frequently used and internal configuration options available for the `pgactive` extension.
| Parameter | Unit | Default | Description |
| --- | --- | --- | --- |
| pgactive.debug\$1apply\$1delay | `integer` | – | Sets an apply delay (in milliseconds) for configured connections that don't have an explicit apply delay in their `pgactive.pgactive_connections` entry. This delay is set during node creation or join time, and pgactive won't replay a transaction on peer nodes until at least the specified number of milliseconds have elapsed since it was committed. Primarily used to simulate high-latency networks in testing environments to make it easier to create conflicts. For example, with a 500ms delay on nodes A and B, you have at least 500ms to perform a conflicting insert on node B after inserting a value on node A. Requires a server reload or restart of apply workers to take effect. |
| pgactive.connectability\$1check\$1duration | `integer` | – | Specifies the duration (in seconds) that a database worker attempts to establish connections during failed attempts. The worker makes one connection attempt per second until it succeeds or reaches this timeout value. This setting is useful when the database engine starts before the worker is ready to establish connections. |
| pgactive.skip\$1ddl\$1replication | `boolean` | `on` | Controls how DDL changes are replicated or handled in Amazon RDS with `pgactive` enabled. When set to `on`, the node processes DDL changes like a non-pgcctive node. The following requirements apply when working with this parameter: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) You can modify this parameter in two ways with super user privileges: globally, locally (session level). Changing this parameter incorrectly can break your replication setups. |
| pgactive.do\$1not\$1replicate | `boolean` | – | This parameter is for internal use only. When you set this parameter in a transaction, the changes are not replicated to other nodes in your DB cluster. Changing this parameter incorrectly can break your replication setups. |
| pgactive.discard\$1mismatched\$1row\$1attributes | `boolean` | – | This parameter is intended for specialist use only. We recommend using this parameter only when troubleshooting specific replication issues. Use this parameter when: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) This setting overrides the following error message and allows data divergence to arise to let replication continue: `cannot right-pad mismatched attributes; attno %u is missing in local table and remote row has non-null, non-dropped value for this attribute` Changing this parameter incorrectly can break your replication setups. |
| pgactive.debug\$1trace\$1replay | `boolean` | – | When set to `on`, it emits a log message for each remote action that downstream apply workers process. The logs include: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) The logs also capture queued DDL commands and table drops.para> By default, the logs do not include row field contents. To include row values in the logs, you must recompile with the following flags enabled: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) Enabling this logging setting can impact performance. We recommend enabling it only when needed for troubleshooting. Changes to this setting take effect when you reload the configuration. You don't need to restart the server. |
| pgactive.extra\$1apply\$1connection\$1options | | – | You can configure connection parameters for all peer node connections with pgactive nodes. These parameters control settings such as keepalives and SSL modes. By default, pgactive uses the following connection parameters: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) To override the default parameters, use the following similar command: pgactive.extra\$1apply\$1connection\$1options = 'keepalives=0' Individual node connection strings take precedence over both these settings and pgactive's built-in connection options. For more information about connection string formats, see [libpq connection strings](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). We recommend keeping the default keepalive settings enabled. Only disable keepalives if you experience issues with large transactions completing over unreliable networks. We recommend keeping the default keepalive settings enabled. Only disable keepalives if you experience issues with large transactions completing over unreliable networks. Changes to this setting take effect when you reload the configuration. You don't need to restart the server. |
| pgactive.init\$1node\$1parallel\$1jobs (int) | | – | Specifies the number of parallel jobs that `pg_dump` and `pg_restore` can use during logical node joins with the `pgactive.pgactive_join_group` function. Changes to this setting take effect when you reload the configuration. You don't need to restart the server. |
| pgactive.max\$1nodes | `int` | 4 | Specifies the maximum number of nodes allowed in a pgactive extension group. The default value is 4 nodes. You must consider the following when setting the value of this parameter: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.pgactive.parameters.html) You can set this parameter in two ways: in the configuration file, using the `ALTER SYSTEM SET` command Default value for this parameter is `4`, meaning, there can be maximum of 4 nodes allowed in the `pgactive` extension group at any point of time. The change takes effect after you restart the server. |
| pgactive.permit\$1node\$1identifier\$1getter\$1function\$1creation | `boolean` | – | This parameter is intended for internal use only. When enabled, `pgactive` extension allows creation of pgactive node identifier getter function. |
# Understanding active-active conflicts
When you use pgactive in active-active mode, writing to the same tables from multiple nodes can create data conflicts. While some clustering systems use distributed locks to prevent concurrent access, pgactive takes an optimistic approach that's better suited for geographically distributed applications.
Some database clustering systems prevent concurrent data access by using distributed locks. While this approach works when servers are in close proximity, it doesn't support geographically distributed applications because it requires extremely low latency for good performance. Instead of using distributed locks (a pessimistic approach), the pgactive extension uses an optimistic approach. This means it:
+ Helps you avoid conflicts when possible.
+ Allows certain types of conflicts to occur.
+ Provides conflict resolution when conflicts happen.
This approach gives you more flexibility when building distributed applications.
## How conflicts happen
Inter-node conflicts arise from sequences of events that could not happen if all the involved transactions occurred concurrently on the same node. Because the nodes only exchange changes after transactions commit, each transaction is individually valid on the node it committed on but would not be valid if run on another node that has done other work in the meantime. Since pgactive apply essentially replays the transaction on the other nodes, the replay operation can fail if there is a conflict between a transaction being applied and a transaction that was committed on the receiving node.
The reason most conflicts can't happen when all transactions run on a single node is that PostgreSQL has inter-transaction communication mechanisms to prevent it, including:
+ UNIQUE indexes
+ SEQUENCEs
+ Row and relation locking
+ SERIALIZABLE dependency tracking
All of these mechanisms are ways to communicate between transactions to prevent undesirable concurrency issues
pgactive achieves low latency and handles network partitions well because it doesn't use a distributed transaction manager or lock manager. However, this means transactions on different nodes run in complete isolation from each other. While isolation typically improves database consistency, in this case, you need to reduce isolation to prevent conflicts.
## Types of conflicts
Conflicts that can occur include:
**Topics**
+ [
### PRIMARY KEY or UNIQUE conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict1)
+ [
### INSERT/INSERT conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict2)
+ [
### INSERTs that violate multiple UNIQUE constraints
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict3)
+ [
### UPDATE/UPDATE conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict4)
+ [
### UPDATE conflicts on the PRIMARY KEY
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict5)
+ [
### UPDATEs that violate multiple UNIQUE constraints
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict6)
+ [
### UPDATE/DELETE conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict7)
+ [
### INSERT/UPDATE conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict8)
+ [
### DELETE/DELETE conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict9)
+ [
### Foreign Key Constraint conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict10)
+ [
### Exclusion constraint conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict11)
+ [
### Global data conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict12)
+ [
### Lock conflicts and deadlock aborts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict13)
+ [
### Divergent conflicts
](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict14)
### PRIMARY KEY or UNIQUE conflicts
Row conflicts occur when multiple operations attempt to modify the same row key in ways not possible on a single node. These conflicts represent the most common type of data conflicts.
pgactive resolves detected conflicts through last-update-wins handling or your custom conflict handler.
Row conflicts include:
+ INSERT vs INSERT
+ INSERT vs UPDATE
+ UPDATE vs DELETE
+ INSERT vs DELETE
+ DELETE vs DELETE
+ INSERT vs DELETE
### INSERT/INSERT conflicts
This most common conflict occurs when INSERTs on two different nodes create a tuple with the same PRIMARY KEY values (or identical UNIQUE constraint values when no PRIMARY KEY exists).
pgactivelink resolves INSERT conflicts by using the timestamp from the originating host to keep the most recent tuple. You can override this default behavior with your custom conflict handler. While this process requires no special administrator action, be aware that pgactivelink discards one of the INSERT operations across all nodes. No automatic data merging occurs unless your custom handler implements it.
The pgactivelink can only resolve conflicts involving a single constraint violation. If an INSERT violates multiple UNIQUE constraints, you must implement additional conflict resolution strategies.
### INSERTs that violate multiple UNIQUE constraints
An INSERT/INSERT conflict can violate multiple UNIQUE constraints, including the PRIMARY KEY. pgactivelink can only handle conflicts that involve a single UNIQUE constraint. When conflicts violate multiple UNIQUE constraints, the apply worker fails and returns the following error:
`multiple unique constraints violated by remotely INSERTed tuple.`
In older versions, this situation generated a 'diverging uniqueness conflict' error instead.
To resolve these conflicts, you must take manual action. Either DELETE the conflicting local tuples or UPDATE them to remove conflicts with the new remote tuple. Be aware that you might need to address multiple conflicting tuples. Currently, pgactivelink provides no built-in functionality to ignore, discard, or merge tuples that violate multiple unique constraints.
**Note**
For more information, see UPDATEs that violate multiple UNIQUE constraints.
### UPDATE/UPDATE conflicts
This conflict occurs when two nodes concurrently modify the same tuple without changing its PRIMARY KEY. pgactivelink resolves these conflicts using last-update-wins logic or your custom conflict handler, if defined. A PRIMARY KEY is essential for tuple matching and conflict resolution. For tables without a PRIMARY KEY, pgactivelink rejects UPDATE operations with the following error:
`Cannot run UPDATE or DELETE on table (tablename) because it does not have a primary key.`
### UPDATE conflicts on the PRIMARY KEY
pgactive has limitations when handling PRIMARY KEY updates. While you can perform UPDATE operation on a PRIMARY KEY, pgactive can't automatically resolve conflicts using last-update-wins logic for these operations. You must ensure that your PRIMARY KEY updates don't conflict with existing values. If conflicts occur during PRIMARY KEY updates, they become divergent conflicts that require your manual intervention. For more information about handling these situations, see [Divergent conflicts](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict14).
### UPDATEs that violate multiple UNIQUE constraints
pgactivelink cannot apply last-update-wins conflict resolution when an incoming UPDATE violates multiple UNIQUE constraints or PRIMARY KEY values. This behavior is similar to INSERT operations with multiple constraint violations. These situations create divergent conflicts that require your manual intervention. For more information, see [Divergent conflicts](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict14).
### UPDATE/DELETE conflicts
This conflicts occur when one node UPDATEs a row while another node simultaneously DELETEs it. In this case a UPDATE/DELETE conflict occurs on replay. The resolution is to discard any UPDATE that arrives after a DELETE, unless your custom conflict handler specifies otherwise.
pgactivelink requires a PRIMARY KEY to match tuples and resolve conflicts. For tables without a PRIMARY KEY, it rejects DELETE operations with the following error:
`Cannot run UPDATE or DELETE on table (tablename) because it does not have a primary key.`
**Note**
pgactivelink cannot distinguish between UPDATE/DELETE and INSERT/UPDATE conflicts. In both cases, an UPDATE affects a nonexistent row. Due to asynchronous replication and lack of replay ordering between nodes, pgactivelink can't determine if the UPDATE is for a new row (INSERT not yet received) or a deleted row. In both scenarios, pgactivelink discards the UPDATE.
### INSERT/UPDATE conflicts
This conflict can occur in multi-node environments. It happens when one node INSERTs a row, a second node UPDATEs it, and a third node receives the UPDATE before the original INSERT. By default, pgactivelink resolves these conflicts by discarding the UPDATE, unless your custom conflict trigger specifies otherwise. Be aware that this resolution method can result in data inconsistencies across nodes. For more information about similar scenarios and their handling, see [UPDATE/DELETE conflicts](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict7).
### DELETE/DELETE conflicts
This conflict occurs when two different nodes concurrently delete the same tuple. pgactivelink considers these conflicts harmless because both DELETE operations have the same end result. In this scenario, pgactivelink safely ignores one of the DELETE operations without affecting data consistency.
### Foreign Key Constraint conflicts
FOREIGN KEY constraints can cause conflicts when applying remote transactions to existing local data. These conflicts typically occur when transactions are applied in a different sequence than their logical order on the originating nodes.
By default, pgactive applies changes with session\$1replication\$1role as `replica`, which bypasses foreign key checks during replication. In active-active configurations, this can lead to foreign key violations. Most violations are temporary and resolve once replication catches up. However, dangling foreign keys can occur because pgactive doesn't support cross-node row locking.
This behavior is inherent to partition-tolerant asynchronous active-active systems. For example, node A might insert a new child row while node B simultaneously deletes its parent row. The system can't prevent this type of concurrent modification across nodes.
To minimize foreign key conflicts, we recommend the following:
+ Limit foreign key relationships to closely related entities.
+ Modify related entities from a single node when possible.
+ Choose entities that rarely require modification.
+ Implement application-level concurrency control for modifications.
### Exclusion constraint conflicts
pgactive link doesn’t support exclusion constraints and restricts their creation.
**Note**
If you convert an existing standalone database to a pgactivelink database, manually drop all exclusion constraints.
In a distributed asynchronous system, it's not possible to guarantee that no set of rows violates the constraint. This is because all transactions on different nodes are fully isolated. Exclusion constraints can lead to replay deadlocks, where replay can't progress from any node to another due to exclusion constraint violations.
If you force pgactive Link to create an exclusion constraint, or if you don't drop existing ones when converting a standalone database to pgactive Link, replication is likely to break. To restore replication progress, remove or alter the local tuples that conflict with an incoming remote tuple so that the remote transaction can be applied.
### Global data conflicts
When using pgactivelink, conflicts can occur when nodes have different global PostgreSQL system-wide data, such as roles. These conflicts can cause operations—primarily DDL—to succeed and commit on one node but fail to apply to other nodes.
If a user exists on one node but not another, replication issues can occur:
+ Node1 has a user named `fred`, but this user doesn't exist on Node2
+ When `fred` creates a table on Node1, the table is replicated with `fred` as the owner
+ When this DDL command is applied to Node2, it fails because user `fred` doesn't exist
+ This failure generates an ERROR in the PostgreSQL logs on Node2 and increments the `pgactive.pgactive_stats.nr_rollbacks` counter
**Resolution:** Create the user `fred` on Node2. The user doesn't need identical permissions but must exist on both nodes.
If a table exists on one node but not another, data modification operations will fail:
+ Node1 has a table named `foo` that doesn't exist on Node2
+ Any DML operations on the `foo` table on Node1 will fail when replicated to Node2
**Resolution:** Create the table `foo` on Node2 with the same structure.
**Note**
pgactivelink doesn't currently replicate CREATE USER commands or DDL operations. DDL replication is planned for a future release.
### Lock conflicts and deadlock aborts
Because pgactive apply processes operate like normal user sessions, they follow standard row and table locking rules. This can result in pgactivelink apply processes waiting on locks held by user transactions or by other apply processes.
The following types of locks can affect apply processes:
+ Explicit table-level locking (LOCK TABLE ...) by user sessions
+ Explicit row-level locking (SELECT ... FOR UPDATE/FOR SHARE) by user sessions
+ Locking from foreign keys
+ Implicit locking due to row UPDATEs, INSERTs, or DELETEs, either from local activity or apply from other servers
Deadlocks can occur between:
+ A pgactivelink apply process and a user transaction
+ Two apply processes
When deadlocks occur, PostgreSQL's deadlock detector terminates one of the problem transactions. If the pgactivelink apply worker's process is terminated, it automatically retries and typically succeeds.
**Note**
These issues are temporary and generally don't require administrator intervention. If an apply process is blocked for an extended period by a lock on an idle user session, you can terminate the user session to resume replication. This situation is similar to when a user holds a long lock that affects another user session.
To identify locking-related replay delays, enable the `log_lock_waits` facility in PostgreSQL.
### Divergent conflicts
Divergent conflicts occur when data that should be identical across nodes differs unexpectedly. While these conflicts shouldn't happen, not all can be reliably prevented in the current implementation.
**Note**
Modifying a row's PRIMARY KEY can cause divergent conflicts if another node changes the same row's key before all nodes process the change. Avoid changing primary keys, or restrict changes to one designated node. For more information, see [UPDATE conflicts on the PRIMARY KEY](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflict5).
Divergent conflicts involving row data typically require administrator intervention. To resolve these conflicts, you must manually adjust data on one node to match another while temporarily disabling replication using `pgactive.pgactive_do_not_replicate`. These conflicts shouldn't occur when you use pgactive as documented and avoid settings or functions marked as unsafe.
As an administrator, you must manually resolve these conflicts. Depending on the conflict type, you'll need to use advanced options like `pgactive.pgactive_do_not_replicate`. Use these options with caution, as improper use can worsen the situation. Due to the variety of possible conflicts, we can't provide universal resolution instructions.
Divergent conflicts occur when data that should be identical across different nodes unexpectedly differs. While these conflicts shouldn't happen, not all such conflicts can be reliably prevented in the current implementation.
## Avoiding or tolerating conflicts
In most cases, you can use appropriate application design to avoid conflicts or make your application tolerant of conflicts.
Conflicts only occur when simultaneous operations happen on multiple nodes. To avoid conflicts:
+ Write to only one node
+ Write to independent database subsets on each node (for example, assign each node a separate schema)
For INSERT vs INSERT conflicts, use Global sequences to prevent conflicts entirely.
If conflicts are not acceptable for your use case, consider implementing distributed locking at the application level. Often, the best approach is to design your application to work with pgactive's conflict resolution mechanisms rather than trying to prevent all conflicts. For more information, see [Types of conflicts](#Appendix.PostgreSQL.CommonDBATasks.pgactive.actact.conflicttypes).
## Conflict logging
pgactivelink logs conflict incidents in the `pgactive.pgactive_conflict_history` table to help you diagnose and handle active-active conflicts. Conflict logging to this table only occurs when you set `pgactive.log_conflicts_to_table` to true. The pgactive extension also logs conflicts to the PostgreSQL log file when log\$1min\$1messages is set to `LOG` or `lower`, regardless of the `pgactive.log_conflicts_to_table` setting.
Use the conflict history table to:
+ Measure how frequently your application creates conflicts
+ Identify where conflicts occur
+ Improve your application to reduce conflict rates
+ Detect cases where conflict resolutions don't produce desired results
+ Determine where you need user-defined conflict triggers or application design changes
For row conflicts, you can optionally log row values. This is controlled by the `pgactive.log_conflicts_to_table` setting. Note that:
+ This is a global database-wide option
+ There is no per-table control over row value logging
+ No limits are applied to field numbers, array elements, or field lengths
+ Enabling this feature may not be advisable if you work with multi-megabyte rows that might trigger conflicts
Since the conflict history table contains data from every table in the database (each with potentially different schemas), logged row values are stored as JSON fields. The JSON is created using `row_to_json`, similar to calling it directly from SQL. PostgreSQL doesn't provide a `json_to_row` function, so you'll need table-specific code (in PL/pgSQL, PL/Python, PL/Perl, etc.) to reconstruct a composite-typed tuple from the logged JSON.
**Note**
Support for user-defined conflicts is planned as a future extension feature.
# Understanding the pgactive schema
The pgactive schema manages active-active replication in RDS for PostgreSQL. This schema contains tables that store replication configuration and status information.
**Note**
The pgactive schema is evolving and subject to change. Don't modify the data in these tables directly.
The key tables in the pgactive schema include:
+ `pgactive_nodes` – Stores information about nodes in the active-active replication group.
+ `pgactive_connections` – Stores connection details for each node.
## pgactive\$1nodes
The pgactive\$1nodes stores information about the nodes participating in the active-active replication group.
| Column | Type | Collation | Nullable | Default |
| --- | --- | --- | --- | --- |
| node\$1sysid | text | – | not null | – |
| node\$1timeline | oid | – | not null | – |
| node\$1dboid | oid | – | not null | – |
| node\$1status | char | – | not null | – |
| node\$1name | text | – | not null | – |
| node\$1dsn | text | – | not null | – |
| node\$1init\$1from\$1dsn | text | – | not null | – |
| node\$1read\$1only | boolean | – | – | false |
| node\$1seq\$1id | smallint | – | not null | – |
**node\$1sysid**
Unique ID for a node, generated during `pgactive_create_group` or `pgactive_join_group`
**node\$1status**
Readiness of the node:
+ **b** - beginning setup
+ **i** - initializing
+ **c** - catchup
+ **o** - creating outbound slots
+ **r** - ready
+ **k** - killed
This column doesn't indicate if a node is connected or disconnected.
**node\$1name**
User-provided unique node name.
**node\$1dsn**
Connection string or user mapping name
**node\$1init\$1from\$1dsn**
DSN from which this node was created.
## pgactive\$1connection
The pgactive\$1connections stores connection details for each node.
| Column | Type | Collation | Nullable | Default |
| --- | --- | --- | --- | --- |
| conn\$1sysid | text | none | not null | none |
| conn\$1timeline | oid | none | not null | none |
| conn\$1dboid | oid | none | not null | none |
| conn\$1dsn | text | none | not null | none |
| conn\$1apply\$1delay | integer | none | none | none |
| conn\$1replication\$1sets | text | none | none | none |
conn\$1sysid
Node identifier for the node this entry refers to.
conn\$1dsn
Same as pgactive.pgactive\$1nodes `node_dsn`.
conn\$1apply\$1delay
If set, milliseconds to wait before applying each transaction from the remote node. Mainly for debugging. If null, the global default applies.
## Working with replication sets
Replication sets determine which tables to include or exclude from replication operations. By default, all tables are replicated unless you specify otherwise using the following functions:
+ `pgactive_exclude_table_replication_set()` - Excludes specified tables from replication
+ `pgactive_include_table_replication_set()` - Includes specified tables in replication
**Note**
Before you configure replication sets, consider the following:
You can configure table inclusion or exclusion only after running `pgactive_create_group()` but before `pgactive_join_group()`.
After you use `pgactive_exclude_table_replication_set()`, you can't use `pgactive_include_table_replication_set()`.
After you use `pgactive_include_table_replication_set()`, you can't use `pgactive_exclude_table_replication_set()`.
The system handles newly created tables differently based on your initial configuration:
+ If you excluded tables: Any new tables created after `pgactive_join_group()` are automatically included in replication
+ If you included tables: Any new tables created after `pgactive_join_group()` are automatically excluded from replication.
To view the replication set configuration for a specific table, use the `pgactive.pgactive_get_table_replication_sets()` function.
# pgactive functions reference
Following, you can find a list of pgactive functions with their parameters, return values, and practical usage notes to help you effectively use them:
## get\$1last\$1applied\$1xact\$1info
Retrieves the last applied transaction information for a specified node.
**Arguments**
+ sysid (text) - timeline OID
+ dboid (OID)
**Return type**
It records the following:
+ last\$1applied\$1xact\$1id (OID)
+ last\$1applied\$1xact\$1committs (timestamp with time zone)
+ last\$1applied\$1xact\$1at (timestamp with time zone)
**Usage notes**
Use this function to retrieve the last applied transaction information for a specified node.
## pgactive\$1apply\$1pause
Pauses the replication apply process.
**Arguments**
None
**Return type**
boolean
**Usage notes**
Call this function to pause the replication apply process.
## pgactive\$1apply\$1resume
Resumes the replication apply process.
**Arguments**
None
**Return type**
void
**Usage notes**
Call this function to resume the replication apply process.
## pgactive\$1is\$1apply\$1paused
Checks if replication apply is currently paused.
**Arguments**
None
**Return type**
boolean
**Usage notes**
Use this function to check if replication apply is currently paused.
## pgactive\$1create\$1group
Creates a pgactive group by converting a standalone database into the initial node.
**Arguments**
+ node\$1name (text)
+ node\$1dsn (text)
+ apply\$1delay integer DEFAULT NULL::integer - replication\$1sets text[] DEFAULT ARRAY[‘default’::text]
**Return type**
void
**Usage notes**
Creates a pgactive group by converting a standalone database into the initial node. The function performs sanity checks before transforming the node into a pgactive node. Before using this function, ensure that your PostgreSQL cluster has sufficient `max_worker_processes` available to support pgactive background workers.
## pgactive\$1detach\$1nodes
Removes specified nodes from the pgactive group.
**Arguments**
+ p\$1nodes (text[])
**Return type**
void
**Usage notes**
Use this function to remove specified nodes from the pgactive group.
## pgactive\$1exclude\$1table\$1replication\$1set
Excludes a specific table from replication.
**Arguments**
+ p\$1relation (regclass)
**Return type**
void
**Usage notes**
Use this function to exclude a specific table from replication.
## pgactive\$1get\$1replication\$1lag\$1info
Retrieves detailed replication lag information, including node details, WAL status, and LSN values.
**Arguments**
None
**Return type**
SETOF record - node\$1name text - node\$1sysid text - application\$1name text - slot\$1name text - active boolean - active\$1pid integer - pending\$1wal\$1decoding bigint - Approximate size of WAL in bytes to be decoded on the sender node - pending\$1wal\$1to\$1apply bigint - Approximate size of WAL in bytes to be applied on receiving node - restart\$1lsn pg\$1lsn - confirmed\$1flush\$1lsn pg\$1lsn - sent\$1lsn pg\$1lsn - write\$1lsn pg\$1lsn - flush\$1lsn pg\$1lsn - replay\$1lsn pg\$1lsn
**Usage notes**
Call this function to retrieve replication lag information, including node details, WAL status, and LSN values.
## pgactive\$1get\$1stats
Retrieves pgactive replication statistics.
**Arguments**
None
**Return type**
SETOF record - rep\$1node\$1id oid - rilocalid oid - riremoteid text - nr\$1commit bigint - nr\$1rollback bigint - nr\$1insert bigint - nr\$1insert\$1conflict bigint - nr\$1update bigint - nr\$1update\$1conflict bigint - nr\$1delete bigint - nr\$1delete\$1conflict bigint - nr\$1disconnect bigint
**Usage notes**
Use this function to retrieve pgactive replication statistics.
## pgactive\$1get\$1table\$1replication\$1sets
Gets replication set configuration for a specific relation.
**Arguments**
+ relation (regclass)
**Return type**
SETOF record
**Usage notes**
Call this function to get replication set configuration for a specific relation.
## pgactive\$1include\$1table\$1replication\$1set
Includes a specific table in replication.
**Arguments**
+ p\$1relation (regclass)
**Return type**
void
**Usage notes**
Use this function to include a specific table in replication.
## pgactive\$1join\$1group
Adds a node to an existing pgactive group.
**Arguments**
+ node\$1name (text)
+ node\$1dsn (text)
+ join\$1using\$1dsn (text)
+ apply\$1delay (integer, optional)
+ replication\$1sets (text[], default: ['default'])
+ bypass\$1collation\$1check (boolean, default: false)
+ bypass\$1node\$1identifier\$1creation (boolean, default: false)
+ bypass\$1user\$1tables\$1check (boolean, default: false)
**Return type**
void
**Usage notes**
Call this function to add a node to an existing pgactive group. Ensure your PostgreSQL cluster has sufficient max\$1worker\$1processes for pgactive background workers.
## pgactive\$1remove
Removes all pgactive components from the local node.
**Arguments**
+ force (boolean, default: false)
**Return type**
void
**Usage notes**
Call this function to remove all pgactive components from the local node.
## pgactive\$1snowflake\$1id\$1nextval
Generates node-specific unique sequence values.
**Arguments**
+ regclass
**Return type**
bigint
**Usage notes**
Use this function to generate node-specific unique sequence values.
## pgactive\$1update\$1node\$1conninfo
Updates connection information for a pgactive node.
**Arguments**
+ node\$1name\$1to\$1update (text)
+ node\$1dsn\$1to\$1update (text)
**Return type**
void
**Usage notes**
Use this function to update connection information for a pgactive node.
## pgactive\$1wait\$1for\$1node\$1ready
Monitors the progress of group creation or joining operations.
**Arguments**
+ timeout (integer, default: 0)
+ progress\$1interval (integer, default: 60)
**Return type**
void
**Usage notes**
Call this function to monitor the progress of group creation or joining operations.
# Handling conflicts in active-active replication
The `pgactive` extension works per database and not per cluster. Each DB instance that uses `pgactive` is an independent instance and can accept data changes from any source. When a change is sent to a DB instance, PostgreSQL commits it locally and then uses `pgactive` to replicate the change asynchronously to other DB instances. When two PostgreSQL DB instances update the same record at nearly the same time, a conflict can occur.
The `pgactive` extension provides mechanisms for conflict detection and automatic resolution. It tracks the time stamp when the transaction was committed on both the DB instances and automatically applies the change with the latest time stamp. The `pgactive` extension also logs when a conflict occurs in the `pgactive.pgactive_conflict_history` table.
The `pgactive.pgactive_conflict_history` will keep growing. You may want to define a purging policy. This can be done by deleting some records on a regular basis or defining a partitioning scheme for this relation (and later detach, drop, truncate partitions of interest). To implement the purging policy on a regular basis, one option is to use the `pg_cron` extension. See the following information of an example for the `pg_cron` history table, [Scheduling maintenance with the PostgreSQL pg\$1cron extension](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_pg_cron.html).
# Handling sequences in active-active replication
An RDS for PostgreSQL DB instance with the `pgactive` extension uses two different sequence mechanisms to generate unique values.
**Global Sequences**
To use a global sequence, create a local sequence with the `CREATE SEQUENCE` statement. Use `pgactive.pgactive_snowflake_id_nextval(seqname)` instead of `usingnextval(seqname)` to get the next unique value of the sequence.
The following example creates a global sequence:
```
app=> CREATE TABLE gstest (
id bigint primary key,
parrot text
);
```
```
app=>CREATE SEQUENCE gstest_id_seq OWNED BY gstest.id;
```
```
app=> ALTER TABLE gstest \
ALTER COLUMN id SET DEFAULT \
pgactive.pgactive_snowflake_id_nextval('gstest_id_seq');
```
**Partitioned sequences**
In split-step or partitioned sequences, a normal PostgreSQL sequence is used on each node. Each sequence increments by the same amount and starts at different offsets. For example, with step 100, the node 1 generates sequence as 101, 201, 301, and so on and the node 2 generates sequence as 102, 202, 302, and so on. This scheme works well even if the nodes can't communicate for extended periods, but requires that the designer specify a maximum number of nodes when establishing the schema and requires per-node configuration. Mistakes can easily lead to overlapping sequences.
It is relatively simple to configure this approach with `pgactive` by creating the desired sequence on a node as follows:
```
CREATE TABLE some_table (generated_value bigint primary key);
```
```
app=> CREATE SEQUENCE some_seq INCREMENT 100 OWNED BY some_table.generated_value;
```
```
app=> ALTER TABLE some_table ALTER COLUMN generated_value SET DEFAULT nextval('some_seq');
```
Then call `setval` on each node to give a different offset starting value as follows.
```
app=>
-- On node 1
SELECT setval('some_seq', 1);
-- On node 2
SELECT setval('some_seq', 2);
```
# Reducing bloat in tables and indexes with the pg\$1repack extension
You can use the `pg_repack` extension to remove bloat from tables and indexes as an alternative to `VACUUM FULL`. This extension is supported on RDS for PostgreSQL versions 9.6.3 and higher. For more information on the `pg_repack` extension and the full table repack, see the [GitHub project documentation](https://reorg.github.io/pg_repack/).
Unlike `VACUUM FULL`, the `pg_repack` extension requires an exclusive lock (AccessExclusiveLock) only for a short period of time during the table rebuild operation in the following cases:
+ Initial creation of log table – A log table is created to record changes that occur during initial copy of the data, as shown in the following example:
```
postgres=>\dt+ repack.log_*
List of relations
-[ RECORD 1 ]-+----------
Schema | repack
Name | log_16490
Type | table
Owner | postgres
Persistence | permanent
Access method | heap
Size | 65 MB
Description |
```
+ Final swap-and-drop phase.
For the rest of the rebuild operation, it only needs an `ACCESS SHARE` lock on the original table to copy rows from it to the new table. This helps the INSERT, UPDATE, and DELETE operations to proceed as usual.
## Recommendations
The following recommendations apply when you remove bloat from the tables and indexes using the `pg_repack` extension:
+ Perform repack during non-business hours or over a maintenance window to minimize its impact on performance of other database activities.
+ Closely monitor blocking sessions during the rebuild activity and ensure that there is no activity on the original table that could potentially block `pg_repack`, specifically during the final swap-and-drop phase when it needs an exclusive lock on the original table. For more information, see [Identifying what is blocking a query](https://repost.aws/knowledge-center/rds-aurora-postgresql-query-blocked).
When you see a blocking session, you can terminate it using the following command after careful consideration. This helps in the continuation of `pg_repack` to finish the rebuild:
```
SELECT pg_terminate_backend(pid);
```
+ While applying the accrued changes from the `pg_repack's` log table on systems with a very high transaction rate, the apply process might not be able to keep up with the rate of changes. In such cases, `pg_repack` would not be able to complete the apply process. For more information, see [Monitoring the new table during the repack](#Appendix.PostgreSQL.CommonDBATasks.pg_repack.Monitoring). If indexes are severely bloated, an alternative solution is to perform an index only repack. This also helps VACUUM's index cleanup cycles to finish faster.
You can skip the index cleanup phase using manual VACUUM from PostgreSQL version 12, and it is skipped automatically during emergency autovacuum from PostgreSQL version 14. This helps VACUUM complete faster without removing the index bloat and is only meant for emergency situations such as preventing wraparound VACUUM. For more information, see [ Avoiding bloat in indexes](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.diag-table-ind-bloat.html#AuroraPostgreSQL.diag-table-ind-bloat.AvoidinginIndexes) in the Amazon Aurora User Guide.
## Pre-requisites
+ The table must have PRIMARY KEY or not-null UNIQUE constraint.
+ The extension version must be the same for both the client and the server.
+ Ensure that the RDS instance has more `FreeStorageSpace` than the total size of the table without the bloat. As an example, consider the total size of the table including TOAST and indexes as 2TB, and total bloat in the table as 1TB. The required `FreeStorageSpace` must be more than value returned by the following calculation:
`2TB (Table size)` - `1TB (Table bloat)` = `1TB`
You can use the following query to check the total size of the table and use `pgstattuple` to derive bloat. For more information, see [Diagnosing table and index bloat](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.diag-table-ind-bloat.html) in the Amazon Aurora User Guide
```
SELECT pg_size_pretty(pg_total_relation_size('table_name')) AS total_table_size;
```
This space is reclaimed after the completion of the activity.
+ Ensure that the RDS instance has enough compute and IO capacity to handle the repack operation. You might consider to scale up the instance class for optimal balance of performance.
**To use the `pg_repack` extension**
1. Install the `pg_repack` extension on your RDS for PostgreSQL DB instance by running the following command.
```
CREATE EXTENSION pg_repack;
```
1. Run the following commands to grant write access to temporary log tables created by `pg_repack`.
```
ALTER DEFAULT PRIVILEGES IN SCHEMA repack GRANT INSERT ON TABLES TO PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA repack GRANT USAGE, SELECT ON SEQUENCES TO PUBLIC;
```
1. Connect to the database using the `pg_repack` client utility. Use an account that has `rds_superuser` privileges. As an example, assume that `rds_test` role has `rds_superuser` privileges. The following syntax performs `pg_repack` for full tables including all the table indexes in the `postgres` database.
```
pg_repack -h db-instance-name.111122223333.aws-region.rds.amazonaws.com -U rds_test -k postgres
```
**Note**
You must connect using the -k option. The -a option is not supported.
The response from the `pg_repack` client provides information on the tables on the DB instance that are repacked.
```
INFO: repacking table "pgbench_tellers"
INFO: repacking table "pgbench_accounts"
INFO: repacking table "pgbench_branches"
```
1. The following syntax repacks a single table `orders` including indexes in `postgres` database.
```
pg_repack -h db-instance-name.111122223333.aws-region.rds.amazonaws.com -U rds_test --table orders -k postgres
```
The following syntax repacks only indexes for `orders` table in `postgres` database.
```
pg_repack -h db-instance-name.111122223333.aws-region.rds.amazonaws.com -U rds_test --table orders --only-indexes -k postgres
```
## Monitoring the new table during the repack
+ The size of the database is increased by the total size of the table minus bloat, until swap-and-drop phase of repack. You can monitor the growth rate of the database size, calculate the speed of the repack, and roughly estimate the time it takes to complete initial data transfer.
As an example, consider the total size of the table as 2TB, the size of the database as 4TB, and total bloat in the table as 1TB. The database total size value returned by the calculation at the end of the repack operation is the following:
`2TB (Table size)` \$1 `4 TB (Database size)` - `1TB (Table bloat)` = `5TB`
You can roughly estimate the speed of the repack operation by sampling the growth rate in bytes between two points in time. If the growth rate is 1GB per minute, it can take 1000 minutes or 16.6 hours approximately to complete the initial table build operation. In addition to the initial table build, `pg_repack` also needs to apply accrued changes. The time it takes depends on the rate of applying ongoing changes plus accrued changes.
**Note**
You can use `pgstattuple` extension to calculate the bloat in the table. For more information, see [pgstattuple ](https://www.postgresql.org/docs/current/pgstattuple.html).
+ The number of rows in the `pg_repack's` log table, under the repack schema represents the volume of changes pending to be applied to the new table after the initial load.
You can check the `pg_repack's` log table in `pg_stat_all_tables` to monitor the changes applied to the new table. `pg_stat_all_tables.n_live_tup` indicates the number of records that are pending to be applied to the new table. For more information, see [pg\$1stat\$1all\$1tables](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ALL-TABLES-VIEW).
```
postgres=>SELECT relname,n_live_tup FROM pg_stat_all_tables WHERE schemaname = 'repack' AND relname ILIKE '%log%';
-[ RECORD 1 ]---------
relname | log_16490
n_live_tup | 2000000
```
+ You can use the `pg_stat_statements` extension to find out the time taken by each step in the repack operation. This is helpful in preparation for applying the same repack operation in a production environment. You may adjust the `LIMIT` clause for extending the output further.
```
postgres=>SELECT
SUBSTR(query, 1, 100) query,
round((round(total_exec_time::numeric, 6) / 1000 / 60),4) total_exec_time_in_minutes
FROM
pg_stat_statements
WHERE
query ILIKE '%repack%'
ORDER BY
total_exec_time DESC LIMIT 5;
query | total_exec_time_in_minutes
-----------------------------------------------------------------------+----------------------------
CREATE UNIQUE INDEX index_16493 ON repack.table_16490 USING btree (a) | 6.8627
INSERT INTO repack.table_16490 SELECT a FROM ONLY public.t1 | 6.4150
SELECT repack.repack_apply($1, $2, $3, $4, $5, $6) | 0.5395
SELECT repack.repack_drop($1, $2) | 0.0004
SELECT repack.repack_swap($1) | 0.0004
(5 rows)
```
Repacking is completely an out-of-place operation so the original table is not impacted and we do not anticipate any unexpected challenges that require recovery of the original table. If repack fails unexpectedly, you must inspect the cause of the error and resolve it.
After the issue is resolved, drop and recreate the `pg_repack` extension in the database where the table exists, and retry the `pg_repack` step. In addition, the availability of compute resources and concurrent accessibility of the table plays a crucial role in the timely completion of the repack operation.
# Upgrading and using the PLV8 extension
PLV8 is a trusted Javascript language extension for PostgreSQL. You can use it for stored procedures, triggers, and other procedural code that's callable from SQL. This language extension is supported by all current releases of PostgreSQL.
If you use [PLV8](https://plv8.github.io/) and upgrade PostgreSQL to a new PLV8 version, you immediately take advantage of the new extension. Take the following steps to synchronize your catalog metadata with the new version of PLV8. These steps are optional, but we highly recommend that you complete them to avoid metadata mismatch warnings.
The upgrade process drops all your existing PLV8 functions. Thus, we recommend that you create a snapshot of your RDS for PostgreSQL DB instance before upgrading. For more information, see [Creating a DB snapshot for a Single-AZ DB instance for Amazon RDS](USER_CreateSnapshot.md).
**Important**
Starting with PostgreSQL version 18, Amazon RDS for PostgreSQL will deprecate the `plcoffee` and `plls` PostgreSQL extensions. We recommend that you stop using CoffeeScript and LiveScript in your applications to ensure you have an upgrade path for future engine version upgrades.
**To synchronize your catalog metadata with a new version of PLV8**
1. Verify that you need to update. To do this, run the following command while connected to your instance.
```
SELECT * FROM pg_available_extensions WHERE name IN ('plv8','plls','plcoffee');
```
If your results contain values for an installed version that is a lower number than the default version, continue with this procedure to update your extensions. For example, the following result set indicates that you should update.
```
name | default_version | installed_version | comment
--------+-----------------+-------------------+--------------------------------------------------
plls | 2.1.0 | 1.5.3 | PL/LiveScript (v8) trusted procedural language
plcoffee| 2.1.0 | 1.5.3 | PL/CoffeeScript (v8) trusted procedural language
plv8 | 2.1.0 | 1.5.3 | PL/JavaScript (v8) trusted procedural language
(3 rows)
```
1. Create a snapshot of your RDS for PostgreSQL DB instance if you haven't done so yet. You can continue with the following steps while the snapshot is being created.
1. Get a count of the number of PLV8 functions in your DB instance so you can validate that they are all in place after the upgrade. For example, the following SQL query returns the number of functions written in plv8, plcoffee, and plls.
```
SELECT proname, nspname, lanname
FROM pg_proc p, pg_language l, pg_namespace n
WHERE p.prolang = l.oid
AND n.oid = p.pronamespace
AND lanname IN ('plv8','plcoffee','plls');
```
1. Use pg\$1dump to create a schema-only dump file. For example, create a file on your client machine in the `/tmp` directory.
```
./pg_dump -Fc --schema-only -U master postgres >/tmp/test.dmp
```
This example uses the following options:
+ `-Fc` – Custom format
+ --schema-only – Dump only the commands necessary to create schema (functions in this case)
+ `-U` – The RDS master user name
+ `database` – The database name for our DB instance
For more information on pg\$1dump, see [pg\$1dump](https://www.postgresql.org/docs/current/static/app-pgdump.html ) in the PostgreSQL documentation.
1. Extract the "CREATE FUNCTION" DDL statement that is present in the dump file. The following example uses the `grep` command to extract the DDL statement that creates the functions and save them to a file. You use this in subsequent steps to recreate the functions.
```
./pg_restore -l /tmp/test.dmp | grep FUNCTION > /tmp/function_list
```
For more information on pg\$1restore, see [pg\$1restore](https://www.postgresql.org/docs/current/static/app-pgrestore.html) in the PostgreSQL documentation.
1. Drop the functions and extensions. The following example drops any PLV8 based objects. The cascade option ensures that any dependent are dropped.
```
DROP EXTENSION plv8 CASCADE;
```
If your PostgreSQL instance contains objects based on plcoffee or plls, repeat this step for those extensions.
1. Create the extensions. The following example creates the plv8, plcoffee, and plls extensions.
```
CREATE EXTENSION plv8;
CREATE EXTENSION plcoffee;
CREATE EXTENSION plls;
```
1. Create the functions using the dump file and "driver" file.
The following example recreates the functions that you extracted previously.
```
./pg_restore -U master -d postgres -Fc -L /tmp/function_list /tmp/test.dmp
```
1. Verify that all your functions have been recreated by using the following query.
```
SELECT * FROM pg_available_extensions WHERE name IN ('plv8','plls','plcoffee');
```
The PLV8 version 2 adds the following extra row to your result set:
```
proname | nspname | lanname
---------------+------------+----------
plv8_version | pg_catalog | plv8
```
# Using PL/Rust to write PostgreSQL functions in the Rust language
PL/Rust is a trusted Rust language extension for PostgreSQL. You can use it for stored procedures, functions, and other procedural code that's callable from SQL. The PL/Rust language extension is available in the following versions:
+ RDS for PostgreSQL 17.1 and higher 17 versions
+ RDS for PostgreSQL 16.1 and higher 16 versions
+ RDS for PostgreSQL 15.2-R2 and higher 15 versions
+ RDS for PostgreSQL 14.9 and higher 14 versions
+ RDS for PostgreSQL 13.12 and higher 13 versions
For more information, see [PL/Rust](https://github.com/tcdi/plrust#readme) on GitHub.
**Topics**
+ [
## Setting up PL/Rust
](#PL_Rust-setting-up)
+ [
## Creating functions with PL/Rust
](#PL_Rust-create-function)
+ [
## Using crates with PL/Rust
](#PL_Rust-crates)
+ [
## PL/Rust limitations
](#PL_Rust-limitations)
## Setting up PL/Rust
To install the plrust extension on your DB instance, add plrust to the `shared_preload_libraries` parameter in the DB parameter group associated with your DB instance. With the plrust extension installed, you can create functions.
To modify the `shared_preload_libraries` parameter, your DB instance must be associated with a custom parameter group. For information about creating a custom DB parameter group, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
You can install the plrust extension using the AWS Management Console or the AWS CLI.
The following steps assume that your DB instance is associated with a custom DB parameter group.
### Console
**Install the plrust extension in the `shared_preload_libraries` parameter**
Complete the following steps using an account that is a member of the `rds_superuser` group (role).
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose **Databases**.
1. Choose the name of your DB instance to display its details.
1. Open the **Configuration** tab for your DB instance and find the DB instance parameter group link.
1. Choose the link to open the custom parameters associated with your DB instance.
1. In the **Parameters** search field, type `shared_pre` to find the **`shared_preload_libraries`** parameter.
1. Choose **Edit parameters** to access the property values.
1. Add plrust to the list in the **Values** field. Use a comma to separate items in the list of values.
1. Reboot the DB instance so that your change to the `shared_preload_libraries` parameter takes effect. The initial reboot may require additional time to complete.
1. When the instance is available, verify that plrust has been initialized. Use `psql` to connect to the DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
```
Your output should look similar to the following:
```
shared_preload_libraries
--------------------------
rdsutils,plrust
(1 row)
```
### AWS CLI
**Install the plrust extension in the shared\$1preload\$1libraries parameter**
Complete the following steps using an account that is a member of the `rds_superuser` group (role).
1. Use the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) AWS CLI command to add plrust to the `shared_preload_libraries` parameter.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=shared_preload_libraries,ParameterValue=plrust,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the [reboot-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/reboot-db-instance) AWS CLI command to reboot the DB instance and initialize the plrust library. The initial reboot may require additional time to complete.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. When the instance is available, you can verify that plrust has been initialized. Use `psql` to connect to the DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
```
Your output should look similar to the following:
```
shared_preload_libraries
--------------------------
rdsutils,plrust
(1 row)
```
## Creating functions with PL/Rust
PL/Rust will compile the function as a dynamic library, load it, and execute it.
The following Rust function filters multiples out of an array.
```
postgres=> CREATE LANGUAGE plrust;
CREATE EXTENSION
```
```
CREATE OR REPLACE FUNCTION filter_multiples(a BIGINT[], multiple BIGINT) RETURNS BIGINT[]
IMMUTABLE STRICT
LANGUAGE PLRUST AS
$$
Ok(Some(a.into_iter().filter(|x| x.unwrap() % multiple != 0).collect()))
$$;
WITH gen_values AS (
SELECT ARRAY(SELECT * FROM generate_series(1,100)) as arr)
SELECT filter_multiples(arr, 3)
from gen_values;
```
## Using crates with PL/Rust
In RDS for PostgreSQL versions 16.3-R2 and higher, 15.7-R2 and higher 15 versions, 14.12-R2 and higher 14 versions, and 13.15-R2 and higher 13 versions, PL/Rust supports additional crates:
+ `url`
+ `regex`
+ `serde`
+ `serde_json`
In RDS for PostgreSQL versions 15.5-R2 and higher, 14.10-R2 and higher 14 versions, and 13.13-R2 and higher 13 versions, PL/Rust supports two additional crates:
+ `croaring-rs`
+ `num-bigint`
Starting with Amazon RDS for PostgreSQL versions 15.4, 14.9, and 13.12, PL/Rust supports the following crates:
+ `aes`
+ `ctr`
+ `rand`
Only the default features are supported for these crates. New RDS for PostgreSQL versions might contain updated versions of crates, and older versions of crates may no longer be supported.
Follow the best practices for performing a major version upgrade to test whether your PL/Rust functions are compatible with the new major version. For more information, see the blog [Best practices for upgrading Amazon RDS to major and minor versions of PostgreSQL](https://aws.amazon.com/blogs/database/best-practices-for-upgrading-amazon-rds-to-major-and-minor-versions-of-postgresql/) and [Upgrading the PostgreSQL DB engine for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.PostgreSQL.html) in the Amazon RDS User Guide.
Examples of using dependencies when creating a PL/Rust function are available at [Use dependencies](https://tcdi.github.io/plrust/use-plrust.html#use-dependencies).
## PL/Rust limitations
By default, database users can't use PL/Rust. To provide access to PL/Rust, connect as a user with rds\$1superuser privilege, and run the following command:
```
postgres=> GRANT USAGE ON LANGUAGE PLRUST TO user;
```
# Managing spatial data with the PostGIS extension
PostGIS is an extension to PostgreSQL for storing and managing spatial information. To learn more about PostGIS, see [PostGIS.net](https://postgis.net/).
Starting with version 10.5, PostgreSQL supports the libprotobuf 1.3.0 library used by PostGIS for working with map box vector tile data.
Setting up the PostGIS extension requires `rds_superuser` privileges. We recommend that you create a user (role) to manage the PostGIS extension and your spatial data. The PostGIS extension and its related components add thousands of functions to PostgreSQL. Consider creating the PostGIS extension in its own schema if that makes sense for your use case. The following example shows how to install the extension in its own database, but this isn't required.
**Topics**
+ [
## Step 1: Create a user (role) to manage the PostGIS extension
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.Connect)
+ [
## Step 2: Load the PostGIS extensions
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.LoadExtensions)
+ [
## Step 3: Transfer ownership of the extension schemas
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.TransferOwnership)
+ [
## Step 4: Transfer ownership of the PostGIS tables
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.TransferObjects)
+ [
## Step 5: Test the extensions
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.Test)
+ [
## Step 6: Upgrade the PostGIS extension
](#Appendix.PostgreSQL.CommonDBATasks.PostGIS.Update)
+ [PostGIS extension versions](#CHAP_PostgreSQL.Extensions.PostGIS)
+ [Upgrading PostGIS 2 to PostGIS 3](#PostgreSQL.Extensions.PostGIS.versions.upgrading.2-to-3)
## Step 1: Create a user (role) to manage the PostGIS extension
First, connect to your RDS for PostgreSQL DB instance as a user that has `rds_superuser` privileges. If you kept the default name when you set up your instance, you connect as `postgres`.
```
psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password
```
Create a separate role (user) to administer the PostGIS extension.
```
postgres=> CREATE ROLE gis_admin LOGIN PASSWORD 'change_me';
CREATE ROLE
```
Grant this role `rds_superuser` privileges, to allow the role to install the extension.
```
postgres=> GRANT rds_superuser TO gis_admin;
GRANT
```
Create a database to use for your PostGIS artifacts. This step is optional. Or you can create a schema in your user database for the PostGIS extensions, but this also isn't required.
```
postgres=> CREATE DATABASE lab_gis;
CREATE DATABASE
```
Give the `gis_admin` all privileges on the `lab_gis` database.
```
postgres=> GRANT ALL PRIVILEGES ON DATABASE lab_gis TO gis_admin;
GRANT
```
Exit the session and reconnect to your RDS for PostgreSQL DB instance as `gis_admin`.
```
postgres=> psql --host=111122223333.aws-region.rds.amazonaws.com --port=5432 --username=gis_admin --password --dbname=lab_gis
Password for user gis_admin:...
lab_gis=>
```
Continue setting up the extension as detailed in the next steps.
## Step 2: Load the PostGIS extensions
The PostGIS extension includes several related extensions that work together to provide geospatial functionality. Depending on your use case, you might not need all the extensions created in this step.
Use `CREATE EXTENSION` statements to load the PostGIS extensions.
```
CREATE EXTENSION postgis;
CREATE EXTENSION
CREATE EXTENSION postgis_raster;
CREATE EXTENSION
CREATE EXTENSION fuzzystrmatch;
CREATE EXTENSION
CREATE EXTENSION postgis_tiger_geocoder;
CREATE EXTENSION
CREATE EXTENSION postgis_topology;
CREATE EXTENSION
CREATE EXTENSION address_standardizer_data_us;
CREATE EXTENSION
```
You can verify the results by running the SQL query shown in the following example, which lists the extensions and their owners.
```
SELECT n.nspname AS "Name",
pg_catalog.pg_get_userbyid(n.nspowner) AS "Owner"
FROM pg_catalog.pg_namespace n
WHERE n.nspname !~ '^pg_' AND n.nspname <> 'information_schema'
ORDER BY 1;
List of schemas
Name | Owner
--------------+-----------
public | postgres
tiger | rdsadmin
tiger_data | rdsadmin
topology | rdsadmin
(4 rows)
```
## Step 3: Transfer ownership of the extension schemas
Use the ALTER SCHEMA statements to transfer ownership of the schemas to the `gis_admin` role.
```
ALTER SCHEMA tiger OWNER TO gis_admin;
ALTER SCHEMA
ALTER SCHEMA tiger_data OWNER TO gis_admin;
ALTER SCHEMA
ALTER SCHEMA topology OWNER TO gis_admin;
ALTER SCHEMA
```
You can confirm the ownership change by running the following SQL query. Or you can use the `\dn` metacommand from the psql command line.
```
SELECT n.nspname AS "Name",
pg_catalog.pg_get_userbyid(n.nspowner) AS "Owner"
FROM pg_catalog.pg_namespace n
WHERE n.nspname !~ '^pg_' AND n.nspname <> 'information_schema'
ORDER BY 1;
List of schemas
Name | Owner
--------------+---------------
public | postgres
tiger | gis_admin
tiger_data | gis_admin
topology | gis_admin
(4 rows)
```
## Step 4: Transfer ownership of the PostGIS tables
**Note**
Do not change ownership of the PostGIS functions. Proper operation and future upgrades of PostGIS require these functions to retain original ownership. For more information about PostGIS permissions, see [PostgreSQL Security](https://postgis.net/workshops/postgis-intro/security.html).
Use the following function to transfer ownership of the PostGIS tables to the `gis_admin` role. Run the following statement from the psql prompt to create the function.
```
CREATE FUNCTION exec(text) returns text language plpgsql volatile AS $f$ BEGIN EXECUTE $1; RETURN $1; END; $f$;
CREATE FUNCTION
```
Next, run the following query to run the `exec` function that in turn runs the statements and alters the permissions.
```
SELECT exec('ALTER TABLE ' || quote_ident(s.nspname) || '.' || quote_ident(s.relname) || ' OWNER TO gis_admin;')
FROM (
SELECT nspname, relname
FROM pg_class c JOIN pg_namespace n ON (c.relnamespace = n.oid)
WHERE nspname in ('tiger','topology') AND
relkind IN ('r','S','v') ORDER BY relkind = 'S')
s;
```
## Step 5: Test the extensions
To avoid needing to specify the schema name, add the `tiger` schema to your search path using the following command.
```
SET search_path=public,tiger;
SET
```
Test the `tiger` schema by using the following SELECT statement.
```
SELECT address, streetname, streettypeabbrev, zip
FROM normalize_address('1 Devonshire Place, Boston, MA 02109') AS na;
address | streetname | streettypeabbrev | zip
---------+------------+------------------+-------
1 | Devonshire | Pl | 02109
(1 row)
```
To learn more about this extension, see [Tiger Geocoder](https://postgis.net/docs/Extras.html#Tiger_Geocoder) in the PostGIS documentation.
Test access to the `topology` schema by using the following `SELECT` statement. This calls the `createtopology` function to register a new topology object (my\$1new\$1topo) with the specified spatial reference identifier (26986) and default tolerance (0.5). To learn more, see [CreateTopology](https://postgis.net/docs/CreateTopology.html) in the PostGIS documentation.
```
SELECT topology.createtopology('my_new_topo',26986,0.5);
createtopology
----------------
1
(1 row)
```
## Step 6: Upgrade the PostGIS extension
Each new release of PostgreSQL supports one or more versions of the PostGIS extension compatible with that release. Upgrading the PostgreSQL engine to a new version doesn't automatically upgrade the PostGIS extension. Before upgrading the PostgreSQL engine, you typically upgrade PostGIS to the newest available version for the current PostgreSQL version. For details, see [PostGIS extension versions](#CHAP_PostgreSQL.Extensions.PostGIS).
After the PostgreSQL engine upgrade, you then upgrade the PostGIS extension again, to the version supported for the newly upgraded PostgreSQL engine version. For more information about upgrading the PostgreSQL engine, see [How to perform a major version upgrade for RDS for PostgreSQL](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.Process.md).
You can check for available PostGIS extension version updates on your RDS for PostgreSQL DB instance at any time. To do so, run the following command. This function is available with PostGIS 2.5.0 and higher versions.
```
SELECT postGIS_extensions_upgrade();
```
If your application doesn't support the latest PostGIS version, you can install an older version of PostGIS that's available in your major version as follows.
```
CREATE EXTENSION postgis VERSION "2.5.5";
```
If you want to upgrade to a specific PostGIS version from an older version, you can also use the following command.
```
ALTER EXTENSION postgis UPDATE TO "2.5.5";
```
Depending on the version that you're upgrading from, you might need to use this function again. The result of the first run of the function determines if an additional upgrade function is needed. For example, this is the case for upgrading from PostGIS 2 to PostGIS 3. For more information, see [Upgrading PostGIS 2 to PostGIS 3](#PostgreSQL.Extensions.PostGIS.versions.upgrading.2-to-3).
If you upgraded this extension to prepare for a major version upgrade of the PostgreSQL engine, you can continue with other preliminary tasks. For more information, see [How to perform a major version upgrade for RDS for PostgreSQL](USER_UpgradeDBInstance.PostgreSQL.MajorVersion.Process.md).
## PostGIS extension versions
We recommend that you install the versions of all extensions such as PostGIS as listed in [Extension versions for Amazon RDS for PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html) in the *Amazon RDS for PostgreSQL Release Notes.* To get a list of versions that are available in your release, use the following command.
```
SELECT * FROM pg_available_extension_versions WHERE name='postgis';
```
You can find version information in the following sections in the *Amazon RDS for PostgreSQL Release Notes*:
+ [ PostgreSQL version 16 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-16x)
+ [ PostgreSQL version 15 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-15x)
+ [ PostgreSQL version 14 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-14x)
+ [ PostgreSQL version 13 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-13x)
+ [ PostgreSQL version 12 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-12x)
+ [ PostgreSQL version 11 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-11x)
+ [ PostgreSQL version 10 extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-101x)
+ [ PostgreSQL version 9.6.x extensions supported on Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-extensions.html#postgresql-extensions-96x)
## Upgrading PostGIS 2 to PostGIS 3
Starting with version 3.0, the PostGIS raster functionality is now a separate extension, `postgis_raster`. This extension has its own installation and upgrade path. This removes dozens of functions, data types, and other artifacts required for raster image processing from the core `postgis` extension. That means that if your use case doesn't require raster processing, you don't need to install the `postgis_raster` extension.
In the following upgrade example, the first upgrade command extracts raster functionality into the `postgis_raster` extension. A second upgrade command is then required to upgrade `postgis_raster` to the new version.
**To upgrade from PostGIS 2 to PostGIS 3**
1. Identify the default version of PostGIS that's available to the PostgreSQL version on your RDS for PostgreSQL DB instance. To do so, run the following query.
```
SELECT * FROM pg_available_extensions
WHERE default_version > installed_version;
name | default_version | installed_version | comment
---------+-----------------+-------------------+------------------------------------------------------------
postgis | 3.1.4 | 2.3.7 | PostGIS geometry and geography spatial types and functions
(1 row)
```
1. Identify the versions of PostGIS installed in each database on your RDS for PostgreSQL DB instance. In other words, query each user database as follows.
```
SELECT
e.extname AS "Name",
e.extversion AS "Version",
n.nspname AS "Schema",
c.description AS "Description"
FROM
pg_catalog.pg_extension e
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = e.extnamespace
LEFT JOIN pg_catalog.pg_description c ON c.objoid = e.oid
AND c.classoid = 'pg_catalog.pg_extension'::pg_catalog.regclass
WHERE
e.extname LIKE '%postgis%'
ORDER BY
1;
Name | Version | Schema | Description
---------+---------+--------+---------------------------------------------------------------------
postgis | 2.3.7 | public | PostGIS geometry, geography, and raster spatial types and functions
(1 row)
```
This mismatch between the default version (PostGIS 3.1.4) and the installed version (PostGIS 2.3.7) means that you need to upgrade the PostGIS extension.
```
ALTER EXTENSION postgis UPDATE;
ALTER EXTENSION
WARNING: unpackaging raster
WARNING: PostGIS Raster functionality has been unpackaged
```
1. Run the following query to verify that the raster functionality is now in its own package.
```
SELECT
probin,
count(*)
FROM
pg_proc
WHERE
probin LIKE '%postgis%'
GROUP BY
probin;
probin | count
--------------------------+-------
$libdir/rtpostgis-2.3 | 107
$libdir/postgis-3 | 487
(2 rows)
```
The output shows that there's still a difference between versions. The PostGIS functions are version 3 (postgis-3), while the raster functions (rtpostgis) are version 2 (rtpostgis-2.3). To complete the upgrade, you run the upgrade command again, as follows.
```
postgres=> SELECT postgis_extensions_upgrade();
```
You can safely ignore the warning messages. Run the following query again to verify that the upgrade is complete. The upgrade is complete when PostGIS and all related extensions aren't marked as needing upgrade.
```
SELECT postgis_full_version();
```
1. Use the following query to see the completed upgrade process and the separately packaged extensions, and verify that their versions match.
```
SELECT
e.extname AS "Name",
e.extversion AS "Version",
n.nspname AS "Schema",
c.description AS "Description"
FROM
pg_catalog.pg_extension e
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = e.extnamespace
LEFT JOIN pg_catalog.pg_description c ON c.objoid = e.oid
AND c.classoid = 'pg_catalog.pg_extension'::pg_catalog.regclass
WHERE
e.extname LIKE '%postgis%'
ORDER BY
1;
Name | Version | Schema | Description
----------------+---------+--------+---------------------------------------------------------------------
postgis | 3.1.5 | public | PostGIS geometry, geography, and raster spatial types and functions
postgis_raster | 3.1.5 | public | PostGIS raster types and functions
(2 rows)
```
The output shows that the PostGIS 2 extension was upgraded to PostGIS 3, and both `postgis` and the now separate `postgis_raster` extension are version 3.1.5.
After this upgrade completes, if you don't plan to use the raster functionality, you can drop the extension as follows.
```
DROP EXTENSION postgis_raster;
```
# Working with the supported foreign data wrappers for Amazon RDS for PostgreSQL
A foreign data wrapper (FDW) is a specific type of extension that provides access to external data. For example, the `oracle_fdw` extension allows your RDS for PostgreSQL DB cluster to work with Oracle databases. As another example, by using the PostgreSQL native `postgres_fdw` extension you can access data stored in PostgreSQL DB instances external to your RDS for PostgreSQL DB instance.
Following, you can find information about several supported PostgreSQL foreign data wrappers.
**Topics**
+ [
# Using the log\$1fdw extension to access the DB log using SQL
](CHAP_PostgreSQL.Extensions.log_fdw.md)
+ [
# Using the postgres\$1fdw extension to access external data
](postgresql-commondbatasks-fdw.md)
+ [
# Working with MySQL databases by using the mysql\$1fdw extension
](postgresql-mysql-fdw.md)
+ [
# Working with Oracle databases by using the oracle\$1fdw extension
](postgresql-oracle-fdw.md)
+ [
# Working with SQL Server databases by using the tds\$1fdw extension
](postgresql-tds-fdw.md)
# Using the log\$1fdw extension to access the DB log using SQL
RDS for PostgreSQL DB instance supports the `log_fdw` extension, which you can use to access your database engine log using a SQL interface. The `log_fdw` extension provides two functions that make it easy to create foreign tables for database logs:
+ `list_postgres_log_files` – Lists the files in the database log directory and the file size in bytes.
+ `create_foreign_table_for_log_file(table_name text, server_name text, log_file_name text)` – Builds a foreign table for the specified file in the current database.
All functions created by `log_fdw` are owned by `rds_superuser`. Members of the `rds_superuser` role can grant access to these functions to other database users.
By default, the log files are generated by Amazon RDS in `stderr` (standard error) format, as specified in `log_destination` parameter. There are only two options for this parameter, `stderr` and `csvlog` (comma-separated values, CSV). If you add the `csvlog` option to the parameter, Amazon RDS generates both `stderr` and `csvlog` logs. This can affect the storage capacity on your DB cluster, so you need to be aware of the other parameters that affect log handling. For more information, see [Setting the log destination (`stderr`, `csvlog`)](USER_LogAccess.Concepts.PostgreSQL.overview.parameter-groups.md#USER_LogAccess.Concepts.PostgreSQL.Log_Format).
One benefit of generating `csvlog` logs is that the `log_fdw` extension lets you build foreign tables with the data neatly split into several columns. To do this, your instance needs to be associated with a custom DB parameter group so that you can change the setting for `log_destination`. For more information about how to do so, see [Working with parameters on your RDS for PostgreSQL DB instance](Appendix.PostgreSQL.CommonDBATasks.Parameters.md).
The following example assumes that the `log_destination` parameter includes `cvslog`.
**To use the log\$1fdw extension**
1. Install the `log_fdw` extension.
```
postgres=> CREATE EXTENSION log_fdw;
CREATE EXTENSION
```
1. Create the log server as a foreign data wrapper.
```
postgres=> CREATE SERVER log_server FOREIGN DATA WRAPPER log_fdw;
CREATE SERVER
```
1. Select all from a list of log files.
```
postgres=> SELECT * FROM list_postgres_log_files() ORDER BY 1;
```
A sample response is as follows.
```
file_name | file_size_bytes
------------------------------+-----------------
postgresql.log.2023-08-09-22.csv | 1111
postgresql.log.2023-08-09-23.csv | 1172
postgresql.log.2023-08-10-00.csv | 1744
postgresql.log.2023-08-10-01.csv | 1102
(4 rows)
```
1. Create a table with a single 'log\$1entry' column for the selected file.
```
postgres=> SELECT create_foreign_table_for_log_file('my_postgres_error_log',
'log_server', 'postgresql.log.2023-08-09-22.csv');
```
The response provides no detail other than that the table now exists.
```
-----------------------------------
(1 row)
```
1. Select a sample of the log file. The following code retrieves the log time and error message description.
```
postgres=> SELECT log_time, message FROM my_postgres_error_log ORDER BY 1;
```
A sample response is as follows.
```
log_time | message
----------------------------------+---------------------------------------------------------------------------
Tue Aug 09 15:45:18.172 2023 PDT | ending log output to stderr
Tue Aug 09 15:45:18.175 2023 PDT | database system was interrupted; last known up at 2023-08-09 22:43:34 UTC
Tue Aug 09 15:45:18.223 2023 PDT | checkpoint record is at 0/90002E0
Tue Aug 09 15:45:18.223 2023 PDT | redo record is at 0/90002A8; shutdown FALSE
Tue Aug 09 15:45:18.223 2023 PDT | next transaction ID: 0/1879; next OID: 24578
Tue Aug 09 15:45:18.223 2023 PDT | next MultiXactId: 1; next MultiXactOffset: 0
Tue Aug 09 15:45:18.223 2023 PDT | oldest unfrozen transaction ID: 1822, in database 1
(7 rows)
```
# Using the postgres\$1fdw extension to access external data
You can access data in a table on a remote database server with the [postgres\$1fdw](https://www.postgresql.org/docs/current/static/postgres-fdw.html) extension. If you set up a remote connection from your PostgreSQL DB instance, access is also available to your read replica.
**To use postgres\$1fdw to access a remote database server**
1. Install the postgres\$1fdw extension.
```
CREATE EXTENSION postgres_fdw;
```
1. Create a foreign data server using CREATE SERVER.
```
CREATE SERVER foreign_server
FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (host 'xxx.xx.xxx.xx', port '5432', dbname 'foreign_db');
```
1. Create a user mapping to identify the role to be used on the remote server.
**Important**
To redact the password so that it doesn't appear in the logs, set `log_statement=none` at the session level. Setting at the parameter level doesn't redact the password.
```
CREATE USER MAPPING FOR local_user
SERVER foreign_server
OPTIONS (user 'foreign_user', password 'password');
```
1. Create a table that maps to the table on the remote server.
```
CREATE FOREIGN TABLE foreign_table (
id integer NOT NULL,
data text)
SERVER foreign_server
OPTIONS (schema_name 'some_schema', table_name 'some_table');
```
# Working with MySQL databases by using the mysql\$1fdw extension
To access a MySQL-compatible database from your RDS for PostgreSQL DB instance, you can install and use the `mysql_fdw` extension. This foreign data wrapper lets you work with RDS for MySQL, Aurora MySQL, MariaDB, and other MySQL-compatible databases. The connection from RDS for PostgreSQL DB instance to the MySQL database is encrypted on a best-effort basis, depending on the client and server configurations. However, you can enforce encryption if you like. For more information, see [Using encryption in transit with the extension](#postgresql-mysql-fdw.encryption-in-transit).
The `mysql_fdw` extension is supported on Amazon RDS for PostgreSQL version 14.2, 13.6, and higher releases. It supports selects, inserts, updates, and deletes from an RDS for PostgreSQL DB to tables on a MySQL-compatible database instance.
**Topics**
+ [
## Setting up your RDS for PostgreSQL DB to use the mysql\$1fdw extension
](#postgresql-mysql-fdw.setting-up)
+ [
## Example: Working with an RDS for MySQL database from RDS for PostgreSQL
](#postgresql-mysql-fdw.using-mysql_fdw)
+ [
## Using encryption in transit with the extension
](#postgresql-mysql-fdw.encryption-in-transit)
## Setting up your RDS for PostgreSQL DB to use the mysql\$1fdw extension
Setting up the `mysql_fdw` extension on your RDS for PostgreSQL DB instance involves loading the extension in your DB instance and then creating the connection point to the MySQL DB instance. For that task, you need to have the following details about the MySQL DB instance:
+ Hostname or endpoint. For an RDS for MySQL DB instance, you can find the endpoint by using the Console. Choose the Connectivity & security tab and look in the "Endpoint and port" section.
+ Port number. The default port number for MySQL is 3306.
+ Name of the database. The DB identifier.
You also need to provide access on the security group or the access control list (ACL) for the MySQL port, 3306. Both the RDS for PostgreSQL DB instance and the RDS for MySQL DB instance need access to port 3306. If access isn't configured correctly, when you try to connect to MySQL-compatible table you see an error message similar to the following:
```
ERROR: failed to connect to MySQL: Can't connect to MySQL server on 'hostname.aws-region.rds.amazonaws.com:3306' (110)
```
In the following procedure, you (as the `rds_superuser` account) create the foreign server. You then grant access to the foreign server to specific users. These users then create their own mappings to the appropriate MySQL user accounts to work with the MySQL DB instance.
**To use mysql\$1fdw to access a MySQL database server**
1. Connect to your PostgreSQL DB instance using an account that has the `rds_superuser` role. If you accepted the defaults when you created your RDS for PostgreSQL DB instance , the user name is `postgres`, and you can connect using the `psql` command line tool as follows:
```
psql --host=your-DB-instance.aws-region.rds.amazonaws.com --port=5432 --username=postgres –-password
```
1. Install the `mysql_fdw` extension as follows:
```
postgres=> CREATE EXTENSION mysql_fdw;
CREATE EXTENSION
```
After the extension is installed on your RDS for PostgreSQL DB instance , you set up the foreign server that provides the connection to a MySQL database.
**To create the foreign server**
Perform these tasks on the RDS for PostgreSQL DB instance . The steps assume that you're connected as a user with `rds_superuser` privileges, such as `postgres`.
1. Create a foreign server in the RDS for PostgreSQL DB instance :
```
postgres=> CREATE SERVER mysql-db FOREIGN DATA WRAPPER mysql_fdw OPTIONS (host 'db-name.111122223333.aws-region.rds.amazonaws.com', port '3306');
CREATE SERVER
```
1. Grant the appropriate users access to the foreign server. These should be non-administrator users, that is, users without the `rds_superuser` role.
```
postgres=> GRANT USAGE ON FOREIGN SERVER mysql-db to user1;
GRANT
```
PostgreSQL users create and manage their own connections to the MySQL database through the foreign server.
## Example: Working with an RDS for MySQL database from RDS for PostgreSQL
Suppose that you have a simple table on an RDS for PostgreSQL DB instance . Your RDS for PostgreSQL users want to query (`SELECT`), `INSERT`, `UPDATE`, and `DELETE` items on that table. Assume that the `mysql_fdw` extension was created on your RDS for PostgreSQL DB instance, as detailed in the preceding procedure. After you connect to the RDS for PostgreSQL DB instance as a user that has `rds_superuser` privileges, you can proceed with the following steps.
1. On the RDS for PostgreSQL DB instance, create a foreign server:
```
test=> CREATE SERVER mysqldb FOREIGN DATA WRAPPER mysql_fdw OPTIONS (host 'your-DB.aws-region.rds.amazonaws.com', port '3306');
CREATE SERVER
```
1. Grant usage to a user who doesn't have `rds_superuser` permissions, for example, `user1`:
```
test=> GRANT USAGE ON FOREIGN SERVER mysqldb TO user1;
GRANT
```
1. Connect as *user1*, and then create a mapping to the MySQL user:
```
test=> CREATE USER MAPPING FOR user1 SERVER mysqldb OPTIONS (username 'myuser', password 'mypassword');
CREATE USER MAPPING
```
1. Create a foreign table linked to the MySQL table:
```
test=> CREATE FOREIGN TABLE mytab (a int, b text) SERVER mysqldb OPTIONS (dbname 'test', table_name '');
CREATE FOREIGN TABLE
```
1. Run a simple query against the foreign table:
```
test=> SELECT * FROM mytab;
a | b
---+-------
1 | apple
(1 row)
```
1. You can add, change, and remove data from the MySQL table. For example:
```
test=> INSERT INTO mytab values (2, 'mango');
INSERT 0 1
```
Run the `SELECT` query again to see the results:
```
test=> SELECT * FROM mytab ORDER BY 1;
a | b
---+-------
1 | apple
2 | mango
(2 rows)
```
## Using encryption in transit with the extension
The connection to MySQL from RDS for PostgreSQL uses encryption in transit (TLS/SSL) by default. However, the connection falls back to non-encrypted when the client and server configuration differ. You can enforce encryption for all outgoing connections by specifying the `REQUIRE SSL` option on the RDS for MySQL user accounts. This same approach also works for MariaDB and Aurora MySQL user accounts.
For MySQL user accounts configured to `REQUIRE SSL`, the connection attempt fails if a secure connection can't be established.
To enforce encryption for existing MySQL database user accounts, you can use the `ALTER USER` command. The syntax varies, depending on the MySQL version, as shown in the following table. For more information, see [ALTER USER](https://dev.mysql.com/doc/refman/8.0/en/alter-user.html) in *MySQL Reference Manual*.
| MySQL 5.7, MySQL 8.0 | MySQL 5.6 |
| --- | --- |
| `ALTER USER 'user'@'%' REQUIRE SSL;` | `GRANT USAGE ON *.* to 'user'@'%' REQUIRE SSL;` |
For more information about the `mysql_fdw` extension, see the [mysql\$1fdw](https://github.com/EnterpriseDB/mysql_fdw) documentation.
# Working with Oracle databases by using the oracle\$1fdw extension
To access an Oracle database from your RDS for PostgreSQL DB instance you can install and use the `oracle_fdw` extension. This extension is a foreign data wrapper for Oracle databases. To learn more about this extension, see the [oracle\$1fdw](https://github.com/laurenz/oracle_fdw) documentation.
The `oracle_fdw` extension is supported on RDS for PostgreSQL 12.7, 13.3, and higher versions.
**Topics**
+ [
## Turning on the oracle\$1fdw extension
](#postgresql-oracle-fdw.enabling)
+ [
## Example: Using a foreign server linked to an Amazon RDS for Oracle database
](#postgresql-oracle-fdw.example)
+ [
## Working with encryption in transit
](#postgresql-oracle-fdw.encryption)
+ [
## Understanding the pg\$1user\$1mappings view and permissions
](#postgresql-oracle-fdw.permissions)
## Turning on the oracle\$1fdw extension
To use the oracle\$1fdw extension, perform the following procedure.
**To turn on the oracle\$1fdw extension**
+ Run the following command using an account that has `rds_superuser` permissions.
```
CREATE EXTENSION oracle_fdw;
```
## Example: Using a foreign server linked to an Amazon RDS for Oracle database
The following example shows the use of a foreign server linked to an Amazon RDS for Oracle database.
**To create a foreign server linked to an RDS for Oracle database**
1. Note the following on the RDS for Oracle DB instance:
+ Endpoint
+ Port
+ Database name
1. Create a foreign server.
```
test=> CREATE SERVER oradb FOREIGN DATA WRAPPER oracle_fdw OPTIONS (dbserver '//endpoint:port/DB_name');
CREATE SERVER
```
1. Grant usage to a user who doesn't have `rds_superuser` privileges, for example `user1`.
```
test=> GRANT USAGE ON FOREIGN SERVER oradb TO user1;
GRANT
```
1. Connect as `user1`, and create a mapping to an Oracle user.
```
test=> CREATE USER MAPPING FOR user1 SERVER oradb OPTIONS (user 'oracleuser', password 'mypassword');
CREATE USER MAPPING
```
1. Create a foreign table linked to an Oracle table.
```
test=> CREATE FOREIGN TABLE mytab (a int) SERVER oradb OPTIONS (table 'MYTABLE');
CREATE FOREIGN TABLE
```
1. Query the foreign table.
```
test=> SELECT * FROM mytab;
a
---
1
(1 row)
```
If the query reports the following error, check your security group and access control list (ACL) to make sure that both instances can communicate.
```
ERROR: connection for foreign table "mytab" cannot be established
DETAIL: ORA-12170: TNS:Connect timeout occurred
```
## Working with encryption in transit
PostgreSQL-to-Oracle encryption in transit is based on a combination of client and server configuration parameters. For an example using Oracle 21c, see [About the Values for Negotiating Encryption and Integrity](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbseg/configuring-network-data-encryption-and-integrity.html#GUID-3A2AF4AA-AE3E-446B-8F64-31C48F27A2B5) in the Oracle documentation. The client used for oracle\$1fdw on Amazon RDS is configured with `ACCEPTED`, meaning that the encryption depends on the Oracle database server configuration and it uses Oracle Security Library (libnnz) for encryption.
If your database is on RDS for Oracle, see [Oracle native network encryption](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.Options.NetworkEncryption.html) to configure the encryption.
## Understanding the pg\$1user\$1mappings view and permissions
The PostgreSQL catalog `pg_user_mapping` stores the mapping from an RDS for PostgreSQL user to the user on a foreign data (remote) server. Access to the catalog is restricted, but you use the `pg_user_mappings` view to see the mappings. In the following, you can find an example that shows how permissions apply with an example Oracle database, but this information applies more generally to any foreign data wrapper.
In the following output, you can find roles and permissions mapped to three different example users. Users `rdssu1` and `rdssu2` are members of the `rds_superuser` role, and `user1` isn't. The example uses the `psql` metacommand `\du` to list existing roles.
```
test=> \du
List of roles
Role name | Attributes | Member of
-----------------+------------------------------------------------------------+-------------------------------------------------------------
rdssu1 | | {rds_superuser}
rdssu2 | | {rds_superuser}
user1 | | {}
```
All users, including users that have `rds_superuser` privileges, are allowed to view their own user mappings (`umoptions`) in the `pg_user_mappings` table. As shown in the following example, when `rdssu1` tries to obtain all user mappings, an error is raised even though `rdssu1``rds_superuser` privileges:
```
test=> SELECT * FROM pg_user_mapping;
ERROR: permission denied for table pg_user_mapping
```
Following are some examples.
```
test=> SET SESSION AUTHORIZATION rdssu1;
SET
test=> SELECT * FROM pg_user_mappings;
umid | srvid | srvname | umuser | usename | umoptions
-------+-------+---------+--------+------------+----------------------------------
16414 | 16411 | oradb | 16412 | user1 |
16423 | 16411 | oradb | 16421 | rdssu1 | {user=oracleuser,password=mypwd}
16424 | 16411 | oradb | 16422 | rdssu2 |
(3 rows)
test=> SET SESSION AUTHORIZATION rdssu2;
SET
test=> SELECT * FROM pg_user_mappings;
umid | srvid | srvname | umuser | usename | umoptions
-------+-------+---------+--------+------------+----------------------------------
16414 | 16411 | oradb | 16412 | user1 |
16423 | 16411 | oradb | 16421 | rdssu1 |
16424 | 16411 | oradb | 16422 | rdssu2 | {user=oracleuser,password=mypwd}
(3 rows)
test=> SET SESSION AUTHORIZATION user1;
SET
test=> SELECT * FROM pg_user_mappings;
umid | srvid | srvname | umuser | usename | umoptions
-------+-------+---------+--------+------------+--------------------------------
16414 | 16411 | oradb | 16412 | user1 | {user=oracleuser,password=mypwd}
16423 | 16411 | oradb | 16421 | rdssu1 |
16424 | 16411 | oradb | 16422 | rdssu2 |
(3 rows)
```
Because of implementation differences between `information_schema._pg_user_mappings` and `pg_catalog.pg_user_mappings`, a manually created `rds_superuser` requires additional permissions to view passwords in `pg_catalog.pg_user_mappings`.
No additional permissions are required for an `rds_superuser` to view passwords in `information_schema._pg_user_mappings`.
Users who don't have the `rds_superuser` role can view passwords in `pg_user_mappings` only under the following conditions:
+ The current user is the user being mapped and owns the server or holds the `USAGE` privilege on it.
+ The current user is the server owner and the mapping is for `PUBLIC`.
# Working with SQL Server databases by using the tds\$1fdw extension
You can use the PostgreSQL `tds_fdw` extension to access databases that support the tabular data stream (TDS) protocol, such as Sybase and Microsoft SQL Server databases. This foreign data wrapper lets you connect from your RDS for PostgreSQL DB instance to databases that use the TDS protocol, including Amazon RDS for Microsoft SQL Server. For more information, see [tds-fdw/tds\$1fdw](https://github.com/tds-fdw/tds_fdw) documentation on GitHub.
The `tds_fdw` extension is supported on Amazon RDS for PostgreSQL version 14.2, 13.6, and higher releases.
## Setting up your Aurora PostgreSQL DB to use the tds\$1fdw extension
In the following procedures, you can find an example of setting up and using the `tds_fdw` with an RDS for PostgreSQL DB instance. Before you can connect to a SQL Server database using `tds_fdw`, you need to get the following details for the instance:
+ Hostname or endpoint. For an RDS for SQL Server DB instance, you can find the endpoint by using the Console. Choose the Connectivity & security tab and look in the "Endpoint and port" section.
+ Port number. The default port number for Microsoft SQL Server is 1433.
+ Name of the database. The DB identifier.
You also need to provide access on the security group or the access control list (ACL) for the SQL Server port, 1433. Both the RDS for PostgreSQL DB instance and the RDS for SQL Server DB instance need access to port 1433. If access isn't configured correctly, when you try to query the Microsoft SQL Server you see the following error message:
```
ERROR: DB-Library error: DB #: 20009, DB Msg: Unable to connect:
Adaptive Server is unavailable or does not exist (mssql2019.aws-region.rds.amazonaws.com), OS #: 0, OS Msg: Success, Level: 9
```
**To use tds\$1fdw to connect to a SQL Server database**
1. Connect to your PostgreSQL DB instance using an account that has the `rds_superuser` role:
```
psql --host=your-DB-instance.aws-region.rds.amazonaws.com --port=5432 --username=test –-password
```
1. Install the `tds_fdw` extension:
```
test=> CREATE EXTENSION tds_fdw;
CREATE EXTENSION
```
After the extension is installed on your RDS for PostgreSQL DB instance, you set up the foreign server.
**To create the foreign server**
Perform these tasks on the RDS for PostgreSQL DB instance using an account that has `rds_superuser` privileges.
1. Create a foreign server in the RDS for PostgreSQL DB instance:
```
test=> CREATE SERVER sqlserverdb FOREIGN DATA WRAPPER tds_fdw OPTIONS (servername 'mssql2019.aws-region.rds.amazonaws.com', port '1433', database 'tds_fdw_testing');
CREATE SERVER
```
To access non-ASCII data on the SQLServer side, create a server link with the character\$1set option in the RDS for PostgreSQL DB instance:
```
test=> CREATE SERVER sqlserverdb FOREIGN DATA WRAPPER tds_fdw OPTIONS (servername 'mssql2019.aws-region.rds.amazonaws.com', port '1433', database 'tds_fdw_testing', character_set 'UTF-8');
CREATE SERVER
```
1. Grant permissions to a user who doesn't have `rds_superuser` role privileges, for example, `user1`:
```
test=> GRANT USAGE ON FOREIGN SERVER sqlserverdb TO user1;
```
1. Connect as user1 and create a mapping to a SQL Server user:
```
test=> CREATE USER MAPPING FOR user1 SERVER sqlserverdb OPTIONS (username 'sqlserveruser', password 'password');
CREATE USER MAPPING
```
1. Create a foreign table linked to a SQL Server table:
```
test=> CREATE FOREIGN TABLE mytab (a int) SERVER sqlserverdb OPTIONS (table 'MYTABLE');
CREATE FOREIGN TABLE
```
1. Query the foreign table:
```
test=> SELECT * FROM mytab;
a
---
1
(1 row)
```
### Using encryption in transit for the connection
The connection from RDS for PostgreSQL to SQL Server uses encryption in transit (TLS/SSL) depending on the SQL Server database configuration. If the SQL Server isn't configured for encryption, the RDS for PostgreSQL client making the request to the SQL Server database falls back to unencrypted.
You can enforce encryption for the connection to RDS for SQL Server DB instances by setting the `rds.force_ssl` parameter. To learn how, see [Forcing connections to your DB instance to use SSL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Concepts.General.SSL.Using.html#SQLServer.Concepts.General.SSL.Forcing). For more information about SSL/TLS configuration for RDS for SQL Server, see [Using SSL with a Microsoft SQL Server DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Concepts.General.SSL.Using.html).
# Working with Trusted Language Extensions for PostgreSQL
Trusted Language Extensions for PostgreSQL is an open source development kit for building PostgreSQL extensions. It allows you to build high performance PostgreSQL extensions and safely run them on your RDS for PostgreSQL DB instance. By using Trusted Language Extensions (TLE) for PostgreSQL, you can create PostgreSQL extensions that follow the documented approach for extending PostgreSQL functionality. For more information, see [Packaging Related Objects into an Extension](https://www.postgresql.org/docs/current/extend-extensions.html) in the PostgreSQL documentation.
One key benefit of TLE is that you can use it in environments that don't provide access to the file system underlying the PostgreSQL instance. Previously, installing a new extension required access to the file system. TLE removes this constraint. It provides a development environment for creating new extensions for any PostgreSQL database, including those running on your RDS for PostgreSQL DB instances.
TLE is designed to prevent access to unsafe resources for the extensions that you create using TLE. Its runtime environment limits the impact of any extension defect to a single database connection. TLE also gives database administrators fine-grained control over who can install extensions, and it provides a permissions model for running them.
TLE is supported on the following RDS for PostgreSQL versions:
+ Version 18.1 and higher 18 versions
+ Version 17.1 and higher 17 versions
+ Version 16.1 and higher 16 versions
+ Version 15.2 and higher 15 versions
+ Version 14.5 and higher 14 versions
+ Version 13.12 and higher 13 versions
The Trusted Language Extensions development environment and runtime are packaged as the `pg_tle` PostgreSQL extension, version 1.0.1. It supports creating extensions in JavaScript, Perl, Tcl, PL/pgSQL, and SQL. You install the `pg_tle` extension in your RDS for PostgreSQL DB instance in the same way that you install other PostgreSQL extensions. After the `pg_tle` is set up, developers can use it to create new PostgreSQL extensions, known as *TLE extensions*.
In the following topics, you can find information about how to set up Trusted Language Extensions and how to get started creating your own TLE extensions.
**Topics**
+ [
# Terminology
](PostgreSQL_trusted_language_extension-terminology.md)
+ [
# Requirements for using Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension-requirements.md)
+ [
# Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance
](PostgreSQL_trusted_language_extension-setting-up.md)
+ [
# Overview of Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension.overview.md)
+ [
# Creating TLE extensions for RDS for PostgreSQL
](PostgreSQL_trusted_language_extension-creating-TLE-extensions.md)
+ [
# Dropping your TLE extensions from a database
](PostgreSQL_trusted_language_extension-creating-TLE-extensions.dropping-TLEs.md)
+ [
# Uninstalling Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension-uninstalling-pg_tle-devkit.md)
+ [
# Using PostgreSQL hooks with your TLE extensions
](PostgreSQL_trusted_language_extension.overview.tles-and-hooks.md)
+ [
# Using Custom Data Types in TLE
](PostgreSQL_trusted_language_extension-custom-data-type.md)
+ [
# Function reference for Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension-functions-reference.md)
+ [
# Hooks reference for Trusted Language Extensions for PostgreSQL
](PostgreSQL_trusted_language_extension-hooks-reference.md)
# Terminology
To help you better understand Trusted Language Extensions, view the following glossary for terms used in this topic.
**Trusted Language Extensions for PostgreSQL**
*Trusted Language Extensions for PostgreSQL* is the official name of the open source development kit that's packaged as the `pg_tle` extension. It's available for use on any PostgreSQL system. For more information, see [aws/pg\$1tle](https://github.com/aws/pg_tle) on GitHub.
**Trusted Language Extensions**
*Trusted Language Extensions* is the short name for Trusted Language Extensions for PostgreSQL. This shortened name and its abbreviation (TLE) are also used in this documentation.
**trusted language**
A *trusted language* is a programming or scripting language that has specific security attributes. For example, trusted languages typically restrict access to the file system, and they limit use of specified networking properties. The TLE development kit is designed to support trusted languages. PostgreSQL supports several different languages that are used to create trusted or untrusted extensions. For an example, see [Trusted and Untrusted PL/Perl](https://www.postgresql.org/docs/current/plperl-trusted.html) in the PostgreSQL documentation. When you create an extension using Trusted Language Extensions, the extension inherently uses trusted language mechanisms.
**TLE extension**
A *TLE extension* is a PostgreSQL extension that's been created by using the Trusted Language Extensions (TLE) development kit.
# Requirements for using Trusted Language Extensions for PostgreSQL
The following are requirements for setting up and using the TLE development kit.
+ ** RDS for PostgreSQL versions** – Trusted Language Extensions is supported on RDS for PostgreSQL versions 13.12 and higher 13 versions, 14.5 and higher 14 versions, and 15.2 and higher versions only.
+ If you need to upgrade your RDS for PostgreSQL instance, see [Upgrades of the RDS for PostgreSQL DB engine](USER_UpgradeDBInstance.PostgreSQL.md).
+ If you don't yet have an Amazon RDS DB instance running PostgreSQL, you can create one. For more information, see RDS for PostgreSQL DB instance, see [Creating and connecting to a PostgreSQL DB instance](CHAP_GettingStarted.CreatingConnecting.PostgreSQL.md).
+ **Requires `rds_superuser` privileges** – To set up and configure the `pg_tle` extension, your database user role must have the permissions of the `rds_superuser` role. By default, this role is granted to the `postgres` user that creates the RDS for PostgreSQL DB instance.
+ **Requires a custom DB parameter group** – Your RDS for PostgreSQL DB instance must be configured with a custom DB parameter group.
+ If your RDS for PostgreSQL DB instance isn't configured with a custom DB parameter group, you should create one and associate it with your RDS for PostgreSQL DB instance. For a short summary of steps, see [Creating and applying a custom DB parameter group](#PostgreSQL_trusted_language_extension-requirements-create-custom-params).
+ If your RDS for PostgreSQL DB instance is already configured using a custom DB parameter group, you can set up Trusted Language Extensions. For details, see [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
## Creating and applying a custom DB parameter group
Use the following steps to create a custom DB parameter group and configure your RDS for PostgreSQL DB instance to use it.
### Console
**To create a custom DB parameter group and use it with your RDS for PostgreSQL DB instance**
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. Choose Parameter groups from the Amazon RDS menu.
1. Choose **Create parameter group**.
1. In the **Parameter group details** page, enter the following information.
+ For **Parameter group family**, choose postgres14.
+ For **Type**, choose DB Parameter Group.
+ For **Group name**, give your parameter group a meaningful name in the context of your operations.
+ For **Description**, enter a useful description so that others on your team can easily find it.
1. Choose **Create**. Your custom DB parameter group is created in your AWS Region. You can now modify your RDS for PostgreSQL DB instance to use it by following the next steps.
1. Choose **Databases** from the Amazon RDS menu.
1. Choose the RDS for PostgreSQL DB instance that you want to use with TLE from among those listed, and then choose **Modify.**
1. In the Modify DB instance settings page, find **Database options** in the Additional configuration section and choose your custom DB parameter group from the selector.
1. Choose **Continue** to save the change.
1. Choose **Apply immediately** so that you can continue setting up the RDS for PostgreSQL DB instance to use TLE.
To continue setting up your system for Trusted Language Extensions, see [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
For more information working with DB parameter groups, see [DB parameter groups for Amazon RDS DB instances](USER_WorkingWithDBInstanceParamGroups.md).
### AWS CLI
You can avoid specifying the `--region` argument when you use CLI commands by configuring your AWS CLI with your default AWS Region. For more information, see [Configuration basics](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-config) in the *AWS Command Line Interface User Guide*.
**To create a custom DB parameter group and use it with your RDS for PostgreSQL DB instance**
1. Use the [create-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-parameter-group.html) AWS CLI command to create a custom DB parameter group based on postgres14 for your AWS Region.
For Linux, macOS, or Unix:
```
aws rds create-db-parameter-group \
--region aws-region \
--db-parameter-group-name custom-params-for-pg-tle \
--db-parameter-group-family postgres14 \
--description "My custom DB parameter group for Trusted Language Extensions"
```
For Windows:
```
aws rds create-db-parameter-group ^
--region aws-region ^
--db-parameter-group-name custom-params-for-pg-tle ^
--db-parameter-group-family postgres14 ^
--description "My custom DB parameter group for Trusted Language Extensions"
```
Your custom DB parameter group is available in your AWS Region, so you can modify RDS for PostgreSQL DB instance to use it.
1. Use the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) AWS CLI command to apply your custom DB parameter group to your RDS for PostgreSQL DB instance. This command immediately reboots the active instance.
For Linux, macOS, or Unix:
```
aws rds modify-db-instance \
--region aws-region \
--db-instance-identifier your-instance-name \
--db-parameter-group-name custom-params-for-pg-tle \
--apply-immediately
```
For Windows:
```
aws rds modify-db-instance ^
--region aws-region ^
--db-instance-identifier your-instance-name ^
--db-parameter-group-name custom-params-for-pg-tle ^
--apply-immediately
```
To continue setting up your system for Trusted Language Extensions, see [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
For more information, see [Parameter groups for Amazon RDS](USER_WorkingWithParamGroups.md).
# Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance
The following steps assume that your RDS for PostgreSQL DB instance is associated with a custom DB parameter group. You can use the AWS Management Console or the AWS CLI for these steps.
When you set up Trusted Language Extensions in your RDS for PostgreSQL DB instance, you install it in a specific database for use by the database users who have permissions on that database.
## Console
**To set up Trusted Language Extensions**
Perform the following steps using an account that's a member of the `rds_superuser` group (role).
1. Sign in to the AWS Management Console and open the Amazon RDS console at [https://console.aws.amazon.com/rds/](https://console.aws.amazon.com/rds/).
1. In the navigation pane, choose your RDS for PostgreSQL DB instance.
1. Open the **Configuration** tab for your RDS for PostgreSQL DB instance. Among the Instance details, find the **Parameter group** link.
1. Choose the link to open the custom parameters associated with your RDS for PostgreSQL DB instance.
1. In the **Parameters** search field, type `shared_pre` to find the `shared_preload_libraries` parameter.
1. Choose **Edit parameters** to access the property values.
1. Add `pg_tle` to the list in the **Values** field. Use a comma to separate items in the list of values.
![\[Image of the shared_preload_libraries parameter with pg_tle added.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/apg_rpg_shared_preload_pg_tle.png)
1. Reboot the RDS for PostgreSQL DB instance so that your change to the `shared_preload_libraries` parameter takes effect.
1. When the instance is available, verify that `pg_tle` has been initialized. Use `psql` to connect to the RDS for PostgreSQL DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
shared_preload_libraries
--------------------------
rdsutils,pg_tle
(1 row)
```
1. With the `pg_tle` extension initialized, you can now create the extension.
```
CREATE EXTENSION pg_tle;
```
You can verify that the extension is installed by using the following `psql` metacommand.
```
labdb=> \dx
List of installed extensions
Name | Version | Schema | Description
---------+---------+------------+--------------------------------------------
pg_tle | 1.0.1 | pgtle | Trusted-Language Extensions for PostgreSQL
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
```
1. Grant the `pgtle_admin` role to the primary user name that you created for your RDS for PostgreSQL DB instance when you set it up. If you accepted the default, it's `postgres`.
```
labdb=> GRANT pgtle_admin TO postgres;
GRANT ROLE
```
You can verify that the grant has occurred by using the `psql` metacommand as shown in the following example. Only the `pgtle_admin` and `postgres` roles are shown in the output. For more information, see [Understanding the rds\$1superuser role](Appendix.PostgreSQL.CommonDBATasks.Roles.rds_superuser.md).
```
labdb=> \du
List of roles
Role name | Attributes | Member of
-----------------+---------------------------------+-----------------------------------
pgtle_admin | Cannot login | {}
postgres | Create role, Create DB +| {rds_superuser,pgtle_admin}
| Password valid until infinity |...
```
1. Close the `psql` session using the `\q` metacommand.
```
\q
```
To get started creating TLE extensions, see [Example: Creating a trusted language extension using SQL](PostgreSQL_trusted_language_extension-creating-TLE-extensions.md#PostgreSQL_trusted_language_extension-simple-example).
## AWS CLI
You can avoid specifying the `--region` argument when you use CLI commands by configuring your AWS CLI with your default AWS Region. For more information, see [Configuration basics](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-config) in the *AWS Command Line Interface User Guide*.
**To set up Trusted Language Extensions**
1. Use the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) AWS CLI command to add `pg_tle` to the `shared_preload_libraries` parameter.
```
aws rds modify-db-parameter-group \
--db-parameter-group-name custom-param-group-name \
--parameters "ParameterName=shared_preload_libraries,ParameterValue=pg_tle,ApplyMethod=pending-reboot" \
--region aws-region
```
1. Use the [reboot-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/reboot-db-instance) AWS CLI command to reboot the RDS for PostgreSQL DB instance and initialize the `pg_tle` library.
```
aws rds reboot-db-instance \
--db-instance-identifier your-instance \
--region aws-region
```
1. When the instance is available, you can verify that `pg_tle` has been initialized. Use `psql` to connect to the RDS for PostgreSQL DB instance, and then run the following command.
```
SHOW shared_preload_libraries;
shared_preload_libraries
--------------------------
rdsutils,pg_tle
(1 row)
```
With `pg_tle` initialized, you can now create the extension.
```
CREATE EXTENSION pg_tle;
```
1. Grant the `pgtle_admin` role to the primary user name that you created for your RDS for PostgreSQL DB instance when you set it up. If you accepted the default, it's `postgres`.
```
GRANT pgtle_admin TO postgres;
GRANT ROLE
```
1. Close the `psql` session as follows.
```
labdb=> \q
```
To get started creating TLE extensions, see [Example: Creating a trusted language extension using SQL](PostgreSQL_trusted_language_extension-creating-TLE-extensions.md#PostgreSQL_trusted_language_extension-simple-example).
# Overview of Trusted Language Extensions for PostgreSQL
Trusted Language Extensions for PostgreSQL is a PostgreSQL extension that you install in your RDS for PostgreSQL DB instance in the same way that you set up other PostgreSQL extensions. In the following image of an example database in the pgAdmin client tool, you can view some of the components that comprise the `pg_tle` extension.
![\[Image showing some of the components that make up the TLE development kit.\]](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/images/apg-pg_tle-installed-view-in-pgAdmin.png)
You can see the following details.
1. The Trusted Language Extensions (TLE) for PostgreSQL development kit is packaged as the `pg_tle` extension. As such, `pg_tle` is added to the available extensions for the database in which it's installed.
1. TLE has its own schema, `pgtle`. This schema contains helper functions (3) for installing and managing the extensions that you create.
1. TLE provides over a dozen helper functions for installing, registering, and managing your extensions. To learn more about these functions, see [Function reference for Trusted Language Extensions for PostgreSQL](PostgreSQL_trusted_language_extension-functions-reference.md).
Other components of the `pg_tle` extension include the following:
+ **The `pgtle_admin` role** – The `pgtle_admin` role is created when the `pg_tle` extension is installed. This role is privileged and should be treated as such. We strongly recommend that you follow the principle of *least privilege* when granting the `pgtle_admin` role to database users. In other words, grant the `pgtle_admin` role only to database users that are allowed to create, install, and manage new TLE extensions, such as `postgres`.
+ **The `pgtle.feature_info` table** – The `pgtle.feature_info` table is a protected table that contains information about your TLEs, hooks, and the custom stored procedures and functions that they use. If you have `pgtle_admin` privileges, you use the following Trusted Language Extensions functions to add and update that information in the table.
+ [pgtle.register\$1feature](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.register_feature)
+ [pgtle.register\$1feature\$1if\$1not\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.register_feature_if_not_exists)
+ [pgtle.unregister\$1feature](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.unregister_feature)
+ [pgtle.unregister\$1feature\$1if\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.unregister_feature_if_exists)
# Creating TLE extensions for RDS for PostgreSQL
You can install any extensions that you create with TLE in any RDS for PostgreSQL DB instance that has the `pg_tle` extension installed. The `pg_tle` extension is scoped to the PostgreSQL database in which it's installed. The extensions that you create using TLE are scoped to the same database.
Use the various `pgtle` functions to install the code that makes up your TLE extension. The following Trusted Language Extensions functions all require the `pgtle_admin` role.
+ [pgtle.install\$1extension](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.install_extension)
+ [pgtle.install\$1update\$1path](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.install_update_path)
+ [pgtle.register\$1feature](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.register_feature)
+ [pgtle.register\$1feature\$1if\$1not\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.register_feature_if_not_exists)
+ [pgtle.set\$1default\$1version](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.set_default_version)
+ [pgtle.uninstall\$1extension(name)](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.uninstall_extension-name)
+ [pgtle.uninstall\$1extension(name, version)](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.uninstall_extension-name-version)
+ [pgtle.uninstall\$1extension\$1if\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.uninstall_extension_if_exists)
+ [pgtle.uninstall\$1update\$1path](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.uninstall_update_path)
+ [pgtle.uninstall\$1update\$1path\$1if\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.uninstall_update_path_if_exists)
+ [pgtle.unregister\$1feature](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.unregister_feature)
+ [pgtle.unregister\$1feature\$1if\$1exists](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.unregister_feature_if_exists)
## Example: Creating a trusted language extension using SQL
The following example shows you how to create a TLE extension named `pg_distance` that contains a few SQL functions for calculating distances using different formulas. In the listing, you can find the function for calculating the Manhattan distance and the function for calculating the Euclidean distance. For more information about the difference between these formulas, see [Taxicab geometry](https://en.wikipedia.org/wiki/Taxicab_geometry) and [Euclidean geometry](https://en.wikipedia.org/wiki/Euclidean_geometry) in Wikipedia.
You can use this example in your own RDS for PostgreSQL DB instance if you have the `pg_tle` extension set up as detailed in [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
**Note**
You need to have the privileges of the `pgtle_admin` role to follow this procedure.
**To create the example TLE extension**
The following steps use an example database named `labdb`. This database is owned by the `postgres` primary user. The `postgres` role also has the permissions of the `pgtle_admin` role.
1. Use `psql` to connect to RDS for PostgreSQL DB instance.
```
psql --host=db-instance-123456789012.aws-region.rds.amazonaws.com
--port=5432 --username=postgres --password --dbname=labdb
```
1. Create a TLE extension named `pg_distance` by copying the following code and pasting it into your `psql` session console.
```
SELECT pgtle.install_extension
(
'pg_distance',
'0.1',
'Distance functions for two points',
$_pg_tle_$
CREATE FUNCTION dist(x1 float8, y1 float8, x2 float8, y2 float8, norm int)
RETURNS float8
AS $$
SELECT (abs(x2 - x1) ^ norm + abs(y2 - y1) ^ norm) ^ (1::float8 / norm);
$$ LANGUAGE SQL;
CREATE FUNCTION manhattan_dist(x1 float8, y1 float8, x2 float8, y2 float8)
RETURNS float8
AS $$
SELECT dist(x1, y1, x2, y2, 1);
$$ LANGUAGE SQL;
CREATE FUNCTION euclidean_dist(x1 float8, y1 float8, x2 float8, y2 float8)
RETURNS float8
AS $$
SELECT dist(x1, y1, x2, y2, 2);
$$ LANGUAGE SQL;
$_pg_tle_$
);
```
You see the output, such as the following.
```
install_extension
---------------
t
(1 row)
```
The artifacts that make up the `pg_distance` extension are now installed in your database. These artifacts include the control file and the code for the extension, which are items that need to be present so that the extension can be created using the `CREATE EXTENSION` command. In other words, you still need to create the extension to make its functions available to database users.
1. To create the extension, use the `CREATE EXTENSION` command as you do for any other extension. As with other extensions, the database user needs to have the `CREATE` permissions in the database.
```
CREATE EXTENSION pg_distance;
```
1. To test the `pg_distance` TLE extension, you can use it to calculate the [Manhattan distance](https://en.wikipedia.org/wiki/Taxicab_geometry) between four points.
```
labdb=> SELECT manhattan_dist(1, 1, 5, 5);
8
```
To calculate the [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_geometry) between the same set of points, you can use the following.
```
labdb=> SELECT euclidean_dist(1, 1, 5, 5);
5.656854249492381
```
The `pg_distance` extension loads the functions in the database and makes them available to any users with permissions on the database.
## Modifying your TLE extension
To improve query performance for the functions packaged in this TLE extension, add the following two PostgreSQL attributes to their specifications.
+ `IMMUTABLE` – The `IMMUTABLE` attribute ensures that the query optimizer can use optimizations to improve query response times. For more information, see [Function Volatility Categories](https://www.postgresql.org/docs/current/xfunc-volatility.html) in the PostgreSQL documentation.
+ `PARALLEL SAFE` – The `PARALLEL SAFE` attribute is another attribute that allows PostgreSQL to run the function in parallel mode. For more information, see [CREATE FUNCTION](https://www.postgresql.org/docs/current/sql-createfunction.html) in the PostgreSQL documentation.
In the following example, you can see how the `pgtle.install_update_path` function is used to add these attributes to each function to create a version `0.2` of the `pg_distance` TLE extension. For more information about this function, see [pgtle.install\$1update\$1path](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.install_update_path). You need to have the `pgtle_admin` role to perform this task.
**To update an existing TLE extension and specify the default version**
1. Connect to RDS for PostgreSQL DB instance using `psql` or another client tool, such as pgAdmin.
```
psql --host=db-instance-123456789012.aws-region.rds.amazonaws.com
--port=5432 --username=postgres --password --dbname=labdb
```
1. Modify the existing TLE extension by copying the following code and pasting it into your `psql` session console.
```
SELECT pgtle.install_update_path
(
'pg_distance',
'0.1',
'0.2',
$_pg_tle_$
CREATE OR REPLACE FUNCTION dist(x1 float8, y1 float8, x2 float8, y2 float8, norm int)
RETURNS float8
AS $$
SELECT (abs(x2 - x1) ^ norm + abs(y2 - y1) ^ norm) ^ (1::float8 / norm);
$$ LANGUAGE SQL IMMUTABLE PARALLEL SAFE;
CREATE OR REPLACE FUNCTION manhattan_dist(x1 float8, y1 float8, x2 float8, y2 float8)
RETURNS float8
AS $$
SELECT dist(x1, y1, x2, y2, 1);
$$ LANGUAGE SQL IMMUTABLE PARALLEL SAFE;
CREATE OR REPLACE FUNCTION euclidean_dist(x1 float8, y1 float8, x2 float8, y2 float8)
RETURNS float8
AS $$
SELECT dist(x1, y1, x2, y2, 2);
$$ LANGUAGE SQL IMMUTABLE PARALLEL SAFE;
$_pg_tle_$
);
```
You see a response similar to the following.
```
install_update_path
---------------------
t
(1 row)
```
You can make this version of the extension the default version, so that database users don't have to specify a version when they create or update the extension in their database.
1. To specify that the modified version (version 0.2) of your TLE extension is the default version, use the `pgtle.set_default_version` function as shown in the following example.
```
SELECT pgtle.set_default_version('pg_distance', '0.2');
```
For more information about this function, see [pgtle.set\$1default\$1version](PostgreSQL_trusted_language_extension-functions-reference.md#pgtle.set_default_version).
1. With the code in place, you can update the installed TLE extension in the usual way, by using `ALTER EXTENSION ... UPDATE` command, as shown here:
```
ALTER EXTENSION pg_distance UPDATE;
```
# Dropping your TLE extensions from a database
You can drop your TLE extensions by using the `DROP EXTENSION` command in the same way that you do for other PostgreSQL extensions. Dropping the extension doesn't remove the installation files that make up the extension, which allows users to re-create the extension. To remove the extension and its installation files, do the following two-step process.
**To drop the TLE extension and remove its installation files**
1. Use `psql` or another client tool to connect to the RDS for PostgreSQL DB instance.
```
psql --host=.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=dbname
```
1. Drop the extension as you would any PostgreSQL extension.
```
DROP EXTENSION your-TLE-extension
```
For example, if you create the `pg_distance` extension as detailed in [Example: Creating a trusted language extension using SQL](PostgreSQL_trusted_language_extension-creating-TLE-extensions.md#PostgreSQL_trusted_language_extension-simple-example), you can drop the extension as follows.
```
DROP EXTENSION pg_distance;
```
You see output confirming that the extension has been dropped, as follows.
```
DROP EXTENSION
```
At this point, the extension is no longer active in the database. However, its installation files and control file are still available in the database, so database users can create the extension again if they like.
+ If you want to leave the extension files intact so that database users can create your TLE extension, you can stop here.
+ If you want to remove all files that make up the extension, continue to the next step.
1. To remove all installation files for your extension, use the `pgtle.uninstall_extension` function. This function removes all the code and control files for your extension.
```
SELECT pgtle.uninstall_extension('your-tle-extension-name');
```
For example, to remove all `pg_distance` installation files, use the following command.
```
SELECT pgtle.uninstall_extension('pg_distance');
uninstall_extension
---------------------
t
(1 row)
```
# Uninstalling Trusted Language Extensions for PostgreSQL
If you no longer want to create your own TLE extensions using TLE, you can drop the `pg_tle` extension and remove all artifacts. This action includes dropping any TLE extensions in the database and dropping the `pgtle` schema.
**To drop the `pg_tle` extension and its schema from a database**
1. Use `psql` or another client tool to connect to the RDS for PostgreSQL DB instance.
```
psql --host=.111122223333.aws-region.rds.amazonaws.com --port=5432 --username=postgres --password --dbname=dbname
```
1. Drop the `pg_tle` extension from the database. If the database has your own TLE extensions still running in the database, you need to also drop those extensions. To do so, you can use the `CASCADE` keyword, as shown in the following.
```
DROP EXTENSION pg_tle CASCADE;
```
If the `pg_tle` extension isn't still active in the database, you don't need to use the `CASCADE` keyword.
1. Drop the `pgtle` schema. This action removes all the management functions from the database.
```
DROP SCHEMA pgtle CASCADE;
```
The command returns the following when the process completes.
```
DROP SCHEMA
```
The `pg_tle` extension, its schema and functions, and all artifacts are removed. To create new extensions using TLE, go through the setup process again. For more information, see [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
# Using PostgreSQL hooks with your TLE extensions
A *hook* is a callback mechanism available in PostgreSQL that allows developers to call custom functions or other routines during regular database operations. The TLE development kit supports PostgreSQL hooks so that you can integrate custom functions with PostgreSQL behavior at runtime. For example, you can use a hook to associate the authentication process with your own custom code, or to modify the query planning and execution process for your specific needs.
Your TLE extensions can use hooks. If a hook is global in scope, it applies across all databases. Therefore, if your TLE extension uses a global hook, then you need to create your TLE extension in all databases that your users can access.
When you use the `pg_tle` extension to build your own Trusted Language Extensions, you can use the available hooks from a SQL API to build out the functions of your extension. You should register any hooks with `pg_tle`. For some hooks, you might also need to set various configuration parameters. For example, the `passcode` check hook can be set to on, off, or require. For more information about specific requirements for available `pg_tle` hooks, see [Hooks reference for Trusted Language Extensions for PostgreSQL](PostgreSQL_trusted_language_extension-hooks-reference.md).
## Example: Creating an extension that uses a PostgreSQL hook
The example discussed in this section uses a PostgreSQL hook to check the password provided during specific SQL operations and prevents database users from setting their passwords to any of those contained in the `password_check.bad_passwords` table. The table contains the top-ten most commonly used, but easily breakable choices for passwords.
To set up this example in your RDS for PostgreSQL DB instance, you must have already installed Trusted Language Extensions. For details, see [Setting up Trusted Language Extensions in your RDS for PostgreSQL DB instance](PostgreSQL_trusted_language_extension-setting-up.md).
**To set up the password-check hook example**
1. Use `psql` to connect to RDS for PostgreSQL DB instance.
```
psql --host=db-instance-123456789012.aws-region.rds.amazonaws.com
--port=5432 --username=postgres --password --dbname=labdb
```
1. Copy the code from the [Password-check hook code listing](#PostgreSQL_trusted_language_extension-example-hook_code_listing) and paste it into your database.
```
SELECT pgtle.install_extension (
'my_password_check_rules',
'1.0',
'Do not let users use the 10 most commonly used passwords',
$_pgtle_$
CREATE SCHEMA password_check;
REVOKE ALL ON SCHEMA password_check FROM PUBLIC;
GRANT USAGE ON SCHEMA password_check TO PUBLIC;
CREATE TABLE password_check.bad_passwords (plaintext) AS
VALUES
('123456'),
('password'),
('12345678'),
('qwerty'),
('123456789'),
('12345'),
('1234'),
('111111'),
('1234567'),
('dragon');
CREATE UNIQUE INDEX ON password_check.bad_passwords (plaintext);
CREATE FUNCTION password_check.passcheck_hook(username text, password text, password_type pgtle.password_types, valid_until timestamptz, valid_null boolean)
RETURNS void AS $$
DECLARE
invalid bool := false;
BEGIN
IF password_type = 'PASSWORD_TYPE_MD5' THEN
SELECT EXISTS(
SELECT 1
FROM password_check.bad_passwords bp
WHERE ('md5' || md5(bp.plaintext || username)) = password
) INTO invalid;
IF invalid THEN
RAISE EXCEPTION 'Cannot use passwords from the common password dictionary';
END IF;
ELSIF password_type = 'PASSWORD_TYPE_PLAINTEXT' THEN
SELECT EXISTS(
SELECT 1
FROM password_check.bad_passwords bp
WHERE bp.plaintext = password
) INTO invalid;
IF invalid THEN
RAISE EXCEPTION 'Cannot use passwords from the common password dictionary';
END IF;
END IF;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;
GRANT EXECUTE ON FUNCTION password_check.passcheck_hook TO PUBLIC;
SELECT pgtle.register_feature('password_check.passcheck_hook', 'passcheck');
$_pgtle_$
);
```
When the extension has been loaded into your database, you see the output such as the following.
```
install_extension
-------------------
t
(1 row)
```
1. While still connected to the database, you can now create the extension.
```
CREATE EXTENSION my_password_check_rules;
```
1. You can confirm that the extension has been created in the database by using the following `psql` metacommand.
```
\dx
List of installed extensions
Name | Version | Schema | Description
-------------------------+---------+------------+-------------------------------------------------------------
my_password_check_rules | 1.0 | public | Prevent use of any of the top-ten most common bad passwords
pg_tle | 1.0.1 | pgtle | Trusted-Language Extensions for PostgreSQL
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(3 rows)
```
1. Open another terminal session to work with the AWS CLI. You need to modify your custom DB parameter group to turn on the password-check hook. To do so, use the [modify-db-parameter-group](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-parameter-group.html) CLI command as shown in the following example.
```
aws rds modify-db-parameter-group \
--region aws-region \
--db-parameter-group-name your-custom-parameter-group \
--parameters "ParameterName=pgtle.enable_password_check,ParameterValue=on,ApplyMethod=immediate"
```
When the parameter is successfully turned on, you see the output such as the following.
```
(
"DBParameterGroupName": "docs-lab-parameters-for-tle"
}
```
It might take a few minutes for the change to the parameter group setting to take effect. This parameter is dynamic, however, so you don't need to restart the RDS for PostgreSQL DB instance for the setting to take effect.
1. Open the `psql` session and query the database to verify that the password\$1check hook has been turned on.
```
labdb=> SHOW pgtle.enable_password_check;
pgtle.enable_password_check
-----------------------------
on
(1 row)
```
The password-check hook is now active. You can test it by creating a new role and using one of the bad passwords, as shown in the following example.
```
CREATE ROLE test_role PASSWORD 'password';
ERROR: Cannot use passwords from the common password dictionary
CONTEXT: PL/pgSQL function password_check.passcheck_hook(text,text,pgtle.password_types,timestamp with time zone,boolean) line 21 at RAISE
SQL statement "SELECT password_check.passcheck_hook(
$1::pg_catalog.text,
$2::pg_catalog.text,
$3::pgtle.password_types,
$4::pg_catalog.timestamptz,
$5::pg_catalog.bool)"
```
The output has been formatted for readability.
The following example shows that `pgsql` interactive metacommand `\password` behavior is also affected by the password\$1check hook.
```
postgres=> SET password_encryption TO 'md5';
SET
postgres=> \password
Enter new password for user "postgres":*****
Enter it again:*****
ERROR: Cannot use passwords from the common password dictionary
CONTEXT: PL/pgSQL function password_check.passcheck_hook(text,text,pgtle.password_types,timestamp with time zone,boolean) line 12 at RAISE
SQL statement "SELECT password_check.passcheck_hook($1::pg_catalog.text, $2::pg_catalog.text, $3::pgtle.password_types, $4::pg_catalog.timestamptz, $5::pg_catalog.bool)"
```
You can drop this TLE extension and uninstall its source files if you want. For more information, see [Dropping your TLE extensions from a databaseDropping your TLE extensions from a database](PostgreSQL_trusted_language_extension-creating-TLE-extensions.dropping-TLEs.md).
### Password-check hook code listing
The example code shown here defines the specification for the `my_password_check_rules` TLE extension. When you copy this code and paste it into your database, the code for the `my_password_check_rules` extension is loaded into the database, and the `password_check` hook is registered for use by the extension.
```
SELECT pgtle.install_extension (
'my_password_check_rules',
'1.0',
'Do not let users use the 10 most commonly used passwords',
$_pgtle_$
CREATE SCHEMA password_check;
REVOKE ALL ON SCHEMA password_check FROM PUBLIC;
GRANT USAGE ON SCHEMA password_check TO PUBLIC;
CREATE TABLE password_check.bad_passwords (plaintext) AS
VALUES
('123456'),
('password'),
('12345678'),
('qwerty'),
('123456789'),
('12345'),
('1234'),
('111111'),
('1234567'),
('dragon');
CREATE UNIQUE INDEX ON password_check.bad_passwords (plaintext);
CREATE FUNCTION password_check.passcheck_hook(username text, password text, password_type pgtle.password_types, valid_until timestamptz, valid_null boolean)
RETURNS void AS $$
DECLARE
invalid bool := false;
BEGIN
IF password_type = 'PASSWORD_TYPE_MD5' THEN
SELECT EXISTS(
SELECT 1
FROM password_check.bad_passwords bp
WHERE ('md5' || md5(bp.plaintext || username)) = password
) INTO invalid;
IF invalid THEN
RAISE EXCEPTION 'Cannot use passwords from the common password dictionary';
END IF;
ELSIF password_type = 'PASSWORD_TYPE_PLAINTEXT' THEN
SELECT EXISTS(
SELECT 1
FROM password_check.bad_passwords bp
WHERE bp.plaintext = password
) INTO invalid;
IF invalid THEN
RAISE EXCEPTION 'Cannot use passwords from the common password dictionary';
END IF;
END IF;
END
$$ LANGUAGE plpgsql SECURITY DEFINER;
GRANT EXECUTE ON FUNCTION password_check.passcheck_hook TO PUBLIC;
SELECT pgtle.register_feature('password_check.passcheck_hook', 'passcheck');
$_pgtle_$
);
```
# Using Custom Data Types in TLE
PostgreSQL supports commands to register new base types (also known as scalar types) for efficiently handling complex data structures in your database. A base type allows you to customize how the data is stored internally, and how to convert it to and from an external textual representation. These custom data types are helpful when extending PostgreSQL to support functional domains where a built-in type such as number or text can't provide sufficient search semantics.
RDS for PostgreSQL enables you to create custom data types in your trusted language extension and define functions that support SQL and index operations for these new data types. Custom data types are available for the following versions:
+ RDS for PostgreSQL 15.4 and higher 15 versions
+ RDS for PostgreSQL 14.9 and higher 14 versions
+ RDS for PostgreSQL 13.12 and higher 13 versions
For more information, see [Trusted Language Base types](https://github.com/aws/pg_tle/blob/main/docs/09_datatypes.md).
# Function reference for Trusted Language Extensions for PostgreSQL
View the following reference documentation about functions available in Trusted Language Extensions for PostgreSQL. Use these functions to install, register, update, and manage your *TLE extensions*, that is, the PostgreSQL extensions that you develop using the Trusted Language Extensions development kit.
**Topics**
+ [
## pgtle.available\$1extensions
](#pgtle.available_extensions)
+ [
## pgtle.available\$1extension\$1versions
](#pgtle.available_extension_versions)
+ [
## pgtle.extension\$1update\$1paths
](#pgtle.extension_update_paths)
+ [
## pgtle.install\$1extension
](#pgtle.install_extension)
+ [
## pgtle.install\$1update\$1path
](#pgtle.install_update_path)
+ [
## pgtle.register\$1feature
](#pgtle.register_feature)
+ [
## pgtle.register\$1feature\$1if\$1not\$1exists
](#pgtle.register_feature_if_not_exists)
+ [
## pgtle.set\$1default\$1version
](#pgtle.set_default_version)
+ [
## pgtle.uninstall\$1extension(name)
](#pgtle.uninstall_extension-name)
+ [
## pgtle.uninstall\$1extension(name, version)
](#pgtle.uninstall_extension-name-version)
+ [
## pgtle.uninstall\$1extension\$1if\$1exists
](#pgtle.uninstall_extension_if_exists)
+ [
## pgtle.uninstall\$1update\$1path
](#pgtle.uninstall_update_path)
+ [
## pgtle.uninstall\$1update\$1path\$1if\$1exists
](#pgtle.uninstall_update_path_if_exists)
+ [
## pgtle.unregister\$1feature
](#pgtle.unregister_feature)
+ [
## pgtle.unregister\$1feature\$1if\$1exists
](#pgtle.unregister_feature_if_exists)
## pgtle.available\$1extensions
The `pgtle.available_extensions` function is a set-returning function. It returns all available TLE extensions in the database. Each returned row contains information about a single TLE extension.
### Function prototype
```
pgtle.available_extensions()
```
### Role
None.
### Arguments
None.
### Output
+ `name` – The name of the TLE extension.
+ `default_version` – The version of the TLE extension to use when `CREATE EXTENSION` is called without a version specified.
+ `description` – A more detailed description about the TLE extension.
### Usage example
```
SELECT * FROM pgtle.available_extensions();
```
## pgtle.available\$1extension\$1versions
The `available_extension_versions` function is a set-returning function. It returns a list of all available TLE extensions and their versions. Each row contains information about a specific version of the given TLE extension, including whether it requires a specific role.
### Function prototype
```
pgtle.available_extension_versions()
```
### Role
None.
### Arguments
None.
### Output
+ `name` – The name of the TLE extension.
+ `version` – The version of the TLE extension.
+ `superuser` – This value is always `false` for your TLE extensions. The permissions needed to create the TLE extension or update it are the same as for creating other objects in the given database.
+ `trusted` – This value is always `false` for a TLE extension.
+ `relocatable` – This value is always `false` for a TLE extension.
+ `schema` – Specifies the name of the schema in which the TLE extension is installed.
+ `requires` – An array containing the names of other extensions needed by this TLE extension.
+ `description` – A detailed description of the TLE extension.
For more information about output values, see [Packaging Related Objects into an Extension > Extension Files](https://www.postgresql.org/docs/current/extend-extensions.html#id-1.8.3.20.11) in the PostgreSQL documentation.
### Usage example
```
SELECT * FROM pgtle.available_extension_versions();
```
## pgtle.extension\$1update\$1paths
The `extension_update_paths` function is a set-returning function. It returns a list of all the possible update paths for a TLE extension. Each row includes the available upgrades or downgrades for that TLE extension.
### Function prototype
```
pgtle.extension_update_paths(name)
```
### Role
None.
### Arguments
`name` – The name of the TLE extension from which to get upgrade paths.
### Output
+ `source` – The source version for an update.
+ `target` – The target version for an update.
+ `path` – The upgrade path used to update a TLE extension from `source` version to `target` version, for example, `0.1--0.2`.
### Usage example
```
SELECT * FROM pgtle.extension_update_paths('your-TLE');
```
## pgtle.install\$1extension
The `install_extension` function lets you install the artifacts that make up your TLE extension in the database, after which it can be created using the `CREATE EXTENSION` command.
### Function prototype
```
pgtle.install_extension(name text, version text, description text, ext text, requires text[] DEFAULT NULL::text[])
```
### Role
None.
### Arguments
+ `name` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `version` – The version of the TLE extension.
+ `description` – A detailed description about the TLE extension. This description is displayed in the `comment` field in `pgtle.available_extensions()`.
+ `ext` – The contents of the TLE extension. This value contains objects such as functions.
+ `requires` – An optional parameter that specifies dependencies for this TLE extension. The `pg_tle` extension is automatically added as a dependency.
Many of these arguments are the same as those that are included in an extension control file for installing a PostgreSQL extension on the file system of a PostgreSQL instance. For more information, see the [Extension Files](http://www.postgresql.org/docs/current/extend-extensions.html#id-1.8.3.20.11) in [Packaging Related Objects into an Extension](https://www.postgresql.org/docs/current/extend-extensions.html) in the PostgreSQL documentation.
### Output
This functions returns `OK` on success and `NULL` on error.
+ `OK` – The TLE extension has been successfully installed in the database.
+ `NULL` – The TLE extension hasn't been successfully installed in the database.
### Usage example
```
SELECT pgtle.install_extension(
'pg_tle_test',
'0.1',
'My first pg_tle extension',
$_pgtle_$
CREATE FUNCTION my_test()
RETURNS INT
AS $$
SELECT 42;
$$ LANGUAGE SQL IMMUTABLE;
$_pgtle_$
);
```
## pgtle.install\$1update\$1path
The `install_update_path` function provides an update path between two different versions of a TLE extension. This function allows users of your TLE extension to update its version by using the `ALTER EXTENSION ... UPDATE` syntax.
### Function prototype
```
pgtle.install_update_path(name text, fromvers text, tovers text, ext text)
```
### Role
`pgtle_admin`
### Arguments
+ `name` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `fromvers` – The source version of the TLE extension for the upgrade.
+ `tovers` – The destination version of the TLE extension for the upgrade.
+ `ext` – The contents of the update. This value contains objects such as functions.
### Output
None.
### Usage example
```
SELECT pgtle.install_update_path('pg_tle_test', '0.1', '0.2',
$_pgtle_$
CREATE OR REPLACE FUNCTION my_test()
RETURNS INT
AS $$
SELECT 21;
$$ LANGUAGE SQL IMMUTABLE;
$_pgtle_$
);
```
## pgtle.register\$1feature
The `register_feature` function adds the specified internal PostgreSQL feature to the `pgtle.feature_info` table. PostgreSQL hooks are an example of an internal PostgreSQL feature. The Trusted Language Extensions development kit supports the use of PostgreSQL hooks. Currently, this function supports the following feature.
+ `passcheck` – Registers the password-check hook with your procedure or function that customizes PostgreSQL's password-check behavior.
### Function prototype
```
pgtle.register_feature(proc regproc, feature pg_tle_feature)
```
### Role
`pgtle_admin`
### Arguments
+ `proc` – The name of a stored procedure or function to use for the feature.
+ `feature` – The name of the `pg_tle` feature (such as `passcheck`) to register with the function.
### Output
None.
### Usage example
```
SELECT pgtle.register_feature('pw_hook', 'passcheck');
```
## pgtle.register\$1feature\$1if\$1not\$1exists
The `pgtle.register_feature_if_not_exists` function adds the specified PostgreSQL feature to the `pgtle.feature_info` table and identifies the TLE extension or other procedure or function that uses the feature. For more information about hooks and Trusted Language Extensions, see [Using PostgreSQL hooks with your TLE extensions](PostgreSQL_trusted_language_extension.overview.tles-and-hooks.md).
### Function prototype
```
pgtle.register_feature_if_not_exists(proc regproc, feature pg_tle_feature)
```
### Role
`pgtle_admin`
### Arguments
+ `proc` – The name of a stored procedure or function that contains the logic (code) to use as a feature for your TLE extension. For example, the `pw_hook` code.
+ `feature` – The name of the PostgreSQL feature to register for the TLE function. Currently, the only available feature is the `passcheck` hook. For more information, see [Password-check hook (passcheck)](PostgreSQL_trusted_language_extension-hooks-reference.md#passcheck_hook).
### Output
Returns `true` after registering the feature for the specified extension. Returns `false` if the feature is already registered.
### Usage example
```
SELECT pgtle.register_feature_if_not_exists('pw_hook', 'passcheck');
```
## pgtle.set\$1default\$1version
The `set_default_version` function lets you specify a `default_version` for your TLE extension. You can use this function to define an upgrade path and designate the version as the default for your TLE extension. When database users specify your TLE extension in the `CREATE EXTENSION` and `ALTER EXTENSION ... UPDATE` commands, that version of your TLE extension is created in the database for that user.
This function returns `true` on success. If the TLE extension specified in the `name` argument doesn't exist, the function returns an error. Similarly, if the `version` of the TLE extension doesn't exist, it returns an error.
### Function prototype
```
pgtle.set_default_version(name text, version text)
```
### Role
`pgtle_admin`
### Arguments
+ `name` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `version` – The version of the TLE extension to set the default.
### Output
+ `true` – When setting default version succeeds, the function returns `true`.
+ `ERROR` – Returns an error message if a TLE extension with the specified name or version doesn't exist.
### Usage example
```
SELECT * FROM pgtle.set_default_version('my-extension', '1.1');
```
## pgtle.uninstall\$1extension(name)
The `uninstall_extension` function removes all versions of a TLE extension from a database. This function prevents future calls of `CREATE EXTENSION` from installing the TLE extension. If the TLE extension doesn't exist in the database, an error is raised.
The `uninstall_extension` function won't drop a TLE extension that's currently active in the database. To remove a TLE extension that's currently active, you need to explicitly call `DROP EXTENSION` to remove it.
### Function prototype
```
pgtle.uninstall_extension(extname text)
```
### Role
`pgtle_admin`
### Arguments
+ `extname` – The name of the TLE extension to uninstall. This name is the same as the one used with `CREATE EXTENSION` to load the TLE extension for use in a given database.
### Output
None.
### Usage example
```
SELECT * FROM pgtle.uninstall_extension('pg_tle_test');
```
## pgtle.uninstall\$1extension(name, version)
The `uninstall_extension(name, version)` function removes the specified version of the TLE extension from the database. This function prevents `CREATE EXTENSION` and `ALTER EXTENSION` from installing or updating a TLE extension to the specified version. This function also removes all update paths for the specified version of the TLE extension. This function won't uninstall the TLE extension if it's currently active in the database. You must explicitly call `DROP EXTENSION` to remove the TLE extension. To uninstall all versions of a TLE extension, see [pgtle.uninstall\$1extension(name)](#pgtle.uninstall_extension-name).
### Function prototype
```
pgtle.uninstall_extension(extname text, version text)
```
### Role
`pgtle_admin`
### Arguments
+ `extname` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `version` – The version of the TLE extension to uninstall from the database.
### Output
None.
### Usage example
```
SELECT * FROM pgtle.uninstall_extension('pg_tle_test', '0.2');
```
## pgtle.uninstall\$1extension\$1if\$1exists
The `uninstall_extension_if_exists` function removes all versions of a TLE extension from a given database. If the TLE extension doesn't exist, the function returns silently (no error message is raised). If the specified extension is currently active within a database, this function doesn't drop it. You must explicitly call `DROP EXTENSION` to remove the TLE extension before using this function to uninstall its artifacts.
### Function prototype
```
pgtle.uninstall_extension_if_exists(extname text)
```
### Role
`pgtle_admin`
### Arguments
+ `extname` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
### Output
The `uninstall_extension_if_exists` function returns `true` after uninstalling the specified extension. If the specified extension doesn't exist, the function returns `false`.
+ `true` – Returns `true` after uninstalling the TLE extension.
+ `false` – Returns `false` when the TLE extension doesn't exist in the database.
### Usage example
```
SELECT * FROM pgtle.uninstall_extension_if_exists('pg_tle_test');
```
## pgtle.uninstall\$1update\$1path
The `uninstall_update_path` function removes the specific update path from a TLE extension. This prevents `ALTER EXTENSION ... UPDATE TO` from using this as an update path.
If the TLE extension is currently being used by one of the versions on this update path, it remains in the database.
If the update path specified doesn't exist, this function raises an error.
### Function prototype
```
pgtle.uninstall_update_path(extname text, fromvers text, tovers text)
```
### Role
`pgtle_admin`
### Arguments
+ `extname` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `fromvers` – The source version of the TLE extension used on the update path.
+ `tovers` – The destination version of the TLE extension used on the update path.
### Output
None.
### Usage example
```
SELECT * FROM pgtle.uninstall_update_path('pg_tle_test', '0.1', '0.2');
```
## pgtle.uninstall\$1update\$1path\$1if\$1exists
The `uninstall_update_path_if_exists` function is similar to `uninstall_update_path` in that it removes the specified update path from a TLE extension. However, if the update path doesn't exist, this function doesn't raise an error message. Instead, the function returns `false`.
### Function prototype
```
pgtle.uninstall_update_path_if_exists(extname text, fromvers text, tovers text)
```
### Role
`pgtle_admin`
### Arguments
+ `extname` – The name of the TLE extension. This value is used when calling `CREATE EXTENSION`.
+ `fromvers` – The source version of the TLE extension used on the update path.
+ `tovers` – The destination version of the TLE extension used on the update path.
### Output
+ `true` – The function has successfully updated the path for the TLE extension.
+ `false` – The function wasn't able to update the path for the TLE extension.
### Usage example
```
SELECT * FROM pgtle.uninstall_update_path_if_exists('pg_tle_test', '0.1', '0.2');
```
## pgtle.unregister\$1feature
The `unregister_feature` function provides a way to remove functions that were registered to use `pg_tle` features, such as hooks. For information about registering a feature, see [pgtle.register\$1feature](#pgtle.register_feature).
### Function prototype
```
pgtle.unregister_feature(proc regproc, feature pg_tle_features)
```
### Role
`pgtle_admin`
### Arguments
+ `proc` – The name of a stored function to register with a `pg_tle` feature.
+ `feature` – The name of the `pg_tle` feature to register with the function. For example, `passcheck` is a feature that can be registered for use by the trusted language extensions that you develop. For more information, see [Password-check hook (passcheck)](PostgreSQL_trusted_language_extension-hooks-reference.md#passcheck_hook).
### Output
None.
### Usage example
```
SELECT * FROM pgtle.unregister_feature('pw_hook', 'passcheck');
```
## pgtle.unregister\$1feature\$1if\$1exists
The `unregister_feature` function provides a way to remove functions that were registered to use `pg_tle` features, such as hooks. For more information, see [Using PostgreSQL hooks with your TLE extensions](PostgreSQL_trusted_language_extension.overview.tles-and-hooks.md). Returns `true` after successfully unregistering the feature. Returns `false` if the feature wasn't registered.
For information about registering `pg_tle` features for your TLE extensions, see [pgtle.register\$1feature](#pgtle.register_feature).
### Function prototype
```
pgtle.unregister_feature_if_exists('proc regproc', 'feature pg_tle_features')
```
### Role
`pgtle_admin`
### Arguments
+ `proc` – The name of the stored function that was registered to include a `pg_tle` feature.
+ `feature` – The name of the `pg_tle` feature that was registered with the trusted language extension.
### Output
Returns `true` or `false`, as follows.
+ `true` – The function has successfully unregistered the feature from extension.
+ `false` – The function wasn't able to unregister the feature from the TLE extension.
### Usage example
```
SELECT * FROM pgtle.unregister_feature_if_exists('pw_hook', 'passcheck');
```
# Hooks reference for Trusted Language Extensions for PostgreSQL
Trusted Language Extensions for PostgreSQL supports PostgreSQL hooks. A *hook* is an internal callback mechanism available to developers for extending PostgreSQL's core functionality. By using hooks, developers can implement their own functions or procedures for use during various database operations, thereby modifying PostgreSQL's behavior in some way. For example, you can use a `passcheck` hook to customize how PostgreSQL handles the passwords supplied when creating or changing passwords for users (roles).
View the following documentation to learn about the passcheck hook available for your TLE extensions. To learn more about the available hooks including the client authentication hook, see [Trusted Language Extensions hooks](https://github.com/aws/pg_tle/blob/main/docs/04_hooks.md).
## Password-check hook (passcheck)
The `passcheck` hook is used to customize PostgreSQL behavior during the password-checking process for the following SQL commands and `psql` metacommand.
+ `CREATE ROLE username ...PASSWORD` – For more information, see [CREATE ROLE](https://www.postgresql.org/docs/current/sql-createrole.html) in the PostgreSQL documentation.
+ `ALTER ROLE username...PASSWORD` – For more information, see [ALTER ROLE](https://www.postgresql.org/docs/current/sql-alterrole.html) in the PostgreSQL documentation.
+ `\password username` – This interactive `psql` metacommand securely changes the password for the specified user by hashing the password before transparently using the `ALTER ROLE ... PASSWORD` syntax. The metacommand is a secure wrapper for the `ALTER ROLE ... PASSWORD` command, thus the hook applies to the behavior of the `psql` metacommand.
For an example, see [Password-check hook code listing](PostgreSQL_trusted_language_extension.overview.tles-and-hooks.md#PostgreSQL_trusted_language_extension-example-hook_code_listing).
**Contents**
+ [
### Function prototype
](#passcheck_hook-prototype)
+ [
### Arguments
](#passcheck_hook-arguments)
+ [
### Configuration
](#passcheck_hook-configuration)
+ [
### Usage notes
](#passcheck_hook-usage)
### Function prototype
```
passcheck_hook(username text, password text, password_type pgtle.password_types, valid_until timestamptz, valid_null boolean)
```
### Arguments
A `passcheck` hook function takes the following arguments.
+ `username` – The name (as text) of the role (username) that's setting a password.
+ `password` – The plaintext or hashed password. The password entered should match the type specified in `password_type`.
+ `password_type` – Specify the `pgtle.password_type` format of the password. This format can be one of the following options.
+ `PASSWORD_TYPE_PLAINTEXT` – A plaintext password.
+ `PASSWORD_TYPE_MD5` – A password that's been hashed using MD5 (message digest 5) algorithm.
+ `PASSWORD_TYPE_SCRAM_SHA_256` – A password that's been hashed using SCRAM-SHA-256 algorithm.
+ `valid_until` – Specify the time when the password becomes invalid. This argument is optional. If you use this argument, specify the time as a `timestamptz` value.
+ `valid_null` – If this Boolean is set to `true`, the `valid_until` option is set to `NULL`.
### Configuration
The function `pgtle.enable_password_check` controls whether the passcheck hook is active. The passcheck hook has three possible settings.
+ `off` – Turns off the `passcheck` password-check hook. This is the default value.
+ `on` – Turns on the `passcode` password-check hook so that passwords are checked against the table.
+ `require` – Requires a password check hook to be defined.
### Usage notes
To turn the `passcheck` hook on or off, you need to modify the custom DB parameter group for your RDS for PostgreSQL DB instance.
For Linux, macOS, or Unix:
```
aws rds modify-db-parameter-group \
--region aws-region \
--db-parameter-group-name your-custom-parameter-group \
--parameters "ParameterName=pgtle.enable_password_check,ParameterValue=on,ApplyMethod=immediate"
```
For Windows:
```
aws rds modify-db-parameter-group ^
--region aws-region ^
--db-parameter-group-name your-custom-parameter-group ^
--parameters "ParameterName=pgtle.enable_password_check,ParameterValue=on,ApplyMethod=immediate"
```