Heroku Postgres Follower Databases
Last updated July 26, 2024
Table of Contents
You can also use heroku pg:psql
with followers to safely run ad-hoc queries against your production data.
Database replication serves many purposes including increasing read throughput with a leader-follower configuration, additional availability with a hot standby, serving as a reporting database, and seamless migrations and upgrades. Though these strategies all serve different purposes, they’re all based on the ability to create and manage copies of a lead database. On Heroku Postgres, this functionality is exposed as the follow feature.
Followers are only supported on Standard, Premium, and Private and Shield tier database plans. To upgrade from an Essential-tier plan to a plan in another tier, follow these instructions.
Followers, like all Heroku Postgres databases, are charged on a pro-rated basis based on the plan of the follower.
A database follower is a read-only copy of the leader database that stays up to date with the leader database data. As writes and other data modifications are committed in the leader database, the changes are streamed, in real time, to the follower databases.
Create a Follower
A follower can be created for any Standard, Premium, or Enterprise tier database that is itself not a follower (that is, followers can’t be chained). Followers can’t be created for a period on newly forked databases (this applies to both explicit forks and forks created through unfollow). The exact timeframe varies depending on the size of the database to be followed and is typically between a few minutes and a few hours.
The following table summarizes where and how you can implement followers:
Leader | Follower | Allowed |
---|---|---|
Common Runtime | Common Runtime | Yes |
Common Runtime | Private Space | Yes |
Private Space | Common Runtime | Yes 1 |
Private Space | Private Space (same Leader’s Space) | Yes |
Private Space | Private Space (different from Leader’s Space) | Yes 1 |
Shield Private Space | Shield Private Space (same Leader’s Space) | Yes |
Shield Private Space | Shield Private Space (different from Leader’s Space) | Yes 2 |
Shield Private Space | Private Space / Common Runtime | No |
Notes:
- Enable Trusted IP ranges for data services to avoid follower lag
- Replication is subject to lag as Trusted IP ranges for data services aren’t currently available for Shield Spaces
The follower must accommodate the current data volume of the leader database. If you attempt to create a follower that can’t accommodate the data volume, no follower is created and an error message indicates the minimum plan needed.
The number of followers on a single leader is limited. The exact limit depends on your plan, the number of data connectors, and the number of existing followers. You see an error message if you try to create a follower that exceeds the calculated limit. In general, it’s not recommended to create more than 10 followers on the same leader.
When a database is ready to support followers, that information shows in heroku pg:info
:
$ heroku pg:info
=== DATABASE_URL, HEROKU_POSTGRESQL_PURPLE_URL
...
Fork/Follow: Available
...
The lag between a leader and follower databases varies greatly depending on the amount and frequency of data updates. It’s possible for long-running queries on the follower to increase your lag time, though after those queries are done your follower catches up. Under normal usage, it’s common for your follower to be within a few seconds or less of your leader.
To create a follower database you must first know the add-on name of the leader database. Use heroku pg:info
to find its HEROKU_POSTGRESQL_*COLOR*_URL
name.
$ heroku pg:info
=== DATABASE_URL, HEROKU_POSTGRESQL_CHARCOAL_URL
Plan: Standard 0
Status: available
...
If more than one database is listed, the one currently serving as the leader will most often be the one assigned to the DATABASE_URL
(listed after the database name).
Create a follower database by provisioning a new heroku-postgresql Standard-tier or higher add-on database and specify the leader database to follow with the --follow
flag. The flag can take either the config var name of the database on the same app, an argument of the form appname::HEROKU_POSTGRES_COLOR_URL
, or the full URL of any Heroku Postgres database.
Followers don’t have to be the same database plan as their leader. Followers can be up to 4 plans below their leader’s plan, unless the follower’s plan has burstable characteristics (Postgres Plan Characteristics), in which case the follower can only be up to 1 plan below their leader’s plan. It’s recommended that unless you’re downgrading plans, a follower be created using the same or higher plan than the leader database.
The addons:create
example follows the syntax for Heroku CLI v9.0.0 or later. If you’re on v8.11.5 or earlier, use the command:
$ heroku addons:create heroku-postgresql:standard-2 --follow HEROKU_POSTGRESQL_CHARCOAL_URL
$ heroku addons:create heroku-postgresql:standard-2 -- --follow HEROKU_POSTGRESQL_CHARCOAL_URL
Adding heroku-postgresql:standard-2 to example-app... done, v71 ($200/mo)
Attached as HEROKU_POSTGRESQL_WHITE
Follower will become available for read-only queries when up-to-date
Use `heroku pg:wait` to track status
Preparing a follower can take anywhere from several minutes to several hours, depending on the size of your dataset. The heroku pg:wait
command outputs the provisioning status of any new databases.
$ heroku pg:wait
Waiting for database HEROKU_POSTGRESQL_WHITE_URL... available
Creating a Follower via the Heroku Data Dashboard
You can also create followers through the web dashboard:
- Go to data.heroku.com.
- Use the search and select the database you want to create a follower from.
- Click the
Durability
tab. - Scroll down to the Followers section.
- Click the
Add a Follower
button. - Choose the plan for the follower. Review the Create a follower section for notes on selecting a plan size.
- Click the
Add follower
button.
The dashboard shows the status of the follower database and updates when the provisioning is complete.
Create a Follower on a Different App
Followers don’t need to be tied to the same application. With sharable add-ons, the databases can be on two separate applications. The steps are the same as the steps outlined for creating a follower on the same application with a few slight differences.
First, use heroku pg:info
to find the resource name of the database that you want to follow:
$ heroku pg:info -a example-app
=== DATABASE_URL, HEROKU_POSTGRESQL_CHARCOAL_URL
Plan: Standard 0
Status: available
Add-on: looking-simply-2449
...
After the resource name has been found, type the add-ons command to create the follower on the app that you want it attached to. For example, if a second application is called other-app
and I want to have a follower attached to it using the main database from example-app
:
The addons:create
example follows the syntax for Heroku CLI v9.0.0 or later. If you’re on v8.11.5 or earlier, use the command:
$ heroku addons:create heroku-postgresql:standard-0 --follow looking-simply-2449 -a other-app
$ heroku addons:create heroku-postgresql:standard-0 -a other-app -- --follow looking-simply-2449
Unfollow
The heroku pg:unfollow
command stops the follower from receiving updates from its leader database and transforms it into a full read/write database containing all of the data received up to that point. This command creates a database fork.
$ heroku pg:unfollow HEROKU_POSTGRESQL_WHITE_URL
! HEROKU_POSTGRESQL_WHITE_URL will become writable and no longer
! follow HEROKU_POSTGRESQL_CHARCOAL. This cannot be undone.
! WARNING: Potentially Destructive Action
! This command will affect the app: example-app
! To proceed, type "example-app" or re-run this command with --confirm example-app
> example-app
Unfollowing... done
Unfollowing a database isn’t the same as de-provisioning it. You’re still charged for the database. To completely de-provision a database, use the heroku addons:destroy HEROKU_POSTGRESQL_WHITE
command.
Database Upgrades and Migrations with Changeovers
In addition to providing data redundancy, followers can also be used to change database plans, underlying infrastructure, or minor versions with minimal downtime. To update your database with a follower, see this guide on updating Heroku Postgres.
Monitoring Followers
Because followers are asynchronously updated, they can be behind their leader by some number of commits. You can view the number of commits your follower is behind by running heroku pg:info
. If your follower is increasing in the number of commits it’s behind, it can be due to long-running transactions on your database. Using the pg-extras plugin, you can run heroku pg:ps
to get currently running transactions. Then if any have been running for longer than expected, you can cancel those transactions with:
$ heroku pg:kill PROCESSID
If you aren’t able to cancel them, you can try terminating them with the -f
parameter.
Heroku Postgres has an automated monitoring system for followers. It checks if the follower is more than 64 write-ahead log segments (1024 MB) behind. If the lag is caused by long-running transactions/queries, we notify you via email that you have some transactions/queries that are causing the lag.
If the situation doesn’t improve after one hour, we terminate any long-running transactions/queries automatically. You see a log entry of terminating connection due to administrator command
for terminated transactions/queries.
High Availability with Followers
As a general practice, use a hot standby for high availability:
- For primary Standard databases, create your own standby by adding a follower. Even if not being actively used as a read replica, creating a follower ensures that it’s available for promotion in situations where the primary database becomes corrupted or unavailable.
- Primary databases on Heroku Postgres Premium, Private, and Shield plans have the HA feature with automated failover which creates a hidden standby for you. There’s no need to create a follower to achieve high availability.
Follower databases don’t have a hidden standby. If you require high availability on a follower itself, in addition to the primary database, provision multiple followers.
Follower databases are guaranteed to be provisioned on geographically separate infrastructure than the primary database.
Manually Promoting Followers in a Failover Event
Heroku doesn’t automatically promote a follower database when the primary database is corrupt or inaccessible. If this functionality is required, use a premium or enterprise tier plan that does offer HA. Performing a manual database failover is the same process as a database migration starting with the enter maintenance mode step.
Distributing App Reads to Followers
Heroku allows you to easily horizontally scale your app by adding additional dynos to meet capacity. Similarly, as detailed earlier, Heroku Postgres allows you to horizontally scale your database by adding read-only followers to the lead database. While these followers are great for analytical purposes, you can also use them via your application for handling read-only queries to your data. This type of architecture can be used to improve app performance as well as work around Heroku Postgres connection limits.
Redirecting your app’s read operations to one or more followers requires changes at the code level of your app. Since each app on Heroku is different, it’s not possible to recommend how to proceed. If your app is built using a framework, it’s likely the framework has support for using multiple databases. In other cases, there are third-party libraries available to help with the handling of multiple databases. In some cases, especially as your app gets more complex, writing custom code is necessary.