Importing and Exporting Heroku Postgres Databases with PG Backups
Last updated 02 March 2018
On the surface, PG Backups provides a way to capture regular backups of your Heroku Postgres database. However, because of its general-purpose architecture and use of standard PostgreSQL utilities, it is also a useful tool capable of exporting to or importing from external PostgreSQL databases.
An alternative to using the dump and restore method of import/export if you have a Postgres instance on your local machine is to use the pg:push and pg:pull CLI commands to automate the process.
Export
PG Backups uses the native pg_dump PostgreSQL tool to create its backup files, making it trivial to export to other PostgreSQL installations. Note that the resulting backup file uses the custom format option in pg_dump. As compared to the plain-text format, the custom format options results in backup files that can be much smaller.
In general, PGBackups are intended for moderately loaded databases up to 20 GB. Contention for the I/O, memory and CPU needed for backing up a larger database becomes prohibitive at a moderate load and the longer run time increases the chance of an error that will end your backup capture prematurely. For databases that are larger than 20 GB, physical backups and continuous protection are recommended.
Download backup
To export the data from your Heroku Postgres database, create a new backup and download it.
$ heroku pg:backups:capture
$ heroku pg:backups:download
Restore to local database
Load the dump into your local database using the pg_restore tool. If objects exist in a local copy of the database already, you might run into inconsistencies when doing a pg_restore. Pg_restore does not drop all of the objects in the database when loading the dump.
This will usually generate some warnings, due to differences between your Heroku database and a local database, but they are generally safe to ignore.
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump
If you are using an old version of pg_restore, you may see an error such as pg_restore: [archiver] unsupported version (1.13) in file header when you try to run pg_restore. Please make sure that you are using pg_restore version 10.3, 9.6.8, 9.5.12, 9.4.17, and 9.3.22 or up. You can check the version of your pg_restore by running pg_restore --version.
Import
PG Backups can be used as a convenient tool to import database dumps from other sources into your Heroku Postgres database.
If you are importing data as part of the initialization of a new application you will need to first create and configure the app on Heroku before performing the import.
Create dump file
Dump your local database in compressed format using the open source pg_dump tool:
$ PGPASSWORD=mypassword pg_dump -Fc --no-acl --no-owner -h localhost -U myuser mydb > mydb.dump
Import to Heroku Postgres
In order for PG Backups to access and import your dump file you will need to upload it somewhere with an HTTP-accessible URL. We recommend using Amazon S3. Wherever you place the file, ensure that the file is publicly accessible. This may require changing the permissions on the uploaded file. For security, we recommend obscuring the filename and also removing the file once the import has completed.
Note that the pg:backups restore command drops any tables and other database objects before recreating them.
Use the raw file URL in the pg:backups restore command:
$ heroku pg:backups:restore 'https://s3.amazonaws.com/me/items/3H0q/mydb.dump' DATABASE_URL
DATABASE_URL represents the HEROKU_POSTGRESQL_COLOR_URL of the database you wish to restore to. You must specify a database configuration variable to restore the database.
If you’re using a Unix-like operating system be sure to use single quotes around the temporary S3 URL, because it might contain ampersands and other characters that will confuse your shell. If you’re running Windows, you must use double-quotes.
When you have completed the import process, delete the dump file from its storage location if it’s no longer needed.