Heroku Connect Logging and Maintenance Operations
Last updated 28 September 2020
Table of Contents
Application logging
The format of log messages is subject to change in future releases of Heroku Connect.
Heroku Connect emits log messages to Logplex making them available to your application via the CLI or using logging add-ons.
Log entries follow the standard log format with a source of app
, a dyno of herokuconnect
and a message that contains structured data including standard fields such as event
and level
.
An example of the Logplex entries for a reload of the Contact
mapping is shown below:
$ heroku logs -d herokuconnect
2018-11-08T20:05:56 app[herokuconnect]: event="RELOAD TABLE contact" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=mapping object_id=7a63b0c9-7145-4717-8193-72881b55c434 state=RELOAD_TABLE level=info
2018-11-08T20:05:56 app[herokuconnect]: event="Cleared table contact for load" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=mapping object_id=7a63b0c9-7145-4717-8193-72881b55c434 state=RELOAD_TABLE level=info
2018-11-08T20:05:56 app[herokuconnect]: event="Loading table contact via SOAP" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=mapping object_id=7a63b0c9-7145-4717-8193-72881b55c434 state=RELOAD_TABLE level=info
2018-11-08T20:05:57 app[herokuconnect]: event="Contact QUERY ↷SALESFORCE 500 rows (0.77 secs)" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=sync object_id=82b1c244-592d-4165-99e7-89a270459752 state=IDLE level=info
2018-11-08T20:06:04 app[herokuconnect]: event="Contact INSERT ↓DATABASE 500 rows (0.02 secs)" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=sync object_id=82b1c244-592d-4165-99e7-89a270459752 state=IDLE level=debug
2018-11-08T20:06:05 app[herokuconnect]: event="RELOAD COMPLETE" addon_id=82b1c244-592d-4165-99e7-89a270459752 object_type=mapping object_id=7a63b0c9-7145-4717-8193-72881b55c434 state=RELOAD_TABLE level=info
For information about specific log messages, please refer to our Log Messages article.
Changing the plan of your Heroku Postgres Database
If you want to change the plan of your Heroku Postgres Database, you can use the heroku addons:upgrade command to do so. Heroku Connect is able to handle this operation transparently since it behaves identically to an HA failover.
While not recommended, you could use the Follower Changeover method instead. If you wish to do that then you would need to take additional steps as those documented in the section below, omitting step 5.
If you’re upgrading from a hobby tier database, you won’t be able to create a follower. Instead, use the PG Copy process instead to copy data into the new database. Simply pause Heroku Connect at the same time as you put your application into maintenance mode, and resume it again when you reactivate your application.
Upgrading the Version of a Heroku Postgres Database
Heroku Connect operates continuously on a single database, which requires a few steps in order to upgrade the version of that database and allow sync operations to continue as expected. The upgrade itself is done using a follower database and pg:upgrade, and these instructions illustrate how Heroku Connect fits into that process.
These steps ensure that the new database is an exact copy of your current database. Heroku Connect will not create tables or transfer data to an empty database, so be sure to follow the proper upgrade process.
Create your follower database and allow it to get mostly caught up with your lead database. This will minimize the amount of downtime required for the upgrade.
Pause your connection for the same reason that you put your application into maintenance mode, to prevent Heroku Connect from trying to write to your database during the upgrade.
Wait for your follower to catch up. Now that Heroku Connect and your application have both stopped writing to your database, the follower can get completely caught up. You can check its status using
heroku pg:info
.Upgrade the follower database using
pg:upgrade
. This will have the follower un-follow the leader and perform the Postgres Version upgrade.You now need to promote the follower database to become the new lead database. The steps you need to take depend on whether your Heroku Connect is using a standard database URL (
DATABASE_URL
) or a custom database URL (e.g.HEROKU_POSTGRESQL_ROSE_URL
).- If you’re using a standard database URL, Promote the follower to become the new lead database. This will make it accessible as
DATABASE_URL
. - If you’re using a custom database URL, you will need to update the Heroku Connect Database Settings with the new custom database URL for the promoted follower or use heroku addons:attach to attach the database to a name of your choosing.
- If you’re using a standard database URL, Promote the follower to become the new lead database. This will make it accessible as
Resume Heroku Connect using the same process used to pause it in step 2, and bring your application out of maintenance mode. Heroku Connect will automatically pick up the new
DATABASE_URL
for your application ( or customer database URL, if you have that configured ) and resume synchronization using the new database.
Pausing synchronization
When performing certain tasks, such as upgrading your database, you may want to pause synchronization activity in Heroku Connect. To do this navigate to the Settings
tab, Manage Connection
and click the Pause
button.
Heroku Connect will complete any pending operations before entering the paused state. When your connection is paused then changes to data in the database will continue to be added to the trigger log and changes made in Salesforce will not be polled. Push Topic notifications from for mappings using the streaming mode will also be ignored.
To resume synchronization return to the Settings
tab, Manage Connection
and click the Resume
button. Heroku Connect will continue where it left off: entries in the trigger log will be processed and pushed to Salesforce and changes made in Salesforce will be retrieved using polling (including those mappings that would normally use the Streaming API).
Any changes made to data in the database or in Salesforce while your connection is paused should automatically be synchronized when your connection is resumed.
Recreating a connection
If you’ve reloaded a sandbox organization or you need to upgrade from a hobby database to a professional one, you’ll need to delete your existing connection and create a new one.
The following settings are manually configured:
- Collaborators and their notifications settings
- Beta features
- The Write Algorithm
- The Salesforce API version
The rest of the recreation can be automated as follows:
- Export your configuration from the Connect Dashboard. Click
Settings
, then clickImport/Export Configuration
, then click theExport Configuration
button. - If you have any read/write mappings, make sure you aren’t writing any new data to those tables, and make sure all the mappings are in
DATA_SYNCED
to ensure all your data is present in Salesforce. You may need to clickSettings
,Manage Connection
and click thePause
button to stop activity on the connection. - Destroy the Connection from the heroku CLI
heroku addons:destroy -a appname herokuconnect-resource-name
- Create a new connection from https://elements.heroku.com or
heroku addons:create herokuconnect
from the command line. - Authenticate your new connection, and import the config you exported from step 1.
Review apps
Heroku Connect does not support review apps.
Heroku Connect does not currently support review apps. It is, however, possible to manually provision Heroku Connect once the review app has been created.
Changing Field Definitions in Salesforce
Field definition changes can delay synchronization with Heroku Connect if not coordinated properly.
Heroku Connect does not automatically change the table schema in PostgreSQL after you’ve changed the corresponding field definition in Salesforce. This is because attempting to change the field may result in data loss or corruption.
After changing a field in Salesforce, users must edit the object mapping in Heroku Connect to trigger an update to the PostgreSQL table schema by Heroku Connect. In simpler cases, all users need to do is click Edit
for the mapping and then click Save
. In more complex cases, users will need to deselect the field, save the mapping, edit the mapping again, and then re-add the field. This will cause Connect to drop the column, re-add it, and load the data back into the column.
Simpler cases include extending or shortening the length of a field. More complex cases include changing the types of fields.