Heroku Connect API
Last updated November 13, 2020
Table of Contents
- Heroku CLI plugin
- Endpoints
- Step 1: Create a Heroku app and Heroku PostgreSQL database
- Step 2: Create the Heroku Connect add-on
- Step 3: Link the new add-on to your Heroku user account
- Step 4: Configure the database key and schema for the connection
- Step 5: Authenticate the connection to your Salesforce Org
- Step 6: Import a mapping configuration
- Step 7: Monitor the connection and mapping status
- Additional endpoints
Heroku Connect provides an API to automate the creation, maintenance, and monitoring of sync operations between Salesforce and a Heroku PostgreSQL database. This tutorial is an overview that describes how to use the Heroku Connect API to provision a new connection to synchronize data between a Salesforce org and a Heroku PostgreSQL database.
Heroku CLI plugin
Heroku provides a very full-featured command-line interface via the Heroku CLI. A good way to work with the Heroku Connect API is to use the Heroku Connect CLI plugin in the CLI.
$ heroku plugins:install heroku-connect-plugin
The CLI plugin is useful for scripting Heroku Connect and offers some demonstration on how to make Heroku Connect API calls.
Endpoints
Heroku Connect operates in many regions, each responding on a different domain. The root API URLs are as follows:
Region | Root API URL |
---|---|
Dublin (EU) | https://connect-dublin.heroku.com/api/v3 , orhttps://connect-2-dublin.heroku.com/api/v3 (see note below) |
Frankfurt | https://connect-frankfurt.heroku.com/api/v3 |
Oregon | https://connect-oregon.heroku.com/api/v3 |
Sydney | https://connect-sydney.heroku.com/api/v3 |
Tokyo | https://connect-tokyo.heroku.com/api/v3 |
Virginia (US) | https://connect-virginia.heroku.com/api/v3 ,https://connect-2-virginia.heroku.com/api/v3 , orhttps://connect-3-virginia.heroku.com/api/v3 (see note below) |
In the Virginia (US) and Dublin (EU) regions, the root API URL will depend on the cell that your connection is in. You can find the base URL of the cell by going to the connection’s Manage Connection page and using the Cell URL value.
Previously, we listed the Root API URL of the EU region as https://connect-eu.heroku.com/api/v3
and the Root API URL of the US region as https://connect-us.heroku.com/api/v3
. These are aliases of the Dublin (EU) and Virginia (US) URLs, respectively, and will continue to work.
You must use the API endpoint that corresponds to the region your app resides in, which you can check by using the Heroku CLI.
$ heroku info -a <app_name>
=== <app_name>
Git Url: git@heroku.com:<app_name>.git
Web Url: https://<app_name>.herokuapp.com/
Addons: heroku-postgresql:standard-0
herokuconnect:enterprise
…
Region: us
…
Select the endpoint prefix for a central location for your app; this can later be changed if your needs expand into multiple regions. Examples in this tutorial will assume the US region.
Authentication
The Heroku Connect API requires a Heroku Platform API direct authorization token. This token is for your own application’s Heroku user only. Heroku Connect does not support API access on behalf of other Heroku customers.
Supply this token with the Authorization
header, like this:
Authorization: Bearer <token>
JSON encoding
All responses are JSON-encoded, as are POST
/PATCH
request payloads. You must supply the Content-Type
header on all such requests:
Content-Type: application/json
Request methods
Most of the Heroku Connect API responds to either GET
or POST
requests, but there are some cases where we use other methods, including PUT
, DELETE
and PATCH
. Most HTTP libraries support these, but some environments, including Apex code on the Force.com platform, only support a subset. In order to use these additional endpoints, it’s possible to use a POST
request with an additional header to indicate what HTTP method you’re really trying to use.
X-HTTP-Method-Override: PATCH
Within the API, this will work just like if you had used the real HTTP method itself, but this can be used even when those methods are unavailable to you.
Rate limits
The API is rate-limited at 5,000 requests per connection per day. Attempting to access the API after this limit has been reached will result in 429 Too Many Requests
. The response message will also contain information about when to expect to be able to make another API request.
{"message":"Request was throttled. Expected available in 72170 seconds."}
Compatibility policy
The Heroku Connect API is under regular development, and changes will occur over time. Some changes, such as new endpoints, new input or new payload content, will not affect existing behavior, and may therefore happen at any time. But other changes will modify or remove existing behavior, so to help ensure compatibility with existing applications, we have a policy in place for those incompatible changes.
For stable endpoints, we will communicate any upcoming incompatible changes to all users at least 30 days prior to their deployment. This notification will include the nature of the change, the reasoning behind it, and any applicable steps to be taken to migrate from the old behavior to the new behavior. These details will also be added to this document for the 30-day duration. Once the 30 days have passed, the new behavior will go live and the old behavior will be removed from this document.
Unless otherwise stated, all documented API endpoints are considered stable. Any experimental endpoints or behaviors will be identified as such in this document. Experimental features are made available in order to get feedback from users of the API, and may change at any time, with no guaranteed notice.
The API is currently at version 3. When a new version is released, it will be made available as an experimental feature until it’s ready to replace the existing version. At that time, we will change the base endpoint listed above, and communicate the same 30-day notice as with other features, before removing the old version.
Step 1: Create a Heroku app and Heroku PostgreSQL database
You’ll need to have created a Heroku app, and added a Heroku PostgreSQL database add-on to that app. The free database tier is sufficient for basic testing, but you’ll need to upgrade to a paid plan before importing a significant amount of data, in order to avoid reaching the database row limits.
$ heroku apps:create <app_name>
$ heroku addons:create heroku-postgresql -a <app_name>
Step 2: Create the Heroku Connect add-on
Use the Heroku Platform API to add the herokuconnect
add-on to your app, or create the add-on via the CLI:
$ heroku addons:create herokuconnect -a <app_name>
Step 3: Link the new add-on to your Heroku user account
By default, no users are present on a Heroku Connect add-on. This means no user can access it. heroku addons:open herokuconnect -a <app_name>
will add a user if Heroku Connect can determine they’re a member with proper permissions. This, however, opens https://connect.heroku.com
in a browser. To avoid that, you need to use the Heroku Connect API to add your Heroku user account to the access list:
POST https://hc-central.heroku.com/auth/<app_name_or_app_uuid>
$ curl -X POST -H "Authorization: Bearer <token>" https://hc-central.heroku.com/auth/<app_name_or_app_uuid>
This will return all Connect add-ons across all regions that you’re known to have access to, including your newest connection. You can filter it with a tool like jq
. This data includes a details_url
field which allows you to interact with that connection via the Connect API.
You can verify this was successful with the Connect CLI plugin:
$ heroku connect:info -a <app_name>
=== Connection [<connection_id>] / <resource_name> (NEW)
No mappings
Step 4: Configure the database key and schema for the connection
Now, configure the connection with the config var name of your database (typically DATABASE_URL
) and an unused schema name in which to place Salesforce data (typically salesforce
).
PATCH /connections/<connection_id>
$ curl -X PATCH -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{"schema_name": "salesforce", "db_key": "DATABASE_URL"}' https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>
{
"id": "<connection_id>",
"name": "<app_name>",
"resource_name": "<resource_name>",
"schema_name": "salesforce",
"db_key": "DATABASE_URL",
…
}
Or, using the CLI plugin:
$ heroku connect:db:set --schema salesforce --db DATABASE_URL -a <app_name>
Configuring connection with
{ schema_name: 'salesforce', db_key: 'DATABASE_URL' }
Setup complete
Step 5: Authenticate the connection to your Salesforce Org
Now that you’ve established the link to the database, you need to authenticate Heroku Connect to talk to your Salesforce Org. This is done by retrieving the authorization URL, and then having the Salesforce user open this URL to complete the authentication. In a typical application you could retrieve the authorization URL and link it to a button to show to the user.
POST /connections/<connection_id>/authorize_url
Available JSON parameters:
environment
:production
,sandbox
, orcustom
[defaults tosandbox
]domain
: specify a custom login domain (if using acustom
environment)api_version
: specify a Salesforce API version to use [defaults to the latest supported version]next
: Final URL to redirect the user [defaults to the Heroku Connect dashboard]
$ curl -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" -d '{"environment": "production"}' https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/authorize_url
{"redirect": "https://connect-3-virginia.heroku.com/oauth/start/…"}
Or, using the CLI plugin:
$ heroku connect:sf:auth -a <app_name>
fetching authorizing URL... done
If your browser doesn't open, please copy the following URL to proceed:
https://connect-3-virginia.heroku.com/oauth/start/…
There is not a programmatic way to complete this step. Salesforce authentication requires manually visiting the URL and authenticating with both Heroku and Salesforce.
Step 6: Import a mapping configuration
Now that the connection is set up, the easiest way to create mappings to synchronize data from Salesforce is to import the JSON mapping configuration exported from an existing connection. You can export a configuration file from the Heroku Connect dashboard, by clicking Settings
, then click Import/Export Configuration
, then click the Export Configuration
button. This endpoint accepts the configuration details either as a standard file upload or directly as JSON in the request body.
POST /connections/<connection_id>/actions/import
$ curl -X POST -H "Authorization: Bearer <token>" -F "config=@config.json" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/actions/import
$ curl -X POST -H "Authorization: Bearer <token>" -H "Content-Type: application/json" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/actions/import -d @config.json
Or, using the CLI plugin:
$ heroku connect:import <exported_config.json> -a <app_name>
Upload complete
Step 7: Monitor the connection and mapping status
Use the connection detail API endpoint with the “deep” argument to retrieve connection status and mapping status information in one call.
GET /connections/<connection_id>
Available query string parameters:
deep
:true
will return information about the connection’s mappings, in addition to the connection itself [defaults tofalse
]
$ curl -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>?deep=true
{
"id": "<connection_id>",
"name": "<app_name>",
"resource_name": "<resource_name>",
"schema_name": "salesforce",
"db_key": "DATABASE_URL",
"state": "IDLE",
"mappings": [
{
"id": "<mapping_id>",
"object_name": "Account",
"state": "SCHEMA_CHANGED",
…
},
{
"id": "<mapping_id>",
"object_name": "Contact",
"state": "SCHEMA_CHANGED",
…
},
…
]
…
}
Or, using the CLI plugin:
$ heroku connect:info -a <app_name>
Connection [<connection_id> / <resource_name>] (IDLE)
--> Account (SCHEMA_CHANGED)
--> Contact (SCHEMA_CHANGED)
…
Additional endpoints
Once you have your connection up and running, there are some additional operations available to help maintain the connection.
Pause the connection
An active connection can be paused, after which it won’t move any data in either direction.
POST /connections/<connection_id>/actions/pause
$ curl -X POST -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/actions/pause
Or, using the Heroku CLI plugin:
$ heroku connect:pause -a <app_name>
Connection [<connection_id> / <resource_name>] paused
Once the connection is paused, it can be resumed using a related endpoint.
POST /connections/<connection_id>/actions/resume
$ curl -X POST -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/actions/resume
Or, using the Heroku CLI plugin:
$ heroku connect:resume -a <app_name>
Connection [<connection_id> / <resource_name>] resumed
Restart the connection
You can restart a connection at any time, which will clear system errors and attempt to sync data again. If the errors have been resolved, this will get the connection back in working order. Otherwise, the connection and its mappings may revert to an error state when errors are next encountered.
POST /connections/<connection_id>/actions/restart
$ curl -X POST -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/connections/<connection_id>/actions/restart
Or, using the Heroku CLI plugin:
$ heroku connect:recover -a <app_name>
recovering connection... done
Reload a mapping
If a table in your PostgreSQL database gets out of sync with Salesforce for any reason, you can use the API to clear out the Postgres table and pull fresh data from Salesforce. This requires a mapping ID, which can be obtained by using deep=true
on the connection detail endpoint, as described in Step 7 above.
POST /mappings/<mapping_id>/actions/reload
$ curl -X POST -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/mappings/<mapping_id>/actions/reload
This command is not yet implemented in the Heroku CLI plugin.
Get mapping details
GET /mappings/<mapping_id>
$ curl -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/mappings/<mapping_id>
The response contains configuration details for the mapping, current status and additional URLs for obtaining related information.
{
"access": "read_only",
"actively_writing": false,
"config": {
"access": "read_only",
"fields": {
"CreatedDate": {},
"Id": {},
"IsDeleted": {},
"SystemModstamp": {}
},
"indexes": {
"Id": {
"unique": false
},
"SystemModstamp": {
"unique": false
}
},
"revision": 12345,
"sf_notify_enabled": true,
"sf_polling_seconds": 600
},
"connection": {
"id": "<connection_id>"
},
"counts": {
"db": 0,
"errors": 0,
"pending": 0,
"sf": 0
},
"detail_url": "<url>",
"id": "<mapping_id>",
"object_name": "<object_name>",
"state": "DATA_SYNCED",
"state_description": "OK",
"times": {
"db_poll": "<date>",
"db_write": "<date>",
"sf_poll": "<date>",
"sf_write": "<date>",
},
}
Create a new mapping
In addition to being able to import an entire configuration at once, you can add a single mapping at a time to an existing connection.
POST /connections/<connection_id>/mappings
The JSON payload of this API endpoint contains detailed configuration for the mapping itself:
object_name
: the name of the Salesforce object to map
The remaining details are provided in an object attached to the config
attribute, matching the output format for the mapping detail above.
access
: indicates whether the mapping should be configured for read-only access from Salesforce or read-write in both directionsread_only
: data will flow only from Salesforce to PostgreSQLread_write
: data will flow in both directions
fields
: an object representing the fields to map for the object. Each field name is an attribute on the object, with its value being an empty object. In the future, we may accept additional options for each field in the mapping.indexes
: an object representing the indexes to include for the object. Each index is provided as an attribute whose name is the field name, and the value is an object with additional settings:unique
: a Boolean indicating whether the index should require unique values
Example payload:
{
"object_name": "Account",
"config": {
"access": "read_write",
"sf_notify_enabled": true,
"sf_polling_seconds": 600,
"fields": {
"CreatedDate": {},
"Id": {},
"IsDeleted": {},
"Name": {},
"SystemModstamp": {}
},
"indexes": {
"Id": {
"unique": false
},
"SystemModstamp": {
"unique": false
}
}
}
}
If the values provided are valid, the response code to this endpoint will be a 201 Created, with full details for the created mapping in the payload of the response. The response payload will be the same as the output from the mapping detail listed above.
If there were any errors in the input, the response code will be 400 Bad Request, with details about the errors in the JSON payload for the response.
Edit an existing mapping
Similar to creating a new mapping, you can use the API to edit an existing mapping as well.
PUT /mappings/<mapping_id>
The payload is nearly identical to what you’d send when creating a new mapping. The difference is the presence of a revision
attribute. Existing mappings will have a revision
attribute in their mapping details, which can be retrieved via our API. The revision
value needs to be sent back, unchanged, in the PUT
request. This allows the API to ensure that the mapping hasn’t been updated by someone else before you had a chance to post it back. The ideal workflow would be to retrieve the details for the mapping, extract the config
attribute, make appropriate changes, then PUT
it back to the same URL.
Like the creation endpoint, most errors will return a 400 Bad Request with details included in the response payload. If the revision
sent back doesn’t match what’s current for the mapping, a 409 Conflict will be returned, to prevent multiple people (or processes) from modifying the same mapping in conflicting ways. Successful responses will be 200 OK, with the entire mapping payload included.
Delete an existing mapping
Mappings can also be deleted individually.
DELETE /mappings/<mapping_id>
$ curl -X DELETE -H "Authorization: Bearer <token>" https://connect-3-virginia.heroku.com/api/v3/mappings/<mapping_id>
This endpoint simply returns with a 204 No Content indicating that the mapping was deleted. Additional tasks, including removing data from your database, may still run after this response returns.