Migrating to V3 of the Add-on Partner API

Last updated 07 January 2019

If you built an add-on before May 2018 (the date we made Add-on Partner API V3 the default version), you’re already familiar with V1 of the Add-on Partner API. V3 of the Add-on Partner API fixes a number of problems inherent to V1 and introduces better asynchronous patterns.

This article will explain the differences between V1 and v3 and give you a possible path forward.

Summary of Differences between V1 and V3

V3 of the Add-on Partner API:

  • Encourages partners to support asynchronous add-on provisioning,
  • Enables use of the Platform API for Partners by default,
  • Versions request via an Accept header as described here,
  • Does not include the ambiguous heroku_id in provisioning, plan change, and other requests,
  • Includes a uuid in the provisioning request that’s used to identify add-on resources unambiguously,
  • The callback_url in the provision request is a url for the Platform API for Partners, rather than the legacy /vendor namespace,
  • SSO integration uses a different token and the new uuid instead of the provider_id created by add-on partners.

Why we’ve made these changes

All the changes are focused on improving clarity, security, and implementing more modern distributed computing patterns. We’ll talk about why we made some of these changes below, to help you understand why we think they’re important.

Asynchronous Provisioning

Some add-on partners have provisioning processes that take longer than a few seconds to coordinate, and asynchronous provisioning is made for them.

The new V3 api includes asynchronous provisioning of add-ons as a well-defined first-class feature. We may implement asynchronous plan changes and deprovisionings in future API releases as well.

We had a few different ad-hoc asynchronous provisioning patterns under our legacy V1 API, and those will continue to be supported for the near future.

The Platform API for Partners

The Platform API for Partners allows add-ons to access a subset of our main Platform API via scoped OAuth tokens. This allows add-on partners to integrate more deeply and securely into the Heroku ecosystem, and replaces our legacy App Info API. It’s the preferred way to update add-on config vars and retrieve app, collaborator, attachment and other add-on resource specific metadata.

You can use the Platform API for Partners under legacy V1 integrations. You might consider backfilling OAuth tokens and other metadata via the Platform API for Partners before undertaking your V3 migration.

Versioning requests via HTTP headers

Versioning requests via HTTP headers as described here brings our Add-on Partner API in line with how other public Heroku APIs work, improving consistency.

The inclusion or exclusion of the new Accept: application/vnd.heroku-addons+json; version=3 HTTP header allows you to distinguish between v1 and v3 requests to your integration API endpoints.

Deprecation of heroku_id, provider_id and inclusion of UUIDs

There was frequent confusion from add-on partners about how to uniquely identify resources under the legacy V1 api - heroku_id? provider_id? Something else?

Additionally, the heroku_id identifier does not allow for disambiguation if your add-on supports many_per_app and there are multiple resources on an application.

We’ve clarified identifying resources uniquely in V3 with a uuid we provide for each resource, sent during provisioning. We then include this resource specific uuid in all subsequent add-on request paths and JSON payloads.

Refactor SSO to use the new UUID

We’ve updated our single sign-on implementation to use the new uuid under V3, rather than the legacy provider_id.

Steps to upgrade from V1 to V3

Upgrading from V1 to V3 is a multi-step process that needs to be orchestrated carefully. We hope to present a way forward that should minimize risk, downtime, and engineering effort.

Create a staging version of your add-on

You should create a separate <your add-on slug>-staging version of your add-on integration endpoints to test the V1 to V3 upgrade process, as it can be fairly involved. To do this:

  1. Deploy a staging instance of your provisioning infrastructure,

  2. Copy your existing add-on manifest file via the addons-admin CLI plugin.

    $ cp addon-manifest.json <add-on slug>-staging/
$ cd <add-on slug>-staging/
$ # Open `addon-manifest.json` in your favorite editor and modify id (the slug), the password,
$ # the sso_salt, and URLs match your staging add-on API endpoints and infrastructure.
$ # It's important to change the "id" attribute or you could interrupt your
$ # existing add-on infrastructure.
$ # Finally, use the addons-admin CLI plugin to push your manifest.
$ heroku addons:admin:manifest:push

  3. Replicate a set of common add-on provisioning scenarios. You might create a few test apps and provision your staging add-on to these apps. If your app is attachable, you might create a few attachments as well. The Heroku CLI or direct Platform API calls can help script creating this testing environment.

Please contact Ecosystem support if you’d like assistance configuring your staging add-on for parity with your production add-on.

Start capturing additional attributes during provisioning

We POST a few new attributes during provisioning that you should capture in addition to the plan, region, and others. Here are a few V3 specific attributes of particular interest:

  • uuid - the UUID we use in lieu of the deprecated heroku_id and provider_id values to uniquely identify an add-on resource,
  • oauth_grant:code - Used to exchange for a resource scoped OAuth token (described in more detail below),
  • oauth_grant:expires_at - A timestamp defining when the oauth_grant:code above expires. You have 5 minutes in which to exchange your grant code.

Backfill the add-on resource uuid

You can do this in at least two ways:

  1. Via the legacy Get All Apps API call under the resource.uuid value in the JSON payload, or
  2. By backfilling OAuth access tokens and then using the Platform API for Partners add-on resource related endpoints.

Implement OAuth token exchange

You’ll need to implement a grant code exchange to start taking advantage of the Platform API for Partners. You might implement this as part of capturing additional provisioning attributes.

You must use the Platform API for Partners if you want to use asynchronous provisioning, and this requires that you implement OAuth token exchange.

Backfill the OAuth access and refresh tokens

Backfill the access and refresh tokens for each add-on resource so you can begin using the Platform API for Partners. This requires that you submit a ticket so that we can temporarily open a backfill endpoint for you.

Replace your calls to the App Info API

Replace your calls to the legacy Add-on Partner App Info API with calls to the new Platform API for Partners via the OAuth tokens you’ve exchanged. Your calls for additional information about add-ons and their environment will be more secure as they’re using a token scoped to the context of the add-on itself.

Implement Asynchronous Provisioning

If you have a provisioning process that takes longer than a few seconds, you should implement asynchronous provisioning. Under asynchronous provisioning, you:

  1. Return a 202 Accepted response status code to a provisioning request while storing relevant metadata
  2. Provision your add-on resource in the background
  3. Update config vars for your add-on instance with the appropriate values
  4. Mark your add-on as provisioned

The last two steps are accomplished via the Platform API for Partners.

You can mix asynchronous and synchronous provisioning - the defining factors are:

  1. That you’re using a V3 integration, and
  2. The response status code.
    • 200 OK means the add-on instance is available immediately, and you should send config vars in the response.
    • 202 Accepted means you’re provisioning asynchronously and will update in a separate request with config vars. You’ll also send another request to mark the add-on as provisioned and therefore available for use.

Final migration plan

After you’ve tested your migration strategy thoroughly in your staging add-on integration environment, you should be ready to roll out integration with our V3 API to your customers.

Below is one possible migration strategy involving minimal downtime:

  1. Reject all provision, deprovision, and plan change requests during the migration period with a 503 - Service Unavailable status code. We’ll retry relevant requests for up to 12 hours and display relevant error messages to customers.
  2. Apply the necessary database changes to store additional metadata.
  3. Run the backfill strategies you’ve devised to backfill UUIDs, OAuth tokens, and other metadata for V1 resources.
  4. Modify and publish your add-on manifest with a version of 3. Pushing your manifest with the new version is the trigger that tells us you’re ready to use the V3 API.
  5. Ensure your background job runner is dispatching jobs for asynchronous provisioning where appropriate.
  6. Remove the 503 response gateway you put in place to start accepting responses.
  7. Test that provisioning, deprovisioning, and plan changes are working correctly.

An alternate migration strategy involving no downtime might look like:

  1. Deploy code to accept V3 requests (as identified by the Accept: header) while continuing to accept V1 requests.
  2. Apply the necessary database changes to store additional metadata.
  3. Start acquiring OAuth tokens for new resources sent under V3.
  4. Run the backfill strategies you’ve devised to backfill UUIDs, OAuth tokens, and other metadata for V1 resources.
  5. Modify and publish your add-on manifest with a version of 3. Pushing your manifest with the new version is the trigger that tells us you’re ready to use the V3 API.
  6. After our integration API has started to send V3 requests, remove V1 specific code from your integration API.
  7. Test that provisioning, deprovisioning, and plan changes are working correctly.

Please don’t hesitate to contact Ecosystem support as you work on your migration.

Feedback

Log in to submit feedback.