Last updated 09 December 2016
Users will need to be granted special access to trial Heroku CI and testpacks. You can request access to the Heroku CI limited private beta here.
Testpacks are the scripts that power the “it just works” functionality of Heroku CI. A testpack supplements a buildpack with scripts that understand how to build, compile, and run tests.
Please note that until Heroku CI is released to General Availability, this API is subject to change without notice. Both Heroku CI and this Testpack API are currently in Private Beta. For more information e-mail firstname.lastname@example.org .
The Testpack API is a supplement to the Buildpack API. As such, this document covers only the additions to the Buildpack API to support Heroku CI. Any testpack implementation should also respect the rules, conventions, and styles set forth in the Buildpack API documentation.
A testpack consists of two scripts:
bin/test-compile: Used to transform application source code into a testable app.
bin/test: Runs the app’s tests.
bin/test-compile BUILD_DIR CACHE_DIR ENV_DIR
This script is similar to
bin/compile, but instead of preparing source for deployment, it prepares source for testing. Commonly,
bin/test-compile performs the same operations as
bin/compile, but also installs test dependencies, and skips any productionization steps.
Please see the
bin/compile documentation for usage, style, and examples.
bin/test BUILD_DIR ENV_DIR
This script should run the application’s test suite. Most implementations will execute the same command a user would run locally to start a full test run.
This script will be executed with all .profile.d scripts and ENV_DIR entries already loaded and set as environment variables.
Heroku CI will evaluate this script’s exit code to determine if the test run was a pass or failure. An exit code of
0 will indicate a successful test run; any other exit code will indicate a test failure.
Heroku CI will evaluate this script’s output and attempt to parse test case passes and failures according to the Test Anything Protocol (TAP). If Heroku CI is able to detect TAP test passes and failures, the developer will experience an enhanced UI for viewing test runs. Output in TAP format is not mandatory to implement a testpack, though it is desirable.
The easiest way to get started with a testpack may be to look at existing implementations. Here are a few buildpacks that have implemented the Testpack API:
While Heroku CI is in private beta, this API is likely to improve. We’d like your help in guiding it’s direction, so we can make it a great experience for implementors and consumers. Please reach out to us at email@example.com if you have any suggestions, difficulties, or any other feedback.