Last updated 09 April 2020
Testpacks are the scripts that power the “it just works” functionality of Heroku CI. A testpack supplements a buildpack with scripts that understand how to build, compile, and run tests.
The Testpack API is a supplement to the Buildpack API. As such, this document covers only the additions to the Buildpack API to support Heroku CI. Any testpack implementation should also respect the rules, conventions, and styles set forth in the Buildpack API documentation.
A testpack consists of two scripts:
bin/test-compile: Used to transform application source code into a testable app.
bin/test: Runs the app’s tests.
bin/test-compile BUILD_DIR CACHE_DIR ENV_DIR
This script is similar to
bin/compile, but instead of preparing source for deployment, it prepares source for testing. Commonly,
bin/test-compile performs the same operations as
bin/compile, but also installs test dependencies, and skips any productionization steps.
Please see the
bin/compile documentation for usage, style, and examples.
bin/test BUILD_DIR ENV_DIR
This script should run the application’s test suite. Most implementations will execute the same command a user would run locally to start a full test run.
This script will be executed with all .profile.d scripts and ENV_DIR entries already loaded and set as environment variables.
Heroku CI will evaluate this script’s exit code to determine if the test run was a pass or failure. An exit code of
0 will indicate a successful test run; any other exit code will indicate a test failure.
Heroku CI will evaluate this script’s output and attempt to parse test case passes and failures according to the Test Anything Protocol (TAP). If Heroku CI is able to detect TAP test passes and failures, the developer will experience an enhanced UI for viewing test runs. Output in TAP format is not mandatory to implement a testpack, though it is desirable.
The easiest way to get started with a testpack may be to look at existing implementations. Here are a few buildpacks that have implemented the Testpack API:
This API is likely to improve. We’d like your help in guiding it’s direction, so we can make it a great experience for implementers and consumers. Please reach out to us at firstname.lastname@example.org if you have any suggestions, difficulties, or any other feedback.