Testpack API
Last updated 17 May 2017
Table of Contents
Testpacks are the scripts that power the “it just works” functionality of Heroku CI. A testpack supplements a buildpack with scripts that understand how to build, compile, and run tests.
API
The Testpack API is a supplement to the Buildpack API. As such, this document covers only the additions to the Buildpack API to support Heroku CI. Any testpack implementation should also respect the rules, conventions, and styles set forth in the Buildpack API documentation.
A testpack consists of two scripts:
-
bin/test-compile: Used to transform application source code into a testable app. -
bin/test: Runs the app’s tests.
bin/test-compile
Usage
bin/test-compile BUILD_DIR CACHE_DIR ENV_DIR
Summary
This script is similar to bin/compile, but instead of preparing source for deployment, it prepares source for testing. Commonly, bin/test-compile performs the same operations as bin/compile, but also installs test dependencies, and skips any productionization steps.
Please see the bin/compile documentation for usage, style, and examples.
bin/test
Usage
bin/test BUILD_DIR ENV_DIR
Summary
This script should run the application’s test suite. Most implementations will execute the same command a user would run locally to start a full test run.
Environment
This script will be executed with all .profile.d scripts and ENV_DIR entries already loaded and set as environment variables.
Exit code
Heroku CI will evaluate this script’s exit code to determine if the test run was a pass or failure. An exit code of 0 will indicate a successful test run; any other exit code will indicate a test failure.
Output
Heroku CI will evaluate this script’s output and attempt to parse test case passes and failures according to the Test Anything Protocol (TAP). If Heroku CI is able to detect TAP test passes and failures, the developer will experience an enhanced UI for viewing test runs. Output in TAP format is not mandatory to implement a testpack, though it is desirable.
Example testpacks
The easiest way to get started with a testpack may be to look at existing implementations. Here are a few buildpacks that have implemented the Testpack API:
Feedback
This API is likely to improve. We’d like your help in guiding it’s direction, so we can make it a great experience for implementers and consumers. Please reach out to us at heroku-ci-feedback@heroku.com if you have any suggestions, difficulties, or any other feedback.