This add-on is operated by Slash Loop Artur Trzop
Run 1-hour tests in 2 min on your current CI server with optimal parallelisation
Last updated 17 September 2020
The Knapsack Pro add-on is currently in beta.
Table of Contents
Ruby supported test runners:
Supported CI servers:
- Heroku CI
- Circle CI
- GitLab CI
- Travis CI
- Semaphore CI
- Solano CI
- Cirrus CI
- Any other CI server using an ENV variables custom setup
How Knapsack Pro Queue Mode works
This video describes how dynamic test suites split across parallel CI nodes and how they work.
Provisioning the add-on
Knapsack Pro can be attached to a Heroku application using the Heroku CLI:
$ heroku addons:create knapsack-pro --app YOUR_HEROKU_APP_NAME
Once Knapsack Pro has been added you may want to attach it to another Heroku app and then sign into Knapsack Pro user dashboard to get your API token for your first test suite. In the user dashboard, you will also find the installation guide for your test runner and CI provider.
All users of Beta version of Knapsack Pro add-on can use an unlimited version of Knapsack Pro for free during Beta testing.
When you use a trial plan, you can test Knapsack Pro for 14 days with active API usage. For instance, when you don’t use Knapsack Pro API over the weekend or when you are busy with other work, your trial days are waiting for you. CI configuration can take time especially when you are migrating from one CI provider to another. If you need more time for testing just let us know. We would be happy to extend your trial period and help you with CI and project configuration.
Attaching the add-on to another app
The Knapsack Pro add-on is shareable across Heroku apps. You can attach it to multiple Heroku apps so your developers can use it with many projects. Remember you need to generate a new API token for each project’s test suite in Knapsack Pro user dashboard. You can create many projects in Knapsack Pro user dashboard.
Attaching the add-on can be useful for instance when you have staging and production Heroku app. You can provision Knapsack Pro add-on for production application and later attach it to staging application so all developers could easily access the add-on from Heroku dashboard.
# First we need to check what's the name generated for Knapsack Pro add-on # which was provisioned for your first Heroku app. # Show list of add-ons and copy add-on name for Knapsack Pro $ heroku addons --app YOUR_HEROKU_APP_NAME # replace knapsack-pro-staging-infinite-32142 with your Knapsack Pro add-on name $ heroku addons:attach knapsack-pro-staging-infinite-32142 --app YOUR_HEROKU_APP_NAME
The Knapsack Pro user dashboard can be accessed via the CLI:
$ heroku addons:open knapsack-pro --app YOUR_HEROKU_APP_NAME
or by visiting the Heroku Dashboard, selecting the application, and then selecting Knapsack Pro from the Add-ons menu.
In the user dashboard, you can find a list of your projects and all associated API tokens to each project. Each test suite should have a separate API token. For instance, you have Ruby on Rails project and you have RSpec tests for backend API. You also have Cucumber tests for end to end testing. In such a case, you need to generate 2 API tokens. One for RSpec and second API token for Cucumber tests.
If you run the single test suite in multiple steps on your CI server, for instance, you have RSpec tests but you run unit tests first and as a second step on your CI you run RSpec features tests then you should generate 2 API tokens. First API token is needed for unit tests and second API token for features tests.
Installation depends on a test runner and CI provider you use. You can follow installation guide that fits your project.
The most common problems can be found in FAQ but here are listed the top questions developers ask for.
For Ruby projects:
- Why when I use Queue Mode for RSpec then my tests fail? - When you use RSpec please use Knapsack Pro Regular Mode first before you try Queue Mode as it is suggested in the Ruby installation guide.
- How to retry only single CI node with failed tests for Queue Mode? - If your CI provider allows retrying a single CI node from list of parallel nodes instead of running a whole CI build then you should set KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true to allow Knapsack Pro remember what tests where executed on particular CI node. Thanks to that when you retry failed CI node then the same set of tests will be executed there.
- Why my CI tests output freezes?
- How to use JUnit formatter with Knapsack Pro?
- How to use simplecov in Queue Mode?
- How to use CodeClimate with knapsack_pro ruby gem?
Answers to common questions specific to your test runner or your CI provider can be found in the documentation for particular Knapsack Pro client.
- How to encrypt test files names for Ruby tests?
- Questions around data usage and security - Knapsack Pro does not have access to your project source code. It collects only branch names, git commit hashes and test file paths with its recorded execution time.
Migrating between plans
Plans migration is easy and instant. It requires no code change and does not affect your existing collected tests timing data for CI builds. You can use the below command to migrate to a new plan:
An example of how to upgrade to a parallel-10 plan.
$ heroku addons:upgrade knapsack-pro:parallel-10 --app YOUR_HEROKU_APP_NAME
List of all plans can be found here.
The Knapsack Pro add-on allows exceeding current plan limits before plan upgrade. We understand how CI testing is critical to engineering teams so we want to ensure your work is never interrupted. When you exceed over 100% current plan usage then we will send you a reminder about plan upgrade.
All Knapsack Pro Heroku add-on support and runtime issues should be submitted via the Heroku Support channels. You can send us your suggestions and feedback at knapsackpro.com via online chat where we can help in realtime with your project and CI setup.
Knapsack Pro supports many CI providers and various test runners. You can see parallel testing examples with video for a few CI providers here. If you use Heroku CI you should take a look at this article explaining how to run tests faster on Heroku CI with parallel dynos.
Removing the add-on
Knapsack Pro can be removed using the Heroku CLI.
This will not destroy associated data immediately. Can be undone only after support contact. Removing add-on will mark your organization account in Knapsack Pro as deprovisioned (you won’t be able to use API tokens to split tests). Within 90 days your collected tests timing data will be destroyed.
$ heroku addons:destroy knapsack-pro --app YOUR_HEROKU_APP_NAME