Load Testing Guidelines
Last updated 12 September 2017
Table of Contents
During the course of running production applications on the Heroku Platform, you will likely run into the need to perform load tests. This document establishes some guidelines around what testing is permissible on Heroku.
Common Runtime limits
There are varying limits depending upon the scale of load test you want to perform. This section outlines the limits that apply at various testing volumes.
Low-medium scale tests
These may be conducted without any need for approval from Heroku, so long as they’re limited to 10,000 requests/second for any given application. This applies to all regions for the Common Runtime. You may run these tests at any point without notice, but limit this to one test at a time – do not run multiple 10,000 requests/second tests concurrently. Tests at this scale can be performed on any dyno sizes.
- RPS Limit: < 10,000 reqs/sec
- Allowed Dyno Size: Professional tier or higher
- Approval Required: No
Large scale tests
You may conduct tests involving traffic volumes greater than 10,000 requests/second, but these require getting approval from Heroku by contacting Heroku support at least two business days before you wish to perform the test.
Heroku’s architecture is designed to scale with organic traffic growth. In the case of benchmarks, especially those with heavy burst traffic, our systems may not react fast enough to accommodate the sudden influx of traffic. Giving us notice will allow us to pre-warm our systems and ensure that your benchmark provides accurate, reliable results.
Tests at this scale should be limited to Performance dynos. Do not run load tests at volumes greater than 10,000 requests/second against non-Performance dynos.
- RPS Limit: > 10,000 reqs/sec
- Allowed Dyno Sizes: Performance tier only
- Approval Required: Yes
Possible ramifications of not getting approval
Without proper preparation, load tests over 10,000 requests/second will most likely give inaccurate results. Also, in the worst case, your load testing may be misinterpreted as an abusive (or abused) app and we may inadvertently suspend your application while investigating.
Private Spaces limits
You may conduct tests involving any traffic volume in Private Spaces without any prior approval. They have completely isolated environments and routing, so there is no potential impact on other customers when conducting load tests from here. The only limits you’ll run into directly correlate to the available throughput of your environment. Note that every application in a Private Space is behind an ELB, so you’ll want to request an ELB pre-warm prior to your test by contacting Heroku support.
- RPS Limit: None
- Allowed Dyno Sizes: Any
- Approval Required: No
You must be running a minimum of two web dynos in order to do prewarming.
Other considerations
In addition to the scale of the test and getting approval where required, there are some other items that are worth considering before conducting a load test.
SSL Endpoint pre-warming
If you’re using an SSL Endpoint (not to be confused with the new Heroku SSL offering), then you should request an ELB pre-warm as described here. If you don’t, then your results will likely not provide an accurate assessment of the volume you can realistically serve. Note that applications in Private Spaces are also behind an ELB that requires pre-warming, so you would want to do this for those as well.
Datastore connection limits
Most datastores have connection limits that you may run into while scaling your application for a load test. It’s important to note what connection limits your datastores have and ensure you don’t surpass them when scaling up for your test. You can find a good discussion of this here. It’s specific to Ruby and ActiveRecord, but many of the concepts have parallels in other languages and ORMs.