Application Load Testing
Last updated 19 May 2020
Table of Contents
Load tests allow you to see how your application performs under real-world traffic. Use load testing to test a new feature at scale before it launches, or to prepare your app for traffic growth.
This article covers how to set up a load testing environment, run a load test, and use the results to improve app performance.
Set up a load testing environment
To avoid impacting real users, set up a separate load testing environment.
Add separate data services (Postgres, Redis, Kafka) to the load testing app, using the same plans as production.
If your app depends on an internal service or API, provision a separate instance of that service if possible.
Ensure the load testing app does not take “real-world” actions like sending e-mail or billing
Add seed data
If your app depends on Heroku Postgres, your load testing app should use its own database with production-like (but non-sensitive) data, called seeds.
Create seeds that reflect the variety of data in production and have production-like row counts. Do not include personal or sensitive information.
Research how your language or framework addresses seeds. For instance:
- Libraries like Rails and Node.js’s Sequelize have conventions for writing seeds
Avoid using production data for load testing, as it may be sensitive or impact real users.
Install logging and monitoring add-ons
Before running a load test, install logging and monitoring add-ons on load testing app. You’ll use these tools to evaluate results:
- A logging add-on, to view app and database logs from the test
- An application performance monitoring (APM) add-on, such as New Relic, Scout, or AppOptics, to identify slow endpoints and services
- An infrastructure monitoring add-on, such as Librato or AppSignal, to measure load on the app and database
Select a load testing tool
Your load testing tool executes the test, hitting endpoints on your load testing app. Add-ons like Loader.io run load tests on a managed platform. Open source tools like Apache AB and JMeter run load tests locally or on cloud virtual machines.
Configure your load testing tool
In your load testing tool, specify the amount of traffic in the load test. For instance, Loader.io offers multiple ways to specify the size of a test, such as clients per second. Match the load test to the production load you have now or anticipate in the near future.
Next, decide which endpoints to include in your test. If you are testing a specific feature, focus on that feature’s endpoints. If you are testing the scalability of the entire app, use your APM tool to find the most popular and highest latency endpoints, and focus the test on those.
Some endpoints of your may require authentication. Loader.io supports HTTP basic authentication, custom cookies, and custom headers.
Run your load test
Review Load Testing Guidelines to see if your load test needs pre-approval.
Use your load testing tool to execute the test on the separate load testing app.
Use results to improve app performance
After your load test, look for these key signals in your metrics to determine the next steps. Follow the links in the text for more information about these issues.
Navigate to your app on the Heroku Dashboard, and go to Metrics tab. Check the response time graph and focus on the 95th percentile response time. Does it increase over the course of the test, or spike during the test?
If you see elevated response times, use your APM to determine which services are responsible. It may show, for instance, that the database made up the majority of the response time. Finally, check your APM to find the endpoints that were slowest during the test.
Consider these actions to address elevated response time:
- Scale dynos or upgrade dyno type
- Tune concurrency. See guidance for Node and Ruby apps.
- If your APM found particularly slow endpoints, see guidance on code-level optimizations
- If your APM shows the database is slowing down during the test, see Slow Queries and Database Load
In your app’s Metrics tab on the Heroku Dashboard, check the throughput graph. Does it show elevated
5xx failed responses during the test? If so, check the errors graph for Heroku error codes during that period.
Consider these actions to address elevated error rates during the test:
- If there is a spike in
H12 - Request timeouterrors, see Response Time and read more about request timeouts
- If there is a spike in
R14 - Memory quota exceedederrors, see Memory
- Errors may originate from your app and not show in the errors graph. Check your APM for failed responses and consider adding an error monitoring add-on.
Memory use can significantly impact app performance and increase response times.
Consider these actions to address memory use that exceeds the quota and results in R14 errors:
- Optimize your app’s memory usage by tuning configuration and running profiling. See guidance for Ruby, Node, and Java apps.
- Consider upgrading to a higher-memory dyno type
Web requests to your app are served by dynos, so it’s important to look at dyno-level metrics from the load test.
Consider these actions to address dyno load that exceeds recommended limits:
- Consider upgrading to a higher-CPU dyno type
- Evaluate concurrency settings, which can impact dyno load. See guidance for Node and Ruby apps.
Slow queries and database load
Slow queries are best diagnosed in production, as they can occur over a long time period. However, you may see hints in your load result.
Check your APM for slow queries during the time of the load test. In your log tool, look for query duration logs, which exist for queries longer than 2 seconds.
Review database load using a platform monitoring tool like Librato. If it exceeds
1 then it may impact query performance.
Consider these actions to address slow queries or high database load: