This add-on is operated by Rafael Ancheta

Autoscaling and load monitoring via CPU, memory and response times supercharged

Last updated January 17, 2023

The 123 Dyno addon provides the fastest, most configurable, autoscaling and load analytics for web and worker dynos. 123 Dyno provides tools to help you analyze, respond, and autoscale your app for changing load intelligently.

Provisioning the Add-on

Attach 123 Dyno to a Heroku application via the CLI:

$ heroku addons:create d123
Creating d123 on exceptional-meercat-5555... free
Your add-on has been provisioned successfully

123 Dyno also requires Heroku Labs log-runtime-metrics to use Dyno Load options:

$ heroku labs:enable log-runtime-metrics
Enabling log-runtime-metrics for myapp... done
$ heroku restart

You must configure the add-on in the dashboard after provisioning.

Dashboard

Access the dashboard via the CLI:

$ heroku addons:open d123
Opening d123 for jovial-brook-1515

The dashboard opens to the App Overview page. Use the dashboard to configure autoscaling.

Overview

View an overview of application response time metrics, configured workers, web URL blocklist paths, overall cost reductions, and load alerts on the Overview page. The system may not have enough data to display if you recently provisioned the add-on.

Autoscaling

Select the tab on the left to configure autoscaling. The autoscaling tab contains configuration elements for the current web and worker processes selectable via the top dropdown menu. You can hover the help icons for more information.

Autoscaling On - Toggles autoscaling on or off for the application.

Dyno Scaling Range - Specifies the min and max number of dyno’s, capped at your account maximum.

Autoscaling Metric - Specifies the autoscaling metric used to scale the dyno:

• Average Response Times - scale off of average response times in a time window (smooth scaling)
• Maximum Response Times - scale off of the maximum response times in a time window (conservative scaling)
• M95 - (95th percentile of Response Times) scale off of the 95th percentile of response times in a time window (middle ground scaling)
• Queue Times - scale off of the amount of time a request is queued for processing (concurrent/parallel frameworks)
• Dyno Load - scale off of CPU load, with Standard Dynos around max 4 load and Performance Dynos around 8 max load (CPU intensive processes)
• Memory Utilization - scale off of percentage of dyno memory used (Memory intensive processes)

Sample Window - The look-back time period used to calculate your autoscaling metric. For example, a 15-minutes window autoscales based on the last 15 minutes of data.

Metric Thresholds - The application tries to keep metrics in between these values in a sort of “Goldilocks range”. Choose threshold values for the scale metric you specified. Signals to upscale occur when metrics break the max threshold. Signals to downscale occur when metrics drop below the minimum.

Autoscaling Speed - Determines how your application scales and how quickly it scales when given an up or down signal. Upscaling and downscaling can be configured separately. The static setting increases your formations quantity by a static amount while the dynamic setting increases based on the magnitude from your desired thresholds. Autoscaling intervals determine how long to wait in-between autoscaling events. The default setting is the most conservative setting. Autoscaling is set up to scale up dynamically and scale down statically by increments of one, allowing you to accommodate increased traffic and prepare for more.

Analytics

The dashboard provides in-depth analytics of how your application functions under load. The topmost chart features dyno count and response times. To view only one metric against another, toggle the boxes in the top legend to hide and show metrics.

Below the graph are path-specific max response times and average response times. Use these values to debug which routes respond the slowest in your application or block paths to help tune scaling.

Next is queue time scaling used to view how long requests are taking before your application begins processing, followed by CPU Load, and Memory Utilization.

Load Alerting

123 Dyno comes preset with default email load alerting if you’re application starts to exceed 4000ms or worker process exceed 3.5 load. Create your own rules and alerting metrics via the buttons on the page.

• Alert Name… the name for the alert shown on the overview page and sent in email subject line
• Alert me when… choose a scaling metric to trigger the alert
• Crosses threshold… the numerical threshold that must be exceeded
• At most every… how long to wait before resending when metric has been crossed
• * With sensitivity…* the time window for metric calculation, for example the average of the past 5 minutes
• Send to Emails… a comma seperated list of email to send the alert to

Logs

Transparent logs for web router, cpu load, memory utilization, and internal scaling are enabled in the Logs section of 123 Dyno. Logs can help you troubleshooting and fine-tune your autoscaling configuration showing exactly how your dyno moves in response to traffic. For example, you can determine scaling speed of your application and adjust to meet needs.

Path Blocklist

Some endpoints take longer to process than others. For example, an endpoint might process file uploads, stream data, or have long polling socket.io backups. The response service time for WebSockets in Heroku represents the entire session length and must be omitted from metrics, check 123 Dyno analytics to see if these are an issue. If these endpoints aren’t removed from metric calculations, autoscaling can get stuck at maximums. Standard string matching is used for the blocklist.

Troubleshooting

• If dynos seem to be getting stuck at max then monitor service logs and put variant long response time endpoints into the path blocklist
• If dyno load charts are always empty, ensure that Heroku Labs log-runtime-metrics are enabled as in the provisioning section of this article

Migrating Between Plans

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade d123:startup
-----> Upgrading d123:startup to filly-dexter-6006... done, v19 ($35/mo)
       Your plan has been updated to: d123:startup

Removing the Add-on

Remove 123 Dyno via the CLI:

$ heroku addons:destroy d123
-----> Removing d123 from verdant-lake-3088... done, v20 (free)

Support

Submit all 123 Dyno support and runtime issues via one of the Heroku Support channels. Any non-support-related issues or product feedback is welcome at info@123dyno.com.