Skip Navigation
Show nav
Heroku Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
View categories

Categories

  • Heroku Architecture
    • Dynos (app containers)
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Command Line
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery
    • Continuous Integration
  • Language Support
    • Node.js
    • Ruby
      • Rails Support
      • Working with Bundler
    • Python
      • Background Jobs in Python
      • Working with Django
    • Java
      • Working with Maven
      • Java Database Operations
      • Working with the Play Framework
      • Working with Spring Boot
      • Java Advanced Topics
    • PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
    • Heroku Data For Redis
    • Apache Kafka on Heroku
    • Other Data Stores
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
    • Compliance
  • Heroku Enterprise
    • Private Spaces
      • Infrastructure Networking
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
    • Single Sign-on (SSO)
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • Language Support
  • Node.js
  • Optimizing Node.js Application Concurrency

Optimizing Node.js Application Concurrency

English — 日本語に切り替える

Last updated November 28, 2022

Table of Contents

  • Enabling concurrency in your app
  • Testing locally
  • Tuning the concurrency level
  • See it in action

Node has a limited ability to scale to different container sizes. It’s single-threaded, so it can’t automatically take advantage of additional CPU cores.

Heroku Enterprise customers with Premier or Signature Success Plans can request in-depth guidance on this topic from the Customer Solutions Architecture (CSA) team. Learn more about Expert Coaching Sessions here or contact your Salesforce account executive.

 

Node versions < 12 cannot automatically take advantage of additional memory. Since Node is built on top of V8, which has a hard memory limit of about 1.5 GB, the process will allocate the same.

Instead, Node.js apps must fork multiple processes to maximize their available resources. This is called “clustering,” and is supported by the Node.js Cluster API. You can invoke the Cluster API directly in your app, or you can use one of many abstractions over the API. Here, we’ll use throng.

With Cluster, you can optimize your app’s performance across various dyno types. The Heroku Node.js buildpack provides environment variables to help.

Enabling concurrency in your app

We recommend that all applications support clustering. Concurrency, even if you don’t anticipate running more than a single process, grants you greater control and flexibility over your app’s performance in the future. Let’s take a look at an example.

First, we determine how many processes we should cluster:

var WORKERS = process.env.WEB_CONCURRENCY || 1;

Second, we define a start function that will be the entry point for each newly clustered process:

function start() {
  // ...
}

Finally, we use throng to cluster the app into multiple processes. We specify a lifetime of Infinity to tell throng that, if a worker dies, it should be respawned - so we will always have WORKERS processes running:

throng({
  workers: WORKERS,
  lifetime: Infinity
}, start);

Testing locally

With that implemented, we can observe the cluster:

$ npm start

> example-concurrency@1.0.0 start example-concurrency
> node server.js

Listening on 3000
$ WEB_CONCURRENCY=4

$ npm start

> example-concurrency@1.0.0 start example-concurrency
> node server.js

Listening on 3000
Listening on 3000
Listening on 3000
Listening on 3000

Tuning the concurrency level

Each app has unique memory, CPU, and I/O requirements, so there’s no such thing as a one-size-fits-all scaling solution. The Heroku buildpack provides reasonable defaults through two environment variables: WEB_MEMORY and WEB_CONCURRENCY. Both of these can be overridden to fit your specific application.

  • WEB_MEMORY specifies, in MB, the expected memory requirements of your application’s processes. It defaults to 512MB.

  • WEB_CONCURRENCY specifies the recommended number of processes to cluster for your application. It’s essentially MEMORY_AVAILABLE / WEB_MEMORY.

Read more about configuring your application’s memory use when clustering

Defaults:

Common Runtime

Dyno TypeNumber of Cluster workers
eco, basic, standard-1x1
standard-2x2
performance-M5
performance-L28

Private Spaces

Dyno TypeNumber of Cluster workers
private-S2
private-M5
private-L28

For Performance-L dynos, applications will fare well with the 28 workers suggested for its 14 GB of memory. However, it is prudent to test an application to see whether it can in fact support so many concurrent processes.

These are reasonable defaults for most apps. In most cases, clustering more than one worker on a Standard-1x dyno will hurt, rather than help performance. However, feel free to try any combination of WEB_CONCURRENCY with any dyno size to see what works best for your workload.

Decreasing the WEB_MEMORY will increase WEB_CONCURRENCY. Increasing WEB_MEMORY will, similarly, reduce concurrency. When the size of your dyno changes, WEB_CONCURRENCY will be recalculated automatically to fill available memory.

You can also set WEB_CONCURRENCY directly, but this will prevent your app from automatically re-clustering when you change dyno sizes.

See it in action

To log Node concurrency settings on startup, set the LOG_CONCURRENCY config var:

$ heroku config:set LOG_CONCURRENCY=true

Once you’ve deployed a cluster-able app to Heroku, you can tail its log to observe it scaling into different container sizes:

$ heroku logs --tail
$ heroku scale web=1:standard-1x

heroku[api]: Resize web to standard-1x
heroku[api]: Scale to web=1
heroku[web.1]: State changed from up to starting
heroku[web.1]: State changed from up to starting
heroku[web.1]: Starting process with command `npm start`
app[web.1]: Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
app[web.1]: Recommending WEB_CONCURRENCY=1
heroku[web.1]: Stopping all processes with SIGTERM
app[web.1]:
app[web.1]: > example-concurrency@1.0.0 start /app
app[web.1]: > node server.js
app[web.1]:
app[web.1]: Listening on 51077
heroku[web.1]: State changed from starting to up
$ heroku scale web=1:performance-l

heroku[api]: Resize web to performance-l
heroku[api]: Scale to web=1
heroku[web.1]: State changed from up to starting
heroku[web.1]: Starting process with command `npm start`
app[web.1]: Recommending WEB_CONCURRENCY=12
app[web.1]: Detected 6144 MB available memory, 512 MB limit per process (WEB_MEMORY)
heroku[web.1]: Stopping all processes with SIGTERM
app[web.1]:
app[web.1]: > example-concurrency@1.0.0 start /app
app[web.1]: > node server.js
app[web.1]:
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
heroku[web.1]: Process exited with status 143
heroku[web.1]: State changed from starting to up

Keep reading

  • Node.js

Feedback

Log in to submit feedback.

Using WebSockets on Heroku with Node.js Scaling an Express.js Application with Memcache

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Podcasts
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing

Subscribe to our monthly newsletter

Your email address:

  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Heroku Podcasts
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Facebook
  • Instagram
  • Github
  • LinkedIn
  • YouTube
Heroku is acompany

 © Salesforce.com

  • heroku.com
  • Terms of Service
  • Privacy
  • Cookies
  • Cookie Preferences