Last updated September 06, 2023

Node has a limited ability to scale to different container sizes. It’s single-threaded, so it can’t automatically take advantage of additional CPU cores.

Node.js apps must fork multiple processes to maximize their available resources. This “clustering” is supported by the Node.js Cluster API. You can invoke the Cluster API directly in your app, or you can use one of many abstractions over the API. Here, we use Throng.

With Cluster API, you can optimize your app’s performance across various dyno types. The Heroku Node.js buildpack provides environment variables to help.

Enabling Concurrency in Your App

We recommend that all applications support clustering. Even if you don’t anticipate running more than a single process, clustering offers greater control and flexibility over your app’s performance. Let’s take a look at an example.

First, we determine how many processes to cluster.

var WORKERS = process.env.WEB_CONCURRENCY || 1;

Second, we define a start function as the entry point for each newly clustered process.

function start() {
  // ...
}

Third, we use throng to cluster the app into multiple processes. We specify a lifetime of Infinity to tell throng to respawn if a worker dies, so we always have WORKERS processes running.

throng({
  workers: WORKERS,
  lifetime: Infinity
}, start);

Testing Locally

With concurrency implemented, we can observe the cluster.

$ npm start

> example-concurrency@1.0.0 start example-concurrency
> node server.js

Listening on 3000

$ WEB_CONCURRENCY=4

$ npm start

> example-concurrency@1.0.0 start example-concurrency
> node server.js

Listening on 3000
Listening on 3000
Listening on 3000
Listening on 3000

Tuning the Concurrency Level

Each app has unique memory, CPU, and I/O requirements, so there’s no such thing as a one-size-fits-all scaling solution. The Heroku buildpack provides reasonable defaults through two environment variables: WEB_MEMORY and WEB_CONCURRENCY. You can override both to fit your specific application.

• WEB_MEMORY specifies, in MB, the expected memory requirements of your application’s processes. It defaults to 512 MB.

• WEB_CONCURRENCY specifies the recommended number of processes to cluster for your application. It’s essentially MEMORY_AVAILABLE / WEB_MEMORY.

Read more about configuring your application’s memory use when clustering.

Defaults

Common Runtime

Private Spaces

These defaults are reasonable for most apps. In most cases, clustering more than one worker on a Standard-1x dyno hurts rather than helps performance. However, try any combination of WEB_CONCURRENCY with any dyno size to see what works best for your workload.

Decreasing the WEB_MEMORY increases WEB_CONCURRENCY. Similarly, increasing WEB_MEMORY reduces concurrency. When the size of your dyno changes, WEB_CONCURRENCY is recalculated automatically to fill available memory.

See It in Action

To log Node concurrency settings on startup, set the LOG_CONCURRENCY config var.

$ heroku config:set LOG_CONCURRENCY=true

After you deploy an app with clustering to Heroku, you can tail its log to observe it scaling into different container sizes.

$ heroku logs --tail

$ heroku scale web=1:standard-1x

heroku[api]: Resize web to standard-1x
heroku[api]: Scale to web=1
heroku[web.1]: State changed from up to starting
heroku[web.1]: State changed from up to starting
heroku[web.1]: Starting process with command `npm start`
app[web.1]: Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
app[web.1]: Recommending WEB_CONCURRENCY=1
heroku[web.1]: Stopping all processes with SIGTERM
app[web.1]:
app[web.1]: > example-concurrency@1.0.0 start /app
app[web.1]: > node server.js
app[web.1]:
app[web.1]: Listening on 51077
heroku[web.1]: State changed from starting to up

$ heroku scale web=1:performance-l

heroku[api]: Resize web to performance-l
heroku[api]: Scale to web=1
heroku[web.1]: State changed from up to starting
heroku[web.1]: Starting process with command `npm start`
app[web.1]: Recommending WEB_CONCURRENCY=12
app[web.1]: Detected 6144 MB available memory, 512 MB limit per process (WEB_MEMORY)
heroku[web.1]: Stopping all processes with SIGTERM
app[web.1]:
app[web.1]: > example-concurrency@1.0.0 start /app
app[web.1]: > node server.js
app[web.1]:
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
app[web.1]: Listening on 50092
heroku[web.1]: Process exited with status 143
heroku[web.1]: State changed from starting to up