Last updated April 13, 2020

Processes running in Heroku Private Spaces can communicate easily with one another by using DNS to obtain the IP address of other dynos in the same space. This makes it easy to create and deploy microservices. For example, suppose you want to create an image-processing app that consists of several small services, including:

• A web process for accepting jobs
• A worker process for processing incoming images
• An agent process for handling back-office interactions

With DNS Service Discovery, processes in systems like this have a simple way to reach one another to perform tasks. When the web process accepts a new job, it can alert the back-office service via its DNS name that a new job has arrived. It can then send the job to the worker process via its DNS name as well.

Unlike dynos in the Common Runtime, dynos in Private Spaces can freely exchange TCP and UDP traffic on a private network. A process type of any kind (web or worker) can choose to listen on any available port, and other dynos in the space can connect to that port on the dyno host.

To communicate with another process, you need to know the IP address of the dyno(s) the process is running on, along with the port number the process is listening on. Typically you know the port number in advance, because server processes listen on a fixed port number. It is not possible, however, to know the dyno’s IP address in advance. To resolve this, Heroku provides a DNS-based naming scheme for dynos. This lets you find and connect to any dyno in a Private Space by specifying:

• The name of the process’s associated Heroku app
• The process type (such as web or worker) of the process you want to connect to
• Optionally, the dyno number to connect to (useful when a process is running on more than one dyno)

Setup: Enable DNS lookups

DNS Service Discovery is enabled by default on all applications created after May 31, 2017. For existing applications, you need to enable this by running:

$ heroku features:enable spaces-dns-discovery --app <your app>

If the app is already running, you need to restart it before this setting takes effect:

$ heroku restart --app <your app>

After the feature is enabled, your app’s dynos have a slightly different DNS configuration that enables DNS lookups. There is a slight risk that this change might cause issues for your application. See known technical considerations below. If you encounter any such issues, please disable the feature and restart your app:

$ heroku features:disable spaces-dns-discovery --app <your app>
$ heroku restart --app <your app>

Usage

Binding to the private network

All dyno-to-dyno communication in Private Spaces takes places via the private network. To communicate via the private network, your process needs to bind to its dyno’s private IP address on a port between 1024 and 65535. The HEROKU_PRIVATE_IP environment variable contains your dyno’s private IP address.

The app and process you want to use with Private Space Service Discovery should not use the value of the $PORT environment variable. $PORT varies for different dynos powering the same process types and changes when dynos are restarted. Instead, pick a static port (like 5000) and use that. For example (using Node.js and Express):

const express = require('express')
const PORT = 5000

express()
  .get('/', (req, res) => res.send('Hello World!'))
  .listen(PORT, () => console.log(`Listening on ${ PORT }`))

This could be deployed as the foo process type by using this Procfile:

foo: node index.js

After your process binds to the private IP address, it’s reachable on the chosen port via its DNS name and port. The above example would be available on this URL:

http://foo.[APP_NAME].app.localspace:5000/

Resolving other processes

DNS Service Discovery uses the following naming scheme:

[DYNO_NUMBER (optional)].[PROCESS_TYPE].[APP_NAME].app.localspace

For example, performing the following nslookup from a one-off dyno returns IP addresses for all dynos running the web process for an app named sushi:

$ nslookup web.sushi.app.localspace
web.sushi.app.localspace. 0 IN A 10.10.10.11
web.sushi.app.localspace. 0 IN A 10.10.10.10
web.sushi.app.localspace. 0 IN A 10.10.10.9

If your process connects to a DNS name with the format shown above, the DNS registry routes it to one of the associated process’s dynos in round-robin fashion. If you instead want to connect to a specific dyno running the associated process, you can specify its dyno number at the beginning of the DNS name.

This example demonstrates performing an nslookup specifically for the IP address of dyno number 2 running the sushi app’s web process:

$ nslookup 2.web.sushi.app.localspace
2.web.sushi.app.localspace. 0 IN A 10.10.10.10

Note that the last two components of the naming scheme (app and localspace) are constant for all DNS names.

Environment variables

Dynos with DNS Service Discovery enabled have the following environment variables set:

Unlike HEROKU_DNS_DYNO_NAME and HEROKU_DNS_FORMATION_NAME, HEROKU_DNS_APP_NAME does not resolve by itself. It is provided to simplify determining the dyno’s application-level hostname.

DNS registry details

Update frequency

The DNS registry’s set of dyno locations is updated every 10 seconds. Take this timeframe into account when designing apps that use DNS Service Discovery. Different dynos might observe these registry updates at slightly different times, but results should converge after about 10 seconds.

TTLs

DNS Service Discovery records always have a time-to-live (TTL) of zero seconds, meaning that the requester should not itself cache the information. It is important that your application is properly configured to respect the TTLs in the DNS response, so it does not cache these local requests.

Technical considerations

Routing

Networking using Private Space DNS Service Discovery uses raw TCP/IP. Even if you are running an HTTP service, there’s no Heroku routing layer, no router logs or measurements and no port-binding related boot timeouts. This can hamper troubleshooting HTTP services deployed as non-web processes, but this can also allow you to run non-HTTP services in Heroku Private Spaces.

IPv6

When you enable DNS Service Discovery for an app, IPv6 is disabled for the app’s dynos. Although Private Spaces only offers connectivity via IPv4, your app’s behavior nevertheless might change when IPv6 is no longer available on the dyno. For example, Go apps need to Listen via tcp4 instead of tcp. Code similar to the following:

conn, err := net.Dial("tcp", "golang.org:80")
ln, err := net.Listen("tcp", ":8080")

needs to be replaced with the following:

conn, err := net.Dial("tcp4", "golang.org:80")
ln, err := net.Listen("tcp4", ":8080")