Table of Contents
- Request distribution
- Request queueing
- Dyno connection behavior
- Simultaneous connections
- Request buffering
- Response buffering
- Heroku headers
- Heroku router log format
- Gzipped responses
- Supported HTTP Methods
- Expect: 100-continue
- HTTP versions supported
- HTTP validation and restrictions
- Protocol upgrades
- Not supported
The Heroku platform automatically routes HTTP requests sent to your app’s hostname(s) to your web dynos. The entry point for all applications on the Cedar stack is the
herokuapp.com domain which offers a direct routing path to your web dynos.
This Heroku HTTP router has built-in WebSocket support. In addition, the router offers request buffering, extended Heroku headers, clearly documented “Expect: 100-continue” behavior, and improved HTTP version support and HTTP validations.
This article provides a detailed reference of how the router behaves, and how it conforms to the HTTP specification.
Inbound requests are received by a load balancer that offers SSL termination. From here they are passed directly to a set of routers.
The routers are responsible for determining the location of your application’s web dynos and forwarding the HTTP request to one of these dynos.
A request’s unobfuscated path from the end-client through the Heroku infrastructure to your application allows for full support of HTTP 1.1 features such as chunked responses, long polling, websockets, and using an async webserver to handle multiple responses from a single web process. HTTP 1.0 compatibility is also maintained.
Routers use a random selection algorithm for HTTP request load balancing across web dynos.
Each router maintains an internal per-app request queue. For Cedar apps, routers limit the number of active requests per dyno to 50 and queue additional requests. There is no coordination between routers however, so this request limit is per router. The request queue on each router has a maximum backlog size of 50n (n = the number of web dynos your app has running). If the request queue on a particular router fills up, subsequent requests to that router will immediately return an H11 (Backlog too deep) response.
Dyno connection behavior
When Heroku receives an HTTP request, a router establishes a new upstream TCP connection to a randomly selected web dyno. If the connection is refused or has not been successfully established after 5 seconds, the dyno will be quarantined and no other connections will be forwarded from that router to the dyno for up to 5 seconds. The quarantine only applies to a single router. Because each router keep its own list of quarantined dynos, other routers may continue to forward connections to the dyno.
When a connection to a dyno is refused or times out, the router processing the request will retry the request on another dyno. A maximum of 10 attempts (or fewer if you don’t have 10 running web dynos) will be made before returning an H19 or H21 error.
If all dynos are quarantined, the router will retry for up to 75 seconds (with an incremental back off), before serving an H99 error.
After a dyno connection has been established, HTTP requests have an initial 30 second window in which the web process must return response data (either the completed response or some amount of response data to indicate that the process is active). Processes that do not send response data within the initial 30-second window will see an H12 error in their logs.
After the initial response, each byte sent (either from the client or from your app process) resets a rolling 55 second window. If no data is sent during this 55 second window then the connection is terminated and a H15 error is logged.
Additional details can be found in the Request Timeout article.
herokuapp.com routing stack allows many concurrent connections to web dynos. For production apps, you should always choose an embedded webserver that allows multiple concurrent connections to maximize the responsiveness of your app. You can also take advantage of concurrent connections for long-polling requests.
Almost all modern web frameworks and embeddable webservers support multiple concurrent connections. Examples of webservers that allow concurrent request processing in the dyno include Unicorn (Ruby), Goliath (Ruby), Puma (JRuby), Gunicorn (Python), and Jetty (Java).
When processing an incoming request, a router sets up a buffer to receive the entire HTTP request line and request headers. Each of these have further limitations described in HTTP validation and restrictions. The body of a request is transmitted by using a 1024 byte buffer, filled and flushed continuously. This represents a size that encompasses the vast majority of requests in terms of volume. The request will start being dispatched to a dyno once the entire set of HTTP headers has been received.
As a result, each router buffers the header section of all requests, and will deliver this to your dyno as fast as our internal network will run. The dyno is protected from slow clients until the request body needs to be read. If you need protection from clients transmitting the body of a request slowly, you’ll have the request headers available to you in order to make a decision as to when you want to drop the request by closing the connection at the dyno.
The router maintains a 1MB buffer for responses from the dyno per connection. This means that you can send a response up to 1MB in size before the rate at which the client receives the response will affect the dyno - even if the dyno closes the connection, the router will keep sending the response buffer to the client. The transfer rate for responses larger than the 1MB buffer size will be limited by how fast the client can receive data.
All headers are considered to be case-insensitive, as per HTTP Specification.
X-Forwarded-For: the originating IP address of the client connecting to the Heroku router
X-Forwarded-Proto: the originating protocol of the HTTP request (example: https)
X-Forwarded-Port: the originating port of the HTTP request (example: 443)
X-Request-Start: unix timestamp (milliseconds) when the request was received by the router
X-Request-Id: the Heroku HTTP Request ID
Via: a code name for the Heroku router
Heroku router log format
2012-10-11T03:47:20+00:00 heroku[router]: at=info method=GET path=/ host=myapp.herokuapp.com fwd="22.214.171.124" dyno=web.1 connect=1ms service=18ms status=200 bytes=13
method: HTTP request method
path: HTTP request path
host: HTTP request
fwd: HTTP request
dyno: name of the dyno that serviced the request
connect: amount of time in milliseconds spent establishing a connection to the backend web process
service: amount of time in milliseconds spent proxying data between the backend web process and the client
status: HTTP response code
bytes: Number of bytes transferred from the backend web process to the client
2012-10-11T03:47:20+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=GET path=/ host=myapp.herokuapp.com fwd="126.96.36.199" dyno=web.1 connect= service=30000ms status=503 bytes=0
code: Heroku error code
desc: description of error
Apps serving large amounts of static assets can take advantage of HTTP caching to improve performance and reduce load.
WebSocket functionality is available as part of this router version.
Since requests to Cedar apps are made directly to the application server – not proxied through an HTTP server like nginx – any compression of responses must be done within your application.
Supported HTTP Methods
The HTTP protocol has a few built-in mechanisms to help clients cooperate with
servers in order to get better service overall. One of such mechanisms is the
Expect: 100-continue header that can be sent along with
This header and value are to be used by friendly clients when they are sending large HTTP requests, and want to know if the server can safely accept it before sending it, in order to prevent denial of service issues, and allow some optimizations. The cURL HTTP client is the most known library using this mechanism, doing it for any content-body larger than 1kb (undocumented behavior).
Whenever asking for the permission, the server is allowed to respond with a
100 Continue HTTP status,
which lets the client know that it is ready to
accept the request and to let the client proceed. If the server cannot deal
with it, it can return any other HTTP response that makes sense for it, such as
413 Request Entity Too Large
if it doesn’t want to handle the load, and can
optionally close the connection right after the fact.
As such, between any two clients and server, the mechanism that takes place may look a bit like this for a regular call:
[Client] [Server] |-------- Partial Request -------->| |<--------- 100 Continue ----------| |--------- Request Body ---------->| |<----------- Response ------------|
Or, for a denied request:
[Client] [Server] |-------- Partial Request -------->| |<-- 413 Request Entity Too Large -|
The mechanism needs to be more resilient, however, because not all servers and
clients can understand that mechanism. As such, if the client doesn’t know
whether the server can accept and honor the
Expect: 100-Continue headers,
it should be sending the actual body after having waited a period of time:
[Client] [Server] |-------- Partial Request -------->| | | | | | | |--------- Request Body ---------->| |<----------- Response ------------|
By default, many web servers do not handle
Expect: 100-continue as a mechanism. Therefore, the Heroku HTTP routers will automatically insert a
100 Continue response on behalf of the application it routes to, and will later forward the data.
This more or less disables the mechanism entirely as far as the dyno’s webserver is concerned.
Enabling 100-Continue Support
End-to-end continue support is now available (to members of the Heroku beta program) through a Heroku labs feature:
$ heroku labs:enable http-end-to-end-continue
If the extension is enabled, the general flow of 100-Continue feature is restored, and the router will pass on the
expect: 100-continue headers and their associated
100 continue responses transparently.
This feature comes with its own set of corner cases and behaviors that have to be specified, however.
There are a set of specific corner cases that may come with this kind of request.
The original HTTP 1.1 RFC (RFC 2068)
allowed use of the
100 Continue partial response to let the server say “the
entire request isn’t done parsing, but keep going, I’m not denying it right
away”. Later RFCs (RFC
2616 for example)
Expect: 100-continue mechanism, and reappropriated the
Continue partial response as part of the mechanism defined in the previous
section of the text.
For backwards compatibility reasons, the server could possibly send a
Continue without having received the related
Expect header, but is strongly
advised against doing so.
A second corner case is that a server may decide not to send back a
Continue response if it has started receiving and processing body data first
hand, which the client should be aware of.
Servers also have to be careful when parsing
Expect headers, given they
could contain more than one value.
Expect: 100-continue, auth
could be sent when expecting the server to handle a large body, while asking to
be authenticated before doing so. Technically, there can be as many values as
desired within the
Expect header with the behavior being defined purely for
the client and the server’s sake.
Other special cases come from unexpected interactions coming from having multiple headers that manage connection flow. For example:
- What happens if a client request contains both a connection Upgrade request
(for websockets) and an
- What happens if a server responds with a
100 Continuestatus code, but also includes headers such as
Connection: close, which should terminate the connection upon reception?
These behaviors are undefined by the original specifications, and the Heroku router has to make a decision regarding them in order to provide consistent behavior.
Even though the
Expect header is defined to be an End-To-End header (only the
client and the server would have to care about them, explaining why any term
can be used in it), the
100 Continue mechanism itself (and the general
Expect behavior support) requires coordination with proxies and is hop-by-hop.
The RFC has special conditions added for them:
- The proxy should pass the header as-is, whether it knows if the server can handle it or not
- If the proxy knows that the server to which it’s routing has an HTTP version of 1.0
or lower, it must deny the request with a
417 Expectation Failedstatus, but it’s possible it doesn’t know about it.
- An HTTP 1.0 client (or earlier) could send a request without an
Expect: 100-Continueheader, and the server could still respond to it using the
100 ContinueHTTP code (as part of RFC 2068), in which case the proxy should strip the response entirely and wait to relay the final status.
Heroku router 100-continue support
The router takes some liberties regarding the undefined behaviors and corner cases mentioned earlier:
100 Continuewill be stripped as a response if the client is a HTTP 1.0 (or earlier) client and the
Expect: 100-continueheader isn’t part of the request, and will be forwarded otherwise no matter what.
- The router will not require a
100 Continueresponse to start sending the request body, but will leave the wait time up to the client (without breaking the regular inactivity rules for connections on Heroku)
- The router will automatically respond with a
417 Expectation Failedresponse and close the connection to the dyno if the
Expectheader contains any value other than
100 Continue(case insensitive)
- If a WebSocket upgrade is requested, it will be sent as-is to the dyno, and the
router will honor whichever response comes in: a
100 Continuestatus may ignore the WebSocket upgrade and return any code (as usual), and a
101 Switching Protocolwill ignore the
Expectheaders' behavior. Note that in order to respect the HTTPbis Draft, The router will still look for a
101 Switching Protocolfollowing a
100 Continuereceived from the server.
- The router will ignore
Connection: closeon a
100 Continueand only honor it after having received the final response. Given that the RFC specifies that the connection should be closed “after the current request/response is complete”, and that
100is not a terminal status, the connection will be closed only after having received a terminal status. Note however, that because
Connection: closeis a hop-by-hop mechanism, the router will not necessarily close the connection to the client, and may not forward it.
- The router will strip all headers from a
100 Continueresponse, given no header is prescribed by the RFC and it makes the implementation much simpler.
- The router will return a 5xx error code if the server returns a
100 Continuefollowing an initial
100 Continueresponse. The router does not yet support infinite 1xx streams.
- The router will close the connection to the server following a
terminal status code, whether it was preceded by a
100 Continueor not beforehand – connections to dynos aren’t keep-alive.
- The router will close the connection to the client following a
terminal status code that was not preceded by a
100 Continueresponse. This will avoid having the client need to send the request body anyway before having the server being able to process the next request.
Other mechanisms should be respected as-is by the protocol, and the router should forward requests as specified by the RFC.
HTTP versions supported
Three main versions of HTTP are used in the wild: HTTP/0.9, HTTP/1.0, and HTTP/1.1. Side protocols such as SPDY and drafts for HTTP/2.0 are being worked on, but do not yet see widespread adoption.
The Heroku router only supports HTTP/1.0 and HTTP/1.1 clients. HTTP/0.9 and earlier are no longer supported. SPDY and HTTP/2.0 are not supported at this point.
The router’s behavior is to be as compliant as possible with the HTTP/1.1 specifications. Special exceptions must be made for HTTP/1.0 however:
- The router will advertise itself as using HTTP/1.1 no matter if the client uses HTTP/1.0 or not.
- The router will take on itself to do the necessary conversions from a chunked response to a regular HTTP response. In order to do so without accumulating potentially gigabytes of data, the response to the client will be delimited by the termination of the connection (See Point 4.4.5)
- The router will assume that the client wants to close the connection on each request (no keep-alive).
- An HTTP/1.0 client may send a request with an explicit
connection:keep-aliveheader. Despite the keep-alive mechanism not being defined back in 1.0 (it was ad-hoc), the router makes the assumption that the behavior requested is similar to the HTTP/1.1 behavior at this point.
HTTP validation and restrictions
- In the case of chunked encoding and content-length both being present in the request, the router will give precedence to chunked encoding.
- If multiple content-length fields are present, and that they have the same length, they will be merged into a single content-length header
- If a content-length header contain multiple values (
content-length: 15,24) or a request contains multiple content-length headers with multiple values, the request will be denied with a code 400.
- Headers are restricted to 8192 bytes per line (and 1000 bytes for the header name)
- Hop-by-hop headers will be stripped to avoid confusion
- At most, 1000 headers are allowed per request
- The request line of the HTTP request is limited to 8192 bytes
- The request line expects single spaces to separate between the verb, the path, and the HTTP version.
- Hop-by-hop headers will be stripped to avoid confusion
- Headers are restricted to 512kb per line
- Cookies are explicitly restricted to 8192 bytes. This is to protect against common restrictions (for example, imposed by CDNs) that rarely accept larger cookie values. In such cases, a developer could accidentally set large cookies, which would be submitted back to the user, who would then see all of his or her requests denied.
- The status line (
HTTP/1.1 200 OK) is restricted to 8192 bytes in length
Applications that break these limits will see their requests fail with a 502 Bad Gateway response, and an H25 error will be injected into the application log stream.
Additionally, while HTTP/1.1 requests and responses are expected to be
keep-alive by default, if the initial request had an explicit
close header from the router to the dyno, the dyno can send a response
delimited by the connection termination, without a specific content-encoding nor
an explicit content-length.
Whereas the previous Heroku router restricted HTTP protocol upgrades to WebSockets only, the new router tolerates any upgrade at all.
Specific points related to the implementation:
- Any HTTP verb can be used with an upgradable connection
- Even though
HEADHTTP verbs usually do not require having a proper response sent over the line (regarding content-length, for example),
HEADrequests are explicitly made to work with
101 Switching Protocolsresponses. A dyno that doesn’t want to upgrade should send a different status code, and the connection will not be upgraded
Expectheaders with any content other than
100-continue(yields a 417)
- HTTP Extensions such as WEBDAV
- A HEAD, 1xx, 204, or 304 response with a content-length or chunked encoding doesn’t have the proxy try to relay a body that will never come
- Header line endings other than CRLF (
- Caching of HTTP Content
- Caching the HTTP versions of servers running on dynos
- Long-standing preallocated idle connections. The limit is set to 1 minute before an idle connection is closed.
- HTTP/1.0 requests without the
Hostheader, even when the full URL is submitted in the request line.