Skip Navigation
Show nav
Heroku Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
View categories

Categories

  • Heroku Architecture
    • Dynos (app containers)
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Command Line
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery
    • Continuous Integration
  • Language Support
    • Node.js
    • Ruby
      • Rails Support
      • Working with Bundler
    • Python
      • Background Jobs in Python
      • Working with Django
    • Java
      • Working with Maven
      • Java Database Operations
      • Working with the Play Framework
      • Working with Spring Boot
      • Java Advanced Topics
    • PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
    • Heroku Data For Redis
    • Apache Kafka on Heroku
    • Other Data Stores
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
    • Compliance
  • Heroku Enterprise
    • Private Spaces
      • Infrastructure Networking
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
    • Single Sign-on (SSO)
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • App Performance
  • Testing Cedar-14 Memory Use

Testing Cedar-14 Memory Use

English — 日本語に切り替える

Last updated January 27, 2022

Table of Contents

  • Test results
  • How we tested?

The Cedar-14 stack is end-of-life. Please upgrade your apps to a supported stack as soon as possible. See also the Cedar-14 End-of-Life FAQ.

As part of our work to the Cedar-14 stack, we benchmarked apps to learn how changes in system libraries from Cedar to Cedar-14 manifested into real world differences in app performance. One thing we found was that memory behavior varied between stacks for some apps. We dug deeper into those variations and this article presents our findings.

For details on how to tune and optimize memory use, see the Tuning glibc memory behavior article.

Memory usage can be higher on Cedar-14 than on Cedar because of an underlying change in glibc’s malloc implementation. The change generally improves app performance when apps do memory allocations from threads. This is because more memory arenas are available to the app. This can cause performance for apps that create many threads (and use glibc’s malloc implementation internally) to have different performance characteristics on Cedar-14.

The way that glibc manages memory arenas can be fine-tuned by setting the environment variable MALLOC_ARENA_MAX.

Test results

For the testing presented in this article, we used Cedar as the baseline and compared it to Cedar-14 with various values of MALLOC_ARENA_MAX. The test candidate was a Ruby app configured to use the Puma web server.

Memory usage (RSS) averages (base is Cedar)

ConfigurationMultiplier
Cedar-14 1.73x
Cedar-14 with MALLOC_ARENA_MAX=1 0.86x
Cedar-14 with MALLOC_ARENA_MAX=2 0.87x

It’s worth noting that, while memory consumption on Cedar-14 was higher than Cedar, this app never used more than the 512 MB available in the dyno.

Median response time averages (base is Cedar)

ConfigurationMultiplier
Cedar-14 0.90x
Cedar-14 with MALLOC_ARENA_MAX=1 1.15x
Cedar-14 with MALLOC_ARENA_MAX=2 1.03x

95th percentile response time averages (base is Cedar)

ConfigurationMultiplier
Cedar-14 0.91x
Cedar-14 with MALLOC_ARENA_MAX=1 1.41x
Cedar-14 with MALLOC_ARENA_MAX=2 1.03x

The glibc default number of memory pools on 64bit systems is 8 times the number of CPU cores (the number of CPU cores seen by dynos on Heroku varies with dyno type).

Limiting the number of arenas from this default generally reduces memory usage. We tested with values of “1” and “2” and both resulted similar memory reductions. The performance differences between arena values of “1” and “2”, however, are measurable. In particular, “1” has worse performance than “2”. Not setting a value for MALLOC_ARENA_MAX gives default glibc behavior and has the the best performance but also consumes the most memory. For details on how to optimize app performance with MALLOC_ARENA_MAX see the Tuning glibc memory behavior article.

How we tested?

The Bundler API, the Ruby gem repository, runs on Heroku and we contribute to its maintenance. This let us test it on multiple stacks with different glibc configurations.

We built a service that reads through the logs of the production app and replays all eligible requests on a number of staging apps. Each staging app ran the latest release of Bundler API but was configured independently and with fully isolated resources.

The test ran for a total of 24 hours.

Keep reading

  • App Performance

Feedback

Log in to submit feedback.

Worker Dynos, Background Jobs and Queueing Using Amazon CloudFront CDN

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Podcasts
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing

Subscribe to our monthly newsletter

Your email address:

  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Heroku Podcasts
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Facebook
  • Instagram
  • Github
  • LinkedIn
  • YouTube
Heroku is acompany

 © Salesforce.com

  • heroku.com
  • Terms of Service
  • Privacy
  • Cookies
  • Cookie Preferences