Last updated March 20, 2023

A segmentation fault can happen when software tries to issue invalid memory commands. For example, when a program attempts to free memory already freed (double free) or accesses a memory location not yet allocated (Illegal memory access). There are two primary ways to trigger a segfault when running Ruby code:

• Segfault in the Ruby virtual machine
• Segfault in a native extension library

This output is an example of what a segfault might look like in your Ruby application logs:

2016-05-10T03:04:11.572681+00:00 app[web.1]: -- Control frame information -----------------------------------------------
2016-05-10T03:04:11.582158+00:00 app[web.1]:   975 /app/vendor/bundle/ruby/2.2.0/gems/sass-3.4.22/lib/sass/supports.rb
2016-05-10T03:04:11.572682+00:00 app[web.1]: c:0001 p:---- s:0002 e:000001 TOP    [FINISH]
2016-05-10T03:04:11.582159+00:00 app[web.1]:   976 /app/vendor/bundle/ruby/2.2.0/gems/sass-3.4.22/lib/sass/engine.rb
2016-05-10T03:04:11.582160+00:00 app[web.1]:   977 /app/vendor/bundle/ruby/2.2.0/gems/sass-3.4.22/lib/sass/railtie.rb
2016-05-10T03:04:11.572683+00:00 app[web.1]:
2016-05-10T03:04:11.582162+00:00 app[web.1]:   978 /app/vendor/bundle/ruby/2.2.0/gems/sass-3.4.22/lib/sass/features.rb
2016-05-10T03:04:11.582162+00:00 app[web.1]:   979 /app/vendor/bundle/ruby/2.2.0/gems/sass-3.4.22/lib/sass.rb
2016-05-10T03:04:11.572684+00:00 app[web.1]:
2016-05-10T03:04:11.582163+00:00 app[web.1]:   980 /app/vendor/bundle/ruby/2.2.0/gems/sprockets-3.6.0/lib/sprockets/sass_functions.rb
2016-05-10T03:04:11.572684+00:00 app[web.1]: -- C level backtrace information -------------------------------------------
2016-05-10T03:04:11.580053+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(rb_vm_bugreport+0x51f) [0x7fc9c783dfbf] vm_dump.c:693
2016-05-10T03:04:11.580095+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(rb_bug+0xca) [0x7fc9c78af95a] error.c:409
2016-05-10T03:04:11.580156+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(newobj_of+0x1be) [0x7fc9c770932e] gc.c:1635
2016-05-10T03:04:11.580241+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(rb_str_dup+0x1b) [0x7fc9c77c3fcb] string.c:552
2016-05-10T03:04:11.582165+00:00 app[web.1]:   981 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/helpers.rb
2016-05-10T03:04:11.580402+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(rb_class_path+0x83) [0x7fc9c77fea03] variable.c:259
2016-05-10T03:04:11.580460+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(thread_start_func_2+0x1e1) [0x7fc9c784c5c1] thread.c:2717
2016-05-10T03:04:11.580543+00:00 app[web.1]: /app/vendor/ruby-2.2.2/bin/ruby(thread_start_func_1+0xbb) [0x7fc9c784d0eb] thread_pthread.c:846
2016-05-10T03:04:11.580604+00:00 app[web.1]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xc2) [0x7fc9c7285182]
2016-05-10T03:04:11.580654+00:00 app[web.1]: /lib/x86_64-linux-gnu/libc.so.6(__clone+0x6d) [0x7fc9c686f47d]

Segfault in the Ruby virtual machine

A segfault can occur in the Ruby virtual machine (VM) written in C. However, this is very unlikely: the VM for Ruby 2+ has been very stable, and raising a segfault by writing pure Ruby code is rare. Usually, a segfault is caused by a native extension using a memory-unsafe language like C or C++.

The Ruby language is considered memory-safe as users do not manually manage memory. Instead, Ruby uses a garbage collector. Any segfault triggered by pure Ruby code (without a native extension) is considered a bug by Ruby core.

While Ruby is a memory-safe language, it is implemented in a memory-unsafe language. Most Ruby internals and the garbage collector are written in C, which is memory-unsafe. However, finding a segfault that can be triggered with pure Ruby code is uncommon.

If your application segfaults, first make sure you’re using the latest Ruby version patch release. If the cause was a bug in Ruby internals, the problem could have been reported and fixed. If that’s the case, upgrading may resolve your issue.

If the segfault persists after ensuring your application is on the latest Ruby version, follow the directions below to rule out the possibility of a native extension causing the problem. If you can create a minimal reproduction for the problem without native extensions, then report it to Ruby core on their bug tracker.

Segfault in a native extension

Most libraries that come from rubygems.org are written in Ruby. Some libraries are written in other languages and use Ruby’s provided foreign function interface (FFI) or C API to interact with Ruby code. When these languages are compiled into native machine code, they’re known as native extensions. Most segfaults come from an error in a memory-unsafe language used to implement a native extension. Most commonly, the segfault will come from a C or C++ extension.

There are many reasons to use or write a native extension. They are written to improve performance or to utilize libraries on Heroku directly by importing their headers and using their APIs.

You can find out what libraries you have that contain C-extensions by running this command:

$ bundle show --paths | ruby -e "STDIN.each_line {|dep| puts dep.split('/').last if File.directory?(File.join(dep.chomp, 'ext')) }"
bcrypt-3.1.18
bindex-0.8.1
bluecloth-2.2.0
bootsnap-1.13.0
concurrent-ruby-1.1.10
ffi-1.15.4
json-2.6.2
msgpack-1.5.6
nio4r-2.5.8
nokogiri-1.13.8
oj-3.13.21
pg-1.2.3
puma-5.6.5
rb-fsevent-0.11.0
rbtrace-0.4.14
sassc-2.4.0
scout_apm-5.3.1
skylight-5.3.3
stackprof-0.2.17
unf_ext-0.0.7.7
websocket-driver-0.7.5

This example output comes from the open source application CodeTriage.

Once you have a list of native extensions for your application, try upgrading the versions to see if the segfault has been reported and fixed already. Also, ensure you’ve upgraded your Ruby version to the latest patch release.

If upgrading native extensions and your Ruby version does not work, you can use the Ruby backtrace in the segfault for more information. Here is a shortened example:

2016-05-10T03:04:11.580659+00:00 app[web.1]: -- Other runtime information -----------------------------------------------
2016-05-10T03:04:11.580659+00:00 app[web.1]:
2016-05-10T03:04:11.580661+00:00 app[web.1]: * Loaded script: puma: cluster worker 1: 27 [app]
2016-05-10T03:04:11.580662+00:00 app[web.1]:
2016-05-10T03:04:11.580663+00:00 app[web.1]: * Loaded features:
2016-05-10T03:04:11.580664+00:00 app[web.1]:
2016-05-10T03:04:11.580667+00:00 app[web.1]:     0 enumerator.so
2016-05-10T03:04:11.580668+00:00 app[web.1]:     1 rational.so
2016-05-10T03:04:11.580668+00:00 app[web.1]:     2 complex.so
2016-05-10T03:04:11.580671+00:00 app[web.1]:     3 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/x86_64-linux/enc/encdb.so
2016-05-10T03:04:11.580671+00:00 app[web.1]:     4 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/x86_64-linux/enc/trans/transdb.so
2016-05-10T03:04:11.580672+00:00 app[web.1]:     5 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/unicode_normalize.rb
2016-05-10T03:04:11.580673+00:00 app[web.1]:     6 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/x86_64-linux/rbconfig.rb
2016-05-10T03:04:11.580675+00:00 app[web.1]:     7 thread.rb
2016-05-10T03:04:11.580676+00:00 app[web.1]:     8 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/x86_64-linux/thread.so
2016-05-10T03:04:11.580676+00:00 app[web.1]:     9 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/compatibility.rb
2016-05-10T03:04:11.580677+00:00 app[web.1]:    10 /app/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/defaults.rb
2016-05-10T03:04:11.582166+00:00 app[web.1]:   982 /app/vendor/bundle/ruby/2.2.0/gems/sprockets-3.6.0/lib/sprockets/sass_importer.rb
2016-05-10T03:04:11.582166+00:00 app[web.1]:   983 /app/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.2/lib/tilt/mapping.rb
2016-05-10T03:04:11.582168+00:00 app[web.1]:   984 /app/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.2/lib/tilt/template.rb
2016-05-10T03:04:11.582169+00:00 app[web.1]:   985 /app/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.2/lib/tilt.rb
2016-05-10T03:04:11.582169+00:00 app[web.1]:   986 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/importer.rb
2016-05-10T03:04:11.582172+00:00 app[web.1]:   987 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/cache_store.rb
2016-05-10T03:04:11.582172+00:00 app[web.1]:   988 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/template.rb
2016-05-10T03:04:11.582173+00:00 app[web.1]:   989 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/logger.rb
2016-05-10T03:04:11.582175+00:00 app[web.1]:   990 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails/railtie.rb
2016-05-10T03:04:11.582175+00:00 app[web.1]:   991 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass/rails.rb
2016-05-10T03:04:11.582176+00:00 app[web.1]:   992 /app/vendor/bundle/ruby/2.2.0/gems/sass-rails-5.0.4/lib/sass-rails.rb
2016-05-10T03:04:11.582178+00:00 app[web.1]:   993 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/version.rb
2016-05-10T03:04:11.582178+00:00 app[web.1]:   994 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/module.rb
2016-05-10T03:04:11.582179+00:00 app[web.1]:   995 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/encoding.rb
2016-05-10T03:04:11.582182+00:00 app[web.1]:   996 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/runtime.rb
2016-05-10T03:04:11.582182+00:00 app[web.1]:   997 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/disabled_runtime.rb
2016-05-10T03:04:11.582183+00:00 app[web.1]:   998 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/duktape_runtime.rb
2016-05-10T03:04:11.582185+00:00 app[web.1]:   999 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/external_runtime.rb
2016-05-10T03:04:11.582186+00:00 app[web.1]:  1000 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/ruby_racer_runtime.rb
2016-05-10T03:04:11.582187+00:00 app[web.1]:  1001 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/ruby_rhino_runtime.rb
2016-05-10T03:04:11.582187+00:00 app[web.1]:  1002 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs/runtimes.rb
2016-05-10T03:04:11.582190+00:00 app[web.1]:  1003 /app/vendor/bundle/ruby/2.2.0/gems/execjs-2.6.0/lib/execjs.rb
2016-05-10T03:04:11.582190+00:00 app[web.1]:  1004 /app/vendor/bundle/ruby/2.2.0/gems/uglifier-3.0.0/lib/uglifier/version.rb
2016-05-10T03:04:11.582191+00:00 app[web.1]:  1005 /app/vendor/bundle/ruby/2.2.0/gems/uglifier-3.0.0/lib/uglifier.rb
2016-05-10T03:04:11.582194+00:00 app[web.1]:  1006 /app/vendor/bundle/ruby/2.2.0/gems/coffee-script-source-1.10.0/lib/coffee_script/source.rb
2016-05-10T03:04:11.582194+00:00 app[web.1]:  1007 /app/vendor/bundle/ruby/2.2.0/gems/coffee-script-2.4.1/lib/coffee_script.rb

Work backwards to try and identify the cause using the backtrace. The gem name in the backtrace might be different from the gem on the system. In this case the unf_ext gem shows up as unf-<version> for example:

2016-05-10T03:04:11.590668+00:00 app[web.1]:  1346 /app/vendor/bundle/ruby/2.2.0/gems/unf-0.1.4/lib/unf/version.rb
2016-05-10T03:04:11.581498+00:00 app[web.1]:   517 /app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.4/lib/active_support/core_ext/string/access.rb
2016-05-10T03:04:11.590669+00:00 app[web.1]:  1347 /app/vendor/bundle/ruby/2.2.0/gems/unf-0.1.4/lib/unf.rb

If you determine that a gem is causing your segfault, please open up an issue with the gem maintainer. Provide the total segfault error output, and whenever possible, include a small example application that reproduces the segfault.

“There was an error parsing your Gemfile” Segfault

A segfault can be caused by an interaction between the environment variable, RUBY_THREAD_VM_STACK_SIZE, and the Ruby buildpack. The segfault will look like this:

-----> Compiling Ruby/Rails
 !
 !     There was an error parsing your Gemfile, we cannot continue
 !     /tmp/codon/tmp/buildpacks/50d5eddf222a9b7326028041d4e6509f915ccf2c/vendor/ruby/heroku-20/lib/ruby/3.0.0/x86_64-linux/enc/encdb.so: [BUG] Segmentation fault at 0x0000000000000018
 !     ruby 3.0.3p157 (2021-11-24 revision 3fb7d2cadc) [x86_64-linux]
 !
 !     -- Control frame information -----------------------------------------------
 !     c:0002 p:-11861427355948 s:0006 e:000005 TOP    [FINISH]
 !     c:0001 p:0000 s:0003 E:001870 (none) [FINISH]

Note the error message is There was an error parsing your Gemfile, we cannot continue. The location of the segfault is in encdb.so. Also, note that this exception happens after the buildpack determines your bundler version but before it determines your Ruby version.

If you get this segfault, remove the following config var from your application:

$ heroku config:unset RUBY_THREAD_VM_STACK_SIZE

This exception occurs because the Ruby buildpack uses a version of Ruby to run itself. On February 24th of 2022, Heroku engineers upgraded the Ruby buildpack version from 2.x to 3.x. This version of Ruby (3.x+) will segfault with values in RUBY_THREAD_VM_STACK_SIZE that previously did not. The solution is to remove this environment variable from your configuration.