This add-on is operated by RadioKit LTD

Pro Web Audio API

Last updated August 16, 2021

Audiomatic is an add-on for processing and analyzing audio files of any kind!

Neutrino Audiomatic provides power of professional music studio as a service. Lightning fast generation of waveform? Check! Duration of audio file in millisecond precision? No problem. Conversion of any obscure format to web-ready mp3? 3.. 2… 1… Done! Say ‘no’ to hours of googling, installing hundreds of plugins or obscure software. Leave these bad parts of audio editing to us, so you have time for the important stuff.

Neutrino Audiomatic is accessible via an API and has supported client libraries for Ruby. More coming soon.

Provisioning the add-on

Neutrino Audiomatic can be attached to a Heroku application via the CLI:

$ heroku addons:create neutrino-audiomatic
-----> Adding neutrino-audiomatic to sharp-mountain-4005... done, v18 (free)

Once Neutrino Audiomatic has been added a NEUTRINO_AUDIOMATIC_URL setting will be available in the app configuration and will contain the canonical URL used to access the newly provisioned Audiomatic service instance. This can be confirmed using the heroku config:get command.

$ heroku config:get NEUTRINO_AUDIOMATIC_URL
http://user:pass@instance.ip/resourceid

After installing Neutrino Audiomatic the application should be configured to fully integrate with the add-on.

Http Api

NeutrinoAudiomatic Request

Scheduling file processing requires POSTing two parameters to audiomatic api: - url - location of processed file, - callback_url - location where results of analysis should be returned as JSON.

Url depends on desired processor. Root of API2.0 requests: - https://audiomatic.radiokitapp.org/api/process/v2.0 should be concatenated with processor’s path:

In short, to obtain duration, POST to: - https://audiomatic.radiokitapp.org/api/process/v2.0/analysis/audio/duration

JSON body:

{
 "callback_url":"https://example.org/callback",
 "url":"https://example.org/file.mp3"
}

Don’t forget about authentication. Use Basic Auth with login and token you obtained during registration. You can use either Authorization Header or URL encoding. For details, see https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side

NeutrinoAudiomatic Response

After sends request to NeutrinoAudiomatic, server answers twice. First response will be sent immediately and it will contain status of request. Second, will be sent results of analyze.

First response:

If status is 202, mean that NeutrinoAudiomatic get your request and start working on it. Body of successful response contain reference number which is unique attribute of single analyze. Otherwise, something went wrong.

Seccond response:

Results are in json similar to:

{"result"=>[{"value"=>"122323", "key"=>"duration"}]} # => duration

{"result"=>[{"value"=>"AC/DC", "key"=>"artist"}, {"value"=>"T.N.T", "key"=>"title"}]} # => tags

{"result"=>[{"value"=>"http://www.example.org/file.mp3", "key"=>"webbrowser"}]} # => webbrowser

{"result"=>[{"value"=>0.05550943315029144, "key"=>"replaygain-track-peak"}, {"value"=>20.32999999999999, "key"=>"replaygain-track-gain"}, {"value"=>89, "key"=>"replaygain-reference-level"}]} # => replay gain

{"result"=>[{"value"=>"http://www.example.org/file.png", "key"=>"waveform"}]} # => waveform

Using with Rails 4.x, 5.x

Ruby on Rails applications will need to add the following entry into their Gemfile specifying the Neutrino Audiomatic client library.

gem 'neutrino-audiomatic-rails'

Update application dependencies with bundler.

$ bundle install

Configuration

For heroku users, configuration is really easy. Just install Neutrino_Audiomatic (url) plugin and paste code-block below to config/initializer/audiomatic_config.rb:

  Neutrino::Audiomatic.config do |c|
    c.processor_host = ENV['NEUTRINO_AUDIOMATIC_URL']
    c.audiomatic_secret = ENV['NEUTRINO_AUDIOMATIC_SHARED_SECRET']
  end

you can also provide your own unique url:

  Neutrino::Audiomatic.config do |c|
    c.processor_host = "https://login:password@audiomatic.radiokitapp.org/api/process"
    c.audiomatic_secret = your_secret
  end

also you have to set the routes default_url_options in environments/production.rb

  Rails.application.routes.default_url_options[:host] = 'your_app_url'

Getting start

1.Store file which you want to analyze. This gem provides method store which simplifies it. Example of controller:

  def create
    @audio_file = AudioFile.new
    @audio_file.store params[:audio_file][:audio]
    @audio_file.save
  end

2.Include Neutrino::AudiomaticRails::Model in your ActiveRecord Model

  class YourModel < ApplicationRecord

    include Neutrino::AudiomaticRails::Model

  end

3.Configure processors. Set field where result should be saved (don’t forget to adding column to database):

  # model should have 'duration' field
  analyze :duration, [:duration]

  # or (if file is named different then result)
  analyze :duration {duration: {field: :duration_field}}

4.Keep adding processors:

  analyze :duration, {duration: {transformation: to_second= }}
  analyze :tags, [:artist, :title]
  analyze :waveform, mount_on: :waveform
  analyze :browser, mount_on: :mp3_file
  analyze :replaygain, save_to: replaygain_method=

5.Last step: tell NeutrinoAudiomatic where results should be sent back. Add to your routes.rb:

  # config/routes.rb

  Rails.application.routes.draw do

    audiomatic_for(YourModel)

  end

And that’s it!

Sets analyze

NeutrinoAudiomaticRails gives many possibilities of processors configuration which will be done on your files. Gem has a lot of default configuration, but also gives capabilities to customize it. Method analyze describes which result we want to get, and how the result is handled.

Saving directly to ActiveRecord field

The most default option: Select processor and put result to Array. It saves results to fields in yourModel (name of result must be the same as name of field)

    #processor #result
  analyze :duration, [:duration]

To customize this strategy you can use Hash instead of Array, and select extra options:

• :field - allows select field for result
• :transformation - allows convert result before saving to db, transformations methods have to be in Array

Example:

  analyze :replaygain, {'replaygain-track-peak': {field: :replaygain_track_peak}, 'replaygain-track-gain': {field: :'replaygain_track_gain', transformation: [:to_s=]}}

  analyze :duration, {duration: {transformation: [:to_s=]}}

Handle answer

If you want to get all results from processor and save or interpret it on your own, you have to use option :save_to which lets you write own method that would get result and do everything you want. Example:

      #processor #result
  analyze :tags, save_to: :tags=

  def tags= result # result is json {'key': name, 'value': value }
    #your code...
  end

With CarrierWave

Because in some cases NeutrinoAudiomatic returns file, neutrino_audiomatic_rails provides easy way to store this file in your Uploader. Example:

  mount_uploader :waveform, WaveformUploader
    mount_uploader :mp3_file, Mp3Uploader

    analyze :webbrowser, mount_on: :mp3_file
    analyze :waveform, mount_on: :waveform

It means mount_on: :uploader is equal to self.uploader.store! file. Now, results are in your Uploader.

Stores files

To analyze file NeutrinoAudiomatic needs a URL to your file. Neutrino_audiomatic_rails provides store method which saves your file in tmp folder and creates routes to get it. It will be used during sending request to NeutrinoAudiomatic.

  def create
    @audio_file = AudioFile.new
    @audio_file.store params[:audio_file][:audio]
    @audio_file.save
  end

If you already have audio file stored. you can provide your own method which would return url to file to analyze.

  audiomatic_file_url :set_url
  def set_url
    #your code
  end

NeutrinoAudiomatic errors

Sending request to NeutrinoAudiomac starts after saving instance of yourModel. While sending request to NeutrinoAudiomatic something can go wrong. In default configuration if server sends response with status different then 202, method in gem raises exception, rollbacks transaction and method save returns false.

Second strategy allows you write your own reaction on error. All you have to do is to write method and set it as audiomatic_error. This method needs to have one argument. If your method returns true, then gem uses ActiveJob and tries sending request to server again after 15 minutes. If method returns false then gem raises exception and deletes transaction.

  audiomatic_error :handle_error=

  def handle_error= error
    #your code
  end

Response authentication

Response sent from NeutrinoAudiomatic can be verified by calculation a signature.

All response contain HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256 header which is generated using the app’s shared secret and body of response.

To verify that the request come from NeutrinoAudiomatic, compute the HMAC digest and then compere value and HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256 header. If they the same, you can be sure the response come from NeutrinoAudiomatic.

Simple example:

  def self.verify_neutrino(data, hmac_header)
    secret = ENV['AUDIOMATIC_SHARED_SECRET']
    calculated_hmac = Digest::SHA1.hexdigest(data + ":" + secret)
    calculated_hmac == hmac_header
  end

  def self.verify_response request
    request.body.rewind
    data = request.body.read
    verified = verify_neutrino(data,        request.headers["HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256"])
  end

Gem neutrino_audiomatic_rails authenticates response for you

Standalone client

If Rails is not your thing, you can still use Neutrino Audiomatic with the standalone Ruby gem. Just add this to your Gemfile:

gem 'neutrino-audiomatic'

And somewhere in your initialization path:

Neutrino::Audiomatic.config do |c|
  c.processor_host = ENV['NEUTIRNO_AUDIOMATIC_URL']
end

Then you have to figure out how you want to receive results.Neutrino Audiomatic doesn’t want to slow down your app - instead of waiting for Neutrino Audiomatic to finish processing, we send you your data the moment it has been processed. Where we send it is up to you - your app should provide secure endpoints for this. Usually it requires creating a dedicated route and controller for receiving the data.

In Sinatra, it would look something like:

require 'sinatra'

post "/endpoint/for/neutrino_audiomatic/:track_id" do
   params["results"].each do |result|
      if result["key"] == "duration"
         duration = result["value"]
         ## Do something with value
      end
   end

   # return status 200
   status 200
end

If you don’t return status 200, Neutrino Audiomatic will retry at least 5 times in varying intervals.

With an endpoint in place, you are free to use Neutrino Audiomatic processors:

    file_url = "http://mysite.com/path/to/file.mp3"
    callback_url = "http://mysite.com//endpoint/for/neutrino_audiomatic/id_of_my_track"
    NeutrinoAudiomatic::Processor.replaygain(file_url, callback_url: callback_url)

Security

You shouldn’t leave your endpoint unprotected. Since the callback URL is not changed by Neutrino Audiomatic, you should sign it.

Processors

Duration

Provides accurate duration of audio file with millisecond precision.

Response:

  {"result": [{"key": "duration", "value": 100000}]} # Value is duration in milliseconds

Standalone:

    Neutrino::Audiomatic::Processor.duration(file_url, callback_url: callback_url)

ActiveRecord

    analyze :duration, [:duration]

Replay Gain

Provides Replay Gain reference level, track peak and track gain.

ReplayGain is a standard to measure the perceived loudness of audio in computer audio formats. It allows players to normalize loudness for individual tracks or albums. This avoids the common problem of having to manually adjust volume levels between tracks when playing audio files from albums that have been mastered at different loudness levels.

Response:

  {"result": [{"key": "replaygain-track-peak", "value": 1.0},{"key": "replaygain-track-gain", "value": 0.96},{"key": "replaygain-reference-level", "value": 86.0},]} # Values are decibels as floats

Standalone:

    NeutrinoAudiomatic::Processor.replaygain(file_url, callback_url: callback_url)

ActiveRecord

    analyze :replaygain, [:]

Tags

Extracts tags metadata from a file.

Response:

  {"result": [{"key": "artist", "value": "Led Zeppelin"}, {"key": "album", "Led Zeppelin"}]} # Tags are mirrored in keys and values.

Standalone:

    Neutrino::Audiomatic::Processor.tags(file_url, callback_url: callback_url)

ActiveRecord:

    analyze :tags, [:artist, :album]

Webbrowser

Converts any file to mp3 format

Response:

  {"result": [{"key": "webbrowser", "value": "http://example.org/url/to/file"}]} # Value is url with converted file. URL is accessible for 24 hours.

Standalone:

    NeutrinoAudiomatic::Processor.webbrowser(file_url, callback_url: callback_url)

ActiveRecord:

    analyze :webbrowser, [:mp3]

Waveform:

Returns a PNG file with the waveform.

Response:

  {"result": [{"key": "waveform", "value": "http://example.org/url/to/file"}]} # Value is url with waveform. URL is accessible for 24 hours.

Standalone:

    NeutrinoAudiomatic::Processor.waveform(file_url, callback_url: callback_url)

ActiveRecord:

    analyze :waveform, [:waveform]

Dashboard

For more information on the features available within the Audiomatic dashboard please see the docs at neutrino-audiomatic.readthedocs.io.

The Neutrino Audiomatic dashboard allows you to check your data usage, browse logs, confirm status of processed files and more.

The dashboard can be accessed via the CLI:

$ heroku addons:open neutrino-audiomatic
Opening neutrino-audiomatic for sharp-mountain-4005

or by visiting the Heroku Dashboard and selecting the application in question. Select Neutrino Audiomatic from the Add-ons menu.

Advanced

Custom setters

Sometimes you may not be that eager to throw whatever we send you into your precious database. No worries, we have you covered.

Instead of using a database field as save-to point, you can just define your own setter:

  def save_duration= duration
    if duration.is_a?(Integer) and duration.between?(0, 3600*1000)
      self.write_attribute :duration, duration.to_s
    end
  end

  analyze :duration, [:save_duration]

With processors returning variable fields, you can get more granular.

Let’s say you need to extract artist and album data from metadata and save them to fields “artist_name” and “album_title”, but album should always be capitalized. Just specify fields and transformations:

  analyze :tags, {artist: {field: :artist_name}, album: {field: :album_title, transformation: [:capitalize]}}

  def capitalize value
    value.capitalize
  end

Troubleshooting

Migrating between plans

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade neutrino-audiomatic:small
-----> Upgrading neutrino-audiomatic:small to sharp-mountain-4005... done, v18 ($7/mo)
       Your plan has been updated to: neutirno-audiomatic:small

Removing the add-on

Audiomatic can be removed via the CLI.

$ heroku addons:destroy neutrino-audiomatic
-----> Removing neutrino-audiomatic from sharp-mountain-4005... done, v20 (free)

Support

All Neutrino Audiomatic support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at info@neutrino.audio.