Neutrino Audiomatic
Last updated July 27, 2023
This article is a work in progress, or documents a feature that is not yet released to all users. This article is unlisted. Only those with the link can access it.
The Neutrino Audiomatic add-on is currently in beta.
Table of Contents
Audiomatic is an add-on for processing and analyzing audio files of any kind!
Neutrino Audiomatic provides power of professional music studio as a service. Lightning fast generation of waveform? Check! Duration of audio file in millisecond precision? No problem. Conversion of any obscure format to web-ready mp3? 3.. 2… 1… Done! Say ‘no’ to hours of googling, installing hundreds of plugins or obscure software. Leave these bad parts of audio editing to us, so you have time for the important stuff.
Neutrino Audiomatic is accessible via an API and has supported client libraries for Ruby. More coming soon.
Provisioning the add-on
Neutrino Audiomatic can be attached to a Heroku application via the CLI:
A list of all plans available can be found here.
$ heroku addons:create neutrino-audiomatic
-----> Adding neutrino-audiomatic to sharp-mountain-4005... done, v18 (free)
Once Neutrino Audiomatic has been added a NEUTRINO_AUDIOMATIC_URL
setting will be available in the app configuration and will contain the canonical URL used to access the newly provisioned Audiomatic service instance. This can be confirmed using the heroku config:get
command.
$ heroku config:get NEUTRINO_AUDIOMATIC_URL
http://user:pass@instance.ip/resourceid
After installing Neutrino Audiomatic the application should be configured to fully integrate with the add-on.
Http Api
NeutrinoAudiomatic Request
Scheduling file processing requires POSTing two parameters to audiomatic api: - url - location of processed file, - callback_url - location where results of analysis should be returned as JSON.
Url depends on desired processor. Root of API2.0 requests: - https://audiomatic.radiokitapp.org/api/process/v2.0 should be concatenated with processor’s path:
Processors | Path |
---|---|
duration: | “/analysis/audio/duration” |
replaygain: | “/analysis/audio/replaygain” |
tags: | “/analysis/audio/tags” |
webbrowser: | “/transcode/audio/webbrowser” |
waveform: | “/visualisation/audio/waveform” |
In short, to obtain duration, POST to: - https://audiomatic.radiokitapp.org/api/process/v2.0/analysis/audio/duration
JSON body:
{
"callback_url":"https://example.org/callback",
"url":"https://example.org/file.mp3"
}
Don’t forget about authentication. Use Basic Auth with login and token you obtained during registration. You can use either Authorization Header or URL encoding. For details, see https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side
NeutrinoAudiomatic Response
After sends request to NeutrinoAudiomatic, server answers twice. First response will be sent immediately and it will contain status of request. Second, will be sent results of analyze.
First response:
If status is 202, mean that NeutrinoAudiomatic get your request and start working on it. Body of successful response contain reference number which is unique attribute of single analyze. Otherwise, something went wrong.
Seccond response:
Results are in json similar to:
{"result"=>[{"value"=>"122323", "key"=>"duration"}]} # => duration
{"result"=>[{"value"=>"AC/DC", "key"=>"artist"}, {"value"=>"T.N.T", "key"=>"title"}]} # => tags
{"result"=>[{"value"=>"http://www.example.org/file.mp3", "key"=>"webbrowser"}]} # => webbrowser
{"result"=>[{"value"=>0.05550943315029144, "key"=>"replaygain-track-peak"}, {"value"=>20.32999999999999, "key"=>"replaygain-track-gain"}, {"value"=>89, "key"=>"replaygain-reference-level"}]} # => replay gain
{"result"=>[{"value"=>"http://www.example.org/file.png", "key"=>"waveform"}]} # => waveform
Processor | Results |
---|---|
duration | “duration” [ms] |
replaygain | “replaygain-track-peak”,“replaygain-track-gain” “replaygain-reference-level” |
tags | all tags of audio file |
webbrowser | url to mp3 file |
Using with Rails 4.x, 5.x
Ruby on Rails applications will need to add the following entry into their Gemfile
specifying the Neutrino Audiomatic client library.
gem 'neutrino-audiomatic-rails'
Update application dependencies with bundler.
$ bundle install
Configuration
For heroku users, configuration is really easy. Just install Neutrino_Audiomatic
(url) plugin and paste code-block below to
config/initializer/audiomatic_config.rb
:
Neutrino::Audiomatic.config do |c|
c.processor_host = ENV['NEUTRINO_AUDIOMATIC_URL']
c.audiomatic_secret = ENV['NEUTRINO_AUDIOMATIC_SHARED_SECRET']
end
you can also provide your own unique url:
Neutrino::Audiomatic.config do |c|
c.processor_host = "https://login:password@audiomatic.radiokitapp.org/api/process"
c.audiomatic_secret = your_secret
end
also you have to set the routes default_url_options in environments/production.rb
Rails.application.routes.default_url_options[:host] = 'your_app_url'
Getting start
1.Store file which you want to analyze. This gem provides method store
which simplifies it. Example of controller:
def create
@audio_file = AudioFile.new
@audio_file.store params[:audio_file][:audio]
@audio_file.save
end
2.Include Neutrino::AudiomaticRails::Model in your ActiveRecord Model
class YourModel < ApplicationRecord
include Neutrino::AudiomaticRails::Model
end
3.Configure processors. Set field where result should be saved (don’t forget to adding column to database):
# model should have 'duration' field
analyze :duration, [:duration]
# or (if file is named different then result)
analyze :duration {duration: {field: :duration_field}}
4.Keep adding processors:
analyze :duration, {duration: {transformation: to_second= }}
analyze :tags, [:artist, :title]
analyze :waveform, mount_on: :waveform
analyze :browser, mount_on: :mp3_file
analyze :replaygain, save_to: replaygain_method=
5.Last step: tell NeutrinoAudiomatic where results should be sent back. Add to your routes.rb:
# config/routes.rb
Rails.application.routes.draw do
audiomatic_for(YourModel)
end
And that’s it!
Sets analyze
NeutrinoAudiomaticRails gives many possibilities of processors configuration which will be done on your files. Gem has a lot of default configuration, but also gives capabilities to customize it. Method analyze
describes which result we want to get, and how the result is handled.
Saving directly to ActiveRecord field
The most default option: Select processor and put result to Array. It saves results to fields in yourModel (name of result must be the same as name of field)
#processor #result
analyze :duration, [:duration]
To customize this strategy you can use Hash instead of Array, and select extra options:
- :field - allows select field for result
- :transformation - allows convert result before saving to db, transformations methods have to be in Array
Example:
analyze :replaygain, {'replaygain-track-peak': {field: :replaygain_track_peak}, 'replaygain-track-gain': {field: :'replaygain_track_gain', transformation: [:to_s=]}}
analyze :duration, {duration: {transformation: [:to_s=]}}
Handle answer
If you want to get all results from processor and save or interpret it on your own, you have to use option :save_to
which lets you write own method that would get result and do everything you want.
Example:
#processor #result
analyze :tags, save_to: :tags=
def tags= result # result is json {'key': name, 'value': value }
#your code...
end
With CarrierWave
Because in some cases NeutrinoAudiomatic returns file, neutrino_audiomatic_rails provides easy way to store this file in your Uploader. Example:
mount_uploader :waveform, WaveformUploader
mount_uploader :mp3_file, Mp3Uploader
analyze :webbrowser, mount_on: :mp3_file
analyze :waveform, mount_on: :waveform
It means mount_on: :uploader
is equal to self.uploader.store! file
.
Now, results are in your Uploader.
Stores files
To analyze file NeutrinoAudiomatic needs a URL to your file. Neutrino_audiomatic_rails provides store
method which saves your file in tmp folder and creates routes to get it. It will be used during sending request to NeutrinoAudiomatic.
def create
@audio_file = AudioFile.new
@audio_file.store params[:audio_file][:audio]
@audio_file.save
end
If you already have audio file stored. you can provide your own method which would return url to file to analyze.
audiomatic_file_url :set_url
def set_url
#your code
end
NeutrinoAudiomatic errors
Sending request to NeutrinoAudiomac starts after saving instance of yourModel. While sending request to NeutrinoAudiomatic something can go wrong. In default configuration if server sends response with status different then 202, method in gem raises exception, rollbacks transaction and method save
returns false.
Second strategy allows you write your own reaction on error. All you have to do is to write method and set it as audiomatic_error. This method needs to have one argument. If your method returns true, then gem uses ActiveJob and tries sending request to server again after 15 minutes. If method returns false then gem raises exception and deletes transaction.
audiomatic_error :handle_error=
def handle_error= error
#your code
end
Response authentication
Response sent from NeutrinoAudiomatic can be verified by calculation a signature.
All response contain HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256
header which is generated using the app’s shared secret and body of response.
To verify that the request come from NeutrinoAudiomatic, compute the HMAC digest and then compere value and HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256
header. If they the same, you can be sure the response come from NeutrinoAudiomatic.
Simple example:
def self.verify_neutrino(data, hmac_header)
secret = ENV['AUDIOMATIC_SHARED_SECRET']
calculated_hmac = Digest::SHA1.hexdigest(data + ":" + secret)
calculated_hmac == hmac_header
end
def self.verify_response request
request.body.rewind
data = request.body.read
verified = verify_neutrino(data, request.headers["HTTP_X_NEUTRINO_AUDIOMATIC_HMAC_SHA256"])
end
Gem neutrino_audiomatic_rails authenticates response for you
Standalone client
If Rails is not your thing, you can still use Neutrino Audiomatic with the standalone Ruby gem. Just add this to your Gemfile:
gem 'neutrino-audiomatic'
And somewhere in your initialization path:
Neutrino::Audiomatic.config do |c|
c.processor_host = ENV['NEUTIRNO_AUDIOMATIC_URL']
end
Then you have to figure out how you want to receive results.Neutrino Audiomatic doesn’t want to slow down your app - instead of waiting for Neutrino Audiomatic to finish processing, we send you your data the moment it has been processed. Where we send it is up to you - your app should provide secure endpoints for this. Usually it requires creating a dedicated route and controller for receiving the data.
In Sinatra, it would look something like:
require 'sinatra'
post "/endpoint/for/neutrino_audiomatic/:track_id" do
params["results"].each do |result|
if result["key"] == "duration"
duration = result["value"]
## Do something with value
end
end
# return status 200
status 200
end
If you don’t return status 200, Neutrino Audiomatic will retry at least 5 times in varying intervals.
With an endpoint in place, you are free to use Neutrino Audiomatic processors:
file_url = "http://mysite.com/path/to/file.mp3"
callback_url = "http://mysite.com//endpoint/for/neutrino_audiomatic/id_of_my_track"
NeutrinoAudiomatic::Processor.replaygain(file_url, callback_url: callback_url)
Security
You shouldn’t leave your endpoint unprotected. Since the callback URL is not changed by Neutrino Audiomatic, you should sign it.
Processors
Duration
Provides accurate duration of audio file with millisecond precision.
Response:
{"result": [{"key": "duration", "value": 100000}]} # Value is duration in milliseconds
Standalone:
Neutrino::Audiomatic::Processor.duration(file_url, callback_url: callback_url)
ActiveRecord
analyze :duration, [:duration]
Replay Gain
Provides Replay Gain reference level, track peak and track gain.
ReplayGain is a standard to measure the perceived loudness of audio in computer audio formats. It allows players to normalize loudness for individual tracks or albums. This avoids the common problem of having to manually adjust volume levels between tracks when playing audio files from albums that have been mastered at different loudness levels.
Response:
{"result": [{"key": "replaygain-track-peak", "value": 1.0},{"key": "replaygain-track-gain", "value": 0.96},{"key": "replaygain-reference-level", "value": 86.0},]} # Values are decibels as floats
Standalone:
NeutrinoAudiomatic::Processor.replaygain(file_url, callback_url: callback_url)
ActiveRecord
analyze :replaygain, [:]
Tags
Extracts tags metadata from a file.
Response:
{"result": [{"key": "artist", "value": "Led Zeppelin"}, {"key": "album", "Led Zeppelin"}]} # Tags are mirrored in keys and values.
Standalone:
Neutrino::Audiomatic::Processor.tags(file_url, callback_url: callback_url)
ActiveRecord:
analyze :tags, [:artist, :album]
Webbrowser
Converts any file to mp3 format
Response:
{"result": [{"key": "webbrowser", "value": "http://example.org/url/to/file"}]} # Value is url with converted file. URL is accessible for 24 hours.
Standalone:
NeutrinoAudiomatic::Processor.webbrowser(file_url, callback_url: callback_url)
ActiveRecord:
analyze :webbrowser, [:mp3]
Waveform:
Returns a PNG file with the waveform.
Response:
{"result": [{"key": "waveform", "value": "http://example.org/url/to/file"}]} # Value is url with waveform. URL is accessible for 24 hours.
Standalone:
NeutrinoAudiomatic::Processor.waveform(file_url, callback_url: callback_url)
ActiveRecord:
analyze :waveform, [:waveform]
Dashboard
For more information on the features available within the Audiomatic dashboard please see the docs at neutrino-audiomatic.readthedocs.io.
The Neutrino Audiomatic dashboard allows you to check your data usage, browse logs, confirm status of processed files and more.
The dashboard can be accessed via the CLI:
$ heroku addons:open neutrino-audiomatic
Opening neutrino-audiomatic for sharp-mountain-4005
or by visiting the Heroku Dashboard and selecting the application in question. Select Neutrino Audiomatic from the Add-ons menu.
Advanced
Custom setters
Sometimes you may not be that eager to throw whatever we send you into your precious database. No worries, we have you covered.
Instead of using a database field as save-to point, you can just define your own setter:
def save_duration= duration
if duration.is_a?(Integer) and duration.between?(0, 3600*1000)
self.write_attribute :duration, duration.to_s
end
end
analyze :duration, [:save_duration]
With processors returning variable fields, you can get more granular.
Let’s say you need to extract artist and album data from metadata and save them to fields “artist_name” and “album_title”, but album should always be capitalized. Just specify fields and transformations:
analyze :tags, {artist: {field: :artist_name}, album: {field: :album_title, transformation: [:capitalize]}}
def capitalize value
value.capitalize
end
Troubleshooting
Migrating between plans
Use the heroku addons:upgrade
command to migrate to a new plan.
$ heroku addons:upgrade neutrino-audiomatic:small
-----> Upgrading neutrino-audiomatic:small to sharp-mountain-4005... done, v18 ($7/mo)
Your plan has been updated to: neutirno-audiomatic:small
Removing the add-on
Audiomatic can be removed via the CLI.
This will destroy all associated data and cannot be undone!
$ heroku addons:destroy neutrino-audiomatic
-----> Removing neutrino-audiomatic from sharp-mountain-4005... done, v20 (free)
Support
All Neutrino Audiomatic support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at info@neutrino.audio.