Audio previews in Rails without a job queue

Suppose you have this model:

class Recording < ApplicationRecord
  has_one_attached :audio
end

Users upload 40-minute WAV files to it. Your app has a couple of different requirements:

The feed needs a 30-second preview,
the player needs a waveform,
downloads should be MP3,
streaming should be Opus,
everything should be loudness-normalised, because half of what people upload is recorded too quiet.

The Rails-shaped answer to that is familiar:

a Recording::Variant model, with an attachment per variant,
a job that shells out to ffmpeg,
a status column, because jobs fail,
a rake task to backfill, for the day somebody decides the preview should be 45 seconds.

I've built that (at least) twice. It works, but it's a lot of code that has nothing to do with the product.

audioproxy is the other option. Think imgproxy, for audio: one container between your storage and your users, rendering variants on demand. You simply describe the variant you want in the URL. It renders the first time somebody requests it, streams the bytes while ffmpeg is still encoding, and writes the result to a bucket or a directory you own, so every request after that hits the cache.

Look at the tally:

No variants table.
No job.
Nothing rendered in advance.
Nothing to backfill when you change your mind.

The previous post makes that case in general. This one is about integrating it into your Rails app.

Two Minutes of Setup in Your Rails App

# Gemfile
gem "audioproxy-rails"

You need a proxy running somewhere for this to point at. In development you can just start one with docker run with your audio directory mounted read only. Take a look at the quickstart walkthrough.

The gem needs to know where it is, and what to sign with. It reads AP_ENDPOINT, AP_KEY and AP_SALT from the environment, or the same three from credentials:

# bin/rails credentials:edit
audioproxy:
  endpoint: https://audio.example.com
  key: 00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff
  salt: ffeeddccbbaa99887766554433221100

Those variable names reflect the ones used when starting the proxy container. In development, you can also just use one env file to feed your app service and the proxy service in docker-compose.yml.

# .env

# Both services read these. Your app signs with them, the proxy verifies with them.
AP_KEY=00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff
AP_SALT=ffeeddccbbaa99887766554433221100

# Your app only: where the gem points. In compose, that's the service name.
AP_ENDPOINT=http://audioproxy:4000

# The proxy only: where it reads sources and writes finished variants.
AP_LOCAL_ROOT=/var/audio
AP_VARIANT_STORE=file:///var/cache/audioproxy
AP_SERVE_MODE=proxy

Two notes on the proxy half of that file. AP_LOCAL_ROOT has to name the same directory as your Disk service's root, which is the coupling described further down. And AP_SERVE_MODE=proxy is not optional next to a file:// variant store, because the proxy refuses to boot in its default redirect mode:

** (AudioProxy.Config.Error) AP_SERVE_MODE=redirect serves cache hits via
presigned URLs, which a file:// AP_VARIANT_STORE cannot produce; set
AP_SERVE_MODE=proxy or use a store that can presign

You can use an initializer to override these settings, if you like:

# config/initializers/audioproxy.rb
Audioproxy.configure do |config|
  config.endpoint = "https://audio-staging.example.com"
  config.default_options = { format: "opus", bitrate: 96 }
end

One Line in Your View

Now you have that recording and you'd like to show it on your site. Just add this:

<%= audioproxy_audio_tag @recording.audio,
      format: "opus", bitrate: 96,
      html: { controls: true, preload: "none" } %>

<audio controls="controls" preload="none"
  src="https://audio.example.com/QuA5t4QcZDzy.../f:opus/br:96/enc/bG9jYWw6Ly91Yy94cS91Y3hxMW52azZ5YWo1NW5yc2FwcnNqYzh5dGo0"></audio>

Let's quickly unravel this:

the first segment is a signature computed over everything after it, so nobody can point a URL at arbitrary work,
then the options, f: for the format and br: for the bitrate in kbps,
then the source, base64url-encoded because it's an s3:// or local:// URI.

That is the whole feature. The first person to load the page triggers the render, and everybody after that simply gets it from the variant store (if configured).

Three things happened in that one line:

the attachment became a source string,
the options became a path, and
the path got signed.

The options are where you will spend most of your time, so there are three rules worth knowing up front.

Proxy options are keyword arguments. HTML attributes go in html:. Without that split a mistyped bitrat: 96 would land on the <audio> element as an attribute while the URL quietly shipped the default format. With it, an unknown option raises:

ArgumentError: unknown Audioproxy option :bitrat; known keys are bd, br, cb, ch,
dl, exp, f, fade, gain, norm, pk_fmt, pts, q, sr, t, each also accepted as its
spelled-out alias (bitrate, sample_rate, peak_format, ...)

Different spellings coalesce to the same URL. f: :opus, br: 96 and format: "opus", bitrate: 96 produce identical paths, so they're one cache key.

Durations work where seconds are meant:

Audioproxy.url_for(source, format: "mp3", bitrate: 128,
                   trim: [0, 30.seconds], fade: [0, 1.5])
# => ".../f:mp3/br:128/t:0:30/fade:0:1.5/enc/..."

How Your Attachment Becomes a Source

A blob, an attachment, or the association itself all work:

Audioproxy.url_for(recording.audio)        # the association
Audioproxy.url_for(recording.audio.blob)   # the blob

If nothing is attached to recording, you get Audioproxy::UnattachedError: nothing is attached to Recording#audio (rather than a URL that would 404 an hour later).

On Disk storage there's one coupling you have to get right. For local:// sources the proxy resolves the path against its own AP_LOCAL_ROOT, so that has to point at the same directory as your Disk service's root:

# config/storage.yml
local:
  service: Disk
  root: /var/audio

# the proxy
AP_LOCAL_ROOT=/var/audio

Waveforms Are Free, Too

Peaks are a format, so they sign, cache and invalidate exactly like audio does:

Audioproxy.url_for(recording.audio, format: "peaks", peak_count: 800)
# => ".../f:peaks/pts:800/enc/..."

What comes back is audiowaveform's own version 2 format, which is what peaks.js already reads. Here is an example with 8 points instead of 800:

{"data":[-16583,15023,-18258,19488,-19352,17923,-22081,20980,-17073,18456,
         -20101,18077,-23225,19427,-17036,17673],
 "version":2,"length":8,"bits":16,"channels":1,
 "sample_rate":44100,"samples_per_pixel":1157904}

The good part is that the processing chain runs before the peaks are computed. Ask for the peaks of a normalised, trimmed, speech-cleaned variant, and you get the waveform of that variant rather than of the master.

So the drawing and the audio always agree. That is surprisingly hard to arrange when two separate jobs produce them at two different times.

Links That Expire

You can let your Audioproxy links expire like this:

Audioproxy.url_for(recording.audio, format: "mp3", expires_in: 10.minutes)
# => ".../f:mp3/exp:1789718075/enc/..."

Or set an expiry period once, globally:

Audioproxy.configure { |config| config.expires_in = 1.hour }

exp is signed into the path, but it deliberately doesn't invalidate the cache entry. So handing out a fresh link every hour re-signs the URL, it doesn't re-render the variant. Once that timestamp passes, the proxy answers 410.

What the First Request Costs

Because variants are computed fresh for a cold visit, the first request takes longer. Signing a URL with the gem and curling the container twice, on my laptop, for a 30-second Opus preview of a three-and-a-half minute mono MP3:

req1 status=200 bytes=381666 time=2.27s x-audio-proxy: MISS
req2 status=200 bytes=381666 time=0.04s x-audio-proxy: HIT

Two things are worth noting here:

The MISS is answered chunked while ffmpeg is still encoding, so playback starts well before those 2.27 seconds are up, and the bytes land in the variant store on the way past.
The HIT adds accept-ranges: bytes, which is what makes seeking work.

When You Still Want a Job Queue

It doesn't replace everything, though:

Converting an archive of 40,000 files tonight is a batch job. A proxy is the wrong shape for it.
Already have a working pipeline with one small hole? Patch the hole. A new runtime dependency isn't worth it for a clip shape you rendered once.
Need the variant to exist whether or not anyone asks for it, say as a contractual deliverable? Then pre-rendering is the requirement, not an implementation detail.

Try It Yourself

The project home page has a URL explorer running against a real deployment: change an option, watch the waveform redraw from a fresh render, see the cache verdict flip.