Granular Synthesis

Granular synthesis is a technique for creating textures out of recorded audio material. The texture consists of grains, which are very short slices of the original source material. For sufficiently small grains, the temporal features of the source material are hidden, but some timbral qualities carry over.

Several parameters can be used to control the texture coming out of the granular synth. Grains are enveloped, transposed, spatialized. Random variance can be introduced. In addition, granular synthesis can be used for time stretching.

Using Audio Material

We are going to use a sampled sound. I'm using a clip I downloaded from freesound.org. Please feel free to download that or to use any clip you may have handy.

The Kronos backend can import audio assets with the help of Audio:File. Audio files are large, and the Kronos compiler is not designed for passing them around as values. That's why the convenience function creates a function wrapper over the audio file, which you can freely pass around and use. It goes like this:

; change this to whatever the path is on your system:
path = "/Users/vnorilo/sounds/countdown.wav"

; obtain audio data
(waveform-sr waveform-frames waveform) = Audio:File(path)

; sample rate
waveform-sr#44100

; number of frames (samples) in file
waveform-frames#541856

; audio data function wrapper
waveform{Closure :Fn{} /Users/vnorilo/sounds/countdown.wav nil}

The file sample rate and frame count are reported as invariant constants. The actual audio data is accessible through waveform, which is a function that, given an index, returns an audio frame. In the case of monophonic audio, as the clip shown here, a frame is simply a sample.

Playback

We should hear something if we plug a moving signal into the waveform function:

Import Gen

; drive audio read head from an oscillator
pos = waveform-sr * 3 * (1 + Gen:Sin(1 / 16))
snd = waveform(pos)

Resampling

As you may hear, the resampling is of very low quality. That is because there is none! If we are going to transpose the sound, as is often the case in granular synthesis, we should add interpolation. There is a handy helper in the Function package that can wrap any integer-domain function with polynomial Hermite interpolation.

Import Function

waveform-ip = Function:Wrap-Hermite(waveform)

; waveform-ip is a function just like waveform, but samples
; four points in the source function in order to provide
; a smoothly interpolated curve.

snd = waveform-ip(pos)

Please note that waveform-ip is a function that behaves exactly as waveform, but is wrapped with an interpolator for better resampling quality. The function abstraction of audio data enables us to compose any sort of behavior into the audio material. One nifty addition is a timebase translation to seconds, which compensates for sample rate differences with resampling, now that we have interpolation in place.

; Use seconds instead of samples as audio timebase. We can do this
; by composing a multiplication in front of the interpolated
; waveform function.

waveform-t = Function:Compose(waveform-ip (* waveform-sr))

snd = waveform-t(1 + Gen:Sin(0.25))

Make it Grain

The building block of our granulator is a single grain. We basically need to scan a short region in the waveform function and apply an amplitude envelope. A grain can be seen as a product of two functions driven by a shared timeline.

Let's define the timeline in proportion to the grain interval, so that 0 corresponds to grain start and 1 corresponds to the end. We can compute the scan range and volume envelope based on that range.

One possible volume envelope is the parabolic shape:

; Simple parabolic envelope for level that is
; nonzero between 0 and 1, peaking with unity at 0.5.
Parabolic = t => Max(0 #4 * (t - t * t))

Algorithm:Map(Parabolic 0 0.25 0.5 0.75 1)0 0.75 1 0.75 0

Let's produce some sound grains by combining all the tools from above:

; Derive timeline from a phasor; let's use a simple
; periodic ramp from 0 to 1 at 1Hz.
grain-rate = 5

time = Gen:Phasor(grain-rate)

grain-pos = 5.8 + time * (1 / grain-rate)

snd = waveform-t(grain-pos) * Parabolic(time)

Sometimes we want the grain envelope to be shorter than the grain interval. That creates a distinct texture by leaving gaps in between the grains. We can do that by accelerating envelope time, simply by multiplying the argument fed into Parabolic.

; We can change envelope duration like so:

dur = 0.01 + Gen:Phasor(0.25)
snd = waveform-t(grain-pos) * Parabolic(time / dur)

; Or modulate the grain rate

grain-rate' = 1 + 30 * Gen:Phasor(0.2)
time' = Gen:Phasor(grain-rate')
grain-pos' = 5.8 + time' * (1 / grain-rate')
snd = waveform-t(grain-pos') * Parabolic(time' * 2)

Note that in this example, modulating the grain rate also generates intra-grain pitch modulation. That is usually not what we want. Instead, we should think about a way to have per-grain parameters and thus, grain identity.

Per-Grain Parameters

The first building block for a general granulator is a routine for cleanly retrieving audio from a specific source region. We want to position the read head whenever there is a new grain, and read with potentially varying rate (for grain transposition).

We can solve this with a grain stream timeline. Let us utilize a linear ramp signal with a rate equal to our grain rate. Then we can denote the integer part of the timeline as the sequential identifier of the grain, and the fractional part as the intra-grain phase proprotional to grain duration.

Grain-Read-Point is a function we use to keep track of grain position.

Whenever the sequential identifier seq changes, the read point is reset to the provided start parameter. In addition, grains can be resampled by providing a rate coefficient.

Import Control

Grain-Read-Point(seq phase start rate) {

	; Compute audio sample interval for read speed, and
	; multiply by the rate parameter
	inc = Interval-of(Audio:Signal(0)) * rate

	; advance read head and update read head
	; location at audio rate
	next-pos = Audio:Signal(pos + inc)

	; identify new grain by detecting upward
	; edges in the seq signal, which represents
	; the grain identifier
	new-grain? = Control:Edge+(seq)

	; for new grains, reset position to the start
	; parameter, otherwise allow read head to advance
	; normally.
	pos = Algorithm:Choose(
			new-grain?
				start
				z-1(0 next-pos))

	; finally, output the read head location
	pos
}

Application: Time Stretching

Granulators can be used to build time stretchers, that is, processors that change the speed of the audio without changing the pitch. The idea is that audio grains taken from near a point in time retain the timbre at that point, but are too short to encode temporal behavior. We can then rebuild the timeline by moving the grain source location forward (or backward) at any speed we may want.

In this example we generate a granulator timeline with Gen:Ramp, which produces a linearly increasing (forever) signal. Let the integer part of that ramp be the sequential grain identifier, and let the fractional part represent intra-grain phase. That way, the rate provided to Gen:Ramp becomes our grain rate.

We can make use of the fact that the grains are sequentially numbered. To play the audio material with the original speed, we can place each grain at seq / rate. Multiply that with stretch ratio, and we have a rudimentary time stretcher.

Stretch-Grain(source-fn timeline rate ratio) {

	; grain seq id: integer part of timeline
	seq = Floor(timeline)

	; grain phase: fraction of timeline
	phase = Min(1 timeline - seq)

	; generate grain read position
	pos = Grain-Read-Point(
		seq phase
		seq * (ratio / rate)
		#1)

	; sample source function at that position,
	; and apply envelope at current phase
	source-fn(pos) * Parabolic(phase)
}

; 8 grains per second
timeline = Gen:Ramp(0 8)
snd = Stretch-Grain(waveform-t timeline 8 0.5)

Polyphony

For a better quality time stretch, we should use multiple overlapping grain streams.

Stretch(source-fn rate ratio voices) {
	Use Algorithm

	; spread grain stream offsets equidistantly between
	; zero and 1.
	offsets = Expand(voices (+ (1 / voices)) 0)

	; generate timeline for all grains
	t = Gen:Ramp(0 rate)

	; for each voice, generate grains.
	; offsets are produced by subtracting the offset
	; from the timeline value.
	v = Map(o => Stretch-Grain(source-fn t - o rate ratio) offsets)

	; sum voices and compensate volume by dividing with
	; the square root of polyphony count
	Reduce((+) v) / Math:Sqrt(voices)
}

; let's modulate the stretch ratio with a ramp
ramp = 0.1 + Gen:Phasor(0.08)
r3 = ramp * ramp * ramp

voices = #4

; 16 grains per second, 4 overlapping streams
snd = Stretch(waveform-t 16 r3 * 4 voices)

Advanced Granulation

Often we will want more per-grain parameters. The grain position provides a template: detect a change in grain seq, and interject a new value. Let's explore per-grain transposition!

The challenge is that we want to make it easy to modulate grain rates and envelopes and hide the complexity of fixing some of those parameters for the duration of any one grain. Yet, we want to allow the possibility of dynamic modulation within grains.

The solution is function abstraction: provide rate and amplitude envelopes as closures. Because closures are normal values in Kronos, we can use Control:Sample-and-Hold to fix their value for the duration of the grain. Thus, any values the closure captures are per-grain, but dynamic behavior within the closure is still possible.

Grain(timeline source-fn start rate-env amp-env) {

	; compute sequential grain id and phase as before
	seq = Control:Signal(Coerce(Int32 timeline))
	phase = Min(1 timeline - seq)

	; detect new grain
	new-grain? = Control:Edge+-(seq)

	; this is our grain closure that captures rate-env
	; and amp-env. Please note that it's important *not*
	; to capture seq and phase, or those too would be
	; immutable during a grain.

	grain-proto(seq phase) {
		rate = rate-env(phase)
		pos = Grain-Read-Point(seq phase start rate)
		source-fn(pos) * amp-env(phase)
	}

	; if we have a new grain starting, admit a new prototype
	; into the grain synthesizer. this prototype has captured
	; rate-env and amp-env as they were upon grain start.
	grain = Control:Sample-and-Hold(grain-proto new-grain?)

	; synthesize!
	grain(seq phase)
}

; test bench
Test(rate) {
	; simulate randomized read position with a complex
	; long-preiod modulator (product of non-aligned sines)
	pos = 6 + 6 * Gen:Sin(0.02) * Gen:Sin(0.07)

	; modulate grain rates by generating an unsteady timeline
	timeline = Gen:Ramp(0 rate * (1 + 0.5 * Gen:Tri(rate * 0.1)))

	; this will be a per-grain parameter. the modulator runs here
	; and the captured value will be held by the grain prototype
	transp = 1 + 0.5 * Gen:Sin(9) * Gen:Sin(11)

	; rate-env is a trivial closure over transp, and
	; Parabolic is the amplitude envelope.
	Grain(timeline waveform-t pos () => transp Parabolic)
}

snd = Algorithm:Reduce(
		(+)
		Algorithm:Map(
			Test
			11 13 17 21 29))

Phew, that got abstract quickly.

Explore Further

In the example above, we used complex vibration to give an impression of randomness. True granulators usually have more advanced pseudorandom facilities. A linear congruential generator could drive any parameter variance you might like. However, I felt this post was getting dense enough as is. Happy hacking!