A Musical Introduction to Generators

What a generator allows you to do is to take code that is writen in a sequential, looping style, and then treat that code’s output as a collection to be iterated over.

To illustrate what this means, and the generators package in general, I will use R to perform a suitable piece of music, specifically Steve Reich’s “Clapping Music.”

The piece starts with a loop counting out groups of 3, 2, 1, 2, 3, 2, 1, 2, 3… with each group separated by one rest. This adds up to a 12-note loop:

Musical score showing loop

In code, we could implement that idea like the following, printing a 1 for each clap and a 0 for each rest:

print_pattern <- function(counts = c(3, 2, 1, 2), repeats=4) {
  for (i in seq_len(repeats)) {
    for (c in counts) {
        for (j in 1:c)
          cat(1)
        cat(0)
    }
  }
  cat("\n")
}

Testing this, we should see groups of 1, 2, or 3 1s separated by a single 0:

print_pattern(repeats=4);

## 111011010110111011010110111011010110111011010110

“Clapping Music” is based on manipulating this 12-count loop. But it’s hard to manipulate the output of a program that only prints. The calls to cat produce output on the terminal, but they don’t produce data that we can easily manipulate with more programming – we need to make this pattern into data, rather than terminal output.

The async package allows us to enclose a data-generating process into an object.

To make a generator for this pattern, we just enclose the body of the function in a call to gen(), and change each cat() to yield(). We can also move the outer loop into the recycle argument of an [iteror].

library(async)
library(iterors)
counts <- c(3, 2, 1, 2)
pattern <- gen({
  repeat {
    for (n in counts) {
      for (j in 1:n)
        yield(1)
      yield(0)
    }
  }
})

The code inside gen(...) does not run, yet. The call to gen constructs an [iteror][]. Iteror is an implementation of iterators defined in the async package; its objects are cross compatible with the established iterators package but can have somewhat better performance. The main method for an iteror is nextOr(x, or) where the second argument specifies what to do to signal the end of iteration. So we can get out our pattern like this:

for (i in 1:24) {
  cat(nextOr(pattern, break))
}

## 111011010110111011010110

cat("\n")

When nextOr() is called on a generator, the generator runs its code only up to where yield is called. The generator returns this value, and saves its state, pausing until the next call to nextOr().

We’ll have to create a few copies of this kind of generator, so we can reuse it by making a function. You can wrap gen around a function expression; this will define a generator function (a function that constructs a generator.)

gen_pattern <- gen(function(counts = c(3, 2, 1, 2)) {
  repeat {
    for (n in counts) {
      for (j in 1:n)
        yield(1)
      yield(0)
    }
  }
})

Because gen(...) builds an [iteror][], you you can apply iteror methods to it. For instance you can collect just the first 24 items with i_limit():

show_head <- function(x, n=24) {
  x |> as.list(n=n) |> deparse() |> cat(sep="\n")
}
show_head(gen_pattern(), 24)

## list(1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 
##     1, 0, 1, 1, 0)

Making noise

We’re a good way into what I advertised as a musical endeavour and haven’t made any sounds yet. First let’s download some handclap samples. I located some on GitHub:

tmp <- tempdir()
baseurl <- "https://github.com/octoblu/drum-kit/raw/master/public/assets/samples"
samplepaths <- paste0(tmp, c("x" = "/clap4.wav","X" = "/clap5.wav"))
curl::curl_download(paste0(baseurl, "/clap%20(4).WAV"), samplepaths[1])
curl::curl_download(paste0(baseurl, "/clap%20(5).WAV"), samplepaths[2])

Although R is not really known for audio performance, there is an audio package playing sound samples, which we can use like this:

library(audio) # for load.wave, play
claps <- lapply(samplepaths, load.wave)
play(claps[[1]])
play(claps[[2]])

We want to play sounds at a consistent tempo, so here’s a routine that takes in a generator and a sample list, and plays at a given tempo. The profvis package has a pause function that’s more accurate than Sys.sleep().

library(profvis) # for pause
iplay <- function(g, samples, bpm) {
  interval <- 60 / bpm
  target <- Sys.time()
    repeat {
      x <- nextOr(g, break)
      target <- target + interval
      while({now <- Sys.time(); Sys.time() - now > 0.15})
        Sys.sleep(target - now - 0.15)
      if (is.numeric(x) && x >= 1 && x <= length(samples)) {
        cat(x)
        pause(target - Sys.time())
        play(samples[[x]])
      } else {
        cat(".")
      }
    }
}

So we should hear our pattern now:

gen_pattern() |> i_limit(36) |> iplay(claps, 360)

Some iteror (iterator) functions

The object constructed by gen has class iteror. Iterors are an iteration construct included in the companion package iterors. Iterors are cross compatible with iterators from the iterators package and its nextElem method, but using their preferred method nextOr can lead to better performance and more compact code.

We can work with generators using iteror methods. Here are two methods that will come in handy. One is an iteror equivalent of lapply, which I’ll call iapply. The other one is isink which just consumes and throws away all elements from an iteror.

iapply <- function(it, f, ...) { list(it, f, ...)
  iteror(function(or) {
    f(nextOr(it, return(or)), ...)
  })
}

isink <- function(it, then=invisible(NULL)) {
  repeat nextOr(it, break)
  then
}

Note that the second argument to nextOr is only lazily evaluated when the iterator terminates; this is what allows you to place control flow operators like break.

For example, we can print the contents of a generator by iapplying cat and throwing away the results:

g <- gen_pattern() |> i_limit(24) |> iapply(cat) |> isink(cat("\n"))

## 111011010110111011010110

Some iterator functions, using generators

Generators enjoy some syntax extensions over base R; one extension is that they can use a regular for loop with an iteror. So we can equivalently write the above iapply and isink using even more concise generator syntax.

gapply <- gen(function(it, f, ...) for (x in it) yield(f(x, ...)) )
gsink <- function(it, then=NULL) {run(for (i in it) NULL); then}

Here [run][] is like gen, except rather than constructing an iteror, run executes immediately, collecting all values passed to yield and returning a list. So gsink is a function that when called will consume all items from the iterator it before returning.

Phasing and combining

“Clapping Music” is a piece for two performers, who both play the same pattern, but after every 12 loops, one of the performers skips forward by one step. Over the course of the piece, the two parts move out and back into in phase with each other. We can write a generator function that does this “skip,” by consuming a value without yielding it:

drop_one_after <- gen(function(g, n, sep=character(0)) {
  repeat {
    for (i in 1:n) yield(nextOr(g, break))
    nextOr(g, break) #drop
    cat(sep) # print a seperator after every skip
  }
})

Here’s what 12 loops look like, where you after three (i.e. skipping every fourth):

iseq() |>
i_limit(12) |>
drop_one_after(3, "\n") |>
iapply(cat, "") |>
isink()

## 1 2 3 
## 5 6 7 
## 9 10 11

The performance directions for “Clapping Music” request that the two performers should make their claps sound similar, so that their lines blend into an overall pattern. We can interpret that as combining the two lines by adding two generators, resulting in 0, 1, or 2 claps at every step, playing the louder sample for a value of 2.

Then, all together:

clapping_music <- function(n=12, counts=c(3,2,1,2), sep=" ") {
  cell <- sum(counts+1) # how long?
  a <- gen_pattern(counts)
  b <- gen_pattern(counts) |> drop_one_after(n*cell, sep)
  # add them together and limit the output
  gen(for (i in 1:(n*(cell+1)*cell)) {
    x <- nextOr(a, break)
    y <- nextOr(b, break)
    yield(x+y)
  })
}

To narrate this: we are constructing two independent instances of our 12-note generator. One of these patterns is made to skip one beat every N bars. Then we create a third generator that adds together the two.

clapping_music(4, sep="\n") |> iapply(cat) |> isink(cat("\n"))

## 222022020220222022020220222022020220222022020220
## 221121111211221121111211221121111211221121111211
## 212112021121212112021121212112021121212112021121
## 122021120221122021120221122021120221122021120221
## 221112111220221112111220221112111220221112111220
## 212022021211212022021211212022021211212022021211
## 121121121121121121121121121121121121121121121121
## 212112120220212112120220212112120220212112120220
## 122022111211122022111211122022111211122022111211
## 221122021120221122021120221122021120221122021120
## 212121120211212121120211212121120211212121120211
## 122112111121122112111121122112111121122112111121
## 222022020220222022020220222022020220222022020220

A performance

Now we should be able to hear our performance:

iplay(clapping_music(n=4, sep="\n"), claps, 480)

It has to be said that R is not particularly built to be a multimedia environment, plus the audio package will be using the OS alert sound facility, which is typically not built for precise timing, so you may hear some glitches and hiccups. Nevertheless, I hope this has illustrated how generators allows control to be interleaved among different sequential processes.