The Cybernetic Forest Synthesizer
How to Write Music With An Eagle and Three Satellites
Birds don’t care about our symphonies.
The Cybernetic Forest Synthesizer is the result of an investigation into communication between animals and machines. It’s a program, written in Python, that translates interactions between birds and GPS sensors into music. Along the way, it inspired a lot of thinking about the limits of encoding the natural world as digital information. It’s a prototype, and my final project for term 1 of my Applied Cybernetics Masters at the 3A Institute at the Australian National University. You can open the Colab Notebook for a deeper exploration of the issues and the process, or read below the button to see a summary and hear some results.
This is a post that ties together a lot of threads. The gist is this: everything about data, from how we collect to how we work with it, reduces the complexity of the world. We do this because computers need it that way. In the process, we make decisions based on simplified models of the world. I’ll give you some examples, starting with bird song. I’ll look at one man’s attempt to break away from simplifications in the early days of computing. And then we’ll look at how artists have imagined new combinations of the natural world and the digital world.
Then, I’ll tell you how to make music from an eagle and three satellites.
A page of music written from birdsong, by Olivier Messiaen.
Birds Are Avant-Garde
When you hear music, you’re actually just hearing vibrations in your ear drum. Certain pitches come from certain frequencies — literally, how frequently the sound waves rise and fall over one second. A mosquito’s wings are high pitched because those sound waves are very close together (“high frequency”) while a whale song is deep because those gaps are very wide (“low frequency”).
Western musical notation, the stuff of piano lessons and sheet music, gave names to those frequencies: A, B, C, D, E, F, G, with sharps and flats in between. Those notes get grouped together based on what rings pleasantly to Western ears. Notably, other systems of music select different frequencies; as with any system of representation, this is all a matter of taste.
These vibrations are assigned a symbol (a “note”) and then they are arranged on sheets of paper, following certain rules about how they should be arranged and performed.
Birds do not care about it.
That didn’t stop 20th-century French composer Olivier Messiaen from making “ornithography” the core of his work. He would go into forests, listen for birds, and write down their songs in a series of sharps and flats on a treble clef. Birds didn’t design Western musical notation, and Western musical notation wasn’t designed for what Messiaen was trying to do. Nonetheless, he did his best to try to make it work through a series of compromises.
Trevor Hold made a list of Messiaen’s struggles (1971): birds sing at pitches higher than the highest keys of a piano can reach. They sing faster than humans can play, and in some cases, faster than what humans can hear. Birds do not keep time, because (as Hold points out) there’s no reason for them to care. Birds do not keep in scale, and no acoustic instruments can vary in timbre as much as a bird’s throat.
Transforming a bird into a piano can be considered a coding exercise. Messiaen used the “code” of musical notation to “program” a musical score for a piano. The birdsong is a kind of data, and the score is a set of instructions, much like a computer program.
Messiaen was an excellent data scientist. His musical scores were all about context. He recorded the time of day and season that the bird was heard; he even included instructions for performers to go on morning walks to hear when birds would be most active. But even with all this effort, the data was a compromise.
Messiaen knew that he was wrestling with challenges of abstracting “the world” into “data,” that is, “the bird” into the “note.” This is impossible, but he worked vigorously to protect the bird from getting lost in the code. Messiaen knew his work was approximate, but this did not stop journalists from describing the music as “accurate.” Reviews of his work sometimes treat these adaptations as if they were being performed by birds themselves. They marveled at the authenticity of the work, “done without the use of tape recorders!” What they meant, though, is that the music was accurate enough, given the circumstances of how they were executed. This is how we ought to think about contemporary computing.
Anything we put into a computer will be digitized. This means converting observations, such as the music sung by a bird, into representations of what that bird sounds like. In digital information, today, this means numbers. Even if we are recording music, we have to convert sound to bits and bytes and re-assemble those bits and bytes into a different, approximate sound.
Organizing pieces of the world into pleasing slices is always going to be lossy and simplistic. To imagine that a map is the territory, or that a symphony can reproduce a forest, is a nice way of imagining a sense of order to the world. It allows us to invent uncomplicated versions of the prettier pieces of our experience, and to reduce away the odd, frustrating, and traumatizing stuff of natural volatility. But it’s a fantasy, borne from a sense that the world is, or could be, stable and manageable. It is not. It is dynamic, complex, and fragile. But that reality is harder to represent in ways that computers can render into something useful. At best, we are forced to contend with models and representations.
But like music journalists at a Messiaen performance, we all too often mistake “good enough for the circumstances” for accuracy. That is where the trouble starts.
It’s easy to think of digitization as the only way to get things done. Corporations and governments are in the midst of a data-collecting frenzy, reducing real-life complexity into simplified models. These companies are churning out research to improve digital machines that can analyze and interpret results from this vast trove of data. But the more complex our data gets, and the more complex our algorithms need to be to sort them, the further we go into building new layers of complexity on top of the real world. We create simulations of problems, then solve the simulations, and bring the simulated solution out into the real world.
The problem is, if we are to work with machines, this process of simplification seems inevitable. But what if there was a different way to imagine computing? Something that didn’t reduce the world to representations, but used the world to generate information and actions?
Managed By Leeches
Stafford Beer saw his answer to digitization at the bottom of a pond in 1959. Beer was unnerved by the rise of machines as management tools for complex systems. Machines were hyped up as promising new tools to reveal the world and strip away the mystery, but Beer knew that some mysteries were too big to be resolved: some events, behavior, and systems could never be predicted. The belief that human behavior, ecosystems or financial markets could be understood and stabilized would prove to be one of the most dangerous ideas of the following century. Pickering writes: “Such systems can never be dominated by knowledge, and instead we have to learn somehow to cope with them. Cybernetics was thus a science of dealing with the unknown—an extremely odd sort of science.” (2004, p.30).
Beer was interested in systems that would respond in real time and counteract the unknown: emergent chaos and catastrophes that rise from complexity. Cybernetics is, at its core, a very detailed science of pendulums. If a system swings too far one way, we would want to design things that help swing the pendulum back.
Beer was working at a steel mill and had been tasked with automating the entire factory. (Spoiler alert: he failed. But Messiaen failed to reproduce bird song by converting it to notes, and there is still something to learn from the process). Beer had invoices and shipments that had to be sorted out and mill work to perform. One way to go would be computers, but it was 1959 and as he wrote, “you would have to build the damn thing.” So, he began by training mice to perform certain tasks, but gave up. And then his idea was to turn to a pond.
As he describes it:
“The contents of the tank were randomly sampled from ponds in Derbyshire and Surrey. Currently there are a few of the usual creatures visible to the naked eye (Hydra, Cyclops, Daphnia, and a leech); microscopically there is the expected multitude of micro-organisms. The state of this research at the moment is that I tinker with this tank from time to time in the middle of the night.” (Beer cited in Pickering, 2010).
Beer hoped that the pond would serve as the thinking hub for an entirely self-sustaining factory, in particular, regulating the supply that would be produced each day according to the demand that was coming from outside. By modern standards, the pond would be in charge of running the Excel spreadsheets and calculating work orders. The inhabitants of the pond would ingest small iron fillings and magnets would respond to their location, triggering lights and switches. There would be not binary reduction or digitization of this information. Instead, the pond would be a kind of model for stability, a biological system that represented itself. A factory would be arranged to follow the pond’s rules. It was an inversion of the “reduce to data, process and expand” model of computerization. This was “Find a process (the pond), build a system that reflects the process (the factory), and then follow the rules of the process (automation of the factory by a pond).”
The problem, in the end, was how to make the pond care about the physical demands of the factory. The critters did not organize themselves in ways that applied to the factory’s demands, and so the factory could not be organized around the pond. Just as Messiaen couldn’t directly reproduce birdsong as data, the Beer could not translate a factory into a pond. A translation could have been imposed by a computer, but if you were to plug an ecosystem into IBM note cards, you might as well just build a program for factory management. And anyway, you would lose too much of the pond’s complexity for any of the effort to be worthwhile (see Kwa, 1993). But it wasn’t an idea without merit. Decades later, we’d have “unconventional computing” figure out how to make logic gates from crabs.
Beer gave up, but that inversion of the computational relationship is a fascinating case history. The thread between these stories is reminiscent of the subtitle of that original book on Cybernetics: communication between the animal and the machine. Beer built a computer but couldn’t sort out how to get information in or our of the pond. For Messiaen, communication meant endless compromises between the languages of birds and the language of musical notation, losing complexity. These two represent the tightrope of “computerism” that we face today: how do you balance the need to preserve the world with the need to simplify it? Beer is right: we have systems that are too complex to reproduce digitally, and the temptation to do so is dangerous. Messiaen was right, too: to be useful, compromises have to be made. But how?
Mutually Programming Harmony
The tension between digital representations of naturally occurring complex systems interested me. I wanted to spend time thinking through relationships between non-human systems of nature and technology. Extinction crises, climate change, and pandemics are all natural disasters, predictable but not to any useful degree. We know there will be fires, we don’t know where they will start. It seems crucial to imagine systems that go beyond privileging a single user:
“Technology designers and design researchers are implicated in this wave of change and uncertainty because we have claimed a stake in the production of futures. As makers, we can choose to have a role in producing alternative narratives for present generations of humans and those who depend on them, such as other species and unborn children.” (Light et al 2017, p.1)
I wanted to imagine a “different kind of narrative” for how we think about encoding the world of observable nature into data by thinking about that point of translation, where animals and machines come together and interact. Every system that interacts has these touch points where information is exchanged and lost. I was curious to find them and look at how they work. And since much of today’s computer science is built on “rational” “logical” foundations, in which relationships between data and output are assumed and all too often unquestioned, I wanted to look beyond engineering for different ways to imagine and consider these relationships.
The Richard Brautigan poem, "All Watched Over By Machines of Loving Grace," imagines a "cybernetic forest" where "mammals and computers live together in mutually programming harmony" (1967). As part of the 3Ai reading list (which includes that poem) I started connecting this idea of mutuality with the thinking of Aboriginal and Torres Strait Islander people presented through author Tyson Yunkaporta’s work. In Sand Talk, Yunkaporta writes about "kinship-mind," in which "every element must be considered in relation to the other elements and the context. Areas of knowledge are integrated, not separated" (2019, 176). The challenge of computational logic is that it relies on the isolation and parsing of relationships into discrete data “points.” We reduce relationships out of data by necessity, and then ask computers to re-invent them. I wanted to prioritize looking at the connections of pieces within systems instead.
Kinship may be a deeper way of thinking about “mutually programming harmony.” I’ve spent some years living in, researching, and writing about Japan, and have been moved and fascinated by Shinto. Shinto is a loose collection of local traditions, most of which focus on kami (roughly translated as “spirits” within things). Kami are a part of trees, objects, technologies, and people. They are distinct, but if everything goes right, they work together to create a balance in the system, observable through the experience of awe. Shinto focuses on sustaining harmonious interactions between kami, and creates a different way of thinking about relationship and communication between technology and nature. Humans don’t have dominion over these systems, but are part of the mix.
This is beautifully and surrealistically illustrated in a short vignette from the 2005 film Funky Forest, in which the spirits of the sounds of nature, machines, and humankind form a band, jamming with human musicians in the trees. The young women in hats are the spirits of various kinds of sounds. Notti, a human (confusingly in a bear costume) encounters them through playing the violin. It is a different, more literal take on the phrase “mutually programming harmony.”
I also considered the work of artist Michael Prime, who generates music from the electrical current pulsing through growing mushrooms. Prime is interesting because, as far as I can tell, he does not “translate” this activity into computational abstraction in this work. It isn’t digital: the sensors aren’t taking data, running it through a machine, and producing sounds based on the data. Instead, the sensors are carrying electricity from the fungus direct to an oscillator, which merely amplifies the sound. This is an example of representation without a translation to the digital symbolic — the mushrooms never become “data.” And yet, it is a “program” — it produces sound based on inputs. Or rather, it is a complete computer, executing a single task of making music. This is closest in practice to Beer’s pond. Nothing is being reduced, rather, the system simply responds.
The Synthesizer
Music is a comfortable space to test these ideas, because they can be designed in ways that do not interfere with natural ecosystems and can be modeled as relationships, “an open-ended and performative interplay between agents that are not capable of dominating each other” (Pickering 2010, p86). A synthesizer is an electronic instrument that relies on data (an electrical signal) to create sound. The term "synthesizer" is appropriate to our uses, as it originates with the term synthetic, as in "devised, arranged, or fabricated for special situations to imitate or replace usual realities," but equally applies to synthesis, "the combining of often diverse conceptions into a coherent whole." The root of both words is the Greek syn, "together."
The Cybernetic Forest Synthesizer is the result of all of this thinking. It synthesizes scores for music, first from bird migration data, but more crucially, from GPS drift. The design journey, which is painstakingly described in the Colab Notebook, involved rethinking the representational relationship of sound to notation, and to focus on the ways information changes meaning when it becomes described as “data”. That’s outlined in the notebook. (Spoiler #2: it failed, because ultimately my desire to produce music that made sense to my own ear meant reinventing music theory. But at least I got to take it apart and determine what I needed along the way.)
I focused on the contact points where interacting systems touch, and the struggle to translate activity into a language that could be understood between systems. This is a lot like human communication: something is lost from the original experience by placing it into words; but something is also gained by being in communication. I am focusing on literal connection points between human, technological, and natural systems.
The code, my first project ever written in Python, ended up mostly assembled and repurposed from the work of Michael Corey on MidiTime, which was intended to create sonification pieces from Earthquakes. Admittedly, much of the “coding” of this project on my end was high-level algorithms that Corey had already solved. The code generates a midi file with a score rendered from the data, and can be used for a multitude of musical scores from various data sets. I have selected two below. A very detailed description of the project, including the process of attempting an original algorithm for this project and the final, working code, can be found in the Colab Notebook.
Music for Birds and Sensors
The first piece is the result of running data from a two-year migration tracking study for a pair of Lark Sparrows (Ross 2013). The synthesizer takes the distance traveled, scales it, and corresponds it to a musical notation system (currently set at A# Phrygian mode). The peaks and valleys you can see in the “waveform” of the audio file above correspond to the rise and fall of intensity in the music, which reflects the distance the birds have traveled. The program creates a score as a MIDI file: a full piece for piano. I then ran that score untouched through a digital synthesizer (Alchemy, using the FL Studio DAW). Aside from the instrumentation change, the only other edit was to remove extended moments of silence between migration seasons.
This piece is a proof of concept, and represents a kind of audio form for data visualization (“data sonification”). But ultimately it still represented data: the sound you hear is a reduction: it’s built on data from a spreadsheet. It’s based on the GPS tag’s coordinates at any given moment and responds to distances between sets of numbers. It does not generate music from the interactions of sparrows and sensors per se, which was my actual goal.
If I wanted to zoom out and focus more on the negotiation that takes place between a bird (nature), a sensor (technology), and musical notation (a human language), I could find an intersection of these in the “problem” of GPS drift.
GPS drift is the distance between where the sensor is and where the sensor reports itself to be. If you’ve ever used a phone to find yourself and found you were momentarily floating over the sea or hovered over a mountain as your train went underground, you’ve experienced GPS drift. The music below emerges from drift, and reflects the distance between the sensor and the thing it is sensing. It rises out of an "interaction" between the measurement language of the sensor and the concrete movements of a bird.
We can hear music generated by that relationship, rather than representing another translation of data (as in the above sonification piece).
Above: An image documenting the Cybernetic Forest Synthesizer's role within a larger systems network, in this case, tracking the relationship between a Golden Eagle and GPS drift, which was used in the second piece, shared below. We see the eagle's motivations for food, nests, and mates / avoidance of rivals in the top left, influencing it's movements, tracked by the GPS sensor. This sensor then sends signals to satellites, and we see common sources of interference -- fog, dense tree cover; or blocked entirely by mountains, or passing through clear blue sky.
It illustrates, through sound, a way of imagining "standpoint" from a non-human perspective: the kinship between a sensor and a bird. Centering the relationship in this way reveals the often ignored or deliberately hidden touch points that connect systems. These touch points are where information is reduced to “data.” It is where the bird becomes a ping and the tree becomes a pair of numerical lat-long coordinates, while the GPS and satellites work to find out where they are.
The eagle has its own motives within its own ecosystem (graphed to the right): it seeks safety, food, mates, and places to rest, for example. It operates within a system of forests, weather, predators and prey.
The GPS device, attached to its claw, connects the eagle to a broader system, one that literally hovers above them in outer space. That brings in systems of orbits, atmospheric conditions, atomic clocks, telecom services and physical obstacles that could reduce data reception. The bird moves the GPS tracker through blockages and clear spaces, creating a relationship between its own goals and the engineered goals of the GPS. The signal from the eagle has to make it up to three satellites, which send back the time from space. The elapsed between the signature from the satellites is then used to determine the location of the receiver (and then, the eagle).
When the satellites are wildly out of sync, perhaps because one signal is blocked but two are accessible, there is a brief phase error that causes uncertainty. That’s GPS drift. This music is created by a network of machines trying to reconcile mysterious discrepancies while trying to find an eagle from outer space. It is about the eagle’s passive resistance to reduction (it just exists), and about the interaction between the two systems on each side of that translation.
When that data is finally collected and processed, we introduce another system: excel spreadsheets and python programming. While Beer struggled to figure out how to get data out of a system without abstracting it —treating their inherent complexity as a feature worth preserving, rather than a bug to solve for — I, and most of the modern world, have opted to simplify things through machines, and for machines, however tragic that surrender may be.
Here, the only way to perfectly represent a system was to focus on a system that could actually be represented as representational. This music is speaking the “language of the sensor,” rather than the language of the bird. Just as Messiaen was not “reproducing the sound of birds,” these pieces are not “based on the location of the birds,” but based on how the location of the bird influenced the sensor’s idea of where it was.
The combination of these systems generate a “mutually programming harmony” heard in the pieces. I think about a Messiaen piece where, instead of struggling to notate the birds as accurately as possible, he imagines ways to highlight the difference between what he heard and what he wrote: inviting us to imagine the remainders left over from reduction, to see what we lose. In the words of Wallace Stevens, to see “nothing that is not there and the nothing that is.”
UPDATE: A mastered version of this piece can be found on my LP released as The Organizing Committee, “The Central Memory.” More info and links to streams are here.
Works Cited
Beer, Stafford (1959). Cybernetics and Management. English Universities Press, London.
Brautigan, Richard (1967). All watched over by machines of loving grace. First published by the Communication Company.
D'Ignazio, Catherine & Klein, Lauren F. (2020). Data Feminism. MIT Press, Cambridge, MA.
Funky Forest (Naisu no Mori), dirs. Katsuhito, Ishii; Ishimine, Hajime, and Miki, Shunichiro. AOI Promotions [2005] video clip retrieved from YouTube via https://youtu.be/ttjHcG6AOQg
Hold, Trevor. “Messiaen's Birds.” Music & Letters, vol. 52, no. 2, 1971, pp. 113–122. JSTOR, www.jstor.org/stable/732949. Accessed 25 Apr. 2020.
Kwa, Chunglin (1993). “Modeling the Grasslands.” Historical Studies in the Physical and Biological Sciences (1993) 24 (1): 125–155. https://doi.org/10.2307/27757714
Light, Ann; Powell, Alison & Shklovski, Irina (2017). Design for Existential Crisis in the Anthropocene Age. In Proceedings of the 8th International Conference on Communities and Technologies (C&T '17), 9 pages. DOI: 10.1145/3083671.3083688
Pickering, Andrew (2010). The Cybernetic Brain. University of Chicago Press.
Ross JD, Bridge ES, Rozmarynowycz MJ, Bingman VP (2013). Geolocator-tracked variation in migratory behavior within a highly site-faithful population of Lark Sparrows (Chondestes grammacus). Animal Migration 2(1): 29–33. doi:10.2478/ami-2014-0003 These data have been published by the Movebank Data Repository with DOI 10.5441/001/1.5jd56s8h.
Smith JP (2010). Raptor migration dynamics in New Mexico. In Cartron J-L E (ed) Raptors of New Mexico. University of New Mexico Press, Albuquerque, NM. p 29-67. ISBN: 978-0826341457 unmpress.com/books/raptors-new-…xico/9780826341457
Smith JP (2019). Data from: Study "HawkWatch International Golden Eagles". Movebank Data Repository. doi:10.5441/001/1.95r77m9k
Stevens, Wallace (1921). “The Snow Man.” The Poetry Foundation via https://www.poetryfoundation.org/poems/45235/the-snow-man-56d224a6d4e90 [26 June 2020].
Yunkaporta, Tyson (2019). Sand talk: How Indigenous thinking can save the world. Text Publishing.