The SYstems Of Systems
The “Systems” in cyberphysical systems
The Systems element of Cyber-Physical Systems is focused on relationships and feedback loops that emerge when distinct systems are put into contact with one another, as in sensors, actuators, communities, individuals, ecosystems, and more.
Scaled systems require a complex network of sensors, processors, and actuators to successfully operate. Today we have engineers well versed in the production of individualized pieces, but a shortage of engineers responsible for understanding the connective fabric of these systems. This is not physical engineering — not wires and cords, or the flow of data through a network. It is the interaction and navigation between these points of contact. Each new step requires a translation between the device sending signal and the device receiving it. These translation-interactions are unique to each arrangement of pieces, and go beyond technological applications. It includes direct influences, such as human decisions on omissions in datasets to make them comprehensible between programs, but also within ecosystems — as when crows disrupt cell towers or powerlines to transportation networks.
My coursework provided me with an opportunity to explore the relationships within “systems of systems,” with a personal focus on bridging technological and natural ecosystems. When we think about cyber-physical systems, how might we move beyond the “internet of things” — things being vague enough to include any object, but narrow enough to exclude nature and communities. Below, I present two case studies which explored these questions by looking at the connection points between technology and birds.
“We are beginning to play with ideas of ecology, and although we immediately trivialize these ideas into commerce or politics, there is at least an impulse still in the human breast to unify and thereby sanctify the total natural world, of which we are.”
Case Study:
The Network Map
A network map is often considered a technical document, detailing the flow of energy or information between connection points within a technological system. But every network map is also a story of relationships. Those connections and the protocols of communication between them illustrate loss, transformation, evolution, and feedback within systems.
For the Owl Smart Home, I conceived of a cyber-physical system that relied upon a feedback loop between a sensor and a living creature: a bird house for owls. The system would protect owl eggs from predatory species, such as crows. After researching our “user,” we discovered that owls do not migrate; therefore, we did not need to worry about a wireless signal interfering with their biological navigation systems. I was able to observe owls through a live stream from the Cornell University Ornithology Lab, and came to see patterns in how the birds hunted, slept, and defended their homes.
A clip from the owl camera from the Cornell University Ornithology Lab.
This lead me to design a house system that would use an image recognition camera to monitor the ledge of the birdhouse, and close the door only if a crow arrived, so as to be minimally disruptive to the owl’s many movements back to the eggs or chicks during its hunting rounds. The camera would send digital photographic data of any other bird’s arrival to the wireless card in the bird house, which would transmit the data to a wireless router and pass it to a local PC with a database. That database would check the image, and send a signal to close the door if it matched the image of a predator.
Here is the preliminary systems map of that network, created in a freeware tool, Network Notepad, which I learned for this task.
Making the map yielded a few discoveries about its components. For example, I traced a path for the digital image all the way to Samsung’s Korean data center, a relationship hardwired into the camera. This was an intriguing bit of data forensics that showed me the importance of analyzing a network map (and all technical documents). Because I had gone through all of the documentation for each component, I was able to see that facial recognition data is sent to Samsung by default! This added depth to my understanding of how face recognition and surveillance tools keep and preserve data beyond the local storage of the camera’s owner.
The map rendered a conceptual understanding of a system into something more concrete: not only the set of components, but the relationships between them.
Maps should tell a clear story of how information within a network is exchanged and what protocols need to be translated. As a way of telling this story in a more accessible way, I reduced my map for clarity, showing the flow of communication signal from the birdhouse to the local server, and then tracing the flow of data. I had to carefully weigh what information could be represented differently, what elements were most important for that audience, and how best to convey it in a way that was accessible, useful, and possibly even “delightful.” But the key thing was to make sure the bird was present — in this case, the crow — as the crow activates the system.
Download the Owl Smart Home Slide Deck
The second map is limited to a specific audience, and so exclusions were made: a truly comprehensive map of this network would extend into a systems map. This would allow us to explore more about the bird’s ecosystem, and the feedback that leads the crow to egg thieving behaviors. The next case study looked more deeply at an interaction point between eco- and techno- systems and the relationship between birds, data, and sensors.
Case Study:
Interactions Between Ecosystems
“I like to think (and
the sooner the better!)
of a cybernetic meadow
where mammals and computers
live together in mutually
programming harmony
like pure water
touching clear sky.”
Mapping relationships can take many forms. As a final project, I designed a cyber-physical system that navigated between natural and technological systems. This would inspire deeper thinking about what we build and how non-human systems might interact.
The project, Cybernetic Forest Synthesizer, creates music from data gathered by sensors recording migration patterns of a golden eagle. This connects the eagle’s migration (representing a natural ecosystem) and GPS drift (representing a technological system) and identified ways to harmonize the two.
Can a piece of music be described as a systems map? I would suggest that it can, as it depicts points of interaction between two larger systems. Systems mapping is a process, rather than a result. In that view, a systems map is any product of a systems mapping exercise.
The relationship between the location of the eagle and the sensor was recorded in a data set collected for a migration study (Smith, 2019). By focusing on this unit of measurement, we can think of this as a “feedback loop” between the bird, which is observed, and the GPS data, which is responding to the bird’s location. This error is adjusted as the bird moves in and out of certain areas and the sensors become aligned to its “true” location, or misaligned due to natural barriers or breakdowns in signal transmissions.
An illustrated systems map to the Cybernetic Synthesizer project, visualizing relationships between the golden eagle and its movements (inspired by food, nests, and the presence of potential mates or rivals); the GPS sensor and various conditions which may challenge the signal’s reception; and finally, the processing music from gathered data. To the right, we see three conditions which contribute to GPS drift: dense forest cover and/or fog weakens signal; rocky mountains block it completely, and clear blue skies allow it to move through. We also see the flow from GPS to spreadsheet to Python to music.
By highlighting GPS drift, we are able to imagine a feedback loop between the bird and the machine: the bird creates the feedback loop through its movements, and the machine interprets that data. Its errors create another level of feedback loop, as the machine drifts in and out of alignment with the bird, correcting itself in fits and starts which come through as spikes in the data. These spikes have been translated into sound. Again, these are just two “touch points” between broader complex systems: for the eagle, the patterns are determined by food and nesting sites, temperature, and other environmental factors. The GPS sensor is working through its own system of tree coverage, weather impacts, and the limitations of its own design — be it materials, ranges, or more.
I was struck by the vastness of this system when broken into components: an eagle in a tree, in communication with a sensor, in communication with a satellite floating in space, sending signals back to Earth. We overlook these relationships when they become pieces of our everyday lives. I hoped to create something that could serve as a meditation on the vast scales of these systems, to invite new ways of conceptualizing them.
The exercise forced me to focus on the connection points, and to think deeply about the relationship between data and what that data represents. Additionally, it forced me to consider “points of translation” between even more systems: for example, how can “location” be mapped to a system of musical notation in the most representative way? How might we preserve that relationship in converting data into an algorithmic process for a computer to execute? What do we gain and lose in processing, and then communicating, that data as sound?
In this case, I relied on an algorithm that would take ranges of GPS drift measurements — how “off” the machine was, in meters, to where the bird was observed — and scale these measurements into 7 ranges. For each timestamp, the drift range was mapped to the seven notes of a musical scale, and added to a musical notation file, which could be played in any digital audio workstation. The eagle could contribute to this drift through its choices of movement. If it went to into heavy canopy, it would inspire greater drift.
This auditory experience is a the result of a systems map, and hopefully provides a compelling meditation on relationships between those concrete pieces. That is to say, the piece is not about interactions between eagles and sensors, but is the result of interactions between eagles and sensors. It is an exercise that helps us imagine undirected exchanges between components of parallel systems, even when those components are unaware of one another. It gets us closer to Richard Brautigan’s “Mutually Programming Harmony” between computers and animals, and a new way of thinking about “kinship” and “symbiosis” between our technological and ecological worlds.
The process of creating this piece is documented in greater detail in the form of the Colab Notebook submitted as the final form of the project, with its own documentation of my process and lessons.
Key Learnings
“Big whirls have little whirls,
That feed on their velocity;
And little whirls have lesser whirls,
And so on to viscosity.”
In conceiving of both of these projects, I wanted to explore ideas of “human absent design,” a reversal of the design thinking approach that focuses on human consumers. It seems critical to look beyond a single human as the sole benefactor of products or environments.
The development of these projects took place alongside multiple catastrophic events, from long-ignored crises of climate change to coral bleaching and the animal-to-human transmission of a novel virus giving rise to the covid-19 pandemic. As I write, mass civil rights demonstrations have started in the United States at an unprecedented scale, giving rise to a long-simmering crisis of community. Designing for a single person is incomprehensible in these times; it speaks to the urgency of centering relationships, communities, and ecosystems.
The new branch of engineering must be focused on interactions between organic, social, and technological systems. Here’s how it might be done.
First, understanding input: feedback loops are extraordinarily powerful, and great care must be placed into thinking about what elements of a system have access to those loops and which are excluded. What assumptions do we have to consider, research, and surrender when deciding who to exclude or include — for example, we assume crows can find an alternate food source than owl eggs, so we are comfortable “closing the door” to the house to those birds as predators. But this is still meddling in the dynamics of a system — ultimately why the system has to remain theoretical. Who controls the feedback? My projects considered an open feedback loop and a closed one: tracking the golden eagle’s free movements, compared to restricting the natural instincts of crows. In implementing systems, I will continue to make these decisions carefully. We have to be open to new forms and varieties of feedback.
Secondly, we should uncover hidden relationships. Identifying and responding to touch points is an essential skill that requires deep investigation, and tools such as network maps, as seen above, can help us imagine systems as a series of touch points. Not all of them are obvious, and not all of them will be useful. But it would be better to decide what to omit, rather than ignore what we don’t know. In the next section, I will describe how these ideas will shape my approach to systems as I move forward.
What’s next:
Three Dimensional Systems Maps
My experiences have shown me that interactions between systems are an inevitable, critical component of future systems design. How might we create a more dynamic, systems-based approach to analyzing the impacts of these tools?
3D Chess, CC0 (public domain) image via Wikimedia Commons.
I am reminded of 3D chess, a game played on three boards, where interactions on each layer affect the motions of all three layers of the board. If today’s network maps are a chess board showing movements within a frame; and systems maps typically contextualize the technological contours of a system, then the time has come to render the systems maps in more inclusive dimensions. I can imagine maps that take into consideration ecosystems and environments behind each interaction point. Where might sensors come into contact with vulnerable populations, endangered environments, or other factors? How could we keep track of ever-changing cycles of feedback? Moving forward, I ask myself how we might make these maps within a disciplined framework of practice. What forms they might take, and who else might we include in their design? Asking these questions is a key component of the skills I’ve learned, and will extend my learning as a practitioner of the NBE beyond the course.
Skills Mastered
Systems Thinking — a broader understanding of how different social, natural, and technological systems rely on and transmit data to one another.
Network mapping, in ways that tell the story of relationships and feedback loops between systems.
Basic programming in Python, in Terminal and Colab environments; structuring large programming projects, and adapting open source solutions.