Eryk Salvaggio: CECS 6001 Learning Portfolio
Submission Date: 12 June 2020

Imagining a new sort of world begins with imagining new kinds of systems. That means asking questions in a new kind of way.

As a masters student in Applied Cybernetics, I study how systems shape this “new sort of world.” It’s a world that emerges from relationships. There are technological pieces, such as sensors, their data, and the ways they connect. There are human elements, too. We want to grow healthy communities and thriving cultures. There are ecosystems, including weather, animals, plants and habitats. As we create more autonomous technologies, we introduce another unique actor to these relationships.

The World Economic Forum brands this new era “the fourth industrial revolution” (Schwab 2016). Genevieve Bell, leader of the 3A Institute, observes that technologies such as big data, AI, algorithms and automated machines that had once “seemed like individual technological interventions, are starting to become a system, working in concert" (2017). As these distinct pieces connect to one another, managing them requires a new layer of thinking about them. We still must understand the components individually. But we must also look at them as components of a “cyber-physical system” (CPS). Otherwise, cannot manage the complexity that emerges as systems create feedback loops with other systems, and so on.

A CPS is a system of systems, brimming with complex interactions. On their own, a series of sensors and actuators integrated through networks — which are, themselves, systems — is enough complexity to require a full set of engineers. But that CPS exists within the world. It interacts with social, natural, and political systems (Geisberger 2015, p.21). If we map a CPS, we must describe it down to the tiniest details of its every connection: where the sensor connects to the network, connects to the data center, connects to the satellite, connects to the home PC, and so on. The languages we use, from code and its persnickety syntax to network maps with excruciatingly detailed lists of IP addresses and protocols, are deliberately specific.

The social and natural systems, on the other hand, tend to thrive in unrestricted spaces. What can be broken down is done in an entirely different language of sociology or biology. Culture is messy, and our expression depends on it: we write poems where words slip between meanings; we create music where sonic fluctuations create harmonies that rise and fall and reconnect. We always manage to reconcile these fuzzy spectrums into something that stirs us. Our systems must preserve the things that make life enjoyable, and we cannot reduce complex worlds to make them more compatible.

The quality of our future depends on understanding the connections between social, technological, and ecological pieces. The complications of one system touches all the others. Grasping the complexity of these systems, and how they interact, can elude and confuse us. We turn difficult relationships into short hands and frameworks and try not to omit crucial details. But we often can’t see what we are losing.

Margaret Mead wrote, “the attempt to introduce automated systems into a society which does not understand them is dangerous.” (1968, p.6). When white settlers began giving steel axes to the Aboriginal Yir Yoront group of Australia, it disrupted long-standing social relationships (Sharp 1952). Today, we rely on social media algorithms and interfaces to navigate friendships. As one student explained it — “sometimes you don’t have time for your friends except if they’re online” (Turkle 2011, p.172). Technology changes our relationships.

In this portfolio, I examine the role of imagination in identifying possible consequences before they emerge, and how imagination can help us understand relationships and systems to investigate new ideas and opportunities. New ways of describing systems helps us anticipate the impacts of things we build. It helps us talk to others about pieces of systems and how they might fit together.

This portfolio relies on a definition of cybernetics from Norbert Wiener, who used the term as the title of his 1948 book. This quote will appear again, but is essential:

"The most fruitful areas for the growth of the sciences [are] those which had been neglected as the no-man's land between the various established fields. ... Today there are few scholars who can call themselves mathematicians or physicists or biologists without restriction. [They] may be a topologist or an acoustician or a coleopterist. [They] will be filled with the jargon of [their] field, and will know all its literature and all its ramifications, but, more frequently than not, [they] will regard the next subject as belonging to his colleague three doors down the corridor, and will consider any interest in it on [their] own part as an unwarrantable breach of privacy." (1968, p.2)

We must find new ways to frame questions graspable in that “no-man’s land” where established disciplines draw boundaries. If machine learning does not overlap with sociology, we must ask sociological questions. Where legal systems do not touch algorithmic bias, we must ask legal and technological questions. New Practitioners are expected to develop a methodology for understanding, critiquing, and designing scalable, complex systems. That starts with expeditions into the “no-man’s land” and the deliberate blurring of borders that confine fields of knowledge. I will show that my training prepares me as a practitioner of the “New Branch of Engineering,” the 3A Institute’s endeavor to create practitioners capable of shaping cyber-physical systems ethically, sustainably, and at scale.

It begins by asking questions. How can we make space to talk “in a new sort of way" about this “new sort of world,” while we imagine and build vast complex cyber-physical systems (Beer 1975, p.1)? How might we deploy creativity and critical thinking in tandem — “the seriousness of a child at play” (Heraclitus) — to help us reveal the unexpected ethical, environmental, and social impacts of these systems? 

This portfolio presents three tasks that show my approach to answering these questions.

• First, I explore the history of color as data. This activity gives us insight into labeling and classifying the natural world, a process that has always been tied to the limits of available technologies, articulating specific questions (in the text) on how we understand data and its context.

• Next, I examine trust in algorithms and machine learning systems to process our data for predictions. The content asks, how much do we trust these predictions? The form, a video using sports as a metaphor, explores how we might engage new audiences in asking these questions.

• Finally, I examine how we know what is true at an even larger scale. The content of this group project maps an evolving, dynamic field, the New Branch of Engineering. This creative reinterpretation of a systems map exercise imagines the NBE as a boundary space where forms of knowledge engage and spill into one another, creating new relationships. It serves as a means of exploring the NBE. Its form is intended to ask questions about contextualizing and visualizing complexity: can we create a useful map of a system that is constantly redefining its boundaries?

Asking what can be hidden and revealed in the gaps between fields of knowledge is a common thread connecting these works. However, each piece raises distinct sets of questions. They have helped me to understand the value of my own artistic lens in building a New Branch of Engineering. As an artist, I explore technology through a creative lens, but also raise questions of disappearance and absence. I am interested in the ways that technology can obscure or reveal points of connection. As a practitioner, I ask myself how my training as an artist — part of a broad, interdisciplinary skill set that also involves psychology, ICTs, and research backgrounds — interacts with other fields of knowledge. How does art move out of the gallery and become a lens for critical thinking? What are the limits of this approach?

I have opted to present the work as a website to accommodate the multimedia nature of the presentation, from text to video to slide deck. You can navigate to the original pieces by clicking on the “open” button to load the document in a new window. Other pieces are directly embedded. When complete, please close any open window to return to this one.

item_0:

Who Decides the Colors of Birds?

In this piece I examine one of the building blocks of cyber-physical systems: data. With a focus on the W3C’s “safe web colors” list, I looked at the history of collecting, naming, and communicating color. I reflect on technology’s role in shaping our imagination of the world. It was selected to show skills relevant to the NBE, especially the skill of taking systems apart. That is, re-evaluating familiar knowledge or technologies to find deeper context: how they were shaped, and how they shape us. Further lessons are identified within the piece. It is a testament to the value of design research, sparking new questions as a result of asking just the one: “Who decided the colors of birds?”

In form, the initial poster (shown here) displayed the difference in how a bird would be portrayed based on the inks available in 1886 and 1814. It is a concrete visualization of the idea that technology shapes our understanding and imagining of the world. As the piece explains, the arrival of new and cheaper inks allowed for a widening imagination of color. Based on feedback from Ellen Broad, the piece has been expanded beyond a single poster into a piece of writing examining the history of color; it contains a set of questions and new pieces of visual art created by a style-transfer machine learning algorithm to mix historical color guides and images of birds that inspired the names of those colors. I have extended the form to the entire post, as I believe the story told by the poster is compelling, and I ask that it be appraised in the full context rather than the visual.

In the future, I hope to continue excavating insights from taking apart and re-contextualizing datasets. Examining technology’s influence on the categorization of the natural world highlights epistemological questions about what we know and how we know it. This is a useful tool in framing new sets of questions from the heart of Wiener’s “No-man’s land.” Since creating this piece, I have also explored musical notation as it has been used to document birdsong, which informed my maker’s project.

This piece had a strong impact beyond the classroom as it was shared as a blog post with more than 400 readers in the fortnight that it was published, which was a strong start for a new website.

Task #3, Revised: Note for the sake of the brief, the “poster” has been broadly redefined to include the blog post, in keeping with the brief that the size “may suit the approach.” The image at the header is a variation of the submitted poster with adjusted dates and cleaner lines. The text, initially on the poster, has been expanded to the full post. They should be considered as a whole.

Item_1:

How much would you bet on a prediction?

This short video raises questions for machine learning and algorithms in public policy. How might machines make mistakes? How do we know when to trust a prediction? Aimed at politicians, I explore Simpson's Paradox using the lens of sport. This paradox arises when we combine multiple data sets, and can obscure the stories told even by reliable data. This is not limited to machine learning applications, but can emerge anywhere that varied data sets get combined (Ma, 2011).

I selected this project to show that I can speak to a diverse set of audiences. Using sports to explain machine learning to politicians is not at all intuitive. Furthermore, I am not a sports fan. This took me well beyond my comfort zone to create something that speaks to a new audience about the topic. It also highlights skills in researching problems.

Now I can add Simpson's Paradox to a tool box for analyzing cyber-physical systems. It is a useful to see how machines can use reliable data to create unreliable results (Krakauer, 2020). Though I had done previous work in algorithmic bias, this example is different. I know I must look at how we prepare data, even if the data itself is thoughtfully collected. Simpson’s Paradox is a crisis of context. We can only see it when combined data sets are taken apart and analyzed.

Based on feedback, the piece has transformed from a spoken presentation to a video. I added a section explaining the difference between AI and Machine Learning. Unfortunately, I did not have time to add an example of Simpson’s Paradox in society. I'd found a compelling case study about gender admissions at UC Berkeley in the 1970’s (Bickle, 1975). I cut it for time after weighing science communication research. Depending on one's goals, an abstract approach can be more useful in raising questions among a diverse set of policymakers. Research has shown that partisan identity markers can trigger a rejection of evidence (Hart, 2011). Unfortunately, "gender bias" is still a politically wrought topic. By showing that machine learning can make mistakes in a politically neutral space of sports, I hope that the message moves through partisan resistance. The video “leads to new sets of questions” but does not point to specific policies. I must trust that the audience will arrive at their own questions about algorithms. In my future practice, knowing what to share, and what to withhold can be of vital importance to a message. On the other hand, I fear that this will create an excuse that distracts me from fully engaged advocacy. This is a set of questions I am still addressing as a practitioner, and will need to be considered carefully in any new context.

As a way of demonstrating impact and application of this skill, I’m sharing another piece created using this methodology. “Nothing to See Here” was created just after the above sports presentation, and was picked up by the London-based new media arts organization Furtherfield’s podcast, “News From Where We Are.” It relied on the same prompt, raising questions about machine learning and image classification models through the lens of photo-sharing websites, focused on automatic text created for covid-19 stock photography. The piece raises questions about what happens when the models we train cannot “keep up” with the contexts they were designed for. You are invited to listen to the six-minute segment below, at the 48:30 mark. You can also read the piece with a slideshow and transcription.

Task #4, Revised. Podcast is not submitted for formal evaluation, only to show impact & application of the skillset beyond the classroom. Revised video made use of feedback from instructors to explain AI and machine learning; expanding on visualization and storytelling skills.

Item_2:

The New Branch of Engineering: A Cybernetic Systems Map

Can we create objects that help many disciplines communicate with each other? Can we build objects to encourage the “crossing of conceptual wires” (Geertz 1973, p.82) and new combinations of ideas? This systems map prototype has a foundation in cybernetics. We applied cybernetics toward understanding the New Branch of Engineering. We relied on systems thinking approaches and the concept of the boundary object (Star 2010). It imagines a dynamic, open-ended representation of a rapidly changing, transdisciplinary field of knowledge, the NBE.

"Boundary objects are a sort of arrangement that allow different groups to work together without consensus," writes Star (2010, p.602). Star notes that such objects include "serious play endeavors" such as skiing or hiking; "what is important is how practices structure, and language emerge, for doing things together" (2010, p.602). The “boundary object” overlaps with another product, the systems map. For Meadows, “a system must consist of three kinds of things: elements, interconnections, and a function or purpose,” (2015, p.11) while for Bateson, the most complete maps of a system account for their own incompleteness (Bateson, 2010). This game board/systems map is an attempt to bridge all of these ideas in a way that captures the emergence and dynamism of the New Branch of Engineering.

I selected this piece to showcase my engagement with an open, creative approach to systems mapping. It raises provocative questions about itself and its own status as a “map,” following the course practice of playfully challenging the distinction between “map” as a noun and “map(ping)” as a process. It also illuminates relationships and exchanges between contributing branches of knowledge from the NBE.

Based on feedback, the map now includes more versatility in what fields are represented, based on new encounters. This follows the guidance that the New Branch of Engineering, as a transdisciplinary field, should be open. Balancing this openness within a structure was a challenge that raised interesting questions about what is ideal versus what is observed. Therefore, we suggest that it is not a perfect map, but one that is forever awaiting completion by inviting visitors and guests to contribute to the range of represented fields and lived experiences.

This project is distinct from the video piece in that it taught me skills beyond speaking to audiences. Instead, I thought of how to include participants in imagining their part within a system (in this case, the NBE). The video shows a concept, the map shows a context. I believe the NBE must show the contexts of complex systems with fewer boundaries, and highlight points where many systems converge and interact. This is a deep communications challenge that involves charting moving pieces and interactions with feedback loops. This game board helps us imagine complex sets of relationships within fields of knowledge. I will continue to explore novel methods for understanding and revealing how other systems mingle, react, compete, and co-exist.

In my future practice, I also hope to explore the use of deliberately designed boundary objects as conversation starters. I believe these objects and activities can connect people across disciplines and invite discussions between forms of knowledge. In this sense, an example in practice would be Boho Interactive’s “Best Festival Ever,” an “interactive theatre performance based on Systems Science” that uses a tabletop game setting to provide tactile, emotionally engaging examples of complex systems (Boho Interactive, 2020).

The team for this project also hoped that, when there are open maker spaces on campus, we could print the board and its pieces as a 3D object, and test its deployment among the cohort and staff of 3Ai (and its guests). This would help us see if the game inspires the discussions it is intended to create, and ideally serve as a conversation piece for future generations of the Applied Cybernetics masters program.

Task #5, Revised. Process of revision explicitly documented within the slidedeck. Collaborative project with Dianna Gaetjens and Maxwell Phillis; iteration was done independently based on classroom presentation feedback and observations.

Works Cited