What does the mouse see?
Notes from an observer at the 2021 Computer Mouse Conference
(Here is the official conference site)
A computer mouse conference (see the official site) has no business being as good as this one. Activists, early career artists and scholars in semiotics, anthropology, and design, each sought to answer to the question, “what does the mouse see?” A number of the presentations embraced playfulness, vulnerability, and earnestness, which seems like an apt and appropriate way to run an online conference in 2021.
Conference organizer and artist-researcher emma rae brunl norton (alongside Ashley Jane Lewis) summed her thesis up straight away: “the mouse is a way to understand our relationships with computers.” Dissecting the mouse, as metaphors and objects, cracks open our relationships with information machines. The conference offered new ways “in” to complicate that relationship and generate new sets of questions.
Sound issues and an antsy dog sometimes made arguments and connections harder to follow. But the gaps that emerged as we moved from history, theory, and design were potent places for the imagination to thrive. In that spirit, this is a collection of notes from that conference — which had me glued to my screen whenever I wasn’t walking my dog. (Apologies to Daniel Shiffman whose talk I missed!)
Day One
The Spinning Wheel
Ingrid Burrington presented a short visual history of the wait cursor: “the spinning wheel of death,” tracking its development from black and white circle to stopwatch to the swirling rainbow whirlpool. The swirling rainbow, Burrington suggests, is based on the refraction of light on a compact disc or hard drive. In a brief literature review, Burrington found the counterintuitive impulse to romanticize the swirl, as something signaling the resolution of an outcome, a wish about to be granted. On the flip side, Angels are also spinning wheels, full of eyes, telling those who see it, “Do not be afraid.” The appearance of the lagging disc is part terror, part hope.
All Sensation is Touch
Nabil Hassein tells us that Cleopatra lived closer to the invention of the computer mouse than to the construction of the Pyramids of Giza. Citing Gera Lerner, Hassein notes that “Knowledge of history is knowledge that things have changed and that things can change.” (The Creation of Patriarchy, p.3). Today, the keyboard and mouse work in tandem, with some coders preferring the keyboard and hotkeys to navigate — and this is often the interface of choice for a variety of accessible interfaces. The mouse works in tandem, but is less intuitive. Its physical location does not map 1:1 with the space on the screen. Hassein ends with the question: What does the mouse feel? In the end, Hassein suggests, all sensation is touch: your eyes make contact with light to see, your ears make contact with vibrations to hear. The mouse feels our intention, and processes that into action and movement.
This is not the Windows XP Desktop.
What does the desktop want?
For Cezar Mocan, the green hills and booting chime of Windows XP was “a seductive invitation to possibility” while growing up. But it’s led to a question: why is every manufacturer using landscape photographs for their default desktops? As the metaphor on the screen moved from a terminal to a visual one, it relied on objects people were familiar with: office spaces. Later, menus and submenus moved users more toward memory: knowing what was possible, and how to find it, was the core compromise of the desktop metaphor, moving away from the complex possibility of the command line to the constrained simplicity of the desktop: a visual arrangement of pre-selected choices.
Drawing on a history of landscape photography, Mocan sees that the landscape desktop brings nature to that environment: the mountains and skies suggest a return to vast possibilities; but also a sense of orientation to space. They lack people and animals (“can’t have cows compete with desktop icons,” one commenter noted). This suggests timelessness — an unbroken moment in time — but also suggests an idea of nature without the complexity of life. In answer to the question, “What does the desktop want?” Mocan suggests the desktop wants to forget. It suggests to us a timeless ideal of a natural world, capable of sustaining us forever in its green lush grasses, and removing any sources of friction that might intervene with bliss.
What do we point at?
Kameelah Janan Rasheed is a learner and artist, presenting work using mouse coordinates while typing poetry to generate music. While this was algorithmic sound composition, it was achieved using analog means: recording the screen while she typed, she then replayed that video with a transparency on the screen, making marks where ever the mouse moved. This was translated, on paper, into a grid, which created a structure for generating sound compositions. Rasheed ended with a note on the mouse as a kind of divination tool, a tool for pointing at our thoughts.
Semiotics of the Click
David Bering Porter’s incredibly deep presentation focused on the “click.” We measure the life of a mouse in terms of our physical interactions with it: 60 million clicks, for example, is the lifespan of a mouse used by extreme gamers. I was surprised to be reminded that the click is a physical sound, not only a verb — I had to click my mouse a few times to remind myself that yes, there’s an audible sound in that tiny switch being moved on and off with the tap. Porter tells us a click is a gesture, a sound, a metric. It’s a relationship: we click when we find affinity, the result of feedback. (For me, it’s also an idea sliding into focus: it finally clicked.)
All of these tell us something about the way our relationships, physical gestures, and interactions have been flattened into a single space: the space of clicks and clicking.
What I found fascinating from Porter’s talk is that the click was accidental, but kept, and in some cases it is simulated. Looking further, there are ways to rig a system to play a different sound when you click the mouse button.
I’m dreaming of a surveillance capitalism mouse, which plays a terrible cash register “ka-ching!” anytime your click is collected by data brokers. If anyone wants to build that with me, let me know. (Really).
Computer Mouse X Magic Wand
CyX is “a black queer non-binary storyteller and cyber witch merging sound, video art, installation, and performance.” Bringing in the background of magic wands, tools designed for health and focus (look at the World Health Organization logo) CyX suggests that the mouse can be treated as connectors between energy forces, moving our intentions into the digital realm. In a workshop, we were guided through tasks such as braiding plants and flowers into the tail of the mouse, the scents providing a reminder of that connection across two spaces. The mouse is a tool that connects our and work and relationships; why wouldn’t we adorn it in welcoming, contemplative, and inspiring ways? If time, relationships, and creativity is sacred, why don’t we create more space for the sacred in digital spaces?
The Mouse House
The always-inspiring Shannon Mattern (whose writing on Plexiglass is my go-to answer when people ask me “so what do you want to do?”) offered us a tour of tiny houses occupied by mice: the mammalian ones, the ones that Beatrix Potter anthropomorphized into clumsy, but studious working-class complexity. From that imagining of mice living alongside us in tiny homes built from our debris, we moved to the medical mouse: standing in for the bodies of humans in medical tests, living in caged grids, the original “mouse pad.” The mouse pad — the subject of this talk — rose from the need to prevent dust from gumming up the tracking ball inside the original mice. The optical mouse arrived, and then lasers — “turning the whole world into a mousepad” — and finally trackpads, which were the mouse and the pad and neither. But for precision work, and the market for gaming, the industry of the mouse pad is still alive and well, in a constant search for the perfect surface.
Does the mouse see race?
Charlton McIlwain — whose book, Black Software, is high up on my reading list — asks if the mouse sees race. To begin with, he points out that the father of the mouse (Douglas Engelbart) was a white man who was trying to solve a series of relational problems and used his body to solve them. Mark Dean, a black IBM engineer, would two decades later create a device that allowed for a broad range of peripherals to work with the same machine. These developments “made computer hardware bend to the users will,” rather than people bending to it, McIlwain says (citing a Usenet discussion). These tools opened the world of computing to more people, creating more points of relationship, including for black and brown people.
McIlwain then pointed to the Implicit Associations test: a tool designed to reveal unconscious bias. The tool scrambles pictures of diverse faces and words with negative and positive associations. You are supposed to click and sort specified combinations, and changes in reaction times for certain tasks can be measured. This test is not a diagnostic tool, and isn’t mentioned as an endorsement, but because of how much this test has shaped ideas and conversations around race. But the mouse is an intermediary for this unconscious bias and the screen used to test it, suggesting that the way we have come to understand implicit bias is now informed by our relationship with the mouse. In the end, the mouse “sees” race through our interactions.
Neta Bomani’s live stream is also an archived audio stream of the conference. Watching the Zine created over the course of the event is also a different way in.
Day Two
Nature’s Notifications
Ryan Clarke “Nature’s Notifications” was a short film and discussion. These are my notes:
Animals looking at cameras, animals mating calls, birds that imitate car alarms / camera shutters / even forester’s chainsaws. A range of natural calamities scored by the nostalgic sounds of technology. A bug trapped by lines drawn by a pen. Animated trees burning in vaporwave. The captions were missing! Tensions were clear: How can we see technology in terms of geological time? How does the Windows Desktop mountain change after watching a series of landslides?
Protecting the Microchips
Dorothy Santos shared the story of Fairchild Semiconductors (acknowledging the research of Lisa Nakamura), a microchip manufacturer built on a Navajo reservation to access Navajo labor. Through women’s work, indigenous bodies became part of this technology: women’s eyes and fingers distinctly present in their construction, informed by histories of Navajo labor of craft, metal work, and weaving. Santos then turns to another way of telling this story, the docu-poetic work of Janice Lobo Sapigao in her book, “Microchips for Millions.” It’s a collection of poems that “captures her mother’s traumatic experience as an assembly line worker in Silicon Valley, as well as the larger social, economic, and environmental impacts of the high tech industry. The poems switch between English, Ilokano, and binary code, and between documentary, visual, ethnographic, and lyric modes.”
In one poem, The Clean Room, Santos tells us, we see the rules of entering the clean room coldly stated on one side of the page, with imagined conversations between Sapigao’s mother and her supervisors at the plant and with her daughter, explaining the reasons why she can only call during certain times of day, signs to the left telling us that cell phones must be left outside. Poetry of this kind, Santos tells us, is a different way in to seeing technology: “outside the reach of the object and outside the reach of capitalism.”
The End
Allison Parrish, Ayo Okunseinde, and Mimi Onuoha closed the event with a panel moderated by Ashley Jane Lewis and emma rae brunl norton which I am not quick enough to document well. But some quotes seemed like great “next steps” to end on:
“What is the gaze of the mouse, and what is it that draws its attention?” — Allison Parrish
“Does the movement of the mouse ever leave the body?” — Ayo Okunseinde
I highly recommend going over to the website to see the full archive of the conference.
(If you’re a panelist and don’t like the way you’re represented just let me know and I’ll take it down.)