Some people have the privilege to be recognized in our society. We’ve started to resurrect history and tell stories of people whose contributions have been studiously omitted. In America, February is Black history month. I didn’t learn Black history in school. I value this time for remedial studies, even as I feel a bit disturbed that we need to aggregate people by race to notice their impact. I had hoped that we, as a society, would have come farther along by now, in treating each other with fairness and respect. Instead, we are encoding our bias about what is noticed and who is recognized.

At the M.I.T. Media Lab, researcher Joy Buolamwini has studied facial recognition software, finding error rates increased with darker skin (via NYT). Specifically, algorithms by Microsoft, IBM and Face++ more frequently failed to identify the gender of black women than white men.

When the person in the photo is a white man, the software is right 99 percent of the time.
But the darker the skin, the more errors arise — up to nearly 35 percent for images of darker skinned women

A lack of judgement in choosing a data set is cast as an error of omission, a small lapse in attention on the part of software developers, yet the persistence of these kinds of errors illustrates a systemic bias. The systems that we build (ones made of code and others made of people) lack checks and balances where we actively notice whether our peers and our software are exercising good judgement, which includes treating people fairly and with respect.

Errors made by humans are amplified by the software we create.

Google "knowledge card" for Bessie Blount Griffen, shows same photo for Marie Van Brittan Brown and Miriam Benjamin

The Google “knowledge card” that appears next to the search results for “Bessie Blount Griffen” shows “people also searched for” two other women who are identified with the same photo. It’s hard to tell when the error first appeared, but we can guess that it was amplified by Google search results and perhaps by image search.

I discovered this error first when reading web articles about two different inventors and noticed that the photos used many of the articles were identical. This can be seen clearly in two examples below where the photo is composited with an image of the corresponding patent drawings. The patents were awarded to two unique humans, but somehow we, collectively, blur their individual identities, anonymizing them with a singular black female face.

I found a New York Times article of Marie Van Brittan Brown, and it seems that the oft replicated photo is Bessie Blount Griffen.

Additional confirmation by @SamMaggs tweet, author of “Wonder Women: 25 Innovators, Inventors, and Trailblazers Who Changed History”

photo of black women and patent drawing of medical apparatus
source: Black Then

patent drawing with home security system, with text: Marie Van Brittan Brown invented First Home Security System in 1966
source: Circle City Alarm blog

Newspaper article with photo of black woman and man behind her, caption: Mr and Mrs Albert L Brown

In 1991, the Company of Science & Art created a music video for Phish. It featured richly colored pastel drawings by Scott Nybakken with just a little bit of vector art and compositing that was likely done in Photoshop 1.0.

The Esther animation (below) was recorded in real-time from Apple Macintosh IIfx running PACo a software-based animation engine developed by The Company of Science & Art

At the time that this music video was made, CD-ROMs were new and Apple Macintosh computers had recently started supporting color monitors. The IIfx was the high-end of the Mac product line, and I remember that computer had 8MB of RAM, which was the most of any Mac in the office.

The sound file had to be annotated with sync points, which were produced by importing the audio into SoundEdit and then adding labels using the graphical user interface and then I think the labels could be exported as its own file (or maybe it was embedded in the sound file, I don’t remember). We used HyperCard as the original user interface for PACo, so we could experiment with new features quickly. The software would combine the labels, audio and a folder full of images, and also allow you somehow to specify transition between the images.

Images were 8-bit color, which means that each pixel was stored as a number which was an index into a color palette. Some of the color shifts in the Esther flying sequence were animated by a specific animation technique that was accomplished by rotating the color palette while leaving the pixels on the screen unchanged. This allowed for faster transitions and smoother animations than would otherwise have been possible. Even the fastest personal computer at that time was too slow to redraw the full screen at the framerate required for animation to look smooth. Notice that where elements are animated only a portion of the screen changes at any one time, such as with the church window sequence (2:07) and the spinning girl (3:40).

It was really fun to work with Scott Nybakken and John Greene who created effects that seemed to stretch beyond what was possible with the software and hardware.

“You can’t do that, it’s illegal,” was a common response to our attempts to bring modern tech practices to the US government. I worked for the Smithsonian Institution in 2013, then for the General Services Administration (GSA) from 2014-2016. First as a Presidential Innovation Fellow, then at 18F, I was part of a large group of industry experts who were recruited to transform government in partnerships with a few talented and experienced civil servants who understood policy and law.

I think it was Stephanie Rivera who first coined the term, full-stack bureaucrat. She had a much more formal title in real life, but this informal moniker helped me understand the work of policy makers through the lens of a software engineer. Whenever we were blocked by someone telling us something was illegal, it was important to find out what rule or law was actually causing the perceived or real conflict. This was akin to debugging the source of a software problem, isolating a bug to a specific place in the “stack.”

Layers of Rules

I think of the bureaucracy stack with practice at the top and the constitution at the bottom, even though the whiteboard photo I saved from one of the full-stack bureaucrat tech talks (below) lists them in the opposite order. In the list below, the top of the stack is the part that is most easy to change.

  • Practice is what people do, which is designed to be within the law. It’s actually rather easy to accidentally break the law by doing something outside of your job description as a civil servant. This is the easiest place to change things if you can establish trust and do some research to show precedent or get a manager, someone from senior leadership or the general counsel to confirm that the new thing is legal. You often find helpful documentation from lower down in the stack…
  • Guidance comes in the form of a memo or documentation that explains some nuanced point. Senior leaders, usually in consultation with general counsel, can issue new guidance when a new practice needs to be established. Sometimes you can find guidance that supports the change you are trying to make, or at least
  • Policy is written by an agency for itself or for all agencies by a few special agencies like the Office of Management & Budget (OMB) or Office of Personel Management (OPM).
  • Regulation is sometimes specific to an agency and sometimes applies to the whole government. Whatever governing body wrote the regulation is the same one who needs to modify it. Sometimes regulations are written into law. By the time you get here in the stack, and below, you are working in an entirely different realm.
  • Law requires an act of congress.
  • The Constitution

It was a little scary at first, when someone would tell me that what I wanted to do was illegal. After a while, I became adept at asking for more information. Meeting opposition with curiosity is a powerful technique. “Really?” I would ask “Could you tell me more about that?” or “Who could explain this to me in more detail?” Usually the policies and laws weren’t actually designed to prevent what I was seeking to do, and it was just a matter of changing the words that I used to allow me to create the change that I needed.

Notes on Terminology

The term full-stack bureaucrat is a nod to 18F’s staff of full-stack developers. The term “full-stack” describes software engineers who are capable of writing code at any level in the “stack” — from HTML that runs in the browser, to middle tier ruby, python or java to SQL that runs in the database (the bottom of the typical web app “stack”). In the early days of 18F, having a staff of full-stack developers was essential for the technical work, where a small staff needed to be prepared to hack anything.

As it turned out, we also needed to learn how to hack bureaucracy. Just like in debugging software, in order to isolate the problem and come up with a fix that will work, we need to figure out where in the stack the source of the problem lies. Understanding the “stack” that composes the legal framework of our government helped us navigate and modify the processes that sometimes prevented us from doing the technical work that was needed.

On a whiteboard from top to bottom:  Congress-Law, OMB/Agency:Regulation, OMB/Agency:Policy, Agency:Guidance, Agency:Practice