Measuring the Societal Impacts of AI: The MIDAS Initiative

15 Dec 2024

U.S. Census Bureau Emerging Technology (xD) Fellowship

Problem

The column nobody can fill: a table of hiring, credit, housing, and benefits decisions, showing what the system does at each but leaving 'what we know nationally' blank for all four, captioned 'No representative measure of where these systems are deployed, how people experience them, or whether they know it happened.'

Algorithmic systems make consequential decisions about people at scale — screening job applicants, setting credit eligibility, allocating housing and benefits — and the harms aren’t distributed evenly. But the government’s response runs into a basic gap: no systematic, nationally representative picture of where AI is deployed, how people experience it, or whether those most affected even know it’s happening.

The lever is structural and sits where two systems meet. The organizational one already exists — Census reaches the households, businesses, and governments where algorithmic impacts land — and the question was whether that infrastructure could be turned toward accountability, not just AI development. The cultural one is why it matters: without population-level evidence, policy gets made in a vacuum and harms get documented reactively, through lawsuits, while the communities most affected have no way to be counted in aggregate.

Solution

The instrument already reaches them: households, businesses, and governments already feed the Census survey infrastructure, which already produces economic and demographic statistics. A second arrow points from that same infrastructure to algorithmic impact measurement via recurring Household Pulse Survey questions, captioned 'Same reach, pointed at accountability.'

I led with learning — mapping who was already trying to answer these questions and what blocked them before proposing what to build.

I helped design the measurement infrastructure from two directions: the survey instruments that generate the data, and the analysis framework that determines what it can answer. I helped develop Household Pulse Survey questions that asked about behaviors first and used framing to orient rather than gate respondents.

In analysis, I framed the questions that should drive the data — not just how many people use AI, but how AI could compound privacy violations, and how that exposure occurs across race, income, and geography. The framing itself was the deliverable, held lightly enough for others to reshape.

Impact

The count is not the question: the default question 'How many people are using AI?' is struck through. What the data should answer instead: where does AI compound existing privacy violations, who is screened by a system without being told, and how does exposure differ by race, income, and geography, captioned 'The framing was the deliverable, held lightly enough for others to reshape.'

The proximate users are fairness researchers, model auditors, state enforcement offices pursuing disparate-impact cases, and federal policymakers — state Privacy and Responsible Technology offices were already using Census data to estimate demographic exposure in bias investigations and needed better upstream measurement to scale it.

But the real goal is to change the conditions under which AI accountability is possible at all: a recurring national program gives advocates a baseline, gives Congress evidence to legislate from, and makes visible the populations AI harm data currently renders invisible.

Stack & Methods

Household Pulse Survey instrument design · behavior-first question framing · analysis framework design · algorithmic impact measurement · U.S. Census Bureau Emerging Technology (xD) Fellowship.