Case StudyAdvocacy Organizations

Proof, in the wild

How Change Research measured whether pro-Biden digital ads actually moved voters in the field, using a matched control district instead of a message-testing survey alone.

+8 ptsfavorability gap vs. matched no-ad district
+10 ptsjob approval gap vs. matched no-ad district
Progress Iowa
The Challenge

Message testing can tell you which ad voters say they like. It can’t tell you whether that ad, aired in the real world, amid an actual media environment and a barrage of competing messages, changes anyone’s mind. Ahead of the 2024 cycle, Progress Iowa needed to know whether its pro-Biden storytelling ads were actually working once they were in the field, not just in a lab.

The stakes were specific: August 2023 was a period of heavy Republican presidential campaigning in Iowa, and Progress Iowa needed evidence that its ads could hold the line against that pressure among “no party” voters — the audience most in play for 2024.

The Approach

After an upstream message-testing phase identified two testimonial ads (community economic development and farm conservation) as the strongest candidates, Progress Iowa put them into the field: two weeks of paid media across Meta, YouTube, Connected TV, and programmatic video, targeted to “no party” voters in Iowa’s 3rd congressional district, backed by $90,000 in spend.

Rather than stop at message testing, Change Research built a field study to measure what the campaign actually did to real voters. Change Research fielded matched pre- and post-campaign polls in two places: IA-3, where the ads ran (n=421 before, n=501 after), and IA-1, a demographically equivalent district that received no ads (n=417 before, n=491 after). The two districts matched closely on sex, age, race/ethnicity, education, and 2020 vote choice going in, which meant any difference in how the two districts moved over the campaign period could be attributed to the ads themselves — a difference-in-differences design that measures real-world impact rather than stated ad preference.

The Results
-9 ptsBIDEN FAVORABILITY DECLINE, NO-AD DISTRICT
-1 ptBIDEN FAVORABILITY CHANGE, AD DISTRICT
-8 ptsJOB APPROVAL DECLINE, NO-AD DISTRICT
+2 ptsJOB APPROVAL CHANGE, AD DISTRICT

In the matched district that received no ads, Biden’s net favorability fell 9 points and his net job approval fell 8 points over the campaign period — the erosion Progress Iowa was trying to get ahead of. In the district that saw the ads, both numbers held essentially flat: a non-significant 1-point dip in favorability and a non-significant 2-point increase in job approval.

That gap between the two districts — measured in the field, among real voters going about their lives, not in a survey about the ads themselves — is the campaign’s real-world effect: roughly an 8-point swing on favorability and a 10-point swing on approval, attributable to the ads rather than to broader national trends, since both districts started from the same place and were exposed to the same national environment.

Success

Ready to write your own success story?

Let's talk about how Change Research can deliver results like these for your campaign.

Let's talk strategySee our methodology