Accuracy

We launched Change Research in 2017 with the belief that polling needed to modernize. The world had moved online, and polling needed to do the same. Few people will answer calls from an unknown number, so how can accurate polling rely on them doing so? We also launched with the belief that polling would be at least as accurate, if not more accurate, with a methodology that finds a unique set of respondents online for every poll. This has proven to be true. Our teams of engineers, data scientists and survey analysts constantly monitor and fine-tune our processes, looking for improvements. We expect to only get better. Here is a brief summary of where we’ve been. A longer analysis is available here.

In 2020, Change Research’s polls in battleground states more than held their own. Our public and private polls were nearly 25 percent more accurate than other polls in battleground states. Change Research’s presidential forecasts in 2020 were uncannily accurate: the model projection was within a point of the actual result in New York, New Jersey, Pennsylvania, Massachusetts, Maryland, and Rhode Island, and within 0.01 in Connecticut.

In 2020, we conducted 27 polls for U.S. Senate races. Our average error in these polls was 5.7, while the average error of over 500 Senate polls aggregated by FiveThirtyEight was 6.1.

We also conducted 28 polls in congressional races in 2020, with an average error of 6.9. There were 214 congressional polls aggregated by FiveThirtyEight after Labor Day, with an average error of 8.09. We were 15% more accurate than other pollsters in Congressional polls.

Finally, we also conducted hundreds of polls for down-ballot races across the country. These polls, many of which were conducted in districts with under 100,000 people, which are notoriously difficult to poll, stood out for their accuracy. Overall, we conducted 77 polls in state senate districts with an average error of 5.94, and 84 polls in state house/assembly districts with an average error of 5.95. The table below shows our average error in a number of these states.

In 2021, we polled dozens of electoral contests. The majority were local, nonpartisan races with candidates who had little name recognition. Many of the races had more than two candidates. Despite each of these polling constraints, we were overwhelmingly accurate, showing the ultimate winner ahead in 88 percent of the races we polled within one month of election day.

We did a series of polls for nonpartisan races in Seattle on seven races – mayor, city attorney, two city council districts, and two school board districts. These were challenging races to poll, covering relatively small geographies, and with more than fifty percent of voters undecided in a couple of the races one month out from election day. We accurately predicted the winner in all seven races, with an average absolute error of eight points.