Methodology & Accuracy
2022 Accuracy Report
2020 Accuracy Report
The New Gold Standard
MEET PEOPLE WHERE THEY ARE – ONLINE AND ON THE GO
Change Research’s methodology is unique from any other pollster’s approach. We recruit respondents through a combination of online platforms, including digital ads and text messaging, to participate in online polls. We never use panels or incentivize respondents.
As we are collecting survey responses, Change Research’s Dynamic Online Sampling Engine adjusts our targeting to ensure a sample that accurately reflects the electorate or population, including populations that are traditionally underrepresented in polling.
We recruit fresh participants online for each and every poll, so Change Research polls are not affected by the dwindling response rates of phone polling.
We reach millennials and seniors, rural and urban dwellers, and members of every gender, race, creed, and political persuasion. In 2020, our methodology was more accurate than any other.
Reach a truly representative, inclusive sample
Change Research’s Dynamic Online Sampling Engine delivers a cross-section of respondents that accurately reflects the electorate or population.
We establish — and can rapidly adjust — advertising targets across age, gender, race, and partisanship to reflect a district’s demographics. Any imbalances or lack of representation in the survey sample can be quickly detected and corrected while the survey is still fielding, rather than relying solely on post-survey techniques for big corrections.
Target the right respondents – FAST
We use our proprietary geo-targeting technology to find respondents in a specific district, be it a city council district, Congressional district, or the entire nation.
We locate respondents rapidly and can deliver results as little as hours after a poll launched. For instance, we fielded an 800-person survey in only 24 hours in Iowa’s 4th Congressional District before the 2018 elections; we found that Democratic challenger JD Scholten was within a point of unseating Republican incumbent Steve King. The poll results sparked a flood of donations to the Scholten campaign, with over $900,000 raised within 48 hours.
Apply best practices in weighting
Leverage AI and big data
How does Dynamic Online Sampling compare to other methods?
Polling accuracy is deeply linked to how pollsters reach voters. Good sampling can make the difference between a poll that tells the right story, and one that is fundamentally flawed.
polling methodologies
Change Research
Change Research recruits respondents through a combination of online platforms, including digital ads, to participate in online polls. Change Research’s patent-pending Dynamic Online Sampling Engine delivers a cross-section of respondents that accurately reflects the electorate or population. It doesn’t offer financial incentives to respondents for participation.
Strengths
Change Research is fast, accurate, and affordable.
Change Research’s use of a combination of multiple platforms ensures that strengths and weaknesses are balanced. Change Research’s Dynamic Online Sampling engine allows for a balancing of demographics as the survey fields; additionally, Change Research is able to more easily and effectively reach voters under the age of 50 who rarely answer their phones.
ONLINE METHODS
Online Ad Invitation
Online Ad Invitation targets respondents through digital ads. These ads reach hundreds of thousands of voters in places they spend their time, allowing for a more representative sample.
Strengths
Affordably survey large sample sizes in a matter of days.
Reach a representative sample even in many small state legislative districts.
Test any sort of visual content ranging from images to videos.
Can field a survey quickly, rather than relying on slow and expensive call centers.
Weaknesses
Due to the targeting methodology, only 20-40% of respondents can be directly matched back to the voter file.
Text to web
Text-to-web polling is a newer methodology that surveys voters by texting them a link to a survey which they then take online. Polling firms that traditionally fielded live phone polls have increasingly adopted text-to-web polling to supplement their samples.
Strengths
Text-to-web polls can be fielded quickly as compared to phone polling.
The percentage of voters who can be directly matched to the file is high because polls are often fielded by texting voters on a voter file.
Text-to-web can be helpful for reaching respondents who are younger and nonwhite.
Weaknesses
Many people still do not click on links sent from an unknown number.
As cell phone carriers and phone developers take active spam measures, these texts may be marked as spam.
Text-to-web polls must be sent one at a time, which can be a time-consuming and labor-intensive process.
Online Survey Panels
Online panels are large universes of people that agree to join a panel of survey respondents, sometimes because the panel offers respondents incentives like money or gift cards. Online panels then invite certain participants to respond to certain surveys depending on their pre-identified demographics or other characteristics.
Strengths
You can survey the exact same person over time, which is beneficial for longitudinal studies.
Weaknesses
Most online panels do not have enough participants in small geographies to get a representative sample.
Because respondents are usually compensated, online panels have been found to have higher levels of “bogus” respondents than other methodologies.
phone polling METHODS
Interactive Voice Response (IVR) Phone Polls
Interactive Voice Response (IVR) polls are automated calls where respondents give their answers by pressing corresponding numbers on the phone keyboard.
Strengths
IVR polls are typically less expensive and don’t depend on calling house centers.
Like live calling polls, most IVR polls are fielded by calling voters from a voter file, which means you can match respondents directly to a voter file.
Weaknesses
IVR polls have even lower response rates in many cases than live phone polling.
Voice response systems are prone to errors or issues, seem to encourage more false responses, and cannot legally call cell phones. IVR surveys are also illegal in some states.
The survey experience can be dreadful if many candidates and issues are being tested.
Live Phone Polls
In live phone polls, interviewers call a random sample of landlines and cell phones oftentimes usually from a list of voters (the voter file) and ask those who answer survey questions.
Strengths
You can match most survey respondents directly to a voter on voter file.
Weaknesses
The survey industry has been experiencing declining survey response rates for years. Respondents who do answer their phones to unknown numbers are arguably not a representative sample of any electorate.
Polls conducted through live phone polling tend to be expensive, take longer to field, and have a hard time reaching larger sample sizes.
Live phone polling also relies on calling house centers, which are prone to error and often book up during peak campaign season.
Speaking with an individual over the phone may make respondents more likely to answer questions in a “socially desirable” way that does not reflect their true views. This is especially true for culturally sensitive issues like race or abortion.
Put more simply…
