Emerson College Polling: Methodology, Accuracy, and Innovation

Emerson College Polling (ECP) is a polling firm associated with Emerson College in Boston, Massachusetts. Known for its accuracy and innovative methodologies, ECP has become a prominent voice in public opinion research, frequently cited by media outlets across the country, including The New York Times. This article delves into the methodology employed by Emerson College Polling, its evolution, accuracy, and its broader impact on understanding public sentiment.

Origins and Evolution

Emerson College Polling emerged from the college's polling program, which was developed in the 1990s. In 2012, Spencer Kimball, an assistant professor of communications, began advising the student-run Emerson College Polling Society, marking a significant step in its development. Since then, ECP has played a crucial role in political analysis, conducting numerous polls across various states. For example, in 2024, Emerson conducted 54 polls in 20 states across a variety of congressional, gubernatorial, and Senate races.

Methodology: A Mixed-Mode Approach

ECP has developed a mixed-mode methodology that continues to evolve as communication preferences change. The firm studies modes of data collection and survey research to better understand the most effective ways to collect data and reach individuals. This adaptability is a key component of their success. As communication evolves, Emerson College Polling evolves its methodology to reach respondents where they can best communicate.

From Landlines to Online Panels and Text-to-Web

In its early stages, in 2016, ECP used a landline-only sample design. Recognizing the shifting landscape of communication, ECP expanded to a mixed-mode methodology in 2017 by integrating online panels into its sample. By 2019, Emerson College Polling began to incorporate Text-to-Web data collection into its methodology to achieve the most representative sample possible. In 2022, ECP’s mixed-mode methodology included IVR, online panels, and text-to-web data collection.

ECP leverages new technologies, such as text-to-web polling and online panels, to maintain its position as a leader in the polling field. Text-to-web polls contact people from voter registration lists. Participants receive a text with an image that indicates the poll is from Emerson College Polling, and a link to a survey they answer at a time convenient for them. According to Kimball, Emerson College Polling is "always innovating."

Key Components of the Mixed-Mode Methodology

IVR (Interactive Voice Response): Automated calls are placed to landline telephones of selected registered voters/likely voters/residents. Respondents answer the survey using their touch-tone telephone. IVR is not used in states where it is prohibited.
Online Opt-in Panel: Respondents are invited and screened for voter registration and basic demographic information (state of residence, gender, age, and ethnicity) through an online opt-in panel provided by CINT. Those who pass the screening questionnaire are directed to an online survey, with additional screening questions to ensure data quality.
MMS-to-Online: Target population (registered voters, likely voters, or residents) receive multimedia text messages with a custom graphic inviting them to take the survey online. The online surveys are hosted on Qualtrics.

Weighting the Data

To achieve a representative sample, ECP employs a weighting technique of demographic breakdowns. The first step in weighting is to survey more than enough people. This allows them to then be able to systematically reject individual surveys from demographics that are over-represented. The data sets are weighted by gender, education, race, age, voter registration data, and region.

Screening for Likely Voters

Through the use of a screening question, Emerson College Polling usually seeks out "likely voters" which has been shown to give a much more representative sample of the populations for most political polling.

Data Analysis: Univariate and Bivariate Approaches

ECP employs both univariate and bivariate analysis of the polling data. The results presented in their reports include univariate and bivariate analysis of the data. Frequency distributions for each item included on the questionnaire are shown in the tables. In all cases, cross-tabulation results are also shown. This type of bivariate analysis examines differences between sub-groups of the overall population.

The modes of data analyses are non-probability and deductive (from “general” to “specific”) in nature, so instead of using classical statistics to quantify the ‘uncertainty’ or range of scores a poll represents, they use Bayesian Statistical methods, that produce the mathematical likelihood of an event of interest, and that are inductive in nature (from “specific” to “general”), for data analyses in best estimate the true population mean proportion.

Accuracy and Transparency

The New York Times gives additional weight to Emerson College Polling results, and media across the country frequently cite their polls, because of its track record of accuracy and commitment to transparency. Emerson College Polling (ECP) is a Charter Member of the American Association of Public Opinion Research (AAPOR) Transparency Initiative.

Read also: Emerson College Expenses

Continuous Refinement

“Elections reveal if our data is skewed, helping us refine our methods,” Kimball explained. In 2018, Emerson’s most accurate results came from a mix of online and landline polling.

Innovative Research: Challenging Conventional Wisdom

Emerson College Polling studies modes of data collection and survey research to better understand the most effective ways to collect data and reach individuals.

Interactive Voice Recognition (IVR) Study

One example of ECP's innovative research is a study that tested the conventional wisdom that interactive voice recognition (IVR; also known as robocalls or auto calls) are not listened to by receivers. The study found that three out of four people (75%) listen to over 19 seconds of a message, which equates to over 40 words. The vast majority of people, 97%, listen to at least 6 seconds. This innovative, unobtrusive field approach for measuring actual listening time eliminates self-report bias.

People Behind Emerson College Polling

Spencer Kimball: Spencer Kimball is the founding Director of the Emerson College Polling Center. Kimball is an established expert in public opinion research. His work is published in mainstream news sources around the globe and journals including Nature Medicine and American Behavioral Scientist. Kimball has been advising the CUNY School of Public Health on COVID-19 related surveys.
Camille Mumford: Camille Mumford is the Director of Communications at Emerson College Polling. Mumford is also an affiliated faculty member in Emerson’s Communication Studies Department, teaches a class called Survey Research Methods. “Students in Communication Studies and Political Communication learn the survey research industry in the classroom,” said Mumford.
Collin Robisheaux: Collin Robisheaux is a communications assistant at Emerson College Polling.

Emerson College Polling and The Midwest Newsroom

The Midwest Newsroom and Emerson College Polling Center partnered on a survey that asked more than 1,000 registered voters in each of Missouri, Kansas, Iowa, and Nebraska about measures on Nov. 5 ballots as well as a variety of social and economic subjects. The separate surveys of each state were conducted between Sept. 26 to Oct.

Key findings:

Read also: Discover Emerson College

More than 60% of respondents in all four states said a college degree is not worth the cost.
About 10% of respondents in each state said climate change is caused “entirely by human activities” while about 24% said it’s caused “mostly by human activities.”
When asked where they find news about state and local politics they trust, about 35% of respondents across all four states responded “local media,” while about 25% said “social media sites.”

Spencer Kimball, executive director of Emerson College Polling said: “Collaborating with The Midwest Newsroom gives us a rare opportunity to blend the insights from pre-election polling with the on-the-ground insights of local journalists, providing a unique and invaluable perspective ahead of the 2024 election. By going beyond the ballot test questions, this partnership elevates the critical issues that matter most to communities in Iowa, Kansas, Missouri, and Nebraska.”

Holly Edgell, managing editor of The Midwest Newsroom, said the project aimed to gauge voter sentiment across a range of issues, themes, and topics that recur in popular discourse. “These are the questions of our time,” she said. “We were less interested in asking how people are planning to vote in November than asking how people are thinking about issues close to home as they grapple with them in daily life.”

Edgell said the project will help reporters in the region in their coverage of issues that elicit strong opinions but can also be more nuanced than conventional wisdom suggests. “Emerson College Polling worked closely with us to develop surveys that speak very specifically to the four states that The Midwest Newsroom serves,” she said.

Project Methodology

The sample of registered voters for each state, n=1,000, has a credibility interval, similar to a poll’s margin of error (MOE), of +/- 3 percentage points. Data was collected September 24-27, 2024. The data sets were weighted by gender, education, race, age, voter registration data, and region.

The Future of Polling: Beyond Politics

Election polling is a billion-dollar industry and survey research is about an $8 billion industry, Kimball said. “We’ve only scratched the surface,” said Kimball. “We’ve done COVID, vaccine, and health studies like bird flu and fatty liver disease.

Adapting to a Changing Landscape

As the presidential election approaches, news reports and social media feeds are increasingly filled with data from public opinion polls. Emerson College Polling leads a dynamic survey operation that has continuously evolved to keep pace with shifting trends and technologies in survey research.

In the early days of the survey industry, being asked to participate in a poll was novel, and response rates were high. With fewer landlines, busy parents juggling work and family, and younger adults who rarely answer calls, preferring text communication, it has become much harder to engage respondents. In the broadest possible terms, polls and surveys have two elements - choosing whom to contact, and reaching them in a way that’s likely to get a response.

Stratified Sampling

By the 1990s, pollsters began moving away from random digit dialing, which was time-consuming and expensive because the random selection often picked phone numbers that were out of service or not useful for opinion surveys, such as businesses or government offices. The information in these and other associated public records, such as those detailing gender, age, and educational attainment, allowed a refinement of random sampling called stratified sampling. Survey-takers then chose randomly from among those subgroups in proportion to the population as a whole.

Interactive Voice Response

Other advances in ways to reach respondents emerged late in the 20th century, such as interactive voice response, which did not require live operators. Instead, automated systems played recordings of the questions and registered the spoken responses.

The Rise of Cellphones and Online Platforms

Over the past two decades, the rise of cellphones, text messaging, and online platforms has dramatically changed survey research. The traditional gold standard of using only live operator telephone polls has become nearly obsolete. Even the random sampling that was once standard has given way to a nonprobability sampling approach based on increasingly specific population proportions.

Quota Sampling

In quota sampling, participants may not be selected randomly but rather chosen as participants because they have specific demographic attributes. This method is less statistically rigorous and more prone to bias, though it may yield a representative sample with relative efficiency.

River Sampling

A newer technique pollsters are using to reach respondents is something called river sampling, an online method in which individuals encounter a survey during their regular internet browsing and social media activity, often through an ad or pop-up. This combination allows ECP to reach a broader, more representative audience, which is essential for accurate polling in today’s fragmented social and media landscape.

Tailoring Communication Methods

When contacting the people in their stratified samples, ECP takes into account differences between each communication method. For example, older people tend to answer landlines, while men and middle-aged people are more responsive to mobile text-to-web surveys. They also use information about whom they sample and how to calculate the margin of error, which measures the precision of poll results.

tags: #emerson #college #polling #methodology