Title PresidentialAdWarTest

Text E

S

EVOLVING
STRATEGI ES

A fool, Mr. Edgeworth, is one who has never made an experiment.

— Erasmus Darwin

Who’s Winning the Presidential Ad War?

The impact of Romney and Obama campaign ads on weak partisan & pure independent voters.

Overview

Ad spending this election year is pushing toward the $1 Billion mark,
and there’s no sign of slowing. So who is winning the Presidential ad
war? To discover whether Obama’s or Romney’s ads are more effective
in shifting the vote, we partnered with Qualtrics and executed a
PocketTrial™ testing the individual and combined impacts of seven
Romney and Obama campaign ads.

Our PocketTrial™ quantifies the true impact of a message using a
customized, double-blind, and fully controlled experimental design. It’s
like a small-scale clinical drug trial, but it’s a message we’re testing.

We exposed more than 2,300 registered, pure independents and weak
partisans - no strong Republicans or Democrats - to one or more of
seven political ads from the Obama and Romney camps (the Control
group watched no ads). Respondents were randomly assigned to one of
11 conditions in a true experiment - 10 treatment groups and a control.
Statistical analyses were performed to identify significant “treatment”
impacts.

We tested the individual and combined impact of matched Obama
and Romney ads from three “themes,” 1) Medicare ads, 2) Economic
Attack ads, and 3) Comprehensive, economic-focused “Plan” ads. In
addition, we tested 4) the impact of the “Disappointed” ad from
Americans for Prosperity, which has been highly rated in focus groups
and received heavy air time.

Key Findings

So which side is winning the Presidential Ad War? It depends on how
much enthusiasm matters compared to marginal vote shifts this year . . .

The Obama ads have a significant impact on vote preference, increasing
Obama's vote and decreasing Romney's. The Romney ads aren’t
effective in shifting preferences.

Obama’s ads increase his vote by about 6 points on
average - a 15% bump in Obama's vote

They decrease Romney’s vote by about 8 points - an
18% slide in Romney's vote.

Round 1 to Obama: Obama ads increase the percentage
of marginal undecided and wavering voters who prefer him.

The twist . . . both the Romney and Obama ads are effective at
increasing enthusiasm among likely Romney voters (McCain voters from
2008). The Obama ads don’t significantly impact the enthusiasm of ’08
Obama voters.

Romney and Obama ads increase the highest level of
voter enthusiasm of ’08 McCain voters by 13 points - a
42% surge in the number of McCain ’08 voters who are
extremely enthusiastic to vote this year.

Enthusiasm for ’08 Obama voters remains flat.

Round 2 to Romney: Romney ads impact marginal,
decided voters, increasing the likelihood that voters who
say they prefer him will actually turn out to vote.

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

— Erasmus Darwin

Experiment, Survey Design & Analysis

Experiment & Survey Design

Respondents were randomly assigned (within Party identification blocks) to one of eleven
conditions in a true experiment - ten treatment groups and a control:

1. Control Condition

Respondents were not exposed to either ad before the policy and election questions
were answered.

2. Medicare Ad Conditions

Respondents viewed either a Romney Medicare ad, an Obama Medicare ad, or both.

3. Economic Attack Ad Conditions

Respondents viewed either a Romney economic attack ad, an Obama economic attack
ad, or both.

4. Comprehensive “Plan” Ad Conditions

Respondents viewed either a Romney ad laying out his plan for the country, an Obama
“plan” ad that also defends his record, or both of these ads as well as Romney’s
Economic Attack Ad.

5. AFP “Disappointed” Ad Condition

Respondents viewed an widely-aired ad produced by American for Prosperity, in which
Obama voters express their disappointment with the President.

All respondents otherwise answered the same survey questions in an identical survey
instrument. The order in which the ads were presented to respondents in the combined
treatment conditions was randomized, as were other questions and responses where
appropriate. The sample was drawn from an opt-in panel to approximate the 2008 electorate
in the CCES 2008 validated dataset (non-strong partisans only) on education, age, gender, and
race. All respondents were screened to be registered, pure independents and weak partisans.
The survey was conducted from Sept. 28-Oct. 3.

Analysis

Impact analysis was conducted using multinomial logistic regression models, which
included variables for treatment interactions, treatment/gender or ’08 vote choice interactions
as well as party identification, liberal/conservative self-identification, age group, attention to
the news, and being “born-again.” Predicted probabilities are adjusted to match the ’08 vote,
by gender, in the CCES 2008 validated dataset. All percentages are predicted probabilities
based on the marginal impact of each treatment. Significance was determined using a pairwise
comparison of treatment impact to control for all respondents or by gender and/or ’08 vote.

The basic structure of a
PocketTrial™ . . .

1. Respondents enter the PocketTrial . . .

2. Respondents answer screening and
control questions . . .

3. Respondents randomly assigned to a
treatment or control group condition . . .

m , XlX

flHm oil iits flfffl

Condition 1 Condition 2 Condition 3 Condition 4

4. Respondents are exposed to their message . . .

1 1

1

M M

(Control) (Message 1) (Message 2) (Message 1 + 2)

5. All respondents answer the same questions that
the message is supposed to impact . . .

6. Statistical analysis identifies the impact of each
message or combination on opinion compared
to the Control.

altncs.com

ES

EVOLVI NG
STRATEGIES

MoneyVote ® A fo°'- Mr Rlgei

Modeling Voter Behavior in a Survey

A fool, Mr. Edgeworth, is one who has never made an experiment.

— Erasmus Darwin

How do you turn a vote preference into an action?

Your vote is a strange thing. It is valuable, but you can’t spend
it on yourself; you can only give it away. Sometimes people give
their vote to a candidate because they like and want to help him.
And a lot of the time, people give their vote to a candidate
because they dislike the other candidate and want to hurt him.

A vote is a real action, something of value that voters use to
help or harm candidates. But asking a respondent who they would
vote for if the election was held today is a very abstract thing . . .
there is little, if anything, of value attached to their answer.

So how can we make an early “vote” real for survey
respondents? How can we put respondents in a decision
framework where their response will really help or harm one of
the candidates?

Taking inspiration from experiments in behavioral economics, we
developed a game that is analogous to a real-life vote decision.

We gave each respondent $100 to give to Barrack Obama or
Mitt Romney candidate. But that’s not all; they could also choose
to take away $100 from one of the candidates. And if they were
really uncertain, they could choose to let the money go to the
default winner of the survey.

This makes an abstract early “vote” concrete by attaching real
value to their answer in the survey. It allows us to better capture
who someone will vote for, or against, and what motivates their
vote.

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

— Erasmus Darwin

Presidential Vote - Measured by the MoneyVote (

Impact on the MoneyVote ® - All Respondents

The Obama ads have a significant impact on vote preference,
increasing Obama's vote and decreasing Romney's. The Romney
ads aren’t effective in shifting preferences.

Obama’s ads increase his vote by about 6 points on
average from 42% to 48% - a 15% bump in
Obama's vote.

They decrease Romney’s vote by about 8 points from
47% to 39% - an 18% slide in Romney's vote.

Obama ads increase the percentage of marginal undecided and
wavering voters who prefer him. And this impact seems to hold
when respondents watch both Romney and Obama ads together.

Even when respondents watch ads from both sides,

Obama maintains his gains with 48% of the vote to
Romney’s 41%.

N=2,384/Sept. 28-Oct. 3. 2012 qualtncscom
ES All Ad Impacts on Vote
(weak partisans & pure independents by gender) 100%
* = statistically significant impact
All Respondents I
47% 49% 48% —*
42% 39% —* 41%
0%
Control Rep Ads Dem Ads All Ads

N=2.384/Sept. 28-Oct. 3. 2012 qualtrics.™
ES 100% All Ad Impacts on Vote (weak partisans & pure independents by gender) * = statistically significant impact 100%
Women _ Obama Vote Romney Vote Men ~ 0bama Vote 1V1CU _ RomneyVote
48% 49% 48% 47% 47%
39% 40% 42% ★ 38% ■"41%
0% Control Rep Ads Dem Ads All Ads Control Rep Ads Dem Ads 0% All Ads

Impact on the MoneyVote ® - by Gender

The ads have a fairly consistent impact on men and women,
although there is some indication in the individual ads that
men might respond more to appeals from Romney.

The Romney vote shifts to a net advantage,

47% to 42%, when men watch a Romney ad,
but the movement isn’t statistically significant.

The Obama ads appear to impact the male
Romney vote but not the male Obama vote.

The Obama ads appear to impact the female
Obama vote more than the male Obama vote.

The overall picture here is one of relative consistency
overall, with men and women exposed to ads from both sides
going for Obama by a 6 point margin.

a

CS.com

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

Voter Enthusiasm

- Erasmus Darwin

Impact on Enthusiasm - All Respondents

While the Obama ads win hands down in shifting voter
preferences, both the Romney and Obama ads are effective at
increasing enthusiasm among likely Romney voters (McCain voters from
2008). The Obama ads don’t significantly impact the enthusiasm of
’08 Obama voters.

Romney and Obama ads increase the highest level of
voter enthusiasm of ’08 McCain voters by 13 points
- a 42% surge in the number of McCain ’08 voters
who are extremely enthusiastic to vote this year.

Enthusiasm for ’08 Obama voters remains flat.

Romney ads impact marginal, decided voters,
increasing the likelihood that voters who say they
prefer him will actually turn out to vote.

The enthusiasm impact of the ads overall are very
similar for men and women. Specific ads, however,
impact men and women very differently.

N=1,742/Sept. 28-Oct. 3, 2012

All Ad Impacts on Enthusiasm

(weak partisans & pure independents by gender)

* = statistically significant impact

100%

All Respondents

Obama '08 Voters w/ High Enthusiasm
McCain '08 Voters

Control

Enthusiasm, “Likely Voters,”Polls & Turnout

Voter Enthusiasm Thermometer

Now we'd like to know how
enthusiastic you are about voting in
the election this fall on a scale of 0 to
10.

Zero means you aren't enthusiastic at
all to vote, and 10 means that you are
extremely enthusiastic to vote. A 5
means you're somewhere in the
middle.

Don't think too much about it . . . just
go with your gut feeling and move the
thermometer to the level that feels
right to you.

Most polls are now using "likely voter" screens for their results. What
does that mean? It means that which candidate registered voters in
general prefer doesn't matter . . . you have to prefer a candidate, and say
that you are very likely to vote and/or are enthusiastic to vote this fall.

These "likely voter" polls, in other words, are capturing shifts in voter
enthusiasm in addition to any shifts in actual vote preference. For
instance, if people who prefer Romney are much more enthusiastic
about voting and people people who prefer Obama are less enthusiastic,
that means more Romney "voters" will get counted because they are
now "likely voters" and fewer Obama "voters" will be counted.

In other words, you can have a significant shift in the polls with very
little shift in underlying voter preference for the candidates. And you can
have a shift in the underlying voter preference without a change in the
likely outcome if those citizens are unlikely to turn out.

I I

a

CS.com

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

Medicare Ad Impacts

- Erasmus Darwin

Medicare Theme - MoneyVote ® Impact

While exposure to the Obama ad alone significantly boosts
Obama’s vote, the effect on Romney’s share is muted and the
combined Romney plus Obama ad condition brings the vote to
parity. Romney’s vote sags about for men who watch the Obama
ad, but the drop is not statistically significant.

& Obama’s ad increases his vote by about 10 points on
average from 39% to 49%. The impact on
Romney’s vote is less dramatic and not significant.

□ Although not statistically significant, the Romney
Medicare ad does seem to shift female voters away
from Obama and toward Romney.

□ In general, women seem to move more than men in
response to both candidate’s Medicare ads.

N=2,384/Sept. 28-Oct. 3, 2012 qualtrics,™
E S 100% Medicare Ad Impacts on Vote (weak partisans & pure independents by gender) * = statistically significant impact 100%
XAfniYipTi Obama Vote vvumrn _ RomneyVote Men ~ 0bama Vote iviciL Romneyvote
48% —49% 0 47% 46%
39%" — 45% 42%
0% Control Rep Ad Dem Ad Both Control Rep Ad Dem Ad 0% Both

Medicare Theme - Enthusiasm Impact

N=1,742/Sept. 28-Oct. 3, 2012 Medicare Ad Impacts on Enthusiasm (weak partisans & pure independents by gender) * = statistically significant impact qualtrics™
ES 100% 100%
Women — Obama ’08 Voters w/ High Enthusiasm — McCain '08 Voters Men — Obama ’08 Voters w/ High Enthusiasm McCain '08 Voters
32%^. 51% 46% 45%
\ """'39%
28% 31 % 00/0
0% Control Rep Ad Dem Ad Both Control Rep Ad Dem Ad 0% Both

The Medicare ads have a fairly strong and variable impact
on enthusiasm to vote for both men and women and Obama/
McCain ’08 voters. Both the Romney and Obama ads appear
to boost enthusiasm amongfemale Obama ’08 voters, but have
variable effects on male Obama voters. Among McCain ’08
voters, men seem to be motivated by exposure to both Obama
and Romney ads. Female McCain ’08 voters appear to get a
boost from both the Obama and Romney ads, but the
combined effect of watching both is a wash.

& The Obama Medicare ad increases high
enthusiasm among female McCain ’08 voters
by 24 points - an 85% increase in the highest
level of enthusiasm.

O Although not statistically significant, exposure
to both ads seems to increase enthusiasm
among female Obama ’08 voters and male
McCain ’08 voters.

CS.com

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

Economic Attack Ad Impacts

- Erasmus Darwin

Economic Attack Theme - MoneyVote ® Impact

Exposure to the Obama Economic attack ad alone significantly
boosts Obama’s vote and soften’s Romney’s. And this effect is
maintained when voters are exposed to both candidates ads at the
same time. Men seem relatively responsive to both ads alone.

Obama’s ad increases his female vote and decreases
Romney’s by about 10 points.

□ The Romney Economic attack ad appears
ineffective with women.

O Although not statistically significant, the Romney
Economic attack ad does seem to shift male voters
away from Obama and toward Romney by about 7
or 8 points.

□ In general, men seem more responsive to the
Romney ad, and women to the Obama ad.

ES

100%

N=2,384/Sept. 28-Oct. 3, 2012

Econ Attack Ad Impacts on Vote

(weak partisans & pure independents by gender)

* = statistically significant impact

qualtncscc

100%

48%

Women ~ °bama^

— Romney Vote

49%

Mm ~ 0bamaVote

ivj.cn _ Romney vote

52%

39%

39%

38%



0%

0%

Control Rep Ad Dem Ad Both Control Rep Ad Dem Ad Both

N=1,742/Sept. 28-Oct. 3, 2012 Econ Attack Ad Impacts on Enthusiasm '&C (paltriest*
ES
100% (weak partisans <3? pure independents by gender) * = statistically significant impact 100%
Women — Obama '08 Voters w/ High Enthusiasm — McCain '08 Voters Men — Obama '08 Voters w/ High Enthusiasm McCain '08 Voters
52%
-J9% 469^^ 40%
32V: 28% 33% 33% 35%
0% Control Rep Ad Dem Ad Both Control Rep Ad Dem Ad 0% Both

Economic Attack Theme - Enthusiasm Impact

The Economic attack ads in general appear to decrease
enthusiasm among male Obama ’08 voters and increase
enthusiasm for McCain ’08 voters. Female McCain ’08 voters
seem quite responsive to the ads, with increasing enthusiasm in
response to the Romney ad and a slight, but insignificant sag
after exposure to the Obama ad.

The Romney Economic attack ad increases
high enthusiasm among female McCain ’08
voters by 25 points - almost a 90% increase in
the highest level of enthusiasm.

□ Although not statistically significant, exposure
to both ads seems to shift the male McCain ’08
voter enthusiasm gap with Obama voters from
an 11-point deficit to a 5-point advantage.

I I

a

CS.com

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

Candidate “Plan” Ad Impacts

- Erasmus Darwin

Candidate “Plan” Theme - MoneyVote® Impact

The “Plan” ads in combination with Romney’s
Economic attack ad boosts t high enthusiasm
among female McCain ’08 voters by 36 points -
an almost 130% increase in the highest level of
enthusiasm.

Obama’s minute-long “Plan” ad seems quite effective at shifting
the vote his way when seen alone. Men again shift more on their
Romney vote than their Obama vote. Watching both “Plan” ads in
addition to Romney’s Economic attack ad causes a substantial
increase in Obama’s female vote and drop in the female Romney
vote, but results in a wash with men.

The Romney “Plan” ad and the combined “Plan” ads plus
the Romney Economic attack ad have a huge, positive impact
on female McCain ’08 voter enthusiasm. While the impact of
the ads on men doesn’t reach statistical significance, the “Plan”
ad conditions do seem to boost male McCain ’08 voter
enthusiasm and sap male Obama ’08 voter enthusiasm.

The Romney “Plan” ad increases high
enthusiasm among female McCain ’08 voters by
20 points - an over 70% increase in the highest
level of enthusiasm.

100%

N=2,384/Sept. 28-Oct. 3, 2012 W qualtrics™

Candidate “Plan” Ad Impacts on Vote

(weak partisans & pure independents by gender)

★ = statistically significant impact 100%

Women ~ 0bamaVote
vvutium _ RomneyVote

Men ~ °bamaVote
ivj.cn RomneyVote

0% 0%
Control Rep Ad Dem Ad Both+Econ Control Rep Ad Dem Ad Both+Econ

Obama’s ad increases his female vote and decreases
Romney’s by about 10 points. Obama’s ad also
decreases the male Romney vote about 10 points.

The “Plan” ads in combination with Romney’s
Economic attack ad boosts the female Obama vote
12 points and turns a 9-point deficit with women
into a 9-point advantage.

The “Plan” ads in combination with Romney’s
Economic attack ad appears ineffective with men.

N=1,742/Sept. 28-Oct. 3, 2012

H qualtricsu

Candidate “Plan” Theme - Enthusiasm Impact

100%

Candidate “Plan” Ad Impacts on Enthusiasm

(weak partisans & pure independents by gender)

* = statistically significant impact

100%

Women

— Obama '08 Voters w/ High Enthusiasm

— McCain '08 Voters 64%

Men

— Obama '08 Voters w/ High Enthusiasm

— McCain ’08 Voters

36% 33%

0% 0%
Control Rep Ad Dem Ad Both+Econ Control Rep Ad Dem Ad Both+Econ

a

CS.com

EVOLVI NG
STRATEGIES

A fool, Mr. Edgeworth, is one who has never made an experiment.

AFP “Disappointed” Ad Impacts

- Erasmus Darwin

AFP “Disappointed” Theme - MoneyVote ® Impact

Although none of the Romney ads appear to have a significant
positive impact on Romney’s share of the vote or a significant
negative impact on Obama’s, the AFP “Disappointed” ad appears
to have a possibly negative impact on Romney’s relative standing
with women. This widely-aired ad, in which Obama voters express
their disappointment with the President, consistently won focus-
groups as an ad that would shift swing voters and appeal to
women. The ad, however, doesn’t seem to move votes at all.

□ Although the impacts are not statistically
significant, Obama’s 9-point deficit in the Control
condition becomes a 1-point advantage when
women watch the AFP “Disappointed” ad.

□ Overall, this ad appears to be, at the least,
ineffective for Romney and, at worst, a net negative
with the female vote.

ES

100%

N=2,384/Sept. 28-Oct. 3, 2012

H qualtricsu

AFP “Disappointed” Ad Impact on Vote

(weak partisans & pure independents by gender)

* = statistically significant impact

100%

WTnrnpr) ' Obama Vote
vvumrn _ RomneyVote

48%

39%

0%

Control

Mm “ 0bamaVote

iv±tn _ Romney Vote

Rep Ad Control

0%
Rep Ad

N=1,742/Sept. 28-Oct. 3, 2012 ^

“Disappointed” Ad Impacts on Enthusiasm

(weak partisans & pure independents by gender)

^ * = statistically significant impact

Women Men

— Obama ’08 Voters w/ High Enthusiasm — Obama '08 Voters w/ High Enthu!

— McCain’08 Voters — McCain’08 Voters

0%

Control

Control

AFP “Disappointed” Theme - Enthusiasm Impact

While the “Disappointed” ad fails to shift voters, it hugely
increases enthusiasm among men who voted for McCain in ’08.
Surprisingly, the “Disappointed” ad is terrible as a soft-edged
appeal to swing voters, but seems to be very effective red-meat for
male voters in Romney’s base.

0 The Disappointed” ad increases high enthusiasm
among male McCain ’08 voters by 31 points - a
nearly 95% increase in the highest level of
enthusiasm.

The Disappointed” ad decreases high enthusiasm
among male Obama ’08 voters by 21 points - a
decline of more than 45% in the highest level of
enthusiasm.

I I

a

CS.com

Highligther

Un-highlight all Un-highlight selectionu Highlight selectionh