Right on the money* – Polling in Calgary’s Election

How are we to read the polls with the results? What can the amateur poll reader to look out for next time.

I’m going to look at two polls which built samples from the entire city, and weren’t content to treat the entire city as a single unit.

First we had ThinkHQ. They were in the field from September 13 to 16th. They used an online forum, Angus Reid. (full poll)

There are two things to tease out, and they only come with healthy assumptions. If you don’t agree, let me know!

  1. Samples which ensure wider representation take more effort and the panel might start fill up fast in some areas of the city, but not in others where it may take multiple days to fill the sample.
  2. Events can shake the electorate along the way, but instead of going back to measure again, they can be observed as just regional differences.

I think that is what may have happened here. And we can tease it out on the ThinkHQ page. More time in the field to fill the sample from the NE and Inner City? Well, that is more time after huge events to start shifting the trajectory of the election.

On September 15th we had the Alberta government move forward with vaccine passports, and this viral tweet from Jyoti Gondek.

The reach of this tweet would have been large, and primed the pump for the next main event. With active Twitter accounts in Canada belonging to ¼ of the adult population, I’d hazard that the Tweet, would have been served even to those unengaged in politics by the Twitter algorithm.

Then we have the second tweet, a week later.   

If the first, a week earlier, hadn’t breached non-politicos’ feeds, this one surely would. On an issue people care about. On an issue that is at the height of the issue-attention cycle.

So onto the second poll, from Janet Brown Opinion Research. (full poll)

This poll was in the field from September 22 to 30th (9 days) and partly used an existing panel, and partly a panel recruited for this survey. (by way of comparison, their similarly sized provincial panel in 2019 was 6 days in the field)

There are some interesting bits here. The NE is pretty close to the ThinkHQ Mayoral result. Ok, so maybe confirmation of the time shift? Or is it regional? The NW seems to have disconnected from the SW for Farkas. I’ll take that as residual hangover from the Farkas’ SW Council Seat. But we still have a Gondek bump in the SW, just not as large.

When we get the more detailed results from Elections Calgary, we will see if the SW caught up with the NW, or if the shift to Gondek accelerated generally to get us to the end results.

We can use a bit of math to check on the results of the Janet Brown poll without undecideds (Don’t know/none of the above) – with much apologies for statistical crimes, and comparing them to the election’s actual result.

With a reported margin of error of 3.1%, and the effect of time, this looks pretty good to me. The polling companies which put more effort to ensure all areas of the city were right on the money.

The poll even likely helped with consolidation behind Jyoti Gondek – at least the Gondek campaign thought as much!

They spent between $7000 and $8000 to put the ad below on Calgarians’ screens 600-700 thousand times from October 6th to 12th.

So, next time around, what I will be looking for:

Polls which sample and weight by quadrants, or another measure. The ‘rings’ approach by ThinkHQ is useful too, and for provincial I’d add a third ‘ring’—the CMA but outside of the City limits, given the divergent results we’ve seen when pollsters have split city proper from their ex-urban and true suburbs.

Who are our Mayoral candidates trying to reach?

When using usually scarce advertising dollars, who is being targeted in the run up to the last weekend of the campaign?

Using Facebook’s ad tools, broadly you can target based on geography (down to 1 km radius circles); interests; age; gender identification. In general I would classify those targeting methods as ‘broadcasting’—Facebook will look to serve the ads to people most active on its platform that in the past have shown they watch videos, engage with posts, or click your links if those objectives are selected, or to show the ad to as many people as possible.

For motivating likely supporters to turn-out to the polling stations, more strategic tools are available. A campaign will pivot to deploy its budget strategically to motivate engaged voters, not solely attempt to recruit new supporters. Here Facebook gives you other useful tools to re-target promising leads. You can target based on visitors to your website (server-side event tracking or less reliable these days, a ‘pixel’), email addresses, phone numbers, past interactions with your campaign’s page, or past interactions with advertising.

A well-resourced campaign will run both strategies simultaneously through the pivot, and Facebook with a bit of finessing will work its magic to broadcast to people not yet in your likely supporters’ group while serving different content to the already engaged. This will move people along an engagement spectrum, saving more targeted and potentially polarizing messaging for voters predisposed for your candidate. The last thing you want is for your get out to vote messaging to be more successful at motivating supporters of the other candidate!

These last few weeks we have seen a huge growth in spending by leading Mayoral candidates. The Gondek and Farkas campaigns have been ramping up since October 1st, with October 11th having over $4,000 in spending for Gondek, and over $1,000 for Farkas. Farkas’s spending continued to grow, reaching $2,518 on October 12th. Prior to October 1, the highest spending day belonged to Jan Damery on June 17th, at $594.

Ad Library Summary

Have candidates pivoted from broadcasting?

Jeromy Farkas

The Farkas campaign is solely in broadcast mode. Its various ads target at least 500,000 thousand people. As of October 13th, it is launching new broadcast ads, videos which we’re likely to see as YouTube pre-rolls, and possibly even on broadcast TV. Will the Farkas team launch a narrower targeted get out the vote pivot? It hasn’t run a targeted ad since the end of March, so the campaign might not see the value in engaging with its strongest fans differently than Calgarians as a whole when it is putting $2,500+ a day into its broadcast campaign.

Jyoti Gondek

The Gondek campaign has suspended its broadcast campaign, and hasn’t run ads since October 12th. Having spent $45,000 since September 28th broadcasting, it has likely recruited its fans, and is ready to pivot. Some signs of an early pivot: targeting a smaller subset of Calgarians for lawn signs (second ad here) with more than $16,000 in ad placements. For campaigns who aren’t door knocking, sign locations can be particularly useful as these more engaged voters can be empowered to act as ‘mini-get out the vote captains’ for their personal networks. They’ve committed themselves by getting a sign, surely you can count on them to check in on 5 or 10 friends or family members to get out to vote.

The campaign also spent over $7,000 on what I would call a ‘consolidation’ ad—meant to arm both a targeted group, and wider Calgarians with polling data showing Gondek as the frontrunner. The ad ran in three versions, one sent to a much smaller group which I am going to hazard a guess as being website vistors and previous content engagers. These ads armed supporters going into the Thanksgiving weekend as they talked to their less engaged friends and relatives.

Update:

The Gondek campaign has launched a distinct GOTV ad campaign, one with a wide audience (left), and one targeting likely supporters (right).

In the past the Gondek campaign has run ads like the left to to broad audiences, but without significant money behind them ($1200 in total) – leaving the algorithm to serve the right language to the right person. Expect a video ad for broadcasting to come soon (they’re cheaper to run per impression).

The rest

Jeff Davison has mostly been broadcasting, but his ads are getting swamped due to the relative spend of Farkas (4-1), and Gondek (15-1). The campaign ran one targeted ad with potential reach of less than 50,000 people but more than 10,000. This is likely indicative of an audience of all people who have visited their campaign website, engaged with the Facebook content, or both.

Brad Field is an a similar boat as Jeff Davison. The campaign is broadcasting, but is being overwhelmed to even a greater degree than the Davison campaign. Their last targeted ad was similarly in the less than 50,000 people but more than 10,000 range.

Jan Damery hasn’t advertised on Facebook since October 8th. Prior to this, ads were targeting broad (100-500k), narrow (10-50k) and very narrow groups (1000 and less)

The Pivot away from trying to reach everyone: Still to come

Over the next few days we will likely see the pivot being implemented by leading campaigns, but we aren’t there yet. It will be interesting to see how targeted the campaigns make their advertising as you can target as narrowly as people who have engaged with particularly parts of your website. Read about transit? Get a get out the vote ad about the candidates vision for transit. Visit a page on taxes and stay for 15+ seconds? Receive a tax focused message. Read a message in a language other than english? Get ads in that language!

The possibilities are nearly endless if a campaign wishes to drill down into sub-audiences and setup their website integrations to do so months ago.