A Democratic Pollster In Frank Luntz's Focus Group
Alternative title: a lesson in how not to screen participants and conduct a focus group
Hi, everyone. With the election just a week away, I personally need a distraction from the crushing anxiety and I figure you probably do too. So here’s a story from November 2018 the week after the midterms when I was invited to, and participated in, one of Republican pollster and focus group moderator Frank Luntz’s made-for-TV focus groups. For those of you not familiar, a focus group is when political or market researchers gather a target audience to discuss a particular issue, find out how they talk about it and how they react to various stimuli…or, in groups like the one I went to, have people yell at each other so you can put it on TV later. Below is the exact text of the email I sent to my coworkers on the way home from groups (with a couple of typos fixed), recounting the craziness.
Now, as a professional who’s run some successful focus groups (and some less successful ones), I feel obligated to point out several best practices that were very much not followed here:
When recruiting political focus group participants, always screen out people who work in politics or journalism!
Keep your groups relatively homogenous, so that people will feel comfortable discussing things and not just shout at each other.
Start with light topics before moving into heavy, controversial stuff.
Try to sort out tech issues beforehand so participants don't have to stare at the moderator being mic'ed up.
But that’s enough actual focus group knowledge. Story time!
Gather 'round, everyone, and I'll tell you the story of how a Democratic pollster spent three hours in a Frank Luntz focus group. (If you really want the tl;dr version: that was weird as fuck, I wasn't on TV tonight but it's possible I will be later, Michael Bennet might be running for president, and you're all better at your jobs than Frank Luntz.) I'm probably forgetting some stuff and remembering others out of order, but this is the best I can do to describe what happened.
Just so everyone not in the DC office knows: I received a Facebook ad to participate in a Frank Luntz focus group. I was naturally curious, so I filled it out. For the record, I answered everything honestly, including all partisanship questions (indicating I was a very strong Dem) and, when asked for my occupation, listed it as "Senior Associate, GBA Strategies." There was no question asking me if I worked in politics, market research, or anything of that nature. And, as I mentioned in chat to Brian, no rescreener when I got there. They just had me sign in, sign a confidentiality agreement (sorry, Frank Luntz), and pointed out the snacks and drinks and where to sit while waiting. We're all women, a mix of ages but tilting younger, mostly but not exclusively white.
When they announced it was time, they had all the Trump voters (plus one Gary Johnson voter) stand up from the waiting area and sit on the left side of the room. Then the Clinton voters sat on the other side. There are three cameras, a lot of staffers working the cameras and sound (including one guy who would lower a mic in the direction of whoever was speaking), and a projector at the front. Each seat has a dial. They spend several minutes arranging us and getting camera placement and lighting ready before anything happens.
Andrew Shue stands up at the front of the room to kick things off. (For my fellow millennials, he was an actor on the 90s show Melrose Place - I had to Google him after, and I still don't understand what he was doing there.) He shows us two slides on the projector: one of income growth over the last 40 years for the top 1% vs. everyone else, and one of the various things the United States ranks last in among 25 industrialized nations (finances, obesity, political polarization and excessive campaign spending...there were more but I don't remember them and it generally seemed rather subjective). He also said that this would be like therapy (um...I've done my share of both therapy and focus groups, and they are very different) and that it was important to be brutally honest.
Then Frank Luntz steps in. And that holds everything up several minutes while he has to be mic'ed up. He mentions that groups held last night are airing on Fox News tonight - so maybe the same will happen to us? I hope they tell us if it does. (Note from 2022: I don’t believe we were ever on TV. If we were, they didn’t tell us.)
And we begin! We start off with something light and not at all controversial: "Give me one word or phrase to describe how you feel about the Kavanaugh hearing." We give our words, and predictably, people start shouting at each other as soon as we can all talk at once. Many of the Republicans actually said something negative, but it becomes clear once they go into more detail that their reactions are about what they see as a false accusation, not being upset over what he did. There is one Black woman on the pro-Trump side (don't ask me how they found her) and she and a Black Latina woman on the Clinton side immediately get into an argument - the pro-Trump Black woman said the hearings were great because Kavanaugh fought back, the other woman expressed disgust that she'd say that as a Black woman, and they were off to the races and shouting over each other.
We do our first dial test and it's clips of Susan Collins' speech when she said she'd support Kavanaugh. After the clips, Luntz turns to our side and called us out for all turning down our dials every time she said "presumption of innocence." There's some back and forth about how it's a job interview, not a trial, and he lied under oath. Luntz tries to play gotcha with us, asking if a president who lied under oath and was disbarred (read: Clinton) was also disqualified. People give middling, mostly unsatisfying answers. He then turns to the Trump voters and asks if it's okay that Kavanaugh lied under oath. They make excuses and say Christine Blasey Ford lied too. (Congratulations on proving all partisans are hypocrites!)
There are several clips of Ben Sasse saying anti-Trump things and how we're having the wrong debate in Washington (this made slightly more sense in context, but I forget the context). I bite my tongue and do not say anything about how he's a United States Senator and could be doing something about all of that.
We talk Trump. Our side is asked what we think of him, and when it's very clearly negative, are asked if we give him credit for anything (uh, no). This leads the Republican side to predictably conclude we're all a bunch of jerks. There's some discussion of the economy, and on the Dem side, a couple of people mention getting new jobs recently but still don't feel like Trump deserves credit for the economy. Luntz tries to call us out on this. (Full disclosure: I tried to be quiet in the beginning, but this is where frustration overwhelmed good sense and when he pulled that shit on a woman who'd just been hired at HRC, I shot back "You're trying to give Trump credit for the Human Rights Campaign hiring? Are you kidding me?")
Frank Luntz takes us to task for all the yelling we're doing at each other several times throughout the group. He accuses us of deliberately using words to hurt each other. At one point, a woman who used to be a journalist (great screening, Luntz!) called him out on it, saying he'd separated us by partisanship and was very clearly trying to get us to do this and it's not how we'd react in a different setting. He argues back with her, saying she'd never accept that as an excuse from someone she was interviewing. (For the record, I am Team Former Journalist. They were very clearly putting hardcore partisans on opposite sides in hopes we'd duke it out, despite all their protestations. Not sure if Frank Luntz and Andrew Shue learned anything from that, but it'll probably make for entertaining TV.)
We watch a lot of clips with the dial tester. Mostly uplifting things about the American Dream and how we all need to come together. Hilariously, he used the Melania "your word is your bond" bit she plagiarized from Michelle Obama and then acted surprised when all the Dem dials went down at that line. It is surprisingly heavy on Michael Bennet - two ads about how he'll fix Washington (nobody else had ads in their clips, only speeches) and two clips of him at what I think was the Betsy DeVos hearing talking education. Luntz prompted us after that to ask if education was something we wanted to see talked about in Presidential campaigns. Almost all of us said yes and then we started arguing about low teacher pay vs. unions supposedly protecting bad teachers and how involved the federal government should be in education. There are other clips of likely Democratic candidates for President - I remember Kamala Harris, Cory Booker, and Elizabeth Warren - but also a mix of historical speeches (Barbara Jordan, Geraldine Ferraro, Hillary's Wellesley graduation speech) and people like Oprah, Melania Trump, Michelle Obama, Ben Carson, Sandra Day O'Connor...definitely others I'm forgetting or couldn't recognize.
Luntz spends some time talking to us at the end - the upshot of it is that he's horribly depressed over how divided we seem, and how having people with different viewpoints in your life enriches it, etc. Andrew Shue comes back one last time and rambles at us about how he thinks other countries are moving together in the right direction (he used a metaphor of rowing their boats) while all we do in the U.S. is argue with each other. But he has hope! He thinks we can come together! He wants to gather a group of 700 people to chart a path forward and is interested in us...I don't know what this would entail and don't want to, but apparently they're going to email me about it, great.
And that was that. I chat with a couple women who also work in Dem politics and make the Facebook/LinkedIn connections. I get my $125 and go home.
I don't know what Frank Luntz and Andrew Shue could possibly have learned from all that shouting, but I guess I'm happy to help.
Happy November, everyone, and may all your focus groups be free of shouting.