Dueling Kansas Gov Polls: who to Believe

There are now two recent polls with very contrasting results out on the Kansas Governors race. A poll from Survey USA showed the Democratic team of Paul Davis and Jill Docking with a 48-40 lead over incumbent Republican Governor Sam Brownback and Lt. Governor Jeff Coyler. Then, on Sunday a poll done by British YouGov polling firm for the New York Times and CBS News showed Brownback with 47 to 37 lead, and including leaners at 52 to 40 lead.

Naturally, this has prompted some questions. How can such divergent results, around a 20 percent swing, be explained? Which poll is most reliable?

The first thing I noticed is that the YouGov poll didn't include Libertarian candidate Keen Umbehr who draws about 5 percent of the vote in the Survey USA poll.

The NYT/CBS/YouGov poll is part of massive national poll of every Gubernatorial and Senate race in every state, with over 100,000 taking part.

Taniel at DailyKos notes

In 2012, the margin of YouGov's final polls favored Republicans by an average 5 points, including large errors in competitive races in Nevada, Wisconsin, Connecticut, and Virginia—all in the GOP's favor.

YouGov doesn't use the same random-digit dialing use by Survey USA and traditional pollsters. Instead they use a panel of respondents, who aren't random selected. In a scientifically valid poll every voter should have an equal chance of being randomly selected, making the sample representative.

Nate Cohn of the New York Times observes:

 YouGov attempts to build a large, diverse panel and then match its panelists to demographically similar respondents from the American Community Survey, an extremely rigorous probability survey conducted by the Census Bureau. This step is intended to mimic probability sampling. But it can require significant assumptions about the composition of the electorate, including partisanship. These assumptions are contestable and based on varying amounts of evidence.

Some groups tend to be underrepresented in web panels: the less educated, the less affluent, Hispanics and those over age 65. You Gov then blows up the under represented groups. So those in the subgroups in the panel, if they have unrepresentative views can skew the entire poll.

Comparing Internals/Crosstabs

Let's peer underneath the topline to understand the differences and see if there are signs that one or another poll is implausible.

 Black voters. One of the more implausible results in the YouGov poll is its claim that black voters prefer Davis by only a 53-46 margin.  There is no "race" breakdown in the latest Survey USA poll, but a November 2013 had Davis up by 65-21 margin.  I think the YouGov result  is implausible and quite likely that Davis-Docking will carry the black vote by a bigger margin.

If blacks are five percent of the electorate, the difference between a 50-50 split and a 90-10 split would be about 4 percent, about one-third of YouGov's Brownback edge.

By the way, YouGov interviewed only 31 black out of 1274, or 2.4% of the panel. In contrast blacks are estimated to have been 5 percent of the 2012 electorate.

YouGov also interviewed only 25 Hispanics.  

Women voters. Incredibly, the YouGov poll has Brownback leading Davis 50-41, while Survey USA has Davis up 51 to 35.

A news report on an exit poll for the Kansas 2012 said that among women Obama was almost equal, while 69 percent of men favored Romney. That is a gender gap of about 20 points in 2012, compared to a YouGov gender gap of 6 and a Survey USA of 13 points.

Republicans.  YouGov, in contrast to Survey USA has Brownback doing exceptionally well with Republican voters grabbing 82 percent to only 9 for Davis. In contrast, Survey USA has Davis getting an impressive 29 percent of Republican voters, compared to only 60 percent for Brownback. It may just be a coincidence, but the Survey USA poll shows "some dudete" symbolic candidate Jennifer Winn drawing 30 percent in the Republican primary.

Independents.  YouGov shows this segment, breaking narrowly for Brownback 46 to 43, while Survey USA has Davis leading 48 to 27. (Independents are 36% of the YG panel and 24% of Survey USA.)

Senior Voters.  YouGov has Brownback leading 57-39, Survey USA has Davis on top 56-36. (65+ voters are  projected to be about 29% of voters.)

Summing up: YouGov has a historical +5 GOP bias, excluding the Libertarian candidate likely gives another +5 bias for Brownback.  Adding in the very suspicious cross tabs for blacks (and Hispanics) and women, not to mention the other categories, makes a strong case that the Survey USA poll is closer to the truth and that the Davis-Docking team is very well positioned to win in November.