15 Responses to “Polling Averages”

1. kitstolz, on July 16th, 2008 at 11:03 pm Said:

   Arguing the latest numbers on Obama is enjoyable, I think.

   A contrasting view would point out that Obama’s lead is outside the margin of error in Iowa, Michigan, Minnesota, Wisconsin, and of course Illinois. People who should know Obama, like Obama.

   Meanwhile “red states” such as Missouri, Florida, and Virginia are neck-and-neck. Obama is tied with McCain in Virginia, and is making a huge push for the state, with the total support of the popular Democratic leadership.

   And to toss aside the LA Times poll and the Newsweek poll as somehow beneath inclusion is arbitrary. The latest Newsweek poll is much closer to the average of about 3.5 percent point lead for Obama, but who is to say that this snapshot of the national electorate is right and that one was not?

   Not to mention pollster George Barna, whose very experienced firm frequently polls for religious organizations, and gave Obama a 15% lead shortly before July 4th.

   http://www.barna.org/FlexPage.aspx?Page=BarnaUpdateNarrowPreview&BarnaUpdateID=300
   (Hope that link is live!) Here’s another one, from my interview with Barna:
   http://achangeinthewind.typepad.com/achangeinthewind/2008/07/ventura-pollste.html

2. Daniel Koffler, on July 16th, 2008 at 11:18 pm Said:

   Polling averages tend to be less precise but more accurate than individual polls (there’s frequently a tradeoff between accuracy and precision). That’s also demonstrably true. The Newsweek and LAT/Bloomberg polls are methodologically suspected by the McCain campaign, but the grounds for that suspicion is suspect. (We’re really shooting in the dark about party-ID breakdowns and effects and turnout models, even more so than usual this year.)

   It’s at best misleading to make anything of leads in polls “disappearing” and “reappearing” on a day-to-day basis. Sure, it’s possible that those shifts are significant of daily shifts in voter preferences, but overwhelmingly more likely (and militated for by Occam’s razor) that nothing at all changed in that period, and the poll had simply captured statistical noise. That’s why polling averages are useful. They help discriminate between real shifts in opinion and noise. Writing an article about fluctuations in a single daily tracking poll is a good sign of an innumerate hack, and cherrypicking a single poll that conforms to one’s biases before any verifiable trend can be adduced is breathtakingly mendacious — i.e., exactly what you’d expect from Dick Morris. True, we shouldn’t impute more errors to him than he makes on his own, but an epistemology that extends Morris the benefit of the doubt we are accustomed to extend to reasonable and honest people is bound to lead to error.

   You’re right to object to treating the RCP average as canonical, though, since despite the fact that it does correct somewhat for inaccuracy in individual polls, it’s a crude, error-ridden first stab at doing so. Fortunately, there’s a much more sophisticated 2.0 version of what RCP is trying to do, namely FiveThirtyEight’s popular vote and electoral vote projectiions, based on not just on taking the arithmetic mean of incommensurate data, but on a fairly rigorous assignment of weights to various polls, demographic analysis of states on a district by district basis, nearest-neighbor analysis of states and districts (since information about a state or district gives us information about states and districts that closely resemble it), etc. As with any polling analysis, FiveThirtyEight’s method is imperfect. But it’s orders of magnitude more deserving of canonical authority than anything else in the general sphere of political opinion polling.

   What does it show? It shows Obama winning the national popular vote by something in the neighborhood of 4 points and clearing 300 electoral votes since the general election campaign began. His numbers dipped slightly, reflecting the tightening Gallup and Rasmussen showed last week, and should tick back up with the latest tranche of polls showing high single digit leads. The point being that Obama’s lead has been consistent and stable since the general election began.

3. Grumpy Old Man, on July 16th, 2008 at 11:26 pm Said:

   Obama’s lead did narrow, and in the polls I tend to trust more (Rasmussen and Gallup), and then it widened a bit.

   On the popular vote polls the gap has remained quite narrow, with BHO in the lead, and BHO leads a bit in most electoral vote projections.

   Unless McCain comes up with something remarkable, the election is Obama’s to lose. As long as Jennifer Rubin and Abe Greenwald keep trashing him, I will confess to a strange new respect for the man.

   And no, a toe fetish does NOT disqualify one from the punditocracy.

4. conradg, on July 17th, 2008 at 6:44 am Said:

   Mostly, what DK said. I’d definitely second the idea that you rigorously study http://www.fivethirtyeight.com/ for a better understanding of the statistical realities behind these polls, and how to analyze them. Most journalists and Byzantine scholars are understandably not very good at math, and could use a little guidance. Unfortunately, most are uninterested in learning, so big misunderstandings tend to get spread across the board and aren’t easily cleared up.

   BTW, it’s still far from clear that the big leads Newsweek put up early were false, or that the current narrowing is true. Voter models are in great flux right now, and results always vary, which is why there is such a thing as “margin of error”. Also, tracking polls have their own biases which are hard to eliminate.

5. John Schwenkler, on July 17th, 2008 at 9:17 am Said:

   Sheesh, conradg - do you think you could be a little more snide there?

6. Daniel Larison, on July 17th, 2008 at 10:04 am Said:

   It’s true that I’m not especially good at math, but I do have some understanding of statistics. As for what “most Byzantine scholars” can or cannot do, I’m fairly confident that you have no idea. If you could actually bother to point out what I have gotten seriously wrong, that might aid in the learning process in which I allegedly have no interest.

   The Newsweek and LAT polls were not “false”–they really did find respondents who said the things they reported. However, they were probably unrepresentative of the state of the presidential race at that time. Once they had a sample that was much more like others in terms of party ID, their results were much more in line with what everyone else has been finding.

   There’s a margin of error? Really? Thanks for the tip!

7. Daniel Larison, on July 17th, 2008 at 10:28 am Said:

   Thanks for the useful correction, Mr. Koffler. Clearly, I hadn’t thought about this nearly enough. I will keep your post foremost in my mind should I have something else to say about polling.

8. conradg, on July 17th, 2008 at 2:54 pm Said:

   Daniel,

   First, you would benefit by taking yourself a little less seriously, If you want to be known as someone who dishes out harsh and unrelenting criticism, you have to welcome the same in return. If you can’t take the heat, get out of the blogosphere.

   Second, DK does a great job of pointing out the problems with your analysis, which is just at the junior high school level, to be honest, and there’s so much wrong with it the best I can recommend is that you read assiduously through sources like 538 and study the general literature on the subject. I’m sure there are some Byzantine scholars with good statistical math skills, but clearly it’s not a job requirement, and it’s also clear you are deeply lacking in those skills by both your own admission and demonstration. If you want to get on a high horse, don’t use a pony.

9. Daniel Koffler, on July 17th, 2008 at 3:47 pm Said:

   Ah, don’t mention it Daniel. I wouldn’t have spent this much time on this if I weren’t a big admirer of your stuff in general and cheering as loud as anybody can from the sidelines for your side to win the conservative civil war. You’ll have a huge leg up if you guys get good at math. Have you read Commentary much recently? I’ll lay odds for anybody willing to bet that Jennifer Rubin can process calculations of numbers an order of magnitude larger than quantities she can use parts of her body to count.

10. Daniel Koffler, on July 17th, 2008 at 4:01 pm Said:

    Oh one other thing, I rarely give myself any practice writing a bunch of jokes into my posts, which means the risk of telling a dud or being misunderstood is amplified. Just to be clear, the target of my snark is primarily Morris and the kind of journalism typified by CNN and the AP

11. Daniel Larison, on July 17th, 2008 at 4:03 pm Said:

    I was not in the best mood when I was reading comments earlier today, and so I responded intemperately. Generally, I try not to take myself seriously, but I also have a short temper, which was even worse earlier, so I apologise for my curt remarks. I took a couple statistics classes in college, but have rarely made use of them since then. In fact, I do understand how I went wrong here and would have avoided the blunders had I not been posting in haste.

    Thanks, Mr. Koffler. I’ll keep that in mind.

12. Daniel Larison, on July 17th, 2008 at 4:13 pm Said:

    I understood what you were trying to do, and I agree with your critique of this sort of journalism. A fairly good example of this sort of thing was when Newsweek changed the composition of their sample, came up with dramatically different results and then tried to explain–in a confused voice–how the race could have changed so quickly.

13. c-joe, on July 17th, 2008 at 4:16 pm Said:

    The Gallop and Rass. polls are weighted by party ID, so their extreme stability in always showing the race somewhere between tied and Obama by 7 is only reflected changes within parties, not shifts to one party or another.

    An interesting point I learned at 538 is that Obama has a serious chance of winning the electoral vote but not the popular vote because of population shifts to Red States from 2000-2008 are not yet reflected in the electoral college. In other words, if redistricting were dynamic, Bush’s 2004 states would have gained several electoral votes.

    Also, a perfect tie would go to the House and result in Obama winning.

    So even if national polls are perfectly accurate, McCain would need to be at about +1.5 to actually win.

14. c-joe, on July 17th, 2008 at 4:22 pm Said:

    I also have a pretty strong gut feeling that black turnout this election is not being accounted for.

    Typically blacks vote at the white rate in the deep south, but well below the white rate in several swing states such as Ohio. I think it will be higher than the white rate virtually everywhere in 2008.

    How many of these black voters don’t get polled at all because they only have cell phones, or fail to pass the “likely-voter” screen because they have never voted before? A lot will not even pass the registered voter screen because they aren’t registered yet but will be by the election, or don’t remember (”not sure” answerers to “are you registered at your current address” get booted.)

15. conradg, on July 18th, 2008 at 8:42 am Said:

    Daniel,

    No big deal about your anger. It’s endearing, really.

Leave a Reply

You must be logged in to post a comment.