Politics and Public Opinion

Is Intrade really that useful?

I don’t want to start a war with my AEI colleagues, specifically Intrade junkies Mark Perry and Jim Pethokoukis, but can I offer a small bit of skepticism about Intrade?

I understand why prediction markets are interesting. But am I the only one who thinks they are incredibly overblown? On any given day, some friend of mine will blog or tweet or otherwise opine about how Mitt Romney is now at X on Intrade or how Newt Gingrich now has a 29.3 percent chance of Y on Intrade. I am always at a loss about how much, if at all, I should care about this information.

From what I can tell, the “prices” for shares in political candidates have been all over the place over the last year. So how predictive are they, really? It seems to me they don’t really measure the likelihood of anything so much as the prevalence of certain aspects of conventional wisdom. It’s a clever way to poll people in a given moment, not some ingenious new mechanism for gleaning the future.

When I complain about Intrade to some of my Intrade-obsessive friends, they say that the numbers change because the facts on the ground change. And in the end, the accuracy is great. Well, first of all, isn’t that true of conventional wisdom, pundits, polls, etc. too? In the end, everyone’s accuracy is great. The closer you get to an actual event, the more ironclad the predictions that that event will occur become. Predictions that your plane will crash in a giant fireball decrease precipitously once the plane’s wheels safely hit the ground, and they drop to zero when the plane parks at the gate.

It reminds me a bit of that scene from Fletch when Chevy Chase pretends to know someone who died.

Dr. Joseph Dolan: You know, it’s a shame about Ed.

Fletch: Oh, it was. Yeah, it was really a shame. To go so suddenly like that.

Dr. Joseph Dolan: He was dying for years.

Fletch: Sure, but… the end was very… very sudden.

Dr. Joseph Dolan: He was in intensive care for eight weeks.

Fletch: Yeah, but I mean the very end, when he actually died. That was extremely sudden.

At the very end, when Mitt Romney, Newt Gingrich, or Rick Santorum actually wins the Republican nomination, Intrade will predict that outcome perfectly. Until then, it’s just another kind of focus group.

Or am I wrong? What is the metric that proves the value of Intrade? I am open to correction on all of this.

Editor’s note: See Jim Pethokoukis’s response here.

21 thoughts on “Is Intrade really that useful?

  1. Nope. Occasionally it’s gone against the CW and been right (at least in non-political arenas), but it mapped pretty closely to the polls in 2008.

    Electoralmap dotNET has some examinations of InTrade vs. polls and election results.

  2. Well yeah it is over rated. How much money is traded on intrade? Is it possible to steer the results with just one big trade? Even the stockmarket, which is so much larger and more representative of the “wisdom of crowds” gets short term issues wrong all the time. Intrade is useful as a gut check close to an election, but so are the latest polls. As a tool of prediction it is roughly correct, but in a close election not that much help.

  3. Interesting, but not predictive. If I were going to place money on Intrade, it would be on Obama winning. That’s because I’d be hedging my financial risk (if Obama wins, my Intrade gains could offset my higher taxes, assuming there is still a functioning society).

    “Prediction Markets” are no more predictions than futures markets in, say, commodities. It’s not a forecast. It’s today’s price for a future uncertain event, which weighs all the differing motivations and costs of a future payout. But unlike futures markets, Intrade trade depth is very shallow.

  4. You are completely right. And I dispute that it even reflects the facts on the ground. From late August, the fundamentals of the race were set. In September, a useful prediction market would have shown a hike in Gingrich’s price before the Gingrich surge actually happened, based on his strong performance in the debates, etc. Investors only need to calculate that the price will go up in the future (not that the candidate will actually win in the end) in order to make it a smart buy, and an intelligent person in early September could have predicted that at some point the price for Gingrich would likely go up. But it flat-lined, and it flat-lined until the polls showed him surging.

    Intrade does not predict the polls. It reflects them.

  5. I see a few key benefits of prediction markets. First, they provide for a quantitative probability of an event occurring (e.g. 90% chance Romney will be nominee vs “Romney is the frontrunner”). Second, they provide a collective market assessment (aggregating public opinion, focus groups, punditry). Finally, they are meant to provide insight into future events, rather than just a snapshot (the reason Herman Cain never rose above 10% despite leading in the polls).

    It’s worth noting that futures markets are not meant to be entirely predictive of what will actually happen. Just because contract A is at 80% doesn’t mean Intrade is “wrong” if Contract A doesn’t happen.

    The right metric for the value of Intrade is to see among all contracts, at all points in time, whether the contract’s probability is the actual likelihood that the event eventually happens. I think studies have shown that it’s generally in line, but I may be wrong.

    How much value you put in Intrade is obviously up to you – it’s the difference between reading a bunch of sports columnists and checking the betting line on a football game to try to predict the outcome of the Super Bowl.

  6. It’s like your GPS. It predicts the time of arrival, which just adjusts as you go. No matter what happens on your ride, by the time you get there, lo and behold, the estimated time matches the current time.

  7. I think that for an event like elections it’s nothing more than a poll. It’s just more exciting because the poll is constantly being taken. I’m also skeptical of the trading volume as being a weakness.

  8. Well I do think it does do tell you something useful, but the one thing I think is worth mentioning every time is that it appears that the people doing the real buying and selling aren’t Americans. You can’t send them any money with an American Credit Card.

  9. I don’t know how well it predicts things, but it serves as a good gut check for me. Thinking you’re sure that event ‘X’ will happen is one thing. Pulling out your wallet and putting money on it, is something else.

  10. I do think that you’re probably missing the point. As long as there’s sufficient volume, your theory that Intrade is inaccurate is basically a theory that we’re all leaving a bunch of free money on the table – that we should be buying/selling shares of Mitt Romney until the price is no longer out of whack. This doesn’t make sense.

    As for your Fletch arguments, those are really just arguments against the straw man that Intrade is great “at the end” of the election. The real argument in favor of Intrade is much stronger. and goes like this: If Mitt Romney is trading at 88 cents to win the nomination right now, that means that given all publicly available information, the odds of him winning are roughly 88%.

    A percentage is just a reflection of the world’s uncertainty about something. You might think the odds of a man flipping a coin heads is 50%. Because he knows it’s a weighted coin, he thinks the odds are 80%. Someone with a supercomputer tracking the coin’s trajectory might think the odds are 100%. None of you are wrong. If we had a crystal ball, all Intrade predictions would be 0% or 100%. If you and I had spent the last 50 years on a desert island, we’d have to call the “Gingrich/Romney” odds 50/50, as we’d know nothing about either guy.

    Does the fact that it has only imperfect information “until the end” make Intrade worthless? Not in my mind, as long as it’s by far the best, easiest way to understand what’s most likely to happen. I knew long before the latest polls came out that Romney was a big favorite tomorrow. because I was able to get the best information from those who 1) knew the most and 2) had bet accordingly.

  11. Generally, the biggest help intrade (and similar markets like it) provide is when the external information about a given topic is not immediately clear. If there were to be a series of contradictory polls on a given race, and the context of the race suggests some ambiguity, then Intrade would provide an excellent metric with which to evaluate the likelihood of a given event. Since it is consumer driven, it has tremendous ability to weigh the volatility of a given metric and weigh them accordingly, but it will not tell you things that are beyond the bounds of conventional wisdom. It does what it does exceedingly well, especially when tracked over time, but it is not going to replace or even properly supplant polling.

  12. Intrade is the best prediction system so far designed. That doesn’t mean it doesn’t suck at prediction, just that it sucks marginally less than anything else. If anyone found something better, they would arbitrage it and the prices will adjust. In this sense, it pays analysts for being correct rather than for selling ad copy or stroking the preconceived notions of their audience.

    That said, some things are just plain unknown. Others (“black swan events”) are too improbable for markets to form around any one of them, but the universe of possible black swans is so large that that it is virtually certain that some of them will occur.

  13. I see your point. I was hoping Intrade would have better long range predictive ability. It seems accurate for the short term, but inadequate as a long range tool. I guess what it really shows is that humans are pretty good at determining things that will take place this week, but not so great at figuring out how things will play out months from now.

    I used to be a weather forecaster. I was great at predicting the weather for the next 72 hours, but after that my accuracy went down considerably. I feel Intrade has some similarities to weather forecasting.

  14. It may have mapped the polls in ’08 but on election day in ’04 when the uncorrected exit poll results were leaked in the afternoon, Kerry soared into the ’90s and Bush dropped toward zero. By evening, when it became clearer that the leaked info was faulty Bush soared and Kerry dropped like a stone.

    It’s a reflection of the news. Like a 5th grade history test, spitting out what’s been drilled in.

  15. Like any other market, the point isn’t really whether it’s perfect, but whether or not it’s better than anything else out there. When you’re talking predictions, there’s always going to be a high error rate the farther out you go, because there are still so many “unknown unknowns.” This is true of stocks, commodities, and political predictions. However, just as the market will, over time, beat out the individual stock-picker, the market tends to reflect the aggregated wisdom of many people with small, discrete bits of information, none of which they could individually use to be as accurate, that through the pricing mechanism reach a better conclusion. Of course, one set of sources of important data for these markets are the polls themselves, so it’s probably not true that intrade renders them obsolete…

  16. You are correct. I watched “Romney to win SC” and “Gingrich to win SC” (these are not actual trade names, but summaries) on Intrade very closely, checking several times daily, in the final week before the SC primary. Both numbers fluctuated wildly and were useless until about the day before the event. Similarly, “Obama to win in November 2012″ has hovered around 50% quite steadily, but spiked to the high 70s for about two days after OBL was killed. Then it came right back down.

    Oh, look! A squirrel!

  17. Intrade is an efficient aggregation of information available at any given time. It is not a predictor of events so much as the best possible expression of an incentivized group’s understanding of an event’s probable outcome.

    When people stake money on an outcome, they are more likely to render an accurate assessment based on facts than they are to dash off an unstudied and superficial opinion. This is something different than “conventional wisdom.” It is closer to “financial wisdom.” Which isn’t always accurate, but it is more accurate than the combined bloviations of costless opinionating.

    On a related matter, the substitution of reportage with polling is the death of the republic.

    Now, if you’ll excuse me, my car just hit a water buffalo.

  18. Rather than speculate, read the University of Iowa papers concerning prediction markets vs. polls. Also read up on the cone of uncertainty to understand why no forecasting method can be “accurate.”

  19. To judge the accuracy of a prediction mechanism, test it against competing mechanisms. Feed the same information to several different systems (e.g., Intrade and polls), and then see which one turns out to be right more often.

    In contrast, you suggest that your method of judging Intrade has been to look only at Intrade’s numbers, relative the outcome. But since the available information is always changing, Intrade’s numbers should be changing all the time too. (Paraphrasing Keynes: What do *you* do when the facts change?)

    So make the right comparison: Intrade versus others, given the same info.

  20. How does Intrade do against other prediction mechanisms (including polls), if they are all given the same information? That’s the question to ask. The fact that Intrade’s numbers move a lot has nothing to do with it. Compare Intrade to other polls, not to itself.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>