I'm indebted to the Felix Salmon blog on Reuters alerting me to this paper by Robert Hodgson. Not only is it very interesting but also covers a topic of some importance in the wine world and really should send a shudder through those who believe they have superior palates. I've had my doubts for some time. I quote at length from: An Analysis of the Concordance Among 13 U.S. Wine Competitions; Robert T. Hodgson; Journal of Wine Economics, Vol. 4, No. 1, 1-9.

The abstract reads: "An analysis of over 4000 wines entered in 13 U.S. wine competitions shows little concordance among the venues in awarding Gold medals. Of the 2,440 wines entered in more than three competitions, 47 percent received Gold medals, but 84 percent of these same wines also received no award in another competition. Thus, many wines that are viewed as extraordinarily good at some competitions are viewed as below average at others. An analysis of the number of Gold medals received in multiple competitions indicates that the probability of winning a Gold medal at one competition is stochastically independent of the probability of receiving a Gold at another competition, indicating that winning a Gold medal is greatly influenced by chance alone."

Some other major points are:

"The benefit of this expense is the belief by wineries that entry fees offer a valid return on investment: Gold medals sell wine. However a recent article in Wine Business Monthly (Thach, 2008) conducted as a joint effort by 10 global universities with specialties in wine business and marketing found that consumers are not particularly motivated by medals when purchasing wine in retail stores. Perhaps consumers are beginning to realize, as discussed below, that winning gold medals may be more a matter of chance than a predictor of quality."

"This supports the opinion of Hodgson (2008) that wine judges concur in what they do not like, but are uncertain about what they do."

"Thus, 98 percent of Gold medal wines were regarded as just above average or below in another competition."

"Thus one is led to the conclusion that a wine's performance in one competition is not Correlated with its performance in another."

"For the 375 wines entered in five competitions, one would expect by chance alone (for p = 0.09), 234 wines receiving no Golds, 116 receiving a Gold in just one competition, 23 receiving Golds in two competitions, two receiving Golds in three competitions and no wine receiving Golds in more than three competitions. The observed frequencies closely mirror these numbers."

"That is, winning a Gold medal at B does not depend on whether or not the wine won a Gold medal at A. This seems to be the case as will be shown below, and supports the conclusion that chance alone may account for the number of Gold medals that a wine receives. By contrast, if wine competitions were a reliable predictor of wine quality, one might expect, say, 10 percent of the wines to be ranked consistently superior. For the case of five competitions described above, one might expect to see 30 to 40 consistent Gold medal winners (which was not observed)."

"An examination of the results of 13 U.S. wine competitions shows that (1) there is almost no consensus among the 13 wine competitions regarding wine quality, (2) for wines receiving a Gold medal in one or more competitions, it is very likely that the same wine received no award at another, (3) the likelihood of receiving a Gold medal can be statistically explained by chance alone."

This is pretty damming stuff as much of the edifice of the wine business is based a swirling, sniffing and spitting and proclaiming which wines are better. We would expect if you enter a show you would put the best foot forward so the poorer wines presumably do not get a start. So these results show that of the pool entered it's particularly hard to say which are the best wines.

Now I'm not arguing that above a basic level all wines are as good as each other; only that tasting panels are not nearly as good as they think they are. There is simply no other way to view this result.