Tastings Wine Scores and Shapley Values
Thursday, 27th March, 2014
- David Farmer
We all approach assessing wine in our own way but mention that all wines in a group must be given a score and the background dynamic changes. A score means that a tally can be done to identify the best wine and alarmingly the worst wine. No one wants to be seen as a supporter of the worse wines so how do you head off this embarrassing possibility?
One way is to not go out on a limb, which means you do not suggest any wine is better than any other. In the early 1980s the National Wine Show in Canberra introduced the idea of judges from overseas joining the panel to add diversity of opinion. To keep an eye on this for a few years I used to peruse their scores to see how they saw things next to local judges. A famous Italian turned up one year and I admired his technique which was to point all wines in every class half a point either side of 15½. You do not get the applause of a star but you also develop no enemies.
If you join a tasting group you will have a lot of fun but at some point it will get serious, as it should, and scores will be called for. What is amazing, and this is often not apparent from discussion, is the wide spread of scores. You can be certain that any tasting of a dozen wines by a dozen tasters will produce a result where the favourites of some are the least liked by others.
The opinions of professionals also diverge widely, though perhaps not to the extent of amateur circles, and normally a lot of talking happens in the background to align scores.
The great marketing genius Robert Parker knew that exposure to masked tastings should be kept to a minimum and normally knew what he was tasting. This of course reduces risk which is an excellent idea for staying on the top.
Working out the best wine in a tasting group is generally done by adding up the scores and dividing by the number of tasters. What happens is that the extreme highs and lows, the diversity of opinion, is compressed and the final result normally shows many wines clustered together with only a small numerical difference. Naturally there is a best wine but the least liked is usually not far behind.
This may lead to heated debate particularly if your most favoured is turned into an also ran. After you have done this for decades you might also wonder if the results are a true indicator of the best wines and perhaps ponder the statistical validity of this approach.
Researchers have begun pondering the statistics behind tasting and I found this paper of interest:
"Increasingly, economists and cultural critics are arguing that wine tasting is junk science. This column argues that the problem with wine tasting lies not with the impossibility to consistently tell a good wine from a bad wine but with how the wines are ranked. If a new system of game-theory-based rating and ranking using the Shapley value were implemented, wine tasting might get a better hearing from its critics". Ranking wines using the Shapley value; Victor Ginsburgh, Israel Zang, 1st August, 2013.
They go on to say:
"Some judges are generous, others are less so, and the ranges of marks used by the judges may vary as well." As noted by Ashenfelter and Quandt (1999), this "may give greater weight to judges who put a great deal of scatter in their numerical scores and thus express strong preferences by numerical differences".
They nominate a simple scoring system:
"We suggest a new game theory-based rating and ranking method for wines, in which the Shapley value of each wine is computed, and wines are ranked according to their Shapley values."
This is simple and easy to apply. Each taster has one vote worth one. You can assign your one vote to your top few wines or assign it to a single wine. Thus if you give your single vote to your top five wines they each get 0.20 points.
A tally is made and the top wines of the tasting are ranked.
Whether this would work better than the numerical rating or the ranking method now used I do not know. Certainly the idea of not giving any score to the wines you dislike has merit. As the authors say this method needs to be tried over many tastings to see whether the results make more sense.
What I do know is that judging and pointing wine is probably little more than an approximation. The big men like Len Evans and Robert Parker who can sweep aside inbuilt prejudice to see what the rest do not are few and far between.
|
|
Which is Better - Young Wine or Aged?Wednesday, 27th August, 2014
John Moody and Brave or Foolish Judging by Winestate September/October 2013 : Part 2.Monday, 21st July, 2014
My Replies to Frequently Asked QuestionsSunday, 22nd June, 2014
John Moody and Brave or Foolish Judging by Winestate September/October 2013 : Part 1Thursday, 1st May, 2014
Tastings Wine Scores and Shapley ValuesThursday, 27th March, 2014
Views on Tasting with Penfolds Notes for the RecordThursday, 27th February, 2014
Broken Glasses, Tasting Cups, Chinese Misadventures and TailgatingThursday, 8th August, 2013
A Wine for RichardMonday, 5th August, 2013
Morrisons - Never Fail Method for Buying WineTuesday, 9th July, 2013
Writing Tasting Notes about Great WineWednesday, 2nd May, 2013
Is a Tasting Note Helpful when Shopping?Wednesday, 24th April, 2013
Do Tasting Notes Have any Value?Friday, 22nd March, 2013
The Five Secrets of Stylish Women Applied to Drinking WineTuesday, 12th February, 2013
Playing the 'Hit Me' Wine GameMonday, 10th December, 2012
A Response to an eBay Customer About Barossa CabernetMonday, 27th August, 2012
Starting a Cellar and General Thoughts on Ageing Wines - The Experience of a Hardcore Wine CollectorMonday, 20th August, 2012
Starting a Cellar and General Thoughts on Ageing WinesWednesday, 18th July, 2012
Do Tastings Protect the Brand Image of Wine Australia? Tuesday, 17th April, 2012
Do the Waitrose Taste TestSunday, 26th February, 2012
Wine Writers Misleading CustomersTuesday, 10th January, 2012
Steps in Becoming a Wine SnobSaturday, 10th December, 2011
A Morning at the Mount Barker WA Wine ShowSunday, 20th November, 2011
Natural Flavours from Wine and Beer YeastsFriday, 26th August, 2011
Status in St Emilion Raises IssuesSunday, 24th July, 2011
What Should You Pay for Great Wine?Sunday, 15th May, 2011
Gordon Ramsay Says "Consult the Sommelier" - Is He Right? Tuesday, 13th July, 2010
Driven to Drink - A Reflection by Frank DevineFriday, 25th June, 2010
The John Vickery Leo Buring Tasting 1997Monday, 24th May 2010
Tasting Budget Reds in LondonTuesday, 27th April 2010
A Rosemount Story from Tesco UKTuesday, 23rd March, 2010
Descriptions of Marlborough Sauvignon Blanc from Unexpected SourcesTuesday, 23rd March, 2010
Sadat X Amusing? Yes. But Not Yet a Challenger to Robert ParkerThursday, 18th February, 2010
Tasting the Latest Noon WinesFriday, 5th February, 2010
The Barossa Wine Show Public Tasting 19th September
Part 3 of the Barossa Tastings
Monday, 4th January, 2010
The Power of GoldWednesday, 25th November, 2009
First Write the Review then Taste the WineWednesday, 25th November, 2009
The Artisans of the Barossa 12th September Part 2 of the Barossa TastingsFriday, 30th October, 2009
The Whites do go Better With FishFriday, 30th October, 2009
A Parker Tasting Should Make Us PauseTuesday, 13th October, 2009
Why You Drink Champagne by the SeaThursday, 8th October, 2009
Tasting at Three Barossa Wine Shows
Part 1 - The Small Winemakers Show
Sunday, 4th October, 2009
It's In The Bubbles, StupidFriday, 2nd October, 2009
How Good is Your Palate?Monday, 21st September, 2009
Another Know All Wine WankerMonday, 21st September, 2009
Further Thoughts on the Australian Wine Show SystemThursday, 3rd September, 2009
Wine Advice from CraigslistThursday, 3rd September, 2009
Comparing Wine Types and Dog BreedsSaturday, 1st August, 2009
Electronic Experts Give Wine AdviceMonday, 15th June, 2009
In Search of a Holy GrailMonday, 5th May, 2009
The Gimblett Gravels Travelling Road Show Surprises LondonMonday, 20th April, 2009
The Beauty of the Five Point Judging SystemThursday, 26th March, 2009
Ideas for a New Wine Judging ScaleThursday, 12th March, 2009
The First Growth GimblettsMonday, 26th January, 2009
The Story of One Lonely BottleFriday, 26th December, 2008
The Future of Capital City Wine ShowsFriday, 5th December, 2008
The More You Spend the Better the TasteFriday, 2nd May, 2008
The Key to Tasting WinesFriday, 11th April, 2008
Buying Wines That Have a Sense of PlaceFriday, 11th April, 2008
What Are We to Make Of Wine Tasting NotesMonday, 4th February, 2007
Study of the Label Alters Our TastesSunday, 9th December, 2007
One in Five Cannot Smell Alpha-ylangeneMonday, 24th September, 2007
Charles Must Love ItFriday, 24th August, 2007
Liking Rough RedThursday, 28th June, 2007
The Ultimate Masked BottleFirday, 24th May, 2007
Create the Atmosphere for the Tasting Result You WantMonday, 19th March, 2007
Confessions of Wine Deceit, Other Deceptions and Fooling YourselfMonday, 29th January, 2007
Two Sides of the Wine CoinSunday, 12th November, 2006
Wine Labels, Ranking, Ratings, Experts and Wine PricesFriday, 27th October, 2006
A Story About VerdelhoWednesday, 4th October, 2006
The Magnetic Appeal of WineSunday, 24th September, 2006
Geology Cannot be Found In WineThursday, 18th September, 2006
More on the Impact of Scoring Wines Out of 100Thursday, 14th September, 2006
How a Three Star Wine Became a 90 Point WineSaturday, 9th September, 2006
High Alcohol - The Debate Rolls OnTuesday, 5th September, 2006
So You Suffer from AnosmiaMonday, 7th August, 2006
Can’t Tell Your Brett from Oak FlavoursTuesday, 18th July, 2006
Terroir - Can It Possible Shine Through the Background NoiseTuesday, 4th July, 2006
A Little Spat Over Chateau PavieThursday, 15th June, 2006
After All This Time - Now They Tell UsTuesday, 13th June, 2006
Winemaking and the French TouchFriday, 9th June, 2006
The Judgement of Paris Part IIMonday, 29th May, 2006
Why Do Chardonnays Seldom Win Trophies?Friday, 5th May, 2006
The Other French ParadoxTuesday, 2nd May, 2006
Show Me the Chardonnay TastingSunday, 30th April, 2006
Judging Wine and the View of the American 'Wine Spectator'Thursday, 6th April, 2006
The World Versus Robert M. Parker, Part ThreeSaturday, 18th March, 2006
Halliday v Parker Continues to Attract International AttentionThursday, 5th January, 2005
"Thank You Mr. Evans and Sorry Mr. Parker"Thursday, 8th December, 2005
The Differences of Opinion Continue: Mount Mary Quintets vs. Robert M. Parker Jr. Part 2Monday, 5th December, 2005
Sharp Differences of Opinion Over Mount MaryMonday, 14th November, 2005
The Views of the Economists With a Sense of HumourFriday, 14th October, 2005
Wine Quality: Does Terroir Matter?Friday, 14th October, 2005
Thoughts on Wine JudgingThursday, 13th October, 2005
What You See Affects What You Smell and TasteTuesday, 13th September, 2005
The New Taste of WineFriday, 2nd September, 2005
Consumers Disagree with Wine ExpertsSaturday, 27th August, 2005
Drinking From Special Wine GlassesThursday, 4th August, 2005
Now, This Wine Drinks Well, Night After NightMonday, 18th April, 2005
Corks, Stelvin Caps and OxygenWednesday, 6th April 2005
A Peep Behind the Wine Show DoorThursday, 17 March 2005
Going To Your Second Wine TastingSaturday, 11th December, 2004
What You Bring to Your First Wine TastingSaturday, 4th December, 2004
What Do Show Medals Mean - Part 2Tuesday, 7th December, 2004
What Do Show Medals Mean - Part 1Friday, 3rd September, 2004
|
|