Username:    Password:
Thanks for stopping by...

On Tasting
John Moody and Brave or Foolish Judging by Winestate September/October 2013 : Part 2.
Monday, 21st July, 2014  - David Farmer

In Part 1, I discussed how the judging or grading of wine is fraught with difficulties. I then pondered if a better system was possible and found encouragement in the methods of John Moody in his rating of financial paper.

To restate the problem in a different way, consider two teams tasting the same set of wines in different rooms at the same time. What we know is they will come up with different results. The results will diverge further if the wine classes do not have a narrow focus; and for example included wines from all countries, many varieties and many vintages.

In Part 1 mention was made that knowing such things as the region, the provenance, the lineage or the length of history of the wine, its ability to age and the maker, would give an extra dimension to the tasting and such factors need exploring.

Great wines have taken their position in the wine hierarchy after countless assessments by the market place over many years and such considerations cannot be dismissed.

So let us return to the Winestate tasting. 'World's Greatest Syrah and Shiraz Tasting', of 582 wines.

I have been reading Winestate for decades and know its style pretty well. I confess though that despite the process of judging and scoring wines being clearly set out I have always thought something was missing, no doubt because of my sceptical view of most things.

Winestate say: " ..the wines are not know to judges. The three judges taste the wines blind and assign a score without reference to each other. Only then do they compare scores, and if there is dissension they re-taste the wines and come to an agreement. Scores are compiled using the 20 point international system... These final 'medals' are then converted into a star rating system... A gold medal means 5 stars, silver is 4 stars and bronze is three stars."

The star rating also includes 3.5 stars and 4.5 stars, which in my view mean it is a ten point scoring method. It is worth remembering that Winestate is a buying guide and to my knowledge they do not review wines with scores lower than three stars, presumably because these are not worth drinking.

It worries me that judges use the 20 point system to make assessments and this is converted backwards into the published five star system. Doesn't it make more sense to judge from the start using stars, or publish the score out of 20.

It is also stated: "Wine judging is an inexact art, not a science-even at the highest levels of proficiency. Accordingly, Winestate uses the star rating system which reflects a range, rather than a specific point score. Point systems indicate a level of accuracy that simply does not exist."

Then to clarify a star versus a point score, Winestate produce a table which shows for example a 4 star wine equals, 17 -17.9/20 which becomes 94-95/100.

Here is the Winestate comparison table.

This triggers another worry. Scaling up a 20 point system to 100 points means 4 stars equalling 17-17.9 should become the range 85-90. As I understand it the Winestate comparison table illustrates how the stars compare to magazines which use the 100 point method. Thus four stars undergo a correction factor and become 94-95.

Further clues about the judging process were set out in the Winestate editorial by Peter Simic.

"Of course the problem we face at Winestate, and the reason why our scores tend to be lower, is that we use the International 20 points judging system with three judges tasting each wine blind. Then we use the Winestate \'majority rules\' system where rather than averaging scores, two judges have to recommend the wine and the closest two scores go through. So, for example, if two judges give a wine 15/20 and the third gives it 18 points it is out because two judges have said it is out, rather than averaging up."...

I see no logic in the 'majority rules' method which does at least confirm my suspicion that the results instead of being fearless are massaged. Why not average the three tasters, and where is the case that this 'majority rules' leads to more accurate judging? It strikes me as the opposite as if the judges are equal how can one judge be dismissed even if that score is at a large variant to the others. [Note: Similar pressure or correction is used in Show Judging as well.]

All this confirms is that stars and numbers are a useful buying guide but customers must remain wary. I also have the view that for the five stars rating to offer 'a range, rather than a specific point score' the divisions of 3.5 and 4.5 stars should be excluded as the divisions between 15.5 and 20.00 are not stepped evenly when they are re-calibrated to stars.

The editorial also says:

..."At the higher priced level it is like 'taking the Rolls out on a dirt track for a spin', ...without giving extra points for the provenance of history and reputation. But of course we cannot make allowances for some iconic brands when all our wines are judged blind."

I have already offered caution that not taking into account the pedigree of the best wines can lead to very odd results and I often remark that at the top level it is not the wine being judged so much as the judges themselves.

So a total of 17 different judges worked their way through 582 wines over 6 days. The large number of judges also worries me and adds another random factor to the final result. For example did each panel taste a random selection across all price points or did they know the price category they were judging. We are not told.

My difficulty over decades of reading Winestate is a simple one and revolves around the wine price. Surely a four star wine at a low price cannot be better than a three star wine selling for say ten times the price, yet we are told this is not so.

So what do we find? A few bargains were unearthed at the low price end with 4.5 stars going to Brookland Valley, Taylors Promised Land and Red Knot McLaren Vale while 4 stars were given to Shot in the Dark, Johnny Q, Wolf Blass Red Label and a few others.

Going back to the Winestate table I read 4.5 stars equates to 18-18.4 and 96-97 on the 100 point scale. These are indeed bargains, but what room is left for the far more expensive great wines of this challenge?

So I turn to the eight wines priced over $200 to find two with 5 stars, one with 4 stars and 5 with 3 stars. The 3 star bracket caught some beauties including Guigal (France), Penfolds Grange 2008 and Hill of Grace 2008.

So as a consumer I am expected to believe that Grange, which the Wine Advocate for example gave 100/100, is a lesser wine than a humble Wolf Blass Red Label.

As Winestate has said all wines are judged equally I graphed the results.

Much of the towering edifice of the wine business is built on the basic idea that expensive wines will taste better. Thus the graph should rise as the wines increase in price.

Thankfully I am saved from having to offer a detailed view of the results as my experience allows me to take the easy path by saying I do not believe the results of this tasting.

I turned back to John Moody to see what he might have advised. Moody charged the investors of financial paper for his company's opinions; advice worth paying for if it helps in avoiding losses. Built into the grading is another useful feature as it can assist in deciding your appetite for risk. In 1970 Moody's joined similar firms in also charging the issuers of financial paper for their helpful ratings.

To walk both sides of the street is very hard and I am ever mindful that Moody's with a bunch of others had some role to play in the recent GFC debacle. Collecting money from the issuer and the investor it seems can lead to the corruption of ratings.

Thus I will be very careful about how I use Moody in thinking about wine tasting.

I have the increasing suspicion that some large International tastings are beginning to walk both sides of the street. Tastings have evolved from a service to wine makers, to being offered for a small charge to assist consumers, to the current vision of a global event that can be a useful money maker, with fees being gathered to enter while charging for the use of the results.

Winestate of course has always been scrupulously fair with their assessments but whether the results of this Shiraz tasting are helpful is what this article is about.

I recall that the Winestate results came out about the time Wine Australia ran its Savour Australia 2013 programme with guest arriving from overseas to listen to experts explaining why they should buy Australian wine. I wonder if any visitors stopped to think why two of our most famous wines, Penfolds Grange and Hill of Grace, had put up such a miserable showing in the country's premier wine magazine.

I have wondered for a long time about the point of large omnibus wine tastings and those saying 'we assess all wines equally and masked', seems to offer no better approach than that of the marketing genius Robert Parker and others who knew it made sense at the top end to know what they were tasting.

As you move from agricultural shows which helped instruct amateur makers how to avoid faults to shows becoming part of marketing so they must constantly evolve. I have also learnt that having forthright opinions on wines can leave you badly exposed and to stare down the market place is very risky.

Keeping things in proportion is often hard to do and I do this by reminding myself that wine is just a drink.

La Tâche 1962 and Grand Old Fake Bottles

Tuesday, 19th December, 2017

How You Can Drink Smarter

Monday, 28th August, 2017

Thoughts on Wine and Alcohol

Monday, 28th August, 2017

Which is Better - Young Wine or Aged?

Wednesday, 27th August, 2014

John Moody and Brave or Foolish Judging by Winestate September/October 2013 : Part 2.

Monday, 21st July, 2014

My Replies to Frequently Asked Questions

Sunday, 22nd June, 2014

John Moody and Brave or Foolish Judging by Winestate September/October 2013 : Part 1

Thursday, 1st May, 2014

Tastings Wine Scores and Shapley Values

Thursday, 27th March, 2014

Views on Tasting with Penfolds Notes for the Record

Thursday, 27th February, 2014

Broken Glasses, Tasting Cups, Chinese Misadventures and Tailgating

Thursday, 8th August, 2013

A Wine for Richard

Monday, 5th August, 2013

Morrisons - Never Fail Method for Buying Wine

Tuesday, 9th July, 2013

Writing Tasting Notes about Great Wine

Wednesday, 2nd May, 2013

Is a Tasting Note Helpful when Shopping?

Wednesday, 24th April, 2013

Do Tasting Notes Have any Value?

Friday, 22nd March, 2013

The Five Secrets of Stylish Women Applied to Drinking Wine

Tuesday, 12th February, 2013

Playing the 'Hit Me' Wine Game

Monday, 10th December, 2012

A Response to an eBay Customer About Barossa Cabernet

Monday, 27th August, 2012

Starting a Cellar and General Thoughts on Ageing Wines - The Experience of a Hardcore Wine Collector

Monday, 20th August, 2012

Starting a Cellar and General Thoughts on Ageing Wines

Wednesday, 18th July, 2012

Do Tastings Protect the Brand Image of Wine Australia?

Tuesday, 17th April, 2012

Do the Waitrose Taste Test

Sunday, 26th February, 2012

Wine Writers Misleading Customers

Tuesday, 10th January, 2012

Steps in Becoming a Wine Snob

Saturday, 10th December, 2011

A Morning at the Mount Barker WA Wine Show

Sunday, 20th November, 2011

Natural Flavours from Wine and Beer Yeasts

Friday, 26th August, 2011

Status in St Emilion Raises Issues

Sunday, 24th July, 2011

What Should You Pay for Great Wine?

Sunday, 15th May, 2011

Gordon Ramsay Says "Consult the Sommelier" - Is He Right?

Tuesday, 13th July, 2010

Driven to Drink - A Reflection by Frank Devine

Friday, 25th June, 2010

The John Vickery Leo Buring Tasting 1997

Monday, 24th May 2010

Tasting Budget Reds in London

Tuesday, 27th April 2010

A Rosemount Story from Tesco UK

Tuesday, 23rd March, 2010

Descriptions of Marlborough Sauvignon Blanc from Unexpected Sources

Tuesday, 23rd March, 2010

Sadat X Amusing? Yes. But Not Yet a Challenger to Robert Parker

Thursday, 18th February, 2010

Tasting the Latest Noon Wines

Friday, 5th February, 2010

The Barossa Wine Show Public Tasting 19th September
Part 3 of the Barossa Tastings

Monday, 4th January, 2010

The Power of Gold

Wednesday, 25th November, 2009

First Write the Review then Taste the Wine

Wednesday, 25th November, 2009

The Artisans of the Barossa 12th September
Part 2 of the Barossa Tastings

Friday, 30th October, 2009

The Whites do go Better With Fish

Friday, 30th October, 2009

A Parker Tasting Should Make Us Pause

Tuesday, 13th October, 2009

Why You Drink Champagne by the Sea

Thursday, 8th October, 2009

Tasting at Three Barossa Wine Shows
Part 1 - The Small Winemakers Show

Sunday, 4th October, 2009

It's In The Bubbles, Stupid

Friday, 2nd October, 2009

How Good is Your Palate?

Monday, 21st September, 2009

Another Know All Wine Wanker

Monday, 21st September, 2009

Further Thoughts on the Australian Wine Show System

Thursday, 3rd September, 2009

Wine Advice from Craigslist

Thursday, 3rd September, 2009

Comparing Wine Types and Dog Breeds

Saturday, 1st August, 2009

Electronic Experts Give Wine Advice

Monday, 15th June, 2009

In Search of a Holy Grail

Monday, 5th May, 2009

The Gimblett Gravels Travelling Road Show Surprises London

Monday, 20th April, 2009

The Beauty of the Five Point Judging System

Thursday, 26th March, 2009

Ideas for a New Wine Judging Scale

Thursday, 12th March, 2009

The First Growth Gimbletts

Monday, 26th January, 2009

The Story of One Lonely Bottle

Friday, 26th December, 2008

The Future of Capital City Wine Shows

Friday, 5th December, 2008

The More You Spend the Better the Taste

Friday, 2nd May, 2008

The Key to Tasting Wines

Friday, 11th April, 2008

Buying Wines That Have a Sense of Place

Friday, 11th April, 2008

What Are We to Make Of Wine Tasting Notes

Monday, 4th February, 2007

Study of the Label Alters Our Tastes

Sunday, 9th December, 2007

One in Five Cannot Smell Alpha-ylangene

Monday, 24th September, 2007

Charles Must Love It

Friday, 24th August, 2007

Liking Rough Red

Thursday, 28th June, 2007

The Ultimate Masked Bottle

Firday, 24th May, 2007

Create the Atmosphere for the Tasting Result You Want

Monday, 19th March, 2007

Confessions of Wine Deceit, Other Deceptions and Fooling Yourself

Monday, 29th January, 2007

Two Sides of the Wine Coin

Sunday, 12th November, 2006

Wine Labels, Ranking, Ratings, Experts and Wine Prices

Friday, 27th October, 2006

A Story About Verdelho

Wednesday, 4th October, 2006

The Magnetic Appeal of Wine

Sunday, 24th September, 2006

Geology Cannot be Found In Wine

Thursday, 18th September, 2006

More on the Impact of Scoring Wines Out of 100

Thursday, 14th September, 2006

How a Three Star Wine Became a 90 Point Wine

Saturday, 9th September, 2006

High Alcohol - The Debate Rolls On

Tuesday, 5th September, 2006

So You Suffer from Anosmia

Monday, 7th August, 2006

Canít Tell Your Brett from Oak Flavours

Tuesday, 18th July, 2006

Terroir - Can It Possible Shine Through the Background Noise

Tuesday, 4th July, 2006

A Little Spat Over Chateau Pavie

Thursday, 15th June, 2006

After All This Time - Now They Tell Us

Tuesday, 13th June, 2006

Winemaking and the French Touch

Friday, 9th June, 2006

The Judgement of Paris Part II

Monday, 29th May, 2006

Why Do Chardonnays Seldom Win Trophies?

Friday, 5th May, 2006

The Other French Paradox

Tuesday, 2nd May, 2006

Show Me the Chardonnay Tasting

Sunday, 30th April, 2006

Judging Wine and the View of the American 'Wine Spectator'

Thursday, 6th April, 2006

The World Versus Robert M. Parker, Part Three

Saturday, 18th March, 2006

Halliday v Parker Continues to Attract International Attention

Thursday, 5th January, 2005

"Thank You Mr. Evans and Sorry Mr. Parker"

Thursday, 8th December, 2005

The Differences of Opinion Continue: Mount Mary Quintets vs. Robert M. Parker Jr. Part 2

Monday, 5th December, 2005

Sharp Differences of Opinion Over Mount Mary

Monday, 14th November, 2005

The Views of the Economists With a Sense of Humour

Friday, 14th October, 2005

Wine Quality: Does Terroir Matter?

Friday, 14th October, 2005

Thoughts on Wine Judging

Thursday, 13th October, 2005

What You See Affects What You Smell and Taste

Tuesday, 13th September, 2005

The New Taste of Wine

Friday, 2nd September, 2005

Consumers Disagree with Wine Experts

Saturday, 27th August, 2005

Drinking From Special Wine Glasses

Thursday, 4th August, 2005

Now, This Wine Drinks Well, Night After Night

Monday, 18th April, 2005

Corks, Stelvin Caps and Oxygen

Wednesday, 6th April 2005

A Peep Behind the Wine Show Door

Thursday, 17 March 2005

Going To Your Second Wine Tasting

Saturday, 11th December, 2004

What You Bring to Your First Wine Tasting

Saturday, 4th December, 2004

What Do Show Medals Mean - Part 2

Tuesday, 7th December, 2004

What Do Show Medals Mean - Part 1

Friday, 3rd September, 2004

©2019 Glug  |  Contact Us  |  Privacy Policy  |   RSS Feed
Liquor Licensing Act 1997: It is an offence to sell or supply liquor to a person under the age of 18 years, or to obtain liquor on behalf of a person under the age of 18 years.
All transactions in $AUD. This web site is operated by Glug Management Company Pty Ltd ABN: 64 116 647 780 Licence No: 57701982