"Is wine judging a pleasure?"

Thomas P recently posted an excellent and interesting blog entry on his Blog Vinofictions-Is wine judging a pleasure?

http://www.vinofictions.com/

Salute

An interesting read indeed. Before commenting, I must state that because I have so little faith in wine competitions, I very rarely sit on such panels and when I do it is as much to evaluate the judging procedures as it is to evaluate the wines. My response to the question is thus based on the reality that even the critic sittig in isolation in his/her own tasting room is, of course, a "wine judge".

As to whether sitting and evaluating (judging if you like) 10, 50 or more wines is a pleasure - yes and no. Yes in the fact that you may be sampling some very good wines and even though you may give them a mere minute, two or three, thus gaining pleasure wine in its present and predicted future form. Yes also in the fact that every time one sits down to taste and judge, one is enterig into a challenge, that challenge to the critic's "senses and sensibilities" - the ability to focus, to remain accurate, to find that for which one is searching. Indeed, challenges, even when one occasionally fails the test, are a positive part of life and thus enjoyable.

As to the "no" - whether sitting down to judge a few or a great many wines, one is setting into a period of intense focus and concentration and, frankly, sometimes even axhaustig work. I suppose though that even this has its up-side, for hard work wel acomplished gives satisfaction or, if we will pleasure.

I won't sit on competition or other panels but so long as my senses remain intact I would not dream of giving up tasting and finding that activity one of intense pleasure.

Best
Rogov

P.S. Thomas errs, albeit in a minor way in only one thing. When he states that most who serve i the status of being judges at competitions "do it gratis (that’s without pay)", he may have forgotten that many of the most "prestigious" competitions do pay for air fare, hotel costs, a good number of meals and in addition to attending the wine fairs attendant to their competitions also offer gratis trips and winery visits after the fair and competition time. There is a not insignificant number of judges who "perform" like the circuit judges of old, making their way around the world by using competitions as their "jump-off" point for their travels. Not "pay" indeed, but for many hardly altruistic and certainly not without its rewards.

Hi Daniel,
I hear what your saying and I agree with most of it. The only difference is the “wine critic” sitting in isolation in his/her own tasting room could use his/her own “standard, criteria, or philosophy” when it comes to evaluating wines. Now when you get a hundred or more wine critics and wine judges together to evaluate wines at a competition, it’s up to the organizer to get them on “the same page”. Thus the organizers will more or less now set the “standard or criteria” for judging.

Salute

Hi Daniel,
I hear what your saying and I agree with most of it. The only difference is the “wine critic” sitting in isolation in his/her own tasting room could use his/her own “standard, criteria, or philosophy” when it comes to evaluating wines. Now when you get a hundred or more wine critics and wine judges together to evaluate wines at a competition, it’s up to the organizer to get them on “the same page”. Thus the organizers will more or less now set the “standard or criteria” for judging.

Indeed, one of my many problems with competitions by judges - the entire panel system leads to thes statistical phenomenon of regression to the mean, that is to say where the mediocre rises and where the exceptional falls. Not far from the large numbers theory that means that a site containing hundreds of thousands of tasting notes will reflect the"average" tasting note - and the average, as we know is not often enough reflective of the realities.

Best
Rogov

The pleasures of wine competitions are:
1. trying new and distinctive wines from grapes and regions you almost didn't know existed.
2. meeting some very nice people.
3. sometimes a tour is thrown in - of a place you actually wanted to tour.

The non-pleasures are:
1. tasting many really dreadful wines - my own Michigan Wine Competition excepted.
2. tasting way too many wines
3. sometimes being forced to taste wines out of order - i.e. cabernet after concord or sherry.
4. having to taste with judges who are trained on 'sensory evaluation' - read 20 pt scale. (they actually have no idea how good a wine is untill they have totaled up their scores). the urge to maim.

P.S. Thomas errs, albeit in a minor way in only one thing. When he states that most who serve i the status of being judges at competitions "do it gratis (that’s without pay)", he may have forgotten that many of the most "prestigious" competitions do pay for air fare, hotel costs, a good number of meals and in addition to attending the wine fairs attendant to their competitions also offer gratis trips and winery visits after the fair and competition time. There is a not insignificant number of judges who "perform" like the circuit judges of old, making their way around the world by using competitions as their "jump-off" point for their travels. Not "pay" indeed, but for many hardly altruistic and certainly not without its rewards.

As someone who has helped out in a number of wine shows in the 'back room', for every judge there is generally a steward who does the pouring of the wines and delivery of them to the judges table. This crew, in my experience, is almost always on a gratis basis, even those that might travel long distances at their own expense just to get to the venue. These people also do the job because they enjoy it. This is no mean feat. In the last show I walked about 14,000 steps on each of the two days of the show. Without them there is no wine show. Another unpaid crew organises the wines over the previous days and washes the thousands of glasses that are used.

I think we have to distinguish between true international competitions (e.g. VinItaly, London Wine and Spirits, Brussels, VinExpo) and local competitions (e.g. state fairs)

At many of the international competitions there are as many as 150 judges. Those generally sit in panels of seven and indeed one sommelier-in-traiing is assigned to each panel as a "server". Those generallly young people are there as part of their academic training and indeed, withthe exception of meals during the day, are not paid. Those working the "back room" are indeed paid, whether dishwashers, sorters, bottle numbering, whatever. And most certainly those from whatever legal of accounting firm there are paid as well. Worth keeping in mind too that such competitions are far from "charitable events" and those competitions (whether at the most respected venues or at such out-of-the-way places as Tiblisi, Larnaca, or Santiago are business ventures meant to make a profit.

As to more local events - indeed those are done to encourage local wineries, the local wine industry and yes, what some call "the culture of wine" and because of that many people are actually volunteers.

Best
Rogov

Indeed, one of my many problems with competitions by judges - the entire panel system leads to thes statistical phenomenon of regression to the mean, that is to say where the mediocre rises and where the exceptional falls.

That doesn't sound like 'regression to the mean'.

From my understanding, regression to the mean would mean that if you gave a wine 86 points on one day, you might give it 90 points the next day, 89 points the following day, and eventually your scores would regress to a mean.

Otherwise, I'm not sure how a large tasting panel would prefer mediocrity. Unless of course you had a panel full of mediocre judges. (Witness tv ratings). It all depends who is doing the judging.

yes, it all depends on who is doing the judging; and that depends on who invites them. competitions that give lots of gold medals get lots of entries and make lots of money. some invite judges who will give lots of medals. really critical judging is not necessarily the norm.

i judge at a competition which just this past year instituted a "you decide" criteria. It assumes that we've all been tasting wines for many many years and if we like it - for whatever reason - that is as good a reason as there is to give it a medal. because it WILL be good; because it is good now; because it is the best possible of its type? good, knowledgable judges need the freedom to make these choices. i'm against the 'judge it for what it is now' group.

Thomas was up here recently for the NY State Fair tasting as a judge..we chatted briefly but were not able to get together..Looks to me like he tasted a lot of swill and quite a few medals were handed out to wines that can only be described as swill, IMHO..Perhaps that is what Daniel is hinting at re regression..My daughter and I tasted through quite a few of those "medal winning "wines when we visited the wineries and were appalled at how bad they were ..but that is just my 2 cents, your mileage may vary

Otherwise, I'm not sure how a large tasting panel would prefer mediocrity. Unless of course you had a panel full of mediocre judges. (Witness tv ratings). It all depends who is doing the judging.

Rahsaan,
Rating by committee, whether in wine competition or scientific review, almost always favors safe choices devoid of risk over the truly innovative or unusual. Did you ever read the panel tastings of wine in the SF Comical? Their preferred choices are almost always bland and commercial. The reason is that, in a large group, you have a spectrum of tastes. Wines that you and I prefer get tossed out by the majority as "thin and shrilly acidic" whereas big, alcoholic frootbombs get tossed out as either atypical or undrinkable. In both cases, there's a passionate minority that loves those rejected wines, but majority rules, no? So, the group favorites tend to be clean, inoffensive and boring. And that's when the judges are competent at detecting flaws.

Mark Lipton

I think we have to distinguish between true international competitions (e.g. VinItaly, London Wine and Spirits, Brussels, VinExpo) and local competitions (e.g. state fairs)

At many of the international competitions there are as many as 150 judges. Those generally sit in panels of seven and indeed one sommelier-in-traiing is assigned to each panel as a "server". Those generallly young people are there as part of their academic training and indeed, withthe exception of meals during the day, are not paid. Those working the "back room" are indeed paid, whether dishwashers, sorters, bottle numbering, whatever. And most certainly those from whatever legal of accounting firm there are paid as well. Worth keeping in mind too that such competitions are far from "charitable events" and those competitions (whether at the most respected venues or at such out-of-the-way places as Tiblisi, Larnaca, or Santiago are business ventures meant to make a profit.

As to more local events - indeed those are done to encourage local wineries, the local wine industry and yes, what some call "the culture of wine" and because of that many people are actually volunteers.

Best
Rogov

Yes, our competitions are not in the same class as the biggies, having only maybe 25-30 judges. But if they only have one 'server' per team of 7 they must take a very long time to get through the wines that they have to judge. In NZ shows each judge manages to get through about 150 wines per day, plus call backs and trophy judging where applicable.

[quote="Rahsaan]
...That doesn't sound like 'regression to the mean'. From my understanding, regression to the mean would mean that if you gave a wine 86 points on one day, you might give it 90 points the next day, 89 points the following day, and eventually your scores would regress to a mean. [/quote]

Rahasaan, Hi....

The regression to the mean to which I refer involves the fact that for every panel of six or seven judges both the highest and the lowest scores for each wine tasted are discarded when calculating the data. That tendency is compounded even more when two different panels taste the same wine, this meanng that four scores are automatically discarded.


[quote="Neil]
...But if they only have one 'server' per team of 7 they must take a very long time to get through the wines [/quote]

Neil Hi...

At such competitions three minutes is allowed per wine. The wines are bagged and ready in the "back room" for the sommeliers-in-training and the only real "delays" come about when one of the panels calls for a second bottle, having deemed the first corked or otherwise faulted. At the most major competitions of this type wines are tasted sixty at a sitting, generallly in flights of 10-12 wines per flight (the glasses, albeit empty, are already on the table). During a first break the tables are cleared, fresh glasses are set out, etc. Most such events call for anywhere from 120-150wines per day.

Best
Rogov

Every time I read what judging entails, I remember why I love being being an enthusiastic amateur. Charles Bukowski summed it up best in The Red Porsche(truncated version]

it feels good to be driven about in a red porsche
by a woman better- read than I am.
it feels good to be driven about in a red porsche
by a woman who can explain things about classical music to me.

......

I like being driven about in a red porsche
while I smoke cigarettes in gentle languor.
it's better to be driven around in a red porsche
than to own one. the luck of the fool is inviolate.

Rating by committee, whether in wine competition or scientific review, almost always favors safe choices devoid of risk over the truly innovative or unusual. Did you ever read the panel tastings of wine in the SF Comical? Their preferred choices are almost always bland and commercial. The reason is that, in a large group, you have a spectrum of tastes. Wines that you and I prefer get tossed out by the majority as "thin and shrilly acidic" whereas big, alcoholic frootbombs get tossed out as either atypical or undrinkable. In both cases, there's a passionate minority that loves those rejected wines, but majority rules, no? So, the group favorites tend to be clean, inoffensive and boring. And that's when the judges are competent at detecting flaws.

I don't think this is just about numbers and committee review but about the composition of the committee and their incentives. For scientific review, I can sort of see how people might favor safe choices in order not to appear foolish but then there is also the bias towards 'bold' 'pathbreaking' work that may in fact later turn out to be wrong.

For wine reviews, the motivations behind a newspaper panel (selecting widely available wines that would please a broad readership) are not the same as a medal panel. And the judges on that medal panel are not randomly selected from the public so why can't they have better credentials and be pre-disposed to liking the shrilly acidic wines we do. I understand that they may not have those preferences, but surely that is a function of who gets selected to serve on silly wine judging panels (and one of the reasons I ignore such medals) as opposed to the concept of a panel. I think there are enough people on here and Disorder to form a better panel!

Rahasaan, Hi....

The regression to the mean to which I refer involves the fact that for every panel of six or seven judges both the highest and the lowest scores for each wine tasted are discarded when calculating the data. That tendency is compounded even more when two different panels taste the same wine, this meanng that four scores are automatically discarded.

Again, I think most of the action (statistical and otherwise) comes in selecting the judges. Although of course these institutional effects are important as well.

Not only is the selection of judges is important, but how are the judges “instructed “ or “guided” to come up with their scores? (Is a modified UC Davis 20 point scale geared to wine appreciation used or a modified UC Davis 20 point scale geared to winemaking used). What are the criteria for actual awarding a medal to a wine? All of this should be determined by those organizing such an event, and should be determined by what their main goal or objective is.

Salute

My only experience with judging is at the amateur level. For several years I helped organize the Western New York Home Wine Competition, held at the Niagara County Fair. Some years I judged, some years I just totaled scores, prepares flights, etc. The judges are local and some of whom are AWS certified. They receive $20 for their efforts and are encouraged to write notes about each wine on the score sheets. The score sheets are later mailed back to the participants to provide feedback. The AWS 20-point scoring system is used. One thing about that scoring system is that it seems to lead the judges to look for flaws first. If a wine has no distinct flaws, it will usually get at least a bronze. Also, since RS masks flaws, wines with some sweetness tend to score higher than dry wines in such competitions. If my memory serves me, there has been only one dry red that ever won best of show. That was a 1977 DeChaunac that spent 2 years in an oak barrel and 3 years bottle aging, winning in 1982.

For many reasons (including a lot enumerated above) I have never seen the point in participating in organized judging. The result is often useless information with the least offensive wine getting the gold, even if it is like awarding a first to a class at a dog show just because the 'winner' was the only mutt that had 4 legs.

I pay zero attention to whether or not some local fair awarded some plastic trophy to a wine. It sure as heck doesn't take the place of tasting it yourself. If anything, it may be a negative indicator to me - a wine to steer away from on the basis that anything that pleased a group of amateur judges must be so bland and uninteresting that I am unlikely to enjoy it.

I don't think this is just about numbers and committee review but about the composition of the committee and their incentives. For scientific review, I can sort of see how people might favor safe choices in order not to appear foolish but then there is also the bias towards 'bold' 'pathbreaking' work that may in fact later turn out to be wrong.

In the review panels I've served on, appearing foolish isn't really a concern as the participants are incredibly astute and well-read. No, it's more about the inherent conservatism of panel review when there are limited resources to award. We all acknowledge at NIH review that we're "lower halving" (aka triaging) some very worthy proposals, but we have to do it when only 5-8% of the proposals submitted will be funded. Moreover, the psychology of those review panels is to look for flaws (sound familiar?) which almost any ground-breaking research will have. The best support for proposed experiments is precedent, which unprecedented research by its very nature lacks. But since NIH codifies "feasability" as one of its 6 review criteria, we have no choice, really.

And the judges on that medal panel are not randomly selected from the public so why can't they have better credentials and be pre-disposed to liking the shrilly acidic wines we do. I understand that they may not have those preferences, but surely that is a function of who gets selected to serve on silly wine judging panels (and one of the reasons I ignore such medals) as opposed to the concept of a panel. I think there are enough people on here and Disorder to form a better panel!

You have an overly optimistic view of how many wine lovers share your preferences, Rahsaan. Sure, here and at Disorder you are among (mostly) like-minded individuals, but venture out to the Spectator, WCWN and eBob boards and poll their preferences. Roberto Rogness over at eBob isn't exactly preaching to the choir, y'know? And look at professional wine critics: how many of the Spectator and WA reviewers share your tastes? If a majority of critics don't, why should judges at competitions? It's a big, harsh world out there, my friend. Stay cloistered here, amongst the Wine Taliban

Gleefully radical,
Mark Lipton

In the review panels I've served on, appearing foolish isn't really a concern as the participants are incredibly astute and well-read. No, it's more about the inherent conservatism of panel review when there are limited resources to award.

Ok. I was thinking more of peer-reviewed journals.

You have an overly optimistic view of how many wine lovers share your preferences, Rahsaan. Sure, here and at Disorder you are among (mostly) like-minded individuals, but venture out to the Spectator, WCWN and eBob boards and poll their preferences.

Maybe that example of appreciating 'shrilly-acidic' wines was too extreme. But I still maintain that it is all about who is doing the judging. And if there aren't enough competent wine people to make a competition worthwhile (for us) then all the more reason (for us) to disregard wine competitions, but not the broader concept of competitions (or their utility for other people).

As to who does the judging at major competitions...... winemakers, people in the wine trade and, as the organizers like to say ,"journalists of high repute".

With regard to the journalists - true, some of those who participate as judges are well versed, experienced and knowledgeable wine critics. On the other hand, the policy in many cases is to send invitations not to a specific journalist but to the editors or varioius newspapers. Many of those papers do not have a regular wine critic so that "gift" of a free trip abroad goes to a loyal buddy.

Two horror stories will suffice: At VinItaly (one of the most respected competitions), on the night before the judging began two journalists of high repute, one from Japan who wrote primarily about business matters and one from Korea whose specialty is automobiles approached me, asking "You know something about wine?" I nodded. They asked "Perhaps you can teach us about how to taste wine?"

Same competition a journalist from Poland. I never figured out what he wrote about but every morning as the bus left the hotel for the tasting center, he boarded with a full bottle of vodka. The trip took about 15 minutes and by the time we had arrived that bottle was empty. Throughout the day he tasted diligently....never spitting mind you but drinking through the day. He did prove his consistency. He awarded every wine 84 points. We had to half carry him back to the bus at the end of the tasting day.

With regard to the winemaker-judges - a small problem as well, for as knowledgeable as many of these people are they tend to overscore. After all, who would want to be nasty to his/her colleagues?

As to those "in the wine trade", all showed up for the first day of tasting;half for the second day; and about one-third for eah of the following days.

C'est la vie!

Best
Rogov

...On the other hand, the policy in many cases is to send invitations not to a specific journalist but to the editors or various newspapers. Many of those papers do not have a regular wine critic so that "gift" of a free trip abroad goes to a loyal buddy.
Two horror stories will suffice: ...

At the aforementioned Niagara County Fair, one year we decided to invite the food editor of one of the two large Buffalo newspapers to be a judge. She was born and raised in France, so, we thought she might be a pretty good judge. After the judging and awards ceremony, there would always be a social gathering for all contestants, judges and staff, so folks could taste each others' wines, etc. However, it seemed that all the award winning wines had disappeared. After searching the area, we found the esteemed food editor, in a very drunken state, loading all the good wines into the trunk of her car.

Questions for Daniel Rogov,
At these “large” International Competitions isn’t one of the seven judges of a panel “elected” or given the title of “Head-Judge”? Do the organizers instruct the judges on the type of scoring system used? Or the criteria for awarding medals? Are medals awarded only to wines of quality or deemed worthy or by how many entries in a category?

Howie- I bet that was the last time she was invited to come back and judge!

Salute