Bob's Blog #11 Can You Crush Grapes with Dirty Feet?
Northern Rio Grande Chapter
New Mexico Vine and Wine Society
Tasting and Evaluating Wine – You got to be kidding me!
It’s been a while since I added a blog to this site. I feel compelled to do so because our last gathering was, by my way of thinking, was not only very informative, but quite surprising. Hmmm, okay, that’s a big lie. It was not at all surprising to me. But it absolutely was to most participants.
On October 6th our Chapter of the NM Vine and Wine Society held a wine tasting and “judging” event. For those of you that were unable to attend I will provide a semi comprehensive accounting of the day’s events.
It was at least two years ago that Mike and I discussed doing this convoluted tasing and judging event. Frankly, I was quite apprehensive when the time came to do it. I felt for sure that I would irritate at least some of the folks that participated in the tasting. Wendy had warned me to rein in my normally contrary and surly approach to making a point. I knew that was going to be hard. Nonetheless, I believed that we would all benefit, to one extent or another, from participating in this wine judging event. With considerable effort, and lots of upfront thought and research, I was able to not make a mule of myself.
We tasted three wines. Of the approximately 45 folks that attended the event all, but a couple participated in the judging. I started the tasting by stating that we would be tasting three wines. All would be New Mexico red wines; the same type and vintage and that we had three bottles of each. During the first round several participants speculated on the type of red wine. The guesses ranged from Cabernet Sauvignon, Syrah, Merlot and Pitit Verdot. One person guessed the right type of wine. Between each wine I asked everyone to cleanse their pallet. The first wine received an informal “average” score of 7.5 out of 10. Based on prior tasting and judging within our group, that was a very high score. The second wine received an average score of 4 out of 10. I was more than surprised, and I made a big deal over the fact that it absolutely could not be so low. The third tasting received an informal average score of 6.5. Based on prior experience with this group I’ve learned that if you add two points to the points given and multiply the resulting score by 10, the scores will be close to how the wine is “professionally” rated. Not that that means much.
Mike revealed each wine. They were all Vivac 2022 Petit Verdot. The wine sold at Vivac’s tasting room for $28 per bottle. All nine bottles were randomly numbered.
As this gathering was intended to have an “educational” focus, I spent considerable effort researching how well educated and trained wine judges very often lack consistency in there judging scores. I believe that it is important to point out that swallowing the wine allows it to coat the back of your palate and release more aromas when you exhale. It can also give you a sense of what drinking a full glass of the wine would taste like. However, swallowing can make it harder to taste new wines. During wine judging competition, wine judges do not drink the wine. Nonetheless, their palette is influenced by previous wines tasted.
I found several articles online regarding the lack of consistency when it came to wine judging. I focused on judging at the international competition level. I found a couple of articles that covered this topic. I was particularly impressed by two articles based on a study conducted by wine judge researcher Robert Hodgson. The following is a very brief synopsis of what he found and is mostly a cut and paste.
In 2000 California winemaker and retired statistics professor Robert Hodgson became curious when some wines that he entered in multiple competitions earned very different evaluations. For example, his 1993 Zinfandel won the San Francisco International Wine Competition Best of Show yet earned no award in a subsequent event. Being curious, and I assume, a little bewildered, this experience prompted Hodgson to set out on a comprehensive and systematic study to determine whether the inconsistencies in wine judging that he encountered were infrequent or commonplace. As a retired professor of statistics, he used his background to develope an approach to prove or disprove his theory that there was no perfect approach to consistently evaluate the quality of a given wine. In 2005, as a member of the advisory board for the California State Fair Commercial Wine Competition, Hodgson asked and received permission to conduct tests to check the reliability of the judges. It was not his intent to discredit the judges or the judging process. In these tests, four-judge panels were presented with their usual flights of wine samples to sniff, taste, and examine. However, to measure the judges’ consistency, some of the wines were presented to the panel three times and poured from the same bottle each time. Hodgson conducted these tests at the state fair each year over an eight-year period. Several different wines were evaluated by multiple judge panels. I should point out that I have no idea what variety of wines were tasted and evaluated. I only point this out because I do not know if scores given were more consistent for one type of wine rather than another.
The wine evaluation scale used by the judges ranged from a low score of 50 to a high score of 100; most of the rated wines were given scores in the 70s, 80s, and low 90s. By the way, in nearly all international wine competition almost all wines evaluated fall within a narrow band between 85 and 100 points. The scale doesn’t start at zero, it starts at 50. It’s important to realize that no wine maker is going to enter a competition unless he or she believes that their wine will be judged in the 85 or better range.
Who were these judges? Were they equivalent in competency to the judges that evaluate the New Mexico State Fair wine competition? No. They were recognized experts in the American wine industry, including professional winemakers, certified sommeliers, well-known wine critics and wine consultants, and even university professors who taught classes in winemaking and conducted research in winemaking.
The approach they used was common in wine judging. In this study, the judges would first indicate the wine’s score independently, which was then recorded. Afterwards, the judges discussed the wine, and based on the discussion, some judges modified their initial score. A key point is only the first independent score was used by Hodgson to analyze an individual judge’s consistency in scoring wines. As it turns out, it is not unusual for judges to be influenced by other judges especially judges that have great influence.
Results from the first four years of Hodgson’s study were published in 2008 in the Journal of Wine Economics, and they indicated that, overall, these wine experts were not very consistent judges. Over the three blind tastings, a typical judge’s scores for the “planted” wine varied by about plus or minus four points. For example, a wine rated as a good 90 would be rated a few minutes later by the same judge as an acceptable 86 and then a bit later as an excellent 94. An eight-point spread is considerable when it comes to awarding medals. Only about 10% of the judges were consistent in rating the identical wines, meaning that they gave the same wine presented to them three different times scores that varied by just plus or minus two points. In other words, these were the only judges whose ratings of the same wine typically stayed within the range of one medal. Yet even those judges who were consistent in their evaluations at Year 1 often were not very consistent at Year 2, so inconsistency was a more accurate descriptor of the vast majority of the judges’ performances. At the opposite end, another 10% of the judges gave the same wine far different ratings, ranging from a wine receiving a gold medal to the same wine receiving no medal at all. Consider our own judging of the Pitit Verdot, our range was 4 to 7.5. Assuming that you are the average wine consumer and do not have considerable training in evaluating wine, what does this mean to you? Don’t pay a whole lot of attention to how wine is scored by the “professionals”. Heck, based on our own tasting exercise, we can’t even trust our own ability to judge a wine.
In additional studies beyond California’s borders, Hodgson tracked wine that had been entered in at least two competitions in the United States and found that about 99% of the wines that earned gold medals in one event received no award in a different event. Several gold medal-winning wines were entered in five competitions. None of them received five gold medals, or even four gold medals. In commenting on his research findings as a whole and their implications for wine judging events generally, Hodgson stated, “Chance appears to have a great deal to do with the awards that wines achieve or miss out on.” He readily admitted that there are many individuals who are expert wine tasters with exceptional abilities to critically judge wines when a few samples are placed before them. Yet this is not what generally takes place at many wine-judging events. Hodgson explained, “When you sit 100 wines in front of a judge, the task is beyond anyone’s ability.”
In a different study I found that $100 bottle of wine is no more likely to get a medal than a $20 bottle of wine. Oh, one more thing, most people don’t take the time to taste their wine. They just drink it.
Raise your hand if you made it all the way through this blog.
Michele, welcome
to Our Site
Michele Padberg of Vivàc Winery was kind enough to write an article for our blog page. I believe that you will find it to be very witty, interesting and informative. Thank you, Michele.
Heading into the fall, we also head into the holiday party season! Halloween, Thanksgiving and December celebrations all require you to host your friends and family and if you are an avid wine drinker, you are headed toward an expensive bar bill. Naturally one of the most common questions I receive is how to host a party without breaking the bank. In other words, how do I serve cheap wine and not embarrass myself? First thing is, don’t be shy about being on a budget! If you drink a lot of wine or host a lot of get-togethers, it adds up fast. The trick to not going broke is to buy the best wine for the price bracket. How does one do that? Remember to always buy in bulk, 6 or 12 bottles get you added discounts. If shopping online, make sure to check for free shipping options and end of lot sales (one of my favorite ways to shop on Wine.com for example). If shopping in a grocery store, look at the sale tags and purchase wines that have a $5 or more (sometimes even over $10!) discount from the regular shelf price, they use this strategy to get people engaged with a product and then increase the price over time after you have attached to it. Another key trick is to do some research and look up winners on some of the Wine Competition sites you respect. You can actually look up winners by grape and format. Now, I am about to say something that freaks some people out, but the format you should be looking up is BOXED WINE. Cue shrieks of horror. Yes, I said it, shop boxed wine for big parties. There are actually some decent, and in specialty shops, great options out there these days and they are a bargain! Is she actually suggesting I set a box out for my wino friends?!? No, no, I am suggesting that you disguise it! Pour the boxed wine into charming carafes or even into a decanter (a little bit of a mean trick to the wine elite at your shin-dig that will inevitably think that it is the special older bottle you’ve been promising to open one day. Maybe give them a heads up. You then set out the plastic cocktail cups, you know the ones that open wide making all aromas and nuances impossible to trace. People at parties rarely are paying that close attention to the wine in the cup. The room is loud, the food is aromatic and the merriment is palpable, all these things distract your sense of taste. Your guests want something tasty, that's it. Disclosure here, this is for a BIG party with many levels of wine drinkers, not a wine tasting get together! Also set out the nicer bottles of “sale” wines you collected, it gives people options, but you watch, people will be plenty happy with the wine in the carafe. Another great way to stretch the wine buck is to make a sangria. No this is not sacrilege, well it would be if it was a vintage champagne or a collector’s special bottle, but with cheaper wine, you are absolutely ok doing this. My favorite is to take one liter of cheap white wine, one bottle of cheap sparkling wine, one liter of bubbly water and add sliced nectarines, and strawberries to it. Serve with ice. For red wine, I do the exact same recipe substituting red wine for the liter of white and substituting the fruit choices to be blueberries and raspberries. If you really want to get the party started, add 2 cups of vodka to the mix. The final tip is to focus on the FUN! Decorating with a theme not only makes your party a lot more enjoyable, it also helps mask your cheap wine more enjoyable. Halloween is all about red wine. It is the easy go to “blood” gimmick and served in IV bags (yes, this is a thing and yes people freaking love them! Shop on Amazon) is hilarious. Then there is white wine with green food coloring in a large punch bowl sitting on dry ice, looks like witches potion, and the kid inside every adult totally loves this classic, yet alcoholic, trick. There are also tons of cool ice molds out there! If you go on Pinterest you can spend hours looking at clever things to add to your ice cube trays or order the specialty forms that allow you to make absolutely any shape you can think of. Ice chills and dilutes wine which, when done in a fun way, masks your cheap wine too. Thanksgiving and the December holidays beg for bubbles! Cheap bubbly with a little juice or edible glitter takes the focus off the wine itself and is festive. As weather gets colder there is mulled wine as an option, but also the simplicity of the carafe wins again! My point in sharing these tips is to allow you to be more playful with wine. People have gotten far too uptight about wine losing the perspective to have fun with it. You know that saying ‘there is a wine for every occasion'. Well the cheap wine has its place too! So keep collecting those magical bottles and bring them out for the more intimate dinners with family and friends, the meaningful celebrations, and those lavish nights where you are alone with your feet up and decide you deserve a reward.