[Iliff Ave]

Lately I seem to notice a trend in comps I enter where I have two judges of different experience levels. The less experienced judges seem to give me scores that are about 4-6 points lower than the more experienced judges. Has any one else experienced this? What would be the explanation?

The only thing that makes sense is that the less experienced judges are looking for more flaws to prove they know what they are doing. It's almost they are looking for the good parts of a beer secondarily or as an afterthought. It's been very frustrating for me so I pretty much ignore those scores even though the two scores get averaged for my final.

[jeffy]

I have been observing the opposite lately.  As the more experienced judge I usually score a little less than the newer guy I am paired with.  We generally have a lot of the same comments and criticisms but I don't feel they are adhering to the little boxes at the bottom left that tell where a beer with such and such number of flaws should fit.  For example we may both notice flaws with carbonation and attenuation, but he/she still scores it mid-30's, while I am under 30. 
I think newer judges tend to be more lenient.

[dmtaylor]

Most typically I find that the less experienced judges very obviously copy some of the results from higher ranked judges, and it drives me up the wall because I would prefer that each judge give me THEIR OWN impressions.  I don't care at all about consensus, whether as an entrant or as a judge myself.  If I got totally different results from two different judges, I'd be both shocked AND pleased.

I understand what you are talking about though too.  I agree that less experienced judges can have a tendency to try to show off their knowledge, even to the point of their comments becoming absurd.  Some judges detect diacetyl in every single beer they taste, whether it's actually there or not.  Some judges will say every beer is oxidized and "probably better when it was fresher".  On the other hand I also have seen a Master level judge who claimed he was able to detect astringency in the aroma.  Yeah, right.  Scoresheets like this just go straight into the recycle bin for me.

It is for many of these reasons, in part, that I enter very few competitions anymore these days.  I enjoy judging, not saying I'm perfect at it either though, but I'd really rather judge than enter.  I think I entered one a couple years ago.  Some judges do an outstanding job, but they are few and far between IMO.  One thing I can promise to anyone who receives a scoresheet from yours truly -- I actually wrote down what *I* tasted, and I didn't give a damn about what the guy sitting next to me wrote on his sheet.  What you see is what I myself actually perceived.  I sure wish more people would judge that way.

Cheers.

[James K]

I actually wrote down what *I* tasted, and I didn't give a damn about what the guy sitting next to me wrote on his sheet.  What you see is what I myself actually perceived.  I sure wish more people would judge that way.

Cheers.

Im not the most experienced judge but I agree with this 100% and perhaps it’s the format of how my club has taught me to judge. When we judge anything, club meeting or a competition, we don’t even talk to one another while tasting. We all drink our little 4oz taster, mostly we don’t even need that much, and then we score, referring back to the beer if needed. After everyone’s card has been filled out, we share our points and then say why or why not. If someone needs to go up or down/up in score we adjust but no one is taking the information off of another judges card.

I recently judged a mead competition and I don’t know a whole lot about mead. I just followed the guidelines though and when one of the meads we moved forward made it to the finals everyone agreed that it should have been there.

I don’t see how you can copy someone’s score card though when everyone’s pallet is different. I’m hortible with detecting some things. And I’m great at detecting and talking about other things. You have to have a balance, otherwise it’s all subjective. IMO.

[Iliff Ave]

Thanks all. The only reason I enter comps is to get feedback. The most frustrating part lately is even decent scores get more negative feedback than positive.

[dls5492]

I like entering the same beer in a couple of competitions. What I look for is if the different judges are making similar comments. I look for a pattern.
I don't mind negative comments. Unless I get a 50, I want idea(s) to improve my beer. I appreciate all the work judges put in and thank every one of you!

[ethinson]

Thanks all. The only reason I enter comps is to get feedback. The most frustrating part lately is even decent scores get more negative feedback than positive.

If they are listing flaws and not listing ways to correct it then I would be worried, but if they are offering possible solutions than that's a good thing.  In theory, a judge should list everything they smell, see and taste.  So they might write "Light DMS" for an adjunct lager.  That's OK to style so they won't list a reason to fix that.  So yes, the more comments will be about flaws that need fixing so there will be more "negative" than positive.  But on the flip side of that coin, not everything listed is a flaw.  That took me a long time to wrap my head around both as a judge and as a competitor.  I'd get scoresheets back with some flavor listed and be like "That's supposed to be in there!!" assuming they had docked it for it.  Probably not. 

As far as the point difference, the BJCP guide says for judges to be within 7 points, so 4-6 is not bad.  Oddly enough in my experience, novice judges tend to score higher than experienced judges, but everyone is different.  I think your theory is valid though of a less experienced judge thinking "Ok, this is a point off and this is a point off and this is a point off" and shaving a lot of points off for minor flaws.

[Iliff Ave]

Good to hear others feedback. I am in no way disparaging any judges but just thought it was an interesting occurrence. It is likely just the pair of judges I got. It seems that as I have become a better brewer, my scores have gotten worse which is a bit frustrating. I don't think I will truly ever understand competitions but that's okay.

[dmtaylor]

I am in no way disparaging any judges

But *I* am.   

but just thought it was an interesting occurrence. It is likely just the pair of judges I got. It seems that as I have become a better brewer, my scores have gotten worse which is a bit frustrating. I don't think I will truly ever understand competitions but that's okay.

I've said this a hundred times, so I'll say it again:  The best way to handle all the inherent flaws with competitions is either: A) to give up and never enter at all, or B) to enter each beer into AT LEAST 3 or 4 competitions, then sift through all 6-10 scoresheets, throw out the 60% worthless scoresheets, and keep just the 40% that have any useful comments.  Yes, that's right.  Throw the bad ones straight into the recycle bin and forget that they ever happened.  This might sound funny to some but I swear I am dead serious.  That's how to resolve the issue, because BJCP can't fix it on the other end, never can, never will, impossible.

Cheers.

[reverseapachemaster]

You can get that difference in scores just by whether the judge scores by looking to take points away or build points up. In each situation a judge who thinks flavor could be a six or seven is going to round one way or the other. Have a judge with each paradigm at the table do that on two or three scoring categories and you'll consistently have a four to six point spread.

[reverseapachemaster]

Good to hear others feedback. I am in no way disparaging any judges but just thought it was an interesting occurrence. It is likely just the pair of judges I got. It seems that as I have become a better brewer, my scores have gotten worse which is a bit frustrating. I don't think I will truly ever understand competitions but that's okay.

Easily possible that as your beers improve they are improving towards what you like to drink rather than what scores well in a competition. An IPA with citra and mosaic is more likely to score well over an IPA with the 4Cs even if the latter better satiates the guidelines, just because judges tend to vote in favor of what they like and what they like is often whatever is popular at the moment.

[denny]

Based on my experience, any generalization about judging is bound to be wrong.....Dave....

[dmtaylor]

Based on my experience, any generalization about judging is bound to be wrong.....Dave....

I won't argue with that.  We're all over the dang place.  Even the same judge on two different days will often (usually?) give two different results.