Author Topic: Question about competition judging  (Read 3145 times)

Offline tmsnyder

  • Cellarman
  • **
  • Posts: 38
    • View Profile
Re: Question about competition judging
« Reply #15 on: August 06, 2013, 11:21:49 AM »
It's up to the judge director to match up judging styles and personalities.  So if it were up to me, I'd put you with someone equally bullheaded and let you duke it out!  Nothing wrong at all with a heated debate over a beer, as long as the comments go on the scoresheet too, so the brewer gets the benefit of the debate.

I'd rather get two well written scoresheets from any rank of judges than four poorly written ones.  It's up to the judge director to keep an eye one these and make sure the brewers are getting their money's worth from the judges.  You should be able to tell what's going on with your beer from two good scoresheets. 




So I got to thinking some more about this...<snip>  If I think the other guy is wrong, I ain't changing my score, and I would expect the same the other way around as well.  I think there needs to be a place for bullheads like me.  <snip>

Perhaps my real wish is that we could get a whole lot more feedback from any one competition.  One Recognized BJCP judge plus a Gump just ain't cutting it.  I wish we could get feedback from 3 or 4 BJCP ranked judges at every comp.  Perhaps not feasible today, but maybe in 50 years it will be common.  That'd be sweet.  Otherwise the only way to know if your beer is really REALLY good is to enter each beer into at least 3 competitions.  And that gets real expensive real quick.  So anyway...

 ;D

Offline kramerog

  • Brewmaster
  • *****
  • Posts: 789
    • View Profile
Re: Question about competition judging
« Reply #16 on: August 06, 2013, 11:30:42 AM »

And it's true, after a couple beers the judges will often end up in lockstep and after judging independently will find that they are making the same comments and scoring them the same.


Lockstep is interesting.  To the extent that judges are trying to mirror each other then lockstep is not necessarily a good thing.  To the extent that judges are in lockstep because they have a common (hopefully correct) understanding of the style, which is often not the case initially, lockstep is generally good.
Brewers of South Suburbia
Brixie's Brewers
Oak Park Homebrewers

Offline tmsnyder

  • Cellarman
  • **
  • Posts: 38
    • View Profile
Re: Question about competition judging
« Reply #17 on: August 06, 2013, 11:38:49 AM »
Dan,

Send me a bottle of your barleywine, I'll sit down with two or three other judges and give you some feedback for free!  :)  LOL!

What were the individual scores?  Was the 23 an outlier?

Any chance you had bottle to bottle variability? 

It's also possible that the competitions didn't temper the beers correctly.  An ice cold barleywine can come across thin and bitter, even though the judges are supposed to ensure the beer is served at the proper temperature. 

Some competitions are more stingy with points too, and some are more generous.  A lot of times it comes from the judge director reminding the judges at the kick off speech that they shouldn't be afraid to give points out in the 40's, and be sure they use the entire range possible.  Without such encouragement, I see it all the time, scores range from 20 to high 30's and that's it.  So we're judging beers over an 18 point range instead of a 35 point range, from 15-50 (in general, we're limited to 15 as the minimum so as not to be hurtful).  So some of the variability you see is from that.  But within the competition they should be consistent, just more or less stingy with points. 



interesting...I put my American Barley Wine this year in three different comps all within a time frame of 2-3 months.  I've received scores from 23-43 on the same beer...yeah a 20 point difference from BJCP judges.  The 43 was from the highest ranking judge, so I 'll have to agree with him ;)

Offline tmsnyder

  • Cellarman
  • **
  • Posts: 38
    • View Profile
Re: Question about competition judging
« Reply #18 on: August 06, 2013, 11:57:05 AM »
It just take a few beers to get dialed sometimes.  But they should be in agreement on what the style should smell/look/taste/feel like.  If one thinks a style is one way, and another thinks the style is another way it would be tough to come to any consensus. That's why the style guidelines are so great.

As far as mirroring one another, that would mean that they're tasting the same things, and comparing the beer to the same standard for the style, and coming to the same conclusion and score.  So yes, as long as the standard they're using (the style guideline) is correct, then mirroring is good. If they're in lockstep and using the wrong standard then that's bad!



And it's true, after a couple beers the judges will often end up in lockstep and after judging independently will find that they are making the same comments and scoring them the same.


Lockstep is interesting.  To the extent that judges are trying to mirror each other then lockstep is not necessarily a good thing.  To the extent that judges are in lockstep because they have a common (hopefully correct) understanding of the style, which is often not the case initially, lockstep is generally good.