Author Topic: Double blind taste testing (I think)  (Read 1593 times)

Offline micsager

  • Senior Brewmaster
  • ******
  • Posts: 1179
    • View Profile
Double blind taste testing (I think)
« on: December 01, 2010, 03:34:07 PM »
Last night I took out a bottle of my standard American Amber that had been in the fridge about 6 months.  And one of the same recipe that was only a month old.  We did a test with three glasses, with folks trying to figure out which of the three was "different", and then if they got that right, which was the more aged beer.

Out of four of us, only one identified the different beer, and he got the aged vs fresh wrong. 

I was happy with those results.  Plus, it was fun to do.

Offline James Lorden

  • Brewer
  • ****
  • Posts: 337
  • Forest Hill, MD
    • View Profile
Re: Double blind taste testing (I think)
« Reply #1 on: December 01, 2010, 03:45:04 PM »
Interesting.  I did a taste test this weekend that I will post more about elsewhere in the same theme.  I had a few bottles of Lagunitas Brown Shugga from last year then bought a fresh six pack to taste against. This beer is brewed in the fall so the beer I bought recently was pretty fresh.  It was amazing how the beer rounded out vs the angular brewery fresh example.  I very much enjoyed the aged version.  The assimilation of the hops into the overall flavor profile was most interesting.
James Lorden
Beer Drinker Beer Maker & Beer Judge

Offline johnf

  • Brewer
  • ****
  • Posts: 434
    • View Profile
Re: Double blind taste testing (I think)
« Reply #2 on: December 01, 2010, 03:53:34 PM »
Last night I took out a bottle of my standard American Amber that had been in the fridge about 6 months.  And one of the same recipe that was only a month old.  We did a test with three glasses, with folks trying to figure out which of the three was "different", and then if they got that right, which was the more aged beer.

Out of four of us, only one identified the different beer, and he got the aged vs fresh wrong. 

I was happy with those results.  Plus, it was fun to do.

I think you have repeated a common mistake with these triangle tests which is to assume that anyone who got the odd one correct did so because they could tell a difference and so their opinion is valid.

If you all guessed blindly, 4/3 of you would have gotten it right accidentally. In practice, 3/3 of you did which is very close to what you would expect if there were no discernible difference. Therefore the correct interpretation of the results is that there is not statistically significant difference among the samples and the observations of the 1 taster who chose the correct odd sample are not weighted above the observations of the other three.

Offline denny

  • Administrator
  • Retired with too much time on my hands
  • *****
  • Posts: 19939
  • Noti OR [1991.4, 287.6deg] AR
    • View Profile
    • Dennybrew
Re: Double blind taste testing (I think)
« Reply #3 on: December 01, 2010, 04:28:53 PM »
This is the test that I wish homebrewers would do more often.  I think many people would be surprised if they did.
Life begins at 60.....1.060, that is!

www.dennybrew.com

The best, sharpest, funniest, weirdest and most knowledgable minds in home brewing contribute on the AHA forum. - Alewyfe

"The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts." - Bertrand Russell

Offline micsager

  • Senior Brewmaster
  • ******
  • Posts: 1179
    • View Profile
Re: Double blind taste testing (I think)
« Reply #4 on: December 01, 2010, 04:36:50 PM »
Last night I took out a bottle of my standard American Amber that had been in the fridge about 6 months.  And one of the same recipe that was only a month old.  We did a test with three glasses, with folks trying to figure out which of the three was "different", and then if they got that right, which was the more aged beer.

Out of four of us, only one identified the different beer, and he got the aged vs fresh wrong. 

I was happy with those results.  Plus, it was fun to do.

I think you have repeated a common mistake with these triangle tests which is to assume that anyone who got the odd one correct did so because they could tell a difference and so their opinion is valid.

If you all guessed blindly, 4/3 of you would have gotten it right accidentally. In practice, 3/3 of you did which is very close to what you would expect if there were no discernible difference. Therefore the correct interpretation of the results is that there is not statistically significant difference among the samples and the observations of the 1 taster who chose the correct odd sample are not weighted above the observations of the other three.
Not really following, can you explain more?

Offline gordonstrong

  • Senior Brewmaster
  • ******
  • Posts: 1355
    • View Profile
    • BJCP
Re: Double blind taste testing (I think)
« Reply #5 on: December 01, 2010, 04:46:39 PM »
You're describing a Triangle Test.  Double blind trials are done in things like medical research, where the researchers and the study participants don't know who is in the experimental group and who is in the control group.
Gordon Strong • Beavercreek, Ohio • AHA Member since 1997 • Twitter: GordonStrong

Offline micsager

  • Senior Brewmaster
  • ******
  • Posts: 1179
    • View Profile
Re: Double blind taste testing (I think)
« Reply #6 on: December 01, 2010, 04:52:20 PM »
You're describing a Triangle Test.  Double blind trials are done in things like medical research, where the researchers and the study participants don't know who is in the experimental group and who is in the control group.
thanks.  I heard about it from Brewstrong, so thought I would give it a shot......

Offline oscarvan

  • Senior Brewmaster
  • ******
  • Posts: 1707
    • View Profile
Re: Double blind taste testing (I think)
« Reply #7 on: December 01, 2010, 04:56:14 PM »
There is another interesting one.......

Blindfold someone, and in no particular order let him/her smell and taste a dry white and red wine several times......

The results may surprise you.
Wooden Shoe Brew Works (not a commercial operation) Bethlehem, PA
http://www.woodenshoemusic.com/WSBW/WSBW_All_grain_Setup.html
I brew WITH style..... not necessarily TO style.....

Offline gordonstrong

  • Senior Brewmaster
  • ******
  • Posts: 1355
    • View Profile
    • BJCP
Re: Double blind taste testing (I think)
« Reply #8 on: December 01, 2010, 04:59:26 PM »
Even better is to not use a blindfold.  Ask someone to describe a white wine.  Then give them the same wine with red food coloring added and ask them to describe it.  Then you'll understand one of the sources of bias.
Gordon Strong • Beavercreek, Ohio • AHA Member since 1997 • Twitter: GordonStrong

Offline denny

  • Administrator
  • Retired with too much time on my hands
  • *****
  • Posts: 19939
  • Noti OR [1991.4, 287.6deg] AR
    • View Profile
    • Dennybrew
Re: Double blind taste testing (I think)
« Reply #9 on: December 01, 2010, 05:05:06 PM »
after the flurry of protest I got with my decoction experiment, I did a lot of research on things like this.  I found one particularly interesting experiment in which Bamforth gave a tasting panel identical beers, but told them one had been produced with traditional German techniques while the other used sugar and other things to speed up fermentation.  The panel reported difference in the beers, even though they were exactly the same.
Life begins at 60.....1.060, that is!

www.dennybrew.com

The best, sharpest, funniest, weirdest and most knowledgable minds in home brewing contribute on the AHA forum. - Alewyfe

"The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts." - Bertrand Russell

Offline bluesman

  • Global Moderator
  • I must live here
  • *****
  • Posts: 8825
  • Delaware
    • View Profile
Re: Double blind taste testing (I think)
« Reply #10 on: December 01, 2010, 05:23:50 PM »
I'm not trying to challenge mics good news but one begs to question, can the highly experienced beer connoisseur and/or Master/National BJCP beer judge distinguish or taste the difference between a six month AAA and a month old AAA repeatably after a calibration. The same goes for a double or triple decocted beer vs. a single infusion mashed beer of the same recipe.  I would be curious to realize the results.
Ron Price

Offline micsager

  • Senior Brewmaster
  • ******
  • Posts: 1179
    • View Profile
Re: Double blind taste testing (I think)
« Reply #11 on: December 01, 2010, 05:39:26 PM »
I'm not trying to challenge mics good news but one begs to question, can the highly experienced beer connoisseur and/or Master/National BJCP beer judge distinguish or taste the difference between a six month AAA and a month old AAA repeatably after a calibration. The same goes for a double or triple decocted beer vs. a single infusion mashed beer of the same recipe.  I would be curious to realize the results.

I hear what your saying....  (although I'm still happy)

Offline bluesman

  • Global Moderator
  • I must live here
  • *****
  • Posts: 8825
  • Delaware
    • View Profile
Re: Double blind taste testing (I think)
« Reply #12 on: December 01, 2010, 05:40:32 PM »
I hear what your saying....  (although I'm still happy)

...and that's all that really matters.  :)
Ron Price

Offline denny

  • Administrator
  • Retired with too much time on my hands
  • *****
  • Posts: 19939
  • Noti OR [1991.4, 287.6deg] AR
    • View Profile
    • Dennybrew
Re: Double blind taste testing (I think)
« Reply #13 on: December 01, 2010, 06:16:35 PM »
I'm not trying to challenge mics good news but one begs to question, can the highly experienced beer connoisseur and/or Master/National BJCP beer judge distinguish or taste the difference between a six month AAA and a month old AAA repeatably after a calibration. The same goes for a double or triple decocted beer vs. a single infusion mashed beer of the same recipe.  I would be curious to realize the results.

As a highly experienced BJCP National/almost Master, I can tell you that the answer is "maybe more often than some people, but not consistently".
Life begins at 60.....1.060, that is!

www.dennybrew.com

The best, sharpest, funniest, weirdest and most knowledgable minds in home brewing contribute on the AHA forum. - Alewyfe

"The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts." - Bertrand Russell

Offline Mikey

  • Brewer
  • ****
  • Posts: 397
    • View Profile
Re: Double blind taste testing (I think)
« Reply #14 on: December 01, 2010, 07:55:26 PM »
I'm not trying to challenge mics good news but one begs to question, can the highly experienced beer connoisseur and/or Master/National BJCP beer judge distinguish or taste the difference between a six month AAA and a month old AAA repeatably after a calibration. The same goes for a double or triple decocted beer vs. a single infusion mashed beer of the same recipe.  I would be curious to realize the results.

As a highly experienced BJCP National/almost Master, I can tell you that the answer is "maybe more often than some people, but not consistently".

When I used to work on a fishing boat as a baiter, I almost got to the level of master. :D