Author Topic: The Importance Of Being Same  (Read 1405 times)

Offline klickitat jim

  • I must live here
  • **********
  • Posts: 8600
    • View Profile
The Importance Of Being Same
« on: October 07, 2015, 07:26:59 PM »
Probably since the beginning of brewing there has been discision regarding differences. At least I've noticed it since I started way back in 2012. For the sake of discussion,  spurred by something I read on the forum today, I thought it might be interesting to air out and debate ways of determining differences and how important it is or isn't.

Let's pick on Denny for a minute. He is a big proponent for the blind triangle test. Two of one, one of another, pick which one is different. If enough testers with decent palates can pick the odd ball, then there is a difference. If they cant pick it, there's not a difference. Thats how I understand it anyway.

But then recently Denny posted results of his Noti Brown brewed using Mark's starter method. He didn't use the triangle, nor did he use any other tasters other than hisself. He said that it was his best Noti Brown ever. Judged from memory only.

Also recently, there was some debate over possible differences in an experiment Marshall did with old hops. He caught a little scrutiny flak for having two different worts. Same grain bill, water, etc... they were just two seperate batches. Seems to me those two worts were probably different, but only minimally,  and only in theory since they weren't charted on a GC/MS. So its possible they actually were the same, though a long shot.

Anywho...
When does difference matter so much that it requires charting on a GC/MS?
When does it matter enough to do a blind triangle with BJCP GMs? Masters? Nationals? Certified? Pro brewers? Novices?
When does difference matter enough that brewer's/taster's memory is good enough?

I have some thoughts, but I'll leave it to you guys for now.

Offline HoosierBrew

  • Global Moderator
  • I must live here
  • *****
  • Posts: 13030
  • Indianapolis,IN
    • View Profile
Re: The Importance Of Being Same
« Reply #1 on: October 07, 2015, 07:41:00 PM »
I think if you're telling people categorically that one way works better than another way, being able to provide some data like a triangle done with what you consider good palates is more solid than just anecdotal info from your brew days. I don't think a set rank has to be achieved to do a triangle, just good palates. OTOH, if I'm offering up what I've observed in my brewing as only shared anecdotal info, then I don't think I need to have done a triangle to share it. That's why I at least try hard not to be categorical, because we all know there are multiple ways to do most of this stuff.

EDIT-  And even a single triangle test doesn't necessarily categorically prove anything. It's just a good data point to consider.
« Last Edit: October 07, 2015, 07:50:06 PM by HoosierBrew »
Jon H.

Offline dmtaylor

  • Official Poobah of No Life.
  • *
  • Posts: 3429
  • Two Rivers, WI
    • View Profile
    • Manty Malters - Meet the Malters! - Dave Taylor
Re: The Importance Of Being Same
« Reply #2 on: October 07, 2015, 07:49:14 PM »
Data is (or "are" for you snooty grammar folks!) data.  Some data is better than others.  It is up to each individual to decide which data he or she likes and which he or she does not.  The human tendency to be biased/prejudiced dictates that we will tend to gravitate towards data that agrees with our own hypotheses, and ignore or attempt to refute that which does not.  Also, we just love to argue, for no sake other than we just love to argue.

"Differences"/discrepancies/anomalies matter when they matter to several people.  When they matter only to few or one, they don't seem to matter.  Still, it doesn't prevent us from arguing about it, because we so love to argue just for the fun of it.
Dave

The world will become a much more pleasant place to live when each and every one of us realizes that we are all idiots.

Offline pete b

  • Official Poobah of No Life.
  • *
  • Posts: 3123
  • Barre, Ma
    • View Profile
Re: The Importance Of Being Same
« Reply #3 on: October 07, 2015, 07:53:03 PM »
A lot depends on your personality type and what sort of brewer you are. I don't brew for competitions. My goal is to brew beer that subjectively tastes good to me in a way that is fun and takes a reasonable amount of time. If people are making starters in an easy way that works I'll try it and see if it works for me. No triangle tests, double blind studies, blah blah blah. Someone else might find that aspect of brewing fun but brewing is just one of the things I do so...yeah.
Don't let the bastards cheer you up.

Offline Village Taphouse

  • Brewer
  • ****
  • Posts: 410
  • Ken from Chicago
    • View Profile
    • Mayfair Court Brewhouse
Re: The Importance Of Being Same
« Reply #4 on: October 07, 2015, 08:09:56 PM »
I'll be honest... when I hear about things like the exbeeriments or blind tasting, I take that information and file it away in case I ever need it.  Otherwise, I generally don't pay much attention to it.  I say this because of all the variables that go into this type of thing.  I almost equate it to a movie reviewer... he liked it or didn't like it but that's not me, it's him.  I follow what I think works the best in my experience.  I take detailed notes on each batch and when I make that recipe again I look and see what I thought should be changed or stay the same.  I mentioned in our FB group that I really like the character I get from FWHing a beer.  One of Marshall's exbeeriments (I think...) was comparing FWH'd beers to non-FWH'd beers.  The results were that no one could tell the difference.  I find that interesting but anytime I'm drinking a beer and think it's exceptional and then I go check the recipe, it contained a FWH.  Nothing scientific about my findings, just consistently good results as far as my tastebuds are concerned.  I should also state for the record that I *REALLY* like when our Super-Brewers conduct these experiments and share their findings with the rest of us.  It makes us all better brewers and shines a light into some dark corners of the brewing world. 
« Last Edit: October 07, 2015, 08:12:12 PM by Village Taphouse »

Offline Wort-H.O.G.

  • Official Poobah of No Life.
  • *
  • Posts: 4439
  • Play Nice
    • View Profile
    • Harvey's Brewhaus
Re: The Importance Of Being Same
« Reply #5 on: October 07, 2015, 08:17:28 PM »
good comments so far. I too appreciate the information shared by others, and evaluate if its absolute or something different that provided good results. I don't need a triangle test (although sometimes it helps my evaluation) - just some basic information about what they did and their results. From there, I decide if it has any value for me, and if its worth  my time to give it a shot.

I ran a test on pure pitch without a starter- didn't make any claims or statements indicating my results were based upon anything other than my past experiences and sensory observations. I made no claims that it's the new or only way to proceed, just shared my findings and what I thought...the rest is up to others.

that's what I love about this forum and the people on it.
Ken- Chagrin Falls, OH
CPT, U.S.Army
https://www.facebook.com/pages/Harveys-Brewhaus/405092862905115

http://braukaiser.com/wiki/index.php?title=The_Science_of_Mashing

Serving:        In Process:
Vienna IPA          O'Fest
Dort
Mead                 
Cider                         
Ger'merican Blonde
Amber Ale
Next:
Ger Pils
O'Fest

Offline BrodyR

  • Brewmaster
  • *****
  • Posts: 512
  • Philadelphia, PA
    • View Profile
Re: The Importance Of Being Same
« Reply #6 on: October 07, 2015, 08:24:32 PM »
I love the triangle tests/multiple tasters Brulosophy style! I agree that everyones tastes things differently so if something works for you (especially something as effortless as FWH) no need to change it from a random test but that being said, that sort of data does help me make decisions.

There seems to be a lot of misinformation out there that's accumulated over the years so it's great to see people test theories out and be able to back it up with some taste tests. In my opinion the proving or disproving or homebrew dogma is the most exciting thing happening in homebrewing at the moment.

Offline Village Taphouse

  • Brewer
  • ****
  • Posts: 410
  • Ken from Chicago
    • View Profile
    • Mayfair Court Brewhouse
Re: The Importance Of Being Same
« Reply #7 on: October 07, 2015, 08:24:50 PM »
I also find that subtle nuances in various beers get swept away by variables.  The taste of a single 5-gallon batch of beer changes over a short time.  Our tastebuds are different every day based on what we have eaten, what we drank yesterday or already today, whether we're sick or on medication, etc.  If I'm making a beer that I would rate 97 out of 100, I'm probably not overly interested in making it better.  If I'm making a beer I would rate a 60 out of 100, I am tirelessly looking for an answer because I'm making beer I don't want to drink.  Always good to bang these questions around with good brewers who are good people too... echoing Ken's words.

Offline narcout

  • Senior Brewmaster
  • ******
  • Posts: 1811
  • Los Angeles, CA
    • View Profile
Re: The Importance Of Being Same
« Reply #8 on: October 07, 2015, 08:25:25 PM »
I feel that I can't even trust my own taster's memory. 

I can't think of how often I've purchased a commercial beer that I've had many times in the past and been surprised by how it actually tastes on the particular day I am drinking it.
It's too close to home
And it's too near the bone

Offline Village Taphouse

  • Brewer
  • ****
  • Posts: 410
  • Ken from Chicago
    • View Profile
    • Mayfair Court Brewhouse
Re: The Importance Of Being Same
« Reply #9 on: October 07, 2015, 08:36:24 PM »
I feel that I can't even trust my own taster's memory. 

I can't think of how often I've purchased a commercial beer that I've had many times in the past and been surprised by how it actually tastes on the particular day I am drinking it.
I agree.  I have had bad commercial beers that I know are usually better than that.  Or a draft version of a beer that is *WAY* better than the bottled versions I have had.  So.  Many.  Variables.

Offline Joe Sr.

  • Official Poobah of No Life.
  • *
  • Posts: 4378
  • Chicago - NORTH SIDE
    • View Profile
Re: The Importance Of Being Same
« Reply #10 on: October 07, 2015, 08:46:04 PM »
I think it depends on what you're trying to test.
It's all in the reflexes. - Jack Burton

Offline dilluh98

  • Brewmaster
  • *****
  • Posts: 575
    • View Profile
Re: The Importance Of Being Same
« Reply #11 on: October 07, 2015, 08:48:46 PM »
I love Marshall’s experiments. I think they’re both highly informative and entertaining but it’s also a single brew day. While he has multiple taste testers, his N still equals 1 regarding the experiment itself. BUT, he never has claimed his experiments to be the final verdict on a topic - only fodder for further discussion and exploration. That caveat is the nature of good scientific process.

In some respects I actually trust Denny’s recent experiment with the shaken not stirred starter method more than I would a single Brulosophy experiment. Why? Because Denny has brewed that particular beer A LOT of times and probably has a well-calibrated sense of how it tastes. If the only thing that has changed is his methodology in preparing the starter and he finds a significant taste difference - well, that’s a lot of data points suggesting that starter preparation method matters. I'd put up the red flag of confirmation bias in this example except Denny was so insistent that the stir plate was the way to go before committing to the experiment.

Replication and repeatability of results is something that is starting to hit home, in a big way, with a lot of pharmaceutical and biochemical research now. So much so that some high-end journals publishing this research are beginning to flesh out protocols regarding repeatability of the experiments. If we are interested in the effect of home-brew techniques on beer taste, replication and repeatability should be on the mind.

Offline toby

  • Senior Brewmaster
  • ******
  • Posts: 1021
  • Galvez, LA
    • View Profile
    • Beer Judge Chronicles
Re: The Importance Of Being Same
« Reply #12 on: October 07, 2015, 08:55:20 PM »
I think it depends on what you're trying to test.

^^This^^

All day.

Offline denny

  • Administrator
  • Retired with too much time on my hands
  • *****
  • Posts: 19858
  • Noti OR [1991.4, 287.6deg] AR
    • View Profile
    • Dennybrew
Re: The Importance Of Being Same
« Reply #13 on: October 07, 2015, 08:59:14 PM »
Probably since the beginning of brewing there has been discision regarding differences. At least I've noticed it since I started way back in 2012. For the sake of discussion,  spurred by something I read on the forum today, I thought it might be interesting to air out and debate ways of determining differences and how important it is or isn't.

Let's pick on Denny for a minute. He is a big proponent for the blind triangle test. Two of one, one of another, pick which one is different. If enough testers with decent palates can pick the odd ball, then there is a difference. If they cant pick it, there's not a difference. Thats how I understand it anyway.

But then recently Denny posted results of his Noti Brown brewed using Mark's starter method. He didn't use the triangle, nor did he use any other tasters other than hisself. He said that it was his best Noti Brown ever. Judged from memory only.

Also recently, there was some debate over possible differences in an experiment Marshall did with old hops. He caught a little scrutiny flak for having two different worts. Same grain bill, water, etc... they were just two seperate batches. Seems to me those two worts were probably different, but only minimally,  and only in theory since they weren't charted on a GC/MS. So its possible they actually were the same, though a long shot.

Anywho...
When does difference matter so much that it requires charting on a GC/MS?
When does it matter enough to do a blind triangle with BJCP GMs? Masters? Nationals? Certified? Pro brewers? Novices?
When does difference matter enough that brewer's/taster's memory is good enough?

I have some thoughts, but I'll leave it to you guys for now.

Jim, it simply depends on what kind of results you're looking for.  In my case, I wasn't trying to empirically prove anything.  It was just a trial and I posted my impressions.  I hope I made that clear in my blog post.  But due to those impressions, I intend to do a more thorough test.  At this point, I won't try to convince anyone of the validity of the validity of my impressions.  They simply are what they are.  If I do a real test, and can get others to repeat it and verify the conclusion, that will be another matter.

Does that all make sense?
Life begins at 60.....1.060, that is!

www.dennybrew.com

The best, sharpest, funniest, weirdest and most knowledgable minds in home brewing contribute on the AHA forum. - Alewyfe

"The whole problem with the world is that fools and fanatics are always so certain of themselves, and wiser people so full of doubts." - Bertrand Russell

Offline klickitat jim

  • I must live here
  • **********
  • Posts: 8600
    • View Profile
Re: The Importance Of Being Same
« Reply #14 on: October 07, 2015, 10:04:09 PM »
I think we wonder about differences from ingredients and differences from process. For the sake of simplification, I'm narrowing it to just plain difference, regardless of cause. Its important to think about wether or not the difference is important before choosing a way to determine difference.

In a criminal trial, things must be proven beyond reasonable doubt. If there is no doubt for which there is a reason, the thing is considered proven. In a civil trial, the thing must only be proven by a preponderance of evidence. A weighing out of evidence showing that more likely than not, it happened. If there is 1 reason to doubt but 2 reasons not to doubt, then you have a preponderance of evidence it happened. Whereas in the criminal trial there must be evidence it happened and there can be no reasons to doubt.

So what? Well, sometimes we hold trial on presented ingredients issues or process issues. What level of proof do we require? Is that level of proof in line with how much difference is made and how important that difference actually is? If, for a home brewing process, we require greater level of proof than is needed to put a Murderer in prison... maybe that's going too far.

For a minute, imagine you're trying to repeat a beer. Make it the same as the last one. Is that possible? The person who would require absolute evidence would say No. Its impossible to rebrew a pefectly identical beer. I would say Yes. As far as it is only going to be tested in my mouth and compared to my memory, yes, I can rebrew beers that are the same according to my level of proof.

So, if I am looking to make a change, will that change be discernable in my mouth as compared to my memory? And is the discernible difference desirable? Then, is the ingredient or process change worth it to obtain the difference?

That thought process shows the importance of personally experiencing changes with home brewing, rather than putting too much weight on the credibility of the claims of others. Ultimately, beer is a personal experience. It doesn't matter that a scientific analysys proves a difference, or if some expert says so. If its not there, for you, its not there.

Case in point. I brewed a beer with US05 sprinkled dry. Then I read an expert who claimed that 50% of them die because of osmotic preasure. So I rehydrated. Then I read that liquid yeast was better so I bought Wy1056 and followed the instructions. Then I read that it wasn't enough yeast, so I made a starter. Then i read that repitching was way better, but i didnt repitch enough, then i was repitching way too much. Then I read that my starter needed to be on a stirplate. Then I read that it needed to not be on a stirplate, but shaken instead. Most recently, I chose to hit it with O2 because I'm too lazy to shake. And pitch at high krausen, whereas I used to chill and decant because I was told that the spent wort was bad. Now unspent high krausen wort seems to be the thing.

So in that last paragraph theres a lot about what I had read or been told but nothing about what I experienced from those different methods. All the expert stuff is just academic. A good starting point if you dont know what to do. The only thing that really matters is what you think of your results with your beer. And when comparing changes, memory is all that really matters, because its awfully hard to preserve a perfect sample from last year's beer in order to do a triangle test.

 Some times some of us need to prove things to others, but all of us need to prove it to ourselves.