Here is a rough draft of my fauxpils results, along with my raw data. I'm hoping with some discussion I'll be able to update and change anything I may have neglected. Differences between the beers was very small and frequently contradictory, so I caution you from reading too much into my results.
This study was really easy to put together, very hard to analyze. I'm providing my data so you can look at it for yourself to decide if I draw a reasonable conclusion. If you disagree with my methods or results, it's easy to find qualified people on the AHA forum to serve as evaluators. Any study is useless if it can't be replicated, so I encourage anyone interested in this topic to organize their own study.
https://docs.google.com/open?id=0B29juefwLV4gSk0xN0NfRDFONnM < Latest version of results and summary.
https://docs.google.com/open?id=0B29juefwLV4gcnNjZFRCakpVa2shttps://docs.google.com/open?id=0B29juefwLV4gVnNBanpUaDNscFkhttps://docs.google.com/open?id=0B29juefwLV4gRHhZUExfUmlBcHMBig picture:
1) Decoction probably won't make your beer better.
2) Decoction mashing extracts more gravity from malt, and may extract more compounds that can be perceived as "dry" as well. Whether that's good or bad for a given recipe will depend on personal preference and your targeted beer profile.
3) The wort from the decocted beer was noticeably clearer. Hot break can bind with hop acids and reduce hop utilization, so the difference in perceived bitterness may be due to reduced hot break in the boil kettle.
4) Using melanoidin malt doesn't emulate decoction mashing
5) There were a lot of contradictory descriptions of the beers. How people perceive aroma and flavor is complex and not easy to anticipate.
What the data supports:
1) Decoction increases mash efficiency.
2) There was no statistically significant correlation between the BJCP scores and recipe.
3) 57% of evaluators preferred the no-sparge beer with 5% melanoidin malt (5%) over the triple-decocted beer (3X), 29% had no preference, and 14% preferred 3X.
4) Judges were significantly more likely to correctly identify duplicate beers than expected based on a random guess.
What the data probably supports:
1) Small difference with 3X leaning toward dry/bitter, 5% leaning toward malty/balanced.
What the data might support:
1) No difference other than color
2) Small difference, but no agreed-upon difference.
3) Either no-sparge or decoction had no effect, and the only difference was due to melanoidin malt.
What the data doesn't support:
1) Decoction makes a better beer
2) Decoction makes your beer maltier than using melanoidin malt w/no sparge.
3) Decoction makes a smoother beer
4) Decoction makes a beer more people prefer
Problems with the study:
1) Small sample size
2) I only have room for two fermentors in my freezer, so I could only make two beers at once to compare.
3) There were some issues with inconsistent carbonation from bottle priming. I've never noticed a difference in carbonation levels in my beer before, so this was really interesting for me.
4) A couple samples may have had a low-level infection. If I could consistently brew perfect beer, I wouldn't spend so much time on the AHA forum trying to learn about brewing.
Mixed variables:
I framed the study as a comparison of no-sparge vs decoction because I wanted to confound evaluators' expectations. If decoction could provide some special
je ne sai quoi beyond just darker color and increased maltiness in a way that melanoidin malt can't emulate, that should have shown up in the results, with more people preferring 3X or more people describing 3X in more favorable terms. In any case, it's possible the no-sparge, decoction or the melanoidin malt had no effect, but I'd say it's more likely that no-sparge or decoction had no effect and melanoidin malt had some effect.
