661

**General Homebrew Discussion / Re: Experiments, beer experiments and statistics**

« **on:**January 31, 2017, 06:28:39 PM »

...not finding a significant result is of minimal value. What that result means is that if there is a correlation between the two variables, then the experiment design was not sensitive enough to find it, not necessarily that the null hypothesis is true. This is why so many exbeeriments return no statistical significance. If they fail to find a statistical significance, it is nowhere near as strong of a result as a positive finding.

I like how you phrased this. This is sort of the point I've been trying to make, and the OP as well if I understood correctly. Yet I still feel a need also to expound on this: if the tasters are almost never able to reliably detect a difference, then personally I figure, what's the real value of making a conclusion statement to this effect? What if instead of aiming for these exbeeriments to give a 95% reliable result, we use a p value of like 0.15 or 0.2 which will tell you "well, MAYBE there's something going on here, but more exbeeriments will be needed to try to find out". Then again, to be fair, maybe it's just easier this way to avoid too much speculation in the conclusion statements. I guess the argument might be a horse apiece; regardless of the p, we can always say like, wow, we came really close to the 95% confidence, so maybe more xBmts are warranted. I dunno, I guess I'll be quiet now as we're talking in circles again as we love to do.

None of this is meant as a criticism for either Brulosophy or XB. I think they are both doing a great job - far better than I can do in my kitchen. The caution comes in with the interpretation of the results.

Ditto. I've got nothing but love and respect for anyone who runs xBmts or IGORs and publishes the results publicly and free of charge! Thanks all so much -- we and you all know who you are. Cheers.