For those of us who had statistics so long ago that there were no desktop computers, can you explain the symbol and the formula for this "statistical significance" (p less than .05, for example)? Near as I can tell, it comes out to about half...which leads to Sean's question about what level is significant, whether statistically relevant or just cuz it means something that can be considered reasonably dependable to base conclusions upon.
I'll try…
The thing that makes a triangle test so statistically useful is that it's binary; each participant either picks correctly, or doesn't. It's a coin toss, and if you toss a coin enough times you can collect valid statistics about how likely you are to get various combinations of heads/tails. The probability that a given heads/tails combination will come up for a fair coin is the p-value. In this case we're saying that, for something not to be due to chance, there has to be only a 5% probability that it occurred by chance, and therefore a 95% probability of it being an actual result. In the case of a triangle test, we'd expect people to guess correctly about 1/3 of the time, and so, as you noted, once the numbers of correct responses gets to something more like 1/2, it becomes significant. Exactly what that fraction needs to be for it to be significant depends on the desired p-value.
This is called a binomial distribution, by the way, and isn't limited to equally weighted outcomes or true/false outcomes, they just make the math much easier.
Edit: In the hard sciences (or at least in my experience), p<0.05 is generally acceptable for publication of empirical data. p<0.01 is generally desirable and would be accepted for publication pretty much anywhere.
Thanks, Sean, I was hoping one of you smart guys could set it straight for me - too many years away from college. I remember some things, but others, not so much. P is for Probability, but here maybe the high ranges for science publication are not necessary, so you are asking if there is an industry standard for non-science probabilities that would still be considered reasonable to use to authenticate a conclusion. If I caught your drift....