Grand Master III here.
I became a judge to improve my palate and better understand what I was tasting (and smelling, etc).
I was brewing what I thought were decent beers, but after taking the training and going through all the classes,
I learned so much that I didn't know I didn't know. My beers improved a lot, and it definitely made me a better brewer.
On the topic of "palate holes", it's very true, and it goes both ways. I'm particularly sensitive to phenolics, sulphur, and diacetyl, but I have a high threshold for picking up oxidation, and isovaleric acid is tasteless to me. Tom Schmidlin can't find diacetyl even at high levels. I let whoever I'm judging with know that, and ask them about their strengths and weaknesses so that we can kind of cover for each other when we are judging. Finding those sensitivities and holes is an important part of learning to judge. I'd try to point them out to examinees if their writeup indicates a mismatch when grading an exam, too. It's become a standard part of the RTP (exam grading report) to list them now, but it wasn't always the case.
It's true that many competitions have a hard time finding enough qualified judges, and that hurts us all. Entrants get a bad experience, and the volunteers feel overwhelmed. We made it a point up here in WA to increase our judge pool. I've personally taught, administered, and proctored the exam many times. The only way to make things get better is to pitch in and help.
Oh, and as far as Mini-BOS and BOS tables go, the best beers really do float to the top in my experience. The difference between the top few is very small usually. But, I've never walked away from a BOS table where I thought the beer that won wasn't a contender (regardless of if the one I thought should win didn't). I try to call them as I see them, and if the others disagree, that's fine, I'm not going to make a stink over it.