I have the old standard dial, analog type thermometer. Not attempted to calibrate for couple of years. I'm not sure of the process for calibration but what I just did was put it in an ice bath, note lowest temp I could get it to and did the same in boiling water.
In the ice I could not get below 35deg and it got to 213deg in boiling. I would guess that means that, on average, my thermometer is showing about 2 deg higher than true.
Unfortunately, that's not a great test. It could be WAY off around 150 where you really need it to be accurate. The best way to calibrate it to get a calibrated lab thermometer and compare your to it at mash temps.
No, I did not make a starter. Frankly, I'm not sure the process of doing that (ie, what medium to use, how long to wait before adding to wort, etc). And I don't know if it is necessary. I don't want to start a new topic here, though. I'm sure there is another thread I can find about it.
Unless your beer is under 1.040 OG, making a starter will always make you better beer IMO. For great info, see mrmalty.com .
+1 on both accounts, I've gotten into the habit of double checking with the lab thermometer every time I brew just about. As far as making a starter go, couldn't be simpler, just like making a small batch of 1.040 beer. DME as the medium (or you brew up several batches as AG.) I know you said you didn't want to "start a new topic" - but had you made a starter (or, more accurately, started with the appropriate pitch of yeast) you would probably have not run into this problem. Pitching enough healthy, fresh, viable yeast is one of the most important aspects of brewing.