I have run 2 professional breweries that used crushed grain. In both the grain sometimes sat for months until it was used. I could never tell the difference. When grain goes "slack" you can tell. It feels soft, not as dusty. If you have slack grain, throw it out.
That said, a mill will give you the ultimate control over your efficiency. Mash efficiency starts with the crush. But if you are happy with the efficiency you have, no need to stop using it. Until the mill gap changes, then don't be surprised if there are unexpected changes.