Gravity and time seem to work remarkably well on my Barley Crusher. No matter how much I bang/spin it after I mill a batch of grain, there's a mysterious pile of dust underneath the mill after a week or two. I'm not overly concerned about the last little bit of grain remnants that stick to the rollers after that.
I suppose if the last thing you milled was a whole lot of Black Patent all by itself, and you are brewing a helles as your next brew, that may be more of a concern. I tend to mix in my specialty grains with my base malt in the hopper specifically to avoid this scenario.