I turned the passive roller around and it helped a little bit. I reach under and turn it until it grabs now.
If it is binding you need to disassemble the mill and clean the spindles and bushings. I have had to clean mine a couple times in 7 years. I add a very small amount of silicone grease to each spindle while I reassemble it. When I say small amount I really mean a small amount. If you can see grease on the spindle its probably too much.
The rubber bushing is completely unnecessary if everything is spinning smoothly.