Given that many people just pour the wort right on top of the old yeast cake in the used fermentor, you'll be fine. The problem is getting your pitch rate right. I estimate and then forget about it. If you look at Mr. Malty, and guess a low concentration of cells to be safe, and a high percentage of non yeast to be safe, then about 300ml of day old unwashed slurry is fine for 5 gallons of 1.060 ale. That's about a cup and a quarter.
What I'd do (just did this morning actually) is rack my beer off the cake. Leave a 1/4" of beer. Swirl that slurry up till there's none stuck to the bottom. Pour that into a sanitized quart jar and put it in the fridge. When you're ready to pitch just pour off the beer at the top and pitch a cup to a cup and a half of the yeast. Cup if you're wort is 1.040-60 cup and a half if over 60. If it's huge, like 1.070 or bigger you'll need more. Double if it's a lager.
This is the Klickitat Jim scientific method dumbed down to reality in the homebrew world.