I think a starter might be a good idea as the yeast might also be a bit stressed by the high gravity environment they are coming from.
I might also be a bit worried about color carryover, but making a starter with just a couple table spoons of slurry would probably help with that as well.