How does the port taste? They are still in the port or just recently removed, right?
I don't see why you can't use them, port often sits in barrels for a lot longer than 2 years. Granted there isn't such a high surface/volume ratio, but that will more affect the flavor of the port than the flavor imparted from putting the cubes in something else. If you're really worried do a small test on a gallon of something, but I don't think it will be a problem.