It's way less than ideal to do it that way, I think the beer will be either flat or practically impossible to put in a growler without a lot of trouble. You could chill the keg and then bottle the whole batch. Or you could get some of those mugs you put in the freezer where the sides are filled with water, those work well. Or you could build a jockey box and just throw some ice in it when you want to pour a beer.
Ok. Well, if I use my fermentation freezer to carbonate but then remove the keg and keep it at room temperature will that mess up the carbonation?
If you are serving at room temp, yes - as it warms up the co2 will be less soluble, so some of it will come out of solution. You'll end up with higher pressure in the keg headspace and your beer will be flatter and pour foamy.
If you serve at carb temp/pressure with a period of warmth in the middle, it will not affect the carbonation.