Yes. Coffee flavor fades with time, but it does not go away. I don't mind a coffee stout with some age on it. I think the coffee becomes less prominent, but it's still there. You'll lose some of the aromatics, mostly, in my experience.
How are you adding the coffee? There are many ways to do so, none of them wrong. I prefer to add cracked beans to the fermenter/keg. My coffee guy tells me you'll get everything from the beans in 24 hours, but I usually leave them longer.