I have had this happen with mead. No worries, just replace with a clean sanitized airlock if it seems like it is not getting more active. If its getting more active run a blow off tube into some sanitizing solution.
I assume this is secondary?
I'm not sure if the ongoing fermentation was expected by you or not. I fill carboys up to the neck for aging mead, for instance, only if I'm reasonably certain that all fermentation has stopped.
If I'm expecting more activity in secondary I leave a bit of space to avoid this and with the assurance that the ongoing activity will fill the space with co2. The purpose of filling into the neck is to make the surface area potentially exposed to oxygen tiny. If its going to be still producing co2 that's not necessary.