I think you have the right approach. You can add the spirit in doses. It will have extracted some wood flavor already, so you don't really need to add the wood directly. The spirit may have extracted the harshest tannins from the oak, however. I'd give it a taste to see.
You could also just add the wood for a bit, see how it comes out, and then dose with the spirit.