Brian,Gotta scoop the punky wood out. The remaining hard wood should be very persistent, and will harden more when exposed.
Curly leaves in fall = blooms in winter. I like the tree, but not sure I can see where you're taking this design.
Let me know when you get tired of that pot...