What do you think about a conversation with the social worker about your ex's behavior? You could ask her what she could do to help with his destructiveness. Only you could know if communicating that he is abusive is beneficial for you or not. It depends on her and her "agency". My experience is that everyone ignores men being abusive. Document everything. Keep a notebook. He is abusive and he is the same person with your son.
I just listened to this Dr. Ramni video on you tube called: "Are smear campaigns large scale gaslighting?". I recommend searching her other videos on "smear campaign". Smear campaign is done to us and is damage control for them. My ex spends excessive energy trying to hurt me this way. It is control and abuse. And since I got away from my now ex, people treat me terribly. I have done nothing and have kept to myself. A woman unattached is a huge problem for people. It is painful.
I am concerned that you said that you were not innocent because you did not connect with your then husband. We carry guilt, shame and blame that is not ours to carry. I don't want you to have one ounce of blame or responsibility carrying it around. He did this to you. I distanced myself as I was planning to escape. Boundaries were to protect me and help me have enough space to get me and my kids out of the situation.
Tell your son the truth in the spirit of your love for him. Gradually point out and identify and label what the behavior is that his dad is doing. It is best for him to know the truth about toxic behaviors. The focus is on the behavior. Again, I recommend Dr. Ramani videos (too on trying to co parent). She is great (compassion and a fierce defender of those who are dealing with these issues.) She gives us the language and knowledge to navigate and heal from the twisted, toxic behavior.
I wrote an older article on this platform about why I tell my kids the truth about their dad. They see it and understand the behaviors. They know how to deal with people like this. They have the knowledge and tools. And it will always serve them well because there are so many toxic people that we encounter on our paths. It is amazing to see them identifying these issues with other people. It is a beautiful thing to empower our kids and ourselves. It is social programming that says we can't speak "negatively" about a toxic parent.
I am sad that you have to endure this.