Stay away from bad people and keep your kids away from bad people. You can't control your mother, and if she continues to associate with this loser, she can do it without her children and grandchildren. Tell her that you will come a different time when he won't be around or that she can come visit you sometime without him.
This isn't about sending a message to either of them. It's about protecting yourself and your kids.