Then it's logical to conclude that you aren't taking the grip correctly. The heel of your lead hand is not on top of the club enough. When you get it right you will eliminate the grip movement and feel very in control the the club.
I've posted this before, but we will risk redundancy. I go for my one Brian lesson so far, and the first thing he zeroed in on was my grip. I have had a decent grip for at least 40 years, but it wasn't as good as Brian wanted it to be. We spent around 45 minutes, or so it seemed, going over exactly how to do the grip and check the clubface alignment.
One key element is holding the club up at a 45 degree angle to the ground, then placing the grip in the fingers of the lead hand with some diagonal orientation. Then the trick is wrapping your hand around the grip by leading with the heel of the hand. His method is to pretend you are shooting a water pistol with the lead hand index finger and the thumb. You take them off the grip and shoot the water to left. This makes the top of your hand go concave and gets the heel of your lead hand on top of the grip.
For a person used to a weaker grip, like me, it feels strong, but it's simply less weak.