Hmmm, my guess is that somewhere...maybe at the top, your face is a little open. When you bend the shaft in the transition, most people will have an opening influence on the face. If you dont naturally downarch your left wrist, make sure the face is okay. Then do a youtube search for Brian show, i think episode 2(?). It talks alot about the "umbrella drill" and how to get some bend or stress in the shaft.
Another possibility is your backswing could be a little arm dominated and you're getting a little rebound at the top resulting in a loss of force into the ground.