Yeah I had that big time. It all disappeared (to some extent) after we we doing some non-contact one step sparring in TSD and the lady I was partnered against (who clearly had no such reservations about punching people) smacked me full on in the face. She said sorry but I think she enjoyed the doing it a lot more than she was sorry.
I thought 'o.k, you're getting that back at some point went we spar.'
By the way, welcome to the forum! Where did you get your name from, are you a Gnarls Barkley fan?