Teaching ground manners to a tough QH?

I ride my boyfriend's aunts QH. He's an ex rodeo horse (barrels, poles, everything western), and was broken by a tough old man that tied him to a trailer and beat him. He's 14 years old, and a dream to ride. As long as he knows you know what your doing, he will respect you - when your riding. On the ground is a different story. He bites, kicks, tramples - you can't go into the pasture with him, and have to be very careful putting on his halter, as he likes to bite under your arm, or for women, your breasts. He has gotten an attitude with me before, while I was walking him into the barn, so I tightened my grip on his halter and gave him a firm "No sir!". He put his ears back up, though kept his lip tight. Well, I was trying to saddle him up alone (it usually takes 2) the other day, and he turned around to bite me (and he did) as a was tightening the cinch. I followed what my boyfriends aunt does with the other horses, yanked his halter down and said "NO!" He then turned to kick me, but I slipped under the fence before he could. How to stop this behavior? Round pens don't work, if you try and lunge him he will trample you. When I walk him on a lead line I have to swing it around to let him know if he cuts up ill smack him with it. What else can I do?