Sorry to be bringing this thing back to life but it piqued my interest.
Hazza, I had exactly the same problem, and still get the odd twinge when picking up that phone or making a call. As you said, it's that fear of judgement (self-imposed and/or from others) which presents the biggest problem. In my case I think it's to do with being a perfectionist: there's no way you're going to have a flawless phone call and this just makes you more flustered as the call goes on. After a bad call that's all you'll remember, and you won't want to pick up the damn phone ever again.
But fear not! When you said you wanted to be able to do it like everyone else, I felt compelled to let you know that the problem is very common. I worked in a customer service call centre until recently, and in plenty of those inbound calls I could tell that the person on the other end wasn't comfortable on the phone. You are certainly not alone! Keep it in mind that these operators or whoever you need to call aren't thinking any less of you: they deal with it all the time and it's their job to make you feel at ease on the phone.
Finally, the best way to get through this sort of thing is just to bite the bullet and do it. Make calls: keep making them until you think you're going to chew through the cord. Eventually you'll be desensitised to it. In my past job I didn't have a choice: I came in one day and found out that we were moving from office work to telephony cover. Yes, I couldn't stand it at first, but after about a month it became second nature. Brutal but effective.
Good luck whatever you do, Hazza!