As a child, I thought that "fear itself" meant some imaginary creature like the boogie man. So my kid brain thought that we were told to not be afraid of real things, but to instead fear the things that can't hurt you.
As an adult, I finally get it. No matter how bad things get, the human mind can adjust to it. The person you love most could spurn you. You could lose your job. You could lose your leg. And in any (or all) of these cases, you don't die. You wake up the next day, still alive, and still with more of your story to be written.
The number one thing that stops us from enjoying ourselves is our own fear. Not people with guns, not rabid gorillas, merely the fear of social discomfort. And the reality is that when somebody strongarms you into doing something you're afraid of, no matter if it ended up good, bad, or indifferent, it was not nearly as bad as you thought it was going to be.
And now we have the title: nothing is as scary as your own fear. However nasty it is, it's never going to be as bad as our imagination can come up with. Use that knowledge to get over it.