Yes, I'm afraid of it. Greatly so.
But a while ago I read post on another forum concerning the subject. It said that we shouldn't worry because we didn't exist before our birth either, and how did that feel? Well, nothing. So why be afraid if you won't even have the consciousness? It'll just be the same thing.
I'm not religious at all, but I do believe our presence moves to another body after death. The gap in-between, it'll just be nothing. When that nothing hits us, we won't even be afraid of it or worry about it or give a damn. Because we can't.