Am I being worried for no reason? Or is it normal to feel the way I do?

Since communication is one of the best aspects of your present relationship I think you should discuss your feelings with your girlfriend. If I were in the same situation I certainly would feel at least a little uncomfortable, but you say that your present girlfriend wouldn't believe things the ex may say anyway. Maybe your ex has matured some and has gotten over your break-up. Good luck and God Bless.