No you shouldn't be worried unless you start bleeding really badly. It's just something that happens. Next time use lube and that should help a bit. I'm about 28 weeks and my bf and I still have sex, and if we dont use lube, it hurts a lot worse!! So try that and if it's just too painful then I would just stop all together.
-Rebecca