Baby not moving as much as usual? Should I be concerned?

I'm 20 weeks pregnant and for the past week and a half, I've felt the baby move on a regular basis. He's been very active with strong kicks and jabs in my lower abdomen and when he does wake up, he moves around for a decent length of time, however for the past few days I've been sick with a stomach virus and yesterday morning I took a Phenergan pill to help with my nausea/vomiting and he hasn't moved much ever since. The only movements I feel are very small compared to how active he was being. It just feels like he is barely moving at all and not as much. Is the change in his activity level something I should be concerned about or is this normal? Could the stomach virus/Phenergan have affected him in some way?
Just wanted to throw in there, Emerald, the Phenergan was prescribed by my ob/gyn to help with the nausea and vomiting. I would never take anything like that without talking with my Dr. first. I seen my Dr. yesterday and was told that with the viral infection, all I could do was wait it out. Just to rest up and drink plenty of fluids and take Phenergan to help with the nausea.