No, the exhaust gas is exerting a force on the rocket, not the atmosphere. It works the same in space as it does near the earth.
If I throw a ball toward you I must exert a force on the ball. Newton's third law says that the ball must exert an equal and opposite force on me. I push the ball toward you, but the ball pushes me away from you. But you know that I don't move, or I don't move very much. This is because I'm also exerting a force on the ground and the ground is exerting a force on me.
Here we must to get to the classic example of a frictionless frozen lake surface. If there is no frictional force between me and the surface I'm standing on, you will see that I start to move once I throw something. Take some time to think about this because you really never see frictionless surfaces.