I would say it will continue moving in a straight line with a velocity equal to the last velocity it had at the moment the acceleration ceased.
I would think at this situation as having a rock tied to a string and made to go around in a circle as we hold one end of the string. If we release the string the rock will lose the centripetal force (string tension) resposible for the acceleration and it will fly tangentially away from the original circular trajectory. :
Hope it helps!