Why do you think Kennedy lost popularity for supporting civil rights?
Civil Rights were very unpopualr among poor whites who voted for the Democratic Party in the South
The South had long supported the Democratic Party for being the opposite party to the Republican Party(the party of Lincoln who ravaged the South during the Civil War), Blacks had been disenfranchised because of several spiteful tricks such as literary tests.
Kennedy's support of the Civil Rights led many voters to desert the Democratic Party. For instance in 1964, four Deep South states voted in favor of Barry Goldwater, the Republican candidate who was very controversial but won the support of those states for opposing the enforcing of Civil Rights by the Federal Government.
George Wallace ended up running with his own party the Independent American Party with his slogan "segregation now, segregation tomorrow, segregation forever".