Should the south have been punished for seceding from the union?
As with any war, the Civil War was started by southern politicians, not the people themselves. But starting in 1863 at Vicksburg MS and continuing through the end of the war in 1865 the south was ravaged by war. Sherman's march to the sea used a scorched earth attitude towards everything which was in his way.
With the war ended, all southern politicians and many high ranking military leaders were banned from public office. There was no need for further punishments of any sort.
Andrew Johnson and the US Congress had the task of rebuilding the south and unifying the country. Any further punishments would only have made that much more difficult.