Why might nuclear energy be controversial?
There are numerous disadvantages.
Nuclear energy can be a very enticing prospect as a replacement for fossil fuels, and it was actually really popular in the 20th century.
Here's some (many) downsides. In order to produce the rods used in nuclear reactors, uranium or plutonium has to be mined, which is as bad as mining coal or fracking for oil. Refinement is also costly process.
Then there's the waste. Nuclear reactors cost millions to build (many times more expensive than building solar panels, windmills or coal-fired power plants), and once they can't be used anymore, it costs millions to take them apart. All of the materials that come into contact with radioactive materials can't be recycled, so they have to be stored somewhere safe. Along with those are the used uranium/plutonium rods, which won't decay for thousands of years. Burying it is costly, and risks contaminating groundwater, escaping containment, or being stolen for malicious purposes.
In the containment shell of a nuclear reactor, water is used to cool the fuel rods. More water is pumped from outside to cool that water (the cooling process happens through a barrier, so the outside water doesn't become radioactive). This hot water is released into an ocean/river/lake, and the heat (thermal pollution) can kill kelp, fish, and other aquatic life. It's better than a coal power plant's emissions, but it's not at all harmless.
Most scientists are in agreement that sustainable energy is not going to come from nuclear energy (especially given the fact that nuclear energy actually isn't sustainable since plutonium and uranium are limited resources). Sources such as biofuels and hydrogen fuel cells, solar, wind, and wave/tidal power are much more viable alternatives to coal, oil, and natural gas.
Mainly because it's not properly understood.
If you are not a trained scientist most of what is told to you about nuclear power will come from either printed media, online information, or TV programs, all of which are made by people (who are often not scientists either) and all of whom have political agendas.
If you want to, you can take anything and pump it up into a hysterical chest beating "oh its terrible its going to kill everyone" propaganda fest.
The actual science is far less sensational, far less dramatic, but unfortunately (especially in these days of "4 second attention spans" and "never mind all the details just give me the basics") not something that will be of interest to the public at large.
That's why it's controversial. What the perceived advantages or disadvantages are is a different question.