For f(x) a function of x, we can denote the derivative in several ways.
On family of notations uses a "differential operator".
This is a notation that tells us "take the derivative".
For example D_x(5x^2+3) means "The derivative, with respect to x, of the function f(x)=5x^2+3. SO D_x(5x^2+3) = 10x.
d/dx is another notation for "the derivative with respect to x of . . . #
So d/dx(5x^2+3) means the same thing as D_x(5x^2+3).
To take the second derivative, we take the derivative twice:
D_x(D_x(5x^2+3)) is written D_x^2(5x^2+3)
d/dx(d/dx(5x^2+3)) could, perhaps, be written (d/dx)^2(5x^2+3), but the standard notation is d^2/dx^2(5x^2+3).
If we let y = 5x^2+3, then these examples look like:
First derivative: D_x(y) and d/dx(y) which is also written dy/dx.
Second derivative: D_x^2(y) and d^2/dx^2(y) which is also written (d^2y)/dx^2.
In this notation, we do not think of dx as d times x
Instead, we are thinking of dx as a single quantity. That is why we do NOT write d^2/(dx)^2(y)
Additional Note
I would interpret dy^2/dx as d/dx(y^2). Using the chain rule, I would evaluate to get d/dx(y^2) = 2y * dy/dx.
Using y=5x^2+3 again, we have d/dx(5x^2+3)^2 = 2(5x^2+3)*5 = 10(5x^2+3)
Note 2
(dy/dx)^2 means the square of the derivative of y.
Using y=5x^2+3 again, we have
(dy/dx)^2 = (10x)^2 = 100x^2.
This kind of expression can come up if using implicit differentiation to find 2nd or higher derivatives. (If we don't pause to solve for dy/dx before our second differentiation.)