To help follow what's going on I'll put in red text things that have changed from the line before.
#cot^2theta/(sintheta+costheta)=(cos^2thetasintheta-cos^3theta)/(2sin^4theta-sin^2theta)#
I'll first use #cot^2theta=cos^2theta/sin^2theta# on the left side and factor the numerator and denominator on the right side. We can also see a #sintheta-costheta# term setting up in the right numerator and so I'll multiply through on the left using that term:
#((color(red)(cos^2theta/sin^2theta))/(sintheta+costheta))color(red)(((sintheta-costheta)/(sintheta-costheta)))=color(red)((cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1))#
#cos^2theta/((sin^2theta)(sintheta+costheta))((sintheta-costheta)/(sintheta-costheta))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
#(cos^2thetacolor(red)((sintheta-costheta)))/((color(red)(sin^2theta))(sintheta+costheta)color(red)((sintheta-costheta)))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
#(cos^2theta(sintheta-costheta))/((sin^2theta)color(red)((sin^2theta-cos^2theta)))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
We can now use #sin^2theta+cos^2theta=1 => cos^2theta=1-sin^2theta#
#(cos^2theta(sintheta-costheta))/((sin^2theta)(sin^2theta-color(red)((1-sin^2theta))))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
#(cos^2theta(sintheta-costheta))/((sin^2theta)(sin^2thetacolor(red)(-1+sin^2theta)))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
#(cos^2theta(sintheta-costheta))/((sin^2theta)color(red)((2sin^2theta-1)))=(cos^2theta(sintheta-costheta))/(sin^2theta(2sin^2theta-1)#
