Given
#(a^2-b^2)sintheta+2abcostheta=a^2+b^2#
#=>(a^2-b^2)/(a^2+b^2)sintheta+(2ab)/(a^2+b^2)costheta=1.....(1)#
Now if #(a^2-b^2)/(a^2+b^2)=cosalpha#
then
#sinalpha=sqrt(1-cos^2alpha)#
#=sqrt(1- ((a^2-b^2)/(a^2+b^2))^2#
#=(2ab)/(a^2+b^2)#
So the relation (1) becomes
#cosalphasintheta+sinalphacostheta=1#
#=>sin(theta+alpha)=sin(pi/2)#
#=>alpha=pi/2-theta#
#=>cosalpha=cos(pi/2-theta)#
#=>cosalpha=sintheta#
#:.sintheta=cosalpha= (a^2-b^2)/(a^2+b^2)#
Alternative-1
Given
#(a^2-b^2)sintheta+2abcostheta=a^2+b^2#
Transferring to RHS we get
#a^2(1-sintheta)+b^2(1+sintheta)-2abcostheta=0#
#=>(asqrt(1-sintheta))^2+(bsqrt(1+sintheta))^2-2ab sqrt(1-sin^2theta)=0#
#=>[(asqrt(1-sintheta))-(bsqrt(1+sintheta))]^2=0#
#=>asqrt(1-sintheta)=bsqrt(1+sintheta)#
#=>a^2(1-sintheta)=b^2(1+sintheta)#
#=>sintheta=(a^2-b^2)/(a^2+b^2)#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Alternative-2
Putting #b=atanphi# in the relation (1) we get
#=>(a^2-b^2)/(a^2+b^2)sintheta+(2ab)/(a^2+b^2)costheta=1#
#=>(a^2-a^2tan^phi)/(a^2+a^2tan^2phi)sintheta+(2a*atanphi)/(a^2+a^2tan^2phi)costheta=1#
#=>cos2phisintheta+sin2phicostheta=1#
#=>sin(theta+2phi)=sin(pi/2)#
#=theta=pi/2-2phi#
#=sintheta=sin(pi/2-2phi)=cos2phi#
#=>sintheta=(1-tan^2phi)/(1+tan^2phi)#
#=>sintheta=(1-(b/a)^2)/(1+(b/a)^2)=(a^2-b^2)/(a^2+b^2)#