A conic equation of the type of #Ax^2+Bxy+Cy^2+Dx+Ey+F=0# is rotated by an angle #theta#, to form a new Cartesian plane with coordinates #(x',y')#, if #theta# is appropriately chosen, we can have a new equation without term #xy# i.e. of standard form.
The relation between coordinates #(x,y)# and #(x'.y')# can be expressed as
#x=x'costheta-y'sintheta# and #y=x'sintheta+y'costheta#
or #x'=xcostheta+ysintheta# and #y=-xsintheta+ycostheta#
for this we need to have #theta# given by #cot2theta=(A-C)/B#
In the given case as equation is #x^2+2sqrt3xy-y^2=7#, we have #A=1#, #B=2sqrt3# and #C=-1# and hence #cot2theta=2/(2sqrt3)=1/sqrt3# i.e. #theta=pi/6#
Hence relation is give by #x=x'cos(pi/6)-y'sin(pi/6)# and #y=x'sin(pi/6)+y'cos(pi/6)# i.e.
#x=(x'sqrt3)/2-(y')/2# and #y=(x')/2+(y'sqrt3)/2#
and #x^2+2sqrt3xy-y^2=7# is
#((x'sqrt3)/2-(y')/2)^2+2sqrt3((x'sqrt3)/2-(y')/2)((x')/2+(y'sqrt3)/2)-((x')/2+(y'sqrt3)/2)^2=7#
which simplifies to #2(x'^2-y'^2)=7# or we can say
#2x^2-2y^2-7=0#
graph{(2x^2-2y^2-7)(x^2+2sqrt3xy-y^2-7)=0 [-10, 10, -5, 5]}
