A conic equation of the type of #Ax^2+Bxy+Cy^2+Dx+Ey+F=0# is rotated by an angle #theta#, to form a new Cartesian plane with coordinates #(x',y')#, if #theta# is appropriately chosen, we can have a new equation without term #xy# i.e. of standard form.
The relation between coordinates #(x,y)# and #(x'.y')# can be expressed as
#x=x'costheta-y'sintheta# and #y=x'sintheta+y'costheta#
or #x'=xcostheta+ysintheta# and #y=-xsintheta+ycostheta#
for this we need to have #theta# given by #cot2theta=(A-C)/B#
In the given case as equation is #x^2+4xy-2y^2-6=0#, we have #A=1#, #C=-2# and #B=4# and hence #cot2theta=3/4# i.e. #(cot^2theta-1)/(2cottheta)=3/4#, which gives us #4cot^2theta-6cottheta-4=0# giving us #cottheta=2# or #-1/2#.
But as #cot2theta>0#, #2theta# is in #Q1# and so is #theta#. Hence, #cottheta=2# which gives us #costheta=2/sqrt5# and #sintheta=1/sqrt5#
Hence relation is give by #x=2/sqrt5x'-1/sqrt5y'# and #y=1/sqrt5x'+2/sqrt5y'#
Hence, we get the equation as #(2/sqrt5x'-1/sqrt5y')^2+4(2/sqrt5x'-1/sqrt5y')(1/sqrt5x'+2/sqrt5y')-2(1/sqrt5x'+2/sqrt5y')^2-6=0#
or #((4x'^2)/5+(y'^2)/5-4/5x'y')+4((2x'^2)/5-(2y'^2)/5+3/5x'y')-2((x'^2)/5+(4y'^2)/5+4/5x'y')-6=0#
or #2x'^2-3y'^2-6=0#
or #2x^2-3y^2-6=0#
As coefficients of #x^2# and #y^2# are of different sign, it is a hyperbola.
The two graphs are as follows:
graph{x^2+4xy-2y^2-6=0 [-10, 10, -5, 5]}
and
graph{2x^2-3y^2=6 [-10, 10, -5, 5]}
