With the rotation
#R_(theta)=((ctheta,-stheta),(stheta,ctheta))#, calling #p = ((x),(y))# and #P=((X),(Y))# we have first
#p^T M p + 10 = 0# representing the conic, with #M = ((2,-3/2),(-3/2,-2))#. Here #.^T# represents transposition. Now introducing a change of coordinates
#R_(theta)P = p# we will have
#P^TR_(theta)^TMR_(theta)P + 10 = 0#
Calling #M_(theta) = R_(theta)^TMR_(theta)=((2 Cos(2 theta) - 3 Cos(theta) Sin(theta), -3/2Cos(2 theta) -
2 Sin(2 theta)),(-3/2 Cos(2 theta) - 2 Sin(2 theta), -2 Cos(2 theta) +
3 Cos(theta) Sin(theta)))#
Now, choosing #theta# such that #-3/2 Cos(2 theta) - 2 Sin(2 theta)=0# we obtain
#tan(2theta)=-3/4# and
#theta = 1/2(-0.643501pmkpi)# Choosing for #k=0# we have
#theta =-0.321751# and
#M_(theta) = ((5/2,0),(0,-5/2))# and in the new coordinates, the conic reads
#5/2X^2-5/2Y^2+10=0# or
#X^2-Y^2+4=0# which is a hyperbola.
