This is a good question, but a little bit vague. I'll try to guess:
When you say, "a conic," do you (1) have an equation and you want to rotate the axes to get a simpler equation? Or do you (2) have a graph or some points that the curve goes through?
The general conic equation is a relation between x and y:
#A x^2 + B x y +C y^2 + Dx + Ey + F = 0#
(1) • If you want to rotate this curve to "get rid of" the xy-term, you can substitute coordinates u and v, rotated by angle #theta#, where
#u = x cos theta + y sin theta#
#v = -x sin theta + y cos theta#
or to substitute in the general conic equation,
#x = u cos theta - v sin theta#
#y = u sin theta + v cos theta#
If you use the specific angle #theta# where
#cot (2 theta) = (A-C)/B#
then you will get an equation in u and v that has no xy-term.
(2) • If you just have some points that the conic goes through, you need five points to determine a unique conic. You then have 5 equations in the 6 variables A, B, C, D, E, F.
They are scalable so this determines a unique conic. For example #x^2 + y^2 - 25# is the same circle as #3x^2 + 3y^2 - 75 = 0#.
Note: If you have six or more points, they may not all lie on the same conic.
// dansmath strikes again! //
p.s. Here's the story, and an example, from Stewart's Calculus: