Before we get started, let's remember the essential equation for converting between Cartesian and polar coordinates:
#x = rcos(theta), y = rsin(theta) #
So once we have x and y, we can substitute in. Hence,
#2 = (8y-x)^2 - 5y - x#
#2 = 64y^2 - 16xy + x^2 - 5y - x #
#2 = 64r^2sin^2theta - 16r^2 sin theta cos theta + r^2 cos^2 theta - 5r sin theta - r cos theta #
#2=r^2(64sin^2 theta + cos^2 theta - 16 cos theta sin theta) - r (5sin theta + cos theta) #
This is kinda the end if we want to have an implicit answer. However, if we want to have a formula for #r# in terms of #theta# we need to realize that the above is a quadratic. So that the formula will fit on one line, let #c = cos(theta)# and #s = sin(theta)#:
#r = ((5s+c) pm sqrt((5s+c)^2 + 4 cdot 2 cdot (64 s^2 + c^2 - 16sc)))/(2(64s^2 + c^2 - 16 sc)) #
#r = ((5s+c) pm sqrt((5s+c)^2 + 8(8s-c)^2))/(2(8s-c)^2) #
yielding two curves.
