As #cos2A=2cos^2A-1#, we have
#cos4A=cos2(2A)=2cos^2 2A-1#
= #2(2cos^2A-1)^2-1#
= #2(4cos^4A-4cos^2A+1)-1#
= #8cos^4A-8cos^2A+1#
Hence #r^2=cos4theta# is same as #r^2=8cos^4theta-8cos^2theta+1#
Now relation between #(r,theta)# in polar coordinates and #(x,y)# in Cartesian coordinates is given by
#x=rcostheta# and #y=rsintheta# i.e. #r^2=x^2+y^2# and hence #costheta=y/sqrt(x^2+y^2)#
and #r^2=8cos^4theta-8cos^2 2theta+1#
#hArrx^2+y^2=8(y/sqrt(x^2+y^2))^4-8(y/sqrt(x^2+y^2))^2+1#
or #x^2+y^2=(8y^4)/(x^2+y^2)^2-(8y)/(x^2+y^2)+1#
or #(x^2+y^2)^3=8y^4-8y(x^2+y^2)+(x^2+y^2)^2#