Multiply both sides by #cos^2x#

#6sin^2x - 4sin^2xcos^2x = cos^2x#

Use the pythagorean identity #sin^2x = 1-cos^2x#

#6(1-cos^2x) -4(1-cos^2x)cos^2x = cos^2x#

Simplify

#6 - 6cos^2x -4cos^2x +4cos^4x = cos^2x#

#6 - 10cos^2x + 4cos^4x = cos^2x#

Subsitute #cos^2x# for #z#

#6 - 10z + 4z^2 = z#

Make one side equal zero

#4z^2 - 11z + 6 = 0#

Solve using the quadratic equation

#z = (11 +-sqrt(121 -4*4*6))/8#

#z = (11 +- sqrt(25))/8 = (11+-5)/8#

#z_+ = (11+5)/8 = 16/8 = 2#

#z_(-) =(11-5)/8 = 6/8 = 3/4 #

Subsitute #z# for #cos^2x#

#cos^2(x) = 2# is invalid because the square cosine has a range of #0 <= y <= 1#

Which leaves us with #cos^2x = 3/4 rarr cosx = +-sqrt(3)/2#

The cosine has one of those values on the first quadrant when #theta = 30º#, one of those values on the second quadrant when #theta = 180º - 30º = 150º#, one of those values on the third quadrant when #theta = 180º+30º = 210º# and one of the values on the fourth quadrant when #theta = 360º - 30º = 330º#

So, the set of solutions #S# is #S = {30º,150º,210º,330º}#