All the terms are multiples of 6y, so we find:
6x^2y+36xy-66y = 6y(x^2+6x-11)
x^2+6x-11 is of the form ax^2+bx+c with a=1, b=6 and c=-11, which has discriminant given by the formula:
Delta = b^2-4ac = 6^2-(4xx1xx-11) = 36+44 = 80
This is positive, but not a perfect square. So the zeros of x^2+6x-11 are not rational and its linear factors will have irrational coefficients.
The roots of x^2+6x-11 = 0 are given by the formula:
x=(-b+-sqrt(Delta))/(2a) = (-6+-sqrt(80))/2
=(-6+-4sqrt(5))/2
=-3+-2sqrt5
This gives us factors (x+3+2sqrt(5)) and (x+3-2sqrt(5))
So the full factorization (if we allow irrational coefficients) is:
6x^2y+36xy-66y = 6y(x+3+2sqrt(5))(x+3-2sqrt(5))