First remember that #cos^2(2x) = 1-sin^2(2x)# so we can rewrite the first part of the integrand as
#cos^3(2x) = cos^2(2x)*cos(2x) = (1-sin^2(2x))*cos(2x)#
Also we'll break this into two integrals:
#int (1-sin^2(2x))*cos(2x)dx - int sin(6x)dx#
For the first integral, let #u = sin(2x)# so #du = 2cos(2x)dx#, which means that #cos(2x)dx = 1/2 du#. So our first integral becomes:
#1/2 int (1-u^2)du = 1/2(u-1/3u^3)+C#
So the first integral gives
#1/2sin(2x)-1/6sin^3(2x)+C#
For the second integral, we let #u=6x# so #du = 6dx# or #dx=1/6du#. So our integral can be rewritten as:
#1/6int sin(u)du = -1/6cos(u)+C#
which gives #-1/6cos(6x)+C#
Now we combine the answers (and combine C in such a way that we just get a new #+C# at the end):
#int (cos^3(2x)-sin(6x))dx = 1/2sin(2x)-1/6sin^3(2x) - (-1/6cos(6x)) +C#
#=1/2sin(2x)-1/6sin^3(2x) +1/6cos(6x) +C#