Given: #intcos^2xsin^5xdx#

Let #u=cosx=>color(blue)(du=-sinxdx#

We can strip a #sinx# out of the integral and rewrite the integral as

#color(blue)(-)intcos^2xsin^4xcolor(blue)(sinxdx#

Simplify

#color(blue)(-)intcos^2x(sin^2x)^2color(blue)(sinxdx#

Note: I will not make the substitution right away, I will still have to manipulate the integral to make it easier to solve. With that being said, whatever is in #color(blue)("blue"# will not be involved in the following procedure until it's time to make the subsitution.

Convert the sines to cosines

Use the identity #cos^2x+sin^2x=1=>sin^2x=1-cos^2x#

#color(blue)(-)intcos^2x(1-cos^2x)^2color(blue)(sinxdx#

Expand #(1-cos^2x)^2#

#color(blue)(-)intcos^2x(1-2cos^2x+cos^4x)color(blue)(sinxdx#

Distribute #cos^2x# to each term

#color(blue)(-)intcos^2x-2cos^4x+cos^6xcolor(blue)(sinxdx#

Now I will make use of the substitution:

#-intu^2-2u^4+u^6du#

Integrating each term we get...

#=-[u^3/3-(2u^5)/5+u^7/7]+C#

#=-u^3/3+(2u^5)/5-u^7/7+C#

Reverse the substiution:

#=-cos^3x/3+(2cos^5x)/5-cos^7x/7+C#