Write the integrand as: #sec^5(x) = sec^2(x) sec^3(x)# and integrate by parts considering that:
#d/dx (tanx) = sec^2(x) #,
so:
#int sec^5x dx = int sec^2(x) sec^3(x)dx#
#int sec^5x dx = int sec^3(x)d(tanx)#
#int sec^5x dx = tanxsec^3x - int tanx d(sec^3(x))#
and as:
#d/dx (sec^3(x)) = 3sec^2(x) d/dx sec(x) = 3sec^3(x) tanx#
we have:
#int sec^5x dx = tanxsec^3x - 3int tan^2x sec^3x dx#
use now the trigonometric identity:
#tan^2 theta = sin^2 theta/cos^2 theta = (1-cos^2 theta)/cos^2theta = sec^2theta -1#
to have:
#int sec^5x dx = tanxsec^3x - 3int (sec^2x -1) sec^3x dx#
and using the linearity of the integral:
#int sec^5x dx = tanxsec^3x + 3int sec^3x dx -3 int sec^5x dx#
The integral now appears on both sides of the equation and we can solve for it obtaining a reduction formula:
#int sec^5x dx = 1/4(tanxsec^3x + 3int sec^3x dx)#
Solve now the resulting integral with the same procedure:
#int sec^3x dx = int secx d(tanx)#
#int sec^3x dx = tanxsecx - int tanx d(secx)#
#int sec^3x dx = tanxsecx - int tan^2x secx dx#
#int sec^3x dx = tanxsecx - int (sec^2x-1) secx dx#
#int sec^3x dx = tanxsecx + int secx dx - int sec^3x dx#
#int sec^3x dx = 1/2(tanxsecx + int secx dx)#
To solve the resulting integral note that:
#d/dx (tanx + secx) = sec^2x +secx tanx = secx(tanx+secx)#
so dividing and multiplying the integrand by #(secx+tanx)#:
#int secx dx = int (secx(secx+tanx))/(secx+tanx) dx#
#int secx dx = int (d(secx+tanx))/(secx+tanx)#
#int secx dx = ln abs(secx+tanx) +C#
Putting it all together:
#int sec^5x dx = (2tanxsec^3x+ 3tanxsecx + 3ln abs(secx+tanx))/8 +C#