int x^4*sqrt(x^2-4)*dx
After using x=2secu and dx=2secu*tanu*du transforms, this integral became,
int (2secu)^4*sqrt[(2secu)^2-4]*2secu*tanu*du
=int 32(secu)^5*tanu*sqrt[4(tanu)^2]*du
=int 64(secu)^5*(tanu)^2*du
=int 64(secu)^5*[(secu)^2-1]*du
=int 64(secu)^7*du-int 64(secu)^5*du
After using int (secu)^n*du=1/(n-1)*(secu)^(n-2)*tanu+(n-2)/(n-1)*int (secu)^n*du reduction formula,
=int 64(secu)^7*du-int 64(secu)^5*du
=64*[1/6*(secu)^5*tanu+5/6*int (secu)^5*du]-int 64(secu)^5*du
=32/3*(secu)^5*tanu+160/3*int (secu)^5*du-int 64(secu)^5*du
=32/3*(secu)^5*tanu-32/3*int (secu)^5*du
=32/3*(secu)^5*tanu-32/3*[1/4*(secu)^3*tanu+3/4*int (secu)^3*du]
=32/3*(secu)^5*tanu-8/3*(secu)^3*tanu-8int (secu)^3*du
=32/3*(secu)^5*tanu-8/3*(secu)^3*tanu-8*[1/2*secu*tanu+1/2*int secu*du]
=32/3*(secu)^5*tanu-8/3*(secu)^3*tanu-4secu*tanu-4int secu*du
=32/3*(secu)^5*tanu-8/3*(secu)^3*tanu-4secu*tanu-4ln(secu+tanu)+C1
After using x=2secu, secu=x/2 and tanu=sqrt(x^2-4)/2 inverse transformations, I found
int x^4*sqrt(x^2-4)*dx
=(1/6*x^5-1/6*x^3-x)*sqrt(x^2-4)-4ln[(x+sqrt(x^2-4))/2]+C1
=1/6*(x^5-x^3-6x)*sqrt(x^2-4)-4ln(x+sqrt(x^2-4))+C
Note: C=C1+4Ln2