Let's rewrite it like this:
#int (x^2)/(sqrt(x^2-4))dx#
With this, #a^2 = 4# so #a = 2#. We have the form:
#sqrt(x^2 - a^2)#
which looks like:
#sqrt((asectheta)^2 - a^2) = sqrt(a^2sec^2theta - a^2) = asqrt(sec^2theta-1) = atantheta = 2tantheta#
So:
Let #x = asectheta = 2sectheta#
#dx = asecthetatanthetad##theta = 2secthetatanthetad##theta#
#int x^2/(sqrt(x^2-4))dx = int (4sec^2theta)/(cancel(2tantheta))*cancel(2)secthetacancel(tantheta)d##theta#
#= int 4sec^3thetad##theta#
#= 4int sec^2thetasecthetad##theta#
From here, we have a pretty difficult integral... We have to use a trick with integration by parts. Notice how this can be written as:
#= 4int (1+tan^2theta)secthetad##theta#
Let:
#u = sectheta#
#v = tantheta#
#du = secthetatanthetad##theta#
#dv = sec^2thetad##theta#
#= 4[overbrace(secthetatantheta)^(u*v) - intoverbrace(tantheta)^(v)overbrace(secthetatanthetadɵ)^(du)]#
#= 4secthetatantheta - 4int(sec^2theta - 1)secthetad##theta#
#= 4secthetatantheta - 4intsec^3theta - secthetad##theta#
#= 4secthetatantheta - 4intsec^3thetad##theta + 4intsecthetad##theta#
#= 4secthetatantheta + 4ln|tantheta+sectheta| - 4intsec^3thetad##theta#
where we already are supposed to know that #intsecthetad##theta = ln|tantheta+sectheta|#. You should have been shown this in class prior to doing this integral.
Now here's the weird part. The whole time we were solving #intsec^3thetad##theta#, so now we can explicitly say that and use that knowledge.
#4int sec^3thetad##theta = 4secthetatantheta + 4ln|tantheta+sectheta| + C - 4intsec^3thetad##theta#
#8int sec^3thetad##theta = 4secthetatantheta + 4ln|tantheta+sectheta| + C#
#int sec^3thetad##theta = 1/2secthetatantheta + 1/2ln|tantheta+sectheta| + C#
where #C# is now a new constant, #1/8C#, which we don't need to know.
And finally, we can plug things back in.
#sectheta = x/2#
#tantheta = sqrt(x^2-4)/2#
We get:
#= x/4(sqrt(x^2-4))/2 + 1/2ln|sqrt(x^2-4)/2+x/2| + C#
