You do not really need a trigonometric substitution here. As:
#d(3+x^2) = 2xdx#
you can just substitute:
#t = 3+x^2#
and have:
#int x sqrt(3+x^2) dx = 1/2 int sqrt(t) dt = 1/3t^(3/2)+C = 1/3(3+x^2)sqrt(3+x^2) +C#
However if you want to perform a trigonometric substitution, the right substitution when under the square root there is a sum of square is:
#x = sqrt(3)tant# with #t in (-pi/2,pi/2)#
#dx = sqrt(3)dt/(cos^2t)#
In this case we have:
#int x sqrt(3+x^2) dx = 3 int tant sqrt (3 + 3 tan^2t) (dt)/cos^2t = 3sqrt(3) int tant/cos^2t sqrt (1+tan^2t) dt#
we now use the trigonometric identity:
#1+tan^2t = 1/cos^2t#
and have:
#int x sqrt(3+x^2) dx = 3sqrt(3) int tant/cos^2t sqrt (1/cos^2t) dt#
Then we note that #cost > 0 # in the interval #t in (-pi/2,pi/2)#, so:
#int x sqrt(3+x^2) dx = 3sqrt(3) int tant/cos^3t dt = 3sqrt(3) int sint/cos^4t dt = -3sqrt(3) int (dcost)/cos^4t = sqrt(3)/cos^3t+ C#
To substitute back #x#, we need to express #cost# in terms of #tant#:
#cost = sqrt (1/(1+tan^2t)) = sqrt( 1/(1+x^2/3))= sqrt(3)/sqrt(3+x^2)#
so that:
#int x sqrt(3+x^2) dx = sqrt(3)/ (sqrt(3)/sqrt(3+x^2))^3 + C = 1/3(3+x^2)sqrt(3+x^2) + C#
