You do not really need to use trigonometric substitutions.
As:
#d/dx (sqrt(2x^2+4)) = (2x)/sqrt(2x^2+4)#
we can integrate by parts:
#int x^3/sqrt(2x^2+4)dx = int x^2/2 d/dx (sqrt(2x^2+4))dx#
#int x^3/sqrt(2x^2+4)dx = ( x^2sqrt(2x^2+4))/2 - int d/dx(x^2/2) sqrt(2x^2+4)dx#
#int x^3/sqrt(2x^2+4)dx = ( x^2sqrt(2x^2+4))/2 - int xsqrt(2x^2+4)dx#
Solve the resulting integral by substitution:
#t = 2x^2+4#
#dt = 4xdx#
#int xsqrt(2x^2+4)dx = 1/4 int sqrt t dt#
#int xsqrt(2x^2+4)dx = 1/4 t^(3/2)/(3/2)+ C = 1/6 t sqrt t +C#
and undoing the substitution:
#int xsqrt(2x^2+4)dx = (x^2+2)/3 sqrt(2x^2+4)+C#
Putting the results together:
#int x^3/sqrt(2x^2+4)dx = ( x^2sqrt(2x^2+4))/2 - (x^2+2)/3 sqrt(2x^2+4)+C#
and simplifying:
#int x^3/sqrt(2x^2+4)dx = (( x^2)/2 - (x^2+2)/3) sqrt(2x^2+4)+C#
#int x^3/sqrt(2x^2+4)dx = (3x^2 - 2x^2-4) sqrt(2x^2+4)/6+C#
#int x^3/sqrt(2x^2+4)dx = ((x^2 -4) sqrt(2x^2+4))/6+C#
