Calculate the first and second derivative by the product rule
(uv)'=u'v-uv'
f(x)=x^3e^x
f'(x)=3x^2e^x+x^3e^x=(x^3+3x^2)e^x
f''(x)=(3x^2+6x)e^x+(x^3+3x^2)e^x
=x(x^2+6x+6)e^x
The points of inflections are when, f''(x)=0
x(x^2+6x+6)e^x, e^x>0
=>, x(x^2+6x+6)=0
=>, {(x=0),(x^2+6x+6=0):}
=>, x=(-6+-sqrt(36-24))/(2)=-3+-sqrt3
There are 3 points of inflections
Therefore,
There are 4 intervals to consider are
I_1=(-oo,-3-sqrt3) and I_2=(-3-sqrt3, -3+sqrt3) and I_3=(-3+sqrt3,0) and I_4=(0,+oo)
Let's consider a variation chart
color(white)(aaaa)"Interval"color(white)(aaaaaa)I_1color(white)(aaaaa)I_2color(white)(aaaa)I_3color(white)(aaaa)I_4
color(white)(aaaa)"sign f''(x)"color(white)(aaaa)-color(white)(aaaa)+color(white)(aaaa)-color(white)(aaa)+
color(white)(aaaa)" f(x)"color(white)(aaaaaaaa)nncolor(white)(aaaa)uucolor(white)(aaaa)nncolor(white)(aaaa)uu
The function is concave for x in (-oo,-3-sqrt3)uu(-3+sqrt3,0) and convex for x in (-3-sqrt3, -3+sqrt3)uu(0,+oo)
graph{x^3e^x [-10, 10, -5, 5]}