Calculate the first and second derivative by the product rule
#(uv)'=u'v-uv'#
#f(x)=x^3e^x#
#f'(x)=3x^2e^x+x^3e^x=(x^3+3x^2)e^x#
#f''(x)=(3x^2+6x)e^x+(x^3+3x^2)e^x#
#=x(x^2+6x+6)e^x#
The points of inflections are when, #f''(x)=0#
#x(x^2+6x+6)e^x#, #e^x>0#
#=>#, #x(x^2+6x+6)=0#
#=>#, #{(x=0),(x^2+6x+6=0):}#
#=>#, #x=(-6+-sqrt(36-24))/(2)=-3+-sqrt3#
There are #3# points of inflections
Therefore,
There are #4# intervals to consider are
#I_1=(-oo,-3-sqrt3)# and #I_2=(-3-sqrt3, -3+sqrt3)# and #I_3=(-3+sqrt3,0)# and #I_4=(0,+oo)#
Let's consider a variation chart
#color(white)(aaaa)##"Interval"##color(white)(aaaaaa)##I_1##color(white)(aaaaa)##I_2##color(white)(aaaa)##I_3##color(white)(aaaa)##I_4#
#color(white)(aaaa)##"sign f''(x)"##color(white)(aaaa)##-##color(white)(aaaa)##+##color(white)(aaaa)##-##color(white)(aaa)##+#
#color(white)(aaaa)##" f(x)"##color(white)(aaaaaaaa)##nn##color(white)(aaaa)##uu##color(white)(aaaa)##nn##color(white)(aaaa)##uu#
The function is concave for #x in (-oo,-3-sqrt3)uu(-3+sqrt3,0)# and convex for #x in (-3-sqrt3, -3+sqrt3)uu(0,+oo)#
graph{x^3e^x [-10, 10, -5, 5]}