bp has one great solution Method 1. There are other solutions:
Both of the solution presented below use Integration by Parts.
I use the form:
#int u dv = uv-intvdu#.
Both of the solution presented below use #int lnx dx = xlnx - x +C#, which can be done by integration by parts. (And, of course, verified by differentiating the answer.)
Method 2
#int (lnx)^2 dx#
Let #u = (lnx)^2# and #dv = dx#.
Then #du = (2lnx)/x dx# and #v = x#
Integration by parts gives us:
#int (lnx)^2 dx = x(lnx)^2 - 2int lnx dx##
#color(white)"sssssss"# # =x(lnx)^2-2(xlnx - x) +C#
#color(white)"sssssss"# # =x(lnx)^2-2xlnx + 2x +C#
Method 3
#int (lnx)^2 dx = int (lnx)(lnx)dx#
Let #u=lnx# and #dv = lnx dx#
So, #du = 1/x dx# and #v= xlnx -x#
The parts formula gives us:
#int (lnx)^2 dx = (lnx)(xlnx -x)-int(xlnx-x)/x dx#
#color(white)"sssssss"# # =x(lnx)^2-xlnx -int (color(red)(lnx) - color(green)(1))dx#
#color(white)"sssssss"# # =x(lnx)^2-xlnx -(color(red)(xlnx-x) - color(green)(x)) +C#
#color(white)"sssssss"# # =x(lnx)^2-2xlnx +2x +C#