Here ,
#I=intarc cosx*xdx#
Using Integration by parts:
#I=arc cosx intxdx-int(d/(dx)(arc cosx)intxdx)dx#
#I=arc cosx(x^2/2)-int(-1)/sqrt(1-x^2)xxx^2/2dx#
#I=x^2/2arc cosx+1/2intx^2/sqrt(1-x^2)dx#
#color(red)(I=x^2/2arc cosx+1/2I_1............to(A)#
where, # I_1=intx^2/sqrt(1-x^2)dx#
Subst. #color(blue)(x=sinu=>dx=cosudu#
So,
#I_1=intsin^2u/sqrt(1-sin^2u)cosudu#
#I_1=intsin^2u/cosucosudu#
#I_1=intsin^2udu#
#I_1=int(1-cos2u)/2du#
#I_1=1/2[u-(sin2u)/2]+c#
#I_1=1/2[u-sinucosu]+c#
#I_1=1/2[u-sinusqrt(1-sin^2u)]+c#
Subst. back #color(blue)(sinu=x and u=arcsinx#
#I_1=1/2[arc sinx-xsqrt(1-x^2)]+c#
Subst. value of #I_1# into #color(red)((A)# we get
#I=x^2/2arc cosx+1/2{1/2[arc sinx-xsqrt(1-x^2)]}+C#
#I=x^2/2arc cosx+1/4arc sinx-x/4sqrt(1-x^2)+C#