As #cotu=3# and we have #pi < u < (3pi)/2# i.e. #u# is in #Q3#
for #pi/2# we have #pi/2 < u/2 < (3pi)/4#
i.e. #u/2# is in #Q2# and while #sin(u/2)# is positive, #cos(u/2)# and #tan(u/2)# are negative.
As #cotu=3#, we have #tanu=1/3# and #cscu=-sqrt(1+cot^2u)=-sqrt10# i.e. #sinu=-1/sqrt10# and #cosu=sinu/tanu=(-1/sqrt10)/(1/3)=-3/sqrt10#.
Now #cosu=2cos^2(u/2)-1# gives us #2cos^2(u/2)=1+(-3sqrt10)#
and #cos(u/2)=-sqrt((1-3sqrt10)/2)#
Similarly from #cosu=1-2sin^2(u/2)# we get
#sin(u/2)=sqrt((1-cosu)/2)=sqrt((1+3sqrt10)/2)#
and #tan(u/2)=-sqrt((1+3sqrt10)/(1-3sqrt10))#