Given: #3tan^-1(x)+ cot^-1(x)=pi#
Use the cosine function on both sides:
#cos(3tan^-1(x)+ cot^-1(x))=cos(pi)#
Use the identity #cos(A+B) = cos(A)cos(B)-sin(A)sin(B)# on the left and #cos(pi) = -1# on the right:
#cos(3tan^-1(x))cos(cot^-1(x))-sin(3tan^-1(x))sin(cot^-1(x))=-1#
From the reference Inverse Trigonometric Functions, we obtain the identities #cos(cot^-1(x)) = x/sqrt(1+x^2)# and #sin(cot^-1(x)) = 1/sqrt(1+x^2)#:
#cos(3tan^-1(x))x/sqrt(1+x^2)-sin(3tan^-1(x))1/sqrt(1+x^2)=-1#
Multiply both sides by #sqrt(1+x^2)#
#cos(3tan^-1(x))x-sin(3tan^-1(x))=-sqrt(1+x^2)#
I used WolframAlpha to find an alternate form for the left side:
#-(2x)/sqrt(1+x^2) = -sqrt(1+x^2)#
Multiply both sides by #-sqrt(1+x^2)#:
#2x = x^2+1#
Write in standard form:
#x^2-2x+1=0#
#(x -1)^2=0#
#x = 1#