Here,
#cos^-1 x + cos^-1 y=pi/2...to(1)#
#tan^-1 x +tan^-1 y =0...to(2)#
So,
#tan^-1x=-tan^-1y#
#=>tan^-1x=tan^-1(-y)#
#=>x=-y...to(3)#
#=>x+y=0...to(4)#
Subst. #x=-y# , in #(1)#
#cos^-1 (-y) + cos^-1 y=pi/2#
#pi-cos^-1y+cos^-1y=pi/2#
Then, #cos^-1 x + cos^-1 y=pi/2# does not exists.
Please check your question.
................................................................................................
My suggestion:
If #color(red)(cos^-1 x + cos^-1 y=pi/2 and tan^-1 x -tan^-1 y =0#
,
then find the value of #color(red)(x^2+y^2+xy# .
#cos^-1 x + cos^-1 y=pi/2...to(1)#
#tan^-1 x -tan^-1 y =0...to(2)#
So,
#tan^-1x=tan^-1y#
#=>x=y...to(3)#
Subst, #y=x# in #(1)#
#cos^-1x+cos^-1x=pi/2=>2cos^-1x=pi/2#
#=>cos^-1x=pi/4=>x=1/sqrt2=y#
So, #x^2+y^2+xy=(1/sqrt2)^2+(1/sqrt2)^2+(1/sqrt2)(1/sqrt2)#
#=1/2+1/2+1/2=3/2#
If your question is different than given answer,then type your
question again.
If my suggestion is true,then please like it.