We determine the coordinates of the center of inscribed circle by investigation of the intersection of the angular bisectors.

Let,#A-=(3,2), B-=(6,7), and C-=(5,8) #represent the vertices of the triangle ABC

Angular bisector atA:

#A-=(3,2)
B-=(6,7)
C-=(5,8)#

Slope of line AB is #m_(AB)=(7-2)/(6-3)=5/3#

Slope of line AB is #m_(AC)=(8-2)/(5-3)=6/2=3#

Let the slope of the bisector be m

#(m-5/3)/(1+mxx5/3)=(3-m)/(1+3xxm)#

#=(3m-5)/(3+5m)=(3-m)/(1+3m)#

Cross multiplying

#(1+3m)(3m-5)=(3-m)(3+5m)#

#3m-5+9m^2-15m=9-3m+15m-5m^2#

#9m^2+5m^2+3m-15m+3m-15m-5-9=0#

#14m^2-24m-14=0#

Dividing by 14

#m^2-12/7m-1=0#

#m=2.17, m=-0.46#

Equation of the bisector through A(3,2) is

#(y-2)/(x-3)=2.17#

Simplifying

#y-2=2.17(x-3)#

#y-2=2.17x-6.51#

#y=2.17x-4.51#

Angular bisector atB:

A-=(3,2)

B-=(6,7)

C-=(5,8)

Slope of line AB is #m_(AB)=(7-2)/(6-3)=5/3#

Slope of line BC is #m_(BC)=(8-7)/(5-6)=1/-1=-1#

Let the slope of the bisector be m

#(m-5/3)/(1+mxx5/3)=(-1-m)/(1+(-1)xxm)#

#=(3m-5)/(3+5m)=(-1-m)/(1-m)#

Cross multiplying

#(1-m)(3m-5)=(-1-m)(3+5m)#

#3m-5-3m^2+5m=-3-5m-3m-5m^2#

#5m^2-3m^2+5m+3m+5m+3m+3-5=0#

#2m^2+16m-2=0#

Dividing by 2

#m^2+8m-1=0#

#m=0.12, m=-8.12#

Equation of the bisector through B(6,7) is

#(y-7)/(x-6)=0.12#

Simplifying

#y-7=0.12(x-6)#

#y-7=0.12x-0.75#

#y=0.12x+6.25#

The lines #y=2.17x-4.51# and #y=0.12x+6.25# intersect at the center of the incircle

Equating rhs

#2.17x-4.51=0.12x+6.25#

#2.17x-0.12x=6.25+4.51#

#2.05x=10.76#

#x=10.76/2.05#

#x=5.25#

#y=2.17x-4.51#

#y=2.17xx5.25-4.51#

#y=11.39-4.51#

#y=6.88#

#y=0.12x+6.25#

#y=0.12xx5.25+6.25=0.63+6.25#

#y=6.88#

Verified

The coordinates of the centre of incircle is

#O-=(5.25,6.88)#

Slope of line AB

#m_(AB)=5/3#

Point A is #A-=(3,2)#

Equation of the line AB is

#(y-2)/(x-3)=5/3#

#3(y-2)=5(x-3)#

#3y-6=5x-15#

#5x-3y+6-15=0#

#5x-3y-9=0#

Center is #O-=(5.25,6.88)#

Tangent is #5x-3y-9=0#

The distance from O to the line AB is

given by

#|(5xx5.25-3xx6.88-9)/sqrt(5^2+(-3)^2)|=|(-3.39)/5.83|=0.581#

The radius of the circle is 0.581