Let #R# be the radius of the incircle.
Radius of incircle of a right triangle #R = (a*b)/((a+b+c)#, where a and b are the legs of the triangle and c is the hypotenuse.
To get the largest possible radius of the incircle, 12 should be the smallest side of the triangle.
As this is a multiple-choice (objective) question, let's start with trying Option #D (R=5)#
Let #a# be the smallest side =#12#
#=> R=(12b)/(12+b+c)=5#
#=> c=(7b-60)/5# ...............(1)
As the triangle is right-angled, #=> c^2=a^2+b^2#
#=> c^2=144+b^2# ....... (2)
Substituting (1) into (2), we get #b=35, c=37#
So the triangle is in the ratio of : #12:35:37#
Check : #12^2+35^2=1369=37^2#, (OK)
#R=(axxb)/(a+b+c)=(12xx35)/(12+35+37)=(420/84)=5#,
Hence, option #D# is the answer.
...........................................................................................................
Footnote : the solution below is a much better one. A big "thank you" to #dk#_#ch# for kindly providing such an excellent solution.
We have #c^2−b^2=144#, where #c# is the hypotenuse and #b# is another side. #c>b#. By the given condition both #c and b# are integers.
Now #(c+b)(c−b)=144=72⋅2#...[1]
Here both #(c+b)and(c−b)# are even integers as their product #144# is even one.
To get maximum integer value satisfying the given condition both #c and b# will have large integer value and the value of #c−b# will have minimum possible even integer value.
So the minimum integer value of #(c−b)# should be #2#, i.e., #(c−b)=2#.
So from relation [1]
we have #(c+b)=72 and (c−b)=2#
Thus we get #c=37 and b=35#.
