#(3x+y)/8 = (x-y)/5 = (x^2-y^2)/5#
Start with #(x-y)/5 = (x^2-y^2)/5#. Multiply by #5# and factor the right side:
#(x-y) = (x - y)(x+y)#.
Collect on one side:
#(x - y)(x+y) - (x-y) = 0 #.
Factor #(x-y)#
#(x - y)(x+y - 1) = 0 #.
So #x-y=0# or #x+y-1 = 0#
This gives us: #y=x# or #y = 1-x#
Now use the first two expressions together with these solutions for #y#.
#(3x+y)/8 = (x-y)/5#
Leads to: #15x+5y=8x-8y#.
So #7x+13y =0#
Solution 1
Now, when #y=x#, we get #20x = 0#, so #x=0# and thus #y=0#
Solution 2
When #y=1-x#, we get
#7x+13(1-x)=0#
#7x + 13 -13x =0#
#-6x = -13#
#x=13/6# and
#y = 1-x = 1- 13/6 = -7/6#
Checking these solutions
#(3x+y)/8 = (x-y)/5 = (x^2-y^2)/5#
For #(0,0)#, we get
#0/8 = 0/5 =0/5#
For #(13/6, -7/6)#, we get:
#(3(13/6)+(-7/6))/8 = (39-7)/48 = 32/48 = 2/3#
#((13/6)-(-7/6))/5 = 20/30 = 2/3#
#((13/6)^2-(-7/6)^2)/5 = (169 - 49)/(36*5) = 120/(36*5) = 20/(6*5) = 2/3#