The box should be a cube and each dimension should be #\frac{2\sqrt{21}}{3}\approx 3.05# units. The maximum volume is #\frac{56\sqrt{21}}{9}\approx 28.5# cubic units.
Let #x#, #y#, and #z# be the dimensions of the box. Then #V=xyz# and #S=2xy+2xz+2yz# are the volume and surface areas of the box.
It might make some intuitive sense to you that the answer is a cube so that #x=y=z#. If it does, then the surface area being 56 square units implies that #S=6x^2=56# so that #x^2=y^2=z^2=\frac{56}{6}=\frac{28}{3}# and #x=y=z=\sqrt{\frac{28}{3}}=\frac{2\sqrt{7}}{\sqrt{3}}=\frac{2\sqrt{21}}{3}# and #V=xyz=(\frac{2\sqrt{21}}{3})^3=\frac{8\cdot 21\sqrt{21}}{27}=\frac{56\sqrt{21}}{9}#
More rigorously, we can use calculus:
Since #S=56#, we can say that #2z(x+y)=56-2xy# so that #z=\frac{28-xy}{x+y}#. This means we can write the volume as a function of two variables:
#V=\frac{xy(28-xy)}{x+y}=\frac{28xy-x^2y^2}{x+y}. #
The partial derivatives of this function are:
#\frac{\partial V}{\partial x}=\frac{(x+y)(28y-2xy^2)-(28xy-x^2y^2)\cdot 1}{(x+y)^2}=\frac{y^2(28-x^2-2xy)}{(x+y)^2}#
and
#\frac{\partial V}{\partial y}=\frac{(x+y)(28x-2x^2y)-(28xy-x^2y^2)\cdot 1}{(x+y)^2}=\frac{x^2(28-y^2-2xy)}{(x+y)^2}.#
Setting these both equal to zero results in the system of equations #x^2+2xy=28, y^2+2xy=28#. The first of these equations can be solved for #y# to get #y=\frac{28-x^2}{2x}#.
Plugging this into the second equation leads to #(\frac{28-x^2}{2x})^{2}+2x\cdot \frac{28-x^{2}}{2x}=28#. Multiplying both sides of this last equation by #4x^2# gives #(28-x^2)^2+112x^2-4x^4=112x^2#, which reduces to #3x^4+56x^2-784=0#.
Using the quadratic formula, #x^2=\frac{-56\pm\sqrt{56^2-4\cdot 3\cdot (-784)}}{2}=\frac{-56\pm\sqrt{12544}}{6}=\frac{-56\pm 112}{6}#. The positive one is #x^2=\frac{56}{6}=\frac{28}{3}# so that #x=\sqrt{\frac{28}{3}}=\frac{2\sqrt{7}}{\sqrt{3}}=\frac{2\sqrt{21}}{3}\approx 3.05#. The symmetry in the original equations implies that #y=\frac{2\sqrt{21}}{3}\approx 3.05# as well. If you use the equation #z=\frac{28-xy}{x+y}# you'll get #z=\frac{2\sqrt{21}}{3}\approx 3.05# as well. That this gives a maximum value can be checked with the 2nd derivative test.
