Physically speaking, energy can be defined in terms of the maximum amount of work that can be performed. To explain this more carefully, let us first think about the notion of work. I will only talk about classical physics here.
In classical physics, the motion of objects is governed by Newtons second law #vecF=mveca#, where #vecF# is a force, #m# an objects mass and #veca# an obects acceleration. This means that a force is something that changes the way an object moves.
Of course we can vary the force we act on a particle through time, or rather, through the path it takes. We therefore define a quantity we call work, (#W#), by the following expression #W=intvecF*dvecs#. Here #dvecs=vecvdt# a vector pointing along the path that a particle takes proportional to the velocity of the particle. When the path is straight and the force in the same direction as the path, this reduces to #W=FDeltas#.
Even though we have defined this work in terms of the path along which a force acts, we can work out that the work needed to change the state of a particle from one to another (for instance change the speed of a particle) is only dependant on the initial and final situation. To see this we just work out the integral using Newtons second law.
Now we use #d/dt(v^2)=d/dt((dvecs)/dt*(dvecs)/dt)=2(d^2vecs)/dt^2*(dvecs)/dt# via the product rule, so #W=m/2intd/dt(v^2)dt=m/2[v^2]_"initially"^"finally"=m/2(v_f^2-v_i^2)#.
So indeed we only need to know the initial and final velocities and the mass to know the work.
Now we define something called the kinetic energy of an object #E_"kin"=m/2v^2#, so #W=DeltaE_"kin"#. Note that #W# can be both negative or positive. If #W# is positive, we say that work has been performed on the object, if it is negative, we say the object has performed work. Since #v^2>0#, the maximum amount of work a moving object can perform is given by its kinetic energy.
Until now we have only talked about moving particles, but there are many other things where we see this quantity of work, think about compression of gas, electric and magnetic fields. However in general it is possible to assign a value to an object that changes when work is being performed. So when we can somehow write down an expression for a value #E# for an object that changes when the object performs work via #W=DeltaE#, and when #E=0# the object can't perform work, we call this value an energy.