We start with taking the constant factor 5 out of the integral, which leaves us with
5 int x cos(2x)\ dx.
If we look at the integrand now, then we have a factor x, whose derivative is 1, and a factor cos(2x), which is super-easy to integrate with an anti-derivative of sin(2x) / 2. Therefore, this calls for integration by parts, which is described by the general rule
int u' v = u v - int u v'.
In our particular case, we choose u = cos(2x) and v = x, and thus get
5 int x cos(2x)\ dx = \frac{5}{2} [ x sin(2x) - int sin(2x)\ dx ].
The last remaining integral is again an easy one,
int sin(2x)\ dx = - cos(2x)/2,
and so we arrive at the final result
\frac{5}{2} [ x sin(2x) - int sin(2x)\ dx ] =
= \frac{5}{2} [ x sin(2x) + cos(2x)/2 ] + C =
= \frac{5}{4} [ 2 x sin(2x) + cos(2x) ] + C.