For this, we will need the Chain rule. The chain rule states that when confronted with a function #f(x) = g(h(x))#, our derivative #f'(x) = (h'(x)) g'(h(x))#. Using this rule, along with the Power Rule and the formula for the derivative of a cosine function, we can find that if #y(x) = cos(x^5), y'(x) = (5x^4)(-sin(x^5))#,
In the interests of brevity, we will skip a proof of the Chain Rule. However, for those interested, a simple web search for "chain rule proof" will turn up several good results on the first few pages. It is recommended one sticks to the PDF ones, those seem to be more tidily designed.
It is worth noting, however, that even when dealing with a function where #h(x)=x#, as in the equation #f(x) = sin(x)# where #g = sin# and #h=x#, we use the chain rule. However, as #(d/dx)x=1#, our #h'(x) = 1#, and thus #f'(x) = h'(x) * g'(h(x)) = 1*g'(h(x)) = g'(h(x)) = g'(x) = (d/dx)sin(x) = cos(x)#.
For a more in depth look at your example, let us find the quantities represented by #g(h), h(x), g'(h), h'(x)#.
#f(x) = y(x) = cos(x^5)#
From the formula #f(x) = g(h(x))#, we have
#g(h) = cos(h(x)), h(x) = x^5#
The power rule quickly tells us that #h'(x) = 5x^4#. For #g'(h)#, we must look at the "cycle" of the derivatives of sine and cosine functions. Namely, the following, for #g(x) = sin(x)#:
#g(x) = sin (x), g'(x) = cos(x), g''(x) = -sin(x), g'''(x) = -cos(x), g''''(x) = sin(x)# Or in words, the first derivative of #g(x)=sin(x)# is #cos(x)#, the second derivative is #-sin(x)#, the third derivative is #-cos(x)#, and the fourth derivative "circles back" to the beginning of the cycle and is #sin(x)# again.
There exists a proof for this (Euler's Formula), but the proof will not be given here. Suffice to say that this shows us that for #g(h) = cos(h), g'(h) = -sin(h)#. Given this along with our initial formula, #y'(x) = h'(x)g'(h(x))#, we arrive at:
#y'(x) = (5x^4)(-sin(x^5)) = -5x^4sin(x^5)#