In your function #f(x)#, let #g(x) = sqrt(x) = x^(1/2)#
Let #h(x) = (x^2 - 2x + 1)^4#
Since the application of the product rule uses the derivatives of both functions, we need to differentiate both. #g(x)# is relatively straight-forward. By the power rule we have that:
#g'(x) = 1/2 xx x^(1/2 - 1)#
#g'(x) = 1/2 xx x^(-1/2)#
#g'(x) = 1/(2x^(1/2))#
Differentiating #h(x)# is a little trickier. We will have to make use of the chain rule.
Let #y = u^4# and #u = x^2 - 2x + 1#
The chain rule states that #dy/dx = dy/(du) xx (du)/dx#, therefore we will have to differentiate both #y# and #u# and then find their product.
#y' = 4u^(4 - 1)#
#y' = 4u^3#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#u' = 2x - 2#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#h'(x) = 4u^3(2x - 2)#
#h'(x) = 4(x^2 - 2x + 1)^3(2x - 2)#
#h'(x) = (8x - 8)(x^2 - 2x + 1)^3#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Finally, we can do our calculation using the product rule.
#f'(x) = (g(x) xx h'(x)) + (g'(x) xx h(x))#
#f'(x) = (x^(1/2)((8x - 8)(x^2 - 2x + 1)^3) + (1/(2x^(1/2))(x^2 - 2x + 1)^4)#
Simplifying:
#f'(x) = (sqrt(x)(8(x - 1))(x^2 - 2x + 1)^3) + ((x^2 - 2x + 1)^4)/(2sqrt(x))#
#f'(x) = sqrt(x)(8x - 8)(x^2 - 2x + 1)^3 +((x^2 - 2x + 1)^4)/(2sqrt(x))#
Hopefully this helps!