Find the first derivative. If #f'(pi/4)<0#, then the function is decreasing at that point. If #f'(pi/4)>0#, then the function is increasing at that point.
To find #f'(x)#, we will need to use the chain rule:
#u=x+(5pi)/6#
#f(x)=cos(u)#
#d/dx[cosu]=-sin(u)*(du)/dx#
#f'(x)=-sin(x+(5pi)/6)*d/dx[x+(5pi)/6]#
Note that #d/dx[x+(5pi)/6]=1#, so:
#f'(x)=-sin(x+(5pi)/6)#
#f'(pi/4)=-sin(pi/4+(5pi)/6)#
#f'(pi/4)=-sin((13pi)/12)#
You could calculate #sin((13pi)/12)#, but I think a more elegant approach would be to recognize that #(13pi)/12# is just barely larger than #pi#, so a reference angle of #(13pi)/12# would land in the third quadrant.
Thus, the sine of that angle would be NEGATIVE, and since there is a negative sign outside of the sine, #f'(pi/4)# is POSITIVE, meaning the function is increasing when #x=pi/4#.
