We get the slope (#m_t#) of the tangent line from the value of the first derivative #(df)/dx# at the given #x# value. The slope of the line normal to that we get from noting that the slopes of two perpendicular lines multiply to -1 - assuming that neither line is vertical.

Firstly note that we may immediately simplify the given function - #cosx# has period #2pi#, so #cos2x# has period #pi#, so #cos(2x-pi)=cos2x# and #f(x)=secx+cos^2 2x#.

**Find the function derivative**

Use the quotient rule for differentiation for the first term of #f# and the chain rule twice for the second term:

#(df)/dx=secxtanx+2cos2x*(-sin2x)*2#

#(df)/dx=secxtanx-4sin2xcos2x#

Recall the identity #sin2x=2sinxcosx#

#(df)/dx=secxtanx-2sin4x#

**Evalute the derivative**

Now evaluate #(df)/dx# at the given #x#:

#(df)/dx((11pi)/8)=sec((11pi)/8)tan((11pi)/8)-2sin((11pi)/2)#

Recall that #cos(x+pi)=-cosx# (so #sec(x+pi)=-secx#). Then #sec((11pi)/8)=-sec((3pi)/8)#

Similarly, recall that #tan(x+pi)=tanx#, so #tan((11pi)/8)=tan((3pi)/8)#. If these aren't easy to recall, think of the graphs of the functions.

As #sin(x+pi)=-sinx#, #sin((11pi)/2)=-sin(pi/2)=-1#

So

#(df)/dx((11pi)/8)=-sec((3pi)/8)tan((3pi)/8)+2#

Now the trig values of the angle #(3pi)/8# are not standard ones, but they can be calculated.

#sin((3pi)/8)=sqrt(2+sqrt(2))/2#, as derived here:

https://socratic.org/questions/how-do-you-use-the-half-angle-identity-to-find-exact-value-of-sin-3pi-8

#cos((3pi)/8)=sqrt(2-sqrt(2))/2#, as derived here:

https://socratic.org/questions/how-do-you-find-the-exact-values-of-cos-3pi-8-using-the-half-angle-formula

Thus

#sec((3pi)/8)=1/cos((3pi)/8)=2/sqrt(2-sqrt(2))#

#tan((3pi)/8)=sin((3pi)/8)/cos((3pi)/8)=sqrt((2+sqrt(2))/(2-sqrt(2)))#

So

#(df)/dx((11pi)/8)=-sec((3pi)/8)tan((3pi)/8)+2=-2/sqrt(2-sqrt(2))sqrt((2+sqrt(2))/(2-sqrt(2)))+2#

#(df)/dx((11pi)/8)=2-(2sqrt(2+sqrt(2)))/(2-sqrt(2))=2/(2-sqrt(2))(2-sqrt(2)-sqrt(2+sqrt(2)))#

This (complicated) value is the slope of the tangent line at the given point.

**Calculate the normal slope**

We now calculate the normal slope #m_n#via #m_nm_t=-1#.

#m_t=2-(2sqrt(2+sqrt(2)))/(2-sqrt(2))=2/(2-sqrt(2))(2-sqrt(2)-sqrt(2+sqrt(2)))#

so

#m_n=-1/m_t=-(2-sqrt(2))/(2[2-sqrt(2)-sqrt(2+sqrt(2))])#

Numerically, we calculate to three significant figures that #m_t=-4.31# and #m_n=0.232#.

Sanity check these answers by plotting the graphs of the function and these lines for the given #x# value

graph{(y-secx-(cos(2x))^2)(y+4.308644x-16.49888)(y-0.232x+3.115689)=0 [2, 6, -5, 0]}