Solving trig antiderivatives usually involves breaking the integral down to apply Pythagorean Identities, and them using a #u#-substitution. That's exactly what we'll do here.
Begin by rewriting #inttan^4xdx# as #inttan^2xtan^2xdx#. Now we can apply the Pythagorean Identity #tan^2x+1=sec^2x#, or #tan^2x=sec^2x-1#:
#inttan^2xtan^2xdx=int(sec^2x-1)tan^2xdx#
Distributing the #tan^2x#:
#color(white)(XX)=intsec^2xtan^2x-tan^2xdx#
Applying the sum rule:
#color(white)(XX)=intsec^2xtan^2xdx-inttan^2xdx#
We'll evaluate these integrals one by one.
First Integral
This one is solved using a #u#-substitution:
Let #u=tanx#
#(du)/dx=sec^2x#
#du=sec^2xdx#
Applying the substitution,
#color(white)(XX)intsec^2xtan^2xdx=intu^2du#
#color(white)(XX)=u^3/3+C#
Because #u=tanx#,
#intsec^2xtan^2xdx=(tan^3x)/3+C#
Second Integral
Since we don't really know what #inttan^2xdx# is by just looking at it, try applying the #tan^2=sec^2x-1# identity again:
#inttan^2xdx=int(sec^2x-1)dx#
Using the sum rule, the integral boils down to:
#intsec^2xdx-int1dx#
The first of these, #intsec^2xdx#, is just #tanx+C#. The second one, the so-called "perfect integral", is simply #x+C#. Putting it all together, we can say:
#inttan^2xdx=tanx+C-x+C#
And because #C+C# is just another arbitrary constant, we can combine it into a general constant #C#:
#inttan^2xdx=tanx-x+C#
Combining the two results, we have:
#inttan^4xdx=intsec^2xtan^2xdx-inttan^2xdx=((tan^3x)/3+C)-(tanx-x+C)=(tan^3x)/3-tanx+x+C#
Again, because #C+C# is a constant, we can join them into one #C#.
