Start with the identity #cos^2(theta) + sin^2(theta) = 1#
Divide both sides by #cos^2(theta)#:
#cos^2(theta)/cos^2(theta) + sin^2(theta)/cos^2(theta) = 1/cos^2(theta)#
Use the identity #sin(theta)/cos(theta) = tan(theta)# and #cos^2(theta)/cos^2(theta) = 1#:
#1 + tan^2(theta) = 1/cos^2(theta)#
Solve for #cos(theta)#:
#cos^2(theta) = 1/(1 + tan^2(theta))#
#cos(theta) = +-sqrt(1/(1 + tan^2(theta)))#
The domain restriction #pi < theta < (3pi)/2# tells us that #theta# is in the third quadrant (where the cosine function is negative), therefore, we change the #+-# to - only:
#cos(theta) = -sqrt(1/(1 + tan^2(theta)))#
Substitute #(2/3)^2# for #tan^2(theta)#
#cos(theta) = -sqrt(1/(1 + (2/3)^2))#
#cos(theta) = -sqrt(1/(1 + 4/9))#
#cos(theta) = -sqrt(1/(13/9))#
#cos(theta) = -sqrt(9/13)#
#cos(theta) = -(3sqrt(13))/13#
