\[ \int \cot ^5(e+f x) \left (a+b \tan ^2(e+f x)\right )^2 \, dx \]
Optimal antiderivative \[ \frac {a \left (a -2 b \right ) \left (\cot ^{2}\left (f x +e \right )\right )}{2 f}-\frac {a^{2} \left (\cot ^{4}\left (f x +e \right )\right )}{4 f}+\frac {\left (a -b \right )^{2} \ln \left (\cos \left (f x +e \right )\right )}{f}+\frac {\left (a -b \right )^{2} \ln \left (\tan \left (f x +e \right )\right )}{f} \]
command
integrate(cot(f*x+e)^5*(a+b*tan(f*x+e)^2)^2,x, algorithm="giac")
Giac 1.9.0-11 via sagemath 9.6 output
\[ -\frac {\frac {12 \, a^{2} {\left (\cos \left (f x + e\right ) - 1\right )}}{\cos \left (f x + e\right ) + 1} - \frac {16 \, a b {\left (\cos \left (f x + e\right ) - 1\right )}}{\cos \left (f x + e\right ) + 1} + \frac {a^{2} {\left (\cos \left (f x + e\right ) - 1\right )}^{2}}{{\left (\cos \left (f x + e\right ) + 1\right )}^{2}} - 32 \, {\left (a^{2} - 2 \, a b + b^{2}\right )} \log \left (\frac {{\left | -\cos \left (f x + e\right ) + 1 \right |}}{{\left | \cos \left (f x + e\right ) + 1 \right |}}\right ) + 64 \, {\left (a^{2} - 2 \, a b + b^{2}\right )} \log \left ({\left | -\frac {\cos \left (f x + e\right ) - 1}{\cos \left (f x + e\right ) + 1} + 1 \right |}\right ) + \frac {{\left (a^{2} + \frac {12 \, a^{2} {\left (\cos \left (f x + e\right ) - 1\right )}}{\cos \left (f x + e\right ) + 1} - \frac {16 \, a b {\left (\cos \left (f x + e\right ) - 1\right )}}{\cos \left (f x + e\right ) + 1} + \frac {48 \, a^{2} {\left (\cos \left (f x + e\right ) - 1\right )}^{2}}{{\left (\cos \left (f x + e\right ) + 1\right )}^{2}} - \frac {96 \, a b {\left (\cos \left (f x + e\right ) - 1\right )}^{2}}{{\left (\cos \left (f x + e\right ) + 1\right )}^{2}} + \frac {48 \, b^{2} {\left (\cos \left (f x + e\right ) - 1\right )}^{2}}{{\left (\cos \left (f x + e\right ) + 1\right )}^{2}}\right )} {\left (\cos \left (f x + e\right ) + 1\right )}^{2}}{{\left (\cos \left (f x + e\right ) - 1\right )}^{2}}}{64 \, f} \]
Giac 1.7.0 via sagemath 9.3 output
\[ \text {Exception raised: NotImplementedError} \]________________________________________________________________________________________