\[ \int \cot ^2(c+d x) (a+i a \tan (c+d x))^2 (A+B \tan (c+d x)) \, dx \]
Optimal antiderivative \[ -2 a^{2} \left (-i B +A \right ) x +\frac {a^{2} B \ln \left (\cos \left (d x +c \right )\right )}{d}+\frac {a^{2} \left (2 i A +B \right ) \ln \left (\sin \left (d x +c \right )\right )}{d}-\frac {A \cot \left (d x +c \right ) \left (a^{2}+i a^{2} \tan \left (d x +c \right )\right )}{d} \]
command
integrate(cot(d*x+c)**2*(a+I*a*tan(d*x+c))**2*(A+B*tan(d*x+c)),x)
Sympy 1.10.1 under Python 3.10.4 output
\[ - \frac {2 i A a^{2}}{d e^{2 i c} e^{2 i d x} - d} + \frac {B a^{2} \log {\left (e^{2 i d x} + e^{- 2 i c} \right )}}{d} + \frac {i a^{2} \cdot \left (2 A - i B\right ) \log {\left (e^{2 i d x} + \frac {\left (A a^{2} - i B a^{2} - a^{2} \cdot \left (2 A - i B\right )\right ) e^{- 2 i c}}{A a^{2}} \right )}}{d} \]
Sympy 1.8 under Python 3.8.8 output
\[ \text {Exception raised: NotInvertible} \]________________________________________________________________________________________