\[ \int \cot (c+d x) \sqrt {a+b \tan (c+d x)} (A+B \tan (c+d x)) \, dx \]
Optimal antiderivative \[ -\frac {2 A \arctanh \left (\frac {\sqrt {a +b \tan \left (d x +c \right )}}{\sqrt {a}}\right ) \sqrt {a}}{d}+\frac {\left (-i B +A \right ) \arctanh \left (\frac {\sqrt {a +b \tan \left (d x +c \right )}}{\sqrt {-i b +a}}\right ) \sqrt {-i b +a}}{d}+\frac {\left (i B +A \right ) \arctanh \left (\frac {\sqrt {a +b \tan \left (d x +c \right )}}{\sqrt {i b +a}}\right ) \sqrt {i b +a}}{d} \]
command
integrate(cot(d*x+c)*(a+b*tan(d*x+c))^(1/2)*(A+B*tan(d*x+c)),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \text {output too large to display} \]
Fricas 1.3.7 via sagemath 9.3 output \[ \text {Timed out} \]