\[ \int \csc (a+b x) (d \tan (a+b x))^{5/2} \, dx \]
Optimal antiderivative \[ \frac {d^{2} \csc \left (b x +a \right ) \sqrt {\frac {1}{2}+\frac {\sin \left (2 b x +2 a \right )}{2}}\, \EllipticF \left (\cos \left (a +\frac {\pi }{4}+b x \right ), \sqrt {2}\right ) \left (\sqrt {\sin }\left (2 b x +2 a \right )\right ) \sqrt {d \tan \left (b x +a \right )}}{3 \sin \left (a +\frac {\pi }{4}+b x \right ) b}+\frac {2 d \csc \left (b x +a \right ) \left (d \tan \left (b x +a \right )\right )^{\frac {3}{2}}}{3 b} \]
command
integrate(csc(b*x+a)*(d*tan(b*x+a))^(5/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \frac {\sqrt {i \, d} d^{2} \cos \left (b x + a\right ) {\rm ellipticF}\left (\cos \left (b x + a\right ) + i \, \sin \left (b x + a\right ), -1\right ) + \sqrt {-i \, d} d^{2} \cos \left (b x + a\right ) {\rm ellipticF}\left (\cos \left (b x + a\right ) - i \, \sin \left (b x + a\right ), -1\right ) + 2 \, d^{2} \sqrt {\frac {d \sin \left (b x + a\right )}{\cos \left (b x + a\right )}}}{3 \, b \cos \left (b x + a\right )} \]
Fricas 1.3.7 via sagemath 9.3 output
\[ {\rm integral}\left (\sqrt {d \tan \left (b x + a\right )} d^{2} \csc \left (b x + a\right ) \tan \left (b x + a\right )^{2}, x\right ) \]