\[ \int \sec ^5(a+b x) (d \tan (a+b x))^{3/2} \, dx \]
Optimal antiderivative \[ \frac {4 d^{2} \sqrt {\frac {1}{2}+\frac {\sin \left (2 b x +2 a \right )}{2}}\, \EllipticF \left (\cos \left (a +\frac {\pi }{4}+b x \right ), \sqrt {2}\right ) \sec \left (b x +a \right ) \left (\sqrt {\sin }\left (2 b x +2 a \right )\right )}{77 \sin \left (a +\frac {\pi }{4}+b x \right ) b \sqrt {d \tan \left (b x +a \right )}}-\frac {4 d \sec \left (b x +a \right ) \sqrt {d \tan \left (b x +a \right )}}{77 b}-\frac {2 d \left (\sec ^{3}\left (b x +a \right )\right ) \sqrt {d \tan \left (b x +a \right )}}{77 b}+\frac {2 d \left (\sec ^{5}\left (b x +a \right )\right ) \sqrt {d \tan \left (b x +a \right )}}{11 b} \]
command
integrate(sec(b*x+a)^5*(d*tan(b*x+a))^(3/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \frac {2 \, {\left (2 \, \sqrt {i \, d} d \cos \left (b x + a\right )^{5} {\rm ellipticF}\left (\cos \left (b x + a\right ) + i \, \sin \left (b x + a\right ), -1\right ) + 2 \, \sqrt {-i \, d} d \cos \left (b x + a\right )^{5} {\rm ellipticF}\left (\cos \left (b x + a\right ) - i \, \sin \left (b x + a\right ), -1\right ) - {\left (2 \, d \cos \left (b x + a\right )^{4} + d \cos \left (b x + a\right )^{2} - 7 \, d\right )} \sqrt {\frac {d \sin \left (b x + a\right )}{\cos \left (b x + a\right )}}\right )}}{77 \, b \cos \left (b x + a\right )^{5}} \]
Fricas 1.3.7 via sagemath 9.3 output
\[ {\rm integral}\left (\sqrt {d \tan \left (b x + a\right )} d \sec \left (b x + a\right )^{5} \tan \left (b x + a\right ), x\right ) \]