\[ \int \frac {\cos ^{\frac {3}{2}}(c+d x) \left (A+B \sec (c+d x)+C \sec ^2(c+d x)\right )}{(a+b \sec (c+d x))^{5/2}} \, dx \]
Optimal antiderivative \[ \frac {2 \left (A \,b^{2}-a \left (b B -a C \right )\right ) \sin \left (d x +c \right ) \left (\sqrt {\cos }\left (d x +c \right )\right )}{3 a \left (a^{2}-b^{2}\right ) d \left (a +b \sec \left (d x +c \right )\right )^{\frac {3}{2}}}+\frac {2 \left (10 a^{2} A \,b^{2}-6 A \,b^{4}-7 a^{3} b B +3 a \,b^{3} B +4 a^{4} C \right ) \sin \left (d x +c \right ) \left (\sqrt {\cos }\left (d x +c \right )\right )}{3 a^{2} \left (a^{2}-b^{2}\right )^{2} d \sqrt {a +b \sec \left (d x +c \right )}}-\frac {2 \left (16 A \,b^{4}+9 a^{3} b B -8 a \,b^{3} B -2 a^{2} b^{2} \left (8 A -C \right )-a^{4} \left (A +3 C \right )\right ) \sqrt {\frac {\cos \left (d x +c \right )}{2}+\frac {1}{2}}\, \EllipticF \left (\sin \left (\frac {d x}{2}+\frac {c}{2}\right ), \sqrt {2}\, \sqrt {\frac {a}{a +b}}\right ) \sqrt {\frac {b +a \cos \left (d x +c \right )}{a +b}}}{3 \cos \left (\frac {d x}{2}+\frac {c}{2}\right ) a^{4} \left (a^{2}-b^{2}\right ) d \sqrt {\cos \left (d x +c \right )}\, \sqrt {a +b \sec \left (d x +c \right )}}+\frac {2 \left (8 A \,b^{4}+8 a^{3} b B -4 a \,b^{3} B +a^{4} \left (A -5 C \right )-a^{2} b^{2} \left (13 A -C \right )\right ) \sin \left (d x +c \right ) \left (\sqrt {\cos }\left (d x +c \right )\right ) \sqrt {a +b \sec \left (d x +c \right )}}{3 a^{3} \left (a^{2}-b^{2}\right )^{2} d}-\frac {2 \left (16 A \,b^{5}-3 B \,a^{5}+15 B \,a^{3} b^{2}-8 B a \,b^{4}-2 a^{2} b^{3} \left (14 A -C \right )+a^{4} \left (8 A b -6 C b \right )\right ) \sqrt {\frac {\cos \left (d x +c \right )}{2}+\frac {1}{2}}\, \EllipticE \left (\sin \left (\frac {d x}{2}+\frac {c}{2}\right ), \sqrt {2}\, \sqrt {\frac {a}{a +b}}\right ) \left (\sqrt {\cos }\left (d x +c \right )\right ) \sqrt {a +b \sec \left (d x +c \right )}}{3 \cos \left (\frac {d x}{2}+\frac {c}{2}\right ) a^{4} \left (a^{2}-b^{2}\right )^{2} d \sqrt {\frac {b +a \cos \left (d x +c \right )}{a +b}}} \]
command
integrate(cos(d*x+c)^(3/2)*(A+B*sec(d*x+c)+C*sec(d*x+c)^2)/(a+b*sec(d*x+c))^(5/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \text {output too large to display} \]
Fricas 1.3.7 via sagemath 9.3 output \[ {\rm integral}\left (\frac {{\left (C \cos \left (d x + c\right ) \sec \left (d x + c\right )^{2} + B \cos \left (d x + c\right ) \sec \left (d x + c\right ) + A \cos \left (d x + c\right )\right )} \sqrt {b \sec \left (d x + c\right ) + a} \sqrt {\cos \left (d x + c\right )}}{b^{3} \sec \left (d x + c\right )^{3} + 3 \, a b^{2} \sec \left (d x + c\right )^{2} + 3 \, a^{2} b \sec \left (d x + c\right ) + a^{3}}, x\right ) \]