\[ \int (a+a \cos (c+d x))^{5/2} (A+B \cos (c+d x)) \sec ^3(c+d x) \, dx \]
Optimal antiderivative \[ \frac {a^{\frac {5}{2}} \left (19 A +20 B \right ) \arctanh \left (\frac {\sin \left (d x +c \right ) \sqrt {a}}{\sqrt {a +a \cos \left (d x +c \right )}}\right )}{4 d}-\frac {a^{3} \left (9 A -4 B \right ) \sin \left (d x +c \right )}{4 d \sqrt {a +a \cos \left (d x +c \right )}}+\frac {a A \left (a +a \cos \left (d x +c \right )\right )^{\frac {3}{2}} \sec \left (d x +c \right ) \tan \left (d x +c \right )}{2 d}+\frac {a^{2} \left (7 A +4 B \right ) \sqrt {a +a \cos \left (d x +c \right )}\, \tan \left (d x +c \right )}{4 d} \]
command
integrate((a+a*cos(d*x+c))^(5/2)*(A+B*cos(d*x+c))*sec(d*x+c)^3,x, algorithm="giac")
Giac 1.9.0-11 via sagemath 9.6 output
\[ \frac {\sqrt {2} {\left (32 \, B a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right ) - \sqrt {2} {\left (19 \, A a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) + 20 \, B a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right )\right )} \log \left (\frac {{\left | -2 \, \sqrt {2} + 4 \, \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right ) \right |}}{{\left | 2 \, \sqrt {2} + 4 \, \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right ) \right |}}\right ) - \frac {4 \, {\left (22 \, A a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )^{3} + 8 \, B a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )^{3} - 13 \, A a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right ) - 4 \, B a^{2} \mathrm {sgn}\left (\cos \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right ) \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )\right )}}{{\left (2 \, \sin \left (\frac {1}{2} \, d x + \frac {1}{2} \, c\right )^{2} - 1\right )}^{2}}\right )} \sqrt {a}}{16 \, d} \]
Giac 1.7.0 via sagemath 9.3 output
\[ \text {Timed out} \]________________________________________________________________________________________