\[ \int (a+b \cos (c+d x))^2 (A+B \cos (c+d x)) \sec ^{\frac {5}{2}}(c+d x) \, dx \]
Optimal antiderivative \[ \frac {2 a \left (5 A b +3 B a \right ) \sin \left (d x +c \right ) \left (\sqrt {\sec }\left (d x +c \right )\right )}{3 d}+\frac {2 a A \left (b +a \sec \left (d x +c \right )\right ) \sin \left (d x +c \right ) \left (\sqrt {\sec }\left (d x +c \right )\right )}{3 d}-\frac {2 \left (2 A a b +B \,a^{2}-b^{2} B \right ) \sqrt {\frac {\cos \left (d x +c \right )}{2}+\frac {1}{2}}\, \EllipticE \left (\sin \left (\frac {d x}{2}+\frac {c}{2}\right ), \sqrt {2}\right ) \left (\sqrt {\cos }\left (d x +c \right )\right ) \left (\sqrt {\sec }\left (d x +c \right )\right )}{\cos \left (\frac {d x}{2}+\frac {c}{2}\right ) d}+\frac {2 \left (a^{2} A +3 A \,b^{2}+6 a b B \right ) \sqrt {\frac {\cos \left (d x +c \right )}{2}+\frac {1}{2}}\, \EllipticF \left (\sin \left (\frac {d x}{2}+\frac {c}{2}\right ), \sqrt {2}\right ) \left (\sqrt {\cos }\left (d x +c \right )\right ) \left (\sqrt {\sec }\left (d x +c \right )\right )}{3 \cos \left (\frac {d x}{2}+\frac {c}{2}\right ) d} \]
command
integrate((a+b*cos(d*x+c))^2*(A+B*cos(d*x+c))*sec(d*x+c)^(5/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \frac {\sqrt {2} {\left (-i \, A a^{2} - 6 i \, B a b - 3 i \, A b^{2}\right )} \cos \left (d x + c\right ) {\rm weierstrassPInverse}\left (-4, 0, \cos \left (d x + c\right ) + i \, \sin \left (d x + c\right )\right ) + \sqrt {2} {\left (i \, A a^{2} + 6 i \, B a b + 3 i \, A b^{2}\right )} \cos \left (d x + c\right ) {\rm weierstrassPInverse}\left (-4, 0, \cos \left (d x + c\right ) - i \, \sin \left (d x + c\right )\right ) - 3 \, \sqrt {2} {\left (i \, B a^{2} + 2 i \, A a b - i \, B b^{2}\right )} \cos \left (d x + c\right ) {\rm weierstrassZeta}\left (-4, 0, {\rm weierstrassPInverse}\left (-4, 0, \cos \left (d x + c\right ) + i \, \sin \left (d x + c\right )\right )\right ) - 3 \, \sqrt {2} {\left (-i \, B a^{2} - 2 i \, A a b + i \, B b^{2}\right )} \cos \left (d x + c\right ) {\rm weierstrassZeta}\left (-4, 0, {\rm weierstrassPInverse}\left (-4, 0, \cos \left (d x + c\right ) - i \, \sin \left (d x + c\right )\right )\right ) + \frac {2 \, {\left (A a^{2} + 3 \, {\left (B a^{2} + 2 \, A a b\right )} \cos \left (d x + c\right )\right )} \sin \left (d x + c\right )}{\sqrt {\cos \left (d x + c\right )}}}{3 \, d \cos \left (d x + c\right )} \]
Fricas 1.3.7 via sagemath 9.3 output
\[ {\rm integral}\left ({\left (B b^{2} \cos \left (d x + c\right )^{3} + A a^{2} + {\left (2 \, B a b + A b^{2}\right )} \cos \left (d x + c\right )^{2} + {\left (B a^{2} + 2 \, A a b\right )} \cos \left (d x + c\right )\right )} \sec \left (d x + c\right )^{\frac {5}{2}}, x\right ) \]