\[ \int x \left (a x+b x^3\right )^{3/2} \, dx \]
Optimal antiderivative \[ \frac {2 x^{2} \left (b \,x^{3}+a x \right )^{\frac {3}{2}}}{13}-\frac {8 a^{3} x \left (b \,x^{2}+a \right )}{65 b^{\frac {3}{2}} \left (\sqrt {a}+x \sqrt {b}\right ) \sqrt {b \,x^{3}+a x}}+\frac {8 a^{2} x \sqrt {b \,x^{3}+a x}}{195 b}+\frac {4 a \,x^{3} \sqrt {b \,x^{3}+a x}}{39}+\frac {8 a^{\frac {13}{4}} \sqrt {\frac {\cos \left (4 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right )}{2}+\frac {1}{2}}\, \EllipticE \left (\sin \left (2 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right ), \frac {\sqrt {2}}{2}\right ) \left (\sqrt {a}+x \sqrt {b}\right ) \sqrt {x}\, \sqrt {\frac {b \,x^{2}+a}{\left (\sqrt {a}+x \sqrt {b}\right )^{2}}}}{65 \cos \left (2 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right ) b^{\frac {7}{4}} \sqrt {b \,x^{3}+a x}}-\frac {4 a^{\frac {13}{4}} \sqrt {\frac {\cos \left (4 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right )}{2}+\frac {1}{2}}\, \EllipticF \left (\sin \left (2 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right ), \frac {\sqrt {2}}{2}\right ) \left (\sqrt {a}+x \sqrt {b}\right ) \sqrt {x}\, \sqrt {\frac {b \,x^{2}+a}{\left (\sqrt {a}+x \sqrt {b}\right )^{2}}}}{65 \cos \left (2 \arctan \left (\frac {b^{\frac {1}{4}} \sqrt {x}}{a^{\frac {1}{4}}}\right )\right ) b^{\frac {7}{4}} \sqrt {b \,x^{3}+a x}} \]
command
integrate(x*(b*x^3+a*x)^(3/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \frac {2 \, {\left (12 \, a^{3} \sqrt {b} {\rm weierstrassZeta}\left (-\frac {4 \, a}{b}, 0, {\rm weierstrassPInverse}\left (-\frac {4 \, a}{b}, 0, x\right )\right ) + {\left (15 \, b^{3} x^{5} + 25 \, a b^{2} x^{3} + 4 \, a^{2} b x\right )} \sqrt {b x^{3} + a x}\right )}}{195 \, b^{2}} \]
Fricas 1.3.7 via sagemath 9.3 output
\[ {\rm integral}\left ({\left (b x^{4} + a x^{2}\right )} \sqrt {b x^{3} + a x}, x\right ) \]