\[ \int \coth ^5(x) \sqrt {a+b \text {sech}^2(x)} \, dx \]
Optimal antiderivative \[ -\frac {\left (8 a^{2}+12 a b +3 b^{2}\right ) \arctanh \left (\frac {\sqrt {a +b \mathrm {sech}\left (x \right )^{2}}}{\sqrt {a +b}}\right )}{8 \left (a +b \right )^{\frac {3}{2}}}+\arctanh \left (\frac {\sqrt {a +b \mathrm {sech}\left (x \right )^{2}}}{\sqrt {a}}\right ) \sqrt {a}-\frac {\left (4 a +3 b \right ) \left (\coth ^{2}\left (x \right )\right ) \sqrt {a +b \mathrm {sech}\left (x \right )^{2}}}{8 \left (a +b \right )}-\frac {\left (\coth ^{4}\left (x \right )\right ) \sqrt {a +b \mathrm {sech}\left (x \right )^{2}}}{4} \]
command
integrate(coth(x)^5*(a+b*sech(x)^2)^(1/2),x, algorithm="fricas")
Fricas 1.3.8 (sbcl 2.2.11.debian) via sagemath 9.6 output
\[ \text {output too large to display} \]
Fricas 1.3.7 via sagemath 9.3 output \[ \text {Timed out} \]