\[ \int \frac {\coth ^{-1}(a x)^2}{x^5} \, dx \]
Optimal antiderivative \[ -\frac {a^{2}}{12 x^{2}}-\frac {a \,\mathrm {arccoth}\left (a x \right )}{6 x^{3}}-\frac {a^{3} \mathrm {arccoth}\left (a x \right )}{2 x}+\frac {a^{4} \mathrm {arccoth}\left (a x \right )^{2}}{4}-\frac {\mathrm {arccoth}\left (a x \right )^{2}}{4 x^{4}}+\frac {2 a^{4} \ln \left (x \right )}{3}-\frac {a^{4} \ln \left (-a^{2} x^{2}+1\right )}{3} \]
command
integrate(arccoth(a*x)^2/x^5,x, algorithm="giac")
Giac 1.9.0-11 via sagemath 9.6 output
\[ \frac {1}{6} \, {\left (4 \, a^{3} \log \left (\frac {a x + 1}{a x - 1} + 1\right ) - 4 \, a^{3} \log \left (\frac {a x + 1}{a x - 1}\right ) + \frac {2 \, {\left (a x + 1\right )} a^{3}}{{\left (a x - 1\right )} {\left (\frac {{\left (a x + 1\right )}^{2}}{{\left (a x - 1\right )}^{2}} + \frac {2 \, {\left (a x + 1\right )}}{a x - 1} + 1\right )}} + \frac {3 \, {\left (\frac {{\left (a x + 1\right )}^{3} a^{3}}{{\left (a x - 1\right )}^{3}} + \frac {{\left (a x + 1\right )} a^{3}}{a x - 1}\right )} \log \left (\frac {a x + 1}{a x - 1}\right )^{2}}{\frac {{\left (a x + 1\right )}^{4}}{{\left (a x - 1\right )}^{4}} + \frac {4 \, {\left (a x + 1\right )}^{3}}{{\left (a x - 1\right )}^{3}} + \frac {6 \, {\left (a x + 1\right )}^{2}}{{\left (a x - 1\right )}^{2}} + \frac {4 \, {\left (a x + 1\right )}}{a x - 1} + 1} + \frac {2 \, {\left (\frac {3 \, {\left (a x + 1\right )}^{2} a^{3}}{{\left (a x - 1\right )}^{2}} + \frac {3 \, {\left (a x + 1\right )} a^{3}}{a x - 1} + 2 \, a^{3}\right )} \log \left (\frac {a x + 1}{a x - 1}\right )}{\frac {{\left (a x + 1\right )}^{3}}{{\left (a x - 1\right )}^{3}} + \frac {3 \, {\left (a x + 1\right )}^{2}}{{\left (a x - 1\right )}^{2}} + \frac {3 \, {\left (a x + 1\right )}}{a x - 1} + 1}\right )} a \]
Giac 1.7.0 via sagemath 9.3 output
\[ \int \frac {\operatorname {arcoth}\left (a x\right )^{2}}{x^{5}}\,{d x} \]________________________________________________________________________________________