\[ \int x^{11} \left (a+b \text {ArcTan}\left (c x^3\right )\right ) \, dx \]
Optimal antiderivative \[ \frac {b \,x^{3}}{12 c^{3}}-\frac {b \,x^{9}}{36 c}-\frac {b \arctan \left (c \,x^{3}\right )}{12 c^{4}}+\frac {x^{12} \left (a +b \arctan \left (c \,x^{3}\right )\right )}{12} \]
command
integrate(x**11*(a+b*atan(c*x**3)),x)
Sympy 1.10.1 under Python 3.10.4 output
\[ \begin {cases} \frac {a x^{12}}{12} + \frac {b x^{12} \operatorname {atan}{\left (c x^{3} \right )}}{12} - \frac {b x^{9}}{36 c} + \frac {b x^{3}}{12 c^{3}} - \frac {b \operatorname {atan}{\left (c x^{3} \right )}}{12 c^{4}} & \text {for}\: c \neq 0 \\\frac {a x^{12}}{12} & \text {otherwise} \end {cases} \]
Sympy 1.8 under Python 3.8.8 output
\[ \text {Timed out} \]________________________________________________________________________________________