\[ y'(x)=\frac {2 a x^3 y(x)^2+2 b x^5-y(x)+x y(x) \log (x)}{x (x \log (x)-1)} \] ✗ Mathematica : cpu = 300.033 (sec), leaf count = 0 , timed out
$Aborted
✓ Maple : cpu = 0.067 (sec), leaf count = 38
\[ \left \{ y \left ( x \right ) ={\frac {x}{a}\tan \left ( 2\,\sqrt {ab} \left ( {\it \_C1}+\int \!{\frac {{x}^{3}}{x\ln \left ( x \right ) -1}}\,{\rm d}x \right ) \right ) \sqrt {ab}} \right \} \]