\[ y'(x)=\frac {2 a x^3 y(x)^2+2 b x^5-y(x)+x y(x) \log (x)}{x (x \log (x)-1)} \] ✗ Mathematica : cpu = 300.025 (sec), leaf count = 0 , timed out
$Aborted
✓ Maple : cpu = 0.052 (sec), leaf count = 45
\[ \left \{ y \left ( x \right ) ={\frac {x}{a}\tan \left ( 2\,\int \!{\frac {{x}^{3}}{x\ln \left ( x \right ) -1}}\,{\rm d}x\sqrt {ab}+2\,{\it \_C1}\,\sqrt {ab} \right ) \sqrt {ab}} \right \} \]