These are release notes for the independent CAS integration tests summer 2023 edition This release is dedicated to the memory of Albert D Rich, author of the Rubi integration software. Copy of this release notes in plain text is here Time to complete all tests was 7 months due to having to restart some tests. CAS systems used: ------------------ Mathematica 13.3.1 (August 16, 2023) on windows 10. Rubi 4.16.1 (Dec 19, 2018) on Mathematica 13.3 on windows 10 Maple 2023.1 (July, 12, 2023) on windows 10. Maxima 5.47 (June 1, 2023) using Lisp SBCL 2.3.0/sagemath 10.1 FriCAS 1.3.9 (July 8, 2023) based on sbcl 2.3.0/sagemath 10.1 Giac/Xcas 1.9.0-57 (June 26, 2023) on Linux/sagemath 10.1 Sympy 1.12 (May 10, 2023) Using Python 3.11.3 on Linux. Mupad using Matlab 2021a/Symbolic Math Toolbox Version 8.7 Number of integrals ------------------- sqlite> select COUNT(*) from main; 85963 Number of test input files: [212] Full database (SQL) is updated which contains all integrals and all results used to generate all the reports.(1.3 GB). This link has instructions how to use and install the database. Solved percentage ------------------- 1. Mathematica % 98.233 2. Rubi % 94.217 3. Maple % 86.609 4. Fricas % 80.892 5. Giac % 59.034 6. Maxima(*) % 57.28 --> 57.28+5.516 = % 62.796 (see below on exceptions) 7. Mupad % 57.09 8. Sympy % 43.562 A grade percentage -------------------- 1. Rubi % 87.36 2. Mathematica % 75.34 3. Maple % 60.52 4. Fricas % 49.95 5. Maxima % 39.7 6. Giac % 38.63 7. Sympy % 25.87 8. Mupad N/A (not graded) Average CPU time per integral. Applied only for passed integrals (timed out and exception/failed integrals do not count) --------------------------------------------------------- 1. Rubi 0.28 sec. 2. Maxima 0.44 sec. 3. Fricas 1.18 sec. 4. Giac 2.03 sec. 5. Mathematica 2.31 sec. 6. Sympy 5.36 sec. 7. Maple(*) 5.73 sec. (slower than normal as it tries all algorithms). 8. Mupad 6.82 sec. Average Leaf size ----------------- No simplification is made to the result. This is the leaf size of the antiderivative as is. 1. Rubi 154.14 2. Mathematica 311.05 3. Sympy 373.47 4. Maxima 587.33 5. Giac 632.64 6. Mupad 664.7 7. Fricas 1161.58 8. Maple(*) 64603.4 (Maple typically have larger leaf size as it does not simplify result by default) Main highlights and changes ---------------------------- 1. Added verification to result of Maple. 2. Added number of new statistics diagrams per file and also for the overall results. Statistics now show - percentage solved. - grades. - CPU time and leaf size. - Performance based on number of rules Rubi used. - Performance based on number of steps Rubi used. - Solved integrals histogram based on leaf size of result. - Solved integrals histogram based on CPU time used. - Leaf size vs. CPU time used. - Performance per integrand type. - Maximum leaf size ratio for each CAS against the optimal result. - Pass/Fail per test file for each CAS system. 3. New chapter that lists Integrals which generated an exception failure F(-2) for each CAS. Fricas: ------- sqlite> select DISTINCT fricas_anti from main where fricas_pass=-2; Error detected within library code: integrate: implementation incomplete (constant residues) Error detected within library code: integrate: implementation incomplete (has polynomial part) Error detected within library code: not invertible Error detected within library code: Curve not irreducible after change of variable 0 -> infinity Error detected within library code: integrate: implementation incomplete (residue poly has multiple non-linear factors) Error detected within library code: integrate: implementation incomplete (trace 0) Error detected within library code: catdef: division by zero RuntimeError >> no explicit roots found Error detected within library code: alglogextint: unimplemented Error detected within library code: Not integrable (provided residues have no relations) Error detected within library code: Shouldn not happen Error detected within library code: Too many variables Error detected within library code: failed of mode Union(SparseUnivariatePolynomial(Expression(Complex(Integer))),failed) Error detected within library code: failed of mode Union(SparseUnivariatePolynomial(Expression(Integer)),failed) System error: Heap exhausted 45588576 requested.PROCEED WITH CAUTION. AttributeError Error detected within library code: integrate: implementation incomplete (irrational residues) Error detected within library code: do_alg_rde: unimplemented kernel System error: Heap exhausted (no more space for allocation).PROCEED WITH CAUTION. Number of problems failed in Fricas due to exception is sqlite> select COUNT(*) from main where fricas_pass=-2; 2017 Out of total 85963 integrals, this gives 2.34%. Maxima: -------- select DISTINCT maxima_anti from main where maxima_pass=-2; ValueError >> Computation failed since Maxima requested additional constraints; RuntimeError >> ECL says: expt: undefined: 0 to a negative exponent. RuntimeError >> ECL says: THROW: The catch RAT-ERR is undefined. RuntimeError >> ECL says: In function CAR, the value of the first argument is 1which is not of the expected type LIST TypeError >> unable to make sense of Maxima expression 'li[2]' in Sage RuntimeError >> ECL says: Memory limit reached. Please jump to an outer pointer, quit program and enlarge thememory limits before executing the program again. RuntimeError >> ECL says: BINDING-STACK overflow at size 10240. Stack can probably be resized.Proceed with caution. RuntimeError >> Encountered operator mismatch in maxima-to-sr translation RuntimeError >> ECL says: sign: argument cannot be imaginary; found %i RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_d*_SAGE_VAR_x^2)-2) RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_x^2)-2) RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt(-_SAGE_VAR_b) RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_b*_SAGE_VAR_x^2)-_SAGE_VAR_a) RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt(sqrt(5)-5) The majority of Maxima exceptions are "failed since Maxima requested additional constraints;" sqlite> select COUNT(*) from main where maxima_pass=-2; 6538 The number of exceptions due to asking for assumptions is sqlite> select COUNT(*) from main where maxima_pass=-2 and maxima_anti LIKE '%Maxima requested additional constraints%'; 4742 Which is 72.53% of all the exceptions! Out of total 85963 integrals, this gives 5.516%. So in theory, if there is a way to give Maxima these assumptions in batch mode, and assuming it can solve the integral after that, its percentage solved can increase by 5.516 % from what it currently obtained. GIAC: ------ select DISTINCT giac_anti from main where giac_pass=-2; Invalid _EXT in replace_ext Error: Bad Argument ValueInvalid _EXT in replace_ext Error: Bad Argument ValueDone The choice was done assuming [sageVARx]=[79]sym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: B Limit: Max order reached or unable to make series expansion Error: Bad Argument Value index.cc index_m i_lex_is_greater Error: Bad Argument Value Unable to divide, perhaps due to rounding error%%%{1,[0,3,1,0,0]%%%} / %%%{1,[0,0,0,1,1]%%%} Error: Bad Argument Value The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument Typeintegrate(1/4*16*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^10/(-((sa The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^6/(-((sageVARa int(sage0,sageVARx):;OUTPUT:The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument TypeDone ext_reduce Error: Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^4/(-((sageVARa Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^6/((-((sageVAR Unable to convert to real sageVARb Error: Bad Argument ValueRecursive assumption AttributeError >> type ignoredsym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: Bad Argument Value index.cc index_m operator + Error: Bad Argument Value sym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: Bad Argument Valuesym2poly/r2sym(const gen & unable to parse Giac output Francis algorithm failure for[-1.0,infinity,infinity,infinity,infinity]proot error [1.0,infinity,infinity,infinity,inf Error index.cc index_gcd Error: Bad Argument ValueError index.cc index_gcd Error: Bad Argument ValueDone Not invertible Error: Bad Argument Value Polynomial exponent overflow. Error: Bad Argument Value The choice was done assuming 0=[0,0,0]ext_reduce Error: Bad Argument TypeThe choice was done assuming 0=[0,0,0]ext_red index_gcd Error: Bad Argument Value index.cc index_m operator + Error: Bad Argument Valueindex.cc index_m operator + Error: Bad Argument ValueDone The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument TypeThe choice was done assuming 0=[0,0]ext_reduce Non regular value [0] was discarded and replaced randomly by 0=[-38]Warning, replacing -38 by 97, a substitution varia Unable to find common minimal polynomial Error: Bad Argument ValueDone Warning, replacing 0 by 15, a substitution variable should perhaps be purged.Warning Non regular value [0] was discarded and replaced randomly by 0=[18]Warning, replacing 18 by 97, a substitution variabl index.cc index_m i_lex_is_greater Error: Bad Argument Value ignoredRecursive assumption Limit: Max order reached or unable to make series expansion Error: Bad Argument Value ln of unsigned or minus infinity Error: Bad Argument Value Invalid series expansion: non tractable function acos at +infinity Minimal poly. in rootof must be fraction free Error: Bad Argument ValueMinimal poly. in rootof must be fraction free E sym2poly exception caught Unable to convert to real %%{poly1[1.0000000000000000000000000000000,0.000000000000000000000 Unable to transpose Error: Bad Argument ValueUnable to transpose Error: Bad Argument Valuesym2poly/r2sym(const gen & e Invalid _EXT in replace_ext Error: Bad Argument Valueintegrate rootof minimal polynomial must be unitary Error: Bad Argument Valuerootof minimal polynomial must be unitary Error: Ba Warning: increasing stack size to 2048000. *** Warning: increasing stack size to 2048000. *** Warning: increasing stack size to proot error [1,0,0,0,1,0,0,0,1]proot error [1,0,0,0,-1,0,0,0,1]proot error [1,0,-10,0,1]proot error [1,0,-10,0,1]proot index.cc index_gcd Error: Bad Argument ValueError index.cc index_gcd Error: Bad Argument ValueError index.cc ind Bad Argument Valuesym2poly/r2sym(const gen & e,const index_m & i,con int(sage0,sageVARx):;OUTPUT:int(sage0,sageVARx) Error: Invalid dimension ln of unsigned or minus infinity Error: Bad Argument Value NotImplementedError >> unable to parse Giac output simplify: Polynomials do not have the same dimension Error: Bad Argument Value The majority of exceptions are "Unable to divide, perhaps due to rounding error" Number of problems failed in GIAC due to exception is sqlite> select COUNT(*) from main where giac_pass=-2; 3341 Out of total 85963 integrals, this gives 3.88%. Sympy ------ select DISTINCT sympy_anti from main where sympy_pass=-2; TypeError >> Invalid NaN comparison RecursionError >> maximum recursion depth exceeded TypeError >> Invalid comparison of non-real I HeuristicGCDFailed >> no luck MellinTransformStripError >> Pole inside critical strip? NotImplementedError >> no valid subset found PolynomialError >> 1/(128*_t**4 - 16*sqrt(2)*_t**2 + 1) contains an element of the set of generators. TypeError >> Invalid comparison of non-real zoo TypeError >> '>' not supported between instances of 'Poly' and 'int' AttributeError >> 'NoneType' object has no attribute 'primitive' TypeError >> Invalid comparison of non-real -I CoercionFailed >> Cannot convert _t0**2 + exp(2*I*a) of type to QQ_I[x,b,_t0,exp(I*a)] TypeError >> cannot determine truth value of Relational ValueError >> Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit CoercionFailed >> Cannot convert 5*exp(exp(exp(5))) of type to ZZ[exp(exp(exp(5))/2),sqrt(16 + 5*exp(exp(exp(5))))] GeneratorsError >> polynomial ring and it's ground domain share generators IndexError >> Index out of range: a[1] The majority of exceptions are "PolynomialError" Number of problems failed in sympy due to exception is sqlite> select COUNT(*) from main where sympy_pass=-2; 823 Out of total 85963 integrals, this gives 0.957%. The number of exceptions due to "PolynomialError" is sqlite> select COUNT(*) from main where sympy_pass=-2 and sympy_anti LIKE '%PolynomialError%'; 64 Which is 7.77% of all the sympy exceptions. 4. Listing of integrals solved by CAS which has no known antiderivatives. Note about CPU timing ---------------------- For Mathematica, Rubi and Mupad 3 CPU minutes were used to complete each integration problem. All on windows 10. For Maple it was given 5 minutes because 'method'=':-_RETURNVERBOSE' was used which makes it try all available methods. For Fricas, Giac and Maxima, 5 minutes were also given to compensate for calling them via sagemath which adds extra time to start new proccess each time for each problem. Note that the time recorded for Fricas, Giac and Maxima is inflated by some amount due the time it takes to start new process each time. About 20-30% of the time used can be attributed to the overhead of using sagemath to interface to these systems. Fricas, Giac, Maxima and sympy were all run on Linux inside Virtual box hosted on windows 10 but with over 50 GB of RAM and on very fast PC. Maximum anti derivatives generated per CAS ------------------------------------------ Maple: ------ leaf size 10,839 sqlite> select command_in_maple, MAX(LENGTH(maple_anti)) from main; int(-2^(1/2)*(x^4+2*x^2+4*x+1)^(1/2)+x*(-1+2*2^(1/2)),x,method=_RETURNVERBOSE)|10839 Mathematica: --------------- leaf size 10,005 sqlite> select command_in_mma, MAX(LENGTH(mma_anti)) from main; Integrate[((a + b*Sec[c + d*x])^(3/2)*(A + B*Sec[c + d*x] + C*Sec[c + d*x]^2))/Sec[c + d*x]^(7/2),x]|10005 Rubi: ----- leaf size 41,795 sqlite> select command_in_mma, MAX(LENGTH(rubi_anti)) from main; Integrate[(E^(30 + 8*x)*(56 + 8*E^10 + E^5*(24 - 16*x) - 24*x + 8*x^2) + E^(15 + 4*x)*(-2016 - 8*E^30 + 2208*x - 1760*x^2 + 800*x^3 - 280*x^4 + 56*x^5 - 8*x^6 + E^25*(-56 + 48*x) + E^20*(-280 + 280*x - 120*x^2) + E^15*(-800 + 1120*x - 560*x^2 + 160*x^3) + E^10*(-1760 + 2400*x - 1680*x^2 + 560*x^3 - 120*x^4) + E^5*(-2208 + 3520*x - 2400*x^2 + 1120*x^3 - 280*x^4 + 48*x^5))*Log[3])/(7776 + E^50 + E^45*(10 - 10*x) - 12960*x + 15120*x^2 - 11520*x^3 + 6960*x^4 - 3152*x^5 + 1160*x^6 - 320*x^7 + 70*x^8 - 10*x^9 + x^10 + E^40*(70 - 90*x + 45*x^2) + E^35*(320 - 560*x + 360*x^2 - 120*x^3) + E^30*(1160 - 2240*x + 1960*x^2 - 840*x^3 + 210*x^4) + E^25*(3152 - 6960*x + 6720*x^2 - 3920*x^3 + 1260*x^4 - 252*x^5) + E^20*(6960 - 15760*x + 17400*x^2 - 11200*x^3 + 4900*x^4 - 1260*x^5 + 210*x^6) + E^15*(11520 - 27840*x + 31520*x^2 - 23200*x^3 + 11200*x^4 - 3920*x^5 + 840*x^6 - 120*x^7) + E^10*(15120 - 34560*x + 41760*x^2 - 31520*x^3 + 17400*x^4 - 6720*x^5 + 1960*x^6 - 360*x^7 + 45*x^8) + E^5*(12960 - 30240*x + 34560*x^2 - 27840*x^3 + 15760*x^4 - 6960*x^5 + 2240*x^6 - 560*x^7 + 90*x^8 - 10*x^9)),x]|41795 Mupad ------ Leaf size 1,099,610 select command_in_mupad, MAX(LENGTH(mupad_anti)) from main; int(1/((a + b*sin(e + f*x))^3*(c + d*sin(e + f*x))^3),x)|1099610 Fricas ------ Leaf size 25,209 sqlite> select command_in_fricas, MAX(LENGTH(fricas_anti)) from main; integrate(1/((-1+2*x)^(1/2)*(4+3*x)+(1+x)*(-3+4*x)^(1/2)),x, algorithm="fricas")|25209 Giac ----- Leaf size 5,171,142 sqlite> select command_in_giac, MAX(LENGTH(giac_anti)) from main; integrate(cos(b*x+a)^3*sin(b*x+a)^2/(d*x+c)^4,x, algorithm="giac")|5171142 Maxima ------- Leaf size 3,541,205 sqlite> select command_in_maxima, MAX(LENGTH(maxima_anti)) from main; integrate(cos(d*x+c)^2*(A+B*cos(d*x+c)+C*cos(d*x+c)^2)/(a+a*cos(d*x+c))^(1/2),x, algorithm="maxima")|3541205 Sympy: ------ Leaf size 1,139,416 sqlite> select command_in_sympy, MAX(LENGTH(sympy_anti)) from main; integrate(sinh(d*x+c)/(a+b*sinh(d*x+c)**2),x)|1139416