The following is a Bar chart illustration of the data in the above table.
This report gives the result of running the computer algebra independent integration problems.
The listing of the problems used by this report are
The above zip files were downloaded from rulebasedintegration.org.
The current number of problems in this test suite is [71994].
The following systems were tested at this time.
Mathematica 12.3 (64 bit) on windows 10.
Rubi 4.16.1 in Mathematica 12 on windows 10.
Maple 2021.1 (64 bit) on windows 10.
Maxima 5.44 on Linux. (via sagemath 9.3)
Fricas 1.3.7 on Linux (via sagemath 9.3)
Sympy 1.8 under Python 3.8.8 using Anaconda distribution.
Giac/Xcas 1.7 on Linux. (via sagemath 9.3)
Mupad using Matlab 2021a with Symbolic Math Toolbox Version 8.7 under windows 10 (64 bit)
Maxima, Fricas and Giac/Xcas were called from inside SageMath. This was done using SageMath integrate command by changing the name of the algorithm to use the different CAS systems.
Sympy was called directly using Python.
Important note: A number of problems in this test suite have no antiderivative in closed form. This means the antiderivative of these integrals can not be expressed in terms of elementary, special functions or Hypergeometric2F1 functions. RootSum and RootOf are not allowed.
If a CAS returns the above integral unevaluated within the time limit, then the result is counted as passed and assigned an A grade.
However, if CAS times out, then it is assigned an F grade even if the integral is not integrable, as this implies CAS could not determine that the integral is not integrable in the time limit.
If a CAS returns an antiderivative to such an integral, it is assigned an A grade automatically and this special result is listed in the introduction section of each individual test report to make it easy to identify as this can be important result to investigate.
The results given in in the table below reflects the above.
| System | solved | Failed |
| Rubi | % 99.52 ( 71651 ) | % 0.48 ( 343 ) |
| Mathematica | % 98.39 ( 70834 ) | % 1.61 ( 1160 ) |
| Maple | % 83.58 ( 60173 ) | % 16.42 ( 11821 ) |
| Fricas | % 68.7 ( 49460 ) | % 31.3 ( 22534 ) |
| Giac | % 52.43 ( 37749 ) | % 47.57 ( 34245 ) |
| Maxima | % 52.72 ( 37956 ) | % 47.28 ( 34038 ) |
| Sympy | % 34.47 ( 24814 ) | % 65.53 ( 47180 ) |
| Mupad | % 52.55 ( 37830 ) | % 47.45 ( 34164 ) |
The table below gives additional break down of the grading of quality of the antiderivatives generated by each CAS. The grading is given using the letters A,B,C and F with A being the best quality. The grading is accomplished by comparing the antiderivative generated with the optimal antiderivatives included in the test suite. The following table describes the meaning of these grades.
| grade |
description |
| A |
Integral was solved and antiderivative is optimal in quality and leaf size. |
| B |
Integral was solved and antiderivative is optimal in quality but leaf size is larger than twice the optimal antiderivatives leaf size. |
| C |
Integral was solved and antiderivative is non-optimal in quality. This can be due to one or more of the following reasons
|
| F |
Integral was not solved. Either the integral was returned unevaluated within the time limit, or it timed out, or CAS hanged or crashed or an exception was raised. |
Grading is implemented for all CAS systems in this version except for CAS Mupad where a grade of B is automatically assigned as a place holder for all integrals it completes on time.
The following table summarizes the grading results.
| System | % A grade | % B grade | % C grade | % F grade |
| Rubi | 98.89 | 0.23 | 0.41 | 0.48 |
| Mathematica | 74.75 | 6.17 | 17.47 | 1.61 |
| Maple | 52.77 | 22.85 | 7.96 | 16.42 |
| Maxima | 41.43 | 10.17 | 1.11 | 47.28 |
| Fricas | 48.82 | 18.36 | 1.52 | 31.3 |
| Sympy | 26.94 | 4.69 | 2.84 | 65.53 |
| Giac | 38.1 | 13.49 | 0.84 | 47.57 |
| Mupad | 4.93 | 47.61 | 0. | 47.45 |
The following is a Bar chart illustration of the data in the above table.
The figure below compares the CAS systems for each grade level.
The table below summarizes the performance of each CAS system in terms of time used and leaf size of results.
| System | Mean time (sec) | Mean size | Normalized mean | Median size | Normalized median |
| Rubi | 0.28 | 156.75 | 1. | 107. | 1. |
| Mathematica | 1.84 | 799.85 | 2.8 | 92. | 0.94 |
| Maple | 0.79 | 62768.5 | 746.4 | 132. | 1.27 |
| Maxima | 0.92 | 209.89 | 1.76 | 81. | 1. |
| Fricas | 1.56 | 408.24 | 2.78 | 121. | 1.37 |
| Sympy | 8.89 | 250.16 | 2.77 | 70. | 1.13 |
| Giac | 1.05 | 261. | 2. | 92. | 1.12 |
| Mupad | 2.73 | 743.09 | 3.35 | 76. | 1. |
The following are the different integrand types the test suite contains.
Algebraic Binomial problems (products involving powers of binomials and monomials).
Algebraic Trinomial problems (products involving powers of trinomials, binomials and monomials).
Miscellaneous Algebraic functions.
Exponentials.
Logarithms.
Trigonometric.
Inverse Trigonometric.
Hyperbolic functions.
Inverse Hyperbolic functions.
Special functions.
Independent tests.
The following table gives percentage solved of each CAS per integrand type.
| Integrand type | problems | Rubi | Mathematica | Maple | Maxima | Fricas | Sympy | Giac | Mupad |
| Independent tests | 1892 | 98.31 | 98.73 | 93.5 | 81.98 | 94.56 | 72.46 | 86.15 | 82.03 |
| Algebraic Binomial | 14276 | 99.99 | 99.7 | 82.87 | 60.35 | 71.54 | 60.43 | 63.63 | 60.65 |
| Algebraic Trinomial | 10187 | 99.99 | 98.94 | 90.73 | 52.97 | 77. | 40.04 | 66.14 | 56.66 |
| Algebraic Miscellaneous | 1519 | 98.62 | 98.16 | 87.56 | 52.27 | 74.26 | 45.23 | 59.58 | 61.88 |
| Exponentials | 965 | 99.17 | 96.68 | 80.21 | 66.22 | 91.19 | 44.04 | 49.43 | 71.61 |
| Logarithms | 3085 | 98.51 | 97.83 | 54.26 | 56.24 | 58.06 | 32.64 | 45.02 | 43.18 |
| Trigonometric | 22551 | 99.56 | 97.66 | 85.76 | 47.39 | 64.1 | 15.49 | 42.22 | 49.39 |
| Inverse Trigonometric | 4585 | 99.65 | 98.26 | 83.69 | 36.23 | 48.42 | 35.92 | 40.52 | 38.43 |
| Hyperbolic | 5166 | 98.32 | 97.99 | 82.6 | 62.18 | 85.06 | 23.11 | 64.42 | 54.72 |
| Inverse Hyperbolic | 6626 | 99.52 | 98.46 | 79.94 | 47.75 | 62.56 | 27.3 | 35.69 | 39.6 |
| Special functions | 999 | 100. | 95.6 | 69.97 | 39.54 | 48.85 | 42.84 | 34.73 | 40.24 |
In addition to the above table, for each type of integrand listed above, 3D chart is made which shows how each CAS performed on that specific integrand type.
These charts and the table above can be used to show where each CAS relative strength or weakness in the area of integration.
The following table gives the largest ratio found in each test file, between each CAS antiderivative and the optimal antiderivative.
For each test input file, the problem with the largest ratio \(\frac {\text {CAS leaf size}}{\text {Optimal leaf size}}\) is recorded with the corresponding problem number.
In each column in the table below, the first number is the maximum leaf size ratio, and the number that follows inside the parentheses is the problem number in that specific file where this maximum ratio was found. This ratio is determined only when CAS solved the the problem and also when an optimal antiderivative is known.
If it happens that a CAS was not able to solve all the integrals in the input test file, or if it was not possible to obtain leaf size for the CAS result for all the problems in the file, then a zero is used for the ratio and -1 is used for the problem number.
This makes it easy to locate the problem. In the future, a direct link will be added as well.
| | ||||||||
| file # | Rubi | Mathematica | Maple | Maxima | FriCAS | Sympy | Giac | Mupad |
| 1 | 1. (1) | 3.9 (50) | 16.6 (114) | 3.8 (169) | 4. (45) | 7.5 (169) | 4.2 (164) | 42.4 (169) |
| 2 | 7.3 (21) | 7.7 (14) | 3.6 (17) | 1.9 (4) | 14.3 (13) | 16.8 (5) | 4.6 (2) | 3.3 (26) |
| 3 | 1. (1) | 16.1 (6) | 17. (6) | 11.1 (7) | 2. (8) | 1.9 (5) | 1.9 (5) | 11.3 (5) |
| 4 | 6.4 (5) | 14.3 (13) | 40.7 (46) | 16.6 (43) | 5.5 (43) | 4.4 (40) | 5.3 (1) | 6.9 (4) |
| 5 | 1. (65) | 54.7 (278) | 12737.8 (278) | 8.1 (280) | 7.7 (280) | 16.1 (175) | 19.5 (141) | 14.1 (204) |
| 6 | 1. (1) | 1.4 (3) | 2.2 (4) | 1.9 (1) | 1.4 (7) | 0.8 (4) | 2.3 (5) | 1.3 (3) |
| 7 | 2.2 (3) | 5.6 (7) | 1.8 (3) | 2.8 (3) | 6.7 (9) | 5. (2) | 1.9 (3) | 1.7 (3) |
| 8 | 1.6 (50) | 5.3 (31) | 7.9 (70) | 6.5 (11) | 5. (42) | 26.4 (71) | 5.2 (70) | 22.5 (70) |
| 9 | 1.2 (365) | 7.2 (80) | 4.3 (341) | 12.1 (328) | 4.2 (341) | 8. (75) | 15. (328) | 6. (9) |
| 10 | 3.2 (335) | 242.6 (327) | 3343.5 (327) | 36.9 (399) | 32.1 (595) | 76.3 (215) | 18.8 (537) | 12.8 (253) |
| 11 | 529. (82) | 127. (82) | 317. (82) | 2.7 (2) | 70. (82) | 41.3 (17) | 6.6 (50) | 207. (82) |
| 12 | 1.8 (6) | 2.3 (4) | 1.2 (8) | 1.5 (2) | 3.3 (3) | 3.4 (3) | 1.6 (2) | 0.9 (8) |
| 13 | 7.1 (369) | 23.8 (1323) | 30.9 (1323) | 32.9 (1323) | 32.9 (1323) | 136.1 (671) | 34. (1323) | 38.1 (1323) |
| 14 | 2. (870) | 16.5 (1101) | 22.6 (1101) | 22.2 (1716) | 21.8 (1101) | 84.5 (67) | 46.7 (827) | 328.9 (2300) |
| 15 | 3.3 (97) | 9.3 (99) | 28.5 (100) | 2.8 (119) | 10.8 (21) | 49.2 (119) | 10. (119) | 23.6 (21) |
| 16 | 1. (1) | 1.5 (17) | 11. (25) | 4. (25) | 9. (25) | 59.8 (27) | 19.9 (25) | 1.7 (3) |
| 17 | 2.6 (35) | 10.1 (67) | 39.8 (66) | 1.7 (35) | 6.4 (7) | 5.3 (35) | 16.5 (52) | 330.8 (32) |
| 18 | 1. (3) | 27.5 (31) | 68. (35) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) |
| 19 | 8.2 (664) | 6.9 (663) | 7.9 (196) | 10. (196) | 10. (196) | 55.3 (528) | 8. (434) | 10.1 (196) |
| 20 | 1.6 (254) | 6.4 (94) | 147.4 (69) | 4.4 (73) | 19.9 (160) | 10.2 (24) | 5.9 (69) | 37.7 (26) |
| 21 | 1. (596) | 12.6 (337) | 46.7 (754) | 3.1 (313) | 17.2 (1016) | 47.7 (335) | 8.6 (553) | 224.1 (502) |
| 22 | 1.3 (64) | 2.6 (63) | 15.2 (57) | 1.3 (15) | 8.1 (60) | 3. (21) | 3. (98) | 1.7 (21) |
| 23 | 1. (1) | 1.1 (50) | 10.4 (15) | 2. (15) | 7. (15) | 43. (16) | 13.8 (15) | 2.5 (1) |
| 24 | 1.2 (173) | 1.9 (45) | 2. (162) | 3.6 (161) | 5.2 (26) | 47.9 (55) | 4. (157) | 1.8 (133) |
| 25 | 8.4 (2686) | 13.4 (2913) | 141.8 (2913) | 13.2 (2285) | 23. (2913) | 170.1 (2672) | 28.4 (2813) | 16.4 (2913) |
| 26 | 4.3 (116) | 9.5 (335) | 17.9 (265) | 4. (40) | 15.1 (265) | 47.5 (290) | 6. (292) | 62.5 (172) |
| 27 | 4.2 (760) | 12.3 (1051) | 77.4 (546) | 29.1 (1063) | 17.4 (317) | 36.6 (124) | 9.8 (1052) | 78.2 (494) |
| 28 | 1.2 (46) | 0.9 (45) | 51.1 (15) | 2.5 (15) | 28.4 (15) | 78.7 (3) | 49.5 (16) | 7.2 (15) |
| 29 | 1.2 (552) | 3.8 (45) | 10.4 (43) | 10. (43) | 47.2 (73) | 14.9 (577) | 8.1 (591) | 34.3 (171) |
| 30 | 1.3 (278) | 10. (328) | 51.5 (297) | 11.2 (348) | 10.2 (348) | 10. (328) | 10.6 (331) | 12.5 (348) |
| 31 | 1. (1) | 6.4 (283) | 4.9 (269) | 3.2 (114) | 4. (269) | 21.6 (269) | 6.3 (269) | 3.4 (190) |
| 32 | 2.8 (83) | 3.9 (25) | 5.8 (74) | 2.2 (83) | 7.2 (127) | 16.4 (63) | 3.1 (74) | 3.5 (25) |
| 33 | 2. (2419) | 23.9 (2302) | 70.8 (2351) | 28.7 (557) | 36.4 (2293) | 67.3 (1423) | 39.4 (2023) | 209.3 (2300) |
| 34 | 1.3 (1471) | 15.6 (1635) | 82.2 (1180) | 50.9 (2170) | 46.2 (1452) | 64.1 (1011) | 28.9 (1593) | 179.2 (2207) |
| 35 | 2.1 (833) | 58.6 (507) | 116.1 (801) | 5.9 (579) | 34. (616) | 71.6 (920) | 20. (925) | 201.9 (818) |
| 36 | 1. (1) | 10.3 (6) | 425.1 (78) | 2.7 (95) | 30.2 (112) | 1.2 (19) | 13.3 (5) | 3. (100) |
| 37 | 1. (129) | 9.7 (37) | 14197.2 (12) | 6.6 (27) | 30.2 (117) | 8.6 (14) | 5.8 (37) | 110.2 (16) |
| 38 | 1.8 (76) | 42.8 (204) | 421. (278) | 89. (278) | 123.4 (278) | 114.1 (278) | 119.2 (278) | 101.3 (278) |
| 39 | 1.7 (636) | 8.8 (109) | 9.5 (885) | 5.4 (515) | 25.4 (1077) | 28.5 (1105) | 13.7 (885) | 91.3 (1088) |
| 40 | 1.7 (212) | 13.9 (409) | 50.7 (220) | 6.5 (88) | 33.5 (109) | 15.8 (283) | 110.2 (216) | 360. (274) |
| 41 | 1.9 (327) | 32.6 (381) | 26. (136) | 5.6 (70) | 55.6 (305) | 47.5 (220) | 35.3 (309) | 123.2 (309) |
| 42 | 1. (59) | 1.5 (25) | 15.8 (54) | 1.4 (111) | 2.6 (46) | 43. (11) | 21.7 (25) | 107.5 (41) |
| 43 | 1.6 (135) | 2.4 (136) | 13.8 (37) | 1.6 (131) | 48.1 (60) | 27.3 (39) | 20.4 (60) | 94.8 (130) |
| 44 | 1.9 (1) | 6.3 (24) | 6.4 (29) | 0. (-1) | 4.2 (35) | 0.8 (1) | 2.5 (42) | 3.8 (34) |
| 45 | 1. (1) | 4.9 (4) | 0.9 (4) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) |
| 46 | 2.1 (154) | 20. (601) | 54.7 (609) | 6.3 (609) | 46.7 (637) | 21.4 (598) | 46.6 (597) | 99.1 (318) |
| 47 | 1. (1) | 25.5 (83) | 2.7 (37) | 1.8 (68) | 12.2 (37) | 42.2 (68) | 15.3 (37) | 116. (41) |
| 48 | 1. (67) | 25.1 (143) | 2909.3 (93) | 88.7 (96) | 90.4 (93) | 82.9 (93) | 73.6 (96) | 165.9 (51) |
| 49 | 1. (1) | 11. (17) | 1.7 (11) | 2.1 (16) | 2.2 (16) | 3.2 (11) | 3.3 (16) | 215.3 (1) |
| 50 | 1. (1) | 1.7 (99) | 4. (72) | 1.1 (72) | 9.5 (102) | 18.1 (72) | 12.1 (79) | 41.4 (99) |
| 51 | 6.2 (424) | 11.6 (162) | 1223.1 (192) | 42.3 (63) | 93.1 (192) | 84.3 (192) | 27.1 (202) | 166.7 (20) |
| 52 | 4.1 (997) | 172.1 (1010) | 3059.3 (1010) | 5.1 (612) | 40.5 (871) | 58.4 (180) | 16.9 (414) | 54.2 (410) |
| 53 | 1. (1) | 1.2 (82) | 9.5 (87) | 2.2 (2) | 2. (81) | 2.5 (2) | 55.8 (2) | 1.8 (2) |
| 54 | 1. (1) | 1. (1) | 16. (46) | 1.9 (25) | 4.6 (58) | 2.2 (32) | 37.5 (25) | 1.8 (20) |
| 55 | 1.2 (655) | 5.3 (636) | 38.7 (267) | 125.2 (267) | 28.5 (292) | 11.2 (281) | 54.7 (563) | 14.9 (268) |
| 56 | 1. (1) | 1.3 (133) | 83.5 (150) | 4.9 (149) | 5. (150) | 24. (150) | 10.2 (149) | 2.6 (61) |
| 57 | 1.7 (115) | 3.9 (363) | 97.5 (440) | 5.1 (348) | 21.1 (440) | 65.5 (442) | 10.8 (392) | 2.5 (65) |
| 58 | 1.5 (176) | 12.4 (64) | 375.9 (87) | 2.9 (166) | 10.1 (237) | 3.6 (165) | 13.4 (237) | 2.2 (166) |
| 59 | 7.5 (71) | 39.2 (308) | 376.9 (168) | 7.4 (10) | 7.3 (171) | 6.5 (59) | 49.8 (231) | 6.6 (239) |
| 60 | 26.4 (88) | 16.8 (81) | 1428.6 (228) | 79.5 (81) | 9.2 (212) | 8. (71) | 26.8 (1) | 13.6 (83) |
| 61 | 1.6 (79) | 55.1 (51) | 14.7 (74) | 14.2 (44) | 4.5 (15) | 20.3 (12) | 12.3 (34) | 7. (34) |
| 62 | 1.8 (383) | 9.3 (340) | 161.9 (62) | 9.1 (340) | 8.8 (404) | 58.5 (427) | 35.8 (456) | 3.6 (52) |
| 63 | 1.5 (390) | 4.3 (45) | 54.3 (175) | 7.4 (390) | 33.9 (197) | 45.6 (183) | 13.8 (45) | 5.7 (197) |
| 64 | 1.2 (284) | 13.1 (44) | 2190.9 (91) | 10.6 (23) | 11.2 (91) | 15.9 (189) | 15.3 (28) | 8.4 (91) |
| 65 | 1. (1) | 114.1 (497) | 33.3 (493) | 3.9 (111) | 7.8 (301) | 137.4 (62) | 5.7 (105) | 6.2 (210) |
| 66 | 1. (1) | 8.6 (249) | 7.6 (83) | 18.6 (185) | 13.1 (209) | 29.5 (193) | 55.1 (7) | 12. (328) |
| 67 | 1. (1) | 9.2 (12) | 4.3 (51) | 2.4 (21) | 3.6 (5) | 17.1 (49) | 4.3 (7) | 1.8 (21) |
| 68 | 1. (1) | 1.8 (113) | 7.7 (65) | 21.3 (45) | 2.3 (38) | 2.2 (12) | 55.1 (36) | 1.7 (12) |
| 69 | 1. (1) | 3.3 (203) | 7.8 (201) | 168.3 (37) | 4.7 (44) | 8.4 (115) | 10.8 (197) | 2.7 (37) |
| 70 | 2. (615) | 113.3 (353) | 447.5 (605) | 9. (151) | 23.4 (476) | 68.3 (344) | 140.9 (122) | 19.7 (467) |
| 71 | 1. (1) | 1.1 (10) | 1.4 (29) | 8.1 (33) | 1.1 (10) | 3.9 (12) | 2.3 (30) | 1.1 (8) |
| 72 | 1.6 (103) | 56.7 (138) | 3.6 (200) | 4. (53) | 7. (201) | 2.6 (40) | 128.8 (16) | 14.1 (189) |
| 73 | 1.9 (621) | 1029.2 (406) | 4914.7 (790) | 30.7 (256) | 15. (563) | 45.9 (462) | 34.1 (48) | 853.8 (722) |
| 74 | 1.6 (1108) | 1478. (937) | 149.3 (174) | 8.6 (46) | 14.7 (937) | 70.5 (1236) | 42.8 (257) | 41.4 (1131) |
| 75 | 1.3 (12) | 3375. (37) | 688.4 (48) | 7.2 (16) | 28.8 (35) | 3.4 (1) | 1.7 (39) | 132.2 (39) |
| 76 | 1.2 (206) | 85. (202) | 8067.4 (353) | 35.1 (48) | 16.5 (327) | 51.8 (79) | 157.2 (15) | 82. (352) |
| 77 | 1. (1) | 6.7 (10) | 3.9 (2) | 12.4 (1) | 2.3 (2) | 412.4 (8) | 5.4 (12) | 2.4 (3) |
| 78 | 1.4 (32) | 72.5 (30) | 4.4 (33) | 3.3 (20) | 2.2 (18) | 2.3 (32) | 0.9 (32) | 2.9 (1) |
| 79 | 1.8 (236) | 228.2 (240) | 51493.3 (593) | 17.6 (487) | 43.6 (260) | 29. (236) | 19.3 (510) | 295.9 (392) |
| 80 | 1. (1) | 2.2 (2) | 2.1 (4) | 1.3 (2) | 4.6 (1) | 11.7 (4) | 2.3 (2) | 35.6 (4) |
| 81 | 1. (1) | 1.5 (16) | 6. (13) | 1. (19) | 25.3 (1) | 2.8 (11) | 1.8 (14) | 122.9 (1) |
| 82 | 1. (1) | 3.7 (284) | 8.3 (12) | 16.5 (170) | 4.1 (42) | 2.7 (64) | 12.5 (64) | 3.5 (41) |
| 83 | 1. (1) | 4. (62) | 8.3 (76) | 12.1 (133) | 6.9 (33) | 4.1 (9) | 66. (8) | 2.4 (1) |
| 84 | 1. (1) | 2.4 (61) | 3.4 (50) | 2. (5) | 2.7 (5) | 6. (41) | 2. (5) | 1.3 (4) |
| 85 | 1. (1) | 1.3 (94) | 4.2 (26) | 4.2 (86) | 1.5 (35) | 6. (61) | 4.3 (35) | 1.1 (87) |
| 86 | 4.3 (11) | 4.1 (60) | 13.2 (78) | 3.2 (3) | 4.2 (32) | 16.8 (26) | 3.7 (11) | 16.1 (24) |
| 87 | 1. (1) | 1. (10) | 1.4 (29) | 8.1 (32) | 1.1 (10) | 3.8 (12) | 2.3 (30) | 1.1 (8) |
| 88 | 1. (1) | 3.2 (1) | 3.4 (3) | 4.1 (3) | 4.1 (20) | 0. (-1) | 3. (3) | 14.7 (10) |
| 89 | 1.4 (370) | 35.3 (773) | 9.3 (642) | 46.6 (119) | 7.2 (484) | 23. (452) | 57.9 (116) | 29. (479) |
| 90 | 1. (1) | 2.8 (2) | 2.9 (2) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) |
| 91 | 1. (1) | 3. (1) | 1.8 (1) | 3.5 (1) | 1.7 (1) | 0. (-1) | 0. (-1) | 1.2 (1) |
| 92 | 1.1 (40) | 36.7 (454) | 14.3 (436) | 56.4 (95) | 7.5 (278) | 36.2 (252) | 28.6 (78) | 32.8 (267) |
| 93 | 1. (1) | 53.3 (393) | 8. (29) | 20.3 (115) | 3.4 (319) | 9. (35) | 80.5 (35) | 2.8 (122) |
| 94 | 1.4 (940) | 84.8 (1350) | 18. (1154) | 47.3 (96) | 7.2 (590) | 29.3 (565) | 55.5 (78) | 193.6 (981) |
| 95 | 1.2 (81) | 4.9 (91) | 6.9 (70) | 9.4 (53) | 44.5 (85) | 400.9 (20) | 3.7 (91) | 20.3 (14) |
| 96 | 1. (1) | 2.1 (9) | 7.6 (21) | 1. (2) | 8.4 (13) | 13.9 (4) | 7. (4) | 113.8 (15) |
| 97 | 1. (1) | 1.9 (5) | 10.5 (13) | 0.8 (11) | 25.1 (13) | 3.2 (12) | 35.4 (15) | 139.2 (13) |
| 98 | 1. (1) | 105. (358) | 326.6 (179) | 1.3 (7) | 8.5 (251) | 3.1 (376) | 14.2 (7) | 26.5 (105) |
| 99 | 1. (1) | 4.4 (44) | 8.6 (29) | 11.1 (49) | 4.9 (54) | 2.5 (24) | 6.5 (22) | 2.8 (16) |
| 100 | 1. (1) | 3.8 (44) | 1.5 (21) | 7.8 (52) | 4.5 (39) | 16.9 (21) | 1.7 (21) | 2.3 (28) |
| 101 | 1.5 (562) | 75.5 (641) | 173.9 (617) | 18.9 (393) | 8.6 (80) | 40. (172) | 52.2 (73) | 11.3 (560) |
| 102 | 1. (1) | 7. (46) | 4.1 (61) | 2.9 (67) | 7.5 (75) | 1.3 (2) | 77.5 (13) | 6.3 (58) |
| 103 | 1.4 (891) | 200.5 (678) | 9431.6 (611) | 141. (1121) | 111.4 (1249) | 68.2 (1213) | 27.4 (1203) | 547.4 (1257) |
| 104 | 1. (1) | 941.7 (463) | 15282.6 (454) | 144. (373) | 90.9 (369) | 42.2 (280) | 24.3 (257) | 304.5 (428) |
| 105 | 1. (130) | 3975.5 (145) | 172.7 (123) | 3. (83) | 11.4 (83) | 42.2 (37) | 26.2 (51) | 688.1 (115) |
| 106 | 1. (1) | 44.6 (159) | 2905.5 (351) | 18.1 (272) | 20.7 (379) | 67.8 (245) | 31.7 (199) | 2360. (138) |
| 107 | 1. (1) | 777.6 (45) | 31763.8 (14) | 0. (-1) | 16.1 (45) | 0. (-1) | 0. (-1) | 0. (-1) |
| 108 | 1. (1) | 21.6 (47) | 288.2 (43) | 1.3 (4) | 4.9 (20) | 2.6 (1) | 4.2 (3) | 5.9 (7) |
| 109 | 1. (1) | 5.5 (42) | 10.1 (27) | 18.6 (47) | 4.9 (59) | 2.4 (22) | 40.3 (8) | 2.2 (16) |
| 110 | 1. (1) | 2.5 (11) | 3.4 (16) | 3.3 (11) | 4. (7) | 1.3 (2) | 2.5 (7) | 6.7 (17) |
| 111 | 1. (1) | 2.4 (5) | 4.3 (9) | 4.3 (7) | 3.3 (7) | 1.2 (2) | 2.7 (6) | 8.7 (15) |
| 112 | 1. (1) | 3.9 (15) | 69.4 (103) | 1.9 (94) | 4.3 (6) | 35.7 (93) | 2.4 (94) | 51.1 (103) |
| 113 | 1. (1) | 23.7 (22) | 35. (29) | 13.6 (8) | 13. (57) | 64.2 (7) | 19.6 (37) | 32.4 (7) |
| 114 | 1. (1) | 1997.4 (22) | 36459.7 (8) | 0. (-1) | 25.5 (27) | 0. (-1) | 0. (-1) | 0. (-1) |
| 115 | 1. (1) | 14.7 (42) | 9.6 (259) | 25.9 (47) | 5.7 (42) | 3.3 (1) | 11.7 (42) | 3.9 (223) |
| 116 | 1. (1) | 10. (40) | 4.1 (29) | 14.8 (16) | 5.1 (6) | 0. (-1) | 6.4 (18) | 1.8 (18) |
| 117 | 1. (1) | 3.2 (18) | 5.9 (73) | 120.4 (20) | 4.5 (68) | 2.2 (53) | 4.2 (20) | 5.5 (15) |
| 118 | 1.4 (423) | 249. (874) | 14.7 (578) | 52.4 (255) | 7. (515) | 2.6 (5) | 5.8 (513) | 29.9 (520) |
| 119 | 1. (1) | 45.2 (153) | 12.6 (284) | 2.9 (65) | 5.8 (227) | 0. (-1) | 7. (196) | 15.1 (230) |
| 120 | 1.7 (340) | 55.8 (191) | 46.5 (339) | 3.7 (67) | 34.3 (339) | 13.1 (90) | 7.1 (286) | 48.1 (295) |
| 121 | 1.3 (115) | 2602.3 (169) | 1152.4 (153) | 37.5 (109) | 8.8 (159) | 0. (-1) | 5.3 (197) | 42.4 (195) |
| 122 | 2.2 (197) | 1877.2 (240) | 7.1 (238) | 43.2 (130) | 15.3 (263) | 3. (170) | 4.4 (256) | 112. (263) |
| 123 | 1.3 (265) | 350.5 (634) | 15.8 (385) | 52.6 (259) | 8.2 (336) | 2.2 (47) | 6.9 (335) | 33.7 (333) |
| 124 | 1. (1) | 3.6 (65) | 24.1 (25) | 13.5 (25) | 2.6 (58) | 2.9 (33) | 2.7 (41) | 6. (58) |
| 125 | 1.2 (870) | 383.4 (1373) | 19.8 (970) | 45.9 (1289) | 7.3 (808) | 3. (930) | 7.6 (489) | 193.5 (903) |
| 126 | 1.3 (231) | 66.8 (138) | 544.5 (433) | 47.4 (379) | 27.3 (461) | 7.4 (459) | 4.6 (15) | 1013.7 (117) |
| 127 | 1. (1) | 5.6 (42) | 12.4 (21) | 33.4 (39) | 3.8 (42) | 3.1 (1) | 3.1 (41) | 3.7 (61) |
| 128 | 1. (1) | 4. (25) | 5.2 (74) | 39.4 (15) | 4.6 (69) | 2.2 (53) | 2.9 (61) | 23. (27) |
| 129 | 1. (1) | 5.3 (36) | 19.6 (18) | 6.4 (13) | 7.8 (20) | 0. (-1) | 13.6 (15) | 34.8 (50) |
| 130 | 1. (1) | 2.5 (8) | 4. (9) | 4.9 (8) | 3.7 (14) | 0. (-1) | 2.2 (8) | 15.8 (9) |
| 131 | 1.3 (20) | 3.3 (10) | 2.3 (22) | 3.5 (1) | 5. (22) | 0. (-1) | 2.2 (10) | 28.1 (19) |
| 132 | 1. (1) | 2.7 (3) | 2.2 (8) | 2.5 (8) | 2.3 (9) | 4.9 (18) | 3.3 (12) | 3.9 (4) |
| 133 | 1. (1) | 1.2 (1) | 1.8 (1) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) | 0. (-1) |
| 134 | 1. (12) | 3.1 (18) | 26.8 (15) | 26.6 (13) | 16.6 (11) | 0. (-1) | 5.3 (16) | 22.2 (7) |
| 135 | 1. (1) | 29.1 (187) | 4879055.9 (170) | 85. (57) | 7.2 (231) | 330.6 (40) | 78. (71) | 8. (233) |
| 136 | 3.3 (23) | 25.3 (272) | 5.9 (146) | 9.5 (209) | 8.5 (143) | 18.9 (124) | 65.7 (238) | 54.3 (22) |
| 137 | 1.1 (281) | 9.2 (164) | 14.6 (80) | 58.4 (391) | 13.8 (273) | 10.3 (396) | 81.4 (293) | 3.1 (81) |
| 138 | 1. (1) | 2.7 (1) | 6.9 (9) | 0.4 (5) | 12.2 (4) | 1.1 (5) | 0.7 (5) | 2.2 (5) |
| 139 | 4.3 (259) | 8. (318) | 12.8 (259) | 90.9 (225) | 3.1 (173) | 7.6 (18) | 73.7 (126) | 4.1 (224) |
| 140 | 19.2 (34) | 9.1 (133) | 40.1 (34) | 81.3 (34) | 4.2 (63) | 10.8 (42) | 266.8 (31) | 8.6 (63) |
| 141 | 10.8 (759) | 718.9 (434) | 651.2 (860) | 178.8 (64) | 27.7 (503) | 1334.1 (478) | 84.2 (904) | 47.9 (509) |
| 142 | 1.4 (107) | 2.5 (95) | 4.8 (156) | 1.7 (155) | 1.8 (7) | 2.3 (11) | 9.9 (145) | 3. (150) |
| 143 | 1.7 (100) | 9.5 (655) | 19.9 (90) | 3.3 (195) | 6. (642) | 2.9 (413) | 56.7 (620) | 3. (662) |
| 144 | 1.9 (147) | 7. (85) | 13.9 (55) | 12.1 (177) | 8.6 (103) | 8.1 (206) | 14. (233) | 3. (12) |
| 145 | 1.3 (168) | 4.9 (41) | 2.8 (156) | 1.8 (155) | 3. (7) | 2.3 (11) | 26.4 (147) | 2. (150) |
| 146 | 1. (1) | 1.9 (10) | 2.8 (13) | 2.4 (11) | 5.1 (33) | 2. (23) | 36.5 (23) | 1.1 (21) |
| 147 | 1. (1) | 3.8 (13) | 7.8 (12) | 2.4 (24) | 5.7 (29) | 2. (58) | 3.9 (31) | 2.4 (27) |
| 148 | 10. (146) | 4.8 (83) | 28.1 (148) | 1.5 (165) | 3.4 (112) | 8.9 (105) | 1.9 (134) | 1.8 (21) |
| 149 | 1.2 (25) | 4.2 (25) | 44.1 (20) | 1.8 (8) | 19.4 (21) | 44.8 (8) | 4.4 (24) | 4.1 (7) |
| 150 | 1.3 (152) | 6.4 (429) | 85.8 (146) | 4.8 (218) | 9.9 (1223) | 4.2 (197) | 2.4 (1279) | 7.7 (1159) |
| 151 | 1. (1) | 3.3 (36) | 80. (56) | 26.3 (61) | 3. (30) | 9.8 (12) | 1. (27) | 5.2 (1) |
| 152 | 2. (344) | 2.7 (248) | 13.6 (329) | 9.6 (180) | 10. (375) | 11.5 (375) | 8.4 (375) | 4.2 (376) |
| 153 | 1.1 (117) | 11.4 (54) | 27.1 (147) | 5.4 (67) | 5.6 (50) | 13.1 (131) | 5.8 (125) | 5.5 (1) |
| 154 | 1.3 (109) | 11.4 (164) | 72.1 (110) | 13.3 (107) | 6.8 (64) | 5.9 (106) | 27.2 (135) | 5.5 (149) |
| 155 | 1. (1) | 1.2 (7) | 1. (2) | 1. (2) | 1.1 (5) | 2.7 (4) | 1.1 (2) | 0.9 (5) |
| 156 | 1.2 (68) | 2.6 (104) | 11.9 (105) | 3.4 (31) | 8.7 (151) | 2.5 (12) | 75.8 (1) | 2. (29) |
| 157 | 1. (1) | 3.3 (42) | 4.2 (26) | 1.7 (14) | 4. (24) | 2.7 (8) | 2.6 (2) | 1.2 (5) |
| 158 | 1.4 (51) | 2.8 (111) | 11.9 (112) | 1.9 (22) | 8.7 (156) | 2.6 (12) | 27.3 (91) | 1.9 (29) |
| 159 | 1. (1) | 3.3 (40) | 4.9 (26) | 1.6 (13) | 4. (23) | 2.7 (8) | 3.5 (26) | 1.3 (5) |
| 160 | 1. (1) | 16. (299) | 7.5 (379) | 3.7 (327) | 18.6 (329) | 7.4 (297) | 8.7 (6) | 16.6 (489) |
| 161 | 1. (1) | 5.4 (53) | 3.4 (98) | 12.9 (90) | 6.6 (20) | 1.9 (10) | 6.9 (29) | 1.7 (50) |
| 162 | 1. (1) | 1.5 (24) | 1.9 (28) | 6. (7) | 5.4 (21) | 0. (-1) | 1.6 (29) | 0. (-1) |
| 163 | 1. (1) | 8.7 (365) | 8.4 (198) | 21.3 (134) | 32.5 (87) | 16.3 (253) | 25.9 (273) | 22.4 (234) |
| 164 | 1.3 (16) | 9.9 (394) | 15.3 (316) | 21.9 (315) | 64.5 (502) | 103. (65) | 23. (273) | 45.2 (327) |
| 165 | 1. (1) | 13.6 (173) | 6. (1) | 3.6 (1) | 16. (36) | 4.1 (8) | 8.7 (6) | 2.4 (16) |
| 166 | 1. (1) | 1.8 (38) | 3.6 (79) | 3.5 (5) | 4.3 (108) | 2.3 (12) | 18. (32) | 1.7 (12) |
| 167 | 1. (1) | 2.1 (3) | 3.4 (64) | 12.9 (56) | 6.6 (20) | 1.9 (10) | 5.2 (25) | 1.9 (24) |
| 168 | 1. (1) | 1.5 (12) | 1.9 (28) | 6. (7) | 5.4 (21) | 0. (-1) | 1.6 (29) | 0. (-1) |
| 169 | 1. (1) | 8.7 (328) | 7.4 (165) | 11.4 (196) | 40.1 (177) | 22. (152) | 25.9 (246) | 21.4 (180) |
| 170 | 1.3 (60) | 2.5 (11) | 8. (38) | 7.4 (13) | 56.7 (12) | 414.7 (16) | 5. (38) | 53.8 (12) |
| 171 | 1. (1) | 3.7 (3) | 9.8 (43) | 3.2 (8) | 23.5 (11) | 1.7 (8) | 3.4 (8) | 1.6 (32) |
| 172 | 1.2 (109) | 3.5 (212) | 6.6 (102) | 12.6 (188) | 65.9 (200) | 42.3 (62) | 4.2 (102) | 11.1 (85) |
| 173 | 1.3 (257) | 10.5 (252) | 14.4 (114) | 23.3 (190) | 89.5 (249) | 36.6 (195) | 21.1 (108) | 27.4 (106) |
| 174 | 1. (1) | 5.1 (48) | 11. (27) | 3.7 (8) | 20. (47) | 11.7 (27) | 3.2 (8) | 1.6 (8) |
| 175 | 1. (1) | 7.6 (113) | 6.2 (35) | 13. (193) | 66.3 (205) | 11.6 (148) | 7.1 (113) | 11.1 (119) |
| 176 | 1. (1) | 6.7 (10) | 9.2 (24) | 6.3 (10) | 89.6 (41) | 6.4 (5) | 8.7 (10) | 19.2 (7) |
| 177 | 1. (1) | 3.8 (6) | 2.9 (5) | 2.5 (7) | 16. (9) | 0. (-1) | 2.7 (7) | 1.7 (7) |
| 178 | 1. (1) | 3.5 (18) | 3.6 (79) | 2. (15) | 22.6 (82) | 0. (-1) | 2. (31) | 4.7 (79) |
| 179 | 3.5 (186) | 6.4 (145) | 12.7 (186) | 8.6 (59) | 84. (136) | 2.2 (119) | 5.4 (186) | 7.5 (116) |
| 180 | 1.4 (54) | 14.4 (168) | 13.9 (169) | 21.9 (158) | 87.8 (209) | 5.4 (142) | 5. (167) | 18.2 (31) |
| 181 | 1. (1) | 9.1 (26) | 5. (29) | 3.1 (7) | 15.7 (9) | 0. (-1) | 2.8 (7) | 2.2 (25) |
| 182 | 1. (1) | 5.2 (18) | 3.9 (78) | 2.3 (15) | 28.8 (15) | 0. (-1) | 1.9 (5) | 9.7 (81) |
| 183 | 3.3 (160) | 6.7 (24) | 22.3 (24) | 8.9 (91) | 33.8 (124) | 0. (-1) | 9. (24) | 8.2 (120) |
| 184 | 1.1 (12) | 3.4 (24) | 16.5 (8) | 6.2 (1) | 58. (14) | 0. (-1) | 8.8 (22) | 9.5 (1) |
| 185 | 1.9 (192) | 515.8 (777) | 140.9 (767) | 26. (100) | 63.7 (794) | 57.3 (808) | 12.2 (11) | 24.9 (100) |
| 186 | 1. (1) | 1.9 (141) | 2.7 (38) | 1.4 (15) | 3.3 (7) | 1. (22) | 2.3 (19) | 0.9 (5) |
| 187 | 2.1 (73) | 3.5 (230) | 10. (313) | 3.5 (219) | 6. (651) | 3.5 (255) | 2.6 (118) | 1.9 (531) |
| 188 | 1.2 (170) | 4.7 (46) | 6.2 (151) | 12.3 (115) | 5.3 (11) | 8.2 (147) | 8.2 (115) | 2.7 (354) |
| 189 | 1.1 (163) | 3.2 (39) | 2.3 (18) | 1.2 (135) | 2. (7) | 1.1 (135) | 2. (19) | 0.9 (136) |
| 190 | 1.7 (322) | 5.5 (516) | 17.2 (93) | 2.6 (22) | 7. (508) | 1.7 (528) | 2.2 (528) | 1.6 (347) |
| 191 | 1.3 (73) | 6.9 (167) | 26. (291) | 9.2 (93) | 9.7 (20) | 7.6 (122) | 6.1 (93) | 88.8 (279) |
| 192 | 8.1 (149) | 1.9 (31) | 49.6 (28) | 5.3 (202) | 9.9 (216) | 29.6 (63) | 7.5 (1) | 3.3 (200) |
| 193 | 1.6 (21) | 4. (12) | 56.2 (20) | 2.3 (40) | 27.6 (32) | 62.6 (8) | 19. (8) | 8.9 (28) |
| 194 | 1.6 (538) | 4. (156) | 74.3 (235) | 16.1 (244) | 7.1 (516) | 4.5 (307) | 6.7 (15) | 6.9 (244) |
| 195 | 1. (43) | 8.2 (42) | 62.1 (46) | 5.2 (15) | 5. (37) | 20.4 (22) | 28.3 (37) | 16. (22) |
| 196 | 2. (172) | 3.7 (868) | 16.4 (867) | 18.3 (1152) | 12.2 (1368) | 30.7 (997) | 9.7 (1368) | 11.7 (652) |
| 197 | 1.7 (81) | 24. (319) | 24.6 (312) | 4.3 (72) | 6.3 (315) | 2.9 (277) | 7.6 (133) | 30.3 (131) |
| 198 | 1.2 (78) | 24. (238) | 3055.9 (185) | 3.9 (95) | 8.4 (181) | 14.4 (107) | 0.8 (298) | 9.4 (176) |
| 199 | 1.9 (172) | 4.9 (430) | 14.9 (117) | 3.5 (37) | 4.3 (130) | 12.1 (767) | 5.1 (235) | 2.8 (584) |
| 200 | 1. (1) | 16.3 (85) | 19.3 (124) | 1.4 (47) | 8.9 (168) | 1.5 (35) | 0. (-1) | 1.2 (27) |
| 201 | 2.8 (38) | 5.6 (18) | 40.2 (80) | 1. (34) | 9.3 (6) | 0.7 (93) | 3.3 (47) | 9.3 (74) |
| 202 | 1.2 (75) | 2.9 (111) | 11.5 (112) | 2.1 (10) | 9.7 (156) | 1.2 (9) | 0. (-1) | 1.2 (9) |
| 203 | 1.6 (55) | 8.2 (13) | 5.2 (65) | 2.7 (31) | 7.3 (71) | 2. (31) | 2.8 (31) | 1.5 (31) |
| 204 | 1. (1) | 1.7 (102) | 2.5 (221) | 1.1 (31) | 2. (140) | 2.7 (221) | 1.6 (18) | 4.4 (48) |
| 205 | 1. (1) | 2.5 (57) | 1.5 (92) | 0. (-1) | 0. (-1) | 2. (179) | 0. (-1) | 0. (-1) |
| 206 | 1. (1) | 2.6 (41) | 3.3 (134) | 0. (-1) | 0. (-1) | 7.3 (69) | 62.4 (135) | 0. (-1) |
| 207 | 1. (1) | 2.5 (131) | 1.3 (35) | 0. (-1) | 0. (-1) | 8.5 (69) | 0. (-1) | 0. (-1) |
| 208 | 1.1 (174) | 1.3 (195) | 2.4 (144) | 4.1 (155) | 2.7 (28) | 4.9 (30) | 0. (-1) | 1.6 (155) |
The following table gives the number of passed integrals and number of failed integrals per test number. There are 208 tests. Each tests corresponds to one input file.
| Test # | Rubi
| MMA
| Maple
| Maxima
| FriCAS
| Sympy
| Giac
| Mupad
| ||||||||
| Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | Pass | Fail | |
| 1 | 175 | 0 | 175 | 0 | 173 | 2 | 166 | 9 | 172 | 3 | 158 | 17 | 169 | 6 | 169 | 6 |
| 2 | 33 | 2 | 35 | 0 | 27 | 8 | 15 | 20 | 24 | 11 | 7 | 28 | 17 | 18 | 9 | 26 |
| 3 | 13 | 1 | 13 | 1 | 11 | 3 | 8 | 6 | 12 | 2 | 9 | 5 | 10 | 4 | 11 | 3 |
| 4 | 48 | 2 | 50 | 0 | 33 | 17 | 24 | 26 | 48 | 2 | 19 | 31 | 41 | 9 | 12 | 38 |
| 5 | 279 | 5 | 283 | 1 | 282 | 2 | 252 | 32 | 280 | 4 | 249 | 35 | 269 | 15 | 270 | 14 |
| 6 | 3 | 4 | 7 | 0 | 5 | 2 | 3 | 4 | 7 | 0 | 5 | 2 | 5 | 2 | 7 | 0 |
| 7 | 7 | 2 | 9 | 0 | 9 | 0 | 7 | 2 | 9 | 0 | 5 | 4 | 9 | 0 | 9 | 0 |
| 8 | 113 | 0 | 112 | 1 | 113 | 0 | 111 | 2 | 112 | 1 | 104 | 9 | 109 | 4 | 106 | 7 |
| 9 | 376 | 0 | 376 | 0 | 376 | 0 | 374 | 2 | 376 | 0 | 348 | 28 | 375 | 1 | 372 | 4 |
| 10 | 705 | 0 | 705 | 0 | 655 | 50 | 564 | 141 | 652 | 53 | 430 | 275 | 587 | 118 | 542 | 163 |
| 11 | 100 | 16 | 95 | 21 | 77 | 39 | 20 | 96 | 89 | 27 | 29 | 87 | 31 | 85 | 37 | 79 |
| 12 | 8 | 0 | 8 | 0 | 8 | 0 | 7 | 1 | 8 | 0 | 8 | 0 | 8 | 0 | 8 | 0 |
| 13 | 1917 | 0 | 1917 | 0 | 1560 | 357 | 1328 | 589 | 1603 | 314 | 1201 | 716 | 1276 | 641 | 1241 | 676 |
| 14 | 3201 | 0 | 3201 | 0 | 2870 | 331 | 2048 | 1153 | 2535 | 666 | 1547 | 1654 | 2403 | 798 | 1884 | 1317 |
| 15 | 158 | 1 | 155 | 4 | 128 | 31 | 39 | 120 | 47 | 112 | 40 | 119 | 42 | 117 | 49 | 110 |
| 16 | 34 | 0 | 34 | 0 | 28 | 6 | 16 | 18 | 28 | 6 | 13 | 21 | 28 | 6 | 4 | 30 |
| 17 | 78 | 0 | 78 | 0 | 78 | 0 | 27 | 51 | 44 | 34 | 14 | 64 | 33 | 45 | 40 | 38 |
| 18 | 35 | 0 | 35 | 0 | 35 | 0 | 0 | 35 | 0 | 35 | 0 | 35 | 0 | 35 | 0 | 35 |
| 19 | 1071 | 0 | 1071 | 0 | 755 | 316 | 632 | 439 | 674 | 397 | 1013 | 58 | 616 | 455 | 695 | 376 |
| 20 | 349 | 0 | 349 | 0 | 260 | 89 | 79 | 270 | 141 | 208 | 103 | 246 | 108 | 241 | 66 | 283 |
| 21 | 1156 | 0 | 1156 | 0 | 1041 | 115 | 682 | 474 | 856 | 300 | 594 | 562 | 822 | 334 | 730 | 426 |
| 22 | 115 | 0 | 114 | 1 | 105 | 10 | 27 | 88 | 30 | 85 | 27 | 88 | 31 | 84 | 27 | 88 |
| 23 | 51 | 0 | 51 | 0 | 14 | 37 | 14 | 37 | 14 | 37 | 25 | 26 | 14 | 37 | 14 | 37 |
| 24 | 174 | 0 | 174 | 0 | 170 | 4 | 170 | 4 | 158 | 16 | 140 | 34 | 170 | 4 | 129 | 45 |
| 25 | 3078 | 0 | 3044 | 34 | 2591 | 487 | 2196 | 882 | 2370 | 708 | 2688 | 390 | 1990 | 1088 | 2228 | 850 |
| 26 | 385 | 0 | 383 | 2 | 197 | 188 | 167 | 218 | 213 | 172 | 144 | 241 | 134 | 251 | 170 | 215 |
| 27 | 1081 | 0 | 1081 | 0 | 749 | 332 | 391 | 690 | 663 | 418 | 380 | 701 | 535 | 546 | 531 | 550 |
| 28 | 46 | 0 | 46 | 0 | 12 | 34 | 12 | 34 | 12 | 34 | 14 | 32 | 11 | 35 | 12 | 34 |
| 29 | 594 | 0 | 594 | 0 | 577 | 17 | 422 | 172 | 341 | 253 | 444 | 150 | 420 | 174 | 449 | 145 |
| 30 | 454 | 0 | 454 | 0 | 385 | 69 | 153 | 301 | 256 | 198 | 114 | 340 | 238 | 216 | 193 | 261 |
| 31 | 298 | 0 | 296 | 2 | 275 | 23 | 212 | 86 | 228 | 70 | 126 | 172 | 213 | 85 | 197 | 101 |
| 32 | 143 | 0 | 143 | 0 | 113 | 30 | 108 | 35 | 113 | 30 | 47 | 96 | 106 | 37 | 132 | 11 |
| 33 | 2590 | 0 | 2584 | 6 | 2325 | 265 | 1428 | 1162 | 2096 | 494 | 1037 | 1553 | 1678 | 912 | 1589 | 1001 |
| 34 | 2646 | 0 | 2646 | 0 | 2584 | 62 | 1720 | 926 | 2299 | 347 | 1214 | 1432 | 2153 | 493 | 1685 | 961 |
| 35 | 958 | 0 | 942 | 16 | 729 | 229 | 331 | 627 | 594 | 364 | 260 | 698 | 279 | 679 | 276 | 682 |
| 36 | 123 | 0 | 123 | 0 | 121 | 2 | 67 | 56 | 111 | 12 | 43 | 80 | 90 | 33 | 53 | 70 |
| 37 | 143 | 0 | 142 | 1 | 141 | 2 | 15 | 128 | 69 | 74 | 11 | 132 | 50 | 93 | 19 | 124 |
| 38 | 400 | 0 | 394 | 6 | 388 | 12 | 290 | 110 | 330 | 70 | 141 | 259 | 335 | 65 | 195 | 205 |
| 39 | 1126 | 0 | 1126 | 0 | 1062 | 64 | 688 | 438 | 846 | 280 | 476 | 650 | 799 | 327 | 695 | 431 |
| 40 | 412 | 1 | 378 | 35 | 399 | 14 | 113 | 300 | 210 | 203 | 188 | 225 | 177 | 236 | 184 | 229 |
| 41 | 413 | 0 | 400 | 13 | 376 | 37 | 173 | 240 | 256 | 157 | 126 | 287 | 264 | 149 | 218 | 195 |
| 42 | 111 | 0 | 103 | 8 | 111 | 0 | 83 | 28 | 83 | 28 | 47 | 64 | 100 | 11 | 106 | 5 |
| 43 | 145 | 0 | 145 | 0 | 143 | 2 | 73 | 72 | 115 | 30 | 79 | 66 | 139 | 6 | 143 | 2 |
| 44 | 42 | 0 | 38 | 4 | 40 | 2 | 0 | 42 | 9 | 33 | 6 | 36 | 5 | 37 | 1 | 41 |
| 45 | 4 | 0 | 4 | 0 | 4 | 0 | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 |
| 46 | 664 | 0 | 662 | 2 | 496 | 168 | 303 | 361 | 535 | 129 | 282 | 382 | 412 | 252 | 360 | 304 |
| 47 | 96 | 0 | 92 | 4 | 49 | 47 | 17 | 79 | 47 | 49 | 40 | 56 | 37 | 59 | 49 | 47 |
| 48 | 156 | 0 | 147 | 9 | 137 | 19 | 69 | 87 | 108 | 48 | 75 | 81 | 110 | 46 | 122 | 34 |
| 49 | 17 | 0 | 14 | 3 | 2 | 15 | 2 | 15 | 7 | 10 | 1 | 16 | 4 | 13 | 5 | 12 |
| 50 | 140 | 0 | 139 | 1 | 136 | 4 | 24 | 116 | 129 | 11 | 53 | 87 | 106 | 34 | 72 | 68 |
| 51 | 491 | 3 | 494 | 0 | 489 | 5 | 409 | 85 | 431 | 63 | 431 | 63 | 421 | 73 | 485 | 9 |
| 52 | 1007 | 18 | 997 | 28 | 841 | 184 | 385 | 640 | 697 | 328 | 256 | 769 | 484 | 541 | 455 | 570 |
| 53 | 98 | 0 | 98 | 0 | 78 | 20 | 64 | 34 | 93 | 5 | 40 | 58 | 55 | 43 | 58 | 40 |
| 54 | 93 | 0 | 84 | 9 | 75 | 18 | 72 | 21 | 93 | 0 | 50 | 43 | 52 | 41 | 53 | 40 |
| 55 | 766 | 8 | 751 | 23 | 621 | 153 | 503 | 271 | 694 | 80 | 335 | 439 | 370 | 404 | 580 | 194 |
| 56 | 193 | 0 | 193 | 0 | 98 | 95 | 106 | 87 | 123 | 70 | 76 | 117 | 102 | 91 | 60 | 133 |
| 57 | 456 | 0 | 449 | 7 | 309 | 147 | 245 | 211 | 280 | 176 | 231 | 225 | 200 | 256 | 146 | 310 |
| 58 | 249 | 0 | 243 | 6 | 78 | 171 | 68 | 181 | 90 | 159 | 42 | 207 | 58 | 191 | 46 | 203 |
| 59 | 288 | 26 | 298 | 16 | 187 | 127 | 238 | 76 | 210 | 104 | 118 | 196 | 152 | 162 | 200 | 114 |
| 60 | 249 | 14 | 249 | 14 | 98 | 165 | 179 | 84 | 156 | 107 | 50 | 213 | 88 | 175 | 127 | 136 |
| 61 | 106 | 2 | 108 | 0 | 24 | 84 | 68 | 40 | 39 | 69 | 22 | 86 | 34 | 74 | 35 | 73 |
| 62 | 543 | 4 | 543 | 4 | 309 | 238 | 223 | 324 | 221 | 326 | 164 | 383 | 214 | 333 | 209 | 338 |
| 63 | 641 | 0 | 621 | 20 | 337 | 304 | 389 | 252 | 393 | 248 | 177 | 464 | 350 | 291 | 326 | 315 |
| 64 | 314 | 0 | 314 | 0 | 234 | 80 | 219 | 95 | 279 | 35 | 127 | 187 | 191 | 123 | 183 | 131 |
| 65 | 538 | 0 | 538 | 0 | 442 | 96 | 243 | 295 | 286 | 252 | 99 | 439 | 191 | 347 | 248 | 290 |
| 66 | 348 | 0 | 348 | 0 | 264 | 84 | 194 | 154 | 322 | 26 | 113 | 235 | 162 | 186 | 143 | 205 |
| 67 | 72 | 0 | 72 | 0 | 47 | 25 | 32 | 40 | 39 | 33 | 32 | 40 | 39 | 33 | 36 | 36 |
| 68 | 113 | 0 | 113 | 0 | 113 | 0 | 53 | 60 | 113 | 0 | 26 | 87 | 65 | 48 | 20 | 93 |
| 69 | 357 | 0 | 345 | 12 | 245 | 112 | 260 | 97 | 305 | 52 | 105 | 252 | 182 | 175 | 129 | 228 |
| 70 | 653 | 0 | 638 | 15 | 562 | 91 | 288 | 365 | 358 | 295 | 96 | 557 | 278 | 375 | 258 | 395 |
| 71 | 36 | 0 | 36 | 0 | 34 | 2 | 34 | 2 | 36 | 0 | 20 | 16 | 34 | 2 | 16 | 20 |
| 72 | 206 | 2 | 203 | 5 | 178 | 30 | 142 | 66 | 178 | 30 | 4 | 204 | 127 | 81 | 154 | 54 |
| 73 | 837 | 0 | 820 | 17 | 635 | 202 | 217 | 620 | 512 | 325 | 153 | 684 | 302 | 535 | 344 | 493 |
| 74 | 1560 | 3 | 1519 | 44 | 1380 | 183 | 984 | 579 | 1216 | 347 | 222 | 1341 | 1111 | 452 | 1131 | 432 |
| 75 | 51 | 0 | 51 | 0 | 50 | 1 | 16 | 35 | 30 | 21 | 4 | 47 | 6 | 45 | 13 | 38 |
| 76 | 358 | 0 | 348 | 10 | 290 | 68 | 133 | 225 | 275 | 83 | 90 | 268 | 130 | 228 | 178 | 180 |
| 77 | 19 | 0 | 15 | 4 | 12 | 7 | 13 | 6 | 13 | 6 | 8 | 11 | 12 | 7 | 13 | 6 |
| 78 | 34 | 0 | 32 | 2 | 5 | 29 | 7 | 27 | 8 | 26 | 1 | 33 | 1 | 33 | 9 | 25 |
| 79 | 590 | 4 | 583 | 11 | 521 | 73 | 332 | 262 | 397 | 197 | 64 | 530 | 321 | 273 | 334 | 260 |
| 80 | 9 | 0 | 9 | 0 | 9 | 0 | 2 | 7 | 9 | 0 | 5 | 4 | 9 | 0 | 9 | 0 |
| 81 | 19 | 0 | 19 | 0 | 19 | 0 | 5 | 14 | 15 | 4 | 6 | 13 | 9 | 10 | 19 | 0 |
| 82 | 294 | 0 | 294 | 0 | 196 | 98 | 92 | 202 | 93 | 201 | 15 | 279 | 26 | 268 | 80 | 214 |
| 83 | 189 | 0 | 187 | 2 | 135 | 54 | 137 | 52 | 133 | 56 | 54 | 135 | 111 | 78 | 74 | 115 |
| 84 | 62 | 0 | 62 | 0 | 45 | 17 | 37 | 25 | 39 | 23 | 32 | 30 | 39 | 23 | 35 | 27 |
| 85 | 99 | 0 | 99 | 0 | 87 | 12 | 81 | 18 | 91 | 8 | 33 | 66 | 51 | 48 | 30 | 69 |
| 86 | 88 | 0 | 88 | 0 | 88 | 0 | 27 | 61 | 32 | 56 | 22 | 66 | 32 | 56 | 34 | 54 |
| 87 | 34 | 0 | 34 | 0 | 32 | 2 | 32 | 2 | 34 | 0 | 18 | 16 | 32 | 2 | 15 | 19 |
| 88 | 22 | 0 | 22 | 0 | 22 | 0 | 17 | 5 | 21 | 1 | 1 | 21 | 20 | 2 | 18 | 4 |
| 89 | 932 | 0 | 923 | 9 | 854 | 78 | 291 | 641 | 443 | 489 | 95 | 837 | 269 | 663 | 310 | 622 |
| 90 | 4 | 0 | 4 | 0 | 4 | 0 | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 |
| 91 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
| 92 | 644 | 0 | 634 | 10 | 634 | 10 | 189 | 455 | 317 | 327 | 63 | 581 | 196 | 448 | 231 | 413 |
| 93 | 393 | 0 | 389 | 4 | 236 | 157 | 119 | 274 | 121 | 272 | 9 | 384 | 17 | 376 | 75 | 318 |
| 94 | 1541 | 0 | 1534 | 7 | 1533 | 8 | 451 | 1090 | 734 | 807 | 122 | 1419 | 505 | 1036 | 629 | 912 |
| 95 | 98 | 0 | 98 | 0 | 98 | 0 | 70 | 28 | 73 | 25 | 18 | 80 | 75 | 23 | 67 | 31 |
| 96 | 21 | 0 | 21 | 0 | 21 | 0 | 2 | 19 | 16 | 5 | 6 | 15 | 15 | 6 | 19 | 2 |
| 97 | 20 | 0 | 20 | 0 | 20 | 0 | 4 | 16 | 16 | 4 | 5 | 15 | 12 | 8 | 20 | 0 |
| 98 | 387 | 0 | 386 | 1 | 264 | 123 | 137 | 250 | 153 | 234 | 13 | 374 | 67 | 320 | 122 | 265 |
| 99 | 62 | 1 | 63 | 0 | 58 | 5 | 45 | 18 | 63 | 0 | 28 | 35 | 31 | 32 | 32 | 31 |
| 100 | 66 | 0 | 66 | 0 | 36 | 30 | 49 | 17 | 48 | 18 | 36 | 30 | 36 | 30 | 38 | 28 |
| 101 | 700 | 0 | 700 | 0 | 580 | 120 | 405 | 295 | 452 | 248 | 122 | 578 | 247 | 453 | 369 | 331 |
| 102 | 91 | 0 | 90 | 1 | 83 | 8 | 79 | 12 | 83 | 8 | 8 | 83 | 75 | 16 | 83 | 8 |
| 103 | 1328 | 0 | 1210 | 118 | 1114 | 214 | 577 | 751 | 921 | 407 | 289 | 1039 | 478 | 850 | 835 | 493 |
| 104 | 855 | 0 | 798 | 57 | 780 | 75 | 428 | 427 | 583 | 272 | 205 | 650 | 225 | 630 | 529 | 326 |
| 105 | 171 | 0 | 169 | 2 | 122 | 49 | 84 | 87 | 84 | 87 | 58 | 113 | 79 | 92 | 103 | 68 |
| 106 | 499 | 0 | 497 | 2 | 406 | 93 | 269 | 230 | 406 | 93 | 91 | 408 | 209 | 290 | 283 | 216 |
| 107 | 51 | 0 | 51 | 0 | 40 | 11 | 0 | 51 | 16 | 35 | 0 | 51 | 0 | 51 | 0 | 51 |
| 108 | 52 | 0 | 52 | 0 | 37 | 15 | 37 | 15 | 21 | 31 | 8 | 44 | 16 | 36 | 26 | 26 |
| 109 | 61 | 0 | 61 | 0 | 58 | 3 | 46 | 15 | 61 | 0 | 28 | 33 | 35 | 26 | 28 | 33 |
| 110 | 23 | 0 | 23 | 0 | 23 | 0 | 19 | 4 | 23 | 0 | 6 | 17 | 22 | 1 | 23 | 0 |
| 111 | 19 | 0 | 19 | 0 | 19 | 0 | 15 | 4 | 19 | 0 | 4 | 15 | 19 | 0 | 19 | 0 |
| 112 | 106 | 0 | 105 | 1 | 103 | 3 | 79 | 27 | 31 | 75 | 2 | 104 | 3 | 103 | 103 | 3 |
| 113 | 64 | 0 | 64 | 0 | 63 | 1 | 21 | 43 | 64 | 0 | 12 | 52 | 36 | 28 | 39 | 25 |
| 114 | 32 | 0 | 32 | 0 | 25 | 7 | 0 | 32 | 16 | 16 | 0 | 32 | 0 | 32 | 0 | 32 |
| 115 | 299 | 0 | 299 | 0 | 225 | 74 | 93 | 206 | 106 | 193 | 21 | 278 | 35 | 264 | 78 | 221 |
| 116 | 46 | 0 | 45 | 1 | 42 | 4 | 29 | 17 | 46 | 0 | 20 | 26 | 21 | 25 | 24 | 22 |
| 117 | 83 | 0 | 79 | 4 | 51 | 32 | 28 | 55 | 63 | 20 | 37 | 46 | 43 | 40 | 47 | 36 |
| 118 | 879 | 0 | 869 | 10 | 735 | 144 | 309 | 570 | 393 | 486 | 49 | 830 | 262 | 617 | 323 | 556 |
| 119 | 305 | 1 | 304 | 2 | 267 | 39 | 175 | 131 | 191 | 115 | 7 | 299 | 191 | 115 | 193 | 113 |
| 120 | 364 | 1 | 344 | 21 | 331 | 34 | 213 | 152 | 260 | 105 | 40 | 325 | 251 | 114 | 181 | 184 |
| 121 | 240 | 1 | 227 | 14 | 216 | 25 | 96 | 145 | 145 | 96 | 5 | 236 | 44 | 197 | 56 | 185 |
| 122 | 286 | 0 | 273 | 13 | 262 | 24 | 166 | 120 | 236 | 50 | 1 | 285 | 82 | 204 | 191 | 95 |
| 123 | 634 | 0 | 634 | 0 | 586 | 48 | 193 | 441 | 300 | 334 | 8 | 626 | 209 | 425 | 195 | 439 |
| 124 | 70 | 0 | 70 | 0 | 70 | 0 | 48 | 22 | 49 | 21 | 3 | 67 | 46 | 24 | 49 | 21 |
| 125 | 1373 | 0 | 1340 | 33 | 1263 | 110 | 459 | 914 | 732 | 641 | 11 | 1362 | 545 | 828 | 552 | 821 |
| 126 | 468 | 2 | 424 | 46 | 431 | 39 | 286 | 184 | 401 | 69 | 21 | 449 | 163 | 307 | 243 | 227 |
| 127 | 70 | 0 | 70 | 0 | 53 | 17 | 28 | 42 | 31 | 39 | 9 | 61 | 31 | 39 | 16 | 54 |
| 128 | 84 | 0 | 80 | 4 | 52 | 32 | 35 | 49 | 64 | 20 | 37 | 47 | 44 | 40 | 47 | 37 |
| 129 | 59 | 0 | 53 | 6 | 41 | 18 | 25 | 34 | 41 | 18 | 3 | 56 | 38 | 21 | 33 | 26 |
| 130 | 16 | 0 | 16 | 0 | 16 | 0 | 12 | 4 | 16 | 0 | 0 | 16 | 16 | 0 | 16 | 0 |
| 131 | 23 | 0 | 23 | 0 | 23 | 0 | 18 | 5 | 23 | 0 | 0 | 23 | 23 | 0 | 23 | 0 |
| 132 | 24 | 0 | 24 | 0 | 24 | 0 | 24 | 0 | 24 | 0 | 9 | 15 | 24 | 0 | 24 | 0 |
| 133 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
| 134 | 27 | 0 | 27 | 0 | 27 | 0 | 17 | 10 | 27 | 0 | 0 | 27 | 17 | 10 | 8 | 19 |
| 135 | 254 | 0 | 252 | 2 | 215 | 39 | 159 | 95 | 209 | 45 | 54 | 200 | 139 | 115 | 169 | 85 |
| 136 | 294 | 0 | 294 | 0 | 289 | 5 | 271 | 23 | 290 | 4 | 64 | 230 | 279 | 15 | 290 | 4 |
| 137 | 397 | 0 | 396 | 1 | 359 | 38 | 302 | 95 | 365 | 32 | 121 | 276 | 219 | 178 | 155 | 242 |
| 138 | 9 | 0 | 9 | 0 | 9 | 0 | 1 | 8 | 9 | 0 | 1 | 8 | 1 | 8 | 1 | 8 |
| 139 | 254 | 76 | 305 | 25 | 107 | 223 | 139 | 191 | 148 | 182 | 65 | 265 | 74 | 256 | 149 | 181 |
| 140 | 140 | 2 | 142 | 0 | 114 | 28 | 114 | 28 | 115 | 27 | 38 | 104 | 63 | 79 | 50 | 92 |
| 141 | 944 | 6 | 938 | 12 | 908 | 42 | 651 | 299 | 851 | 99 | 418 | 532 | 705 | 245 | 700 | 250 |
| 142 | 227 | 0 | 226 | 1 | 216 | 11 | 66 | 161 | 79 | 148 | 100 | 127 | 163 | 64 | 75 | 152 |
| 143 | 700 | 3 | 694 | 9 | 554 | 149 | 251 | 452 | 265 | 438 | 202 | 501 | 234 | 469 | 146 | 557 |
| 144 | 472 | 2 | 464 | 10 | 376 | 98 | 112 | 362 | 185 | 289 | 162 | 312 | 246 | 228 | 89 | 385 |
| 145 | 227 | 0 | 227 | 0 | 214 | 13 | 67 | 160 | 79 | 148 | 100 | 127 | 163 | 64 | 73 | 154 |
| 146 | 33 | 0 | 33 | 0 | 30 | 3 | 12 | 21 | 15 | 18 | 11 | 22 | 15 | 18 | 3 | 30 |
| 147 | 118 | 0 | 113 | 5 | 78 | 40 | 31 | 87 | 48 | 70 | 33 | 85 | 51 | 67 | 22 | 96 |
| 148 | 157 | 9 | 163 | 3 | 144 | 22 | 93 | 73 | 92 | 74 | 91 | 75 | 80 | 86 | 108 | 58 |
| 149 | 29 | 2 | 28 | 3 | 30 | 1 | 14 | 17 | 11 | 20 | 11 | 20 | 7 | 24 | 14 | 17 |
| 150 | 1301 | 0 | 1287 | 14 | 1200 | 101 | 416 | 885 | 554 | 747 | 553 | 748 | 365 | 936 | 754 | 547 |
| 151 | 70 | 0 | 67 | 3 | 69 | 1 | 37 | 33 | 28 | 42 | 23 | 47 | 6 | 64 | 30 | 40 |
| 152 | 385 | 0 | 368 | 17 | 203 | 182 | 134 | 251 | 286 | 99 | 71 | 314 | 117 | 268 | 147 | 238 |
| 153 | 153 | 0 | 153 | 0 | 133 | 20 | 86 | 67 | 131 | 22 | 50 | 103 | 61 | 92 | 55 | 98 |
| 154 | 234 | 0 | 230 | 4 | 228 | 6 | 142 | 92 | 168 | 66 | 82 | 152 | 111 | 123 | 108 | 126 |
| 155 | 12 | 0 | 12 | 0 | 6 | 6 | 1 | 11 | 6 | 6 | 3 | 9 | 1 | 11 | 6 | 6 |
| 156 | 174 | 0 | 169 | 5 | 139 | 35 | 84 | 90 | 108 | 66 | 67 | 107 | 92 | 82 | 53 | 121 |
| 157 | 50 | 0 | 49 | 1 | 37 | 13 | 18 | 32 | 28 | 22 | 13 | 37 | 27 | 23 | 10 | 40 |
| 158 | 178 | 0 | 173 | 5 | 144 | 34 | 82 | 96 | 110 | 68 | 63 | 115 | 94 | 84 | 57 | 121 |
| 159 | 49 | 0 | 49 | 0 | 36 | 13 | 15 | 34 | 27 | 22 | 12 | 37 | 25 | 24 | 12 | 37 |
| 160 | 502 | 0 | 466 | 36 | 340 | 162 | 294 | 208 | 456 | 46 | 118 | 384 | 199 | 303 | 209 | 293 |
| 161 | 102 | 0 | 101 | 1 | 80 | 22 | 84 | 18 | 78 | 24 | 31 | 71 | 54 | 48 | 29 | 73 |
| 162 | 33 | 0 | 33 | 0 | 31 | 2 | 31 | 2 | 33 | 0 | 9 | 24 | 31 | 2 | 9 | 24 |
| 163 | 369 | 0 | 369 | 0 | 318 | 51 | 266 | 103 | 304 | 65 | 115 | 254 | 276 | 93 | 221 | 148 |
| 164 | 525 | 0 | 501 | 24 | 488 | 37 | 196 | 329 | 366 | 159 | 72 | 453 | 267 | 258 | 247 | 278 |
| 165 | 183 | 0 | 181 | 2 | 110 | 73 | 143 | 40 | 150 | 33 | 61 | 122 | 103 | 80 | 70 | 113 |
| 166 | 111 | 0 | 111 | 0 | 111 | 0 | 64 | 47 | 111 | 0 | 26 | 85 | 71 | 40 | 20 | 91 |
| 167 | 68 | 0 | 68 | 0 | 58 | 10 | 62 | 6 | 60 | 8 | 23 | 45 | 43 | 25 | 21 | 47 |
| 168 | 33 | 0 | 33 | 0 | 31 | 2 | 31 | 2 | 33 | 0 | 9 | 24 | 31 | 2 | 9 | 24 |
| 169 | 336 | 0 | 335 | 1 | 293 | 43 | 208 | 128 | 283 | 53 | 103 | 233 | 259 | 77 | 190 | 146 |
| 170 | 85 | 0 | 84 | 1 | 85 | 0 | 34 | 51 | 67 | 18 | 17 | 68 | 46 | 39 | 55 | 30 |
| 171 | 72 | 5 | 71 | 6 | 69 | 8 | 63 | 14 | 64 | 13 | 30 | 47 | 46 | 31 | 39 | 38 |
| 172 | 206 | 41 | 247 | 0 | 207 | 40 | 151 | 96 | 209 | 38 | 67 | 180 | 187 | 60 | 175 | 72 |
| 173 | 263 | 0 | 263 | 0 | 249 | 14 | 177 | 86 | 246 | 17 | 40 | 223 | 233 | 30 | 185 | 78 |
| 174 | 61 | 0 | 60 | 1 | 58 | 3 | 55 | 6 | 61 | 0 | 28 | 33 | 35 | 26 | 28 | 33 |
| 175 | 183 | 41 | 224 | 0 | 164 | 60 | 105 | 119 | 177 | 47 | 32 | 192 | 137 | 87 | 131 | 93 |
| 176 | 53 | 0 | 53 | 0 | 43 | 10 | 16 | 37 | 50 | 3 | 6 | 47 | 27 | 26 | 32 | 21 |
| 177 | 16 | 0 | 16 | 0 | 8 | 8 | 5 | 11 | 12 | 4 | 3 | 13 | 4 | 12 | 4 | 12 |
| 178 | 84 | 0 | 80 | 4 | 50 | 34 | 39 | 45 | 63 | 21 | 34 | 50 | 44 | 40 | 47 | 37 |
| 179 | 201 | 0 | 192 | 9 | 140 | 61 | 90 | 111 | 142 | 59 | 9 | 192 | 114 | 87 | 94 | 107 |
| 180 | 220 | 0 | 220 | 0 | 180 | 40 | 147 | 73 | 211 | 9 | 10 | 210 | 141 | 79 | 121 | 99 |
| 181 | 29 | 0 | 29 | 0 | 19 | 10 | 13 | 16 | 25 | 4 | 4 | 25 | 8 | 21 | 8 | 21 |
| 182 | 83 | 0 | 74 | 9 | 49 | 34 | 55 | 28 | 62 | 21 | 34 | 49 | 43 | 40 | 47 | 36 |
| 183 | 175 | 0 | 175 | 0 | 136 | 39 | 111 | 64 | 132 | 43 | 0 | 175 | 107 | 68 | 91 | 84 |
| 184 | 27 | 0 | 27 | 0 | 14 | 13 | 10 | 17 | 24 | 3 | 0 | 27 | 20 | 7 | 5 | 22 |
| 185 | 1059 | 0 | 1049 | 10 | 936 | 123 | 762 | 297 | 975 | 84 | 313 | 746 | 802 | 257 | 740 | 319 |
| 186 | 156 | 0 | 156 | 0 | 109 | 47 | 51 | 105 | 43 | 113 | 48 | 108 | 36 | 120 | 30 | 126 |
| 187 | 663 | 0 | 663 | 0 | 498 | 165 | 259 | 404 | 241 | 422 | 185 | 478 | 95 | 568 | 133 | 530 |
| 188 | 370 | 1 | 370 | 1 | 233 | 138 | 117 | 254 | 147 | 224 | 93 | 278 | 97 | 274 | 78 | 293 |
| 189 | 166 | 0 | 166 | 0 | 112 | 54 | 57 | 109 | 52 | 114 | 53 | 113 | 39 | 127 | 32 | 134 |
| 190 | 569 | 0 | 558 | 11 | 471 | 98 | 245 | 324 | 236 | 333 | 154 | 415 | 106 | 463 | 144 | 425 |
| 191 | 295 | 1 | 288 | 8 | 185 | 111 | 81 | 215 | 122 | 174 | 80 | 216 | 84 | 212 | 64 | 232 |
| 192 | 216 | 27 | 231 | 12 | 195 | 48 | 154 | 89 | 147 | 96 | 83 | 160 | 127 | 116 | 128 | 115 |
| 193 | 46 | 3 | 47 | 2 | 48 | 1 | 29 | 20 | 16 | 33 | 10 | 39 | 17 | 32 | 17 | 32 |
| 194 | 538 | 0 | 536 | 2 | 508 | 30 | 270 | 268 | 256 | 282 | 144 | 394 | 175 | 363 | 175 | 363 |
| 195 | 62 | 0 | 60 | 2 | 61 | 1 | 34 | 28 | 17 | 45 | 14 | 48 | 17 | 45 | 17 | 45 |
| 196 | 1378 | 0 | 1353 | 25 | 1100 | 278 | 619 | 759 | 1118 | 260 | 454 | 924 | 675 | 703 | 698 | 680 |
| 197 | 361 | 0 | 361 | 0 | 342 | 19 | 267 | 94 | 338 | 23 | 78 | 283 | 257 | 104 | 239 | 122 |
| 198 | 300 | 0 | 294 | 6 | 273 | 27 | 246 | 54 | 223 | 77 | 100 | 200 | 30 | 270 | 153 | 147 |
| 199 | 935 | 0 | 914 | 21 | 784 | 151 | 502 | 433 | 838 | 97 | 200 | 735 | 490 | 445 | 518 | 417 |
| 200 | 190 | 0 | 185 | 5 | 151 | 39 | 86 | 104 | 120 | 70 | 48 | 142 | 45 | 145 | 52 | 138 |
| 201 | 100 | 0 | 98 | 2 | 74 | 26 | 21 | 79 | 69 | 31 | 1 | 99 | 6 | 94 | 56 | 44 |
| 202 | 178 | 0 | 173 | 5 | 100 | 78 | 84 | 94 | 113 | 65 | 32 | 146 | 46 | 132 | 49 | 129 |
| 203 | 71 | 0 | 71 | 0 | 53 | 18 | 42 | 29 | 49 | 22 | 32 | 39 | 23 | 48 | 41 | 30 |
| 204 | 311 | 0 | 300 | 11 | 179 | 132 | 140 | 171 | 258 | 53 | 168 | 143 | 133 | 178 | 203 | 108 |
| 205 | 218 | 0 | 190 | 28 | 154 | 64 | 60 | 158 | 60 | 158 | 114 | 104 | 60 | 158 | 60 | 158 |
| 206 | 136 | 0 | 134 | 2 | 118 | 18 | 34 | 102 | 34 | 102 | 50 | 86 | 104 | 32 | 34 | 102 |
| 207 | 136 | 0 | 136 | 0 | 104 | 32 | 34 | 102 | 34 | 102 | 50 | 86 | 34 | 102 | 34 | 102 |
| 208 | 198 | 0 | 195 | 3 | 144 | 54 | 127 | 71 | 102 | 96 | 46 | 152 | 16 | 182 | 71 | 127 |
The command AboluteTiming[] was used in Mathematica to obtain the elapsed time for each integrate call. In Maple, the command Usage was used as in the following example
cpu_time := Usage(assign ('result_of _int',int(expr,x)),output='realtime'
For all other CAS systems, the elapsed time to complete each integral was found by taking the difference between the time after the call has completed from the time before the call was made. This was done using Python’s time.time() call.
All elapsed times shown are in seconds. A time limit of 3 minutes was used for each integral. If the integrate command did not complete within this time limit, the integral was aborted and considered to have failed and assigned an F grade. The time used by failed integrals due to time out is not counted in the final statistics.
A verification phase was applied on the result of integration for Rubi and Mathematica. Future version of this report will implement verification for the other CAS systems. For the integrals whose result was not run through a verification phase, it is assumed that the antiderivative produced was correct.
Verification phase has 3 minutes time out. An integral whose result was not verified could still be correct. Further investigation is needed on those integrals which failed verifications. Such integrals are marked in the summary table below and also in each integral separate section so they are easy to identify and locate.
Since these integrals are run in a batch mode, using an automated script, and by using sagemath (SageMath uses Maxima), then any integral where Maxima needs an interactive response from the user to answer a question during evaluation of the integral in order to complete the integration, will fail and is counted as failed.
The exception raised is ValueError. Therefore Maxima result below is lower than what could result if Maxima was run directly and each question Maxima asks was answered correctly.
The percentage of such failures were not counted for each test file, but for an example, for the Timofeev test file, there were about 14 such integrals out of total 705, or about 2 percent. This pecrentage can be higher or lower depending on the specific input test file.
Such integrals can be indentified by looking at the output of the integration in each section for Maxima. The exception message will indicate of the error is due to the interactive question being asked or not.
Maxima integrate was run using SageMath with the following settings set by default
'besselexpand : true' 'display2d : false' 'domain : complex' 'keepfloat : true' 'load(to_poly_solve)' 'load(simplify_sum)' 'load(abs_integrate)' 'load(diag)'
SageMath loading of Maxima abs_integrate was found to cause some problem. So the following code was added to disable this effect.
from sage.interfaces.maxima_lib import maxima_lib maxima_lib.set('extra_definite_integration_methods', '[]') maxima_lib.set('extra_integration_methods', '[]')
See https://ask.sagemath.org/question/43088/integrate-results-that-are-different-from-using-maxima/ for reference.
There are Few integrals which failed due to SageMath not able to translate the result back to SageMath syntax and not because these CAS system were not able to do the integrations.
These will fail With error Exception raised: NotImplementedError
The number of such cases seems to be very small. About 1 or 2 percent of all integrals.
Hopefully the next version of SageMath will have complete translation of FriCAS and XCAS syntax and I will re-run all the tests again when this happens.
For Mathematica, Rubi and Maple, the buildin system function LeafSize is used to find the leaf size of each antiderivative.
The other CAS systems (SageMath and Sympy) do not have special buildin function for this purpose at this time. Therefore the leaf size for Fricas and Sympy and Giac antiderivatives is determined using the following function, thanks to user slelievre at https://ask.sagemath.org/question/57123/could-we-have-a-leaf_count-function-in-base-sagemath/
For Sympy, which is called directly from Python, the following code is used to obtain the leafsize of its result
Matlab’s symbolic toolbox does not have a leaf count function to measure the size of the antiderivative, Maple was used to determine the leaf size of Mupad output by post processing.
Currently no grading of the antiderivative for Mupad is implemented. If it can integrate the problem, it was assigned a B grade automatically as a placeholder. In the future, when grading function is implemented for Mupad, the tests will be rerun again.
The following is an example of using Matlab’s symbolic toolbox (Mupad) to solve an integral
Which gives sin(x)^2/2
The following diagram gives a high level view of the current test build system.
