Skip to content
Snippets Groups Projects
benchmarks.org 45.7 KiB
Newer Older
#+TITLE: Benchmarks for Weighted Tree Automata
#+PROPERTY: header-args  :exports both :eval never-export

* What fits in 16GB RAM

Finding the number of states that copar can solve in 16GB RAM for a few
different types of WTAs.

** Powerset
*** GHC 8.6.4 with symbols 0,8 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 0,8 --zero-frequency 0.7 --good 1400 --bad 1600 --start-states 1500
#+END_SRC

#+RESULTS:
: Trying 1500...
: Trying 1450...
: Trying 1475...
: Trying 1487...
: Trying 1481...
: Trying 1478...
: Trying 1479...
: First bad state count: 1479

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,8_0.7_1478*
#+END_SRC

#+RESULTS:
: 82M bench/wta_powerset_0,8_0.7_1478_0.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_1.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_2.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_3.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_4.coalgebra

*** GHC 8.4.4 with symbols 0,8 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 0,8 --zero-frequency 0.7 --start-states 1000
#+END_SRC

#+RESULTS:
: Trying 1000...
: Trying 2000...
: Trying 1500...
: Trying 1250...
: Trying 1375...
: Trying 1437...
: Trying 1468...
: Trying 1484...
: Trying 1476...
: Trying 1480...
: Trying 1478...
: Trying 1479...
: First bad state count: 1479

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,8_0.7_1478*
#+END_SRC

#+RESULTS:
: 82M bench/wta_powerset_0,8_0.7_1478_0.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_1.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_2.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_3.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_4.coalgebra

**** Automaton size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,8 --zero-frequency 0.7 --states 1478 --indiv --header
#+end_src

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 5243856 | 10484756 |                     10 |                   18 |                             2 |             0 |    115.339077581 |   34.439478263 |       79.458232767 |        27.712258503 |    37.668249537 |
| 1 | 5245405 | 10487854 |                     10 |                   18 |                             2 |             0 |    114.001430765 |   34.477874858 |        78.07613373 |        27.764347556 |    37.074617863 |
| 2 | 5243549 | 10484142 |                     10 |                   18 |                             2 |             0 |    114.862812706 |   34.668409012 |       78.756016051 |        27.741448159 |     37.09480403 |
| 3 | 5241660 | 10480364 |                     10 |                   18 |                             2 |             0 |    114.228588077 |   34.362576104 |       78.424601434 |        27.648526393 |    37.043127108 |
| 4 | 5243032 | 10483108 |                     10 |                   18 |                             2 |             0 |    114.692551901 |   34.827430372 |       78.414037199 |        27.723198479 |    38.276163878 |

*** GHC 8.4.4 with symbols 1,0,4 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 1,0,4 --zero-frequency 0.7
#+END_SRC

#+RESULTS:
: Trying 50...
: Trying 100...
: Trying 200...
: Trying 150...
: Trying 175...
: Trying 162...
: Trying 156...
: Trying 153...
: Trying 151...
: Trying 152...
: First bad state count: 152

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_1,0,4_0.7_152*
#+END_SRC

#+RESULTS:
: 83M bench/wta_powerset_1,0,4_0.7_152_0.coalgebra
: 83M bench/wta_powerset_1,0,4_0.7_152_1.coalgebra
: 83M bench/wta_powerset_1,0,4_0.7_152_2.coalgebra
: 83M bench/wta_powerset_1,0,4_0.7_152_3.coalgebra
: 83M bench/wta_powerset_1,0,4_0.7_152_4.coalgebra

**** Automaton Size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 1,0,4 --zero-frequency 0.7 --states 151 --indiv --header
#+end_src

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4131380 | 12393591 |                      7 |                   69 |                             4 |             0 |     92.530296805 |   32.925871573 |       58.292789973 |        13.413784138 |     32.25285307 |
| 1 | 4131667 | 12394466 |                      7 |                   69 |                             4 |             0 |    104.072447103 |     32.6810851 |       70.054731043 |        22.601698388 |    34.657530397 |
| 2 | 4130798 | 12391837 |                      7 |                   69 |                             4 |             0 |     104.65358267 |   32.641475967 |       70.686110886 |        22.881971907 |    34.580390995 |
| 3 | 4131273 | 12393268 |                      7 |                   69 |                             4 |             0 |    104.436764929 |   33.010643372 |       70.097279338 |        22.722527765 |    34.948100296 |
| 4 | 4131755 | 12394730 |                      7 |                   69 |                             4 |             0 |    103.940596569 |   32.874641172 |       69.748179673 |        22.561661198 |    34.530464508 |

*** GHC 8.4.4 with symbols 4,3,2 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 4,3,2 --zero-frequency 0.7
#+END_SRC

#+RESULTS:
: Trying 50...
: Trying 100...
: Trying 200...
: Trying 150...
: Trying 175...
: Trying 187...
: Trying 193...
: Trying 190...
: Trying 191...
: First bad state count: 191

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_4,3,2_0.7_190*
#+END_SRC

#+RESULTS:
: 83M bench/wta_powerset_4,3,2_0.7_190_0.coalgebra
: 83M bench/wta_powerset_4,3,2_0.7_190_1.coalgebra
: 83M bench/wta_powerset_4,3,2_0.7_190_2.coalgebra
: 83M bench/wta_powerset_4,3,2_0.7_190_3.coalgebra
: 83M bench/wta_powerset_4,3,2_0.7_190_4.coalgebra

**** Automaton Size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 4,3,2 --zero-frequency 0.7 --states 190 --indiv --header
#+end_src

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4150153 | 12416791 |                     11 |                72964 |                           190 |       7474719 |    141.894074912 |    32.94816078 |      104.894412823 |         19.27142335 |    73.292672481 |
| 1 | 4149881 | 12415853 |                     11 |                72964 |                           190 |       7374059 |    143.936708293 |   32.959668876 |      106.847647981 |        19.335579933 |    74.389142529 |
| 2 | 4147912 | 12410153 |                     11 |                72964 |                           190 |       7530637 |    143.039477927 |   32.922891848 |      106.049882108 |        19.238034371 |    73.786648681 |
| 3 | 4149832 | 12415896 |                     11 |                72964 |                           190 |       7400820 |    141.899762867 |   32.763790399 |       105.08945468 |        19.081253852 |    73.103108932 |
| 4 | 4147931 | 12410198 |                     11 |                72964 |                           190 |       7356721 |    142.277553489 |   33.004372363 |      105.135364747 |        19.217062907 |     72.82642293 |

*** GHC 8.4.4 with symbols 0,0,0,0,0,3 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --start-states 2
#+END_SRC

#+RESULTS:
: Trying 2...
: Trying 4...
: Trying 8...
: Trying 16...
: Trying 12...
: Trying 10...
: Trying 11...
: First bad state count: 12

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,0,0,0,0,3_0.7_11*
#+END_SRC

#+RESULTS:
: 47M bench/wta_powerset_0,0,0,0,0,3_0.7_11_0.coalgebra
: 47M bench/wta_powerset_0,0,0,0,0,3_0.7_11_1.coalgebra
: 47M bench/wta_powerset_0,0,0,0,0,3_0.7_11_2.coalgebra
: 47M bench/wta_powerset_0,0,0,0,0,3_0.7_11_3.coalgebra
: 47M bench/wta_powerset_0,0,0,0,0,3_0.7_11_4.coalgebra

**** Automaton Size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --states 11 --indiv --header
#+end_src

#+RESULTS:
| i |  states |   edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 1593503 | 9560952 |                      5 |                   98 |                             2 |             0 |     44.433319213 |   19.470309466 |       24.352231629 |         11.34834025 |    10.606600771 |
| 1 | 1594688 | 9568062 |                      5 |                   98 |                             2 |             0 |      44.35660646 |   19.503070403 |       24.284648797 |        11.546398372 |    10.351405421 |
| 2 | 1595746 | 9574410 |                      5 |                   98 |                             2 |             0 |     44.979436845 |   19.622696684 |       24.756055326 |        11.448256352 |    10.958179969 |
| 3 | 1594811 | 9568800 |                      5 |                   98 |                             2 |             0 |     44.518512407 |   19.486959131 |       24.465812029 |        11.501689163 |     10.63837111 |
| 4 | 1595187 | 9571056 |                      5 |                   98 |                             2 |             0 |     44.449440224 |   19.537019031 |       24.300846734 |        11.389487771 |      10.5363747 |

** Z,max
*** GHC 8.4.4 with symbols 0,8 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Z,max' --symbols 0,8 --zero-frequency 0.7
#+END_SRC

#+RESULTS:
: Trying 50...
: Trying 100...
: Trying 200...
: Trying 400...
: Trying 800...
: Trying 1600...
: Trying 1200...
: Trying 1400...
: Trying 1500...
: Trying 1450...
: Trying 1475...
: Trying 1462...
: Trying 1456...
: Trying 1453...
: Trying 1451...
: First bad state count: 1451

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Z,max_0,8_0.7_1450*
#+END_SRC

#+RESULTS:
: 182M bench/wta_Z,max_0,8_0.7_1450_0.coalgebra
: 182M bench/wta_Z,max_0,8_0.7_1450_1.coalgebra
: 182M bench/wta_Z,max_0,8_0.7_1450_2.coalgebra
: 182M bench/wta_Z,max_0,8_0.7_1450_3.coalgebra
: 182M bench/wta_Z,max_0,8_0.7_1450_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Z,max' --symbols 0,8 --zero-frequency 0.7 --states 1450 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 5047826 | 10092752 |                   1458 |                13050 |                          1450 |       9457197 |    127.724360284 |   43.258927035 |       81.171137974 |        26.874901612 |    40.422676395 |
| 1 | 5047514 | 10092128 |                   1458 |                13050 |                          1450 |       9456756 |    125.832804664 |   42.853972016 |       79.686342664 |        26.577174808 |    39.966384527 |
| 2 | 5049772 | 10096644 |                   1458 |                13050 |                          1450 |       9460968 |    126.336593045 |    42.68312591 |       80.349875424 |        26.745235991 |    40.031230363 |
| 3 | 5046916 | 10090932 |                   1458 |                13050 |                          1450 |       9455805 |     125.95962352 |   42.526542018 |       80.176310943 |        26.550162046 |    40.227319833 |
| 4 | 5049229 | 10095558 |                   1458 |                13050 |                          1450 |       9460347 |    126.718920083 |   42.559013501 |       80.863052096 |        26.756405734 |    40.145229524 |

*** GHC 8.4.4 with symbols 1,0,4 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Z,max' --symbols 1,0,4 --zero-frequency 0.7
#+END_SRC

#+RESULTS:
: Trying 50...
: Trying 100...
: Trying 200...
: Trying 150...
: Trying 175...
: Trying 162...
: Trying 156...
: Trying 153...
: Trying 151...
: First bad state count: 151

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Z,max_1,0,4_0.7_150*
#+END_SRC

**** File Size

#+RESULTS:
: 162M bench/wta_Z,max_1,0,4_0.7_150_0.coalgebra
: 162M bench/wta_Z,max_1,0,4_0.7_150_1.coalgebra
: 162M bench/wta_Z,max_1,0,4_0.7_150_2.coalgebra
: 162M bench/wta_Z,max_1,0,4_0.7_150_3.coalgebra
: 162M bench/wta_Z,max_1,0,4_0.7_150_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Z,max' --symbols 1,0,4 --zero-frequency 0.7 --states 150 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4051480 | 12153896 |                    155 |                90151 |                           150 |       7087519 |    120.395749162 |   40.871788157 |        74.90886142 |        22.238988672 |     39.72932804 |
| 1 | 4050707 | 12151583 |                    155 |                90151 |                           150 |       7087488 |    120.463605814 |    40.81720097 |       75.096220332 |         22.78192433 |    39.512466086 |
| 2 | 4048842 | 12145992 |                    155 |                90151 |                           150 |       7083654 |    120.367574362 |   40.847872293 |       74.879748499 |        22.174812232 |    39.882652278 |
| 3 | 4052208 | 12156090 |                    155 |                90151 |                           150 |       7090069 |    120.237306821 |   40.747046417 |       74.945644255 |        22.185753095 |    39.520022235 |
| 4 | 4048823 | 12145929 |                    155 |                90151 |                           150 |       7084229 |    121.046334089 |   41.206127141 |       75.236966807 |        22.108849249 |    39.746848148 |

*** GHC 8.4.4 with symbols 4,3,2 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Z,max' --symbols 4,3,2 --zero-frequency 0.7 --start-states 100
#+END_SRC

#+RESULTS:
: Trying 100...
: Trying 200...
: Trying 150...
: Trying 175...
: Trying 187...
: Trying 193...
: Trying 190...
: Trying 188...
: Trying 189...
: First bad state count: 189

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Z,max_4,3,2_0.7_188*
#+END_SRC

#+RESULTS:
: 162M bench/wta_Z,max_4,3,2_0.7_188_0.coalgebra
: 162M bench/wta_Z,max_4,3,2_0.7_188_1.coalgebra
: 162M bench/wta_Z,max_4,3,2_0.7_188_2.coalgebra
: 162M bench/wta_Z,max_4,3,2_0.7_188_3.coalgebra
: 162M bench/wta_Z,max_4,3,2_0.7_188_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Z,max' --symbols 4,3,2 --zero-frequency 0.7 --states 188 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4020872 | 12029832 |                    197 |                71444 |                           188 |       6045852 |    115.942447016 |    40.75075879 |       71.271681594 |        18.625916768 |     39.91853539 |
| 1 | 4017121 | 12018620 |                    197 |                71444 |                           188 |       6040501 |    113.960683621 |   40.210261362 |       69.908854506 |        18.715510494 |    39.429755908 |
| 2 | 4017092 | 12018743 |                    197 |                71444 |                           188 |       6040038 |    114.417975118 |   40.056454421 |       70.449543532 |        18.443439893 |    39.714702929 |
| 3 | 4013171 | 12006729 |                    197 |                71444 |                           188 |       6034912 |    104.421419394 |   40.426583126 |       60.116296609 |        10.279226244 |    37.258447268 |
| 4 | 4018149 | 12021895 |                    197 |                71444 |                           188 |       6042204 |    104.595285757 |   40.484957759 |        60.29935373 |        10.313693438 |    37.315322541 |

*** GHC 8.4.4 with symbols 0,0,0,0,0,3 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Z,max' --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --start-states 6
#+END_SRC

#+RESULTS:
: Trying 6...
: Trying 12...
: Trying 9...
: Trying 10...
: Trying 11...
: First bad state count: 12

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Z,max_0,0,0,0,0,3_0.7_11*
#+END_SRC

#+RESULTS:
: 79M bench/wta_Z,max_0,0,0,0,0,3_0.7_11_0.coalgebra
: 79M bench/wta_Z,max_0,0,0,0,0,3_0.7_11_1.coalgebra
: 79M bench/wta_Z,max_0,0,0,0,0,3_0.7_11_2.coalgebra
: 79M bench/wta_Z,max_0,0,0,0,0,3_0.7_11_3.coalgebra
: 79M bench/wta_Z,max_0,0,0,0,0,3_0.7_11_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Z,max' --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --states 11 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |   edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 1595450 | 9572634 |                     14 |               473731 |                            11 |       2679440 |     59.567674089 |   22.815217966 |       34.656233896 |        10.712862563 |    21.527223791 |
| 1 | 1596274 | 9577578 |                     14 |               473599 |                            11 |       2680789 |     59.667949978 |   22.964709813 |       34.595654672 |        10.671440318 |    21.504327033 |
| 2 | 1593533 | 9561132 |                     14 |               473725 |                            11 |       2676188 |      59.60896951 |   22.965296594 |       34.468009878 |        10.694780509 |    21.357284246 |
| 3 | 1593597 | 9561516 |                     14 |               473516 |                            11 |       2675256 |     59.618517937 |   22.900218752 |       34.517770264 |        10.699678956 |    21.404437124 |
| 4 | 1595682 | 9574026 |                     14 |               473759 |                            11 |       2679419 |     59.829033166 |    22.84743033 |       34.719394797 |        10.690592405 |    21.632768706 |

** Word,or
*** GHC 8.4.4 with symbols 0,8 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Word,or' --symbols 0,8 --zero-frequency 0.7 --start-states 1000
#+END_SRC

#+RESULTS:
: Trying 1000...
: Trying 2000...
: Trying 1500...
: Trying 1250...
: Trying 1375...
: Trying 1437...
: Trying 1406...
: Trying 1421...
: Trying 1413...
: Trying 1409...
: Trying 1407...
: Trying 1408...
: First bad state count: 1409

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Word,or_0,8_0.7_1408*
#+END_SRC

#+RESULTS:
: 164M bench/wta_Word,or_0,8_0.7_1408_0.coalgebra
: 164M bench/wta_Word,or_0,8_0.7_1408_1.coalgebra
: 165M bench/wta_Word,or_0,8_0.7_1408_2.coalgebra
: 165M bench/wta_Word,or_0,8_0.7_1408_3.coalgebra
: 165M bench/wta_Word,or_0,8_0.7_1408_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Word,or' --symbols 0,8 --zero-frequency 0.7 --states 1408 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |   edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4757493 | 9512170 |                   1416 |                12672 |                          1408 |       8913294 |    122.529064018 |   49.673619826 |       69.872621378 |        24.487186873 |    37.636595831 |
| 1 | 4757305 | 9511794 |                   1416 |                12672 |                          1408 |       8912679 |    121.871882143 |   49.468009438 |       69.396700752 |        24.548033181 |    37.526833514 |
| 2 | 4760364 | 9517912 |                   1416 |                12672 |                          1408 |       8918643 |    122.195401637 |   49.367139075 |       69.859922188 |        24.551202679 |    37.745627532 |
| 3 | 4760232 | 9517648 |                   1416 |                12672 |                          1408 |       8918564 |    121.744240716 |   49.628682967 |       69.111452032 |        24.573090085 |    37.819264634 |
| 4 | 4759753 | 9516690 |                   1416 |                12672 |                          1408 |       8917624 |    118.671354206 |   49.213411686 |       66.440058464 |        25.139216435 |    33.058420248 |

*** GHC 8.4.4 with symbols 1,0,4 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Word,or' --symbols 1,0,4 --zero-frequency 0.7 --start-states 120
#+END_SRC

#+RESULTS:
: Trying 120...
: Trying 240...
: Trying 180...
: Trying 150...
: Trying 135...
: Trying 142...
: Trying 146...
: Trying 148...
: Trying 149...
: First bad state count: 149

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Word,or_1,0,4_0.7_148*
#+END_SRC

#+RESULTS:
: 151M bench/wta_Word,or_1,0,4_0.7_148_0.coalgebra
: 151M bench/wta_Word,or_1,0,4_0.7_148_1.coalgebra
: 151M bench/wta_Word,or_1,0,4_0.7_148_2.coalgebra
: 151M bench/wta_Word,or_1,0,4_0.7_148_3.coalgebra
: 151M bench/wta_Word,or_1,0,4_0.7_148_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Word,or' --symbols 1,0,4 --zero-frequency 0.7 --states 148 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 3889583 | 11668189 |                    153 |                87765 |                           148 |       6805177 |    117.573809962 |   44.159477335 |       69.176479615 |         20.97881901 |     36.97313268 |
| 1 | 3891836 | 11675004 |                    153 |                87765 |                           148 |       6809318 |    118.060187599 |   44.059742822 |       69.709799716 |        20.986764025 |    37.089169403 |
| 2 | 3889498 | 11667962 |                    153 |                87765 |                           148 |       6805428 |    117.680086041 |   43.801892931 |       69.639925802 |        20.888711457 |    36.894495934 |
| 3 | 3890675 | 11671481 |                    153 |                87765 |                           148 |       6807190 |    118.213230214 |   44.026034118 |       69.934017821 |        20.807216331 |    36.920223682 |
| 4 | 3889335 | 11667465 |                    153 |                87765 |                           148 |       6804884 |    118.020940476 |   44.097067581 |       69.813347698 |        20.923033139 |    36.907214948 |

*** GHC 8.4.4 with symbols 4,3,2 and zero-freq 0.7

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Word,or' --symbols 4,3,2 --zero-frequency 0.7
#+END_SRC

#+RESULTS:
: Trying 50...
: Trying 100...
: Trying 200...
: Trying 150...
: Trying 175...
: Trying 187...
: Trying 181...
: Trying 184...
: Trying 185...
: Trying 186...
: First bad state count: 187

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Word,or_4,3,2_0.7_186*
#+END_SRC

#+RESULTS:
: 152M bench/wta_Word,or_4,3,2_0.7_186_0.coalgebra
: 152M bench/wta_Word,or_4,3,2_0.7_186_1.coalgebra
: 152M bench/wta_Word,or_4,3,2_0.7_186_2.coalgebra
: 152M bench/wta_Word,or_4,3,2_0.7_186_3.coalgebra
: 152M bench/wta_Word,or_4,3,2_0.7_186_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Word,or' --symbols 4,3,2 --zero-frequency 0.7 --states 186 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 3890269 | 11638838 |                    195 |                69940 |                           186 |       5848699 |    114.733470175 |   43.499787726 |         67.5948969 |        17.660572753 |    37.835620352 |
| 1 | 3893818 | 11649123 |                    195 |                69940 |                           186 |       5854005 |    114.756947419 |   43.702682275 |       67.505722972 |        17.675712966 |    37.818639736 |
| 2 | 3893265 | 11647592 |                    195 |                69940 |                           186 |       5854134 |    115.418354168 |   43.606980048 |       68.133828807 |        17.837028275 |    37.992011256 |
| 3 | 3892206 | 11644421 |                    195 |                69940 |                           186 |       5852423 |    113.998060941 |   43.769845642 |       66.618120946 |        17.720427448 |    37.714434677 |
| 4 | 3892036 | 11643814 |                    195 |                69940 |                           186 |       5851296 |    115.965290499 |   44.138476758 |       68.234040288 |        17.763408679 |    38.356640369 |

*** GHC 8.4.4 with symbols 0,0,0,0,0,3 and zero-freq 0.7 
#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid 'Word,or' --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --start-states 6                                                                            
#+END_SRC

#+RESULTS:
: Trying 6...                          
: Trying 12...
: Trying 9...
: Trying 10...
: Trying 11...
: First bad state count: 12

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_Word,or_0,0,0,0,0,3_0.7_11*
#+END_SRC

#+RESULTS:
: 77M bench/wta_Word,or_0,0,0,0,0,3_0.7_11_0.coalgebra
: 77M bench/wta_Word,or_0,0,0,0,0,3_0.7_11_1.coalgebra
: 77M bench/wta_Word,or_0,0,0,0,0,3_0.7_11_2.coalgebra
: 77M bench/wta_Word,or_0,0,0,0,0,3_0.7_11_3.coalgebra
: 77M bench/wta_Word,or_0,0,0,0,0,3_0.7_11_4.coalgebra

**** Automaton Size

#+BEGIN_SRC sh
python bench.py run ../../copar/bin/copar --monoid 'Word,or' --symbols 0,0,0,0,0,3 --zero-frequency 0.7 --states 11 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |   edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 1594413 | 9566412 |                     14 |               473587 |                            11 |       2676959 |      61.53939505 |    24.79942316 |       34.477272309 |        10.822832613 |    21.313535924 |
| 1 | 1596360 | 9578094 |                     14 |               473404 |                            11 |       2681034 |     61.147599248 |   24.824030133 |       34.101896988 |           10.793184 |    20.989457877 |
| 2 | 1595591 | 9573480 |                     14 |               473700 |                            11 |       2679546 |     61.459443653 |   24.801616514 |       34.389540062 |        10.893824596 |    21.133849133 |
| 3 | 1593825 | 9562884 |                     14 |               473580 |                            11 |       2676789 |     61.147557505 |    24.98879381 |       33.942496577 |        10.818043846 |    20.839884809 |
| 4 | 1596528 | 9579102 |                     14 |               473600 |                            11 |       2681048 |     61.864094058 |   24.884111862 |       34.762990208 |        10.893522867 |    21.485077775 |
* Same Question, but with limited transition count
** Powerset
*** GHC 8.4.4 with symbols 0,8 and transitions 500

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/{random-wta,copar} --monoid powerset --symbols 0,8 --transitions 5000
#+END_SRC

#+RESULTS:
: Trying 1000...
: Trying 2000...
: Trying 1500...
: Trying 1250...
: Trying 1375...
: Trying 1437...
: Trying 1468...
: Trying 1484...
: Trying 1476...
: Trying 1480...
: Trying 1478...
: Trying 1479...
: First bad state count: 1479

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,8_0.7_1478*
#+END_SRC

#+RESULTS:
: 82M bench/wta_powerset_0,8_0.7_1478_0.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_1.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_2.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_3.coalgebra
: 82M bench/wta_powerset_0,8_0.7_1478_4.coalgebra

**** Automaton size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,8 --zero-frequency 0.7 --states 1478 --indiv --header
#+end_src

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 5243856 | 10484756 |                     10 |                   18 |                             2 |             0 |    115.339077581 |   34.439478263 |       79.458232767 |        27.712258503 |    37.668249537 |
| 1 | 5245405 | 10487854 |                     10 |                   18 |                             2 |             0 |    114.001430765 |   34.477874858 |        78.07613373 |        27.764347556 |    37.074617863 |
| 2 | 5243549 | 10484142 |                     10 |                   18 |                             2 |             0 |    114.862812706 |   34.668409012 |       78.756016051 |        27.741448159 |     37.09480403 |
| 3 | 5241660 | 10480364 |                     10 |                   18 |                             2 |             0 |    114.228588077 |   34.362576104 |       78.424601434 |        27.648526393 |    37.043127108 |
| 4 | 5243032 | 10483108 |                     10 |                   18 |                             2 |             0 |    114.692551901 |   34.827430372 |       78.414037199 |        27.723198479 |    38.276163878 |

*** GHC 8.4.4 with symbols 0,8 and out-degree 1000

#+BEGIN_SRC sh
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,8 --out-degree 1000 --good 1000
#+END_SRC

#+RESULTS:
: Trying 1001...
: Trying 2002...
: Trying 4004...
: Trying 8008...
: Trying 6006...
: Trying 7007...
: Trying 7507...
: Trying 7257...
: Trying 7382...
: Trying 7444...
: Trying 7413...
: Trying 7428...
: Trying 7420...
: Trying 7416...
: Trying 7414...
: Trying 7415...
: First bad state count: 7416

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,8_t7415000__7415*
#+END_SRC

#+RESULTS:
: 120M bench/wta_powerset_0,8_t7415000__7415_0.coalgebra
: 120M bench/wta_powerset_0,8_t7415000__7415_1.coalgebra
: 120M bench/wta_powerset_0,8_t7415000__7415_2.coalgebra
: 120M bench/wta_powerset_0,8_t7415000__7415_3.coalgebra
: 120M bench/wta_powerset_0,8_t7415000__7415_4.coalgebra

**** Automaton Size

#+begin_src sh :hlines yes
python bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,8 --transitions 7415000 --states 7415 --indiv --header
#+end_src

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 7422415 | 14830000 |                     10 |                   18 |                             2 |             0 |    157.872966092 |   51.221314999 |      104.615248956 |          33.7012898 |    58.664739269 |
| 1 | 7422415 | 14830000 |                     10 |                   18 |                             2 |             0 |    157.428380106 |   51.433510926 |      103.926494829 |        33.216537153 |    58.775334589 |
| 2 | 7422415 | 14830000 |                     10 |                   18 |                             2 |             0 |    156.808413994 |   51.343226544 |       103.40051371 |        32.969125294 |    58.865798525 |
| 3 | 7422415 | 14830000 |                     10 |                   18 |                             2 |             0 |    157.791363724 |   51.873644617 |      103.855838314 |         33.05713406 |    58.931293578 |
| 4 | 7422415 | 14830000 |                     10 |                   18 |                             2 |             0 |    157.126256596 |   51.400301031 |      103.666329167 |        32.976280595 |    58.830566981 |

* Next Try

We now use the following parameters:

- Signature :: 4×X^k k ∈ { 0, ..., 5 }
- GHC :: 8.4.4

** Powerset
Hans-Peter Deifel's avatar
Hans-Peter Deifel committed
*** 4xX mit out-degree 500

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 500 --good 8000 --start-states 12000 --bad 16000
#+END_SRC

: Trying 12000...
: Trying 14000...
: Trying 15000...
: Trying 14500...
: Trying 14250...
: Trying 14375...
: Trying 14312...
: Trying 14281...
: Trying 14296...
: Trying 14288...
: Trying 14284...
: Trying 14282...
: First bad state count: 14282

**** File Size

#+BEGIN_SRC sh :results output
ls -sh bench/wta_powerset_0,4_*__14281*
#+END_SRC

#+RESULTS:
: 118M bench/wta_powerset_0,4_t7140500__14281_0.coalgebra

**** Automaton Size

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,4 --out-degree 500 --states 14281
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 500 --states 14281 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 7154781 | 14281000 |                      6 |                   10 |                             2 |             0 |    155.635043092 |   50.511676534 |      103.239235361 |        30.135940008 |    61.445316625 |
| 1 | 7154781 | 14281000 |                      6 |                   10 |                             2 |             0 |    155.345664537 |   50.391815858 |      103.046448356 |         29.52881681 |    61.823502784 |
| 2 | 7154781 | 14281000 |                      6 |                   10 |                             2 |             0 |    154.746389702 |   50.356318464 |      102.487287332 |        29.561887582 |    61.245616512 |
| 3 | 7154781 | 14281000 |                      6 |                   10 |                             2 |             0 |    156.681042711 |   50.477922803 |      104.306181251 |        30.132009036 |    62.231759233 |
| 4 | 7154781 | 14281000 |                      6 |                   10 |                             2 |             0 |    156.206931801 |   50.552217812 |      103.775304876 |        30.106012782 |    61.793692847 |

*** 4xX mit out-degree 20

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 20 --good 8000 --start-states 14281
#+END_SRC

: Trying 14281...
: Trying 28562...
: Trying 57124...
: Trying 114248...
: Trying 228496...
: Trying 456992...
: Trying 342744...
: Trying 399868...
: Trying 371306...
: Trying 357025...
: Trying 349884...
: Trying 353454...
: Trying 355239...
: Trying 356132...
: Trying 355685...
: Trying 355462...
: Trying 355573...
: Trying 355629...
: Trying 355657...
: Trying 355671...
: Trying 355678...
: Trying 355681...
: Trying 355683...
: Trying 355684...
: First bad state count: 355685

**** File Size


**** Automaton Size

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,4 --out-degree 20 --states 355684
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 20 --states 355684 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 7469364 | 14227360 |                      6 |              1768728 |                        355684 |      10883373 |    344.399732107 |   61.942569869 |      262.572157652 |        32.047148894 |   209.333188906 |
| 1 | 7469364 | 14227360 |                      6 |              1768935 |                        355684 |      10883077 |    340.732038227 |   60.912573518 |      260.176273796 |        31.928906963 |   208.837646082 |
| 2 | 7469364 | 14227360 |                      6 |              1768908 |                        355684 |      10885631 |    340.838739608 |   61.120402905 |      260.008098908 |        31.951679391 |   208.775189688 |
| 3 | 7469364 | 14227360 |                      6 |              1768880 |                        355684 |      10880372 |    340.237713603 |   61.448840572 |      259.027481579 |        31.936433445 |   207.318948419 |
| 4 | 7469364 | 14227360 |                      6 |              1768875 |                        355684 |      10886155 |    342.792217839 |   61.487509806 |      261.434824046 |        32.045849362 |   209.995181997 |

*** 4xX mit out-degree 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 50 --good 130000 --bad 140000
#+END_SRC

: Trying 135000...
: Trying 132500...
: Trying 131250...
: Trying 131875...
: Trying 132187...
: Trying 132031...
: Trying 132109...
: Trying 132148...
: Trying 132167...
: Trying 132177...
: Trying 132182...
: Trying 132179...
: Trying 132178...
: First bad state count: 132179


**** Automaton Size

#+BEGIN_SRC shell
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,4 --out-degree 50 --states 132178 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 6741078 | 13217800 |                      6 |               660888 |                        132178 |       9275215 |    263.557139582 |   53.134795016 |      197.899988998 |        33.273151066 |   153.369899287 |

*** 4xX^2 mit out-degree 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,0,4 --out-degree 50 --start-states 80000
#+END_SRC

: Trying 80000...
: Trying 160000...
: Trying 120000...
: Trying 100000...
: Trying 90000...
: Trying 95000...
: Trying 97500...
: Trying 98750...
: Trying 98125...
: Trying 98437...
: Trying 98593...
: Trying 98671...
: Trying 98632...
: Trying 98651...
: Trying 98661...
: Trying 98666...
: Trying 98668...
: Trying 98669...
: Trying 98670...
: First bad state count: 98671

**** Automaton Size

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,0,4 --out-degree 50 --states 98670
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,0,4 --out-degree 50 --states 98670 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 5032170 | 14800500 |                      6 |              5031861 |                         98670 |      11563672 |    320.579655827 |   46.524207289 |       261.58724583 |        31.537501676 |   213.989949977 |

*** 4xX^3 mit out-degree 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,0,0,4 --out-degree 50 --start-states 60000
#+END_SRC

: Trying 60000...
: Trying 120000...
: Trying 90000...
: Trying 75000...
: Trying 82500...
: Trying 86250...
: Trying 84375...
: Trying 85312...
: Trying 84843...
: Trying 85077...
: Trying 84960...
: Trying 85018...
: Trying 84989...
: Trying 85003...
: Trying 85010...
: Trying 85014...
: Trying 85016...
: Trying 85017...
: First bad state count: 85018

Hans-Peter Deifel's avatar
Hans-Peter Deifel committed
**** Automaton Size 

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,0,0,4 --out-degree 50 --states 85016
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,0,0,4 --out-degree 50 --states 85016 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 4335816 | 17003200 |                      6 |              4335816 |                         85016 |      11932210 |    261.096600392 |   47.614392976 |      203.643561011 |        20.446586332 |   167.587154416 |

*** 4xX^4 mit out-degree 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,0,0,0,4 --out-degree 50 --start-states 48000
#+END_SRC

: Trying 48000...
: Trying 96000...
: Trying 72000...
: Trying 60000...
: Trying 54000...
: Trying 57000...
: Trying 58500...
: Trying 59250...
: Trying 59625...
: Trying 59437...
: Trying 59531...
: Trying 59578...
: Trying 59601...
: Trying 59589...
: Trying 59595...
: Trying 59598...
: Trying 59596...
: First bad state count: 59596

Hans-Peter Deifel's avatar
Hans-Peter Deifel committed
**** Automaton Size

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,0,0,0,4 --out-degree 50 --states 59596
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,0,0,0,4 --out-degree 50 --states 59596 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 3039396 | 14899000 |                      6 |              3039396 |                         59596 |      13024716 |    200.034099951 |   41.065064963 |      152.485306449 |        25.383501709 |   121.316305478 |

*** 4xX^5 mit out-degree 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid powerset --symbols 0,0,0,0,0,4 --out-degree 50 --start-states 40000
#+END_SRC

: Trying 40000...
: Trying 80000...
: Trying 60000...
: Trying 50000...
: Trying 45000...
: Trying 47500...
: Trying 48750...
: Trying 49375...
: Trying 49687...
: Trying 49531...
: Trying 49453...
: Trying 49414...
: Trying 49394...
: Trying 49384...
: Trying 49379...
: Trying 49377...
: Trying 49376...
: First bad state count: 49376
Hans-Peter Deifel's avatar
Hans-Peter Deifel committed

**** Automaton Size 

#+BEGIN_SRC shell
./bench.py generate ../../copar/bin/random-wta --monoid powerset --symbols 0,0,0,0,0,4 --out-degree 50 --states 49375
./bench.py run ../../copar/bin/copar --monoid powerset --symbols 0,0,0,0,0,4 --out-degree 50 --states 49375 --indiv --header
#+END_SRC

#+RESULTS:
| i |  states |    edges | initial-partition-size | final-partition-size | explicit-final-partition-size | size1-skipped | overall-duration | parse-duration | algorithm-duration | initialize-duration | refine-duration |
| 0 | 2518125 | 14812500 |                      6 |              2518125 |                         49375 |      13940781 |    164.986249384 |   38.769441334 |      119.843574846 |        24.415685672 |    90.703821725 |

*** Z,max
**** 4xX mit out-degree 50 und different-values 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid Z,max --symbols 0,4 --out-degree 50 --good 100000 --bad 130000 --different-values 50
#+END_SRC

: Trying 115000...
: Trying 107500...
: Trying 111250...
: Trying 113125...
: Trying 114062...
: Trying 114531...
: Trying 114765...
: Trying 114882...
: Trying 114941...
: Trying 114911...
: Trying 114896...
: Trying 114889...
: Trying 114885...
: Trying 114887...
: Trying 114888...
: First bad state count: 114889

**** 4xX^2 mit out-degree 50 und different-values 50

#+BEGIN_SRC shell
./bench.py bisect ../../copar/bin/random-wta ../../copar/bin/copar --monoid Z,max --symbols 0,0,4 --out-degree 50 --good 50000 --bad 100000 --different-values 50
#+END_SRC

: Trying 75000...
: Trying 87500...
: Trying 93750...
: Trying 96875...
: Trying 95312...