Skip to content

[TEST] Benchmark pure Queue implementations #235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 82 commits into
base: main
Choose a base branch
from
Draft

[TEST] Benchmark pure Queue implementations #235

wants to merge 82 commits into from

Conversation

Kamirus
Copy link
Contributor

@Kamirus Kamirus commented Mar 24, 2025 •
edited
Loading

To be abandoned once the Real-Time queue gets merged.

TODO:

  • A single benchmark that compares all 3 queue implementations should be added in a separate PR (this PR should serve as archive with more comprehensive benchmarks)

Kamirus and others added 30 commits March 5, 2025 12:49
@Kamirus
types + some funcs
d97cb2c
@Kamirus
Merge branch 'main' into kamil/deque
847bbab
@Kamirus
bigstate, smallstate, commonstate
aa66dbe
@Kamirus
missing small state functions + states
5826b9e
@Kamirus
pushFront and Back
f1ba61a
@Kamirus
pop operations
1eaf4b3
@Kamirus
Merge branch 'main' into kamil/deque
f7cab44
@Kamirus
fix missing List.push after merge
af4e79b
@Kamirus
empty, isEmpty, singleton, size. Also: docs, refactor, fixes
d05beb1
@Kamirus
stacks as module
da12689
@Kamirus
current as module
a818404
@Kamirus
refactor
13d747d
@Kamirus
docs, refactors and pre-contains
610f44e
@Kamirus
first tests, fix popBack signature
6a2236b
@Kamirus
remove wrong assert
460503e
@Kamirus
almost all tests migrated
5b4aaaf
@Kamirus
missing operations
20205b8
@Kamirus
whole test suite ported
b366ebf
@Kamirus
all operations done + refactor
d756b15
@Kamirus
test missing functions
7c203a5
@Kamirus
more code coverage
8b04421
@Kamirus
more coverage
2908292
@Kamirus
add reverse test
5503764
@Kamirus
more edge case tests
df049d6
@Kamirus
debug size invariants
380f3c8
@Kamirus
remove done todos
6a03bad
@Kamirus
peek without pop
7effbae
@Kamirus
documentation
7e45ba6
@Kamirus
improve docs
c996edf
@Kamirus
arbitrary order in operations for limiting allocations
d1d56ab
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 24, 2025 •
edited
Loading

✨ Documentation preview for 80d8012:

https://dfinity.github.io/new-motoko-base/pull/235 (source code)

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 24, 2025 •
edited
Loading

Benchmark Results

Benchmark files:
• bench/FromIters.bench.mo
• bench/Skewed.bench.mo
• bench/ZLongPopsLarge.bench.mo
• bench/ZLongUniform.bench.mo

Benchmarking the fromIter functions

Columns describe the number of elements in the input iter.

Instructions

100 10_000 100_000
Array.fromIter 53_243 $({\color{gray}0\%})$ 5_152_204 $({\color{gray}0\%})$ 51_503_819 $({\color{gray}0\%})$
List.fromIter 39_313 $({\color{gray}0\%})$ 3_801_900 $({\color{gray}0\%})$ 38_004_901 $({\color{gray}0\%})$

Heap

100 10_000 100_000
Array.fromIter 272 B $({\color{gray}0\%})$ 272 B $({\color{gray}0\%})$ 272 B $({\color{gray}0\%})$
List.fromIter 272 B $({\color{gray}0\%})$ 272 B $({\color{gray}0\%})$ 272 B $({\color{gray}0\%})$

Garbage Collection

100 10_000 100_000
Array.fromIter 2.76 KiB $({\color{gray}0\%})$ 234.79 KiB $({\color{gray}0\%})$ 2.29 MiB $({\color{gray}0\%})$
List.fromIter 3.53 KiB $({\color{gray}0\%})$ 312.91 KiB $({\color{gray}0\%})$ 3.05 MiB $({\color{gray}0\%})$

No previous results found "/home/runner/work/new-motoko-base/new-motoko-base/.bench/Skewed.bench.json"

Compare queue implementations

Instructions

Real-Time Amortized Mutable
Initialize with 2 elements 2_583 3_480 3_324
Push 500 elements 1_058_212 103_177 243_222
Pop front 2 elements 4_283 76_248 3_909
Pop back 2 elements 5_798 4_469 4_721
Push 10 front&back; Pop 5 front&back 27_315 13_714 20_393
Pop 150 front&back 388_890 107_602 146_139

Heap

Real-Time Amortized Mutable
Initialize with 2 elements 300 B 324 B 352 B
Push 500 elements 8.17 KiB 8.08 KiB 19.8 KiB
Pop front 2 elements 240 B 240 B 192 B
Pop back 2 elements 240 B 240 B 192 B
Push 10 front&back; Pop 5 front&back 432 B 432 B 672 B
Pop 150 front&back -1.22 KiB -4.42 KiB -11.45 KiB

Garbage Collection

Real-Time Amortized Mutable
Initialize with 2 elements 472 B 516 B 456 B
Push 500 elements 176.28 KiB 10.1 KiB 344 B
Pop front 2 elements 592 B 8.29 KiB 424 B
Pop back 2 elements 736 B 448 B 424 B
Push 10 front&back; Pop 5 front&back 4.33 KiB 1.32 KiB 832 B
Pop 150 front&back 61.52 KiB 15.61 KiB 12.1 KiB

No previous results found "/home/runner/work/new-motoko-base/new-motoko-base/.bench/ZLongPopsLarge.bench.json"
2025-03-26 12:23:54.192309673 UTC: [Canister avqkn-guaaa-aaaaa-qaaea-cai] Real-Time {avg = 2_248; max = ?6_880; mean = 1_045; total = 34_746_012}
2025-03-26 12:23:54.433322850 UTC: [Canister avqkn-guaaa-aaaaa-qaaea-cai] Amortized {avg = 358; max = ?1_445_509; mean = 188; total = 5_536_647}
2025-03-26 12:23:54.673164044 UTC: [Canister avqkn-guaaa-aaaaa-qaaea-cai] Mutable {avg = 449; max = ?595; mean = 469; total = 6_946_759}

Compare queue implementations

PushFront 10_000 elements at step 0, then perform 110 steps of 50 x popBack

Instructions

Real-Time Amortized Mutable
0 27_343_307 13_589_160 16_399_262
1 115_826 1_522_599 84_503
2 115_808 77_345 84_485
3 118_151 79_688 86_828
4 115_808 77_345 84_485
5 118_151 79_688 86_828
6 115_808 77_345 84_485
7 115_808 77_345 84_485
8 118_151 79_688 86_828
9 115_808 77_345 84_485
10 118_258 79_858 86_872
11 115_897 77_371 84_511
12 115_897 77_371 84_511
13 118_240 79_714 86_854
14 115_834 77_371 84_511
15 115_834 77_371 84_511
16 259_113 79_714 86_854
17 293_997 77_371 84_511
18 296_340 79_714 86_854
19 293_997 77_371 84_511
20 293_971 77_408 84_485
21 296_314 79_688 86_828
22 293_908 77_345 84_485
23 296_251 79_688 86_828
24 293_908 77_345 84_485
25 293_908 77_345 84_485
26 296_251 79_688 86_828
27 278_767 77_345 84_485
28 234_451 79_688 86_828
29 232_108 77_345 84_485
30 232_108 77_345 84_485
31 234_451 79_688 86_828
32 232_108 77_345 84_485
33 234_451 79_688 86_828
34 232_108 77_345 84_485
35 237_355 77_345 84_485
36 245_051 79_688 86_828
37 242_708 77_345 84_485
38 242_708 77_345 84_485
39 306_371 79_688 86_828
40 331_908 77_345 84_485
41 334_314 79_688 86_828
42 331_971 77_345 84_485
43 331_971 77_345 84_485
44 334_314 79_688 86_828
45 331_971 77_345 84_485
46 334_377 79_688 86_828
47 331_971 77_345 84_485
48 331_971 77_345 84_485
49 334_314 79_688 86_828
50 334_118 77_345 84_485
51 352_427 79_688 86_828
52 349_958 77_345 84_485
53 289_782 77_345 84_485
54 279_314 79_688 86_828
55 276_971 77_345 84_485
56 277_127 77_438 84_578
57 279_284 79_595 86_735
58 276_971 77_345 84_485
59 279_251 79_688 86_828
60 276_908 77_345 84_485
61 276_908 77_345 84_485
62 184_716 79_688 86_828
63 115_808 77_345 84_485
64 118_151 79_688 86_828
65 115_808 77_345 84_485
66 115_808 77_345 84_485
67 118_151 79_688 86_828
68 115_808 77_345 84_485
69 118_151 79_688 86_828
70 115_808 77_345 84_485
71 115_808 77_345 84_485
72 118_151 79_688 86_828
73 115_808 77_345 84_485
74 118_151 79_688 86_828
75 115_808 77_345 84_485
76 115_871 77_408 84_485
77 154_178 79_688 86_828
78 304_571 77_471 84_485
79 304_571 77_345 84_485
80 306_914 79_688 86_828
81 304_508 77_345 84_485
82 306_851 79_688 86_828
83 304_508 77_345 84_485
84 304_508 77_345 84_485
85 276_260 79_688 86_828
86 242_708 77_345 84_485
87 245_051 79_688 86_828
88 242_708 77_345 84_485
89 242_708 77_345 84_485
90 245_051 79_688 86_828
91 242_708 77_345 84_485
92 245_051 79_688 86_828
93 312_948 77_345 84_485
94 331_908 77_345 84_485
95 334_251 79_688 86_828
96 331_908 77_345 84_485
97 334_251 79_688 86_828
98 331_908 77_345 84_485
99 331_908 77_345 84_485
100 335_611 79_735 86_875
101 350_005 800_065 84_532
102 310_283 77_518 84_532
103 279_361 79_735 86_875
104 277_018 77_392 84_532
105 279_424 79_735 86_875
106 277_081 77_392 84_532
107 277_018 77_392 84_532
108 217_046 79_735 86_875
109 124_932_195 122_626_272 121_910_513

Heap

Real-Time Amortized Mutable
0 198.72 KiB 198.65 KiB 433 KiB
1 -528 B -528 B -1.69 KiB
2 -528 B -528 B -1.69 KiB
3 -8 B -8 B -1.18 KiB
4 -528 B -528 B -1.69 KiB
5 -8 B -8 B -1.18 KiB
6 -528 B -528 B -1.69 KiB
7 -528 B -528 B -1.69 KiB
8 -8 B -8 B -1.18 KiB
9 -528 B -528 B -1.69 KiB
10 -8 B -8 B -1.18 KiB
11 -528 B -528 B -1.69 KiB
12 -528 B -528 B -1.69 KiB
13 -8 B -8 B -1.18 KiB
14 -528 B -528 B -1.69 KiB
15 -528 B -528 B -1.69 KiB
16 5.05 KiB -8 B -1.18 KiB
17 5.73 KiB -528 B -1.69 KiB
18 6.24 KiB -8 B -1.18 KiB
19 5.73 KiB -528 B -1.69 KiB
20 5.73 KiB -528 B -1.69 KiB
21 6.24 KiB -8 B -1.18 KiB
22 5.73 KiB -528 B -1.69 KiB
23 6.24 KiB -8 B -1.18 KiB
24 5.73 KiB -528 B -1.69 KiB
25 5.73 KiB -528 B -1.69 KiB
26 6.24 KiB -8 B -1.18 KiB
27 4.97 KiB -528 B -1.69 KiB
28 3.12 KiB -8 B -1.18 KiB
29 2.61 KiB -528 B -1.69 KiB
30 2.61 KiB -528 B -1.69 KiB
31 3.12 KiB -8 B -1.18 KiB
32 2.61 KiB -528 B -1.69 KiB
33 3.12 KiB -8 B -1.18 KiB
34 2.61 KiB -528 B -1.69 KiB
35 2.61 KiB -528 B -1.69 KiB
36 3.12 KiB -8 B -1.18 KiB
37 2.61 KiB -528 B -1.69 KiB
38 2.61 KiB -528 B -1.69 KiB
39 3.11 KiB -8 B -1.18 KiB
40 2.61 KiB -528 B -1.69 KiB
41 3.12 KiB -8 B -1.18 KiB
42 2.61 KiB -528 B -1.69 KiB
43 2.61 KiB -528 B -1.69 KiB
44 3.12 KiB -8 B -1.18 KiB
45 2.61 KiB -528 B -1.69 KiB
46 3.12 KiB -8 B -1.18 KiB
47 2.61 KiB -528 B -1.69 KiB
48 2.61 KiB -528 B -1.69 KiB
49 3.12 KiB -8 B -1.18 KiB
50 2.16 KiB -528 B -1.69 KiB
51 -8 B -8 B -1.18 KiB
52 -528 B -528 B -1.69 KiB
53 -29.99 KiB -528 B -1.69 KiB
54 -808 B -8 B -1.18 KiB
55 -1.3 KiB -528 B -1.69 KiB
56 -1.3 KiB -528 B -1.69 KiB
57 -808 B -8 B -1.18 KiB
58 -1.3 KiB -528 B -1.69 KiB
59 -808 B -8 B -1.18 KiB
60 -1.3 KiB -528 B -1.69 KiB
61 -1.3 KiB -528 B -1.69 KiB
62 -108.74 KiB -8 B -1.18 KiB
63 -528 B -528 B -1.69 KiB
64 -8 B -8 B -1.18 KiB
65 -528 B -528 B -1.69 KiB
66 -528 B -528 B -1.69 KiB
67 -8 B -8 B -1.18 KiB
68 -528 B -528 B -1.69 KiB
69 -8 B -8 B -1.18 KiB
70 -528 B -528 B -1.69 KiB
71 -528 B -528 B -1.69 KiB
72 -8 B -8 B -1.18 KiB
73 -528 B -528 B -1.69 KiB
74 -8 B -8 B -1.18 KiB
75 -528 B -528 B -1.69 KiB
76 -528 B -528 B -1.69 KiB
77 1.3 KiB -8 B -1.18 KiB
78 5.73 KiB -528 B -1.69 KiB
79 5.73 KiB -528 B -1.69 KiB
80 6.24 KiB -8 B -1.18 KiB
81 5.73 KiB -528 B -1.69 KiB
82 6.24 KiB -8 B -1.18 KiB
83 5.73 KiB -528 B -1.69 KiB
84 5.73 KiB -528 B -1.69 KiB
85 4.7 KiB -8 B -1.18 KiB
86 2.61 KiB -528 B -1.69 KiB
87 3.12 KiB -8 B -1.18 KiB
88 2.61 KiB -528 B -1.69 KiB
89 2.61 KiB -528 B -1.69 KiB
90 3.12 KiB -8 B -1.18 KiB
91 2.61 KiB -528 B -1.69 KiB
92 3.12 KiB -8 B -1.18 KiB
93 2.6 KiB -528 B -1.69 KiB
94 2.61 KiB -528 B -1.69 KiB
95 3.12 KiB -8 B -1.18 KiB
96 2.61 KiB -528 B -1.69 KiB
97 3.12 KiB -8 B -1.18 KiB
98 2.61 KiB -528 B -1.69 KiB
99 2.61 KiB -528 B -1.69 KiB
100 2.83 KiB -8 B -1.18 KiB
101 -528 B -528 B -1.69 KiB
102 -20.15 KiB -528 B -1.69 KiB
103 -808 B -8 B -1.18 KiB
104 -1.3 KiB -528 B -1.69 KiB
105 -808 B -8 B -1.18 KiB
106 -1.3 KiB -528 B -1.69 KiB
107 -1.3 KiB -528 B -1.69 KiB
108 -72.8 KiB -8 B -1.18 KiB
109 -528 B -528 B -1.69 KiB

Garbage Collection

Real-Time Amortized Mutable
0 2.68 MiB 393.67 KiB 198.34 KiB
1 10.73 KiB 159.96 KiB 3.12 KiB
2 10.73 KiB 3.7 KiB 3.12 KiB
3 10.73 KiB 3.7 KiB 3.12 KiB
4 10.73 KiB 3.7 KiB 3.12 KiB
5 10.73 KiB 3.7 KiB 3.12 KiB
6 10.73 KiB 3.7 KiB 3.12 KiB
7 10.73 KiB 3.7 KiB 3.12 KiB
8 10.73 KiB 3.7 KiB 3.12 KiB
9 10.73 KiB 3.7 KiB 3.12 KiB
10 10.73 KiB 3.7 KiB 3.12 KiB
11 10.73 KiB 3.7 KiB 3.12 KiB
12 10.73 KiB 3.7 KiB 3.12 KiB
13 10.73 KiB 3.7 KiB 3.12 KiB
14 10.73 KiB 3.7 KiB 3.12 KiB
15 10.73 KiB 3.7 KiB 3.12 KiB
16 29.96 KiB 3.7 KiB 3.12 KiB
17 35.15 KiB 3.7 KiB 3.12 KiB
18 35.15 KiB 3.7 KiB 3.12 KiB
19 35.15 KiB 3.7 KiB 3.12 KiB
20 35.15 KiB 3.7 KiB 3.12 KiB
21 35.15 KiB 3.7 KiB 3.12 KiB
22 35.15 KiB 3.7 KiB 3.12 KiB
23 35.15 KiB 3.7 KiB 3.12 KiB
24 35.15 KiB 3.7 KiB 3.12 KiB
25 35.15 KiB 3.7 KiB 3.12 KiB
26 35.15 KiB 3.7 KiB 3.12 KiB
27 32.85 KiB 3.7 KiB 3.12 KiB
28 25.77 KiB 3.7 KiB 3.12 KiB
29 25.77 KiB 3.7 KiB 3.12 KiB
30 25.77 KiB 3.7 KiB 3.12 KiB
31 25.77 KiB 3.7 KiB 3.12 KiB
32 25.77 KiB 3.7 KiB 3.12 KiB
33 25.77 KiB 3.7 KiB 3.12 KiB
34 25.77 KiB 3.7 KiB 3.12 KiB
35 25.77 KiB 3.7 KiB 3.12 KiB
36 25.77 KiB 3.7 KiB 3.12 KiB
37 25.77 KiB 3.7 KiB 3.12 KiB
38 25.77 KiB 3.7 KiB 3.12 KiB
39 35.23 KiB 3.7 KiB 3.12 KiB
40 39.45 KiB 3.7 KiB 3.12 KiB
41 39.45 KiB 3.7 KiB 3.12 KiB
42 39.45 KiB 3.7 KiB 3.12 KiB
43 39.45 KiB 3.7 KiB 3.12 KiB
44 39.45 KiB 3.7 KiB 3.12 KiB
45 39.45 KiB 3.7 KiB 3.12 KiB
46 39.45 KiB 3.7 KiB 3.12 KiB
47 39.45 KiB 3.7 KiB 3.12 KiB
48 39.45 KiB 3.7 KiB 3.12 KiB
49 39.45 KiB 3.7 KiB 3.12 KiB
50 39.7 KiB 3.7 KiB 3.12 KiB
51 41.4 KiB 3.7 KiB 3.12 KiB
52 41.4 KiB 3.7 KiB 3.12 KiB
53 64.16 KiB 3.7 KiB 3.12 KiB
54 33.98 KiB 3.7 KiB 3.12 KiB
55 33.98 KiB 3.7 KiB 3.12 KiB
56 33.98 KiB 3.7 KiB 3.12 KiB
57 33.98 KiB 3.7 KiB 3.12 KiB
58 33.98 KiB 3.7 KiB 3.12 KiB
59 33.98 KiB 3.7 KiB 3.12 KiB
60 33.98 KiB 3.7 KiB 3.12 KiB
61 33.98 KiB 3.7 KiB 3.12 KiB
62 128.82 KiB 3.7 KiB 3.12 KiB
63 10.73 KiB 3.7 KiB 3.12 KiB
64 10.73 KiB 3.7 KiB 3.12 KiB
65 10.73 KiB 3.7 KiB 3.12 KiB
66 10.73 KiB 3.7 KiB 3.12 KiB
67 10.73 KiB 3.7 KiB 3.12 KiB
68 10.73 KiB 3.7 KiB 3.12 KiB
69 10.73 KiB 3.7 KiB 3.12 KiB
70 10.73 KiB 3.7 KiB 3.12 KiB
71 10.73 KiB 3.7 KiB 3.12 KiB
72 10.73 KiB 3.7 KiB 3.12 KiB
73 10.73 KiB 3.7 KiB 3.12 KiB
74 10.73 KiB 3.7 KiB 3.12 KiB
75 10.73 KiB 3.7 KiB 3.12 KiB
76 10.73 KiB 3.7 KiB 3.12 KiB
77 15.32 KiB 3.7 KiB 3.12 KiB
78 35.15 KiB 3.7 KiB 3.12 KiB
79 35.15 KiB 3.7 KiB 3.12 KiB
80 35.15 KiB 3.7 KiB 3.12 KiB
81 35.15 KiB 3.7 KiB 3.12 KiB
82 35.15 KiB 3.7 KiB 3.12 KiB
83 35.15 KiB 3.7 KiB 3.12 KiB
84 35.15 KiB 3.7 KiB 3.12 KiB
85 30.51 KiB 3.7 KiB 3.12 KiB
86 25.77 KiB 3.7 KiB 3.12 KiB
87 25.77 KiB 3.7 KiB 3.12 KiB
88 25.77 KiB 3.7 KiB 3.12 KiB
89 25.77 KiB 3.7 KiB 3.12 KiB
90 25.77 KiB 3.7 KiB 3.12 KiB
91 25.77 KiB 3.7 KiB 3.12 KiB
92 25.77 KiB 3.7 KiB 3.12 KiB
93 36.6 KiB 3.7 KiB 3.12 KiB
94 39.45 KiB 3.7 KiB 3.12 KiB
95 39.45 KiB 3.7 KiB 3.12 KiB
96 39.45 KiB 3.7 KiB 3.12 KiB
97 39.45 KiB 3.7 KiB 3.12 KiB
98 39.45 KiB 3.7 KiB 3.12 KiB
99 39.45 KiB 3.7 KiB 3.12 KiB
100 39.61 KiB 3.7 KiB 3.12 KiB
101 41.4 KiB 81.83 KiB 3.12 KiB
102 56.61 KiB 3.7 KiB 3.12 KiB
103 33.98 KiB 3.7 KiB 3.12 KiB
104 33.98 KiB 3.7 KiB 3.12 KiB
105 33.98 KiB 3.7 KiB 3.12 KiB
106 33.98 KiB 3.7 KiB 3.12 KiB
107 33.98 KiB 3.7 KiB 3.12 KiB
108 97.37 KiB 3.7 KiB 3.12 KiB
109 132.61 KiB 125.59 KiB 124.84 KiB

No previous results found "/home/runner/work/new-motoko-base/new-motoko-base/.bench/ZLongUniform.bench.json"
2025-03-26 12:25:20.200187904 UTC: [Canister asrmz-lmaaa-aaaaa-qaaeq-cai] Real-Time {avg = 2_163; max = ?6_658; mean = 1_078; total = 1_081_697}
2025-03-26 12:25:20.444925128 UTC: [Canister asrmz-lmaaa-aaaaa-qaaeq-cai] Amortized {avg = 366; max = ?21_287; mean = 185; total = 183_358}
2025-03-26 12:25:20.709968288 UTC: [Canister asrmz-lmaaa-aaaaa-qaaeq-cai] Mutable {avg = 442; max = ?592; mean = 466; total = 221_271}

Compare queue implementations

Start with empty, then perform 100 random steps of pushFront/popBack (each repeated 5 times) with push twice as likely. Spikes in real-time operations are due to rebalancing

Instructions

Real-Time Amortized Mutable
push 1 17_005 17_117 18_309
pop 2 15_642 14_882 13_988
push 3 11_956 12_068 13_260
push 4 23_199 13_882 15_377
push 5 15_451 12_324 13_819
push 6 13_108 11_810 13_305
push 7 34_770 14_621 16_116
pop 8 16_006 14_586 11_367
push 9 18_543 11_773 13_268
push 10 30_464 11_910 13_405
push 11 14_356 11_229 12_724
pop 12 15_115 11_392 11_937
push 13 27_266 15_479 16_974
push 14 31_862 11_229 12_724
push 15 15_019 11_892 13_387
push 16 12_620 11_322 12_817
push 17 13_097 11_799 13_294
pop 18 20_632 19_202 11_937
push 19 25_248 11_229 12_724
push 20 34_278 11_866 13_361
push 21 20_225 11_866 13_361
push 22 12_501 11_203 12_698
push 23 13_164 11_866 13_361
pop 24 14_545 10_822 11_367
push 25 13_071 11_773 13_268
push 26 18_415 17_117 18_612
push 27 30_991 11_203 12_698
pop 28 26_627 10_729 11_274
push 29 34_241 11_978 13_473
push 30 32_570 11_203 12_698
push 31 12_501 11_203 12_698
push 32 12_594 11_296 12_791
push 33 13_183 11_885 13_380
push 34 12_501 11_203 12_698
push 35 12_501 11_203 12_698
pop 36 15_227 11_504 12_049
push 37 12_501 11_203 12_698
pop 38 23_531 10_729 11_274
pop 39 32_110 27_117 12_049
push 40 23_684 11_203 12_698
pop 41 32_721 10_729 11_274
push 42 34_278 11_978 13_473
pop 43 33_883 10_729 11_274
push 44 14_995 11_203 12_698
pop 45 15_227 11_504 12_049
push 46 12_501 11_203 12_698
push 47 12_501 11_203 12_698
pop 48 14_545 10_822 11_367
pop 49 27_775 11_411 11_956
push 50 27_551 11_203 12_698
pop 51 27_142 10_729 11_274
push 52 40_463 18_781 20_276
push 53 33_958 11_203 12_698
push 54 26_822 11_203 12_698
push 55 13_276 11_978 13_473
push 56 12_501 11_203 12_698
pop 57 14_452 10_729 11_274
push 58 13_276 11_978 13_473
pop 59 14_452 10_729 11_274
pop 60 14_452 10_729 11_274
pop 61 31_493 11_504 12_049
push 62 26_987 11_203 12_698
push 63 24_489 11_203 12_698
push 64 32_478 11_296 12_791
push 65 34_416 11_885 13_443
push 66 34_311 11_203 12_698
push 67 14_145 11_203 12_698
pop 68 15_290 32_485 12_049
push 69 12_501 11_203 12_698
pop 70 14_515 10_729 11_274
push 71 13_276 11_978 13_473
pop 72 14_515 10_729 11_274
pop 73 14_515 10_729 11_274
pop 74 24_369 11_504 12_049
pop 75 32_325 10_729 11_274
pop 76 27_293 10_729 11_274
push 77 26_365 12_104 13_599
push 78 33_466 11_203 12_698
push 79 33_783 11_203 12_698
pop 80 33_913 10_822 11_367
push 81 15_677 11_885 13_380
push 82 12_627 11_203 12_698
pop 83 14_515 10_855 11_274
push 84 13_339 11_978 13_536
pop 85 14_515 10_792 11_274
push 86 12_564 11_203 12_761
pop 87 15_290 11_567 12_112
push 88 28_263 11_203 12_698
pop 89 31_079 10_729 11_274
push 90 25_201 11_978 13_473
push 91 28_269 11_203 12_698
push 92 33_466 11_203 12_698
pop 93 37_789 11_504 12_049
pop 94 33_148 10_729 11_274
pop 95 14_452 30_677 11_274
push 96 12_594 11_296 12_791
push 97 13_183 11_885 13_380
push 98 12_501 11_203 12_698
push 99 12_501 11_203 12_698
push 100 3_024_803 2_999_760 2_919_893

Heap

Real-Time Amortized Mutable
push 1 528 B 456 B 556 B
pop 2 148 B 220 B 120 B
push 3 460 B 388 B 488 B
push 4 684 B 400 B 520 B
push 5 116 B 400 B 520 B
push 6 368 B 368 B 488 B
push 7 892 B 416 B 536 B
pop 8 -284 B 192 B 72 B
push 9 704 B 392 B 512 B
push 10 620 B 392 B 512 B
push 11 -188 B 352 B 472 B
pop 12 224 B 224 B 104 B
push 13 904 B 432 B 552 B
push 14 548 B 352 B 472 B
push 15 -276 B 392 B 512 B
push 16 352 B 352 B 472 B
push 17 392 B 392 B 512 B
pop 18 544 B 232 B 112 B
push 19 756 B 352 B 472 B
push 20 592 B 392 B 512 B
push 21 -524 B 392 B 512 B
push 22 352 B 352 B 472 B
push 23 392 B 392 B 512 B
pop 24 192 B 192 B 72 B
push 25 384 B 384 B 504 B
push 26 464 B 464 B 584 B
push 27 1.12 KiB 352 B 472 B
pop 28 500 B 192 B 72 B
push 29 744 B 424 B 544 B
push 30 -1.04 KiB 352 B 472 B
push 31 352 B 352 B 472 B
push 32 352 B 352 B 472 B
push 33 424 B 424 B 544 B
push 34 352 B 352 B 472 B
push 35 352 B 352 B 472 B
pop 36 264 B 264 B 144 B
push 37 352 B 352 B 472 B
pop 38 632 B 192 B 72 B
pop 39 856 B 264 B 144 B
push 40 672 B 352 B 472 B
pop 41 500 B 192 B 72 B
push 42 656 B 424 B 544 B
pop 43 -40 B 192 B 72 B
push 44 -1.28 KiB 352 B 472 B
pop 45 264 B 264 B 144 B
push 46 352 B 352 B 472 B
push 47 352 B 352 B 472 B
pop 48 192 B 192 B 72 B
pop 49 824 B 256 B 136 B
push 50 864 B 352 B 472 B
pop 51 512 B 192 B 72 B
push 52 804 B 496 B 616 B
push 53 472 B 352 B 472 B
push 54 -1.44 KiB 352 B 472 B
push 55 424 B 424 B 544 B
push 56 352 B 352 B 472 B
pop 57 192 B 192 B 72 B
push 58 424 B 424 B 544 B
pop 59 192 B 192 B 72 B
pop 60 192 B 192 B 72 B
pop 61 960 B 264 B 144 B
push 62 848 B 352 B 472 B
push 63 672 B 352 B 472 B
push 64 660 B 352 B 472 B
push 65 608 B 424 B 544 B
push 66 328 B 352 B 472 B
push 67 -1.59 KiB 352 B 472 B
pop 68 264 B 264 B 144 B
push 69 352 B 352 B 472 B
pop 70 192 B 192 B 72 B
push 71 424 B 424 B 544 B
pop 72 192 B 192 B 72 B
pop 73 192 B 192 B 72 B
pop 74 704 B 264 B 144 B
pop 75 832 B 192 B 72 B
pop 76 544 B 192 B 72 B
push 77 732 B 424 B 544 B
push 78 672 B 352 B 472 B
push 79 504 B 352 B 472 B
pop 80 -40 B 192 B 72 B
push 81 -1.52 KiB 424 B 544 B
push 82 352 B 352 B 472 B
pop 83 192 B 192 B 72 B
push 84 424 B 424 B 544 B
pop 85 192 B 192 B 72 B
push 86 352 B 352 B 472 B
pop 87 264 B 264 B 144 B
push 88 1.02 KiB 352 B 472 B
pop 89 768 B 192 B 72 B
push 90 744 B 424 B 544 B
push 91 660 B 352 B 472 B
push 92 672 B 352 B 472 B
pop 93 352 B 264 B 144 B
pop 94 -2.07 KiB 192 B 72 B
pop 95 192 B 192 B 72 B
push 96 352 B 352 B 472 B
push 97 424 B 424 B 544 B
push 98 352 B 352 B 472 B
push 99 352 B 352 B 472 B
push 100 416 B 416 B 536 B

Garbage Collection

Real-Time Amortized Mutable
push 1 800 B 752 B 636 B
pop 2 1.38 KiB 1.03 KiB 780 B
push 3 712 B 664 B 548 B
push 4 2.15 KiB 680 B 580 B
push 5 1.6 KiB 640 B 540 B
push 6 1008 B 648 B 548 B
push 7 3.58 KiB 696 B 596 B
pop 8 2.3 KiB 1.25 KiB 740 B
push 9 1.65 KiB 640 B 540 B
push 10 3.34 KiB 640 B 540 B
push 11 1.85 KiB 640 B 540 B
pop 12 1.57 KiB 888 B 748 B
push 13 2.38 KiB 712 B 612 B
push 14 3.63 KiB 640 B 540 B
push 15 1.98 KiB 640 B 540 B
push 16 1000 B 640 B 540 B
push 17 1000 B 640 B 540 B
pop 18 2.24 KiB 1.69 KiB 740 B
push 19 2.7 KiB 640 B 540 B
push 20 3.82 KiB 640 B 540 B
push 21 2.93 KiB 640 B 540 B
push 22 1000 B 640 B 540 B
push 23 1000 B 640 B 540 B
pop 24 1.56 KiB 880 B 740 B
push 25 1008 B 648 B 548 B
push 26 1.08 KiB 744 B 644 B
push 27 3.43 KiB 640 B 540 B
pop 28 3.16 KiB 880 B 740 B
push 29 3.77 KiB 640 B 540 B
push 30 5.18 KiB 640 B 540 B
push 31 1000 B 640 B 540 B
push 32 1000 B 640 B 540 B
push 33 1000 B 640 B 540 B
push 34 1000 B 640 B 540 B
push 35 1000 B 640 B 540 B
pop 36 1.56 KiB 880 B 740 B
push 37 1000 B 640 B 540 B
pop 38 2.73 KiB 880 B 740 B
pop 39 3.86 KiB 2.54 KiB 740 B
push 40 2.38 KiB 640 B 540 B
pop 41 3.98 KiB 880 B 740 B
push 42 3.8 KiB 640 B 540 B
pop 43 4.43 KiB 880 B 740 B
push 44 3.01 KiB 640 B 540 B
pop 45 1.56 KiB 880 B 740 B
push 46 1000 B 640 B 540 B
push 47 1000 B 640 B 540 B
pop 48 1.56 KiB 880 B 740 B
pop 49 3.22 KiB 888 B 748 B
push 50 2.95 KiB 640 B 540 B
pop 51 3.07 KiB 880 B 740 B
push 52 3.86 KiB 776 B 676 B
push 53 3.86 KiB 640 B 540 B
push 54 4.8 KiB 640 B 540 B
push 55 1000 B 640 B 540 B
push 56 1000 B 640 B 540 B
pop 57 1.56 KiB 880 B 740 B
push 58 1000 B 640 B 540 B
pop 59 1.56 KiB 880 B 740 B
pop 60 1.56 KiB 880 B 740 B
pop 61 3.7 KiB 880 B 740 B
push 62 2.9 KiB 640 B 540 B
push 63 2.38 KiB 640 B 540 B
push 64 3.66 KiB 640 B 540 B
push 65 3.82 KiB 640 B 540 B
push 66 3.96 KiB 640 B 540 B
push 67 3.22 KiB 640 B 540 B
pop 68 1.56 KiB 3.11 KiB 740 B
push 69 1000 B 640 B 540 B
pop 70 1.56 KiB 880 B 740 B
push 71 1000 B 640 B 540 B
pop 72 1.56 KiB 880 B 740 B
pop 73 1.56 KiB 880 B 740 B
pop 74 2.73 KiB 880 B 740 B
pop 75 4 KiB 880 B 740 B
pop 76 3.16 KiB 880 B 740 B
push 77 2.61 KiB 640 B 540 B
push 78 3.77 KiB 640 B 540 B
push 79 3.84 KiB 640 B 540 B
pop 80 4.43 KiB 880 B 740 B
push 81 3.32 KiB 640 B 540 B
push 82 1000 B 640 B 540 B
pop 83 1.56 KiB 880 B 740 B
push 84 1000 B 640 B 540 B
pop 85 1.56 KiB 880 B 740 B
push 86 1000 B 640 B 540 B
pop 87 1.56 KiB 880 B 740 B
push 88 3.05 KiB 640 B 540 B
pop 89 3.82 KiB 880 B 740 B
push 90 2.38 KiB 640 B 540 B
push 91 3.04 KiB 640 B 540 B
push 92 3.77 KiB 640 B 540 B
pop 93 4.56 KiB 880 B 740 B
pop 94 6.43 KiB 880 B 740 B
pop 95 1.56 KiB 3 KiB 740 B
push 96 1000 B 640 B 540 B
push 97 1000 B 640 B 540 B
push 98 1000 B 640 B 540 B
push 99 1000 B 640 B 540 B
push 100 6.04 KiB 5.59 KiB 5.41 KiB

Note: Renamed benchmarks cannot be compared. Refer to the current baseline for manual comparison.

@Kamirus Kamirus mentioned this pull request Mar 25, 2025
Open
@Kamirus
Copy link
Contributor Author

Kamirus commented Mar 26, 2025 •
edited
Loading

Implementations and their names in benchmarks:

  • Amortized = 'pure/Queue' currently in new-base
  • Real-Time = a potential replacement for the 'pure/Queue'
  • Mutable = 'Queue' in new-base

Summary

  • 'Amortized' can cause huge spikes in instructions, e.g. 76_248 instructions per 2 pop operations
    • this is work of splitting 500 element list into two halves
    • with 10_000 elements: 1_445_509 instructions for a single pop
  • A single 'Real-time' pop/push operation costs max ~7_000 (in recorded benchmarks, but it should be true for any size as there is O(1) bound)
  • 'Real-time' pop operations are on average 4x more expensive
  • 'Real-time' push operations are on average 10x more expensive
    • which is expected as the rebalancing steps are performed on each operation and 'Amortized' push operations are most of the time as cheap as taking a head and tail from pure/List
  • 'Amortized' is on average slightly cheaper than 'Mutable' (see statistics below)

Statistics

Real-Time {avg = 2_163; max =     6_658; mean = 1_078; total =  1_081_697}
Real-Time {avg = 2_248; max =     6_880; mean = 1_045; total = 34_746_012}
Amortized {avg =   366; max =    21_287; mean =   185; total =    183_358}
Amortized {avg =   358; max = 1_445_509; mean =   188; total =  5_536_647}
Mutable   {avg =   442; max =       592; mean =   466; total =    221_271}
Mutable   {avg =   449; max =       595; mean =   469; total =  6_946_759}

with queue sizes both ~10_000 and ~250

@Kamirus Kamirus self-assigned this Apr 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Kamirus Kamirus

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Kamirus