You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JetStream 3 is a JavaScript and WebAssembly benchmark suite.
4
+
For more information see the index and in-depth pages of the deployed benchmark.
5
+
6
+
A preview of the current main branch is available at [https://webkit-jetstream-preview.netlify.app/](https://webkit-jetstream-preview.netlify.app/).
7
+
8
+
<imgsrc="./resources/screenshot.png">
9
+
10
+
## Open Governance
11
+
12
+
See [Governance.md](Governance.md) for more information.
13
+
14
+
## Getting Started, Setup Instructions
15
+
16
+
- Install Node.js and (optionally) [jsvu](https://github.com/GoogleChromeLabs/jsvu) for conveniently getting recent builds of engine shells.
17
+
-`npm install` the necessary dependencies.
18
+
-`npm run server` for starting a local development server, then browse to http://localhost:8010.
19
+
-`npm run test:shell` for running the benchmark in engine shells, or alternatively running directly, e.g., via `jsc cli.js`.
20
+
21
+
See [package.json](package.json) and [.github/workflows/test.yml](.github/workflows/test.yml) for more details and available commands.
22
+
23
+
### Shell Runner
24
+
25
+
For the shell runner, see the available options by passing `--help` to `cli.js`. (Note that this requires `--` for JavaScriptCore and V8 to separate VM arguments from script arguments.):
26
+
27
+
```
28
+
$ v8 cli.js -- --help
29
+
JetStream Driver Help
30
+
31
+
Options:
32
+
--help Print this help message.
33
+
--iteration-count Set the default iteration count.
34
+
...
35
+
36
+
Available tags:
37
+
all
38
+
...
39
+
40
+
Available tests:
41
+
8bitbench-wasm
42
+
...
43
+
```
44
+
45
+
### Browser Runner
46
+
47
+
The browser version also supports passing parameters as URL query parameters, e.g., pass the `test` parameter (aliases are `tests` or `testList`) with a comma-separated list to run only specific workloads: [https://webkit-jetstream-preview.netlify.app/?test=8bitbench-wasm,web-ssr](https://webkit-jetstream-preview.netlify.app/?test=8bitbench-wasm,web-ssr).
48
+
See [utils/params.js](utils/params.js) and [JetStreamDriver.js](JetStreamDriver.js) for more details.
49
+
50
+
## Technical Details
51
+
52
+
The main file of the benchmark harness is `JetStreamDriver.js`, which lists the individual workloads and their parameters, implements measurement and scoring, etc.
53
+
The individual workloads are in subdirectories.
54
+
55
+
### Preloading and Compression
56
+
57
+
The JetStream driver (both in the browser and shell runners) _preloads_ some large assets and source files.
58
+
That is, it reads those files from disk or fetches them from the network before running the workloads.
59
+
This is in order to exclude network latency and disk I/O from the benchmark measurements and reduce variance. Otherwise, OS scheduling or CPU frequency scaling may affect the measurement.
60
+
61
+
Some workloads also utilize large assets (e.g., ML models, heavy JavaScript bundles in the order of 10s of MBs).
62
+
In order to limit the repository size and network transfers, such large assets are stored as compressed .z files.
63
+
Preloading handles the decompression of these assets (using `DecompressionStream` or a Wasm-based zlib polyfill) upfront so that decompression overhead does not affect the benchmark score.
64
+
65
+
Both preloading and compression can be disabled, e.g., to inspect raw files or because it sometimes helps with debugging (e.g., proper URLs instead of Blobs for resources).
66
+
67
+
- Compression: Run `npm run decompress` to decompress all .z files before running the benchmark.
68
+
- No prefetching for shells: Pass the `--no-prefetch` flag, e.g., `jsc cli.js -- --no-prefetch`.
69
+
- No prefetching in browsers: Append the query parameter `?prefetchResources=false` to the URL.
70
+
71
+
See `JetStreamDriver.js` and `utils/compress.mjs` for more details.
72
+
73
+
### Score Calculation
74
+
75
+
Scores in JetStream are dimensionless floats, where a higher score is better.
76
+
When scores are aggregated (e.g., multiple sub-scores for each workload, or to determine the total score of the full benchmark suite), we use the [geometric mean](https://en.wikipedia.org/wiki/Geometric_mean).
77
+
The geometric mean ensures that a multiplicative improvement of any individual score has the same effect on the aggregated score, regardless of the absolute value of the individual score.
78
+
For example, an improvement by 5% of the sub score of benchmark A has the same effect on the total score as an improvement by 5% of the sub score of benchmark B.
79
+
80
+
See the [in-depth.html](https://webkit-jetstream-preview.netlify.app/in-depth.html) and `JetStreamDriver.js` for more details.
JetStream 3 combines together a variety of JavaScript and Web Assembly benchmarks, covering a variety of
46
+
JetStream 3 combines together a variety of JavaScript and WebAssembly benchmarks, covering a variety of
47
47
advanced workloads and programming techniques, and reports a single score that
48
48
balances them using a geometric mean.
49
49
</p>
@@ -56,9 +56,11 @@ <h2>In-Depth Analysis</h2>
56
56
</p>
57
57
58
58
<p>
59
-
Each benchmark in JetStream 3 computes its own individual score. JetStream 3 weighs each
60
-
benchmark equally, taking the geometric mean over each individual benchmark's score to compute
61
-
the overall JetStream 3 score.
59
+
Each benchmark in JetStream 3 computes its own individual score.
60
+
Scores in JetStream are dimensionless floats, where a higher score is better.
61
+
JetStream weighs each benchmark equally, taking the <ahref="https://en.wikipedia.org/wiki/Geometric_mean">geometric mean</a> over each individual benchmark's score to compute the overall JetStream 3 score.
62
+
The geometric mean ensures that a multiplicative improvement of any individual score has the same effect on the aggregated score, regardless of the absolute value of the individual score.
63
+
For example, an improvement by 5% of the sub score of benchmark A has the same effect on the total score as an improvement by 5% of the sub score of benchmark B.
62
64
</p>
63
65
64
66
<p>
@@ -70,7 +72,7 @@ <h2>In-Depth Analysis</h2>
70
72
</p>
71
73
72
74
<p>
73
-
For most of the JavaScript benchmarks in JetStream 3, individual scores
75
+
For most of the JavaScript and WebAssembly benchmarks in JetStream 3, individual scores
74
76
equally weigh startup performance, worst case performance, and average case
75
77
performance. These three metrics are crucial to running performant JavaScript
76
78
in the browser. Fast startup times lead browsers to loading pages more quickly. Good
@@ -79,11 +81,10 @@ <h2>In-Depth Analysis</h2>
79
81
</p>
80
82
81
83
<p>
82
-
For JetStream 3's Web Assembly benchmarks, individual scores equally weigh startup time and
83
-
total execution time. An important component of JetStream 1 were the asm.js subset of benchmarks. With the release
84
-
of Web Assembly, the importance of asm.js has lessened since many users of asm.js are
85
-
now using Web Assembly. JetStream 3 has converted many of the asm.js benchmarks from
86
-
JetStream 1 into Web Assembly.
84
+
An important component of JetStream 1 were the asm.js subset of benchmarks. With the release
85
+
of WebAssembly, the importance of asm.js has lessened since many users of asm.js are
86
+
now using WebAssembly. JetStream 3 has converted many of the asm.js benchmarks from
87
+
JetStream 1 into WebAssembly.
87
88
</p>
88
89
89
90
<p>
@@ -106,14 +107,13 @@ <h2>In-Depth Analysis</h2>
106
107
<p>
107
108
JetStream 3 includes parts of these benchmark suites that came before it: <ahref="https://webkit.org/perf/sunspider/sunspider.html">SunSpider</a>,
<ahref="https://browserbench.org/ARES-6/">ARES-6</a>, and <ahref="https://v8.github.io/web-tooling-benchmark/">Web Tooling Benchmark</a>.
110
-
JetStream 3 also includes new benchmarks inspired by <ahref="https://krakenbenchmark.mozilla.org">Kraken</a>.
111
-
JetStream 3 also includes a new set of benchmarks that measure the performance of Web Assembly, Web Workers,
110
+
<ahref="https://browserbench.org/ARES-6/">ARES-6</a>, <ahref="https://v8.github.io/web-tooling-benchmark/">Web Tooling Benchmark</a>, and benchmarks inspired by <ahref="https://krakenbenchmark.mozilla.org">Kraken</a>.
111
+
JetStream 3 also includes a new set of benchmarks that measure the performance of WebAssembly, Web Workers,
112
112
Promises, async iteration, unicode regular expressions, and JavaScript parsing.
113
113
</p>
114
114
115
115
<p>
116
-
JetStream 3 runs the same benchmarks as JetStream 3, but updates the benchmark driver to
116
+
JetStream 3 includes several benchmarks from earlier JetStream versions, but updates the benchmark driver to
117
117
improve score stability. This is achieved by pre-fetching network resources prior to running
118
118
the benchmarks. This can reduce perturbations on the measurement of JavaScript execution
119
119
time due to second order effects of pause times induced by network latency.
@@ -168,7 +168,7 @@ <h3>
168
168
<dtid="tsf-wasm">tsf-wasm</dt>
169
169
<dd>
170
170
Runs Filip Pizlo's — of the WebKit team — implementation of a <ahref="http://www.filpizlo.com/tsf/"> Typed Stream Format </a>
171
-
in Web Assembly. The original code is compiled from C to Web Assembly using <ahref="https://emscripten.org">Emscripten</a>.
171
+
in WebAssembly. The original code is compiled from C to WebAssembly using <ahref="https://emscripten.org">Emscripten</a>.
172
172
Source code: <ahref="wasm/TSF">TSF</a>
173
173
</dd>
174
174
@@ -179,7 +179,7 @@ <h3>
179
179
180
180
<dtid="argon2-wasm">argon2-wasm</dt>
181
181
<dd>
182
-
Tests <ahref="https://github.com/P-H-C/phc-winner-argon2">Argon2</a>, a password-hashing function (the winner of Password Hashing Competition), in Web Assembly. This is test is based on <ahref="https://github.com/antelle/argon2-browser">argon2-browser</a> library. Source code: <ahref="wasm/">ARGON2</a>.
182
+
Tests <ahref="https://github.com/P-H-C/phc-winner-argon2">Argon2</a>, a password-hashing function (the winner of Password Hashing Competition), in WebAssembly. This is test is based on <ahref="https://github.com/antelle/argon2-browser">argon2-browser</a> library. Source code: <ahref="wasm/">ARGON2</a>.
183
183
</dd>
184
184
185
185
<dtid="tagcloud-SP">tagcloud-SP</dt>
@@ -246,8 +246,8 @@ <h3>
246
246
<dtid="richards-wasm">richards-wasm</dt>
247
247
<dd>
248
248
Martin Richard's <ahref="http://www.cl.cam.ac.uk/~mr10/Bench.html">system language
249
-
benchmark</a> compiled to a hybrid of Web Assembly and JavaScript. It stresses how quickly
250
-
JavaScript can call into Web Assembly code.
249
+
benchmark</a> compiled to a hybrid of WebAssembly and JavaScript. It stresses how quickly
Copy file name to clipboardExpand all lines: index.html
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -97,7 +97,7 @@ <h1 class="logo">
97
97
</h1>
98
98
99
99
<main>
100
-
<pclass="summary">JetStream 3 is a JavaScript and WebAssembly benchmark suite focused on the most advanced web applications. It rewards browsers that start up quickly, execute code quickly, and run smoothly. For more information, read the <ahref="resources/in-depth.html">in-depth analysis</a>. Bigger scores are better.</p>
100
+
<pclass="summary">JetStream 3 is a JavaScript and WebAssembly benchmark suite focused on the most advanced web applications. It rewards browsers that start up quickly, execute code quickly, and run smoothly. For more information, read the <ahref="/in-depth.html">in-depth analysis</a>. Bigger scores are better.</p>
0 commit comments