Skip to content

Conversation

@tigrannajaryan
Copy link
Collaborator

Protobuf generation does not need to happen during a regular build. This is only necessary when proto changes and that is very rare and can be run manually when needed.

Re-generation during "all" target currently causes the generated files to be updated unpredictably if the locally installed version of protoc is different. In the future we will want to fix this by using a fixed version of protoc tools.

Protobuf generation does not need to happen during a regular build.
This is only necessary when proto changes and that is very rare and
can be run manually when needed.

Re-generation during "all" target currently causes the generated
files to be updated unpredictably if the locally installed version
of protoc is different. In the future we will want to fix this by
using a fixed version of protoc tools.
@github-actions
Copy link

Benchmark Result

Benchmark diff with base branch
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │     sec/op     │    sec/op     vs base               │
SerializeNative/STEF/serialize-4                     13.24m ± 15%   10.66m ±  7%  -19.51% (p=0.004 n=6)
SerializeNative/STEFU/serialize-4                    35.42m ±  1%   35.38m ±  4%        ~ (p=1.000 n=6)
DeserializeNative/STEF/deser-4                       2.555m ±  1%   2.562m ±  3%        ~ (p=0.818 n=6)
DeserializeNative/STEFU/deser-4                      7.315m ±  0%   7.329m ±  1%        ~ (p=0.394 n=6)
SerializeFromPdata/STEF/serialize-4                  139.7m ±  4%   138.0m ±  2%        ~ (p=0.699 n=6)
SerializeFromPdata/STEFU/serialize-4                 36.00m ±  2%   34.19m ±  3%   -5.01% (p=0.002 n=6)
DeserializeToPdata/STEF/deserialize-4                46.34m ±  1%   45.13m ±  1%   -2.62% (p=0.002 n=6)
DeserializeToPdata/STEFU/deserialize-4               63.61m ±  2%   62.73m ±  3%        ~ (p=0.180 n=6)
STEFReaderRead-4                                     2.605m ±  2%   2.586m ±  1%   -0.73% (p=0.026 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4        3.575 ± 20%    3.412 ± 22%        ~ (p=0.093 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     72.53m ± 16%   75.59m ± 11%        ~ (p=0.310 n=6)
ReadSTEF-4                                           2.736m ±  4%   2.775m ±  1%        ~ (p=0.065 n=6)
ReadSTEFZ-4                                          3.470m ±  1%   3.735m ±  6%   +7.65% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 8.330m ±  5%   8.506m ±  3%   +2.11% (p=0.026 n=6)
geomean                                              22.17m         21.83m         -1.57%

                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │   sec/point    │  sec/point    vs base               │
SerializeNative/STEF/serialize-4                     198.0n ± 15%   159.4n ±  7%  -19.52% (p=0.004 n=6)
SerializeNative/STEFU/serialize-4                    529.8n ±  1%   529.2n ±  4%        ~ (p=1.000 n=6)
DeserializeNative/STEF/deser-4                       38.22n ±  1%   38.32n ±  3%        ~ (p=0.818 n=6)
DeserializeNative/STEFU/deser-4                      109.5n ±  0%   109.7n ±  1%        ~ (p=0.413 n=6)
SerializeFromPdata/STEF/serialize-4                  2.089µ ±  4%   2.065µ ±  2%        ~ (p=0.734 n=6)
SerializeFromPdata/STEFU/serialize-4                 538.4n ±  2%   511.4n ±  3%   -5.01% (p=0.002 n=6)
DeserializeToPdata/STEF/deserialize-4                693.2n ±  1%   675.2n ±  1%   -2.60% (p=0.002 n=6)
DeserializeToPdata/STEFU/deserialize-4               951.5n ±  2%   938.4n ±  3%        ~ (p=0.180 n=6)
STEFReaderRead-4                                     38.96n ±  2%   38.67n ±  1%   -0.72% (p=0.026 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4       4.544µ ± 20%   4.337µ ± 22%        ~ (p=0.093 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     92.19n ± 16%   96.08n ± 11%        ~ (p=0.329 n=6)
ReadSTEF-4                                           40.95n ±  4%   41.53n ±  1%        ~ (p=0.065 n=6)
ReadSTEFZ-4                                          51.93n ±  1%   55.90n ±  6%   +7.65% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 124.6n ±  5%   127.3n ±  3%   +2.13% (p=0.017 n=6)
geomean                                              233.2n         229.6n         -1.56%

                                                 │ bench-main.txt │            bench-new.txt             │
                                                 │      B/op      │     B/op      vs base                │
SerializeNative/STEF/serialize-4                     3.347Mi ± 0%   3.341Mi ± 0%  -0.17% (p=0.009 n=6)
SerializeNative/STEFU/serialize-4                    7.557Mi ± 0%   7.557Mi ± 0%  -0.00% (p=0.039 n=6)
DeserializeNative/STEF/deser-4                       951.4Ki ± 0%   951.4Ki ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                      1.715Mi ± 0%   1.715Mi ± 0%       ~ (p=1.000 n=6)
SerializeFromPdata/STEF/serialize-4                  76.55Mi ± 0%   76.55Mi ± 0%       ~ (p=0.394 n=6)
SerializeFromPdata/STEFU/serialize-4                 7.557Mi ± 0%   7.557Mi ± 0%       ~ (p=0.797 n=6)
DeserializeToPdata/STEF/deserialize-4                31.99Mi ± 0%   31.99Mi ± 0%       ~ (p=0.069 n=6)
DeserializeToPdata/STEFU/deserialize-4               38.88Mi ± 0%   38.88Mi ± 0%       ~ (p=0.996 n=6)
STEFReaderRead-4                                     953.1Ki ± 0%   953.1Ki ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4       3.384Gi ± 0%   3.384Gi ± 0%       ~ (p=0.818 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     20.30Mi ± 0%   20.30Mi ± 0%       ~ (p=0.892 n=6)
ReadSTEF-4                                           953.2Ki ± 0%   953.2Ki ± 0%       ~ (p=0.905 n=6)
ReadSTEFZ-4                                          10.29Mi ± 0%   10.29Mi ± 0%       ~ (p=0.494 n=6)
ReadSTEFZWriteSTEF-4                                 13.44Mi ± 0%   13.44Mi ± 0%       ~ (p=0.411 n=6)
geomean                                              10.66Mi        10.66Mi       -0.01%
¹ all samples are equal

                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │   allocs/op    │  allocs/op   vs base                │
SerializeNative/STEF/serialize-4                      2.670k ± 0%   2.659k ± 0%  -0.41% (p=0.009 n=6)
SerializeNative/STEFU/serialize-4                      885.0 ± 0%    885.0 ± 0%       ~ (p=0.455 n=6)
DeserializeNative/STEF/deser-4                         463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
DeserializeNative/STEFU/deser-4                        496.0 ± 0%    496.0 ± 0%       ~ (p=1.000 n=6) ¹
SerializeFromPdata/STEF/serialize-4                   134.7k ± 0%   134.7k ± 0%       ~ (p=0.405 n=6)
SerializeFromPdata/STEFU/serialize-4                   886.0 ± 0%    886.0 ± 0%       ~ (p=1.000 n=6) ¹
DeserializeToPdata/STEF/deserialize-4                 756.2k ± 0%   756.2k ± 0%       ~ (p=1.000 n=6) ¹
DeserializeToPdata/STEFU/deserialize-4                944.9k ± 0%   944.9k ± 0%       ~ (p=1.000 n=6) ¹
STEFReaderRead-4                                       463.0 ± 0%    463.0 ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4        13.15M ± 0%   13.15M ± 0%       ~ (p=0.251 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      1.956k ± 0%   1.956k ± 0%       ~ (p=0.545 n=6)
ReadSTEF-4                                             464.0 ± 0%    464.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZ-4                                            502.0 ± 0%    502.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZWriteSTEF-4                                  1.231k ± 0%   1.231k ± 0%       ~ (p=1.000 n=6) ¹
geomean                                               6.308k        6.306k       -0.03%
¹ all samples are equal
Benchmark result
benchstat bench-new.txt
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-new.txt │
                                                 │    sec/op     │
SerializeNative/STEF/serialize-4                    10.66m ±  7%
SerializeNative/STEFU/serialize-4                   35.38m ±  4%
DeserializeNative/STEF/deser-4                      2.562m ±  3%
DeserializeNative/STEFU/deser-4                     7.329m ±  1%
SerializeFromPdata/STEF/serialize-4                 138.0m ±  2%
SerializeFromPdata/STEFU/serialize-4                34.19m ±  3%
DeserializeToPdata/STEF/deserialize-4               45.13m ±  1%
DeserializeToPdata/STEFU/deserialize-4              62.73m ±  3%
STEFReaderRead-4                                    2.586m ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4       3.412 ± 22%
STEFDeserializeMultipart/astronomy-otelmetrics-4    75.59m ± 11%
ReadSTEF-4                                          2.775m ±  1%
ReadSTEFZ-4                                         3.735m ±  6%
ReadSTEFZWriteSTEF-4                                8.506m ±  3%
geomean                                             21.83m

                                                 │ bench-new.txt │
                                                 │   sec/point   │
SerializeNative/STEF/serialize-4                    159.4n ±  7%
SerializeNative/STEFU/serialize-4                   529.2n ±  4%
DeserializeNative/STEF/deser-4                      38.32n ±  3%
DeserializeNative/STEFU/deser-4                     109.7n ±  1%
SerializeFromPdata/STEF/serialize-4                 2.065µ ±  2%
SerializeFromPdata/STEFU/serialize-4                511.4n ±  3%
DeserializeToPdata/STEF/deserialize-4               675.2n ±  1%
DeserializeToPdata/STEFU/deserialize-4              938.4n ±  3%
STEFReaderRead-4                                    38.67n ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4      4.337µ ± 22%
STEFDeserializeMultipart/astronomy-otelmetrics-4    96.08n ± 11%
ReadSTEF-4                                          41.53n ±  1%
ReadSTEFZ-4                                         55.90n ±  6%
ReadSTEFZWriteSTEF-4                                127.3n ±  3%
geomean                                             229.6n

                                                 │ bench-new.txt │
                                                 │     B/op      │
SerializeNative/STEF/serialize-4                    3.341Mi ± 0%
SerializeNative/STEFU/serialize-4                   7.557Mi ± 0%
DeserializeNative/STEF/deser-4                      951.4Ki ± 0%
DeserializeNative/STEFU/deser-4                     1.715Mi ± 0%
SerializeFromPdata/STEF/serialize-4                 76.55Mi ± 0%
SerializeFromPdata/STEFU/serialize-4                7.557Mi ± 0%
DeserializeToPdata/STEF/deserialize-4               31.99Mi ± 0%
DeserializeToPdata/STEFU/deserialize-4              38.88Mi ± 0%
STEFReaderRead-4                                    953.1Ki ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4      3.384Gi ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4    20.30Mi ± 0%
ReadSTEF-4                                          953.2Ki ± 0%
ReadSTEFZ-4                                         10.29Mi ± 0%
ReadSTEFZWriteSTEF-4                                13.44Mi ± 0%
geomean                                             10.66Mi

                                                 │ bench-new.txt │
                                                 │   allocs/op   │
SerializeNative/STEF/serialize-4                     2.659k ± 0%
SerializeNative/STEFU/serialize-4                     885.0 ± 0%
DeserializeNative/STEF/deser-4                        463.0 ± 0%
DeserializeNative/STEFU/deser-4                       496.0 ± 0%
SerializeFromPdata/STEF/serialize-4                  134.7k ± 0%
SerializeFromPdata/STEFU/serialize-4                  886.0 ± 0%
DeserializeToPdata/STEF/deserialize-4                756.2k ± 0%
DeserializeToPdata/STEFU/deserialize-4               944.9k ± 0%
STEFReaderRead-4                                      463.0 ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4       13.15M ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4     1.956k ± 0%
ReadSTEF-4                                            464.0 ± 0%
ReadSTEFZ-4                                           502.0 ± 0%
ReadSTEFZWriteSTEF-4                                 1.231k ± 0%
geomean                                              6.306k

@tigrannajaryan tigrannajaryan marked this pull request as ready for review January 15, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant