Skip to content

Commit f07a751

Browse files
TomWambsgansTomWambsgans
committed
Improve ISA and compiler.
Squashed commit of the following: commit 197cb9c Author: Tom Wambsgans <[email protected]> Date: Wed Feb 25 02:06:54 2026 +0400 bench commit 5dc4ed0 Author: Tom Wambsgans <[email protected]> Date: Wed Feb 25 01:41:34 2026 +0400 w commit f25f9ad Author: Tom Wambsgans <[email protected]> Date: Wed Feb 25 00:59:35 2026 +0400 w commit 15ad13d Author: Tom Wambsgans <[email protected]> Date: Tue Feb 24 23:38:22 2026 +0400 some cleaning commit e27ed1f Author: Tom Wambsgans <[email protected]> Date: Tue Feb 24 13:36:32 2026 +0400 wip commit d6ce63e Author: Tom Wambsgans <[email protected]> Date: Tue Feb 24 12:52:04 2026 +0400 improve compilation.rs commit 13d7910 Author: Tom Wambsgans <[email protected]> Date: Tue Feb 24 12:15:52 2026 +0400 compilation.rs commit 637e7d9 Author: Tom Wambsgans <[email protected]> Date: Tue Feb 24 12:09:47 2026 +0400 simpler commit 0fd87f6 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:46:04 2026 +0400 w commit eeb3d1a Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:44:42 2026 +0400 faster slice_hash_chunks_with_iv commit aa4752e Merge: 1e5efee edaad0c Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:41:58 2026 +0400 Merge remote-tracking branch 'origin/improve_isa' into improve_isa commit 1e5efee Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:41:35 2026 +0400 wip commit edaad0c Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:31:11 2026 +0400 wip commit 31accfa Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 22:20:16 2026 +0400 fix commit 0ef6dbc Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 21:19:15 2026 +0400 wip commit 664e783 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 20:23:02 2026 +0400 recursion: merkle tree -> 4 by 4 (decomposing index in nibbles rather than bits) commit 96d5f18 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 18:26:59 2026 +0400 wip commit b7c6ce8 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 17:38:02 2026 +0400 wip commit c2083c6 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 17:12:06 2026 +0400 sample_bits_and_compute_root_pow commit a42628e Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 16:33:02 2026 +0400 wip commit 46503d6 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 16:15:27 2026 +0400 wip commit 242e6b0 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 16:04:52 2026 +0400 naming commit 5ce9d59 Merge: 6d0400f a2fd1fa Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 15:59:18 2026 +0400 Merge branch 'hint_merkle_data_locally' into improve_isa commit a2fd1fa Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 14:12:31 2026 +0400 fix pruning commit 7bdb3ce Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 13:50:20 2026 +0400 niceee commit 76d524d Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 12:45:04 2026 +0400 negative fp-relative offsets commit 37a3ed2 Merge: fa9ee48 6d0400f Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 02:14:22 2026 +0400 Merge branch 'improve_isa' into hint_merkle_data_locally commit fa9ee48 Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 02:14:03 2026 +0400 wip hint_merkle_data_locally commit 6d0400f Author: Tom Wambsgans <[email protected]> Date: Mon Feb 23 02:12:58 2026 +0400 WIP Replace DotProduct precompile by a more general toolkit for extension field operations commit 2b907a9 Merge: ce6f9fa a56d29c Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 15:51:22 2026 +0400 Merge branch 'main' into improve_isa commit ce6f9fa Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 15:49:16 2026 +0400 wip commit c6d9ddc Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 14:04:12 2026 +0400 better memory usage commit 84bdbdf Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 13:53:12 2026 +0400 wip commit d9f3d1c Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 13:32:29 2026 +0400 wip commit 19e90e6 Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 12:28:34 2026 +0400 display commit b099b65 Author: Tom Wambsgans <[email protected]> Date: Sun Feb 22 00:33:30 2026 +0400 wip commit 78a7862 Author: Tom Wambsgans <[email protected]> Date: Sat Feb 21 17:15:07 2026 +0400 wip commit d12a31e Author: Tom Wambsgans <[email protected]> Date: Sat Feb 21 02:07:31 2026 +0400 wip first draft commit ec73b82 Author: Tom Wambsgans <[email protected]> Date: Fri Feb 20 22:52:36 2026 +0400 ISA experiment: more flexible precompile instructions with fp-relative offsets commit ca71cb5 Author: Tom Wambsgans <[email protected]> Date: Fri Feb 20 21:47:30 2026 +0400 wip commit f451bfa Author: Tom Wambsgans <[email protected]> Date: Fri Feb 20 19:58:49 2026 +0400 implem the ISA simplification commit 655f4e7 Author: Tom Wambsgans <[email protected]> Date: Fri Feb 20 17:03:16 2026 +0400 wip improvement of the ISA
1 parent a56d29c commit f07a751

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+2538
-1806
lines changed

‎README.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,22 +36,23 @@ cargo run --release -- xmss --n-signatures 1400
3636

3737
| WHIR rate \ regime | Proven | Conjectured |
3838
| ------------------ | -------------------- | -------------------- |
39-
| 1/2 | 610 XMSS/s - 383 KiB | 610 XMSS/s - 209 KiB |
40-
| 1/4 | 490 XMSS/s - 252 KiB | 490 XMSS/s - 148 KiB |
39+
| 1/2 | 775 XMSS/s - 374 KiB | 775 XMSS/s - 204 KiB |
40+
| 1/4 | 650 XMSS/s - 246 KiB | 650 XMSS/s - 146 KiB |
4141

4242
(Proving throughput - proof size)
4343

4444
### Recursion
4545

46+
2 to 1 recursion (WHIR rate = 1/4):
47+
48+
4649
```bash
4750
cargo run --release -- recursion --n 2
4851
```
4952

50-
2 to 1 recursion (WHIR rate = 1/4):
51-
5253
| Proven | Conjectured |
5354
| --------------- | --------------- |
54-
| 1.08s - 223 KiB | 1.03s - 134 KiB |
55+
| 0.77s - 223 KiB | 0.59s - 128 KiB |
5556

5657

5758
### Bonus: unbounded recursive aggregation

‎crates/backend/air/src/symbolic.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@ impl<F: Field> PrimeCharacteristicRing for SymbolicExpression<F> {
9898
const ONE: Self = Self::Constant(F::ONE);
9999
const TWO: Self = Self::Constant(F::TWO);
100100
const NEG_ONE: Self = Self::Constant(F::NEG_ONE);
101+
const INVERSE_OF_TWO: Self = Self::Constant(F::INVERSE_OF_TWO);
101102

102103
#[inline]
103104
fn from_prime_subfield(f: Self::PrimeSubfield) -> Self {

‎crates/backend/fiat-shamir/src/prover.rs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ where
3030
}
3131
}
3232

33-
pub fn raw_proof(&self) -> Vec<PF<EF>> {
34-
self.transcript.raw_proof()
33+
pub fn raw_proof(self) -> RawProof<PF<EF>> {
34+
self.transcript.into_raw_proof()
3535
}
3636

3737
pub fn pruned_proof(&self) -> PrunedProof<PF<EF>> {
@@ -74,14 +74,14 @@ where
7474

7575
fn state(&self) -> String {
7676
format!(
77-
"state: {} (len: {})",
77+
"state: {} (n_items: {})",
7878
self.challenger
7979
.state
8080
.iter()
8181
.map(|f| f.to_string())
8282
.collect::<Vec<_>>()
8383
.join(", "),
84-
self.transcript.raw_proof().len()
84+
self.transcript.0.len()
8585
)
8686
}
8787

‎crates/backend/fiat-shamir/src/traits.rs‎

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
use field::ExtensionField;
22

3-
use crate::{MerklePath, PF, ProofError, flatten_scalars_to_base, pack_scalars_to_extension};
3+
use crate::{MerkleOpening, MerklePath, PF, ProofError, flatten_scalars_to_base, pack_scalars_to_extension};
44

55
pub trait ChallengeSampler<EF> {
66
fn sample_vec(&mut self, len: usize) -> Vec<EF>;
@@ -40,9 +40,8 @@ pub trait FSProver<EF: ExtensionField<PF<EF>>>: ChallengeSampler<EF> {
4040

4141
pub trait FSVerifier<EF: ExtensionField<PF<EF>>>: ChallengeSampler<EF> {
4242
fn state(&self) -> String;
43-
fn transcript(&self) -> Vec<PF<EF>>;
4443
fn next_base_scalars_vec(&mut self, n: usize) -> Result<Vec<PF<EF>>, ProofError>;
45-
fn receive_hint_base_scalars(&mut self, n: usize) -> Result<Vec<PF<EF>>, ProofError>;
44+
fn next_merkle_opening(&mut self) -> Result<MerkleOpening<PF<EF>>, ProofError>;
4645
fn check_pow_grinding(&mut self, bits: usize) -> Result<(), ProofError>;
4746

4847
fn next_extension_scalars_vec(&mut self, n: usize) -> Result<Vec<EF>, ProofError> {
@@ -54,10 +53,4 @@ pub trait FSVerifier<EF: ExtensionField<PF<EF>>>: ChallengeSampler<EF> {
5453
fn next_extension_scalar(&mut self) -> Result<EF, ProofError> {
5554
Ok(self.next_extension_scalars_vec(1)?[0])
5655
}
57-
58-
fn receive_hint_extension_scalars(&mut self, n: usize) -> Result<Vec<EF>, ProofError> {
59-
Ok(pack_scalars_to_extension(
60-
&self.receive_hint_base_scalars(n * EF::DIMENSION)?,
61-
))
62-
}
6356
}

‎crates/backend/fiat-shamir/src/transcript.rs‎

Lines changed: 30 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,18 @@ use crate::{PrunedMerklePaths, challenger::RATE};
88

99
pub(crate) const DIGEST_LEN_FE: usize = 8;
1010

11+
#[derive(Debug, Clone)]
12+
pub struct MerkleOpening<F> {
13+
pub leaf_data: Vec<F>,
14+
pub path: Vec<[F; DIGEST_LEN_FE]>,
15+
}
16+
17+
#[derive(Clone)]
18+
pub struct RawProof<F> {
19+
pub transcript: Vec<F>,
20+
pub merkle_openings: Vec<MerkleOpening<F>>,
21+
}
22+
1123
#[derive(Debug, Clone, Serialize, Deserialize)]
1224
pub enum TranscriptData<F, MerklePaths> {
1325
Interraction(Vec<F>),
@@ -30,34 +42,37 @@ pub struct MerklePaths<Data, F>(pub(crate) Vec<MerklePath<Data, F>>);
3042
pub struct Proof<F>(pub(crate) Vec<TranscriptData<F, MerklePaths<F, F>>>);
3143

3244
impl<F: Field> Proof<F> {
33-
pub fn raw_proof(&self) -> Vec<F> {
34-
let mut proof = Vec::new();
35-
for item in &self.0 {
45+
pub fn into_raw_proof(self) -> RawProof<F> {
46+
let mut transcript = Vec::new();
47+
let mut merkle_openings = Vec::new();
48+
for item in self.0 {
3649
match item {
3750
TranscriptData::Interraction(scalars) => {
38-
proof.extend_from_slice(scalars);
3951
let padding = scalars.len().next_multiple_of(RATE) - scalars.len();
40-
proof.extend(repeat_n(F::ZERO, padding));
52+
transcript.extend(scalars);
53+
transcript.extend(repeat_n(F::ZERO, padding));
4154
}
4255
TranscriptData::GrindingWitness(scalar) => {
43-
proof.push(*scalar);
44-
proof.extend(repeat_n(F::ZERO, RATE - 1));
56+
transcript.push(scalar);
57+
transcript.extend(repeat_n(F::ZERO, RATE - 1));
4558
}
4659
TranscriptData::MerklePaths(paths) => {
47-
for path in &paths.0 {
48-
proof.extend_from_slice(&path.leaf_data);
60+
for path in paths.0 {
4961
assert!(path.leaf_data.len() % RATE == 0);
50-
}
51-
for path in &paths.0 {
52-
for hash in &path.sibling_hashes {
53-
proof.extend_from_slice(hash);
54-
}
62+
merkle_openings.push(MerkleOpening {
63+
leaf_data: path.leaf_data,
64+
path: path.sibling_hashes,
65+
});
5566
}
5667
}
5768
}
5869
}
59-
proof
70+
RawProof {
71+
transcript,
72+
merkle_openings,
73+
}
6074
}
75+
6176
pub fn prune(self) -> PrunedProof<F> {
6277
PrunedProof(
6378
self.0

‎crates/backend/fiat-shamir/src/verifier.rs‎

Lines changed: 24 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ use symetric::Compression;
1010
pub struct VerifierState<EF: ExtensionField<PF<EF>>, P> {
1111
challenger: Challenger<PF<EF>, P>,
1212
transcript: Vec<PF<EF>>,
13-
index: usize,
13+
merkle_openings: Vec<MerkleOpening<PF<EF>>>,
14+
transcript_index: usize,
15+
merkle_opening_index: usize,
1416
_extension_field: std::marker::PhantomData<EF>,
1517
}
1618

@@ -19,12 +21,14 @@ where
1921
PF<EF>: PrimeField64,
2022
{
2123
#[must_use]
22-
pub fn new(transcript: Vec<PF<EF>>, compressor: P) -> Self {
24+
pub fn new(raw_proof: RawProof<PF<EF>>, compressor: P) -> Self {
2325
assert!(EF::DIMENSION <= RATE);
2426
Self {
2527
challenger: Challenger::new(compressor),
26-
transcript,
27-
index: 0,
28+
transcript: raw_proof.transcript,
29+
merkle_openings: raw_proof.merkle_openings,
30+
transcript_index: 0,
31+
merkle_opening_index: 0,
2832
_extension_field: std::marker::PhantomData,
2933
}
3034
}
@@ -48,34 +52,31 @@ where
4852
{
4953
fn state(&self) -> String {
5054
format!(
51-
"state {} (len: {})",
55+
"state {} (transcript_idx: {}, merkle_opening_idx: {})",
5256
self.challenger
5357
.state
5458
.iter()
5559
.map(|f| f.to_string())
5660
.collect::<Vec<_>>()
5761
.join(", "),
58-
self.index
62+
self.transcript_index,
63+
self.merkle_opening_index,
5964
)
6065
}
6166

62-
fn transcript(&self) -> Vec<PF<EF>> {
63-
self.transcript.clone()
64-
}
65-
6667
fn next_base_scalars_vec(&mut self, n: usize) -> Result<Vec<PF<EF>>, ProofError> {
6768
let n_padded = n.next_multiple_of(RATE);
68-
if n_padded > self.transcript.len() - self.index {
69+
if n_padded > self.transcript.len() - self.transcript_index {
6970
return Err(ProofError::ExceededTranscript);
7071
}
71-
let scalars = self.transcript[self.index..][..n].to_vec();
72-
if self.transcript[self.index + n..self.index + n_padded]
72+
let scalars = self.transcript[self.transcript_index..][..n].to_vec();
73+
if self.transcript[self.transcript_index + n..self.transcript_index + n_padded]
7374
.iter()
7475
.any(|&x| x != PF::<EF>::ZERO)
7576
{
7677
return Err(ProofError::InvalidPadding);
7778
}
78-
self.index += n_padded;
79+
self.transcript_index += n_padded;
7980
for chunk in scalars.chunks(RATE) {
8081
let mut buffer = [PF::<EF>::ZERO; RATE];
8182
for (i, val) in chunk.iter().enumerate() {
@@ -87,32 +88,32 @@ where
8788
Ok(scalars)
8889
}
8990

90-
fn receive_hint_base_scalars(&mut self, n: usize) -> Result<Vec<PF<EF>>, ProofError> {
91-
if n > self.transcript.len() - self.index {
91+
fn next_merkle_opening(&mut self) -> Result<MerkleOpening<PF<EF>>, ProofError> {
92+
if self.merkle_opening_index >= self.merkle_openings.len() {
9293
return Err(ProofError::ExceededTranscript);
9394
}
94-
let scalars = self.transcript[self.index..self.index + n].to_vec();
95-
self.index += n;
96-
Ok(scalars)
95+
let opening = self.merkle_openings[self.merkle_opening_index].clone();
96+
self.merkle_opening_index += 1;
97+
Ok(opening)
9798
}
9899

99100
fn check_pow_grinding(&mut self, bits: usize) -> Result<(), ProofError> {
100101
if bits == 0 {
101102
return Ok(());
102103
}
103104

104-
if self.index + RATE > self.transcript.len() {
105+
if self.transcript_index + RATE > self.transcript.len() {
105106
return Err(ProofError::ExceededTranscript);
106107
}
107108

108-
let witness = self.transcript[self.index];
109-
if self.transcript[self.index + 1..self.index + RATE]
109+
let witness = self.transcript[self.transcript_index];
110+
if self.transcript[self.transcript_index + 1..self.transcript_index + RATE]
110111
.iter()
111112
.any(|&x| x != PF::<EF>::ZERO)
112113
{
113114
return Err(ProofError::InvalidPadding);
114115
}
115-
self.index += RATE;
116+
self.transcript_index += RATE;
116117

117118
self.challenger.observe({
118119
let mut value = [PF::<EF>::ZERO; RATE];

‎crates/backend/field/src/field.rs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,8 @@ pub trait PrimeCharacteristicRing:
110110
/// If the field has characteristic 2 this is equal to ONE.
111111
const NEG_ONE: Self;
112112

113+
const INVERSE_OF_TWO: Self;
114+
113115
/// Embed an element of the prime field `ℤ/p` into the ring `R`.
114116
///
115117
/// Given any element `[r] ∈ ℤ/p`, represented by an integer `r` between `0` and `p - 1`

‎crates/backend/koala-bear/src/monty_31/aarch64_neon/packing.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,7 @@ impl<FP: FieldParameters> PrimeCharacteristicRing for PackedMontyField31Neon<FP>
188188
const ONE: Self = Self::broadcast(MontyField31::ONE);
189189
const TWO: Self = Self::broadcast(MontyField31::TWO);
190190
const NEG_ONE: Self = Self::broadcast(MontyField31::NEG_ONE);
191+
const INVERSE_OF_TWO: Self = Self::broadcast(MontyField31::INVERSE_OF_TWO);
191192

192193
#[inline]
193194
fn from_prime_subfield(f: Self::PrimeSubfield) -> Self {

‎crates/backend/koala-bear/src/monty_31/data_traits.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ pub trait FieldParameters: PackedMontyParameters + Sized {
5555
const MONTY_ONE: MontyField31<Self> = MontyField31::new(1);
5656
const MONTY_TWO: MontyField31<Self> = MontyField31::new(2);
5757
const MONTY_NEG_ONE: MontyField31<Self> = MontyField31::new(Self::PRIME - 1);
58+
const MONTY_INVERSE_OF_TWO: MontyField31<Self> = MontyField31::new((Self::PRIME + 1) >> 1);
5859

5960
// A generator of the fields multiplicative group. Needs to be given in Monty Form.
6061
const MONTY_GEN: MontyField31<Self>;

‎crates/backend/koala-bear/src/monty_31/monty_31.rs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,7 @@ impl<FP: FieldParameters> PrimeCharacteristicRing for MontyField31<FP> {
173173
const ONE: Self = FP::MONTY_ONE;
174174
const TWO: Self = FP::MONTY_TWO;
175175
const NEG_ONE: Self = FP::MONTY_NEG_ONE;
176+
const INVERSE_OF_TWO: Self = FP::MONTY_INVERSE_OF_TWO;
176177

177178
#[inline(always)]
178179
fn from_prime_subfield(f: Self) -> Self {

0 commit comments

Comments
 (0)