Internal precompile binaries hook #22

BitcoinZavior · 2026-01-18T00:45:16Z

Precompiled Binaries Feature

Resolves #20
This PR introduces a comprehensive precompiled binaries system that significantly improves the developer experience by reducing build times and providing secure, signed artifacts.

Key Features

Secure Distribution

Ed25519-signed precompiled binaries with public key verification
Automated signing and verification pipeline
Secure artifact hosting via GitHub Releases

Performance Improvements

Skip local Rust compilation for supported platforms
Automatic fallback to local builds when precompiled binaries unavailable
Configurable modes: auto, always, or never

Developer Experience

Zero-configuration setup for most users
Comprehensive CLI tooling for maintainers
Detailed documentation and troubleshooting guides

Configuration

Users can configure precompiled binaries in their pubspec.yaml:
Defaulting to auto which most users would want.

bdk_dart:
  precompiled_binaries:
    mode: auto  # auto | always | never

reez · 2026-01-22T17:50:07Z

This is looking nice, added a bunch of comments/questions, and if you can run dart format to fix the ci failure too.

I’ll set up the signing secret later, let me know if any other maintainer setup is needed.

reez · 2026-01-22T16:24:28Z

.github/workflows/ci.yml

 on:
  push:
-    branches: [main, "**"]
+    branches: [main]


Did you remove ** to reduce duplicate CI runs on branch pushes?

Yes, but this is something you can assess and configure.

reez · 2026-01-22T16:28:09Z

.github/workflows/precompile_binaries.yml

+      GITHUB_TOKEN: ${{ github.token }}
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions-rs/toolchain@v1


This uses toolchain: stable, but native/rust-toolchain.toml pins 1.85.1 and the CLI reads that channel when running rustup run. Should we switch to toolchain-file: native/rust-toolchain.toml (or set toolchain: 1.85.1) so precompiled artifacts are built with the pinned toolchain? Same applies to the Android job.

We switched both precompile jobs from stable to the pinned 1.85.1 toolchain. Pinning to 1.85.1 keeps a single source of truth and avoids subtle build/compatibility drift.

reez · 2026-01-22T16:36:34Z

README.md

+```
+
+`mode` controls when the precompiled path is used:
+- `auto` prefers precompiled binaries when available, otherwise builds locally


auto is described as preferring precompiled, but the hook currently disables precompiled when rustup is present (UserOptions.defaultUsePrecompiledBinaries). Should we update this wording to reflect the rustup heuristic, or change the behavior to match the doc

I suggest updating the wording in README.md. It now explicitly states that auto prefers local builds when the Rust toolchain (rustup) is detected; otherwise, it uses precompiled binaries.

reez · 2026-01-22T16:38:57Z

docs/precompiled_binaries.md

+- CI builds and uploads precompiled binaries via `.github/workflows/precompile_binaries.yml`.
+- Artifacts are tagged by the crate hash and uploaded to a GitHub release.
+- Each binary is signed with an Ed25519 key; the public key is embedded in `pubspec.yaml`.
+- The build hook tries to download a verified binary first and falls back to a local build if needed.


The doc says ‘download verified binary first and fall back’; should we note that mode: always is intended to disable fallback (once that behavior is enforced), or explicitly call out the rustup heuristic for auto?

Added a “Mode behavior” section. It clarifies that always currently falls back but is intended to fail if a verified binary isn’t present, and reiterates how never operates.

reez · 2026-01-22T16:42:33Z

lib/src/precompiled/artifacts_provider.dart

+      );
+    }
+
+    if (precompiled.mode == PrecompiledBinaryMode.never) {


mode: always currently just returns null on download/verify failure, which triggers fallback in PrecompiledBuilder. Should always fail the build instead so it truly requires precompiled binaries?

lib/src/precompiled/artifacts_provider.dart now tracks when mode == PrecompiledBinaryMode.always and throws PrecompiledBinaryRequiredException on any download/verification failure instead of returning null. Since PrecompiledBuilder does not catch that exception, the build fails rather than falling back whenever mode=always cannot obtain a signed artifact—so the builder enforces the “require precompiled” promise.

reez · 2026-01-22T16:57:04Z

lib/src/precompiled/cli/commands/precompile_binaries.dart

+    exitCode = 2;
+    return;
+  }
+  if (repository == null || repository.trim().isEmpty) {


--repository is required but never used in any gh release calls; should we pass --repo $repository to view/create/upload (or drop the flag if it’s not needed)?

Forgot to call it out earlier: --repository was left unused in the release commands by mistake, but we’ve since wired it back in. This lets us rerun the tool against another repo (e.g., for testing or experimentation) without editing the workflow. In production it still defaults to the current repo, so the rest of the release flow behaves as before.

reez · 2026-01-22T16:58:41Z

lib/src/precompiled/cli/commands/precompile_binaries.dart

+          '--package',
+          cratePackage,
+          '--release',
+          '--locked',


We pass --locked to cargo, but there’s no committed native/Cargo.lock in this repo; cargo build --locked will fail without it. Should we add native/Cargo.lock or remove --locked?

Lock file added and .gitignore updated to allow lock file changes.

reez · 2026-01-22T17:01:52Z

lib/src/precompiled/crate_hash.dart

+    final output = AccumulatorSink<Digest>();
+    final input = sha256.startChunkedConversion(output);
+
+    void addTextFile(File file) {


We concatenate lines from multiple files without a delimiter or filename marker which could (rarely) lead to hash collisions if file boundaries change. Should we consider prefixing each file with its path (or a separator) before hashing to make the hash more robust?

That scenario is extremely rare isn't it? we would have to arrange contents to collide via a prefix/suffix shift. I believe a lightweight fix ( a separator before its contents) should be sufficient.

reez · 2026-01-22T17:15:20Z

lib/src/precompiled/precompiled_builder.dart

+    );
+
+    final downloaded = await provider.tryGetPrecompiledArtifact();
+    if (downloaded != null) {


If we enforce mode: always, should we be surfacing a hard failure here (or in PrecompiledArtifactProvider) so the build doesn’t silently fall back when always is set?

PrecompiledArtifactProvider now throws PrecompiledBinaryRequiredException on any download/verify failure when mode == always. Since PrecompiledBuilder already propagates provider exceptions, the build will now fail fast instead of silently falling back—honoring the “require precompiled binaries” promise.

pubspec.yaml

BitcoinZavior · 2026-01-24T06:23:05Z

Thanks for the review 🧡 PR updated to address review comments and to provide more info.

BitcoinZavior added 30 commits January 13, 2026 09:00

Add core utility functions for hex encoding and HTTP retry

b7fe450

Add utility functions and target triple mapping

85dfab4

Add options and configuration parsing

feb7f85

Add crate hash calculation

e126921

Add Rust toolchain detection

9d582c6

Add Cargo build integration

1e2df77

Add artifacts provider for download and verification

6c6f974

Add precompiled builder main integration

c19cef8

Add CLI support utilities

7dab321

Add CLI framework

224d288

Add CLI commands: targets and hash

bb8ca56

Add CLI commands: gen_key and sign

ac5f174

Add CLI command: precompile_binaries

8434ef4

Add build tool executable

e2781fe

Add tests for precompiled binaries

0d197a2

Update artifact host references in tests

d7d4a3e

Add documentation for precompiled binaries

675d6e9

Add CI workflow for precompiling binaries

e9051e6

Integrate precompile workflow into main CI

518cfa5

Update build hook to use precompiled builder

7728dda

Format crate_hash.dart code

c062e86

Add dependencies and configuration to pubspec.yaml

8576a9c

Update .gitignore and README documentation

fbf3f6f

Update artifact host to bitcoindevkit/bdk-dart

e73cf33

Format rust_toolchain.dart code

8ed8ef6

Format crate_hash_test.dart code

5edfa04

Remove trailing newlines from CLI files

c572e77

Remove trailing newline from target.dart

0087a44

update readme

4e3094e

Pin Rust toolchain to 1.85.1 in CI workflow

00b0daf

reez reviewed Jan 22, 2026

View reviewed changes

BitcoinZavior added 3 commits January 22, 2026 14:30

Add exception class for required precompiled binaries

8412bc8

Update precompiled binaries documentation intro

1f98a55

Add mode behavior documentation section

f6ee437

BitcoinZavior added 3 commits January 24, 2026 09:00

Update README precompiled binaries intro

c1446c1

Update README mode behavior descriptions

cdc2fc2

Add repository parameter to GitHub CLI commands

229cdeb

Internal precompile binaries hook #22

Are you sure you want to change the base?

Internal precompile binaries hook #22

Uh oh!

Conversation

BitcoinZavior commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Precompiled Binaries Feature

Key Features

Configuration

Uh oh!

reez commented Jan 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BitcoinZavior Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BitcoinZavior Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

BitcoinZavior commented Jan 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

BitcoinZavior commented Jan 18, 2026 •

edited

Loading

BitcoinZavior Jan 22, 2026 •

edited

Loading

BitcoinZavior Jan 24, 2026 •

edited

Loading