-
-
Notifications
You must be signed in to change notification settings - Fork 312
clarify flakes as erring to implicit management of recursive dependencies #1197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…cies while i'm not sure this is the right place for this, i think defaulting to implicit management of recursive dependencies is currently a feature that sets it apart from competing dependency management systems for Nix. in my understanding, that makes this a relevant consideration in whether one might choose to adopt them. i.e., if one wants a better `apt` offering packages that span multiple nixy repositories, flakes may well be the closest thing they can find right now. on the other hand, if one is coding a project themselves, i.e. may want more control over their (recursive) dependencies, then this distinguishing feature in itself may be sufficient reason to opt to pass up on them, at least for the purpose of a default mode of dependency management. Signed-off-by: cinereal <[email protected]>
|
I'm not sure that I understand the title or the proposed wording. Flake inputs do indeed allow users to inject their own inputs, but as I see it, there's an unpaved road that leads to making use of that input in the derivations from Nixpkgs that they're likely trying to use. That usually ends in a long and somewhat complicated overlay. Am I misunderstanding here? |
|
what i tried to say here is, while flake this differs from how dependency management would work traditionally, e.g. how package derivations would specify all their dependencies as module inputs, making the caller specify inputs explicitly. that explicit approach seems facilitated by the traditional dependency pinning mechanisms such as i hope that helps a bit - feel free to suggest changing the wording if i wasn't communicating that intent well. |
|
So "1000 instances of Nixpkgs" as a positive reference rather than a negative one? |
|
depending on your use-case, it could be. i think most current nix(os) users are rather in the category of coding up declarative set-ups rather than using nix imperatively, so i believe for most of us in the community '1000 instances of nixpkgs' is not what we want. i agree that, given that, this might not be the right place to clarify flakes offer implicit-leaning dependency management. |
|
in retrospect, i'd consider rewording the 'why controversial' section into a broader 'why you might consider alternatives' one. i feel like controversy perhaps reflected the time of writing, but doesn't necessarily seem like the reasons one should base their decisions off of today. |
|
Yeah, this whole section (before the proposed change) reads like it was written some years ago, which it likely was. I'm game for some broader reworking. Want to propose that? |
|
sure - let me try and draft something up |
Signed-off-by: cinereal <[email protected]>
|
i tried expanding this further now - feedback welcome |
Signed-off-by: cinereal <[email protected]>
ghost
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left my suggestions, it looks pretty good so far.
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
|
@normalcea thanks for your swift feedback! i've tried to address them now. |
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
philiptaron
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I let my fingers fly this morning. I'm mostly just trying to re-write in order to understand.
source/concepts/flakes.md
Outdated
| You have to judge for yourself based on your needs. | ||
|
|
||
| [schema]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-format | ||
| While flakes reduce complexity in some regards, they also introduce some complexity with additional mechanisms. | ||
| You will have to learn more about the system to fully understand how it works. | ||
|
|
||
| 1. References to other flakes can be specified using a dedicated [URL-like syntax](https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references). | ||
| A [flake registry] allows using symbolic identifiers for further brevity. | ||
| References can be automatically locked to their current specific version and later updated programmatically. | ||
| Both paradigms have their own set of unique concepts and support tooling that have to be learned, with varying ease of use, implementation quality, and support status. | ||
| At the moment, neither the stable nor the experimental interface is clearly superior to the other in all aspects. | ||
|
|
||
| [flake registry]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-registry.html | ||
| There are different levels of buy-in in the use of flakes, with each stage introducing their own trade-offs: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a few minutes to see if I could make these points in a more direct way. Is this what you're saying?
| You have to judge for yourself based on your needs. | |
| [schema]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-format | |
| While flakes reduce complexity in some regards, they also introduce some complexity with additional mechanisms. | |
| You will have to learn more about the system to fully understand how it works. | |
| 1. References to other flakes can be specified using a dedicated [URL-like syntax](https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references). | |
| A [flake registry] allows using symbolic identifiers for further brevity. | |
| References can be automatically locked to their current specific version and later updated programmatically. | |
| Both paradigms have their own set of unique concepts and support tooling that have to be learned, with varying ease of use, implementation quality, and support status. | |
| At the moment, neither the stable nor the experimental interface is clearly superior to the other in all aspects. | |
| [flake registry]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-registry.html | |
| There are different levels of buy-in in the use of flakes, with each stage introducing their own trade-offs: | |
| You should use flakes if you want to make your software easy for others to run without them needing to manage dependencies themselves. Flakes prioritize convenience for consumers: `nix run github:user/repo` just works, using the dependency versions the author tested with. | |
| You should avoid flakes if you want explicit control over your entire dependency tree. By default, flakes honor the lockfiles of your dependencies, meaning transitive dependency versions are chosen upstream, not by you. You can override this with follows declarations, but the default is implicit. Tools like [`npins`](https://nix.dev/guides/recipes/dependency-management.html) take the opposite approach: you explicitly declare every dependency. | |
| Other considerations: | |
| - Flakes remain an https://nix.dev/manual/nix/stable/development/experimental-features#xp-feature-flakes with no stabilization timeline. The interface could change. | |
| - {term}Nixpkgs does not use flakes in its internal tooling, though it provides the most-consumed flake reference. | |
| - Below the surface of `flake.nix`, the Nix language works identically either way. The same reproducibility is achievable with or without flakes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
an https://nix.dev/manual/nix/stable/development/experimental-features#xp-feature-flakes
i think we could use a hyperlink there as well.
Below the surface of
flake.nix, the Nix language works identically either way. The same reproducibility is achievable with or without flakes.
The same reproducibility is achievable with or without flakes.
compared to the wording on reproducibility i was using ('promoting a style of writing programs more likely to make them reproducible[^reproducible]'), i feel this suggested wording could be read so as to mean that nix could inherently grant reproducibility.
Below the surface of
flake.nix, the Nix language works identically either way.
like, yes and no.
as in, it is the same language, yes, but if one were to take this at face value, thereby regarding any level of flake buy-in as harmless, i feel that misrepresents the added difficulty to a vanilla consumer that would be posed by functionality exposed only through a project structured using the dendritic pattern.
combining these concerns on these two sentences, i'm feeling a bit hesitant as to what you intended to communicate here, and would perhaps personally consider leaving them out.
to make your software easy for others to run without them needing to manage dependencies themselves
By default, flakes honor the lockfiles of your dependencies, meaning transitive dependency versions are chosen upstream, not by you.
Tools like
npinstake the opposite approach: you explicitly declare every dependency.
i think these do not necessarily accurately represent traditional dependency management.
if a libraries manages dependencies by npins, if they offer a default.nix entrypoint handling overridable dependencies like { sources ? import ./npins, ... }: { inherit sources; } then inject those into exposed imported plain modules, that simultaneously offers implicit dependency management through the entrypoint, whereas overriding sources (or directly importing the module files) would offer explicit dependency management.
flake inputs, by contrast, to my knowledge don't have such an explicit option as as setting sources to wholesale overriding the entirety of inputs, instead relying on the user to learn about and unset/override these one by one, unless using proprietary alternatives.
You should use flakes if you want to make your software easy for others to run without them needing to manage dependencies themselves. Flakes prioritize convenience for consumers:
nix run github:user/repojust works, using the dependency versions the author tested with.
You should avoid flakes if you want explicit control over your entire dependency tree. By default, flakes honor the lockfiles of your dependencies, meaning transitive dependency versions are chosen upstream, not by you. You can override this with follows declarations, but the default is implicit. Tools likenpinstake the opposite approach: you explicitly declare every dependency.
overall, an underlying intent i had with the article was to break the traditional dichotomy of use vs. non-use, which i think the suggested phrasing here follows as well.
i think overcoming that dichotomy in favor of an understanding of the nuances of the different levels of buy-in is important, since it enables 'golden mean' choices that i think can beat the drawbacks the dichotomy holds.
(while i have yet to find a choice with no actual drawbacks, i think buying into 2 or 3 out of 4 levels comes somewhat close (edit: presuming one wants to expose the flake entry-point and wants the CI benefits), facilitating consumption by either method, tho i think only properly facilitating overrides in one of the two ways, depending on the level of buy-in.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that is to say, with the experimental feature enabled, #1202 handles nix run nixpkgs#hello. (under v2 that looks like nix-shell -p '(import (builtins.fetchTree "flake:nixpkgs").outPath { }).hello' --run 'hello', or (without experimental features) nix-shell -p '(import (builtins.fetchTree "github:nixos/nixpkgs?ref=nixos-25.11").outPath { }).hello' --run 'hello' for packages from arbitrary repositories.)
for vanilla entrypoints, with fetch-tree enabled and a look-up path <nixpkgs> for lib.getExe, the convenience of nix run could be achieved using bash aliases like:
- drop-in
nix-runfor use likenix-run github:nixos/nixpkgs?ref=nixos-25.11#hello(i.e. arguments likenix run's):alias nix-run='run() { $(nix-instantiate --raw --impure --eval --expr "(import <nixpkgs> {}).lib.getExe (import (builtins.fetchTree \"$(cut -d "#" -f 1 <<< "$1")\").outPath { }).$(cut -d "#" -f 2 <<< "$1")"); }; run' - simplified implementation for use like
nix-run github:nixos/nixpkgs?ref=nixos-25.11 hello(so args spaced):alias nix-run='run() { $(nix-instantiate --raw --impure --eval --expr "(import <nixpkgs> {}).lib.getExe (import (builtins.fetchTree \"$1\").outPath { }).$2"); }; run'
(/cc @fricklerhandwerk i think you talked about stuff sort of like this before?)
(edit: filed as suggestion to add as a built-in command at NixOS/nix#14992)
to be fair, how you want to run commands from remote repositories can technically be a separate decision from what cli/entrypoint you use when writing code yourself, with the latter a bit more of a slippery slope in terms of no longer testing if your code can be used easily with just stable features.
| ### Discoverability | ||
|
|
||
| [new command line interface]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix.html | ||
| A first level of use of flakes consists of including a `flake.nix` file specifying `outputs`. | ||
|
|
||
| Nix handles flakes differently than regular {term}`Nix files <Nix file>` in the following ways: | ||
| Pros: | ||
|
|
||
| - Allows use of the code through the flake, including the v3 command line interface, downstream flake projects, as well as flake registries. | ||
| - The `flake.nix` file is checked for schema validity. | ||
|
|
||
| In particular, the metadata fields cannot be arbitrary Nix expressions. | ||
| This is to prevent complex, possibly non-terminating computations while querying the metadata. | ||
| Cons: | ||
|
|
||
| - Flakes are not [parameterized]. | ||
| This means that flakes [downgrade] ease of use of the `system` parameter of derivations, for producers and consumers, | ||
| leading to related [utility functions]. | ||
| - As an experimental feature, flakes could still be changed by Nix developers. | ||
|
|
||
| [parameterized]: https://github.com/NixOS/nix/issues/2861 | ||
| [downgrade]: https://github.com/NixOS/nix/issues/3843 | ||
| [utility functions]: https://github.com/numtide/flake-utils | ||
|
|
||
| Alternatives: | ||
|
|
||
| - Use flakes as thin wrappers over existing Nix code, so code can be used in both ways. | ||
| - Settle for exposing traditional Nix modules, and let flake users import the project with `flake = false;`. | ||
|
|
||
| [`import`]: https://nix.dev/tutorials/nix-language#import |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a stab at trying to be more direct here. I think this is what you're saying.
| ### Discoverability | |
| [new command line interface]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix.html | |
| A first level of use of flakes consists of including a `flake.nix` file specifying `outputs`. | |
| Nix handles flakes differently than regular {term}`Nix files <Nix file>` in the following ways: | |
| Pros: | |
| - Allows use of the code through the flake, including the v3 command line interface, downstream flake projects, as well as flake registries. | |
| - The `flake.nix` file is checked for schema validity. | |
| In particular, the metadata fields cannot be arbitrary Nix expressions. | |
| This is to prevent complex, possibly non-terminating computations while querying the metadata. | |
| Cons: | |
| - Flakes are not [parameterized]. | |
| This means that flakes [downgrade] ease of use of the `system` parameter of derivations, for producers and consumers, | |
| leading to related [utility functions]. | |
| - As an experimental feature, flakes could still be changed by Nix developers. | |
| [parameterized]: https://github.com/NixOS/nix/issues/2861 | |
| [downgrade]: https://github.com/NixOS/nix/issues/3843 | |
| [utility functions]: https://github.com/numtide/flake-utils | |
| Alternatives: | |
| - Use flakes as thin wrappers over existing Nix code, so code can be used in both ways. | |
| - Settle for exposing traditional Nix modules, and let flake users import the project with `flake = false;`. | |
| [`import`]: https://nix.dev/tutorials/nix-language#import | |
| ### Adding a `flake.nix` | |
| Adding a `flake.nix` exposes your project to the flake ecosystem. Users can reference it in their flake inputs, build it with `nix build`, run it with `nix run`, or add it to a registry. | |
| The trade-off is that [flakes must enumerate outputs for each system](https://github.com/NixOS/nix/issues/2861) (x86_64-linux, aarch64-darwin, etc.) rather than accepting system as an argument. Libraries like [`flake-utils`](https://github.com/numtide/flake-utils) and [`blueprint`](https://github.com/numtide/blueprint) reduce this boilerplate but do not eliminate it. | |
| You can support both paradigms by keeping logic in regular Nix files and using `flake.nix` as a thin wrapper. Flake users can also import non-flake projects directly with `flake = false` in their inputs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
while i realized this is based on my phrasing, i guess the wording here reminds me of some nuances to this:
Adding a
flake.nixexposes your project to the flake ecosystem. Users can reference it in their flake inputs
- non-flake projects can in fact be sourced, the difference being that for projects with a
flake.nix, you'd get the flake'soutputsover just the project path, while for flake projects additionally usinginputssuch (recursive) dependencies would be sourced as well.
build it with
nix build, run it withnix run
- the v3 cli (
nix build,nix run) would support loading arbitrary nix files using-f <path-to-nix-file>
or add it to a registry.
rechecking this, i think what you can add is not confined to locations containing a flake.nix. (adding references to the relevant JSON configuration by NixOS's nix.registry further seems to require no experimental features - making use of the registry i'm not sure, but i'm seeing it described as more closely related to fetch-tree.)
blueprint
hm, this is making me realize my split into four sequential-ish sections was maybe still less granular than the actual considerations involved.
for the last section (flake-only) i had been thinking essentially of the dendritic pattern, which makes modules very hard to consume through anything but the flake entrypoint.
blueprint tho uses modules with flakes' module arg inputs, which is not nearly as bad as the dendritic pattern for vanilla consumers, but does still make for a gap with vanilla modules.
(as an additional consideration there, that technically isn't even exactly about adding another section in-between dependency management and flake-only, in the sense the self parameter would be available before use of flakes' inputs for dependency management as well.)
maybe this also feels a bit awkward in the sense that for compatibility blueprint would be closer to such later sections, while at the same time i understand that it's on-topic to the system section you added it for here.
Flake users can also import non-flake projects directly with
flake = falsein their inputs.
i'm probably nitpicking here, but i feel the new phrasing here could be misread such as to imply that being able to import both types of entrypoints is specific to opting in to flakes (to e.g. the point of enabling the experimental features). given e.g. flake-inputs i think the potential for such a reading could end up misleading, regardless of intent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, i think the point about being parameterizable went lost in the new wording
| ### Running commands | ||
|
|
||
| Given the above, one may further access the created flake by Nix's [v3 command line interface], enabled by separate experimental feature [nix-command], which can build or run programs given a flake references. | ||
|
|
||
| [v3 command line interface]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix.html | ||
| [nix-command]: https://nix.dev/manual/nix/stable/development/experimental-features#xp-feature-nix-command | ||
|
|
||
| Pros: | ||
|
|
||
| - Flakes cache evaluations of builds to save time on subsequent identical builds, which can save time when say running unchanged builds in Continuous Integration. | ||
| - The v3 command-line interface, together with flakes, promotes making programs easy to run (using `nix run` to run executables in `outputs.apps`), including from remote repositories. | ||
| - Flakes default to running in [pure mode], promoting a style of writing programs more likely to make them reproducible[^reproducible]. | ||
|
|
||
| [pure mode]: https://nix.dev/manual/nix/stable/tutorials/nix-language.html#impurities | ||
|
|
||
| [^reproducible]: Even in pure mode, reproducibility is [not actually guaranteed]. | ||
|
|
||
| [not actually guaranteed]: https://discourse.nixos.org/t/nix-flakes-explained-what-they-solve-why-they-matter-and-the-future/72302/7 | ||
|
|
||
| Cons: | ||
|
|
||
| - The entire flake directory is copied to Nix store before evaluation. | ||
| This is used in evaluation caching, but also adds a [performance penalty], especially for large repositories such as {term}`Nixpkgs`. | ||
| - There are still various outstanding issues with the implementations of [flakes] and the [v3 `nix` command line interface]. | ||
| - In projects using [Git] version control, files must be staged for flakes to see them. | ||
|
|
||
| This allows for effective evaluation caching, which is relevant for large expressions such as Nixpkgs, but also requires copying the entire flake directory again on each change. | ||
| [performance penalty]: https://github.com/NixOS/nix/issues/3121 | ||
| [flakes]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Aflakes+sort%3Areactions-%2B1-desc | ||
| [v3 `nix` command line interface]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Anew-cli+sort%3Areactions-%2B1-desc | ||
|
|
||
| - No external variables, parameters, or impure language values are allowed. | ||
| Alternatives: | ||
|
|
||
| It means full reproducibility of a Nix expression, and, by extension, the resulting build instructions by default, but also prohibits parametrisation of results by consumers. | ||
| - The [v2 command line interface] (e.g. [nix-build], [nix-shell]) is used to interfaces with traditional Nix files. | ||
| - The [`--file` flag] allows the v3 commands to operate on traditional Nix files. | ||
| - In {term}`NixOS`, to use the v3 commands with a package set `pkgs` rather than | ||
| with a flake NixOS configuration's {term}`Nixpkgs` instantiation, | ||
| one may use [`nixpkgs.flake.source = pkgs.path;`]. | ||
|
|
||
| (flakes-controversy)= | ||
| ## Why are flakes controversial? | ||
| [Git]: https://git-scm.com/ | ||
| [v2 command line interface]: https://nix.dev/manual/nix/stable/command-ref/main-commands | ||
| [`--file` flag]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-build.html#options-that-change-the-interpretation-of-installables | ||
| [nix-build]: https://nix.dev/manual/nix/stable/command-ref/nix-build.html | ||
| [nix-shell]: https://nix.dev/manual/nix/stable/command-ref/nix-shell.html | ||
| [`nixpkgs.flake.source = pkgs.path;`]: https://search.nixos.org/options?channel=unstable&show=nixpkgs.flake.source&query=nixpkgs.flake.source |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the longest section. Here's my attempt at summarizing what I think you're getting at here.
| ### Running commands | |
| Given the above, one may further access the created flake by Nix's [v3 command line interface], enabled by separate experimental feature [nix-command], which can build or run programs given a flake references. | |
| [v3 command line interface]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix.html | |
| [nix-command]: https://nix.dev/manual/nix/stable/development/experimental-features#xp-feature-nix-command | |
| Pros: | |
| - Flakes cache evaluations of builds to save time on subsequent identical builds, which can save time when say running unchanged builds in Continuous Integration. | |
| - The v3 command-line interface, together with flakes, promotes making programs easy to run (using `nix run` to run executables in `outputs.apps`), including from remote repositories. | |
| - Flakes default to running in [pure mode], promoting a style of writing programs more likely to make them reproducible[^reproducible]. | |
| [pure mode]: https://nix.dev/manual/nix/stable/tutorials/nix-language.html#impurities | |
| [^reproducible]: Even in pure mode, reproducibility is [not actually guaranteed]. | |
| [not actually guaranteed]: https://discourse.nixos.org/t/nix-flakes-explained-what-they-solve-why-they-matter-and-the-future/72302/7 | |
| Cons: | |
| - The entire flake directory is copied to Nix store before evaluation. | |
| This is used in evaluation caching, but also adds a [performance penalty], especially for large repositories such as {term}`Nixpkgs`. | |
| - There are still various outstanding issues with the implementations of [flakes] and the [v3 `nix` command line interface]. | |
| - In projects using [Git] version control, files must be staged for flakes to see them. | |
| This allows for effective evaluation caching, which is relevant for large expressions such as Nixpkgs, but also requires copying the entire flake directory again on each change. | |
| [performance penalty]: https://github.com/NixOS/nix/issues/3121 | |
| [flakes]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Aflakes+sort%3Areactions-%2B1-desc | |
| [v3 `nix` command line interface]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Anew-cli+sort%3Areactions-%2B1-desc | |
| - No external variables, parameters, or impure language values are allowed. | |
| Alternatives: | |
| It means full reproducibility of a Nix expression, and, by extension, the resulting build instructions by default, but also prohibits parametrisation of results by consumers. | |
| - The [v2 command line interface] (e.g. [nix-build], [nix-shell]) is used to interfaces with traditional Nix files. | |
| - The [`--file` flag] allows the v3 commands to operate on traditional Nix files. | |
| - In {term}`NixOS`, to use the v3 commands with a package set `pkgs` rather than | |
| with a flake NixOS configuration's {term}`Nixpkgs` instantiation, | |
| one may use [`nixpkgs.flake.source = pkgs.path;`]. | |
| (flakes-controversy)= | |
| ## Why are flakes controversial? | |
| [Git]: https://git-scm.com/ | |
| [v2 command line interface]: https://nix.dev/manual/nix/stable/command-ref/main-commands | |
| [`--file` flag]: https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-build.html#options-that-change-the-interpretation-of-installables | |
| [nix-build]: https://nix.dev/manual/nix/stable/command-ref/nix-build.html | |
| [nix-shell]: https://nix.dev/manual/nix/stable/command-ref/nix-shell.html | |
| [`nixpkgs.flake.source = pkgs.path;`]: https://search.nixos.org/options?channel=unstable&show=nixpkgs.flake.source&query=nixpkgs.flake.source | |
| ### Using the v3 CLI | |
| The v3 command-line interface (`nix build`, `nix run`, `nix develop`) is a separate experimental feature (`nix-command`) designed around flakes and flake references. The goal is to make running software convenient: `nix run github:user/repo` fetches and executes in one command. | |
| Since flakes cache evaluations, identical builds are fast if run-run. Pure mode is enforced by default, encouraging reproducible expressions. | |
| The main downside is that the entire flake directory is copied to the Nix store before evaluation. This enables caching but adds a [time and storage tax](https://github.com/NixOS/nix/issues/3121) for large repositories like Nixpkgs. In addition, in Git repositories, unstaged files aren't copied: evaluation fails when it tries to access them, one file at a time, requiring repeated staging and retrying. | |
| You can use the v3 CLI without flake semantics via the `--file` flag, or stick with the v2 CLI (`nix-build`, `nix-shell`). The v2 CLI defaults to impure evaluation and evaluates files in place without copying, avoiding both downsides. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The v3 command-line interface [...] is a separate experimental feature (
nix-command) designed around flakes [...]
i feel this gives the impression this is an add-on experimental feature on top of the flakes experimental feature, tho the wiki seems to imply now that flakes need both while the v3 cli just needs nix-command. i think that's because v3 commands can be used (with --expr or --file without touching anything flake-related.
and flake references
while these references were designed for flakes originally, given they appear to have been split out into fetch-tree, i think referring to them in this way would seem to falsely imply they are a feature specific to flakes, whereas fetch-tree exposes them thru builtins.fetchTree as well.
identical builds are fast if run-run
if re-run?
on this one, i think the CI scenario was worthy of mention as a sane use-case because the baseline to repeating nix run seems repeatedly running commands through nix-shell (whether with --command / --run or by loading up the shell first then invoking the loaded programs), which similarly reuses cached derivations.
running nix-build twice tho on the same code admittedly would take up time twice - but then, as a human, why even do it?
Pure mode is enforced by default, encouraging reproducible expressions.
this wording i feel is a bit ambiguous. i think this allows for a reading that enabled pure mode makes for reproducible expressions, which as per the original footnote is inaccurate.
the note on nixpkgs.flake.source is for enabling use of like nix run nixpkgs#hello without a flake.nix in one's NixOS configuration. for repositories pinned in one's NixOS config, that could offer the convenience of the v3 cli. i've now filed PR #1202 to explain that for any pins at the dependency management page. in case that makes it in, it could make sense to link that section here.
on the original note on implementation issues, a reason i considered having it at this section was that i felt it served a place in user considerations among the levels of buy-in. that is to say, if one were to include a flake.nix for discoverability/compatibility for other people, while not actually running v3 commands themselves, they would effectively opt out of those implementation issues themselves, leaving those as problems for whoever were to use such an entrypoint.
| - Design | ||
| - The flakes proposal was criticised for trying to solve too many problems at once and at the wrong abstraction layer. | ||
| - The design still has [various problems] including around versioning, composability, cross-compilation, and tight coupling with nixpkgs. | ||
| - There are also still many [open design questions around the `nix` command line interface]. | ||
| - Implementation | ||
| - A number of [problems with the implementation] remain. | ||
| - Process | ||
| - While there were still outstanding concerns about the design, | ||
| the implementation was merged without the RFC having been accepted (and in fact being withdrawn on merge), | ||
| raising questions about proper process. | ||
| - The RFC was closed with no timeline to conclude the experiment. | ||
| - Flakes had become depended on by many projects, making it harder to iterate on their design without breaking many people's code. | ||
| - Community | ||
| - The design had not been accepted by all parts of the community, with e.g. {term}`Nixpkgs` not using it in its internal tooling. | ||
| As a result of this, branching approaches to flakes have been made, | ||
| with e.g. company Determinate Systems (which offers proprietary features around flakes) unilaterally declaring the feature stable, | ||
| while community-driven Nix fork Lix [consolidated the featureset] to a de-facto 'v1'. | ||
| Such branching could potentially end up breaking the promise of a unified interface that propelled the flakes experiment in the first place. | ||
|
|
||
| Other than that, and below the surface of the flake schema, Nix and the Nix language work exactly the same in both cases. | ||
| In principle, the same level of reproducibility can be achieved with or without flakes. | ||
| In particular, the process of adding software to {term}`Nixpkgs` or maintaining {term}`NixOS` modules and configurations is not affected by flakes at all. | ||
| There is also no evidence that flakes could help solve the scalability challenges of either. | ||
|
|
||
| Finally, there are downsides to relying on [experimental features][experimental feature] in general: | ||
|
|
||
| - Interfaces and behavior of experimental features could still be changed by Nix developers. | ||
| This may require you to adapt your code at some point in the future, which will be more effort when it has grown in complexity. | ||
| [Currently there is no concrete timeline for stabilising flakes.](https://discourse.nixos.org/t/stabilising-the-new-nix-command-line-interface/35531#p-123372-how-does-this-relate-to-flakes-3) | ||
| In contrast, stable features in Nix can be considered stable indefinitely. | ||
| - The [Nix maintainer team](https://nixos.org/community/teams/nix.html) focuses on fixing bugs and regressions in stable interfaces, supporting well-understood use cases, as well as improving the internal design and overall contributor experience in order to ease future development. | ||
| Improvements to experimental features have low priority. | ||
| - The [Nix documentation team](https://nixos.org/community/teams/documentation.html) focuses on improving documentation and learning materials for stable features and common principles. | ||
| When using flakes, you will have to rely more heavily on user-to-user support, third-party documentation, and the source code. | ||
| [various problems]: https://wiki.lix.systems/books/lix-contributors/page/flakes-feature-freeze#bkmrk-design-issues-of-fla | ||
| [open design questions around the `nix` command line interface]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Anew-cli+sort%3Areactions-%2B1-desc | ||
| [problems with the implementation]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Aflakes+sort%3Areactions-%2B1-desc | ||
| [consolidated the featureset]: https://wiki.lix.systems/books/lix-contributors/page/flake-stabilisation-proposal |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a stab at turning these bullet points into prose. I think I captured what you were going for, but this one might be more "me" than the others.
| - Design | |
| - The flakes proposal was criticised for trying to solve too many problems at once and at the wrong abstraction layer. | |
| - The design still has [various problems] including around versioning, composability, cross-compilation, and tight coupling with nixpkgs. | |
| - There are also still many [open design questions around the `nix` command line interface]. | |
| - Implementation | |
| - A number of [problems with the implementation] remain. | |
| - Process | |
| - While there were still outstanding concerns about the design, | |
| the implementation was merged without the RFC having been accepted (and in fact being withdrawn on merge), | |
| raising questions about proper process. | |
| - The RFC was closed with no timeline to conclude the experiment. | |
| - Flakes had become depended on by many projects, making it harder to iterate on their design without breaking many people's code. | |
| - Community | |
| - The design had not been accepted by all parts of the community, with e.g. {term}`Nixpkgs` not using it in its internal tooling. | |
| As a result of this, branching approaches to flakes have been made, | |
| with e.g. company Determinate Systems (which offers proprietary features around flakes) unilaterally declaring the feature stable, | |
| while community-driven Nix fork Lix [consolidated the featureset] to a de-facto 'v1'. | |
| Such branching could potentially end up breaking the promise of a unified interface that propelled the flakes experiment in the first place. | |
| Other than that, and below the surface of the flake schema, Nix and the Nix language work exactly the same in both cases. | |
| In principle, the same level of reproducibility can be achieved with or without flakes. | |
| In particular, the process of adding software to {term}`Nixpkgs` or maintaining {term}`NixOS` modules and configurations is not affected by flakes at all. | |
| There is also no evidence that flakes could help solve the scalability challenges of either. | |
| Finally, there are downsides to relying on [experimental features][experimental feature] in general: | |
| - Interfaces and behavior of experimental features could still be changed by Nix developers. | |
| This may require you to adapt your code at some point in the future, which will be more effort when it has grown in complexity. | |
| [Currently there is no concrete timeline for stabilising flakes.](https://discourse.nixos.org/t/stabilising-the-new-nix-command-line-interface/35531#p-123372-how-does-this-relate-to-flakes-3) | |
| In contrast, stable features in Nix can be considered stable indefinitely. | |
| - The [Nix maintainer team](https://nixos.org/community/teams/nix.html) focuses on fixing bugs and regressions in stable interfaces, supporting well-understood use cases, as well as improving the internal design and overall contributor experience in order to ease future development. | |
| Improvements to experimental features have low priority. | |
| - The [Nix documentation team](https://nixos.org/community/teams/documentation.html) focuses on improving documentation and learning materials for stable features and common principles. | |
| When using flakes, you will have to rely more heavily on user-to-user support, third-party documentation, and the source code. | |
| [various problems]: https://wiki.lix.systems/books/lix-contributors/page/flakes-feature-freeze#bkmrk-design-issues-of-fla | |
| [open design questions around the `nix` command line interface]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Anew-cli+sort%3Areactions-%2B1-desc | |
| [problems with the implementation]: https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Aflakes+sort%3Areactions-%2B1-desc | |
| [consolidated the featureset]: https://wiki.lix.systems/books/lix-contributors/page/flake-stabilisation-proposal | |
| Flakes were criticized [during the RFC process](https://github.com/NixOS/rfcs/pull/49) for trying to solve too many problems at once and at the wrong abstraction layer. Despite unresolved design concerns, the implementation was merged before the RFC was accepted, then the [RFC was withdrawn](https://github.com/NixOS/rfcs/pull/49#issuecomment-659379115) in order to re-assess core design concerns. There is no timeline to conclude the experiment. | |
| This created a catch-22: flakes became widely adopted while fundamental questions remained open. [Various questions](https://wiki.lix.systems/books/lix-contributors/page/flakes-feature-freeze#bkmrk-design-issues-of-fla) around versioning, composability, and cross-compilation persist. [Problems with the implementation](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+label%3Aflakes+sort%3Areactions-%2B1-desc) remain. Yet changing the design now risks breaking the many projects that depend on it. | |
| The community remains divided. Nixpkgs, the largest Nix project, does not use flakes in its internal tooling. Different groups have taken divergent approaches: Determinate Systems unilaterally declared flakes stable (with proprietary extensions), while the Lix fork has [frozen](https://wiki.lix.systems/books/lix-contributors/page/flake-stabilisation-proposal) the current implementation into a de-facto "v1". This fragmentation of underlying implementations undermines the unified interface that motivated flakes in the first place. | |
| Despite all this, flakes through the v3 CLI remain a useful and relatively ergonomic tool for composing code across disparate ecosystems. As evidenced by adoption, they are the default choice for sharing Nix code in a federated or distributed manner. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the implementation was merged before the RFC was accepted, then the RFC was withdrawn in order to re-assess core design concerns
i was not involved personally, tho if this were the intent, i think it would have been more logical to not merge it - not to mention it seems to clash with graham declaring the design stable. as such, i'd maybe be inclined not to add the extra 'in order to re-assess core design concerns'.
Determinate Systems unilaterally declared flakes stable (with proprietary extensions)
originally what i had in mind they offer was lazy trees and flakehub.
'extensions' maybe describes lazy trees better than an external platform like flakehub.
maybe 'tooling' would better cover these? edit: nah this seems right, i forgot parameterizable-flakes. edit 2: lazy trees isn't proprietary from what i can see.
Despite all this, flakes through the v3 CLI remain a useful and relatively ergonomic tool for composing code across disparate ecosystems. As evidenced by adoption, they are the default choice for sharing Nix code in a federated or distributed manner.
for composing code
for what it's worth, composability has also been mentioned among points of criticism toward flakes:
- https://jade.fyi/blog/flakes-arent-real/
- https://wiki.lix.systems/books/lix-contributors/page/flake-stabilisation-proposal
Despite all this, flakes through the v3 CLI remain a useful and relatively ergonomic tool for composing code across disparate ecosystems. As evidenced by adoption, they are the default choice for sharing Nix code in a federated or distributed manner.
i feel this tends to conflate all the considerations involved. in practice we see this type of thinking devolve into what ends up as code hard to consume (particularly libraries using flake-parts) without workarounds, which i would consider harmful for the ecosystem as a whole, forcing everyone into this dependency management system that requires proprietary features to suck less, threatening community self-determination.
as a result, on mentions such as default choice (do we have stats better than graham's repos with flake.nix vs repos specifically containing default/shell files?), i think as a collective it seems in our interest to promote decisions based on merit - groupthink has seemed plenty common already, and i think is kind of what got us into this spot in the first place.
Co-authored-by: Philip Taron <[email protected]>
thank you! i tried to kinda add some of my considerations now - hopefully that might clarify a bit over the original wording. as some background, i used flakes for a few years, then last year redid my nixos config following jade's blog post using npins. for work we struggled with that more so still, having ended up with a project that partly used conversely, i think doing vanilla would not make it hard to pick up flakes: in fact, it would rather get one closer to the thin-wrapper type of combination that i think could actually make for the golden mean here, getting one the compatibility with the schema for discoverability, as well as with the v3 cli for caching in CI, potentially without having to get into the mess of exploding dependencies. i kind of hope we could move closer to that golden mean collectively - and if i've sounded bearish in my writing, that's kind of the experiences underlying that. |
Signed-off-by: cinereal <[email protected]>
8d2f17f to
6aa98ed
Compare
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
hsjobeki
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this definitely an improvement.
If possible can i ask you for a second pass over the document.
I left some comments.
- Split up complex sentences that say two things at once.
- Provide examples before explaining. (Show then tell)
This documentation mostly tells; but it doesn't show any examples at all, which makes it possibly alienating for new users.
source/concepts/flakes.md
Outdated
| ## What are flakes? | ||
|
|
||
| Technically, a [flake](https://nix.dev/manual/nix/stable/command-ref/new-cli/nix3-flake.html#description) is a file system tree that contains a file named `flake.nix` in its root directory. | ||
| Flakes are an [experimental feature] in Nix, proposed in [RFC 49], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a rather technical description of the status and history of flakes without saying what they actually are.
I'd suggest a slightly different approach.
Technically we are in /concepts but i didn't find any flake example in nix.dev. This is a clear documentation gap.
- Short sentence that says what a flake is
1.2 Experimental & status note. - Example of a flake.nix
- Explain the concepts
A flake is a Nix file that declares inputs and outputs with a standardized structure.
> Note: Experimental, requires nix > 2.4
Here's a minimal example (`flake.nix`):
```nix
....
```
... rest of the document
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note that the blog post that introduced flake defined them as:
a source tree (such as a Git repository) containing a file named flake.nix
.. clashing with the colloquial usage of the file itself as the flake
edit: i sort of worked around this by... leaving this implicit. i think it's understandable now nevertheless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for your feedback! i now rewrote most of it - should be a little better already.
the wiki article still makes for a more comprehensive tutorial, but yeah, maybe there's a scope to a concept page.
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
Signed-off-by: cinereal <[email protected]>
1465066 to
247e757
Compare
while i'm not sure this is the right place for this, i think defaulting to implicit management of recursive dependencies is currently a feature that sets it apart from competing dependency management systems for Nix. in my understanding, that makes this a relevant consideration in whether one might choose to adopt them.
i.e., if one wants a better
aptoffering packages that (individually) span multiple nixy repositories, flakes may well be the closest thing they can find right now.on the other hand, if one is coding a project themselves, i.e. may want more control over their (recursive) dependencies, then this distinguishing feature in itself may be sufficient reason to opt to pass up on them, at least for the purpose of a default mode of dependency management.