Skip to content

Revise build process and correct errors#522

Open
phreed wants to merge 24 commits intoyuezk:mainfrom
phreed:pixi-build
Open

Revise build process and correct errors#522
phreed wants to merge 24 commits intoyuezk:mainfrom
phreed:pixi-build

Conversation

@phreed
Copy link
Contributor

@phreed phreed commented Jul 14, 2025

I am attempting to use your system with our campus network.
I made a number of significant changes.
I hope you like some of them.

  • pixi build, while the devcontainer approach works it can be clumsy I have provided a brief comparison below.
  • Provide a cross platform installation package, pixi works well with conda packages. I have added a rattler-build task and recipe.yaml for building the package.
  • Provide installation scripts (these are only for linux)
  • Provide developer's guide, operator's guide, and a roadmap (in asciidoc). The roadmap is presumptive.

With these changes and those below it now work with our university servers.

phreed added 12 commits July 10, 2025 13:59
Development with devcontainers can be problematic. `pixi` environments
are light weight.
* A developer's guide * An operator's guide * A roadmap * An index
(could be rolled into the top level README.md)
The commit adds robust setup tools and improves file permission
handling:

- Add gp-setup script for automated system configuration - Add
gp-welcome script with setup guidance and status checks - Switch to
user-specific runtime directories instead of /var/run - Add proper
permission validation and error messaging - Support both root and
non-root deployment modes - Add test scripts and conda package
improvements - Document runtime directory selection and setup process
@yuezk yuezk requested a review from Copilot July 15, 2025 09:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR overhauls the build and packaging workflows by adopting a Pixi-based setup, adds cross-platform installation scripts and conda packaging recipes, and provides comprehensive test suites along with developer and operator guides.

  • Introduce Pixi tasks and update pixi.toml for build, test, package, and publish workflows.
  • Add shell scripts for installation, environment setup, task running, and version override.
  • Provide extensive end-to-end tests and AsciiDoc documentation for developers and operators.

Reviewed Changes

Copilot reviewed 60 out of 72 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/test-task-refactoring.sh Add strict naming convention test suite for Pixi tasks
tests/test-gp-setup-logging-fix.sh Validate gp-setup logging permission fixes
scripts/task-runner.sh New enhanced task runner with logging, workflows, and error handling
pixi.toml Define Pixi project config, dependencies, and task definitions
recipe.yaml Conda recipe for full application package build
recipe-cli.yaml Conda recipe for CLI-only package build
scripts/create-gp-version-override.sh Generate LD_PRELOAD library for GlobalProtect version override
docs/sects/*.adoc Developer and operator guide sections added/updated
Comments suppressed due to low confidence (3)

tests/test-gp-setup-logging-fix.sh:96

  • The grep pattern uses a literal '$USER' instead of expanding to the actual username, so the test for a user-specific log file will always fail. Change the pattern to "/tmp/gp-setup-$USER.log" without escaping the dollar sign.
    if grep -q "/tmp/gp-setup-\$USER\.log" "$gp_setup_script"; then

packaging/files/usr/lib/NetworkManager/dispatcher.d/gpclient-nm-hook:43

  • The comment refers to gpclient, but the code sends SIGUSR2 to the gpservice PID. Update the comment to match the actual process being signaled or adjust the signal target if incorrect.
      # Send a USR2 signal to the gpservice process to resume the VPN connection

scripts/task-runner.sh:179

  • The workflow references show-package-info, but no such task is defined in Pixi (it should be show-package-status). This will cause the workflow to fail at runtime.
            tasks=("clean" "build-cli" "test-cli" "package-cli" "show-package-info")

@yuezk
Copy link
Owner

yuezk commented Jul 15, 2025

@phreed Thank you very much for enhancing the project. I will review it gradually.

@phreed
Copy link
Contributor Author

phreed commented Jul 15, 2025

@phreed Thank you very much for enhancing the project. I will review it gradually.

I have it to the point where it works for me on my bluefin system.

I was able to correct some longstanding problems:

I am going to stop working on it now.
Let me know if any of it does not make sense.

I might come back to this when the openconnect issue 807 is resolved and propagated to conda-forge.
Then I can make a nice conda-forge package which should allow for broader distribution of this package.
Without that update there is just too much rube-goldberg.

And you are very welcome, thanks for making this.

@phreed
Copy link
Contributor Author

phreed commented Jul 15, 2025

I prefer to use pixi over devcontainers (though pixi can certainly be used in a devcontainer).

Key differences and use cases between pixi and devcontainers

Scope of control

Pixi focuses on managing the software packages and their versions within your existing operating system or container. Dev Containers, on the other hand, provide a completely isolated and pre-configured environment,
including the operating system itself.

Setup complexity

Setting up a Pixi environment is often less complex than a Dev Container, especially for basic use cases.
Dev Containers involve Docker and container configuration, which can have a steeper learning curve.

Resource usage

Dev Containers can be more resource-intensive due to running a separate operating system,
although optimized base images can mitigate this.
Pixi environments are generally lighter as they work within your existing OS.

@yuezk
Copy link
Owner

yuezk commented Jul 15, 2025

pixi is a new thing to me, but it looks modern. I will learn it and try to integrate it based on your work.

@phreed
Copy link
Contributor Author

phreed commented Jul 15, 2025

pixi is a new thing to me, but it looks modern. I will learn it and try to integrate it based on your work.

Yes, pixi is very new.
It is is part of the conda -> mamba -> micromamba -> pixi line.
The original conda was written in python, it was slow and required python be installed.
'pixi' is small, fast, written it rust, it is very much under active development.
'rattler-build' is a tool for building conda packages (primarily for conda-forge).
The big idea is that all conda-forge packages are compatible with each other and do not depend on the host os for very much.
This makes conda-forge packages very portable.
If you have questions about pixi just ask.

@phreed
Copy link
Contributor Author

phreed commented Jul 22, 2025

Where are we at with this? Do you need something from me?

@yuezk
Copy link
Owner

yuezk commented Jul 23, 2025

Hi @phreed, thank you for your work on this.

I just wanted to mention that I'm not currently full-time on this project—apologies for any delays in my responses.

I tested the gp-version-override feature and noticed that it may not be functioning as expected. Specifically, while the GP Override: Active - will report version ... message is logged, I’m not seeing the GP Override: Using app-version ... message, which suggests it might not be fully taking effect.

@phreed
Copy link
Contributor Author

phreed commented Nov 4, 2025

Hi @phreed, thank you for your work on this.

I just wanted to mention that I'm not currently full-time on this project—apologies for any delays in my responses.

I tested the gp-version-override feature and noticed that it may not be functioning as expected. Specifically, while the GP Override: Active - will report version ... message is logged, I’m not seeing the GP Override: Using app-version ... message, which suggests it might not be fully taking effect.

Hi, @yuezk ,
I am just getting back to this.
I have not needed the VPN, so...
I have a branch which is working for me now.
Here is what I do:

GP VPN working.
https://github.com/phreed/GlobalProtect-openconnect

git clone git@github.com:phreed/GlobalProtect-openconnect.git gpoc
cd gpoc
git checkout vanderbilt
pixi run build-cli

./vanderbilt-vpn 

For more information look in https://github.com/phreed/GlobalProtect-openconnect/blob/vanderbilt/docs/gpclientw-wrapper-guide.adoc

@phreed
Copy link
Contributor Author

phreed commented Nov 4, 2025

That branch has a bunch of vanderbilt specific stuff so I will need to back the changes into this PR.
Which pieces are you interested in having incorporated?

p.s. I know you are busy and I would appreciate your thoughts.

@yuezk
Copy link
Owner

yuezk commented Nov 5, 2025

Hi @phreed, I really appreciate your contribution! Your PR motivates me to learn more about the Pixi build and the monkey patching mechanism. Although I still couldn't override the append_opt function in your repository.

After learning the code, I believe that the Pixi build system is modern and can achieve the same goal that I did with the Makefile and the dev container. However, I'm concerned that it cannot be used to build release packages for different Linux distributions because it requires online installation and internet access to download dependencies during the build process.

When releasing the DEB/RPM packages, the build machine cannot access the internet, and we can only use the tools and dependencies from their own package managers' repositories. Therefore, we cannot use the Pixi build system to create release packages for different Linux distributions.

Considering this, I am unable to introduce the Pixi build system. Otherwise, we have to maintain two build systems for the same project, which is not a good practice. In addition to the Pixi-related changes, I will incorporate some of your other changes in future releases. Thank you again for your contribution!

@phreed
Copy link
Contributor Author

phreed commented Nov 5, 2025

Hi @phreed, I really appreciate your contribution! Your PR motivates me to learn more about the Pixi build and the monkey patching mechanism. Although I still couldn't override the append_opt function in your repository.
Are you asking for the ability to edit my repository?
I have added you as a collaborator.

@phreed
Copy link
Contributor Author

phreed commented Nov 5, 2025

After learning the code, I believe that the Pixi build system is modern and can achieve the same goal that I did with the Makefile and the dev container. However, I'm concerned that it cannot be used to build release packages for different Linux distributions because it requires online installation and internet access to download dependencies during the build process.

When releasing the DEB/RPM packages, the build machine cannot access the internet, and we can only use the tools and dependencies from their own package managers' repositories. Therefore, we cannot use the Pixi build system to create release packages for different Linux distributions.

Are you saying that the building of the DEB/RPM are happening on machines with the proper packages pre-loaded?
You are talking about https://github.com/yuezk/GlobalProtect-openconnect/blob/main/.github/workflows/release.yaml , correct?
There are several ways to pre-load (or mirror) pixi packages.

@yuezk
Copy link
Owner

yuezk commented Nov 6, 2025

Are you saying that the building of the DEB/RPM are happening on machines with the proper packages pre-loaded?
You are talking about main/.github/workflows/release.yaml , correct?

Not really. It is not the GitHub CI job. It is the distribution's CI machine, for example, the Ubuntu's Launchpad and the OpenSUSE's Build Service. These services don't allow access to the internet. We also need to support different architectures (i.e., x86_64 and aarch64). So it may not be that easy to prepare the package.

@phreed
Copy link
Contributor Author

phreed commented Nov 6, 2025

@yuezk I am considering packaging the command line application as a conda-forge package.

Here is an example of a substantial illustrative package (both command-line and GUI):

image As you can see it has variants for multiple architectures and operating systems.

I realize you have a liberal license for the command line version, so, that is the part I am thinking of packaging.
The GUI version could be packaged, but the license would make it difficult to include in conda-forge.
There are repositories and ways of publishing packages with commercial licenses, e.g., https://prefix.dev/channels, if you are interested.

Note:: I generally run FCOS based systems so installing application level packages (*.rpm *.deb) is not something I do.
System level packages are a different story.

@yuezk
Copy link
Owner

yuezk commented Nov 10, 2025

I am considering packaging the command line application as a conda-forge package.

Got the idea, this is much easier to achieve. Publishing the CLI to the conda-forge and user can install it with related package manager, e.g., pixi.

@yuezk yuezk mentioned this pull request Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants