Feature/pacemaker implementation by yuxzhou · Pull Request #485 · autoatml/autoplex

yuxzhou · 2026-03-03T15:01:29Z

Description

This PR integrates Pacemaker (P-ACE) as a newly supported MLIP backend for autoplex. This addition allows users to fit P-ACE potentials and perform ACE-RSS workflows using Pacemaker potentials, expanding the current support for GAP, MACE, NEQUIP, and others.

Key changes:

Core integration (src/autoplex):

New MLIP backend: Added P-ACE to the supported mlip_type options across fitting and RSS workflows.
Configuration: Introduced PacemakerSettings in settings.py. This class handles Pacemaker-specific hyperparameters.
Fitting logic: Updated machine_learning_fit in fitting/common/jobs.py to start the pace_fitting workflow when mlip_type="P-ACE".
RSS support: Updated process_rss in data/rss/utils.py to load and evaluate P-ACE potentials (via AutoplexPyACECalculator) during structure minimisation.

Testing (tests/)

Unit tests: Added two tests in tests/fitting/common/test_jobs.py to verify:
-- Default hyperparameter handling (test_pace_fit_maker_defaults).
-- Custom hyperparameter handling (test_pace_fit_maker_custom).
Workflow tests: Added full integration tests for RSS cycles in tests/auto/rss/test_flows.py (test_mock_workflow_for_PACE), ensuring the cycle of Fitting $\rightarrow$ RSS $\rightarrow$ Selection $\rightarrow$ Refitting works correctly for P-ACE.
Test data: Added necessary reference YAML configurations and test data in tests/test_data/fitting/PACE.

README.md file

Dependencies & Installation
To utilise the Pacemaker-ACE fitting, users need to install some dependencies. Details are shown in the modified README file.

…g_fit

Added installation instructions for TensorFlow, TensorPotential, and pyace.

…/yuxzhou/autoplex into feature/pacemaker-implementation

JaGeo · 2026-03-04T08:42:27Z

README.md

+git clone https://github.com/ICAMS/python-ace.git
+pip install --upgrade ./python-ace


I would recommend to install a specific commit at least if no proper versioning is available. Otherwise, the code will be brittle extremely fast and results might not be reproducible anymore.

I see. Based on my current tests, the code goes well within my expection. But I agree that there might be some unexpected side effect. Shall we wait for now and see if Tensorpotential team has any response on that?

We can also install the code (or a specific commit to be backward compatible) without further dependencies and then add the dependecies to autoplex itself.

naik-aakash · 2026-03-04T12:14:00Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

+        config_params.update(mlip_hypers)
+
+        print("\n[Step 4] After flattening (del mlip_hypers + update)")
+        print(f"  'mlip_hypers' in config_params: {'mlip_hypers' in config_params}")


Are these print statements in testing needed?

Ah good catch! I only printed the information out for myself to check. We can delete that!

naik-aakash · 2026-03-04T12:17:15Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

This complete testing module seems to does not follow the other tests in package as a whole. Is this format needed , can it be adapted to follow similar convention to keep things consistent?

This test is specifically for the hyperparameter parsing. The main unit tests regarding jobs and flows (following the similar format of other unit tests previously in the code) were written in the test_flows.py and test_jobs.py files. This is why I moved the hyperparameter parsing tests to the specific folder tests/test_data/fitting/PACE/. What convention shall I follow to keep it there or maybe let's remove it as it is a just simple test for certain functions?

The tests should not live in the "test_data" folder, as this is really only for data.

JaGeo · 2026-03-04T12:57:50Z

Just as a note here, @yuxzhou . I did not yet fully go through the code and mostly added info on the installation etc. I will hopefully find some time end of this week.

yuxzhou · 2026-03-04T13:14:14Z

Just as a note here, @yuxzhou . I did not yet fully go through the code and mostly added info on the installation etc. I will hopefully find some time end of this week.

Thank you Janine!!!

JaGeo

Hi @yuxzhou ,
I had a few miniutes to take a first look.
I think it looks already good! :) Unfortunately, there is still some work to do.

Just to clarify the purpose: we would like to directly replace GAP with ACE in the RSS workflows, right?

Here are some comments on the implementation:

In any case, pacemaker needs to be installed as part of our continous testing environment. Otherwise, the tests will not run. If you do not know how to do this, please let us know. Please use a specific commit of pyace and tensorpotential for now and try to install it without dependencies ("pip install --no-deps git+https://github.com/USER/REPO.git@COMMIT_HASH"); Then add the additional dependencies in our workflow as well.
Then, I would suggest implementing pacemaker in such a way that even if the installation fails, Autoplex can still be used. I left a few comments and pointers on how to do this. Specifically, one can use monty's requires for this.
I believe it would be good to think about the output of a fitting job. Currently, this is quite sparse. However, users might want to have additional data in their database. This could also include information on the potential itself and the hyperparameters.

JaGeo · 2026-03-04T14:41:08Z

src/autoplex/data/rss/utils.py

+            raise FileNotFoundError(
+                f"Could not find 'output_potential.yaml' in {mlip_path} for P-ACE."
+            )
+


Could you try importing AutoplexPyACECalculator here, rather than at the beginning? I would like to make that we step by step disentangle the fitting installations.

Thank you Janine! All sound very good. I will quickly reply to your points regarding pacemaker installation / inplementation in this chat (and show more details in the following sperate chats).

For the purpose: yes, we want to have a separate ACE-driven RSS flow that is independent of any GAP-RSS runs. I am not saying ACE will completely displace GAP (people can still run GAP-RSS if they want!), but we will have other MLIP options for the RSS process (ACE is particularlly important because it enables super large-scale MD simulations).

Regarding the dependency conflicts, I completely agree that we shall separate the installation of tensorpotential / python-ace and their original dependencies. To this end, I have added necessary dependencies in the pyproject.toml file, and modified the README.md file so that we guide users to properly install tensorflow / tensorpotential / python-ace without dependencies (i.e., via the way you suggest here pip install --no-deps git+https://github.com/USER/REPO.git@COMMIT_HASH).

For implementing pacemaker, that's a great idea. I understand that autoplex should still be able to run even without the installation of tensorflow / tensorpotential / python-ace (especially for those who don't want to run ACE-RSS at all!). I will move all importings into the if clauses.

Great! Thanks!
In the future, we should seperate all imports for the different MLIPs but we will do this stepwise! 😃

JaGeo · 2026-03-04T14:41:39Z

src/autoplex/fitting/common/jobs.py

+    pace_fitting,
 )
+from autoplex.settings import PacemakerSettings


Same here, could you move the import to the if clause?

JaGeo · 2026-03-04T14:41:56Z

src/autoplex/fitting/common/jobs.py

        mlip_paths.append(train_test_error["mlip_path"])

+    elif mlip_type == "P-ACE":
+


Please do the import here.

JaGeo · 2026-03-04T14:42:52Z

src/autoplex/fitting/common/utils.py

 from monty.serialization import dumpfn
 from nequip.ase import NequIPCalculator
 from numpy import ndarray
+from pyace.asecalc import PyACECalculator


Could you use the following to import?

try: from pyace.asecalc import PyACECalculator except ImportError: PyACECalculator=None

JaGeo · 2026-03-04T14:43:53Z

src/autoplex/fitting/common/utils.py

    MACE_HYPERS,
    NEP_HYPERS,
    NEQUIP_HYPERS,
+    PACEMAKER_HYPERS,


If this does not depend on pacemaker, it can stay here. Otherwise, move it to the specific functionality.

src/autoplex/fitting/common/utils.py

JaGeo · 2026-03-04T14:45:51Z

src/autoplex/fitting/common/utils.py

        return res


+class AutoplexPyACECalculator(PyACECalculator):


add in front of it:

@requires(PyAceCalculator, "pyace must be installed")

See examples for similar import solutions here in atomate2: from monty.dev import requires

Btw, I hope you don't mind the German directness: these are all suggestions to improve the code and long-term stability of the code. The directness is hard to control sometimes 😅

of course no! They are super useful!! Let me do the very best to standarise the code implementation. To be honest, I have so many to learn "how to properly code" so it's good to follow the "best practice" at the very beginning!! Thank you!!

JaGeo · 2026-03-04T14:47:50Z

src/autoplex/fitting/common/utils.py

+    return {
+        "train_error": train_error,
+        "test_error": test_error,
+        "mlip_path": Path.cwd(),
+    }


Can we think about what else would be useful to return? Do we just want to give an energy error? What other information would be useful for the user to save in a database? Do we want to add specific information on the hyper parameters as well? This is useful for further development of the fitting modules.

src/autoplex/fitting/common/utils.py

JaGeo · 2026-03-04T14:50:47Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

The tests should not live in the "test_data" folder, as this is really only for data.

yuxzhou · 2026-03-04T17:37:01Z

I am about to submit a new commit for modified README.md and pyproject.toml. Some updates:

I added these into the dependencies list in pyproject.toml:

     "pandas>=2.2",
     "numpy<=1.26.4",
     "scipy",
     "ruamel.yaml",
     "psutil",
     "scikit-learn<=1.4.2",

and these to the strict list:

     "numpy==1.26.4",
     "pandas>=2.2",
     "scipy",
     "ruamel.yaml",
     "psutil",
     "scikit-learn==1.4.2",

@JaGeo @naik-aakash Do you think they look OK (not sure if I did anything stupid)?

and in README.md file, I added the following:

ℹ️ To fit and validate Pacemaker ACE potentials, one also needs to install tensorflow, tensorpotential, and python-ace.
Please note that Pacemaker ACE fitting can be run on both CPU and GPU.
⚠️ Please also note on versioning: to prevent dependency conflicts (e.g., with pandas versions) and ensure stability, please install the exact commit hashes listed below using the --no-deps flag. These specific versions have been fully tested and validated for use with Autoplex.

pip install tensorflow==2.8.0
pip install --no-deps git+https://github.com/ICAMS/TensorPotential.git@1e44b2558356800ae070658c0bb856ff9bf74538
# Ensure CMake is available before running this:
pip install --no-deps git+https://github.com/ICAMS/python-ace.git@d1c213a7d9c5b809a3ae83b2e5a916be26d921f0

I re-run the whole installation process - no warning / errors.

JaGeo · 2026-03-04T17:51:00Z

I think this looks good.

You would also need to install it on the repo and make it part of our docker container.

I believe, you have to add it here for the continous testing:

autoplex/.github/workflows/python-package.yml

Line 50 in a9d8f82

python -m uv pip install --prerelease=allow .[strict,tests,aims] && \

And here for the docker, I believe:

autoplex/Dockerfile

Line 86 in a9d8f82

&& uv pip install --system --prerelease=allow ".[strict,docs]" \

autoplex/.devcontainer/devcontainer.json

Line 17 in a9d8f82

    
           "postCreateCommand": "pip cache purge && uv pip install --prerelease=allow -e .[strict,docs] && pre-commit install",

@naik-aakash does this look right (for when you are available again)?

naik-aakash · 2026-03-05T07:02:29Z

I think this looks good.

You would also need to install it on the repo and make it part of our docker container.

I believe, you have to add it here for the continous testing:

autoplex/.github/workflows/python-package.yml

Line 50 in a9d8f82

python -m uv pip install --prerelease=allow .[strict,tests,aims] && \

And here for the docker, I believe:

autoplex/Dockerfile

Line 86 in a9d8f82

&& uv pip install --system --prerelease=allow ".[strict,docs]" \

autoplex/.devcontainer/devcontainer.json

Line 17 in a9d8f82

"postCreateCommand": "pip cache purge && uv pip install --prerelease=allow -e .[strict,docs] && pre-commit install",

@naik-aakash does this look right (for when you are available again)?

yes this seems fine. Adding to devcontainer.json would not be strictly needed, as it will already use the new Docker image when a new release is made. Unless one wants to use an existing image for now, this change would be necessary.

But doc build workflow would also need pyace to be installed. Change would be needed here https://github.com/yuxzhou/autoplex/blob/91770a12d537a2bf5022dca61150e5f979a6752d/.github/workflows/docs.yml#L44

Add the same pip install pyace command added to the README for pyace installation.

naik-aakash · 2026-03-05T07:16:27Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

+
+
+@pytest.fixture
+def clean_dir(tmp_path, monkeypatch):


For example, this is duplicate fixture, we already have this fixture,

autoplex/tests/conftest.py

Line 200 in a9d8f82

def clean_dir(debug_mode):

could be reused if one moves these tests out of test_data to the appropriate module tests/*

I use the top imports a guide to decide under which module these tests can reside or new test_*.py module need to be added.

Same goes for tests added in test_pacemaker.py - they can be relocated

naik-aakash · 2026-03-05T07:22:59Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

+        return config_file
+
+    @pytest.fixture
+    def mock_training_data(self, tmp_path):


all fixtures needed can reside in conftest.py , at the top of tests modules

https://github.com/autoatml/autoplex/blob/main/tests/conftest.py

or also in sub test module level if they are very specific for a particular case

see https://github.com/autoatml/autoplex/tree/main/tests/misc/fhi-aims

naik-aakash · 2026-03-05T07:29:11Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

+        print("="*80)
+
+
+class TestPacemakerConfigFlow:


we are mostly using directly test_* based convention in tests, class based test exists only for one case which we will be replaced soon at some point. This is what also meant module having bit different convention as it is mixing fixtures and using class based tests 😅

naik-aakash · 2026-03-05T07:32:58Z

tests/test_data/fitting/PACE/test_pacemaker_config_flow.py

+        assert dumped["P-ACE"]["cutoff"] == 8.0
+
+
+class TestOtherMLIPsNotAffected:


These tests are probably not needed, I'm not 100 % though, as we have tests for other MLIP fits. If anything is affected by the new changes, it should be seen in existing test failures, I assume.

@JaGeo, do you have any opinions?

JaGeo · 2026-04-08T15:04:20Z

@yuxzhou Thanks for your contributions. I think the changes look good already.

Unfortunately, there appear to be version conflicts during installation. Could you take a look? It could take a bit more time to exactly figure out what is going on here.

…optional groups

…/yuxzhou/autoplex into feature/pacemaker-implementation

naik-aakash · 2026-04-10T06:40:00Z

pyproject.toml

+pacemaker = [
+    "protobuf<3.20 ; python_version == '3.10'",
+    "tensorflow==2.8.0 ; python_version == '3.10'",
+    "tensorpotential @ git+https://github.com/ICAMS/TensorPotential.git@1e44b2558356800ae070658c0bb856ff9bf74538 ; python_version == '3.10'",


I think we will not be able to release the autoplex on PyPI with URL-based dependencies.

Can a new version for tensorpotential / pyace be requested for release on PyPI?

@yuxzhou, I assume a PyPI release of autoplex is expected after this PR is merged. Thus asking

That it is true. We likely need to keep the additional Readme.txt stuff.

JaGeo · 2026-04-10T07:19:17Z

@yuxzhou thank you for the latest changes.

Would it be possible that instead of having Python 3.10 tests including Pacemamer that you have Python 3.10 tests with and without Pacemaker. You would need to add new test workflows for pacemaker. Similar to what we have done for the tutorials. Within these tests, we should then run only tests related to the pacemaker implementation.
You could keep everything similar to the file here but would need to replace, among others, this last line:

autoplex/.github/workflows/python-tutorial.yml

Line 51 in 08472a0

    
                     OMP_NUM_THREADS=1 pytest -vv --cache-clear  --nbmake  --nbmake-timeout=-1 ./tutorials

And, as @naik-aakash pointed out correctly, installations directly from git will not work within a pypi release. We would need to keep the additional Installation in the CI and the installation instructions in the readme.

yuxzhou and others added 11 commits February 3, 2026 17:59

Implement Pacemaker support: codes are all updated and unit tests passed

e1d6ad2

Implement Pacemaker support: error parsing and unit tests passed

96b2886

Fix Pacemaker fitting: implementation of correct parameter passing

3d7326e

Fix P-ACE: extract hyperparameters from fit_kwargs in machine_learnin…

f7bbf62

…g_fit

fix hypers passing issue for pacemaker

4aebbd1

implementation: Pacemaker ACE interface

51b8650

fix: export extxyz file during pacemaker fitting

ffa424a

check: e0 reading

10288d2

fix: species input

201d12b

Update README with installation instructions for Pacemaker

943b00f

Added installation instructions for TensorFlow, TensorPotential, and pyace.

update: remove all unnecessary comments; No changes on codes

8a1b04e

YuanbinLiu requested review from YuanbinLiu and removed request for YuanbinLiu March 3, 2026 15:31

yuxzhou and others added 5 commits March 3, 2026 17:15

Merge branch 'main' into feature/pacemaker-implementation

4d691aa

pre-commit auto-fixes

3e66741

Fix SIM102: Combine nested if statements in jobs.py

45ecc77

Merge branch 'feature/pacemaker-implementation' of https://github.com…

7de6ed8

…/yuxzhou/autoplex into feature/pacemaker-implementation

fix an error in pacemaker fit unit test

cf27c74

JaGeo reviewed Mar 4, 2026

View reviewed changes

naik-aakash reviewed Mar 4, 2026

View reviewed changes

JaGeo requested changes Mar 4, 2026

View reviewed changes

Fix dependency conflicts and update installation docs for Pacemaker

91770a1

naik-aakash reviewed Mar 5, 2026

View reviewed changes

yuxzhou and others added 4 commits April 6, 2026 23:09

fix: update for tensorpotential dependency

5585c47

pre-commit auto-fixes

2e0e7c0

style: add noqa tags to bypass ruff PLC0415 for conditional imports

ea285ce

chore: resolve merge conflict with pre-commit auto-fixes

f8dbae4

yuxzhou and others added 7 commits April 8, 2026 17:31

ci: fix git dubious ownership error in docker container for python 3.10

d0843b4

ci: edit python-package.yml

75465b8

chore: remove hardcoded dependencies to fully rely on pyproject.toml …

1aca90d

…optional groups

Merge branch 'main' into feature/pacemaker-implementation

78fcee5

fix space typo

51af9f8

Merge branch 'feature/pacemaker-implementation' of https://github.com…

02ec6fb

…/yuxzhou/autoplex into feature/pacemaker-implementation

ci: fix 3.10 version conflicts

31c5fea

naik-aakash reviewed Apr 10, 2026

View reviewed changes

		git clone https://github.com/ICAMS/python-ace.git
		pip install --upgrade ./python-ace

		mlip_paths.append(train_test_error["mlip_path"])

		elif mlip_type == "P-ACE":

		assert dumped["P-ACE"]["cutoff"] == 8.0


		class TestOtherMLIPsNotAffected:

Conversation

yuxzhou commented Mar 3, 2026

Description

Key changes:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JaGeo Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JaGeo commented Mar 4, 2026

Uh oh!

yuxzhou commented Mar 4, 2026

Uh oh!

JaGeo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JaGeo Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yuxzhou commented Mar 4, 2026

Uh oh!

JaGeo commented Mar 4, 2026

Uh oh!

naik-aakash commented Mar 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

naik-aakash Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JaGeo commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

naik-aakash Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

JaGeo Mar 4, 2026 •

edited

Loading

JaGeo Mar 4, 2026 •

edited

Loading

naik-aakash Mar 5, 2026 •

edited

Loading

JaGeo commented Apr 8, 2026 •

edited

Loading

naik-aakash Apr 10, 2026 •

edited

Loading

JaGeo commented Apr 10, 2026 •

edited

Loading