Skip to content

Add installation instructions & llama.cpp as a submodule#1

Open
HaroldBenoit wants to merge 7 commits intoIST-DASLab:mainfrom
HaroldBenoit:feat/repo_installation
Open

Add installation instructions & llama.cpp as a submodule#1
HaroldBenoit wants to merge 7 commits intoIST-DASLab:mainfrom
HaroldBenoit:feat/repo_installation

Conversation

@HaroldBenoit
Copy link
Copy Markdown

Hello, thank you very much for this toolkit. This is very useful for the community, and this PR aims to make a few improvements.

  • It adds an explicit requirements.txt & installation instructions in the README.md.

    • The current dependencies specified in the README.md don't work out of box with pip or uv.
  • It adds explicitly llama.cpp as a submodule.

    • This allows a clear way to obtain llama binaries such as llama-quantize necessary for scripts such as quant/gguf/run_quant.sh .
    • This also fixes import errors (e.g. MistralTokenizerType is not importable) when running quant/gptq/pack_gptq_into_gguf.py, as the pypi version of gguf is not up-to-date, and llama.cpp implictly requires to point to the local upstream version stored in the gguf-py folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant