Ryzen AI LLM - Onnxruntime GenAI

Ryzen AI Software includes support for deploying LLMs on Ryzen AI PCs using the ONNX Runtime generate() API (OGA).

Pre-optimized Models

AMD provides a set of pre-optimized LLMs ready to be deployed with Ryzen AI Software and the supporting runtime for hybrid and NPU execution. These models can be found on Hugging Face:

Published models:

• Ryzen AI Hybrid models.
• Ryzen AI NPU models.

Ryzen AI Installation

• The steps for installing Ryzen AI along with it's requirement can be found in the Official Ryzen AI Software documantion page here - https://ryzenai.docs.amd.com/en/latest/inst.html

Steps to compile and run LLM example.

• Activate Ryzen AI environment:

conda activate ryzen-ai-1.6.0

• Download the model: This example uses the qwen model.

#hyrbid model:
git clone https://huggingface.co/amd/Qwen3-4B-awq-quant-onnx-hybrid

#npu model:
git clone https://huggingface.co/amd/Qwen2.5-7B-Instruct-onnx-ryzenai-npu

• Navigate to OGA_API folder:

cd path\to\RyzenAI-SW\example\llm\oga_api

• Copy necessary DLLs and header files:

xcopy /I "%RYZEN_AI_INSTALLATION_PATH%\LLM\lib\onnxruntime-genai.lib" libs
xcopy /I "%RYZEN_AI_INSTALLATION_PATH%\LLM\include\*" include

xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime-genai.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\onnxruntime.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\ryzen_mm.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\onnx_custom_ops.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\libutf8_validity.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\abseil_dll.dll" libs\
xcopy /Y "%RYZEN_AI_INSTALLATION_PATH%\deployment\DirectML.dll" libs\

• Compile and build the code:

   mkdir build
   cd build
   cmake .. -A x64
   cmake --build . --config Release
   cd bin\Release

• Execute code:

.\example.exe -m "<path_to_model>"

• Sample command

.\example.exe -m path\to\Qwen3-4B-awq-quant-onnx-hybrid   -n 2048 -c 1

• Sample output:

Initializing ORT GenAI...
Loading Model from: 
C:\Users\Ai_test\ryzenai1.6\model\Qwen3-4B-awq-quant-onnx-hybrid\
Model loaded.
Creating Tokenizer...
Tokenizer created.
Creating Generator...
Generator created.
--------------------------------
Enter prompt: Explain the basics of object oriented programming
Generating response:
  Of course! Object-oriented programming (OOP) is a programming paradigm that organizes software design around objects, which are instances of classes, rather than functions and data. Here are the basics of OOP:

1. Classes and Objects: In OOP, a class is a blueprint or template for creating objects. A class defines the properties and behaviors of an object, and it can contain other classes or objects as members. An object is an instance of a class, and it has its own set of attributes (data) and methods (functions).
2. Inheritance: Inheritance is the process of creating a new class based on an existing class. The new class (the subclass) inherits the properties and behaviors of the existing class (the superclass), and it can also add new properties and behaviors.
3. Polymorphism: Polymorphism is the ability of an object to take on many forms. In OOP, polymorphism can occur in two ways: method overriding and method overloading. Method overriding occurs when a subclass provides a different implementation of a method that is already defined in its superclass. Method overloading occurs when a class provides multiple definitions for a method with the same name but different parameters.
4. Encapsulation: Encapsulation is the practice of hiding the implementation details of an object from the outside world. In OOP, encapsulation is used to protect the data and methods of an object from external interference or misuse.
5. Abstraction: Abstraction is the process of representing complex real-world objects or systems in a simplified way. In OOP, abstraction is used to focus on the essential features of an object and to hide the irrelevant details.
6. Composition: Composition is the process of combining objects or classes to create a new object or system. In OOP, composition is used to create complex objects by combining simpler objects or classes.
7. Inheritance Hierarchy: Inheritance hierarchy is a tree-like structure that represents the relationship between classes. A class can inherit properties and behaviors from its parent class, and it can also have its own subclasses that inherit properties and behaviors from it.
8. Interfaces: Interfaces are used to define a set of methods that a class must implement. Interfaces are used to define a contract between a class and its clients, and they are used to ensure that a class implements a set of methods that are common to all classes in a particular category.
9. Abstract Classes: Abstract classes are classes that cannot be instantiated. They are used to define a blueprint for a class, and they can contain methods that are intended to be overridden by subclasses.
10. Final Classes: Final classes are classes that cannot be subclassed. They are used to define a class that cannot be modified or extended.

These are the basic concepts of object-oriented programming. Of course, there are many more advanced concepts and techniques that can be used in OOP, but these are the fundamental building blocks upon which all other concepts are based.

Copyright

Copyright(C) 2025 Advanced Micro Devices, Inc. All rights reserved.