Skip to content

Add basic low-level operatations for CXL-capable PCI devices #12415

Merged
ardbiesheuvel merged 6 commits intotianocore:masterfrom
gonzoua:cxl-upstream
May 1, 2026
Merged

Add basic low-level operatations for CXL-capable PCI devices #12415
ardbiesheuvel merged 6 commits intotianocore:masterfrom
gonzoua:cxl-upstream

Conversation

@gonzoua
Copy link
Copy Markdown
Contributor

@gonzoua gonzoua commented Apr 8, 2026

Description

The pull request introduced a new CxlIo protocol that provides a basic operations to communicate with CXL-capable devices:

  • Read area of register block of certain type
  • Write area of register block of certain type
  • Perform a DOE transaction: send a request and read a response.

CxlDxe driver implements the protocol and binds it to CXL-capable PCI devices.

Driver code is partially based on two patchsets:

The patchsets' authors are attributed in the commit message.

On the Google's side @NicholasGraves and Ryan Heise did most of the work on the CxlIo design and implementation. The development history was squashed for brevity.

The PR also introduces cxl command for a basic introspection of CXL devices:

  • List all CXL devices
  • Dump CDAT structures for a specified device
  • Breaking change?
  • Impacts security?
  • Includes tests?

How This Was Tested

Build

host$ docker run --rm -ti -v "${HOME}":"${HOME}" -e EDK2_DOCKER_USER_HOME="${HOME}" \
  -v $(pwd):/host -w /host --name edk2-ubuntu-dev \
  ghcr.io/tianocore/containers/ubuntu-22-dev:latest\
 /bin/bash
docker$ export WORKSPACE=$(pwd)
docker$ export EDK_TOOLS_PATH=$(pwd)/BaseTools
docker$ source edksetup.sh]
docker$ make -j$(nproc) -C BaseTools
docker$ build -t GCC -a X64 -p OvmfPkg/OvmfPkgX64.dsc -D FD_SIZE_4MB -D NVME

Test

Start QEMU:

host$ export CODE=Build/OvmfX64/DEBUG_GCC/FV/OVMF_CODE.fd
host$ export VARS=Build/OvmfX64/DEBUG_GCC/FV/OVMF_VARS.fd

host$ qemu-system-x86_64 -drive if=none,id=code,format=raw,file=${CODE},readonly=on \
  -drive if=none,id=vars,format=raw,file=${VARS},snapshot=on \
  -drive file=fat:rw:root,id=fat1,format=vvfat \
  -machine q35,cxl=on,pflash0=code,pflash1=vars \
  -object memory-backend-ram,id=m1,size=1G \
  -object memory-backend-ram,id=m2,size=512M \
  -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=lsa0.raw,size=4k \
  -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=lsa1.raw,size=4k \
  -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
  -device cxl-rp,port=0,bus=cxl.1,id=root_port_cxl1,chassis=0,slot=2 \
  -device cxl-rp,port=1,bus=cxl.1,id=root_port_cxl2,chassis=1,slot=2 \
  -device cxl-type3,bus=root_port_cxl1,memdev=m1,id=cxl-mem0,lsa=cxl-lsa0 \
  -device cxl-type3,bus=root_port_cxl2,memdev=m2,id=cxl-mem1,lsa=cxl-lsa1 \
  -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=10G \
  -display none -nographic

Navigate menu and start UEFI shell, in shell use the following commands

Shell> cxl
    00   0D   00    00
             Vendor 8086 Device 0D93
    00   0E   00    00
             Vendor 8086 Device 0D93
Shell> cxl 0d 00 00
Device Scoped Memory Affinity (DSMAS):
   Type          00
   DSMADHandle   00
   Flags         04
   DPABase       0
   DPALength     40000000
...
Shell> cxl 0e 00 00
Device Scoped Memory Affinity (DSMAS):
   Type          00
   DSMADHandle   00
   Flags         04
   DPABase       0
   DPALength     20000000

Integration Instructions

N/A

@ardbiesheuvel
Copy link
Copy Markdown
Member

Thanks for this contribution.

High level feedback before even having looked at the actual code: MdePkg is meant for protocols and other definitions that are derived from an external (i.e., non-edk2) spec, e.g., UEFI, PI, ACPI, etc. As I understand from the commit log, the CxlIo protocol being proposed here is not defined in such a spec today.

So there are two options:

  • propose/contribute the CXL I/O protocol to UEFI or CXL or another spec (which may take a long time),
  • move the code to MdeModulePkg, and omit EFI_ prefixes for identifiers (use EDKII_ instead)

I think the second option is more appropriate here, and we can always start a parallel track to promote the protocol once it lands in one of the industry specifications.

(Btw no need to go and repaint everything right way - we can take a bit of time to review and discuss the implementation first)

@gonzoua gonzoua marked this pull request as draft April 8, 2026 21:23
@gonzoua gonzoua force-pushed the cxl-upstream branch 5 times, most recently from 839cdf2 to 045be1f Compare April 9, 2026 19:54
@gonzoua gonzoua marked this pull request as ready for review April 10, 2026 13:39
@gonzoua gonzoua force-pushed the cxl-upstream branch 2 times, most recently from 3ee87dd to 198c86b Compare April 11, 2026 13:31
@leiflindholm
Copy link
Copy Markdown
Member

@gonzoua can you please add some comments as to what you're changing with each update?

@gonzoua
Copy link
Copy Markdown
Contributor Author

gonzoua commented Apr 13, 2026

@leiflindholm Do you mean comments for PR iterations (force-pushes)?

Series of force-pushes up until commit 045be1f was a CI faliures clean-up: there was a number of issues with Doxygen comments and implicit int types conversion that caused VS build to fail.

Commit 3ee87dd push - addressed @ardbiesheuvel's comment. I moved the CxlIo protocol to MdeModulePkg and switched it to using EDKII prefix/namespace.

Commit 198c86b push - commit message fix, I forgot to change MdePkg to MdeModulePkg in the previous modification.

@ardbiesheuvel
Copy link
Copy Markdown
Member

@jkolakow any thoughts?

Copy link
Copy Markdown
Contributor

@pierregondois pierregondois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello Oleksandr,
I added some comments to the PR

Comment thread MdePkg/Include/IndustryStandard/Cxl30.h Outdated
Comment thread MdeModulePkg/Bus/Pci/CxlDxe/CxlDxe.c
Comment thread MdeModulePkg/Bus/Pci/CxlDxe/CxlDxe.c
Comment thread MdeModulePkg/Bus/Pci/CxlDxe/CxlDxe.c Outdated
Comment thread ShellPkg/Library/UefiShellDebug1CommandsLib/Cxl.h Outdated
Comment thread ShellPkg/Library/UefiShellDebug1CommandsLib/UefiShellDebug1CommandsLib.uni Outdated
Comment thread MdeModulePkg/Bus/Pci/CxlDxe/CxlDxe.c Outdated
Comment thread MdeModulePkg/Bus/Pci/CxlDxe/CxlDxe.c Outdated
Comment thread ShellPkg/Library/UefiShellDebug1CommandsLib/Cxl.c Outdated
Comment thread ShellPkg/Library/UefiShellDebug1CommandsLib/Cxl.c Outdated
@gonzoua
Copy link
Copy Markdown
Contributor Author

gonzoua commented Apr 29, 2026

Force-push to commit ebd8abd addressed @pierregondois's feedback:

  • Fixed out-of-bound accesses for DVSEC registers read/writes
  • CDAT definitions moved to its separate standards header file CxlCdat.h
  • Added logic to wait for DOE object being ready with timeout
  • Various code clean-ups

@ardbiesheuvel ardbiesheuvel added the push Auto push patch series in PR if all checks pass label Apr 29, 2026
@ardbiesheuvel
Copy link
Copy Markdown
Member

@Mergifyio queue

@mergify
Copy link
Copy Markdown

mergify Bot commented Apr 30, 2026

Merge Queue Status

  • 🟠 Waiting for queue conditions
  • ⏳ Enter queue
  • ⏳ Run checks
  • ⏳ Merge
Waiting for:
  • -closed [📌 queue requirement]
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
All conditions
  • -closed [📌 queue requirement]
  • any of [🔀 queue conditions]:
    • all of [📌 queue conditions of queue rule default]:
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • base~=(^main|^master|^stable/)
      • label=push
      • any of [🛡 GitHub branch protection]:
        • check-success = tianocore.PatchCheck
        • check-neutral = tianocore.PatchCheck
        • check-skipped = tianocore.PatchCheck
      • any of [🛡 GitHub branch protection]:
        • check-success = ArmVirtPkg - Ubuntu GCC - PR
        • check-neutral = ArmVirtPkg - Ubuntu GCC - PR
        • check-skipped = ArmVirtPkg - Ubuntu GCC - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = EmulatorPkg - Ubuntu GCC - PR
        • check-neutral = EmulatorPkg - Ubuntu GCC - PR
        • check-skipped = EmulatorPkg - Ubuntu GCC - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = EmulatorPkg - Windows VS - PR
        • check-neutral = EmulatorPkg - Windows VS - PR
        • check-skipped = EmulatorPkg - Windows VS - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = OvmfPkg - Ubuntu GCC - PR
        • check-neutral = OvmfPkg - Ubuntu GCC - PR
        • check-skipped = OvmfPkg - Ubuntu GCC - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = OvmfPkg - Windows VS - PR
        • check-neutral = OvmfPkg - Windows VS - PR
        • check-skipped = OvmfPkg - Windows VS - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = Windows VS - PR
        • check-neutral = Windows VS - PR
        • check-skipped = Windows VS - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = Ubuntu GCC - PR
        • check-neutral = Ubuntu GCC - PR
        • check-skipped = Ubuntu GCC - PR
      • any of [🛡 GitHub branch protection]:
        • check-success = Validate Pull Request Formatting
        • check-neutral = Validate Pull Request Formatting
        • check-skipped = Validate Pull Request Formatting
      • any of [🛡 GitHub branch protection]:
        • check-success = ArmVirtPkg - Ubuntu - CLANGPDB
        • check-neutral = ArmVirtPkg - Ubuntu - CLANGPDB
        • check-skipped = ArmVirtPkg - Ubuntu - CLANGPDB
      • any of [🛡 GitHub branch protection]:
        • check-success = OvmfPkg - Ubuntu - CLANGPDB
        • check-neutral = OvmfPkg - Ubuntu - CLANGPDB
        • check-skipped = OvmfPkg - Ubuntu - CLANGPDB
      • any of [🛡 GitHub branch protection]:
        • check-success = Ubuntu - CLANGPDB
        • check-neutral = Ubuntu - CLANGPDB
        • check-skipped = Ubuntu - CLANGPDB
      • any of [🛡 GitHub branch protection]:
        • check-success = Windows - CLANGPDB
        • check-neutral = Windows - CLANGPDB
        • check-skipped = Windows - CLANGPDB
      • any of [🛡 GitHub branch protection]:
        • check-success = EmulatorPkg - Windows - CLANGPDB
        • check-neutral = EmulatorPkg - Windows - CLANGPDB
        • check-skipped = EmulatorPkg - Windows - CLANGPDB
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of [📌 queue -> configuration change requirements]:
    • -mergify-configuration-changed
    • check-success = Configuration changed

NicholasGraves and others added 6 commits April 30, 2026 21:56
- Add Data Object Exchange registers to the PCIe headers.
- Add Data Object Exchange bits to the CXL 3.0 headers.

Signed-off-by: Nick Graves <[email protected]>
Signed-off-by: Oleksandr Tymoshenko <[email protected]>
Add CXL CDAT structure definitions to newly introduced CxlCdat.h

Signed-off-by: Nick Graves <[email protected]>
Signed-off-by: Oleksandr Tymoshenko <[email protected]>
Create a CXL protocol for interacting with CXL endpoint devices. CXL
devices are necessarily also PCI devices, so the PCI IO protocol is also
provided as part of the CXL protocol.

The protocol provides access to the following operations for CXL devices:
  - Read from a DVSEC register block
  - Write to a DVSEC register block
  - Perform DOE transaction

This set of operation is sufficient to implement BIOS-level CXL
functionality.

Signed-off-by: Nick Graves <[email protected]>
Signed-off-by: Oleksandr Tymoshenko <[email protected]>
Add CxlDxe driver that provides CxlIo protocol.

Co-authored-by: Abhishek Narvaria <[email protected]>
Co-authored-by: Alok Rathore <[email protected]>
Co-authored-by: Nick Graves <[email protected]>
Co-authored-by: Ryan Heise <[email protected]>
Co-authored-by: Sayanta Pattanayak <[email protected]>
Co-authored-by: Sweta Kumari <[email protected]>

Signed-off-by: Oleksandr Tymoshenko <[email protected]>
Include CXL support in the form of CxlDxe to OVMF package.

Signed-off-by: Oleksandr Tymoshenko <[email protected]>
Add a 'cxl' command to list and query CXL devices.

Signed-off-by: Oleksandr Tymoshenko <[email protected]>
@gonzoua
Copy link
Copy Markdown
Contributor Author

gonzoua commented May 1, 2026

How do I trigger mergify re-evaluation? It seems to wait for all the review threads to be resolved but there seems to be no unresolved ones.

@ardbiesheuvel
Copy link
Copy Markdown
Member

I'll just merge this one by hand - mergify has been playing up recently.

@ardbiesheuvel ardbiesheuvel merged commit 68662fd into tianocore:master May 1, 2026
161 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

push Auto push patch series in PR if all checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants