Use Vcpkg manifest mode and add arm64 mac support

[ssunday]

No Mac CI job yet, will raise in subsequent PR.

Mac dependencies do build a functioning OpenMW that runs the game.

MacOS specifics taken from https://gitlab.com/OpenMW/openmw-dep

New Notes:

This raises the target to MacOS 15

Also the 7z contains more than just what we'd need—I'm guessing windows is the same way with the raw export. One day we could maybe trim what we store.

[ssunday]

Ended up adding mac CI

[ssunday]

My fork's CI pushed it:

https://gitlab.com/ssunday/openmw-deps/-/blob/vcpkg-4fd9ce9684a4fe2760119a5db91d0f51a18880cc/macos/vcpkg-arm64-osx-TEST-manifest.txt?ref_type=heads

[ssunday]

Quick update: I'm working on testing it with the CMAKE toolchain way rather than what I was doing. I'm also realizing I can have this include icu + qt, which reduces the need heavily for brew installed packages on the OpenMW side, so I'm seeing if that can work.

[AnyOldName3]

Qt is huge and slow to build, and we don't get any particular advantage from having it in a precompiled deps package we maintain when upstream provides a perfectly good precompiled distribution and we're never going to want to patch it. For Windows, we use aqt to install mainline Qt to a subdirectory of the OpenMW repo so it's kept separate from any system-wide install, and on-and-off have supported using an existing system-wide install instead. As far as I know, there's nothing forcing us to do anything differently for MacOS. Qt's the poster-child for using VCPKG_POLICY_EMPTY_PACKAGE to just reuse a precompiled version even when building everything else with vcpkg.

[ssunday]

Updates:

• Using vcpkg toolchain seems to cause some runtime problem with luajit for reasons unknown. Using the lua vcpkg works. Someone else was having a similar luajit crash issue. Interestingly, luajit is not OK with arm64 windows...maybe it shouldn't be OK with arm64 osx? I don't know.
• Not using the vcpkg toolchain causes a cyclical dependency warning that doesn't seem to actually cause any problems (it seems to be related to both debug and release deps included in the path). Unless this is a deep concern, I'm going to call this good enough....

[ssunday]

This should be ready to review

[ssunday]

Navmesh tool error with bullet :

Assertion failed: (point.getY() <= m_bvhAabbMax.getY()), function quantize, file btQuantizedBvh.h, line 331.
Process 77926 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
    frame #4: 0x00000001012bd118 libBulletCollision.3.25.dylib`btQuantizedBvh::quantize(this=0x000000012743a080, out=0x000000016fdfd464, point=0x000000016fdfd4d8, isMax=1) const at btQuantizedBvh.h:331:3
   328 			btAssert(m_useQuantization);
   329 	
   330 			btAssert(point.getX() <= m_bvhAabbMax.getX());
-> 331 			btAssert(point.getY() <= m_bvhAabbMax.getY());
   332 			btAssert(point.getZ() <= m_bvhAabbMax.getZ());
   333 	
   334 			btAssert(point.getX() >= m_bvhAabbMin.getX());
Target 0: (openmw-navmeshtool) stopped.

Did not happen with latest build from master.

[ssunday]

I'd like to get this merged in some capacity so we have manifest mode in place so other platforms can mess with it. I can split out just to do the manifest stuff if preferred to that end. I mean, even if we merge this, we don't need to have Mac use it yet if there are kinks...

[AnyOldName3]

There's a decent chance that that error would happen with the existing deps if they were to have assertions enabled, which I assume they don't. I'm not seeing a separate debug and release lib for Bullet in the archive, and I'm pretty sure it defaults to only enabling assertions in debug. Vcpkg, however, will give you both, so if you've built the navmesh tool in the Debug configuration, it'll link with a debug version of Bullet, and you'll see assertions that wouldn't fire before.

[AnyOldName3]

This isn't a review comment, just a note so I'll see it if this line looks weird to me later.

Unlike on Windows, where vcpkg will install its own local isolated versions of tools like these, it explicitly recommends you install these yourself via Homebrew for MacOS.

[ssunday]

There's a decent chance that that error would happen with the existing deps if they were to have assertions enabled, which I assume they don't. I'm not seeing a separate debug and release lib for Bullet in the archive, and I'm pretty sure it defaults to only enabling assertions in debug. Vcpkg, however, will give you both, so if you've built the navmesh tool in the Debug configuration, it'll link with a debug version of Bullet, and you'll see assertions that wouldn't fire before.

Hmm. This makes sense. I'll rebuild in release mode and see if the issue persists. If it does NOT persist (i.e. what you said is the case), that would mean we must always build MacOS in release or else things would break that wouldn't before (i.e. introduce functional regression). Or we fix the underlying problem. I don't know quite why we wouldn't build in release for CI builds by default, esp. if they are the ones used in RC. I don't have any opinion on the current situation vs. what it should be, just saying that it would be a change in build procedure (whether it should happen anyway IDK).

[ssunday]

We build in RelWithDebInfo*

[AnyOldName3]

We'll probably want to upstream the unshield port eventually.

[Assumeru]

Hence "in a VS context" i.e. I think we shouldn't bother to support two versions of Visual Studio. At least long term. Obviously this isn't particularly relevant until we actually decide to start using C++23.

[ssunday]

I am targeting 14.0, seems to be fine. I dropped windows 2019, I can revert if that wasn't the conclusion.

[ssunday]

As the last thing to do before this PR is merged, I want to download the deps from the runner and build with it rather than just the local deps. In case CI setup is missing something....

[AnyOldName3]

If we're dropping 2019 support (and we might not have to - I opened microsoft/vcpkg#49654 after isolating the right PR), then the upload-deps job needs to be updated to not try and upload its artefacts.

When we were originally setting up this repo, we tested it by just downloading the deps zips that the jobs spat out from the summary page (for some reason they're not available from the individual job pages), putting them in the directory that before_script.msvc.sh downloads deps to, and setting the filename in that script to the one of the zip we'd just downloaded. The equivalent should work for MacOS, too.

[ssunday]

Whoops, I can fix that...if we want.

When we were originally setting up this repo, we tested it by just downloading the deps zips that the jobs spat out from the summary page (for some reason they're not available from the individual job pages), putting them in the directory that before_script.msvc.sh downloads deps to, and setting the filename in that script to the one of the zip we'd just downloaded. The equivalent should work for MacOS, too. Yes - that is exactly what I was going to do. I was just declaring that I want to absolutely do that after everything else is approved.

[ssunday]

Can wait and see if Vcpkg maintainers reply with fix.

[AnyOldName3]

The plan was always to have dropped MSVC 2019 support by now anyway. The only spanner in the works is that VS 2026 still comes with MSVC 2022, but seeing as how they're going to stop updating the MSVC 2022 that comes with VS 2022, that's really just a branding thing rather than a change to how they do things. We've got a slight extra problem that we can't support VS 2026 until we support CMake 4.2, though, and we're still stuck with 3.31.x until the recast situation is resolved.

Historically (i.e. since MSVC 2015 froze the ABI and this became possible), we've transitioned between MSVC versions by first changing before_script.msvc.sh to use the new toolset and generator with the old deps, and then we've switched the deps over later when we had to rebuild them anyway, so it would be a change if we led from the deps this time. If not for the recast situation, I could make that point moot by making and testing an MR to switch from supporting MSVC 2019 to MSVC-2022-via-VS-2026 right now.

PS: the CMake docs say the VS 2026 generator will default to the v145 toolset that VS 2026 is bundled with, but it only goes up to v143 in the VS installer, so that means either Kitware know something we don't or Kitware made a mistake in their docs.

[ssunday]

Works from github generated deps, just had to tell it to trust luajit

[ssunday]

seems to work with https://gitlab.com/OpenMW/openmw/-/merge_requests/5136

[AnyOldName3]

This isn't building stuff for AMD64 MacOS, and I'm not sure we've officially decided to drop support for that yet. In principle, it might just be a case of adding an extra triplet to a build matrix and having vcpkg handle all the cross compilation stuff for us (although that precludes forcing the host and target triplets to match - I guess we'd want to put that in the build matrix, too, so it uses a matched triplet for native builds and the default triplet for genuine cross builds).

If it doesn't Just Work™, it might be time to have the do we stop supporting Intel macs discussion.

[ssunday]

AMD64 can be handled in a separate PR and any weirdness dealt with there if anything comes up. We can still have Intel releases using the old deps so it's not a blocker to handle in two separate waves.

[AnyOldName3]

There's not a promote to issue button, so you'll need to manually make issues for things like this not working for Intel-based Macs yet.

[ssunday]

I'm not sure how to clear the cache on Github Actions, doesn't seem to show any caches ATM.