How to approach ensuring some simulation "determinism" in a dynamic level?

[adamwych]

Hi, I'm currently working on a game project, where the player places various physical objects in order to move a ragdoll puppet through a level. The level is hand-made and half-finished - the player is not allowed to move props, that the level designer has put there, but they can add new ones.

I'm having trouble figuring out how to ensure, that the level can actually be finished.

In my tests any slight change might cause the puppet to completely change its flight path, even if that change does not directly intersect its flight path. See video for demonstration.

puppetqa.mov

Here, in the second run, I added a bush, that the puppet did not hit and yet it changed its path.

Basically - the designer had created a certain path for the puppet to go through, tested it on their PC and it was OK. Then, a player adds a (seemingly) irrelevant prop somewhere, which completely changes the flight path and thus the level can no longer be finished.

I understand that actual determinism depends on inputs being exactly the same, and adding a new body breaks that rule, but it seems to me like such change to the flight path might actually be an error on my side.

I'm using Godot, and I've already ensured that the entire physics system (physics space in Godot terms) is recreated on every run and all bodies are recreated in the new space. From my understanding - in those two runs in the demo above, from the perspective of the physics system there are no differences other than the additional static body.

I would greatly appreciate any tips and info whether what I'm trying to do here is practically achievable or not.

[jrouwe]

Jolt in godot is not compiled with JPH_CROSS_PLATFORM_DETERMINISTIC and I'm not sure if it uses fused multiply add (FMA) instructions. This means it's not deterministic across different hardware.

You'd still expect determinism on the same PC. You have to guarantee that bodies / constraints are created in the same order (the body ID is used to sort). You need ensure that the physics space is completely destroyed between runs (just clearing it is not enough). If you don't, there will be caches that survive that affect determinism. I don't know how you recreate the space, but that could be the problem.

[adamwych]

Thanks for the answer. I forgot to mention, but I'm already using a custom Godot build where I added JPH_CROSS_PLATFORM_DETERMINISTIC, -ffp-model=precise and -ffp-contract=off as suggested in the docs, but for now the goal is to at least make it run consistently on one platform.

You need ensure that the physics space is completely destroyed between runs [...] I don't know how you recreate the space, but that could be the problem.

I create a completely new World3D, which creates a new PhysicsSystem. Then, Godot automatically sends a notification to all nodes in the tree and they re-create the bodies.

Is there anything that's shared by all PhysicsSystems that I also need to clear?

[jrouwe]

There's another issue: The code uses a hash of the body ID to sort contacts. This means that if you first create the world bodies and then the ragdoll, inserting a table will change the body IDs of the ragdoll. This changes the hash and thus the sorting. This changes the outcome of the simulation. You can work around this by ensuring that you always create the ragdoll first and that any bodies that you add to the level are added last so that all bodies retain the IDs they had previously.

[adamwych]

Okay, yes, I do in fact spawn the puppet as the very last thing - I moved it to be the very first thing, and at first glance it seems to be working reliably.

But I have more dynamic objects in the level, e.g. a ball launcher (at 0:02 in the demo). Those objects can get "activated" at any point - it depends on the level.

Do I understand correctly that I can pre-spawn a pool of all those dynamic objects (e.g. a ball that gets launched), and just put their bodies to sleep until they are needed and this way their IDs will be stable, and thus the simulation should be reliable?

So I would create bodies in this order:

• Puppet
• Dynamic object pool 1#
• ...
• Dynamic object pool N#
• Static objects

[jrouwe]

Do I understand correctly that I can pre-spawn a pool of all those dynamic objects (e.g. a ball that gets launched), and just put their bodies to sleep until they are needed and this way their IDs will be stable, and thus the simulation should be reliable?

I think that should work. But I think godot also allows creating bodies without adding them to the physics space. This will be cheaper than sleeping the bodies. You can create a pool of X bodies in advance and then later change their shape / add them to the world as you need them. This would mean the ball launcher could also launch 2 balls if needed (just take an extra body from the pool). It would also not break levels if your ragdoll needs 1 extra body later in development (again, just take a body from the pool).

[mihe]

But I think godot also allows creating bodies without adding them to the physics space.

While this is true, JPH::BodyInterface::CreateBody won't happen until the body is in fact added to the physics space. A body only exists in JPH::BodyCreationSettings form until then.

Also, since JPH_CROSS_PLATFORM_DETERMINISTIC was brought up, keep in mind that Godot itself uses the platform trigonometry functions, so any input directly or indirectly derived from many of the Basis/Transform3D operations will be tainted from a cross-platform perspective. I'm sure there are more pitfalls like this to be found.

No consideration of determinism (cross-platform or not) was taken during development of the Godot bindings, and Godot itself provides no explicit guarantees about it, so I would be very cautious of basing the playability of your game on it, especially if you're involving Godot's scene tree. There have been (and might still be) cases where sorted containers are keyed with pointers and then iterated over, etc. etc.

[adamwych]

Thanks a lot. It took a bit of work, but I got Windows, macOS, Android and iOS all playing the same level identically.

I went with the Jolt body pooling idea, but had to modify Godot to add a function that allows me to migrate a CollisionObject3D from its original RID (created during scene instantiation) with another one (from the pre-allocated pool) + replace trigonometry funcs in Godot with Jolt's, and also expose those to C# scripts since .NET's built-in float.Sin/Cos/etc. are not deterministic.

I give each prop a budget of 10 bodies and use a permanent, auto-increment ID to select which "slice" of the pool each prop gets. This way, prop with ID 8 will always be assigned the exact same bodies, even if I then remove prop with ID 6.

pmdet1.mov

[jrouwe]

Is there anything that's shared by all PhysicsSystems that I also need to clear?

Not that I'm aware of. But please look at the response above, my guess is that inserting new bodies in the middle is what is going wrong.