VR soft body physics with Nvidia Flex

I gave up on trying out Nvidia Flex in VR using the Unreal Engine a few months ago – entirely due to just not wanting to work with C++ and Unreal (posted on that here). I still had a cooking game idea I wanted to try with soft body physics, so once I saw someone had created a Nvidia Flex plugin for Unity called uFlex, I decided to put a weekend into trying it out.

Really all I had to do was fix some rendering issues to get it drawing correctly in both eyes – then add hand interaction to allow the player to grab particles. To grab particles it just checks what particles are within a certain radius to the controller when the trigger button is pressed. If they are being held it will set their velocity to be the delta position between the particle and where the particle was in relation to the controller (setting the particle positions directly would really mess up the physics).

Soft body physics in VR is incredibly satisfying, but there is no way I can see a viable game using this. Performance is not there yet for any large amount of particles. And VR games already need to be pretty minimalist for the best performance. I think I recorded these videos running at 45 FPS – which is nausea inducing for me (despite spending a lot of time in VR I am still easily motion sick from bad experiences, something I want to keep so I don’t make bad experiences myself)

Even ignoring the performance issues, every small game test idea I started to work on eventually hit a wall. I wasn’t intending to make a product out of this, but I at least wanted something I could distribute free online as a test demo. At first my plan was to make a quick cooking game – throw some bacon, butter, pancake batter – basically a bunch of different materials with varying hardness and viscosity on a pan, then slosh it all around as it cooks. I couldn’t even get this to work since there was no continuous collision detection on the particles – so if you picked up the pan and quickly moved it (which would happen a lot) the particles would all fall out of the pan.

What I really want to see is mesh deformation like in this demo video – but when that can actually be used in a VR game, and run well, I have no idea