Multiplayer, Bevy 0.18, and Fosdem

Overview

This month was basically solely focused on adding multiplayer functionality with a small break for a bevy 0.18 migration.

If you find this interesting and find the game interesting I would encourage you to give the game a wishlist on steam. It helps a ton.

I will say that I regret not starting the multiplayer code earlier. Like internationalization, and saving, multiplayer is something that should be architected for from the beginning. The good news is that the code was clean enough that it wasn't too hard to refactor.

In other news migrating from 0.17 to 0.18 was pretty painless. I had to drop a few more less maintained crates, but the core migration itself was simple.

Lastly, I took the time to head out to Fosdem to hang out with the Bevy community. While I didn't give a talk Thierry Berger gave an excellent one on mobile bevy game development.

Multiplayer

The game supports simultaneous Direct Connect, LAN, and Steam multiplayer hosting and joining.
The game supports simultaneous Direct Connect, LAN, and Steam multiplayer hosting and joining.

For people writing games in Bevy I had been giving the advice to consider your internationalization, gamepad, and save stories early. I now regret that I missed likely the most important factor to consider early this being multiplayer and netcode.

So with that my updated list of things to consider early are your:

  1. Multiplayer / Netcode Story
  2. Gamepad Story
  3. Save Story
  4. Internationalization Story

Adding these later will be annoying and painful. (Ask me how I know! /s)

Exofactory Specifics

Exofactory is a Factory game where the player plays as an AI who interacts with the world via little robots called "Exoframes" that must be built by the player. While not occupying an Exoframe the player can zip around their factories in the global camera mode.

This leads to a sort of natural co-optative approach where players can join the game in the global mode where they can look around, and can configure existing machines. Once there is an Exoframe that is not being occupied the new player can "inhabit" it building and exploring the world beyond the factory sensor range.

This leads to a nice sort of multiplayer metaplot while at the same time providing a clear technical story about what must be done in the netcode these being:

  1. Interact with the world in Global mode including building interaction.
  2. Selecting and controlling one of a Pool of Exoframes.
  3. Running the rest of the world to update the relevant components.
  4. Ensuring the client sees the world in an accurate way.

Personally, I think this naturally leads to a server authoritative model. The server instance of the game runs the world, and the client instances just participate in it.

Picking a Library

At the time on Bevy 0.18 there seem to be two well supported crates these being bevy_replicon and lightyear

I encourage you to take a look at both as both have their unique advantages. In my (potentially flawed) understanding, lightyear adds more features like client side prediction, deterministic replication and interest management (only syncing relevant data), at the expense of initial overhead and complexity, while replicon omits these features by default, but is simpler to setup with less dev time overhead. If my understanding is wrong here I would be happy to correct this if anyone reaches out.

For Exofactory, given that the game is definitively NOT a twitchy shooter, and my planned pace of feature development, and my relative newness to game dev net code I decided to go with the simpler to set up replicon.

This all being said, I suspect that if your game architecture is already well structured switching later would not be a painful process.

From Single Player to Multi

At the heart of it replicon does two things. Replicate components, and handle network events/messages to/from clients.

In order to enable multiplayer I had to refactor several systems and logically group them.

Exofactory Multiplayer Architecture

Authoritative Core Systems

These only run on the server instance and update the world (all the components that store game state) for everything not directly manipulated by the player.

This includes things like the machine cycle that turns iron ore into iron ingots, what sounds should be played, and even the rate of mining that any exoframe is doing while the player watches. There are many many more systems but these are the type that run in the group.

These systems are allowed to write to components directly. Replicon automatically replicates these changes to the client instance of the components.

Player UI Systems

These run only on game instances where a player plays. These play the sound, show the building UIs, and let the player know the state of the world. These systems read the state of the world by reading the local components that are kept in sync automatically.

The part that required major refactor work is that in these systems no direct manipulation of components is allowed. The clients must send an event/message to the server which then manipulates the components on behalf of the client and updates the components which are replicated to the client.

Network Messages & Events

In the above diagram I only included one example, but in reality basically everything you do in the game beyond "looking at stuff" happens via event or message. This includes the player on the server instance of the game.

This I would say is the MOST important factor to consider when working on a bevy game that might eventually support multiplayer. If I had done this from the beginning adding the multiplayer functionality would have been easy. Even if you don't want to think about networking at the beginning, just ensuring all player manipulations of the world go through the native bevy event systems or message systems will save you time later.

Ramifications & Outcomes

Exofactory Multiplayer Architecture Headless

There were two sorts of artifacts that came out of this. The first one, was something that I was planning on working on later, but due to said refactoring became much easier. That being a headless server instance that the player can self host in docker/podman etc. This would allow the factory to continuously run without anyone needing to keep the game running on their PC.

With the refactoring and a new feature based build of the game that uses MinimalPlugins instead of DefaultPlugins we get a clean lean headless server instance of the game that can run basically anywhere including most RISC-V SBCs!

This still needs testing but things are looking good. Above is a similar diagram but for the headless version of the game.

The second interesting outcome is that I am using way fewer Bevy resources namely because Replicon does not replicate resources. Resultingly I am using way more pseudo-singletons (aka an "empty" entity that just has the component that holds the data the resource used to.)

I am not a huge fan of object oriented design patterns, so it's a bit funny and feels a bit dirty.

Bevy 0.18 Migration

I have seen the light! For real though any Bevy game that has light should prioritize migrating. Major work has been done and the game looks notably better. This being said, even if your game does not use light you should still prioritize migrating.

Overall beyond a few shader bind group things (i.e. BindGroupLayout -> BindGroupLayoutDescriptor) and Atmosphere changes it was a simple migration.

Glad to see the release schedule back on track.

I will say though, with each upgrade I see more and more plugins falling by the wayside. This makes me appreciate the maintainers of plugins even more. Keeping up with Bevy's release schedule is a major task compared to any other game engine.

Fosdem!

Speaking of, I had the privilege to attend Fosdem a libre get together that is hosted in Brussels every year.

If you are interested in open source game engines, or in reality open source anything at all I would highly recommend going. There are a ton of fun and interesting talks, but the best part (in my book anyways) is being able to meet and see and hang out with the people whom you normally only talk to online. I know the Bevy dinner with Thierry Berger, Gee, Niklas Eicker, Firestar99 and several other charming individuals was very fun.

Conclusion

I thought I could get multiplayer done in a week, but it took a month as somehow expected. I am a better game dev for the experience. You can't just depend on back end patterns for everything.

Bevy 0.18 was simple to migrate to.

I strongly recommend going to Fosdem if you have the chance.