Docs for Reliable Messaging

modules/ROOT/nav.adoc

@@ -87,6 +87,7 @@
             **** xref:Development/Satisfactory/FactoryLegs.adoc[Factory Legs]
             **** xref:Development/Satisfactory/AbstractInstance.adoc[Abstract Instances]
             **** xref:Development/Satisfactory/ConditionalPropertyReplication.adoc[Conditional Property Replication]
+            **** xref:Development/Satisfactory/ReliableMessaging.adoc[Reliable Messaging]
             **** xref:Development/Satisfactory/Savegame.adoc[SaveGame]
             **** xref:Development/Satisfactory/IconLibrary.adoc[Icon Libraries]
             **** xref:Development/Satisfactory/DedicatedServerAPIDocs.adoc[Vanilla Dedicated Server API]

modules/ROOT/pages/Development/Satisfactory/Multiplayer.adoc

@@ -264,6 +264,12 @@ xref:Development/Satisfactory/ConditionalPropertyReplication.adoc[Conditional Pr
 a system custom written by Coffee Stain to reduce the amount of unnecessary data sent to clients.
 See the linked page for more information on how to replicate inventory components.
 
+[id="ReliableMessagingReplication"]
+== Replication with Reliable Messaging
+
+Since the 1.1 release you can use xref:Development/Satisfactory/ReliableMessaging.adoc[Reliable Messaging] 
+to replicate large amount of data while bypassing unreal engine's build in replication 
+
 == Replicated Maps
 
 For unknown reasons, Unreal does not provide systems that allow TMaps to be replicated.
@@ -279,6 +285,7 @@ There are multiple approaches you can implement yourself to work around this:
   Such an approach is most certainly not for the faint of heart, though.
   If your map is updating so often that the overhead of converting it to/from an array is important,
   reconsider if you really need to replicate all that data, and if you would encounter network problems first.
+* Maps can also be replicated by a custom implementation using link:#ReliableMessagingReplication[Replication with Reliable Messaging]
 
 Note that replicating one array of keys and one array of values is not suggested
 because changes to each array are not guaranteed to arrive at the same time.

modules/ROOT/pages/Development/Satisfactory/ReliableMessaging.adoc

@@ -0,0 +1,232 @@
+= Reliable Messaging
+
+[NOTE]
+====
+This page assumes you already have working knowledge of Unreal Engine's replication system.
+
+Read the xref:Development/Satisfactory/Multiplayer.adoc[Multiplayer]
+page for information about Unreal's replication system and special cases for Satisfactory.
+====
+
+Reliable Messaging was introduced in the 1.1 release
+and is a Coffee Stain custom implementation that completely bypasses Unreal's normal replication system.
+It sends messages directly to clients over an TCP connection, this comes at the cost of a slightly higher latency, 
+But this has the advantage that the developer has fine grained control over the contents of the message send and 
+it wont ever trigger the Reliable Buffer Overflow.
+
+The purpose of the system is to replicate large amounts of data without hitting Unreal Engine's its limits.
+Example use cases are for replicating foliage removals, recipe and schematic managers.
+
+== Replicating Data
+
+=== Setting up the build dependency
+
+To use the system to replicate data, we must first make it available in our cpp code
+we need to add `"ReliableMessaging"` to our `PublicDependencyModuleNames` in the `YourModReference.Build.cs`
+
+=== Choosing a channel Id
+
+Reliable Messaging supports up 255 different channels for messages for different systems, 
+Its important that the channel ids do not overlap, even across different mods and base game.
+Else your client will receive data that it does not know how to parse, so be creative with choosing your channel id.
+In any case choose a channel id higher than 50 so it will not conflict with https://github.com/satisfactorymodding/SatisfactoryModLoader/blob/master/Source/FactoryGame/Public/FactoryGame.h[channel ids used by the base game].
+You can define your channel id in your header like
+
+[source,c++]
+----
+constexpr uint8 RELIABLE_MESSAGING_CHANNEL_ID_FOR_MY_MOD = ; //choose your own number
+----
+
+=== Deciding the message to transfer
+
+For this example we will assume we want to send and array of fictional player infos.
+We will need to define our custom data struct `FPlayerInfo`, the message struct and the message id.
+The message id allows you to give different messages different ids so they can all be send over the same channel.
+It will look something like this
+
+[source,h]
+----
+// some custom data struct (can be USTRUCT(BlueprintType)) if desired
+struct ARCHIPELAGO_API FPlayerInfo
+{
+	FString Name;
+	FString Job;
+
+	FPlayerInfo(FString name, FString job) : Name(name), Job(job) {}
+	FPlayerInfo() : FPlayerInfo(TEXT(""), TEXT("")) {}
+	FPlayerInfo(const FPlayerInfo& Other) : Name(Other.Name), Job(Other.Job) {}
+};
+
+// message id, to distingush between different messages
+enum class EModPlayerInfoMessageId : uint32
+{
+	PlayerJobs = 0x01
+};
+
+// the message struct to send
+struct FModPlayerInfoJobsMessage
+{
+	static constexpr EModPlayerInfoMessageId MessageId = EModPlayerInfoMessageId::PlayerJobs;
+	TArray<FPlayerInfo> PlayerInfos;
+
+	friend FArchive& operator<<(FArchive& Ar, FModPlayerInfoJobsMessage& Message);
+};
+----
+
+We would also need to implement some operator overloads to the `<<` operator 
+so that the compiler knows how to serialize our custom struct
+
+[source,c++]
+----
+// serializer for our custom data struct
+FArchive& operator<<(FArchive& Ar, FPlayerInfo& Info)
+{
+	Ar << Info.Name;
+	Ar << Info.Job;
+	return Ar;
+}
+
+// serializer for our message
+FArchive& operator<<(FArchive& Ar, FModPlayerInfoJobsMessage& Message)
+{
+	Ar << Message.PlayerInfos;
+	return Ar;
+}
+----
+
+=== Setting up a handler for receiving data
+
+On the client we want to register a callback to be called when the server sends us messages.
+This is done by retrieving the `UReliableMessagingPlayerComponent` for your local player controller
+and calling `RegisterMessageHandler` on it.
+Its generally a good idea to setup this handler early so you can start receiving your data early.
+For that its recommended to setup this handler in the `BeginPlay()` of your `APlayerController`.
+This can be achieved using a xref:Development/ModLoader/ActorMixins.adoc[Actor Mixin] 
+that overrides the Begin Play event on `BP_PlayerController`
+
+image::Satisfactory/ReliableMessaging/BlueprintMixinTarget.png[Actor mixin target example]
+image::Satisfactory/ReliableMessaging/OverrideBeginPlay.png[Override begin play]
+
+Then from the mixin call into cpp, 
+this can for example be done in locally spawned xref:Development/ModLoader/Subsystems.adoc[Mod Subsystem] 
+or in blueprint function library.
+
+[source,c++]
+----
+void AModReliableMessagingSubsystem::OnPlayerControllerBeginPlay(const APlayerController* PlayerController)
+{
+	// We are local player connected to a server, register the message handler
+	if (!PlayerController->HasAuthority() && PlayerController->IsLocalController())
+	{
+		if (UReliableMessagingPlayerComponent* PlayerComponent = UReliableMessagingPlayerComponent::GetFromPlayer(PlayerController))
+		{
+			PlayerComponent->RegisterMessageHandler(RELIABLE_MESSAGING_CHANNEL_ID_FOR_MY_MOD,
+				UReliableMessagingPlayerComponent::FOnBulkDataReplicationPayloadReceived::CreateUObject(this, 
+                    &AModReliableMessagingSubsystem::OnRawDataReceived));
+		}
+	}
+}
+----
+
+Now we will also need need to implement that `OnRawDataReceived`. 
+Assuming we are going to receive messages like the one defined above, we can implement it like this.
+
+[source,c++]
+----
+void AModReliableMessagingSubsystem::OnRawDataReceived(TArray<uint8>&& InMessageData)
+{
+	FMemoryReader RawMessageMemoryReader(InMessageData);
+	FNameAsStringProxyArchive NameAsStringProxyArchive(RawMessageMemoryReader);
+
+	EModPlayerInfoMessageId MessageId{};
+
+    // this actually writes the id from `NameAsStringProxyArchive` to `MessageId`
+	NameAsStringProxyArchive << MessageId; 
+	if (NameAsStringProxyArchive.IsError()) return;
+
+    // if we support different messages we can switch here to the correct types we expect
+	switch (MessageId) 
+	{
+        case EModPlayerInfoMessageId::PlayerJobs:
+        {
+            FModPlayerInfoJobsMessage JobsMessage;
+
+            // this actually writes the message from `NameAsStringProxyArchive` to `JobsMessage`
+            NameAsStringProxyArchive << JobsMessage;
+            if (NameAsStringProxyArchive.IsError()) return;
+
+            HandlePlayerInfoJobsMessage(JobsMessage);
+
+            break;
+        }
+	}
+}
+----
+
+=== Sending the message from the server
+
+If you want to directly replicate some initial data to clients that connect to the server,
+we can reuse same `OnPlayerControllerBeginPlay` that we defined above.
+To do this we will need to extend it to directly send the data to connecting clients.
+
+[source,c++]
+----
+void AModReliableMessagingSubsystem::OnPlayerControllerBeginPlay(const APlayerController* PlayerController)
+{
+	// We are Server, and this is a remote player. Send descriptor lookup array to the client
+	if (PlayerController->HasAuthority() && !PlayerController->IsLocalController())
+	{
+        TArray<FPlayerInfo> PlayerInfos
+        PlayerInfos.Add(FPlayerInfo(TEXT("Sander"), TEXT("Rouge")));
+        PlayerInfos.Add(FPlayerInfo(TEXT("Henk"), TEXT("Warrior")));
+        PlayerInfos.Add(FPlayerInfo(TEXT("Melisa"), TEXT("Mage")));
+
+        FModPlayerInfoJobsMessage JobsMessage;
+        JobsMessage.PlayerInfos = PlayerInfos;
+
+        SendRawMessage(PlayerController, JobsMessage);
+	}
+}
+----
+
+And the actual implementation of `SendRawMessage` will look something like this.
+
+[source,c++]
+----
+void AModReliableMessagingSubsystem::SendRawMessage(const APlayerController* PlayerController, FModPlayerInfoJobsMessage Message) const
+{
+	TArray<uint8> RawMessageData;
+	FMemoryWriter RawMessageMemoryWriter(RawMessageData);
+	FNameAsStringProxyArchive NameAsStringProxyArchive(RawMessageMemoryWriter);
+
+	NameAsStringProxyArchive << Message.MessageId;
+	NameAsStringProxyArchive << Message;
+
+	UReliableMessagingPlayerComponent* PlayerComponent = UReliableMessagingPlayerComponent::GetFromPlayer(PlayerController);
+	if (ensure(PlayerComponent))
+	{
+		PlayerComponent->SendMessage(RELIABLE_MESSAGING_CHANNEL_ID_FOR_MY_MOD, MoveTemp(RawMessageData));
+	}
+}
+----
+
+You are in full control of whether or not to send data to certain player controllers, 
+and it doesn't have to be some initial data, you can call the `SendMessage` whenever you like.
+If for example you do want to send a message to all players but you don't directly access to their controllers
+you can use and actor iterator like this.
+
+[source,c++]
+----
+    for (TActorIterator<APlayerController> actorIterator(GetWorld()); actorIterator; ++actorIterator) {
+        APlayerController* PlayerController = *actorIterator;
+        if (!IsValid(PlayerController) || PlayerController->IsLocalController())
+            continue;
+
+        SendRawMessage(PlayerController, Message...);
+    }
+----
+
+=== Example implementations
+
+ * On the Satisfactory modding discord https://discord.com/channels/555424930502541343/1036634533077979146/1463650672137212114[the .cpp code] for the `UFGConveyorChainSubsystemReplicationComponent` was shared. 
+ * There is also an https://github.com/Jarno458/SatisfactoryArchipelagoMod/blob/main/Source/Archipelago/Private/Subsystem/ApPlayerInfoSubsystem.cpp[open source implementation] in the https://ficsit.app/mod/Archipelago[Archipelago] mod.