Beamable SDK Review – December 2022

In December, Beamable released SDK versions 1.7.0, 1.8.0, and 1.9.0, and 1.9.1. Here are the highlights from the previous month of releases. 

Microservice Performance

Microservices received many performance improvements in December. Previous to Beamable 1.7, Microservices logged all inbound and outbound messages, including the request bodies and headers. The volume of log messages for high-scaled services was degrading performance, so going forward, Microservices no longer emit logs for sending and receiving messages. Additionally, logs do not automatically include the request body or header information. Of course, these can be added manually to logs on a case-by-case basis. Regarding logs, Beamable 1.7 introduced limits on max log size. Nested objects are only destructured to a depth of 3. Only 10 objects will be serialized in a collection. The max character size of a destructure operation is 500. These logging changes reduce the memory footprint of deployed Microservices which allows them to run more efficiently.

In Beamable 1.7, each Microservice instance opens multiple WebSocket connections to Beamable. Maintaining multiple connections allows the service to spread the network load across several Beamable Gateway instances, which improves reliability and performance. It is expected that each WebSocket connection may expire, but with multiple connections available, a single connection loss won’t cause a major service outage.

Due to the logging changes, the multiple connection changes, and a variety of smaller improvements to memory management and garbage collection, the overall performance characteristics of Beamable 1.7 are much better than 1.6.2. We used Locust to simulate a stress test on the two Beamable versions. In the stress test, we experimented to see how many requests a Microservice could handle before it started experiencing 504 HTTP timeout errors. 

The key takeaway is that Beamable 1.7 never encountered latency above 2.5 seconds while 1.6.2 had p95 latencies of around 4 seconds and experienced 504 HTTP timeouts due to response times exceeding the 10-second timeout period.

Prefetch Content

In Beamable 1.8, Microservices prefetch all Beamable Content before accepting user traffic. When the service starts up, it will download the entire content manifest. Subsequently, any request to fetch content will be handled by the content cache. 

Cancellation Token

Finally, In Beamable 1.9, we introduced a CancellationToken accessible on the Context property in each [ClientCallable] function. Consider the code snippet below, 

[ClientCallable]
public void SpinForever()
{
  while (true)
  {
     // do nothing.
  }
}

The SpinForever function will never complete due to the non-terminating while loop. This is extremely bad for performance and efficiency, especially in a deployed environment. Every [ClientCallable] function is run in its own Task instance. In dotnet, many Tasks are scheduled per Thread, and then Threads are scheduled by the underlying operating system. If a single Task keeps running, then any other Task scheduled on the same Thread will never get a chance to execute. Essentially, the non-terminating while loop kills an entire Thread instance. This reduces the performance of the service and also causes the CPU to spike to 100%. If you see your deployed service CPU at 100%, you should double-check your code for non-terminating loops or recursion. When the CPU is pinned at 100%, it will cause additional server instances to spawn in the Beamable Cloud. Unfortunately, those new instances will be susceptible to the same underlying issue, and will likely also get pinned at 100% CPU.

In Beamable 1.9, you can change the above code to this snippet. The ThrowIfCancelled function will throw a TaskCancelledException after 20 seconds, and cause the Task to finish. There is also a IsCancelled boolean available on the Context that will return true when the request has timed out.

[ClientCallable]
public void SpinForever()
{
  while (true)
  {
     // do nothing.
     Context.ThrowIfCancelled();
  }
}

Editor Performance

Beamable 1.8 and 1.9 include several performance improvements for the Unity Editor experience, specifically concerning Content and Microservices. From our internal testing using the Editor Iteration Profiler, we saw a 20% speed improvement for domain reloads in a project with 2.5 thousand content objects, and about a 30% speed improvement in projects using several Microservices. 

In both cases, these improvements can be attributed to fewer memory allocations and Reflection operations. Unfortunately, in a few critical places in the Beamable SDK prior to 1.8, System.Linq commands were causing a lot of memory allocation which led to large garbage collection passes. Additionally, we identified several opportunities to cache information instead of recomputing it every domain reload. 

Visual Changes

In Beamable 1.7.0, we continued moving the Beamable tool windows to a modern flat User Interface design. The Content Manager and Toolbox have refined interfaces that make it easier to recognize key features. 

Additionally, in Beamable 1.8, the default avatars have been replaced with grey-box user icons. If you are upgrading from a previous Beamable version, you will continue to see the old avatars until you delete the Assets/Beamable/Resources/AvatarConfiguration file and it is regenerated.

Local Content Mode

Beamable 1.7 introduced Local Content Mode. From the Project Settings, you can enable Local Content Mode, which changes all content fetching in PlayMode to read content from your local disk instead of downloading it from the Beamable Cloud. This can help speed up your workflow when making subtle content changes to custom content objects. 

Change Logs

For a complete list of changes, please review the at https://beamable.github.io/changes