Potential Games

In Part 1 of this article, I defined non-blocking transaction processing in the game client. In Part 3, the cheating exploits discussed in Part 2 are addressed. First, I adapt the common database technique of resource locking to the Potential Engine (the MMORPG game server). Then, I introduce the concept of resource isolation, which potentially allows for server-side concurrency and distributed load balancing.

To recap the previous discussion, it may be desirable to process server-side transactions concurrently. However, this may result in various exploits and cheats, if the server follows a faulty parallel processing paradigm.

When many threads may be processing a store of data, it is imperative that transactions are performed atomically. That is, each logical action (even if executing concurrently with other actions) must access and manipulate required pieces of data without interference from other transactions. This is the goal of the concurrency model: to allow actions to execute in parallel without manipulating data in a conflicting way.

The common database approach is to use resource locking. In a relational database, locking occurs on tables, and/or rows, and/or cells. In the Potential Engine, content objects are the resources. In this context, each action knows the content objects it needs to transact the action. The engine obtains an exclusive lock on each before any data is read/manipulated. This ensures no two actions will interfere with each other’s data.

This approach to resource locking, as defined here, is lacking in many details, such as deadlock avoidance. Notice that this technique is very low-level (involving individual content objects). This model also implies a system-wide, centralized database.

Next, I submit for your consideration, a concept of resource isolation. This approach to concurrent transaction processing operates at a higher level than resource locking. In the context of a game server, resource isolation involves defining what content is associated, according to game rules. This forms clusters of content that are mutually isolated. Each cluster can operate in parallel with all other clusters.

For example, an MMORPG may define each cave as a geographic region, in which all gameplay is isolated from the rest of the online world. Then, each cave can be considered a cluster. Minimally, this model allows a game server to process transactions sequentially within each cluster, but process all clusters concurrently. Of course, resource locking can be utilized within each cluster to improve internal throughput.

Furthermore, since the content for a cluster has been isolated, that content can reside in a completely separate processing context. That is, resource isolation also defines a model of distributed data processing. This allows, for example, a game’s data to be managed by a server farm rather than one centralized server/database.

The resource isolation technique is more theoretical, whereas the resource locking approach is a rather well-defined approach. These resource isolation concepts are not unique, but I believe further formalization and definition of the concept is important to understanding the nature of distributed software design. Is that an unusual hobby?

In Part 1 of this article, I defined non-blocking transaction processing in the game client. Part 2 discusses security concerns of concurrent transactions in the game server, which could expose cheating and exploits. A simple server-side sequential transaction approach is described that can fix the security issues. Part 3 will discuss how to safely perform server-side parallel processing, using two techniques.

Security in this context is defined in terms of game rules. The security concern is that players may be able to exploit the transaction model to cheat the rules. For example, suppose two players’ characters attempt to pick up the same item from the world surface. The failure case is that the server processes both transactions at the same time, placing the item into each character’s inventory. Similarly, suppose one player finds a way to send two concurrent requests to pick up an item. This could result in the item showing up twice in inventory. This represents a common item duplication exploit in online games.

To avoid the item-dupe exploit, the server must be more careful how it processes transactions. There are several potential approaches. The simplest technique is to serialize all transaction processing in the server. That is, each transaction is executed sequentially (one-at-a-time). No matter the timing between players picking up the item, the server will accept the first and deny the second.

Notice that the order in which requests (e.g., to pick up an item) are processed depends on client and network timing. This is a fairness concern, rather than a transaction security issue, which is not addressed here. (Solving the fairness problem involves lots of interesting techniques, such as distributed coordinated game time. This would make a fine future discussion.)

The simplest solution is the most secure, and should be used until proven insufficient. If performance testing proves that serial transaction processing is the bottleneck preventing the server from supporting the required number of online players, parallel transaction processing techniques can be employed, under the presumption that concurrency will provide greater transaction throughput. Concurrency always results in a multitude of really interesting (read: tricky) issues to deal with.

In Part 3, I discuss safe server-side concurrent transaction processing, using resource locking and resource isolation.

Looking for exciting new gameplay? Sorry, this month is a busy time. I have managed to implement some transaction processing improvements. Consequently, watch out for new client behavioral anomalies!

Specifically, I’m testing the client with non-blocking transactions. Previously, client processing would block (stall) to complete a transaction with the server. A transaction (round-trip client-server request-response message) is needed for almost all game interaction (walking, inventory control, combat). Transactions are designed to be quick, but interaction could become choppy under high network latency (lag).

Non-blocking mode allows the client to perform transactions concurrently and without waiting for the server. This prevents stalling the client, but has the side effect of delayed updates. For example, when dropping an item, blocking-mode would fully complete the transaction before the client became responsive again (the item would be removed from inventory and show on the world surface). In non-blocking mode, you can drop an item and keep right on playing, but the item may linger in your backpack a bit before appearing on the ground.

User interface smarts could hide some of this out-of-sync behavior. For example, when you drop an item to the ground, the client’s user interface could shade (and block use of) the item in your backpack until the transaction actually completes. In the event the transaction fails (e.g., something blocks the drop location), the user interface would relinquish control over the item.

These timing issues bring up stability and security concerns. In Part 2 of this article, I’ll discuss how the server deals with transactions to ensure stability and prevent cheating exploits.