From a management standpoint, a mature software development process should not allow that first design to ever be implemented. Another round of simplification should be performed before any code is written. The initial design is not wasted time, as it provides insight to future improvements based on the original designer’s concepts. I’ve found that, working on my own, there is often too little oversight in this regard.

So it was with the content management system for the MMORPG platform I’m developing. Since I’m on my own, I could cope with the inelegant-yet-functional initial design. It had several programming-theoretic benefits. For example, I’m a big believer in strong typing and layering, but to the extreme it adds boilerplate code at each layer of architecture. In addition, (against my own advice) much of the design was overly rigid and complicated for the sake of theoretic (unproven) efficiency gains.

The straw that broke the camel’s back was gameplay integration. Adding and changing game content was a multi-step process to integrate properly with the underlying content system. Due to the aggressive byte-packing strategy, database storage was fragile, requiring a complex process to cope with code changes. Altering the gameplay needed to be more fluid, robust, and simple.

The new content design is fundamentally the same as before, but with a radically simpler implementation. As evidence, I dropped over 12,000 lines of code after removing the legacy content structure from the code base. Granted, not every little thing is fully functional yet, but the client and server operate with basic functionality, at least as well as before. Every line of content integration is simpler, with cleaner behavior. Database storage and network transport are more robust.

As a postmortem to the legacy content system, it provided a solid testbed for developing the features needed to support an MMORPG. It served as a proof-of-concept of the fundamental design principles, which are now more elegantly realized in the new content system. Most importantly, the new system shall usher in a renaissance of gameplay integration.

From a software project standpoint, I had to weigh the consequences of the new design versus maintaining the existing (mostly working) design. Anticipating major ripple effect throughout the code, I decided to take the plunge (pun intended).

From a software design standpoint, I planned out the enhancement to avoid breaking existing systems until the last minute, by implementing the new code separately. I moved old classes to legacy packages for easy identification, for later removal. After the new content implementation was working on its own (granted, only with minor developer testing), it was time to integrate the rest of the game systems. To my body of code, it was like breaking every one of its bones, just to reset them to heal in a better position.

Since dis-integrating the old system, it has taken me about five days of furious coding to re-integrate with the new classes. The code is remarkably simpler, and is even more capable than the previous model. For example, I can more easily manipulate content without fear of destroying the entire database! When the old system is expunged entirely, I expect another steep drop in lines of code.

From a software management standpoint, I should have isolated my changes in a branch, to be merged back to trunk after complete. That would have allowed me to support existing releases while the branch remained broken. Also, that would have allowed me to easily abandon my changes if this whole upgrade when terribly wrong…

… which it might still do. At this point, I can compile my 1659 source files, but I’ve not yet run the game with the changes. In fact, I know of several gaps that will cause the system to crash spectacularly, until corrected. Once I have a working game again, I’ll post an update for Alpha Playtesters to enjoy. With any luck, you won’t notice a single difference.

As an initial disclaimer, I am not entirely discrediting the Java Logging API. From a software engineering standpoint, java.util.logging provides a flexible, well-designed logging system, which is fully capable of providing logging services for a variety of applications. However, I have found it to be several times too complicated in the common case, and I disagree with some of its basic principles.

Although I argue against static design in almost all cases, logging is one situation in which you do not want to be passing an instance reference through all your methods. Languages with aspects, other cross-cutting features, or even multiple inheritance can allow logging to be mixed into classes. Since Java does not (natively) support these language features, a static logging API is the cleanest solution.

I do not fault the static design of the Java Logging API, but I take exception to calling Logger.getLogger(this.getClass().getName()) just to obtain a logging instance. The first design simplification I made was to write my own static logging API to encapsulate the ugly details of obtaining the Java Logger. My logging calls look something like Log.warn(this, "message"), which is somewhat cleaner.

All logging systems I’ve ever used have been based on logging levels, and I’ve never agreed with this concept. Logging messages are generated for different purposes, not at different levels. Severity levels make sense for informational/warning/error messages. Other message types do not fit into this model, such as debug, audit, start, stop, and configuration logging. Furthermore, levels named fine, finer, and finest (all lower than config messages) force programmers to arbitrarily decide how important a log message should be. Java arbitrarily includes exactly seven levels, placing further restrictions on extensibility.

I do not blame the Java Logging engineers for following a well-established design, even though it is faulty, in my opinion. Nonetheless, the second design decision I made was to isolate Java’s logging levels behind my own logging categories (fail, warn, info, init, stop, user, debug). My categories are mapped to Java’s logging levels, which are required to call the underlying Java Logging API. Custom formatters output my category names in place of the Java logging levels.

Registering custom formatters and handlers is another example of good engineering becoming somewhat overbearing in the common case. Based on any criteria, Java logging messages can be reformatted and redirected to different files (e.g., summary, detail, error) and sent over the network. While this is a potentially useful feature, the configuration is more complicated than desired for common use.

The third design simplification I made was to limit my logging to one flat file. If category or message source filtering is needed, grep is a perfectly adequate command-line tool. It is more reliable to post-process a complete log file, rather than rely on code changes to adjust logging output. This simplification alleviated any need for complicated Java handler and filter configuration. Since remote logging is useful, I added a few lines of code to (optionally) open a socket and send log messages to a remote server.

At this point, my logging API provided simple static integration from anywhere in my code, featuring source-named messages with meaningful categories. The complexities of configuring and calling the Java Logging API were all hidden behind my static Log class.

With all this effort heaped upon my code just to avoid reinventing yet-another logging system, two straws finally broke the proverbial camel’s back. For one, I had exhausted the seven available Java logging levels and could not add any additional categories. Even more severe, Java tended to stop logging before the application was finished. Log files lost the last lines of logging, because the logging API stopped outputting at the first signal of application shutdown. There does not appear to be any programmatic control over Java logging shutdown timing.

These issues prompted me to make my fourth, and definitive, simplification to using the Java Logging API; I abandoned it entirely. Since the rest of my code was integrated with my logging facade, I didn’t have to touch any code outside my static Log class. My application now initializes my logging system at startup, which involves opening a file, and stops it at shutdown, which involves closing the file. To preserve the best of the Java logging design, I have a simple formatter interface, which allows custom output. To prevent file output from blocking the application, I implemented a simple thread handoff queue, so all log calls return immediately. Flushing this queue is coordinated as part of application shutdown, so no messages are lost (even in the case of unhandled application exceptions).

In conclusion, I agree with the sentiment of not reinventing the wheel by using available software libraries. However, there is little added value when more time is spent dealing with the complexities, anomalies, and inadequacies of another library. There is a tradeoff, which can be both a strength and a weakness, between learning to integrate a software library versus developing it yourself. Although the Java Logging API is entirely capable, in this case, I would have made better use of time by implementing my own, simplified, logging API in the first place.

Attention Alpha Playtesters: I’ll be testing the new Java version soon, with the goal of officially updating the game’s base Java requirements to this version. Playtesters are encouraged to try the new update and report any issues/improvements.

In the rest of this article, I give my two cents on Sun’s approach to Java thus far, and examine a couple improvements I’ve been eagerly anticipating.

Since the beginning (and to its credit), Java has been a language by developers and for developers. By learning the Java Programming Language, developers can leverage the platform’s capabilities for a multitude of applications, including client/GUI applications, networking, server and enterprise applications, database interaction, server-side Web processing (Servlets), and browser-based applications (Applets). Under this developer-centric approach, Sun has underestimated the frustration experienced by end users trying to install and configure the Java Runtime Environment. Although I applaud Sun for remaining true to its target audience of developers, I see its lack of user attention as the major reason Java is not as prolific as it could/should be.

Take, for example, Java Applets. I’m not a big fan of in-browser applications (especially when abused for purposes of eye-candy), but there are some useful features that can be offered by interactive browser-based widgets. As a Java developer, I could leverage the platform to create fully-featured browser-based applications. Sadly, I have had little confidence that a Java Applet would actually execute properly in a user’s browser. The Next-Generation Java Plug-In promises to improve Applet support.

As a more specific case-in-point, Java Web Start should be the ideal approach for players to install-update-launch the Potential Games, Java-based, in-development MMORPG game client (“with a single click”). My experience has shown JWS to be problematic (to put it lightly) under many common conditions. Since Java includes this integrated deployment capability, I’d prefer to use JWS over a third-party deployment system. Java 6 Update 10 promises to make application deployment more robust and transparent to the end user.

In summary, and in conclusion, I feel that Java has been incubating within its developer-centric shell and is only now beginning to break out into an environment of mass acceptance, where it can realize its full potential.

First, consider the cases in which data serialization (a.k.a. marshalling and unmarshalling bytes) are used on this project. I am serializing objects for client-server network transport, client data cache, and server persistent storage. Without any special care, any class changes could/would break compatibility with old versions of the data. This situation is evident to any Alpha Playtester, as I usually wipe out the database after major updates.

Notice that client-side cache data can be (and is) expunged if any data structure has become incompatible. Also, since I’m using Java Web Start (at least provisionally), I can assume all users are running the latest version of the client software. Therefore, transport compatibility is not a big concern. Nonetheless, other projects may support clients of varying versions. Although I’m not explicitly dealing with this scenario, the technique described below could be used to support forward compatibility.

For the server, serialized data must survive code changes.

The standard advice is to avoid the problem; either maintain yet another external data format, or never change your objects. Since the former is undesirable, and the latter impossible, I’ve attempted to come up with a more robust approach.

Although Potential RPG is (so far) 100% Pure Java, I am not using Java’s built-in Serialization mechanism. Java Serialization is designed (rather elegantly, actually) for convenience at the expense of robustness. Even for core Java classes, the documentation explicitly warns that serialized data is likely to break between Java versions. Using Java’s Externalizable interface is an option, but this still binds the developer to Java’s Serialization mechanics.

The approach I’ve developed is an object packing/unpacking system, in which each serializable class has an accompanying byte-handling Packer helper class. This is functionally equivalent to a class-based Externalizable system (but with a cleaner API, IMHO).

The problem of survivability remains. The one-byte solution is to insert a packing schema version byte before each object’s data. This version is given to the unpacking method when unmarshalling the bytes. The unpacking method can include logic to (a) skip defunct fields and/or (b) set default values for missing fields. This technique provides backward compatibility. (Adapting to future versions is more complex, so I’ll ignore that on this project…)

Under this design, maintaining backward data compatibility for a class is a matter of augmenting the unpacking routine. Notice that the packing routine always produces an updated serialized data structure. Therefore, it is possible to write a routine that upgrades the database in one pass; or, the server can be left to lazily update objects as it encounters them.

For Playtesters, the next Alpha release will impose a mandatory database reset. After that, if I am diligent in my code, the test world should survive future updates.

Further reading: Durable Java: Serialization

I expect everyone to be walking around at Level 16 with Double Battleaxes and Expert Regeneration skill by tomorrow.

… That gives you one hour until tomorrow. Have at it.