Thursday, 18 August 2016

Real-Time Java: An Introduction Part - 1

I've worked in investment banking for several years and my experience suggests that most problems in financial software arise from the lack of real-time support. Many large financial IT systems use a Java platform, and even an unplanned two-second stop for full garbage collection in an application can cause a loss of tens of thousands of dollars. The situation is worsened by the fact that garbage collections usually occur during a high-application activity, when the app is especially sensitive to any breaks in execution. The same situation occurs in other high-tech and production industries, which is why they are taking a careful look at real-time Java specification and its implementations.
Java and real-time--some might think that these words are from separate contexts, but actually one of the oldest JSRs (JSR number 1, to be exact) is about a real-time extension for the Java platform. However, order of submission does not guarantee order of implementation. The fact that Sun implemented it only recently does not mean it's a low-priority feature; it's actually a very complicated and thorough piece of work. But are real-time requirements compatible with the Java religion? Many issues must be addressed, such as GC semantics, synchronization, thread scheduling, and high-resolution time management. In this article, I will shed some light on them.

What does "real time" mean?


According to Greg Bollella, a distinguished engineer at Sun Microsystems and one of the authors of the real-time Java specification, real time means "the ability to reliably and predictably reason about and control the temporal behavior of program logic." Real time does not mean "fast," as many developers might think; it means predictable and reliable when a reaction to real-world events is required. The real-time computer will always respond before a particular deadline that you've assigned it. Depending on how you set your deadlines, numerous systems could be called real-time.

A large domain of applications cannot afford delaying even for one second; this domain includes, as mentioned earlier, many financial applications, on-board aircraft software, nuclear power plant software, and more. So this is all not about speed, although real-time platform designers did their best to make it fast. Obviously, the standard Java platform does not meet the needs of real-time systems, and this is addressed in the license agreement accompanying J2SE and J2EE packages, which explicitly states that Java cannot be used in nuclear power plant software, defense systems, etc.

Real-time Java


Real-time application development requires an API set and semantics that allow developers to correctly control the temporal behavior of application, i.e., how it will behave in real-world time. A real-time edition of Java must therefore provide some semantic JVM enhancements and a new API set appropriate for real-time applications. It is not surprising that the main obstacle in achieving real-time characteristics for Java is its garbage collector. A real-time garbage collector became a revolutionary and central component of Sun's recently-released Java real-time edition RTS 1.0, although its first implementation does not include one (it is expected in the next release). Java RTS addresses other issues, making strong deterministic guarantees for thread scheduling, synchronization overhead, lock queuing order, class initialization, and maximum interrupt response latency. Java RTS is intended only for suitable underlying operating systems, which means that only a real-time operating system, such as QNX, is appropriate for implementing the JVM.

The first official commercial implementation of the real-time Java specification is implemented for Solaris 10, works on UltraSparc hardware, and requires a J2SE 1.4.1 as a base. Future releases will support Java 5 and some other platforms. Sun Java Real-Time System is already being used by the U.S. Navy, Raytheon, and Boeing. Of course Sun Java RTS 1.0 is not the first real-time implementation of Java; some real-time features were implemented previously by embedded systems vendors, but their implementations covered only specific needs and did not conform to JSR-1 specification. The Java RTS is good news for developers who use Java as a main platform for any reason, but need a real-time JVM.

Okay, this all sounds good, but what does it mean from a developer point of view, and what will be required to adapt existing applications to an RTS API? Will it allow us to get rid of the main problem--unpredictable garbage collection pauses? Unfortunately, I must disappoint you: it's not that easy. Simply installing an RTS extension and renaming java.lang.Thread instances to javax.realtime.RealtimeThread will not turn the application into a real-time app.

However, this is a good start; with that change you will get a revolutionary real-time garbage collection. It's also worth mentioning that existing J2SE application will successfully run under Java RTS because the RTSJ specification only restricts the semantics of the JLS and JVM specification to a subset; it does not allow syntactic extensions which could break existing applications.

To make the real-time garbage collector predictable, you should understand how your program demands memory from the heap, because the garbage collector and your program both use it. The program makes garbage, the garbage collector turns garbage into free memory, and they have to interact in the heap. Since they can't do that at the same time, there must be some "synchronization" between the two. Thus, you have to tell the garbage collector something about how fast your program will create garbage so it knows how fast it must collect it. Getting that number can be somewhat tricky, but you must consider memory, no matter what you do.

If running in RealtimeThread is not enough, and after reworking the code, the GC pauses are still too long or unpredictable, you might use an execution context other than RealtimeThread, such as javax.realtime.NoHeapRealtimeThread. It achieves predictability by using memory other than the regular Java heap, such as immortal and scoped memory, which will be discussed below. Getting predictability surely requires trade-offs; typically, you sacrifice average throughput performance.

What's new in Java RTS?


Let's examine what's new in Java RTS platform.
  • Direct memory access. Java RTS allows direct access to physical memory, making it similar to J2ME. No surprise there: one of the main target platforms of real-time Java is embedded systems. This means that now you can create device drivers written in pure Java. Although memory access is not directly a real-time issue, physical memory access is desirable for many applications. Java RTS defines a new class that allows programmers byte-level access to physical memory, as well as a class that allows the construction of objects in physical memory. One might think that physical memory access is the point where Java gives up its main principles--reliability and safety--and takes a step back towards C, but this isn't the case. Java maintains strong security protections by controlling memory bounds and data contents.
  • Asynchronous communications. Java RTS provides two forms of asynchronous communication: asynchronous event handling, and asynchronous transfer of control. Asynchronous event handling means the developer can now schedule the response to events coming from outside the JVM. Asynchronous transfer of control provides a carefully controlled way for one thread to interrupt another thread in a safe manner.
  • High-resolution timing. There are several ways of specifying high-resolution time including absolute and relative time. A nanosecond accuracy is available for time scheduling and measurements.
  • Memory management. There are two new types of memory areas that help prevent unpredictable delays commonly caused by traditional garbage collectors in real-time applications: immortal memory and scoped memory. Immortal memory holds objects without destroying them, except when the program ends. This means that objects created in immortal memory must be carefully allocated and managed, as with C programs. Scoped memory is used only while a process works within a particular section, or scope, of the program (such as in a method). Objects are automatically destroyed when the process leaves the scope. Neither immortal nor scoped memories are garbage collected, so using them avoids problems of GC interference. The Java RTS also provides limited support for providing memory allocation budgets for threads using memory areas. Maximum memory area consumption and maximum allocation rates for individual real-time threads may be specified when the thread is created.
  • Real-time threads. As mentioned previously, Java RTS supports two new thread models: real-time threads (javax.realtime.RealtimeThread) and no-heap real-time threads (javax.realtime.NoHeapRealtimeThread). Both thread types cannot be interrupted by garbage collection. These thread classes have 28 levels of priority and, unlike standard Java, their priority is strictly enforced. Real-time threads are synchronized and are not subject to so-called "priority inversion" situations where a lower priority thread has a block on a resource needed by a higher priority thread and thus prevents the higher priority thread from running. Thorough testing has proven that Java RTS completely avoids any priority inversions, which is crucial for mission-critical applications.

How it works


Let's take a brief look at how easily a programmer can use the new real-time Java features. We'll consider only the most interesting parts of the new API: threads and memory.

One of the main goals of the specification authors was to keep RTS programming simple, despite real-time problem complexity; that means the operating system must handle as much work as possible, leaving the programmer only the challenging task of real-time application design.

Threads


The RealtimeThread class extends java.lang.Thread. It has several constructors, giving the developer an opportunity to tune thread behavior:

public RealtimeThread()
public RealtimeThread(SchedulingParameters scheduling)
public RealtimeThread(SchedulingParameters scheduling,
                      ReleaseParameters release)

The ReleaseParameters and SchedulingParameters provided to the RealtimeThread constructor (as well as MemoryParameters, used by a non-spec constructor) allow the temporal and processor demands of the thread to be communicated to the system. RealtimeThread implements the Schedulable interface. The key is that Schedulable objects can be placed in memory represented by instances of ImmortalMemory, HeapMemory, ScopedPhysicalMemory, and PhysicalImmortal.

A NoHeapRealtimeThread is a specialized form of RealtimeThread. Because an instance of NoHeapRealtimeThread may immediately preempt any implemented garbage collector, the logic contained in its run() method is never allowed to allocate or reference any object allocated in the heap, or to manipulate the references to objects in the heap. For example, if A and B are objects in immortal memory, B.p is a reference to an object in the heap, and A.p is type-compatible with B.p, then a NoHeapRealtimeThread is not allowed to execute anything like the following:

A.p = B.p;
B.p = null;

Given these restrictions, a NoHeapRealtimeThread object must be placed in a memory area such that thread logic may unexceptionally access instance variables. This is why the constructors of NoHeapRealtimeThread require a reference to ScopedMemory or ImmortalMemory. When the thread is started, all execution occurs in the scope of the given memory area. Thus, all memory allocation performed with the new operator is taken from this given area.

Memory management


We have already noted some memory-related classes. To be more exact, MemoryArea is the base abstract class of all classes dealing with representation of allocatable memory areas, including the immortal memory area, physical memory, and scoped memory areas. The HeapMemory class is a singleton object that allows logic within other memory areas to allocate objects in the Java heap. This method returns a pointer to the singleton instance of HeapMemory representing the Java heap:

public static HeapMemory instance()

A much more interesting class is ImmortalMemory, which is a memory resource that is shared among all threads. Objects allocated in the immortal memory live until the end of the application and are never subject to garbage collection, although some GC algorithms may require a scan of the immortal memory.

A ScopedMemory area is a connection to a particular region of memory, and is a class dealing with representations of memory spaces that have a limited lifetime.

An instance of RawMemoryAccess models a range of physical memory as a fixed sequence of bytes. A full complement of accessor methods allows the contents of the physical area to be accessed through offsets from the base, interpreted as byte, short, int or long data values, or as arrays of these types. If you need float- or double-type access, you should use the RawMemoryFloatAccess class instead. Whether the offset addresses the high-order or low-order byte depends on the value of the BYTE_ORDER static boolean variable in the RealtimeSystem class. A raw memory area surely cannot contain references to Java objects, because such a capability would be unsafe. The RawMemoryAccess class is instantiated with the following constructor:

public RawMemoryAccess(java.lang.Object type,
                              long base, long size)
       throws SecurityException,
              OffsetOutOfBoundsException,
              SizeOutOfBoundsException,
              UnsupportedPhysicalMemoryException,
              MemoryTypeConflictException,
              MemoryInUseException

It is noteworthy that this constructor declares so many possible exceptions types to be thrown. The type parameter is on Object representing the type of memory required. It is used to define the base address and control the mapping.