1 Introduction

Most databases in deployment are driven by collections of rows and columns organized into tables. There types of databases are called relational databases, and to retrieve information from them it is necessary to perform a query that returns what is called a record set. A record set contains the rows and columns that are the answer to the query that the operator had asked the database. These records sets are static collections of data that are immutable on the client side.

On the other hand, there is a fundamentally different way of approaching data via objects. Objects are generally more like human ways of thinking, and thus easier to understand. For example a “person” object would represent a person, and one would be able to change the name or age of this object, rather than in a relational database where one cell of a row-column relationship would be updated. Objects have made their presence well known in the development community by the popularization of the C++ programming language and more recently the Java language.

This paper discuses the research ideas and implementation of an object oriented database system that supports pass-through objects via a lazy threaded mechanism that we call Lazy Threaded Object Persistence (LTOP).

1.1 Java and Java Features

Java [5] is a multi-platform, hardware-independent programming language developed by Sun Microsystems. The Java compiler produces byte codes that are designed to run on a Java Virtual Machine. This virtual machine is designed to provide a level of abstraction on top of the host system so that Java code compiled on heterogeneous environments can be run on any system that supports the Virtual Machine. Since its current inception Java has gone through three major revisions, and is currently in version 1.3.

Java supports a feature that provides ways to manipulate objects in an abstract manner, called Reflection [5]. Reflection allows an application to introspectively examine unknown objects to determine the fields, methods, superclasses, and the like. Not only can one look at the internals of an object, one may also read and write the values of Fields.

1.2 Lazy Threads & Pass Through Objects

The general concept of lazy threads is not new. A lazy thread in the context of this paper is defined as a thread that runs at a lower priority than the main processes of the program. Lazy threads are often used to perform some soft of delayed execution, such as the creation of auxiliary windows of a GUI while a user is interacting with an initial window. Lazy threads are most often used so that a user of a program can quickly be able to interact with it while it may be performing a processor intensive task.

Pass through objects are defined as objects that can be retrieved from an object database and are capable of being transparently manipulated. Thus, if one was to change an object, the server would automatically be aware of the changes and vice versa: if an object was changed on the server while a client had it open, the changes would be propagated to the client. For example if one was to retrieve a “Person” object that had previously been stored on the database, the client would be able to modify the local object and have the server updated without making any type of “save” calls. This can be accomplished with the support of lazy threads and the Java language.

Once an object has been persisted and later retrieved, the client can use native ways to deal with the object and have the object’s status transparently be updated on the server. By the lazy threads watching over the local object, they are aware of changes to the object just as well as they are aware of changes on the server, and by using Java’s reflection they changes can be read and persisted or the object can updated.

1.3 Motivation and Needs

Conventional databases have very procedural methods in which data is updated. Our works deals with trying to ease in an object database the updating of returned records. Our goals were to come up with some approach to retrieved objects that would make manipulation of the retrieved data more natural to an “end-user” developer.

We found that in developing and utilizing this system, it is much easier to deal with information. In our opinion, this method of updating data objects reduces some of the programmatic overhead involved with object databases in a data driven environment.

Further, our research involved the development of status driven systems [1, 3], such as electrical transformers, or semaphore driven systems. In status systems there is often a write only actor whose status is constantly changed and another actor who is watching this change. We concentrated on making each of these actors have to worry less about the method in which they would have to update their status reports, and more on generating good real time results. Thus, by way of lazy threads and reflection, a potential “status” object be natively updated and have the listener actor see the change with minimal client effort.

1.4 Paper Structure

This paper is ordered to provide an understanding into the research work from the design and development of the database mechanism along side with the experiences in implementing the ideas presented. Section 2 deals with the design and implementation of the client of this system. Section 3 deals with the support mechanisms for the client and object persistence, primarily the server end of the system, and the protocol that is used to communicate between the client and the server. Section 4 provides a general discussion of problems that were encountered in the development of the system as well as the usefulness of the system.

2 Design and Main Mechanisms

LTOP is designed in a client server environment. All reflective and lazy threaded work occurs in the client, while all data storage, query answering and searching occur on the server. The client is a Java package, named com.fullenergy.ltop.client and the server is stand-alone Java program that runs out of the package com.fullenergy.ltop.server. The task of the client is to store and retrieve objects. However, its most important purpose to provide pass through objects via the lazy threaded mechanism. It is implemented as an importable namespace, with the design goal to provide a high level interface to accessing and retrieving stored objects as well as creating newly persisted objects. On the client side is also support of “Contexts” by which objects are atomized to the client level depending on locking levels. This will be discussed in more detail in Section 2.2. The servers purpose is to store the objects that the client passes to it, and to propagate change information to connected clients. The server speaks to clients on TCP port 1162 communicating information about the objects that are being handled by the client, objects by that have been remotely updated and need updating on the client, and other general administrative and security tasks. To use LTOP, it would look like:

// create a new client

Client ltops = new Client();

// give the client some connect information

ltops.init("sa", "", "localhost");

// start the connection, and get a context manager

ContextManager conMan = ltops.start();

// start a new context

Context context = conMan.newContext("sampledb");

2.1 Contexts

Before we delve into a discussion of the way in which the client operates its reflection and lazy threading, one more design and implementation must be discussed. One of the original design goals of LTOP was to be able for a program utilizing LTOP to be able to persist status objects regarding certain states of things. For example, in a water plant scenario there are multiple pumps that push water from areas of low pressure to move water along a municipal distribution network [1]. Hypothetically, if each of the pumps was equipped with a LTOP client and there was an object representing the status of the each of the pumps, each pump would update the object on the LTOP server. This in its own right is nothing unusual; however, at the monitoring station LTOP clients who had contexts open on the same objects on the server would transparently have their local copy updated while the pumps where updating information. This is possible via Contexts, which encapsulate the lazy threading mechanism in the clients as well as the server – client communication.

In its most simple form a Context is a transaction in which all client operations to a server are atomized. The Context provides the locking mechanism for an object and is one of the fundamental pieces to the lazy threaded system as well. Objects are “locked” into a context when they are requested from a server. There are 4 locks currently defined in LTOP, represented by a one-byte value. They are quite simple and are: 1 – Read Only (RO), 2 – Write Only (WO), 3 – Read Only, Write Locked (ROWL), and 4 – Read Write (RW). The first mode, RO, is the default lock granted to a client when it asks for an object. The object can have an infinite number of RO locks opened against it. The RO lock, however, does not guaranty that the object will stay static in the local client Context. If there is a change on the object by another client that RW lock against it, the local representation of the object will be updated as soon as the server can push the new object values to the client. For example, if a water pump had a RW or RO lock on its status object, and there were 10 monitoring stations that had RO locks that same object, if the pressure of the pumped changed by 3 PSI, the pump would update the status object, and the new pressure information would be propagated to all the clients with RO locks.

The WO lock is primarily defined so that a system only needs to update an object and has no care for querying the status of that object. The main uses for such a lock would be in the previous pump example. This lock is exclusive, and if obtained by a client, all clients requesting locks into their Contexts are only able to gain RO locks. The next lock, ROWL is used for a client that is unsure at first weather or not it will need to update an object. If it obtains a ROWL lock, it is ensured that no other client can obtain any other type of write-based lock. If the client wishes to update the object, it must ask the Context for the lock upgrade. The last kind of lock needs little explanation. The RW lock is not defined to have any difference characteristics in LTOP than in any other database or file system. As a few last words about LTOP locking, an object may only have one WO, ROWL, and RW lock against it across all client Contexts, and a RO lock can be obtained (security premising) without restriction on any locked object.

As each object is locked, it is copied to the local client’s Context. Once there, the Context on the client works to ensure that the object remains valid across the server systems, e.g. if an object is updated, the Context will verify the change is allowed, if not it will actually undo the change on the client to the correct value. Ergo, if a client has a RO lock on a status object, and it calls its setPressure method, which updates the pressure field, when the lazy thread checks on that object, and notices that the pressure field had been incorrectly reassigned a value, it will actually change the value of the pressure field to the correct server based value. Conversely, if the client has WO, ROWL or RW locks against an object and it updates a field, the Context will pick this change up, and propagate it to the server.

2.2 Client

As mentioned earlier, the client is contained in the com.fullenergy.ltop.client package. To use the client, an application creates a new instance of the Client class. The Client class does nothing upon instantiation; rather, it waits for the application to call the init method of the Client class. The init method is a static method and performs some crucial startup tasks, mainly the creation of the lazy threads. Currently, the LTOP design specifics 16 lazy threads of type ObjectManager to be spawned on initialization. These 16 threads are broken down into groups of 4 Java Thread Groups, named ObjectManager0 through ObjectManager3 (abbreviated OM_n). Lastly, the init method sets the priorities of the OM₀, OM₁, OM₂, and OM₃ Thread Groups to Thread.NORM_PRIORITY, Thread.NORM_PRIORITY – 1, Thread.NORM_PRIORITY – 2, and Thread.MIN_PRIORITY respectively. Only the main thread group runs at Thread.NORM_PRIORITY because it would be unwise to have all the LTOP threads lock processing time from the main application’s needs. A detailed discussing of the threading mechanism will be done in section 2.2.1.

After the application had initialized the client, it then may call the start method in order to begin communication with the Server. Simply, the start method opens a connection the specified machine on port 1162, and then attempts to authenticate with the given username and password credentials. Once it is authenticated, it is assigned a 16-bit session identifier that is used to keep track of client-server communication. The start method then returns a ContextManager class, which is an embodiment of the session identifier and the method of doing administrative work before the client is has its own Context. From the ContextManager, the application may then ask for a new Context, to join an old Context. The request to create a new Context is nearly almost always granted, as long as the credentials are allowed on the specific object database on the server. An application may only join an existing Context if it was one that was previously created by the same set of credentials. However if a Context was flagged to be world joinable, it may be joined if not already open by another set of credentials. This feature is not currently implemented in LTOP, but considerations have been pre-built in.

Once the application has a Context, it is fully able to use the LTOP features. The two primary tasks that it can then perform are to ask for new object to be persisted, or to ask to retrieve an object from the database. In either case, once an object has been copied and locked into the local Context, the work of the lazy threads and reflection to support the pass through mechanisms begin.

2.2.1 Threads

As mentioned earlier, there are 16 threads that are spawned upon initialization of the LTOP Client of type ObjectManager. These are the threads that are used to update and set the values of the persisted objects that are in the local Context. Each object is assigned to a specific thread in a specific group. The most frequently accessed and modified objects are assigned to the Group with the highest priorities. The Context maintains a Most Often Used (MOU) list of objects in order to ensure that the objects that have the most frequent rate of change are assigned to the Thread Group that is most appropriate priority. As an object is used more and more, it can be reassigned to a higher Thread Group in order to facilitate the new rate of object change. Just as well, it can move back down to a lower priority Thread Group if it is accessed less often.

The MOU is a list of all the objects in the Context and the number of times that they have been modified, either by the server or the client. This number is divided by the number of seconds the object has existed in the Context. Every 64 accesses, the used number and timestamps are cleared so that if an object’s rate of usage drops, it will not take processing time in a high priority Thread Group. The MOU formula is as follows:

§ Define “a” as the number of accesses, and “t” as the time,

1. If a > 64, set a = 0; t_incontext = 0,

2. Determine access rate da/dt, via (a_server + a_client) / (t_incontext – t_now),

3. If da/dt in the current thread group is greater than the lowest da/dt in the next highest thread group, swap objects,

4. If da/dt in the current thread group is less than the highest da/dt in the next lowest thread group, swap objects.

The threads each process at a lower level of execution than the Application’s default, and they generally do not process in real time, because they do not check their assigned objects at every processing cycle. All this is done to ensure that the pass through mechanism does not overwhelm the application by hogging CPU time.

Each thread can be assigned multiple objects to watch over in the Context. Currently LTOP defines that each thread may monitor four objects, for a total of 64 objects to be passed through the Context (16 threads multiplied by 4 objects per thread). The main purpose of the thread is to watch the objects that are assigned to it in sub real time. If the object changes by the client, the thread is to propagate it (as its duty in the Context) to the server. If the Context finds from the server that the object that it has in the Context has changed, it is the threads duty to update the object on the client to reflect the new value. The Context, for each basic object, maintains the java.lang.Object hash code. For more advanced objects, mainly user-defined objects that contain numerous subtypes, the thread builds a singular hash code by recursively introspecting the contained objects until their fundamental types are found and using those hash codes. These “unified hash codes” are used by the Context and threads to determine whether or not a change has been made on the client. If the change has been made, it is then sent to the server.

The threads are able to look at the values of the fields of an object by using Reflection. Upon each other their times of execution and object inspection, they use reflection to look at the all the values of all the fields in the object, and then to delve deeper into the objects by using reflection again on the contained objects. That brings us to the second half of what makes pass through objects work: Java Reflection.

2.2.2 Reflection

A client is allowed to persist any type of object possible, no matter what it extends from or what its fields are. The LTOP client must be able to compensate for this by being able to handle no matter what type of object is thrown at it. This is done by utilizing reflection. Once an object is persisted for the first time, the object is broken down into all of its constituent parts. A water pump object may be methods such as setPressure, getPressure, setFlowRate, and getFlowRate and may have such fields as pressure and flowRate. When the object is persisted, the two fields are stored in the object repository, and their hash codes are constructed. The Java class Class provides a method named getDeclaredFields, which returns an array of Field objects that represent the fields of the object. We then iterate the array to get the values of all of the elements by using the Field object’s get method. It is very possible to find that when iterating through the fields that the field is not a fundamental data type. Thus, it is necessary to perform a recursive analysis of object to drill down to each of the contained fields fundamental type. This technique is also used to generate the hash codes that are used by the lazy threads to monitor each object. Once all of these fields are reflected and persisted, the object in its entirety has been persisted.

Once an object is persisted, it can be retrieved from a repository. Upon its retrieval, all of the stored fundamental values are written into the local objects fields by way of the Field objects set method. After that, the Application may use the object in any fashion it wishes to. When an object is updated on the server, and the changes need to be propagated, the client writes the new field information into the persisted object, in a nearly identical fashion as if it was a new object. The lazy threads, as described earlier, monitor the objects and use the reflection to keep watch over the object. This combination of reflection and lazy threads equal pass through objects. All this work is done on the client, and is backed by communication to a server that maintains master copies of the information that is being reflected in and out of the client.

3 Support Mechanisms

While most of our research has focused on the client side of pass through objects, the status support systems end of it would be impossible without some sort of repository to store and push changes to call subscribed clients. The support mechanisms of LTOP consist of a server and the communication protocol that it uses.

3.1 Server

The LTOP server is the relay of all the information and updating that is occurring in the LTOP environment. The server is responsible for communicating via the client server protocol, informing clients about updates, and propagating change information. The information store for the LTOP client is a Microsoft SQL Server, which is used to store field information as well as general administrative information. The main reason that we chose to use a relational database is because it is easy to use from Java and because it is very low overhead and requires no third party software as an object oriented database would.

The main tasks of the server are as follows: maintain a list of the sessions of the clients, maintain a list of the contexts, maintain an object list of what is locked by each context, keep all the field information for each persisted object. Along side all of these are the associated tasks of ensuring transactional safety, expiring aged contexts. Sessions are created each time a client connects to a server. There is not necessarily a one to one correspondence of sessions to contexts, as one client may have multiple contexts that it may be working in. Each context has the objects that have been locked out of it and these are the objects that the server actively monitors and accepts updated for. Updates are propagated and processed as follows:

1. A client (C₁) informs the server (S) that it has changed an object,

2. S verifies the lock is correct; if not, it denies the update and the C₁’s value is reset,

3. If correct, then S updates the field information,

4. S scans the list of contexts to find any other contexts that may have a lock on that same object,

5. If one is found, the server pushes a packet to the other client (C₂) with the changed data, and C₂ then reflects the change into the object.

By this mechanism all changes that occur anywhere in the LTOP system are pushed across all subscribed clients. The locks against the objects dictate all the updates that occur. For any type of lock that involves reading (RO or ROWL), the lock is basically expendable. If all the locks in that context are of reading only, the context then is deemed expendable. When a context is expendable we delete the context regardless of the fact if the context is closed cleanly or abruptly. If there are any write locks in the context, then it is dangerous if the context does not close cleanly. Thus, the server will roll back any changes that were not explicitly committed if the time out period of a context expires (which we have defined as one hour). When all the locks have been cleared, the context will then delete the context.

3.2 Protocol

The client server communication protocol is a packet-based protocol that is used for all things relating to objects. The types of communications are broken down into three categories: Authorization and Authentication, Contexts and Objects. The AA subpart of the protocol is simply used by the client to initiate a new session with the server, and from there the client receives a Session ID from which it communicates to the server. Using the Session ID, the client then can request to join or create a new context, by use of the Context subpart of the protocol. Once a Context ID has been received, the client then can use the Object subpart to perform all tasks related to persisting an object.

The protocol is very simplistic, and it does not include many provisions for security. The protocol is non-encrypted and passwords are sent as clear-text. However, its simplicity leads to it being a low impact protocol on a network, and the associated code that deciphers the packets simple. Each packet has three header bytes: the category byte, the subtype byte and the length byte. The last two bytes of the packet are the checksum (we have chosen to use the Fletcher checksum). The protocol is structured in ask and answer fashion. For example, a client asks the server for a context and it answers with a context ID. In the reverse direction the server answers, it will never ask a client for information (with the exception of error processing information). An example of a typical packet is as follows:

Client Says	0x02	0x01	0x09	0x00 0x00 0x00 0x01
	Context Packet	New Context Request	Packet length of 9	Session ID 0001

There are two bytes for checksums that have been omitted for compactness. In this example after the client has talked to the server, the server will response by either saying that the new context request was granted along with a context identifier or it will say the request was denied.

4 Discussion and Conclusions

In our internal uses of LTOP we have found it much easier to use pass through objects to persist data than object databases. Because we were able to updated objects directly and not have to worry about putting the objects back into the database we were able to cut the amount of code required. The code requirement to use the LTOP system is much less than it is to use other object persistence systems.

We believe the most useful use of a pass through object system would be for the implementation of a status support system. As in the example we have been using throughout this paper, pass through objects would be perfect to monitor systems where there is constant activity occurring. The client has no need ever to ask to refresh its object, thus cutting on the code requirements. Further, updating the status of the system is quite simple as all one has to do is to update the object in a normal fashion. This also cuts down on code requirements. The ability to also use reflection to allow or deny updates to an object allows for tight control of security. Since all events are verified through the server, the server may make decisions based upon its implementation.

FullEnergy Labs will be making the source to LTOP available in the near future.

5 Acknowledgment

We would like to acknowledge Mr. Tobin Fricke and Mr. David Ochi for their help in developing the concepts in this paper and giving guidance in the development of LTOP.

6 References

1. Colby C., Jategaonkar Jagadeesan L., Jagadeesan R., Läufer K., Puchol C., Objects and Concurrency in Triveni: A Telecommunication Case Study in Java, Proc. of the USENIX Conference on Object Oriented Technologies and Systems, 1998.

2. Elliot J., Moss B., Working with Persistent Objects: To Swizzle or Not to Swizzle?, IEEE Transactions on Software Engineering, 18(8):657-673, August 1992.

3. Franklin M., and Zdonik S., A Framework for Scalable Dissemination-Based Systems, Proc. of OOPSLA '97, pp Oct 1997.

4. Frolund S., Quality of Service – Aware Distributed Object Systems, Proc. of the USENIX Conference on Object Oriented Technologies and Systems, 1999.

5. Gosling J., Joy B. and Steele G., The Java Language Specification, Addison Wesley, Reading, MA, 1996.

6. Kakkad, S., Address Translation Strategies in the Texas Persistent Store, Proc. USENIX Conference on Object Oriented Technologies and Systems, 1999.

7. Milner R., Communication and Concurrency, Series in Computer Science. Prentice Hall, 1989.

8. Wollrath A., Riggs R., and Waldo J., A Distributed Object Model for the Java System, Proc. of the USENIX Object-Oriented Technology and Systems '96, July 1996.

0 Abstract