Welcome to Fulmine. Fulmine is a near-real-time, distributed, data model system. It was created as a foundation for NRT data systems; it incorporates all the required features and has the experience of over 10 years in distributed NRT data systems applied to it. Fulmine's principle goal is to allow developers to focus on the business logic for NRT distributed data systems - the 'what we need to do' not the 'how do we do it'; Fulmine takes care of the 'how'.
Fulmine has been designed from the ground up to be a near-real-time data distribution system for Java. Fulmine uses the concept of a context to encapsulate a unique runtime instance (a VM can host multiple contexts). Data entities are represented as records within the context. The attributes of the entities are represented as fields in the records. A record is a map/dictionary of fields indexed by a string name. The fields can only be scalar data types (e.g. integer, string, boolean). Records are thus 'flat' data structures.
Records created in a context are 'local' to that context. They can be replicated to other contexts (termed remote contexts) simply by the remote contexts subscribing for the record from the local context. Any changes made to the record in the local context will be distributed, in near-real-time, to the remote contexts. When a record is subscribed for, in this way, it is known as a remote record in the remote context (because it is not native to the subscribing remote context). Records are defined by a name, type and domain. When subscribing for a remote record, the identity, type and domain must be supplied. Regular expressions and wildcards can be used for the identity, type and domain. Application code in the local context can also subscribe for local records. Remote contexts cannot directly alter the state of the remote records; an RPC is invoked to pass on the requested change to the local context.
Subscribers receive copies of the record when any fields change. Changes to a record's fields are performed in an 'execution frame'. This is analogous to a transaction (but without rollback capability). The execution frame is started, then changes are made to the fields, then the frame is committed. When an execution frame has been started on a record, no other frames can begin until the current frame is committed. When the frame is committed, subscribers for the record will see all the changes that occurred in that execution frame as an atomic change; they will receive a new copy of the record.
Contexts use UDP to detect one another. They issue heartbeats to allow each other to detect when a context goes offline. Contexts connect to each other, when subscriptions are issued, using TCP/IP. Record changes are sent over the TCP/IP connection. A context will detect how many CPUs are available and create a thread per CPU. All records of the same type are bound to the same thread (CPU). Fulmine thus scales its throughput based on how many CPUs are available and how many distinct record types the application code requires.
Fulmine has a built-in permissioning framework that gives control over what record fields are visible depending on the subscriber's permission token. The local context assigns permission rights to the fields of its local records. Only if the subscribing context presents a compatible permission profile will the relevant fields be available to the remote subscribing context. The permission checking for fields occurs on de-serialisation within the remote context.
The distrubution is handled using a custom binary wire format. Fields are given wire codes and changed fields are identified by their wire codes. As fields are added or removed, the wire format is updated to add or remove the wire codes. Only changes are sent in the wire format (this is the Fulmine delta transmission protocol). This provides efficient replication of the records.
A Fulmine context allows application code to enable methods to be remotely called via its RPC mechanism. There are some caveats to this; only primitive types (including the String object) are supported for method arguments and return types. The RPC framework is also used as the mechanism for remote contexts to updated remote records. The RPC framework does not employ any implicit permissioning.
A fault tolerance (FT) system has been built into Fulmine. A FT cluster is composed of any number of FT context instances that connect to each other over a separate FT network. The FT contexts of a cluster each host an identical application context. Only one FT context instance will be active and will start its application context; this application context will then be discoverable by external application contexts. All the other FT instances will be in a "warm" standby mode and will not start their hosted application context. Should the active FT context go down, another will move in to take its place and its application context will be started. Because the FT contexts in a cluster host an identical application context, external application contexts will simply see a 'blip' in the FT application context as it 'moves' to another FT context instance. The FT contexts are given a priority to decide which one becomes the active instance when there is no active instance.
Fulmine has been built, as far as possible, to be a 'zero config' distributed system; with very little code, a fully operational Fulmine instance can be created. The more advanced features require only a minimal set of configuration.
Fulmine has comprehensive javadoc information about all its features.
