A Short Dissertation on EDA

There is a lot of literature on EDA, event stream processing, CEP, etc; that is, on event and event processing technologies. Although all of them are very good, it can get a little overwhelming. Following, I attempt to describe EDA and how EDA relates to other technologies, such as SOA, real-time, and Java, in a pragmatic form.

Event-driven architecture is an architectural style composed of decoupled applications that interact by exchanging events. These applications are called event-driven applications. Event-driven applications may play the role of an emitter of events, and of a responder or processor of events.

Event-driven architecture is important, because the real-world is event-driven. One example is the financial world, in which trader applications react to events (or changes) made to the financial exchange market. Event-driven situations should be modeled by event-driven architecture.

Event driven applications are sense-and-respond applications, that is, applications that react to and process events.

Events are state changes that are meaningful to an observer. Generally, events are in the form of a protocol message. Events may be simple or complex. Simple events contain no meaningful member events. Complex events contain meaningful member events, which are significant on their own. An example of a simple event is a stock bid event, and a stock offer event; an example of a complex event is a stock trade event, which includes both a bid event and an offer event.

Events may be delivered through different mediums, two of which are channels and streams. Channels are non-active virtual pipes, that is, a producer component is responsible for inserting data into one side of the pipe and another consumer component is responsible for removing the data at the other side of the pipe. The data is stored in the channel as long as it is not removed by a component. Of course, channels may be bound, in which case it may stop accepting new data or purging existing data as it sees fit. Examples of channels are JMS queues and topics. In the contrary, streams are active virtual pipes, that is, they support a continuous flow of data. If a producer component does not directly listen to the stream, it is likely to miss some data. Because streams do not need to store data, streams are able to support a high-volume of streaming data flowing through them. An example of a stream is the of the air TV broadcast.

Having received events, the next task of an event-driven application is to process the
events. Event Processing is defined as a computation stage that consumes and optionally generates events. Currently, as specified by Roy Schulte, there are four ways to categorize event processing:

  • Event passing:
    Events are simply handled off between components, there is
    mostly no processing, and it generally deals only with simple events. Event-passing applications are asynchronous, staged, and trigged by the arrival of one event from a single event stream or channel. Sometimes they are referenced as message-driven or document-driven applications. Examples are simple pub-sub applications.
  • Event mediation (or brokering):
    Events are filtered, routed (e.g. content-based), and transformed (e.g. enriched). Event mediators are stateless, and deal with both simple and complex events; however they do not synthesize new complex events of their own, that is, event mediators cannot combine (i.e. aggregate) simple events into complex events, mostly due to the fact that they do not keep state. Generally, there is a single event stream or channel fan-in, and multiple event
    streams or channels fan-out. Examples are integration brokers.
  • Complex Event Processing (CEP):
    Events are processed by matching for complex patterns, and for complex relationships, such as causality, timing, correlation and aggregation. CEP applications are state-full; simple and complex events are received from several event streams and new complex events may be synthesized. CEP applications must be able to handle a very high volume of events, and hence generally only using streams.
  • Non-linear Complex BPM:
    Event-based business processes modeling non-linear complex work flows. The business process is able to handle unpredictable situations, including complex patterns, and complex event relations.

Event Stream Processing (ESP) is event processing solely on streams, as opposed to channels. Hence, CEP is always part of ESP; however ESP includes other event processing types, such as event passing and event mediation, when those are performed on streams, rather than on channels.

An event-driven application may play the roles of event source, event sink, or both. An event source generates events to event sinks. Note that event sources do not necessarily create the event, nor events sinks are necessarily the consumer of events. Furthermore, event sources and event sinks are completely decoupled from each other:

  • An event source does not pass control to event sinks, which is the case of service consumers delegating work to providers; and
  • Event sinks do not provide services to event sources, which is the case of consumers that initiate and consume work from providers; and
  • One can add and remove event sources and sinks as needed without impacting other event sources and sinks.

How does EDA compare to SOA? That depends on how the loosely term SOA is defined. If SOA is defined as an architecture that promotes re-use of modular, distributed components, then EDA is a type of SOA. If SOA is defined as an architecture where modules provide services to consumer modules, then EDA is not SOA.

The concepts previously described are based upon work from Roy Schulte, Mani Chandy, David Luckham, and others.

Next, let’s focus on real-time concepts.

Real-time is the capability of a system on being able to ensure the timely and predictable execution of code. In another words, if a developer specifies that an object must be executed in the next 100 milliseconds (or in the next 100 minutes for that matter), a real-time infrastructure will guarantee the execution of this object within this temporal constraint.

Event-driven architectures are suitable for real-time. Event-driven applications are generally implemented using asynchronous mechanisms; this lack of synchronicity improves resource usage, which in turn helps guarantee real-time quality of service.

Objects that have temporal constraints are named schedulable objects. The system measures how well the temporal constraints are being met by means of a particular metric, for example, the number of missed deadlines. Schedulers order the execution of schedulable objects attempting to maximize these metrics. Schedulers make use of different algorithms or policies to do this, one of which is the Rate Monotonic Analyze (RMA). RMA relies on thread priority as a scheduling parameter and determines that the highest priority should be associated to the shortest tasks.

Let’s re-consider CEP. CEP allows one to specify temporal constraints in the processing of events. For example, one can specify to match for an event that happens within 100 milliseconds of another event. Hence, CEP rules (e.g. queries) are essentially a type of schedulable object, and therefore a CEP agent must be a real-time agent.

In a very loosely form, CEP can be further characterized by two functions, a guarding function, and an action function. The former determines whether an event should trigger a response, and the latter specifies the responses to be taken if the guard is satisfied.

Consider a system that supports CEP agents whose action functions are coded in Java. This implies that the system must support the development, and deployment of Java applications, and hence, in this regards, it must be to some extent a Java application server, or rather as we have concluded previously, a real-time Java application server.

To be more exact, CEP Java action functions do not need the full services of a complete application server, for instance, part of the transactional, persistence, and security container services may not be needed. What is needed is a minimal-featured application server. This minimalist aspect is also applicable to the real-time capability. We do not need a full set of real-time features that enables the development of any type of applications, but rather a minimal set of real-time features that enables the support of CEP agents.

A system that supports CEP also supports other event processing types, such as event passing and event mediation. Therefore, a light-weight real-time Java application server that is able to host CEP agents is a good overall solution for achieving EDA.

Advertisements

2 Responses to A Short Dissertation on EDA

  1. […] EDA Programming Model In the previous post A Short Dissertation for EDA, I try to describe what is EDA, and why it is […]

  2. Alexandre Alves says:

    “Very interesting. So from what you say I gather that many applications are architected in this event style? Are there architectural frameworks/guidelines for this type of event architecture?”
    Posted by: jonmountjoy on November 19, 2006 at 5:28 PM

    “Yes, EDA as an architecture is not a new concept and is being used for some time now; what is more recent is some of the new event processing technologies, such as CEP. Benefits of why one would use EDA and guidelines for doing so is a long subject, maybe it deserves a separate blog entry. :-)”
    Posted by: aalves on November 20, 2006 at 6:33 PM

    “Or an article 🙂 Grab the article template at http://dev2dev.bea.com/editorial_guidelines.html and send me the result!”
    Posted by: jonmountjoy on November 21, 2006 at 7:57 AM

    “In the world of CEP, I’m having difficultly understanding the difference between “events” and “messages”. At times, they seem to be used interchangeably. When I think of an event, this is an action, such as a change of state which you clearly articulate above. Where things start to get confusing is when we talk about sending/receiving events. It seems like data, in the form of messages, is what is actually sent through channels or streams. Is there something that I am missing here or should be looking at differently?”
    Posted by: larry.dino@bea.com on December 13, 2006 at 4:22 PM

    “Good article, Alexandre.
    Many people think of SOA as synchronous RPC (mostly over Web services). Others say EDA is SOA. And there are people who say that the best of EDA and SOA is combined in SOA 2.0. But an architectural distinction can be made between a request- and-reply pattern and a publish-and-subscribe pattern. Both patterns are an inverse of each other. Because of the completely different nature and use of the two patterns, it is necessary to be able to distinguish between the both and to name them. You might say making such a distinction is a universal architectural principle. Combining both of the patterns into an increment of the version number of one of them is not a very clever act. It is appropriate and desirable to use the acronyms SOA and EDA to make this distinction, because SOA and EDA are both positioned in the same architectural domain; SOA focusing on (the decomposition of) business functions and EDA focusing on business events. This article explains the differences between the two patterns, when to use the one or the other and how to combine them.

    An additional clarifying article and a blog devoted to this subject…”
    Posted by: jack.vanhoof on January 3, 2007 at 11:26 PM

    “Good overview. As an old techie (42 YEARS IT/ICT), first on mainframes and then on VAXe’s and the like, we did a lot of “real-time” programming, which I see nowadays referred to as “concurrent programming”. We used a lot of event-based designs/architecures in both firmware and user-applications (above the operating system). A recurring problem was missing events. To cater for this, we normally employed “uninterruptible” (by the hardware) instructions that would execute in a single clock cycle to test for the arrival of an event (using semaphores or event control blocks that could cater for a list of types of events for a single session/address-space/thread or in other cases the scheduling-type list signalling the arrival of an event for a particular session of any type). Hardware made use of levels of interruption (event-signal), tied with escalating priority (in some hardware up to 4, with power and timer interrupts on higher level, etc.) For this reason, stacking of interrupts were done to cater for the loss of a lower-priority interrupt. I think queues of some kind can separate events that are time-constrained vs “lazy” events (time-tolerant), just to cater for missing events. This is where the TIBCO-type message bus is useful, as it retains the message that was published, until aging or explicit deletes removes the event/message. You, see, I also have a bit of a problem differentiating between messaging and event-based processing (I buy your light app server and such for an EDA, but…) The EDA paradigm (and allied pub-sub)is in my opinion, not always compatible/suitable with any/all kinds of business transactions, especially the logical request-reply models, although implemented asynchronously from each other, the pair still constitutes a full closed-loop process model from a business “transaction” point of view. Particularly important in banking, etc., where missing or duplication of events cannot be tolerated. Guaranteed delivery is also of critical importance in the media employed to the event sink. So, “horses for courses”. What do you think?”
    Posted by: ottohm on August 15, 2007 at 9:11 AM

    “A lot of interesting questions in your comment; let me give you my opinion on them:
    Events, which sometimes are called Event Objects, represent a (local) entity at the application layer. Messages represent an entity at the communication layer. For example, an event may be created at some event source (application code), transformed into a message and sent over the wire to some remote endpoint (infra code), which then unmarshals the message into an event and delivers it to an event sink (application code). Note that an event may be created and consumed locally, in which case there won’t be a need for a message.

    It is layered architecture. When one talks in terms of processing data to aggregate several items, the context is of events. When one talks in terms of marshaling, unmarshaling, routing, the context is of messages. If one goes even lower, the message becomes a packet at some data link layer between network components.

    Exchange of events is one interaction pattern. The other common pattern is Request/Response, which actually, is better described as Request/Indication (peer)/Response (peer)/Confirmation. The main difference is that the event pattern is push-based, the Request/Response pattern is pull-based.

    Real-time, Event-Driven systems generally behave better when based upon the push model, where as service oriented systems (as you pointed out) use/need the pull model.

    Both are definitely needed. The Event Server provides the missing push-model support to an already existing pull-model infrastructure.”
    Posted by: aalves on August 16, 2007 at 10:15 AM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: