Event Processing Technologies

The slides from the topic presentation are available for download: http://bpt.hpi.uni-potsdam.de/pub/Public/BPT-Masterseminar-SS2013/Event-Processing-Technologies-SS13.pdf

Seminar Introduction

Today’s organizations strive to evaluate their executed processes with respect to usage, performance and conformance using process monitoring and process analysis techniques. During the execution of business processes data is produced and several events occur that are valuable for gaining insights about the processes and their execution. Utilizing these events for different purposes is the discipline of "Complex Event Processing" (CEP). According to the Event Processing Glossary (Vers. 2.0) compiled by David Luckham & W. Roy Schulte (http://www.complexevents.com/wp-content/uploads/2011/08/EPTS_Event_Processing_Glossary_v2.pdf) complex event processing is the "computing that performs operations on complex events, including reading, creating, transforming, abstracting, or discarding them."

In the seminar, the students will work on a specific topic in the area of event processing technologies (see below). First, the students present their approach synopsis (short presentation, 5 minutes), then present an application of their approach (technical presentation, 10 minutes), and finally give an overview of their solution and results (final presentation, 20 minutes). In addition, the results have to be documented in a scientific paper (max. 15 pages). The paper has to be submitted as draft version (mid. of the lecture) which will be reviewed by two other students. Thus, each student has to do two reviews which are graded as well. Based on the reviews the final paper has to be submitted at the end of the lecture time. To enable the students for holding scientific presentations as well as writing and reviewing papers two lectures will take place.

Seminar Topic Overview

  1. Scalability: complex event processing solutions for high performance and low latency
    1. Managing a high number of event types in a repository
  2. Aggregation concepts: event processing approaches to extract business information from raw events
    1. Knowledge-based Event Processing
    2. Complex event processing for run-time decision making
    3. Rapid detection of rare events
  3. Correlation: combining BPM and CEP
    1. Classification and challenges for combining CEP and BPM
    2. Framework for event-based monitoring in BPM
    3. Discovering event correlation rules for semi-structured business processes
    4. Using Events for Monitoring Business Process Interaction
    5. Deriving Event Processing Rules from Business Process Models
  4. Uncertainty: handling of noise in data streams
    1. Dynamic event subscription
    2. Complex event processing in the presence of uncertainty (co-supervisor: Andreas Rogge-Solti)
    3. Monitoring events in manual process execution environments
  5. Prediction: predict future events
    1. Predicting events and providing corresponding actions
  6. Heterogeneity: Processing heterogeneous events
    1. Identifying meaningful events in heterogeneous data streams
    2. Semantic matching of heterogeneous events to subscriptions

Important Dates

April 10th, 2013, 11am till 12.30pm - Opening Presentation (A-1.1)

April 15th, 2013 - Topic Application

April 16th, 2013 - Topic Assignment Notification

April 17th, 2013, 11am till 12.30pm - Lecture: How to make a scientific presentation (A-1.1)

April 24th, 2013, 11am till 12.30pm - Lecture: How to write a scientific paper (A-1.1)

May 8th, 2013 - Short Presentation (motivation, goal, approach, schedule) (A-1.1)

June 12th, 2013 - Technical Presentation (Approach Application) (H-E.51)

June 14th, 2013 - Paper draft submission

June 15th, 2013 - Paper draft distribution

June 21st, 2013 - Review submission

June 22nd, 2013 - Review distribution

July 4th, 2013 - Final Presentation (A-1.1)

July 26th, 2013 - Final Paper Submission

All the submission deadlines are 23:59 CET.

Seminar Structure

The seminar will contain two introductory lectures:
  • How to write a research paper
  • How to present a scientific talk

Seminar Topic Proposals

The topics are related to our current research projects:

Scalability

  1. Managing a high number of event types in a repository
    • Context: Today’s logistics service provider (LSP) strive to evaluate their transportation processes with respect to CO2 emissions, performance, capacity utilization, and conformance using process monitoring and process analysis techniques. During the transportation data and messages are produced and several events occur that are valuable for gaining insights about the transportations. This data needs to be provided in a structured form, namely event objects, for further event processing techniques, such as event composition, aggregation, and enrichment. Event objects are described by a event object type that specifies, among others, the structure of the event content and the procedure about the creation of the event content. Managing this event object types in environments with a high number of these is a challenge.
    • Task: Propose a conceptual solution for a repository for event objects types, e.g., in the style of the SOA paradigm. The repository should provide functionalities for, among others, searching event object types for further usage and guided creation of event object types. Evaluate which attributes and parameters of an event object type are necessary to be stored in the repository, e.g., several meta-data could be addressed. Describe the link between the event object repository and the event object store, a storage that contain all event objects that are created according the event object type specification. Derive the requirements for an event object repository from the logistics domain and utilize a use case motivated by the current research project "GET Service" for evaluation.
    • Literature:

Aggregation concepts

  1. Knowledge-based Event Processing
    • Context: In an organization, a huge amount of data is stored in external knowledge bases. Existing approaches combine this data with event processing in order to achieve more knowledgeable complex event processing.
    • Task: Review the different event processing approaches used for the fusion of background knowledge with run-time event streams. Evaluate if and how the approaches can be applied in our current research project "GET Service".
    • Literature:
      • Kia Teymourian, Malte Rohde, and Adrian Paschke. 2012. Fusion of background knowledge and streams of events. In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems (DEBS '12). ACM, New York, NY, USA, 302-313. (http://dx.doi.org/10.1145/2335484.2335517)
  2. Complex event processing for run-time decision making
    • Context: Novel approaches have evolved that gather data from various sources, analyze it, and extract events for decision making in emergency rescue operations and public transport.
    • Task: Evaluate how the decision making approach based on events can be applied to the logistic processes as they appear in our current research project "GET Service".
    • Literature:
      • Pekka Kaarela, Mika Varjola, Lucas P.J.J. Noldus, and Alexander Artikis. 2011. PRONTO: support for real-time decision making. In Proceedings of the 5th ACM international conference on Distributed event-based system (DEBS '11). ACM, New York, NY, USA, 11-14. (http://dx.doi.org/10.1145/2002259.2002262)
      • Pottebaum J., Artikis A., Marterer R., Paliouras G. and Koch R. Event definition for the application of event processing to intelligent resource management. International Conference on Information Systems for Crisis Response and Management (ISCRAM), 2011.
  3. Rapid detection of rare events
    • Context: Complex event processing is very crucial for rare events - such as events that occur at most once a month - that have very high costs for tardy detection and for false positives. Rare events can be earthquakes, tsunamis, nuclear radiation, or fires. In this context, the rapid, accurate detection of events and dissemination of warnings to people and systems that must respond quickly is of high importance.
    • Task: Review the theory, architecture, and early experience with applications that help to respond to earthquakes in the given literature and match it to the context of logistic processes in our current research project "GET Service".
    • Literature:
      • Michael Olson, Annie Liu, Matthew Faulkner, and K. Mani Chandy. 2011. Rapid detection of rare geospatial events: earthquake warning applications. In Proceedings of the 5th ACM international conference on Distributed event-based system (DEBS '11). ACM, New York, NY, USA, 89-100. (http://doi.acm.org/10.1145/2002259.2002276)
      • Annie Liu, Michael Olson, Julian Bunn, and K. Mani Chandy. 2012. Towards a discipline of geospatial distributed event based systems. In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems (DEBS '12). ACM, New York, NY, USA, 95-106. (http://doi.acm.org/10.1145/2335484.2335495)

Correlation

  1. Classification and challenges for combining CEP and BPM
    • Context: Today's organizations use heavily business process management systems (BPMS). Combining these with complex event processing (CEP) capabilities would gain them a lot more efficient business process execution.
    • Task: Evaluate the classification and challenges for CEP having a BPMS as data source given in the paper below. Then, review the literature for similar approaches. Show how these CEP and BPM are related, present relevant approaches, and compare them with appropriate means. Identify approaches and concepts that are applicable for our research project "GET Service".
    • Literature:
      • Michael Daum, Manuel Götz, and Jörg Domaschka. 2012. Integrating CEP and BPM: how CEP realizes functional requirements of BPM applications (industry article). In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems (DEBS '12). ACM, New York, NY, USA, 157-166. (http://dx.doi.org/10.1145/2335484.2335503)
  2. Framework for event-based monitoring
    • Context: Business process execution as well as corresponding events are monitored to provide information about a business process. For example, to ensure processes are executed as expected or to trigger corrective actions when needed. The monitoring activity needs to be operated in a non intrusive and dynamic way. The monitoring framework EasierBSM covers coarse-grained as well as fine-grained abstractions, from choreography to orchestration and services.
    • Task: Summarize the given literature and present the monitoring framework EasierBSM. Evaluate the suitability of EasierBSM for an application in the GET project with appropriate measures.
    • Literature:
  3. Discovering event correlation rules for semi-structured business processes
    • Context: Events are particularly important pieces of knowledge, as they represent activities of special significance within an organization. The events originating from different kinds of sources have to be correlated in order to track instances of a running business process for the purposes of monitoring, discovery and other applications. The relationships between events can be determined via correlation rules. The definition of correlation rules, however, is a very time-consuming and labor intensive process that requires tool-support and automation. Presently, a novel approach was published that describes an algorithm to discover event correlation rules from arbitrary data sources with a high level of accuracy. This is a step towards semi-automating the task of detecting correlations.
    • Task: Evaluate if and how the algorithm can be applied in the context of the "GET Service" project to discover event correlation rules for logistic processes.
    • Literature:
      • Szabolcs Rozsnyai, Aleksander Slominski, and Geetika T. Lakshmanan. 2011. Discovering event correlation rules for semi-structured business processes. In Proceedings of the 5th ACM international conference on Distributed event-based system (DEBS '11). ACM, New York, NY, USA, 75-86. (http://dx.doi.org/10.1145/2002259.2002272)
  4. Using Events for Monitoring Business Process Interaction
    • Context: Today’s organizations strive to evaluate their executed processes with respect to performance and conformance, but also with respect to collaboration with other organizations.
    • Task: Explore the given literature to reveal which techniques for business process collaboration monitoring are described. For a comparison, link the work to be done to the to the research project "GET Services".
    • Literature:
      • A. Norta, P. Grefen. Discovering Patterns for Inter-Organizational Business Collaboration. International Journal of Cooperative Information Systems, Vol. 16, No. 3/4, World Scientific, 2007. pp. 507-544.
      • A. Norta, R. Eshuis. Specification and verification of harmonized business-process collaborations. Information Systems Frontiers 12, 4 (September 2010), 457-479. DOI=10.1007/s10796-009-9164-1-
      • A. Norta. Exploring Dynamic Inter-Organizational Business Process Collaboration. PhD Thesis, Beta Research School for Operations Management and Logistics, 2007. Eindhoven, The Netherlands. (http://alexandria.tue.nl/extra2/200710444.pdf)
      • R. Eshuis, A. Norta. A Framework for Service Outsourcing using Process Views. Proceedings 2010 14th IEEE International Enterprise Distribution Object Computing Conference (EDOC), IEEE Computer Science pages 99-108 Vitoria, Brazil.
  5. Deriving Event Processing Rules from Business Process Models
    • Context: Business process execution is monitored to provide information about a business process. For example, monitoring can be used to detect deviations from business process specifications or evaluate the execution time of a (collection) of activities in a process instance. Complex event processing (CEP) can be used to support business process monitoring. Existing approaches propose methods to derive CEP rules from a business process model.
    • Task: Review the given literature and present existing approaches that derive complex event processing rules from a business process model to monitor business process execution. Do a literature review to identify similar approaches and compare them. Based on the literature review, the approaches should be demonstrated for a subset of logistic processes as they appear in our current research project "GET Service".
    • Literature:
      • M. Weidlich, H. Ziekow, J. Mendling, O. Günther, M. Weske, and N. Desai. "Event-Based Monitoring of Process Execution Violations". BPM 2011, LNCS 6896, 2011
      • F. Koetter, M. Kochanowski. "A Model-Driven Approach for Event-Based Business Process Monitoring". BPM Workshops, LNBIP, Volume 132, 2013
      • A. Baouab, O. Perrin, C. Godart. "An Optimized Derivation of Event Queries to Monitor Choreography Violations". 10th International Conference on Service Oriented Computing, Vol. 7636, 2012

Uncertainty

  1. Dynamic event subscription
    • Context: In the context of the Internet of Things, it is impossible to know all event sources that might be relevant or needed. One approach to tackle this problem is a mechanism that enables a service to decide which (complex) event to subscribe to and for how long. Situation action networks (SAN) are used to produce the dynamic subscriptions based on detected situations.
    • Task: Review the given literature and evaluate the suitability of SAN for producing dynamic event subscriptions in the context of logistic processes. This work should be related to our current research project "GET Service".
    • Literature:
      • Yiannis Verginadis, Nikos Papageorgiou, Ioannis Patiniotakis, Dimitris Apostolou, and Gregoris Mentzas. 2012. A goal driven dynamic event subscription approach. In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems (DEBS '12). ACM, New York, NY, USA, 81-84. (http://dx.doi.org/10.1145/2335484.2335493)
      • B. Montreuil. 2011. Toward a Physical Internet: meeting the global logistics sustainability grand challenge. Logistics Research, 3(2), 71-87. (http://dx.doi.org/10.1007/s12159-011-0045-x)
  2. Complex Event Processing in the Presence of Uncertainty (co-supervisor: Andreas Rogge-Solti)
    • Context: In many real settings information might be not 100% reliable. One example is location information from GPS-sensors that might be off to some degree. But there are other sources of information that might be noisy or even fake. Luckily, there exist approaches to deal with uncertain information, originating from the artificial intelligence area. Probabilistic approaches based on probabilistic networks, like Bayesian Networks or Markov Chains can be used to classify uncertain events correctly. (A good understanding of probability theory or an interest in artificial intelligence techniques is helpful for this topic.)
    • Task: Evaluate the given literature and present their proposed approach to handle uncertainty in CEP. Explain the problems that arise when using Bayesian networks in this domain. If possible apply probabilistic approaches in the context of the "GET Service" project.
    • Literature:
      • S. Wasserkrug, A. Gal, O. Etzion, and Y. Turchin. "Complex Event Processing over Uncertain Data". DEBS'08 Rome, Italy 2008
      • Skarlatidis, Anastasios; Paliouras, Georgios; Vouros, George A.; Artikis, Alexander. Probabilistic Event Calculus based on Markov Logic Networks. In: Proceedings of 5th International Symposium on Rules (RuleML 2011) Part 2 , Fort Lauderdale, USA, 03-05 November 2011.
      • E. Charniak. "Bayesian networks without tears". AI Magazine, 12(4), 1991
  3. Monitoring events in manual process execution environments
    • Context: During the execution of business processes several events occur. These events are valuable for process monitoring and analysis but are subject of a couple of problems origin from the manual execution. First issue is that these events are not represented in the IT system landscape in real-time, but in a deferred way, e.g., during a ward round the doctor is writing down some information about a patient's treatment to a sheet of paper or is using his dictating machine. The information about the treatment event is recorded in the clinical information system several hours/days later (use case motivated by our research project PIGE: www.pige-projekt.de). A second issue occurs in the correlation of these events to a specific node. Some of the events may be expected between two activities, e.g., when a patient is passed from the operating room back to the ward. In between several events could occur, however, in the process model the surgery activity and the aftercare activity is modeled. The paper "Towards process evaluation in non-automated process execution environments" presents the definition of event monitoring points on every node, e.g., an activity, in a process model. A definition of event monitoring points on other elements is not supported yet.
    • Task: Do a literature research about methodologies that are dealing with deferred event recording. Summarize these methodologies and compare them. If this topic is not discussed in literature yet, come up with an own approach, how this could be handled. As an idea: How could a process monitoring and analysis system that is using event information know that some events might have already occurred in real-world but are not recorded in the IT system landscape yet? Categorize which constellations of event recording can occur, e.g., event recorded delayed, but timestamp is the original one, event recorded delayed and timestamp is delayed as well. Think about scenarios in which it is necessary to have event monitoring points between two nodes also, e.g., on control flow edges. How could the definition of event monitoring points on these elements look like?
    • Literature:
      • C. Malessa, K. Kirchner, O. Habrecht, N. Herzberg, S. Krumnow, H. Scheuerlein, A. Bauschke, and U. Settmacher. Can Liver Transplantation Be Supported by Intelligent Clinical Pathways? 24th International Congress of The Transplantation Society, 2012. DOI: 10.3252/pso.eu.24tts.2012.
      • N. Herzberg, M. Kunze, A. Rogge-Solti. Towards process evaluation in non-automated process execution environments. 4th Central-European Workshop on Services and their Composition (ZEUS), 2012.

Prediction

  1. Predicting events and providing corresponding actions
    • Context: Nowadays, a challenging problem in all domains is the reaction to unplanned events. Selected approaches show that event-driven systems can be used to predict future events and react to them before they occur. In the logistics domain, a reactive application might react to congestion and its caused delay by rescheduling the subsequent routes for a transport; a proactive application, in contrast, will attempt to predict the congestion and prevent the delay by suggesting another route in advance. Here, proactive event-driven systems can exploit existing event-based technology to monitor current events and infer future situations based on that information.
    • Task: Review the given literature and develop a proactive model for specific use cases in the logistics domain as they appear in our current research project "GET Service".
    • Literature:
      • Yagil Engel, Opher Etzion, and Zohar Feldman. 2012. A basic model for proactive event-driven computing. In Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems (DEBS '12). ACM, New York, NY, USA, 107-118. (http://dx.doi.org/10.1145/2335484.2335496)

Heterogeneity

  1. Identifying meaningful events in heterogeneous data streams
    • Context: During the execution of business processes several events occur that can be monitored to analyze the execution of business processes. Data streams, often seen as sources of events, provide a variety of heterogeneous events that have to be processed. To cope with the heterogeneity of event stream, the Sparkwave system includes pattern matching as well as several stream reasoning features such as support for fairly expressive pattern definitions, time-based sliding windows and schema-entailed knowledge.
    • Task: Summarize the given literature and present Sparkwave. Evaluate the suitability of Sparkwave for an application in the "GET Service" project with appropriate measures.
    • Literature:
  2. Semantic matching of heterogeneous events to subscriptions
    • Context: During the execution of business processes several events occur. These events are monitored via corresponding subscriptions. However, in today's dynamic environments event sources can be changed, added, or removed regularly. We are then forced to handle heterogeneous events. This makes it difficult to develop and maintain event-based systems and corresponding subscriptions. For example, logistics service provider (LSP) strive to evaluate their transportation processes with respect to CO2 emissions. This can be represented by a subscription listening to updates of the carbon emission value. To receive this value, the company has to identify different potential sources and events that affect the calculation. Some existing event matching approaches address this problem with techniques to identify events and sources for particular subscriptions.
    • Task: Starting from the given literature, identify event matching approaches and corresponding systems that are able to specify a (dynamic) matching between events and subscriptions. Reveal how these approaches can be used in the context of logistic processes as they appear in our current research project "GET Service".
    • Literature:

General links and literature to BPM
  • M. Weske. "Business Process Management: Concepts, Languages, Architectures", 2nd ed. 2012, XV, 403 p. 300 illus. Hardcover ISBN 978-3-642-28615-5, Springer-Verlag Berlin Heidelberg 2012
General links and literature to complex event processing
  • D. Luckham. The Power of Events - An Introduction to Complex Event Processing in Distributed Enterprise Systems. ISBN 0201727897. 2010.
  • O. Etzion, P. Niblett. Event Processing in Action. ISBN 9781935182214. 2011.
  • http://www.ep-ts.com/
  • Gianpaolo Cugola and Alessandro Margara. 2012. Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44, 3, Article 15 (June 2012) (http://dx.doi.org/10.1145/2187671.2187677)
General links and literature to the combination of BPM and CEP Links and literature for all GET-related topics