# Detecting Context-Aware Deviations in Process Executions

- Type: Conference paper
- URL: https://www.gyunam.com/publication/2022-bpm-forum/
- Date: 2022-09-01
- Authors: Gyunam Park, Janik-Vasily Benzin, Wil M. P. van der Aalst
- Venue: Business Process Management Forum - BPM 2022 Forum, Münster, Germany, September 11-16, 2022, Proceedings
- DOI: https://doi.org/10.1007/978-3-031-16171-1_12
- PDF: https://www.gyunam.com/publication/2022-bpm-forum/paper.pdf
- Link (URL): https://doi.org/10.1007/978-3-031-16171-1_12
- Research lines: [Process discovery and diagnostics](https://www.gyunam.com/research-line/process-discovery-and-diagnostics/index.md)

## Abstract

A deviation detection aims to detect deviating process instances, e.g., patients in the healthcare process and products in the manufacturing process. A business process of an organization is executed in various contextual situations, e.g., a COVID-19 pandemic in the case of hospitals and a lack of semiconductor chip shortage in the case of automobile companies. Thus, context-aware deviation detection is essential to provide relevant insights. However, existing work 1) does not provide a systematic way of incorporating various contexts, 2) is tailored to a specific approach without using an extensive pool of existing deviation detection techniques, and 3) does not distinguish positive and negative contexts that justify and refute deviation, respectively. In this work, we provide a framework to bridge the aforementioned gaps. We have implemented the proposed framework as a web service that can be extended to various contexts and deviation detection methods. We have evaluated the effectiveness of the proposed framework by conducting experiments using 255 different contextual scenarios.

## Practical problem

Hospitals, manufacturers, and service companies flag unusual cases in their processes to catch errors, fraud, or quality problems, but a case that looks abnormal in isolation may be perfectly explainable once the circumstances are known: a long wait for admission during a pandemic wave, or a slow order week when half the team is on leave. Conversely, cases that look normal may hide a problem, for instance throughput that is only on target because staff worked the weekend. Existing deviation detection techniques either ignore such context entirely or bake one specific kind of context into one specific algorithm, so an organization cannot combine the detector it trusts with the contextual factors that matter to it. None of them separates context that excuses a deviation from context that casts doubt on an apparently normal case, which is exactly the distinction a process owner needs when deciding what to investigate.

## Research questions

1. How can arbitrary contextual factors be incorporated systematically into existing unsupervised deviation detection methods, rather than building one method per context?
2. How should positive context (which justifies deviations) and negative context (which refutes apparent normality) be modeled and used to revise deviation scores?
3. Does context-aware post-processing improve the classification of deviating and normal traces across different detection methods and contextual scenarios?

## Contributions

- Formalizes context-aware deviation detection as a four-class labeling problem combining non-context and context classifications (context-insensitive or context-sensitive, deviating or normal).
- A context ontology for deviation detection that merges prior business process context work into intrinsic, relational, and external contexts, and extends it with positive and negative context semantics.
- A post-processing framework with four components (deviation detection, context analysis, context linking to events and traces, score revision) that plugs in any score-based detector.
- An extensible cloud-based web service, `contect`, on GitHub with a user manual, implementing workload, weekend, waiting time, and capacity utilization contexts and four detectors (Inductive Miner with alignments, Profiles, ADAR, Autoencoder).
- An evaluation protocol that injects four deviation types and four contextual scenarios into CPN Tools simulations of an order management process, yielding 225 scenarios per detector.

## Key findings

- Across 225 simulated scenarios per detector, context-aware post-processing with grid-searched positive and negative degrees raises accuracy, precision, and recall over the context-unaware baseline for all four detectors.
- For the Inductive Miner based detector, accuracy rises from 0.389118 to 0.426846, precision from 0.248691 to 0.293496, and recall from 0.389118 to 0.425686; precision gains are largest for Autoencoder (+0.133433), Profiles (+0.111686) and ADAR (+0.104261).
- Average class accuracy decreases slightly for every detector (e.g. 0.326856 to 0.311832 for Inductive), so the gain is not uniform across the four classes.
- Summed confusion matrices show the improvement comes from context-sensitive deviating traces: 54,186 of 163,160 are correctly labeled with Inductive and 47,951 of 159,417 with Autoencoder.
- Limitation: context-sensitive normal traces are almost never recovered (0 of 21,381 for Inductive, 83 of 21,240 for Autoencoder), results depend on several tunable parameters and on the underlying detector, and external contexts are only measured indirectly.