Reliable and Configurable Process Simulations via Probabilistic White-Box Models

Abstract

Business process simulation (BPS) has emerged as a crucial tool that offers a risk-free virtual environment to analyze, test, and optimize complex service compositions and orchestrations in cloud and edge computing environments. BPS enables the evaluation of alternative scenarios (a.k.a. “what-if” scenarios) by capturing the dynamic behavior of service interactions, including control-flow, task durations, and resource utilization. Several techniques to discover BPS models employ black-box predictors to characterize the run-time simulation aspects. The downside of these approaches is that the rules guiding the predictors cannot be provided in an intelligible form. This means that service architects and process analysts cannot modify or explore the decision-making logic that drives service orchestration and composition in these simulations. Moreover, these models are often deterministic and thus unable to capture uncertainty, which is essential to realistically simulate dynamic environments. This paper presents white-box predictors based on probabilistic decision trees, which are intelligible and easy to configure. The experiments show that white-box predictors can improve the simulation accuracy and variability, while being naturally intelligible.

Publication
Service-Oriented Computing

Practical problem

Process analysts and service architects use simulation to test what-if scenarios such as adding staff, changing service-level agreements or automating a task without touching the live system. Recent simulators plug deep learning models into the simulation to predict durations and routing, and these are accurate but opaque: nobody can read or edit the rules, so the what-if scenarios the simulation exists for cannot be expressed. Such models also need large amounts of data and return the same output for the same input, so they fail to reproduce the natural variability of real operations. Classic white-box simulators are editable but rely on simple distributions and are noticeably less accurate.

Research questions

  1. Can probabilistic decision trees characterize the run-time behavior of a process simulation model (routing, execution times, waiting times, arrivals) while remaining interpretable and editable?
  2. How does such a white-box simulation compare in accuracy with the white-box baseline Simod and with the black-box deep learning simulators DSim and RIMS?
  3. Does probabilistic rather than deterministic prediction yield more realistic variability in the simulated behavior?

Contributions

  • Decision-Aware Business Process Simulation (DBPS) models: a Petri net combined with probabilistic decision trees that set transition weights, activity execution times, resource-specific waiting times and inter-arrival times from trace history, attributes, resource, workload, weekday and hour.
  • A discovery technique that aligns the log to the control-flow model, trains CART decision trees per transition, activity and resource, and fits a per-leaf distribution (normal, exponential, log-normal, uniform or constant) by minimizing Wasserstein distance.
  • Three what-if interaction operations on the trees: changing leaf distributions, replacing a subtree by a leaf, and replacing a leaf by a subtree.
  • An open-source Python package (prosit on GitHub) that discovers a model from an event log and Petri net or lets users define each element manually.
  • An evaluation protocol on five logs (Purchase-to-Pay, a pharmacy process, academic credential recognition, and the BPI Challenge 2012 and 2017 workflow logs) with control-flow, cycle-time, arrival, rule-level and entropy measures.

Key findings

  • Across the five logs, DBPS beats the white-box baseline Simod on all but two metric-log combinations; average N-gram distance (NGD) 0.28 versus 0.66, cycle time distribution distance (CTD) 153.83 versus 356.06, case arrival rate distance (CAR) 200.12 versus 232.37.
  • DBPS is competitive with the black-box simulators: average NGD 0.28 versus 0.49 (DSim) and 0.48 (RIMS), average CTD 153.83 versus 194.92 and 170.53, average CAR 200.12 versus 229.91 and 240.39.
  • At the level of decision rules, DBPS has lower average distances than Simod on all four rule metrics, most clearly control-flow rule distance 0.07 versus 0.23 and execution-time rule distance 9.70 versus 32.07.
  • Average execution-time entropy is 1.47 for DBPS versus 1.32 for Simod, 0.73 for DSim, 0.71 for RIMS and 0.57 for a deterministic DBPS variant, so the probabilistic leaves are what restores variability.
  • Limitation: on the synthetic pharmacy log DBPS has the lowest entropy (0.36 versus 0.64 for Simod), and the evaluation is purely quantitative; whether analysts can actually use trees of depth up to 5 for what-if analysis is untested.