A Generic Approach to Extract Object-Centric Event Data from Databases Supporting SAP ERP

Alessandro Berti, Gyunam Park, Majid Rafiei, Wil MP Van Der Aalst
July, 2023
PDF Cite DOI

Abstract

Process mining provides a collection of techniques to gain insights into business processes by analyzing event logs. Organizations can gain various insights into their business processes by using process mining techniques. Such techniques use event logs extracted from relational databases supporting the business process as input. However, extracting event logs is challenging due to the size of the data, and it remains ad-hoc. Existing commercial tools partly support the extraction of event logs, but they are proprietary and focus on the mainstream processes such as Purchase-To-Pay (P2P) and Order-To-Cash (O2C). Moreover, the extracted event logs suffer from well-known deficiency, convergence, and divergence issues. For example, due to convergence events are unintentionally duplicated causing unreliable or confusing performance diagnostics. In this paper, we propose an approach to extract event logs while avoiding the aforementioned issues. More in detail, we extract object-centric event logs by using an abstraction layer of the database, called Graph of Relationships (GoRs), designing blueprints with domain knowledge, and converting the database and blueprint into object-centric event logs.We fully implemented the proposed approach, which can extract object-centric event logs from SAP ERP systems, and evaluate the utility and scalability of the proposed approach.

Type
Journal article
Publication
Journal of Intelligent Information Systems

Practical problem

Before a company can analyze a process such as purchasing or sales with process mining, someone has to pull an event log out of the database of its Enterprise Resource Planning (ERP) system. In an SAP ERP installation that database holds hundreds of thousands of tables, so analysts struggle to find which tables belong to a process, and writing the extraction queries demands deep domain knowledge. Commercial extractors solve this only for the mainstream Purchase-to-Pay and Order-to-Cash processes, are proprietary, and need customization as soon as a process deviates from the template. Worse, they force every event under a single case identifier, which silently drops events, duplicates them (so performance figures become unreliable), or breaks the real ordering between them.

Research questions

  1. How can the relational structure of an ERP database be abstracted so that analysts can identify processes and select the relevant tables without writing queries?
  2. How can table entries be translated into events with activities, timestamps and related objects so that the result is an object-centric event log free of deficiency, convergence and divergence?
  3. Is such an extraction feasible in acceptable time on a real SAP ERP instance, and how useful are the resulting logs?

Contributions

  • A technology-independent database abstraction (tables, entries, object types, transactions, keys, object relationships) and a derived Graph of Relationships (GoR) that supports process identification and table selection.
  • The concept of blueprints that map table entries to primitive events, with a basic blueprint needing no domain knowledge and a domain-knowledge blueprint grouping related entries into one event.
  • A formal conversion from database plus blueprint to an object-centric event log in the OCEL standard, including a projection to a chosen subset of tables.
  • A concrete realization for SAP ERP: the SQL queries that build the GoR from SAP dictionary tables and a classification of SAP tables into change, transaction and object tables.
  • The open-source Interactive SAP Explorer tool (Neo4J graph database, Angular front end, Python extractor on GitHub) plus two extracted O2C and P2P logs published in JSON-OCEL.

Key findings

  • On an educational SAP ERP instance, the full database abstraction (662,161 tables, 934,546 attributes, 9.6 million attribute arcs, 1.67 million relationship arcs) is built in about 24 seconds of query and postprocessing time.
  • Extracting entries from Procure-to-Pay tables is fast (1.25 s for 1,844,532 BSEG payment rows), but computing object relationships dominates runtime: 137.79 s of postprocessing for 2,323,792 BSEG relationships.
  • Starting from the purchasing-document object type EINKBELEG, GoR expansion reaches the expected purchase-requisition, invoice and payment tables, and process identification surfaces twelve processes, e.g. sales documents with 412,228 events across 11 tables.
  • Object-centric directly-follows graphs discovered from the extracted O2C and P2P logs show the document flows and change activities of both processes without convergence or divergence artifacts.
  • The basic blueprint yields activity names of lower quality than ad-hoc commercial extractors, the extractor runs only against SAP ECC on Oracle with memory-bound Python extraction, and table-set completeness remains the user’s responsibility.