# Extracting declarative constraints for process modeling from natural language descriptions with large language models

- Type: Journal article
- URL: https://www.gyunam.com/publication/2026-dss-declarativenl/
- Date: 2026-01-30
- Authors: Gyunam Park, Julian Kofferath, Minsu Cho
- Venue: Decision Support Systems
- DOI: https://doi.org/10.1016/j.dss.2026.114627
- PDF: https://www.gyunam.com/publication/2026-dss-declarativenl/paper.pdf
- Research lines: [Neuro-symbolic process learning](https://www.gyunam.com/research-line/neuro-symbolic-process-learning/index.md)

## Abstract

With the growing availability of unstructured text data in organizations, automating the extraction of process models from natural language descriptions has become increasingly crucial. Traditional rule-based techniques face challenges such as limited generalization and constraints to narrow sets of control-flow constructs. We present a systematic empirical study of Large Language Models (LLMs) for translating natural language sentences into declarative process constraints using the Declare modeling language. Through comprehensive evaluation across seven LLM architectures, we demonstrate that fine-tuned models achieve 97.8% template accuracy compared to 53.8% for existing rule-based approaches. Our fine-tuning approach substantially outperforms prompting techniques (99.4% vs. 56.5% template accuracy), establishing clear guidance for practical deployment. We contribute a benchmark dataset of 969 sentence-constraint pairs across 11 Declare templates, direction-sensitive evaluation metrics, and complete reproducibility materials. The extracted constraints enable practical decision support applications including compliance monitoring, conformance checking, and automated governance dashboards.

## Practical problem

Companies describe how their work should be done in text documents, handbooks and regulations, but analysis tools need formal process models, and the domain experts who know the processes rarely have the modeling skills or time to build them. Automated extraction tools based on hand-written language rules exist, yet they break on unfamiliar sentence structures or noisy wording and recognize only a handful of ordering constraints. There is also no shared dataset or metric for judging such tools, so it is hard to tell which one works. Analysts therefore still translate text into models manually before they can check compliance or compare documented and actual behavior.

## Research questions

1. Can a fine-tuned large language model translate natural language sentences into Declare constraints more accurately than rule-based pipelines and than prompting approaches?
2. Can such a model cover a broader range of Declare templates than existing extraction methods support?
3. How can constraints extracted sentence by sentence be merged into one coherent Declare process model with consistent activity labels?

## Contributions

- A pipeline that translates each sentence of a process description into a Declare constraint with a fine-tuned decoder-only large language model (LLM), matches activity labels across constraints and merges them into a Declare process model.
- Parameter-efficient fine-tuning with Low-Rank Adaptation (LoRA) of five open seven-to-eight-billion-parameter models (Gemma 7B, Mistral 7B, Falcon 7B, Llama-2 7B, Llama-3 8B) on a question-and-answer prompt format.
- A publicly released benchmark dataset of 969 sentences paired with expert-annotated Declare constraints covering 11 templates across healthcare, compliance, education and finance.
- An evaluation metric that weights template errors above activity errors and penalizes direction errors, which the slot-filling precision and recall used previously cannot detect.
- Zero-shot and few-shot prompting strategies for matching semantically equivalent activity labels (verb-, noun- and gerund-based) across extracted constraints.

## Key findings

- In 10-fold cross-validation on the 969-sentence benchmark, all five fine-tuned models reach template accuracy between 0.970 and 0.980; Gemma 7B scores highest with F1 0.884 (alpha 2), 0.918 (alpha 4) and template accuracy 0.980.
- Against the rule-based state of the art of van der Aa et al. on validation set V1 (80 sentences; Init, End, Response, Precedence), fine-tuned Gemma 7B reaches template accuracy 0.988 versus 0.538; on V2 (45 sentences from their data) 0.956 versus 0.822.
- Gains are largest for unary templates: on V1, Init improves from 0.273 to 0.910 and End from 0.0 to 1.0; the baseline ties on V2 Precedence (0.963) only because its fallback rule emits precedence constraints by default.
- Fine-tuning beats prompting the same Gemma 7B over all 11 templates: template accuracy 0.994 versus 0.459 for few-shot and 0.565 for few-shot chain-of-thought prompting; F1 (alpha 2) 0.911 versus 0.268 and 0.598.
- Limitation: the method assumes one constraint per sentence, so a sentence describing two constraints collapses into one; branched responses and time conditions are unsupported, and remaining template errors stem from linguistic ambiguity.