ProcessM-NoSQL — PQL Query Engine for Process Mining Event Logs

A custom SQL-like query engine for PQL (Process Query Language), using ANTLR4 to parse queries and Apache CouchDB as the underlying document database for process-mining event logs in the XES format.

Bachelor's thesis project in Computer Science — Poznań University of Technology.

Overview

Process-mining event logs in the IEEE XES format are hierarchical (log → trace → event) and schema-flexible. This makes them difficult to represent with a fixed relational schema, especially when querying data across different levels of the hierarchy.

This project combines Apache CouchDB with a purpose-built PQL query engine. It allows users to write declarative, SQL-like queries instead of working directly with CouchDB's native JSON query syntax.

Note: The PQL language specification was created by the ProcessM research project. The original contribution of this repository is the engine that parses, translates, and executes PQL queries against a NoSQL document database.

Key Features

PQL Query Engine

• ANTLR4-based parser for the PQL language.
• Query translation from PQL into CouchDB Mango _find selectors.
• Query operations including filtering, aggregation, grouping, sorting, and per-scope pagination.
• Cross-scope queries over log, trace, and event attributes.

XES Processing

• Streaming parser based on StAX for processing XES XML without loading the entire document into memory.
• Format support for .xes, .xes.gz, and .zip archives.
• Batched and parallelized ingestion into CouchDB.
• XES export that reconstructs the hierarchical event-log structure from the stored documents.

Application

• Spring Boot REST API for interacting with the query engine.
• Built-in web GUI for importing logs and executing queries.
• Docker Compose setup with a pre-configured CouchDB instance.

Architecture

flowchart TD
    Client(["Browser GUI or HTTP client"]) --> Ctrl["PqlRestController<br/>Spring Boot REST API"]

    Ctrl -->|"POST /api/pql/import"| Detect["LogFileDetector"]
    Detect --> Parse["StaxXesParser<br/>(streaming StAX reader)"]
    Parse --> Map["StreamingXesToCouchDBMapper<br/>(flatten, batch, parallel write)"]
    Map -->|"_bulk_docs"| DB[("Apache CouchDB<br/>event_logs")]

    Ctrl -->|"POST /api/pql/query"| Grammar["AntlrPqlParser<br/>(ANTLR4 grammar)"]
    Grammar --> Translate["PqlToCouchDbTranslator<br/>(query → Mango selector)"]
    Translate -->|"_find"| DB
    DB --> Exec["PqlQueryExecutor<br/>(grouping, aggregation,<br/>sorting, cross-scope logic)"]
    Exec --> JSONOut["JSON result"] --> Ctrl

    Ctrl -->|"GET /api/pql/export/{logId}"| Rebuild["XesExporter"]
    DB --> Rebuild
    Rebuild --> XESOut[".xes file"] --> Ctrl

Loading

PQL Query Examples

PQL provides a declarative interface for querying process-mining data. The l:, t:, and e: prefixes refer to log, trace, and event scopes respectively.

-- Event frequency, most common first
SELECT e:name, count(e:name)
GROUP BY e:name
ORDER BY count(e:name) DESC

-- Cross-scope filter
SELECT e:name, t:name
WHERE t:name = 'Order_2'

-- Regex filter on event names
SELECT e:name, t:name
WHERE e:name MATCHES '.*(Invoice|Goods).*'

-- Process-variant mining using the ^ hoist operator
SELECT count(t:name)
GROUP BY ^e:name
ORDER BY count(t:name) DESC

-- Per-scope hierarchical pagination
SELECT e:name, t:name
ORDER BY t:name ASC, e:timestamp ASC
LIMIT e:10 OFFSET t:1, e:5

REST API

Base path: /api/pql

Method	Endpoint	Body	Description
POST	/query	text/plain	Executes a PQL query and returns the result as JSON
POST	/import	multipart/form-data	Imports an event log (.xes, .gz, .zip) and returns its logId
GET	/export/{logId}	—	Streams the log back out as a .xes file
DELETE	/deleteAll	—	Drops and recreates the database

Example:

curl -X POST http://localhost:8080/api/pql/query \
  -H "Content-Type: text/plain" \
  -d "SELECT e:name, count(e:name) GROUP BY e:name ORDER BY count(e:name) DESC"

Tech Stack

Layer	Technology
Language	Kotlin 2.1.10
JVM	JDK 23
Framework	Spring Boot 3.4.0
Query parsing	ANTLR4 4.13.1
Database	Apache CouchDB 3.3.3
HTTP client	OkHttp 4.12.0
JSON	Gson 2.11.0
Archive handling	Apache Commons Compress 1.26.0
Build tool	Gradle 8.10 (Kotlin DSL)
Testing	JUnit 5.10.0
Infrastructure	Docker / Docker Compose

Project Structure

src/main
├── antlr/                 # PQL lexer and parser grammars
├── kotlin/
│   ├── app/               # REST controllers, log importer, XES exporter
│   ├── db/                # CouchDB HTTP client and Mango integration
│   ├── input/             # Streaming XES parser
│   ├── mapper/            # XES-to-CouchDB document mapping
│   └── pql/               # Parser listeners, translators, and query executor
└── resources/static/      # Built-in web GUI

Getting Started

Prerequisites

• JDK 23
• Docker and Docker Compose
• Git

1. Clone the repository

git clone https://github.com/sa111nt/ProcessM-NoSQL-Diploma.git
cd ProcessM-NoSQL-Diploma

2. Start CouchDB

cd docker
docker compose up -d
cd ..

The Docker Compose setup includes an initialization script that configures CouchDB for bulk event-log ingestion.

3. Run the application

./gradlew bootRun

4. Open the application

Open http://localhost:8080 in your browser to access the built-in GUI.

From there you can:

1. Upload an XES event log.
2. Execute PQL queries.
3. Inspect query results.
4. Export an imported log back to XES.

Tests & Evaluation

The repository includes an integration test suite that runs against a live CouchDB instance.

Key evaluation areas include:

• Scalability benchmarks: query execution across datasets ranging from small sample logs to approximately 1.05 million events, including aggregation, cross-scope filtering, regex filtering, sorting, and grouping.
• Storage overhead: measurement of the storage cost introduced by event denormalization in CouchDB.
• Language compliance: regression tests covering PQL parsing, translation, and query execution.

Run the test suite with:

./gradlew test

Some scalability and storage experiments require the sample datasets available under src/test/resources/.

Known Limitations

This is an academic thesis project rather than a production-ready system. Known limitations include:

• CouchDB credentials are currently hard-coded as admin / admin for local evaluation.
• The test suite is integration-based and requires a running CouchDB instance; there is no isolated unit-test layer.
• The REST API currently allows cross-origin requests from any origin (@CrossOrigin(origins = ["*"])).
• LogFileDetector recognizes OCEL_JSON, but actual ingestion is currently limited to XES formats.
• No CI/CD pipeline is configured in the repository.

Acknowledgments

• PQL Language Specification: ProcessM Research Project
• Event Log Format: IEEE 1849-2016 XES Standard

Author

@sa111nt

Bachelor's thesis project, Poznań University of Technology.