Where to Source and Structure Data for Process Mining

Where the Data Comes From

Every process leaves a trail in the systems that run it: orders are created in an ERP, tickets move through a service desk, approvals are logged in a workflow tool. Process mining reads that trail. This page covers where to find it and how to shape it into an event log. For the fields every log must contain, see What data you need for process mining; for the accepted file types, see Supported Data Formats.

Two questions decide whether a system is worth extracting from:

• Does it record a timestamp for the step, not just the current state?
• Does it record an identifier you can use as the case ID across the whole process?

If a system only shows the latest state, look for its history or audit trail. Most enterprise systems keep one.

Where Events Are Usually Recorded

System type	Examples	Where the events live
ERP	SAP, Oracle E-Business Suite or Fusion, Microsoft Dynamics	Document and change tables: purchase orders, invoices, and material documents each carry a created or changed date and the user who acted
Service desk and ITSM	ServiceNow, Jira, Zendesk	Audit and history tables that log every field change (for example, ServiceNow sys_audit), plus each task’s own created and updated timestamps
CRM	Salesforce, Microsoft Dynamics	Object field-history tables that keep the old and new value with a timestamp and user
Workflow and BPM tools	Camunda, Power Automate, custom workflow engines	The completed step history of each process instance
In-house applications	Custom web apps, legacy systems	Application logs or an audit table; ask the team that owns the app which table records state changes
Spreadsheets	Manual handoffs, offline approvals	The export itself, as long as the case ID and timestamp columns are filled in consistently

Example: Purchase-to-Pay in SAP

A purchase-to-pay process is spread over several SAP tables, one per step:

Step	SAP table
Purchase requisition	EBAN
Purchase order	EKKO (header), EKPO (items)
Goods receipt	MKPF (header), MSEG (items)
Invoice receipt	BKPF (header), BSEG (items)
Payment	PAYR, REGUH

On SAP ECC you can read these tables with an ABAP report or a custom program; on SAP S/4HANA, use CDS views, an OData service, or SAP Datasphere. Extract one lean query per step, with an activity name, the case ID, and the timestamp (plus user, amount, or vendor if you want them), and combine the results into a single event log:

SELECT 'PO Created' AS Activity, EBELN AS CaseID, AEDAT AS Timestamp FROM EKKO
UNION
SELECT 'Goods Receipt', EBELN, BUDAT FROM MSEG JOIN MKPF ON MSEG.MBLNR = MKPF.MBLNR
UNION
SELECT 'Invoice Posted', BELNR, BUDAT FROM BKPF
ORDER BY CaseID, Timestamp;

Narrow the extract before it grows: filter by company code, document type, or fiscal year instead of exporting everything.

Structure the Export

• One row per event — a state change, not a whole case. Keep the Case ID, Activity, and Timestamp columns, plus any optional columns (user, cost, amount) you want to analyze.
• Consistent case IDs across systems — when a process spans an ERP and a CRM, reconcile the identifiers so every event ends up under one case.
• One time zone — normalize timestamps to UTC or to a single zone so events from different systems stay in order.
• Consistent activity names — “Approve Order” and “Order Approval” are the same step; pick one wording per step.
• Leave out noise — test documents, cancelled records, and internal system steps add rows without adding insight.

Header row, delimiters, and timestamp formats are detected on upload; the full requirements are listed in Supported Data Formats. If a step is missing from your systems altogether, Data Cleaning and Preparation covers how to fill the gap.

Handle Personal Data Before You Export

Event logs usually carry names, customer numbers, or user IDs. Anonymize or mask what the analysis does not need, limit who can download the extract, and follow the retention rules that apply to your organization (for example, GDPR). ProcessMind keeps datasets per tenant, and access follows the user’s role and seat — see Users, Roles & Permissions.

Next Step: Upload and Map

Once the log is structured, upload it and map the columns to the dataset:

1. Configure your dataset — map Case ID, Activity, and Timestamp, and check the Data Quality badge.
2. Map your data to the model — connect attributes to activities and connections.
3. For continuously growing data, use incremental (delta) uploads instead of reloading everything.

Ready-made sample logs are available in Example Files if you want to try the flow before extracting your own data.

Related topics

• What data you need for process mining - the three columns every log needs
• Supported Data Formats - file types and structure requirements
• Data Cleaning and Preparation - clean the extract and fix the usual problems before upload
• Configuring Your Dataset - map and validate the upload