Process Mining Performance: Benchmarks and Tips

Process mining performance depends on more than event count. File size, case count, query shape, and network throughput all affect how long you wait. The benchmarks below cover datasets from 100K to 100M events. They were run on production AWS infrastructure in March 2026, using the file formats and test conditions described here.

What drives process mining performance?

Four factors shape performance: the amount of data, the structure of the event log, the query being run, and the environment used to transfer and process it.

• Event count and case count: More events generally mean more data to upload and process. Case count determines how many process instances your analysis covers.
• Query shape: A summary statistic and a process flow require different calculations. Queries that calculate transitions between activities can take longer than queries over pre-aggregated data.
• Activity count and case length: More distinct activities and longer cases increase the work involved in calculating flows and variants.
• File size and network speed: These determine upload time. A wide CSV with long text fields can be much larger than a compact columnar file containing the same events.

Event count alone cannot predict how long a dataset will take. Use the benchmarks as reference points, then test the event log and queries you plan to use. That is the most useful form of process mining performance analysis for your environment.

The data pipeline: where does the time go?

A dataset passes through three stages: upload, preprocessing, and recalculation when you change the process model.

1. Upload: Your file travels over the internet to ProcessMind’s cloud infrastructure. Network throughput and file size determine transfer time. For example, a 50 million event CSV file of 11 GB takes about 2 minutes at 1 Gbps, 18 minutes at 100 Mbps, or over 3 hours at 10 Mbps. The same data in a 1.7 GB Parquet file takes about 19 seconds, 3 minutes, or 28 minutes at those speeds.
2. Preprocessing: After upload, the system converts the data into optimized columnar storage, indexes events, calculates activity transitions and process variants, and generates summary statistics. In the benchmarks below, this takes 35 seconds for 100K events and up to 2.5 minutes for 100M events.
3. Model changes: When you change activity mappings or add or remove activities, ProcessMind updates the model-dependent calculations. In the benchmarks below, this takes about 14 seconds for 1M events and 52 seconds for 100M events. Filter changes are instant.

For large datasets, reducing file size is often the most effective way to shorten the initial wait. After preprocessing, query performance depends on what your dashboard needs to calculate.

How does query performance vary?

Different queries have different costs. These characteristics help explain what to expect and how to shape your data.

• Activity count: Models with 10–20 distinct activities are easier to analyze and display. Above 50 activities, process flows take longer to calculate and can become harder to read. Group related steps when that accurately reflects the process.
• Variant diversity: A process where most cases follow a small number of paths is generally faster to analyze than one where every case follows a different path. High variation may be meaningful, but it can increase query time.
• Column count: The core fields, CaseId, Activity, and Timestamp, are needed for process analysis. Additional attributes support filtering and categorization, but add data to transfer and scan.
• Case length: A case with 50 events requires more work than one with 5. Processes with cases spanning hundreds of events can take longer to query.

Dashboard response times

These medians come from multiple benchmark runs. Each dashboard component queries independently and loads in parallel.

Statistics stayed at about 0.6 seconds across the tested sizes. Process Flow took longer as event count grew because it calculates transitions between activities. Variants and Categories scaled moderately, while the Data Browser stayed responsive through pagination. Animation time varied with the number of active cases displayed.

At 1M to 10M events, measured response times ranged from 0.6 to 3.4 seconds, depending on the component. At 50M events, Process Flow and Categories took about 5–6 seconds. These are benchmark results, not a guarantee for every dataset or environment.

Benchmark data: what were the test conditions?

These benchmarks were run on production AWS infrastructure in March 2026. Upload and preprocessing times are averages across multiple test runs. Dashboard response times are medians across multiple runs. The test covered more than 50 query types per dataset size.

The event counts below are the actual counts in the test files. File sizes are for uncompressed CSV with a typical event log schema: CaseId, Activity, Timestamp, and 5–8 business attributes. Your file size and results will vary with column count, content, network conditions, distance to the data center, and current load.

Upload and preprocessing times

The measured 1 Gbps transfer rate was 88 MB/s to AWS eu-central-1. Other speeds are extrapolated using practical throughputs: about 5 MB/s at 50 Mbps, 10 MB/s at 100 Mbps, and 1 MB/s at 10 Mbps. Actual throughput depends on your network and conditions.

In this test, preprocessing ranged from 35 seconds to 2.5 minutes. Upload time grew with file size, making it the largest part of the wait for large files.

File format: Parquet and ORC versus CSV

ProcessMind supports CSV, Parquet, ORC, Excel, and XES. This file-size comparison uses the same event counts and schema as the benchmark above.

In these tests, Parquet files were about 85% smaller than CSV, and ORC files were about 82% smaller. Both are columnar formats with built-in compression. Data tools such as Spark, Databricks, dbt, and BigQuery may support exporting to these formats.

Preprocessing times were similar for Parquet, ORC, and CSV. CSV.GZ took longer at larger event counts because gzip files cannot be split for parallel decompression.

Total time combines upload and preprocessing. These examples use a 100 Mbps connection.

For large datasets, Parquet and ORC reduce transfer time because their files are smaller. CSV remains a straightforward option for smaller datasets or when your export process does not support columnar formats. CSV.GZ can reduce transfer time on a slow connection, but its decompression cost increases with dataset size.

Which query patterns can slow analysis?

Some data and dashboard patterns require more work than others. Use these checks to identify likely sources of slower queries.

• Too many activities: A process flow with more than 50 distinct activities can take longer to calculate and may be difficult to interpret. Group related steps where appropriate.
• Many unique variants: When cases follow many different paths, variant calculations have more work to do. High variation can also point to process behavior worth investigating.
• Long cases: Cases with many events require more processing than short cases.
• Unnecessary columns: Extra attributes increase file size and can add work when queried. Keep the fields that support your analysis.
• Large, crowded dashboards: Each chart or tile runs its own query. Keep the components that answer a clear question and put other views on a separate dashboard.
• Expensive visualizations: On large datasets, charts with many categories can take longer to render. Use a summary view when you do not need a detailed breakdown.
• Animation with many active cases: Animation time depends on how many cases are displayed. Reduce the speed or turn off tails and effects if you only need to inspect the flow.

How should you choose the right data size?

Start with the smallest dataset that can answer your question. A focused dataset is quicker to upload and easier to validate. Expand it when you need broader coverage, rare events, or longer-term trends.

Build and validate with a smaller dataset

Start with a recent sample, configure your model, and check the data before scaling up.

1. Start with a sample. For example, extract 1M events from a recent three-month period. In the benchmark, upload took 3 seconds at 1 Gbps or 22 seconds at 100 Mbps, followed by 55 seconds of preprocessing.
2. Build your model. Configure activities, filters, and views. Model changes took about 14–20 seconds for the tested datasets from 1M to 10M events.
3. Validate the findings. Check whether activity names and process paths make sense, and resolve data quality issues.
4. Add more data if needed. Increase the event count when you need to investigate rare cases or compare longer time periods.

The table compares the initial wait at different event counts over a 100 Mbps connection.

The right event count depends on your question. A smaller dataset may be enough for common process patterns, but you may need more data to study rare events or meet a requirement for complete population coverage.

Segment the data when the question is specific

If you are investigating one country, department, or product line, start with that segment rather than combining unrelated groups. Segmentation can make comparisons clearer and reduce the amount of data you need to process.

For example, the benchmark describes a European logistics dataset with 42M shipment events across 8 countries:

• All countries: 42M events, 9.3 GB, 16 minutes to upload at 100 Mbps, and 2 minutes to preprocess.
• Germany: 8.5M events, 1.9 GB, 3 minutes to upload, and 1.5 minutes to preprocess.
• Netherlands: 3.1M events, 690 MB, 1 minute to upload, and 1 minute to preprocess.

You can compare segments after validating each one. Useful dimensions include geography, organization, product, time period, and customer segment.

Filter for the cases you need

If you are investigating exceptions, exporting only relevant cases can reduce the dataset substantially. For example, the benchmark describes a purchase-to-pay process with 1.2 million purchase orders and 8.4M events. It reports that 96,000 orders, or 8%, had exceptions such as rejections, returns, duplicate invoices, or missing approvals.

For that analysis, exporting the exception cases reduces the dataset from 8.4M events to 670K events. The reported file size falls from 1.9 GB to 150 MB, and upload time at 100 Mbps falls from 3 minutes to 15 seconds. Exporting the same data as a 15 MB Parquet file takes under 2 seconds to upload in the benchmark.

Filter by status, activity, case duration, time period, or business unit when those fields match the question you are investigating.

Select only the columns you need

Every column contributes to file size. Long text fields, redundant identifiers, audit fields, and technical metadata may not be useful for your analysis.

The benchmark describes an SAP export of 1.8M purchase order events with 45 columns. Reducing it to 12 essential columns lowered the file size from 2.1 GB to 380 MB. Exporting those columns as Parquet reduced it further to 58 MB. The reported upload time at 100 Mbps fell from 3.5 minutes to 6 seconds.

Keep the fields that support your analysis, including CaseId, Activity, Timestamp, and relevant business attributes such as status, amount, category, or region. Remove personal information when it is not needed for the analysis.

When should you use a larger dataset?

A larger event count may be necessary when your question depends on coverage or time span.

• Rare events: Finding exceptions that occur once in 100,000 cases needs a population large enough to contain them. If an exception occurs 0.01% of the time and you want to study 50 of them, you need around 500K cases.
• Low-frequency variants: Variants that occur 0.1% of the time can be invisible in a 1M event sample and clearly measurable in a 50M event population.
• Compliance and audit: Some requirements call for complete population coverage rather than sampling.
• Long-term trends: Comparing Q1 2024 with Q1 2025 and Q1 2026 needs data from all three periods. Delta loading builds that coverage incrementally.

For datasets above 50M events, the benchmark suggests using Parquet, uploading through the API, and using a fast network connection where available. The 50M event CSV in the test was 11.1 GB, compared with 1.7 GB in Parquet.

How can you keep the model and interface responsive?

Data volume is only part of the picture. The process model and dashboard design also affect how quickly you can work.

• Keep the model readable: Break large processes into modular Subprocesses. A canvas with many visible elements can take longer to render and be harder to navigate. Use auto-layout after structural changes.
• Choose dashboard components carefully: Each chart and tile requires a calculation. Keep the views that answer your current questions.
• Use charts suited to the data: On large datasets, a visualization with many categories may take longer to calculate. Use a summary when you do not need a detailed breakdown.
• Limit filters to the question: Filters can help narrow an analysis. Avoid combining filters that do not contribute to the question you are answering.
• Adjust animation settings: Animation cost increases with the number of active cases. Lower the speed or turn off tails and effects when you only need to inspect the flow. See Process Animation.
• Recalculate only what changed: Model changes update model-dependent calculations without repeating the full preprocessing step. In the benchmark, changes took from about 14 seconds for 1M events to 52 seconds for 100M events.

How does delta loading reduce upload time?

Delta loading lets you append new events to an existing dataset instead of uploading the full dataset again. It reduces the amount of data transferred when you refresh a growing event log.

After a delta upload, preprocessing runs again on the combined dataset. In the benchmark, that took 1–2.5 minutes. The upload savings come from transferring only the new events.

A delta file must use the same file format and column structure as the original upload. See the incremental data loading guide for details.

When should you use an API upload?

For recurring uploads or files of several gigabytes, an API upload can be easier to manage than a browser transfer. A script or command-line tool can fit into an ETL pipeline, support scheduled uploads, and show transfer progress.

ProcessMind provides presigned URLs for direct uploads to cloud storage. You can copy the upload URL from the dataset settings menu in the ProcessMind UI.

# Upload a Parquet file directly using a presigned URL
curl -X PUT "$PRESIGNED_URL" --upload-file data.parquet

See the API documentation for Bash, JavaScript, and Python examples, including how to obtain presigned URLs and upload delta files.

When should you split a dataset?

Split a dataset when a specific analysis only needs a subset of the data, or when separate groups need to be compared independently. You can segment by geography, department, product line, time period, or case type, then analyze each dataset on its own.

Keep the data together when your question depends on comparing groups or tracking the full process across them. Before splitting, check whether the split changes the meaning of the case or removes events needed to understand its path. For example, a time-based split should account for cases that begin in one period and finish in another.

Test performance with your own data

Benchmarks help you plan, but they cannot predict results for every event log, network, or query. Test the dataset you expect to use, then check upload time, preprocessing time, and response times for the dashboard views that matter to you.

Read the data preparation guidance first if you need help shaping your event log. You can also review how to analyze your process and how to continuously monitor your process.

Load your largest dataset and try it

Test it against your own data volume