Frankly, process mining is just a business analysis to get a handle on what’s going on.
Of course, process mining is able to conduct business analysis much more accurately and efficiently than conventional interviews and observational surveys with field staff, and for this reason, it has rapidly gained attention in recent years.
In many cases, process mining is used in BPR(business process re-engineering) or business process improvement projects.
Therefore, it is important not only to understand the procedure of process mining as an analytical method, but also to understand and utilize the procedure and framework of BPR or business process improvement.
There are various approaches to BPR and business process improvement, but I think it is best to adopt Six Sigma’s methodology as a foundation.
Six Sigma is a business process improvement method developed by Motorola based on the Toyota Production System concept. That is, it includes many frameworks that originated in Japan.
The process improvement project at Six Sigma is known as the “DMAIC”. It is an acronym for the following five phases of activity.
A brief description of the relationship between each activity phase and process mining.
First, identify the processes that need to be improved, as well as potential problems and issues. Basically, problems that can be identified as phenomena, such as excessive overtime and exhausted employees, or an increase in customer complaints due to delivery delays, are identified and identified as problems in which processes are likely to have problems, and are targeted for improvement.
This is the prerequisite phase of the process mining project “scoping” or “analysis planning”.
Collect the data and information needed to understand the current situation. As mentioned earlier, it has traditionally been customary to hold interviews with field staff, conduct observation surveys, and hold workshops with all concerned parties to take inventory of current business processes.
In process mining, in addition to the information gathering methods described above, “Event Log”, which is historical data of business performance on IT systems, is extracted and analyzed.
The process model (as is Process) obtained from process mining enables accurate reproduction of fact-based business processes. However, it is of course impossible to grasp the operations that are being conducted outside of the system. Therefore, complementation through interviews and observational surveys is essential.
We conduct analysis from various perspectives based on information and data obtained in the previous phase.
In the case of a project using process mining, based on a process model derived from an event log, it is possible to easily find bottlenecks where work is taking too long, inefficient, or stagnating, so that it is possible to delve into the analysis of interviews and field investigations where such problems have been identified.
If process mining is not adopted, field interviews must begin with the question, “What are you doing and how are you doing it? (What, How)” However, if you look at the process model, you can ask the question, “Why does this happen? (Why)” making it easier to pursue the root cause.
Once the root causes of problems and issues that appear to be phenomena (Inefficiency, bottlenecks, etc.) have been clarified, specific improvement measures will be formulated and improvement activities will be carried out. Since it is the execution stage, the process mining is retracted to the stage, but the event log is steadily accumulated for the next Control (establishment and management).
The last is to establish and manage improved business processes. You can develop better business processes and deploy them in the field, but if you leave them as they are, you can go back to the old ways.
Therefore, continuous monitoring and appropriate guidance are needed to ensure that new processes take root. By utilizing process mining, it is possible to extract and analyze in real time operation history data that is executed on an IT system on a daily basis, and to quickly find deviations and problem points in an ongoing project and issue alerts, thereby ensuring that new business processes are implemented.
As new IT tools emerge, some people have the illusion that they are a universal solution.
Of course, there’s no one-size-fits-all tool, and since it’s just a tool in the first place, user skills are more important. Even if you have a kitchen knife with the best sharpness, you’ll know that if you’re not used to using a kitchen knife, you can’t cook ingredients well.
Process mining tools are, of course, more than one-size-fits-all; they’re just analytical tools. Most importantly, there is no “What should I do?” solution in the analysis. In fact, the results of the analysis do not explain why “Why is it so inefficient?” or “Why bottlenecks?”.
What can be discovered not only by process mining tools, but also by analytical tools such as BI is nothing more than “location of the problem”. By analyzing the flow diagram drawn from the event log, that is, the “as is process model” from various angles, it is easy to identify where the problem lies in the business procedure.
But they don’t tell us why there’s a problem. By digging deeper into the problem, you can narrow down what is causing the problem.
For example, in the sales order process (O2C), you know that the throughput (overall lead time) from the sales order to the delivery date is longer than expected, and you rate it as “matter”. Using process mining tools, we can analyze by product, customer, etc. to find out which products or customers tend to take longer to deliver.
This is very useful information, but we still don’t know the “Why is that?”. If you don’t ask why, you won’t be able to come up with an effective solution, but you need to interview the person in charge of the site and do some observational research.
The ultimate purpose of process mining is not to conduct analysis, but to solve management issues such as improving business processes and promoting digital transformation (DX).
However, the direct purpose of process mining as an analytical method is to “elucidation of the underlying cause”. We should not immediately consider measures to improve the situation just because we have identified the problem areas through analysis.
For example, suppose you discover that a process is taking too long, or inefficient. So, I think there are people who would rush to take easy improvement measures, thinking that it would be more efficient if we introduced RPA and automated it.
In the first place, I asked a field representative why it took so long, and before the RPA, it was easy to save time by skipping the scheduled steps.
So once you get the results, you have to go back and try to figure out the root causes. Only when the root cause is known, it is possible to plan effective improvement measures, not superficial therapy.
“Digital twin” is not limited to business as a pure technology, but in process mining, it stands for “Digital twin of an organization”.
The literal translation of “Digital twin of an organization” is “one digital half of twins of an organization “.
On the other hand, the other half is “Analog half of twins of an organization” where employees work together in the analog field. However, since much of the work is now performed by IT systems, the work remains as a digital footprints.
By analyzing digital footprints, or “Event Log” through process mining, it has become possible to visualize previously unseen business operations. The flow of business processes can be “discovered” as a flowchart.
With process mining, we can calculate the number of cases processed, the “Processing time (service time)” per activity, and the transition time from the previous process to the next process, i.e., the “Wait Time (waiting time)”, etc., making it easier to identify areas with high workload and bottlenecks with stagnant business.
In addition, it is possible to clearly understand who is in charge of what kind of work and who is collaborating and collaborating with each other through work.
It is important to be able to clarify, based on the facts, the flow of work, the number of processes, the time required, and the functions involved in collaboration that were only vaguely understood. The flow charts and diagrams visualized by process mining are truly “digital twin” that reproduce the way an organization works and the contents of business operations based on digital data.
The benefits of a digital twin are not just accurate fact-based reality. With a digital twin, you can simulate what happens if you delete or change some of the processes, or if you automate some of the processes with the RPA and see the overall impact.
In other words, after examining how to improve business processes to reduce lead time, reduce costs, and eliminate bottlenecks, it is possible to apply the method to an analog twin, that is, a real process.
In addition, by flowing the event log continuously recorded on the IT system into the process mining tool in real time, the work execution situation in the field can be monitored in the digital twin, and the problem can be corrected immediately.
As you can see, process mining is an essential tool and solution for achieving a digital twin.
Recently, there has been increasing interest in process mining as a trump card for business innovation. Many people marvel at the “process model” shown in the process mining demos, or business flow charts automatically created from event logs, saying, “Oh, my God.”.
In contrast to the challenges of traditional approaches to visualizing business processes such as interviews and workshops, you find it great to be able to draw business flow diagrams easily and quickly from data extracted from IT systems.
But everything has a bright side (Bright side) and a dark side (Shadow side). When you operate the process mining tool, the flow of business processes and the relationship between business and the person in charge, which has not been seen until now, are shown clearly in the diagram, is “bright side”. On the other hand, there is a “dark side” that is rarely talked about because it raises the psychological hurdle of introduction.
There are actually two dark sides. There are 2 phases: “data preprocessing” which is the work required before the stage of operating the process mining tool and performing the actual analysis, and “Evaluation and interpretation of analysis results” which is performed after the actual analysis.
It’s easy and fun to navigate through process mining tools and switch between visual screens. On the other hand, data preprocessing and the evaluation and interpretation of analysis results require a great deal of labor. However, this is a phase that cannot be avoided in order to achieve business reform through process mining. This is an inconvenient truth of process mining.
In this article, I will first explain “data preprocessing” which is a pre-process of process mining analysis.
Once a business process to be subjected to process mining is determined, necessary data is extracted from the IT system executing the business process, but the extracted data (raw data: transaction data) cannot be directly uploaded to the process mining tool.
This is because the files to be uploaded to the process mining tool need to be unified into a clean file with all the necessary data items.
In general, the data extracted from the DB in the IT system is dirty data or dirty data, for example, the file is divided by the year, the transaction file and the master file are separated, and there are omissions of data (Blank) and garbled characters.
“data preprocessing” is the process of cleaning such multiple (Often dozens) dirty data and processing them into 1 data file = clean event log.
We’ll discuss how to do this in a separate article, but for example, you might want to delete all data with blanks or enter some correction. To perform these preprocessing operations on 100,000 to 1 million raw data items, a data preprocessing tool “ETL” is basically used.
ETL, which stands for Extract, Tranform, and Load, is a multi-functional tool that literally extracts data, transforms it (Machining), uploads it to other tools, and even provides analysis capabilities, but is used exclusively for data transformation (Machining) in process mining.
The ETL tool I recommend is an open source one called “KNIME”. It’s not localized in Japanese, but it’s free and very intuitive.
KNIME can perform various data processing without programming, so even non-engineers can perform data preprocessing. Of course, if your engineers preprocess data, they can do it faster than KNIME by using SQL, Python, R, or other scripts that you’re good at.
In data mining projects, it is often said that data preprocessing consumes about 80% of the project effort. Process mining analysis and the evaluation and interpretation of analysis results become meaningful only when proper data is prepared.
“Process mining” was born in the late 1990s and last year turned 20 years old. In 2019, a new concept called “task mining” appeared.
In this article, I would like to organize and sort out the differences in purpose and positioning, including “SIEM: Security Information and Event Management”, which is a similar solution to process mining and task mining.
First, the difference between process
mining and task mining. In simple terms, the data to be analyzed is different.
Process mining analyzes the event logs
(transaction data) recorded and accumulated in business systems such as ERP,
CRM, and SFA. The recorded data is based on activities such as “purchase
request” and “purchase approval” when the “send” or
“update” button of the system is pressed, and the granularity of only
the “milestone” of the business Is a rough thing.
On the other hand, task mining analyzes the
detailed operations on PCs that employees operate individually, specifically,
the “PC operation log” that records application launches, file opens, mouse
clicks, copy and paste, etc. Eligible. Compared to the event log extracted from
the business system, it is “atomic” detailed data that cannot be
further decomposed and can be analyzed at the task level. Since these PC
operation logs are not recorded anywhere, install software called sensors or
agents on the PC to be analyzed and actively capture and collect PC operations
as data. A mechanism to accumulate on the server is required.
“SIEM” is a similar solution
adjacent to process mining and task mining. It analyzes security logs, network
devices, and various logs remaining on servers to find security-related issues
such as cyber attacks and data leaks, and manages IT devices as assets. And so
Now, since these solutions basically
analyze data generated in the “workplace”, they can be broadly put into the
framework of “Workplace Analytics”.
Now let’s position process mining, task
mining, SIEM, and their key solutions within the framework of workplace
analytics. (See the figure below)
Look around the double arrow at the bottom
of the figure. Process mining is “process improvement oriented”,
while “SIEM” is “risk aversion and management oriented”.
Task mining is located in the middle. This is because task mining can be used
for attendance management because it allows you to understand the entire daily
work of employees. (In process mining, since only the data of operations performed
on the business system is the analysis target, it is not possible to grasp the
entire business of the day.)
In addition, process mining and task mining
can be surrounded by the framework of “process intelligence”, but SIEM is not
included because “process” is not analyzed.
And process mining is “DX-driven” because it is effective for process reform of the entire company and approach from the viewpoint of digital transformation (DX), while task mining is ultimately an automation at the task level Because it is often aimed at a certain RPA, it can be said that it is “RPA-driven”.
Let’s look at the key solutions in each
category. At this time (February 2020), two key players in the Japanese process
mining market are Celonois and myInvenio. Both tools are enterprise solutions
with rich functions and excellent operability, and the number of enterprises,
especially large enterprises, is increasing. And recently, both tools have
added a “task mining function”. By being able to create not only
event log data from business systems, but also flow charts (process models)
from PC operation logs, it can be said that it meets the analysis needs
necessary for RPA to aim for task-level automation Will be.
In the task mining category, heartcore,
myInvenio’s sole agent in Japan, provides Heartcore Task Mining. In addition,
MeeCap, which has a track record of introduction in the banking industry, has
begun to expand to a process mining function that analyzes event logs from ERP
and other sources.
In the SIEM category, Splunk and Skysea View are known, but Splunk has added a process flowchart function. However, it seems that analysis cannot be performed until the event log is imported.
プロセスマイニング市場はまだまだ新しいため、市場全体を把握できるデータや資料がほとんど存在しません。そんな中、イタリアのITコンサルティング会社、「HSPI Management Consulting」が2018年から毎年発行している「Process Mining: A DATABASE OF APPLICATION」は、プロセスマイニングプロジェクト件数ベースでの概要を伝えてくれる貴重な調査資料です。