“Before starting a new project, we should verify the available data for correctness and completeness”. With this attitude, many logistics analysis has come to a premature standstill. It seems so obvious, but is this attitude really helpful? This article presents an alternative view, based on our logistics experience.
Logistic processes generate data. Almost every sequential step in a logistic process creates at least a timestamp. The main purpose of these registrations is just triggering the next step in the process. When that is achieved, most registrations lose their primary value.
However, these data still contain valuable information about the course of our processes in the real world. Analyzing process data provides unexpected understanding of bottlenecks and obstacles, unlocking potential process improvements. Now we are talking about Data Discovery, as opposed to the more traditional approach of Business Intelligence.
Business Intelligence provides answers to management questions, such as “what service level did we achieve”.
Data Discovery digs deeper, it facilitates a quest, focused on solving a real-world problem, but meandering beyond preset boundaries and beyond known hypotheses. An example for Data Discovery could be: a data analysis, searching for effective levers to improve the service level.
In our quest we should experiment, exploiting the dimensions in our data. In this context each column in the data matrix can serve as a dimension. For example we can visualize the achieved service level as a function of the region, or a function of the customer, the product weight, the price, or whatever column we wish to test.
Many IT projects do not achieve their goal because the process-owner has not been the involved as closely as needed. A good understanding of the real-world process-flow is a pre-condition for assessing the project outcome against practical use and informative value. The direct dialog between analyst and process owner leads to new insights and feasible solutions.
Often we see logistics IT projects being postponed because the data quality is disputed. Is it essential to wait for perfect data before starting a logistic analysis? No! It is almost unachievable to obtain a ’clean‘ dataset. While during the analysis most ’dirty‘ data will show up automatically, allowing you to correct, ignore or delete it in your interpretation of the outcome.
Try to plot a dataset in a scatter diagram, using colors to represent a specific attribute in a data field. Most likely the relevant correlations will become evident, while the ’dirty‘ data will appear as outliers at the corners of the diagram.
Data Discovery is just like laying-out a large jig-saw puzzle. You just have to make a start. It does not help to check the number of pieces first. The result of your count is not really relevant. Suppose you count 1002 pieces, what would you do? It wouldn’t stop you solving this puzzle.
Imagine yourself putting a puzzle together of a nice mountain view. You could start with the horizon, but you will never start with the blue sky. How would you approach this in the real world? First you will sort the pieces, selecting colors and patterns. Then you might start with the lower edges or other parts of the puzzle that are easier to combine. Gradually the picture will become clear, long before you finish the blue sky.
With the help of good Data Discovery software it becomes easier to combine, sort and select your data. That being done, the valuable work can start: putting together the interesting parts of your jig-saw.
At Districon we have often derived valuable results from large, raw logistic datasets. At the start of the logistic analysis we do not worry too much about data-completeness and data-quality. During the analysis we get a better understanding of the quality and we take appropriate measures. We prefer to go for the information-treasure hidden in the data, rather than burning our energy pursuing a perfect data-universe with limited value.
Our advice:
- Focus on available data and start creating value
- Assure the involvement of the process owner. Dialogue is the basis for insight and understanding.
This is often sufficient for an adequate logistic analysis, leading to the right decision. Gathering data can never be the goal by itself; enabling good decisions is the real game.