Data Mining

Data mining — today more often called data science — is about explaining the past and predicting the future by means of data analysis. Organizations accumulate years of operational data; data mining extracts valuable, actionable knowledge from it, and that knowledge turns into more clients, more sales, better engineering, and better medicine.

It is a genuinely multidisciplinary field. No single discipline owns it, and a working data scientist borrows from all of them daily.

The five disciplines behind data mining

Statistics supplies the inferential rigor, machine learning supplies the algorithms, and database and warehousing technology supplies the data itself. AI is the umbrella ambition that ties them together.

The Data Mining Map

Data mining is a process, not an algorithm. A project moves through six stages: Problem Definition, Data Exploration, Data Preparation, Modeling, Evaluation, and Deployment. This lineage descends directly from CRISP-DM (Cross-Industry Standard Process for Data Mining), the industry-standard process model published in the late 1990s, which remains the most widely used framework for structuring analytics projects.

Two loops matter. First, the process is iterative: exploration and evaluation routinely send you back to preparation or even to the problem definition. Second, the work splits into two goals — explaining the past through descriptive analysis, and predicting the future through modeling.

The nine areas of this guide

• Foundations — what data mining is and how a project is framed.
• Data Exploration — univariate and bivariate analysis.
• Data Preparation — encoding and binning.
• Classification — predicting categories, from ZeroR to SVMs.
• Regression — predicting numbers.
• Clustering — finding structure in unlabeled data.
• Association Rules — discovering interesting co-occurrences.
• Model Evaluation — measuring whether a model is any good.
• Further Readings — where to go deeper.

In practice

The map is tooling-agnostic, but today each stage has a default stack: pandas and Polars for exploration and preparation, scikit-learn for classical models, XGBoost and LightGBM for tabular prediction, and MLflow for tracking experiments across the loop. CRISP-DM’s iterative spirit survives intact in modern MLOps — deployment is a stage of the process, not the end of it.

Common pitfalls

• Jumping to modeling before defining the problem — the most common way projects fail.
• Treating the process as a straight line instead of an iterative loop.
• Confusing explaining the past (description) with predicting the future (generalization); they require different evidence.
• Ignoring data quality until preparation, when it should shape the problem definition itself.

Summary

Data mining is a multidisciplinary process for turning stored data into knowledge: explain the past, predict the future. The map — problem definition, exploration, preparation, modeling, evaluation, deployment — descends from CRISP-DM and organizes every page of this guide.