Feature Engineering

Feature engineering is the craft of transforming raw data into features — numerical, categorical, or structured inputs — that expose the relevant structure to a machine-learning model. For most of ML history, feature engineering was where projects were won and lost: the right log transform, ratio feature, aggregation window, or interaction term could move a model's accuracy more than any algorithmic change. Experienced practitioners built libraries of domain-specific feature recipes — customer RFM features, time-series lag features, text TF-IDF features, image SIFT/HOG features — that the underlying model could consume.

The advent of **deep learning** and **foundation models** reduced the prominence of hand-crafted feature engineering for unstructured data. A CNN learns its own image features; a Transformer learns its own text features; a pretrained foundation model already knows features for nearly any modality you feed it. The centre of gravity shifted from feature design to **representation selection**: which pretrained model's embeddings to use, how much to fine-tune, what retrieval context to feed in. Feature engineering did not die — it moved up the stack, from pixel-level filters to prompt design and retrieval query construction.

For tabular data — which is the dominant data modality at most APAC mid-market enterprises — feature engineering remains decisive. XGBoost, LightGBM, and CatBoost continue to beat most neural architectures on tabular problems, and their performance depends heavily on the features you hand them. Time-based features (day-of-week, hour-of-day, time-since-event), categorical encodings (target, mean, or frequency encoding for high-cardinality categoricals), and domain-specific transforms (log of monetary values, ratios of related columns) routinely move models more than hyperparameter tuning does. Automated feature engineering (Featuretools, DataRobot) can help but rarely replaces domain knowledge.

The non-obvious operational warning: **features that leak the label kill models silently**. A feature that encodes information from the future, or from a downstream consequence of the label, will produce spectacular in-sample performance that collapses in production. The classic example is "account status" in churn prediction — if the account is already closed, the feature tells the model the answer. Feature pipelines need careful temporal auditing ("would this feature have been available at prediction time?") before any new signal enters the model.