BERT (Bidirectional Encoder Representations from Transformers)

BERT (Bidirectional Encoder Representations from Transformers), released by Google in October 2018, was the model that proved pretraining at scale plus task-specific fine-tuning could beat bespoke architectures on essentially every NLP benchmark that existed. The two design choices that mattered: an **encoder-only transformer** (no autoregressive decoder — the model reads the whole input at once) and a **masked-language-modelling** objective (randomly mask 15% of tokens, predict them from both left and right context). This produced general-purpose text embeddings good enough to be reused for classification, entity extraction, question-answering, and search ranking with only a small fine-tuning dataset.

BERT did not disappear when GPT-style decoders became dominant — it quietly remains the default backbone for a wide band of production systems where you need understanding without generation. Semantic search, retrieval re-ranking, toxic-content classifiers, PII detection, entity extraction pipelines, and most of the embedding-for-vector-DB work still runs on BERT descendants: **RoBERTa** (Meta's better-tuned BERT), **DistilBERT** (lighter, faster, 97% quality), **DeBERTa** (disentangled attention), and multilingual variants like **XLM-RoBERTa** and **mBERT** that matter enormously for APAC use cases.

The practical decision split for teams building today: use a BERT-family encoder when the task is **understanding text** (classifying, scoring, embedding for retrieval) and you want low latency, cheap inference, and on-device deployability. Use a decoder-only LLM (GPT, Claude, Llama) when the task is **generating text** or when flexibility across many tasks matters more than raw inference cost. The embeddings that power modern RAG systems are almost always BERT-family under the hood, even when a decoder-only LLM sits in front of them.

For APAC mid-market, the unsexy truth is that 2018-vintage BERT architecture, fine-tuned on your data, is still the right answer for the majority of production NLP work. The hype has moved on; the engineering reality has not.