Neural Network

A neural network is a machine-learning model composed of layers of interconnected nodes ("neurons") that compute weighted sums of their inputs, apply a nonlinear activation function, and pass the result forward. Learning happens by adjusting the connection weights via backpropagation of error gradients — the algorithmic discovery (Rumelhart, Hinton & Williams, 1986) that made multi-layer networks practical. The biological inspiration gave the technique its name, but modern networks have diverged considerably from biology; the useful part of the analogy is the layered-function-composition idea.

The field moved through several architectural eras. **Feed-forward MLPs** dominated early applications — tabular classification, regression, basic pattern recognition. **Convolutional networks** (LeCun 1989, exploded post-AlexNet 2012) specialised for images by exploiting spatial locality and translation invariance. **Recurrent networks** (LSTMs, GRUs) handled sequences until Transformers (2017) displaced them for nearly every application. Today's state-of-the-art is dominated by **Transformers** and their cousins — decoder-only LLMs, ViTs for vision, diffusion-backbone networks for generation, and the emerging state-space architectures (Mamba family) as a competitor on long sequences. The common thread is gradient-descent-trained differentiable functions over large datasets.

For APAC mid-market teams, the relevant framing is that neural networks are no longer the novel exotic choice they were a decade ago — they are the default workhorse for anything involving text, images, audio, video, or sufficiently-rich tabular data. The strategic decisions have shifted up the stack: which pretrained architecture to adopt, how much of it to fine-tune, what form of adaptation (prompting, RAG, fine-tuning, adapters), what MLOps infrastructure to run it on. Rolling your own architecture from scratch is rarely justified outside research settings.

The non-obvious practical note: **neural networks are data-hungry and compute-hungry in ways that shift over time**. A decade ago that was the central obstacle. Foundation models invert the calculus — someone else paid the pretraining cost, and your job is to adapt their work cheaply. Budget your team's effort for data curation and evaluation design, not for training from scratch, unless you have a genuinely unique data asset that no pretrained model has seen.