A neural network is a parameterised function. It transforms input features through layers of weighted operations and nonlinear activations, then produces an output such as a probability, numeric estimate, embedding or generated sequence. “Deep” learning usually means that the model contains multiple representation-building layers; it does not mean that the result is automatically intelligent, correct or suitable for production.
The useful question is not whether a model sounds advanced. It is whether its inputs, target, evaluation and operating controls fit a defined decision or workflow.
Follow one training step precisely
Training is easier to reason about when four operations are kept separate:
- Forward pass: the current parameters transform a batch of inputs into predictions.
- Loss calculation: a chosen loss function measures disagreement between predictions and the training target.
- Backpropagation or automatic differentiation: the system applies the chain rule through the recorded computation to calculate gradients of the loss with respect to parameters.
- Optimiser step: an algorithm such as stochastic gradient descent or Adam uses those gradients and its own state to update the parameters.
Backpropagation does not select the learning rate and does not itself update weights. Conversely, an optimiser needs gradients but is not the mechanism that differentiates the model. PyTorch documents these responsibilities separately in its autograd mechanics and optimisation API.
A simplified iteration looks like this:
clear old gradients
predictions = model(batch.features)
loss = loss_function(predictions, batch.targets)
compute gradients of loss
optimiser updates parameters
record metrics and diagnosticsFrameworks may combine or reorder implementation details, use gradient accumulation, mixed precision or distributed execution, but the conceptual distinction remains important when debugging exploding gradients, stale accumulation or an unexpected update.
Layers learn representations, not explanations
A dense layer combines its inputs and parameters, while an activation such as ReLU introduces nonlinearity. Convolutional layers encode useful locality assumptions for grid-like data. Recurrent designs maintain a sequential state. Attention allows elements to condition on other elements and underpins modern transformer architectures.
These are design biases, not proof that a model has learned the intended concept. A classifier may exploit a watermark, scanner type, postcode proxy or annotation habit instead of the phenomenon the team meant to model. Inspect slices, counterexamples and failure modes rather than treating a high aggregate score as an explanation.
Start with the simplest credible baseline. A linear model, ruleset or manual workflow can expose whether a neural network creates enough additional value to justify its data, latency, operational and governance costs. Google’s current Machine Learning Crash Course places neural networks alongside data quality, generalisation, production systems and fairness rather than presenting architecture alone as the project.
Split data before learning from it
Use separate training, validation and test data, with the split designed around how the model will encounter the real world. Random row splits can leak information when records from the same customer, document, device or time period appear on both sides. For a future-facing forecast, use a temporal holdout. For a system expected to generalise to new organisations, consider holding out entire organisations.
Any learned preprocessing—normalisation statistics, feature selection, vocabulary building, imputation or synthetic sampling—must be fitted using training data only. Duplicate and near-duplicate examples require deliberate handling. A test set repeatedly consulted during tuning is no longer an untouched final test.
Choose metrics from the cost of errors. Accuracy can conceal failure on a rare but important class. Depending on the task, review precision, recall, false-positive and false-negative rates, calibration, ranking measures, latency and abstention behaviour. Report uncertainty and results for meaningful cohorts. Do not optimise one convenient metric while leaving the business decision undefined.
Make experiments reproducible enough to investigate
Record the code revision, framework and library versions, data snapshot and query, preprocessing configuration, random seeds, model configuration, hardware, checkpoint and evaluation script. Reproducibility is not equivalent to typing one seed. Parallel execution and some hardware kernels can be nondeterministic, while deterministic alternatives can be slower.
PyTorch explicitly warns that completely reproducible results are not guaranteed across releases, commits, platforms or CPU and GPU execution in its reproducibility notes. TensorFlow likewise documents the software, hardware, input-pipeline and random-state conditions around deterministic operations. Treat those settings as debugging and assurance tools, then measure their performance impact.
Evaluate the system, not only the checkpoint
A production model sits inside a larger system: collection, validation, feature computation, inference, human review, logging, fallback, appeal and retraining. Test malformed and missing inputs, out-of-distribution cases, dependency failures, timeouts, rollback and version compatibility. Define what the system must do when confidence is low or a required input is unavailable.
Monitoring should cover input quality, operational health, outcome quality where ground truth eventually arrives, and effects on people. Drift is not automatically harmful, and a stable input distribution does not guarantee stable outcomes. Establish a named owner, review cadence, incident path and retirement condition.
The voluntary NIST AI Risk Management Framework organises risk work around Govern, Map, Measure and Manage. Its emphasis on validity, reliability, transparency, privacy and harmful-bias management is a useful reminder that a model can be technically functional yet unsuitable in context.
Choose tools after defining the constraints
TensorFlow, PyTorch and other frameworks can all support serious work. Select on required operators, deployment target, team competence, maintenance horizon, observability and ecosystem compatibility—not an outdated claim that one framework is only for research and another only for production. Prototype the riskiest deployment path early and pin versions before a reproducible evaluation.
Before release, require evidence for:
- a documented task, user and unacceptable outcome;
- a leakage-resistant data split and representative test set;
- a baseline and decision-relevant metrics;
- cohort and edge-case evaluation;
- reproducible artefacts and dependency records;
- privacy, security and access controls;
- human review or safe fallback where consequences justify it; and
- monitoring, rollback and accountable ownership.
For help framing a responsible prototype or evaluation plan, see Ozlin Info’s AI and automation services or contact Ozlin Info.
Related reading: Machine-learning paradigms and evaluation and reading an older PyTorch HMER project responsibly.
General-information disclaimer
This article provides general technical information, not a guarantee of model accuracy, fairness, safety, regulatory compliance or business outcomes. Independent domain, privacy, legal, security and statistical review may be required for the actual use case.
AI-assistance disclosure
AI tools assisted with source discovery, outlining and copyediting. A human reviewer must verify the technical claims, examples, links and risk controls against the intended dataset, framework version and deployment context before publication or use.
Primary sources checked
- Google for Developers — Machine Learning Crash Course
- PyTorch — Autograd mechanics
- PyTorch — Optimisation API
- PyTorch — Reproducibility notes
- TensorFlow — Enabling operation determinism
- NIST AI Resource Center — AI Risk Management Framework
Source access date: 29 August 2026.
Leave a Reply