Let’s get one thing straight: managing tensors by hand is like juggling chainsaws while blindfolded. It sounds impressive, but it’s a guaranteed mess if you don’t know exactly what you’re doing. Tensors aren’t just arrays; they’re multidimensional beasts with their own quirks and traps. You might think, “Hey, it’s just a matrix with more dimensions,” but as soon as you dive into broadcasting rules, shapes, gradients, and memory layouts, the waters get murky fast.
For instance, imagine you have a 3D tensor representing, say, an RGB image batch with shape (batch_size, height, width, channels). You want to apply a transformation across the batch, but you accidentally mix up your axes, and suddenly your code runs without errors but produces garbage output. That happens more often than you’d like.
Here’s a quick example. Suppose you want to normalize each image in the batch by subtracting the mean pixel value per channel:
import numpy as np batch = np.random.randn(32, 64, 64, 3) # 32 images, 64x64 pixels, 3 color channels # Calculate mean per channel across the batch and spatial dimensions mean_per_channel = batch.mean(axis=(0, 1, 2), keepdims=True) # Subtract mean from the batch normalized_batch = batch - mean_per_channel
Looks straightforward, right? But what if you accidentally forget keepdims=True? Your mean_per_channel becomes shape (3,) instead of (1,1,1,3), and broadcasting still works… but only because NumPy is lenient. Other frameworks might choke or, worse, produce subtle bugs.
And that’s just the start. Once you bring gradients into the picture-because, of course, you’re training a neural network-you quickly realize that keeping track of which operations are differentiable, which tensors require gradients, and how to chain them together manually is a nightmare. The computational graph concept is not just a fancy abstraction; it’s essential for avoiding hours of debugging cryptic errors.
Let’s not forget memory management. When you do in-place operations on tensors that participate in gradient computation, you can easily corrupt the computation graph or cause unexpected side effects. For example, this:
import torch x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True) y = x * 2 y += 1 # In-place operation that might cause issues in autograd
It’s tempting to write neat, concise code, but those in-place operations can sabotage gradient calculations. PyTorch throws a warning for a reason.
So, if you’re thinking you can just “roll your own” tensor management and backpropagation from scratch, be prepared for a deep dive into edge cases, broadcasting pitfalls, and subtle bugs that only show up after hours of training. There’s a reason frameworks like PyTorch and TensorFlow exist-they do the heavy lifting, so you don’t have to.
That said, understanding what’s under the hood isn’t wasted effort. When you finally grasp why certain operations are allowed or not, how autograd tracks history, and the subtle details of tensor shapes, you become a better programmer, able to debug and optimize your models effectively. But if you want to jump in without understanding, you’re just setting yourself up for an uphill battle.
Still, let’s try to demystify some of these concepts by building a tiny framework that handles tensors and gradients manually. This will be a toy example, but it’ll expose you to the core ideas without the noise of a full-blown library.
class Tensor:
def __init__(self, data, requires_grad=False):
self.data = data
self.requires_grad = requires_grad
self.grad = None
self._backward = lambda: None
self._prev = set()
def __add__(self, other):
other = other if isinstance(other, Tensor) else Tensor(other)
out = Tensor(self.data + other.data, requires_grad=self.requires_grad or other.requires_grad)
def _backward():
if self.requires_grad:
self.grad = self.grad + out.grad if self.grad is not None else out.grad
if other.requires_grad:
other.grad = other.grad + out.grad if other.grad is not None else out.grad
out._backward = _backward
out._prev = {self, other}
return out
def backward(self):
topo = []
visited = set()
def build_topo(t):
if t not in visited:
visited.add(t)
for child in t._prev:
build_topo(child)
topo.append(t)
build_topo(self)
self.grad = np.ones_like(self.data)
for t in reversed(topo):
t._backward()
Notice how this class keeps track of the data, gradients, and backward functions. It’s minimal, but it lays the foundation for automatic differentiation. The backward method traverses the graph in topological order, calling _backward on each tensor to propagate gradients.
Try adding multiplication and other operations on your own, and you’ll see how this quickly grows. Managing the graph, ensuring correct gradient shapes, and handling edge cases soon becomes a full-time job. Which is why you should appreciate the libraries that do this for you, even if you sometimes curse their cryptic error messages.
Up next, we’ll see how to package this into a neat little toolbox so you never have to think about these internals again-until you want to. Because that’s the real magic of modern machine learning: your new box of magical Lego bricks.
HP DeskJet 2855e Wireless All-in-One Color Inkjet Printer, Scanner, Copier, Best-for-home, 3 month Instant Ink trial included. This printer is only 2.4 ghz capable. (588S5A)
$49.89 (as of July 17, 2026 15:16 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)Your new box of magical Lego bricks
So, you’ve seen the gory details of manual backpropagation. It’s messy, error-prone, and something you should only do once, for educational purposes, like dissecting a frog in biology class. You learn how the guts work, but you wouldn’t want to do it every day. The smart folks who build deep learning frameworks realized this and gave us a box of magical Lego bricks. The core idea is abstraction. Instead of fiddling with raw tensors for weights and biases, you get pre-packaged components called “layers” or “modules.”
In PyTorch, the fundamental building block is the torch.nn.Module. Think of it as a super-powered Python class. Anything that has learnable parameters and some logic to process data-a linear layer, a convolutional layer, or even an entire network-is a subclass of nn.Module. It does more than just hold your code; it automatically tracks all its sub-modules and, more importantly, all their learnable parameters. You just define the bricks, and the module keeps an inventory for you. This is incredibly useful when you need to tell an optimizer, “Here are all the knobs you can turn.”
Let’s look at the most basic brick: a fully connected, or linear, layer. Manually, you’d define a weight matrix and a bias vector, then perform the operation y = x @ W.T + b. With a framework, you just instantiate a Linear layer.
import torch import torch.nn as nn # A batch of 64 data points, each with 100 features input_data = torch.randn(64, 100) # Create a Linear layer that maps 100 input features to 10 output features linear_layer = nn.Linear(in_features=100, out_features=10) # Pass the data through the layer output = linear_layer(input_data) # The output has the correct shape print(output.shape) # torch.Size([64, 10])
What just happened? When you created nn.Linear(100, 10), it didn’t just create an object. It internally created two tensors: a weight tensor of shape (10, 100) and a bias tensor of shape (10). These aren’t just plain Tensor objects; they are wrapped in a special class called nn.Parameter. This is a tiny but crucial detail. An nn.Parameter is a subclass of Tensor that automatically sets requires_grad=True and, most importantly, signals to any containing nn.Module that it is a trainable parameter.
When you later call something like model.parameters() on your network, the module recursively scans for all attributes that are instances of nn.Parameter and returns them. This is how an optimizer gets a list of all the weights to update. You don’t have to manually track every single weight tensor in a giant list. The framework does the bookkeeping.
This Lego brick approach is powerful because it’s compositional. The output of one layer is just a tensor, which can be fed directly into another layer. You can stack them up to build complex architectures. And it’s not just linear layers. The toolbox is vast: convolutional layers (nn.Conv2d) for images, recurrent layers (nn.LSTM) for sequences, normalization layers (nn.BatchNorm2d) to stabilize training, and dropout layers (nn.Dropout) to prevent overfitting. They all inherit from nn.Module and follow the same simple pattern: instantiate it with its configuration, then call it with your data.
How to teach your Python class to be a neural network
Now that you understand the power of nn.Module, let’s see how you can teach your own Python class to behave like one of these neural network bricks. Say you want to create a simple custom layer that applies a fixed linear transformation, but with a twist: it scales the output by a learnable scalar parameter. Here’s how you might do it from scratch, without relying on built-in layers.
import torch
import torch.nn as nn
class ScaledLinear(nn.Module):
def __init__(self, in_features, out_features):
super().__init__()
# Weight matrix and bias vector as Parameters
self.weight = nn.Parameter(torch.randn(out_features, in_features))
self.bias = nn.Parameter(torch.randn(out_features))
# Learnable scalar to scale the output
self.scale = nn.Parameter(torch.tensor(1.0))
def forward(self, x):
# Linear transformation: x @ W^T + b
linear_out = x.matmul(self.weight.t()) + self.bias
# Scale the output
return self.scale * linear_out
# Usage example
batch_size, input_dim, output_dim = 8, 5, 3
layer = ScaledLinear(input_dim, output_dim)
input_tensor = torch.randn(batch_size, input_dim)
output_tensor = layer(input_tensor)
print(output_tensor.shape) # torch.Size([8, 3])
Notice how the key to making your class “play nice” with PyTorch’s ecosystem is inheriting from nn.Module and registering parameters as nn.Parameter. The magic is that this registration hooks your parameters into the module’s machinery automatically. No need to write boilerplate code to collect parameters or handle gradients.
Also, the convention is to implement a method called forward that defines the computation. When you call the module instance like a function, e.g., layer(input_tensor), Python internally dispatches to forward. This syntactic sugar keeps your code clean and readable.
But what about submodules? Suppose you want to build a tiny multilayer perceptron (MLP) with two linear layers and a non-linearity in between. Instead of writing all the operations manually in one class, you can compose smaller modules. This is the composability that makes deep learning code elegant and scalable.
class SimpleMLP(nn.Module):
def __init__(self, input_dim, hidden_dim, output_dim):
super().__init__()
self.fc1 = nn.Linear(input_dim, hidden_dim)
self.relu = nn.ReLU()
self.fc2 = nn.Linear(hidden_dim, output_dim)
def forward(self, x):
x = self.fc1(x)
x = self.relu(x)
x = self.fc2(x)
return x
# Instantiate and run
model = SimpleMLP(10, 20, 5)
x = torch.randn(4, 10)
out = model(x)
print(out.shape) # torch.Size([4, 5])
Here, fc1, relu, and fc2 are all submodules of the SimpleMLP class. The moment you assign them as attributes, PyTorch detects and registers them. When you call model.parameters(), it returns parameters from both fc1 and fc2, recursively. This means your optimizer gets the full list of trainable weights with zero extra effort on your part.
Want to add dropout or batch normalization? Just instantiate those modules and insert them in the forward method. You don’t have to write the low-level tensor operations yourself. The framework handles all the gradient calculations and device placements behind the scenes.
One subtle but important detail is that modules can behave differently during training and evaluation. For example, dropout turns off when you switch to evaluation mode. This is controlled by calling model.train() or model.eval(). Your custom modules can implement this behavior by overriding the train and eval hooks if needed.
Here’s a quick example of a custom module that behaves differently in training vs. evaluation:
class CustomDropout(nn.Module):
def __init__(self, p=0.5):
super().__init__()
self.p = p
def forward(self, x):
if self.training:
mask = (torch.rand_like(x) > self.p).float()
return x * mask / (1.0 - self.p)
else:
return x
# Usage
dropout = CustomDropout(p=0.3)
dropout.train()
train_output = dropout(torch.ones(5))
dropout.eval()
eval_output = dropout(torch.ones(5))
print(train_output)
print(eval_output)
Notice the use of the self.training flag inside forward. This is automatically set by the framework when you toggle modes. It’s a simple mechanism, but it lets you build modules that adapt their behavior seamlessly.
Finally, to make your model trainable, you just need to plug it into an optimizer and a loss function. The optimizer calls model.parameters() to get the weights and updates them based on the gradients computed during backpropagation. Here’s a minimal example of a training step:
model = SimpleMLP(10, 20, 5) optimizer = torch.optim.SGD(model.parameters(), lr=0.01) criterion = nn.MSELoss() # Dummy data inputs = torch.randn(16, 10) targets = torch.randn(16, 5) # Forward pass outputs = model(inputs) loss = criterion(outputs, targets) # Backward pass optimizer.zero_grad() loss.backward() optimizer.step()
And that’s the crux of it: by teaching your Python class to be a module, you unlock all the power of the framework’s autograd, parameter management, and optimization routines. You get to focus on the architecture and the math, while the framework handles the plumbing.
Of course, this is just the beginning. Once you start stacking modules, experimenting with different architectures, or implementing custom layers that do unusual things, you’ll appreciate the flexibility and elegance of this design pattern. But before you get ahead of yourself, make sure you master the basics of subclassing nn.Module and composing your own networks. It’s the foundation upon which everything else is built.
Next up, we’ll look at how to tell the data where to go during training and inference, orchestrating the flow between your modules, datasets, and the optimizer. Because building a network is only half the battle-you also need to make sure it learns.
Now just tell the data where to go
So you’ve meticulously crafted your neural network by subclassing nn.Module. It’s a beautiful piece of software architecture. It has layers, parameters, and a forward method. Now what? It just sits there, a collection of matrices and functions, completely useless until you feed it data and tell it to learn. This process, the orchestration of data flow and gradient updates, is what we call the training loop. It’s the engine that drives your model from a state of random initialization to a state where it can actually make useful predictions.
The first problem you’ll hit is data. Your dataset is probably huge-thousands, maybe millions of examples. You can’t just load it all into your GPU’s memory; you’d run out of VRAM before you could say “out of memory.” The solution is to process the data in small, manageable chunks called mini-batches. But slicing up your data, shuffling it every epoch to prevent the model from learning the order, and feeding it to the model one batch at a time is tedious boilerplate. This is where the framework steps in again with two handy classes: Dataset and DataLoader.
A Dataset is an object that knows how to access your data. It’s a simple contract: you give it an index, and it returns the corresponding data point (e.g., an image and its label). You can create your own by subclassing torch.utils.data.Dataset and implementing two methods: __len__ to tell the framework how many samples you have, and __getitem__ to retrieve a specific sample.
from torch.utils.data import Dataset
class MyDummyDataset(Dataset):
def __init__(self, num_samples=1000):
self.num_samples = num_samples
# Generate some dummy data and labels
self.data = torch.randn(num_samples, 20)
self.labels = torch.randint(0, 2, (num_samples,))
def __len__(self):
return self.num_samples
def __getitem__(self, idx):
return self.data[idx], self.labels[idx]
# Instantiate the dataset
dataset = MyDummyDataset()
print(f"Dataset size: {len(dataset)}")
# Get one sample
features, label = dataset[0]
print(f"Sample shape: {features.shape}, Label: {label}")
The DataLoader is the real workhorse. It takes your Dataset and gives you an iterator that serves up batches of data, automatically handling shuffling, batching, and even using multiple CPU cores to load data in parallel so your GPU doesn’t sit idle waiting for the next batch. You just tell it your dataset, the batch size, and whether to shuffle, and it does the rest.
from torch.utils.data import DataLoader
# Create a DataLoader
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
# Iterate over one epoch of data
for features_batch, labels_batch in dataloader:
print(f"Batch of features shape: {features_batch.shape}")
print(f"Batch of labels shape: {labels_batch.shape}")
# This is where your training logic would go
break # Stop after one batch for demonstration
With data handling sorted, the next piece of the puzzle is the hardware. Your laptop’s CPU can run these operations, but it’s like using a screwdriver to hammer a nail. It’ll work, but it’s painfully slow. Neural network computations are massively parallel matrix multiplications, which is exactly what GPUs are designed for. To use a GPU, you need to explicitly tell PyTorch to move both your model’s parameters and your data onto the GPU’s memory. If you forget to move even one tensor, you’ll be greeted with a runtime error that every practitioner knows and loathes: Expected all tensors to be on the same device.
The standard practice is to define a device object at the beginning of your script, which points to the GPU if one is available ("cuda") or falls back to the CPU. Then, you use the .to(device) method on your model and on every batch of data you get from your DataLoader.
# Set device
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Using {device} device")
# Assume 'model' is an nn.Module instance
model = SimpleMLP(20, 50, 2).to(device)
# Inside the training loop
for features_batch, labels_batch in dataloader:
# Move data to the selected device
features_batch = features_batch.to(device)
labels_batch = labels_batch.to(device)
# Now you can perform operations on the GPU
# ...
Now we can assemble the complete training loop. It’s a standard recipe: loop over epochs, and inside each epoch, loop over your data loader. For each batch, you perform a forward pass, compute the loss, perform a backward pass to get gradients, and ask the optimizer to update the weights. You also need to remember to zero out the gradients from the previous step using optimizer.zero_grad(), otherwise they will accumulate.
# Re-using the SimpleMLP and MyDummyDataset from before
input_dim, hidden_dim, output_dim = 20, 50, 2
model = SimpleMLP(input_dim, hidden_dim, output_dim).to(device)
dataset = MyDummyDataset(num_samples=1000)
dataloader = DataLoader(dataset, batch_size=64, shuffle=True)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
num_epochs = 5
for epoch in range(num_epochs):
model.train() # Set the model to training mode
for features, labels in dataloader:
features = features.to(device)
labels = labels.to(device)
# Forward pass
outputs = model(features)
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()
print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}")
And don’t forget validation. After each epoch of training, you typically want to evaluate your model’s performance on a separate validation dataset. The validation loop is similar, but with two key differences: you set the model to evaluation mode with model.eval() to disable behaviors like dropout, and you wrap the whole thing in a with torch.no_grad(): context. This tells PyTorch not to build a computation graph, which saves memory and speeds up computation, since you don’t need to calculate gradients for validation.
