Multi-Output Machine Learning from Scratch: Intuition, Matrix Multiplication & Debugging Journey

From Linear Regression to Multi-Output Models (Built from Scratch)

For the past 14 days, I’ve been building machine learning concepts from scratch – no libraries, no shortcuts, just raw Python and first-principles thinking.

This post is not about “how to implement linear regression.”

  • how intuition develops
  • how confusion happens
  • how errors teach better than theory
  • and how a simple model evolves into something much deeper

Where It Started: Single Output Thinking

I began with a simple idea:

Given features → predict one value

Example:

  • Inputs: SqFt, Age
  • Output: Price

Mathematically:

[y=Xw+b][ y = Xw + b ]

At this stage:

X(n×d)X → (n × d)
w(d)w → (d)

Output → scalar per row

Everything was clean, simple, and intuitive.

Implementation: https://ai.ppradosh.com/2026/april/11/the-capstone-the-batch-processing-regressor

The Shift: Multiple Outputs

Then I asked:

Why stop at predicting price?

What if I also predict:

  • Luxury score
  • Maybe even more attributes later

Now the model becomes:

[f(x)[price,luxury]][ f(x) → [price, luxury] ]

This changes everything.

The Key Realization

Each output is not “extra data” – it’s a separate mapping.

Instead of:

  • one function

I now have:

  • multiple functions applied simultaneously
[Y=XW][ Y = XW ]

Where:

  • W is no longer a vector
  • It becomes a matrix of mappings

The Moment Everything Clicked

This didn’t click while coding.

It clicked while walking.

“Matrix multiplication is just applying multiple mappings at once.”

That one line changed everything.

The Reality: Errors Everywhere

Understanding didn’t come from theory.

It came from breaking things.

Error 1: List vs Scalar

TypeError: can only concatenate list (not "float") to list

What I learned:

  • Batch output is a list
  • Bias must be applied per element

Error 2: Matrix vs List

AttributeError: 'list' object has no attribute 'number_of_rows'

What I learned:

  • Data structure matters
  • A matrix is not just a list of lists — it has behavior

Error 3: Shape Mismatch (Big One)

ValueError: columns != rows

This was the turning point.

What I realized:

Matrix multiplication is not about numbers — it’s about alignment.

This is where transpose entered the picture.

The Role of Transpose

I stored weights as:

(k×d)(k × d)

But matrix multiplication required:

(d×k)(d × k)

So I had to transpose:

[X(n×d)WT(d×k)][ X (n×d) \cdot W^T (d×k) ]

This wasn’t a trick.

It was:

Aligning input space with output mappings

Error 4: Nested Structures

TypeError: type list doesn't define __round__

What I learned:

  • My weights are no longer simple numbers
  • They represent multiple mappings

Error 5: Loss Function Chaos

TypeError: unsupported operand type(s) for +: 'int' and 'list'

What I learned:

  • Loss must respect output structure
  • Multi-output = multi-dimensional error

Error 6: Loop Mistakes

TypeError: 'list' object cannot be interpreted as an integer

What I learned:

  • Iterating over structure vs length matters

Error 7: The Classic

NameError: returnb


Yes… that happened too 😄

What I Actually Built

After all the iterations, the model became:

[Y=XWT+b][ Y = X \cdot W^T + b ]
  • X(n×d)X → (n × d)
  • W(k×d)W → (k × d)
  • Y(n×k)Y → (n × k)

And finally:

Prediction: [[285.05, 77.8]]

Implementation

import random
class Matrix:
def __init__(self, data: list[list[float]]):
if data:
self.__validate(data)
self.data = data
self.number_of_rows = len(data)
self.number_of_cols = len(data[0])
else:
self.data = []
self.number_of_rows = 0
self.number_of_cols = 0
def __validate(self, data: list[list[float]]) -> None:
"""Private method to ensure matrix is a perfect rectangle."""
number_of_cols = len(data[0])
for row in data:
if len(row) != number_of_cols:
raise ValueError("All rows must have the same number of columns to form a valid matrix.")
@property
def shape(self) -> tuple[int, int]:
"""Returns the shape of the matrix as (rows, columns)."""
return (self.number_of_rows, self.number_of_cols)
def __mul__(self, scalar: float) -> "Matrix":
"""Scalar multiplication: scales every element by the scalar."""
return Matrix([[element * scalar for element in row] for row in self.data])
def __add__(self, other: "Matrix") -> "Matrix":
"""Matrix addition: adds elements of identically shaped matrices."""
if isinstance(other, Matrix):
if self.shape != other.shape:
raise ValueError("Matrices must have the same shape for addition")
return Matrix([
[a + b for a, b in zip(row1, row2)]
for row1, row2 in zip(self.data, other.data)
])
else:
raise TypeError(f"Unsupported operand type for +: 'Matrix' and '{type(other).__name__}'")
def dot_vector(self, vector: list[float]) -> list[float]:
"""Multiplies the matrix by a 1D vector (Batch Dot Product)."""
if self.number_of_cols != len(vector):
raise ValueError("The number of columns in the matrix must exactly equal the number of elements in the vector")
return [sum(a * b for a, b in zip(row, vector)) for row in self.data]
def dot_matrix(self, other: "Matrix") -> "Matrix":
"""Multiplies the matrix by another matrix (Batch Matrix Multiplication)."""
if self.number_of_cols != other.number_of_rows:
raise ValueError("The number of columns in the first matrix must equal the number of rows in the second matrix for multiplication")
result = [
[
sum(self.data[i][k] * other.data[k][j] for k in range(other.number_of_rows))
for j in range(other.number_of_cols)
]
for i in range(self.number_of_rows)
]
return Matrix(result)
def get_column(self, index: int) -> list[float]:
"""Returns a specific column from the matrix as a 1D list."""
if not 0 <= index < self.number_of_cols:
raise IndexError("Column index is out of bounds")
return [row[index] for row in self.data]
@property
def T(self) -> "Matrix":
"""Returns the transpose of the matrix."""
return Matrix([[self.data[i][j] for i in range(self.number_of_rows)] for j in range(self.number_of_cols)])
def __repr__(self) -> str:
"""Helper to print the matrix cleanly in the terminal."""
rows_str = "\n ".join(str(row) for row in self.data)
return f"Matrix(\n {rows_str}\n)"
def mean_squared_error(actuals, predictions):
return [
sum((a[j] - p[j]) ** 2 for a, p in zip(actuals, predictions)) / len(actuals)
for j in range(len(predictions[0]))
]
class BatchLinearRegressor:
def __init__(self, learning_rate: float = 1.0, epochs: int = 10000):
self.learning_rate = learning_rate
self.epochs = epochs
# We start with empty weights because we don't know how many features X has yet!
self.weights = []
self.bias = []
self.best_loss = [float('inf'), float('inf')]
def fit(self, X: Matrix, y: list[list[float]]) -> None:
# 1. Initialize self.weights with 0.0 for every column in X
if not self.weights:
self.weights = [[0.0 for _ in range(X.shape[1])] for _ in range(len(y[0]))]
if not self.bias:
self.bias = [0.0 for _ in range(len(y[0]))]
for epoch in range(self.epochs):
test_weights = [[weight + random.uniform(-self.learning_rate, self.learning_rate) for weight in row] for row in self.weights]
test_bias = [bias + random.uniform(-self.learning_rate, self.learning_rate) for bias in self.bias]
test_predictions = [[a + b for a, b in zip(output, test_bias)] for output in X.dot_matrix(Matrix(test_weights).T).data]
test_loss = mean_squared_error(y, test_predictions)
# 4. If the error is lower, keep the new weights!
for i in range(len(test_loss)):
if test_loss[i] < self.best_loss[i]:
self.best_loss[i] = test_loss[i]
self.weights[i] = test_weights[i]
self.bias[i] = test_bias[i]
def predict(self, X: Matrix) -> list[list[float]]:
"""Generates predictions for new batch data."""
return [[a + b for a, b in zip(output, self.bias)] for output in X.dot_matrix(Matrix(self.weights).T).data]
# --- Example Usage: The Capstone Test ---
# Columns: [Age, Size, Num of Bedrooms]
X_train = Matrix([
[15.0, 10.0, 2.0], # House 1
[3.0, 15.0, 3.0], # House 2
[40.0, 30.0, 4.0] # House 3
])
# Columns: [Price, Luxury Score]
y_train = [
[130.0, 60.0], # House 1
[240.0, 95.0], # House 2 (High luxury driven by young age)
[310.0, 40.0] # House 3 (Low luxury driven by old age, despite high price)
]
# Train the engine
model = BatchLinearRegressor(learning_rate=1.0, epochs=20000)
model.fit(X_train, y_train)
# Test on completely unseen data: A 4000 SqFt house that is 2 years old
X_test = Matrix([[4.0, 20.0, 3]])
print(f"Trained Weights: {[model.weights]}")
print(f"Trained Bias: {model.bias}")
print(f"Prediction for 4000 SqFt, 2 yrs old: {[[round(i, 2) for i in p] for p in model.predict(X_test)]}")


Discover more from PRADOSH

Subscribe to get the latest posts to your email.

Leave a comment