From Linear Regression to Multi-Output Models (Built from Scratch)
For the past 14 days, I’ve been building machine learning concepts from scratch – no libraries, no shortcuts, just raw Python and first-principles thinking.
This post is not about “how to implement linear regression.”
- how intuition develops
- how confusion happens
- how errors teach better than theory
- and how a simple model evolves into something much deeper
Where It Started: Single Output Thinking
I began with a simple idea:
Given features → predict one value
Example:
- Inputs: SqFt, Age
- Output: Price
Mathematically:
At this stage:
Output → scalar per row
Everything was clean, simple, and intuitive.
Implementation: https://ai.ppradosh.com/2026/april/11/the-capstone-the-batch-processing-regressor
The Shift: Multiple Outputs
Then I asked:
Why stop at predicting price?
What if I also predict:
- Luxury score
- Maybe even more attributes later
Now the model becomes:
This changes everything.
The Key Realization
Each output is not “extra data” – it’s a separate mapping.
Instead of:
- one function
I now have:
- multiple functions applied simultaneously
Where:
- W is no longer a vector
- It becomes a matrix of mappings
The Moment Everything Clicked
This didn’t click while coding.
It clicked while walking.
“Matrix multiplication is just applying multiple mappings at once.”
That one line changed everything.
The Reality: Errors Everywhere
Understanding didn’t come from theory.
It came from breaking things.
Error 1: List vs Scalar
TypeError: can only concatenate list (not "float") to list
What I learned:
- Batch output is a list
- Bias must be applied per element
Error 2: Matrix vs List
AttributeError: 'list' object has no attribute 'number_of_rows'
What I learned:
- Data structure matters
- A matrix is not just a list of lists — it has behavior
Error 3: Shape Mismatch (Big One)
ValueError: columns != rows
This was the turning point.
What I realized:
Matrix multiplication is not about numbers — it’s about alignment.
This is where transpose entered the picture.
The Role of Transpose
I stored weights as:
But matrix multiplication required:
So I had to transpose:
This wasn’t a trick.
It was:
Aligning input space with output mappings
Error 4: Nested Structures
TypeError: type list doesn't define __round__
What I learned:
- My weights are no longer simple numbers
- They represent multiple mappings
Error 5: Loss Function Chaos
TypeError: unsupported operand type(s) for +: 'int' and 'list'
What I learned:
- Loss must respect output structure
- Multi-output = multi-dimensional error
Error 6: Loop Mistakes
TypeError: 'list' object cannot be interpreted as an integer
What I learned:
- Iterating over structure vs length matters
Error 7: The Classic
NameError: returnb
Yes… that happened too 😄
What I Actually Built
After all the iterations, the model became:
And finally:
Prediction: [[285.05, 77.8]]
Implementation
import randomclass Matrix: def __init__(self, data: list[list[float]]): if data: self.__validate(data) self.data = data self.number_of_rows = len(data) self.number_of_cols = len(data[0]) else: self.data = [] self.number_of_rows = 0 self.number_of_cols = 0 def __validate(self, data: list[list[float]]) -> None: """Private method to ensure matrix is a perfect rectangle.""" number_of_cols = len(data[0]) for row in data: if len(row) != number_of_cols: raise ValueError("All rows must have the same number of columns to form a valid matrix.") property def shape(self) -> tuple[int, int]: """Returns the shape of the matrix as (rows, columns).""" return (self.number_of_rows, self.number_of_cols) def __mul__(self, scalar: float) -> "Matrix": """Scalar multiplication: scales every element by the scalar.""" return Matrix([[element * scalar for element in row] for row in self.data]) def __add__(self, other: "Matrix") -> "Matrix": """Matrix addition: adds elements of identically shaped matrices.""" if isinstance(other, Matrix): if self.shape != other.shape: raise ValueError("Matrices must have the same shape for addition") return Matrix([ [a + b for a, b in zip(row1, row2)] for row1, row2 in zip(self.data, other.data) ]) else: raise TypeError(f"Unsupported operand type for +: 'Matrix' and '{type(other).__name__}'") def dot_vector(self, vector: list[float]) -> list[float]: """Multiplies the matrix by a 1D vector (Batch Dot Product).""" if self.number_of_cols != len(vector): raise ValueError("The number of columns in the matrix must exactly equal the number of elements in the vector") return [sum(a * b for a, b in zip(row, vector)) for row in self.data] def dot_matrix(self, other: "Matrix") -> "Matrix": """Multiplies the matrix by another matrix (Batch Matrix Multiplication).""" if self.number_of_cols != other.number_of_rows: raise ValueError("The number of columns in the first matrix must equal the number of rows in the second matrix for multiplication") result = [ [ sum(self.data[i][k] * other.data[k][j] for k in range(other.number_of_rows)) for j in range(other.number_of_cols) ] for i in range(self.number_of_rows) ] return Matrix(result) def get_column(self, index: int) -> list[float]: """Returns a specific column from the matrix as a 1D list.""" if not 0 <= index < self.number_of_cols: raise IndexError("Column index is out of bounds") return [row[index] for row in self.data] property def T(self) -> "Matrix": """Returns the transpose of the matrix.""" return Matrix([[self.data[i][j] for i in range(self.number_of_rows)] for j in range(self.number_of_cols)]) def __repr__(self) -> str: """Helper to print the matrix cleanly in the terminal.""" rows_str = "\n ".join(str(row) for row in self.data) return f"Matrix(\n {rows_str}\n)"def mean_squared_error(actuals, predictions): return [ sum((a[j] - p[j]) ** 2 for a, p in zip(actuals, predictions)) / len(actuals) for j in range(len(predictions[0])) ]class BatchLinearRegressor: def __init__(self, learning_rate: float = 1.0, epochs: int = 10000): self.learning_rate = learning_rate self.epochs = epochs # We start with empty weights because we don't know how many features X has yet! self.weights = [] self.bias = [] self.best_loss = [float('inf'), float('inf')] def fit(self, X: Matrix, y: list[list[float]]) -> None: # 1. Initialize self.weights with 0.0 for every column in X if not self.weights: self.weights = [[0.0 for _ in range(X.shape[1])] for _ in range(len(y[0]))] if not self.bias: self.bias = [0.0 for _ in range(len(y[0]))] for epoch in range(self.epochs): test_weights = [[weight + random.uniform(-self.learning_rate, self.learning_rate) for weight in row] for row in self.weights] test_bias = [bias + random.uniform(-self.learning_rate, self.learning_rate) for bias in self.bias] test_predictions = [[a + b for a, b in zip(output, test_bias)] for output in X.dot_matrix(Matrix(test_weights).T).data] test_loss = mean_squared_error(y, test_predictions) # 4. If the error is lower, keep the new weights! for i in range(len(test_loss)): if test_loss[i] < self.best_loss[i]: self.best_loss[i] = test_loss[i] self.weights[i] = test_weights[i] self.bias[i] = test_bias[i] def predict(self, X: Matrix) -> list[list[float]]: """Generates predictions for new batch data.""" return [[a + b for a, b in zip(output, self.bias)] for output in X.dot_matrix(Matrix(self.weights).T).data]# --- Example Usage: The Capstone Test ---# Columns: [Age, Size, Num of Bedrooms]X_train = Matrix([ [15.0, 10.0, 2.0], # House 1 [3.0, 15.0, 3.0], # House 2 [40.0, 30.0, 4.0] # House 3])# Columns: [Price, Luxury Score]y_train = [ [130.0, 60.0], # House 1 [240.0, 95.0], # House 2 (High luxury driven by young age) [310.0, 40.0] # House 3 (Low luxury driven by old age, despite high price)]# Train the enginemodel = BatchLinearRegressor(learning_rate=1.0, epochs=20000)model.fit(X_train, y_train)# Test on completely unseen data: A 4000 SqFt house that is 2 years oldX_test = Matrix([[4.0, 20.0, 3]]) print(f"Trained Weights: {[model.weights]}")print(f"Trained Bias: {model.bias}")print(f"Prediction for 4000 SqFt, 2 yrs old: {[[round(i, 2) for i in p] for p in model.predict(X_test)]}")
Leave a comment