Logistic Regression

What

Despite the name, it’s a classification model. Linear regression + sigmoid function → outputs a probability.

z = w₁x₁ + w₂x₂ + ... + b          (linear part)
P(y=1) = sigmoid(z) = 1 / (1 + e⁻ᶻ)  (squash to [0, 1])

Why it matters

• First classification algorithm to learn
• Outputs calibrated probabilities (not just class labels)
• Interpretable: coefficients show feature influence
• Fast, works well as a baseline

Training

Minimize binary cross-entropy (log loss):

Loss = -[y·log(ŷ) + (1-y)·log(1-ŷ)]

from sklearn.linear_model import LogisticRegression
 
model = LogisticRegression()
model.fit(X_train, y_train)
 
model.predict(X_test)        # class labels
model.predict_proba(X_test)  # probabilities

Multi-class

• One-vs-Rest (OvR): train one classifier per class
• Softmax: generalization of sigmoid to multiple classes — used in neural nets

# sklearn handles multi-class automatically
model = LogisticRegression(multi_class="multinomial")

Links

• Linear Regression — the regression foundation
• Loss Functions — cross-entropy
• Decision Trees — non-linear alternative
• Evaluation Metrics — precision, recall, F1