Binary outcomes show up everywhere in analytics: yes/no, spam/not spam, default/no default, clicked/did not click. Logistic regression is a standard model for these problems because it produces probabilities while keeping the math consistent with how binary data is generated. In many learning paths—especially a data scientist course in Delhi—logistic regression becomes a foundation topic because it connects probability, optimisation, and interpretability in one place.
1) From binary labels to a probability model
Assume each label yiy_iyi is binary: yi∈{0,1}y_i \in \{0,1\}yi∈{0,1}. The natural distribution for a single binary event is the Bernoulli distribution:
P(yi=1)=pi,P(yi=0)=1−piP(y_i=1) = p_i,\quad P(y_i=0)=1-p_iP(yi=1)=pi,P(yi=0)=1−piSo the likelihood for one observation is:
P(yi∣pi)=piyi(1−pi)(1−yi)P(y_i \mid p_i) = p_i^{y_i}(1-p_i)^{(1-y_i)}P(yi∣pi)=piyi(1−pi)(1−yi)For a dataset with nnn independent observations, the likelihood is the product:
L(p)=∏i=1npiyi(1−pi)(1−yi)L(\mathbf{p})=\prod_{i=1}^{n} p_i^{y_i}(1-p_i)^{(1-y_i)}L(p)=i=1∏npiyi(1−pi)(1−yi)The only remaining question is: how should we model pip_ipi using features xix_ixi?
2) Why we need a link function
A linear model like pi=β0+β⊤xip_i = \beta_0 + \beta^\top x_ipi=β0+β⊤xi is problematic because it can produce values less than 0 or greater than 1. We need a function that maps any real number to (0,1)(0,1)(0,1). We start with a linear predictor:
ηi=β0+β⊤xi\eta_i = \beta_0 + \beta^\top x_iηi=β0+β⊤xiThen we use a link function g(⋅)g(\cdot)g(⋅) to connect probability to ηi\eta_iηi:
g(pi)=ηig(p_i)=\eta_ig(pi)=ηiFor logistic regression, the link is the logit:
g(p)=log(p1−p)g(p)=\log\left(\frac{p}{1-p}\right)g(p)=log(1−pp)This choice is not arbitrary. It has strong statistical and practical reasons, and it leads directly to the familiar sigmoid probability function.
3) Deriving the logit via odds and invertibility
Start with the odds of the positive class:
odds=p1−p\text{odds}=\frac{p}{1-p}odds=1−ppOdds are always positive, but can become very large. Taking the log transforms them to the entire real line:
log(odds)=log(p1−p)\log(\text{odds}) = \log\left(\frac{p}{1-p}\right)log(odds)=log(1−pp)Now set this equal to the linear predictor:
log(pi1−pi)=ηi\log\left(\frac{p_i}{1-p_i}\right)=\eta_ilog(1−pipi)=ηiExponentiate both sides:
pi1−pi=eηi\frac{p_i}{1-p_i}=e^{\eta_i}1−pipi=eηiSolve for pip_ipi:
pi=eηi1+eηi=11+e−ηip_i = \frac{e^{\eta_i}}{1+e^{\eta_i}}=\frac{1}{1+e^{-\eta_i}}pi=1+eηieηi=1+e−ηi1This is the logistic (sigmoid) function. It guarantees valid probabilities and creates a clean interpretation: each coefficient βj\beta_jβj represents an additive change in log-odds per unit increase in xjx_jxj (holding other variables constant). This interpretability is a big reason the model is taught early in a data scientist course in Delhi and used widely in industry.
4) Maximum likelihood estimation and the optimisation target
To estimate β\betaβ, logistic regression uses maximum likelihood. Substitute pi=σ(ηi)p_i=\sigma(\eta_i)pi=σ(ηi) into the Bernoulli likelihood and take logs (log-likelihood is easier to optimise):
ℓ(β)=∑i=1n[yilog(pi)+(1−yi)log(1−pi)]\ell(\beta)=\sum_{i=1}^{n} \left[y_i\log(p_i)+(1-y_i)\log(1-p_i)\right]ℓ(β)=i=1∑n[yilog(pi)+(1−yi)log(1−pi)]Using pi=σ(ηi)p_i=\sigma(\eta_i)pi=σ(ηi), this becomes:
ℓ(β)=∑i=1n[yiηi−log(1+eηi)]\ell(\beta)=\sum_{i=1}^{n}\left[y_i\eta_i-\log(1+e^{\eta_i})\right]ℓ(β)=i=1∑n[yiηi−log(1+eηi)]There is no closed-form solution for β\betaβ here (unlike ordinary least squares). Instead, we maximise ℓ(β)\ell(\beta)ℓ(β) numerically.
Gradient (what the optimiser follows)
The gradient has a neat form:
∂ℓ∂β=∑i=1n(yi−pi)xi\frac{\partial \ell}{\partial \beta}=\sum_{i=1}^{n}(y_i-p_i)x_i∂β∂ℓ=i=1∑n(yi−pi)xiThis says the update direction is driven by the residual yi−piy_i-p_iyi−pi. If the model underpredicts an event (yi=1y_i=1yi=1 but pip_ipi is low), the gradient pushes parameters to increase pip_ipi for similar feature patterns.
Hessian and convexity (why it converges reliably)
The second derivative (Hessian) is negative semi-definite for logistic regression, so the negative log-likelihood is convex. That means there is a single global optimum, and algorithms like Newton–Raphson or IRLS (iteratively reweighted least squares) converge predictably when the data is well-behaved.
5) Practical intuition: why logit is a sensible “scale” for binary outcomes
Probabilities compress near 0 and 1, but business decisions often need a linear score to rank cases. Log-odds provide that linear scale. For example, in credit risk, doubling the odds of default is meaningful, and in marketing, log-odds shifts can be compared across segments. Logistic regression gives both: a probability for decisioning and a log-odds interpretation for explanation and reporting.
Conclusion
The logit link in logistic regression comes from a clean requirement: model Bernoulli outcomes with valid probabilities while keeping the predictor linear in features. By modelling the log-odds as a linear function and estimating parameters via maximum likelihood, we naturally obtain the sigmoid probability function and an optimisation objective that is stable and interpretable. If you are revising classification fundamentals for a data scientist course in Delhi, focusing on this derivation is valuable because it explains not just what logistic regression does, but why the logit is the right bridge between linear models and binary data.