Distributions
What
A probability distribution describes all possible values a random variable can take and how likely each is.
Key distributions
Discrete
| Distribution | What it models | Example |
|---|
| Bernoulli | Single yes/no trial | Coin flip, spam/not-spam |
| Binomial | Count of successes in n trials | How many emails are spam out of 100 |
| Categorical | One of k classes | Which digit (0-9) an image shows |
| Poisson | Count of events in an interval | Number of requests per second |
Continuous
| Distribution | What it models | Example |
|---|
| Normal (Gaussian) | Bell curve, most common | Heights, measurement errors, weight initialization |
| Uniform | Equal probability everywhere | Random number generation |
| Exponential | Time between events | Time between server failures |
Why it matters
- Normal distribution: weight initialization, batch normalization, noise injection
- Softmax output: categorical distribution over classes
- Generative models: learn to sample from the data distribution
- Central Limit Theorem: averages of anything → normal distribution (why it’s everywhere)
Key properties
- PDF/PMF: the function that gives probability (density) at each value
- CDF: cumulative — P(X ≤ x)
- Parameters: normal has μ (mean) and σ (std dev), Bernoulli has p
Links