ML Percentile

ML Percentiles & Quantiles: Distribution Analysis

Percentiles indicate the relative standing of a value within a data distribution. The $P$-th percentile is the value below which $P%$ of data observations fall.

Quartiles & Box Plot Metrics

flowchart LR
    Min["Minimum"] --> Q1["Q1 (25th Percentile)"]
    Q1 --> Q2["Q2 / Median (50th Percentile)"]
    Q2 --> Q3["Q3 (75th Percentile)"]
    Q3 --> Max["Maximum"]
    Q1 --- IQR["Interquartile Range (IQR = Q3 - Q1)"] --- Q3

Practical Code Example

import numpy as np
from typing import Dict

def analyze_percentiles(data: np.ndarray) -> Dict[str, float]:
    q1 = float(np.percentile(data, 25))
    median = float(np.percentile(data, 50))
    q3 = float(np.percentile(data, 75))
    iqr = q3 - q1

    # Outlier thresholds via IQR rule
    lower_bound = q1 - (1.5 * iqr)
    upper_bound = q3 + (1.5 * iqr)

    return {
        "Q1_25%": q1,
        "Median_50%": median,
        "Q3_75%": q3,
        "IQR": iqr,
        "Outlier_Lower": lower_bound,
        "Outlier_Upper": upper_bound
    }

if __name__ == "__main__":
    scores = np.array([35, 55, 60, 65, 70, 75, 80, 85, 90, 95, 150])  # 150 is outlier
    p_info = analyze_percentiles(scores)

    print("--- Percentile Summary ---")
    for key, val in p_info.items():
        print(f" {key:<15}: {val:.2f}")

Best Practices & Gotchas

• IQR Outlier Detection: Any data point smaller than $Q1 - 1.5 imes IQR$ or larger than $Q3 + 1.5 imes IQR$ is classified as a statistical outlier.
• Interpolation Methods: NumPy support different percentile interpolation methods (method='linear', 'nearest', 'lower', 'higher').
• Use np.quantile() for Decimals: np.quantile(data, 0.5) is identical to np.percentile(data, 50).

Self-Check Challenge

Find the 90th percentile score of np.array([10, 20, 30, 40, 50, 60, 70, 80, 90, 100]) using np.percentile().