Total Variation Distance
Total Variation Distance (TVD) is a statistical measure of the difference between two probability distributions, quantifying the maximum possible discrepancy in probabilities assigned to any event. It is defined as half the sum of absolute differences between the probabilities of all outcomes, ranging from 0 (identical distributions) to 1 (completely disjoint distributions). This metric is widely used in probability theory, statistics, and machine learning to assess distributional similarity or convergence.
Developers should learn TVD when working on tasks involving probabilistic models, such as evaluating generative models (e.g., GANs), testing statistical hypotheses, or analyzing algorithm performance in randomized settings. It is particularly useful in machine learning for measuring how well a learned distribution approximates a target distribution, and in cryptography for assessing statistical security properties. TVD provides a rigorous, interpretable bound on distribution differences, making it essential for applications requiring precise probabilistic guarantees.