In statistics, the logit (/ˈloʊdʒɪt/ LOH-jit) function is the quantile function associated with the standard logistic distribution. It has many uses in data analysis and machine learning, especially in data transformations.
![](http://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Logit.svg/290px-Logit.svg.png)
Mathematically, the logit is the inverse of the standard logistic function , so the logit is defined as
Because of this, the logit is also called the log-odds since it is equal to the logarithm of the odds where p is a probability. Thus, the logit is a type of function that maps probability values from to real numbers in ,[1] akin to the probit function.
Definition
editIf p is a probability, then p/(1 − p) is the corresponding odds; the logit of the probability is the logarithm of the odds, i.e.:
The base of the logarithm function used is of little importance in the present article, as long as it is greater than 1, but the natural logarithm with base e is the one most often used. The choice of base corresponds to the choice of logarithmic unit for the value: base 2 corresponds to a shannon, base e to a “nat”, and base 10 to a hartley; these units are particularly used in information-theoretic interpretations. For each choice of base, the logit function takes values between negative and positive infinity.
The “logistic” function of any number is given by the inverse-logit:
The difference between the logits of two probabilities is the logarithm of the odds ratio (R), thus providing a shorthand for writing the correct combination of odds ratios only by adding and subtracting:
History
editSeveral approaches have been explored to adapt linear regression methods to a domain where the output is a probability value , instead of any real number . In many cases, such efforts have focused on modeling this problem by mapping the range to and then running the linear regression on these transformed values.[2]
In 1934, Chester Ittner Bliss used the cumulative normal distribution function to perform this mapping and called his model probit, an abbreviation for "probability unit". This is, however, computationally more expensive.[2]
In 1944, Joseph Berkson used log of odds and called this function logit, an abbreviation for "logistic unit", following the analogy for probit:
"I use this term [logit] for following Bliss, who called the analogous function which is linear on for the normal curve 'probit'."
— Joseph Berkson (1944)[3]
Log odds was used extensively by Charles Sanders Peirce (late 19th century).[4] G. A. Barnard in 1949 coined the commonly used term log-odds;[5][6] the log-odds of an event is the logit of the probability of the event.[7] Barnard also coined the term lods as an abstract form of "log-odds",[8] but suggested that "in practice the term 'odds' should normally be used, since this is more familiar in everyday life".[9]
Uses and properties
edit- The logit in logistic regression is a special case of a link function in a generalized linear model: it is the canonical link function for the Bernoulli distribution.
- More abstractly, the logit is the natural parameter for the binomial distribution; see Exponential family § Binomial distribution.
- The logit function is the negative of the derivative of the binary entropy function.
- The logit is also central to the probabilistic Rasch model for measurement, which has applications in psychological and educational assessment, among other areas.
- The inverse-logit function (i.e., the logistic function) is also sometimes referred to as the expit function.[10]
- In plant disease epidemiology, the logistic, Gompertz, and monomolecular models are collectively known as the Richards family models.
- The log-odds function of probabilities is often used in state estimation algorithms[11] because of its numerical advantages in the case of small probabilities. Instead of multiplying very small floating point numbers, log-odds probabilities can just be summed up to calculate the (log-odds) joint probability.[12][13]
Comparison with probit
editClosely related to the logit function (and logit model) are the probit function and probit model. The logit and probit are both sigmoid functions with a domain between 0 and 1, which makes them both quantile functions – i.e., inverses of the cumulative distribution function (CDF) of a probability distribution. In fact, the logit is the quantile function of the logistic distribution, while the probit is the quantile function of the normal distribution. The probit function is denoted , where is the CDF of the standard normal distribution, as just mentioned:
As shown in the graph on the right, the logit and probit functions are extremely similar when the probit function is scaled, so that its slope at y = 0 matches the slope of the logit. As a result, probit models are sometimes used in place of logit models because for certain applications (e.g., in item response theory) the implementation is easier.[14]
See also
edit- Sigmoid function, inverse of the logit function
- Discrete choice on binary logit, multinomial logit, conditional logit, nested logit, mixed logit, exploded logit, and ordered logit
- Limited dependent variable
- Logit analysis in marketing
- Multinomial logit
- Ogee, curve with similar shape
- Perceptron
- Probit, another function with the same domain and range as the logit
- Ridit scoring
- Data transformation (statistics)
- Arcsin (transformation)
- Rasch model
References
editThis article includes a list of general references, but it lacks sufficient corresponding inline citations. (November 2010) |
- ^ "Logit/Probit" (PDF).
- ^ a b Cramer, J. S. (2003). "The origins and development of the logit model" (PDF). Cambridge UP.
- ^ Berkson 1944, p. 361, footnote 2.
- ^ Stigler, Stephen M. (1986). The history of statistics : the measurement of uncertainty before 1900. Cambridge, Massachusetts: Belknap Press of Harvard University Press. ISBN 978-0-674-40340-6.
- ^ Hilbe, Joseph M. (2009), Logistic Regression Models, CRC Press, p. 3, ISBN 9781420075779.
- ^ Barnard 1949, p. 120.
- ^ Cramer, J. S. (2003), Logit Models from Economics and Other Fields, Cambridge University Press, p. 13, ISBN 9781139438193.
- ^ Barnard 1949, p. 120,128.
- ^ Barnard 1949, p. 136.
- ^ "R: Inverse logit function". Archived from the original on 2011-07-06. Retrieved 2011-02-18.
- ^ Thrun, Sebastian (2003). "Learning Occupancy Grid Maps with Forward Sensor Models". Autonomous Robots. 15 (2): 111–127. doi:10.1023/A:1025584807625. ISSN 0929-5593. S2CID 2279013.
- ^ Styler, Alex (2012). "Statistical Techniques in Robotics" (PDF). p. 2. Retrieved 2017-01-26.
- ^ Dickmann, J.; Appenrodt, N.; Klappstein, J.; Bloecher, H. L.; Muntzinger, M.; Sailer, A.; Hahn, M.; Brenk, C. (2015-01-01). "Making Bertha See Even More: Radar Contribution". IEEE Access. 3: 1233–1247. doi:10.1109/ACCESS.2015.2454533. ISSN 2169-3536.
- ^ Albert, James H. (2016). "Logit, Probit, and other Response Functions". Handbook of Item Response Theory. Vol. Two. Chapman and Hall. pp. 3–22. doi:10.1201/b19166-1.
- Berkson, Joseph (1944). "Application of the Logistic Function to Bio-Assay". Journal of the American Statistical Association. 39 (227 (September)): 357–365. doi:10.2307/2280041. JSTOR 2280041.
- Barnard, George Alfred (1949). "Statistical Inference". Journal of the Royal Statistical Society. B. 11 (2): 115–149. JSTOR 2984075.
External links
editFurther reading
edit- Ashton, Winifred D. (1972). The Logit Transformation: with special reference to its uses in Bioassay. Griffin's Statistical Monographs & Courses. Vol. 32. Charles Griffin. ISBN 978-0-85264-212-2.