Talk:Kelly criterion

Citation 14 is wrong

"Optimal Gambling System for Favorable Games" is by Breiman, not Thorpe. I haven't read the Breiman paper so I don't know if the correct citation is that paper by Breiman, or some other paper by Thorpe. TimothyFreeman (talk) 08:25, 14 April 2023 (UTC)

The citation is correct. There is a paper by Thorpe [1] of that name. Limit-theorem (talk) 11:47, 14 April 2023 (UTC)

A sentence in the introduction is wrong / confusing / deeply problematic

"This is done by maximizing the logarithm of wealth, which is mathematically simpler to do, and also maximizes wealth."

Wealth is a random variable (not a scalar), and you can't maximize wealth or the logarithm of wealth. You can maximize expected wealth or the expected logarithm of wealth, but those are NOT equivalent! $\max _{W\in {\mathcal {A}}}\operatorname {E} [W]$ is NOT equivalent to $\max _{W\in {\mathcal {A}}}\operatorname {E} [\log W]$. The former maximizes the expected return, the latter maximizes the expected geometric growth rate of wealth. The former leads to going broke with probability 1 (and measure 0 probability of infinite wealth), while the latter leads to the Kelly criteria.

The distinction between maximizing expected wealth and the expected logarithm of wealth is absolutely critical. Someone reading this introduction may come away with serious misconceptions.

I suggest instead

"... is a formula for bet sizing that leads almost surely to higher wealth compared to any other strategy in the long run. This bet size is found by maximizing the expected geometric growth rate (which is equivalent to maximizing the expected logarithm of wealth).

Mgunn (talk) 15:53, 9 January 2019 (UTC)

I went ahead and changed it. Mgunn (talk) 01:09, 10 January 2019 (UTC)

I disagree. Mgunn's statement above does correct the vague statement about "maximizing wealth", but the conclusion is incorrect. The first proof of the Kelly criterion given in the article maximizes the expected value of wealth after n bets of size x, which is (1+bx)^(pn) * (1-x)^(qn). It maximizes this by maximizing the log of the expected value of wealth, NOT the expected value of the logarithm of wealth. log(E[W]) = log((1+bx)^(np) * (1-x)^(nq)) = np*log(1+bx) + n(1-p)*log(1-x). We divide by n before maximizing, to maximize the log of the geometric mean of E[W]. So it does maximize the expected geometric growth rate, but maximizes the logarithm of expected wealth, not the expected logarithm of wealth. Reply with counterargument by Sept. 15, or I'll change it. Philgoetz (talk) 03:21, 3 September 2019 (UTC)

I believe it is computing the expected log of wealth, not the log of expected wealth. Wouldn't maximizing the log of expected wealth give simply the same solution as maximizing expected wealth directly? (An expected value is a deterministic number, not a random variable, and a logarithm is a simple monotonic function, so whatever maximizes the expected value of something will also maximize the log of that expected value. Wealth, however, is a random variable, so the log of wealth is also a random variable, and maximizing the expected value of wealth might provide a different solution from maximizing the expected value of the log of wealth.) —BarrelProof (talk) 00:47, 4 September 2019 (UTC)

General Formula Wrong

The formula f = p/a-q/b is wrong. It should be f = p-q*a/b. It simplifies to f = (pb-qb)/b = expected profit / gains if success.

The way it's stated, f is usually bigger than 1, which is incorrect. This should be corrected. — Preceding unsigned comment added by 189.112.213.146 (talk) 23:04, 21 March 2013 (UTC) Agreed the formula given gives ridiculous numbers (despite the apparent "proof"). E.g. p=0.8, q=0.2, a=0.25, b=0.10, so four out of five times we invest we make a 10% profit, the fifth time we make a 25% loss. The f = p/a-q/b forumula tells us to invest 120% of our capital (0.8/0.25 - 0.2/0.1 = 3.2 - 2.0 = 1.2). Not only is this incorrect but it's dangerous as users may take this formula and invest/bet amounts which are detrimental to them, and possibly bankrupting. There are no references to the wrong formula, so I intend to correct it to the formula you have mentioned, which is backed up my many sources elsewhere on the internet and reference them. In the event I am interrupted before I can put the edit in, please can somebody else do the correction? Here's an example of a reference with the correct formula (p-q*a/b): http://www.investopedia.com/terms/k/kellycriterion.asp — Preceding unsigned comment added by 85.255.232.117 (talk) 18:43, 7 May 2016 (UTC)

You're right. Sorry it took so long to fix. -- Derek Ross | Talk 17:37, 31 August 2019 (UTC)

You guys are going off the rails here. The formula p-qa/b gives the Kelly fraction to RISK. The formula p/a-q/b gives the Kelly fraction to BET. You want the second one, and these are not the same in your example. You are risking the amount a, but you are betting the different amount 1 dollar ("the value of your investment"). The 2 formulas differ by a factor of a, and they are only the same when a=1, that is, when you can lose your entire bet. In the example given above, the answer of 120% is neither ridiculous nor incorrect - it is correct. You need leverage to bet the Kelly optimal in this case, but betting 120% of the bankroll means that we are only placing 30% of the bankroll at risk since we can only lose 25% of our bet. To see that this is optimal, simply note that 120% indeed maximizes the expected value of the log of the bankroll 0.8log(1+0.1f)+0.2log(1-0.25f). This should be clear from the derivation of p/a-q/b under PROOF. This last revision should be reversed. Brucezas (talk) 19:46, 3 September 2019 (UTC)

There should probably be a statement pointing out that this formula can advise betting more than 100% of the bankroll, but that the amount at risk will be less than 100%. The amount at risk can be obtained by scaling the result by a. Otherwise this is likely to be a common source of confusion. Brucezas (talk) 19:53, 4 September 2019 (UTC)

Factual basis of Kelly Criterion is in dispute

The Kelly Criterion is not universally accepted in the mathematical community. For example, see http://www.bjmath.com/bjmath/kelly/mandk.htm. The dispute seems to hinge on the fact that the choice of utility function is arbitrary. There is no reason to prefer the log utility function in the current version of the article over others a priori, as any monotonically increasing utility function will result in infinite predicted wealth with time.

Sorry but this has nothing to do with utility theory. Kelly's theory that you should choose the result with the highest expected log result. This is the same as Bernoulli's theory with a choice of bets you choose that with the highest geometric mean of outcomes. Perhaps utility theory can be derived from Kelly, but it's not necessary for Kelly. This source isn't great it's from a blackjack forum, and the writer doesn't appear to have academic credentials. Please don't forget to sign your posts. --Dilaudid (talk) 12:13, 24 January 2008 (UTC)

To quote from Kelly's original paper "The reason (for the log rule) has nothing to do with the value which (the gambler) attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets" 158.143.86.159 (talk) 16:14, 25 January 2010 (UTC)

Reading some of his paper, the math is about maximizing the log of the expected value, but he seems to justify it on the basis of maximizing the mode. Someone who uses the Kelly Criterion will probably do better than someone who does not, even though it's easy to get a higher expected value than someone who uses it. — DanielLC 22:09, 10 December 2014 (UTC)

The formula can be simplified:

(bp-q)/b => b(p-q/b)/b => q=1-p so k=p-(1-p)/b

--Geremy78 09:49, 28 January 2006 (UTC)

Generalized Kelly Strategy

The "generalized form of the formula" given in the article isn't really the most general. The most general expression of the Kelly criterion is to find the fraction f of the bankroll that maximizes the expectation of the logarithm of the results. For simple bets with two outcomes, one of which involves losing the entire amount bet, the formula given in the article is correct and is easily derived from the general form. For bets with many possible outcomes (such as betting on the stock market), the calculation is naturally more complicated.

One statement in the article,

In addition to maximizing the long-run growth rate, the formula has the added benefit of having zero risk of ruin, as the formula will never allow 100% of the bankroll to be wagered on any gamble having less than 100% chance of winning.

isn't strictly true from a theoretical standpoint. It is always true that Kelly strategy has zero risk of ruin, but in the general case it is not true that a bet of 100% of the bankroll is not allowed. If the probability of losing the entire amount of the bet is zero, then bets of 100% and even larger (buying stocks on margin, for example) are allowed. Investing in a stock index (as opposed to a single stock or small number of stocks) could allow such percentages, if we assume the index can never go broke (even though individual stocks might), and that the index has a positive expectation of outcome (adjusted for inflation, since we are dealing with money invested over time). Of course, any real-world investment will have a non-zero chance of going bust, and therefore Kelly strategy will indicate a bet of less than 100% of bankroll.

Rsmoore 07:55, 4 February 2006 (UTC)

Formula presentation

While the Kelly Criterion formula can be "simplified" to remove the q term, it actually becomes longer, less intuitive, and harder to remember. As a result, it is generally presented as (bp-q)/b.

Whoever "corrected" the formula to (bp-1)/(b-1), this is incorrect. I have changed it back to the correct formula. I cite as my source William Poundstone's book Fortune's Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street. Additional sources can also probably be found all over the internet.

(bp-1)/(b-1) is not incorrect, it depends on the definition of odds, i.e. how "b" is defined. (bp-1)/(b-1) is for European/decimal odds. —Preceding unsigned comment added by Cwberg (talk • contribs) 13:16, 11 January 2008 (UTC)

Added disadvantages section

The information in that section is all from reading Poundstone's book. The book is very very verbose and completely non-mathematical - designed for bedtime reading I expect. It could have been condensced to 1/10th. of the length without losing much. I have seen more concise explainations of Kelly in other popular books about chance, although I do not remember reading before about the volatility problem or over-betting. Something only mentioned in one sentance is that its easy to overestimate the true odds and unconciously overbet, leading to ruin. It is suggested this happened to Long Term Capital Management.

There are no explainations of the maths behind the information stated in the book - you have to take it on trust. For those with busy lives you can find all the relevant info by looking up Kelly criterion and geometric average in the index. The pages in the 2005 hardback edition I thought were most relevant were pgs. 73, 191, 194-201,229,231,232, 297, 298.

It does have an extensive bibliography, and there is a reference for: Bernoulli, Daniel (1954) "Exposition of a New Theory on the Measurement of Risk" Trans. Louise Sommer, Econometrica 22:23-36. Wonder if its available online? Henry A Latane/ did some academic papers about the geometric mean criterion. The books (academic or popular) of William T Ziemba also seem of interest, including Beat the Racetrack.

Poundstone describes the Kelly criterion in his own way (pg.73): he says you should gamble the fraction edge/odds of your bankroll. Edge is how much you expect to win on average. Odds are the public or 'tote-board' odds. Example: the tote board odds for a horse are 5 to 1. You think the horse has a 1 in 3 chance of winning. So by betting on the horse you on average get $200 back for a $100 stake, giving a net profit of $100. The edge is the $100 profit divided by the $100 stake, giving 1. So in this case the edge is 1. The odds are 5 to 1 - you only need the 5. So edge/odds is 1/5 - you should bet one fifth of your bankroll.

As someone who has never gambled on races, I find "odds" confusing. I wish someone would also provide a formula in the article where only p is used, that is more suitable for use with investments.

Where the book really falls down is in describing multiple bets. Poundstone just baldly says you can bet more of your bankroll with simultaneous bets - but he dosnt give any how or why, although this would be useful to know. Perhaps he doesnt understand this himself.

As someone who is currently making heavily geared real-estate investments, I think the encyc. article should go into much more detail than currently, including practical applications. I wish I had some guidance on how much I should optimally invest. I find the idea of choosing the greatest geometric mean much easier to understand than the Kelly criterion. In business investments I suppose you would take the geometric mean of the expected net present values - or would you?

As Poundstone points out (I think), the geometric average rule does have a flaw. For example, if you had a bet for a $10 stake where you had a 99% chance of winning $1000000 and a 1% chance of winning $0, then the geometric mean criteria would tell you to ignore this bet completely! (Please tell me if I've got this wrong.)

The book is about 90% chat about financial things only tenuously linked to Kellys criteria - about various imprisoned and/or ruined Wall St. multi-millionaires, about the links one large well known entertainment company is said to have/had with the Mafia. It says nothing about Shannon's communication theory, and zilch about the links between this and Kelly's criterion, which was my reason for ordering the book. It does describe Thorp quite a lot though.

Perhaps someone could add some references to some more concise popular expositions of the criterion.

For example, if you had a bet for a $10 stake where you had a 99% chance of winning $1 000 000 and a 1% chance of winning $0, then the geometric mean criteria would tell you to ignore this bet completely! (Please tell me if I've got this wrong.)

Let's calculate:

f == (p*b - q)/b

in this case, p = 0.99, q = 1-p = 0.01, and b = 999 990, so

f = ( 0.99*(999 990) - 0.01) / ( 999 990 ) = 0.99000

. So the geometric mean criteria recommends you keep 1% in your pocket, and bet everything else.

--68.0.120.35 19:55, 3 March 2007 (UTC)

Hi I think RE the 1% chance of winning $0, I think you meant a 1% chance of losing all your money. Then the Kelly criterion would say do not bet. —Preceding unsigned comment added by 82.26.92.226 (talk) 09:49, 4 January 2008 (UTC)

A lot of this section is factually incorrect and needs to be revised. (I have no clue how to sign this, nor do I just wanna lop the whole section out of the page. But the 2nd and 4th paragraphs in this section are either misleading or factually incorrect)

The clue how to sign this is just above the edit panel. – Do you mean part of the article is wrong, or part of this section of the Talk page? The Talk page is not part of the article, it's for discussing what belongs in the article, so errors and misunderstandings are to be expected; we do not "correct" what others post here. —Tamfang (talk) 03:22, 8 December 2009 (UTC)

Shannons stock system

Continuing from the above, Poundstones book also mentions an interesting (theorectical) investment system devised by Shannon.

Shannons actual stock investments (the book says) were buy and hold. He selected stocks by extrapolating earnings growth (using human judgement). Two or three of Shannons stocks accounted for nearly all the value of his portfolio.

He also devised an interesting theorectical system for investing in stock with a lot of volatility but no trend (pg. 202). Put half your capital into stock and half into cash. Each day rebalance by shifting from stock to cash or vice versa to keep these proportions. Surprisingly, the total value grows. In practice the dealing commissions would remove any profit.

This system is now known as a "constant-proportion rebalanced portfolio", and has been studied by economists Mark Rubenstein, Eugene Fama, and Thomas Cover.

Mark Rubinstein was an economist (or "financial engineer") and Eugene Fama is an economist, but Thomas Cover was an information theorist rather than an economist. Proportional rebalancing is a very common strategy in portfolio management. It is advocated in a very large amount of the personal finance literature, there are "balanced" mutual funds based on it, etc. —BarrelProof (talk) 01:09, 1 September 2019 (UTC)

Kelly Criterion For Stock Market

Kelly Criterion For Stock Market should be merged into this article. (Nuggetboy) (talk) (contribs) 19:11, 25 January 2007 (UTC)

I think merging will create a lot of confusion. For practicality they should be separated because Kelly Criterion For Stock Market requires the reader to have some math and finance background. (User:Zfang)

I would support merging. The article here is much more encyclopedic, the one to merge has some tone/content issues - the approach is more instructional and doesn't sound appropriate here.--Gregalton 22:44, 15 February 2007 (UTC)

There's a huge probleme with the objectif of this quadratic problem. At first, Q is not positive definite and Dim(Ker(Q)) = N-1, so under these constraints, in the best case, we can find N-3 different solutions which make the result unstable. Just try do find Xi+1 = Xi + dXi and look at the norm of dXi. If you use a "regular" solver with an iterative method, you'll not converge in dXi but in dU (utility). That implies the problem is ill-posed. We cannot use it that way. The Bad Boy (talk) 14:33, 9 October 2013 (UTC)

The Application's to Stock Market section is quite poorly written and should, in my opinion, be removed. — Preceding unsigned comment added by 104.162.109.38 (talk) 09:12, 12 September 2016 (UTC)

Rigorous statement of the objective function

I am still puzzled what the objective function is that we are trying to maximise and what the constraints are. In my view, the article would benefit from a precise mathematical expression. I take it it's the limit of the expected wealth as the number of periods goes to infinity? For any finite number of periods T, I suppose I could beat Kelly, for instance, by betting Kelly until time t and put all my money on one side in the last round. — Preceding unsigned comment added by Derfugu (talk • contribs) 11:25, 26 March 2011 (UTC)

Derfugu (talk) 11:26, 26 March 2011 (UTC)

It's maximizing the log of the wealth. The reason for this doesn't seem to be very justified. It mentions that using a different probability consistently will result in less wealth with a probability that approaches unity, but the example you gave will do better most of the time (assuming you're putting all your money on the most likely side, even if this reduces the expected value). — DanielLC 22:14, 10 December 2014 (UTC)

In the article it seems to be missing the fact that we are looking to maximise the expected value of the log of the wealth. Since I find it very confusing (I only found out the answer by checking this talk page), I'm going to add a line in the intro to make it explicit. Student73 (talk) 13:29, 18 November 2017 (UTC)

formula derivation

Rather "take it on trust" that the formula is correct, (or, worse, refuse to believe and repeatedly substitute random variations), I would much rather people check my calculation and fix any flaws I introduce.

• f is the fraction of the current bankroll to wager;
• b is the odds received on the wager;
• p is the probability of winning;
• q is the probability of losing, which is 1 − p.
• m0 is your initial bankroll.
• m1 is your bankroll after 1 round of the game.
• When you lose, you lose the entire amount of your wager.
  • when you lose, the amount in-the-bankroll becomes m1=m0*(1-f).
• When you win, you win back your wager, plus the amount bet times b, the payoff odds.
  • when you win, the amount in-the-bankroll becomes m1=m0*(1+b*f).

Before I derive it, let me list some characteristics I expect the "correct" formula to have:

• as long as p is strictly less than 1.0, f is less than 1.0 : "never bet the whole wad".
• If p is 1.0 and b is greater than zero, f should be exactly 1.0 : "bet the whole wad on a sure win"
• If p is 0, f should be ... less than 0 ?
• as b increases from q/p ("expect to break even") up, f should increase monotonically ("non-decreasing") from 0 towards 1.0. "Even if there is only a 1 in 10 chance of a horse winning, you can still make money if the odds are 11 to 1"
• As p increases from 1/(1+b) ("expect to break even") up to 1.0, f should increase monotonically from 0 to 1.0 .

We want to maximize the geometric mean of the ... (fill in here ...). To do that, we pick f to maximize the expected value of the log of the final amount in-the-bankroll m1.

pick f to maximize g(f), where

 g(f) = expectation( log( m1 ) ) =
 g(f) = p*log( m1_when_we_win ) + q*log( m1_when_we_lose ) =
 g(f) = p*log( m0*(1+b*f) ) + q*log( m0*(1-f) ) =
 g(f) = log(m0) + p*log(1+b*f) + q*log(1-f).

For a smooth function like this, the maximum is either at the endpoints (f=0 or f=1.0) or where the derivative of the function is zero: ( k1 depends on whether we use log10(), log2(), loge(), etc. -- but it turns out to be irrelevant. )

 (d/df)g(f)= 0 + p*k1*( 1/(1+b*f) )*b + q*k1( 1/(1-f) )*(-1) =
 (d/df)g(f)= p*k1*b/(1+b*f) - q*k1/(1-f)
 find f where 0 == (d/df)g(f).
 0 == p*b/(1+b*f) - q/(1-f)
 0*(1+b*f)*(1-f) == p*b*(1-f) - q(1+b*f)
 0 == p*b - p*b*f - q - q*b*f
 0 == p*b - q - ( p*b + q*b )*f
 0 == p*b - q - ( b )*f
 f == (p*b - q)/b

And there we have it.

(Should I cut-and-paste this derivation into the article, like Kelly Criterion For Stock Market includes the derivation in the article?)

new term:

• n is the "know-nothing" estimate of the probability given only the posted odds b, n = 1/(b+1), b=(1-n)/n
  • If the true probability of winning exactly matches the posted odds (p=n), I expect f=0.

Other ways of expressing the value of f:

 f == (p*b - q)/b
 f == p - (1-p)/b
 f == p - (q/b)
 f == (p(b+1) - 1)/b
 f == (p-n)/(1-n)
 f == 1 - q/(1-n)

Special cases:

 for even-money bets (b=1, so n=0.5), f=p-q.
 for "huge payoff" bets, where 1 << b but p << 1, we can approximate f ≅ p - 1/b ≅ p - n.

-- User:DavidCary --68.0.120.35 19:55, 3 March 2007 (UTC)

A minor historical worry: In Kelly's paper, he uses the word "odds" but not to mean what you mean here. He had his odds such that the (fair) odds are the reciprocal of the probability. This is the situation where eg. odds on a fair coin landing heads are 2 to 1, not 1 to 1. See p. 921 of his paper. This doesn't fit with odds nor with your "b" above. I'm not questioning the derivation, I'm wondering how (or whether) to present the divergence of terminology... 158.143.86.159 (talk) 16:18, 25 January 2010 (UTC)

--User:derfugu

I thought this is rather clear in the article. We speak of "to" odds, when we mean the fraction of the wager that we receive in addition to the original wager and of "for" odds when we mean including the wager. Hence the 'fair' odds (fair in a sense that the implied probabilities sum up to one) in coin tossing are 1 to 1 and 2 for 1. — Preceding unsigned comment added by Derfugu (talk • contribs) 11:05, 26 March 2011 (UTC)

Proof section

Hi, I deleted some stuff from the proof section - it appeared to me that the section on how to underbet and beat kelly was original research and I don't think it works - here's the section: suppose you bet a small amount less than Kelly on the first bet, and double the amount less than Kelly every bet until you finally lose a bet. At that point, you will be a small amount ahead of Kelly. If you follow Kelly thereafter, you will end up better than Kelly. This will happen unless you start with such a long string of wins that you can no longer double the amount you underbet Kelly. You can make the probability of this as low as you want by making the initial underbet very small. Thus you can come up with a strategy that will always outperform Kelly.. This strategy cannot be better than Kelly - the reason is simple, if it were better than Kelly, I would use the strategy on itself. Then I would have a strategy that was better than the better strategy. Ad infinitum. --165.222.184.132 (talk) 09:01, 16 March 2009 (UTC)

You are correct that the argument can be used on itself to form an infinite series of better strategies. But that does not invalidate its point that it is possible to find a strategy better than Kelly. Whether this doubling strategy actually is better than Kelly, or even different from Kelly is a matter of dispute. Certainly it doesn't point to a practical way to improve on Kelly.

Anyway, the doubling argument is often made, and it was referenced. Whether you agree with it or not is not the point in Wikipedia. I do not think it should have been deleted. On the other hand, I don't have any strong feeling that it needs to be included. AaCBrown (talk) 21:04, 10 October 2013 (UTC)

This scheme is like the argument that is known as Leib's paradox, and a proof concerning it appears in the 2008 article Understanding The Kelly Criterion by Edward O. Thorp--- the man. The article is a revised reprint from two columns from the series A Mathematician on Wall Street in Wilmott Magazine, May and September 2008. Thorp explains, contrary to the commentary here to the effect that nothing like this could work, that Lieb's trick works. Here's Thorp's summary of it:

"To prove the first part, we show how to get ahead of Kelly with probability 1 − ε within a finite number of trials. The idea is to begin by betting less than Kelly by a very small amount. If the first outcome is a loss, then we have more than Kelly and use the strategy from the proof of the second part to stay ahead. If the first outcome is a win, we’re behind Kelly and now underbet on the second trial by enough so that a loss on the second trial will put us ahead of Kelly. We continue this strategy until either there is a loss and we are ahead of Kelly or until even betting 0 is not enough to surpass Kelly after a loss. Given any N , if our initial underbet is small enough, we can continue this strategy for up to N trials. The probability of the strategy failing is p N , 12 < p < 1. Hence, given ε > 0, we can choose N such that p N < ε and the strategy therefore succeeds on or before trial N with probability 1 − p N > 1 − ε."

My first thoughts--- I have to study the paper more--- are that the resolution may be to understand that the stated and mathematically proven Kelly "dominance", as Thorp sometimes calls it, the superiority, is established in a gross statistical sense and only with respect to what he calls (and mathematically defines) "essentially different" strategies. A strategy that is a close shadow of Kelley that is generally implemented for only a few trials is not essentially different. The basic meaning of that could be that the Leib strategy could in the long run amount to small potatoes compared to the difference between Kelley and an essentially different strategy.Mihael O'Connor (talk) 12:04, 6 April 2014 (UTC)

Two disagreements

There are two sentences in this article I disagree with:

"However, as is evident in the derivation below, the Kelly strategy is applicable only when the same proportion of one's bankroll is invested in the same investment vehicle ad infinitum, or at least a huge number of times, e.g., thousands or even millions of times.[citation needed]"

There is no assumption in Kelly of either the same proportion of bankroll or the same investment vehicle. It does assume a large number of independent bets, but this is discussed later in the article. And "large" need not mean thousands or millions.

"In conclusion, the Kelly strategy is the best strategy for beating a casino, because it is (upper limit) invariant."

Kelly is irrelevant for most casino games, because they have negative expectation and do not allow negative bets. The last clause requires a lot more discussion to be meaningful, and it doesn't support the rest of the sentence.

I'm going to take them out.

AaCBrown (talk) 02:42, 21 June 2010 (UTC)

Definition of a goal function that is to maximized

The article does not clearly state what goal is being maximized and where the logarithm comes from.

So my first attempt to make sense of the article was to maximize the expected profit after n bets. This leads to f being 1 under favourable conditions (p(b + 1) > 1) and 0 otherwise. Clearly not the Kelly criterion.

Then I realized that, under favourable conditions, a skilled gambler will be able to break the bank with probability 1. (Or to reach a target wealth). That lead me to second goal function, namely to minimize the expected time to break the bank. With this goal, he will never bet all his money, because then there is a positive probability that he will never break the bank leading (making the expected time infinite). With f* < 1, the logarithm of the capital follows a one dimensional Random walk and the gambler wants to maximize the rate at which he is moving towards the target. Now the Proof section makes sense. -- Nic Roets (talk) 10:54, 4 January 2012 (UTC)

Link to Kelly Calculator

I believe having a link to an interactive kelly calculator adds value to this page for the reader. I have a kelly calculator on my site and I posted a link to it from here under a section titled External Links at the bottom of the page. Over time, people saw it, and posted links to their own calculators and it became spammy. An editor recently removed them all of them and asked that anyone who wants to have their link restored has to justify it here first. I think the value it brings to the reader justifies the link itself and the record of the page will show that the link to the calculator on the website RLD Investments was the first one posted. — Preceding unsigned comment added by RyanRLD (talk • contribs) 17:21, 12 December 2013 (UTC)

Blacklisted Links Found on Kelly criterion

Cyberbot II has detected links on Kelly criterion which have been added to the blacklist, either globally or locally. Links tend to be blacklisted because they have a history of being spammed or are highly inappropriate for Wikipedia. The addition will be logged at one of these locations: local or global If you believe the specific link should be exempt from the blacklist, you may request that it is white-listed. Alternatively, you may request that the link is removed from or altered on the blacklist locally or globally. When requesting whitelisting, be sure to supply the link to be whitelisted and wrap the link in nowiki tags. Please do not remove the tag until the issue is resolved. You may set the invisible parameter to "true" whilst requests to white-list are being processed. Should you require any help with this process, please ask at the help desk.

Below is a list of links that were found on the main page:

• http://www.bjmath.com/bjmath/thorp/ch2.pdf

  Triggered by \bbjmath\.com\b on the global blacklist

If you would like me to provide more information on the talk page, contact User:Cyberpower678 and ask him to program me with more info.

From your friendly hard working bot.—^{cyberbot II}_{Talk to my owner:Online} 16:56, 11 August 2015 (UTC)

The bjmath link in the article (and above) returns a 404 error.

What link are you talking about? I replaced the link months ago with this one from Ed Thorp's own site: http://www.edwardothorp.com/sitebuildercontent/sitebuilderfiles/beatthemarket.pdf AndyBloch (talk) 23:45, 11 November 2015 (UTC)

Parallel bets

The Kelly criterion assumes that only one bet is available during each time period, and maximizes the growth over time by using that one opportunity to its fullest. If however multiple bets are available at any given time, that run in parallel and whose outcomes are not completely correlated, then it makes sense to distribute a smaller amount of money over each bet.

For example if p=0.5 and B=2, the Kelly bet is 25%. If three of these bets are available at the same time and are uncorrelated, then the optimum bet (maximizing EV of log) is 21% each -- a total of 63% betted. If ten are available, then optimum bet is 9.96% each (total 99.6%). As a rough guide, if N equivalent and uncorrelated bets are available and Kelly says to bet much less than 1/N, then the N bets should be made with values slightly less than Kelly. On the other hand if Kelly says to bet more than 1/N then one should (and can only) do N bets of a bit less than 1/N. [these guides change drastically if there are correlations]

In other words, another reason to de-rate the Kelly bet is its opportunity cost, that the money betted could be spread out into other avenues. Moreover, if the opportunities have partially correlated outcomes, then this gives additional push in EV(log) to strongly de-rate the Kelly bet on each. The combination of these effects thus offers a naturally emerging reason for fractional Kelly betting. I am sure these reasons are mentioned in some finance book somewhere and would be worth mentioning in the introduction, besides the "practical reasons, such as wishing to reduce volatility, or protecting against non-deterministic errors in their advantage (edge) calculations".

(In, general with correlated probabilities one sees the emergence of hedged strategy like long/short equity. This is similar to the section "Many Assets" but without the Taylor approximation.) --Nanite (talk) 20:18, 22 November 2016 (UTC)

I was searching for the Kelly optimum at parallel bets, what you write intuitively seems correct. Can you give a link to a paper or similar? Thanks. Richard — Preceding unsigned comment added by 141.69.58.57 (talk) 16:29, 2 November 2017 (UTC)

Irrational coin tossers

In paragraph 2 I think it's worth noting that the sample of 61 people was made up of a combination of college-age students in finance and economics and some young professionals at finance firms (including 14 who worked for fund managers)[1], as a random sample of the general population would probably have fared even worse.

[1] https://www.economist.com/blogs/buttonwood/2016/11/investing

111.220.125.72 (talk) 12:42, 4 March 2018 (UTC)

Lead excessively anecdotal

... and can be counterintuitive. In one study, each participant was given $25 and asked to bet on a coin that would land heads 60% of the time. Participants had 30 minutes to play, so could place about 300 bets, and the prizes were capped at $250. Behavior was far from optimal. "Remarkably, 28% of the participants went bust, and the average payout was just $91. Only 21% of the participants reached the maximum. 18 of the 61 participants bet everything on one toss, while two-thirds gambled on tails at some stage in the experiment." Using the Kelly criterion and based on the odds in the experiment, the right approach would be to bet 20% of the pot on each throw (see first example below). If losing, the size of the bet gets cut; if winning, the stake increases.

This lead is very heavy for the length of the article to begin with. I'd personally consider moving the anecdotal material out of the lead. — MaxEnt 00:55, 11 August 2018 (UTC)

I find the example interesting. I think it gives the reader a good feeling for the basic idea of the topic, and it seems much more interesting than just a dry description of mathematical properties. It also introduces the reader to the difference between mathematics and human behavior, and brings to mind interesting questions such as why people behaved differently from the described strategy. But I notice some different problems with it:

• It says that only 21% of the experiment participants reached the maximum of $250, but it does not say what the percentage of people would be who reach the maximum if they had followed the Kelly criterion, so we have no way of seeing how far from optimal the behavior was.
• It says that the average payout was just $91, but it does not say what the average payout would be if the players were following the Kelly criterion, so again we cannot see the difference from optimiality.
• In this game, the goal is not the same as the goal of the Kelly criterion.
  • This game has a maximum payout of $250. The strategy for maximizing the probability of a payout of $250 would presumably be somewhat different from the strategy for a gain with an unlimited maximum payout.
  • For example, if someone playing the game reaches a winnings total of more than $250 after only 20 coin tosses, they should just bet nothing for the rest of the period of the game, because they have no incentive to continue; they can only lose by further participation.
  • This game has a limited duration, and the Kelly criterion is for a game of unlimited duration.
  • I don't know whether these factors would make a significant difference in the optimal strategy or not, and the article doesn't seem to say whether they would or not.

—BarrelProof (talk) 00:45, 1 September 2019 (UTC)

After some work to calculate the precise probabilities, I added some information about these issues to the article. It turns out that the presence of the cap does significantly affect the optimal strategy. Ceasing to bet after reaching the cap also helps a little. If players used the Kelly rule of betting 20% of the pot, 94% of them would reach the cap and the average payoff would be $237.36 (assuming they stop betting if they reach the cap, since further betting can only cause them a loss of value). Betting 12% of the pot on each toss rather than the suggested Kelly bet of 20% produces even better results (95% probability of reaching the cap and an average payout of $242.03). Further refinement of the playing strategy also seems reasonable – e.g., a smart player would never bet more than the difference between the current pot value and the cap, since such a bet increases downside risk without increasing the reward. But calculating the probabilities for that strategy seems more difficult. —BarrelProof (talk) 05:54, 5 September 2019 (UTC)

Incidentally, I also moved the example out of the lead section, as suggested. —BarrelProof (talk) 15:14, 5 September 2019 (UTC)

Lead lacks concision

The conventional alternative is expected utility theory which says bets should be sized to maximize the expected utility of the outcome (to an individual with logarithmic utility, the Kelly bet maximizes expected utility, so there is no conflict; moreover, Kelly's original paper clearly states the need for a utility function in the case of gambling games which are played finitely many times). Even Kelly supporters usually argue for fractional Kelly (betting a fixed fraction of the amount recommended by Kelly) for a variety of practical reasons, such as wishing to reduce volatility, or protecting against non-deterministic errors in their advantage (edge) calculations.

Okay, semicolon inside a parenthetical—I do that all the time (but not on Wikipedia). And then it veers off into camp dissection (even Alex Jones concedes that Hillary stands by her man).

Couldn't we just say that Kelly theory is equivalent to expected utility theory under a logarithmic utility function?

This seems too prolix for what it conveys. — MaxEnt 01:04, 11 August 2018 (UTC)

Yes, one may shorten, but while stressing that it is coincidental. It so happened that maximizing the rate of growth maps to log utility. Limit-theorem (talk) 11:02, 11 August 2018 (UTC)

Grammar: "probability of win" vs. "probability of a win" (940400173)

@Stdazi:

No need to add legs for the snakes on a painting, don't you think so?

— Wikipedian Right (talk) 13:10, 16 February 2020 (UTC)

"Multiple outcomes" formulas are wrong

If you read the cited paper for this section, the steps taken in the paper are quite different to those given here.

Step 1 (calculating earning rate) is given in the paper (formula 2.3) as:

$er_{i}={\frac {Dp_{i}}{\beta _{i}}}=p_{i}(Q_{i}+1)$

In step 4, you need to recalculate the reserve rate. The paper cited suggest this is the correct formula (6.8 in the paper):

$R(S)={\frac {D\sum _{k\in S}p_{k}}{D-\sum _{k\in S}\beta _{k}}}$

The paper then suggests (on page 9) that the optimal Kelly fractions are then calculated using formula 6.5:

$f_{i}=p_{i}-\beta _{i}{\frac {\sum _{k\in S}p_{k}}{(D-\sum _{k\in S}\beta _{k})}}$

I would like to propose these changes, but am concerned I'm misinterpreting the paper.

- PaulRobinson (talk) 18:11, 19 May 2020 (UTC)

A year ago, I used this page in the past to create a software implementation of this algorithm, and it worked. Today, I tried creating another software implementation, again using this page, but I was getting some very strange results. I can say almost definitively that the algorithm now described in this section as a consequence of these changes is incorrect. If you try the case of a single outcome, the algorithm will always recommend you bet nothing.

Meowxr (talk) 11:22, 19 February 2021 (UTC)

Removed paragraph from introduction

I removed this paragraph. The video (questionable source on its own) was not backing the claim. The video shows that bets that are too large (way larger than the Kelly criterion) will ruin you. That's correct, but that's not betting according to the Kelly criterion. The rest of that pararaph (completely unsourced) was just wrong or applies to fair bets only, but for fair bets the Kelly criterion says not to bet. --mfb (talk) 02:44, 3 October 2020 (UTC)

• Reverted you. Gambler's Ruin and Impossibility of a gambling system are not "nonsense" Ipso facto, "Impossibile" means Kelly is not a possibility. No more credible reference than Taleb and his video/posting. Richard J Kinch (talk) 22:50, 5 October 2020 (UTC)

These two don't apply here, as you easily see from the articles already. --mfb (talk) 02:29, 6 October 2020 (UTC)

Don't apply? You believe otherwise? If something is proved impossible, that applies universally. If a successful gambling system is proven to be impossible, and Kelly is a gambling system, then Kelly is proven to be not successful. It is imperative for this article to say so, because there is a widespread misunderstanding that Kelly is a system to gamble successfully, when it is only a system to gamble optimally by some metric. Optimization is not success. Maximizing temporary winnings does not prevent inevitably going broke. Censoring these well-established and referenced facts is a cruel disservice to those who look to an encyclopedia for answers to controversial subjects. Kelly is optimal, but only in being the slowest way or highest peak before inevitable ruin. Stop denying this truth. Richard J Kinch (talk) 03:44, 8 October 2020 (UTC)

Read what exactly is impossible. It's impossible to have a winning strategy if your bets are even or have a disadvantage for you (Gambler's Ruin). Irrelevant for this article, based on Kelly you only bet if you have an advantage. It's impossible to avoid going broke if you increase bets but don't derease them (Gambler's Ruin). Again irrelevant for this article, because the Kelly criterion reduces bets after losses. It's impossible to improve the chance to win in a specific round (impossibility of a gambling system). Again irrelevant for this article, Kelly doesn't try that.

Here is a simple example that we can fully analyze on the talk page: You bet all your money, with 40% you get half of your money back, with 60% you get twice your money back. Kelly will tell you that's not optimal but that's okay. On a logarithmic scale this simply means going one step to the left (losing) or right (winning) in a biased random walk. If we start at $10.24 then 1 cent ("broke") is 10 steps to the left. No limit to the right. This is discussed e.g. here but with Roulette, i.e. a bias towards the left. The probability to reach a state -m to the left is ((1-p)/p)^m (second to last equation in 20.2.3). Plugging in values (p=0.4, m=-10) we get a 1.7% chance to go broke (at any point in the future) and a 98.3% chance to play forever. --mfb (talk) 05:07, 8 October 2020 (UTC)

Your position is essentially to deny Gambler's Ruin and Impossibility of a gambling system. Yet you cite an MIT course that teaches those precise concepts. By your logic any rational person can gamble, play forever, and hence random-walk into eventually owning the universe. That's absurd. Let me refer you to Taleb again, and his criticism of IYIs. Unfortunately I don't have Taleb's courage or vigor to carry on an edit war, so I hope someone else will step in. In the event these principles continue to be stripped out of the article, I'll leave this dissent for the objectively curious. Richard J Kinch (talk) 03:06, 10 October 2020 (UTC)

I discussed in detail how these two concepts are not relevant here. They exist, they are correct, they are important in most situations, but they do not apply here. The statements "with a fair bet you cannot win" and "with a bet in your favor you can win in some cases" are not in conflict, they look at mutually exclusive conditions so different results are not surprising. Your response is completely missing this, and doing nothing to argue against my example. If you have the conditions where the Kelly criterion tells you to gamble, then yes, a rational person should do that, forever. A real-life gambling system will never provide you these conditions, because that gambling system would go out of business quickly. But if you have these conditions for whatever reason then do gamble as long as you can. --mfb (talk) 22:04, 13 October 2020 (UTC)

Is the example given incorrect?

Shouldn't $b$ in ″(e.g. betting $10, on win, rewards $4 plus wager; then $b=0.4$)" be $1.4$ instead? — Preceding unsigned comment added by Convolutional Network (talk • contribs) 05:00, 4 February 2021 (UTC)

If you include the initial wager in b then you need to replace (b+1) by b and b by (b-1). That's a possible alternative way to write things. --mfb (talk) 05:41, 4 February 2021 (UTC)

If b is the "net odds," and if b is 0.4 in the given example, then this definition of b directly conflicts with other statements within the main article, including, "if b=q/p, then the criterion recommends for the gambler to bet nothing," and, "the gambler receives 1-to-1 odds on a winning bet (b=1)." These statements are correct, as they are verified by Rotando and Thorp, 1992, page 925, "Further, suppose that on each trial the win probability is p > 0 and pb - q > 0 so the game is advantageous to player A." The "net odds" defintion of b would require that 1-to-1 odds is when b=0, not 1. The variable b is better expressed as "gross odds," not "net odds," and b in the given example would be 1.4, not 0.4. I edited the article appropriately. If any mathematical statements in the article were based on the error, then they will need to be likewise edited.

Look how much else needed to be adjusted to make it gross odds. That's not covering the "Proof" section yet, which would need to change as well. It's awkward. The net odds for 1-to-1 bet is b=1, which means breakeven is p=q, i.e. both are 1/2. That's exactly what we expect. Now with your proposed change we have (b-1) all over the place. --mfb (talk) 02:31, 28 June 2021 (UTC)

I’m planning on switching the (b-1)s back to b. Like just about every other article, including Kelly’s paper. I’m also thinking the article is a bit proof heavy… seems like the various authors of the article are expanding it to justify their points, rather than make it readable… thoughts? --Zojj tc 02:20, 10 August 2021 (UTC)

I went ahead and switched it, reorganized a bit, but left all the proofs. Should be more accessible now to the average reader. --Zojj tc 03:56, 13 August 2021 (UTC)

Growth rate / error in study paper

So I had initially added a growth rate formula to the article. It came from the 60/40 coin flip study that listed a 4% expected growth rate, and a $3 million expected value. I assume this came from the simple formula $r=f^{*}(pb-qa)=0.2\times (0.6\times 1-0.4\times 1)=0.04$. But notice that the growth rate according to this formula increases with a higher bet fraction, which conflicts with the Kelly criterion. So something isn't right... thoughts? --Zojj tc 18:13, 14 August 2021 (UTC)

I show the expected growth rate in the study should be 2%, and a $10,500 expected value; far less than the 3 million.

$r=(1+fb)^{p}\cdot (1-fa)^{q}=1.12^{0.6}\times 0.92^{0.4}=1.0203$

I don't have a source for the formula, but it's pretty straightforward... --Zojj tc 23:12, 14 August 2021 (UTC)

Turns out that formula reduces to the logarithmic wealth function as listed in the proof; I think we're good. I added a note about the error in the study. --Zojj tc 03:17, 15 August 2021 (UTC)

The paper is correct that the average expected winnings should be $3,220,637 if the payout is left uncapped. The paper has the math listed in their footnotes. You are showing the expected utility which is the median result while expected value is the mean result. This is discussed in footnotes [13] and [14] of the paper. This is also easy to verify with a quick Monte Carlo sim. I can share mine with you if you'd like.

So two things in the "Optimal betting example" need to be corrected:

1) the average gain should be 4% not 2% (.6*.2-.4*.2)

2) the average expected winnings for uncapped prizes after 300 rounds should be $3,220,637 -- PayPigman (talk) 00:04, 25 October 2021 (UTC)

Yes you're correct, the paper has it right. Thanks for pointing out the footnote. --Zojj tc 08:13, 8 January 2022 (UTC)

Which formula is the "Kelly criterion"?

The article is titled "Kelly criterion", and says that the Kelly criterion is "a formula". However, the article gives a bunch of different formulas and never actually says which one is the "Kelly criterion". Is the Kelly criterion one of the formulas in the article, or something else?

In Kelly's paper the thing he refers to as a "criterion" is the idea that at every bet the gambler should maximise the expected value of the logarithm of their capital. I think this is probably what is meant by "Kelly criterion" -- which means that the Kelly criterion is not in fact a formula at all. If that's right then the article should be corrected. Otherwise, it should be explicitly stated which formula is the one called "Kelly criterion."

Nathaniel Virgo (talk) 01:37, 27 February 2023 (UTC)

@Nathanielvirgo

The exact parameterization depends on the problem at hand. It would be more accurate to say that the Kelly Criterion produces a unique formula for a betting mechanism. 131.220.249.229 (talk) 15:02, 26 April 2024 (UTC)

logarithms and values

This was anonymously added a few days ago:

It assumes that the expected returns are known and is optimal for a bettor who values their wealth logarithmically.

with the edit summary

The Kelly Criterion is only optimal given a utility function logarithmic in wealth

If your utility function is monotone, its precise shape doesn't matter much: more is better. It seems to me that Kelly optimization assumes that your wager is not the only wager you'll ever make; you want to grow your pot over the long run, to have more to bet with next time, and this means multiplying your wealth by an optimum factor rather than adding to it.

If you optimize linearly, won't you bet your shirt every time the odds are in your favor? —Tamfang (talk) 04:46, 29 March 2023 (UTC)

If you want to optimize the expected linear wealth then you should bet everything every time, indeed. This is clearly not how people behave in practice. There are more options than just linear and logarithmic, however. Different options lead to different strategies. --mfb (talk) 08:45, 29 March 2023 (UTC)

half Kelly

William Poundstone's book Fortune's Formula says somewhere that sophisticated gamblers often bet half the Kelly amount, because of uncertainties. Perhaps this could be mentioned somewhere. (I'll dig out the book one of these days.) —Tamfang (talk) 06:58, 13 April 2023 (UTC)

Agreed, the article currently only mentions this very briefly, but it deserves more thorough and prominent discussion. The Kelly bet will maximise mean growth rate, but it is highly volatile, it does not maximise median growth rate. One person will do very well, but most bettors will lose money. The winner could compensate the losers, but so what? The same logic applies to betting your entire bankroll -- this maximises expected return but almost everyone will go bust. Again the winner could compensate the losers, but so what? There is something seriously irrational about betting more than the Kelly bet. Even betting the full Kelly bet is only for people who are truly risk-neutral, but such people are very rare and there is something irrational about them.

A risk averse person will bet some fraction of the Kelly bet. Because of the shape of the curve, betting half the Kelly bet gets you a substantial decrease in risk but only a small decrease in return. In addition, you may have estimated the odds wrong, and a fractional Kelly bet based on estimated odds helps reduce the chance you exceed the Kelly bet for the true (unknown) odds. cagliost (talk) 14:07, 5 November 2023 (UTC)

only for people who are truly risk averse – did you mean risk-neutral?

I wonder what does maximize median growth rate. —Tamfang (talk) 05:29, 7 November 2023 (UTC)

Thanks, I did mean that. I have updated my comment.

I intend to update the article in this regard using "The Missing Billionaires" as a source (the book is basically about this topic).

I think the fraction of the Kelly bet which maximises median growth rate depends on the edge; I will check. cagliost (talk) 11:15, 7 November 2023 (UTC)

I would intuitively expect it to so depend, yes. —Tamfang (talk) 16:34, 7 November 2023 (UTC)

I got it wrong, the full Kelly bet maximises median return. cagliost (talk) 18:49, 7 November 2023 (UTC)

Holy anticlimax, Batman! —Tamfang (talk) 03:55, 12 November 2023 (UTC)

Now I wonder whether there is a standard mathematical model of risk aversion – ah, apparently there is. —Tamfang (talk) 04:01, 12 November 2023 (UTC)

a and b descriptions are wrong, or example values are

re: a is the fraction that is lost in a negative outcome. If the security price falls 10%, then

b is the fraction that is gained in a positive outcome. If the security price rises 10%, then .

We're saying "negative outcome" and "positive outcome" but the example is an odd ratio. Which is correct? Dforootan (talk) 04:12, 26 January 2024 (UTC)

I believe the % should be used e.g. 0.10, see ref :

"One claim was that one can only lose the amount bet so there was no reason to consider the (simple) generalization of this formula to the situation where a unit wager wins b with probability p > 0 and loses a with probability q."

Dforootan (talk) 07:25, 26 January 2024 (UTC)

Maximising log wealth and maximising CAGR are *not* equivalent

The second sentence claims that maximising expected log wealth and maximising the expected CAGR are equivalent. But this is false. Most obviously, if running the bet for only one round, CAGR is maximised by better 100% of wealth. What is true is only that in the limit of infinitely many repeated bets, nearly all the probability is in the typical set where the fraction of Heads is very nearly np, and that maximising CAGR for that set is equivalent to maximising log wealth. Echidna44 (talk) 09:16, 16 March 2024 (UTC)

The real problem with using Kelly Criterion in the Real World

The real problem with Kelly Criterion in the Real World is that it expects that you know the TRUE probability of winning. But you do not. You can only estimate the probability of winning based on the data you have. So your estimate of the TRUE probability of winning has a degree of uncertainty. See the wikipedia article of estimating the probability of a coin toss.

Another problem is that the TRUE probability of winning could be changing over time in the real world as the real world environment changes. Ohanian (talk) 06:02, 18 May 2024 (UTC)

I think the negative edge part is wrong

"The Kelly bet is -1/19 , meaning the gambler should bet one-nineteenth of their bankroll that red will notcome up. " when the edge is negative the Kelly fraction can be lower than -1 (consider limit b goes to 0). So I disagree with this interpretation and recommend removal. Andy Jiang (talk) 07:50, 4 October 2024 (UTC)

I agree that the interpretation isn't clear. The Kelly bet in this case is negative, meaning you should get someone else to make the bet and you cover the action, so your amount at risk is b times the other side of the bet. It could also be explained that if you're betting on the other side, P' = 1-P, and b' = 1/b. Then f' = P' - (1-P')/b' = 1 - P + P/(1/b) = 1-P + bP = b ((1-P) /b + P) = b * f. But without a source this is all OR. I recommend removal of the example and someone finding a good reference for this entire section. AndyBloch (talk) 00:16, 10 October 2024 (UTC)