Convergence proof techniques

Convergence proof techniques are canonical patterns of mathematical proofs that sequences or functions converge to a finite limit when the argument tends to infinity.

There are many types of sequences and modes of convergence, and different proof techniques may be more appropriate than others for proving each type of convergence of each type of sequence. Below are some of the more common and typical examples. This article is intended as an introduction aimed to help practitioners explore appropriate techniques. The links below give details of necessary conditions and generalizations to more abstract settings. Proof techniques for the convergence of series, a particular type of sequences corresponding to sums of many terms, are covered in the article on convergence tests.

Convergence in Rⁿ

It is common to want to prove convergence of a sequence $f:\mathbb {N} \rightarrow \mathbb {R} ^{n}$ or function $f:\mathbb {R} \rightarrow \mathbb {R} ^{n}$ , where $\mathbb {N}$ and $\mathbb {R}$ refer to the natural numbers and the real numbers, respectively, and convergence is with respect to the Euclidean norm, $||\cdot ||_{2}$ .

Useful approaches for this are as follows.

First principles

The analytic definition of convergence of $f$ to a limit $f_{\infty }$ is that^[1] for all $\epsilon$ there exists a $k_{0}\in \mathbb {N}$ such for all $k>k_{0}$ , $\|f(k)-f_{\infty }\|<\epsilon$ . The most direct proof technique from this definition is to find such a $k_{0}$ and prove the required inequality. If the value of $f_{\infty }$ is not known in advance, the techniques below may be useful.

Contraction mappings

In many cases, the function whose convergence is of interest has the form $f(k+1)=T(f(k))$ for some transformation $T$ . For example, $T$ could map $f(k)$ to $f(k+1)=Af(k)$ for some conformable matrix $A$ , so that $f(k)=A^{k}f(0)$ , a matrix generalization of the geometric progression. Alternatively, $T$ may be an elementwise operation, such as replacing each element of $f(k)$ by the square root of its magnitude.

In such cases, if the problem satisfies the conditions of Banach fixed-point theorem (the domain is a non-empty complete metric space) then it is sufficient to prove convergence to prove that $T$ is a contraction mapping to prove that it has a fixed point. This requires that $\|T(x)-T(y)\|<\|\lambda (x-y)\|$ for some constant $|\lambda |<1$ which is fixed for all $x$ and $y$ . The composition of two contraction mappings is a contraction mapping, so if $T=T_{1}\circ T_{2}$ , then it is sufficient to show that $T_{1}$ and $T_{2}$ are both contraction mappings.

Example

Famous examples of applications of this approach include

If $T$ has the form $T(x)=Ax+B$ for some matrices $A$ and $B$ , then $T^{k}(x)$ converges to $(I-A)^{-1}B$ if the magnitudes of all eigenvalues of $A$ are less than 1^{[citation needed]}.

Non-expansion mappings

If both above inequalities in the definition of a contraction mapping are weakened from "strictly less than" to "less than or equal to", the mapping is a non-expansion mapping. It is not sufficient to prove convergence to prove that $T$ is a non-expansion mapping. For example, $T(x)=-x$ is a non-expansion mapping, but the sequence $T^{n}(x)$ does not converge for any $x\neq 0$ . However, the composition of a contraction mapping and a non-expansion mapping (or vice versa) is a contraction mapping.

Contraction mappings on limited domains

If $T$ is not a contraction mapping on its entire domain, but it is on its codomain (the image of the domain), that is also sufficient for convergence. This also applies for decompositions. For example, consider $T(x)=\cos(\sin(x))$ . The function $\cos$ is not a contraction mapping, but it is on the restricted domain $[-1,1]$ , which is the codomain of $\sin$ for real arguments. Since $\sin$ is a non-expansion mapping, this implies $T$ is a contraction mapping.

Convergent subsequences

Every bounded sequence in $\mathbb {R} ^{n}$ has a convergent subsequence, by the Bolzano–Weierstrass theorem. If these subsequences all have the same limit, then the original sequence also converges to that limit. If it can be shown that all of the subsequences of $f$ must have the same limit, such as by showing that there is a unique fixed point of the transformation $T$ and that there are no invariant sets of $T$ that contain no fixed points of $T$ , then the initial sequence must also converge to that limit.

Monotonicity (Lyapunov functions)

Every bounded monotonic sequence in $\mathbb {R} ^{n}$ converges to a limit.

This fact can be used directly and can also be used to prove the convergence of sequences that are not monotonic using techniques and theorems named for Aleksandr Lyapunov. In these cases, one defines a function $V:\mathbb {R} ^{n}\rightarrow \mathbb {R}$ such that $V(f(k))$ is monotonic in $k$ and thus $V(f(k))$ converges. If $V$ satisfies the conditions to be a Lyapunov function then Lyapunov's theorem implies that $f$ is also convergent. Lyapunov's theorem is normally stated for ordinary differential equations, but it can also be applied to sequences of iterates by replacing derivatives with discrete differences.

The basic requirements on $V$ to be a Lyapunov function are that

$V(x)>0$ for all $x\neq 0$ and $V(0)=0$
$V(f(k+1))-V(f(k))<0$ for $f(k)\neq 0$ (discrete case) or ${\dot {V}}(x)<0$ for $x\neq 0$ (continuous case)
$V$ is "radially unbounded", i.e., that ${\textstyle \lim _{k\rightarrow \infty }V(f(k))=\infty }$ for any sequence with ${\textstyle \lim _{k\rightarrow \infty }||f(k)||=\infty }$ .

In many cases a quadratic Lyapunov function of the form $V(x)=x^{T}Ax$ can be found, although more complex forms are also common, for instance entropies in the study of convergence of probability distributions.

For delay differential equations, a similar approach applies with Lyapunov functions replaced by Lyapunov functionals also called Lyapunov-Krasovskii functionals.

If the inequality in the condition 2 is weak, LaSalle's invariance principle may be used.

Convergence of sequences of functions

To consider the convergence of sequences of functions,^[2] it is necessary to define a distance between functions to replace the Euclidean norm. These often include

Convergence in the norm (strong convergence) -- a function norm, such as ${\textstyle \|g\|_{f}=\int _{x\in A}\|g(x)\|dx}$ is defined, and convergence occurs if $||f(n)-f_{\infty }||_{f}\rightarrow 0$ . For this case, all of the above techniques can be applied with this function norm.
Pointwise convergence -- convergence occurs if for each $x$ , $f_{n}(x)\rightarrow f_{\infty }(x)$ . For this case, the above techniques can be applied for each point $x$ with the norm appropriate for $f(x)$ .
uniform convergence -- In pointwise convergence, some (open) regions can converge arbitrarily slowly. With uniform convergence, there is a fixed convergence rate such that all points converge at least that fast. Formally, $\lim _{n\to \infty }\,\sup\{\,\left|f_{n}(x)-f_{\infty }(x)\right|:x\in A\,\}=0,$ where $A$ is the domain of each $f_{n}$ .

Convergence of random variables

Random variables^[3] are more complicated than simple elements of $\mathbb {R} ^{n}$ . (Formally, a random variable is a mapping $x:\Omega \rightarrow V$ from an event space $\Omega$ to a value space $V$ . The value space may be $\mathbb {R} ^{n}$ , such as the roll of a dice, and such a random variable is often spoken of informally as being in $\mathbb {R} ^{n}$ , but convergence of sequence of random variables corresponds to convergence of the sequence of functions, or the distributions, rather than the sequence of values.)

There are multiple types of convergence, depending on how the distance between functions is measured.

Convergence in distribution -- pointwise convergence of the distribution functions of the random variables to the limit
Convergence in probability
Almost sure convergence -- pointwise convergence of the mappings $x_{n}:\Omega \rightarrow V$ to the limit, except at a set in $\Omega$ with measure 0 in the limit.
Convergence in the mean

Each has its own proof techniques, which are beyond the current scope of this article.

Topological convergence

For all of the above techniques, some form the basic analytic definition of convergence above applies. However, topology has its own definitions of convergence. For example, in a non-Hausdorff space, it is possible for a sequence to converge to multiple different limits.

References

^ Ross, Kenneth. Elementary Analysis: The Theory of Calculus. Springer.
^ Haase, Markus. Functional Analysis: An Elementary Introduction. American Mathematics Society.
^ Billingsley, Patrick (1995). Probability and Measure. John Wesley.

[1] Ross, Kenneth. Elementary Analysis: The Theory of Calculus. Springer.

[2] Haase, Markus. Functional Analysis: An Elementary Introduction. American Mathematics Society.

[3] Billingsley, Patrick (1995). Probability and Measure. John Wesley.

[1]

[2]

[3]