In probability and statistics , given two stochastic processes
{
X
t
}
{\displaystyle \left\{X_{t}\right\}}
{
Y
t
}
{\displaystyle \left\{Y_{t}\right\}}
cross-covariance is a function that gives the covariance of one process with the other at pairs of time points. With the usual notation
E
{\displaystyle \operatorname {E} }
expectation operator , if the processes have the mean functions
μ
X
(
t
)
=
E
[
X
t
]
{\displaystyle \mu _{X}(t)=\operatorname {\operatorname {E} } [X_{t}]}
μ
Y
(
t
)
=
E
[
Y
t
]
{\displaystyle \mu _{Y}(t)=\operatorname {E} [Y_{t}]}
K
X
Y
(
t
1
,
t
2
)
=
cov
(
X
t
1
,
Y
t
2
)
=
E
[
(
X
t
1
−
μ
X
(
t
1
)
)
(
Y
t
2
−
μ
Y
(
t
2
)
)
]
=
E
[
X
t
1
Y
t
2
]
−
μ
X
(
t
1
)
μ
Y
(
t
2
)
.
{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2})=\operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} [(X_{t_{1}}-\mu _{X}(t_{1}))(Y_{t_{2}}-\mu _{Y}(t_{2}))]=\operatorname {E} [X_{t_{1}}Y_{t_{2}}]-\mu _{X}(t_{1})\mu _{Y}(t_{2}).\,}
Cross-covariance is related to the more commonly used cross-correlation of the processes in question.
In the case of two random vectors
X
=
(
X
1
,
X
2
,
…
,
X
p
)
T
{\displaystyle \mathbf {X} =(X_{1},X_{2},\ldots ,X_{p})^{\rm {T}}}
Y
=
(
Y
1
,
Y
2
,
…
,
Y
q
)
T
{\displaystyle \mathbf {Y} =(Y_{1},Y_{2},\ldots ,Y_{q})^{\rm {T}}}
p
×
q
{\displaystyle p\times q}
K
X
Y
{\displaystyle \operatorname {K} _{XY}}
cov
(
X
,
Y
)
{\displaystyle \operatorname {cov} (X,Y)}
K
X
Y
(
j
,
k
)
=
cov
(
X
j
,
Y
k
)
.
{\displaystyle \operatorname {K} _{XY}(j,k)=\operatorname {cov} (X_{j},Y_{k}).\,}
cross-covariance is used in order to distinguish this concept from the covariance of a random vector
X
{\displaystyle \mathbf {X} }
matrix of covariances between the scalar components of
X
{\displaystyle \mathbf {X} }
In signal processing , the cross-covariance is often called cross-correlation and is a measure of similarity of two signals , commonly used to find features in an unknown signal by comparing it to a known one. It is a function of the relative time between the signals, is sometimes called the sliding dot product , and has applications in pattern recognition and cryptanalysis .
Cross-covariance of random vectors
edit
Cross-covariance of stochastic processes
edit
The definition of cross-covariance of random vectors may be generalized to stochastic processes as follows:
Let
{
X
(
t
)
}
{\displaystyle \{X(t)\}}
{
Y
(
t
)
}
{\displaystyle \{Y(t)\}}
K
X
Y
{\displaystyle K_{XY}}
[ 1] : p.172
K
X
Y
(
t
1
,
t
2
)
=
d
e
f
cov
(
X
t
1
,
Y
t
2
)
=
E
[
(
X
(
t
1
)
−
μ
X
(
t
1
)
)
(
Y
(
t
2
)
−
μ
Y
(
t
2
)
)
]
{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2}){\stackrel {\mathrm {def} }{=}}\ \operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} \left[\left(X(t_{1})-\mu _{X}(t_{1})\right)\left(Y(t_{2})-\mu _{Y}(t_{2})\right)\right]}
(Eq.1 )
where
μ
X
(
t
)
=
E
[
X
(
t
)
]
{\displaystyle \mu _{X}(t)=\operatorname {E} \left[X(t)\right]}
μ
Y
(
t
)
=
E
[
Y
(
t
)
]
{\displaystyle \mu _{Y}(t)=\operatorname {E} \left[Y(t)\right]}
If the processes are complex-valued stochastic processes, the second factor needs to be complex conjugated :
K
X
Y
(
t
1
,
t
2
)
=
d
e
f
cov
(
X
t
1
,
Y
t
2
)
=
E
[
(
X
(
t
1
)
−
μ
X
(
t
1
)
)
(
Y
(
t
2
)
−
μ
Y
(
t
2
)
)
¯
]
{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2}){\stackrel {\mathrm {def} }{=}}\ \operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} \left[\left(X(t_{1})-\mu _{X}(t_{1})\right){\overline {\left(Y(t_{2})-\mu _{Y}(t_{2})\right)}}\right]}
Definition for jointly WSS processes
edit
If
{
X
t
}
{\displaystyle \left\{X_{t}\right\}}
{
Y
t
}
{\displaystyle \left\{Y_{t}\right\}}
jointly wide-sense stationary , then the following are true:
μ
X
(
t
1
)
=
μ
X
(
t
2
)
≜
μ
X
{\displaystyle \mu _{X}(t_{1})=\mu _{X}(t_{2})\triangleq \mu _{X}}
t
1
,
t
2
{\displaystyle t_{1},t_{2}}
μ
Y
(
t
1
)
=
μ
Y
(
t
2
)
≜
μ
Y
{\displaystyle \mu _{Y}(t_{1})=\mu _{Y}(t_{2})\triangleq \mu _{Y}}
t
1
,
t
2
{\displaystyle t_{1},t_{2}}
and
K
X
Y
(
t
1
,
t
2
)
=
K
X
Y
(
t
2
−
t
1
,
0
)
{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2})=\operatorname {K} _{XY}(t_{2}-t_{1},0)}
t
1
,
t
2
{\displaystyle t_{1},t_{2}}
By setting
τ
=
t
2
−
t
1
{\displaystyle \tau =t_{2}-t_{1}}
K
X
Y
(
τ
)
=
K
X
Y
(
t
2
−
t
1
)
≜
K
X
Y
(
t
1
,
t
2
)
{\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {K} _{XY}(t_{2}-t_{1})\triangleq \operatorname {K} _{XY}(t_{1},t_{2})}
The cross-covariance function of two jointly WSS processes is therefore given by:
K
X
Y
(
τ
)
=
cov
(
X
t
,
Y
t
−
τ
)
=
E
[
(
X
t
−
μ
X
)
(
Y
t
−
τ
−
μ
Y
)
]
=
E
[
X
t
Y
t
−
τ
]
−
μ
X
μ
Y
{\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {cov} (X_{t},Y_{t-\tau })=\operatorname {E} [(X_{t}-\mu _{X})(Y_{t-\tau }-\mu _{Y})]=\operatorname {E} [X_{t}Y_{t-\tau }]-\mu _{X}\mu _{Y}}
(Eq.2 )
which is equivalent to
K
X
Y
(
τ
)
=
cov
(
X
t
+
τ
,
Y
t
)
=
E
[
(
X
t
+
τ
−
μ
X
)
(
Y
t
−
μ
Y
)
]
=
E
[
X
t
+
τ
Y
t
]
−
μ
X
μ
Y
{\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {cov} (X_{t+\tau },Y_{t})=\operatorname {E} [(X_{t+\tau }-\mu _{X})(Y_{t}-\mu _{Y})]=\operatorname {E} [X_{t+\tau }Y_{t}]-\mu _{X}\mu _{Y}}
Two stochastic processes
{
X
t
}
{\displaystyle \left\{X_{t}\right\}}
{
Y
t
}
{\displaystyle \left\{Y_{t}\right\}}
uncorrelated if their covariance
K
X
Y
(
t
1
,
t
2
)
{\displaystyle \operatorname {K} _{\mathbf {X} \mathbf {Y} }(t_{1},t_{2})}
[ 1] : p.142
{
X
t
}
,
{
Y
t
}
uncorrelated
⟺
K
X
Y
(
t
1
,
t
2
)
=
0
∀
t
1
,
t
2
{\displaystyle \left\{X_{t}\right\},\left\{Y_{t}\right\}{\text{ uncorrelated}}\quad \iff \quad \operatorname {K} _{\mathbf {X} \mathbf {Y} }(t_{1},t_{2})=0\quad \forall t_{1},t_{2}}
Cross-covariance of deterministic signals
edit
The cross-covariance is also relevant in signal processing where the cross-covariance between two wide-sense stationary random processes can be estimated by averaging the product of samples measured from one process and samples measured from the other (and its time shifts). The samples included in the average can be an arbitrary subset of all the samples in the signal (e.g., samples within a finite time window or a sub-sampling of one of the signals). For a large number of samples, the average converges to the true covariance.
Cross-covariance may also refer to a "deterministic" cross-covariance between two signals. This consists of summing over all time indices. For example, for discrete-time signals
f
[
k
]
{\displaystyle f[k]}
g
[
k
]
{\displaystyle g[k]}
(
f
⋆
g
)
[
n
]
=
d
e
f
∑
k
∈
Z
f
[
k
]
¯
g
[
n
+
k
]
=
∑
k
∈
Z
f
[
k
−
n
]
¯
g
[
k
]
{\displaystyle (f\star g)[n]\ {\stackrel {\mathrm {def} }{=}}\ \sum _{k\in \mathbb {Z} }{\overline {f[k]}}g[n+k]=\sum _{k\in \mathbb {Z} }{\overline {f[k-n]}}g[k]}
where the line indicates that the complex conjugate is taken when the signals are complex-valued .
For continuous functions
f
(
x
)
{\displaystyle f(x)}
g
(
x
)
{\displaystyle g(x)}
(
f
⋆
g
)
(
x
)
=
d
e
f
∫
f
(
t
)
¯
g
(
x
+
t
)
d
t
=
∫
f
(
t
−
x
)
¯
g
(
t
)
d
t
{\displaystyle (f\star g)(x)\ {\stackrel {\mathrm {def} }{=}}\ \int {\overline {f(t)}}g(x+t)\,dt=\int {\overline {f(t-x)}}g(t)\,dt}
The (deterministic) cross-covariance of two continuous signals is related to the convolution by
(
f
⋆
g
)
(
t
)
=
(
f
(
−
τ
)
¯
∗
g
(
τ
)
)
(
t
)
{\displaystyle (f\star g)(t)=({\overline {f(-\tau )}}*g(\tau ))(t)}
and the (deterministic) cross-covariance of two discrete-time signals is related to the discrete convolution by
(
f
⋆
g
)
[
n
]
=
(
f
[
−
k
]
¯
∗
g
[
k
]
)
[
n
]
{\displaystyle (f\star g)[n]=({\overline {f[-k]}}*g[k])[n]}
^ a b Kun Il Park, Fundamentals of Probability and Stochastic Processes with Applications to Communications, Springer, 2018, 978-3-319-68074-3
![{\displaystyle \mu _{X}(t)=\operatorname {\operatorname {E} } [X_{t}]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6464417d52bc920c016773f523d4dbceb9b293d4)
![{\displaystyle \mu _{Y}(t)=\operatorname {E} [Y_{t}]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/01da5dd04fc26f6b21e177f10ea9736e2d56a4e7)
![{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2})=\operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} [(X_{t_{1}}-\mu _{X}(t_{1}))(Y_{t_{2}}-\mu _{Y}(t_{2}))]=\operatorname {E} [X_{t_{1}}Y_{t_{2}}]-\mu _{X}(t_{1})\mu _{Y}(t_{2}).\,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/05f4077f8173bfb323a68313e4287d017cc0c281)
![{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2}){\stackrel {\mathrm {def} }{=}}\ \operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} \left[\left(X(t_{1})-\mu _{X}(t_{1})\right)\left(Y(t_{2})-\mu _{Y}(t_{2})\right)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/43a9557f62a2b87ad3e9ea3fe34fd2886a555452)
![{\displaystyle \mu _{X}(t)=\operatorname {E} \left[X(t)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/b66ff50c83aada12cb690a425b435f229b9f72ed)
![{\displaystyle \mu _{Y}(t)=\operatorname {E} \left[Y(t)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e9fbc8f1d47e8c8e0ef3b108fb5ff1f50d622ff3)
![{\displaystyle \operatorname {K} _{XY}(t_{1},t_{2}){\stackrel {\mathrm {def} }{=}}\ \operatorname {cov} (X_{t_{1}},Y_{t_{2}})=\operatorname {E} \left[\left(X(t_{1})-\mu _{X}(t_{1})\right){\overline {\left(Y(t_{2})-\mu _{Y}(t_{2})\right)}}\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4af132571e88ed273022677215e210816f4f9b93)
![{\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {cov} (X_{t},Y_{t-\tau })=\operatorname {E} [(X_{t}-\mu _{X})(Y_{t-\tau }-\mu _{Y})]=\operatorname {E} [X_{t}Y_{t-\tau }]-\mu _{X}\mu _{Y}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e9d33105bfcc12b9bd9a3d240d78fad1df3cf524)
![{\displaystyle \operatorname {K} _{XY}(\tau )=\operatorname {cov} (X_{t+\tau },Y_{t})=\operatorname {E} [(X_{t+\tau }-\mu _{X})(Y_{t}-\mu _{Y})]=\operatorname {E} [X_{t+\tau }Y_{t}]-\mu _{X}\mu _{Y}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c329ddae8b4e41431582575fd8156e025c26f34d)
![{\displaystyle f[k]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c0c2cac4f7a962e0e8b6e35a6ddb99c8b39763fb)
![{\displaystyle g[k]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e5e0eb43b8f34360a3158d48d0d908a104dd8ccf)
![{\displaystyle (f\star g)[n]\ {\stackrel {\mathrm {def} }{=}}\ \sum _{k\in \mathbb {Z} }{\overline {f[k]}}g[n+k]=\sum _{k\in \mathbb {Z} }{\overline {f[k-n]}}g[k]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/02f82f14c2e14226808b1b3517153c75716e658d)
![{\displaystyle (f\star g)[n]=({\overline {f[-k]}}*g[k])[n]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c04fa8cd0152aa79efeee16ac48bbd6170a4b52a)