In mathematics, for a given complex Hermitian matrix
and nonzero vector
, the Rayleigh quotient (also known as the Rayleigh–Ritz ratio),
, is defined as [1]:
![{\displaystyle R(M,x):={x^{*}Mx \over x^{*}x}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f49fede4ae2a2f37755f251c408594638b463529)
For real matrices and vectors, the condition of being Hermitian reduces to that of being symmetric, and the conjugate transpose
to the usual transpose
. Note that
for any real scalar
. Recall that a Hermitian (or real symmetric) matrix has real eigenvalues. It can be shown that, for a given matrix, the Rayleigh quotient reaches its minimum value
(the smallest eigenvalue of
) when
is
(the corresponding eigenvector). Similarly,
and
. The Rayleigh quotient is used in min-max theorem to get exact values of all eigenvalues. It is also used in eigenvalue algorithms to obtain an eigenvalue approximation from an eigenvector approximation. Specifically, this is the basis for Rayleigh quotient iteration.
The range of the Rayleigh quotient is called a numerical range.
Special case of covariance matrices
[editar]
A covariance matrix M can be represented as the product
. Its eigenvalues are positive:
![{\displaystyle Mv_{i}==A'Av_{i}=\lambda _{i}v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a6e0241b96cc550acaffe06a70b1477f876d83f2)
![{\displaystyle v_{i}'A'Av_{i}=v_{i}'\lambda _{i}v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ca92c9941e043db1a2f4fb9e47b2fb428091ce97)
![{\displaystyle \left\|Av_{i}\right\|^{2}=\lambda _{i}\left\|v_{i}\right\|^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/a6d20e9d7be35e7a8d31c65ce54d4fce04e1ef03)
![{\displaystyle \lambda _{i}={\frac {\left\|Av_{i}\right\|^{2}}{\left\|v_{i}\right\|^{2}}}\geq 0.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/dff09e4783fa18c3067547cd8d2623108dbfbda4)
The eigenvectors are orthogonal to one another:
![{\displaystyle Mv_{i}=\lambda _{i}v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/5c50764b54f59cacc713336abda323033ae0034d)
![{\displaystyle v_{j}'Mv_{i}=\lambda _{i}v_{j}'v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/83f6ce720322e3ba48ccfb69adda8196d92aeb6d)
![{\displaystyle (Mv_{j})'v_{i}=\lambda _{i}v_{j}'v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/606c6fecc5d9ea501e97204ea3808820ad3f6320)
![{\displaystyle \lambda _{j}v_{j}'v_{i}=\lambda _{i}v_{j}'v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/92690662b310482ae3f8f74fc25c54c74f58fa84)
![{\displaystyle (\lambda _{j}-\lambda _{i})v_{j}'v_{i}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/2a1ec1970e7c03aea5ad5786128019d372cd9961)
(different eigenvalues, in case of multiplicity, the basis can be orthogonalized).
The Rayleigh quotient can be expressed as a function of the eigenvalues by decomposing any vector
on the basis of eigenvectors:
, where
is the coordinate of x orthogonally projected onto ![{\displaystyle v_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7dffe5726650f6daac54829972a94f38eb8ec127)
![{\displaystyle R(M,x)={\frac {x'A'Ax}{x'x}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/743fb967ae0693b807a3584e452f0e0ff667b25d)
![{\displaystyle R(M,x)={\frac {(\sum _{j=1}^{n}\alpha _{j}v_{j})'A'A(\sum _{i=1}^{n}\alpha _{i}v_{i})}{(\sum _{j=1}^{n}\alpha _{j}v_{j})'(\sum _{i=1}^{n}\alpha _{i}v_{i})}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d66a93e9fcea2076b9d4f3a5e7eb06ee06ea29f9)
which, by orthogonality of the eigenvectors, becomes:
![{\displaystyle R(M,x)={\frac {\sum _{i=1}^{n}\alpha _{i}^{2}\lambda _{i}}{\sum _{i=1}^{n}\alpha _{i}^{2}}}=\sum _{i=1}^{n}\lambda _{i}{\frac {(x'v_{i})^{2}}{(x'x)(v_{i}'v_{i})}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0cbdf58297c400547be0d08e305014a7985392ae)
In the last representation we can see that the Rayleigh quotient is the sum of the squared cosines of the angles formed by the vector x and each eigenvector
, weighted by corresponding eigenvalues.
If a vector
maximizes
, then any vector
(for
) also maximizes it, one can reduce to the Lagrange problem of maximizing
under the constraint that
.
Since all the eigenvalues are non-negative, the problem is convex and the maximum occurs on the edges of the domain, namely when
and
(when the eigenvalues are ordered in decreasing magnitude).
Alternatively, this result can be arrived at by the method of Lagrange multipliers. The problem is to find the critical points of the function
,
subject to the constraint
I.e. to find the critical points of
![{\displaystyle {\mathcal {L}}(x)=x^{T}Mx-\lambda (x^{T}x-1),}](https://wikimedia.org/api/rest_v1/media/math/render/svg/154f1959f31e8db1baf2327a18db40089be40009)
where
is a Lagrange multiplier. The stationary points of
occur at
![{\displaystyle {\frac {d{\mathcal {L}}(x)}{dx}}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c98dce5b82321942deee71c83252d2969e4e975a)
![{\displaystyle \therefore 2x^{T}<-2\lambda x^{T}=0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e49dff3bb9fefe431ec1a610256d689e81fcf948)
![{\displaystyle \therefore Mx=\lambda x}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d784a134003dc151f6c9be50232128e2fa5cc338)
and
Therefore, the eigenvectors
of M are the critical points of the Raleigh Quotient and their corresponding eigenvalues
are the stationary values of R.
This property is the basis for principal components analysis and canonical correlation.
Use in Sturm–Liouville theory
[editar]
Sturm–Liouville theory concerns the action of the linear operator
![{\displaystyle L(y)={\frac {1}{w(x)}}\left(-{\frac {d}{dx}}\left[p(x){\frac {dy}{dx}}\right]+q(x)y\right)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/371147c6e1ef449ffd33e67888014e4866665d32)
on the inner product space defined by
![{\displaystyle \langle {y_{1},y_{2}}\rangle =\int _{a}^{b}w(x)y_{1}(x)y_{2}(x)\,dx}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f468853e4dcc2902529e7bb5eca8cd836212ccaa)
of functions satisfying some specified boundary conditions at a and b. In this case the Rayleigh quotient is
![{\displaystyle {\frac {\langle {y,Ly}\rangle }{\langle {y,y}\rangle }}={\frac {\int _{a}^{b}{y(x)\left(-{\frac {d}{dx}}\left[p(x){\frac {dy}{dx}}\right]+q(x)y(x)\right)}dx}{\int _{a}^{b}{w(x)y(x)^{2}}dx}}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/e5f2d8b104320e364d15f3863e08434f92ba1015)
This is sometimes presented in an equivalent form, obtained by separating the integral in the numerator and using integration by parts:
![{\displaystyle {\frac {\langle {y,Ly}\rangle }{\langle {y,y}\rangle }}={\frac {\int _{a}^{b}{y(x)\left(-{\frac {d}{dx}}\left[p(x)y'(x)\right]\right)}dx+\int _{a}^{b}{q(x)y(x)^{2}}\,dx}{\int _{a}^{b}{w(x)y(x)^{2}}\,dx}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/ed0d8d34fdd179dff34c6376ee2e90d81524e627)
![{\displaystyle ={\frac {-y(x)\left[p(x)y'(x)\right]|_{a}^{b}+\int _{a}^{b}{y'(x)\left[p(x)y'(x)\right]}\,dx+\int _{a}^{b}{q(x)y(x)^{2}}\,dx}{\int _{a}^{b}{w(x)y(x)^{2}}\,dx}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/1f72c17873203ef837577d8d1ce10af282434eeb)
![{\displaystyle ={\frac {-p(x)y(x)y'(x)|_{a}^{b}+\int _{a}^{b}\left[p(x)y'(x)^{2}+q(x)y(x)^{2}\right]\,dx}{\int _{a}^{b}{w(x)y(x)^{2}}\,dx}}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/580e27b2083d13d2627b3b575cc0db2aa0f1054f)
For a given pair
of real symmetric positive-definite matrices, and a given non-zero vector
, the generalized Rayleigh quotient is defined as:
![{\displaystyle R(A,B;x):={\frac {x^{T}Ax}{x^{T}Bx}}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/b906c0dfe3d5eb8b30e87dfd81ef856adbf5ba4a)
The Generalized Rayleigh Quotient can be reduced to the Rayleigh Quotient
through the transformation
where
is the Cholesky decomposition of matrix
.
- ↑ Horn, R. A. and C. A. Johnson. 1985. Matrix Analysis. Cambridge University Press. pp. 176–180.