Académique Documents
Professionnel Documents
Culture Documents
In linear algebra, a symmetric real matrix is said to be positive definite if the scalar is strictly positive for
every non-zero column vector of real numbers. Here denotes the transpose of .[1] When interpreting as the output of
an operator, , that is acting on an input, , the property of positive definiteness implies that the output always has a positive
inner product with the input, as often observed in physical processes.
More generally, a complex Hermitian matrix is said to be positive definite if the scalar is strictly positive for
every non-zero column vector of complex numbers. Here denotes the conjugate transpose of . Note that is
automatically real since is Hermitian.
Positive semi-definite matrices are defined similarly, except that the above scalars or must be positive or zero (i.e.
non-negative). Negative definite and negative semi-definite matrices are defined analogously. A matrix that is not positive semi-
definite and not negative semi-definite is called indefinite.
The matrix is positive definite if and only if the bilinear form is positive definite (and similarly for a
positive definite sesquilinear form in the complex case). This is a coordinate realization of an inner product on a vector space.[2]
Some authors use more general definitions of definiteness, including some non-symmetric real matrices, or non-Hermitian
complex ones.
Contents
Definitions
Definitions for real matrices
Definitions for complex matrices
Consistency between real and complex definitions
Notation
Examples
Eigenvalues
Connections
Characterizations
Quadratic forms, convexity, optimization
Simultaneous diagonalization
Properties
Induced partial ordering
Inverse of positive definite matrix
Scaling
Addition
Multiplication
Cholesky decomposition
Square root
Submatrices
Trace
Hadamard product
Kronecker product
Frobenius product
Convexity
Further properties
Block matrices
Extension for non-symmetric square matrices
Applications
Heat conductivity matrix
See also
Notes
References
External links
Definitions
In the following definitions, is the transpose of , is the conjugate transpose of and denotes the n-dimensional zero-
vector.
A symmetric real matrix is said to be positive semidefinite or non-negative definite if for all non-zero
in . Formally,
A symmetric real matrix is said to be negative definite if for all non-zero in . Formally,
A symmetric real matrix is said to be negative semidefinite or non-positive definite if for all non-zero
in . Formally,
A symmetric real matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.
A Hermitian complex matrix is said to be positive definite if for all non-zero in . Formally,
A Hermitian complex matrix is said to be positive semi-definite or non-negative definite if for all non-
zero in . Formally,
A Hermitian complex matrix is said to be negative definite if for all non-zero in . Formally,
A Hermitian complex matrix is said to be negative semi-definite or non-positive definite if for all non-
zero in . Formally,
A Hermitian complex matrix which is neither positive semidefinite nor negative semidefinite is called indefinite.
For complex matrices, the most common definition says that " is positive definite if and only if is real and positive for
all non-zero complex column vectors ". This condition implies that is Hermitian (i.e. its transpose is equal to its conjugate).
To see this, consider the matrices and , so that and
. The matrices and are Hermitian, therefore and are individually real. If is
real, then must be zero for all . Then is the zero matrix and , proving that is Hermitian.
By this definition, a positive definite real matrix is Hermitian, hence symmetric; and is positive for all non-zero real
column vectors . However the last condition alone is not sufficient for to be positive definite. For example, if
then for any real vector with entries and we have , which is always positive if
is not zero. However, if is the complex vector with entries and , one gets
On the other hand, for a symmetric real matrix , the condition " for all nonzero real vectors " does imply that is
positive definite in the complex sense.
Notation
If a Hermitian matrix is positive semi-definite, one sometimes writes and if is positive definite one writes .
To denote that is negative semi-definite one writes and to denote that is negative definite one writes .
The notion comes from functional analysis where positive semidefinite matrices define positive operators.
A common alternative notation is , , and for positive semi-definite and positive definite, negative
semi-definite and negative definite matrices, respectively. This may be confusing, as sometimes nonnegative matrices
respectively nonpositive matrices are also denoted in this way.
Examples
The identity matrix is positive definite (and as such also positive semi-definite). It is a real
symmetric matrix, and, for any non-zero column vector z with real entries a and b, one has
Seen as a complex matrix, for any non-zero column vector z with complex entries a and b one has
Either way, the result is positive since is not the zero vector (that is, at least one of and is not zero).
The real symmetric matrix
is positive definite since for any non-zero column vector z with entries a, b and c, we have
This result is a sum of squares, and therefore non-negative; and is zero only if , that is, when z is
the zero vector.
For any real invertible matrix , the product is a positive definite matrix. A simple proof is that for any non-
zero vector , the condition since the invertibility of matrix means that
The example above shows that a matrix in which some elements are negative may still be positive definite.
Conversely, a matrix whose entries are all positive is not necessarily positive definite, as for example
for which
Eigenvalues
Let be an Hermitian matrix.
Connections
A general purely quadratic real function on real variables can always be written as where is the
column vector with those variables, and is a symmetric real matrix. Therefore, the matrix being positive definite means that
has a unique minimum (zero) when is zero, and is strictly positive for any other .
More generally, a twice-differentiable real function on real variables has local minimum at arguments if its
gradient is zero and its Hessian (the matrix of all second derivatives) is positive semi-definite at that point. Similar statements can
be made for negative definite and semi-definite matrices.
In statistics, the covariance matrix of a multivariate probability distribution is always positive semi-definite; and it is positive
definite unless one variable is an exact linear function of the others. Conversely, every positive semi-definite matrix is the
covariance matrix of some multivariate distribution.
Characterizations
Let be an Hermitian matrix. The following properties are equivalent to being positive definite:
A symmetric matrix is positive definite if and only if its quadratic form is a strictly convex function.
More generally, any quadratic function from to can be written as where is a symmetric
matrix, is a real -vector, and a real constant. This quadratic function is strictly convex, and hence has a unique finite global
minimum, if and only if is positive definite. For this reason, positive definite matrices play an important role in optimization
problems.
Simultaneous diagonalization
A symmetric matrix and another symmetric and positive definite matrix can be simultaneously diagonalized, although not
necessarily via a similarity transformation. This result does not extend to the case of three or more matrices. In this section we
write for the real case. Extension to the complex case is immediate.
Let be a symmetric and a symmetric and positive definite matrix. Write the generalized eigenvalue equation as
where we impose that be normalized, i.e. . Now we use Cholesky decomposition to write the
inverse of as . Multiplying by and letting , we get , which can be rewritten as
where . Manipulation now yields where is a matrix having as columns the
generalized eigenvectors and is a diagonal matrix of the generalized eigenvalues. Now premultiplication with gives the
final result: and , but note that this is no longer an orthogonal diagonalization with respect to the
inner product where . In fact, we diagonalized with respect to the inner product induced by .
Note that this result does not contradict what is said on simultaneous diagonalization in the article Diagonalizable matrix, which
refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous
diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other.[3]
Properties
Scaling
If is positive definite and is a real number, then is positive definite.[6]
Addition
If and are positive definite, then the sum is also positive definite.[6]
Multiplication
If and are positive definite, then the products and are also positive definite. If ,
then is also positive definite.
If is positive semidefinite, then is positive semidefinite. If is positive definite and has full rank,
then is positive definite.[7]
Cholesky decomposition
For any matrix , the matrix is positive semidefinite, and . Conversely, any Hermitian positive
semi-definite matrix can be written as , where is lower triangular; this is the Cholesky decomposition. If is
not positive definite, then some of the diagonal elements of may be zero.
A hermitian matrix is positive definite if and only if it has a unique Cholesky decomposition, i.e. the matrix is positive
definite if and only if there exists a unique lower triangular matrix , with real and strictly positive diagonal elements, such that
.
Square root
A matrix is positive semi-definite if and only if there is a positive semi-definite matrix with . This matrix is
unique,[8] is called the square root of , and is denoted with (the square root is not to be confused with the matrix
in the Cholesky factorization , which is also sometimes called the square root of ).
If then .
Submatrices
Every principal submatrix of a positive definite matrix is positive definite.
Trace
The diagonal entries of a positive definite matrix are real and non-negative. As a consequence the trace, .
Furthermore,[9] since every principal sub-matrix (in particular, 2-by-2) is positive definite,
and thus
Hadamard product
If , although is not necessary positive semidefinite, the Hadamard product (this result is often
called the Schur product theorem).[10]
Regarding the Hadamard product of two positive semidefinite matrices , , there are two notable
inequalities:
.[12]
Kronecker product
If , although is not necessary positive semidefinite, the Kronecker product .
Frobenius product
If , although is not necessary positive semidefinite, the Frobenius product (Lancaster–Tismenetsky,
The Theory of Matrices, p. 218).
Convexity
The set of positive semidefinite symmetric matrices is convex. That is, if and are positive semidefinite, then for any
between 0 and 1, is also positive semidefinite. For any vector :
This property guarantees that semidefinite programming problems converge to a globally optimal solution.
Further properties
1. If is a symmetric matrix of the form , and the strict inequality holds
where each block is . By applying the positivity condition, it immediately follows that and are hermitian, and
.
A similar argument can be applied to , and thus we conclude that both and must be positive definite matrices, as well.
Converse results can be proved with stronger conditions on the blocks, for instance using the Schur complement.
Indeed, with this definition, a real matrix is positive definite if and only if for all nonzero real vectors , even if is
not symmetric.
In general, we have for all complex nonzero vectors if and only if the Hermitian part of is
positive definite in the narrower sense. Similarly, we have for all real nonzero vectors if and only if the symmetric
part of is positive definite in the narrower sense. Accordingly, common tests for positive definiteness in the
narrow sense must be applied only to the symmetric part of a matrix, never the original matrix itself. For example, the matrix
has positive eigenvalues yet is not positive definite; in particular a negative value of is obtained with the
choice (which is the eigenvector associated with the negative eigenvalue of the symmetric part of ).
In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex
Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no
longer true in the real case.
Applications
Notes
1. "Appendix C: Positive Semidefinite and Positive Definite Matrices". Parameter Estimation for Scientists and
Engineers: 259–263. doi:10.1002/9780470173862.app3 (https://doi.org/10.1002%2F9780470173862.app3).
2. Stewart, J. (1976). Positive definite functions and generalizations, an historical survey. Rocky Mountain J. Math,
6(3). (http://projecteuclid.org/DPubS?verb=Display&version=1.0&service=UI&handle=euclid.rmjm/1250130219&p
age=record) Archived (https://web.archive.org/web/20140202092645/http://projecteuclid.org/DPubS?verb=Displa
y&version=1.0&service=UI&handle=euclid.rmjm%2F1250130219&page=record) 2014-02-02 at the Wayback
Machine
3. Horn & Johnson (1985), p. 218 ff.
4. Horn & Johnson (1985), p. 397
5. Horn & Johnson (1985), Corollary 7.7.4(a)
6. Horn & Johnson (1985), Observation 7.1.3
7. Horn, Roger A.; Johnson, Charles R. (2013). "7.1 Definitions and Properties". Matrix Analysis (Second Edition).
Cambridge University Press. p. 431. ISBN 978-0-521-83940-2. "Observation 7.1.8 Let be Hermitian
and let :
References
Horn, Roger A.; Johnson, Charles R. (1990). Matrix Analysis. Cambridge University Press. ISBN 978-0-521-
38632-6.
Bhatia, Rajendra (2007). Positive definite matrices. Princeton Series in Applied Mathematics. ISBN 978-0-691-
12918-1.
Bernstein, B.; Toupin, R.A. (1962). Some Properties of the Hessian Matrix of a Strictly Convex Function, J. fűr die
Reine und Angew. Math.210, 67-72.
External links
Hazewinkel, Michiel, ed. (2001) [1994], "Positive-definite form" (https://www.encyclopediaofmath.org/index.php?ti
tle=p/p073880), Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic
Publishers, ISBN 978-1-55608-010-4
Wolfram MathWorld: Positive Definite Matrix (http://mathworld.wolfram.com/PositiveDefiniteMatrix.html)
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using
this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia
Foundation, Inc., a non-profit organization.