2026 Prof.Jiang@ECE NYU 253 Lecture VI Extensions to Complex Matrices, in particular

Hermitian Matrices. Key Notions: * Unitary matrices * Unitary equivalence * Schur’s unitary triangularization * QR factorization * Congruence and simultaneous diagonalization 2026 Prof.Jiang@ECE NYU 254 Orthogonality Between Complex

Vectors * 1 1 2 2 Given any pair of ( ) vectors , , the inner product is defined as

,

. They are said to be , if

, o

0 rthogon l . a n n n complex x y x y y x x y x y x y x y          2026 Prof.Jiang@ECE NYU 255 Facts about the Inner Product It can be easily checked that the inner product enjoys

the following properties:

,

,

,

,

, , .

,

,

,

,

scalar. 0,

;

,

0,

if and only if 0. n n x y z x y x z x y z x y x y x x x x                       2026 Prof.Jiang@ECE NYU 256 Orthogonal & Orthonomal Sets of

Vectors

A set of vectors

is said to be , if

,

0,

1 ,

, .

A set of vectors

is said t orthogonal orthono be

if, add orma itionally, : ,

1,

1 . l

i n n i j i i i i x x x i j k i j x x x x i k                 2026 Prof.Jiang@ECE NYU 257 Remark   1 Any orthogonal set of

vectors

can be made an orthonormal set, by defining 1

no

: ,

1 . nzer , o ki i i i i i y x y i k y y      2026 Prof.Jiang@ECE NYU 258 Fundamental Results 1) Any orthogonal set of nonzero vectors

is linearly independent. 2) Any orthonormal set of vectors is

linearly independent. 2026 Prof.Jiang@ECE NYU 259 Unitary Matrix * * A matrix

is said to be

if .

(Recall that ) Of course, a real orthogonal matrix

is unitary, but the converse is not true. C unita an you find some examp r e y l s? n n n n TU U U U U I O         Complex Orthogonal Matrix 2026 Prof.Jiang@ECE NYU 260 complex orthogA matrix

is said to be , if:

ona

l . n n T A A A I    Remark: A complex orthogonal matrix is unitary if and only if it is real. 2026 Prof.Jiang@ECE NYU 261 Equivalent Characterizations * 1 * * The following are equivalent:

is unitary;

is nonsingular and ;

;

is unitary;

The columns of

form an orthonormal set;

The rows of

form an orthonormal set;

For any

U U U U UU I U U U x            * *,

satisfies .n y Ux y y x x  Exercise 2026 Prof.Jiang@ECE NYU 262   1) For any given real parameters ,

1 ,

is always unitary. 2) Any diagonal unitary matrix can always be put into

the above form. 3) Any diagonalizable unitary matrix can be transformed

to

k i j i n U diag e      the above form. Are the following statements true or false? 2026 Prof.Jiang@ECE NYU 263 Question How to apply a unitary matrix, instead of

a real orthogonal matrix, to transform a

Hermitian matrix into a canonical

diagonal form? 2026 Prof.Jiang@ECE NYU 264 Review: Canonical Form of a Real

Symmetrical Matrix   1 1 Let

be a real symmetric matrix. Then, it can be transformed into the diagonal form by using an orthogonal matrix

so that

where

are the eigenvalues of . 0 0 n n T n n i i A O O AO A                  2026 Prof.Jiang@ECE NYU 265 Extension * * It is possible to generalize this important result to (possibly complex)

matrices ,

Hermitian unitary matr . ., . In this case, we use

, instead of orthogonal matrices, ices

. ., . H i e H H U i e U U I   2026 Prof.Jiang@ECE NYU 266 Examples 1 2

The matrix

is Hermitian. 2 3 1 2

The matrix

is

Hermitian, 2 3 but is a complex symmetrical matri not x. i i i i             2026 Prof.Jiang@ECE NYU 267 Eigenvalues of Hermitian Matrices The eigenvalues of a Hermitian matrix are real, and eigenvectors associated with distinct eigenvalues are orthogonal. 2026 Prof.Jiang@ECE NYU 268 Canonical Transformation 1 * If

is a Hermitian matrix, there exists a unitary matrix

such that

. In particular,

becomes a real orthogonal matrix when

is a real symmetric matrix. 0 0 n H U U HU U H             2026 Prof.Jiang@ECE NYU 269 Idea of Proof As in the case of real symmetric matrices, we

use the Gram-Schmidt Orthogonalization

Process, noting the following: 1 For complex vectors ,

, the inner product is defined

as follows:

, . n n T i i i x y x y y x x y       2026 Prof.Jiang@ECE NYU 270 Exercise 1 2 1 2 1 Compute the eigenvalues ,

of 1 2

2 3 and find a unitary matrix

that 0 reduces

to the diagonal form . 0 ( : use , =

for

vectors ,

in the orth Hint ogonaliz n i i i i H i U H x y x y complex x y               ation process.)

2026 Prof.Jiang@ECE NYU 271 Schur’s Unitary Triangularization 1 * For

square,

necessarily Hermitian,

matrix ,

there is a unitary matrix

for which

* 0

*

0

* being zero or nonzero scala not rs. * 0 n any n n A U U AU with             2026 Prof.Jiang@ECE NYU 272 Algorithm     1 1 2 1 2

Take a normalized eigenvector

of

associated with an eigenvalue ,

and

find 1

vectors , ,

so that

,

, ,

1: are linearly independent. n n x A n y y x y Step y     2026 Prof.Jiang@ECE NYU 273 Algorithm 1 2 1 2 1 2 1

Apply the Gram-Schimidt orthonormalization

procedure to ,

, ,

to produce an

orthonormal set ,

, , .

Define ,

, ,

which, c

learly, 2 : n n n Step x y y x z z U x z z       is

a unitary matrix. 2026 Prof.Jiang@ECE NYU 274 Algorithm   1 2 1 ( 1) 11* 1 1 1 1 1 2 Under ,

, , ,

*

,

with . 0

Of course,

has eigenvalues , , .

2 (cont'd) : n n n n U x z z U AU A A tep A S                   2026 Prof.Jiang@ECE NYU 275 Algorithm     ( 1) 1 1 2 3 1 1 1 ( 1) 12 3 2 1 1 2* 2 1 2 2

For , apply Steps 1-2

to arrive at an orthonormal set ,

,

,

and a unitary matrix

,

,

,

so that *

3

: 0

n n n n n nn A x z z U x z Ste z U A A p U                       ( 2) 2 2,

with

n nA        2026 Prof.Jiang@ECE NYU 276 Algorithm     2 1 2 2 1 2 * 1 2 1 2 ( 2 ) 2 2

It is easy to check that,

1 0

and

0

are both unitary. In addition, *

*

* 0

*

4

*

:

n n n V U V U U V A U V St p O e A                             2026 Prof.Jiang@ECE NYU 277 Algorithm ( 1)( 1)

Continuing these steps to arrive at the

last step, where we have produced

unitary matrices ,

and

,

2,3,

: , 1

n i n i i n n i Last t p V e U n S i             1 2 1 1 *

so that

,

and *

*

. 0 n n U U V V U AU                    2026 Prof.Jiang@ECE NYU 278 Some Applications of Schur’s

Theorem • Useful for solving algebraic,

differential or

difference linear equations. Do you know why? 2026 Prof.Jiang@ECE NYU 279 Applications of Schur’s Theorem • Cayley-Hamilton Theorem         1 1 1 1 Let

be the characteristic polynomial of ,

,

det . Then, : 0. A n n A n n n A n p A that is p I A p A A A I                         See the textbook of Horn & Johnson (2nd ed., 2013),

pp. 109~110. 2026 Prof.Jiang@ECE NYU 280 Comment Cayley-Hamilton Theorem is extremely

important in linear systems theory. 2026 Prof.Jiang@ECE NYU 281 Technical Remark 1 1 1 1 For any square

matrix ,

for any integer , there exist constants , ,

such that

,

. i in i n i in in n n A i n c c A c A c A c I i n            2026 Prof.Jiang@ECE NYU 282 Exercise 2 3 4 1 3 2 Consider the matrix . 1 0

Use Cayley-Hamilton Theorem to

express ,

,

as linear combinations

of ,

.

Use Cayley-Hamilton Theorem to find

the inverse . A A A A A I A        2026 Prof.Jiang@ECE NYU 283 QR Factorization For any (possibly nonsquare) matrix ,

,

,

such that

The columns of

form an orthonormal set,

and

is an upper triangular matrix;

. If, in addition,

is nonsingula n m n m m m A with Q R Q R A n QR m A              r, then the diagonal

entries of

are positive. Moreover, in this case,

and

are unique. R Q R 2026 Prof.Jiang@ECE NYU 284 Remark The factors Q and R may be taken real, if

A is a real matrix. Proof: See the textbook, pp.89~90, for the

constructive procedure closely tied to the

Gram-Schmidt (G-S) algorithm. 2026 Prof.Jiang@ECE NYU 285 An Example What is the

factorization of 1 0

2 3 QR A      2026 Prof.Jiang@ECE NYU 286 Solution     1 2 1 1 1 2 2 1* 2 1 1 0 For simplicity, denote :

. 2 3 1 2 Then, let /

and, 5 5 like in the G-S process, compute 6 3

5 5 T T A a a q a a y a q a q                   2026 287 Solution (cont’d)     2 2 2 1 2 1 2 1 Now, let /

. 5 5

which, by construction, is orthonormal. Then, , (with =0 )

can be determined according to the general formula:

,

1, 2,..., T ij kj j j k kj k q y y Set Q q q R r r k j a r q j m              m = 2, here R is upper-triangular.Prof.Jiang@ECE NYU 2026 Prof.Jiang@ECE NYU 288 Solution (end) 11 21 12 22 6 3 So, 5,

0,

, . 5 5 6 5 5 That is:

3 0 5 It is directly verified that . r r r r R A QR              2026 Prof.Jiang@ECE NYU 289 Application to Cholesky factorization * * By means of

factorization, any matrix

taking the form ,

with ,

can be written as:

,

with

lower triangular. Moreover, this factorization is unique,

if

is nonsingu n n n n n n QR B B A A A B LL L A            lar. Indeed, it suffices to write

to obtain .A QR L R  B: Positive semi-definite 2026 Prof.Jiang@ECE NYU 290 QR Numerical Algorithm This is a powerful tool for computing the

eigenvalues of a matrix. 2026 Prof.Jiang@ECE NYU 291 QR Numerical Algorithm 0 0 0 0 1 0 0 1 1 1 1

For any given , factorize

Define ,

and factorize

Continuing this process, we have

1,

1:

2 : n n k k k k k k Step St A A Q R A R Q A Q R A Q R k e A Q p R            2026 Prof.Jiang@ECE NYU 292 Proposition 0 0

Each

is unitarily equivalent to ,

and thus

they have the same eigenvalues.

If

has distinct eigenvalues, then

converges

to an upper triangular matrix. k k A A A A   2026 Prof.Jiang@ECE NYU 293 A Numerical Exercise Use MATLAB simulation to validate

the

algorithm for the matrix 1 0

. 2 3 QR A      Congruence 2026 Prof.Jiang@ECE NYU 294 * Consider two matrices ,

. (1)

is said to be

,

if

for some nonsingular matrix . (2)

is said to be

,

if

for some nons *

, or

ingular m

T n n T congruent to congruent congruent t A B B A B SAS S B A B SAS o     atrix .S Notice that both congruence are equivalence relations.

(Horn-Johnson, 2nd ed., 2013; p. 281) Inertia 2026 Prof.Jiang@ECE NYU 295              0 0 Consider a Hermitian matrix . Its

is defined as the ordered triple:

( ) ,

,

the number of positive eigenvalues of ; the number of negative eigenval inerti ues of ; a n nA i A i A i A i A where i A A i A A i A           the number of zero eigenvalues of .A Sylvester’s Law of Inertia 2026 Prof.Jiang@ECE NYU 296 Hermitian matrices ,

are *congruent

if and only if they have the same inertia,

i.e., the same number of positive eigenvalues and

the same number of negative eigenvalues. n nA B  For the proof, see (Horn-Johnson, 2nd Ed., 2013, p. 282) Simultaneous Diagonalization 2026 Prof.Jiang@ECE NYU 297 * * Consider two Hermitian matrices , . There is a unitary matrix

and real diagonal

matrices ,

such that ,

is Hermitian, that is, . n n n n A B U M A U U B UMU iff AB AB BA            See (Horn-Johnson, 2nd Edition, 2013, page 286.) 2026 Prof.Jiang@ECE NYU 298 Homework VI 2 1. Transform the following Hermitian matrix 0 2 1

2 5 6 1 6 8

into a diagonal form. 2.

If a (real) Hermitian matrix

is positive definite,

prove that

,

for a positive H H H P         

definite matrix .P 51作业君版权所有