Matrix Differential Calculus Cheat Sheet Page 3
ADVERTISEMENT
1
2
3
4Matrix Differential Calculus Cheat Sheet
Stefan Harmeling
Blue Note 142 (started on 27-08-2013)
compiled on 30-8-2013 15:45
2
4. Find the derivative of φ(x) = (y
Ax)
.
T
T
φ = ((y
Ax)
(y
Ax)) =
2(y
Ax) A x thus
φ(x) =
2A
(y
Ax)
(47)
T
1
T
T
1
5. Find the derivative of f (X) = (X
X)
X
y. We write A = (X
X)
.
T
1
T
f = (X
X)
X
y
(48)
T
T
=
A(( X) X + X
( X))AX
y
(49)
T
T
T
T
=
A( X) XAX
y
AX
( X)AX
y
A( X)
y
(50)
T
T
T
T
T
=
(A
(XAX
y)
) vec X
((XAX
y)
A) vec X
(A
y
) vec X
(51)
T
T
T
T
T
=
(A
(XAX
y)
+ (XAX
y)
A + A
y
)
vec X
(52)
D ( )
T
T
1
T
6. Often it is easier to find the differential of a scalar function φ(X) = c
(X
X)
X
y.
T
T
T
T
T
T
T
φ =
c
A( X) XAX
y
c
AX
( X)AX
y
c
A( X)
y
(53)
T
T
T
T
T
T
T
=
tr(c
A( X) XAX
y)
tr(c
AX
( X)AX
y)
tr(c
A( X)
y)
(54)
T
T
T
T
T
T
T
T
=
tr(yXA
X
( X)A
c)
tr(c
AX
( X)AX
y)
tr(y
( X)A
c)
(55)
T
T
T
T
T
T
T
T
=
tr(A
c yXA
X
X)
tr(AX
yc
AX
X)
tr(A
cy
X)
(56)
T
T
T
T
T
T
T
T
=
tr((A
c yXA
X
+ AX
yc
AX
+ A
cy
) X)
(57)
7. Sometimes it is good to rewrite with indices:
T
T
T
T
T
tr(A Diag v) = tr(A(I
v1
)) = tr((A
I)v1
= 1
Diag(A)v = diag(A)
v = diag(A)
v (58)
T
T
tr(A Diag v) =
A v = tr diag(A)
v = tr diag(A)
v
(59)
T
T
8. Find the derivative of Rayleigh coefficient φ(x) = x
Ax/(x
x) for symmetric A.
T
T
T
T
2x
A( x)(x
x)
2x
Axx
x
φ =
(60)
T
2
(x
x)
T
T
T
T
2(x
x)x
A( x)
2x
Axx
x
=
(61)
T
2
(x
x)
T
T
T
T
2(x
x)x
A
2x
Axx
=
x
(62)
T
2
(x
x)
2
T
T
T
=
x
(xx
A
Axx
) x
(63)
T
2
(x
x)
(64)
Thus the derivative is:
2
T
T
φ(x) =
(Axx
xx
A)x
(65)
T
2
(x
x)
T
Ax
x
Ax
= 2
2
x
(66)
2
x x
(x x)
More difficult examples
9. Consider a steepest descent algorithm for minimizing the previous function:
( +1)
( )
T
( )
x
= x
ξA
(y
Ax
)
(67)
(1)
(0)
( +1)
( )
(a) Find the derivative of x
(x
) and of x
(x
).
(1)
(0)
T
(0)
T
(0)
x
= x
+ ξA
A x
= (I + ξA
A) x
(68)
( +1)
T
( )
x
= (I + ξA
A) x
(69)
3
ADVERTISEMENT
0 votes
Related Articles
Related forms
Related Categories
Parent category: Education