Matrix Differential Calculus Cheat Sheet Page 4
ADVERTISEMENT
1
2
3
4Matrix Differential Calculus Cheat Sheet
Stefan Harmeling
Blue Note 142 (started on 27-08-2013)
compiled on 30-8-2013 15:45
(2)
(0)
(b) Find the derivative of x
(x
).
(2)
T
(1)
T
T
(0)
x
= (I + ξA
A) x
= (I + ξA
A) (I + ξA
A) x
(70)
( )
(0)
(c) Find the derivative of x
(x
).
( )
T
(0)
x
= (I + ξA
A)
x
(71)
(1)
(d) Find the derivative of x
(ξ).
(1)
T
(0)
x
=
A
(y
Ax
) ξ
(72)
(2)
(e) Find the derivative of x
(ξ).
(2)
T
(1)
(1)
x
= ( ξA
(y
Ax
) + x
)
(73)
T
(1)
T
(1)
(1)
=
A
(y
Ax
) ξ + ξA
A x
+ x
(74)
T
(1)
T
(1)
=
A
(y
Ax
) ξ + (ξA
A + I ) x
(75)
T
(1)
T
T
(0)
= ( A
(y
Ax
)
(ξA
A + I )A
(y
Ax
)) ξ
(76)
(77)
(3)
(f) Find the derivative of x
(ξ).
(3)
T
(2)
(2)
x
= ( ξA
(y
Ax
) + x
)
(78)
T
(2)
T
(2)
=
A
(y
Ax
) ξ + (ξA
A + I ) x
(79)
T
(2)
T
T
(1)
T
2
T
(0)
=
A
(y
Ax
) ξ
(ξA
A + I )A
(y
Ax
)
(ξA
A + I )
A
(y
Ax
) ξ
(80)
2
T
T
(2
)
=
(ξA
A + I ) A
(y
Ax
)
ξ
(81)
=0
( +1)
(g) Find the derivative of x
(ξ).
( +1)
T
T
(
)
x
=
(ξA
A + I ) A
(y
Ax
)
ξ
(82)
=0
(1)
(h) Find the derivative of x
(A).
(1)
(0)
T
(0)
x
= (x
ξA
(y
Ax
))
(83)
T
(0)
T
(0)
=
ξ( A)
(y
Ax
) + ξA
( A)x
(84)
(0)
T
(0)
T
T
=
ξ(I
(y
Ax
)
) vec A + ξ((x
)
A
) vec A
(85)
(0)
T
(0)
T
T
=
ξ(I
(y
Ax
)
+ (x
)
A
) vec A
(86)
2
T
1
T
10. Consider the norm φ = r
= r
r of the residual r = y
Ax. Plug into φ the minimizer x = (A A)
A
y,
we obtain
1
T
2
1
T
2
φ = (y
A(A A)
A
y)
= ((I
A(A A)
A
)y)
(87)
Additionally we have an expression for A:
T
A = C
Diag(Za)B
(88)
So we can view φ as a function of a. Find the derivative of φ(a). Left as an exercise ;)
References
[1] H. L¨ u tkepohl. Handbook of matrices. Wiley, 1996.
[2] J.R. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistics and Economet-
rics. John Wiley, Chichester, 1999.
4
ADVERTISEMENT
0 votes
Related Articles
Related forms
Related Categories
Parent category: Education