Matrix Differential Calculus Cheat Sheet

ADVERTISEMENT

Matrix Differential Calculus Cheat Sheet
Stefan Harmeling
Blue Note 142 (started on 27-08-2013)
compiled on 30-8-2013 15:45
1
Rules for the differentials
Let α, a, A be constants and φ, ψ, u, v, x, f , U , V , F be functions.
α = 0
a = 0
A = 0
(1)
(αφ) = α φ
(αu) = α u
(αU ) = α U
(2)
(φ + ψ) = φ + ψ
(u + v) = u + v
(U + V ) = U + V
(3)
T
T
T
(φ ψ) = ( φ)ψ + φ ψ
(u
v) = ( u)
v + u
v
(U V ) = ( U )V + U V
(4)
2
T
T
T
T
(φ/ψ) = (( φ)ψ
φ ψ) / ψ
(u
) = ( u)
(U
) = ( U )
(5)
vec U = vec U
tr U = tr U
(6)
(U
V ) = ( U )
V + U
V
(U
V ) = ( U )
V + U
V
(7)
1
1
1
1
(φ ) = αφ
φ
(U
) =
U
( U ) U
(8)
1
1
det U = det(U ) tr(U
U )
log(det U ) = tr(U
U )
(9)
exp φ = exp(φ) φ
tr( exp U ) = tr(exp(U ) U )
(10)
Identification table
Note that the differential has always the same shape as the function.
function
differential
derivative
shape of derivative
φ(ξ)
φ = α(ξ) ξ
φ(ξ) = α(ξ)
1
1
(11)
T
T
φ(x)
φ = a(x)
x
φ(x) = a(x)
1
n
(12)
T
T
φ(X)
φ = tr A(X)
X
φ(X) = (vec A(X))
1
nq
(13)
f (ξ)
f = a(ξ) ξ
f (ξ) = a(ξ)
m
1
(14)
f (x)
f = A(x) x
f (x) = A(x)
m
n
(15)
f (X)
f = A(X) vec X
f (X) = A(X)
m
nq
(16)
F (ξ)
F = A(ξ) ξ
F (ξ) = vec A(ξ)
mp
1
(17)
F (x)
vec F = A(x) x
F (x) = A(x)
mp
n
(18)
F (X)
vec F = A(X) vec X
F (X) = A(X)
mp
nq
(19)
Notation
α, β, φ, ξ, . . .
lower case Greek letters are scalars
(20)
a, b, c, f, x, . . .
lower case Latin letters are (column) vectors
(21)
A, B, C, F, X, . . .
capital Latin letters are matrices
(22)
φ
differential of φ
(23)
φ(x)
derivative of φ at x
(24)
0 , 0
n vector of zeros, m
n matrix of zeros
(25)
1 , 1
n vector of ones, m
n matrix of ones
(26)
I , I
n
n identity matrix, m
n identity matrix
(27)
T
A
, tr A, det A
transpose of A, trace of A, determinant of A
(28)
vec A
vector containing the stacked columns of A
(29)
diag A
vector containing the diagonal of A
(30)
Diag a
diagonal matrix with a along the diagonal
(31)
exp α
scalar exponential function
(32)
exp A
matrix exponential function, det(exp(A)) = exp(tr(A))
(33)
A
B
Hadamard product (component-wise product)
(34)
A
B
component-wise division
(35)
T
A
B
Kronecker product, vec ab
= b
a
(36)
1
Most of the material of this document is from [2], available at at the author’s webpage
mdc2007-3rdedition.pdf. Another good resource for matrix wisdom is [1].
1

ADVERTISEMENT

00 votes

Related Articles

Related forms

Related Categories

Parent category: Education
Go
Page of 4