# An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation

@inproceedings{Giles2008AnEC, title={An extended collection of matrix derivative results for forward and reverse mode algorithmic dieren tiation}, author={Michael B. Giles}, year={2008} }

This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation (AD). It highlights in particular the remarkable contribution of a 1948 paper by Dwyer and Macphail which derives the linear and adjoint sensitivities of a matrix product, inverse and determinant, and a number of related results motivated by applications in multivariate analysis in statistics.
This is an extended version of a paper which will appear… Expand

#### 60 Citations

Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation

- Mathematics
- 2008

This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation. It highlights in particular the remarkable… Expand

On evaluating higher-order derivatives of the QR decomposition of tall matrices with full column rank in forward and reverse mode algorithmic differentiation

- Mathematics, Computer Science
- Optim. Methods Softw.
- 2012

We address the task of higher-order derivative evaluation of computer programs that contain QR decompositions of tall matrices with full column rank. The approach is a combination of univariate… Expand

Algorithmic Differentiation of Linear Algebra Functions with Application in Optimum Experimental Design (Extended Version)

- Mathematics, Computer Science
- ArXiv
- 2010

We derive algorithms for higher order derivative computation of the rectangular QR and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of… Expand

Derivatives of partial eigendecomposition of a real symmetric matrix for degenerate cases

- Physics, Computer Science
- ArXiv
- 2020

The forward and backward derivatives of partial eigendecomposition, i.e. where it only obtains some of the eigenpairs, of a real symmetric matrix for degenerate cases are presented. Expand

Algorithmic differentiation in Python with AlgoPy

- Computer Science
- J. Comput. Sci.
- 2013

AlgoPy provides the means to compute derivatives of arbitrary order and Taylor approximations of such programs as NumPy, based on a combination of univariate Taylor polynomial arithmetic and matrix calculus in the (combined) forward/reverse mode of Algorithmic Differentiation (AD). Expand

The Kähler Mean of Block-Toeplitz Matrices with Toeplitz Structured Blocks

- Mathematics, Computer Science
- SIAM J. Matrix Anal. Appl.
- 2016

The generalized barycenter is derived, or generalized Kahler mean, and a greedy approximation is shown to be close to the generalized mean with a significantly lower computational cost. Expand

N A ] 4 S ep 2 01 9 Automatic Differentiation for Complex Valued SVD

- 2019

Automatic differentiation(AD) evaluates derivatives or gradients of any functions specified by computer programs[1]. It is implemented by propagating derivatives of primitive operations via chain… Expand

The Bordering Method of the Cholesky Decomposition and its Backward Differentiation

- Computer Science
- 2017

The backward differentiation of the Cholesky decomposition by the bordering method is described and it is found that the resulting algorithm can be adapted to vector processing, as is also true of the algorithms developed from the inner product form and outer product form. Expand

Differentiation of the Cholesky decomposition

- Mathematics, Computer Science
- ArXiv
- 2016

New `blocked' algorithms, based on differentiating the Cholesky algorithm DPOTRF in the LAPACK library, are recommended, which uses `Level 3' matrix-matrix operations from BLAS, and so is cache-friendly and easy to parallelize. Expand

A review of automatic differentiation and its efficient implementation

- Computer Science, Mathematics
- Wiley Interdiscip. Rev. Data Min. Knowl. Discov.
- 2019

Automatic differentiation is a powerful tool to automate the calculation of derivatives and is preferable to more traditional methods, especially when differentiating complex algorithms and mathematical functions. Expand

#### References

SHOWING 1-10 OF 19 REFERENCES

Some Applications of Matrix Derivatives in Multivariate Analysis

- Mathematics
- 1967

Abstract It is claimed that the reasons for using matrices of derivatives, in appropriate situations, are as compelling as those for using matrices. This paper provides basic material for such use.… Expand

Old and New Matrix Algebra Useful for Statistics

- Mathematics
- 2000

The partials with respect to the numerator are laid out according to the shape of Y while the partials with respect to the denominator are laid out according to the transpose of X. For example, dy/dx… Expand

Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition

- Computer Science
- Frontiers in applied mathematics
- 2000

This second edition has been updated and expanded to cover recent developments in applications and theory, including an elegant NP completeness argument by Uwe Naumann and a brief introduction to scarcity, a generalization of sparsity. Expand

An efficient overloaded implementation of forward mode automatic differentiation in MATLAB

- Mathematics, Computer Science
- TOMS
- 2006

The Mad package described here facilitates the evaluation of first derivatives of multidimensional functions that are defined by computer codes written in MATLAB through the separation of the linear combination of derivative vectors into a separate derivative vector class derivvec. Expand

Matrix inversion algorithms by means of automatic differentiation

- Mathematics
- 1994

Abstract There are many matrix inversion algorithms, some being widely known and others not as widely known. We will show that some of known elaborate formulas for matrix inversion can be derived by… Expand

ADMAT : Automatic differentiation in MATLAB using object oriented methods ∗

- 2007

Differentiation is one of the fundamental problems in numerical mathematics. The solution of many optimization problems and other applications require knowledge of the gradient, the Jacobian matrix,… Expand

ADMIT-1: automatic differentiation and MATLAB interface toolbox

- Computer Science
- TOMS
- 2000

This article provides an introduction to the design and usage of ADMIT-1, a generic automatic differentiation tool that enables the computation of sparse Jacobian and Hessian matrices from a MATLAB environment. Expand

The Scaling and Squaring Method for the Matrix Exponential Revisited

- Mathematics
- 2009

The scaling and squaring method is the most widely used method for computing the matrix exponential, not least because it is the method implemented in the MATLAB function expm. The method scales the… Expand

Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs

- Computer Science
- Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation
- 2002

A novel software tool is proposed to automatically transform a given MATLAB program into another MATLab program capable of computing not only the original function but also user-specified derivatives of that function. Expand

An Introduction to Multivariate Statistics

- Computer Science, Medicine
- Canadian journal of psychiatry. Revue canadienne de psychiatrie
- 1993

There are costs associated with these benefits, such as increased complexity, decreased power, multiple ways of answering the same question, and ambiguity in the allocation of shared variance. Expand