# Verifying a neat identity: showing a connection between linear algebra and calculus

My PhD student Nikoleta and I are doing some work relating the Prisoner’s Dilemma to quadratic forms (a linear algebraic generalisation of quadratics) and it’s led me to really come to appreciate a specific identity regarding their derivatives. In this post I’ll describe what the identity is and what it means but also verify it using Python.

Let us assume we have 2 symbolic variables: \(x, y\) and we have a polynomial expression in these two variables where the degree of each term is 2:

Here is a heatmap of it:

A quadratic form has the nice property that it can be represented using a linear algebraic representation. If we let:

and

then:

(\(v ^ T\) denotes the transpose of \(v\) so it’s a row vector.)

Let’s start by verifying this using Sympy (the Python library for symbolic mathematics):

```
>>> import sympy as sym
>>> x, y = sym.symbols("x, y")
>>> quadratic_form = - x ** 2 - 4 * x * y + y ** 2
>>> quadratic_form
-x**2 - 4*x*y + y**2
>>> v = sym.Matrix([[x], [y]])
>>> A = sym.Matrix([[-sym.S(1), - sym.S(2)], [-sym.S(2), sym.S(1)]])
>>> ((v.transpose() * A * v)[0, 0]).expand() == quadratic_form
True
```

The identity that my student and I have been using is for the derivative of a quadratic form:

and more specifically in the case where \(A ^ T = A\) (\(A ^ T\) is the transpose of \(A\)):

A good overview of the mathematical background behind this is given here: http://michael.orlitzky.com/articles/the_derivative_of_a_quadratic_form.xhtml.

One of the intuitive reasons why this is “nice” and/or “looks right” is that it looks eerily familiar to \(\frac{dx^2}{dx}=2x\): the derivative of the most basic quadratic.

The first thing to realise about \(2v^TA\) is that it is a row vector. Each element of the vector corresponds to the partial derivate of the quadratic form according to each variable.

In our case:

Here is how to do these derivatives using Sympy:

```
>>> sym.diff(quadratic_form, x)
-2*x - 4*y
>>> sym.diff(quadratic_form, y)
-4*x + 2*y
```

The identify above shows that we do not need to do any differentiation at all to obtain this: we can instead do a linear algebraic manipulation:

We see that the only stationary point is \((x, y)=(0, 0)\) (which we can see in the picture above).

This can extend to larger quadratic forms: indeed this holds for any quadratic form over any number of variables.