When studying multivariable calculus, we often come across the use of matrices to represent different concepts. We often come across the Jacobian, the Hessian and the gradient.
These concepts are close to each other, by virtue of being matrices having to do with derivatives of functions, However, each matrix has its own derivation, and signification.
What is the difference between the Jacobian, the Hessian and the gradient 🤔? We will discover it together in this article. During this mathematical journey, we will also study the concepts of the difference between vectorvalued functions and scalarvalued functions.
The Gradient
We start with the gradient which we already talked about a few weeks ago. The gradient is a vector composed of the partial derivatives of a scalarvalued function, $f$:
where $\nabla$ is called nabla. It is sometimes called “del”. It denotes the vector differential operator.
Recall that the gradient measures the direction and the fastest rate of increase of a function $f$ at a given point.
Let’s imagine that we are at the foot of a mountain, and we wanted to climb it as quickly as possible. We see that the right track is to climb it in the direction of the arrow:
This arrow represents the gradient because it is the fastest ascent up the mountain.
A scalarvalued function
A scalarvalued function is a function (multivariable / of dimension $n$) that returns a scalar value:
For example, if we evaluate $f(x,y) = x + y$ at the point (2,1), we get the scalar, 3:
How to calculate the Gradient
Let’s take an example. I have a function defined as $f(x,y) = 5x^2 + 3xy + 3y^3$. First, we need to find the partial derivatives with respect to the variables $x$ and $y$ as follows:
This gives us a gradient:
The Hessian
If we follow the explanation of the gradient, the Hessian will also be easy to understand. It is the derivative of the gradient of a scalarvalued function $f: \mathbb{R}^n \rightarrow \mathbb{R}$. For instance :
Using again the function $f(x,y) = 5x^2 + 3xy + 3y^3$, we will see a Hessian matrix as follows:
Some applications of the Hessian matrix are the following:

Quadratic approximations of a multivariable function. This is a closer approximation to the function than the local linear approximation we already discussed several weeks ago.

The partial second derivative test  it is used to find saddle points, the maxima and the minima of a function.
The Hessian determinant
The Hessian determinant plays a role in finding the local maxima/minima of a multivariable function. We will see what a determinant is in a later article. For now, I’ll just explain how to calculate it. I present to you once again the Hessian matrix that we have already calculated in the last part:
In order to find the determinant, we need to follow the framework below:
Therefore,
The Jacobian
The Jacobian is a matrix that holds all firstorder partial derivatives of a vectorvalued function:
The Jacobian form of a vectorvalued function $h(f(x,y), g(x, y))$ is therefore the following:
For example  if we have a vectorvalued function like:
The Jacobian matrix will be :
By using the Jacobian, we can locally linearize a vector valued function at a specific point.
Linear functions are simple enough that we can understand them well (using linear algebra), and often understanding the local linear approximation of $f(x,y) allows us to draw conclusions about$ f $$ itself.
To learn more about the calculation, here is a video that helped me a lot:
The determinant of the Jacobian matrix
We can use Jacobian matrix of a vector valued transformation to find its determinant. To do this with a transformation of $f: \mathbb{R}^3 \rightarrow \mathbb{R}^3$, we use the framework:
Consider this example using this Jacobian matrix:
We can calculate the determinant as follows:
What does the determinant mean? 🤔 The function $2xy+x^2$ gives us the amplitude in which space contracts or expands during a transformation around a point $(x, y, z)$.
For example, if we evaluate it at the point $(1, 0, 2)$, we will see:
Which tells us that the space does not change around this point! However, around the point $(3, 1, 0)$, we see another story:
I recommend this resource in order to visualize the transformations and the determinant for each one!
Resources
 Ask Ethan: What is A Scalar Field?  Forbes
 A Gentle Introduction To Hessian Matrices  Machine Learning Mastery
 A Gentle Introduction to the Jacobian  Machine Learning Mastery
 Vector and scalar functions and fields. Derivatives
 What is the Jacobian matrix?  Stack Exchange
 The Hessian  KhanAcademy
 What is the difference between the Jacobian, Hessian and the Gradient?  Stack Exchange