Preliminary Mathematics for online MSc programmes in Data AnalyticsUnit 4: Functions of more than one variable and partial derivatives
Functions of more than one variable
In Unit 1 we defined a function, , from a set to another set to be a rule that assigns for each a unique element . In that case the output depended only on one input; the value of .
However, we can also have two inputs (or more) to a function, each of which can be chosen independently. As before, the function can then be used to produce the output. Note that even though we can have two or more inputs, we will still assume that there is just a single output. These inputs can be represented by any letter (e.g. ) and the output can either be represented by a letter (one that we have not used for any of the inputs) or as a function of the input (e.g. ).
As an important special case, a function of two variables is a rule that produces a single output when specific values of the two variables are chosen. Similarly to the definition of Unit 1; a function, , from a set to another set defines a rule that assigns for each pair a unique element .
Example 1
A function is defined as .
If we want to calculate its value for and we just need to find .
Similarly, if we want to calculate its value for and we need to find .
Example 2
Let's assume that we now have a function that has two inputs :
The figure below shows its graph for and .
Because is a function of and we can now ask questions like "How does change as a function of and ?" or "What happens to the function when increases and takes small values?". Would our answer be the same even if was taking different values?
Another graphical technique for representing a 3-dimensional surface is by plotting constant f(x,y) slices, called contours, on a 2-dimensional format. That is, given a value for f(x,y), lines are drawn for connecting the (x,y) coordinates where that f(x,y) value occurs. Figure 2 shows the contour plot of .
Can you see the connection between the contour plot and the graph of the function (in the previous page)?
Focusing on the left half of the contour plot, we can see that we get exactly the same values at the top and the bottom of the contour plot (they both have the dark blue colour).
(Hint: What happens if I find the values of for the two pairs and respectively?)
Partial differentiation
Introduction to partial differentiation
In Unit 2 we showed how one can differentiate a function of a single variable and compute the derivative of such a function. This gave us information about the slope of the graph of the function at different points. If was the input and (or ) was the output, we used the notation or to obtain the derivative. As a reminder, the latter way would be read as "the derivative of with respect to the variable ".
Consider now that we have (or ) as the function that depends on the two variables and . Thus, can be differentiated with respect to and produce one derivative while it can also be differentiated with respect to and produce another (different) derivative. The derivative with respect to gives us the slope in the -direction, whereas the derivative with respect to gives us the slope in the -direction. So, for functions of two variables we can no longer talk about a single (unique) derivative of .
From now on, when we differentiate a function of two variables, we will refer to this process as partial differentiation. Instead of using the letter in we will use a curly instead and write is as . As an example, when we differentiate with respect to (or ), we will denote the resulting partial derivative as (or ). This is just a different notation, (almost) nothing else changes.
The rules for partial differentiation are the same as when we differentiate a function of a single input, with just one addition. When we differentiate with respect to a specific variable (let's say ) we treat all other variables as they were constants (i.e. numbers). Let's look at an example.
Example 3
Let's assume we have the function
If we differentiate with respect to we should consider any occurence of all other variables as though they were constants. In this case, we only have one more variable that we will consider as a constant.
Thus,
since the derivative of is while the derivative of the constant is . If it helps, imagine that had a specific value () and the derivative of any number is .
If we differentiate with respect to we should consider any occurence of all other variables (namely ) as though they were constants.
since the derivative of with respect to is and the derivative of with respect to is .
Example 4
Let's now assume we have the function
and want to find the partial derivatives.
We start with the partial derivative with respect to . For this we have to treat as a constant and use the product rule.
Note that we have used the product rule to differentiate with respect to .
The partial derivative with respect to is a lot simpler to calculate.
Sometimes, we arrange the partial derivatives into a vector, called the gradient of the function. It is often denoted as , , or .
Higher-order derivatives
In the same way that a function of one variable has a second derivative (which is found by differentiating the first derivative), so too does a function of two variables. The second (order) partial derivatives are found by differentiating the first (order) partial derivatives.
We can differentiate either of the first partial derivatives with respect to or with respect to to obtain various second partial derivatives.
Differentiating with respect to produces .
Differentiating with respect to produces .
Differentiating with respect to produces .
Differentiating with respect to produces .
In most circumstances (if the corresponding derivatives are continuous) the order of differentiation doesn't matter, in which case
The second derivatives are often arranged as a matrix, called the Hessian:
Example 5
Let's assume we have the previous function and want to find the second partial derivatives.
Let's start with . We have already shown that . If we differentiate again with respect to we obtain . In more formal notation,
Let's turn to StartFraction partial differential squared z Over partial differential y partial differential x EndFraction. Derivatives are evaluated right-to-left, so we first have to differentiate with respect to x, just like before. This gave StartFraction partial differential z Over partial differential x EndFraction equals 30 x. Now we have to take the derivative with respect to y. Given that there is no y in 30 x, the derivative with respect to y is 0. StartFraction partial differential z Over partial differential x EndFraction equals 30 x.
StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential x EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 30 x Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 30 x right parenthesis equals 0
Let's turn to StartFraction partial differential squared z Over partial differential x partial differential y EndFraction, so we now swap the order of taking derivatives. We differentiate with respect to y first, giving StartFraction partial differential z Over partial differential y EndFraction equals 1, and differentiate the result with respect to x. As there is no x in the constant expression 1, the derivative is 0. More formally,
StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential Over partial differential x EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 1 Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0
We can see that, as expected, this is the same as StartFraction partial differential squared z Over partial differential y partial differential x EndFraction.
Finally,
StartFraction partial differential squared z Over partial differential y squared EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y right parenthesis With bottom brace Underscript equals 1 Endscripts equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0 period
Hence, the Hessian is
Start 2 By 2 Matrix 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column StartFraction partial differential squared z Over partial differential x partial differential y EndFraction 2nd Row 1st Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 2nd Column StartFraction partial differential squared z Over partial differential y squared EndFraction EndMatrix equals Start 2 By 2 Matrix 1st Row 1st Column 30 2nd Column 0 2nd Row 1st Column 0 2nd Column 0 EndMatrix
Tasks
Task 1
In each of the following cases, calculate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction
(a) z equals 5 x plus 12 y
(b) z equals 9 minus 3 y Superscript 4 Baseline plus 12 x squared
(c) z equals 10 left parenthesis x plus y plus 5 right parenthesis
(d) z equals 9 x squared y
(e) z equals minus 9 y x
(a) StartFraction partial differential z Over partial differential x EndFraction equals 5 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 12
(b) StartFraction partial differential z Over partial differential x EndFraction equals 24 x semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 12 y cubed
(c) StartFraction partial differential z Over partial differential x EndFraction equals 10 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 10
(d) StartFraction partial differential z Over partial differential x EndFraction equals 18 x y semicolon StartFraction partial differential z Over partial differential y EndFraction equals 9 x squared
(e) StartFraction partial differential z Over partial differential x EndFraction equals minus 9 y semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 9 x
Task 2
If z equals 11 x plus 2 y squared, evaluate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction at the point left parenthesis 4 comma negative 3 right parenthesis.
Show answer
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction 4th Column equals 4 y EndLayout
Evaluating these partial derivatives at x equals 4 and y equals negative 3 gives
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 4th Column equals 4 times left parenthesis negative 3 right parenthesis equals negative 12 EndLayout
Task 3
Calculate StartFraction partial differential z Over partial differential x EndFraction