Preliminary Mathematics for online MSc programmes in Data AnalyticsUnit 4: Functions of more than one variable and partial derivatives
Functions of more than one variable
In Unit 1 we defined a function, , from a set to another set to be a rule that assigns for each a unique element . In that case the output depended only on one input; the value of .
However, we can also have two inputs (or more) to a function, each of which can be chosen independently. As before, the function can then be used to produce the output. Note that even though we can have two or more inputs, we will still assume that there is just a single output. These inputs can be represented by any letter (e.g. ) and the output can either be represented by a letter (one that we have not used for any of the inputs) or as a function of the input (e.g. ).
As an important special case, a function of two variables is a rule that produces a single output when specific values of the two variables are chosen. Similarly to the definition of Unit 1; a function, , from a set to another set defines a rule that assigns for each pair a unique element .
Example 1
A function is defined as .
If we want to calculate its value for and we just need to find .
Similarly, if we want to calculate its value for and we need to find .
Example 2
Let's assume that we now have a function that has two inputs :
The figure below shows its graph for and .
Because is a function of and we can now ask questions like "How does change as a function of and ?" or "What happens to the function when increases and takes small values?". Would our answer be the same even if was taking different values?
Another graphical technique for representing a 3-dimensional surface is by plotting constant f(x,y) slices, called contours, on a 2-dimensional format. That is, given a value for f(x,y), lines are drawn for connecting the (x,y) coordinates where that f(x,y) value occurs. Figure 2 shows the contour plot of .
Can you see the connection between the contour plot and the graph of the function (in the previous page)?
Focusing on the left half of the contour plot, we can see that we get exactly the same values at the top and the bottom of the contour plot (they both have the dark blue colour).
(Hint: What happens if I find the values of for the two pairs and respectively?)
Partial differentiation
Introduction to partial differentiation
In Unit 2 we showed how one can differentiate a function of a single variable and compute the derivative of such a function. This gave us information about the slope of the graph of the function at different points. If was the input and (or ) was the output, we used the notation or to obtain the derivative. As a reminder, the latter way would be read as "the derivative of with respect to the variable ".
Consider now that we have (or ) as the function that depends on the two variables and . Thus, can be differentiated with respect to and produce one derivative while it can also be differentiated with respect to and produce another (different) derivative. The derivative with respect to gives us the slope in the -direction, whereas the derivative with respect to gives us the slope in the -direction. So, for functions of two variables we can no longer talk about a single (unique) derivative of .
From now on, when we differentiate a function of two variables, we will refer to this process as partial differentiation. Instead of using the letter in we will use a curly instead and write is as . As an example, when we differentiate with respect to (or ), we will denote the resulting partial derivative as (or ). This is just a different notation, (almost) nothing else changes.
The rules for partial differentiation are the same as when we differentiate a function of a single input, with just one addition. When we differentiate with respect to a specific variable (let's say ) we treat all other variables as they were constants (i.e. numbers). Let's look at an example.
Example 3
Let's assume we have the function
If we differentiate with respect to we should consider any occurence of all other variables as though they were constants. In this case, we only have one more variable that we will consider as a constant.
Thus,
since the derivative of is while the derivative of the constant is . If it helps, imagine that had a specific value () and the derivative of any number is .
If we differentiate with respect to we should consider any occurence of all other variables (namely ) as though they were constants.
since the derivative of with respect to is and the derivative of with respect to is .
Example 4
Let's now assume we have the function
and want to find the partial derivatives.
We start with the partial derivative with respect to . For this we have to treat as a constant and use the product rule.
Note that we have used the product rule to differentiate with respect to .
The partial derivative with respect to is a lot simpler to calculate.
Sometimes, we arrange the partial derivatives into a vector, called the gradient of the function. It is often denoted as , , or .
Higher-order derivatives
In the same way that a function of one variable has a second derivative (which is found by differentiating the first derivative), so too does a function of two variables. The second (order) partial derivatives are found by differentiating the first (order) partial derivatives.
We can differentiate either of the first partial derivatives with respect to or with respect to to obtain various second partial derivatives.
Differentiating with respect to produces .
Differentiating with respect to produces .
Differentiating with respect to produces .
Differentiating with respect to produces .
In most circumstances (if the corresponding derivatives are continuous) the order of differentiation doesn't matter, in which case
The second derivatives are often arranged as a matrix, called the Hessian:
Example 5
Let's assume we have the previous function and want to find the second partial derivatives.
Let's start with . We have already shown that . If we differentiate again with respect to we obtain . In more formal notation,
Let's turn to StartFraction partial differential squared z Over partial differential y partial differential x EndFraction. Derivatives are evaluated right-to-left, so we first have to differentiate with respect to x, just like before. This gave StartFraction partial differential z Over partial differential x EndFraction equals 30 x. Now we have to take the derivative with respect to y. Given that there is no y in 30 x, the derivative with respect to y is 0. StartFraction partial differential z Over partial differential x EndFraction equals 30 x.
StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential x EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 30 x Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 30 x right parenthesis equals 0
Let's turn to StartFraction partial differential squared z Over partial differential x partial differential y EndFraction, so we now swap the order of taking derivatives. We differentiate with respect to y first, giving StartFraction partial differential z Over partial differential y EndFraction equals 1, and differentiate the result with respect to x. As there is no x in the constant expression 1, the derivative is 0. More formally,
StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential Over partial differential x EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 1 Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0
We can see that, as expected, this is the same as StartFraction partial differential squared z Over partial differential y partial differential x EndFraction.
Finally,
StartFraction partial differential squared z Over partial differential y squared EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y right parenthesis With bottom brace Underscript equals 1 Endscripts equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0 period
Hence, the Hessian is
Start 2 By 2 Matrix 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column StartFraction partial differential squared z Over partial differential x partial differential y EndFraction 2nd Row 1st Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 2nd Column StartFraction partial differential squared z Over partial differential y squared EndFraction EndMatrix equals Start 2 By 2 Matrix 1st Row 1st Column 30 2nd Column 0 2nd Row 1st Column 0 2nd Column 0 EndMatrix
Tasks
Task 1
In each of the following cases, calculate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction
(a) z equals 5 x plus 12 y
(b) z equals 9 minus 3 y Superscript 4 Baseline plus 12 x squared
(c) z equals 10 left parenthesis x plus y plus 5 right parenthesis
(d) z equals 9 x squared y
(e) z equals minus 9 y x
(a) StartFraction partial differential z Over partial differential x EndFraction equals 5 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 12
(b) StartFraction partial differential z Over partial differential x EndFraction equals 24 x semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 12 y cubed
(c) StartFraction partial differential z Over partial differential x EndFraction equals 10 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 10
(d) StartFraction partial differential z Over partial differential x EndFraction equals 18 x y semicolon StartFraction partial differential z Over partial differential y EndFraction equals 9 x squared
(e) StartFraction partial differential z Over partial differential x EndFraction equals minus 9 y semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 9 x
Task 2
If z equals 11 x plus 2 y squared, evaluate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction at the point left parenthesis 4 comma negative 3 right parenthesis.
Show answer
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction 4th Column equals 4 y EndLayout
Evaluating these partial derivatives at x equals 4 and y equals negative 3 gives
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 4th Column equals 4 times left parenthesis negative 3 right parenthesis equals negative 12 EndLayout
Task 3
Calculate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction
(a) z equals e Superscript x Baseline e Superscript y
(b) z equals e Superscript x y
(c) z equals e Superscript 5 x
(d) z equals e Superscript 2 y
Show answer
(a) StartFraction partial differential z Over partial differential x EndFraction equals e Superscript x Baseline e Superscript y Baseline semicolon StartFraction partial differential z Over partial differential y EndFraction equals e Superscript x Baseline e Superscript y
(b) StartFraction partial differential z Over partial differential x EndFraction equals y e Superscript x y Baseline semicolon StartFraction partial differential z Over partial differential y EndFraction equals x e Superscript x y
(c) StartFraction partial differential z Over partial differential x EndFraction equals 5 e Superscript 5 x Baseline semicolon StartFraction partial differential z Over partial differential y EndFraction equals 0
(d) StartFraction partial differential z Over partial differential x EndFraction equals 0 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 2 e Superscript 2 y
Task 4
In each of the following cases, calculate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction:
(a) z equals y x e Superscript x
(b) z equals 3 x y cubed e Superscript x
(c) z equals x ln left brace x y right brace
(d) z equals StartFraction 1 Over x squared plus y squared EndFraction
(e) z equals StartFraction x Over x squared plus y squared EndFraction
(a) StartFraction partial differential z Over partial differential x EndFraction equals y e Superscript x Baseline plus x y e Superscript x Baseline semicolon StartFraction partial differential z Over partial differential y EndFraction equals x e Superscript x
(b) StartFraction partial differential z Over partial differential x EndFraction equals 3 e Superscript x Baseline y cubed plus 3 x e Superscript x Baseline y cubed semicolon StartFraction partial differential z Over partial differential y EndFraction equals 9 x y squared e Superscript x
(c) StartFraction partial differential z Over partial differential x EndFraction equals 1 plus ln left parenthesis x y right parenthesis semicolon StartFraction partial differential z Over partial differential y EndFraction equals StartFraction x Over y EndFraction
(d) StartFraction partial differential z Over partial differential x EndFraction equals minus StartFraction 2 x Over left parenthesis x squared plus y squared right parenthesis squared EndFraction semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus StartFraction 2 y Over left parenthesis x squared plus y squared right parenthesis squared EndFraction
(e) StartFraction partial differential z Over partial differential x EndFraction equals StartFraction left parenthesis y squared minus x squared right parenthesis Over left parenthesis x squared plus y squared right parenthesis squared EndFraction semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus StartFraction 2 x y Over left parenthesis x squared plus y squared right parenthesis squared EndFraction
Task 5
Find all the second partial derivatives in each of the following cases:
(a) z equals 8 x plus 2 y plus 11
(b) z equals 10 y squared x plus 2
(c) z equals minus 2 x Superscript 4 Baseline y squared
(a) StartFraction partial differential squared z Over partial differential x squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals 0
(b) StartFraction partial differential squared z Over partial differential x squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals 20 x semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals 20 y
(c) StartFraction partial differential squared z Over partial differential x squared EndFraction equals minus 24 x squared y squared semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals minus 4 x Superscript 4 Baseline semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals minus 16 x cubed y
(d) StartFraction partial differential squared z Over partial differential x squared EndFraction equals 8 y squared e Superscript x y Baseline semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals 8 x squared e Superscript x y Baseline semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals 8 left bracket x y e Superscript x y Baseline plus e Superscript x y Baseline right bracket
(e) StartFraction partial differential squared z Over partial differential x squared EndFraction equals StartFraction 2 Over x cubed EndFraction semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals 0
(f) StartFraction partial differential squared z Over partial differential x squared EndFraction equals StartFraction 2 y Over x cubed EndFraction semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals minus StartFraction 1 Over x squared EndFraction
(g) StartFraction partial differential squared z Over partial differential x squared EndFraction equals 0 semicolon StartFraction partial differential squared z Over partial differential y squared EndFraction equals StartFraction 2 x Over y cubed EndFraction semicolon StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals minus StartFraction 1 Over y squared EndFraction
In Unit 2 we used differentiation to find the maximum and minimum values of a function of a single variable. You may ask yourselves "Is there a similar procedure for functions of two variables?" There is.
To locate stationary points in a function of 2 variables we have to set the first partial derivatives StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction equal to zero and find the values of x and y that satisfy this.
Example 6
Let's go back to the function f left parenthesis x comma y right parenthesis equals minus StartFraction x squared Over 2 EndFraction plus x y squared that we were looking at earlier.
The first order partial derivatives are StartFraction partial differential z Over partial differential x EndFraction equals negative x plus y squared and StartFraction partial differential z Over partial differential y EndFraction equals 2 x y. We now need to find the values of x and y for which both partial derivatives are zero.
Setting the partial derivative with respect to x to zero yields x equals y squared. We can now plug this into the derivative with respect to y (i.e. replace every occurrence of x by y squared), which yields StartFraction partial differential z Over partial differential y EndFraction vertical bar Subscript x equals y squared Baseline equals 2 y cubed. Setting this to zero gives y equals 0. Then x equals y squared equals 0. Thus the only stationary point of the function is at x equals 0 and y equals 0.
The value of the function at that stationary point is the z coordinate and is obtained using z equals minus StartFraction 0 squared Over 2 EndFraction plus 0 dot 0 squared equals 0. The graph of the function along with the stationary point upper A can be seen in the figure below.
Distinguishing between stationary points
So far, we have seen how to find the stationary points of a function but not how to distinguish between them. In order to do that we have to look at the second order derivatives.
Specifically, if we want to find the (local) maximum or (local) minimum values in a function of two variables we can:
locate the position of stationary points, let's say x 1 comma y 1, by looking for points where StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction are equal to zero, and
calculate the expression left parenthesis StartFraction partial differential squared z Over partial differential x squared EndFraction right parenthesis left parenthesis StartFraction partial differential squared z Over partial differential y squared EndFraction right parenthesis minus left parenthesis StartFraction partial differential squared z Over partial differential x partial differential y EndFraction right parenthesis squared at the stationary points x 1 comma y 1.
If the expression is:
positive and StartFraction partial differential squared z Over partial differential x squared EndFraction is positive we have a (local) minimum point,
positive and StartFraction partial differential squared z Over partial differential x squared EndFraction is negative we have a (local) maximum point,
negative then we have what is known as a saddle point (a point that the slope is zero but its neither a minimum nor a maximum),
zero the test is inconclusive and we need further tests to decide whether it is a minimum or a maximum.
Example 7
Let's determine whether the stationary point from Example 6 is a (local) minimum, maximum or saddle point. We found the first partial derivatives in Example 6.
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals negative x plus y squared 3rd Column StartFraction partial differential z Over partial differential y EndFraction 4th Column equals 2 x y EndLayout
The second derivatives are
StartLayout 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column equals StartFraction partial differential Over partial differential x EndFraction left parenthesis negative x plus y squared right parenthesis equals negative 1 3rd Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 4th Column equals StartFraction partial differential Over partial differential y EndFraction left parenthesis negative x plus y squared right parenthesis equals 2 y 5th Column StartFraction partial differential squared z Over partial differential y squared EndFraction 6th Column equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 2 x y right parenthesis equals 2 x EndLayout
We now need to evaluate
left parenthesis StartFraction partial differential squared z Over partial differential x squared EndFraction right parenthesis left parenthesis StartFraction partial differential squared z Over partial differential y squared EndFraction right parenthesis minus left parenthesis StartFraction partial differential squared z Over partial differential x partial differential y EndFraction right parenthesis squared equals negative 1 times 2 x minus left parenthesis 2 y right parenthesis squared equals minus 2 x minus 4 y squared
at x equals 0 and y equals 0, for which the quantity is 0. Our test is hence inconclusive.
From the figure in Example 6 we can however see that it is a saddle point. (To see this from the derivatives we would have to take third-order partial derivatives).
Tasks
Task 6
Locate the stationary points (and distinguish between them) of the following functions:
(a) z equals 3 x y plus x plus y
(b) z equals x squared plus y squared minus 3 y
(c) z equals x squared plus y squared minus 3 x y
(d) z equals StartFraction 1 Over x EndFraction plus StartFraction 1 Over y EndFraction minus StartFraction 3 Over x y EndFraction
StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals 3 y plus 1 3rd Column StartFraction partial differential z Over partial differential y EndFraction 4th Column equals 3 x plus 1 EndLayout
Setting both partial derivatives to zero yields x equals negative one third and y equals negative one third.
The second derivatives are
StartLayout 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column equals 0 3rd Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 4th Column equals 3 5th Column StartFraction partial differential squared z Over partial differential y squared EndFraction 6th Column equals 0 EndLayout
Thus
StartFraction partial differential squared z Over partial differential x squared EndFraction StartFraction partial differential squared z Over partial differential y squared EndFraction minus left parenthesis StartFraction partial differential squared z Over partial differential x partial differential y EndFraction right parenthesis squared equals 0 times 0 minus 3 squared equals negative 9 less than 0 comma
hence there is a saddle point at x equals negative one third and y equals negative one third.
(b) x equals 0, y equals three halves; minimum
(c) x equals 0, y equals 0; saddle point
(d) x equals 3, y equals 3; saddle point
(e) x equals 0, y equals 0; saddle point
Task 7
Determine the stationary points of f left parenthesis x comma y right parenthesis equals 2 x squared plus 3 y squared plus 5 x plus 12 y plus 19.
The stationary point is at x equals negative five fourths, y equals negative 2. It is a minimum.
Task 8
Calculate StartFraction partial differential z Over partial differential x EndFraction when
(a) z equals StartFraction y Over x squared EndFraction minus StartFraction x Over y squared EndFraction
(b) z equals e Superscript x squared minus 4 x y
(c) z equals StartFraction x squared minus 3 y squared Over x squared plus y squared EndFraction
Show answer
(a) StartFraction partial differential z Over partial differential x EndFraction equals minus StartFraction 2 y Over x cubed EndFraction minus StartFraction 1 Over y squared EndFraction
(b) StartFraction partial differential z Over partial differential x EndFraction equals left parenthesis 2 x minus 4 y right parenthesis e Superscript x squared minus 4 x y
(c) StartFraction partial differential z Over partial differential x EndFraction equals StartFraction 8 x y squared Over left parenthesis x squared plus y squared right parenthesis squared EndFraction