Preliminary Mathematics for online MSc programmes in Data AnalyticsUnit 4: Functions of more than one variable and partial derivatives

Functions of more than one variable

In Unit 1 we defined a function, f left parenthesis x right parenthesis, from a set upper A to another set upper B to be a rule that assigns for each x element of upper A a unique element f left parenthesis x right parenthesis element of upper B. In that case the output f left parenthesis x right parenthesis depended only on one input; the value of x.

However, we can also have two inputs (or more) to a function, each of which can be chosen independently. As before, the function can then be used to produce the output. Note that even though we can have two or more inputs, we will still assume that there is just a single output. These inputs can be represented by any letter (e.g. x comma y comma z comma ellipsis) and the output can either be represented by a letter (one that we have not used for any of the inputs) or as a function of the input (e.g. f left parenthesis x comma y comma z right parenthesis).

As an important special case, a function of two variables is a rule that produces a single output when specific values of the two variables are chosen. Similarly to the definition of Unit 1; a function, f left parenthesis x comma y right parenthesis, from a set upper A to another set upper B defines a rule that assigns for each pair left parenthesis x comma y right parenthesis element of upper A a unique element f left parenthesis x comma y right parenthesis element of upper B.

Example 1

A function f left parenthesis x comma y right parenthesis is defined as f left parenthesis x comma y right parenthesis equals StartRoot x squared plus y squared plus 3 x EndRoot.

If we want to calculate its value for x equals 3 and y equals 2 we just need to find f left parenthesis 3 comma 2 right parenthesis equals StartRoot 3 squared plus 2 squared plus 3 dot 3 EndRoot equals StartRoot 22 EndRoot.

Similarly, if we want to calculate its value for x equals 2 and y equals 3 we need to find f left parenthesis 2 comma 3 right parenthesis equals StartRoot 2 squared plus 3 squared plus 3 dot 2 EndRoot equals StartRoot 19 EndRoot.

Example 2

Let's assume that we now have a function f left parenthesis x comma y right parenthesis that has two inputs x comma y:

f left parenthesis x comma y right parenthesis equals minus StartFraction x squared Over 2 EndFraction plus x y squared period

The figure below shows its graph for negative 2 less than x less than 2 and negative 2 less than y less than 2.

x -2 -1 0 1 2 y -2 -1 0 1 2 f(x,y) -10 -5 0 5 -8 -6 -4 -2 0 2 4
Figure 1

Because f is a function of x and y we can now ask questions like "How does f left parenthesis x comma y right parenthesis change as a function of x and y?" or "What happens to the function when x increases and y takes small values?". Would our answer be the same even if y was taking different values?

Another graphical technique for representing a 3-dimensional surface is by plotting constant f(x,y) slices, called contours, on a 2-dimensional format. That is, given a value for f(x,y), lines are drawn for connecting the (x,y) coordinates where that f(x,y) value occurs. Figure 2 shows the contour plot of f left parenthesis x comma y right parenthesis.

-10 -5 0 5 -2 -1 0 1 2 -2 -1 0 1 2 x y
Figure 2

Can you see the connection between the contour plot and the graph of the function (in the previous page)?

Focusing on the left half of the contour plot, we can see that we get exactly the same values at the top and the bottom of the contour plot (they both have the dark blue colour).

(Hint: What happens if I find the values of f left parenthesis x comma y right parenthesis for the two pairs left parenthesis x equals negative 2 comma y equals negative 2 right parenthesis and left parenthesis x equals negative 2 comma y equals 2 right parenthesis respectively?)

Partial differentiation

Introduction to partial differentiation

In Unit 2 we showed how one can differentiate a function of a single variable and compute the derivative of such a function. This gave us information about the slope of the graph of the function at different points. If x was the input and f left parenthesis x right parenthesis (or y) was the output, we used the notation f prime left parenthesis x right parenthesis or StartFraction normal d y Over normal d x EndFraction to obtain the derivative. As a reminder, the latter way would be read as "the derivative of y with respect to the variable x".

Consider now that we have f left parenthesis x comma y right parenthesis (or z) as the function that depends on the two variables x and y. Thus, z can be differentiated with respect to x and produce one derivative while it can also be differentiated with respect to y and produce another (different) derivative. The derivative with respect to x gives us the slope in the x-direction, whereas the derivative with respect to y gives us the slope in the y-direction. So, for functions of two variables we can no longer talk about a single (unique) derivative of z.

From now on, when we differentiate a function of two variables, we will refer to this process as partial differentiation. Instead of using the letter normal d in StartFraction normal d y Over normal d x EndFraction we will use a curly normal d instead and write is as partial differential. As an example, when we differentiate z with respect to x (or y), we will denote the resulting partial derivative as StartFraction partial differential z Over partial differential x EndFraction (or StartFraction partial differential z Over partial differential y EndFraction). This is just a different notation, (almost) nothing else changes.

The rules for partial differentiation are the same as when we differentiate a function of a single input, with just one addition. When we differentiate with respect to a specific variable (let's say x) we treat all other variables as they were constants (i.e. numbers). Let's look at an example.

Example 3

Let's assume we have the function

z equals f left parenthesis x comma y right parenthesis equals 15 x squared plus y period

If we differentiate with respect to x we should consider any occurence of all other variables as though they were constants. In this case, we only have one more variable y that we will consider as a constant.

Thus,

StartFraction partial differential z Over partial differential x EndFraction equals 30 x

since the derivative of 15 x squared is 30 x while the derivative of the constant y is 0. If it helps, imagine that y had a specific value (y equals 20) and the derivative of any number is 0.

If we differentiate with respect to y we should consider any occurence of all other variables (namely x) as though they were constants.

StartFraction partial differential z Over partial differential y EndFraction equals 1

since the derivative of 15 x squared with respect to y is 0 and the derivative of y with respect to y is 1.

Example 4

Let's now assume we have the function

z equals f left parenthesis x comma y right parenthesis equals y x e Superscript 2 x

and want to find the partial derivatives.

We start with the partial derivative with respect to x. For this we have to treat y as a constant and use the product rule.

StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals StartFraction partial differential Over partial differential x EndFraction y x e Superscript 2 x Baseline 2nd Row 1st Column Blank 2nd Column equals y StartFraction partial differential Over partial differential x EndFraction x e Superscript 2 x Baseline 3rd Row 1st Column Blank 2nd Column equals y left parenthesis 1 times e Superscript 2 x Baseline plus x times 2 e Superscript 2 x Baseline right parenthesis equals e Superscript 2 x Baseline y left parenthesis 1 plus 2 x right parenthesis EndLayout

Note that we have used the product rule to differentiate x e Superscript 2 x with respect to x.

The partial derivative with respect to y is a lot simpler to calculate.

StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential y EndFraction 2nd Column equals StartFraction partial differential Over partial differential y EndFraction y x e Superscript 2 x Baseline 2nd Row 1st Column Blank 2nd Column equals x e Superscript 2 x Baseline StartFraction partial differential Over partial differential y EndFraction y 3rd Row 1st Column Blank 2nd Column equals x e Baseline 2 x times 1 equals x e Superscript 2 x EndLayout

Sometimes, we arrange the partial derivatives into a vector, called the gradient of the function. It is often denoted as f prime left parenthesis x comma y right parenthesis, nabla f left parenthesis x comma y right parenthesis, or StartFraction normal d Over normal d left parenthesis x comma y right parenthesis EndFraction.

f prime left parenthesis x comma y right parenthesis equals nabla f left parenthesis x comma y right parenthesis equals StartFraction normal d Over normal d left parenthesis x comma y right parenthesis EndFraction f left parenthesis x comma y right parenthesis equals StartBinomialOrMatrix StartFraction partial differential Over partial differential x EndFraction f left parenthesis x comma y right parenthesis Choose StartFraction partial differential Over partial differential y EndFraction f left parenthesis x comma y right parenthesis EndBinomialOrMatrix

Higher-order derivatives

In the same way that a function of one variable has a second derivative (which is found by differentiating the first derivative), so too does a function of two variables. The second (order) partial derivatives are found by differentiating the first (order) partial derivatives. We can differentiate either of the first partial derivatives with respect to x or with respect to y to obtain various second partial derivatives.

  • Differentiating StartFraction partial differential z Over partial differential x EndFraction with respect to x produces StartFraction partial differential squared z Over partial differential x squared EndFraction equals StartFraction partial differential Over partial differential x EndFraction left parenthesis StartFraction partial differential z Over partial differential x EndFraction right parenthesis.

  • Differentiating StartFraction partial differential z Over partial differential x EndFraction with respect to y produces StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals StartFraction partial differential Over partial differential y EndFraction left parenthesis StartFraction partial differential z Over partial differential x EndFraction right parenthesis.

  • Differentiating StartFraction partial differential z Over partial differential y EndFraction with respect to x produces StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential Over partial differential x EndFraction left parenthesis StartFraction partial differential z Over partial differential y EndFraction right parenthesis.

  • Differentiating StartFraction partial differential z Over partial differential y EndFraction with respect to y produces StartFraction partial differential squared z Over partial differential y squared EndFraction equals StartFraction partial differential Over partial differential y EndFraction left parenthesis StartFraction partial differential z Over partial differential y EndFraction right parenthesis.

In most circumstances (if the corresponding derivatives are continuous) the order of differentiation doesn't matter, in which case

StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals StartFraction partial differential squared z Over partial differential x partial differential y EndFraction period

The second derivatives are often arranged as a matrix, called the Hessian:

Start 2 By 2 Matrix 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column StartFraction partial differential squared z Over partial differential x partial differential y EndFraction 2nd Row 1st Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 2nd Column StartFraction partial differential squared z Over partial differential y squared EndFraction EndMatrix

Example 5

Let's assume we have the previous function f left parenthesis x comma y right parenthesis equals 15 x squared plus y and want to find the second partial derivatives.

Let's start with StartFraction partial differential squared z Over partial differential x squared EndFraction. We have already shown that StartFraction partial differential z Over partial differential x EndFraction equals 30 x. If we differentiate again with respect to x we obtain 30. In more formal notation,

StartFraction partial differential squared z Over partial differential x squared EndFraction equals StartFraction partial differential Over partial differential x EndFraction ModifyingBelow StartFraction partial differential Over partial differential x EndFraction left parenthesis 15 x squared plus y right parenthesis With bottom brace Underscript equals 30 x Endscripts equals StartFraction partial differential Over partial differential x EndFraction left parenthesis 30 x right parenthesis equals 30

Let's turn to StartFraction partial differential squared z Over partial differential y partial differential x EndFraction. Derivatives are evaluated right-to-left, so we first have to differentiate with respect to x, just like before. This gave StartFraction partial differential z Over partial differential x EndFraction equals 30 x. Now we have to take the derivative with respect to y. Given that there is no y in 30 x, the derivative with respect to y is 0. StartFraction partial differential z Over partial differential x EndFraction equals 30 x.

StartFraction partial differential squared z Over partial differential y partial differential x EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential x EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 30 x Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 30 x right parenthesis equals 0

Let's turn to StartFraction partial differential squared z Over partial differential x partial differential y EndFraction, so we now swap the order of taking derivatives. We differentiate with respect to y first, giving StartFraction partial differential z Over partial differential y EndFraction equals 1, and differentiate the result with respect to x. As there is no x in the constant expression 1, the derivative is 0. More formally,

StartFraction partial differential squared z Over partial differential x partial differential y EndFraction equals StartFraction partial differential Over partial differential x EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y With bottom brace right parenthesis Subscript equals 1 Baseline equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0

We can see that, as expected, this is the same as StartFraction partial differential squared z Over partial differential y partial differential x EndFraction.

Finally,

StartFraction partial differential squared z Over partial differential y squared EndFraction equals StartFraction partial differential Over partial differential y EndFraction ModifyingBelow StartFraction partial differential Over partial differential y EndFraction left parenthesis 15 x squared plus y right parenthesis With bottom brace Underscript equals 1 Endscripts equals StartFraction partial differential Over partial differential y EndFraction left parenthesis 1 right parenthesis equals 0 period

Hence, the Hessian is

Start 2 By 2 Matrix 1st Row 1st Column StartFraction partial differential squared z Over partial differential x squared EndFraction 2nd Column StartFraction partial differential squared z Over partial differential x partial differential y EndFraction 2nd Row 1st Column StartFraction partial differential squared z Over partial differential y partial differential x EndFraction 2nd Column StartFraction partial differential squared z Over partial differential y squared EndFraction EndMatrix equals Start 2 By 2 Matrix 1st Row 1st Column 30 2nd Column 0 2nd Row 1st Column 0 2nd Column 0 EndMatrix

Tasks
Task 1

In each of the following cases, calculate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction

(a) z equals 5 x plus 12 y (b) z equals 9 minus 3 y Superscript 4 Baseline plus 12 x squared (c) z equals 10 left parenthesis x plus y plus 5 right parenthesis (d) z equals 9 x squared y (e) z equals minus 9 y x

Show answer
VideoVideo model answers for part (a)Duration1:12VideoVideo model answers for part (d)Duration1:02

(a) StartFraction partial differential z Over partial differential x EndFraction equals 5 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 12 (b) StartFraction partial differential z Over partial differential x EndFraction equals 24 x semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 12 y cubed (c) StartFraction partial differential z Over partial differential x EndFraction equals 10 semicolon StartFraction partial differential z Over partial differential y EndFraction equals 10 (d) StartFraction partial differential z Over partial differential x EndFraction equals 18 x y semicolon StartFraction partial differential z Over partial differential y EndFraction equals 9 x squared (e) StartFraction partial differential z Over partial differential x EndFraction equals minus 9 y semicolon StartFraction partial differential z Over partial differential y EndFraction equals minus 9 x

Task 2

If z equals 11 x plus 2 y squared, evaluate StartFraction partial differential z Over partial differential x EndFraction and StartFraction partial differential z Over partial differential y EndFraction at the point left parenthesis 4 comma negative 3 right parenthesis.

Show answer

StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction 4th Column equals 4 y EndLayout

Evaluating these partial derivatives at x equals 4 and y equals negative 3 gives

StartLayout 1st Row 1st Column StartFraction partial differential z Over partial differential x EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 2nd Column equals 11 3rd Column StartFraction partial differential z Over partial differential y EndFraction vertical bar Subscript x equals 4 comma y equals negative 3 4th Column equals 4 times left parenthesis negative 3 right parenthesis equals negative 12 EndLayout

Task 3

Calculate StartFraction partial differential z Over partial differential x EndFraction