Preliminary Mathematics for online MSc programmes in Data AnalyticsUnit 2: Differentiation in 1D (minima and maxima)

Differentiation

Introduction to differentiation

We are often interested in the rate at which some variable is changing. For example, we may be interested in the rate at which the temperature is changing in a chemical reaction or in the rate at which the pressure in a vessel is changing. Rapid rates of change of a variable may indicate that a system is not operating normally and is approaching critical values.

Rates of change may be positive, zero, or negative. A positive rate of change means that the variable is increasing; a zero rate of change means that the variable is not changing; while a negative change of rate means that the variable is decreasing.

Consider the function f left parenthesis x right parenthesis equals minus x cubed plus x squared plus e Superscript x for x element of left bracket negative 1.5 comma 4 right bracket, shown below.

x f(x)= x 3 + x 2 + e x -1 0 1 2 3 4 0 1 2 3 4 5 6
Figure 1

Between x equals negative 2 and x equals negative 1, the function is decreasing rapidly. Across this interval the rate of change of the function f left parenthesis x right parenthesis is large and negative. Between x equals negative 1 and x equals negative 0.3 the function is still decreasing but not as rapidly as before. Across this interval the rate of change of the function f left parenthesis x right parenthesis is small and negative. There is a small interval, left parenthesis negative 0.3 comma negative 0.2 right parenthesis that the function seems to not change at all. Across that interval the rate of change is zero. Between x equals negative 0.2 and x equals 1.8 the function is increasing rapidly; the rate of change is large and positive.

It is often not sufficient to describe a rate of change as "large and positive" or "small and negative". A precise value is needed. The technique for calculating the rate of change of any function is called differentiation. Use of differentiation provides a precise value or expression for the rate of change of a function.

Average rate of change across an interval

We have already seen that a function can have different rates of change at different points on its graph. Let's first define and calculate the average rate of change of a function across an interval and later on we will also define the rate of change at a point. The figure below shows a function f left parenthesis x right parenthesis; two possible argument values, a and b, and their two respective outputs f left parenthesis a right parenthesis and f left parenthesis b right parenthesis.

x f(x) a b f(a) f(b) f(b)-f(a) b-a
Figure 2

Consider that x is increasing from a to b. The change in x is b minus a. As x increases from a to b, then the function f left parenthesis x right parenthesis increases from f left parenthesis a right parenthesis to f left parenthesis b right parenthesis. The change in f left parenthesis x right parenthesis is f left parenthesis b right parenthesis minus f left parenthesis a right parenthesis. Then the average rate of change of y across the interval is StartFraction change in y Over change in x EndFraction equals StartFraction f left parenthesis b right parenthesis minus f left parenthesis a right parenthesis Over b minus a EndFraction

Another way to think of the average rate of change of a function is by visualising it as the slope of a line that passes through two points on the function. This line, called a secant line, can be drawn on a graph of a function so that we can quantify the value of the slope of the line. A secant line passing through the points left parenthesis a comma f left parenthesis a right parenthesis right parenthesis and left parenthesis b comma f left parenthesis b right parenthesis right parenthesis has a vertical rise of f left parenthesis b right parenthesis minus f left parenthesis a right parenthesis and a horizontal run of b minus a. The slope of the line, between the points a and b, is StartFraction f left parenthesis b right parenthesis minus f left parenthesis a right parenthesis Over b minus a EndFraction (which is exactly the same as the average rate of change).

Example 1

Let's calculate the average rate of change of f left parenthesis x right parenthesis equals x squared across the following intervals

(a) x equals 1 to x equals 4 (b) x equals negative 2 to x equals 0

For the first interval the change in x is equal to 4 minus 1 equals 3. When x equals 1, f left parenthesis x right parenthesis equals 1; while when x equals 4 , f left parenthesis x right parenthesis equals 16. Thus, the change of f left parenthesis x right parenthesis is 16 minus 1 equals 15. So, the avarage rate of change across the interval left bracket 1 comma 4 right bracket is StartFraction 15 Over 3 EndFraction equals 5. What does this mean though? It means that across the interval left bracket 1 comma 4 right bracket, on average the f left parenthesis x right parenthesis value increases by 5 for every 1 unit increase in x.

x f(x)= x 2 -4 -2 0 2 4 0 5 10 15 20 16-1=15 4-1=3
Figure 3

This is a good time for you to try out the second interval. (The average rate of change turns out to be -2.)

Rate of change at a point

We often need to know the rate of change of a function at a point, and not simply an average rate of change across an interval. Let's assume that b is really close to a. To better reflect this is our notation, we will call what we used to call a, x, and what we used to call b, x plus h, with h being a very small number.

x f(x) x x+h f(x) f(x+h) δ y = f(x+h)-f(x) δ x =h
Figure 4

As mentioned earlier, the average rate of change of y across the interval left bracket x comma x plus h right bracket is

StartLayout 1st Row 1st Column StartFraction change in y direction Over change in x direction EndFraction 2nd Column equals StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over x plus h minus x EndFraction 2nd Row 1st Column Blank 2nd Column equals StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over h EndFraction EndLayout

What do you think would happen if we assumed that the distance, h, between the two points was made increasingly small (in Mathematics notation h right arrow 0)?

If we assumed that, it would mean that the second point x plus h is really close to x. This is exactly what we will assume in order to find the rate of change at the point x. Let's say that we assumed that h right arrow 0. If we now focus again on the graph above and assume that h right arrow 0, the distance between the two points x and x plus h would get smaller and likewise the difference between their respective outputs, f left parenthesis x right parenthesis and f left parenthesis x plus h right parenthesis, would also get smaller. We can define those respective differences as delta x and delta y respectively. The term delta x reads as "delta x" and represents a small change in the x direction. In our case delta x equals x plus h minus x equals h and delta y equals f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis.

Thus, the rate of change at a point a is StartLayout 1st Row 1st Column StartFraction small change in y direction Over small change in x direction EndFraction 2nd Column equals limit Underscript delta x right arrow 0 Endscripts StartFraction delta y Over delta x EndFraction 2nd Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over h EndFraction EndLayout

Let's look at a couple of examples first and then focus on terminology and notation.

Example 2

One of the simplest functions to consider is a linear function. Let's assume that we have f left parenthesis x right parenthesis equals 2 x plus 3.

x f(x)=2x+3 x x+h f(x) f(x+h) δ y = f(x+h)-f(x) δ x =h
Figure 5

What should we do if we want to find the rate of change at any point of the function? (We want to essentially answer the question "What is the change in the y direction when the change in the x direction is small")

Let's use the definition we saw earlier and calculate the rate of change at any point x of the function (think of it as looking at the two points x and x plus h with h right arrow 0).

StartLayout 1st Row 1st Column StartFraction small change in y direction Over small change in x direction EndFraction 2nd Column equals limit Underscript delta x right arrow 0 Endscripts StartFraction delta y Over delta x EndFraction 2nd Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over x plus h minus x EndFraction 3rd Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction 2 left parenthesis x plus h right parenthesis plus 3 minus left parenthesis 2 x plus 3 right parenthesis Over x plus h minus x EndFraction 4th Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction 2 CrossOut h EndCrossOut Over CrossOut h EndCrossOut EndFraction 5th Row 1st Column Blank 2nd Column equals 2 EndLayout

Wait. The rate of change for the function f left parenthesis x right parenthesis at any point x is 2? What does that mean?

It means that the f left parenthesis x right parenthesis value increases by 2 h for every small increase, h, in x. So it doesn't matter which x value we are looking at (e.g. x equals 2 or x equals 58.5); the f left parenthesis x right parenthesis value will always increase by 2 for every small increase, h, in x (i.e. x equals 2 plus h or x equals 58.5 plus h where h right arrow 0).

For non-linear functions a one unit increase in the value of x leads to different increases in f left parenthesis x right parenthesis.

Example 3

Consider a quadratic function f left parenthesis x right parenthesis equals x squared.

Before we use the previous definition and calculate the rate of change at any point, let's try something else.

x f(x)= x 2 x x+h f(x) f(x+h) δ y = f(x+h)-f(x) δ x =h
Figure 6

What will happen to the f left parenthesis x right parenthesis values:

  • if x equals 1 and we increase it by 1 unit (i.e. x equals 2)? The f left parenthesis x right parenthesis values will increase by 3 (i.e. 2 squared minus 1 squared).

  • if x equals 2 and we increase it by 1 unit (i.e. x equals 3)? The f left parenthesis x right parenthesis values will increase by 5 (i.e. 3 squared minus 2 squared).

  • if x equals 3 and we increase it by 1 unit (i.e. x equals 4)? The f left parenthesis x right parenthesis values will increase by 7 (i.e. 4 squared minus 3 squared).

Thus, in a quadratic function a 1 unit increase in x leads to different increases in the f left parenthesis x right parenthesis values.

Let's now use the definition to find out what is happening in the f left parenthesis x right parenthesis values when x is increased by h with h right arrow 0 (instead of x being increased by 1).

StartLayout 1st Row 1st Column StartFraction small change in y direction Over small change in x direction EndFraction 2nd Column equals limit Underscript delta x right arrow 0 Endscripts StartFraction delta y Over delta x EndFraction 2nd Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over x plus h minus x EndFraction 3rd Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction left parenthesis x plus h right parenthesis squared minus x squared Over x plus h minus x EndFraction 4th Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction CrossOut x squared EndCrossOut plus 2 x h plus h squared CrossOut minus x squared EndCrossOut Over x plus h minus x EndFraction 5th Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts StartFraction CrossOut h EndCrossOut left parenthesis 2 x plus h right parenthesis Over CrossOut h EndCrossOut EndFraction 6th Row 1st Column Blank 2nd Column equals limit Underscript h right arrow 0 Endscripts left parenthesis 2 x plus h right parenthesis 7th Row 1st Column Blank 2nd Column equals 2 x EndLayout

So, the rate of change for the function f left parenthesis x right parenthesis at a point x is 2 x. This means that the f left parenthesis x right parenthesis value increases by 2 x for every small increase, h, in x. Thus, the rate of change along a quadratic function is changing constantly (according to the value of x we are looking at), the rate of change has to be computed separately at each possible value of x. The rate of change is thus a local phenomenon: it does not give us any information about the rate of change globally.

Note that the rate of change, 2 x, for the function f left parenthesis x right parenthesis is itself a function of x.

Terminology and notation

The process of finding the rate of change of a given function is called differentiation. The function is said to be differentiated. If f left parenthesis x right parenthesis identical to y (read "f left parenthesis x right parenthesis is equivalent to y") is a function of x we say that y is differentiated with respect to x. The rate of change of a function is also known as the derivative of the function.

There is a notation for writing down the derivative of a function. If the function is f left parenthesis x right parenthesis identical to y, we denote the derivative of y equals f left parenthesis x right parenthesis by

StartFraction normal d y Over normal d x EndFraction equals StartFraction normal d f left parenthesis x right parenthesis Over normal d x EndFraction equals f prime left parenthesis x right parenthesis equals limit Underscript delta x right arrow 0 Endscripts StartFraction delta y Over delta x EndFraction equals limit Underscript h right arrow 0 Endscripts StartFraction f left parenthesis x plus h right parenthesis minus f left parenthesis x right parenthesis Over h EndFraction

(read "dee y (by) dee x", "dee f of x dee x" and "f prime").

This is the point where you should start asking yourselves "Wait a minute, do I have to compute limit Underscript delta x right arrow 0 Endscripts StartFraction delta y Over delta x EndFraction every time I need to find the derivative of a function at a point x?". Thankfully, the answer is no.

Table of derivatives

Table 1 lists some of the common functions used in Mathematics and Statistics and their corresponding derivatives. The symbols k and n are constants while the symbol x represents a variable.

Function f left parenthesis x right parenthesisDerivative f prime left parenthesis x right parenthesis
constant0
x1
k xk
x Superscript nn x Superscript n minus 1
k x Superscript nk n x Superscript n minus 1
e Superscript xe Superscript x
e Superscript k xk e Superscript k x
a Superscript xlog left parenthesis a right parenthesis a Superscript x
log left parenthesis x right parenthesisStartFraction 1 Over x EndFraction
log left parenthesis k x right parenthesisStartFraction 1 Over x EndFraction
Example 4

Find the derivative of f left parenthesis x right parenthesis equals 3 x.

We note that 3 x is of the form k x where k equals 3. This means that f prime left parenthesis x right parenthesis equals 3.

Example 5

Find the derivative of f left parenthesis x right parenthesis equals 3.

This function is constant, hence its derivative is zero.

Example 6

Find the derivative of f left parenthesis x right parenthesis equals 6 x squared.

This function is of the form k x Superscript n with k equals 6 and n equals 2, hence its derivative is 12 x.

Example 7

Find the derivative of f left parenthesis x right parenthesis equals StartRoot x EndRoot.

We first rewrite the function as f left parenthesis x right parenthesis equals StartRoot x EndRoot equals x Superscript one half. This means that the function is of the form k x Superscript n with k equals 1 and n equals one half. This means that f prime left parenthesis x right parenthesis equals one half x Superscript negative one half Baseline equals StartFraction 1 Over 2 x Superscript one half Baseline EndFraction equals StartFraction 1 Over 2 StartRoot x EndRoot EndFraction.

Example 8

Find the derivative of f left parenthesis x right parenthesis equals StartFraction 3 Over x squared EndFraction.

We first rewrite the function as f left parenthesis x right parenthesis equals StartFraction 3 Over x squared EndFraction equals 3 x Superscript negative 2. This means that the function is of the form k x Superscript n with k equals 3 and n equals negative 2. This means that f prime left parenthesis x right parenthesis equals 3 left parenthesis negative 2 right parenthesis x Superscript negative 3 Baseline equals minus 6 x Superscript negative 3 Baseline equals minus StartFraction 6 Over x cubed EndFraction.

Example 9

Find the derivative of f left parenthesis x right parenthesis equals e Superscript 3 x.

This function is of the form e Superscript k x with k equals 3, hence its derivative is f prime left parenthesis x right parenthesis equals 3 e Superscript 3 x.

Ok, that is a good start but what do we do with functions like f left parenthesis x right parenthesis equals 2 x plus 3, g left parenthesis x right parenthesis equals x Superscript 5 Baseline log left parenthesis x right parenthesis and h left parenthesis x right parenthesis equals StartFraction x squared Over e Superscript x Baseline EndFraction?

The first function involves adding two functions (the first one being of the form k x while the second one is a constant function).

The second function, g left parenthesis x right parenthesis, involves multiplying two functions (x Superscript 5 and log left brace x right brace) while the last one, h left parenthesis x right parenthesis, involves dividing two functions (x squared and e Superscript x).

We need to introduce some simple rules to enable us to extend the range of functions that we can differentiate.

Rules of differentiation

  • Differentiation is linear: For any functions f and g and any real numbers a and b, the derivative of the function h left parenthesis x right parenthesis equals a f left parenthesis x right parenthesis plus or minus b g left parenthesis x right parenthesis with respect to x is h prime left parenthesis x right parenthesis equals a f prime left parenthesis x right parenthesis plus or minus b g prime left parenthesis x right parenthesis

  • Product rule: For any functions f and g the derivative of a function h left parenthesis x right parenthesis equals f left parenthesis x right parenthesis g left parenthesis x right parenthesis with respect to x is h prime left parenthesis x right parenthesis equals f prime left parenthesis x right parenthesis g left parenthesis x right parenthesis plus f left parenthesis x right parenthesis g prime left parenthesis x right parenthesis period

  • Quotient rule: For any functions f and g the derivative of a function h left parenthesis x right parenthesis equals StartFraction f left parenthesis x right parenthesis Over g left parenthesis x right parenthesis EndFraction, where g left parenthesis x right parenthesis not equals 0, with respect to x is h prime left parenthesis x right parenthesis equals StartFraction f prime left parenthesis x right parenthesis g left parenthesis x right parenthesis minus f left parenthesis x right parenthesis g prime left parenthesis x right parenthesis Over g squared left parenthesis x right parenthesis EndFraction period

  • Chain rule: The derivative of the function of a composite function h left parenthesis x right parenthesis equals f left parenthesis g left parenthesis x right parenthesis right parenthesis with respect to x is h prime left parenthesis x right parenthesis equals f prime left parenthesis g left parenthesis x right parenthesis right parenthesis g prime left parenthesis x right parenthesis period What is a composite function you ask? It is a function that takes another function as its argument. So, instead of having a function f left parenthesis x right parenthesis that has x as its input, we have a function f which takes g left parenthesis x right parenthesis as its input. Thus, it becomes f left parenthesis g left parenthesis x right parenthesis right parenthesis.

Function h left parenthesis x right parenthesisDerivative h prime left parenthesis x right parenthesis
a f left parenthesis x right parenthesis plus b g left parenthesis x right parenthesisa f prime left parenthesis x right parenthesis plus b g prime left parenthesis x right parenthesis
a f left parenthesis x right parenthesis minus b g left parenthesis x right parenthesisa f prime left parenthesis x right parenthesis minus b g prime left parenthesis x right parenthesis
f left parenthesis x right parenthesis g left parenthesis x right parenthesisf prime left parenthesis x right parenthesis g left parenthesis x right parenthesis plus f left parenthesis x right parenthesis g prime left parenthesis x right parenthesis
StartFraction f left parenthesis x right parenthesis Over g left parenthesis x right parenthesis EndFractionStartFraction f prime left parenthesis x right parenthesis g left parenthesis x right parenthesis minus f left parenthesis x right parenthesis g prime left parenthesis x right parenthesis Over g squared left parenthesis x right parenthesis EndFraction
f left parenthesis g left parenthesis x right parenthesis right parenthesisf prime left parenthesis g left parenthesis x right parenthesis right parenthesis g prime left parenthesis x right parenthesis

Example 10

Find the derivative of h left parenthesis x right parenthesis equals 4 x Superscript 5 Baseline plus 5 x squared.

This function is of the form a f left parenthesis x right parenthesis plus b g left parenthesis x right parenthesis with a equals 4, f left parenthesis x right parenthesis equals x Superscript 5, b equals 5 and g left parenthesis x right parenthesis equals x squared. Hence, f prime left parenthesis x right parenthesis equals 5 x Superscript 4 and g prime left parenthesis x right parenthesis equals 2 x, which yields

h prime left parenthesis x right parenthesis equals a f prime left parenthesis x right parenthesis plus b g prime left parenthesis x right parenthesis equals 4 times 5 x Superscript 4 Baseline plus 5 times 2 x equals 20 x Superscript 4 Baseline plus 10 x

(We could have also used a equals 1, f left parenthesis x right parenthesis equals 4 x Superscript 5, b equals 1 and g left parenthesis x right parenthesis equals 5 x squared.)

Example 11

Find the derivative of h left parenthesis x right parenthesis equals 3 x minus 6 x Superscript 6.

This function is of the form a f left parenthesis x right parenthesis minus b g left parenthesis x right parenthesis with a equals 3, f left parenthesis x right parenthesis equals x, b equals 6 and g left parenthesis x right parenthesis equals x Superscript 6. Hence, f prime left parenthesis x right parenthesis equals 1 and g prime left parenthesis x right parenthesis equals 6 x Superscript 5