The Taylor Series

1.1. The Taylor Series#

The Taylor series is an infinite sum of terms where each term is a derivative of the function evaluated at a specific point. Named after English mathematician Brook Taylor the Taylor series expansion of the function $f(t)$ at the point $a$ is

\[\begin{split} \begin{align*} f(t) &= f(a) + \frac{f'(a)}{1!}(x - a) + \frac{f''(a)}{2!}(x - a)^2 + \frac{f'''(a)}{3!}(x - a)^3 + \cdots \\ &= \sum_{n=0}^\infty \frac{f^{(n)}(a)}{n!}(x - a)^n, \end{align*} \end{split}\]

where $f^{(n)}(t)$ denotes the $n$-th derivative of $f(t)$. If we replace $a$ with $t$ and $t$ with $t+h$ then we can write the Taylor series as

\[\begin{split} \begin{align*} f(t+h) &= f(t) + \frac{h}{1!}f'(t) + \frac{h^2}{2!}f''(t) + \frac{h^3}{3!}f'''(t) + \cdots \\ &= \sum_{n=0}^\infty \frac{h^n}{n!}f^{(n)}(t). \end{align*} \end{split}\]

In this form the Taylor series can be used to derive numerical methods for solving differential equations. From now on we shall be referring to this form as the Taylor series.

Definition 1.3 (The Taylor series)

If $f(t)$ is a differentiable function of the variable $t$ then the Taylor series expansion of $f(t+h)$ where $h$ is some scalar value is

(1.2)#\[f(t+h) = \sum_{n = 0}^\infty \frac{h^n}{n!}f^{(n)}(t).\]

We can use equation (1.2) to calculate the value of $f(t+h)$. However, we cannot sum an infinite number of terms so instead we only consider the first few terms. For example, if we include the terms up to and including the $n$-th derivative term then

\[ \begin{align*} f(t+h) &= \underbrace{f(t) + hf'(t) + \frac{h^2}{2!}f''(t) + \cdots + \frac{h^n}{n!}f^{(n)}(t)}_{\text{$n$-th order Taylor series expansion}} + \underbrace{\frac{h^{n+1}}{(n+1)!}f^{(n+1)}(t) + \cdots}_{\text{Highter Order Terms (HOT)}} \end{align*} \]

Since we are omitting the higher order terms in the truncation of the Taylor series we only have an approximation of $f(t+h)$. When we omit the Higher-Order Terms (HOT) from the Taylor series we say we have truncated the Taylor series. So if we omit all terms higher than fourth-order we have say that we have the fourth-order Taylor series expansion of $f(t+h)$ which is

\[f(t+h) = f(t) + hf' (t)+\frac{h^2}{2}f''(t) + \frac{h^3}{3!}f'''(t) + \frac{h^4}{4!}f^{(4)}(t).\]

For example lets use the first, second, third and fourth-order Taylor series expansions to approximate the value of $\cos(1+h)$ where $h=0.1$. Here $f(t) = \cos(t)$ so the derivatives of $f(t)$ are

\[\begin{split} \begin{align*} f'(t) &= -\sin(t), \\ f''(t) &= -\cos(t), \\ f'''(t) &= \sin(t), \\ f^{(4)}(t) &= \cos(t), \\ & \vdots \end{align*} \end{split}\]

then the fourth-order Taylor series expansions of $\cos(t+h)$ is

\[ \begin{align*} \cos(t+h) &= \cos(t) - h \sin(t) - \frac{h^2}{2} \cos(t) + \frac{h^3}{6} \sin(t) + \frac{h^4}{24} \cos(t). \end{align*} \]

Substituting $t=1$ and $h=0.1$ and truncating the Taylor series expansion to the first-order term then

\[ \cos(1 + 0.1) = \cos(1) - 0.1 \sin(1) = 0.456155. \]

The exact value is $\cos(1.1) = 0.453596$ (correct to 6 decimal places) so the error in the first-order Taylor series approximation is $|0.456155 - 0.453596| = 0.002559$. Doing similar for the second, third and fourth-order Taylor series gives

\[\begin{split} \begin{align*} \text{second-order}: && \cos(1 + 0.1) &= \underbrace{0.456155}_{\text{first-order}} - \frac{0.1^2}{2} \cos(1) = 0.453454, \\ \text{third-order}: && \cos(1 + 0.1) &= \underbrace{0.453454}_{\text{second-order}} + \frac{0.1^3}{6} \sin(1) = 0.453594, \\ \text{fourth-order}: && \cos(1 + 0.1) &=\underbrace{0.453594}_{\text{third-order}} + \frac{0.1^4}{24} \cos(1) = 0.453596. \end{align*} \end{split}\]

The Taylor series approximations of $\cos(1.1) = 0.453596\ldots$ and the corresponding errors for the first, second, third and fourth order Taylor series expansions is shown in Table 1.1. Note that as we include more terms in the Taylor series expansion the error tends to zero.

Table 1.1 Taylor series approximations of $\cos(1.1)$ up to fourth-order accuracy.#
Order	Approximation	Error
first	0.456155	2.56e-03
second	0.453454	1.42e-04
third	0.453594	2.18e-06
fourth	0.453596	7.09e-08