Taylor Series - Anby's blog

691 words

3 minutes

Taylor Series

2025-10-16

Mathematic

Taylor’s Formula#

Let’s consider a polynomial:

p=p(x)=a_0+a_1x+a_2x^2+\dots+a_nx^n

Using high school differentiation rules, we can easily obtain the following results:

\begin{aligned} p'&=a_1+2a_2x+3a_3x^2+\dots+na_nx^{n-1},\\ p''&=2a_2+3\cdot 2a_3x+\dots+n\cdot (n-1)a_nx^{n-2},\\ &\dots\\ p^{(n)}&=n\cdot(n-1)\cdot(n-2)\dots2\cdot a_n \end{aligned}

Now, let $x=0$ , then we have:

p^{(i)}(0)=a_i\cdot i!\\ \Rightarrow a_i=\frac{p^{(i)}(0)}{i!}

Substituting back into the original expression, we get:

p(x)=p(0)+\frac{p'(0)}{1!}x+\frac{p''(0)}{2!}x^2+\dots+\frac{p^{(n)}(0)}{n!}x^n

If we had initially let $x=(x-x_0)$ , then at this step, we would obtain:

p(x)=p(x_0)+\frac{p'(x_0)}{1!}(x-x_0)+\frac{p''(x_0)}{2!}(x-x_0)^2+\dots+\frac{p^{(n)}(x_0)}{n!}(x-x_0)^n

This is the well-known Taylor’s formula.

Expansion of Arbitrary Functions#

If a function $f(x)$ is $n$ -times differentiable at $x_0$ on the interval $[a,b]$ , then we can construct the polynomial

p_n(x)=f(x_0)+\frac{f'(x_0)}{1!}(x-x_0)+\frac{f''(x_0)}{2!}(x-x_0)^2+\dots+\frac{f^{(n)}(x_0)}{n!}(x-x_0)^n

However, at this point, $f(x)=p_n(x)$ does not necessarily hold true. The $p(x)$ we obtained can only be regarded as an approximation of the function $f(x)$ .
We denote their error as $r_n(x)=f(x)-p_n(x)$ .
We find that when $x\rightarrow x_0$ , $r_n(x)=o((x-x_0)^n)$ .

Proof

Consider its equivalent proposition:

r_n(x_0)=r_n'(x_0)=\dots=r_n^{(m)}(x_0)=0

We use mathematical induction for the proof.
When $m=0,1$ , clearly:

r_n(x_0)=r_n'(x_0)=0

Assume it holds for $1\le m=k$ . To prove it also holds for $m=k+1$ , i.e.,

r_n(x_0)=r_n'(x_0)=\dots=r_n^{(k+1)}(x_0)=0

then

r_n(x)=o((x-x_0)^{k+1})

Thus,

r_n'(x)=o((x-x_0)^k)

But (A chinese meme) by Lagrange’s Mean Value Theorem, there must exist $\exists x_t\in[x,x_0]$ such that

r_n'(x_t)(x-x_0)=r_n(x)-r_n(x_0)\\ \Rightarrow r_n'(x_t)=o((x-x_0)^k)\\

Then we very easily obtain:

f(x)=\sum^n_{i=0}{\frac{f^{(i)}(x_0)}{i!}(x-x_0)^i}+o((x-x_0)^n)

This is called the Taylor’s formula with Peano remainder.
Next, we prove its uniqueness.

Proof of Uniqueness

Suppose we simultaneously have:

f(x)=\sum^n_{i=0}{A_i(x-x_0)^i}+o((x-x_0)^n)

and

f(x)=\sum^n_{i=0}{A'_i(x-x_0)^i}+o((x-x_0)^n)

Letting $x\to x_0$ , we get $A_0=A'_0$ . Subtract the two equations, then divide by $(x-x_0)$ , to get $A_1=A'_1$ . Continuing this process, we obtain:

A_i=A'_i(\forall i\in[1,n]\cap\mathbb{N})

Thus, it’s unique!

Substituting $x_0=0$ , we get a more concise expression called the Maclaurin expansion. Giving it a new name?! Actually quite ironic.
Below are the Maclaurin expansions of some basic elementary functions.

\begin{aligned} e^x&=\sum_{i=0}^n\frac{x^i}{i!}+o(x^n)\\ \sin(x)&=\sum_{i=0}^n(-1)^{i+1}\frac{x^{2i-1}}{(2i-1)!}+o(x^{2n})\\ \cos(x)&=\sum_{i=0}^n(-1)^i\frac{x^{2i}}{(2i)!}+o(x^{2n+1})\\ \end{aligned}

Taylor Series#

In the previous Taylor’s formula, note that $n$ can take any value, so naturally, we consider letting it tend to infinity.
Thus, the Taylor series refers to the Taylor formula when $n\to\infty$ ~
Formally, the Taylor series has the following form:

f(x)=\sum^\infty_{i=0}{A_i(x-x_0)^i}

where

A_n=\frac{f^{(i)}(x_0)}{i!}

are called the Taylor coefficients.
Although this seems very simple, it raises many problems. The most direct one is: Does that series converge? And will it still equal the original function? This is a very tricky problem.

$f(x)$ can equal that summation if and only if the remainder term $r_n(x)$ satisfies: $\lim_{n\to\infty}r_n(x)=0$ The proof is obvious.
But to prove this equality holds, we might need to study other forms of the remainder term. Below, I will discuss the case for $x_0=0$ ; other cases are left for the reader to explore.

Lagrange form: $r_n(x)=\frac{f^{(n+1)}(\theta x)}{(n+1)!}x^{n+1}$
Cauchy form: $r_n(x)=\frac{f^{(n+1)}(\theta x)}{n!}(1-\theta)x^{n+1}$

where $\theta\in[0,1]$ .
Proof? The Lagrange form is proved using Lagrange’s Mean Value Theorem, and the Cauchy form is proved using Cauchy’s Mean Value Theorem.

Here are the specific Taylor expansions of some functions.

Taylor Expansions of Exponential and Basic Trigonometric Functions#

If a function $f(x)$ has derivatives of all orders on the interval $[0,H]$ or $[-H,0](H>0)$ , and when $x$ varies within the given interval, the absolute values of all these derivatives are bounded by the same number: $|f^{(n)}(x)|\le L$ Then this function can be expanded as a Taylor series.

Proof

We take the remainder in the Lagrange form, then we easily get: $|r_n(x)|=\frac{|f^{(n+1)}(\theta x)|}{(n+1)!}|x|^{n+1}\le L\cdot\frac{H^{n+1}}{(n+1)!}$ Let’s prove a stronger conclusion. Summing both sides, we get:

\sum_{n=0}^\infty|r_n(x)|\le L\cdot\sum_{n=0}^\infty \frac{H^{n+1}}{(n+1)!}

The right-hand side clearly converges, thus we can deduce $r_n(x)\to 0$ !

When $f(x)=e^x,\sin(x),\cos(x)$ , the conditions are clearly satisfied, so we can write their expansions:

\begin{aligned} e^x&=\sum_{i=0}^\infty\frac{x^i}{i!}\\ \sin(x)&=\sum_{i=1}^\infty(-1)^{i+1}\frac{x^{2i-1}}{(2i-1)!}\\ \cos(x)&=\sum_{i=0}^\infty(-1)^i\frac{x^{2i}}{(2i)!}\\ \arctan(x)&=\sum_{i=1}^\infty(-1)^{i+1}\frac{x^{2i-1}}{2i-1} \end{aligned}

Here, for the arctangent expansion, substituting $x=1$ , we get:

\frac\pi4=1-\frac13+\frac15-\cdots

which is the classic Leibniz formula.

Logarithmic Function#

Consider expanding $f(x)=\ln(1+x)$ . The reason we expand $\ln(1+x)$ is that it’s convenient for calculation.
First, write its Taylor series:

\ln(1+x)=\sum_{n=1}^\infty(-1)^{n+1}\frac{x^n}{n}

We use the Lagrange form of the remainder to consider its convergence.

\lim_{n\to\infty}r_n(x)=\frac{(-1)^n}{n+1}\cdot(\frac x{\theta x+1})^{n+1}=0

But unfortunately, the above only holds when $x\in(-1,1]$ , meaning its expansion converges only in this interval.

Stirling’s Formula#

Most applications of Taylor series involve estimation. Here is a classic example of estimation.
Consider the Taylor expansion of $\ln(1+x)$ we obtained earlier. We can easily get: