my mathematics cheat sheet

A place to keep quick notes about Math that I keep forgetting. This is meant to be a scratch notes and cheat sheet for me to write math notes before I forget them or move them somewhere else. Can and will contain errors and/or not complete description in number of places. Use at your own risk.

1 What is a ﬁrst integral of a diﬀerential equation and how to ﬁnd it?

Lets start with ﬁrst order ode. This generalizes to any order. Let say our ode is

\frac{d y}{d x} = f (x, y)

The ﬁrst integral of the above ode is any function $Φ (x, y)$ such that its rate of change along $x$ is zero. i.e. $\frac{d}{d x} Φ (x, y) = 0$ . This means

\begin{aligned} \frac{d}{d x} Φ (x, y) & = \frac{\partial Φ}{\partial x} + \frac{\partial Φ}{\partial y} \frac{d y}{d x} \\ = Φ_{x} + Φ_{y} \frac{d y}{d x} \end{aligned}

But $\frac{d y}{d x} = f (x, y)$ , hence the above is

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

If the above comes out to be zero, then $Φ (x, y)$ is called the ﬁrst integral of the ode $\frac{d y}{d x} = f (x, y)$ . In the above, we should make sure to replace any $y$ in the RHS with the solution itself of the ode.

Notice that the ﬁrst integral itself is not constant. But its rate of change as the independent variable changes is what is constant. We should not mix these two things.

But how to ﬁnd the ﬁrst integral function $Φ (x, y)$ ? This is easy. We have to solve the ode itself. Then move all terms to one side, and this is what $Φ (x, y)$ is. Let us look at few examples to make this clear.

It is also possible to ﬁnd ﬁrst integral $Φ (x, y)$ without solving the ode. We just need to ﬁnd any $Φ (x, y)$ such that when we diﬀerentiate it w.r.t. $x$ which gives $Φ_{x} + Φ_{y} f (x, y)$ becomes zero. In here $f (x, y)$ must be the RHS of the ode. So if we by inspection or other means can ﬁnd such $Φ (x, y)$ then no need to solve the ode to ﬁnd it. For some easy ode’s, method of inspection might be possible. There are more advanced methods to ﬁnding ﬁrst integrals. But here, for simplicity, we assume we have the solution to the ode available.

If we want to ﬁrst ﬁrst integral without having the solution to the ﬁrst order ode, and if the ode is already exact ode, then the same method used in solving exact ode can be used to ﬁnd the ﬁrst integral. i.e. if the ode has form

\begin{matrix} (1) & M (x, y) d x + N (x, y) d y = 0 \end{matrix}

Where this is exact (i.e. $\frac{\partial M}{\partial y} = \frac{\partial N}{\partial x}$ ) then we assume ﬁrst integral exist and given by $Φ (x, y) = c_{1}$ . Hence

\begin{matrix} (2) & \frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} \frac{d y}{d x} = 0 \end{matrix}

Comparing (1,2) we see that

\begin{aligned} Φ_{x} & = M \\ Φ_{y} & = N \end{aligned}

From these two equations we can now ﬁnd $Φ (x, y)$ using same methods we use when solving exact ﬁrst order ode. So this is an example where we can ﬁnd $Φ (x, y)$ without knowing the solution to the ode. The above works if the ode is exact.

If the ode is not exact, then we try to ﬁnd an integrating factor which makes the ode exact ﬁrst.

The point of this note is to show that ﬁrst integral is a function $Φ (x, y)$ which happens to be constant along solution curves.

The ﬁrst integral $Φ (x, y)$ of an ode is not unique. We just need to ﬁnd one. Even though the ode itself can have unique solution, there can be many diﬀerent ﬁrst integrals $Φ (x, y)$ . Only condition is that $\frac{d}{d x} Φ (x, y) = 0$ .

1.1 Example $y^{'} = - \frac{x}{y}$

The solution can be found to be

y^{2} = c_{1} - x^{2}

Hence the ﬁrst integral is (moving everything to one side)

Φ (x, y) = y^{2} + x^{2} - c_{1}

To show this the ﬁrst integral, we have to show that $\frac{d}{d x} Φ (x, y) = 0$ . This is given by

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

Looking at our ode we see that $f (x, y) = - \frac{x}{y}$ . The above becomes

\frac{d}{d x} Φ (x, y) = Φ_{x} - Φ_{y} (\frac{x}{y})

But $Φ_{x} = 2 x$ and $Φ_{y} = 2 y$ , then the above becomes

\begin{aligned} \frac{d}{d x} Φ (x, y) & = 2 x - 2 y (\frac{x}{y}) \\ = 0 \end{aligned}

Since $\frac{d}{d x} Φ (x, y) = 0$ then $Φ (x, y) = y^{2} + x^{2} - c_{1}$ is ﬁrst integral.

1.2 Example $y^{'} = x y$

This is linear ode, the solution is

y = c_{1} e^{\frac{x^{2}}{2}}

Hence the ﬁrst integral is (moving everything to one side)

Φ (x, y) = y - c_{1} e^{\frac{x^{2}}{2}}

To show this the ﬁrst integral, we have to show that $\frac{d}{d x} Φ (x, y) = 0$ . This is given by

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

Looking at our ode we see that $f (x, y) = x y$ . The above becomes

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} (x y)

But $Φ_{x} = - c_{1} x e^{\frac{x^{2}}{2}}$ and $Φ_{y} = 1$ , then the above becomes

\frac{d}{d x} Φ (x, y) = - c_{1} x e^{\frac{x^{2}}{2}} + (x y)

But $y$ is the solution $y = c_{1} e^{\frac{x^{2}}{2}}$ , hence the above becomes

\begin{aligned} \frac{d}{d x} Φ (x, y) & = - c_{1} x e^{\frac{x^{2}}{2}} + x c_{1} e^{\frac{x^{2}}{2}} \\ = 0 \end{aligned}

Since $\frac{d}{d x} Φ (x, y) = 0$ then $Φ (x, y) = y - c_{1} e^{\frac{x^{2}}{2}}$ is ﬁrst integral.

1.3 Example $y^{'} = 2 x^{2} y^{2}$

The solution can be found to be

- \frac{1}{y} = \frac{2}{3} x^{3} + c_{1}

Hence the ﬁrst integral is (moving everything to one side)

Φ (x, y) = \frac{1}{y} + \frac{2}{3} x^{3} + c_{1}

To show this is indeed the ﬁrst integral, we have to show that $\frac{d}{d x} Φ (x, y) = 0$ . This is given by

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

Looking at our ode we see that $f (x, y) = 2 x^{2} y^{2}$ . The above becomes

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} (2 x^{2} y^{2})

But $Φ_{x} = 2 x^{2}$ and $Φ_{y} = \frac{- 1}{y^{2}}$ , then the above becomes

\begin{aligned} \frac{d}{d x} Φ (x, y) & = 2 x^{2} - \frac{1}{y^{2}} (2 x^{2} y^{2}) \\ = 0 \end{aligned}

Since $\frac{d}{d x} Φ (x, y) = 0$ then

Φ (x, y) = \frac{1}{y} + \frac{2}{3} x^{3} + c_{1}

Is ﬁrst integral of the ode $y^{'} = 2 x^{2} y^{2}$

1.4 Example $y^{'} = \frac{x - 1}{y}$

The solution is

\frac{y^{2}}{2} = - x + \frac{1}{2} x^{2} + c_{1}

Hence the ﬁrst integral is

Φ (x, y) = \frac{y^{2}}{2} + x - \frac{1}{2} x^{2} - c_{1}

To show this is the ﬁrst integral, we have to show that $\frac{d}{d x} Φ (x, y) = 0$

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

Looking at our ode we see that $f (x, y) = \frac{x - 1}{y}$ , then the above becomes

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} (\frac{x - 1}{y})

But $Φ_{x} = 1 - x$ and $Φ_{y} = y$ , then the above becomes

\begin{aligned} \frac{d}{d x} Φ (x, y) & = 1 - x + y (\frac{x - 1}{y}) \\ = 0 \end{aligned}

Since $\frac{d}{d x} Φ (x, y) = 0$ then

Φ (x, y) = \frac{y^{2}}{2} + x - \frac{1}{2} x^{2} - c_{1}

Is ﬁrst integral of the ode $y^{'} = \frac{x - 1}{y}$ .

1.5 Example $y^{'} = y \sin x$

The solution is

y = c_{1} e^{- \cos x}

Hence the ﬁrst integral is

Φ (x, y) = y - c_{1} e^{- \cos x}

To show this is the ﬁrst integral, we have to show that $\frac{d}{d x} Φ (x, y) = 0$

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} f (x, y)

Looking at our ode we see that $f (x, y) = y \sin x$ , then the above becomes

\frac{d}{d x} Φ (x, y) = Φ_{x} + Φ_{y} (y \sin x)

But $Φ_{x} = - c_{1} \sin (x) e^{- \cos x}$ and $Φ_{y} = 1$ , then the above becomes

\frac{d}{d x} Φ (x, y) = - c_{1} \sin (x) e^{- \cos x} + (y \sin x)

Notice that in this example, we have $y$ in the RHS above that did not cancel as the case was with the ﬁrst two examples. In this case, we have to replace this $y$ by the solution $y = c_{1} e^{- \cos x}$ and the above now becomes

\begin{aligned} \frac{d}{d x} Φ (x, y) & = - c_{1} \sin (x) e^{- \cos x} + c_{1} e^{- \cos x} \sin x \\ = 0 \end{aligned}

Since $\frac{d}{d x} Φ (x, y) = 0$ then

Φ (x, y) = y - c_{1} e^{- \cos x}

Is ﬁrst integral of the ode $y^{'} = y \sin x$ .

2 Special ode’s and their solutions

These are ode’s whose solution is in terms of special functions. Will update as I ﬁnd more. Most of the special functions come up from working out the solution in series of second order ode which has regular singular point at expansion point. These are the more interesting odes which will generate these special functions.

2.1 Airy $y^{''} + a x y = 0$

solution is

y (x) = c_{1} AiryAi (- a^{\frac{1}{3}} x) + c_{2} AiryBi (- a^{\frac{1}{3}} x)

2.2 Chebyshev $(1 - x^{2}) y^{''} - x y^{'} + n^{2} y = 0$

For

(1 - x^{2}) y^{''} - x y^{'} + n^{2} y = 0

Singular points at $x = 1, - 1$ and $\infty$ . Solution valid for $| x | < 1$ . Maple gives solution

y (x) = c_{1} \frac{1}{{(x + \sqrt{x^{2} - 1})}^{n}} + c_{2} {(x + \sqrt{x^{2} - 1})}^{n}

For

(1 - x^{2}) y^{''} - a x y^{'} + n^{2} y = 0

Maple gives solution

\begin{matrix} y (x) = c_{1} {(x^{2} - 1)}^{\frac{1}{2} - \frac{a}{4}} LegendreP (\frac{\sqrt{a^{2} + 4 n^{2} - 2 a + 1}}{2} - \frac{1}{2}, - 1 + \frac{a}{2}, x) \\ + c_{2} {(x^{2} - 1)}^{\frac{1}{2} - \frac{a}{4}} LegendreQ (\frac{\sqrt{a^{2} + 4 n^{2} - 2 a + 1}}{2} - \frac{1}{2}, - 1 + \frac{a}{2}, x) \end{matrix}

If $n$ positive integer, then solution in series gives polynomial solution of degree $n$ . Called Chebyshev polynomials.

2.3 Hermite $y^{''} - 2 x y^{'} + 2 n y = 0$

Converges for all $x$ . If $n$ is positive integer, one series terminates. Series solution in terms of Hermite polynomials.

Maple gives solution

y (x) = c_{1} x KummerM (\frac{1}{2} - \frac{n}{2}, \frac{3}{2}, x^{2}) + c_{2} x KummerU (\frac{1}{2} - \frac{n}{2}, \frac{3}{2}, x^{2})

2.4 Legendre $(1 - x^{2}) y^{''} - 2 x y^{'} + n (n + 1) y = 0$

Series solution in terms of Legendre functions. When $n$ is positive integer, one series terminates (i.e. becomes polynomial).

Maple gives solution

y (x) = c_{1} LegendreP (n, x) + c_{2} LegendreQ (n, x)

If the ode is given in form

\sin (θ) P^{''} (θ) + \cos (θ) P^{'} (θ) + n \sin (θ) P (θ) = 0

Then using $x = \cos θ$ transforms it to the earlier more familar form. Maple gives this as solution

P (θ) = c_{1} LegendreP (\frac{\sqrt{4 n + 1}}{2} - \frac{1}{2}, \cos θ) + c_{2} LegendreQ (\frac{\sqrt{4 n + 1}}{2} - \frac{1}{2}, \cos θ)

2.5 Bessel $x^{2} y^{''} + x y^{'} + (x^{2} - n^{2}) y = 0$

$x = 0$ is regular singular point. Solution in terms of Bessel functions

y (x) = c_{1} BesselJ (n, x) + c_{2} BesselY (n, x)

2.6 Reduced Riccati $y^{'} = a x^{n} + b y^{2}$

For the special case of $n = - 2$ the solution is

y (x) = \frac{λ}{x} - \frac{x^{2 b λ}}{\frac{b x}{2 b λ + 1} x^{2 b λ} + c_{1}}

Where in the above $λ$ is a root of $b λ^{2} + λ + a = 0$ .

For $n \neq - 2$

\begin{aligned} w & = \sqrt{x} {\begin{array}{cc} c_{1} BesselJ (\frac{1}{2 k}, \frac{1}{k} \sqrt{a b} x^{k}) + c_{2} BesselY (\frac{1}{2 k}, \frac{1}{k} \sqrt{a b} x^{k}) & a b > 0 \\ c_{1} BesselI (\frac{1}{2 k}, \frac{1}{k} \sqrt{- a b} x^{k}) + c_{2} BesselK (\frac{1}{2 k}, \frac{1}{k} \sqrt{- a b} x^{k}) & a b < 0 \end{array} \\ y & = - \frac{1}{b} \frac{w^{'}}{w} \\ k & = 1 + \frac{n}{2} \end{aligned}

2.7 Gauss Hypergeometric ode $x (1 - x) y^{''} + (c - (a + b + 1) x) y^{'} - a b y = 0$

Solution is for $| x | < 1$ is in terms of hypergeom function. Has 3 regular singular points, $x = 0, x = 1, x = \infty$ .

Maple gives this solution

y (x) = c_{1} hypergeom ([a, b], [c], x) + c_{2} x^{1 - c} hypergeom ([1 + a - c, 1 + b - c], [2 - c], x)

And Mathematica gives

y (x) = c_{1} HypergeometricF 1 (a, b, c, x) + {(- 1)}^{1 - c} x^{1 - c} c_{2} HypergeometricF 1 (1 + a - c, 1 + b - c, 2 - c, x)

3 Change of variables and chain rule in diﬀerential equation

These are examples doing change of variable for an ode.

3.1 Example 1 Change of the independent variable using $z = g (x)$

Given the ode

\frac{d^{2} y}{d x^{2}} + \frac{d y}{d x} + y = \sin (x)

And we are asked to do change of variables from $x$ to $z$ where $z = g (x)$ . In this, we can also write

x = g^{- 1} (z)

Where $g^{- 1} (z)$ is the inverse function. Using chain rule gives

\frac{d y}{d x} = \frac{d y}{d z} \frac{d z}{d x}

And for second derivative

\begin{aligned} \frac{d^{2} y}{d x^{2}} & = \frac{d}{d x} (\frac{d y}{d x}) \\ = \frac{d}{d x} (\frac{d y}{d z} \frac{d z}{d x}) \end{aligned}

And now we use the product rule, which is $\frac{d}{d x} (a b) = a^{'} b + a b^{'}$ on the above, which gives

\begin{matrix} (1) & \frac{d^{2} y}{d x^{2}} = (\frac{d}{d x} \frac{d y}{d z}) (\frac{d z}{d x}) + (\frac{d y}{d z}) (\frac{d}{d x} \frac{d z}{d x}) \end{matrix}

Let us do each of the terms on the right above one by one. The second term on the RHS above is easy. It is

\begin{matrix} (2) & (\frac{d y}{d z}) (\frac{d}{d x} \frac{d z}{d x}) = (\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}}) \end{matrix}

It is the ﬁrst term in (1) which needs more care. The problem is how to handle $\frac{d}{d x} \frac{d y}{d z}$ ? Since the denominators are diﬀerent. The trick is to write $\frac{d}{d x} \frac{d y}{d z}$ as $\frac{d}{d z} \frac{d z}{d x} (\frac{d y}{d z})$ which does not change anything, but now we can change the order and write this as $\frac{d z}{d x} \frac{d}{d z} (\frac{d y}{d z})$ which now makes the denominator the same and now it is free sailing:

\begin{aligned} \frac{d}{d x} \frac{d y}{d z} & = \frac{d}{d z} \frac{d z}{d x} (\frac{d y}{d z}) \\ = \frac{d z}{d x} \frac{d}{d z} (\frac{d y}{d z}) \\ = \frac{d z}{d x} (\frac{d^{2} y}{d z^{2}}) \end{aligned}

Therefore, the ﬁrst term in (1) becomes

\begin{aligned} (\frac{d}{d x} \frac{d y}{d z}) (\frac{d z}{d x}) & = \frac{d z}{d x} (\frac{d^{2} y}{d z^{2}}) (\frac{d z}{d x}) \\ (3) & = {(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}}) \end{aligned}

Using (2,3) then we have

\frac{d^{2} y}{d x^{2}} = {(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}}) + (\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})

Hence the original ode now becomes

\begin{aligned} \frac{d^{2} y}{d x^{2}} + \frac{d y}{d x} + y & = \sin (x) \\ \overset{y^{''} (x)}{\overset{⏞}{{(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}}) + (\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})}} + \overset{y^{'} (x)}{\overset{⏞}{\frac{d y}{d z} \frac{d z}{d x}}} + y (z) & = \sin (g^{- 1} (z)) \end{aligned}

We could have written the RHS above as just $\sin (x)$ instead of $\sin (g^{- 1} (z))$ but since the independent variable is now $z$ , this seemed better to do it this way. But both are correct. Now, since $z = g (x)$ the above can also be written as

\begin{aligned} {(\frac{d g}{d x})}^{2} (\frac{d^{2} y}{d z^{2}}) + (\frac{d y}{d z}) (\frac{d^{2} g}{d x^{2}}) + \frac{d y}{d z} \frac{d g}{d x} + y (z) & = \sin (g^{- 1} (z)) \\ {(g^{'} (x))}^{2} y^{''} (x) + y^{'} (z) g^{''} (x) + y^{'} (z) g^{'} (x) + y (z) & = \sin (x) \end{aligned}

OK, since the above was so much fun, lets do third derivative $\frac{d^{3} y}{d x^{3}}$

\begin{aligned} \frac{d^{3} y}{d x^{3}} & = \frac{d}{d x} (\frac{d^{2} y}{d x^{2}}) \\ = \frac{d}{d x} ({(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}}) + (\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})) \\ (4) & = \frac{d}{d x} [{(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}})] + \frac{d}{d x} [(\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})] \end{aligned}

Each term above is now found. Looking at ﬁrst term in (4)

\frac{d}{d x} [{(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}})]

Using the product rule, which is $\frac{d}{d x} (a b) = a^{'} b + a b^{'}$ on the above gives

\frac{d}{d x} [{(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}})] = \frac{d}{d x} [{(\frac{d z}{d x})}^{2}] \frac{d^{2} y}{d z^{2}} + {(\frac{d z}{d x})}^{2} \frac{d}{d x} [\frac{d^{2} y}{d z^{2}}]

But $\frac{d}{d x} [{(\frac{d z}{d x})}^{2}] = 2 \frac{d z}{d x} \frac{d^{2} z}{d x}$ and for $\frac{d}{d x} (\frac{d^{2} y}{d z^{2}})$ we have to use the same trick as before by writing $\frac{d}{d x} (\frac{d^{2} y}{d z^{2}}) = \frac{d}{d z} \frac{d z}{d x} (\frac{d^{2} y}{d z^{2}}) = \frac{d z}{d x} \frac{d}{d z} (\frac{d^{2} y}{d z^{2}})$ and now we have $\frac{d}{d x} (\frac{d^{2} y}{d z^{2}}) = \frac{d z}{d x} \frac{d^{3} y}{d z^{3}}$ . Hence the ﬁrst term in (4) is now done.

\begin{aligned} \frac{d}{d x} [{(\frac{d z}{d x})}^{2} (\frac{d^{2} y}{d z^{2}})] & = 2 \frac{d z}{d x} \frac{d^{2} z}{d x^{2}} \frac{d^{2} y}{d z^{2}} + {(\frac{d z}{d x})}^{2} \frac{d z}{d x} \frac{d^{3} y}{d z^{3}} \\ (5) & = 2 \frac{d z}{d x} \frac{d^{2} z}{d x^{2}} \frac{d^{2} y}{d z^{2}} + {(\frac{d z}{d x})}^{3} \frac{d^{3} y}{d z^{3}} \end{aligned}

Now we look at the second term in (4) which is $\frac{d}{d x} [(\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})]$ and apply the product rule, this gives

\begin{aligned} \frac{d}{d x} [(\frac{d y}{d z}) (\frac{d^{2} z}{d x^{2}})] & = \frac{d}{d x} [\frac{d y}{d z}] (\frac{d^{2} z}{d x^{2}}) + \frac{d y}{d z} \frac{d}{d x} [\frac{d^{2} z}{d x^{2}}] \\ = \frac{d}{d z} \frac{d z}{d x} [\frac{d y}{d z}] (\frac{d^{2} z}{d x^{2}}) + \frac{d y}{d z} \frac{d^{3} z}{d x^{3}} \\ = \frac{d z}{d x} \frac{d}{d z} [\frac{d y}{d z}] (\frac{d^{2} z}{d x^{2}}) + \frac{d y}{d z} \frac{d^{3} z}{d x^{3}} \\ (6) & = \frac{d z}{d x} \frac{d^{2} y}{d z^{2}} (\frac{d^{2} z}{d x^{2}}) + \frac{d y}{d z} \frac{d^{3} z}{d x^{3}} \end{aligned}

That is it. We are done. (5,6) are the two terms in (4). Therefore

\begin{aligned} \frac{d^{3} y}{d x^{3}} & = 2 \frac{d z}{d x} \frac{d^{2} z}{d x^{2}} \frac{d^{2} y}{d z^{2}} + {(\frac{d z}{d x})}^{3} \frac{d^{3} y}{d z^{3}} + \frac{d z}{d x} \frac{d^{2} y}{d z^{2}} (\frac{d^{2} z}{d x^{2}}) + \frac{d y}{d z} \frac{d^{3} z}{d x^{3}} \\ = 3 \frac{d z}{d x} \frac{d^{2} z}{d x^{2}} \frac{d^{2} y}{d z^{2}} + {(\frac{d z}{d x})}^{3} \frac{d^{3} y}{d z^{3}} + \frac{d y}{d z} \frac{d^{3} z}{d x^{3}} \end{aligned}

Now, since $z = g (x)$ the above can also be written as

y^{'''} (x) = 3 g^{'} (x) g^{''} (x) y^{''} (z) + {(g^{'} (x))}^{3} y^{'''} (z) + y^{'} (z) g^{'''} (x)

This table show summary of transformation for each derivative $y^{(n)} (x)$ when using change of variables $z = g (x)$


$y^{'} (x)$	$y^{'} (z) g^{'} (x)$

$y^{''} (x)$	${(g^{'} (x))}^{2} y^{''} (z) + y^{'} (z) g^{''} (x)$

$y^{'''} (x)$	$3 g^{'} (x) g^{''} (x) y^{''} (z) + {(g^{'} (x))}^{3} y^{'''} (z) + y^{'} (z) g^{'''} (x)$

$y^{''''} (x)$	$3 {(g^{''} (x))}^{2} y^{''} (z) + 4 g^{'} (x) y^{''} (z) g^{'''} (x) + 6 {(g^{'} (x))}^{2} g^{''} (x) y^{'''} (z) + y^{'} (z) g^{''''} (x) + {(g^{'} (x))}^{4} y^{''''} (z)$

Strictly speaking, it would be better to use diﬀerent variable than $y$ when changing the independent variable. i.e. instead of writing $y (z)$ in all the above, we should write $u (z)$ in its place. So the above table will look like


$y^{'} (x)$	$u^{'} (z) g^{'} (x)$

$y^{''} (x)$	${(g^{'} (x))}^{2} u^{''} (z) + u^{'} (z) g^{''} (x)$

$y^{'''} (x)$	$3 g^{'} (x) g^{''} (x) u^{''} (z) + {(g^{'} (x))}^{3} u^{'''} (z) + u^{'} (z) g^{'''} (x)$

$y^{''''} (x)$	$3 {(g^{''} (x))}^{2} u^{''} (z) + 4 g^{'} (x) u^{''} (z) g^{'''} (x) + 6 {(g^{'} (x))}^{2} g^{''} (x) u^{'''} (z) + y^{'} (z) g^{''''} (x) + {(g^{'} (x))}^{4} u^{''''} (z)$

So any place where $y (z)$ shows in the transformed expression, it should be written with new letter for the dependent variable $u (z)$ . But this is not always enforced.

3.2 Example 2 Change of the independent variable using $t = \ln (x)$ Euler ode

Given the ode

x^{2} \frac{d^{2} y}{d x^{2}} + 2 x \frac{d y}{d x} + y = 0

And asked to do change of variable $t = \ln (x)$

\begin{aligned} \frac{d y}{d x} & = \frac{d y}{d t} \frac{d t}{d x} \\ = \frac{d y}{d t} \frac{1}{x} \end{aligned}

And

\begin{aligned} \frac{d^{2} y}{d x^{2}} & = \frac{d}{d x} (\frac{d y}{d x}) \\ = \frac{d}{d x} (\frac{d y}{d t} \frac{1}{x}) \\ = \frac{d}{d x} [\frac{d y}{d t}] \frac{1}{x} + \frac{d y}{d t} \frac{d}{d x} (\frac{1}{x}) \\ = \frac{d}{d t} \frac{d t}{d x} [\frac{d y}{d t}] \frac{1}{x} - \frac{d y}{d t} \frac{1}{x^{2}} \\ = \frac{d t}{d x} \frac{d^{2} y}{d t^{2}} \frac{1}{x} - \frac{d y}{d t} \frac{1}{x^{2}} \\ = \frac{1}{x} \frac{d^{2} y}{d t^{2}} \frac{1}{x} - \frac{d y}{d t} \frac{1}{x^{2}} \\ = \frac{1}{x^{2}} \frac{d^{2} y}{d t^{2}} - \frac{d y}{d t} \frac{1}{x^{2}} \end{aligned}

Hence the original ode becomes

\begin{aligned} x^{2} (\frac{1}{x^{2}} \frac{d^{2} y}{d t^{2}} - \frac{d y}{d t} \frac{1}{x^{2}}) + 2 x (\frac{d y}{d t} \frac{1}{x}) + y & = 0 \\ \frac{d^{2} y}{d t^{2}} - \frac{d y}{d t} + 2 \frac{d y}{d t} + y & = 0 \\ \frac{d^{2} y}{d t^{2}} + \frac{d y}{d t} + y & = 0 \end{aligned}

3.3 Example 3 Change of the dependent variable using $y = x^{r}$ Euler ode

Given the ode

x^{2} \frac{d^{2} y}{d x^{2}} + 2 x \frac{d y}{d x} + y = 0

And asked to do change of variable $y = x^{r}$

\frac{d y}{d x} = r x^{r - 1}

And

\begin{aligned} \frac{d^{2} y}{d x^{2}} & = \frac{d}{d x} (r x^{r - 1}) \\ = r (r - 1) x^{r - 2} \end{aligned}

Hence the original ode becomes

\begin{aligned} x^{2} (r (r - 1) x^{r - 2}) + 2 x (r x^{r - 1}) + x^{r} & = 0 \\ r (r - 1) x^{r} + 2 r x^{r} + x^{r} & = 0 \\ r (r - 1) + 2 r + 1 & = 0 \end{aligned}

Solving for $r$ gives the roots. Hence solutions are $y_{1} = x^{r_{1}}$ and $y_{2} = x^{r_{2}}$ . Final solution is therefore

\begin{aligned} y & = c_{1} y_{1} + c_{2} y_{2} \\ = c_{1} x^{r_{1}} + c_{2} x^{r_{2}} \end{aligned}

This method of solving the Euler ode is much simpler than using $t = \ln (x)$ change of variables but for some reason most text books use the later one.

4 Changing the role of independent and dependent variable in an ode

(added Dec 14, 2024).

Given an ode $y^{'} (x) = f (x, y)$ , we want to change it so that instead of $y (x)$ being the dependent variable, to make the ode so that $x (y)$ is the dependent variable. For example, given the ode

\frac{d^{2} y}{d x^{2}} = \frac{d y}{d x} e^{y (x)}

The new ode becomes

\frac{d^{2} x}{d y^{2}} = - {(\frac{d x}{d y})}^{2} e^{y}

Which is easier to solve for $x (y)$ . Once solved, we ﬂip back and ﬁnd $y$ from the solution. Sometimes this trick can make solving hard ode very easy. It also can make solving easy ode very hard. Only way to ﬁnd out, is to try it. So if we have an ode that we are having hard time solving, we can try this trick.

For ﬁrst order ode, the method is easy. We just isolate $\frac{d y}{d x}$ and then ﬂip the left hand side and ﬂip the right hand side, and change all $y (x)$ to just $y$ and change all the $x$ to $x (y)$ .

More formally, this can also be done using change of variables, like this. The ﬁrst step is to do change of variables.

\begin{aligned} x & = v (t) \\ y & = t \end{aligned}

If we carry the above change of variables, the new ode will be in terms of $v (t), v^{'} (t)$ and so on.

Now we replace all the $v^{(n)} (t)$ with $x^{(n)} (y)$ where $n$ here is the order of derivative. And replace any $t$ by $y$ (not $y (x)$ but just $y$ ). And replace any $v (t)$ by $x$ . The new ode will be the ﬂipped ode.

When we do the above change of variables using chain rule, these will result in the following

\begin{aligned} \frac{d y}{d x} & \to \frac{1}{\frac{d v}{d t}} = \frac{d t}{d v} \\ \frac{d^{2} y}{d x^{2}} & \to - \frac{\frac{d^{2} v}{d t^{2}}}{{(\frac{d v}{d t})}^{3}} \\ \frac{d^{3} y}{d x^{3}} & \to \frac{\frac{2 {(\frac{d^{2} v}{d t^{2}})}^{2}}{{(\frac{d v}{d t})}^{2}} - \frac{\frac{d^{3} v}{d t^{3}}}{{(\frac{d v}{d t})}^{3}}}{\frac{d v}{d t}} \end{aligned}

And so on. Once the above is done, the rest is easy. We just replace any $\frac{d v}{d t}$ by $\frac{d x}{d y}$ and any $t$ by $y$ and any $v$ by $x$ . We will not do change roles for ode higher than two in these examples.

4.1 Example 1

Change the role for the ode

\frac{d y}{d x} = x

This has solution

\begin{matrix} (1) & y (x) = \frac{1}{2} x^{2} + c_{1} \end{matrix}

Since this is ﬁrst order, we can do it the easy way without change of variable. Flip the left side and ﬂip the right side and do the renaming

\frac{d x}{d y} = \frac{1}{x}

If we want to do it via change of variables, the method is: Let

\begin{aligned} x & = v (t) \\ y & = t \end{aligned}

Then

\frac{d y}{d x} = \frac{d y}{d t} \frac{d t}{d v} \frac{d v}{d x}

But $\frac{d y}{d t} = 1$ and the above becomes

\frac{d y}{d x} = \frac{d t}{d v} \frac{d v}{d x}

And $\frac{d v}{d x} = 1$ and the above becomes

\frac{d y}{d x} = \frac{d t}{d v}

Hence ode becomes

\begin{aligned} \frac{d t}{d v} & = v (t) \\ \frac{d v}{d t} & = \frac{1}{v} \end{aligned}

Now we replace $v^{'} (t)$ by $x^{'} (y)$ and $v$ by $x$ The above becomes (which is the ﬂipped ode)

\frac{d x}{d y} = \frac{1}{x}

Solving for $x (y)$ gives

\begin{aligned} x_{1} & = \sqrt{2 y + c_{1}} \\ x_{2} & = - \sqrt{2 y + c_{1}} \end{aligned}

Lets take the ﬁrst solution and solve for $y$ , this gives

\begin{aligned} x^{2} & = 2 y + c_{1} \\ y & = \frac{1}{2} x^{2} - \frac{1}{2} c_{1} \\ (2) & = \frac{1}{2} x^{2} + c_{1} \end{aligned}

Which is the same as (1). Of course, in this example there is no point of changing the roles, but this was just an example.

4.2 Example 2

Change the role for the ode

\frac{d y}{d x} = e^{y}

This has solution

\begin{matrix} (1) & y (x) = \ln (\frac{- 1}{x + c_{1}}) \end{matrix}

Since this is ﬁrst order, we will do it the easy way. Flip the left side and ﬂip the right side and do the renaming. This gives

\frac{d x}{d y} = e^{- y}

Solving this gives

x = - e^{- y} + c_{1}

Solving for $y$ gives

\begin{aligned} - x + c_{1} & = e^{- y} \\ \ln (- x + c_{1}) & = - y \\ y & = - \ln (- x + c_{1}) \\ = \ln (\frac{1}{- x + c_{1}}) \\ = \ln (\frac{- 1}{x - c_{1}}) \\ = \ln (\frac{- 1}{x + c_{2}}) \end{aligned}

Which is same as (1).

4.3 Example 3

Change the role for the ode

\begin{matrix} (1) & y \ln y + (x - \ln y) \frac{d y}{d x} = 0 \end{matrix}

Solving the above gives

\begin{aligned} (2) & y_{1} & = e^{x - \sqrt{x^{2} - 2 c_{1}}} \\ y_{1} & = e^{x + \sqrt{x^{2} - 2 c_{1}}} \end{aligned}

Since this is ﬁrst order, we will do it the easy way. First isolate $\frac{d y}{d x}$ then ﬂip the left side and the right side and rename. Solving for $\frac{d y}{d x}$ from (1) gives

\frac{d y}{d x} = \frac{- y \ln y}{x - \ln y}

Flipping

\begin{aligned} \frac{d x}{d y} & = \frac{\ln (y) - x}{y \ln y} \\ = \frac{1}{y} - \frac{x}{y \ln y} \\ (3) & \frac{d x}{d y} + \frac{x}{y \ln y} & = \frac{1}{y} \end{aligned}

In this example, we see that changing roles really paid oﬀ as Eq. (3) is linear ode in $x (y)$ but (1) is very hard to solve for $y (x)$ and needs Lie symmetry to solve it. Solving (3) gives

x = \frac{\ln y}{2} + \frac{c_{1}}{\ln y}

Solving the above for $y$ gives same solutions as (2).

4.4 Example 4

Change the role for the ode

\begin{matrix} (1) & \frac{d^{2} y}{d x^{2}} = \frac{d y}{d x} e^{y (x)} \end{matrix}

This has solution

\begin{matrix} (2) & y (x) = c_{1} c_{2} + c_{1} x + \ln (\frac{- c_{1}}{e^{c_{1} c_{2}} e^{x c_{1}} - 1}) \end{matrix}

Since this is not ﬁrst order, we can not do the easy method as with ﬁrst order and we have to do change of variables since with second derivative it is more complicate. Let

\begin{aligned} x & = v (t) \\ y & = t \end{aligned}

Using the rules gives above, we know that

\begin{aligned} (3) & \frac{d y}{d x} & = \frac{1}{\frac{d v}{d t}} \\ \frac{d^{2} y}{d x^{2}} & = - \frac{\frac{d^{2} v}{d t^{2}}}{{(\frac{d v}{d t})}^{3}} \end{aligned}

Substituting (3) into (1) (and changing at $y (x)$ by $t$ and any $x$ by $v (t)$ ) gives

\begin{aligned} - \frac{\frac{d^{2} v}{d t^{2}}}{{(\frac{d v}{d t})}^{3}} & = \frac{1}{\frac{d v}{d t}} e^{t} \\ - \frac{d^{2} v}{d t^{2}} & = \frac{{(\frac{d v}{d t})}^{3}}{\frac{d v}{d t}} e^{t} \\ - \frac{d^{2} v}{d t^{2}} & = {(\frac{d v}{d t})}^{2} e^{t} \end{aligned}

We now replace each $\frac{d v}{d t}$ by $\frac{d x}{d y}$ and each $t$ by $y$ . The above becomes

\frac{d^{2} x}{d y^{2}} = - {(\frac{d x}{d y})}^{2} e^{y}

And the above is the ﬁnal ﬂipped ode. The solution is

x = - \frac{1}{c_{1}} \ln (e^{y}) + \frac{1}{c_{1}} \ln (e^{y} - c_{1}) + c_{2}

To obtain $y$ as function of $x$ , we just isolate $y$ from the above.

\begin{aligned} c_{1} x & = - \ln (e^{y}) + \ln (e^{y} - c_{1}) + c_{1} c_{2} \\ c_{1} x - c_{1} c_{2} & = \ln (\frac{e^{y} - c_{1}}{e^{y}}) \\ e^{(c_{1} x - c_{1} c_{2})} & = \frac{e^{y} - c_{1}}{e^{y}} \\ e^{(c_{1} x - c_{1} c_{2})} & = 1 - c_{1} e^{- y} \\ 1 - e^{(c_{1} x - c_{1} c_{2})} & = c_{1} e^{- y} \\ \frac{1 - e^{(c_{1} x - c_{1} c_{2})}}{c_{1}} & = e^{- y} \\ - y & = \ln (\frac{1 - e^{(c_{1} x - c_{1} c_{2})}}{c_{1}}) \\ y & = \ln (\frac{c_{1}}{1 - e^{(c_{1} x - c_{1} c_{2})}}) \end{aligned}

Which is the solution to the original ode obtain by ﬁrst ﬂipping the ode.

4.5 Example 5

Change the role for the ode

\begin{matrix} (1) & 1 + x y (1 + x y^{2}) \frac{d y}{d x} = 0 \end{matrix}

As this stands, it is hard to solve as it needed Lie symmetry. The solution is

\begin{aligned} y_{1} & = \frac{1}{x} \sqrt{- x (2 x LambertW (- \frac{1}{2} c_{1} e^{\frac{- 2 x - 1}{2 x}}) + 2 x + 1)} \\ y_{2} & = - \frac{1}{x} \sqrt{- x (2 x LambertW (- \frac{1}{2} c_{1} e^{\frac{- 2 x - 1}{2 x}}) + 2 x + 1)} \end{aligned}

By ﬂipping roles, the ode becomes Bernoulli, which is much easier. Since this is ﬁrst order, we will use the easy method. First we isolate $\frac{d y}{d x}$ from (1) then ﬂip both sides and rename. Solving for $\frac{d y}{d x}$ in (1) gives

\frac{d y}{d x} = \frac{- 1}{x y (1 + x y^{2})}

Flipping and renaming $y (x)$ to $y$ and $x$ to $x (y)$ gives

\frac{d x}{d y} = - x y - x^{2} y^{3}

This is in the form

x^{'} = P x + Q x^{n}

Where $n = 2$ here. Hence Bernoulli, which is easily solved. The solution is

x = \frac{1}{- 2 + c_{1} e^{\frac{y^{2}}{2}} - y^{2}}

The last step is to solve for $y$ as function of $x$ .

\begin{aligned} x (- 2 + c_{1} e^{\frac{y^{2}}{2}} - y^{2}) & = 1 \\ - 2 x + c_{1} x e^{\frac{y^{2}}{2}} - x y^{2} & = 1 \\ c_{1} e^{\frac{y^{2}}{2}} - y^{2} & = \frac{1 + 2 x}{x} \end{aligned}

Solving for $y$ from the above gives same answer as above. This is an example where ﬂipping roles paid oﬀ well. But only way to know is to try it and see.

4.6 Example 6

Change the role for the ode

\begin{matrix} (1) & (1 - 4 x y^{2}) \frac{d y}{d x} = y^{3} \end{matrix}

As this stands, this is homogeneous class G. The solution is

\begin{aligned} y_{1} & = - \frac{1}{2 x} \sqrt{x (1 + \sqrt{16 c_{1} x + 1})} \\ y_{2} & = \frac{1}{2 x} \sqrt{x (1 + \sqrt{16 c_{1} x + 1})} \\ y_{3} & = - \frac{1}{2 x} \sqrt{- x (- 1 + \sqrt{16 c_{1} x + 1})} \\ y_{4} & = \frac{1}{2 x} \sqrt{- x (- 1 + \sqrt{16 c_{1} x + 1})} \end{aligned}

By ﬂipping roles, the ode becomes linear, which is much easier to solve. Since this is ﬁrst order, we will use the easy method. First we isolate $\frac{d y}{d x}$ from (1) then ﬂip both sides and rename. Solving for $\frac{d y}{d x}$ in (1) gives

\frac{d y}{d x} = \frac{y^{3}}{1 - 4 x y^{2}}

Flipping and renaming $y (x)$ to $y$ and $x$ to $x (y)$ gives

\begin{aligned} \frac{d x}{d y} & = \frac{1 - 4 x y^{2}}{y^{3}} \\ = \frac{1}{y^{3}} - 4 \frac{x}{y} \end{aligned}

\frac{d x}{d y} + \frac{4}{y} x = \frac{1}{y^{3}}

Which is linear ode in $x (y)$ . Solving gives

x = \frac{1}{y^{4}} (\frac{y^{2}}{2} + c_{1})

The last step is to solve for $y$ which will give same solution as above.

4.7 Example 7

Change the role for the ode

\begin{matrix} (1) & \frac{d y}{d x} = \frac{x}{y^{2} x^{2} + y^{5}} \end{matrix}

As this stands, this can be solved using Lie symmetry or as an exact ode but with an integrating factor that needs to be found ﬁrst, The solution is

y_{1} = \frac{1}{2} {(- 8 x^{2} - 12 LambertW (\frac{4}{3} c_{1} e^{- \frac{2}{3} x^{2} - 1}) - 12)}^{\frac{1}{3}}

And 2 more (too long to type). By ﬂipping roles the new ode becomes

\begin{aligned} \frac{d x}{d y} & = \frac{y^{2} x^{2} + y^{5}}{x} \\ = x y^{2} + y^{5} x^{- 1} \end{aligned}

This has form

x^{'} = P (y) x + Q (y) x^{n}

Which is Bernoulli ode. Which is simpler to solve solve. Solving gives

\begin{aligned} x & = - \frac{1}{2} \sqrt{- 6 - 4 y^{3} + c_{1} 4 e^{\frac{2}{3} y^{3}}} \\ x & = \frac{1}{2} \sqrt{- 6 - 4 y^{3} + c_{1} 4 e^{\frac{2}{3} y^{3}}} \end{aligned}

Finally, we solve for $y$ from the above. This will give same solutions as above.

5 general notes

$◼$ Some rules to remember. This is in the real domain

$\sqrt{a b} = \sqrt{a} \sqrt{b}$ only for $a \geq 0, b \geq 0$ . In general ${(a b)}^{\frac{1}{n}} = a^{\frac{1}{n}} b^{\frac{1}{n}}$ for $a \geq 0, b \geq 0$ where $n$ is positive integer.
$\sqrt{y} = x$ implies $y = x^{2}$ only when $x > 0$ . So be careful when squaring both sides to get rid of sqrt root on one side. To see this, let $\sqrt{y} = 4$ then $y = 16$ because $4$ is positive. But if we had $\sqrt{y} = - 4$ then we can’t say that $y = 16$ since $\sqrt{16}$ is $4$ and not $- 4$ . (we always take the positive root). So each time we square both sides of equation to get rid of $\sqrt{}$ on one side, always say this is valid when the other side is not negative.
Generalization of the above: given ${(a b)}^{\frac{n}{m}}$ where both $n, m$ integers then ${(a b)}^{\frac{n}{m}} = a^{\frac{n}{m}} b^{\frac{n}{m}}$ only when $a \geq 0, b \geq 0$ . This applies if $\frac{n}{m} < 1$ such as $\frac{2}{3}$ or when $\frac{n}{m} > 1$ such as $\frac{3}{2}$ . Only time we can write ${(a b)}^{n} = a^{n} b^{n}$ for any $a, b$ is when $n$ is an integer (positive or negative). When the power is ratio of integers, then was can split it only under the condition that all terms are positive.
$\sqrt{\frac{1}{b}} = \frac{1}{\sqrt{b}}$ only for $b > 0$ . This can be used for example to simplify $\sqrt{\frac{1}{1 - x^{2}}} \sqrt{1 - x^{2}}$ to $1$ under the condition $1 - x^{2} > 0$ or $- 1 < x < 1$ . Because in this case the input becomes $\frac{1}{\sqrt{1 - x^{2}}} \sqrt{1 - x^{2}} = 1$ .
Generalization of the above: $\sqrt{\frac{a}{b}} = \frac{\sqrt{a}}{\sqrt{b}}$ only for $a \geq 0, b > 0$
$\sqrt{x^{2}} = x$ only for $x \geq 0$
Generalization of the above: ${(x^{n})}^{\frac{1}{n}} = x$ only when $x \geq 0$ (assuming $n$ is integer).

$◼$ Given $u \equiv u (x, y)$ then total diﬀerential of $u$ is

d u = \frac{\partial u}{\partial x} d x + \frac{\partial u}{\partial y} d y

$◼$ Lyapunov function is used to determine stability of an equilibrium point. Taking this equilibrium point to be zero, and someone gives us a set of diﬀerential equations $(\begin{matrix} x^{'} (t) \\ y^{'} (t) \\ z^{'} (t) \end{matrix}) = (\begin{matrix} f_{1} (x, y, z, t) \\ f_{2} (x, y, z, t) \\ f_{2} (x, y, z, t) \end{matrix})$ and assuming $(0, 0, 0)$ is an equilibrium point. The question is, how to determine if it stable or not? There are two main ways to do this. One by linearization of the system around origin. This means we ﬁnd the Jacobian matrix, evaluate it at origin, and check the sign of the real parts of the eigenvalues. This is the common way to do this. Another method, called Lyapunov, is more direct. There is no linearization needed. But we need to do the following. We need to ﬁnd a function $V (x, y, z)$ which is called Lyapunov function for the system which meets the following conditions

$V (x . y, z)$ is continuously diﬀerentiable function in $R^{3}$ and $V (x . y, z) \geq 0$ (positive deﬁnite or positive semideﬁnite) for all $x, y, z$ away from the origin, or everywhere inside some ﬁxed region around the origin. This function represents the total energy of the system (For Hamiltonian systems). Hence $V (x, y, z)$ can be zero away from the origin. But it could never be negative.
$V (0, 0, 0) = 0$ . This says the system has no energy when it is at the equilibrium point. (rest state).
The orbital derivative $\frac{d V}{d t} \leq 0$ (i.e. negative deﬁnite or negative semi-deﬁnite) for all $x, y, z$ , or inside some ﬁxed region around the origin. The orbital derivative is same as $\frac{d V}{d t}$ along any solution trajectory. This condition says that the total energy is either constant in time (the zero case) or the total energy is decreasing in time (the negative deﬁnite case). Both of which indicate that the origin is a stable equilibrium point.

If $\frac{d V}{d t}$ is negative semi-deﬁnite then the origin is stable in Lyapunov sense. If $\frac{d V}{d t}$ is negative deﬁnite then the origin is asymptotically stable equilibrium. Negative semi-deﬁnite means the system, when perturbed away from the origin, a trajectory will remain around the origin since its energy do not increase nor decrease. So it is stable. But asymptotically stable equilibrium is a stronger stability. It means when perturbed from the origin the solution will eventually return back to the origin since the energy is decreasing. Global stability means $\frac{d V}{d t} \leq 0$ everywhere, and not just in some closed region around the origin. Local stability means $\frac{d V}{d t} \leq 0$ in some closed region around the origin. Global stability is stronger stability than local stability.

Main diﬃculty with this method is to ﬁnd $V (x . y, z)$ . If the system is Hamiltonian, then $V$ is the same as total energy. Otherwise, one will guess. Typically a quadratic function such as $V = a x^{2} + c x y + d y^{2}$ is used (for system in $x, y$ ) then we try to ﬁnd $a, c, d$ which makes it positive deﬁnite everywhere away from origin, and also more importantly makes $\frac{d V}{d t} \leq 0$ . If so, we say origin is stable. Most of the problems we had starts by giving us $V$ and then asks to show it is Lyapunov function and what kind of stability it is.

To determine if $V$ is positive deﬁnite or not, the common way is to ﬁnd the Hessian and check the sign of the eigenvalues. Another way is to ﬁnd the Hessian and check the sign of the minors. For $2 \times 2$ matrix, this means the determinant is positive and the entry $(1, 1)$ in the matrix is positive. Similar thing to check if $\frac{d V}{d t} \leq 0$ . We ﬁnd the Hessian of $\frac{d V}{d t}$ and do the same thing. But now we check for negative eigenvalues instead.

$◼$ Methods to ﬁnd Green function are

Fredholm theory
methods of images
separation of variables
Laplace transform

reference Wikipedia I need to make one example and apply each of the above methods on it.

$◼$ In solving an ODE with constant coeﬃcient just use the characteristic equation to solve the solution.

$◼$ In solving an ODE with coeﬃcients that are functions that depends on the independent variable, as in $y^{''} (x) + q (x) y^{'} (x) + p (x) y (x) = 0$ , ﬁrst classify the point $x_{0}$ type. This means to check how $p (x)$ and $q (x)$ behaves at $x_{0}$ . We are talking about the ODE here, not the solution yet.

There are 3 kinds of points. $x_{0}$ can be normal, or regular singular point, or irregular singular point. Normal point $x_{0}$ means $p (x)$ and $q (x)$ have Taylor series expansion $y (x) = \sum_{n = 0}^{\infty} a_{n} {(x - x_{0})}^{n}$ that converges to $y (x)$ at $x_{0}$ .
Regular singular point $x_{0}$ means that the above test fails, but $lim_{x \to x_{0}} (x - x_{0}) q (x)$ has a convergent Taylor series, and also that $lim_{x \to x_{0}} {(x - x_{0})}^{2} p (x)$ now has a convergent Taylor series at $x_{0}$ . This also means the limit exist.

All this just means we can get rid of the singularity. i.e. $x_{0}$ is a removable singularity. If this is the case, then the solution at $x_{0}$ can be assumed to have a Frobenius series $y (x) = \sum_{n = 0}^{\infty} a_{n} {(x - x_{0})}^{n + α}$ where $a_{0} \neq 0$ and $α$ is the root of the Frobenius indicial equation. There are three cases to consider. See https://math.usask.ca/~cheviakov/courses/m338/text/Frobenius_Case3_ill.pdf for more discussion on this.

The third type of point, is the hard one. Called irregular singular point. We can’t get rid of it using the above. So we also say the ODE has an essential singularity at $x_{0}$ (another fancy name for irregular singular point). What this means is that we can’t approximate the solution at $x_{0}$ using either Taylor nor Frobenius series.

If the point is an irregular singular point, then use the methods of asymptotic. See advanced mathematical methods for scientists and engineers chapter 3. For normal point, use $y (x) = \sum_{n = 0}^{\infty} a_{n} x^{n}$ , for regular singular point use $y (x) = \sum_{n = 0}^{\infty} a_{n} x^{n + r}$ . Remember, to solve for $r$ ﬁrst. This should give two values. If you get one root, then use reduction of order to ﬁnd second solution.

$◼$ Asymptotic series $S (z) = c_{0} + \frac{c_{1}}{z} + \frac{c_{2}}{z^{2}} + \dots$ is series expansion of $f (z)$ which gives good and rapid approximation for large $z$ as long as we know when to truncate $S (z)$ before it becomes divergent. This is the main diﬀerence Asymptotic series expansion and Taylor series expansion.

$S (z)$ is used to approximate a function for large $z$ while Taylor (or power series) is used for local approximation or for small distance away from the point of expansion. $S (z)$ will become divergent, hence it needs to be truncated at some $n$ to use, where $n$ is the number of terms in $S_{n} (z)$ . It is optimally truncated when $n \approx {| z |}^{2}$ .

$S (x)$ has the following two important properties

$lim_{| z | \to \infty} z^{n} (f (z) - S_{n} (z)) = 0$ for ﬁxed $n$ .
$lim_{n \to \infty} z^{n} (f (z) - S_{n} (z)) = \infty$ for ﬁxed $z$ .

We write $S (z) \sim f (z)$ when $S (z)$ is the asymptotic series expansion of $f (z)$ for large $z$ . Most common method to ﬁnd $S (z)$ is by integration by parts. At least this is what we did in the class I took.

$◼$ For Taylor series, leading behavior is $a_{0}$ no controlling factor? For Frobenius series, leading behavior term is $a_{0} x^{α}$ and controlling factor is $x^{α}$ . For asymptotic series, controlling factor is assumed to be $e^{S (x)}$ always. proposed by Carlini (1817)

$◼$ Method to ﬁnd the leading behavior of the solution $y (x)$ near irregular singular point using asymptotic is called the dominant balance method.

$◼$ When solving $ϵ y^{''} + p (x) y^{'} + q (x) y = 0$ for very small $ϵ$ then use WKB method, if there is no boundary layer between the boundary conditions. If the ODE non-linear, can’t use WKB, has to use boundary layer (B.L.). Example $ϵ y^{''} + y y^{'} - y = 0$ with $y (0) = 0, y (1) = - 2$ then use BL.

$◼$ good exercise is to solve say $ϵ y^{''} + (1 + x) y^{'} + y = 0$ with $y (0) = y (1)$ using both B.L. and WKB and compare the solutions, they should come out the same. $y \sim \frac{2}{1 + x} - \exp (\frac{- x}{ϵ} - \frac{x^{2}}{2 ϵ}) + O (ϵ) .$ with BL had to do the matching between the outer and the inner solutions. WKB is easier. But can’t use it for non-linear ODE.

$◼$ When there is rapid oscillation over the entire domain, WKB is better. Use WKB to solve Schrodinger equation where $ϵ$ becomes function of $ℏ$ (Planck’s constant, $6.62606957 \times 10^{- 34}$ m $^{2}$ kg/s)

$◼$ In second order ODE with non constant coeﬃcient, $y^{''} (x) + p (x) y^{'} (x) + q (x) y (x) = 0$ , if we know one solution $y_{1} (x)$ , then a method called the reduction of order can be used to ﬁnd the second solution $y_{2} (x)$ . Write $y_{2} (x) = u (x) y_{1} (x)$ , plug this in the ODE, and solve for $u (x)$ . The ﬁnal solution will be $y (x) = c_{1} y_{1} (x) + c_{2} y_{2} (x)$ . Now apply I.C.’s to ﬁnd $c_{1}, c_{2}$ .

$◼$ To ﬁnd particular solution to $y^{''} (x) + p (x) y^{'} (x) + q (x) y (x) = f (x)$ , we can use a method called undetermined coeﬃcients. But a better method is called variation of parameters, In this method, assume $y_{p} (x) = u_{1} (x) y_{1} (x) + u_{2} (x) y_{2} (x)$ where $y_{1} (x), y_{2} (x)$ are the two linearly independent solutions of the homogeneous ODE and $u_{1} (x), u_{2} (x)$ are to be determined. This ends up with $u_{1} (x) = - \int \frac{y_{2} (x) f (x)}{W} d x$ and $u_{2} (x) = \int \frac{y_{1} (x) f (x)}{W} d x$ . Remember to put the ODE in standard form ﬁrst, so $a = 1$ , i.e. $a y^{''} (x) + \dots$ . In here, $W$ is the Wronskian $W = | \begin{matrix} y_{1} (x) & y_{2} (x) \\ y_{1}^{'} (x) & y_{2}^{'} (x) \end{matrix} |$

$◼$ Two solutions of $y^{''} (x) + p (x) y^{'} (x) + q (x) y (x) = 0$ are linearly independent if $W (x) \neq 0$ , where $W$ is the Wronskian.

$◼$ For second order linear ODE deﬁned over the whole real line, the Wronskian is either always zero, or not zero. This comes from Abel formula for Wronskian, which is $W (x) = k \exp (- \int \frac{B (x)}{A (x)} d x)$ for ODE of form $A (x) y^{''} + B (x) y^{'} + C (x) y = 0$ . Since $\exp (- \int \frac{B (x)}{A (x)} d x) > 0$ , then it is decided by $k$ . The constant of integration. If $k = 0$ , then $W (x) = 0$ everywhere, else it is not zero everywhere.

$◼$ For linear PDE, if boundary condition are time dependent, can not use separation of variables. Try Transform method (Laplace or Fourier) to solve the PDE.

$◼$ If unable to invert Laplace analytically, try numerical inversion or asymptotic methods. Need to ﬁnd example of this.

$◼$ Green function takes the homogeneous solution and the forcing function and constructs a particular solution. For PDE’s, we always want a symmetric Green’s function.

$◼$ To get a symmetric Green’s function given an ODE, start by converting the ODE to a Sturm-Liouville form ﬁrst. This way the Green’s function comes out symmetric.

$◼$ For numerical solutions of ﬁeld problems, there are basically two diﬀerent problems: Those with closed boundaries and those with open boundaries but with initial conditions. Closed boundaries are elliptical problems which can be cast in the form $A u = f$ , and the other are either hyperbolic or parabolic.

$◼$ For numerical solution of elliptical problems, the basic layout is something like this:

Always start with trial solution $u (x)$ such that $u_{t r i a l} (x) = \sum_{i = 0}^{i = N} C_{i} ϕ_{i} (x)$ where the $C_{i}$ are the unknowns to be determined and the $ϕ_{i}$ are set of linearly independent functions (polynomials) in $x$ .

How to determine those $C_{i}$ comes next. Use either residual method (Galerkin) or variational methods (Ritz). For residual, we make a function based on the error $R = A - u_{t r i a l} f$ . It all comes down to solving $\int f (R) = 0$ over the domain. This is a picture

| 
+---------------+-------------------------------------+ 
|                                                     | 
residual                       Variational (sub u_trial in I(u) 
|                          where I(u) is functional to minimize. 
| 
+----------------+-------------+----------+ 
|                |             |          | 
Absolute error   collocation   subdomain orthogonality 
.... 
+----------------------+------------+ 
|                      |            | 
method of moments   Galerkin     least squares

$◼$ Geometric probability distribution. Use when you want an answer to the question: What is the probability you have to do the experiment $N$ times to ﬁnally get the output you are looking for, given that a probability of $p$ showing up from doing one experiment.

For example: What is the probability one has to ﬂip a fair coin $N$ times to get a head? The answer is $P (X = N) = (1 - p)^{k - 1} p$ . So for a fair coin, $p = \frac{1}{2}$ that a head will show up from one ﬂip. So the probability we have to ﬂip a coin $10$ times to get a head is $P (X = 10) = (1 - 0.5)^{9} (0.5) = 0.00097$ which is very low as expected.

$◼$ To generate random variable drawn from some distribution diﬀerent from uniform distribution, by only using uniform distribution $U (0, 1)$ do this: Lets say we want to generate random number from exponential distribution with mean $μ$ .

This distribution has $p d f (X) = \frac{1}{μ} e^{\frac{- x}{μ}}$ , the ﬁrst step is to ﬁnd the cdf of exponential distribution, which is known to be $F (x) = P (X <= x) = 1 - e^{\frac{- x}{μ}}$ .

Now ﬁnd the inverse of this, which is $F^{- 1} (x) = - μ \ln (1 - x)$ . Then generate a random number from the uniform distribution $U (0, 1)$ . Let this value be called $z$ .

Now plug this value into $F^{- 1} (z)$ , this gives a random number from exponential distribution, which will be $- μ \ln (1 - z)$ (take the natural log of both side of $F (x)$ ).

This method can be used to generate random variables from any other distribution by knowing on $U (0, 1)$ . But it requires knowing the CDF and the inverse of the CDF for the other distribution. This is called the inverse CDF method. Another method is called the rejection method

$◼$ Given $u$ , a r.v. from uniform distribution over [0,1], then to obtain $v$ , a r.v. from uniform distribution over [A,B], then the relation is $v = A + (B - A) u$ .

$◼$ When solving using F.E.M. is best to do everything using isoparametric element (natural coordinates), then ﬁnd the Jacobian of transformation between the natural and physical coordinates to evaluate the integrals needed. For the force function, using Gaussian quadrature method.

$◼$ A solution to diﬀerential equation is a function that can be expressed as a convergent series. (Cauchy. Briot and Bouquet, Picard)

$◼$ To solve a ﬁrst order ODE using integrating factor.

x^{'} (t) + p (t) x (t) = f (t)

then as long as it is linear and $p (t), f (t)$ are integrable functions in $t$ , then follow these steps

multiply the ODE by function $I (t)$ , this is called the integrating factor.
$I (t) x^{'} (t) + I (t) p (t) x (t) = I (t) f (t)$
We solve for $I (t)$ such that the left side satisﬁes
$\frac{d}{d t} (I (t) x (t)) = I (t) x^{'} (t) + I (t) p (t) x (t)$
Solving the above for $I (t)$ gives
$\begin{aligned} I^{'} (t) x (t) + I (t) x^{'} (t) & = I (t) x^{'} (t) + I (t) p (t) x (t) \\ I^{'} (t) x (t) & = I (t) p (t) x (t) \\ I^{'} (t) & = I (t) p (t) \\ \frac{d I}{I} & = p (t) d t \end{aligned}$

Integrating both sides gives
$\begin{aligned} \ln (I) & = \int p (t) d t \\ I (t) & = e^{\int p (t) d t} \end{aligned}$
Now equation (1) can be written as
$\frac{d}{d t} (I (t) x (t)) = I (t) f (t)$
We now integrate the above to give
$\begin{aligned} I (t) x (t) & = \int I (t) f (t) d t + C \\ x (t) & = \frac{\int I (t) f (t) d t + C}{I (t)} \end{aligned}$

Where $I (t)$ is given by (2). Hence
$x (t) = \frac{\int e^{\int p (t) d t} f (t) d t + C}{e^{\int p (t) d t}}$
$◼$ A polynomial is called ill-conditioned if we make small change to one of its coeﬃcients and this causes large change to one of its roots.

$◼$ To ﬁnd rank of matrix $A$ by hand, ﬁnd the row echelon form, then count how many zero rows there are. subtract that from number of rows, i.e. $n$ .

$◼$ To ﬁnd the basis of the column space of $A$ , ﬁnd the row echelon form and pick the columns with the pivots, there are the basis (the linearly independent columns of $A$ ).

$◼$ For symmetric matrix $A$ , its second norm is its spectral radius $ρ (A)$ which is the largest eigenvalue of $A$ (in absolute terms).

$◼$ The eigenvalues of the inverse of matrix $A$ is the inverse of the eigenvalues of $A$ .

$◼$ If matrix $A$ of order $n \times n$ , and it has $n$ distinct eigenvalues, then it can be diagonalized $A = V Λ V^{- 1}$ , where

Λ = (\begin{matrix} e^{λ_{1}} & 0 & 0 \\ 0 & ⋱ & 0 \\ 0 & 0 & e^{λ n} \end{matrix})

and $V$ is matrix that has the $n$ eigenvectors as its columns.

$◼$ $lim_{k \to \infty} \int_{x_{1}}^{x_{2}} f_{k} (x) d x = \int_{x_{1}}^{x_{2}} lim_{k \to \infty} f_{k} (x) d x$ only if $f_{k} (x)$ converges uniformly over $[x_{1}, x_{2}]$ .

$◼$ $A^{3} = I$ , has inﬁnite number of $A$ solutions. Think of $A^{3}$ as 3 rotations, each of $120^{0}$ , going back to where we started. Each rotation around a straight line. Hence inﬁnite number of solutions.

$◼$ How to integrate $I = \int \frac{\sqrt{x^{3} - 1}}{x} d x$ .

Let $u = x^{3} + 1$ , then $d u = 3 x^{2} d x$ and the above becomes

I = \int \frac{\sqrt{u}}{3 x^{3}} d u = \frac{1}{3} \int \frac{\sqrt{u}}{u - 1} d u

Now let $u = \tan^{2} v$ or $\sqrt{u} = \tan v$ , hence $\frac{1}{2} \frac{1}{\sqrt{u}} d u = \sec^{2} v d v$ and the above becomes

\begin{aligned} I & = \frac{1}{3} \int \frac{\sqrt{u}}{\tan^{2} v - 1} (2 \sqrt{u} \sec^{2} v) d v \\ = \frac{2}{3} \int \frac{u}{\tan^{2} v - 1} \sec^{2} v d v \\ = \frac{2}{3} \int \frac{\tan^{2} v}{\tan^{2} v - 1} \sec^{2} v d v \end{aligned}

But $\tan^{2} v - 1 = \sec^{2} v$ hence

\begin{aligned} I & = \frac{2}{3} \int \tan^{2} v d v \\ = \frac{2}{3} (\tan v - v) \end{aligned}

Substituting back

I = \frac{2}{3} (\sqrt{u} - \arctan (\sqrt{u}))

Substituting back

I = \frac{2}{3} (\sqrt{x^{3} + 1} - \arctan (\sqrt{x^{3} + 1}))

$◼$ (added Nov. 4, 2015) Made small diagram to help me remember long division terms used.

$◼$ If a linear ODE is equidimensional, as in $a_{n} x^{n} y^{(n)} + a_{n - 1} x^{n - 1} y^{(n 01)} + \dots$ for example $x^{2} y^{''} - 2 y = 0$ then use ansatz $y = x^{r}$ this will give equation in $r$ only. Solve for $r$ and obtain $y_{1} = x^{r_{1}}, y_{2} = x^{r_{2}}$ and the solution will be

y = c_{1} y_{1} + c_{2} y_{2}

For example, for the above ode, the solution is $c_{1} x^{2} + \frac{c_{2}}{x}$ . This ansatz works only if ODE is equidimensional. So can’t use it on $x y^{''} + y = 0$ for example.

If $r$ is multiple root, use $x^{r}, x^{r} \log (x), x^{r} (\log (x))^{2} \dots$ as solutions.

$◼$ for $x^{i}$ , where $i = \sqrt{- 1}$ , write it as $x = e^{\log x}$ hence $x^{i} = e^{i \log x} = \cos (\log x) + i \sin (\log x)$

$◼$ Some integral tricks: $\int \sqrt{a^{2} - x^{2}} d x$ use $x = a \sin θ$ . For $\int \sqrt{a^{2} + x^{2}} d x$ use $x = a \tan θ$ and for $\int \sqrt{x^{2} - a^{2}} d x$ use $x = a \sec θ$ .

$◼$ $y^{''} + x^{n} y = 0$ is called Emden-Fowler form.

$◼$ For second order ODE, boundary value problem, with eigenvalue (Sturm-Liouville), remember that having two boundary conditions is not enough to fully solve it.

One boundary condition is used to ﬁnd the ﬁrst constant of integration, and the second boundary condition is used to ﬁnd the eigenvalues.

We still need another input to ﬁnd the second constant of integration. This is normally done by giving the initial value. This problem happens as part of initial value, boundary value problem. The point is, with boundary value and eigenvalue also present, we need 3 inputs to fully solve it. Two boundary conditions is not enough.

$◼$ If given ODE $y^{''} (x) + p (x) y^{'} (x) + q (x) y (x) = 0$ and we are asked to classify if it is singular at $x = \infty$ , then let $x = \frac{1}{t}$ and check what happens at $t = 0$ . The $\frac{d^{2}}{d x^{2}}$ operator becomes $(2 t^{3} \frac{d}{d t} + t^{4} \frac{d^{2}}{d t^{2}})$ and $\frac{d}{d x}$ operator becomes $- t^{2} \frac{d}{d t}$ . And write the ode now where $t$ is the independent variable, and follow standard operating procedures. i.e. look at $lim_{t \to 0} x p (t)$ and $lim_{t \to 0} x^{2} q (t)$ and see if these are ﬁnite or not. To see how the operator are mapped, always start with $x = \frac{1}{t}$ then write $\frac{d}{d x} = \frac{d}{d t} \frac{d t}{d x}$ and write $\frac{d^{2}}{d x^{2}} = (\frac{d}{d x}) (\frac{d}{d x})$ . For example, $\frac{d}{d x} = - t^{2} \frac{d}{d t}$ and

\begin{aligned} \frac{d^{2}}{d x^{2}} & = (- t^{2} \frac{d}{d t}) (- t^{2} \frac{d}{d t}) \\ = - t^{2} (- 2 t \frac{d}{d t} - t^{2} \frac{d^{2}}{d t^{2}}) \\ = (2 t^{3} \frac{d}{d t} + t^{4} \frac{d^{2}}{d t^{2}}) \end{aligned}

Then the new ODE becomes

\begin{aligned} (2 t^{3} \frac{d}{d t} + t^{4} \frac{d^{2}}{d t^{2}}) y (t) + p (t) (- t^{2} \frac{d}{d t} y (t)) + q (t) y (t) & = 0 \\ t^{4} \frac{d^{2}}{d t^{2}} y + (- t^{2} p (t) + 2 t^{3}) \frac{d}{d t} y + q (t) y & = 0 \\ \frac{d^{2}}{d t^{2}} y + \frac{(- p (t) + 2 t)}{t^{2}} \frac{d}{d t} y + \frac{q (t)}{t^{4}} y & = 0 \end{aligned}

The above is how the ODE will always become after the transformation. Remember to change $p (x)$ to $p (t)$ using $x = \frac{1}{t}$ and same for $q (x)$ . Now the new $p$ is $\frac{(- p (t) + 2 t)}{t^{2}}$ and the new $q$ is $\frac{q (t)}{t^{4}}$ . Then do $lim_{t \to 0} t \frac{(p (t) + 2 t^{3})}{t^{4}}$ and $lim_{t \to 0} t^{2} \frac{q (t)}{t^{4}}$ as before.

$◼$ If the ODE $a (x) y^{''} + b (x) y^{'} + c (x) y = 0$ , and say $0 \leq x \leq 1$ , and there is essential singularity at either end, then use boundary layer or WKB. But Boundary layer method works on non-linear ODE’s (and also on linear ODE) and only if the boundary layer is at end of the domain, i.e. at $x = 0$ or $x = 1$ .

WKB method on the other hand, works only on linear ODE, but the singularity can be any where (i.e. inside the domain). As rule of thumb, if the ODE is linear, use WKB. If the ODE is non-linear, we must use boundary layer.

Another diﬀerence, is that with boundary layer, we need to do matching phase at the interface between the boundary layer and the outer layer in order to ﬁnd the constants of integrations. This can be tricky and is the hardest part of solving using boundary layer.

Using WKB, no matching phase is needed. We apply the boundary conditions to the whole solution obtained. See my HWs for NE 548 for problems solved from Bender and Orszag text book.

$◼$ In numerical, to ﬁnd if a scheme will converge, check that it is stable and also check that if it is consistent.

It could also be conditionally stable, or unconditionally stable, or unstable.

To check it is consistent, this is the same as ﬁnding the LTE (local truncation error) and checking that as the time step and the space step both go to zero, the LTE goes to zero. What is the LTE? You take the scheme and plug in the actual solution in it. An example is better to explain this part. Lets solve $u_{t} = u_{x x}$ . Using forward in time and centered diﬀerence in space, the numerical scheme (explicit) is

U_{j}^{n + 1} = U_{j}^{n} + \frac{k}{h^{2}} (U_{j - 1}^{n} - 2 U_{j}^{n} + U_{j + 1}^{n})

The LTE is the diﬀerence between these two (error)

L T E = U_{j}^{n + 1} - (U_{j}^{n} + \frac{k}{h^{2}} (U_{j - 1}^{n} - 2 U_{j}^{n} + U_{j + 1}^{n}))

Now plug-in $u (t^{n}, x_{j})$ in place of $U_{j}^{n}$ and $u (t^{n} + k, x_{j})$ in place of $U_{j}^{n + 1}$ and plug-in $u (t^{n}, x + h)$ in place of $U_{j + 1}^{n}$ and plug-in $u (t^{n}, x - h)$ in place of $U_{j - 1}^{n}$ in the above. It becomes

\begin{matrix} (1) & L T E = u (t + k, x_{j}) - (u (t^{n}, x_{j}) + \frac{k}{h^{2}} (u (t, x - h) - 2 u (t^{n}, x_{j}) + u (t, x + h))) \end{matrix}

Where in the above $k$ is the time step (also written as $Δ t$ ) and $h$ is the space step size. Now comes the main trick. Expanding the term $u (t^{n} + k, x_{j})$ in Taylor,

\begin{matrix} (2) & u (t^{n} + k, x_{j}) = u (t^{n}, x_{j}) + k {\frac{\partial u}{\partial t} |}_{t^{n}} + \frac{k^{2}}{2} {\frac{\partial^{2} u}{\partial t^{2}} |}_{t^{n}} + O (k^{3}) \end{matrix}

And expanding

\begin{matrix} (3) & u (t^{n}, x_{j} + h) = u (t^{n}, x_{j}) + h {\frac{\partial u}{\partial x} |}_{x_{j}} + \frac{h^{2}}{2} {\frac{\partial^{2} u}{\partial x^{2}} |}_{x_{j}} + O (h^{3}) \end{matrix}

And expanding

\begin{matrix} (4) & u (t^{n}, x_{j} - h) = u (t^{n}, x_{j}) - h {\frac{\partial u}{\partial x} |}_{x_{j}} + \frac{h^{2}}{2} {\frac{\partial^{2} u}{\partial x^{2}} |}_{x_{j}} - O (h^{3}) \end{matrix}

Now plug-in (2,3,4) back into (1). Simplifying, many things drop out, and we should obtain that

L T E = O (k) + O (h^{2})

Which says that $L T E \to 0$ as $h \to 0, k \to 0$ . Hence it is consistent.

To check it is stable, use Von Neumann method for stability. This check if the solution at next time step does not become larger than the solution at the current time step. There can be condition for this. Such as it is stable if $k \leq \frac{h^{2}}{2}$ . This says that using this scheme, it will be stable as long as time step is smaller than $\frac{h^{2}}{2}$ . This makes the time step much smaller than space step.

$◼$ For $a x^{2} + b x + c = 0$ , with roots $α, β$ then the relation between roots and coeﬃcients is

\begin{aligned} α + β & = - \frac{b}{a} \\ α β & = \frac{c}{a} \end{aligned}

$◼$ Leibniz rules for integration

\begin{aligned} \frac{d}{d x} \int_{a (x)}^{b (x)} f (t) d t & = f (b (x)) b^{'} (x) - f (a (x)) a^{'} (x) \\ \frac{d}{d x} \int_{a (x)}^{b (x)} f (t, x) d t & = f (b (x)) b^{'} (x) - f (a (x)) a^{'} (x) + \int_{a (x)}^{b (x)} \frac{\partial}{\partial x} f (t, x) d t \end{aligned}

$◼$ $\int_{a}^{b} f (x) d x = \int_{a}^{b} f (a + b - x) d x$

$◼$ Diﬀerentiable function implies continuous. But continuous does not imply diﬀerentiable. Example is $| x |$ function.

$◼$ Mean curvature being zero is a characteristic of minimal surfaces.

$◼$ How to ﬁnd phase diﬀerence between 2 signals $x_{1} (t), x_{2} (t)$ ? One way is to ﬁnd the DFT of both signals (in Mathematica this is Fourier, in Matlab ﬀt()), then ﬁnd where the bin where peak frequency is located (in either output), then ﬁnd the phase diﬀerence between the 2 bins at that location. Value of DFT at that bin is complex number. Use Arg in Mathematica to ﬁnd its phase. The diﬀerence gives the phase diﬀerence between the original signals in time domain. See https://mathematica.stackexchange.com/questions/11046/how-to-find-the-phase-difference-of-two-sampled-sine-waves for an example.

$◼$ Watch out when squaring both sides of equation. For example, given $y = \sqrt{x}$ . squaring both sides gives $y^{2} = x$ . But this is only true for $y \geq 0$ . Why? Let us take the square root of this in order to get back to the original equation. This gives $\sqrt{y^{2}} = \sqrt{x}$ . And here is the problem, $\sqrt{y^{2}} = y$ only for $y \geq 0$ . Why? Let us assume $y = - 1$ . Then $\sqrt{y^{2}} = \sqrt{{(- 1)}^{2}} = \sqrt{1} = 1$ which is not $- 1$ . So when taking the square of both sides of the equation, remember this condition.

$◼$ do not replace $\sqrt{x^{2}}$ by $x$ , but by $| x |$ , since $x = \sqrt{x^{2}}$ only for non negative $x$ .

$◼$ Given an equation, and we want to solve for $x$ . We can square both sides in order to get rid of sqrt if needed on one side. But be careful. Even though after squaring both sides, the new equation is still true, the solutions of the new equation can introduce extraneous solution that does not satisfy the original equation. Here is an example I saw on the internet which illustrate this. Given $\sqrt{x} = x - 6$ . And we want to solve for $x$ . Squaring both sides gives $x = {(x - 6)}^{2}$ . This has solutions $x = 9, x = 4$ . But only $x = 9$ is valid solution for the original equation before squaring. The solution $x = 4$ is extraneous. So need to check all solutions found after squaring against the original equation, and remove those extraneous one. In summary, if $a^{2} = b^{2}$ then this does not mean that $a = b$ . But if $a = b$ then it means that $a^{2} = b^{2}$ . For example ${(- 5)}^{2} = 5^{2}$ . But $- 5 \neq 5$ .

$◼$ How to ﬁnd Laplace transform of product of two functions?

There is no formula for the Laplace transform of product $f (t) g (t)$ . (But if this was convolution, it is diﬀerent story). But you could always try the deﬁnition and see if you can integrate it. Since $L (f (t)) = \int_{0}^{\infty} e^{- s t} f (t) d t$ then $L (f (t) g (t)) = \int_{0}^{\infty} e^{- s t} f (t) g (t) d t$ . Hence for $f (t) = e^{a t}, g (t) = t$ this becomes

\begin{aligned} L (t e^{a t}) & = \int_{0}^{\infty} e^{- s t} t e^{a t} d t \\ = \int_{0}^{\infty} t e^{- t (s - a)} d t \end{aligned}

Let $s - a \equiv z$ then

\begin{aligned} L (t e^{a t}) & = \int_{0}^{\infty} t e^{- t z} d t \\ = L_{z} (t) \\ = \frac{1}{z^{2}} \\ = \frac{1}{{(s - a)}^{2}} \end{aligned}

Similarly for $f (t) = e^{a t}, g (t) = t^{2}$

\begin{aligned} L (t^{2} e^{a t}) & = \int_{0}^{\infty} e^{- s t} t^{2} e^{a t} d t \\ = \int_{0}^{\infty} t^{2} e^{- t (s - a)} d t \end{aligned}

Let $s - a \equiv z$ then

\begin{aligned} L (t e^{a t}) & = \int_{0}^{\infty} t^{2} e^{- t z} d t \\ = L_{z} (t^{2}) \\ = \frac{2}{z^{3}} \\ = \frac{2}{{(s - a)}^{3}} \end{aligned}

Similarly for $f (t) = e^{a t}, g (t) = t^{3}$

\begin{aligned} L (t^{2} e^{a t}) & = \int_{0}^{\infty} e^{- s t} t^{3} e^{a t} d t \\ = \int_{0}^{\infty} t^{3} e^{- t (s - a)} d t \end{aligned}

Let $s - a \equiv z$ then

\begin{aligned} L (t e^{a t}) & = \int_{0}^{\infty} t^{3} e^{- t z} d t \\ = L_{z} (t^{3}) \\ = \frac{6}{z^{4}} \\ = \frac{6}{{(s - a)}^{4}} \end{aligned}

And so on. Hence we see that for $f (t) = e^{a t}, g (t) = t^{n}$

L (t^{n} e^{a t}) = \frac{n!}{{(s - a)}^{n + 1}}

6 Converting ﬁrst order ODE which is homogeneous to separable ODE

(Added July, 2017).

If the ODE $M (x, y) + N (x, y) \frac{d y}{d x} = 0$ has both $M$ and $N$ homogenous functions of same power, then this ODE can be converted to separable. Here is an example. We want to solve

\begin{matrix} (1) & (x^{3} + 8 x^{2} y) + (4 x y^{2} - y^{3}) y^{'} = 0 \end{matrix}

The above is homogenous in $M, N$ , since the total powers of each term in them is $3$ .

(\overset{3}{\overset{⏞}{x^{3}}} + 8 \overset{3}{\overset{⏞}{x^{2} y}}) + (4 \overset{3}{\overset{⏞}{x y^{2}}} - \overset{3}{\overset{⏞}{y^{3}}}) y^{'} = 0

So we look at each term in $N$ and $M$ and add all the powers on each $x, y$ in them. All powers should add to same value, which is $3$ in this case. Of course $N, M$ should be polynomials for this to work. So one should check that they are polynomials in $x, y$ before starting this process. Once we check $M, N$ are homogeneous, then we let

y = x v

Therefore now

\begin{aligned} M & = x^{3} + 8 x^{2} (x v) \\ (2) & = x^{3} + 8 x^{3} v \end{aligned}

And

\begin{aligned} N & = 4 x {(x v)}^{2} - {(x v)}^{3} \\ (3) & = 4 x^{3} v^{2} - x^{3} v^{3} \end{aligned}

And

\begin{matrix} (4) & y^{'} = v + x v^{'} \end{matrix}

Substituting (3,4,5) into (1) gives

\begin{aligned} (x^{3} + 8 x^{3} v) + (4 x^{3} v^{2} - x^{3} v^{3}) (v + x v^{'}) & = 0 \\ (x^{3} + 8 x^{3} v) + (4 x^{3} v^{3} - x^{3} v^{4}) + (4 x^{4} v^{2} - x^{4} v^{3}) v^{'} & = 0 \end{aligned}

Dividing by $x^{3} \neq 0$ it simpliﬁes to

(1 + 8 v) + (4 v^{3} - v^{4}) + x (4 v^{2} - v^{3}) v^{'} = 0

Which can be written as

\begin{aligned} x (4 v^{2} - v^{3}) v^{'} & = - ((1 + 8 v) + (4 v^{3} - v^{4})) \\ v^{'} & = \frac{- ((1 + 8 v) + (4 v^{3} - v^{4}))}{(4 v^{2} - v^{3})} (\frac{1}{x}) \end{aligned}

We see that it is now separable. We now solve this for $v (x)$ by direct integration of both sides And then using $y = x v$ ﬁnd $y (x)$ .

7 Direct solving of some simple PDE’s

Some simple PDE’s can be solved by direct integration, here are few examples.

Example 1

\frac{\partial z (x, y)}{\partial x} = 0

Integrating w.r.t. $x$ ., and remembering that now constant of integration will be function of $y$ , hence

z (x, y) = f (y)

Example 2

\frac{\partial^{2} z (x, y)}{\partial x^{2}} = x

Integrating once w.r.t. $x$ gives

\frac{\partial z (x, y)}{\partial x} = \frac{x^{2}}{2} + f (y)

Integrating again gives

z (x, y) = \frac{x^{3}}{6} + x f (y) + g (y)

Example 3

\frac{\partial^{2} z (x, y)}{\partial y^{2}} = y

Integrating once w.r.t. $y$ gives

\frac{\partial z (x, y)}{\partial y} = \frac{y^{2}}{2} + f (x)

Integrating again gives

z (x, y) = \frac{y^{3}}{6} + y f (x) + g (x)

Example 4

\frac{\partial^{2} z (x, y)}{\partial x \partial y} = 0

Integrating once w.r.t $x$ gives

\frac{\partial z (x, y)}{\partial y} = f (y)

Integrating again w.r.t. $y$ gives

z (x, y) = \int f (y) d y + g (x)

Example 5

Solve $u_{t} + u_{x} = 0$ with $u (x, 1) = \frac{x}{1 + x^{2}}$ . Let $u \equiv u (x (t), t)$ , therefore

\frac{d u}{d t} = \frac{\partial u}{\partial t} + \frac{\partial u}{\partial x} \frac{d x}{d t}

Comparing the above with the given PDE, we see that if $\frac{d x}{d t} = 1$ then $\frac{d u}{d t} = 0$ or $u (x (t), t)$ is constant. At $t = 1$ we are given that

\begin{matrix} (1) & u = \frac{x (1)}{1 + x {(1)}^{2}} \end{matrix}

To ﬁnd $x (1)$ , from $\frac{d x}{d t} = 1$ we obtain that $x (t) = t + c$ . At $t = 1$ , $c = x (1) - 1$ . Hence $x (t) = t + x (1) - 1$ or

x (1) = x (t) + 1 - t

Hence solution from (1) becomes

u = \frac{x - t + 1}{1 + {(x - t + 1)}^{2}}

Example 6

Solve $u_{t} + u_{x} + u^{2} = 0$ .

Let $u \equiv u (x (t), t)$ , therefore

\frac{d u}{d t} = \frac{\partial u}{\partial t} + \frac{\partial u}{\partial x} \frac{d x}{d t}

Comparing the above with the given PDE, we see that if $\frac{d x}{d t} = 1$ then $\frac{d u}{d t} = - u^{2}$ or $\frac{- 1}{u} = - t + c .$ Hence

u = \frac{1}{t + c}

At $t = 0$ , $c = \frac{1}{u (x (0), 0)}$ . Let $u (x (0), 0) = f (x (0))$ . Therefore

u = \frac{1}{t + \frac{1}{f (x (0))}}

Now we need to ﬁnd $x (0)$ . From $\frac{d x}{d t} = 1$ , then $x = t + c$ or $c = x (0)$ , hence $x (0) = x - t$ and the above becomes

u (x, t) = \frac{1}{t + \frac{1}{f (x - t)}} = \frac{f (x - t)}{t f (x - t) + 1}

8 Fourier series ﬂow chart

(added Oct. 20, 2016)

pict

8.1 Theorem on when we can do term by term diﬀerentiation

If $f (x)$ on $- L \leq x \leq L$ is continuous (notice, NOT piecewise continuous), this means $f (x)$ has no jumps in it, and that $f^{'} (x)$ exists on $- L < x < L$ and $f^{'} (x)$ is either continuous or piecewise continuous (notice, that $f^{'} (x)$ can be piecewise continuous (P.W.C.), i.e. have ﬁnite number of jump discontinuities), and also and this is very important, that $f (- L) = f (L)$ then we can do term by term diﬀerentiation of the Fourier series of $f (x)$ and use $=$ instead of $\sim$ . Not only that, but the term by term diﬀerentiation of the Fourier series of $f (x)$ will give the Fourier series of $f^{'} (x)$ itself.

So that main restriction here is that $f (x)$ on $- L \leq x \leq L$ is continuous (no jump discontinuities) and that $f (- L) = f (L)$ . So look at $f (x)$ ﬁrst and see if it is continuous or not (remember, the whole $f (x)$ has to be continuous, not piecewise, so no jump discontinuities). If this condition is met, look at see if $f (- L) = f (L)$ .

For example $f (x) = x$ on $- 1 \leq x \leq 1$ is continuous, but $f (- 1) \neq f (1)$ so the F.S. of $f (x)$ can’t be term be term diﬀerentiated (well, it can, but the result will not be the Fourier series of $f^{'} (x)$ ). So we should not do term by term diﬀerentiation in this case.

But the Fourier series for $f (x) = x^{2}$ can be term by term diﬀerentiated. This has its $f^{'} (x)$ being continuous, since it meets all the conditions. Also Fourier series for $f (x) = | x |$ can be term by term diﬀerentiated. This has its $f^{'} (x)$ being P.W.C. due to a jump at $x = 0$ but that is OK, as $f^{'} (x)$ is allowed to be P.W.C., but it is $f (x)$ which is not allowed to be P.W.C.

There is a useful corollary that comes from the above. If $f (x)$ meets all the conditions above, then its Fourier series is absolutely convergent and also uniformly convergent. The M-test can be used to verify that the Fourier series is uniformly convergent.

8.2 Relation between coeﬃcients of Fourier series of $f (x)$ Fourier series of $f^{'} (x)$

If term by term diﬀerentiation allowed, then let

\begin{aligned} f (x) & = \frac{a_{0}}{2} + \sum_{n = 1}^{\infty} a_{n} \cos (n \frac{π}{L} x) + b_{n} \sin (n \frac{π}{L} x) \\ f^{'} (x) & = \frac{α_{0}}{2} + \sum_{n = 1}^{\infty} α_{n} \cos (n \frac{π}{L} x) + β_{n} \sin (n \frac{π}{L} x) \end{aligned}

Then

\begin{aligned} α_{n} & = n b_{n} \\ β_{n} & = - n a_{n} \end{aligned}

And Bessel’s inequality instead of $\frac{a_{0}^{2}}{2} + \sum_{n = 1}^{\infty} (a_{n}^{2} + b_{n}^{2}) < \infty$ now becomes $\sum_{n = 1}^{\infty} n^{2} (a_{n}^{2} + b_{n}^{2}) < \infty$ . So it is stronger.

8.3 Theorem on convergence of Fourier series

If $f (x)$ is piecewise continuous on $- L < x < L$ and if it is periodic with period $2 L$ and if on any point $x$ on the entire domain $- \infty < x < \infty$ both the left sided derivative and the right sided derivative exist (but these do not have to be the same !) then we say that the Fourier series of $f (x)$ converges and it converges to the average of $f (x)$ at each point including points that have jump discontinuities.

9 Laplacian in diﬀerent coordinates

(added Jan. 10, 2019)

10 Linear combination of two solution is solution to ODE

If $y_{1}, y_{2}$ are two solutions to $a y^{''} + b y^{'} + c y = 0$ then to show that $c_{1} y_{1} + c_{2} y_{2}$ is also solution:

\begin{aligned} a y_{1}^{''} + b y_{1}^{'} + c y_{1} & = 0 \\ a y_{2}^{''} + b y_{2}^{'} + c y_{2} & = 0 \end{aligned}

Multiply the ﬁrst ODE by $c_{1}$ and second ODE by $c_{2}$

\begin{aligned} a {(c_{1} y_{1})}^{''} + b {(c_{1} y_{1})}^{'} + c (c_{1} y_{1}) & = 0 \\ a {(c_{2} y_{2})}^{''} + b {(c_{2} y_{2})}^{'} + c (c_{2} y_{2}) & = 0 \end{aligned}

Add the above two equations, using linearity of diﬀerentials

a {(c_{1} y_{1} + c_{2} y_{2})}^{''} + b {(c_{1} y_{1} + c_{2} y_{2})}^{'} + c (c_{1} y_{1} + c_{2} y_{2}) = 0

Therefore $c_{1} y_{1} + c_{2} y_{2}$ satisﬁes the original ODE. Hence solution.

11 To ﬁnd the Wronskian ODE

Since

W (x) = | \begin{matrix} y_{1} & y_{2} \\ y_{1}^{'} & y_{2}^{'} \end{matrix} | = y_{1} y_{2}^{'} - y_{2} y_{1}^{'}

Where $y_{1}, y_{2}$ are two solutions to $a y^{''} + b y^{'} + c y = 0.$ Write

\begin{aligned} a y_{1}^{''} + p y_{1}^{'} + c y_{1} & = 0 \\ a y_{2}^{''} + p y_{2}^{'} + c y_{2} & = 0 \end{aligned}

Multiply the ﬁrst ODE above by $y_{2}$ and the second by $y_{1}$

\begin{aligned} a y_{2} y_{1}^{''} + p y_{2} y_{1}^{'} + c y_{2} y_{1} & = 0 \\ a y_{1} y_{2}^{''} + p y_{1} y_{2}^{'} + c y_{1} y_{2} & = 0 \end{aligned}

Subtract the second from the ﬁrst

\begin{matrix} (1) & a (y_{2} y_{1}^{''} - y_{1} y_{2}^{''}) + p (y_{2} y_{1}^{'} - y_{1} y_{2}^{'}) = 0 \end{matrix}

But

\begin{matrix} (2) & p (y_{2} y_{1}^{'} - y_{1} y_{2}^{'}) = - p W \end{matrix}

And

\begin{aligned} \frac{d W}{d x} & = \frac{d}{d x} (y_{1} y_{2}^{'} - y_{2} y_{1}^{'}) \\ = y_{1}^{'} y_{2}^{'} + y_{1} y_{2}^{''} - y_{2}^{'} y_{1}^{'} - y_{2} y_{1}^{''} \\ (3) & = y_{1} y_{2}^{''} - y_{2} y_{1}^{''} \end{aligned}

Substituting (2,3) into (1) gives the Wronskian diﬀerential equation

\begin{aligned} - a (\frac{d W}{d x}) - p W & = 0 \\ a W^{'} + p W & = 0 \end{aligned}

Whose solution is

W (x) = C e^{- \int \frac{p}{a} d x}

Where $C$ is constant of integration.

Remember: $W (x_{0}) = 0$ does not mean the two functions are linearly dependent. The functions can still be Linearly independent on other interval, It just means $x_{0}$ can’t be in the domain of the solution for two functions to be solutions. However, if the two functions are linearly dependent, then this implies $W = 0$ everywhere. So to check if two functions are L.D., need to show that $W = 0$ everywhere.

12 Green functions notes

$◼$ Green function is what is called impulse response in control. But it is more general, and can be used for solving PDE also.

Given a diﬀerential equation with some forcing function on the right side. To solve this, we replace the forcing function with an impulse. The solution of the DE now is called the impulse response, which is the Green’s function of the diﬀerential equation.

Now to ﬁnd the solution to the original problem with the original forcing function, we just convolve the Green function with the original forcing function. Here is an example. Suppose we want to solve $L [y (t)] = f (t)$ with zero initial conditions. Then we solve $L [g (t)] = δ (t)$ . The solution is $g (t)$ . Now $y (t) = g (t) ⊛ f (t)$ . This is for initial value problem. For example. $y^{'} (t) + k x = e^{a t}$ , with $y (0) = 0$ . Then we solve $g^{'} (t) + k g = δ (t)$ . The solution is $g (t) = {\begin{array}{cc} e^{- k t} & t > 0 \\ 0 & t < 0 \end{array}$ , this is for causal system. Hence $y (t) = g (t) ⊛ f (t)$ . The nice thing here, is that once we ﬁnd $g (t)$ , we can solve $y^{'} (t) + k x = f (t)$ for any $f (t)$ by just convolving the Green function (impulse response) with the new $f (t)$ .

$◼$ We can think of Green function as an inverse operator. Given $L [y (t)] = f (t)$ , we want to ﬁnd solution $y (t) = \int_{- \infty}^{\infty} G (t; τ) f (τ) d τ$ . So in a sense, $G (t; τ)$ is like $L^{- 1} [y (t)]$ .

$◼$ Need to add notes for Green function for Sturm-Liouville boundary value ODE. Need to be clear on what boundary conditions to use. What is B.C. is not homogeneous?

$◼$ Green function properties:

$G (t; τ)$ is continuous at $t = τ$ . This is where the impulse is located.
The derivative $G^{'} (t)$ just before $t = τ$ is not the same as $G^{'} (t)$ just after $t = τ$ . i.e. $G^{'} (t; t - ε) - G^{'} (t; t + ε) \neq 0$ . This means there is discontinuity in derivative.
$G (t; τ)$ should satisfy same boundary conditions as original PDE or ODE (this is for Sturm-Liouville or boundary value problems).
$L [G (t; τ)] = 0$ for $t \neq τ$
$G (x; τ)$ is symmetric. i.e. $G (x; τ) = G (τ; x)$ .

$◼$ When solving for $G (t; τ)$ , in context of 1D, hence two boundary conditions, one at each end, and second order ODE (Sturm-Liouville), we now get two solutions, one for $t < τ$ and one for $t > τ$ .

So we have $4$ constants of integrations to ﬁnd (this is for second order ODE) not just two constants as normally one would get , since now we have 2 diﬀerent solutions. Two of these constants from the two boundary conditions, and two more come from property of Green function as mentioned above. $G (t; τ) = {\begin{array}{cc} A_{1} y_{1} + A_{2} y_{2} & 0 < t < τ \\ A_{3} y_{1} + A_{4} y_{2} & τ < t < L \end{array}$

13 Laplace transform notes

$◼$ Remember that $u_{c} (t) f (t - c) ⟺ e^{- c s} F (s)$ and $u_{c} (t) f (t) ⟺ e^{- c s} L {f (t + c)}$ . For example, if we are given $u_{2} (t) t$ , then $L (u_{2} (t) t) = e^{- 2 s} L {t + 2} = e^{- 2 s} (\frac{1}{s^{2}} + \frac{2}{s}) = e^{- 2 s} (\frac{1 + 2 s}{s^{2}})$ . Do not do $u_{c} (t) f (t) ⟺ e^{- c s} L {f (t)}$ ! That will be a big error. We use this allot when asked to write a piecewise function using Heaviside functions.

14 Series, power series, Laurent series notes

$◼$ if we have a function $f (x)$ represented as series (say power series or Fourier series), then we say the series converges to $f (x)$ uniformly in region $D$ , if given $ε > 0$ , we can number $N$ which depends only on $ε$ , such that $| f (x) - S_{N} (x) | < ε$ .

Where here $S_{N} (x)$ is the partial sum of the series using $N$ terms. The diﬀerence between uniform convergence and non-uniform convergence, is that with uniform the number $N$ only depends on $ε$ and not on which $x$ we are trying to approximate $f (x)$ at. In uniform convergence, the number $N$ depends on both $ε$ and $x$ . So this means at some locations in $D$ we need much larger $N$ than in other locations to convergence to $f (x)$ with same accuracy. Uniform convergence is better. It depends on the basis functions used to approximate $f (x)$ in the series.

If the function $f (x)$ is discontinuous at some point, then it is not possible to ﬁnd uniform convergence there. As we get closer and closer to the discontinuity, more and more terms are needed to obtained same approximation away from the discontinuity, hence not uniform convergence. For example, Fourier series approximation of a step function can not be uniformly convergent due to the discontinuity in the step function.

$◼$ Geometric series:

\begin{aligned} \sum_{n = 0}^{N} r^{n} & = 1 + r + r^{2} + r^{3} + \dots + r^{N} = \frac{1 - r^{N + 1}}{1 - r} \\ \sum_{n = 1}^{N} r^{n} & = - 1 + \sum_{n = 0}^{N} r^{n} = - 1 + \frac{1 - r^{N + 1}}{1 - r} = r \frac{1 - r^{N}}{1 - r} \\ \sum_{n = 0}^{\infty} r^{n} & = 1 + r + r^{2} + r^{3} + \dots = \frac{1}{1 - r} | r | < 1 \\ \sum_{n = 0}^{\infty} {(- 1)}^{n} r^{n} & = 1 - r + r^{2} - r^{3} + \dots = \frac{1}{1 + r} | r | < 1 \end{aligned}

$◼$ Binomial series:

General binomial is

{(x + y)}^{n} = x^{n} + n x^{n - 1} y + \frac{n (n - 1)}{2!} x^{n - 2} y^{2} + \frac{n (n - 1) (n - 2)}{3!} x^{n - 3} y^{3} + \dots

From the above we can generate all other special cases. For example,

{(1 + x)}^{n} = 1 + n x + \frac{n (n - 1) x^{2}}{2!} + \frac{n (n - 1) (n - 2) x^{3}}{3!} + \dots

This work for positive and negative $n$ , rational or not. The sum converges when only for $| x | < 1$ . From this, we can derive the above sums also for the geometric series. For example, for $n = - 1$ the above becomes

\begin{aligned} \frac{1}{(1 + x)} & = 1 - x + x^{2} - x^{3} + \dots | x | < 1 \\ \frac{1}{(1 - x)} & = 1 + x + x^{2} + x^{3} + \dots | x | < 1 \end{aligned}

For $| x | > 1$ , we can still ﬁnd series expansion in negative powers of $x$ as follows

\begin{aligned} {(1 + x)}^{n} & = {(x (1 + \frac{1}{x}))}^{n} \\ = x^{n} {(1 + \frac{1}{x})}^{n} \end{aligned}

And now since $| \frac{1}{x} | < 1$ , we can use binomial expansion to expand the term ${(1 + \frac{1}{x})}^{n}$ in the above and obtain a convergent series, since now $| \frac{1}{x} | < 1 .$ This will give the following expansion

\begin{aligned} {(1 + x)}^{n} & = x^{n} {(1 + \frac{1}{x})}^{n} \\ = x^{n} (1 + n (\frac{1}{x}) + \frac{n (n - 1)}{2!} {(\frac{1}{x})}^{2} + \frac{n (n - 1) (n - 2)}{3!} {(\frac{1}{x})}^{3} + \dots) \end{aligned}

So everything is the same, we just change $x$ with $\frac{1}{x}$ and remember to multiply the whole expansion with $x^{n}$ . For example, for $n = - 1$

\begin{aligned} \frac{1}{(1 + x)} & = \frac{1}{x (1 + \frac{1}{x})} = \frac{1}{x} (1 - \frac{1}{x} + {(\frac{1}{x})}^{2} - {(\frac{1}{x})}^{3} + \dots) | x | > 1 \\ \frac{1}{(1 - x)} & = \frac{1}{x (1 - \frac{1}{x})} = \frac{1}{x} (1 + \frac{1}{x} + {(\frac{1}{x})}^{2} + {(\frac{1}{x})}^{3} + \dots) | x | > 1 \end{aligned}

These tricks are very useful when working with Laurent series.

$◼$ Arithmetic series:

\begin{aligned} \sum_{n = 1}^{N} n & = \frac{1}{2} N (N + 1) \\ \sum_{n = 1}^{N} a_{n} & = N (\frac{a_{1} + a_{N}}{2}) \end{aligned}

i.e. the sum is $N$ times the arithmetic mean.

$◼$ Taylor series: Expanded around $x = a$ is

f (x) = f (a) + (x - a) f^{'} (a) + \frac{{(x - a)}^{2} f^{''} (a)}{2!} + \frac{{(x - a)}^{3} f^{(3)} (a)}{3!} + \dots + R_{n}

Where $R_{n}$ is remainder $R_{n} = \frac{{(x - a)}^{n + 1}}{(n + 1)!} f^{(n + 1)} (x_{0})$ where $x_{0}$ is some point between $x$ and $a$ .

$◼$ Maclaurin series: Is just Taylor expanded around zero. i.e. $a = 0$

f (x) = f (0) + x f^{'} (0) + \frac{x^{2} f^{''} (0)}{2!} + \frac{x^{3} f^{(3)} (0)}{3!} + \dots

$◼$ This diagram shows the diﬀerent convergence of series and the relation between them

The above shows that an absolutely convergent series ( $B$ ) is also convergent. Also a uniformly convergent series ( $D$ ) is also convergent. But the series $B$ is absolutely convergent and not uniform convergent. While $D$ is uniform convergent and not absolutely convergent.

The series $C$ is both absolutely and uniformly convergent. And ﬁnally the series $A$ is convergent, but not absolutely (called conditionally convergent). Examples of $B$ (converges absolutely but not uniformly) is

\begin{aligned} \sum_{n = 0}^{\infty} x^{2} \frac{1}{{(1 + x^{2})}^{n}} & = x^{2} (1 + \frac{1}{1 + x^{2}} + \frac{1}{{(1 + x^{2})}^{2}} + \frac{1}{{(1 + x^{2})}^{3}} + \dots) \\ = x^{2} + \frac{x^{2}}{1 + x^{2}} + \frac{x^{2}}{{(1 + x^{2})}^{2}} + \frac{x^{2}}{{(1 + x^{2})}^{3}} + \dots \end{aligned}

And example of $D$ (converges uniformly but not absolutely) is

\sum_{n = 1}^{\infty} {(- 1)}^{n + 1} \frac{1}{x^{2} + n} = \frac{1}{x^{2} + 1} - \frac{1}{x^{2} + 2} + \frac{1}{x^{3} + 3} - \frac{1}{x^{4} + 4} + \dots

Example of $A$ (converges but not absolutely) is the alternating harmonic series

\sum_{n = 1}^{\infty} {(- 1)}^{n + 1} \frac{1}{n} = 1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots

The above converges to $\ln (2)$ but absolutely it now becomes the harmonic series and it diverges

\sum_{n = 1}^{\infty} \frac{1}{n} = 1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \dots

For uniform convergence, we really need to have an $x$ in the series and not just numbers, since the idea behind uniform convergence is if the series convergence to within an error tolerance $ε$ using the same number of terms independent of the point $x$ in the region.

$◼$ The sequence $\sum_{n = 1}^{\infty} \frac{1}{n^{a}}$ converges for $a > 1$ and diverges for $a \leq 1$ . So $a = 1$ is the ﬂip value. For example

1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \dots

Diverges, since $a = 1$ , also $1 + \frac{1}{\sqrt{2}} + \frac{1}{\sqrt{3}} + \frac{1}{\sqrt{4}} + \dots$ diverges, since $a = \frac{1}{2} \leq 1$ . But $1 + \frac{1}{4} + \frac{1}{9} + \frac{1}{16} + \dots$ converges, where $a = 2$ here and the sum is $\frac{π^{2}}{6}$ .

$◼$ Using partial sums. Let $\sum_{n = 0}^{\infty} a_{n}$ be some sequence. The partial sum is $S_{N} = \sum_{n = 0}^{N} a_{n}$ . Then

\sum_{n = 0}^{\infty} a_{n} = lim_{N \to \infty} S_{n}

If $lim_{N \to \infty} S_{n}$ exist and ﬁnite, then we can say that $\sum_{n = 0}^{\infty} a_{n}$ converges. So here we use set up a sequence who terms are partial sum, and them look at what happens in the limit to such a term as $N \to θ$ . Need to ﬁnd an example where this method is easier to use to test for convergence than the other method below.

$◼$ Given a series, we are allowed to rearrange order of terms only when the series is absolutely convergent. Therefore for the alternating series $1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots$ , do not rearrange terms since this is not absolutely convergent. This means the series sum is independent of the order in which terms are added only when the series is absolutely convergent.

$◼$ In an inﬁnite series of complex numbers, the series converges, if the real part of the series and also the complex part of the series, each converges on their own.

$◼$ Power series: $f (z) = \sum_{n = 0}^{\infty} a_{n} {(z - z_{0})}^{n}$ . This series is centered at $z_{0}$ . Or expanded around $z_{0}$ . This has radius of convergence $R$ is the series converges for $| z - z_{0} | < R$ and diverges for $| z - z_{0} | > R$ .

$◼$ Tests for convergence.

Always start with preliminary test. If $lim_{n \to \infty} a_{n}$ does not go to zero, then no need to do anything else. The series $\sum_{n = 0}^{\infty} a_{n}$ does not converge. It diverges. But if $lim_{n \to \infty} a_{n} = 0$ , it still can diverge. So this is a necessary but not suﬃcient condition for convergence. An example is $\sum \frac{1}{n}$ . Here $a_{n} \to 0$ in the limit, but we know that this series does not converge.
For Uniform convergence, there is a test called the weierstrass M test, which can be used to check if the series is uniformly convergent. But if this test fails, this does not necessarily mean the series is not uniform convergent. It still can be uniform convergent. (need an example).
To test for absolute convergence, use the ratio test. If $L = lim_{n \to \infty} | \frac{a_{n + 1}}{a_{n}} | < 1$ then absolutely convergent. If $L = 1$ then inconclusive. Try the integral test. If $L > 1$ then not absolutely convergent. There is also the root test. $L = lim_{n \to \infty} \sqrt[n]{| a_{n} |} = lim_{n \to \infty} {| a_{n} |}^{\frac{1}{n}}$ .
The integral test, use when ratio test is inconclusive. $L = lim_{n \to \infty} \int^{n} f (x) d x$ where $a (n)$ becomes $f (x)$ . Remember to use this only of the terms of the sequence are monotonically decreasing and are all positive. For example, $\sum_{n = 1}^{\infty} \ln (1 + \frac{1}{n})$ , then use $L = lim_{N \to \infty} \int^{N} \ln (1 + \frac{1}{x}) d x = {((1 + x) \ln (1 + x) - x \ln (x) - 1)}^{N}$ . Notice, we only use the upper limit in the integral. This becomes (after simpliﬁcations) $lim_{N \to \infty} \frac{N}{N + 1} = 1$ . Hence the limit $L$ is ﬁnite, then the series converges.
Radius of convergence is called $R = \frac{1}{L}$ where $L$ is from (3) above.
Comparison test. Compare the series with one we happen to already know it converges. Let $\sum b_{n}$ be a series which we know is convergent (for example $\sum \frac{1}{n^{2}}$ ), and we want to ﬁnd if $\sum a_{n}$ converges. If all terms of both series are positive and if $a_{n} \leq b_{n}$ for each $n$ , then we conclude that $\sum a_{n}$ converges also.

$◼$ For Laurent series, lets say singularity is at $z = 0$ and $z = 1$ . To expand about $z = 0$ , get $f (z)$ to look like $\frac{1}{1 - z}$ and use geometric series for $| z | < 1$ . To expand about $z = 1$ , there are two choices, to the inside and to the outside. For the outside, i.e. $| z | > 1$ , get $f (z)$ to have $\frac{1}{1 - \frac{1}{z}}$ form, since this now valid for $| z | > 1$ .

$◼$ Can only use power series $\sum a_{n} {(z - z_{0})}^{n}$ to expand $f (z)$ around $z_{0}$ only if $f (z)$ is analytic at $z_{0}$ . If $f (z)$ is not analytic at $z_{0}$ need to use Laurent series. Think of Laurent series as an extension of power series to handle singularities.

14.1 Some tricks to ﬁnd sums

14.1.1 Example 1

Find $\sum_{n = 1}^{\infty} \frac{e^{i n x}}{n}$

solution Let $f (x) = \sum_{n = 1}^{\infty} \frac{e^{i n x}}{n}$ , taking derivative gives

\begin{aligned} f^{'} (x) & = i \sum_{n = 1}^{\infty} e^{i n x} \\ = i \sum_{n = 1}^{\infty} {(e^{i x})}^{n} \\ = i (\sum_{n = 0}^{\infty} {(e^{i x})}^{n} - 1) \\ = \frac{i}{1 - e^{i x}} - i \end{aligned}

Hence

\begin{aligned} f (x) & = \int (\frac{i}{1 - e^{i x}} - i) d x \\ = i \int \frac{d x}{1 - e^{i x}} - i x + C \\ = i (x + i \ln (1 - e^{i x})) - i x + C \\ = i x - \ln (1 - e^{i x}) - i x + C \\ = - \ln (1 - e^{i x}) + C \end{aligned}

We can set $C = 0$ to obtain

\sum_{n = 1}^{\infty} \frac{e^{i n x}}{n} = - \ln (1 - e^{i x})

More tricks to add...

14.2 Methods to ﬁnd Laurent series

Let us ﬁnd the Laurent series for $f (z) = \frac{5 z - 2}{z (z - 1)}$ . There is a singularity of order $1$ at $z = 0$ and $z = 1$ .

14.2.1 Method one

Expansion around $z = 0$ . Let

\begin{aligned} g (z) & = z f (z) \\ = \frac{5 z - 2}{(z - 1)} \end{aligned}

This makes $g (z)$ analytic around $z$ , since $g (z)$ do not have a pole at $z = 0$ , then it is analytic around $z = 0$ and therefore it has a power series expansion around $z = 0$ given by

\begin{matrix} (1) & g (z) = \sum_{n = 0}^{\infty} a_{n} z^{n} \end{matrix}

Where

a_{n} = \frac{1}{n!} {g^{(n)} (z) |}_{z = 0}

But

g (0) = 2

And

\begin{aligned} g^{'} (z) & = \frac{5 (z - 1) - (5 z - 2)}{{(z - 1)}^{2}} = \frac{- 3}{{(z - 1)}^{2}} \\ g^{'} (0) & = - 3 \end{aligned}

And

\begin{aligned} g^{''} (z) & = \frac{- 3 (- 2)}{{(z - 1)}^{3}} = \frac{6}{{(z - 1)}^{3}} \\ g^{''} (0) & = - 6 \end{aligned}

And

\begin{aligned} g^{'''} (z) & = \frac{6 (- 3)}{{(z - 1)}^{4}} = \frac{- 18}{{(z - 1)}^{4}} \\ g^{''} (0) & = - 18 \end{aligned}

And so on. Therefore, from (1)

\begin{aligned} g (z) & = g (0) + g^{'} (0) z + \frac{1}{2!} g^{''} (0) z^{2} + \frac{1}{3!} g^{'''} (0) z^{3} + \dots \\ = 2 - 3 z - \frac{6}{2} z^{2} - \frac{18}{3!} z^{3} - \dots \\ = 2 - 3 z - 3 z^{2} - 3 z^{3} - \dots \end{aligned}

Therefore

\begin{aligned} f (z) & = \frac{g (z)}{z} \\ = \frac{2}{z} - 3 - 3 z - 3 z^{2} - \dots \end{aligned}

The residue is $2$ . The above expansion is valid around $z = 0$ up and not including the next singularity, which is at $z = 1$ . Now we ﬁnd the expansion of $f (z)$ around $z = 1$ . Let

\begin{aligned} g (z) & = (z - 1) f (z) \\ = \frac{5 z - 2}{z} \end{aligned}

This makes $g (z)$ analytic around $z = 1$ , since $g (z)$ do not have a pole at $z = 1$ . Therefore it has a power series expansion about $z = 1$ given by

\begin{matrix} (1) & g (z) = \sum_{n = 0}^{\infty} a_{n} {(z - 1)}^{n} \end{matrix}

Where

a_{n} = \frac{1}{n!} {g^{(n)} (z) |}_{z = 1}

But

g (1) = 3

And

\begin{aligned} g^{'} (z) & = \frac{5 z - (5 z - 2)}{z^{2}} = \frac{2}{z^{2}} \\ g^{'} (1) & = 2 \end{aligned}

And

\begin{aligned} g^{''} (z) & = \frac{2 (- 2)}{z^{3}} = \frac{- 4}{z^{3}} \\ g^{''} (1) & = - 4 \end{aligned}

And

\begin{aligned} g^{'''} (z) & = \frac{- 4 (- 3)}{z^{4}} = \frac{12}{z^{4}} \\ g^{''} (1) & = 12 \end{aligned}

And so on. Therefore, from (1)

\begin{aligned} g (z) & = g (1) + g^{'} (1) (z - 1) + \frac{1}{2!} g^{''} (1) {(z - 1)}^{2} + \frac{1}{3!} g^{'''} (1) {(z - 1)}^{3} + \dots \\ = 3 + 2 (z - 1) - \frac{4}{2} {(z - 1)}^{2} + \frac{12}{3!} {(z - 1)}^{3} - \dots \\ = 3 + 2 (z - 1) - 2 {(z - 1)}^{2} + 2 {(z - 1)}^{3} - \dots \end{aligned}

Therefore

\begin{aligned} f (z) & = \frac{g (z)}{z - 1} \\ = \frac{3}{z - 1} + 2 - 2 (z - 1) + 2 {(z - 1)}^{2} - 2 {(z - 1)}^{3} + \dots \end{aligned}

The residue is $3$ . The above expansion is valid around $z = 1$ up and not including the next singularity, which is at $z = 0$ inside a circle of radius $1$ .

Putting the above two regions together, then we see there is a series expansion of $f (z)$ that is shared between the two regions, in the shaded region below.

Let check same series in the shared region give same values. Using the series expansion about $f (0)$ to ﬁnd $f (z)$ at point $z = \frac{1}{2}$ , gives $- 2$ when using $10$ terms in the series. Using series expansion around $z = 1$ to ﬁnd $f (\frac{1}{2})$ using $10$ terms also gives $- 2$ . So both series are valid produce same result.

14.2.2 Method Two

This method is simpler than the above, but it results in diﬀerent regions. It is based on converting the expression in order to use geometric series expansion on it.

f (z) = \frac{5 z - 2}{z (z - 1)}

Since there is a pole at $z = 0$ and at $z = 1$ , then we ﬁrst ﬁnd expansion for $0 < | z | < 1$ . To do this, we write the above as

\begin{aligned} f (z) & = \frac{5 z - 2}{z} (\frac{1}{z - 1}) \\ = \frac{2 - 5 z}{z} (\frac{1}{1 - z}) \end{aligned}

And now expand $\frac{1}{1 - z}$ using geometric series, which is valid for $| z | < 1$ . This gives

\begin{aligned} f (z) & = \frac{2 - 5 z}{z} (1 + z + z^{2} + z^{3} + \dots) \\ = \frac{2}{z} (1 + z + z^{2} + z^{3} + \dots) - 5 (1 + z + z^{2} + z^{3} + \dots) \\ = (\frac{2}{z} + 2 + 2 z + 2 z^{2} + \dots) - (5 + 5 z + 5 z^{2} + 5 z^{3} + \dots) \\ = \frac{2}{z} - 3 - 3 z - 3 z^{2} - 3 z^{3} - \dots \end{aligned}

The above is valid for $0 < | z | < 1$ which agrees with result of method 1.

Now, to ﬁnd expansion for $| z | > 1$ , we need a term that looks like $(\frac{1}{1 - \frac{1}{z}})$ . Since now it can be expanded for $| \frac{1}{z} | < 1$ or $| z | > 1$ which is what we want. Therefore, writing $f (z)$ as

f (z) = \frac{5 z - 2}{z (z - 1)} = \frac{5 z - 2}{z^{2} (1 - \frac{1}{z})} = \frac{5 z - 2}{z^{2}} (\frac{1}{1 - \frac{1}{z}})

But for $| \frac{1}{z} | < 1$ the above becomes

\begin{aligned} f (z) & = \frac{5 z - 2}{z^{2}} (1 + \frac{1}{z} + \frac{1}{z^{2}} + \frac{1}{z^{3}} + \dots) \\ = \frac{5}{z} (1 + \frac{1}{z} + \frac{1}{z^{2}} + \frac{1}{z^{3}} + \dots) - \frac{2}{z^{2}} (1 + \frac{1}{z} + \frac{1}{z^{2}} + \frac{1}{z^{3}} + \dots) \\ = (\frac{5}{z} + \frac{5}{z^{2}} + \frac{5}{z^{3}} + \frac{5}{z^{4}} + \dots) - (\frac{2}{z^{2}} + \frac{2}{z^{3}} + \frac{2}{z^{4}} + \frac{2}{z^{5}} + \dots) \\ = \frac{5}{z} + \frac{3}{z^{3}} + \frac{3}{z^{4}} + \frac{3}{z^{5}} + \dots \end{aligned}

With residue $5$ . The above is valid for $| z | > 1$ . The following diagram illustrates the result obtained from method 2.

14.2.3 Method Three

For expansion about $z = 0$ , this uses same method as above, giving same series valid for $| z | < 1 .$ This method is a little diﬀerent for those points other than zero. The idea is to replace $z$ by $z - z_{0}$ where $z_{0}$ is the point we want to expand about and do this replacement in $f (z)$ itself. So for $z = 1$ using this example, we let $ξ = z - 1$ hence $z = ξ + 1$ . Then $f (z)$ becomes

\begin{aligned} f (z) & = \frac{5 z - 2}{z (z - 1)} \\ = \frac{5 (ξ + 1) - 2}{(ξ + 1) (ξ)} \\ = \frac{5 (ξ + 1) - 2}{ξ} (\frac{1}{ξ + 1}) \\ = \frac{5 ξ + 3}{ξ} (\frac{1}{1 + ξ}) \end{aligned}

Now we expand $\frac{1}{1 + ξ}$ for $| ξ | < 1$ and the above becomes

\begin{aligned} f (z) & = \frac{5 ξ + 3}{ξ} (1 - ξ + ξ^{2} - ξ^{3} + ξ^{4} - \dots) \\ = \frac{5 ξ + 3}{ξ} (1 - ξ + ξ^{2} - ξ^{3} + ξ^{4} - \dots) \\ = (\frac{5 ξ + 3}{ξ} - (5 ξ + 3) + (5 ξ + 3) ξ - (5 ξ + 3) ξ^{2} + \dots) \\ = (5 + \frac{3}{ξ} - 5 ξ - 3 + 5 ξ^{2} + 3 ξ - 5 ξ^{3} - 3 ξ^{2} + \dots) \\ = (2 + \frac{3}{ξ} - 2 ξ + 2 ξ^{2} - 2 ξ^{3} + \dots) \end{aligned}

We now replace $ξ = z - 1$ and the above becomes

f (z) = (\frac{3}{(z - 1)} + 2 - 2 (z - 1) + 2 {(z - 1)}^{2} - 2 {(z - 1)}^{3} + 2 {(z - 1)}^{4} - \dots)

The above is valid for $| ξ | < 1$ or $| z - 1 | < 1$ or $- 1 < (z - 1) < 1$ or $0 < z < 2$ . This gives same series and for same region as in method one. But this is little faster as it uses Binomial series short cut to ﬁnd the expansion instead of calculating derivatives as in method one.

14.2.4 Conclusion

Method one and method three give same series and for same regions. Method three uses binomial expansion as short cut and requires one to convert $f (z)$ to form to allow using Binomial expansion. Method one does not use binomial expansion but requires doing many derivatives to evaluate the terms of the power series. It is more direct method.

Method two also uses binomial expansion, but gives diﬀerent regions that method one and three.

If one is good in diﬀerentiation, method one seems the most direct. Otherwise, the choice is between method two or three as they both use Binomial expansion. Method two seems a little more direct than method three. It also depends what the problem is asking form. If the problem asks to expand around $z_{0}$ vs. if it is asking to ﬁnd expansion in $| z | > 1$ for example, then this decides which method to use.

15 Gamma function notes

$◼$ Gamma function is deﬁned by

Γ (x) = \int_{0}^{\infty} t^{x - 1} e^{- t} d t x > 0

The above is called the Euler representation. Or if we want it deﬁned in complex domain, the above becomes

Γ (z) = \int_{0}^{\infty} t^{z - 1} e^{- t} d t Re (z) > 0

Since the above is deﬁned only for right half plane, there is way to extend this to left half plane, using what is called analytical continuation. More on this below. First, some relations involving $Γ (x)$

\begin{aligned} Γ (z) & = (z - 1) Γ (z - 1) Re (z) > 1 \\ Γ (1) & = 1 \\ Γ (2) & = 1 \\ Γ (3) & = 2 \\ Γ (4) & = 3! \\ Γ (n) & = (n - 1)! \\ Γ (n + 1) & = n! \\ Γ (\frac{1}{2}) & = \sqrt{π} \\ Γ (z + 1) & = z Γ (z) recursive formula \\ Γ (\bar{z}) & = \overset{―}{Γ (z)} \\ Γ (n + \frac{1}{2}) & = \frac{1 \cdot 3 \cdot 5 \dots (2 n - 1)}{2^{n}} \sqrt{π} \end{aligned}

$◼$ To extend $Γ (z)$ to the left half plane, i.e. for negative values. Let us deﬁne, using the above recursive formula

\bar{Γ} (z) = \frac{Γ (z + 1)}{z} Re (z) > - 1

For example

\bar{Γ} (- \frac{1}{2}) = \frac{Γ (\frac{1}{2})}{- \frac{1}{2}} = - 2 Γ (\frac{1}{2}) = - 2 \sqrt{π}

And for $Re (z) > - 2$

\bar{Γ} (- \frac{3}{2}) = \frac{\bar{Γ} (- \frac{3}{2} + 1)}{- \frac{3}{2}} = (\frac{1}{- \frac{3}{2}}) \bar{Γ} (- \frac{1}{2}) = (\frac{1}{- \frac{3}{2}}) (\frac{1}{- \frac{1}{2}}) Γ (\frac{1}{2}) = (\frac{1}{- \frac{3}{2}}) (\frac{1}{- \frac{1}{2}}) \sqrt{π} = \frac{4}{3} \sqrt{π}

And so on. Notice that for $x < 0$ the functions $Γ (x)$ are not deﬁned for all negative integers $x = - 1, - 2, \dots$ it is also not deﬁned for $x = 0$

$◼$ The above method of extending (or analytical continuation) of the Gamma function to negative values is due to Euler. Another method to extend Gamma is due to Weierstrass. It starts by rewriting from the deﬁnition as follows, where $a > 0$

\begin{aligned} Γ (z) & = \int_{0}^{\infty} t^{z - 1} e^{- t} d t \\ (1) & = \int_{0}^{a} t^{z - 1} e^{- t} d t + \int_{a}^{\infty} t^{z - 1} e^{- t} d t \end{aligned}

Expanding the integrand in the ﬁrst integral using Taylor series gives

\begin{aligned} \int_{0}^{a} t^{z - 1} e^{- t} d t & = \int_{0}^{a} t^{z - 1} (1 + (- t) + \frac{{(- t)}^{2}}{2!} + \frac{{(- t)}^{3}}{3!} + \dots) d t \\ = \int_{0}^{a} t^{z - 1} (1 + (- t) + \frac{{(- t)}^{2}}{2!} + \frac{{(- t)}^{3}}{3!} + \dots) d t \\ = \int_{0}^{a} t^{z - 1} \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n} t^{n}}{n!} d t \\ = \int_{0}^{a} \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n} t^{n + z - 1}}{n!} d t \\ = \sum_{n = 0}^{\infty} \int_{0}^{a} \frac{{(- 1)}^{n} t^{n + z - 1}}{n!} d t \\ = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n!} \int_{0}^{a} t^{n + z - 1} d t \\ = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n!} {[\frac{t^{n + z}}{n + z}]}_{0}^{a} \\ = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n + z)} a^{n + z} \end{aligned}

This takes care of the ﬁrst integral in (1). Now, since the lower limits of the second integral in (1) is not zero, then there is no problem integrating it directly. Remember that in the Euler deﬁnition, it had zero in the lower limit, that is why we said there $Re (z) > 1$ . Now can can choose any value for $a$ . Weierstrass choose $a = 1$ . Hence (1) becomes

\begin{aligned} Γ (z) & = \int_{0}^{a} t^{z - 1} e^{- t} d t + \int_{a}^{\infty} t^{z - 1} e^{- t} d t \\ (2) & = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n + z)} + \int_{1}^{\infty} t^{z - 1} e^{- t} d t \end{aligned}

Notice the term $a^{n + z}$ now is just $1$ since $a = 1$ . The second integral above can now be integrated directly. Let us now verify that Euler continuation $\bar{Γ} (z)$ for say $z = - \frac{1}{2}$ gives the same result as Weierstrass formula. From above, we found that $\bar{Γ} (z) = - 2 \sqrt{π}$ . Equation (2) for $z = - \frac{1}{2}$ becomes

\begin{matrix} (3) & \bar{Γ} (- \frac{1}{2}) = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n - \frac{1}{2})} + \int_{1}^{\infty} t^{- \frac{3}{2}} e^{- t} d t \end{matrix}

Using the computer

\sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n - \frac{1}{2})} = - 2 \sqrt{π} + 2 \sqrt{π} (1 - \erf (1)) - 2 \frac{1}{e}

And direct integration

\int_{1}^{\infty} t^{- \frac{3}{2}} e^{- t} d t = - 2 \sqrt{π} + 2 \sqrt{π} \erf (1) + \frac{2}{e}

Hence (3) becomes

\begin{aligned} \bar{Γ} (- \frac{1}{2}) & = (- 2 \sqrt{π} + 2 \sqrt{π} (1 - \erf (1)) - 2 \frac{1}{e}) + (- 2 \sqrt{π} + 2 \sqrt{π} \erf (1) + \frac{2}{e}) \\ = - 2 \sqrt{π} \end{aligned}

Which is the same as using Euler method. Let us check for $z = - \frac{2}{3}$ . We found above that $\bar{Γ} (- \frac{3}{2}) = \frac{4}{3} \sqrt{π}$ using Euler method of analytical continuation. Now we will check using Weierstrass method. Equation (2) for $z = - \frac{3}{2}$ becomes

\bar{Γ} (- \frac{3}{2}) = \sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n - \frac{3}{2})} + \int_{1}^{\infty} t^{- \frac{5}{2}} e^{- t} d t

Using the computer

\sum_{n = 0}^{\infty} \frac{{(- 1)}^{n}}{n! (n - \frac{3}{2})} = \frac{4 \sqrt{π}}{3} - \frac{4 \sqrt{π} (1 - \erf (1))}{3} + \frac{2}{3 e}

And

\int_{1}^{\infty} t^{- \frac{5}{2}} e^{- t} d t = - \frac{4 \sqrt{π} \erf (1)}{3} + \frac{4 \sqrt{π}}{3} - \frac{2}{3 e}

Hence

\begin{aligned} \bar{Γ} (- \frac{3}{2}) & = (\frac{4 \sqrt{π}}{3} - \frac{4 \sqrt{π} (1 - \erf (1))}{3} + \frac{2}{3 e}) + (- \frac{4 \sqrt{π} \erf (1)}{3} + \frac{4 \sqrt{π}}{3} - \frac{2}{3 e}) \\ = \frac{4}{3} \sqrt{π} \end{aligned}

Which is the same as using the Euler method. Clearly the Euler method for analytical continuation of the Gamma function is simpler to compute.

$◼$ Euler reﬂection formula

\begin{aligned} Γ (x) Γ (1 - x) & = \int_{0}^{\infty} \frac{t^{x - 1}}{1 + t} d t 0 < x < 1 \\ = \frac{π}{\sin (π x)} \end{aligned}

Where contour integration was used to derive the above. See Mary Boas text book, page 607, second edition, example 5 for full derivation.

$◼$ $Γ (z)$ has singularities at $z = 0, - 1, - 2, \dots$ and $Γ (1 - z)$ has singularities at $z = 1, 2, 3, \dots$ so in the above reﬂection formula, the zeros of $\sin (π x)$ cancel the singularities of $Γ (x)$ when it is written as

Γ (1 - x) = \frac{π}{Γ (x) \sin (π x)}

$◼$ $\frac{1}{Γ (z)}$ is entire.

$◼$ There are other representations for $Γ (x)$ . One that uses products by Euler also is

\begin{aligned} Γ (z) & = \frac{1}{z} Π_{n = 1}^{\infty} \frac{{(1 + \frac{1}{n})}^{z}}{1 + \frac{z}{n}} \\ = lim_{n \to \infty} \frac{n! {(n + 1)}^{z}}{z (z - 1) \dots (z + n)} \end{aligned}

And another due to Weierstrass is

\begin{aligned} Γ (z) & = \frac{e^{- γ z}}{z} Π_{n = 1}^{\infty} \frac{e^{\frac{z}{n}}}{1 + \frac{z}{n}} \\ = e^{- γ z} lim_{n \to \infty} \frac{n! \exp (z (1 + \frac{1}{2} + \dots + \frac{1}{n}))}{z (z + 1) (z + 2) \dots (z + n)} \end{aligned}

16 Riemann zeta function notes

$◼$ Given by $ζ (s) = \sum_{n = 1}^{\infty} \frac{1}{n^{s}}$ for $Re (s) > 1$ . Euler studied this and It was extended to the whole complex plane by Riemann. So the Riemann zeta function refer to the one with the extension to the whole complex plane. Euler only looked at it on the real line. It has pole at $s = 1$ . Has trivial zeros at $s = - 2, - 4, - 6, \dots$ and all its non trivial zeros are inside the critical strip $0 < s < 1$ and they all lie on the critical line $s = \frac{1}{2}$ . $ζ (s)$ is also deﬁned by integral formula

ζ (s) = \frac{1}{Γ (s)} \int_{0}^{\infty} \frac{1}{e^{t} - 1} \frac{t^{s}}{t} d t Re (s) > 1

$◼$ The connection between $ζ (s)$ prime numbers is given by the Euler product formula

\begin{aligned} ζ (s) & = Π_{p} \frac{1}{1 - p^{- s}} \\ = (\frac{1}{1 - 2^{- s}}) (\frac{1}{1 - 3^{- s}}) (\frac{1}{1 - 5^{- s}}) (\frac{1}{1 - 7^{- s}}) \dots \\ = (\frac{1}{1 - \frac{1}{2^{s}}}) (\frac{1}{1 - \frac{1}{3^{s}}}) (\frac{1}{1 - \frac{1}{5^{s}}}) (\frac{1}{1 - \frac{1}{7^{s}}}) \dots \\ = (\frac{2^{s}}{2^{s} - 1}) (\frac{3^{s}}{3^{s} - 1}) (\frac{5^{s}}{5^{s} - 1}) (\frac{7^{s}}{7^{s} - 1}) \dots \end{aligned}

$◼$ $ζ (s)$ functional equation is

ζ (s) = 2^{s} π^{s - 1} \sin (\frac{π s}{2}) Γ (1 - s) ζ (1 - s)

17 Complex functions notes

$◼$ Complex identities

\begin{aligned} {| z |}^{2} & = z \bar{z} \\ \overset{―}{(\bar{z})} & = z \\ \overset{―}{(z_{1} + z_{2})} & = {\bar{z}}_{1} + {\bar{z}}_{2} \\ | \bar{z} | & = | z | \\ | z_{1} z_{2} | & = | z_{1} | | z_{2} | \\ Re (z) & = \frac{z + \bar{z}}{2} \\ Im (z) & = \frac{z + \bar{z}}{2 i} \\ \arg (z_{1} z_{2}) & = \arg (z_{1}) + \arg (z_{2}) \end{aligned}

$◼$ A complex function $f (z)$ is analytic in a region $D$ if it is deﬁned and diﬀerentiable at all points in $D$ . One way to check for analyticity is to use the Cauchy Riemann (CR) equations (this is a necessary condition but not suﬃcient). If $f (z)$ satisﬁes CR everywhere in that region then it is analytic. Let $f (z) = u (x, y) + i v (x, y)$ , then these two equations in Cartesian coordinates are

\begin{aligned} \frac{\partial u}{\partial x} & = \frac{\partial v}{\partial y} \\ - \frac{\partial u}{\partial y} & = \frac{\partial v}{\partial x} \end{aligned}

Sometimes it is easier to use the polar form of these. Let $f (z) = r \cos θ + i \sin θ$ , then the equations become

\begin{aligned} \frac{\partial u}{\partial r} & = \frac{1}{r} \frac{\partial v}{\partial θ} \\ - \frac{1}{r} \frac{\partial u}{\partial θ} & = \frac{\partial v}{\partial r} \end{aligned}

To remember them, think of the $r$ as the $x$ and $θ$ as the $y$ .

Let us apply these on $\sqrt{z}$ to see how it works. Since $z = r e^{i θ + 2 n π}$ then $f (z) =$ $\sqrt{r} e^{i \frac{θ}{2} + n π}$ .This is multi-valued function. One value for $n = 0$ and another for $n = 1$ . The ﬁrst step is to make it single valued. Choosing $n = 0$ gives the principal value. Then $f (z) = \sqrt{r} e^{i \frac{θ}{2}}$ . Now we ﬁnd the branch points. $z = 0$ is a branch point. We can pick $- π < θ < π$ and pick the negative real axis as the branch cut (the other branch point being $- \infty$ ). This is one choice.

We could have picked $0 < θ < 2 π$ and had the positive $x$ axis as the branch cut, where now the second branch point is $+ \infty$ but in both cases, origin is still part of the branch cut. Let us stick with $- π < θ < π$ .

Given all of this, now $\sqrt{z} = \sqrt{r} e^{i \frac{θ}{2}} = \sqrt{r} (\cos (\frac{θ}{2}) + i \sin (\frac{θ}{2}))$ , hence $u = \sqrt{r} \cos (\frac{θ}{2})$ and $v = \sqrt{r} \sin (\frac{θ}{2})$ . Therefore $\frac{\partial u}{\partial r} = \frac{1}{2} \frac{1}{\sqrt{r}} \cos (\frac{θ}{2}),$ and $\frac{\partial v}{\partial θ} = \frac{1}{2} \sqrt{r} \cos (\frac{θ}{2})$ and $\frac{\partial u}{\partial θ} = - \frac{1}{2} \sqrt{r} \sin (\frac{θ}{2})$ and $\frac{\partial v}{\partial r} = \frac{1}{2} \frac{1}{\sqrt{r}} \sin (\frac{θ}{2})$ . Applying Cauchy-Riemann above gives

\begin{aligned} \frac{1}{2} \frac{1}{\sqrt{r}} \cos (\frac{θ}{2}) & = \frac{1}{r} \frac{1}{2} \sqrt{r} \cos (\frac{θ}{2}) \\ \frac{1}{2} \frac{1}{\sqrt{r}} \cos (\frac{θ}{2}) & = \frac{1}{2} \frac{1}{\sqrt{r}} \cos (\frac{θ}{2}) \end{aligned}

Satisﬁed. and for the second equation

\begin{aligned} - \frac{1}{r} (- \frac{1}{2} \sqrt{r} \sin (\frac{θ}{2})) & = \frac{1}{2} \frac{1}{\sqrt{r}} \sin (\frac{θ}{2}) \\ \frac{1}{2} \frac{1}{\sqrt{r}} \sin (\frac{θ}{2}) & = \frac{1}{2} \frac{1}{\sqrt{r}} \sin (\frac{θ}{2}) \end{aligned}

so $\sqrt{z}$ is analytic in the region $- π < θ < π$ , and not including branch points and branch cut.

$◼$ We can’t just say $f (z)$ is Analytic and stop. Have to say $f (z)$ is analytic in a region or at a point. When we say $f (z)$ analytic at a point, we mean analytic in small region around the point.

If $f (z)$ is deﬁned only at an isolated point $z_{0}$ and not deﬁned anywhere around it, then the function can not be analytic at $z_{0}$ since it is not diﬀerentiable at $z_{0}$ . Also $f (z)$ is analytic at a point $z_{0}$ if the power series for $f (z)$ expanded around $z_{0}$ converges to $f (z)$ evaluated at $z_{0}$ . An analytic complex function mean it is inﬁnitely many times diﬀerentiable in the region, which means the limit exist $lim_{Δ z \to 0} \frac{f (z + Δ z) - f (z)}{Δ z}$ and does not depend on direction.

$◼$ Before applying the Cauchy Riemann equations, make sure the complex function is ﬁrst made to be single valued.

$◼$ Remember that Cauchy Riemann equations as necessary but not suﬃcient condition for function to be analytic. The extra condition needed is that all the partial derivatives are continuous. Need to ﬁnd example where CR is satisﬁed but not the continuity on the partial derivatives. Most of the HW problems just needs the CR but good to keep an eye on this other condition.

$◼$ Cauchy-Goursat: If $f (z)$ is analytic on and inside closed contour $C$ then $\oint_{C} f (z) d z = 0$ . But remember that if $\oint_{C} f (z) d z = 0$ then this does not necessarily imply $f (z)$ is analytic on and inside $C$ . So this is an IF and not an IFF relation. For example $\oint_{C} \frac{1}{z^{2}} d z = 0$ around unit circle centered at origin, but clearly $\frac{1}{z^{2}}$ is not analytic everywhere inside $C$ , since it has a singularity at $z = 0$ .

proof of Cauchy-Goursat: The proof uses two main ideas. It uses the Cauchy-Riemann equations and also uses Green theorem. Green’s Theorem says

\begin{matrix} (1) & \int_{C} P d x + Q d y = \int_{D} (\frac{\partial Q}{\partial x} - \frac{\partial P}{\partial y}) d A \end{matrix}

So Green’s Theorem transforms integration on the boundary $C$ of region $D$ by integration over the area inside the boundary $C$ . Let $f (z) = u + i v$ . And since $z = x + i y$ then $d z = d x + i d y$ . Therefore

\begin{aligned} \oint_{C} f (z) d z & = \oint_{C} (u + i v) (d x + i d y) \\ = \oint_{C} u d x + u i d y + i v d x - v d y \\ (2) & = \oint_{C} (u d x - v d y) + i \oint_{C} v d x + u d y \end{aligned}

We now apply (1) to each of the two integrals in (3). Hence the ﬁrst integral in (2) becomes

\oint_{C} (u d x - v d y) = \int_{D} (- \frac{\partial v}{\partial x} - \frac{\partial u}{\partial y}) d A

But from CR, we know that $- \frac{\partial u}{\partial y} = \frac{\partial v}{\partial x}$ , hence the above is zero. And the second integral in (2) becomes

\oint_{C} v d x + u d y = \int_{D} (\frac{\partial u}{\partial x} - \frac{\partial v}{\partial y}) d A

But from CR, we know that $\frac{\partial u}{\partial x} = \frac{\partial v}{\partial y}$ , hence the above is zero. Therefore the whole integral in (2) is zero. Therefore $\oint_{C} f (z) d z = 0$ . QED.

$◼$ Cauchy residue: If $f (z)$ is analytic on and inside closed contour $C$ except at some isolated points $z_{1}, z_{2}, \dots, z_{N}$ then $\oint_{C} f (z) d z = 2 π i \sum_{j = 1}^{N} Res {(f (z))}_{z = z_{j}}$ . The term $Res {(f (z))}_{z = z_{j}}$ is the residue of $f (z)$ at point $z_{j}$ . Use Laurent expansion of $f (z)$ to ﬁnd residues. See above on methods how to ﬁnd Laurent series.

$◼$ Maximum modulus principle: If $f (z)$ is analytic in some region $D$ and is not constant inside $D$ , then its maximum value must be on the boundary. Also its minimum on the boundary, as long as $f (z) \neq 0$ anywhere inside $D$ . In the other hand, if $f (z)$ happened to have a maximum at some point $z_{0}$ somewhere inside $D$ , then this implies that $f (z)$ is constant everywhere and will have the value $f (z_{0})$ everywhere. What all this really mean, is that if $f (z)$ is analytic and not constant in $D$ , then its maximum is on the boundary and not inside.

There is a complicated proof of this. See my notes for Physics 501. Hopefully this will not come up in the exam since I did not study the proof.

$◼$ These deﬁnitions from book of Joseph Bak

$f$ is analytic at $z$ if $f$ is diﬀerentiable in a neighborhood of $z$ . Similarly $f$ is analytic on set $S$ if $f$ is diﬀerentiable at all points in some open set containing $S$ .
$f (z)$ is analytic on open set $U$ is $f (z)$ if diﬀerentiable at each point of $U$ and $f^{'} (z)$ is continuous on $U$ .

$◼$ Some important formulas.

If $f (z)$ is analytic on and inside $C$ then
$\oint_{C} f (z) d z = 0$
If $f (z)$ is analytic on and inside $C$ then and $z_{0}$ is a point in $C$ then
$\begin{aligned} 2 π i f (z_{0}) & = \oint_{C} \frac{f (z)}{z - z_{0}} d z \\ 2 π i f^{'} (z_{0}) & = \oint_{C} \frac{f (z)}{{(z - z_{0})}^{2}} d z \\ \frac{2 π i}{2!} f^{''} (z_{0}) & = \oint_{C} \frac{f (z)}{{(z - z_{0})}^{3}} d z \\ ⋮ \\ \frac{2 π i}{n!} f^{(n)} (z_{0}) & = \oint_{C} \frac{f (z)}{{(z - z_{0})}^{n + 1}} d z \end{aligned}$
From the above, we ﬁnd, where here $f (z) = 1$
$\oint_{C} \frac{1}{{(z - z_{0})}^{n + 1}} d z = {\begin{array}{ccc} 2 π i & n = 0 \\ 0 & n = 1, 2, \dots \end{array}$

17.1 Find $b_{n}$ coeﬃcients in the Laurent series expansion

On Finding coeﬃcient of the principle part of the Laurent series expansion around $z_{0}$ . Let

\begin{aligned} (1) & f (z) & = \sum_{n = 0}^{\infty} c_{n} {(z - z_{0})}^{n} + \sum_{n = 1}^{N} \frac{b_{n}}{{(z - z_{0})}^{n}} \\ = \sum_{n = 0}^{\infty} c_{n} {(z - z_{0})}^{n} + \frac{b_{1}}{(z - z_{0})} + \frac{b_{2}}{{(z - z_{0})}^{2}} + \frac{b_{3}}{{(z - z_{0})}^{3}} + \dots + \frac{b_{N}}{{(z - z_{0})}^{N}} \end{aligned}

The goal is to determine all the coeﬃcients $b_{1}, b_{2}, \dots, b_{N}$ in Laurent series expansion. This assumes the largest order of the pole is ﬁnite. To ﬁnd $b_{1}$ , we multiply both side of the above by ${(z - z_{0})}^{N}$ which gives

\begin{matrix} (2) & {(z - z_{0})}^{N} f (z) = \sum_{n = 0}^{\infty} c_{n} {(z - z_{0})}^{n + N} + b_{1} {(z - z_{0})}^{N - 1} + b_{2} {(z - z_{0})}^{N - 2} + b_{3} {(z - z_{0})}^{N - 3} + \dots + b_{N} \end{matrix}

Diﬀerentiating both sides $N - 1$ times w.r.t. $z$ gives

\frac{d^{N - 1}}{d z^{(N - 1)}} ({(z - z_{0})}^{N} f (z)) = \sum_{n = 0}^{\infty} \frac{d^{N - 1}}{d z^{(N - 1)}} (c_{n} {(z - z_{0})}^{n + N}) + b_{1} (N - 1)!

Evaluating at $x = x_{0}$ the above gives

b_{1} = \frac{lim_{z \to z_{0}} \frac{d^{N - 1}}{d z^{(N - 1)}} ({(z - z_{0})}^{N} f (z))}{(N - 1)!}

To ﬁnd $b_{2}$ we diﬀerentiate both sides of (2) $N - 2$ times which gives

\frac{d^{N - 2}}{d z^{(N - 2)}} ({(z - z_{0})}^{N} f (z)) = \sum_{n = 0}^{\infty} \frac{d^{N - 2}}{d z^{(N - 2)}} (c_{n} {(z - z_{0})}^{n + N}) + b_{1} (N - 1)! (x - x_{0}) + b_{2} (N - 2)!

Hence

b_{2} = \frac{lim_{z \to z_{0}} \frac{d^{N - 2}}{d z^{(N - 2)}} ({(z - z_{0})}^{N} f (z))}{(N - 2)!}

We keep doing the above to ﬁnd $b_{3}, b_{4}, \dots, b_{N}$ . Therefore the general formula is

\begin{matrix} (3A) & b_{n} = \frac{lim_{z \to z_{0}} \frac{d^{N - n}}{d z^{(N - n)}} ({(z - z_{0})}^{N} f (z))}{(N - n)!} \end{matrix}

And for the special case of the last term $b_{N}$ the above simpliﬁes to

\begin{matrix} (3B) & b_{k} = \frac{lim_{z \to z_{0}} {(z - z_{0})}^{N} f (z)}{(N - k)!} \end{matrix}

Where in (3) $n$ is the coeﬃcient $b_{n}$ needed to be evaluated and $N$ is the pole order and $z_{0}$ is the expansion point. The special value $b_{1}$ is called the residue of $f (z)$ at $z_{0}$ .

18 Hints to solve some problems

18.1 Complex analysis and power and Laurent series

Laurent series of $f (z)$ around point $z_{0}$ is $\sum_{n = - \infty}^{\infty} a_{n} {(z - z_{0})}^{n}$ and $a_{n} = \frac{1}{2 π i} \oint \frac{f (z)}{{(z - z_{0})}^{n + 1}} d z$ . Integration is around path enclosing $z_{0}$ in counter clockwise.
Power series of $f (z)$ around $z_{0}$ is $\sum_{0}^{\infty} a_{n} {(z - z_{0})}^{n}$ where $a_{n} = \frac{1}{n!} {f^{(n)} (z) |}_{z = z_{0}}$
Problem asks to use Cauchy integral formula $\oint_{C} \frac{f (z)}{z - z_{0}} d z = 2 π i f (z_{0})$ to evaluate another integral $\oint_{C} g (z) d z$ . Both over same $C$ . The idea is to rewrite $g (z)$ as $\frac{f (z)}{z - z_{0}}$ by factoring out the poles of $g (z)$ that are outside $C$ leaving one inside $C$ . Then we can write
$\begin{aligned} \oint_{C} g (z) d z & = \oint_{C} \frac{f (z)}{z - z_{0}} d z \\ = 2 π i f (z_{0}) \end{aligned}$

For example, to solve $\oint_{C} \frac{1}{(z + 1) (z + 2)} d z$ around $C$ unit circle. Rewriting this as $\oint_{C} \frac{\frac{1}{z + 2}}{(z - (- 1))} d z$ where now $f (z) = \frac{1}{z + 2}$ and now we can use Cauchy integral formula. So all what we have to do is just evaluate $\frac{1}{z + 2}$ at $z = - 1$ , which gives $\oint_{C} \frac{1}{(z + 1) (z + 2)} d z = 2 π i$ . This works if $g (z)$ can be factored into $\frac{f (z)}{z - z_{0}}$ where $f (z)$ is analytic on and inside $C$ . This would not work if $g (z)$ has more than one pole inside $C$ .
Problem asks to ﬁnd $\oint_{C} f (z) d z$ where $C$ is some closed contour. For this, if $f (z)$ had number of isolated singularities inside $C$ , then just use
$\oint_{C} f (z) d z = 2 π i \sum residues of f (z) at each singularity inside C$
Problem asks to ﬁnd $\int_{C} f (z) d z$ where $C$ is some open path, i.e. not closed (if it is closed, try Cauchy), such as a straight line or a half circle arc. For these problem, use parameterization. This converts the integral to line integration. If $C$ is straight line, use standard $t$ parameterization, which is found by using
$\begin{aligned} x (t) & = (1 - t) x_{0} + t x_{1} \\ y (t) & = (1 - t) y_{0} + t y_{1} \end{aligned}$

where $(x_{0}, y_{0})$ in the line initial point and $(x_{1}, y_{1})$ is the line end point. This works for straight lines. Now use the above and rewrite $z = x + i y$ as $z (t) = x (t) + i y (t)$ and then plug-in in this $z (t)$ in $f (z)$ to obtain $f (t)$ , then the integral becomes
$\int_{C} f (z) d z = \int_{t = 0}^{t = 1} f (t) z^{'} (t) d t$
And now evaluate this integral using normal integration rules. If the path is a circular arc, then no need to use $t$ , just use $θ$ . Rewrite $x = r e^{i θ}$ and use $θ$ instead of $t$ and follow same steps as above.
Problem gives $u (x, y)$ and asks to ﬁnd $v (x, y)$ in order for $f (x, y) = u (x, y) + i v (x, y)$ to be analytic in some region. To solve these, use Cauchy Riemann equations. Need to use both equations. One equation will introduce a constant of integration (a function) and the second equation is used to solve for it. This gives $v (x, y)$ . See problem 2, HW 2, Physics 501 as example.
Problem asks to evaluate $\oint_{C} \frac{f (z)}{{(z - z_{0})}^{n}} d z$ where $n$ is some number. This is the order of the pole, and $f (z)$ is analytic on and inside $C$ . Then use the Cauchy integral formula for higher pole order. $\oint_{C} \frac{f (z)}{{(z - z_{0})}^{n}} d z = 2 π i Residue (z_{0})$ . The only diﬀerence here is that this is pole of order $n$ . So to ﬁnd residue, use
$\begin{aligned} Residue (z_{0}) & = lim_{z \to z_{0}} \frac{d^{n - 1}}{d z^{n}} \frac{{(z - z_{0})}^{n}}{(n - 1)!} \frac{f (z)}{{(z - z_{0})}^{n}} \\ = lim_{z \to z_{0}} \frac{d^{n - 1}}{d z^{n}} \frac{f (z)}{(n - 1)!} \end{aligned}$
Problem gives $f (z)$ and asks to ﬁnd branch points and branch cuts. One way is to ﬁrst ﬁnd where $f (z) = 0$ and for each zero, make a small circle around it, starting from $θ = 0$ to $θ = 2 π$ . If the function at $θ = 0$ has diﬀerent value from $θ = 2 π$ , then this is a branch point. Do this for other zeros. Then connect the branch points. This will give the branch cut. It is not always clear how to connect the branch point though, might need to try diﬀerent ways. For example $f (z) = \sqrt{z^{2} + 1}$ has two zeros at $z = \pm i$ . Both turn out to be branch points. The branch cut is the line between $- i$ to $+ i$ on the imaginary axis.
Problem gives a series $\sum_{n = 0}^{\infty} a_{n} z^{n}$ and asks to ﬁnd radius of convergence $R$ . Two ways, ﬁnd $L = lim_{n \to \infty} \frac{| a_{n + 1} |}{| a_{n} |}$ and then $R = \frac{1}{L}$ . Another way is to ﬁnd $L$ using $L = lim_{n \to \infty} {| a_{n} |}^{\frac{1}{n}}$ .
Problem gives integral $\int_{0}^{2 π} f (θ) d θ$ and asks to evaluate using residues. We start by converting everything to $z$ using $z = e^{i θ}$ using $| z | = 1$ . No need to use $z = r e^{i θ}$ . The idea is to convert it to $\oint f (z) d z$ which then we can use $\oint f (z) d z = 2 π i \sum$ residues inside. Replace $f (θ)$ to become $f (z)$ , this could require using Euler relation such as $\cos n θ = \frac{z^{n} + z^{- n}}{2}$ and similar for $\sin θ$ . Now all what is needed is to ﬁnd residues of any poles inside the unit circle. Do not worry about poles outside the unit circle. To ﬁnd residues use short cut tricks. No need to ﬁnd Laurent series.
For an example, to evaluate $\int_{0}^{2 π} \frac{1}{5 + 4 \cos θ} d θ$ , then $\frac{1}{5 + 4 \cos θ}$ becomes $\frac{1}{(2 z + 1) (z + 2)}$ and there is only one pole inside unit circle, at $z = - \frac{1}{2}$ .
Problem gives integral $\int_{0}^{\infty} f (θ) d θ$ and asks to evaluate using residues. The contour here goes from $- R$ to $+ R$ and then a semi circle in upper half plane. This works for even $f (θ)$ since we can write $\int_{0}^{\infty} f (θ) d θ = \frac{1}{2} \int_{- \infty}^{\infty} f (θ) d θ$ . If there is a pole inside the upper half plane, then the integral over the semi circle is $2 π i$ times the sum of residues. If there is a pole on the real line, then make a small semi circle around pole, say at $z = a$ and then the integral for the small semi circle is $- π i$ times the residue at $a$ . The minus sign here is due to moving clock wise on the small circle.
Problem gives a series $\sum_{n = 0}^{\infty} a_{n} z^{n}$ and asks if it is uniformly convergent. For general series, use the M-test. But for this kind of series, just ﬁnd radius of convergence as above using ratio test, and if it is absolutely convergent, then say it converges uniformly for $| z | \leq r < R$ . It is important to write it this way, and not just $| z | < R$ .
Problems gives $\sum_{n = 0}^{\infty} a_{n}$ and asks to ﬁnd the sum. Sometimes this trick works for some series. For example the alternating series $\sum_{n = 1}^{\infty} {(- 1)}^{n + 1} \frac{1}{n} = 1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots$ , then write it as $x - \frac{x^{2}}{2} + \frac{x^{3}}{3} - \frac{x^{4}}{4} + \dots$ which is the same when $x = 1$ , and now notice that this is the Taylor series for $\ln (1 + x)$ which means when $x = 1$ then $1 - \frac{1}{2} + \frac{1}{3} - \frac{1}{4} + \dots = \ln (2)$ .
Problem gives $f (z)$ and asks to ﬁnd residue at some $z = z_{0}$ . Of course we can always expand $f (z)$ around $z = 0$ using Laurent series and ﬁnd the coeﬃcient of $\frac{1}{z}$ . But this is too much work. Instead, if $f (z)$ has a simple pole of order one, then we use
$R (z_{0}) = lim_{z \to z_{0}} (z - z_{0}) f (z)$
In general, if $f (z) = \frac{g (z)}{h (z)}$ then there are two cases. If $h (z_{0}) = 0$ or not. If $h (z_{0}) \neq 0$ , then we can just use the above. For example, if $f (z) = \frac{z}{(2 z + 1) (5 - z)}$ and we want the residue at $z_{0} = 5$ , then since it simple pole, then using
$\begin{aligned} R (5) & = lim_{z \to 5} (z - 5) \frac{z}{(2 z + 1) (5 - z)} \\ = lim_{z \to 5} \frac{- z}{(2 z + 1)} \\ = - \frac{3}{11} \end{aligned}$

But if $h (z_{0}) = 0$ then we need to apply La’Hopital like this. If $f (z) = \frac{\sin z}{1 - z^{4}}$ and we want to ﬁnd residue at $z = i$ . Then do as above, but with extra step, like this
$\begin{aligned} R (i) & = lim_{z \to i} (z - i) \frac{\sin z}{1 - z^{4}} \\ = (lim_{z \to i} \sin z) (lim_{z \to i} (z - i) \frac{1}{1 - z^{4}}) \\ = \sin i (lim_{z \to i} \frac{(z - i)}{1 - z^{4}}) Now apply La’Hopital \\ = \sin i (lim_{z \to i} \frac{1}{- 4 z^{3}}) \\ = \frac{\sin i}{- 4 i^{3}} \\ = \frac{1}{4} \sinh (1) \end{aligned}$

Now if the pole is not a simple pole or order one,.say of order $m$ , then we ﬁrst multiply $f (z)$ by ${(z - z_{0})}^{m}$ then diﬀerentiate the result $m - 1$ times, then divide by $(m - 1)!$ , and then evaluate the result at $z = z_{0} .$ in other words,
$R (z_{0}) = lim_{z \to z_{0}} \frac{1}{(m - 1)!} \frac{d^{m - 1}}{d z^{m - 1}} ({(z - z_{0})}^{m} f (z))$
For example, if $f (z) = \frac{z \sin z}{{(z - π)}^{3}}$ and we want residue at $z = π$ . Since order is $m = 3$ , then
$\begin{aligned} R (z_{0}) & = lim_{z \to π} \frac{1}{2!} \frac{d^{2}}{d z^{2}} ({(z - π)}^{3} \frac{z \sin z}{{(z - π)}^{3}}) \\ = lim_{z \to π} \frac{1}{2} \frac{d^{2}}{d z^{2}} (z \sin z) \\ = lim_{z \to π} \frac{1}{2} (- z \sin z + 2 \cos z) \\ = - 1 \end{aligned}$

The above methods will work on most of the HW problems I’ve seen so far but If all else fails, try Laurent series, that always works.

18.2 Errors and relative errors

A problem gives an expression in $x, y$ such as $f (x, y)$ and asks how much a relative error in both $x$ and $y$ will aﬀect $f (x, y)$ in worst case. For these problems, ﬁnd $d f$ and then ﬁnd $\frac{d f}{f}$ . For example, if $f (x, y) = \sqrt{\frac{x}{y^{3}}}$ and relative error is in $x$ and $y$ is $2 %$ then what is worst relative error in $f (x, y) ?$ . Then since
$\begin{aligned} d f & = \frac{\partial f}{\partial x} d x + \frac{\partial f}{\partial y} d y \\ = \frac{1}{2} x^{- \frac{1}{2}} b^{- \frac{3}{2}} d x - \frac{3}{2} x^{\frac{1}{2}} y^{- \frac{5}{2}} d y \end{aligned}$

Then
$\frac{d f}{f} = \frac{1}{2} \frac{d x}{x} - \frac{3}{2} \frac{d y}{y}$
But $\frac{d x}{x}$ and $\frac{d y}{y}$ are the relative errors in $x$ and $y$ . So if we plug-in $2$ for $\frac{d x}{x}$ and $- 2$ for $\frac{d y}{y}$ we get $4 %$ is worst relative error in $f (x, y)$ . Notice we used $- 2 %$ relative error for $y$ and $+ 2 %$ relative error for $x$ since we wanted the worst (largest) relative error. If we wanted the least relative error in $f (x, y)$ , then we will use $+ 2 %$ for $y$ also, which gives $1 - 3 = - 2$ or $2 %$ relative error in $f (x, y)$ .

19 Some CAS notes

$◼$ in Mathematica Exp is a symbol. Head[Exp] gives Symbol but in Maple it is not.

In Maple

indets(z^2-exp(x^2-1)+1+Pi+Gamma*foo()-sin(y),'name');

gives ${Γ, π, x, y, z}$ but in Mathematica

expr=z^2-Exp[x^2-1]+1+Pi+Gamma*foo[]-Sin[y]; 
Cases[expr,_Symbol,Infinity]

gives ${e, x, π, z, Gamma, y}$

Notice that $e$ shows up in Mathematica, but not in Maple.

20 d’Alembert’s Solution to wave PDE

(added December 13, 2018)

The PDE is

\begin{matrix} (1) & \frac{\partial^{2} ψ}{\partial t^{2}} = c^{2} \frac{\partial^{2} ψ}{\partial x^{2}} \end{matrix}

Let

\begin{aligned} u & = x - c t \\ v & = x + c t \end{aligned}

Then

\begin{aligned} \frac{\partial ψ}{\partial t} & = \frac{\partial ψ}{\partial u} \frac{\partial u}{\partial t} + \frac{\partial ψ}{\partial v} \frac{\partial v}{\partial t} \\ (2) & = - c \frac{\partial ψ}{\partial u} + c \frac{\partial ψ}{\partial v} \end{aligned}

And

\begin{aligned} \frac{\partial ψ}{\partial x} & = \frac{\partial ψ}{\partial u} \frac{\partial u}{\partial x} + \frac{\partial ψ}{\partial v} \frac{\partial v}{\partial x} \\ (3) & = \frac{\partial ψ}{\partial u} + \frac{\partial ψ}{\partial v} \end{aligned}

Then, from (2)

\begin{aligned} \frac{\partial^{2} ψ}{\partial t^{2}} & = - c (\frac{\partial^{2} ψ}{\partial u^{2}} \frac{\partial u}{\partial t} + \frac{\partial^{2} ψ}{\partial u \partial v} \frac{\partial v}{\partial t}) + c (\frac{\partial^{2} ψ}{\partial v^{2}} \frac{\partial v}{\partial t} + \frac{\partial^{2} ψ}{\partial v \partial u} \frac{\partial u}{\partial t}) \\ = - c (- c \frac{\partial^{2} ψ}{\partial u^{2}} + c \frac{\partial^{2} ψ}{\partial u \partial v}) + c (c \frac{\partial^{2} ψ}{\partial v^{2}} - c \frac{\partial^{2} ψ}{\partial v \partial u}) \\ = c^{2} \frac{\partial^{2} ψ}{\partial u^{2}} - c^{2} \frac{\partial^{2} ψ}{\partial u \partial v} + c^{2} \frac{\partial^{2} ψ}{\partial v^{2}} - c^{2} \frac{\partial^{2} ψ}{\partial v \partial u} \\ (4) & = c^{2} \frac{\partial^{2} ψ}{\partial u^{2}} + c^{2} \frac{\partial^{2} ψ}{\partial v^{2}} - 2 c^{2} \frac{\partial^{2} ψ}{\partial v \partial u} \end{aligned}

And from (3)

\begin{aligned} \frac{\partial^{2} ψ}{\partial x^{2}} & = (\frac{\partial^{2} ψ}{\partial u^{2}} \frac{\partial u}{\partial x} + \frac{\partial^{2} ψ}{\partial u \partial v} \frac{\partial v}{\partial x}) + (\frac{\partial^{2} ψ}{\partial v^{2}} \frac{\partial v}{\partial x} + \frac{\partial^{2} ψ}{\partial v \partial u} \frac{\partial u}{\partial x}) \\ = (\frac{\partial^{2} ψ}{\partial u^{2}} + \frac{\partial^{2} ψ}{\partial u \partial v}) + (\frac{\partial^{2} ψ}{\partial v^{2}} + \frac{\partial^{2} ψ}{\partial v \partial u}) \\ (5) & = \frac{\partial^{2} ψ}{\partial u^{2}} + \frac{\partial^{2} ψ}{\partial v^{2}} + 2 \frac{\partial^{2} ψ}{\partial v \partial u} \end{aligned}

Substituting (4,5) into (1) gives

\begin{aligned} - 2 c^{2} \frac{\partial^{2} ψ}{\partial v \partial u} & = 2 c^{2} \frac{\partial^{2} ψ}{\partial v \partial u} \\ - 4 c^{2} \frac{\partial^{2} ψ}{\partial v \partial u} & = 0 \end{aligned}

Since $c \neq 0$ then

\frac{\partial^{2} ψ}{\partial v \partial u} = 0

Integrating w.r.t $v$ gives

\frac{\partial ψ}{\partial u} = f (u)

Integrating w.r.t $u$

ψ (x, t) = F (u) + G (v)

Therefore

\begin{matrix} (6) & ψ (x, t) = F (x - c t) + G (x + c t) \end{matrix}

The functions $F (x, t), G (x, t)$ are arbitrary functions found from initial and boundary conditions if given. Let initial conditions be

\begin{aligned} ψ (x, 0) & = f_{0} (x) \\ \frac{\partial}{\partial t} ψ (x, 0) & = g_{0} (x) \end{aligned}

Where the ﬁrst condition above is the shape of the string at time $t = 0$ and the second condition is the initial velocity.

Applying ﬁrst condition to (6) gives

\begin{matrix} (7) & f_{0} (x) = F (x) + G (x) \end{matrix}

Applying the second condition gives

\begin{aligned} g_{0} (x) & = {[\frac{\partial}{\partial t} F (x - c t)]}_{t = 0} + {[\frac{\partial}{\partial t} G (x + c t)]}_{t = 0} \\ = {[\frac{d F (x - c t)}{d (x - c t)} \frac{\partial (x - c t)}{\partial t}]}_{t = 0} + {[\frac{d G (x + c t)}{d (x + c t)} \frac{\partial (x + c t)}{\partial t}]}_{t = 0} \\ = {[- c \frac{d F (x - c t)}{d (x - c t)}]}_{t = 0} + {[c \frac{d G (x + c t)}{d (x + c t)}]}_{t = 0} \\ (8) & = - c \frac{d F (x)}{d x} + c \frac{d G (x)}{d x} \end{aligned}

Now we have two equations (7,8) and two unknowns $F, G$ to solve for. But the (8) has derivatives of $F, G$ . So to make it easier to solve, we integrate (8) w.r.t. to obtain

\begin{matrix} (9) & \int^{x} g_{0} (s) d s = - c F (x) + c G (x) \end{matrix}

So we will use (9) instead of (8) with (7) to solve for $F, G$ . From (7)

\begin{matrix} (10) & F (x) = f_{0} (x) - G (x) \end{matrix}

Substituting (10) in (9) gives

\begin{aligned} \int^{x} g_{0} (s) d s & = - c (f_{0} (x) - G (x)) + c G (x) \\ = - c f_{0} (x) + 2 c G (x) \\ G (x) & = \frac{(\int^{x} g_{0} (s) d s) + c f_{0} (x)}{2 c} \\ (11) & = \frac{1}{2 c} (\int^{x} g_{0} (s) d s + c f_{0} (x)) \end{aligned}

Using the above back in (10) gives $F (x)$ as

\begin{matrix} (12) & F (x) = f_{0} (x) - \frac{1}{2 c} (\int^{x} g_{0} (s) d s + c f_{0} (x)) \end{matrix}

Using (11,12) in (6) gives the ﬁnal solution

\begin{aligned} ψ (x, t) & = F (x - c t) + G (x + c t) \\ = f_{0} (x - c t) - \frac{1}{2 c} (\int^{x - c t} g_{0} (s) d s + c f_{0} (x - c t)) + \frac{1}{2 c} (\int^{x} g_{0} (s) d s + c f_{0} (x)) \\ = f_{0} (x - c t) - \frac{1}{2 c} \int^{x - c t} g_{0} (s) d s - \frac{1}{2} f_{0} (x - c t) + \frac{1}{2 c} \int^{x + c t} g_{0} (s) d s + \frac{1}{2} f_{0} (x + c t) \\ = \frac{1}{2} (f_{0} (x - c t) + f_{0} (x - c t)) + \frac{1}{2 c} \int_{x - c t}^{x + c t} g_{0} (s) d s \end{aligned}

The above is the ﬁnal solution. So if we are given initial position and initial velocity of the string as function of $x$ , we can ﬁnd exact solution to the wave PDE.

21 Convergence

$◼$ Deﬁnition of pointwise convergence: $f_{n} (x)$ converges pointwise to $f_{*} (x)$ if for each $ε > 0$ there exist integer $N (ε, x)$ such that $| f_{n} (x) - f_{*} (x) | < ε$ for all $n \geq N$ .

$◼$ Deﬁnition of uniform convergence: $f_{n} (x)$ converges uniformly to $f_{*} (x)$ if for each $ε > 0$ there exist integer $N (ε)$ such that $| f_{n} (x) - f_{*} (x) | < ε$ for all $n \geq N$ .

$◼$ Another way to ﬁnd uniform convergence, ﬁrst ﬁnd pointwise convergence of $f_{n} (x)$ . Say it converges to $f_{*} (x)$ . Now show that

‖ f_{n} - f_{*} ‖ = sup_{x \in I} (f_{n} - f_{*})

goes to zero as $n \to \infty$ . To ﬁnd $sup (f_{n} - f_{*})$ might need to ﬁnd the maximum of $f_{n} - f_{*}$ . i.e. diﬀerentiate this, set to zero, ﬁnd $x$ where it is Max, then evaluate $f_{n} (x) - f_{*} (x)$ at this maximum. This gives the $sup$ . Then see if this goes to zero as $n \to \infty$

$◼$ If sequence of functions $f_{n}$ converges uniformly to $f_{*}$ , then $f_{*}$ must be continuous. So this gives a quick check if uniform convergence exist. First ﬁnd the pointwise convergence $f_{*} (x)$ and check if this is continuous or not. If not, then no need to check for uniform convergence, it does not exist. But if $f_{*} (x)$ is continuous function, we still need to check because it is possible there is no uniform convergence.

22 Note on using when to raise ln to exp solving an ode

Sometimes in the middle of solving an ode, we get $\ln$ on both sides. We can raise both sides to $\exp$ as soon as these show up, or wait until the end, after solving the constant of integration to do that. This shows we get same result in both cases.

22.1 Example 1

\begin{matrix} (1) & y^{'} = 2 \frac{2 y x - x}{x + y} \end{matrix}

With initial conditions $y (0) = 2$ . This is homogenous type ode. It solved by substitution $u = \frac{y}{x}$ which results in the new ode in $u$ given by

u^{'} = \frac{1}{x} (\frac{- u^{2} + 3 u - 2}{1 + u})

This is now separable

\begin{aligned} \frac{d u}{d x} & = \frac{1}{x} (\frac{- u^{2} + 3 u - 2}{1 + u}) \\ \int \frac{1 + u}{- u^{2} + 3 u - 2} d u & = \frac{1}{x} d x \end{aligned}

Integrating gives

\begin{matrix} (1A) & 2 \ln (1 - u) - 3 \ln (2 - u) = \ln x + c \end{matrix}

Replacing $u$ by $\frac{y}{x}$ which gives

\begin{aligned} 2 \ln (1 - \frac{y}{x}) - 3 \ln (2 - \frac{y}{x}) & = \ln x + c \\ \ln (\frac{{(1 - \frac{y}{x})}^{2}}{{(2 - \frac{y}{x})}^{3}}) & = \ln x + c \\ \ln (\frac{\frac{1}{x^{2}} {(x - y)}^{2}}{\frac{1}{x^{3}} {(2 x - y)}^{3}}) & = \ln x + c \\ \ln (x \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}}) & = \ln x + c \\ (1B) & \ln x + \ln \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} & = \ln x + c \end{aligned}

$\ln x$ cancels out giving

\begin{matrix} (2) & \ln \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} = c \end{matrix}

Now lets try to solve for $c$ from IC $y (0) = 2$ . The above becomes

\begin{aligned} \ln (\frac{{(- 2)}^{2}}{{(- 2)}^{3}}) & = c \\ c & = \ln (\frac{4}{- 8}) \\ = \ln (- \frac{1}{2}) \end{aligned}

So the solution (2) is

\ln \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} = \ln (- \frac{1}{2})

And only now after $c$ is found, we raise both sides to $\exp$ (to simplify it) which gives the solution as

\frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} = \frac{- 1}{2}

\begin{matrix} (3) & \frac{{(x - y)}^{2}}{{(y - 2 x)}^{3}} = \frac{1}{2} \end{matrix}

Lets see what happens if we had raised both sides to $\exp$ earlier on, instead of waiting until after solving for the constant of integration. i.e. from step (1A) above

\begin{aligned} 2 \ln (1 - u) - 3 \ln (2 - u) & = \ln x + c \\ \ln \frac{{(1 - u)}^{2}}{{(2 - u)}^{3}} & = \ln x + c \\ \frac{{(1 - u)}^{2}}{{(2 - u)}^{3}} & = e^{\ln x + c} \\ \frac{{(1 - u)}^{2}}{{(2 - u)}^{3}} & = A x \end{aligned}

Where $A$ is new constant. And only now we replace $u$ by $\frac{y}{x}$ which gives

\begin{aligned} \frac{{(1 - \frac{y}{x})}^{2}}{{(2 - \frac{y}{x})}^{3}} & = A x \\ x \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} & = A x \\ (4) & \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} & = A \end{aligned}

Using IC $y (0) = 2$ . The above becomes

\begin{aligned} \frac{{(- 2)}^{2}}{{(- 2)}^{3}} & = A \\ A & = - \frac{1}{2} \end{aligned}

Hence (4) becomes

\begin{aligned} \frac{{(x - y)}^{2}}{{(2 x - y)}^{3}} & = - \frac{1}{2} \\ \frac{{(x - y)}^{2}}{{(y - 2 x)}^{3}} & = \frac{1}{2} \end{aligned}

Which is the same answer obtained earlier in (3). This shows both methods work. It might be better to delay the raising to exponential to the very end so it is all done in one place.

22.2 Example 2

\begin{aligned} (1) & y^{'} & = \frac{y^{2} - 2 x y - x^{2}}{y^{2} + 2 x y - x^{2}} \\ y (1) & = - 1 \end{aligned}

This is a homogenous ode, solved by the substitution $u = \frac{y}{x}$ which results in new ode in $u$ given by

u^{'} = \frac{1}{x} \frac{- u^{3} - u^{2} - u - 1}{u^{2} + 2 u - 1}

This is separable

\frac{u^{2} + 2 u - 1}{- u^{3} - u^{2} - u - 1} d u = \frac{1}{x} d x

Integrating gives

\begin{matrix} (1) & \ln (u + 1) - \ln (u^{2} + 1) = \ln (x) + c_{1} \end{matrix}

There are two choices now. Raise both sides to $\exp$ to simplify the $u$ solution or wait until the end. Option 1:

Replacing $u$ by $\frac{y}{x}$ in (1) gives

\begin{aligned} \ln (\frac{y}{x} + 1) - \ln ({(\frac{y}{x})}^{2} + 1) & = \ln (x) + c_{1} \\ \ln (\frac{\frac{y}{x} + 1}{{(\frac{y}{x})}^{2} + 1}) & = \ln (x) + c_{1} \\ \ln (\frac{\frac{1}{x} (y + x)}{\frac{1}{x^{2}} (y^{2} + x^{2})}) & = \ln (x) + c_{1} \\ \ln (x \frac{(y + x)}{(y^{2} + x^{2})}) & = \ln (x) + c_{1} \\ \ln x + \ln (\frac{y + x}{y^{2} + x^{2}}) & = \ln (x) + c_{1} \\ (2) & \ln (\frac{y + x}{y^{2} + x^{2}}) & = c_{1} \end{aligned}

Now lets try to solve for $c_{1}$ from IC $y (1) = - 1$ . The above becomes

\begin{aligned} \ln (\frac{0}{2}) & = c_{1} \\ c & = - \infty \end{aligned}

Hence (2) becomes

\ln (\frac{y + x}{y^{2} + x^{2}}) = - \infty

Now raising both sides to $\exp$ gives

\begin{aligned} \frac{y + x}{y^{2} + x^{2}} & = e^{- \infty} \\ \frac{y + x}{y^{2} + x^{2}} & = 0 \\ y + x & = 0 \\ y & = - x \end{aligned}

Lets try to see what happens if we raise to $\exp$ after solving for $u$ immeadilty which is the second option. From (1)

\ln (\frac{u + 1}{u^{2} + 1}) = \ln (x) + c_{1}

Raising both to $\exp$ gives

\frac{u + 1}{u^{2} + 1} = A x

Where $A$ new constant. Now we replace $u$ by $\frac{y}{x}$

\begin{aligned} (2) & \frac{\frac{y}{x} + 1}{{(\frac{y}{x})}^{2} + 1} & = A x \\ x \frac{y + x}{y^{2} + x^{2}} & = A x \\ \frac{y + x}{y^{2} + x^{2}} & = A \end{aligned}

Solving for $A$ from IC $y (1) = - 1$ from the above gives

\begin{aligned} \frac{0}{2} & = A \\ A & = 0 \end{aligned}

Hence the solution (2) becomes

\frac{\frac{y}{x} + 1}{{(\frac{y}{x})}^{2} + 1} = 0

\begin{aligned} \frac{y}{x} + 1 & = 0 \\ y & = - x \end{aligned}

So both method worked. The early one and the later on one. Both give same result.

22.3 Example 3

\begin{aligned} (1) & (x + 2 y) y^{'} & = 1 \\ y (0) & = - 1 \end{aligned}

This is tricky as how it is solved needs special handling of the initial conditions. Let us solve by subtituting $z = x + 2 y$ . Then $z^{'} = 1 + 2 y^{'}$ . The ode now becomes

\begin{aligned} z \frac{(z^{'} - 1)}{2} & = 1 \\ z^{'} - 1 & = \frac{2}{z} \\ z^{'} & = \frac{2}{z} + 1 \end{aligned}

This is separable

\frac{d z}{1 + \frac{2}{z}} = d x

Integrating

\begin{aligned} \int \frac{d z}{1 + \frac{2}{z}} & = \int d x \\ (1) & z - 2 \ln (2 + z) & = x + c \end{aligned}

We could raise both sides to $\exp$ now or wait until after converting back to $y$ . Lets look what happens in both cases. Raising to $\exp$ now gives

\begin{aligned} e^{z - 2 \ln (2 + z)} & = A e^{x} \\ \frac{e^{z}}{{(2 + z)}^{2}} & = A e^{x} \end{aligned}

But $z = x + 2 y$ and the above becomes

\begin{aligned} \frac{e^{x + 2 y}}{{(2 + x + 2 y)}^{2}} & = A e^{x} \\ (2) & \frac{e^{2 y}}{{(2 + x + 2 y)}^{2}} & = A \end{aligned}

Which is the correct solution. Now IC is used to ﬁnd $A$ . Using $y (0) = - 1$ the above becomes

\frac{e^{- 2}}{0} = A

So $A = \infty$ . Hence the solution (2) is

\frac{e^{2 y}}{{(2 + x + 2 y)}^{2}} = \infty

When this happens, to simplify the above we say that ${(2 + x + 2 y)}^{2} = 0$ or $2 + x + 2 y = 0$ . This gives $2 y = - 2 - x$ . Hence

y = - 1 - x

23 References

Too many references used, but will try to remember to start recording books used from now on. Here is current list

Applied partial diﬀerential equation, by Haberman
Advanced Mathematical Methods for Scientists and Engineers, Bender and Orszag, Springer.
Boundary value problems in physics and engineering, Frank Chorlton, Van Norstrand, 1969
Class notes. Math 322. University Wisconsin, Madison. Fall 2016. By Professor Smith. Math dept.
Mathematical methods in the physical sciences. Mary Boas, second edition.
Mathematical methods in physics and engineering. Riley, Hobson, Bence. Second edition.
various pages Wikipedia.
Mathworld at Wolfram.
Fourier series and boundary value problems 8th edition. James Brown, Ruel Churchill.
good note on Sturm-Liouville http://ramanujan.math.trinity.edu/rdaileda/teach/s12/m3357/lectures/lecture_4_10_short.pdf