7 · The Matrix Exponential
Chapter 7

The Matrix Exponential

Chapter 6 ended with a problem. To find the flow matrix Φ(𝑡) of a linear system 𝑋′ =𝐴(𝑡)𝑋, we had to solve 𝑁 separate initial value problems, one for each standard solution. And when the equations are intertwined, even one of them can be hard.

Let's zoom out once more. Instead of solving for the columns one at a time, treat the whole flow matrix as a single unknown. What equation does Φ satisfy? Each column of Φ is a standard solution, and each standard solution obeys the system. So we should expect the whole matrix to obey the system too.

Proposition 7.1 (The flow matrix solves a matrix equation). The flow matrix of 𝑋′ =𝐴(𝑡)𝑋 is the unique solution of

Φ′=𝐴(𝑡)Φ,Φ(𝑡0)=𝐼.

Proof. The columns of Φ are the standard solutions 𝑅1,…,𝑅𝑁, and we differentiate a matrix entry by entry, so we can work one column at a time:

Φ′=(𝑅′1 ⋯ 𝑅′𝑁)=(𝐴(𝑡)𝑅1 ⋯ 𝐴(𝑡)𝑅𝑁)=𝐴(𝑡)(𝑅1 ⋯ 𝑅𝑁)=𝐴(𝑡)Φ.

The second step uses that each 𝑅𝑗 solves the system. The third step pulls out 𝐴(𝑡): multiplying a matrix by 𝐴(𝑡) multiplies each of its columns by 𝐴(𝑡). At the starting time, each 𝑅𝑗(𝑡0) is 𝑒𝑗, so Φ(𝑡0) =𝐼.

Could another matrix-valued function also solve this problem? Its columns would solve 𝑋′ =𝐴(𝑡)𝑋, starting from 𝑒1,…,𝑒𝑁. Each of those initial value problems has only one solution, so its columns must be 𝑅1,…,𝑅𝑁. So it is Φ. ∎

The equation Φ′ =𝐴(𝑡)Φ is the system 𝑋′ =𝐴(𝑡)𝑋, applied to the whole flow at once. Each column of Φ records where one standard basis vector has been carried, and each one moves by the same rule as every other state. The condition Φ(𝑡0) =𝐼 says that at the start, nothing has moved. So instead of 𝑁 problems, we have one: find the matrix-valued function that starts at the identity and moves by the rule 𝐴(𝑡).

From here on, we take the coefficient matrix to be constant and start at 𝑡0 =0. The problem is then: given a constant 𝑁 ×𝑁 matrix 𝐴, find the matrix-valued function Φ with

Φ′=𝐴Φ,Φ(0)=𝐼,

directly from 𝐴. For the spring, with coefficient matrix 𝐽 =(01−10), example 6.19 tells us the answer should be the rotation matrix (cos⁡𝑡sin⁡𝑡−sin⁡𝑡cos⁡𝑡). Can we get it from 𝐽 alone?

7.1What Is an Exponential?

The problem Φ′ =𝐴Φ, Φ(0) =𝐼 has the same shape as 𝑦′ =𝑎𝑦, 𝑦(0) =1, whose solution is the exponential 𝑒𝑎𝑡. So we'd like an exponential of a matrix. But what would 𝑒𝑡𝐴 mean? We can't multiply 𝑒 by itself a matrix number of times.

Even for real numbers, that picture is incomplete. Multiplying 𝑒 by itself gives 𝑒3, and taking roots gives 𝑒1/2, but neither gives 𝑒√2. Multiplying and taking roots only ever reach rational exponents. So before asking how to extend the exponential to matrices, we should ask how it gets extended to all real numbers.

Calculus gives us another way to describe the exponential: its derivative is itself, and its value at zero is 1. Could these two properties serve as a definition? Consider the initial value problem

𝐸′=𝐸,𝐸(0)=1.

Our existence and uniqueness theorem for linear equations says that there is exactly one solution, defined for every real 𝑡. So this equation defines a function 𝐸, even before we have a formula for it. We'll call it the exponential, but we still owe ourselves an explanation of how it agrees with the powers and roots we already know.

First, can we find a formula? Try a power series,

𝐸(𝑡)=𝑐0+𝑐1𝑡+𝑐2𝑡2+𝑐3𝑡3+⋯.

Differentiating term by term gives

𝐸′(𝑡)=𝑐1+2𝑐2𝑡+3𝑐3𝑡2+⋯.

For 𝐸′ =𝐸, the coefficients of each power of 𝑡 must match. The coefficient of 𝑡𝑘 in 𝐸′ is (𝑘 +1)𝑐𝑘+1, and in 𝐸 it is 𝑐𝑘, so

(𝑘+1)𝑐𝑘+1=𝑐𝑘.

Each coefficient is the previous one divided by the next whole number. The initial condition gives 𝑐0 =1, and then

𝑐1=1,𝑐2=12,𝑐3=13⋅2,…,𝑐𝑘=1𝑘!.

The equation has written its own solution:

𝐸(𝑡)=∞∑𝑘=0𝑡𝑘𝑘!=1+𝑡+𝑡22!+𝑡33!+⋯.

So far this is a formal calculation: we differentiated an infinite series term by term without knowing that it converges. It does. For 𝑡 ≠0, consecutive terms have absolute ratio |𝑡|/(𝑘 +1), which tends to 0, so the ratio test gives convergence; at 𝑡 =0 the series is just 1. Calculus also tells us that a power series can be differentiated term by term inside its interval of convergence, which here is all of ℝ. So the series really does satisfy 𝐸′ =𝐸, and its value at 𝑡 =0 is 1. By uniqueness, it is the function we defined.

Now we have a formula. Why does this function deserve the name exponential? The key property of familiar powers is the law 𝑒𝑠+𝑡 =𝑒𝑠𝑒𝑡, and uniqueness proves the corresponding law for 𝐸. Fix 𝑠, and compare 𝐹(𝑡) =𝐸(𝑠 +𝑡) with 𝐺(𝑡) =𝐸(𝑠)𝐸(𝑡). Both solve 𝑦′ =𝑦: the first by the chain rule, and the second because 𝐸(𝑠) is a constant. Both equal 𝐸(𝑠) at 𝑡 =0. So they are the same function:

𝐸(𝑠+𝑡)=𝐸(𝑠)𝐸(𝑡).

Now write 𝑒 =𝐸(1). For a positive integer 𝑛, repeated addition gives

𝐸(𝑛)=𝐸(1+⋯+1⏟𝑛 times)=𝐸(1)𝑛=𝑒𝑛.

What about negative exponents? Since

𝐸(−𝑥)𝐸(𝑥)=𝐸(0)=1,

we have 𝐸( −𝑥) =1/𝐸(𝑥): negating the input takes the reciprocal.

And what is 𝐸(1/2)? Adding two halves gives

𝑒=𝐸(1)=𝐸(12+12)=𝐸(12)2.

So 𝐸(1/2) is a square root of 𝑒. It is the positive root: the series for 𝐸(𝑡) is positive when 𝑡 ≥0, and the reciprocal identity makes it positive when 𝑡 <0. Thus 𝐸(1/2) =√𝑒.

Similarly, for an integer 𝑝 and a positive integer 𝑞, adding 𝑞 copies of 𝑝/𝑞 gives

𝐸(𝑝/𝑞)𝑞=𝐸(𝑝)=𝑒𝑝.

So 𝐸(𝑝/𝑞) is the positive 𝑞th root of 𝑒𝑝. So at every rational number, the series gives exactly what multiplying and taking roots gave. At every other number, it gives an answer those operations couldn't: 𝐸(√2) is 𝑒√2.

For any constant 𝑎, the chain rule gives 𝑑𝑑𝑡𝐸(𝑎𝑡) =𝑎𝐸(𝑎𝑡), and 𝐸(𝑎 ⋅0) =1. So the solution of 𝑦′ =𝑎𝑦, 𝑦(0) =1 is

𝑦(𝑡)=𝐸(𝑎𝑡)=𝑒𝑎𝑡=∞∑𝑘=0𝑎𝑘𝑡𝑘𝑘!.

The differential equation defined the function, and the power series gave us a way to compute it. We can now try the same construction for matrices.

7.2The Matrix Exponential

Now repeat the calculation from section 7.1 for the matrix problem

Φ′=𝐴Φ,Φ(0)=𝐼.

Try a power series whose coefficients are constant matrices:

Φ=𝐶0+𝐶1𝑡+𝐶2𝑡2+𝐶3𝑡3+⋯.

Differentiating term by term, and multiplying on the left by 𝐴,

Φ′=𝐶1+2𝐶2𝑡+3𝐶3𝑡2+⋯,𝐴Φ=𝐴𝐶0+𝐴𝐶1𝑡+𝐴𝐶2𝑡2+⋯.

Matching the coefficient of 𝑡𝑘 gives (𝑘 +1)𝐶𝑘+1 =𝐴𝐶𝑘, exactly as before. The initial condition gives 𝐶0 =𝐼, and then

𝐶1=𝐴,𝐶2=𝐴22,𝐶3=𝐴33⋅2,…,𝐶𝑘=𝐴𝑘𝑘!.

Almost nothing changed. The recursion is the same, and so are the coefficients, with powers of the matrix 𝐴 in place of powers of the number 𝑎. Only two things are new. The number 1 has become the identity matrix 𝐼. And because the order of matrix multiplication matters, we kept 𝐴 on the left, where the equation puts it. In the end the order makes no difference here, because every coefficient is a power of 𝐴.

Definition 7.2 (Matrix exponential). For a square matrix 𝐴, the matrix exponential is the matrix-valued function

𝑒𝑡𝐴=∞∑𝑘=0𝑡𝑘𝐴𝑘𝑘!=𝐼+𝑡𝐴+𝑡2𝐴22!+𝑡3𝐴33!+⋯.

The notation does not mean exponentiating each entry of 𝐴. It names the matrix built from the powers 𝐼,𝐴,𝐴2,… by this series. So far, as in section 7.1, it is the result of a formal calculation.

Before checking that the series converges, let's see whether it gives the right answer for the one flow we know, the spring. Its matrix 𝐽 has 𝐽2 = −𝐼, so the powers repeat in a cycle of four:

𝐽2=−𝐼,𝐽3=−𝐽,𝐽4=𝐼,𝐽5=𝐽,….

Every term of the series is therefore a multiple of 𝐼 or of 𝐽. Collect them:

𝑒𝑡𝐽=𝐼+𝑡𝐽−𝑡22!𝐼−𝑡33!𝐽+𝑡44!𝐼+𝑡55!𝐽−⋯=(1−𝑡22!+𝑡44!−⋯)𝐼+(𝑡−𝑡33!+𝑡55!−⋯)𝐽.

Look at the two series in parentheses. They are familiar from calculus:

cos⁡𝑡=1−𝑡22!+𝑡44!−⋯,sin⁡𝑡=𝑡−𝑡33!+𝑡55!−⋯.

The exponential series has split into the power series for cosine and sine! So

𝑒𝑡𝐽=𝐼cos⁡𝑡+𝐽sin⁡𝑡=(cos⁡𝑡sin⁡𝑡−sin⁡𝑡cos⁡𝑡).

This is the rotation matrix from the opening. Starting from 𝐽 alone, the series has produced the spring's flow.

Figure 7.1 In the live figure, choose how many terms of the exponential series to keep. The resulting polynomial path follows the true circle for longer as more terms are added, then eventually peels away. Drag the gold state around the circle to compare its true position with the position predicted by the chosen truncation. In the limit, the series produces the flow 𝑒𝑡𝐽.

Remark 7.3 (The matrix as a velocity rule). The equation 𝑋′ =𝐴𝑋 assigns a velocity 𝐴𝑋 to every state 𝑋, and 𝑒𝑡𝐴 is the motion this rule produces. For the spring, the rule explains why the motion is a rotation. At the state 𝑋 =(𝑥,𝑣), the velocity is 𝐽𝑋 =(𝑣, −𝑥). It is perpendicular to 𝑋, since 𝑥 ⋅𝑣 +𝑣 ⋅( −𝑥) =0, and it has the same length, since 𝑣2 +𝑥2 =𝑥2 +𝑣2. A velocity perpendicular to the position never moves the state toward or away from the origin; it only turns it. So every state travels clockwise around a circle, at unit angular speed.

Figure 7.2 For 𝑋′ =𝐽𝑋, every velocity arrow is perpendicular to the position vector and grows in proportion to its length. The state therefore travels around a circle, while the grid rotates rigidly with it. In the live figure, drag the gold state and watch the entries of 𝑒𝑡𝐽 change as the motion continues.

The series for 𝑒𝑡𝐽 converged because it split into the sine and cosine series. A general matrix has no such pattern, so we need another way to see that the series converges. What exactly needs to converge? Take, for example,

𝐴=⎛⎜ ⎜ ⎜⎝110011101⎞⎟ ⎟ ⎟⎠,𝐴2=⎛⎜ ⎜ ⎜⎝121112211⎞⎟ ⎟ ⎟⎠.

Writing out the entries of 𝐼 +𝑡𝐴 +𝑡22!𝐴2 +⋯ gives

⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝1+𝑡+𝑡22!+⋯𝑡+2𝑡22!+⋯𝑡22!+⋯𝑡22!+⋯1+𝑡+𝑡22!+⋯𝑡+2𝑡22!+⋯𝑡+2𝑡22!+⋯𝑡22!+⋯1+𝑡+𝑡22!+⋯⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠.

There are nine ordinary infinite series here, one in each entry. For the matrix series to converge, all nine must converge. But we have only written their first few terms. The later coefficients come from more and more matrix multiplications, and we don't have a simple formula for them. How can we prove a series converges when we don't know its terms explicitly?

Calculus developed a collection of tools for exactly this kind of problem: convergence tests. A particularly useful one is comparison. If each term of a series is bounded in absolute value by the corresponding term of a convergent series of nonnegative numbers, then our series converges too. Why? The partial sums of the absolute values increase, but can never exceed the sum of the comparison series. So the series of absolute values converges, and therefore the original series converges as well.

We will do this for each of our nine series, or each of the 𝑛2 series for an 𝑛 ×𝑛 matrix. Happily, we won't need a different comparison for every entry: we can construct a single series that bounds them all! It will be an exponential series

𝑒𝑀|𝑡|=∞∑𝑘=0𝑀𝑘|𝑡|𝑘𝑘!,

where 𝑀 is a nonnegative number constructed from the entries of 𝐴. This series converges for every 𝑡, no matter how large 𝑀 is! So if we can find 𝑀 such that

|(𝐴𝑘)𝑖𝑗|≤𝑀𝑘

for every entry and every power, each term of each of our 𝑛2 series will be bounded in absolute value by the corresponding term of this one exponential series. The comparison test will prove that all of them converge. We win!

The remaining question is how to construct 𝑀. It must control not just the entries of 𝐴, but the entries of every power of 𝐴. The largest absolute row sum does the job, as the optional argument below shows.

7.2.1Optional Reading: The Convergence Argument

This argument is not required reading. You can continue at section 7.2.2, using the fact that every entry of the matrix exponential series converges for every 𝑡.

To measure the size of a matrix, take the absolute values of its entries, add them along each row, and keep the largest sum. For example, the rows of

𝐵=(1−231)

have absolute sums 1 +2 =3 and 3 +1 =4, so our measure of its size is 4. We give this measure a name and a notation.

Definition 7.4 (Row-sum norm). The row-sum norm of a matrix 𝐵 =(𝑏𝑖𝑗) is

‖𝐵‖=max𝑖∑𝑗|𝑏𝑖𝑗|,

the largest total of the absolute values in any one row.

Every entry is at most its own row's absolute sum, which is at most the largest row sum. Thus |𝑏𝑖𝑗| ≤‖𝐵‖. This controls the entries of 𝐵. To control powers, we need to know what happens when we multiply matrices.

Proposition 7.5 (Norm of a product). For square matrices 𝐵 and 𝐶 of the same size, ‖𝐵𝐶‖ ≤‖𝐵‖ ‖𝐶‖.

Proof. Fix a row 𝑖 of 𝐵𝐶. Its entry in column 𝑗 is ∑ℓ𝑏𝑖ℓ𝑐ℓ𝑗. The absolute value of a sum is at most the sum of the absolute values, so the absolute row sum is bounded by

∑𝑗|(𝐵𝐶)𝑖𝑗|≤∑𝑗∑ℓ|𝑏𝑖ℓ||𝑐ℓ𝑗|=∑ℓ|𝑏𝑖ℓ|∑𝑗|𝑐ℓ𝑗|.

The last step just changes the order of two finite sums. Now the inner sum is the absolute sum of row ℓ of 𝐶, so it is at most ‖𝐶‖. Therefore

∑𝑗|(𝐵𝐶)𝑖𝑗|≤∑ℓ|𝑏𝑖ℓ|‖𝐶‖=‖𝐶‖∑ℓ|𝑏𝑖ℓ|≤‖𝐶‖‖𝐵‖.

The remaining sum is the absolute sum of row 𝑖 of 𝐵, which explains the last inequality. The bound holds for every row of 𝐵𝐶, so it holds for the largest row sum too. ∎

Applying the product estimate repeatedly gives ‖𝐴2‖ ≤‖𝐴‖2, ‖𝐴3‖ ≤‖𝐴‖3, and in general ‖𝐴𝑘‖ ≤‖𝐴‖𝑘. Since every entry is bounded by the norm, we have found our number: 𝑀 =‖𝐴‖. For every entry of every term,

∣𝑡𝑘(𝐴𝑘)𝑖𝑗𝑘!∣≤|𝑡|𝑘‖𝐴‖𝑘𝑘!.

The numbers on the right are the terms of the series for 𝑒|𝑡|‖𝐴‖, which converges for every 𝑡. By the comparison test, all 𝑛2 entry series converge absolutely, for every 𝑡.

7.2.2Verifying the Flow

Each entry of 𝑒𝑡𝐴 is a power series in 𝑡 that converges everywhere. Calculus tells us that we can differentiate such a series term by term, just as in section 7.1. Differentiating the matrix series gives

𝑑𝑑𝑡𝑒𝑡𝐴=∞∑𝑘=1𝑘𝑡𝑘−1𝐴𝑘𝑘!=𝐴∞∑𝑘=1𝑡𝑘−1𝐴𝑘−1(𝑘−1)!=𝐴𝑒𝑡𝐴.

At 𝑡 =0, every term but the first vanishes, so 𝑒0𝐴 =𝐼. The series really does solve Φ′ =𝐴Φ, Φ(0) =𝐼, and by proposition 7.1, it is the flow matrix.

Theorem 7.6 (Flow of a constant linear system). For a constant matrix 𝐴, the solution of

𝑋′=𝐴𝑋,𝑋(𝑡0)=𝑋0

is 𝑋(𝑡) =𝑒(𝑡−𝑡0)𝐴𝑋0.

Proof. The derivative of 𝑒(𝑡−𝑡0)𝐴𝑋0 is 𝐴𝑒(𝑡−𝑡0)𝐴𝑋0, by the chain rule and the calculation above, so it solves the equation. At 𝑡 =𝑡0 it equals 𝑒0𝐴𝑋0 =𝑋0. By uniqueness, it is the solution. ∎

Multiplying by 𝑒𝑡𝐴 is the solution map, built from 𝐴 alone.

7.3Properties of the Matrix Exponential

For numbers, exponentiating a sum gives a product of exponentials. Does the same hold for matrices? It does when the matrices commute. To prove this, we'll compare two solutions of the same differential equation, just as we did for the scalar exponential. First we need a fact about commuting matrices.

Lemma 7.7 (Commuting with an exponential). If 𝐵𝐶 =𝐶𝐵, then 𝑒𝐵𝐶 =𝐶𝑒𝐵.

Proof. Since 𝐵 and 𝐶 commute, we can move 𝐶 past each factor of 𝐵: 𝐵𝑘𝐶 =𝐶𝐵𝑘. Moving it through the exponential series gives

𝑒𝐵𝐶=(∞∑𝑘=0𝐵𝑘𝑘!)𝐶=∞∑𝑘=0𝐵𝑘𝐶𝑘!=∞∑𝑘=0𝐶𝐵𝑘𝑘!=𝐶(∞∑𝑘=0𝐵𝑘𝑘!)=𝐶𝑒𝐵.

Multiplication by a fixed matrix uses only finite sums of entries, so it passes through the limits of these convergent series. ∎

Now we can prove the addition law. Remember that a matrix-valued differential equation is an ordinary system for the entries of the matrix. If 𝑍 is 𝑛 ×𝑛, then 𝑍′ =𝑀𝑍 is a homogeneous linear system with 𝑛2 unknown functions. It can be a pretty complicated system, but our existence and uniqueness theorem applies just the same!

Theorem 7.8 (Exponential of a sum). If 𝐵𝐶 =𝐶𝐵, then 𝑒𝐵+𝐶 =𝑒𝐵𝑒𝐶.

Proof. Define two matrix-valued functions

𝐹(𝑡)=𝑒𝑡(𝐵+𝐶),𝐺(𝑡)=𝑒𝑡𝐵𝑒𝑡𝐶.

We will show that both solve the initial value problem

𝑍′=(𝐵+𝐶)𝑍,𝑍(0)=𝐼.

For 𝐹, the derivative formula for the matrix exponential gives

𝐹′=(𝐵+𝐶)𝑒𝑡(𝐵+𝐶)=(𝐵+𝐶)𝐹,𝐹(0)=𝐼.

For 𝐺, use the product rule. Since 𝑡𝐵 commutes with 𝐶, lemma 7.7 gives 𝑒𝑡𝐵𝐶 =𝐶𝑒𝑡𝐵. Thus

𝐺′=𝐵𝑒𝑡𝐵𝑒𝑡𝐶+𝑒𝑡𝐵𝐶𝑒𝑡𝐶=𝐵𝑒𝑡𝐵𝑒𝑡𝐶+𝐶𝑒𝑡𝐵𝑒𝑡𝐶=(𝐵+𝐶)𝐺.

Also 𝐺(0) =𝐼𝐼 =𝐼. Both functions solve the same linear system with the same initial condition, so uniqueness gives 𝐹(𝑡) =𝐺(𝑡) for every 𝑡. Taking 𝑡 =1 gives 𝑒𝐵+𝐶 =𝑒𝐵𝑒𝐶. ∎

The commuting hypothesis let us move 𝐶 past 𝑒𝑡𝐵 in the second line of the calculation. Without it, the law can fail; an exercise works out an example.

The familiar flow laws now follow from this one result. First, any two multiples of the same matrix commute: (𝑠𝐴)(𝑡𝐴) =𝑠𝑡𝐴2 =(𝑡𝐴)(𝑠𝐴).

Corollary 7.9 (Adding elapsed times). For any square matrix 𝐴 and real numbers 𝑠,𝑡,

𝑒(𝑠+𝑡)𝐴=𝑒𝑠𝐴𝑒𝑡𝐴.

Apply theorem 7.8 to 𝑠𝐴 and 𝑡𝐴. In words, evolving for time 𝑡 and then for time 𝑠 has the same effect as evolving for time 𝑠 +𝑡.

Taking 𝑠 = −𝑡 gives 𝑒−𝑡𝐴𝑒𝑡𝐴 =𝑒0𝐴 =𝐼. Reversing the order gives the identity too, so we also have:

Corollary 7.10 (Reversing the flow). For any square matrix 𝐴 and real number 𝑡,

(𝑒𝑡𝐴)−1=𝑒−𝑡𝐴.

Chapter 6 showed that the flow matrix is always invertible. Now we have a formula for its inverse: undoing the flow for time 𝑡 is the same as running it for time −𝑡.

7.4Matrix Exponentials We Can Compute

The series defines 𝑒𝑡𝐴 for every square matrix, but an infinite series is not always the most revealing form. When the powers of 𝐴 follow a pattern, as the powers of 𝐽 did, the series collapses into a formula we can read.

The simplest pattern belongs to a multiple of the identity, 𝐴 =𝜆𝐼. Every power is (𝜆𝐼)𝑘 =𝜆𝑘𝐼, so

𝑒𝑡𝐴=(∞∑𝑘=0(𝜆𝑡)𝑘𝑘!)𝐼=𝑒𝜆𝑡𝐼.

The velocity rule 𝐴𝑋 =𝜆𝑋 points along the line through 𝑋: outward when 𝜆 >0, and inward when 𝜆 <0. So every state slides along its own ray, and the flow scales the whole plane by 𝑒𝜆𝑡.

This calculation gives a third useful consequence of the addition law. A multiple of the identity commutes with every matrix, so we can separate uniform scaling from the rest of a flow.

Corollary 7.11 (Separating uniform scaling). For any square matrix 𝐵 and real numbers 𝑎,𝑡,

𝑒𝑡(𝑎𝐼+𝐵)=𝑒𝑎𝑡𝑒𝑡𝐵.

Indeed, theorem 7.8 and the calculation above give

𝑒𝑡(𝑎𝐼+𝐵)=𝑒𝑡𝑎𝐼𝑒𝑡𝐵=(𝑒𝑎𝑡𝐼)𝑒𝑡𝐵=𝑒𝑎𝑡𝑒𝑡𝐵.

Adding 𝑎𝐼 to the velocity rule multiplies the resulting motion by 𝑒𝑎𝑡. It expands or contracts the whole flow uniformly.

A diagonal matrix gives each coordinate its own rate. If

𝐷=(𝜆100𝜆2),then𝐷𝑘=(𝜆𝑘100𝜆𝑘2),𝑒𝑡𝐷=(𝑒𝜆1𝑡00𝑒𝜆2𝑡).

We exponentiate the diagonal entries. The system is just the two separate equations 𝑥′ =𝜆1𝑥 and 𝑦′ =𝜆2𝑦. For example, 𝐷 =diag⁡(1, −1) stretches the 𝑥-direction while squeezing the 𝑦-direction.

Some matrices simplify for a different reason: their powers eventually vanish. Take

𝑁=(0100),

acting on states (𝑥,𝑣). Here 𝑁2 =0, so the series stops after two terms:

𝑒𝑡𝑁=𝐼+𝑡𝑁=(1𝑡01),𝑒𝑡𝑁(𝑥0𝑣0)=(𝑥0+𝑣0𝑡𝑣0).

The velocity rule 𝑁𝑋 =(𝑣,0) is horizontal, and it is larger farther from the 𝑥-axis. So horizontal layers slide past one another, and points on the 𝑥-axis stay put: the flow is a shear. We have met this matrix before. It is the system 𝑥′ =𝑣, 𝑣′ =0, which is 𝑥″ =0: a particle with no force acting on it. The flow says what we expect. The particle keeps its starting velocity, and its position changes at that constant rate.

Figure 7.3 These four matrices have simple patterns in their powers but produce quite different motion. Uniform scaling pushes every ray outward, diagonal scaling gives the two axes different rates, 𝐽 rotates the plane, and the shear slides horizontal layers past one another while leaving the 𝑥-axis fixed. The gold states move at the speeds prescribed by their respective velocity rules.

These examples worked because the powers of each matrix followed a pattern we could recognize. Now return to the damped spring from section 4.4. With mass 1, spring constant 𝑘, and drag coefficient 𝑏, its equation is 𝑥″ = −𝑘𝑥 −𝑏𝑥′, and with state (𝑥,𝑣) it becomes

𝑋′=𝐴𝑋,𝐴=(01−𝑘−𝑏).

In section 6.4 we solved it by guessing, with a different guess for each kind of motion. Now theorem 7.6 gives the solution as a single formula, 𝑋(𝑡) =𝑒𝑡𝐴𝑋0, for every 𝑏 and 𝑘. But the powers of 𝐴 don't settle into a pattern:

𝐴2=(−𝑘−𝑏𝑏𝑘𝑏2−𝑘),𝐴3=(𝑏𝑘𝑏2−𝑘𝑘2−𝑏2𝑘2𝑏𝑘−𝑏3).

Each new power mixes the entries further, and we can't sum the series by recognizing it.

The series still works, though. A computer can add up as many terms as we like. Take the motion from Part I, with 𝑘 =1, 𝑏 =0.4, 𝑥(0) =1, and 𝑥′(0) = −0.2, and keep more and more terms.

Figure 7.4 The damped spring with 𝑘 =1 and 𝑏 =0.4, starting from 𝑥(0) =1 and 𝑥′(0) = −0.2, as in section 4.4. The blue curve is the true motion. The gold curve keeps the first 𝑛 terms of the series for 𝑒𝑡𝐴𝑋0. Increase 𝑛: the gold curve follows the true motion for longer before flying off, and with enough terms it matches across the whole window.

With a dozen terms, the partial sum follows the true motion only until about 𝑡 =4. With sixty, it matches across the whole window, decaying oscillation and all. So the matrix exponential computes the damped spring's motion. What it doesn't do is explain it. Nothing in the powers of 𝐴 shows why the motion oscillates, why it decays, or how fast.

We understand diagonal motion, rotation, and shear because their matrices have simple powers. Could a change of coordinates make a matrix like the damped spring's just as simple? Perhaps the motion is one of these familiar flows, combined with uniform scaling, seen in different coordinates. That is the question of chapter 8.

7.5Appendix: Euler's Method and the Exponential

This appendix is optional. It connects the matrix exponential to Euler's method from chapter 5.

For a first-order system 𝑋′ =𝐹(𝑡,𝑋), Euler's method takes the steps 𝑋𝑘+1 =𝑋𝑘 +ℎ𝐹(𝑡𝑘,𝑋𝑘). For 𝑋′ =𝐴𝑋 with a constant matrix, 𝐹(𝑡,𝑋) =𝐴𝑋, so

𝑋𝑘+1=𝑋𝑘+ℎ𝐴𝑋𝑘=(𝐼+ℎ𝐴)𝑋𝑘.

Every step multiplies the state by the same matrix 𝐼 +ℎ𝐴. To go from time 0 to time 𝑡 in 𝑛 equal steps, take ℎ =𝑡/𝑛. After 𝑛 steps,

𝑋𝑛=(𝐼+𝑡𝑛𝐴)𝑛𝑋0.

Euler's method takes the equation literally: over each short step, the state changes by 𝐴 times its current value, times the length of the step.

For the spring, we can watch this happen. Take 𝐴 =𝐽. Each Euler step moves the state from 𝑋 to 𝑋 +ℎ𝐽𝑋. By remark 7.3, the step ℎ𝐽𝑋 is perpendicular to 𝑋, and its length is ℎ‖𝑋‖. So the old state, the step, and the new state form a right triangle, and by Pythagoras,

‖𝑋+ℎ𝐽𝑋‖2=‖𝑋‖2+ℎ2‖𝑋‖2=(1+ℎ2)‖𝑋‖2.

Every step stretches the state by √1+ℎ2. It also turns the state clockwise through the angle arctan⁡ℎ, since the legs of the triangle have ratio ℎ. In matrix form, one step is a stretch times a rotation:

𝐼+ℎ𝐽=√1+ℎ2𝑒(arctan⁡ℎ)𝐽.

Take 𝑛 steps with ℎ =𝑡/𝑛. The stretches multiply and, by the law of exponents, the angles add:

(𝐼+𝑡𝑛𝐽)𝑛=(1+𝑡2𝑛2)𝑛/2𝑒(𝑛arctan⁡𝑡𝑛)𝐽.

So the Euler polygon spirals outward, and after 𝑛 steps it has not quite turned through the angle 𝑡. But both errors disappear as 𝑛 grows. For the stretch, log⁡(1 +𝑥) ≤𝑥 gives

0≤𝑛2log⁡(1+𝑡2𝑛2)≤𝑡22𝑛⟶0,

so the stretch factor tends to 1. For the angle, when 𝑡 ≠0,

𝑛arctan⁡𝑡𝑛=𝑡⋅arctan⁡(𝑡/𝑛)𝑡/𝑛⟶𝑡,

since arctan⁡𝑢/𝑢 →1 as 𝑢 →0. So (𝐼 +𝑡𝑛𝐽)𝑛 →𝑒𝑡𝐽: the Euler steps converge to the exact rotation.

Figure 7.5 Each Euler step multiplies the state by 𝐼 +ℎ𝐽. This turns the state through arctan⁡ℎ, but it also stretches lengths by √1+ℎ2, so the polygon spirals outside the true circle. In the live figure, increase the number of steps per loop: the accumulated stretch shrinks, and (𝐼 +𝑡𝑛𝐽)𝑛 approaches the rotation 𝑒𝑡𝐽.

For numbers, calculus gives the limit (1 +𝑎𝑡𝑛)𝑛 →𝑒𝑎𝑡. The same holds for matrices.

Theorem 7.12 (Euler's method converges to the exponential). For every square matrix 𝐴 and every 𝑡,

(𝐼+𝑡𝑛𝐴)𝑛⟶𝑒𝑡𝐴as 𝑛→∞.

Proof. Since 𝐼 and 𝐴 commute, the binomial theorem expands the power:

(𝐼+𝑡𝑛𝐴)𝑛=𝑛∑𝑘=0(𝑛𝑘)𝑡𝑘𝐴𝑘𝑛𝑘=𝑛∑𝑘=0𝛽𝑛,𝑘𝑡𝑘𝐴𝑘𝑘!,𝛽𝑛,𝑘=𝑛(𝑛−1)⋯(𝑛−𝑘+1)𝑛𝑘.

Each 𝛽𝑛,𝑘 lies between 0 and 1, and for each fixed 𝑘 it tends to 1 as 𝑛 →∞. Set 𝛽𝑛,𝑘 =0 for 𝑘 >𝑛. Then

𝑒𝑡𝐴−(𝐼+𝑡𝑛𝐴)𝑛=∞∑𝑘=0(1−𝛽𝑛,𝑘)𝑡𝑘𝐴𝑘𝑘!.

The norm of a sum is at most the sum of the norms, and ‖𝐴𝑘‖ ≤‖𝐴‖𝑘. So, writing 𝑠 =|𝑡| ‖𝐴‖,

∥𝑒𝑡𝐴−(𝐼+𝑡𝑛𝐴)𝑛∥≤∞∑𝑘=0(1−𝛽𝑛,𝑘)𝑠𝑘𝑘!=𝑒𝑠−(1+𝑠𝑛)𝑛.

The last equality is the same binomial expansion, for the number 𝑠. The right side tends to 0 by the scalar limit, so the left side does too. ∎

Problems

1The Hyperbolic Functions from a Matrix Exponential

We return to the equation

𝑦″=𝑦,

this time solving it with the matrix exponential. Along the way, we will find the power series of the hyperbolic functions.

(a)

Set 𝑣 =𝑦′ and write the equation as a first-order system 𝑋′ =𝐴𝑋 with state 𝑋 =(𝑦,𝑣). Compute 𝐴2, 𝐴3, and 𝐴4. What pattern do the powers of 𝐴 follow?

(b)

Substitute your pattern into the series

𝑒𝑡𝐴=∞∑𝑘=0𝑡𝑘𝐴𝑘𝑘!.

Collect the even and odd powers to write

𝑒𝑡𝐴=𝑐(𝑡)𝐼+𝑠(𝑡)𝐴,

where 𝑐(𝑡) and 𝑠(𝑡) are scalar power series. Find these two series, and write 𝑒𝑡𝐴 as a 2 ×2 matrix with entries 𝑐(𝑡) and 𝑠(𝑡).

(c)

The matrix 𝑒𝑡𝐴 is the flow matrix of the system, so its first column is the state (𝑦(𝑡),𝑣(𝑡)) of the solution starting from (1,0). Use your work in the Chapter 6 exercise Discovering the hyperbolic functions to identify that solution and its state. What functions are 𝑐(𝑡) and 𝑠(𝑡)? What power series have you found?

2The Exponential of a Sum

For numbers, 𝑒𝑎+𝑏 =𝑒𝑎𝑒𝑏. In this problem we test whether the same law holds for matrices, using

𝐵=(0100),𝐶=(0010).
(a)

Compute 𝐵2 and 𝐶2. Use your answers to find 𝑒𝑡𝐵 and 𝑒𝑡𝐶 from the exponential series.

(b)

Compute 𝐵 +𝐶. Where have you seen this matrix before, and what is 𝑒𝑡(𝐵+𝐶)? Now compute the products 𝑒𝑡𝐵𝑒𝑡𝐶 and 𝑒𝑡𝐶𝑒𝑡𝐵. Does 𝑒𝑡(𝐵+𝐶) =𝑒𝑡𝐵𝑒𝑡𝐶? Does the order of the product matter?

(c)

Why does the law fail? Let 𝐵 and 𝐶 now be any two square matrices of the same size. Write out the terms of 𝑒𝑡(𝐵+𝐶) and of 𝑒𝑡𝐵𝑒𝑡𝐶 up through 𝑡2. Show that the 𝑡2 terms agree exactly when 𝐵𝐶 =𝐶𝐵. Compute 𝐵𝐶 and 𝐶𝐵 for our matrices. Why does this issue never arise for numbers?

3A Series That Stops

Consider the third-order equation

𝑥‴=0.
(a)

Solve this equation with initial data 𝑥(0) =𝑥0, 𝑥′(0) =𝑣0, 𝑥″(0) =𝑤0 by integrating three times.

(b)

Set 𝑣 =𝑥′ and 𝑤 =𝑥″, and write the equation as a first-order system 𝑋′ =𝑁𝑋 with state 𝑋 =(𝑥,𝑣,𝑤). Compute 𝑁2 and 𝑁3.

(c)

Compute 𝑒𝑡𝑁 from the exponential series. Use it to solve the system from the initial state 𝑋0 =(𝑥0,𝑣0,𝑤0), and check that you recover your answer from the first part.

4The flow of 𝑥′ =2𝑦, 𝑦′ = −3𝑥

In example 6.20 we found the flow matrix of the system 𝑥′ =2𝑦, 𝑦′ = −3𝑥 by checking a family of solutions. In this problem, find it from the coefficient matrix alone.

(a)

Write the system as 𝑋′ =𝐴𝑋. Compute 𝐴2 and 𝐴3. What pattern do the powers of 𝐴 follow?

(b)

Substitute your pattern into the series

𝑒𝑡𝐴=∞∑𝑘=0𝑡𝑘𝐴𝑘𝑘!.

Collect the even and odd powers to write 𝑒𝑡𝐴 =𝑐(𝑡) 𝐼 +𝑠(𝑡) 𝐴, where 𝑐(𝑡) and 𝑠(𝑡) are scalar power series. Compare them with the series for cosine and sine, evaluated at √6 𝑡, and identify 𝑐 and 𝑠.

(c)

Write 𝑒𝑡𝐴 as a 2 ×2 matrix, and check that it is the flow matrix from example 6.20.