Chapter 6 left us with a matrix initial value problem. For the system
𝑋′=𝐴(𝑡)𝑋, the flow matrix beginning at time 𝑡0 satisfies
Φ′𝑡0(𝑡)=𝐴(𝑡)Φ𝑡0(𝑡),Φ𝑡0(𝑡0)=𝐼.
We can construct this matrix by solving once from each standard initial
state, either exactly or numerically. That procedure computes the columns of
Φ𝑡0 separately. It does not yet tell us how to construct the whole
matrix directly from the coefficient matrix 𝐴(𝑡).
We now restrict attention to a constant system 𝑋′=𝐴𝑋. In this case the
starting time can be removed by resetting the clock. If 𝑋(𝑡0)=𝑋0, define
𝑌(𝜏)=𝑋(𝑡0+𝜏). The chain rule gives
𝑌′(𝜏)=𝑋′(𝑡0+𝜏)=𝐴𝑋(𝑡0+𝜏)=𝐴𝑌(𝜏),𝑌(0)=𝑋0.
Thus the evolution depends only on the elapsed time 𝜏=𝑡−𝑡0:
Φ𝑡0(𝑡)=Φ0(𝑡−𝑡0). We may set the initial time equal to zero,
construct Φ0(𝑡), and replace 𝑡 by 𝑡−𝑡0 when the original clock is
needed.
The problem is therefore precise: given a constant 𝑁×𝑁 matrix 𝐴,
construct the matrix-valued solution of
Φ′(𝑡)=𝐴Φ(𝑡),Φ(0)=𝐼
directly from 𝐴.
7.1The Generator of Rotation
The solutions of the normalized spring 𝑥″=−𝑥 are already known. With state
𝑋=(𝑥,𝑣), where 𝑣=𝑥′, its equation becomes the constant system
𝑋′=(𝑥′𝑣′)=(𝑣−𝑥)=(01−10)(𝑥𝑣)=𝐽𝑋,𝐽=(01−10).
At a state 𝑋=(𝑥,𝑣), the matrix 𝐽 assigns the velocity
𝐽𝑋=(𝑣,−𝑥). This vector is perpendicular to 𝑋, since
(𝑥,𝑣)⋅(𝑣,−𝑥)=𝑥𝑣−𝑣𝑥=0, and it has the same length because
‖𝐽𝑋‖2=𝑣2+𝑥2=‖𝑋‖2. Thus the differential
equation prescribes clockwise motion tangent to the circle through the current
state, with unit angular speed.
We also know the solution from an arbitrary initial state
𝑋0=(𝑥0,𝑣0). The position and velocity are
𝑥(𝑡)=𝑥0cos𝑡+𝑣0sin𝑡,𝑣(𝑡)=−𝑥0sin𝑡+𝑣0cos𝑡.
Putting these two equations into a single matrix equation gives
𝑋(𝑡)=𝑅(𝑡)𝑋0,𝑅(𝑡)=(cos𝑡sin𝑡−sin𝑡cos𝑡).
The matrix 𝐽 describes the velocity at one instant, while 𝑅(𝑡) advances
every initial state through a time interval of length 𝑡. These matrices are
connected by the same equation that governed the flow matrix in Chapter 6.
Indeed,
Therefore 𝑅′(𝑡)=𝐽𝑅(𝑡), and direct evaluation gives 𝑅(0)=𝐼. In particular,
𝑅′(0)=𝐽𝑅(0)=𝐽.
Here we obtained 𝑅(𝑡) from a solution formula that was already available.
If the sine and cosine formulas were not known, how could we recover 𝑅(𝑡)
from the matrix 𝐽 alone?
Figure 7.1 For 𝑋′=𝐽𝑋, every velocity arrow is perpendicular to the position vector
and grows in proportion to its length. The state therefore travels around
a circle while the grid rotates rigidly with it. In the live figure, drag
the gold state and watch the entries of the rotation matrix 𝑅(𝑡) change as
the motion continues.
7.2Repeated Small Steps
Before trying to recover the rotation matrix from 𝐽, consider the scalar
equation with the same constant-coefficient form:
𝑦′=𝑎𝑦,𝑦(0)=𝑦0.
Euler's method with step size ℎ replaces the continuous evolution by the
repeated update
𝑦𝑘+1=𝑦𝑘+ℎ𝑎𝑦𝑘=(1+ℎ𝑎)𝑦𝑘.
To travel from time 0 to time 𝑡 in 𝑛 equal steps, take ℎ=𝑡/𝑛.
Repeating the update 𝑛 times gives
𝑦𝑛=(1+𝑎𝑡𝑛)𝑛𝑦0.
Calculus identifies the limit of these growth factors:
𝑒𝑎𝑡=lim𝑛→∞(1+𝑎𝑡𝑛)𝑛.
Now apply exactly the same numerical idea to 𝑋′=𝐴𝑋. One Euler step sends
the current state 𝑋𝑘 to
𝑋𝑘+1=𝑋𝑘+ℎ𝐴𝑋𝑘=(𝐼+ℎ𝐴)𝑋𝑘.
Because 𝐴 is constant, every step uses the same matrix. After 𝑛 steps of
size 𝑡/𝑛, the computed state is therefore
𝑋𝑛=(𝐼+𝑡𝑛𝐴)𝑛𝑋0.
For the spring generator 𝐽, we can see precisely what these repeated
updates do. The preceding section showed that 𝐽𝑋 is perpendicular to 𝑋
and has the same length. Thus the new radius 𝑋+ℎ𝐽𝑋 is the hypotenuse of a
right triangle whose perpendicular legs are 𝑋 and ℎ𝐽𝑋. The Pythagorean
theorem gives
‖𝑋+ℎ𝐽𝑋‖2=‖𝑋‖2+ℎ2‖𝐽𝑋‖2=(1+ℎ2)‖𝑋‖2.
One step therefore multiplies every length by √1+ℎ2. Its signed
clockwise turning angle is arctanℎ: the ratio of the perpendicular leg
to the original radius is ℎ. In terms of the rotation matrix from the
previous section, the entire Euler update is
𝐼+ℎ𝐽=√1+ℎ2𝑅(arctanℎ).
Repeating this same scaling and rotation 𝑛 times, with ℎ=𝑡/𝑛, yields
(𝐼+𝑡𝑛𝐽)𝑛=(1+𝑡2𝑛2)𝑛/2𝑅(𝑛arctan𝑡𝑛).
Both finite-step errors are visible here. The first factor makes the state
spiral outward, while the angle after 𝑛 steps is not quite 𝑡. But both
errors disappear as the step size shrinks. For the radial factor,
0≤𝑛2log(1+𝑡2𝑛2)≤𝑡22𝑛⟶0,
so (1+𝑡2/𝑛2)𝑛/2→1. When 𝑡≠0, the accumulated
angle satisfies
𝑛arctan𝑡𝑛=𝑡arctan(𝑡/𝑛)𝑡/𝑛⟶𝑡,
and the case 𝑡=0 is immediate. Consequently,
(𝐼+𝑡𝑛𝐽)𝑛⟶𝑅(𝑡).
Starting only from the local rule 𝐽, Euler's small linear turns recover the
exact rotation in the limit.
Figure 7.2 Each Euler step multiplies the state by 𝐼+ℎ𝐽. This turns the state through
arctanℎ, but it also stretches lengths by √1+ℎ2, so the polygon
spirals outside the true circle. In the live figure, increase the number of
steps per loop: the accumulated stretch shrinks and
(𝐼+𝑡𝑛𝐽)𝑛 approaches the rotation 𝑅(𝑡).
The scalar and matrix calculations now have the same shape:
𝑒𝑎𝑡=lim𝑛→∞(1+𝑎𝑡𝑛)𝑛,𝑅(𝑡)=lim𝑛→∞(𝐼+𝑡𝑛𝐽)𝑛.
For a general constant matrix 𝐴, this resemblance suggests both a name and
a possible limiting formula for the exact flow:
𝑒𝑡𝐴?=lim𝑛→∞(𝐼+𝑡𝑛𝐴)𝑛.
The question mark matters. We have proved this matrix limit only for the
special generator 𝐽, and an Euler approximation does not by itself define
the exact solution. The exact equation Φ′=𝐴Φ, Φ(0)=𝐼 must tell us
what the matrix 𝑒𝑡𝐴 is.
7.3The Matrix Exponential
The Euler product suggested what to call the flow, but the exact equation
must construct it. We return to
Φ′=𝐴Φ,Φ(0)=𝐼.
For the scalar equation 𝑦′=𝑎𝑦, the initial condition 𝑦(0)=1
characterizes 𝑒𝑎𝑡. The matrix equation has precisely the same form:
the scalar 1 has become the identity matrix 𝐼, and multiplication by the
number 𝑎 has become left multiplication by the matrix 𝐴. If a
matrix-valued function deserves to be called 𝑒𝑡𝐴, it is the solution of
this initial value problem.
There is nothing mysterious about differentiating such a function. The set
𝑀𝑁(ℝ) of real 𝑁×𝑁 matrices is a vector space with 𝑁2
coordinates, one for each entry. A matrix-valued function is a curve in this
space, and its derivative is computed entry by entry. Moreover, the map
𝐵↦𝐴𝐵 is linear on 𝑀𝑁(ℝ). Thus the equation above is an
ordinary linear system in 𝑁2 scalar coordinates. Viewed column by column,
it is also exactly the collection of 𝑁 standard-history problems from
Chapter 6.
How might we construct its solution? In Calculus II, power series gave a way
to solve differential equations when no formula was yet available. Try the
same idea here by seeking a candidate of the form
Φ(𝑡)=𝐶0+𝐶1𝑡+𝐶2𝑡2+𝐶3𝑡3+⋯,
where each coefficient 𝐶𝑘 is a constant matrix. We are not assuming that
an unknown solution must be analytic; we are asking whether a power series
can produce one. Formally differentiating the candidate and multiplying it
by 𝐴 give
Φ′(𝑡)=𝐶1+2𝐶2𝑡+3𝐶3𝑡2+⋯,𝐴Φ(𝑡)=𝐴𝐶0+𝐴𝐶1𝑡+𝐴𝐶2𝑡2+⋯.
The initial condition requires 𝐶0=𝐼. Matching equal powers of 𝑡 in the
differential equation then requires
𝐶1=𝐴𝐶0=𝐴,2𝐶2=𝐴𝐶1=𝐴2,3𝐶3=𝐴𝐶2=𝐴32!.
In general, (𝑘+1)𝐶𝑘+1=𝐴𝐶𝑘, so the pattern continues by induction:
𝐶𝑘=𝐴𝑘𝑘!.
We have therefore arrived at one possible solution,
𝐼+𝑡𝐴+𝑡2𝐴22!+𝑡3𝐴33!+⋯.
So far this is only a formal candidate. Before differentiating it as though
it were a function, we must know that its partial sums actually converge.
For a matrix 𝐵=(𝑏𝑖𝑗), define the row-sum norm
‖𝐵‖=max𝑖∑𝑗|𝑏𝑖𝑗|.
This is also commonly called the infinity norm, though we will use the more
descriptive name here.
This norm interacts especially well with matrix multiplication. If
𝐵=(𝑏𝑖𝑗) and 𝐶=(𝑐𝑖𝑗), then
Applying this inequality repeatedly gives
‖𝐴𝑘‖≤‖𝐴‖𝑘. Consequently,
∥𝑡𝑘𝐴𝑘𝑘!∥≤|𝑡|𝑘‖𝐴‖𝑘𝑘!.
The scalar series formed from the quantities on the right is
∞∑𝑘=0|𝑡|𝑘‖𝐴‖𝑘𝑘!=𝑒|𝑡|‖𝐴‖,
which converges for every real 𝑡. The comparison therefore proves that our
matrix series converges for every 𝑡 as well. Since 𝑀𝑁(ℝ) has
only finitely many coordinates, convergence in this norm is equivalent to
convergence of every matrix entry.
We may now make the promised definition.
Definition 7.1 (Matrix exponential). For a real square matrix 𝐴, its matrix exponential is the
matrix-valued function
𝑒𝑡𝐴:=∞∑𝑘=0𝑡𝑘𝐴𝑘𝑘!.
The notation does not mean that we exponentiate the entries of 𝐴
separately. It names the matrix built from the powers
𝐼,𝐴,𝐴2,𝐴3,… by this convergent series.
Each entry of the matrix exponential is now a scalar power series with
infinite radius of convergence. The power-series theorem from calculus
therefore permits term-by-term differentiation. Reindexing the resulting
series gives
𝑑𝑑𝑡𝑒𝑡𝐴=∞∑𝑘=1𝑘𝑡𝑘−1𝐴𝑘𝑘!=𝐴∞∑𝑘=1𝑡𝑘−1𝐴𝑘−1(𝑘−1)!=𝐴𝑒𝑡𝐴.
At 𝑡=0, every term except the first vanishes, so 𝑒0𝐴=𝐼. The series
really does solve the matrix initial value problem that produced it.
Uniqueness now identifies it with the flow matrix.
Theorem 7.2 (Flow of a constant linear system). For a constant real matrix 𝐴, the unique solution of
𝑋′=𝐴𝑋,𝑋(𝑡0)=𝑋0
is
𝑋(𝑡)=𝑒(𝑡−𝑡0)𝐴𝑋0.
Proof. Set 𝑋(𝑡)=𝑒(𝑡−𝑡0)𝐴𝑋0. The calculation above gives
𝑋′(𝑡)=𝐴𝑒(𝑡−𝑡0)𝐴𝑋0=𝐴𝑋(𝑡),
while 𝑋(𝑡0)=𝑒0𝐴𝑋0=𝑋0. Thus this function satisfies both the
differential equation and the initial condition. Uniqueness says it is the
solution. ∎
The spring shows what this construction has accomplished. Since
𝐽2=−𝐼, the powers repeat in groups of four:
𝐽3=−𝐽 and 𝐽4=𝐼. Because the exponential series converges absolutely,
we may collect its even and odd powers to obtain
We began with the instantaneous rule 𝐽 and, without assuming the
trigonometric solution, constructed the entire rotation flow from its powers.
Figure 7.3 In the live figure, choose how many terms of the exponential series to keep.
The resulting polynomial path follows the true circle for longer as more
terms are added, then eventually peels away. Drag the gold state around the
circle to compare its true position with the position predicted by the chosen
truncation. In the limit, the series produces the flow we denote by 𝑒𝑡𝐽.
The matrix exponential also obeys the familiar law for adding elapsed times.
Fix 𝑠 and consider, as functions of 𝑡,
𝐹(𝑡)=𝑒(𝑡+𝑠)𝐴,𝐺(𝑡)=𝑒𝑡𝐴𝑒𝑠𝐴.
Both satisfy 𝑍′=𝐴𝑍, and both equal 𝑒𝑠𝐴 when 𝑡=0. Uniqueness gives
𝑒(𝑡+𝑠)𝐴=𝑒𝑡𝐴𝑒𝑠𝐴.
Taking 𝑠=−𝑡 then gives
𝑒𝑡𝐴𝑒−𝑡𝐴=𝑒0𝐴=𝐼,(𝑒𝑡𝐴)−1=𝑒−𝑡𝐴.
Thus every constant linear system produces an invertible flow for every
elapsed time. The remaining difficulty is computational: when the powers of
𝐴 do not repeat as transparently as the powers of 𝐽, how can we recognize
the motion encoded by their series?
7.4Matrix Flows We Can Compute
The exponential series gives an exact flow for every constant matrix, but an
infinite series is not always the most revealing form of the answer. For some
matrices, the pattern in the powers turns the series into a familiar formula
and makes the resulting motion visible.
Begin with a scalar multiple of the identity, 𝐴=𝜆𝐼. Since
(𝜆𝐼)𝑘=𝜆𝑘𝐼, every term of the matrix series is a scalar
multiple of the identity:
𝑒𝑡𝐴=∞∑𝑘=0𝑡𝑘(𝜆𝐼)𝑘𝑘!=(∞∑𝑘=0(𝜆𝑡)𝑘𝑘!)𝐼=𝑒𝜆𝑡𝐼.
The local rule 𝐴𝑋=𝜆𝑋 points along the line through 𝑋, outward when
𝜆>0 and inward when 𝜆<0. Its flow multiplies every state by
the same factor 𝑒𝜆𝑡, so every state remains on its original ray.
The coordinates can also grow at different rates. If
𝐷=(𝜆100𝜆2),
then every power of 𝐷 is still diagonal:
𝐷𝑘=(𝜆𝑘100𝜆𝑘2).
Substitution into the series gives
𝑒𝑡𝐷=(𝑒𝜆1𝑡00𝑒𝜆2𝑡),
and therefore
𝑒𝑡𝐷(𝑥0𝑦0)=(𝑒𝜆1𝑡𝑥0𝑒𝜆2𝑡𝑦0).
This formula may look like entrywise exponentiation, but it works only because
the diagonal matrix does not mix the coordinates. The two scalar equations
𝑥′=𝜆1𝑥 and 𝑦′=𝜆2𝑦 evolve independently. In particular,
𝐷=diag(1,−1) expands the first coordinate while contracting
the second as time moves forward.
Some matrices simplify for a different reason: their higher powers vanish.
Consider
𝑁=(0100).
Here 𝑁2=0, so every term of the exponential series after the linear term
vanishes:
𝑒𝑡𝑁=𝐼+𝑡𝑁=(1𝑡01).
Thus the flow sends
(𝑥0,𝑦0)⟼(𝑥0+𝑡𝑦0,𝑦0).
The second coordinate remains fixed while the first changes at the constant
rate 𝑦0. Every point on the 𝑥-axis is fixed, and horizontal layers slide
past one another faster when they lie farther from that axis. This is a
shear.
Figure 7.4 These four matrices have simple patterns in their powers but produce quite
different motion. Uniform scaling pushes every ray outward, diagonal scaling
gives the two axes different rates, 𝐽 rotates the plane, and the nilpotent
matrix slides horizontal layers past one another while leaving the 𝑥-axis
fixed. The gold states move at the speeds prescribed by their respective
vector fields.
The first and third of these motions can occur simultaneously. Suppose
𝐴=𝑎𝐼+𝜔𝐽.
At each state 𝑋, the term 𝑎𝑋 points radially while 𝜔𝐽𝑋 points
tangentially. This suggests a flow that scales by 𝑒𝑎𝑡 while rotating
through the angle 𝜔𝑡. To check that the two motions really combine in
this way, set
The two expressions agree, and 𝐹(0)=𝐼. Uniqueness therefore proves
𝑒𝑡(𝑎𝐼+𝜔𝐽)=𝑒𝑎𝑡(𝐼cos(𝜔𝑡)+𝐽sin(𝜔𝑡)).
When 𝑎>0, states spiral outward; when 𝑎<0, they spiral inward; and when
𝑎=0, we recover the circular motion of the normalized spring. The magnitude
of 𝜔 sets the angular speed, while its sign sets the direction.
We can also write the formula as
𝑒𝑡(𝑎𝐼+𝜔𝐽)=𝑒𝑡𝑎𝐼𝑒𝑡𝜔𝐽.
This factorization works because 𝑎𝐼 and 𝜔𝐽 commute. It is not a
general law: if two matrices 𝐵 and 𝐶 do not commute, then
𝑒𝐵+𝐶 need not equal 𝑒𝐵𝑒𝐶.
These examples succeeded because the powers of each matrix followed a pattern
we could recognize. The damped spring brings us to a matrix for which that
pattern is not immediately visible. If the spring constant is 𝑘 and the
damping coefficient is 𝑏, its state 𝑋=(𝑥,𝑣) satisfies
𝑋′=(01−𝑘−𝑏)𝑋.
The theorem from the preceding section gives the exact solution
𝑋(𝑡)=𝑒𝑡𝐴𝑋0,𝐴=(01−𝑘−𝑏).
But the powers 𝐴,𝐴2,𝐴3,… do not reveal the motion as readily as the
powers of 𝐼, 𝐷, 𝑁, or 𝐽. The exponential notation has constructed the
flow without yet making it easy to calculate or interpret.
What makes the earlier matrices simple is that they act predictably along
particular directions. Can we find directions adapted to a more complicated
matrix, so that its action becomes simple there? That question leads from the
matrix exponential to the geometry of linear systems.