## Jim Hefferon's Linear Algebra: A free textbook with fascinating applications

Posted by Eric Kidd Wed, 07 Mar 2007 08:01:00 GMT

My college linear algebra course was held early in the morning, and it was devoted almost entirely to blackboard proofs. The professor would stand in front of the room, half asleep, and write:

“Theorem. Lemma. Lemma. Proof. Theorem…”

Despite this experience, I somehow managed to learn about eigenvectors and kernels. Or at least, I learned how to write proofs about them. But I had no intuition for linear algebra: I couldn’t visualize it, and I couldn’t explain why anybody, anywhere, ever cared about eigenvectors.

Years later, in a computer vision class, I finally learned to care about linear algebra. It could solve all sorts of cool problems! (Eigenfaces, in particular, blew me away.) And since then, I’ve encountered linear algebra everywhere. But my intuition is still piecemeal, built from half-a-dozen applications over the years.

My motto for math is, “If it keeps showing up, build a rock-solid intuition for how it works.” And towards that end, I’ve been looking for a good linear algebra textbook.

My ideal linear algebra textbook would:

1. Include plenty of motivating examples.
2. Show how to solve real-world problems.
3. Devote plenty of time to proofs.

The proofs, after all, are necessary in the real world. If you ever attempt to do something slightly odd, you’ll want to prove that it actually works.

### Jim Hefferon’s Linear Algebra

Professor Jim Hefferon’s Linear Algebra is available as a free PDF download. But don’t be fooled by the price: Hefferon’s book is better than most of the expensive tomes sold in college bookstores.

Everything in Hefferon’s book is superbly motivated. The first chapter begins with two real-world examples: Unknown weights placed on balances, and the ratios of complex molecules in chemical reactions. These examples are used to introduce Gauss’s method for solving systems of linear equations. Further into the book, the examples begin to tie back to earlier chapters. Determinants, for example, are motivated by the usefulness of recognizing isomorphisms and invertible matrices.

But Hefferon’s emphasis on real-world examples is admirably balanced by an abundance of proofs. The first proof appears on page 4, and nearly everything is proven either in the main text or in the exercises. This will be helpful for readers who (like me) are trying to bring more rigor to their mathematical thinking.

### The “Topics”: Fascinating real-world problems

The most delightful part of the book, however, are the “Topics” at the end of each chapter. These cover a wide range of fields, including biology, economics, probability and abstract algebra. The topic “Stable Populations” begins:

Imagine a reserve park with animals from a species that we are trying to protect. The park doesn’t have a fence and so animals cross the boundary, both from the inside out and in the other direction. Every year, 10% of the animals from inside of the park leave, and 1% of the animals from the outside find their way in. We can ask if we can find a stable level of population for this park: is there a population that, once established, will stay constant over time, with the number of animals leaving equal to the number of animals entering?

Hefferon relates the solution to Markov chains and eigenvalues, cementing several important intuitions firmly in place.

Other topics include basic electronics, space shuttle O-rings, and the number of games required to win the World Series. There are plenty of CS-related discussions, too: a survey of things that can go wrong in naive numeric code, the time required to calculate determinants, and how the memory hierarchy affects array layout.

Hefferon’s love for linear algebra is infectious, and his “Topics” will appeal to anybody who does recreational math.

### “Free” as in “freedom”

Linear Algebra is published under the GNU Free Documentation License and the Creative Commons Share Alike license.

What this means: You may make copies of the book, or even print them out at a copyshop and charge students a fee. You may also create a custom version of the textbook, and share it with anybody who’s interested. The only restriction: You must “share alike,” honoring the original author’s terms as you pass along the textbook.

### Miscellaneous notes

Hefferon has put out a call for extra material. In particular, he’d love to have a section on quantum mechanics:

Several people have asked me about a Topic on eigenvectors and eigenvalues in Quantum Mechanics. Sadly, I don’t know any QM. If you can help, that’d be great.

On the downside, the internal PDF links in Linear Algebra are broken in MacOS X Preview. This is odd, because the LaTeX `hyperref` package usually works fine with Preview.

The reddit discussion of Linear Algebra has pointers to several other linear algebra textbooks, with varying emphasis. And many other free math textbooks are available online.

If you have any favorite math books (paper or PDF, for any area of mathematics), please feel free to recommend them in the comment thread!

Tags

1. Michael said about 3 hours later:

I definitely second your enthusiastic recommendation of Hefferon’s book—I have also found it a great resource for those problems where I need something more than the essential theorems in order to really get my teeth in.

2. Jon Shea said about 3 hours later:

I couldn’t agree more with your assessment of undergraduate Linear Algebra classes. They are used primarily, if not exclusively, as a vehicle for teaching proofs. I’m furious that I wasn’t taught the SVD until graduate school.

I can’t wait to look at the book, thanks for the recommendation.

3. JJ said about 4 hours later:

I fully agree when you said Hefferon’s text was superior to a lot of college Linear Algebra books selling for over \$ 100.

4. Darrin Thompson said about 6 hours later:

I got eaten alive by the time I reached chapter 3. But I’m getting better with producing proofs because I found the Haskell Road to Logic Maths and Programming. It’s a good remedial course for those of us familiar with programming but weak in abstract math chops.

It isn’t free, but quite good.

5. Eric said about 7 hours later:

Jon: I agree, SVD is insanely useful, and it should certainly be taught before grad school.

Darrin: The Haskell Road rocks my world. It’s an amazing book, especially for programmers who want to become better at discrete math. (No Haskell experience is required, and no math beyond high school algebra.) I read about six chapters in a weekend (ouch!), and my understanding of logic, relations and functions improved dramatically.

In general, doing abstract math in Haskell makes it a lot more accessible to me. In particular, it’s nice knowing the types of all the equations.

6. Peteris Krumins said about 8 hours later:

Check out this blog post about free mathematics video lectures: Mathematics Video Lectures @ Free Science Online

There is a free Linear Algebra video course available from MIT! MIT’s OCW Linear Algebra Video Lectures

7. Fred Ross said about 8 hours later:

Funnily enough, I wrote something apropos yesterday: A Mathematics Curriculum [Ed: Converted URL to link.]

Mind you, it’s aimed at an engineer I know, and I know his background (including quantum mechanics) in detail.

8. Rob Beezer said 2 days later:

Another GFDL’ed linear algebra text is A First Course in Linear Algebra. Lots of worked examples and exercises, and very careful attention to proofs. Links in PDF or MathML versions reference previous theorems or definitions. SVD is on the list to be added as part of next Thursday’s update.

9. Eric said 2 days later:

Peteris, Fred, Rob: Thanks for the links!

There’s also an SVD overview at UW La Crosse. I haven’t read through this one yet, but it looks quite interesting.

10. Sanchit! said 11 days later:

Hey I must say this book is great with so many examples and problems!

Thankxxxxxxx

11. Serious reviewer said 81 days later:

A text book should be lucid, expressing ideas clearly and simply. Every time its revisited it should leave one with a newer better insight. Hefferon’s book is too simplistic. Not the book i would want to go to once i understood the subject a little but needed a solid reference. Not worth the trouble of printing.

All professors who teach LA dont look asleep. That you get what you pay for is pretty true as far books (and linear algebra courses) go.

12. Michael said 81 days later:

Serious reviewer said: “A text book should be lucid, expressing ideas clearly and simply. Every time its revisited it should leave one with a newer better insight. Hefferon’s book is too simplistic.”

I’m surprised to hear you say this—can you give us an example of some section of the text you found to be particularly lacking clarity or lucidity? As someone who frequently has to present tricky mathematical ideas to sharp but inexperienced students, I actually found Dr. Hefferon’s presentation quite good. So, I would be interested to hear of some specific points that you found lacking.

If we use tuition cost as the measure of “you get what you pay for,” then I don’t think we get a very good correlation. An hour of class where Eric went to school costs a lot more than the same hour at St. Michael’s College where Dr. Hefferon teaches. And yet, if the textbook is any proxy for the quality of his lectures, then it seems like the cost doesn’t track the quality at all.

13. Bjorn said 357 days later:

Funny, I was googling linear algebra Mac os x and I found this page because of the comment on the broken links in Preview. Well, they work well in Preview in Leopard so maybe you could put a footnote pointing that. And now I have this book to refresh my linear algebra.

14. Arun J. C. said 364 days later:

I just searched linear agebra and this came up wonderful , really benifical better than any other book. thanks ….

15. Anon said 606 days later:

You should also take a look at Gilbert Strang’s Linear Algebra video lectures on MIT’s open courseware site. They’re pure gold.

http://web.mit.edu/18.06/www/Video/video-fall-99-new.html

16. Anand said 687 days later:

The book “Music: A Mathematical Offering” is another book that I find fascinating. It explains the mathematics behind signal processing in a very interesting manner. I love the way the book is titled and it does not disappoint you as you read through it.

Enjoy!

17. Anand said 687 days later:

http://www.maths.abdn.ac.uk/~bensondj/html/music.pdf

18. Roger said 789 days later:

Jim Hefferon’s Linear Algebra is a very balanced, substantial yet concise, with fascinating exercises. Thank you so much Mr. Hefferon for making such a book free. More power to you!!!

19. Regnessem said 817 days later:

“SVD is insanely useful” is true but in very limited sense. There’s plenty of richfull life out of the realm of ortonormal basis. Jim Hefferon really paves the road to move on beyond SVD!