He is 78, an age when you usually only appear on television with the words "Missing" or "Chinese Premier" under you name. But Lou Holtz is leaving ESPN on a high note--namely that after years on television, he leaves with the same image coming out that he had going into the game, i.e. that Lou Holtz at 78 was just as barking insane as young Lou Holtz was.
Honestly, you probably won't miss Lou that much, even when you get the knee-jerk emotional reaction of "awww, that guy will no longer be here, even though I really only noticed him when he did crazy things once or twice a year." Holtz's departure and Rece Davis's shifting role at ESPN mean College Football Final is dead in the format as you know it. (They're probably not going to build a show around Mark May, unless that show is based strictly on Mark May's now-dormant Twitter feed. Which would be FINE.)
That doesn't mean the show itself is dead, mind you: as long as something's sponsored, ESPN will do it, and college football only sits second to the NFL in terms of being a hole advertisers are all too happy to throw money down. That's largerly a good thing, even with the cast change. Take three people, make them watch 14 hours of college football coverage, and then ask them to try to talk on television after midnight, and the results will be pretty similar no matter who you're working with in on screen. Holtz was a cast member, sure. But the real star of the show has been and will always be sleep deprivation.