Although I use Taylor series regularly, I have never understood precisely why they work.
The basic idea is simple enough. Suppose, by magic, it just happens to be true that
for some infinite list of real numbers .
Then by differentiating both sides of the sum times, we have
and for the particular value , we can simplify this to
. Plugging the back into the original sum, we obtain the Taylor series for about the point .
This is not satisfying because, as I said, it involves magic when assuming we can write a function as an infinite sum of polynomials. What’s been worrying me in particular is the following:
Here is a rough picture of a smooth function.
Here is a different one.
These pictures are exactly the same most of the way – after I made the first one, I made the second one by erasing the last part and redoing it – giving it a twist at the end.
The problem is that if I calculate a Taylor series for these functions centered about , I do it using solely local knowledge – just a bunch of derivatives right at that point.
In order to calculate derivatives at some point, you only need the value of the function there, and the values of the function within an infinitesimal neighborhood (by which I mean any finite neighborhood, no matter how small, is big enough). Is it really true that if I draw just a tiny little box around one part of a smooth function, and show you just what’s inside the box, you can tell me all the wild fluctuations the full function goes through anywhere from here to infinity?
If I only tweak the function way out on one of the ends, how are you supposed to know about it, just looking at your isolated little box? Shouldn’t my two example functions have the same Taylor series about ? Then they should be the same function. But they’re different.
Evidently, it is impossible to have two smooth functions that agree for a while, then go their separate ways. In fact, unless they’re the same function, it’s impossible for them to agree in any open interval at all. Why should this be true?
Let’s suppose the two smooth functions are and , and that they agree everywhere on the open interval . However, they do not have the same value everywhere. We’re looking for a contradiction in this setup.
Let’s define as the difference . Then is smooth because and are. We also know on , but is not zero everywhere. Form the set of all points for which . (If there are no such points, find all the points where and flip the rest of this argument around to match.)
is a lower bound for this set, and so the set must have a greatest lower bound. Because the real numbers are complete, the greatest lower bound must be a real number. Let's denote this number by , so that now on , but . This implies is discontinuous, because , but . This contradicts with being smooth.
We must conclude that the original assumption was false. If and are two smooth functions that agree on some open interval, they agree everywhere. (This is not intended to be a rigorous proof, since I am mathematically incompetent to produce one.)
So somehow, when I tried to make those two pictures that fit together smoothly, I messed up. When I erased the right hand end of the first one and drew a new ending in for it, I was supposed to match all the derivatives of the original function with my new one. I might have matched the first derivative, second derivative, and third derivative, but somewhere along that line I went awry.
The only way I could have fit my new function to the old one for all their derivatives is to have changed the old one just a little bit in the process. I would have had to change it at every value, including all the way over at , even though I'm just trying to put a new little blip on the function way over at .
This tiny little change to the original function shows up in that box you were using to monitor a little chunk of the function, and if your spectacles are powerful enough, you can extrapolate to determine exactly what changes I've made out on the periphery.
This seems pretty nice. It eases some of my concerns. It doesn't prove that the Taylor series of a smooth function converges, but it does show that the concept, that of describing an entire smooth function by its local characteristics at a given point, makes sense.
If I want to prove that Taylor series converge, I now only have to do it on some infinitesimal interval. Then, so long as the series converges to some value, it must converge to the correct one.