In this note, we prove the Weierstrass approximation theorem.
1. Bernstein’s approximation
Theorem. Let be continuous and let be given. Then there exists a polynomial such that .
We can construct a polynomial explicitly. For each , we define
We call this polynomial as Bernstein polynomials. The polynomial can be interpreted in terms of probabilistic idea.
For we interpret as a probability of getting a head of the coin(success) and as a probability of getting a tail of the coin(fail). In tosses, the probability of -success is
In the polynomial, has the meaning ` dollars is paid out when exactly heads turn up when tosses are made). So the average amount paid out when tosses are made is
As then in a typical game, . So the average payout converges to .
This is an intuitive idea, not a rigorous proof. Since most of people don’t know the theory of probability, we give a proof which does not contains any probabilistic interpretation.
Differentiating the identity with respect to . Then
Again differentiating the identity with respect to . Then
Put . Then
Choose so that on . Since is uniformly continuous, for given , there exists such that implies .
We divide into two parts: (i) (ii) .
If , then . So in this case, the summation is bounded by since and . For the second type, the corresponding summation is bounded by
since . By (1), this is bounded by
since on . Thus, for every , there is a such that
Choose so that . Then for any ,
This completes the proof.
2. Landau’s approximation
There is another proof using special `approximation to the identity’. The definition of approximation to the identity is the following:
Definition. We say a family is said to be an approximation to the identity if
- For all ,
- There exists such that for all
- For every ,
Roughly speaking, this concept concentrate and localize a function. One motivation of this concept is a dirac delta function. We leave the following theorem as an exercise.
Theorem. Let be a family of approximation to the identity. If is continuous everywhere, then
converges uniformly to .
Hence the above theorem gives an uniform approximation to continuous function. From now on, we denote .
Define the Landau kernels by
where is chosen so that .
Then one can check is a family of approximation to the identity.
Now we are ready to prove the Weierstrass approximation theorem again. By considering the translation, it suffices to consider a continuous function supported in .
Since is a family of approximation to the identity, uniformly. Note that is a sequence of polynomials on . This completes the proof of Weierstrass approximation theorem.
First of all, with some a slight modification, Bernstein polynomial can be generalized to multivariable setting. Moreover, the following theorem holds.
Theorem. Let be a real-valued function defined on with continuous derivatives on for all . Then there exists a sequence of polynomials in such that converges to , respectively, for all , uniformly on any compact set.
The proof of this theorem is quite technical. So we omit the proof. One can see the proof e.g. Krylov’s introduction to the theory of diffusion. But this book is not appropriate to those who are taking undergraduate analysis course. Here the proof uses some theory of probability theory, the weak law of large numbers. One may ask why such theorem is needed. This theorem is a quite good lemma for proving some fundamental theorem in the theory of stochastic integrals.
Another way to generalize the theorem in the following sense. Let denotes the space of polynomials. The Weierstrass theorem gives is dense in
with the uniform norm. Actually, the class can be regarded as a special case. This was proved by Stone.
A family of of real functions defined on a set is said to be an algebra if (i) , (ii) and (iii) for all and for all real constants .
If has the property that whenever and uniformly on , we say uniformly closed.
We say separate points on if to every pair of distinct points , there exists a function such that .
Note that the class clearly satisfies separate points on .
Theorem (Stone). Let be an algebra of real continuous function on a compact set . If separates points on and if vanishes at no point of , then the set of all functions which is the limits of uniformly convergent sequence of members of consists of all real continuous functions
In this sense, the theorem holds not only in but also . Moreover, the theorem does not depend on Euclidean structure. Those who are interested in its proof, see Rudin’s PMA.