Ciro Santilli
OurBigBook.comWell summarized as "the branch of mathematics that deals with limits".
A fancy name for calculus, with the "more advanced" connotation.
The fundamental concept of calculus!
The reason why the epsilon delta definition is so venerated is that it fits directly into well known methods of the formalization of mathematics, making the notion completely precise.
This is a general philosophy that Ciro Santilli, and likely others, observes over and over.
Basically, continuity, or higher order conditions like differentiability seem to impose greater constraints on problems, which make them more solvable.
Some good examples of that:
- complex discrete problems:
- simple continuous problems:
- characterization of Lie groups
Something that is very not continuous.
Notably studied in discrete mathematics.
Chuck Norris counted to infinity. Twice.
There are a few related concepts that are called infinity in mathematics:
- limits that are greater than any number
- the cardinality of a set that does not have a finite number of elements
- in some number systems, there is an explicit "element at infinity" that is not a limit, e.g. projective geometry
The derivative of a function gives its slope at a point.
More precisely, it give sthe inclination of a tangent line that passes through that point.
Here's an example of the chain rule. Suppose we want to calculate:So we have:and so:Therefore the final result is:
math.stackexchange.com/questions/1786964/is-it-possible-to-construct-a-smooth-flat-top-bump-function
Given a function :we want to find the points of the domain of where the value of is smaller (for minima, or larger for maxima) than all other points in some neighbourhood of .
- from some space. For beginners the real numbers but more generally topological spaces should work in general
- to the real numbers
In the case of Functionals, this problem is treated under the theory of the calculus of variations.
Tagged
Nope, it is not a Greek letter, notably it is not a lowercase delta. It is just some random made up symbol that looks like a letter D. Which is of course derived from delta, which is why it is all so damn confusing.
I think the symbol is usually just read as "D" as in "d f d x" for .
This notation is not so common in basic mathematics, but it is so incredibly convenient, especially with Einstein notation as shown at Section "Einstein notation for partial derivatives":
This notation is similar to partial label partial derivative notation, but it uses indices instead of labels such as , , etc.
The total derivative of a function assigns for every point of the domain a linear map with same domain, which is the best linear approximation to the function value around this point, i.e. the tangent plane.
The easy and less generic integral. The harder one is the Lebesgue integral.
"More complex and general" integral. Matches the Riemann integral for "simple functions", but also works for some "funkier" functions that Riemann does not work for.
Ciro Santilli sometimes wonders how much someone can gain from learning this besides the beauty of mathematics, since we can hand-wave a Lebesgue integral on almost anything that is of practical use. The beauty is good reason enough though.
Advantages over Riemann:
- Lebesgue integral of is complete but Riemann isn't.
- youtu.be/PGPZ0P1PJfw?t=710 you are able to switch the order of integrals and limits of function sequences on non-uniform convergence. TODO why do we care? This is linked to the Fourier series of course, but concrete example?
Riemann integral vs. Lebesgue integral by The Bright Side Of Mathematics (2018)
Source. youtube.com/watch?v=PGPZ0P1PJfw&t=808 shows how Lebesgue can be visualized as a partition of the function range instead of domain, and then you just have to be able to measure the size of pre-images.
One advantage of that is that the range is always one dimensional.
But the main advantage is that having infinitely many discontinuities does not matter.
Infinitely many discontinuities can make the Riemann partitioning diverge.
But in Lebesgue, you are instead measuring the size of preimage, and to fit infinitely many discontinuities in a finite domain, the size of this preimage is going to be zero.
So then the question becomes more of "how to define the measure of a subset of the domain".
Which is why we then fall into measure theory!
In "practice" it is likely "useless", because the functions that it can integrate that Riemann can't are just too funky to appear in practice :-)
Its value is much more indirect and subtle, as in "it serves as a solid basis of quantum mechanics" due to the definition of Hilbert spaces.
Bibliography:
is:
- complete under the Lebesgue integral, this result is may be called the Riesz-Fischer theorem
- not complete under the Riemann integral: math.stackexchange.com/questions/397369/space-of-riemann-integrable-functions-not-complete
And then this is why quantum mechanics basically lives in : not being complete makes no sense physically, it would mean that you can get closer and closer to states that don't exist!
TODO intuition
A measurable function defined on a closed interval is square integrable (and therefore in ) if and only if Fourier series converges in norm the function:
TODO
Riesz-Fischer theorem is a norm version of it, and Carleson's theorem is stronger pointwise almost everywhere version.
Note that the Riesz-Fischer theorem is weaker because the pointwise limit could not exist just according to it: norm sequence convergence does not imply pointwise convergence.
There are explicit examples of this. We can have ever thinner disturbances to convergence that keep getting less and less area, but never cease to move around.
If it does converge pointwise to something, then it must match of course.
The Fourier series of an function (i.e. the function generated from the infinite sum of weighted sines) converges to the function pointwise almost everywhere.
The theorem also seems to hold (maybe trivially given the transform result) for the Fourier series (TODO if trivially, why trivially).
Only proved in 1966, and known to be a hard result without any known simple proof.
This theorem of course implies that Fourier basis is complete for , as it explicitly constructs a decomposition into the Fourier basis for every single function.
TODO vs Riesz-Fischer theorem. Is this just a stronger pointwise result, while Riesz-Fischer is about norms only?
One of the many fourier inversion theorems.
Integrable functions to the power , usually and in this text assumed under the Lebesgue integral because: Lebesgue integral of is complete but Riemann isn't
is by far the most important of because it is quantum mechanics states live, because the total probability of being in any state has to be 1!
has some crucially important properties that other don't (TODO confirm and make those more precise):
- it is the only that is Hilbert space because it is the only one where an inner product compatible with the metric can be defined:
- Fourier basis is complete for , which is great for solving differential equation
Some sources say that this is just the part that says that the norm of a function is the same as the norm of its Fourier transform.
Others say that this theorem actually says that the Fourier transform is bijective.
The comment at math.stackexchange.com/questions/446870/bijectiveness-injectiveness-and-surjectiveness-of-fourier-transformation-define/1235725#1235725 may be of interest, it says that the bijection statement is an easy consequence from the norm one, thus the confusion.
TODO does it require it to be in as well? Wikipedia en.wikipedia.org/w/index.php?title=Plancherel_theorem&oldid=987110841 says yes, but courses.maths.ox.ac.uk/node/view_material/53981 does not mention it.
As mentioned at Section "Plancherel theorem", some people call this part of Plancherel theorem, while others say it is just a corollary.
This is an important fact in quantum mechanics, since it is because of this that it makes sense to talk about position and momentum space as two dual representations of the wave function that contain the exact same amount of information.
But only for the proper Riemann integral: math.stackexchange.com/questions/2293902/functions-that-are-riemann-integrable-but-not-lebesgue-integrable
Main motivation: Lebesgue integral.
The Bright Side Of Mathematics 2019 playlist: www.youtube.com/watch?v=xZ69KEg7ccU&list=PLBh2i93oe2qvMVqAzsX1Kuv6-4fjazZ8j
The key idea, is that we can't define a measure for the power set of R. Rather, we must select a large measurable subset, and the Borel sigma algebra is a good choice that matches intuitions.
Approximates an original function by sines. If the function is "well behaved enough", the approximation is to arbitrary precision.
Fourier's original motivation, and a key application, is solving partial differential equations with the Fourier series.
Can only be used to approximate for periodic functions (obviously from its definition!). The Fourier transform however overcomes that restriction:
The Fourier series behaves really nicely in , where it always exists and converges pointwise to the function: Carleson's theorem.
But what is a Fourier series? by 3Blue1Brown (2019)
Source. Amazing 2D visualization of the decomposition of complex functions.See: math.stackexchange.com/questions/579453/real-world-application-of-fourier-series/3729366#3729366 from heat equation solution with Fourier series.
Separation of variables of certain equations like the heat equation and wave equation are solved immediately by calculating the Fourier series of initial conditions!
Other basis besides the Fourier series show up for other equations, e.g.:
Output: another sequence of complex numbers such that:Intuitively, this means that we are braking up the complex signal into sinusoidal frequencies:and is the amplitude of each sine.
- : is kind of magic and ends up being a constant added to the signal because
- : sinusoidal that completes one cycle over the signal. The larger the , the larger the resolution of that sinusoidal. But it completes one cycle regardless.
- : sinusoidal that completes two cycles over the signal
- ...
- : sinusoidal that completes cycles over the signal
We use Zero-based numbering in our definitions because it just makes every formula simpler.
Motivation: similar to the Fourier transform:In particular, the discrete Fourier transform is used in signal processing after a analog-to-digital converter. Digital signal processing historically likely grew more and more over analog processing as digital processors got faster and faster as it gives more flexibility in algorithm design.
- compression: a sine would use N points in the time domain, but in the frequency domain just one, so we can throw the rest away. A sum of two sines, only two. So if your signal has periodicity, in general you can compress it with the transform
- noise removal: many systems add noise only at certain frequencies, which are hopefully different from the main frequencies of the actual signal. By doing the transform, we can remove those frequencies to attain a better signal-to-noise
Sample software implementations:
- numpy.fft, notably see the example: numpy/fft.py
DFT of with 25 points
. This is a simple example of a discrete Fourier transform for a real input signal. It illustrates how the DFT takes N complex numbers as input, and produces N complex numbers as output. It also illustrates how the discrete Fourier transform of a real signal is symmetric around the center point.See sections: "Example 1 - N even", "Example 2 - N odd" and "Representation in terms of sines and cosines" of www.statlect.com/matrix-algebra/discrete-Fourier-transform-of-a-real-signal
The transform still has complex numbers.
Summary:Therefore, we only need about half of to represent the signal, as the other half can be derived by conjugation.
- is real
"Representation in terms of sines and cosines" from www.statlect.com/matrix-algebra/discrete-Fourier-transform-of-a-real-signal then gives explicit formulas in terms of .
NumPy for example has "Real FFTs" for this: numpy.org/doc/1.24/reference/routines.fft.html#real-ffts
DFT of with 25 points
. Source at: numpy/fft_plot.py. This plot illustrates how the DFT of a real signal is symmetric around the middle point, and so only half of the transform points are needed to reconstruct the original signal. We also see how the phase of the sinusoids determines if their DFT components are real or imaginary.There are actually two possible definitions for the DFT:
- 1/N, given as "the default" in many sources:
- , known as the "normalized DFT" by some sources: www.dsprelated.com/freebooks/mdft/Normalized_DFT.html, definition which we adopt:
The is nicer mathematically as the inverse becomse more symmetric, and power is conserved between time and frequency domains.
An efficient algorithm to calculate the discrete Fourier transform.
Continuous version of the Fourier series.
Can be used to represent functions that are not periodic: math.stackexchange.com/questions/221137/what-is-the-difference-between-fourier-series-and-fourier-transformation while the Fourier series is only for periodic functions.
Of course, every function defined on a finite line segment (i.e. a compact space).
Therefore, the Fourier transform can be seen as a generalization of the Fourier series that can also decompose functions defined on the entire real line.
As a more concrete example, just like the Fourier series is how you solve the heat equation on a line segment with Dirichlet boundary conditions as shown at: Section "Solving partial differential equations with the Fourier series", the Fourier transform is what you need to solve the problem when the domain is the entire real line.
Lecture notes:
- www.robots.ox.ac.uk/~az/lectures/ia/lect2.pdf Lecture 2: 2D Fourier transforms and applications by A. Zisserman (2014)
How the 2D FFT works by Mike X Cohen (2017)
Source. Animations showing how the 2D Fourier transform looks like for simple inpuf functions.A set of theorems that prove under different conditions that the Fourier transform has an inverse for a given space, examples:
The Laplace Transform: A Generalized Fourier Transform by Steve Brunton (2020)
Source. Explains how the Laplace transform works for functions that do not go to zero on infinity, which is a requirement for the Fourier transform. No applications in that video yet unfortunately.First published by Fourier in 1807 to solve the heat equation.
Topology is the plumbing of calculus.
The key concept of topology is a neighbourhood.
Just by havin the notion of neighbourhood, concepts such as limit and continuity can be defined without the need to specify a precise numerical value to the distance between two points with a metric.
As an example. consider the orthogonal group, which is also naturally a topological space. That group does not usually have a notion of distance defined for it by default. However, we can still talk about certain properties of it, e.g. that the orthogonal group is compact, and that the orthogonal group has two connected components.
Basically it is a larger space such that there exists a surjection from the large space onto the smaller space, while still being compatible with the topology of the small space.
We can characterize the cover by how injective the function is. E.g. if two elements of the large space map to each element of the small space, then we have a double cover and so on.
The key concept of topology.
We map each point and a small enough neighbourhood of it to , so we can talk about the manifold points in terms of coordinates.
Does not require any further structure besides a consistent topological map. Notably, does not require metric nor an addition operation to make a vector space.
A notable example of a Non-Euclidean geometry manifold is the space of generalized coordinates of a Lagrangian. For example, in a problem such as the double pendulum, some of those generalized coordinates could be angles, which wrap around and thus are not euclidean.
Collection of coordinate charts.
The key element in the definition of a manifold.
A generalized definition of derivative that works on manifolds.
TODO: how does it maintain a single value even across different coordinate charts?
TODO find a concrete numerical example of doing calculus on a differentiable manifold and visualizing it. Likely start with a boring circle. That would be sweet...
TODO what's the point of it.
Bibliography:
- www.youtube.com/watch?v=j1PAxNKB_Zc Manifolds #6 - Tangent Space (Detail) by WHYB maths (2020). This is worth looking into.
- www.youtube.com/watch?v=oxB4aH8h5j4 actually gives a more concrete example. Basically, the vectors are defined by saying "we are doing the Directional derivative of any function along this direction".One thing to remember is that of course, the most convenient way to define a function and to specify a direction, is by using one of the coordinate charts.We can then just switch between charts by change of basis.
- jakobschwichtenberg.com/lie-algebra-able-describe-group/ by Jakob Schwichtenberg
- math.stackexchange.com/questions/1388144/what-exactly-is-a-tangent-vector/2714944 What exactly is a tangent vector? on Stack Exchange
A member of a tangent space.
www.youtube.com/watch?v=tq7sb3toTww&list=PLxBAVPVHJPcrNrcEBKbqC_ykiVqfxZgNl&index=19 mentions that it is a bit like a dot product but for a tangent vector to a manifold: it measures how much that vector derives along a given direction.
A metric is a function that give the distance, i.e. a real number, between any two elements of a space.
A metric may be induced from a norm as shown at: Section "Metric induced by a norm".
Because a norm can be induced by an inner product, and the inner product given by the matrix representation of a positive definite symmetric bilinear form, in simple cases metrics can also be represented by a matrix.
Canonical example: Euclidean space.
Tagged
TODO examples:
- metric space that is not a normed vector space
- norm vs metric: a norm gives size of one element. A metric is the distance between two elements. Given a norm in a space with subtraction, we can obtain a distance function: the metric induced by a norm.
In plain English: the space has no visible holes. If you start walking less and less on each step, you always converge to something that also falls in the space.
One notable example where completeness matters: Lebesgue integral of is complete but Riemann isn't.
Subcase of a normed vector space, therefore also necessarily a vector space.
Appears to be analogous to the dot product, but also defined for infinite dimensions.
Vs metric:
- a norm is the size of one element. A metric is the distance between two elements.
- a norm is only defined on a vector space. A metric could be defined on something that is not a vector space. Most basic examples however are also vector spaces.
Metric space but where the distance between two distinct points can be zero.
Notable example: Minkowski space.
Tagged
When a disconnected space is made up of several smaller connected spaces, then each smaller component is called a "connected component" of the larger space.
See for example the
There are two cases:
- (topological) manifolds
- differential manifolds
Questions: are all compact manifolds / differential manifolds homotopic / diffeomorphic to the sphere in that dimension?
- for topological manifolds: this is a generalization of the Poincaré conjecture.Original problem posed, for topological manifolds.Last to be proven, only the 4-differential manifold case missing as of 2013.Even the truth for all was proven in the 60's!Why is low dimension harder than high dimension?? Surprise!AKA: classification of compact 3-manifolds. The result turned out to be even simpler than compact 2-manifolds: there is only one, and it is equal to the 3-sphere.For dimension two, we know there are infinitely many: classification of closed surfaces
- for differential manifolds:Not true in general. First counter example is . Surprise: what is special about the number 7!?Counter examples are called exotic spheres.Totally unpredictable count table:is an open problem, there could even be infinitely many. Again, why are things more complicated in lower dimensions??
Dimension | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | Smooth types | 1 | 1 | 1 | ? | 1 | 1 | 28 | 2 | 8 | 6 | 992 | 1 | 3 | 2 | 16256 | 2 | 16 | 16 | 523264 | 24 |
So simple!! You can either:
- cut two holes and glue a handle. This is easy to visualize as it can be embedded in : you just get a Torus, then a double torus, and so on
- cut a single hole and glue a Möbius strip in it. Keep in mind that this is possible because the Möbius strip has a single boundary just like the hole you just cut. This leads to another infinite family that starts with:
A handle cancels out a Möbius strip, so adding one of each does not lead to a new object.
You can glue a Mobius strip into a single hole in dimension larger than 3! And it gives you a Klein bottle!
Intuitively speaking, they can be sees as the smooth surfaces in N-dimensional space (called an embedding), such that deforming them is allowed. 4-dimensions is enough to embed cover all the cases: 3 is not enough because of the Klein bottle and family.
Important 4D spaces:
Simulate it. Just simulate it.
math.stackexchange.com/questions/466707/what-are-some-examples-of-infinite-dimensional-vector-spaces
This section is about the definition of the dot product over , which extends the definition of the dot product over .
Some motivation is discussed at: math.stackexchange.com/questions/2459814/what-is-the-dot-product-of-complex-vectors/4300169#4300169
The complex dot product is defined as:
E.g. in :
Just like the usual dot product, this will be a positive definite symmetric bilinear form by definition.
with extra structure added to make it into a metric space.
The identity matrix.
Each elliptic space can be modelled with a real projective space. The best thing is to just start thinking about the real projective plane.
Generalize function to allow adding some useful things which people wanted to be classical functions but which are not,
It therefore requires you to redefine and reprove all of calculus.
For this reason, most people are tempted to assume that all the hand wavy intuitive arguments undergrad teachers give are true and just move on with life. And they generally are.
One notable example where distributions pop up are the eigenvectors of the position operator in quantum mechanics, which are given by Dirac delta functions, which is most commonly rigorously defined in terms of distribution.
Distributions are also defined in a way that allows you to do calculus on them. Notably, you can define a derivative, and the derivative of the Heaviside step function is the Dirac delta function.
The "0-width" pulse distribution that integrates to a step.
There's not way to describe it as a classical function, making it the most important example of a distribution.
Applications:
- position operator in quantum mechanics. It's not a coincidence that the function is named after Paul Dirac.
The surprising thing is that a bunch of results are simpler in complex analysis!
Being a complex holomorphic function is an extremely strong condition.
The existence of the first derivative implies the existence of all derivatives.
Another extremely strong consequence is the identity theorem.
"Holos" means "entire" in Greek, so maybe this is a reference to the fact that due to the identity theorem, knowing the function on a small open ball implies knowing the function everywhere.
visualizing the Riemann hypothesis and analytic continuation by 3Blue1Brown (2016) is a good quick visual non-mathematical introduction is to it.
The key question is: how can this continuation be unique since we are defining the function outside of its original domain?
The answer is: due to the identity theorem.
Good ultra quick visual non-mathematical introduction to the Riemann hypothesis and analytic continuation.
Essentially, defining an holomorphic function on any open subset, no matter how small, also uniquely defines it everywhere.
This is basically why it makes sense to talk about analytic continuation at all.
One way to think about this is because the Taylor series matches the exact value of an holomorphic function no matter how large the difference from the starting point.
Therefore a holomorphic function basically only contains as much information as a countable sequence of numbers.
visualizing the Riemann hypothesis and analytic continuation by 3Blue1Brown (2016) is a good quick visual non-mathematical introduction is to it.
One of the Millennium Prize Problems and Hilbert's problems.
What is the Riemann hypothesis REALLY about? by HexagonVideos (2022)
Source. Key for quantum mechanics, see: mathematical formulation of quantum mechanics, the most important example by far being .
Finding a complete basis such that each vector solves a given differential equation is the basic method of solving partial differential equation through separation of variables.
The first example of this you must see is solving partial differential equations with the Fourier series.
Notable examples:
- Fourier series for the heat equation as shown at Fourier basis is complete for and solving partial differential equations with the Fourier series
- Hermite functions for the quantum harmonic oscillator
- Legendre polynomials for Laplace's equation in spherical coordinates
- Bessel function for the 2D wave equation on a circular domain in polar coordinates
Sample software implementations:
The name is a bit obscure if you don't think in very generalized terms right out of the gate. It refers to a linear polynomial of multiple variables, which by definition must have the super simple form of:and then we just put the unknown and each derivative into that simple polynomial:except that now the are not just constants, but they can also depend on the argument (but not on or its derivatives).
Explicit solutions exist for the very specific cases of:
- constant coefficients, any degree. These were known for a long time, and are were studied when Ciro was at university in the University of São Paulo.
- degree 1 and any coefficient
Order of the highest derivative that appears.
Technique to solve partial differential equations
Naturally leads to the Fourier series, see: solving partial differential equations with the Fourier series, and to other analogous expansions:
One notable application is the solution of the Schrödinger equation via the time-independent Schrödinger equation.
Bibliography:
The finite element method is one of the most common ways to solve PDEs in practice.
Used for example in FreeFem and FEniCS Project as the input description of the PDEs, TODO why.
Used to solve partial differential equation.
TODO understand, give intuition, justification of bounds and JavaScript demo.
The majority likely comes from physics:
Like a heat equation but for functions without time dependence, space-only.
TODO confirm: does the solution of the heat equation always converge to the solution of the Laplace equation as time tends to infinity?
In one dimension, the Laplace equation is boring as it is just a straight line since the second derivative must be 0. That also matches our intuition of the limit solution of the heat equation.
Uniqueness: Uniqueness theorem for Poisson's equation.
Show up when solving the Laplace's equation on spherical coordinates by separation of variables, which leads to the differential equation shown at: en.wikipedia.org/w/index.php?title=Legendre_polynomials&oldid=1018881414#Definition_via_differential_equation.
Generalization of Laplace's equation where the value is not necessarily 0.
A solution to Laplace's equation.
Correspond to the angular part of Laplace's equation in spherical coordinates after using separation of variables as shown at: en.wikipedia.org/wiki/Spherical_harmonics#Laplace's_spherical_harmonics
Besides being useful in engineering, it was very important historically from a "development of mathematics point of view", e.g. it was the initial motivation for the Fourier series.
Some interesting properties:
- TODO confirm: for a fixed boundary condition that does not depend on time, the solutions always approaches one specific equilibrium function.This is in contrast notably with the wave equation, which can oscillate forever.
- TODO: for a given point, can the temperature go down and then up, or is it always monotonic with time?
- information propagates instantly to infinitely far. Again in contrast to the wave equation, where information propagates at wave speed.
Sample numerical solutions:
See: math.stackexchange.com/questions/579453/real-world-application-of-fourier-series/3729366#3729366
Describes perfect lossless waves on the surface of a string, or on a water surface.
As mentioned at: math.stackexchange.com/questions/579453/real-world-application-of-fourier-series/3729366#3729366 from solving partial differential equations with the Fourier series citing courses.maths.ox.ac.uk/node/view_material/1720, analogously to the heat equation, the wave linear equation can be be solved nicely with separation of variables.
Staring from a triangle wave, this explains why we always get the same musical notes:See also: solving partial differential equations with the Fourier series.
- www.math.hmc.edu/~ajb/PCMI/lecture7.pdf "7.5.1. Musical instruments" is very good. Also mentions that in the piano it is more like an initial speed is applied, and it is not the same as plucking
- music.stackexchange.com/questions/135635/confusion-about-overtones-and-a-slow-motion-video-of-a-plucked-string
- music.stackexchange.com/questions/60833/what-determines-the-relative-volumes-of-the-harmonics-when-plucking-a-guitar-str
TODO: do higher overtones decay faster in time than the base ones?
- www.physicsforums.com/threads/why-do-harmonics-decay-faster-than-the-fundamental.955731/ But presumaby yes, damping force is proportional to speed, and higher harmonics have higher speeds going up and down
Slow motion: rubber string pulled and released by Pavel Radzivilovsky
. Source. Good symmetric example. But unfortunately the video is too short.
This section talks about solvers/simulators dedicated solving the wave equation. Of course, any serious solver will likely be able to solve a wider range of PDE, so this section contains mostly fun toys. For more serious stuff see: Section "PDE solver".
JavaScript toy solvers:
- jtiscione.github.io/webassembly-wave/index.html circular domain, create waves with mouse click
- dionyziz.com/graphics/wave-experiment/ with useless 3D WebGL visualization :-), waves with mouse click. Solving itself done on CPU, not GPU.
web.archive.org/web/20200621205928/https://courses.maths.ox.ac.uk/node/view_material/1720 also mentioned at math.stackexchange.com/questions/579453/real-world-application-of-fourier-series/3729366#3729366 from heat equation solution with Fourier series.
TODO confirm, see also: coupled oscillators. And then this idea can be used to define/motivate quantum field theory in terms of quantum harmonic oscillators with second quantization.
- youtu.be/SMmFgIEGYtw?t=324 Quantum Field Theory 2a - Field Quantization I by ViaScience (2018)
Standing wave caused by wave interference of two waves
. Source. 
Start with: Section "String polarization".
Then go to: Section "Polarization of light".
This is about the polarization of a string in 3D space. That is the first concept of polarization you must have in mind!
Tagged
Approximation to Huygens-Fresnel principle.
Far field approximation to Kirchhoff's diffraction formula, i.e. when the plane of observation is far from the object diffracting.
Near field approximation to Kirchhoff's diffraction formula, i.e. when the plane of observation is near the object diffracting.
Resonance is a really cool thing.
Examples:
- mechanical resonance, notably:
- pipe instruments
- electronic oscillators, notably:
- LC oscillator, and notably the lossy version RLC circuit
Perhaps a key insight of resonance is that the reonant any lossy system tends to look like the resonance frequency quite quickly even if the initial condition is not the resonant condition itself, because everything that is not the resonant frequency interferes destructively and becomes noise. Some examples of that:
- striking a bell or drum can be modelled by applying an impuse to the system
- playing a pipe instrument comes down to blowing a piece that vibrates randomly, and then leads the pipe to vibrate mostly in the resonant frequency. Likely the same applies to bowed string instruments, the bow must be creating a random vibration.
- playing a plucked string instrument comes down to initializing the system to an triangular wave form and then letting it evolve. TODO find a simulation of that!
Another cool aspect of resonance is that it was kind of the motivation for de Broglie hypothesis, as de Broglie was kind of thinking that electroncs might show discrete jumps on atomic spectra because of constructive interference.
What you see along a line or plane in a wave interference.
Notably used for the pattern of the double-slit experiment.
Shows up when trying to solve 2D wave equation on a circular domain in polar coordinates with separation of variables, where we have to decompose the initial condition in termes of a fourier-Bessel series, exactly like the Fourier series appears when solving the wave equation in linear coordinates.
For the same fundamental reasons, also appears when calculating the Schrödinger equation solution for the hydrogen atom.
Completeness: math.stackexchange.com/questions/2192665/is-this-set-of-bessel-functions-a-basis-for-all-c10-a-functions TODO
This is the Bessel function analogue to Fourier basis is complete for .
eigenvalue problem of Laplace's equation.
If you have a PDE that models physical phenomena, it is fundamental that:
- there must exist a solution for every physically valid initial condition, otherwise it means that the equation does not describe certain cases of reality
- the solution must be unique, otherwise how are we to choose between the multiple solutions?
Unlike for ordinary differential equations which have the Picard–Lindelöf theorem, the existence and uniqueness of solution is not well solved for PDEs.
For example, Navier-Stokes existence and smoothness was one of the Millennium Prize Problems.
Started in 1987 and written in Pascal, by the French from Pierre and Marie Curie University, the French are really strong in numerical analysis.
Ciro wasn't expecting it to be as old. Ported to C++ in 1992.
The fact that French wrote it can be seen in the documentation, for example doc.freefem.org/tutorials/index.html uses file extension
mycode.edp instead of mycode.pde where dep stands for "Équation aux dérivées partielles".Besides the painful build, using FreeFem is relatively simple, as can be seen from the examples on the website.
They do use a domain-specific language on the examples, which appears to be the main/only interface, which is a bad thing, Ciro would rather have a Python API as the "main API", which is more the approach taken by the FEniCS Project, but so be it. This domain-specific language business means that you always stumble upon basic stuff you want to do but can't, and then you have to think about how to share data between the simulation and the plotting. The plotting notably is super complex and they can't implement all of what people want, upstream examples often offload that to gnuplot. This is potentially a big advantage of FEniCS Project.
It nice though that they do have some graphics out of the box, as that allows to quickly debug common problems.
Uses variational formulation of a partial differential equation, which is not immediately obvious to beginners? The introduction doc.freefem.org/tutorials/poisson.html gives an ultra quick example, but your are mostly on your own with that.
On Ubuntu 20.04, the
freefem is a bit out-of-date (3.5.8, there isn't even a tag for that in the GitHub repo, and refs/tags/release_3_10 is from 2010!) and fails to run the examples from the website. It did work with the example package though, but the output does not have color, which makes me sad :-)sudo apt install freefem freefem-examples
freefem /usr/share/doc/freefem-examples/heat.pdeSo let's just compile the latest v4.6 it from source, on Ubuntu 20.04:
sudo apt build-dep freefem
git clone https://github.com/FreeFem/FreeFem-sources
cd FreeFem-sources
# Post v4.6 with some fixes.
git checkout 3df0e2370d9752801ac744b11307b14e16743a44
# Won't apply automatically due to tab hell.
# https://superuser.com/questions/607410/how-to-copy-paste-tab-characters-via-the-clipboard-into-terminal-session-on-gnom
git apply <<'EOS'
diff --git a/3rdparty/ff-petsc/Makefile b/3rdparty/ff-petsc/Makefile
index dc62ab06..13cd3253 100644
--- a/3rdparty/ff-petsc/Makefile
+++ b/3rdparty/ff-petsc/Makefile
@@ -204,7 +204,7 @@ $(SRCDIR)/tag-make-real:$(SRCDIR)/tag-conf-real
$(SRCDIR)/tag-install-real :$(SRCDIR)/tag-make-real
cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr install
-test -x "`type -p otool`" && make changer
- cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
+ #cd $(SRCDIR) && $(MAKE) PETSC_DIR=$(PETSC_DIR) PETSC_ARCH=fr check
test -e $(DIR_INSTALL_REAL)/include/petsc.h
test -e $(DIR_INSTALL_REAL)/lib/petsc/conf/petscvariables
touch $@
@@ -293,7 +293,6 @@ $(SRCDIR)/tag-tar:$(PACKAGE)
-tar xzf $(PACKAGE)
patch -p1 < petsc-hpddm.patch
ifeq ($(WIN32DLLTARGET),)
- patch -p1 < petsc-metis.patch
endif
touch $@
$(PACKAGE):
EOS
autoreconf -i
./configure --enable-download --enable-optim --prefix="$(pwd)/../FreeFem-install"
./3rdparty/getall -a
cd 3rdparty/ff-petsc
make petsc-slepc
cd -
./reconfigure
make -j`nproc`
make install
cd ../FreeFem-install
PATH="${PATH}:$(pwd)/bin" ./bin/FreeFem++ ../FreeFem-sources/examples/tutorial/Ciro's initial build experience was a bit painful, possibly because it was done on a relatively new Ubuntu 20.04 as of June 2020, but in the end it worked: github.com/FreeFem/FreeFem-sources/issues/141
The main/only dependency appears to be PETSc which is used by default, which is a good sign, as that library appears to automatically parallelize a single input to several backends (single CPU, MPI, GPU) so you know things will scale up as you reach simulations.
The problem is that it compiling such a complex dependency opens up much more room for hard to solve compilation errors, and takes a lot more time.
1-dimensional heat equation example with Dirichlet boundary condition
2-dimensional heat equation example with Dirichlet boundary condition:
One big advantage over FreeFem is that it uses plain old Python to describe the problems instead of a domain-specific language. Matplotlib is used for plotting by default, so we get full Python power out of the box!
Also uses variational formulation of a partial differential equation like FreeFem which is a pain.
One downside is that its documentation is a Springer published PDF link.springer.com/content/pdf/10.1007%2F978-3-319-52462-7.pdf which is several years out-of-date (tested with FEnics 2016.2. Newbs. This causes problems e.g.: stackoverflow.com/questions/53730427/fenics-did-not-show-figure-nameerror-name-interactive-is-not-defined/57390687#57390687
system of partial differential equations are mentioned at: link.springer.com/content/pdf/10.1007%2F978-3-319-52462-7.pdf 3.5 "A system of advection–diffusion–reaction equations". You don't need to manually iterate between the equations.
On Ubuntu 20.04 as per fenicsproject.org/download/Before 2020-06, it was failing with:but they seem to have created the Ubuntu 20.04 package as of 2020-06, so it now worked! askubuntu.com/questions/866901/what-can-i-do-if-a-repository-ppa-does-not-have-a-release-file
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:fenics-packages/fenics
sudo apt-get update
sudo apt-get install --no-install-recommends fenics
sudo apt install fenics
python3 -m pip install -u matplotlibE: The repository 'http://ppa.launchpad.net/fenics-packages/fenics/ubuntu focal Release' does not have a Release file.TODO heat equation hello world.
GitHub account: github.com/hplgit
It should be mentioned that when you start Googling for PDE stuff, you will reach Han's writings a lot under his GitHub Pages: hplgit.github.io/, and he is one of the main authors of the FEniCS Project.
Unfortunately he died of cancer in 2016, shame, he seemed like a good educator.
He also published to GitHub pages with his own crazy markdown-like multi-output markup language: github.com/hplgit/doconce.
Rest in peace, Hans.
In many important applications, what you have to solve is not just a single partial differential equation, but multiple partial differential equations coupled to each other. This is the case for many key PDEs including:
One major application of this classification is that different boundary conditions are suitable for different types of partial differential equations as explained at: which boundary conditions lead to existence and uniqueness of a second order PDE.
Bibliography:
www.cns.gatech.edu/~predrag/courses/PHYS-6124-12/StGoChap6.pdf 6.1 "Classification of PDE's" clarifies which boundary conditions are needed for existence and uniqueness of each type of second order of PDE:
This idea comes up particularly in the phase space coordinate of Hamiltonian mechanics.
Basically a subset of the boundary condition for when one of the parameters is time and we are specifying values for the time 0.
Specifies fixed values.
Can be used for elliptic partial differential equations and parabolic partial differential equations.
Numerical examples:
Specifies the derivative in a direction normal to the boundary.
Can be used for elliptic partial differential equations and parabolic partial differential equations.
Sets both a Dirichlet boundary condition and a Neumann boundary condition for a single part of the boundary.
Can be used for hyperbolic partial differential equations.
We understand intuitively that this imposes stricter requirements on solutions, which makes it easier to guarantee uniqueness, but also harder to have existence. TODO intuitively why hyperbolic need this extra level of restriction.
Linear combination of a Dirichlet boundary condition and Neumann boundary condition at each point of the boundary.
Examples:
- heat equation when metal plaque is immersed in a large external environment of fixed temperature.In this case, the normal derivative at the boundary is proportional to the difference between the temperature of the boundary and the fixed temperature of the external environment.The result as time tends to infinity is that the temperature of the plaque tends to that of the environment.Shown a solved example in the FreeFem tutorial: doc.freefem.org/tutorials/thermalConduction.html (github.com/FreeFem/FreeFem-doc/blob/1d5996d8b891fd553fd318321249c2c30f693fc3/source/tutorials/thermalConduction.rst)
In the context of wave-like equations, an open-boundary condition is one that "lets the wave go through without reflection".
This condition is very useful when we want to simulate infinite domains with a numerical method. Ciro Santilli wants to do this all the time when trying to come up with demos for his physics writings.
Here are some resources that cover such boundary conditions:
- www.asc.tuwien.ac.at/~arnold/pdf/graz/graz.pdf lots of slides
- hplgit.github.io/wavebc/doc/pub/._wavebc_cyborg002.html mentions them and gives a 1D formula. It mentions that things get complicated in 2D and 3D TODO why.The other page: hplgit.github.io/wavebc/doc/pub/._wavebc_cyborg003.html shows solution demos.
Multiple boundary conditions for different parts of the boundary.
Most commonly, boundary conditions such as the Dirichlet boundary condition are taken to be fixed values in time.
But it also makes sense to think about cases where those values vary in time.
Some bibliography:
This basically adds one more ingredient to partial differential equations: a function that we can select.
And then the question becomes: if this function has such and such limitation, can we make the solution of the differential equation have such and such property?
It's quite fun from a mathematics point of view!
Control theory also takes into consideration possible discretization of the domain, which allows using numerical methods to solve partial differential equations, as well as digital, rather than analogue control methods.
This is particularly relevant in electronics, where the oscilloscope's time origin is set to match the wave.
Points in the direction in which a wind spinner spins fastest.
As if Greek letters weren't enough, physicists and mathematicians also like to make up tons of symbols, some of which look like the could actually be Greek letters!
Nabla is one of those: it was completely made up in modern times, and just happens to look like an inverted upper case delta to make things even more confusing!
Nabla means "harp" in Greek, which looks like the symbol.
Oh, and if it weren't enough, mathematicians have a separate name for the damned nabla symbol : "del" instead of "nabla".
TODO why is it called "Del"? Is is because it is an inverted uppercase delta?
Takes a vector field as input and produces a scalar field.
Mnemonic: it gives out the amount of fluid that is going in or out of a given volume per unit of time.
Therefore, if you take a cubic volume:
- the input has to be the 6 flows across each face, therefore 3 derivatives
- the output is the variation of the quantity of fluid, and therefore a scalar
Takes a scalar field as input and produces a vector field.
Mnemonic: the gradient shows the direction in which the function increases fastest.
Think of a color gradient going from white to black from left to right.
Therefore, it has to:
- take a scalar field as input. Otherwise, how do you decide which vector is larger than the other?
- output a vector field that contains the direction in which the scalar increases fastest locally at each point. This has to give out vectors, since we are talking about directions
Can be denoted either by:Our default symbol is going to be:
- the upper case Greek letter delta
- nabla symbol squared
The laplace operator for Minkowski space.
Can be nicely written with Einstein notation as shown at: Section "D'alembert operator in Einstein notation ".