linear-algebra.bigb

= Linear algebra
{wiki}

= Linear function
{parent=Linear algebra}
{wiki}

= Linear
{synonym}

= Linearly
{synonym}

The term is not very clear, as it could either mean:
* a <real number> function whose graph is a line, i.e.:
  $$
  f(x) = ax + b
  $$
  or for higher dimensions, a <hyperplane>:
  $$
  f(x_1, x_2, \ldots, x_n) = c_1 x_1 + c_2 x_2 + \ldots + c_n x_n + b
  $$
* a <linear map>. Note that the above linear functions are not linear maps unless $b = 0$ (known as the homogeneous case), because e.g.:
  $$
  f(x + y) = ax + ay + b
  $$
  but
  $$
  f(x) + f(y) = ax + b + ay + b
  $$
  For this reason, it is better never to refer to linear maps as linear functions.

= Linear map
{parent=Linear algebra}
{title2=linear operator}
{wiki}

A linear map is a function $f : V_1(F) \to V_2(F)$ where $V_1(F)$ and $V_2(F)$ are two vector spaces over <underlying field of a vector space>[underlying fields] $F$ such that:
$$
\forall v_{1}, v_{2} \in V_1, c_{1}, c_{2} \in F \\
f(c_{1} v_{1} + c_{2} v_{2}) = c_{1} f(v_{1}) + c_{2} f(v_{2})
$$

A common case is $F = \R$, $V_1 = \R_m$ and $V_2 = \R_n$.

One thing that makes such functions particularly simple is that they can be fully specified by specifyin how they act on all possible combinations of input basis vectors: they are therefore specified by only a finite number of elements of $F$.

Every linear map in <finite dimension> can be represented by a <matrix>, the points of the <domain (function)> being represented as <vectors>.

As such, when we say "linear map", we can think of a generalization of <matrix multiplication> that makes sense in <infinite dimensional> spaces like <Hilbert spaces>, since calling such infinite dimensional maps "matrices" is stretching it a bit, since we would need to specify infinitely many rows and columns.

The prototypical building block of <infinite dimensional> linear map is the <derivative>. In that case, the vectors being operated upon are <functions>, which cannot therefore be specified by a finite number of parameters, e.g. 

For example, the left side of the <time-independent Schrödinger equation> is a linear map. And the <time-independent Schrödinger equation> can be seen as a <eigenvalue> problem.

= Form
{disambiguate=mathematics}
{parent=Linear map}

A form is a <function> from a <vector space> to elements of the <underlying field of the vector space>.

Examples:
* <linear form>
* <bilinear form>
* <multilinear form>

= Linear form
{parent=Linear map}
{wiki}

A <Linear map> where the <image (mathematics)> is the <underlying field of the vector space>, e.g. $\R^n \to \R$.

The set of all <linear forms> over a <vector space> forms another vector space called the <dual space>.

= Matrix representation of a linear form
{parent=Linear form}

For the typical case of a linear form over <\R^n>, the form can be seen just as a row vector with n elements, the full form being specified by the value of each of the <basis vectors>.

= Dual space
{parent=Linear form}
{title2=$V^*$}
{wiki}

The dual space of a <vector space> $V$, sometimes denoted $V^*$, is the vector space of all <linear forms> over $V$ with the obvious addition and scalar multiplication operations defined.

Since a linear form is completely determined by how it acts on a \x[basis]{magic}, and since for each basis element it is specified by a scalar, at least in finite dimension, the dimension of the dual space is the same as the $V$, and so they are isomorphic because <all vector spaces of the same dimension on a given field are isomorphic>, and so the dual is quite a boring concept in the context of finite dimension.

Infinite dimension seems more interesting however, see: https://en.wikipedia.org/w/index.php?title=Dual_space&oldid=1046421278#Infinite-dimensional_case

One place where duals are different from the non-duals however is when dealing with <tensors>, because they transform differently than vectors from the base space $V$.

= Dual vector
{parent=Dual space}
{title2=$e^i$}

Dual vectors are the members of a <dual space>.

In the context of <tensors> , we use raised indices to refer to members of the dual basis vs the underlying basis:
$$
\begin{aligned}
e_1 & \in V \\
e_2 & \in V \\
e_3 & \in V \\
e^1 & \in V^* \\
e^2 & \in V^* \\
e^3 & \in V^* \\
\end{aligned}
$$
The dual basis vectors $e^i$ are defined to "pick the corresponding coordinate" out of elements of V. E.g.:
$$
\begin{aligned}
e^1 (4, -3, 6) & =  4 \\
e^2 (4, -3, 6) & = -3 \\
e^3 (4, -3, 6) & =  6 \\
\end{aligned}
$$
By expanding into the basis, we can put this more succinctly with the <Kronecker delta> as:
$$
e^i(e_j) = \delta_{ij}
$$

Note that in <Einstein notation>, the components of a dual vector have lower indices. This works well with the upper case indices of the dual vectors, allowing us to write a dual vector $f$ as:
$$
f = f_i e^i
$$

In the context of <quantum mechanics>, the <bra-ket>[bra] notation is also used for dual vectors.

= Linear operator
{parent=Linear map}
{wiki}

= Operator
{synonym}

We define it as a <linear map> where the <domain (function)> is the same as the <image (mathematics)>, i.e. an <endofunction>.

Examples:
* a 2x2 matrix can represent a <linear map> from <\R^2> to <\R^2>, so which is a linear operator
* the <derivative> is a <linear map> from <C^{\infty}> to <C^{\infty}>, so which is also a linear operator

= Adjoint operator
{parent=Linear operator}
{title2=$A^\dagger$}

Given a <linear operator> $A$ over a space $S$ that has a <inner product> defined, we define the adjoint operator $A^\dagger$ (the $\dagger$ symbol is called "dagger") as the unique operator that satisfies:
$$
\forall v, w \in S, <Av, w> = <v, A^{\dagger} w>
$$

= Self-adjoint operator
{parent=Linear map}
{wiki}

= Self-adjoint
{synonym}

= Multilinear map
{parent=Linear map}
{wiki}

= Bilinear map
{parent=Multilinear map}
{wiki}

= Bilinear product
{synonym}

<Linear map> of two variables.

More formally, given 3 <vector spaces> X, Y, Z over a single <field (mathematics)>, a bilinear map is a function from:
$$
f : X \times Y \to Z
$$
that is linear on the first two arguments from X and Y, i.e.:
$$
f(a_1\vec{x_1} + a_2\vec{x_2}, \vec{y}) = a_1f(\vec{x_1}, \vec{y}) + a_2f(\vec{x_2}, \vec{y})
$$
Note that the definition only makes sense if all three vector spaces are over the same field, because linearity can mix up each of them.

The most important example by far is the <dot product> from $\R^n \times \R^n \to \R$, which is more specifically also a <symmetric bilinear form>.

= Bilinear form
{parent=Multilinear map}
{title2=$B(x, y)$}
{wiki}

Analogous to a <linear form>, a bilinear form is a <Bilinear map> where the <image (mathematics)> is the <underlying field of the vector space>, e.g. $\R^n \times \R^m \to \R$.

Some definitions require both of the input spaces to be the same, e.g. $\R^n \times \R^n \to \R$, but it doesn't make much different in general.

The most important example of a bilinear form is the <dot product>. It is only defined if both the input spaces are the same.

= Matrix representation of a bilinear form
{parent=Bilinear form}

As usual, it is useful to think about how a <bilinear form> looks like in terms of <vectors> and <matrices>.

Unlike a <linear form>, which <matrix representation of a linear form>[was a vector], because it has two inputs, the bilinear form is represented by a matrix $M$ which encodes the value for each possible pair of <basis vectors>.

In terms of that <matrix>, the form $B(x,y)$ is then given by:
$$
B(x,y) = x^T M y
$$

= Effect of a change of basis on the matrix of a bilinear form
{parent=Matrix representation of a bilinear form}
{title2=$B_2 = C^T B C$}

If $C$ is the <change of basis matrix>, then the <matrix representation of a bilinear form> $M$ that looked like:
$$
B(x,y) = x^T M y
$$
then the matrix in the new basis is:
$$
C^T M C
$$
<Sylvester's law of inertia> then tells us that the number of positive, negative and 0 eigenvalues of both of those matrices is the same.

Proof: the value of a given bilinear form cannot change due to a \x[change of basis]{magic}, since the bilinear form is just a <function (mathematics)>, and does not depend on the choice of basis. The only thing that change is the matrix representation of the form. Therefore, we must have:
$$
x^T M y = x_{new}^T M_{new} y_{new}
$$
and in the new basis:
$$
x = C x_{new} \\
y = C y_{new} \\
x_{new}^T M_{new} y_{new} = x^T M y =  (Cx_{new})^T M (Cy_{new}) = x_{new}^T (C^T M C) y_{new} \\
$$
and so since:
$$
\forall x_{new}, y_{new} x_{new}^T M_{new} y_{new} = x_{new}^T (C^T M C) y_{new} \implies M_{new} = C^T M C \\
$$

Related:
* https://proofwiki.org/wiki/Matrix_of_Bilinear_Form_Under_Change_of_Basis

= Multilinear form
{parent=Multilinear map}
{wiki}

See <form (mathematics)>.

Analogous to a <linear form>, a multilinear form is a <Multilinear map> where the <image (mathematics)> is the <underlying field of the vector space>, e.g. $\R^{n_1} \times \R^{n_2} \times \R^{n_2} \to \R$.

= Symmetric bilinear map
{parent=Multilinear map}
{wiki}

Subcase of <symmetric multilinear map>:
$$
f(x, y) = f(y, x)
$$

Requires the two inputs $x$ and $y$ to be in the same <vector space> of course.

The most important example is the <dot product>, which is also a <positive definite symmetric bilinear form>.

= Symmetric bilinear form
{parent=Symmetric bilinear map}
{wiki}

<symmetric bilinear maps> that is also a <bilinear form>.

= Matrix representation of a symmetric bilinear form
{parent=Symmetric bilinear form}

= Matrix representation of the symmetric bilinear form
{synonym}

Like the <matrix representation of a bilinear form>, it is a <matrix>, but now the matrix has to be a <symmetric matrix>.

We can then immediately see that the matrix is symmetric, then so is the form. We have:
$$
B(x,y) = x^T M y
$$
But because $B(x,y)$ is a <scalar>, we have:
$$
B(x,y) = B(x,y)^T
$$
and:
$$
B(x,y) = B(x,y)^T = (x^T M y)^T = y^T M^T x = y^T M^T x = y^T M x = B(y,x)
$$

= Hermitian form
{c}
{parent=Symmetric bilinear map}
{wiki}

The <complex number> analogue of a <symmetric bilinear form>.

The prototypical example of it is the <complex dot product>.

Note that this form is neither strictly <symmetric bilinear map>[symmetric], it satisfies:
$$
<x, y> = \overline{<y, x>}
$$
where the over bar indicates the <complex conjugate>, nor is it linear for complex scalar multiplication on the second argument.

Bibliography:
* https://mathworld.wolfram.com/HermitianForm.html

= Matrix representation of a Hermitian form
{parent=Hermitian form}

;

A <Hermitian matrix>.

= Quadratic form
{parent=Symmetric bilinear map}
{wiki}

<Multivariate polynomial> where each term has degree 2, e.g.:
$$
f(x,y) = 2y^2 + 10yx + x^2
$$
is a quadratic form because each term has degree 2:
* $y^2$
* $xy$
* $x^2$
but e.g.:
$$
f(x,y) = 2y^2 + 10yx + x^3
$$
is not because the term $x^3$ has degree 3.

More generally for any number of variables it can be written as:
$$
f(x_1, x_2, \ldots, x_n) = \sum_{i,j} a_i a_j x_i x_j
$$

There is a <1-to-1> relationship between <quadratic forms> and <symmetric bilinear forms>. In matrix representation, this can be written as:
$$
\vec{x}^T B \vec{x}
$$
where $\vec{x}$ contains each of the variabes of the form, e.g. for 2 variables:
$$
\vec{x} = [x, y]
$$

Strictly speaking, the associated <bilinear form> would not need to be a <symmetric bilinear form>, at least for the <real numbers> or <complex numbers> which are <commutative>. E.g.:
$$
\begin{bmatrix}x y\end{bmatrix}
\begin{bmatrix}0 & 1 \\ 2 & 0 \\ \end{bmatrix}
\begin{bmatrix}x \\ y \\ \end{bmatrix}
=
\begin{bmatrix}x y\end{bmatrix}
\begin{bmatrix}y \\ 2x \\\end{bmatrix}
= xy + 2yx
= 3xy
$$
But that same matrix could also be written in symmetric form as:
$$
\begin{bmatrix}0 & 1.5 \\ 1.5 & 0 \\ \end{bmatrix}
$$
so why not I guess, its simpler/more restricted.

= Positive definite symmetric bilinear form
{parent=Symmetric bilinear map}
{wiki}

<Symmetric bilinear form> that is also <positive definite>, i.e.:
$$
\forall x, B(x, x) > 0
$$

= Matrix representation of a positive definite symmetric bilinear form
{parent=Positive definite symmetric bilinear form}

A <positive definite matrix> that is also a <symmetric matrix>.

= Skew-symmetric bilinear map
{parent=Symmetric bilinear map}

= Antisymmetric bilinear map
{synonym}

Subcase of <antisymmetric multilinear map>:
$$
f(x, y) = -f(y, x)
$$

= Skew-symmetric bilinear form
{parent=Symmetric bilinear map}
{wiki}

<Skew-symmetric bilinear map> that is also a <bilinear form>.

= Symmetric multilinear map
{parent=Multilinear map}

Same value if you swap any input arguments.

= Antisymmetric multilinear map
{parent=Symmetric multilinear map}

Change sign if you swap two input values.

= Alternating multilinear map
{parent=Multilinear map}

Implies <antisymmetric multilinear map>.

= Dot product
{parent=Linear algebra}
{wiki}

The definition of the "dot product" of a general space varies quite a lot with different contexts.

Most definitions tend to be <bilinear forms>.

We use the unqualified generally refers to the dot product of <Real coordinate spaces>, which is a <positive definite symmetric bilinear form>. Other important examples include:
* the <complex dot product>, which is not strictly <symmetric bilinear map>[symmetric] nor <linear>, but it is <positive definite>
* <Minkowski inner product>, sometimes called" "Minkowski dot product is not <positive definite>
The rest of this section is about the <\R^n> case.

The <positive definite> part of the definition likely comes in because we are so familiar with <metric spaces>, which requires a positive <norm> in the <norm induced by an inner product>.

The default <Euclidean space> definition, we use the <matrix representation of a symmetric bilinear form> as the identity matrix, e.g. in <\R^3>:
$$
M =
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 \\
\end{bmatrix}
$$
so that:
$$
\vec{x} \cdot \vec{y}
=
\begin{bmatrix}
x_1 & x_2 & x_3 \\
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix}
y_1 \\
y_2 \\
y_3 \\
\end{bmatrix}
=
x_1y_1 + x_2y_2 + x_3y_3
$$

= Orthogonality
{parent=Dot product}
{wiki}

= Orthogonal
{synonym}

= Orthonormality
{parent=Orthogonality}
{wiki}

= Orthonormal
{synonym}

= Angle
{parent=Dot product}
{wiki}

= Cross product
{parent=Linear algebra}
{title2=$\va{x} \times \va{y}$}
{wiki}

= Jacobi identity
{c}
{parent=Cross product}
{wiki}

= Index picking function
{parent=Linear algebra}

= Kronecker delta
{c}
{parent=Index picking function}
{title2=$\delta_{ij}$}
{wiki}

= Levi-Civita symbol
{c}
{parent=Index picking function}
{title2=$\varepsilon$}
{wiki}

Denoted by the <Greek letter epsilon> with `\varepsilon` encoding in <LaTeX>.

Definition:
* <odd permutation>: -1
* <even permutation>: 1
* not a <permutation>: 0. This happens iff two more more indices are repeated

= Levi-Civita symbol as a tensor
{parent=Levi-Civita symbol}

<An Introduction to Tensors and Group Theory for Physicists by Nadir Jeevanjee (2011)> shows that this is a <tensor> that represents the <volume of a parallelepiped>.

It takes as input three vectors, and outputs one real number, the volume. And it is linear on each vector. This perfectly satisfied the definition of a tensor of <order of a tensor>[order] (3,0).

Given a basis $(e_i, e_j, e_k)$ and a function that return the volume of a parallelepiped given by three vectors $V(v_1, v_2, v_3)$, $\varepsilon_{ikj} = V(e_i, e_j, e_k)$.

= Projection
{disambiguate=mathematics}
{parent=Linear algebra}

= Matrix
{parent=Linear algebra}
{wiki}

= Matrix operation
{parent=Matrix}
{wiki}

= Determinant
{parent=Matrix operation}
{title2=$det$}
{wiki}

Name origin: likely because it "determines" if a matrix is <invertible matrix>[invertible] or not, as a matrix is invertible iff determinant is not zero.

= Matrix inverse
{parent=Matrix operation}
{title2=$M^{-1}$}

When it exists, which is not for all matrices, only <invertible matrix>, the inverse is denoted:
$$
M^{-1}
$$

= Invertible matrix
{parent=Matrix inverse}
{tag=Named matrix}
{wiki}

The set of all <invertible matrices> forms a <group>: the <general linear group> with <matrix multiplication>. Non-invertible matrices don't form a group due to the lack of inverse.

= Transpose
{parent=Matrix operation}
{title2=$M^T$}
{wiki}

= Transpose of a matrix multiplication
{parent=Transpose}
{wiki}

When it distributes it inverts the order of the <matrix multiplication>:
$$
(MN)^T = N^T M^T
$$

= Inverse of the transpose
{parent=Transpose}
{wiki}

The <transpose> and <matrix inverse> commute:
$$
(M^T)-1 = (M^{-1})^T
$$

= Matrix multiplication
{parent=Matrix}
{wiki}

= Matrix product
{synonym}

Since a <matrix> $M$ can be seen as a <linear map> $f_M(\vec{x})$, the product of two matrices $MN$ can be seen as the composition of two <linear maps>:
$$
f_M(f_N(\vec{x}))
$$
One cool thing about linear functions is that we can easily pre-calculate this product only once to obtain a new matrix, and so we don't have to do both multiplications separately each time.

= System of linear equations
{parent=Matrix multiplication}
{wiki}

= Application of systems of linear equations
{parent=System of linear equations}
{wiki}

No 2x2 examples please. I'm talking about large matrices that would be used in <supercomputers>.

= System of linear equations algorithm
{parent=System of linear equations}

= LINPACK benchmarks
{c}
{parent=System of linear equations algorithm}
{tag=Benchmark}
{wiki}

= Conjugate gradient method
{parent=System of linear equations algorithm}
{wiki}

For <positive definite matrices> only.

TODO application.

TODO speedup over algorithm for general matrices.

https://www.studentclustercompetition.us/ comments:
\Q[The HPCG benchmark uses a preconditioned conjugate gradient (PCG) algorithm to measure the performance of HPC platforms with respect to frequently observed but challenging patterns of computing, communication, and memory access. While HPL provides an optimistic performance target for applications, HPCG can be considered as a lower bound on performance. Many of the top 500 supercomputers also provide their HPCG performance as a reference.]

= Application of matrix multiplication
{parent=Matrix multiplication}

https://math.stackexchange.com/questions/41706/practical-uses-of-matrix-multiplication/4647422#4647422[] highlights <deep learning> applications.

= Matrix multiplication algorithm
{parent=Matrix multiplication}
{tag=Computational problem}

https://math.stackexchange.com/questions/30330/fast-algorithm-for-solving-system-of-linear-equations/259372#259372

= General Matrix Matrix Multipliation
{parent=Matrix multiplication algorithm}
{wiki}

= GEMM
{c}
{synonym}
{title2}

The terminology <GEMM> is present on <BLAS>, and has stuck pretty much.

= Matrix decomposition
{parent=Matrix multiplication}
{wiki}

= Eigenvalues and eigenvectors
{parent=Matrix}
{wiki}

= Applications of eigenvalues and eigenvectors
{parent=Eigenvalues and eigenvectors}

* https://math.stackexchange.com/questions/23312/what-is-the-importance-of-eigenvalues-eigenvectors/3503875#3503875
* https://math.stackexchange.com/questions/1520832/real-life-examples-for-eigenvalues-eigenvectors
* https://matheducators.stackexchange.com/questions/520/what-is-a-good-motivation-showcase-for-a-student-for-the-study-of-eigenvalues

= Characteristic polynomial
{parent=Eigenvalues and eigenvectors}
{wiki}

= Eigenvalue
{parent=Eigenvalues and eigenvectors}

See: <eigenvalues and eigenvectors>.

= Spectrum
{disambiguate=functional analysis}
{parent=Eigenvalue}

Set of <eigenvalues> of a <linear operator>.

= Continuous spectrum
{disambiguate=functional analysis}
{parent=Spectrum (functional analysis)}

= Continuous spectrum
{synonym}

Unlike the simple case of a <matrix>, in <infinite dimensional> vector spaces, the spectrum may be continuous.

The quintessential example of that is the spectrum of the <position operator> in <quantum mechanics>, in which any <real number> is a possible <eigenvalue>, since the particle may be found in any position. The associated <eigenvectors> are the corresponding <Dirac delta functions>.

= Eigendecomposition of a matrix
{parent=Eigenvalues and eigenvectors}
{wiki}

= Eigendecomposition
{synonym}

Every <invertible matrix> $M$ can be written as:
$$
M = QDQ^{-1}
$$
where:
* $D$ is a <diagonal matrix> containing the <eigenvalues> of $M$
* columns of $Q$ are <eigenvectors> of $M$
Note therefore that this decomposition is unique up to swapping the order of eigenvectors. We could fix a canonical form by sorting eigenvectors from smallest to largest in the case of a <real number>.

Intuitively, Note that this is just the \x[change of basis]{magic} formula, and so:
* $Q^{-1}$ changes basis to align to the eigenvectors
* $D$ multiplies eigenvectors simply by eigenvalues
* $Q$ changes back to the original basis

= Eigendecomposition of a real symmetric matrix
{parent=Eigendecomposition of a matrix}

= The eigendecomposition of a real symmetric matrix is done with orthogonal matrices
{synonym}

The general result from <eigendecomposition of a matrix>:
$$
M = QDQ^{-1}
$$
becomes:
$$
M = ODO^T
$$
where $O$ is an <orthogonal matrix>, and therefore has $O^{-1} = O^T$.

= Sylvester's law of inertia
{c}
{parent=Eigendecomposition of a matrix}
{wiki}

The main interest of this theorem is in <classifying (mathematics)> the <indefinite orthogonal groups>, which in turn is fundamental because the <Lorentz group> is an <indefinite orthogonal groups>, see: <all indefinite orthogonal groups of matrices of equal metric signature are isomorphic>.

It also tells us that a \x[change of basis]{magic} does not the alter the <metric signature> of a <bilinear form>, see <matrix congruence can be seen as the change of basis of a bilinear form>.

The theorem states that the number of 0, 1 and -1 in the <metric signature> is the same for two <symmetric matrices> that are <congruent matrices>.

For example, consider:
$$
A = \begin{bmatrix}2 & \sqrt{2} \\ \sqrt{2} & 3 \\\end{bmatrix}
$$

The <eigenvalues> of $A$ are $1$ and $4$, and the associated eigenvectors are:
$$
v_1 = [-\sqrt{2}, 1]^T
v_4 = [\sqrt{2}/2, 1]^T
$$
<symPy> code:
``
A = Matrix([[2, sqrt(2)], [sqrt(2), 3]])
A.eigenvects()
``
and from the <eigendecomposition of a real symmetric matrix> we know that:
$$
A = PDP^T =
\begin{bmatrix}-\sqrt{2} & \sqrt{2}/2 \\ 1 & 1\\\end{bmatrix}
\begin{bmatrix}1 & 0 \\ 0 & 4\\\end{bmatrix}
\begin{bmatrix}-\sqrt{2} & 1 \\ \sqrt{2}/2 & 1\\\end{bmatrix}
$$

Now, instead of $P$, we could use $PE$, where $E$ is an arbitrary <diagonal matrix> of type:
$$
\begin{bmatrix}e_1 & 0 \\ 0 & e_2\\\end{bmatrix}
$$
With this, would reach a new matrix $B$:
$$
B = (PE)D(PE)^T = P(EDE^T)P^T = P(EED)P^T
$$
Therefore, with this congruence, we are able to multiply the eigenvalues of $A$ by any positive number $e_1^2$ and $e_2^2$. Since we are multiplying by two arbitrary positive numbers, we cannot change the signs of the original eigenvalues, and so the <metric signature> is maintained, but respecting that any value can be reached.

Note that the <matrix congruence> relation looks a bit like the <eigendecomposition of a matrix>:
$$
D = SMS^T
$$
but note that $D$ does not have to contain <eigenvalues>, unlike the <eigendecomposition of a matrix>. This is because here $S$ is not fixed to having <eigenvectors> in its columns.

But because the matrix is symmetric however, we could always choose $S$ to actually diagonalize as mentioned at <eigendecomposition of a real symmetric matrix>. Therefore, the <metric signature> can be seen directly from <eigenvalues>.

Also, because $D$ is a <diagonal matrix>, and thus symmetric, it must be that:
$$
S^T = S^{-1}
$$

What this does represent, is a general \x[change of basis]{magic} that maintains the matrix a <symmetric matrix>.

Related:
* https://math.stackexchange.com/questions/1817906/sylvesters-law-of-inertia
* https://math.stackexchange.com/questions/1284601/what-is-the-lie-group-that-leaves-this-matrix-invariant
* https://physics.stackexchange.com/questions/24495/metric-signature-explanation

= Congruent matrix
{parent=Sylvester's law of inertia}

= Matrix congruence
{synonym}

Two <symmetric matrices> $A$ and $B$ are defined to be congruent if there exists an $S$ in <GL(n)> such that:
$$
A = S B S^T
$$

= Matrix congruence can be seen as the change of basis of a bilinear form
{parent=Congruent matrix}

From <effect of a change of basis on the matrix of a bilinear form>, remember that a change of basis $C$ modifies the <matrix representation of a bilinear form> as:
$$
C^T M C
$$

So, by taking $S = C^T$, we understand that two matrices being congruent means that they can both correspond to the same <bilinear form> in different bases.

= Matrix similarity
{parent=Sylvester's law of inertia}
{wiki}

= Similar matrix
{synonym}

= Metric signature
{parent=Sylvester's law of inertia}
{wiki}

= Metric signature matrix
{parent=Metric signature}

= Eigenvector
{parent=Eigenvalues and eigenvectors}

See: <eigenvalues and eigenvectors>.

= Eigenvectors and eigenvalues of the identity matrix
{parent=Eigenvalues and eigenvectors}

https://math.stackexchange.com/questions/1507290/linear-algebra-identity-matrix-and-its-relation-to-eigenvalues-and-eigenvectors/3934023#3934023

= Spectral theorem
{parent=Eigenvalues and eigenvectors}
{wiki}

* https://math.stackexchange.com/questions/1557878/do-infinite-dimensional-hermitian-operators-admit-a-complete-basis-of-eigenvecto
* https://mathoverflow.net/questions/45426/diagonalization-of-infinite-hermitian-matrices

= Hermitian matrix
{c}
{parent=Spectral theorem}
{title2=complex analogue of symmetric matrix}
{wiki}

= Hermitian operator
{c}
{parent=Hermitian matrix}

= Hermitian
{c}
{synonym}

This is the possibly infinite dimensional version of a <Hermitian matrix>, since <linear operators> are the possibly infinite dimensional version of <matrices>.

There's a catch though: now we don't have explicit matrix indices here however in general, the generalized definition is shown at: https://en.wikipedia.org/w/index.php?title=Hermitian_adjoint&oldid=1032475701#Definition_for_bounded_operators_between_Hilbert_spaces

= Riesz representation theorem
{c}
{parent=Hermitian operator}

= Kronecker product
{c}
{parent=Matrix}
{wiki}

= Named matrix
{parent=Matrix}
{wiki=List_of_named_matrices}

= Dense and sparse matrices
{parent=Named matrix}

A good definition is that the <sparse matrix> has non-zero entries proportional the number of rows. Therefore this is <Big O notation> less than something that has $N^2$ non zero entries. Of course, this only makes sense when generalizing to larger and larger matrices, otherwise we could take the constant of proportionality very high for one specific matrix.

Of course, this only makes sense when generalizing to larger and larger matrices, otherwise we could take the constant of proportionality very high for one specific matrix.

= Dense matrix
{parent=Dense and sparse matrices}

= Sparse matrix
{parent=Dense and sparse matrices}
{wiki}

= Diagonal matrix
{parent=Named matrix}
{wiki}

Forms a <normal subgroup> of the <general linear group>.

= Scalar matrix
{parent=Diagonal matrix}
{title2=$Z(V)$}
{{wiki=Diagonal_matrix#Scalar_matrix}}

Forms a <normal subgroup> of the <general linear group>.

= Identity matrix
{parent=Scalar matrix}
{title2=$I_n$}
{wiki}

= Square matrix
{parent=Named matrix}
{wiki}

= Matrix ring
{parent=Square matrix}
{wiki}

= Matrix ring of degree n
{synonym}
{title2}

= $M_n$
{synonym}
{title2}

= Set of all n-by-y square matrices
{synonym}
{title2}

The matrix ring of degree n $M_n$ is the set of all n-by-n square matrices together with the usual <vector space> and <matrix multiplication> operations.

This set forms a <ring (mathematics)>.

Related terminology:
* https://math.stackexchange.com/questions/412200/what-is-the-notation-for-the-set-of-all-m-times-n-matrices

= Orthogonal matrix
{parent=Named matrix}
{wiki}

Members of the <orthogonal group>.

= Unitary matrix
{parent=Orthogonal matrix}
{wiki}

= Unitary operation
{synonym}

<complex number>[Complex] analogue of <orthogonal matrix>.

Applications:
* in <quantum computers> programming basically comes down to creating one big unitary matrix as explained at: <quantum computing is just matrix multiplication>

= Triangular matrix
{parent=Named matrix}
{wiki}

= Symmetric matrix
{parent=Named matrix}
{wiki}

A <matrix> that equals its <transpose>:
$$
M = M^T
$$

Can represent a <symmetric bilinear form> as shown at <matrix representation of a symmetric bilinear form>, or a <quadratic form>.

= Definite matrix
{parent=Symmetric matrix}
{wiki}

The definition implies that this is also a <symmetric matrix>.

= Positive definite matrix
{parent=Definite matrix}

= Positive definite
{synonym}

The <dot product> is a <positive definite matrix>, and so we see that those will have an important link to familiar <geometry>.

= Skew-symmetric matrix
{parent=Symmetric matrix}
{wiki}

= Antisymmetric matrix
{synonym}
{title2}

WTF is a skew? "Antisymmetric" is just such a better name! And it also appears in other definitions such as <antisymmetric multilinear map>.

= Skew-symmetric form
{parent=Skew-symmetric matrix}

= Vector space
{parent=Linear algebra}
{wiki}

= Basis
{disambiguate=linear algebra}
{parent=Vector space}
{wiki}

= Basis
{synonym}

= Basis vector
{synonym}

= Change of basis
{parent=Basis (linear algebra)}
{wiki}

$$
N = BMB^{-1}
$$
where:
* $M$: matrix in the old basis
* $N$: matrix in the new basis
* $B$: change of basis matrix

= Change of basis matrix
{parent=Change of basis}

The change of basis matrix $C$ is the matrix that allows us to express the new basis in an old basis:
$$
x_{old} = Cx_{new}
$$

Mnemonic is as follows: consider we have an initial basis $(x_{old}, y_{old})$. Now, we define the new basis in terms of the old basis, e.g.:
$$
\begin{aligned}
x_{new} &= 1x_{old} + 2y_{old} \\
y_{new} &= 3x_{old} + 4y_{old} \\
\end{aligned}
$$
which can be written in matrix form as:
$$
\begin{bmatrix}x_{new} \\ y_{new} \\\end{bmatrix} =
\begin{bmatrix}1 && 2 \\ 3 && 4 \\\end{bmatrix}
\begin{bmatrix}x_{old} \\ y_{old} \\\end{bmatrix}
$$
and so if we set:
$$
M = \begin{bmatrix}1 && 2 \\ 3 && 4 \\\end{bmatrix}
$$
we have:
$$
\vec{x_{new}} = M\vec{x_{old}}
$$

The usual question then is: given a vector in the new basis, how do we represent it in the old basis?

The answer is that we simply have to calculate the <matrix inverse> of $M$:
$$
\vec{x_{old}} =  M^{-1}\vec{x_{new}}
$$

That $M^{-1}$ is the matrix inverse.

= Change of basis between symmetric matrices
{parent=Change of basis}

When we have a <symmetric matrix>, a \x[change of basis]{magic} keeps symmetry iff it is done by an <orthogonal matrix>, in which case:
$$
N = BMB^{-1} = OMO^T
$$

= Linear independence
{parent=Basis (linear algebra)}
{wiki}

= Linearly independent
{synonym}

= Classification of vector spaces
{parent=Vector space}

= All vector spaces of the same dimension on a given field are isomorphic
{synonym}

https://en.wikipedia.org/wiki/Dimension_(vector_space)#Facts

= Underlying field of a vector space
{parent=Vector space}

= Underlying field of the vector space
{synonym}

Every vector space is defined over a <field (mathematics)>.

E.g. in $\R^3$, the underlying <field (mathematics)> is $\R$, the <real numbers>. And in $\C^2$ the underlying field is $\C$, the <complex numbers>.

Any field can be used, including <finite field>. But the underlying thing has to be a field, because the definitions of a vector need all field properties to hold to make sense.

Elements of the underlying field of a vector space are known as <scalar (mathematics)>.

= Tensor product
{parent=Vector space}
{title2=$\otimes$}
{wiki}

= Vector
{disambiguate=mathematics}
{parent=Vector space}
{wiki=Vector (mathematics and physics)}

= Vector
{synonym}

= Vectorized
{synonym}

= Scalar
{disambiguate=mathematics}
{parent=Vector (mathematics)}
{wiki}

= Scalar
{synonym}

A member of the <underlying field of a vector space>. E.g. in $\R^3$, the underlying field is $\R$, and a scalar is a member of $\R$, i.e. a <real number>.

= Tensor
{parent=Linear algebra}
{wiki}

A <multilinear form> with a <domain (function)> that looks like:
$$
V^m \times {V*}^n \to \R
$$
where $V*$ is the <dual space>.

Because a tensor is a <multilinear form>, it can be fully specified by how it act on all combinations of basis sets, which can be done in terms of components. We refer to each component as:
$$
T_{i_1 \ldots i_m}^{j_1 \ldots j_n} = T(e_{i_1}, \ldots, e_{i_m}, e^{j_1}, \ldots, e^{j_m})
$$
where we remember that the raised indices refer <dual vector>.

Some examples:
* <levi-Civita symbol as a tensor>{child}
* <a linear map is a (1,1) tensor>

= A linear map is a (1,1) tensor
{parent=Tensor}

A linear map $A$ can be seen as a (1,1) <tensor> because:
$$
T(w, v*) = v* A w
$$
is a number, $v*$. is a <dual vector>, and <W> is a <vector>. Furthermoe, $T$ is linear in both $v*$ and $w$. All of this makes $T$ fullfill the definition of a (1,1) tensor.

= Tensor space
{parent=Tensor}
{title2=$T^{(m, n)}$}

Bibliography:
* https://mathworld.wolfram.com/TensorSpace.html

= Order of a tensor
{parent=Tensor space}

$T^{(m, n)}$ has order $(m, n)$

= Einstein notation
{c}
{parent=Tensor}
{wiki}

= Einstein summation convention
{c}
{synonym}
{title2}

The https://en.wikipedia.org/w/index.php?title=Einstein_notation&oldid=1021244532[Wikipedia page] of this article is basically a masterclass why <ourbigbook com/Wikipedia>[Wikipedia is useless for learning technical subjects]. They are not even able to teach such a simple subject properly there!

Bibliography:
* https://www.maths.cam.ac.uk/postgrad/part-iii/files/misc/index-notation.pdf gives a definition that does not consider upper and lower indexes, it only counts how many times the indices appear

  Their definition of the <Laplacian> is a bit wrong as only one $i$ appears in it, they likely meant to have written $\pdv{}{x_i}\pdv{F}{x_i}$ instead of $\pdv{^2 F}{x_i^2}$, related: 

= Raised and lowered indices
{parent=Einstein notation}

TODO what is the point of them? Why not just sum over every index that appears twice, regardless of where it is, as mentioned at: https://www.maths.cam.ac.uk/postgrad/part-iii/files/misc/index-notation.pdf[].

Vectors with the index on top such as $x^i$ are the "regular vectors", they are called <covariant vectors>.

Those in indices on bottom are called <contravariant vectors>.

It is possible to change between them by <Raising and lowering indices>.

The values are different only when the <metric signature matrix> is different from the <identity matrix>.

= Raised index
{parent=Raised and lowered indices}

= Lowered index
{parent=Raised and lowered indices}

= Raising and lowering indices
{c}
{parent=Raised and lowered indices}
{wiki}

= Implicit metric signature in Einstein notation
{parent=Einstein notation}

Then a specific <metric> is involved, sometimes we want to automatically add it to products.

E.g., in a context considering the common <Minkowski inner product matrix> where the $\eta$ 4x4 matrix and $\mu$ is a vector in <\R^4>
$$
x^{\mu} x_{\mu} = x^{\mu} \eta_{\mu \nu} x^{\nu} = -x_0^2 + x_1^2 + x_2^2 + x_3^2;
$$
which leads to the change of sign of some terms.

= Einstein notation for partial derivatives
{parent=Einstein notation}

The <Einstein summation convention> works will with <partial derivatives> and it is widely used in <particle physics>.

In particular, the <divergence> and the <Laplacian> can be succintly expressed in this notation:
* <Divergence in Einstein notation>{full}
* <Laplacian in Einstein notation>{full}

In order to expresse partial derivatives, we must use what <Ciro Santilli> calls the "<partial index partial derivative notation>", which refers to variales with indices such as $x_0$, $x_1$, $x_2$, $\partial_0$, $\partial_1$ and $\partial_2$  instead of the usual letters $x$, $y$ and $z$.

= Divergence in Einstein notation
{parent=Einstein notation for partial derivatives}
{title2=$\partial_i$}

First we write a <vector field> as:
$$
F(x_0, x_1, x_2) = (F^0(x_0, x_1, x_2), F^1(x_0, x_1, x_2), F^2(x_0, x_1, x_2)) : \R^3 \to \R^3
$$
Note how we are denoting each component of $F$ as $F^i$ with a <raised index>.

Then, the <divergence> can be written in <Einstein notation> as:
$$
\div{F} = \pdv{F^0(x_0, x_1, x_2)}{x_0} + \pdv{F^1(x_0, x_1, x_2)}{x_1} + \pdv{F^2(x_0, x_1, x_2)}{x_2} = \partial_i F^i(x_0, x_1, x_2) = \pdv{F^i(x_0, x_1, x_2)}{x^i}
$$

It is common to just omit the variables of the function, so we tend to just say:
$$
\div{F} = \partial_i F^i
$$
or equivalently when referring just to the <operator>:
$$
\div{} = \partial_i
$$

= Laplacian in Einstein notation
{c}
{parent=Einstein notation for partial derivatives}
{title2=$\partial_i \partial^i$}

Consider a real valued function of three variables:
$$
F(x_0, x_1, x_2) = : \R^3 \to \R
$$

Its <Laplacian> can be written as:
$$
\laplacian{F(x_0, x_1, x_2)} = \\
\partial_0^2 F(x_0, x_1, x_2) + \partial_1^2 F(x_0, x_1, x_2) + \partial_2^2 F(x_0, x_1, x_2) = \\
\partial_0 \partial^0 F(x_0, x_1, x_2) + \partial_1 \partial^1 F(x_0, x_1, x_2) + \partial_2 \partial^2 F(x_0, x_1, x_2) = \\
\partial_i \partial^i F(x_0, x_1, x_2)
$$

It is common to just omit the variables of the function, so we tend to just say:
$$
\laplacian{F} = \partial_i \partial^i F
$$
or equivalently when referring just to the <operator>:
$$
\laplacian{} = \partial_i \partial^i
$$

= D'alembert operator in Einstein notation
{c}
{parent=Laplacian in Einstein notation}
{title2=$\partial_i \partial^i$}

Given the function $\psi$:
$$
\psi : \R^4 \to \C
$$
the operator can be written in <Planck units> as:
$$
\partial_i \partial^i \psi(x_0, x_1, x_2, x_3) - m^2 \psi(x_0, x_1, x_2, x_3) = 0
$$
often written without function arguments as:
$$
\partial_i \partial^i \psi
$$
Note how this looks just like the <Laplacian in Einstein notation>,  since the <D'alembert operator> is just a generalization of the <laplace operator> to <Minkowski space>.

= Klein-Gordon equation in Einstein notation
{parent=D'alembert operator in Einstein notation}

The <Klein-Gordon equation> can be written in terms of the <D'alembert operator> as:
$$
\Box \psi + m^2 \psi = 0
$$
so we can expand the <D'alembert operator in Einstein notation> to:
$$
\partial_i \partial^i \psi - m^2 \psi = 0
$$

= Covariance and contravariance of vectors
{parent=Einstein notation}
{wiki}

= Covariant vector
{parent=Covariance and contravariance of vectors}

= Contravariant vector
{parent=Covariance and contravariance of vectors}

= Linear algebra bibliography
{parent=Linear algebra}

= Interactive Linear Algebra by Margalit and Rabinoff
{c}
{parent=Linear algebra bibliography}
{tag=Visual math HTML book}
{tag=GNU Free Documentation License}

https://textbooks.math.gatech.edu/ila/index.html

Source: https://github.com/QBobWatson/ila[].

Written in <MathBook XML>.