\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Eigenvectors of projection and reflection matricies Let \( P \) and \( R \) be the projection and reflection matricies for a vector \( a \). \( P \) has eigenvalues \(1\) and \(0\), while \( R \) has eigenvalues \(1\) and \(-1\). Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Projection matrix Projection matrix We wish to project the vector \( b \) onto the vector \( a \). The projection matrix \( P \) achieves this by \( Pb \), and is given by: Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Projection matrix Projection matrix We wish to project the vector \( b \) onto the vector \( a \). The projection matrix \( P \) achieves this by \( Pb \), and is given by: Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Affine function Affine function Let \( I \) be a real interval. A function \( \alpha : \mathbb{R} \to \mathbb{R} \) is affine iff [\[ \alpha(p x_1 + (1-p) x_2) = \;\; ? Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Affine function Affine function Let \( I \) be a real interval. A function \( \alpha : \mathbb{R} \to \mathbb{R} \) is affine iff \[ \alpha(p x_1 + (1-p) x_2) = p \alpha(x_1) + (1-p) \alpha(x_2) \] for all \( x_1, x_2 \in I \) and \( p \in [0,1] \). Read more...

Math and science::INF ML AI Projectile arc visualization for Jensen's inequality Another Jensen's inequality visualization. Picture the long parabola arc of a thrown ball from the thrower's hand to where it landed yonder. The first sense of average is the midpoint of the distance traveled between the thrower and the landing spot. To see the second sense of average, stand behind the thrower so the projectile's arc is just a vertical line. Read more...

Math and science::INF ML AI Projectile arc visualization for Jensen's inequality Another Jensen's inequality visualization. Picture the long parabola arc of a thrown ball from the thrower's hand to where it landed yonder. The first sense of average is the midpoint of the distance traveled between the thrower and the landing spot. To see the second sense of average, stand behind the thrower so the projectile's arc is just a vertical line. Read more...

Math and science::INF ML AI Summand on Entropy Consider an element of the form \( x_i \log(\frac{1}{x_i}) \), which makes up the sum to calculate entropy. As a function itself, \( f(x) = x \log(\frac{1}{x}) \) has the below shape: and for a log of base \( b \) the maximum is reached at [\( \;x=\;?\;\)].

Math and science::INF ML AI Summand on Entropy Consider an element of the form \( x_i \log(\frac{1}{x_i}) \), which makes up the sum to calculate entropy. As a function itself, \( f(x) = x \log(\frac{1}{x}) \) has the below shape: and for a log of base \( b \) the maximum is reached at \( x=b \). So with a natural logarithm, the maximum is around 3.6788. Proof The derivative is \( \log(\frac{1}{x}) - 1 \), and setting this to zero gives the result. Read more...

Math and science::INF ML AI Adam optimizer Adam combines momentum and RMSProp into a single optimizer. Momentum The update step is not the gradient, but the exponential moving average of the gradient. This smooths out the gradients to maintain faster updates in an underlying direction. It's best to think of the parameters as being multi-dimensional vectors, as direction and magnitude of the gradients are typically what are manipulated by the optimizer. Read more...