\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Viewing matrices through SVD SVD recap: Singular value decomposition (SVD) Let A be an \( m \times n \) matrix. SVD decomposes \( A \) as: \[ A = U \Sigma V^T \] where \( U \) and \( V \) are orthogonal matrices and \( \Sigma \) is a diagonal matrix with non-negative real numbers on the diagonal. Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Spectral Theorem Spectral theorem Every real symmetric matrix \( S \) can be expressed as [ \[ S = \; ? \; \]] There are two arguments on the reverse that motivate this statement. Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Spectral Theorem Spectral theorem Every real symmetric matrix \( S \) can be expressed as \[ S = Q \Lambda Q^T \] There are two arguments on the reverse that motivate this statement. Read more...

Pytorch's Module class Are there any methods that you think you should know? class Module: def __init__(self, *args, **kwargs) -> None: pass forward: Callable[..., Any] = _forward_unimplemented def register_buffer(self, name: str, tensor: Optional[Tensor], persistent: bool = True) -> None: r"""Add a buffer to the module.""" pass def register_parameter(self, name: str, param: Optional[Parameter]) -> None: r"""Add a parameter to the module.""" pass def add_module(self, name: str, module: Optional['Module']) -> None: r" Read more...

Pytorch's Module class Are there any methods that you think you should know? class Module: def __init__(self, *args, **kwargs) -> None: pass forward: Callable[..., Any] = _forward_unimplemented def register_buffer(self, name: str, tensor: Optional[Tensor], persistent: bool = True) -> None: r"""Add a buffer to the module.""" pass def register_parameter(self, name: str, param: Optional[Parameter]) -> None: r"""Add a parameter to the module.""" pass def add_module(self, name: str, module: Optional['Module']) -> None: r" Read more...

Pytorch's nn.init functionality Below is the main contents of Pytorch's init.h (csrc/api/include/torch/nn/init.h) file. Can you recall the definitions of these functions? The most important are: kaiming_normal_ and kaiming_uniform_ xavier_normal_ and xavier_uniform_ _calculate_fan_in_and_fan_out namespace nn { namespace init { /// Return the recommended gain value for the given nonlinearity function. TORCH_API double calculate_gain( NonlinearityType nonlinearity, double param = 0.01); /// Fills the given `tensor` with the provided `value` in-place, and returns it. Read more...

Pytorch's nn.init functionality Below is the main contents of Pytorch's init.h (csrc/api/include/torch/nn/init.h) file. Can you recall the definitions of these functions? The most important are: kaiming_normal_ and kaiming_uniform_ xavier_normal_ and xavier_uniform_ _calculate_fan_in_and_fan_out namespace nn { namespace init { /// Return the recommended gain value for the given nonlinearity function. TORCH_API double calculate_gain( NonlinearityType nonlinearity, double param = 0.01); /// Fills the given `tensor` with the provided `value` in-place, and returns it. Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Eigenvector decomposition. Equivalent forms. The following is true, for a diagonizable matrix \( A \): [\[ S^{-1} A S = \;\; ? \]]

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Eigenvector decomposition. Equivalent forms. The following is true, for a diagonizable matrix \( A \): \[ S^{-1} A S = \Lambda \] The above is equivalent to the more commonly seen: {{\[ A = S \Lambda S^{-1} \] And another equivalent is: Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra Markov matrices and their eigenvectors. An NZ-AU immigration example. What is the population of Australia and New Zealand after 100 years of the following migration pattern? Starting with 1 million people in Australia and no one in New Zealand. Read more...