\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra::Aluffi Groups. Order for commutative product. Order for commutative product If \( gh = hg \), then \( |gh| \) divides [...].

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::Algebra::Aluffi Groups. Order for commutative product. Order for commutative product If \( gh = hg \), then \( |gh| \) divides \( \operatorname{lcm}(\; |g|, \; |h| \;) \). Proof Let \( N = \operatorname{lcm}(\; |g|, \; |h| \; ) \). Read more...

Overview of progress in experiment 2.

Experiment 1.5.1, but with ResNet18 instead of ResNet50.

This is the second attempt at experiment 1.1. Same setup, more data collected.

Experiment 1.3, but with the single circle being varied by size and position.

A green-blue version of 1.5, carried out for the purposes of comparison.

Below are some videos that show how weights, activations and gradients change as a network is trained. The videos were made while trying to test the idea that layers closer to the input stabilize earlier than layers closer to the output. The below videos suggest this hypothesis is wrong. In fact, quite often the updates to the last layer are the first to slow down, and the updates to the first layer are the last. Read more...

This is a tl;dr post of a longer (and not yet existing) post on variational auto-encoders. Derivation idea The evidence lower bound (ELBO) expression appears naturally when you try to sample the posterior distribution with an approximate distribution. I think this way of arriving at the evidence lower bound is intuitive and reveals more about why concessions are being made. Importance sampling allows us to calculate the expectation: \[ \E_{z \sim \mathrm{P}_z}[f(z)] \] by instead calculating: Read more...

\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \newcommand{\multiBetaReduction}[0] {\twoheadrightarrow_{\beta}} \newcommand{\betaReduction}[0] {\rightarrow_{\beta}} \newcommand{\betaEq}[0] {=_{\beta}} \newcommand{\string}[1] {\texttt{"}\mathtt{#1}\texttt{"}} \newcommand{\symbolq}[1] {\texttt{`}\mathtt{#1}\texttt{'}} \newcommand{\groupMul}[1] { \cdot_{\small{#1}}} \newcommand{\groupAdd}[1] { +_{\small{#1}}} \newcommand{\inv}[1] {#1^{-1} } \newcommand{\bm}[1] { \boldsymbol{#1} } \require{physics} \require{ams} \require{mathtools} \) Math and science::INF ML AI Importance sampling and rejection sampling [Rejection or importance?] sampling is useful for generating samples that are distributed according to a difficult-to-sample-from distribution. [Rejection or importance?] sampling is useful for evaluating a function (such as expectation) of a random variable with a difficult-to-sample-from distribution. Read more...