Math and science::INF ML AI

Belief networks: independence

It's not immediately obvious how to interpret the conditional relationships represented by a belief network. For example, consider the networks below.

Network(s) b, c and d represent the distribution:

\begin{aligned}p(x_1, x_2, x_3) &= p(x_1 \vert x_3)p(x_2 \vert x_3)p(x_3) \\&= p(x_2 \vert x_3)p(x_3 \vert x_1)p(x_1) \\&= p(x_1 \vert x_3)p(x_3 \vert x_2)p(x_2) \end{aligned}

Network(s) a represent the distribution:

$p(x_1, x_2, x_3) = p(x_3 \vert x_1, x_2)p(x_1)p(x_2)$

Next, consider conditional independence.

• In a), $$x$$ and $$y$$ are unconditionally [...], and conditioned on $$z$$ they are [...]. $$p(x, y \vert z) = p(x \vert z)p(y \vert z)$$
• In b), $$x$$ and $$y$$ are unconditionally [...], and conditioned on $$z$$ they are [...]. $$p(x, y \vert z) \varpropto p(z \vert x)p(x)p(y \vert z)$$
• In c), $$x$$ and $$y$$ are unconditionally [...], and conditioned on $$z$$ they are [...]. $$p(x, y \vert z) \varpropto p(z \vert x, y)p(x)p(y)$$
• Id d), $$x$$ and $$y$$ are unconditionally [...], and conditioned on $$z$$ or $$w$$, are [...]. See book for full equation.