Math and science::INF ML AI
Belief networks: independence
It's not immediately obvious how to interpret the conditional relationships represented by a belief network. For example, consider the networks below. 
Network(s) b, c and d represent the distribution:
\[\begin{aligned}p(x_1, x_2, x_3) &= p(x_1 \vert x_3)p(x_2 \vert x_3)p(x_3) \\&= p(x_2 \vert x_3)p(x_3 \vert x_1)p(x_1) \\&= p(x_1 \vert x_3)p(x_3 \vert x_2)p(x_2) \end{aligned} \]
Network(s) a represent the distribution:
\[ p(x_1, x_2, x_3) = p(x_3 \vert x_1, x_2)p(x_1)p(x_2)\]
Next, consider conditional independence.
- In a), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) = p(x \vert z)p(y \vert z) \)
- In b), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) \varpropto p(z \vert x)p(x)p(y \vert z) \)
- In c), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) they are [...]. \( p(x, y \vert z) \varpropto p(z \vert x, y)p(x)p(y) \)
- Id d), \( x \) and \( y \) are unconditionally [...], and conditioned on \( z \) or \( w \), are [...]. See book for full equation.