\( \newcommand{\matr}[1] {\mathbf{#1}} \newcommand{\vertbar} {\rule[-1ex]{0.5pt}{2.5ex}} \newcommand{\horzbar} {\rule[.5ex]{2.5ex}{0.5pt}} \)
header
Show Answer
\( \newcommand{\cat}[1] {\mathrm{#1}} \newcommand{\catobj}[1] {\operatorname{Obj}(\mathrm{#1})} \newcommand{\cathom}[1] {\operatorname{Hom}_{\cat{#1}}} \)
Math and science::INF ML AI

Belief networks: independence

It's not immediately obvious how to interpret the conditional relationships represented by a belief network. For example, consider the networks below.

Network(s) [...] represent the distribution:

\[\begin{aligned}p(x_1, x_2, x_3) &= p(x_1 \vert x_3)p(x_2 \vert x_3)p(x_3) \\&= p(x_2 \vert x_3)p(x_3 \vert x_1)p(x_1) \\&= p(x_1 \vert x_3)p(x_3 \vert x_2)p(x_2) \end{aligned} \]

Network(s) [...] represent the distribution:

\[ p(x_1, x_2, x_3) = p(x_3 \vert x_1, x_2)p(x_1)p(x_2)\]

Next, consider conditional independence.


  • In a), \( x \) and \( y \) are unconditionally dependent, and conditioned on \( z \) they are independent. Knowing either gives information on the distribution of \( z \), which in turn informs of the others distribution.. \( p(x, y \vert z) = p(x \vert z)p(y \vert z) \)
  • In b), \( x \) and \( y \) are unconditionally dependent, and conditioned on \( z \) they are independent. \( p(x, y \vert z) \varpropto p(z \vert x)p(x)p(y \vert z) \)
  • In c), \( x \) and \( y \) are unconditionally independent, and conditioned on \( z \) they are dependent. \( p(x, y \vert z) \varpropto p(z \vert x, y)p(x)p(y) \)
  • Id d), \( x \) and \( y \) are unconditionally independent, and conditioned on \( z \) or \( w \), are dependent. See book for full equation.