Machine Learning — Variational Inference

Modified from source

Overview

Source
  1. Expand the ELBO into terms of individual variables
  2. Compute the expected value
  3. Optimize the ELBO
Source

KL-divergence

Source

Evidence lower bound (ELBO)

The graph is originated from Wikipedia
Modified from source
Modified from source

Mean Field Variational Inference

Recap

Example

  1. Initialize τn to some arbitrary value.
  2. Solve bn with the equation above.
  3. Solve τn with the equation above.
  4. Repeat the last two steps until the values converge.

Sampling v.s. Variational inference

More readings

Credit and references

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store