Consensus Monte Carlo. Consider a large data set y that is too big or just impractical to process at one time. The idea of consensus Monte Carlo (Scott et al, 2016) is to break the prohibitively big data y into small “shards,”

ys

, s = 1, . . . , S , carry out posterior computation for each shard separately, and then appropriately combine the subset posteriors to recover, or at least approximate, what posterior inference under the complete data could have been.

Assume ys

| θ ∼ N(θ, Σs), s = 1, . . . , S , for known Σs (for example, Σs =

σ

2 I), with conjugate prior h(θ) = N(m, T).

a. Define a subset posterior as hs(θ | y) ∝ f(ys

| θ) h(θ)

1/S

, i.e., as posterior conditional on the subset ys under the modified prior hs(θ) ∝ h(θ)

1/S

.

Show that hs(θ | ys) = N(ms

, Vs), and that h(θ | y) = Πshs(θ | ys) =

N(m, V). Find ms

, Vs

, m, and V.

b. Let θs ∼ hs(θ | y) denote draws from the subset posteriors, s = 1, . . . , S .

Let θ = V

P s V

−1 s

θs



. Show that θ ∼ N(m, V), i.e., θ is a draw from the joint posterior.

For multivariate normal posteriors, the described algorithm provides an exact draw from the joint posterior. In general, one can argue that it provides a reasonable approximate draw from the joint posterior that can be obtained by carrying out computation for small shards only.