Universiteit Utrecht

Department of Mathematics


Abstract


Hidden variables, missing data, and optimal design of the experiment of the century.
Richard Gill (UU), October 2, 2002

Consider four binary random variables X_1, X_2, Y_1, Y_2 such that one can only collect data from the joint distribution of each of the pairs (X_i,Y_j), i=1,2, j=1,2, but never from the complete four-tuple. There is an experimental set-up in physics where this picture is applicable, where moreover two competing theories say something very different about the possible probability distributions of the data: according to classical physics (C), the four bivariate distributions from which data can be collected, are bivariate marginals of a single four-variate distribution, while according to quantum mechanics (Q), this need not necessarily be the case.

Now there are a lot of variations possible in an experimental test of Q's claims. The literature contains competing claims that one or another experimental design is the best one, usually based on rather vague criteria. I will argue that an objective way to compare different possible experiments is in terms of the weight of statistical evidence provided, per observation, for Q versus the competing classical theories C.

I will explain how this weight of evidence can be quantified using the Kullback-Leibler divergence between the two competing probability models for the data [the mean value under Q of the log-likelihood ratio of Q versus C]. Next I will show how finding the optimal experimental design corresponds to finding a saddle-point in a certain two-person zero-sum game, played between the quantum experimentalist Q and the classical theorist C, in which Q chooses various features of experimental design while C tries to come up with a theory which explains the resulting data.

Some of the games we will consider turn out to have solutions (saddlepoints) and hence values, which lead to an objective comparison between various proposals.

This is joint-work-in-progress with Wim van Dam (Berkeley) and Peter Grunwald (CWI). Some of our results are numerical and are obtained using programs developed by Piet Groeneboom for solving nonparametric missing data problems - the connection will be explained.


Back to the history of the seminar or the Colloquium Stochastiek homepage.
Martijn Pistorius (pistorius@math.uu.nl)