## Xian's Og

### Vivons-nous pour être heureux ? [bacc. 2014]

** T**his year is my daughter’s final year in high school and she is now taking the dreaded baccalauréat exams. Just like a few hundred thousands French students. With “just like” in the strict sense since all students with the same major take the very same exam all over France… The first written composition is in the “mother of all disciplines”, philosophy, and the theme of one dissertation this year was *“do we live to be happy?”*. Which suited well my daughter as she was hoping for a question around that theme. She managed to quote Plato and Buddha, The Pursuit of Happiness and The Wolf of Wall-street… So sounded happy enough with her essay. This seemed indeed like a rather safe notion (as opposed to ethics, religion, politics or work), with enough material to fill a classical thesis-antithesis-synthesis plan (and my personal materialistic conclusion about the lack of predetermination in our lifes).

Filed under: Books, Kids Tagged: Baccalauréat, essay, exam, France, happiness, philosophy

### Bayes at the Bac’ [again]

**W**hen my son took the mathematics exam of the baccalauréat a few years ago, the probability problem was a straightforward application of Bayes’ theorem. (Problem which was later cancelled due to a minor leak…) Surprise, surprise, Bayes is back this year for my daughter’s exam. Once again, the topic is a pharmaceutical lab with a test, test with different positive rates on two populations (healthy vs. sick), and the very basic question is to derive the probability that a person is sick given the test is positive. Then a (predictable) application of the CLT-based confidence interval on a binomial proportion. And the derivation of a normal confidence interval, once again compounded by a CLT-based confidence interval on a binomial proportion… Fairly straightforward with no combinatoric difficulty.

**T**he other problems were on (a) a sequence defined by the integral

(b) solving the equation

in the complex plane and (c) Cartesian 2-D and 3-D geometry, again avoiding abstruse geometric questions… A rather conventional exam from my biased perspective.

Filed under: Kids, Statistics Tagged: Baccalauréat, Cartesian geometry, complex numbers, exam, high school, integrals, polynomials, sequence, Thomas Bayes

### the Poisson transform

**I**n obvious connection with an earlier post on the “estimation” of normalising constants, Simon Barthelmé and Nicolas Chopin just arXived a paper on The Poisson transform for unormalised statistical models. Obvious connection because I heard of the Guttmann and Hyvärinen (2012) paper when Simon came to CREST to give a BiP talk on this paper a few weeks ago. (A connected talk he gave in Banff is available as a BIRS video.)

**W**ithout getting too much into details, the neat idea therein is to turn the observed likelihood

into a joint likelihood

which is the likelihood of a Poisson point process with intensity function

This is an alternative model in that the original likelihood does not appear as a marginal of the above. Only the modes coincide, with the conditional mode in ν providing the normalising constant. In practice, the above Poisson process likelihood is unavailable and Guttmann and Hyvärinen (2012) offer an approximation by means of their logistic regression.

**U**navailable likelihoods inevitably make me think of ABC. Would ABC solutions be of interest there? In particular, could the Poisson point process be simulated with no further approximation? Since the “true” likelihood is not preserved by this representation, similar questions to those found in ABC arise, like a measure of departure from the “true” posterior. Looking forward the Bayesian version!* (Marginalia: Siméon Poisson died in Sceaux, which seemed to have attracted many mathematicians at the time, since Cauchy also spent part of his life there…)*

Filed under: Books, Kids, pictures, Statistics, University life Tagged: Banff, Cauchy, logistic regression, Paris, Poisson point process, Sceaux, Siméon Poisson, untractable normalizing constant

### last Big MC [seminar] before summer [June 19, 3pm]

**L**ast session of our Big’MC seminar at Institut Henri Poincaré this year, on Tuesday** Thursday**, **June 19**, with

Chris Holmes (Oxford) at 3pm on

**Robust statistical decisions via re-weighted Monte Carlo samples**

and Pierre Pudlo (iC3M, Université de Montpellier 2) at 4:15pm on [our joint work]

**ABC and machine learning**

Filed under: pictures, Statistics, University life Tagged: ABC, Big'MC, Chris Holmes, IHP, Institut Henri Poincaré, machine learning, Monte Carlo s, Montpellier, Paris, Pierre Pudlo, seminar, University of Oxford

### Le Monde sans puzzle

**T**his week, Le Monde mathematical puzzle: is purely geometric, hence inappropriate for an R resolution. In the Science & Médecine leaflet, there is however an interesting central page about random generators, from the multiple usages of those in daily life to the consequences of poor generators on cryptography and data safety. The article is compiling an interview of Jean-Paul Delahaye on the topic with recent illustrations from cybersecurity. One final section gets rather incomprehensible: when discussing the dangers of seed generation, it states that “a poor management of the entropy means that an hacker can saturate the seed and take over the original randomness, weakening the whole system”. I am sure there is something real behind the imagery, but this does not make sense… Another insert mentions a possible random generator built out of the light detectors on a smartphone. And quantum physics. The society IDQ can indeed produce ultra-rapid random generators that way. And it also ran randomness tests summarised here. Using in particular George Marsaglia’s diehard battery.

**A**nother column report that a robot passed the Turing test last week, on Turing‘s death anniversary. Meaning that 33% of the jury was convinced the robot’s answers were given by a human. This reminded me of the Most Human Human book sent to me by my friends from BYU. (A marginalia found in Le Monde is that the test was organised by Kevin Warwick…from the University of Coventry, a funny reversal of the University of Warwick sitting in Coventry! However, checking on his website showed that he has and had no affiliation with this university, being at the University of Reading instead.)

Filed under: Books, Kids, Statistics, University life Tagged: Alan Turing, DieHard, George Marsaglia, Jean-Paul Delahaye, Le Monde, mathematical puzzle, pseudo-random generator, randomness

### Caen, June 1944-2014

Filed under: pictures, Running, Travel Tagged: Allied troops, Caen, D-Day beaches, flags, June 1944, Normandy

### Statistical modeling and computation [apologies]

**I**n my book review of the recent book by Dirk Kroese and Joshua Chan, *Statistical Modeling and Computation*, I mistakenly and persistently typed the name of the second author as Joshua Chen. This typo alas made it to the printed and on-line versions of the subsequent CHANCE **27**(2) column. I am thus very much sorry for this mistake of mine and most sincerely apologise to the authors. Indeed, it always annoys me to have my name mistyped (usually as Roberts!) in references. *[If nothing else, this typo signals it is high time for a change of my prescription glasses.]*

Filed under: Books, R, Statistics, University life Tagged: apologies, Australia, Bayesian statistics, Dirk Kroese, introductory textbooks, Joshua Chan, Monte Carlo methods, Monte Carlo Statistical Methods, R, state space model, typo

### checking for finite variance of importance samplers

**O**ver a welcomed curry yesterday night in Edinburgh I read this 2008 paper by Koopman, Shephard and Creal, testing the assumptions behind importance sampling, which purpose is to check on-line for (in)finite variance in an importance sampler, based on the empirical distribution of the importance weights. To this goal, the authors use the upper tail of the weights and a limit theorem that provides the limiting distribution as a type of Pareto distribution

over (0,∞). And then implement a series of asymptotic tests like the likelihood ratio, Wald and score tests to assess whether or not the power ξ of the Pareto distribution is below ½. While there is nothing wrong with this approach, which produces a statistically validated diagnosis, I still wonder at the added value from a practical perspective, as raw graphs of the estimation sequence itself should exhibit similar jumps and a similar lack of stabilisation as the ones seen in the various figures of the paper. Alternatively, a few repeated calls to the importance sampler should disclose the poor convergence properties of the sampler, as in the above graph. Where the blue line indicates the true value of the integral.

Filed under: R, Statistics, Travel, University life Tagged: Abraham Wald, curry, Edinburgh, extreme value theory, importance sampling, infinite variance estimators, Pareto distribution, score function, Scotland

### AppliBUGS day celebrating Jean-Louis Foulley

**I**n case you are in Paris tomorrow and free, there will be an AppliBUGS day focussing on the contributions of our friend Jean-Louis Foulley. (And a regular contributor to the ‘Og!) The meeting takes place in the ampitheatre on second floor of ENGREF-Montparnasse (19 av du Maine, 75015 Paris, Métro Montparnasse Bienvenüe). I will give a part of the O’Bayes tutorial on alternatives to the Bayes factor.

Filed under: pictures, Statistics, University life Tagged: AppliBUGS, Bayes factor, EnGREF, Jean-Louis Foulley, Montparnasse-Bienvenüe, O'Bayes, Paris

### Edinburgh snapshot (#6)

### posterior likelihood ratio is back

*“The PLR turns out to be a natural Bayesian measure of evidence of the studied hypotheses.”*

**I**sabelle Smith and André Ferrari just arXived a paper on the posterior distribution of the likelihood ratio. This is in line with Murray Aitkin’s notion of considering the likelihood ratio

as a *prior* quantity, when contemplating the null hypothesis that θ is equal to θ0. (Also advanced by Alan Birnbaum and Arthur Dempster.) A concept we criticised (rather strongly) in our Statistics and Risk Modelling paper with Andrew Gelman and Judith Rousseau. The arguments found in the current paper in defence of the posterior likelihood ratio are quite similar to Aitkin’s:

- defined for (some) improper priors;
- invariant under observation or parameter transforms;
- more informative than tthe posterior mean of the posterior likelihood ratio, not-so-incidentally equal to the Bayes factor;
- avoiding using the posterior mean for an asymmetric posterior distribution;
- achieving some degree of reconciliation between Bayesian and frequentist perspectives, e.g. by being equal to some p-values;
- easily computed by MCMC means (if need be).

One generalisation found in the paper handles the case of* composite versus composit*e hypotheses, of the form

which brings back an earlier criticism I raised (in Edinburgh, at ICMS, where as one-of-those-coincidences, I read this paper!), namely that using the product of the marginals rather than the joint posterior is no more a standard Bayesian practice than using the data in a prior quantity. And leads to multiple uses of the data. Hence, having already delivered my perspective on this approach in the past, I do not feel the urge to “raise the flag” once again about a paper that is otherwise well-documented and mathematically rich.

Filed under: Statistics, University life Tagged: Alan Birnbaum, Arthur Dempster, Bayesian hypothesis testing, Bayesian p-values, composite hypotheses, Edinburgh, ICMS, invariance, Murray Aitkin, posterior likelihood ratio

### Edinburgh snapshot (#5)

Filed under: pictures, Running, Travel Tagged: Edinburgh, ICMS, Royal Mile, Scotland, St. Giles' cathedral

### lazy ABC

*“A more automated approach would be useful for lazy versions of ABC SMC algorithms.”*

**D**ennis Prangle just arXived the work on lazy ABC he had presented in Oxford at the i-like workshop a few weeks ago. The idea behind the paper is to cut down massively on the generation of pseudo-samples that are “too far” from the observed sample. This is formalised through a stopping rule that puts the estimated likelihood to zero with a probability 1-α(θ,x) and otherwise divide the original ABC estimate by α(θ,x). Which makes the modification unbiased when compared with basic ABC. The efficiency appears when α(θ,x) can be computed much faster than producing the entire pseudo-sample and its distance to the observed sample. When considering an approximation to the asymptotic variance of this modification, Dennis derives a optimal (in the sense of the effective sample size) if formal version of the acceptance probability α(θ,x), conditional on the choice of a “decision statistic” φ(θ,x). And of an importance function g(θ). (I do not get his Remark 1 about the case when π(θ)/g(θ) only depends on φ(θ,x), since the later also depends on x. Unless one considers a multivariate φ which contains π(θ)/g(θ) itself as a component.) This approach requires to estimate

as a function of φ: I would have thought (non-parametric) logistic regression a good candidate towards this estimation, but Dennis is rather critical of this solution.

**I** added the quote above as I find it somewhat ironical: at this stage, to enjoy laziness, the algorithm has first to go through a massive calibration stage, from the selection of the subsample [to be simulated before computing the acceptance probability α(θ,x)] to the construction of the (somewhat mysterious) decision statistic φ(θ,x) to the estimation of the terms composing the optimal α(θ,x). The most natural choice of φ(θ,x) seems to be involving subsampling, still with a wide range of possibilities and ensuing efficiencies. (The choice found in the application is somehow anticlimactic in this respect.) In most ABC applications, I would suggest using a quick & dirty approximation of the distribution of the summary statistic.

**A** slight point of perplexity about this “lazy” proposal, namely the static role of ε, which is impractical because not set in stone… As discussed several times here, the tolerance is a function of many factors incl. all the calibration parameters of the lazy ABC, rather than an absolute quantity. The paper is rather terse on this issue (see Section 4.2.2). It seems to me that playing with a large collection of tolerances may be too costly in this setting.

Filed under: Books, Statistics, University life Tagged: ABC, ABC-SMC, accelerated ABC, calibration, i-like, lazy ABC, logistic regression, University of Oxford

### Edinburgh snapshot (#4)

Filed under: pictures, Travel, University life Tagged: Edinburgh, pub, Scotland, The Guildford Arms, Waverley Station

### Le Monde puzzle [#869]

**A**n uninteresting Le Monde mathematical puzzle:

*Solve the system of equations
*

*a+b+c=16,**b+c+d=12,**d+c+e=16,**e+c+f=18,**g+c+a=15*

* for 7 different integers 1≤a,…,g**≤*9.

**I**ndeed, the final four equations determine *d=a-4, e=b+4, f=a-2, g=b-1* as functions of *a* and *b*. While forcing *5≤a*, 2*≤**b**≤5,* and 7*≤**a+b**≤15*. Hence, 5 possible values for a and 4 for b. Which makes 20 possible solutions for the system. However the fact that* a,b,c,d,e,f,g* are all different reduces considerably the possibilities. For instance, *b* must be less than *a-4*. The elimination of impossible cases leads in the end to consider *b=a-5* and *b=a-7*. And eventually to* a=8, b=3*… Not so uninteresting then. A variant of Sudoku, with open questions like what is the collection of the possible values of the five sums, i.e. of the values with one and only one existing solution? Are there cases where four equations only suffice to determine *a,b,c,d,e,f,g*?

**A**part from this integer programming exercise, a few items of relevance in this Le Monde Science & Medicine leaflet. A description of the day of a social sciences worker in front of a computer, in connection with a sociology (or sociometry) blog and a conference on Big Data in sociology at Collège de France. A tribune by the physicist Marco on data sharing (and not-sharing) illustrated by an experiment on dark matter called Cogent. And then a long interview of Matthieu Ricard, who argues about the “scientifically proven impact of meditation”, a sad illustration of the ease with which religions permeate the scientific debate [or at least the science section of Le Monde] and mingle scientific terms with religious concepts (e.g., the fusion term of “contemplative sciences”). *[As another "of those coincidences", on the same day I read this leaflet, Matthieu Ricard was the topic of one question on a radio quizz.]*

Filed under: Books, Kids, Statistics, University life Tagged: big data, Cogent, Collège de France, dark matter, Le Monde, mathematical puzzle, Matthieu Ricard, neurosciences, religions, social sciences

### June 7, 1944

*[I wrote this post a few years ago, but the 70th anniversary of the D-day brought back those memories and I thought it worth re-posting...]*

**T**his is the day I almost got un-born, not that I was born at the time (!) but my mother, then almost seven, came close to dying under the Allied bombs that obliterated Saint-Lô (Manche, western France) from the map that night, in conjunction with the D Day landing in the nearby beaches of Utah Beach and Omaha Beach. (The city was supposed to be taken by the end of June 6, but it was only on July 19 that Allied troops entered Saint-Lô.) Most of the town got destroyed under 60,000 pounds of bombs in an attempt by the Allied forces to cut access to the beaches from German reinforcements from Brittany. (Saint-Lô got the surname of “capital of the ruins” from Samuel Beckett after this bombing and it took many years to reconstruct.) My granparents and their three daughters barely went out of their house before it collapsed and had to flee the ablaze Saint-Lô with a single cartwheel to carry two suitcases and the three girls. Several times did my grandfather hide them under his leather jacket for power lines were collapsing around them…

**T**hey eventually (and obviously) made it alive out of Saint-Lô, only to be rounded up with other refugees by German troops who parked them in a field, most likely to be used as hostages. Taking advantage of the night, my grandfather managed once again to get his family away by crawling under the barriers on the darkest side of the field and they then reached (by foot) a most secluded village in the countryside where my great-grandmother was living at the time. From when I was a child, I have heard this story so many times from my mother that it is almost pictured in my brain, as if I had seen the “movie”, somehow.

Filed under: Kids Tagged: Allied troops, bombing, capital of the ruins, D Day, Saint-Lô, WW II

### computational methods for statistical mechanics [day #4]

**M**y last day at this ICMS workshop on molecular simulation started [with a double loop of Arthur's Seat thankfully avoiding the heavy rains of the previous night and then] Chris Chipot‘s magistral entry to molecular simulation for proteins with impressive slides and simulation movies, even though I could not follow the details to really understand the simulation challenges therein, just catching a few connections with earlier talks. A typical example of a cross-disciplinary gap, where the other discipline always seems to be stressing the ‘wrong” aspects. Although this is perfectly unrealistic, it would immensely to prepare talks in pairs for such interdisciplinary workshops! Then Gersende Fort presented results about convergence and efficiency for the Wang-Landau algorithm. The idea is to find the optimal rate for updating the weights of the elements of the partition towards reaching the flat histogram in minimal time. Showing massive gains on toy examples. The next talk went back to molecular biology with Jérôme Hénin‘s presentation on improved adaptive biased sampling. With an exciting notion of orthogonality aiming at finding the slowest directions in the target and putting the computational effort. He also discussed the tension between long single simulations and short repeated ones, echoing a long-going debate in the MCMC community. (He also had a slide with a picture of my first 1983 Apple IIe computer!) Then Antonietta Mira gave a broad perspective on delayed rejection and zero variance estimates. With impressive variance reductions (although some physicists then asked for reduction of order 10¹⁰!). Johannes Zimmer gave a beautiful maths talk on the connection between particle and diffusion limits (PDEs) and Wasserstein geometry and large deviations. (I did not get most of the talk, but it was nonetheless beautiful!) Bert Kappen concluded the day (and the workshop for me) by a nice introduction to control theory. Making connection between optimal control and optimal importance sampling. Which made me idly think of the following problem: what if control cannot be completely… controlled and hence involves a stochastic part? Presumably of little interest as the control would then be on the parameters of the distribution of the control.

*“The alanine dipeptide is the fruit fly of molecular simulation.”*

**T**he example of this alanine dipeptide molecule was so recurrent during the talks that it justified the above quote by Michael Allen. Not that I am more proficient in the point of studying this protein or using it as a benchmark. Or in identifying the specifics of the challenges of molecular dynamics simulation. Not a criticism of the ICMS workshop obviously, but rather of my congenital difficulty with continuous time processes!!! So I do not return from Edinburgh with a new research collaborative project in molecular dynamics (if with more traditional prospects), albeit with the perception that a minimal effort could bring me to breach the vocabulary barrier. And maybe consider ABC ventures in those (new) domains. (Although I fear my talk on ABC did not impact most of the audience!)

Filed under: Mountains, pictures, Running, Statistics, Travel, University life Tagged: ABC, Arthur's Seat, computational physics, control theory, control variate, delayed rejection sampling, Edinburgh, Highlands, ICMS, Langevin diffusion, large deviation, MCMC, molecular simulation, Monte Carlo Statistical Methods, Scotland, Wasserstein distance, zero variance importance sampling

### computational methods for statistical mechanics [day #4]

**M**y last day at this ICMS workshop on molecular simulation started [with a double loop of Arthur's Seat thankfully avoiding the heavy rains of the previous night and then] Chris Chipot‘s magistral entry to molecular simulation for proteins with impressive slides and simulation movies, even though I could not follow the details to really understand the simulation challenges therein, just catching a few connections with earlier talks. A typical example of a cross-disciplinary gap, where the other discipline always seems to be stressing the ‘wrong” aspects. Although this is perfectly unrealistic, it would immensely to prepare talks in pairs for such interdisciplinary workshops! Then Gersende Fort presented results about convergence and efficiency for the Wang-Landau algorithm. The idea is to find the optimal rate for updating the weights of the elements of the partition towards reaching the flat histogram in minimal time. Showing massive gains on toy examples. The next talk went back to molecular biology with Jérôme Hénin‘s presentation on improved adaptive biased sampling. With an exciting notion of orthogonality aiming at finding the slowest directions in the target and putting the computational effort. He also discussed the tension between long single simulations and short repeated ones, echoing a long-going debate in the MCMC community. (He also had a slide with a picture of my first 1983 Apple IIe computer!) Then Antonietta Mira gave a broad perspective on delayed rejection and zero variance estimates. With impressive variance reductions (although some physicists then asked for reduction of order 10¹⁰!). Johannes Zimmer gave a beautiful maths talk on the connection between particle and diffusion limits (PDEs) and Wasserstein geometry and large deviations. (I did not get most of the talk, but it was nonetheless beautiful!) Bert Kappen concluded the day (and the workshop for me) by a nice introduction to control theory. Making connection between optimal control and optimal importance sampling. Which made me idly think of the following problem: what if control cannot be completely… controlled and hence involves a stochastic part? Presumably of little interest as the control would then be on the parameters of the distribution of the control.

*“The alanine dipeptide is the fruit fly of molecular simulation.”*

**T**he example of this alanine dipeptide molecule was so recurrent during the talks that it justified the above quote by Michael Allen. Not that I am more proficient in the point of studying this protein or using it as a benchmark. Or in identifying the specifics of the challenges of molecular dynamics simulation. Not a criticism of the ICMS workshop obviously, but rather of my congenital difficulty with continuous time processes!!! So I do not return from Edinburgh with a new research collaborative project in molecular dynamics (if with more traditional prospects), albeit with the perception that a minimal effort could bring me to breach the vocabulary barrier. And maybe consider ABC ventures in those (new) domains. (Although I fear my talk on ABC did not impact most of the audience!)

Filed under: Mountains, pictures, Running, Statistics, Travel, University life Tagged: ABC, Arthur's Seat, computational physics, control theory, control variate, delayed rejection sampling, Edinburgh, Highlands, ICMS, Langevin diffusion, large deviation, MCMC, molecular simulation, Monte Carlo Statistical Methods, Scotland, Wasserstein distance, zero variance importance sampling