Bayesian News Feeds

the luminaries [book review]

Xian's Og - Fri, 2015-04-17 18:15

I bought this book by Eleanor Catton on my trip to Pittsburgh and Toronto in 2013 (thanks to Amazon associates’ gains!), mostly by chance (and also because it was the most recent Man Booker Prize). After a few sleepless nights last week (when I should not have been suffering from New York jet lag!, given my sleeping pattern when abroad), I went through this rather intellectual and somewhat contrived mystery. To keep with tradition (!), the cover was puzzling me until I realised those were phases of the moon, in line with [spoiler!] the zodiacal underlying pattern of the novel, pattern I did not even try to follow for it sounded so artificial. And presumably restricted the flow of the story by imposing further constraints on the characters’ interactions.

The novel has redeeming features, even though I am rather bemused at it getting a Man Booker Prize. (When compared with, say, The Remains of the Day…) For one thing, while a gold rush story of the 1860’s, it takes place on the South Island of New Zealand instead of Klondike, around the Hokitika gold-field, on the West Coast, with mentions of places that brings memory of our summer (well, winter!) visit to Christchurch in 2006… The mix of cultures between English settlers, Maoris, and Chinese migrants, is well-documented and information, if rather heavy at times, bordering on the info-dump, and a central character like the Maori Te Rau Tauwhare sounds caricaturesque. The fact that the story takes place in Victorian times call Dickens to mind, but I find very little connection in either style or structure, nor with Victorian contemporaries like Wilkie Collins, and Victorian pastiches like Charles Palliser‘s Quincunx…. Nothing of the sanctimonious and moral elevation and subtle irony one could expect from a Victorian novel!

While a murder mystery, the plot is fairly upside down (or down under?!): the (spoiler!) assumed victim is missing for most of the novel, the (spoiler!) extracted gold is not apparently stolen but rather lacks owner(s), and the most moral character of the story ends up being the local prostitute. The central notion of the twelve men in a council each bringing a new light on the disappearance of Emery Staines is a neat if not that innovative literary trick but twelve is a large number which means following many threads, some being dead-ends, to gather an appearance of a view on the whole story. As in Rashomon, one finishes the story with a deep misgiving as to who did what, after so many incomplete and biased accountings. Unlike Rashomon, it alas takes forever to reach this point!

Filed under: Books, Kids, Mountains, Travel Tagged: Charles Dickens, Christchurch, Dunedin, gold rush, Man Booker Prize, New Zealand, South Island, The Quincunx, Wilkie Collins
Categories: Bayesian Bloggers

vertical likelihood Monte Carlo integration

Xian's Og - Thu, 2015-04-16 18:15

A few months ago, Nick Polson and James Scott arXived a paper on one of my favourite problems, namely the approximation of normalising constants (and it went way under my radar, as I only became aware of it quite recently!, then it remained in my travel bag for an extra few weeks…). The method for approximating the constant Z draws from an analogy with the energy level sampling methods found in physics, like the Wang-Landau algorithm. The authors rely on a one-dimensional slice sampling representation of the posterior distribution and [main innovation in the paper] add a weight function on the auxiliary uniform. The choice of the weight function links the approach with the dreaded harmonic estimator (!), but also with power-posterior and bridge sampling. The paper recommends a specific weighting function, based on a “score-function heuristic” I do not get. Further, the optimal weight depends on intractable cumulative functions as in nested sampling. It would be fantastic if one could draw directly from the prior distribution of the likelihood function—rather than draw an x [from the prior or from something better, as suggested in our 2009 Biometrika paper] and transform it into L(x)—but as in all existing alternatives this alas is not the case. (Which is why I find the recommendations in the paper for practical implementation rather impractical, since, were the prior cdf of L(X) available, direct simulation of L(X) would be feasible. Maybe not the optimal choice though.)

“What is the distribution of the likelihood ordinates calculated via nested sampling? The answer is surprising: it is essentially the same as the distribution of likelihood ordinates by recommended weight function from Section 4.”

The approach is thus very much related to nested sampling, at least in spirit. As the authors later demonstrate, nested sampling is another case of weighting, Both versions require simulations under truncated likelihood values. Albeit with a possibility of going down [in likelihood values] with the current version. Actually, more weighting could prove [more] efficient as both the original nested and vertical sampling simulate from the prior under the likelihood constraint. Getting away from the prior should help. (I am quite curious to see how the method is received and applied.)

Filed under: Books, pictures, Running, Statistics, Travel, University life Tagged: Chicago Booth School of Business, importance sampling, Monte Carlo integration, Monte Carlo Statistical Methods, nested sampling, normalising constant, slice sampling, Wang-Landau algorithm
Categories: Bayesian Bloggers

ah ces enseignants..!

Xian's Og - Thu, 2015-04-16 13:18

Filed under: Kids, pictures, Travel
Categories: Bayesian Bloggers

abc [with brains]

Xian's Og - Thu, 2015-04-16 08:18

Filed under: Statistics
Categories: Bayesian Bloggers

reis naar Amsterdam

Xian's Og - Wed, 2015-04-15 18:15

On Monday, I went to Amsterdam to give a seminar at the University of Amsterdam, in the department of psychology. And to visit Eric-Jan Wagenmakers and his group there. And I had a fantastic time! I talked about our mixture proposal for Bayesian testing and model choice without getting hostile or adverse reactions from the audience, quite the opposite as we later discussed this new notion for several hours in the café across the street. I also had the opportunity to meet with Peter Grünwald [who authored a book on the minimum description length principle] pointed out a minor inconsistency of the common parameter approach, namely that the Jeffreys prior on the first model did not have to coincide with the Jeffreys prior on the second model. (The Jeffreys prior for the mixture being unavailable.) He also wondered about a more conservative property of the approach, compared with the Bayes factor, in the sense that the non-null parameter could get closer to the null-parameter while still being identifiable.

Among the many persons I met in the department, Maarten Marsman talked to me about his thesis research, Plausible values in statistical inference, which involved handling the Ising model [a non-sparse Ising model with O(p²) parameters] by an auxiliary representation due to Marc Kac and getting rid of the normalising (partition) constant by the way. (Warning, some approximations involved!) And who showed me a simple probit example of the Gibbs sampler getting stuck as the sample size n grows. Simply because the uniform conditional distribution on the parameter concentrates faster (in 1/n) than the posterior (in 1/√n). This does not come as a complete surprise as data augmentation operates in an n-dimensional space. Hence it requires more time to get around. As a side remark [still worth printing!], Maarten dedicated his thesis as “To my favourite random variables , Siem en Fem, and to my normalizing constant, Esther”, from which I hope you can spot the influence of at least two of my book dedications! As I left Amsterdam on Tuesday, I had time for a enjoyable dinner with E-J’s group, an equally enjoyable early morning run [with perfect skies for sunrise pictures!], and more discussions in the department. Including a presentation of the new (delicious?!) Bayesian software developed there, JASP, which aims at non-specialists [i.e., researchers unable to code in R, BUGS, or, God forbid!, STAN] And about the consequences of mixture testing in some psychological experiments. Once again, a fantastic time discussing Bayesian statistics and their applications, with a group of dedicated and enthusiastic Bayesians!

Filed under: Books, Kids, pictures, Running, Statistics, Travel, University life, Wines Tagged: Amsterdam, Bayesian statistics, BUGS, canals, Holland, Ising model, JASP, Marc Kac, minimal description length principle, normalising constant, psychology, R, STAN, UvA
Categories: Bayesian Bloggers

Bernoulli, Montmort and Waldegrave

Xian's Og - Tue, 2015-04-14 18:15

In the last issue of Statistical Science, David Belhouse [author of De Moivre’s biography]  and Nicolas Fillion published an accounting of a discussion between Pierre Rémond de Montmort, Nicolaus Bernoulli—”the” Bernoulli associated with the St. Petersburg paradox—, and Francis Waldegrave, about the card game of Le Her (or Hère, for wretch). Here is the abridged description from the paper:

“Le Her is a game (…) played with a standard deck of fifty-two playing cards. The simplest situation is when two players [Pierre and Paul] play the game, and the solution is not simply determined  even in that situation (…) Pierre deals a card from the deck to Paul and then one to himself. Paul has the option of switching his card for Pierre’s card. Pierre can only refuse the switch if he holds a king (the highest valued card). After Paul makes his decision to hold or switch, Pierre now has the option to hold whatever card he now has or to switch it with a card drawn from the deck. However, if he draws a king, he must retain his original card. The player with the highest card wins the pot, with ties going to the dealer Pierre (…) What are the chances of each player (…) ?” (p.2)

As the paper focus on the various and conflicting resolutions by those 18th Century probabilists, reaching the solution [for Paul to win]

“where a is Paul’s probability of switching with seven, b is Paul’s probability of holding the seven, c is Pierre’s probability of switching with an eight, and d is Pierre’s probability of holding on to an eight”

[which sounds amazing for the time, circa 1713!], where I do not see how a+b or c+d are different from 1,  I ran a small R code to check the probability that Paul wins if he switches when there are more larger than smaller values in the remaining cards and Pierre adopts the same strategy if Paul did not switch:

cards=rep(1:13,4) win=0 T=10^6 for (t in 1:T){ deal=sample(cards,2) #Alice has deal[1] switch=0 rest=cards[-deal[1]] if ((deal[2]<13)&amp;(sum(rest<=deal[1])<sum(rest>=deal[1]))){ switch=deal[2];deal[2]=deal[1];deal[1]=switch} #Bob's turn if (switch>0){ rest=cards[-deal] if (deal[2]<deal[1]){ #sure loss worse than random one draw=sample(rest,1) if (draw<13) deal[2]=draw} }else{ rest=cards[-deal[2]] if (sum(rest<=deal[2])<sum(rest>=deal[2])){ draw=sample(rest,1) if (draw<13) deal[2]=draw}} win=win+(deal[2]>=deal[1]) } 1-win/T

Returning a winning probability of 0.5128 [at the first try] for Paul. However, this is not the optimal strategy for either Paul or Pierre, since randomisation for card values of 7 and 8 push Paul’s odds slightly higher!

Filed under: Books, Kids, R, Statistics
Categories: Bayesian Bloggers

thumbleweed [no] news

Xian's Og - Tue, 2015-04-14 16:25

Just realised today is the second year since my climbing accident and the loss of my right thumb. Even less to say than last anniversary: while it seems almost impossible not to think about it, the handicap is quite minimal. (Actually, the only time I truly forgot about it was when I was ice-climbing in Scotland this January, the difficulty of the [first] climb meaning I had to concentrate on more immediate issues!) Teaching on the blackboard is fine when I use a chalk holder, I just bought a new bike with the easiest change of gears, and except for lacing my running shoes every morning, most chores do not take longer and, as Andrew pointed out in his March+April madness tornament, I can now get away with some missing-body-part jokes!

Filed under: Kids, Mountains, Running, Travel Tagged: amputation, anniversary, blackboard, climbing accident, indoor climbing, Scotland, stolen bike, thumb

Categories: Bayesian Bloggers

failures and uses of Jaynes’ principle of transformation groups

Xian's Og - Mon, 2015-04-13 18:15

This paper by Alon Drory was arXived last week when I was at Columbia. It reassesses Jaynes’ resolution of Bertrand’s paradox, which finds three different probabilities for a given geometric event depending on the underlying σ-algebra (or definition of randomness!). Both Poincaré and Jaynes argued against Bertrand that there was only one acceptable solution under symmetry properties. The author of this paper, Alon Drory, argues this is not the case!

“…contrary to Jaynes’ assertion, each of the classical three solutions of Bertrand’s problem (and additional ones as well!) can be derived by the principle of transformation groups, using the exact same symmetries, namely rotational, scaling and translational invariance.”

Drory rephrases as follows:  “In a circle, select at random a chord that is not a diameter. What is the probability that its length is greater than the side of the equilateral triangle inscribed in the circle?”.  Jaynes’ solution is indifferent to the orientation of one observer wrt the circle, to the radius of the circle, and to the location of the centre. The later is the one most discussed by Drory, as he argued that it does not involve an observer but the random experiment itself and relies on a specific version of straw throws in Jaynes’ argument. Meaning other versions are also available. This reminded me of an earlier post on Buffon’s needle and on the different versions of the needle being thrown over the floor. Therein reflecting on the connection with Bertrand’s paradox. And running some further R experiments. Drory’s alternative to Jaynes’ manner of throwing straws is to impale them on darts and throw the darts first! (Which is the same as one of my needle solutions.)

“…the principle of transformation groups does not make the problem well-posed, and well-posing strategies that rely on such symmetry considerations ought therefore to be rejected.”

In short, the conclusion of the paper is that there is an indeterminacy in Bertrand’s problem that allows several resolutions under the principle of indifference that end up with a large range of probabilities, thus siding with Bertrand rather than Jaynes.

Filed under: Books, Kids, R, Statistics, University life Tagged: Bertrand's paradox, E.T. Jaynes, Henri Poincaré, Joseph Bertrand, randomness
Categories: Bayesian Bloggers

a vignette on Metropolis

Xian's Og - Sun, 2015-04-12 18:15

Over the past week, I wrote a short introduction to the Metropolis-Hastings algorithm, mostly in the style of our Introduction to Monte Carlo with R book, that is, with very little theory and worked-out illustrations on simple examples. (And partly over the Atlantic on my flight to New York and Columbia.) This vignette is intended for the Wiley StatsRef: Statistics Reference Online Series, modulo possible revision. Again, nothing novel therein, except for new examples.

Filed under: Books, Kids, R, Statistics, Travel, University life Tagged: Columbia University, Introducing Monte Carlo Methods with R, Metropolis-Hastings algorithm, mixture, New York city, testing as mixture estimation, vignette
Categories: Bayesian Bloggers

chocolate mousse [recipe]

Xian's Og - Sat, 2015-04-11 18:15

Chocolate mousse may be my favourite desert, most likely because it was considered the top desert by both my mother and my grandmother!, but it is very easy both to miss and to mess the outcome. Which is why I never eat mousse in a restaurant and why I statistically fail to make a proper mousse about 40% of the time [no reproducible experiment available, though]. My recipe is indeed both incredibly simple in its ingredients: only use a bar of baking chocolate and 7 eggs, and highly variable in the output: a wrong kind of baking chocolate, an improperly melted chocolate, an imperfect separation between whites and yolks, those are sufficient reasons for the mousse not to hold, producing instead a chocolatey custard of no particular appeal… Here is the recipe (essentially the one provided inside the Nestlé dessert wrap):Ingredients and utensils

  • 7 fresh eggs, fresh as they are used raw (avoid double-yolkers as well!, as they are hellish to separate);
  • a 250g bar of (bitter) cooking chocolate like [my grandmother’s] Menier dessert, Nestlé dessert, or Poulain dessert, or bitter dark chocolate, with a high cocoa butter content;
  • two large bowls and a smaller one;
  • an egg-beater or a cooking robot


  1. Break the chocolate bar into small chunks and add four spoons of water;
  2. Melt the chocolate by bain-marie or microwave using a temperature and duration as low as possible, and let it cool down;
  3. Separate white from yolk one egg at a time in a small bowl, setting aside [for another recipe] any egg where yolk could have gotten mixed with white, dropping each white into a large bowl and 5 or 6 of the yolks into another large bowl;
  4. beat the whites into a hard and even harder foam;
  5. mix well the melted chocolate with the yolks;
  6. incorporate as gently as possible the whites in the chocolate mix, to avoid breaking the foam, by portions of ¼, until the preparation is roughly homogeneous;
  7. cover and put in the fridge for at least 5 hours.


Since this recipe uses raw eggs, the mousse should be eaten rather quickly, within the next 48 hours. Although it usually vanishes in one meal!

Filed under: Kids Tagged: chocolate bar, chocolate mousse, dessert, Menier, Nestlé, Poulain, raw eggs
Categories: Bayesian Bloggers

George’s dream

Xian's Og - Sat, 2015-04-11 11:09

While I have shared this idea with many of my friends [in both senses that I mentioned it and that they shared the same feeling that it would be a great improvement], the first time I heard of the notion was in George Casella‘s kitchen in Ithaca, New York, in the early 1990’s… We were emptying the dishwasher together and George was reflecting that it would be so convenient to have a double dishwasher and remove the need to empty it altogether! Although, at the moral level, I think that we should do without dishwashers, I found this was a terrific idea and must have told the joke to most of my friends. I was nonetheless quite surprised and very pleased to receive the news from Nicole today that Fisher & Paykel (from Auckland, New Zealand) had gone all the way to produce a double dishwasher, or more exactly a double dishdrawer, perfectly suited to George’s wishes! (Pleased that she remembered the notion after all those years, not pleased with the prospect of buying a double dish washer for more than double the cost of [and a smaller volume than] a regular dishwasher!)

Filed under: Kids, Travel Tagged: Auckland, dishwasher, George Casella, Ithaca, New York, New Zealand
Categories: Bayesian Bloggers

Fool’s Assassin

Xian's Og - Fri, 2015-04-10 18:15

When I learned that Robin Hobb had started a new Assassin’s trilogy, Fitz and the Fool, I got a bit wary, given the poor sequel to the Liveship Traders trilogy I read in the hospital two years ago, and the imperfect Soldier Son trilogy… But also excited, for The Farseer Trilogy is one of the best fantasy series ever! Now that I have read Fool’s Assassin, the first volume of the trilogy, I can only wait for the second one, Fool’s Quest, to appear next summer.  Unsurprisingly, reconnecting with the universe of The Farseer Trilogy is almost enough per se to make reading this book a pleasure, even though it seems to draw too much from the past volumes to gain independent praise, except in the accelerating final chapters. The style conveys too much the homely feeling of Fitz as a retired country squire, surrounded by family and friends. There is obviously a new plot, a new danger to the Six Duchies, and new characters, one of which is singularly attaching!, while Fitz remains as obtuse and whining as in earlier volumes (which is a joy to behold once again!). So now that the setting has been painstakingly and that the game is afoot, I hope the second volume will keep up with the pace of the final chapters… (Nice cover by the way if unrelated to the contents of the book, apart from the snow!)

Filed under: Books, Kids Tagged: Farseer trilogy, Fool's Assassin, hospital, Liveship Traders, Rivership Traders, Robin Hobb
Categories: Bayesian Bloggers

statistical modelling of citation exchange between statistics journals

Xian's Og - Thu, 2015-04-09 18:15

Cristiano Varin, Manuela Cattelan and David Firth (Warwick) have written a paper on the statistical analysis of citations and index factors, paper that is going to be Read at the Royal Statistical Society next May the 13th. And hence is completely open to contributed discussions. Now, I have written several entries on the ‘Og about the limited trust I set to citation indicators, as well as about the abuse made of those. However I do not think I will contribute to the discussion as my reservations are about the whole bibliometrics excesses and not about the methodology used in the paper.

The paper builds several models on the citation data provided by the “Web of Science” compiled by Thompson Reuters. The focus is on 47 Statistics journals, with a citation horizon of ten years, which is much more reasonable than the two years in the regular impact factor. A first feature of interest in the descriptive analysis of the data is that all journals have a majority of citations from and to journals outside statistics or at least outside the list. Which I find quite surprising. The authors also build a cluster based on the exchange of citations, resulting in rather predictable clusters, even though JCGS and Statistics and Computing escape the computational cluster to end up in theory and methods along Annals of Statistics and JRSS Series B.

In addition to the unsavoury impact factor, a ranking method discussed in the paper is the eigenfactor score that starts with a Markov exploration of articles by going at random to one of the papers in the reference list and so on. (Which shares drawbacks with the impact factor, e.g., in that it does not account for the good or bad reason the paper is cited.) Most methods produce the Big Four at the top, with Series B ranked #1, and Communications in Statistics A and B at the bottom, along with Journal of Applied Statistics. Again, rather anticlimactic.

The major modelling input is based on Stephen Stigler’s model, a generalised linear model on the log-odds of cross citations. The Big Four once again receive high scores, with Series B still much ahead. (The authors later question the bias due to the Read Paper effect, but cannot easily evaluate this impact. While some Read Papers like Spiegelhalter et al. 2002 DIC do generate enormous citation traffic, to the point of getting re-read!, other journals also contain discussion papers. And are free to include an on-line contributed discussion section if they wish.) Using an extra ranking lasso step does not change things.

In order to check the relevance of such rankings, the authors also look at the connection with the conclusions of the (UK) 2008 Research Assessment Exercise. They conclude that the normalised eigenfactor score and Stigler model are more correlated with the RAE ranking than the other indicators.  Which means either that the scores are good predictors or that the RAE panel relied too heavily on bibliometrics! The more global conclusion is that clusters of journals or researchers have very close indicators, hence that ranking should be conducted with more caution that it is currently. And, more importantly, that reverting the indices from journals to researchers has no validation and little information.

Filed under: Books, Statistics, University life Tagged: citation index, impact factor, JRSSB, Read paper, Royal Statistical Society, University of Warwick
Categories: Bayesian Bloggers