I had quite a special day today as I travelled through Birmingham, made a twenty minutes stop in Coventry to drop my bag, went down to London to collect a loaned city bike and took the train back to Coventry with the bike… On my way from Bristol to Warwick, I decided to spend the night in downtown Birmingham as it was both easier and cheaper than to find accommodation on Warwick campus. However, while the studio I rented was well-designed and brand-new, my next door neighbours were not so well-designed in that I could hear them and the TV through the wall, despite top-quality ear-plugs! After a request of mine, they took the TV off but kept to the same decibel level for their uninteresting exchanges. In the morning I tried to go running in the centre of Birmingham but, as I could not find the canals, I quickly got bored. As Mark had proposed to lend me a city bike for my commuting in Warwick, I then decided to take the opportunity of a free Sunday to travel down to London to pick the bike, change the pedals, add an anti-theft device and head back to Coventry. Which gave me the opportunity to bike in London by Abbey Road, Regent Park, and Hampstead, before boarding a fast train back to Coventry and biking up to the University of Warwick campus. (Sadly to discover that all convenience stores had closed by then…)
Filed under: pictures, Running, Travel Tagged: biking, Birmingham, Coventry, England, London, Regent Park, University of Warwick
The title of this recent arXival had potential appeal, however the proposal ends up being rather straightforward and hence anti-climactic! The paper by Hu, Hendry and Heng proposes to run a mixture of proposals centred at the various modes of the target for an efficient exploration. This is a correct MCMC algorithm, granted!, but the requirement to know beforehand all the modes to be explored is self-defeating, since the major issue with MCMC is about modes that are omitted from the exploration and remain undetected throughout the simulation… As provided, this is a standard MCMC algorithm with no adaptive feature and I would rather suggest our population Monte Carlo version, given the available information. Another connection with population Monte Carlo is that I think the performances would improve by Rao-Blackwellising the acceptance rate, i.e. removing the conditioning on the (ancillary) component of the index. For PMC we proved that using the mixture proposal in the ratio led to an ideally minimal variance estimate and I do not see why randomising the acceptance ratio in the current case would bring any improvement.
Filed under: Books, Statistics, University life Tagged: acceptance probability, Metropolis-Hastings algorithms, multimodal target, population Monte Carlo, Rao-Blackwellisation
Filed under: Mountains, pictures, Running, Travel Tagged: agriculture, Auvergne, Besse-en-Chandesse, fields, France, hay, hiking, landscape, Murol, POST, summer
Last and maybe most exciting day of the “High-dimensional Stochastic Simulation and Optimisation in Image Processing” in Bristol as it was exclusively about simulation (MCMC) methods. Except my own talk on ABC. And Peter Green’s on consistency of Bayesian inference in non-regular models. The talks today were indeed about using convex optimisation devices to speed up MCMC algorithms with tools that were entirely new to me, like the Moreau transform discussed by Marcelo Pereyra. Or using auxiliary variables à la RJMCMC to bypass expensive Choleski decompositions. Or optimisation steps from one dual space to the original space for the same reason. Or using pseudo-gradients on partly differentiable functions in the talk by Sylvain Lecorff on a paper commented earlier in the ‘Og. I particularly liked the notion of Moreau regularisation that leads to more efficient Langevin algorithms when the target is not regular enough. Actually, the discretised diffusion itself may be geometrically ergodic without the corrective step of the Metropolis-Hastings acceptance. This obviously begs the question of an extension to Hamiltonian Monte Carlo. And to multimodal targets, possibly requiring as many normalisation factors as there are modes. So, in fine, a highly informative workshop, with the perfect size and the perfect crowd (which happened to be predominantly French, albeit from a community I did not have the opportunity to practice previously). Massive kudos to Marcello for putting this workshop together, esp. on a week where family major happy events should have kept him at home!
As the workshop ended up in mid-afternoon, I had plenty of time for a long run with Florence Forbes down to the Avon river and back up among the deers of Ashton Court, avoiding most of the rain, all of the mountain bikes on a bike trail that sounded like trail running practice, and building enough of an appetite for the South Indian cooking of the nearby Thali Café. Brilliant!
Filed under: pictures, Statistics, Travel, University life Tagged: Ashton Court, Avon river, Bristol, England, Hamiltonian Monte Carlo, MALA, Markov chains, Moreau regularisation, multimodality, RJMCMC, south Indian cuisine, SuSTain, thali
Filed under: Mountains, pictures, Running, Statistics, Travel Tagged: Auvergne, Besse-en-Chandesse, Bleu d'Auvergne, Cantal, cow, Fourme d'Amber, France, Laguiole, Mont-Dore, Puy-de-Dôme, Saint-Nectaire, Salers, sunrise, vacations, volcanoes
After a nice morning run down Leigh Woods and on the muddy banks of the Avon river, I attended a morning session on hyperspectral image non-linear modelling. Topic about which I knew nothing beforehand. Hyperspectral images are 3-D images made of several wavelengths to improve their classification as a mixture of several elements. The non-linearity is due to the multiple reflections from the ground as well as imperfections in the data collection. I found this new setting of clear interest, from using mixtures to exploring Gaussian processes and Hamiltonian Monte Carlo techniques on constrained spaces… Not to mention the “debate” about using Bayesian inference versus optimisation. It was overall a day of discovery as I am unaware of the image processing community (being the outlier in this workshop!) and of their techniques. The problems mostly qualify as partly linear high-dimension inverse problems, with rather standard if sometimes hybrid MCMC solutions. (The day ended even more nicely with another long run in the fields of Ashton Court and a conference diner by the river…)
Filed under: pictures, Statistics, Travel, Uncategorized, University life, Wines Tagged: Bristol, England, Hamiltonian Monte Carlo, image classification, MALA, SuSTain, variational Bayes methods
Filed under: Mountains, pictures, Running, Statistics, Travel Tagged: Auvergne, France, Mont-Dore, Puy de Sancy, Puy-de-Dôme, vacations, volcanoes
Even though I flew through Birmingham (and had to endure the fundamental randomness of trains in Britain), I managed to reach the “High-dimensional Stochastic Simulation and Optimisation in Image Processing” conference location (in Goldney Hall Orangery) in due time to attend the (second) talk by Christophe Andrieu. He started with an explanation of the notion of controlled Markov chain, which reminded me of our early and famous-if-unpublished paper on controlled MCMC. (The label “controlled” was inspired by Peter Green who pointed out to us the different meanings of controlled in French [meaning checked or monitored] and in English . We use it here in the English sense, obviously.) The main focus of the talk was on the stability of controlled Markov chains. With of course connections with out controlled MCMC of old, for instance the case of the coerced acceptance probability. Which happened to be not that stable! With the central tool being Lyapounov functions. (Making me wonder whether or not it would make sense to envision the meta-problem of adaptively estimating the adequate Lyapounov function from the MCMC outcome.)
As I had difficulties following the details of the convex optimisation talks in the afternoon, I eloped to work on my own and returned to the posters & wine session, where the small number of posters allowed for the proper amount of interaction with the speakers! Talking about the relevance of variational Bayes approximations and of possible tools to assess it, about the use of new metrics for MALA and of possible extensions to Hamiltonian Monte Carlo, about Bayesian modellings of fMRI and of possible applications of ABC in this framework. (No memorable wine to make the ‘Og!) Then a quick if reasonably hot curry and it was already bed-time after a rather long and well-filled day!z
Filed under: pictures, Statistics, Travel, Uncategorized, University life, Wines Tagged: Bristol, control, controlled MCMC, England, faux-ami, Hamiltonian Monte Carlo, image classification, Lyapounov function, MALA, Markov chains, SuSTain, variational Bayes methods
Filed under: Mountains, pictures, Running, Travel Tagged: Auvergne, Besse-en-Chandesse, countryside, France, Puy de Sancy, Puy-de-Dôme, vacations
In the newspaper I grabbed in the corridor to my plane today (flying to Bristol to attend the SuSTaIn image processing workshop on “High-dimensional Stochastic Simulation and Optimisation in Image Processing” where I was kindly invited and most readily accepted the invitation), I found a two-page entry on estimating the number of homeless deaths using capture-recapture. Besides the sheer concern about the very high mortality rate among homeless persons (expected lifetime, 48 years; around 7000 deaths in France between 2008 and 2010) and the dreadful realisation that there are an increasing number of kids dying in the streets, I was obviously interested in this use of capture-recapture methods as I had briefly interacted with researchers from INED working on estimating the number of (living) homeless persons about 15 years ago. Glancing at the original paper once I had landed, there was alas no methodological innovation in the approach, which was based on the simplest maximum likelihood estimate. I wonder whether or not more advanced models and [Bayesian] methods of inference could [or should] be used on such data. Like introducing covariates in the process. For instance, when conditioning the probability of (cross-)detection on the cause of death.
Filed under: Statistics, Travel, University life Tagged: Bristol, capture-recapture, covariate, death rate, generalised linear models, homeless, image processing, INED
Filed under: Mountains, pictures, Running, Travel Tagged: Auvergne, Besse-en-Chandesse, countryside, France, Puy de Sancy, Puy-de-Dôme, vacations
Yesterday night, we went to a very special restaurant in down-town Paris, called “dans le noir” where meals take place in complete darkness (truly “dans le noir”!). Complete in the sense it is impossible to see one’s hand and one’s glass. The waiters are blind and the experiment turns them into our guides, as we are unable to progress or eat in the dark! In addition to this highly informative experiment, it was fun to guess the food (easy!) and even more to fail miserably at guessing the colour of the wine (a white Minervois made from Syrah that tasted very much like a red, either from Languedoc-Roussillon or from Bordeaux…!) The food was fine if not outstanding (the owner told us how cooking too refined a meal led to terrible feedbacks from the customers as they could not guess what they were eating) and the wine very good (no picture for the ‘Og, obviously!). This was my daughter’s long-time choice for her 18th birthday dinner and a definitely outstanding idea! So if you have the opportunity to try one of those restaurants (in Barcelona Paseo Picasso, London Clerkenwell, New York, Paris Les Halles, or Saint-Petersbourg), I strongly suggest you to make the move. Eating will never feel the same!
Filed under: Kids, pictures, Travel, Wines Tagged: birthday, blindness, Bordeaux, dans le noir, Languedoc-Roussillon, Les Halles, Minervois, Paris
Filed under: Mountains, pictures, Running, Travel Tagged: Auvergne, Besse-en-Chandesse, France, Puy de Sancy, Puy-de-Dôme, sunrise, vacations
David Minh and Paul Minh [who wrote a 2001 Applied Probability Models] have recently arXived a paper on “understanding the Hastings algorithm”. They revert to the form of the acceptance probability suggested by Hastings (1970):
where s(x,y) is a symmetric function keeping the above between 0 and 1, and q is the proposal. This obviously includes the standard Metropolis-Hastings form of the ratio, as well as Barker’s (1965):
which is known to be less efficient by accepting less often (see, e.g., Antonietta Mira’s PhD thesis). The authors also consider the alternative
which I had not seen earlier. It is a rather intriguing quantity in that it can be interpreted as (a) a simulation of y from the cutoff target corrected by reweighing the previous x into a simulation from q(x|y); (b) a sequence of two acceptance-rejection steps, each concerned with a correspondence between target and proposal for x or y. There is an obvious caveat in this representation when the target is unnormalised since the ratio may then be arbitrarily small… Yet another alternative could be proposed in this framework, namely the delayed acceptance probability of our paper with Marco and Clara, one special case being
is an arbitrary decomposition of the target. An interesting remark in the paper is that any Hastings representation can alternatively be written as
where k(x,y) is a (positive) symmetric function. Hence every single Metropolis-Hastings is also a delayed acceptance in the sense that it can be interpreted as a two-stage decision.
The second part of the paper considers an extension of the accept-reject algorithm where a value y proposed from a density q(y) is accepted with probability
and else the current x is repeated, where M is an arbitrary constant (incl. of course the case where it is a proper constant for the original accept-reject algorithm). Curiouser and curiouser, as Alice would say! While I think I have read some similar proposal in the past, I am a wee intrigued at the appear of using only the proposed quantity y to decide about acceptance, since it does not provide the benefit of avoiding generations that are rejected. In this sense, it appears as the opposite of our vanilla Rao-Blackwellisation. (The paper however considers the symmetric version called the independent Markovian minorizing algorithm that only depends on the current x.) In the extension to proposals that depend on the current value x, the authors establish that this Markovian AR is in fine equivalent to the generic Hastings algorithm, hence providing an interpretation of the “mysterious” s(x,y) through a local maximising “constant” M(x,y). A possibly missing section in the paper is the comparison of the alternatives, albeit the authors mention Peskun’s (1973) result that exhibits the Metropolis-Hastings form as the optimum.
Filed under: Books, Statistics Tagged: accept-reject algorithm, acceptance probability, Barker's algorithm, delayed acceptance, Metropolis-Hastings algorithms, vanilla Rao-Blackwellisation
Filed under: Mountains, pictures, Running, Travel Tagged: Auvergne, Besse-en-Chandesse, fog, France, Lac Pavin, Puy de Sancy, Puy-de-Dôme, sunrise, vacations
Following a proposal put forward by Ted Meeds, Max Welling, Richard Wilkinson, Neil Lawrence and myself, our ABC in Montréal workshop has been accepted by the NIPS 2014 committee and will thus take place on either Friday, Dec. 11, or Saturday, Dec. 12, at the end of the main NIPS meeting (Dec. 8-10). (Despite the title, this workshop is not part of the ABC in … series I started five years ago. It will only last a single day with a few invited talks and no poster. And no free wine & cheese party.) On top of this workshop, our colleagues Vikash K Mansinghka, Daniel M Roy, Josh Tenenbaum, Thomas Dietterich, and Stuart J Russell have also been successful in their bid for the 3rd NIPS Workshop on Probabilistic Programming which will presumably be held on the opposite day to ours, as Vikash is speaking at our workshop, while I am speaking in this workshop. I am yet undecided as to whether or not to attend the main conference, given that I am already travelling a lot this semester and have to teach two courses, incl. a large undergraduate statistics inference course… Obviously, I will try to attend if our joint paper is accepted by the editorial board! Even though Marco will then be the speaker.
Filed under: Kids, Statistics, Travel, University life Tagged: ABC, ABC in Montréal, Canada, delayed acceptance, machine learning, Montréal, NIPS 2014, prefetching, Québec
Filed under: Kids, pictures, Running, Travel Tagged: beach, Drake Island, grass, Maine, sunrise, USA, vacations
In a theme connected with one argument in Dawkins’ The God Delusion, The New York Time just published a piece on the 20th anniversary of the debate between Carl Sagan and Ernst Mayr about the likelihood of the apparition of intelligent life. While 20 years ago, there was very little evidence if any of the existence of Earth-like planets, the current estimate is about 40 billions… The argument against the high likelihood of other inhabited planets is that the appearance of life on Earth is an accumulation of unlikely events. This is where the paper goes off-road and into the ditch, in my opinion, as it makes the comparison of the emergence of intelligent (at the level of human) life to be “as likely as if a Powerball winner kept buying tickets and — round after round — hit a bigger jackpot each time”. The later having a very clearly defined probability of occurring. Since “the chance of winning the grand prize is about one in 175 million”. The paper does not tell where the assessment of this probability can be found for the emergence of human life and I very much doubt it can be justified. Given the myriad of different species found throughout the history of evolution on Earth, some of which evolved and many more which vanished, I indeed find it hard to believe that evolution towards higher intelligence is the result of a basically zero probability event. As to conceive that similar levels of intelligence do exist on other planets, it also seems more likely than not that life took on average the same span to appear and to evolve and thus that other inhabited planets are equally missing means to communicate across galaxies. Or that the signals they managed to send earlier than us have yet to reach us. Or Earth a long time after the last form of intelligent life will have vanished…
Filed under: Books, Kids Tagged: Carl Sagan, cosmology, Ernst Mayr, evolution, exoplanet, Powerball, The God Delusion, The New York Times
Filed under: Kids, pictures, Running, Travel Tagged: beach, Drake Island, Maine, sunrise, USA, vacations, Wells
I had planned my summer read long in advance to have an Amazon shipment sent to my friend Natesh out of my Amazon associate slush funds. While in Boston and Maine, I read Richard Dawkins’ The God delusion, the fourth Kelly McCullough’s Fallen Blade novel, Blade reforged, the second Ancient Blades novel, unrelated to the above, A thief in the night, by David Chandler, and also the second Tad Williams’ Bobby Dollar novel, Happy Hour in Hell. The God delusion is commented on another post.
Blade reforged is not a major novel, unsurprisingly for a fourth entry, but pleasant nonetheless, especially when reading in the shade of a pavilion on Revere Beach! The characters are mostly the same as previously and it could be that the story has (hopefully) come to an end, with (spoilers!) the evil ruler replaced by the hero’s significant other and his mystical weapons returned to him. A few loose ends and a central sword fight with a more than surprising victory, but a good summer read. Checking on Kelly McCullough’s website, I notice that two more novels are in the making….
Tad Williams’ second novel Happy Hour in Hell is much less enjoyable as the author was unable to keep up with the pace and tone of the highly imaginative first novel, full of witty and hard-boiled exchanges. The first novel introduced the (after-)life of a guardian angel in California, Doloriel (a.k.a. Bobby Dollar), with enough levels of political intrigue between Heaven and Hell and Earth and plots, pursuits, assassination attempts, etc., to make it a page-turner. This second novel sends Doloriel on a suicide mission to Hell… and the reader to a Hell of sorts where the damnation is one of eternal boredom! What made the first novel so original, namely the juxtaposition of the purpose of a guardian with his every-day terrestrial life, is lost. All we have there is a fantastic creature (from Heaven) transposed in another fantastic environment (Hell) and trying to survive without a proper guide book. The representation of Hell is not particularly enticing (!), even with acknowledged copies from Dante’s Inferno and Hieronymus Bosch’s paintings. There is a very low tolerance level to my reading of damned souls being tortured, dismembered, eaten or resuscitated, even when it gets to the hero’s turn. Add to that a continuation of the first book’s search for a particular feather. And an amazing amount of space dedicated to the characters’ meals. This makes for a very boring book. Even for a rainy day on a Maine lake! The depiction of the levels and inhabitants of Hell reminded me of another endless book by Tad Williams, Shadowmarch, where some characters end up in a subterranean semi-industrial structure, with a horde of demon-like creatures and no fun [for the reader!]. Ironically, the funniest part of reading Happy Hour in Hell was to do it after Dawkins’ as some reflections of the angel about the roles of Heaven and Hell (and religion) could have fitted well into The God delusion! (Too bad my Maine rental had Monty Python’s Holy Grail instead of The Life of Brian, as it would have made a perfect trilogy!)
Most sadly, David Chandler’s A thief in the night had exactly the same shortcomings as another book I had previously read and maybe reviewed, even though I cannot trace the review or even remember the title of the book (!), and somewhat those of Tad Williams’ Happy Hour in Hell as well, that is, once again a subterranean adventure in a deserted mythical mega-structure that ends up being not deserted at all and even less plausible. I really had to be stuck on a beach or in an airport lounge to finish it! The points noted about Den of Thieves apply even more forcibly here, that is, very charicaturesque characters and a weak and predictable plot. With the addition of the unbearable underground hidden world… I think I should have re-read my own review before ordering this book.
Filed under: Books, Travel Tagged: amazon associates, Ancient Blades, blade reforged, Boston, David Chandler, Kelly McCullough, Maine, Richard Dawkins, Tad Williams, The God Delusion, Wells