Bayesian News Feeds

two, three, five, …, a million standard deviations!

Xian's Og - Thu, 2014-09-25 18:14

I first spotted Peter Coles’ great post title “Frequentism: the art of probably answering the wrong question” (a very sensible piece by the way!, and mentioning a physicist’s view on the Jeffreys-Lindley paradox I had intended to comment) and from there the following site jumping occured:

“I confess that in my early in my career as a physicist I was rather cynical about sophisticated statistical tools, being of the opinion that “if any of this makes a difference, just get more data”. That is, if you do enough experiments, the confidence level will be so high that the exact statistical treatment you use to evaluate it is irrelevant.” John Butterworth, Sept. 15, 2014

After Val Johnson‘s suggestion to move the significant level from .05 down to .005, hence roughly from 2σ up to 3σ, John Butterworth, a physicist whose book Smashing Physics just came out, discusses in The Guardian the practice of using 5σ in Physics. It is actually induced by Louis Lyons’ arXival of a recent talk with the following points (discussed below):

  1. Should we insist on the 5 sigma criterion for discovery claims?
  2. The probability of A, given B, is not the same as the probability of B, given A.
  3. The meaning of p-values.
  4. What is Wilks Theorem and when does it not apply?
  5. How should we deal with the `Look Elsewhere Effect’?
  6. Dealing with systematics such as background parametrisation.
  7. Coverage: What is it and does my method have the correct coverage?
  8. The use of p0 versus p1 plots.

Butterworth’s conclusion is worth reproducing:

“…there’s a need to be clear-eyed about the limitations and advantages of the statistical treatment, wonder what is the “elsewhere” you are looking at, and accept that your level of certainty may never feasibly be 5σ. In fact, if the claims being made aren’t extraordinary, a one-in-2million chance of a mistake may indeed be overkill, as well being unobtainable. And you have to factor in the consequences of acting, or failing to act, based on the best evidence available – evidence that should include a good statistical treatment of the data.” John Butterworth, Sept. 15, 2014

esp. the part about the “consequences of acting”, which I interpret as incorporating a loss function in the picture.

Louis’s paper-ised talk 1. [somewhat] argues in favour of the 5σ because 2σ and 3σ are not necessarily significant on larger datasets. I figure the same could be said of 5σ, no?! He also mentions (a) “systematics”, which I do not understand. Even though this is not the first time I encounter the notion in Physics. And (b) “subconscious Bayes factors”, which means that the likelihood ratio [considered here as a transform of the p-value] is moderated by the ratio of the prior probabilities, even when people do not follow a Bayesian procedure. But this does not explain why a fixed deviation from the mean should be adopted. 2. and 3. The following two points are about the common confusion in the use of the p-value, found in most statistics textbooks. Even though the defence of the p-value against the remark that it is wrong half the time (as in Val’s PNAS paper) misses the point. 4. Wilk’s theorem is a warning that the χ² approximation only operates under some assumptions. 5. Looking elsewhere is the translation of multiple testing or cherry-picking. 6. Systematics is explained here as a form of model misspecification. One suggestion is to use a Bayesian modelling of this misspecification, another non-parametrics (why not both together?!). 7. Coverage is somewhat disjunct from the other points as it explains the [frequentist] meaning of the coverage of a confidence interval. Which hence does not apply to the actual data. 8. p0 versus p1 plots is a sketchy part referring to a recent proposal by the author. So in the end a rather anticlimactic coverage of standard remarks, surprisingly giving birth to a sequence of posts (incl. this one!)…

Filed under: Books, Statistics, University life Tagged: Bayesian modeling, five sigma, John Butterworth, likelihood ratio, Louis Lyons, p-values, PNAS, The Guardian, Valen Johnson
Categories: Bayesian Bloggers

interesting mis-quote

Xian's Og - Wed, 2014-09-24 18:14

At a recent conference on Big Data, one speaker mentioned this quote from Peter Norvig, the director of research at Google:

“All models are wrong, and increasingly you can succeed without them.”

quote that I found rather shocking, esp. when considering the amount of modelling behind Google tools. And coming from someone citing Kernel Methods for Pattern Analysis by Shawe-Taylor and Christianini as one of his favourite books and Bayesian Data Analysis as another one… Or displaying Bayes [or his alleged portrait] and Turing in his book cover. So I went searching on the Web for more information about this surprising quote. And found the explanation, as given by Peter Norvig himself:

“To set the record straight: That’s a silly statement, I didn’t say it, and I disagree with it.”

Which means that weird quotes have a high probability of being misquotes. And used by others to (obviously) support their own agenda. In the current case, Chris Anderson and his End of Theory paradigm. Briefly and mildly discussed by Andrew a few years ago.

Filed under: Books, pictures, Statistics, Travel, University life Tagged: Alan Turing, all models are wrong, artificial intelligence, George Box, misquote, Peter Norvig, statistical modelling, The End of Theory, Thomas Bayes
Categories: Bayesian Bloggers

a weird beamer feature…

Xian's Og - Tue, 2014-09-23 18:14

As I was preparing my slides for my third year undergraduate stat course, I got a weird error that got a search on the Web to unravel:

! Extra }, or forgotten \endgroup. \endframe ->\egroup \begingroup \def \@currenvir {frame} l.23 \end{frame} \begin{slide} ?

which was related with a fragile environment

\begin{frame}[fragile] \frametitle{simulation in practice} \begin{itemize} \item For a given distribution $F$, call the corresponding pseudo-random generator in an arbitrary computer language \begin{verbatim} > x=rnorm(10) > x [1] -0.021573 -1.134735 1.359812 -0.887579 [7] -0.749418 0.506298 0.835791 0.472144 \end{verbatim} \item use the sample as a statistician would \begin{verbatim} > mean(x) [1] 0.004892123 > var(x) [1] 0.8034657 \end{verbatim} to approximate quantities related with $F$ \end{itemize} \end{frame}\begin{frame}

but not directly the verbatim part: the reason for the bug was that the \end{frame} command did not have a line by itself! Which is one rare occurrence where the carriage return has an impact in LaTeX, as far as I know… (The same bug appears when there is an indentation at the beginning of the line. Weird!) [Another annoying feature is wordpress turning > into > in the sourcecode environment...]

Filed under: Books, Kids, Linux, R, Statistics, University life Tagged: Beamer, bootstrap, course, fragile environment, LaTeX, R, random number generation, rnorm(), slides, Statistics, Université Paris Dauphine, verbatim, \end{frame}
Categories: Bayesian Bloggers

Statistics second slides

Xian's Og - Tue, 2014-09-23 18:14

This is the next chapter of my Statistics course, definitely more standard, with some notions on statistical models, limit theorems, and exponential families. In the first class, I recalled the convergence notions with no proof but counterexamples and spend some time on a slide not included here, borrowed from Chris Holmes’ talk last Friday on the linear relation between blood pressure and the log odds ratio of an heart condition. This was a great example, both to illustrate the power of increasing the number of observations and of using a logistic regression model. Students kept asking questions about it.

Filed under: Books, Kids, Statistics, University life Tagged: blood pressure, exponential families, logistic regression, statistical modelling, undergraduates, Université Paris Dauphine
Categories: Bayesian Bloggers

BAYSM’14 recollection

Xian's Og - Mon, 2014-09-22 18:14

When I got invited to BAYSM’14 last December, I was quite excited to be part of the event. (And to have the opportunities to be in Austria, in Wien and on the new WU campus!) And most definitely and a posteriori I have not been disappointed given the high expectations I had for that meeting…! The organisation was seamless, even by Austrian [high] standards, the program diverse and innovative, if somewhat brutal for older Bayesians and the organising committee (Angela Bitto, Gregor Kastner, and Alexandra Posekany) deserves an ISBA recognition award [yet to be created!] for their hard work and dedication. Thanks also to Sylvia Früwirth-Schnatter for hosting the meeting in her university. They set the standard very high for the next BAYSM organising team. (To be hold in Firenze/Florence, on June 19-21, 2016, just prior to the ISBA World meeting not taking place in Banff. A great idea to associate with a major meeting, in order to save on travel costs. Maybe the following BAYSM will take place in Edinburgh! Young, local, and interested Bayesians just have to contact the board of BAYS with proposals.)

So, very exciting and diverse. A lot of talks in applied domains, esp. economics and finance in connection with the themes of the guest institution, WU.  On the talks most related to my areas of interest, I was pleased to see Matthew Simpson working on interweaving MCMC with Vivek Roy and Jarad Niemi, Madhura Killedar constructing her own kind of experimental ABC on galaxy clusters, Kathrin Plankensteiner using Gaussian processes on accelerated test data, Julyan Arbel explaining modelling by completely random measures for hazard mixtures [and showing his filliation with me by (a) adapting my pun title to his talk, (b) adding an unrelated mountain picture to the title page, (c) including a picture of a famous probabilist, Paul Lévy, to his introduction of Lévy processes and (d) using xkcd strips], Ewan Cameron considering future ABC for malaria modelling,  Konstantinos Perrakis working on generic importance functions in data augmentation settings, Markus Hainy presenting his likelihood-free design (that I commented a while ago), Kees Mulder explaining how to work with the circular von Mises distribution. Not to mention the numerous posters I enjoyed over the first evening. And my student Clara Grazian who talked about our joint and current work on Jeffreys priors for mixture of distributions. Whose talk led me to think of several extensions…

Besides my trek through past and current works of mine dealing with mixtures, the plenary sessions for mature Bayesians were given by Mike West and Chris Holmes, who gave very different talks but with the similar message that data was catching up with modelling and with a revenge and that we [or rather young Bayesians] needed to deal with this difficulty. And use approximate or proxy models. Somewhat in connection with my last part on an alternative to Bayes factors, Mike also mentioned a modification of the factor in order to attenuate the absorbing impact of long time series. And Chris re-set Bayesian analysis within decision theory, constructing approximate models by incorporating the loss function as a substitute to the likelihood.

Once again, a terrific meeting in a fantastic place with a highly unusual warm spell. Plus enough time to run around Vienna and its castles and churches. And enjoy local wines (great conference evening at a Heuriger, where we did indeed experience Gemütlichkeit.) And museums. Wunderbar!

Filed under: Books, Kids, pictures, Statistics, Travel, University life, Wines Tagged: ABC, approximate likelihood, architecture, Austria, BAYSM 2014, Donau, econometrics, Heuriger, interweaving, MCMC, Vienna, Wien, WU Wien, young Bayesians
Categories: Bayesian Bloggers

An der schönen blauen Donau

Xian's Og - Mon, 2014-09-22 08:18
Categories: Bayesian Bloggers

new kids on the block

Xian's Og - Sun, 2014-09-21 18:14

This summer, for the first time, I took three Dauphine undergraduate students into research projects thinking they had had enough R training (with me!) and several stats classes to undertake such projects. In all cases, the concept was pre-defined and “all they had to do” was running a massive flow of simulations in R (or whatever language suited them best!) to check whether or not the idea was sound. Unfortunately, for two projects, by the end of the summer, we had not made any progress in any of the directions I wanted to explore… Despite a fairly regular round of meetings and emails with those students. In one case the student had not even managed to reproduce the (fairly innocuous) method I wanted to improve upon. In the other case, despite programming inputs from me, the outcome was impossible to trust.  A mostly failed experiment which makes me wonder why it went that way. Granted that those students had no earlier training in research, either in exploiting the literature or in pushing experiments towards logical extensions. But I gave them entries, discussed with them those possible new pathways, and kept updating schedules and work-charts. And the students were volunteers with no other incentive than discovering research (I even had two more candidates in the queue).  So it may be (based on this sample of 3!) that our local training system is missing in this respect. Somewhat failing to promote critical thinking and innovation by imposing too long presence hours and by evaluating the students only through standard formalised tests. I do wonder, as I regularly see [abroad] undergraduate internships and seminars advertised in the stats journals. Or even conferences.

Filed under: Kids, R, Statistics, University life Tagged: academic research, research internships, training of researchers, undergraduates
Categories: Bayesian Bloggers

snapshot from Vienna

Xian's Og - Sun, 2014-09-21 08:18
Categories: Bayesian Bloggers

Le Monde puzzle [#879]

Xian's Og - Sat, 2014-09-20 18:14

Here is the last week puzzle posted in Le Monde:

Given an alphabet with 26 symbols, is it possible to create 27 different three-symbol words such that

  1. all symbols within a word are different
  2. all triplets of symbols are different
  3. there is no pair of words with a single common symbol

Since there are


such three-symbol words, it could be feasible to write an R code that builds the 27-uplet, assuming it exists. However, by breaking those words into primary words [that share no common symbols] and secondary words [that share two symbols with one primary word], it seems to me that there can be a maximum of 26 words under those three rules…

Filed under: Books, Kids Tagged: combinatorics, Le Monde, mathematical puzzle
Categories: Bayesian Bloggers

Basil the chipmunk (#2)

Xian's Og - Sat, 2014-09-20 08:18
Categories: Bayesian Bloggers

someone to watch over me [Horfðu á mig]

Xian's Og - Fri, 2014-09-19 18:14

And yet another roman noir taking place in Iceland! My bedside read over the past two months was “Someone to watch over me” by Yrsa Sigurðardóttir. (It took that long because I was mostly away in July and August, not because the book was boring me to sleep every night!) It is a fairly unusual book in several respects: the setting is an institution for mentally handicapped patients that was set on fire, killing five of the patients as a result, the investigator is an Icelandic lawyer, Þóra Guðmundsdóttir, along with her German unemployed-banker boyfriend, the action takes place at the height [or bottom!] of the Icelandic [and beyond!] economic crisis, when most divorce settlements are about splitting the debts of the household, and when replacing a computer becomes an issue, some of the protagonists, including the main suspects, are mentally ill, and the police and justice are strangely absent from most of the story. The the book tells a lot about the Icelandic society, where a hit-and-run is so unheard of that the police is clueless. Or seems to be. And where people see ghosts. Or think they do, as the author plays (heavily?) on the uncertainty about those ghosts. (At least, there are no elves. Nor trolls.) Definitely more in tune with the “true” Iceland than Available dark. (Well, as far as I can tell!) The mystery itself is a wee bit stretched and the final resolution slightly disappointing, implying some unlikely behaviour from the major characters. In particular, I do not buy the explanation motivating the arson itself. Terrible cover too. And not a great title in English (Watch me or Look at me would have been better) given the many books, movies and songs with the same title. Nonetheless, I liked very much the overall atmosphere of the book, enough to recommend it.

Filed under: Books, Travel Tagged: autism, Þóra Guðmundsdóttir, financial crisis, Horfðu á mig, Iceland, Reykjavik, Yrsa Sigurðardóttir
Categories: Bayesian Bloggers

an ISBA tee-shirt?!

Xian's Og - Fri, 2014-09-19 08:18

 Sonia Petrone announced today at BAYSM’14 that a competition was open for the design of an official ISBA tee-shirt! The deadline is October 15 and the designs are to be sent to Clara Grazian, currently at CEREMADE, Université Dauphine [that should be enough to guess her email!]. I will most certainly submit my mug design. And maybe find enough free time to design a fake eleven Paris with moustache tee-shirt. With Bayes’ [presumed] portrait of course…

Filed under: Kids, pictures, University life Tagged: BAYSM 2014, ISBA, Markov chain, mug, tee-shirt, werewolf
Categories: Bayesian Bloggers

Am Belvedere

Xian's Og - Thu, 2014-09-18 18:14
Categories: Bayesian Bloggers

up [and down] Pöstlingberg

Xian's Og - Thu, 2014-09-18 08:18

Early morning today, following my Linz guests’ advice, I went running towards the top of Pöstlingberg, a hill 250m over Linz and the Danube river. A perfect beacon thus avoiding wrong turns and extra-mileage, but still a wee climb on a steep path for the last part. The reward of the view from the top was definitely worth the [mild] effort and I even had enough time to enjoy a good Austrian breakfast before my ABC talk

Filed under: Mountains, pictures, Running, Travel, University life Tagged: Austria, breakfast, Danube, Donau, IFAS, Linz, montain running, Pöstlingberg, seminar, sunrise
Categories: Bayesian Bloggers

my life as a mixture [slides]

Xian's Og - Wed, 2014-09-17 18:14

Here are the slides of my talk today at the BAYSM’14 conference in Vienna. Mostly an overview of some of my papers on mixtures, with the most recent stuff…

Filed under: pictures, Statistics, Travel, University life Tagged: Austria, BAYSM 2014, church, mixtures, Neue Jesuitenkirche, slides, Universitätkirche, Vienna, Wien
Categories: Bayesian Bloggers

position at Warwick

Xian's Og - Wed, 2014-09-17 08:18

A new position for the of Professor Of Statistics and Data Science / Director of the [newly created] Warwick Data Science Institute has been posted. To quote from the job description, “the position arises from the Department of Statistics’ commitment, in collaboration with the Warwick Mathematics Institute and the Department of Computer Science, to a coherent methodological approach to the fundamentals of Data Science and the challenges of complex data sets (for example big data).”  The interview date is November 27, 2014. All details available here.

Filed under: Statistics, University life Tagged: academic position, big data, data science, England, job opening, professor of statistics, University of Warwick, Warwick Data Science Institute
Categories: Bayesian Bloggers