Statistical Modeling, Causal Inference, and Social Science

The property of learning a function parametrized by weights on examples is a general property of optimizing functionals which are the sum of an empirical loss functional and the norm in a Reproducing Kernel Hilbert Space. The SVM is one example, but there are many others. The business about the examples closest to the boundary, and actually the whole geometric conception of SVMs in terms of distance to the hyperplane ("margin"), is a bit of a red-herring. In practice, SVMs are nearly always used in a context where "errors" are allowed but penalized, and in this case, all points which are errors are also support vectors. In this framework, the nice geometric notion of the support vectors being objects closest to the boundary is lost --- there's still a boundary, but there can be errors which are arbitrarily close to it, and the margin is not well-defined. I prefer to think of the SVM as arising from a particular choice of loss function (the hinge loss) in a functional optimization problem.

Loved the ape quiz.

The complexity of the "statistical learning" approach is staggering and comparable to the scope of the statistical school. But a statistician would use the likelihood function instead of an empirical loss functional, the prior instead of a regularizer, probability of having generated the data instead of hinge loss. These tools are analogous, and differences may be irrelevant. RKHS is very nice, but it takes one or two lectures to explain it properly.

However, expressing the model in terms of weights assigned to cases is something that one doesn't see too often in statistics, and would be interesting to see more often.

I agree completely with you. Have you considered presenting something like gaussian process regression? It's fairly simple, it's the same equations as regularized least squares "under the covers", but it's a nice bayesian interpretation, it gives you a confidence interval on your outputs, AND the function you learn is expressed in terms of weights assigned to the observations, just like an SVM.

I'm going to ignore the serious statistical isues here and just focus on the fun stuff. I'm pleased to say that I got 100% correct on the Artist or Ape quiz, and got 83% correct on the True Art or Fake quiz. I'm ashamed to admit that I got exactly 50% on the Sokal and Bricmont or Lenin quiz.

But as far as telling us anything, two of these quizzes really don't.

(1) Artist or Ape is uninformative because I'm pretty sure the author of the quiz chose the Artist pictures that look _most_ like an ape might have drawn them, and chose the Ape pictures to look _most_ like an artist might have drawn them. This doesn't really tell us anything about whether modern artists draw like apes draw.

(2) True Art or Fake is uninformative because the "Fake" art was generated by someone who had seen the True Art and was deliberately trying to compose something that looked similar. The "skill" in drawing a Mondrian isn't in drawing the lines and coloring in the squares, it's conceiving of the idea of making a pattern like that out of lines and squares. I happen to not like Mondrian, but I give the guy some credit: until he came along, nobody was making (or at least selling or showing) art like that. Sure, NOW you can copy him and make something that looks pretty similar, but you're still copying him. If I stand exactly where Ansel Adams stood to take his famous Half Dome picture, and expose the film at the same time of day and develop and print it the same way, I'll have something that is nearly indistinguishable from an Ansel Adams picture, but I will not have demonstrated that Adams was as talentless as I am.

Sokal and Bricmont or Lenin seems more fair, though. It suffers somewhat from the same shortcoming as (1) above --- these are the S-B quotes that are most like Lenin quotes and vice versa --- but given that S-B is just one book about one subject, the fact that there are _any_ non-trivial S-B quotes that sound just like non-trivial Lenin quotes is already noteworthy.

If I were to try to put all of these thoughts into a scholarly statistical context, I would say that there is a strong selection bias in the quizzes. Using these results to say that artists paint like apes, or that modern artists are no more talented than non-artists, is like comparing the temperate of the warmest winter days and the coolest summer days and saying that winter temperatures are about the same as summer.

All of that said, the quizzes are a ton o' fun and I'll be looking for more.

The bias asserted by rif in the previous comment does not exist.

Sokal & Bricmont's views on philosophy of science are identical to those of Lenin.

Not all abstract paintings look like a work of an ape: many contain geometric figures, which apes are unable to draw. However, the paintings within the branch of abstract art, called "Abstract Expressionism", all look like they were produced by an ape.

Rif's comments about imitating Mondrian are irrelevant as I did not imitate him (or any other artist). Instead I presented the paintings creared using The new method in Abstract Art, invented by myself.

Regarding warmest winter and coolest summer days: in some places indeed there is no difference between the seasons of the year. In San Francisco you don't swim in summer and don't ski in winter. One can say that there is no winter or summer in San Francisco (just like in the San Francisco Museum of Modern Art there is no art).

Pretty interesting stuff, the use of dots inspired me to make a painting