“Love by numbers: maths professor’s formula for romantic success”

Alex Frankel sent in this:

A professor at Oxford University and his team have perfected a model whereby they can calculate whether the relationship will succeed. In a study of 700 couples, Professor James Murray, a maths expert, predicted the divorce rate with 94 per cent accuracy. His calculations were based on 15-minute conversations between couples who were asked to sit opposite each other in a room on their own and talk . . . Professor Murray and his colleagues recorded the conversations and awarded each husband and wife positive or negative points depending on what was said. Partners who showed affection, humour or happiness as they talked were given the maximum points, while those who displayed contempt or belligerence received the minimum. . . .

I looked up James Murray and couldn’t find any article describing these results; 94% accuracy sounds pretty good to me, but it’s difficult to make any comment based only on news reports. It appears, though, that Murray’s main home is the University of Washington, not Oxford–at least, there seems to be a lot more info on Murray at UW than at Oxford–and he’s cowritten a book on The Mathematics of Marriage, so this isn’t a new area for him.

There must be a bit of a discussion of this sort of thing in the clinical psychology literature? Perhaps this would be a good topic for teaching logistic regression forecasting, better than our usual boring examples.

One thing about the news report puzzled me, though; at the end, it says:

The forecast of who would get divorced in his study of 700 couples over 12 years was 100 per cent correct, he said. But “what reduced the accuracy of our predictions was those couples who we thought would stay married and unhappy actually ended up getting divorced”.

Huh?? If the accuracy was 100%, then what does he mean by “what reduced the accuracy of our predictions”? Were they hoping for 110%?

18 thoughts on ““Love by numbers: maths professor’s formula for romantic success”

  1. I think it means that everyone they predicted would get divorced did get divorced. Some of the people they thought would stay married got divorced, also.

  2. Professor: I think he's saying that every couple whom he thought would get a divorce did; but some couples whom he thought would stay together also got divorced. So "the forecast of who would get divorced" was accurate, but he was wrong about about "the couples who he thought would stay married and unhappy." I agree it's poorly phrased.

    Also, this sounds like some research mentioned in _Blink_, but I don't have a copy with me so I can't say anything more.

  3. Every couple Murray predicted would get divorced did get divorced, so that part of the forecast was 100% correct. However, some couples Murray said would stay together were divorced, which lowered the overall accuracy.

  4. Murray describes some modeling of this sort in chapter 5 of his book Mathematical Biology, and you can view most of it online. I'm not sure if the methods used for the recent results are different but I believe they were reporting similar accuracy numbers with what's in the book.

  5. @Jadagul: Yes, it's a continuation of the research described in _Blink_, just a new study from the same people, near as I can tell.

    Murray and Gottman have been criticized before for reporting the success rate of their model on the training dataset rather than doing something like cross-validation. From the wording of his quotes in the article, it sounds like they did a real predictive study this time, but the Life & Style section of the Sydney Morning Herald is not my preferred source for science news.

    Huh. Looks like there's an R implementation of their model, too.

  6. Who knows what this might mean. You are being too logical. He might be predicting a 40% divorce rate overall (not by couple) and have a 6% error against that rate — hence "predicted the divorce rate with 94 per cent accuracy".

    Before you say "that's stupid", remember they've already got the guy moved to Oxford — fact checking might not be a high priority.

  7. I saw Murray speak a couple times at UW – he's also done some interesting studies with same sex couples (where he claims the prediction is even higher than heterosexual couples). I haven't seen publications yet about this yet…

  8. The book Blink by Malcolm Gladwell talks about something similar. Chapter one talks about a psychologist John Gottman in University of Washington who worked in this area. When I checked his profile it looks like James Murray worked with him.

  9. I actually didn't have an opinion about the study itself — it sounded fishy, but I couldn't find more details about their methods. (In particular, what percent of couples stay together in their sample? 75% maybe? Then guessing that no couple will get divorced gives you 75% accuracy right away. Now if you take any correlates at all of marital happiness and run the regression in-sample, as Robert Kern suggests they might have done, you're going to be pretty accurate…).

    The reason I sent it in was because I thought this description of the scientific process was funny:

    Professor Murray, a fellow of the Royal Society in Britain, said the scores were fed into a mathematical model and plotted on a graph. The point at which the lines met illustrated the chance of success or failure.

    By the way, Hal — and I point this out only because you're the chief economist of Google — you might want to use html tags for your links. :)

  10. It is poorly phrased, but from a signal detection point of view, we would say that the false alarm rate for predicting divorces was zero but the hit rate was somewhat less than 1.0.

    Which is why psychologists who have studied psychophysics never (or should never) quote an "accuracy" value in isolation. The performance of a detector can only be characterised by pairs of hit/false alarm rates or by a single true measure of sensitivity such as d prime or area-under-the-(ROC)-curve. 'Accuracy' is inherently ambiguous and should generally be avoided.

  11. Think of it in terms of sensitivity and specificity instead of overall accuracy rate and it makes a lot of sense.

    Sens: # predicted to get divorced/# who actually got divorced (True Pos./(TruePos+FalseNeg))
    Spec # predicted to stay married/#who actually stayed married. (True Neg/(True Neg+False Pos))

    Since the terms vary independantly it is actually possible to have 1 term be 100% and the other something abysmal like 50%….

  12. This is in the news because Murray was awarded the Royal Society Bakerian prize "for his groundbreaking work in mathematical biology" and gave his prize lecture last week. In his lecture he talked about this work on marital interaction. The Telegraph picked this up and ran a story on it. But after the Sydney Morning Herald had edited the story, these details had been dropped, and the whole thing became a human interest/scientists-develop-magic-formula-story, of the kind that the press seems to love.

    Anyway, a webcast of the lecture should eventually be available for download from The Royal Society web site

  13. I have the unfortunate vision of a news story focusing on that last claim and summarizing the whole thing as, "Scientists find it is easier to tell which couples will definitely break up than which will definitely stay together." I am getting cynical after seeing too many versions of, "Oh, those silly academics, researching what everyone already knows."

  14. @Alex Frankel: Actually, that description is actually a fairly accurate, if missing all of the important details, description of the specific model they use. Murray and Gottman fit the kinematics of a conversation between the couple to a dynamical model for each person. The model can be viewed graphically as curves in RR^2. The intersections of the two curves are supposedly the equilibria the couple's conversations inevitably gravitate towards. The theory roughly goes that if these equilibria are where both partners are happy, the model predicts they will stay together. If they are both unhappy at the equilibria, then it predicts they will get divorced.

Comments are closed.