Composite sampling and the safety of canned tuna

Seth forwarded me an article from the Chicago Tribune on mercury poisoning in fish. Near the end of the article, there’s a mini-debate about composite sampling, which brought back some memories, since I did a project in 1994 on composite sampling of fish.

I’ll post the relevant bits from my article and then give my comments.

FDA tests show risk in tuna
U.S. agency finds high mercury levels in some cans and in samples of Chilean sea bass

By Sam Roe and Michael Hawthorne
Tribune staff reporters

January 27, 2006

Newly released government data provide the best evidence to date that some cans of light tuna–one of America’s favorite seafoods–contain high levels of mercury.

Testing by the Food and Drug Administration found that 6 percent of canned light tuna samples contained large amounts of mercury, a toxic metal that can cause learning disabilities in children and neurological problems in adults.
. . .

In the 216 samples of canned light tuna tested by the FDA, the mercury levels averaged 0.12 parts per million, in line with previous limited testing and well below the legal limit of 1.0 parts per million. But 12 samples exceeded 0.35 parts per million, an amount the government considers high. When the Tribune recently tested 36 cans of the same type of canned tuna, none of the samples exceeded that level. The discrepancy might be due to the difference in sample size or because mercury levels can vary widely in all fish.

When asked about the FDA’s latest testing results on light tuna, an agency official said consumers should not be concerned that 6 percent of canned light tuna tested high in mercury. What’s important, the official said, is that on average, such tuna tested relatively low.
. . .

The U.S. Tuna Foundation, the industry’s leading lobbying group, said the FDA’s new data actually confirm the safety of canned light tuna.
. . .

“FDA’s latest findings about mercury levels in canned tuna should end the debate over whether canned tuna is a safe and healthy food for all Americans,” David Burney, the foundation’s executive director, said in a statement. “No one is at risk from the minute amounts of mercury in any form of canned tuna.”
. . .

In 2004, the FDA and the U.S. Environmental Protection Agency jointly warned high-risk consumers to eat no more than 6 ounces of albacore canned tuna per week because of high mercury levels.

Even if women of childbearing age and young children followed that suggestion, the EPA’s own calculations show they would absorb too much mercury.
. . .

Among the fish testing relatively low in mercury in the FDA’s latest round of tests was tilefish, a species the agency warns pregnant women and young children not to eat.

Previous testing in the Gulf of Mexico found high mercury levels in tilefish. The latest samples came from waters off the Atlantic Coast, raising questions about the reliability of the FDA’s consumer advice.
. . .

Just how much mercury might be in a single can of tuna is unclear. That is because the FDA does not test individual cans. Instead, it removes small pieces of tissue from 12 cans and mixes the tissue together. The agency then tests the mixture, masking any extreme amounts of mercury in a single can. This is done with other fish species as well.
. . .

In the FDA’s recent testing, one sample of light tuna showed mercury levels of 0.72 parts per million–a high amount but still within the 1.0 legal limit. But because this result was a composite of 12 cans, it is likely that some of the individual cans had higher levels.

It is impossible to know whether one of those cans tested over the legal limit.

The FDA said it tests a mixture of cans rather than individual cans partly to save money.

“It would cost 12 times as much to test 12 separate cans and then average the data, which is what we would have to do,” said the FDA official who requested anonymity.

That methodology troubles some doctors.

“I find that incredibly disturbing,” said Jane Hightower, a San Francisco internist who treats patients with mercury-related ailments. “That is falsifying data as far as I am concerned.”

The FDA’s data are here. It’s not just the tuna that has mercury. Don’t each the sharks! See here for the summary of the FDA data and here for their advice.

My thoughts (in no particular order):

1. My understanding of heavy-metal poisoning is that it’s the cumulative dose that matters, not the size of any particular dose. I might be wrong on this, but if I’m right, the FDA is right to just look at averages. The article correctly states that composite sampling is “masking any extreme amounts of mercury in a single can,” but that’s not a problem if the concern is total exposure.

2. Similarly, I don’t think it’s relevant to say that 6% of cans exceeded some threshold. What’s relevant is the average amount of mercury per can. If I’m eating a lot of tuna, it doesn’t matter whether I’m getting it all from one can, or if it’s evenly distributed in many cans.

3. I’m confused about the recommendations. Ignoring the irrelevant (but headline-producing) 6%, I see that, for all the cans tested, “the mercury levels averaged 0.12 parts per million, in line with previous limited testing and well below the legal limit of 1.0 parts per million.” So it sounds like the mercury is not a problem. But then later on it says, “the EPA’s own calculations show they would absorb too much mercury.” So what’s the scientific consensus?

4. There are some elaborate methods in the statistical literature for getting standard errors for estimates from composite sampling. The simplest and most reliable approach, however, is to just replicate the composite sampling, then use simple inference for the average of the composite samples. That’s what we did for my consulting project in 1994 analyzing fish sampled from a lake.

5. Maybe I’m missing something, but I can’t see how that doctor can say that composite sampling is “falsifying data.” And since when are doctors considered experts in statistics? Well, I guess that’s OK if a spokesman from the Tuna Association is considered an expert in medicine. . . .

The article is actually a little frustrating, because (as noted in point 3 above), there seems to be some ambiguity in the recommendations, but this just about gets lost amid the quotes giving extreme views of the issue.

But it was fun to see composite sampling in the news. Maybe next time they’ll quote a statistician . . .

P.S.

A commenter pointed out that there’s a question at Scatterbox about the tuna study. I don’t know enough about this to evaluate it, but as noted above, my first thought is that it makes more sense to look at averages than to look at the percent of cans that are above some threshold. And composite sampling is fine. What’s not clear to me are the esitmated risks of a given dose of mercury (or, I suppose, the health benefits of eating fish compared to whatever the alternative is for any particular person).

4 thoughts on “Composite sampling and the safety of canned tuna

  1. Totally agree with you. Adding to point 1… since when is sampling used to identify extreme points? If they need to be certain to find that "single can" with extreme value, then they need to test every can!

    Ironically, composite sampling is not even that bad if we were to test everything. If there is 1 bad can in 1 million, then the bad can will inflate the average value in its own 12-can batch but all other batches will have much lower average values, so we can at least narrow the culprit to 12 cans.

  2. Andrew, I agree with your line of reasoning. I fear that the good doctor does not understand the sampling issue or statistics in general.

  3. Andrew, in your #3, the first quote is about light tuna. The second quote seems to be about albacore tuna (and seems to say the EPA thinks the FDA's upper limit is too high).

Comments are closed.