Tag Archives: study

Scientific Studies Should Be Taken With A Giant Grain Of Salt

Posted by apt46 on May 14, 2012 None comments

An article in Nature last week brought up a fact that’s been getting more and more attention in the past decade: most published research findings are false. It all started when a paper with that very title was published in 2005, containing mathematical proof showing that it’s very easy for a study to be wrong, due to three main reasons:

  • Researcher bias: this can be confirmation bias, where the researcher, maybe even unconsciously, sets the experiment up to succeed out of fear of bring proven wrong; there are plenty of ways to massage every experiment, or to interpret the numbers, so that the outcome is the one hoped for. The bias can also be incentive-based, in which the researcher skews the results because of what’s at stake: a good job, grant money, a promising career, or even prestige and scientific stardom.
  • Selective reporting: the vast majority of studies published have positive results — that is, they confirmed what the researcher was looking for, e.g. that cell phones cause brain cancer. But if the research shows no effect from cell phones, it doesn’t get sent in for publication. And even if it does, the journal may decide to not publish because it’s not sexy enough.
  • Poor study design: maybe the samples were too small, the study wasn’t double-blinded or not randomized, it tested too many relationships, or relied on self-reporting; it could be the question was too broad, the results were mis-analyzed, or any of dozens of possible flaws that can make its way into a study, usually due to constraints on time or money.

John Ioannidis

 

The paper became the most downloaded article in the PLoS Medicine journal. The author, John Ioannidis, was the subject of an in-depth profile by The Atlantic in 2010. He has made it his mission in life to root out bad studies and prove them wrong. From the article, emphasis added:

Ioannidis laid out a detailed mathematical proof that, assuming modest levels of researcher bias, typically imperfect research techniques, and the well-known tendency to focus on exciting rather than highly plausible theories, researchers will come up with wrong findings most of the time. Simply put, if you’re attracted to ideas that have a good chance of being wrong, and if you’re motivated to prove them right, and if you have a little wiggle room in how you assemble the evidence, you’ll probably succeed in proving wrong theories right. His model predicted, in different fields of medical research, rates of wrongness roughly corresponding to the observed rates at which findings were later convincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials. (The Atlantic, November 2010)

Ioannidis then published a study in the Journal of the American Medical Association, which confirmed his predictions: 22% of the 49 most widely cited medical studies in the most widely cited journals were never even replicated; and 29% of the ones that were, were proven wrong. Therefore, his advice is to largely ignore studies: besides the fact that they often contradict each other, there are so many factors at play in something as complex as the human body, that researchers most often find flukes in large, but limited data sets — not actual facts. It’s somewhat like claiming words in a giant bowl of alphabet soup have some significance.

 

But even if researchers stumble onto something significant, the study can only predict effects in a test environment, not in the real world. And very few studies go on for long enough to see if a factor affects important things, like death rate. (The few that do, generally contradict shorter studies.) Then, even in a magical unicorn of a perfect study, the results are averages over hundreds or thousands of people and are not even remotely tailored to our individual needs. And finally, even if you are the unlikely beneficiary of a perfect study done on a subset of the population that you belong to, the effects found are generally meager. In fact, we can’t even be sure that a study hasn’t already been refuted: sometimes it takes over ten years for researchers to stop citing a study which was proven to be wrong. For example, people still think plastic water bottles leech toxic chemicals, even though the original study was severely flawed.

“The odds that anything useful will survive from any of these studies are poor,” says Ioannidis—dismissing in a breath a good chunk of the research into which we sink about $100 billion a year in the United States alone. (The Atlantic, November 2010)

And these problems exist in all fields of research, not just in bio-medicine. The New Yorker also interviewed Ioannidis in 2010, as part of an article on a related problem in scientific research, known as the Decline Effect. A very curious phenomenon, it makes significant findings of studies disappear over time — and it happens a lot. A study will find a drug very effective, then a subsequent study will find it less effective, and a third one even less so; the effect is generally explained as flukes being worked out of the studies over time, known as regression toward the mean.

 

It has even been shown that all economic studies might be wrong, and throughout history, instances where reality didn’t agree with the laws of physics have caused our understanding of the universe to become more and more complex — from the Earth sitting on the back of a turtle, to being suspended in the void by a force we can measure, but do not quite understand. And after centuries of discovering the laws of physics and building the complex language of advanced mathematics to describe them, scientists now faced with the similar feat of discovering the laws that govern the other important and extremely complex systems: our bodies, our environment and our society.

All of the evidence points to us not even being close to understanding those systems. But eventually, we will get to the same level of clarity about them that we have about classical physics. Until then, Ioannidis came up with some rules for judging truthiness:

A study is more likely to be wrong if

  • it is small
  • the effect sizes are small
  • there are large financial interests, or prejudices
  • the scientific field is flexible with respect to study design
  • the scientific field is teeming with competing teams

See also:

 

From Nature, PLoS Medicine, Journal of the American Medical Association, The New Yorker and The Atlantic

Jogging 3x A Week Helps You Live 6 Years Longer

Posted by apt46 on May 7, 2012 None comments

The Copenhagen City Heart Study was started in 1976 and tracks about 20,000 people in that city in order to learn more about heart disease and other health issues. After 35 years, there’s a lot of data there, so the reseearchers decided to crunch it and find out if jogging really is good for you. They looked at about 2,000 joggers at various periods within the 35 years, recorded the amount of time they jog per week and the intensity they do it with, and compared their age at death with that of non-joggers. On average, jogging men lived 6.2 years longer than the non-jogging ones, and the women lived 5.6 years longer.

 

They also tried to figure out the best jogging regimen: the data shows that running three times a week for about 35 minutes (+/- 15) had the optimum correlation with life expectancy.

“The relationship appears much like alcohol intakes. Mortality is lower in people reporting moderate jogging, than in non-joggers or those undertaking extreme levels of exercise,” said [chief cardiologist of the Copenhagen City Heart Study, Peter] Schnohr. The ideal pace can be achieved by striving to feel a little breathless. “You should aim to feel a little breathless, but not very breathless,” he advised.

Jogging, like exercise in general, does all kinds of good things for the body: boosts the immune system, makes the heart work more efficiently, lowers blood pressure, makes the body more sensitive to insulin, prevents cancer, migraines, the weakness of old age, and Alzheimer’s among other things. Therefore, even though the joggers probably took better care of themselves in other ways too (e.g., diet, not smoking), it’s clear that there is causation at play here, not just correlation.

See also:

 

From European Society of Cardiology, via The Atlantic

Laziness Correlated With Alzheimer’s

Posted by apt46 on April 19, 2012 None comments

A study used Philips’ Actical device to measure the amount of energy 700 old people (average age of 82) expended. Over the course of the study, about 10% of those people developed clinical Alzheimer’s disease. The ones that did develop Alzheimer’s tended to consume less energy per day. And the lazier they were, the more demented they became. Sloth: just one of the deadly sins.

Sloth in the Amazon. Photo by Carol Schaffer

 

In related news, Nike makes a very cool energy-measuring device, called Fuelband, that looks like a futuristic watch, and it works with your iPhone. It has a “fuel” display that goes up throughout the day as you use more energy, so you can easily tell if you’ve been too lazy. If your fuel display is down in the red, you’re not doing well. It sells for 150$ in the Nike Store.

Nike Fuelband

From Neurology, via NPR

Diet Sodas Definitely Aren’t Good For You

Posted by apt46 on April 13, 2012 None comments

NPR highlights three studies on diet sodas, none of which do them any favors:

  • A study on how diet sodas relate to metabolic syndrome, which is generally found in fat people and causes heart disease and diabetes. The researchers created three groups and measured how many of them developed the disorder. The first group drank diet sodas and had an awful diet (think McDonald’s all the time). The second group drank diet sodas and were on a healthy diet (fruit, fish, nuts, veggies). The third group didn’t drink sodas at all and were on the healthy diet, too. Metabolic syndrome was highest in the diet soda + bad food group, followed by diet soda + healthy food one, and lowest in the no soda + healthy food collective.
  • A study on how weight change relates to various lifestyle factors, including amount of exercise, of TV watched and various foods eaten, found that diet sodas didn’t affect people’s weight. The study did find out factors that correlated with people losing weight: exercising and eating healthy (fruits, veggies, whole grains, nuts and yogurt). They also found out what factors correlated with people gaining weight: watching TV, drinking, smoking, sleeping too little or too much, and a poor diet (potatoes, potato chips, sugary drinks, processed meat, and red meat).
  • Just to conflict with the one above, another study showed that people who drank diet sodas gained more weight than ones who didn’t.

But no studies showed that there’s anything diet about diet sodas. They either do nothing, or make you fat. What all of this sums up to is diet soda being at best a crutch that doesn’t help your health; more likely though, it’s a crutch that slowly kills you. Just stick to water.

 

From The American Journal of Clinical Nutrition, The New England Journal of Medicine, via NPR and CBS News

Interesting Divorce Statistics

Posted by apt46 on April 9, 2012 None comments

Mental Floss dug up some data on what circumstances are correlated with either higher- or lower-than-normal divorce rates. Of course, correlation is not causation, so just because something is related to a higher divorce rate, doesn’t mean that it causes divorce. Still, they’re interesting to consider:

  • Smoking: couples where one person smokes are almost twice as likely to get divorced. It’s a little higher if the smoker is the wife. And even when both spouses are smokers, the divorce rate is still higher than non-smokers; however, this may have a common cause, since poorer people smoke more and are also more likely to get divorced.
  • Jobs: some professions have lower divorce rates than normal. Optometrists, shuttle car drivers, transit cops, farmers, nuclear engineers and clergy. Again, this is probably just due to the kind of people that choose those professions, and the spouses that marry them — the jobs themselves don’t really make them less divorce-prone. Massage therapists and mathematicians were among the most likely to get divorced.
  • Generosity: couples that split chores evenly tend to have a lower divorce rate. This was also corroborated by the National Marriage Project, which showed that generosity was the 3rd biggest predictor of a happy marriage.
  • Ideology: conservative states have a higher rate of divorce than progressive ones. This is probably due a few factors, including the average level of education and marriage age being lower in conservative states: people there get married younger, then probably grow apart as they age, and don’t have coping skills like good communication techniques to work problems out. It’s worth noting that while some liberal states do have the lowest rates of divorce, others have the highest likelihood of marriages ending in divorce.
  • Influence: people who know more divorced couples tend to get divorced more. The social effect is very powerful and its presence can be seen in a lot of other problems, such as smoking and obesity: people tend to be like their friends and family. When it comes to people, birds of a feather don’t necessarily flock together, but rather those who flock together start growing the same feathers.
  • Beginnings: couples who met during high school or college are more likely to stay together; those who met in bars are more likely to get divorced. This likely has to do with shared history, values and other things the spouses should have in common.
  • Kids: couples who have more daughters have a higher rate of divorce; couples who have more sons have a lower rate.
  • Wife: 73% of divorces are initiated by the wife.

 

See also:

 

From Mental Floss, via Neatorama

More Evidence For The Toxicity Of Sugar

Posted by apt46 on April 3, 2012 None comments

Almost a year ago, The New York Times reported on Dr. Robert Lustig’s theory that the amounts of sugar present in the typical Western diet is toxic, because it causes heart disease, type 2 diabetes, obesity, high blood pressure, metabolic syndrome and probably cancer. Since then, it’s been confirmed that sugar consumption does indeed raise the risk of heart disease, and Lustig and his team have begun lobbying that the government regulate sugar like it does alcohol.

The heart of Lustig’s theory is that in nature, sugar is locked away inside fibrous fruit, which makes it impossible for us to eat too many sweets. But in these modern times, through the wonder of technology, we can cheaply extract the sugar like heroin from poppy, then add it to everything under the sun because it tastes good and acts like a preservative: drinks, desserts, bread, sauce, peanut butter, etc, etc, ad nauseam. As a result, the amount of hidden sugar we actually eat is so heavily disguised, that we don’t even notice the raw quantities we eat — quantities that would make us sick in the form of table sugar, and quantities that would be physically impossible to eat solely from fruit.

This past Sunday, 60 Minutes got into the game too, with a segment featuring Dr. Lustig and other scientists, all telling us why they’ve quit eating added sugar:

  • A study at UC Davis showed that within two weeks of eating 25% of their calories in the form of high-fructose corn syrup (which is the same thing as sugar), subjects had increased levels of LDL cholesterol and were are higher risk for plaque in their arteries, as well as heart attacks.
  • A Harvard professor and biochemist explains how eating sugar increases the risk of cancer: a third of common cancers have insulin receptors; eating sugar causes insulin to spike, which in turn is ingested by the receptors on tumors, which fuels the tumors and causes them to grow
  • A neuroscientist shows, via fMRI brain scans, how sugar activates reward centers in the brain in the same way that drugs like alcohol and cocaine do, which makes it very addictive. As with those other drugs, people also develop a tolerance to it, and need more and more to get the same pleasure from eating it.
  • A spokesman for the sugar industry is skeptical, and says the science “is not completely clear”.

See also:

From 60 Minutes, via a vigilant reader

Cleanliness Is Next To Sickliness

Posted by apt46 on March 30, 2012 None comments

When germ theory firmly took hold in the public mind some 150 years ago, hindsight kicked into high gear: of course everyone was sick and dying, catching diseases from all the filth in which humanity was immersed, ripe with bacteria from all manner of rodents, vermin and dirty, sickly people. Ergo, if dirt made you sick, lack of dirt kept you healthy. Of course, the concept wasn’t new: religions had commanded cleanliness for centuries, and John Wesley, the founder of Methodism, preached that cleanliness was next Godliness in the 18th century. Until science brought it home though, cleanliness was seen as kind of like praying: good, but not necessary outside of church. This paradigm shift brought with it all manner of life-saving cleaning measures: soap, detergent, toothpaste, bleach and ammonia — all popularized around the beginning of the 20th century.

19th Century Babbitt's Soap Advertisement

 

And so the pendulum swung the other way: it was much easier to be clean, and being clean became the norm. In the middle of the 20th century, urban enclaves — once cesspools that served as fodder for plague – became cleaner than the sparsely populated farms their inhabitants had always envied. Fast forward a few more decades and we find ourselves in the meticulously clean present in which each household has a handful of cleaning agents under the sink, anti-bacterial soap is ubiquitous, and Purell is in every proper lady’s handbag. Along with that, however, came the rise of chronic diseases of the immune system, which, while very rare in farm-dwellers and less-developed (meaning, less-clean) countries, abound in city folk: asthma, allergies, intestinal problems, and autoimmunity.

Buy the Lysol to clean, then the Benadryl for the side effects. Buy the pizza and Coke to enjoy, then the gym membership for the side effects. Selling the poison and its antidote: a cornerstone of the modern economy.

 

Scientists weren’t quite sure why that was, until recently, when a study found out that bacteria in our intestines keep iNKT cells of the immune system from getting too vigilant, which if they do, leads to the chronic diseases mentioned above. And so, a century later, after realizing they overstated the whole cleanliness issue, scientists swing the pendulum back the other way: some exposure to dirt is good for you, especially as a kid. To summarize:

  • modern, germ-free environment: not good
  • 19th century urban filth: bad
  • moderately clean farm: good

So stop using Purell and anti-bacterial soap, enforce the five-second rule (or even lengthen it), and take up dirty activities like gardening or rugby. Pretend you’re on a farm, get back in touch with nature and re-learn to tolerate some amount of dirt: it will help keep your immune system functioning properly.

See also:

From Science, via NPR

Negative Facebookers Have Low Self-Esteem

Posted by apt46 on March 27, 2012 None comments

This makes a lot of sense in hindsight, but people whose status updates on Facebook often sound like this, probably have low self-esteem:

“Some people are just such liars!”

“The only heaven I’ll know is if my friends remember me when I die.”

“‘In a real dark night of the soul, it is always three o’clock in the morning, day after day.’ — F. Scott Fitzgerald”

“Finally getting over this cold. Maybe I’ll actually live through it.”

“People who criticize you behind your back are not your friends, they’re a waste of good air.”

It is known that people with low-self esteem are pessimists who complain more than normal, have negative views on life, are very cautious, and seek approval from others. Because of that last trait, researchers thought that maybe such people censored themselves on Facebook and tried to present a more likable image. Armed with a testable hypothesis and maybe some grant money, they obtained almost 1800 status updates from volunteers and also had them fill out a self-esteem questionnaire. Then they had another group of people rate the status messages based on how positive or negative they were. The status ratings were analyzed against their authors’ self-esteem and it turned out the lower the self-esteem, the more negative the statuses. So if they do censor themselves, they’re not doing a good job — which should come  as no surprise to people with poor self-image, since they aren’t good at anything, and are also ugly.

 

More interestingly, this creates a feedback loop: the people rating the negative statuses didn’t want to get to know the people writing them. Researchers even asked friends of the Debbie Downers if they liked the negative updates, to see if maybe they did just out of empathy for their friend: nope. Therefore, people with low self-esteem tend to think life sucks, which causes people to not want to hang out with them, which causes them to be lonely and think life sucks even more.

Moral of the story: if you want people’s approval, negativity should be very much the exception, not the rule. Also, no one likes you.

 

 

From Psychological Science, via NPR

More Drilling Doesn’t Make Gas Cheaper

Posted by apt46 on March 23, 2012 2 comments

This is something that’s been known for decades: the oil market is a global one, and its price is also global. If a barrel of oil costs 100$, that is its price in the US, in Britain, Angola, Japan, Iran, etc. And that price is determined by the market forces of supply and demand: if people want more oil than there is, the price goes up; if there’s more oil than demand, the price goes down. But really, there is much more oil in the ground than we need, so the limiting factor is production: how much of it is pumped out of the ground and turned into gasoline, kerosene and diesel.  Therefore, like that of diamonds, the supply of oil is artificially controlled and companies can ramp production up or down as they see fit, based on the market price of oil. OPEC is notorious for doing this: the 12-country cartel holds 79% of the world’s oil reserves and is responsible for 44% of world oil production. Due to their large share of the pie, they can effectively control the price of oil — and in turn, the price of gas — simply by producing more or less oil.

OPEC countries

 

American oil production, on the other hand, accounts for only about 9% of the world’s supply. Because even double the production would still be dwarfed by OPEC, the price of gas cannot be affected much by American supply. This fact is evident via statistical data that shows gas prices being unrelated to how much oil the US produces. For example, while US production increased by large amounts in the past three years, gas prices have not only not gone down, but almost doubled from 2.10$ to 3.58$. Therefore, expanded drilling — be it in the Gulf of Mexico, Alaska or North Dakota — will only result in more profits for (American) oil companies, not cheaper prices for consumers. In fact, the American drilling boom in the past several years has only happened because the (global) price of oil is now so high, that it’s worth trying to get it out of the bottom of the ocean, the tundra, or rocks.

Politicians pretend they can lower gas prices by expanding drilling: George W. Bush was for it; Sarah Palin made “drill baby drill” famous; Barrack Obama is also for it, despite the BP oil spill. The truth is, the government could easily lower the price of oil by reducing the gas tax, but that money goes to repairing the roads on which gas is consumed. Otherwise, their hands are tied. Yes, expanded drilling will increase our energy independence, but that just means we’ll still have gas if OPEC decides to stop exporting; it doesn’t mean that gas won’t cost 10$/gallon.

 

However, increasing supply isn’t the only way to reduce gas prices: reducing demand will have the same effect. The US consumes about a quarter of the world’s oil, or 20 million barrels per day. That is by far the highest rate: the next country is China, who only consumes 7 million barrels per day. This means America has a lot more influence as an oil consumer than an oil producer: while we can’t flood the market with oil to make gas cheaper, we can buy more hybrids and build more nuclear power plants, to achieve the same effect.

See also:

 

From Minnesota Public Radio, via Slashdot

Overheating Is A Serious Barrier To Exercise

Posted by apt46 on March 14, 2012 None comments

A study done by Stanford had two dozen middle-aged obese women walk on a treadmill for a mile and a half. Half the women got their bodies cooled by sticking their palms in a device that runs ice water through itself — kind of like a car radiator cools the engine. The other half also had their hands in the device, but water at room temperature ran through it. After three months: the cool women shaved five minutes off their walk, lost 3″ off their waist, and were more likely to not quit the study early.

 

Avacore Rapid Thermal Exchange

 

Walking 1.5 miles should take somewhere around 27 minutes, give or take, so shaving off five minutes is a pretty big deal: almost 20% faster; three inches thinner is also nothing to sneeze at. The NPR article on the study implies that overheating is mostly a psychological problem, possibly because if you overheat too much you’ll likely just faint. Elite athletes also overheat, but they push through it. Novices on the other hand, especially fat ones, think it’s the end of the world. It also doesn’t help that fat is an insulator, so the feeling of overheating gets worse the more of it there is.

The bottom line is that psychological or not, overheating is a significant deterrent to working out, and the more weight you need to lose, the more of a deterrent it is. As such, it’s important that measures are taken against overheating, because the less barriers there are in front of exercising, the better. The ice-water device used in the study is a non-starter because it costs 4,000$. But there are less expensive workarounds:

  • When working out, always wear clothing made out of moisture-wicking technical fiber. These fabrics use capillary action to make the sweat evaporate faster, which helps you sweat more, which helps to cool down more.
  • Run outside when it’s cool (around sunrise, after sunsent, in the winter) or work out indoors, with serious air conditioning
  • Put a lot of ice in your water bottle
  • Hold something cold in your hand and touch it to your face when overheating — ice pack, bottle of ice water, etc
  • Go swimming instead of running: the 80° pool water is like a giant cooler, and swimming is one of the best forms of exercising due to the use of the whole body and its low-impact nature
  • Go biking instead of running: even when it’s hot out, the breeze generated at 14mph does a lot to both cool you down and evaporate sweat; and it’s also a low-impact sport

Via NPR