Skip to main content

"Any idiot can do a study and make it show whatever they want."

31 July 2022

34% of adults in the United States who have tried to lose weight have tried at least one supplement to help. As these are sold as supplements, and sold over the counter, they are not subject to regulation or testing by the FDA. This allows the companies to utilize any products they want, regardless of whether they have ever been tested in a controlled environment to show they help with weight loss.

Due to the robust and growing weight loss market, more and more companies are popping up promising a miracle cure to obesity. Whether it’s using electricity, lasers, natural supplements derived from plants or compounds that work on our hormones, they all promise the same thing – lose weight fast without significant dieting or exercise.

However, as we all learn early on … if something sounds too good to be true it probably is. Let’s start with broad brush strokes. But first, an introduction. I’m Dr. Chris Dietrich, and I’m a teaching physician with an internal medicine residency program. Think of Scrubs or Grey’s Anatomy – I’m one of the attending physicians. This means my residents come to me every day with questions. Some I know the answer to, and some I don’t. When I don’t know the answer, I need to look it up. But I need to make sure what I look up is accurate and correct, as a generation of future doctors is going to take what I tell them and use it to treat patients for the remainder of their careers.

This is where the critical appraisal of literature comes in. When I was a kid, I hated brushing my teeth. The two minutes seemed like an eternity and I had other things to do with my time. I remember saying to my dad “there was a study in the paper that said you only have to brush your teeth for 30 seconds!”   My father was a scientist, so I was hopeful this would convince my father to let me brush my teeth less. Instead, he told me “Any idiot can do a study and make it show whatever they want.” I wouldn’t know how true this was until I started medical school.

There’s big bucks in alternative medicine, and in order to make big money companies need to make sure whatever they’re selling at least looks like it works. As such, companies will build studies to show their drugs work (even if they don’t). This is even worse in the supplement market – these studies tend to be published in journals without real peer review.  They essentially allow the supplement maker to pay to have their study published showing their weight loss drug is a magical cure all. They then tout this data to their customers to justify charging them hundreds of dollars a month.

How do they do this? Imagine for a second I want to prove Tic Tacs cause people to lose weight. So I recruit 50 people, 25 of which are going to eat Tic Tacs, and 25 of which aren’t. Then I watch them over the next 3 months to see who lost weight.  However, in order to make sure my study shows what I want it to, I have the people who are eating Tic Tacs also only eat 1200 calories a day (Tic Tacs included). The other 25 people I will tell to keep eating whatever they want.

Normally this would be caught by peer review when I try to publish, but there are journals out there that are willing to look past that for some extra money. Now I have a study that says “People who ate Tic Tacs every day for 3 months lost 25 pounds compared to people who didn’t!” Suddenly Tic Tacs are the next big thing in weight loss! And if you think to yourself – “well I would see that the people eating Tic Tacs were also eating way fewer calories” – when was the last time you read the study of the weight loss supplement you were taking?

In medical school we learn how to see through these poorly designed studies. You look at their data and what they did to manipulate it. And we also learn what journals and types of studies are more trustworthy. So now whenever someone asks me about a drug or treatment, I always look at studies through this lens of “critical appraisal”.

Back to the broad brush strokes – in 2021 a group of doctors looked at 315 studies identifying supplements used for weight loss. Of these, only 16 actually showed weight loss. Of the 16, the degree of weight loss ranged from less than one pound to ten pounds during the study. The authors note many of the studies were paid for by the supplement maker, or were of a very small group of people or over a very short period of time. When this occurs, the information the study shows is much more likely to be due to chance, and not because of the effect of the supplement.

Next let’s look at specific claims of a supplement (which costs well over $50 a month, and they suggest be taken in conjunction with other supplements which also cost over $50/month).

Estro-One – this is marketed as an estrogen blocking supplement to help men lose weight. The theory is that estrogen causes men to gain fat and have a hard time losing it. But this isn’t the truth! At least not the whole truth. Fat tissue is estrogenic – meaning it actually creates estrogen. Men and women with obesity tend to have higher estrogen levels than non obese people. It’s hard to say the estrogen is what causes you to be obese when it’s the fat tissue making the estrogen.

And this isn’t just a “Which came first, the chicken or the egg?” scenario. In the 1990’s, there was a group of men who had a genetic abnormality causing them to not make estrogen. These men were found to have increased central obesity, insulin resistance (leading to diabetes) and increased incidence of non-alcoholic fatty liver disease. Because they DIDN’T have estrogen.

In the early 2000’s scientists set out to prove it was in fact lack of estrogen that caused these men to be obese. To do this they took a group of male mice, and in half of them changed their genetics so they would not make estrogen. These male mice exhibited increased central adiposity that became more pronounced with age, to the point by the time the mice were older they had twice as much fat tissue as their estrogen normal counterparts.

Male mice with an aromatase deficiency, which inhibits the conversion of testosterone to estrogen, were also found to have more fat tissue and actually had decreased physical activity and glucose oxidation. These mice actually made more fatty acids in their livers, had higher insulin resistance levels and higher incidence of fatty liver disease. This is important as there are several weight loss supplements on the market that tout they “inhibit the conversion of testosterone to estrogen”. 

This was legitimately the first supplement I came across for weight loss, and the very idea of how it works is flawed. It sounds good in theory – as a man I don’t want estrogen. But it isn’t true. I did a cursory search into several other supplements, and came up with similar flaws for every one of them. Every. Single. One. Some utilize chemicals or compounds which in theory may work to help people lose weight, but it’s never been proven to actually work in humans over extended periods of time. That doesn’t mean it couldn’t work, but it does mean it hasn’t been proven to work. 

So, before you waste money on the next miracle weight loss cure, consider talking to your doctor about it first. They can look to see if there is any evidence it actually works, or even if the theory behind how it works makes sense. There are products out there that can help you lose weight, but they need a prescription pad and a doctor’s approval to be prescribed. This is why I always recommend people see a board-certified weight loss specialist to help them lose weight.