Is drinking coffee good for you? Is there a link between eating red meat and breast cancer? Does a vitamin D deficiency put men at risk for prostate cancer? Reports on new medical studies come at you all the time, and frequently they contradict one another. How can you gauge which new treatments or basic scientific discoveries might help you or a loved one?
The bad news is there are few hard and fast rules for judging the validity and value of medical studies, says Charis Eng, M.D., Ph.D., Chair of the Cleveland Clinic’s Genomic Medicine Institute. And even if you do your homework, there are no guarantees that the results reported will work on you.
The good news is you can take steps to increase the chances of connecting with new science that can help. While there’s little that’s black and white, understanding some basics can increase your odds of finding a valid study that might hit home.
"When you see something in the press, stop and be critical," says Dr. Eng, who is also Director of the Cleveland Clinic’s Center for Personalized Genetic Healthcare. "Be critical. Be open minded. Ask questions of your friendly professional."
Dr. Eng offers these tips:
Go to the source. If results of a new study sound promising to your situation, don’t rely on reports in the popular press. Several scientific journals issue news releases on the studies they publish that are edited for accuracy by the studies’ lead researchers. They’re available on the journals’ Web sites and generally in Internet searches as well, and they’re written in language that laypersons can understand.
Reading a study itself may be difficult. If you don’t understand it, ask your physician for a translation.
- Remember: replication, replication, replication. "If the study’s new, that’s great for science," Dr. Eng says. "But replication is vital if you’re making that leap to the bedside." The result reported in the first test of a drug might be just a coincidence. Look for confirmation and reconfirmation in subsequent studies, preferably by different sets of researchers. One large-scale replication is usually sufficient.
Consider the reputations. Do the researchers have a good track record of publishing in this or related fields? Have the researchers been editors of journals? Have they been invited to write clinical practice guidelines? This information might be difficult for laypersons to assess. A search using Google Scholar (scholar.google.com) or PubMed (pubmed.gov) might help.
Less important, says Dr. Eng, is the reputation of the institution hosting the study. Big-name institutions can have researchers who are "cowboys." On the other hand, lesser known institutions might specialize in a particular field.
Similarly, top medical journals have published inaccurate reports, and many reputable journals are not widely known, Dr. Eng says. Look for studies that are reported in peer-reviewed journals.
Take comfort in numbers — usually. Was the treatment tested on a large number of people? "The bigger the better," says Dr. Eng. "That’s a good rule of thumb." With smaller numbers, the results are more likely due to chance.
But don’t automatically rule out a small study. If it confirms the findings of two larger studies, it can be the replication needed to move the treatment to clinical use. And a small study might be valid in other situations as well (e.g., if researchers find a genetic alteration in all members of five families who have a specific disease, but not in healthy members of those families or in a control group of the general population).
- Check the time. By the same token, don’t automatically rule out short-term studies. The length of time that should be spent on a study depends on the medical condition at issue. A new treatment that’s still working after nine months on a patient with a disease that usually kills in three months might be viewed as a remarkable advance.
Consider the methodology. Although other study methods exist, the standard for therapeutic trials consists of three phases, according to Dr. Eng. In Phase 1, researchers determine whether the drug is safe and how high a dose can be administered without side effects. In Phase 2, the response rate to this dose is measured on patients who have not responded to standard therapy. Phase 3 is a randomized controlled trial, in which patients are randomly assigned to one group that takes the drug being tested or to a control group that takes standard therapy or a placebo.
In a double-blind study, neither the patient nor the doctor knows which group the patient is in. It can be important if the doctor is the only professional evaluating the responses or if the patient has to report subjective reactions. If test results are measured by specialists who don’t interact with the patients, a double-blind format is not so important, Dr. Eng says.
A randomized controlled trial that was conducted at many centers is more likely to have valid results than the same study conducted in only one center.
- Look at the populations tested. Sometimes it doesn’t matter, but sometimes it’s important that the characteristics of the study subjects are similar to yours or your loved one’s. If you are a 32-year-old Ohio woman whose ancestors came from Europe, research conducted on elderly Asian men might not apply.
- Follow the money. Does the researcher stand to profit financially from the drug or device being studied? Is the study sponsored by the pharmaceutical company that developed the drug? Dr. Eng says consumers should be aware of financial ties, but that a study can still be valid as long as the scientist is reputable, rigorous in his or her work and upfront about the connection. Most top medical journals run disclosure statements. Otherwise, try an Internet search, although ultimately this information may be difficult to find.
- Ask a professional. "Ask your doctor, but be critical of what he or she says, depending on that person’s field of expertise," Dr. Eng says. Don’t hesitate to ask for a referral to a specialist or to a physician who sees patients and does research.
We rely on the results of research studies to help us stay healthy. We modify our diets and lifestyles. We pay more attention to our family history, and we schedule routine health screenings. The benefits of being "in the know" are many. The trick is knowing what to know. With a little research and critical thinking, you’ll be able to judge the merits of health news and make informed decisions to benefit you and your family.
© Copyright 1995-2009 Cleveland Clinic. All rights reserved.
Can't find the health information you’re looking for?
Ask a Health Educator, Live!
This article appears in the Health Extra Newsletter (now Be Well).
Know someone who could use this information?...send them this link.
This information is provided by the Cleveland Clinic and is not intended to replace the medical advice of your doctor or health care provider. Please consult your health care provider for advice about a specific medical condition. This document was last reviewed on: 5/1/2007
