As companies in the dietary supplement space seek to navigate the regulatory requirements around claims substantiation, understanding the terminology used is critical to ensuring the science meets the message. Both FTC and FDA require that claims are backed by competent and reliable scientific evidence, which considers the quality of studies, the fit of studies to the claim, and the amount and type of data. FTC released the “Health Products Compliance Guide” in December 2022 to replace its previous guide, issued more than 25 years ago. Designed to illustrate how FTC identifies express and implied claims and evaluates the scientific support, it offers insight into the agency’s thinking about how to develop an appropriate substantiation dossier.
Competent and reliable scientific evidence has been defined by FTC as “tests, analyses, research, or studies that have:
- been conducted and evaluated in an objective manner by experts in the relevant disease, condition, or function to which the representation relates; and
- are generally accepted in the profession to yield accurate and reliable results.” This includes looking at the reliability of the results, study design, data analysis, and whether there is sufficient positive evidence when considered in light of the entire body of evidence.
This brings up the term totality of evidence, whereby the breadth of the full body of evidence must be considered. In fact, “The Totality of the Evidence” is a whole section within the FTC guidance. The agency notes that the context of the evidence is just as important as the individual studies. It states: “Advertisers should consider all relevant well-conducted research relating to the claimed benefit and shouldn’t focus only on research that supports an effect, while discounting research that doesn’t. ... Where there are inconsistencies in the evidence, it is important to examine whether there is a sound explanation for those inconsistencies. … The totality of evidence also will affect how a claim is presented—that is, how carefully the claim is qualified to reflect accurately the strengths of the evidence.”
This concept means companies must evaluate the breadth of evidence—good, bad and otherwise—as it relates to the desired claim. Megan Olsen, SVP and legal counsel to the Council for Responsible Nutrition (CRN), explains it thus: “You can’t cherry pick studies, but you have to have this good fit and understanding as to if there are studies that are not supportive and those that are, what is the direction of the evidence and is it strong enough to support the claim?”
When inconsistencies do come to light, Rend Al-Mondhiry, partner at Amin Wasserman Gurnani, suggests that a deeper dive might yield an explanation, whether that’s a study in a population that is outside of the target demographic or one that is lacking appropriate controls; the same type of analysis should be done on positive studies as well. “If, after making this inquiry into the evidence, the balance of relevant and reliable evidence points to the claim being unsupported, a marketer may want to think twice about using the claim,” she advises, considering regulatory agencies and attorneys are reviewing the same data and comparing to the messaging.
That said, another aspect to consider, according to Doug Kalman, a research and regulatory expert, is the timeframe; does totality mean every study since PubMed started indexing research, or is a decade enough? “The courts have not defined that, and usually within the scientific realm, even on meta-analysis, you indicate the time period you’re looking at,” he says.
When looking back and building out the totality, companies must also look at the type of studies that fit into that “bucket” of evidence. While randomized controlled trials (RCTs) are the gold standard, particularly if done on the actual finished product rather than similar products or solely ingredients, there are other types of studies that contribute, provided they are methodologically sound. In addition to these studies, other types of information can add to the totality of evidence in support of claim; this could include meta-analyses, testimonials, review articles and more that may further elucidate the mechanism of action of the ingredient(s) and/or formulation. Al-Mondhiry noted those type of resources could summarize and give a sense of what the weight of evidence shows, as well as the strength and validity of studies.
It’s incumbent upon companies to truly substantiate their substantiation. “Really look at your studies and ensure not only are they high quality and provide sufficient, statistical, significant and proper end points, but they really match the product and the type of claims you’re selling to consumers,” Olsen advises.
That issue of relevance is critical, Al-Mondhiry agrees. “You can have a ‘gold standard’ study that meets all of FTC’s criteria,” she said, “but if it isn’t relevant to your specific product and the claim, then that study will not substantiate the claim, no matter how well-conducted it is.”
The federal agencies are generally aligned regarding both the level of competent and reliable scientific evidence and the alignment of any claims related to the totality of evidence. At the same time, there are newer variations in study design and execution to be considered. COVID accelerated the move toward decentralized and virtual RCTs, although there are long-term examples (such as the National Health And Nutrition Examination Survey) of decentralized, questionnaire-based trials. “When we consider the totality of the evidence, to me,” Kalman says, “that includes all types of research, both in-person and decentralized.”
Andrea Wong, Ph.D., CRN’s SVP, scientific & regulatory affairs, advises companies to look at FDA’s guidance, “Decentralized Clinical Trials for Drugs, Biological Products, and Devices,” which could be used as reference material for health products. “That can be a useful tool for companies as they’re looking to design and execute trials that way, whether it’s hybrid or fully virtual,” she says.
When considering the data, Kalman notes, “As long as a study is well-designed and well-executed, how the data is accumulated doesn’t really matter. All types of studies that are done in a good, ethical manner add to the story. They must be well designed, because one of the biggest things in research is asking the right question. And then are you asking the right question in the right population?”
There is also the opportunity to include data around user perceptions of product efficacy, such as surveys, customer feedback or information on repeat purchasing, according to Kalman, although this should be contextualized and weighted within the full data set. FTC’s guidance is clear that anecdotal evidence alone isn’t sufficient for substantiation. “If we’re talking about a general consumer survey querying consumers about their experience with a product, that is unlikely to support a health-related claim,” Al-Mondhiry adds.
It does raise the issue of how finished product formulations perform in a population, and whether the products themselves should have clinical studies. The National Advertising Division (NAD) of the Better Business Bureau (BBB) has emphasized the importance of testing on finished products to support product performance claims; again, the point is that the claim must match the evidence.
As one example, NAD recently determined that Vision Elements Inc. appropriately substantiated the shelf-life stability and safety claims for its Early Defense dietary supplement. However, it recommended other claims related to its “science-driven formula” and eyesight performance claims be discontinued or modified, in part because the finished product itself had not been studied in published clinicals. Al-Mondhiry comments: “[This] case underscores the importance of ensuring that the studies supporting your claims are both reliable AND relevant. The advertiser submitted numerous studies (30+ in total), but NAD recommended nearly all of the claims be discontinued because they weren’t a good fit.”
This is not to say that studies on ingredients aren’t part of the totality of evidence. “Ingredient studies have always been an acceptable form of evidence for structure/function claims,” Olsen clarifies. “But, again, you have to be cautious that your claim itself is being appropriately qualified or positioned.”
One such example is the class action lawsuit filed against VNGR Beverage that alleges its Poppi Soda doesn’t contain enough inulin prebiotic fiber per serving to deliver the purported gut health benefits. While each serving delivers 2 grams of inulin, the lawsuit claims the science shows consumers would need to consume more than four cans of soda per day for three weeks to deliver the gut health benefits. VNGR has responded in the press that it will vigorously defend against the claims, although the slogan “gut healthy” has been removed from its website and product labeling.
While Olsen didn’t comment on any specific cases, she did note in the case of a class action suit, for example, the substantiation for any given claim isn’t public. “Accusations alone are not enough to form an opinion as to whether or not a claim is supported,” she says. “But it all comes back to looking at the full picture, with companies needing to be careful that they’ve done fact-specific analysis.”
Certainly, there are benefits to companies that invest in studies on their finished products in their target population. Kalman likens it to creating a cake recipe; while someone may be able to pick out multiple ingredients with similar health effects, only an experienced formulator could determine how those ingredients may react or the impact on bioavailability, and testing that for its efficacy yields strong data.
In its guidance, FTC offers an example about claims made for a sports drink using a clinically tested ingredient for improving blood flow and increasing endurance; the claim not only connected the positive test results, but also conveyed that the sports drink will provide those benefits. FTC advises: “Because the drink also contains other ingredients, the marketer should consult with a qualified expert in the relevant field to determine whether experts in that field would generally require a clinical test of the sports drink itself, rather than the isolated ingredient, to confirm the blood flow and endurance benefits.” Similarly, FTC offers an example of an energy drink making claims about the safety and efficacy of its product based on well-conducted clinical studies on the ingredients; it concluded the studies likely would not support a safety claim because the combination of the ingredients could affect the body differently.
Kalman concludes: “The finished product study is the ultimate handshake with your consumer that we test our product, we believe in our product, and here’s research. It also allows companies to potentially learn unexpected findings from their product that may give it new uses. The totality of the evidence encompasses much more than just the science or scientific studies.”
