Leveraging Real-World Evidence for Treating Multiple Sclerosis

Podcast Transcript

Introduction: Neuro Pathways, a Cleveland Clinic podcast exploring the latest research discoveries and clinical advances in the fields of neurology, neurosurgery, neuro rehab, and psychiatry.

Glen Stevens, DO, PhD:

Disease-modifying therapies improve outcomes by reducing disease activity and progression in multiple sclerosis, but the emerging disease-modifying therapy landscape remains complex. As new therapies with variable efficacy and safety profiles become available, these complexities become greater and result in challenges in disease-modifying therapy clinical decision-making. While randomized trials provide the highest level of evidence for disease-modifying therapy, safety, and efficacy, they are cost and time prohibitive and restrictive in the clinical setting due to inclusion and exclusion criteria. In response, observational studies harnessing real world data are being used to allow direct comparison of disease-modifying therapies in larger, more heterogeneous patient population to answer clinical questions with broad applicability.

In today's episode of Neuro Pathways, we're taking a closer look at this real world evidence used for multiple sclerosis clinical decision-making. I'm your host Glen Stevens, neurologist/neuro-oncologist in Cleveland Clinic's Neurological Institute, and I'm excited to be joined by Dr. Carrie Hersh. Dr. Hersh is a neuro-immunologist and director of the Multiple Sclerosis Health and Wellness Program at the Lou Ruvo Center for Brain Health in Las Vegas, Nevada. Carrie, welcome to Neuro Pathways.

Carrie Hersh, DO:

Thank you so much for having me today.

Glen Stevens, DO, PhD:

So Carrie, I'm not sure if you know, but my mother had multiple sclerosis, so I was raised by someone that had the disease, and eventually, unfortunately, she died from complications related to the disease. But before we get to that, and I've known you for many years, but tell our audience a little bit about how you made your way into the Cleveland Clinic and just a little bit about yourself.

Carrie Hersh, DO:

Absolutely. So I've actually been with the Cleveland Clinic since I was doing my internship back in 2009, and I went ahead to complete my adult neurology residency training program at the Cleveland Clinic, where I had the honor and privilege of working under the tutelage of Dr. Stevens. And then my interest in MS and neuro-immunology blossomed during my latter years of adult neurology residency, ended up doing a two-year combination of a clinical and research fellowship at the Mellen Center. After I completed my fellowship training, I really wanted to stay with the Cleveland Clinic and continue the good work that I had started with the neuro-immunology and MS team at the Mellen Center. But being from Florida, I really needed some more sunshine again. So when this opportunity opened up in Las Vegas, where I would be able to translate my information out to the west, where there was a really huge need for specialty MS care, it just seemed like the universe and the world was just aligning and all the stars and everything just kind of came together. And I've been here now for seven and a half years.

Glen Stevens, DO, PhD:

Well, listen, I'm just thrilled that you stayed with the Cleveland Clinic. I'm sorry that I don't get to see you. I always loved working with you. When I was on-call, I could rest easy because I know you were looking after everything, so appreciate your efforts back in the day.

Carrie Hersh, DO:

Thank you.

Glen Stevens, DO, PhD:

In today's episode, the discussion is really centered on areas where you focused your own practice, thankfully, and that is with real world experience, which is really what's important, and also outcomes-based research and comparative effectiveness of these studies. I can tell you with my mother, the decision was, do we give steroids or don't we give steroids? There was really nothing. And in terms of the disease-modifying therapy, sometimes, even though I don't look after MS patients, I think it must just be overwhelming. There are so many therapies that are out there. So I'm just fascinated by the work that you're doing now because as we realize, we just can't keep doing two-drug comparisons when there's 15 drugs out there. So tell us a little bit about what you're doing, it's quite fascinating, and how you're able to do it.

Carrie Hersh, DO:

Yes. Well, no, that's a fabulous question, lovely segue into some of the work that I've been doing out here. So my interest in real world evidence in comparative effectiveness started at the Mellen Center. And it started as a two-drug comparison. And this was back in 2015, really before we had this huge explosion of new DMTs over the past five years. And there was specifically a lot of interest in the newly approved oral disease-modifying therapies, specifically dimethyl fumarate, Tecfidera, and fingolimod or Gilenya. But there was a lot of clinical equipoise in terms of, well, does one medicine work better than the other medicine? And because there was such a huge interest in folks with MS transitioning from these injectable platform therapies that were low effective, and there was a lot of intolerability with injection site reactions, there was certainly a lot of interest for folks to go ahead and move on to oral therapies that also showed higher efficacy too.

But the big question was, well, which one might work better? Does one work better than the other one? And so that's where my interest in comparative effectiveness really blossomed. And with the tutelage of a well-known biostatistician in a specific kind of statistical analysis called propensity scoring that I learned at Case Western Reserve University, I was able to translate that knowledge base to conduct this three-year study it turned into of comparing the clinical and radiographic effectiveness of Tecfidera and Gilenya.

Ultimately, what we were able to see in our study, just using real world evidence from the electronic medical record, nothing really fancy, we did show that there was pretty much comparable effectiveness between the two. The only thing that we have to consider, however, is it was in one center, one location, and we were using data from one electronic medical record source. So even though we were able to adjust for a lot of confounding that we can see that makes these real world evidence studies messy, especially when they're retrospective, we have to consider the source of the data and where they're coming from and how generalizable, what is the external validity, when we are trying to broaden that scope to the larger MS population in the country and then across the world.

So with that being said, that led to further interest down the line, especially once I was out here in Las Vegas and the advent of the MS Partners Advancing Technology and Health Solutions program or MS PATHS came to light. We were able to use multiple, multiple data points from a much larger and heterogeneous patient population coming not from one or two centers, but coming from 10 different academic healthcare centers across the world. And so that's where some of my work has continued to evolve and blossom over the years.

Glen Stevens, DO, PhD:

So I'm really glad that you mentioned the propensity scores and that I don't have to explain that to any great detail. But I was reading one of the clinical trials, and it's great that everybody's collaborating. I just got off a podcast with Dr. Malkin. We were talking about clinical trials and brain tumor. We're kind of an orphan disease, and one of the things that we realize is that we really need to have collaborative sites where you can store data and you can answer questions that way. But then it's always, who owns the data, those types of things. But missing data becomes a big issue. And one of the DMT trials that recently published that I was looking at, I think almost half of the patients had missing MRI data, other types of data were missing. How do we get past that? Or is it just education and just being strict up front of making it easier to transfer files? Because it's always tough to transfer files, especially big files with imaging, right?

Carrie Hersh, DO:

That's right, yes. And you raise an excellent question. And the quality of the results and the conclusions are highly dependent on the data quality that is coming in. And missing data are a large part and a large limitation of retrospective observational studies, even when we are working with a large collaboration program like the MS PATHS program, because it still relies on patients coming in and then the real world implications of getting things done that a highly regimented clinical trial just has a much better opportunity to control.

So at this point, when there are missing data, I mean, we can use statistical methods like imputation, but that only works for half of the data. So that's the information coming in. But what about the information coming out? So what we can do as a learning healthcare system is that we can look at doing retrospective observational studies in a couple of different ways.

We can use intention to treat, which means that we are analyzing all of the patients who are enrolled in the study, whether or not they have all of the complete endpoints available. And then we can do a sensitivity analysis looking at the per protocol, so only including patients who have a complete data set. The more patients that we include in a particular study, so meaning the higher the sample size, the more information we'll be able to glean even with that missing is available. So again, having the availability of such a large learning health system or database, if you're working on something that goes outside of a highly standardized program like MS PATHS, can only help with the quality of the data that are coming through. But it continues to be a huge limitation and probably, if not the biggest, one of the biggest limitations of the work that I do.

Glen Stevens, DO, PhD:

So do you have an active consortium that you're working with? Or is it just silent handshakes that we're going to share stuff? Or is it a functional consortium?

Carrie Hersh, DO:

Yeah, so the way that the MS PATHS program operates, it has a high governance structure. Luckily, the way that the program is structured is that there are 10 healthcare centers that are highly motivated to not only be a part of the program, but to enroll patients and to standardize the way that data are being collected so that way we have it not only available at point of care, so that will help us learn about our patient population and deliver the best possible care that we can, but then we also have the data at our fingertips for putting out fantastic research across different facets and modalities, looking at biomarkers and MRI data and then doing the work that I'm doing and looking at personalized treatment of our patients using various disease-modifying therapies. So while it's not just that silent handshake, there certainly is this spoken agreement that we are going to do our very best to get as many patients enrolled as we can, but to make this work at our center and incorporate this and highly integrate this into our workflow structure.

Glen Stevens, DO, PhD:

So I hate to use the F word, but how do you fund it?

Carrie Hersh, DO:

No, that's a great question. So we're very lucky in that this is a collaboration project through our funding source, Biogen, and the 10 healthcare academic institutions across the world. So we are very lucky in this sense that we have a highly motivated team through industry.

Glen Stevens, DO, PhD:

And there must be a central office somewhere that coordinates it. Is that right? Or a data ...

Carrie Hersh, DO:

Correct.

Glen Stevens, DO, PhD:

... a big data super computer somewhere that's collecting it all?

Carrie Hersh, DO:

Yeah. So there's Biogen headquarters in Cambridge. That's the physical location of the folks who collaborate with us in the MS PATHS program. And then we have a huge data storage house that makes up the learning health system and is made available to all of the investigators who are part of the 10 active healthcare institutions. And we have a new system in place now that, what we call external investigators, can actually get access to the data so long as they're working with a principal investigator or another co-investigator that are part of the other institutions that are embedded in the MS PATHS program.

Glen Stevens, DO, PhD:

So if I'm living in a publish-or-perish environment and I'm one of these 10 centers, how do I know that I can get access to the data or a study that I want to do? Is there some sort of a committee that defines the projects that will go forward in this? Or how do you determine that?

Carrie Hersh, DO:

Yeah, great question. So we actually do have a couple of committees that oversee the different projects, and we actually have a pretty highly structured system of how projects are approved. So that way we don't have multiple people working on a similar question, but they're doing it differently because then we have to consider the reliability and the validity of the data. So we do have those committees that are in place so that we have some sort of a structure on what is being done, what data are being collected, and what studies are being conducted.

Currently, the way that external investigators are being made aware of this is currently word of mouth. I don't think that we have a highly systematic way to announce that these data are available for other investigators. And I think that part of that is we want at least some level of, and I don't know if control is the right word for it, but we need to have some structure of who is doing projects and where they're being done and how they're being done. So right now it's essentially word of mouth through the other investigators, through MS PATHS.

Glen Stevens, DO, PhD:

So the MS PATHS again stands for Multiple Sclerosis Partners Advancing Technology and Health Solutions, if I got that correct, hopefully.

Carrie Hersh, DO:

That's correct. Yes.

Glen Stevens, DO, PhD:

Can you share a couple of examples of the comparative effectiveness studies that have been completed?

Carrie Hersh, DO:

Yeah, sure.

Glen Stevens, DO, PhD:

And the findings, maybe?

Carrie Hersh, DO:

So there are two of note that I can talk to you about. One of the projects that we published a couple of years ago in 2021 that was entitled, Impact of Natalizumab on Quality of Life in a Real World Cohort of Patients with MS. And specifically, what we were interested in looking at was the comparative quality of life effectiveness between two medications that are considered highly effective therapies. So one is called natalizumab or Tysabri, and the other medication is called ocrelizumab or Ocrevus.

And the reason why we were specifically interested in looking at these two couple of disease-modifying therapies is because they are commonly used in clinical practice. And once you are looking at highly effective therapies, doing a comparative analysis gets kind of tricky. And the reason for that is because they are so good at what they do in preventing new disease activity that looking at differences in treatment effects between these two therapies would be rather difficult, especially when the event rates are relatively low. So essentially, it would probably take thousands and thousands of patients in order to do a properly-conducted comparative effectiveness study looking at relapse on MRIs.

So instead, we decided to look at quality of life. And the reason why this is so important is because what the patient is experiencing from day-to-day in terms of their physical, their mental, and their social well-being is so key. And when we're already talking about highly effective therapies, where we know that generally speaking, they're probably going to be pretty well controlled from the disease standpoint, we can start looking at some other factors that might be able to help separate the two when we are at the bedside and we're talking to our patients about starting one or the other.

So that's where this interest primarily came from. And the reason why we decided to look at specifically natalizumab or Tysabri against ocrelizumab or Ocrevus is so that if we really were seeing any improvements in quality of life as patients were treated on this medication, we could say that well, compared to another medication, we actually saw even more improvements than what we otherwise might only expect with expectation bias, so meaning that the patient just feels good because they know that they're on a good medicine. So that's the reason why we decided to do that comparative analysis to see how it stands and works against another highly effective disease-modifying therapy.

Glen Stevens, DO, PhD:

That's great. We actually, in the cancer field, have had a number of drugs that have been released as anti-cancer drugs, and they received FDA-approval because they, as well, affected quality of life in a positive direction, where progression-free survival, overall survival weren't necessarily changed, but it was felt that quality of life was so important that it was worthwhile to approve it. I think you mentioned one other comparison of drugs that you looked at as well. Won't you share that with us?

Carrie Hersh, DO:

Yes, absolutely. And this is actually a segue from the work that I had talked about earlier in the podcast looking at dimethyl fumarate or Tecfidera versus fingolimod or Gilenya. So as I had mentioned before, one of the issues that we have to consider when we're doing a comparative effectiveness study just using one site is that, well, we're only looking at patients at that one site, and the information is not standardized. People might be getting MRIs from this state and then coming to our location for this MRI. So we don't have a systematic way that we are collecting the data. So that might impact the endpoints that we are making conclusions on when we're talking about treatment effect differences.

So we decided to look at a similar comparative effectiveness between these two agents, but instead of just using data from the EMR through the Cleveland Clinic, we decided to use data from the MS PATHS learning health system, so that way, not only are we including a more heterogeneous larger patient population, but we are also using standardized information, including how the data are collected as part of routine clinical care, the standardization of MRIs, and then the standardization of how we are collecting patient-reported outcomes. And not only that, we were able to look at the comparative analysis between both of these medications on a biomarker called serum neurofilament light chain, which is actually gaining a lot of interest in the MS space as an inflammatory and a predictor of how patients are responding to individual treatments.

So we're able to include all of these things in this study. And the great part is that even though the data are different, coming from different places, we still showed comparability between these two disease-modifying therapies, which is reassuring because it gives us more clarification and more reassurance that the information we get from different study designs, from different data populations, they're lining up.

Glen Stevens, DO, PhD:

And I guess the low-hanging fruit, if the data is there, as you know, they're comparable. So now you can look at, is the quality of life the same, and that would potentially alter how you would give the drug, right?

Carrie Hersh, DO:

That's exactly right. And the great part about using these data are that we are looking at this in the real world population. They're not highly regimented, high rigorous, randomized clinical trials. So it's a little bit easier to translate these data to the bedside when we're educating our patients because the patients who are included in these studies are the patients who are seen in clinical practice.

Glen Stevens, DO, PhD:

Well, as a patient, this is data that I would want to see. That's for sure. This is very exciting. So where do we go from here? Just more patients, more centers?

Carrie Hersh, DO:

The next frontier, at least in my world of real world evidence, is a concept called heterogeneous treatment effects, if I can go back and give a little bit of background of what this is and what's been done previously and then how we're trying to encapsulate that into the MS PATHS program. So what we understand is that randomized trials, they provide estimates of average treatment effects, but they're really not well-suited to understand a concept of heterogeneity of treatment effects, so that means the non-random variability in the direction or magnitude of a treatment effect across individuals, across people. So to put it simply, in clinical practice, not all patients respond the same to a particular disease-modifying therapy or to a specific class of disease-modifying therapy. And we really don't have a good understanding of that. And sometimes we do have a pretty difficult time personalizing and individualizing the selection of a disease-modifying therapy to a patient.

And so this work has already been done, looking at these heterogeneous treatment effects, using data from clinical trials. But again, the problem is that these data are highly regimented, very strict inclusion-exclusion criteria. So we have a difficult time translating that to a more generalized patient population. So this is where we can take this concept and we can now embed this into a framework that uses real world data. And not only that, but in MS PATHS, we probably have over 18,000 unique patients, so a very large heterogeneous treatment population to begin with. And essentially what we're trying to do is understand a person's baseline risk of having an outcome of interest, let's say relapses, and then being able to determine, based on those disease characteristics, where that baseline risk exists, and how can these different classes of disease-modifying therapies categorized into low, moderate, and highly effective, how can that translate?

So meaning a patient who has these baseline disease characteristics, are they going to be better off on this particular medicine compared to this particular medicine based off of their baseline characteristics? So what we're trying to do is build these highly sophisticated logistic regression models so that way we can do a better job of individualizing and personalizing treatment to an individual that is based on their disease characteristics and their demographics. So imagine if, in the Epic EMR, you have a patient coming in, and we could include all of this relevant information, their biological sex and their MS phenotype, how long they've had MS, so on and so forth. And then we could graph that very nicely on a graph that's embedded in the EMR to determine, well, where would you fall if we decided to choose a moderately effective therapy as opposed to a highly effective therapy? So it would individualize the care a lot better than maybe we can do currently.

Glen Stevens, DO, PhD:

Carrie, I didn't realize that you are such a statistical wizard.

Carrie Hersh, DO:

I don't know if I would call myself a statistical wizard, but I would say that I was very lucky to have the training that I did at the Cleveland Clinic and through the Case CRISP program. So for those who are listening, who are considering additional education and training through the CRISP program, it's fantastic. And that's where I learned all of my knowledge base with the propensity score analysis.

Glen Stevens, DO, PhD:

Well, I know that my mother would be crying with all the great work you're doing if she was here today and how the field has moved forward. So I'm so excited for everything that you're doing. Anything that we missed that you want to talk about?

Carrie Hersh, DO:

I would just like to say, this is such an exciting time in the MS space, and I'm so pleased to be in this space, in the scientific space, in 2023. There are so many incredible things that are happening in the MS world, and the importance of real world evidence is just blossoming. And I really do feel that through the work that we're doing at the Cleveland Clinic and that other folks are doing across the country and the world, we'll be able to personalize care a whole lot better than we've been able to before.

Glen Stevens, DO, PhD:

Well, Carrie, it was great having you join us today. I really appreciate your time and insights, and I knew that you would be where you are today when I first met you. So what you're doing is no surprise to me. I'm so proud of you, and I look forward to seeing you down the road.

Carrie Hersh, DO:

It was a pleasure, Dr. Stevens. Thank you so much for having me today.

Conclusion: This concludes this episode of Neuro Pathways. You can find additional podcast episodes on our website, clevelandclinic.org/neuropodcast, or subscribe to the podcast on iTunes, Google Play, Spotify, or wherever you get your podcasts. And don't forget, you can access real-time updates from experts in Cleveland Clinic's Neurological Institute on our Consult QD website. That's consultqd.clevelandclinic.org/neuro, or follow us on Twitter @CleClinicMD, all one word. And thank you for listening.