The Frontier Psychiatrists? It’s your source for grump columns reviewing data, among other things. It’s written by Owen Scott Muir, M.D., DFAACAP. It’s also a live-action series of conferences called “Rapid Acting Mental Health Treatment”—the next is in NYC on May 5th. You should get your ticket now!
Today, we continue our thrilling journey into the story of Seroquel (Quetiapine) reviewing one of my favorite papers to teach trainees of all time. Not, I will warn readers, because it’s a “great paper.” It’s research so bad it landed in The NY Times as a cautionary tale about clinical trial nonsense by big Pharma in which some of the drugs ended up in morning oatmeal for sex offenders. This is not a joke. Is there any more NYT headline than the following?
However, I will argue the Times reporting gets this almost completely wrong:
Last fall, an article in The American Journal of Psychiatry caught the attention of specialists who treat borderline personality disorder, an intractable condition for which no approved drug treatment exists.
The article seemed to offer a glimmer of hope: The antipsychotic drug Seroquel XR reduced some of the disorder’s worst symptoms in a significant number of patients. “It was an exciting development,” recalled Mark F. Lenzenweger, a professor at Binghamton University and Weill Cornell Medical College and an expert in borderline personality disorder.
Let us dig into why, in that very article, anyone who is a reader of this newsletter could have called bullish*t long before the Times reported this gem:
One morning in May 2010, residents and staff members at Alpha Human Services, a residential treatment facility for sex offenders in Minneapolis, sat down to breakfast and noticed something strange about their oatmeal. It was pink.
Later in the day, several people reported feeling unusually tired. One employee approached the director, Gerald T. Kaplan. “There’s something going on here,” he recalled being told. “I’m a morning person, and I can’t keep my eyes open.”
A study subject had dumped their study meds into the oatmeal of other members of their halfway house. Recall, from yesterday, Quetiapine is reliably sedating.
This study, published in AJP, had the following title, which I will argue, is a LIE1.
Comparison of Low and Moderate Dosages of Extended-Release Quetiapine in Borderline Personality Disorder: A Randomized, Double-Blind, Placebo-Controlled Trial
Where do readers of the Frontier Psychiatrists start with any paper? Yes. Table 1.
In this paper by Black et. al., they attempt to distract our attention from a problem with the trial design by pushing what should be all of table one into two tables, Table One and Table Two. DON'T BE FOOLED. It’s all baseline findings for the experimental groups. If the randomization was successful, there would be no differences between groups.
They had a Table Two, which shows differences in the baseline rating scales, and they provided us a statistical assessment.
Oh, sorry, here's the table without me getting in the way of it:
Over there on the right, you'll notice there is a p-value. And now, I'm gonna interrupt your casual perusing of that table with some commentary, in this helpful visual format:
The randomization failed. The groups are different at baseline. This is not a randomized control trial. We can stop reading now. I'm gonna shit post this a little bit more, but if you were reading this for the first time, you wouldn't make the same mistake the times did, and think there was any promise to be frayed.
Don't take researchers at their word, look at the data!
What happens when one group is reliably sicker than another group? Because that is what the Study did. It compared a group of people, who are overwhelmingly randomized to the moderate dose quetiapine group and compared them to less sick people in the placebo and higher dose quetiapine group. Unremarkably, thanks to regression to the mean, by the end of the study the group who was more sick became less sick. This is exactly what we would expect in the sample by chance. The reason we go through all the trouble of randomizing between the groups is to make sure that any difference at the end of the study is due to the intervention, not due to this reliable statistical movement of numbers. Things that started far from the average will become more average over time. We've known this since the very birth of statistics, thanks to Francis Galton, who I wrote about previously here.
Everything the study says after you've evaluated Table 2— which should've just been more Table 1—and determined the groups were not successfully randomized? Nonsense. But let's continue, to make the point.
The authors claimed the following was the “result” of the study:
Participants treated with 150 mg/day of quetiapine had a significant reduction in the severity of borderline personality disorder symptoms compared with those who received placebo. Adverse events were more likely in participants taking 300 mg/day of quetiapine.
We are asked to believe that moderate dose quetiapine is somehow effective compared to placebo but—at the same time— higher dose quetiapine isn’t more effective than placebo. So either low-dose Seroquel has some magic compared to twice as much of the very same drug… or they screwed up the study. Our readers know something that writers at The Times didn’t catch—they screwed up the randomization. The failed randomization led to greater observed improvements in the moderate dose group—not some magic power of slightly less Seroquel.
There are a few other highlights of bad trial design here, which, again, you can see just by reading the published study and applying common sense. This is how the study was recruited:
Persons 18–45 years of age with moodiness, impulsivity, distrustfulness, and difficult relationships were recruited through referral, advertisements, and word of mouth. After screening with the Diagnostic Interview for DSM-IV Personality Disorders, to confirm the presence of DSM-IV borderline personality disorder, we administered the Structured Clinical Interview for DSM-IV to assess comorbid disorders
and the following were the inclusion/exclusion criteria—I promise this is worth reading:
Participants had to meet the Revised Diagnostic Interview for Borderlines criteria (21) for borderline personality disorder and could not meet current criteria for major depressive disorder, post-traumatic stress disorder, panic disorder, or obsessive-compulsive disorder. They were required to have a total score ≥9 on the Zanarini Rating Scale for Borderline Personality Disorder (“Zanarini scale”) at visit 2. Individuals were excluded if they had ever met criteria for a psychotic disorder, had a primary neurological condition, or were cognitively impaired; had current substance dependence or had recently abused opiates, amphetamine, barbiturates, cocaine, or hallucinogens; were medically unstable; had a history of lack of response to an atypical antipsychotic; were pregnant or lactating; or were acutely suicidal.
However, with all of those criteria, we are told the following:
A total of 111 individuals were screened for the study, and 95 were randomly assigned to a treatment group.
So we are asked to believe that when 111 people, all of whom replied to a newspaper ad, by chance, 85% of them happened to meet all inclusion and no exclusion criteria. If you can find me one person off the street in the next two weeks who happens to…have full diagnostic criteria for BPD, but also happens to not have MDD, PTSD, panic disorder, or OCD, and also doesn’t have a substance use disorder, or even recent abuse of drugs, or had a history—and this is just bonkers—of a lack of response to an atypical antipsychotic? Keeping in mind, reckless substance use is a symptom of BPD.
As a “credulity test” here is the rate of comorbidity in BPD,2 and spoiler alert, it’s not only 15%, which is what the unicorns who responded to the “Do you have BPD” newspaper ad to enroll in the study showed up with…
The people enrolling for the trial? They lied about having BPD. So they could get paid money for their participation. In a poorly designed trial, for a condition, only some of them may have had, the investigators, despite the obviously implausible answers to screening questions, enrolled them in the trial anyway. They went on to report that their intention to randomize this group—enhanced with liars about having BPD recruited from a sex offender treatment setting—led to a real study result. Which, I will point out, it didn't. No subsequent paper has replicated this result, and no oral medicines work in core BPD symptoms to date.
Don't believe everything you read.
Please consider becoming a paid subscriber to this newsletter so I am appropriately incentivized to continue cranking out these gems for you all.
—Owen Scott Muir, M.D., DFAACAP.
Black, D. W., Zanarini, M. C., Romine, A., Shaw, M., Allen, J., & Schulz, S. C. (2014). Comparison of low and moderate dosages of extended-release quetiapine in borderline personality disorder: a randomized, double-blind, placebo-controlled trial. American Journal of Psychiatry, 171(11), 1174-1182.
Zimmerman, M., & Mattia, J. I. (1999). Axis I diagnostic comorbidity and borderline personality disorder. Comprehensive psychiatry, 40(4), 245-252.
I haven't journal access since I was a grad student a million years ago. Is the American Journal of Psychiatry considered a well-regarded journal or a pay to play kind of journal? Either way, shame on them for publishing something with such clear study flaws!
Why bother even reading Table 1 or 2? If you would have scrolled down to the financial conflicts of interest, you would have realized it would be a garbage piece of scholarship due to this sentence "Supported by a grant from AstraZeneca to Dr. Schulz, with subcontracts to Drs. Black and Zanarini." The drug companies have paid roughly 20 billion dollars to the US DOJ for lying about psychiatric pills, and this 'research' is just part of the fabric of deception they weave.