The Headline Shook Me.
“TikTok Withheld a Safety Feature From Millions. One Died by Suicide”
I can’t unsee it. I put on my “innocent consumer” hat and it seems ridiculous.
I put on my “boss” hat and I’m perplexed.
I put on my “dad” hat and I’m devastated.
Now, I’m wearing my “keyboard warrior” hat and I’m furious.
How did we get here? That story was published a week ago and it barely seems to matter. I’ve seen no repercussions, socially or economically. Does our News cycle moves too fast? Is there just too much horrible news? Or are people already numb to this?
I’m probably overthinking it. People make dozens of choices every day that, analyzed accordingly, would make it hard to put one foot in front of the other. After all, we need electricity to power our nose-hair trimmers, and electricity also powers the electric chair. We give guns to law enforcement officers and soldiers who use them to try to keep people safe. Guns also do horrific harm to innocent victims, and some days, I wish they’d never been invented. We have an obesity epidemic in this country, yet so many among our children go hungry.
There are plenty of nuanced problems that need measured, balanced approaches to solve them. How do we end up in a place where it’s okay for a social media company to keep a control group of 15 million people using their product without the safety features they’d already built, just to see what happens? How is it we allow a company to show people, including our most vulnerable, thousands of videos about suicide, sadness, hopelessness, and loneliness? How is this a thing? I want to scream: What are we doing here?! If you read the article I linked above, tell me you’re not furious.
How does anyone who works at that company go into the office the next day? It reminds me of when I worked at Google in 2019. There was a widely reported story about the so-called “rabbit hole”, where the algorithm led viewers to more and more radicalized content. I turned to a colleague in the Google office that day and said, “Crazy story... not much to be proud of there.” She hadn’t seen it, so I explained. Her reply, “Oh, that’s YouTube. I don’t work on that product.”
And there it was. I watched, in real time, someone compartmentalize without missing a beat. Keep your head down, focus on your work, don’t worry about the rest, keep cashing the paycheck, keep vesting the stock.
I know media and technology is my wheelhouse, so maybe I’m oversensitive to it. But in other markets, am I any less guilty? I still drive a gas-powered car and do my part to wreck the environment. I watch college (now professional) and professional football, which hands its gladiators concussions, torn ligaments, crushed bones, and life-altering neurological trauma on a weekly basis. All for our entertainment.
So where do we go from here? Delete TikTok? (Maybe)
Quit social media altogether? (Unlikely)
Kill algorithmically-fed content? (Yes)
The least I can do is remind you. Be aware of it and champion balanced analysis with nuanced answers. Maybe together we can make a dent.
(Bloomberg’s Tiktok article below if you couldn’t access above)
TikTok Withheld a Safety Feature From Millions. One Died by Suicide
A confidential document says an algorithm adjustment was kept from many users “by design,” as part of an experiment to measure engagement.
August 4, 2026 at 9:00 PM UTC
Save
Translate
When TikTok tweaked its algorithm in 2021 to stop users from being overwhelmed with harmful content, the company didn’t roll out the safer version to everyone, a confidential internal document shows. Instead, to see if the change might reduce the app’s stickiness, the company conducted an experiment. It created a control group of 10% of US users — at the time, that would have been about 15 million people — who kept the old version of the app. This group, according to the document, included 16-year-old Chase Nasca. TikTok’s algorithm pushed him thousands of videos about suicide, sadness, hopelessness and loneliness, right up until he killed himself.
Listen to the Story
TikTok Withheld a Safety Feature From Millions. One Died by Suicide
00:00 / 19:40
Chase’s account was stuck in a so-called filter bubble, an online echo chamber that pumped out repetitive content the algorithm predicted he’d want to watch. There he received an “onslaught” of suicide and self-harm content, according to the document. Because Chase was in the control group, it says, “TikTok’s filter bubble prevention strategies did not take effect on this user by design.”
Those two words, “by design,” carry extra weight against the backdrop of litigation over social media addiction that’s playing out in US courts. The crux of the legal argument being wielded against the world’s biggest social media platforms is that their products are dangerous by design. The companies, including TikTok, vehemently deny this and point to safety features they’ve introduced to protect users. But the document shows TikTok intentionally withheld a safety feature from millions of people, and why.
“The user impact decision,” it says, “was a delicate balance across safety and the ability to measure impact on DAU and core metrics.” DAU, or daily active users, is one of the top data points by which Wall Street judges social media companies. Within the hypercompetitive industry, user growth is often prioritized over user safety. That trade-off has been well documented for years, and it’s now the backbone of thousands of lawsuits against the companies.
In response to questions about the document, a TikTok spokesperson sent a written statement saying the company was “deeply committed to the safety and well-being of users,” especially teens. “Our hearts break for any family that experienced a tragic loss,” the spokesperson said. “To help protect our community, we continue to invest significantly in Trust & Safety, including robust detection systems and dedicated enforcement teams that proactively remove content that violates our Community Guidelines.”
Written in March 2023, the document is an in-depth review of Chase’s account, prepared by members of TikTok’s safety and algorithm teams. It provides extraordinary detail about his watch history in the weeks, days and hours before he died. Marked “privileged and confidential,” it’s currently under a court-ordered seal.
Two weeks after the review was conducted, TikTok Chief Executive Officer Shou Chew testified before Congress. He told lawmakers, “Safety and wellness — in particular for teens — is a core priority for TikTok.” During questioning, he said that “we have a commitment to transparency.” Chew made this point so often that one representative, Alabama Republican Gary Palmer, used it to call him out: “You have repeatedly used the word ‘transparency’ throughout this hearing. And every time you’ve said it, what I’ve heard is deception.”
After the hearing, members of Congress sent TikTok written questions. One asked: “How many cases is TikTok aware of where users have committed suicide after receiving sad, depressing or suicide-promoting videos on TikTok?” In response the company wrote, “TikTok does not believe that use of the platform has caused users to commit suicide.” That answer was dated May 4, 2023 — 15 months after Chase’s death and two months after the company’s investigation into his account.
“His account was recommended a repetitive amount of content surrounding Suicide and Self Harm and depressive topics, particularly leading up to the date of his suicide”
TikTok doesn’t typically examine the watch history of individual users. The company analyzed Chase’s after Bloomberg Businessweek reached out in March 2023, seeking comment for a story about TikTok’s impact on teen users. The article centered on Chase, a free-spirited 16-year-old soccer prodigy with no history of mental health issues, who in February 2022 killed himself by stepping in front of a train in Bayport, New York.
He left no suicide note, and his death at first mystified his friends and family. But when Chase’s mother gained access to his TikTok account, she found a clue: The algorithm was feeding him videos about suicide. Five days before Chase’s death, she discovered, he’d been sent a video of a person taking a selfie in front of an oncoming train, with the caption “went for a quick lil walk to clear my head.”
Following Businessweek’s inquiry, TikTok’s algorithm and safety teams reconstructed Chase’s watch history. They summarized their findings in a 14-page document, titled “Bloomberg Businessweek, Youth Safety & Filter Bubbles: Escalation Summary.” A list of “key contacts” shows it was shared across 10 teams within TikTok, including communications, legal response, policy, security and search. The report’s first line alerts readers that it contains “potentially disturbing” information.
“Overall, 16-year-old user Chase Nasca’s account fell into a filter bubble where his account was recommended a repetitive amount of content surrounding Suicide and Self Harm and depressive topics, particularly leading up to the date of his suicide,” the document says. Many of Chase’s video searches were about sports and comedy, it says. But he also searched for songs about loneliness and hopelessness. Those songs, written “by popular musical artists who struggled with depression and suicidal ideation,” likely triggered the algorithm to keep feeding him this content, the document says.
TikTok analyzed 7,563 videos served to Chase in the two weeks before he died. Its review found that 73% carried common themes — “testimonials of suicide survivors, mental health struggles, suicide awareness and longing to be in a relationship or sadness over a breakup” — but didn’t violate TikTok’s rules. An additional group of videos, almost 10%, did violate the rules, which ban anything that promotes or normalizes suicide or self-harm. These videos included “images with memes and messages about deep depression, low self-esteem, loneliness, feeling tired and suicidal ideation.” In all, thousands of videos that were pushed to Chase had a current of despair.
The analysis provides a glimpse into the last moments of Chase’s life. The day before he died, he scrolled TikTok for more than five hours and searched for posts about suicide and trains. On the day of his death, he favorited videos alluding to suicide. Hours before he died, he watched (and rewatched) a video about a teen boy who left a goodbye missive on social media, saying “this world is going to forget me,” before jumping from the Manhattan Bridge. The last video Chase saw was of someone crying.
The document explains that Chase was fed this kind of content because his account had been randomly selected for an “internal backtest,” making his account part of the control group. So, for him, TikTok’s safeguard wasn’t switched on. Backtesting, the document states, is a common data science technique, often referred to as an A/B experiment, that establishes two groups to measure a change’s impact and “validate the gains.”
To “prevent future incidents,” the report provides two recommendations: shrink the control group to less than 1% and reduce the test duration for the same users “from a half year/full year to 7 days.” But an update included in the report warned that cutting the test pool to one-tenth the size still meant “at least 1.8 million US users have been excluded,” putting them “at risk for falling into bubbles.” (The 2022 A/B test was a routine one, used to evaluate whether safety changes were working as intended, the company told Businessweek on Tuesday. TikTok said it has since refined how it conducts such tests, without elaborating.)
In the past four years, more than 4,000 personal injury lawsuits have been filed accusing social media companies of harming children’s mental health, with many cases joined into a huge, consolidated litigation. So far only one lawsuit has made it to trial. In March a jury in California found Google’s YouTube and Meta Platforms Inc.’s Instagram negligent for intentionally targeting children with addictive features. Jurors awarded the plaintiff, who was 17 when she sued, $6 million in damages. (TikTok and Snap Inc. settled on the eve of trial. Google and Meta say they plan to appeal.) In the run-up to the case, all four companies went through an exhaustive two-year discovery process connected to the broader litigation and handed over more than 6 million internal documents to the other side. The review of Chase’s account was unearthed during that discovery process.
On Monday, TikTok finalized settlements in three more cases. The company has settled every personal injury case alleging social media addiction, five in all, that had been set for trial this year.
When TikTok exploded into popular culture in 2018, what set it apart was its stickiness — the irresistible carousel of hyperpersonalized content on its For You page, optimized for engagement. By the summer of 2021 it had reached 1 billion users worldwide, faster than any other social media platform. Rivals tried to mimic the algorithm driving its recommendation engine, and journalists started investigating it. The Wall Street Journal created more than 100 automated TikTok accounts to track how it works. Within about 36 minutes, one bot programmed to engage with posts about depression was being fed a stream of content that was 93% about sad topics.
In December 2021, TikTok issued a press release saying it was testing ways to “interrupt repetitive patterns” of subjects that could be “problematic if viewed in clusters,” such as extreme dieting and loneliness. By early 2022, TikTok had rolled out its newly safeguarded algorithm to 90% of US users. It held back 10% in the control group, according to the confidential document.
The report pinpoints Jan. 25, 2022, as the day Chase’s account was randomly selected for the backtest experiment. That’s around the time a high school friend, Brandon Maffei, noticed a change in the videos Chase was sharing, according to an interview Maffei gave Businessweek the following year. Chase used to share funny videos with his friends. Now he was sending posts about suicide. Maffei remembered getting one in particular: “I opened it up, and I was just like, ‘Damn. I don’t know why he’s sending this to me.’” Within a month, Chase was dead.
Chase’s account remained active, and a year later the algorithm was still sending it suicide content, his mother discovered. When Businessweek reached out to the company for comment for the 2023 article, TikTok employees traded messages discussing a public-relations strategy for the story. (These internal communications have become an exhibit in litigation against social media companies. The exhibit was unsealed earlier this year, but references to Chase’s name and the backtest experiment were redacted.)
The internal dialogue, visible in the court filing, provides a behind-the-scenes account of a corporate crisis team mobilizing. Two dozen employees are looped into the multiday thread, with messages being sent late into the night. After one employee from the trust and safety team shares a summary of Chase’s watch history, three others respond with the crying emoji. A communications team member then shares that they are “developing on-the-record quotes to show empathy for these issues” and are hoping to expedite the rollout of a feature allowing people to refresh their For You feeds, to “show progress ahead of the story.”
In the end, TikTok provided a statement to Businessweek for the 2023 story that’s similar to the one it gave for this story. The company said it was committed to the safety and well-being of teen users. “Our hearts break for any family that experiences a tragic loss,” said Jamie Favazza, a spokeswoman. “We strive to provide a positive and enriching experience and will continue our significant investment in safeguarding our platform.”
When Chew, the CEO, went before the House Committee on Energy and Commerce on March 23, 2023, it was two weeks after Businessweek had reached out to TikTok and the company had reviewed Chase’s account. The hearing lasted five hours. Members asked him about TikTok’s ties to its Beijing-based parent company, ByteDance Ltd., about possible national security risks — and about child safety. Florida Republican Gus Bilirakis motioned toward Chase’s parents, who were sitting in the public gallery. They’d filed a wrongful death lawsuit against TikTok in New York two days earlier. “Mr. Chew,” Bilirakis said, “your company destroyed their lives.” For Chase, TikTok “was sadly a window to discover suicide,” he said. Bilirakis then played a 30-second recording of videos sent to Chase’s account, including one in which a user says, “I’m going to put a shotgun in my mouth and blow the brains out the back of my head.”
“Your technology is literally leading to death,” Bilirakis said to Chew. “Yes or no, do you have full responsibility for your algorithms?” Chew began to answer, saying, “I would just like to, respectfully, if you don’t mind, I would just like to start by saying it is devastating to hear about the news, as a father myself,” but Bilirakis cut him off. “Yes or no?” he demanded. “We do take these issues very seriously, and we do provide resources for anyone who types in anything like ‘suicide,’” Chew said, before Bilirakis interrupted again: “I see you are not willing to answer the question or take any responsibility for your parent company, the technology and the harm it creates.”
The experiment should have excluded minors and dangerous content such as suicide videos, a former trust and safety worker says
The following month, the committee sent TikTok a long list of follow-up questions. TikTok responded on May 4, providing 53 pages of answers. One question asked: “When did TikTok first become aware that its algorithms send sad, depressing and suicide-promoting videos to the ‘For You’ feeds of teenage users?”
The reply began: “TikTok does not believe that this is an accurate sentiment regarding its algorithms. The safety of the TikTok community is of the utmost importance to TikTok.” The company added that it “proactively blocks, redirects, and warns users who attempt to search” for “dangerous or otherwise harmful content.”
The next question: “What if anything has TikTok done to stop young users from receiving sad, depressing and suicide-promoting videos in their ‘For You’ feeds?”
The reply began: “TikTok’s Community Guidelines prohibit content depicting, promoting, normalizing, or glorifying activities that could lead to suicide or self-harm.” The company added that it’s “aware of the potential concern with receiving recurrent content” and that it’s introduced strategies to help users avoid filter bubbles.
Two questions down: “How many cases is TikTok aware of where users have committed suicide after receiving sad, depressing or suicide-promoting videos on TikTok?”
The reply: User safety is an “important priority” for TikTok; its policies “aim to support people who may be struggling”; it “provides regional suicide prevention hotline numbers.” This answer also said, “TikTok does not believe that use of the platform has caused users to commit suicide.”
Then, six more questions down: “Please list all cases where algorithmic AB testing has resulted in harm to TikTok’s users.”
To this, TikTok’s answer, in its entirety, was: “The safety of the TikTok community is of the utmost importance to TikTok and the Community Guidelines include rules and standards for using TikTok. The guidelines apply to everyone and everything on our platform, including content that is presented to users during AB testing.”
The company’s answers made no mention of the backtest experiment described in the confidential document.
The Businessweek story was published in April 2023, two weeks before TikTok’s written answers were submitted to Congress. The article cited former employees of the company’s safety team who said they were concerned about the high level of secrecy around the algorithm, how it was weighted and how it could be changed. They insisted they’d been set up to fail — asked to enhance the safety of an app whose underpinnings they couldn’t comprehend.
One of those trust and safety workers had been employed as a team leader at the time the backtest experiment was rolled out. When contacted for this story, he said he hadn’t been aware of the experiment. After hearing details from the document, he confirmed that such tests are commonly used in the social media industry but said there were flaws in this one’s methodology. The share of users assigned to the control group was much too high, he said, and the experiment should have excluded minors and dangerous content such as suicide videos.
The wrongful death lawsuit brought by Chase’s parents against TikTok was dismissed by a judge last year, after the company argued that Section 230 of the Communications Decency Act shielded it from liability. An appeal is pending. The Nascas didn’t respond to requests for comment for this story. (Soon after the 2023 Businessweek story was published, Chase’s parents were interviewed for a documentary Bloomberg was co-producing about social media harms. They later decided they didn’t want to be involved and filed a lawsuit to prevent the interview from being used. The interview wasn’t included in the documentary, the lawsuit was dismissed, and Chase’s parents have filed a notice of appeal.)
At least one effort has been made to get the confidential document in front of Chase’s parents. The Arkansas attorney general’s office, in a lawsuit against TikTok, filed a motion seeking to depose Chase’s father in June. It wanted to show him 15 documents, including the one about his son being part of an experiment. TikTok responded in a court filing, saying Chase’s father “has never seen these documents and has no personal knowledge of their contents.” The deposition request should be rejected, the company argued, and the documents should remain sealed because they contain “sensitive, internal, confidential discussions.” A judge sided with TikTok.
Get the Businessweek Daily newsletter.
Fresh perspectives on business, economics, politics and tech.
Sign Up
By continuing, I agree to the Privacy Policy and Terms of Service.
The use of A/B testing on teen social media users was cited in a wrongful death lawsuit filed in July in Delaware, alleging that two young people who died by suicide were addicted to TikTok and other platforms. Such testing, the complaint says, has ended in direct harm to young users: “It’s like relaxing the seatbelt standards in a child’s car seat to determine whether consumers will buy more of those carseats, knowing that some number of children died in car accidents because of that choice.” —With Madlin Mekelburg
More coverage:
Unsafe Online: Investigating Social Media’s Impact on Child Safety
TikTok’s Algorithm Keeps Pushing Suicide to Vulnerable Kids
Levittown Podcast: A Real-Life AI Deepfake Horror Story
Big Take Podcast: The Dark Side of TikTok’s Algorithm
If you or someone you know is having thoughts of suicide or self-harm, a global list of helplines is available at blog.opencounseling.com/suicide-hotlines/.
Please! Just some level of actual accountability for these platforms. Enough is enough already, people (children) are actually dying!
Feeling shook feels like the proper response to *waves hand* all of this. It's also unfortunate that so many bright people give their time and skill sets to these companies because of the golden handcuffs.