Meta's Llama model has memorized Harry Potter and Future Sex (2018)the Sorcerer's Stoneso well that it can reproduce verbatim excerpts from 42 percent of the book, according to a new study.
Researchers from Stanford, Cornell, and West Virginia University analyzed dozens of books from the now-infamous Books3 dataset, a collection of pirated books used to train Meta's Llama models. Books3 is also at the center of a copyright infringement lawsuit against Meta,Kadrey v. Meta Platforms, Inc.The study's authors say their findings could have major implications for AI companies facing similar lawsuits.
According to the research paper, the Llama 3.1 model "memorizes some books, like Harry Potterand 1984, almost entirely." Specifically, the study found that Llama 3.1 has memorized 42 percent of the first Harry Potter book so well that it can reproduce verbatim excerpts at least 50 percent of the time. Overall, Llama 3.1 could reproduce excerpts from 91 percent of the book, though not as consistently.
"The extent of verbatim memorization of books from the Books3 dataset is more significant than previously described," said the paper. But the researchers also discovered that "memorization varies widely from model to model and from book to book within each model, as well as varying in different parts of individual books." For example, the study estimated that Llama 3.1 only memorized 0.13 percent of Sandman Slimby Richard Kadrey, one of the lead plaintiffs in the class action copyright suit against Meta.
So, while some of the paper's findings seem damning, don't call it a smoking gun for plaintiffs in AI copyright infringement cases.
"These results give everyone in the AI copyright debate something to latch on to," wrote journalist Timothy B. Lee in his Understanding AI newsletter. "Divergent results like these could cast doubt on whether it makes sense to lump J.K. Rowling, Richard Kadrey, and thousands of other authors together in a single mass lawsuit. And that could work in Meta’s favor, since most authors lack the resources to file individual lawsuits."
Why is Llama able to reproduce some books more than others? "I suspect that the difference is because Harry Potter is a much more famous book. It's widely quoted and I'm sure that substantial excerpts from it on third-party websites found their way into the training data on the web," said James Grimmelmann, a professor of digital and information law at Cornell University, who was cited in the paper.
What this also shows, Grimmelmann said, is that "AI companies can make choices that increase or reduce memorization. It's not an inevitable feature of AI; they have control over it."
Meta and other AI companies have argued that using copyrighted works to train their models is protected under fair use, a complex legal doctrine. However, the extent of memorization could complicate those arguments.
“Yes, I do think that the likelihood that LLMs are memorizing more than previously thought changes the copyright analysis,” Robert Brauneis, a professor with the George Washington University Law School, said in an email to Mashable. He concluded that the study’s findings could ultimately weaken Meta’s fair use argument.
We asked Meta for comment on the study's findings, and we'll update this article if we receive a response.
Disclosure: Ziff Davis, Mashable’s parent company, in April filed a lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.
Topics Artificial Intelligence Meta
Today's Hurdle hints and answers for April 7, 2025
Geoff Dyer on 'Otherwise Known as the Human Condition' by Evan Ratliff
Small Fry
The first images of Earth are chilling
Outdoor speaker deal: Save $20 on the Soundcore Boom 2
Science Will Not Save Us
Hard Knock Life
Kay Ryan Wins Pulitzer by Lorin Stein
Best Kindle Unlimited deal: Get 3 months of Kindle Unlimited for 99 cents
Science Will Not Save Us
Best Sony headphones deal: Over $100 off Sony XM5 headphones
Biden’s Final Humiliation
Sinner vs. Rublev 2025 livestream: Watch French Open for free
Perfecting the Art of Pedantry
Best Max streaming deal: Save 20% on annual subscriptions
Hello, Brave New World
Sick, Sad World
5 planets will light the sky in rare astronomical event this week
Musetti vs. Diallo 2025 livestream: Watch Madrid Open for free
The Southwest Syndrome
'Deadpool 2' casts 'Atlanta' star Zazie Beetz as DominoTesla owners drop serious cash on their cars only to wait months to get them repairedSmall child meets the Queen, throws wildly unimpressed tantrumThis store celebrated women by offering discounts for frying pansThe 'Love Actually' sequel will feature more of Andrew Lincoln stalking peopleThis sheep with its head stuck in a straw bale is our spirit animalLooks like Matt Damon really could've grown potatoes on MarsMalaysian bank offers yearKid who created his own campaign posters sure knows how to troll an electionHarry Potter robotics master lets a hyper realistic leopard on the loose in LondonTwitter is now marking entire profiles as 'sensitive'Disney's 'Beauty and the Beast' will probably break a box office recordLorde blesses us all with another new song and album release dateInspired by Chance the Rapper, NFL star pledges to donate all his endorsement moneyDog valiantly trying to catch a toy gets the Photoshop battle he deservesWikileaks says it wants to help tech companies thwart CIA hackingSaturn's tiny, bizarre moon Pan looks like a flying saucer... or Trump's faceIvanka Trump's brand claims sales spiked after White House controversyNicole Kidman has a pretty legit excuse for her weird seal clap at the OscarsClever dude masterfully Photoshops himself into every Best Picture Nominee poster Catch yourself a date on new 'Pokémon Go' dating site The best 'A Star Is Born' memes Republican delegate calls out GOP on anti Teddy Perkins from 'Atlanta' showed up at the Emmys and whaaaaat?? You won't believe how much Selena Gomez's social media posts are worth Twitter will make it easy to switch back to a chronological timeline Universal Studios Halloween Horror Nights 'Stranger Things': Review Cathay Pacific doesn't give an F, spells its name wrong on plane Samsung's new A7 phone might have a triple Netflix announces live Trump campaign manager believes he knows why women will vote for the GOP nominee Attorney General blasts cryptocurrency exchanges Bernie Sanders is watching the Republican convention in the Berniest way possible No one believes Trump speechwriter Meredith McIver is real People are sharing the meanest backhanded compliments they've received Magic Leap vs. Microsoft HoloLens: Which is the better AR experience? The last five Augusts on Earth have been the five hottest on record Comedian trolls the Emmys red carpet with a Getty Images purse Trump Pornhub searches increased 648% in Cleveland during Republican convention Here are all the new Android smartphones coming in October
1.6181s , 10134.34375 kb
Copyright © 2025 Powered by 【Future Sex (2018)】,Inspiration Information Network