What just happened? Another incident has taken place that illustrates the need to be careful what you tell AI. A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to “shoot up” the Sheriff’s office. After a human reviewer examined the statements, they were reported to police.
According to the arrest report, Carli Michelle Heller, of Bonita Springs, Florida, wrote on September 26 that she would attack the Sheriff’s office. She later said that she uses Anthropic’s chatbot like a “diary.”
Claude’s safety systems flagged the entry and it was escalated to a human reviewer. After deciding it was a credible threat, the reviewer reported it to law enforcement.
The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.
Deputies identified Heller and visited her home. She was detained without incident before an LCSO intelligence detective took over the investigation.
Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
Anthropic isn’t going to be taking any chances when it comes to anything it deems a potential threat. Last month, it was reported that OpenAI and Sam Altman are being sued by British Columbia over claims that the company could have prevented a mass shooting in the Canadian province.
The shooter, eighteen-year-old former pupil Jesse Van Rootselaar, had previously been flagged by OpenAI’s safety team for her conversations about gun violence, but OpenAI never alerted police because the conversations did not meet the threshold for legal referral.
In June, Florida also sued OpenAI and Altman, alleging that ChatGPT had contributed to real-world harms, including the 2025 Florida State University shooting.
The latest incident is another reminder to think before you enter something into a chatbot that could get you into trouble. It’s certainly not a private diary whose contents are for your eyes only.
Reports last month revealed that human contractors reviewing Microsoft Copilot’s image editor can see users’ prompts, uploaded photos and AI-generated edits. Documents show that some of those assignments contain sexual, disturbing or potentially illegal material, though the reviewers are not there to flag the content – only to assess whether the output is accurate.
And people were not-so-long-ago concerned that phones are spying after them. Now they just send all the info themselves.
Up until the 80s, there was still some debate over whether wiretaps should be legal. It’s sad that we have come to accept them as normal.
It was less worrisome back then. The cost to have someone listen to and transcribe your conversations was quite high. They’d really have to want to go after someone to do it.
Now, turning voice conversations into searchable text is nearly free.
Not only that - wiretaps have Judicial Oversight in that they require a Court Order (well, in most of the World, not sure if that’s the case in present day America) which a Judge supposedly only issues if there’s reason to suspect the person whose lines are tapped of committing or preparing to commit a crime, whilst Antrophic’s surveillance of people’s conversations with their chatbot has no Judicial Oversight at all and has no minimum legal threshold that must be reached before it is allowed to happen.
I think this is one of the greatest dangers AI presents. Sure it makes some of us stupider, sure it drinks our water and eats our GPUs and raises power prices… but once upon a time, it took actual human effort to run a dragnet, even with judicial approval.
Now, you plug in an LLM to the giant pile of corporate surveillance data and ask it to find troublemakers. Anonymity is dead and its death made retroactive and since there’s no government oversight, the government can just lease access to the tools they used to need approval to use.
Back in the day the 31337 found a way around the bots.
Too bad they weren’t more elite, or they could have prevented the modern internet from becoming what it is
People will fight you tooth and nail to prevent something they view as immediately threatening to them. On the other hand, they’ll pay you to implement something they view as immediately useful to them. In either case, they don’t care at all about any non-immediate repercussions for anything.
I’m building a fully private assistant with local Home Assistant, local model, and a local Wikipedia database for lookups. Will probably make a post/release of it soon once I finish replacing all my Google Nest displays.
I’d like to subscribe to your newsletter.
I want to do this, but only got as far as setting up a minimal Home Assistant. Only put a few hours into it, though, then got sidetracked by trying to turn an esp32 into a speaker/microphone puck.
Hah, you can watch my site: https://nowsci.com/ as I put everything there.
I’m very nearly at daily usable by everyone, including my partner who has a very different accent to mine. A lot of it was tuning sentences/words to catch poor STT to skip going to an LLM to speed things up and make it reliable. Here’s the current hardware in action:
@SuspiciousCarrot78@aussie.zone yes, Ive tried a lot of LLMs and have settled on Gemma 4 2b with a very specific prompt and local MCP tools for unit conversions, weather, lookups, etc.
For Q&A, my current build uses an MCP container I made to do a SearXNG search and top result retrieval for context (I use a trick to weight Wikipedia results). The final step I haven’t completed is for this to be a backup to Kiwix, for which I also haven’t found a great search tool/index/MCP, so if you want to collab on that I’d be interested. One less thing for me to build myself ;)
i know very few people who wereor are concerned about being spied on regardless of how much they should be.
I get called a downer for mentioning how this all works. You shouldn’t connect your smart tv to the network, you shouldn’t use Alexa, anything you do online is public, and it always gets recorded poorly.
Maybe my partner is the only person who listens, and she has all her stuff disabled, there are no network attached smart things in it house, but she still uses windows and she’s not yet receptive to moving to Linux (which is a hard sell).
deleted by creator
Smart TVs just find those “open” hotspots that cable providers leave on everyone’s modem now; you don’t even need to agree to TOS before it starts locating and spying.
Every device is a wiretap. Every device is a wiretap. We live in a panopticon. The purpose of AI is to price us out of the component market. You can’t escape if you can’t afford your own hardware.
I haven’t seen any proof of this, so I’m still skeptical.
I didn’t approve any TOS setting up my tv, so this would be legally a nightmare for them.
Gamers Nexus recently released a 2-hour video on LG smart TVs and how they spy on you. I haven’t watched the entire video yet, but it seems that the damn things start recording data as soon as they receive power.
so this would be legally a nightmare for them
To many corporations such as LG, laws are just the cost of doing business. Assuming they even receive a fine it will be so small that it won’t matter.
I just mean there’s enough tech illiterate people out there that they don’t need that liability.
95% of people will just turn on the tracking and not know or think about it. Connecting to nearby wifi is more of a risk.
I suppose it should be pretty easily verifiable. Just set up a new SSID without protection and see what connects.
Might as well try it with my own TV. IIRC I never accepted any TOS either.
I find it interesting that, if their AI gets out of control, there’s nothing they can do about it, but if a USER of that AI gets out of control, there is something they can do about it…
Also depends on how mega rich that USER is.
Why would you even use Claude as a fuckin diary?
I always knew it was going to happen. In fact, I won’t be surprised if the FBI starts coming up.
Telling the wire tap all your secrets, and expecting it to honor them?! is fucking next level delusional.
AI and idiots go together like peanut butter and jelly.
Americans doing anything but proper gun control
As a leftist gun owner. I’m currently not interested in handing over my weapons.
Cool, that’s not what gun control means
You morons always think some boogyman is coming for your guns. It’s con CONTROL, not gun CONFISCATION.
Kinda unrelated. But what happens when OpenAI or Anthropic’s loans come due and they have no legal way to turn a profit, but are sitting on a treasure trove of corporate secrets?
Are they sitting on a treasure trove of corporate secrets? Or is it just a thousand executives writing “Claude, synergize expediency to increase value for shareholders. Maximize usefulness. Implement Six Cigma methodologies to unlock a more agile work cycle. Generate a billion dollars in new cashflow. Make no mistakes.”
And bank account numbers
If I had to take a wild guess: Government bailouts
ZDR policies really are a must. If a service doesn’t have it, don’t use it.
ZDR?
They’re just going to be like “Google it, I’m not your Mom!” or something to that effect. This happens a lot here. Someone comes along, drops some niche term and doesn’t bother to explain it at all because they either don’t know how to imagine what it’s like to not know what they know, or they just don’t give a shit about not being understood by people who don’t already know.
You somehow managed to craft a reply that added even less value to the conversation than the one you’re complaining about. Amazing!
Stupid, stupid, stupid human. Well, those are the consequences of what she chose to do. Which is write a threat. Threats are not protected speech, or so the courts have decided. We created these circumstances, so we can either navigate them with care or run afoul and be chewed up and shat out by the institutional systems we ourselves built.
sigh.
it’s exhausting, really.
Writing a threat is not illegal. Hell people vent all the time, threatening to do things they know damn well they never will. The law says transmitting a threat to another person is illegal because it can be used as intimidation or conspiracy. I think there’s a damn good argument that she didn’t knowingly transmit it to anyone. She thought she was writing in confidence to a non-person. And if it’s not her first time expressing outrage to AI, she would know it would only try to deescalate her.
Who knows how the case will go. I can see it going either way — probably depending on how much money she has. Laws are going to have to catch up to technology in this case, and it’s going to be bumpy. I definitely agree with erring on the side of caution. Private, local AI is far safer, but of course it’s less capable for now.
The article very much states that it is in fact illegal to write down a threat under Florida law.
I think this is a combination of the law being poorly written and the woman in question making a significantly questionable decision. Her actions have consequences (even if I think that law is problematic at best). There’s no reason for her to have had an expectation that Claude chats would be private (we’ve had repeated news reports that prove otherwise).
The article very much states that it is in fact illegal to write down a threat under Florida law.
I don’t know that that’s my read here. The emphasis on the below quote from the article is mine.
Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism. The communication must be made in a manner in which another person may view it.
It sounds like it’s not illegal to write the threat down, but it’s illegal to share it with others. If you do keep a private journal, or write your feelings onto slips of paper and burn them (idk if this is actually common, but it’s always been recommended to me after a breakup), that’s not illegal, even if you describe wanting to do something illegal. It’s the sort of hair splitting that wouldn’t normally be relevant, but could make a huge difference here.
Of course, she probably will be legally considered to have sent, posted, and transmitted it, so she’s SOL on that end, but she could credibly (imo) claim she was unaware that another person might view it.
She did transmit such a threat though. Which is against the law quoted in the article.
Yes but not in a way she would think another person may view it, most likely. Obviously IANAL but surely intent matters here?
If people actively realised someone can read all their chats, I’m sure these services would be far less popular.
And here’s the autism slop because I can’t think of the proper term for “actively realising” and do not have the energy to make it readable anymore:
What I mean is, I’m sure plenty of people know in the back of their mind that their data is not protected there, but it’s not the first, or second, or probably even tenth thing they think about when starting a chat.https://vger.to/lemmy.world/comment/26171736
This is a link to my original comment on the subject. I did not say what you think I said in the comment you replied to or the original comment I linked. Her ignorance doesn’t absolve her of her actions or the consequences. Intent does matter to some extent but there have also been a lot of news reports about how data you share with corporate LLM’S isn’t private and so then the question becomes should she have known and not just if she did or didn’t know.
If someone else steals something and sells it to you or gifts it to you and you had no knowledge that it was stolen you can still be prosecuted under the law for receiving stolen goods. Your intent in the matter should be considered because there is a difference between knowingly receiving something you know is stolen and not reporting it, and just getting something without knowing it’s legal provenance and not reporting it. But you can still be charged either way.
In this case though an attempt was made to notify the user that her chats were not private in a EULA or TOS. They changed that TOS relatively recently (something like a year ago I think). At that time it looks like users would have been required to agree to an updated TOS with specific verbage about the sharing of chats that doesn’t look like it would have required users to read the entire TOS, just the agree popup. I linked the article for that below because while that was for allowing AI to train on user data, it should have been a clue that the chats were not private.
https://www.bgr.com/1953473/anthropic-train-claude-chats-how-opt-out/
https://www.anthropic.com/news/updates-to-our-consumer-terms
I appreciate the links! I still don’t think it’s a slam dunk either way, because even in Anthropic’s release, it never actually says that a human will be involved. “Training” could be anything, and while intent does not always matter, a threat is one of the few crimes where it really does.
“The company says it may share user information in limited emergencies if it believes disclosure is necessary to prevent death or serious physical injury.”
Also keep in mind: Your chat logs are company assets. Even if today’s decision makers believe maintaining user trust is net profitable, that will not always be true. Some day, maybe when the pullback comes and the wolves are at the gates, or when they sell the company, or when a shareholder action asserts fiduciary duty to maximize shareholder value, your chat logs will be sold and mined in the most lucrative way possible. The way that extracts the most value from you, the product.
when a shareholder action asserts fiduciary duty to maximize shareholder value
People really should stop spreading this fiction that there is some legal requirement to maximize shareholder profits embedded in the concept of fiduciary duty.
These aren’t the kind of saber-rattling disputes that make it to trial (or even go beyond the sternly worded letters stage). Neither side wants to fight it in court, because it’s a murky. The most noteworthy recent curtailing of it was when SCOTUS went 5-4 in favor of religion over profit in Hobby Lobby, which is hardly a strong case for board independence.
Self host if you’re gonna use AI, in fact, self host whenever possible. There are numerous examples of companies using centralized data to sell, report to police, or even having it fall into the hands of hackers.
I really think people think that all data input to a website is actually stored locally, nope, don’t be dumb.
Self host if you’re gonna use AI
or, you know, Don’t fucking use AI period cause its fucking stupid, and you don’t need to be consuming the power of the fucking sun to make a journal.
Fully this.
In my day to day life it improves nothing. I certainly could write a complicated harness over a local model to get it to automate a ton of my day to day tasks but… Why?
Why would I do that? Regular deterministic tools and regular programming languages already give me the ability to automate the stuff that is truly tedious. Aside from that most of the tasks that people seem to use it for are tasks were the friction of doing them is were all the benefit lies.
People using this shit for journaling or taking notes is INSANE.
Better yet… just don’t us AI
AI is good. It’s just that the people behind it suck.
Fuck AI there is nothing good about anything going on with AI right now. And please don’t go into the whole “bUt ThErEs DiFfErEnT kInDs So ThEy’Re NoT aLl BaD” because we know. We fucking know.
But the mass majority of AI use is fucking bullshit so please don’t cape for this insanity because 1% of AI isn’t bad.
Can someone actually self host a reasonable model on a standard PC?
Depends on what you need. You can get away with hosting QWEN 27B on a laptop with about 64 GB or RAM and a dedicated GPU. You’ll be using mostly system RAM, not VRAM, so generation would be slow, but not unbearable.
Good for creating short stories and other creative work when paired with a front end that can handle character profiles, world building, and has memory recall.
It’s better to build your own server and set it up so you, and only you, can remotely access it.
