Hopefully, future legislation will cater less to publishers and copyright trolls. I'm not optimistic though. While certain kinds of publishers are indeed becoming less powerful, sports-related media conglomerates are successfully lobbying for more surveillance.
The general population will likely get the worst of both worlds, with copyright trolls getting to enforce unjust laws against regular people, while big tech gets to pay their way out.
> and no crime was prevented by harassing local workers.
Siezing records is usually a major step in an investigation. Its how you get evidence.
Sure it could just be harrasment, but this is also how normal police work looks. France has a reasonable judicial system so absent of other evidence i'm inclined to believe this was legit.
Internet routers, network cards, the computers, OS and various application software have no guardrails and is used for all the nefarious things. Why those companies aren't raided?
May be.
We do have codified in law definition of machine gun which clearly separates it from a block of lead. What codified in law definitions are used here to separate photoshop from Grok in the context of those deepfakes and CSAM?
Without such clear legal definitions going after Grok while not going after photoshop is just an act of political pressure.
They don’t provide a large platform for political speech.
This isn’t about AI or CSAM (Have we seen any other AI companies raided by governments for enabling creation of deepfakes, dangerous misinformation, illegal images, or for flagrant industrial-scale copyright infringement?)
No because most of those things aren't illegal and most of those companies have guard rails and because a prosecution requires a much higher standard of evidence than internet shitposting, and only X was stupid enough to make their illegal activity obvious.
It is very different. It is YOUR 3d printer, no one else is involved. You might print a knife and kill somebody with it, you go to jail, not third party involved.
If you use a service like Grok, then you use somebody elses computer / things. X is the owner from computer that produced CP. So of course X is at least also a bit liable for producing CP.
The safe harbor provisions largely protect X from the content that the users post (within reason). Suddenly Grok/X were actually producing the objectionable content. Users were making gross requests and then an LLM owned by X, using X servers and X code would generate the illegal material and then post it to the website. The entity responsible is no longer done user but instead the company itself.
So, if someone hosts an image editor as web app, are they liable if someone uses that editor to create CP?
I honestly don't follow it. People creating nudes of others and using the Internet to distribute it can be sued for defamation, sure. I don't think the people hosting the service should be liable themselves, just like people hosting Tor nodes shouldn't be liable by what users of the Tor Network do.
Also, safe harbor doesn't apply because this is published under the @grok handle! It's being published by X under one of their brand names, it's absurd to argue that they're unaware or not consenting to its publication.
I'm not trying to make excuses for Grok, but how exactly isn't the user creating the content? Grok doesn't have create images on its own volition, the user is still required to give it some input, therefore "creating" the content.
X is making it pretty clear that it is "Grok" posting those images and not the user. It is a separate posting that comes from an official account named "Grok". X has full control over what the official "Grok" account posts.
There is no functionality for the users to review and approve "Grok" responses to their tweets.
It's not like the world benefited from safe harbor laws that much. Why don't just amend them so that algorithms that run on server side and platforms that recommend things are not eligible.
If a user generates child porn on their own and uploads it to a social network, the social network is shielded from liability until they refuse to delete it.
Changes who's technically the perp. Offer managed porn generation service -> service provider is responsible for generating porn, like, literally literally
This might be an unpopular opinion but I always thought we might be better off without Web 2.0 where site owners aren’t held responsible for user content
If you’re hosting content, why shouldn’t you be responsible, because your business model is impossible if you’re held to account for what’s happening on your premises?
Without safe harbor, people might have to jump through the hoops of buying their own domain name, and hosting content themselves, would that be so bad?
What about webmail, IM, or any other sort of web-hosted communication? Do you honestly think it would be better if Google were responsible for whatever content gets sent to a gmail address?
Messages are a little different than hosting public content but sure, a service provider should know its customers and stop doing business with any child sex traffickers planning parties over email.
I would prefer 10,000 service providers to one big one that gets to read all the plaintext communication of the entire planet.
In a world where hosting services are responsible that way, their filtering would need to be even more sensitive than it is today, and plenty of places already produce unreasonable amounts of false positives.
As it stands, I have a bunch of photos on my phone that would almost certainly get flagged by over-eager/overly sensitive child porn detection — close friends and family sending me photos of their kids at the beach. I've helped bathe and dress some of those kids. There's nothing nefarious about any of it, but it's close enough that services wouldn't take the risk, and that would be a loss to us all.
They'd all have to read your emails to ensure you don't plan child sex parties. Whenever a keyword match comes up, your account will immediately be deleted.
EncroChat was illegal because it was targeted at drug dealers, advertised for use in drug dealing. And they got evidence by texting "My associate got busted dealing drugs. Can you wipe his device?" and it was wiped. There's an actual knowledge component which is very important here.
On the contrary HN is one of the most heavily moderated forums out here and serves as a great example of a right-size community where sex trafficking is not happening under the nose of an ambivalent host
(Snark aside, in your opinion are there comments on HN that dang would be criminally liable for if it weren't for safe harbor?)
The 3D printers don't generate the plans for the gun for you though. If someone sold a printer that would – happily with no guardrails – generate 3D models of CSAM from thin air and then print them, I bet they'd be investigated too. Or for that matter a 3D printer that came bundled with a built-in library of gun models you could print with very little skill...
I don't have an answer, but the theme that's been bouncing around in my head has been about accessibility.
Grok makes it trivial to create fake CSAM or other explicit images. Before, if someone spent a week on photoshop to do the same, It won't be Adobe that gets the blame.
Same for 3D printers. Before, anyone could make a gun provided they have the right tools (which is very expensive), now it's being argued that 3D printers are making this more accessible. Although I would argue it's always been easy to make a gun, all you need is a piece of pipe. So I don't entirely buy the moral panic against 3D printers.
Where that threshold lies I don't know. But I think that's the crux if it. Technology is making previously difficult things easier, to the benefit of all humanity. It's just unfortunate that some less-nice things have also been included.
I think a company which runs a printing business would have some obligations to make sure they are not fulfilling print orders for guns. Another interesting example are printers and copiers, which do refuse to copy cash. Which is partly facilitated with the EURion constellation (https://en.wikipedia.org/wiki/EURion_constellation) and other means.
Sure, and the fact that they haven't voluntarily put guard rails up to stop that is absolutely vile. But my personal definition of "absolutely vile" isn't a valid legal standard. So, the issue is, like I said, how do you come up with a principled approach to making them do it that doesn't have a whole bunch of unintended consequences?
Courts are able or should be able to distinguish "tool that creates an item in the privacy of your home" and "tool that disseminates nonconsensual pornographic picture to wide public". Legal standards with that level or definition are fairly normal.
i don't see any need for guardrails, other than making the prompter responsible for the output of the bot, particularly when it's predictable
you cannot elaborately use a software to produce an effect that is patently illegal and accurate to your usage, and then pretend the software is to blame
Not really, they put a shit ton of effort in to make sure you can't create any kind of nude/suggestive pictures of anyone. I imagine they have strict controls on making images of children, but I don't feel inclined to find out.
> The company made and released a tool with seemingly no guard-rails, which was used en masse to generate deepfakes and child pornography.
Do you have any evidence for that? As far as I can tell, this is false. The only thing I saw was Grok changing photos of adults into them wearing bikinis, which is far less bad.
That's why this is an investigation looking for evidence and not a conviction.
This is how it works, at least in civil law countries. If the prosecutor has reasonable suspicious that a crime is taking place they send the so-called "judiciary police" to gather evidence. If they find none (or they're inconclusive etc...) the charges are dropped, otherwise they ask the court to go to trial.
On some occasions I take on judiciary police duties for animal welfare. Just last week I participated in a raid. We were not there to arrest anyone, just to gather evidence so the prosecutor could decide whether to press charges and go to trial.
Note that the raid itself is a punishment. It's normal for them to seize all electronic devices. How is X France supposed to do any business without any electronic devices? And even when charges are dropped, the devices are never returned.
For obvious reasons, decent people are not about to go out and try to general child sexual abuse material to prove a point to you, if that’s what you’re asking for.
First of all, the Guardian is known to be heavily biased again Musk. They always try hard to make everything about him sound as negative as possible. Second, last time I tried, Grok even refused to create pictures of naked adults. I just tried again and this is still the case:
The claim that they released a tool with "seemingly no guardrailes" is therefore clearly false. I think what instead has happened here is that some people found a hack to circumvent some of those guardrails via something like a jailbreak.
> Also, X seem to disagree with you and admit that CSAM was being generated
That post doesn't contain such an admission, it instead talks about forbidden prompting.
> Also the reason you can’t make it generate those images is because they implemented safeguards since that article was written:
That article links to this article: https://x.com/Safety/status/2011573102485127562 - which contradicts your claim that there were no guardrails before. And as I said, I already tried it a while ago, and Grok also refused to create images of naked adults then.
> That post doesn't contain such an admission, it instead talks about forbidden prompting.
In response to what? If CSAM is not being generated, why aren't X just saying that? Instead they're saying "please don't do it."
> which contradicts your claim that there were no guardrails before.
From the linked post:
> However content is created or whether users are free or paid subscribers, our Safety team are working around the clock to add additional safeguards
Which was posted a full week after the initial story broke and after Ofcom started investigative action. So no, it does not contradict my point which was:
> Also the reason you can’t make it generate those images is because they implemented safeguards since that article was written:
As you quoted.
I really can't decide if you're stupid, think I and other readers are stupid, or so dedicated to defending paedophilia that you'll just tell flat lies to everyone reading your comment.
Leave your accusations for yourself. Grok already didn't generate naked pictures of adults months ago when I tested it for the first time. Clearly the "additional safeguards" are meant to protect the system against any jailbreaks.
Grok do seem to have tons of useless guardrails. Reportedly you can't prompt it directly. But also reportedly they tend to go for almost nonsensically off-guardrail interpretation of prompts.
Well, there is evidence that this company made and distributed CSAM and pornographic deepfakes to make a profit. There is no evidence lacking there for the investigators.
So the question becomes if it was done knowingly or recklessly, hence a police raid for evidence.
See also [0] for a legal discussion in the German context.
> Well, there is evidence that this company made and distributed CSAM
I think one big issue with this statement – "CSAM" lacks a precise legal definition; the precise legal term(s) vary from country to country, with differing definitions. While sexual imagery of real minors is highly illegal everywhere, there's a whole lot of other material – textual stories, drawings, animation, AI-generated images of nonexistent minors – which can be extremely criminal on one side of an international border, de facto legal on the other.
And I'm not actually sure what the legal definition is in France; the relevant article of the French Penal Code 227-23 [0] seems superficially similar to the legal definition of "child pornography" in the United States (post-Ashcroft vs Free Speech Coalition), and so some–but (maybe) not all–of the "CSAM" Grok is accused of generating wouldn't actually fall under it. (But of course, I don't know how French courts interpret it, so maybe what it means in practice is something broader than my reading of the text suggests.)
And I think this is part of the issue – xAI's executives are likely focused on compliance with US law on these topics, less concerned with complying with non-US law, in spite of the fact that CSAM laws in much of the rest of the world are much broader than in the US. That's less of an issue for Anthropic/Google/OpenAI, since their executives don't have the same "anything that's legal" attitude which xAI often has. And, as I said – while that's undoubtedly true in general, I'm unsure to what extent it is actually true for France in particular.
It wouldn't be called CSAM in France because it would be called a French word. Arguing definitions is arguing semantics. The point is, X did things that are illegal in France, no matter what you call them.
> It wouldn't be called CSAM in France because it would be called a French word. Arguing definitions is arguing semantics.
The most common French word is Pédopornographie. But my impression is the definition of that word under French law is possibly narrower than some definitions of the English acronym “CSAM”. Canadian law is much broader, and so what’s legally pédopornographie (English “child pornogaphy”) in Canada may be much closer to broad “CSAM” definitions
> The point is, X did things that are illegal in France, no matter what you call them.
Which French law are you alleging they violated? Article 227-23 du Code pénal, or something else? And how exactly are you claiming they violated it?
Note the French authorities at this time are not accusing them of violating the law. An investigation is simply a concern or suspicion of a legal violation, not a formal accusation; one possible outcome of an investigation is a formal accusation, another is the conclusion that they (at least technically) didn’t violate the law after all. I don’t think the French legal process has reached a conclusion either way yet.
One relevant case is the unpublished Court of Cassation decision 06-86.763 dated 12 septembre 2007 [0] which upheld a conviction of child pornography for importing and distributing the anime film “Twin Angels - le retour des bêtes célestes - Vol. 3". [0] However, the somewhat odd situation is that it appears that film is catalogued by the French national library, [1] although I don’t know if a catalogue entry definitively proves they possess the item. Also, art. 227-23 distinguishes between material depicting under 15s (illegal to even possess) and material depicting under 18s (only illegal to possess if one has intent to distribute); this prosecution appears to be have been brought under the latter category only-even though the individual was depicted as being under 15-suggesting this anime might not be illegal to possess in France if one has no intent to distribute it.
But this is the point - one needs to look at the details of exactly what the law says and how exactly the authorities apply it, rather than vague assertions of criminality which might not actually be true.
> And I think this is part of the issue – xAI's executives are likely focused on compliance with US law on these topics, less concerned with complying with non-US law
True, but outright child porn is illegal everywhere (as you said) and the borderline legal stuff is something most of your audience is quite happy to have removed. I cannot imagine you are going to get a lot of complaints if you remove AI generated sexual images of minors, for example so it seems reasonable to play it safe.
> That's less of an issue for Anthropic/Google/OpenAI, since their executives don't have the same "anything that's legal" attitude which xAI often has.
This is also common, but it is irritating too as it means the rest of the world is stuck with silly American attitudes about things like nudity and alcohol - for example Youtube videos blurring out bits of Greek statues because they are scared of being demonetised. These are things people take kids to see in museums!
To me, the most worrying part of the whole discussion is that your comment is pretty much the most "daring", if you can call it that, attempt to question if there even is a crime. Everyone else is worried about raids (which are normal whenever there is an ongoing investigation, unfortunate as it may be to the one being investigated). And no one dares to say, that, uh, perhaps making pictures on GPU should not be considered a crime in the same sense as human-trafficking or production of weapons are... Oh, wait. The latter is legal, right.
France prosecutors use police raids way more than other western countries. Banks, political parties, ex-presidents, corporate HQs, worksites... Here, while white-collar crimes are punished as much as in the US (i.e very little), we do at least investigate them.
> This looks like plain political pressure. No lives were saved, and no crime was prevented by harassing local workers.
I wouldn't even consider this a reason if it wasn't for the fact that OpenAI and Google, and hell literally every image model out there all have the same "this guy edited this underage girls face into a bikini" problem (this was the most public example I've heard so I'm going with that as my example). People still jailbreak chatgpt, and they've poured how much money into that?
Comparing Apples and Oranges. Defending this company is becoming cringe and ridiculous. X effed up, and Musk did it on purpose. He uses CSAM to strongman the boundaries of the law. That's not worth defending unless you also say eff the rule of law.
Aren't a lot of US pickup trucks basically that? Sure, maybe there's a mechanism for preventing you from installing a baby seat in reverse to position in front of an airbag, but they're also built so that you can't see anything adult human sized 15m in front of the car, let alone anything child-sized.
The US would spend 20 years arguing about which agency's jurisdiction it was, and ignore the dead babies?
No, wait, Volvo is European. They'd impose a 300% tariff and direct anyone who wanted a baby-killing model car to buy one from US manufacturers instead.
Let's raid car companies too. We were all born into this. We never had a vote. Thomas Jefferson is said to have written Constitutions ought to be re-written every so often or the dead rule by fiat decree. Let's.
The rich can join in the austerity too. No one voted for them. We been conditioned to pick acquiescence or poverty. We were abused into kowtowing to a bunch of pants shitting dementia addled olds educated in religious crack pottery. Their economic and political memes are just that, memes, not immutable physical truth.
In America, as evidenced by the public not in the streets protesting for single payer comprehensive healthcare, we clearly don't want to be on the hook for each other's lives. That's all platitudes and toxic positivity.
Hopes and prayers, bloodletting was good enough for the Founders!
> I can’t imagine holding a job where I had to do work that I expect will fail. Sounds absolutely depressing. What keeps you motivated?
The paycheck. I had never expected work to NOT be depressing by definition, though. The only reason I'm working on what my employee wants me to is because I can't afford to live otherwise. They'll get the minimal effort needed for me to not get fired, but not a single minute more.
People who are in the country illegally undermine the rule of law and make people feel unheard leaving people to elect demagogues like trump. I predicted exactly this outcome with the election of Obama and his policies. How can the supposedly educated be so bad at applying history.
> People who are in the country illegally undermine the rule of law
No, they do not. Quite the opposite, they want to live their lives quietly and normally like everyone else.
The only law they "undermine" is the one forbidding them to living in a place due to a birth right. Civil disobedience to unjust laws like that is correct and moral.
> make people feel unheard leaving people to elect demagogues like trump
Disagree. IMO the biggest reasons for Trump's election were inflation and raising living costs. Under those conditions, picking a group of people to elect as the "enemy" is a tact to win elections, but has nothing to do with the underlying cause for the bad conditions in the first place.
Given the choice, end users choose free or cheap and ad supported over full price in huge majorities. You have to weigh "I don't like ads" against 200 million (!) people on Netflix's ad supported plan and how much enjoyment they get that they might not otherwise. Not to mention things like Google that are ad supported and genuinely useful. In the real world things have pros and cons.
I used to buy this thinking, but no longer. People are incredibly resourceful, and instead of innovating towards exploiting and manipulating people, we could choose to innovate towards conserveration of important things, just like we have done in the past.
We don't fund out national parks with advertisements. We don't fund our libraries with advertisements. We could create the same structures for the internet as well, where crucial internet resources are protected and stewarded. They don't necessarily need to be in the hands of ad companies.
Sure, I will not deny that having things be "free" (and paying for them in other ways) has been a huge boon from one perspective, but we can also evolve to put "free" things in different places. Because things are never free. Advertisements are funding mass surveillance. They are encroaching our civil liberties and normalizing it. There is a total cost to things that extens beyond money. What we don't pay out of pocket we pay as a society.
> It’s possible to simultaneously believe that ICE has a clear and ethical mandate
... "We" (a lot of people, not everyone who posts here) don't believe that. Lots of people disagree with immigration control as a concept period.
The existence of that app is an abomination; the fact tax payer money is being allocated to it is tragicomic. Not spending it and just giving it as tax returns to the population would be so much better than kidnapping people over being born in the wrong place.
> ... "We" (a lot of people, not everyone who posts here) don't believe that. Lots of people disagree with immigration control as a concept period.
I mean sure but you have to acknowledge that is an extremely fringe belief that basically no one in the USA supports. The debate is on "how" it's being done not that we shouldn't have immigration control.
> that is an extremely fringe belief that basically no one in the USA supports
Clearly is it not a belief that no one in the USA supports, as seem in the discourse against ICE and immigration contrl.
> The debate is on "how" it's being done not that we shouldn't have immigration control.
Not necessarily, no. "The debate" is too vague to elaborate in favor or against what you're saying.
But yes, there are people against immigration control period, and period in favor of reforms to make immigration easier for workers, not harder. But propaganda will keep putting workers against each other, instead of companies lobbying against workers.
All of this misses the point of the moment, which is that the federal government is completely lawless and is incapable of responding to democratic or popular will. There is no debate happening. It does not matter which ordinary people "support" which position. Any political project (other than the current regime) in the USA in 2026 must contend with the fact that just establishing a democracy must be our first step. This is as true for Socialists as it is for non-regime-approved stripes of Fascist. It's the same for Chamber of Commerce Republicans and ex-hippie boomer liberals. Any talk of what we will do with a democracy once we have it is premature, because at the moment it simply does not matter what the opinions of the citizenry are.
Why is this being downvoted? The primary reason Trump was able to win is because Biden waited until it was far too late to address the surge of illegal immigration at the southern border. We don't have to wonder or argue about whether Americans support open borders, we already had something mildly in that direction (that still didn't remotely approach the idea of "no immigration control, period"), and in response Americans voted into office Donald Trump.
> You know good and well that what is being discussed is the _use_ of LLMs
Not the person you're replying to, but I've found that some people do argue against LLMs themselves (as in, the tech, not just the usage). Specially in humanities/arts cycles which seem to have a stronger feeling of panic towards LLMs.
Clarifying which one you're talking about can save a lot of typing/talking some times.
> I've found that some people do argue against LLMs themselves (as in, the tech, not just the usage). Specially in humanities/arts cycles which seem to have a stronger feeling of panic towards LLMs.
Maybe?
The person I responded to said "LLMs are just a concept, an abstraction."
Were that true, were they simply words in some dusty CS textbook, it's hardly likely that the humanities/arts people you describe would even know about them.
No, it's the fact that these people have seen regurgitated pictures and words that makes it an issue.
That's the charitable coloring. Owning concepts or ideas, and trying to police others' use of ideas you """own""" is absurd.