I would rather not. While it is already highly questionable to use it normally because it steals opensource code, but let's give it a pass for this thought experiment, it probably scrapped the multiple git repository of Windows leaked source code. In which case it would ABSOLUTELY undermine the project's ability to say it's a clean room implementation
"it probably scrapped the multiple git repository of Windows leaked source code. In which case it would ABSOLUTELY undermine the project's ability to say it's a clean room implementation"
If an LLM model has been fed leaked code, then that is a general problem for that model and for its use for anything. Singling out its use for an open-source project and denouncing that as a potential problem while otherwise keeping quiet about it just makes no sense. Just take legal action against the model if there's anything plausible to warrant that, don't weaponize it against open-source projects.
All LLM have probably as they scrape github, and there are still to this day multiple Windows XP source code live on it (I won't give links but they are pretty easy to find). And I'd bet there is way more than just windows leaks on there...
Various versions of Windows have had their source code leaked out in part or almost whole. If Claude produces an exact copy, like LLMs used to do with the fast inverse square root from Doom, Microsoft would have good reason to sue and it'd be on the project to prove that the copyright violation was done by a bot (which makes it legal now).
With the project essentially implementing the entire API method by method, the chances of LLMs repeating some of the leaked source code would be tremendous.
A one-directional fork of ReactOS might be able to make some fast progress for a few people who desperately need certain programs to work, but I don't think the project will benefit from LLMs.
But, if any such model got fed with leaked code, then how is this a specific open-source project's problem and not of all projects (either open-source or private) that got to ever use that model?
Then, (having thought this just now) how can an argument relying on (legally) undisclosed information be used against anything public? Isn't the onus on the party having the undisclosed information to prove that it preceded the public one? How can that precedence be trusted by an independent judging party if the undisclosed information (source-code and systems managing that source code) is and always has been in the hands of the accusing (thus biased) party?
I think it's not ready yet but I agree that eventually it will be. The 40th anniversary of ReactOS might have some substantial features. This is the decade of ReactOS!
The new graphics driver stack they're touting (capable of running unmodified modern windows display drivers) along with support for x86_64 landing may result in increased interest in the project. They've already made a lot of progress with almost no resources as is. It's truly an impressive project.
Ask people to do things for you. Then you will learn how to work with something/someone who has faults but can overall be useful if you know how to view the interaction.
Though remember that it's not a human. It's easy to waste a lot of time convincing it to do something in a certain way, then one prompt later it forgets everything you said and reverts back to its previous behavior. (Yes humans can do that too, but not usually to this level).
It's important (though often surprisingly hard!) to remember it's just a tool, so if it's not doing things the way you want, start over with something else. Don't spend too much time on a lost cause.
I agree strongly with this take but find it hard to convince others of it. Instead, people keep thinking there is a magic bullet to discover resulting in a lot of wasted resources and money.
Autoencoders should output these kinds of splats instead of pixel outputs and likely obtain better representations of the world at the bottleneck. These features can be used for downstream tasks.
I am interested in doing research like this. Is there any way I can be a part of it or a similar group? I have been fighting for funding from DoD for many years but to no avail so I largely have to do this research on my own time or solve my current grant's problems so that i can work on this. In my mind, this kind of research is the most interesting and important right now in the deep learning field. I am a hard worker and a high-throughput thinking... how can i get connected to otherwise with a similar mindset?
I am glad they evaluated this hypothesis using weight decay which is primarily thought of to induce a structured representation. My first thought was that the entire paper was useless if they didn't do this experiment.
I find it rather interesting that the structured representations go from sparse to full to sparse as a function of layer depth. I have noticed that applying weight decay penalty as an exponential function of layer depth gives improved results over using a global weight decay.
Not quite. For an underlying semantic concept (e.g., smiling face), you can go from a basis vector [0,1,0,...,0] to the original latent space via a single rotation. You could then induce said concept by manipulating the original latent point by traversing along that linear direction.
I think we are saying the same thing. Please correct me though where I am wrong. You could look at the maps in some way but instead of the basis being one hot dimensions (the standard basis), it could be rotated.
