you may want to re-evaluate your understanding of the word hackers. It has had a few definitions during my lifetime and none of the fit with how you have used it here.
My understanding of hackers is entirely based on people on this message board.
And of course hackers and other similarly minded people would be outraged if something they bought on Amazon could be found for cheaper elsewhere, and then the company would have to process a thousand percent increase in returns as well as losing the sales.
Why so negative lol. The speed and very reduced power use of this thing are nothing to be sneezed at. I mean, hardware accelerated LLMs are a huge step forward. But yeah, this is a proof of concept, basically. I wouldn't be surprised if the size factor and the power use go down even more, and that we'll start seeing stuff like this in all kinds of hardware. It's an enabler.
You don't know. You just have marketing materials, not independent analysis. Maybe it actually takes 2 years to design and manufacture the hardware, so anything that comes out will be badly out of date. Wouldn't be the first time someone lied. A good demo backed by millions of dollars should not allow such doubts.
Did you not see the chatbot they posted online (https://chatjimmy.ai/)? That thing is near instantaneous, it's all the proof you need that this is real.
And if the hardware is real and functional, as you can independently verify by chatting with that thing, how much more effort would it be to etch more recent models?
The real question is of course: what about LARGER models? I'm assuming you can apply some of the existing LLM inference parallelization techniques and split the workload over multiple cards. Some of the 32B models are plenty powerful.
I mean, you're right but at the same time you're talking about something completely different. Software with security vulnerabilities is not a useful product. You don't address the raised issues.
I'm completely ignorant about this, but wouldn't it be possible to compile separately your project to improve compilation times? for instance, if you're using OP's vector library, which is self contained, you could compile that first and just once?
int add(int a, int b){
// Long logic and then this
return a+b;
}
Let's say this is your main.c.
#include "add.h"
int main(void) {
return add(5,6);
}
The preprocessor just copies the contents of add.h into your main.c whenever you're trying to compile main.c. (let's ignore the concept of precompiled headers for now).
What you can instead do is just put the add function declaration in add.h that just tells the compiler that add function takes two integers and returns an integer.
int add(int a, int b);
You can then put the add function definition in add.c , compile that to an add.o and link it to your main.o at link time to get your final binary - without having to recompile add.o every time you change your main.c.
reply