Hacker Newsnew | past | comments | ask | show | jobs | submit | linl's commentslogin

Thanks for sharing. I helped write this guide. My team and I collectively wrote connectivity for around 80-140 trading venues at major trading firms before this, and we tried to distill it down to the basics you need to know. Happy to answer any other questions!


Recently our team at Databento was debugging a memory leak on an experimental debugging branch. For context, this is on a Rust application that distributes billions of market data messages per day and we're using shared memory/mmap queues for interprocess communication. It turned out to be a false alarm due to adding tokio-console to a component.


I agree with this. B-trees vs hash indices, I/O hierarchy, process model, etc.

Also in the modern day, it's valuable to learn common strategies behind column-oriented databases such as late tuple materialization, deferred execution, linear scans vs. binary search, instruction pipelining.

Once you're familiar with all this, you'd find sometimes in the field what you really need is a simple flat file or an embeddable database like RocksDB, not a DBMS.


>"I agree with this. B-trees vs hash indices, I/O hierarchy, process model, etc."

I'm familiar with all of the concepts in the context of databases except "process model." Is this the same as data model? Could you elaborate if not?


There are several types of process models that's suitable for a DBMS: 1 process per DBMS worker, 1 thread per DBMS worker, a lightweight thread pool, a process pool, etc.

Hellerstein, Stonebraker and Hamilton (2007) is a great introduction on this.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: