Hacker Newsnew | past | comments | ask | show | jobs | submit | Tokutek's commentslogin

Ok, the website is starting to come back online after our huge spike in traffic.

Thanks for all the interest and for your patience...

-Tokutek


What is your performance and/or scaling issue and what alternatives are you considering?

In this particular case the scaling needs and performance issues made TokuDB an economical decision. As the researcher noted in the piece: "With InnoDB [the default MySQL storage engine], going to larger servers, adding 100s of GBs of additional RAM along with many additional drives would have easily cost $20,000 or more, and still would not have addressed all our needs. TokuDB was by far both a cheaper and simpler solution.”


The alternative people normally have in mind when they make comments like "I wish I could afford X" is not "I guess I'll still do whatever it was X would have helped me with, but at much larger pain and cost" but "I guess I won't be able to do the thing X would have helped with at all".

There are many people who have interesting data-related ideas and problems (maybe for companies, maybe for research, or maybe just for fun) who are not going to be able to afford (as the company wouldn't be profitable enough, the research not grant-worthy enough, or with sufficiently little disposable income to play with) large servers with 100s of GB's of additional RAM.

So, one might imagine this same person, were he to have just read an article about a company that successfully managed to scale MySQL by buying the worlds largest computer to run it on from a company specializing in building them--let's call it the supremum-computer--would have made a very similar comment in that context: "I wish I could afford a supremum-computer".

Thereby, I will point out that while it is nice that your solution makes such tasks cheaper, possibly even by an order of magnitude, for people who are making enough money to be considering such things, and while I have no qualms with your business model and wish you a lot of luck, it feels like "rubbing it in our faces" to point out that if we were seriously embarking down one of these paths you are saving us money by buying your solution: that doesn't somehow mean we can afford it.


I run http://DNSDigger.com as a (big) hobby project but trying to commercialize it. I will add features and API access but as far i actually only made like $100USD from it since i started it ~2006. I only recently got a proper server for it but need beefier ones if i gonna expand. My row size is really small but in total i have over a billion rows. Insert speed is crucial as i update/insert new IP data in the millions per day and when the new features is in place i will grow minimum 100 million rows per month. I currently have all tables as MyISAM because insert speed is important and the DB would not fit on the 240GB SSD if i changed to InnoDB. So without any money now i have to use whatever tool i can.


To expand the MVCC list noted above, MySQL + TokuDB is always in MVCC as well (since version 5.0 last year). Also note that the Quora link describes Fractional Cascading. TokuDB uses Fractal Tree indexes. To learn more about Fractal Tree indexes as opposed to basic LSM, see here - http://www.tokutek.com/2011/09/write-optimization-myths-comp...


Good insight. As to your point -- keeping up many indexes with MyISAM or InnoDB is definitely a problem, especially if the indexes grow too big for RAM. In these cases, TokuDB (http://en.wikipedia.org/wiki/TokuDB), a drop in storage engine replacement for MySQL, can help by replacing dated B-tree technology with more modern Fractal Tree indexes, since B-trees suffer in performance with high insertion rates (especially random insertions) into disk. By allowing for faster write performance, Fractal Tree engines allow for more indexes to be maintained, and hence better query performance can be obtained.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: