Hey there, I wrote a simple AI grammar checker.
Research on AI applications in the field of Natural Language Processing (NLP), including grammar errors correction, is growing quickly. However, despite this progress, easily accessible tools are still missing on the web.
With this project, I'm trying exactly to cover this gap, a web app with an easy-to-use interface, powered with an artificially intelligent algorithm and trained on common errors made by not native English speakers.
Would be nice to get some feedback from you!
Thanks,
100%. I've been going to their meetups for a few years now.
If there's anyone in London who is interested in Java, give it a go. The meetups have a variety of topics, covering many skill levels. You can go sit at the back and slink off if you want, or sometimes they have pizza/beer and you can chat afterwards.
That always amazed me. Why people keep torturing themselves applying Java to the usecase where Java is clearly not a right technology. I've watched very interesting talk by LMAX people and half of it was about how to overcome garbage collection gaps, latency, etc.
I've researched this some a few months back. I don't blame you for saying that Java is wrong for HFT. My first reaction was the same, but there's much more to it.
Java has a few things going for it in HFT. The obvious pluses are it's mature and memory safe. What's less obvious is that you can make it low-latency. It takes a lot of work, but it's doable, at which point you have all the nice things: mature ecosystem, speed, latency, safety. It takes a lot of work, because Java was always oriented towards server use cases, as in high-throughput, not low-latency. That's changing by the way, there are two new GC engines coming out that are low-latency oriented. Also, there's been third-party JVMs with low-latency guarantees for quite a while.
Of course, what's between the lines is that there isn't any easy answers for HFT people. You either choose mature safety and do gc gymnastics (because everything is throughput oriented), or you choose manual memory management, which is its own gymnastics.
Anyway, that's my take. I welcome input and contradictions.
Mostly I agree, but there is one factor that makes C++ (or any native compiled language) better than java for HFT: the ability to lie to the optimizer about what the hot path is. in HFT you have thousands of no trades for every trade, so the java optimizer will optimize the no-trade code path as more likely, then when the trade happens java pays a CPU branch prediction miss penalty at the only time low latency matters.
You have to have good algorithms optimized to the max for this to matter though.
If you have a low-latency trading component written in Java, a common trick is to continuously bombard it with ‘fake’ inputs to keep the desired code paths nice and hot.
The fake inputs should be virtually indistinguishable from real ones that you would normally act on. The more subtle the difference, the better, e.g., just flip the sign on the timestamp field.
You can use that subtle difference to pick whether the order goes out to the real exchange or a fake exchange. The decision needs to avoid actual branching instructions, though, or the JVM will likely optimize out the ‘real’ hot path, and you’ll fall back to interpreted mode when an actionable ‘real’ event comes in. I usually use a branchless selector to index into an array ([0] goes to a real socket, [1] goes to a black hole socket).
You can also use this technique to make sure you can respond to very rare events quickly. For example, you may want to respond to news signals from Bloomberg. Actionable news is rare, so if you want to keep your news parsing/analysis code warmed up and in the cache, it needs to constantly be reacting to warmup data.
Thankfully, no (knocks on wood). But you have to base your design around the idea that real and fake trades are indistinguishable until the last possible moment. That critical requirement needs to always be in your mind.
I would never try to bolt those kinds of optimizations onto an existing system. It’d be too easy to miss something.
I was with you until you mentioned branch prediction... isn't branch prediction a hardware feature? How do you trick the HW branch predictor into predicting the unlikely case?
The cpu still needs to load code in via instruction cacheline fetches. For every instruction fetch, that core isn't doing much.
The compiler alleviates this somewhat by putting the hot path right under the branch instruction so that the fetch that grabs the branch also grabs the start of the hot path as part of the same cacheline.
It sounds minimal, but if that fetch is swapped out of L2 cache due to long periods of inactivity, it can take upwards of 100ns, which starts to add up in HFT.
Yes it is a hardware feature. However the hardware can be given hints as to which branch is more likely. This is generally documented by the manufacturer, in one of those technical documents aimed at compiler writers.
With profile guided optimization it is possible for the compiler to have much better information about branches than the CPU can guess. Java applies profile guided optimization in real time, with C++ it much more complex to apply.
> However the hardware can be given hints as to which branch is more likely.
I don't think that is the case for modern (last 8 years or so) Intel processors. For example, I'm under the impression that gcc's __builtin_expect only affects the layout of the generated code. However I'd love to learn something new here; do you have a source or any additional info you could share?
The hints are purely for the compiler. When branch probabilities are available (either via heuristics, annotations or profile data), it will optimize hot paths differently from cold paths. For example it might be more aggressive with inlinig or vice versa optimize for size. Also will attempt to put cold code in separate pages so that it doesn't get pollute the cache. Also non taken braches are marginally "faster" than taken so it is worth, when possible to put hot code in the non taken branch.
I'm not a compiler writer, I'm sure there is more.
To me, that section says that you can emit machine code that will allow the branch predictor to do a better job, not that you can control what the branch predictor does.
My understanding is the layout is the hint. Modern CPUs would not want to use cache space for any hints, not to mention all the silicon to decode the hint.
Compiler writers are smart people and can layout code to give the CPU the right hints, so long as they know what the CPU does. CPU manufactures want the compilers writers to do this as the compiler is a significant factor in making one CPU faster than the competition in benchmarks.
It seems to me that it's misleading to call that a hint. Rather, certain code paths create hard-to-predict branches, and others create predictable branches. A hint would be some kind of metadata that says "you'll want to predict this branch this way based on your algorithm, but please don't!"
I think you're discussing two different phases of optimization. PGO and Java's JIT use branch information to emit different machine code. Hardware branch prediction takes machine code and determines which branches in the machine code are taken, and speculates based on that information. There's an underlying pattern that both follow, but they're very different.
PGO and JIT both use their information to change the machine code to suggest which branch is more likely.
PGO and JIT also do a lot of other things that are unrelated to this discussion. Some of those things can have a much larger gain than branch prediction.
Unless you write HFT code, or follow talks by those writing HFT code you probably wouldn't. When you write HFT code you have to look at profiles and think about cache misses until branch missed become something to consider. If you don't come up with that idea someone else will and they will beat you to every trade and put you out of business.
So, if I want to manipulate the market I need to work out (or poison) the hot path and have a system that's tailored to being faster on profitable - if colder - paths?
I have been out of the HFT space for many years now, but your conclusions make sense. Here is my own take on the subject from my short tenure at an HFT firm: https://news.ycombinator.com/item?id=12053159
One important thing to know is that allocating memory in C or C++ has high and/or unpredictable latency relative to the target latency of HFT code. So for critical paths, you end up needing to do the same kind of pre-allocation tricks in C/C++ that you would need to do in Java. There is some benefit to being able to write more natural code in the non-critical paths, but that has to be weighed against the overall development advantages that lead people to choose Java over C/C++ in other industries.
A very good point. Most of my GC whispering is done in .NET, which also has value types, but I was guessing that escape analysis in Java would solve most of the problems in this regard. Is that not actually the case?
The most important factor is how well can your language be optimized. HFT is about winner takes all. If the rust optimizer is even slightly worse than your competitors language you will make nothing. Development speed might get your a faster algorithm for a few weeks but your competition will notice you making that money and will catch up despite the slower pace of development and then their faster language will make the difference and you lose all future trades.
llvm is not known as the best optimizer though. (but benchmarks tend to lie and llvm is always pretty close). There are also subtle areas where the front end can generate code that the backend cannot optimizer as well (though given equal effort I'd expect this advantage to go to newer languages that are design for modern optimizers - but effort is not equal with C++ getting for more love)
Yes, my point is mostly informative; we get benefits from using their optimizer, and all the time and resources others pour in, so it's not solely about what the Rust team does. Your point about front-end optimizations is true though; we generally try to copy what clang does, but there's stuff we can do better as well. And also implementing some of our own front-end optimizations.
In trading, correctness is still a lot more important than latency. Using Java instead is about risk avoidance, which is why you also see people using OCaml.
It's incredible how well documented and rich of information is the article. I'm Italian and I could not find in any Italian newspapers all the information that I needed! It makes me aware of how poor is the information in my country!
True, but those are technical ones. I couldn't find any article explaining what a "strallo" is, in fact "cos'è uno strallo" (what is a straw) seems to be a common search based on Google's autocomplete.
>True, but those are technical ones. I couldn't find any article explaining what a "strallo" is, in fact "cos'è uno strallo" (what is a straw) seems to be a common search based on Google's autocomplete.
It's not a "straw", it is a "stay" in English.
Strallo is a term mutuated from boats, a sail boat has a mast (albero) that is kept vertical and strong by one or more sets of "stays" (stralli):
On the sail boat the function of the stay is "opposite" to that of a bridge stay, it is used to keep the mast pressed to the base and more rigid, on a bridge they are simply "suspension supports" allowing to carry the weight of the viaduct deck and transferring the vertical load to the tower or pylon (torre o antenna).
In South Italy professionist graduated, certificated, with ten yeas of experience as Java programmer, earn net 1500 euros. I’m graduated and work from 2 years and earn net 1200 euros. I prefer be waiter, i’m moving away from Italy to start as waiter, it’s better!
Consider remote working as a freelancer in other European countries, they pay 70€+ per hour if they need you (and you are good ;)). All you need is a fast internet connection and a coffee machine at home :)
