I have also wondered this for years, and always was told "because JITs work better", but that felt a bit handwavy. Luckily for both of us David Chisnall just published an article on ACM about designing ISAs that properly explains the reasoning behind Jazelle and why it did not work in the long run:
> Small code is also important [for a simple single-issue in-order core]. A small microcontroller core may be as small as 10KB of SRAM (static random access memory). A small decrease in encoding efficiency can dwarf everything when considering the total area cost: If you need 20 percent more SRAM for your code, then that can be equivalent to doubling the core area. Unfortunately, this constraint almost directly contradicts the previous one [about decoder complexity]. This is why Thumb-2 and RISC-V focused on a variable length encoding that is simple to decode: They save code size without significantly increasing decoder complexity.
> This is a complex tradeoff that is made even more complicated when considering multiple languages. For example, Arm briefly supported Jazelle DBX (direct bytecode execution) on some of its mobile cores. This involved decoding Java bytecode directly, with Java VM (virtual machine) state mapped into specific registers. A Java add instruction, implemented in a software interpreter, requires at least one load to read the instruction, a conditional branch to find the right handler, and then another to perform the add. With Jazelle, the load happens via instruction fetch, and the add would add the two registers that represented the top of the Java stack. This was far more efficient than an interpreter but did not perform as well as a JIT (just-in-time) compiler, which could do a bit more analysis between Java bytecodes.
> Jazelle DBX is an interesting case study because it made sense only in the context of a specific set of source languages and microarchitectures. It provided no benefits for languages that didn't run in a Java VM. By the time devices had more than about 4MB of RAM, Jazelle was outperformed by a JIT. Within that envelope, however, it was a good design choice.
> Jazelle DBX should serve as a reminder that optimizations for one size of core can be incredibly bad choices for other cores
So: a decent JIT works better if you can afford the overhead of the JIT. Jazelle was only a good idea in a very brief period of time when this wasn't true, and even then only if you insist on running a Java VM.
> Small code is also important [for a simple single-issue in-order core]. A small microcontroller core may be as small as 10KB of SRAM (static random access memory). A small decrease in encoding efficiency can dwarf everything when considering the total area cost: If you need 20 percent more SRAM for your code, then that can be equivalent to doubling the core area. Unfortunately, this constraint almost directly contradicts the previous one [about decoder complexity]. This is why Thumb-2 and RISC-V focused on a variable length encoding that is simple to decode: They save code size without significantly increasing decoder complexity.
> This is a complex tradeoff that is made even more complicated when considering multiple languages. For example, Arm briefly supported Jazelle DBX (direct bytecode execution) on some of its mobile cores. This involved decoding Java bytecode directly, with Java VM (virtual machine) state mapped into specific registers. A Java add instruction, implemented in a software interpreter, requires at least one load to read the instruction, a conditional branch to find the right handler, and then another to perform the add. With Jazelle, the load happens via instruction fetch, and the add would add the two registers that represented the top of the Java stack. This was far more efficient than an interpreter but did not perform as well as a JIT (just-in-time) compiler, which could do a bit more analysis between Java bytecodes.
> Jazelle DBX is an interesting case study because it made sense only in the context of a specific set of source languages and microarchitectures. It provided no benefits for languages that didn't run in a Java VM. By the time devices had more than about 4MB of RAM, Jazelle was outperformed by a JIT. Within that envelope, however, it was a good design choice.
> Jazelle DBX should serve as a reminder that optimizations for one size of core can be incredibly bad choices for other cores
So: a decent JIT works better if you can afford the overhead of the JIT. Jazelle was only a good idea in a very brief period of time when this wasn't true, and even then only if you insist on running a Java VM.
[0] https://queue.acm.org/detail.cfm?id=3639445