JIT approach

[CeleritasCelery]

Emacs already offers AOT compilation via native compilation. By doing so you can remove the overhead of the bytecode interpreter and optimizing the code as a single compilation unit (which results in better code).

However there are a few things you loose by compiling ahead-of-time:

1. You don’t have any type information. Since elisp is dynamically typed, you have to assume that your input arguments can be any type. Sometimes you can do type inference because the built-in function usually have type requirements, but it is limited in the elisp world.
2. You don’t know what code paths are most important. Since the code has never been run when it is compiled, you don’t know what code paths are “hot”. So everything is compiled the same.
3. You have limited ability to inline. Only builtin function can be inlined, because any function in elisp can be dynamically updated.

Both of these can be solved with a little run time information. If you are able to profile the code as it runs, you can see what types it gets called with (which is usually the only types it will use) and you know which functions get called frequently. This allows for more aggressive optimizations then AOT and let’s you only compile the functions that actually matter, because 95% of them are not worth the effort.

tracing vs method JIT's

tracing JIT's start by looking for backwards jumps (i.e. the end of loops). They then trace every instruction until the same point is hit again, essentially make a "trace" of the loop body. This trace is then JIT compiled and spliced into the program stream. Since most hot code is in loops this tends to work very well. It also has the advantage that the trace has no branches and so better take advantage of code motion. Also you don't need to worry about inlining, because methods are automatically inlined in the trace.

The other camp is method JITs. Here you just look for "hot" methods, record type information about the arguments and branches, and then JIT compile the method.

From what I understand of the real world use cases, tracing JIT's can have better peak performance, but also tend to have more pathological cases that are harder to debug. Tracing JIT's are very sensitive to parameter tuning. method JIT's don't have the same max speedup but are more consistent and less pitfalls. Overall I see them as simpler to comprehend and debug.