Apache APISIX Architecture: Coroutines, Event-Driven Design, and OpenResty vs Envoy

Posted on Views: Word count in article: 5.6k Reading time ≈ 5 mins.
A look at the Apache APISIX architecture: how event-driven design, coroutines, and LuaJIT on top of OpenResty deliver its performance, plus a comparison with Envoy as a foundation.

You have surely come across articles about how fast APISIX is. But how is that performance actually achieved?
This post is both an analysis and my own study notes from the book “OpenResty: From Beginner to Practice” and the official APISIX documentation.

A first look

Start with the architecture diagram from the official site:

flow software architecture

As the diagram shows, APISIX is built on OpenResty and Nginx.

One thing to note here: OpenResty is not a fork of Nginx, nor is it Nginx repackaged with some common libraries bolted on — it simply uses Nginx as an underlying networking library.

So how exactly does it use Nginx as a networking library?

It’s simple: a few lines of configuration in Nginx.conf route all traffic through the gateway’s Lua code.

Note: Nginx does not support Lua by default — that capability comes from OpenResty.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
server {
    listen 9080;

    init_worker_by_lua_block {
        apisix.http_init_worker()
    }

    location / {
        access_by_lua_block {
            apisix.http_access_phase()
        }
        header_filter_by_lua_block {
            apisix.http_header_filter_phase()
        }
        body_filter_by_lua_block {
            apisix.http_body_filter_phase()
        }
        log_by_lua_block {
            apisix.http_log_phase()
        }
    }
}
# simplified illustration only; for the full configuration see the nginx.conf file at the end of this article

In this example, the gateway listens on port 9080 and, via location /, intercepts every request to that port, processing each one through the access, rewrite, header filter, body filter, and log phases in order — calling the corresponding plugin function in each phase. The rewrite phase is handled together inside the apisix.http_access_phase function.

Here is a complete Nginx configuration file: APISIX Nginx.conf

For reference, the OpenResty phases:

openresty phases

To achieve sufficiently high performance, Apache APISIX implements a radix-tree-based route matching algorithm in C, and on top of it uses LuaJIT’s FFI to expose a Lua interface. The flexibility of Lua also lets the APISIX routing module easily support matching sub-routes under the same prefix with expressions and similar methods. The result replaces NGINX’s native routing while delivering dynamic configuration that is both high-performance and highly flexible. For the detailed implementation, see lua-resty-radixtree and route.lua.

But how does the Lua layer keep performance high?

Coroutines + events

At the OpenResty layer, Lua coroutines work together with Nginx’s event mechanism. When Lua code performs an I/O operation — querying a MySQL database, say — it first calls yield to suspend itself and registers a callback with Nginx. Once the I/O completes (or times out, or errors out), Nginx calls resume to wake the Lua coroutine back up. That is how Lua coroutines and Nginx’s event-driven model cooperate, avoiding callbacks written by hand in Lua code.

openresty lua 协程 nginx事件机制

LuaJIT

First, where LuaJIT sits in the overall OpenResty architecture:

openresty luajit

OpenResty’s worker processes are all forked from the master process, and the LuaJIT VM in the master is forked along with it. Every coroutine within a worker shares that one LuaJIT VM, and Lua code executes inside it.

Standard Lua vs LuaJIT

Standard Lua actually ships a VM of its own for performance reasons, so Lua code is not interpreted directly: the Lua compiler first compiles it into bytecode, and the Lua virtual machine then executes that bytecode.

LuaJIT’s runtime, besides an assembly-implemented Lua interpreter, also includes a JIT compiler that can emit machine code directly. At startup, LuaJIT behaves like standard Lua — Lua code is compiled to bytecode, and the bytecode is interpreted by LuaJIT’s interpreter.

The difference: while interpreting bytecode, LuaJIT’s interpreter records runtime statistics — the actual number of times each Lua function entry point is invoked, and the actual number of iterations of each Lua loop. Once those counts exceed a certain randomized threshold, the corresponding function entry or loop is considered hot enough, and the JIT compiler kicks in.

The JIT compiler starts from the hot function’s entry point or some point in the hot loop and tries to compile the corresponding Lua code path. Compilation converts LuaJIT bytecode into LuaJIT’s own intermediate representation (IR), and then into machine code for the target architecture.

So, LuaJIT performance tuning is, in essence, about getting as much Lua code as possible compiled into machine code by the JIT compiler, instead of falling back to the Lua interpreter’s interpreted mode.

Worth noting:

  • LuaJIT is not complete — there are NYI (Not Yet Implemented) gaps
  • LuaJIT’s author is semi-retired
  • The LuaJIT used by OpenResty is a branch maintained by LuaJIT itself

Selected passages

While working through “OpenResty: From Beginner to Practice”, a few passages really resonated with me. Quoting them here for mutual encouragement:

🎯 In my view, understanding why a technology exists, and figuring out its differences and advantages versus similar technologies, matters far more than merely being fluent in calling its APIs. That kind of technical perspective gives you a measure of foresight and insight — arguably an important difference between an engineer and an architect.

📚 No amount of technical coursework can replace careful reading of the official documentation. That slow, unglamorous work cannot be skipped by anyone.

🌍 OpenResty’s official documentation is currently available in English only, so engineers in China will inevitably miss the key points — or even misunderstand the content — because of the language barrier. But precisely because of that, there is no shortcut: you should read the documentation carefully from beginning to end, and when in doubt, work through the test case suites and your own experiments to pin down the answer. That is the right path to learning OpenResty.

⚖️ It’s fine to have preferences when choosing technologies, but we should not turn them into blanket absolutes — there is no silver bullet that fits every caching scenario. Building a minimally viable solution for the actual requirements and growing it step by step is a sound approach.

OpenResty vs Envoy as a foundation

With the rise of cloud native, north-south traffic gateways in the Kubernetes stack have become common as well: North–South Entry Gateway Selection

Feature-wise, these gateways may still lag behind Kong or APISIX, which sit on an OpenResty foundation, but the cloud-native community is clearly more active. Below, a comparison across four dimensions — contributors, stars, commits, and releases (data collected 2025-03-22):

By current functionality, Kong and APISIX are more complete; but long-term, I personally favor gateways built on the Kubernetes Gateway API.

It also feels like Cloudflare has moved off OpenResty to Pingora: how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet

Pingora is not a tool; it is a library.

This article is the English edition of a post originally published in Chinese.