Dynamic Modules Development

Dynamic Modules Development

The following is adapted from a presentation given by Ruslan Ermilov at nginx.conf 2015, held in San Francisco in September. You can view the video of the talk on YouTube.

Table of Contents
0:00 Introduction
2:39 Module Growth
3:22 Core Modules
4:30 Event Modules
4:58 HTTP, Stream and Mail Modules
5:47 More Modules
7:11 Static Modules
8:05 Fears and Excuses
9:57 The Demand for Dynamic Loading
11:12 Benefits of Dynamic Loading
12:12 Working Prototype
12:45 Building Dynamic Modules
13:15 Building Dynamic Modules – Example
14:04 Building Third Party Modules Dynamically
14:22 Loading Modules
15:36 Gory Details
17:29 More Gory Details
19:44 For Module Developers
21:05 Who is Involved
22:50 Questions and Answers

0:00 Introduction

My name is Ruslan, but I don’t mind if you call me Ruslam. I’m a developer for NGINX.

I’ve been with the company since early 2011. Amongst other things, I have been writing and heading support for IPv6. When we started NGINX Plus, I worked closely on dynamic configuration of upstreams and health checks. I also helped review Valentin’s code for SPDY and HTTP/2.

In this talk, I’m going to describe what we are doing in the ongoing project that will add support for dynamically loadable modules into NGINX.

2:39 Module Growth

Since its public launch, NGINX has worked with the idea of modules. The first public version, 0.1.0, was released in October 2004 and already had more than 30 modules. This included 22 HTTP modules, five core modules, and nine event modules. Today, after a decade of continuous development, we have nearly 100 modules.

3:22 Core Modules

Let’s take a look under the hood. Core modules implement basic things such as generic types and objects such as numbers, streams, arrays, lists, hashes, buffers, queues, and trees.

Core modules also provide a memory management API. We have a pool allocator, shared memory with slab allocator, and locking primitives for them such as mutexes, spin locks, read/write locks; almost an entire operating system. We implement several checksum and hashing algorithms, configuration file parsing and configuration management interfaces.

We have logging, including syslog support, timekeeping, asynchronous resolver, connection management. The core modules are also responsible for specific tasks such as file and socket I/O, process management, IPC timers, CPU affinity, and thread pools for offloading blocking I/O operations.

4:30 Event Modules

The event modules are responsible for connection processing. This includes epoll support on Linux, kqueue on BSD-like systems, /dev/poll on all the Solaris and some other UNIX derivatives, event port on Solaris 10 and later, and archaic methods such as select and poll. NGINX will automatically choose the most efficient method available for the platform.

4:58 HTTP, Stream and Mail Modules

HTTP modules – these are what make NGINX a good web server. Stream modules – in the 1.9.0 release, we added these. They allow NGINX to work as a generic TCP proxy server.

Mail modules have existed for a long time. These enable NGINX to proxy IMAP, POP3, and SMTP traffic.

5:47 More Modules

Over the years, the growing NGINX community has developed more than 10 dozen third-party modules, and we know quite a few companies which even write their own modules for NGINX.

Our commercial product NGINX Plus also adds another dozen modules. Among these are modules for media streaming – HLS and HDS. Modules that implement session logging instead of just the simple access logging available in the open source version. We have extended monitoring capabilities. We have dynamic configuration for upstream servers and health checks for them. We also have additional load balancing features such as NTLM support and least time balancing, as well as session stickiness.

Besides new modules, NGINX Plus also extends existing modules with features such as limiting the maximum number of connections to upstream servers and keeping upstream configurations up-to-date by tracking DNS changes.

7:11 Static Modules

So, how does it all work together? The set of modules that constitute an NGINX binary is currently fixed at compile time by the configure script. Standard modules are added or excluded to the list using the with and without module options.

The ability to compile in external modules appeared in the early version of NGINX, but the demand for dynamic module loading was low at the time. The number of external modules was pretty moderate, so supporting dynamic loading wasn’t a high priority task.

8:05 Fears and Excuses

Nevertheless, dynamic module loading was considered even early on. dlopen() allows you to easily load dynamically the piece of code into a new binary. But while dlopen() is a major system interface long available in nearly all UNIX-like operating systems, it opens up a can of worms, namely dependency hell.

One of the typical issues with dynamic loading is when two modules are linked against different versions of the same library. If this happens, it can result in random crashes and other hard-to-diagnose problems.

So, what NGINX offers now is a little-known feature: the ability to upgrade the executable on the fly. You can find more information here. Using this feature, it’s possible to run a new NGINX binary built with the modules you need, without any interruption in service. You just tell NGINX to upgrade the binary, providing the new one, and it will switch to the new process while the old process handles all existing connections until they complete.

It works well when you have source code for NGINX and source code for all the modules that you need as well as the freedom of mixing them in the way you need.

9:57 The Demand for Dynamic Loading

Unfortunately, this is not always the case. Linux vendors ship NGINX in binary packages. Usually they provide several packages, each with its own set of modules. For example, on Ubuntu 14.04, there are five base packages of NGINX. These are NGINX Core, NGINX Light, NGINX Full, NGINX Extras, and NGINX Nexus. NGINX Plus also comes as several packages.

What if you’re not happy with the set of modules in a package? You can’t just add another module to the crew; you have to recompile NGINX. So sometimes the provided packages may not suit you.

As the number of third-party modules grew, and with the launch of NGINX Plus in August 2013, it became obvious that we should make an effort to allow modules to be loaded dynamically without the need for recompiling NGINX.

11:12 Benefits of Dynamic Loading

Dynamic loading has the potential of reducing the number of packages that vendors have to ship, because individual modules can be shipped as distinct packages instead of just being part of base packages.

If you look at NGINX Plus, it is partly closed-source, so you can’t recompile it with a custom set of modules. If we support dynamic loading, we can allow modules that pass our integrity checks to be loaded along with NGINX Plus.

Finally, there is growing demand from companies interested in running NGINX Plus with their own modules or supplying binary modules for open-source NGINX.

12:12 Working Prototype

When it came to time to implement dynamic loading, we had a working prototype written in two hours. It could load the empty_gif and geoip modules dynamically.

You can see the traces of this attempt. If you’re familiar with UNIX, in this case FreeBSD, it shows you how different binary objects are loaded and in what order.

12:45 Building Dynamic Modules

The syntax to build dynamic modules simply extends existing syntax for building static modules. You simply say that you want to build some modules dynamically, and the rest is handled by the configure script. It generates the necessary makefile rules to build NGINX and the requested modules.

13:15 Building Dynamic Modules – Example

For example, if you build the GeoIP module dynamically , the dependency on the GeoIP library shifts from the main NGINX binary to the module. The module can then be packaged and distributed separately.

This is also why we don’t put modules that have external dependencies into the base package. We would like to avoid unnecessary dependencies on other vendors, other repositories and so on. And for some modules, it could create legal issues because of different licensing.

14:04 Building Third Party Modules Dynamically

Building a third party module dynamically is not much different. There is only a slight change to this existing syntax. I will talk more about third party modules later.

14:22 Loading Modules

For the configuration file, we added a new directive load_module. It tells NGINX to load a module at program start-up.

Once NGINX is running, you can change the list of modules by simply editing the configuration file, adding or removing the necessary load_module directives.

When you’re happy with your changes, you can signal NGINX to reload its configuration just as you do today. NGINX will check the syntax validity and try to apply the new configuration. If this fails for some reason, NGINX will roll back changes and continue to work with the old configuration.

If some modules became unused in between the changes in configuration, they will be unloaded after applying the new configuration.

15:36 Gory Details

Some modules have interdependencies that should be taken into account. For example, the three standard index modules: random_index, index and autoindex have to be loaded in this particular order or they won’t be able to run correctly. The same holds true for the filter modules.

Not all modules can be made dynamic. There are some modules, the core modules among them, that are an integral part of NGINX, and these can’t be cut into loadable objects.

There are also some complex modules. For example, mail is actually a set of several modules, but they’re written in such a way that makes it impossible to separate specific modules. So right now, mail is a single loadable module that provides eight NGINX modules.

Some third party modules like RTMP and Lua are also complex modules.

17:29 More Gory Details

Another obstacle is that NGINX doesn’t yet have a cleanly separated programming API. In contrast to libraries, when you program for NGINX, you may have to muck with its internal structures, call its functions and sometimes you would even like to make some static functions external.

NGINX also doesn’t provide some abstraction layer like some other programs that use external modules. So, when you program an external module, it’s like you’re extending NGINX with the new module.

Well, what’s the problem with this? NGINX has a lot of conditional compilation defines that may influence the size and contents of various structures. This has to be taken into account when building a custom module.

So when you already have a NGINX binary and just want to build another module for it, you have to use not only the same NGINX header file versions, but also the same set of compilation defines.

There are also some special cases like, what to do when a new configuration removes some filter module, but we have to rollback to a previous configuration due to some issue.

Internally, the filter modules are organized as chains, and in this case we would have to restore the filter chain to its previous state as well. This is one of the tasks that is still not solved.

19:44 For Module Developers

So what does this all mean for module developers?

If you are not interested in dynamically loading your module, you don’t have to worry. Everything will continue to work as before and you are not required to modify the source code of your module.

If you do, however, you just need to make some simple changes to your module’s configuration file. This example shows what such a config may look like. You specify the type and name of your module, supply the list of header files, an optional pass to them, then the list of source files and libraries the module depends on, if any. You can find another example config file and more details in our wiki. Everything is like before, just in a slightly different format.

The new auto/module script will handle the rest. It will generate makefile rules to build the modules statically or dynamically as you have requested.

More complex modules may require more changes to configuration files and maybe multiple calls to the auto/module script.

21:05 Who is Involved

There are two developers involved in the project, these are their names. I am one of them.

Dynamic Modules are now available in NGINX 1.9.11. The next NGINX Plus release, NGINX Plus R9, will also have dynamic modules. You can read more about it in our announcement blog post and find out implementation details in our wiki documentation.

22:50 Questions and Answers

Q: You mentioned that dynamic loading will let you build plugins without the source. Does that means there’ll be a published API with header files to build against soon?

A:That was on my last slide. It’s still an ongoing project, so we don’t have any ready for public code yet. We are still experimenting, but we’re hoping to have a library and an API published soon. And it’s going to be possible to compile modules outside of NGINX.

The post Dynamic Modules Development appeared first on NGINX.

Source: Dynamic Modules Development