Harnessing the Power and Convenience of JavaScript for Each Request with the NGINX JavaScript Module

Harnessing the Power and Convenience of JavaScript for Each Request with the NGINX JavaScript Module

table.nginx-blog, table.nginx-blog th, table.nginx-blog td {
border: 2px solid black;
border-collapse: collapse;
}
table.nginx-blog {
width: 100%;
}
table.nginx-blog th {
background-color: #d3d3d3;
align: left;
padding-left: 5px;
padding-right: 5px;
padding-bottom: 2px;
padding-top: 2px;
line-height: 120%;
}
table.nginx-blog td {
padding-left: 5px;
padding-right: 5px;
padding-bottom: 2px;
padding-top: 5px;
line-height: 120%;
}
table.nginx-blog td.center {
text-align: center;
padding-bottom: 2px;
padding-top: 5px;
line-height: 120%;
}

Editor – The blog post titled “Introduction to the NGINX JavaScript Module” redirects here. The post has been updated to use the NGINX JavaScript Module directives and features supported as of April 2021.

The NGINX JavaScript module (njs) became generally available as a stable module in NGINX Open Source 1.11.10 and NGINX Plus R12. [The module was originally called nginScript, and that name appears in some older posts.] We have been working steadily on NGINX JavaScript since its launch in September 2015, adding the features and language support included in the stable module.

NGINX JavaScript is a unique JavaScript implementation for NGINX and NGINX Plus, designed specifically for server‑side use cases and per‑request processing. It extends NGINX configuration syntax with JavaScript code in order to implement sophisticated configuration solutions.

The use cases are extensive, especially as the NGINX JavaScript module is available for both HTTP and TCP/UDP protocols. Sample use cases for NGINX JavaScript include:

• Generating custom log formats with values not available from regular NGINX variables (for example, diagnostic logging)
• Modifying responses from proxied servers<!– (response filtering) –>
• Implementing custom authentication schemes (for example, OAuth 2.0 token introspection)
• Parsing TCP/UDP protocols for application‑level sticky sessions (for example, MQTT load balancing)

Before discussing NGINX JavaScript in more detail, let’s first address two common misconceptions.

NGINX JavaScript Is Not Lua

The NGINX community has created several programmatic extensions over the years. At the time of writing, Lua is the most popular of these; it’s available as a module for NGINX and a supported, prebuilt third‑party dynamic module for NGINX Plus. The Lua module and add‑on libraries provide deep integration with the NGINX core and a rich set of functionality, including a driver for Redis.

Lua is a powerful scripting language. It, however, remains fairly niche in terms of adoption and is not typically found in the “skillset toolbox” of the frontend developer or DevOps engineer.

NGINX JavaScript does not seek to replace Lua and it will be some time before NGINX JavaScript has a comparable level of functionality. The goal of NGINX JavaScript is to provide programmatic configuration solutions to the widest possible community by using a popular programming language.

NGINX JavaScript Is Not Node.js

NGINX JavaScript does not aim to turn NGINX or NGINX Plus into an application server. In simple terms, the use cases for NGINX JavaScript are akin to middleware, as the execution of JavaScript code happens between the client and the content. Technically speaking, while Node.js shares two things with the combination of NGINX JavaScript and NGINX or NGINX Plus – an event‑driven architecture and the JavaScript programming language – the similarities end there.

Node.js uses the Google V8 JavaScript engine, whereas NGINX JavaScript is a bespoke implementation of the ECMAScript standards, designed specifically for NGINX and NGINX Plus. Node.js has a persistent JavaScript virtual machine (VM) in memory and performs routine garbage collection for memory management, whereas NGINX JavaScript initializes a new JavaScript VM and the necessary memory for each request and frees the memory when the request is completed.

JavaScript as a Server-Side Language

As mentioned above, NGINX JavaScript is a bespoke implementation of the JavaScript language. All other existing JavaScript runtime engines are designed to be executed within a web browser. The nature of client‑side code execution is different from server‑side code execution in many ways – from the availability of system resources to the possible number of concurrent runtimes.

We decided to implement our own JavaScript runtime in order to meet the requirements of server‑side code execution and fit elegantly with NGINX’s request‑processing architecture. Our design principles for NGINX JavaScript are these:

• Runtime environment lives and dies with the request

  The NGINX JavaScript module uses single‑threaded bytecode execution, designed for quick initialization and disposal. The runtime environment is initialized per request. Startup is extremely quick, because there is no complex state or helpers to initialize. Memory is accumulated in pools during execution and released at completion by freeing the pools. This memory management scheme eliminates the need to track and free individual objects or to use a garbage collector.

• Non‑blocking code execution

  NGINX and NGINX Plus’ event‑driven model schedules the execution of individual NGINX JavaScript runtime environments. When an NGINX JavaScript rule performs a blocking operation (such as reading network data or issuing an external subrequest), NGINX and NGINX Plus transparently suspend execution of the associated NGINX JavaScript VM and reschedule it when the event completes. This means that you can write rules in a simple, linear fashion and NGINX and NGINX Plus schedule them without internal blocking.

• Implement only the language support that we need

  The specifications for JavaScript are defined by the ECMAScript standards. NGINX JavaScript follows ECMAScript 5.1 with some ECMAScript 6 for mathematical functions. Implementing our own JavaScript runtime gives us the freedom to prioritize language support for server‑side use cases and ignore what we don’t need. We maintain a list of the currently supported language elements.

• Close integration with request‑processing phases

  NGINX and NGINX Plus process requests in distinct phases. Configuration directives typically operate at a specific phase and native NGINX modules often take advantage of the ability to inspect or modify a request at a particular phase. NGINX JavaScript exposes some of the processing phases through configuration directives to give control over when the JavaScript code is executed. This integration with the configuration syntax promises the power and flexibility of native NGINX modules with the simplicity of JavaScript code.

  The table below indicates which processing phases are accessible via NGINX JavaScript at the time of writing, and the configuration directives that provide it.

  Processing Phase	HTTP Module	Stream Module
  Access – Authentication and access control	auth_request and js_content	js_access
  Pre-read – Read/write payload	N/A	js_preread
  Filter – Read/write response during proxy	js_body_filter js_header_filter	js_filter
  Content – Send response to client	js_content	N/A
  Log / Variables – Evaluated on demand	js_set	js_set

Getting Started with NGINX JavaScript – A Real‑World Example

NGINX JavaScript is implemented as a module that you can compile into an NGINX Open Source binary or dynamically load into NGINX or NGINX Plus. Instructions for enabling NGINX JavaScript with NGINX and NGINX Plus appear at the end of this article.

In this example we use NGINX or NGINX Plus as a simple reverse proxy and use NGINX JavaScript to construct access log entries in a specialized format, that:

• Includes the request headers sent by the client
• Includes the response headers returned by the backend
• Uses key‑value pairs for efficient ingestion into and searching with log processing tools such as the ELK Stack (now called Elastic Stack), Graylog, and Splunk

The NGINX configuration for this example is extremely simple.

As you can see, NGINX JavaScript code does not sit inline with the configuration syntax. Instead we use the js_import directive to specify the file that contains all of our JavaScript code. The js_set directive defines a new NGINX variable, $access_log_headers, and the JavaScript function that populates it. The log_format directive defines a new format called kvpairs which writes each log line with the value of $access_log_headers.

The server block defines a simple HTTP reverse proxy that forwards all requests to https://www.example.com. The access_log directive specifies that all requests will be logged with the kvpairs format.

Let’s now look at the JavaScript code that prepares a log entry.

log += ` in.${h[0]}=${h[1]}`);
r.rawHeadersOut.forEach(h => log += ` out.${h[0]}=${h[1]}`);
return log;
}

export default { kvAccess }[logging.js] –>

The return value from the kvAccess function – a log entry – is passed to the js_set configuration directive in rawheader_logging.conf. Bear in mind that NGINX variables are evaluated on demand and this in turn means that the JavaScript function defined by js_set is executed when the value of the variable is required. In this example, $access_log_headers is used in the log_format directive and so kvAccess() is executed at log time. Variables used as part of map or rewrite directives (not illustrated in this example) trigger the corresponding JavaScript execution at an earlier processing phase.

We can see this NGINX JavaScript‑enhanced logging solution in action by passing a request through our reverse proxy and observing the resulting log file entry, which includes request headers with the in. prefix and response headers with the out. prefix.

$ curl http://127.0.0.1/
$ tail --lines=1 /var/log/nginx/access_headers.log
2021-04-23T10:08:15+00:00 client=172.17.0.1 method=GET uri=/index.html status=200 in.Host=localhost:55081 in.User-Agent=curl/7.64.1 in.Accept=*/* out.Content-Type=text/html out.Content-Length=612 out.ETag=x22606339ef-264x22 out.Accept-Ranges=bytes

Much of the utility of NGINX JavaScript is a result of its access to NGINX internals. This example utilizes several properties of the request (r) object. The Stream NGINX JavaScript module (for TCP and UDP applications) utilizes a session object (s) with its own set of properties. For other examples of NGINX JavaScript solutions for both HTTP and TCP/UDP, see Use Cases for the NGINX JavaScript Module.

We’d love to hear about the use cases that you come up with for NGINX JavaScript – please tell us about them in the comments section below.

---

Use Cases for the NGINX JavaScript Module

Check out these blog posts to explore other HTTP and TCP/UDP use cases for the NGINX JavaScript module:

• Validating Request Bodies in “Deploying NGINX Plus as an API Gateway, Part 2: Protecting Backend Services”
• Diagnostic Logging with the NGINX JavaScript Module
• Announcing NGINX Plus R22 – Logging the exact set of headers sent by the client whenever an error is encountered
• Announcing NGINX Plus R21 – Chaining subrequests in a single code sequence without using callbacks; sending a copy of request headers to a security information and event management (SIEM) system
• Just One POST: Enabling Declarative DNS with F5 and the NGINX JavaScript Module
• Over-the-Air Updates to IoT Devices with NGINX
• Announcing NGINX Plus R18 – Logging a hashed (masked) version of the client IP address instead of the real address
• Virtual Patching with the NGINX JavaScript Module
• Batching API Requests with NGINX Plus and the NGINX JavaScript Module
• Announcing NGINX Plus R15 – Issuing an HTTP request to two different backends simultaneously, then forwarding the first response and ignoring the second; adding data integrity to application cookies
• Scaling Out Change Detection with JavaScript Hashing in “Detecting Homepage Defacement With Active Health Checks”
• Extending TCP/UDP Load Balancing with nginScript in “TCP/UDP Load Balancing with NGINX: Overview, Tips, and Tricks”
• Using Client Certificates to Authenticate MQTT Clients in “NGINX Plus for the IoT: Encrypting and Authenticating MQTT Traffic”
• Load Balancing MQTT for Session Persistence with the NGINX JavaScript Module in “NGINX Plus for the IoT: Load Balancing MQTT”
• Data Masking for User Privacy with the NGINX JavaScript Module
• Advanced Logging with the NGINX JavaScript Module in “Scaling MySQL with TCP Load Balancing and Galera Cluster”
• Announcing NGINX Plus R11 – Searching for key patterns in a message in a MySQL protocol stream to identify the SQL operation
• Using the NGINX JavaScript Module to Progressively Transition Clients to a New Server

Enabling NGINX JavaScript for NGINX and NGINX Plus

• Loading the NGINX JavaScript Module for NGINX Plus
• Loading the NGINX JavaScript Module for NGINX Open Source
• Compiling NGINX JavaScript as a Dynamic Module for NGINX Open Source

Loading the NGINX JavaScript Module for NGINX Plus

NGINX JavaScript is available as a free dynamic module for NGINX Plus subscribers. For loading instructions, see the NGINX Plus Admin Guide.
<!– tmauro 6-Aug-2019 commented out and added pointer to Admin Guide

1. Obtain the module itself by installing it from the NGINX Plus repository.

   • For Ubuntu and Debian systems:

     [terminal]$ sudo apt‑get install nginx-plus-module-njs[/terminal]

   For RedHat, CentOS, and Oracle Linux systems:

   [terminal]$ sudo yum install nginx-plus-module-njs[/terminal]

2. Enable the module by including a load_module directive for it in the top‑level (“main”) context of the nginx.conf configuration file (not in the http or stream context). This example loads the NGINX JavaScript modules for both HTTP and TCP/UDP traffic.

   [config]load_module modules/ngx_http_js_module.so;
   load_module modules/ngx_stream_js_module.so;[/config]

3. Reload NGINX Plus to load the NGINX JavaScript modules into the running instance.

   [terminal]$ sudo nginx -s reload[/terminal]

–>

Loading the NGINX JavaScript Module for NGINX Open Source

If your system is configured to use the official prebuilt packages for NGINX Open Source and your installed version is 1.9.11 or later, then you can install NGINX JavaScript as a prebuilt package for your platform.

1. Install the prebuilt package.

   • For Ubuntu and Debian systems:

     $ sudo apt-get install nginx-module-njs

   • For RedHat, CentOS, and Oracle Linux systems:

     $ sudo yum install nginx-module-njs

2. Enable the module by including a load_module directive for it in the top‑level (“main”) context of the nginx.conf configuration file (not in the http or stream context). This example loads the NGINX JavaScript modules for both HTTP and TCP/UDP traffic.

   load_module modules/ngx_http_js_module.so;
   load_module modules/ngx_stream_js_module.so;

3. Reload NGINX Plus to load the NGINX JavaScript modules into the running instance.

   $ sudo nginx -s reload

Compiling NGINX JavaScript as a Dynamic Module for NGINX Open Source

If you prefer to compile an NGINX module from source:

1. Follow these instructions to build either or both the HTTP and TCP/UDP NGINX JavaScript modules from the open source repository.
2. Copy the module binaries (ngx_http_js_module.so, ngx_stream_js_module.so) to the modules subdirectory of the NGINX root (usually /etc/nginx/modules).
3. Perform Steps 2 and 3 in Loading the NGINX JavaScript Module for NGINX Open Source.

The post Harnessing the Power and Convenience of JavaScript for Each Request with the NGINX JavaScript Module appeared first on NGINX.

Source: Harnessing the Power and Convenience of JavaScript for Each Request with the NGINX JavaScript Module