BLOG | NGINX

Announcing NGINX Plus R25

NGINX-Part-of-F5-horiz-black-type-RGB
Zach Westall Thumbnail
Zach Westall
Published September 28, 2021

We’re happy to announce the availability of NGINX Plus Release 25 (R25). Based on NGINX Open Source, NGINX Plus is the only all-in-one software web server, load balancer, reverse proxy, content cache, and API gateway.

New features in NGINX Plus R25 include:

Important Changes in Behavior

Increased Memory Requirement for Upstream Zones

As described in detail in More Granular Reporting of HTTP Response Codes, NGINX Plus R25 provides counts of individual status codes as well as per‑class aggregate counts. When NGINX Plus is configured as a reverse proxy or load balancer, you might need to increase the size of the shared memory zone used to monitor upstream “peers” (backend servers), if there are more than 20 peers.

NGINX Plus R25 does not start if shared memory zones are under‑provisioned, causing failed upgrades. Before upgrading, it is important to check the memory utilization of each upstream zone. For instructions on checking and adjusting memory allocation, see Allocate Sufficient Memory to Prevent Startup Failures.

Changes to the NGINX Plus Repo Organization and Installation Procedure

At the release of NGINX Plus R24, the package repositories for all NGINX software were reorganized, resulting in changes to the NGINX Plus installation procedure.

When you install or upgrade NGINX Plus, the operating system’s package manager (apt, yum, or equivalent) is configured with the software repository for NGINX Plus. On existing systems configured to use the old repo (those running NGINX Plus R23 or earlier), you need to update the package manager to refer to the new repo. See the instructions in the F5 Knowledge Base.

If performing an initial installation of NGINX Plus R25, see Installing NGINX Plus in the NGINX Plus Admin Guide.

Note: You must use the new software repository. The old repo will no longer be updated and may cause errors for future installs and upgrades.

Deprecated Cookie-Flag Module

As announced at the release of NGINX Plus R23, the third‑party Cookie‑Flag module is deprecated and will be removed from the dynamic modules repo in NGINX Plus R26. The set_cookie_flag directive defined in that module is replaced by the built‑in proxy_cookie_flags directive. We recommend that you replace any set_cookie_flag directives in your configuration with proxy_cookie_flags directives as soon as possible.

Changes to Platform Support

  • New operating systems supported:
    • Debian 11 (x86_64, aarch64)
    • Alpine Linux 3.14 (x86_64, aarch64)
  • Older operating systems removed:
    • Alpine Linux 3.10; oldest supported version is 3.11
    • Amazon Linux 1 (2018.03+); switch to Amazon Linux 2 LTS
    • FreeBSD 11.4+; oldest supported version is 12.1+
    • Ubuntu 16.04 LTS; oldest supported version is 18.04 LTS
  • Older operating systems deprecated and scheduled for removal in NGINX Plus R26:
    • Alpine Linux 3.11

New Features in Detail

Additional JSON Web Token Use Cases and Functionality

NGINX Plus R24 introduced initial support for encrypted JSON Web Tokens (JWE), extending one of the most popular methods of client authentication with data confidentiality. NGINX Plus R25 builds on that capability and introduces support for additional and more advanced authentication use cases that help improve the security of JWT‑based authentication for both signed (JWS) and encrypted (JWE) use cases. The enhancements both reduce the risk of leaking personally identifiable information (PII) and offer more flexibility. The new JWT functionality and enhancements for NGINX Plus include:

Variable for Decrypted JWE Ciphertext

JWTs are a widely used and trusted method for stateless authentication of HTTP requests (that is, token‑based authentication). By transmitting end‑user attributes in the token payload, JWTs help make requests more secure. However, security researchers and DevSecOps practitioners agree that the risk presented by storing unencrypted PII on web clients is a pressing concern – hence the development of the JSON Web Encryption standard (RFC 7516), which provides guidelines for implementing encrypted tokens.

NGINX Plus R24 introduced support for JWE, providing data integrity and confidentiality for the entire claim set. Encoding PII within the encrypted token greatly reduces the risk of data leaks when using JWTs.

NGINX Plus R25 builds on the initial JWE support with a new variable, $jwt_payload, that enables NGINX Plus to decrypt the JWE and ciphertext, and then access the plaintext during processing of the HTTP request. This new functionality can be used for implementing advanced access‑control policies and request‑routing decisions, as well as enabling users to deploy NGINX Plus as a JWE decryption layer for backend applications.

Support for Nested JWTs

JWE tokens are effective at protecting PII, with ciphertext serving to preserve confidentiality even across CDNs and other TLS‑terminating proxies. But encryption is a double‑edged sword, as JWEs can introduce complexity into the application environment. A way to address this issue is to use nested JWTs.

Diagram showing structure of a nested JWT
Anatomy of a nested JWT

Nested JWTs embed JWS as the ciphertext of a JWE. NGINX Plus R25 enables nested JWTs as a method for authenticating HTTP requests by supporting both JWS and JWE simultaneously. In one operation, NGINX Plus can now decrypt the ciphertext in the JWE, validate the resulting plaintext as a JWS, and use the exp (expiration time) and nbf (not before) claims in the enclosed token to verify that it is valid. This enables an existing JWS environment to be upgraded with payload encryption by wrapping it in a JWE.

The following configuration snippet illustrates the process. The new nested parameter to the auth_jwt_type directive tells NGINX Plus to perform both JWE decryption and JWS validation. It also affects how the variables for JWT headers and JWT claims are evaluated:

 

With the following configuration, NGINX Plus constructs and sends additional HTTP request headers to the backend application. The encryption algorithm from the JWE header (the $jwt_header_enc variable) and the JWT subject claim from the JWS payload (the $jwt_claim_sub variable) are proxied with the original request. This means that applications can leverage JWE‑based authentication simply by consuming HTTP headers, with no need to implement cryptographic code or libraries.

 

For users operating in zero‑trust architectures, the following configuration enables the backend application to validate the JWS token as captured in the $jwt_payload variable. The variable contains the decrypted plaintext version of the JWE ciphertext. If there is a nested JWT, the JWS can be passed to the backend application as a bearer token.

 

In essence, nested JWTs streamline operations and improve application security by enabling NGINX Plus to simultaneously decrypt and validate secure tokens, all while parsing or proxying the “regular JWT” signed token for backend applications.

Asymmetric Encryption for JWE

NGINX Plus R24 introduced support for symmetric encryption of tokens using the AES standard. NGINX Plus R25 adds support for asymmetric encryption when using RSA key pairs. Asymmetric cryptography uses different keys (public and private) for encryption and decryption which are mathematically linked to each other with the RSA (Rivest–Shamir–Adleman) algorithm, the same mechanism used for SSL/TLS encryption of HTTPS traffic between client and server.

NGINX Plus R25 supports RSA with Optimal Asymmetric Encryption Padding (OEAP) as a key‑management algorithm, with no explicit configuration required on the NGINX Plus side. Supported values for the JWS alg (algorithm) header are RSA-OAEP, RSA-OAEP-256, RSA-OAEP-384, and RSA-OAEP-512.

The following configuration shows how all that’s needed to implement asymmetric encryption for JWE is to specify the JSON Web Key (JWK) file that contains the RSA private (unwrap) keys as the parameter to the auth_jwt_key_file directive (the auth_jwt_key_request directive can also be used).

 

Multi-Source JWK Support

Leveraging nested JWTs means that the associated JWKs are likely to come from more than one source. NGINX Plus R25 supports multiple auth_jwt_key_file and auth_jwt_key_request directives in the same context. NGINX Plus loads keys from all specified sources and looks for the matching verification key among the combined set of keys – extremely useful when working with a JWS that was issued by an external identity provider and is nested inside a JWE, where the encryption was done by a separate process.

This feature adds further flexibility, security, and power to NGINX Plus deployed as an API gateway.

Custom JWT Validation Rules

When NGINX Plus is deployed as an API gateway, it is common to inspect JWT claims to implement fine‑grained access control. This can lead to complex configurations. In previous NGINX Plus releases, you could use if configuration blocks to evaluate JWT claims, but this approach was somewhat limited and did not work for encrypted tokens.

NGINX Plus R25 addresses this limitation by providing a native way to perform additional checks on the token during the JWT validation process. The auth_jwt_require directive adds an optional, customizable step into the JWT validation process. It accepts a space‑separated list of strings, all of which must be non‑empty and not equal to 0 (zero) for JWT validation to succeed. This adds huge flexibility into the JWT validation process.

The JWT standard does not prescribe which claims are mandatory, so you can use auth_jwt_require to list the appropriate claims for each environment.

The following configuration ensures that both the exp and sub claims are present in every token.

 

You can configure more complex use cases by using map blocks or the NGINX Plus key‑value store to pass boolean values to the auth_jwt_require directive. You can also use the NGINX JavaScript module to implement rich authentication solutions such as mutual TLS (mTLS) client certificate‑bound access tokens (defined in RFC 8705).

The following configuration performs both client certificate authentication (mTLS) and JWT validation. The auth_jwt_require directive on line 14 calls for the $thumbprint_match variable to be evaluated, and JWT validation succeeds only if it has a non‑zero value. This variable is evaluated by executing the JavaScript function invoked on line 2.

 

Here’s the code for the validate function invoked on line 2 of the previous snippet:

 

More Granular Reporting of HTTP Response Codes

Monitoring and visibility are cornerstones of application performance, availability, and security. HTTP response codes are one of the most crucial sources of insight into application health. The NGINX Plus API tracks HTTP response codes both for interactions between NGINX and clients, and between NGINX and upstream servers; the counts are reported on the relevant tabs of the NGINX Plus live activity monitoring dashboard.

Previous versions of the NGINX Plus API aggregated HTTP response code counts by class (2xx, 4xx, and so on). Now codes are also counted individually (200, 404, 503, etc.). This gives you deeper insight into exactly what’s going on with an application – distinguishing between errors that have very different causes, such as spikes in failed authentications (403) and missing content (404). For information about configuring metrics gathering, see the NGINX Plus Admin Guide.

The latest version of the NGINX Plus API (version 7), released with NGINX Plus R25, includes a codes object within the responses object to enable counting of individual HTTP response codes. Aggregated responses are still available and previous versions of the NGINX Plus API – which don’t include the codes object – can still be used with NGINX Plus R25. Here’s an example of the new set of metrics:

$ curl -s http://localhost:8080/api/7/http/server_zones/www.example.com | jq{
  "processing": 31,
  "requests": 63192,
  "responses": {
    "1xx": 0,
    "2xx": 54368,
    "3xx": 8454,
    "4xx": 330,
    "5xx": 9,
    "codes": {
      "200": 54368,
      "302": 8454,
      "401": 30,
      "404": 200,
      "429": 100,
      "503": 9
    },
    "total": 63161
  },
  "discarded": 0,
  "received": 693436,
  "sent": 13843478
}

Allocate Sufficient Memory to Prevent Startup Failures

Important Note: When NGINX Plus is configured as a reverse proxy or load balancer, counting of individual codes increases memory utilization in the shared memory zone for each upstream group. If there are more than 20 peers in the upstream group, you may need to increase the memory size, as configured with the zone directive.

NGINX Plus R25 does not start if upstream zones are under‑provisioned, causing failed upgrades.

Here’s a typical configuration of the shared memory zone for an upstream group:

 

Before upgrading to NGINX Plus R25, it is important that you check the memory utilization for existing upstream zones. To do this, ensure the NGINX Plus API is enabled before using one of these methods:

  • Check the HTTP Upstreams tab of the live activity monitoring dashboard, as in this example from demo.nginx.com, where utilization is 54%:

  • Use the NGINX Plus API directly from the host by running the following command. First:

    • Install the jq utility if necessary
    • Set the API variable to the location where the api directive is enabled
    $ API=http://localhost:8080/api; for i in `curl -s $API/1/http/upstreams | jq -r '.[].zone | @uri'`; do echo -n $i; curl -s $API/1/slabs/$i | jq -r '.pages | 100*(.used / (.used + .free)) | " \(round)% used"'; done

If current utilization is above 40% (as in the screenshot), increase the second (size) parameter to the zone directive by at least 2.5x. For example, we recommend increasing 64k to 160k in the configuration snippet above.

Dynamic SSL/TLS Client Certificate Loading for Proxying

Mutual TLS (mTLS) is a common authentication practice that involves verifying the identity of both client and server. With NGINX Plus, you can define the servers in an upstream group dynamically and with variables. That implies you may also need to be able to dynamically select the TLS certificate used by NGINX Plus to verify itself to those upstream servers.

NGINX Plus R25 extends the configuration directives used for mTLS to a backend server, to accept a variable representing the certificate. The variable can refer to either of these:

  • A file on disk
  • Raw data in PEM format, prefixed with data:

This enables NGINX Plus to select a certificate and private key dynamically – useful for modern application environments that are subject to constant change. You can store the certificate and private key in the NGINX Plus key‑value store, which enhances security because the private key is stored in memory rather than on disk. Another use case is automated certificate rotation, where you use an API call to update certificates in the key‑value store.

In the following configuration, NGINX Plus routes requests to different upstream groups based on the hostname. The proxied connection is made using mTLS, and the appropriate client certificate for each upstream is dynamically selected.

 

The following directives support dynamic certificate loading for mTLS with upstream servers:

Hardened Security for HTTP Request Processing

One of the core tenets of the NGINX philosophy is continuous improvement, especially concerning security. We use all available resources, including collaboration with security researchers, integration of F5’s industry‑leading security technologies into our products, and internal development efforts.

As an example of the last, NGINX Plus R25 performs several additional checks on HTTP requests to protect your applications against potential attacks, such as request smuggling. It returns an error for these conditions:

  • An HTTP/1.0 request includes the Transfer-Encoding header
  • The Transfer-Encoding and Content-Length headers are both present
  • There are spaces or control characters in the request line or any header name
  • There are spaces or control characters in the Host header
  • The CONNECT method is used

In addition, the following characters are now always escaped in a proxied URI: ", <, >, \, ^, `, {, |, }.

Please note that these changes are proactive security enhancements and are not made in response to any known vulnerability.

Persistence of Mandatory Health Check Status on Reload for TCP/UDP Applications

NGINX Plus uses mandatory health checks to ensure that new servers added to upstream groups are tested and healthy before client requests are proxied to them. In NGINX Plus R23 and earlier, upstream servers were considered unhealthy after a configuration reload regardless of their status before the reload. As a result, NGINX Plus didn’t send requests to a server until the first mandatory check had passed.

NGINX Plus R24 introduced optional persistence of mandatory health check status across reloads, for HTTP applications. If the last mandatory health check before a reload was successful, NGINX Plus can send requests to a server immediately rather than having to wait for a post‑reload mandatory health check to succeed.

NGINX Plus R25 extends this functionality to TCP/UDP applications (within the stream context). As for HTTP, add the persistent parameter to the health_check directive along with the mandatory parameter.

 

Enhancements to the NGINX JavaScript Module

The NGINX JavaScript module (njs) has been updated to version 0.6.2 with several bug fixes and some functional enhancements that strengthen compatibility with JavaScript ES6.

Variable Declaration with the let and const Keywords

NGINX JavaScript now supports declaration of scoped variables using the let and const keywords. Previous versions of NGINX Plus and njs supported only the var keyword for declaring and assigning variables. This didn’t provide for variables that are limited to the scope of a particular block statement, which are needed to deal with the complexity that often arises when different projects, programming languages, and engineering teams collaborate on shared codebases or libraries.

JavaScript ES6 provides the let and const keywords for defining scoped variables:

  • let variables are limited to the scope of a block statement or an expression on which it is used. In contrast, the var keyword declares a variable globally, or locally to an entire function regardless of block scope.
  • const variables are block‑scoped, much like variables declared using the let keyword. The value of a constant can’t be changed through reassignment, and it can’t be redeclared.

Support for All Promise Constructor Methods

With the addition of the Promise.all(), Promise.allSettled(), Promise.any(), and Promise.race() constructor methods, njs now supports the complete set of constructor methods defined in the JavaScript ES6 standard.

Upgrade or Try NGINX Plus

If you’re running NGINX Plus, we strongly encourage you to upgrade to NGINX Plus R25 as soon as possible. You’ll also pick up several additional fixes and improvements, and it will help NGINX to help you when you need to raise a support ticket.

If you haven’t tried NGINX Plus, we encourage you to try it out – for security, load balancing, and API gateway, or as a fully supported web server with enhanced monitoring and management APIs. You can get started today with a free 30-day trial.

Read more blog posts about F5 NGINX ›

"This blog post may reference products that are no longer available and/or no longer supported. For the most current information about available F5 NGINX products and solutions, explore our NGINX product family. NGINX is now part of F5. All previous NGINX.com links will redirect to similar NGINX content on F5.com."