AWS - Abusing Lambda Extensions

Lambda Extensions

Lambda extensions enhance functions by integrating with various monitoring, observability, security, and governance tools. These extensions, added via .zip archives using Lambda layers or included in container image deployments, operate in two modes: internal and external.

Internal extensions merge with the runtime process, manipulating its startup using language-specific environment variables and wrapper scripts. This customization applies to a range of runtimes, including Java Correto 8 and 11, Node.js 10 and 12, and .NET Core 3.1.
External extensions run as separate processes, maintaining operation alignment with the Lambda function's lifecycle. They're compatible with various runtimes like Node.js 10 and 12, Python 3.7 and 3.8, Ruby 2.5 and 2.7, Java Corretto 8 and 11, .NET Core 3.1, and custom runtimes.

For more information about how lambda extensions work check the docs.

External Extension for Persistence, Stealing Requests & modifying Requests

This is a summary of the technique proposed in this post: https://www.clearvector.com/blog/lambda-spy/

It was found that the default Linux kernel in the Lambda runtime environment is compiled with “process_vm_readv” and “process_vm_writev” system calls. And all processes run with the same user ID, even the new process created for the external extension. This means that an external extension has full read and write access to Rapid’s heap memory, by design.

Moreover, while Lambda extensions have the capability to subscribe to invocation events, AWS does not reveal the raw data to these extensions. This ensures that extensions cannot access sensitive information transmitted via the HTTP request.

The Init (Rapid) process monitors all API requests at http://127.0.0.1:9001 while Lambda extensions are initialized and run prior to the execution of any runtime code, but after Rapid.

The variable AWS_LAMBDA_RUNTIME_API indicates the IP address and port number of the Rapid API to child runtime processes and additional extensions.

⚠️ Warning

By changing the **`AWS_LAMBDA_RUNTIME_API`** environment variable to a **`port`** we have access to, it's possible to intercept all actions within the Lambda runtime (**man-in-the-middle**). This is possible because the extension runs with the same privileges as Rapid Init, and the system's kernel allows for **modification of process memory**, enabling the alteration of the port number.

Because extensions run before any runtime code, modifying the environment variable will influence the runtime process (e.g., Python, Java, Node, Ruby) as it starts. Furthermore, extensions loaded after ours, which rely on this variable, will also route through our extension. This setup could enable malware to entirely bypass security measures or logging extensions directly within the runtime environment.

The tool lambda-spy was created to perform that memory write and steal sensitive information from lambda requests, other extensions requests and even modify them.

AWS - Abusing Lambda Extensions

Lambda Extensions

External Extension for Persistence, Stealing Requests & modifying Requests

References