How Aptible gracefully handles memory management

Memory policies play a major role in how Docker applications function. Docker containers share the same underlying hardware, which can result in issues if a single process eats up a core resource (such as memory). Known as the noisy neighbor problem, one memory-hungry process could shut down an entire application without the right safeguards.

Given that the core tenet of containerized applications is to separate subprocesses into their own isolated units, memory management is an important feature. Docker handles memory via dynamic memory limits. However, Docker’s approach involves using SIGKILL, a termination-forcing technique that has downsides.

Today, let’s discuss the issues with Docker’s native approach and how Aptible addresses it.

What is Aptible?

Aptible is a platform as a service (PaaS) that automatically scales infrastructure to meet an application's demands. Aptible is built on top of Docker, handling all the grunt work of building robust systems on containerized systems. Aptible uses Docker’s native features when they meet our users’ needs, but we aren’t afraid to engineer a better experience when called for.

When it comes to memory, Aptible differs from Docker in two ways: (i) how memory limits work and (ii) what happens to memory in the case of container recovery.

How does Docker’s memory limit work?

Docker enables users to set dynamic memory limits.

At a base level, Docker’s memory limits restrict how much memory an application is allowed to use. When applications run short on memory, they may swap it with disk—Docker has a setting to restrict how much memory can be swapped to disk. This can also be set as a percentage.

Additionally, Docker containers can have a soft limit. A soft limit is a memory limit that isn’t always enforced (but recommended to the application). In the case that the kernel is low on memory, the soft limit will be enforced. A soft limit could be combined with a hard limit, where the hard limit is always greater than the soft limit.

What happens if a limit condition is crossed? Docker will kill the process. It accomplishes this by issuing a SIGKILL signal to the process (or, more specifically, instructing the kernel to do so). However, SIGKILL is not a “graceful” approach to terminating a process; it’s equivalent to force quitting an application on Mac—it forces the application to immediately shut down. Even if the application had a shut-down process, it wouldn’t be able to carry it out because SIGKILL cannot be blocked or ignored, even for a brief period of time.

Quick Refresher—SIGKILL versus SIGTERM: Unix has two built-in signals for terminating a process: SIGTERM and SIGKILL. While SIGKILL immediately terminates a process, SIGTERM instructs it to shut down. This enables the process to error-handle incomplete requests and return any unfinished jobs to any external queue. Once the process cleans up its state, it can terminate itself. If a process ignores a SIGTERM signal for a lengthy period of time, a SIGKILL will typically be issued to force it to quit.

How does Aptible’s memory management work?

Aptible handles memory limits via a step-by-step procedure. This process is managed by a feature named memory management. Memory management’s goal is to gracefully terminate a container before having to force it to quit.

It accomplishes this by:

1. Dispatching a log message stating that a container has crossed the memory limit. The message includes a list of processes that the container is running for debugging.
2. Giving the container 10% more memory if space is available. This gives the container a better shot of exiting gracefully.
3. Issuing a SIGTERM signal to the container to instruct it to self-terminate.
4. If the container doesn’t halt within 10 seconds, issuing a SIGKILL signal to force the container to terminate.
5. Triggering container recovery to restart the container.

The general benefit of Aptible’s memory management feature is that it increases a container’s chance of exiting gracefully, enabling it to clean up its work before self-terminating. Aptible prioritizes predictability in performance and therefore doesn’t support or recommend swap limits or soft limits. Instead, Aptible is able to respond more gracefully when an out-of-memory error occurs.

Container recovery

A feature that’s directly connected to memory limits is container recovery. Container recovery is the practice of restarting a container if it has exited unexpectedly. Container recovery is extremely common—most applications eventually hit a snag.

Container recovery differs from process management solutions like upstart or PM2 that attempt to restart failed applications at the application layer. It’s generally advised to not combine container recovery and process management solutions because it muddles which process is in charge of restarting applications.

Just like memory limits, Docker’s and Aptible’s container recovery features work a bit differently.

Docker’s memory-preserving container recovery

Docker’s container recovery—named container restart policy—is a configurable policy that governs when a container should be restarted. For instance, a container restart policy could dictate that containers be restarted in all cases or only if the code had faulted.

Docker only executes the container restart policy if a container previously started successfully and lived for at least 10 seconds. This condition prevents an infinite loop of container restarts. However, an infinite loop is still possible if the container’s fault happens after some initial application booting.

Docker doesn’t wipe the container’s memory before restarting it. This could lead to a future crash if an artifact in memory was the crash’s origin.

Aptible’s filesystem-wiping container recovery

Aptible keeps track of which containers should be alive. When Aptible’s ledger doesn’t match a container’s reported dead state, Aptible performs container recovery. While Aptible invokes docker start to restart the container, it differs from Docker when it comes to the filesystem.

Pristine state

Before restarting the container, Aptible wipes the container’s filesystem. This technique is known as starting a container from a pristine state.

Doing this manually is possible, but is a hassle. Only the application’s created files need to be deleted; otherwise, applications might break.

Aptible’s memory auto-wipe feature can be optionally disabled. This might be necessary if an application is using its file storage to manage its state (which is discouraged when following the twelve-factor app model).

To be clear, the local file system is still an important resource to a process. For instance, a process might store temporary JSON files for query responses. It’s important to preserve this file system in case it lends details about the crash.

To address this, Aptible copies the file system into a folder outside of the container, retaining the file system while maintaining a pristine state.

Closing thoughts

Aptible designs its platform with these specific behaviors to provide the best experience to our customers, even when they run into unexpected crashes or uncontrolled memory consumption. Conversely, Docker enables developers to set more complex memory limits, it terminates processes using SIGKILL, which prevents processes from gracefully ending. In comparison, Aptible gives containers extra memory and time to end on their own terms, allowing them to clean up any remaining work. Of course, if a process doesn’t terminate itself, Aptible will also issue a SIGKILL signal.

After a memory limit is crossed, Aptible and Docker have different strategies to restart a container. Docker attempts to restart a container immediately. Aptible has a few more steps, including wiping the file system after copying it to an external location.

Overall, Aptible’s memory management features and container recovery policies were designed to address Docker’s trigger-happy faults.