Core Concepts
Reference
How-to Guides
Troubleshooting
Container Recovery
When Containers on Aptible exit unexpectedly (i.e., Aptible did not terminate them as part of a deploy or restart), they are automatically restarted. This feature is called Container Recovery. For most apps, Aptible will automatically restart containers in the event of a crash without requiring user action.
Overview
When Containers exit, Aptible automatically restarts them from a pristine state. As a result, any changes to the filesystem will be undone (e.g., PID files will be deleted, etc.). As a user, the implication is that if a Container starts properly, Aptible can automatically recover it. Whenever this happens, Aptible notifies Log Drains with a pair of container has exited
/ container has started
events. If an App is continuously restarting, Aptible will throttle recovery to a rate of one attempt every 2 minutes.
Cases where Container Recovery will not work
Container Recovery restarts Containers that exit, so if an app crashes but the Container does not exit, then Container Recovery can't help.
Here's an example Procfile demonstrating this issue:
app: (my-app &) && tail -F log/my-app.log
In this case, since my-app
is running in the background, the Container will not exit when my-app
exits. Instead, it would exit if tail
exited.
To ensure Container Recovery effectively keeps an App up, make sure that:
- Each Container is only running one App.
- The one App each Container is supposed to run is running in the foreground.
For example, rewrite the above Procfile like so:
app: (tail -F log/my-app.log &) && my-app
Use a dedicated process manager in a Container, such as Forever or Supervisord, if multiple processes need to run in a Container or something else needs to run in the foreground.
Contact Aptible Support when in doubt.
Disabling filesystem wipes
Container Recovery automatically restarting containers with a pristine filesystem maximizes the odds of a Container coming back up when recovered and mimics what happens when restarting an App using aptible restart
.
Set the APTIBLE_DO_NOT_WIPE
Configuration variable on an App to any non-null value (e.g., set it to 1
) to prevent the filesystem from being wiped (assuming it is designed to handle being restarted properly).