Operations

Learn more about operations work on Aptible - with minimal downtime and rollbacks

Overview

An operation is performed and recorded for all changes made to resources, environments, and stacks. As operations are performed, operation logs are outputted and stored within Aptible. Operations are designed with reliability in mind - with minimal downtime and automatic rollbacks.

A collective record of operations is referred to as activity.

Type of Operations

• backup: Creates a database backup
• configure: Sets the configuration for an app
• copy: Creates a cross-region copy database backup
• deploy: Deploys a Docker image for an app
• deprovision: Stops all running containers and deletes the resources
• execute: Creates an ephemeral SSH session for an app
• logs: Streams logs to CLI
• modify: Modifies a database volume type (gp3, gp2, standard) or provisioned IOPS (if gp3)
• provision: Provisions a new database, log drain, or metric drain
• purge: Purges a database backup
• rebuild: Rebuilds the Docker image for an app and deploys the app with the newly built image
• reload: Restarts the database in place (does not alter size)
• replicate: Creates a replica for databases that support replication.
• renew: Renews a certificate for an app endpoint using Managed HTTPS.
• restart: Restarts an app or database
• restore: Restores a database backup into a new database
• scale: Scales a service for an app
• scan: Generates a security scan for an app

Operation Logs

For all operations performed, Aptible collects operation logs. These logs are retained only for active resources and can be viewed in the following ways:

• Within the Aptible Dashboard:
  • Within the resource summary by:
    • Navigating to the respective resource
    • Selecting the Activity tab
  • Within the Activity dashboard by:
    • Navigating to the Activity page
    • Selecting the Logs button for the respective operation
      • Note: This page only shows operations performed in the last 7 days.
• Within the Aptible CLI by using the aptible operation:logs command
  • Note: This command only shows operations performed in the last 90 days.

Activity Dashboard

The Activity dashboard provides a real-time view of operations for active resources in the last seven days. Through the Activity page, you can:

• View operations for resources you have access to
• Search operations by resource name, operation type, and user
• View operation logs for debugging purposes

📘 Tip: Troubleshooting with our team? Link the Aptible Support team to the logs for the operation you are having trouble with.

Activity Reports

Activity Reports provide historical data of all operations in a given environment, including operations executed on resources that were later deleted. These reports are generated on a weekly basis for each environment, and they can be accessed for the duration of the environment's existence.

Activity Reports can be downloaded in CSV format within the Aptible Dashboard by:

• Selecting the respective Environment
• Selecting the Activity Reports tab

All Activity Reports for an Environment are accessible for the lifetime of the Environment.

Operation Logs

Aptible collects Operation Logs, which can be viewed in the following ways:

• By navigating to the Aptible Dashboard
  • Navigating to the Environment in which your App or Database lives in
  • Selecting the respective App or Database
  • Selecting the Activity tab
  • Selecting the download ↓ icon

• Using the aptible operation:logs command

Minimal downtime operations

To further mitigate the impact of failures, Aptible Operations are designed to be interruptible at any stage whenever possible.

In particular, when deploying a web application, Aptible performs Zero-Downtime Deployment. This ensures that if the Operation is interrupted at any time and for any reason, it still won't take your application down.

When downtime is inevitable (such as when resizing a Database volume or redeploying a Database to a bigger instance), Aptible optimizes for minimal downtime.

For example, when redeploying a Database to another instance, Aptible must perform the following steps:

• Shut down the old Database Container.
• Unmount and then detach the Database volume from the instance the Database was originally scheduled on.
• Attach then remount the Database volume on the instance the Database is being re-scheduled on.
• Start the new Database Container.

When performing this Operation, Aptible will minimize downtime by ensuring that all preconditions are in place to start the new Database Container on the new instance before shutting down the old Database Container. In particular, Aptible will ensure the new instance is available and has pre-pulled the Docker image for your Database.

Operation Rollbacks

Aptible was designed with reliability in mind. To this extent, Aptible provides automatic rollbacks for failed operations. Users can also manually rollback an operation should they need to.

🧠 Why do Operation Failures happen?
Reliability is a top priority at Aptible in general and for Aptible in particular. That said, occasional failures during Operations are inevitable and may be caused by the following:\-Failing third-party services: Aptible strives to minimize dependencies on the critical path to deploying an App or restarting a Database, but Aptible nonetheless depends on a number of third-party services. Notably, Aptible depends on AWS EC2, AWS S3, AWS ELB, and the Docker Hub (with a failover or Quay.io and vice-versa). These can occasionally fail and when they do, they may cause Aptible Operations to fail.\-Crashing instances: Aptible is built on a fleet of Linux instances running Docker. Like any other software, Linux and Docker have bugs and may occasionally crash. Here again, when they do, Aptible operations may fail

Automatic Rollbacks

All Aptible operations are designed to support automatic rollbacks in the event of a failure (with the exception of a handful of trivial operations with no side effects (such as launching Ephemeral SSH Sessions).

When a failure occurs, and an automatic rollback is initiated, a message will be displayed within the operation logs. The logs will indicate whether the rollback succeeded (i.e., everything was restored back to the way it was before the Operation) or failed (some changes could not be undone).

❗️ Some side-effects of deployments cannot be rolled back by Aptible. In particular, database migrations performed in before_release commands cannot be rolled back (unless you design your migrations to roll back on failure, of course!). We strongly recommend designing your database migrations so that they are backwards compatible across at least one release. This is a very good idea in general (not just on Aptible), and a best practice for zero-downtime deployments (see Concurrent Releases for more information).

Manual Rollbacks

A rollback can be manually initiated within the Aptible CLI by using the aptible operation:cancel command.

Featured Guides

• How to access operation logs