Automatic App Restarts

CloudHub 2.0 restarts your Mule applications during infrastructure failure or maintenance, security patching, replica crashes, or replica rebalancing.

The CloudHub 2.0 infrastructure gracefully shuts down by draining connections, transitioning traffic to a new instance of your application replica before terminating the older instance.

To ensure business resiliency, design your application to be idempotent and deploy across multiple replicas with clustering enabled with a layer of persistence (Object Store v2).

Understand Local Storage Limitations

CloudHub 2.0 replica local storage is ephemeral. Unlike local development environments, CloudHub 2.0 clears local data when an app restarts.

This limitation affects:

Persistent object stores: Configuring persistent="true" writes data to local disk. This works in Anypoint Studio and standalone Mule runtimes, but data doesn’t persist across CloudHub 2.0 restarts. Use Object Store v2 for persistent storage.
Local file operations: The File connector and other file-based operations write to the replica’s local disk. CloudHub 2.0 deletes these files when the app restarts.
Temporary files and caches: App-generated files, custom logs, and local cache data don’t persist.

To ensure data persistence in CloudHub 2.0:

Use Object Store v2 instead of persistent object stores for app state and synchronization data.
Store files in external cloud storage services.
Design stateless integrations by persisting data in external databases or messaging queues.

For more information about designing resilient applications, see CloudHub 2.0 High Availability and Disaster Recovery.

Shutdown Timeout Behavior

CloudHub 2.0 overrides the Mule runtime’s default shutdownTimeout value to avoid disruption for long-running flows. By default, the Mule runtime sets shutdownTimeout to five seconds. CloudHub 2.0 extends the shutdown grace period to accommodate applications that require more time to complete in-flight processing.

For infrastructure-triggered restarts (such as infrastructure failure, maintenance, security patching, or replica rebalancing):

CloudHub 2.0 provides a grace period up to a maximum of 12 hours.
If no custom shutdownTimeout is configured, CloudHub 2.0 applies a 12-hour maximum grace period to ensure long-running flows complete without disruption.

For user-triggered restarts (such as stop, restart, update, redeploy, or terminate actions):

CloudHub 2.0 honors the configured shutdownTimeout value from your application configuration.
CloudHub 2.0 doesn’t provide the extended 12-hour grace period.

You can optionally configure the shutdownTimeout attribute in your Mule application configuration. See Global Configurations Reference.

<mule ...>
  <configuration shutdownTimeout="150000" /> <!-- This sets shutdownTimeout to 2 minutes, 30 seconds. Default is 5 seconds. -->
</mule>

Considerations

Review these details about how each replica strategy manages application optimization:

Single Replica with Recreate Strategy: The sole replica is terminated first, allowing it to gracefully shut down. A new replica launches as soon as the previous replica enters Stopping status. Incoming traffic experiences a brief downtime. If the application schedule triggers during this downtime, the trigger is missed.
Single Replica with Rolling Restart: A new replica launches before the old replica begins its graceful shutdown. Two replicas run concurrently until the old replica is fully terminated. Once in Stopping status, the old replica stops receiving new connections or triggering application schedules.
Multiple Replicas with Recreate or Rolling Restart: At least two thirds of the replicas remain available throughout the optimization process. Some replicas are gracefully shut down and restarted on new infrastructure.

Infrastructure Failure and Maintenance

If the compute infrastructure suffers a failure, CloudHub 2.0 moves your application to a healthy infrastructure and restarts the application replicas. Likewise, during infrastructure maintenance, the platform moves application replicas running on a compute infrastructure that becomes impaired or end-of-life onto a new infrastructure to ensure availability.

Security Patching

To mitigate security vulnerabilities, CloudHub 2.0 patches and restarts applications every month. See the Critical Security Vulnerabilities policy.

Replica Crash

If an application replica crashes for any reason, CloudHub 2.0 automatically restarts it. If it continues to fail to start, the platform attempts to restart it using an exponential backoff policy.

Replica Rebalancing

The CloudHub 2.0 infrastructure optimizes application efficiency through replica rebalancing. This process gracefully moves and restarts replicas, provided there are at least two replicas available.

During rebalancing, a maximum of 30% of an application’s replicas are rebalanced concurrently. The system rounds up this percentage to the number of replicas that can be disrupted. For example, if your application has two replicas, a disruption affects one replica, exceeding the configured 30%.

This table shows the potential impact of rebalancing operations on your applications based on the number of replicas.

Total Application Replicas	Maximum Replicas Disrupted
1	1
2	1
3	1
4	2
5	2
6	2
7	3
8	3
9	3
10	3
11	4
12	4
13	4
14	5
15	5
16	5

This method ensures that the application’s configuration, including the Mule runtime version and the infrastructure, remains consistent after all restarts, except for restarts occurring after a security patching. For more details about security patching, see CloudHub 2.0 Operating System Patch Updates.