Learn how to put your digital team to work with MuleSoft for Agentforce.
Contact Us 1-800-596-4880

Setting Up Monitoring for Hybrid Deployments

Install the Anypoint Monitoring agent on an on-premises server to monitor applications that are running on that server and managed in the cloud.

The Anypoint Monitoring agent sends monitoring data to the Anypoint Monitoring cloud endpoint. Anypoint Monitoring data is transferred using an open-source protocol for transferring log data over a highly efficient bidirectional binary channel. Data is securely transmitted using mutual TLS encryption.

To use Anypoint Monitoring across a server group or cluster, you must install Anypoint Monitoring on each individual server.

Before You Begin

Before installing the Anypoint Monitoring agent, set up and run a Mule runtime engine instance on your server. For guidance, see Run Mule Runtime Engine On-Premises.

If you have an outbound firewall, open ports based on the am.zip installer version for the Anypoint Monitoring agent:

  • 443 for all installer versions

  • 8443 for 3.x installer versions

Outbound connections are made to the following endpoints:

  • US Control Plane

    • 3.x installers:

      • us1.ingest.mulesoft.com:8443

      • data-authenticator.anypoint.mulesoft.com:443

      • s3.amazonaws.com:443

    • 2.x installers:

      • dias-ingestor-router.us-east-1.prod.cloudhub.io:443

      • data-authenticator.anypoint.mulesoft.com:443

      • s3.amazonaws.com:443

  • US-GOV Control Plane

    • data-authenticator.gov.anypoint.mulesoft.com:443

    • dias-ingestor-router.us-gov-west-1.gprod.msap.io:443

  • EU Control Plane

    • 3.x installers:

      • eu1.ingest.mulesoft.com:8443

      • data-authenticator.anypoint.mulesoft.com:443

      • s3.eu-central-1.amazonaws.com:443

    • 2.x installers:

      • dias-ingestor-router.eu-central-1.prod-eu.msap.io:443

      • data-authenticator.eu1.anypoint.mulesoft.com:443

      • s3.eu-central-1.amazonaws.com:443

Ports

Region Name Port

US

us1.ingest.mulesoft.com

8443

US

anypoint.mulesoft.com

443

US

mule-manager.anypoint.mulesoft.com

443

US

runtime-manager.anypoint.mulesoft.com

443

US

analytics-ingest.anypoint.mulesoft.com

443

US

arm-auth-proxy.prod.cloudhub.io

443

US

data-authenticator.anypoint.mulesoft.com

443

US

exchange-files.anypoint.mulesoft.com

443

US

exchange2-asset-manager-kprod.s3.amazonaws.com

443

US-GOV

runtime-manager.gov.anypoint.mulesoft.com

443

US-GOV

gov.anypoint.mulesoft.com

443

US-GOV

exchange-files.gov.anypoint.mulesoft.com

443

US-GOV

exchange2-asset-manager-kgprod.s3.us-gov-west-1.amazonaws.com

443

EU

eu1.ingest.mulesoft.com

8443

EU

eu1.anypoint.mulesoft.com

443

EU

mule-manager.eu1.anypoint.mulesoft.com

443

EU

runtime-manager.eu1.anypoint.mulesoft.com

443

EU

analytics-ingest.eu1.anypoint.mulesoft.com

443

EU

arm-auth-proxy.prod-eu.msap.io

443

EU

data-authenticator.eu1.anypoint.mulesoft.com

443

EU

exchange2-asset-manager-kprod-eu.s3.eu-central-1.amazonaws.com

443

Supported Operating Systems

Install the Anypoint Monitoring agent on a supported operating systems:

  • 3.x versions of the installer:

    • RedHat Enterprise Linux 9

    • Ubuntu 20.04

    • Windows Server 2019 (not currently supported by the 3.x agent installer)

  • 2.x versions of the installer:

    • CentOS 7

    • RedHat Enterprise Linux 7

    • Ubuntu 16.04 and 18.04

    • Windows Server 2016, Windows Server 2019

To install the agent, you must have root or administrative privileges.

Supported Versions of Mule Runtime Engine

Run your Mule app on a supported Mule runtime version. For supported versions, see Mule Runtime Release Notes.

Supported Versions of Mule Runtime Manager Agent

You must install a supported version of the Anypoint Runtime Manager agent in the Mule runtime engine running on the server. For more information, see Runtime Manager Agent.

  • 2.7.x

    An Anypoint Monitoring agent from the version 3.x installer requires Mule Runtime Manager agent version 2.7.1 or higher.

  • 2.6.x

  • 2.5.x

  • 2.4.x

  • 2.3.0

  • 2.2.x

  • 2.1.9

  • 2.1.7

  • 2.1.6

  • 1.14.x

  • 1.13.0

  • 1.12.x

  • 1.11.x

  • 1.10.0

You must manage Mule deployments in Runtime Manager. You can add and check for your hybrid deployments from the Runtime Manager UI or the Runtime Manager REST API.

Hardware Prerequisites

Ensure that the server has the minimum hardware requirements for Anypoint Monitoring, which are:

  • 2GHz CPU or 1 virtual CPU

  • 1GB RAM

  • 4GB disk space

  • Intel x86 or x64 compatible CPU architecture

Install the Anypoint Monitoring Agent

  1. Meet the prerequisites in Before You Begin:

    Mule runtime engine must be installed and running.

  2. Follow the steps in Managing CloudHub and Hybrid Settings to download the latest version of the agent’s am.zip file and install the agent from Settings, in Anypoint Monitoring.

    • When you extract the am.zip file, a folder called am is created in the location where you extracted the ZIP.

    • The installation script asks if you want Anypoint Monitoring installed as a service.

      • If you don’t want to have it preset as a service by the installer, select No.

      • You can call the installation script with the -x argument to skip this prompt. If you skip the prompt, Anypoint Monitoring is not installed as a service.

    • You cannot install the Anypoint Monitoring agent on a server that has an existing filebeat instance running.

    • When you install the Anypoint Monitoring agent on a hybrid server in a server group, use Runtime Manager to restart the server to enable Anypoint Monitoring to begin collecting metrics.

Proxy Settings

If you proxy your monitoring data as it leaves the data center, you must configure the Anypoint Monitoring agent to communicate through the proxy.

The configuration and supported protocol depends on the version of the Anypoint Monitoring agent’s installer:

  • 3.x: Supports the HTTP/HTTPS protocol and does not support SOCKS5

  • 2.x: Supports the SOCKS5 protocol and does not support HTTP/HTTPS

Configure Proxy Communication for Monitoring Agents from Version 3.x Installers (HTTP/HTTPS Protocol)

If you proxy your monitoring data as it leaves the data center and you’re using an Anypoint Monitoring agent from a version 3.x installer, the proxy must support the HTTP/HTTPS protocol, and you must configure the Anypoint Monitoring agent to communicate through the proxy.

To configure the agent for an HTTP or HTTPS proxy:

  1. Set up Runtime Manager with a proxy. See Create a Server in Runtime Manager.

  2. Install the Anypoint Monitoring agent and use the -p option. For example, on a Linux server, at the command line, enter:

    $ ./install -p https://user:password@https-server:1080

    For a user or password of the proxy configuration that contains URI-reserved characters, you must percent-encode the characters according to the encoding rules of RFC 3986.

Configure Proxy Communication for Monitoring Agents from Version 2.x Installers (SOCKS5 Protocol)

If you proxy your monitoring data as it leaves the data center and you’re using an Anypoint Monitoring agent from a version 3.x installer, the proxy must support the SOCKS5 protocol, and you must configure the Anypoint Monitoring agent to communicate through the proxy.

To configure the agent for the SOCKS5 proxy:

  1. Set up Runtime Manager with a proxy. See Create a Server in Runtime Manager.

  2. Install the Anypoint Monitoring agent and use the -p option. For example, on a Windows server, at the command line, enter:

    <location-of-am-folder\bin>powershell -file install.ps1 -p socks5://user:password@socks5-server:1080

    For a Linux server, enter:

    $ ./install -p socks5://user:password@socks5-server:1080

    For a user or password of the proxy configuration that contains URI-reserved characters, you must percent-encode the characters according to the encoding rules of RFC 3986.

Verify the Anypoint Monitoring Agent is Running

  1. Wait 10 minutes for the configuration to propagate.

  2. Log in to Anypoint Platform with a user account that has permission to access Anypoint Monitoring content.

  3. Navigate to Anypoint Monitoring.

  4. On the Built-in Dashboards page, select an app that is deployed on the server where you installed the Anypoint Monitoring agent.

  5. Check whether the CPU Utilization and Memory Utilization charts are displaying data.
    If you see data in the charts, the Anypoint Monitoring agent is installed and running.
    If you do not see data in the charts, verify that Anypoint Monitoring is able to connect to the endpoint.

    If necessary, you can add the endpoint to an allowlist for outbound firewall rules so your server allows connections. The endpoints to add to the allowlist are listed in Before You Begin.
  6. Log into the server on which the Anypoint Monitoring agent is running.

  7. Open the log file for the data-transfer agent:

    • 3.x log file path: ./am/logs

      If this agent is running as a service on Linux, the log file path is /var/log/otel-collector.

    • 2.x log file path: /var/log/filebeat

      If this agent is running as a service on Linux, the log file path is /var/log/filebeat`.

  8. Look for entries that mention connection failures or connection retries.

Update the Anypoint Monitoring Agent

When you upgrade the Runtime Manager agent, you must reinstall the Anypoint Monitoring agent.

If you need to update the Anypoint Monitoring agent at some point in the future, follow these steps:

  1. Stop Mule.

  2. In the am folder, run the following command:
    ./bin/uninstall

  3. Delete the am folder.

  4. Follow the instructions for installing the Anypoint Monitoring agent on-premises.

  5. Restart Mule.

If you add or remove a Mule runtime from the cluster or a server group, you must reinstall the Anypoint Monitoring agent.

Performance Impact

Enabling monitoring can impact CPU utilization and memory utilization depending on your specific application configuration. Before enabling monitoring in your production environment, test any performance impact in your pre-production environment to ensure optimal performance and sizing.

Troubleshooting

  • If you find connection failure or retry entries, ensure that port 8443 (Anypoint Monitoring agents from installer version 3.x and later only) and 443 (all Anypoint Monitoring agents) in your outbound firewall are open.

  • If you don’t find connection failure or retry entries, contact the support team.

Disable Log Forwarding for Hybrid Runtime Servers

info Availability: Advanced and Titanium tiers only

You can disable log forwarding to Anypoint Monitoring if you have AM installer v.2.5.10 or later. To disable this feature, you must configure a JVM property on each hybrid server for which you want to disable log forwarding. You cannot disable log forwarding per application.

To disable log forwarding:

  1. Upgrade the Anypoint Monitoring agent to a version that supports disabling log forwarding (am-2.5.10.0.zip or later).

  2. Stop Mule.

  3. Add the following line in the MULE_HOME/conf/wrapper.conf file (where <N> is the next available property index):
    wrapper.java.additional.<N>=-Danypoint.platform.monitoring.logs_forwarding.enabled=false

  4. Restart Mule.