Contact Us 1-800-596-4880
  1. Homepage
  2. Mule Runtime (4.11)

  3. Mule Troubleshooting Plugin

Mule Troubleshooting Plugin

Use the Mule Troubleshooting plugin to generate structured diagnostic information, simplify troubleshooting, and provide consistent data for Mule runtime support.

The Mule Troubleshooting plugin provides a unified way to collect diagnostic data from Mule runtime environments. It generates a structured diagnostic archive called the Diagnostic Information Analysis File (DIAF), which consolidates Mule runtime information, application metrics, and system data into a single, standardized output.

This Java-based plugin provides an extensible, environment-agnostic solution that simplifies troubleshooting for Mule runtime engineers, MuleSoft Support teams, customers running self-service diagnostics, and AI-assisted analysis.

Before You Begin

Before using the plugin, make sure that you have these prerequisites:

  • Supported Mule runtime distributions include version 4.10 and later, or LTS versions 4.9 (with patch 4.9.10 or later) and 4.6 (with patch 4.6.23 or later).

  • Java 8 or later, matching the Mule runtime version requirements.

  • Access to the $MULE_HOME directory. The CLI script diag automatically locates the Mule home directory.

The plugin works out-of-the-box in the Standalone, CloudHub, and CloudHub 2.0 deployment models without installing additional dependencies. For details about CloudHub 2.0 diagnostics, see Generating Apps Diagnostics and Analyzing with Einstein.

Using the Mule Troubleshooting Plugin

Run this command from your Mule runtime installation at $MULE_HOME/tools/diag to generate the DIAF and a thread dump. By default, the tool saves the files unzipped in the logs directory of the distribution.

Use the ./diag --extended command to generate a heap dump. Use ./diag --output some/dir/name/ to create the directories if they don’t exist and save unzipped files there. Use ./diag --output some/dir/name (without a trailing /) to create a ZIP file at that path containing all output files.

On Windows, run diag.bat. The --stdout option isn’t supported.

The plugin’s help output lists the available commands and options.

➜  mule-enterprise-standalone-4.6.23 ./tools/diag help
Mule Troubleshooting Tool
=========================

Usage: ./diag [options] [command] [command-options]

Commands:
  diaf                    Generate a complete Mule diagnostic dump (default)
  help                    Show this help message
  <operation-name>        Execute a specific troubleshooting operation

Global Options:
  --stdout                Output the diagnostic dump to standard output
  --output <path>         Specify custom output directory or file path
  --extended              Enable extended mode (includes heap dump)
  --debug                 Enable debug mode with remote debugging on port 5005

Examples:
  ./diag                          # Generate diagnostic dump to logs directory
  ./diag --stdout                 # Output diagnostic dump to stdout
  ./diag --extended               # Include heap dump in diagnostic
  ./diag --output /tmp/mule.zip   # Save to specific file
  ./diag --output /tmp/           # Save to specific directory
  ./diag <operation-name>         # Execute specific operation

Output:
  By default, the tool creates a ZIP file containing:
  - mule_dump_<timestamp>.diaf    # Diagnostic information
  - thread_dump_<timestamp>.txt   # Thread dump
  - heap_dump_<timestamp>.hprof   # Heap dump (if --extended is used)

  The ZIP file is saved to the 'logs' directory by default.

Understanding Diagnostic Information Analysis File (DIAF)

The Diagnostic Information Analysis File (DIAF) organizes all diagnostic data collected by the Mule Troubleshooting plugin into structured sections. Use this reference to understand the content of each section:

Title

This section shows the report generation timestamp.

Field Description

Report Generation Timestamp

The report generation time, expressed in the local time zone.

Basic Information

This section shows details about the environment where the Mule runtime instance is running.

Available in Mule runtime 4.11.6 and later. The JVM resource metrics in this section appear under the JVM Resources block and use the jvm. prefix (for example, jvm.memory.used). In earlier patch versions, these metrics appear at the top level without the prefix (for example, memory.used). The metrics have the same meaning in both formats. Operating system details and the system-wide CPU load no longer appear in this section. See System Info for those metrics.
Field Description

Mule Product/version

Product (CE/EE), version, and build number of the Mule runtime. Formatted as [productName] [version] (build [buildNumber]).

mule_home

Absolute path to MULE_HOME for the Mule runtime.

mule_base

Absolute path to MULE_BASE for the Mule runtime.

mule.* System Properties

All system properties starting with mule., including those defined by DataWeave and API Gateway. Listed with values and sorted alphabetically.

Java Version

Version of the JVM running the Mule runtime.

Java Vendor

Vendor of the JVM running the Mule runtime.

Java VM Name

Full name of the JVM running the Mule runtime.

JAVA_HOME

Location of the JVM running the Mule runtime.

Running Time

Total time the Mule runtime has been running.

PID

Process ID of the JVM running the Mule runtime.

Report Millis Time

Report generation time in milliseconds since epoch (System.currentTimeMillis).

Report Nano Time

Report generation time in nanoseconds (System.nanoTime).

memory.used (jvm.memory.used)

Amount of used memory in the JVM.

memory.free (jvm.memory.free)

Amount of available memory in the JVM.

memory.total (jvm.memory.total)

Total amount of memory in the JVM.

memory.max (jvm.memory.max)

Maximum amount of memory the JVM attempts to use.

memory.used/total (jvm.memory.used/total)

Percentage of used memory compared to the total allocated memory.

memory.used/max (jvm.memory.used/max)

Percentage of used memory compared to the maximum available memory.

processors.available (jvm.processors.available)

Number of processors available to the JVM process.

load.process (jvm.load.process)

Percentage of recent CPU usage for the JVM process; negative value if unavailable.

License

(Enterprise Edition only) License information for Enterprise Edition distributions appears under the License block. This block doesn’t appear in Community Edition or when no Enterprise license is registered. When present, it can include:

  • valid: Indicates whether the license is valid.

  • validationError: Indicates the reason the license is not valid, when applicable.

  • evaluation: Indicates whether the license is an evaluation license.

  • expirationDate: The license expiration date.

  • licenseFile: The license file in use.

  • contact: Contact information, including name, email, telephone, company, and country.

  • entitlements: The list of entitlements granted by the license.

In Mule runtime patches earlier than 4.11.6, this section also reports OS Name, OS Version, OS Arch, load.system, and load.systemAverage. Starting with Mule 4.11.6, the operating system details (host.os.) and the host CPU load (host.cpu.load.) are reported in the System Info section instead.

System Info

Available in Mule runtime 4.11.6 and later.

This section shows host-level diagnostics for the machine, container, or pod where the Mule runtime instance is running. The diagnostics are collected from the host operating system. Unlike Basic Information, which reports JVM-level metrics, this section reports host-level information. CPU load percentages are sampled over a short interval. Some fields are only populated on Linux hosts. On other operating systems, these fields can be absent or display n/a. If the runtime can’t collect host information, the report shows Host info not available instead.

Operating System

Field Description

host.os.name

Name of the host operating system.

host.os.version

Version of the host operating system.

host.os.arch

Architecture of the host operating system (for example, amd64 or aarch64).

CPU

Field Description

host.cpu.model

Processor model name.

host.cpu.cores.physical

Number of physical CPU cores on the host.

host.cpu.cores.logical

Number of logical CPUs (hardware threads) on the host.

host.cpu.load.user

Percentage of CPU time spent in user space during the sampling interval.

host.cpu.load.system

Percentage of CPU time spent in kernel or system space during the sampling interval.

host.cpu.load.idle

Percentage of idle CPU time during the sampling interval.

host.cpu.load.iowait

(Linux only) Percentage of CPU time spent waiting for input or output operations to complete. High values indicate disk or network I/O bottlenecks rather than CPU saturation.

host.cpu.load.steal

(Linux only) Percentage of CPU time taken by the hypervisor to serve other tenants. High values indicate a noisy-neighbor or over-committed virtualized host.

Memory

This section reports host physical memory, not the JVM heap.

Field Description

host.memory.total

Total physical RAM on the host.

host.memory.available

Physical RAM currently available.

host.memory.used

Physical RAM currently in use (total minus available).

host.memory.page.size

Memory page size, in bytes.

host.memory.swap.total

Total swap space configured on the host.

host.memory.swap.used

Swap space currently in use. Non-zero swap usage often indicates memory pressure.

host.memory.virtual.max

Maximum virtual memory (RAM plus swap) available.

host.memory.virtual.in.use

Virtual memory currently in use.

Disk Input and Output

(Linux only) For each physical disk device, the report includes information about the device. The report excludes pseudo-devices such as loop, ram, and dm-, and zero-size devices. A device can show (stats unavailable) if the runtime can’t read its per-device counters.

Field Description

model

Disk model.

size

Total size of the disk.

reads / writes

Number of read or write operations the device has served since the runtime started.

read.bytes / write.bytes

Total amount of bytes the device has read or written since the runtime started.

transfer.time.ms

Total time spent on transfers, in milliseconds.

JVM Agents

Available in Mule runtime 4.11.6 and later.

This section lists the JVM agents attached at JVM startup through the -javaagent, -agentpath, or -agentlib arguments. JVM agents can instrument the runtime, intercept class loading, redefine classes, and add overhead. Use this information to identify third-party, APM, and observability agents such as profilers, monitoring, security, and bytecode instrumentation agents, that can affect runtime behavior or interfere with the Mule runtime. The report doesn’t include agents attached dynamically after the JVM starts by using the Attach API. If no agents were attached at startup, the report shows No JVM agents detected. instead. Option values that appear to contain secrets, such as license keys, tokens, and passwords, are masked as ** in both the report and the debug logs.

Java Agents

Jar-based agents attached with -javaagent. For each agent, the report shows the JAR path, the option string, and metadata read from the JAR’s META-INF/MANIFEST.MF. If the JAR is missing or has no usable manifest, the report shows (no manifest metadata available).

Field Description

Path

Absolute path to the agent JAR.

Options

Option string passed to the agent. The report shows (none) if no options are given or (empty) if an empty option string is given. Sensitive values are masked.

Premain-Class

Class with the premain method that runs before the application’s main method, when the agent is attached at startup.

Agent-Class

Class with the agentmain method, used when the agent is attached to a running JVM.

Launcher-Agent-Class

Class used to launch the agent from an executable JAR.

Can-Redefine-Classes

Indicates whether the agent can redefine classes.

Can-Retransform-Classes

Indicates whether the agent can retransform classes.

Can-Set-Native-Method-Prefix

Indicates whether the agent can set a native method prefix.

Boot-Class-Path

Paths the agent adds to the bootstrap class loader search.

Implementation-Title

Title of the agent implementation.

Implementation-Vendor

Vendor of the agent implementation.

Implementation-Version

Version of the agent implementation.

Native Agents

Agents attached through -agentpath or -agentlib are native libraries, which don’t carry a manifest.

Field Description

Name

Name (for -agentlib) or path (for -agentpath) of the native agent.

Flag

Argument used to attach the agent through -agentpath or -agentlib.

Options

Option string passed to the agent. The report shows (none) if no options are given or (empty) if an empty option string is given. Sensitive values are masked.

File exists

Whether the native library is found on disk. Shown only for agents attached through -agentpath.

Statistics

This section shows detailed statistics information about deployed Mule applications and their performance metrics. Metrics reflect the runtime state since the last start or redeployment. They reset after redeployments and don’t capture complete historical data. Note that this information represents a snapshot, or point in time, of the runtime behavior and can differ from the information in the Anypoint Platform usage report, which reflects a period of time.

Set the mule.enable.statistics system property to collect General Application Metrics and Flow Statistics.

Flow Summary Statistics

Field Description

Private Flows Declared

Total number of private flows declared in the application. A private flow doesn’t contain a MessageSource and isn’t used by an APIkit router.

Private Flows Active

Number of private flows that are currently in a started state.

Trigger Flows Declared

Total number of trigger flows declared in the application. A trigger flow contains a MessageSource.

Trigger Flows Active

Number of trigger flows currently in a started state.

API Kit Flows Declared

Total number of APIkit flows declared in the application. An APIkit router uses an APIkit flow, but the flow doesn’t contain a MessageSource.

API Kit Flows Active

Number of APIkit flows currently in a started state.

General Application Metrics

This section shows counter metrics for the application or flow. Counter metrics include cumulative totals and rolling counts for the last 1, 5, 15, and 60 minutes (shown next to the total as (rolling) 1m: …​ 5m: …​ 15m: …​ 60m: …​).

Field Description

Events Received

Number of events received by the application or flow.

Events Processed

Number of events processed by the application or flow.

Messages Dispatched

Total number of messages dispatched from message sources within the application.

Execution Errors

Number of execution errors encountered.

Fatal Errors

Number of fatal errors that cause the application to fail or stop processing.

Connection Errors

Number of connection-related errors that occur.

Average Processing Time

Average time (in milliseconds) required to process an event.

Min Processing Time

Minimum time (in milliseconds) required to process an event.

Max Processing Time

Maximum time (in milliseconds) required to process an event.

Total Processing Time

Cumulative time (in milliseconds) spent processing all events.

Processing Time (rolling)

Aggregated processing-time figures for rolling windows of the last 1, 5, 15, and 60 minutes. Each window reports min, max, avg, and count (number of processed events that contributed to the window). When count is 0, min, max, and avg are reported as 0.

Flow Statistics

This section shows counter metrics for the flow. Counter metrics include cumulative totals and rolling counts for the last 1, 5, 15, and 60 minutes (shown next to the total as (rolling) 1m: …​ 5m: …​ 15m: …​ 60m: …​).

Field Description

Events Received

Total number of events received by the application since it started.

Events Processed

Total number of events successfully processed by the application.

Messages Dispatched

Total number of messages dispatched from message sources within the application.

Execution Errors

Number of execution errors that occur during event processing.

Fatal Errors

Number of fatal errors that cause the application to fail or stop processing.

Connection Errors

Number of connection-related errors that occur.

Average Processing Time

Average time (in milliseconds) required to process an event.

Processing Time (rolling)

Aggregated processing-time figures for rolling windows of the last 1, 5, 15, and 60 minutes. Each window reports min, max, avg, and count (number of processed events that contributed to the window). When count is 0, min, max, and avg are reported as 0.

Alerts

This section shows alerts for known Mule runtime issues. The report lists how many times each alert triggers during the last 1, 5, 15, and 60 minutes along with the context of the alert at the time of triggering. Some alerts, like the backpressure alert, trigger multiple times with the same context, so the plugin shows the context once and indicates how many times it happens to avoid flooding the report. Alerts that don’t trigger in any of the time intervals aren’t included in the report.

Field Description

MULE:UNKNOWN error raised

The runtime generated MULE:UNKNOWN errors, which go unhandled. If such an error raises or appears in the app log, it indicates an error in the Mule runtime. The context shows the details of the errors categorized as MULE:UNKNOWN.

Reactor discarded event

A discarded event is one that a component explicitly filters in a flow. This cuts the processing of such an event, causing the execution to hang. The context shows the correlation ID of each discarded event.

Reactor dropped event

A dropped event doesn’t pass to the following component in a flow through a reactor chain and doesn’t complete. Its symptom is that the event is “hanged”. The alert doesn’t show context because the event dump provides the information.

Reactor dropped error

A dropped error doesn’t pass to the corresponding error handler in a flow through a reactor chain, and so doesn’t complete. Its symptom is that the event is “hanged” when an error occurs. The context shows the string representation of each dropped error.

Not consumed stream

A stream that is garbage collected before being completely consumed can provoke leaks on certain conditions (the most common one is connections from a DB connection pool that remain taken until the data is fully read). The context shows the originating location of the components that generated the streams.

Backpressure triggered

Backpressure is the mechanism that rejects incoming events that exceed the current capacity. This happens because of a spike of incoming events or a longer than usual processing time of the flows. A common sign is when backpressure triggers on systems that have a CPU and memory capacity. The context shows the flow or component that exceeded capacity and the reason for backpressure.

XA recovery start error

Triggered if recovery of an XA connection fails to start. The context shows the unique name (including the configuration name) of the connection for which recovery fails.

Async logger ringbuffer full

When a log appender writes logs slower than the log entries are generated, the logger ringbuffer fills up. When full, threads attempting to log either wait for space in the ringbuffer or log synchronously, depending on the configuration. In either case, a thread that shouldn’t block or wait does so, causing performance issues in the Mule runtime. No context is available for this alert because it always means the same, the buffer is full.

Event dispatched after flow stop

If a source dispatches an event after the flow stops, the graceful shutdown period extends unnecessarily. The context shows the connector that owns the flow’s source, and a thread dump helps you identify why the source doesn’t finish stopping.

OTel export queue full

If OpenTelemetry export queues fill up, Mule runtime either drops the exceeding spans and logs or blocks the thread that tries to add them to the export queue until space is available. Dropping these resources results in missing data, while blocking threads can cause performance issues. System logs specify which spans or logs were affected and whether they were dropped or blocked.

The report provides hints about potential issues. For details on a specific alert, query the log. This report isn’t intended to replace the log for detailed analysis.

Event Dump

This section shows a hierarchical listing of in-flight events. For each event hierarchy executing through a flow in Mule has at least one entry in the report. For each child context for the event, a nested entry appears, sorted in a stack order: children on top, parents on bottom. If an event is dropped, the legend DROPPED appears next to it.

Field Description

eventId

A unique identifier for the event. For child events, it has the ID of the parent event context as prefix.

runningTime

How long the event has been running. For child events, this time refers to the execution of this child context. The format is “mm:ss”.

eventContextState

  • PENDING: Event has been created but not yet accepted by the flow for processing.

  • EXECUTING: Event is being executing by the flow or executable component, or has finished but the response is still being processed.

  • WAITING_NON_BLOCKING: Event is waiting for a non-blocking operation to complete.

  • RESPONSE_PROCESSED: Event execution is complete and the response is handled.

  • COMPLETE: Same as RESPONSE_PROCESSED, and all child events are RESPONSE_PROCESSED.

  • TERMINATED: After COMPLETE, and all completion callbacks of the context execute.

flowStack

flowStack contains zero-to-many lines, each with this format.

at [componentId]@[componentLocation]([muleFileName]:[muleFileLineNumber]) [timeInLocation] ms

flowStack.componentId

Identifier of the component (for example, http:request).

flowStack.componentLocation

Unique identifier of a component within a Mule application. The first part is the flow or policy name, followed by the index and chains that nests the component.

flowStack.muleFileName

Name of the Mule configuration file that contains the component.

flowStack.muleFileLineNumber

Line number in the Mule configuration file that contains the component.

flowStack.timeInLocation

Duration in milliseconds the event spends at the flowStack entry.

Schedulers

This section shows the status and metrics of schedulers provided by the scheduler service, which the Mule runtime manages internally, not the source components themselves. For Mule runtime instances with multiple deployed applications, entries are grouped by application.

Field Description

schedulerName

Name assigned to the scheduler when created, showing where in the code it happened.

threadType

Type of tasks the scheduler runs:

  • IO: A task that spends most of its execution waiting for I/O operations to complete.

  • CPU_INTENSIVE: A task that runs longer than 10 milliseconds, with less than 20% of time blocked.

  • CPU_LIGHT: A task that never blocks and runs is less than 10 milliseconds.

  • CUSTOM: Threads that aren’t managed by Mule runtime or shared among schedulers. Used when a thread pool needs exclusive use (for example, NIO selectors).

shutdown

A shutdown scheduler doesn’t accept new tasks. Tasks still running are allowed a graceful period to complete.

terminated

A terminated scheduler is shut down and all in progress tasks are completed or forcefully terminated after a graceful shutdown period.

activeTasks

Number of tasks currently executing by the scheduler.

queuedTasks

Number of tasks waiting in a queue. Not shown if there’s no queue, the queue size can’t be queried, or no tasks are queued.

rejections

Number of tasks rejected because the scheduler is at capacity. Shows rejections in the last 1, 5, 15, and 60 minutes. If there aren’t any in those intervals, the alert isn’t shown.

throttles

Number of tasks throttled because the scheduler is at capacity. Shows throttles in the last 1, 5, 15, and 60 minutes. If there aren’t any in those intervals, the alert isn’t shown.

CPU Usage

This section shows the CPU usage for each thread in the Mule runtime process. Use it to identify high CPU consumption that can affect application performance. The data is sorted by current CPU usage to highlight the most active threads.

Field Description

Thread ID

Unique identifier of the thread within the Mule runtime process.

%CPU

Percentage of CPU consumed by the thread during the last three-second sampling interval, calculated like the Linux top command. This metric provides a short-term view of CPU activity instead of a lifetime average.

Thread Name

Name of the thread as reported by the JVM.

State

Current execution state of the thread (for example, RUNNABLE, WAITING, TIMED_WAITING, or BLOCKED).

CPU Time (3s)

Total CPU time the thread consumed during the most recent three-second measurement interval, in milliseconds.

Total CPU Time

Total CPU time the thread has consumed since the Mule runtime process started, in milliseconds.

Deployments

This section shows the status and metadata of all artifacts currently deployed in the Mule runtime instance: applications, domains, policy templates, and policies applied to applications. It also shows undeployed artifacts still present in memory, which can indicate possible leaks.

Applications

For each application, the report includes this information:

Field Description

Artifact Name

Name of the application.

Artifact Type

APPLICATION.

State

Current state of the application:

  • INITIAL

  • DEPLOYED

  • STARTED

  • STOPPED

  • UNDEPLOYED

  • FAILED

Domain

Name of the domain the application uses. Shown only if the application uses a non-default domain.

Policies

List of policy IDs applied to the application. Shown only if the application has policies.

Plugins

List of all plugins available in the application’s deployment.

Time since Last State Update

Timestamp of the most recent state update, formatted as yyyy-MM-dd HH:mm:ss.

Feature Flags

List of all enabled feature flags for the application, sorted alphabetically. Shown only for DEPLOYED, STARTED, or STOPPED applications.

Domains

For each domain (excluding the default domain), the report includes this information:

Field Description

Artifact Name

Name of the domain.

Artifact Type

DOMAIN.

State

Current state of the domain:

  • INITIAL

  • DEPLOYED

  • STARTED

  • STOPPED

  • UNDEPLOYED

  • FAILED

Plugins

List of all plugins available in the domain’s deployment.

Time since Last State Update

Timestamp of the most recent state update, formatted as yyyy-MM-dd HH:mm:ss.

Policy Templates

Policy templates appear as separate top-level entries. Each entry represents a unique policy template used by one or more applications.

Field Description

Template Name/ID

Name or identifier of the policy template.

Artifact Type

POLICY_TEMPLATE.

Version

Version of the policy template.

Plugins

List of all plugins available in the policy template.

Connectors State

Available in Mule runtime 4.11.6 and later.

This section reports the state of each connector configuration in the deployed applications, including the connection lifecycle, connection pool information, recent errors, and TLS details. When more than one application is deployed, the report groups connector configurations by application. If an application isn’t started, the report shows Application is not started — no artifact context available.. If an application has no connector configurations, the report shows No connector configurations found..

For each connector configuration, the report includes a header with the name, the Extension it belongs to, and whether the configuration is Dynamic (resolved per event from expressions). The report fields depend on whether the configuration is static or dynamic.

Field Description

Connection Provider

Class name of the resolved connection provider. If the configuration doesn’t declare one, the report shows (configuration does not declare a connection provider).

TLS details

TLS context information for the connection provider, including certificate validity where applicable. The report shows this information only for providers that use TLS.

Connection State

A block of connection statistics for the configuration, shown as (no statistics available) when none have been recorded. When present, the block contains:

  • Last Used: Timestamp of the most recent use of the connection.

  • Active Components: Number of components currently using this configuration.

  • Last Connection Attempt: Timestamp of the most recent connection attempt.

  • Last Successful Conn.: Timestamp of the most recent successful connection.

  • Pool Info: For pooling connection providers, the connection pool state: active, idle, maxTotal, maxIdle, created, borrowed, returned, and destroyed connection counts. An active count at or near maxTotal indicates the pool is saturated.

  • Last Error: Timestamp and details of the most recent connection error, when one has occurred.

Usages

(Static configurations) Component locations that reference this configuration, or (not referenced by any component) / (no usages created yet) if there are none.

Last Resolution Failure

(Dynamic configurations) Timestamp and details of the most recent failure to resolve the dynamic configuration, or Not available if there are none.

Last Successful Creation

(Dynamic configurations) Timestamp of the most recent successful resolution of a configuration instance, or Not available if there are none.

Active Usages

(Dynamic configurations) Currently active resolved instances, each with its own Connection State block, or (no active dynamic usages) if there are none.

Cluster Info

This section applies to Enterprise Edition only and is available in Mule runtime 4.11.6 and later.

This section reports the clustering configuration of the local node. When lock diagnostics are enabled, it also reports the distributed locks currently held across the cluster. This section isn’t available in Community Edition or non-clustered runtimes. If the node isn’t part of an active cluster, the report shows Cluster is not active. No cluster information available..

Cluster Configuration

The report mirrors the cluster configuration of the local node.

Field Description

clusterId

Identifier of the cluster the node belongs to.

clusterNodeId

Identifier of the local node in the cluster.

clusterSize

Number of nodes currently in the cluster.

clusterSchema

Cluster schema in use.

multicastEnabled

Indicates whether multicast is enabled for cluster discovery.

tcpInboundPort

TCP port the node uses for inbound cluster communication.

nodes

IP addresses of the nodes in the cluster.

networkInterfaces

Network interfaces used for cluster communication.

Lock Diagnostics

Available only when lock diagnostics are enabled on the runtime. Reports the local member, the number of reporting members, and the distributed locks currently held. For each held lock, the report includes the lockId and the lock holder details.

Field Description

holder.nodeId

Node holding the lock, shown as LOCAL or REMOTE.

holder.thread

Name of the thread that holds the lock.

holder.threadId

Identifier of the thread holding the lock.

holder.threadState

State of the holding thread, or n/a for remote nodes.

holder.acquiredAt

Timestamp when the lock was acquired.

holder.heldFor

How long the lock has been held.

holder.callerClass

(When acquisition stack capture is enabled) Class that acquired the lock.

holder.acquireStack

(When acquisition stack capture is enabled) Stack frames captured when the lock was acquired.

hz.isLocked

Hazelcast-level lock state, or hz.state: unavailable if it can’t be queried.

The report also lists orphan locks, which are held locks that have no owning node in any member’s registry, for example, the holding node stopped before releasing the lock. If acquisition stack traces aren’t enabled, the report includes a note explaining how to enable them.

Considerations

  • DIAF provides investigation hints. Check the logs for complete details.

  • Heap dumps can contain sensitive data. Enable the --extended option only in secure environments.

  • In Mule runtime instances with multiple applications, DIAF sections are grouped by application.

  • Use DIAF for initial troubleshooting before collecting heap or thread dumps manually.

  • Correlate events in the event dump section with logs by using the eventId for deeper analysis.

  • Collect scheduled diagnostics during maintenance windows in production environments.

  • To verify if all the hosts defined in your deployable artifacts (domains, applications, policies) support TLS 1.2 and 1.3 connectivity, enable the mule.extractConnectionData.enable system property. On UNIX, the tool generates <ORIGINAL_CSV_NAME>_tls_results.csv along with DIAF output. Enable the mule.extractConnectionData.silentErrors system property to log errors without failing deployment. Not available for Windows.

See Also