Auto-Clustering Configuration - Documentation topics on: auto clustering,cache,cluster,cluster_autowire,cluster configuration,clustered plugins,clusters,elasticsearch,enterprise only,lb,license management,load balancer,osgi,osgi cluster,.

Auto-Clustering Configuration

The Auto-clustering feature allows you to automatically add servers to a cluster, and automatically manages the cluster configuration for you. The following configuration steps are required to set up auto-clustering.

Note: You must complete all installations steps in the Cluster Setup document before performing any of these installation steps.

Download and Apply a License Pack

Auto-clustering requires a valid Enterprise license pack. Enterprise clients can download license packs from the dotCMS Support website. Contracts that have licenses available provide the option to download license packs.

If you are an Enterprise client (or would like to move to Enterprise), please contact dotCMS Support for more information about Enterprise contracts and support portal options.

Once downloaded from the support portal, license packs for auto-clustering should be uploaded to only one server that is part of the cluster. Once the license pack is uploaded, a license can be chosen for the current server.

For more information on applying licenses, see the License Management documentation.

Add Additional Servers to the Cluster

The rest of the server nodes in the cluster do not need their own database or assets folder, and do not need separate licenses. Instead, each additional server in the cluster must only:

  • Have a symbolic link to the same asset directory as the first server.
  • Share a connection to the same database.

No other configuration is required. However the following settings found in the file can be used to modify Auto-clustering behavior:

AUTOWIRE_CLUSTER_TRANSPORTtrueActivates the auto-wire process at cluster cache transport level.
AUTOWIRE_CLUSTER_EStrueActivates the auto-wire process at the elastic search index level.
(30 minutes)
Timeout (in seconds) before a node will be considered unresponsive (please see Unresponsive Nodes, below).
Note: 30 minutes without a heart beat provides a reasonable time window to perform normal server changes without dropping the node from the cluster.
(once every minute)
The frequency to check for dead servers in the cluster.

Note: It is strongly recommended that all changes to the file be made through a properties file extension.

As each additional node starts up, it will recognize that it shares the same database and assets directory as the first node in the cluster, and will automatically search for an available license from the license pack already uploaded to the first server. If a license is available, it will automatically be assigned, as long as there are enough valid licenses in the license pack for each node. Additional license packs can always be uploaded to the first server, allowing scaling of the cluster over time.

Unresponsive Nodes

If a node's “heartbeat” is not detected before the HEARTBEAT_TIMEOUT expires, then the node will be dropped from the cluster. If a node is dropped, dotCMS will:

  1. Free the license.
  2. Update the Elasticsearch replica count.
  3. Rewire the cache and Elasticsearch.

If a temporarily disconnected server is re-discovered within 30 minutes, the server can still re-join the cluster normally.

Activate License Icon

Elasticsearch Replica Management

By default, the Auto-clustering feature handles the replicas for Elasticsearch. The default configuration setting AUTOWIRE_MANAGE_ES_REPLICAS=true disables backend UI manual management of clustered elastic search replicas. However, this can be changed by changing the setting to false AUTOWIRE_MANAGE_ES_REPLICAS=false in the file. This enables manual right click management of Elastic Search replicas in the Network tab of the System -> Configuration tool in the dotCMS back-end. In this case dotCMS is no longer actively handling replica management. When AUTOWIRE_MANAGE_ES_REPLICAS is set to false, the default ElasticSearch replica count can still be configured using the es.index.number_of_replicas setting. If dotCMS autowire replica management is disabled, then replicas must be managed manually via the System→Configuration→Network portlet and/or by using dotCMS REST API commands.

If you are using an external ElasticSearch cluster with dotCMS running in Elasticsearch client-mode, it is recommended that you disable autowire replica management and manage your replicas directly in ElasticSearch. For more information, please consult the ElasticSearch documentation or dotCMS Support for the best practices in your environment.

For more information on setting the number of replicas, please see the Manual Cluster Configuration documentation.

Auto-Cluster Configuration Checklist

The following checklist outlines the steps required to establish an auto-cluster environment:

  1. Have a valid Enterprise contract with available server licenses.
  2. Request a license pack by submitting a ticket on the dotCMS Support website.
  3. Start up the first dotCMS server (server #1).
  4. Upload the license pack to the License Manager of server #1.
  5. Select a license from the pack to be used by server #1.
  6. Configure the second dotCMS server to share the same assets and database.
  7. Start up the second server (it will automatically join the cluster).

To add additional server nodes to the cluster, repeat steps 6 & 7 and each node will automatically be joined to the cluster (as long as unused licenses are still available*).

* Warning: If multiple nodes in the cluster are on the same physical server, then the Tomcat port and shutdown ports will need to be different for each node in the cluster. Ports can be configured in the server.xml file (/dotserver/tomcat-X.xx/conf/server.xml), to avoid conflicts.

Verifying Your Cluster

After adding all nodes to your cluster, please see the Testing Your Cluster section in the Cluster Setup documentation for steps to verify the proper operation of your cluster.