forked from wildfly/wildfly.org
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[wildfly#623] Add guilde for configuring messaging in high availabili…
…ty for Wildfly
- Loading branch information
Showing
3 changed files
with
234 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,228 @@ | ||
| = Configure WildFly With Messaging (ActiveMQ Artemis) Cluster And High Availability | ||
| :summary: Learn how to configure WildFly with messaging (ActiveMQ Artemis) cluster and high availability | ||
| :includedir: _includes | ||
| include::{includedir}/_attributes.adoc[] | ||
| // you can override any attributes eg to lengthen the | ||
| // time to complete the guide | ||
| :prerequisites-time: 20 | ||
|
|
||
| In this guide you will learn how to configure two WildFly servers with messaging (ActiveMQ Artemis broker) in high availability topology with shared journal. | ||
|
|
||
| Apache ActiveMQ Artemis broker is integrated in WildFly and allows Artemis brokers to be linked together in primary-secondary pairs. | ||
| It is active-passive high availability design such that one (the primary node) handles all the workload under normal operating conditions, | ||
| while the other (the secondary node) remains on standby, ready to take over if the active node fails. As usually it’s not desirable to keep | ||
| one Wildfly server idle on standby mode, we will configure one additional ActiveMQ Artemis server on each WidFly server as a secondary for the other. | ||
| So there will be two primary-secondary pairs, each pair with its own journal directory. | ||
|
|
||
| image::artemis-collocated-topology.png[WildFly with ActiveMQ Artemis in collocated HA topology] | ||
|
|
||
| If one WildFly server fails, the secondary ActiveMQ Artemis broker on the other WildFly server will activate and take over all duties. | ||
| All HA enabled connections from the original primary broker will fail over to the secondary broker. The secondary broker will | ||
| access all messages stored on the primary broker by loading them from the shared journal directory. Once the original WildFly server is restarted, | ||
| the secondary broker will automatically switch back to standby mode, and all clients will fail back to the primary broker. | ||
| Something like: | ||
|
|
||
| [NOTE] | ||
| -- | ||
| For simplicity, we will use just one machine to run WildFly instances and local directories for shared journals. In real world scenario those should be two separate machines with shared journal mount from shared file system (like NFS4). | ||
| -- | ||
|
|
||
| [[prerequisites]] | ||
| == Prerequisites | ||
|
|
||
| To complete this guide, you need: | ||
|
|
||
| * Roughly {prerequisites-time} minutes | ||
| * JDK {jdk-minimal-version} installed with `JAVA_HOME` configured appropriately | ||
|
|
||
| == Prepare shared journal directories | ||
|
|
||
| Create two directories for each primary-secondary pair (Change <path> based on your environment): | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| mkdir messaging-journal-a | ||
| mkdir messaging-journal-b | ||
| ---- | ||
|
|
||
| == Prepare WildFly Servers | ||
|
|
||
| Now there will be configured two WildFly servers with two primary-secondary pairs. ActiveMQ Artemis configured | ||
| as primary on the first WildFly server and its secondary on the second WildFly server will use `messaging-journal-a` | ||
| directory for their shared journal. Similarly primary broker on second WildFly server and secondary broker on | ||
| the first WildFly server will use `messaging-journal-b` directory for their shared journal. | ||
|
|
||
| [NOTE] | ||
| -- | ||
| All ActiveMQ Artemis brokers must be in cluster so they know about each other and can provide cluster topology to connected clients. In case that any WildFly instance crashes, clients know the location of the secondary broker and are able to fail over. | ||
| -- | ||
|
|
||
| * Copy WildFly into two directories `wildfly-1` and `wildfly-2` | ||
| * Start `wildfly-1` in `full-ha` profile in admin only mode and connect through CLI | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| ./wildfly-1/bin/standalone.sh -c standalone-full-ha.xml --admin-only | ||
| ---- | ||
|
|
||
| * In Another terminal connect to CLI: | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| ./wildfly-1/bin/jboss-cli.sh -c | ||
| ---- | ||
|
|
||
| * Run following CLI commands on `wildfly-1`: | ||
|
|
||
| [NOTE] | ||
| -- | ||
| In the following CLI script change `<password>` and `<path>` values based on your environment. | ||
| -- | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| # Change ActiveMQ Artemis cluster <password> | ||
| /subsystem=messaging-activemq/server=default:write-attribute(name=cluster-password, value=<password>) | ||
| # Set location of journal for pair A - change path to messaging-journal-a directory based on your environment | ||
| /subsystem=messaging-activemq/server=default/path=bindings-directory:write-attribute(name=path,value=<path>/messaging-journal-a/bindings) | ||
| /subsystem=messaging-activemq/server=default/path=journal-directory:write-attribute(name=path,value=<path>/messaging-journal-a/journal) | ||
| /subsystem=messaging-activemq/server=default/path=large-messages-directory:write-attribute(name=path,value=<path>/messaging-journal-a/largemessages) | ||
| /subsystem=messaging-activemq/server=default/path=paging-directory:write-attribute(name=path,value=<path>/messaging-journal-a/paging) | ||
| # Set "default" Artemis broker as primary | ||
| /subsystem=messaging-activemq/server=default/ha-policy=shared-store-primary:add(failover-on-server-shutdown=true) | ||
| :reload | ||
| # Add secondary broker | ||
| /subsystem=messaging-activemq/server=secondary:add(security-enabled=true, elytron-domain=ApplicationDomain) | ||
| # Change ActiveMQ Artemis cluster <password> | ||
| /subsystem=messaging-activemq/server=secondary:write-attribute(name=cluster-password, value=password) | ||
| # Create http-connector/http-acceptor pointing to this broker | ||
| /subsystem=messaging-activemq/server=secondary/http-acceptor=acceptor:add(http-listener=default) | ||
| /subsystem=messaging-activemq/server=secondary/http-connector=connector:add(endpoint=acceptor,socket-binding=http) | ||
| # Create a broadcast group to advertise the broker in the cluster and a corresponding discovery group to be used by the cluster connection to discover other brokers, enabling the formation of the cluster. | ||
| /subsystem=messaging-activemq/server=secondary/jgroups-broadcast-group=bg-group1:add(jgroups-cluster=activemq-cluster, connectors=[connector]) | ||
| /subsystem=messaging-activemq/server=secondary/jgroups-discovery-group=dg-group1:add(jgroups-cluster=activemq-cluster) | ||
| /subsystem=messaging-activemq/server=secondary/cluster-connection=my-cluster:add(cluster-connection-address=jms,connector-name=connector,discovery-group=dg-group1) | ||
| # Set "secondary" Artemis broker as secondary | ||
| /subsystem=messaging-activemq/server=secondary/ha-policy=shared-store-secondary:add(failover-on-server-shutdown=true) | ||
| # Set location of journal for pair B - change path to messaging-journal-a directory based on your environment | ||
| /subsystem=messaging-activemq/server=secondary/path=bindings-directory:write-attribute(name=path,value=<path>/messaging-journal-b/bindings) | ||
| /subsystem=messaging-activemq/server=secondary/path=journal-directory:write-attribute(name=path,value=<path>/messaging-journal-b/journal) | ||
| /subsystem=messaging-activemq/server=secondary/path=large-messages-directory:write-attribute(name=path,value=<path>/messaging-journal-b/largemessages) | ||
| /subsystem=messaging-activemq/server=secondary/path=paging-directory:write-attribute(name=path,value=<path>/messaging-journal-b/paging) | ||
| # Shutdown the server | ||
| shutdown | ||
| ---- | ||
|
|
||
| * Start `wildfly-2` in `full-ha` profile in admin only mode and connect through CLI (set port offset to 1000 to avoid port conflicts) | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| ./wildfly-2/bin/standalone.sh -c standalone-full-ha.xml -Djboss.socket.binding.port-offset=1000 --admin-only | ||
| ---- | ||
|
|
||
| * In Another terminal connect to CLI: | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| ./wildfly-2/bin/jboss-cli.sh -c --controller=127.0.0.1:10990 | ||
| ---- | ||
|
|
||
| * Run following CLI commands on `wildfly-2`: | ||
|
|
||
| [NOTE] | ||
| -- | ||
| In the following CLI script change `<password>` and `<path>` values based on your environment. | ||
| -- | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| # Change ActiveMQ Artemis cluster <password> | ||
| /subsystem=messaging-activemq/server=default:write-attribute(name=cluster-password, value=<password>) | ||
| # Set location of journal for pair B - change path to messaging-journal-b directory based on your environment | ||
| /subsystem=messaging-activemq/server=default/path=bindings-directory:write-attribute(name=path,value=<path>/messaging-journal-b/bindings) | ||
| /subsystem=messaging-activemq/server=default/path=journal-directory:write-attribute(name=path,value=<path>/messaging-journal-b/journal) | ||
| /subsystem=messaging-activemq/server=default/path=large-messages-directory:write-attribute(name=path,value=<path>/messaging-journal-b/largemessages) | ||
| /subsystem=messaging-activemq/server=default/path=paging-directory:write-attribute(name=path,value=<path>/messaging-journal-b/paging) | ||
| # Set "default" Artemis broker as primary | ||
| /subsystem=messaging-activemq/server=default/ha-policy=shared-store-primary:add(failover-on-server-shutdown=true) | ||
| :reload | ||
| # Add secondary broker | ||
| /subsystem=messaging-activemq/server=secondary:add(security-enabled=true, elytron-domain=ApplicationDomain) | ||
| # Change ActiveMQ Artemis cluster <password> | ||
| /subsystem=messaging-activemq/server=secondary:write-attribute(name=cluster-password, value=password) | ||
| # Create http-connector/http-acceptor pointing to this broker | ||
| /subsystem=messaging-activemq/server=secondary/http-acceptor=acceptor:add(http-listener=default) | ||
| /subsystem=messaging-activemq/server=secondary/http-connector=connector:add(endpoint=acceptor,socket-binding=http) | ||
| # Create a broadcast group to advertise the broker in the cluster and a corresponding discovery group to be used by the cluster connection to discover other brokers, enabling the formation of the cluster. | ||
| /subsystem=messaging-activemq/server=secondary/jgroups-broadcast-group=bg-group1:add(jgroups-cluster=activemq-cluster, connectors=[connector]) | ||
| /subsystem=messaging-activemq/server=secondary/jgroups-discovery-group=dg-group1:add(jgroups-cluster=activemq-cluster) | ||
| /subsystem=messaging-activemq/server=secondary/cluster-connection=my-cluster:add(cluster-connection-address=jms,connector-name=connector,discovery-group=dg-group1) | ||
| # Set "secondary" Artemis broker as secondary | ||
| /subsystem=messaging-activemq/server=secondary/ha-policy=shared-store-secondary:add(failover-on-server-shutdown=true) | ||
| # Set location of journal for pair A - change path to messaging-journal-a directory based on your environment | ||
| /subsystem=messaging-activemq/server=secondary/path=bindings-directory:write-attribute(name=path,value=<path>/messaging-journal-a/bindings) | ||
| /subsystem=messaging-activemq/server=secondary/path=journal-directory:write-attribute(name=path,value=<path>/messaging-journal-a/journal) | ||
| /subsystem=messaging-activemq/server=secondary/path=large-messages-directory:write-attribute(name=path,value=<path>/messaging-journal-a/largemessages) | ||
| /subsystem=messaging-activemq/server=secondary/path=paging-directory:write-attribute(name=path,value=<path>/messaging-journal-a/paging) | ||
| # Shutdown the server | ||
| shutdown | ||
| ---- | ||
|
|
||
| == Test High Availability | ||
|
|
||
| We’ll test HA by crashing first WildFly server and checking that secondary ActiveMQ Artemis broker on the second WildFly server is active. | ||
|
|
||
| * Start both WildFly servers, each in a separate terminal. | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| ./wildfly-1/bin/standalone.sh -c standalone-full-ha.xml | ||
| ./wildfly-2/bin/standalone.sh -c standalone-full-ha.xml -Djboss.socket.binding.port-offset=1000 | ||
| ---- | ||
|
|
||
| * Shutdown or crash `wildfly-1` and check that secondary broker started in `wildfly-2` in the server log: | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| 16:25:44,921 INFO [org.apache.activemq.artemis.core.server] (AMQ229000: Activation for server ActiveMQServerImpl::name=secondary) AMQ221010: Backup Server is now active | ||
| ---- | ||
|
|
||
| * Start `wildfly-1` server and verify that fail-back happened and secondary broker on `wildfly-2` went to standby mode: | ||
|
|
||
| [source, bash ,options="nowrap"] | ||
| ---- | ||
| 16:26:53,043 INFO [org.apache.activemq.artemis.core.server] (Thread-4 (ActiveMQ-scheduled-threads)) AMQ221008: primary server wants to restart, restarting server in backup | ||
| ... | ||
| 16:26:55,488 INFO [org.apache.activemq.artemis.core.server] (Thread-0 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@2845f281)) AMQ221031: backup announced | ||
| ---- | ||
|
|
||
| == What's next? | ||
|
|
||
| ActiveMQ Artemis in WildFly allows also to use replicated journal where each primary-secondary pair replicates | ||
| data over network. This has its pros and cons as it does not require to mount shared journal directories on | ||
| each machine but usually suffers by lower performance caused by needed network round trip times between paired | ||
| brokers. More information can be found in https://activemq.apache.org/components/artemis/documentation/[ActiveMQ Artemis documentation]. | ||
|
|
||
| [[references]] | ||
| == References | ||
|
|
||
| * https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.4/html-single/configuring_messaging/index#messaging-ha[Configuring Messaging guide for EAP 7.4] | ||
| * https://activemq.apache.org/components/artemis/documentation/[ActiveMQ Artemis documentation] |