OX6:RHEL5 Cluster - Revision history

Choeger: Choeger moved page RHEL5 Cluster to OX6:RHEL5 Cluster

2015-11-27T09:34:41Z

Choeger moved page RHEL5 Cluster to OX6:RHEL5 Cluster

← Older revision

Revision as of 09:34, 27 November 2015

Holgi: /* Configure LVS */

2011-02-24T10:25:39Z

Configure LVS

← Older revision		Revision as of 10:25, 24 February 2011
Line 308:		Line 308:

	* monitoring script for the Open-Xchange Server <tt>ox-mon.sh</tt> :		* monitoring script for the Open-Xchange Server <tt>ox-mon.sh</tt> :
	~~tbd~~
			#!/bin/sh

			/usr/bin/curl $1/servlet/TestServlet 2>&1 \| grep TestServlet 2>&1
			1>/dev/null
			if test $? -gt 0
			then
			echo "FALSE"
			else
			echo "OK"
			fi

	=== Remarks ===		=== Remarks ===

Choeger: moved Rhel cluster to RHEL5 Cluster

2009-10-21T10:10:23Z

moved Rhel cluster to RHEL5 Cluster

← Older revision

Revision as of 10:10, 21 October 2009

Holgi: /* Installation */

2009-10-19T11:18:34Z

Installation

← Older revision		Revision as of 11:18, 19 October 2009
Line 340:		Line 340:
	On both systems a RHEL 5.3 operating system has to be installed and on top of the RHEL cluster suite. Because the Conga web interface is used to ease the configuration tasks the packages for the client-server system of Conga: ricci and luci need to be available on both systems.		On both systems a RHEL 5.3 operating system has to be installed and on top of the RHEL cluster suite. Because the Conga web interface is used to ease the configuration tasks the packages for the client-server system of Conga: ricci and luci need to be available on both systems.

	* All packet filters are disabled to avoid impacts on the cluster: ~~<tt>~~system-config-securitylevel disable; service iptables stop; chkconfig –del iptables. ~~</tt>~~Check/edit the file /<tt>etc/sysconfig/system-config-securitylevel</tt>, the service must be set to ~~“disabled”~~.		* All packet filters are disabled to avoid impacts on the cluster:
	* Also selinux is disabled resp. set to permissive mode: ~~<tt>~~system-config-securitylevel disable~~. </tt>~~Check/edit the file <tt>/etc/selinux/</tt>~~config~~, the service must be set either to disabled or to permissive<~~ref name="ftn3"~~>If selinux was set to disabled and should be enabled again: set to permissive, boot, touch /.autorelabel; boot; set to enabled</~~ref~~>.		system-config-securitylevel disable
			service iptables stop
			chkconfig –del iptables.
			Check/edit the file <tt>/etc/sysconfig/system-config-securitylevel</tt>, the service must be set to <tt>disabled</tt>.
			* Also selinux is disabled resp. set to permissive mode:
			system-config-securitylevel disable
			Check/edit the file <tt>/etc/selinux/config</tt>, the service must be set either to <tt>disabled</tt> or to <tt>permissive</tt>. If selinux was set to <tt>disabled</tt> and should be enabled again: set to <tt>permissive</tt>, boot, <tt>touch /.autorelabel</tt>, boot and set to <tt>enabled</tt>.
	* IP forwarding is enabled in <tt>/etc/sysctl.conf</tt>, set <tt>net.ip4.ip_forward=1</tt>		* IP forwarding is enabled in <tt>/etc/sysctl.conf</tt>, set <tt>net.ip4.ip_forward=1</tt>
	* Network zeroconf is disabled in <tt>/etc/sysconfig/network;</tt> a line is added <tt>NOZEROCONF=yes</tt>		* Network zeroconf is disabled in <tt>/etc/sysconfig/network</tt> a line is added <tt>NOZEROCONF=yes</tt>
	* On both systems ACPI/the acpid is disabled to allow for immediate shutdown via fence device: ~~<tt>~~chkconfig --del acpid~~</tt>~~		* On both systems ACPI/the acpid is disabled to allow for immediate shutdown via fence device:
	* To configure both systems from conga the ricci serivce is started on both system:~~<tt>~~ chkconfig --level 2345 ricci on; service ricci start~~</tt>~~		chkconfig --del acpid
	* On the system node8888 also the luci service is enabled:~~<tt>~~ chkconfig --level 2345 luci on~~</tt>~~		* To configure both systems from conga the ricci serivce is started on both system:
			chkconfig --level 2345
			ricci on
			service ricci start
			* On the system node8888 also the luci service is enabled:
			chkconfig --level 2345 luci on

	=== Configuration ===		=== Configuration ===

Holgi: /* Abstract */

2009-10-19T11:11:31Z

Abstract

← Older revision		Revision as of 11:11, 19 October 2009
Line 9:		Line 9:
	LVS is running in an active-passive configuration on two nodes. A “reduced” three-tier topology is used: the “real servers” are provided as virtual services by another two node RHEL cluster. The OX service will benefit from load balancing because it is running in active-active mode as two virtual services on the RHEL cluster and it is ensured that each virtual OX will run on one of the physical nodes of the cluster. The Mysql and Mail services are configured to each run on a single node and will failover in case of a node outage. LVS is using NAT as routing method and the round robin algorithm for scheduling. See the sketch for IPs and general configuration.		LVS is running in an active-passive configuration on two nodes. A “reduced” three-tier topology is used: the “real servers” are provided as virtual services by another two node RHEL cluster. The OX service will benefit from load balancing because it is running in active-active mode as two virtual services on the RHEL cluster and it is ensured that each virtual OX will run on one of the physical nodes of the cluster. The Mysql and Mail services are configured to each run on a single node and will failover in case of a node outage. LVS is using NAT as routing method and the round robin algorithm for scheduling. See the sketch for IPs and general configuration.

	* LVS is configured on two nodes: <tt>node1111/192.168.109.105</tt> and ~~'''~~node2222/192.168.109.106~~'''~~		* LVS is configured on two nodes: <tt>node1111/192.168.109.105</tt> and <tt>node2222/192.168.109.106</tt>
	* A single “external” IP is configured for all services: ~~'''~~192.168.109.117~~'''.~~		* A single “external” IP is configured for all services: <tt>192.168.109.117</tt>
	* The routing IP (used as gateway from the “internal” part of the setup) is configured as ~~'''~~192.168.109.116~~'''~~.		* The routing IP (used as gateway from the “internal” part of the setup) is configured as <tt>192.168.109.116</tt>.
	* The LVS heartbeat network is using the private network ~~'''~~10.10.10.0/24~~'''~~.		* The LVS heartbeat network is using the private network <tt>10.10.10.0/24</tt>.
	* Network devices are bundled together as bond devices to gain higher redundancy.		* Network devices are bundled together as bond devices to gain higher redundancy.
	* Firewall marks are used to get session persistence.		* Firewall marks are used to get session persistence.

Holgi: /* Abstract */

2009-10-19T11:10:26Z

Abstract

← Older revision		Revision as of 11:10, 19 October 2009
Line 323:		Line 323:
	=== Abstract ===		=== Abstract ===

	The RHEL HA Cluster is running on the two nodes node8888/192.168.109.102 and node9999/192.168.109.103. A “reduced” three-tier topology is used: in front of the cluster a LVS load balancer is routing the traffic to the services on the cluster. See sketch for the setup, IPs etc. Dell DRAC management hardware is used for fencing.		The RHEL HA Cluster is running on the two nodes <tt>node8888/192.168.109.102</tt> and <tt>node9999/192.168.109.103</tt>. A “reduced” three-tier topology is used: in front of the cluster a LVS load balancer is routing the traffic to the services on the cluster. See sketch for the setup, IPs etc. Dell DRAC management hardware is used for fencing.

	[[File:Illustration7.png]]		[[File:Illustration7.png]]
Line 329:		Line 329:


	* RHEL HA Cluster is running on node8888/192.168.109.102 and node9999/192.168.109.103		* RHEL HA Cluster is running on <tt>node8888/192.168.109.102</tt> and <tt>node9999/192.168.109.103</tt>
	* Virtual Services are defined for Ox (active-active), IMAP+SMTP and Mysql. The services are build from globally defined cluster resources (e.g. IPs, filesystems etc.).		* Virtual Services are defined for Ox (active-active), IMAP+SMTP and Mysql. The services are build from globally defined cluster resources (e.g. IPs, filesystems etc.).
	* Shared Storage is configured with LVM and three logical volumes are defined as resources for Ox (GFS2), Mysql (EXT3) and IMAP/SMTP (EXT3)		* Shared Storage is configured with LVM and three logical volumes are defined as resources for Ox (GFS2), Mysql (EXT3) and IMAP/SMTP (EXT3)
	* Virtual IPs (seen as Real IPs from LVS) are defined for OX1/2 192.168.109.114 and 192.168.109.15, Mysql 192.168.109.112 and 192.168.109.118 and IMAP/SMTP 192.168.109.113		* Virtual IPs (seen as Real IPs from LVS) are defined for <tt>OX1/2 192.168.109.114</tt> and <tt>192.168.109.15</tt>, Mysql <tt>192.168.109.112</tt> and <tt>192.168.109.118</tt> and IMAP/SMTP <tt>192.168.109.113</tt>
	* Dell DRAC devices host fence devices are configured (192.168.105.234/192.168.105.235)		* Dell DRAC devices host fence devices are configured (<tt>192.168.105.234/192.168.105.235</tt>)
	* The Conga web configuration interface is used to configure the cluster		* The Conga web configuration interface is used to configure the cluster

Holgi: /* Configure LVS */

2009-10-19T11:07:57Z

Configure LVS

← Older revision		Revision as of 11:07, 19 October 2009
Line 133:		Line 133:

	* A password for the piranha-gui needs to be set:		* A password for the piranha-gui needs to be set:
	~~iranha~~-passwd		piranha-passwd
	* The piranha-gui is started:		* The piranha-gui is started:
	service piranha-gui start		service piranha-gui start
	The gui can be accessed ~~with~~ the URL:<tt>http://192.168.109.105:3636/</tt>. It should be considered to limit access to the piranha GUI.		The gui can be accessed at the URL: <tt>http://192.168.109.105:3636/</tt>. It should be considered to limit access to the piranha GUI.

	The following screenshots are showing configuration examples:		The following screenshots are showing configuration examples:
Line 306:		Line 306:
	}		}
	}		}

			* monitoring script for the Open-Xchange Server <tt>ox-mon.sh</tt> :
			tbd

	=== Remarks ===		=== Remarks ===

Holgi: /* Configure LVS */

2009-10-19T11:03:01Z

Configure LVS

← Older revision		Revision as of 11:03, 19 October 2009
Line 132:		Line 132:
	To ease the configuration of LVS the RHEL piranha-gui is used.		To ease the configuration of LVS the RHEL piranha-gui is used.

	* A password for the piranha-gui needs to be set ~~with the command p<tt>~~iranha-passwd~~</tt>~~		* A password for the piranha-gui needs to be set:
	* The piranha-gui is started: ~~<tt>~~service piranha-gui start~~</tt><br/> To access~~ the ~~gui open a browser and point it to~~ <tt>http://192.168.109.105:3636/<~~ref name="ftn2"><~~tt>It should be considered to limit access to the piranha GUI.~~</tt></ref>~~		iranha-passwd
			* The piranha-gui is started:
			service piranha-gui start
			The gui can be accessed with the URL:<tt>http://192.168.109.105:3636/</tt>. It should be considered to limit access to the piranha GUI.

	The following screenshots are showing ~~some~~ configuration examples:		The following screenshots are showing configuration examples:

	[[File:Illustration2.png\|center\|750px\|]]		[[File:Illustration2.png\|center\|750px\|]]
Line 156:		Line 159:

	After the configuration has been finished the configuration must be synchronized to the second server. To accomplish this task the following files are copied via scp to node2222:		After the configuration has been finished the configuration must be synchronized to the second server. To accomplish this task the following files are copied via scp to node2222:
			/etc/sysconfig/ha/lvs.cf
			/etc/sysctl.conf
			/etc/sysconfig/iptables

	* /etc/sysconfig/ha/lvs.cf		Firewall/packet filtering, ip forwarding and the pulse service must be enabled by rebooting the system.
	* /etc/sysctl.conf
	* /etc/sysconfig/iptables

	Firewall/packet filtering, ip forwarding and the pulse service must be enabled ~~(e.g. reboot~~ the system).


	The LVS configuration on the systems as example:		The LVS configuration on the systems as example:


	serial_no = 213		serial_no = 213

Holgi: /* Configuration */

2009-10-19T10:10:17Z

Configuration

← Older revision		Revision as of 10:10, 19 October 2009
Line 43:		Line 43:
	'''Services'''		'''Services'''

	* On both systems the sshd and pulse services are enabled: ~~<tt>~~chkconfig –level 35 sshd\|pulse on~~</tt>~~		* On both systems the sshd and pulse services are enabled:
			chkconfig –level 35 sshd\|pulse on

	* On the piranha system (192.168.109.105/node1111) the piranha web service is enabled: ~~<tt>~~chkconfig –level 35 piranha-gui on~~</tt>~~		* On the piranha system (192.168.109.105/node1111) the piranha web service is enabled:
			chkconfig –level 35 piranha-gui on

	* Also the apache web server is enabled and started: ~~<tt>~~chkconfig –add httpd; service httpd start~~</tt>~~		* Also the apache web server is enabled and started:
			chkconfig –add httpd
			service httpd start

	* The access to the apache server is allowed to anybody via /etc/sysconfig/ha/web/secure/.htaccess. This can be changed and adapted to the needs/policies ~~of OpenXchange~~.		* The access to the apache server is allowed to anybody via <tt>/etc/sysconfig/ha/web/secure/.htaccess</tt>. This can be changed and adapted to the needs/policies in place.

	==== Network ====		==== Network ====

	A single network 192.168.109.0/24 is used for all real and virtual IPs and the private network 10.10.10.0/24 for the LVS internal communication. Via bonding of two physical devices the network devices bond0/1 need to be configured for both networks during the base installation.		A single network <tt>192.168.109.0/24</tt> is used for all real and virtual IPs and the private network <tt>10.10.10.0/24</tt> for the LVS internal communication. Via bonding of two physical devices the network devices bond0/1 need to be configured for both networks during the base installation.

	On node1111 the bond devices are configured as:		On node1111 the bond devices are configured as:
Line 101:		Line 105:
	Firewall marks together with a timing parameter are used by LVS to make considerations about packages from the same source and the same destination but different ports. Marked packages are assumed to belong to the same sessions if they bear the same mark and appear during a configurable time frame. The following marking rules are configured:		Firewall marks together with a timing parameter are used by LVS to make considerations about packages from the same source and the same destination but different ports. Marked packages are assumed to belong to the same sessions if they bear the same mark and appear during a configurable time frame. The following marking rules are configured:

	* Mark sessions to port 80/http, 443/https and 44335/oxtender with “80”:		Mark sessions to port 80/http, 443/https and 44335/oxtender with “80”:
	* iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 80 -j MARK --set-mark 80		iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 80 -j MARK --set-mark 80
	* iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 –dport 443 -j MARK --set-mark 80		iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 –dport 443 -j MARK --set-mark 80
	* iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 44335 -j MARK --set-mark 80		iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 44335 -j MARK --set-mark 80
	* Mark sessions to port 143/imap and port 993/imaps with “143”:		Mark sessions to port 143/imap and port 993/imaps with “143”:
	* iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 143 -j MARK --set-mark 143		iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 143 -j MARK --set-mark 143
	* iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 993 -j MARK --set-mark 143		iptables -t mangle -A PREROUTING -p tcp -d 192.168.109.117/32 --dport 993 -j MARK --set-mark 143
	* The rules are saved and the enabled via: ~~<tt>~~service iptables save; chkconfig –add iptables~~</tt>~~		The rules are saved and the enabled via:
			service iptables save
			chkconfig –add iptables

	After the configuration the file<tt>/etc/sysconfig/iptables </tt>should contain a paragraph with the following rules:		After the configuration the file<tt>/etc/sysconfig/iptables </tt>should contain a paragraph with the following rules:

Holgi at 10:06, 19 October 2009

2009-10-19T10:06:15Z

← Older revision		Revision as of 10:06, 19 October 2009
Line 15:		Line 15:
	* Network devices are bundled together as bond devices to gain higher redundancy.		* Network devices are bundled together as bond devices to gain higher redundancy.
	* Firewall marks are used to get session persistence.		* Firewall marks are used to get session persistence.

			[[File:Illustration1.png\|center\|600px]]

			'''''Illustration 1: Setup of LVS and HA cluster'''''

	=== Installation ===		=== Installation ===
Line 34:		Line 38:
	* IP forwarding is enabled in <tt>/etc/sysctl.conf</tt>, set <tt>net.ip4.ip_forward=1</tt>		* IP forwarding is enabled in <tt>/etc/sysctl.conf</tt>, set <tt>net.ip4.ip_forward=1</tt>
	* Network zeroconf is disabled in <tt>/etc/sysconfig/network</tt> a line is added <tt>NOZEROCONF=yes</tt>		* Network zeroconf is disabled in <tt>/etc/sysconfig/network</tt> a line is added <tt>NOZEROCONF=yes</tt>

	~~[[File:Illustration1.png\|center\|600px]]~~

	~~'''''Illustration 1: Setup of LVS and HA cluster with all IPs '''''~~

	=== Configuration ===		=== Configuration ===