OX HE Tutorial 100K: Difference between revisions

From Open-Xchange
(Created page with "= High Available OX HE Deployment for up to 100.000 users = This article describes a typical OX HE Setup for up to 100.000 Users, which is fully clustered and high available. I...")
 
 
(30 intermediate revisions by 7 users not shown)
Line 1: Line 1:
= High Available OX HE Deployment for up to 100.000 users =
= Tutorial: High Available OX HE Setup for up to 100.000 users =


This article describes a typical OX HE Setup for up to 100.000 Users, which is fully clustered and high available.
'''This article describes what you need for a typical OX HE Setup for up to 100.000 Users, which is fully clustered and high available.'''


It contains everything you need to:
It contains everything you need to:
* Understand the design of the OX HE setup including additional services
* Understand the design of the OX HE setup including additional services
* Install the whole system based on the relevant articles
* Install the whole system based on the relevant articles
* Find pointers to the next steps of integration


= System Design =
= System Design =


The following image illustrates the recommended architecture for this setup:
[[Image:SaaS-100k-1.jpg]]
 
The system is designed to provide maximum functionality and availability with a minimum of necessary hardware. If the services on one server fail, it is enough to take over the IP address to the other machine and service will stay up and running.
 
== Core Components for OX HE ==
 
* Two OX HE servers (HW recommendation: 16GB RAM / 4 cores each)
* MySQL installed directly on these server
* NFS Server to store documents and files
 
== Infrastructure Components not delivered by OX ==
 
* An email system providing IMAP and SMTP
* A control panel for creation and administration of users
* A Load Balancer in front of the OX servers (optional, recommended)
 
= Overview Installation Steps =
 
'''To deploy the described OX setup, the following steps need to be done. '''
 
 
== Mandatory Steps ==
# Initialize and configure MySQL database on both servers
# Install and configure OX on both servers
 
== Steps depending on your environment ==
# Implement Load Balancer
# Connect Control Panel
# Connect Email System
 
== Recommended Optional Next Steps ==
# Automated Frontend Tests
# Upsell Plugin
# Mobile Autoconfiguration
# Automatic FailOver
# Branding
 
 
= Mandatory Installation Steps - Instructions & Recommendations =
 
 
'''The following steps need to be done in every case to get OX up and running:'''
 
== Initialize and configure MySQL database on both servers ==
 
MySQL will run on both servers. MySQL will be configured as Master-Master configuration to ensure data consistency on both servers.
If one machine fails, the other machine will take over all functionality.
 
[[OXLoadBalancingClustering_Database|Database setup for clustered environments]]
 
== Install and configure OX on both servers ==


[[Image:SaaS-100k-1.jpg]]
OX will be installed on both servers. It will be configured to '''write''' to the '''first''' MySQL database and to '''read''' from the '''second''' MySQL database. This will distribute the load during normal operation as smooth as possible. During FailOver the IP address of the failed server will be taken over to the working server, the system stays operable.
 
[[OXLoadBalancingClustering_OXConfiguration|Open-Xchange setup and configuration for clustered environments]]
 
The NFS server will be mounted on both machines and registered as filestore.
 
[[OXLoadBalancingClustering_Filestore|Filestore setup for clustered environments]]
 
When multiple Open-Xchange Servers are configured within a cluster Session and Loadbalancing needs to be set up.
 
[[OXLoadBalancingClustering_SessionLoadbalancing|Session and Loadbalancing for clustered environments]]
 
[[OXLoadBalancingClustering_NetworkConfiguration|Network configuration for clustered environments]]
 
You also should install and configure the OXtender for Business Mobility
 
[[OXtender_for_Business_Mobility_Installation_Guide|Installation of the OXtender for Business Mobility]]
 
Let your users connect to their data from other services like Twitter or LinkedIn (removed since 7.10.0) by configuring the "SocialOX"
 
[[SocialOX|SocialOX-Configuration]]
 
= Installation Steps depending on your environment - Instructions & Recommendations =
 
'''The following components need to be implemented in your environment.'''
 
 
== Implement Load Balancer ==
 
A load balancer in front of the OX servers is recommended, but optional in this deployment size. (In small environments, DNS Round Robin may be sufficient).
 
If you already have a hardware load balancing solution in place, this can be used. OX is known to work with the standard load balancing solutions from BigIP, Barracuda, Foundry, ...
 
If you do not have a load balancing solution already in place, we recommend to use [http://www.keepalived.org/ Keepalived] as reliable and cost effective solution.
 
Read more about configuring [[Keepalived | Keepalived for Open-Xchange]]
 
 
{{OX_HE_Tutorial_CP}}
 
 
{{OX_HE_Tutorial_HGP}}
 
 
{{OX_HE_Tutorial_POA}}


= Requirements =


* A working Master-Master database setup (see LINK_TO_DATABASE_SETUP)
{{OX_HE_Tutorial_Auth}}
* Two basic configured Open-Xchange HE installations
* Any number of Fileservers which have been mounted into the local filesystem of the HE installations
* A Load Balancer which will forward the requests to the Open-Xchange HE installations
* Optional: Plesk provisioning




== Connect Email System ==


The hardware recommendations for an Open-Xchange cluster setup which is able to serve up to 100.000 users is:
Every email system providing IMAP and SMTP can be used as backend to OX. Best experiences are made with the widespread Linux based IMAP servers [http://dovecot.org/ Dovecot],  [http://www.cyrusimap.org/ Cyrus] or [http://www.courier-mta.org/imap/ Courier].


* 2 Open-Xchange HE installations (Hardware requirements: 16GB RAM / 4 cores each)
Other IMAP servers need to be tested thoroughly before going into production.
* 1 Master-Master configured database cluster (Hardware requirements: 16GB RAM / 4 cores each)
* 1 Load balancer (Software or Hardware)
* X NFS servers (Filestores), depends on the amount of storage which is required


= Database setup and configuration =
There are several possibilities to implement the Email system:


In this section you will find information of how to create the MySQL Master-Master replication setup that will be connected to the Open-Xchange HE setup during the next step. A detailed description of how this can be done is described here:
# You already have an email system available: Nothing needs to be done, it just needs to be configured
# You use Parallels Automation (POA): Nothing special needs to be done, everything you need is contained in the APS package
# You want to setup a new Email system: It is recommended to use Dovecot, as this is very stable, fast, feature rich and easy to scale


[[Load_balancing_and_clustering#Master.2FSlave_database_setup|MySQL Master-Master replication setup]]


= Basic Open-Xchange HE setup and configuration =
{{OX_HE_Tutorial_Dovecot}}


To install the Open-Xchange Hosting Edition that is required for this setup please follow the Quick installation guides for the according OS:


* [[Open-Xchange_Installation_Guide_for_Debian_5.0|Download and Installation Guide for Debian GNU/Linux 5.0 (Lenny)]]
{{OX_HE_Tutorial_Next}}
* [[Open-Xchange_Installation_Guide_for_SLES10|Download and Installation Guide for SUSE Linux Enterprise Server 10]]
* [[Open-Xchange_Installation_Guide_for_SLES11|Download and Installation Guide for SUSE Linux Enterprise Server 11]]
* [[Open-Xchange_Installation_Guide_for_RHEL5|Download and Installation Guide for RedHat Enterprise Linux 5]]
* [[Installing_OX_Language_Packages|Installing Open-Xchange Language Packages]]
* [http://software.open-xchange.com/OX6/stable/ Software repository]


IMPORTANT: During the installation process you will need to create the Config Database which holds all necessary Database cluster information, but as this setup provides a dedicated MySQL Master-Master database. Simply skip the following step during the setup:


* Open-Xchange configuration (usually step 5)
{{OX_HE_Tutorial_FailOver}}

Latest revision as of 08:16, 8 February 2018

Tutorial: High Available OX HE Setup for up to 100.000 users

This article describes what you need for a typical OX HE Setup for up to 100.000 Users, which is fully clustered and high available.

It contains everything you need to:

  • Understand the design of the OX HE setup including additional services
  • Install the whole system based on the relevant articles
  • Find pointers to the next steps of integration


System Design

SaaS-100k-1.jpg

The system is designed to provide maximum functionality and availability with a minimum of necessary hardware. If the services on one server fail, it is enough to take over the IP address to the other machine and service will stay up and running.

Core Components for OX HE

  • Two OX HE servers (HW recommendation: 16GB RAM / 4 cores each)
  • MySQL installed directly on these server
  • NFS Server to store documents and files

Infrastructure Components not delivered by OX

  • An email system providing IMAP and SMTP
  • A control panel for creation and administration of users
  • A Load Balancer in front of the OX servers (optional, recommended)

Overview Installation Steps

To deploy the described OX setup, the following steps need to be done.


Mandatory Steps

  1. Initialize and configure MySQL database on both servers
  2. Install and configure OX on both servers

Steps depending on your environment

  1. Implement Load Balancer
  2. Connect Control Panel
  3. Connect Email System

Recommended Optional Next Steps

  1. Automated Frontend Tests
  2. Upsell Plugin
  3. Mobile Autoconfiguration
  4. Automatic FailOver
  5. Branding


Mandatory Installation Steps - Instructions & Recommendations

The following steps need to be done in every case to get OX up and running:

Initialize and configure MySQL database on both servers

MySQL will run on both servers. MySQL will be configured as Master-Master configuration to ensure data consistency on both servers. If one machine fails, the other machine will take over all functionality.

Database setup for clustered environments

Install and configure OX on both servers

OX will be installed on both servers. It will be configured to write to the first MySQL database and to read from the second MySQL database. This will distribute the load during normal operation as smooth as possible. During FailOver the IP address of the failed server will be taken over to the working server, the system stays operable.

Open-Xchange setup and configuration for clustered environments

The NFS server will be mounted on both machines and registered as filestore.

Filestore setup for clustered environments

When multiple Open-Xchange Servers are configured within a cluster Session and Loadbalancing needs to be set up.

Session and Loadbalancing for clustered environments

Network configuration for clustered environments

You also should install and configure the OXtender for Business Mobility

Installation of the OXtender for Business Mobility

Let your users connect to their data from other services like Twitter or LinkedIn (removed since 7.10.0) by configuring the "SocialOX"

SocialOX-Configuration

Installation Steps depending on your environment - Instructions & Recommendations

The following components need to be implemented in your environment.


Implement Load Balancer

A load balancer in front of the OX servers is recommended, but optional in this deployment size. (In small environments, DNS Round Robin may be sufficient).

If you already have a hardware load balancing solution in place, this can be used. OX is known to work with the standard load balancing solutions from BigIP, Barracuda, Foundry, ...

If you do not have a load balancing solution already in place, we recommend to use Keepalived as reliable and cost effective solution.

Read more about configuring Keepalived for Open-Xchange


Connect Control Panel

You need a Control Panel to create and edit users.

OX is designed to integrate into every solution you may already run in your environment and also into wide spread solutions, like the Parallels Control Panels.

If you do not run hosting services today and do not have a Control Panel in place, it is recommend to use Plesk to manage OX. With that combination you will get a full functional hosting platform containing everything you need.


Integrate your own Control Panel

If you already have a Control Panel in production, you should integrate OX with it. It is recommended to use the SOAP provisioning Interface for that purpose.

Read more about: Provisioning using SOAP

A good start to test and to understand the necessary commands are the Command Line Tools. They have exactly the same calls like the SOAP API.

Read more about: Open-Xchange CLT


Integrate with Parallels Automation (POA)

Parallels Operations Automation (POA) is an operations support system (OSS) for service providers, who want to differentiate their offerings in order to reduce customer churn and attract new customers. Additional, the APS package adds a high performance, best in class email service to Parallels Plesk Panel customers.


Authentication

To avoid password synchronization issues, it is recommended to use your existing email authentication mechanism within OX. Then you do not need to add user passwords to OX, you simply use a plugin to authenticate against your IMAP server.

Read more about the IMAP Authentication Plugin


Connect Email System

Every email system providing IMAP and SMTP can be used as backend to OX. Best experiences are made with the widespread Linux based IMAP servers Dovecot, Cyrus or Courier.

Other IMAP servers need to be tested thoroughly before going into production.

There are several possibilities to implement the Email system:

  1. You already have an email system available: Nothing needs to be done, it just needs to be configured
  2. You use Parallels Automation (POA): Nothing special needs to be done, everything you need is contained in the APS package
  3. You want to setup a new Email system: It is recommended to use Dovecot, as this is very stable, fast, feature rich and easy to scale


Dovecot Setup

If you want to setup a new Email system based on Dovecot, it is recommended to use NFS as storage backend and to install at least two Dovecot servers, accessing this storage. With that setup you have best scalability and high availability with a minimum of complexity and hardware.

Read more in the Dovecot documentation including a QuickConfiguration guide


Recommended Optional Next Steps

You will find plenty of additional documentation for customization of OX in our knowledge base [1]

When the main setup is completed, we recommend to start with the following articles to enhance your system and to become more attractive for your users.


Automated Frontend Tests

It is a good idea, to verify the functionality of your freshly set up and integrated system. Our QA department does that with tests, running automatically on the web frontend. We release this tests with every release and recommend you to use them to verify your environment with every update.

Read more about Automated_GUI_Tests


Monitoring / Statistics

It is recommended to implement at least a minimal monitoring/Statistics solution to get an overview of the systems health. If you have a support contract with Open-Xchange, it is very helpful, if the support can access the monitoring graphs. There are example scripts for a basic monitoring with [Munin] available.

Read more about installing and configuring Munin scripts for Open-Xchange


Upsell Plugin / Webmail Replacement

If you want to use your OX based Webmail system to upsell premium functions like full groupware functionality or like push to mobile phones, it is strongly recommended to use the in-app sales process.

Read more about Upsell


Branding

If you want OX to look more like your own Corporate Identity, including your logo, product name and maybe your colors, this can be easily achieved by changing the logos and stylesheets.

Read more about: Gui_Theming_Description

Read more about: Gui Branding Plugins

Read more about: Branding via the ConfigCascade

Backup

It is recommended to run regular backups for your OX installation. This can be done with every backup solution for Linux.

Read more about Backup your Open-Xchange installation


IP FailOver

For automatic IP failover, several solutions are available for Linux. A reliable working framework is:

Heartbeat