AppSuite:OX Guard Upgrade OSGI

Upgrading OX Guard from 2.2.x to 2.4.0

Introduction

With OX Guard 2.4.0 the first step was made and the product was moved into the OSGi stack. Although OX Guard now has a tighter integration with the core that does not mean that it lost its micro-service character. OX Guard still relies on the support API to accomplish its tasks; tasks like capability check, HTML sanitising and mail address resolving. The database calls however were completely eliminated (along with their support API) and now the core's OSGi services are used for that purpose. The same applies to the configuration.

In the next few sections it is described what was changed and the necessary steps to upgrade from 2.2.x to 2.4.0. For a fresh installation refer to the main OX Guard article.

Changes

This section covers the changes introduced with OX Guard 2.4.0.

Configuration Structure

Alongside with the OSGi-fication of the product, a bit of clean up took place, regarding the configuration files and their structure.

The most obvious change was to get rid of the separate /opt/open-xchange/guard directory that the entire configuration was residing in. The contents of this directory are now spread accross the appropriate folders in /opt/open-xchange.

Templates

The templates (originally located under /opt/open-xchange/guard/templates) were moved to the templates directory of the open-xchange installation, i.e. to /opt/open-xchange/templates/guard.

Translations

The translations (originally located under /opt/open-xchange/guard/l10n) were moved to the i18n directory of the open-xchange installation, i.e. to /opt/open-xchange/i18n. The translations formerly had a prefix main-*_*.po (i.e. main-de_DE.po) should now start with guard- (i.e. guard-de_DE.po) to be able to differentiate between middleware translations and those provided by OX Guard. Even the languages.xml and those starting with templates-* should be located within the above mentioned i18n directory.

Properties

guard.properties

Prior to 2.4.0, there were two guard.properties files, one for the core groupware which contained properties that were used to enable the OX Guard capability to various modules as well as some extra functionality, and one for OX Guard itself, containing all properties for that product. Since the /opt/open-xchange/guard directory was deemed obsolete and subject to be removed and to avoid the name clashing of the two property files, they had to be renamed. A merge was not option, simply because they were provided and used by different packages. Therefore, the properties file guard.properties (under /opt/open-xchange/etc/) which accomodated the properties for the AppSuite Guard bundle is now renamed to guard-api.properties.

The OX Guard guard.properties file (under /opt/open-xchange/guard/etc) which housed all the OX Guard properties, is now renamed and moved under /opt/open-xchange/guard/etc/guard-core.properties.

private_dns_key and public_dns_key

Those two property files were not needed anymore, thus they were marked as obsolete and removed with OX Guard 2.4.0.

Farewell Jetty, welcome Grizzly

With the big step OX Guard took and moved into the OSGi stack, Jetty was rendered obsolete and had to be removed as well. OX Guard now uses Grizzly to register its servlets.

Apache

The first change in the Apache configuration is the port number of the balancer directive regarding OX Guard. Since Jetty was removed and Grizzly is in charge now, the port was adjusted to reflect that change.

The second change is the order of the ProxyPass locations. The OX Guard one, had to be moved forward because it was getting masked and overwritten by the AppSuite ones.

Guest Reader moved to a separate package

In order to allow more flexibility on distributed deployments, the "Guest Reader" component has moved to a separate package that is being installed on the frontend server. The UI is then provided in a similar way to App Suite UI.

Command Line Tool

The command line tool also got a "lifting" and was brought to the standards of the rest of the command line tools of the OX family. The CLT now includes optional JMX authentication. Furthermore, the functionality of initialising the OX Guard installation was removed from the command line tool. It is now being done automatically upon server start up. The rest of the functionality is still there, though.

usage: guard [-i [-d /custom/path]] [-t john.doe@example.com] [-u
             joan.doe@ox.io] [-e john.doe@example.com] [-n
             joan.doe@example.com] [-j jmxUser] [-w jmxPassword] [-x
             jmx.host] [-o <JMX_PORT>] [-p <RMI_PORT>] [--responsetimeout
             <TIMEOUT>]
 -d,--directory <arg>         Full path of the 'oxguardpass' file
                              (defaults to current directory). This flag
                              should be used in conjunction with the
                              '--init' switch.
                              e.g. guard --init --directory
                              /opt/open-xchange/etc
 -e,--reset <arg>             Resets the specified e-mail address and
                              sends a new password to the user
                              e.g. guard --reset john@somewhere.com
 -h,--help                    Prints a help text
 -i,--init                    Initialise guard.
 -j,--jmx-user <arg>          JMX user
 -n,--remove-pin <arg>        Removes the PIN for the specified user
                              e.g. guard --remove-pin john@somewhere.com
 -o,--jmx-port <arg>          JMX port (default: '9999')
 -p,--port <arg>              The optional RMI port (default:1099)
    --responsetimeout <arg>   The optional response timeout in seconds
                              when reading data from server (default: 0s;
                              infinite)
 -s,--server <arg>            The optional RMI server (default: localhost)
 -t,--test <arg>              Test the specified e-mail address against
                              the MailResolver
                              e.g. guard --test john@somewhere.com
 -u,--upgrade <arg>           Upgrades the specified guest account to an
                              OX account
                              e.g. guard --upgrade john@somewhere.com
 -w,--jmx-password <arg>      JMX password
 -x,--jmx-host <arg>          JMX host (default: 'localhost')

Command line tool for OX Guard

Logging

The /opt/open-xchange/guard/etc/logback.xml file for OX Guard was completely removed. Now logging is being configured globally in /opt/open-xchange/etc/logback.xml. All OX Guard events that were previously logged in /var/log/open-xchange/guard/guard.log are now being logged in the log file as the groupware, namely in /var/log/open-xchange/open-xchange.log.0.

The SMTP relevant section is no longer present, but it can be configured separately by adding it again to the previous mentioned logback.xml file.

<!-- Configure SMTP Triggered emails here.  Once done, uncomment out appender under root below -->
  <appender name="SMTP" class="ch.qos.logback.classic.net.SMTPAppender">
    <smtpHost>smtphost</smtpHost>
    <to>user@admin</to>
    <from>notify@admin</from>
    <subject>Guard - %marker - %msg%n </subject>
    <layout class="ch.qos.logback.classic.html.HTMLLayout">
      <pattern>%date%level%logger{24}%msg</pattern>
    </layout>

    <evaluator class="ch.qos.logback.classic.boolex.JaninoEventEvaluator">
      <expression>
        marker != null && (marker.contains("NOTIFY_MAJOR") || marker.contains("NOTIFY_MINOR"))
      </expression>
    </evaluator>
  </appender>

The following tag has to be added to the root level in order for this to work.

<appender-ref ref="SMTP" />

For example:

<root level="INFO">
    <appender-ref ref="ASYNC"/>
    <appender-ref ref="SMTP" />
</root>

Upgrading from 2.2.x to 2.4.0

There are a few things to consider when upgrading from a version 2.2.x to 2.4.0. In this section we will cover that upgrade path.

The first thing is that the file storage implementations are now being kept into two separate packages, i.e. open-xchange-guard-file-storage and open-xchange-guard-s3-storage for the file storage and S3 storage support respectively. This means that one of the packages has to be manually installed after the upgrade.

The second thing is that the open-xchange-guard system service is now gone. Since, OX Guard is intergraded into the OSGi stack, it will start automatically with the backend, i.e. when the open-xchange system service starts.

With those two things in mind, OX Guard can be upgraded as described here. Again, after the upgrade is complete there is no need to restart the open-xchange-guard system service; neither the open-xchange system service should be restarted just yet. There are a few things to do before having a fully functional OX Guard backend.

The Guest Reader component has moved to the open-xchange-guard-reader package, which needs to be installed on the frontend server next to App Suite UI.

Step 1 - Enhance repository sources

If you currently do not have an Open-Xchange middleware installed you have to adapt your Open-Xchange repository file to be able to solve required dependencies to the middleware packages. Add the following to the file that already should contain an entry for the OX Guard repository.

Debian 7.x

deb https://software.open-xchange.com/products/appsuite/stable/backend/DebianWheezy /

Debian 8.x

deb https://software.open-xchange.com/products/appsuite/stable/backend/DebianJessie /

RedHat Enterprise Linux 6 or CentOS 6

[ox-backend]
name=Open-Xchange-backend
baseurl=https://software.open-xchange.com/products/appsuite/stable/backend/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

Redhat Enterprise Linux 7 or CentOS 7

[ox-backend]
name=Open-Xchange-backend
baseurl=https://software.open-xchange.com/products/appsuite/stable/backend/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

SUSE Linux Enterprise Server 11.x

$ zypper ar https://software.open-xchange.com/products/appsuite/stable/backend/SLES11 ox-backend

SUSE Linux Enterprise Server 12.x

$ zypper ar https://software.open-xchange.com/products/appsuite/stable/backend/SLE_12 ox-backend

After adding new repositories retrieve new lists of packages by executing your desired package managers update call.

Step 2 - Update OX Guard

Upgrade the existing installation by executing your desired package mangers upgrade call, for instance apt-get dist-upgrade, yum upgrade and so on.

If you are running an OX Guard-only node (because of infelicitous set dependencies) you have to install an implementation of the package open-xchange-authentication and open-xchange-spamhandler which will never be used but checked for. We suggest to install open-xchange-authentication-database and open-xchange-spamhandler-default. Installing those packages as well as chosing a open-xchange-guard-storage implementation will install all required middleware dependencies.

After the update /etc/init.d/open-xchange-guard shouldn't be available any more but OX Guard will start when starting the open-xchange system service.

Step 3 - Choose and install your storage

According to your previous installation, you will have to manually install the relevant open-xchange-guard-storage package. Of course, you can switch to the one or the other storage if you wish, but be aware that no data will be migrated. You will have to do this manually.

File Storage Support

If you want the file storage support, and depending on your Linux distribution, you can install it with the following command:

Debian 7.x/8.x

apt-get install open-xchange-guard-file-storage

RedHat Enterprise Linux 6.x/7.x or CentOS 6.x/7.x

yum install open-xchange-guard-file-storage

SUSE Linux Enterprise Server 11.x/12.x

zypper in open-xchange-guard-file-storage

S3 Storage Support

If you want the S3 storage support, and depending on your Linux distribution, you can install it with the following command:

Debian 7.x/8.x

apt-get install open-xchange-guard-s3-storage

RedHat Enterprise Linux 6.x/7.x or CentOS 6.x/7.x

yum install open-xchange-guard-s3-storage

SUSE Linux Enterprise Server 11.x/12.x

zypper in open-xchange-guard-s3-storage

Step 4 - Install open-xchange-rest

OX Guard requires access to existing middleware hosts. The endpoint is configured within the guard-core.properties which you can find in /opt/open-xchange/etc. The endpoint configured via com.openexchange.guard.restApiHostname need to have the package open-xchange-rest installed to be able to retrieve and process OX Guard REST requests.

Normally the post installation scripts from the open-xchange-guard package should take over the formerly configured endpoint so that you do not have to do anything.

Step 5 - Ensure that the server keys are readable by the sever

The oxguardpass file should be owned by the open-xchange user and have the permission 400, meaning read access only by the owner of the file.

ls -l /opt/open-xchange/etc/oxguardpass
-r-------- 1 open-xchange root 64 Nov 17 19:14 /opt/open-xchange/etc/oxguardpass

Step 6 - Ensure that the Apache balancer directive uses the correct port

Since OX Guard is part of the OSGi stack and Grizzly is used to register and serve the OX Guard servlets, the oxguard and oxguardpgp balancer directives in the Apache configuration have to be adjusted in order to set the correct port.

Find this lines in your apache configuration:

BalancerMember http://localhost:8080/oxguard timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1
BalancerMember http://localhost:8080/pgp timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1

and adjust the port to reflect the Grizzly setting:

BalancerMember http://localhost:8009/oxguard timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1
BalancerMember http://localhost:8009/pgp timeout=1800 smax=0 ttl=60 retry=60 loadfactor=100 route=OX1

Step 7 - Adapt configuration

This chapter is only valid for OX Guard-only node where no additional middleware services haven't been desired and installed before! As Guard is now using existing middleware mechanisms to access the databases you have to configure the following properties.

• configdb.properties
• copy an already configured file from a middleware node to /opt/open-xchange/etc on the OX Guard node.
• system.properties
• copy an already configured file from a middleware node to /opt/open-xchange/etc on the OX Guard node.
• server.properties
• set com.openexchange.connector.networkListenerHost to your desired hosts (e. g. *)
• logback.xml
• consolidate your old OX Guard logback.xml with the new one from within /opt/open-xchange/etc
• jolokia.properties
• copy an already configured file from a middleware node to /opt/open-xchange/etc on the OX Guard node.
• mpasswd
• copy your existing oxadminmaster credential storage to /opt/open-xchange/etc on the OX Guard node.
• guard-api.properties
• Adapt the property com.openexchange.guard.endpoint. This can be as simple as replacing the old port 8080 with 8009 but may differ depending on your setup

After copying those files, please make sure that they are owned by the group open-xchange, otherwise they cannot be read by the middleware service. In addition to the above mentioned properties you may would like to change the default configuration if the underlying middleware instance is only used for OX Guard:

• Disable Hazelcast cluster discovery
• hazelcast.properties: com.openexchange.hazelcast.network.join=empty
• remove all permissions set via configuration
• permissions.properties: set entries to false

Step 8 - Restart the groupware

Last, and certainly not least, you need to restart the groupware process:

Debian 7.x/RedHat Enterprise Linux 6.x/CentOS 6.x/SUSE Linux Enterprise Server 11.x

service open-xchange restart

Debian 8.x/RedHat Enterprise Linux 7.x/CentOS 7.x/SUSE Linux Enterprise Server 12.x

systemctl restart open-xchange

Step 9 - Execute update tasks

Before OX Guard should receive the first requests you have to ensure that all database schemas are up to date. This can be done by executing /opt/open-xchange/sbin/runupdate.

Congratulations! You should now have a running OX Guard 2.4.0!

Common Problems

When trying to decode a message I get a 404

This happens probably because the AppSuite ProxyPass Location directives are overwriting the OX Guard ones. To solve this, you will have to move the OX Guard ProxyPass Location directive right before the AppSuite ones.

Change the ordering of the locations from this:

<Location /ajax>
  ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api>
  ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api/oxguard>
  ProxyPass balancer://oxguard
</Location>

to this:

<Location /appsuite/api/oxguard>
  ProxyPass balancer://oxguard
</Location>
<Location /ajax>
  ProxyPass balancer://oxcluster/ajax
</Location>
<Location /appsuite/api>
  ProxyPass balancer://oxcluster/ajax
</Location>

After migration, OX Guard fails to accept any request

This may happen if configuration files that were copied during migration do not have the correct permission set. Please have a look at the OX Guard middleware log file to find any ERROR messages. The getmissingservices command-line tool may also help to identify services that have not been started and why they failed to start.