AppSuite:Debugging production servers - Revision history

Khgras: Replaced content with "The content on this page has moved to https://documentation.open-xchange.com/latest/ui/miscellaneous/debugging-prodserver.html Note: Open-Xchange is in the process of mig..."

2017-05-22T09:59:16Z

Replaced content with "The content on this page has moved to https://documentation.open-xchange.com/latest/ui/miscellaneous/debugging-prodserver.html Note: Open-Xchange is in the process of mig..."

← Older revision		Revision as of 09:59, 22 May 2017
Line 1:		Line 1:
	~~<!-- !!! -->~~		The content on this page has moved to https://documentation.open-xchange.com/latest/ui/miscellaneous/debugging-prodserver.html
	~~<!-- PLEASE APPLY CHANGES ONLY TO THE NEW TECHNICAL DOCUMENTATION~~: ~~wd/frontend~~/~~web~~/documentation -->
	~~<!-- !!! -->~~

	~~{{Stability~~-~~experimental}}~~		Note: Open-Xchange is in the process of migrating all its technical documentation to a new and improved documentation system (documentation.open-xchange.com). Please note as the migration takes place more information will be available on the new system and less on this system. Thank you for your understanding during this period of transition.
	~~<div class="title">Debugging production servers</div>~~

	~~A how-to for debugging your local UI plugin~~ in ~~combination with production and staging servers, which use redirects, HTTPS, and other things which break with~~ the ~~auto-generated Grunt configuration.~~

	~~== Introduction ==~~

	~~Not everybody has access~~ to a ~~dedicated test server. Sometimes, a plugin requires a backend~~ system ~~which is hard to access outside of existing infrastructure~~. ~~And sometimes, a bug appears only on a production server. There are many reasons why a server which is used by <tt>grunt dev</tt> can not be reconfigured and has to be used as~~-is.

	The most frequent reason, why debugging with a remote server fails, is a redirect to an external login page. After the login, the browser is usually redirected to the original domain, and all the cookies used for authentication are also set for that domain, not http://localhost:8337, where Grunt listens.

	~~The solution is to use grunt~~ as ~~an HTTP proxy server, which intercepts all requests to~~ the ~~server's domain.~~

	~~In this article, <tt>''example.com''</tt> is user for the server domain, which you should replace by the real domain of the OX server.~~

	~~== Configuring Grunt ==~~

	~~Following is an example <tt>grunt/local.conf.json</tt> file:~~

	{
	~~"proxy": true,~~
	~~"appserver": {~~
	~~"protocol": "https",~~
	~~"livereload": false,~~
	~~"server": "https://''example.com''/''custom-path''/",~~
	~~"path": "/''custom-path''/",~~
	~~"rejectUnauthorized": false,~~
	~~"verbose": [],~~
	~~"prefixes": [~~
	~~"build/",~~
	~~"../../web/ui/build"~~
	],
	~~"manifests": [~~
	~~"build/manifests/",~~
	~~"../../web/ui/build/manifests"~~
	]
	},
	~~"debug": true,~~
	~~"coreDir": "../../web/ui/build"~~
	}

	~~The relevant settings are:~~

	~~;<tt>"proxy": true</tt>~~
	~~: instructs Grunt to start an HTTP proxy on port 8080. Note that the value is outside the <tt>"appserver"</tt> section.~~
	~~;<tt>"protocol": "https"</tt>~~
	~~: to use HTTPS on port 8337.~~
	~~;<tt>"livereload": false</tt>~~
	~~: because LiveReload can't be proxied.~~
	~~;<tt>"path": "/''custom-path''/"</tt>~~
	~~: must correspond to the path in the <tt>"server"</tt> entry and~~ will ~~in most cases~~ be ~~<tt>"/appsuite/"</tt>.~~
	~~;<tt>"rejectUnauthorized": false</tt>~~
	~~: is helpful with certificate problems, especially for integration tests of~~ the ~~login mechanism on a POC~~ system ~~with self-signed certificates.~~
	~~;<tt>"index": "/"</tt>~~
	: If the main path uses an HTTP redirect for a custom login page, then the initial request should not be intercepted by Grunt. Adding this entry in the <tt>"appserver"</tt> section disables the interception. This has two side-effects:
	~~:# The main HTML file is always served by the original server,~~ and ~~therefore~~
	~~:# The timestamps used for cache-busting are not updated by Grunt.~~
	~~:Unfortunately,~~ this ~~means clearing the cache will be required more often. If that becomes too much overhead, you can log in, then remove the option and restart Grunt.~~

	~~== Generating certificates ==~~

	~~To avoid unnecessary warnings in the browser, Grunt can use a set of certificates which are trusted by the browser. For local use, a self-created CA should be enough~~. ~~By default, Grunt looks for the certificates in the subdirectory <tt>ssl</tt>, so~~ you ~~can create them there with the following commands:~~

	~~mkdir ssl~~
	~~cd ssl~~
	~~openssl req -x509 -newkey rsa:4096 -keyout rootCA.key -days 3650 -out rootCA.crt~~
	~~openssl req -newkey rsa:4096 -nodes -keyout host.key -out host.csr~~

	While most data entered for the certificates doesn't matter, the Common Name for the certificate signing request in the second <tt>openssl</tt> command must match the domain of the <tt>"server"</tt> setting in <tt>local.conf.json</tt>. Alternatively, it can be a higher domain with a wildcard, e.g. <tt>*.example.com</tt> for ~~<tt>"server": "https://my.example.com/appsuite/"</tt>.~~

	~~openssl x509 -req -in host.csr -CA rootCA.crt -CAkey rootCA.key \~~
	~~-CAcreateserial -out host.crt -days 365~~

	<tt>host.csr</tt> is only temporary and can be deleted now. Only the files <tt>rootCA.crt</tt>, <tt>host.key</tt> and <tt>host.crt</tt> are used by Grunt. The <tt>rootCA.</tt> files can be reused to create multiple host certificates. The first <tt>openssl</tt> command then only needs to be executed once. The <tt>host.</tt> files can also be reused as long as the domain(-wildcard) matches.

	~~The file <tt>rootCA.crt</tt> needs to be imported into~~ your browser as a trusted certificate authority (only once, if you are going to reuse the <tt>rootCA.*</tt> files). For Firefox, look in Preferences → Advanced → Certificates → View Certificates → Authorities → Import...
	~~For Chrome, look in Settings → Show advanced settings... → HTTPS/SSL → Manage certificates... → Authorities → Import...~~
	~~Since the authority can sign certificates for any domain, and its key is not protected much, it's better to not use the browser(-profile) which trusts~~ this ~~authority for normal Internet browsing.~~

	~~== Launching the browser ==~~

	After starting the proxy with <tt>grunt dev</tt>, you just need to use it when debugging your code in the browser. Most browsers have their own proxy settings, which override system-wide settings. They also respect environment variables like <tt>http_proxy</tt>.

	~~http_proxy=http://localhost:8080 firefox~~

	~~Chrome does not have its own settings (it starts the global settings dialog) and it ignores <tt>http_proxy</tt>. Instead, you have to use a command line parameter:~~

	~~chromium --proxy-server=http://localhost:8080~~

Frank.paczynski at 05:36, 11 February 2016

2016-02-11T05:36:52Z

← Older revision		Revision as of 05:36, 11 February 2016
Line 1:		Line 1:
			<!-- !!! -->
			<!-- PLEASE APPLY CHANGES ONLY TO THE NEW TECHNICAL DOCUMENTATION: wd/frontend/web/documentation -->
			<!-- !!! -->

	{{Stability-experimental}}		{{Stability-experimental}}
	<div class="title">Debugging production servers</div>		<div class="title">Debugging production servers</div>

J.ohny.b: generate host key without password, since there currently is no easy way to provide a password to the connect middleware and it's cumbersome anyway

2016-01-29T13:09:25Z

generate host key without password, since there currently is no easy way to provide a password to the connect middleware and it's cumbersome anyway

← Older revision		Revision as of 13:09, 29 January 2016
Line 65:		Line 65:
	cd ssl		cd ssl
	openssl req -x509 -newkey rsa:4096 -keyout rootCA.key -days 3650 -out rootCA.crt		openssl req -x509 -newkey rsa:4096 -keyout rootCA.key -days 3650 -out rootCA.crt
	openssl req -newkey rsa:4096 -keyout host.key -out host.csr		openssl req -newkey rsa:4096 -nodes -keyout host.key -out host.csr

	While most data entered for the certificates doesn't matter, the Common Name for the certificate signing request in the second <tt>openssl</tt> command must match the domain of the <tt>"server"</tt> setting in <tt>local.conf.json</tt>. Alternatively, it can be a higher domain with a wildcard, e.g. <tt>*.example.com</tt> for <tt>"server": "https://my.example.com/appsuite/"</tt>.		While most data entered for the certificates doesn't matter, the Common Name for the certificate signing request in the second <tt>openssl</tt> command must match the domain of the <tt>"server"</tt> setting in <tt>local.conf.json</tt>. Alternatively, it can be a higher domain with a wildcard, e.g. <tt>*.example.com</tt> for <tt>"server": "https://my.example.com/appsuite/"</tt>.

J.ohny.b: "path" options seems to be mandatory, at least we had problems when not setting this option to any value

2016-01-29T13:08:04Z

"path" options seems to be mandatory, at least we had problems when not setting this option to any value

← Older revision		Revision as of 13:08, 29 January 2016
Line 49:		Line 49:
	: because LiveReload can't be proxied.		: because LiveReload can't be proxied.
	;<tt>"path": "/''custom-path''/"</tt>		;<tt>"path": "/''custom-path''/"</tt>
	: must correspond to the path in the <tt>"server"</tt> entry and ~~can~~ be ~~omitted if it's~~ <tt>"/appsuite/"</tt>.		: must correspond to the path in the <tt>"server"</tt> entry and will in most cases be <tt>"/appsuite/"</tt>.
	;<tt>"rejectUnauthorized": false</tt>		;<tt>"rejectUnauthorized": false</tt>
	: is helpful with certificate problems, especially for integration tests of the login mechanism on a POC system with self-signed certificates.		: is helpful with certificate problems, especially for integration tests of the login mechanism on a POC system with self-signed certificates.

Viktor.Pracht: Added certificate generation

2015-09-22T12:35:11Z

Added certificate generation

← Older revision		Revision as of 12:35, 22 September 2015
Line 58:		Line 58:
	:Unfortunately, this means clearing the cache will be required more often. If that becomes too much overhead, you can log in, then remove the option and restart Grunt.		:Unfortunately, this means clearing the cache will be required more often. If that becomes too much overhead, you can log in, then remove the option and restart Grunt.

	~~After the configuration is complete, Grunt is started like always with~~		== Generating certificates ==

	~~grunt dev~~		To avoid unnecessary warnings in the browser, Grunt can use a set of certificates which are trusted by the browser. For local use, a self-created CA should be enough. By default, Grunt looks for the certificates in the subdirectory <tt>ssl</tt>, so you can create them there with the following commands:

			mkdir ssl
			cd ssl
			openssl req -x509 -newkey rsa:4096 -keyout rootCA.key -days 3650 -out rootCA.crt
			openssl req -newkey rsa:4096 -keyout host.key -out host.csr

			While most data entered for the certificates doesn't matter, the Common Name for the certificate signing request in the second <tt>openssl</tt> command must match the domain of the <tt>"server"</tt> setting in <tt>local.conf.json</tt>. Alternatively, it can be a higher domain with a wildcard, e.g. <tt>*.example.com</tt> for <tt>"server": "https://my.example.com/appsuite/"</tt>.

			openssl x509 -req -in host.csr -CA rootCA.crt -CAkey rootCA.key \
			-CAcreateserial -out host.crt -days 365

			<tt>host.csr</tt> is only temporary and can be deleted now. Only the files <tt>rootCA.crt</tt>, <tt>host.key</tt> and <tt>host.crt</tt> are used by Grunt. The <tt>rootCA.</tt> files can be reused to create multiple host certificates. The first <tt>openssl</tt> command then only needs to be executed once. The <tt>host.</tt> files can also be reused as long as the domain(-wildcard) matches.

			The file <tt>rootCA.crt</tt> needs to be imported into your browser as a trusted certificate authority (only once, if you are going to reuse the <tt>rootCA.*</tt> files). For Firefox, look in Preferences → Advanced → Certificates → View Certificates → Authorities → Import...
			For Chrome, look in Settings → Show advanced settings... → HTTPS/SSL → Manage certificates... → Authorities → Import...
			Since the authority can sign certificates for any domain, and its key is not protected much, it's better to not use the browser(-profile) which trusts this authority for normal Internet browsing.

	== Launching the browser ==		== Launching the browser ==

	~~Now that~~ the proxy ~~is running~~, you just need to use it when debugging your code in the browser. Most browsers have their own proxy settings, which override system-wide settings. They also respect environment variables like <tt>http_proxy</tt>.		After starting the proxy with <tt>grunt dev</tt>, you just need to use it when debugging your code in the browser. Most browsers have their own proxy settings, which override system-wide settings. They also respect environment variables like <tt>http_proxy</tt>.

	http_proxy=http://localhost:8080 firefox		http_proxy=http://localhost:8080 firefox

	Chrome does not have its own settings (it starts the global settings dialog) and it ignores http_proxy. Instead, you have to use a command line parameter:		Chrome does not have its own settings (it starts the global settings dialog) and it ignores <tt>http_proxy</tt>. Instead, you have to use a command line parameter:

	chromium --proxy-server=http://localhost:8080		chromium --proxy-server=http://localhost:8080

Viktor.Pracht: Described the new proxy feature of grunt dev

2015-09-22T09:30:47Z

Described the new proxy feature of grunt dev

Show changes

Viktor.Pracht: /* Squid */ Added SquidMan link

2015-09-01T09:56:17Z

Squid: Added SquidMan link

← Older revision		Revision as of 09:56, 1 September 2015
Line 38:		Line 38:
	# echo 'hosts_file /home/''user''/proxy/hosts' >> /etc/squid3/squid.conf		# echo 'hosts_file /home/''user''/proxy/hosts' >> /etc/squid3/squid.conf
	# service squid3 reload		# service squid3 reload

			For Macs, there's [http://squidman.net/squidman/ SquidMan]. It has a preferences item where you can edit the configuration file.

	=== Grunt ===		=== Grunt ===

Viktor.Pracht: Created page with "{{Stability-experimental}}
Debugging production servers
A how-to for debugging your local UI plugin in combination with production and staging server..."

2015-05-29T09:14:09Z

Created page with "{{Stability-experimental}} <div class="title">Debugging production servers</div> A how-to for debugging your local UI plugin in combination with production and staging server..."

New page

{{Stability-experimental}}
<div class="title">Debugging production servers</div>

A how-to for debugging your local UI plugin in combination with production and staging servers, which use redirects, HTTPS, and other things which break with a simple <tt>grunt dev</tt> command.

== Introduction ==

Not everybody has access to a dedicated test server. Sometimes, a plugin requires a backend system which is hard to access outside of existing infrastructure. And sometimes, a bug appears only on a production server. There are many reasons why a server which is used by <tt>grunt dev</tt> can not be reconfigured and has to be used as-is.

The most frequent reason, why debugging with a remote server fails, are redirects to an external login page. After the login, the browser is usually redirected to the original domain, and all the cookies used for authentication are also set for that domain, not http://localhost:8337, where Grunt listens.

The solution is to use a proxy server, which redirects all requests to the server's domain to Grunt. Until <tt>grunt dev</tt> can be used as an HTTP proxy directly, a dedicated proxy is required. This proxy needs to support custom hosts files. Any kind of rewriting or forwarding usually ends up in the wrong domain being used, and therefore still breaking cookies and redirects. The global <tt>/etc/hosts</tt> can't be used because Grunt still needs to access the original server, and use the original domain name for that to properly set the <tt>Host</tt> header.

The usual way to use a custom hosts file is to set it in the <tt>HOSTALIASES</tt> environment variable. Unfortunately, for security reasons, this variable is explicitly cleared when executing setuid processes. And this is exactly what proxy servers tend to do. A proxy server with explicit configuration support for custom hosts files is Squid. So that's what we will use for debugging.

== HOWTO ==

All commands are specified for Debian. Commands which need to be run as root start with <tt>#</tt>, commands which can be run as a normal user start with <tt>$</tt>.

The examples use <tt>''user''</tt> as user name, and all temporary files are stored in <tt>/home/''user''/proxy</tt>. You should use your own directory, and can pick any other directory. For the server domain, we'll use <tt>''example.com''</tt>, which should of course be replaced by the real domain of the OX server.

=== Squid ===

Install Squid.

# apt-get install squid3

Copy your <tt>/etc/hosts</tt> file to another location.

$ cp /etc/hosts /home/''user''/proxy/hosts

Add a line which redirects the target domain to localhost.

$ echo '127.0.0.1 ''example.com''<nowiki/>' >> /home/''user''/proxy/hosts

Now this file can be added to Squid's configuration file.

# echo 'hosts_file /home/''user''/proxy/hosts' >> /etc/squid3/squid.conf
# service squid3 reload

=== Grunt ===

Assuming you already have a working <tt>local.conf.json</tt>, add the following entries to the <tt>"appserver"</tt> section:

"protocol": "https",
"port": 443,
"server": "https://''example.com''/''custom-path''/",
"rejectUnauthorized": false,
"path": "/''custom-path''",

The entry <tt>"path"</tt> is optional, if the default <tt>/appsuite</tt> is used.

If the main path uses an HTTP redirect for a custom login page, then the initial request can't be intercepted by Grunt. The following entry disables it:

"index": "/",

This has two side-effects:
# The mail HTML file is always served by the original server, and therefore
# The timestamps used for cache-busting are not updated by Grunt.

This means clearing the cache will be required more often. If that becomes too much overhead, you can log in, then remove the option and restart Grunt.

=== Running <tt>grunt dev</tt>===

Since Grunt needs to bind to port 443, it requires additional privileges. There are several options. Running Grunt as root is the quickest, but also least secure way. Better alternatives are <tt>authbind</tt> and <tt>setcap</tt>.

Don't forget to stop your local web server (e.g. Apache),

# service apache2 stop

or at least disable SSL.

# a2dismod ssl
# service apache2 restart

==== <tt>authbind</tt> ====

<tt>authbind</tt> is a program which uses a setuid helper to bind to lower ports. It is usually not installed by default.

# apt-get install authbind

The configuration is performed by creating files in <tt>/etc/authbind</tt> and giving execute permissions to the appropriate users.

# touch /etc/authbind/443
# chown ''user'' /etc/authbind/443
# chmod 544 /etc/authbind

Now, Grunt can be started using the wrapper.

$ authbind --deep grunt dev

==== <tt>setcap</tt> ====

This option can be used if your file system supports capabilities. It is less secure since it grants the permission to bind to all low ports to any Node program. But if supported, it at least does not require additional tools.

# setcap "cap_net_bind_service=+ep" "$(readlink -f "$(which node)")"

On OS X, <tt>greadlink</tt> needs to be used instead of <tt>readlink</tt>. If you don't have it, just find out where the actual <tt>node</tt> binary is installed and use it directly as the second parameter to <tt>setcap</tt>.

Grunt can then be run as usual.

$ grunt dev

=== Launching the browser ===

Now that everything is running, you only need to use the proxy when debugging your code in the browser. Most browsers have their own proxy settings, which override system-wide settings. They also respect environment variables like <tt>http_proxy</tt>.

$ http_proxy=http://localhost:3128 firefox

Chrome does not have its own settings (it starts the global settings dialog) and it ignores http_proxy. Instead, you have to use a command line parameter:

$ chromium --proxy-server=http://localhost:3128