1. No category

Concept about how to SSL offload and load balance with Apache2
Version: 2011.02.06-0.2
Author: Margus Pärt
Table of Contents
1. About this document..............................................................................................................................2
2. Description of the idea...........................................................................................................................3
3. SSL offloader and Load balancer...........................................................................................................4
3.1 Tasks................................................................................................................................................4
3.1.1 SSL offloader's functional tasks..............................................................................................4
3.1.2 SSL offloader's informative tasks............................................................................................4
3.1.3 Load balancer's functional tasks..............................................................................................4
3.1.4 Load balancer's informative tasks............................................................................................4
3.1.5 Backend server/application's functional tasks.........................................................................4
3.2 Install...............................................................................................................................................5
3.2.1 Debian Packages......................................................................................................................5
3.2.2 Create base (create two different Apache configurations for one binary)...............................5
3.3 Base configuration...........................................................................................................................7
3.3.1 SSL offloader's functional tasks..............................................................................................7
3.3.2 SSL offloader's informative tasks..........................................................................................10
3.3.3 Load balancer's functional task..............................................................................................10
3.3.4 Load balancer's informative tasks..........................................................................................11
3.4 Configuration procedure examples...............................................................................................12
3.4.1 Add new backend servers, and domain to be SSL offloaded and load balanced...................12
3.4.2 Add a new VirtualHost with sticky-sessions controlled in Load Balancer (can be used for
Apache2, Tomcat, Jboss and Weblogic backends.)........................................................................13
3.5 Upgrading......................................................................................................................................13
3.5.1 OS..........................................................................................................................................13
3.5.2 SSL offloader and Load balancer concept implementation...................................................13
3.6 Backup and restore........................................................................................................................15
3.6.1 Backup...................................................................................................................................15
3.6.2 Restore...................................................................................................................................15
4. Backend server.....................................................................................................................................16
4.1 Apache2.........................................................................................................................................16
4.2 Weblogic........................................................................................................................................17
5. Configuration recommendations/notes................................................................................................18
5.1 Apache...........................................................................................................................................18
5.2 Loadbalancing...............................................................................................................................18
6. Links.....................................................................................................................................................19
1. About this document
Newest version is always kept in: https://apache2-ssloffload-andloadbalance.googlecode.com/svn/trunk/Documentation/ (.odt and .pdf files).
Concept described in this document can be used on every OS, copy-paste to Bash is tested to work on
Ubuntu Maverick (10.10) and Debian Squeeze (6.0).
Version history:
What
When
Who
Added “About this document”
and “Upgrade steps”.
2011.02.06-0.2
Margus Pärt
Initial.
2011.02.06-0.1
Margus Pärt
2
2. Description of the idea
SSL offload and balancing.
Not to repeat configuration so much, also to make logic more separated, one Apache2 binary is ran
with two different configurations:
• SSL offloader (In folder: /etc/apache2-ssloffloader) (It takes also HTTP requests from user)
• Load balancer (In folder: /etc/apache2-balancer)
Listen addresses:
• SSL offloader listens at external IP
• Load balancer listens at 127.0.0.1, only SSL offloader can send request to this address
Requests path steps (abstract example, there are more variables and headers involved):
1. Client opens connection to 80 or 443, sends HTTP request: “GET /something HTTP/1.0 \n
Host: www.example.ee \n SSL_HEADER: h2xx”
2. SSL offloader deletes SSL_HEADER and sets a new one from Apache2 env variable named
SSL_HEADER, adds client info, and with ProxyPass sends request to Load balancer:
“GET /something HTTP/1.0 \n Host: www.example.ee \n SSL_HEADER: fixed \n XForwarded-For: 123.231.123.231”
3. Load balancer sends request to correct place
4. Backend server (response to requests)
5. .. and reverse way through the chain back to the client
Using SSL offloader and Load balance proxy combination gives us following upsides and downsides:
• + you don't have to repeat configuration in both 443 or 80
• + you can have multiple different domains behind one wilcard certificate
• - logic differs from conventional Apache2 (but I see it as defining standard on top of another
standard, what makes life easier, if you have a lot of VirtualHosts)
3
3. SSL offloader and Load balancer
3.1 Tasks
3.1.1
SSL offloader's functional tasks
1. Take requests on ports 80 and 443 from clients, SSL VirtualHosts need to be defined in
directory: /etc/apache2-ssloffloader/sites-enabled, certificates are kept in directory:
/etc/certificates-apache2.
2. Clean headers from client sent data (unset SSL_CLIENT_CERT etc); set correct headers for
backend server from env values, so backend server knows if client is authenticated. Setting
headers for backend server is done in file: /etc/apache2-ssloffloader/conf.d/ssl_offload_headers
3. Default SSLVerifyClient URL-s for all the hosts is defined in file: /etc/apache2ssloffloader/conf.d/ssl_smartcard_auth_url
4. Forward request to balancer.proxy, proxy configuration is defined in file: /etc/apache2ssloffloader/mods-enabled/proxy.conf and ProxyPass has to be done in VirtualHost definition
for SSL offloader file: /etc/apache2-ssloffloader/sites-enabled/name.of.site.conf
3.1.2
SSL offloader's informative tasks
1. Log requests, logging is defined in file: /etc/apache2-ssloffloader/conf.d/logging
2. Show server status at http://server/ssloffloader-status, defined in file: /etc/apache2ssloffloader/conf.d/serverinfo-status
3.1.3
Load balancer's functional tasks
1. Take requests for port 80, name based virtualhosts are defined in directory: /etc/apache2balancer/sites-enabled (I'd recommend use filname format: domain.subdomain.subdomain.conf)
2. Proxy requests for correct backend node, using balancers configured in directory: /etc/apache2balancer/balancers and proxy configured in /etc/apache2-balancer/mods-enabled/proxy.conf
3.1.4
Load balancer's informative tasks
1. Log requests, logging is defined in file: /etc/apache2-balancer/conf.d/logging
2. Show server status at http://server/balancer-status, defined in file: /etc/apache2balancer/conf.d/serverinfo-status
3. Show and let configure balancers at http://server/balancer-manager, defined in file:
/etc/apache2-balancer/conf.d/serverinfo-balancermanager
3.1.5
Backend server/application's functional tasks
1. Receive request and understand if user has done smartcard authentication: for apache:
/etc/apache2/conf.d/ssl_env_values_from_headers, for weblogic (Client Cert Proxy Enabled in
Console, or “<client-cert-proxy-enabled>” in web.xml)
4
2. Response
3.2 Install
3.2.1
Debian Packages
# Install Apache2
apt-get install apache2 libapache2-mod-rpaf
3.2.2
Create base (create two different Apache configurations for one binary)
# Please set correct env value for external IP
LB_EXTERNAL_IP='192.168.0.9'
LB_INTERNAL_IP='127.0.0.1'
# Create host file for our needs (So we can duplicate configurations to other servers, without changing them.)
echo $LB_EXTERNAL_IP ssloffloader.proxy >> /etc/hosts
echo $LB_INTERNAL_IP balancer.proxy >> /etc/hosts
# Remove unnessesary VirtualHostss
rm -rf /etc/apache2/sites-enabled/* /etc/apache2/sites-available/*
# Copy (or create) nessesary structure
cp -a /etc/apache2 /etc/apache2-ssloffloader
cp -a /etc/default/apache2 /etc/default/apache2-ssloffloader
cp -a /var/log/apache2 /var/log/apache2-ssloffloader
#
cp -a /etc/apache2 /etc/apache2-balancer
cp -a /etc/default/apache2 /etc/default/apache2-balancer
cp -a /var/log/apache2 /var/log/apache2-balancer
mkdir -p /etc/apache2-balancer/balancers
# Disable default Apache2 configuration
update-rc.d apache2 disable
chmod 000 /etc/apache2
# Create startup script for apache2-ssloffloader
cat > /etc/init.d/apache2-ssloffloader <<EOF
#!/bin/sh
### BEGIN INIT INFO
# Provides: apache2-ssloffloader
# Required-Start: \$local_fs \$remote_fs \$network \$syslog \$named
# Required-Stop: \$local_fs \$remote_fs \$network \$syslog \$named
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: true
# Short-Description: Start/stop apache2-ssloffloader web server
### END INIT INFO
APACHE_CONFDIR='/etc/apache2-ssloffloader' /etc/init.d/apache2 \$1
EOF
#
chmod 755 /etc/init.d/apache2-ssloffloader
5
# Create startup script for apache2-balancer
cat > /etc/init.d/apache2-balancer <<EOF
#!/bin/sh
### BEGIN INIT INFO
# Provides:
apache2-balancer
# Required-Start: \$local_fs \$remote_fs \$network \$syslog \$named
# Required-Stop: \$local_fs \$remote_fs \$network \$syslog \$named
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: true
# Short-Description: Start/stop apache2-balancer web server
### END INIT INFO
APACHE_CONFDIR='/etc/apache2-balancer' /etc/init.d/apache2 \$1
EOF
#
chmod 755 /etc/init.d/apache2-balancer
# Correct files for our need
sed -i 's/NameVirtualHost \*:80/NameVirtualHost ssloffloader.proxy:80/g' /etc/apache2-ssloffloader/ports.conf
sed -i 's/Listen 80/Listen ssloffloader.proxy:80/g' /etc/apache2-ssloffloader/ports.conf
sed -i 's/Listen 443/Listen ssloffloader.proxy:443/g' /etc/apache2-ssloffloader/ports.conf
sed -i 's/NameVirtualHost \*:80/NameVirtualHost balancer.proxy:80/g' /etc/apache2-balancer/ports.conf
sed -i 's/Listen 80/Listen balancer.proxy:80/g' /etc/apache2-balancer/ports.conf
sed -i 's/Listen 443/Listen balancer.proxy:443/g' /etc/apache2-balancer/ports.conf
# Set default DocumentRoot
echo DocumentRoot /var/www > /etc/apache2-ssloffloader/conf.d/documentroot
echo DocumentRoot /var/www > /etc/apache2-balancer/conf.d/documentroot
# Enable/disable nessesary modules
APACHE_CONFDIR='/etc/apache2-ssloffloader' a2enmod proxy proxy_connect proxy_http rewrite headers ssl
APACHE_CONFDIR='/etc/apache2-balancer' a2enmod proxy proxy_connect proxy_http rewrite headers proxy_balancer rpaf
APACHE_CONFDIR='/etc/apache2-ssloffloader' a2dismod rpaf
# Create directory for internal balancers and do that content of this folder is loaded
mkdir -p /etc/apache2-balancer/balancers
echo Include /etc/apache2-balancer/balancers/*conf > /etc/apache2-balancer/conf.d/include_balancers
# Set automatic start after reboot
update-rc.d apache2-ssloffloader defaults
update-rc.d apache2-balancer defaults
# Restart both services. In result you have two different Apache configuration on different IP-s running.
/etc/init.d/apache2-ssloffloader restart
/etc/init.d/apache2-balancer restart
6
3.3 Base configuration
3.3.1
SSL offloader's functional tasks
# Take requests for both 80 and 443 directly from client,
# SSL VirtualHosts need to be defined in directory: /etc/apache2-ssloffloader/sites-enabled, certificates are kept in directory:
/etc/certificates-apache2
cat > /etc/apache2-ssloffloader/sites-enabled/default <<EOF
<VirtualHost ssloffloader.proxy:80>
ProxyPass / http://balancer.proxy/
</VirtualHost>
EOF
#
cat > /etc/apache2-ssloffloader/sites-enabled/default-ssl <<EOF
<VirtualHost ssloffloader.proxy:443>
ProxyPass / http://balancer.proxy/
# + Certificates
SSLEngine on
SSLCertificateFile /etc/certificates-apache2/sites/wildcard.example.ee.crt
SSLCertificateKeyFile /etc/certificates-apache2/sites/wildcard.example.ee.key
SSLCertificateChainFile /etc/certificates-apache2/sites/juur-thawte.crt
SSLCACertificateFile /etc/certificates-apache2/ssl.crt/id.crt
SSLVerifyClient none
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
</VirtualHost>
EOF
# Headers cleaning from client sent data; setting headers for backend server is done in file: /etc/apache2ssloffloader/conf.d/ssl_offload_headers
cat > /etc/apache2-ssloffloader/conf.d/ssl_offload_headers <<EOF
#############################################
# Apache
#############################################
RequestHeader unset HTTPS
RequestHeader unset SSL_PROTOCOL
RequestHeader unset SSL_SESSION_ID
RequestHeader unset SSL_CIPHER
RequestHeader unset SSL_CIPHER_EXPORT
RequestHeader unset SSL_CIPHER_USEKEYSIZE
RequestHeader unset SSL_CIPHER_ALGKEYSIZE
RequestHeader unset SSL_VERSION_INTERFACE
RequestHeader unset SSL_VERSION_LIBRARY
RequestHeader unset SSL_CLIENT_M_VERSION
RequestHeader unset SSL_CLIENT_M_SERIAL
RequestHeader unset SSL_CLIENT_S_DN
RequestHeader unset SSL_CLIENT_S_DN_x509
RequestHeader unset SSL_CLIENT_I_DN
RequestHeader unset SSL_CLIENT_I_DN_x509
RequestHeader unset SSL_CLIENT_V_START
RequestHeader unset SSL_CLIENT_V_END
RequestHeader unset SSL_CLIENT_A_SIG
RequestHeader unset SSL_CLIENT_A_KEY
RequestHeader unset SSL_CLIENT_CERT
RequestHeader unset SSL_CLIENT_CERT_CHAINn
RequestHeader unset SSL_CLIENT_VERIFY
RequestHeader unset SSL_SERVER_M_VERSION
7
RequestHeader unset SSL_SERVER_M_SERIAL
RequestHeader unset SSL_SERVER_S_DN
RequestHeader unset SSL_SERVER_S_DN_x509
RequestHeader unset SSL_SERVER_I_DN
RequestHeader unset SSL_SERVER_I_DN_x509
RequestHeader unset SSL_SERVER_V_START
RequestHeader unset SSL_SERVER_V_END
RequestHeader unset SSL_SERVER_A_SIG
RequestHeader unset SSL_SERVER_A_KEY
RequestHeader unset SSL_SERVER_CERT
RequestHeader set HTTPS "%{HTTPS}s" env=HTTPS
RequestHeader set SSL_PROTOCOL "%{SSL_PROTOCOL}s" env=SSL_PROTOCOL
RequestHeader set SSL_SESSION_ID "%{SSL_SESSION_ID}s" env=SSL_SESSION_ID
RequestHeader set SSL_CIPHER "%{SSL_CIPHER}s" env=SSL_CIPHER
RequestHeader set SSL_CIPHER_EXPORT "%{SSL_CIPHER_EXPORT}s" env=SSL_CIPHER_EXPORT
RequestHeader set SSL_CIPHER_USEKEYSIZE "%{SSL_CIPHER_USEKEYSIZE}s" env=SSL_CIPHER_USEKEYSIZE
RequestHeader set SSL_CIPHER_ALGKEYSIZE "%{SSL_CIPHER_ALGKEYSIZE}s" env=SSL_CIPHER_ALGKEYSIZE
RequestHeader set SSL_VERSION_INTERFACE "%{SSL_VERSION_INTERFACE}s" env=SSL_VERSION_INTERFACE
RequestHeader set SSL_VERSION_LIBRARY "%{SSL_VERSION_LIBRARY}s" env=SSL_VERSION_LIBRARY
RequestHeader set SSL_CLIENT_M_VERSION "%{SSL_CLIENT_M_VERSION}s" env=SSL_CLIENT_M_VERSION
RequestHeader set SSL_CLIENT_M_SERIAL "%{SSL_CLIENT_M_SERIAL}s" env=SSL_CLIENT_M_SERIAL
RequestHeader set SSL_CLIENT_S_DN "%{SSL_CLIENT_S_DN}s" env=SSL_CLIENT_S_DN
RequestHeader set SSL_CLIENT_S_DN_x509 "%{SSL_CLIENT_S_DN_x509}s" env=SSL_CLIENT_S_DN_x509
RequestHeader set SSL_CLIENT_I_DN "%{SSL_CLIENT_I_DN}s" env=SSL_CLIENT_I_DN
RequestHeader set SSL_CLIENT_I_DN_x509 "%{SSL_CLIENT_I_DN_x509}s" env=SSL_CLIENT_I_DN_x509
RequestHeader set SSL_CLIENT_V_START "%{SSL_CLIENT_V_START}s" env=SSL_CLIENT_V_START
RequestHeader set SSL_CLIENT_V_END "%{SSL_CLIENT_V_END}s" env=SSL_CLIENT_V_END
RequestHeader set SSL_CLIENT_A_SIG "%{SSL_CLIENT_A_SIG}s" env=SSL_CLIENT_A_SIG
RequestHeader set SSL_CLIENT_A_KEY "%{SSL_CLIENT_A_KEY}s" env=SSL_CLIENT_A_KEY
RequestHeader set SSL_CLIENT_CERT "%{SSL_CLIENT_CERT}s" env=SSL_CLIENT_CERT
RequestHeader set SSL_CLIENT_CERT_CHAINn "%{SSL_CLIENT_CERT_CHAINn}s" env=SSL_CLIENT_CERT_CHAINn
RequestHeader set SSL_CLIENT_VERIFY "%{SSL_CLIENT_VERIFY}s" env=SSL_CLIENT_VERIFY
RequestHeader set SSL_SERVER_M_VERSION "%{SSL_SERVER_M_VERSION}s" env=SSL_SERVER_M_VERSION
RequestHeader set SSL_SERVER_M_SERIAL "%{SSL_SERVER_M_SERIAL}s" env=SSL_SERVER_M_SERIAL
RequestHeader set SSL_SERVER_S_DN "%{SSL_SERVER_S_DN}s" env=SSL_SERVER_S_DN
RequestHeader set SSL_SERVER_S_DN_x509 "%{SSL_SERVER_S_DN_x509}s" env=SSL_SERVER_S_DN_x509
RequestHeader set SSL_SERVER_I_DN "%{SSL_SERVER_I_DN}s" env=SSL_SERVER_I_DN
RequestHeader set SSL_SERVER_I_DN_x509 "%{SSL_SERVER_I_DN_x509}s" env=SSL_SERVER_I_DN_x509
RequestHeader set SSL_SERVER_V_START "%{SSL_SERVER_V_START}s" env=SSL_SERVER_V_START
RequestHeader set SSL_SERVER_V_END "%{SSL_SERVER_V_END}s" env=SSL_SERVER_V_END
RequestHeader set SSL_SERVER_A_SIG "%{SSL_SERVER_A_SIG}s" env=SSL_SERVER_A_SIG
RequestHeader set SSL_SERVER_A_KEY "%{SSL_SERVER_A_KEY}s" env=SSL_SERVER_A_KEY
RequestHeader set SSL_SERVER_CERT "%{SSL_SERVER_CERT}s" env=SSL_SERVER_CERT
#############################################
# Weblogic
#############################################
RequestHeader unset WL-Proxy-SSL
RequestHeader unset WL-Proxy-Client-Cert
RequestHeader unset WL-Proxy-Client-Keysize
RequestHeader unset WL-Proxy-Client-Secretkeysize
RequestHeader unset WL-Proxy-Client-IP
RequestHeader unset Proxy-Client-IP
RequestHeader unset X-Forwarded-For
RequestHeader unset X-WebLogic-KeepAliveSecs
RequestHeader unset X-WebLogic-Request-ClusterInfo
RequestHeader unset x-weblogic-cluster-hash
RequestHeader set WL-Proxy-SSL "true" env=HTTPS
RequestHeader set WL-Proxy-Client-Keysize "256" env=HTTPS
8
RequestHeader set WL-Proxy-Client-Secretkeysize "256" env=HTTPS
RequestHeader set WL-Proxy-Client-IP "%{REMOTE_ADDR}s"
RequestHeader set Proxy-Client-IP "%{REMOTE_ADDR}s"
RequestHeader set X-Forwarded-For "%{REMOTE_ADDR}s"
RequestHeader set X-WebLogic-KeepAliveSecs "30"
# Set Cert from SSL_CLIENT_CERT env value + clean it for Weblogic (only cert content)
RequestHeader set WL-Proxy-Client-Cert "%{SSL_CLIENT_CERT}s" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "-----BEGIN CERTIFICATE----- (.*) -----END CERTIFICATE-----" "$1"
env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
RequestHeader edit WL-Proxy-Client-Cert "[^\w\d\/+=]" "" env=SSL_CLIENT_CERT
EOF
# Default SSLClientVerify path for all the hosts is defined in file: /etc/apache2-ssloffloader/conf.d/ssl_smartcard_auth_url
cat > /etc/apache2-ssloffloader/conf.d/ssl_smartcard_auth_url <<EOF
# URL for client cert auth - base websites
<Location ~ "auth/smartcard">
SSLOptions +StdEnvVars +ExportCertData
SSLVerifyClient optional
SSLVerifyDepth 2
</Location>
# One Java app
<Location ~ "idLogin">
SSLOptions +StdEnvVars +ExportCertData
SSLVerifyClient optional
SSLVerifyDepth 2
</Location>
EOF
# Forward request to balancer.proxy, proxy configuration is defined in file: /etc/apache2/mods-enabled/proxy.conf
cat > /etc/apache2-ssloffloader/mods-enabled/proxy.conf <<EOF
<IfModule mod_proxy.c>
#turning ProxyRequests on and allowing proxying from all may allow
9
#spammers to use your proxy to send email.
ProxyRequests Off
<Proxy *>
AddDefaultCharset off
Order deny,allow
Allow from all
</Proxy>
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
ProxyVia Off
# Nessesary that Host: in header would remain intact
ProxyPreserveHost On
ProxyTimeout 6000
</IfModule>
EOF
3.3.2
SSL offloader's informative tasks
#Log requests, logging is defined in file: /etc/apache2-ssloffloader/conf.d/logging
cat > /etc/apache2-ssloffloader/conf.d/logging <<EOF
LogFormat "%V:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" ria_vhost_combined
ErrorLog "|/usr/bin/logger -p local1.error -t apache2-ssloffloader"
CustomLog "|/usr/bin/logger -p local1.info -t apache2-ssloffloader" ria_vhost_combined
EOF
#Show server status, defined in file: /etc/apache2-ssloffloader/conf.d/serverinfo-status
cat > /etc/apache2-ssloffloader/conf.d/serverinfo-status <<EOF
ExtendedStatus On
<Location /ssloffloader-status>
SetHandler server-status
Order Allow,Deny
Allow from 192.168.252 172.19
</Location>
ProxyPass
EOF
3.3.3
/ssloffloader-status
!
Load balancer's functional task
# Take requests for port 80, name based virtualhosts are defined in directory: /etc/apache2-balancer/sites-enabled
# (I'd recommend use filname format: domain.subdomain.subdomain.conf)
cat > /etc/apache2-balancer/sites-enabled/ee.example.example.conf <<EOF
<VirtualHost balancer.proxy:80>
ServerName example.example.ee
ServerAlias data.example.ee
ProxyPass
/
balancer://kit.avalik.vm2-apache-1/
</VirtualHost>
EOF
# Proxy requests for correct backend node, using balancers configured in directory: /etc/apache2-balancer/balancers and proxy
configured in /etc/apache2-balancer/mods-enabled/proxy.conf
10
cat > /etc/apache2-balancer/balancers/kit.avalik.vm2-apache-1.conf <<EOF
# this is example balancer, you have to change it later
<Proxy balancer://kit.avalik.vm2-apache-1>
BalancerMember http://10.0.6.153:80
BalancerMember http://10.0.6.154:80
</Proxy>
EOF
cat > /etc/apache2-balancer/mods-enabled/proxy.conf <<EOF
<IfModule mod_proxy.c>
#turning ProxyRequests on and allowing proxying from all may allow
#spammers to use your proxy to send email.
ProxyRequests Off
<Proxy *>
AddDefaultCharset off
Order deny,allow
Allow from all
</Proxy>
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
ProxyVia Off
# Nessesary that Host: in header would remain intact
ProxyPreserveHost On
ProxyTimeout 6000
# FIX: needed so that mod-itk would not exit (same tcpsession different host problem)
# TODO:
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
</IfModule>
EOF
3.3.4
Load balancer's informative tasks
# Log requests, logging is defined in file: /etc/apache2-balancer/conf.d/logging
cat > /etc/apache2-balancer/conf.d/logging <<EOF
LogFormat "%V:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" ria_vhost_combined
ErrorLog "|/usr/bin/logger -p local1.error -t apache2-balancer"
CustomLog "|/usr/bin/logger -p local1.info -t apache2-balancer" ria_vhost_combined
EOF
# Show server status, defined in file: /etc/apache2-balancer/conf.d/balancer-status
cat > /etc/apache2-balancer/conf.d/balancer-status <<EOF
ExtendedStatus On
<Location /balancer-status>
SetHandler server-status
Order Allow,Deny
Allow from 192.168.252 172.19
</Location>
ProxyPass
EOF
/balancer-status
!
# Show and let configure balancers: /etc/apache2-balancer/conf.d/balancer-manager
cat > /etc/apache2-balancer/conf.d/balancer-manager <<EOF
# Show LB balancer status
11
<Location /balancer-manager>
SetHandler balancer-manager
Order Allow,Deny
Allow from 192.168.252 172.19
</Location>
ProxyPass
EOF
/balancer-manager
!
3.4 Configuration procedure examples
3.4.1
Add new backend servers, and domain to be SSL offloaded and load balanced.
Description of steps:
1.) * Only if you need a new VirtualHost with different certificate for that domain *, define new
SSL VirtualHost in file: /etc/apache2-ssloffloader/sitesenabled/com.anotherdomain.subdomain.conf from what ProxyPass to balancer.proxy (You don't
have to define new HTTP VirtualHost, default will take care of everything.)
2.) Define new balancer for “anotherwebservers.subnet.kit” server group in file: /etc/apache2balancer/balancers/kit.subnet.anotherwebservers.conf
3.) Create new named VirtualHost for “subdomain.anotherdomain.com” in file: /etc/apache2balancer/sites-enabled/com.anotherdomain.subdomain.conf
Steps to do:
cat > /etc/apache2-ssloffloader/sites-enabled/com.anotherdomain.subdomain.conf <<EOF
Listen ssloffloader.proxy:444
<VirtualHost ssloffloader.proxy:444>
ProxyPass / http://balancer.proxy/
# + Certificates
SSLEngine on
SSLCertificateFile /etc/certificates-apache2/sites/subdomain.anotherdomain.com.crt
SSLCertificateKeyFile /etc/certificates-apache2/sites/subdomain.anotherdomain.com.key
SSLCertificateChainFile /etc/certificates-apache2/sites/juur-thawte.crt
SSLCACertificateFile /etc/certificates-apache2/ssl.crt/id.crt
SSLVerifyClient none
SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
</VirtualHost>
EOF
cat > /etc/apache2-balancer/balancers/kit.subnet.anotherwebservers.conf <<EOF
<Proxy balancer://kit.subnet.anotherwebserver>
BalancerMember http://10.0.6.201:80
BalancerMember http://10.0.6.202:80
</Proxy>
EOF
12
cat > /etc/apache2-balancer/sites-enabled/com.anotherdomain.subdomain.conf <<EOF
<VirtualHost balancer.proxy:80>
ServerName subdomain.anotherdomain.com
ProxyPass
/
balancer://vm2-apache-1.avalik.kit/
</VirtualHost>
EOF
Testing:
1. Change your hosts file and make usual HTTP(S) request (or telnet server 80\n GET /
HTTP/1.0\n Host: subdomain.anotherdomain.com)
3.4.2
Add a new VirtualHost with sticky-sessions controlled in Load Balancer (can be used for
Apache2, Tomcat, Jboss and Weblogic backends.)
Description of steps:
1. Do as in step 3.4.1 Add new backend servers, and domain to be SSL offloaded and load
balanced. but create different balancer.
2. Create proxy balancer and set route_id's for nodes.
3. Enable mod_headers, if not enabled, and set stickysession name + create rule for adding cookie
with that name and route_id, if it changes. (Route_id is taken from
stickysession_name=sometext.this_value_is_taken. )
Steps to do:
cat > /etc/apache2-balancer/balancers/kit.subnet.weblogic-app-servers__application.conf <<EOF
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy balancer://kit.subnet.weblogic-app-servers__application>
BalancerMember http://10.0.6.136:7010 route=1
BalancerMember http://10.0.6.136:7010 route=2
ProxySet stickysession=ROUTEID
</Proxy>
EOF
Testing:
1. Order of getting correct routeid from stickysession_name is: 1.) from URL, 2.) from cookie.
That means that you can test solution by going: http://server/?ROUTEID=.change_id_to_test
and see results from https://server/balancer-manager and https://server/balancer-status
3.5 Upgrading
3.5.1
OS
Uprading from Debian Squeeze or Ubuntu Maverick to newer should be without complications and
additional steps. It will be tested, and if needed, additional steps will be added to here.
3.5.2
SSL offloader and Load balancer concept implementation
1. Backup (3.6.1 Backup)
2. Do install steps by this document's manual (3.2 Install)
13
3.6 Backup and restore
3.6.1
Backup
/etc/*apache2* (apache2-ssloffloader, apache2-balancer, certificates-apache2) must be backed up
regularly.
3.6.2
Restore
Install steps + copy apache2-ssloffloader, apache2-balancer and certificates-apache2 back up /etc
folder.
14
4. Backend server
4.1 Apache2
# Additional install
apt-get install libapache2-mod-rpaf
# Enable nessesary modules
a2enmod headers rpaf
# Edit /etc/apache2/mods-enabled/rpaf.conf (Read more: http://stderr.net/apache/rpaf/)
RPAFproxy_ips
# Create SSL env values from HTTP headers (only from SSL offloader and balancer should be requests allowed, or major security
problem)
cat > /etc/apache2/conf.d/ssl_env_values_from_headers <<EOF
SetEnvIf HTTPS "(..*)" HTTPS=$1
SetEnvIf SSL_PROTOCOL "(..*)" SSL_PROTOCOL=$1
SetEnvIf SSL_SESSION_ID "(..*)" SSL_SESSION_ID=$1
SetEnvIf SSL_CIPHER "(..*)" SSL_CIPHER=$1
SetEnvIf SSL_CIPHER_EXPORT "(..*)" SSL_CIPHER_EXPORT=$1
SetEnvIf SSL_CIPHER_USEKEYSIZE "(..*)" SSL_CIPHER_USEKEYSIZE=$1
SetEnvIf SSL_CIPHER_ALGKEYSIZE "(..*)" SSL_CIPHER_ALGKEYSIZE=$1
SetEnvIf SSL_VERSION_INTERFACE "(..*)" SSL_VERSION_INTERFACE=$1
SetEnvIf SSL_VERSION_LIBRARY "(..*)" SSL_VERSION_LIBRARY=$1
SetEnvIf SSL_CLIENT_M_VERSION "(..*)" SSL_CLIENT_M_VERSION=$1
SetEnvIf SSL_CLIENT_M_SERIAL "(..*)" SSL_CLIENT_M_SERIAL=$1
SetEnvIf SSL_CLIENT_S_DN "(..*)" SSL_CLIENT_S_DN=$1
SetEnvIf SSL_CLIENT_S_DN_x509 "(..*)" SSL_CLIENT_S_DN_x509=$1
SetEnvIf SSL_CLIENT_I_DN "(..*)" SSL_CLIENT_I_DN=$1
SetEnvIf SSL_CLIENT_I_DN_x509 "(..*)" SSL_CLIENT_I_DN_x509=$1
SetEnvIf SSL_CLIENT_V_START "(..*)" SSL_CLIENT_V_START=$1
SetEnvIf SSL_CLIENT_V_END "(..*)" SSL_CLIENT_V_END=$1
SetEnvIf SSL_CLIENT_A_SIG "(..*)" SSL_CLIENT_A_SIG=$1
SetEnvIf SSL_CLIENT_A_KEY "(..*)" SSL_CLIENT_A_KEY=$1
SetEnvIf SSL_CLIENT_CERT "(..*)" SSL_CLIENT_CERT=$1
SetEnvIf SSL_CLIENT_CERT_CHAINn "(..*)" SSL_CLIENT_CERT_CHAINn=$1
SetEnvIf SSL_CLIENT_VERIFY "(..*)" SSL_CLIENT_VERIFY=$1
SetEnvIf SSL_SERVER_M_VERSION "(..*)" SSL_SERVER_M_VERSION=$1
SetEnvIf SSL_SERVER_M_SERIAL "(..*)" SSL_SERVER_M_SERIAL=$1
SetEnvIf SSL_SERVER_S_DN "(..*)" SSL_SERVER_S_DN=$1
SetEnvIf SSL_SERVER_S_DN_x509 "(..*)" SSL_SERVER_S_DN_x509=$1
SetEnvIf SSL_SERVER_I_DN "(..*)" SSL_SERVER_I_DN=$1
SetEnvIf SSL_SERVER_I_DN_x509 "(..*)" SSL_SERVER_I_DN_x509=$1
SetEnvIf SSL_SERVER_V_START "(..*)" SSL_SERVER_V_START=$1
SetEnvIf SSL_SERVER_V_END "(..*)" SSL_SERVER_V_END=$1
SetEnvIf SSL_SERVER_A_SIG "(..*)" SSL_SERVER_A_SIG=$1
SetEnvIf SSL_SERVER_A_KEY "(..*)" SSL_SERVER_A_KEY=$1
SetEnvIf SSL_SERVER_CERT "(..*)" SSL_SERVER_CERT=$1
#RequestHeader unset HTTPS
RequestHeader unset SSL_PROTOCOL
RequestHeader unset SSL_SESSION_ID
RequestHeader unset SSL_CIPHER
RequestHeader unset SSL_CIPHER_EXPORT
15
RequestHeader unset SSL_CIPHER_USEKEYSIZE
RequestHeader unset SSL_CIPHER_ALGKEYSIZE
RequestHeader unset SSL_VERSION_INTERFACE
RequestHeader unset SSL_VERSION_LIBRARY
RequestHeader unset SSL_CLIENT_M_VERSION
RequestHeader unset SSL_CLIENT_M_SERIAL
RequestHeader unset SSL_CLIENT_S_DN
RequestHeader unset SSL_CLIENT_S_DN_x509
RequestHeader unset SSL_CLIENT_I_DN
RequestHeader unset SSL_CLIENT_I_DN_x509
RequestHeader unset SSL_CLIENT_V_START
RequestHeader unset SSL_CLIENT_V_END
RequestHeader unset SSL_CLIENT_A_SIG
RequestHeader unset SSL_CLIENT_A_KEY
RequestHeader unset SSL_CLIENT_CERT
RequestHeader unset SSL_CLIENT_CERT_CHAINn
RequestHeader unset SSL_CLIENT_VERIFY
RequestHeader unset SSL_SERVER_M_VERSION
RequestHeader unset SSL_SERVER_M_SERIAL
RequestHeader unset SSL_SERVER_S_DN
RequestHeader unset SSL_SERVER_S_DN_x509
RequestHeader unset SSL_SERVER_I_DN
RequestHeader unset SSL_SERVER_I_DN_x509
RequestHeader unset SSL_SERVER_V_START
RequestHeader unset SSL_SERVER_V_END
RequestHeader unset SSL_SERVER_A_SIG
RequestHeader unset SSL_SERVER_A_KEY
RequestHeader unset SSL_SERVER_CERT
EOF
Known problem: Currently in multiline texts newlines are replaced with spaces, because of that
SSL_*_CERT will not work.
4.2 Weblogic
Configuration for the Weblogic is the same as you would be using mod_weblogic or F5, you have to set
checkbox in Weblogic Console to Client Cert Proxy Enabled, or in deployment's weblogic.xml enable
tag client-cert-proxy-enabled.
•
•
16
http://www.google.com/search?q=Client+Cert+Proxy+Enabled+weblogic
http://www.google.com/search?q=client-cert-proxy-enabled
5. Configuration recommendations/notes
5.1 Apache
1. Keep in mind, that Apache2 configuration is read linearly. (If you first do ProxyPass and then
set some headers or do some cheks, user will be already at proxyed.)
2. In configurations don't use RewriteRule /something /otherthing [QSA,P] or the webserver will
make queries to its DNS resolve, but use [QSA,PT] (passthrough, not proxy) – also using P flag
is security hole through what your internal or other websites can be attacked (it acts as
anonymous proxy).
5.2 Loadbalancing
1. Use sticky sessions if you are not certain, that your applications fully and correctly support fail
over – if one server should die, then only users from that server are directed to other server.
2. If your backendserver uses mod-itk (or for some other reason) can't handle multiple requests in
same TCP session to different virtualhosts, use session terminating for that host.
(http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxypass search: “disablereuse”)
17
6. Links
1. http://httpd.apache.org/docs/current/mod/mod_proxy.html
2. http://httpd.apache.org/docs/current/mod/mod_proxy_balancer.html
18