Scaling Progress
OpenEdge Appservers
Syed Irfan Pasha
Principal QA Engineer
Progress Software
Michael Jackson Dies
and Twitter Fries
Twitter’s – Fail Whale
3
© 2014 Progress Software Corporation. All rights reserved.
Twitter’s – Scalability Problem
 Takeaways from the story of Twitter
• Understand your software’s behavior and foresee its usage
• Have a mechanism to identify Production issues and raise an alarm smartly
before your customers do it
• Scalability acts as a key measure for the Quality of a Software
4
© 2014 Progress Software Corporation. All rights reserved.
Agenda
 OpenEdge Applications and Focus Areas for Scalability
 Optimization Techniques for Appserver
 Tomcat Web Server
• Tuning Techniques
• Different Types of Tomcat Connectors
 Scaling OpenEdge Applications With Load Balancing
 Q&A
5
© 2014 Progress Software Corporation. All rights reserved.
Focus for Scalability in Progress OpenEdge Components
Presentation Layer
OpenEdge Clients Open Clients Rollbase Mobile
Application Server
OpenEdge Appserver
Database Server
OpenEdge RDBMS
6
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Replication
Enterprise Services
JMS
Web Services
ESB
Where to focus
on scalability ?
OpenEdge Appservers and Applications
2
Web Server
(OpenEdge Services/
Applications)
OpenEdge
3
Pacific Appserver
for OE
Non-OpenEdge
Clients
Classic Appserver
1
7
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Appservers
Key Areas to be Focused for Scalability
Classic Appserver


Optimization Techniques
Properties for high
concurrency
Webserver(Tomcat)


© 2014 Progress Software Corporation. All rights reserved.
Different Tomcat
Connectors and their uses

Types of load balancing
to be applied for
OpenEdge applications

Methods for applying
load balancing

Best Practices

Most of the OpenEdge Adapters and certified using
standard Java Container, which is Tomcat

Tomcat acts a Java Container for “Pacific AS for OE”
Why Tomcat?
8
Tuning techniques
Load Balancing
Typical Issues With Application Servers
 No response from the server
 Client request takes too long to execute
 Server goes down very often
• Out of Memory
• Out of Heap space
• All kinds of requests goes to the same server
 Web server needs to be restarted as the Application needs to be upgraded
 Cannot execute more than a specific number of requests
9
© 2014 Progress Software Corporation. All rights reserved.
Tuning Classic Appserver
for Better Scalability
Tuning Options of Classic Appserver
ABL
Web
Clients
Open
Clients
 maxClientInstance
• Set to 512 by default
Client
Requests
 Heap Memory of the broker process
• jvmArgs=-Xms<value> -Xmx<value>
 Queuelimit
• (Size of the Payload)/8k =
Queuelimit value
Classic
Appserver
11
© 2014 Progress Software Corporation. All rights reserved.
 Deleting Persistent and Connection
Procedures
OpenEdge Applications and its Environment
Mobile Clients
SOAP Clients
Webspeed Clients
Web Server
Webspeed
Adapter
SOAP
Adapter
AIA
Adapter
REST
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
12
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Appserver
OpenEdge Appserver
Understanding the Tomcat
Web Application Server
Tomcat and its Pieces
Instance of the Tomcat Web
Application Server which
contains one or more services
Server
Service
Routes requests between
Clients and Web Applications
Connectors
Engine
Host1
Host2
WebApp1
WebApp3
WebApp2
WebApp4
Handles communications
between client and the Engine
Highest level of Container
for processing servlets
Web Applications deployed
in multiple hosts
14
© 2014 Progress Software Corporation. All rights reserved.
Tuning Options in Tomcat Webserver
 Tuning Webserver to scale for maximum client requests
15
JVM Tuning
Threads
and its
Properties
Compression
Cache for
Static Files
© 2014 Progress Software Corporation. All rights reserved.
Tuning Options in Tomcat Webserver – JVM Tuning
 The defaults are not for the Production
Environment
JVM Tuning
Threads
and its
Properties
Compression
Cache for
Static Files
 Higher heap memory causes long pauses in
Garbage Collection which will be affected in
the response time
 Know your needs and configure optimistic
memory
 Permgen space – Optimistic approach is to
set almost 25–30% of total Java heap memory
setenv.{bat/sh}
JAVA_OPTS -Xms <value> -Xmx <value> -XX:MaxGCPauseMillis <value>
-XX:+PrintGC -XX:+PrintGCDetails
16
© 2014 Progress Software Corporation. All rights reserved.
Tuning Options in Tomcat Webserver – Threads
Concurrent User Requests
AcceptCount
∝
Kernel’s listen queue
Compression
Cache for
Static Files
Accept Count
Further client
requests will
be refused
OS buffers TCP
(kernel’s listen queue)
minSpareThreads(10)
maxThreads(50)
Threads Configuration
17
JVM Tuning
Threads
and its
Properties
© 2014 Progress Software Corporation. All rights reserved.
+ acceptCount(50)
Maxthreads : 200 – 800
Optimize based on your Application
Tuning Options in Tomcat Webserver – Monitoring Threads
http://<hostname>:<port>/manager
Server Status
Properties can be configured in server.xml as below
<Connector port="8080“ connectionTimeout="20000“ maxThreads="500“ />
18
© 2014 Progress Software Corporation. All rights reserved.
Tuning Options in Tomcat Webserver – Compression
 Compression
• Reduces the network payload but gobbles the computational power
• Client and the webserver both should be configured for compression
Client Execution Time for the Payload of 10000 Records
Execution Time in ms
18000
16000
Improvement of 110% in execution
time after enabling compression
14000
12000
10000
8000
Compression Disabled
6000
Compression Enabled
4000
2000
0
Create
Update
Read
Operations Performed
19
© 2014 Progress Software Corporation. All rights reserved.
Delete
Network Payload/Bytes
Transferred were
150 times less with
compression
Tuning Options in Tomcat Webserver – Caching Static files
 Allows caching the static files in the browser
with a future expiration date
JVM Tuning
Threads
and its
Properties
Compression
Cache for
Static Files
 Reduces the load of calling the static files
every time from the webserver
 Mostly used for css, images, javascript and
static html
content.xml
<Context cacheMaxSize=”10240” cacheTTL=”60000” cachingAllowed=”true”>
20
© 2014 Progress Software Corporation. All rights reserved.
Tomcat Connectors
Tomcat Connectors – From the Perspective of OpenEdge
Handles communications
client and the Engi
Server
Listens on a defined single
port for connections
Connectors
HTTP & AJP
Ubroker
Engine
Web
Applications
http://localhost:8080
Starts Agents
Handles and manages
communication between
Client and Agent
HTTP - HTTP 1.1 Protocol AJP – Apache Jserv Protocol
22
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Clients
Agents
ABL Business Logic
Tomcat Connectors – HTTP
 BIO Connector
• Executes multiple requests on multiple threads
• Stable performance for applications with moderate
traffic
• Consumes more system resources on a burst of
requests
 NIO Connector
Web Server
HTTP
AJP
HTTP – HTTP 1.1 Protocol
• Handles multiple connections using a couple
of poller threads
• Better managing of threads which shares threads
for multiple concurrent users
 APR Connector
BIO – Blocking Input Output
NIO – Non-blocking Input Output
• Uses Apache Native Runtime library
• Mostly used for SSL as it uses OpenSSL library
which is fastest
23
© 2014 Progress Software Corporation. All rights reserved.
APR – Apache Portable Runtime
Tomcat HTTP Connectors – Which One to Choose
24
Option-1
Option-2
Stability
BIO
APR
SSL
APR
NIO
Low Concurrency
BIO
APR
High Concurrency No Keep-Alive
BIO
APR
High Concurrency Keep-Alive
APR
NIO
© 2014 Progress Software Corporation. All rights reserved.
Tomcat Connectors – AJP
 Tomcat to Apache over a Wire Protocol
 Mostly used to serve tomcat behind
Apache httpd and handle the traffic to
the Tomcat clusters
 Apache serves better static content and
better caching mechanism
 Secures tomcat with its inbuilt security
features and third party libraries
Tomcat Connectors – Why AJP
HTTP
AJP
 Widely used as secure proxy servers
 Examples – modjk, mod_proxy,
mod_http_proxy
25
© 2014 Progress Software Corporation. All rights reserved.
AJP – Apache Jserv Protocol
After Applying Tuning in Appserver and Webserver
 What if your application is getting more and more hits
• Followed all tuning possibilities and best practices
– Optimal JVM options, heap space etc.
– Choosing right Connector for the Application
– Enabled Compression
– Optimal number of threads, connection timeouts
• What might go wrong
– More and more requests conquer your webserver
– Webserver Out of Memory
– Client Requests exceeds the maximum number
of requests
26
© 2014 Progress Software Corporation. All rights reserved.
Is adding more
Computational resources
an ultimate solution
for Scalability?
With all the tuning
options, can we take
advantage of horizontal
scalability?
Applying Scalability with a Load Balancer
Mobile
Clients
SOAP
Clients
Webspeed
Clients
Nameserver(alike)
Web Server Instance 1
Webspeed
Adapter
SOAP
Adapter
AIA
Adapter
Web Server Instance 2
REST
Adapter
Webspeed
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
27
OpenEdge Appserver
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Appserver
SOAP
Adapter
AIA
Adapter
REST
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
OpenEdge Appserver
OpenEdge Appserver
Applying Scalability with a Load Balancer
Mobile
Clients
SOAP
Clients
Webspeed
Clients
Nameserver(alike)
Web Server Instance 1
Webspeed
Adapter
SOAP
Adapter
AIA
Adapter
Web Server Instance 2
REST
Adapter
Webspeed
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
28
OpenEdge Appserver
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Appserver
SOAP
Adapter
AIA
Adapter
REST
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
OpenEdge Appserver
OpenEdge Appserver
Applying Scalability with a Load Balancer
Mobile
Clients
SOAP
Clients
Webspeed
Clients
Load Balancing using AJP
Web Server Instance 1
Webspeed
Adapter
SOAP
Adapter
AIA
Adapter
Web Server Instance 2
REST
Adapter
Webspeed
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
29
OpenEdge Appserver
© 2014 Progress Software Corporation. All rights reserved.
OpenEdge Appserver
SOAP
Adapter
AIA
Adapter
REST
Adapter
OpenEdge Services/Applications
OpenEdge Appserver
OpenEdge Appserver
OpenEdge Appserver
Load Balancing and its Techniques
Load Balancing With Apache
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
Tomcat instance 4
Configurations
Tomcat – Set the AJP Port
Mod_jk – Configure worker.properties with Tomcat instances details
Apache HTTPD – Configure to send the traffic to the balancer
31
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
 Mod_jk Configuration
• Mod_jk acts a load balancer that is configured in the Apache HTTPD Server
• Properties of this connector is configured in worker.properties file
• Structure of the properties file
Properties for
Each Node
– Worker.<tomcat node>.type = ajp13
– Worker.<tomcat-node>.port = <ajp port>
– Worker.<tomcat-node>.host = < hostname of the tomcat-node>
– Worker.list = <list of workers>
Generic Properties
– Worker.balancer.type = lb
– Worker.balancer.balancer_workers = <tomcat-node1>,<tomcat-node2>
– Worker.stat.type = <any defined name for your worker status>
32
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
1st Request
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
Tomcat instance 4
33
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Load Balancer
2nd Request
(mod_jk)
Tomcat instance 3
Tomcat instance 4
34
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
3rd Request
Tomcat instance 4
35
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
Tomcat instance 4
4th Request
36
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Load Balancer
(mod_jk)
X
Tomcat
stance 2
Tomcat instance 3
Tomcat instance 4
37
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
5th Request
Apache HTTPD
Load Balancer
(mod_jk)
X
Tomcat
stance 2
Tomcat instance 3
Tomcat instance 4
38
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Apache
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Load Balancer
(mod_jk)
X
Tomcat
stance 2
Tomcat instance 3
6th Request
Tomcat instance 4
39
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Tomcat Clusters
HTTP Stateless Clients
Basic Load Balancer
Allows a user to route the
traffic across different
nodes
40
© 2014 Progress Software Corporation. All rights reserved.
HTTP Stateful Clients
Sticky Sessions/
Session Affinity
Persists the users session
for a single node
Session Replication
Replicates the session
across multiple nodes of
the load balancer
Load Balancing With Tomcat Clusters
 Session Replication
• Provides High Availability and Fail-over tolerance in a clustered environment
Multicast
Enables all the cluster nodes join the
multicast group and sends the heartbeat
signals for certain intervals
Tomcat
Session
Manager
As part of creating and managing the
session, it replicates the session to all
tomcat nodes
• Enable multicast routing in your server machine
• Enable/Add Cluster configuration in Tomcat
• Add <distributable> tag to the Web Application
41
© 2014 Progress Software Corporation. All rights reserved.
Configuring Session Replication
Load Balancing With Tomcat Clusters
Session created
for 1st Request
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Session
Context
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
Tomcat instance 4
1st Request
42
© 2014 Progress Software Corporation. All rights reserved.
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
Session
Context
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Load Balancer
(mod_jk)
Tomcat instance 3
Tomcat instance 4
1st Request
43
© 2014 Progress Software Corporation. All rights reserved.
Replicating
the session
across
multiple
instances
using Apache
Tribes Group
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Session
Context
Session
Context
Load Balancer
(mod_jk)
Tomcat instance 3
Session
Context
Tomcat instance 4
1st Request
44
© 2014 Progress Software Corporation. All rights reserved.
Session
Context
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Session
Context
Session
Context
Load Balancer
(mod_jk)
Tomcat instance 3
Session
Context
Tomcat instance 4
2nd Request
45
© 2014 Progress Software Corporation. All rights reserved.
Session
Context
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Session
Context
Session
Context
Load Balancer
(mod_jk)
Tomcat instance 3
Session
Context
Tomcat instance 4
3rd Request
46
© 2014 Progress Software Corporation. All rights reserved.
Session
Context
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
Tomcat instance 1
Apache HTTPD
Tomcat instance 2
Session
Context
Session
Context
Load Balancer
(mod_jk)
Tomcat instance 3
Session
Context
Tomcat instance 4
4th Request
47
© 2014 Progress Software Corporation. All rights reserved.
Session
Context
Load Balancing With Tomcat Clusters
http://<hostname>:<apacheport>/myApp
X
Tomcat
Apache HTTPD
tance 1
Tomcat instance 2
Session
Context
Session
Context
Load Balancer
(mod_jk)
Tomcat instance 3
Session
Context
Tomcat instance 4
4th Request
48
© 2014 Progress Software Corporation. All rights reserved.
Session
Context
Summary
1. Tuning Classic Appserver
1. Applying Load balancing for OpenEdge Applications
2. Properties and best practices
2. Stateless and Stateful Requests
Classic
Appserver
Load
Balancing
Scalability
Tomcat
Web Server
1. Tuning Tomcat Webserver
2. Choosing the right Connector
49
© 2014 Progress Software Corporation. All rights reserved.
References
50

http://tomcat.apache.org/tomcat-7.0doc/config/http.html

http://tomcatexpert.com/sites/default/files/PerformanceT
uningApacheTomcat-Part2.pdf

http://tomcat.apache.org/connectorsdoc/generic_howto/loadbalancers.html
© 2014 Progress Software Corporation. All rights reserved.
Q&A
Want to Learn More About OpenEdge 11?
 Role-based learning paths are available for OpenEdge 11
 Each course is available as Instructor-led training or eLearning
 Instructor-led training:
• $500 per student per day
• https://www.progress.com/support-and-services/education/instructor-led-training
 eLearning:
• Via the Progress Education Community (https://wbt.progress.com):
• OpenEdge Developer Catalog: $1500 per user per year
• OpenEdge Administrator Catalog: $900 per user per year
 User Assistance videos: https://www.progress.com/products/pacific/help/openedge
26
© 2014 Progress Software Corporation. All rights reserved.
Visit the Resource Portal
 Get session details & presentation downloads
 Complete a survey
 Access the latest Progress product literature
www.progress.com/exchange2014