1. No category

M/Gateway Developments Ltd
ewd-federator
Reference Guide
M/Gateway Developments Ltd
http://www.mgateway.com
Copyright ©2015, M/Gateway Developments Ltd. All Rights Reserved
Table of Contents
Introduction
1
Background
1
Background and Architecture
2
Overview
2
ewd-federator Architecture
2
Choice of Database for ewd-federator
5
Installing, Configuring & Running
6
Initial Installation Steps
6
Installing ewd-federator
6
Configuring ewd-federator
6
Caché:
7
GT.M:
7
Starting ewd-federator
8
Testing ewd-federator
8
Stopping ewd-federator
9
ewd-federator Configuration
10
Configuration Basics 10
The server Object
12
Re-routing to EWD.js Servers
12
The service Object
ewd-federator Reference Guide (Build 0.13)
10
i
Copyright ©2015, M/Gateway Developments Ltd. All Rights Reserved
EWD.js URL Mapping
12
Signing the EWD.js Request
15
Grouping Destinations
16
Defining a Destination Group
16
Using a Group Destination
17
Customising ewd-federator
19
Why Customise ewd-federator?
19
Creating an Extension Module
19
Defining and Handling Events in your Extension Module
20
Handling the request Event
20
Manually Sending a Request to a Destination
23
Defining Custom Response Events
24
The ScratchPad Object
26
Response Event Handlers for Destination Groups
26
ewd-federator Reference Guide (Build 0.13)
ii
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Introduction
Background
ewd-federator is an Apache 2-licensed Open Source Node.js-based customisable platform that
allows you to create a federation of servers, each of which provide access via HTTP-based or RESTbased Web Services. Key features are as follows:
• ewd-federator is accessed as a REST-ful service. ewd-federator makes use of the Node.js Restify
module.
• incoming REST requests can be re-routed to one or more of the federated Web Service servers.
Routing to more than one servers simultaneously is achieved by defining groups of servers. You
may define as many groups as you like, and groups can overlap. Groups can also be defined
dynamically.
• responses from a group of servers are automatically aggregated by ewd-federator
• if one or more of the federated servers is running EWD.js, ewd-federator will correctly sign the
requests (provided it is registered for access by the EWD.js server). ewd-federator can be used with
EWD.js to provide a fully REST-ful interface to your EWD.js Web Services.
• you can optionally customise ewd-federator by adding your own Node.js module that allows you to
intercept and modify/manipulate not only incoming REST requests to ewd-federator, but also
incoming responses from the back-end federated servers.
• ewd-federator uses the same master/child-process architecture as EWD.js, and similarly makes use
of a persistent JSON database (Caché, GT.M, GlobalsDB). The purpose of this integrated database
is to allow you to cache or store custom information, such as Single Sign On (SSO) authentication
tokens. How you make use of this database is entirely up to you. It is not used or required by ewdfederator itself.
ewd-federator is therefore a lightweight, but very flexible and powerful federation platform.
This document explains how to install, configure, run and customise ewd-federator.
ewd-federator Reference Guide (Build 0.13)
1
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Background and Architecture
Overview
ewd-federator acts as an intermediate tier between REST clients and a federation of back-end Web Service and REST
servers (which may include EWD.js servers), as summarised below:
You can federate as many back-end servers as you wish, and they can be any combination of HTTP-based Web Service
servers, REST service servers and EWD.js servers.
ewd-federator would normally be installed on its own server. It runs as a set of Node.js processes (see below), and requires
the use of a database that you may use for local cacheing and data persistence within your customisation logic. Currently 3
databases are supported: Caché, GlobalsDB and GT.M, all of which behave as persistent JSON storage engines.
ewd-federator Architecture
ewd-federator’s architecture is similar to that used by EWD.js, which best exploits the behaviour and characteristics of the
Node.js interface to the databases it uses. All incoming REST requests are handled by a fully asynchronous master Node.js
process. The main work, however, is carried out by a pre-forked, constantly-running pool of child processes, each of which
has an in-process, synchronous connection to the selected database. This is summarised in the diagram below:
ewd-federator Reference Guide (Build 0.13)
2
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
You can configure the size of the child process pool in the ewd-federator start-up file, and you can modify and control the
size of the pool when ewd-federator is running.
Communication with the back-end Web Service and REST servers occurs within the ewd-federator module that runs in each
child process (as summarised below):
The master process uses the Restify module to provide the REST server interface through which REST clients communicate.
Whenever an incoming REST request is received, it is placed in a queue which is processed immediately. If a child process
is available, the request is passed to that child process which is removed from the available pool: a child process therefore
only handles a single request at a time.
The ewd-federator child process module re-routes and re-writes the REST request and forwards it to one or more back-end
servers. If the back-end server is an EWD.js server, ewd-federator uses the ewdliteclient module to digitally sign the request:
ewd-federator Reference Guide (Build 0.13)
3
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Responses from the back-end server(s) are returned to the child process and returned to the master process which forwards
it to the awaiting REST client. As soon as the child process has returned the response to the master process, it is returned
to the available pool.
Re-routing of incoming REST requests is performed by default by a JSON object that maps the destination (as defined in the
incoming REST request) to a physical back-end server’s host address and port. Re-writing of the REST request to meet the
characteristics of the remote server’s web service interface is carried out, as far as is possible, with a JSON object map.
For more complex re-routing and re-writing, and in order to create complex “dances” between the remote servers (eg for
Single Sign On logic), ewd-federator allows you to define your own extension Node.js module in which you can intercept the
incoming requests and returned responses and modify them as required. For example, on receipt of a response from a
remote server, your extension module can send out one or more requests to back-end servers instead of allowing the
response to be returned to the original REST client. Once your extension module’s logic determines that the information has
been assembled to construct the required federated response, you can then return it to the awaiting REST client. Along the
way, your extension module logic can save information to the database and/or retrieve information from the database: this is
useful as a means of cacheing information to avoid the need for repeated “dances” between the back-end servers, and also
allows authentication tokens to be stored on a permanent or temporary basis. The diagram below summarises the
customised flow:
ewd-federator Reference Guide (Build 0.13)
4
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Choice of Database for ewd-federator
The databases currently supported by ewd-federator are Caché, GlobalsDB and GT.M. ewd-federator uses the globals.js
module to abstract all three databases to behave as persistent JSON stores.
Due to the currently available versions of the interfaces between these databases and Node.js, ewd-federator is limited to
running with Node.js version 0.10.x. This will change in due course.
ewd-federator itself does not require the use of a database for its own default behaviour, but you will almost certainly want to
customise ewd-federator, The included database allows you to easily persist and cache information to support your
federation logic as JSON documents.
ewd-federator can be installed on Windows, Linux or Mac OS X. Caché or GlobalsDB can be used as the database on any
of these operating systems. GT.M can only be used on Linux systems.
If you need to choose which one to use, bear the following in mind:
• GlobalsDB is a free, but closed source product. However, it has no limitations in terms of its use or re-distribution. It is
essentially the core database engine from Caché. Versions are available for Windows, Mac OS X and Linux. No technical
support or maintenance is available for GlobalsDB from InterSystems: it is provided for use on an “as-is” basis. GlobalsDB
is the quickest and simplest of the databases to install, and is probably the best option to use if you are new to ewdfederator.
• GT.M is a free, Open Source, industrial-strength product. It is limited to Linux systems.
• Caché is a proprietary and commercially-licensed industrial-strength product, with versions available for Windows, Mac
OS X and Linux. ewd-federator only requires its core database engine, but if you’re already a Caché user, you may prefer
to use it. Caché also provides excellent built-in solutions for scaling out ewd-federator instances where high scalability
and availability is essential.
For more details about these databases, see:
• GlobalsDB: http://www.globalsdb.org/
• GT.M: http://www.fisglobal.com/products-technologyplatforms-gtm
• Caché: http://www.intersystems.com/cache/index.html
All three databases are extremely fast, and both Caché and GT.M can scale to extremely large enterprise levels. The Node.js
interface for GlobalsDB and Caché currently has about twice the speed of performance as the NodeM interface for GT.M.
Note that when you use ewd-federator, all three Mumps databases appear to be identical in terms of how you access and
manipulate data from within your customisation logic. The only difference you’ll need to be aware of is a small difference in
the configuration settings within the startup file you’ll use for the ewd-federator module (see later). Otherwise, ewd-federator
customisation logic that you develop for, say, Caché, should normally work without any coding changes with a GT.M
database.
ewd-federator Reference Guide (Build 0.13)
5
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Installing, Configuring & Running
Initial Installation Steps
• Install Node.js: See http://nodejs.org/ ewd-federator can currently be used with version 0.10.x of Node.js. Don’t install
the latest 0.12.x version.
• Install your chosen database: see the relevant web-site for details (the relevant URLs are listed in the previous chapter)
A very quick and simple way to get started with ewd-federator, particularly if you’re not familiar with these databases is to
use the EWD.js installers that are documented in Appendix I and II in the EWD.js Reference Guide. These will install Node.js,
GT.M or GlobalsDB and EWD.js on a Ubuntu Linux platform (physical machine, VM, Amazon EC2 instance). You can simply
ignore EWD.js when the installation is complete.
The EWD.js Reference Guide also provides further information about the Node.js interfaces for the databases used by ewdfederator.
Installing ewd-federator
Simply navigate to the correct directory where you want ewd-federator to be installed and use NPM, eg:
cd ~/ewdjs
npm install ewd-federator
You may see some errors reported, but you can usually ignore these as they relate to some of the Restify dependencies that
aren’t actually used by ewd-federator.
Configuring ewd-federator
Next you need to configure ewd-federator. This is done in the JavaScript file that you’ll use to start ewd-federator. Initially
we’ll just create a simple “hello world” setup that will make use of a test REST server.
Make sure you’re in the same directory as when you installed ewd-federator, eg in the example above ~/ewdjs.
Create and save your startup file as shown below. You can name your startup file anything you like, but it should have a file
extension of .js. For example, save it as startFederator.js:
ewd-federator Reference Guide (Build 0.13)
6
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
var federator = require('ewd-federator');
var params = {
restPort: 8081,
poolSize: 2,
traceLevel: 3,
};
database: {
type: 'globals',
path: '/home/ubuntu/globalsdb/mgr',
},
server: {
placeholder: {
host: 'jsonplaceholder.typicode.com',
port: 80,
ssl: false,
ewdjs: false
}
}
federator.start(params);
This example above assumes we’re using GlobalsDB as the database, and that GlobalsDB Manager directory path is:
/home/ubuntu/globalsdb/mgr
You should modify this path to match where you’ve installed GlobalsDB.
If you’re using Caché or GT.M you’ll need to modify the database section of the params object, for example:
Caché:
database: {
type: 'cache',
path: '/usr/local/cachesys/mgr',
username: 'demoUser',
password: 'xyz1234',
namespace: 'USER'
},
Modify the path to match the one where Caché’s mgr directory is installed. Note that the cache.node interface needs to log
into Caché using a valid username and password, and note that you also must specify the namespace into which it will
connect.
GT.M:
database: {
type: 'gtm'
},
This assumes that the Node.js interface for GT.M (NodeM) has been installed using NPM - see https://github.com/dlwicksell/
nodem
The startup file has configured a single back-end server that we have given a destination of placeholder, and which points to
a host name jsonplaceholder.typicode.com. This is a useful, publicly-available demonstration REST Server.
ewd-federator Reference Guide (Build 0.13)
7
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Starting ewd-federator
Once we have a startup file defined, we can fire it up. Once again, do this from the directory in which you installed ewdfederator (which is also the directory where your start-up file should reside):
cd ~/ewdjs
node startFederator
Provided that everything is correctly installed and that the configuration paths etc are correct, ewd-federator should start,
and will then be sitting waiting to receive incoming REST requests. For example, at the end of the startup trace report, it
should say something like this:
requireAndWatch: /home/ubuntu/ewd-test/node_modules/globalIndexer.js loaded by process
2758
** Global Indexer loaded: /home/ubuntu/ewd-test/node_modules/globalIndexer.js
child process returned response
{"pid":2758,"type":"initialise","release":true,"empty":true,"processNo":1}
process 2758 returned to available pool; type=initialise
******************************************************
*******
ewd-federator is Ready
***********
******************************************************
Started: Tue, 17 Mar 2015 15:18:55 GMT
Listening on port 8081
*** checking child process pool at Tue, 17 Mar 2015 15:18:57 GMT
pid: 2754 available for 1000
pid: 2758 available for 924
Testing ewd-federator
Although you can send simple GET requests from any browser, you probably want to use a proper REST client. For
example, if you use the Chrome browser, try installing the Advanced REST Client extension.
Try the following GET request, substituting the IP address or domain name of the server on which you’re running ewdfederator:
http://xx.xx.xxx.xxx:8081/placeholder/posts/1
You should get back a response similar to this:
ewd-federator Reference Guide (Build 0.13)
8
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
{
userId: 1
id: 1
title: "sunt aut facere repellat provident occaecati excepturi optio reprehenderit"
body: "quia et suscipit suscipit recusandae consequuntur expedita et cum reprehenderit molestiae ut ut quas totam
nostrum rerum est autem sunt rem eveniet architecto"
}
What’s happened is, of course, that ewd-federator has relayed the REST request to the placeholder server. Try sending the
equivalent request directly to this server:
http://jsonplaceholder.typicode.com/posts/1
If this example worked for you, you now have ewd-federator up and working.
Stopping ewd-federator
If you’re running ewd-federator as a foreground process, you can stop it simply by typing CTRL & C within the process
window.
If you’re running it as a background service, send a SIGTERM message to it (eg using kill {process id})
ewd-federator Reference Guide (Build 0.13)
9
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
ewd-federator Configuration
Configuration Basics
ewd-federator is configured by a JSON object that is used as the argument for the ewd-federator start() function. A simple
example was shown in the previous chapter.
The configuration object has the following mandatory properties:
• restPort: the port on which ewd-federator listens for incoming REST requests
• poolSize: the number of child processes that are pre-forked when ewd-federator is started
• traceLevel: the level of verbosity of logging information shown in the ewd-federator console.log (0 = none, 3 = maximum)
• timeout: the number of milliseconds ewd-federator will wait for a response from a remote server, before giving up and
returning a 408 error (default 120,000ms)
• database: an object that defines the type of database and its configuration parameters (see previous chapter)
• server: an object defining the destination alias names and associated connection details of the back-end servers
accessible via ewd-federator. You can define as many servers as you like
There are a number of additional, optional properties:
• service: an object defining mappings from REST paths to corresponding EWD.js module and function (service) names
• destination: an object defining group alias names that represent aggregations of back-end servers, eg _florida.
• extensionModule: the name of an optional extension module in which you can customise the behaviour of ewd-federator
• password: the management password for system-level access to ewd-federator when it is running (see later).
The server Object
The server object allows you to define the connection details and other key characteristics of each back-end server within
your federation. Each server is given an alias (otherwise known as a destination name). Against each alias is an object that
defines the specific server, ie:
ewd-federator Reference Guide (Build 0.13)
10
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
var params = {
...
server: {
<alias1>: {
host: <host name or IP address>,
port: <port>,
ssl: <true | false>,
ewdjs: <true | false>
},
<alias2>: {
host: <host name or IP address>,
port: <port>,
ssl: <true | false>,
ewdjs: <true | false>
},
...etc
}
};
The alias name can be any valid JSON property name. This is the name that will be used as the destination to which REST
requests are routed. The way this works is very simple: when sending a REST request to ewd-federator, the first part of the
URL path must correspond to a configured server alias name. So in the example from the previous chapter, a server named
placeholder was defined:
server: {
placeholder: {
host: 'jsonplaceholder.typicode.com',
port: 80,
ssl: false,
ewdjs: false
}
}
We were then able to route REST requests to it via ewd-federator using a URL that contained /placeholder as the first part of
its path, eg:
http://localhost:8081/placeholder/posts/1
Each individual server is defined by a sub-object with the following properties:
• host: the IP address or domain name of the remote server
• port: the port on which the remote server listens for REST / Web Service requests
• ssl: true | false - defines whether or not the remote server requests must be sent in the clear or over SSL/TLS
• ewdjs: true | false (default if not specified = true) - if true, the remote server is an EWD.js server, and ewd-federator must
digitally sign the requests that are forwarded to it. In order to do this, the following two properties must be defined:
• accessId: the accessId assigned by the EWD.js server for use by this instance of the ewd-federator.
• secretKey: the corresponding secret key, assigned by the EWD.js server. This is used to digitally sign all request to the
EWD.js server, using the HMAC-SHA256 cipher.
Note: accessId and secretKey can be omitted if the ewdjs property is set to false.
So in the example above, the REST request took the first part of the URL path (ie placeholder) and looked that up in the
server object. It then had the information needed to route the request to the physical host (jsonplaceholder.typicode.com).
The first part of the path was removed from the URL sent to the remote site (ie the URL path sent was posts/1).
ewd-federator Reference Guide (Build 0.13)
11
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Re-routing to EWD.js Servers
When a REST request is forwarded to an EWD.js server, ewd-federator takes a couple of extra steps:
• the incoming REST URL is mapped to a corresponding EWD.js HTTP Web Service URL using the service configuration
object.
• the HTTP request that is forwarded to the EWD.js is digitally signed using the accessId and secretKey that was allocated
to the ewd-federator instance by that particular EWD.js server. Note also that the ewd-federator instance can only send
requests that access the modules that the EWD.js server has permitted it to use. The accessId, secretKey and application
permissions are granted on the EWD.js server by using a security option within the ewdMonitor application. Consult the
EWD.js Reference Guide for details.
The service Object
The service configuration sub-object defines the mappings between incoming REST URLs and remote EWD.js web service
modules, eg:
var params = {
...
service: {
<module alias 1>: {
module: <EWD.js module name>,
service: <function name within module>,
contentType: 'application/json'
},
<module alias 2>: {
module: <EWD.js module name>,
service: <function name within module>,
contentType: 'application/json'
},
...etc
}
};
EWD.js URL Mapping
The structure of EWD.js-mapped REST requests is as follows:
http://localhost:8081/<destination>/<module alias name>/<operation name> ....
Any further sub-paths or querystring name/value pairs that follow the operation name are specific to the operation name.
The destination mapping has been described in the previous section. The second part of the URL path (ie the module alias
name) is looked up in the service object. If it can be found, then the corresponding EWD.js Web Service URL is constructed.
For example, if the server and service objects contained the following:
ewd-federator Reference Guide (Build 0.13)
12
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
var params = {
...
service: {
demo: {
module: myDemo,
service: parse,
contentType: 'application/json'
}
},
server: {
london: {
host: '99.9.9.9',
port: 8080,
ssl: false,
ewdjs: true,
secretKey: 'rob123',
accessId: 'rob'
}
}
};
then the following incoming REST request:
http://localhost:8081/london/demo/helloworld?id=1234
would be re-written as:
http://99.9.9.9:8080/json/myDemo/parse?id=1234&rest_path=demo/helloworld&...etc
Notice that the operation name in the REST request (helloworld) is not directly mapped to the EWD.js URL, but is conveyed
within the value of the rest_path name/value pair.
Provided this request is correctly digitally signed (see later), the remote EWD.js server will invoke the myDemo module and
execute the parse() function within it. The parse() function should parse/inspect the rest_path name/value pair to determine
which function within the myDemo module is actually required.
In fact, all the relevant information about the originating REST request is forwarded to the EWD.js server as name value pairs
attached to the URL query string. The full list is as follows:
• rest_url: the full original REST URL, eg http://localhost:8081/london/demo/helloworld?id=123
• rest_path: the path part of the original REST URL, with the destination removed, eg /demo/helloworld
• rest_auth: if an Authorization HTTP Header was included with the request, its value is included here
• rest_method: the original REST HTTP method (eg GET, POST, PUT etc)
The parse() method (or whatever function you specify for processing all REST requests for a particular operation within an
EWD.js module) may need to use several or all of these name/value pairs in order to determine what to do. For example,
suppose on the EWD.js server you have three operations defined for a module (aka application) named Scheduling:
• getAppointments
• setAppointment
ewd-federator Reference Guide (Build 0.13)
13
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
• deleteAppointment
Expressed via REST requests sent to ewd-federator, these would probably be expressed using a single resource name Appointments - and the appropriate HTTP verb, ie:
• GET /Scheduling/Appointments
• PUT /Scheduling/Appointments
• DELETE /Scheduling/Appointments
So, for example, the REST GET /Scheduling/Appointments URL might look like this:
GET http://localhost:8081/london/Scheduling/Appointments?id=1234&date=today
As a result of the service object mapping, this might get re-written to the following EWD.js URL that is sent to the london
server:
http://99.9.9.9:8080/json/scheduling/parse?id=1234&date=today&
rest_path=Scheduling/Appointments&rest_method=GET&...etc
The scheduling.js module on the EWD.js server might then look something like this:
var operations = {
Appointments: {
GET: function(ewd) {
// code to get appointments
var id = ewd.query.id;
var date = ewd.query.date;
var appointments = {};
//...etc
return appointments;
},
PUT: function(ewd) {
// code to set an appointment
},
DELETE: function(ewd) {
// code to delete an appointment
}
}
}
module.exports: {
parse: function(ewd) {
var resource = ewd.query.rest_path.split('/')[1];
var method = ewd.query.rest_method;
if (operations[resource] && operations[resource][method]) {
return operations[resource][method](ewd);
}
else {
return {error: 'Invalid request'};
}
}
};
ewd-federator Reference Guide (Build 0.13)
14
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Signing the EWD.js Request
The final step performed by ewd-federator is to digitally sign the re-written EWD.js request using the accessId and SecretKey
values that are held in the server object for the specified destination EWD.js server. ewd-federator uses the ewdliteclient
module to perform this task.
The request is then dispatched to the remote EWD.js server. Provided the accessId and secretKey are correct and provided
the EWD.js server allows the ewd-federator instance to invoke the specified module (myDemo.js in the example above), then
the request will be accepted and invoked on the EWD.js server.
For full details on how to define and write Web Services on EWD.js systems, see the chapter “Web Service Interface” in the
EWD.js Reference Manual.
ewd-federator Reference Guide (Build 0.13)
15
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Grouping Destinations
Defining a Destination Group
So far, you’ve seen how individual destinations can be defined and mapped to the connection details of a specific back-end
server. However, ewd-federator gives you much more flexibility by allowing you to define groups of destinations. For
example, suppose you had the following destinations defined:
• crawley
• reigate
• guildford
• leeds
• york
Suppose you want to get all of a particular set of results from the servers in Surrey (ie crawley, reigate and guildford), you can
define a Group Destination named surrey, by using the destinations configuration object. Provided you’ve already defined
the three servers above in the servers object, you simply add the following to the configuration object:
var params = {
...
destination: {
Surrey: [
'crawley',
'reigate',
'guildford'
],
Yorkshire: [
'leeds',
'york'
],
...etc
}
};
You can define as many group destinations as you want, and individual servers can appear in as many group destinations as
you want. Note that each member of a group must be an alias defined in the server object. You can’t use a group
destination name as a member of another group definition array. For example, to define a group named England that
included all the sites in England, you’d have to define it as follows:
ewd-federator Reference Guide (Build 0.13)
16
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
var params = {
...
destination: {
Surrey: [
'crawley',
'reigate',
'guildford'
],
Yorkshire: [
'leeds',
'york'
],
England: [
'crawley',
'reigate',
'guildford',
'leeds',
'york'
],
}
};
You cannot do this, however:
var params = {
...
destination: {
Surrey: [
'crawley',
'reigate',
'guildford'
],
Yorkshire: [
'leeds',
'york'
],
England: [
'Surrey',
'Yorkshire'
],
}
};
Using a Group Destination
if you specify the destination as /Surrey in the REST request that you send to ewd-federator, it will send identical copies of
the mapped request to the servers with aliases of crawley, reigate and guildford. These requests are sent out in parallel, at
the same time and asynchronously. For example, your REST request might look like this:
GET http://localhost:8081/Surrey/Warehouse/StockReport?item=widgets
ewd-federator will automatically aggregate the responses that are returned from the three servers in Surrey. When it has
received all the expected responses, it returns the aggregated JSON response to the waiting REST client. So, for example,
the aggregated response might look like this:
ewd-federator Reference Guide (Build 0.13)
17
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
{
}
"crawley": {
"stockLevel": 23,
"orderNo": "AB13"
},
"reigate": {
"stockLevel": 0,
"orderNo": "RBV531A"
},
{"guildford": {
"stockLevel": 143,
"orderNo": "GU7FGY5"
}
Note that if an error occurs on one of the servers within the group, eg if it times out, the error response is added as a JSON
object for that server to the aggregated response.
ewd-federator Reference Guide (Build 0.13)
18
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
Customising ewd-federator
Why Customise ewd-federator?
ewd-federator’s default behaviour and functionality that has been described so far may meet all your requirements - in which
case that is great: read no further!
However, you will probably find situations where you’d like to take control over the mapping between the incoming REST
requests and the outgoing requests sent to some or all of the back-end servers. Perhaps you want to keep an audit trail on
ewd-federator of the activity, or logging information. Perhaps you want to implement a Single Sign On mechanism where
one back-end server is the the main authentication server which returns a token that is then used for signing onto other
back-end servers. Perhaps you want to intercept responses from the back-end servers and, dependent on the results, fire
off some more requests to another set of back-end servers before creating your own customised aggregated response.
ewd-federator lets you take complete control and perform any level of complexity of “dance” that you want to create
between sending in a REST request to ewd-federator and ultimately getting back a response.
The key to customising ewd-federator is for you to define an extension module, and instruct ewd-federator to use it via the
startup configuration object.
Creating an Extension Module
An ewd-federator extension module is simply a standard Node.js module file, written in JavaScript. It has the following
signature / pattern:
var events = require('events');
var federator = new events.EventEmitter();
module.exports = {
event: federator,
events: [
// array of names of events to be handled
]
};
// Event Handler functions
....
// Event Workflow
....
You typically save this file in the node_modules directory that exists below the directory into which you originally installed
ewd-federator. Give it any name you like, eg: ~/ewdjs/node_modules/myFederator.js
Next, add it to the configuration object in your ewd-federator startup file. Use the filename you’ve used, but remove the .js
file extension, eg:
ewd-federator Reference Guide (Build 0.13)
19
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
var params = {
...
extensionModule: 'myFederator',
...
};
Stop and restart ewd-federator using your start-up file. Provided you don’t have any syntax errors in your extension module
file, ewd-federator should start successfully.
Defining and Handling Events in your Extension Module
Essentially an ewd-federator extension module defines a list of named events that are exposed to the main ewd-federator
child process module so that it can trigger them when required for you.
There are two types of events that you can define:
• request: this fires whenever ewd-federator receives an incoming REST request, allowing you to handle and, if you wish,
modify ewd-federator’s behaviour when receiving that request
• your own named custom response events. You assign a name of your choice to each request that your extension
module sends to a back-end server or group destination, and when the response is returned, your corresponding custom
event handler fires.
Handling the request Event
The request event is a built-in event that ewd-federator triggers every time it receives an incoming REST request.
Below is a simple example of how to define your own handler for this event within your extension module:
var events = require('events');
var federator = new events.EventEmitter();
module.exports = {
event: federator,
events: [
'request'
]
};
// Event Handler functions
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
ewd.util.sendRequest(request.destination, request);
}
// Event Workflow
federator.on('request', onRequest);
Stop and restart ewd-federator to ensure that this module is loaded (subsequently, ewd-federator should automatically
reload your extension module every time you edit it)
If you send a REST request into the federator, you should see something like this appear in amongst the console.log traces
generated within the ewd-federator terminal window:
ewd-federator Reference Guide (Build 0.13)
20
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
onRequest: request = {
"type": "posts/1",
"path": "posts/1",
"method": "GET",
"contentType": "application/json",
"nvps": {
"rest_url": "http://99.9.9.9:8081/placeholder/posts/1",
"rest_path": "posts/1",
"rest_method": "GET"
},
"post_data": {},
"destination": "placeholder"
}
Otherwise, you should see the normal response coming back from the remote site.
What happened was that the ewd-federator child process triggered your event handler via your workflow function:
federator.on('request', onRequest);
It knew to invoke this because you’d registered this event in your extension module’s module.exports:
module.exports = {
event: federator,
events: [
'request'
]
};
As a result, your handler function - onRequest() - fired.
Your handler function should define two arguments:
• request
• ewd
As you can see from the console.log trace created by your handler function, the request object contains all the information
that you need in order to manipulate and/or deal with the incoming request. The ewd object provides you with a number of
key objects used by ewd-federator in addition to a number of functions that you’ll want to use within your customisation
logic.
Notice the second line in your handler function:
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
ewd.util.sendRequest(request.destination, request);
}
If you intercept the request event, it becomes your responsibility to handle what happens next, otherwise ewd-federator will
just sit waiting. In this case we’re going to forward the request to the remote destination. As we haven’t defined any other
event handlers, ewd-federator will return the response from the remote site automatically.
To forward a request to a remote site, you use the ewd.util.sendRequest() function. This takes two arguments:
ewd-federator Reference Guide (Build 0.13)
21
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
• the destination name of the site to which you want the request to be sent. The destination name can be a group
destination name
• the request object that defines the request to be sent.
In our example we’re just forwarding the incoming request unchanged to its originally intended destination. However, if we’d
wanted, we could have created a completely different request object and sent it to an entirely different destination (including
a destination group).
Something else we could have done is to have sent our own customised response back to the REST client. ewd.util
provides two functions for this:
• ewd.util.returnRESTError()
• ewd.util.returnRESTResponse()
So, for example, you could modify your handler function as follows to simply return your own success response:
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
var responseObj = {
intercepted: 'by me!'
};
ewd.util.returnRESTResponse(responseObj, 'application/json');
}
The incoming request never got forwarded to its originally intended destination!
Alternatively you could return an error response:
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
var errorObj = {
statusCode: 400,
restCode: 'BadRequest',
message: 'I decided that no requests are allowed'
};
ewd.util.returnRESTError(errorObj, 'application/json');
}
This time you’ll get an HTTP 400 error returned with an associated JSON error object, and the incoming request will not be
forwarded to its originally intended destination.
You can therefore use the request event to write your own filters for what incoming requests you want to allow, and send
error responses for requests whose object contents indicate that you don’t want them handled.
You could also create a log in the database for each incoming request, eg:
ewd-federator Reference Guide (Build 0.13)
22
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
var logGlo = new ewd.mumps.GlobalNode('log', []);
var recNo = logGlo.$('recNo')._increment();
logGlo.$('request').$(recNo)._setDocument(request);
}
ewd.util.sendRequest(request.destination, request);
If you’re familiar with how you access the database from within EWD.js, you’ll recognise that ewd-federator behaves
identically. If you’re new to handling GT.M, Cache or GlobalsDB via the globals.js abstraction, read the chapter “JavaScript
Access to Mumps Data” in the EWD.js Reference Guide.
Manually Sending a Request to a Destination
A key part of customising ewd-federator’s behaviour is understanding how to send your own manually-constructed requests
to remote servers (or groups of servers). We’ve already seen the basic mechanism in the previous section, ie using the
ewd.util.sendRequest() function
As previously noted, this function takes two arguments:
• the destination name of the site to which you want the request to be sent. The destination name can be a group
destination name
• the request object that defines the request to be sent.
The key part is understanding how to construct a valid request object. The properties of the request object are:
• type: This is for you to define. It is simply a string value that you assign to this request (or type of request), so that you
can later recognise its corresponding response when it comes back
• path: the path part of the URL that is to be sent to the remote server
• method: the HTTP method for the request you’re sending
• contentType: the MIME type for the response that will be returned by the remote site. application/json is recommended
• nvps: an object containing any query string name/value pairs that should be added to the outgoing URL
• post_data: if the method is POST or PUT, this object defines the payload of the request
See the section above for the example we saw in the log when we intercepted the incoming REST request.
If you’re manually sending a request to a group destination, then you can optionally define different name/value pair values to
be added to the URLs sent to each site in the group. To do this, instead of using the nvps property, you can define:
• nvpsBySite: an object containing any query string name/value pairs that should be added to the outgoing URL, specified
by site name, eg:
ewd-federator Reference Guide (Build 0.13)
23
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
nvpsBySite: {
reigate: {
a: 111,
b: 222
},
guildford: {
a: 222,
b: 3333
},
...etc
}
Note: if the destination is an EWD.js server:
The path must define the EWD.js module and function name, eg scheduler/parse
ewd-federator will automatically digitally sign requests
Defining Custom Response Events
Custom response event handling hinges around the types that you assign to any requests that you manually send out from
within your extension module.
For each request that you define and send using the ewd.util.sendRequest() function, you expose the type name that you
assigned to it through the events array. For example, let’s go back to that original demo we used at the start of this chapter
where we displayed the request in the console.log. It looked like this:
onRequest: request = {
"type": "posts/1",
"path": "posts/1",
"method": "GET",
"contentType": "application/json",
"nvps": {
"rest_url": "http://99.9.9.9:8081/placeholder/posts/1",
"rest_path": "posts/1",
"rest_method": "GET"
},
"post_data": {},
"destination": "placeholder"
}
ewd-federator automatically assigns a type to the incoming REST request - as you can see it makes it the same as the path.
We’ll revert the onRequest function logic to forward this request to its intended destination, ie:
function onRequest(request, ewd) {
console.log('onRequest: request = ' + JSON.stringify(request, null, 2));
ewd.util.sendRequest(request.destination, request);
}
Now we can add a response event handler to intercept the response for the request type ‘posts/1’. First add the type
‘posts/1’ to the module.exports events list in your extension module:
ewd-federator Reference Guide (Build 0.13)
24
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
module.exports = {
event: federator,
events: [
'request',
'posts/1'
]
};
Now add the event handler to your workflow at the bottom of your extension module:
federator.on('request', onRequest);
federator.on('posts/1', onPosts1);
And finally add the handler function. I’ve named it onPosts1() in the example above, but of course you can name it anything
you like:
function onPosts1(responseObj, ewd) {
console.log('Got the response: ' + JSON.stringify(responseObj, null, 2));
ewd.util.returnRESTResponse(responseObj.response, 'application/json');
};
Notice that a response handler function takes two arguments:
• an object containing the properties that make up the received response
• the ewd object that contains the various helper functions you’ll need to use in your handler
If you save your changes (your extension module should automatically be reloaded by ewd-federator into any child process
that is already using it) and send that original request, you’ll now see the response being intercepted in the console.log, eg:
Got the response: {
"type": "posts/1",
"site": "placeholder",
"error": false,
"response": {
"userId": 1,
"id": 1,
"title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
"body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
},
"contentType": "application/json"
}
The actual response payload is in response.response. That’s usually all you’ll need, but the other properties can be useful in
certain situations.
Note that if you intercept a response, it now becomes your responsibility to decide what to do with it. If you don’t do
anything, the ewd-federator child process that is handling the response will be left hanging forever.
At this point you have three options:
ewd-federator Reference Guide (Build 0.13)
25
Copyright ©2013-15, M/Gateway Developments Ltd. All Rights Reserved
• return a response to the awaiting REST client by using the ewd.util.returnRESTResponse() function (see earlier). You can
use the response object that was passed to your handler function, or you could modify it and/or use a completely different
response object entirely
• return an error response to the awaiting REST client by using the ewd.util.returnRESTError() function (see earlier)
• send one or more new requests to remote servers by using the ewd.util.sendRequest() function (see earlier). You can
send requests to any destination or destination group. Make sure you give any message that you send out a type
property so you can handle its response with another custom response event handler.
By now you’ve probably realised that these simple mechanisms and functions provide you with everything you need to
create a “dance” of requests and responses of any complexity across your federation of back-end servers. You simply
intercept each response by type and send out new requests with new types according to what you received back. You build
out the list of event names that you expose through your extension module’s module.exports, and build out the workflow
and associated handler function for each event.
At some point(s) in your workflow you’ll reach a point where you’ve accumulated all the information you need for the final
response, in which case you’ll invoke the ewd.util.returnRESTResponse() function and send back the final federated
response object.
Using a ScratchPad Object
You may need to accumulate a set of information during a complex “dance” between servers in your federation. The easiest
way to do this is instantiate, at the start of your extension module, a globally-scoped empty scratchPad object, eg:
var scratchPad = {};
You should clear this down in the first event handler function in your workflow (eg in your onRequest() function), so that every
time the child process handles a new request, it starts from a clean slate.
You can now access, add to and modify content within this scratchPad object in any or all of your custom event handlers in
your workflow. When you reach the end of your workflow in a complex “dance” between federated servers, you can use the
contents of your scratchPad object to determine the contents of the federated response that finally gets returned to the
awaiting REST client.
Note that because all the processing of any one federated “dance” workflow happens in a single child process, there is no
risk of your scratchPad object being over-written by anyone else’s processing.
You may, of course, want to save some or all of the contents of your scratchPad object to the database, eg as cached
information to save time when processing some subsequent request.
Response Event Handlers for Destination Groups
One important thing to note. If you manually send out a request using ewd.util.sendRequest() to a group destination name,
ewd-federator will automatically aggregate the responses as described earlier, and will trigger your custom response event
only when the aggregated response has been completely assembled - ie not until the last response has been received from
the group of servers. The response object passed into your handler function will be the aggregated response object.
ewd-federator Reference Guide (Build 0.13)
26