Perfect10 - Building A Test Lab

In the 10th edition of my Perfect 10 webcast I explain how and why to build a test lab so you can get to deploying those products you downloaded today. You did download them right?

I was asked recently to share the slides for all these Perfect10 presentations and to be honest I hadn’t thought of it but it’s a good idea so I’ll be sharing them all this week.

Next up: Let’s Talk Domino v10 & Admin

11 mins

All the 10s, Let’s Make Things Simple..

Announced at the Domino 10 launch today: if you have let your licensing for Domino lapse you can renew it until the end of the year saving up to 50%, and if you have licensing and need more then IBM will discount new licensing by up to 20% until the end of the year.  I don’t sell licenses but that seems like a good deal to me.  Why should you? Well, why shouldn’t you?  If you have Domino in your environment even with a lapsed license you are keeping those servers around because of all the data that’s on them - don’t you want to use that data?  Let’s talk about what you get if you are licensed:

  • access to your Domino data using Node and modern javascript development tools
  • a new query language for Domino (DQL) which allows you to access Domino data from external platforms (buh-bye ODBC!)
  • access to your Notes applications (yes even the really really old ones) on an iPad
  • entitlement to not only use instant messaging in your Notes and web clients but also on mobile devices
  • a ton of TCO features including a new 256GB db size limit, auto database repair, cluster symmetry (automatically populating entire directories from one server to another and keeping them in sync), publishing of stats to New Relic and other cloud based reporting tools, new full text indexing engine, …

I’m not mentioning a ton of other features too.  I’m giving credit to the team who did such a great job today and especially Luis Gurigay whose presentation was seamless and showed how your back-end Domino applications work with no required code changes on an iPad, how your Domino data and web apps can be integrated into O365 and (this which he showed live and I took a screenshot) of showing a Notes document being created and then published in Slack, Microsoft Teams and Watson Workspace concurrently.

The code for this will be made available via the IBM Destination Domino site tomorrow.

So Domino 10 is out tomorrow.  The beta for the application development pack which includes the Node module is due out this week, and the beta for Notes on the iPad is due out this month.  If you want to sign up for the betas go to the destination domino site and if you want to talk licensing and you don’t have someone at IBM to talk to let me know and I’ll see who I can point you at (and then step away because I do not want to do licensing :-))

Building a stack in Node.js

[Update: Since I posted this article, I have been informed that the domino-db node integration will be available in beta with Domino10 in a week’s time!]

With Domino 10 nearly upon us, and the Node integration hopefully following soon after, I thought I would talk about building a full-stack application in Node.js, covering how modern JavaScript UI frameworks can be built on top of Node.js and integrated with Domino in the background as a datastore.

This is all part of the IBM and HCL strategy of having Node.js as a parallel development platform alongside the standard Domino development tools, with Node providing a way for web developers to extend existing Domino apps and datastores. However, you don’t have to wait for the Domino node module to start learning about this.

If you consider the development stack acronyms of DEAN and DERN (or NERD), the UI framework is the ‘A’ and the ‘R”. These initials refer to Angular and React respectively, but generally apply to any JavaScript framework, and there are many very good ones.

The main advantage of a development stack is that the UI layer can be independent of the middleware, server, and datastore layers and so you can replace or modify the UI without impacting the rest of the architecture. As an example, you might want to do this to extend an existing web app onto a mobile platform which may require a different UI.

A review of popular frameworks and their features and advantages is beyond the scope if this post, and I may return to that later, but for now I would like to get into the broader topic of how this all fits together, i.e. how does a JavaScript UI sit ‘on top of’ Node?

The first thing to understand is that UI frameworks such as Angular, React, et al, are nothing to do with Node.js. They are not part of Node.js and do not require Node.js to work. They all run perfectly happily on any web server, including Domino. When we use a UI framework with Node, Node is essentially acting as a web server, serving the framework web pages to a browser which then loads the pages and runs the framework code.

You can run framework components inside a regular Domino web form on a Domino server, but the advantage of using the JavaScript development stack is two-fold. First, the stack is all JavaScript, so it makes it easy to talk between layers because all the data is JSON. Second, we are opening up Domino to a new development arena, with an established community, support resources and third-party products.

JavaScript UI frameworks come in all sizes and flavours, but they all work essentially the same way. You embed some code in your HTML web pages and then call the framework library (usually a js file) to enable the framework within these pages. The key thing is that everything is held in the usual web files and folders which are stored on the web server file system. Your server (whether it is Domino, Node, or Apache) simply serves up these files when a browser hits it.

In my earlier post, I talked about how you can use Express to provide middleware routes in your Node server. Here is an example which uses a static route to serve up the contents of the ‘public’ folder on the server:

const express = require('express');
const app = express();
app.use(express.static(__dirname + '/public'));
const hostname = '127.0.0.1';
const port = 3000;
app.listen(port, hostname, () => {
   console.log(`Server running at http://${hostname}:${port}/`);
});

By default, the server opens the index.html file in that folder. Luckily, this is the default filename used in most UI frameworks, so it is very easy to put your framework web app and all its files and folders in the public folder and run it from your website. The folder structure could look something like this:

node
|   server.js
|   package.json
|
+---node-modules
|   \---(node stuff...)
|
+---public
|      index.html
|      other framework files...
|      +---framework folders...
|      \---etc

The Node server runs from the server.js (using modules such as Express which are installed under the node-modules folder), and serves up the framework UI starting with the index.html in the public folder.

So we have a nice UI running on our node server. Now we need to connect this UI to our data backend. In a Domino web form we would typically make a call to a web agent which would return whatever Domino data we need. Here in Node we do pretty much the same thing, specifically our UI will make a call to an API to get the data.

Node is great at two important roles - being a web server and being an API gateway. So we can add middleware routes to our Node server to handle these API calls.

Let’s suppose we want to get a list of people to display on-screen. In our UI framework we will make a call to an API on the same Node server, something like this:

let uri = myDomain + "/api/people";
request.get( uri ).then( displayTheResults );

So this request would hit our Node server with the route “/api/people”. Currently, our server doesn’t know what to do with this, so let’s add in a new Express route to handle it.

app.get('/api/people', (req, res) => {
   let myResults = doLookup();
   res.json( myResults );
})

This will catch requests coming in to “myDomain.com/api/people”, and then our middleware code does the lookup and sends the results back in the response as JSON.

Notice how these methods handle all the JSON stuff for us, making it easy to pass the data back and forth.

This is all we have to do to to get our server responding to the API call. Now we need to look at how we get data from the backend, i.e. what happens in our doLookup().

If we suppose we are going to be using the upcoming domino-db module, then we can use its methods to run queries on Domino data. The details of exactly how the domino-db module works are still under NDA right now, but it could be something like this:

function doLookup() {
const { domServer } = require('domino-db');
   domServer(serverConfig).then(
      async (server) => {
         const db = await server.useDatabase(dbConfig);
         const docs = await db.bulkReadDocuments(DQLquery);
         return docs;
   });
} 

This example function would run a Domino Query Language query on a Domino database which returns a JSON array of document objects, i.e. ‘docs’.

We pass this back as the return value of our doLookup function and this will be sent out as the response from our API route.

Back in our front-end framework UI, we receive this JSON data in our ‘request.get’ call and we can then go ahead and call our ‘displayTheResults’ function.

This really is pretty much all there is to it. We can easily get data, in a standard JSON format, all the way from the datastore to the UI front-end, without needing fiddly data manipulation and all in only a few lines of code.

Also, what is great is that the UI framework is separate from the Express routes and these are separate from the datastore. They can all be on different servers, and we can even have different UIs accessing the same API routes to get at the same data, for example separate web apps and mobile apps.

I hope this gives you an idea of how we will be able to about go about building a full-stack application in a DEAN/DERN environment. In my next blog I plan to expand on how we can use Express to build useful routes to do all sort of things, such as performing CRUD operations and incorporating business logic.

Deletion Logs - What’s Coming In V10

So deletion logs.. currently (without custom code) we cannot tell who deleted a document and what document they deleted in which database.  With v10 deletion logging is now a standard trigger on the database that creates an entry in a delete.log file in the IBM_TECHNICAL_SUPPORT directory detailing every deletion activity.

So how does it work?

Deletion logging is enabled via the compact task on an individual database basis. The option -dl is used when compacting a database along with the fields in that database you want to be part of the log. For example if I wanted to turn it on for my mail file I might do

load compact mail\gdavis.nsf -dl on subject,posteddate,sendto,recipient

Every deletion after that point would then be logged as a single CSV entry in delete.log.  Note there are standard values that are always logged in addition to the custom fields I requested

“20180210T211516,06+01″,
“Mail\gdavis.nsf”,
“80256487:00352154″, “nserver”,”CN=Traveler/O=Turtle”,
“SOFT”, “0001″,”72C0E3F8:44B53FB5DC4EDBF8:A785466D”,
“from”,”””New Relic”  -
 “<marketing@newrelic.com>”, “sendto”,”gabriella@turtle.com”, “deliveredDate”,”02/10/2018 21:05:05”, “posteddate”,”02/10/2018 16:15:18″

There are several interesting aspects to this approach but I see it being particularly powerful for audit purposes, as it shows not only the message but the timestamp of the deletion and who did it.   Note that the server name in the log entry here tells me my Traveler server did the deletion so it was done from my phone, if it had been deleted in the Notes client it would have my name there as the person who did the deletion.

The delete.log itself rolls over each time the server is restarted but obviously depending on the size of your environment and how widely you deploy deletion logging that’s a CSV file you are going to want to have a strategy for.

7 days and counting

 

Java Problems Installing Sametime Community Server

Recently I was asked to install Sametime Community server in a new site.  I’ll be honest, I haven’t done a greenfield site install of Sametime in nearly a year, my work has primarily been upgrading (adding new elements) and maintenance.

As you probably know you can’t just install the Community Server onto Domino,  much of the admin and management features are now controlled solely inside the Sametime System Console running on WebSphere.  When installing WebSphere I installed version 8.5.5 as a base then applied the latest fixpack 12. (now version 13).  The Sametime elements only work with Java SE6 which used to be fine, during the WebSphere install I’d explicitly override its wish to install Java SE8 with a radio button to choose Java SE6, however that option disappeared on fixpack 11 and as of April 2018 Java SE6 is no longer supported even though Sametime still requires it and will continue to do so I suspect well into next year since the next release of a Community server is scheduled for H1 2019 and other elements for H2 2019.

Everything installed fine but then the servers with the applications couldn’t be stopped properly.  I had to uninstall WebSphere and the SSC entirely, then install base 8.5.5 with fixpack 8 (which I had to hand although other early fixpacks may also have worked) that allowed me to choose Java SE6 then install the SSC.  Once it was installed and I tested starting and stopping server elements I went ahead and upgraded the fixpack to 12.  WebSphere will warn you but continue to honour the Java version you originally chose , in this case Java SE 6, and not force an upgrade.

So. Websphere 8.5.5 with FP8 , then FP11.. 12.. 13 whatever you want.  The system requirements still say these are all supported so the loss of the option to choose Java SE 6 during fixpack install is what we are trying to fix.