Connections Futures – IBM Connect Review #3

We interrupt this blog to apologise. Usually I like to sanity check my statements before publishing them but 80%+ of presentations from IBM Connect are missing online. That means my notes are all I have right now.

So let’s talk about IBM Connections and where it’s going.  You’ve probably heard references to Connections 6 as well as Connections Pink maybe Muse or Livegrid so let’s try and clear some of that stuff up.

First Connections 6 will be shipping in Q2 (tbc) and will be an upgrade from Connections 5.5 (possibly 5.x). It will have the same architecture as existing Connections and that means WebSphere, DB2/SQL/Oracle ,TDI etc.  There are some much needed new features in Connections 6 like the ability to customise as well as copy Communities. It will also ship with the first component of Connections Pink in Orient Me which will be an optional service offering an intelligent newspaper like homepage that customises itself for each person and their activities / interests.

To get to Connections Pink you’ll first need to be on Connections 6 so let’s talk about Pink and what it is.

To quote IBM Pink “is a Vision, not a Release”.  It is the future of Connections reconceived from the ground up.  There are several principles behind that Vision

  1. Currently there are multiple code streams for Connections depending on whether you are in the cloud or on premises.  There will be a single code stream
  2. Websphere will be gone
  3. Everything will be delivered as a microservice inside a docker container
  4. DB2/SQL/Oracle will be gone and replaced by MongoDB
  5. You will be able to choose the location of data for each service/component so if you want your Activities data in the cloud but your profiles on premises you can have that
  6. Every service / component will have a published API
  7. You will be able to develop your own extensions and publish them to your own github repository to deploy

There’s a lot more but essentially the ties to the legacy IBM architecture are gone. The keywords (as far as I could tell) are customisable,  flexible and developer focused. If you’re an admin then things should get much easier as you’ll be deploying services that are prebuilt inside docker containers and can automatically pick up updates directly from IBM as you would any other online software update.

So let’s take a brief timeout and discuss docker.  If you’re a developer you probably already know and maybe use docker but as an admin, especially one managing production environments it’s likely you haven’t come across it.  A docker container can be deployed anywhere as inside it are both the software code and application and the operating system needed to run it.  In theory simply starting a docker container will enable you to run the application inside it without ever having installed anything.  As you can imagine that’s a very tempting idea for Connections where we have 14+ individual applications that could then each be run inside their own individual docker containers.  Now that seems simple but as admins we will still have to understand docker behaviour, networking, container to container communications and so on.  It does mean however that add / removing / upating services becomes a simpler task.

One of the important goals of Pink is that there is to be no migration effort  from Connections 6 to Pink.  As they are doing with Orient Me, IBM will gradually introduce Pink elements into the existing Connections architecture and those elements will read data from the legacy components into their new Pink components so that at some time in the future it’s all Pink.  I stole this chart from Maureen Leland’s presentation and there are three important things to note from it.

  1. Yes those are a lot of scary looking technologies
  2. No you do not and will not need to learn or understand any of them. That entire middleware layer will be invisible to you
  3. The takeway from the diagram is that the Connections cloud code (Green) and the Connections blue code (on premises) will seemlessly transition to the Connections Pink code without you touching a thing.  Like magic.

screen-shot-2017-02-28-at-22-04-00

The other feature I want to talk about is the Muse Proxy.  Overlaid on top of Connections it will allow us to customise our Connections UI.  If you want to add a button on screen or  javascript action somewhere one doesn’t exist, you can do that using Muse and without touching the docker containers that have the services in them.  Think of it like a really powerful stylesheet (well I’m an admin so that’s how i’m going to think of it).  Now imagine your docker containers continue to update with the latest code but your configuration settings via your Muse proxy don’t change. We can separate the customisation elements from the service elements.

So I could type pages more on this and I will in future blogs when I want to talk about all of the technologies that you should not need to learn but want to be aware of.  Livegrid for instance is a new development platform for Connections and that gets a blog all of its own. The good news here is that IBM are dismissing the more lumbering dated and complex technologies and trying to replace them with architecture designed as if they were a new startup with all the technologies in the world to choose from and not just ones branded IBM.  It’s a big vision thing and I hope they can pull it off.

To quote Jason Roy Gary the Director of Software Development for IBM Connections

“We don’t want to develop software for you, we want to develop it with you”

– I applaud both the intent and the HUGE effort being put in.  As partners and customers we will do what we can to contribute to that end goal.

So to summarise, you need to get ready for Connections 6 not just for its new features (of which the Community customisations would be worth the upgrade alone) but because you will get the first Pink service (Orient Me) and have everything in place for the Pink updates and services as they arrive.  Connections 6 should be the last whole system upgrade you need to do.(that’s me saying that not IBM) 🙂

 

 

 

 

You Lie! Error Messages and When To Ignore Them

Building Connections this week and troubleshooting some errors reminded me to share the process I have adopted when dealing with IBM error messages – which is to treat them as hints that can set you on the right path but also send you badly down the wrong one.

Problem 1:

Installing Connections itself via Installation Manager.  One of steps during the install requires you to specify the DB2 server, the database names and credentials to connect to them.  I click validate and it fails  with error CLFRP0030E and launch error!.  That points to this technote which says I left a space after the hostname for the DB2 server.

I absolutely didn’t leave a space and didn’t copy/paste.  Just in case (and working on the assumption that it’s always me and not the product) I cleared it all and typed carefully again. I confirmed the hostname was correct and could be reached.  I also relaunched Installation Manager and started from the beginning.  No luck.

It’s  at this point I have to accept the error is referring to something else and that’s all the information I’m going to get from Installation Manager.  So now I move to asking myself “what if I saw no error but it just failed to connect”.  Well the first answer to that is to check if the connection details, hostname, credentials etc actually work at all.   Having confirmed the hostname and ports (there were no firewalls turned on or virus software), I logged into the DB2 server and checked the LCUSER account. Locked out.  I unlocked the account and the install then completed.

Problem 2

The test server in this environment is one box with everything DB2, TDI and all the applications on it.  My base WebSphere install was WAS 8.5.5 FP10 since Connections System Requirements for WebSphere 8.5.5 says FP8 and higher and I wanted to test that out. Everything installed fine right up to when I went to install Connections Surveys.  That’s when I hit a 2 day brick wall.  Installation Manager couldn’t connect to the Deployment manager despite it being on the same server.

screen-shot-2016-12-09-at-18-26-10

Well that’s odd.  Deployment manager is running.  The hostname resolves. The port is listening. I try to find out what the system requirements are for Connections Surveys but for 2 days last week and through the weekend the IBM system requirements pages for Survey were down.  I’m stubborn so I won’t let it go.  Even the Forms Experience Builder requirements for earlier versions were down.  So eventually I had to leave it and move onto the production build. The work needs completing and I was suspicious that the issue might have been installing everything on one server.

I build production across 4 servers and this time I stick with WebSphere 8.5.5 FP8 just in case.  When I get to the Surveys install it goes without a hitch.  So back to the test server I go.  Roll back Websphere to 8.5.5.0 and then forwards to FP8 (thank you Installation Manager!).  Surprise surprise Surveys installed perfectly.

So. Not an issue connecting to deployment manager or port or the server running but instead “Connections Surveys cannot install onto WebSphere 8.5.5.10 at all.

 

 

User Denied Access To Files and Wikis

Another PMR this week on a new 5.5 side by side build. Once built everything looked OK except a couple of users in IT who received access denied errors when going to Files or Wikis, everything else worked.  Those two applications have databases with pretty much the same schema so we often see matching problems in both applications.

Checking the application security I could see that both were set to All Authenticated so there was no reason why those users couldn’t get at files.  The browser error contained

Identifier: LC6C54CE35BA4D41BF8CB2461634B9EAE6 EJPVJ9275E: Unable to add a group with the directory ID [E7F267C7-8811-D8EC-8025-7E57004A5278, 4339D1D3-2F37-ACDB-8025-7E57004A5285, C0085F47-7A84-EFD4-8025-7E57004A51FA, 4DB58BD6-77EA-80AC-8525-6B700078923E, A5456CF5-9FA0-E49B-8025-7E57004A5316, 54578802-623A-2E18-8025-7E57004A5289, 4EEFAFD1-A098-4155-8025-7F1D00522430, 0D1FD4C5-F61E-CB15-8025-7E57004A51F6, 5A3E2519-52BE-F072-8025-7E57004A527B, 04CE2967-BD15-B84D-8025-7E57004A52F1, 0D162A8C-223C-33C3-8025-7EB4002F6ADF, 6DCCEAE9-6A16-2A75-8025-7E57004A5377, 2B5D0EBA-B225-BA42-8025-7E57004A52DF, C6B296E2-5D27-0F89-8025-7E57004A532A].

If I count how many directory IDs are listed there, there are 14 which matched the number of groups that user was a member of when doing an LDAP query.  Still we weren’t using groups for any access and this exact configuration was working for the same users in 5.0.

In the SystemOut.log I could also see

CWWIM4546E  Duplicate entries were found in the external identifier ‘d68bb54dea77ac8085256b700078923e’ in repository ‘d68bb54dea77ac8085256b700078923e’.

That ID (formatted in various ways) would not resolve to any group in Files or Wikis never mind to duplicates.

Eventually David McCarthy @ IBM got me to change the wimconfig.xml file on the deployment manager which fixed the problem.  My configuration didn’t exactly match the documentation which said to change

<config:baseEntries name=”o=ORGX” nameInRepository=”o=ORGX”/>
to
<config:baseEntries name=”” nameInRepository=””/>

my configuration only had <config:baseEntries name=”” – no ORGX and no nameInRepository at all.  I believe that’s because we use  Domino for LDAP and “root” as the base entry so my federated repository looks like this – a configuration that results in no entry for nameInRepository in wimconfig.xml.

Screen Shot 2016-06-29 at 14.52.26

Once more this isn’t a problem in 5.0 but possibly due to a change in WebSphere behaviour in a newer version, I had to manually edit wimconfig.xml to add the nameInRepository=”” value.

At IBM’s request I also added the Group Membership Attribute which is used for resolving nested group memberships.  This customer uses Domino for LDAP and doesn’t really use nested groups in Connections so in 5.0 it was empty and worked fine however 5.5 may have been struggling with resolving group memberships for some individuals.  In 5.5 having it set to empty could have been contributing to the access problem.

The screenshot below is from 5.0. Screen Shot 2016-06-28 at 19.13.56this is how I changed it in 5.5 (same LDAP source, same users, same everything else)

Screen Shot 2016-06-29 at 15.06.31

Resyncing and restarting then fixed the problem and the users concerned could suddenly access Files and Wikis.

Not sure why it didn’t work for those users before the changes but it could have been something to do with one particular group and its nesting or maybe even a replication conflict which I couldn’t find.

Go figure.

 

Ephox Textbox.io – documentation error

When installing Textbox.io, one of the rich text editors for Connections 5.5 from Ephox,  you have the option post install to configure a spellchecker.  It’s actually a very nice feature that spellchecks on the fly in any rich text field within connections.

To enable it you have to install one of the ear files that comes with the Ephox installers and configure a configuration file that allows the spellchecker to run.  It’s a simple thing to do and the instructions are here however a few issues you should be aware of

  1. The documented example refers to you using server ports but if Connections is correctly configured via IHS and you have regenerated the plugin-cfg.xml then you don’t need to add the server ports for access
  2. The example refers to only one  origin URL but often we have more than one.  To add additional origin URLs you add a comma and a space. My example is
     ephox { allowed-origins { origins = [ "http://connections101.turtlehost.net", "https://devtest.turtlehost.net"], url = "https://connections101.turtlehost.net/ephox-allowed-origins/cors" } }
  3. The biggest problem is that the documentation is wrong when it says where to create the application.conf file

    On Windows: BOOT_DRIVE_LETTER:\opt\ephox\application.conf where BOOT_DRIVE_LETTER is the boot drive for your system

    it’s fairly clear it wants me to put the file on the C drive which is the boot drive for Windows but if you do that the spellchecker won’t work and the URL

    https://connections101.turtlehost.net/ephox-allowed-origins/cors will return

    {"value":["http://localhost"]} 

    which is obviously a default value when the file can’t be found.  

    You need to create the application.conf file not on the boot drive but on whatever drive you have installed WebSphere for the deployment manager which could be the D, E or even Z drive. By creating the /opt/ephox directories there and an application.conf file the spellchecker will find it and start working.

Connections Upgrade UI Weirdness

My latest PMR for a Connections 5.5 upgrade was concerning some very strange behaviour in the UI.  Seemingly unrelated but looking suspiciously like the same cause. Here’s what we discovered (and yes it was tested with multiple IDs , browsers and locations)

  1. The public files window had no scroll bar
  2. There were no upload buttons on the Files app
  3. When editing a profile picture I couldn’t remove it, only change it.  If I chose remove it just closed the profile with no changes.  The wsadmin profile command to remove the picture worked fine
  4. Editing a Community I owned just showed a blank page with a Community title but not editable fields

Weird right?

No errors anywhere in SystemOut or even in filebug or fiddler.  IBM could find no problems in the configuration or errors anywhere so today we did a screenshare to see what was going on.  After 90 minutes we had no luck, not even an error so Charlie Price joined the call as we talked over what I did to build the environment I explained it was a side by side upgrade where I

  • Built a new 5.5 server
  • Upgraded it to CR1
  • migrated the data to 5.0 CR1 databases using dbt.jar
  • upgraded those databases first to 5.5 then to CR1
  • used the migration tool to export artifacts from 5.0
  • copied the work folder the migration tool created to the new 5.5 server
  • used migration lc-import (from the updated tool issued D1 Dec 18) to import the artifacts

When I looked at the customization folder after completing the migration it was full of content that had been imported so the first thing I did when I saw this problem was empty that folder (by copying it somewhere else) and getting it to match my Connections101 5.5 CR1 server.  My suspicion was that somehow the migration lc-import had broken something.

Charlie compared with his system and noticed that both the javascript folder under Customizations and the webresources folder under provision were both completely wrong.  When we checked the 5.0 environment we saw that the contents matched that so must have been overwritten during migration lc-import.  The clue was the missing javascript and jar files for Ephox which I had installed and were now missing.  Charlie sent me a zipped up copy of his webresources directory and I overwrote mine with that and everything started working.

So.. be very careful using the migration wizard.  Take copies of your javascript and provision\webresources directories before doing an import and then make sure the files look up to date and there is nothing missing when you’re done.

 

 

 

Connections Failure To Authenticate

Last week was spent working on a PMR where a newly migrated (side by side) Connections 5.5 environment refused to let anyone access any applications.  I could login using any credential but the Homepage wouldn’t load and any application that required authentication failed including Communities.

Here are some of the errors in the logs

CLFRW0016E: Could not retrieve details for the user with login ID gabriella.davis@domainname.com due to an exception. The exception occurred when retrieving the details via the virtual member manager directly: {1} (in system out for utilcluster which contains homepage)

ADMN0022E: Access is denied for the expandVariable operation on AdminOperations MBean because of insufficient or empty credentials. (in ffdc)

“CustomAuthent E com.ibm.connections.httpClient.CustomAuthenticatorFactory <init> SONATA: authenticator class name is missing!  {in SystemOut for InfraCluster)

webapp E com.ibm.ws. webcontainer.webapp.WebApp logServletError SRVE0293E: [Servlet Error]- [action]: com.ibm.tango.exception.AuthContextException: com.ibm. connections.directory.services.exception.DSException: com.ibm. connections.directory.services.exception.DSOutOfServiceException: java. lang.NullPointerException (in Systemout for InfraCluster).

 

Here (amongst others) are the things we tested / changed / reverted that didn’t fix it.  Bear in mind a working 5.0 production environment with the exact same configuration had no problems during this time.

  • LDAP was fine (we could login). For giggles we changed credentials and back again
  • We changed the login options from mail;cn;uid (which we use in this environment and works fine) to uid;mail;cn
  • We removed the mapped credentials for application security that were put there by the installer and put them back again – apparently that sometimes works
  • Set the authentication under application security for Communities and Profiles from None to Everyone just to confirm where the problem was
  • About 100 other things

Basically we managed to establish the issue was any intraservice communication but not why.  Eventually it went to L3 who isolated the error  as being something in the LotusConnections-Config.xml.

CustomAuthent E com.ibm.connections.httpClient.CustomAuthenticatorFactory <init> SONATA: authenticator class name is missing!  

That file had been migrated as an artifact via the migration tool and was the same as 5.0 but in there was the line <tns:customAuthenticator name=”DefaultAuthenticator” xmlns:tns1=”http://www.ibm.com/uiextensions-config”/>;

which they asked to be changed to <customAuthenticator name=”DefaultAuthenticator”/>

That immediately fixed the problem.

No-one is quite sure how that setting ever got into LotusConnections-Config.xml but my guess is during a CCM/Filenet installation.  The interesting thing is that it works in 5.0 but breaks 5.5. Maybe it requires you to have CCM installed to work as the 5.5 environment (mine or IBMs) didn’t have that.

Still a nice simple fix for such a painful problem and maybe somewhere for you to check when doing your own debugging.

Thanks very much to David McCarthy & the IBM L2 team for prioritising and working the problem.