Monthly Archives: September 2014

09.30.14
by

My Connections Migration Checklist

I’ve been doing a lot of Connections upgrades and migrations in the past few months and since I prefer to do a side-by-side upgrade there are lots of steps along the way to make sure the data is moved and upgraded from the existing servers to the new servers.  The documentation on how to do this in the Knowledge Center is good but there’s a lot of jumping around all over the place between tasks and I have found it helpful for me to have a checklist to make sure I don’t miss anything.

Here’s the checklist I’m using right now with some explanation and links to the documents in the Knowledge Center for each.  My steps aren’t  in the same order as in the documentation but they are the order I use

In theory the migration shouldn’t make changes to your production servers, but I’m risk averse and it’s worth the extra few minutes to make sure you can back out of the migration should you need to.

Before starting anything you should have created new empty databases on your new system using the scripts / wizard from the version you are moving from.  Even if you are moving to Connections 5 from Connections 4, you will need to use the Database wizard for Connections 4 to create the databases we are going to move data into.   That makes sense when you consider we are going to transfer the data over from the existing production environment so the format / structure and schema must be identical from source to target.

Begin by stopping everything, all WAS servers and DB2 (or SQL, Oracle) in your production environment as well as any TDI assemblylines you may have running.  The data migration requires the production site to be down and stay down until the new site comes up, that could be anywhere from a day to 3 days depending on how big your environment is and how much data you have as well as the connectivity between old and new environments when transferring the data.

Now let’s back everything up - just get the existing production configuration data somewhere you can access it and make sure you don’t lose any data during migration so backup all the DB2 databases as well as the Connections shared data /Connections/data.. /shared (I personally like to backup /Connections/data which gets local as well but that’s just me.

  • Backup Connections Dmgr Profile by running backupconfig.bat /.sh from the /Dmgr01/bin directory.  This will stop the Dmgr server if it’s not already stopped or if you don’t use the -NoStop parameter. (no need to backup Installation Manager when doing a side by side migration)
  • Backup the Connections shared data
  • Backup customisations somewhere you can access them for reading and manual copying over to the new environment
  • Run the migration.bat / sh to export the Connections configuration data ready for import in your new environment.  This includes the LotusConnections-Config.xml and application specific data.  This is exported to a directory you then copy to your new environment where you can import it
  • Migrate each of the databases, one at a time.  Each one has a pre-script to run to prepare the database, then at least 2 migration scripts, one to move the data and one to clear the scheduler entries on each database.   All the instructions are here however there are a couple of things to bear in mind.

When running the scripts I like to add >filename to the end of each command to pipe the output to a log file.  I usually create a “Logs” directory and call the file by the name of the script _app name e.g predb_blogs.txt.  This way I can check if the scripts ran OK by reading the logs and I have something to send to IBM if it comes down to opening a PMR

See my earlier blog for potential syntax issues running the scripts

To run dbt.jar which migrates the data you create an XML file and a matching Batch file for each application.  I like to create all of these at once and add them to a directory from which I can run for each application (again with the >logfile at the end).  Below are examples of XML and batch files I modify to use (I’ve avoided putting in carriage returns as that messes things up should you copy out of here)

XML (e.g. files.xml below)
<dbTransfer xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance“><database role=”source” driver=”com.ibm.db2.jcc.DB2Driver” url=“jdbc:db2://sourcedbserverhost:50000/FILES” userId=“db2admin” schema=“FILES” dbType=“DB2”/> <database role=”target” driver=”com.ibm.db2.jcc.DB2Driver” url=”jdbc:db2://targetdbserverhost:50000/FILES” userId=“db2admin” schema=“FILES dbType=“DB2”/> </dbTransfer>

BATCH (calls files.xml)
“e:\install\connections\wizards\jvm\win\jre\bin\java” -cp e:\dbt_home\dbt.jar;e:\ibm\sqllib\java\db2jcc.jar;e:\ibm\sqllib\java\db2jcc_license_cu.jar com.ibm.wps.config.db.transfer.CmdLineTransfer -logDir e:\dbt_home\logs -xmlfile e:\dbt_home\files.xml -sourcepassword typedb2passwordhere -targetpassword typedb2passwordhere

  • Upgrade database schemas.  Once all the migrations scripts have been run (don’t forget the clearScheduler and run/updateStats where needed) you can proceed to upgrade the databases.  I like to back them up one more time before running the upgrade though, but that’s just me.  If it took a day or more to migrate the data I don’t want to do that all again.There are two ways to update the databases on your new target server.  Either using the provided (Connections 5) database wizard and choosing “Upgrade” or by running manual scripts.  I prefer to run the scripts manually so I can see what’s going on and IBM recommend that for the Homepage at least you run the script manually rather than use the Wizard.

    Instructions for doing both Wizard and Manual methods are here .  The biggest issue with running the scripts manually is that there are slightly different syntaxes depending on which version you are coming from and it’s fiddly getting the right one, I still prefer it although  I have used the Wizard for several of the applications and it has worked fine.

  • Once you’ve upgraded all the databases, the Homepage requires another step and that’s to do a java migration of its data. This ensures the format and content of each individual’s homepage matches that required for Connections 5.  The Homepage database is by far the largest of all those used and this could take significant time.  Below is an example of the command I run (again I have taken out carriage returns and invalid quotes etc

e:\install\connections\wizards\jvm\win\jre\bin\java -Dfile.encoding=UTF-8 -Xmx1024m -classpath e:\ibm\sqllib\java\db2jcc.jar;e:\ibm\sqllib\java\db2jcc_license_cu.jar;e:\install\connections\wizards\lib\lic.dbmigration.default.jar;e:\install\connections\wizards\lib\commons-logging-1.0.4.jar;e:\install\connections\wizards\lib\news.common.jar;e:\install\connections\wizards\lib\news.migrate.jar com.ibm.lconn.news.migration.next50.NewsMigrationFrom45to50 -dbur1 jdbc://db2://targetdb2hostname:50000/HOMEPAGE -dbuser db2admin -dbpassword targetdb2password >java.out.log 2>&1

  • Importing artifacts.  Using the directory and contents created earlier one when we exported the Connections artifacts, we can now import them into our new Connections environment.  We’re basically doing the reverse of what we did to export but this time running migration.bat /sh lc-import.
  • CommunitiesMemberService.syncMemberExtIdByLogin(“wasadmin”)
  • Migrate or Rebuild the search index.  Migrating can be done if the source version is 4.5 because the search index structure is the same however I prefer to rebuild cleanly if I have the time
  • FilesDataIntegrityService.syncAllCommunityShares()
  • Custom profiles. If you have custom profile settings (strings, languages, profile types) in your existing environment and that is 4.0 these will need to be migrated / converted to the Connections 5 format.  There are also settings that should have come over when restoring your artifacts that it is worth validating

The items below tend to be optional depending on what is installed in your current environment but if these elements exist currently they will need to be migrated too

Cognos

Connections Content Manager

Media Gallery

That’s my list anyway.  Obviously the Knowledge Center is the definitive source for all you installation / documentation needs 🙂

 

09.26.14
by

Speaking at ConnectED 2015, the mysterious “track 6”

This year the new ConnectED conference introduces a brand new track under breakout sessions - Track 6 entitled “Beyond The Everyday”.  The track is being put together by Christian Holsing and myself and it’s something very different we think has been missing from previous IBM conferences.  Although it has replaced the Show and Tell track the two aren’t connected in any way and the “Beyond” sessions are regular (60 mins) session length.  Best of all , these sessions are ideal for Business Partner and Customer speakers.

Let me explain why I’m excited by “Beyond” because I can think of at least 30 sessions without even trying that would be great. Beyond the Everyday looks to fill a gap as it encourages you as speaker to talk, not just about IBM products, but about how extending those products and adding other technologies bring your software to a whole new level.  In previous years I’ve heard from people who didn’t submit that they didn’t think IBM would care to hear what they want to talk about because it’s not on brand enough - well it may not have been true then (I honestly don’t know) but it’s not true now, at least for this track.

Business solutions today aren’t all from one provider, they aren’t all from one OS, many of them aren’t all on premises or Cloud but combining the most innovative tools in each area from different sources makes Connections, Sametime, Domino, Portal etc a very big part of a larger technological story.  It’s about what we can achieve when we reach further.   So I know you all have innovative ways of making things work, of bending technology to your will, of achieving your goals by forcing those technical barriers out of the way and and that’s what we want to hear about 🙂

In short, if you’re doing something freaky, weird, bent and exciting (in a software sense) share it with an audience who want to know how to work without limits.   As before I am very happy to help or advise on abstracts if you are considering submitting, but you only have a week or so to do so.  I’m hoping we were right and this is what people want to hear and hoping even more this is what you all want to talk about!

Send in your abstracts HERE

09.18.14
by

Getting Around Documentation Errors With Connections Scripts

I’ve been meaning to write this blog for a while.  And by “a while” I mean since v4 of Connections.  IBM supply a series of scripts with the Connections install that are found in the install directory under the folder connnections.sql.  These scripts are used for a variety of things but most people will have to use them if migrating from an earlier version of Connections to a new one.  The scripts are under the database type folder for each application so the scripts for the Blogs database on DB is in

/connections.sql/blogs/db2

Now you can put those scripts where you want obviously, but that’s where you will find them.  In that folder there are lots of files that are basically a series of SQL commands written out for you.  Each command line terminates with a ; or a @ to identify that’s the end of the command.  When running these commands with db2 you use a different syntax depending on whether the SQL file ends each line in a ; or @.  For example

;  means our command line is written as “db2 - tvf {filename} >{writetoalogfile} “

@ means our command line is written as “db2 -td@ -vf {filename} >{writetologfile}”

Writing to a log file isn’t compulsory but I always do so I can check if the script ran OK.

The problem is that on the  IBM Documentation site they often give the wrong syntax for each database (oh and they aren’t consistent) so on this page the instructions for the profiles database are

“db2 -tvf predbxferxx.sql”

If you run that (and the clue is it takes less than a second which is suspicious) you will see no errors but if you check your log you will see a single line saying

“End of file reached while reading the command”

That basically means we used the wrong line terminator, we told it -tvf so it looked for a ; at the end of each line but if we open the predbxfer45.sql we can see each line ends in @.  If we change the command to

“db2 -td@ -vf predbxfer45.sql”

it runs perfectly.

It would be nice if the IBM documentation was correct but it’s a simple problem to catch and fix.

09.17.14
by

One Dumb And Two Smart Things - Calling That A Win

Last night / yesterday afternoon I was building a Connections server (for an internal project) when I wiped out hours of work doing something dumb.  I had spent some time downloading all the software and fixes to the server which was Windows 2008 R2 (because I have plenty of licensing for that)  and then I installed DB2 and WAS and created the WAS profile.  Next step was to run dbwizard.bat to create the databases but that’s where weird stuff started happening.  The dumb bit had already occurred I just hadn’t noticed it yet…..

The DBWizard would launch and let me move past the first screen but no amount of clicking on “Next” would let me move off the “Create, Edit, Update” screen.  Clicking ‘Back” actually took me to the next screen (!) but I couldn’t get any further than that.  I refused to believe it could be a DB2 problem because at the point in the Wizard it had no idea I was running DB2 as I hadn’t chosen my database platform because I couldn’t get to that screen.  I started from the assumption that since DBWizard is a java program my version of Java (brand shiny new updated yesterday) was incompatible.  So cue much time spent uninstalling and installing different java versions to try and fix it with no luck.  I could have run DBWizard from another machine but I wanted to fix whatever the underlying problem was.  Then I realised the dumb bit, I had installed 32bit DB2 on a 64bit platform which DB2 is fine with but the DBWizard really isn’t.  I don’t know if that was my problem (I still can’t believe on the early DBWizard screen it even knows to check) but in my attempts to fix uninstall and cleanup DB2 , I corrupted the Windows registry.  At least that’s what I think I did because on restart Windows would only boot to a grey branded screen with no login, even if I chose one of  the Safe modes or tried booting from a CD.

Since this work was about installing Connections and not fixing Windows I decided not to waste more time on it and startover.  Here come the two smart things.

1. I have a pre built Windows 2008 R2 VM disk with a 40GB C drive I use to clone and make new VMs.

2. I had downloaded and installed everything to a separate 100GB virtual disk

I detached the virtual disk from the broken VM

deleted that VM from the host entirely

made a copy of my simple VM disk

created a new virtual machine using that copy as its disk

added the 100GB virtual disk to that new VM

opened it up and changed its ip to match that of the VM I just deleted

and I was back in business.  Total time elapsed about 7 minutes

Of course I now had a D drive with software on it the Windows registry new nothing about but it was simple to just delete those installer folders and reinstall (the right) DB2, WAS etc and get back on track.  Certainly much simpler than trying to fix a broken Windows server!

09.16.14
by

IBM ConnectED 2015 Call For Abstracts - What Do Those Tracks Mean?

So the call for abstracts are out for ConnectED 2015 this coming January.  For those of you who don’t know.. ConnectED is a very different beast that Lotusphere or even Connect prreviously. It’s intention is to be much more technical as well as smaller (encompassing the Swan only). Think of it more like a technical user group, more opportunity to learn and network.

Smaller also means fewer session slots available so you need to submit if you want to speak as soon as you can.  Call for abstracts closes on October 8th.  Some tracks are still here (yay Best Practices!) some have moved on (SNT) and some are brand spanky new!.  Take a look at the track descriptions below to find out where you fit.  I’m particularly excited about Track 6 - Beyond the Everyday which looks at pushing the boundaries of IBM technology and where it coexists with other environments.

https://www-950.ibm.com/events/tools/ibmced/2015ems/

Track ONE: Strategy and Innovation
Who should attend?: CIOs, IT managers, IT professionals of all levels

Take your collaborative environment to the next level! Laying the foundation for the week, and designed for all levels of IT professionals, this track concentrates on these brand-new, “hot-off-the-press” technology innovations coming from the IBM labs and engineers, including exciting new technologies and glimpses of the future from industry gurus. From social transformation and exceptional web experiences to adoption best practices — and with “sneak peeks” at every turn — this track will be full of sessions that you won’t see anywhere else!

Track TWO: Application Development
Who should attend?: Developers and architects of all levels

The speed of business has accelerated to meet the speed at which people interact, digitally. People are “always on”, consuming data at constant rate. Their interactions and the Internet of things, are sources of that data. Traditional business processes are expected to combine that data with that which is inherent in their systems to provide individualized experiences to end users. This experience needs to be always available and customized to the device from which they choose to access it.

In this track, application developers will learn about building applications in a world of social interaction and information. Topics will include Cloud, Analytics, Mobile, and Social, using technologies for IBM Domino, IBM Connections, and IBM Exceptional Customer and Employee Experience Suite.

Track THREE: Infrastructure and Deployment
Who should attend?: System administrators, IT managers and integrators of all levels

The Infrastructure track is at the technical heart of IBM ConnectED; these sessions will provide the knowledge you need to design, deploy and manage a collaboration infrastructure tailored to your business needs. You’ll learn planning considerations, administration/deployment techniques, and daily “tips and tricks” straight from IBM’s software architects, developers, product managers and “in-the-field” experts. Whether you’re a new systems administrator, an experienced collaboration architect, or something in between - this track will provide what you need to get to the next level.

Track FOUR: Best Practices
Who should attend?: IT managers and practitioners: application developers, system administrators, integrators; technologists of all levels

The popular Best Practices track is delivered by the community – for the community - -and is all about technical solutions that can be implemented TODAY. Whether it’s adoption or development — Domino, mobile, cloud or portal, sessions here provide tips, tricks, and quick maneuvers with a focus on learning from the experiences of others. No product or marketing sessions found here, these sessions take away pain, bring back the fun and deliver the technologies. So put on your thinking cap and see if you can create a session that will meet these standards! We want to hear from you!

Track FIVE: Spotlight on IBM Business Partners
Who should attend?: IT managers and practitioners as well as business leaders and professionals

Back by popular demand! Sessions in this track showcase the highly-acclaimed solutions that our Business Partner community is known for - with technical detail and best practices on how the solution was built! Whether your focus is social software, email, unified communications, or creating exceptional digital experiences - on-premise or in the cloud - there’s something here for you!

Track Six: “Beyond the Everyday”
Who should attend?: IT managers and practitioners: application developers, system administrators, integrators, technologists of all levels

In this track, we step outside the standard uses of our IBM technologies and see how people have combined tools and new techniques to innovate and extend what their software can do. Meeting the demands for multiple technologies, platforms, frameworks, mobile, cloud, on premises and beyond, “Beyond the Everyday” is a deep technical track where approaches will be explained and demonstrated. These sessions may feature IBM products as part of a larger story or may be entirely focused around stretching the products we know to the limits of their capabilities. One think is for certain, they all will be about new ideas and will leave the audience excited to try for themselves!

Track Seven: Chalktalk Sessions
Who should attend?: Any and all ConnectED attendees, both IT and business roles

Come one, come all! Chalktalk sessions are informal, interactive discussion groups for attendees to share ideas and experiences in an small group, and open forum setting. Feel free to submit topics that interest you, either those you would like to lead, or topics and discussions that you’d like to participate in! As a reminder, attendees build the agenda by voting on all submissions and ultimately select the sessions that will appear on the agenda. So you’ll be sure find something that’s well worth your while!

Sessions will likely focus on the areas below, but other ideas more than welcome!

  • Messaging and Collaboration
  • Social Software
  • Exceptional Digital Experience portfolio
  • Cloud
  • Mobile
  • and more!
09.16.14
by

Adding External Users To Connections 5

Last week I did a presentation at Icon UK on the new Connections 5 feature that allows you to add external users into your Connections environment.  To write the presentation I built my own environment multiple times using different techniques for adding external users and discovered some interesting stuff along the way.  Since the presentation doesn’t have my commentary on it i’ll try and summarise that here

1. On page 6 are a list of things an external user can do according to IBM documentation.  Some of the items on that page (in italics) actually didn’t, in any of my testing, work.  This is because there are conflicting security limitations on what a user can’t do (see items in bold on page 7)

So for example although the documentation states that an external user can share files with people or communities, it also states that they can’t use type ahead or directory lookups.  Preventing type ahead and directory lookups actually disables the ability to share files with a user since there’s no way to lookup a user.  Sharing files with a Community works fine.

2. The external users can be added via an LDAP attribute from your LDAP server or by a separate LDAP server or branch.  Although an entirely separate LDAP server is more secure and in my opinion preferable, it must use a search base which means flat names in Domino can’t be part of the external LDAP source.

To counteract this in one instance I faked a hierarchy as the users were created (using a simple Xpages app to allow people to self register and manage their own passwords and setting a fake hierarchical name for them in the background).  In the other instance I used the same LDAP source as for internal users but with a specific attribute set to the word “external”

In general the external users feature has been locked down securely enough that i’d highly recommend it for inviting people to work with your Connections communities .

09.11.14
by

Icon UK Presentation - External Users in Connections 5

Today I am finishing my presentation for ICON UK on external users in Connections 5.  There’s a lot to cover and I’m trying to run neither over or under time and pull off the goldilocks of presentations covering…

  • How external access works
  • What can external users do (and not do)
  • How your internal users interact with external users
  • Configuring external user access
  • Securing the perimeter
  • Implications and things to think about…

I’ll post the presentation here once I’m done but of course it won’t come with me talking over each page (is that good or bad?)  - so if you can make it to Icon tomorrow at IBM South Bank in London, I hope to see you there.  My session is at 11am.

 

09.11.14
by

WebSphere Things That Drive Me Insane - Pt..um.. 3

I actually like WebSphere. Honestly I do.  But it really really does not like Domino and Domino is my first love (well 2nd love.. ccMail you’ll always be first in my heart).  I have always run into problems configuring Domino within WebSphere mostly due to the fact that Domino LDAP isn’t always hierarchical the way every other LDAP is.  Back in the original Sametime 8.5 days we couldn’t have users of ST who didn’t have hierarchical names and we used to have to fake a hierarchy (C=US) to trick WebSphere.

My latest hair tearing out insanity is shown below.  To configure external users for Connections you can choose to set up an alternate LDAP source - in this case I’m using a dedicated Domino server I can make publicly available for people to register themselves.  Here are my repositories set up in WebSphere showing the two Domino LDAP sources..

LDAP1 is our internal directory LDAP SSO the external / public facing

LDAP1 is our internal directory
LDAP SSO the external / public facing

Here’s the fun bit.. this is what the federated repositories actually look like in WebSphere

As soon as I added the external Domino LDAP repository it changed the original internal one to the external one so that’s listed twice.  Try and add it again and it adds the same one a third time.  Even more hilarious, only the original (unlisted) one actually works and lists / authenticates users.

And yes, if I try and delete one it actually deletes all three.  Off I go to edit some XML files….I’ll post a fix when I get there

09.09.14
by

IHS Errors or WHY Won’t Connections SSL Work

It happens.  Usually when I’m building a test server on a single box and i’m building in a hurry.  I get everything configured and installed and take a brief stopover at IHS configuration on my way to completing security setup.   I create my keyfile using ikeyman, I import my trusted root certificates from whichever CA I plan to use and I generate a personal certificate.  I think it’s all working fine then I restart IHS and one of two things happen

1. IHS starts but only for 80 not 443

2. IHS starts on both 80 and 443 but I get an error 500 trying to access any Connections page over SSL

The logging on the 2nd error isn’t terribly useful and it’s tempting to run around checking the module mappings and LotusConnections-Config.xml for the source of the problem.  For some reason, even though I’ve seen each of these lots of times, my brain insists on starting at the beginning with debugging and looking at the logs.  So this blog is for you brain - next time just come here and check this first

1. The solution is often that the keyfile either isn’t where I told httpd.conf it was OR where the plugin-cfg.xml is looking for it.  Take time to go check the plugin configuration under your webserver in the ISC and make sure the name and location are what you think they are.  Then go and actually make sure they are there

2. A handshaking error caused by either the signer certificates used by the application servers not being imported into the keyfile OR (and this one drives me batty) installing everything on one box with the same hostname for the WebSphere servers as the IHS server.  In the 2nd instance you can’t have two totally different certificates both claiming to be the same hostname trying to talk to each other.  I export the certificate from WAS trusted key store and import it into ikeyman (or import into WAS and map each of the servers).

In general when I’m configuring IHS it’s always down to a file not being where I told httpd.conf it was.

Here are my rewrite and plugin lines for 64bit IHS on this particular Linux box

LoadModule was_ap22_module “/opt/IBM/WebSphere/Plugins/bin/64bits/mod_was_ap22_http.so”

WebSpherePluginConfig “/opt/IBM/HTTPServer/Plugins/config/webserver1/plugin-cfg.xml”

RewriteEngine On RewriteRule ^/$ https://<hostname>/homepage [R,L]

Update: I should have linked to this document which I found in the past and is always useful. Troubleshooting IHS