The Painful Journey To Abandoning iCloud

As some of you know I’m very committed to the Mac ecosystem.  I have Mac laptops, an iPad with over 4000 books, iPhone (not the latest because who needs that), a watch Apple TVs x 4 etc etc.  I’m also extremely risk averse and cloud wary.  I gave in and let Apple put all my books in the cloud just because iTunes sucks for syncing and cloud syncing worked across all my devices however I also had a lengthy open support call last year with Apple wanting to know where my books were now stored on my Mac so I could find them and back them up

“they are all in the cloud”

“yes I get that but they are also on my laptop so where are they”

“no they are only in the cloud”

“well that’s not true because here I go, switching off wifi and hey I can still read my books in ibooks so they are here somewhere”

..>> pause for several weeks whilst this is escalated>>>>

“they are on your Macbook but stored in a way you can’t find them or access them”

(please no advice on this one, I found my own workaround to find them and backup un-DRM copies)

So.. iCloud. I agreed about 18 months’ ago to let my Documents and Desktop folders sync to iCloud.  My only reason for that was so that I could get at files if I needed to on my iPad or by logging into any browser but tbh I rarely used it.  Still it worked and seemed a decent idea.

Then one Saturday about two weeks’ ago it all went horribly wrong..

I was sat working when I got an alert saying facetime had been added to my watch.  Which was odd.  My watch is 18 months old and was on my wrist and nothing had changed. The watch itself had no alert.  So off I go digging and I find under my account and devices a list of my current watch and an old watch I wiped and sold to a friend to give to his wife 18 months ago.  Well still odd but no big deal.  They hadn’t done anything so clearly just an odd gremlin.  Just in case I removed that old watch from my devices.

Then I got alerts saying my credit cards had been removed from my watch.  Except they hadn’t been removed from the watch on my wrist and the other watch was flattened before I handed it over 18 months ago.

I did some research, found nothing nefarious and let it go.   I did notice I had been logged out of all my Apple accounts on all my devices and things like Sonos had to be re-authorised again.  Weird and annoying but a side effect of whatever happened I assume**

Then a few days later I restarted my laptop.  I probably only restart it every two weeks so this was the first time since that alert.  The laptop restarts but finder and anything that uses finder like spotlight or even terminal were entirely non responsive.  They would briefly work long enough for me to type 2 characters or click on a folder then there would be a spinning ball for about 25 seconds before it would respond.  That gradually got slower and slower over a few minutes.  So off we go to research because I now have a broken laptop.

After several hours research we found this article which gave a bit of a clue as it pointed to a cloud corruption problem http://osxdaily.com/2015/04/17/fix-slow-folder-populating-cloudkit-macosx/

Unlike some of the other Finder troubles, the Finder process usually doesn’t eat much CPU or crash repeatedly, it’s just inordinately slow when loading folder views, populating files, and opening folders.

So I followed the instructions and deleted the files they specify and immediately my laptop was more responsive.  OK.. Well that was a scary afternoon and I’ll just go ahead and disable cloud syncing so that never happens again.

Did you know Apple doesn’t let you do that?  If you disable cloud syncing for Documents anbd Desktop it actually deletes the contents of those folders and keeps the files in the cloud for 30 days in case you want them back.  So that’s dumb.  I decided to move the contents of both folders to temporary folders, disable cloud syncing then move them back but my laptop was working and I was busy so I parked that for later.

Later…. about a week later again the Finder sluggishness came back but this time I knew how to fix it.  Once it was fixed I went ahead and moved the contents of both Documents and Desktop to temporary folders, disabled cloud syncing and moved them back.  My laptop immediately started working, finder was faster than it had been for a very long time and I’ve had no more problems.

Now I wonder if that first alert about this non existent “watch” was a precursor to some cloud corruption on my account.  That cloud corruption caused all the authentication for my account to be lost and also corrupted the authentication for my cloud data which only tried to reconnect when I signed back into the OS.

** for anyone who was wondering if I had asked an apple “genius” about this. Yes I did. No they had no clue what I was talking about since most of them are “iphone experts” in store now and the one who called me back seemed to think I made it up.

Lesson learned. Apple iCloud for all but my books is now disabled.

A New Base – A New Hope – A New Beginning. Factory Tours Episode 2

This week I spent time in Milan at HCL’s 2nd factory tour at their offices.   It was an intense couple of days, with presentations from the development teams working on Domino, Notes, Nomad, Sametime, DQL, etc, as well as HCL executives working on building the support program, the partner program, and HCL’s client advocacy program.

After several great discussions with people from HCL and IBM all I can say is that things are moving F-A-S-T and thank you in particular to Richard Jefts, Russ Holden, John Curtis, Tony Blake, Pat Galvin and Francois Nasser for listening to my ideas even if they end up being unworkable.

Note there was no discussion of Connections at all as that deal with IBM is not finalised and HCL aren’t in a position to discuss it. HCL are targeting April 1st for the deal and early June for everything to be transitioned over, with the v11 beta this summer and the v11 launch around end of Q4.

Here are some highlights that I noted from the conversations and sessions that I want to share here.  I apologise if I have incorrectly noted what I heard.

Domino (presented by Russ Holden) – v11 Features 

  • Directory Sync from Active Directory to Domino Directory. Renames made in AD will trigger a Domino rename.  Attempting to get rid of or at least minimise the HTTP password field and make AD authoritative.  We had discussions about whether attributes in AD should/could be written to by Domino or if AD was to be the single authoritative and management source.
  • HTTP password authentication via ID Vault instead of person document, including the ability to keep the Notes password and HTTP password in the vault in sync.*Those of us who work with Traveler know the risk of changing the HTTP password until we get client certificate authentication in Traveler, and we shared that with Russ as well.
  • DAOS will still exist but in v11 there will also be Cloud Object Storage where a single instance of an attachment per note ID will be stored in Amazon’s S3 cloud by default but with the option for extensions to other servers.The concept is that it would potentially save on both on premises storage costs and backup requirements for the attachment store.  It’s not intended to be a space saving offering over DAOS as although there will be one instance of an attachment regardless of cluster replicas, that instance is based on the note id of the document.  That means whereas with DAOS an email sent to 40 people will generate only a single NLO on each cluster server, with this model there would be 40 attachments all accessible by any server in the cluster.  How appropriate this will be as a solution will be dependent on your storage and backup requirements as well as your typical mail usage.
  • A new PubSub feature that will allow applications to subscribe for updates that Domino will publish rather than poll for them.  Traveler is a good example, currently Traveler scans each user’s mail file on a polled interval to see if there are updates and if there are it then grabs them to send to your device. Polling databases asking for “anything new to tell me” is a lot of unnecessary overhead when the alternative is Domino publishing updates each time there is activity (note adds, deletes, folder adds etc).  The ability to subscribe to specific databases you want immediate activity on and for Domino to publish that activity to you as it happens obviously has a huge scope beyond performance outside of Domino as well.Which leads us to…
  • Using ElasticSearch for searching which will utilise the PubSub functionality in order to get immediate updates to process.  ElasticSearch will be configurable on a database-by-database basis including the option to have it take the place of Domino FT search.  One of my issues with ElasticSearch is its security model and they are also working on an API to address that so that we get security parity with the current FT search model.
  • Message recall for undelivered (scheduled or queued) mail.

The Clients (presented by Andrew Davis, Maxx Sutton, Barry Rosen)

The focus for v11 is very much around the client delivery and UI.

There is a new UI under development for Notes and some of its key templates that modernises it and brings it more in line with the UI design of Verse.  These are early stages but they will be part of the v11 ship.  Verse continues to be extended with very welcome upcoming support for mobile browsers and a goal to have parity with iNotes by the end of this year.

HCL Nomad, currently on public beta for iPad (yes, they know we want it SHIPPED) was demoed for Android and ChromeOS at Think and to us this week.  iPhone is also on the way.  To develop for Android HCL used OpenGL and that code can also be compiled as WebGL which, using WebAssembly, will enable Nomad to run in most browsers with the same fidelity and behaviour as on the mobile devices.

I’m not saying that would give us a lightweight client but it would totally give us a lightweight client that could be used in something like HCL Places for instance.

Sametime (presented by Pat Galvin)

Sametime 10 Limited Use, that now includes mobile entitlement, is on track to be delivered in the first half of this year.  The persistent chat feature that allows chats to be routed to multiple devices you are logged into will, in the first instance, require MongoDB with Domino coming “later”.  I have a big problem with this.  Neither me nor my customers want to bring the overhead of MongoDB into a Domino site just for this single feature regardless of how welcome that feature is.  I hope HCL prioritise “later” as “soonest” to be honest otherwise I suspect we’ll be deploying v10 of Sametime initially without its biggest feature.

Platforms will be Windows initially then Linux.  All 64bit.

Sametime will be released lock step with Domino, so at the end of this year Domino 11 will support the product released as Sametime 11.  Targeted for release with v11 is Docker deployment and support for integration with Zoom, Webex, etc. Stretch goals for v11 include getting rid of the Sametime System Console and the ability to invite external guests into chats.

In addition, they are looking to deliver chat enhancements in v11 such as read status on messages, @mentions, and multi-device file transfer so you can select which device a received file is downloaded to as well as choosing which device to answer an audio / video call on if you are logged into multiple devices.

Finally for meetings I’m delighted to hear that they are working to remove the accursed browser plugins for audio and video from v11.

Sametime 12+ includes targets that are stretch goals on v11 and additional targets such as removing WebSphere and DB2.

DQL (Presented by John Curtis)

The 1.0.1 Appdev pack which contains all the functionality you need to deploy DQL from Node is out this quarter.  The plan is to have quarterly updates to the Appdev pack introducing new features.  Some things planned for future updates include:

OAuth authentication.  This is a huge deal and has to be done right.  DQL only works as a solution if we can maintain the same security model that Domino gives us and OAuth has the ability to give us that.   Currently the OAuth implementation in the Appdev pack is application level, meaning one identity shared by anyone using that application which means no reader fields of custom user security.

They are working to support on-the-fly computation of formulas to support things like computed for display fields.

For searching there will be support for both FT Search and the new ElasticSearch with indexes created across databases and in attachments where required.  Searching rich text and mime is also on the agenda.

These are just some of my highlights.  There was a lot more and if you want to get involved I highly recommend registering for Engage UG in Brussels this May (14/15) https://engage.ug.  It’s a free user group event and HCL will be there in force with a lot more to show, hopefully on the heels of some beta content.

If you want to add your own enhancement requests and suggestions definitely to go the aha! site and add them there.  Everyone who spoke said they monitored that site and many of the features that are coming are based on posts there.

https://domino.ideas.aha.io/

HCL Client Advocacy – (presented by John Immerman)

If you are a customer or a business partner please sign up for the Client Advocacy Program @ HCL.  John Paganetti and his team will connect you with a developer advocate who will work with you to make sure your requests are heard, your PMRs don’t stall, and your ideas are taken seriously.  Much of what they do is learning about how you use the tools and what your pain points and wishes are.  This speaks to the core of who HCL are and I can’t think of any other company who would commit skilled expert resources to these kind of relationships.  If you want to be heard go register here. There are 200 companies registered already with hundreds more requested and being personally contacted.  Nothing about this is automated so don’t pass up the opportunity.

https://www.cwpcollaboration.com

Support – (presented by Michael Fiorentino)

The support model is still transitioning over from IBM including hundreds of thousands of technotes and HCL are moving away from the Salesforce support interface that IBM use to a simpler more streamlined one.  The big question I wanted to ask was whether HCL would do away with the login requirement to read technotes or get patches.  Currently IBM require you to be a customer with a support license to be able to read technotes and find out what may be wrong.  I’ve always thought that’s crazy and I know it frustrates customers and partners alike.  Both Richard Jefts and Michael Fiorentino confirmed that is not how they plan to run things and that’s a great start.

Michael also wanted to understand the business partner requirements so he could structure things to make it easier for us to open calls at the right level of expertise and to do so on behalf of our customers in an easier way.  Michael has a frankly astonishing amount of work ahead of him to get the support structure right, but he was very open to all our ideas and comments and I honestly believe they are committed to doing this right and not “business as usual”.

I heard from more people there of really good IBM’ers who are moving to HCL. I don’t want to name them here as that’s their business but I’m delighted they are joining.

Let’s all work together and do great things.

 

 

Think-Ing From Far Away Pt5 – The Big Wrap Up

Our final podcast of Think-ing from far away is a wrap up of the week, announcements and thoughts in general.  We were joined by Maria Nordin from ISW, Kris de Bisschop from Groupwave and Christoph Adler from Panagenda. All dosed up with coffee at 8am on the last day in San Francisco.

The final wrap up podcast is here

Content from Think is already beginning to be posted here  You must have an IBM ID to login and download although you don’t need to be a registered Think attendee.  Some content already available including:

Supercharged Notes 10 Upgrade: Turning the Worst Notes Deployments into the Best from Christoph Adler and Jared Roberts

Domino on Docker Boot Camp from Thomas Hampel (and Daniel Nashed)

Get Started with IBM Connections Customizer for Dummies  from Wannes Rams and Martin Donnelly

Best Practices for Maximizing Your Investment in IBM Verse On-Premises from Drew Birnbaum and Barry Rosen

Using Node-RED to Bring IBM Domino Content into Your Web and Mobile Applications from Scott Good

Register for one of the IBM Connections in person design “jams”

Keep an eye on the Social Connections site for the announcement of where the European even in September this year with a keynote from Richard Jefts will be held.

Follow Twitter

@IBMSocialBiz

@IBMChampions

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info

Think-Ing From Far Away Pt4 – It’s All Connections

Today’s podcast of Think-ing from far away is all about Connections.  We have heard so much news coming out of Think already regarding Connections so we were joined by Sandra Buehler from Belsoft and Wannes Rams from Ramsit – both kindly sat on the floor in a quiet corner of the Moscone Center!.  We were also joined by Chris Reckling who leads the design team in Littleton MA to talk about what’s new and what’s coming.

The Connections podcast is here

What is new with CR4, what’s coming with CR5, how to work with customiser and how to register for the design jams rolling out this year starting in April and May.

A great summary page of all that’s new in Connections

Register for one of the IBM Connections in person design “jams”

Keep an eye on the Social Connections site for the announcement of where the European even in September this year with a keynote from Richard Jefts will be held.

Great to hear so much news from Community and developments continuing to progress fast during the transitiion from IBM to HCL.

Next up: Connections from Think 2019 !

Follow Twitter

⁦@IBM @ Think

@IBMSocialBiz

@IBMChampions

#Think2019

#IBMThink2019

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info

Think-Ing From Far Away Pt3 – Community Day & Chairman’s Address

Today’s podcast of Think-ing from far away is entirely “far away” as we welcome Andreas Ponte from Belsoft in Switzerland and Mike Smith from The Turtle Partnership to discuss the news from Community Day and the Chairman’s Address from Ginni Rometty that we all watched via live stream.

The Community & Chairman’s address podcast is here

What is IBM’s new message and what does that mean to those of us working in the collaboration space?

Replays including the Chairman’s address:

Watch sessions live stream: https://www.ibm.com/events/think/watch/

Think Today “newsdesk

The new HCL Partner Connect Program https://www.cwpcollaboration.com/hcl-partner-connect-registration.html

Domino 11 Sneak Peek registration on March 14th

Register for one of the IBM Connections in person design “jams”

Great to hear so much news from Community and developments continuing to progress fast during the transitiion from IBM to HCL.

Next up: Connections from Think 2019 !

Follow Twitter

⁦@IBM @ Think

@IBMSocialBiz

@IBMChampions

#Think2019

#IBMThink2019

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info

Think-Ing From Far Away Pt2 -News So Far & What’s Coming Up (Tuesday)

Today’s podcast of Think-ing from far away features is an accidental Lifetime Champion full house with guests Paul Withers from Intec and Daniel Nashed from Nashcom alongside Julian, Theo and myself.  We discuss the news so far including work Daniel has been working on providing Docker scripts that is being presented by Thomas Hampel this week and the relaunch of code snippets by OpenNTF.  We also discuss how us far away users are playing along with Think from home by watching live streams and monitoring blogs.

The Penumbra Group were very happy to present this year’s Prism award given to the IBM’er who most helped the BP Community this year to the amazing Mat Newman , Global Executive, IBM Collaboration Solutions.  More details and photo here http://penumbragroup.com.  A big thank you from everyone to you Mat.

Tuesday’s podcast is here

In the podcast we again mention hashtags , blogs and twitter accounts those of us who aren’t at Think should keep an eye on this week but we now also have live stream content and replays

Replays including the Chairman’s address:

Watch sessions live stream: https://www.ibm.com/events/think/watch/

Think Today “newsdesk

Openntf XSnippets

Daniel’s Docker Github where the scripts will be posted

Paul’s Github for Nodered

Download the Think mobile app for iOS or Android to stream content from there

What’s happening for  ICS during Think

Collaboration Sessions Guide

Tomorrow we hope to have two podcasts on both the Chairman’s address and one focused around Connections announcements so stay tuned.

Follow Twitter

⁦@IBM @ Think

@IBMSocialBiz

@IBMChampions

#Think2019

#IBMThink2019

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info

Think-Ing From Far Away Pt1 – Community

Today’s podcast of Think-ing from far away features guests Libby Ingrasia from IBM, Rob Novak from Snapps, John Paganetti from HCL and Femke Goedhart from panagenda alongside Julian Robichaux, Theo Heselmans and myself.  We discuss the ICS Community, what community events are happening at Think and how to play along from home.

Monday’s podcast is here http://www.nsftools.com/tffa/TFFA_1.mp3

In the podcast we mention hashtags , blogs and twitter accounts those of us who aren’t at Think should keep an eye on this week and I wanted to summarise some of those here

Watch sessions live stream including the Chairman’s address on Tuesday here https://www.ibm.com/events/think/watch/

Tomorrow our podcast will be talking about some specific sessions we hope to hear news from this week and the blogs of the people giving them.

Twitter

⁦@IBM @ Think

@IBMSocialBiz

@IBMChampions

#Think2019

#IBMThink2019

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info

 

 

A No-Brainer For Domino Admins

I made this as short as I could – it should take you 3 minutes at most to read.

Do you want to be able to get a free audit report of your Notes clients, including what hardware they are using, what versions of Notes, what memory and disk each machine has, and what databases are on their workspaces?

Would you like to easily set notes.ini, any other ini, and even windows registry settings without the user noticing or being involved?

Would you like to deploy files to client workstations silently?

All of those things are now part of Notes and Domino 10.0.1 free of charge and with virtually no effort on your part.

Many of you will have heard of MarvelClient from Panagenda. Some of you may have heard that a licensed version of MarvelClient is free of charge with Domino 9.0.1 and later.  Why am I only talking about it now? Well MarvelClient now ships with Notes and Domino 10.0.1, which means that if you install either Domino or Notes 10.0.1 then the library files and databases needed to run MarvelClient are already installed for you.

Let me explain, prior to 10.0.1 you needed to deploy a library file to the clients that you wanted to use MarvelClient on, and although that could be done in a variety of ways – including postopen scripts, buttons and mail triggers – most of them involved some degree of user interaction and deployment configuration, and that hurdle was often too high for many customers to take on.  Now that hurdle is gone.

I want to talk about what you get with MarvelClient Essentials and here I’m going to be brief because I want you to deploy it, you want to deploy it, and so I want to be as clear as I can to get you there.  To install MarvelClient Essentials, you do this:

  1. Install Domino 10.0.1 * – that gives you the MC databases in a folder on the server called panagenda.  There is a configuration database for configuring what you want it to do and an Analyze database to show you the results.
  2. Open the desktop policy for your users and add a single notes.ini setting for EXTMGR_ADDINS
  3. Run your Notes 10.0.1 clients and watch the good stuff roll in.

Here’s the IBM whitepaper that goes into a little more detail, but not much because there isn’t much more you need to know to get started, although there’s a lot more you can learn when you’re ready to do more.

So that’s what I did.  I spent less than 10 minutes setting up the ACLs of the two server databases, signing them and updating the desktop policy and immediately the information started to come in.  These are some of the results that showed for Tim and I accessing our development environment using Macs.  What is great about MarvelClient is that it gives me a view and management over the client environment which I can’t see any other way – for instance:-

What directories Notes is installed in, where the program files are and where the notes.ini file is.

Screenshot 2019-02-07 at 21.22.23

Notes.ini settings for each user (note some are “2” where we both have then set and some are “1” where only one of us has it set).  These can be set, changed and deleted by MarvelClient as well.

Screenshot 2019-02-07 at 21.20.14

Notes client preferences, by preference and by person / machine.

Screenshot 2019-02-07 at 21.28.10

Resources, disk capacity / free, memory / free.

Screenshot 2019-02-07 at 21.19.30

Below is the full list of things you can do with Marvelclient Essentials.  On the right is the additional features you can get by upgrading to the Basic version which is chargeable.  I think it’s very clear just how much Panagenda and HCL are providing to you at no cost.  Very few of my customers are able to provide a good audit of their client environment and even fewer able to easily make changes to that environment.   It’s a testament to HCL’s commitment to lowering the TCO of Notes that they have provided all this functionality in 10.0.1.

Now what are you waiting for?

Essentials

Basic

Analyze
Desktop, Bookmarks

notes.ini, User Preferences

Mailfile Details

IBM Notes Version and Installation Information
OS and HW Overview
Local Databases / Replicas
Eclipse and Plugin Details
Eclipse Settings (incl Sametime)
ECL
Server<->Client Latency
ID File Details
Locations, Connections, Accounts, Certificates
HW / SW Inventory
Mail Archives
Windows Application Usage
Configuration
notes.ini and MC Config Variables
Any .ini File
User Preferences
Windows Registry
Up-/Download
Upload Data for Analyze
Upload Backups for Rollback
File Deployment
Smart File Downloader
Roll Back User Configuration
Run
Run Programs
Run Notes Processes
Run Agents
Run Notes Formulas
Copy, Move, Delete Files
Compact Desktop
Manage
Workspace Pages
Desktop Icons
Local Replicas
Replicator Page
Bookmarks & Bookmark Folders
Locations
Connections
ECL
ID File Management
Profile Documents
Switch Location
Migrate
Mass Change to Update Database Links
Mass Delete to Remove Database Links

 

Think-ing From Far Away

As many of you know, next week is IBM Think in San Francisco.  Unfortunately I won’t be able to be there and neither will my fellow Lifetime Champions Julian Robichaux and Theo Heselmans so we wanted to do our own version of “Think” for those of us who are missing out.  Starting next week we plan to record a daily 30 minute audio podcast discussing the news coming out of Think and some of the presentations and sponsors that are there.

IBM have published a list of sessions that will be available for streaming here https://www.ibm.com/events/think/watch/ so we will be watching remotely and we have several guests who will be at Think joining us each day.  This is our attempt to bring some Think and collaboration news to those (including us) who can’t make it.  Here are our planned topics for each day

Monday: The Collaboration Community
Tuesday: Keynote
Tuesday or Wednesday: Showfloor review
Wednesday: Futures and Innovations
Thursday: Domino News
Friday: Wrap up

The podcasts will be published on our blogs each day, We hope you enjoy them.

 

Domino – Exchange On Premises Migration Pt2: Wrestling the Outlook Client

In part 1 of my blog about Exchange on premises migration from Domino I talked about the challenges of working with Exchange for someone who is used to working with Domino.  If only that were all of it but now I want to talk about the issues around Outlook and other Exchange client options that require those of us used to working with Domino to change our thinking.

In Domino we are used to a mail file being on the server and regardless of whether we used Notes or a browser to see that client, the data is the same.  Unless we are using a local replica, but the use of that is very clear when we are in the database as it visibly shows “on Local” vs the server name.

We can also easily swap between the local and server replicas and even have both open at the same time.

In Outlook you only have the option to open a mailbox in either online or cached mode.

So let’s talk about cached mode because that’s the root of our migration pains. You must have a mail profile defined in Windows in order to run Outlook. The default setting for an Outlook profile is “cached mode” and that’s not very visible to the users. The screenshot below is what the status bar shows when you are accessing Outlook in cached mode.

connectedtoexchange

In cached mode there is a local OST file that syncs with your online Exchange mailbox.  It’s not something you can access or open outside of Outlook.

datafiles

Outlook will always use cached mode unless you modify the settings of the data file or the account to disable it.

cachedsettings

As you can see from the configuration settings below, a cached OST file is not the same as a local replica and it’s not designed to be.   The purpose of the cached mail is to make Outlook more efficient by not having everything accessed on the server.

cachedoffline

Why does this matter during a migration?  Most migration tools can claim to be able to migrate directly to the server mailboxes but in practice the speed of that migration is often unworkably slow.  If that can be achieved it’s by far the most efficient but Exchange has its own default configuration settings that work against you doing that including throttling of activity and filtering / scanning of messages.   Many / most migration tools do not expect to migrate “all data and attachments” which is what we are often asked to do.  If what we are aiming for is 100% data parity between a Domino mail file and an Exchange mailbox then migrating that 5GB, 10GB, 30GB volume directly to the server isn’t an option.  In addition if a migration partially runs to the server and then fails it’s almost impossible to backfill the missing data with incremental updates.  I have worked with several migration tools testing this and just didn’t have confidence in the data population directly on the server.

In sites where I have done migrations to on premises servers I’ve often found the speed of migration to the server mailbox on the same network makes migration impossible so instead I’ve migrated to a local OST file.  The difference between migrating a 10GB file to a local OST (about an hour) vs directly to Exchange (about 2.5 days) is painfully obvious. Putting more resources onto the migration machine didn’t significantly reduce the time and in fact each tool either crashed (running as a Domino task) or crashed (running as a Windows desktop task) when trying to write directly to Exchange.

An hour or two to migrate a Domino mail file to a local workstation OST isn’t bad though right?  That’s not bad at all, and if you open Outlook you will see all the messages, folders, calendar entries, etc, all displaying.  However that’s because you’re looking at cached mode. You’re literally looking at the data you just migrated.  Create a profile for the same user on another machine and the mail file will be empty because at this point there is no data in Exchange, only in the local OST.  Another thing to be aware of is that there is no equivalent of an All Documents view in Outlook so make sure your migration tool knows how to migrate unfoldered messages and your users know where to find them in their new mailbox.

Now to my next struggle.  Outlook will sync that data to Exchange.  It will take between 1 and 3 days to do so.  I have tried several tools to speed up the syncing and I would advise you not to bother.  The methods they use to populate the Exchange mailbox from a local OST file sidestep much of the standard Outlook sync behaviours meaning information is often missing or, in one case, it sent out calendar invites for every calendar entry it pushed to Exchange.  I tried five of those tools and none worked 100%. The risk of missing data or sending out duplicate calendar entries/emails was too high.  I opted in the end to stick with Outlook syncing.  Unlike Notes replication I can only sync one OST / Outlook mailbox at a time so it’s slow going unless I have multiple client machines. What is nice is that I can do incremental updates quickly once the initial multi-GB mailbox has synced to Exchange.

So my wrestling with the Outlook client boils down to

  • Create mail profiles that use cached mode
  • Migrate to a local OST
  • Use Outlook to sync that to Exchange
  • Pay attention to Outlook limits, like a maximum of 500 folders*
  • Be Patient

*On Domino mailboxes we migrated that pushed up against the folder or item limits we found Outlook would run out of system memory repeatedly when trying to sync.

One good way to test whether the Exchange data matches the Domino data is to use Outlook Web Access as that is accessing data directly on the Exchange server.  Except that’s not as identical to the server data as we are used to seeing with Verse or iNotes.  In fact OWA too decides to show you through a browser what it thinks you most need to see versus everything that’s there.  Often folders will claim to be empty and that there is no data when in fact that data is there but hasn’t been refreshed by Exchange (think Updall).  There are few things more scary in OWA than an empty folder and a link suggesting you refresh from the server.  It just doesn’t instill confidence in the user experience.

Finally we have Outlook mobile or even using the native iOS mail application.  That wasn’t a separate configuration and unless you configure Exchange otherwise the default is that mobile access will be granted to everyone.   In one instance a couple of weeks ago, mobile access suddenly stopped working for all users who hadn’t already set up their devices.  When they tried to log in they got invalid name or password.  I eventually tracked that down to a Windows update that had changed permissions in Active Directory that Exchange needed set.  You can see reference to the issue here, and slightly differently here, although note it seems to have been an issue since Exchange 2010 and still with Exchange 2016.  I was surprised it was broken by a Windows update but it was.

I know (and have used) many workarounds for the issues I run into but that’s not for here.  Coming from a Domino and Notes background I believe we’ve been conditioned to think in a certain way about mailfile structure, server performance, local data, and the user experience, and expecting to duplicate that exactly is always going to be troublesome.

#DominoForever