Domino Server Health – Monitoring and Managing @ Engage

This is my session on Domino Server Health given at Engage in Brussels last week.

If you’re a Domino administrator how do you decide what to monitor on your servers and how to manage them ? What are the key things to monitor? How do good practice management tools such as statistics reporting, DDM, cluster symmetry, database repair and policy settings make your work lighter and faster. Finally we’ll talk about some of the “must dos” in the day, week and month of a Domino admin.

Face/Off Domino vs Exchange On Premises @ Engage

Here is my presentation discussing how Exchange and Outlook on premises differs from Domino and Notes given at Engage in Brussels last week.

I hope you find it useful, this was my first presentation pulling together my ideas from the past few years of working with Exchange on premises integration projects.

How do Exchange on premises and the various Outlook clients line up against Domino on premises and its clients? In this session we’ll look at the configuration options and management interfaces for each server as well as the client options and client behaviours. We’ll also discuss the general ecosystems, considerations for migrating or co-existing and lessons learned. A great session for Domino admins who want to know more about the other side.

Exchange 2019 On Prem Install

In a couple of weeks time I’ll be in Brussels presenting at Engage and one of my sessions is Face/Off Domino vs Exchange On Premises (Weds at 8am).  I have an Exchange 2016 install but since Exchange 2019 shipped last October I wanted to update my install with that so I could use the latest version to demo.  In truth very little has changed in Exchange on premises since 2008 but I don’t like using an old version in my presentations.  So this is the story of the 4 days it took me to complete the install.

Four. Days.

Day 1: My big mistake.  I decided to uninstall Exchange 2016 instead of upgrading it. I wanted an entirely clean server to demonstrate.  The uninstall failed half way through.  It wouldn’t uninstall and it was still listed under installed programs.  Several hours of trial and error and internet research confirmed this is a common problem with Exchange uninstalls and the “fix” is to flatten the machine and start over.  The problem was the Exchange install was on the same box as the Active Directory 2016 Domain Controller which I really really didn’t want to flatten.

Day 2: Being Stubborn.  I’d do just about anything to avoid flattening the entire box and rebuilding so some more internet research took me to several blogs that talked about manually removing registry entries in order to clean up the install.  Hundreds of registry entries.  After doing that I still couldn’t delete or rename the folder despite no services being present so then it was into safe mode to do the rename.  That worked and I started the upgrade to Windows 2019 (the only supported platform for Exchange 2019). You can now do an inplace Windows upgrade from 2016 to 2019 and that worked maintaining all my Active Directory settings.

Day 3: Accepting the inevitable. Off I go with an Exchange 2019 install once more which started to install then prompted me for the Exchange installer disk.  It wouldn’t take the mounted disk I had started the installer from.  After a few hours’ research I realised this is a common red herring error that basically means the server can detect some old installation files and won’t complete.  At this point there were no services, no directory, nothing listed under installed programs.  Sometimes you have to accept you’ve strayed too many hacks from your starting point it’s best to startover and do it properly.  Windows 2019 install #2 this time letting it blat the server and rebuilding Active Directory from scratch (luckily it’s just my demo machine and I could do that but good luck if it’s your production environment).

At the end of day 3 I had a new Windows 2019 Domain Controller fully patched and I was ready to start my Exchange 2019 install.

Day 4: The Long Road.  Before Exchange will install the installer program will verify you have all the pre-requisites required on the operating system.  There are many from IIS management tools to .Net 4.7.1 to the basic authentication system.  A scrolling page of missing features is shown with URL links explaining them.  Since 90% of those features were actually Windows features you go to add/remove features to install I don’t know why the Exchange installer doesn’t just offer to install them for me because it took some time to work out where in the multi level hierarchy of features each one was.  In addition serveral of the URLs brought up 404 pages on the Microsoft site refering to Exchange 2003 and that link not being available(!).  Anyway finally after a few hours of digging around, downloading libraries, installing features and restarting it agreed to install Exchange 2019 and I was done.

If you take one lesson from this it should be that the Microsoft solution to many problems seems to be “flatten and start over”.  For that reason I wouldn’t put Exchange on any machine you wouldn’t be happy to flatten and start over or replace.

 

More Apple Cloud Issues & The Solution

If you saw my earlier post on my trouble with my  Apple iCloud account it looked by the end of it I had found a resolution (original post here https://turtleblog.info/2019/03/08/the-painful-journey-to-abandoning-icloud/).

In summary it appeared (from my investigation) that my iCloud account somehow got corrupted or modified in such a way that it became unusable and I ended up disabling iCloud storage for documents and desktop amongst other things to fix it.

Except it didn’t entirely fix it.

For the past few weeks I’ve had endless problems with Finder being slow. Spinning ball and up to 10 seconds to change folders, open a fie dialog in any application – longer for dragging and dropping files.  In addition Spotlight was broken, it would let me type a few characters then take around 10 seconds to fill in any suggestions which I then had to wait another 5 seconds or so to click on.

I tried all the fixes I could find including rebuilding Spotlight, removing the CloudMetadata.xml files and resetting preferences.  Things were definitely “usable” but not the way I found acceptable ie. how it used to be.  So last week I had a call with an Apple “genius”.  Two resets, a boot into safe mode (which took 45 minutes and caused hin to exclaim “oh my god”) and finally I caved and reinstalled the OS entirely (which leaves everything else in place).  The guy ended up ghosting me when I pointed out that he wasn’t actually identifying the problem, just hitting it with ever bigger hammers in the hopes it would go away and that if this final hammer (reinstalling the OS) didn’t work he better have a plan.

So this week I was back on my own with a Mac that was still slow and convinced even flattening and rebuiding wouldn’t necessarily help if the issue was related to my iCloud account.  Then I read a few community comments that put me on the right path – I removed all internet accounts from under system preferences.  In my case all I had was my iCloud account.  To remove it it had to save a copy of all my iCloud data locally and remove all my credit cards.  Then I removed the Cloud metadata (rm ~/Library/Caches/CloudKit/CloudKitMetadata*;killall cloudd) and restarted.

I reattached my iCloud account and configured what I wanted to sync, added my credit cards and went to bed.  This morning everything is working as it should including Finder and Spotlight so we’re back to the status before the iCloud corruption.  It seems the single only fix I needed was to remove the iCloud account from system preferences entirely and then re-add it.

 

The Painful Journey To Abandoning iCloud

As some of you know I’m very committed to the Mac ecosystem.  I have Mac laptops, an iPad with over 4000 books, iPhone (not the latest because who needs that), a watch Apple TVs x 4 etc etc.  I’m also extremely risk averse and cloud wary.  I gave in and let Apple put all my books in the cloud just because iTunes sucks for syncing and cloud syncing worked across all my devices however I also had a lengthy open support call last year with Apple wanting to know where my books were now stored on my Mac so I could find them and back them up

“they are all in the cloud”

“yes I get that but they are also on my laptop so where are they”

“no they are only in the cloud”

“well that’s not true because here I go, switching off wifi and hey I can still read my books in ibooks so they are here somewhere”

..>> pause for several weeks whilst this is escalated>>>>

“they are on your Macbook but stored in a way you can’t find them or access them”

(please no advice on this one, I found my own workaround to find them and backup un-DRM copies)

So.. iCloud. I agreed about 18 months’ ago to let my Documents and Desktop folders sync to iCloud.  My only reason for that was so that I could get at files if I needed to on my iPad or by logging into any browser but tbh I rarely used it.  Still it worked and seemed a decent idea.

Then one Saturday about two weeks’ ago it all went horribly wrong..

I was sat working when I got an alert saying facetime had been added to my watch.  Which was odd.  My watch is 18 months old and was on my wrist and nothing had changed. The watch itself had no alert.  So off I go digging and I find under my account and devices a list of my current watch and an old watch I wiped and sold to a friend to give to his wife 18 months ago.  Well still odd but no big deal.  They hadn’t done anything so clearly just an odd gremlin.  Just in case I removed that old watch from my devices.

Then I got alerts saying my credit cards had been removed from my watch.  Except they hadn’t been removed from the watch on my wrist and the other watch was flattened before I handed it over 18 months ago.

I did some research, found nothing nefarious and let it go.   I did notice I had been logged out of all my Apple accounts on all my devices and things like Sonos had to be re-authorised again.  Weird and annoying but a side effect of whatever happened I assume**

Then a few days later I restarted my laptop.  I probably only restart it every two weeks so this was the first time since that alert.  The laptop restarts but finder and anything that uses finder like spotlight or even terminal were entirely non responsive.  They would briefly work long enough for me to type 2 characters or click on a folder then there would be a spinning ball for about 25 seconds before it would respond.  That gradually got slower and slower over a few minutes.  So off we go to research because I now have a broken laptop.

After several hours research we found this article which gave a bit of a clue as it pointed to a cloud corruption problem http://osxdaily.com/2015/04/17/fix-slow-folder-populating-cloudkit-macosx/

Unlike some of the other Finder troubles, the Finder process usually doesn’t eat much CPU or crash repeatedly, it’s just inordinately slow when loading folder views, populating files, and opening folders.

So I followed the instructions and deleted the files they specify and immediately my laptop was more responsive.  OK.. Well that was a scary afternoon and I’ll just go ahead and disable cloud syncing so that never happens again.

Did you know Apple doesn’t let you do that?  If you disable cloud syncing for Documents anbd Desktop it actually deletes the contents of those folders and keeps the files in the cloud for 30 days in case you want them back.  So that’s dumb.  I decided to move the contents of both folders to temporary folders, disable cloud syncing then move them back but my laptop was working and I was busy so I parked that for later.

Later…. about a week later again the Finder sluggishness came back but this time I knew how to fix it.  Once it was fixed I went ahead and moved the contents of both Documents and Desktop to temporary folders, disabled cloud syncing and moved them back.  My laptop immediately started working, finder was faster than it had been for a very long time and I’ve had no more problems.

Now I wonder if that first alert about this non existent “watch” was a precursor to some cloud corruption on my account.  That cloud corruption caused all the authentication for my account to be lost and also corrupted the authentication for my cloud data which only tried to reconnect when I signed back into the OS.

** for anyone who was wondering if I had asked an apple “genius” about this. Yes I did. No they had no clue what I was talking about since most of them are “iphone experts” in store now and the one who called me back seemed to think I made it up.

Lesson learned. Apple iCloud for all but my books is now disabled.

Think-Ing From Far Away Pt5 – The Big Wrap Up

Our final podcast of Think-ing from far away is a wrap up of the week, announcements and thoughts in general.  We were joined by Maria Nordin from ISW, Kris de Bisschop from Groupwave and Christoph Adler from Panagenda. All dosed up with coffee at 8am on the last day in San Francisco.

The final wrap up podcast is here

Content from Think is already beginning to be posted here  You must have an IBM ID to login and download although you don’t need to be a registered Think attendee.  Some content already available including:

Supercharged Notes 10 Upgrade: Turning the Worst Notes Deployments into the Best from Christoph Adler and Jared Roberts

Domino on Docker Boot Camp from Thomas Hampel (and Daniel Nashed)

Get Started with IBM Connections Customizer for Dummies  from Wannes Rams and Martin Donnelly

Best Practices for Maximizing Your Investment in IBM Verse On-Premises from Drew Birnbaum and Barry Rosen

Using Node-RED to Bring IBM Domino Content into Your Web and Mobile Applications from Scott Good

Register for one of the IBM Connections in person design “jams”

Keep an eye on the Social Connections site for the announcement of where the European even in September this year with a keynote from Richard Jefts will be held.

Follow Twitter

@IBMSocialBiz

@IBMChampions

@HCL_CollabDev

@IBMLive

@planetlotus 

IBM Champions- All – List

Blogs

PlanetLotus http://planetlotus.org

IBM Collaboration Solutions Blog https://www.ibm.com/blogs/collaboration-solutions/

HCL Collaboration Workflow Platforms https://www.cwpcollaboration.com/blogs

Aha! Domino Ideas Lab https://domino.ideas.aha.io

Aha! Connections Ideas Lab https://connections.ideas.aha.io

Collaboration Today https://collaborationtoday.info

Other In Person Events Already Announced For 2019

https://engage.ug

https://collabsphere.org

https://admincamp.de

https://dnug.de

https://isbg.no

https://socialconnections.info