Remember when Apple “just worked”

*The below are issues with Apple’s cloud services, I have zero doubt that same issues would occur and do occur with Android and Microsoft. The problem is the concept of cloud storage and DRM*

I have been an Apple user for about 20 years. I like the ecosystem because I do tech support for a living so when it’s my own work and life I don’t want to have to troubleshoot. For years that was true. I had Macs, iPhones and iPads and it’s still my preferred approach, I would stick with the Mac ecosystem over anything else but it no longer “just works”. You may remember my blog earlier this year about issues with the Cloud that broke my Mac until I turned off Cloud syncing entirely

https://turtleblog.info/2019/04/08/more-apple-cloud-issues-the-solution/

Well that wasn’t the end of my problems. Recently I have been preparing to travel and that meant making sure I had the music I wanted on my phone. Since 2001 Tim has been creating 2 or 3 playlists a year to replace the mixtapes we used to have. I now have over 50 playlists of music that has been important to us in the past few decades. Our music was converted from CDs we owned or sometimes bought digitally and we had over 100GB of it synced.

Note my use of the past tense “had”. Yep I discovered a couple of weeks ago that although my playlists were there they were suddenly all empty. In fact all my music had disappeared and synced that disapperance across every device. I didn’t notice right away because I have been streaming new music recently but when I did notice it was too late to fix it. All my music, all of it, was gone. That’s why the playlists were empty.

Luckily Tim had exported the playlists last year Unfortunately they were txt files and Apple can now only import XML files as playlists. I found a site that can convert text playlists to other formats like Apple Music, Spotify, Youtube etc but when I tried to do that it “matched” the music with stuff in the Apple Music cloud getting the wrong versions of songs or just the wrong songs entirely. Back to square one.

We do have backups but it seems like the music had gradually been disappearing since last April when my cloud account got corrupted. Apple’s level of support at this point comes down to telling you to reformat whatever device you have and “download it from the cloud again”. When I’ve explained it’s no longer in the cloud or anywhere else I get told “that’s not possible”. No point wasting any more time there, I need to fix it myself.

So if you convert playlists to Apple Music , even if they do match correctly, they use Apple music song instances. That means that if i stop paying for Apple music all my music would go away. Once I’m through this disaster I definitely plan to stop paying for Apple music or cloud storage so this never happens again so that’s not an option.

It may be ethically wrong but it felt morally right to explore how to remove the DRM from Apple music tracks that I originally owned before Apple decided to delete them. There are tools out there that do that but in the end I was reluctant and most of our music no longer exists in the versions we bought in the 90s so both unethical and pointless.

Next step, we have an old Mac Mini we use for file storage and never launch iTunes on or have bothered upgrading from Sierra. That had all our music still sat in iTunes since it predated all the cloud syncing. We backed that up onto an external drive and I copied all the tracks into my Apple Music app. That then recognised it as music I own. I could then re-create the playlists using my own music and the text files Tim exported as templates. It’s a manual re-creation but it only takes 5 or 6 minutes per playlist and then I can export them as XML.

I have been watching my Mac upload my music into the Cloud since last night. Still going. I am concerned it’s going to “match” with the wrong versions again but at least my local instances are correct and I can deal with that. As soon as it’s all done I’m cancelling Apple Music and going back to syncing music over wifi with my Mac. The risk is too great that this could happen again.

I go to a lot of trouble to remove DRM off of books so I can back them up and own them. I pay for any content I have and clearly the risk with cloud “don’t worry about it , it just works” is that it doesn’t and if it goes wrong no-one has any clue how to fix it.

Oh and my facetime camera on my iPad is showing the green screen problem as soon as I start a call. Nothing wrong with the camera. This issue has been reported after iOS updates going back to 2013 with no fix. Apple’s response? “format it and reinstall”. I pointed out that would mean downloading over 4k books individually again I was told “that’s all we can recommend”.

Apple – it just works, except when it doesn’t.

Apple iBooks – The Battle Contiunes

As some of you know I buy a lot of books. A LOT. I currently have over 4000 and I like them all downloaded on my phone and iPad, not “ready to download” but actually there. For years I have run into problems with iBooks which doesn’t seem designed to work with the combined models of lots of books that are then downloaded. The last few times I have bought or setup replacement devices (I smash screens a lot) the books wouldn’t download unless I selected each one manually. 4000 times.

This time setting up a new replacement I got a new fun error “this book cannot be downloaded as you don’t have enough storage”. Off to log a genius call I go…

Apple answer 1: Maybe you were out of storage? (Optimistic)

Me: Nope.  I have 170GB free

Apple answer 2: Did you backup your previous iPad? (Hopeful I’m an idiot)

Me: Yep, it’s all automatically done and in the cloud and it must be there because it knows what books need to be downloaded

Apple answer 3 (after more research).  Oh I see, it’s working as designed.  It only downloads new books you order not books you already have in the cloud. (Please don’t ask any more questions , just say thank you and go away)

Me: Really?  So it downloads all apps and all music but not all books

Apple:  Um, yes? (no that doesn’t sound right does it? damn)

Me: Interesting.  Can you send me a link to a page confirming that is the functionality

Apple.  Sure.  Let me find one. (She’s onto me, there’s no link)

Apple Answer 4 (after more research and a call back).  So I can’t find a link but that’s definitely how it works, it won’t download your existing books from the cloud (Sound forceful and confident and she’ll go away)

Me:  Except it did.  It downloaded a few hundred before stopping and then not doing any more for at least 2 hrs and now I can get it to work by individually selecting each book.  All 4000 of them.

Apple:  Ah, well it only downloads what you had backed up in the cloud as downloaded on your previous iPad (clutching at straws)

Me: So all of my books then. Because all of them were on my existing iPad

Apple: Oh. Well I will do more research. Are you OK in downloading each one by clicking on it for the remaining books or you could reset and re-setup the iPad and see if it works this time? .. The cloud was very busy yesterday with everyone downloading catalina (that last one at least made me laugh)

I did suggest it at least be escalated to the iBooks development team who could then review how their iCloud downloads are (not) working under load but since I’ve suggested this many times in the past I don’t hold out hope.

2200 done and counting… click click click click

News for Ex Customers of Domino, Notes, Connections….

Here’s something I found out from HCL recently that I don’t think is widely known and is really good news for anyone who has let their licenses lapse. Previously if you chose not to renew IBM would require a penalty payment (usually equal to 2.5x what would have been your annual subscription) to re-establish your licenses. That meant once lapsed it became prohibitively expensive to start up again.

HCL does not want penalty payments. If you have let your licenses lapse you can re-activate them by paying the last subscription price you paid plus any % annual increases since then. Basically paying just the current annual renewal price even if you didn’t renew 3 year’s ago.

With the investment in the products HCL have already demonstrated and committed to (one major release a year for example) there are lots of good reasons to re-establish your maintenance. If you previously let your licensing lapse but still have Notes, Domino, Connections or Sametime , renewing your subscription gets you access to the latest versions and support so you can benefit from all the TCO and development enhancements already in v10 as well as what’s coming in v11 this year such as

  • the Nomad client for tablet and smartphone enabling you to access your existing Notes applications with no code changes,
  • the app dev pack enabling Node servers to access your Domino data and for web applications to be written using Angular , React or other frameworks accessing Domino data
  • S3 cloud storage for file attachments that can be shared across multiple servers delivering an even bigger saving in storage than DAOS does currently
  • A browser based lightweight Notes client that will support all your existing Notes apps with no additional development
  • Continual development of the web based Verse client for mail, calendar and sametime
  • A browser based low and pro code development environment
  • Sametime on mobile entitlement

Hopefully you’ve been hearing all the announcements coming out of HCL since early July In addition to many TCO features already in v10, we have heard what’s coming up for developers in v11 and the work already underway for v12. If you want to know more go here to find detailed presentations from the recent factory tour where HCL showed their current and future plans.

To get a renewal quote you can contact a Business Partner, or reach out to HCL directly

customersuccess@hcl.com

Or the sales directors for each region

EMEA Paul Mason (paul.mason@hcl.com)

North America Peter Phillips (peter.phillips@hcl.com)

APAC Sushovan Chatterjee (SushovanC@hcl.com)

Onboarding With HCL

It’s now been several weeks since HCL took full ownership of what were the ICS (IBM Collaboration Solutions) products that include Domino, Sametime, Notes, Verse, Traveler, Portal and Connections. In those few weeks there’s been a lot of activity, not least transitioning over the development and support teams and setting up new systems for support, software access and community news.

You’ve probably seen many of the announcements or even attended the multiple webcasts but here are a few in case you missed them.

The new HCL division that holds responsiblity for these products is called Digital Solutions and their homepage for all HCL DS (not sure they use that abbreviation) activity is here https://www.cwpcollaboration.com. The blog you will want to follow is at https://www.cwpcollaboration.com/blogs.

So what does HCL Digital Solutions look like?

Along with the owernship of Notes/Domino, Sametime, Connections, LEAP (previously FEB), you can see all the planned integration points including Rest APIs, docker, Zoom, Salesforce and more. The Digital Solutions story is one that connects all their products together and understands the importance of other applications and services to the whole.


If you’re an existing customer you are going to want to register for both a customer account and a support account. Right now HCL don’t have single sign-on across their sites so you do need to register an account for each service. I’m not saying you “should” but you “could” use the same login and password everywhere.

To register as a customer go here

To register for support go here

To register for your software licenses and downloads (which HCL will have been given a record of by IBM) go here

HCL have also announced the first drop of the v11 beta which will be for Notes (Windows and Mac), Domino (Windows and Linux), Designer (Windows) and the embedded Sametime client. All of these in English language strings only on Sept 16th.

To register for the first beta drop you must sign up here by September 16th, after that date you will be added to the test group for the second beta drop https://registration.hclpartnerconnect.com/D11Beta

If you are interested in licensing, that has also transitioned to HCL entirely. Although I know of a few customers receiving letters from IBM, they are no longer entitled to sell your renewal or additional licenses. If you have any questions about licensing reach out to your Business Partner or visit the eCommerce portal online (you’ll need to register as a customer first) https://buy.hcltechsw.com

HCL have also committed to delivering v12 of the products in Q4 2020 so we are on schedule for one major release (and interim smaller releases) a year with v11 due in Q4 2019.

If the idea of upgrading all your clients it a bit overwhelming, don’t forget you have an entitlement to use Panagenda’s MarvelClient Essentials to upgrade from v9 at no charge. MarvelClient Essentials is built into Notes 10.0.1 and later but you can download it here if you need to get to that point.

So the question is, have you deployed v10 yet ?

Problems With MongoDB During Sametime Install

In my previous blogs I discussed installing Sametime 10. My installs all ran perfectly except for one point – when I tried to install MongoDB as a service it installed but then wouldn’t start with the service error “1053”, reported as “service did not respond to the start or control request in a timely fashion”.

I thought I had fixed the problem by using another method to install Mongo. From my previous blog:

This is what the documentation said to use (sc is found in c:\windows\system32 if your path can’t find it)
sc.exe create MongoDB binPath=”\”C:\Program Files\MongoDB\Server\3.6\bin\mongod.exe\” –service — config=\”C:\Program Files\MongoDb\Server\3.6\mongod.cfg”” DisplayName= “MongoDB” start= “auto”

I ended up removing that service since it wouldn’t start (sc delete MongoDB) and adding it using a different syntax from the Mongo bin directory itself

mongod –directoryperdb -dbpath C:\data\mongodb\ –logpath C:\data\mongodb\log\mongo.log –logappend –service –install

Turns out I should have stuck with the documentation and using “sc” but the 1053 error was caused by two problems – the first was a misprint in the documentation as you can see above, there are two “” after mongod.cfg instead of one so that line should read (with added escape character)

sc.exe create MongoDB binPath=”\”C:\Program Files\MongoDB\Server\3.6\bin\mongod.exe\” –service — config=\”C:\Program Files\MongoDb\Server\3.6\mongod.cfg\”” DisplayName= “MongoDB” start= “auto”

the second problem was in the mongod.cfg file so when Mongo tried to start and read it, it failed.

I had copied the contents of the Mongod.cfg from the documentation into a text file so I didn’t consider that would be an issue but the pasted content removed the spaces and tabs at the beginning of each line in the documentation and that meant my mongod.cfg file wouldn’t work. This is what it should look like, spaces and tabs included.

Once that was complete everything worked perfectly. allowing the service to start in the way Sametime wanted and allowed me to run the command “rs.initiate()” from the Mongo console.

Thank you to Tony Payne @ HCL for working with me on this last week.

Domino Query Language @ Engage

By Tim Davis, Technical Director

This is my session given at Engage 2019 in Brussels last week.

“In this session, Tim Davis (Technical Director at The Turtle Partnership Ltd) takes you through the new Domino Query Language (DQL), how it works, and how to use it in LotusScript, in Java, and in the new domino-db Node.js module. Introduced in Domino 10, DQL provides a simple, efficient and powerful search facility for accessing Domino documents. Originally only used in the domino-db Node.js module, with 10.0.1 DQL also became available to both LotusScript and Java. This presentation will provide code examples in all three languages, ensuring you will come away with a good understanding of DQL and how to use it in your projects.”

Exchange 2019 On Prem Install

In a couple of weeks time I’ll be in Brussels presenting at Engage and one of my sessions is Face/Off Domino vs Exchange On Premises (Weds at 8am).  I have an Exchange 2016 install but since Exchange 2019 shipped last October I wanted to update my install with that so I could use the latest version to demo.  In truth very little has changed in Exchange on premises since 2008 but I don’t like using an old version in my presentations.  So this is the story of the 4 days it took me to complete the install.

Four. Days.

Day 1: My big mistake.  I decided to uninstall Exchange 2016 instead of upgrading it. I wanted an entirely clean server to demonstrate.  The uninstall failed half way through.  It wouldn’t uninstall and it was still listed under installed programs.  Several hours of trial and error and internet research confirmed this is a common problem with Exchange uninstalls and the “fix” is to flatten the machine and start over.  The problem was the Exchange install was on the same box as the Active Directory 2016 Domain Controller which I really really didn’t want to flatten.

Day 2: Being Stubborn.  I’d do just about anything to avoid flattening the entire box and rebuilding so some more internet research took me to several blogs that talked about manually removing registry entries in order to clean up the install.  Hundreds of registry entries.  After doing that I still couldn’t delete or rename the folder despite no services being present so then it was into safe mode to do the rename.  That worked and I started the upgrade to Windows 2019 (the only supported platform for Exchange 2019). You can now do an inplace Windows upgrade from 2016 to 2019 and that worked maintaining all my Active Directory settings.

Day 3: Accepting the inevitable. Off I go with an Exchange 2019 install once more which started to install then prompted me for the Exchange installer disk.  It wouldn’t take the mounted disk I had started the installer from.  After a few hours’ research I realised this is a common red herring error that basically means the server can detect some old installation files and won’t complete.  At this point there were no services, no directory, nothing listed under installed programs.  Sometimes you have to accept you’ve strayed too many hacks from your starting point it’s best to startover and do it properly.  Windows 2019 install #2 this time letting it blat the server and rebuilding Active Directory from scratch (luckily it’s just my demo machine and I could do that but good luck if it’s your production environment).

At the end of day 3 I had a new Windows 2019 Domain Controller fully patched and I was ready to start my Exchange 2019 install.

Day 4: The Long Road.  Before Exchange will install the installer program will verify you have all the pre-requisites required on the operating system.  There are many from IIS management tools to .Net 4.7.1 to the basic authentication system.  A scrolling page of missing features is shown with URL links explaining them.  Since 90% of those features were actually Windows features you go to add/remove features to install I don’t know why the Exchange installer doesn’t just offer to install them for me because it took some time to work out where in the multi level hierarchy of features each one was.  In addition serveral of the URLs brought up 404 pages on the Microsoft site refering to Exchange 2003 and that link not being available(!).  Anyway finally after a few hours of digging around, downloading libraries, installing features and restarting it agreed to install Exchange 2019 and I was done.

If you take one lesson from this it should be that the Microsoft solution to many problems seems to be “flatten and start over”.  For that reason I wouldn’t put Exchange on any machine you wouldn’t be happy to flatten and start over or replace.