Here we go again. My iPad had to be replaced under warranty this week (it was only intermittently charging) so that meant all 4500+ books had to download from the cloud once more. It took about two days but down they came about 50/50 bought from Apple/Amazon. I immediately noticed that several of my books, all of them Amazon epubs, were missing their covers. So…
Off I go to iBooks on my Mac to verify if the covers were still there. They were.
I then try the trick of adding a new cover to a book (just drag and drop an image onto the book in iBooks – that made no difference to the iPad.
Then I tried removing the download on my iPad and redownloading – it again came down with no cover
I tried deleting the books from iBooks entirely and re-adding them – they disappeared from iPad and reappeared. Still with no covers.
Apparently Catalina has broken syncing. If you add epubs to iBooks then have them sync to the cloud and then iOS the covers often disappear. The reverse, syncing from iOS to Catalina works fine. This only applies to non Apple epubs (which is about half my book purchases). Here’s what I did to fix it
Download the Kindle book
Remove the DRM so I can use it / back it up
Drag the book to my iCloud on the Mac
Open “Files” on the iPad and download the book that is now showing in iCloud. iOS will open the downloaded book in iBooks, correctly showing the cover and then sync that back to iBooks cloud and onto my Mac.
It’s not a difficult solution now I know it’s there but it’s concerning how badly Apple are breaking iBooks on OSX when a primary reason for investing in their technology is the cohesive ecosystem.
I found this great short demo video which shows what it is better than I can explain having not touched it myself. Very smart stuff that enables rapid simple application development that can work on mobile, be extended with more complex programming if required, and easily integrated with other systems such as sharepoint or salesforce.
Looking forward to hearing all the news in only 6 hours.
*The below are issues with Apple’s cloud services, I have zero doubt that same issues would occur and do occur with Android and Microsoft. The problem is the concept of cloud storage and DRM*
I have been an Apple user for about 20 years. I like the ecosystem because I do tech support for a living so when it’s my own work and life I don’t want to have to troubleshoot. For years that was true. I had Macs, iPhones and iPads and it’s still my preferred approach, I would stick with the Mac ecosystem over anything else but it no longer “just works”. You may remember my blog earlier this year about issues with the Cloud that broke my Mac until I turned off Cloud syncing entirely
Well that wasn’t the end of my problems. Recently I have been preparing to travel and that meant making sure I had the music I wanted on my phone. Since 2001 Tim has been creating 2 or 3 playlists a year to replace the mixtapes we used to have. I now have over 50 playlists of music that has been important to us in the past few decades. Our music was converted from CDs we owned or sometimes bought digitally and we had over 100GB of it synced.
Note my use of the past tense “had”. Yep I discovered a couple of weeks ago that although my playlists were there they were suddenly all empty. In fact all my music had disappeared and synced that disapperance across every device. I didn’t notice right away because I have been streaming new music recently but when I did notice it was too late to fix it. All my music, all of it, was gone. That’s why the playlists were empty.
Luckily Tim had exported the playlists last year Unfortunately they were txt files and Apple can now only import XML files as playlists. I found a site that can convert text playlists to other formats like Apple Music, Spotify, Youtube etc but when I tried to do that it “matched” the music with stuff in the Apple Music cloud getting the wrong versions of songs or just the wrong songs entirely. Back to square one.
We do have backups but it seems like the music had gradually been disappearing since last April when my cloud account got corrupted. Apple’s level of support at this point comes down to telling you to reformat whatever device you have and “download it from the cloud again”. When I’ve explained it’s no longer in the cloud or anywhere else I get told “that’s not possible”. No point wasting any more time there, I need to fix it myself.
So if you convert playlists to Apple Music , even if they do match correctly, they use Apple music song instances. That means that if i stop paying for Apple music all my music would go away. Once I’m through this disaster I definitely plan to stop paying for Apple music or cloud storage so this never happens again so that’s not an option.
It may be ethically wrong but it felt morally right to explore how to remove the DRM from Apple music tracks that I originally owned before Apple decided to delete them. There are tools out there that do that but in the end I was reluctant and most of our music no longer exists in the versions we bought in the 90s so both unethical and pointless.
Next step, we have an old Mac Mini we use for file storage and never launch iTunes on or have bothered upgrading from Sierra. That had all our music still sat in iTunes since it predated all the cloud syncing. We backed that up onto an external drive and I copied all the tracks into my Apple Music app. That then recognised it as music I own. I could then re-create the playlists using my own music and the text files Tim exported as templates. It’s a manual re-creation but it only takes 5 or 6 minutes per playlist and then I can export them as XML.
I have been watching my Mac upload my music into the Cloud since last night. Still going. I am concerned it’s going to “match” with the wrong versions again but at least my local instances are correct and I can deal with that. As soon as it’s all done I’m cancelling Apple Music and going back to syncing music over wifi with my Mac. The risk is too great that this could happen again.
I go to a lot of trouble to remove DRM off of books so I can back them up and own them. I pay for any content I have and clearly the risk with cloud “don’t worry about it , it just works” is that it doesn’t and if it goes wrong no-one has any clue how to fix it.
Oh and my facetime camera on my iPad is showing the green screen problem as soon as I start a call. Nothing wrong with the camera. This issue has been reported after iOS updates going back to 2013 with no fix. Apple’s response? “format it and reinstall”. I pointed out that would mean downloading over 4k books individually again I was told “that’s all we can recommend”.
As some of you know I buy a lot of books. A LOT. I currently have over 4000 and I like them all downloaded on my phone and iPad, not “ready to download” but actually there. For years I have run into problems with iBooks which doesn’t seem designed to work with the combined models of lots of books that are then downloaded. The last few times I have bought or setup replacement devices (I smash screens a lot) the books wouldn’t download unless I selected each one manually. 4000 times.
This time setting up a new replacement I got a new fun error “this book cannot be downloaded as you don’t have enough storage”. Off to log a genius call I go…
Apple answer 1: Maybe you were out of storage? (Optimistic)
Me: Nope. I have 170GB free
Apple answer 2: Did you backup your previous iPad? (Hopeful I’m an idiot)
Me: Yep, it’s all automatically done and in the cloud and it must be there because it knows what books need to be downloaded
Apple answer 3 (after more research). Oh I see, it’s working as designed. It only downloads new books you order not books you already have in the cloud. (Please don’t ask any more questions , just say thank you and go away)
Me: Really? So it downloads all apps and all music but not all books
Apple: Um, yes? (no that doesn’t sound right does it? damn)
Me: Interesting. Can you send me a link to a page confirming that is the functionality
Apple. Sure. Let me find one. (She’s onto me, there’s no link)
Apple Answer 4 (after more research and a call back). So I can’t find a link but that’s definitely how it works, it won’t download your existing books from the cloud (Sound forceful and confident and she’ll go away)
Me: Except it did. It downloaded a few hundred before stopping and then not doing any more for at least 2 hrs and now I can get it to work by individually selecting each book. All 4000 of them.
Apple: Ah, well it only downloads what you had backed up in the cloud as downloaded on your previous iPad (clutching at straws)
Me: So all of my books then. Because all of them were on my existing iPad
Apple: Oh. Well I will do more research. Are you OK in downloading each one by clicking on it for the remaining books or you could reset and re-setup the iPad and see if it works this time? .. The cloud was very busy yesterday with everyone downloading catalina (that last one at least made me laugh)
I did suggest it at least be escalated to the iBooks development team who could then review how their iCloud downloads are (not) working under load but since I’ve suggested this many times in the past I don’t hold out hope.
Today at Let’s Connect (previously Socal Connections) in Munich, Uffe Sorensen from HCL presented the outline of the new licensing model HCL plan to adopt from v11 of Domino onwards. HCL are looking for feedback on this as they plan to bring all other products into line as things progress.
I apologise in advance for any misunderstandings on my part from Uffe’s presentation but I wanted to highlight the key points as I see them.
Flexnet HCL, as some of you know, are using Flexnet now for software downloads. If you have a license agreement you now download all software and patches from the Flexnet server. In the future (as of v11), you will provision software from the Flexnet server and it will maintain a count of your company’s active sessions.
The cloud-based Flexnet server will be able to track and make visible to you the number of user licenses you are currently consuming. It is not HCL’s intention to disable your services should your license usage exceed what you bought, but they will count “overages” and, after a month’s grace, the plan is to auto-bill those. That would give you a month to bring your licensing back down to the purchased level or be invoiced.
Let’s come back to that in a bit…
Licensing IBM’s licensing model for Domino and Notes was fairly complex. Although there were CEO licenses which were per-user, they had limitations. There were Express licenses which were limited to having only 1000 users in the directory and restricted the number of clustered servers you could have, there was a Utility Server license which allowed unlimited web application access but required you to track your processor usage or be charged for “full capacity”. I could go on but I won’t, let’s leave that in the past.
HCL have committed to two things:
A simpler pricing model
No “sticker shock” for existing customers who will be able to transfer their licensing to this new model.
Model 1: Per User In this model you buy a license for every user or account that will access your servers. Regardless of purpose or client. That’s what Flexnet will track. The user license will allow access to any servers or applications you need. There is no limitation on server use or processor use.
The Per User model can be licensed as a monthly cost or as a persistent license with a S&S renewal. The monthly price was something previously only applied to cloud licenses so that’s now an option for customers who prefer to license that way and maybe have a more dynamic user count.
Model 2: Platform In this model, still very much evolving, instead of licensing a user, you license a server based on its usage and that allows unlimited users access to it. For example the usage may be measured by number of NSFs on each server and the license could be based on a tier level for buying these licenses (such as x hundred NSFs, 1000 NSFs etc) but those haven’t been published (or even decided) yet. In the platform model you pay for server usage not user activity but the measure won’t be around hardware as it was with IBM who measured processor usage, it will likely be by NSF instance, so if you do have 3 servers each with 5 NSFs then that is 15 NSFs out of your license pool.
The platform license will be able to be billed monthly per NSF tier or as a persistent license with annual S&S.
So in summary HCL have attempted to streamline the previous license model to just two options , remove the need for any audits and ensure that you are aware of your license consumption whilst removing the need for you to track things like individual server or processor usage. It’s not a bad approach by any means, and I’ll certainly be glad to see the back of the express licensing and the limited licensing models we had previously that just led to confusion.
Like you I had about 1000 questions, including how the Flexnet licensing monitoring will work for customers that cannot support a cloud connection to the HCL licensing system. For those customers there will be an on-premise Flexnet offering that allows licenses to be tracked internally and subsequently reported to HCL.
The plan is to deploy the cloud-based Flexnet license engine in v11 of Domino and Notes Q4 2019 for per user licensing. As an initial launch this will be cloud tracking only and there will be no overage support / charges. The more fully realised Flexnet licensing including the platform license model and overages will be in 11.0.1 due in Q1 2020.
If you are an existing customer you can today continue to renew your existing licenses on new HCL part numbers but under the same terms you already have.
HCL have emphasised many times that this is an evolving license strategy and they are keen to hear feedback and especially use cases that they may not have allowed for. The details of the licensing including the pricing are still being formulated but this is a sincere attempt to remove previous barriers and confusion around the wide variety of license options, plus the commitment to maintaining license parity (“no sticker shock”) is a good one.
Here’s something I found out from HCL recently that I don’t think is widely known and is really good news for anyone who has let their licenses lapse. Previously if you chose not to renew IBM would require a penalty payment (usually equal to 2.5x what would have been your annual subscription) to re-establish your licenses. That meant once lapsed it became prohibitively expensive to start up again.
HCL does not want penalty payments. If you have let your licenses lapse you can re-activate them by paying the last subscription price you paid plus any % annual increases since then. Basically paying just the current annual renewal price even if you didn’t renew 3 year’s ago.
With the investment in the products HCL have already demonstrated and committed to (one major release a year for example) there are lots of good reasons to re-establish your maintenance. If you previously let your licensing lapse but still have Notes, Domino, Connections or Sametime , renewing your subscription gets you access to the latest versions and support so you can benefit from all the TCO and development enhancements already in v10 as well as what’s coming in v11 this year such as
the Nomad client for tablet and smartphone enabling you to access your existing Notes applications with no code changes,
the app dev pack enabling Node servers to access your Domino data and for web applications to be written using Angular , React or other frameworks accessing Domino data
S3 cloud storage for file attachments that can be shared across multiple servers delivering an even bigger saving in storage than DAOS does currently
A browser based lightweight Notes client that will support all your existing Notes apps with no additional development
Continual development of the web based Verse client for mail, calendar and sametime
A browser based low and pro code development environment
Sametime on mobile entitlement
Hopefully you’ve been hearing all the announcements coming out of HCL since early July In addition to many TCO features already in v10, we have heard what’s coming up for developers in v11 and the work already underway for v12. If you want to know more go here to find detailed presentations from the recent factory tour where HCL showed their current and future plans.
To get a renewal quote you can contact a Business Partner, or reach out to HCL directly
It’s now been several weeks since HCL took full ownership of what were the ICS (IBM Collaboration Solutions) products that include Domino, Sametime, Notes, Verse, Traveler, Portal and Connections. In those few weeks there’s been a lot of activity, not least transitioning over the development and support teams and setting up new systems for support, software access and community news.
You’ve probably seen many of the announcements or even attended the multiple webcasts but here are a few in case you missed them.
Along with the owernship of Notes/Domino, Sametime, Connections, LEAP (previously FEB), you can see all the planned integration points including Rest APIs, docker, Zoom, Salesforce and more. The Digital Solutions story is one that connects all their products together and understands the importance of other applications and services to the whole.
If you’re an existing customer you are going to want to register for both a customer account and a support account. Right now HCL don’t have single sign-on across their sites so you do need to register an account for each service. I’m not saying you “should” but you “could” use the same login and password everywhere.
To register for your software licenses and downloads (which HCL will have been given a record of by IBM) go here
HCL have also announced the first drop of the v11 beta which will be for Notes (Windows and Mac), Domino (Windows and Linux), Designer (Windows) and the embedded Sametime client. All of these in English language strings only on Sept 16th.
If you are interested in licensing, that has also transitioned to HCL entirely. Although I know of a few customers receiving letters from IBM, they are no longer entitled to sell your renewal or additional licenses. If you have any questions about licensing reach out to your Business Partner or visit the eCommerce portal online (you’ll need to register as a customer first) https://buy.hcltechsw.com
HCL have also committed to delivering v12 of the products in Q4 2020 so we are on schedule for one major release (and interim smaller releases) a year with v11 due in Q4 2019.
If the idea of upgrading all your clients it a bit overwhelming, don’t forget you have an entitlement to use Panagenda’s MarvelClient Essentials to upgrade from v9 at no charge. MarvelClient Essentials is built into Notes 10.0.1 and later but you can download it here if you need to get to that point.
In my previous blogs I discussed installing Sametime 10. My installs all ran perfectly except for one point – when I tried to install MongoDB as a service it installed but then wouldn’t start with the service error “1053”, reported as “service did not respond to the start or control request in a timely fashion”.
I thought I had fixed the problem by using another method to install Mongo. From my previous blog:
This is what the documentation said to use (sc is found in c:\windows\system32 if your path can’t find it) sc.exe create MongoDB binPath=”\”C:\Program Files\MongoDB\Server\3.6\bin\mongod.exe\” –service — config=\”C:\Program Files\MongoDb\Server\3.6\mongod.cfg”” DisplayName= “MongoDB” start= “auto”
I ended up removing that service since it wouldn’t start (sc delete MongoDB) and adding it using a different syntax from the Mongo bin directory itself
Turns out I should have stuck with the documentation and using “sc” but the 1053 error was caused by two problems – the first was a misprint in the documentation as you can see above, there are two “” after mongod.cfg instead of one so that line should read (with added escape character)
the second problem was in the mongod.cfg file so when Mongo tried to start and read it, it failed.
I had copied the contents of the Mongod.cfg from the documentation into a text file so I didn’t consider that would be an issue but the pasted content removed the spaces and tabs at the beginning of each line in the documentation and that meant my mongod.cfg file wouldn’t work. This is what it should look like, spaces and tabs included.
Once that was complete everything worked perfectly. allowing the service to start in the way Sametime wanted and allowed me to run the command “rs.initiate()” from the Mongo console.
Thank you to Tony Payne @ HCL for working with me on this last week.
The Sametime Proxy Server is needed if you have a Community server you want people to login to using either a web or mobile client, this includes having Sametime awareness for other products such as Verse or Connections. Since I had already installed a Community Server without using a System Console I now wanted to add a Sametime Proxy Server with as minimal additional software as possible.
The Sametime Proxy server uses a DB2 database for iOS push notifications. If you don’t create that database the server will work in every other way but you won’t have working push notifications to iOS devices. You can create the database on any DB2 10.x or 11.x server but if you don’t have DB2 your Sametime license includes entitlement to a limited for Sametime only use license of DB2 10.5 (at least that’s what is in the download).
To start we need to install Installation Manager. Installation Manager is a package management software that will be used to install WebSphere and the Sametime Proxy Server on top of it. The Sametime download comes with version 1.6.2 but you are going to want to upgrade that to at least 1.8.6 before installing anything. Once Installation Manager is installed we need to install WebSphere 8.5.5 fixpack 14. Don’t worry about the fixpack download for now, if you have internet access during the install that can be done automatically so you will want to start by extracting each of the three WebSphere installers to a directory on your file system. It doesn’t matter which one. I extracted mine to C:\Software\WAS855. This is just the install directory so you don’t need to keep it once the install is complete.
Now we need to add a repository under Installation Manager so it can find the installer for WebSphere. The installer file is always repository.config so I added a respository (File – Preferences) and pointed to c:\software\was855\repository.config
Once the repository is added I can choose “Install” from the main Installation Manager menu and I will be presented with the option to install WebSphere 8.5.5. As you can see it has automatically checked Version 188.8.131.52 and I have checked “Show all versions” so I can install Fixpack 14 directly during this one install step.
Now the WebSphere install is complete we can proceed to installing the Sametime Proxy Server. In my design it’s a standalone server so has no deployment manager or System Console. The instructions to do this require us to perform a silent install which means using a response file specifying the configuration that is accessed from outside the Installation Manager GUI.
The extracted installer for the Sametime Proxy Server contains a response file we can edit for our own install. My response file was in
C:\Software\SametimeProxyServer\responseFiles and was called ip.rsp.
I took a copy of that file and made my own changes – see the bottom of this blog for my ip.rsp file but one very important consideration is that in the sample ip.rsp the default password of “wasadmin” is set. If you run the install without changing that you will end up with a login of wasadmin and a password of wasadmin.
zN39fpCc9SqIryGJM7+02A== is the encoded value of “wasadmin”, you must enter the encoded value of the password you want to set in the response file. To generate the encoded value before adding it to the response file you use the “generateEncodedPassword” utility which can be found in your Sametime Proxy Server extracted install e.g.
From the IBM Installaton Manager install directory (C:\IBM\Installation Manager for me) I ran
C:\Software\SametimeProxyServer\GeneratedEncodedPassword\GenerateEncodedPassword.bat gabpass >pw.txt – which generates the encoded value for “gabpass” and writes it to the pw.txt text file as “rNFGfQ/sPi082+AaXFMNxA==. I write the value to a text file so I can have it recorded whilst I complete the next step.
I can then modify the value of <data key=’user.com.ibm.lotus.sametime.proxyserver.was.password’ value=’rNFGfQ/sPi082+AaXFMNxA==’/>
Important note; do not try and use special characters in your WebSphere passwords especially @ and !.
Now we are ready to run the silent install from the eclipse directory where Installation Manager is installed e.g c:\IBM\Installation Manager\eclipse using the command
With the install completed successfully we can sign onto the node via https://hostname:9043/ibm/console using the credentials we set in our rsp install file and see the Sametime Proxy Server in place. The community server that the Sametime Proxy connects to was specified in my response file when I did the install.
In my environment I wanted to have the Sametime Proxy Server on port 443 securely and to install my own SSL certificate. To modify the ports for the Samtime Proxy Server click on the server name and choose “ports” under “Communication” then edit the value for “wc_defaulthost_secure”.
Once this step is complete you should be able to connect to Sametime through a mobile client or through the web using the new web client on https://hostname:port/chat
Once the server is installed any changes to the configuration such as which Community server or cluster it should connect to should be changed by directly editing the stproxyconfig.xml file.
I could do that but then I hit a problem in that anyone accessing Sametime via the Proxy Server could send but not receive messages or see any they sent. My next blog “Issues With MongoDB During Sametime Install” explains what happened and how HCL helped me identify and fix it.
For the official install instructions from HCL please see this PDF
My ip.rsp file is shown below as an example. You will need to modify it for your own install.
<?xml version="1.0" encoding="UTF-8"?>
<!--The "acceptLicense" attribute has been deprecated. Use "-acceptLicense" command line option to accept license agreements.-->
The following two XML elements specify the host and port of the Sametime community server that
you wish to connect to with this Proxy server. You will need to alter it. The host name needs to be the FQDN of the CS The default port to use is 1516, because this is the default port for Virtual Places
server to server communication (1533 is the default for client to server communication_
If you specified that you wish to install a WAS secondary node profile,
you must edit these attributes to match the host name and soap port of the Deployment manager you wish to have your WAS profile managed by If this secondary node is a standalone node, then edit these attributes to match the secondary node's hostname and soap port values.
Uncomment this attribute if you wish the WAS profile to use a hostname other than the default for this machine
if you do specify a hostname it should be the fully qualified domain name (e.g. "servername.domain.com" not "servername") <data key='user.com.ibm.lotus.sametime.proxyserver.hostname' value='HOSTNAME'/>
The WAS profiles created by the installer will be protected by File based security
for the System Console. The following two XML elements specify the username & password
that can be used to login to the WAS System Console.It is not allowed to omit these elements.
For security reasons you do not store the password in plain text. Instead you can use
the generateEncodedPassword utility contained in the kit to generated an encoded form of the password
For example 'zN39fpCc9SqIryGJM7+02A==' is equivalent to a password of 'wasadmin' It is not possible to choose LDAP security during install, but this can be