Ideas, Demos & Your Last Day To Sign Up for Beta 2

So much interesting activity going on around the IBM/HCL products so in case you missed them I thought I could summarise for you.  All are worthy of your time if you care about the future of Domino, Traveler, Verse or Sametime

BETA

Firstly – no time to lose – the registration for Beta 2 of Domino , Notes and Traveler closes TODAY at 12pm EST/5pm GMT.  If you want access to that Beta due this month hopefully then go and sign up here now https://www.ibm.com/blogs/collaboration-solutions/2018/06/11/announcing-ibm-domino-v10-portfolio-beta-program-sign-today/.  Don’t leave it then be disappointed when you don’t get access.

IDEAS

If you have ideas for what you want in Domino, Notes, Traveler, Sametime or anything else – there is a new site (requiring no login) where you can add your ideas and vote on other people’s.  It’s been running for a few weeks and there are some great ideas there already to vote for so it’s a good place to browse during your next coffee break.  Remember the rule – if you don’t ask you don’t get https://domino.ideas.aha.io/ideas

DEMOS

HCL are publishing a series of videos showing how features that are in v10 will behave.  Here are three interesting features announced so far.

Definitely different – a few days looking into the future with HCL (and IBM)

If this blog is tl:dr then here’s your takeaway

I can’t thank everyone at HCL enough for throwing open the doors and leaving them open. Together we will continue to innovate great things for customers

Last week Tim and I were invited to the 1st CWP Factory tour held by HCL at their offices in Chelmsford.  “CWP” stands for “Collaboration Workflow Platform” and includes not only the products HCL took over from IBM late last year such as Domino, Traveler, Verse on Premises and Sametime but also new products that HCL are developing as extensions of those.  These (that I can talk a little bit about) such as HCL Nomad (Notes for iPad) and HCL Places (a new client runnvetaing against Domino 10 and providing integrated collaborative services such as chat, AV , web and Notes applications) will be leapfrogging Domino far over its competitors.

I want to start by thanking HCL for inviting us inside to see their process. We met and made our voices heard with more than 30 developers and executives, all of who wanted to know “do you like this?”  “what are we missing?”.  I came away from the two days with a to-do list of my own at the request of various people to send in more details of problems or requirements I had mentioned when there.  John Paganetti, who is also a customer advocate at HCL, hosted the “ask the developers” impromptu session (we had so many questions so they threw one into the agenda on day 2).  We were told to get to know and reach out directly to the teams with our feedback and questions.  If you don’t have a route to provide feedback and want one then please reach out.

Back in February I attended a Domino Jam hosted by Andrew Manby (@andrewmanby) from IBM in London.  These were held all over the world and attendees were pushed to brainstorm around features that were missing or needed.  That feedback was used to create priorities for v10 and many of the features requested at my session and others have appeared in the current beta and are committed to a v10 release.  At the end of the 2nd day of the factory tour we again had a Domino Jam hosted by Andrew Manby but this time for Domino 11 features – wheeeeeeee!   With the Jams and the Destination Domino blog as well as the #domino2025 hashtag activity, IBM are really stepping up to the products in a way they haven’t in several years.  I want to recognise the hard work being done by Andrew, by Uffe Sorensen, and by Mat Newman amongst others, to make this IBM/HCL relationship work.

So what was the factory tour? It was a 2 day conference held at HCL’s (still being built) offices. I am pleased to say it was put together very informally, we were split into groups of about 10 (hi Daniel, Francie, Julian, Richard, Paul, Nathan, Devin, Fabrice!) and one by one the development teams came and took our feedback on the work they are doing.  We worked with the Verse (on premises) team, the TCO team (looking at the Domino and Sametime servers), the Notes client team, the Nomad team and the Application Development team.  It was an intense day in a good way with so much information being shared with us and questions being asked of us.  It was also good to be told that the majority of what we saw and discussed could be shared publicly.

A few highlights (out of many) from the two days that were new to me:

  • The new database repair and folder sync features in Domino 10 (shame on me for not remembering what they are called). The database repair feature will detect when a database is corrupted and replace it whilst the server is running with a new instance from a working cluster mate (another good reason to cluster).  The folder sync feature will keep any  Domino database files or NLOs in any listed folders in sync.  This stuff is so cool and exactly what Domino clustering needed so we asked for them to extend the sync feature to include any files in the HTML directory such as HTML CSS and CGI scripts and they are considering that (v10 is a tight delivery timeline right now so no guarantees of anything).
  • Some very candid discussions (I think repeated multiple times by everyone there) about getting rid of WebSphere for Sametime in the future and how to better provide Sametime services purely under Domino.
  • HCL Places looking much evolved even in the few weeks since it was first shown at Engage – this is going to be a game changer client when it comes out.
  • The Domino General Query Facility (DGQF) available in Domino 10 is the biggest investment in Notes/Domino code in 10 years. A query language accessible outside Domino that doesn’t require any  knowledge of Domino design by a developer.  Using DGQF you can rapidly query collections of documents represented by any criteria not necessarily views or forms.  Using DGQF a regular web developer would be able to build a Node application, for instance, using back-end Domino data without ever having to learn the structure of the Domino database or touch Domino Designer.  Here’s a sneaky picture I took of the positioning for DGQF.John Curtis who is the lead designer behind DGQF has been very responsive on twitter to questions about how it will work (@john_d_curtis)IMG_0313
  • A lot of stuff Nomad and Node related which is still NDA but you’ll hear more about them at Collabsphere in Ann Arbor – HCL will be out in force as will IBM speaking, showing and listening so if you can you need to get yourself there.   Turn out and turn up – there’s still time to get your voice heard.

 

The Champion & Confidence Dilemma

I wanted to share today something I’ve been dealing with for a few months and inspired by shares from others.  For those of you who don’t know the IBM Champion program, in short it was set up to acknowledge the work done by people who contribute to their Community outside of their regular jobs.

When I started as a business partner in the mid 90s the IBM community I was introduced to was full of people interested in IBM technology, wanting to learn and wanting to share what they knew with others for no reason other than they were excited about it and enjoyed seeing others doing the same.  In the past 20 years a lot of that has changed and I miss those days.  There are still lots of people who share and want to learn but the days of not wanting credit or taking a back seat are often (not always) gone.

I was encouraged and inspired for 20 years by people many of you will have heard of and many of you wouldn’t.  Without Andrew Pollack to tell me I was smart enough to learn this stuff and present, or Chris Miller offering to present wtih me or Rocky Oliver encouraging me to write, or Ben Langhinrichs asking the tough business questions about why I don’t charge more, or Carl Tyler giving me no leeway to make excuses, or Paul Mooney who was as enthusiastic about educating as I was and happy to work with me – without those people and many more in Penumbra and further afield I wouldn’t have chosen the path I did.  The path that led me to be an IBM Champion and 3 years ago one of the first (along with the amazing Theo Heselmans) IBM Lifetime Champions.

That should have been it right? Validation. The pinnacle of achievement.  Confirmation I was doing something right.

I hadn’t allowed for two things.  People’s misjudgement and their need to tear you down. Those two things in the past few months have brought me near to walking away.

I’ve learned to trust my judgement and my judgement says when people isolate me and ignore me it’s because they want to cut me out, and I assumed because they didn’t like me. I don’t consider myself that likeable so that’s a reasonable, although sad, explanation.  However I have realised in the past few weeks that apparently I am in some sort of competition that I was unaware of:  “Don’t let her get involved, she has enough credit”,  “Don’t get involved in ideas she has, she has enough credit”.  Little comments people have said in passing in my hearing serve to destroy my confidence daily. There have been many of these incidents, all small but incemental.

In a group discussion a few weeks ago I was trying to encourage someone I respect to put themselves forward to be a champion.  Another person in the group asked of the group, “Who thinks they deserve to be a champion?” and I, along with the other couple of champions there, put up my hand thinking we were supporting the discussion. This person said, “I don’t. I don’t think any of us do”.

I felt blindsided

I felt awful.

I still feel awful.

Maybe that person was right.  In which case the validation I had been accepting and working to deserve was just ego.  I didn’t think I had much ego but maybe I do. Maybe that’s what puts people off.

So this is to say to all of you out there:

  • Don’t project onto anyone a motive for their actions. Least of all your own.  Someone once said to me “well we all present for the applause don’t we”.  No. No we don’t.  Some of us do it to learn and to help others learn. That’s it.
  • Don’t project confidence where none exists. Don’t assume how you see someone is how they see themselves.
  • If you’re jealous, own that as your problem. I will put my hand up and admit to in the past being jealous of successful friends (Paul, Rob, Stuart) but that was my problem about where I felt I fell short and I truly hope they never felt the effects of it.
  • Don’t try and tear people down to make yourself feel better.

Your comments hurt. your actions or in-actions hurt. You cause hurt.

I wish it was still the mid 90s and we could still be that community that recognised the success of one is the success of all, but that was pre a lot of things and this is where we are now.

I’ll keep doing what I do because that’s the only way I know how to work and because presenting, blogging , sharing, learning, teaching make me happy.

 

Destination Domino (yes, yes I’m late to the party) **

Well this is a lot of good news all at once.  IBM have launched the Destination Domino site – a one stop shop for all your Domino v10 and future strategy news.  If you doubt their commitment to the future development of Domino and the community that believes in it, well just look at all that yellow.

On Thursday 24th May (the day after Engage) I’ll be participating in a webcast on what’s new for Mail, Verse, and Chat for v10.  I will be joining Andrew Manby (Director of Product Management @ IBM) and Ram Krishnamurthy (Chief Architect, Notes, Designer and Xpages @ HCL) on the call.  Registration is here. I recommend you also sign up to the newsletter on the Destination Domino site to stay on top of the developments happening because those are coming at us pretty fast.

I was fortunate enough to visit HCL’s offices in Chelmsford, MA last week and met many of the development teams working on Domino, Verse, Traveler and Sametime.  Some I have known from their years at IBM before they moved to HCL and some were new to me – most of the day is under NDA and you’ll be hearing more about what they are going to deliver at some point if you attend Engage, DNUG, MWLUG and other conferences this summer. If you can’t attend just keep an eye on the Destination Domino site.

One thing I can share that isn’t under NDA is how impressed I was not just with the rapid development of features many of us have been waiting a long time for but also the innovative and open thinking behind Domino as a development platform and the energy and enthusiasm just about everyone I met that day (over 30 people) had.  We are going to see a lot more on the Notes client for iPad and the integration of Node.JS in the next few months and that’s all very exciting.

**I have a good excuse since I’m currently on holiday in St Lucia BUT we interrupt this pool / beach time because this is really important.

Me vs iBooks: The Return. I win (barely)

This blog is for future me and for anyone else wanting to understand some iBooks structure.  It’s not an attack on Apple – I know I’m an extreme case.

Some of you may know my fondness for books.  A habit that led to me buying so many books when the iPad came out I actually broke the iBooks app (too many books to display on the “purchased” screen) which took a year to fix.  Fast forward several years..

It’s been an unexpected few days of technical support. Rumour is that Apple will be changing the iBooks app in an upcoming release and that always makes me nervous.  I buy around 30 books a month and have 3859 on my iPad and iPhone.  Probably about 60/40 iTunes and Amazon.  Losing my books would be equivalent to someone who cares about music losing all their music or a gamer losing all their games.  It would be bad.  Give her space. Don’t try and talk to her. Back away slowly. Bad.

I carefully backup (and have to remove DRM to do it) about once a month.  Why?  Because Apple may decide to drop iBooks at any time and then where would I be with 4000 (or at least 2000) unreadable books?

So I needed to backup and since upgrading to High Sierra that’s been impossible.  The technology I used only worked up to  Sierra.  That’s OK, I use Parallels , can download Sierra at no cost from the App Store and create a VM running Sierra. Of course I had to authorise that VM with my iTunes account so it could read the books which meant deauthorising everything else first since I was at 5 devices. Top tip, if you buy new kit, make sure you deactivate iTunes before flattening the old kit.

Step 1: Getting the books into my VM

In theory because I sync my books to the cloud I should be able to just launch iBooks and auto redownload. Unfortunately that didn’t happen. The books display as in the cloud but have to be manually downloaded.   Understandably selecting nearly 4000 books and telling iBooks to download them all caused it to crash. Repeatedly.  So I needed a better way.

Step 2: Why not just copy the books from my laptop which is the host machine for the VM?

Some digging uncovered that my epubs are stored in

~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books

so surely I can just copy them over from one machine to another?  Why yes I can and when I launch iBooks they all display – kind of.  They can’t be read and most of the covers are missing but otherwise.. great! Some more digging later and I realised that although I had copied over the books.plist (which is a preferences file containing and index of all the books iBooks knows about), I didn’t get the SQL database that iBooks uses that is in ~/Library/Containers/com.apple.iBooksX.  

So that isn’t going to work. A few hours of trying to get covers to appear or books to be readable and I realised I needed to take a step back.  

Step 3: Maybe I was overthinking this. iBooks builds the index when you add books to the app by choosing “add to library” or just drag and drop them so why not drag the 4000 epubs into iBooks.  I knew they were already there but I tested and it does prompt you with the option to “Replace” all books that are already there instead of creating duplicates (of course what I could really do with is “Skip” rather than “Replace” but I get i’m in a niche situation).

So – drag 4000 books to iBooks and choose “Replace” and wait.  There’s no progress bar. Nothing.  The only way I can see that anything is happening is by launching activity monitor and noting that bkagentservice was consuming 80+% CPU.  Eventually “lots” of books appear.  This is the point where I realise there’s no way to count how many books are in iBooks.  I knew “lots” wasn’t all because I got this dialog “<epub filename> couldn’t be opened because you don’t have permission to view it”

Screen Shot 2018-02-05 at 07.54.25

I click OK and got another, and another and another. Eventually having to Force Quit iBooks and restart.

Fair enough.  Maybe when copying over the files from host to guest the permissions came with them and my new guest account doesn’t have permissions.  I spend some time making sure all permissions are OK, applying my new account as well as “Everyone” to that folder and all files contained in it.  I finally test by dragging and dropping individual files into iBooks that work with no error so I decide that error is a red herring – it’s more a “gah! iBooks can’t handle you doing that and has tripped over itself – try adding fewer books”

So now I have a new problem.  What books are missing?  If I knew what books were  missing I could manually add them.   Unfortunately not only do I not know what books are missing,  I don’t know if it’s 10 books or 2000.

Step 4: The search for the missing books

Those filenames aren’t terribly helpful but I know what books I have so I search in iBooks for certain book titles and discover some that aren’t there that should be (and are in my iBooks on my host machine).  How do I find the filename that matches the book title if I know I have the epub in the correct directory?  Here we head to terminal.  In the directory

~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books

I type grep “some phrase” ./*.epub -r

that “some phrase” could be author, booktitle, any text found in the book.  It’s weirdly powerful so make it as specific as you can.  I find the epub filename for a book I know should be there, I find that the epub  is in the right folder and I drag and drop that epub into iBooks. It works!.  Then I try with some of the files it said I had no permissions for… those work too.  OK so since I know it works and I can’t add all 4000 books at once,  now all I need is a list of what books it thinks I have in my Library to compare with the ones I have on the file system.
Easy right?
Step 5: We’re going to need some XCode
The list of books it thinks I have in the library is in the preferences file books.plist in ~/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books.  Unfortunately the only thing that can easily read  a preferences file is Xcode so off I go into developer territory and installing XCode.  Once I do that I can open and read that preferences file.  Of course XCode is 10GB and my books are 12GB so I’m fast running out of space on the small VM I started with.  
When I do that I see this.  That’s right, an array of 5443 items each one representing a book.  Yes I know I said I had 4000 and it failed to add them all but clearly something is awry in the index too – one problem at a time.
Screen Shot 2018-02-05 at 11.56.37
Step 6: A New Plan
I can now read plist files and in theory get an export of items in that file.  If I can export all the books and filenames in the guest machine and do the same on the host machine I can import both lists into Excel and compare to see what files are missing – then manually add them.  Simple!
I don’t do code. I know what I want to do and what I want to do needs code but I will avoid it if I can.  Unfortunately here it’s the simplest way to get what I want.
Using “Script Editor” (part of the native OS) I write a script like this

tell application “System Events”

tell property list file “/Users/gabrielladavis/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/Books.plist”
set Booklist to value of property list item “Books”
set Output to “”
set Counter to 0
repeat with a from 1 to length of Booklist
set theCurrentListItem to item a of Booklist
try
set author to artistname of theCurrentListItem
set booktitle to itemname of theCurrentListItem
set thefile to sourcepath of theCurrentListItem
set Output to Output & author & “,” & booktitle & “,” & thefile & return
end try
set Counter to Counter + 1
if Counter mod 50 = 0 then
log (Counter)
end if
end repeat
log Counter
return Output
end tell
end tell

The counter was so I could see it was actually doing something as it ran.  The “try” was to check if the item has an author etc since my PDFs often didn’t and the code would fail otherwise.

It may not be pretty but it gave me what I wanted which was thousands of lines like this

Pamela Hartshorne,Time’s Echo,/Users/gabrielladavis/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/1F31185F755DD6B65C00B1CF641409B4.epub

Riggs, Ransom,Miss Peregrine’s Home for Peculiar Children,/Users/gabrielladavis/Library/Containers/com.apple.BKAgentService/Data/Documents/iBooks/Books/46D721416EA9EBB037E767DF155A4395.epub

 

Step 7: An afternoon with Excel

Running the agent twice against the host and guest books.plist gives me the data I need.  The host machine plist gives me 3789 entries and the guest machine 5443 entries. It appears every time I attempted to drag and drop a file in the guest copy of iBooks it created a new plist entry.  I enjoy data manipulation in Excel and after cleaning things up and playing with INDEX/MATCH I discover…. it’s not going to work.

The problem is that the plist filename is only updated when the books are added to the library so there was an unreliable mismatch between the guest and host plists.

Step 8: Take a step back and try playing by Apple’s rules

I take a copy of the iBooks directory into another folder (“movedbooks”)then I launch iBooks itself and (making sure iCloud is completely disabled on the guest machine so there’s absolutely not syncing to any device) I remove every.single.book from within iBooks.  Several scary minutes later iBooks is empty and so is the iBooks folder and the plist file.

Meanwhile I still have a copy of all the books in “movedbooks” – I know iBooks didn’t like me dropping 4000 books in but at this point I’m prepared to meet it half way.  After some trial and error, I copy the books in 250 or so at a time.  I verify they are added correctly by checking the books count that appears in the iBooks folder.  It takes about an hour but when I’m done, the iBooks folder is 170 items smaller than the movedbooks backup.

GAH

Step 9: The search for the missing books

I now need a tool to compare the contents of the movedbooks folder to the ibooks folder and tell me which files re present in the first but missing in the second i.e. are missing from iBooks.  A free app called “Compare Folders” does that for me nicely.  Unfortunately it won’t let me export the list but at list I can see the list of missing files.

Step 10: The final piece

170 is a manageable number so now, one by one, I find the missing files and drop them into iBooks.  That works and I end up 3849 books in iBooks and in the directory.  If you’ve spotted that’s 10 less than I should have then congratulations, that’s not a typo.  10 books completely resisted being added to the guest, no error, nothing, they just won’t add.  Even weird when I check my Excel spreadsheet and decide I don’t care about those 10.  But I make a note in case in care in the future.

So that’s it.  I shouldn’t need to do this again as I can add books in small numbers as I buy them and never again have to add all books I’ve bought.  In theory.

A final note.  If you have a Mac , buy yourself a copy of DiskWarrior, but that’s a story for another day.

{blink}So THAT’S the future{blink}

It’s been a few weeks since I’ve posted here.  The end of last year was a juggle of many different pieces of work and learning new technologies but more on that later. I did however take my eye off the ICS (IBM Collaboration Solutions) ball waiting for the HCL deal to firm up and find out more what was going on.

As you may know from my previous post, at the end of October IBM entered a partnership with HCL whereby the latter took development ownership of all Domino products including Verse and Traveler as well as Sametime whilst IBM continued to own the license model, support and most importantly strategy.  As part of that announcement we were told of an upcoming series of workshops around the world called Domino Jam 2025  which were to determine the future path of both Domino and Sametime (so not just Domino and nothing to do with the year 2025!).

Last Friday I (along with about 50 other people) attended a London Domino Jam at IBM South Bank which was hosted by the senior development team at HCL (who transferred in from the same roles at IBM) and the product management team at IBM.  Up until a week or so ago I would say I was more curious and cautiously optimistic than excited.  I had been in coversation with someone in the development team at HCL who I have known from IBM days for 20 years and his introduction to HCL and his enthusiasm for the future sparked my own.  Let’s just say the HCL team seem to be people who have been long immersed in these products, believe in them and are now being let off the leash to develop for the future. During the day we were casually told that there would be new product releases guaranteed in 2018 including

Domino 10

Notes 10

Traveler 10

Verse (10?)

Sametime Instant Messaging 10

That was and is very encouraging news and IBM seemed bemused that several of us in the room kept asking for clarification and confirmation of a thing they thought we already knew.

As part of the Domino jam workshop we were split into groups and asked to brainstorm things like “your biggest pain points” and “what features would you like to see in the future” and then prioritise and present them.  These workshops are going on around the world right now (there’s one in Geneva I believe today) and you can register for them here https://www.ibm.com/collaboration/announcements/domino-jam2025.  In addition if you can’t attend in person IBM are hosting a Domino Jam Forum from Jan 16 – Jan 19 where you can provide your feedback directly online.  See this blog for more information https://www.ibm.com/blogs/collaboration-solutions/2018/01/11/announcing-domino2025-online-forum-january-16-18-2018/

the “#domino2025 Online Forum” will be active for fifty-five hours across the globe to gather your business and technical input and prioritization.” (I prefer “priorities” but I get what they mean :-).

If you are in any way invested in the future of ICS products either mail or instant messaging or conferencing then this is a serious effort to hear what you want and don’t want.  I can tell you there was some very honest and harsh feedback as well as some pretty extreme ideas thrown out by the teams last Friday and we were encouraged to do that.   I know for some of you it’s asking a lot but dig deep, find that kernel of optimism that’s still left, or just throw in your lot because it does no harm to do so and may do a lot of good.

This is an opportunity to help form the long term strategy and direction of products many of us love and believe in.  Take it.

A New Era, A New Future, A New Domino

Some of you may have already seen the tweets out of HCL and the announcement from IBM on the new strategic partnership for ICS products that IBM have just entered into with HCL.  To explain things as simply as possible this means that HCL will take over the development of most of the ICS products whilst IBM continues to manage the product strategy, marketing and licensing.  The products involved include

Domino
Notes (on premises and SmartCloud)
Traveler
Verse
Sametime
IBM Client Access
IMSMO (mail support for Microsoft Outlook)

So what does this actually mean for us as Business Partners and, more importantly, for customers?  In my opinion this is very good news.  The products will benefit from significantly more development investment and resources,  with the existing IBM development teams on those products moving under HCL  In addition IBM will continue to manage the product themselves (using the existing offerings management teams) as well as the licensing, and the strategic direction.

There is no longer any End of Life date for Domino, not even “at least until” dates.

As far as customers are concerned, nothing will change other than an acceleration in the investment and development of products including the announcement of Domino 10 for 2018 and the #Domino2025 project.  Even submitting PMRs will continue in the same way.  There will be no customer – HCL direct relationship.

What is Domino2025? Otherwise known as Project Sapphire this is a strategic and ongoing product development initiative IBM will be starting in November where they will be taking feedback on what people want to help drive the future of Domino.   You can tweet your feedback using #Domino2025 and look forward to workshops taking place worldwide as a continual process to design future (post v10) Domino.

I have a lot more thoughts which I’ll share over the coming days but when I first heard about this and considered it I realised that this is a huge opportunity and a step forward for all the products involved.  I hope you agree.

The birth of a new Domino!