Domino LDAP And A Failure To Authenticate

Bear with me and try not to shout at the screen “we all know that” - this blog is about the 10 hrs I lost yesterday troubleshooting a problem I distinctly remembering seeing before and realising, once I solved it, that last time it had also taken me hours and ended up being the same issue.  In my defence the last time I had this problem it was with Quickr so that’s a throwback and even if this blog isn’t news to you, it will hopefully be there for me in another 5 years…

I was using Domino as a LDAP source for Connections.  I don’t manage the Domino side of things for this customer so I had just asked them to add a secondary directory (in this case for External users) to Directory Assistance on their LDAP servers. I wanted the DA document set to be LDAP only rather than LDAP & Notes / Internet Authentication**. They did that and I tried to login from Connections to discover that I could login as a user in names.nsf but not as a user in the secondary directory. Time to look at the configuration.  Here’s what I did

1. Confirmed the DA document looked OK.  It actually wasn’t set to trust for credentials so I enabled that.
No luck.

2. Tried “sh xdir” to verify the directory was listed. It was, as Directory #4 out of 6.  Tried sh xdir reload to refresh Directory Assistance and then tried restarting the server
No luck but at least I knew DA was configured correctly

3. Turned on LDAPDebug=3 so I could see the debug information. At this point I could see the failing accounts (any in the secondary directory) were coming up with “authentication failure using internet password” in Domino and in the SystemOut.log of the WAS server that hosts the homepage application I saw references to PasswordFailedCheckException behind CWWIM4529E and SECJ0369E errors. Password failed? That made no sense at all.   One thing that was an issue was that the server I was working on was being probed every few seconds by a remote machine for availability on LDAP so with debug turned on the screen was filling up with thousands of lines of content making it difficult to see and track my own issues.  In retrospect if I’d asked for that to be disabled it would have saved me hours.

4. I then took a step back and installed Softerra’s LDAP Browser so I could test things outside of Connections.  I could bind using any credential in names.nsf but when trying to bind using a credential in the secondary directory I got “invalid credentials” and LDAP wouldn’t bind.

5. I then cut and paste a person document from the secondary directory to names.nsf to verify if the issue was the directory itself or the format of the person documents. I knew those documents worked fine on another server where they were in the names.nsf.  Turns out that if I moved them to names.nsf they worked fine.  I could bind with Softerra and I could login with Connections.

hmmm

6. I go back and check the ACLs of both names.nsf and the secondary directory.  I may even have bumped up default to something stupidly high *cough*Editor*cough* for 30 seconds to rule that out.
No luck

7. I paste the person document back into names.nsf again and bind with Softerra. This time I try and search for a name I know is in both the names.nsf and secondary directory (not the same name, just the same lastname).  Interestingly I get access denied / unauthorised - it finds the two entries but doesn’t let me see the content of them.  The fact that it found the entries meant that it could search LDAP but it can’t display them?  Surely that’s ACL issues.  So back I go to check the -default- rights on both directories and even test effective access for the specific account i’m using.  Nothing.

Then I see it.  As I try searching and searching and trying to catch errors on the server logs amongst the mass of LDAP debug information.. I see
searching directory names.nsf for sn=davis
searching directory directories\custnames.nsf for sn=davis
search directory directories\morenames.nsf for sn=davis unauthorised, skipping
search directory directories\externalnames.nsf for sn=davis
search directory directories\suppliers.nsf for sn=davis

Right there - in the middle. A directory I don’t care about, that has two dummy documents in it but happens to be part of Directory Assistance.  I go look at yes - -Default- is set to No Access. I change that to “Reader” and ta-da! suddenly I can both bind and login.  Then I remember I had this exact problem before at another customer with many directories that I didn’t set up or configure and it took me forever to find because I simply don’t touch what I’m not meant to be managing. In this case a directory that’s nothing to do with me and isn’t being used by my application on a server I don’t manage.

So what happened? It appears that Domino LDAP will search multiple directories but once it comes across one it can’t access with those bind credentials it doesn’t skip over it.. it stops.  The “skipping” isn’t strictly true.  So when the credentials were in directories one or two they worked. in directories four or five they failed because it stopped at directory three.

My lessons are
1. Remove as much extraneous activity as you can or you won’t be able to debug quickly enough
2. Always check everything (or in my case ask permission to check everything) even if it looks unrelated and especially if you didn’t set it up yourself 🙂

You’re welcome Gab of the future….

**Added on this morning.  Using LDAP only for authentication doesn’t work because a Directory Assistance document set to LDAP only doesn’t actually work for anything but LDAP searching. Not for authentication at all.  Foolish me for trying to be logical.  Here’s what the pop up help says - and they’re right. I tested it :-)]

DirectoryAssistance

Domino LDAP Insufficient Access

Here’s where you all get to laugh and point at me for not knowing this sooner.  I was setting up Domino for LDAP access on a server with multiple directories in DA.  Everything was working fine until I wanted to write values from another source into the Domino LDAP.  Insufficient access.  OK so let’s check

  1. Account being use to authenticate has Editor access to the ACL in all directories in Directory Assistance
  2. Global Configuration document in Domino is set to allow LDAP write activity
  3. Global Configuration document in Domino is set to allow write activity that doesn’t conform to the schema
  4. I can login to the web interface of Domino using the LDAP credentials and successfully edit the person document I’m trying to change through LDAP

So what was my problem?  Apparently with LDAP write activity the Global Configuration document enabling LDAP to do writes has to appear in every directory used by Directory Assistance !  I finally got there by trial and error but that makes no sense at all, especially because the secondary directory doesn’t even need to use the pubnames.ntf template.  The Global Configuration document by definition controls LDAP activity for the entire domain which surely includes any secondary directories that are set up.  But that’s what it was.

I created a Global Configuration document in my secondary directory and set it to allow LDAP and write activity and my “Insufficient Access” went away.

Ooh look - wordpress has a poll facility , let’s try it.

Adventures in TDI - Connections and Updating Profiles

Recently (well the past couple of months) I have been working on building custom Assemblylines to sync Connections data held in DB2 to LDAP data held in Domino.  I really struggled with finding good documentation for doing this and that’s because the best documentation was written for 3.0.1 and hasn’t been updated since.  Thanks to help from some people at IBM (who I’d name publicly here but I’m not sure they want emails from everyone) I managed to get hold of some draft updated documentation and get answers to some questions as I went along so I thought it would be helpful to share what I’ve learnt although this is a very streamlined description, hopefully it will give you some pointers.

Firstly if you are using Connections you need to populate profiles.  Populating profiles requires you to have installed TDI (and the right fixpack, don’t try it without) but once you have done that you have 3 techniques for population, two of which involve you never seeing a TDI configuration screen.

1. The population wizard which is a graphical tool supplied by IBM for pulling data from LDAP into your Connections data source (DB2 in this case).  The population wizard is easy to use and meets that needs of I’d say 60%+ of users who are working with Connections.  It’s certainly where most people begin to get that information populated in the first instance.

2. Underlying the population wizard are XML and properties files.  If you run the wizard you’ll find it has written to  properties and XML files on the file system and then it uses IBM supplied scripts to run the import.   What you can do is take a copy of the TDISOL directory which contains all the properties, xml files and scripts (and is updated on Fix Central occasionally) and use those to create you own custom syncing.  The files you will be working with are

tdienv.bat/sh  This is where you tell TDI to find its own installed files

profiles_tdi.properties  This is where you tell TDI how to find the LDAP and DB2 sources as well as how to authenticate with them

map_dbrepos_from_source.properties This is where you tell TDI how and what to map from LDAP (Domino) to Connections (DB2)

map_dbrepos_to_source.properties  This is where you tell TDI how and what to map from DB2 to LDAP if required.

You can then schedule a batch file called sync_all_dns.bat /sh which will bi-directionally update the information according to those files.  I would always recommend a) doing this in a test environment first b) backing up PEOPLEDB before starting c) running collect_dns.bat/sh first to ensure your search scope for LDAP returns the users you expect

3. So if you’re still with me and you want something even more advanced you’ll need to create an Assemblyline using the TDI Configuration Editor. For instance sync_all_dns does a complete bi-directional sync so for 50k users it was taking nearly an hour and for 25k users 20 minutes because it has to check every record in both LDAP and DB2.  That was taking too long so we wanted something that ran faster, in another case we wanted to pull in data from another data source to populate profiles alongside the LDAP data.  Using TDI and creating a custom assemblyline allows you unlimited scope to pull data from any LDAP or JDBC source (amongst others) into your profiles.

You’ll hear a lot of talk about Assemblylines being “real time” but in fact that’s very difficult to do in a Connections environment.  There is no real time monitor of generic LDAP you can use.  There are change control monitors for both Domino and Active Directory if you are using those as your LDAP source you can use them to  continuously monitor the servers and trigger on any change.  I have found however there is a risk associated with a continuously running and monitoring service and, being risk averise,  I prefer to schedule my Assemblylines to run.     There is also a change connector for RDBMS you could use to monitor DB2 but again, I prefer to use the standard Connector with a SQL statement telling it to look for things modified in the past x minutes or whatever.  I then export the project  and create a batch file to call the Assemblylines I want, scheduling that to run repeatedly in Task Scheduler (for Windows) or Linux.

The hard and fast rule when creating an Assemblyline is that you must use IBM’s supplied files.  They have provided Assemblylines that can be copied and modified to do just about anything you need.  To begin with you will need to complete the tdienv.bat/sh and profiles_tdi.properties files which will be used by the Configuration Editor when it’s launched.  Once launched you’re going to create a new project by importing the profiles_tdi.xml file and that will present you with all the Connectors and starter Assemblylines you need.  In particular there is a Connector called the ProfilesConnector which contains everything needed to write to a profile.

Without going into 40 pages more detail, I hope this helped.  This paper from IBM is the absolute best source for setting up your environment, although significantly out of date it will set you on the right path .  I am also anonymising a simple Domino - DB2 and DB2 - Domino Assemblyline  that I am happy to share with people once it’s complete.  I can’t support it and you take it at your own risk but it may help you to see something configured.