A Portal to a Portal

Having spent much of the past week trying to resolve a problem with person tagging in the Profiles feature of Lotus Connections 2.5, I thought I'd share the solution.

To cut a long story short, the customer was seeing a problem whereby an authenticated user was unable to tag his/her own profile, using the Tags widget on the left-hand side of the My Profile page. There was no obvious error in the browser, and the tag cloud/list was never updated.

After much trial and error, we narrowed the problem domain down to it only occuring when we accessed LC via a pair of front-end HTTP server, via an HTTP load balancer - it's my understanding that the LB "merely" directs requests to one of the two IHS servers on a "round robin" basis, so has no real part to play in the story.

We were able to confirm that the problem WAS "caused" by the presence of the front-end IHS servers, by accessing Profiles directly on the WAS instance ( in the client's case, each Connections feature is clustered across two identical WAS ND nodes ). When we accessed Profiles in this way ( e.g. http://connections.uk.ibm.com:9088/profiles ), tagging worked as expected.

However, going via the HTTP servers ( both with the WAS plugin configured correctly to direct the requests to the cluster ), we saw the following string of messages ( some good, some bad ) in the IHS access.log file: -

192.168.1.3 - - [24/Nov/2009:13:07:02 +0000] "GET /profiles/html/myProfileView.do?lang=en HTTP/1.1" 200 41226
192.168.1.3 - - [24/Nov/2009:13:07:04 +0000] "GET /profiles/html/profileLocalTime.do?timezoneId=Europe%2FLondon&timeNow=1259067900000&lang=en HTTP/1.1" 200 17
192.168.1.3 - - [24/Nov/2009:13:07:04 +0000] "GET /profiles/ajax/getStatusMessage.do?key=39464090-6e3a-4543-9370-954f341c3096&lang=en HTTP/1.1" 200 1171
192.168.1.3 - - [24/Nov/2009:13:07:04 +0000] "GET /profiles/ajax/getWallContent.do?key=39464090-6e3a-4543-9370-954f341c3096&filter=PROFILE&type=ALL&lang=en HTTP/1.1" 200 12500
192.168.1.3 - - [24/Nov/2009:13:07:04 +0000] "GET /communities/service/atom/communities/all?userid=E7CEACE4-B7E8-BDD0-8025-76680020359C&ps=5 HTTP/1.1" 500 619
192.168.1.3 - - [24/Nov/2009:13:07:14 +0000] "POST /profiles/html/tagTypeahead.do?useJson=true HTTP/1.1" 200 8
192.168.1.3 - - [24/Nov/2009:13:07:15 +0000] "PUT /profiles/atom/forms/profileTags.do?targetKey=39464090-6e3a-4543-9370-954f341c3096&sourceKey=39464090-6e3a-454                                              3-9370-954f341c3096 HTTP/1.1" 403 319
192.168.1.3 - - [24/Nov/2009:13:07:15 +0000] "GET /profiles/auth/loginRedirect.do?loginReturnPage=%2Fprofiles%2Fhtml%2FmyProfileView.do%3Flang%3Den HTTP/1.1" 302 -
192.168.1.3 - - [24/Nov/2009:13:07:15 +0000] "GET /profiles/html/myProfileView.do?lang=en HTTP/1.1" 200 41226
192.168.1.3 - - [24/Nov/2009:13:07:16 +0000] "GET /profiles/html/profileLocalTime.do?timezoneId=Europe%2FLondon&timeNow=1259067960000&lang=en HTTP/1.1" 200 17
192.168.1.3 - - [24/Nov/2009:13:07:16 +0000] "GET /profiles/ajax/getStatusMessage.do?key=39464090-6e3a-4543-9370-954f341c3096&lang=en HTTP/1.1" 200 1171
192.168.1.3 - - [24/Nov/2009:13:07:16 +0000] "GET /profiles/ajax/getWallContent.do?key=39464090-6e3a-4543-9370-954f341c3096&filter=PROFILE&type=ALL&lang=en HTTP/1.1" 200 12500
192.168.1.3 - - [24/Nov/2009:13:07:16 +0000] "GET /communities/service/atom/communities/all?userid=E7CEACE4-B7E8-BDD0-8025-76680020359C&ps=5 HTTP/1.1" 500 619

Of these, we can ignore the HTTP 500 errors coming from Communities, as that feature is not running ( I've taken the logs from my own LC25 clustered environment, as I was able to recreate the customer's problem ).

The important error is the HTTP 403 coming from the PUT method.

The customer was also using the HTTP mod_write directive, which I've again reproduced on my own setup here: -

LoadModule rewrite_module modules/mod_rewrite.so
RewriteEngine on
RewriteCond %{REQUEST_METHOD} !^(GET|POST|HEAD)$
RewriteRule .* - [F]

Can you see the problem ? I bet you can - sadly, it took me a little longer ...

As mentioned, the person tagging feature uses the HTTP PUT method, rather than the more common HTTP POST method. The customer's configuration for mod_rewrite was explicitly blocking this, via the RewriteCond directive seen above.

Once we change the rule to read: -

RewriteCond %{REQUEST_METHOD} !^(GET|POST|HEAD|PUT)$

and restarted IHS, all was well.

The moral of the story ? Break a problem down into logical domains e.g. test the core of the environment ( in our case, accessing Profiles directly on WAS on port 9088 ) and, if that works, focus on the HTTP servers, in terms of (a) non-standard modules/directives and (b) the access and error logs.

With thanks to Erik Sorensen, the Lotus Connections support team in Mulhuddart, Dublin, and, of course, my customer.

Sorted :-)