ALM integration challenge #1: Same users but different accounts in different systems
CollabNet Connector Framework (CCF) aka CollabNet Connect/sync is an Application Life-cycle Management (ALM) middle-ware that integrates and synchronizes data between CollabNet TeamForge and various ALM platforms like HP Quality Center/ALM 11, ScrumWorks Pro, MS Team Foundation Server and Rational ClearQuest.
One of the most common use cases we encounter when doing artifact synchronization with CCF is that user accounts are not identical on source and target systems. A user called John Doe may have an account jdoe in TeamForge and john_doe in Quality Center. To complicate things, certain users may not even exist in both systems in order to save licenses. This blog post will shortly cover traditional approaches to still map artifacts and introduce a new CCF feature called “dynamic user lookup” which makes user related mapping tasks way easier. As we have just released this new feature, we like to get your feedback on it so that we know how to improve it in future CCF releases.
Traditional approaches to deal with mismatching user bases during ALM integration
The simplest workaround is not to map user fields between different ALM systems at all. While this might seem unacceptable at the first glance, you might be surprised how often this approach is taken once the effort for alternatives has been realisticly estimated.
A not much more complicated solution is to map a user field of the source system to a text field in the target system. In this case, no information gets lost during artifact synchronization but one still not be able to benefit from the ALM reporting features that go with “real” user fields like burn down charts, capacity planning, etc.
If there is a clear algorithm how user account names are generated in the source and in the target system, one can try to set up a more complex mapping rule which simulates this algorithm. If, e.g. a user account in TeamForge is alwasy called <firstname>_<lastname> and <firstname>-<last name> in Team Foundation Server, one could work with XSLT transformation rules, that replace underscores with hyphens. If such an algorithm exists and the user base of the source and the target system are similar, this is probably the best solution. However, if only a subset of users exists in the target system (to save license costs, etc) this approach has to be combined with our new feature, so please read on
If there is no clear algorithm to compute target account names based on source account names, one can define the account mapping user by user with a value map. CCF already provides user mapping capability using value map mapping through graphical mappings (via Altova Mapforce mappings, Mapping rules mapping) and non-graphical mapping (custom xslt). Using a value map mapping each user could be mapped one to one between the user id in the source and target systems. But mapping one to one user ids is a very time-consuming and tedious process. It will aggravate the existing situation when new users are added to the source and target systems.
For the sake of completeness, we like to mention two further approaches to deal with different user accounts: Unify user account naming conventions across all ALM systems used, create missing accounts in all systems or agree upon a subset of user accounts that can be assigned to artifacts to be synched. Those approaches might be the best answers in some companies and completely unrealistic in others.
Our new feature falls into yet another solution category: Dynamically look up the corresponding user account while the artifact transformation is taking place using custom code. In this case, one can look up user mappings in external data sources and do any computation imagineable. However, one always has to take under consideration that executing custom code imposes additional questions around security (who wrote that code, will it only do user mapping and nothing else, how to authenticate against external data sources) and performance (how long will such calls to external systems take, do we have to cache results, what to do in case of a timeout, etc). Our new feature only talks to TeamForge, caches session information and does not allow to execute any arbitrary code.
Introducing CCF’s dynamic user lookup feature
In past customer scenarios, we have used all approaches outlined above. sometimes even combinations of them. The feature we mostly missed while using value maps and custom XSLT transformation blocks was a check whether the account name we have computed for the target system actually exists (as users are constantly created in larger ALM setups and not immediately synchronized to all systems).
We wanted to easily solve use cases like this one:
- A QC defect gets created with BG_RESPONSIBLE field set to a QC user name.
- If the same user (admin in our example) is available in TF, the artifact gets created in TF system with assignedTo set to the matching user account (TeamForge Administrator in our example)
- If the user in question is not available in the target system (ccfsynch in our example), the default user name defined in the mapping (normal in our example) gets assigned to the target system.
- If BG_RESPONSIBLE in QC is empty, the corresponding TF artifact will have None value assigned to the TF artifact.
To implement use cases like this we have created a new, Java backed, XSLT function searchUser()which looks up and resolves the user name in TeamForge. To use this function, we will need a custom XSLT snippet (in mapping rules mapping) as shown in the screen shot below.
The function has two variables – the first variable gets assigned to the field value of BG_RESPONSIBLE for the defect and the second variable defines the default user name that needs to be assigned when the user lookup fails because the passed user account does not exist in TeamForge.
With this function, it is finally possible to conveniently deal with situations where a user just does not exist in the target system.
Where this feature will go
The searchUser() XSLT function will be part of the upcoming CCF 2.2 release and is already available for CCF 2.1 customers upon request (comment on this blog post if you like to give it a try). At the moment, it only supports looking up users in TeamForge. For the future, we are thinking about various improvements like
Lookup users based on permissions (as the user in the target system might not have permissions to be actually assigned to the artifact)
Lookup users based on their email, full name and other characteristics (your suggestions welcome here)
- Lookup users in other ALM systems integrated with TeamForge (TFS, SWP, RCQ) as well
Whether and when those improvements will be developed mostly depends on your feedback to this post. Apart from feature prioritization ideas we are also interested in your approaches to deal with different user bases during ALM integrations, your input will help to make life of integrators way easier and shape the feature scope of CCF 2.3.
The CCF team