Status for Andrew DeFaria

• Implemented Evil Twin trigger. Have not yet tested nor installed trigger yet
• Meeting with Naga Re:
  • Binary Merge problem
  • RM_EMPTY_BRANCH/EVIL_TWIN triggers
  • Lock project problem
• Reviewing both Philip and Chin's performance testing
• Struggling with Cisco VPN client at home!

Evil Twin Trigger

Evil twins are a well known phenomena in Clearcase circles. It is not recommended that you try to stop them by convention rather by Clearcase trigger. I have a trigger that does just that. The code uses cleartool ls so that it works for both dynamic views and snapshot views. The trigger also respects case sensitivity. This means that creating an element named "foo" and another element named "Foo" is allowed.

Evil twins should not be allowed in any vob because, as you rightly point out, it causes merging problems.

Shall I add this trigger to all vobs?

Performance Scripts

Both Philip and Chin have their own Clearcase performance testing scripts. In general performance of Clearcase is a huge topic. These scripts a relatively simple, just timing some basic Clearcase operations. Still this is a good metric from a user's perspective.

We should consolidate these tests and make them more flexible by having them self contained (both creating and tearing down the environment that they need) and generalize them so that new Clearcase operations can be easily added and timed.

These scripts should produce a data file that can be later consumed by say a PHP script to present the results in a readable manner on a web page somewhere.

Also, these scripts need to set up various different environments such as using local/remote snapshot views, dynamic views. These scripts can be used to measure performance of the old (ccase-rmna-1) environment as well as newer environements (ccase-rmna-3 and/or the new Linux server).

Finally it should be made such at adding new tests, i.e. make new baseline, can easily be incorporated into the test suite.

Posted by Andrew DeFaria at 12:04 PM | Permalink

• Inveistigated binary merge problem
• Worked with Mukund regarding CVS access
• Helped Shivdutt with a Clearquest Web problem. User reports that Clearquest Web not working in IE! Suspect the problem is due to configuration issues, specifically JRE. Did some research and sent URLs to user

Binary Merge

Clearcase's diffmerge utility has understandable problems attempting to merge files that are binary (Clearcase searches for NUL characters \000 in the file to determine if it is binary). Sometimes Clearcase can merge such files, if it determines that the merge merely entails a wholesale replace of one version with another. But in non trival merges this is not possible.

Developers and managers here use a series of Perl scripts to help automate common trasks, one of which is to merge to and from different branches. The problem is when delivering a large set of changes there is a chance that some of the files will be binary and that it will not be possible to automatically merge them. Often a developer will start a merge and let it run over night. Sometimes, somewhere in the middle, the merge will prompt with a dialog box saying that it cannot merge this binary file. The result is that the merge is 1/2 done!

The Perl script uses cleartool findmerge to find and merge both directories and files. It also uses the -merge option to say "automatically merge things". Unfortunately when it hits a non trival binary file merge it cannot continue.

We propose to modify the Perl script to call findmerge without the -merge option then obtain the list of files that needs merging and itterate through the list calling cleartool merge for each file with a -abort. This tells merge to automatically merge things if it can, otherwise not to merge and return a status code that that merge was not possible in an automatic mode.

The script would then collect the names of all the files that could not be automatically merged and store that in a file somewhere.

We believe there are 3 possible ways to merge a non trival merge of a binary file (this assumes a simple merge between two different versions of a binary file. A merge with multiple contributors would obviously present N+1 possibilities...):

1. Result of merge is a copy of the first contributor
2. Result of merge is a copy of the second contributor
3. Binary file needs to be rebuilt combining both A and B

A process could be written to read the saved file of non automatic binary merges and present the user with the choices listed above. The user then selects which choice is appropriate. The process then performs the necessary actions to accomplish what the user requested (i.e. if they select A then a merge arrow is drawn from B -> the checked out copy of A. If they select B then a merge arrow is drawn from A to the checked out copy of B). If any C options are choosen then the file remains unmerged.

In this way when the user performs his normal merge, upon completion, if merge conflicts exist in binary files the user is presented, at the end of merging everything else a dialog box allowing them to resolve the binary merge problems right now. The benefit here is that all of the other merging has already been completed and they are only dealing with the problem merges now. If they do not wish to correct these now they can always correct them later (issues about where this saved file will reside and how to restart the binary merge resolver are still open).

Posted by Andrew DeFaria at 06:31 PM | Permalink

• Met with Chris Rumf regarding Clearquest and int impending upgrade/merge of DBs. Chris told us of his trials and tribulations with a 3 month project to merge CQ databases and move them to San Diego.
• Fixed mktriggers to properly specify the path for the trigger script in both Windows and Unix implementations.
• Worked with Shivdutt to try to reproduce the problems with merging binary files.
• I was assigned a ticket to give a user CVS repository access. I still do not have proper login access to Irvine nor do I have permissions to perform this task

Clearquest issues

In all likelihood we will be faced with a merging of DBs and probable writing of Perl scripts to massage and otherwise transform the database. This will take time. Additionally Vinh is leaving Broadcom.

Chris warned that he faced a situation where the new version of Clearquest brings in the concept of internationalization by use of a code page and that he had many problems with users copying and pasting things from MS Word which ended up outside of the normal character set.

Also, Irvine seems to have Clearquest 2001 version which has the old, ASP based Clearquest Web which is decidedly inferior to the new 2003 Apache/RWP/Java based Clearquest Web. He also pointed us to some valuable information about performance tuning the various Clearquest Web pieces. This will be useful if we are to support a large number of users from Isreal.

Triggers

Looked over most of the currently implemented triggers. I can probably add them to my mktriggers.pl script. One problem is that the ADD_EXECUTE trigger is merely a cleartool protect command to add execute permissions to newly created elements. I'm not sure why one would want to do this to every element as most elements are not executable (even though some OSes seem to think so). The mktriggers.pl script is oriented to only adding Perl based triggers. This could be solved by making it a simple Perl script. Also ADD_EXECUTE is added to all vobs except the \Docs vobs. It we can add it to all vobs it would be easier.

Current triggers use -exec and specify a Windows UNC path. This means that if we have any Unix clients they will not be able to execute any of these triggers! Fixed mktriggers.pl to add -execwin and -execunix options with proper pathnames.

CVS

Depending on the CVS situation, Broadcom might be interested in a Web application I had made that manages CVS users. From a web page a user can manage his user (able to change his password, add an email address/phone number, etc). If the user is in a cvsadm group they can add/change/delete other CVS users. With such a web app the help desk could resolve such CVS requests on the spot.

Posted by Andrew DeFaria at 06:00 PM | Permalink

• Finally got Remedy access. Turns out I had it all along just my username was ADEFARIA instead of adefaria!
• Looked into P4 ticket. I am unable to perform the action because I don't have a login to Irvine. P4 stuff gets performed in Irvine and I don't have an account there. Actually I have an account but didn't know the password. Actually I figured out the password - it was my initial password when I came to Broadcom and was never set. The Help desk had me set my password in the beginning but that was in the San Jose domain. I've now set it for Irvine too. There are two problems with my Linux login:
  1. In Irvine I have no home directory! As a result I cannot create ~/public_html and write web pages as well as I cannot set up my startup scripts, etc. I put in a ticket for this. Hopefully they will set up Irvine like San Jose in that my home directory will be share amoungst all three areas.
  2. My default shell is (yuck) csh! I asked for that to be changed to bash.
  3. Worked with Shivdutt on disk space problem on ccase-rmna-3
  4. Added RM_EMPTY_BRANCH trigger to vobs on ccase-rmna-3
  5. Investigating other triggers to include them in mktriggers.pl

San Jose/Irvine Domains

Here in Broadcom there are multiple NIS domains here including sanjose and irvine. For example, ssh'ing into xserver.sj works for me and puts me in /home/adefaria. Insterestingly /home/adefaria corresponds with my Windows home directory. IOW if I touch file on Linux I can see file in my home directory in Windows. Cool. The world is as it should be!

However in Irvine I have no home directory at all. It seems that home directories are automounted from the NIS auto_home map and that San Jose home directories come from fs-sj1-* while Irivine home direcrtories come from fs-irva-*. Ugh!

Posted by Andrew DeFaria at 10:50 AM | Permalink

• Investigated how to setup, configure and failover Clearcase backup registry
• Working with Shivdutt to set up new Clearquest server software
• Ported mktriggers.pl and RemoveEmptyBranch.pl. Added RM_EMPTY_BRANCH to vobs on ccase-rmna-1

Clearcase Backup Registry

I've looked into how to setup a Clearcase Backup Registry as well as how it is supposed to function when a failure occurs and a switchover is necessary.

Introduction

Clearcase provides a mechanism to set up a backup registry server in the event that the primary registry server fails. If done properly switching over to the backup registry server is quick and easy. It is not, however, without it's potential problems¹.

Setup

To setup a Clearcase Backup Registry Server you must first define which servers will be the primary and secondary register servers². Let's assume rgy1 is the primary and rgy2 is the backup. Next you configure rgy1 to tell it that the backup registry server is rgy2 by adding a line to /var/adm/rational/clearcase/rgy/rgy_hosts.conf.

Normal Operation

In normal operation Clearcase runs a scheduled job called Daily Registry Backup on all hosts. On all hosts except the designated backup registry server this job does little if anything.

I have a note that when run on a Windows client it will ask what the backup registry server is and if found and if different, will store that value in the Windows registry, thus Windows clients are self configuring. This should be tested however.

On the backup registry host the job will obtain a snapshot of the registry from the primary registry server. By default 3 days worth of copies are kept.

Switchover

If the primary registry server fails all that needs to be done is to run rgy_switchover which will promote the backup registry server to be a primary registry server. It will also inform all clients of the change. When the primary comes back it is configured as a backup of the new primary and optionally rgy_switchover'd to be a primary again

Notes

1. There is no guarantee that rgy_switchover will be able to successfully switch over all clients. Clients may not be available on the network, for example. It will report which clients were not switched over and they can be fixed manually.
2. What are our registry servers? The Clearcase Admin Console shows me the following:
   

   1. ccase-atla-1
   2. ccase-gera-1
   3. ccase-irva-2
   4. ccase-rmna-3
   5. ccase-sdoa-1
   6. ccase-sj1-1
   7. ccase-sj1-3
   8. ccase-sj1-4
   9. ccase-sj1-5
   10. ccase-sj1-7
   11. ccase-sj1-8
   12. ldt-sdoa-013

   It seems as if multiple registry servers were used instead of using one global registry and dividing things up by Clearcase regions. In any event, which of these primary registry servers need to be backed up and to where?

   Also, it is not a good practice to put your registry on the same box as your other Clearcase objects (i.e. views or vobs). Say, for example, ccase-rmna-3 blows a disk drive and will be down for quite some time. Sure ccase-rmna-2 might be it's backup registry server and we could switch over the clients, etc. However what good would that do if the vob data they wish to get to is on ccase-rmna-3! Answer: No good at all. That machine is down - period.

   I would recommend that we have one global registry server and a backup server. The backup server can be a vob machine or some other machine which houses important Clearcase data. The theory here is that, as a backup server, it's service time is limited - IOW it's only going to be functioning as a primary registry server for the time that the primary is out. While slightly risky it's only used for a limited time and when there's an emergency.

Mktriggers.pl & RemoveEmptyBranch.pl

Shivdutt and I feel that the problem that check_full_baseline is hitting is initially caused by having elements that have a 0 element on a branch. This is a common problem as I've explained before - a user checks out a file, it is branched and a 0 element is created as well as a checked out element:

If the user then cancels the checkout then we are left with:

There is no difference between /main/1 and /main/andys_branch/0. Both /main/andys_branch/0 and /main/andys_branch can be safely removed. I believe that check_full_baseline corrects this situation by creating yet another identical version /main/andys_branch/1 and checking in yet another identical version.

The RemoveEmptyBranch.pl trigger corrects this condition at uncheckout (rmver and rmbranch) time by detecting this situation and removing both /main/andys_branch/0 and /main/andys_branch. It will only do so if the 0 element is the only thing there. If there are labels attached to the 0 element it will not remove it.

In order to implement this trigger I had to get the code working here at Broadcom. The RemoveEmptyBranch.pl trigger runs right out of the box, however things must be placed the proper places here at Broadcom. Additionally this trigger should be added to all vobs. Suffice to say, as a Clearcase Admin, I've hit this problem before.

My solution is a mktriggers.pl script which adds (or replaces) triggers on all public vobs in a region based on data in a data file which describes triggers. Mktriggers.pl is smart to skips private vobs and UCM project vobs.

Finally, mktriggers.pl uses a module of mine called Display.pm, which provides a consistent way of displaying messages.

To this regard I have created/ported the following files in //fs-rmna-01/Projects-V0/cc4:

bin/mktriggers.pl: Script to make/replace triggers in all vobs based on triggers.dat
etc/triggers.dat: Data file describing triggers (currently only describing RM_EMPTY_BRANCH)
triggers/RemoveEmptyBranch.pl: The RM_EMPTY_BRANCH trigger
lib/Display.pm: Perl module for displaying messages, errors, warnings consistently
lib/Logger.pm: Perl Object for handling creating and manipulating log files (Not used yet)

Here's a usage for mktriggers.pl:

    $ mktriggers.pl -u
    Usage mktriggers.pl: [-u] [-n] [-a] [-r] [-v] [ -vobs  ]
    Where:
            -u      Displays this usage
            -n      No execute mode - just echo out what would have been done
            -r      Perform only replacements of triggers
            -a      Perform only adds of triggers that are missing
            -v      Verbose
            -d      Debug
            -vobs   List of vob tags to apply triggers to (default all vobs)

As you can see there is a no execute mode which just shows what would have been done. Add -v for verbose and it will also echo out the commands that would have been performed. You can also limit mktriggers.pl to only doing additions (-a) or replacements (-r).

Mktrigger.pl's data files is in etc/triggers.dat. It's format is relatively simple:

    # Triggers
    #################################################################################
    #
    # File:         triggers.dat
    # Description:  Describes the triggers to be implemented.
    # Author:       Andrew@DeFaria.com
    # Created:      Mon Mar 15 08:48:24 PST 2004
    # Language:     None
    #
    # (c) Copyright 2004, Andrew@DeFaria.com, all rights reserved.
    #
    ################################################################################
    #
    # Only the following keywords are currently recognized:
    #
    #       Trigger:        Introduces the trigger and gives it its name
    #       Description:    Used for the trigger type's comment
    #       Type:           Type of trigger (so far they're all -element -all)
    #       Opkinds:        Operation kinds that will cause the trigger to fire
    #       ScriptEngine:   Currently only supporting ccperl (C:\Program
    #                       Files\Rational\ClearCase\bin\ccperl)
    #       Script:         Script to run (under triggers)
    #       Vobs:           Can be either base, ucm, all or a list of vob tags.
    #                       If base is specified then the trigger is applied to
    #                       all base Clearcase vobs. If ucm is specified then the
    #                       trigger is applied to all ucm vobs. If all is
    #                       specified (or if Vobs is not present) then the trigger
    #                       is applied to all vobs (base and ucm). Otherwise the
    #                       value is considered a space separated list of vob tags
    #                       (without the leading "\") and the trigger is applied
    #                       only to those vobs.
    #       EndTrigger      Ends this trigger definition.
    #
    ################################################################################
    Trigger:        RM_EMPTY_BRANCH
    Description:    Remove empty branches after uncheckout, rmver or rmbranch
    Type:           -element -all
    Opkinds:        -postop rmbranch,rmver,uncheckout
    ScriptEngine:   Perl
    Script:         RemoveEmptyBranch.pl
    EndTrigger

Next I applied the triggers to ccase-rnma-1 since this is our replicated backup of production and here's the output:

    bash-2.05b$ bin/mktriggers.pl -v
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/A1... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/CommEngine... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/NewTest... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/OnePhone... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/SpiceBoxSW... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/TrainCommEngine... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/alpha_video... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/bfc_systems... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/cablex... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/cablex_tools... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/docs... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/ldx_apps... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/ldx_dev... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/ldx_hausware... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/ldx_tools... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/lucentexcel... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/netro_apps... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/phonex... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/prot_callctrl... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/prot_h248... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/prot_mgcp... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/prot_openssl... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/prot_tools... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/rmna_projects... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/sec_uHSMnCipher... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/telecan... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/test_comp1... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/test_docs... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/test_pvob... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/test_trp_vob... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/voice_res_gw... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/widcomm_bluetooth... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_common... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_drivers... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa_cbx... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa_ipp... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa_ldx... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa_op... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xchg_qa_xme... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xme... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/xme_sa... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_Nucleus... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_TCL... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_VxWorks... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_cygwin... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_eCos... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_gnu_mips_elf... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_misc... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_psos... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_ti54x... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_vc... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_x86... done
    Adding trigger RM_EMPTY_BRANCH to vob /vobs/zOEMtools_zsp... done

When neither add (-a) or replace (-r) is specified mktriggers.pl both adds and/or replaces triggers. If run again with only -a (meaning only add missing triggers) no additional triggers are added. This is a good way to check that no new vobs have been created that are missing triggers or to only add triggers to a vob you just created. You could also specify something like mktriggers.pl -a -vobs /vobs/newvob. Combined with no execute mode and you can easily check if there are any missing triggers without adding them (i.e. mktriggers.pl -n -a).

Posted by Andrew DeFaria at 12:54 PM | Permalink

• Investigating check_full_baseline routine and why it is necessary
• Got access to Clearcase storage areas
• Attempting to set up environment to reproduce check_full_baseline problem
• Attended Clearcase World Wide Support meeting
• Attended Plan for ClearQuest meeting

Posted by Andrew DeFaria at 12:26 PM | Permalink

• Installed Clearcase and Clearquest on my system. CC Doctor complains that CQ is old
• Worked with Shivdutt creating new component and the proper parent/child relationships
• Mainly setting up my environment today

Posted by Andrew DeFaria at 06:00 PM | Permalink