Cabie Automated Build System

Cabie is comprised of:

• A client program which submits a build job (by inserting entries into the MySQL database) and can submit build requests for that job to the build servers.
• A mySQL database which accepts job definitions and reporting from the build servers (build request status).
• Numerous build servers which accept the build requests and performs the builds. There is one build server per platform for which builds are to be generated. (i.e. various flavors of Linux, MS/Windows and Unix) The build server will prepare the environment, perform an update and then submit four jobs (shell scripts):
  1. pre-build job: i.e. make clean or ant clean
  2. build job: compilation i.e. make or ant
  3. post-build job: verify make/build, does new build product exist
  4. post-post-build job: regression test build product (application)
  The build server supports the following CM systems:
  • Perforce
  • CVS
  • Subversion
  This tutorial presumes that the CM systems is already installed and configured. See the YoLinux.com Subversion and Trac Server Installation and Configuration tutorial.
• An Apache CGI which interfaces with the database and presents build results logs. It auto-updates the web page to continually show the latest status.

Cabie will build with build monitoring "watchdog" threads to notify of overdue processes or insufficient resources generate email notifications, respond to build inquiries and test. Cabie is written in Perl.

Also see: Cabie process flow diagram.

I'll start by saying that the Cabie perl scripts and configuration are specific to one particular installation and that it will take a lot of tweaking and debugging to get it to work for your specific configuration. It will fail if mis-configured. This tutorial is centric to a Subversion CM integration. A knowledge of Perl is helpful to get Cabie installed and working properly. Unlike other YoLinux.com Tutorials which have a cook book, quick start style, you may find Cabie more complex and difficult to configure. - Good luck!

System Preparation:

The following native Linux packages MUST be installed:

• Web Server: perl, httpd (Apache), mysql, perl-DBI, perl-DBD, smtp server (any email server)
• Build Server: perl, perl-DBI, perl-DBD

Perl:

Downloading / installing Perl with CPAN:

• perl -MCPAN -e shell
• force install threads

Red Hat Enterprise 4 / CentOS 4: Cabie requires the installation of four perl modules (above and beyond the regular Red Hat Enterprise 4 release):

• Mail::Sendmail - rpm -ivh perl-Mail-Sendmail-0.79-1.2.el4.rf.noarch.rpm
• FreezeThaw - rpm -ivh perl-FreezeThaw-0.43-1.2.el4.rf.noarch.rpm
• Archive::Zip - rpm -ivh perl-Archive-Zip-1.16-1.2.el4.rf.noarch.rpm
• Compress::Zlib - rpm -ivh perl-Compress-Zlib-1.33-0.2.el4.rf.i386.rpm

SuSE (9.3 pro): The SuSE 9.3 distribution comes with FreezeThaw, Archive::Zip and Compress::Zlib. Use CPAN to install Mail::Sendmail:

• perl -MCPAN -e shell
• Answer "No"
• install Mail::Sendmail
• quit

Also see the YoLinux SysAdmin Tutorial: Managing Perl

Installation on other platforms is covered below: SGI/IRIX, RHEL3, MS/Windows XP

Cabie Download/Install:

1. Download Cabie source from http://cabie.tigris.org: cabieserver.tar
   Note: This tutorial is for release 0.8.1. Newer releases are available from the CVS repository but may differ slightly from this tutorial.
2. Un-tar in the directory /opt/Cabie/: tar xf cabieserver.tar
   Creates:
   • /opt/Cabie/client/ (Client perl command program)
   • /opt/Cabie/server/ (Perl scripts and libraries, job bash scripts, ...)
   • /opt/Cabie/web/ (CGI, css and icons)
3. Support for Subversion is available by downloading the latest files from the Tigris.org Cabie CVS repository.
4. Remove "^M"s: find /opt/Cabie -name "*" -exec dos2unix {} \;

Update Database interface mysqlwdb.pl:

Download/Install:

• Get the latest version from: MySqlWDB home page
  Download: mysqlwdb-1.5.4.tar.gz
• Untar/compress: tar xzf mysqlwdb-1.5.4.tar.gz
• cp mysqlwdb-1.5.4/mysqlwdb.pl /opt/Cabie/web/cgi-bin/
• Edit the script and specify:
  • database host server
  • MySQL port: 3306
  • Name of database: builds (defined by SQL import file)
  • MySQL user/password (Note: Cabie by default is configured to use a NULL password.)

Cabie Architecture/Topology and System Configuration:

1) Web server mounts the exported drives from build servers

bldsrv1:/share/Cabie/BUILDS /share/Cabie/BUILDS/bldsrv1 nfs auto 0 0
bldsrv2:/share/Cabie/BUILDS /share/Cabie/BUILDS/bldsrv2 nfs auto 0 0
bldsrv3:/share/Cabie/BUILDS /share/Cabie/BUILDS/bldsrv3 nfs auto 0 0
...

/share/Cabie/BUILDS 192.168.1.0/255.255.255.0(rw)

/share/Cabie/BUILDS 192.168.1.0/255.255.255.0(rw)

C:\Cabie\BUILDS

or
2) Build servers mount the exported drive of web server.

/share/Cabie/BUILDS 192.168.1.0/255.255.255.0(rw)

webserver:/share/Cabie/BUILDS /share/Cabie/BUILDS nfs auto 0 0

webserver:/share/Cabie/BUILDS /share/Cabie/BUILDS nfs auto 0 0

The second option is favored if the web server is also a general file server and thus booted first. The clients systems would then be booted afterwards and mount the shared exported drive.

MS/Windows Build Servers: It is required that MS/Windows build servers build on a local drive. This is because Cabie uses the perl disk directive "GetDriveGeometry" which does NOT work on networked Microsoft filesystems. Thus the web server will use an SMB mount to mount the MS/Windows file server.

Cabie Web server SMB mount: /etc/fstab

//bldsrvWIN/bldsrvWIN /share/Cabie/BUILDS/bldsrvWIN smbfs credentials=/home/linux-user-id/.smbpassword,workgroup=MS-WINDOWS-DOMAIN,uid=linux-user-id,gid=linux-user-group-name 0 0

username = ms-windows-login-name
password = super-secret

On MS/Windows, create the share from the directory: C:\Cabie\BUILDS\bldsrvWIN.
(Right click on the directory in MS/Windows Explorer.)

For details see: YoLinux Tutorial: Mounting a MS/Windows share

Another option is to install "Unix services for MS/Windows" and export an NFS partition from the MS/Windows build server. This will be necessary for very strict MS/Windows authentication configurations.

Check-out code from CM: Create the Subversion local repository on the build server. It must exist before starting the Cabie build server.

• cd /share/Cabie/BUILDS/bldsrv1
• svn co http://subversion-server/svn/projectX

The MySQL database is at the heart of Cabie operations. Build definitions are registered in the database. The build servers register and report status and information to the database. The Cabie web CGI uses data stored in the MySQL database to generate a display of build information.

See the YoLinux MySQL tutorial to configure and start the MySQL database.

Create database user buildmgr:

• Assign password: mysqladmin -u root password 'supersecret'
• mysql -h localhost -u root -psupersecret
• mysql> use mysql;
• mysql> INSERT INTO user (Host, User, Password ) VALUES ('localhost','buildmgr',password('buildsupersecret'));
• mysql> INSERT INTO user (Host, User, Password ) VALUES ('%','buildmgr',password('buildsupersecret'));
• mysql> UPDATE user set Select_priv='Y',Insert_priv='Y',Delete_priv='Y',Create_priv='Y' where User='buildmgr';
• mysql> FLUSH PRIVILEGES;
• mysql> CREATE DATABASE builds;
• mysql> GRANT ALL PRIVILEGES ON builds.* TO buildmgr@'%';
• mysql> FLUSH PRIVILEGES;
• mysql> quit

[Potential Pitfall]: The older 1.1 release of Cabie (tar archive) does not have a method for Cabie to use a password and I have found that assigning no password ("") works. I also have been using user "root" for everything. The database login is specified in /opt/Cabie/server/lib/host-name.pm The latest code in the CVS repository does support MySQL login/password.

Import the Cabie database schema:

• cd /opt/Cabie/server/sql
• mysql -h localhost -u login -pPASSWORD
• mysql> use builds;
• mysql> \. schema
• mysql> show tables
• mysql> quit

Tigris.org: Description of Cabie schema

You can now connect with: mysql -h localhost -u buildmgr -pbuildsupersecret

Build Server Process Flow:

• Build Server: /opt/Cabie/server/buildserver.pl: Receives commands from build client program /opt/Cabie/client/build.pl
  • Read configuration from server/lib/host-name.pm
  • Call appropriate subroutine based on hash table of build server commands
    • Command: "createjob":
      • Reads arguments and load database.
      • Data checking performed to see if command arguments are valid.
      OR
    • Command: "build":
      • /opt/Cabie/server/bin/build.pl called by buildserver.pl
      • Gather data and initialize info.
        • Read configuration from server/lib/hostname.pm (98)
        • Store job start time.
        • Calls GetJobInfo() to read database table "configuration" and store values. (325)
        • Calls _initchangeno() and sets BuildNum (363)
      • Start Job:
        • Create Semaphore (blocks new runs till done) (384)
        • Updatejobs: Execute script updatejobs.sh (defined in server/lib/hostname.pm (392)
        • _findbuildscripts() (406)
        • Verify and set: prebuild, postbuild, postpostbuild, onfail, retail and debug.
        • MainBuildProc() (411)
          • Sleep 30 seconds to make sure build area has been updated from CM.
          • Get max build time from build history. (451)
          • Launch watchdog thread. (470)
          • Set environment variables. (486)
          • Update build area: (Call to server/lib/cmbroker.pm)
          • Read job version number. (503)
          • Read file /share/Cabie/BUILDS/buildserver/projectA_job.build (534)
          • Call pre-build script: server/jobs/projectA_job/prebuild.sh (669)
          • _runjob(): Calls server/jobs/projectA_job/rtlbuild.sh (730)
          • Notify via email if build fails.
          • If build OK, then run server/jobs/projectA_job/postbuild.sh
          • Release semaphores for build job.

Build Server Configuration:

Edit/Configure Perl Scripts:

• server/buildserver.pl: Good as is.

• server/bin/builder.pl:
  • (810) In order for server/jobs/projectA_job/postbuild.sh to log results (i.e. log postbuild or postpostbuild test results), capture return value of the script and log to database:
    • Alter "if" block for "postbuild" (search file for "postbuild" in builder.pl) and use the call "system()" instead of "forkprocess()" to use a blocking call which waits till done:

        ..
        ...
      
        @proglist = ("./script","$top","$type","$title","$BuildNum", ...);
        $retVALUE = system(@proglist);
      
        ...
        ..
                  

    • Test "$retVALUE" and log results:

        ..
      
        _logresults($BUILDNUM,$StartTime,$CompileTime,X);
      
        ..
                  

      Logs to database table "jobs" where status=X.

      Web Display	status
      Build Untested	0
      Build Passed Test	3
      Build Failed	2
      Failed Test	1

      These status values correspond to the status colors used.

• server/lib/your-buildserver-host-name-goes-here.pm:
  • This is the Cabie configuration file for the build server.
  • Copy server/lib/unix_example.pm_orig to your-buildserver-host-name-goes-here.pm
    Edit values in this perl module to define your buildserver configuration.
  • Configuration arguments: (partial list)
    

    Variable	Description
    BD	Build Drive: Linux/Posix set to '/' MS/Win set to 'Z:\\'
    BSR	Build Server Root: Name of directory containing buildserver.pl - /opt/Cabie/server MS/Win set to: C:\Cabie\server
    BC	Build Command: Path to build command - /opt/Cabie/server/bin MS/Win: C:\Cabie\server\bin\buildserver.pl
    BTMP	Temp directory. Note that the directory "/tmp" is set as the default destination for debug log files. MS/Win: 'C:\temp'
    CVSROOT	Not used by Subversion.
    COMPANY	Domain to use for email. i.e. your-company.com
    LOGOICON	Icon for company logo. i.e. /CabieIcons/logo.gif
    DFCMD	Command to display free disk space: /bin/df MS/Win: ''
    DEFGROUP	Default group for CM system. Use command "id" to view your group.
    BUGDB	URL of your bug tracking system (i.e. your Trac server) 'http://cm-server/cgi-bin/trac.cgi/ticket/%s'
    WORKDRIVE	Work Drive: Linux/Posix set to '/' MS/Win: '\\'
    NULL	NULL device, Linux: '/dev/null' MS/Win: 'nul'
    JOBDIR	Cabie local builds directory on build server: i.e. /share/Cabie/BUILDS/BuildServerHostname (Match name in database table "configuration" filed "top". Used for checking disk space using command "df") MS/Win: 'C:\Cabie\BUILDS\BuildServerHostname
    MOUNTROOT	Local builds "root" directory on build server: i.e. /share/Cabie/BUILDS MS/Win: 'C:\Cabie\BUILDS
    MAILER	Path to mailer.pl: Use BSR/MAILER: support/mailer Linux: '/opt/Cabie/server/support/mailer.pl MS/Win: C:\Cabie\server\support\mailer.pl'
    PARALLEL	Boolean: set to "0" if jobs are to be run sequentially.
    RLCHG	Obtain "Product Release Number" (corresponds to the "major" version number) from change management (CM) system (var: sccs).
    PRN	Product Release Number: corresponds to the "major" version number.
    INCREMENT	Boolean: Set to false ("0") - default. Set true ("1") to increment "minor" version number even if build fails.
    BLDNUM	Boolean: Incremental build number - "minor" version number read from file build-name.build. Default: 0
    SHAREPOINT	Used for email messages for file URL. 'http://web-server/Cabie-cgi-bin/genweb'
    SMTP	Outgoing SMTP email server. i.e. 'smtp.company.com'
    SMTPDELIM	Delimiter for SMTP addressing: ','
    RELEASE	Name of directory used to copy released builds to. Specify path.
    WEBSERVER	Cabie web server i.e. http://webserver/Cabie/BUILDS
    SQLSERVER	Cabie database server i.e. localhost
    SQLID	Cabie database server login for database connection i.e. buildmgr
    DBPASSWORD	Cabie database server password i.e. supersecret
    POLLFILE	Name of file containing list of build jobs to poll. Default: 'poll.txt'
    PREBUILD	Name of pre-build script file. Default: 'prebuild.sh' MS/Win: 'prebuild.bat'
    RETAIL	Name of retail build script file. Default: 'rtlbuild.sh' MS/Win: 'rtlbuild.bat'
    POSTBUILD	Name of post-build script file. Default: 'postbuild.sh' MS/Win: 'postbuild.bat'
    POSTPOSTBUILD	Name of post-post-build script file. Default: 'postpostbuild.sh' MS/Win: 'postpostbuild.bat'

• server/updatejobs.sh: Called by "server/bin/builder.pl". Not required. Intended to update the Cabie build server job build scripts (/opt/Cabie/server/jobs). Used if Cabie itself is stored in a CM system. Updates Cabie job and build scripts from CM.

  #!/bin/bash
  
  exit 0
      

  Cabie includes an example command if using the CM system Perforce.

• If adding a CM system not included as default, add support in the perl module server/lib/cmbroker.pm

Continuous Build Mode:

projectA_job
projectB_job

...
..

• The file "poll.txt" is specified in the Cabie build server configuration file: server/lib/build-server-host-name.pm as specified by the configuration variable "POLLFILE" in build server configuration file: server/lib/your-host-name-goes-here.pm
• Any job specified in the poll.txt file will be polled and Cabie will operate in "Continuous Automated Build" mode. Cabie will constantly check for any update to the CM repository. If a check-in is detected, then Cabie will start the automated build process. To specify builds manually (or through cron), remove the job in poll.txt (or comment it out with a leading "#") and use the Cabie client to submit the build job (shown below).
• The poll interval is hard coded in server/buildserver.pl for variable "sleepinterval". This can be customized to suit your environment. This variable is used in two places for two different purposes. One in "sub pendingjobs" is for polling the CM repository for updates and inserts results into the "pending" table used only for the mouse-over display of the "I" information icon. The other "sleepinterval" in the "sub polljobs" function is used for the job poll interval for build jobs. This is generally of equal or larger value.

Job Build Scripts:

The scripts postbuild.sh and onfail.sh should be generic and not job specific. You should be able to use the same postbuild.sh and onfail.sh scripts for different jobs. The scripts prebuild.sh and rtlbuild.sh are more likely to have job specific directives.

• prebuild.sh:

  #!/bin/bash
  
  export PATH=$PATH:/opt/bin
  TOP=$1
  TYPE=$2
  TITLE=$3
  
  if [ ! -d "$top/projectX" ];
  then
      echo "Build directory $TOP/$TITLE does not exist!"
      exit 1
  fi
  
  cd  $TOP/$TITLE
  
  if [ -f configure ];
  then
          ./configure --prefix=/opt
  fi
  if [[ $? != 0 ]];
  then
    echo "prebuild.sh configure ERROR!"
    exit 1
  fi
  
  echo "configure completed."
  
  if [ -f Makefile ];
  then
          make clean
  fi
  if [[ $? != 0 ]];
  then
    echo "prebuild.sh ERROR!"
    exit 1
  fi
  echo "make clean completed."
      

  arguments:
  • $1 = Top of the source tree (Directory path: /share/Cabie/BUILDS/BuildServerHostname)
  • $2 = The build type (retail, debug, or both)
  • $3 = Title of build (job name: i.e. projectA_job)

• rtlbuild.sh:

  #!/bin/bash
  
  export PATH=$PATH:$JAVA_HOME/bin:/opt/bin
  export JAVA_HOME=/usr/java/jdk1.5.0_05
  export CLASSPATH=/usr/java/jdk1.5.0_05/lib/tools.jar:/usr/java/jdk1.5.0_05/jre/lib/rt.jar:./
  
  TOP=$1
  TITLE=$2
  NUM=$3
  export DIST_DIR=$TOP/$TITLE.$NUM
  
  export BUILD_NUM=$(cat $TOP/$TITLE.build)
  
  cd  $DIST_DIR
  
  cat > $DIST_DIR/build.properties <<-EOF
          Build Numer=$BUILD_NUM
          Buildserver DIR=$(pwd)
          dist.dir=$DIST_DIR
          C++ compiler=`g++ --version`
          jdk=$JAVA_HOME
          Job Title=$TITLE
  EOF
  
  if [ -f Makefile ];
  then
          make
  fi
  if [[ $? != 0 ]];
  then
    echo "Failure to build"
    exit 1
  fi
  echo "make completed"
  
  exit 0
      

  arguments:
  • $1 = Top of the source tree (Directory path: /share/Cabie/BUILDS/BuildServerHostname)
  • $2 = Title of build
  • $3 = Unique build number
  Note absence of "type". This reflects the configuration of the current release of Cabie.
  This script generates the file /share/Cabie/BUILDS/BuildServerHostname/projectA_job/build.properties which is delivered by the build server to the web CGI when requested.

• dbgbuild.sh:
  This script builds a "debug" version of the build based on the "type" specified with the "createjob" command. The "type" is stored as a field in the "configuration" table which stores the job description. Two types are allowed:
  • retail
  • debug

• postbuild.sh:
  Analyze the build to see if it was successful.

  #!/bin/bash
  
  export PATH=$PATH:/opt/bin
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/lib
  export TOP=$1
  export TYPE=$2
  export TITLE=$3
  export NUM=$4
  
  # Create output directory
  
  export DATA_OUTPUT=$TOP/$TITLE/$NUM
  
  if [ ! -d $DATA_OUTPUT ]
  then
     mkdir -p $DATA_OUTPUT
  fi
  
  cd $TOP/$TITLE
  cp *.log  $DATA_OUTPUT
  
  # Prepare log files for presentation via web
  
  BLDSRVDIR=/opt/Cabie/server
  BLDSRVDIR_SHARED=$BLDSRVDIR/jobs/shared
  
  sleep 10
  
  # This script is only executed id rtlbuild.sh succeeds. 
  # Thus build is good - record this fact.
  
  echo $NUM > $TOP/builds/$TITLE.lastgood
  
  
  perl $BLDSRVDIR_SHARED/getlogs.pl `hostname` $TITLE $NUM
  perl $BLDSRVDIR_SHARED/buildfuncs.pl removecomments
  
  if [ -f "$DATA_OUTPUT/$TITLE.$TYPE.log" ];
  then
     perl -I$BLDSRVDIR/lib \
          $SRV_SHARED_DIR/log2html.pl -e $BLDSRVDIR/config/$TITLE.errors \
                                      -i $DATA_OUTPUT/$TITLE.$TYPE.log   \
                                      -o $DATA_OUTPUT/build_log.html
  else
     perl -I$BLDSRVDIR/lib \
          $SRV_SHARED_DIR/log2html.pl -e $BLDSRVDIR/config/$TITLE.errors  \
                                      -i $DATA_OUTPUT/$TITLE.prebuild.log \
                                      -o $DATA_OUTPUT/prebuild_log.html
  fi
  
  # Test if executable has been created
  
  if [ ! -f application.exe ];
  then
      exit 1
  fi
  
  if [ -f Makefile ];
  then
      make install
  fi
  
  exit 0
      

  arguments:
  • $1 = Top of the source tree (Directory path: /share/Cabie/BUILDS/BuildServerHostname)
  • $2 = The build type
  • $3 = Title of build
  • $4 = Unique build number
  Note: Requires error file build-title.errors used as input to log2html script described below.

• onfail.sh:

  #!/bin/bash
  
  TOP=$1
  TYPE=$2
  TITLE=$3
  NUM=$4
  
  sleep 10
  
  export LOG_OUTPUT=$TOP/$TITLE
  export DATA_OUTPUT=$TOP/$TITLE/$NUM
  
  if [ ! -d $DATA_OUTPUT ]
  then
     mkdir -p $DATA_OUTPUT
  fi
  
  cd $TOP/$TITLE
  cp *.log  $DATA_OUTPUT
  
  # Prepare log files for presentation via web
  
  BLDSRVDIR=/opt/Cabie/server
  BLDSRVDIR_SHARED=$BLDSRVDIR/jobs/shared
  
  perl $BLDSRVDIR_SHARED/getlogs.pl `hostname` $TITLE $NUM &
  perl $BLDSRVDIR_SHARED/buildfuncs.pl removecomments
  
  if [ -f "$DATA_OUTPUT/$TITLE.$TYPE.log" ];
  then
     perl -I$BLDSRVDIR/lib \
          $SRV_SHARED_DIR/log2html.pl -e $BLDSRVDIR/config/$TITLE.errors \
                                      -i $DATA_OUTPUT/$TITLE.$TYPE.log   \
                                      -o $DATA_OUTPUT/build_log.html
  elif [ -f "$DATA_OUTPUT/$TITLE.prebuild.log" ];
  then
     perl -I$BLDSRVDIR/lib \
          $SRV_SHARED_DIR/log2html.pl -e $BLDSRVDIR/config/$TITLE.errors  \
                                      -i $DATA_OUTPUT/$TITLE.prebuild.log \
                                      -o $DATA_OUTPUT/prebuild_log.html
  elif [ -f "$DATA_OUTPUT/$TITLE.sync.log" ];
  then
     perl -I$BLDSRVDIR/lib \
          $SRV_SHARED_DIR/log2html.pl -e $BLDSRVDIR/config/$TITLE.errors \
                                      -i $DATA_OUTPUT/$TITLE.sync.log   \
                                      -o $DATA_OUTPUT/sync_log.html
  fi
  
      

  
  arguments:
  • $1 = Top of the source tree (Directory path: /share/Cabie/BUILDS/BuildServerHostname)
  • $2 = The build type
  • $3 = Title of build
  • $4 = Unique build number

Regression Testing Application:

#!/bin/bash

TOP=$1
TYPE=$2
TITLE=$3
NUM=$4

sleep 10

cd  $TOP/$TITLE

if [ -f Makefile ];
then
        make run
fi

/opt/bin/examine-test-results.pl         - Write this script to suite your app
                                           Read program output from test run, use JUNIT or CppUnit, ...
    

• $1 = Top of the source tree (Directory path: /share/Cabie/BUILDS/BuildServerHostname)
• $2 = The build type
• $3 = Title of build
• $4 = Unique build number

The processing is invoked by the build server (server/buildserver.pl) which calls server/bin/builder.pl which in turn calls the above scripts in succession if no errors are encountered. An error in "prebuild.sh" or "rtlbuild.sh" ends the processing with a final call to the script "onfail.sh". If the build is successful then an install (postbuild.sh) and test (postpostbuild.sh) will occur.

Support Scripts:

fatal error
Fatal error
error
Error
ERROR
Unexpected Error
1 error
cannot resolve symbol
Error while building files
Name assignment wrong
** Error
')' expected
'(' expected

File permissions:

Build Numbers:

1. Cabie can obtain a build number by using the revision number supplied by Subversion by executing the command (in cmbroker.pm): "svn info repository-dir" Any attempt to re-run a build with the same build number will be aborted. Cabie first checks with the "jobs" table in the MySQL database for duplication with any previous build numbers. Duplication results in the refusal to perform a build. The absence of the file /share/Cabie/BUILDS/BuildServerHostname/projectA_job.build and the build server configuration file variable "RLCHG" in server/lib/your-host-name-goes-here.pm defines this behavior.
2. Increment the build number based on a the major version number set in the build server configuration file server/lib/buildserver-host-name.pm variable "PRN" and a minor version number stored in a file. The major version number "PRN" can also be obtained from the file /share/Cabie/BUILDS/BuildServerHostname/projectA_job.version. This is used for CM systems which do not have a global revision number like Clearcase. The "minor" build number is stored in the file $TOP/$TITLE.build (i.e. /share/Cabie/BUILDS/BuildServerHostname/projectA_job.build). To use this methodology, create this file and insert the number "1". The existence of this file triggers its use. Removal of this file will cause Cabie to query CM for a build number. The build server configuration file boolean variable "INCREMENT" can be set to "1" to always increment the build number. The resulting build number will be "major-version-number.minor-version-number".

Job Descriptions:

This can be interrogated later with a working system using the web interface and selecting the link with the job name (projectA_job) or the command: /opt/Cabie/client/build.pl instructions -n projectA_job

<h2>projectA_job Job Information</h2>

<b>Build and Development Info:</b> <a href="http://cm-server/cgi-bin/trac.cgi/wiki/projectA">Trac wiki</a>

<b>Subversions:</b> <a href="http://cm-server/svn/trunk/productA/">SVN productA Repository</a> 
 - <a href="http://cm-server/cgi-bin/trac.cgi/browser/trunk/projectA">Trac Subversion View</a>

<b>Bugs:</b> <a href="http://cm-server/cgi-bin/trac.cgi/report/1">Trac tickets</a>

<b>Permissions:</b> This repository is open for all in group "dev".

<b>Tree Etiquette"</b> Before you check in, please make sure that your code compiles and runs.

<b>Output:</b> Generated binaries and build logs are available 
 <a href="http://cm-server/Cabie/BUILDS/productA_job/">here</a>.

<b>Job description:</b> Build using Makefile.

<b>Questions:</b> <a href="mailto:dude@mail-server">Email Dude</a>

Job Configuration:

[Potential Pitfall]: The build server process must be able to read and execute the scripts and read the information files:

chmod ug+x -R /opt/Cabie/server/jobs
chown buildmgr:buildmgr -R /opt/Cabie

Debugging:

• The Cabie debug mechanism allows you to call _debug("Write your message here with $variable);. This will send a message to the debug log file specified in server/lib/hostname-of-build-server.pm. Look for the debug messages in /tmp/debug.out
• The directory /opt/Cabie/proc/ is where Cabie writes STDOUT and STDERR along with connection and rejection logs. Look at these log files for errors.
• Printing info using Perl:

  ...
  
  open(HNDL, ">>/tmp/cabiedbg.log");
  print HNDL "Debug message goes here. Variable= $var"
  close(HNDL);
  
  ...
          

I tried using the SGI and after market freeware CDs of open source software for MySQL and Perl but found that only the following worked:

• Install support packages from SGI freeware CD's:
  (May 2004: http://freeware.sgi.com/)
  • fw_libz
  • fw_openssl
  • fw_readlines
• MySQL: (client libraries required by perl module DBD::Mysql)
  • Download pre-compiled MySQL tar archive for IRIX from MySQL.org.
  • Install tar archive:
    • cd /usr/local
    • tar xzf mysql-standard-4.1.21-sgi-irix6.5-mips.tar.gz
    • mv mysql-standard-4.1.21-sgi-irix6.5-mips/ mysql/
• Perl: (Must compile from source for threads support required by Cabie.)
  • Download Perl source from http://CPAN.org
  • tar xjf perl-5.8.8-stable.tar.bz2
    (By contrast, RHEL4 uses Perl 5.8.5)
  • cd perl-5.8.8
  • sh Configure -Dcc='cc -n32' -Duseithreads -d
  • make
    (Uses native SGI IRIX make and cc compiler.)
    Note: fails test "stat" but seems to have had no apparent effect.
  • make install
    (Installs to /usr/local)
  • Set environment variables:
    • export PATH=/usr/local/mysql/bin:/usr/local/bin:/usr/freeware/bin:$PATH
      (Note: The command /usr/local/mysql/bin/mysql_config is required for the installation of perl module DBD::mysql)
    • export LD_LIBRARYN32_PATH=/usr/local/mysql/lib:/usr/local/lib:/usr/freeware/lib:$LD_LIBRARY32_PATH
  • Create Cabie perl path: ln -s /usr/local/bin/perl /usr/bin/perl
• Perl Modules:
  • Download Perl modules tar packages from http://CPAN.org:
    (Install in this order.)
    • Mail-Sendmail-0.79.tar.gz
    • FreezeThaw-0.43.tar.gz
    • DBI-1.52.tar.gz
    • DBD-mysql-3.0006.tar.gz
    • Compress-Zlib-1.42.tar.gz
    • Archive-Zip-1.16.tar.gz
    • libwww-perl-5.805.tar.gz
      (Used to supply HTTP::Date Perl module)
  • Install:
    • Mail::Sendmail: (typical)
      • tar xzf Mail-Sendmail-0.79.tar.gz
      • cd Mail-Sendmail-0.79
      • perl Makefile.PL
      • make
      • make install
    • ...
    • DBD::mysql:
      • tar xzf DBD-mysql-3.0006.tar.gz
      • cd DBD-mysql-3.0006
      • perl Makefile.PL --libs="-L/usr/local/mysql/lib -lmysqlclient -lz -lgen -lnsl -lm"
        Where the arguments can be obtained from: /usr/local/mysql/bin/mysql_config --libs
        or
        perl Makefile.PL
        Where the environment variable PATH includes the MySQL command mysql_config
      • make
      • make install
    • ...

See installation and configuration instructions above.

See installation and configuration instructions above.

The version of perl included with Red Hat Enterprise Linux 4 works with Cabie. The version of perl included with Red Hat Enterprise Linux 3 (RHEL3) does not! It is not threaded in the traditional manner as a new process is generated for each thread. If using RHEL3, download the perl RPM from Active State Perl. (The tar.gz bundle does not include many of the basic modules you need, download the RPM as this includes additional Perl modules.)

• Install: rpm -ivh ActivePerl-5.8.8.817-i686-linux-2.2.17-gcc-257965.rpm
• Remove old perl: rpm -e perl
• Create link so "/usr/bin/perl" works: ln -s /opt/ActivePerl-5.8/bin/perl /usr/bin/perl

If using "Active State Perl" you will have to download the Perl database, network, email and file compression interfaces from CPAN.org. Install in the following order:

• DBI:
  • Download DBI-1.52.tar.gz
  • tar xzf DBI-1.52.tar.gz
  • cd DBI-1.52
  • perl Makefile.PL
  • make
  • make install
• DBD-mysql 3.0006
• FreezeThaw 0.43
• Mail-Sendmail 0.79
• Compress-Zlib 1.10
• Archive-Zip 1.16

Install MySQL client, Subversion client, Perl and the appropriate modules to support the Cabie build server.

MySQL Client:

Download MySql for MS/Windows (version 4.1 to be compatible with RHEL4) the installer from MySQL.com (Full install: mysql-4.1.21-win32.zip) During the installation you will be queried for the components to install. Choose "Client software".

Subversion Client:

Download from http://subversion.tigris.org (svn-win32-X.X.X.zip)

Unzip archive into C:\SVN\. Add C:\SVN\bin to your system path. Path must match that used in server/lib/cmbroker.pm for svn.exe if path is fully qualified.

Perl:

Download Active State Perl ".msi" package for MS/Windows:
ActivePerl-5.8.8.819-MSWin32-x86-267479.msi
This MS/Windows installation package will include the "Perl Package Manager" (ppm)

Additional modules are required and can be (1) downloaded as PPM's and installed or (2) installed from an internet connection.

1. Download and install PPMs:
   • ActiveState.com:
     • DBI (DBI-1.50.zip)
     • DBD::mysql (DBD-mysql-3.0002.zip)
     • FreezeThaw (FreezeThaw-0.43.zip)
     • Mail::Sendmail (Mail-Sendmail-0.79.zip)
   • Roth.net:
     • Win32::Daemon (win32-daemon.ppd)
     • Win32::AdminMisc (win32-adminmisc.ppd)

   Unzip the ppm packages and install: Start the Perl Package manager console (command "ppm") and enter the command:

   install C:\....\package-name.ppd

   or from the command line console (cmd):

   ppm install C:\....\package-name.ppd

2. Install from internet archive:
   Start the Perl Package manager console (command "ppm") and enter the commands:
   • ppm> install DBI
   • ppm> install DBD-mysql
   • ppm> install FreezeThaw
   • ppm> install Mail-Sendmail
   • ppm> install http://www.roth.net/perl/packages/win32-daemon.ppd
   • ppm> install http://www.roth.net/perl/packages/win32-adminmisc.ppd
   The first four packages will be obtained from the default archive - ActiveState.com.

   If using a proxy, set the environment variable http_proxy to http://name-of-proxy-server:80

   Check installation with the following commands:

   • View list of all installed packages: ppm> query *
   • View list of family of installed packages: ppm> search mysql

Job scripts:

• prebuild.bat
• rtlbuild.bat
• postbuild.bat
• postpostbuild.bat

Environment variables for use with the MS/Visual Studio compiler can be found in:

• MS/Visual Studio .NET 2003: C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\vsvars32.bat
• MS/Visual Studio 8.0: C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat

MS/Windows build server:

C:\Cabie\server\buildserver.pl -s 8091 -m 8092 -i

1. Start
2. Control Panel
3. Administrator tools
4. Services
   • There will be two build server services. Select each service individually.
   • Right click and select "Start"

The Cabie web interface is a CGI interface to the Cabie data residing in a MySQL database (in this case on the same server: localhost)

Cabie CGI:

• Main interface: /opt/Cabie/web/cgi-bin/genweb
  (Also: server/buildserver.pl, web/cgi-bin/: status.pl, gencomment, defectfix, jobinfo and server/lib: cabbieaddon.pm, hostname-of-buildserver.pm)
  • Edit/change lines which refer to "/icons" to "/CabieIcons/"
  • Edit/change lines which refer to "/cgi-bin/" to "/Cabie-cgi-bin/"
  • Edit/change lines which refer to "/main/static" to "/Cabie/static/"
  • See bugs by running at the command line. Genweb will complain as there are sure to be loads of undefined parameters which can be set in /opt/Cabie/server/lib/host-name.pm for an element of "%fields".
• /opt/Cabie/web/cgi-bin/status.pl
  • Define reference to PWEBSERVER in "%fields"..
    (Used by Perforce CM system. Even if not used set it to something.)
• /opt/Cabie/web/cgi-bin/jobinfo
  • Change path of "/opt/bin/build" to reflect actual path: /opt/Cabie/client/build.pl and arguments "-s name-of-build-server -p 9190 describe -n nameOf_job"
  • Set /opt/Cabie/web/icons/sct_logo.gif to your corporate logo.
• Files missing from Cabie tar file: I noticed that some of the files missing from the Cabie tar file are available from the Cabie CVS repository on Tigris.org. (i.e. web/icons/logo.gif, web/icons/cabie.gif, ...)

• chown apache:apache -R /opt/Cabie/web
• chmod ug+x -R /opt/Cabie/web/cgi-bin

Tip: CGI can be debugged by looking at the errors in : /var/log/httpd/error_log

File:

• Red Hat/Fedora Core/CentOS4: /etc/httpd/conf.d/cabie.conf
• SuSE 9.3: /etc/apache2/conf.d/cabie.conf

Alias /Cabie/BUILDS "/share/Cabie/BUILDS/"
<Directory "/share/Cabie/BUILDS/">
   Options Indexes FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
</Directory>

Alias /main/ "/opt/Cabie/web/htdocs/main/"
<Directory "/opt/Cabie/web/htdocs/main">
   Options Indexes FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
</Directory>

Alias /CabieIcons/ "/opt/Cabie/web/icons/"
<Directory "/opt/Cabie/web/icons">
   Options Indexes FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
</Directory>

ScriptAlias /Cabie-cgi-bin/ "/opt/Cabie/web/cgi-bin/"
<Directory "/opt/Cabie/web/cgi-bin">
   Options Indexes FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
</Directory>

Restart web server: service httpd restart

The Cabie URL: http://name-of-cabie-web-server/Cabie-cgi-bin/genweb
(to view jobs)

Cabie directories:

• Build server: Constructs path based on data held in the database. It uses the table "configuration" and the fields "top" (top directory for build) and "title" (project title). It generates results files based on the path constructed from putting together /"top"/"title"/BuildNumber
  The build server can have a unique path for each build as each build has a unique database record.
• Web server: Constructs the a path for each build but must use the same formula for each build. The path is constructed from: /MOUNTROOT(specified in /opt/Cabie/server/lib/hostname.pm)/buildserver-hostname/"title"/BuildNumber

ln -s /top/title /MOUNTROOT/hostname/title

The same effect may be achieved by selecting a mount point to meet this criteria.

[Potential Pitfall]: If using SELinux, you will have a lot to configure. I suggest turning this mode off. See YoLinux Apache Configuration: SELinux

1. Add authorization privileges to database: Chicken-Egg problem - The Cabie build server can not run to accept the "authorize" command if it is not first authorized to run. Use a SQL insert to make the table entry:

   mysql -h localhost -u root
   mysql> use builds
   mysql> insert into authtable values('build-server','client-machine');
   mysql> quit

   You must also include an entry for the Cabie web server for each build server as it acts as a client to each build server.

   Test: /opt/Cabie/client/build.pl -s build_server -p 9190 authorized

   [Potential Pitfall]: You may have to use the fully qualified name of the computer. i.e. "buildserver1" will not be authorized if the network defines the computer as "buildserver1.megacorp.com".

2. Start the build server daemon on the build server:
   • Linux/Unix:
     • cd /opt/Cabie/server
       (Run from server directory so that perl paths to library work.)
     • /opt/Cabie/server/buildserver.pl -s 9190 -m 9192 -d
     or use system init script shown below: service cabie start
   • MS/Windows:
     • cd C:\Cabie\server
     • C:\Cabie\server\buildserver.pl -s 8091 -m 8092 -i
     [Potential Pitfall]: The server and monitor ports are hard coded in /opt/Cabie/server/lib/winsys.pm On MS/Windows you must use the hard coded values. We never got user specified values to work on MS/Windows.
   • -s: Server port (default 8091)
   • -m: Monitor port (default 8092)
   • -d: Create server daemon (Linux/Unix)
   • -i: Install MS/Windows service. (You must have admin privileges.) Service registration allows for a permanent configuration where the build server will start/stop upon system boot/shutdown.
   • -r: Remove MS/Windows service.
   Upon starting the build server, it will add a record to the table "buildservers".
   Note: Look for progress and errors in the buildserver log files located in /opt/Cabie/proc/.

3. Test build server using the following client commands:
   • List supported commands: /opt/Cabie/client/build.pl -s build-server -p 9190 commands -l -w
   • Show server daemon process id: /opt/Cabie/client/build.pl getid
     (Use environment variable or specify "-s" and "-p" as above.)
   • Show server configuration: /opt/Cabie/client/build.pl dumpconfig
   Note: You can specify the build-server and port using an environment variable: export BLDSERVER=build-server:port

4. Define a build job using Cabie client /opt/Cabie/client/build.pl:
   • Define system configuration:
     • /opt/Cabie/client/build.pl authorize -c server-1
     • /opt/Cabie/client/build.pl authorize -c server-2
     • ...
   • One must first create the job build scripts (prebuild.sh, rtlbuild.sh, ...) as shown above before a build job can be operational.

   • One must then define the build job:
     /opt/Cabie/client/build.pl -s build-server -p9190 createjob -n projectA_job -p http://cm-server:80/svn/ -c projectA -r /share/Cabie/BUILDS/BuildServerHostname -t retail -d "/opt/bin" -k 10 -s subversion -b CM-URL -m 1 -C "Job Description"

     Command argument	Description
     -n job-name	Name of job. i.e.: projectA_job
     -p port-number -p svn-sub-dir	build-server:port Subversion: Adds this info to mouse-over data.
     -c p4-port or -c projectA	Subversion: Subdirectory under "source root" /share/Cabie/BUILDS/BuildServerHostname which is the Subversion local buildserver repository.
     -r source-root	CM repository root directory. Subversion local repository parent directory.
     -t retail	Build type: debug | retail | both.
     -d /path/to/tools	Path to build tools to be included in path: ie: /opt/bin (Win32: C:\SVN\bin)
     -k keeplevel	How many to keep in database: i.e. 10
     -s subversion	CM repository type: perforce, cvs or subversion
     -b URL	CM repository URL: http://cm-server/...
     -m 1	Build server failure notification: 1 or 0
     -C "comment"	Comment to describe build.

     Note:

     • Command arguments MUST be given in the same order as above or you will get an error complaining.
     • Verify entry using the Cabie client command: /opt/Cabie/client/build.pl describe -n projectA_job
     • Defining a job with "createjob" is run once. It can not be re-run with the same job name. Once can:
       • Delete the job definition and then create a new job definition.
       • Edit an existing job definition in the mySQL database.
       • Create a new job definition with a new job name.

     The build job definition will be inserted into the "configuration" table:

     Field	Example value
     server	nameOfBuildserverHostname
     title	ProjectX_job
     port	http://webserver.com:80/svn/stuff/
     client	ProjectX
     top	/opt/Cabie/BUILDS/buildserver/svn/stuff/local/working/dir/projectX
     type	retail
     toolsdir	/opt/bin
     isimake	no
     keeplevel	10
     comment	ProjectX comments goes here
     sccs	subversion
     browserlink	http://webserver.com/trac/browser/trunk/working/dir/projectX
     state	0
     spam	1
     buildsize	1

     [Potential Pitfall]: The error "Unsupported request" can result from a client which is not authorized to submit the "createjob" command (or any other command). See step above to create an entry into the mySQL authorization table "authtable".

   • The build request must then be defined in the database. If you are not in polling mode, you can manually issue a build command to the build server by issuing the client command:
     /opt/Cabie/client/build.pl -s build-server -p9190 build -n projectA_job

     [Potential Pitfall]: If you find or suspect that the client can not connect to the build server, test the connection with the telnet command: telnet builServerHostname 9190

     • OK:

       Connected to builServerHostname
       Escape character is '^]'.
               

     • No connection:

       telnet: connect to address XXX.XXX.XXX.XXX: Connection refused
       telnet: Unable to connect to remote host: Connection refused
               

     Solution: Remove the host name assignment to the localhost IP address "127.0.0.1

     • Linux/Unix: /etc/hosts
     • MS/Windows: C:\WINDOWS\system32\drivers\etc\hosts

     Hosts file: (even if using DNS)

     • OK:

       127.0.0.1        localhost.localdomain  localhost
       192.168.XXX.XXX  buildserver1
       
            

     • Not OK:

       127.0.0.1        buildserver1 localhost.localdomain  localhost
       192.168.XXX.XXX  buildserver1
       
            

     • Not OK: (MS/Windows default)

       127.0.0.1        localhost
                

     Kill and restart the buildserver.pl process if you have performed this change.

     The network information command: netstat -punta | grep 9190 can also show if the build server is attached to 127.0.0.1 only and thus not accepting network connections.

5. View progress via the web:
   • See status and interact with Cabie: http://web-server/genweb
   The web page will show the nine stages of progress in a color coded scheme:

   Job Idle	Not listed in database.
   Syncing	Creating build environment and performing CM update (cmbroker.pm::sccs_update()). Includes 30 sleep.
   Building	Script server/jobs/projectA_job/rtlbuild.sh launched.
   Overdue	Builds lasting over 150% of average build time.
   Generating Email
   Build Untested	Build complete. Next: check with postbuild.sh. Results of "make" OK Table: jobs, Field: status=0
   Build (compile/link) Passed Test	No build test errors according to postbuild.sh Table: jobs, Field: status=3
   Build Failed	Errors (compile or link) found in build by postbuild.sh Table: jobs, Field: status=2
   Failed Test	Run postpostbuild.sh regression test on application. Test results not good. Table: jobs, Field: status=1

   Note: The first few builds generated by Cabie will be listed as "overdue" for part of the duration of the build. This is because there are no build times or an insufficient number of build times stored in the database. Once data on a build is stored in the system, Cabie will have build times which can be used for judging build duration.

   Look for CGI errors in /var/log/httpd/error_log.

The Cabie build server can be terminated with the command: /opt/Cabie/client/build.pl shutdown

[Potential Pitfall]: The case (upper vs lower) of the hostname MUST be consistent throughout the configuration. The host name used in the job definition will be used to define the directory path of the builds directory as viewed by the web server. Entries in the database and the hostname used to define the configuration file /opt/Cabie/server/lib/hostname.pm and Perl module are all tied together in a single configuration. Consistency is important.

[Potential Pitfall]: Database cleanup for a build which gets stuck or for a build in an inconsistent state (i.e. if a build server is shut down during a build, the database will be in an inconsistent state. Cleanup as follows:
Connect to the MySQL database: mysql -h localhost -u buildmgr -p

• delete from joblog where server='buildsrvrx' and title='ProjectX';
• delete from jobs where server='buildsrvrx' and title='ProjectX' and job='1027';
• delete from semaphores where server='buildsrvrx' and title='ProjectX';
• delete from restore where server='buildsrvrx' and title='ProjectX';
• delete from proctree where server='buildsrvrx' and title='ProjectX';

• cd /opt/Cabie/server
• /opt/Cabie/client/build.pl -s buildsrvrx -p 9190 cleanjob -n ProjectX_job -j job-number-to-purge -f

Create a list of users who will receive email notification:

login-name-1:email-1@domain.com:FirstName1 LastName1
login-name-2:email-2@domain.com:FirstName2 LastName2

...
..

This is when "Continuous Automated Build" mode is turned off by removing the reference to a job from server/poll.txt.
The cron job can manually submit the job.

Cron script: /etc/cron.daily/cabie

#!/bin/sh
/opt/Cabie/client/build.pl -s build-server -p9190 -n projectA_job

/opt/Cabie/client/build.pl -s build-server -p9190 -n projectB_job
                

See the YoLinux.com Tutorial on scheduling jobs.

The build server can be started upon system boot with the following init script. See YoLinux Init Tutorial for configuration.

#!/bin/bash
#
# cabie        This starts and stops cabie build server.
#
# chkconfig: 345 98 35
# description: The Cabie build server is an automated distributed build system.
#

# Source function library.
. /etc/init.d/functions

# Get config.
test -f /etc/sysconfig/network && . /etc/sysconfig/network

# Check that networking is up.
[ "${NETWORKING}" = "yes" ] || exit 0

[ -f /opt/Cabie/server/buildserver.pl ] || exit 1
[ -f /opt/Cabie/client/build.pl ] || exit 1

prog="Cabie build server"

start(){
    echo -n $"Starting $prog: "

    ulimit -S -c 0 >/dev/null 2>&1
    RETVAL=0
    initlog $INITLOG_ARGS -c \
       "/opt/Cabie/server/buildserver.pl -s 9190 -m 9192 -d"
    RETVAL=$?
    [ "$RETVAL" -eq 0 ] && success $"Cabie build server startup" || \
       failure $"cabie start"
    echo
    [ "$RETVAL" -eq 0 ] && touch /var/lock/subsys/cabie
    return $RETVAL
}

stop(){
    echo -n $"Stopping $prog: "
    initlog  $INITLOG_ARGS -c \
       "/opt/Cabie/client/build.pl -s `hostname` -p 9190 shutdown >/dev/null 2>&1"
    RETVAL=$?
    [ "$RETVAL" -eq 0 ] && success $"Cabie build server shutdown" || \
       failure $"Cabie buildserver shutdown"
    echo
    [ "$RETVAL" -eq 0 ] && rm -f  /var/lock/subsys/cabie
    return $RETVAL
}

# See how we were called.
case "$1" in
    start)
	start
	;;
    stop)
	stop
	;;
    restart|reload)
	stop
	sleep 3
        start
	;;
    condrestart)
        if [ -f /var/lock/subsys/cabie ]; then
            stop
            sleep 3
            start
        fi
	;;
    status)
	status buildserver.pl
	;;
    *)
	echo $"Usage: $0 {start|stop|status|restart|condrestart|reload}"
	RETVAL=1
esac

exit $RETVAL

Register the service: chkconfig --add cabie
or: chkconfig --level 35 cabie on

Manually start service: service cabie start

[Potential Pitfall]: The above init script does not work for Red Hat Enterprise 3. (Works for RHEL4) For some unknown reason, this script when executed by the "service" command (service cabie start) on RHEL3, will cause the build server to launch four processes instead of the regular two buildserver processes (server and monitor process) thus use the script linked below instead.

SGI IRIX init script: /etc/init.d/cabie
SuSE Linux init script: /etc/init.d/cabie
RHEL3 Linux init script: /etc/init.d/cabie

Be sure to set execute privileges on the script:

chmod +x /etc/init.d/cabie

I have found it necessary to recycle the build scripts weekly using a cron job. Be sure to set execute privileges on the script:

File: /etc/cron.weekly/cabie.sh (Red Hat Enterprise 4)

#!/bin/bash
/etc/init.d/cabie stop

# Ensure jobs have terminated even if stubborn
ps axwww | grep "buildserver" | awk '{print $1 " " $3} | \
           grep Sl | awk '{print $1}' | \
           ( while read pid; do kill -9 $pid; done )

/etc/init.d/cabie start
        

File: /etc/cron.weekly/cabie.sh (Red Hat Enterprise 3)

#!/bin/bash
/etc/init.d/cabie stop

# Ensure jobs have terminated even if stubborn
ps axwww | grep "buildserver" | awk '{print $1}' | \
           ( while read pid; do kill -9 $pid; done )

/etc/init.d/cabie start
        

• Check perl script syntax: perl -c perl-script.pl
  Should respond with: ./perl-script.pl syntax OK

• Verify Perl modules required for Cabie with the following scripts:
  • testFreezeThaw.pl
    

    #!/usr/bin/perl
    eval "use FreezeThaw"; $hasModule = $@ ? 0 : 1;
    printf "FreezeThaw". ($hasModule ? "" : " not") . " installed";
    printf "\n";
            

  • testMailSendmail.pl
    

    #!/usr/bin/perl
    eval "use Mail::Sendmail"; $hasModule = $@ ? 0 : 1;
    printf "Mail::Sendmail". ($hasModule ? "" : " not") . " installed";
    printf "\n";
            

  • etc ...

• Perl library paths: One may add the fully qualified path name:

  BEGIN {
       push @INC, "/opt/Cabie/server/lib";
  }
          

  or specify the path: /usr/bin/perl -I/opt/Cabie/server/lib perl-script.pl

  MS/Windows Active Perl path:

  BEGIN {
       push @INC, "C:/Perl/site/lib";
  }
          

• When debugging the Perl CGI script web/cgi-bin/genweb, the text delivered to the web browser is in 512k blocks from the buffered I/O. The latest statements printed may never show up in the browser because they are stored in the I/O buffer until the buffer is full, at which time it is flushed. This makes it hard to diagnose problems with print statements. Flush the buffer after writing out debugging statements with the Perl directive "$|=1". This will flush the stdout buffer.

• YoLinux.com Tutorial: SysAdmin - Perl Administration

• Cabie home page
• YoLinux.com: Testing frameworks for C/C++ and Java: Unit tests to verify code correctness
• YoLinux.com: List of continuous and nightly build / test systems
  • Jenkins: CI server installation and configuration

Books:

Better Software Magazine Free subscription for software professionals who care about quality. Each issue brings you relevant, timely information to help you build better software. Continuing to deliver in-depth articles on testing, tools, defect tracking, metrics, and management.

Free Subscription