Introduction to Jenkins:
Jenkins can monitor your CM system to detect a check-in.
Upon recognition of this change, Jenkins will update a local working directory of code and perform a series of build steps (e.g. "ant clean", "ant compile" or "make clean" and "make").
Unit tests and tests can be performed after each build. Jenkins can support multiple CM systems (SVN, CVS, GIT, Clearcase, ...) and build frameworks. It can work as a stand-alone server or collectively in a master with multiple reporting nodes configuration to support builds on a variety of OS's and configurations.
Jenkins supports Ant, Maven or a sequence of shell script based builds, code analysis and unit or integration tests.
Jenkins has many plug-ins available to extend its feature set and to integrate with other Software tools.
This tutorial will cover the installation, configuration and use of Jenkins
integrated with an existing Apache server used as a CM web portal.
Other related YoLinux.com Tutorials:
Installation and Configuration:
Download:
Two example installations:
- Basic WAR file and start script
- Linux RPM Package (Red Hat/Fedora/CentOS)
1) Basic Jenkins WAR File Configuration:
This example shows the basic Unix/Linux start script used to launch Jenkins.
All configuration options in this example are set with environment variables and command line arguments:
07 | export JENKINS_HOME=/var/jenkins |
08 | export JENKINS_USER="jenkins" |
09 | export JAVA_HOME=/usr/java/latest |
10 | export CLASSPATH=$JAVA_HOME/lib/tools.jar:./ |
11 | export ANT_HOME=/opt/apache-ant-1.8.1 |
12 | export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH |
13 | export JENKINS_PORT="8080" |
14 | export JENKINS_AJP_PORT="8009" |
15 | export JENKINS_DEBUG_LEVEL="5" |
16 | export JENKINS_ENABLE_ACCESS_LOG="yes" |
17 | export JENKINS_HANDLER_MAX="10" |
18 | export JENKINS_HANDLER_IDLE="20" |
20 | java -Djava.awt.headless=true -DJENKINS_HOME=$JENKINS_HOME -jar $JENKINS_HOME/jenkins.war --accessLoggerClassName=winstone.accesslog.SimpleAccessLogger --simpleAccessLogger.format=combined --simpleAccessLogger.file=$JENKINS_HOME/logs/access_log --prefix=/jenkins --daemon |
Note
--prefix=/jenkins is set for the Apache proxy configuration below
or access with the "/jenkins/" path appended to the URL
http://localhost:8080/jenkins/
2) Red Hat Enterprise / CentOS RPM Configuration:
Jenkins is available as source, pre-built binaries and as an installation package.
Jenkins RPM Installation:
rpm -ivh jenkins-1.487-1.1.noarch.rpm
/etc/init.d/jenkins
/etc/logrotate.d/jenkins
/etc/sysconfig/jenkins
/etc/yum.repos.d/jenkins.repo
/usr/lib/jenkins
/usr/lib/jenkins/jenkins.war
/usr/sbin/rcjenkins
/var/cache/jenkins
/var/lib/jenkins
/var/log/jenkins
Edit the init script: /etc/init.d/jenkins
Add the Java JRE path to line below (line 68):
for candidate in /usr/java/latest/bin/java /usr/lib/jvm/java-1.6.0/bin/java ...
See the
YoLinux Java installation tutorial for more information.
Edit settings in /etc/sysconfig/jenkins
Define variables used by Jenkins init script.
JENKINS_HOME="/var/lib/Jenkins"
JENKINS_JAVA_CMD="/usr/java/latest/bin/java"
JENKINS_USER="jenkins"
JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true"
JENKINS_PORT="8080"
JENKINS_AJP_PORT="8009"
JENKINS_DEBUG_LEVEL="5"
JENKINS_ENABLE_ACCESS_LOG="yes"
JENKINS_HANDLER_MAX="100"
JENKINS_HANDLER_IDLE="20"
JENKINS_ARGS=""
...
JENKINS_HOME can be specified anywhere you like on the file system (eg:
/opt/BUILDS/Jenkins)
An RPM installation will generate the user "jenkins".
A binary installation of Jenkins requires that you create the user:
useradd -d /var/lib/jenkins -c 'Jenkins user' jenkins.
chown -R jenkins.jenkins /var/lib/jenkins
Start Jenkins: /etc/init.d/jenkins start (or service jenkins start)
Note default service configuration: chkconfig --list |grep jenkins
jenkins 0:off 1:off 2:off 3:on 4:off 5:on 6:off
Jenkins can now be access by a web browser: http://jenkins.megacorp.com:8080/
Using Apache proxy to access Jenkins:
If you don't want users to access the Jenkins server directly but would rather that they access via an Apache web server at port 80 and use Apache authentication (e.g. LDAP, NIS, etc), route web traffic through the Apache server and use it as a proxy to access the Jenkins server.
- Note that the Jenkins port is specified in the configuration file /etc/sysconfig/jenkins
JENKINS_PORT="8080"
...
Defines default Jenkins port.
- Proxy a URL path. Change the prefix Jenkins uses so that the default http://servername:8080/ can be accessed as http://servername:8080/jenkins
(Choose one method only, not both)
- Set a /etc/sysconfig/jenkins variable (Red Hat/CentOS/AWS based systems). Default was "".
...
JENKINS_ARGS="--prefix=/jenkins"
Restart Jenkins: service jenkins restart
- Change the init script:
This is achieved by adding the command line argument --prefix=/jenkins
to the init script PARMS variable and restart Jenkins.
Edit Jenkins init script: /etc/init.d/jenkins
From:
PARAMS="--logfile=/var/log/jenkins/jenkins.log --daemon"
...
To:
PARAMS="--logfile=/var/log/jenkins/jenkins.log --daemon --prefix=/jenkins"
...
Restart Jenkins: /etc/init.d/jenkins restart
- Configure Apache as a proxy to the Jenkins server service:
Allow mod_proxy
Restart web server: service httpd restart
or: apachectl restart
- Block external access to the Jenkins server on port 8080 (only localhost allowed) and only allow network proxy access through Apache on port 80 with the following firewall rules:
Add to end of /etc/rc.local so it executes upon system boot.
# Allow loopback access.
# Rule for your computer to be able to access itself via the loopback
# This rule must come before the rules denying port # access!!
iptables -A INPUT -i lo -p all -j ACCEPT
iptables -A OUTPUT -o lo -p all -j ACCEPT
iptables -A INPUT -p tcp -s 0/0 -d 0/0 --dport 8080 -j DROP
SELinux: setsebool -P httpd_can_network_connect true
- Jenkins can now be accessed with a web browser: http://jenkins.yourdomain.com/jenkins/
[Potential Pitfall]: The Apache httpd statement "ProxyRequests On" can cause all sorts of trouble on the public internet (don't use it) as it will allow your server to become an open proxy for all web traffic. If using this statement you must also restrict use as a proxy:
Limit proxy access to a specified IP address block:
4 | Allow from XXX.XXX.XXX.XXX/14 |
Ant Build script: (example)
Jenkins can execute scripts, use Apache Ant or Maven build tools.
In this example I am using Ant to launch a make file.
Typical Java programmers will not need this example and will typically have projects which build with Ant or Maven.
Projects based on Makefile builds (typical for C, C++, FORTRAN, etc) may want to use Apache Ant to call makefile targets as shown in this example.
Installation: (choose one option)
Create Ant build script to launch make file targets:
File: build.xml
01 | <project name="projectX" default="jenkins" basedir="."> |
03 | Jenkins Ant file for projectX |
06 | <property name="app" location="projectx"/> |
07 | <property name="make.cmd" value="/usr/bin/make"/> |
08 | <property name="build.native" value="./"/> |
15 | <target name="compile" |
16 | description="compile the source" > |
17 | <exec dir="${build.native}" executable="${make.cmd}" failonerror="true"> |
22 | description="clean up" > |
23 | <exec dir="${build.native}" executable="${make.cmd}" failonerror="true"> |
Links:
MS/Visual C++:
DOS commands to clean and build:
- devenv.exe projectX.sln /clean debug
- devenv.exe projectX.sln /build debug
Example CMD DOS command scripts to clean and build:
Environment variables for use with the MS/Visual Studio compiler can be found in:
- MS/Visual Studio .NET 2003: C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\vsvars32.bat
- MS/Visual Studio 8.0: C:\Program Files\Microsoft Visual Studio 8\VC\vcvarsall.bat
Also see the
YoLinux DOS / bash command comparison
Wind River Workbench / VxWorks:
Example of a build for embedded RTOS development using Wind River Workbench / VxWorks:
File: build.xml
01 | <project name="Embedded VxWorks Build for ProjectE" default="jenkins" basedir="."> |
03 | Jenkins Ant file for Embedded VxWorks Build for ProjectE |
06 | <property name="make.cmd" value="/opt/WindRiver/utilities-1.0/x86-linux2/bin/make"/> |
07 | <property name="genmake.cmd" value="/opt/WindRiver/workbench-3.1/x86-linux2/bin/wrws_update.sh"/> |
08 | <property name="build.native" value="./"/> |
09 | <property environment="env"/> |
16 | <target name="genmake" description="Generate Makefile from .project file" > |
17 | <exec dir="${build.native}" executable="${genmake.cmd}" failonerror="true"> |
18 | <arg line="-data /home/builds/jobs/ProjectE/workspace -l ProjectE -m"/> |
19 | <env key="WIND_HOME" value="/opt/WindRiver"/> |
20 | <env key="WIND_BASE" value="/opt/WindRiver/vxworks-6.7"/> |
21 | <env key="PATH" value="/opt/WindRiver/diab/5.7.0.0/LINUX386/bin:${env.PATH}"/> |
22 | <env key="LD_LIBRARY_PATH" value="/opt/WindRiver/vxworks-6.7/host/x86-linux2/lib:${env.LD_LIBRARY_PATH}"/> |
23 | <env key="WIND_PLATFORM" value="vxworks-6.7"/> |
24 | <env key="LM_LICENSE_FILE" value="/opt/WindRiver/license/zwrsLicense.lic"/> |
28 | <target name="compile" depends="genmake" description="compile the source" > |
29 | <exec dir="${build.native}" executable="${make.cmd}" failonerror="true"> |
30 | <arg line="-f PPC604diab/Makefile clean main_all"/> |
31 | <env key="WIND_HOME" value="/opt/WindRiver"/> |
32 | <env key="WIND_BASE" value="/opt/WindRiver/vxworks-6.7"/> |
33 | <env key="PATH" value="/opt/WindRiver/diab/5.7.0.0/LINUX386/bin:${env.PATH}"/> |
34 | <env key="LD_LIBRARY_PATH" value="/opt/WindRiver/vxworks-6.7/host/x86-linux2/lib:${env.LD_LIBRARY_PATH}"/> |
35 | <env key="WIND_PLATFORM" value="vxworks-6.7"/> |
36 | <env key="LM_LICENSE_FILE" value="/opt/WindRiver/license/zwrsLicense.lic"/> |
Note that much of this configuration is devoted to setting the environment variables PATH, LD_LIBRARY_PATH and VxWorks environment variables. The VxWork paths should precede all others so that VxWorks provided tools such as tclsh, used for auto code generation, will be used instead of the Linux tools as they may operate differently due to version and release variations.
This ANT build script first generates the Makefile from the Workbench GUI IDE project file.
It then compiles the code using the Makefile.
This configuration is of Apache, Jenkins and the Wind River workbench compiler all running on Linux. (Note that the Windriver Workbench/VxWorks installation CDs contain both the Microsoft Windows and Linux installations on the same CD)
Cross Platform Considerations:
Use the "os" exec attribute to turn on an exec for only that platform.
Thus all paths and platform dependencies within the exec statement will only get executed on the proper platform and ignored on all others.
The OS name can be determined by using and looking for os.name in the output it generates.
Include all platform execs within the same target.
Linux:
File snippet:
build.xml
1 | <target name="run" description="Run application" depends="compile"> |
2 | <exec dir="${build.native}" executable="exampleapp" os="Linux" failonerror="true"> |
MS/Windows:
File snippet:
build.xml
1 | <target name="run" description="Run application" depends="compile"> |
3 | <exec dir="${build.native}" executable="cmd" os="Windows 7" failonerror="true"> |
5 | <arg value="exampleapp.exe"/> |
[Potential Pitfall]: Trying to execute a command directly rather than within a shell on MS/Windows may result in the following error:
Execute failed: java.io.IOException: Cannot run program "..." (in directory "..."):
CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at java.lang.Runtime.exec(Runtime.java:617)
at org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:827)
at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:445)
at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:459)
...
...
Fix: run the "cmd" shell with the executable as an argument to the shell and do not run the executable directly. This is a MS/Windows specific problem.
Jenkins Build Configuration:
From the Jenkins web page: http://jenkins.megacorp.com/jenkins/
- Select "Manage Jenkins" (upper left)
(URL: http://jenkins.megacorp.com/jenkins/manage)
- Select "Configure System":
- Select "create new jobs"
- Enter a job name and type ("Build a free-style software project" + OK
- Source Code Management:
- Subversion:
Select CM (i.e. Subversion). If Subversion then also enter the repository
URL: http//subversion.megacorp.com/svn/ProjectX/trunk/JavaApp
You will be prompted for authentication login/password if required.
Check "Use update"
- GIT:
- Build Triggers:
Select "Poll SCM" and define a schedule:
Poll CM system (e.g. Subversion) every 6 minutes (10 times an hour) for updates.
*/6 * * * *
This will detect changes checked into the CM system to trigger a new build.
Chose a time which is greater than the time to update the local working Subversion repository, compile and run any regression tests.
Configure to handle even the longest of JUnit or CppUnit tests.
Run build at 2:00am every Tuesday-Saturday
0 2 * * 2,3,4,5,6
is the same as:
0 2 * * 2-6
with a random number of minutes chosen:
H 2 * * 2,3,4,5,6
is the same as:
H 2 * * 2-6
This will run like a nightly build cron job.
Five Columns: MINUTE HOUR DOM MONTH DOW
- MINUTE: Minutes within the hour (0–59)
- HOUR: The hour of the day (0–23)
- DOM: The day of the month (1–31)
- MONTH: The month (1–12)
- DOW: The day of the week (0–7) where 0 and 7 are Sunday.
Operators:
- * specifies all valid values
- M-N specifies a range of values
- M-N/X or */X steps by intervals of X through the specified range or whole valid range
- A,B,...,Z enumerates multiple values
- H choose a random value. Good for avoiding conflicts.
If "H 2 * * 2-6" is specified for all projects, the "H" will avoid the launching of multiple jobs all at once.
- Build:
Select from the pull-down list "Invoke Ant" and enter list of targets eg: clean and all
- Unit Tests:
- Select "E-mail Notification" and enter email address.
(Make sure you are running an SMTP daemon: service sendmail start)
- Select "Save"
Test the system by selecting "Build Now" (upper left hand side of page.)
Jenkins Artifacts:
Define list of build artifacts:
- From the Jenkins home page select the Job.
- From the upper left hand side of the page, select the "Configure" link.
- In the section "Post-build Actions" select the option "Archive the artifacts".
- Files to archive: name-your-executable-or-lib-here
(Comma separated list. Can use **/file.ext notation)
- After the next build, the project page will show the package icon and list the "Last Successful Artifacts" you specified.
Jenkins Plug-ins:
Jenkins plug-ins are a way of extending and enhancing the capability of Jenkins.
Many plug-ins provide an interface with other systems to provide improved integration.
One of the wonderful features of Jenkins is the seamless integration of the plug-in installation: Jenkins (start page) + "Manage Jenkins" + "Manage Plugins".
Now select the "Available" tab.
From here you can select the plugin for Jenkins to download and install.
See the current list of
Jenkins plug-ins
Jenkins can download and manage plugins from the web console but the "hpi" plugin files can also be downloaded (Jenkins hpi plugins) independently and placed in the plugins directory: $JENKINS_HOME/plugins or /var/lib/jenkins/plugins/ (Red Hat: defines the environment variable JENKINS_HOME in /etc/sysconfig/jenkins)
Note that this directory gets generated by Jenkins after first use when it starts up.
I do recommend some as part of any Jenkins installation, the Source Lines Of Code (SLOC) plugin "SLOCCount", "Sidebar-Link", Disk Usage and "Green Balls" plugins.
Plugins:
[Potential Pitfall]: Jenkins version 1.4xx requires Java 1.5 or newer. If running on a Solaris 8 platform you will be limited to Java 1.5 as your latest version. Note that some plugins are compiled with later versions of Java and may cause unexpected errors.
A typical error is to make a call to an API method available with one version of Java (eg 1.6) but not the other (1.5).
Sidebar-Link Plugin:
Allows one to generate a list of extra links for Jenkins main page and view tabs.
I use this to generate a link back to our CM server home page.
Jenkins Sidebar-Link Plugin
SLOCCount Plugin:
This plugin will run the sloccount program against your code and report results through Jenkins.
After the build following this configuration, there will now be a new link on the upper left corner of the page labeled "SLOCCount".
Select to show the SLOC report.
Jenkins SLOCCount Plugin
Don't Repeat Yourself (DRY) Plugin:
This is a plugin which reports the output from the PMD Cut and Paste Detector (CPD) which finds code duplication (comments and copyright headers not included).
While the entire PMD suite of static code analysis applies only to Java, the CPD portion applies to Java, C, C++, C#, PHP, Ruby, Fortran and JavaScript.
Jenkins DRY plugin
Jenkins Security:
Security can be handled by Apache authentication and authorization if proxied through Apache as detailed above or by Jenkins itself.
Turning on Jenkins security: Jenkins (home page/dashboard) +"Manage Jenkins" + Configure system"
Options:
- Jenkins's own user database
- LDAP
- Delegate to servlet container
- Unix user/group database (PAM)
Authorization can also be handled by Jenkins options from nothing to setting fine grained Global or Project based authorization control.
Definitions:
- Authentication: user proves that they are indeed who they say they are.
Login and password required.
- Authorization: user is allowed (authorized) to perform certain functions and have specific access.
eg. admin, read only, etc.
Notes:
Jenkins Environment Variables:
Jenkins provides a number of environment variables which can be used by your build tools (eg Ant or Maven)
| Environment Variable | Description |
|---|
| BUILD_NUMBER | The current build number, such as "153" |
| BUILD_ID | The current build id, such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss, defunct since version 1.597) |
| BUILD_URL | The URL where the results of this build can be found (e.g. http://buildserver/jenkins/job/MyJobName/666/) |
| NODE_NAME | The name of the node the current build is running on. Equals 'master' for master node. |
| JOB_NAME | Name of the project of this build. This is the name you gave your job when you first set it up. It's the third column of the Jenkins Dashboard main page. |
| BUILD_TAG | String of jenkins-${JOB_NAME}-${BUILD_NUMBER}. Convenient to put into a resource file, a jar file, etc for easier identification. |
| GIT_COMMIT | For Git-based projects, this variable contains the Git hash of the commit checked out for the build (like ce9a3c1404e8c91be604088670e93434c4253f03) (all the GIT_* variables require git plugin) |
| GIT_URL | For Git-based projects, this variable contains the Git url (like git@github.com:user/repo.git or [https://github.com/user/repo.git]) |
| GIT_BRANCH | For Git-based projects, this variable contains the Git branch that was checked out for the build (normally origin/master) |
The environment variables can then be referenced in build tools like Apache Ant or Maven.
Maven example:
03 | <id>jenkins-build</id> |
06 | <name>env.BUILD_NUMBER</name> |
10 | <release.jenkins.version>${env.BUILD_NUMBER}</release.jenkins.version> |
The default for when the maven build is not using Jenkins, the build number is set to "local":
<properties>
<release.jenkins.version>local</release.jenkins.version>
The artifact names can be appended with -${env.BUILD_NUMBER}
Version numbers can also be set to build numbers as well.
Also see the Jenkins website list of environment variables.
Links:
Books:
