Play with Jenkins Pipeline

Jenkins pipeline

      We know that CI/CD is one of the best practices for DevOps teams to implement. It is also an agile methodology best practice, as it enables software development teams to focus on meeting business requirements, code quality, and security because deployment steps are automated.

    Continuous Integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently. Because most modern applications require developing code in different platforms and tools, the team needs a mechanism to integrate and validate its changes. The technical goal of CI is to establish a consistent and automated way to build, package, and test applications.

     Continuous Delivery picks up where continuous integration ends. Continuous Delivery automates the delivery of applications to selected infrastructure environments. Most teams work with multiple environments other than the production, such as development and testing environments, and Continuous Delivery ensures there is an automated way to push code changes to them.

       CI/CD tools help to store the environment specific parameters that must be packaged with each delivery. CI/CD automation then performs any necessary service calls to web servers, databases, and other services that may need to be restarted or follow other procedures when applications are deployed. In this article I would like to share more on CI/CD Pipeline  using the tool Jenkins.

      Jenkins is a open source Continuous Integration/Continuous Delivery tool. It is used to integrate and automate your product development and testing processes. The purpose of using this tool was to build and test project continuously. In an ascent of agile, this could help developers to integrate the changes to the project as quickly as possible and obtain fresh builds ready for testing. Jenkins is very flexible and provides most of plugins support for integrations and also it is very easy to setup/configure. One of the important part of Jenkins tool is Jenkins Build Pipeline. This gives an overview about various jobs running on builds after commits made by developers. It tells about which tasks, Jenkins is currently executing.

      As name mentioned Pipeline, this will connect one section to another to achieve the goal. In Jenkins Build Pipeline the build can be seen as segmented into sections such as the compile the code, code review, unit test, packaging and deployment phases. These phases can be executed either in series or parallel. If one phase is successful, it automatically moves on to the next phase. Here, we will see how the Jenkins Build Pipeline happening for following:

  • Jenkins Build Pipeline within Development Project.
  • Jenkins Build Pipeline between Development Project and Test Automation Project.

Jenkins Build Pipeline within Development Project

     In this case, I have a development project (created in Maven) and committed the code into a version control system (Git or Bitbucket). I have committed the latest code into the version control system, next integrating the repository into Jenkins, defining tasks in Jenkins, creating pipeline for different phases includes compile code, code review, unit testing and packaging. These phases will execute one after another in Jenkins. Below are the detailed steps:

Step 1: Start Jenkins server and once the server ready, open the Jenkins Dashboard (make sure that all the suggested plugins installed).

Step 2: Create a Freestyle Project “1_compile” for compile the development project.

  • Map the development project repository at Source Code Management section.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as compile. Click Apply and Save.

Step 3: Create a Freestyle Project “2_code review” for code review.

  • Map the same development project repository at Source Code Management section.
  • At Build Triggers section, select Build after other projects are built option and enter project name 1_compile in Projects to watch field and select Trigger only if build is stable option. Project 2_code review will trigger after successful execution of  1_compile.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as -P metrics pmd:pmd. Here I used PMD plugin for static code analyze (it’s deprecated one you can use Warnings Next Generation plugin). Click Apply and Save.

Step 4: Create a Freestyle Project “3_unit test” for unit testing the development project.

  • Map the same development project repository at Source Code Management section.
  • At Build Triggers section, select Build after other projects are built option and enter project name 2_code review in Projects to watch field and select Trigger only if build is stable option. Project 3_unit test will trigger after successful execution of  2_code review.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as test. Click Apply and Save.

Step 5: Create a Freestyle Project “4_packaging” to generate build.

  • Map the same development project repository at Source Code Management section.
  • At Build Triggers section, select Build after other projects are built option and enter project name 3_unit test in Projects to watch field and select Trigger only if build is stable option. Project 4_packaging will trigger after successful execution of  3_unit test.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as package. Click Apply and Save.

jenkins_1

Step 6: Now all the jobs are ready to map to the pipeline operation.

  • Refer above figure, click on plus (+) icon. Enter View name and select Build Pipeline View option. Click OK. If you are not finding the Build Pipeline View option then  you need to install the Build Pipeline plugin in Jenkins.
  • Click Apply and OK. You will get below dashboard view to run the pipeline before the execution,

jenkins_2

  • Once complete the execution, you will get below view,

jenkins_3

Step 7: Now your pipeline is ready to trigger your various phases of development and deployment includes compile, code review, unit test and packaging.

Jenkins Build Pipeline between Development Project and Test Automation Project

     In this case, I have a development project (created in Maven) and also a test automation project (created in Maven). For both projects, the latest code committed into their respective repository in the version control system (Git or Bitbucket). Next integrating the repositories into Jenkins, defining tasks in Jenkins, creating pipeline in Jenkins. Below are the detailed steps to see how the test automation scripts will trigger once the build generated from the development project using pipeline:

Step 1: Start Jenkins server and once the server ready, open the Jenkins Dashboard (make sure that all the suggested plugins installed).

Step 2: Create a Freestyle Project “1_dev_project” to generate build for subsequent test automation script execution.

  • Map the development project repository at Source Code Management section.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as package. Click Apply and Save.

Step 3: Create a Freestyle Project “2_test_project” to execute the test automation script against new build generated from 1_dev_project.

  • Map the test automation project repository at Source Code Management section.
  • At Build Triggers section, select Build after other projects are built option and enter project name 1_dev_project in Projects to watch field and select Trigger only if build is stable option. Project 2_test_project will trigger after successful execution of  1_dev_project.
  • At Build section, select Invoke top-level Maven targets option and enter Goal as clean test. Click Apply and Save.

jenkins_4

Step 4: Now all the jobs are ready to map to the pipeline operation.

  • Refer above figure, click on plus (+) icon right to All. Enter View name and select Build Pipeline View option. Click OK. If you are not finding the Build Pipeline View option then  you need to install the Build Pipeline plugin in Jenkins.
  • Click Apply and OK. You will get below dashboard view to run the pipeline before the execution,

jenkins_5

  • Once complete the execution, you will get below view. My automation test project failed due to expected element not found in the application.

jenkins_6

Step 5: Now your pipeline is ready to trigger the test automation script once after the build successfully generated from the development project.

      I hope you really enjoyed with the Jenkins and Jenkins Pipeline within development project and also the pipeline between development and test automation projects. Try to implement the Jenkins pipeline from your end and enjoy your DevOps strategy.

make it perfect!

Automation execution in the DevOps world using GitLab CI/CD

CICD-Pipeline-Devops-Edureka

      GitLab is getting more popularity in recent times. One of the reasons for popularity because it combines the source code repository and CI server. Generally, GitLab composed of source code repository and Runners (open source project which is written in “Go” language). Runners are light-weight agents that trigger the jobs assigned to them with the help of dockers (it is a container). If you have a GitLab server then no need for a separate CI server.

      Here I would like to explain how the automation scripts trigger as a pipeline when the developer commits the developer code into the repository.

Capture      The whole magic happens with help of .gitlab-ci.yml file (YAML). It is a configuration file that you can create on your project’s root. This file automatically runs whenever you push a commit to the server. This triggers a notification to the runner and then it processes the series of tasks you specified in the YAML file. The following are some important configuration parameters in the .gitlab-ci.yml file.

  • image: use docker images. Telling to docker agent that image going to use for execution.
  • stages: define stages (jobs) in a pipeline.
  • stage: defines a job stage (default is test).
  • before_script: override a set of commands that are executed before the job.

     The following are the important eight steps to achieve automation execution in the DevOps world using GitLab CI/CD. Here I have done for Selenium automation with Gitlab CI/CD

Step 1: Create Automation Test Project using TestNG + Maven. Develop automation test cases.

Step 2: Create .gitlab-ci.yml file in the root directory of your automation project and configure before_script, stages and test stage details in .gitlab-ci.yml. Below is the configuration details of .gitlab-ci.yml,

image: maven:3.6.3-jdk-11
  
before_script:
    ################################################################################################
    # Install Xvfb (X virtual framebuffer) packages.                                               #
    # Xvfb (X virtual framebuffer) is an in-memory display server for a UNIX-like operating system # 
    # It implements the X11 display server protocol without any display.                           #
    ################################################################################################
    - apt-get update
    - apt-get -y install gnupg
    - apt-get install -y unzip xvfb libxi6 libgconf-2-4
    
    ##########################
    # Install Java on system #
    ##########################
    - apt-get install -y default-jdk
    
    ##################################################
    # Install Latest Google chrome package on system #
    ##################################################
    - curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add
    - echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list
    - apt-get -y update
    - apt-get -y install google-chrome-stable
    
stages:
  - test
  
test:
  stage: test
  script:
    - chmod a+x /builds/gitlab_username/sampleautomationproject/chromedriver #If you not given the permission then you may face "java.lang.IllegalStateException: The driver is not executable" error.
    - mvn clean #Compile your scripts
    - mvn test #Run your scripts

Step 3: Login to GitLab Dashboard.

Step 4: Create a New Project in GitLab to keep the automation project > Create a Repository.

Step 5: Commit and push your automation test project to the GitLab automation repository.

Step 6: Just refresh the GitLab and see the Pipeline and Jobs section of CI/CD. You can see your automation scripts are executing.

1

Step 7: Create .gitlab-ci.yml file in the root directory of the development project and configure triggerjob details in .gitlab-ci.yml. Below is the configuration details of .gitlab-ci.yml,

image: maven:latest

stages:
  - build
  - test
  - automationstage

build:
  stage: build
  script:
    - mvn clean

test:
  stage: test
  script:
    - mvn test
    
triggerjob:
    stage: automationstage
    trigger:
        project: gitlab_username/sampleautomationproject
        branch: master

Step 8: Make some changes/add new features in development project and deploy it. Observe that development project pipeline starts, once it success it will trigger the automation stage for automation execution.

2.PNG

     Please try to utilize the above steps to make your automation life easier (with GitLab). In this link https://gitlab.com/sanojs you can find the sample POC which I have implemented for this GitLab CI/CD.

make it perfect!