Configuring Sonarqube with Jenkins for Python Application

https://progressstory.com

Sonarqube is an automated static code analysis tool configured with Jenkins, it will report bugs, vulnerabilities, and tech debt of the project. Sonarqube with Jenkins can be extensively used to raise the bugs that would be missed by some manual efforts. Secondly, Linting and code coverage are also handled by Sonarqube, so we don’t have to have different tools for that. Thirdly, Sonarqube is open-source software with good support and in this doc, we will be going to cover the setup, sonar scanner for one sonarqube-flask project, integration with Jenkins, and setting up the quality gate.

Consider a situation where you are the engineer who is reviewing a pull request with more than 1000 lines of code change. However, you would have a checklist where you will check the

Manual PR Checklist

  • In addition, Does the PR follow the single concern principle?
  • Are the commit messages well-written?
  • Will the code work well with the existing code and not increase duplication?
  • Is the code well organised in terms of the placement of components?
  • Is the PR contains the test cases for the modified code?
  • New code keeping up with the idioms and code patterns of the language?
  • Does the code make use of the language features and standard libraries?
  • Does it comply with PEP-8?
  • Are all language and project conventions followed?
  • Are identifiers given meaningful and style guide-compliant names?
  • Is the code free of implementation bugs that could be exploited?
  • Have all the new dependencies been audited for vulnerabilities?

There’s a good probability that some points will be missed out when you are doing a manual check from the above checklist.

So here comes the Sonarqube, as it will take care of almost everything from the checklist, you just need to chill and watch Netflix.

So let’s dive into the agenda of the blog, what all things I will be covering in this blog.

Agenda

  1. Overview of Sonarqube and where to use it.
  2. Setup/Install Sonarqube.
  3. Configure python project in Sonarqube.
  4. Configure Jenkins with Sonarqube for automated testing.

Overview of Sonarqube

Jenkins pipeline with Sonarqube

Installation of Sonarqube with Jenkins

Prerequisites

  1. docker
  2. docker-compose

Step 1

Sonarqube with Docker

- sonarqube/
- data/ ## All the data of sonarqube is in this folder
- docker-compose.yml ## Sonarqube entrypoint
- extensions/ ## All the extensions are here
- logs/ ## Logs are stored here
- pg_data/ ## Sonarqube needs a db, we are using postgres for it and the data is here
- pg_db/ ## Postgres databases

To do so, use the below commands.

mkdir sonarqube
cd sonarqube
mkdir data extensions logs pg_data pg_db
touch docker-compose.yml

Step 2

anandtripathi5 gists

Step 3

docker stack deploy -c docker-compose.yml sonar

Most importantly, just crosscheck that the docker services are running fine or not by the below command

docker service ls

Step 4

Sonarqube Configuration

Step 5

username: admin

password: admin

  • Firstly, Click on Administrator’s My Account
  • Secondly, Click on the security
  • Thirdly, Change the password to something secure one

Step 6

Disable anonymous access

Administrator>Security>Force User Authentication

Step 7

Enable Email authentication and Setup Email Client

  1. Open My Account
  2. Open the Notifications tab and tick on the following rules

Setup Email Client

Open Administrator tab, then Configurations and then general

>Administrator>Configurations>General

Enter below values

  1. email prefix: [SONARQUBE]
  2. from address: <your email address>
  3. from name: SonarQube
  4. SMTP host: smtp.gmail.com
  5. SMPT password: <your password>
  6. SMTP port: 587
  7. username: <your username>
  8. server base URL: https://localhost:9000(your sonarqube domain)

Step 8

Configuration of Project

  • Give a name to the project
  • Generate a new token for analysis purpose, so that you will not use user credentials for analysing
  • Store the token somewhere safe as it will not be visible once the project is set up.
  • Choose the project main language, as in this case I have chosen “Other” cause the sonarqube-flask project’s main language is python. Then the OS will give you the sonar scanner command that we have to run to populate static code analysis.
  • Finally, the project is set up, we can see the project listing on the projects tab without any analysis

Step 9

Create Project

Sonar Scanner — Project Setup

For sonar-scanner, we will be using docker.

  1. Create a sonar-project.properties file on the root directory. This file will have all the flags or properties of the sonar scanner of that project
sonar.projectKey=sonarqube-flask
sonar.projectBaseDir=/sonarqube-flask
sonar.sources=.
sonar.host.url=http://localhost:9000
sonar.login=e21546a01f24a02b82428c8fe2d3622a5bd0fd8a
sonar.python.coverage.reportPaths=/sonarqube-flask/coverage.xml
sonar.python.xunit.reportPath=/sonarqube-flask/result.xml
sonar.coverage.dtdVerification=false
sonar.inclusions=app.py
sonar.coverage.exclusions=**/__init__.py

2. Install the below requirements file libraries

asgiref==3.3.4
attrs==21.2.0
beautifulsoup4==4.9.3
blinker==1.4
click==8.0.1
coverage==5.5
Flask==2.0.1
Flask-WebTest==0.0.9
iniconfig==1.1.1
itsdangerous==2.0.1
Jinja2==3.0.1
MarkupSafe==2.0.1
packaging==20.9
pluggy==0.13.1
py==1.10.0
pyparsing==2.4.7
pytest==6.2.4
pytest-cov==2.12.1
six==1.16.0
soupsieve==2.2.1
testapp==0.1.0
toml==0.10.2
waitress==2.0.0
WebOb==1.8.7
WebTest==2.0.35
Werkzeug==2.0.1

3. Write test cases for the project

from flask_webtest import TestApp
import pytest
from app import app as current_app
@pytest.fixture
def testapp(app):
return TestApp(app)
_app = None
@pytest.yield_fixture(scope='session')
def app():
global _app
if not _app:
_app = current_app
with _app.app_context():
yield _app
class TestFlask(object): @pytest.fixture(autouse=True)
def setup(self, testapp):
self.testapp = testapp
def test_comic(self):
res = self.comic()
assert res.status_code == 200
def comic(self):
return self.testapp.get(
"/comic",
expect_errors=True,
)

For complete code, check out the repository from here

4. Run the test case to generate the coverage report and result report using the below command

py.test --cov-report xml:coverage.xml --cov=. --junitxml=result.xml  test.py

5. This will generate coverage.xml file and result.xml file in the root directory of the repository

Step 10

Run Sonar Scanner on Project

For the complete blog and implementation, please check out Step by Step Configuration of Sonarqube with Jenkins for Python blog on my blogging website — Progress Story

Conclusion

I hope, it has helped you or it will help you. If you want to discuss anything or anything related to tech, you can contact me on the Contact Page of Progress Story. If you are interested in becoming a part of the Progress story please reach out to me or check out the Create Blog page. You can also reach out to me on Linkedin

See you next time! Peace Out ✌️

Creator of progressstory.com blogging website.