Test Automation Infrastructure
Hello World,
Test Automation infrastructure,
When we move towards the automation in testing, then it's better to have a good automation infrastructure to support our automation testing CI/CD and reporting.
Now, you might ask questions like,
- Why we would be needing an infrastructure in automation?
- Is it necessary to have an infrastructure?- How it will be helpful?
- Will it be useful?
- Will it require maintenance?
and a lot of question like above. So let's discuss these questions.
Why we would be needing an infrastructure in automation?
Well before answering this question directly, we need to understand what does infrastructure mean? So Infrastructure is defined and structured approach to deal with any technological problem, hence infrastructure is a solution to solve the technological integration and its management.
Is it necessary to have an infrastructure?
So, although it is not necessary to have proper infrastructure, but consider this, If we would be using any technology to solve some problem then anyhow we are going to build an infrastructure whether it's proper or not. So it's better to have properly documented infrastructure.
How it will be helpful/useful?
Okay, so now you know what is infrastructure and whether it is necessary or not. So now if you have a proper infrastructure then, of course, it will be helpful and useful in many ways as
- Help understand the system
- Helps in debugging (if something goes wrong)
- Help to improve the current system easily
- Helps in collaboration
- etc
Will it require maintenance?
Well, to be honest, every technological infrastructure solution needs maintenance but the catch is How much maintenance? So here how much your infra system needs to be taken care of is totally dependent on the following things,
- How Complex it is?
- How frequently you need a new feature or upgrade?
But if you build an infra to serve very small limited things then it will not require proper maintenance.
So now, let's talk about the infrastructure architecture.
I have been working on many automation projects and we were developing/maintaining our infra with many tools like Jenkins(CI/CD), Maven(Dependency mgmt tool for java like NPM in js), flask(Micro-server in python), docker(Container service).
Here is a sample architecture that I have used in one of my automation testing projects.
- Java
I have used java to write our automation code
- Serenity-Screenplay
Serenity for the automation reporting, and screenplay for the reporting pattern to enhance the report
- Maven
Test Automation infrastructure,
When we move towards the automation in testing, then it's better to have a good automation infrastructure to support our automation testing CI/CD and reporting.
Now, you might ask questions like,
- Why we would be needing an infrastructure in automation?
- Is it necessary to have an infrastructure?- How it will be helpful?
- Will it be useful?
- Will it require maintenance?
and a lot of question like above. So let's discuss these questions.
Why we would be needing an infrastructure in automation?
Well before answering this question directly, we need to understand what does infrastructure mean? So Infrastructure is defined and structured approach to deal with any technological problem, hence infrastructure is a solution to solve the technological integration and its management.
Is it necessary to have an infrastructure?
So, although it is not necessary to have proper infrastructure, but consider this, If we would be using any technology to solve some problem then anyhow we are going to build an infrastructure whether it's proper or not. So it's better to have properly documented infrastructure.
How it will be helpful/useful?
Okay, so now you know what is infrastructure and whether it is necessary or not. So now if you have a proper infrastructure then, of course, it will be helpful and useful in many ways as
- Help understand the system
- Helps in debugging (if something goes wrong)
- Help to improve the current system easily
- Helps in collaboration
- etc
Will it require maintenance?
Well, to be honest, every technological infrastructure solution needs maintenance but the catch is How much maintenance? So here how much your infra system needs to be taken care of is totally dependent on the following things,
- How Complex it is?
- How frequently you need a new feature or upgrade?
But if you build an infra to serve very small limited things then it will not require proper maintenance.
There is one tool which is best known for CI/CD, and that is JENKINS.
So now, let's talk about the infrastructure architecture.
I have been working on many automation projects and we were developing/maintaining our infra with many tools like Jenkins(CI/CD), Maven(Dependency mgmt tool for java like NPM in js), flask(Micro-server in python), docker(Container service).
Here is a sample architecture that I have used in one of my automation testing projects.
- Java
I have used java to write our automation code
- Serenity-Screenplay
Serenity for the automation reporting, and screenplay for the reporting pattern to enhance the report
- Maven
used for dependency management in our java project- Jenkins
to automate our CI/CD for the automation code
- Tomcat
to serve the Jenkins war
- Flask
a Micro-server in python to serve our automation reports
- Jinja
a Templating Engine in python to serve the automation summary as a dashboard for one or several automation projects
- BeautifulSoup
A Module in python for Scrapping, I have used it to scrap the HTML page to extract the overall pass, fail, ignored, pending, broken, skipped, compromised.
- MongoDB
No RDBMS, to store the automation build results and location of the reports.
So overall I had two documents/structure in the MongoDB, those were:
{
timestamp:datetime,
jenkinsBuild:buildNumber,
pass: p,
failed: f,
ignored: i,
pending: e,
broken: b,
skipped: s,
compromised: c,
total: n,
reportlocation: loc
} => autoOverallReports document
Holds basic automation summary.
{
jenkinsBuild: buildNumber,
timestamp: dateAndTime,
failed: [
{ failedScenario: some-scenario, reason: exception},
...
]
} => failedReports document
Holds the failed scenario. (My plan to build this schema was to analyze the failure and the histogram for any specific failure)
So, I was pushing this info into the templates to present the reports to user/stakeholders.
Explanation of my sample infrastructure
-----------------------------------------
In my sample infrastructure, I divided the whole system into 4 parts
- Automation Code
- Jenkins server
with Build(code) and Post Build(crawling) instructions
- Python's micro-server (Flask+Jinja)
- MongoDB Server
So let me talk about these parts,
1- Automation Code
Our whole automation project was a Maven project and was written in Java using Serenity and Screenplay.
2- Jenkins server with Build and Post Build instructions
I used tomcat apache server to host the Jenkins war. although it is not necessary to have apache tomcat, its good to have it because of robustness and lightness.
Jenkins Build Instruction:
Since our automation project was a maven project so I created a Jenkins job for maven project(There is a maven plugin available for Jenkins) and put the maven goals like "clean verify [other-options-if-any]" (but if you are not using maven plugin for Jenkins then put "mvn clean verify [other-options-if-any]") and make sure that your system where Jenkins is running, Maven is installed and accessible by Jenkins(you can set the maven in the system environment)
Jenkins Post Build Instruction:
In this section, Jenkins was configured to do two things
- First was to copy the output reports to somewhere global place where all the reports along with the build number exist
- And second, was the execution of the crawler for that global directory.
So
the path for the global output folder was:
SomeDrive/AutomationReports
\---------ProjectName
| \----------SubProjectName
| \------------BuildNumber
| \----- output
Crawler: I have written the crawler in the python using the beautifulsoup module. Its main object is to extract the automation summary from the "index.html" available in the created report.
More technically the crawler is like: crawler(root_directory), here root_directory is the directory path for the output folder generated after the build.
3- Python's micro-server (Flask+Jinja)
I have written a micro-server using python's Flask and for the templating, I used the Jinja.
4- MongoDB Server
Mongo server to store all the data collected from the crawler.
The source code for this whole infrastructure will soon be available
@github/nityanarayan44
@github/ashusc
Thank You
Nitya Narayan Gautam
to automate our CI/CD for the automation code
- Tomcat
to serve the Jenkins war
- Flask
a Micro-server in python to serve our automation reports
- Jinja
a Templating Engine in python to serve the automation summary as a dashboard for one or several automation projects
- BeautifulSoup
A Module in python for Scrapping, I have used it to scrap the HTML page to extract the overall pass, fail, ignored, pending, broken, skipped, compromised.
- MongoDB
No RDBMS, to store the automation build results and location of the reports.
So overall I had two documents/structure in the MongoDB, those were:
{
timestamp:datetime,
jenkinsBuild:buildNumber,
pass: p,
failed: f,
ignored: i,
pending: e,
broken: b,
skipped: s,
compromised: c,
total: n,
reportlocation: loc
} => autoOverallReports document
Holds basic automation summary.
{
jenkinsBuild: buildNumber,
timestamp: dateAndTime,
failed: [
{ failedScenario: some-scenario, reason: exception},
...
]
} => failedReports document
Holds the failed scenario. (My plan to build this schema was to analyze the failure and the histogram for any specific failure)
So, I was pushing this info into the templates to present the reports to user/stakeholders.
Explanation of my sample infrastructure
-----------------------------------------
In my sample infrastructure, I divided the whole system into 4 parts
- Automation Code
- Jenkins server
with Build(code) and Post Build(crawling) instructions
- Python's micro-server (Flask+Jinja)
- MongoDB Server
So let me talk about these parts,
1- Automation Code
Our whole automation project was a Maven project and was written in Java using Serenity and Screenplay.
2- Jenkins server with Build and Post Build instructions
I used tomcat apache server to host the Jenkins war. although it is not necessary to have apache tomcat, its good to have it because of robustness and lightness.
Jenkins Build Instruction:
Since our automation project was a maven project so I created a Jenkins job for maven project(There is a maven plugin available for Jenkins) and put the maven goals like "clean verify [other-options-if-any]" (but if you are not using maven plugin for Jenkins then put "mvn clean verify [other-options-if-any]") and make sure that your system where Jenkins is running, Maven is installed and accessible by Jenkins(you can set the maven in the system environment)
Jenkins Post Build Instruction:
In this section, Jenkins was configured to do two things
- First was to copy the output reports to somewhere global place where all the reports along with the build number exist
- And second, was the execution of the crawler for that global directory.
So
the path for the global output folder was:
SomeDrive/AutomationReports
\---------ProjectName
| \----------SubProjectName
| \------------BuildNumber
| \----- output
Crawler: I have written the crawler in the python using the beautifulsoup module. Its main object is to extract the automation summary from the "index.html" available in the created report.
More technically the crawler is like: crawler(root_directory), here root_directory is the directory path for the output folder generated after the build.
3- Python's micro-server (Flask+Jinja)
I have written a micro-server using python's Flask and for the templating, I used the Jinja.
4- MongoDB Server
Mongo server to store all the data collected from the crawler.
The source code for this whole infrastructure will soon be available
@github/nityanarayan44
@github/ashusc
Thank You
Nitya Narayan Gautam
Comments
Post a Comment