Getting Scalable with Continuous Integration and AWS
Let’s first understand what is CI and why do we need to use the stated development paradigm.
Continuous Integration
Grady Booch introduced the world to the art of continuous integration in 1991 which then evolved to become a standard in software engineering.
To put it simply, this is a paradigm under which the product or service is built in chunks or in a batch based manner which is continuously pushed into main code repository.
Although people are inclined towards automation testing scenarios but I have a different viewpoint with respect to the implementation of the infra.
- It provides significant benefits to both developers and customers.
On the developer end it comes handy in following scenarios:
- software quality, faster time-to-market, owner costs of development and decreased occurrences of integration issues throughout the whole software development process.
On the customer end it comes handy in following scenarios:
- Manage continuous checkpoint based deployment of customer custom or standard demands and helps in managing the demands by a ticketing system if required. - CI helps in supporting the Agile culture:
Agile - The most commonly followed Software Development methodology since the last decade is based on twelve principles. Three of these twelve principles emphasize fast development cycle and quick feedback loops which is fulfilled by CI paradigm:
a. Customer satisfaction by early and continuous delivery of valuable software.
b. Working software is delivered frequently (weeks rather than months).
c. Working software is the principal measure of progress.
Assume that you just got a new client who is demanding multiple things w.r.t the application he/she desires for his/her business and asked you to mention the cost for the same.
How will you do that?
Will you go onto a insightful journey to quote the price?
or
Will you be the one who will analyze the complete scenario to present a quote?
From a customers perspective:
Understanding the Story and quoting as per the sprints:
A user story is a formal or an informal communication between the manager and the client. It is basically to understand the technical needs & the business aspects of the customer w.r.t the application.
A product manager, post discussion formulates the complete scenario into a WBS to articulate the application development team in the form of sprints.
Once the sprints are done they are then pushed into the main code repository that holds the legacy code of the application.
During this complete delivery, suppose you engaged:
- A Developer
- A Designer
- A Tester &
- A Dev-ops engineer
These four people consumed a week to deliver the first sprint, and you have 20 more sprints to complete the application as desired by the client.
Now, it comes out to be a simple math problem:
- 1 week for one sprint, so, 20 weeks for 20 sprints.
Now, the PM can easily quote the client with a price according to the team size involved in developing the application and the time/sprint.
When you keep pushing the sprints live as per CI paradigm, you keep growing onto a working application and then simultaneously present the client for feedback on the same, which fulfills the other conditions of agile i.e. “A working software”.
From a product perspective:
Building a product?
Product development is complex journey, an entrepreneur strives to generalize the market and then prepares a one spot solution for the market problems.
The generalization process is long and encapsulates various operational dynamics. The continuous development paradigm helps the team to deliver a minimum viable product for the target audience via continuous integration. Nevertheless, it also helps the team to be focused on the product features which have significant impact rather than catering to the custom needs which is more of a service oriented thing.
One can easily have the sprints prepared in this scenario too and can keep on pushing onto the main code repository.
CI looks like the below self explanatory picture in real scenario.
Jumping into the Infrastructure and Implementation
A. Git Version Control
Configure your Github or Bitbucket account, I will go with bitbucket:
You may choose to create a team first, Bitbucket provides upto 5 members, the free subscription plan.
1. Create a Repository
2. Configure the Repository
- Update the details of Repository
- You can add a default reviewers and description as required so that the developer on commit see what you have to address.
- Add Jenkins Webhook (how to get webhook url is explained below) above and set branch merge permissions accordingly.
- Choose trigger type as “Approved” from list of triggers as shown below
B. Configure Jenkins Server
- Install Jenkins server on AWS
- Create and configure your Jobs on Jenkins
1. Create Freestyle Project
2. Add the repo link to source code management (clone — SSH)
3. In build trigger configure “Generic Webhook Trigger”, here you’ll get the webhook url to add into bitbucket webhook option. Moreover, here you have to add the token as added in buibucket repo above.
4. In “Post content Parameters”, add variable and the JSON path expression
5. You may configure the Build Environment as “Delete workspace after build”
6. In Build section, choose “Execute Shell” and path to your bash script
Example:
#/bin/bash
echo “$variable name”
path to script $variable name
7. In post build action section, choose “Slack” and configure to get updates on your slack channel
As soon as the developer pushes the code, the respective Jenkins server Job is triggered, which then execute the bash script to deploy the code live onto AWS
C. AWS Setup
For this, we need two different EC2 instances!
a. For prod-server
b. Jenkins-server
within Jenkins server you’ll need to write some bash script to deploy our code to prod-server.
We will use API’s provided by Bitbucket to achieve the task (within bash).
Once it is configured, our code is now live on AWS prod-server.
last but not the least, we will have to configure our php server on the prod-server which is to address the domains that you’re about to make live.
Restart the server via system ctrl command and you’re done.
You are now into an automated environment where the developer just have to focus on developing the application and pushing the code, rest is handled automatically.
You then of-course have to manage the domain with Route53 provided by AWS to route the traffic to the prod-server using ELB.
I hope the article is Helpful!
NOTES:
- I am not providing the very specific bash scripts to be used and how they’re written as it may vary from person to person.
- I haven’t showed you how to install and configure Jenkins, this is a pretty big process will increase the length of article, defeating its purpose.
- I am also not disclosing how to configure your php server on AWS which includes multiple scenarios like maintaining access log files, development error logs in production etc.
I hope this article is helpful in giving you the insights to the automated development system architecture.
Please share and clap if you like or comment if you have any doubts or issues!
AWS Notes:
Going Cloud Native: https://rb.gy/ag4edj
Going Secure on Cloud: https://tiny.cc/ts83nz
Vivek Gupta: https://www.linkedin.com/in/vivekg-/
Follow me on Quora: https://www.quora.com/profile/Vivek-Gupta-1493
Check out my legal space here: https://easylaw.quora.com
