CI/CD: How Not to Get Devoured by Competitors

Once upon a time, the world of development was very slow. It's hard to say whether this was due to a generally more relaxed pace of life and lower competition, or something else entirely. But one thing is certain: 15 years ago, sluggish and cumbersome banks didn't roll out new products every few weeks, suddenly offering convenient insurance applications, stock trading, and gas station payments from the comfort of your car, all in a few clicks.

How were programs developed back then? The approach was called Waterfall. Smart people in formal suits gathered strange folks from the IT department and tried to explain what the business wanted. They sat for weeks in conference rooms, drawing hundreds of diagrams, plans, and architectural sketches. They were designed monumentally and for the ages. After several months, the development department held The Plan. This document, soaked in tears and anxiolytics, contained a complete approved instruction of what the Business wanted. And then developers sat down and began writing code for several months to implement business tasks.

Six months of intense development passed, and the Business patiently waited for their ordered product, beautiful and polished. Finally, with a ceremonial drum roll, the new branch was ceremoniously merged into the main code, deployed across all nodes, and everything crashed with a bang. Thousands of bugs, compatibility issues, and other problems that were hard to foresee emerged. Since the new product was integrated in one big piece, quickly identifying the cause was very difficult.

Finally, all bugs were fixed, and the Business received a beautiful Product, developed strictly according to a massive technical specification. Only it turned out that by this time, competitors had already managed to release a rough and ugly analog, attract users, earn money, hire another team, and quickly improve the application, propping it up with crutches as needed. The beautiful and well-thought-out Product was no longer needed by anyone.

Nowadays, no one wants to spend hours filling out paperwork for something other service providers can offer in a few clicks. Businesses are forced to adapt to these demands to stay afloat. Deliver a set of groceries in three clicks from an app within 20 minutes? Done. Somewhere in the backend, there are complex integrations of the central warehouse, intermediate delivery points, geolocation, real-time courier tracking, and much more, so you can get raw eggs for an omelette and a croissant while your coffee brews. Those who couldn't occupy a niche and provide such services were destroyed by competitors and pushed out of the market.

So, businesses need to change quickly. Did a competitor add a new feature? We need to have it too, and it should have been here yesterday. That's why Agile methodology has become widespread. The key task is to deliver value to the business and customers as quickly as possible. There's no time to plan a huge project in advance; it might be changed in dozens of places before reaching its final form. Instead, we quickly implement many small atomic changes that rapidly make our users happy.

The ideal situation is when a task comes in from the business in the morning, developers refine the code within a few hours, push it to the repository, and within minutes, the end user has the new features working. Achieving this is only possible if you have a great team of experts working simultaneously with the code and infrastructure using DevOps practices.

CI/CD is one of the basic concepts related to Agile and DevOps methodology. It consists of two fundamental elements - Continuous Integration (CI) and Continuous Delivery (CD). Let's start with continuous integration.

People are forgetful, prone to fatigue, and generally perform boring routine tasks poorly, especially those requiring long-term focus on something very monotonous. Solving a complex creative task after 15 cups of coffee? No problem. Checking a hundred parameters in 37 manual tests for the 74th time during another deployment? People can't handle that even once.

A skilled DevOps expert, can solve this problem. To begin with, the entire concept of working with code is restructured. The main task is to ensure that developers merge code as often as possible in small atomic portions. DevOps configures automated pipelines using tools like Gitlab or Jenkins, so that every tiny piece of code won't reach users unless it meets all requirements.

A typical scenario without CI usually looks like this:

• A developer gets a task to fix a small bug.
• The developer pushes their patch to the repository and goes for another coffee.
• Two months later, it turns out this patch broke critical functionality elsewhere, which is used every few months.

With CI, the same scenario looks different:

• The developer pushes their patch to the repository.
• An automated pipeline starts.
• Linters check the code for syntax correctness, e.g., Pylint or RuboCop.
• Code style is checked to ensure all team members write in a consistent style, simplifying collaboration. Believe me, inconsistency is very irritating to developers.
• Unit tests are executed. Unit tests check individual parts of your code.
• Static code analysis is performed. Your code is checked for vulnerabilities and issues, even those that appeared in open databases a few hours ago. Tools like SonarQube might be used here.
• Dependency assembly and verification. All necessary elements are compiled, packaged, and sent to the company's artifact repository.
• Integration testing. The ready elements are deployed on test infrastructure, simulating typical behavior to evaluate the interaction of all parts of the architecture.

This extensive list above is impossible to perform manually after each minor code change. If you have a well-configured CI, you can be sure that your patch won't break something that already worked. You won't need to spend time on manual checks. As a result, code that has passed the CI pipeline automatically meets your quality standards. Any changes that don't pass the tests won't reach the production branch.

At WiseOps, for instance, we adhere to quite strict internal code standards, ensuring the uniformity and reliability of the final product.

Great, we have complete automation at the code checking stage, but now we need to somehow deliver it to the appropriate environments. There are many ways to build such pipelines. We configure them considering the client's architecture specifics, like Blue-Green Deployment, and other nuances. In short, we ensure everything updates seamlessly, ideally without downtime or impact on users.

Often, the pipeline looks something like this:

• Code is built into packages, images are created.
• Resources are created or modified using Terraform.
• Server and resource configurations are changed using Ansible. At these stages, we leverage the Infrastructure-as-Code approach.
• Code is copied to the target environment.
• Environment variables are configured.
• All application components are deployed - APIs, web services, and databases.
• Various post-installation actions are performed: correct restarting of daemons, web services.
• Testing the correctness of the entire infrastructure and applications.
• Load switching.

1. If a company wants to survive in fierce competition, it needs to respond quickly to changes and deliver value to its customers.
2. People are weak. Don't give them tasks that can be automated and will be performed with equal efficiency day and night.
3. You can't manually track a critical vulnerability discovered in one of the hundred libraries of your project. Automated tools will do this for you with every new change.
4. Do you want all your code to look clean and be understandable to every team member? Linters and style checks will prevent most ugly fragments that are hard to work with.
5. Deliver changes every day, even if they are small. Strive to ensure that the user gets a solution to their problem or a new feature as quickly as possible.
6. Make sure your infrastructure is described by code (IaC). This will allow you to seamlessly project all changes onto real servers and virtual machines.

And if you need a team of experts to beautifully set up your pipelines and synchronize them with your business needs, we at WiseOps will be happy to help.