Introduction
After more than eight years of experience with SQLI, I have had the opportunity to participate in many projects and contribute to more than half of them. Over time, I have memorized the procedures, actions and changes required for each new project, as well as the steps required to install on our internal servers. Not to mention the configuration of the pipelines, which is systematically copied from one project to the next, without ever being revised, due to lack of time. Eight years is a long time.
One day, I reached a saturation point. It was just unbelievable that our company, which specializes in developing automation solutions for our customers, did not apply it to itself! I am well aware that cobblers are often the worst shod, but at a certain point, there must be no soles left at all! We used to waste days manually setting up each project. Days that could be more profitable if they were devoted to our real added value: development.
Repeatedly creating projects using the same templates and procedures has some advantages in introducing newcomers to the servers, security, procedures and tools we use. However, at a certain point it becomes necessary to move forward.
So I set out to solve this problem at its source by extending this approach to everything that didn’t need to be done manually. I developed an internal platform that automates everything that can be automated.
The underlying idea is that each project has a series of well-defined steps:
- Initialization
- Server configuration
- Setting up the pipelines
- Development
- Tests
- Delivery
- Monitoring
- Closing
Of these eight points, the only step where human intervention is essential is the development phase. All the other steps are of little interest to be carried out manually by one person and can even lead to risks of errors. It is therefore highly recommended to automate them.
I will group these steps into 3 main categories:
The pre-project
The first category includes preliminary steps, such as initializing the code and its Git repository, installing and configuring the internal server dedicated to the project, and setting up pipelines to automate tests and deployments on the server. These steps are systematic from one project to another and do not add any value when done manually. A machine, on the other hand, can perform them quickly and accurately, without omitting important tasks or details.
The internal tool developed allows you to select the project technology to be initialized (Symfony, Drupal, etc.) via a user-friendly interface. It then installs and configures the required packages in the appropriate versions on a blank server (PHP, Apache, etc.). A first commit is automatically created in the project repository, with the chosen technology already initialized and the necessary modifications to our existing infrastructure. Pre-configured pipelines are also integrated into the project, covering most needs. Customizations can be made if necessary.
This project start-up phase, which used to take several days, can now be reduced to a few hours, allowing us to focus on project-specific adjustments, if needed.
The project
The second phase is both the one where human intervention is essential and the one where developers excel in their role. This is the active part of the project, where the developers progress through the different sprints and make regular commits. It is during this phase that the project is built from scratch.
Development cannot be automated at this stage, as this is where our exceptional developers can express their full talent.
Depending on the level of rigor required for the project, unit tests, functional tests, end-to-end tests, visual tests, code quality audits, or a combination of several methods can be implemented.
The internal tool developed does not make significant contributions during this phase. It simply connects to the results of the automatically executed tests via the pipelines and generates reports. These reports can be useful for project managers or customers.
The post-project
Once the project is completed and delivered, the maintenance phase begins. It is then necessary to constantly monitor the availability of the site, the dependencies, the vulnerabilities and to ensure that the new developments requested later do not cause regressions.
This is where the tool comes into its own, as it allows continuous monitoring of dependencies (vendors, node_modules, pom.xml, etc.) in order to detect any potential vulnerability and issue alerts accordingly.
There are still two features to be added to perfect this tool:
- Monitoring of availability and loading time once the site is online.
- Automatic archiving of projects once they are closed.
Once these two features are integrated, the tool will be perfect to accompany projects throughout their life cycle, from initialization to closure, through development, maintenance and monitoring. It will save time at every stage, allowing employees to focus on real value-added aspects, while the machine takes care of all the automatable tasks. In addition, it will make it much easier to take over projects developed by competing agencies, allowing quality and security audits to be performed more quickly and efficiently on the retrieved code.