Once upon a time was IT monitoring. A nice Swiss army knife which had the solution to everyone’s monitoring problems. But that was before… Before the Cloud, SaaS, serverless, agility, automation, machine learning, AI… In just a few years, monitoring has undergone a huge transformation. Firstly, to cater for the new IT practices borne by agility and the digital transformation that closely ties business to the IS. Then, to adapt to Cloud infrastructure. And finally, to be able (at last) to move hand in hand with the development teams in the DevOps movement that has put monitoring back at the center of IT performance. Here is a guided tour of what monitoring owes to DevOps (and vice versa).
DevOps, or the reconciliation of the enemy brothers
Beyond the buzzword, DevOps is a grassroots movement born almost 10 years ago during successive professional conferences. The term DevOps was coined by Patrick Debois and is the result of concatenating the first letters of Development and the Ops abbreviation, which refers to operations in English.
The roots of DevOps are in Lean Management and the agile manifesto, from which it took the methods aimed at continuously improving value for the client, reducing wastage, but also developing collaboration between IT project players, and better structuring teams.
The main benefits can be found in terms of agility and productivity, but also in product quality and customer satisfaction. Teams make services available to users faster, and the services are more in line with user expectations. This allows businesses to reduce time to market, and remain in phase with their markets.
DevOps is founded on the cooperation between the different IT players around good practices in order to create, develop and roll out applications faster, cheaper and of higher quality. It thereby reconciles development and operations teams around Amazon CTO Werner Vogel’s famous rule: “you build it, you run it”. The person who designs is also the person who deploys. The team is therefore one of the main DevOps tools.
An impact on conventional IT processes… and monitoring
The DevOps approach, contrary to ITIL for example, is not a formal or standardized process. It is a set of good practices that directly impact conventional IT processes because they imply:
- Putting the user back at the center of processes,
- Focusing on short and controlled cycles based on continuous integration,
- Changing the way IT is designed, developed and produced,
- Rethinking the relationship with business teams and, in a wider sense, the customer relationship
- Organizing processes, services and products differently in the IT department,
- Sharing responsibilities between teams differently,
- Structuring processes around smaller, autonomous and versatile teams overseen by a Product Owner.
It is therefore the end of working in silos that structurally separated Dev from Ops.
For Operations and Development teams this means:
- Developing a culture around collaboration in order to overcome conventional compartmentalization,
- Automating operational tasks as much as possible in terms of roll-out, version management and monitoring, to increase quality and productivity, limit errors and beef up self-service,
- Measuring performance as a means to continuous improvement using specific DevOps indicators (deployment frequency, restoration speed, etc.),
- Sharing tools and experience based on an exchange and decompartmentalizing logic.
This new philosophy comes with a new relationship between teams and new tools that make
automation easier, encourage short cycles, and consume indicators. Monitoring is therefore at the
center of the DevOps system, because only monitoring can produce these tools.
Monitoring at the heart of DevOps systems
With DevOps, the idea is to collect as much data as possible from monitoring in order to analyze it, so that the business enhances its agility and makes corrections faster. Monitoring’s scope is widening and diversifying. Different types of data are being collected, with different reading levels and varied user cases. In the DevOps system, data analysis is a priority, but it cannot be achieved without collecting almost all IT data. This is true whatever the infrastructure model, application types or monitored systems. All this in a constantly evolving IT environment (mainly due to the Cloud). Monitoring must also evolve to collect data that can feed into business and functional indicators. Technicians are no longer the only people to use the monitoring solution dashboards. All departments
should be able to use monitoring.
It therefore has an essential role at the heart of the DevOps system. This was highlighted by Maslow’s Hierarchy of Needs (*) drawn up by Google to illustrate its SRE (Site Reliability Engineering) development methodology which takes much of its inspiration from DevOps methods. The Google method is based on a “site reliability” technical team that supports the research and hosting activities in Europe.
It puts monitoring at the foundations of the project, suggesting that nothing can be achieved without relevant and suitable monitoring.
With Data driven business, monitoring is moving towards “observability”
This has become a major stake in companies in which business is increasingly linked to IS performance. Data must now be put into context to better understand wavering metrics or behaviors. Rolling out a modification, as insignificant as changing a button on an e-commerce site, for example, can have a big impact on business performance. The monitoring tool must be able to highlight this sort of behavior.
It is becoming essential to have increasing numbers of business indicators by using diverse data (internal and external) and correlating it.
The monitoring trade is changing to directly serve business teams and quantify business
If we look at Netflix: the business proposes films and TV series continuously all over the world via Internet. Its monitoring collects a large amount of data in order to have indications to assess broadcast status in different geographical zones, to upgrade its infrastructure, but also to identify trends to create its new series using the collected and analyzed metrics.
Currently, monitoring tools use datawarehouses to store, historicize and contextualize large volumes of data. They can bring data up easily and offer a global view of all IT and the services provided. The vision of the IS is centralized, making it possible to see the activity of the different resources.
This also makes it possible to provide business management its own indicators to feed the continuous improvement process, which is a pillar of the ITIL and DevOps processes.
Cloud, monitoring, and automation: the three pillars of IT and business performance realtime view
DevOps is based on short cycles. To achieve that, it must be capable of rolling out to production quickly, and therefore automating most of the operations. This imperative is even stronger due to the fact that businesses are moving their infrastructure to the Cloud. IT’s scope is changing, and monitoring must change along with it. It adjusts the scope of monitoring almost instantly by creating the interface between the Cloud and internal tools.
Let’s pause for a second and consider the cloud principle, which is very closely connected to the digital transformation and the implementation of DevOps. The Cloud makes it possible to create and eliminate machines in a few seconds, and to break away from infrastructure hardware management. Previous practices such as feeding data into the CMDB covering infrastructure elements are not necessarily still valid. But the Cloud isn’t a black box.
Infrastructure is clearly moving towards serverless, but that doesn’t mean that the IT will be monitoringless! The IT will still require health checks, and new metrics will need to be included in monitoring to guarantee optimal operations.
Monitoring must move into the automation era, and even the self-configuration era using APIs, so that it can become a support for DevOps teams.
Read the Tribune:
- Le Journal du Net (France) : Ce que la supervision doit au DevOps (et vice versa)
- CanalPress (Spain) : Lo que la monitorización debe a DevOps, y viceversa
- ITCIO (Spain) : Lo que la monitorización debe a DevOps y viceversa
Share
