The journey of a network engineer to automation is not something new, it has been emerging since practically the beginning of networking technologies. In some environments, automation has been a threat to the network engineer when it is the opposite, a help in their day to day.
In recent times the acronym SDN (Software Defined Networking) has become the magic word that floods the portfolios of all manufacturers of communications equipment. One of the reasons is to automate communication infrastructures. Moving from a distributed control plane paradigm to a centralized control plane that orchestrates communications infrastructures.
One of the motivations for the use of this type of technology is to automate operations on the network: provisioning on the network, obtaining usage data, deployment of new equipment. That is why some environment’s automation is identified with these technologies, when the reality is that the relationship between the two is not bijective: SDN technologies use automation, but automation does not have to go hand in hand with SDN.
The RAE (Royal Spanish Academy) defines automate as “Converting certain movements into automatic or unintentional movements.” Within the communications environment, this translates into the identification of repetitive tasks and applying the mechanisms and tools that allows a mechanical execution of them.
Although there are and have been commercial tools for the industrialization and automation of tasks in communications networks, they are suitable for stable production environments or those with little variability. In engineering projects in which efficiency in the execution of the changes rule, whether it is the replacement of equipment, or the design and deployment of new equipment, or testing upon receipt of the equipment that it works correctly, etc., … the use of traditional tools is neither efficient nor effective.
At this point is when the skills of the network engineer are put into value in order to minimize the time necessary to execute the different tasks.
The purchase of automation skills in network engineers is born not only as a market need, but rather as a concern of engineers themselves. This situation is well reflected in this classic internet graphic.
Engineers are motivated to do different, challenging tasks. You can even say that these kinds of tasks is fun for them. When an engineer must perform repetitive tasks, he will find a way to minimize the time he needs to carry out those tasks. And we must recognize that automation is a fun challenge: making a tedious and boring task can be done quickly and with just one click.
For this he will use the available tools. The biggest problem the network engineer finds is the ignorance of what tools there are. The growing interrelation with systems and development departments, the DevOps paradigm, has facilitated the discovery of these types of tools.
If we focus on specific examples, within the world of communications a typical case is the creation of templates for the display of equipment. The way in which this task is carried out has changed over time.
- At first templates were created with variables that were substituted manually or with the famous “search and replace”.
- Afterwards, we go on to use and combine the functionality of MS Word. Sometimes this approach caused some problems due to the automatic conversion of special characters.
- In some cases, this correspondence combination was programmed in a different language (VisualBasic, Perl, etc …), which allowed greater flexibility in this task.
However, this approach only works well when all or most of all equipment and configurations are the same or nearly the same. The moment in which variability is introduced in these templates, there is no choice but manual intervention or ad-hoc programming, the script that makes the templates.
To solve this problem, web development tools such as template programming languages come to the aid of the network engineer. Particularly the Jinja2 language, designed to make web templates in python environments.
This language allows you to design a template that behaves differently depending on the data that is provided. It adds the concept of modularity to templates. It is no longer necessary to have a monolithic template, but we can have small templates that can be applied or not depending on the data with which we want to build our configuration.
The learning curve for this template language for a network engineer is very low. Very elaborate templates can be made without going too far into the complexities of a programming language.
Although Jinja2 is a very powerful language that allows you to do magic, what a network engineer really needs is much less: to be able to configure a variable number of elements, such as network interfaces; to be able to apply or not a configuration based on one or several parameters, such as applying BGP configuration or not to a computer based on its role in the network; etc.,
The use of jinja2 supposes an improvement in how the templates are made, since it allows their industrialization and automation.
However, the use of these templates, same as with traditional templates, but to a greater extent, raises another problem: managing their versions.
Now it is essential to keep track of these versions: what is the latest version, what are the changes compared to the previous version, who has made the latest changes, what is the reason behind them, etc.,
The traditional way is to use a manual naming criterion for the template files, but this estimation has always brought bad experiences: versions with inconsistent names, who hasn´t seen templates with names like “definitive”, “last” or “Final” within the same directory? Quite frequent a file has been replaced by another by mistake, losing all or part of the information. Not to mention disputes when two people have concurrently modified the same template.
The world of development returns to the rescue of the network engineer. In this environment version control is essential and many tools of this type have been used. For years the predominant tool has been Git, a tool developed by the famous Linus Torvalds to manage the versions of the Linux kernel.
This tool is going to solve the two main problems:
- It allows us to have version control. It is no longer necessary to change the name of the files; it is the tool itself that is responsible for creating versions and controlling the differences between them.
- Group work: which allows more than one person to work on a copy of a file at the same time and automatically merge all changes. Only if two people have touched the same lines of a file will manual intervention be required to resolve the conflict.
Next step in our journey is how to obtain the necessary data to feed our template and generate our configuration.
A common type of project in communication environments is the replacement or migration of equipment. At times this equipment is from the same manufacturer and in others it is not, but in all cases, it is necessary to obtain the data of the services in the current equipment in order to migrate them to the new ones.
If up to now it has been possible to avoid the use of programming languages, at this point it is difficult not to have to acquire minimum skills. Most of the current equipment already allows data to be delivered in structured formats (mainly xml or json), but this is not so common in less recent equipment. This equipment delivers raw text, so in many cases it is necessary to have to parse the configuration: analyze the configuration to obtain the necessary data in a format with which we can feed our configuration template.
Among the programming languages used, Python stands out. The main reasons are its simplicity and the amount of resources and examples that can be found online. It is currently the “de facto” language used in most network automation tools. In fact, an interpreter for this language is already included in the equipment of the main network manufacturers, such as Cisco, Juniper or Nokia.
Based on all the above mentioned, we would already be able to analyze and process information on our PCs automatically. It remains to be seen how to obtain that information dynamically and how to dump network configuration in the same way.
Here we have three large groups of options:
- Use of tools embedded in connection managers such as secureCRT. The big problem with this approach is usually twofold
- Many times, these tools are proprietary and have a cost.
- Sharing the scripts requires the use of the same connection manager by the whole team, something that is not usually the case.
Use of tools developed by manufacturers: the main players in the sector have developed open source python libraries that can be used to carry out this task. Examples would be pyATS from Cisco or pyEZ from Juniper.
- Utilization of manufacturer agnostic and community-developed tools. Here we would I have two main aspects:
- Ansible: general automation tool from the DevOps world.
- Network-oriented tools like napalm, nornir or salt.
Among all of them, Ansible stands out for the following reasons:
- There is no need of programming knowledge to use it. The rest of the tools are python libraries that can be used in our scripts.
- It has many specific modules for each manufacturer, developed in most cases by those same manufacturers.
- It is relatively easy to create new modules, programming, though. But once the module is programmed, anyone can use it without having that programming knowledge
Ansible allows great flexibility in automating tasks. It allows for example:
- To connect to a team and find out against which teams you set up transport sessions.
- To connect to those computers and find out what the status of the service is according to the parameters that we determine.
- To put the information obtained with the format that interests us the most to disk.
Until now we did this type of task relatively manually, both obtaining the information and the correlation itself.
The use of ansible as an automation tool allows us to be able to easily replicate a task at any given moment (for example: before and after performing an intervention on the network), and also reduces the risks of errors that occur between the chair and the computer keyboard.
The way ansible is “programmed” is like the so-called playbooks: files in yaml format in which we indicate what actions we want it to perform. Embedded in ansible, jinja2 is used as the template language. Therefore, these playbooks can (and should) also be managed using git.
We mustn’t forget to highlight a meta-tool that flies over all the previous ones: GNU Linux. This operating system allows great automation capabilities since its inception, some that have given as good results as expected, which allows a dialogue to be carried out with a device: I send you some information and before you reply I send you other information, and so on. Some tools like ansible only work on a Linux kernel. Since the introduction of the Windows Subsystem for Linux (WSL) ansible can also be run on Windows, so there is no excuse to learn a minimum of this operating system.
Finally, it is convenient to remember that automation is not an end, but a resource. In our day to day, before starting to execute a new task, we must analyze whether it is worth automating it.
Illustration 1 Source www.xkcd.com
We must also have in mind that the ideal curve for how much time we save with automations does not always fit the above graph of “geeks and repetitive tasks”. If we lose sight of the fact that automation is a resource and not an end, that curve can turn into the following nightmare:
Illustration 2: Source www.xkcd.com
But the only way as a network engineer to get to automation efficiency is to start the journey… and start having fun.
This article was published in October by “A NOSA REDE” by Colegio Oficial Enxeñeiros de Telecomunicación de Galicia . You can access through this link.