by Guest Writer Terry Slattery, CCIE #1026, Principal Architect, NetCraftsmen
Network automation frequently gets lumped into the category of Network Configuration and Change Management (NCCM) tools. This is a common misconception. Modern network automation systems are much more capable.
What is Network Configuration and Change Management (NCCM)?
NCCM products have been used for years to manage the configurations of network equipment. These products aid network operators in saving, comparing, and updating configurations of network equipment like routers and switches. The configuration for each network device is saved within the NCCM product, changes are made, and those changes are pushed out to the network equipment. The changes are performed through an application programming interface or by emulating a human who is interacting with the command line interface (CLI) of the device.
Command syntax differences between devices are typically not handled. The use of templates with variables make these systems better than a per-device configuration file approach. But the device-specific syntax requires that a network engineer create a per-device template for each unique device type.
NCCM products are useful for tracking config drift–the changes in network configurations from day to day. Knowing what changed is valuable because most network failures are due to human error, such as an incorrect configuration change. However, simply comparing each today’s device configuration against a previously captured device configuration is only part of the task. Once a change has been identified, the tools require that a network engineer manually evaluate the configuration change and either accept it or roll it back, a time-consuming process.
NCCM products rarely include the ability to perform pre-change and post-change network validation run arbitrary commands and parse the output to identify and extract important information. As a result, pre-change and post-change network validation in an NCCM environment is often a set of manual verification processes against raw show command output that was automatically collected. These manual processes are especially error-prone when the changes involve multiple steps.
Network Automation is Different
Comparing network automation with NCCM is like comparing a Formula 1 racer with a sedan. It is a paradigm change in comparison.
While there are a few similarities, there are substantial differences. They both manage configurations and can identify changes. They both perform some level of automation, but the broader goal of network automation is to eliminate as many manual processes as possible. This means that network automation includes work-flow automation that is not included in NCCM products. You can think of network automation as a more capable framework that includes NCCM. The current state of network automation is available in the report by Enterprise Management Associates – Research Summary: Enterprise Network Automation for 2020 and Beyond. Juniper Networks also does an annual survey and report, the latest of which is their 2019 State of Network Automation Report.
An interesting finding that cuts across many studies like those above is that most organizations are just starting to use automation. EMA found that the most successful organizations drive automation from Source-of-Truth databases (described below) and integrate with automated operations triggers. They further noted that AIOps is just starting to be adopted by some organizations.
Modern network automation incorporates multiple key practices and concepts:
- Declarative (intent-based) configuration
- Infrastructure-as-code (IaC) model
- Source-of-truth database to store network configuration state
- Automated pre-check and post-check process support
- Handling brownfield (existing network) and greenfield (new network) deployments
- Security vulnerability checking
- Network device OS upgrades
- Basic NCCM functions of config repository, config audit and config drift
- Overall workflow automation that allows building complete processes
- A REST API for interfacing with other network management products
Declarative Configuration
The better configuration systems use model-based declarative configuration. You define what you want the configuration to look like, not the steps that are required to make the change. The automation system determines what actions need to be taken to convert the current configuration to the desired configuration. This means that complex steps may be needed to add, remove, or edit configuration statements.
Infrastructure as Code
Modern network automation tools are built to manage network configurations using an infrastructure as code (IaC) model. IaC doesn’t necessarily mean that a programming language must be used. While some systems use code (typically Python-based), other systems use domain-specific configuration files (Ansible), and finally there are commercial systems that use configuration templates (configuration snippets) based upon per-device configuration variables. The templates replace device-specific configuration elements like IP addresses and device names with variables. A separate database of device-specific variable-value pairs, merged with the snippets, creates per-device configuration snippets.
Network Source-of-Truth Database
The variables that are used in configuration snippets should be stored in a source-of-truth (SoT) database. The SoT data should also be maintained using IaC principles, because the combination of the configuration templates and the per-device variables are used to create network configurations.
Using a SoT decouples device-specific variables from the specific configuration syntax used by a device. For example, an NTP configuration would refer to network clock sources by DNS name or IP address. When replacing a network device, the new configuration may use different syntax than the old device. It is simple for the network automation system to detect the new device type and use the configuration template for that device. Differences in command syntax are automatically handled by the platform, not the network engineer.
Automate Pre-Change and Post-Change Checks
Network automation isn’t just about implementing a change. A critical aspect is identifying network problems prior to implementing a change and then verifying that the network is functioning correctly after the change is made. Deviations in either the pre-change or post-change check causes the change to be aborted and, if necessary, rolled back.
Handling both Brownfield and Greenfield Networks
Good network automation tools must be able to handle both existing brownfield networks and new greenfield networks. This is often trickier than it might otherwise seem because products frequently need to manage entire device configurations.
Security Vulnerability Checking
Network security has become increasingly important. A network automation system should include the ability to check for product security instances (frequently called PSIRT, to reference the teams that identify and handle security vulnerabilities). It’s not just about checking for a vulnerable OS. It is necessary to also determine if the defective feature is enabled within the configuration.
Network Device OS Upgrades
Change and configuration management the ability to automate the process of updating the network device operating system. Automation of OS upgrades is required to keep the network safe from security vulnerabilities. The upgrade process can be quite challenging for an automated system, due to differences across devices, even within a single vendor.
Basic NCCM Functions
Finally, the network automation system must perform basic NCCM functions like configuration drift detection and configuration audits. A critical component of these functions is how the system handles any exceptions. Ideally, the automation system will provide the option for automatic configuration change to put the network device back into compliance. In all cases, the system should alert the network management staff to the exception and the remediation steps that were taken, if any. The ability to automatically correct any exceptions to either drift or audit is a powerful feature that significantly reduces the manual processing load on the networking staff.
Work-flow Automation
Network automation systems don’t just automate the configuration change. They automate an entire work-flow, eliminating many manual steps from network administration processes. A good example is the pre-change and post-change checks of both configuration and operational state, combined with the ability to abort or roll-back a change if exceptions are detected.
REST API
Communications between network management processes is imperative and the REST API has become the network industry’s generally accepted mechanism. For example, a network monitoring system could detect that an existing traffic engineering mechanism is out of sync with the type of traffic transiting the network. It could then make a REST call to the network automation system to enact a change to the traffic engineering configuration to improve the users’ experience.
The Gluware Approach to Network Automation System
Gluware is a comprehensive network automation system. It incorporates declarative configuration with the additional advantage that network engineers can use it without the need to acquire software development skills. Only the parts of the configuration that are modified will be touched, which is a very nice feature for automation of brownfield networks. Configuration variables are decoupled from the configuration snippets to create its own source-of-truth database. Gluware is able to perform a preview of the change along with pre-change and post-change checks in which the configuration and operational state of a device can be verified before or after a change. A change can be aborted or rolled back if a check fails. This capability prevents a change from being applied to a network that has already deviated from the expected state. For example, don’t apply a routing protocol change if the set of neighbors and routes deviates from what is expected.
On the security front, Gluware handles security vulnerability checks, including whether a vulnerable feature is enabled. When vulnerable devices are identified, the network team is alerted, who can then use Gluware to perform the OS upgrade.
Configuration drift and configuration audit are handled smoothly by Gluware, including the ability to automatically remediate any unauthorized changes or exceptions to policy. It can automate work-flows to greatly streamline processes that typically rely on manual steps. A REST API provides the ability to interface with other network management tools, adding to the ability to remove manual steps in complex processes.
While it may seem less expensive to use legacy NCCM tools and attempt to script yourself, you need to take into account the cost in mistakes and time to automate in order to understand the true cost of older solutions and the value of a more modern approach like Gluware. The speed of deployment (weeks instead of months) makes a significant difference to deliver on business requirements.
Want to Learn More?
To learn more about network automation from Gluware, check out collateral, videos and blogs here:
Gluware product collateral
Gluware videos
Gluware blog
