Configuring Nokia SR OS with NETCONF

netlab uses gNMI to configure Nokia SR OS devices. The gNMI Ansible collection has to be installed from a GitHub repository together with an outdated version of the protobuf library (netlab install grpc), which makes the installation fragile.

This document describes how we could replace gNMI with NETCONF while reusing the existing SR OS configuration templates.

Current Implementation

All SR OS configuration templates render YAML data describing a gNMI SetRequest:

updates, replace and delete lists
Every list element is a dictionary with a path (gNMI path) and a val (YANG-JSON data in the nokia-conf YANG model)

The deploy-config/sros.yml task renders the template, parses the YAML data, and passes the lists to the nokia.grpc.gnmi_config module.

updates:
- path: configure/router[router-name=Base]/interface[interface-name=system]
  val:
   admin-state: enable
   description: "System interface"
   ipv4:
    primary:
     address: "10.0.0.1"
     prefix-length: 32

Proposed Approach

SR OS exposes the same nokia-conf YANG model over NETCONF. The gNMI data generated by the configuration templates can thus be converted into NETCONF XML with a mechanical transformation that does not need the YANG schema. The configuration templates (including the users’ custom configuration templates) do not have to change.

The conversion would be implemented as an Ansible filter (for example, netsim/ansible/filter_plugins/sros_netconf.py; Ansible loads filter plugins from the filter_plugins directory next to the playbooks).

gNMI data	NETCONF XML
`elem[key=val]` path element	`<elem><key>val</key>...</elem>`; key leaves are emitted first (taken from the path, duplicates in val are skipped)
Keys containing slashes (`port-id=1/1/c1`)	Bracket-aware path parsing
`configure` top-level element	`<configure xmlns="urn:nokia.com:sros:ns:yang:sr:conf">`
updates	Default `merge` operation
replace	`nc:operation="replace"` on the target node
delete	`nc:operation="remove"` on the target node
JSON list of dictionaries or leaf-list	Repeated elements
`[null]` (YANG empty type) or `{}`	Empty element
`True`/`False`	`true`/`false`
Scalar val (gNMI path points to a leaf)	Leaf element text
Multiple items sharing a path prefix	Merged into a single XML tree

The whole module configuration is sent in a single <edit-config> RPC to the candidate datastore followed by a commit. gNMI applies all delete operations, then all replace operations, then all update operations within a single atomic transaction. The converter processes the items in the same order and resolves overlapping operations to get the same end result:

A replace or delete operation discards the earlier changes within its subtree
An update or replace below (or at) a deleted node changes the remove operation into replace: deleting a node and then creating some of its contents is equivalent to replacing the node with the new contents
An update below (or at) a replaced node is merged into the replacement contents

Converted configuration snippet:

<nc:config xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0">
  <configure xmlns="urn:nokia.com:sros:ns:yang:sr:conf">
    <router>
      <router-name>Base</router-name>
      <interface>
        <interface-name>eth-1/1/c3</interface-name>
        <ipv4>
          <bfd nc:operation="replace">
            <admin-state>enable</admin-state>
            <receive>500</receive>
            <transmit-interval>500</transmit-interval>
            <multiplier>3</multiplier>
          </bfd>
        </ipv4>
      </interface>
    </router>
  </configure>
</nc:config>

Prototype Converter

A prototype converter has been tested on the configurations generated for an SR-SIM lab using the initial, ospf, bgp, bfd, and vrf modules (netlab initial -o). All configuration files were converted; the results have not been tested on a live device.

import re
import xml.etree.ElementTree as ET

NS = { 'configure': 'urn:nokia.com:sros:ns:yang:sr:conf' }
NC = 'urn:ietf:params:xml:ns:netconf:base:1.0'
PATH_ELEM = re.compile(r'([^/\[]+)((?:\[[^\]]*\])*)')

def split_path(path):              # 'a/b[k=1/1/c1]/c' -> [(a,{}),(b,{k:1/1/c1}),(c,{})]
  return [ (m[1], dict(kv.split('=',1) for kv in re.findall(r'\[([^\]]*)\]',m[2])))
           for m in PATH_ELEM.finditer(path) ]

def scalar(v):
  return 'true' if v is True else 'false' if v is False else str(v)

def add_value(parent, name, val):
  if isinstance(val, list):
    if val == [None]:                # YANG 'empty' in JSON encoding
      ET.SubElement(parent, name); return
    for item in val:                 # list entries or leaf-list values
      add_value(parent, name, item)
    return
  e = ET.SubElement(parent, name)
  if isinstance(val, dict):
    for k, v in val.items():
      add_value(e, k, v)
  elif val is not None:
    e.text = scalar(val)

def find_or_add(parent, name, keys):
  for c in parent.findall(name):     # merge entries sharing the same path prefix
    if all(c.findtext(k) == v for k, v in keys.items()):
      return c
  e = ET.SubElement(parent, name)
  for k, v in keys.items():          # keys from the path go first
    ET.SubElement(e, k).text = v
  return e

def to_xml(cfg):
  OP = '{%s}operation' % NC
  root = ET.Element('{%s}config' % NC)
  for op in ('delete', 'replace', 'updates'):   # gNMI processing order
    for item in cfg.get(op) or []:
      elems = split_path(item['path'])
      keys = elems[-1][1]
      node = root
      for idx, (name, path_keys) in enumerate(elems):
        node = find_or_add(node, name, path_keys)
        if idx == 0:
          node.set('xmlns', NS[name])
        if op != 'delete' and node.get(OP) == 'remove':
          node.set(OP, 'replace')        # delete followed by replace/update is a replace
      if op != 'updates':                # replace/delete supersedes earlier changes in the subtree
        for c in [ c for c in node if c.tag not in keys ]:
          node.remove(c)
        node.set(OP, 'remove' if op == 'delete' else 'replace')
      val = item.get('val')
      if isinstance(val, dict):
        for k, v in val.items():
          if k not in keys:              # already emitted from path predicate
            add_value(node, k, v)
      elif val is not None and val != [None]:
        node.text = scalar(val)          # scalar leaf value
  ET.register_namespace('nc', NC)
  ET.indent(root)
  return ET.tostring(root, encoding='unicode')

The prototype does not handle gNMI paths pointing to a leaf-list (a list of scalar values in val); none of the current templates use them.

Required Changes

deploy-config/sros.yml: render the template as today, then use ansible.netcommon.netconf_config with ansible_connection: ansible.netcommon.netconf and port 830 (already defined as netconf: 830 in defaults/ports.yml):

ansible.netcommon.netconf_config:
  content: "{{ yaml_config | from_yaml | sros_netconf_xml }}"
  target: candidate
  commit: true
  lock: always

readiness-check/sros.yml: remove the gRPC plugin check and wait for TCP port 830.
readiness-check/srsim.yml: convert the SR-SIM hardware configuration with the same filter and replace gnmi_get with netconf_get using a <state xmlns="urn:nokia.com:sros:ns:yang:sr:state"> filter.
fetch-config/sros.yml: use netconf_get with a <configure/> filter (XML output) or the md-cli-raw-command RPC executing admin show configuration (MD-CLI output).
Installation: replace netlab install grpc with installing ncclient (ansible.netcommon is already part of the Ansible distribution).
Remove the sros_grpc_port group variable.
Remove the unused OpenConfig templates (sros.openconfig.j2) and the sros_use_openconfig group variable instead of adding module-prefix-to-namespace mapping to the converter.

Open Questions

These have to be verified on a live SR-SIM device:

The NETCONF connection selects the ncclient device handler based on ansible_network_os (sros). ncclient has an sros device handler, but we have to check that the Ansible default NETCONF plugin uses it.
Is NETCONF enabled in the default containerlab SR-SIM startup configuration, and is the admin user allowed to use it?
The converter knows the list keys only for path elements. Within val, it relies on the ordering of the template data. That ordering could be wrong when the template data is generated with to_nice_yaml (which sorts the keys), for example in the SR-SIM hardware configuration. Does SR OS reject list entries in which the keys are not the first child elements?