Building a Private Local AIOps Assistant: Part 3 - Final Modelfile, and AI Model

Prerequisite: Make sure you have Ollama installed to follow along with this tutorial, you can download for FREE from: https://ollama.com/download

In my last post, I embedded my network topology directly into the devops-buddy Ollama Modelfile.

It was a big improvement, but during further testing I discovered something frustrating: getting the model to produce the correct configuration once didn't mean it would consistently produce the correct configuration.

On subsequent runs, it could still invent interfaces or insist on generating IPv6 configuration even though IPv6 wasn't part of my topology.

So today I changed the approach slightly.

Instead of asking the model to work out parts of my topology, I decided to give it the exact information it needs and make its job as deterministic as possible.

And this time, the result was very different.

Stop Making the LLM Calculate the Network

My original topology definition described the P2P addressing logic and expected the model to work out which addresses belonged to which interfaces.

That seemed reasonable to me as a network engineer.

But why make the LLM calculate something that I already know?

I changed the topology section so that every link explicitly defines both devices, interfaces and IP addresses:

Network Topology:


- P2P Link 1: J1 em2 | ip: 172.16.14.1/24 <---> J4 em2 | ip: 172.16.14.4/24
- P2P Link 2: J1 em4 | ip: 172.16.13.1/24 <---> J3 em4 | ip: 172.16.13.3/24
- P2P Link 3: J2 em3 | ip: 172.16.24.2/24 <---> J4 em3 | ip: 172.16.24.4/24
- P2P Link 4: J3 em3 | ip: 172.16.35.3/24 <---> J5 em3 | ip: 172.16.35.5/24
- P2P Link 5: J4 em4 | ip: 172.16.46.4/24 <---> J6 em4 | ip: 172.16.46.6/24

This is a small but important change.

I'm no longer asking the LLM to calculate my network.

I'm giving it the network and asking it to generate configuration from that information.

Meet the New NetDevOpsBuddy

For the next experiment I switched to Gemma 4 12B.

My new Modelfile is:

FROM gemma4:12b


# 1. Force the creativity engine to absolute zero
PARAMETER temperature 0


# Define System persona
SYSTEM """
You are NetDevOpsBuddy, a specialized AIOps and Network Automation Assistant.
Your creator and user is Shahzad Qadir, an experienced Network Engineer building a modern AIOps platform.


Core Network Blueprint
- Network Devices: J1, J2, J3, J4, J5, J6
- Use number prefix in the device name as device number
- Mgmt Network: 10.10.99.0/24, device IP depends on device number for example 10.10.99.1/24 for J1 and so on.
- Loopback Network: 200.0.0.0/24, loopback interface is lo0 and uses unit 0, loopback IP depends on device number, for example 200.0.0.1/24 for J1 and so on.
- Mgmt Interface: em5
- Devices Software: juniper_junos
- Device Software Version: 14.1R1.10


Network Topology:


- P2P Link 1: J1 em2 | ip: 172.16.14.1/24 <---> J4 em2 | ip: 172.16.14.4/24
- P2P Link 2: J1 em4 | ip: 172.16.13.1/24 <---> J3 em4 | ip: 172.16.13.3/24 
- P2P Link 3: J2 em3 | ip: 172.16.24.2/24 <---> J4 em3 | ip: 172.16.24.4/24 
- P2P Link 4: J3 em3 | ip: 172.16.35.3/24 <---> J5 em3 | ip: 172.16.35.5/24
- P2P Link 5: J4 em4 | ip: 172.16.46.4/24 <---> J6 em4 | ip: 172.16.46.6/24


Behavior Guidelines:
1. All generated configs will be for Juniper Junos 14.1R1.10 and in set format
2. For layer 3 ip addressing use unit 0 as that is what Juniper recommends using unless it is a tagged interface.
3. We don't have tagged interfaces in the topology.
4. When generating configs, don't go beyond what is in topology and also consider you are on Juniper Junos Software.
5. Never discover interfaces or IP address, strictly follow whatever is in the topology. If no IPv6 addresses are defined in topology don't generate any IPv6 related
"""

There are two things I particularly like about this version.

First, the topology is explicit.

Second, the model is given clear boundaries.

It doesn't need to decide what interfaces exist.

It doesn't need to discover IP addresses.

It doesn't need to invent IPv6.

It simply needs to use the information I've provided.

The Test

I then asked Buddy something slightly more demanding:

can you generate interface configs for all devices in this network topology?

This time, it correctly understood the complete topology and generated configurations for all six routers.

For example, J4 has three P2P connections, making it a useful test of whether the model can keep multiple relationships straight.

The generated configuration was:

set interfaces em5 unit 0 family inet address 10.10.99.4/24
set interfaces lo0 unit 0 family inet address 200.0.0.4/32
set interfaces em2 unit 0 family inet address 172.16.14.4/24
set interfaces em3 unit 0 family inet address 172.16.24.4/24
set interfaces em4 unit 0 family inet address 172.16.46.4/24

The important thing isn't just that the syntax looks correct.

Buddy correctly understood that:

J4 em2 -> J1
J4 em3 -> J2
J4 em4 -> J6

and used the corresponding addresses from the topology.

The same happened across the rest of the network.

J1
set interfaces em5 unit 0 family inet address 10.10.99.1/24
set interfaces lo0 unit 0 family inet address 200.0.0.1/32
set interfaces em2 unit 0 family inet address 172.16.14.1/24
set interfaces em4 unit 0 family inet address 172.16.13.1/24
J2
set interfaces em5 unit 0 family inet address 10.10.99.2/24
set interfaces lo0 unit 0 family inet address 200.0.0.2/32
set interfaces em3 unit 0 family inet address 172.16.24.2/24
J3
set interfaces em5 unit 0 family inet address 10.10.99.3/24
set interfaces lo0 unit 0 family inet address 200.0.0.3/32
set interfaces em4 unit 0 family inet address 172.16.13.3/24
set interfaces em3 unit 0 family inet address 172.16.35.3/24
J5
set interfaces em5 unit 0 family inet address 10.10.99.5/24
set interfaces lo0 unit 0 family inet address 200.0.0.5/32
set interfaces em3 unit 0 family inet address 172.16.35.5/24
J6
set interfaces em5 unit 0 family inet address 10.10.99.6/24
set interfaces lo0 unit 0 family inet address 200.0.0.6/32
set interfaces em4 unit 0 family inet address 172.16.46.6/24

Six devices.

All the expected interfaces.

All the expected addresses.

No invented interfaces.

No IPv6.

No OSPF.

No BGP.

No configuration outside the request.

This is exactly what I wanted devops-buddy to do.

A Small but Important Lesson

This experiment has changed how I'm thinking about using LLMs for network automation.

My first instinct was to make the model intelligent enough to work things out.

I'm now leaning towards the opposite approach.

Give the model less to figure out.

If my application already knows that:

J1 em2 = 172.16.14.1/24

then there is no reason to ask the LLM to calculate it.

The application should provide the fact.

The LLM should understand the request and generate the appropriate output.

This separation is going to become increasingly important as devops-buddy grows.

From a Modelfile to a Real Application

At the moment, my topology is effectively part of Buddy's personality.

That's useful for experimentation, but I don't want to maintain a production network that way.

If I add J7, change an IP address or replace an interface, I shouldn't have to rebuild my entire LLM.

This is where the next stage of the project begins.

I want to move the network information into a proper data store and let Buddy retrieve the information it needs.

The architecture will start moving towards:

User
  |
  v
devops-buddy
  |
  +---- Network Database
  |       |
  |       +-- Devices
  |       +-- Interfaces
  |       +-- IP Addresses
  |       +-- Topology
  |
  +---- Configuration Generation
  |
  +---- Configuration Validation
  |
  v
Network Automation

In Part 4, we will move back to our coding environment and introduce a local SQLite database to track our session history permanently. So we can do more useful things with devops-buddy like helping us setup a ansible environment and automate network configurations.