Description

This article describes how to configure an event policy to capture different command outputs in case of communication loss.

Solution

For this sample, we chose neighbor IP "10.50.8.1" as the main target. Communication will be tracked using ICMP, and if the target IP becomes unreachable the switch will run a preconfigure set of commands.

Full configuration:

  1. Configure an RPM probe to monitor connectivity with "10.50.8.1" using ICMP traffic:

    set services rpm probe ConnectivityProbe test NeighborState probe-type icmp-ping
    set services rpm probe ConnectivityProbe test NeighborState target address 10.50.8.1
    set services rpm probe ConnectivityProbe test NeighborState probe-interval 5
    set services rpm probe ConnectivityProbe test NeighborState thresholds successive-loss 5

  2. Configure an event policy to be triggered if 10.50.8.1 is not longer reachable:

    set event-options policy collectoutputpolicy events ping_test_failed
    set event-options policy collectoutputpolicy within 30 trigger on
    set event-options policy collectoutputpolicy within 30 trigger 2
    set event-options policy collectoutputpolicy attributes-match ping_test_failed.test-owner matches ConnectivityProbe

  3. Specify the destination folder and file name where outputs will be stored:

    set event-options policy collectoutputpolicy then execute-commands output-filename event_results
    set event-options policy collectoutputpolicy then execute-commands destination   myfolder
    set event-options policy collectoutputpolicy then execute-commands output-format text
    set event-options destinations myfolder   archive-sites /var/tmp

  4. Configure all commands to be run if the event is triggered. By default, ping, monitor and other non-show commands do not stop without manual intervention (Ctrl+C), if they need to be included on the list, make sure to use option " count ", otherwise the  MTR process will spike and run the commands endlessly. We do not recommend including traceroute commands as option " count " may not be available.

    set event-options policy collectoutputpolicy then execute-commands commands "clear arp hostname 10.50.8.1"
    set event-options policy collectoutputpolicy then execute-commands commands "show interfaces xe-0/0/0 extensive"
    set event-options policy collectoutputpolicy then execute-commands commands "show ethernet-switching table interface xe-0/0/0"
    set event-options policy collectoutputpolicy then execute-commands commands "show arp no-resolve expiration-time interface irb.50"
    set event-options policy collectoutputpolicy then execute-commands commands "ping 10.50.8.1 count 4"
    set event-options policy collectoutputpolicy then execute-commands commands "monitor traffic interface xe-0/0/0 size 1500 no-resolve count 15"
    set event-options policy collectoutputpolicy then execute-commands commands "monitor traffic interface irb.50 size 1500 no-resolve count 15"

Full Configuration

{master:0}
root@QFX5100_Lab> show configuration services
rpm {
    probe ConnectivityProbe {
        test NeighborState {
            probe-type icmp-ping;
            target address 10.50.8.1;
            probe-interval 5;
            thresholds {
                successive-loss 5;
            }
        }
    }
}

root@QFX5100_Lab> show configuration event-options
policy collectoutputpolicy {
    events ping_test_failed;
    within 30 {
        trigger on 2;
    }
    attributes-match {
        ping_test_failed.test-owner matches ConnectivityProbe;
    }
    then {
        execute-commands {
            commands {
                "clear arp hostname 10.50.8.1";
                "show interfaces xe-0/0/0 extensive";
                "show ethernet-switching table interface xe-0/0/0";
                "show arp no-resolve expiration-time interface irb.50";
                "ping 10.50.8.1 count 4";
                "monitor traffic interface xe-0/0/0 size 1500 no-resolve count 15";
                "monitor traffic interface irb.50 size 1500 no-resolve count 15";
            }
            output-filename event_results;
            destination myfolder;
            output-format text;
        }
    }
}
destinations {
    myfolder {
        archive-sites {
            /var/tmp;
        }
    }
}

Once the event is triggered, a file will be created and stored on the pre-configured destination folder after all the commands are run. The format of the file includes the name of the host name, date, time and file name. The content of the files could be check using  'file show.'

Note: Once the event is detected, all instructions will run only once regardless the time the target IP remains down. After communication is restored, the cycle will restart as well and create separate files per event.

root@QFX5100_Lab> file list /var/tmp/

/var/tmp/:
.snap/
QFX5100_Lab_20191107_134058_event_results
QFX5100_Lab_20191107_134750_event_results
install/
mmcq_mmdb_rep_mmcq
pics/
rtsdb/
sd-upgrade/
vi.recover/

Sample output:

root@QFX5100_Lab> file show /var/tmp/QFX5100_Lab_20191107_134750_event_results

root@QFX5100_Lab> clear arp hostname 10.50.8.1
10.50.8.1        deleted

root@QFX5100_Lab> show interfaces xe-0/0/0 extensive
Physical interface: xe-0/0/0, Enabled, Physical link is Up
  Interface index: 649, SNMP ifIndex: 509, Generation: 141
  Link-level type: Ethernet, MTU: 1514, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, Source filtering: Disabled
  Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Flow control: Disabled, Media type: Fiber
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Link flags     : None
  CoS queues     : 12 supported, 12 maximum usable queues
(...)

Using ' show services rpm probe-results ' provides a summary of the event results as well:

Results over all tests: 

Probes sent: 6889, Probes received: 6744, Loss percentage: 2.104805
      Measurement: Round trip time
Samples: 6744, Minimum: 1344 usec, Maximum: 1771109 usec, Average: 17630 usec, Peak to peak: 1769765 usec, Stddev: 32491 usec, Sum: 118897779 usec
      Measurement: Positive round trip jitter
Samples: 3444, Minimum: 0 usec, Maximum: 1759984 usec, Average: 12365 usec, Peak to peak: 1759984 usec, Stddev: 44541 usec, Sum: 42584898 usec
      Measurement: Negative round trip jitter
Samples: 3299, Minimum: 1 usec, Maximum: 1759976 usec, Average: 12908 usec, Peak to peak: 1759975 usec, Stddev: 45391 usec, Sum: 42584976 usec