Description

In a scaled PWHT environment, the number of locally terminated circuits might exceed the size of the MAC-address pool associated with a logical tunnel interface. In this case, the excess circuits will remain unusable.

Symptoms

In lt-style configuration of PWHT, there are two types of sub-interfaces (IFLs) configured under a logical tunnel interfaces (lt-):

interfaces {
    lt-1/0/0 {
        unit 100 {                          /* Service IFL */
            encapsulation vlan;
            vlan-id 100;
            peer-unit 1100;
            family inet {
                address 198.19.0.1/30;
            }
        }
        unit 1100 {                         /* Transport IFL */
            encapsulation vlan-ccc;
            vlan-id 100;
            peer-unit 100;
        }
    }
}
protocols {
    l2circuit {
        neighbor 198.18.255.2 {
            interface lt-1/0/0.1100 {
                virtual-circuit-id 1100;
            }
        }
    }
}

To make use of access devices, which are incapable of qualified (per-VLAN) MAC-learning, JUNOS allocates a unique MAC address for every "service" IFL, e.g.:

> show interfaces lt-1/0/0 terse | match "lt-1/0/0.10[0-3]"
lt-1/0/0.100            up    up   inet     198.19.0.1/30
lt-1/0/0.101            up    up   inet     198.19.0.5/30
lt-1/0/0.102            up    up   inet     198.19.0.9/30
lt-1/0/0.103            up    up   inet     198.19.0.13/30


> show interfaces lt-1/0/0 | match "(lt-1/0/0.10[0-3]|MAC)"
  Logical interface lt-1/0/0.100 (Index 332) (SNMP ifIndex 655)
    MAC: 02:24:dc:78:69: 4a
  Logical interface lt-1/0/0.101 (Index 334) (SNMP ifIndex 659)
    MAC: 02:24:dc:78:69: 4b
  Logical interface lt-1/0/0.102 (Index 336) (SNMP ifIndex 3900)
    MAC: 02:24:dc:78:69: 4c
  Logical interface lt-1/0/0.103 (Index 338) (SNMP ifIndex 3901)
    MAC: 02:24:dc:78:69: 4d
 

On the other hand, a typical number of MAC-addresses which are allocated per lt- interface from the global (chassis) pool is 64.  Thus, if the number of locally terminated PHWT circuits crosses the threshold, the newly provisioned interfaces will remain inactive:

<-- Syslog facility = kernel

May 11 12:43:37.823  jtac-MX480-r182 kernel: if_rtb_default_ifl_bitmap_set() rtb ifl bitmap itable op 1 done. rtb id 0 ifl idx 476 iff 0xfffff80403739e80.
May 11 12:43:37.823  jtac-MX480-r182 kernel: RLT:lt_ifl_peer_update:lt-1/0/0.1164 lt-1/0/0.164 op 1
May 11 12:43:37.823  jtac-MX480-r182 kernel: RLT:lt_ifl_peer_update:OLD {lt-1/0/0.1164-lt-1/0/0.164} cooki{0 130}
May 11 12:43:37.823  jtac-MX480-r182 kernel: RLT:lt_ifl_peer_update:NEW {lt-1/0/0.1164-lt-1/0/0.164} cooki{130 130}
May 11 12:43:37.823  jtac-MX480-r182 kernel: lt_ifl_alloc_mac_addr(530), IS_MASTER_RE: 1, error: 28 , errmsg: free index unavailable for mac address
May 11 12:43:37.954  jtac-MX480-r182 kernel: RLT:lt_ifl_peer_update:lt-1/0/0.1164 lt-1/0/0.164 op 2


  Note errno 28  stands for "No space left on device".

> show interfaces lt-1/0/0 terse | match "lt-1/0/0.16[0-7]"
lt-1/0/0.160            up    up   inet     198.19.0.241/30
lt-1/0/0.161            up    up   inet     198.19.0.245/30
lt-1/0/0.162            up    up   inet     198.19.0.249/30
lt-1/0/0.163            up    up   inet     198.19.0.253/30     /* 64th circuit */
lt-1/0/0.164            up    down inet     198.19.1.1/30       /* 65th circuit */
lt-1/0/0.165            up    down inet     198.19.1.5/30
lt-1/0/0.166            up    down inet     198.19.1.9/30
lt-1/0/0.167            up    down inet     198.19.1.13/30

The service IFL is down:

> show interfaces lt-1/0/0.164
  Logical interface lt-1/0/0.164 (Index 453) (SNMP ifIndex 4008)
    Flags: Hardware-Down Up SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.164 ]  Encapsulation: LT-VLAN
    Input packets : 0
    Output packets: 0
    Protocol inet, MTU: 9174
    Max nh cache: 0, New hold nh limit: 0, Curr nh cnt: 0, Curr new hold cnt: 0, NH drop cnt: 0
      Flags: Sendbcast-pkt-to-re
      Addresses, Flags: Dest-route-down Is-Preferred Is-Primary
        Destination: 198.19.1.0/30, Local: 198.19.1.1, Broadcast: 198.19.1.3

The interface state is propagated from the service IFL to the transport IFL (the peer unit):

> show interfaces lt-1/0/0.1164
  Logical interface lt-1/0/0.1164 (Index 454) (SNMP ifIndex 4009)
    Flags: Up Link-Layer-Down SNMP-Traps 0x4000 VLAN-Tag [ 0x8100.164 ]  Encapsulation: VLAN-CCC
    Input packets : 0
    Output packets: 0
    Protocol ccc, MTU: 9192

The corresponding PW is inactive:

> show l2circuit connections interface lt-1/0/0.1164
Layer-2 Circuit Connections:

Legend for connection status (St)
EI -- encapsulation invalid      NP -- interface h/w not present
MM -- mtu mismatch               Dn -- down
EM -- encapsulation mismatch     VC-Dn -- Virtual circuit Down
CM -- control-word mismatch      Up -- operational
VM -- vlan id mismatch           CF -- Call admission control failure
OL -- no outgoing label          IB -- TDM incompatible bitrate
NC -- intf encaps not CCC/TCC    TM -- TDM misconfiguration
BK -- Backup Connection          ST -- Standby Connection
CB -- rcvd cell-bundle size bad  SP -- Static Pseudowire
LD -- local site signaled down   RS -- remote site standby
RD -- remote site signaled down  HS -- Hot-standby Connection
XX -- unknown

Legend for interface status
Up -- operational
Dn -- down
Neighbor: 198.18.255.2
    Interface                 Type  St     Time last up          # Up trans
    lt-1/0/0.1164(vc 1164)    rmt    LD

Solution

To overcome this limit, change the allocation mode from "per-IFL" to "per-interface (IFD)".There will be only a single MAC-address used across all service IFLs:

[edit interfaces lt-1/0/0]
+    logical-tunnel-options {
+        per-unit-mac-disable;
+    }

The service IFLs are all active now:

> show interfaces lt-1/0/0 terse | match "lt-1/0/0.16[0-7]"
lt-1/0/0.160            up    up   inet     198.19.0.241/30
lt-1/0/0.161            up    up   inet     198.19.0.245/30
lt-1/0/0.162            up    up   inet     198.19.0.249/30
lt-1/0/0.163            up    up   inet     198.19.0.253/30
lt-1/0/0.164            up    up    inet     198.19.1.1/30
lt-1/0/0.165            up    up    inet     198.19.1.5/30
lt-1/0/0.166            up    up    inet     198.19.1.9/30
lt-1/0/0.167            up    up    inet     198.19.1.13/30


All the IFLs share the same MAC-address:

> show interfaces lt-1/0/0.16[0-7] | match "(Logical|MAC)"
  Logical interface lt-1/0/0.160 (Index 436) (SNMP ifIndex 4000)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.161 (Index 438) (SNMP ifIndex 4002)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.162 (Index 440) (SNMP ifIndex 4004)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.163 (Index 442) (SNMP ifIndex 4006)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.164 (Index 444) (SNMP ifIndex 4008)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.165 (Index 446) (SNMP ifIndex 4010)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.166 (Index 448) (SNMP ifIndex 4012)
    MAC: 02:24:dc:78:69:4a
  Logical interface lt-1/0/0.167 (Index 450) (SNMP ifIndex 4020)
    MAC: 02:24:dc:78:69:4a

Related Information