RED drop on Juniper due to RSVP Interoperability issue with Cisco.
Limitation on Cisco: To signal Penultimate Hop Popping (PHP), a Cisco egress router sends label 0 to the penultimate hop in the control plane, whereas a Juniper Networks egress router sends label 3 (implicit NULL). If traffic rate increases beyond 25-30% approx. RED (Random Early Detection) drops are seen on the Juniper router due to Pause frames coming from Cisco.
Agilent-----Cisco-3845----Att-M320----Att-640------Agilent <----------Traffic flow------ Transmit Rate: 279.53MB/S
There are two possible solutions to stop RED drop on a Juniper router.
Configure no-flow-control .
no-flow-control
Use mpls traffic-eng signalling advertise implicit-null on Cisco.
mpls traffic-eng signalling advertise implicit-null
Juniper is receiving MAC pause frames from Cisco. Since Cisco router is not able to process packets due to high incoming traffic.
show interfaces ge-6/1/0 extensive <...> MAC statistics: Receive Transmit Total octets 7698137040 814273350791 Total packets 120074767 12090571383 Unicast packets 30165 12090535886 Broadcast packets 24 58 Multicast packets 120044578 35439 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 120025971 0 MAC pause frames 120025971 0
MAC pause frames are increasing on the Juniper router.
Juniper@ATT-M320-Re0# run show interfaces queue ge-6/1/0 Physical interface: ge-6/1/0, Enabled, Physical link is Up Interface index: 202, SNMP ifIndex: 153 Description: Connected to CISCO-3845 ge-0/0 Forwarding classes: 16 supported, 4 in use Egress queues: 4 supported, 4 in use Queue: 0, Forwarding classes: best-effort Queued: Packets : 9162921 538987 pps Bytes : 531449418 250089968 bps Transmitted: Packets : 8462011 497400 pps Bytes : 490796638 230793600 bps Tail-dropped packets : 118476 7097 pps RED-dropped packets : 700441 42160 pps Low, non-TCP : 700441 42160 pps Low, TCP : 0 0 pps High, non-TCP : 0 0 pps High, TCP : 0 0 pps RED-dropped bytes : 44828224 21585920 bps Low, non-TCP : 44828224 21585920 bps Low, TCP : 0 0 bps High, non-TCP : 0 0 bps High, TCP : 0 0 bps
CISCO-3845(config)#do sh int g0/0 GigabitEthernet0/0 is up, line protocol is up Hardware is BCM1125 Internal MAC, address is 0016.c8e6.9fb0 (bia 0016.c8e6.9fb0) Description: "**Connection to Node-1 LAN Network**" Internet address is 192.168.10.214/30 MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, reliability 255/255, txload 1/255, rxload 64/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, link type is autonegotiation, media type is SX output flow-control is XON, input flow-control is XON ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:02, output 00:00:01, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 254280000 bits/sec, 496634 packets/sec 5 minute output rate 1000 bits/sec, 0 packets/sec 591814268 packets input, 1099031276 bytes, 0 no buffer Received 47685 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 47625 multicast, 0 pause input 0 input packets with dribble condition detected 432000 packets output, 56708628 bytes, 0 underruns
No packet drops are seen on the Cisco router.
ge-6/1/0 { gigether-options { no-flow-control; }
No RED Drops are seen anymore
Tail-dropped packets : 0 0 pps RED-dropped packets : 0 0 pps
Now Cisco will drop packets on the incoming interface.
CISCO-3845#sh int g0/0 GigabitEthernet0/0 is up, line protocol is up Hardware is BCM1125 Internal MAC, address is 0016.c8e6.9fb0 (bia 0016.c8e6.9fb0) Description: "**Connection to Node-1 LAN Network**" Internet address is 192.168.10.214/30 MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, reliability 255/255, txload 1/255, rxload 43/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, link type is autonegotiation, media type is SX output flow-control is XON, input flow-control is XON ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: random early detection(RED) 5 minute input rate 170522000 bits/sec, 333042 packets/sec 5 minute output rate 2000 bits/sec, 0 packets/sec 1042618197 packets input, 1176338647 bytes, 0 no buffer Received 52896 broadcasts, 1 runts, 0 giants, 0 throttles 3142573 input errors, 1 CRC, 0 frame, 3142571 overrun, 0 ignored 0 watchdog, 4473 multicast, 0 pause input
PHP Router: lab@ATT-M320-Re0# run show mpls lsp Ingress LSP: 1 sessions To From State Rt ActivePath P LSPname 11.11.11.11 1.1.1.1 Up 0 * to-cisco Total 1 displayed, Up 1, Down 0 Transit LSP: 2 sessions To From State Rt Style Labelin Labelout LSPname 9.9.9.9 11.11.11.11 Up 0 1 SE 164208 3 Tunnel-to-Juniper-T640 11.11.11.11 9.9.9.9 Up 0 1 FF 164400 0 to-cisco3845 Total 2 displayed, Up 2, Down Default Label from Cisco is 0
lab@ATT-M320-Re0# run show interfaces ge-6/1/0 extensive Physical interface: ge-6/1/0, Enabled, Physical link is Up <...> MAC statistics: Receive Transmit Total octets 7698137040 814273350791 Total packets 120074767 12090571383 Unicast packets 30165 12090535886 Broadcast packets 24 58 Multicast packets 120044578 35439 CRC/Align errors 0 0 FIFO errors 0 0 MAC control frames 120025971 0 MAC pause frames 120025971 0 Oversized frames 0
CISCO-3845(config)#do sh proc cpu his 999999999999999999999999999999999999999999999999999999999999 999999999999999999999999999999999999999999999999999999999999 100 ************************************************************ 90 ************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ 0....5....1....1....2....2....3....3....4....4....5....5....6
CISCO-3845(config)#do sh int g0/0 GigabitEthernet0/0 is up, line protocol is up Hardware is BCM1125 Internal MAC, address is 0016.c8e6.9fb0 (bia 0016.c8e6.9fb0) Description: "**Connection to Node-1 LAN Network**" Internet address is 192.168.10.214/30 MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, reliability 255/255, txload 1/255, rxload 64/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, link type is autonegotiation, media type is SX output flow-control is XON, input flow-control is XON ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:02, output 00:00:01, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 254280000 bits/sec, 496634 packets/sec 5 minute output rate 1000 bits/sec, 0 packets/sec 591814268 packets input, 1099031276 bytes, 0 no buffer Received 47685 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 47625 multicast, 0 pause input 0 input packets with dribble condition detected 432000 packets output, 56708628 bytes, 0 underruns 3 output errors, 0 collisions, 3 interface resets 0 unknown protocol drops 0 unknown protocol drops 0 babbles, 0 late collision, 0 deferred 3 lost carrier, 0 no carrier, 0 pause output 0 output buffer failures, 0 output buffers swapped out
CISCO-3845(config)# mpls traffic-eng signalling advertise implicit-nul lab@ATT-M320-Re0# run show mpls lsp Ingress LSP: 1 sessions To From State Rt ActivePath P LSPname 11.11.11.11 1.1.1.1 Up 0 * to-cisco Total 1 displayed, Up 1, Down 0 Transit LSP: 2 sessions To From State Rt Style Labelin Labelout LSPname 9.9.9.9 11.11.11.11 Up 0 1 SE 164208 3 Tunnel-to-Juniper-T640 11.11.11.11 9.9.9.9 Up 0 1 FF 164416 3 to-cisco3845
lab@ATT-M320-Re0# run show interfaces queue ge-6/1/0 Physical interface: ge-6/1/0, Enabled, Physical link is Up Interface index: 202, SNMP ifIndex: 153 Description: Connected to CISCO-3845 ge-0/0 Forwarding classes: 16 supported, 4 in use Egress queues: 4 supported, 4 in use Queue: 0, Forwarding classes: best-effort Queued: Packets : 2183817 546021 pps Bytes : 117926118 235881072 bps Transmitted: Packets : 2183818 546021 pps Bytes : 117926172 235881072 bps Tail-dropped packets : 0 0 ppsRED-dropped packets : 0 0 pps Low, non-TCP : 0 0 pps Low, TCP : 0 0 pps High, non-TCP : 0 0 pps High, TCP : 0 0 pps RED-dropped bytes : 0 0 bps Low, non-TCP : 0 0 bps Low, TCP : 0 0 bps High, non-TCP : 0 0 bps High, TCP : 0 0 bps lab@ATT-M320-Re0# run show interfaces ge-6/1/0 extensive Physical interface: ge-6/1/0, Enabled, Physical link is Up Interface index: 202, SNMP ifIndex: 153, Generation: 203 Description: **Connected to CISCO-3845 ge-0/0** Link-level type: Ethernet, MTU: 1514, Speed: 1000mbps, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online Device flags : Present Running Interface flags: SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 4 maximum usable queues Hold-times : Up 0 ms, Down 0 ms Current address: 00:17:cb:98:1c:73, Hardware address: 00:17:cb:98:1c:73 Last flapped : 2009-08-12 06:57:10 UTC (1d 06:35 ago) Statistics last cleared: 2009-08-13 13:23:49 UTC (00:09:06 ago) Traffic statistics: Input bytes : 107133 616 bps Output bytes : 16126516424 235890000 bps Input packets: 157 0 pps Output packets: 298638911 546041 pps
CISCO-3845(config)#do sh proc cpu his CISCO-3845 08:52:01 AM Thursday Aug 13 2009 UTC 888888888888888888888888888888888888888888888888888888888888 888888888888888888888888888888888999998888899999888888888888 100 90************************************************************ 80 ************************************************************ 70 ************************************************************ 60 ************************************************************ 50 ************************************************************ 40 ************************************************************ 30 ************************************************************ 20 ************************************************************ 10 ************************************************************ CISCO-3845(config)#do sh int g0/0 GigabitEthernet0/0 is up, line protocol is up Hardware is BCM1125 Internal MAC, address is 0016.c8e6.9fb0 (bia 0016.c8e6.9fb0) Description: Connection to Node-1 LAN Network Internet address is 192.168.10.214/30 MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec, reliability 255/255, txload 1/255, rxload 66/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s, link type is autonegotiation, media type is SX output flow-control is XON, input flow-control is XON ARP type: ARPA, ARP Timeout 04:00:00 Last input 00:00:02, output 00:00:00, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 258947000 bits/sec, 526305 packets/sec 5 minute output rate 3000 bits/sec, 0 packets/sec 786785715 packets input, 89016114 bytes, 0 no buffer Received 47872 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 47808 multicast, 0 pause input 0 input packets with dribble condition detected 432270 packets output, 56793795 bytes, 0 underruns 3 output errors, 0 collisions, 3 interface resets 0 unknown protocol drops 0 unknown protocol drops 0 babbles, 0 late collision, 0 deferred 3 lost carrier, 0 no carrier, 0 pause output 0 output buffer failures, 0 output buffers swapped out