On Aloha HQoS MPC cards (MPC2E NG HQoS and MPC3E NG HQoS), IFDs are seen to be missing in the show chassis hardware output after committing the ingress-and-egress knob on the PICs.
show chassis hardware
ingress-and-egress
This article explains why the IFDs go missing and the Aloha HQoS MPC fails to allocate loopback streams, and provides a recommendation on the ideal configuration to resolve this issue.
Note: MPC reboots automatically upon commit to add or delete the ingress-and-egress knob.
This behavior can be observed as follows:
User@mx240> show chassis hardware Hardware inventory: Item Version Part number Serial number Description Chassis JN1267EE7AFC MX240 Midplane REV 39 750-047865 ACRK4054 Enhanced MX240 Backplane FPM Board REV 04 760-059207 CAJW7791 Front Panel Display PEM 0 Rev 01 740-063046 QCS1746U01V PS 1.4-2.52kW; 90-264V AC in PEM 1 Rev 01 740-063046 QCS1744U1FV PS 1.4-2.52kW; 90-264V AC in Routing Engine 0 REV 14 740-031116 9016122650 RE-S-1800x4 CB 0 REV 05 750-062577 CAJL4747 Enhanced MX SCB FPC 1 REV 15 750-054903 CAFY2182 MPC2E NG HQoS CPU REV 12 711-045719 CAFS2278 RMPC PMB MIC 0 REV 26 750-028392 CAAE6165 3D 20x 1GE(LAN) SFP PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 02 740-011613 PHE5YBK SFP-SX Xcvr 1 REV 01 740-031851 AM152428WSF SFP-SX Xcvr 2 REV 01 740-031851 AM152428QUP SFP-SX Xcvr 3 REV 01 740-031851 AM1237SYVUA SFP-SX Xcvr 4 REV 01 740-031851 AM152428WCV SFP-SX Xcvr 5 REV 01 740-031851 AM152428QUY SFP-SX Xcvr 6 REV 01 740-031851 AM152428WRF SFP-SX Xcvr 7 REV 01 740-031851 PP10PWM SFP-SX Xcvr 8 REV 01 740-007326 P5S0P6M SFP-SX Xcvr 9 REV 01 740-031851 AM152428WS9 SFP-SX PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 01 740-011782 PC30N6E SFP-SX Xcvr 1 REV 01 740-031851 PQ34R0C SFP-SX Xcvr 2 REV 01 740-031851 AM152428QUL SFP-SX Xcvr 3 REV 01 740-031851 PND7YMA SFP-SX Xcvr 4 REV 01 740-031851 PM75SMY SFP-SX Xcvr 5 REV 02 740-011613 PNR0Q79 SFP-SX Xcvr 6 REV 01 740-031851 PM75VUM SFP-SX Xcvr 7 REV 02 740-011613 PKC0WHW SFP-SX Xcvr 8 REV 02 740-011613 AM152629CDD SFP-SX Xcvr 9 REV 01 740-031851 AM152428QUQ SFP-SX MIC 1 REV 27 750-028392 CABA7666 3D 20x 1GE(LAN) SFP PIC 2 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 02 740-011613 PGL2LGR SFP-SX Xcvr 1 REV 01 740-031851 AM152428QTG SFP-SX Xcvr 2 REV 01 740-031851 AM152428REH SFP-SX Xcvr 3 REV 01 740-031851 PMS60YK SFP-SX Xcvr 4 REV 01 740-031851 PP10NDU SFP-SX Xcvr 5 REV 02 740-011613 PNQ5S3Q SFP-SX Xcvr 6 REV 01 740-031851 PQ34TDW SFP-SX Xcvr 7 REV 01 740-011613 PBN09TW SFP-SX Xcvr 8 REV 02 740-011613 PJH259D SFP-SX Xcvr 9 REV 01 740-031851 AM152428RET SFP-SX PIC 3 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 01 740-011782 P9B0PUY SFP-SX Xcvr 1 REV 01 740-031851 AM152629EXJ SFP-SX Xcvr 2 REV 02 740-011613 PNR0PNP SFP-SX Xcvr 3 REV 02 740-011613 PU800P8 SFP-SX Xcvr 4 REV 02 740-011613 PHH68MV SFP-SX Xcvr 5 REV 01 740-011782 P8J2HLX SFP-SX Xcvr 6 REV 02 740-011613 PJH25R7 SFP-SX Xcvr 7 REV 01 740-031851 PP10R73 SFP-SX Xcvr 8 REV 01 740-031851 AM1045SU7TE SFP-SX Xcvr 9 REV 01 740-011782 PAN53TE SFP-SX Fan Tray 0 REV 02 711-059360 CAJJ5284 Enhanced Fan Tray
The ingress-and-egress knob is enabled on PIC 0 , 1 , 2 , and 3 .
User@mx240# show |compare [edit chassis] + fpc 1 { + pic 0 { + traffic-manager { + mode ingress-and-egress; + } + max-queues-per-interface 4; + } + pic 1 { + traffic-manager { + mode ingress-and-egress; + } + max-queues-per-interface 4; + } + pic 2 { + traffic-manager { + mode ingress-and-egress; + } + max-queues-per-interface 4; + } + pic 3 { + traffic-manager { + mode ingress-and-egress; + } + max-queues-per-interface 4; + } + } [edit] User@mx240# commit commit complete User@mx240> show configuration chassis fpc 1 { pic 0 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } pic 1 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } pic 2 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } pic 3 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } }
Only 10 SFPs under PIC 0 show up but the others do not.
FPC 1 REV 15 750-054903 CAFY2182 MPC2E NG HQoS CPU REV 12 711-045719 CAFS2278 RMPC PMB MIC 0 REV 26 750-028392 CAAE6165 3D 20x 1GE(LAN) SFP PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 02 740-011613 PHE5YBK SFP-SX Xcvr 1 REV 01 740-031851 AM152428WSF SFP-SX Xcvr 2 REV 01 740-031851 AM152428QUP SFP-SX Xcvr 3 REV 01 740-031851 AM1237SYVUA SFP-SX Xcvr 4 REV 01 740-031851 AM152428WCV SFP-SX Xcvr 5 REV 01 740-031851 AM152428QUY SFP-SX Xcvr 6 REV 01 740-031851 AM152428WRF SFP-SX Xcvr 7 REV 01 740-031851 PP10PWM SFP-SX Xcvr 8 REV 01 740-007326 P5S0P6M SFP-SX Xcvr 9 REV 01 740-031851 AM152428WS9 SFP-SX PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP MIC 1 REV 27 750-028392 CABA7666 3D 20x 1GE(LAN) SFP PIC 2 BUILTIN BUILTIN 10x 1GE(LAN) SFP PIC 3 BUILTIN BUILTIN 10x 1GE(LAN) SFP Fan Tray 0 REV 02 711-059360 CAJJ5284 Enhanced Fan Tray
The following error messages indicate that resources could not be allocated for the last several ge-1/1/* ports.
*** messages *** Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 1 (Unknown) on IF command 3 (IFD add) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 226 (IFD HW aconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 15 (IFD aconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 13 (IFD mconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 151 (Ether set uint32) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 163 (Unknown) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 275 (Unknown) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 1 (Unknown) on IF command 3 (IFD add) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 226 (IFD HW aconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 15 (IFD aconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 13 (IFD mconfig) Jun 6 18:00:55.765 mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean) ############ ge-1/1/4 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ###### Jun 6 18:01:07.643 mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/4 phy-port 14 offset 14 Jun 6 18:01:07.743 mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 Jun 6 18:01:07.915 mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:14 Jun 6 18:01:08.075 mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/4 - ingress queueing traffic stream attach failed. Jun 6 18:01:08.205 mx240 fpc1 cos-halp: setup ifd:240, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes Jun 6 18:01:09.742 mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/4 EGRESS Jun 6 18:01:11.536 mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/4. Ingress queueing 1 ############ ge-1/1/5 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ###### Jun 6 18:01:11.663 mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/5 phy-port 15 offset 15 Jun 6 18:01:11.763 mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 Jun 6 18:01:11.935 mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:15 Jun 6 18:01:12.095 mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/5 - ingress queueing traffic stream attach failed. Jun 6 18:01:12.224 mx240 fpc1 cos-halp: setup ifd:241, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes Jun 6 18:01:13.762 mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/5 EGRESS Jun 6 18:01:15.555 mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/5. Ingress queueing 1 ############ ge-1/1/6 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ###### Jun 6 18:01:20.521 mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/6 phy-port 16 offset 16 Jun 6 18:01:20.622 mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 Jun 6 18:01:20.793 mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:16 Jun 6 18:01:20.954 mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/6 - ingress queueing traffic stream attach failed. Jun 6 18:01:21.083 mx240 fpc1 cos-halp: setup ifd:242, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes Jun 6 18:01:22.620 mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/6 EGRESS Jun 6 18:01:24.414 mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/6. Ingress queueing 1 ############ ge-1/1/7 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ###### Jun 6 18:01:29.405 mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/7 phy-port 17 offset 17 Jun 6 18:01:29.480 mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 Jun 6 18:01:29.652 mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:17 Jun 6 18:01:29.813 mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/7 - ingress queueing traffic stream attach failed. Jun 6 18:01:29.942 mx240 fpc1 cos-halp: setup ifd:243, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes Jun 6 18:01:31.479 mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/7 EGRESS Jun 6 18:01:33.273 mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/7. Ingress queueing 1 ############ ge-1/1/8 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ###### Jun 6 18:01:38.239 mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/8 phy-port 18 offset 18 Jun 6 18:01:38.339 mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 Jun 6 18:01:38.511 mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:18 Jun 6 18:01:38.671 mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/8 - ingress queueing traffic stream attach failed. Jun 6 18:01:38.801 mx240 fpc1 cos-halp: setup ifd:244, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes Jun 6 18:01:40.338 mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/8 EGRESS Jun 6 18:01:42.132 mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/8. Ingress queueing 1
Only 10 ports are available under PIC 0 after committing the ingress-and-egress knob configuration on all the 4 PICs in the FPC. This is due to a hardware limitation on Aloha HQoS cards (MPC2E NG HQoS and MPC3E NG HQoS) for both the 80G and 130G models.
Ingress queuing can be enabled for a maximum of 10 ports per MIC slot, and 20 ports are enabled per Aloha line card with 10 ports per MIC slot. The XM chip has a resource called "loopback streams," which is shared among tunnel interfaces, inline services, ingress queuing, and so on. For ingress queuing, only 10 ports worth of loopback streams are reserved per MIC.
When the configuration is set with PIC 0 and PIC 2 in FPC 1, all the ports are present in the output of the show chassis hardware command and available.
The above issue occurs when all the four PICs with 10 ports each have the configuration. If only 10 per MIC are configured, for example PIC 0 and PIC 2 or PIC 1 and PIC 3, the issue will not be seen.
User@mx240> show configuration chassis fpc 1 { pic 0 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } pic 2 { traffic-manager { mode ingress-and-egress; } max-queues-per-interface 4; } } network-services enhanced-ip; FPC 1 REV 15 750-054903 CAFY2182 MPC2E NG HQoS CPU REV 12 711-045719 CAFS2278 RMPC PMB MIC 0 REV 26 750-028392 CAAE6165 3D 20x 1GE(LAN) SFP PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 02 740-011613 PHE5YBK SFP-SX Xcvr 1 REV 01 740-031851 AM152428WSF SFP-SX Xcvr 2 REV 01 740-031851 AM152428QUP SFP-SX Xcvr 3 REV 01 740-031851 AM1237SYVUA SFP-SX Xcvr 4 REV 01 740-031851 AM152428WCV SFP-SX Xcvr 5 REV 01 740-031851 AM152428QUY SFP-SX Xcvr 6 REV 01 740-031851 AM152428WRF SFP-SX Xcvr 7 REV 01 740-031851 PP10PWM SFP-SX Xcvr 8 REV 01 740-007326 P5S0P6M SFP-SX Xcvr 9 REV 01 740-031851 AM152428WS9 SFP-SX PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 01 740-011782 PC30N6E SFP-SX Xcvr 1 REV 01 740-031851 PQ34R0C SFP-SX Xcvr 2 REV 01 740-031851 AM152428QUL SFP-SX Xcvr 3 REV 01 740-031851 PND7YMA SFP-SX Xcvr 4 REV 01 740-031851 PM75SMY SFP-SX Xcvr 5 REV 02 740-011613 PNR0Q79 SFP-SX Xcvr 6 REV 01 740-031851 PM75VUM SFP-SX Xcvr 7 REV 02 740-011613 PKC0WHW SFP-SX Xcvr 8 REV 02 740-011613 AM152629CDD SFP-SX Xcvr 9 REV 01 740-031851 AM152428QUQ SFP-SX MIC 1 REV 27 750-028392 CABA7666 3D 20x 1GE(LAN) SFP PIC 2 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 02 740-011613 PGL2LGR SFP-SX Xcvr 1 REV 01 740-031851 AM152428QTG SFP-SX Xcvr 2 REV 01 740-031851 AM152428REH SFP-SX Xcvr 3 REV 01 740-031851 PMS60YK SFP-SX Xcvr 4 REV 01 740-031851 PP10NDU SFP-SX Xcvr 5 REV 02 740-011613 PNQ5S3Q SFP-SX Xcvr 6 REV 01 740-031851 PQ34TDW SFP-SX Xcvr 7 REV 01 740-011613 PBN09TW SFP-SX Xcvr 8 REV 02 740-011613 PJH259D SFP-SX Xcvr 9 REV 01 740-031851 AM152428RET SFP-SX PIC 3 BUILTIN BUILTIN 10x 1GE(LAN) SFP Xcvr 0 REV 01 740-011782 P9B0PUY SFP-SX Xcvr 1 REV 01 740-031851 AM152629EXJ SFP-SX Xcvr 2 REV 02 740-011613 PNR0PNP SFP-SX Xcvr 3 REV 02 740-011613 PU800P8 SFP-SX Xcvr 4 REV 02 740-011613 PHH68MV SFP-SX Xcvr 5 REV 01 740-011782 P8J2HLX SFP-SX Xcvr 6 REV 02 740-011613 PJH25R7 SFP-SX Xcvr 7 REV 01 740-031851 PP10R73 SFP-SX Xcvr 8 REV 01 740-031851 AM1045SU7TE SFP-SX Xcvr 9 REV 01 740-011782 PAN53TE SFP-SX Fan Tray 0 REV 02 711-059360 CAJJ5284 Enhanced Fan Tray
2019-11-19: Technical publication link added to Related Links for reference