"TTI TCAM Full" error when adding/deleting Q-in-Q configuration

Description

The ternary content addressable memory (TCAM) Q-in-Q rule limit on a EX4500 is limited to 509 entries. Exceeding this limit will result in the TTI TCAM full message.

Symptoms

Seeing "TTI TCAM Full" messages in the logs when adding and deleting a Q-in-Q (also known as QinQ) configuration to an existing configuration using a single commit. Why?

Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_msg_handler,601: route process failed Oct 23 13:46:13 EXEX4500-77 master Device 0, TTI TCAM full! for cvlan tag range min 1751,cvlan tag range max 1751, on port 32 Oct 23 13:46:13 EXEX4500-77 master MRVL-L2:mrvl_vlan_class_port_set(),5677:Customer-range/priority-tag/mapping <tag> pushPort CVID Rule set failed(3) on port ge-0/0/8 Oct 23 13:46:13 EXEX4500-77 master MRVL-L2:mrvl_brg_port_vmem_create(),10589:vlan_class_port_set () Failed(:3) Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_entry_add_msg_proc,2793: l2_halp_vectors->l2_entry_create failed Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_entry_add_msg_proc,2886: proto VLAN membership,len 144 prefix 00074:native-vlan:QinQ customer-range: 1751-1751 nh 74 Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_msg_handler,601: route process failed Oct 23 13:46:13 EXEX4500-77 master Device 0, TTI TCAM full! for cvlan tag range min 1752,cvlan tag range max 1752, on port 32 Oct 23 13:46:13 EXEX4500-77 master MRVL-L2:mrvl_vlan_class_port_set(),5677:Customer-range/priority-tag/mapping <tag> pushPort CVID Rule set failed(3) on port ge-0/0/8 Oct 23 13:46:13 EXEX4500-77 master MRVL-L2:mrvl_brg_port_vmem_create(),10589:vlan_class_port_set () Failed(:3) Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_entry_add_msg_proc,2793: l2_halp_vectors->l2_entry_create failed Oct 23 13:46:13 EXEX4500-77 master RT-HAL,rt_entry_add_msg_proc,2886: proto VLAN membership,len 144 prefix 00074:native-vlan:QinQ customer-range: 1752-1752 nh 74

Solution

When using a single commit to process a delete and an add, the Q-in-Q rules process the " add" first and then the " delete" .

In the example below, the total number of EX4500 Q-in-Q rules exceeds the TCAM limit of 509.

Example:

An EX4500 has 2 vlans; vlan1790 and vlan 1791. The initial TCAM contains 506 entries.
The customer wants to remove 43 vlans and add 38 different vlans. They issue the delete command, the add command, and then the commit; yet they get the TTI TCAM message in the log. ( starting value:506 + new vlans:38 =544 entries)
They try to rollback, and continue to get the TTI TCAM message in the log.

The recommendation is to commit the delete and add commands separately. Issue the delete and commit commands first; and then add and commit commands second.

At any time, use the " show shim tti qinq " command to check the TCAM utilization in the PFE. (Note: the following example is for checking the Q-in-Q rules for access ports on FPC0 device 0)

root@EX4500-77:RE:0% cprod -A fpc0 -c "show shim tti qinq dev 0" TCAM-Index Rule-Flag PCL-ID Trunk/Port [Un]Tagged VLAN-Tag-Range Action/S-VID Total packet hits ========================================================================================================= 3584 PUSH 3 Port 23 Tagged 0 1790 0 3585 RANGE 3 Port 23 Tagged 1610 - 1610 1790 0 3586 RANGE 3 Port 23 Tagged 1731 - 1731 1790 0 3587 RANGE 3 Port 23 Tagged 1751 - 1751 1790 0 3588 RANGE 3 Port 23 Tagged 1752 - 1752 1790 0 3589 RANGE 3 Port 23 Tagged 1753 - 1753 1790 0 3590 RANGE 3 Port 23 Tagged 1754 - 1754 1790 0 3591 RANGE 3 Port 23 Tagged 1755 - 1755 1790 0 3592 RANGE 3 Port 23 Tagged 1757 - 1757 1790 0 3593 RANGE 3 Port 23 Tagged 1758 - 1758 1790 0 3594 RANGE 3 Port 23 Tagged 1759 - 1759 1790 0

To view the TCAM count, enter the following command. Notice, in this example the count exceeds the limit. The results of the output can help with the planning of the add / delete configuration.

root@EX4500-77:RE:0% cprod -A fpc0 -c "show shim tti qinq dev 0" | wc 510 5544 47274