Description

The flow-table-size statement under [chassis fpc slot-number inline-services] specifies the size of hash tables for inline services sampling.  

In some circumstances, reconfiguring flow-table-size might not take effect. This article explains the possible reason and how to fix the issue.

Symptoms

Case 1:
The initial configuration is shown below:

chassis {
    fpc 0 {
        sampling-instance jflow;
        inline-services {
            flow-table-size {
                ipv4-flow-table-size 15 ;
            }
        }
    }
}

Check inline-jflow status, ipv4-flow-table-size is with the expected value:

user@MX> show services accounting status inline-jflow fpc-slot 0   
  Status information
    FPC Slot: 0
    IPV4 export format: Version9, IPV6 export format: Not set
    VPLS export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 1704, IPv6 Route Record Count: 0, MPLS Route Record Count: 0
    Route Record Count: 1704, AS Record Count: 2
    Route-Records Set: Yes, Config Set: Yes
    Service Status: PFE-0: Steady
    Using Extended Flow Memory?: PFE-0: No PFE-1: No
    Flex Flow Sizing ENABLED?: PFE-0: No PFE-1: No
    IPv4 MAX FLOW Count: 5239800 , IPv6 MAX FLOW Count: 1024  <-- it's correct
    VPLS MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024

Delete the flow-table-size configuration and re-configure it immediately:

user@MX# delete chassis fpc 0 inline-services flow-table-size ipv4-flow-table-size

[edit]
user@MX# commit
commit complete

[edit]
user@MX# set chassis fpc 0 inline-services flow-table-size ipv4-flow-table-size 15

[edit]
user@MX# commit
commit complete

The flow table size value should be set to 15, but it is still be default value:

user@MX# run show services accounting status inline-jflow fpc-slot 0
  Status information
    FPC Slot: 0
    IPV4 export format: Version9, IPV6 export format: Not set
    BRIDGE export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 38, IPv6 Route Record Count: 0, MPLS Route Record Count: 0
    Route Record Count: 38, AS Record Count: 1
    Route-Records Set: Yes, Config Set: Yes
    Service Status: PFE-0: Steady
    Using Extended Flow Memory?: PFE-0: No
    Flex Flow Sizing ENABLED?: PFE-0: No
    IPv4 MAX FLOW Count: 1024, IPv6 MAX FLOW Count: 1024 <<-- it's incorrect
    BRIDGE MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024

Check messages log, might see logs as below:

user@MX# run show log messages | match "flow_table" 
Oct 18 00:02:25  MX fpc0 jnh_services_configure_flow_table_sizes(555): Inst(0) Flow Table Configuration is changed
Oct 18 00:02:25  MX fpc0 jnh_services_jflow_table_cfg_event(942): Inst(0) Processing event(1) state(0)
Oct 18 00:02:59  MX fpc0 jnh_services_configure_flow_table_sizes(551): Service is already in Reconfiguring State.Could not toggle use-extended-flow-memory or change flow-table-size now. please try later.inst(0)
Oct 18 00:03:57  MX fpc0 jnh_services_jflow_table_cfg_event(942): Inst(0) Processing event(2) state(1)
Oct 18 00:03:59  MX fpc0 jnh_services_jflow_table_cfg_event(942): Inst(0) Processing event(3) state(2)
Oct 18 00:03:59  MX fpc0 jnh_services_jflow_table_cfg_event(1007): Inst(0) Disabling AGER and EXPORT thread
Oct 18 00:04:09  MX fpc0 jnh_services_jflow_table_cfg_event(942): Inst(0) Processing event(4) state(3)
Oct 18 00:04:09  MX fpc0 jnh_services_jflow_table_cfg_event(1028): Inst(0) re-configuring Jflow Hash Table
Oct 18 00:04:09  MX fpc0 jnh_services_flow_table_free(597): Inst(0) Freeing flow table
Oct 18 00:04:09  MX fpc0 jnh_services_calculate_jflow_table_size(1505): inst(0) configured flow table info v4: 0, v6: 0, BRIDGE: 0, mpls: 0, flex: 0, nentries: 8196
Oct 18 00:04:09  MX fpc0 jnh_services_calculate_jflow_table_size(1567): inst (0) Setting max flows  - v4_nentries: 1024 v6_nentries: 1024 bridge_nentries: 1024 mpls_nentries: 1024
Oct 18 00:04:09  MX fpc0 jnh_services_jflow_table_cfg_event(1043): Inst(0) Enabing AGER and EXPORT thread
Oct 18 00:04:14  MX fpc0 jnh_services_jflow_table_cfg_event(942): Inst(0) Processing event(5) state(4)

Case 2:
The initial configuration is shown below:

chassis {
    fpc 0 {
        sampling-instance jflow;
        inline-services {
            flow-table-size {
                ipv4-flow-table-size 150 ;
            }
        }
    }
}

Check inline-jflow status, ipv4-flow-table-size is 1024, which is not the configured value:

user@MX> show services accounting status inline-jflow fpc-slot 0   
  Status information
    FPC Slot: 0
    IPV4 export format: Version9, IPV6 export format: Not set
    VPLS export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 1024, IPv6 Route Record Count: 0, MPLS Route Record Count: 0
    Route Record Count: 1024, AS Record Count: 2
    Route-Records Set: Yes, Config Set: Yes
    Service Status: PFE-0: Steady
    Using Extended Flow Memory?: PFE-0: No PFE-1: No
    Flex Flow Sizing ENABLED?: PFE-0: No PFE-1: No
    IPv4 MAX FLOW Count: 1024 , IPv6 MAX FLOW Count: 1024  <-- it's incorrect
    VPLS MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024

Check messages log, might see logs as below:

user@MX# run show log messages | match "table_size" 
Nov  6 08:56:22.345  MX fpc0 jnh_services_calculate_hash_table_size(7394): Configured Flow Table Size is higher than the supported value/chip, max supported table size is 48
Nov  6 08:56:22.345  MX fpc0 jnh_services_calculate_hash_table_size(7395): setting to default flow table size

Solution

Case 1:
The issue occurred because flow table ( flow-table-size ) is changed during inline jflow "Reconfiguring" state, so the new configured value has not taken effect.

Case 2:
The issue is due to flow table size being higher than the specific chip supported, so it sets to default.

Case 1:
The best practice is to always check the inline jflow state and make sure inline jflow is in Steady state, not commit  flow-table-size  configuration change in "Reconfiguring" state.

This KB article is also suitable for sideband ( use-extended-flow-memory ) reconfiguring same as flow table ( flow-table-size ) .

If this issue already exists, delete the configuration first, then commit configuration in steady  state.

The following sample is not ready to reconfigure:

user@MX> show services accounting status inline-jflow fpc-slot 0
Oct 18 00:02:51
  Status information
    FPC Slot: 0
    IPV4 export format: Version9, IPV6 export format: Not set
    BRIDGE export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 38, IPv6 Route Record Count: 0, MPLS Route Record Count: 0
    Route Record Count: 38, AS Record Count: 1
    Route-Records Set: Yes, Config Set: Yes
     Service Status: PFE-0: Reconfiguring
    Using Extended Flow Memory?: PFE-0: No
    Flex Flow Sizing ENABLED?: PFE-0: No
    IPv4 MAX FLOW Count: 1024, IPv6 MAX FLOW Count: 1024
    BRIDGE MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024

The following sample is ready to reconfigure:

user@MX> show services accounting status inline-jflow fpc-slot 0
  Status information
    FPC Slot: 0
    IPV4 export format: Version9, IPV6 export format: Not set
    BRIDGE export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 38, IPv6 Route Record Count: 0, MPLS Route Record Count: 0
    Route Record Count: 38, AS Record Count: 1
    Route-Records Set: Yes, Config Set: Yes
    Service Status: PFE-0: Steady
    Using Extended Flow Memory?: PFE-0: No
    Flex Flow Sizing ENABLED?: PFE-0: No
    IPv4 MAX FLOW Count: 1024, IPv6 MAX FLOW Count: 1024
    BRIDGE MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024

Case 2:
Since there are various FPCs, the configurable units range is 1.. 245. The solution is quite straightforward. It is to reduce the flow table size to a supported number for a specific FPC.

user@MX# set chassis fpc 0 inline-services flow-table-size ?
Possible completions:
+ apply-groups         Groups from which to inherit configuration data
+ apply-groups-except  Don't inherit configuration data from these groups
  ipv4-flow-table-size  IPv4 flow table size in units of 256k (1..245)
  ipv6-extended-attrib  Enable extended flow attribute
  ipv6-flow-table-size  IPv6 flow table size in units of 256k (1..245)
  mpls-flow-table-size  MPLS flow table size in units of 256k (1..245)
  vpls-flow-table-size  VPLS flow table size in units of 256k (1..245)

Modification History

2019-11-15: Added case 2.

Related Information