Description

This articles provides general information about scaling ratio in IP fabric

Symptoms

Scaling up the number of ports in a fabric is accomplished by adjusting the width of the spine and the oversubscription ration.

To increase the overall throughput of an IP-fabric, you simply need to increase the number of spine devices (and the appropriate uplinks from the leaf nodes to those spine devices).

 

Adding and removing both server-facing ports (downlinks from leaf nodes) and spine nodes will affect the oversubscription ration of a fabric. For example, a line rate forwarding would equate to 1to1 (1:1), meaning the aggregate server facing bandwidth is equal to the aggregate uplink bandwidth.

As for a 3:1 oversubscription, that is, the aggregate server-facing bandwidth is x3 times that of the aggregate uplink bandwidth.

Solution

The picture shows a basic 3:1 oversubscription IP fabric:



  • All spine nodes, four in total, are routers (Layer 3 switches) that each have (32) 40GbE interfaces.
  • All leaf nodes, 32 in total, are routers (Layer 3 switches) that have (6) 40GbE uplink interfaces and (48) 10GbE server-facing interfaces.
  • Each of the (48) 10GbE ports for all 32 spine nodes are fully utilized (that is, attached to downstream servers). So, the total server-facing bandwidth is 48 x 32 x 10 Gbps, which equals 15,360 Gbps.
  • Each of the 32 leaf nodes has (4) 40GbE spine-facing interfaces. So, the total uplink bandwidth is 4 x 32 x 40 Gbps, which equals 5120 Gbps. The oversubscription ratio for this fabric is 15360:5120 or 3:1.

Modification History

2025-02-25 : Article Created