Home » Dell Manuals » Hubs & Switches » Dell C9000 Line Cards » Manual Viewer

Dell C9000 Line Cards Networking Configuration Guide for the C9000 Series Vers - Page 387

Equal Cost Multi-Path (ECMP), ECMP for Flow-Based Affinity, Enabling Deterministic ECMP Next Hop

View all Dell C9000 Line Cards manuals

Add to My Manuals
Save this manual to your list of manuals

Page 387 highlights

14 Equal Cost Multi-Path (ECMP) Equal cost multi-path (ECMP) supports multiple paths in next-hop packet forwarding to a destination device. ECMP for Flow-Based Affinity ECMP for flow-based affinity includes link bundle monitoring. Enabling Deterministic ECMP Next Hop Deterministic ECMP next hop arranges all ECMPs in order before writing them into the content addressable memory (CAM). For example, suppose the RTM learns eight ECMPs in the order that the protocols and interfaces came up. In this case, the forwarding information base (FIB) and CAM sort them so that the ECMPs are always arranged. This implementation ensures that every chassis having the same prefixes orders the ECMPs the same. With eight or less ECMPs, the ordering is lexicographic and deterministic. With more than eight ECMPs, ordering is deterministic, but it is not in lexicographic order. To enable deterministic ECMP next hop, use the appropriate command. NOTE: Packet loss might occur when you enable ip/ipv6 ecmp-deterministic for the firsttime only. • Enable IPv4 Deterministic ECMP Next Hop. CONFIGURATION mode. ip ecmp-deterministic • Enable IPv6 Deterministic ECMP Next Hop. CONFIGURATION mode. ipv6 ecmp-deterministic Configuring the Hash Algorithm Seed Deterministic ECMP sorts ECMPs in order even though RTM provides them in a random order. However, the hash algorithm uses as a seed the lower 12 bits of the chassis MAC, which yields a different hash result for every chassis. This behavior means that for a given flow, even though the prefixes are sorted, two unrelated chassis can select different hops. The system provides a command line interface (CLI)-based solution for modifying the hash seed to ensure that on each configured system, the ECMP selection is same. When configured, the same seed is set for ECMP, LAG, and NH, and is used for incoming traffic only. NOTE: While the seed is stored separately on each port-pipe, the same seed is used across all CAMs. Equal Cost Multi-Path (ECMP) 387

Section	Page
Dell Networking Configuration Guide for the C9000 Series Version 9.9(0.0)	3
About this Guide	36
Audience	36
Conventions	36
Related Documents	36
Configuration Fundamentals	37
Accessing the Command Line	37
CLI Modes	37
Navigating CLI Modes	39
The do Command	42
Undoing Commands	43
Obtaining Help	44
Entering and Editing Commands	44
Command History	45
Filtering show Command Outputs	46
Multiple Users in Configuration Mode	47
Getting Started	49
Console Access	49
Serial Console	49
Mounting an NFS File System	50
Important Points to Remember	51
Default Configuration	52
Configuring a Host Name	52
Accessing the System Remotely	52
Accessing the System Remotely	53
Configure the Management Port IP Address	53
Configure a Management Route	53
Configuring a Username and Password	54
Configuring the Enable Password	54
Manage Configuration Files	54
File Storage	55
Copy Files to and from the System	55
Save the Running-Configuration	56
Configure the Overload Bit for a Startup Scenario	57
Viewing Files	57
Changes in Configuration Files	58
Viewing Command History	58
Upgrading the Dell Networking OS	59
Switch Management	60
Configuring Privilege Levels	60
Creating a Custom Privilege Level	60
Removing a Command from EXEC Mode	60
Moving a Command from EXEC Privilege Mode to EXEC Mode	60
Allowing Access to CONFIGURATION Mode Commands	61
Allowing Access to the Following Modes	61
Applying a Privilege Level to a Username	63
Applying a Privilege Level to a Terminal Line	63
Configuring Logging	63
Audit and Security Logs	64
Configuring Logging Format	65
Setting Up a Secure Connection to a Syslog Server	66
Track Login Activity	67
Restrictions for Tracking Login Activity	67
Configuring Login Activity Tracking	67
Display Login Statistics	68
Limit Concurrent Login Sessions	69
Restrictions for Limiting the Number of Concurrent Sessions	69
Configuring Concurrent Session Limit	69
Enabling the System to Clear Existing Sessions	69
Log Messages in the Internal Buffer	70
Configuration Task List for System Log Management	70
Disabling System Logging	71
Sending System Messages to a Syslog Server	71
Configuring a UNIX System as a Syslog Server	71
Display the Logging Buffer and the Logging Configuration	72
Changing System Logging Settings	72
Configuring a UNIX Logging Facility Level	73
Synchronizing Log Messages	74
Enabling Timestamp on Syslog Messages	75
File Transfer Services	75
Configuration Task List for File Transfer Services	76
Enabling the FTP Server	76
Configuring FTP Server Parameters	76
Configuring FTP Client Parameters	77
Terminal Lines	77
Denying and Permitting Access to a Terminal Line	77
Configuring Login Authentication for Terminal Lines	78
Setting Time Out of EXEC Privilege Mode	79
Using Telnet to Access Another Network Device	79
Lock CONFIGURATION Mode	80
Viewing the Configuration Lock Status	81
Recovering from a Forgotten Password	81
Ignoring the Startup Configuration and Booting from the Factory-Default Configuration	82
Recovering from a Failed Start	82
Restoring Factory-Default Settings	83
Important Point to Remember	83
Restoring Factory-Default Boot Environment Variables	83
Using Hashes to Verify Software Images Before Installation	85
Verifying System Images on C9010 Components	86
When System Images on C9010 Components Do Not Match	87
Manually Resetting the System Image on a C9010 Component	88
Logging in to the Virtual Console of a C9010 Component	88
Booting the C9010 from an Image on a Network Server	88
Configuring C9010 Components to Boot from the RPM CP Image	89
802.1X	90
The Port-Authentication Process	91
EAP over RADIUS	93
Configuring 802.1X	93
Related Configuration Tasks	93
Important Points to Remember	94
Enabling 802.1X	95
Configuring dot1x Profile	97
Configuring MAC addresses for a do1x Profile	98
Configuring the Static MAB and MAB Profile	98
Configuring Critical VLAN	99
Configuring Request Identity Re-Transmissions	100
Configuring a Quiet Period after a Failed Authentication	101
Forcibly Authorizing or Unauthorizing a Port	102
Re-Authenticating a Port	103
Configuring Dynamic VLAN Assignment with Port Authentication	104
Guest and Authentication-Fail VLANs	105
Configuring a Guest VLAN	106
Configuring an Authentication-Fail VLAN	107
Configuring Timeouts	108
Multi-Host Authentication	109
Configuring Multi-Host AuthenticationConfiguring Single-Host Authentication	111
Multi-Supplicant Authentication	112
Configuring Multi-Supplicant AuthenticationRestricting Multi-Supplicant Authentication	113
MAC Authentication Bypass	114
MAB in Single-host and Multi-Host Mode	114
MAB in Multi-Supplicant Authentication Mode	114
Configuring MAC Authentication Bypass	115
Dynamic CoS with 802.1X	116
Access Control Lists (ACLs)	118
IP Access Control Lists (ACLs)	118
CAM Usage	119
User-Configurable CAM Allocation	120
Allocating CAM for Ingress ACLs on the Port Extender	120
Allocating CAM for Egress ACLs on the Port Extender	122
Implementing ACLs	123
IP Fragment Handling	124
IP Fragments ACL Examples	125
Layer 4 ACL Rules Examples	125
Configure a Standard IP ACL	126
Configuring a Standard IP ACL Filter	127
Configure an Extended IP ACL	128
Configuring Filters with a Sequence Number	128
Configuring Filters Without a Sequence Number	130
Configure Layer 2 and Layer 3 ACLs	131
Using ACL VLAN Groups	131
Guidelines for Configuring ACL VLAN Groups	132
Configuring an ACL VLAN Group	132
Allocating ACL VLAN CAM	133
Applying an IP ACL to an Interface	134
Applying Ingress ACLs on the Port Extender	135
Applying Egress ACLs	135
Applying Layer 3 Egress ACLs on Control-Plane Traffic	136
Counting ACL Hits	137
IP Prefix Lists	137
Implementation Information	137
Configuration Task List for Prefix Lists	138
ACL Resequencing	141
Resequencing an ACL or Prefix List	142
Route Maps	143
Implementation Information	144
Important Points to Remember	144
Configuration Task List for Route Maps	144
Configuring Match Routes	146
Configuring Set Conditions	148
Configure a Route Map for Route Redistribution	149
Configure a Route Map for Route Tagging	150
Continue Clause	150
Configuring a UDF ACL	150
Hot-Lock Behavior	152
Bidirectional Forwarding Detection (BFD)	153
How BFD Works	153
BFD Packet Format	154
BFD Sessions	156
BFD Three-Way Handshake	156
Session State Changes	157
Important Points to Remember	158
Configure BFD	158
Configure BFD for Static Routes	159
Configure BFD for OSPF	160
Configure BFD for OSPFv3	164
Configure BFD for IS-IS	165
Configure BFD for BGP	168
Configure BFD for VRRP	174
Configuring Protocol Liveness	177
Border Gateway Protocol IPv4 (BGPv4)	178
Autonomous Systems (AS)	178
Sessions and Peers	180
Establish a Session	181
Route Reflectors	181
Communities	182
BGP Attributes	182
Best Path Selection Criteria	183
Weight	185
Local Preference	185
Multi-Exit Discriminators (MEDs)	186
Origin	187
AS Path	188
Next Hop	188
Multiprotocol BGP	188
Implement BGP	189
Additional Path (Add-Path) Support	189
Advertise IGP Cost as MED for Redistributed Routes	189
Ignore Router-ID for Some Best-Path Calculations	190
Four-Byte AS Numbers	190
AS4 Number Representation	190
AS Number Migration	192
BGP4 Management Information Base (MIB)	194
Important Points to Remember	194
Configuration Information	195
BGP Configuration	195
Enabling BGP	196
Configuring AS4 Number Representations	200
Configuring Peer Groups	201
Configuring BGP Fast Fail-Over	204
Configuring Passive Peering	206
Maintaining Existing AS Numbers During an AS Migration	207
Allowing an AS Number to Appear in its Own AS Path	208
Enabling Neighbor Graceful Restart	208
Filtering on an AS-Path Attribute	209
Regular Expressions as Filters	211
Redistributing Routes	212
Enabling Additional Paths	213
Configuring IP Community Lists	213
Configuring an IP Extended Community List	215
Filtering Routes with Community Lists	216
Manipulating the COMMUNITY Attribute	216
Changing MED Attributes	218
Changing the LOCAL_PREFERENCE Attribute	218
Changing the NEXT_HOP Attribute	219
Changing the WEIGHT Attribute	220
Enabling Multipath	220
Filtering BGP Routes	220
Filtering BGP Routes Using Route Maps	222
Filtering BGP Routes Using AS-PATH Information	222
Configuring BGP Route Reflectors	223
Aggregating Routes	224
Configuring BGP Confederations	224
Enabling Route Flap Dampening	225
Changing BGP Timers	228
Enabling BGP Neighbor Soft-Reconfiguration	228
Route Map Continue	229
Enabling MBGP Configurations	230
BGP Regular Expression Optimization	231
Debugging BGP	231
Storing Last and Bad PDUs	232
Capturing PDUs	233
PDU Counters	234
Sample Configurations	234
Content Addressable Memory (CAM)	244
CAM Allocation	244
Test CAM Usage	246
View CAM-ACL Settings	246
View CAM Usage	247
Return to the Default CAM Configuration	248
CAM Optimization	248
Applications for CAM Profiling	248
LAG HashingLAG Hashing Based on Bidirectional Flow	248
Unified Forwarding Table (UFT) Modes	249
Configuring UFT Modes	249
Control Plane Policing (CoPP)	251
CoPP Implementation	251
Protocol-based Control Plane Policing	251
Queue-based Control Plane Policing	251
CoPP Example	253
Configure Control Plane Policing	254
Configuring CoPP for Protocols	254
Examples of Configuring CoPP for Protocols	256
Configuring CoPP for CPU Queues	257
Examples of Configuring CoPP for CPU Queues	257
Displaying CoPP Configuration	258
Troubleshooting CoPP Operation	263
Enabling CPU Traffic Statistics	263
Viewing CPU Traffic Statistics	263
Troubleshooting CPU Packet Loss	264
Viewing Per-Protocol CoPP Counters	265
Viewing Per-Queue CoPP Counters	270
Data Center Bridging (DCB)	272
Enabling Data Center Bridging	272
Ethernet Enhancements in Data Center Bridging	272
Priority-Based Flow Control	274
Enhanced Transmission Selection	275
Data Center Bridging Exchange Protocol (DCBx)	276
Data Center Bridging in a Traffic Flow	277
QoS dot1p Traffic Classification and Queue Assignment	277
SNMP Support for PFC and Buffer Statistics Tracking	278
DCB Maps and its Attributes	279
DCB Map: Configuration Procedure	279
Important Points to Remember	279
Applying a DCB Map on a Port	280
Configuring PFC without a DCB Map	281
Configuring Lossless Queues	281
Applying a DCB Map on a Line Card	282
Data Center Bridging: Default Configuration	283
Configuration Notes: PFC and ETS in a DCB Map	283
PFC Configuration Notes	283
ETS Configuration Notes	284
ETS Prerequisites and Restrictions	285
Priority-Group Configuration Notes	286
Configuring Priority-Based Flow Control	286
Configuring Lossless Queues	287
Configuring Enhanced Transmission Selection	288
Creating an ETS Priority Group	288
ETS Operation with DCBx	289
Configure a DCBx Operation	290
DCBx Operation	290
DCBx Port Roles	290
DCB Configuration Exchange	292
Configuration Source Election	292
Propagation of DCB Information	293
Auto-Detection and Manual Configuration of the DCBx Version	293
Behavior of Tagged Packets	294
Configuration Example for DSCP and PFC Priorities	294
DCBx Example	295
DCBx Prerequisites and Restrictions	296
Configuring DCBx	297
Verifying the DCB Configuration	301
Performing PFC Using DSCP Bits Instead of 802.1p Bits	312
PFC and ETS Configuration Examples	312
Using PFC and ETS to Manage Data Center Traffic	313
PFC and ETS Configuration Command Examples	314
Using PFC and ETS to Manage Converged Ethernet Traffic	315
Hierarchical Scheduling in ETS Output Policies	315
Priority-Based Flow Control Using Dynamic Buffer Method	315
Pause and Resume of Traffic	316
Buffer Sizes for Lossless or PFC Packets	316
Configuring the Dynamic Buffer Method	317
Debugging and Diagnostics	319
Offline Diagnostics	319
Running Port Extender Offline Diagnostics on the Switch	319
Running Offline Diagnostics on a Standalone Switch	326
TRACE Logs	349
Auto Save on Reload, Crash, or Rollover	349
Uploading Trace Logs	349
Last Restart Reason	350
show hardware Commands	350
Environmental Monitoring	353
Displaying Port Extender Environment Information	354
Display Power Supply Status	354
Display Fan Status	355
Display Transceiver Type	355
Recognize an Over-Temperature Condition	357
Troubleshoot an Over-Temperature Condition	358
Troubleshooting Packet Loss	360
Displaying Drop Counters	361
Displaying Dataplane Statistics	362
Displaying Line-Card Counters	364
Accessing Application Core Dumps	364
Mini Core Dumps	365
Full Kernel Core Dumps	366
Enabling TCP Dumps	367
Accessing Port Extender Core and Mini Core Dumps	367
Dynamic Host Configuration Protocol (DHCP)	368
DHCP Packet Format and Options	368
Assign an IP Address using DHCP	370
Implementation Information	371
Configure the System to be a DHCP Server	372
Configuring the Server for Automatic Address Allocation	372
Specifying a Default Gateway	374
Configure a Method of Hostname Resolution	374
Using DNS for Address Resolution	374
Using NetBIOS WINS for Address Resolution	374
Creating Manual Binding Entries	374
Debugging the DHCP Server	375
Using DHCP Clear Commands	375
Configure the System to be a Relay Agent	375
Configure the System to be a DHCP Client	377
DHCP Client on a Management Interface	377
DHCP Client Operation with Other Features	378
Configure Secure DHCP	378
Option 82	379
DHCP Snooping	379
Drop DHCP Packets on Snooped VLANs Only	381
Dynamic ARP Inspection	382
Configuring Dynamic ARP Inspection	383
Source Address Validation	384
Enabling IP Source Address Validation	384
DHCP MAC Source Address Validation	385
Enabling IP+MAC Source Address Validation	385
Viewing the Number of SAV Dropped Packets	386
Clearing the Number of SAV Dropped Packets	386
Equal Cost Multi-Path (ECMP)	387
ECMP for Flow-Based Affinity	387
Enabling Deterministic ECMP Next Hop	387
Configuring the Hash Algorithm Seed	387
Link Bundle Monitoring	388
Managing ECMP Group Paths	388
Creating an ECMP Group Bundle	389
Modifying the ECMP Group Threshold	389
BGP Multipath Operation with Link Bankwidth	390
Dynamic Re-calculation of Link Bankwidth	392
Weighted ECMP for Static Routes	392
ECMP Support in L3 Host and LPM Tables	393
FCoE Transit	394
Fibre Channel over Ethernet	394
Ensure Robustness in a Converged Ethernet Network	394
FIP Snooping on Ethernet Bridges	396
FIP Snooping in a Switch Stack	398
Using FIP Snooping	398
FIP Snooping Prerequisites	398
Important Points to Remember	399
Enabling the FCoE Transit Feature	399
Enable FIP Snooping on VLANs	400
Configure the FC-MAP Value	400
Configure a Port for a Bridge-to-Bridge Link	400
Configure a Port for a Bridge-to-FCF Link	400
Impact on Other Software Features	400
FIP Snooping Restrictions	401
Configuring FIP Snooping	401
Displaying FIP Snooping Information	402
FCoE Transit Configuration Example	408
FIPS Cryptography	410
Configuration Tasks	410
Preparing the System	410
Enabling FIPS Mode	411
Generating Host-Keys	411
Monitoring FIPS Mode Status	412
Disabling FIPS Mode	412
Flex Hash and Optimized Boot-Up	414
Flex Hash Capability Overview	414
Configuring the Flex Hash Mechanism	414
LACP Fast Switchover	415
Configuring LACP Fast Switchover	415
LACP	415
LACP Fast Switchover	415
RDMA Over Converged Ethernet (RoCE) Overview	416
Sample Configurations	417
	417
Preserving 802.1Q VLAN Tag Value for Lite Subinterfaces	420
Force10 Resilient Ring Protocol (FRRP)	422
Protocol Overview	422
Ring Status	423
Multiple FRRP Rings	424
Important FRRP Points	424
Implementing FRRP	424
Important FRRP Concepts	425
FRRP Configuration	426
Creating the FRRP Group	426
Configuring the Control VLAN	427
Configuring and Adding the Member VLANs	428
Setting the FRRP Timers	429
Clearing the FRRP Counters	429
Viewing the FRRP Configuration	430
Viewing the FRRP Information	430
Troubleshooting FRRP	430
Configuration Checks	430
Sample Configuration and Topology	430
GARP VLAN Registration Protocol (GVRP)	433
Important Points to Remember	433
Configure GVRP	434
Related Configuration Tasks	434
Enabling GVRP Globally	435
Enabling GVRP on a Layer 2 Interface	435
Configure GVRP Registration	435
Configure a GARP Timer	436
High Availability (HA)	438
High Availability on Chassis	438
High Availability in a PE Stack	438
Online Insertion and Removal	438
RPM Online Insertion	439
Line Card Online Insertion	439
Pre-configuring a Slot for a Line-Card Type	439
Replacing a Line Card	440
Hitless Behavior	440
Graceful Restart	441
Software Resiliency	441
System Health Monitoring	441
Failure and Event Logging	442
Trace Log	442
Core Dumps	442
System Log	442
Control Plane Redundancy	442
Control-Plane Failover	443
RPM Synchronization	444
Forcing an RPM Failover	444
Specifying an Auto-Failover Limit	444
Disabling Auto-Reboot	444
Internet Group Management Protocol (IGMP)	446
IGMP Implementation Information	446
IGMP Protocol Overview	446
IGMP Version 2	446
IGMP Version 3	448
Configure IGMP	451
Related Configuration Tasks	451
Viewing IGMP Enabled Interfaces	452
Selecting an IGMP Version	452
Viewing IGMP Groups	453
Enabling IGMP Immediate-Leave	453
IGMP Snooping	453
IGMP Snooping Implementation Information	454
Configuring IGMP Snooping	454
Removing a Group-Port Association	454
Disabling Multicast Flooding	455
Specifying a Port as Connected to a Multicast Router	455
Configuring the Switch as Querier	455
Fast Convergence after MSTP Topology Changes	456
Designating a Multicast Router Interface	456
Interfaces	458
Basic Interface Configuration	458
Advanced Interface Configuration	458
Port Numbering	459
Interface Types	461
View Basic Interface Information	462
Resetting an Interface to its Factory Default State	468
Enabling a Physical Interface	469
Physical Interfaces	469

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1,000
1,001
1,002
1,003
1,004
1,005
1,006
1,007
1,008
1,009
1,010
1,011
1,012
1,013
1,014
1,015
1,016
1,017
1,018
1,019
1,020
1,021
1,022
1,023
1,024
1,025
1,026
1,027
1,028
1,029
1,030
1,031
1,032
1,033
1,034
1,035
1,036
1,037
1,038
1,039
1,040
1,041
1,042
1,043
1,044
1,045
1,046
1,047
1,048
1,049
1,050
1,051
1,052
1,053
1,054
1,055
1,056
1,057
1,058
1,059
1,060
1,061
1,062
1,063
1,064
1,065
1,066
1,067
1,068
1,069
1,070
1,071
1,072
1,073
1,074
1,075
1,076
1,077
1,078
1,079
1,080
1,081
1,082
1,083
1,084
1,085
1,086
1,087
1,088
1,089
1,090
1,091
1,092
1,093
1,094
1,095
1,096
1,097
1,098
1,099
1,100
1,101
1,102
1,103
1,104
1,105
1,106
1,107
1,108
1,109
1,110
1,111
1,112
1,113
1,114
1,115
1,116
1,117
1,118
1,119
1,120
1,121
1,122
1,123
1,124
1,125
1,126
1,127
1,128
1,129
1,130
1,131
1,132
1,133
1,134
1,135
1,136
1,137
1,138
1,139
1,140
1,141
1,142
1,143
1,144
1,145
1,146
1,147
1,148

Match case Limit results 1 per page

Equal Cost Multi-Path (ECMP)

Equal cost multi-path (ECMP) supports multiple paths in next-hop packet forwarding to a destination

device.

ECMP for Flow-Based Affinity

ECMP for flow-based affinity includes link bundle monitoring.

Enabling Deterministic ECMP Next Hop

Deterministic ECMP next hop arranges all ECMPs in order before writing them into the content

addressable memory (CAM).

For example, suppose the RTM learns eight ECMPs in the order that the protocols and interfaces came

up. In this case, the forwarding information base (FIB) and CAM sort them so that the ECMPs are always

arranged. This implementation ensures that every chassis having the same prefixes orders the ECMPs the

same.

With eight or less ECMPs, the ordering is lexicographic and deterministic. With more than eight ECMPs,

ordering is deterministic, but it is not in lexicographic order.

To enable deterministic ECMP next hop, use the appropriate command.

NOTE:

Packet loss might occur when you enable

ip/ipv6 ecmp-deterministic

for the first-

time only.

•

Enable IPv4 Deterministic ECMP Next Hop.

CONFIGURATION mode.

ip ecmp-deterministic

•

Enable IPv6 Deterministic ECMP Next Hop.

CONFIGURATION mode.

ipv6 ecmp-deterministic

Configuring the Hash Algorithm Seed

Deterministic ECMP sorts ECMPs in order even though RTM provides them in a random order. However,

the hash algorithm uses as a seed the lower 12 bits of the chassis MAC, which yields a different hash

result for every chassis.

This behavior means that for a given flow, even though the prefixes are sorted, two unrelated chassis can

select different hops.

The system provides a command line interface (CLI)-based solution for modifying the hash seed to

ensure that on each configured system, the ECMP selection is same. When configured, the same seed is

set for ECMP, LAG, and NH, and is used for incoming traffic only.

NOTE:

While the seed is stored separately on each port-pipe, the same seed is used across all

CAMs.

Equal Cost Multi-Path (ECMP)

387