Home » Dell Manuals » Hubs & Switches » Dell C1048P Port Extender » Manual Viewer

Dell C1048P Port Extender Networking Configuration Guide for the C9000 Series - Page 373

VLAN ID 4095., Flex Hash and Optimized Boot-Up

Get Dell C1048P Port Extender PDF manuals and user guides

View all Dell C1048P Port Extender manuals

Add to My Manuals
Save this manual to your list of manuals

Page 373 highlights

the RoCE and the IP networks and sends the RoCE frames over the IP network. RRoCE transmission results in the encapsulation of RoCE packets in IP packets. RRoCE packets are received and transmitted on specific interfaces called lite-subinterfaces. These interfaces are similar to the normal Layer 3 physical interfaces except for the extra provisioning that they offer to enable the VLAN ID for encapsulation. You can configure a physical interface or a Layer 3 Port Channel interface as a lite subinterface. When you configure a lite subinterface, only tagged IP packets with VLAN encapsulation are processed and routed. All other data packets are discarded. To provide lossless service for RRoCE, the QoS service policy must be configured in the ingress and egress directions on lite subinterfaces. A normal Layer 3 physical interface processes only untagged packets and makes routing decisions based on the default Layer 3 VLAN ID (4095). To enable routing of RRoCE packets, the VLAN ID is mapped to the default VLAN ID of 4095 using VLAN translation. After the VLAN translation, the RRoCE packets are processed in the same way as normal IP packets that a Layer 3 interface receives and routes in the egress direction. At the egress interface, the VLAN ID is appended to the packet and transmitted out of the interface as a tagged packet with the dot1Q value preserved. When a storage area network (SAN) is connected over an IP network, the following conditions must be satisfied: • Faster Connectivity: QoS for RRoCE enables faster and lossless nature of disk input and output services. • Lossless connectivity: VMs require the connectivity to the storage network to be lossless always. When a planned upgrade of the network nodes happens, especially with top-of-rack (ToR) nodes where there is a single point of failure for the VMs, disk I/O operations are expected to occur in 20 seconds. If disk in not accessible in 20 seconds, unexpected and undefined behavior of the VMs occurs. You can optimize the booting time of the ToR nodes that experience a single point of failure to reduce the outage in traffic-handling operations. RRoCE has IP headers. RRoCE is bursty and uses the entire 10-Gigabit Ethernet interface. Although RRoCE and normal data traffic are propagated in separate network portions, it may be necessary in certain topologies to combine both the RRoCE and the data traffic in a single network structure. RRoCE traffic is marked with dot1p priorities 3 and 4 (code points 011 and 100, respectively) and these queues are strict and lossless. DSCP code points are not tagged for RRoCE. Both ECN and PFC are enabled for RRoCE traffic. For normal IP or data traffic that is not RRoCE-enabled, the packets comprise TCP and UDP packets and they can be marked with DSCP code points. Multicast is not supported in that network. Flex Hash and Optimized Boot-Up 373

Section	Page
Dell Networking Configuration Guide for the C9000 Series Version 9.10(0.1)	3
About this Guide	33
Audience	33
Conventions	33
Related Documents	33
Configuration Fundamentals	34
Accessing the Command Line	34
CLI Modes	34
Navigating CLI Modes	36
The do Command	39
Undoing Commands	39
Obtaining Help	40
Entering and Editing Commands	40
Command History	41
Filtering show Command Outputs	41
Multiple Users in Configuration Mode	43
Getting Started	44
Console Access	44
Serial Console	45
Mounting an NFS File System	46
Important Points to Remember	46
Default Configuration	47
Configuring a Host Name	47
Accessing the System Remotely	48
Accessing the System Remotely	48
Configure the Management Port IP Address	48
Configure a Management Route	48
Configuring a Username and Password	49
Configuring the Enable Password	49
Manage Configuration Files	49
File Storage	50
Copy Files to and from the System	50
Save the Running-Configuration	51
Configure the Overload Bit for a Startup Scenario	52
Viewing Files	52
Changes in Configuration Files	53
Viewing Command History	53
Upgrading the Dell Networking OS	53
Switch Management	54
Configuring Privilege Levels	54
Creating a Custom Privilege Level	54
Removing a Command from EXEC Mode	54
Moving a Command from EXEC Privilege Mode to EXEC Mode	54
Allowing Access to CONFIGURATION Mode Commands	55
Allowing Access to the Following Modes	55
Applying a Privilege Level to a Username	56
Applying a Privilege Level to a Terminal Line	57
Configuring Logging	57
Audit and Security Logs	57
Configuring Logging Format	59
Setting Up a Secure Connection to a Syslog Server	59
Track Login Activity	61
Restrictions for Tracking Login Activity	61
Configuring Login Activity Tracking	61
Display Login Statistics	61
Limit Concurrent Login Sessions	63
Restrictions for Limiting the Number of Concurrent Sessions	63
Configuring Concurrent Session Limit	63
Enabling the System to Clear Existing Sessions	63
Log Messages in the Internal Buffer	64
Configuration Task List for System Log Management	64
Disabling System Logging	65
Sending System Messages to a Syslog Server	65
Configuring a UNIX System as a Syslog Server	65
Display the Logging Buffer and the Logging Configuration	66
Changing System Logging Settings	66
Configuring a UNIX Logging Facility Level	67
Synchronizing Log Messages	68
Enabling Timestamp on Syslog Messages	68
File Transfer Services	69
Configuration Task List for File Transfer Services	69
Enabling the FTP Server	69
Configuring FTP Server Parameters	70
Configuring FTP Client Parameters	70
Terminal Lines	71
Denying and Permitting Access to a Terminal Line	71
Configuring Login Authentication for Terminal Lines	71
Setting Time Out of EXEC Privilege Mode	72
Using Telnet to Access Another Network Device	73
Lock CONFIGURATION Mode	73
Viewing the Configuration Lock Status	73
Recovering from a Forgotten Password	74
Ignoring the Startup Configuration and Booting from the Factory-Default Configuration	75
Recovering from a Failed Start	75
Restoring Factory-Default Settings	75
Important Point to Remember	76
Restoring Factory-Default Boot Environment Variables	76
Using Hashes to Verify Software Images Before Installation	77
Verifying System Images on C9010 Components	78
When System Images on C9010 Components Do Not Match	79
Manually Resetting the System Image on a C9010 Component	80
Logging in to the Virtual Console of a C9010 Component	80
Booting the C9010 from an Image on a Network Server	80
Configuring C9010 Components to Boot from the RPM CP Image	81
802.1X	82
The Port-Authentication Process	84
EAP over RADIUS	85
Configuring 802.1X	86
Related Configuration Tasks	86
Important Points to Remember	86
Enabling 802.1X	87
Configuring dot1x Profile	89
Configuring MAC addresses for a do1x Profile	89
Configuring the Static MAB and MAB Profile	90
Configuring Critical VLAN	91
Configuring Request Identity Re-Transmissions	91
Configuring a Quiet Period after a Failed Authentication	92
Forcibly Authorizing or Unauthorizing a Port	93
Re-Authenticating a Port	94
Configuring Dynamic VLAN Assignment with Port Authentication	94
Guest and Authentication-Fail VLANs	95
Configuring a Guest VLAN	96
Configuring an Authentication-Fail VLAN	97
Configuring Timeouts	98
Multi-Host Authentication	99
Configuring Multi-Host AuthenticationConfiguring Single-Host Authentication	100
Multi-Supplicant Authentication	102
Configuring Multi-Supplicant AuthenticationRestricting Multi-Supplicant Authentication	102
MAC Authentication Bypass	103
MAB in Single-host and Multi-Host Mode	103
MAB in Multi-Supplicant Authentication Mode	104
Configuring MAC Authentication Bypass	104
Dynamic CoS with 802.1X	105
Access Control Lists (ACLs)	107
IP Access Control Lists (ACLs)	107
CAM Usage	108
User-Configurable CAM Allocation	109
Allocating CAM for Ingress ACLs on the Port Extender	109
Allocating CAM for Egress ACLs on the Port Extender	111
Implementing ACLs	112
IP Fragment Handling	113
IP Fragments ACL Examples	113
Layer 4 ACL Rules Examples	114
Configure a Standard IP ACL	114
Configuring a Standard IP ACL Filter	115
Configure an Extended IP ACL	116
Configuring Filters with a Sequence Number	116
Configuring Filters Without a Sequence Number	118
Configure Layer 2 and Layer 3 ACLs	118
Using ACL VLAN Groups	119
Guidelines for Configuring ACL VLAN Groups	119
Configuring an ACL VLAN Group	120
Allocating ACL VLAN CAM	121
Applying an IP ACL to an Interface	121
Applying Ingress ACLs on the Port Extender	122
Applying Egress ACLs	122
Applying Layer 3 Egress ACLs on Control-Plane Traffic	123
Counting ACL Hits	124
IP Prefix Lists	124
Implementation Information	124
Configuration Task List for Prefix Lists	124
ACL Resequencing	128
Resequencing an ACL or Prefix List	129
Route Maps	130
Implementation Information	130
Important Points to Remember	130
Configuration Task List for Route Maps	130
Configuring Match Routes	133
Configuring Set Conditions	134
Configure a Route Map for Route Redistribution	135
Configure a Route Map for Route Tagging	135
Continue Clause	136
Configuring a UDF ACL	136
Hot-Lock Behavior	138
Bidirectional Forwarding Detection (BFD)	139
How BFD Works	139
BFD Packet Format	140
BFD Sessions	141
BFD Three-Way Handshake	142
Session State Changes	144
Important Points to Remember	144
Configure BFD	144
Configure BFD for Static Routes	145
Configure BFD for OSPF	147
Configure BFD for OSPFv3	150
Configure BFD for IS-IS	151
Configure BFD for BGP	154
Configure BFD for VRRP	160
Configuring Protocol Liveness	163
Border Gateway Protocol IPv4 (BGPv4)	164
Autonomous Systems (AS)	164
Sessions and Peers	166
Establish a Session	167
Route Reflectors	167
Communities	168
BGP Attributes	168
Best Path Selection Criteria	169
Weight	170
Local Preference	171
Multi-Exit Discriminators (MEDs)	171
Origin	172
AS Path	172
Next Hop	173
Multiprotocol BGP	173
Implement BGP	173
Additional Path (Add-Path) Support	173
Advertise IGP Cost as MED for Redistributed Routes	174
Ignore Router-ID for Some Best-Path Calculations	174
Four-Byte AS Numbers	174
AS4 Number Representation	175
AS Number Migration	176
BGP4 Management Information Base (MIB)	178
Important Points to Remember	178
Configuration Information	179
BGP Configuration	179
Enabling BGP	180
Configuring AS4 Number Representations	183
Configuring Peer Groups	184
Configuring BGP Fast Fail-Over	187
Configuring Passive Peering	188
Maintaining Existing AS Numbers During an AS Migration	189
Allowing an AS Number to Appear in its Own AS Path	190
Enabling Neighbor Graceful Restart	190
Filtering on an AS-Path Attribute	191
Regular Expressions as Filters	192
Redistributing Routes	194
Enabling Additional Paths	194
Configuring IP Community Lists	195
Configuring an IP Extended Community List	196
Filtering Routes with Community Lists	197
Manipulating the COMMUNITY Attribute	197
Changing MED Attributes	199
Changing the LOCAL_PREFERENCE Attribute	199
Configuring the local System or a Different System to be the Next Hop for BGP-Learned Routes	200
Changing the WEIGHT Attribute	200
Enabling Multipath	201
Filtering BGP Routes	201
Filtering BGP Routes Using Route Maps	202
Filtering BGP Routes Using AS-PATH Information	203
Configuring BGP Route Reflectors	204
Aggregating Routes	204
Configuring BGP Confederations	205
Enabling Route Flap Dampening	205
Changing BGP Timers	207
Enabling BGP Neighbor Soft-Reconfiguration	208
Route Map Continue	209
Enabling MBGP Configurations	209
BGP Regular Expression Optimization	210
Debugging BGP	210
Storing Last and Bad PDUs	211
Capturing PDUs	212
PDU Counters	213
Sample Configurations	213
Content Addressable Memory (CAM)	222
CAM Allocation	222
Test CAM Usage	224
View CAM-ACL Settings	224
View CAM Usage	225
Return to the Default CAM Configuration	225
CAM Optimization	226
Applications for CAM Profiling	226
LAG HashingLAG Hashing Based on Bidirectional Flow	226
Unified Forwarding Table (UFT) Modes	226
Configuring UFT Modes	227
Control Plane Policing (CoPP)	228
CoPP Implementation	228
Protocol-based Control Plane Policing	228
Queue-based Control Plane Policing	228
CoPP Example	230
Configure Control Plane Policing	231
Configuring CoPP for Protocols	231
Examples of Configuring CoPP for Protocols	232
Configuring CoPP for CPU Queues	233
Examples of Configuring CoPP for CPU Queues	234
Displaying CoPP Configuration	235
Troubleshooting CoPP Operation	238
Enabling CPU Traffic Statistics	238
Viewing CPU Traffic Statistics	238
Troubleshooting CPU Packet Loss	239
Viewing Per-Protocol CoPP Counters	240
Viewing Per-Queue CoPP Counters	243
Data Center Bridging (DCB)	244
Enabling Data Center Bridging	244
Ethernet Enhancements in Data Center Bridging	245
Priority-Based Flow Control	246
Enhanced Transmission Selection	247
Data Center Bridging Exchange Protocol (DCBx)	248
Data Center Bridging in a Traffic Flow	248
QoS dot1p Traffic Classification and Queue Assignment	249
SNMP Support for PFC and Buffer Statistics Tracking	249
DCB Maps and its Attributes	250
DCB Map: Configuration Procedure	250
Important Points to Remember	251
Applying a DCB Map on a Port	251
Configuring PFC without a DCB Map	252
Configuring Lossless Queues	252
Applying a DCB Map on a Line Card	253
Data Center Bridging: Default Configuration	253
Configuration Notes: PFC and ETS in a DCB Map	254
PFC Configuration Notes	254
ETS Configuration Notes	255
ETS Prerequisites and Restrictions	256
Priority-Group Configuration Notes	256
Configuring Priority-Based Flow Control	256
Configuring Lossless Queues	257
Configuring Enhanced Transmission Selection	258
Creating an ETS Priority Group	258
ETS Operation with DCBx	259
Configure a DCBx Operation	259
DCBx Operation	260
DCBx Port Roles	260
DCB Configuration Exchange	261
Configuration Source Election	262
Propagation of DCB Information	262
Auto-Detection and Manual Configuration of the DCBx Version	262
Behavior of Tagged Packets	263
Configuration Example for DSCP and PFC Priorities	263
DCBx Example	265
DCBx Prerequisites and Restrictions	265
Configuring DCBx	266
Verifying the DCB Configuration	269
Performing PFC Using DSCP Bits Instead of 802.1p Bits	279
PFC and ETS Configuration Examples	279
Using PFC and ETS to Manage Data Center Traffic	279
PFC and ETS Configuration Command Examples	281
Using PFC and ETS to Manage Converged Ethernet Traffic	281
Hierarchical Scheduling in ETS Output Policies	281
Priority-Based Flow Control Using Dynamic Buffer Method	282
Pause and Resume of Traffic	282
Buffer Sizes for Lossless or PFC Packets	283
Configuring the Dynamic Buffer Method	283
Debugging and Diagnostics	285
Offline Diagnostics	285
Running Port Extender Offline Diagnostics on the Switch	285
Running Offline Diagnostics on a Standalone Switch	291
TRACE Logs	312
Auto Save on Reload, Crash, or Rollover	312
Uploading Trace Logs	312
Last Restart Reason	313
show hardware Commands	313
Environmental Monitoring	315
Displaying Port Extender Environment Information	315
Display Power Supply Status	316
Display Fan Status	316
Display Transceiver Type	317
Recognize an Over-Temperature Condition	318
Troubleshoot an Over-Temperature Condition	319
Troubleshooting Packet Loss	321
Displaying Drop Counters	322
Displaying Dataplane Statistics	322
Displaying Line-Card Counters	324
Accessing Application Core Dumps	325
Mini Core Dumps	325
Full Kernel Core Dumps	326
Enabling TCP Dumps	326
Accessing Port Extender Core and Mini Core Dumps	327
Dynamic Host Configuration Protocol (DHCP)	328
DHCP Packet Format and Options	328
Assign an IP Address using DHCP	330
Implementation Information	331
Configure the System to be a DHCP Server	331
Configuring the Server for Automatic Address Allocation	332
Specifying a Default Gateway	333
Configure a Method of Hostname Resolution	334
Using DNS for Address Resolution	334
Using NetBIOS WINS for Address Resolution	334
Creating Manual Binding Entries	334
Debugging the DHCP Server	335
Using DHCP Clear Commands	335
Configure the System to be a Relay Agent	335
Configure the System to be a DHCP Client	337
DHCP Client on a Management Interface	337
DHCP Client Operation with Other Features	337
Configure Secure DHCP	338
Option 82	338
DHCP Snooping	339
Drop DHCP Packets on Snooped VLANs Only	341
Dynamic ARP Inspection	342
Configuring Dynamic ARP Inspection	343
Source Address Validation	344
Enabling IP Source Address Validation	344
DHCP MAC Source Address Validation	345
Enabling IP+MAC Source Address Validation	345
Viewing the Number of SAV Dropped Packets	345
Clearing the Number of SAV Dropped Packets	346
Equal Cost Multi-Path (ECMP)	347
ECMP for Flow-Based Affinity	347
Enabling Deterministic ECMP Next Hop	347
Configuring the Hash Algorithm Seed	347
Link Bundle Monitoring	348
Managing ECMP Group Paths	348
Creating an ECMP Group Bundle	349
Modifying the ECMP Group Threshold	349
BGP Multipath Operation with Link Bankwidth	350
Dynamic Re-calculation of Link Bankwidth	351
Weighted ECMP for Static Routes	352
ECMP Support in L3 Host and LPM Tables	352
FCoE Transit	353
Fibre Channel over Ethernet	353
Ensure Robustness in a Converged Ethernet Network	353
FIP Snooping on Ethernet Bridges	355
FIP Snooping in a Switch Stack	357
Using FIP Snooping	357
FIP Snooping Prerequisites	357
Important Points to Remember	357
Enabling the FCoE Transit Feature	358
Enable FIP Snooping on VLANs	358
Configure the FC-MAP Value	358
Configure a Port for a Bridge-to-Bridge Link	359
Configure a Port for a Bridge-to-FCF Link	359
Impact on Other Software Features	359
FIP Snooping Restrictions	359
Configuring FIP Snooping	360
Displaying FIP Snooping Information	361
FCoE Transit Configuration Example	366
FIPS Cryptography	368
Configuration Tasks	368
Preparing the System	368
Enabling FIPS Mode	369
Generating Host-Keys	369
Monitoring FIPS Mode Status	369
Disabling FIPS Mode	370
Flex Hash and Optimized Boot-Up	371
Flex Hash Capability Overview	371
Configuring the Flex Hash Mechanism	371
LACP Fast Switchover	372
Configuring LACP Fast Switchover	372
LACP	372
LACP Fast Switchover	372
RDMA Over Converged Ethernet (RoCE) Overview	372
Sample Configurations	374
	374
Preserving 802.1Q VLAN Tag Value for Lite Subinterfaces	377
Force10 Resilient Ring Protocol (FRRP)	378
Protocol Overview	378
Ring Status	379
Multiple FRRP Rings	379
Important FRRP Points	380
Implementing FRRP	380
Important FRRP Concepts	381
FRRP Configuration	382
Creating the FRRP Group	382
Configuring the Control VLAN	382
Configuring and Adding the Member VLANs	383
Setting the FRRP Timers	384
Clearing the FRRP Counters	385
Viewing the FRRP Configuration	385
Viewing the FRRP Information	385
Troubleshooting FRRP	385
Configuration Checks	385
Sample Configuration and Topology	386
GARP VLAN Registration Protocol (GVRP)	388
Important Points to Remember	388
Configure GVRP	389
Related Configuration Tasks	389
Enabling GVRP Globally	389
Enabling GVRP on a Layer 2 Interface	390
Configure GVRP Registration	390
Configure a GARP Timer	391
High Availability (HA)	392
High Availability on Chassis	392
High Availability in a PE Stack	392
Online Insertion and Removal	392
RPM Online Insertion	393
Line Card Online Insertion	393
Pre-configuring a Slot for a Line-Card Type	393
Replacing a Line Card	394
Hitless Behavior	394
Graceful Restart	395
Software Resiliency	395
System Health Monitoring	395
Failure and Event Logging	395
Trace Log	395
Core Dumps	395
System Log	396
Control Plane Redundancy	396
Control-Plane Failover	396
RPM Synchronization	397
Forcing an RPM Failover	397
Specifying an Auto-Failover Limit	398
Disabling Auto-Reboot	398
Internet Group Management Protocol (IGMP)	399
IGMP Implementation Information	399
IGMP Protocol Overview	399
IGMP Version 2	399
IGMP Version 3	401
Configure IGMP	403
Related Configuration Tasks	404
Viewing IGMP Enabled Interfaces	404
Selecting an IGMP Version	404
Viewing IGMP Groups	405
Enabling IGMP Immediate-Leave	405
IGMP Snooping	405
IGMP Snooping Implementation Information	406
Configuring IGMP Snooping	406
Removing a Group-Port Association	406
Disabling Multicast Flooding	407
Specifying a Port as Connected to a Multicast Router	407
Configuring the Switch as Querier	407
Fast Convergence after MSTP Topology Changes	408
Designating a Multicast Router Interface	408
Interfaces	409
Basic Interface Configuration	409
Advanced Interface Configuration	409
Port Numbering	410
Interface Types	413
View Basic Interface Information	413
Resetting an Interface to its Factory Default State	419
Enabling a Physical Interface	419
Physical Interfaces	420

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1,000
1,001
1,002
1,003
1,004
1,005
1,006
1,007
1,008
1,009
1,010
1,011
1,012
1,013
1,014
1,015
1,016
1,017
1,018
1,019
1,020
1,021
1,022
1,023
1,024
1,025
1,026
1,027
1,028
1,029
1,030
1,031
1,032
1,033
1,034
1,035
1,036
1,037
1,038
1,039
1,040
1,041
1,042
1,043
1,044
1,045
1,046
1,047

Match case Limit results 1 per page

the RoCE and the IP networks and sends the RoCE frames over the IP network. RRoCE transmission results in the encapsulation

of RoCE packets in IP packets.

RRoCE packets are received and transmitted on specific interfaces called lite-subinterfaces. These interfaces are similar to the

normal Layer 3 physical interfaces except for the extra provisioning that they offer to enable the VLAN ID for encapsulation.

You can configure a physical interface or a Layer 3 Port Channel interface as a lite subinterface. When you configure a lite

subinterface, only tagged IP packets with VLAN encapsulation are processed and routed. All other data packets are discarded.

To provide lossless service for RRoCE, the QoS service policy must be configured in the ingress and egress directions on lite

subinterfaces.

A normal Layer 3 physical interface processes only untagged packets and makes routing decisions based on the default Layer 3

VLAN ID (4095).

To enable routing of RRoCE packets, the VLAN ID is mapped to the default VLAN ID of 4095 using VLAN translation. After the

VLAN translation, the RRoCE packets are processed in the same way as normal IP packets that a Layer 3 interface receives and

routes in the egress direction. At the egress interface, the VLAN ID is appended to the packet and transmitted out of the

interface as a tagged packet with the dot1Q value preserved.

When a storage area network (SAN) is connected over an IP network, the following conditions must be satisfied:

•

Faster Connectivity: QoS for RRoCE enables faster and lossless nature of disk input and output services.

•

Lossless connectivity: VMs require the connectivity to the storage network to be lossless always. When a planned upgrade

of the network nodes happens, especially with top-of-rack (ToR) nodes where there is a single point of failure for the VMs,

disk I/O operations are expected to occur in 20 seconds. If disk in not accessible in 20 seconds, unexpected and undefined

behavior of the VMs occurs. You can optimize the booting time of the ToR nodes that experience a single point of failure to

reduce the outage in traffic-handling operations.

RRoCE has IP headers. RRoCE is bursty and uses the entire 10-Gigabit Ethernet interface. Although RRoCE and normal data

traffic are propagated in separate network portions, it may be necessary in certain topologies to combine both the RRoCE and

the data traffic in a single network structure. RRoCE traffic is marked with dot1p priorities 3 and 4 (code points 011 and 100,

respectively) and these queues are strict and lossless. DSCP code points are not tagged for RRoCE. Both ECN and PFC are

enabled for RRoCE traffic. For normal IP or data traffic that is not RRoCE-enabled, the packets comprise TCP and UDP packets

and they can be marked with DSCP code points. Multicast is not supported in that network.

Flex Hash and Optimized Boot-Up

373