Dell VNX5600 Fibre Channel SAN Topologies TechBook
Dell VNX5600 Manual
 |
View all Dell VNX5600 manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell VNX5600 manual content summary:
- Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 1
Fibre Channel SAN Topologies Version 4.2 • Fibre Channel Topology Overview • Simple and Complex Fibre Channel SAN Topologies • Brocade Virtual Fabrics Case Studies • FICON Topologies Erik Smith Richard Hultman Dennis Kloepping - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 2
herein are the property of their respective owners.. For the most up-to-date regulator document for your product line, go to Dell EMC Online Support (https://support.emc.com). Part number H8074.8 2 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 3
Contents Preface...9 Chapter 1 Fibre Channel SAN Topologies Fibre Channel topology overview 16 Instructions for using this TechBook 17 General best practices 19 Switch-specific best practices 24 Connectrix B-Series 24 Connectrix MDS 27 Host and storage layout 27 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 4
switch settings for interop 181 Heterogeneous interoperability test information 187 Monitoring your SAN Introduction 192 Switch-based error types 194 Fabric resiliency features and recommendations 200 Brocade SAN resiliency features 200 Fabric resiliency thresholds 202 Quick reference - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 5
260 Deleting a VF 260 Brocade Base Switches concept 261 FCR with Brocade VFs 261 Chapter 6 FICON Topologies Overview 264 Topology support 266 Zoning practices 267 Cascading 268 Terminology 269 IOCP considerations 271 FICON and Dell EMC ControlCenter 273 CUP (fabric management server - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 6
Contents Supported products 278 Requirements 278 Configuring 279 OCP considerations 280 CUP support 280 FICON configuration files 280 Switch node identifier 281 FICON port numbering 282 References 283 6 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 7
Figures Title Page 1 Single switch topology example 34 2 Single switch SAN using DS-5100B and EZSwitchSetup with zoning ... 35 3 Connectrix MDS 9506 using VSANs 38 4 Two switch fabric example 46 5 Two switch SAN with DS-5100B and DS-5300B using CLI to configure SAN ...48 6 Port - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 8
Figures 27 Phase 4: Completely moving host and storage ports 172 28 Phase 5: Adding Connectrix MDS 9216 173 29 Phase 6: Moving hosts and storage to a new edge 175 30 Phase 7: Adding Connectrix M switch to the core 176 31 Fabric wide effects of a latency bottleneck condition 209 32 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 9
of its product lines, Dell EMC periodically releases revisions of its hardware and software. Therefore, some functions described in this document may not be supported by all versions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 10
• Legacy SAN Technologies Reference Manual • Non-EMC SAN Products Data Reference Manual ◆ RSA security solutions documentation, at http://RSA.com > Content Library Find all of the following documentation and release notes on Dell EMC Online Support. Dell EMC hardware documents and release notes - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 11
following E-Lab documentation is also available: ◆ Host Connectivity Guides ◆ HBA Guides For Cisco and Brocade documentation, refer to the vendor's has held various technical roles in both Dell EMC Engineering and Technical Support. Erik has authored and coauthored several Dell EMC TechBooks. Erik is - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 12
programs, processes, services, applications, utilities, specifically selects, clicks, presses, or types Used in all text (including support, product, and licensing information can be obtained as follows. Dell EMC support, product, and licensing information can be obtained on Dell EMC Online Support - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 13
and service, go to Dell EMC Online Support (registration required). Technical support Dell EMC offers a variety of support options. Support by Customer Support Center. If you are missing a LAC letter, or require further instructions on activating your licenses through the Online Support site, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 14
Preface We'd like to hear from you! Your suggestions will help us continue to improve the accuracy, organization, and overall quality of the user publications. Send your opinions of this document to: [email protected] Your feedback on our TechBooks is important to us! We want our books to be - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 15
Tag 1 Fibre Channel SAN Topologies This chapter provides an overview of Fibre Channel SAN topologies. ◆ Fibre Channel topology overview 16 ◆ Instructions for using this TechBook 17 ◆ General best practices 19 ◆ Switch-specific best practices 24 ◆ Switch and fabric management 29 ◆ Security - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 16
Fibre Channel SAN Topologies Fibre Channel topology overview This chapter provides information on Fibre Channel SAN topologies. For valuable information that may be helpful prior to building a SAN, refer to the Networked Storage Concepts and Protocols TechBook, available on the E-Lab Navigator, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 17
SAN Topologies Instructions for using this TechBook This TechBook was written so that it could be read from front to back or used as a quick reference guide. The concept approach is meant to enhance the readability and supportability of the document. Instructions for using this TechBook 17 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 18
to address all possible host and storage combinations. For a complete list of supported host/switch/storage combinations, and the most up-to-date supported configurations, refer to the Dell EMC Support Matrices, available on Dell EMC E-Lab Navigator. 18 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 19
27 for information on host and storage layout. ◆ Plan cabling Table 1 on page 20 lists the typical distances that can be supported for Fibre Channel with the different fiber types and link speeds. OM2 cable was the standard 50um cable used with Fibre Channel for many years. The higher link speeds - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 20
.pdf. • Single initiator zoning with Cisco MDS Smart Zoning Cisco MDS Smart Zoning supports zoning among more devices by reducing the number of zoning entries to be programmed. Smart Zoning considers device type information (initiators or targets) without increasing the size of the zone set. Smart - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 21
One zone can support multiple initiators (hosts) while still maintaining the single initiator zoning rule. This enables fewer zones to be created and simplifies single initiator zoning. Consider the following recommendations: - Only a host or target be used as the device type. - For array ports - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 22
addresses, subnet mask, and gateway for each switch These values are used in both manual and GUI-based setup methods. ◆ Use a port fencing policy For more information , Documentation > Topology Resource Center. ◆ Use the latest supported firmware version and ensure that the same version of firmware - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 23
Fibre Channel SAN Topologies ◆ Periodically (or following any changes) back up switch configurations ◆ Utilize a syslog server Note: It is also recommended to install an NTP client on the syslog server. ◆ Use persistent Domain IDs ◆ Estimate light budget General best practices 23 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 24
. This information will not only help validate that the firmware download did not disrupt the existing configuration, but also provide the support team with sufficient information in an unlikely event that something goes wrong after a firmware download. 1. Establish a Telnet session and log in - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 25
commands are version dependent) to provide baseline information for advanced support. Note: If you are upgrading a Connectrix B director do not get processed and, in fact, are lost. To address these types of issues, Brocade defects have been opened and resolved. SAN administrators should use - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 26
Fibre Channel SAN Topologies ISL trunking ◆ A zoneset can be managed and activated from any switch in the fabric, but should be managed from a single entry switch within a fabric. This avoids complications arising from multiple users accessing different switches in a fabric to make concurrent zone - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 27
Fibre Channel SAN Topologies Connectrix MDS The following are requirements and guidelines for using IVR NAT: IVR NAT port login (PLOGI) requests received from hosts are delayed for a few seconds to perform the rewrite on the FC ID address. If the host's PLOGI timeout value is set to a value less - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 28
Fibre Channel SAN Topologies preference. However, if the switch being used contains multiple ASICs, try to connect host and storage pairs to the same ASIC. This prevents using the shared internal data transfer bus and reduces switch latency. In addition to performance concerns, consider fault - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 29
Fibre Channel SAN Topologies Switch and fabric management The following are management interfaces common to all platforms: ◆ CLI The command line interface (CLI) can be accessed through Telnet and Secure Shell (SSH, which authenticates remote computers). On some platforms, it is also possible to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 30
for the management of smaller fabrics with Connectrix B Series departmental switches. • CMCNE Professional Plus Edition. Increases fabric scalability and adds support for the ED-DCX-4S-B and ED-DCX8510-4B. • CMCNE Enterprise Edition. A full-featured Enterprise class management solution for the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 31
Fibre Channel SAN Topologies Security It is important to secure your fabric. General security best practices for an FC SAN are as follows: ◆ Implement some form of zoning ◆ Change the default password ◆ Disable unused or infrequently used Management Interfaces ◆ Use SSL or SSH if available ◆ Limit - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 32
Fibre Channel SAN Topologies Connectrix MDS The MDS supports the following protocols: ◆ SSH ◆ SFTP ◆ FCSP ◆ DHCHAP 32 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 33
Ivisible Body Tag 2 Simple Fibre Channel SAN Topologies This chapter provides the following information on simple Fibre Channel SAN topologies. ◆ Single switch fabrics 34 ◆ Two switch fabrics 46 ◆ Blade switch with direct attached storage 69 Simple Fibre Channel SAN Topologies 33 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 34
Simple Fibre Channel SAN Topologies Single switch fabrics This section provides examples of single switch fabrics. Overview of fabric design considerations General layout A single switch fabric consists of only a single switch (Figure 1). The switch is also connected to a single management LAN - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 35
exists, or as complicated as disabling all unused ports and hard-setting port type to F_Port. Refer to "Security" on page 31 for general security information. 5100B and EZSwitchSetup with zoning Supported switches Connectrix DS-220B and DS-5100B are supported with EZSwitchSetup. Best practices - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 36
up this topology When using Typical Zoning in EZSwitchSetup, the GUI guides the SAN administrator on recommended placement of both host and storage. get the license key. ◆ The customer has provided a server or laptop with CD drive, serial DB-9 connector, and a NIC card to be used to configure the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 37
letter H, and a desired storage port, designated by the color green, and the letter S. 9. Connect host and storage using the Configure Typical Zoning GUI as a guide. 10. Use the pull-down menu selections on the Specify Devices screen to select the number of HBA connections and the number of storage - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 38
Simple Fibre Channel SAN Topologies 13. Validate zoning configuration by ensuring that each HBA has access to the storage device(s). Each HBA should see every storage device when using Typical Zoning. The Validate link in the task panel of the Switch Manager page checks for devices that are not - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 39
to this case study: ◆ The switches are installed in a Dell EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. • For Cabinet power - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 40
Simple Fibre Channel SAN Topologies In this example it is assumed that the customer has provided one Ethernet cable and an IP of 172.23.199.22. ◆ The proper number of line cards have been installed in each chassis. In this case, two line cards in each chassis are required and installed in slots 1 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 41
Simple Fibre Channel SAN Topologies Note: This example starts with the switch that has a Domain ID of 1 and an IP address of 172.23.199.22. 7. Repeat the above steps for each switch, supplying the appropriate IP. CLI commands to configure the IP and gateway ◆ Switch# config terminal Enter - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 42
Simple Fibre Channel SAN Topologies Configure a VSAN To configure a VSAN: 1. Open the Device Manager for the switch with an IP address of 172.23.199.22. 2. Open the VSAN dialog box by selecting VSAN. 3. Click Create. 4. Enter the value of 100 into the VSAN ID field. 5. Set the VSAN Name to be " - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 43
Simple Fibre Channel SAN Topologies Slot # 2 2 2 2 2 2 2 Port # 2 3 4 5 6 7 8 Name Blue Host HBA 2 Green Host HBA 2 Green Storage 2 Red Storage 1 Green Storage 1 BlueStorage 1 VSAN ID 300 200 200 100 200 300 Connect cables To connect the cables: 1. Connect host and storage ports. 2. Attach - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 44
Simple Fibre Channel SAN Topologies 4. Provide a descriptive name for the zone. This example zones "Red host HBA 1" and "Red Storage 1", so "RedHBA1_1470_8aa" will be used. Click OK. 5. Locate, then click, "Red Host HBA 1" (WWPN 10000000c938e554) in the Potential zone members list. 6. Click the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 45
host configuration files, are required before the SAN setup is complete. Refer to the Cisco MDS 9000 Family Fabric Manager Configuration Guide for more details. Configure security ◆ Enable Switch Binding. ◆ Enable Port Binding. Configure proactive monitoring and countermeasures ISL thresholds are 80 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 46
FC Switch Storage Subnet A Management network Subnet B Figure 4 Best practices Two switch fabric example Management station GEN-000229 Every switch type can be used in any position in this configuration. The following best practices are specific for two switch fabrics. ◆ ISL subscription - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 47
Simple Fibre Channel SAN Topologies Host and storage layout Switch and fabric management Security to "Buffer-to-buffer credit information" in the Extended Distance Technologies TechBook, available through the on the E-Lab Navigator, Documents > Topology Resource Center. Refer to the individual - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 48
Simple Fibre Channel SAN Topologies General layout Figure 5 illustrates a two switch SAN with a DS-5100B and a DS-5300B using CLI to configure the SAN. Red Host HBA 1 Red Host HBA 2 Green Host HBA 1 Green Host HBA 1 Blue Host HBA 2 Blue Host HBA 2 Emulex 4Gb/sec Emulex 4Gb/sec Emulex 2Gb/sec - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 49
: ◆ The Fibre Channel switches are installed in a Dell EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual on Dell EMC Online Support. ◆ Redundant power sources are available. • For switch power requirements, refer to the Dell - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 50
HyperTerminal: a. Open HyperTerminal by clicking on the Start / Programs / Accessories / Communications / HyperTerminal. The Connection Description dialog box displays. b. Type a descriptive director name in the Name field and click OK. The Connect To dialog box displays. c. Ensure the Connect using - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 51
Simple Fibre Channel SAN Topologies - Bits per second: 9600 - Databits: 8 - Parity: None - Stop bits: 1 - Flow control: None e. Press Return to get a prompt. 4. Log in to the switch using the default values: Username: admin and Password: password. IMPORTANT It is strongly recommended that when - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 52
Simple Fibre Channel SAN Topologies 9. Connect the switch to the 10/100BaseT Ethernet connection for the 172.23.199.x network which was provided by the customer. 10. Power up the switch. The switch can now be accessed via an IP-based management tool. 11. Repeat Step 1 through Step 10 on the DS-5300B - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 53
Operation parameters (yes, y, no, n): [no] - RSCN Transmission Mode (yes, y, no, n): [no] - Arbitrated Loop parameters (yes, y, no, n): [no] - System services (yes, y, no, n): [no] - Portlog events enable (yes, y, no, n): [no] - ssl attributes (yes, y, no, n): [no] - http attributes (yes, y, no - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 54
Simple Fibre Channel SAN Topologies a. From the switch prompt, enter licenseadd . Example: switch:admin> licenseadd byeSRbdzyQkzfTS0 Note: To enable trunking on the switch after unlocking the license, you need to re-initialize the ports. To re-initialize the ports, you can - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 55
be set to Auto-negotiate speed and the port type should not be locked to either L_Port or and time Note: To assist with the review of support logs should the need arise, it is recommended that date and time will be set manually. To set the date and time of a switch manually: 1. Using Telnet, log in - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 56
The date function does not support Daylight Savings Time or time zones, so changes will have to be reset manually. Example: switch:admin> date a connectivity issue is encountered during this phase, it is easier to troubleshoot it now rather than after all cables have been attached. a. Connect ISLs - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 57
Simple Fibre Channel SAN Topologies zonecreate "BlueHBA1_1489_8aa", "21:01:00:e0:8b:8a:c7:6d; 50:06:04:82:cc:19:c4:47" zonecreate "BlueHBA2_1489_9aa", "21:01:00:e0:8b:aa:c7:6d; 50:06:04:82:cc:19:c4:48" zonecreate "GreenHBA1_AllGreenStorage", "10:00:00:00:c9:39:e5:51; 50:06:04:82:cc:19:c4:07; 50:06: - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 58
, or unintentional changes are made, keep a backup copy of the configuration file on a host computer. To upload a configuration file: 1. Verify that the FTP service is running on the host computer. The host must have an FTP server application running. 2. Connect to the switch through the Telnet and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 59
required before the SAN setup is complete. Refer to the OS configuration guide for more details. Connectrix MDS example Note: VSANs will be configured example. Refer to the Dell EMC Connectrix SAN Products Data Reference Manual, at E-Lab Navigator, Documentation > Topology Resource Center, for more - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 60
study: ◆ The switches are installed in an Dell EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. • For switch power requirements - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 61
Simple Fibre Channel SAN Topologies For switch or cabinet network requirements, refer to the EMC Connectrix SAN Products Data Reference Manual, available at E-Lab Navigator, Documentation > Topology Resource Center. Note: Connectrix MDS switches can be placed on either a public or private network. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 62
Simple Fibre Channel SAN Topologies 4. Connect to the serial port of the switch using an RS232 serial cable, with a baud rate of 9600, 8 data bits, no parity, 1 stop bit and no flow control. The login prompt should display. 5. Log in the first time with username admin and password admin. You should - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 63
Simple Fibre Channel SAN Topologies Example 1: IP address 172.23.199.22 2. Enter the IP address of the switch in the address bar. 3. Follow the prompts and accept all defaults to install both Fabric Manager and Device Manager. Fabric Manager and Device Manager can be started using the configured - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 64
Simple Fibre Channel SAN Topologies Example 2: IP address 172.23.200.22 Slot # 1 1 1 2 2 2 2 2 2 2 2 Port # 6 7 8 1 2 3 4 5 6 7 8 Name Blue Host HBA 1 VSAN ID 300 TE ISL to SW 2 1 Green Storage 2 200 Blue Host HBA 1 300 Blue Storage 2 300 For the switch with the IP address 172.23.200. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 65
Simple Fibre Channel SAN Topologies Connecting cables To connect the cables: 1. Connect ISLs. a. Attach fiber cable between switches as shown in Figure 7 on page 59. b. After all cables are connected, use Fabric Manager to verify that all ISL connections are up. c. Re-arrange icons to accurately - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 66
Simple Fibre Channel SAN Topologies 5. Locate, then click, "Red Host HBA 1" (WWPN 10000000c938e554) in the Potential zone members list. 6. Click the right-pointing arrow on the divider between the Potential members list and the zones list to add the HBA to the zone. 7. Locate, then click, "Red - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 67
Simple Fibre Channel SAN Topologies Zone set name = "Green_Fabric" Zone name = "GreenHBA1_AllGreenStorage" Zone Member = "10000000c939a051" Zone Member = "50060482cc19c407" Zone Member = "50060482cc19c408" Zone Member = "50060482cc19c4c7" Zone Member = "50060482cc19c4c8" Zone name = " - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 68
masking and modification of host configuration files, are required before the SAN setup is complete. Refer to the Cisco MDS 9000 Family Fabric Manager Configuration Guide for more details. 68 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 69
server chassis which can house at least ten independent servers, depending on the vendor type (IBM, Dell, HP, and so on). The chassis contains the power, cooling FC SAN world, the basic architecture, and the various EMC-supported blade switch modules are further discussed in the Fibre Channel over - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 70
, and offers fully redundant fabrics. Most switch modules today have 4 G ports. Considering a switch module has at the most 6 ports (depending on the vendor type), a blade server is capable of up to 48 GB/s throughput. General steps to set up a blade server To set up a blade server: 1. Configure - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 71
configuring the management module through the serial console using the Hyperterminal application, it is essential to gather the following information: - Connection type: COM 1 or COM 3 - Other settings: - Bits per second - Data bits - Parity bits - Stop bits If configuring the management through the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 72
ID settings: - Domain ID range for Brocade switch modules: 1 - 239 c. Ensure that the switch firmware revision and the switch management application are supported versions. d. If default zoning enabled does not allow the host ports to communicate with the storage ports, then the appropriate WWNN or - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 73
to have access to the storage port it is zoned with and there is no disruption in traffic between the server and storage. ◆ Always check supported switch module firmware release notes for caveats on direct attach to storage, if any. Host and storage layout For general host and storage layout - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 74
refer to "Switch and fabric management" on page 29. The following information is specific to blade switches. It is recommended that end-users use the supported vendor specific web application and CLI for managing the switch. ◆ The Brocade modules can be managed using Brocade CLI or Web Tools. ◆ The - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 75
identity of a connected switch, host or target and/or authorize a list of devices to join a fabric. ◆ ISL security is supported on E_Ports. ELS security is supported on F_Ports. ◆ Fabric Binding, Port Binding, and Switch Binding are introduced as a means to control the switch composition of a fabric - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 76
this topology: ◆ The IBM Blade server chassis and its components have been installed in the cabinet and powered up as per the IBM hardware installation guide for blade servers. (http://www.redbooks.ibm.com/redbooks/pdfs/sg246342.pdf) 76 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 77
Simple Fibre Channel SAN Topologies ◆ The following components are used for this example: • Two Ethernet modules • Brocade switch I/O modules • Management module • Three server blades ◆ The IP addresses and operating systems on the server blades can be configured by referring to the following - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 78
Simple Fibre Channel SAN Topologies b. Connect the Management Module to a workstation using a cross over network cable. c. Configure a static IP address for the workstation that is in the same subnet as the Management Module default IP addresses. In this case, a static address of 192.168.70.100 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 79
Simple Fibre Channel SAN Topologies One can now connect to the Management Module Web interface and Telnet into its Command Line Interface using the 172.23.199.60 IP address assigned to its external network interface. 2. Configure the Brocade SAN switch modules. The primary set up tasks for the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 80
be disabled by issuing switchdisable. - A new Domain ID can be assigned by running configure. - When prompted to choose yes or no for Configure.Fabric parameters, type y. - For the Domain ID setting, type 1. 80 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 81
by issuing the switchname IBM_brcd_module1 command. f. Ensure that a supported firmware version is running on the switch module by running version known problem with auto-negotiation exists between two port types. We recommend that you leave ports at auto-negotiate for both port type and speed - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 82
3 15 16 Symbolic port name Red Storage 2 Red Blade HBA 2 Blue Blade HBA 2 Green Blade HBA 2 Blue Storage 2 Green Storage 2 Port type F_port F_port F_port F_port F_port F_port Port speed AutoNeg. AutoNeg. AutoNeg. AutoNeg. AutoNeg. AutoNeg. Repeat steps Step a through Step f for the other switch - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 83
Simple Fibre Channel SAN Topologies Zoning is configured on the Brocade modules using either the Brocade CLI or Web Tools. This example uses Brocade Web Tools to do the zoning. a. Point the web browser to the Brocade module (172.23.199.22), which opens Web Tools. b. Select the Zoning icon on the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 84
steps, such as configuring LUN Masking and modification of host configuration files, are required before the SAN setup is complete. Refer to the OS configuration guide for more details. 84 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 85
Invisible Body Tag 3 Complex Fibre Channel SAN Topologies This chapter provides the following information on complex Fibre Channel SAN topologies. ◆ Best practices 86 ◆ Four switch full mesh 87 ◆ Compound core edge topologies 112 ◆ Heterogeneous switch interoperability 147 Complex Fibre - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 86
Complex Fibre Channel SAN Topologies Best practices General best practices for simple and complex Fibre Channel SAN topologies are described in "General best practices" on page 19. The information in this section is specific to complex Fibre Channel SAN topologies only. ISL subscription While - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 87
FC Switch FC Switch Storage Subnet A Management network Subnet B Figure 11 Best practices Management station GEN-000222 Four switch full mesh fabric Each switch type can be used in any position. For general best practices for all SANs, refer to "General best practices" on page 19. For - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 88
Complex Fibre Channel SAN Topologies Host and storage layout Switch and fabric management Security For more information on credit starvation, refer to "BB-Credit Loss" section in the Networked Storage Concepts and Protocols TechBook, available at Dell EMC E-Lab Navigator, , Documentation > - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 89
GEN-000498 Figure 12 Best practices ISL layout Four switch fabric with ED-48000Bs All Connectrix® B series switches supporting v5.2.x Fabric Operating software and higher are supported using this topology. In this example, the Connectrix ED-48000B director is used. For general information on best - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 90
Host and storage layout susceptible to congestion and backpressure. The ISLs are spread out over two different blades to enhance fault tolerance and serviceability. ISLs destined to the same Domain are kept within the same port octet to take advantage of trunking. Both hosts and storage can - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 91
case study: ◆ The switches are installed in an EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. • For switch power requirements - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 92
Complex Fibre Channel SAN Topologies • For help in determining how many ports are required, refer to "Determining customer requirements" in the Networked Storage Concepts and Protocols TechBook at Dell EMC E-Lab Navigator, Documentation > Topology Resource Center. ◆ License keys have been obtained. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 93
Note: For Domains 3 and 4, use the gateway address of 172.23.200.2. Configure FC switches Notes: ◆ WWN Based persistent PID-When enabled, this feature supports both dynamic and static WWN-based PID assignment. In dynamic PID binding, the first area assigned to a device when it logs in is bound to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 94
the area assignments. ◆ For additional switch configuration details and parameters, refer to the Brocade Fabric OS Administration Guide and the Brocade Command Reference Guide at Dell EMC Online Support. To configure FC switches: 1. Set the switch name and fabric parameters for the switch with the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 95
Operation parameters (yes, y, no, n): [no] - RSCN Transmission Mode (yes, y, no, n): [no] - Arbitrated Loop parameters (yes, y, no, n): [no] - System services (yes, y, no, n): [no] - Portlog events enable (yes, y, no, n): [no] - ssl attributes (yes, y, no, n): [no] - http attributes (yes, y, no - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 96
Complex Fibre Channel SAN Topologies Note: You may also press CNTRL D after making the last change in the menu to exit and save the changes. This will eliminate the need to accept the default values for the rest of the menu items. j. Repeat from Step a for switches with the Domain IDs 2, 3, and 4. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 97
Complex Fibre Channel SAN Topologies Zone hosts and storage To zone hosts and storage, Telnet into one of the switches in the fabric and using the following zoning commands: 1. Create zones using the zonecreate commands below: zonecreate "RedHBA1_1470_8aa", "10:00:00:00:c9:38:e5:54; 50:06:04:82:cc: - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 98
, or unintentional changes are made, keep a backup copy of the configuration file on a host computer. To upload a configuration file: 1. Verify that the FTP service is running on the host computer. The host must have an FTP server application running. 2. Connect to the switch through the Telnet and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 99
steps, such as configuring LUN Masking and modification of host configuration files, are required before the SAN setup is complete. Refer to the OS configuration guide for more details. Four switch full mesh 99 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 100
Complex Fibre Channel SAN Topologies Connectrix MDS example General layout Figure 13 illustrates four Connectrix MDS 9506s full mesh configuration. Red Host HBA 1 Red Host HBA 2 Emulex 4Gb/sec Emulex 4Gb/sec WWPN WWPN 10000000c938e554 10000000c938e555 Blue Host HBA 1 QLogic 4Gb/sec WWPN - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 101
case study: ◆ The switches are installed in an EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. • For switch power requirements - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 102
Complex Fibre Channel SAN Topologies Note: Connectrix MDS switches can be directly connected to the customer's LAN. The switches can be placed on either a public or private network. There are advantages to both configurations. For more information, refer to "Public versus private" in the Networked - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 103
Complex Fibre Channel SAN Topologies You should be prompted to supply a new strong password for CLI user admin. 6. For this example, select no when asked if you want to run setup. Note: This example will start with the switch that will have a Domain ID of 1 and an IP address of 172.23.199.22. CLI - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 104
Complex Fibre Channel SAN Topologies Fabric Manager and Device Manager can be started using the configured snmp-server username and password in "CLI commands to configure the IP and gateway" on page 103. Configure a VSAN To configure a VSAN: 1. Open the Device Manager for the switch with an IP - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 105
Slot # 1 1 1 Port # 6 7 8 Complex Fibre Channel SAN Topologies Name TE ISL to SW 2 VSAN ID 1 Slot # 2 2 2 2 2 2 2 2 Port # 1 2 3 4 5 6 7 8 Name TE ISL to SW 3 TE ISL to SW 4 Red Storage 1 Green Storage 1 Red Host HBA 1 Green Host HBA 1 TE ISL to SW 2 VSAN ID 1 1 100 200 100 200 1 b. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 106
Complex Fibre Channel SAN Topologies Slot # 2 2 2 2 2 Port # 4 5 6 7 8 Name Green Storage 3 Green Storage 4 VSAN ID 200 200 TE ISL to SW 1 1 c. Following Step 1 and Step 2, configure the ports of the switch with an IP address of 172.23.200.22 as shown in the tables below. Slot # 1 1 1 1 1 1 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 107
Complex Fibre Channel SAN Topologies d. Following Step 1 and Step 2, configure the ports of the switch with an IP address of 172.23.200.23 as shown in the table below. Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to SW 1 TE ISL to SW 2 VSAN ID 1 1 Blue Host HBA 2 300 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 108
New Zone under Zones. 3. Provide a descriptive name for the zone. This example will zone "Red host HBA 1" and "Red Storage 1". Type "RedHBA1_1470_8aa" and press Enter. 4. In zone "RedHBA1_1470_8aa" select (WWPN 10000000c938e554) select add to zone. 5. Select "Red Storage 1" (WWPN 50060482cc19bf87 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 109
of host configuration files, are required before the SAN setup is complete. Refer to the Cisco MDS 9000 Family Fabric Manager Configuration Guide for more details. Configure IVR with Network Address Translation (NAT) The configuration of VSANs on a fabric allows for security, scalability and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 110
Complex Fibre Channel SAN Topologies Note: To use IVR NAT, it must be enabled in all IVR-enabled switches in the fabric IVR configuration distribution. By default, IVR NAT and IVR configuration distribution are disabled in all switches fin the Cisco MDS 9000 Family. 1. In Fabric Manager: a. Click - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 111
Complex Fibre Channel SAN Topologies 6. Verify all steps that Fabric Manager will take to configure IVR in the fabric. 7. Click Finish if you want to enable IVR NAT and IVR topology and to create the associated IVR zones and IVR zone set. or Click Cancel to exit the IVR Wizard without saving any - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 112
Complex Fibre Channel SAN Topologies Compound core edge topologies This section provides examples of compound core edge switch topologies. Overview of fabric design considerations General layout Figure 14 shows an example of a four switch compound core edge switches. Network A Interswitch Link - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 113
Complex Fibre Channel SAN Topologies Best practices Host and storage layout Switch and fabric management Security Specific information on a compound core edge switch follows. ◆ Layout the host and storage connectivity such that if a switch fails, not all of a particular hosts storage becomes - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 114
Note: Any director class Connectrix B product, such as the ED-24000B or ED-48000B, can be used in the core of this fabric. All EMC-supported Connectrix B director and switch class products can be used as edge switches. 114 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 115
over-subscription management of the different switch port blades (i.e., the supported 16-port, 32-port, and 48-port blades) while connect ISLs to the same number ports on both switches to help assist with troubleshooting should the need arise. For general information on host and storage layout for - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 116
Complex Fibre Channel SAN Topologies Switch and fabric management Security Setting up this topology In this example, both CLI and Web Tools will be used to set up the compound core edge SAN topology, with the four Connectrix ED-48000Bs at the core and the eight DS-4900Bs at the edge. For general - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 117
Complex Fibre Channel SAN Topologies For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. For switch power requirements, refer to "Connectrix B series - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 118
Complex Fibre Channel SAN Topologies Configure the IP address To configure the IP address: Note: Connectrix B switches may ship with a default IP address that is not on the desired subnet. The ED-48000B director uses a maximum of three IPs per unit: one IP for the switch and one IP for each control - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 119
Complex Fibre Channel SAN Topologies 12. When prompted, supply hostname (48K_1_6), IP address (172.23.199.6), subnet mask (255.255.255.0), and gateway address (172.23.199.2). 13. Power down the switch and disconnect the serial cable. 14. Connect the switch to a 10/100BaseT Ethernet connection. 15. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 120
Complex Fibre Channel SAN Topologies Switch H 19 Switch I 20 Switch J 21 Switch K 22 Switch L 23 172.23.199.19 172.23.200.20 172.23.200.21 172.23.200.22 172.23.200.23 4900_19 4900_20 4900_21 4900_22 4900_23 Configure FC switches To configure FC switches: 1. Set the switch name and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 121
Operation parameters (yes, y, no, n): [no] - RSCN Transmission Mode (yes, y, no, n): [no] - Arbitrated Loop parameters (yes, y, no, n): [no] - System services (yes, y, no, n): [no] - Portlog events enable (yes, y, no, n): [no] - ssl attributes (yes, y, no, n): [no] - http attributes (yes, y, no - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 122
Fibre Channel SAN Topologies 4. Ports on the switch may be configured if desired. By default the port type and port speed is set to auto. a. Issue the portcfgeport 1 to configure the port type to E_Port for an ISL connection and to lock it as an E_Port. b. Issue the portcfggport - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 123
Complex Fibre Channel SAN Topologies Index 81 96 97 112 113 Slot/Port# 4/34 5/0 5/1 5/17 5/18 • For Switch B: Index 0 1 2 3 16 32 33 48 64 65 80 81 96 97 112 113 Slot/Port# 1/0 1/1 1/2 1/3 1/16 2/0 2/1 4/0 4/17 4/18 4/33 4/34 5/0 5/1 5/17 5/18 • For Switch C: Index 0 1 2 16 Slot/Port# 1/0 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 124
Complex Fibre Channel SAN Topologies Index 32 33 48 64 65 80 81 96 97 112 113 Slot/Port# 2/0 2/1 4/0 4/17 4/18 4/33 4/34 5/0 5/1 5/17 5/18 • For Switch D: Index 0 1 2 3 16 32 33 48 64 65 80 81 96 97 112 113 Slot/Port# 1/0 1/1 1/2 1/3 1/16 2/0 2/1 4/0 4/17 4/18 4/33 4/34 5/0 5/1 5/17 5/18 Name - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 125
Complex Fibre Channel SAN Topologies • For Switch E: Port# 0 1 2 3 4 • For Switch F: Port# 0 1 2 3 4 • For Switch G: Port# 0 1 2 3 4 Name ISL to domain 10 ISL to domain 10 ISL to domain 4 ISL to domain 4 Red Host HBA 1 Name ISL to domain 10 ISL to domain 10 ISL to domain 4 ISL to domain 4 Red - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 126
Complex Fibre Channel SAN Topologies • For Switch H: Port# 0 1 2 3 4 • For Switch I: Port# 0 1 2 3 4 • For Switch J: Port# 0 1 2 3 4 Name ISL to domain 13 ISL to domain 13 ISL to domain 7 ISL to domain 7 Green Storage 2 Name ISL to domain 4 ISL to domain 4 ISL to domain 10 ISL to domain 10 Red - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 127
Complex Fibre Channel SAN Topologies • For Switch K: Port# 0 1 2 3 4 Name ISL to domain 7 ISL to domain 7 ISL to domain 13 ISL to domain 13 Green Storage 4 • For Switch L: Port# 0 1 2 3 4 Name ISL to domain 7 ISL to domain 7 ISL to domain 13 ISL to domain 13 Green Storage 3 3. Verify port - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 128
Complex Fibre Channel SAN Topologies 6. Expand the WWNs folder (if needed) by clicking on the + sign. Click on the WWN of Red_HBA1 (10:00:00:00:c9:38:e5:54) and click Add Member to move the WWN to Alias Members column. This will create an alias for Blue_HBA1 using one of the WWNs of the host. All - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 129
Complex Fibre Channel SAN Topologies 14. Click the Zoning Actions pull-down menu on top of window. Click Enable Config and select "Oct_31_06_1140" zone and then click OK. This will push the new zone out to the fabric and make it effective. 15. When completed, the zone set, when running the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 130
LUN masking and modification of host configuration files, are required before the SAN setup is complete. Refer to the OS configuration guide for more details. Connectrix MDS example This section contains information on this Connectrix MDS example. 130 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 131
Complex Fibre Channel SAN Topologies General layout Figure 16 shows four MDS 9513s in a full mesh configuration with edge switches attached. Red Host HBA 1 Emulex 4Gb/sec WWPN 10000000c938e554 Red Host HBA 2 Emulex 4Gb/sec WWPN 10000000c938e555 Interswitch Link (ISL) FC (Block I/O) Blue Host HBA - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 132
to this case study: ◆ The switches are installed in a Dell EMC-supplied cabinet. • For installation instructions, refer to the Connectrix EC-1500 Cabinet Installation and Setup Manual at Dell EMC Online Support. ◆ The proper power receptacles have been provided by the customer. • For switch power - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 133
Complex Fibre Channel SAN Topologies Storage Concepts and Protocols TechBook at Dell EMC E-Lab Navigator, Documentation > Topology Resource Center. This example assumes that the customer has provided us with two Ethernet cables and that one of them is on the 172.23.199.x network and that the other - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 134
Complex Fibre Channel SAN Topologies Note: This example starts with the switch that will have a Domain ID of 1 and an IP address of 172.23.199.22. CLI commands to configure the IP and gateway ◆ Switch# config terminal Enter configuration commands, one per line. Switch(config)# interface mgmt 0 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 135
Complex Fibre Channel SAN Topologies Use the IP address of (172.23.199.23), subnet mask (255.255.255.0), and gateway (172.23.199.2). Install Fabric Manager and Device Manager To install Fabric Manager and Device Manager: 1. Open your web browser. 2. Enter the IP address of the switch into the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 136
Complex Fibre Channel SAN Topologies Table 2 14. From the Device Manager menu, select FC/Domain Manager/Configuration and set a static Domain ID for the switches as shown in Table 2 and Table 3. 172.23.199.22 through 172.23.199.27 IP 172.23.199.22 172.23.199.23 172.23.199.24 172.23.199.25 172.23 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 137
Complex Fibre Channel SAN Topologies c. Admin up and configure the ports as shown in the following tables: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 5 TE ISL to Domain 7 TE ISL to Domain 2 TE ISL to Domain 4 TE ISL to Domain 3 TE ISL to Domain 6 TE ISL to Domain 10 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 138
Complex Fibre Channel SAN Topologies d. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.199.23 as shown in the following tables: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 9 TE ISL to Domain 11 TE ISL to Domain 2 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 139
Complex Fibre Channel SAN Topologies e. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.200.22 as shown in the following tables: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 6 TE ISL to Domain 10 TE ISL to Domain 1 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 140
Complex Fibre Channel SAN Topologies f. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.200.23 as shown in the following tables: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 1 TE ISL to Domain 2 TE ISL to Domain TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 141
Complex Fibre Channel SAN Topologies g. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.199.24 as shown in the following table: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 2 TE ISL to Domain 2 TE ISL to Domain 1 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 142
Complex Fibre Channel SAN Topologies i. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.199.25 as shown in the following table: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 2 TE ISL to Domain 2 TE ISL to Domain 1 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 143
Complex Fibre Channel SAN Topologies k. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.199.26 as shown in the following table: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 4 TE ISL to Domain 4 TE ISL to Domain 3 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 144
Complex Fibre Channel SAN Topologies m. Following Step a through Step c, configure the ports of the switch with an IP address of 172.23.199.27 as shown in the following table: Slot # 1 1 1 1 1 1 1 1 Port # 1 2 3 4 5 6 7 8 Name TE ISL to Domain 4 TE ISL to Domain 4 TE ISL to Domain 3 TE ISL to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 145
New Zone under the Zones Tree. 3. Provide a descriptive name for the zone. This example zones "Red host HBA 1" and "Red Storage 1". Type "RedHBA1_1470_8aa" and press Enter. 4. Select "Red Host HBA 1" (WWPN 10000000c938e554) in the potential zone members list. 5. Click the right-pointing arrow - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 146
steps, such as configuring LUN masking and modification of host configuration files, are required before the SAN setup is complete. Refer to the OS configuration guide for more details. 146 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 147
Brocade M series. Switch migration procedures to move customers' existing SAN topology from one EMC-supported switch vendor type to another EMC-supported vendor type are also provided. Support for Native Connectivity between Brocade B-series and M-series platforms remains largely unchanged up to and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 148
for switches that are compatible with the FC-SW standard is listed in the "Switched Fabric Topology Parameters" section of the Dell EMC Simple Support Matrices. This section comprises of the following attributes (or columns): ◆ Name of the switch ◆ Name of the switches it is interoperable with - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 149
A, B, and C, respectively. If the following entries are present in the support matrix: ◆ A is interoperable with B ◆ A is interoperable with C detail in "Blade servers" in the Non-EMC SAN Products Data Reference Manual at Dell EMC E-Lab Navigator, Documentation > Topology Resource Center. Hence, if - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 150
details on setting up an interoperable SAN environment, and studies different combinations of some of the EMC-supported heterogeneous SANs. In addition to providing a detailed step-by-step approach to design some specific type of SANs, this section is intended to give you an insight into the kind of - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 151
to another vendor switch core type. ◆ One of the steps in the migration involves a stage when different vendor switches co-exist in the same fabric. Note: This can help end users set up a supported interoperability fabric based on the switch configuration settings from different vendors that can - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 152
7: More Vendor B switches are added, if required, to the core of the fabric. Based on the switch vendor types and the interoperability modes currently tested and supported by EMC, five case studies are described in this section. Migrating from a Connectrix B homogeneous fabric in Connectrix B native - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 153
at the edge. Refer to the "Switched Fabric Topology Parameters" section of the Dell EMC Simple Support Matrices to obtain a list of all the other Brocade/EMC Connectrix B switches that can be supported in a heterogeneous set up as well as the operating modes specified above for this case study. The - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 154
Complex Fibre Channel SAN Topologies core-edge homogeneous fabric discussed in the "Connectrix B example" on page 88 applies to this base topology. Checkpoints Before adding the Cisco core Connectrix MDS 9513 director to the homogeneous Connectrix B fabric, verify the following fabric - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 155
MDS 9513. Refer to the "Switched Fabric Topology Parameters" section of the Dell EMC Simple Support Matrices to obtain a list of all the Connectrix MDS series switches that can be supported in a heterogeneous fabric with the Connectrix B switches, as well as the operating modes specified above - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 156
Complex Fibre Channel SAN Topologies Before adding the MDS 9513 to the fabric, the following steps need to be executed on the Connectrix MDS switch: 1. Create a VSAN on the Connectrix MDS switch. A Virtual SAN can be used to create multiple logical SANs over the same physical infrastructure. a. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 157
B switches and the Connectrix MDS switch are running the supported firmware versions for the respective interop modes. This can be checked by referring to the Dell EMC Simple Support Matrices entries for these switches 5. Create an ISL between the Connectrix - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 158
Complex Fibre Channel SAN Topologies Phase 3: Moving half of the host and storage ports from the Connectrix B core to the Connectrix MDS 9513 Topology DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 159
them to the Connectrix MDS 9513 core. Again, it is recommended to go follow the "Checkpoints" on page 154 using both Connectrix B and Connectrix MDS supported fabric management tools. Heterogeneous switch interoperability 159 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 160
Complex Fibre Channel SAN Topologies Phase 5: Adding an Connectrix MDS 9216 to the edge Topology MDS 9216 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server Dell Blade Server GEN-000271 Figure 21 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 161
Complex Fibre Channel SAN Topologies d. For VSAN attributes, leave everything other than the InterOperValue and the AdminState at default. VSAN attributes can be seen in the information pane. e. Set InterOperValue to Interop-1 and the AdminState to Suspended. f. Statically assign VSAN membership for - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 162
Complex Fibre Channel SAN Topologies Phase 6: Moving hosts and storage to a new edge Topology MDS 9216 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server Dell Blade Server GEN-000272 Figure 22 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 163
Complex Fibre Channel SAN Topologies switches apply to setting up a heterogeneous Connectrix MDS-Connectrix MDS fabric from the ground up. Phase 7: Adding a Connectrix MDS switch to the core Topology MDS 9140 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 MDS 9513 ED-24000B ED-12000B SW32R1812 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 164
now exists. This is a complete migration from one switch type (Connectrix B) to another type (Connectrix MDS). IMPORTANT In a Connectrix MDS-only fabric a regular zoneset or an IVR zoneset. This issue is seen on EMC-supported Brocade releases: v5.3.x; v6.1.x. The workaround is to bounce (disable and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 165
for managing the fabric: - Cisco Fabric Manager - Web Tools only for Connectrix B switches Refer to the Dell EMC Simple Support Matrices for the latest supported firmware versions on the Cisco MDS switches for interop with Brocade/Connectrix B switches running FOS v6.0.x and higher. When migrating - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 166
), and Dell (SW4016). Refer to the "Switched Fabric Topology Parameters" section of the Dell EMC Simple Support Matrices for a list of other Brocade/EMC Connectrix switches supported in a heterogeneous setup, and for relevant operating modes. The following information discussed in the "Connectrix - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 167
"Case study #1" on page 152, sets Connectrix B switches to interop mode, links them with an ISL to Connectrix MDS switches (operating in their supported Interop-1 mode), and then pulls out the Connectrix B cores from the fabric. Since this is a Connectrix B-only fabric with all switches operating in - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 168
Complex Fibre Channel SAN Topologies Verify that the all host HBA ports and storage ports that are logged into the fabric are listed in the name server. ◆ Proper distribution of Zoning information. Verify that the active zoneset comprises zones that contain the desired mapping of host and storage - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 169
MDS 9513. Refer to the "Switched Fabric Topology Parameters" section of the Dell EMC Simple Support Matrices for a list of Cisco/Connectrix MDS switches that are supported in a heterogeneous configuration with Connectrix B switches. In addition, operating modes are listed. Before adding a Connectrix - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 170
Complex Fibre Channel SAN Topologies 2. Set port settings on the Device Manager for the Connectrix MDS 9513: a. Select the desired switch, in this case the Connectrix MDS 9513. The Device Manager for this switch displays. b. Select the ports participating in this fabric and set the port speed to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 171
Complex Fibre Channel SAN Topologies Phase 3: Moving half of the host and storage ports from the Connectrix B core to the Connectrix MDS 9513 Topology DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 172
Complex Fibre Channel SAN Topologies Phase 4: Completely moving the host and storage ports from the Connectrix B core to the Connectrix MDS 9513 Topology DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 173
Complex Fibre Channel SAN Topologies Phase 5: Adding a Connectrix MDS 9216 to the edge Topology MDS 9216 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server Dell Blade Server GEN-000271 Figure 28 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 174
Complex Fibre Channel SAN Topologies c. Fill in the VSAN ID field with an unused ID number and the VSAN name field with an appropriate name. In this case study, the VSAN ID number = 801 is assigned. d. For VSAN attributes, leave everything other than the InterOperValue and the AdminState at default. - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 175
Complex Fibre Channel SAN Topologies Phase 6: Moving hosts and storage to a new edge Topology MDS 9216 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 ED-12000B ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server Dell Blade Server GEN-000272 Figure 29 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 176
Complex Fibre Channel SAN Topologies Phase 7: Adding a Connectrix MDS switch to the core Topology MDS 9140 DS-16B2 DS-8B3 DS-32B2 DS-4100B MDS 9513 MDS 9513 ED-24000B ED-12000B SW32R1812 SW32R1812 SW4016 SW4016 DS-16B2 DS-220B IBM Blade Server Dell Blade Server GEN-000273 Figure 30 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 177
-only fabric. This is a complete migration from one switch type (Connectrix B) to another type (Connectrix MDS). IMPORTANT In a Connectrix MDS-only fabric, switches to activate zoning changes. This caveat is specific to EMC-supported Brocade FOS v6.1.x. Cisco Inter VSAN Routing (IVR) in a - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 178
Complex Fibre Channel SAN Topologies change the core PID of one of the switches. IVR allows the Connectrix MDS switch to act as a conduit between two different legacy switch interop modes. CAUTION RDI mode must be explicitly enabled in IVR configurations which will use interop mode 1 to connect to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 179
Complex Fibre Channel SAN Topologies Note: Fabric Manager displays an error message if all the switches participating in IVR do not have unique Domain IDs.These switches must be reconfigured before configuring IVR. d. Enter the VSAN ID of the VSAN you want to use as the transit VSAN between the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 180
For more detailed information on setting up different types of IVR or IVR-NAT configurations refer to the such as zone set activation, reactivation or deactivation for zones that do not require IVR services) should always be initiated from the IVR border switch (the IVR-enabled Connectrix MDS - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 181
instructions or non-default settings that need to be configured on all switches belonging to a specific vendor type have been set up to work in a supported interop mode. ◆ Verify that the E_D_TOV and ; if necessary, refer to the appropriate user manual for information on how to set up operating - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 182
-disruptive and seamless migration. Note: At the time of this release, EMC supports Interop-4 using RPQ for migration purposes only. Refer to the Dell EMC Simple Support Matrices for the most up-to-date support. ◆ Manage all IVR-based zoning in a heterogeneous environment with Cisco switches using - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 183
version of switch firmware and Fabric Management software. The vendor-specific non-default settings that must be enabled on each of the supported vendor switches before introducing them into an interop environment follow: Configure a Brocade switch Telnet can be used to change the Domain ID - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 184
can be set depending on the requirement. At the time of this release, Dell EMC supports Interop-4 using RPQ for migration purposes only. Refer to the Dell EMC Simple Support Matrices for the most up-to-date support. 10. Click Create. The new VSAN appears in the left pane. 184 Fibre Channel SAN - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 185
switches when operating in Interop Fabric Mode include: Brocade switches ◆ QuickLoop ◆ QuickLoop Fabric Assist ◆ Remote Switch ◆ Extended Fabrics ◆ Trunking ◆ Secure Fabric OS ◆ Alias Server ◆ Platform Service ◆ Virtual Channels ◆ FCIP Heterogeneous switch interoperability 185 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 186
in Cisco Fabric mode VSANs. ◆ The Quality of Service feature is intended to provide nodes with high bandwidth needs and greater access to the fabric resources . Note: For the most current support information, refer to the Dell EMC Simple Support Matrices. 186 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 187
Brocade core PID=1) N/A Connectrix MDS/ Cisco Interop-2 (if Brocade core PID=0) Interop-3 (if Brocade core PID=1) Interop-1 N/A Not supported Native Interop-1 Interop-2 Interop-3 Interop-4 Heterogeneous interoperability test information Interoperability test plan A list of tests that EMC E-Lab - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 188
Complex Fibre Channel SAN Topologies ◆ Port type and Speed sensing (1/2/4 G performance based tests) ◆ NDCLA: Online code loads ◆ Firmware downgrade/upgrade backward compatible to two (2) firmware revisions ◆ Threshold alert-based tests ◆ Routing Table - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 189
by an end user. E-Lab can also now take point-in-time snapshots of any configuration for future reference when confirming a fix or reproducing a problem. More about the UCS 2910 The UCS 2910 is an Optical to Electrical to Optical (O-E-O) physical layer switch that handles all serial communication - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 190
Complex Fibre Channel SAN Topologies 190 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 191
provides the following information to monitor your SAN to limit errors that can impact performance: ◆ Introduction 192 ◆ Switch-based error types 194 ◆ Fabric resiliency features and recommendations 200 ◆ Brocade fabric resiliency concepts 208 ◆ Configuring FS features case study 215 ◆ Summary - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 192
Monitoring your SAN Introduction As storage needs continue to grow, SANs are not only expanding in size, but are also becoming more complex and more applicationintensive. Over the years, high density, high accessibility, and high scalability have been the quality aspects associated with SANs. So, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 193
how they can be detected in a Brocade or Connectrix B switch fabric. Next, current firmware features that can be used to detect the different types of switch errors, along with recommendations to prevent error impact on other switch components and device functionality in the SAN, are offered. A case - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 194
of errors, pausible causes, and error displays. Error type Invalid Cyclic Redundancy Checks (CRC), or frame errors CRC errors indicate framing or bit errors that can occur on any link with media or transmission problems. The following are some specific areas within the FC frame that can cause - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 195
the "FC SAN Concepts" chapter of the Networked Storage Concepts and Protocols TechBook at Dell EMC E-Lab Navigator, Documentation > Topology Resource Center. Error type Signal integrity errors on an internal link between the switch ASIC and SERDES can also trigger a CRC. It is important to look at - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 196
while the switch was in an online state. Possible causes Possible causes include: ◆ Link reset (LR) A link reset indicates a buffer-to-buffer credit problem between two connected FC ports. The objective of a link reset is to reset the outstanding credit balance between the connected ports. When an - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 197
Monitoring your SAN Error type ◆ Loss of signal A loss of signal condition implies that the light energy being received on a switch port, which has been sequence) or Status Check condition. This causes the affected frame and associated data sequence to be resent. Switch-based error types 197 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 198
with frames but with ordered sets such as IDLEs, R_RDYs, and other primitives. Ordered sets are used to distinguish between data and the various types of control information. Ordered sets are four character/byte transmission words that all begin with the special character/byte K28.5. The next three - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 199
. The counter that should be examined is: er_bad_os counter - Invalid ordered sets State Changes (ST) State changes include the link failure, mainly loss of sync type of errors. The possible causes for these errors are the same as discussed in "Loss of sync" on page 196, including bad physical media - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 200
to prevent data corruption, unavailability, and data loss conditions due to any unpredictable occurrence of the errors described in "Switch-based error types" on page 194. The recommended topologies and SAN design tips discussed in this section have been drafted to enhance load balancing and data - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 201
, making it more accurate. Bottleneck Detection does not require a license and is supported on both 4 and 8 Gb/s platforms. Edge Hold Time (EHT) configuration to apply EHT to target ports because a target port can service a large number of initiators. A large number of frame drops on a - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 202
note that all the fabric resiliency features are non-disruptive except for port fencing. With port fencing, the user is notified about the error type and the affected port gets disabled, blocking all the traffic going through it. User intervention is required to re-enable the fenced port. Disabling - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 203
disabled or fenced. Refer to Table 5 on page 205 for error types and threshold values. Quick reference for steps to address switch-based errors study: Brocade CLI and CMDCE" on page 215. In this section, the error-types for which the steps have been provided include: ◆ "CRC errors, ITWs, State - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 204
inspection and cleaning kit provided by JDSU to address optical contamination based problems in optical networks. (http://www.jdsu.com/en-us/Test-and- issue isolation and rectification steps for the different error types discussed in"Switch-based error types" on page 194. Tables include: ◆ Table 5, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 205
" on page 215. The threshold numbers provided apply to SAN topologies that follow design best practices. Configuration reference table Error types CRC errors ITW State Changes Port fencing Enable Moderate Thresholds: Low 5 High 20 Aggressive Threshold: High 2 Conservative Thresholds: Low 5 High - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 206
the notification that the user should expect, but does not provide the exact syntax of the error messages. Notification reference table Error types Port fencing Bottleneck detection Edge hold time CRC errors User will be notified based N/A N/A on the kind of event notification selected - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 207
about a fenced port or a bottleneck condition (with or without Edge Hold Time). Issue isolation and rectification reference table Error types CRC errors ITW State Changes Link failures C3 Discards Port fencing Bottleneck Detection Edge Hold Time Recommended rectification steps: N/A N/A Look - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 208
How latency on a switch can propagate through the fabric ◆ Different types of latencies based on severity ◆ How Bottleneck Detection, along with to the transmitting switch to which it is directly attached fast enough to support the offered load, even though the offered load is less than the maximum - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 209
on Switches C and B 1. Buffer credits exhausted due to latency bottleneck Switch A Switch B Switch C Figure 31 5. Connection to other storage devices being serviced by the affected servers will now be at risk too Slow drain storage device (SDSD) Primary port affected Secondary ports affected - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 210
Monitoring your SAN Another type of bottleneck is caused due to congestion wherein a port is unable to storage Switch A Switch B Switch C Figure 32 3. Connection to other storage devices being serviced by the affected ISLs will now be at risk too Slow drain storage device Primary ports affected - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 211
device in returning credits to the switch, the more severe the problem. The following latency severities are discussed in this section: ◆ "Moderate switches or the C3 (Class 3) discards discussed in "Switch-based error types" on page 194. Since the effect of device latencies often spreads through - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 212
Monitoring your SAN which the misbehaving device is connected but also on E_Ports carrying traffic to the F_Port. Dropped frames typically cause I/O errors that result in a host retry and can result in significant decreases in application performance. The implications of this behavior are - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 213
can be used to quarantine a high latency device and mitigate the impact on the fabric (8 Gb/s platform support available in Brocade FOS 6.3 and later; 4 Gb/s platform support available in Brocade FOS 6.3.1b and later). Brocade recommends enabling port fencing for transmit timeouts on F_Ports. Once - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 214
Monitoring your SAN Edge Hold Time This applies primarily to a core edge switch topology.To reduce frame drops on E_Ports on core switches, the edge switches can be configured to have a shorter Hold Time compared to the core switches by using the Edge Hold Time feature (available in Brocade FOS - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 215
Monitoring your SAN Configuring FS features case study This section includes information for the following case study: ◆ "Case study: Brocade CLI and CMDCE" on page 215 Case study: Brocade CLI and CMDCE This case study example demonstrates how the Brocade CLI and the Connectrix Manager Data Center - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 216
not displayed that way in the illustration below. Although the Queues located near each port are intended to indicate shared memory type of buffers, the same type of issues can also be experienced in environments utilizing virtual output queues. Frame from initiator port 0 Frame from initiator port - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 217
Initiator Monitoring your SAN Frame from initiator port 0 Frame from initiator port 1 Switch A Domain ID 1 Q 1 Q Target 1 (4 Gb/s) 7 WWPN: 10:00:00:00:00:05:00:00 Port 0 WWPN: 10:00:00:00:00:01:00:00 Port 1 WWPN: 10:00:00:00:00:02:00:00 1 3Q 4Q Switch B Domain ID 2 Q 2 Figure 34 2 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 218
Monitoring your SAN Initiator Frame from initiator port 0 Frame from initiator port 1 Port 0 WWPN: 10:00:00:00:00:01:00:00 Port 1 WWPN: 10:00:00:00:00:02:00:00 1 3Q 4Q Switch B Domain ID 2 Q 2 Switch A Domain ID 1 Q 1 2 Switch C Domain ID 3 456 Figure 35 Buffer Queue for port 7 continues to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 219
Bottleneck Detection Finding a slow drain, especially without the use of a protocol analyzer, is one of the most challenging performance problems to troubleshoot. A slow drain device causes a latency bottleneck and backpressure across the fabric and, in this cas,e leads to congestion bottlenecks - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 220
Monitoring your SAN On the edge switches A and C with target port connections, complete the following steps: 1. Log in with admin level privileges. 2. Enter bottleneckmon --enable to enable Bottleneck Detection on the F_Ports or ports where the target devices are attached by using the following - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 221
Monitoring your SAN The settings are configurable in the event that a user has specific reasons for modifying them or if the design best practices have not been followed but, in most cases, the default settings should not be changed. There are several reasons they should not be changed. For example - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 222
Monitoring your SAN 5. If the bottleneckmon --enable -alert option is selected, RASlog alerts will be sent when the bottleneck conditions at a port exceed a specified threshold. If the alert parameter is not specified, alerts are not sent, but a history of bottleneck conditions for the port can be - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 223
Fabric Watch and is used to enable error reporting on all ports of a specified type and configure the ports to report errors for a specific area. Supported port types include E_Ports and F_Ports. The specified port type can be configured to report errors for one or more areas. This case study uses - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 224
in steps 1 and 2, execute the following steps: • To display that port fencing has been enabled (with a sample output): Switch_A:admin> portfencing -show Port Type Area PF Status E-port CRC enabled ITW enabled LR disabled PE disabled ST disabled FOP-port CRC ITW LR C3TX-TO PE ST - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 225
Monitoring your SAN Using CMDCE Step 1 through Step 4 in the "Using the CLI," section can can be execute d e using CMDCE by completing the following steps: 1. On the main interface, go to Monitor > Fabric Watch > Port Fencing, as shown in Figure 37. Figure 37 Port fencing dialog box The Port - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 226
Monitoring your SAN 2. Select Violation Type: C3 Discard Frames (Fabric OS only) and click Add to create a port fencing threshold that can be applied to all E_Ports or F_Ports on the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 227
Monitoring your SAN 3. Create a customized threshold for the C3 discard errors as shown Figure 39. Figure 39 Create a customized threshold 4. Apply them to the F_Ports on Switch_A as shown in Figure 40. Figure 40 Apply customized threshold Configuring FS features case study 227 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 228
the port has been fenced due to an error, based on the error type, the user needs to intervene, fix the condition causing the error, Series switches is 500ms. However, this does not provide a solution to the problem with the slow drain since average frame latency time through a switch is between - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 229
SAN The Edge Hold Time value is persistently stored in the configuration file. All configuration file operations, such as configupload and configdownload, are supported for this feature. Note: This setting is available only in Brocade FOS v6.3.1b and later. Configuring FS features case study 229 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 230
Monitoring your SAN Summary SAN performance monitoring tools are gaining more attention with some of the newer switch firmware releases. Just as proper SAN administering and design are considered best practices to manage your SAN, proper SAN monitoring best practices are also vital. Effective SAN - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 231
Invisible Body Tag 5 Brocade Virtual Fabrics Case Study This chapter provides a case study for Brocade Virtual Fabrics. ◆ Brocade Virtual Fabrics case study overview 232 ◆ Objectives of Virtual Fabrics architecture 233 ◆ Logical Switch capability 234 ◆ Virtual Fabrics and inter-switch links - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 232
in terms of protocol and management. Each Logical Switch has its own fabric services (name server, zoning, etc.), configuration (port, switch, fabric, etc Brocade Fabric OS v6.2.x provides solutions to these problems. The Virtual Fabrics feature is supported on the Connectrix ED-DCX-B, ED-DCX-4S-B, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 233
Brocade Virtual Fabrics Case Study Objectives of Virtual Fabrics architecture The objective of the Virtual Fabrics architecture is to provide the following: ◆ Virtualize hardware boundaries Traditionally, the SAN design/management is performed at the granularity of a physical switch. The Virtual - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 234
"Default Logical Switch" on page 234 ◆ "Logical Switch" on page 235 ◆ "Base Switch" on page 236 When Virtual Fabrics is enabled on a switch that supports the VF feature, the switch transforms into a Default Logical Switch, which is like a regular L2 switch. All the ports in the switch now belong to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 235
that is being created. Note: Some types of ports and blades cannot be removed in the EMC Connectrix B Series Administrator's Guide for FOS v6.2.x. Table 8 A port product are shown in Table 8. Number of Logical Switches supported per product Product Maximum number of Logical Switches per chassis - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 236
Base switches connecting to each other must match. Base Switches are also used for FCR support/connectivity. All EX_Ports on a switch must be a part of the Base Switch. Base Switches do not support direct device connectivity; therefore, a Base Switch must have only E_Ports or EX_Ports. 236 Fibre - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 237
Brocade Virtual Fabrics Case Study Virtual Fabrics and inter-switch links This section briefly discusses the following: ◆ "Dedicated ISLs (DISLs)," next ◆ "Conventional ISLs" on page 237 ◆ "Extended ISL (XISLs) and Logical ISLs (LISLs)" on page 238 Dedicated ISLs (DISLs) An interswitch link (ISL - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 238
Brocade Virtual Fabrics Case Study Figure 43 ISL connection between Logical Switches and non-VF-capable switch Extended ISL (XISLs) and Logical ISLs (LISLs) An ISL connecting two Base Switches is called an XISL. In Figure 44 on page 239, there is no DISL connecting the two Logical Switches of the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 239
Brocade Virtual Fabrics Case Study Figure 44 XISL connection between Base Switches (BS) As shown in Figure 44, LISL connections are automatically formed between Logical Switches LS 1 and LS2. Virtual Fabrics and inter-switch links 239 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 240
Brocade Virtual Fabrics Case Study How to configure Brocade Virtual Fabrics case study This case study converts topology (A), as shown in Figure 45 on page 240 to topology (B), as shown in Figure 46 on page 241. General layout The example for this case study on Virtual Fabrics comprises of two - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 241
Brocade Virtual Fabrics Case Study Figure 46 Topology B example Note the following: ◆ The Virtual Fabrics (VF) feature is only supported on the ED-DCX-4S-B, ED-DCX-B, and DS-5300B switches in this configuration. ◆ The colored blocks within the ED-DCX-4S-B, ED-DCX-B, and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 242
other switches). ◆ The VF-capable switches (ED-DCX-4S-B, ED-DCX-B, DS-5300B) are running EMC-supported FOS v6.2.x and up and the other switches are running compatible EMC-supported FOS versions. Note: Refer to the Dell EMC Simple Support Matrices (ESM) for the most up-to-date information on Dell EMC - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 243
Brocade Virtual Fabrics Case Study Objectives of this case study This case study can be used as a reference to meet the following objectives: ◆ To enable VFs on the VF-capable switches ◆ To partition a physical switch into multiple Logical Switches ◆ To create base switches that can be used to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 244
be run to verify that the Virtual Fabrics is enabled: switch:admin> fosconfig --show FC Routing service: disabled iSCSI service: Service not supported on this Platform iSNS client service: Service not supported on this Platform Virtual Fabric: enabled In the case of our example, verify that all the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 245
Brocade Virtual Fabrics Case Study c. Disable the base switch. switchdisable d. Configure the switch attributes, including assigning a unique domain ID by running the following command configure - Enter y at the Fabric Parameters prompt. Note: As a best practice and for ease of management, EMC - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 246
Brocade Virtual Fabrics Case Study Note: All ports on the chassis, including the ICL ports, are initially assigned to the default logical switch (FID 128). If ICLs are being deployed in the base switch, then all ports associated with those ICLs must be assigned to the base switch. a. On each of the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 247
Brocade Virtual Fabrics Case Study where 127 is the fabric ID of the base switch where the ICL ports are currently present. b. All the ICL ports that were disabled by default must now be enabled by running the following commands: iclcfg --enable 3/0 (press enter) iclcfg --enable 3/1 (press enter) - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 248
Brocade Virtual Fabrics Case Study Note: When the Logical Switch is created, it is automatically enabled and empty - that is, it does not have any ports in it. After creating the logical/base switch, the user must disable the switch to configure it and set the domain ID. Then the user must assign - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 249
Brocade Virtual Fabrics Case Study - Ctrl D can be used to save changes and exit from this menu. e. Enable the base switch by running the following command: switchenable f. The Logical Switches get a default switch name assigned to them. The user can change the name of these base switches by - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 250
Brocade Virtual Fabrics Case Study Use the following steps s to configure the first row in the above table above; that is, to add ports 1/44, 1/46, 1/47 to the Logical Switch with FID 2 and DID 4 on the ED-DCX-4S-B (172.23.199.4) with switch DID 4. a. On the ED-DCX-4S-B (DID 4) prompt, the context - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 251
Host A Host B 300F 828C DFA1 DFA5 FID 2 1/0 DID:6 1/1 FID 8 1/2 DID:6 1/3 ED-DCX-B (172.23.199.6) FID128 Yellow storage 7FA (AD59) 8 VF not supported DID:7 9 10 DS-5000B (172.23.199.7) 1/44 1/46 1/47 1/45 -1/289 FID 2 DID:4 FID 8 DID:4 FID 6 1/48 DID:4 FID 127 3/0 3/1 DID - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 252
Brocade Virtual Fabrics Case Study The following is an example for one of the Logical Switches, that is, the Logical Switch (LS) with FID 2, LS DID 4 in the ED-DCX-4S-B (DID 4). a. On the ED-DCX-4S-B (DID 4) prompt the context must be set to the Logical Switch to that has to be enabled. For our - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 253
Brocade Virtual Fabrics Case Study Virtual Fabrics ID 4 6 8 Switch Switch FID ID ED-DCX-4S-B 5 4 DS-5300B 8 4 DS-4900B 9 - ED-DCX-4S-B 5 6 DS-5300B 8 6 ED-DCX-4S-B 4 6 ED-DCX-4S-B 5 8 ED-DCX-B 6 8 ED-DCX-4S-B 4 8 Logical Color Switch ID 5 Green 8 - 5 Yellow 8 4 5 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 254
Brocade Virtual Fabrics Case Study This can be verified by running the switchshow command on every Logical Switch and validating that the "Allow XISL use" attribute is set to ON. Note: If Logical Switches have the option to use either a DISL or the XISL, they will use the DISL to route traffic, - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 255
Brocade Virtual Fabrics Case Study Zoning with Virtual Fabrics Each Virtual Fabric has its own zoning configuration. The following table specifies the devices that are attached to each Virtual Fabric, as can also be seen from Figure 47 on page 251. Virtual Fabric ID Initiator color/ WWN Target - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 256
Brocade Virtual Fabrics Case Study Effective configuration: CFG VF2_cfg Zone: HostA_RedHBA_Symm_7FB 10000000c938300f 500009720813ad60 7. One Logical Switch from each of the other FIDs: 8, 4, 6, can be selected and Step 1 through Step 6 must be executed. The zone information on Logical Switches on - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 257
zoning configuration on the switch, use the following command: cfgshow For information on the following show commands and for details on the supported platforms for Virtual Fabrics, Virtual Fabrics interaction with other FOS features, and limitations and restrictions of Virtual Fabrics, refer to the - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 258
Brocade Virtual Fabrics Case Study Brocade Virtual Fabrics versus traditional Cisco Virtual SANs This section explains why Brocade Virtual Fabrics (VF) configuration needs more planning on the user-front than Cisco's Virtual SANs (VSAN) configuration. For more basic information on VSANs, refer to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 259
Brocade Virtual Fabrics Case Study VSAN This is not the case with Cisco VSANs. When VSANs get created, the switches that are participating in a VSAN retain their original Domain ID so the user does not have to reconfigure them. Zoning information may be shared between Logical Switches, even when - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 260
Brocade Virtual Fabrics Case Study Switch name must be configured on the Logical Switches VF When a Logical Switch (LS) is created on a switch, the new LS uses a default switch name. Therefore, in a given Virtual Fabric all Logical Switches will end up with the same switch name if the user does - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 261
VFs VF Brocade's FCR technology can be used to route traffic between different VFs. A Base Switch in the fabric will be used for legacy FCR support. All EX_Ports need to be part of the Base Switch. The EX_Ports will be disabled if they are in non-base switch. "Extended ISL (XISLs - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 262
Brocade Virtual Fabrics Case Study 262 Fibre Channel SAN Topologies TechBook - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 263
Invisible Body Tag 6 FICON Topologies This chapter provides the following information on FICON topologies. ◆ Overview 264 ◆ Topology support 266 ◆ Zoning practices 267 ◆ Cascading 268 ◆ Terminology 269 ◆ IOCP considerations 271 ◆ FICON and Dell EMC ControlCenter 273 ◆ CUP (fabric management - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 264
Signaling), and FC-3 (Common Services). FICON is an FC-4 type. It was designed as a replacement for ESCON to support mainframe attached CKD (Count Key km droop 100 MB/s). The FICON technologies described in this Topology Guide are organized as follows: ◆ General FICON connectivity information - - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 265
to the following for more information: ◆ Connectrix documentation on Dell EMC Online Support: ◆ HCD/IOCP: http://www.ibm.com/servers/resourcelink Select Library, then http://www.ibm.com/redbooks: • FICON Native Implementation and Reference Guide, PN SG24-6266-01 • Getting Started with the Brocade M - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 266
FICON and SRDF on the same switch ◆ Cascading (multiswitch fabric support) ◆ Size of fabric ◆ Interoperability among switch vendors ◆ basis. Refer to the following for more information: Connectrix switch series Topology Guide reference M series "Connectrix MDS series" on page 278 B series " - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 267
that all FICON N_Ports receive their SCNs. Zoning based on World Wide Port Name (WWPN) is recommended, but port-based zoning is also supported. This practice varies from the Single Initiator zoning of Open Systems, because mainframe channels do not depend on the name server for device discovery - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 268
is a limitation of one 'hop' between switches for cascaded FICON. Refer to the following for more information: Connectrix switch series Topology Guide reference M series "Connectrix MDS series" on page 278 B series "Connectrix B series" on page 275 MDS series "Connectrix MDS series" on page - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 269
Terminology FICON Topologies ◆ Entry switch - A FICON switch that is connected to the processor's FICON channel(s) and to a cascaded switch. An entry switch can also be a cascaded switch. ◆ Cascaded switch - A FICON director that connects to a control unit (, for example, a storage array) and to - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 270
the Fabric Binding and Insistent Domain ID features installed. The channel checks this during initialization by sending the Security Attributes Extended Link Services query to the switch. In the response from the switch, the channel checks that Fabric Binding Enforcement and Insistent Domain ID bits - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 271
ports 04, 08, 0C, 10 Storage array CU 8100 Figure 48 Cascaded IOCP A cascaded FICON IOCP based on this figure would look like this: Type keyword CHPID will operate in FICON native mode, required for cascading Switch keyword Logical switch number Channel entry switch.id, required for FICON (FC - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 272
FICON Topologies Once a two-byte link address has been specified for a channel path: ◆ It is recommended that you specify two-byte link addresses for all paths from the same CEC that have the same entry switch. • This allows IOCP to perform better checking of the switch configuration. • HCD - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 273
switch management functions (including zoning) are qualified on a switch vendor basis. Refer to the specific switch vendor information for the latest support. Any time there is more than one management control point, care must be taken to insure that administrative conflicts do not arise - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 274
: ◆ HCD - Port activation/deactivation ◆ HCM - Performance display ◆ SMF - Performance statistics gathering for RMF ◆ ZOS - Error handling, problem determination ◆ DCM - Dynamic CHPID management The services provided by CUP are used by IBM management software System Automation for OS/390. CUP is not - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 275
-12000B ◆ ED-24000B ◆ ED-48000B ◆ DS-4900B ◆ PB-48K-18i blade (FC-IP) ◆ Silkworm 7500 (FC-IP) Configuring Topology support ◆ Single switch is supported. ◆ Cascading is supported with FOS 5.2.1 or higher • Cascading is not supported with ED-12000B. ◆ Intermixing FICON and FCP on the same switch is - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 276
). OCP considerations Switch ID Definition - No offsets on switch ID or port address, but every value must be in hex for the mainframe. CUP support CUP is a separately licensed inband management service installed on each switch. You can manage CUP using Web Tools or Fabric Manager. Limited - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 277
Refer to the IBM document FICON Director Programming Interface with Cascading Support. Switch node identifier You can find switch node information such as IBM-proprietary document FICON Director Programming Interface With Cascading Support. Dell EMC documentation Refer to these documents for more - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 278
" on page 280 ◆ "Switch node identifier" on page 281 ◆ "FICON port numbering" on page 282 ◆ "References" on page 283 Supported products These MDS series products support FICON connectivity: ◆ MDS 9513 ◆ MDS 9509 ◆ MDS 9506 ◆ MDS 9216 ◆ MDS 9216A ◆ MDS 9216i Requirements Requires the purchase and - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 279
for FICON is required, create multiple ISLs dedicated to the FICON VSAN. ◆ Port Channels for FICON or mixed FCP/FICON traffic is supported. FICON ports must be bound to a Port Channel. Recommended FICON environment configuration settings ◆ Configure ports that are connected to 1 GB/s channels for - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 280
Switch ID Definition - No offsets on switch ID or port address, but every value must be in hex for the mainframe. CUP support Control Unit Port (CUP) is supported by switches and directors in the Cisco MDS 9000 Family. The CUP function allows the mainframe to manage the Cisco MDS switches. CUP - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 281
of ports per card. You can find more information on CUP functions and commands in the IBM-proprietary document FICON Director Programming Interface With Cascading Support. Connectrix MDS series 281 - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 282
FICON Topologies FICON port numbering Table 11 represents the logical port number for the MDS 9216, MDS 9506, and MDS 9509. Table 11 FICON port numbering in the MDS 9000 family Product MDS 9200 series MDS 9506 Director MDS 9509 Director Slot Implemented port allocation number To ports To - Dell VNX5600 | Fibre Channel SAN Topologies TechBook - Page 283
on FICON connectivity: ◆ Dell EMC E-Lab Navigator, for supported MDS firmware for FICON usage ◆ Connectrix MDS Release Notes, for specific information related to new firmware ◆ Cisco MDS 9000 Family Configuration Guide, available at http://www.Cisco.com ◆ Implementing the Cisco MDS 9000 in
-
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8
-
9
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
32
-
33
-
34
-
35
-
36
-
37
-
38
-
39
-
40
-
41
-
42
-
43
-
44
-
45
-
46
-
47
-
48
-
49
-
50
-
51
-
52
-
53
-
54
-
55
-
56
-
57
-
58
-
59
-
60
-
61
-
62
-
63
-
64
-
65
-
66
-
67
-
68
-
69
-
70
-
71
-
72
-
73
-
74
-
75
-
76
-
77
-
78
-
79
-
80
-
81
-
82
-
83
-
84
-
85
-
86
-
87
-
88
-
89
-
90
-
91
-
92
-
93
-
94
-
95
-
96
-
97
-
98
-
99
-
100
-
101
-
102
-
103
-
104
-
105
-
106
-
107
-
108
-
109
-
110
-
111
-
112
-
113
-
114
-
115
-
116
-
117
-
118
-
119
-
120
-
121
-
122
-
123
-
124
-
125
-
126
-
127
-
128
-
129
-
130
-
131
-
132
-
133
-
134
-
135
-
136
-
137
-
138
-
139
-
140
-
141
-
142
-
143
-
144
-
145
-
146
-
147
-
148
-
149
-
150
-
151
-
152
-
153
-
154
-
155
-
156
-
157
-
158
-
159
-
160
-
161
-
162
-
163
-
164
-
165
-
166
-
167
-
168
-
169
-
170
-
171
-
172
-
173
-
174
-
175
-
176
-
177
-
178
-
179
-
180
-
181
-
182
-
183
-
184
-
185
-
186
-
187
-
188
-
189
-
190
-
191
-
192
-
193
-
194
-
195
-
196
-
197
-
198
-
199
-
200
-
201
-
202
-
203
-
204
-
205
-
206
-
207
-
208
-
209
-
210
-
211
-
212
-
213
-
214
-
215
-
216
-
217
-
218
-
219
-
220
-
221
-
222
-
223
-
224
-
225
-
226
-
227
-
228
-
229
-
230
-
231
-
232
-
233
-
234
-
235
-
236
-
237
-
238
-
239
-
240
-
241
-
242
-
243
-
244
-
245
-
246
-
247
-
248
-
249
-
250
-
251
-
252
-
253
-
254
-
255
-
256
-
257
-
258
-
259
-
260
-
261
-
262
-
263
-
264
-
265
-
266
-
267
-
268
-
269
-
270
-
271
-
272
-
273
-
274
-
275
-
276
-
277
-
278
-
279
-
280
-
281
-
282
-
283
 |
 |
Fibre Channel SAN Topologies
Version 4.
2
•
Fibre Channel Topology Overview
•
Simple and Complex Fibre Channel SAN Topologies
•
Brocade Virtual Fabrics Case Studies
•
FICON Topologies
Erik Smith
Richard Hultman
Dennis Kloepping