VPC - Back-to-Back configuration example

Below is the VPC Back-to-Back  configuration example.

N7K-1:-

Step1:- Enable Feature VPC

N7K-1(config)# feature vpc

Step 2:- Enable Feature LACP

N7K-1(config)# feature lacp

Step 3:- Create VPC domain. Make sure it is same on the VPC peer otherwise VPC will remain in down state.

N7K-1(config-if-range)# vpc domain 100 N7K-1(config-vpc-domain)# peer-keepalive destination 10.1.1.72 --<< Mgmt IP of N7K-2 Note:  --------:: Management VRF will be used as the default VRF ::--------

We get the below output if domain Id are different on peers switches.

N7K-1(config)# show vpc brief Legend:                 (*) - local vPC is down, forwarding via vPC peer-link vPC domain id                     : 100 Peer status                       : peer link not configured vPC keep-alive status             : peer is alive, but domain IDs do not match  -----<<<<<<< Configuration consistency status  : failed Per-vlan consistency status       : failed Configuration inconsistency reason: vPC peer-link does not exist Type-2 consistency status         : failed Type-2 inconsistency reason       : vPC peer-link does not exist vPC role                          : none established Number of vPCs configured         : 0 Peer Gateway                      : Disabled Dual-active excluded VLANs        : - Graceful Consistency Check        : Disabled (due to peer configuration) Auto-recovery status              : Disabled

We get below output when peer switches are reachable via peer keepalive link.

N7K-1(config-vpc-domain)# show vpc brief Legend:                 (*) - local vPC is down, forwarding via vPC peer-link vPC domain id                     : 100 Peer status                       : peer link not configured  --------------<<<<<<<<<<<<<<<<<<< vPC keep-alive status             : peer is alive  ----------------<<<<<<<<<<<<<<< Configuration consistency status  : failed Per-vlan consistency status       : failed Configuration inconsistency reason: vPC peer-link does not exist  ---<<<<<<<<< Type-2 consistency status         : failed Type-2 inconsistency reason       : vPC peer-link does not exist vPC role                          : none established Number of vPCs configured         : 0 Peer Gateway                      : Disabled Dual-active excluded VLANs        : - Graceful Consistency Check        : Disabled (due to peer configuration) Auto-recovery status              : Disabled

Step 4:- Create port-channel for vpc peer-link. As soon as VPC PEER-LINK command is configured on

port channel the port type is changed to network and bridge assurance is enabled on the port-channel.

N7K-1(config)# int eth1/1-2 N7K-1(config-if-range)# channel-group 1 mode active N7K-1(config-if-range)# no shut N7K-1(config-if-range)# int po1 N7K-1(config-if)# switchport mode trunk N7K-1(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. N7K-1(config-if)# no shut

Step 5:- Configure port-channel connected to 5Ks. Make sure all the interfaces connected to 5Ks are in same port channel.

N7K-1(config)# int eth1/3-4 N7K-1(config-if-range)# channel-group 10 mode active N7K-1(config-if-range)# no shut N7K-1(config-if-range)# int po10 N7K-1(config-if)# switchport mode trunk N7K-1(config-if)#vpc10

N7K-2:-

Step 6:- Enable vpc and lacp feature

N7K-2(config)# feature vpc N7K-2(config)# feature lacp

Step7:- Configure VPC Domain and vpc peer-keepalive link.

N7K-2(config-if-range)# vpc domain 100 N7K-2(config-vpc-domain)# peer-keepalive destination 10.1.1.71 --<< Mgmt IP of N7K-1 Note:  --------:: Management VRF will be used as the default VRF ::--------

Step8:- Configure VPC peer-link

N7K-2(config)# int eth1/1-2 N7K-2(config-if-range)# channel-group 1 mode active N7K-2(config-if-range)# no shut N7K-2(config-if-range)# int po1 N7K-2(config-if)# switchport mode trunk N7K-2(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. N7K-2(config-if)# no shut

Step9:- Configure vpc port-channel connected to N5k

N7K-2(config)# int eth1/3-4 N7K-2(config-if-range)# channel-group 10 mode active N7K-2(config-if-range)# no shut N7K-2(config-if-range)# int po10 N7K-2(config-if)# switchport mode trunk N7K-2(config-if)#vpc10

N5K-1:-

Step 10:- Enable vpc and lacp feature

N5K-1(config)# feature vpc N5K-1(config)# feature lacp

Step 11:- Configure VPC Domain and vpc peer-keepalive link.

N5K-1(config-if-range)# vpc domain 100 N5K-1(config-vpc-domain)# peer-keepalive destination 10.1.1.52 --<< Mgmt IP of N5K-2 Note:  --------:: Management VRF will be used as the default VRF ::--------

Step12:- Configure VPC peer-link

N5K-1(config)# int eth1/1-2 N5K-1(config-if-range)# channel-group 1 mode active N5K-1(config-if-range)# no shut N5K-1(config-if-range)# int po1 N5K-1(config-if)# switchport mode trunk N5K-1(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. N5K-1(config-if)# no shut

Step13:- Configure vpc port-channel connected to N7ks and port channel number must be same on both 5K. Make sure port channel number on 7k and 5K should be different.

For example we have chosen po10 on 7K and po20 on 5K.

N5K-1(config)# int eth1/3-4 N5K-1 (config-if-range)# channel-group 20 mode active N5K-1 (config-if-range)# no shut N5K-1 (config-if-range)# int po20 N5K-1 (config-if)# switchport mode trunk N5K-1 (config-if)#vpc 20

N5K-2:-

Step 14:- Enable vpc and lacp feature

N5K-1(config)# feature vpc N5K-1(config)# feature lacp

Step 15:- Configure VPC Domain and vpc peer-keepalive link.

N5K-1(config-if-range)# vpc domain 100 N5K-1(config-vpc-domain)# peer-keepalive destination 10.1.1.51 --<< Mgmt IP of N5K-1 Note:  --------:: Management VRF will be used as the default VRF ::--------

Step12:- Configure VPC peer-link

N5K-2(config)# int eth1/1-2 N5K-2(config-if-range)# channel-group 1 mode active N5K-2(config-if-range)# no shut N5K-2(config-if-range)# int po1 N5K-2(config-if)# switchport mode trunk N5K-2(config-if)# vpc peer-link Please note that spanning tree port type is changed to "network" port type on vPC peer-link. This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance (which is enabled by default) is not disabled. N5K-2(config-if)# no shut

Step13:- Configure vpc port-channel connected to N7ks .

N5K-2(config)# int eth1/3-4 N5K-2(config-if-range)# channel-group 20 mode active N5K-2(config-if-range)# no shut N5K-2(config-if-range)# int po20 N5K-2(config-if)# switchport mode trunk N5K-2(config-if)#vpc 20

What is Zoning - Storage 12

Zoning is way to restrict the communication between the initiator and the targets. By using Zoning only assigned target is visible to the initiator.

 In Ethernet, host connected to LAN is able to see all the devices connected on the same vlan but in VSAN there is another layer of restriction called zoning to restrict the visibility between initiator and targets. Devices within a zone can able see each other.

There are two types of Zoning available.

1.  Hard Zoning:- It is implemented at the ASIC level and hence devices cannot communicate if hard zoning is a restricting the communication.

It not only creates a barrier to the visibility between devices but also restrict the data transfer between the ports in different zones. It restricts both control and data plane traffic.

Zoning done on the basis of the switchport are called port-based zoning. Sometimes hard zoning also known as port-based zoning.

It can be one of the below three types:-

·  One to one:- Initiator and target are mapped in single zoning configuration and hence initiator can only see one target.

·  One to many:- Initiator and many target are in one zone. Hence many targets are visible to the initiator.

·  Many to many:- Many initiator and many targets are in one zone. A single port can be configured on multiple targets.

Advantage: -

Ø  As it is implemented on the ASIC hardware level, it provides higher level of security as the data can be sent between the ports on same zone only. No data will be allowed to between the hosts of different zone.

Ø  It increases the routing performance as the zoning is applied at the hardware level.

Disadvantage: -

Ø  It is not a flexible solution as devices must be connected to the same port in order to communicate the other devices in the zone. In case of port failure, zoning must be changed on all the switches in the fabric which is a difficult task in large environment.

2.  Soft Zoning:- Zoning based on PWWN and NWWN are known as soft zoning. It is implemented on the operating system level. It only restricts the control packet information not the data plane.

As soon as a device is connected to the fabric, it sends a request to name service about all the devices present in the same zone.

Advantage:-

Ø  It is very flexible as compared to hard zoning.

Disadvantage:-

Ø  Device will not see any devices in other zones. By any chance if the device knows the destination and send the FC frame to the target then Software zoning has no way to restrict the communication.  As a result it is less secure than hard zoning.

Note:- MDS 9000 supports both hard and soft zoning.

Port Types - Storage 11

Below are most common port types in storage.

1. N-Port (Node Port):- Port on the end host device like storage (target) or server (initiator). N-port cannot be configured on the switch side.

2. NL-Port (Node Loop port):- Host that are connected in Arbitrated loop has the NL port.

3. F-port (Fabric Port):- Switchport that are connected to N-port are configured as F-port.

4. FL-Port (Fabric Loop port):- Switchport connected to NL port are configured with FL port.

5. E-Port (Expansion Port):- Port between the switches is configured as E-port. Expansion ports are similar to trunk port in Ethernet. Also known as ISL (Inter switch-link).

6. TE-port (Trunk expansion port):-  It is known as extended ISL which is analogous to 802.1Q to allow multiple VSANs on the E-port. There is no configuration to make a TE port. It is automatically configured as soon as multiple VSANS are configured on a switch.

7. TN-Port (Trunk Node Port):- N-port of hosts that support multiple VSANS are known as TN port. Need of trunk F-port is required in case when host device supports multiple vsan at a time like ESX.

8. TF-Port (Trunk Fabric Port):- Switchport connected to TN port is configured as TF port.

Fibre channel Addressing - Storage 10

1. World Wide Node Name (WWNN):- Sometimes it is also called as NWWN (Node World Wide Name).It is an 8-byte address used to identify device in the fabric.Every device in Fabric has its own WWNN.

No device can have two WWNN. Each node or HBA has one WWNN. If a server has two HBA then they have two WWNN.

2. World Wide Port Name (WWPN):- Also known as PWWN (Port World Wide Name). It is used to identify port on a device (switch or HBA card). Each port on a device has its own PWWN. It is also 8-Byte address.

3. Fabric Assigned Port World Wide Name (FAPWWN):-Some vendors like IBM also support Virtual PWWN called FAPWWN. These can be used to pre-configure the Zoning in order to increase the deployment efficiency.

4. Fabric Channel ID:- Referred to as FCID. Since there is a potential problem if 64 bits PWWN is used for routing, FC has used another address scheme.  FCID is used by the data plane to switch the traffic. FCID persistence is enabled by default to make sure it will not change after the device reboot.

FCID is 24 bit or 3 byte address assigned automatically to each port by switch. A switch can have different FCIDs for different VSANs.

FCID can be manually or dynamically assigned. Some HBAs doesn't allow the targets in the same area, static assignment can be useful in such scenario.

MDS-01# config t MDS-01(config)# fcdomain fcid database MDS-01(config-fcid-db)# vsan 100 wwn 00:00:00:00:00:00:00:01 fcid 0x010101

FCID is made up of three fields:-

• Domain ID: - It is the most significant byte that is a unique identifier assigned to each switch in a fabric. We can have 2 ^8 = 256 address available but few addresses are reserved and hence only 239 addresses can be used. It can either be assigned manually or dynamically.

   Domains ID are assigned by the Principle switch. PS is nothing but a general switch in the fabric that is responsible to assign the Domain IDs. Please refer to the below blog to find the more details about the Domain ID.

• Area ID: - It gives 2^8=256 addresses. In director switches there can be more than 256 switchport. And hence Area ID is the shared between the groups of ports. It cannot be configured and automatically configured by switches. 

• Port ID: - It is used to identify individual ports connected to N or NL port.

VTP type-2 configuration incompatible error in Cisco Nexus switch

If we get VTP type-2 configuration incompatible error in show VPC brief command as shown below.

N7K-6-2(config)# show vPC brief Legend:                 (*) - local vPC is down, forwarding via vPC peer-link vPC domain id                     : 10 Peer status                       : peer adjacency formed ok vPC keep-alive status             : peer is alive Configuration consistency status  : success Per-vlan consistency status       : success Type-2 consistency status         : failed Type-2 inconsistency reason       : VTP type-2 configuration incompatible ---<<<<< vPC role                          : primary Number of VPCs configured         : 0 Peer Gateway                      : Disabled Dual-active excluded VLANs        : - Graceful Consistency Check        : Enabled Auto-recovery status              : Disabled vPC Peer-link status --------------------------------------------------------------------- id   Port   Status Active vlans --   ----   ------ -------------------------------------------------- 1    Po10   up     1,20

Then please make sure VTP version, domain, mode and password is same on the peers.

N7K-6-2# show vpc consistency-parameters interface po10 Note: **** Global type-1 parameters will be displayed for peer-link *****     Legend:         Type 1 : vPC will be suspended in case of mismatch Name                        Type  Local Value            Peer Value -------------               ----  ---------------------- ----------------------- STP Mode                    1     Rapid-PVST             Rapid-PVST STP Disabled                1     None                   None STP MST Region Name         1     ""                     "" STP MST Region Revision     1     0                      0 STP MST Region Instance to  1  VLAN Mapping STP Loopguard               1     Disabled               Disabled STP Bridge Assurance        1     Enabled                Enabled STP Port Type, Edge         1     Normal, Disabled,      Normal, Disabled, BPDUFilter, Edge BPDUGuard        Disabled               Disabled STP MST Simulate PVST       1     Enabled                Enabled VTP domain                  2     lab                    test                ---------------<<<<<< VTP version                 2     1                      1 VTP mode                    2     Server                 Server          ---------------<<<<<< VTP password                2     lab123               test             ----------------<<<<<< VTP pruning status          2     Disabled               Disabled Allowed VLANs               -     1                      1 Local suspended VLANs       -     -                      -

Even if you see the errors then try below steps:- 

1.       Disable and re-enable the VTP feature.

2.       Reconfigure the VTP password.

3.       Change the mode to server to client then back to server.

4.       Also check the layer-2 connectivity between the peers via peer link. It could be a problem with one or more of the peer link member port. Check for errors on the physical link of the peer link port-channel.

Fabric Login Process - Storage Basics-9

Below are the three login processes which are responsible for the creation and maintenance of communication link between initiator and target.

  1. Fabric Login:- It is known as FLOGI. It establishes the session between N-port and F-port. It is mandatory process for point-to-point and Fabric topology, without it N_port will not able to send or receive data. It is optional for Arbitrated loop. It performs below function so that N_port can register to the fabric.

• As soon as N_port is connected to F_port, it will register itself in the fabric by sending the fabric login request. In return F_port will assign a 24-bit FCID to the N-port.
• N_port also negotiate the link flow control parameter i.e. Buffer-to-Buffer credit with F_port.
• If N_port and F_port supports authentication then they are negotiate the authentication parameters during FLOGI.

   N_port sends the Flogi frame (containing WWNN, WWPN and B2B credit) to a well-known address of 0xFFFFFE. In return switch will send the accept frame (ACC).

  2. N-Port Login:- It is also known as PLOGI. It creates a FC4 session between the end devices.  It is responsible to create and maintain the session between two N-Port. During PLOGI, N_port will negotiate the end-to-end credit. PLOGI also enables N_port to learn other characteristics like WWNN, WWPN etc. of other end N_port.

N_port sends the PLOGI on a well-known address of 0xFFFFFC.

  3. Process Login:- Also known as PRLI. It creates and maintains a session between two FC4 processes that are connected via two different N_port between initiator and target.

Disk Subsystem Components - Storage Basics-3

Every intelligent Disk subsystem consists of the below components.

1.SAN:- Server can be connected either directly to the disk subsystem or indirectly via Storage area network. SAN can be made up of SCSI, Fibre channel or iSCSI infrastructure.

2.Connection Points:- Servers are connected to Disk subsystem via connection points. Each subsystem must be connected more than one connection point to provide redundancy. Small Storages have one or two connection points and 6 to 8 hard disks.

3.Controller:- Disk controller are used to handle write and read operation to/from hard disks. With the help of controller, entire disk array appeared as one virtual hard disk to the servers.

Server sends the blocks of data to the controller and its controller responsibility to distribute the data to internal hard disk.

4.Hard disk:-This is the place where actual data is stored. It is very critical to choose the size of the Hard disk as it will limit overall maximum capacity of the storage. More disk in the storage means more read/write heads which will increase the throughput but then maximum capacity of the system will reduce. Application with high throughput will require hard disk with smaller space.

5. Internal IO channels: - It can use vendor proprietary methods or standard IO method i.e SCSI, FC etc.  to connect the controller and internal hard disks.

There are various design methods of Internal IO channel in order to provide redundant path between controller and hard disks.

•    Active: - Every hard disk is connected to controller only via single IO channel. If it is broken hard disk will be isolated from Controller.

It is the cheapest and simplest method to provide the connectivity but not recommended as there is no redundancy.

•    Active/Passive: - There are two traces of IO channel but only one path is active at a time. If primary link is down then controller will use backup link to read and write data to the hard disks.

•    Active/Active No- Load Balancing :- Both the Channel traces are active but hard disk is only using one trace and keeping the other channel as backup.

As shown below Channel-1 is active only for hard disk 1 -3 whereas it is used as backup for Hard disk-2 and 4. Similarly Channel-2 is active for Hard disk 2and 4 and is used as backup for Hard disk1-3.

•    Active/Active Load Balancing: - All hard disks are connected to controller via two separate IO channels and both the channel is used to perform read and write operation by the controller.

It is the best method to provide high fault tolerance.

6. Cache:- It is used by the controller to increase the read and write speed. There are two types of cache.

•     Cache on the hard disk:- Generally the speed of IO channel are higher than the speed at which controller can write on the hard disk. So the data is being cached by the hard disk to make IO channel free and can be used for the data towards other hard disks.
•     Cache on the controller:-Controller has its own cache which caches all the data sent by the servers and allow other servers to send data on the free channel. Controller has its own battery to protect data lose due to power failure.