Thursday, 7 August 2025

Comparison Between CDP and LLDP in Cisco ACI

 Comparison Between CDP and LLDP in Cisco ACI 

Note :- Both CDP and LLDP can be enabled at the same time.

Feature

CDP (Cisco Discovery Protocol)

LLDP (Link Layer Discovery Protocol)

Vendor Support

Cisco proprietary

Vendor-neutral (IEEE 802.1ab standard)

Protocol Layer

Data Link Layer

Data Link Layer

Device Discovery Scope

Cisco devices only

Cisco and non-Cisco devices

Communication Type

Periodic advertisements (multicast)

One-way advertisements

Information Shared

Protocol addresses, platform, SNMP address, hold-time

Device capabilities, identity, configuration via TLVs

TLV Support

Limited to Cisco-defined TLVs

Standardized TLVs (Type-Length-Value)

Max Neighbors per Port

Up to 256

One device per port

ACI Support (from Release 4.2(1))

Supported on leaf/spine management interfaces

Supported on leaf/spine management interfaces

ACI Configuration Scope

Can be enabled globally across fabric

Can be enabled globally across fabric

ACI Use Case

Troubleshooting cabling issues, especially in unstaffed sites

Same as CDP

ACI Interface Support

Physical interfaces and port channels only

Same, but not supported on FEX interfaces

ACI VLAN TLV Limitations

Not applicable

Only 25 VLANs advertised; name TLV limited to 32 characters

ACI Infra-VLAN Advertisement

Not specified

Not advertised even if enabled

ACI Routed Sub-interface VLANs

Not specified

Not advertised

Default Behavior on Fabric Ports

Not supported between fabric-connected interfaces

Enabled by default on fabric ports



ACI BFD Support Overview

 ACI BFD Support Overview

  • Supported Protocols: BFD is supported for BGP external routed networks.
  • Purpose: BFD provides sub-second failure detection between ACI leaf switches and external routers, improving convergence and reliability.
  • Multihop BFD: Supported from APIC Release 5.0(1) onward.
  • C-bit-aware BFD: ACI supports control-plane-aware BFD, allowing you to configure whether BFD sessions are dependent or independent of the control plane



🔧 Configuration Guidelines

To configure BFD in ACI:

  1. Create or Edit a BGP Peer Connectivity Profile:
    • Navigate to L3Out > Node Profile > BGP Peer Connectivity Profile
    • Enable BFD under the peer settings.
  2. Ensure Loopback Interfaces:
    • For multihop BFD, loopback interfaces are required.
    • Each L3Out should have a unique loopback IP for BGP peering.
  3. MTU Considerations:
    • ACI does not support IP fragmentation, so ensure MTU is properly configured on both ends.
    • Recommended: Test MTU using CLI tools like ping df-bit packet-size.
  4. Platform Compatibility:
    • BFD support may vary slightly depending on the ACI hardware model (e.g., N9K-C9336PQ, N9K-C93180YC-EX).
    • Always verify compatibility in the release notes or hardware documentation.

⚠️ Limitations

  • BFD is not supported for OSPF or EIGRP in ACI.
  • BFD configuration is only applicable to external routed networks (L3Out), not internal fabric routing.

 

Wednesday, 6 August 2025

ACI Leaf as Ethernet Hub - Spanning tree handing in ACI

 

🔁 ACI Leaf as Ethernet Hub (Behavioral Analogy)

  • ACI leaf switches forward BPDUs transparently between connected devices.
  • This behavior mimics a hub, where multiple devices share the same broadcast domain.
  • Therefore, STP decisions and transitions are influenced by how the connected switch interprets the topology.

 P2P Mode (Rapid Convergence)

  • When a switch receives a Proposal BPDU on a P2P link:
    • It can immediately respond with an Agreement BPDU.
    • This allows the sender to transition from Blocking to Forwarding without waiting for timers.
  • This is ideal for RSTP-enabled switch-to-switch links.

🕒 Shared Mode (Delayed Convergence)

  • On a Shared link, the receiving switch cannot send an Agreement immediately.
  • The sender must wait for the Forward Delay timer to expire before transitioning.
  • This introduces latency in STP convergence.

🔄 Impact Across All ACI Versions

  • This behavior is consistent across all ACI firmware versions.
  • It’s crucial to explicitly configure STP link-type on external switches connected to ACI leafs to ensure optimal convergence.

 ACI Port Configuration Best Practices for External Switches

1. Determine the Nature of the Connection

Connection Type

Recommended STP Link-Type

Reason

Switch-to-Switch (Trunk or Access)

Point-to-Point (P2P)

Enables rapid STP convergence via RSTP

Switch-to-Hub or Shared Media

Shared

Prevents premature forwarding; slower convergence

Legacy or non-RSTP switch

Shared

Ensures compatibility with older STP implementations


2. ACI Interface Policy Configuration

In ACI, configure the following under Access Policies:

  • Interface Policy Group:
    • Enable STP Interface Policy
    • Set Link Type to either point-to-point or shared based on the external device
  • Attach the Interface Policy Group to the appropriate Leaf Interface Profile


4. Avoid STP Misconfigurations

  • Ensure BPDU Guard is disabled on ACI ports connected to switches.
  • Avoid enabling PortFast on external switch ports facing ACI unless it's an edge port.
  • Monitor STP topology changes to detect misbehaving devices.

5. Use LLDP/CDP for Visibility

Enable LLDP/CDP on both ACI and external switches to:

  • Verify connectivity
  • Identify misconfigured ports
  • Assist in troubleshooting

 


Tuesday, 5 August 2025

COOP: Council of Oracle Protocol - Cisco ACI

COOP: Council of Oracle Protocol – A Modern Overview

The Council of Oracle Protocol (COOP) serves as a critical mechanism for transmitting endpoint mapping data—such as identity and location—from leaf switches to spine proxies within a network. 

This communication is facilitated using Zero Message Queue (ZMQ), enabling leaf switches to relay endpoint details to a designated spine switch known as the "Oracle."

Spine nodes running COOP maintain a synchronized repository of endpoint mappings, ensuring consistency across the network. Additionally, COOP manages a Distributed Hash Table (DHT) that stores identity-to-location mappings, forming the backbone of the protocol’s database infrastructure.

To prioritize secure and efficient data transport, COOP uses high-priority channels and encrypted connections. Security is further reinforced through MD5-based authentication, which safeguards COOP messages against unauthorized traffic injection. Both the APIC controller and network switches support this authentication mechanism.

COOP now supports two distinct ZMQ authentication modes:

  • Strict Mode: Only MD5-authenticated ZMQ connections are permitted, ensuring maximum security.
  • Compatible Mode: Allows both authenticated and non-authenticated ZMQ connections, offering flexibility for diverse network environments.

 

Integrating COOP with Cisco APIC: Secure ZMQ Authentication in ACI Fabric

To enable secure communication across the Cisco Application Centric Infrastructure (ACI), the Application Policy Infrastructure Controller (APIC) incorporates support for COOP Zero Message Queue (ZMQ) authentication. This includes the use of MD5-based password protection and a secure operational mode for COOP messaging.


Configuration of COOP ZMQ Authentication Type

A new managed object, coop:AuthP, has been introduced within the Data Management Engine (DME) under the COOP database path (coop/inst/auth). This object allows administrators to define the authentication mode for COOP ZMQ connections. By default, the mode is set to "compatible", permitting both authenticated and unauthenticated connections. For environments requiring stricter security, the mode can be switched to "strict", which enforces MD5 authentication exclusively.


Managing the MD5 Password for COOP Authentication

The APIC also provides a managed object named fabric:SecurityToken, which includes a dynamic attribute called "token". This token serves as the MD5 password and is refreshed automatically every hour. COOP receives update notifications from the DME to ensure the password remains current. For security reasons, the actual token value is not exposed or displayed.

COOP Strict Mode Behavior During ACI Fabric Upgrades

When performing an upgrade across the Cisco ACI fabric, the system temporarily disables COOP strict mode until all switches have completed the upgrade process. This safeguard is designed to prevent disruptions in COOP communication—specifically, it avoids the risk of a switch rejecting COOP connections due to premature enforcement of strict authentication. By deferring strict mode activation, the fabric ensures seamless interoperability and avoids authentication mismatches during transitional states.


Configuring COOP Authentication Policy in Cisco ACI

Using the Cisco APIC GUI

To set the COOP authentication mode through the APIC interface:

  1. Navigate to System > System Settings from the top menu.
  2. In the left-hand Navigation pane, select COOP Group.
  3. In the Work pane, locate the Policy Property section. Under the Type field, choose either:
    • Compatible Type – allows both authenticated and unauthenticated ZMQ connections.
    • Strict Type – enforces MD5 authentication for all ZMQ connections.
  4. Click Submit to apply the changes.

This completes the configuration of the COOP authentication policy via the APIC GUI.


Using the Cisco NX-OS-Style CLI

To configure COOP authentication using the command-line interface:

This sets the COOP authentication mode to strict, ensuring that only MD5-authenticated ZMQ connections are accepted.

apic1# configure 

apic1(config)# coop-fabric 

apic1(config-coop-fabric)# authentication type ? 

compatible Compatible type strict Strict type

apic1(config-coop-fabric)# authentication type strict

COOP (Council of Oracle Protocol) and ZMQ (Zero Message Queue)

 In Cisco ACI (Application Centric Infrastructure), both COOP (Council of Oracle Protocol) and ZMQ (ZeroMQ) play critical roles in the control-plane communication between switches (leaves and spines), but they serve different purposes and operate at different levels. Let’s break down the difference and their relationship:


🔹 What is COOP in Cisco ACI?

📌 Purpose:

COOP is the control-plane protocol used by leaf switches to register endpoint information with spine switches in Cisco ACI.

📌 Function:

  • It is used by leaf switches to tell the spine(s) about endpoints (MAC, IP, EPG, etc.) they’ve learned.
  • Spines act as COOP databases and maintain a distributed mapping table of which leaf has which endpoint.
  • This allows for location-based routing (instead of flooding like in traditional Ethernet).

🔹 What is ZMQ (Zero Message Queue) in Cisco ACI?

📌 Purpose:

ZMQ (ZeroMQ) is a messaging library used within Cisco ACI to transport messages between system components (e.g., between leaf and spine switches).

📌 Function:

  • Acts like a messaging bus.
  • Enables high-performance publish/subscribe or request/response messaging.
  • It’s used under the hood to deliver control-plane messages—including COOP messages.

 Does COOP Use ZMQ?

Yes.

COOP uses ZMQ as its transport mechanism to send and receive messages between leaf and spine switches.


🔍 How Does COOP Use ZMQ?

Here’s the flow simplified:

  1. leaf switch learns a new endpoint (say, a VM with MAC/IP).
  2. The leaf constructs a COOP message with the endpoint details.
  3. This COOP message is encapsulated in a ZMQ message.
  4. The ZMQ library sends this message to the appropriate spine switch (COOP database).
  5. The spine decodes the COOP message and updates its COOP database.

Diagrammatically:

less

CopyEdit

[Leaf Switch]

    |

    |--> [COOP Message Created]

    |

    |--> [Wrapped in ZMQ Message]

    |

    |--> [ZMQ Sends Message to Spine]

    |

[Spine Switch (COOP DB)]

    |

    |--> [ZMQ Receives Message]

    |

    |--> [COOP Message Extracted & DB Updated]


🆚 Summary: COOP vs ZMQ

Feature

COOP

ZMQ

Purpose

Control-plane protocol for endpoint learning

Messaging library used for data transport

Scope

Endpoint registration between leaf & spine

Messaging between ACI components

Layer

Application-layer protocol

Transport mechanism (middleware)

Relationship

Payload Protocol

Transport Protocol

Used By

Leaf-to-Spine communication

All ACI components (APICs, Leafs, Spines)


Example of Usage:

  • You might see coop messages being passed between leaf and spine switches in packet captures.
  • If you dig deeper, those messages are often encapsulated using ZMQ frames, showing how COOP rides on top of ZMQ.

 


Concept of vPC in ACI

Concept of vPC in ACI

In Cisco ACI, a Virtual Port Channel (vPC) enables two separate leaf switches to present a unified port channel to a connected endpoint—such as a server, firewall, or another switch that supports link aggregation protocols like LACP.

In this setup, two ACI leaf nodes (e.g., Leaf201 and Leaf202) act as vPC peers, forming a logical construct known as a vPC domain. One of these peers is elected as the primary, while the other assumes the secondary role.




ACI’s MCT-Based Architecture

Unlike traditional vPC implementations that rely on a dedicated peer-link, ACI leverages the fabric itself to manage synchronization and control-plane communication. This architecture is referred to as Multichassis EtherChannel Trunk (MCT).

🔧 Key Characteristics:

  • No physical peer-link is required between Leaf201 and Leaf202.
  • Instead, the ACI fabric handles all peer communication and synchronization.
  • ZMQ (Zero Message Queue) replaces traditional CFS (Cisco Fabric Services) for messaging between vPC peers.

How Peer Communication Works in ACI

  • ZMQ, a high-performance messaging library using TCP, is embedded as libzmq on each switch.
  • Applications that require peer communication (like the vPC manager) use this library to exchange messages.

🔄 Peer Reachability Mechanism:

  • The vPC manager subscribes to routing updates via URIB.
  • When IS-IS discovers a route to the peer (e.g., Leaf202 sees Leaf201), URIB notifies the vPC manager.
  • The manager then attempts to establish a ZMQ socket with the peer.
  • If the route is withdrawn (e.g., due to link failure), the vPC manager is notified and the MCT link is brought down accordingly.

Upgrade Best Practices with vPC

To ensure high availability during fabric upgrades, it's recommended to divide switches into at least two upgrade groups. For example:

  • Group A: Leaf201, Leaf203, Spine101
  • Group B: Leaf202, Leaf204, Spine102

This strategy ensures that at least one vPC peer remains active during the upgrade, preventing service disruption for connected endpoints.


Glossary

Term

Description

ACI

Application Centric Infrastructure

vPC

Virtual Port Channel

MCT

Multichassis EtherChannel Trunk

ZMQ

Zero Message Queue

URIB

Unicast Routing Information Base

IS-IS

Intermediate System to Intermediate System

LACP

Link Aggregation Control Protocol


 VPC Design Options:-

Option 1 -VPC with SAME Leaf interfaces across two leafs with Combined Profiles



Option 2 -  VPC with SAME Leaf interfaces across two leafs with Individual Profiles.




Option 3 -  VPC with DIFFERENT Leaf interfaces across two leafs with Individual Profiles





Twinax vs DAC Cable: What's the Difference?

 

Twinax vs DAC Cable: What's the Difference?

In high-speed data center environments, Twinax and Direct Attach Copper (DAC) cables are often mentioned interchangeably—but they’re not exactly the same. Understanding their distinctions helps in selecting the right connectivity solution for your Cisco ACI fabric or any modern network deployment.

🔌 Twinax Cable

  • Definition: Twinax (short for twin axial) is a type of cable that uses two conductors within a single shielded cable to transmit differential signals.
  • Use Case: It’s the physical medium used in many short-range, high-speed connections, especially in data centers.
  • Form Factor: Twinax is the underlying cable technology used in DAC cables.

🔗 DAC Cable (Direct Attach Copper)

  • Definition: DAC is a complete cable assembly that includes Twinax cabling with integrated transceivers at both ends (usually SFP+, QSFP+, or QSFP28).
  • Use Case: Commonly used for short-distance connections between switches, servers, and storage devices—typically up to 7 meters.
  • Types:
    • Passive DAC: No signal amplification; ideal for short distances (up to ~5m).
    • Active DAC: Includes signal conditioning electronics; supports slightly longer distances (up to ~10m).

🆚 Key Differences

Feature

Twinax Cable

DAC Cable

Definition

Cable type with twin conductors

Cable assembly with connectors

Includes Transceivers

No

Yes

Application

Used inside DAC or other assemblies

Plug-and-play for switch/server links

Distance Support

Depends on implementation

Typically 1–10 meters