Cisco ACI Service Graph Management Models Explained

Cisco ACI provides three distinct approaches to manage service graphs, each offering different levels of control and integration:

1. Network Policy Mode (Unmanaged)
   In this mode, ACI configures only the network aspects of the service graph within the fabric. No configuration changes are pushed to the L4-L7 device, making it suitable when device policies are managed externally.

2. Service Policy Mode (Managed)
   Here, ACI not only handles the fabric configuration but also manages VLAN settings on the L4-L7 device. The APIC administrator can directly input device-specific configurations through the APIC interface, ensuring centralized control.

3. Service Manager Mode
   This model allows the firewall or load balancer administrator to define L4-L7 policies. ACI takes care of the fabric and VLAN configurations, while the APIC administrator links these policies with the network policy, enabling a collaborative approach.

Choosing the Right Cisco ACI Service Graph Mode

When designing with Cisco ACI, selecting the right service graph mode depends on your operational needs:

• Dynamic Configuration Needs?
  If firewalls and load balancers must be configured dynamically through APIC, choose Service Policy Mode. If a separate administrator handles device configuration, opt for Network Policy Mode or Service Manager Mode.

• Frequent Commissioning Like Cloud Services?
  For environments where devices are frequently added or removed, Service Policy Mode or Service Manager Mode works best. If services remain static for long periods, Network Policy Mode or Service Manager Mode is more practical.

• Complex Multi-Leg Designs?
  If your design requires multiple interfaces or DMZ configurations, manual service insertion using EPGs and bridge domains may be more convenient than using a service graph.

Bottom Line:
Your choice should align with automation needs, operational flexibility, and design complexity.

Cisco ACI - Node Stateful Vs Stateless reload

 

Aspect	Stateful Reload	Stateless Reload
Definition	Reload where process state is preserved using checkpoints	Reload where process starts fresh without any prior state
State Preservation	Yes – runtime state is saved to Persistent Storage Services (PSS)	No – process is restarted without retaining previous state
Recovery Speed	Faster – resumes from last known state	Slower – requires full reinitialization
System Impact	Minimal – seamless continuation of operations	Higher – may cause temporary disruption or delay
Use Case	Preferred for critical services needing quick recovery	Used when state cannot be preserved or process needs a clean start
Managed By	Persistent Storage Services (PSS)	System Manager
Example Scenario	Restarting a service with session data intact	Replacing a crashed process with a new instance

 

ACI Contracts Components - Contacts Vs Filters Vs Aliases Vs Labels

 In Cisco ACI (Application Centric Infrastructure), Contracts are a key component of the policy model, used to define how endpoints (EPGs) communicate with each other. Within contracts, the terms Labels, Filters, Aliases, and Subjects each play distinct roles. Here's a breakdown of each:

---

🔹 1. Filters

• Purpose: Define the actual traffic (protocols, ports) that is allowed or denied.
• Details:
• Filters are composed of entries specifying Layer 4 information like TCP/UDP ports and protocols.
• They are reusable across multiple contracts.
• Example: A filter might allow TCP traffic on port 80 (HTTP).

---

🔹 2. Subjects

• Purpose: Act as containers within a contract that reference filters and define directionality.
• Details:
• A contract can have multiple subjects.
• Each subject can reference one or more filters.
• You can specify whether the traffic is unidirectional or bidirectional.
• Example: A subject might define that HTTP traffic is allowed from EPG A to EPG B.

---

🔹 3. Aliases

• Purpose: Provide a way to abstract or alias filters for reuse or simplification.
• Details:
• Aliases are less commonly used and are more relevant in complex policy models.
• They can help in referencing filters indirectly, making policy definitions more modular.

---

🔹 4. Labels

• Purpose: Used for categorization and policy enforcement.
• Details:
• Labels can be applied to contracts, EPGs, and other objects.
• They help in grouping and applying policies based on tags.
• Useful in large environments for automation and policy scaling.

---

🧩 How They Work Together in a Contract

• A Contract contains one or more Subjects.
• Each Subject references one or more Filters (or Aliases to filters).
• Labels can be used to tag contracts or EPGs for organizational or policy purposes.

 

Taboo vs vzAny Contract

 

Aspect	Taboo Contract	vzAny Contract
Purpose	Used to explicitly deny certain types of traffic for an EPG	Used to apply contracts to all EPGs within a VRF in a simplified manner
Functionality	Acts as a deny filter for specific traffic (e.g., block port 80 or 23)	Acts as a wildcard to apply contracts to all EPGs in a VRF
Application Scope	Applied to individual EPGs to block traffic	Applied to entire VRF, affecting all EPGs within it
Use Case	Prevent specific traffic types (e.g., cleartext communication)	Enable free intra-VRF communication or many-to-one service consumption
Configuration Complexity	Requires manual filter creation and careful application	Simplifies configuration by automating contract relationships
TCAM Usage	May consume more TCAM entries depending on filter granularity	Optimizes TCAM usage by reducing entries to a single group-level contract
Best Practices	Generally discouraged unless absolutely necessary 1	Recommended for efficient policy management in large-scale environments 2 3
Limitations	Not suitable for inter-EPG communication control	Must follow strict guidelines to avoid unintended traffic leaks 2 3

Recovering and Reinitializing a Standby APIC to active APIC in Cisco ACI

 Recovering and Reinitializing a Standby APIC in Cisco ACI

If you're working with a standby APIC and need to verify access or promote it to an active role, here’s a practical approach to follow.

---

🔐 Step 1: Accessing the Standby APIC

You can attempt to log in using the rescue-user account. This account is specifically designed for emergency access to standby controllers.

• Try using your standard APIC admin password first.
• If the standby APIC is in sync with the cluster, this should work.
• If not, attempt login without a password — some configurations allow password-less access for rescue-user in isolated standby mode.

---

🔄 Step 2: Reinitializing the Standby APIC as Active

If you need to promote a standby APIC (e.g., APIC3) to an active role, it's best to wipe and reconfigure it from scratch. Ensure you have KVM access to the APIC before proceeding.

Run the following commands in sequence:

1. Clean the existing configuration.
2. Trigger the setup script on next boot.
3. Reboot the APIC to begin the reinitialization process.

These commands will:

• acidiag touch clean
• acidiag touch setup
• acidiag reboot

Once rebooted, the APIC will prompt you to configure it as part of the active cluster.

---

🧠 Pro Tip

Always ensure that the APIC you’re reinitializing is physically connected to the fabric and that you have console or KVM access. This process is irreversible and should only be done when you're certain the APIC is no longer needed in standby mode.

 

 Cold Standby APIC – Key Characteristics

• Supported in both single-pod and multi-pod ACI deployments.
• Can be connected to any leaf switch in any pod, restoring edit capabilities in minority scenarios.
• Automatically receives firmware updates to stay in sync with the active APIC cluster.
• During upgrades, once all active APICs are updated, the standby APIC is upgraded automatically.
• Assigned temporary node IDs; a new ID is issued when promoted to active.
• Admin login is disabled on standby APICs.
• Troubleshooting is done via SSH using the rescue-user account.
• During switchover, the replaced active APIC is powered down to avoid dual connectivity.
• Standby APICs do not participate in policy configuration or fabric management.
• Cisco recommends placing standby APICs in the same pod as the active ones they may replace.
• No configuration or credential data is replicated to standby APICs — only rescue-user access is available.

 

L3Out Subnet Scope Options in Cisco ACI

 

L3Out Subnet Scope Options in Cisco ACI

Scope Option	Purpose
Export Route Control Subnet	Advertises specific transit routes out of the ACI fabric. Used for controlling which external routes are exported.
Import Route Control Subnet	Allows specific external routes to be imported into the ACI fabric using BGP or OSPF.
External Subnets for External EPG (Security Import Subnet)	Enables data plane traffic between external EPGs and internal EPGs. Without this, traffic is dropped even if routes are learned.
Shared Route Control Subnet	Used in inter-VRF route leaking. Marks subnets that should be advertised across VRFs.
Shared Security Import Subnet	Similar to External Subnets, but for shared L3Outs. Enables traffic flow between shared external and internal EPGs.
Aggregate Export/Import/Shared Routes	Used to summarize routes, e.g., adding /32 in front of 0.0.0.0/0 for default route aggregation.

---

🧠 Key Notes

• These scope options are critical for controlling route advertisement and enabling secure communication between different parts of the network.
• Cisco ACI uses a contract-based model, so even if routes are advertised, traffic will be dropped unless explicitly allowed by contracts and security prefixes 

Comparison Between CDP and LLDP in Cisco ACI

 Comparison Between CDP and LLDP in Cisco ACI 

Note :- Both CDP and LLDP can be enabled at the same time.

Feature	CDP (Cisco Discovery Protocol)	LLDP (Link Layer Discovery Protocol)
Vendor Support	Cisco proprietary	Vendor-neutral (IEEE 802.1ab standard)
Protocol Layer	Data Link Layer	Data Link Layer
Device Discovery Scope	Cisco devices only	Cisco and non-Cisco devices
Communication Type	Periodic advertisements (multicast)	One-way advertisements
Information Shared	Protocol addresses, platform, SNMP address, hold-time	Device capabilities, identity, configuration via TLVs
TLV Support	Limited to Cisco-defined TLVs	Standardized TLVs (Type-Length-Value)
Max Neighbors per Port	Up to 256	One device per port
ACI Support (from Release 4.2(1))	Supported on leaf/spine management interfaces	Supported on leaf/spine management interfaces
ACI Configuration Scope	Can be enabled globally across fabric	Can be enabled globally across fabric
ACI Use Case	Troubleshooting cabling issues, especially in unstaffed sites	Same as CDP
ACI Interface Support	Physical interfaces and port channels only	Same, but not supported on FEX interfaces
ACI VLAN TLV Limitations	Not applicable	Only 25 VLANs advertised; name TLV limited to 32 characters
ACI Infra-VLAN Advertisement	Not specified	Not advertised even if enabled
ACI Routed Sub-interface VLANs	Not specified	Not advertised
Default Behavior on Fabric Ports	Not supported between fabric-connected interfaces	Enabled by default on fabric ports

ACI BFD Support Overview

 ACI BFD Support Overview

• Supported Protocols: BFD is supported for BGP external routed networks.
• Purpose: BFD provides sub-second failure detection between ACI leaf switches and external routers, improving convergence and reliability.
• Multihop BFD: Supported from APIC Release 5.0(1) onward.
• C-bit-aware BFD: ACI supports control-plane-aware BFD, allowing you to configure whether BFD sessions are dependent or independent of the control plane

---

🔧 Configuration Guidelines

To configure BFD in ACI:

1. Create or Edit a BGP Peer Connectivity Profile:
• Navigate to L3Out > Node Profile > BGP Peer Connectivity Profile
• Enable BFD under the peer settings.
2. Ensure Loopback Interfaces:
• For multihop BFD, loopback interfaces are required.
• Each L3Out should have a unique loopback IP for BGP peering.
3. MTU Considerations:
• ACI does not support IP fragmentation, so ensure MTU is properly configured on both ends.
• Recommended: Test MTU using CLI tools like ping df-bit packet-size.
4. Platform Compatibility:
• BFD support may vary slightly depending on the ACI hardware model (e.g., N9K-C9336PQ, N9K-C93180YC-EX).
• Always verify compatibility in the release notes or hardware documentation.

---

⚠️ Limitations

• BFD is not supported for OSPF or EIGRP in ACI.
• BFD configuration is only applicable to external routed networks (L3Out), not internal fabric routing.

 

COOP (Council of Oracle Protocol) and ZMQ (Zero Message Queue)

 In Cisco ACI (Application Centric Infrastructure), both COOP (Council of Oracle Protocol) and ZMQ (ZeroMQ) play critical roles in the control-plane communication between switches (leaves and spines), but they serve different purposes and operate at different levels. Let’s break down the difference and their relationship:

---

🔹 What is COOP in Cisco ACI?

📌 Purpose:

COOP is the control-plane protocol used by leaf switches to register endpoint information with spine switches in Cisco ACI.

📌 Function:

• It is used by leaf switches to tell the spine(s) about endpoints (MAC, IP, EPG, etc.) they’ve learned.
• Spines act as COOP databases and maintain a distributed mapping table of which leaf has which endpoint.
• This allows for location-based routing (instead of flooding like in traditional Ethernet).

---

🔹 What is ZMQ (Zero Message Queue) in Cisco ACI?

📌 Purpose:

ZMQ (ZeroMQ) is a messaging library used within Cisco ACI to transport messages between system components (e.g., between leaf and spine switches).

📌 Function:

• Acts like a messaging bus.
• Enables high-performance publish/subscribe or request/response messaging.
• It’s used under the hood to deliver control-plane messages—including COOP messages.

---

✅ Does COOP Use ZMQ?

Yes.

COOP uses ZMQ as its transport mechanism to send and receive messages between leaf and spine switches.

---

🔍 How Does COOP Use ZMQ?

Here’s the flow simplified:

1. A leaf switch learns a new endpoint (say, a VM with MAC/IP).
2. The leaf constructs a COOP message with the endpoint details.
3. This COOP message is encapsulated in a ZMQ message.
4. The ZMQ library sends this message to the appropriate spine switch (COOP database).
5. The spine decodes the COOP message and updates its COOP database.

Diagrammatically:

less

CopyEdit

[Leaf Switch]

    |

    |--> [COOP Message Created]

    |

    |--> [Wrapped in ZMQ Message]

    |

    |--> [ZMQ Sends Message to Spine]

    |

[Spine Switch (COOP DB)]

    |

    |--> [ZMQ Receives Message]

    |

    |--> [COOP Message Extracted & DB Updated]

---

🆚 Summary: COOP vs ZMQ

Feature	COOP	ZMQ
Purpose	Control-plane protocol for endpoint learning	Messaging library used for data transport
Scope	Endpoint registration between leaf & spine	Messaging between ACI components
Layer	Application-layer protocol	Transport mechanism (middleware)
Relationship	Payload Protocol	Transport Protocol
Used By	Leaf-to-Spine communication	All ACI components (APICs, Leafs, Spines)

---

✅ Example of Usage:

• You might see coop messages being passed between leaf and spine switches in packet captures.
• If you dig deeper, those messages are often encapsulated using ZMQ frames, showing how COOP rides on top of ZMQ.

 

Complete Steps to Create vPC in Cisco ACI (via APIC GUI)

 Understanding vPC in Cisco ACI: A Modern Approach to High Availability

In the evolving landscape of data center networking, Virtual Port Channel (vPC) stands out as a cornerstone of high availability and link redundancy. While traditional NX-OS environments rely on CLI-driven configurations, Cisco ACI reimagines vPC through a policy-driven, intent-based model that aligns with the fabric’s overarching design philosophy.

Unlike legacy setups, ACI abstracts physical connectivity into logical constructs, allowing administrators to define vPC behavior through interface policy groups, switch profiles, and attachable access entity profiles (AAEPs). This not only simplifies deployment but also ensures consistency across the fabric.

At its core, a vPC in ACI enables two leaf switches to present a unified uplink to a downstream device—be it a server, firewall, or load balancer—without relying on spanning tree protocols. The result is active-active forwarding, improved bandwidth utilization, and seamless failover.

In this guide, we’ll walk through the step-by-step configuration of vPC in Cisco ACI, demystifying each component and highlighting best practices to ensure a robust and scalable deployment.

Note:- In Cisco ACI, a Fabric Extender (FEX) can be integrated using a port channel in a straight-through topology, where each FEX connects directly to a leaf switch. While vPCs can be established between hosts and the FEX for redundancy and load balancing, the FEX itself does not support vPC connectivity to multiple leaf switches.

 Complete Steps to Create vPC in Cisco ACI (via APIC GUI)

Step 1: Leaf Onboarding (One-by-One)

🔍 Monitor Discovery in APIC

1. Log in to the APIC GUI
2. Navigate to:
   Fabric → Inventory → Fabric Membership → Nodes Pending Registration
3. Wait for Leaf101 to appear
• You’ll see its Serial Number
• Node Role: Leaf
• Status: Blank / Not Registered

---

📝 Register Leaf101

1. Right-click on Leaf101’s serial number
2. Click Register
3. In the registration window, enter:
• Node ID: 101
• Node Name: Leaf101
• Click Register
• Wait for it to appear in Registered Nodes

---

📝 Register Leaf102

1. Repeat the same steps for Leaf102:
• Wait for it to appear in Nodes Pending Registration
• Right-click → Register
• Enter:
• Node ID: 102
• Node Name: Leaf102
• Click Register
• Wait for it to appear in Registered Nodes

---

🔢 Step-by-Step ACI Configuration Flow

2. VLAN Pool (VLAN 113)

• Navigate to:
  Fabric → Access Policies → Pools → Right Click on VLAN and click Create Vlan Pool
• Create VLAN Pool:
• Name: VLAN_113_Pool
• Mode: Static
• Click + under Encap Blocks

Ø  Range:  113 – 113

Ø  Allocation mode: Static

• Click Ok - >Submit

---

3. Domain (Physical Domain)

• Go to:
  Fabric → Access Policies → Physical and External Domains ->Right Click on Physical domain ->
• Create Physical Domain:
• Name: PhysDom_VLAN113
• VLAN Pool: VLAN_113_Pool
• Click Submit

---

4. AEP (Attachable Access Entity Profile)

• Navigate to:
  Fabric → Access Policies → Policies-> Global → Right Click on Attachable Access Entity Profiles -> Click Create Attachable Access Entity Profiles
• Create AEP:
• Name: AEP_VLAN113
• Click + under Domains and Associated Domain: PhysDom_VLAN113
• Click Update ->Next -> Finish

---

5. Interface Policy Group (vPC)

• Go to:
  Fabric → Access Policies → Interface → Leaf Interfaces - >Policy Groups->Right click on VPC Interfaces - >Create VPC Interfaces
• Create VPC Interface Policy Group:
• Name:  vPC_LF101_LF102_1_1
• AEP: AEP_VLAN113
• Port Channel Policy: system-lacp-Active
• Link Level Policy: system-link-level-XG-Auto
• Click Next > Finish

6. Create vPC Policy (Your Mentioned Step)

• Go to:
  Fabric → Access Policies → Policies → Switch
• Right-click on Virtual Port Channel Default

Name:VPC_101_102

ID:10

VPC Domain Policy: Default

Switch1: Leaf101

Switch2: Leaf102

This step ensures the vPC behavior is defined at the switch policy level.

7. Interface Profile

• Navigate to:
  Fabric → Access Policies → Interface → Leaf Interface -> Profiles
• Right click on the interface profile and click Create Interface Profile:
• Name: IntProf_Leaf101_102
• Click + under Interface Selector:
• Name: Eth1_4
• Interface ID: 1/4
• Policy Group: vPC_LF101_LF102_1_1
• Click Ok - > Submit

 

8. Switch Profile

• Go to:
  Fabric → Access Policies → Switches → Profiles
• Right Click on Profile and click Create Leaf Profile:
• Name: LeafProf_101_102
• Click + under Leaf Selector:
• Name: Leaf101_102
• Node Block: From 101 to 102
• Click Update -> Next
• Attach Interface Profile:
• IntProf_Leaf101_102

---

9. Create Tenant

• Navigate to:
  Tenants
• Click Add Tenant
• Name: Tenant_WebApp
• Click Submit

10. Create VRF

• Navigate to:
  Tenants
• Click Networking -> VRF -> Right click on VRF -> click Create VRF
• Name: WebApp_VRF
• Uncheck Create A Bridge Domain
• Click Finish

11 Create BD

• Navigate to:
  Tenants
• Click Networking -> Bridge Domain -> Right click on Bridge Domain-> click Create Bridge Domain
• Name: WebApp_BD
• VRF: WebApp_VRF
• Click Next
• Click + under Subnet

Ø  Gateway IP: 10.1.1.1/24

Ø  Check “Make this IP address Primary”

Ø  Scope: check “Advertised Externally”

• Click OK -> Next-> Finish

 

12. Create Application Profile (AP)

• Inside Tenant_WebApp, go to:
  Application Profiles
• Right Click on Application Profile  and Create Application Profile:
• Name: WebApp_AP
• Click Submit

13. Create Endpoint Group (EPG)

• Inside WebApp_AP, go to:
  EPGs
• Right Click on Application EPG and click Create Application EPG:
• Name: WebApp_EPG
• Bridge Domain: WebApp_BD
• Click Finish
• Right Click on WebApp_EPG and click ADD Physical Domain Association:
• Domain Association: PhysDom_VLAN113
• Click Submit

---

14. Create Contract (Allow TCP Port 80)

• Go to:
  Tenant_WebApp → Contracts -> Standard
• Right Click on Standard -> Click Create Contract:
• Name: Allow_HTTP
• Click + under Subject:
• Name: HTTP_Subject
• Filter: Click + under Filter
• Click + Under Name
• Name: HTTP_Filter
• Click + under Entries
• Name: HTTP_Entry
• EtherType: IP
• IP Protocol: TCP
• Destination Port: From http – To http
• Click Update-> Submit
• Click Update -> Ok -> Submit
• Provide contract to/from EPG as needed

---

15. Static Binding of EPG to Port

• Go to Tenant WebApp_EPG-> Application Profiles -> WebApp_AP ->  Application EPGs -> WebApp_EPGs
• Right Click on WebApp_EPGs -> Click Deploy Static EPG on PC,VPC, or Interface
• Path Type: Virtual Port Channel
•  Path: Leaf101/eth1/4 and Leaf102/eth1/4
• Mode: Trunk
• Encapsulation: vlan-113