Cisco ACI Port Security

  Cisco ACI Port Security – Summary

Purpose:
Controls the number of MAC addresses that can be learned on an interface to prevent unauthorized access and MAC flooding.

---

⚙️ Key Features

• MAC Limit: Set a maximum number of MAC addresses per interface (0–12000).
• Protect Mode: Only supported violation action.
• Excess MAC addresses are dropped.
• MAC learning is disabled temporarily.
• Learning resumes after a timeout (default: 60 seconds).
• Supported Interfaces: Physical ports, port channels, and vPCs.
• Monitoring: Faults and syslogs are generated when limits are exceeded.

---

🚫 Restrictions

• Not supported on Fabric Extender (FEX) ports.
• Only MAC address limits are enforced (not MAC+IP).

---

🛠️ Configuration Path in APIC GUI

1. Fabric → Access Policies → Interface Policies → Port Security
2. Create and attach the policy to an Interface Policy Group
3. Bind the group to a Switch Profile

 

Cisco ACI - Fabric Secure Mode Overview

 Fabric Secure Mode Overview

Fabric Secure Mode is a security feature in Cisco ACI that safeguards the infrastructure from unauthorized additions. It ensures that only verified switches and APIC controllers can join the fabric, even if someone has physical access to the equipment.

Starting from release 1.2(1x), Cisco ACI performs a validation check during installation or upgrade. This check confirms that each device has a valid serial number and a Cisco-signed digital certificate.

By default, the system operates in Permissive Mode, allowing existing setups to continue functioning even if some devices lack valid certificates. However, administrators can enable Strict Mode for enhanced security, requiring manual approval for any new device joining the fabric.

---

⚙️ Modes of Operation

Mode	Permissive Mode (Default)	Strict Mode
Device Validation	Valid Cisco serial number and certificate required	Enforces serial number and certificate validation
Existing Fabric	Continues operating even with invalid certificates	Requires all devices to be validated
Authorization	Auto-discovers and allows devices without manual approval	Manual authorization needed for each new device
Security Level	Basic security	Enhanced security and control

---

To change the Fabric Secure Mode in Cisco ACI (e.g., from Permissive to Strict), follow these steps using the Cisco APIC GUI:

🔧 Steps to Change Fabric Secure Mode

1. Log in to the APIC GUI.
2. Navigate to:
   System → System Settings → Fabric Security
3. In the Properties pane, locate the Fabric Secure Mode setting.
4. Select Strict Mode from the available options.
5. Save the configuration.
6. Reboot the APIC and affected switches to apply the change.

⚠️ Important: Changing the mode requires a reboot for the configuration to take effect.

Cisco ACI - Node Stateful Vs Stateless reload

 

Aspect	Stateful Reload	Stateless Reload
Definition	Reload where process state is preserved using checkpoints	Reload where process starts fresh without any prior state
State Preservation	Yes – runtime state is saved to Persistent Storage Services (PSS)	No – process is restarted without retaining previous state
Recovery Speed	Faster – resumes from last known state	Slower – requires full reinitialization
System Impact	Minimal – seamless continuation of operations	Higher – may cause temporary disruption or delay
Use Case	Preferred for critical services needing quick recovery	Used when state cannot be preserved or process needs a clean start
Managed By	Persistent Storage Services (PSS)	System Manager
Example Scenario	Restarting a service with session data intact	Replacing a crashed process with a new instance

 

ACI Node state - undiscovered Vs Unknown

 

  Undiscovered:

• You’ve manually added a node ID in APIC.
• The switch is not yet connected or powered on.
• Could be due to cabling issues or incorrect port configuration.

  Unknown:

• The switch is physically connected and sending LLDP packets.
• APIC detects it but doesn’t have a matching Node ID policy.
• You need to assign a Node ID to complete discovery.

 

What is a Contract Preferred Group in ACI?

 🔷 What is a Contract Preferred Group in ACI?

In Cisco ACI, Endpoint Groups (EPGs) typically require contracts to communicate with each other. This follows the “allow list” model, where communication is explicitly permitted only if a contract exists.

The Preferred Group (PG) feature simplifies this by allowing certain EPGs within the same VRF to communicate freely without contracts.

---

✅ Key Concepts

Term	Description
Included EPGs	EPGs that are part of the preferred group and can communicate with each other without contracts.
Excluded EPGs	EPGs outside the preferred group that still require contracts to communicate.
VRF PG Setting	Must be enabled for the preferred group to work. Without this, even included EPGs won’t communicate freely.

---

🛠️ Configuration Steps

1. Enable Preferred Group on VRF:
• Go to the VRF settings in APIC or Nexus Dashboard Orchestrator (NDO).
• Check the Preferred Group box.
2. Add EPGs to the Preferred Group:
• In the EPG properties, check Include in Preferred Group.
• Save the configuration.
3. Verify Membership:
• You can view all EPGs in the preferred group under the VRF’s properties.

---

🌐 Multi-Site Considerations

• In a stretched VRF across multiple sites, preferred group EPGs are shadowed in other sites to enable inter-site communication.
• This allows, for example, a web EPG in Site 1 to communicate with an app EPG in Site 2 without contracts.

---

⚠️ Limitations

• Preferred Groups are not supported for L3Out external EPGs.
• If vzAny is already consuming/providing a contract in the VRF, you should not configure preferred groups.
• All EPGs in a preferred group must be managed consistently (either all via APIC or all via NDO).

 

ACI Leaf Switch Replacement

 To replace a Cisco ACI leaf switch, follow these step-by-step instructions to ensure a smooth transition without disrupting your fabric:

---

🛠️ Preparation

1. Document the existing switch details:
• POD ID
• Node ID
• Node Name
• Serial Number 

2. Ensure the replacement switch is in ACI mode:
• Connect via console and run show version.
• If in NX-OS mode, convert to ACI mode using Cisco's documented procedure 
• Before adding the new leaf switch to the fabric, ensure it's manually upgraded to the target image or one with a direct upgrade path. Avoid using intermediate images that require multiple upgrade steps, as they can trigger issues and impact your production environment. A final upgrade via policy helps ensure BIOS and FPGA components are properly updated.

3. Clean up the replacement switch:
• Run setup-clean-config.sh and then reload to remove any existing configuration 

---

🔄 Decommission the Faulty Leaf Switch

4. Go to APIC GUI:
   Fabric > Inventory > Fabric Membership
5. Right-click the faulty switch → Select Decommission.
6. Once decommissioned, Remove from Controller and confirm the action 
7. Physically disconnect and unmount the old switch.

---

🔌 Install and Connect the New Leaf Switch

1. Mount the new switch and connect uplinks to spine switches. DONOT CONNECT DOWNLINK AT THIS STAGE
2. Power on the switch.
3. In APIC GUI, go to:
   Fabric > Inventory > Fabric Membership > Nodes Pending Registration
4. Verify serial number, then Register the switch:
• Use the same POD ID, Node ID, and Node Name as the old switch 

5. Once registered, go to:
   Fabric > Inventory > Fabric Membership > Registered Nodes
   → Right-click → Select Commission.
6. Wait for the switch to reach Active state.

---

🔍 Post-Replacement Validation

1. Connect downlink cables (after switch is active).
2. Go to:
   Fabric > Inventory > Topology
   → Verify the switch is visible and operational.
3. SSH into APIC and run:

→ Confirm switch status is active 

4. If you get SSH warnings (e.g., DNS spoofing), update the known_hosts file:

---

🧩 Troubleshooting Tips

• Switch not discovered: Check LLDP neighbors and cable connections.
• Switch shows "Not Supported": Upgrade APIC firmware to match switch model.
• No TEP IP assigned: May be a DHCP issue—contact Cisco TAC.
• SSL issues: Check for established sessions on port 12215 

Common Causes of "Unknown" Leaf State

 

 Common Causes of "Unknown" Leaf State

• Certificate Issues: The leaf might not be presenting a valid certificate chain to the APIC, which prevents proper SSL handshake and authentication.

• LLDP Mismatch or Failure: ACI relies on LLDP (Link Layer Discovery Protocol) for fabric discovery. If LLDP info isn’t exchanged correctly between APIC and leaf, discovery fails.

• Firmware Incompatibility: The leaf switch might be running a version of ACI software that’s not compatible with the APIC or spine switches.

• Hardware Problems: Faulty transceivers, cables, or ports can block communication between APIC and leaf.

• Time Sync Issues: If the leaf’s system time is out of sync with the APIC, certificate validation may fail.

• Incorrect Node ID or Serial Number: If the leaf was previously part of another fabric or misconfigured, it may need to be wiped and re-initialized.

ACI Contracts Components - Contacts Vs Filters Vs Aliases Vs Labels

 In Cisco ACI (Application Centric Infrastructure), Contracts are a key component of the policy model, used to define how endpoints (EPGs) communicate with each other. Within contracts, the terms Labels, Filters, Aliases, and Subjects each play distinct roles. Here's a breakdown of each:

---

🔹 1. Filters

• Purpose: Define the actual traffic (protocols, ports) that is allowed or denied.
• Details:
• Filters are composed of entries specifying Layer 4 information like TCP/UDP ports and protocols.
• They are reusable across multiple contracts.
• Example: A filter might allow TCP traffic on port 80 (HTTP).

---

🔹 2. Subjects

• Purpose: Act as containers within a contract that reference filters and define directionality.
• Details:
• A contract can have multiple subjects.
• Each subject can reference one or more filters.
• You can specify whether the traffic is unidirectional or bidirectional.
• Example: A subject might define that HTTP traffic is allowed from EPG A to EPG B.

---

🔹 3. Aliases

• Purpose: Provide a way to abstract or alias filters for reuse or simplification.
• Details:
• Aliases are less commonly used and are more relevant in complex policy models.
• They can help in referencing filters indirectly, making policy definitions more modular.

---

🔹 4. Labels

• Purpose: Used for categorization and policy enforcement.
• Details:
• Labels can be applied to contracts, EPGs, and other objects.
• They help in grouping and applying policies based on tags.
• Useful in large environments for automation and policy scaling.

---

🧩 How They Work Together in a Contract

• A Contract contains one or more Subjects.
• Each Subject references one or more Filters (or Aliases to filters).
• Labels can be used to tag contracts or EPGs for organizational or policy purposes.

 

Difference between Aliases and Labels - Aliases VS Labels - Cisco ACI

 

Feature	Aliases	Labels
Purpose	Abstract reference to filters for modular contract design	Tagging mechanism for categorization and automation
Used In	Subjects within Contracts	EPGs, Contracts, Bridge Domains, Application Profiles, etc.
Functionality	Indirectly reference filters to simplify reuse	Group and organize objects; used in automation workflows
Visibility	Internal, technical use	Administrative, visible in UI and automation tools
Example Use Case	Reuse a common filter across multiple subjects	Tag all web-tier contracts with web-tier for easy grouping
Impact on Policy	Affects how filters are applied in traffic rules	Helps in applying policies based on tags or roles
Automation Role	Minimal	Significant (used in scripts, templates, etc.)

 

Choosing the Right Cisco 10G SFP+: A Simple Comparison Guide

🟢 Standard Modules

These are full-featured and support everything (including FCoE, OTN, etc.).

• SFP-10G-SR: Short range (up to 400m on multimode fiber)
• SFP-10G-LR: Long range (up to 10km on single-mode fiber)
• SFP-10G-ER: Extended range (up to 40km)
• SFP-10G-ZR: Ultra-long range (up to 80km)

✅ Supports all protocols
✅ TAA compliant
✅ Good for enterprise/data center use

---

🔵 S-Class Modules (e.g., SFP-10G-SR-S, LR-S, ER-S, ZR-S)

These are cost-effective versions of standard modules.

• Same distance and fiber type as standard ones
• ❌ No FCoE support
• ❌ Not TAA compliant
• ✅ Cheaper

Use these if you don’t need advanced features like Fibre Channel over Ethernet.

---

🟠 Special Environment Modules

For harsh or non-standard environments:

• SFP-10G-SR-X: Extended temperature (-5°C to 85°C)
• SFP-10G-SR-I: Industrial temperature (-40°C to 85°C)
• SFP-10G-T-X: Copper (RJ-45), up to 30m at 10G

 

Taboo vs vzAny Contract

 

Aspect	Taboo Contract	vzAny Contract
Purpose	Used to explicitly deny certain types of traffic for an EPG	Used to apply contracts to all EPGs within a VRF in a simplified manner
Functionality	Acts as a deny filter for specific traffic (e.g., block port 80 or 23)	Acts as a wildcard to apply contracts to all EPGs in a VRF
Application Scope	Applied to individual EPGs to block traffic	Applied to entire VRF, affecting all EPGs within it
Use Case	Prevent specific traffic types (e.g., cleartext communication)	Enable free intra-VRF communication or many-to-one service consumption
Configuration Complexity	Requires manual filter creation and careful application	Simplifies configuration by automating contract relationships
TCAM Usage	May consume more TCAM entries depending on filter granularity	Optimizes TCAM usage by reducing entries to a single group-level contract
Best Practices	Generally discouraged unless absolutely necessary 1	Recommended for efficient policy management in large-scale environments 2 3
Limitations	Not suitable for inter-EPG communication control	Must follow strict guidelines to avoid unintended traffic leaks 2 3

What is a Taboo Contract?

 In Cisco ACI (Application Centric Infrastructure), a taboo contract is a special type of contract used to explicitly deny specific types of traffic between endpoint groups (EPGs), even if other contracts would otherwise allow it.

🔍 What is a Taboo Contract?

• Purpose: Taboo contracts are designed to block traffic that matches certain filters. They override any other contracts that might permit that traffic.
• Application: Unlike standard contracts which are applied between EPGs (one consuming and one providing), taboo contracts are applied to an entire EPG. This means they affect all traffic originating from or destined to that EPG 

• Use Case: For example, if you want to ensure that an EPG never uses insecure protocols like HTTP (port 80) or Telnet (port 23), you can apply a taboo contract with filters for those ports 

🛑 Key Characteristics

• Deny Action: Taboo contracts only support the "deny" action. They are used to block traffic, not to permit it.
• Logging: Optionally, taboo contracts can also log the denied traffic for auditing or troubleshooting purposes 
• Priority: They take precedence over regular contracts. If a packet matches a taboo filter, it is dropped—even if a regular contract would otherwise allow it 

⚠️ Best Practices & Cautions

• Many experts recommend avoiding taboo contracts unless absolutely necessary. Instead, it's often better to design your standard contracts carefully to only permit the desired traffic 
• Taboo contracts can add complexity and may lead to unintended traffic drops if not managed properly

 

Cisco ACI - Flood in Encapsulation

🧠 What Is "Flood in Encapsulation"?

Flood in encapsulation is a setting in Cisco ACI that controls how broadcast or unknown traffic is handled within a bridge domain when multiple EPGs use different VLANs.

---

📦 Example Scenario

Let’s say:

• EPG1 uses VLAN X
• EPG2 uses VLAN Y
• Both EPGs are part of the same bridge domain

If flood in encapsulation is enabled, then:

• Broadcast or unknown traffic from EPG1 will only flood within VLAN X
• It won’t reach EPG2 (VLAN Y), even though they share the same bridge domain

---

✅ Why Use It?

This helps limit unnecessary traffic between EPGs and improves security and performance by keeping flooding local to each VLAN.