Cisco ACI Port Security

  Cisco ACI Port Security – Summary

Purpose:
Controls the number of MAC addresses that can be learned on an interface to prevent unauthorized access and MAC flooding.

---

⚙️ Key Features

• MAC Limit: Set a maximum number of MAC addresses per interface (0–12000).
• Protect Mode: Only supported violation action.
• Excess MAC addresses are dropped.
• MAC learning is disabled temporarily.
• Learning resumes after a timeout (default: 60 seconds).
• Supported Interfaces: Physical ports, port channels, and vPCs.
• Monitoring: Faults and syslogs are generated when limits are exceeded.

---

🚫 Restrictions

• Not supported on Fabric Extender (FEX) ports.
• Only MAC address limits are enforced (not MAC+IP).

---

🛠️ Configuration Path in APIC GUI

1. Fabric → Access Policies → Interface Policies → Port Security
2. Create and attach the policy to an Interface Policy Group
3. Bind the group to a Switch Profile

 

Cisco ACI - Fabric Secure Mode Overview

 Fabric Secure Mode Overview

Fabric Secure Mode is a security feature in Cisco ACI that safeguards the infrastructure from unauthorized additions. It ensures that only verified switches and APIC controllers can join the fabric, even if someone has physical access to the equipment.

Starting from release 1.2(1x), Cisco ACI performs a validation check during installation or upgrade. This check confirms that each device has a valid serial number and a Cisco-signed digital certificate.

By default, the system operates in Permissive Mode, allowing existing setups to continue functioning even if some devices lack valid certificates. However, administrators can enable Strict Mode for enhanced security, requiring manual approval for any new device joining the fabric.

---

⚙️ Modes of Operation

Mode	Permissive Mode (Default)	Strict Mode
Device Validation	Valid Cisco serial number and certificate required	Enforces serial number and certificate validation
Existing Fabric	Continues operating even with invalid certificates	Requires all devices to be validated
Authorization	Auto-discovers and allows devices without manual approval	Manual authorization needed for each new device
Security Level	Basic security	Enhanced security and control

---

To change the Fabric Secure Mode in Cisco ACI (e.g., from Permissive to Strict), follow these steps using the Cisco APIC GUI:

🔧 Steps to Change Fabric Secure Mode

1. Log in to the APIC GUI.
2. Navigate to:
   System → System Settings → Fabric Security
3. In the Properties pane, locate the Fabric Secure Mode setting.
4. Select Strict Mode from the available options.
5. Save the configuration.
6. Reboot the APIC and affected switches to apply the change.

⚠️ Important: Changing the mode requires a reboot for the configuration to take effect.

Cisco ACI - Node Stateful Vs Stateless reload

 

Aspect	Stateful Reload	Stateless Reload
Definition	Reload where process state is preserved using checkpoints	Reload where process starts fresh without any prior state
State Preservation	Yes – runtime state is saved to Persistent Storage Services (PSS)	No – process is restarted without retaining previous state
Recovery Speed	Faster – resumes from last known state	Slower – requires full reinitialization
System Impact	Minimal – seamless continuation of operations	Higher – may cause temporary disruption or delay
Use Case	Preferred for critical services needing quick recovery	Used when state cannot be preserved or process needs a clean start
Managed By	Persistent Storage Services (PSS)	System Manager
Example Scenario	Restarting a service with session data intact	Replacing a crashed process with a new instance

 

ACI Node state - undiscovered Vs Unknown

 

  Undiscovered:

• You’ve manually added a node ID in APIC.
• The switch is not yet connected or powered on.
• Could be due to cabling issues or incorrect port configuration.

  Unknown:

• The switch is physically connected and sending LLDP packets.
• APIC detects it but doesn’t have a matching Node ID policy.
• You need to assign a Node ID to complete discovery.

 

What is a Contract Preferred Group in ACI?

 🔷 What is a Contract Preferred Group in ACI?

In Cisco ACI, Endpoint Groups (EPGs) typically require contracts to communicate with each other. This follows the “allow list” model, where communication is explicitly permitted only if a contract exists.

The Preferred Group (PG) feature simplifies this by allowing certain EPGs within the same VRF to communicate freely without contracts.

---

✅ Key Concepts

Term	Description
Included EPGs	EPGs that are part of the preferred group and can communicate with each other without contracts.
Excluded EPGs	EPGs outside the preferred group that still require contracts to communicate.
VRF PG Setting	Must be enabled for the preferred group to work. Without this, even included EPGs won’t communicate freely.

---

🛠️ Configuration Steps

1. Enable Preferred Group on VRF:
• Go to the VRF settings in APIC or Nexus Dashboard Orchestrator (NDO).
• Check the Preferred Group box.
2. Add EPGs to the Preferred Group:
• In the EPG properties, check Include in Preferred Group.
• Save the configuration.
3. Verify Membership:
• You can view all EPGs in the preferred group under the VRF’s properties.

---

🌐 Multi-Site Considerations

• In a stretched VRF across multiple sites, preferred group EPGs are shadowed in other sites to enable inter-site communication.
• This allows, for example, a web EPG in Site 1 to communicate with an app EPG in Site 2 without contracts.

---

⚠️ Limitations

• Preferred Groups are not supported for L3Out external EPGs.
• If vzAny is already consuming/providing a contract in the VRF, you should not configure preferred groups.
• All EPGs in a preferred group must be managed consistently (either all via APIC or all via NDO).

 

ACI Leaf Switch Replacement

 To replace a Cisco ACI leaf switch, follow these step-by-step instructions to ensure a smooth transition without disrupting your fabric:

---

🛠️ Preparation

1. Document the existing switch details:
• POD ID
• Node ID
• Node Name
• Serial Number 

2. Ensure the replacement switch is in ACI mode:
• Connect via console and run show version.
• If in NX-OS mode, convert to ACI mode using Cisco's documented procedure 
• Before adding the new leaf switch to the fabric, ensure it's manually upgraded to the target image or one with a direct upgrade path. Avoid using intermediate images that require multiple upgrade steps, as they can trigger issues and impact your production environment. A final upgrade via policy helps ensure BIOS and FPGA components are properly updated.

3. Clean up the replacement switch:
• Run setup-clean-config.sh and then reload to remove any existing configuration 

---

🔄 Decommission the Faulty Leaf Switch

4. Go to APIC GUI:
   Fabric > Inventory > Fabric Membership
5. Right-click the faulty switch → Select Decommission.
6. Once decommissioned, Remove from Controller and confirm the action 
7. Physically disconnect and unmount the old switch.

---

🔌 Install and Connect the New Leaf Switch

1. Mount the new switch and connect uplinks to spine switches. DONOT CONNECT DOWNLINK AT THIS STAGE
2. Power on the switch.
3. In APIC GUI, go to:
   Fabric > Inventory > Fabric Membership > Nodes Pending Registration
4. Verify serial number, then Register the switch:
• Use the same POD ID, Node ID, and Node Name as the old switch 

5. Once registered, go to:
   Fabric > Inventory > Fabric Membership > Registered Nodes
   → Right-click → Select Commission.
6. Wait for the switch to reach Active state.

---

🔍 Post-Replacement Validation

1. Connect downlink cables (after switch is active).
2. Go to:
   Fabric > Inventory > Topology
   → Verify the switch is visible and operational.
3. SSH into APIC and run:

→ Confirm switch status is active 

4. If you get SSH warnings (e.g., DNS spoofing), update the known_hosts file:

---

🧩 Troubleshooting Tips

• Switch not discovered: Check LLDP neighbors and cable connections.
• Switch shows "Not Supported": Upgrade APIC firmware to match switch model.
• No TEP IP assigned: May be a DHCP issue—contact Cisco TAC.
• SSL issues: Check for established sessions on port 12215 

Common Causes of "Unknown" Leaf State

 

 Common Causes of "Unknown" Leaf State

• Certificate Issues: The leaf might not be presenting a valid certificate chain to the APIC, which prevents proper SSL handshake and authentication.

• LLDP Mismatch or Failure: ACI relies on LLDP (Link Layer Discovery Protocol) for fabric discovery. If LLDP info isn’t exchanged correctly between APIC and leaf, discovery fails.

• Firmware Incompatibility: The leaf switch might be running a version of ACI software that’s not compatible with the APIC or spine switches.

• Hardware Problems: Faulty transceivers, cables, or ports can block communication between APIC and leaf.

• Time Sync Issues: If the leaf’s system time is out of sync with the APIC, certificate validation may fail.

• Incorrect Node ID or Serial Number: If the leaf was previously part of another fabric or misconfigured, it may need to be wiped and re-initialized.