The flashcards below were created by user
on FreezingBlue Flashcards.
BMC: What is the meaning of the acronym for the mangement controller chip in the UCS blade servers.
Baseboad Management Controller
What is the term used to describe one or more blade chassis connected to the same fabric interconnects?
B-250: Full width, 1u, Cisco server, up to 2 Nehalem CPUs, 384GB memory w 8GB DIMMs.
Scale Up is associated with?
- Monolithic, multiprocessor servers. To scale, add hardware to the server.
- Large failure domain (failure of a server impacts multiple apps and users)
Scale Out is associated with?
- Commodity servers. Often one app per server. Usually x86.
- High power and cooling costs. Scale limited to one host. Usually apps don't scale "horizontally".
Scale In is associated with?
- Blade and other commodity hardware, things like GRID, VMware, etc.
- Implementation complicated. Limited scale.
What is hypervisor?
- Software layer that abstracts physical hardware.
- Creates individual virtualized hardware for each VM.
What are some identifiers of a server?
- Mac address
- Firmware version
What is a "stateless server"?
Server whose operating system and application personalities are not tied to the physical hardware. One way to make the server stateless is to use transportable, virtual, unique identifiers.
What is a "server personality"?
Server personality is the operating system configuration and application settings. This fully functional set of programs, files and settings is required to perform the given task of the server.
What are some blade benefits?
- Reduction in redundant equipment costs.
- Power and cooling savings.
- Shared network and storage access. May reduce cabling.
- Rapid hardware provisioning.
- Maximum use of physical space.
What are some blade challenges?
- Increased physical density may create power and cooling challenges
- Increased compute density could increase bandwidth and cabling requirements.
- Each chassis and local switching creates additional management points.
Describe CPU improvements/evolution?
- Single core CPU - CPU is usually the bottleneck
- Without virtualization - multicore CPUs are often underutilized.
- With virtualization and multicore CPUs, memory is often bottleneck.
What is a virtual machine?
A virtualized set of hardware that operates like a physical server.
What is a virtual server?
A virtual set of hardware (virtual machine) along with the OS, apps, and files associated with a "physical server".
What is a hypervisor layer?
A software layer that abstracts the physical hardware and creates the individual virtual hardware for the virtual machines (VM). I.e., VMware ESX, Hyper-V and Xen.
Why a stateless server?
If a server fails, migrating to another server will require that unique identifiers like MAC address, WWN, BIOS and firmware settings be configured. This is a "stateful" server.
What are some problems in managing heterogeneous rack based hardware?
- Independent systems have to be managed including network, storage, host identity, etc.
- Often requires many different management and monitoring interfaces.
What are the components of a UCS system?
- Two Cisco UCS 6120/6140 Fabric Interconnects.
- - Manage up to 320 blades and 40 chassis
- - FCoE
- UCS 5108 blade serve chassis
- - 2 Cisco UCS 2104XP fabric extenders per chassis
- - Up to 8 blades per chassis
Is Cisco UCS is a cable-once, dynamic environment that can be provisioned rapidly.
What are the main UCS components?
- UCS Management and Switching - UCS 6100 series Fabric Interconnects
- UCS Blade Chassis - UCS 5100 blade chassis
- UCS Compute Nodes - Cisco UCS B series blade servers
- UCS Chassis Network Distribution - UCS 2100 series I/O modules
- UCS Server Network Distribution
- - Virtual Interface Card, Converged Network Adapter, 10Gb Ethernet Adapter
What is CEE or DCE?
- CEE - Converged Enhanced Ethernet
- DCE - Date Center Ethernet
- They are the same thing.
- - Priority groups: virtualize links and allocates recources per traffic class.
- - Per priority flow control
- - End-to-end congestion management and notification
- - Shortest-path bridging: Layer 2 multipathing.
Benefits of ethernet (CEE/DCE) enhancements?
- - Lossless fabric: No drop storage links.
- - Provides tools for handling network congestion
- - Deterministic latency for HPC clusters
- - Enables a converged Ethernet fabric for reduced cost and complexity
Descibe an I/O consolidation environment?
In the case of FCoE and TCP/IP; seperate protocols operate as tunnels on the same physical media. Both FC and TCP/IP exist in seperate ethernet payloads.
How does lossless ethernet work?
Effectively, all ethernet fabrics today use a form of packet-loss flow control. Using IEEE 802.3X pause capability in standard ethernet, a lossless fabric can be configured. There is also a priority-based pause capability that can be used to allow multiple lossless fabrics to exist on a single physical infrastructure.
What's the average savings in cable connections with UCS?
40% fewer connections and 66% fewer components.
What are the I/O consolidation benefits of FCoE?
- - Reduced # of NIC/HBA cards/ports.
- - Can be managed same as traditional FC.
- - Same FC model so easy for storage personnel to transition.
- - Remains stateless, don't have to map through stateful gateway. Performance benefits from this.
- - Power efficiency.
- - FC is bridged onto a lossless ethernet fabric.
Definition of a network bridge.
A Network Bridge connects multiple network segments at the data link layer (Layer 2) of the OSI model. In Ethernet networks, the term Bridge formally means a device that behaves according to the IEEE 802.1D standard. A bridge and switch are very much alike; a switch being a bridge with numerous ports. Switch or Layer 2 switch is often used interchangeably with Bridge.
Details of the UCS 5108 blade chassis
- 6U; Blade servers and power supplies install from the front, IOMs and fans from the rear.
- Up to 8 half-width blades, 4 full-width blades, 2 - full width, double height blades
- Physical divider for half height blades can be removed with no tools required.
- Max of 4 - 2500W hot-plug power supplies. 3 configs: non-redundant, N+1, N+N (grid). Single phase AC. They provide up to 550W per half-width server, 1100W per full width server. Power cabling in front (less chance of accident?)
- Chassis cools from front to rear. 8 hot-plug fan modules in rear with status lights per chassis.
What is the IOM?
IOM - I/O Module - extends the 10Gb CEE fabric from the fabric interconnects to the blade servers. No switching in the IOM. 2 - IOMs per chassis, technically only one required. 4 10Gb ports per IOM. If only one IOM installed, must be placed on the left bay. If both used (and cabled correctly?) they are hot-swappable and fully redundant from ethernet perspective. FC ??? Each blade server connects to both IOMs via mezzanine card. Supports up to 8:1 oversubscription.
What is a CMC? Where is it?
CMC - chassis management controller. The CMC collects status data fromt he IOM using the IPMI (Intelligent Platform Management Interface) protocol over the integrated circuit serial bus. The CMC then communicates that information to the management node (UCS management server on the fabric interconnects?) by using the ethernet server link. The CMS controls fan speeds and power supplies, serves as a proxy for the UCS Manager to the blade servers for certain functions. Plays a part in the HA protocols.
What is the UCS 5108 midplane?
- The midplane provides:
- - Redundant power to the IOM and blade servers.
- - Redundant data network (ethernet) connectivity between the IOM and blade servers.
- - Redundant 12C management paths to the blades
- - Dedicated management network (ethernet) connectivity
- - Supports autodiscovery of all components.
What are Cisco 6100 series fabric interconnects?
- - 20 (6120) and 40 (6140) port versions.
- - 1U and 2U high respectively.
- - 6120 has 20 SFP+ (small form-factor pluggable) ports for 10Gb ethernet and one expansion module 560Gb/s capacity
- - 6140 has 40 SFP + (small form-factor pluggable) ports for 10Gb ethernet and one expansion module 1120Gb/s capacity
What type of fabric interconnect expansion modules are offered for the 6120 and 6140?
- - FC only expansion module with 8 SFP ports that support 1,2,4Gb/s FC.
- - Combination expansion module: 4 SFP+ ports that support 10Gb/s ethernet, and 4 SFP ports that support 1,2,4Gb/s FC.
- - Ethernet only expansion module: 6 SFP+ ports that support 10Gb/s ethernet, no FC.
What type of UCS blades are available?
- B200-M1 half-width 12 DIMMs 1 Mezzanine Card 1or 2 Intel Xeon 5500 CPU
- B200-M2 half-width 12 DIMMs 1 Mezzanine Card 1 or 2 Intel Xeon 5600 CPU
- B250-M1 full-width 48 DIMMs 2 Mezzanine Cards 1 or 2 Intel Xeon 5500 CPU
- B250-M2 full-width 48 DIMMs 2 Mezzanine Cards 1 or 2 Intel Xeon 5600 CPU
- B440-M1 2u, full-width 32 DIMMs 2 Mezzanine Cards 2 or 4 Intel Xeon 7500 CPU
- 2 SAS hard drives for all models except B440-M1
What is the BMC? Not who is BMC?
BMC - Baseboard Management Controller on the UCS blades. The BMC uses the IPMI protocol over the 12C serial bus to manage devices on the baseboard. The BMC is responsible for providing remote keyboard, video, mouse (KVM) access to the end user.
What is Cisco extended memory technology?
Used on the B250. Custom ASICs virtualize 8 DIMMs per memory channel. With 3 memory channels per CPU, that adds up to: 8 DIMMS * 3 Channels * 2 CPUs = 48 DIMM slots.
What type of SAS drives can be used in the blade servers?
- 73GB/15K SAS
- 146GB/10K SAS
- SDD soon.
What type of Mezzanine Cards are available?
- Cisco UCS VIC M81KR: Cisco virtual interface card (Palo?)
- Cisco UCS CNA M71KR-E: Emulex CNA
- Cisco UCS CNA M71KR-Q: QLogic CNA
- Cisco UCS 82598KR-CI: 10Gb ethernet adapter
What are the featuers of the Cisco M81KR Mezzanine card?
- Supports multiple vNICs or vHBAs.
- 500,000 IOPs in both initiator and target mode.
- Supports kernel and hypervisro bypass.
- Dual 10Gb ethernet ports and dual FC ports to backplane.
- Provides failover between redundant links.
- No multipathing software is required on the host OS.
What are the featuers of the Cisco M71KR Mezzanine cards?
- 2 host side Gb ethernet ports and 2 FC ports to the backplane.
- Network ports can run either native ethernet or FCoE protocols
- Failover is performed by the FCoE asic Is this feature going to be deprecated?
- Both cards use Intel 10Gb ethernet chipsets.
- FCoE protocol encapsulation is offloaded from the host and peformed by the CNA.
What are the features of the Cisco UCS 82598KR-CI Mezzanine Card?
- Made by Intel, only supports 10Gb ethernet.
- FCoE support must be handled by the OS on the CPU (not offloaded the the Mezzanine cards).
How does the Nexus 1000v work?
- Nexus 1000V bypasses the VMware vSwitch by leveraging the VMware vNetwork distributed virtual switch (DVS). Provides a single point of configuration for networking multiple ESX hosts. Also provides: policy-based connectivity for VMs, network security mobility, NDU software model.
What are the main components of the 1000V architecture?
- VSM - Virtual Supervisor Module - a virtual or physical appliance runing Cisco OS. It performs management, monitoring and configuration tasks. Tight intergration with vCenter.
- VEM - Virtual Ethernet Module - Enables advanced networking capability on the hypervisor. Provides each VM witha virtual dedicated "switch port". The VEM sits on top of the VMware DVS (distributed virtual switch).
Descibe the UCS management architecture?
- - All administrative management activity occurs with the UCS Manager running on the fabric interconnect.
- - Management traffic flows from the fabric interconnect via a dedicated VLAN (4044) on the ethernet downlinks to the chassis management controller (CMC) on the IOMs.
UCS Manager manges the chassis, IOM and blades using what links?
- - Chassis related management tasks are passed via redundant serial links to the appropriate device.
- - IOM management tasks are passwd via integrated circuit (I2C) links to the appropriate IOM chipset.
- - Blade management tasks are passwd via dedicated redundant ethernet links from the BMC (Baseboard Management Controller).
Where does the UCS Manager run and what options are available?
- - UCS Manager runs on the fabric interconnects (UCS 6120 or 6140).
- - UCS Manager runs in the Nexus OS (NX-OS) on the fabric interconnects.
- - UCS Manager can be accessed via GUI, CLI or XML API.
- - UCS Manager provides KVM functionality to manage the blade servers.
What does the CMC do?
CMC - Chassis Management Controller, collects the status of the IOM that it resides in using the IPMI (Intelligent Platform Management Interface) protocol over the I2C serial bus. The CMC communicates information back to the UCS Manager (management node) via ethernet server links. The CMC controls pwoer supply and fan speeds. It is also a proxy between the UCS Manager and the blade servers for some functionality. Also supports some UCS Manager HA functions.
What does the BMC (baseboard management controller) on each blade do?
The BMC communicates "lights-out" status information, KVM and other status information to the CMC.
Descibe UCS Management assignments.
- - Most existing system, LAN and SAN admin responsibilities remain unchanged.
- - RBAC allows current admins to manage appropriate domains with UCS Manager.
What modes does the fabric interconnect operate in?
- - EHV - Ethernet host virtualizer mode
- - Traditional switch mode
- - For storage, the fabric interconnect operates in N_Port virtualization mode for SAN uplinks.
- This allows full compatibility with existing networks that contain switching infrastructure from other vendors.
How does "switch mode" work on the fabric interconnects?
When running is switch mode, the FIs are managed in the same manner as any other Layer 2 switch. The ethernet Network admin is responsible for LAN config including 802.1Q VLAN trunks and etherchannel aggregates.
How does "EHV" (Ethernet host virtualizer) mode work on the fabric interconnects?
When the FIs run in EHV mode, it does not participate in VLAN trunking protocol (VTP) or Spanning Tree Protocol (STP). The FIs are Layer 2 devices that do not participate in routing protocols. Uplink network configuration consists of link aggregation and trunk settings.
Does UCS change SAN admin responsibilities?
1. Why do the UCS fabric interconnects need to be in NPV mode?
2. Why do UCS fabric interconnects need the SAN be in NPIV (N port Indentifier Virtualization)?
- 1. Prevents it from participating in the FSPF (Fabric Short Path First) routing and domain schemes.
- 2. NPIV allows multiple N port IDs to share one physical N port.
In UCS do sysadmin duties change?
What is the role of the UCS admin?
- - Coordination with the data center team.
- - Physical resource assignment
- - Server profile create
- - Creation and management of the UCS internal VSAN and VLAN objects.
- - BIOS and firmware management
- - UCS configuration backup
What diagnostic techniques are available for the UCS?
- - Physical and power status via LEDs
- - Logical status via UCS Manager GUI and CLI
- - Traditional troubleshooting techniques.
UCS LED status lights
What type of external management integration is available for the UCS?
- - XML API (application programming interface)
- - Web-Based Enterprise Management (WBEM)
- - SNMP
- - IPMI
- - SDKs for full control of UCS are available
What can you do with the external management integration tools?
- - Create multitenant portals with your own presentation, authorization and monitoring mechanisms.
- - Integrating with custom or industry-standard orchestration tools (like HP Openview).
- - Populating CMDBs
- - Scripting and integrating with custom management solutions
How does the DME (Data Management Engine) enable interaction between UCS hardware and software components and external applications?
The DME provides API services (all interfaces are ultimately translated into XML AP) and authentication servers. Custom scripting, CMDBs, etc. can be integrated using the XML API. Everything that si available in the user interface in the product is available through the XML API.
How is BMC BladeLogic integrated wth UCS?
BladeLogic uses UCS features like policies and service profiles allowing administrators to manage and deploy servers, perform compliance operations ont he UCS platform. BladeLogic can be used by admins to perform day-to-day Cisco UCS system mangement without using UCS manager GUI or CLI. BladeLogic is a purpose built automation solution for UCS.
What are some features of BMC BladeLogic?
- - Can manage multiple UCS "fabrics" or "pods".
- - Can manage service profiles or "templates".
- - Can manage unique IDs like MAC addresses and WWNs for the entire server farm (multiple pods).
- BladeLogic uses custom objects to manage the Cisco UCS environment. Admins can provision and reprovision an entire technology stack.
What are UCS fabric interconnect HA features?
- - Redundant fabric interconnects synchronize database and state data through dedicated, redundant ethernet links.
- - Architecture prevents split-brain scenarios.
- - Floating management IP addresses allows for seamless failover.
- - Management of redundant fabric interconnects occurs on active device only, changes are synchronized to standby.
- - Only management interface is active/standby; data traffic is active/active.
What are the UCS Manager DR features?
- - Recoveriong a full Cisco UCS Manager implementation after a disaster can use multiple backup types.
- - Backups can be automated to an external TFTP, FTP or SCP server.
- - Full UCS Manager database backups include physical hardware states such as service profile associations.
- - Configuration data can be backed up as XML.
- - XML-format backups do not include service profile associations. Can be edited before restoration.
Summary of management features.
- - All management occurs in the UCS Manager interfaces - no direct device or module management is necessary.
- - Network and system admin models are unchanged.
- - Cicso UCS Manager has many sources of diagnostic and status information to assist troubleshooting.
- - Broad management API support allows for 3rd party or custom integration.
- - Clustered fabric interconnects provide both management and network traffic HA.
- - Recovering the UCS environment after a disaster is possible using the UCS database and configuration backups.
What are the power supply options for the UCS 5108 blade chassis?
- - Nonredundant - Only enough power to meet chassis requirements. Power supply failure results in chassis shutdown.
- - N+1 redundancy - Meets chassis requirements plus one power supply. No disruption if 1 power supply fails. All power supplies provide power and balance workload. Extra power supplies (N+2) are placed in standby mode.
- - Grid redundancy - Requires twice nonredundant configuration. Half supplies are wired to one power source, half to another power source.
More on power supplies.
- - Inbound power connections are not matrixed. Failure of an input causes loss of power to the connected power supply.
- - Outbound (to the blades, etc.) are matrixed. All power supplies pool power for all components.
Details about the fabric interconnect console.
Fabric interconnect management port details.
- Out-of-band management is performed with these.
Fabric Interconnect clustering ports.
- Two RJ45 ports for heartbeat. Use 802.3ad bond managed by NX-OS. Ports are configured LACP. Labeled L1 and L2. Must be cabled L1-L1 and L2-L2.
Does the IOM (I/O Module) have any independent point of management?
No. Managed through the UCS Manager.
How can you connect to a UCS blade to manage it independently?
Blade dongle that includes a DB9 serial connection, USB connector and VGA connector.
Does the chassis have any independent points for management?
No. Managed using UCS Manager.
What are the requirements for HA for the fabric interconnects?
- - Connect Layer 1 (L1) of Fabric Interconnect A to Layer 1 of Fabric Interconnect B.
- - Connect Layer 2 (L2) of Fabric Interconnect A to Layer 2 of Fabric Interconnect B.
- - Connect Fabric Interconnect A to IOM A of each chassis with 1 to 4 uplinks.
- - Connect Fabric Interconnect B to IOM B of each chassis with 1 to 4 uplinks.
What is DCOS (of Cisco UCS Manager)?
Date Center Operating System on the fabric interconnects.
Cisco UCS system connectivity:
- Fabric A and B Data Center Operating System (DCOS) Networks. - these provide connectivity to the CMCs and management entities on the mezzanine cards.
- Fabric A and B Adapter Management (AM) Networks - provide connectivity between the DCOS instance in each FI and the management entities on the mezzanine cards for the purposes of allocating recources.
- Fabric A and B Adapter Management (AM) Infrastructure Networks - provide system wide connectivity between the AM instance in the UCS fabric interconnect, the fabric A and B CMCs, and the baseboard management controller (BMC) of each blade. UCS Manager bootstraps this process by passing out chassis numbers to each CMC. Other addresses are derived from that. The CMC and BMC interfaces are exposed as virtual interfaces on VLAN 4044.
- Fabric A and B Cisco UCS Manager PXE Networks - These isolated networks are used to boot host OS from images that are hosted by the UCS Manager. IP addresses are assigned by DHCP as part of the PXE process. VLAN 4047.
Fabric Interconnect prerequisites:
- VLANS: - configure globally; out-of-box VLAN is default (1); trunk all VLANs defined n profiles across uplinks.
- Port Channels - port channel uplinks created using UCS Manager; must match port channel config upward from Cisco UCS; Uses LACP 802.3ad/802.1ax.
How do you manage VLANs in UCS Manager?
For any VLAN to be supported on the blade servers, a VLAN object must be created in the GLOBAL UCS (both fabrics) configuration (under the LAN tab). VLANs associated with fabric A or fabric B can also be created, but not recommended.
What is DCOS
Data Center Operating System
What are AM Networks
Adapter Manager networks (one each for fabric A and B). These networks provide connectivity between the DCOS instance of each FI (fabric interconnect) and the management entities on the mezzanine cards for the purposes of allocating recoures, defining identities, and monitoring adapter status. Layer 2 protocols are run by both Cisco UCS manager instances. Frames on this network use VLAN 4043.
What are the reserved VLANs?
- 4042: Connectivity between the Cisco UCS Manager and chassis management.
- 4043: Cisco UCS Manager to management entity on mezzanine adapters.
- 4044: Cisco UCS Manager to CMCs
- 4047: PXE boot for Cisco UCS Utility OS (UUOS)
Creating port channels:
As an example, port channel ID 1 can be created with uplink ports 19 and 20 on a Cisco UCS Fabric Interconnect. A matching configuration must be configured on the uplink switch. So the port channels are between the FIs and an external switch.
Can be configured globally or per fabric in UCS manager. Don't use default VSAN 1. Also have to configure a VLAN ID for each VSAN to carry FCoE traffic. Like any other VLAN/VSAN, must be a unique number. Probably best just to reserve a set of VLAN/VSAN IDs for storage.
Cisco 61xx Fabric Interconnect main features:
- - Management of the 61xx FI's can be performed via serial console, out-of-band IP management, or Cisco UCS Manager GUI.
- - the UCS chassis and IOMs are managed by Cisco UCS Manager.
- - UCS 61xx FIs provides management channels for all Cisco UCS components and LAN/SAN data paths for the UCS server nodes.