Common iDRAC Commands used for VMware Cloud Foundation

During several VMware Cloud Foundation deployments I would typically find myself having to change/modify the configuration of Dell PowerEdge Servers.  Changing a single server can be done quickly through the console, however changing multiple racks of servers is more time consuming.  To reduce the amount of time it takes I typically use the racadm commands in a bash script.

You can find the required BIOS settings documented in the VIA User’s Guide here.

Personality Module Check

Check Personality Module

sudo racadm -r IP_Address -u root -p calvin get bios.SysInformation.SystemModelName

Get NUMA

sudo racadm -r IP_Address  -u root -p calvin get BIOS.MemSettings.NodeInterleave

Get Processor Virtualization

sudo racadm -r IP_Address -u root -p calvin get BIOS.ProcSettings.ProcVirtualization

Power Controls

Power Cycle

sudo racadm -r IP_Address -u root -p calvin serveraction powercycle

Power On

sudo racadm -r IP_Address -u root -p calvin serveraction powerup

Power Off

sudo racadm -r IP_Address -u root -p calvin serveraction powerdown

Update BIOS

Check BIOS

sudo racadm -r IP_Address -u root -p calvin get bios.SysInformation.SystemBiosVersion

Update BIOS

sudo racadm -r IP_Address -u root -p calvin update –f filename

sudo racadm -r IP_Address -u root -p calvin serveraction powercycle

Update Storage Controller

Check firmware

sudo racadm -r IP_Address -u root -p calvin storage get controllers -o | grep FirmwareVersion

Update firmware

sudo racadm -r IP_Address -u root -p calvin update –f SAS-RAID_Firmware_2H45F_WN32_25.5.0.0018_A08.EXE

Reboot after firmware installation

sudo racadm -r IP_Address -u root -p calvin serveraction powercycle

Set the HBA controller to HBA MODE

sudo racadm -r IP_Address -u root -p calvin storage get controllers -o | grep CurrentControllerMode

See http://www.dell.com/support/manuals/us/en/19/poweredge-rc-h730/perc9ugpublication/switching-the-controller-to-hba-mode?guid=guid-1fcc87e1-d534-451a-9947-56f1175886c5&lang=en-us

Note: Although it appears as though this can be set using racadm the tool does not actually change it to HBA Mode.

Get  Service Tag Numbers

sudo racadm -r IP_Address -u root -p calvin getsvctag

Update Boot Order

Check Boot Order

sudo racadm -r IP_Address -u root -p calvin get BIOS.BiosBootSettings.bootseq

Check Hard Drive Sequence

sudo racadm -r IP_Address -u root -p calvin get BIOS.BiosBootSettings.Hddseq

Update Hard Drive Sequence

sudo racadm -r IP_Address -u root -p calvin set BIOS.BiosBootSettings.Hddseq Disk.SDInternal.1-1

Update Boot Order

sudo racadm -r IP_Address -u root -p calvin -g cfgServerInfo -o cfgServerBootOnce 0

sudo racadm -r IP_Address -u root -p calvin -g cfgServerInfo -I  cfgServerFirstBootDevice SD

One-time-boot from PXE and reboot immediately

sudo racadm -r IP_Address -u root -p calvin set BIOS.OneTimeBoot.OneTimeBootMode OneTimeBootSeq

sudo racadm -r IP_Address -u root -p calvin set BIOS.OneTimeBoot.OneTimeBootSeqDev NIC.Integrated.1-1-1

sudo racadm -r IP_Address -u root -p calvin create BIOS.Setup.1-1

sudo racadm -r IP_Address -u root -p calvin hardreset

One-time boot from Virtual CD/DVD once and reboot immediately

sudo racadm -r IP_Address -u root -p calvin config -g cfgServerInfo -o cfgServerBootOnce 1

sudo racadm -r IP_Address -u root -p calvin config -g cfgServerInfo -o cfgServerFirstBootDevice VCD-DVD

sudo racadm -r IP_Address -u root -p calvin serveraction powercycle

Update Power Profile

Check Power Profile

sudo racadm -r IP_Address -u root -p calvin get BIOS.SysProfileSettings.SysProfile

Update Power Profile

sudo racadm -r IP_Address -u root -p calvin set BIOS.SysProfileSettings.SysProfile PerfPerWatOptimizedOs

Set Password for Root User

sudo racadm -r IP_Address -u root -p calvin set iDRAC.Users.2.Password PASSWORD

Modify iDRAC IP Address and Host Name

Get Current IP Address

sudo racadm -r IP_Address -u root -p calvin getconfig -g cfgLanNetworking

Set New IP Address

sudo racadm -r IP_Address -u root -p calvin setniccfg -s 192.168.0.103 255.255.255.0 192.168.0.1

Modify iDRAC DNS Name

sudo racadm -r IP_Address -u root -p calvin config -g cfgLanNetworking -o cfgDNSRacName iDRAC02

Advertisements
Posted in Uncategorized | Leave a comment

VMware Cloud Foundation Versions

Quick Note on VMware Cloud Foundation versioning:

Major.Minor.Patch – Build Number

  • Major releases typically have a significant change related to: Features, Functionality, Architecture, and Scale.
  • Minor releases offer small feature enhancements or consolidate multiple patches into a larger release.
  • Patch is for Bug Fixes and is limited in scope to one or two sub-products.

A link to the Release Notes is provided below, you can find the VMware Software Version and Build Numbers for all of the Software Components there.

VMware Cloud Foundation Releases

.5.0 (EVO-RACK) Early Field Trial (EFT)

1.0 (EVO-SDDC) Release to Partners (RTP2) (Arroyo) Release Date: 25 MAR 2016

1.2 (EVO-SDDC) Release to Partners (RTP3) (Big Sur) Release Date: 01 SEPT 2016| Build 4271912

2.0  As of 01 SEPT 2016, VMware Cloud Foundation 2.0 replaces VMware EVO SDDC 1.2

2.1 (R0) (Calaveras) Release Date: 08 DEC 2016| Build 4732657

  • 2.1.1 Release Date: 09 FEB 2017| Build 5022920
  • 2.1.2 Release Date: 06 APR 2017| Build 5022920
  • 2.1.3 Release Date: 25 MAY 2017| Build 5600653
  • 2.1.3a Release Date: 20 JUN 2017
  • 2.1.3b Release Date: 12 OCT 2017 | Build 5022920
  • 2.1.3c Release Date: 11 JAN 2018

2.2 (R1) Release Date: 24 AUGUST 2017 | Build 6383997

2.2.0.1 Release Date: 12 OCTOBER 2017

2.2.0.2 Release Date: 26 OCTOBER 2017

2.2.1 Release Date: 05 DECEMBER 2017 | Build 7236974

2.2.1.1 Release Date: 11 JAN 2018

2.3 (R2) (Dearborn) Release Date: 17 JANUARY 2018 | Build 7597069

  • 2.3.1 Release Date: 06 MARCH 2018 | Build 7898339
  • 2.3.2 Release Date: 24 APRIL 2018 | Build 8318074
Posted in Uncategorized | Leave a comment

Modifing vmnic name in vSphere ESXi 6.5

After installing vSphere ESXi 6.5 on four Dell R630 hosts I noticed that the first host represented my Intel® Ethernet Converged Network Adapter X520-DA2 ports as vmnic4 and vmnic5, whereas all of the other hosts represented the same ports as vmnic0 and vmnic1.

Screen Shot 2017-10-07 at 5.42.57 PM

The problem was that I had forgotten to disable the integrated network adapter in the BIOS, a quad-port 1Gbe daughter board.  After modifying the BIOS and rebooting the ESXi host I noticed that the vmnic numbering remained the same (vmnic4, vmnic5).  I didn’t want to re-install vSphere ESXi 6.5 to fix the ordering so I decided to modify the esx.conf file instead.

To do so, first enable SSH on the ESXi host.

  1. Login to the vSphere WebUI (DCUI) as root
  2. From the left Navigation pane, Select the host.
  3. Click Manage, then click on the Services Tab.
  4. In the Services section locate TSM-SSH and click to select it.
  5. Click the Actions button and select Policy –> Start and Stop with host
  6. Click the Start Button
  7. Verify that the TSM-SSH service has started by reviewing the Status column

Next use an SSH client such as PuTTy to connect to the ESXi Host, then use vi to modify the esx.conf file.  I used cat to find all of the references for vmnic4 (below)

Screen Shot 2017-10-08 at 6.59.50 AM

First I made a backup of the file by typing cp /etc/vmware/esx.conf /etc/vmware/esx.old, then I proceeded to edit the file by typing vi /etc/vmware/esx.conf.

On my system I deleted the existing (vmnic0, vmnic1, vmnic2, and vmnic3) lines in the esx.conf file PRO TIP: Use the VI command dd to delete an entire line.  Next, I modified all vmnic4 entries by changing them to vmnic0 and all vmnic5 entries by changing them to vmnic1. PRO TIP: Use the VI command r to replace a single character.

When finished hit ESC, then type :wq! to save your changes and exit.

When finished I rebooted the system then logged into my ESXi host to verify the changes.

Screen Shot 2017-10-08 at 7.37.52 AM

Posted in Uncategorized | Leave a comment

Home Network Upgrade – Ubiquiti Unifi

IMG_1696

My Home Network is responsible for providing Internet connectivity for my computer, laptop, printer, iPAD, iPhone, security cameras, and a myriad of home automation devices.  It requires 24/7/365 availability.

Recently, one of my Apple Airport Extreme access points servicing my Home Network failed after three years of flawless service.  That somehow turned into me upgrading my entire home network.  A few of my colleagues have been raving about how much they liked UBNT gear and I was anxious to try it out.  Since I was starting to run out of physical network ports and was tired of the SmartNet contract obligation for software upgrades, my Cisco 3560-CG Compact Switch also became a candidate for replacement.   I originally planned on keeping my pfSense Security Appliance which has been problem free since I purchased it – however I wanted to fully experience the entire Ubiquiti ecosystem which meant that it was going to be replaced as well.

Design Requirements

When implementing any solution, it’s always a good idea to jot down a list of requirements.  This was a small list of requirements that I created:

  1. The equipment will be powered-on 24/7/365 all components should be low power, current power consumption for equipment being replaced is a total of 53 watts.
  2. Network Switch must support a minimum of 10-ports, 5-ports are required to provide power to PoE devices.
  3. Network Switch must fit in existing wall mounted 19″ rack which has a maximum depth of  12″
  4. Network Throughput must exceed our current Internet plan – 300/25 Mbps.
  5. Network switch must have at least one SFP port for fiber run to the Garage
  6. Network Switch must be Managed
  7. Network Switch must be quiet, current switch is fanless
  8. Equipment must provide Port Level Statistics
  9. Wireless Network must support 802.11n and 802.11ac, we have several Home Automation devices that use 802.11n

Once I verified that the Unifi components would meet the requirements by reading through product documentation I decided to move ahead with the purchase.

Components & Cost

The Ubiquiti solution was almost 50% less than my existing solution.  B&H Photo carried all of the components and was competitively priced.

Total Cost (B&H Photo Video) = $756.04

As a point of reference, here is a break-down of the costs of the components that were being replaced:

Total Cost = $1,267.99

B&H shipped the Unifi equipment quickly, I ordered everything on a Monday and had it in time to start my installation over the weekend.

Installation Experience

Hardware Installation

The physical installation of the Unifi Switch 16-150W was seamless. Rack mount brackets for the 16-port switch were included in the box and the 8.7″ switch depth left me with 3.3″ of clearance in the back – plenty of room to allow for airflow.  The switch is lightweight at 6.37 lbs which made it easy to mount by myself.  Once the switch was powered-on the very quiet fans (37dBA) came on, but later turned-off.

The installation of the Unifi AC HD was also straightforward, the included wall mount bracket installed quickly onto the ceiling and it was easy to twist the access point on to the mounting bracket.

The Cloud Key included a short re-inforced network cable that allowed me to hang it from the switch port, you can see this in the picture.  Cloud Key is PoE device so no additional wires are left hanging.  This provides for a very clean installation.

The small 5.3″ width of the Unifi Security Gateway allowed it to share a shelf with my   APC BG500 UPS – it was a perfect fit!

Software Installation

The software installation was relatively straightforward, although it was not seamless.  The Ubiquiti Device Discovery Tool Chrome App had difficulty finding the Cloud Key and Unifi Switch – I was surprised since my laptop was plugged into the Unifi Switch when I ran the Scan.

Out-of-the-box every device required a software upgrade, and although this added installation time all upgrades were completed successfully.

I had a hard time getting the Unifi Management controller software to adopt the network switch.  At one point I gave up, went for a 40-minute hike and when I returned the device had been adopted.

Once all of the devices were added to the Unifi Management controller the configuration was simple, although it took a Google search to find out where to enable DPI – deep packet inspection.  Hint: Use the side-bar menu to select Settings and you will find DPI.  For some reason I was looking for this setting on the Configuration Tab of the Unifi Security Gateway device itself.  There is a Configuration tab on each device, and then a separate Settings menu on the side bar – it’s important to review them both so that you understand which options are available.

Power Consumption

Because my home network is on 24/7/365, low power consumption was a key requirement.  Lower power consumption not only reduces electric costs, but also extends the run-time of the UPS in the event of a power outage.  My APC Back-UPS Pro 500 Lithium Ion UPS battery backup can provide 34 minutes of runtime at 40W.

Previous Power Consumption – 53 watts

  • Apple Airport Extreme – 9 watts
  • pfSense Security Gateway – 6 watts
  • Cisco Catalyst 3560CG-8PC-S Compact PoE Switch – 32 watts
  • Estimated Annual Power Consumption = 464.28 kWh
  • Estimated Power Cost $51.07 (0.11 kWh)

New Power Consumption – 38 watts

  • Unifi Switch 16-150W Managed PoE+ Gigabit Switch with SFP – 32 watts
  • Unifi Cloud Key – Powered by Switch (PoE) consumes 2.8 watts
  • Unifi Security Gateway – 6 watts
  • Unifi Access Point AC HD – Powered by Switch (PoE) consumes 6.9 watts
  • Estimated Annual Power Consumption = 332.88 KWh
  • Estimated Power Cost = $36.62

The new Unifi equipment lowered power costs by $14.45 annually, and based on the APC Runtime Graph for the BG500 increased our run-time by 7-minutes.

Noise

My Cisco Catalyst 3560CG-8PC-S Compact PoE Switch is fanless and completely silent.  Although you can find several 16-port unmanaged network switches that are fanless, it is pretty rare to find a 16-port managed switch without fans.  The Unifi Switch 16-150W is no exception, it includes two fans.  The Unifi Switch 16-150 Datasheet reports that fan noise is 34 – 37 dBA., 40 dBA is what a Library is rated at.  The room where the switch is located is approximately 80 degrees Fahrenheit, it has been running for 24-hours and the reported temperature of the switch is 61 degrees Celsius – the fans are not on. The only time the fans have turned on is when the switch was booted.  So yes, it is quiet.

Performance

The only anomaly with performance was the wireless download throughput I received when using my MacBook, it was by no means slow – it just wasn’t as fast as I am accustomed to.

Unifi Switch 16-150W

The Unifi Switch includes a total of 18-ports which provide a non-blocking throughput of  18 Gbps and a forwarding rate of 26.78 Mpps.  The Cisco C3560CG has a total of 10-ports provides a non-blocking throughput of 10 Gbps and a forwarding rate of 13.9 Mpps.  It’s unlikely that I will notice the difference in speed between the two switches.

Unifi Security Gateway

At $104.99 it was about 1/3rd of the cost of my pfsense Security Appliance.  I conducted several Upload and Download Speed tests while directly connected to the LAN port of the device and consistently achieved 300+ Mbps download speed.  One of my friends here locally in Austin, TX mentioned that he consistently was able to get 800 – 900 Mbps of download/upload speed behind his Unifi Security Gateway with a Google Fiber connection.

Unifi AP AC HD

I read several articles including this one Review: Ubiquiti UniFi AP AC HD WiFi Access Point (UAP-AC-HD) which provided examples of download speeds exceeding 300 Mbps.  I consistently get over 300 Mbps when connecting my MacBook Pro Retina (15″ Mid-2014, Broadcom BCM43xx 3×3) to the Apple Airport Extreme.  Unfortunately after running at least a dozen tests, I could never get anything better than 250 Mbps download standing directly under the Unifi access point with my MacBook.

I also own a Dell XPS 15 outfitted with a Dell Wireless 1830 and was able to get 300+ Mbps download speed from the UniFi AP.  So the problem may be related to my MacBook.

Final Thoughts

I have only had the system online for about 24-hours so please keep that in mind.  Overall I am happy with the quality of the hardware components and the software features.  I had hoped to replace two Airport Extreme devices with a single Unifi AP AC HD, however the the Unifi AP is located approximately 60′ from my office and although it provides a respectable 80 – 90 Mbps download speed I am accustomed to having 300+ Mbps.  I considered purchasing a second Unifi AP AC HD to place in my office, however I would like to determine why my MacBook can only download at 250 Mbps before I spend another $293.99 on an additional Unifi AP.  So for now I have an Apple Airport Extreme in my office to support my laptop and iPhone while the UBNT AP AC HD provides Internet Connectivity to the rest of the house.

 

 

 

Posted in Uncategorized | Leave a comment

Upgrade Cisco IoS using MacBook as TFTP Server

I recently needed to upgrade a Cisco Catalyst 3560CPD-8PT-S Compact Switch, which was running an older IoS version.  I knew that my MacBook had a built in TFTP server so this seemed like a great opportunity to use it to upgrade my switch.  Here are the steps that I followed using a MacBook Pro macOS Sierra v10.12.4:

  1. Download latest IoS version for the Cisco Catalyst 3560CPD-8PT-S Compact Switch
  2. Read the Release Notes
  3. Create a TFTP Directory on my Macbook and copy the file
    • Change to the /private/ directory
      • cd /private/tftpboot
    •  Copy the Software Upgrade to the tftpboot directory
      • cp c3560c405ex-universalk9-tar.152-2.E6.tar /private/tftpboot/

    • Change the file permissions
      • chmod 766

        c3560c405ex-universalk9-tar.152-2.E6.tar

  4. Start TFTP on your MacBook
    • sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist

    • Verify the TFTP Service is running
      • netstat -na |grep \*.69
  5. Telnet to the Cisco Switch from the Macbook
    • telnet cisco-switch-address
  6. Ensure that you have IP connectivity to the TFTP server, your MacBook, from the Cisco switch by entering this privileged EXEC command:

    Switch# ping tftp-server-address

  7. Download the image file from the TFTP server, your MacBook, to the switch.
    • archive download-sw /overwrite tftp://MacBook-address/c3560c405ex-universalk9-tar.152-2.E6.tar

  8. Stop TFTP on your MacBook
    • sudo launchctl unload -F /System/Library/LaunchDaemons/tftp.plist
    • Verify the TFTP Service is stopped
      • netstat -na |grep \*.69
Posted in Uncategorized | Leave a comment

Configuring a Cumulus Switch to Send syslog info to vRealize Log Insight

Log Insight is a so much more than a syslog collector, however this post focuses on how to export syslog files from a Cumulus switch.  Today, it doesn’t appear so though Cumulus has a content pack for Log Insight but you may want to check the VMware Solution Exchange.  Until then syslog must be manually configured to export log files.  Fortunately the process is relatively straightforward.

The following steps must be completed:

1. The  switch will need an IP Address on the network where the Log Insight collector is located, this can be done by editing the /etc/network/interfaces file.   You can use cat, nano, or vi to verify that the switch has an interface on the same network as the Log Insight collector.

auto br-rack.3100
iface br-rack.3100
address 192.168.16.1/24

2. If you had to manually add an IP address, the networking service on the Cumulus switch must be restarted using the command service networking restart

3. After successfully restarting networking attempt to ping the IP address of the Log Insight server, in my environment it is 192.168.16.7
Note: Optionally you may wish to establish a Telnet session to verify that you can reach the syslog server.  To do so type telnet 192.168.16.7:514

4. Next configure syslog by creating a file named /etc/rsyslog.d/90-remotesyslog.conf and add the following content:

## Copy all messages to the Log Insight server at 192.168.16.7 port 514
*.*                             @192.168.16.7:514

5. We also want to ship switchd logs to the syslog server by creating a file named /etc/rsyslog.d/24-switchd.conf
## Logging switchd messages to Log Insight
$ModLoad imfile
$InputFileName /var/log/switchd.log
$InputFileStateFile logfile-log
$InputFileTag switchd:
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 5
$InputRunFileMonitor

if $programname == ‘switchd’ then @192.168.16.7:514

6. You may also find it useful to log information for clagd.  I created a file named 20-clagd.conf and added the following

## Logging clagd messages to Log Insight
:programname, isequal, “clagd” -/var/log/clagd.log

7. When finished type service rsyslog restart

8. Next go log in to Log Insight, the default user name is admin.

9. Click on the Interactive Analytics button and filter by source (192.168.16.1), if it has been successfully configured you should see several events from the switch.

Posted in Uncategorized | Leave a comment

Dell FTOS % Error: Port is in Layer-2 mode

I was recently configuring a Dell Force10 S Series S55 1/10GbE top-of-rack (ToR) switch and received the error message “Dell FTOS % Error: Port is in Layer-2 mode” when attempting to remove Hybrid Mode.

The Dell Knowledge Base Article How to configure interfaces in Layer 2 mode on Dell Networking Force10 Switches has a note at the very bottom that states, “Before hybrid mode can be removed from an interface all layer 2 configurations have to be removed. Including any VLANs as tagged or untagged.” Unfortunately it doesn’t provide any details on how to remove the existing configuration.

You are already connected to the switch if you are receiving the error message in the title – so no “How-to connect to the switch” instructions are being provided.

To remove an existing Layer 2 configuration the first step is to look at the current configuration of the port.  You can do this by typing show running-config or simply show run

Screen Shot 2015-12-31 at 1.19.44 PM

Once you have reviewed the current configuration disable the current Layer 2 mode (in my example, Hybrid Mode) enter configuration mode by typing config and typing the commands below. Undo from the bottom up.

First I connected to the interface by typing the following:

interface gigabitethernet 0/4

Next I shutdown the port, this is important since we will also be disabling spanning-tree.  It is a Best Practice to shutdown the port during configuration.

shutdown

Now we are ready to undo the existing Layer 2 configuration type in the following commands:

no spanning-tree 0 portfast

no switchport

no portmode hybrid

end

If you are done configuring the port don’t forget to re-enable it by typing:

no shut

The last step is to save the configuration, type:

copy running-config startup-config

Here is what it looks like from the console

Screen Shot 2015-12-31 at 1.25.51 PM

 

Posted in Uncategorized | Leave a comment