Setup AD Authentication in vCenter 5 Virtual Appliance

Okay, so I’ve setup a vCenter 5 virtual appliance and want to tie it into Active Directory (AD) – now what?!  I searched around and couldn’t find a quick-and-easy “here’s what to do”, so after figuring it out, here’s what I did (hope it helps!).

VMware has a helpful reference at http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.vsphere.install.doc_50/GUID-7C9A1E23-7FCD-4295-9CB1-C932F2423C63.html.

First, there are plenty of references around how to setup AD access.  I setup a service account specifically for vCenter in my AD, then used the account credentials along with the FQDN in the appliance web interface (Authentication -> Active Directory).  BTW, in case you’ve forgotten how to get to your virtual appliance’s web interface, it’s http://<your vCenter IP>:5480 (it’ll redirect you to a secure self-signed page).  Don’t forget that when you enable AD on the appliance, you have to restart the appliance (easily accomplished by going to System -> Reboot).

After you have AD access, log into your vSphere Client (going to your vCenter appliance) and select your root-level item (logging in as the root account at this point).  Click on the Permissions tab and right-click, then click on the “Add Permission…” link, then add your username and/or AD group with the appropriate permissions.

After doing this, things were good – I can sign-in to vCenter using my AD credentials without a problem!

Hope the above helps!  Until next time…

Fixing an LVM in Ubuntu

WARNING: There are steps, references, commands and/or instructions in this article that can be very dangerous to your filesystem and possibly cause irreversible damage to your data.  You are responsible for maintaining your data – use these commands only after you fully understand the implications of what they do and you’re comfortable and competent.  Any damage resulting from using any of these commands or guidance of this article is fully your responsibility – I can’t be responsible for your data.  By continuing, you agree to release the author, the hosting service for this site, anybody posting comments to this site, your dog, etc. free from liability should any damage occur, whether accidental or intentional.

I use Ubuntu quite a bit and have recently started using LVM on an internal FTP/TFTP/SCP “drop box” of sorts. The reason for LVM is that I needed to dynamically “grow” the amount of disk space available for the server (it was full). Being a VM, it’s easy to carve out another virtual disk and assign it to the VM. LVM takes over from there…

This is all fine-and-dandy if your storage is stable, however we ran into an issue where storage was oversubscribed and completely full on the SAN. This particular server was running on a NetApp SAN and the volumes ended up being taken offline. This happened automatically on the SAN. After freeing up some space for the remaining volumes, I brought the volumes back up. Either this process itself or having me shutdown the VM (I hadn’t noticed that the storage was gone at the point I shut the VM down) resulted in a corrupted filesystem.

Upon trying to boot the server back up, I was presented with a BusyBox screen and several error messages indicating that the filesystem was messed up. I received a message like “Target filesystem doesn’t have /sbin/init” on the screen (with the BusyBox prompt).

Here’s what I did to fix the filesystem and restore the system to full functionality, and a lesson I learned.

To restore the system:

  1. Download the SystemRescueCd ISO (www.sysresccd.org)
  2. Create a new virtual disk (for copying data that I couldn’t live without to) and assign this to the VM
  3. Mount the ISO in the VM on startup and boot off of the CD
  4. Setup the networking (assign an IP address, default gateway, etc.)
  5. SSH into the server
  6. Check the LVM for filesystem errors
    1. e2fsck -n /dev/mapper/<LVM name>
    2. If it shows errors, you might want to continue to fix these errors (if possible)
  7. Mount the old LVM
    1. mkdir /mnt/t (t for temp, or whatever directory name you desire)
    2. mount /dev/mapper/<LVM name> /mnt/t
  8. Create a partition on and format the new virtual disk
    1. Plenty of resources on this – Google for the filesystem you’re wanting to use (ext2, ext3, ext4, etc.)
  9. Mount the virtual disk
    1. mkdir /mnt/n (n for new – again, whatever you want)
    2. mount /dev/sdc1 /mnt/n
  10. Copy data that I couldn’t live without from the old partition to the new one (so from /mnt/t to /mnt/n)
  11. Unmount and fix errors on the old LVM partition
    1. umount /dev/mapper/<LVM name>
    2. e2fsck -v /dev/mapper/<LVM name>
      1. I know there are ways to have e2fsck automatically fix errors, but I wanted to see and approve each error, so I went the somewhat slow path
  12. Shutdown the VM, remove the SystemRescueCD ISO and the new virtual disk
  13. Try booting the VM

After following these steps (best as I can remember), the system was back working again.  In this system, I had a single LVM that was pretty much an “everything” partition – root, FTP/TFTP/SCP storage, etc.  This leads up to my lessons learned:

  • On file servers, particularly VMs, it’s so easy to carve out additional virtual disks, keep your root filesystem (boot loader, kernel, etc.) on one virtual disk that’s only used for base OS functionality.  Create another virtual disk(s) for file storage, using LVMs if necessary.  This allows you to still easily access the data by simply assigning the virtual disk to another “clean” Ubuntu install, bypassing the need to boot off of the SystemRescueCD ISO.  In my instance, everything was combined.  Since it was on a SAN, I had assumed very little risk of data corruption (messed up filesystem), however oversubscription can cause problems.
  • Use oversubscription on your storage sparingly and in a planned fashion.  This bit me, and I’ve heard other IT professionals say “Oh, don’t worry about it – it won’t use it”.  While it might be unforeseeable that a system consume all of the allocated space, it is possible and should be guarded against.  What happens if it does consume all of the space (logs, updates, etc. can all contribute to filesystem growth).  Ensure that your core, mission-critical systems are NOT using oversubscribed storage.
  • When working with LVMs, don’t point to the physical disks (/dev/sdb) and partitions (/dev/sdb1) for troubleshooting – it’ll get you nowhere.  Using the pvdisplay, lvdisplay, etc. commands (for examining your LVMs), you’ll be able to see the LVM name, as well as partitions that comprise the LVMs.  Focus on the LVM, not the partition (at least in my case).  This isn’t to say that sometimes there are physical issues occurring (SAN stats or SMART errors on a local disk should help here).

So, your mileage may vary, but this is what I did to fix the LVM filesystem (and restore functionality of my system).

What do you consider best-practices for Linux VM creation as well as general filesystem tasks?  Do you have a different tip or trick than I’ve mentioned above?

Until next time…

ACS 5.2 and AD Client

I had an experience this morning and figured I’d share it. I’m running ACS 5.2 to provide a rich Authentication, Authorization and Accounting (AAA) environment for our equipment. I like it – it provides an extremely rich feature-set and is very extensible.

Our environment uses AD on the back-end, however what happens when AD is inaccessible, what then? This happened to me today – here’s what I did.

  1. SSH to the ACS server
  2. Check the status of the adclient ACS process

  3. ACS/admin# sh app status acs

    ACS role: PRIMARY

    Process 'database' running
    Process 'management' running
    Process 'runtime' running
    Process 'adclient' Execution failed
    Process 'view-database' running
    Process 'view-jobmanager' running
    Process 'view-alertmanager' running
    Process 'view-collector' running
    Process 'view-logprocessor' running

    ACS/admin#

  4. Oops – it’s not running, so let’s stop the whole ACS process and start it back up

  5. ACS/admin# app stop acs

    Stopping ACS.
    Stopping Management and View...............................................................
    Stopping Runtime.......
    Stopping Database...
    Cleanup.....

    ACS/admin# app start acs

    Starting ACS ....

    To verify that ACS processes are running, use the
    'show application status acs' command.

    ACS/admin#

  6. All things running now?

  7. ACS/admin# sh app status acs

    ACS role: PRIMARY

    Process 'database' running
    Process 'management' running
    Process 'runtime' running
    Process 'adclient' running
    Process 'view-database' running
    Process 'view-jobmanager' running
    Process 'view-alertmanager' running
    Process 'view-collector' running
    Process 'view-logprocessor' running

    ACS/admin#

Keep in mind that it takes awhile for ACS to stop, as well as start back up (the CLI returns almost immediately after telling it to startup the ACS app, however it will be several minutes until all of the processes are running).

Now it might be totally unnecessary to take down all of ACS, rather than just telling it to start (would it be smart enough to only start the one failed process?). I didn’t try this today – just got things back up-and-running.

Until next time…

Re-Address (IP Address) EMC Clariion CX4-120

I had to change the IP address on a Clariion CX4-120. Here’s the steps that I went through:

In a web browser, go to https:///setup
Change the IP address, then let it reload the SP (SPA)

https:///setup

Change the second one (SPB) – it might auto-populate the correct peer IP address – if not, change to correct IP

This results in each SP being bounced (one at a time, otherwise you’ll end up with an outage). At the end of the day, you’ll be able to re-IP the box pretty easily.

VLANs in IOS and NX-OS

There are always limitations and system-reserved resources on network platforms.  For years, certain VLANs have been “off-limits” on IOS platforms.  With NX-OS, we also have “off-limit” VLANs, however they are different from the IOS counterparts.

Read more »

Upgrading a Juniper J4350 to 2GB Compact Flash

This is somewhat of an impossibility, right?  Well, not really.  Although not technically supported by Juniper, we are able to go above the recommended 1GB Compact Flash size on J-series routers (at least J4350s).

This was performed on lab equipment, not production equipment.  I wouldn’t advise doing anything that would violate any manufacturer’s support agreement on production equipment.  Use this at your own risk – your mileage may vary, especially as it’s not following Juniper’s recommendations.  Phew – disclaimer finished – now let’s move on…

Read more »

Nexus (NX-OS) protocol support (and cleanup)

Over the years, Cisco has been very instrumental in the design and standardization of many networking protocols.  There are lots of examples where a need for a protocol was identified and Cisco filled the need with a Cisco-proprietary protocol.  Cisco-proprietary can sound bad, but it really isn’t.  Let’s give them some credit here – network equipment vendors are in competition and don’t typically play well together.  Often times vendors pitch proprietary solutions in an attempt to carve out a niche that delineates them from their competitors.

There are several standards organizations in existence today (IEEE, IETF, CableLabs, etc.) which many vendors work with and closely follow.  While this sounds ideal (and is very beneficial), standards often take a significant amount of time to be ratified, leaving any current needs unaddressed from a standards perspective.  The only alternative (for a quick resolution) is a proprietary solution, while the standards process is given time to complete. Read more »

Missing VLANs are not automatically created in NX-OS

In IOS, if we assign a switchport to a non-existent VLAN, the switch will kindly create the missing VLAN for us. NX-OS does not do that – if a switchport is assigned to a missing VLAN, the interface will be placed in the down state. Let’s look at it… Read more »

FEX configs are retained

The N5k maintains the config of the FEXs, even when they’re removed (and even though it’s not visible to us). In this article, we’re going to look into this a little further… We’ll start with a working FEX, using Po100 (with Eth1/19-20 being the physical bundle members). Read more »

Some NX-OS features can’t be manually enabled

The whole idea around features in NX-OS has been intriguing to me. It’s a good idea – I like it. It seems very similar to services on many other OSs (particularly *nix systems), however it doesn’t always strike me as having a rich feature set as of now.

For instance, the NX-OS config guides repeatedly refer to the show feature command to look at which features are enabled.  I’ve already mentioned that this command doesn’t exist today in older NX-OS versions – the currently-available NX-OS versions today support this feature (a reader confirmed this on the N7k and I’ve confirmed it on the N5k).  See this article for more info.  There are some interesting behaviors around features, in that some are manually enabled, while others are automatically enabled and disabled as needed.  Let’s dig into this a little deeper for an example… Read more »

WordPress Themes