Understanding Authentication in vRealize Operations

// This is a guest post by Jeremy (@nakedhitman)

vRealize Operations offers a number of options for user authentication and permission management, but does not do a great job describing the differences between them and why you should care. Most of the options are fairly easy to understand, but the vCenter authentication source is special and can easily cause confusion. Let's break them down and run through the differences:

• Local Shell Users
  
  
  
  • Example: root
  
  
  • These user accounts can not log in or be imported into the application UI.
  
  
  • They can log in to the shell of the system via SSH or VM console.
  
  
  
  


• Local Application Users
  
  
  
  • Example: admin, maintenanceAdmin, automationAdmin
  
  
  • They can not log in to the shell of the system via SSH or VM console.
  
  
  • The full range of permissions can be granted to local users.
  
  
  
  


• Active Directory / LDAP
  
  
  
  • Example: youruser@yourdomain.tld, yourdomain\otheruser
  
  
  • They can not log in to the shell of the system via SSH or VM console.
  
  
  • Technically, AD and LDAP are the same technology, but with different presets. These sources allow users and groups to be imported from a domain so that they can log in with their common credentials.
  
  
  • The full range of vRealize Operations permissions can be granted to imported AD/LDAP users and groups.
  
  
  
  


• SSO SAML
  
  
  
  • Example: administrator@vsphere.local, youruser@yourdomain.tld, yourdomain\otheruser
  
  
  • They can not log in to the shell of the system via SSH or VM console.
  
  
  • Similarly to the AD/LDAP source, importing users from SSO allows you to assign vRealize Operations roles to domain users and groups. The advantage of using the SSO source is that if you have a complex domain configuration, you can use the existing configuration from SSO without having to re-implement your domain delegation and bindings in vRealize Operations.
  
  
  • The full range of vRealize Operations permissions can be granted to imported SSO users and groups.
  
  
  
  


• vCenter
  
  
  
  • Example: administrator@vsphere.local, youruser@yourdomain.tld, yourdomain\otheruser
  
  
  • They can not log in to the shell of the system via SSH or VM console.
  
  
  • Users from this source are not imported in the traditional sense and vRealize Operations simply trusts the vCenter when it says that a user has permission to log in and see objects.
  
  
  • Because vCenter is responsible for assigning permissions, and because vCenter can only ever be aware of objects in its own inventory, it cannot grant permission to view or modify objects that are internal to vRealize Operations, nor to data that comes in to vROps from other sources.
  
  
  • The vRealize Operations roles cannot be modified or expanded for users that come from the vCenter source from the vRealize Operations side.
  
  
  • vCenter users are unable to create and schedule reports in vROps.
  
  
  
  

The vCenter authentication source is enabled automatically when vCenters are registered and is one of the options for authentication on the vRealize Operations login screen. This causes a great deal of confusion for many users that are using their domain credentials and can log in without issues, yet cannot perform various tasks despite being an "administrator" in the vCenter and/or domain.

Even in situations where "real" authentication sources are configured properly, the vCenter authentication source may still cause confusion since it can present itself as the default, allowing domain users to log in with partial permissions if they are not paying close enough attention. Unfortunately, there is no way to disable the vCenter authentication source, nor to change the default, so you must make an effort to educate your users about the differences and to use the appropriate source when logging in.

Hopefully, this clarifies the differences between the authentication sources and allows you to choose the one that best suits your needs. If any of the above is unclear, please drop me a line in the comments. Thanks for stopping by!

Read More

Upgrading vRealize Operations

vRealize Operations (formerly known as vCenter Operations Manager) is an indispensable tool for monitoring a VMware environment. The upgrade process between recent versions is riddled with pitfalls, however. Here, I will cover the process and make notes of all the gotchas so that you can have as smooth of an experience as possible.

The upgrade process goes from 5.x -> 5.8.1+ -> 6.0.x -> 6.2.

Upgrading from 5.x to 5.8.5

Prerequisites and Common Pitfalls:

• You should change your admin password before attempting the upgrade. New security policies that come with this upgrade will expire older passwords for admin. It must be a complex password that you have not used before. http://kb.vmware.com/kb/2013358


• You should ensure that you have a good amount of free disk space on the UI and Analytics VMs. You can check this with “df -h” on the command line.
  
  
  
  • Appliances that were originally deployed as 5.6 and earlier had a smaller root partition that may get filled by the update. If you run into any issues caused by this, you can follow http://kb.vmware.com/kb/2074688
  
  
  • If the /data partition is >85% full on either VM, you should add a disk and reboot. Boot-time scripts will handle data volume expansion onto new disks for you.
  
  
  
  

Steps to Upgrade:

1. Take a snapshot of both VMs.


2. Follow the instructions in the release notes for the upgrade at https://www.vmware.com/support/vcops/doc/vcops-585-vapp-release-notes.html#upgrade


3. After ensuring that you can get in to the UI, delete the snapshots.

Migrating data from 5.8.1+ to 6.0.x

Prerequisites and Common Pitfalls:

• You must be migrating from version 5.8.1 or higher. The data migration will be more reliable with version 5.8.5, but is known to work with all versions from 5.8.1+.


• You must have forward and reverse DNS entries in place for your source 5.8.1+ VMs, as well as for every node in your new 6.0.x cluster.


• Please size your nodes appropriately per the handy Excel spreadsheet attached to the bottom of http://kb.vmware.com/kb/2130551


• When naming and addressing your version 6 nodes, please note:
  
  
  
  • Node and host names can not have underscores
  
  
  • The roles of a node are subject to change, so naming a node according to a role may get confusing in the future.
  
  
  • Node names are extremely difficult to change, and attempting to do so is quite likely to break things.
  
  
  
  


• All nodes of a vROps 6 cluster (with the exception of remote collectors) must be on the same physical LAN (>1MS latency will cause problems) and must not be separated by a firewall.


• You should change your admin password on 5.8.1+ before attempting the migration. An expired password is common, sometimes difficult to identify, and will cause vague errors. It must be a complex password that you have not used before. http://kb.vmware.com/kb/2013358


• If you suspect that performance may be an issue, stopping DT calculation and new data collection on the 5.8.1+ appliance will improve the speed of the data migration greatly. http://kb.vmware.com/kb/2040008


• Although it is not supposed to be an issue anymore, there are some cases where DNS resolution does not work properly and this KB may still be necessary: http://kb.vmware.com/kb/2100944

Steps to upgrade:

1. Install the latest version of 6.0.x per the documentation at http://pubs.vmware.com/vrealizeoperationsmanager-6/index.jsp


2. After you bring your cluster online, you will be presented with a wizard. Don’t select the option for importing data or enter a license key when it asks you. Even though you are going to import data and probably have a license key, it is more reliable to do these things after the setup wizard completes in my experience.


3. Go to Administration>Solutions and select the tab called “Import Data”


4. Follow the prompts for importing data.


5. Once the import is complete, you can run the old 5.8 instance in parallel with 6.0 until you are comfortable with the results, then delete the 5.8 instance when you are ready.

Upgrading from 6.x to 6.2

Prerequisites and Common Pitfalls:

• Because the upgrade process will convert two of the databases to another type and not delete the source data sets (in case of issues), you must plan for this extra disk space to be consumed.
  
  
  
  • To calculate how much extra space will be consumed by the database conversion, you can log in to the shell of each data/master/replica node and run:
    $  du -sch $STORAGE/db/vcops/*xdb* | tail -n1
    You should add around 10% to this value to be safe, which is in addition to the 15% total free space you ought to maintain on /storage/db for general stability.
  
  
  • To check how much total free space is available on the system, run:
    $ df -h $STORAGE/db
  
  
  
  


• If you have nodes that are separated by slow, latent, or unreliable links, the update may time out when the master node pushes the update out to them. This will be apparent if the upgrade fails without presenting you with an EULA. You can pre-stage the update pak files to work around this: http://kb.vmware.com/kb/2127895


• There are a few instances where upgrading to 6.2 or 6.2a will take an indefinite amount of time to complete. If your upgrade takes more than 24 hours to complete, please contact VMware support. They will be able to help finish the upgrade.
  This is fixed in 6.2.1.


• These should no longer be necessary, but I'll leaving them here for reference:
  
  
  
  • Previous updates, patches and add-on Management Packs may cause issues if their files still linger in your update queue. To avoid issues with this, you should log in to each node via console or SSH and move the pak files in “$STORAGE/db/casa/pak/dist_pak_files/” somewhere else.
  
  
  • Large database blobs from a previous data migration may choke the update process. http://kb.vmware.com/kb/2132479
  
  
  • The cluster state files may have some malformed data in them. http://kb.vmware.com/kb/2106950
  
  
  • Expired projects may cause an upgrade failure when going from 6.1 to 6.2 http://kb.vmware.com/kb/2136536
  
  
  • There may be a malformed upgrade state file from previous versions. You should remove this before upgrading. http://kb.vmware.com/kb/2120616
  
  
  • I have a script that can automate many of the above tasks and more at
    https://github.com/nakedhitman/vrops-cluster-repair
  
  
  
  

Steps to upgrade:

1. Take a snapshot of all your nodes.


2. Install the OS Update pak file. (This step is very important and must be done before the application update.)


3. Install the Application Update pak file.
   
   
   
   1. Ensure that you check both the boxes so that the “Reset out of the box content” option is selected. If you do not do this, some parts of the system may not be upgraded. This will not affect custom dashboards or other user-created content. Only the content that ships with the system that you ought not to have changed will be affected. If you do have customizations to the built-in content, you can clone them to preserve your changes.
   
   
   
   


4. Log in to the product UI as admin and ensure that your dashboards, adapter instances, and other data are present and working as expected.
   
   
   
   1. If it makes you go through the first run setup wizard again: don’t panic, its probably fine. Just choose evaluation mode and complete the wizard. Everything should be there when it’s complete. http://kb.vmware.com/kb/2132452
   
   
   2. If you end up with vCenter adapter instances that cannot be configured, missing dashboards or licenses, or end up loosing historical data, you should revert your snapshot and tell VMware about it immediately. If you don’t have a support contract, then make some noise on the communities site. VMware pays attention to this stuff and wants to prevent it from happening.
   
   
   
   


5. Once you’re sure that everything is running OK, delete the snapshots.

Conclusion

Although the upgrade process is a bit of an ordeal that can consume a good chunk of time, the features, stability, and performance of a successful upgrade really are worth it. VMware is working hard to make it better, and does listen to the things that are said on the communities site should you have any issues.

Read More

Use Tags to not lose track of View Linked Clone Replicas

The Problem
If you've managed a View Linked Clone pool for any amount of time, you probably know the frustration that can come with trying to determine which replica is tied to a particular pool. This information can be especially important when trying to find unused replicas that are hanging around using up precious storage space. Or maybe when auditing your inventory, you find you have 5 View Pools, but 6 replica VMs. How do we know if we really are using all 6 replicas?

There is a public VMware Knowledge Base article (2009844) that utilizes SviConfig.exe to automate the process of finding unused Replicas, but sometimes this command can be less than helpful. The command can often fail with vague errors, and only works with registered replicas in vCenter's inventory. It also doesn't tell you what pool the replica is tied to!

There is also KB1031842 that goes over an ESXi command that can show mapping of what VM is using a particular replica. But in a large environment, this can be a tedious task parsing through the hundreds of lines of output.

The Solution
The nice thing about Replicas is that they are simply a clone of our Parent VM. This means that outside of the protection View puts on the VM itself.. it's a regular ol' virtual machine! These objects easily Tagged to help with sorting and inventory management.

Tags are a very powerful organization mechanism that were introduced in vSphere 5.1. Tags are recorded as metadata for the object it's assigned to and can be super helpful for searching the vCenter.

To create a Tag

1. From the vSphere Web Client Home, click Tags


2. Click the Items tab and click Tags


3. Click the New Tag icon


4. In the vCenter Server drop-down menu, select the vCenter you want the Tag to reside on


5. Enter a name for your Tag. I have matched my Tag name to my View Pool Name


6. For Category, select/create a Category. I named mine View


7. For the Tag Category options, choose Many tags per object. This allows multiple tags from the View category to be applied to an object at any one time.
   NOTE: You cannot change the cardinality from Many tags per object to 1 tag per object


8. Click OK
   

You can find more information on Tags at the vSphere 5.1 Documentation Center

Assigning your Tag to a newly created Replica

1. After creating your Linked Clone Pool in View, you'll find the parent VM being cloned to the Replica in the More Tasks pane

     2. Under Related events find the Replica name your Parent VM is cloning to

     NOTE: If your Replica was created earlier, no worries. You can search your vCenter's Task History      to find the Replica name

     3. Now you can go to your Hosts and Clusters view or VMs and Templates view

• Find and Right Click the desired Replica


• Click Assign Tag


• Choose your Tag and click OK


• Your Replica is now tagged!

Congratulations! You can now easily search your vCenter inventory for your Tag name and find which replica is tied to that Tag. Depending on your naming conventions, your search results may vary. As you can see from my example, my tag matches my View Pool's vm naming so that gets returned as well

You can also click the Tag name under Tags and find your assigned Replica

Thanks for reading!

Read More