Category Archives: Tech

Extending vRealize Operations Actions with the vRealize Orchestrator Solution and Workflow Package

When vRealize Operations Management 6.0 was released, VMware increased the flexibility afforded to administrators by adding the concepts of symptoms, recommendations and actions to the product. As you might expect, symptoms are thresholds or characteristics that define when a problem may have occurred or additional guidance may be needed. Recommendations are a customizable way to define what that additional guidance might be – and actions allow you to automate and carry out that guidance.

Since then, one of the most frequent questions from my customers has been “When will we be able to use vRealize Orchestrator for these?”

I’m pleased to report that VMware has now enabled that capability via the vRealize Orchestrator Solution and Workflow Package for vRealize Operations. This package is available at the VMware Solution Exchange right now, and the purpose of this post is to guide you through the installation and configuration of it. The package adds many frequently-requested workflows, including:

  • Decommission a Host
  • Place a Host into Maintenance Mode
  • Perform a Power Off or Reboot on a Host
  • Manage VM or VM Group Snapshots
  • Migrate a VM or VM Group
  • Power Off, Power On or Reboot a VM or VM Group
  • Reconfigure a VM or VM Group (CPU and Memory settings)
  • Upgrade the VMware Tools for a VM or VM Group

Clicking the links above will bring you to the Solution Exchange portal where you can read more about and download the package. Click the blue “Try” button to initiate the download.

VSX_Download_vRealize_Orchestrator_Solution_and_Workflows_for_vRealize_Operations

Once you have downloaded and extracted the ZIP file, it’s time to start the installation. The first thing you’ll want to do is ensure that both your vRealize Orchestrator and vRealize Operations Manager are registered to the same vCenter instance. This can be done by comparing the data shown in the two screenshots below.

Validate_vRealize_Operations_vCenter_Connection

Validate _vRealize_Orchestrator_vCenter_Connection

As you can see above, both systems are taking to the same vCenter. We’re ready to begin!

First, you will need to import the Workflow package into your vRealize Orchestrator instance. Start by logging in to the Orchestrator Client.

Log_Into_vRealize_Orchestrator

Ensure that your client view is set to Administer

Switch_to_Administrator_View

Then, click on the Import Package button in the upper left of the right-hand panel.

Import_vRealize_Orchestrator_Package

Select the Remediation Actions Package (default filename is com.vmware.vrops.remediationactionsall-v15.package) and select Open

Select_Package_to_Import

You will be prompted to verify the software signature. Continue by selecting Import

Accept_Package_Signature

vRealize Orchestrator will then present you with a list of all of the new and changed elements that this package import will affect. No changes here are necessary, simply continue by clicking Import Selected Elements

Import_vRealize_Orchestrator_Package_Elements

Once the import completes, you will be able to view the new workflows. Click the Workflows tab to verify that there’s a whole bunch of new vRealize Operations Manager goodness present.

View_Imported_Workflows

You can also verify that the new workflows are present by switching back to the Run view, clicking the Workflows tab and expanding the new vRealize Operations Manager folder. You can see I already have a ton of great workflows by my friends Eric at Cloud Relevant and Sid at Daily Hypervisor in here.

Switch_to_Run_View_and_View_New_Workflows

That’s it for the vRealize Orchestrator side of things. Now you will need to switch over to your vRealize Operations Manager portal. Log in as a user with appropriate rights to add/update solutions. An admin user will work nicely.

Click on the Administration button, followed by the Solutions section. Then, click the Green + to add a new solution.

Import_New_vRealize_Operations_Solution

Select the solution file using the Browse button and click Upload. Once the upload completes and the PAK file has been verified, click Next to proceed with the installation.

Select_Solution_PAK

Accept the EULA and click Next again. Wait for the installation to complete, then select Finish

Complete_Solution_Installation

You can now verify that your new solution is installed by locating the vRealize Orchestrator Actions Adapter in the solutions list. Note that you may have to scroll down to find it, if you have several solutions installed. You may also notice that the adapter instance is not yet configured. Let’s tackle that next!

Verify_New_Solution_is_Installed

To configure the adapter instance, ensure that the vRealize Orchestrator Actions Adapter is still selected, then click the Gears icon at the top, next to the Green + we clicked a few steps back.

Give your new adapter a name, and enter the IP or hostname of your vRealize Orchestrator instance. Be sure to use the same Orchestrator instance as we verified at the beginning of this process. Click the Green + to add credentials for the instance.

Configure_New_vRealize_Operations_Solution

Enter your credentials and click OK

Add_New_Credential

Next, click Test Connection. You may be presented with a certificate warning – click OK if you trust the certificate, and then your test should be successful!

Accept_vRealize_Orchestrator_Certificate

Solution_Test_Successful

Save your new adapter by clicking Save Settings and finally Close the configuration dialog.

That’s it for the installation! You can verify that the new actions are present by clicking on the Content tab inside vRealize Operations and selecting Actions from the list on the left. If all went well, you should see the 8 new actions present. These can now be combined with symptoms and recommendations to unlock many new possibilities for remediation inside your environment.

View_New_Available_Actions
(Click for larger image)

Since it’s not even 9am yet, today’s post will be brought to you by the Zesty Bacon Bloody Mary from the Boon Fly Cafe in Napa, CA. This exceptional libation combines top-shelf Vodka with Boon Fly’s own special spice blend, a celery salt rim and a massive slab of applewood smoked bacon to top it all off. Paired with Boon Fly’s fresh made donuts, it’s the best breakfast in the valley. Bloody Marys also have the (dubious?) honor of being the drink that’s OK to have first thing in the morning. After all, you’re not an alcoholic, you’re just a little tired.

Bacon_Bloody_Mary_Boon_Fly_Cafe

I hope this guide has proved useful and that you have a chance to head out to Boon Fly and try their delicious concoctions.

 

Important – vRealize Automation VAMI authentication issue when upgrading to 6.2.1

For those of you who will be upgrading your vRealize Automation appliance to 6.2.1 now that the new version is available, please be aware of an issue that you may encounter.

After upgrading your appliance to 6.2.1 via the VAMI and rebooting, you may find that you are unable to authenticate to the VAMI as ‘root’.

This can be fixed by logging in to the vRA Appliance from the console (as root – this account is unaffected) and running the following command:

chage -M 99999 root

This will reset the expired root password account and allow you to authenticate to the appliance VAMI interface again.

Alternatively, you could change the root password to reset the expiration.

Sorry, no wine content on this one – it’s WAY too early. This message pairs nicely with a cup of coffee.

Happy automating!

 

Monitoring vRealize Automation with vRealize Operations and Hyperic

Have you ever deployed vRealize Automation? If so, then you know that it has a highly complex architecture, made up of dozens of individual components – and has historically been a bit of a hassle to properly monitor.

That said, there’s good news for administrators who have both the vRealize Automation and the vRealize Operations Advanced edition  – VMware has released a brand-new way to integrate the two, via the vRealize Automation Management Pack.  This new management pack brings detailed  application-aware monitoring of the full architecture of vRealize Automation, and includes a set of plugins for vRealize Hyperic as well as an updated vRealize Operations Management Pack for Hyperic. With the helo of this management pack and set of plugins, users gain the following capabilities:

  • vRealize Hyperic platform service monitoring for vRealize Automation related services
  • An inventory tree object in vRealize Operations Manager specifically tailored to vRealize Automation
  • A set of pre-defined symptoms, alerts, and recommendations for vRealize Operations specifically revolving around vRealize Automation monitoring

Before diving into implementation details, here are a couple of quick screenshots of what you can expect after deploying the new management pack and plugins.

vRealize Automation Environment View in vRealize Operations
(Click the above image for a larger version)

 

vRealize Automation Inventory Tree View in vRealize Operations

As you can see, it monitors the following high-level capabilities and their sub components :

  • vRealize Automation Appliance
  • vRealize Automation Infrastructure-as-a-Service (IaaS) Server
  • vRealize Business (Formerly ITBM) Appliance
  • vSphere Single Sign-On (SSO)
  • vRealize Orchestrator

Here’s today’s obligatory wine tie-in. Given to a friend when he departed the employ of Viansa, this bottle of 2005 Ossidiana was signed by his friends and co-workers from all aspects of the winery. It’s also a finely blended Bordeaux – representing the perfect marriage of the 5 noble French grapes. The blend is proprietary and not disclosed, but it was clearly more than a little Cab. All sorts of grapes, styles, workers, techniques and technology coming together to produce one harmonious and easily enjoyable product. Can you see why I was reminded of this exciting new marriage of Automation and Management when we opened this bottle last night?

IMG_4734

All that aside, let’s get into some of the nuts and bolts of implementing this new connection.

First, we must assume that you have functioning instances of vRealize Automation 6.1 or above, vRealize Operations Manager 6.0 or above and vRealize Hyperic deployed. Getting all of those up and running in your environment is outside the scope of this article. You will also need Hyperic agents deployed to all of the appliances and servers involved in the vRealize Automation  stack. These can include (but are not limited  to):

  • vSphere SSO
  • vRealize Automation Appliance
  • vRealize Orchestrator Appliance
  • vRealize Business Appliance
  • vRealize Automation Infrastructure-as-a-Service (IaaS) Server
  • Any additional Distributed Execution Managers (DEM)
  • External vRealize Automation IaaS Database Servers

Deploying these agents is also outside the scope of this article. Look for a forthcoming post on getting the agents onto the VMware appliances.

From there, you will log into your vRealize Hyperic server as an administrator with the rights to install plugins. Select the Administration tab and the Plugin Manager link.

Now, if you are currently running vRealize Hyperic 5.8.4, you may see some existing custom vRealize XML Plugins already present in the environment. These need to be removed first, and look like the following. If you don’t see these plugins, skip this step.

vRealize Hyperic XML Plugins for vRealize Monitoring
(Click the above image for a larger version)

To delete them, simply select the Checkbox to the left of each plugin and select Delete Selected Plugin(s) from the bottom left corner. This may take some time to complete.

Now click the Add/Update Plugin(s) button in the lower right corner and upload the two new .JAR plugin files.

After that’s complete, you should see something like the following image. Notice the two new custom JAR plugins, highlighted in red.

vRealize Hyperic JAR Plugins for vRealize Automation
(Click the above image for a larger version)

Now, switch over to your vRealize Operations console. Log in with a user who has the administrative rights to update solutions. Navigate to the Administration tab and select Solutions from the navigation pane. Click the Green + (Add) in the upper left corner of the solutions pane. Follow the wizard that is produced to install or update the solution.

vRealize Operations Solutions

If you already had the vRealize Hyperic solution installed and working, you’re done with this part! If this is your first time installing the solution, you will need to configure the adapter instance. To do so, highlight the vRrealize Hyperic solution and click on the Gears icon in the upper left. Fill in the requested details about your vRealize Hyperic server as seen here, of course using your own settings. Test and save the settings.

vRealize Hyperic Adapter Configuration

Now all you need to do is wait for vRealize Hyperic to auto-discover your new services. Check your Hyperic dashboard after a few minutes and import them; after a few more minutes they will start appearing in your vRealize Operations Manager.

You can confirm which vRealize Hyperic metrics are flowing into vRealize Operations by logging into it with an administrative account, then navigating to the Administration tab and Environment Overview. Expand the Adapter Instances and then your Hyperic Adapter Instance. You will see the name of the Hyperic instance that you configured in the last step – select it and view the related metrics.

vRealize Operations Manager Environment Overview
(Click the above image for a larger version)

That’s all there is to it – now you can navigate to your vRealize Operations Content tab and view the vRealize Automation inventory tree.

vRealize Operations Inventory Trees

From here you can explore the related tabs – environment, analysis, troubleshooting, etc – and begin leveraging the wealth of new metrics at your fingertips.

The new vRealize Operations and vRealize Hyperic integration packs can be downloaded from the VMware Solutions Exchange here and here.

Enjoy!

You can also see this article cross-posted on the VMware Management Blog at https://blogs.vmware.com/management/2015/02/monitoring-vrealize-automation-vrealize-operations-vrealize-hyperic.html

Creating DISA STIG Scorecards with vCM

In my previous life as an InfoSec guy, I was responsible for assessing, enforcing, and ensuring continuous compliance with all the various baselines for which my organization was responsible. At the forefront of this list were a long list of DISA STIGs (Defense Information Systems Agency Security Technical Implementation Guides) – a daunting task in any size environment with any size staff. Of course, this particular environment was fairly large, and the information assurance technical staff consisted basically of… me…  so  automating these processes became something of a necessity.

This is where one of VMware’s most versatile products comes in – the vRealize Configuration Manager (vCM.) This gem of a tool provides unified, cross-platform configuration and compliance management and enforcement of over 80,000 distinct controls from a single interface, complete with fully customizable reports, dashboards, and a whole host of other fun features.

Anyway, enough sales. On to the how-to.

This tutorial assumes you already have vCM installed and configured and able to communicate with at least one managed system. For this example we will be creating a Windows 2008 R2 scorecard.

To start, we will need to download the STIG content and the viewer tool straight from DISA.  The content is what’s known as a “Benchmark” and can be obtained from https://iase.disa.mil/stigs/Pages/a-z.aspx. The one we will want for this example is the “Windows 2008 R2 MS STIG Benchmark – Version 1, Release 15

Benchmarks

A few notes here.

  1. These benchmarks will update quarterly, on a fixed schedule found here.
  2. You will notice that there are “MS” and “DC” STIG bundles for Windows operating systems. MS refers to Member Servers, and DC to Domain Controllers. This is because there are additional and special requirements depending on which role the server fills. Make sure you select the appropriate bundle for your target system.
  3. You will also notice that there are “Benchmark” and “STIG” bundles. The STIG bundle contains much more information as well as a whole host of manual (non-automated) checks which are out of scope for this guide.
  4. Any time you see the *PKI designation next  to a link, this means that a DoD issued PKI certificate (smart card) is required to access this content. If you have one, great! If not, you simply won’t have access to this particular information.

Next, you will need to download the STIG Viewer. This is a Java based JAR application which will allow you to view, interact with and update your STIG scorecards. This can be obtained from https://iase.disa.mil/stigs/Pages/stig-viewing-guidance.aspx. Click the link for “STIG Viewer Version 1.2.0” in this example.

STIGViewer

Note that you will need to have a functioning JRE installed on your vCM server to use this tool.

Once you have the STIG Viewer and the appropriate benchmark for your guest operating system downloaded to the vCM server, we need to place the benchmark in vCM’s SCAP import folder. In a default install of vCM, this folder is found at C:\Program Files (x86)\vmware\vcm\WebConsole\L1033\Files\SCAP\import

SCAPImport

Tip: SCAP is the Security Content Automation Protocol, a standard designed to provide a framework for vulnerability management by the National Vulnerability Database.

Once the file is copied to the import location, it’s time to fire up the vCM console.

  1. Log in as a user with the Admin role or a custom role with access to the Compliance tools.
  2. Click on the Compliance slider on the left
  3. Expand the SCAP Compliance spinner
  4. Click on Benchmarks
  5. Click Import on the right hand panel to bring up the list of available SCAP benchmarks
  6. Using the arrow controls in the middle of the dialog, move the benchmarks you wish to import to the right hand side and click Next, followed by Finish on the next dialogBenchmarkImport

You will now see the new benchmarks listed in the Compliance slider on the left. If you expand them, you will notice that they are broken down into MAC (Mission Assurance Category) and CL (Confidentiality Level) categories. Be sure you know the MAC and CL for the systems you plan to audit –  the affects the stringency of certain technical controls.

MACandCL

Now it’s time to run a collection against the systems you want to audit. There are many ways to accomplish this, and I’m going to assume you have your own preferred method – but here’s a quick one just in case.

  1. Select Collect from the main toolbar at the top of the vCM interface
  2. Select Machine Data and select OK
  3. Choose the machine(s) you wish to audit – either from the list on the left, or use the filter, or machine groups, etc.
  4. Select Select a Collection Filter Set to apply to these machines and click NextCollection1
  5. Select the Regulatory Baseline Filters – Windows (for this example) filter set and click NextCollection2
  6. Click Finish

Now you need to monitor the collection job until it completes successfully. Wait until the job disappears completely from the Jobs list before continuing to the next step. This ensures that the data is fully merged into the vCM database.

JobsView

Now we can return to the Compliance slider.

  1. Expand the SCAP Compliance spinner, followed by the Benchmarks and the appropriate benchmark for the OS you are going to audit
  2. Select the appropriate MAC and CL for the system in question. For this example, we will use MAC-2_Sensitive
  3. Click Run Assessment in the right hand panel
  4. Select the machines you wish to audit from the upper list and move them to the lower list using the arrow controls
    RunSCAP
  5. Click Next
  6. Select if you wish to run the action now or later. For this example we will select Run Action Now and click Next
  7. Click Finish

A Windows SCAP Assessment job will be submitted to the Jobs list. Monitor this until it completes, then select the appropriate MAC and CL from the Benchmarks list again to refresh the view.

You should see a list of your assessed servers that looks like this:

ResultOptions

Now you have quite a few options. You can choose from the pre-configured result types that vCM provides for you – the OVAL HTML result is a nicely formatted human-readable report that’s suitable for a build book, hard copy, etc:

OVALResults

But, to generate content that will work with the DISA STIG Viewer, you need to export an XCCDF-formatted XML file. To do this:

  1. Click Export from the toolbar
  2. Select the machines you wish to export data for. Each machine will generate its own XML file
  3. Click Next
  4. Select XCCDF Results – XML
  5. Click Finish

You will receive a dialog that looks like this when the export is complete.

ExportResults

Navigate to this folder: in a default vCM install it is C:\Program Files (x86)\vmware\vcm\WebConsole\L1033\Files\SCAP\export

Here you will see a list of the exported results files for the servers you selected in the last step.ExportedFiles

We’re almost there. Take a deep breath and another drink of your favorite adult beverage. Today I am personally drinking a truly excellent 2010 Miner Cab from the much-coveted Stagecoach vineyard. This vineyard in the eastern hills of the Napa Valley produces fruit for some of the biggest name wines around, and with good reason.MinerStagecoach

Refreshed? Good – back to the STIGs. Now you’re going to want to fire up that DISA STIG Viewer we downloaded at the beginning. Provided you have a properly installed JRE, you should just be able to double-click the JAR file.

You’ll then be greeted with this friendly government-issue GUI. Never fear.STIGViewerGUI

  1. Select File and Import STIG from ZIP from the menu bar
  2. If this is your first time importing a STIG bundle, you will be prompted to create a savepoint. Select Yes
  3. Navigate to the folder where you stored the STIG Benchmarks we downloaded at the beginning of this guide. Be sure to select the one(s) which apply to the compliance results you exported earlier.
  4. You’ll see the viewer is now populated with STIG controls.STIGControls
  5. Now we must create a checklist from this raw data. Select Checklist and Create Checklist – Current STIG from the menu bar
  6. You will now have a STIG Checklist which you can enter your own data into. Notice that the Host Target Data in the lower left corner is not populated, and all of the vulnerability statuses are set to Not ReviewedChecklistView
  7. This is it: the last step. Select Import from the menu bar, followed by Import XCCDF Results. Navigate to the XML file you exported from vCM earlier. Remember, by default it was located in C:\Program Files (x86)\vmware\vcm\WebConsole\L1033\Files\SCAP\export
  8. Voila! You will see that the checklist has been filled out for you. You can now review the checklist, mark it up, make manual severity/status overrides, etc. If you save it as a .CKL file, it will be an acceptable artifact to most DoD certified auditors for the purposes of DIACAP/RMF. CompletedChecklist

Of course, you must be sure to be aware that every audit is different and you should check with your DOIM or local IA department to confirm these documents will be acceptable/sufficient for your purposes.

Tip: Much of what we just did can be scheduled inside of vCM. This removes a LOT of the manual work.

I hope this  guide has been useful to you.