v-Nick.com is now Systemsgame.com!

As anyone who follows my blog knows, it goes in bursts. I spend a ton of time working with clients on all things Cloud and Automation, but never have enough time to blog about it or get the info out there. Hopefully now that is about to change, and a new name makes all that happen right….right???

Why Systems Game?

I decided my little pun on NIC and my name has come to an end, and most of all I felt like a nice fresh coat of paint was in order. I wanted to think of something that really encompassed what I do on a daily basis. Automation, Orchestration, and Cloud are all related to systems, and well, since I absolutely love my job, and treat it as fun more than work, I figured it was more like a game.
The 2 came together and now we have Systemsgame.com

What’s coming?

I have a list of about 20+ posts to write at the moment around all my work with vRealize Automation, UCS director, the Microsoft stack, vRealize Orchestrator. Starting early next week I will be publishing an awesome series I’ve been working on for some Tintri Storage Automation. Following that I will be publishing a lengthy series on Building a Cloud with vRealize Automation and vRealize Orchestrator, combined with Puppet for Configuration Management.

Thanks for reading, and as always feedback is greatly appreciated!

vRealize Automation 6: Changing Lease Expiration Notification

Vector red expired stamp


One of the features I love about vRealize Automation is the ability to give out leases to various business groups for their VMs. It’s fantastic for Dev and QA environments. One annoying thing however, is that the default notification for expiration is 7 days, and there is no apparent way to change it in the UI. In fact it would be nice if we could send multiple warnings out. Perhaps the first warning 7 days before, and the second warning a day before the VM expires? That’s for a future discussion though.

For now, here is how to change the number of days before the user is sent a lease expiration notification:

1. RDP to your IaaS Windows Server and locate the “ManagerService.exe.config” file located in “C:\Program Files (x86)\VMware\vCAC\Server”



2. Open the file and search for “DaysNotifcationBeforeExpire”. The default should be set to “7”.



Change it to the number of days you desire? I personally like a 2 day notification, but it’s really a preference that makes sense for the users of your Cloud.

Note: While you are here you can also adjust many other automatic e-mail notification events from their defaults. Anything you change to true will result in an e-mail notification. For example: Changing PowerOff=”false”  to  PowerOff=”true” will result in an e-mail notification every time the machine is powered off.

3. Save the File

4. Restart the vCloud Automation Center Service

Select Start > Administrative Tools > Services and restart the vCloud Automation Center service.


And there you have it! Lease expiration notifications when you want them!

While this solves the problem, hopefully VMware will create the LeaseExpirationNotifcation as a Stub so that we can call out directly to a vRO workflow when the lease is about to expire.


vCO Workflow: Change VM Port Group for VM on Standard vSwitch

I was surprised recently to find that no builtin workflow existed for changing the backing information for a VM if you aren’t using a VDS. Now, before I go any further, I’m a big fan of moving to a vSphere Distributed Switch mode,l but there are certainly cases where you might encounter a standard vSwitch environment which you need to automate port group changes upon.

The Approach:

Essentially when it comes to changing NIC settings on a VM, you have to change the “Backing” information for the NIC associated with the VM. In my case this was for VMs which were just built as part of an overall automation process, and had only one NIC.

Step 1: Create Action Item.

I created an action item which has 2 inputs.

“vm” of type VC:VirtualMachine – This is basically so you can select the VM in vCO that you want to modify

“vSwitchPGName” of type String – This is so you can pass in the string value of the portgroup name for the vSwitch.


The code I then used is below. I’ve commented it but please let me know if you have any questions.

var spec = new VcVirtualMachineConfigSpec(); // Initialize a Virtual Machine Config Spec first
var myDeviceChange = new Array(); // Create an array to hold all of your changes
var devices = vm.config.hardware.device;

//Find devices that are VMXNET3 or E1000
for (var i in devices)
		if 	(
				(devices[i] instanceof VcVirtualVmxnet3) ||
				(devices[i] instanceof VcVirtualE1000) 
			System.log("The device we are going to modify is: " + devices[i]);
			var nicChangeSpec = new VcVirtualDeviceConfigSpec(); //This is the specification for the Network adapter we are going to change
			nicChangeSpec.operation = VcVirtualDeviceConfigSpecOperation.edit; //Use edit as we are going to be modifying a NIC
			nicChangeSpec.device = new VcVirtualE1000;
			nicChangeSpec.device.key = devices[i].key; 
			System.log("NicChangeSpec key is : " + nicChangeSpec.device.key);

			nicChangeSpec.device.addressType = devices[i].addressType;
			nicChangeSpec.device.macAddress = devices[i].macAddress;

			System.log("Adding backing info" ) ;
			//Add backing information

			nicChangeSpec.device.backing = new VcVirtualEthernetCardNetworkBackingInfo();
			System.log("Backing info for nicChangeSpec is : " + nicChangeSpec.backing);
			nicChangeSpec.device.backing.deviceName = vSwitchPGName; //Change the backing to the portgroup input
			System.log("Backing info for deviceName on nicChangeSpec is : " + nicChangeSpec.device.backing.deviceName);

			//Push change spec to device change variable


spec.deviceChange = myDeviceChange;
System.log("DeviceChange Spec is: " + spec.deviceChange);
return vm.reconfigVM_Task(spec);

Step 2:

I created a simple workflow which calls this action item and then has a vim3WaitTaskEnd so we can be sure the task is completed before moving on to any other workflows. This is useful if you are going to be incorporating this action into a larger process.

Update Port Group for vSwitch

Running the workflow gives you this simple presentation.

vSwitchPG 2

And that’s basically all there is to it. Select your VM, type in your PortGroup name, and voila!

For a vDS, VMware included a workflow out of the box in vCO so there is no need to create any of the above.


vRealize Air Automation

I posted a new blog posted on the ThinkAhead Blog this morning around vRealize Air Automation which was recently announced at VMworld.


I am excited to see vCAC become simpler and easier to deploy. This means more time to work on vCO workflows and truly understanding the business cases, rather than troubleshooting the management tool.

So now we have… vRAA! (vRAWWWRRR). Definitely nicer to say than vCAKE.

vNick Returning

After a while working on 2ninjas1blog.com and various other projects, I’ve decided to fire up v-nick.com again.

My wife has kindly agreed to provide me with some new graphics, including a vNick with 2 years. This should please some people out there.

The blog will continue to focus on automation, but with a greater focus on Service Catalog enabled with vCAC and ServiceNow.

Expect more in the coming days, especially as VMworld gets underway!

Chicago VMUG Tomorrow!

Hoping to see lots of people at the Chicago VMUG tomorrow. If anyone isn’t already aware of the event, I recommend you go to http://www.chicagovmug.com/

Amy Manley (Twitter:  @wyrdgirl, Blog: http://www.virtualchick.com), and I will be presenting “Real World Automation with vCO” at 3:45pm. We are hoping to give a nice overview of how vCO Automation and Self Healing Datacenter Concepts can be used in the real world.

Look forward to meeting lots of you there and at the Nimble event afterwards!

Workflow: Find the SDRS Pod for a datastore name, and Refresh Recommendations

Summary/Use Cases:

This workflow can be used as part of a self healing solution to ensure SDRS runs whenever you get a datastore alarm. (see Self Healing Datacenter). This ensures SDRS immediately runs and can balance the storage in the cluster.


  • Datastore Name: Type = String

Outputs: None

The Workflow:


How it works:

  1. Takes the Datastore string passed to it and then searches vCenter for a Datastore Object matching the stringtext. This is stored as type VC:Datastore.
  2. Looks through all of the SDRS PODs declared in your general attributes to see if any of them contain the Datastore object found in step 1.
  3. If there is a match, it stores this as the variable “podToRunSDRSOn”, and stops searching.
  4. Run’s refresh recommendations on the pod “podToRunSDRSOn”.

 The Code:

Only 1 scriptable task has been used above, and i’m considering making it an action item if I don’t expand on the workflow further.

//Search vCenter for a Datastore matching the name

var dataStoreObject = System.getModule 
dataStore = dataStoreObject[0];
System.log("Datastore Object Found is " + dataStore);
//Now we have the datastore in question, we need to find out which SDRS POD the Datastore 
is a member of.

//Checking the array of pods
for (var i in podsArray)
 System.log("Checking POD: " + i + "in podsArray. The current POD is" + podsArray 

 //Grab the child entitys of the pod and put them in the arrayCheckDatastore

checkDatastore = podsArray[i].childEntity;

//Now check each datastore against the original one we found to verify the 
objects are the same

for (var t in checkDatastore) 
 System.log ("Datastore at object " + t + "is : " + 
 if (checkDatastore[t] == dataStore) 
 System.log("Datastore Match. The pod we need to 
run SDRS on is " + podsArray[i]);
 podToRunSDRSOn = podsArray[i];
 System.log("There was no match...we could not 
find an SDRS POD");
System.log("CHECKING COMPLETE: Pod to run SDRS on is : " + podToRunSDRSOn);


if (podToRunSDRSOn != null)
var m = podToRunSDRSOn.vimHost.storageResourceManager;
task = m.refreshStorageDrsRecommendation(podToRunSDRSOn);


VMworld 2013 – Self Healing Datacenter Example 1 – Automating HA/DRS Configurations

First of all, thank you if you attended the session I presented with Dan Mitchell on the Self Healing Datacenter. Also a big thanks to Kim Jahnz and Dan for giving me the opportunity to go up there and talk about some of the things I’ve been working on.

Here is the detailed walkthrough of the 1st example I gave.


In this example I demonstrated how you could effectively use vCO as a Configuration Management tool for vSphere settings. Now this does not mean that there aren’t more advanced tools out there for configuration management. Puppet, Chef etc. are your big knife tools for more serious configuration management, but vCO can fill some very easy gaps without needing to go to more complex tools.


Always start with a goal in mind and then work your way through all the components required.

  • HA Settings
    • HA should be turned on
    • HA admission control should be enabled
    • HA Admission Control policy should be set to percentage based and allow for 1 host in the cluster to be out of service. (e.g. for a 4 host cluster, I would set this to 25%)
    • DRS
      • DRS should be turned on
      • DRS should be set to fully automated

Break it down:

The very first time I created this workflow, I used workflows containing scriptable tasks. Later I improved it ahead of VMWorld and turned that workflow of scriptable tasks into an action item.

I don’t think there is a right or wrong answer to this, but using the action item seems cleaner and more reusable, with less chance of error.

The main workflow:


Now for the action item code…

Part 1: Calculating your HA % for the amount of hosts in your cluster.

var Hosts = System.getModule("com.vmware.library.vc.cluster").getAllHostSystemsOfCluster(cluster);

System.log("Number of Hosts in Cluster: " + Hosts.length);

var HApercent = ((1/Hosts.length)*100);
HApercent = HApercent.toFixed(0);

System.log("HA Percent which will be used for cluster is: " + HApercent);

Part 2: The cluster specifications and task to reconfigure the cluster

//Create variables for DRS/HA config

var clusterConfigSpec = new VcClusterConfigSpecEx();
clusterConfigSpec.drsConfig = new VcClusterDrsConfigInfo();
clusterConfigSpec.dasConfig = new VcClusterDasConfigInfo();

//Enable DRS/HA

clusterConfigSpec.dasConfig.enabled = true;
clusterConfigSpec.drsConfig.enabled = true;

//Set DRS to INPUT (Passed to the Action)

System.log("Setting DRS to Fully Automated");

clusterConfigSpec.drsConfig.defaultVmBehavior = drsBehaviour;

//Fix Admissions control policy

System.log("Updating HA Admission Control policy for " + cluster.name);

clusterConfigSpec.dasConfig.admissionControlPolicy = new VcClusterFailoverResourcesAdmissionControlPolicy();
clusterConfigSpec.dasConfig.admissionControlEnabled = true;

//Set host monitoring to the setting passed to the action

clusterConfigSpec.dasConfig.hostMonitoring = haHostMonitoring;
clusterConfigSpec.dasConfig.admissionControlPolicy.cpuFailoverResourcesPercent = HApercent;
clusterConfigSpec.dasConfig.admissionControlPolicy.memoryFailoverResourcesPercent = HApercent;

//Reconfigure the cluster, by adding the True parameter this ensures any previous settings remain

System.log("Executing Cluster Reconfiguration for " + cluster.name);
task = cluster.reconfigureComputeResource_Task(clusterConfigSpec, true);

Putting it all together:

So now we have an action item which can take in the following inputs:

1. Cluster (Type: VCCluster)
2. DRS Behaviour (Type: DRS Behaviour)
3. HA Host Monitoring (Type: Boolean)

Now comes the easy part, you can just create a workflow where the 3 inputs required above are inputs, or general attributes. If you use Inputs then you will be prompted each time. If you choose general attributes then they are set permanently in the vCO workflow unless you change them.
I chose to set the DRS Behaviour/HA Host Monitoring settings as general attributes inside the workflow which I called “Configure HA/DRS Settings for Cluster”. Then I would just select the cluster I wanted to apply the settings to when I ran the workflow.

Now once I decided this worked great, I wanted to push the settings out to ALL of my clusters, so I created the main operational workflow “Configure HA/DRS for ALL Clusters”. I then made an array of clusters as the general attribute, which I could add in, or take out clusters as I wanted to.

Finally, schedule the workflow in vCO to run every night at midnight, and you know that all your clusters are exactly as they should be. If you are working on a cluster, just take it out of the array and put it back in when you are done.

vCenter Orchestrator Appliance – Guest File Operations (Copying a file to guest VM)

One of the things you will often find you need to do with vCO is to get a file to a guest VM, or just run a file from inside the VM. Now for Windows you can use Powershell remote features in many cases, but what if your server isn’t on the network yet? Until version 5.1 we had to rely on VIX as a way to do this, but now VMware has added a number of new workflows under “Guest Operations” which are much more reliable.

vCO Guest Operations

vCO Guest Operations

“Copy file from vCO to guest” is the one I’m going to be using in this example.

First of all copy the workflow into a sandbox area. This way you can move a bunch of the inputs to attributes and not have to key them in each time (e.g. The local administrator username, password, and test VM).

In my example, I’m going to create a text file called test.txt in a new folder under /opt called “vcofiles”.

My target machine is a Windows 2008 R2 server, where I will copy the file and place it in the C:\temp\ folder with the name “testcopy.txt”

If you run the workflow then these are my input parameters:



The problem is that if you run this you will get an error similar to this:

“No permissions on the file for the attempted operation (Workflow: Copying files from vCO appliance to guest/Scriptable task…”



In order to fix this you first need to give the correct rights to the folder and file on your vCO Appliance.

1. Login as root onto the appliance
2. Give Read/Write/Execution rights to the new folder


3. Give Read/Write rights to the Text file you made



Unfortunately we aren’t quite done yet. You also need to tell orchestrator which locations it can read/write/execute from. This involves editing the “js-io-rights.conf” file located in “/opt/vmo/app-server/server/vmo/conf”


Add the line “+rwx /opt/vcofiles/” as shown above.

If anyone isn’t too sure on the linux commands to do this:

  • Type “cd /opt/vmo/app-server/server/vmo/conf” and press enter.
  • Type “vi js-io.rights.conf” and press enter.
  • Use the arrow keys to move the cursor where you want and press the insert key
  • Press Enter and type in the line “+rwx /opt/vcofiles”
  • Press ESC
  • Type “:wq” and press enter.

4. Now, there’s one more thing. You need to restart the vCO service for this to take effect.

Login to the vCO configuration manager, go to startup, and click restart service.


5. Now run your workflow and see if your text file copied across.


You can see a quick video demo of this on youtube. (apologies for the mouse pointer issue..)


Thanks for reading. Let me know if you have any questions.