Ahead Tech Summit 2015 – Building a Highly Automated Hybrid Cloud – and thank you!

As most people know, I work for Ahead, a Partner based out of Chicago. I have been here for over a year and have loved every minute of my time there, continuing to evolve the work my predecessor, Alex Mattson, started here as an Automation/Cloud specialist.

2 weeks ago, we hosted our 2nd Ahead Tech Summit and I was fortunate enough to be asked to present on Building a Highly Automated Hybrid Cloud. The video of my presentation is below.

This is the 2nd year I’ve presented at the Tech Summit, attendance was up and it was an absolute joy. We had more to showcase and I love talking about this. Many customers approached me afterwards and said they really enjoyed the presentation. As always I love feedback and hearing how I can improve, so please post your thoughts in the comments section below.

In addition, as I watched this back, I did want to thank some people who have really helped me grow into a better presenter. When I compared this talk, to my one last year, it feels like it was night and day.

Kim Jahnz – another good friend of mine who I met as a TAM at TransUnion. Kim helped catapult my career, pushing to get me a talk at VMworld in 2013 on Self Healing Datacenter which was extremely successful. The feedback from the customers was great, and I thank you so much Kim for pushing me to do this at a time I almost said no due to other personal commitments.

Sean Sergent – Sean and I worked together in the UK when we developed our own consultancy practice. Thank you for all the leaderships you demonstrated to me and learning by “osmosis” as you once described it. Many of the behaviours I have today are a result of the work we did in the UK.

Adam Cavaliere – my good friend and colleague from Tintri. We worked together at TransUnion and Catamaran. At TransUnion we completed our first presentation together in front of a large audience showing the benefits of VMware. It’s amazing to think how nervous we were there, and now see the things we are doing today. Thanks for being a great friend and being an absolutely pain in the ass when we worked together at TU, but we did achieve great things!

Eric Kaplan – Our CTO at Ahead, who makes sure I remained on track with both of my Tech Summit presentations. Thanks (I think) for always filling up my Trello board with 5 minute meetings. I’m sure I heard that from Chris Wahl once as well.

Justin Lauer – You said one thing to me at dinner last year that I still think about. “I don’t think that nervous feeling you get before you go up for a presentation ever goes away”. I think this says a lot…I think it’s because we care so much and want to achieve great things.

Chris Wahl – 30 minutes sitting with you and getting presentation tips is something I hope to do again.”Don’t say so!”

Alex Mattson – Thanks for recommending me to Ahead. I loved our lunches and chats about Automation when we were at Catamaran. Many of those discussions and ideas we had floating around have helped shape the process we have today.

Rob Warren – Thank you for your amazing leaderships and rules to live by. Everything has a standard. There’s a standard for everything. Standardize, Procedurize, Automate…enough said :)

Frank Denneman – For minor tips, and recommending to me that I don’t orange drink juice the morning of my VMworld presentation.

Doug Barnes – For your amazing attention to detail when we started out in Automation and being a great friend. Would not be where I am today in this space if I hadn’t worked with you in the beginning.

Nick Rodriguez – For the 16 revisions you worked on with me to make an amazing PowerPoint deck.

Plus everyone at Ahead and many others. Definitely did not get this far alone, and I’m very pleased to be able to go on stage and present about the things I love to do in this space.

So with that said. Thank you and enjoy!

 

Automating Tintri with vRO Part 5a: Sync VM

What is Sync VM?

Every now and then a technology on a storage array gets me really excited. When Adam Cavaliere showed me Sync VM last month I was impressed. After talking about it and showing it to customers, they were very interested. More specifically they wanted to see it automated, and in vRealize Automation.

So what is it, and how does it work?

I think it’s best explained by Adam Cavaliere and Emad Younis who have posts and videos on this already.

Video showing Database Example: https://www.youtube.com/watch?v=bWsVLUb9O_A

Blog post by Emad Younis: https://www.tintri.com/blog/2015/04/tech-tuesday-time-travel-syncvm-snapshots

The vRealize Orchestrator Workflow

In this part (5a), I will explain how to use the vRO workflow I’ve created which is available on FlowGrab along with the others in this series. In part 5b I will be showing how to integrate this vRO workflow with vRA, to offer it as a day 2 operation.

Pre Requisites:

1. Completed Part 3 verifying that you can get a Tintri UUID from a virtual machine.
2. Have a source and target virtual machine in vCenter with the same disk configuration upon which you want to sync.
3. Download the latest Tintri Workflow Package I created and uploaded to FlowGrab.

Important!

Do not execute this vRO workflow without an understanding of SyncVM. The code in this example replaces 3 disks (SCSI 0:1, SCSI 0:2, SCSI 0:3 to be precise). If you need to use alternate disk configurations, then you will need to adjust the parameters and action item to compensate. Please post questions if you are unsure how to do this, but this assumes a good understanding of vRealize Orchestrator.

Step 1: Import the Tintri Workflow Package you downloaded from FlowGrab.

You should now see extra workflows in your vRO Design view as per the screenshot below.

syncvmwflows

Take a look at FG-SyncVMExample you can see this utilizes a number of the other workflows previously created.
Sync VM Example

SyncVMInputs

Running FG-Tintri-SyncVM by itself, you can see this requires the RestHost, SessionID, and a snapshot UUID from the source VM, as well as the Tintri VM UUID from the target VM.

In FG-SyncVMExample I use the modular components built previously to retrieve all the information required to initiate a Sync VM operation.

The process is as follows:

  1. Create Rest Session
  2. Tintri Snapshot the Source VM in order to get the latest data.
    1. Get the Tintri VM UUID
    2. Snapshot the VM
    3. Get the Snapshot UUID for the latest snapshot
  3. Get the Tintri VM UUID for the target VM
  4. Shutdown the Guest OS (This is because if you initiate a sync VM via REST API it will do a hard power off of the virtual machine.)
  5. Initiate SyncVM now that we have the UUIDs we needed.
  6. Power back on the target VM.

Step 2: Run the workflow!

Exampleinputsforsyncvm

Select your REST Host, type in a sample name for the snapshot (this could obviously be set as a general attribute or automatically generated in the future), select your source VM and target VM from vCenter…and off we go!

Step 3: End Result

Login to your Tintri VM Store and you should see a message on the target VM indicating that synchronization is complete.

Screen Shot 2015-06-23 at 3.43.37 PM

vRealize Orchestrator will also show logging to indicate the Sync VM operation was successful.

vrologsyncvm

What’s next?

Hopefully this workflow helps. I would recommend adapting the actions and workflow to your use cases. i.e. which disks to sync? How many VMs do you want to sync to?

In the next post I’ll be demonstrating how you can add this capability to vRealize Automation as a Day 2 Operation you can present to Software Development and QA teams in order to get the latest data.

 

 

 

 

Automating Tintri with vRO Part 4: Create Tintri Snapshot on VMware Virtual Machine

In this section I’m going to walk through utilizing the workflow I created to initiate a Tintri Snapshot on your Virtual Machine.

Pre Requisites:

1. Completed Part 3 verifying that you can get a Tintri UUID from a virtual machine.
2. Have a Virtual Machine in vCenter which is running on the Tintri VM Store.
3. Import the latest Tintri Workflow Package I created and uploaded to FlowGrab.

The Tintri Snapshot

It would take far too long to go into all the details of snapshots and how they work, so instead I’m going to refer you to this PDF from Tintri. This gives a detailed overview.

However, to be a little more clear, the workflow we are executing here is going to complete the equivalent of selecting a VM in the Tintri GUI and selecting “Take snapshot…” as shown in the image below.

snapui1

 

Note, that these snapshots will NOT appear in the vSphere client UI. This is expected.

If we want to view all the Tintri snapshots upon the VM, we can also select “View snapshots”, and the list will be displayed.

snapui2

 

Executing the snapshots with vRO

So we have a great feature within Tintri, but we don’t really want to go to the Tintri UI every time. By having the vRO workflow, we can execute them directly from within vCenter or as a Day 2 action within vRealize Automation.

Step 1: Import the latest package from FlowGrab

After importing the package you should see some additional vRO workflows and actions.

vROflowgrab

vROflowgrabActions

As you can see there are 2 SnapVM workflows. The “FG-Tintri-SnapVM” workflow just encapsulates the action “tintriSnapVMfromUUID”. The 2nd workflow “FG-Tintri-GetUUIDandSnapVM” combines all the workflows from Parts 2 and 3 to create the REST session and get the Tintri UUID. As a general rule I try always create the “lego brick” workflow, and then create a larger workflow to wrap all the pieces I need. I include them both, as there may be creative workflows you want to create and need to do multiple snaps etc.

vROflowgrabSchema

Step 2: Run the workflow

Run the vRO workflow and put in your REST Host, VM, and Snapshot name.

snapVMinputs

Check the logs to verify you received a successful snapshot creation.

snaplogs

The number that is returned at the end is actually the Snapshot UUID from Tintri. If you login to Tintri you should see your snapshot listed in the GUI.

snapVMvROUIresultIn addition if you browse to “https://YourTintriIP/api/v310/vm” you will see this listed as the current latest snapshot for your virtual machine.

snapshotID-JSONThat’s it. You can now successfully Tintri snapshot your VMware Virtual Machines using vRO! At the end of the series I’ll be showing how to add these to vRealize Automation as Day 2 Actions.

What’s next…

Part 5: Automating Tintri Sync VM with vRO – Expected June 2nd

 

 

 

 

Automating Tintri with vRO Part 3: Get UUID from VMware Virtual Machine

In this section I’m going to walk through utilizing the workflow I created to get the Tintri UUID from a VMware Virtual Machine.

Why do I need this workflow?

In order for us to be able to perform any actions against VMs from the Tintri perspective, we need to know the UUID of the VM. This workflow allows us to get it.

Pre Requisites:

1. Completed Part 2 verifying that you can create a session to your Tintri VM Store and are able to return a JSESSION ID.
2. Have a Virtual Machine in vCenter which is running on the Tintri VM Store you tested in Part 2.
3. Import the latest Tintri Workflow Package I created and uploaded to FlowGrab.

What is the Tintri UUID?

The Tintri UUID is the unique object identifier for which we need to reference when performing any Tintri operations against the Virtual Machine. This is how Tintri sees the virtual machine as opposed to VMware which uses the Managed Object Reference for the vCenter object.

For example, in VMware we have this VM:

demovm

If we look in vRealize Orchestrator, you can see that the Managed object reference for this VM is:

demovmvRO

In Tintri we have the same VM

demovmTintri

If we take it a step further and look at the Tintri API in your browser, you will see that Tintri has both values for the VM. The VM MOR, and the Tintri UUID. Our goal here is to get the Tintri UUID, so we can perform REST operations against it, like Snapshot, Replication, and Sync VM which are in the next set of posts.

tintriuuid

Step 1: Import the package you downloaded from FlowGrab

Once imported you should see some additional vRO workflows, and Actions.

newworkflows

Step 2: Take a peak inside the workflows

If you look in the workflow for Tintri-Session and Get UUID you will notice that this workflow includes the workflow from Part 2a for Creating the session as well as the other workflow we just imported.

sessionuuidworkflow

We first run the Create Session workflow, store the JSESSIONID as a general attribute, and then run the GetUUID fromVM workflow using the JSESSIONID we stored from the previous workflow.

visualbinding1

 

visualbinding2

Step 3: Run the workflow

Run the workflow and select a Virtual Machine.

runworkflow1

Check the logs and make a note of the MOR and UUID.

workflowrunlogs

Take a look in vRO at the VM from the inventory view and should see the Managed object reference matches the VM you selected.

Take a look in the Tintri API from your browser, and do a search for the MOR. You should see the matching Tintri UUID there as well.

What now…

Now we are able to successfully get the UUID, we can start to perform Tintri actions upon the VM using vRO. In the next part of the series we will be going through executing a Tintri Snapshot on the VM.

Part 4: vRO workflow: Tintri Snapshot a Virtual Machine – coming Tuesday May 26.

 

Automating Tintri with vRO Part 2: Create REST Session

In this post I’m going to walk you through utilizing the vRO workflow I’ve developed for creating a Tintri session over REST. This is the first workflow in the series, and without it, none of the subsequent workflows can be utilized.

Why do I need this workflow?

In order to complete REST requests against the Tintri API, an authenticated session is required. This workflow authenticates with the Tintri REST API, and returns the JSESSION ID. This JSESSIONID variable is then utilized for each of the REST requests in the next set of operations.

Pre Requisites:

1. Tintri VM Store Configured and accessible via REST
2. vRealize Orchestrator installed and configured
3. Download the vRO workflow package from Flowgrab

Step 1: Add your Tintri REST Host

First of all make sure you have the vRealize Orchestrator REST plugin installed. The REST plugin should have been configured automatically when you installed vRealize Orchestrator.

Run the “Add a REST host” workflow to add your Tintri VMstore. If like most users, you have more than one Tintri appliance, go ahead and repeat this process to add all of them.

addresthost

addresthost2

Choose “No” for Use Proxy

addresthost3

Choose “None” for Host’s authentication type

addresthost4

 

If everything is successful, you should see your Tintri Host listed in your inventory list.

addresthost5

Step 2: Import the Tintri vRO workflow package

Import the workflow package you from Flowgrab.

You will find the following objects now available to you in vRO:

Workflow:

The FG-Tintri-CreateSession workflow uses the action and configuration elements discussed below.

import1

Action:

Right now, you will only see the tintriGetSession action item highlighted below. This is the action item which returns the JSESSIONID.

import2

Configuration Element:

import3

The configuration element was created to store the Tintri UserName as opposed to storing it directly in the vRO workflow as a general attribute.

Edit the Tintri-ConfigurationElement

import4

Select the attributes tab, and input your password for the admin account.

Step 3: Run the workflow and verify we can create a session!

Now for the fun part! Run the FG-Tintri-CreateSession workflow.

workflowrun1

Select your REST Host and click submit.

Check the logs and verify you got a status code of 200 which verifies the session was successfully created, along with the JSESSIONID.

Congratulations, you were able to successfully authenticate with your Tintri VM Store using the REST API.

What next…

Part 3: vRO Workflow: Get a Tintri UUID from vCenter VM Object – scheduled for 05/20 – stay with me!

This next workflow allows us to select a Virtual Machine from vRealize Orchestrator, and returns the UUID from Tintri.

Part 2b: Tintri-CreateSession vRO Workflow Code – Go here if you want to see the code and how this was all created – scheduled for 05/21 – stay with me!

Automating Tintri with vRealize Orchestrator Part 1: Let’s begin!

Welcome to part 1 of my series on Automating Tintri with vRealize Orchestrator.

I’m sure almost everyone visiting my blog knows how awesome Tintri is so I’m not going to go into details of the product. In short, I love it because it’s simple and clones VMs incredibly fast.

The other hidden gem about Tintri is that there very well documented REST API allowing us to do all sorts of nice automation without having to click through the GUI. Recently Tintri announced Sync VM which got me really excited. I love all things DevOps, and when I was talking with Adam Cavaliere and other SQL server Engineers, they immediately saw amazing potential for this. After speaking with a few of my customers who had vRealize Automation, they really loved the idea of being able to have Day 2 Operations for tasks like Snapshotting their VM, and were even more excited about Sync VM and how that could be used.

So after further discussion, we decided to build a vRealize Orchestrator Package including actions, workflows, and all awesome stuff for using Tintri. In this series I’m going to walk you through how I built the package, and how you can use it.

The following workflows are included:

Tintri-CreateSession – Creates a REST Session with your Tintri VMstore. This is needed for all subsequent workflows.

Tintri-GetUUIDfromVM – Allows you to select a Virtual Machine from vRO, and will return the corresponding Tintri UUID. This is needed to perform any actions against Tintri objects.

Tintri-SnapshotVM – Utilizes the 2 workflows above, and then creates a Tintri Snapshot of your VM.

Tintri-SyncVM – Magical! Takes a Source VM and TargetVM, then syncs the VMDKs underneath. Tintri does all the heavy lifting. Check out a great video on this technology at https://www.tintri.com/blog/2015/05/tech-tuesday-manage-data-syncvm-video.

Tintri-ReplicateVM – Also magical! Enable Tintri replication on a VM for DR purposes.

As I release the workflows throughout the series, you will be able to find them on Flowgrab via the SystemsGame Page. If you aren’t already familiar with FlowGrab, I highly recommend you check it out. It’s a great place for sharing workflows with the vRealize Orchestrator community.

Finally, a big thank you to Adam Cavaliere for coming up with these cool ideas and working out all the kinks for the REST API calls in Python first. If you want an expert on Tintri, definitely contact him!

Next: Part 2 – Create REST Session

vRealize Automation 6: Changing Lease Expiration Notification

Vector red expired stamp

 

One of the features I love about vRealize Automation is the ability to give out leases to various business groups for their VMs. It’s fantastic for Dev and QA environments. One annoying thing however, is that the default notification for expiration is 7 days, and there is no apparent way to change it in the UI. In fact it would be nice if we could send multiple warnings out. Perhaps the first warning 7 days before, and the second warning a day before the VM expires? That’s for a future discussion though.

For now, here is how to change the number of days before the user is sent a lease expiration notification:

1. RDP to your IaaS Windows Server and locate the “ManagerService.exe.config” file located in “C:\Program Files (x86)\VMware\vCAC\Server”

ManagerServiceExeConfig

 

2. Open the file and search for “DaysNotifcationBeforeExpire”. The default should be set to “7”.

DaysNotifcation

 

Change it to the number of days you desire? I personally like a 2 day notification, but it’s really a preference that makes sense for the users of your Cloud.

Note: While you are here you can also adjust many other automatic e-mail notification events from their defaults. Anything you change to true will result in an e-mail notification. For example: Changing PowerOff=”false”  to  PowerOff=”true” will result in an e-mail notification every time the machine is powered off.

3. Save the File

4. Restart the vCloud Automation Center Service

Select Start > Administrative Tools > Services and restart the vCloud Automation Center service.

vCloudAutomationService

And there you have it! Lease expiration notifications when you want them!

While this solves the problem, hopefully VMware will create the LeaseExpirationNotifcation as a Stub so that we can call out directly to a vRO workflow when the lease is about to expire.

 

vCenter Orchestrator Appliance – Guest File Operations (Copying a file to guest VM)

One of the things you will often find you need to do with vCO is to get a file to a guest VM, or just run a file from inside the VM. Now for Windows you can use Powershell remote features in many cases, but what if your server isn’t on the network yet? Until version 5.1 we had to rely on VIX as a way to do this, but now VMware has added a number of new workflows under “Guest Operations” which are much more reliable.

vCO Guest Operations

vCO Guest Operations

“Copy file from vCO to guest” is the one I’m going to be using in this example.

First of all copy the workflow into a sandbox area. This way you can move a bunch of the inputs to attributes and not have to key them in each time (e.g. The local administrator username, password, and test VM).

In my example, I’m going to create a text file called test.txt in a new folder under /opt called “vcofiles”.

My target machine is a Windows 2008 R2 server, where I will copy the file and place it in the C:\temp\ folder with the name “testcopy.txt”

If you run the workflow then these are my input parameters:

GuestFileOperations-Run

 

The problem is that if you run this you will get an error similar to this:

“No permissions on the file for the attempted operation (Workflow: Copying files from vCO appliance to guest/Scriptable task…”

GuestFileFailure

GuestFileFailure

In order to fix this you first need to give the correct rights to the folder and file on your vCO Appliance.

1. Login as root onto the appliance
2. Give Read/Write/Execution rights to the new folder

FolderRights

3. Give Read/Write rights to the Text file you made

Filerights

 

Unfortunately we aren’t quite done yet. You also need to tell orchestrator which locations it can read/write/execute from. This involves editing the “js-io-rights.conf” file located in “/opt/vmo/app-server/server/vmo/conf”

Java-FolderRights-2

Add the line “+rwx /opt/vcofiles/” as shown above.

If anyone isn’t too sure on the linux commands to do this:

  • Type “cd /opt/vmo/app-server/server/vmo/conf” and press enter.
  • Type “vi js-io.rights.conf” and press enter.
  • Use the arrow keys to move the cursor where you want and press the insert key
  • Press Enter and type in the line “+rwx /opt/vcofiles”
  • Press ESC
  • Type “:wq” and press enter.

4. Now, there’s one more thing. You need to restart the vCO service for this to take effect.

Login to the vCO configuration manager, go to startup, and click restart service.

ServiceRestarted

5. Now run your workflow and see if your text file copied across.

Success

You can see a quick video demo of this on youtube. (apologies for the mouse pointer issue..)

 

Thanks for reading. Let me know if you have any questions.

Nick

 

 

 

Checking for VM Memory Limits

Here is a quick one liner I found to check for any VMs which had memory limits set on them:

Get-VM | Get-VMResourceConfiguration | where {$_.MemlimitMB -ne -1}

If you want to target a specific cluster, just add Get-Cluster “clustername” to the beginning:

Get-Cluster “Clustername” | Get-VM | Get-VMResourceConfiguration | where {$_.MemlimitMB -ne -1}

Now if you want to get rid of the memory limits, add the following:

Set-VMResourceConfiguration -MemlimitMB $null

Final script for all VMs to find and remove limits:

Get-VM | Get-VMResourceConfiguration | where {$_.MemlimitMB -ne -1} | Set-VMResourceConfiguration -MemlimitMB $null

Next step…setting this as a scheduled workflow in Orchestrator to run every night/week and send a report out of any limits discovered.