My Next Big Thing

The Infrastructure Years

For the last 5 years, I have had a front row seat to a major transformation of Information Technology. In 2008 I was hired as employee #9 into a visionary company called Bluelock. Bluelock was a new kind of service company, completely dedicated to Infrastructure as a Service, First coined on Wikipedia my friend and CTO, Pat O’Day. Bluelock’s datacenters did not focus on traditional colocation. Bluelock focused on providing pools of resources dedicated to a particular client in a highly available multi-tenant infrastructure made up of multiple VMware VI 3.5 clusters. These pools are now known as Virtual Datacenters, and now come in all different shapes and sizes. As a Systems Engineer and later Solutions Architect, my passion gravitated toward the needs for automation of the infrastructure, not so much about adding infrastructure faster, but removing the human error from the equation. Automation doesn’t stop at the provider layer either. You may notice most of the content on my blog is about consuming cloud, not providing it.

Bluelock has been an amazing journey for me, and I will miss the Bluelock family that I’ve been with all the way back to just our start-up days.

A few highlights from my Bluelock days include:

• vCloud 1.0 Powershell module
• vExpert award for 2010-2013
• vCloud Organizations Course at TrainSignal
• VMworld session: Bare metal to the Cloud, and Beyond with Alan Renouf and Eric Williams

 

Desired State

I’m excited to announce that I will be joining VMware in Dallas, Texas. It’s a big move for the Robinsons, but I am honored to say I will be working with Nick Weaver and an all-star lineup on the vCHS R&D team, who are building the next-generation of cloud infrastructure automation frameworks. This opportunity is an amazing trifecta of my passions in infrastructure, code, and automation. There will be lots of vCloud, Puppet and Razor in my future. I’m honored to be on the team and hope to share more very soon.

Important: If you are going to VMworld, be sure to attend Nick and Nan’s session on Cloud Automation and Project Zombie.

Thank you again to the VMware community for all the well-wishes and good lucks. Special thank you’s to my friends at Bluelock, and to my new friends at VMware (See you all soon :)).

 

VMware Tools reporting in vCloud with PowerCLI

When running VMs at a public cloud provider like Bluelock, it’s still important to maintain installations and proper versions of VMware Tools on your VMs. While some providers will make an effort to report this to you, others may leave the ball in your court. Thankfully, it’s a simple one-liner to report on VMware Tools in the cloud!

Get-CIVM | Select-Object name,vapp, @{n=“VMTools”; e={$_.ExtensionData.GetRuntimeInfoSection().VMWareTools.version}} | Sort-Object vapp,name

Simple VM reporting in vCloud with PowerCLI

Because of the unique requirement of VMs to exist inside vApps in vCloud Director, it’s a challenge to get a nice view of all the VMs and their locations within your vCloud Organization. This can mostly be satisfied with a Get-CIVM command, which will return great info about your VMs, but it’s missing the storage assignment. Here’s a sweet little script to report specifically on the VM resources in use inside your vCloud Organization:

$vms = get-civm
$objects = @()

foreach($vm in $vms)
{
 $hardware = $vm.ExtensionData.GetVirtualHardwareSection()
 $diskMB = (($hardware.Item | where {$_.resourcetype.value -eq "17"}) | %{$_.hostresource[0].anyattr[0]."#text"} | Measure-Object -Sum).sum
 $row = New-Object PSObject -Property @{"vapp" = $vm.vapp; "name"=$vm.Name;"cpuCount"=$vm.CpuCount;"memoryGB"=$vm.MemoryGB;"storageGB"=($diskMB/1024)}
 $objects += $row
}

# Use select object to get the column order right. Sort by vApp. Force table formatting and auto-width.
$objects | select-Object name,vapp,cpuCount,memoryGB,storageGB | Sort-Object -Property vapp | Format-Table -AutoSize

You can easily export to a CSV by using Export-CSV or just run the report to get output that looks something like this:

Setting Default VDSwitch security for vCloud with PowerCLI

Working as a public vCloud provider periodically gives me the opportunity to help clients that are running their own private vCloud. It’s a great side benefit to working with an experienced public vCloud provider in case you need some advanced help, or custom code/scripting!

In this case, my client needed to set default security permissions for all new VDPortgroups that were created by vCloud Director. This is different than setting security on a single portgroup. This sets the defaults for ALL portgroups created after the setting change! This is very handy when the network creation is out of our hands. 😀

############################################

# Configuration Settings                   #

############################################

$switchName=“dvSwitch”

$promiscuous=$true

$forgedTransmits=$false

$macChanges=$false

 

###################################################

# Shouldn’t need to edit anything below this line #

###################################################

$dvSwitch = Get-VDSwitch $switchname

$spec = New-Object VMware.Vim.VMwareDVSConfigSpec

$spec.configVersion = $dvswitch.ExtensionData.Config.ConfigVersion

$spec.DefaultPortConfig = New-Object VMware.Vim.VMwareDVSPortSetting

$spec.DefaultPortConfig.SecurityPolicy = New-Object VMware.Vim.DVSSecurityPolicy

$spec.DefaultPortConfig.SecurityPolicy.AllowPromiscuous = New-Object VMware.Vim.BoolPolicy

$spec.DefaultPortConfig.SecurityPolicy.AllowPromiscuous.Value = $promiscuous

 

$spec.DefaultPortConfig.SecurityPolicy.MacChanges = New-ObjectVMware.Vim.BoolPolicy

$spec.DefaultPortConfig.SecurityPolicy.MacChanges.Value = $macChanges

 

$spec.DefaultPortConfig.SecurityPolicy.ForgedTransmits = New-Object VMware.Vim.BoolPolicy

$spec.DefaultPortConfig.SecurityPolicy.ForgedTransmits.Value = $forgedTransmits

 

$dvswitch.ExtensionData.ReconfigureDvs_Task($spec)

I’ve written about getting around the missing dvPortgroup inheritance in the past, and there are also a couple good posts from Luc Dekens and Alan Renouf:

http://geekafterfive.com/2011/04/04/dvportgroup-inheritance/

dvSwitch scripting – Part 2 – dvPortgroup

http://blogs.vmware.com/vipowershell/2011/11/vsphere-distributed-switch-powercli-cmdlets.html

Last but not least, special thanks to William Lam for pointing me to the right spot in the API. 🙂

vCloud API 5.1 Metadata

There has been very good coverage and code samples for working with Metadata in vCloud Director 1.5 (See posts from William, Alan, and Clinton), so this will simply be an update post for Metadata in vCloud Director 5.1.

What’s different?

1. Typed Values – Metadata must now be put into a certain type. No more throwing random junk into a single metadata entry. 😀
2. Domain and Visibility – Providers now have the ability to provide System metadata entries on objects, which is not editable by users. Providers can additionally make the metadata private, meaning the users cannot see the System metadata on the object at all.
3. Objects – Metadata can now be added to new vCloud Director 5.1 objects, including disks and storage profiles. Additionally, I believe that because OrgVDC networks are now user manageable, vCloud users can add their own metadata to networks.

For more instruction on Metadata in the vCloud API, read “Working with Object Metadata” in the vCloud Programming Guide.

PowerCLI 5.1 R2 example:

$vapp = Get-CIVapp “MyVApp”

$metadata = $vapp.extensiondata.GetMetadata()

$metadata.MetadataEntry = New-Object VMware.VimAutomation.Cloud.Views.MetadataEntry

# I don’t have any other metadata, so index is [0].

$metadata.MetadataEntry[0].Key = “Jake”

$metadata.MetadataEntry[0].TypedValue = New-Object VMware.VimAutomation.Cloud.Views.MetadataStringValue

$metadata.MetadataEntry[0].TypedValue.Value = “vCloudNerd”

$vapp.ExtensionData.CreateMetadata($metadata)

vCloud VM and VApp Backups with PowerCLI

There are times when a snapshot doesn’t cut it, and asking the Backup Administrator or Cloud Provider to set up backups doesn’t make sense. Just a couple examples I’ve run into as a Cloud Provider:

• Upgrades with testing could take longer than a snapshot should exist for.
• Temporary workloads may need backups for “warm storage”

So where is the in-between? Wouldn’t it be great to hot clone a VM or VApp with some light scheduling and retention capabilities?

vCloudBackups is the Powershell module for doing this.

The vCloudBackups powershell module allows you to create local backups of VMs and vApps at any cloud provider running vCloud Director 1.5 or 5.1. By local, I mean the VMs and vApps will currently be cloned to the same storage the workloads are currently running on. If you don’t trust your SAN with this level of backup, you probably need to talk to you Backup Administrator or cloud provider for that off-site backup. 🙂

How does it work?

Very simply, the module is just doing hot-clones of VMs or vApps through the vCloud API. When backing up just VMs, they are stored inside a vApp simply called ‘Backups.’ Each VM backup is named with the vApp origination, VM name, and a date/time of the backup.

VApp backups are simply just exact copies of the originals, except named titled as Backup with a date/time of the backup.

“Retention” is built into the module as a number to keep. The oldest will automatically be deleted.

Scheduling is accomplished through your Windows Task Scheduler and an included script called MyBackups.ps1. Credentials for your vCloud Director login are stored encrypted in a configuration file, meaning NO PLAIN TEXT PASSWORDS!

If you are interested in the vCloud Backups script, please contact me on Twitter, Google+, or the comments below.

vCloud 5.1 API Support in PowerCLI 5.1 R2

Quite a few people have been asking about vCloud 5.1 feature support in PowerCLI. Guess what: It’s here! PowerCLI 5.1 R2 has officially dropped, and the new vCloud 5.1 features are built in!

You will find all the new 5.1 API properties and methods in the .extensiondata of objects such as a VM returned by Get-CIVM. Snapshots come to mind as a frequent request. Here’s how to find them:

$vm=Get-CIVM “MyVM”

$vm.ExtensionData | Get-Member

The .extensiondata property is really the same object as you would get from the Get-CIView cmdlet.

So, lets say we wanted to find out what new fancy 5.1 VM snapshot properties and methods are available to us:

$vm=Get-CIVM “MyVM”

$vm.ExtensionData | Get-Member*snapshot*

From this,  we see that we now have the ability to automate the creation, removal and revert of snapshots in vCloud Director 5.1!

Creating, Reporting on, and Removing snapshots is now easy!

# Create Snapshot (Snapshot memory false, quiece true, snapshot name, description)

$vm.extensiondata.CreateSnapshot($false,$true,“Snapshot”,“Description”)

# Get VM snapshot

$vm.ExtensionData.GetSnapshotSection().snapshot

# Remove Snapshots

$vm.ExtensionData.RemoveAllSnapshots()

Let’s check out another one. Edge Gateways are new in vCloud Director 5.1. How do we find those without any new cmdlets? Search-Cloud!

Search-Cloud -QueryType EdgeGateway | Get-CIView

Search-Cloud returns a search object, and from this we can get the .NET object with all the properties and methods.

$edge | Get-Member

Note that we don’t have to use the .extensiondata, because we used Get-CIView to directly get the .NET object.

Let’s check out the .configuration!

$edge.configuration

Both the vCloud Admin and Tenant PowerCLI snap-ins have been updated, so if your cloud provider is running on 5.1, enjoy the new features!

Now, this is certainly not all PowerCLI 5.1 R2 has to offer.  Powershell 3 support as well as an official VDS snap-in are also available! You can read more about these updates on the Official PowerCLI Blog! For tons of VDS examples, See Alan’s post!

Creating new vCloud Organization users with PowerCLI

 

There is not a New-CIUser cmdlet *yet*, but I’ll show you the way you can do it now!

We’ll need 3 things: A username, a password, and a role.

To get the role we want to assign the user, we’ll need to use Search-Cloud:

Search-Cloud -QueryType Role | select Name

This will give us a list of roles. Pick a role name. As an example, I want to add an Org admin, so I am going to run this:

$role = Search-Cloud -QueryType Role -Name “Organization Administrator” | Get-CIView

The previous line queried for the role, retrieved the role object using Get-CIView and assigned it to the $role variable.

Congrats, that was the hardest part… on to the good stuff!

We need to do two things next… Get our Org object, and create a new user object:

$org = Get-Org

$user = New-Object VMware.VimAutomation.Cloud.Views.User

Now we assign the fun stuff to our user object (name, password, role)!

$user.Name =“JakeRobinson”

$user.Password = “myPassword”

$user.Role = $role.href

$user.IsEnabled = $true

and finally, we push the user object to our Org:

$org.ExtensionData.createUser($user)

So that’s it! Probably 10 lines of code and we have a new user. The secret is really in the .extensiondata of the objects like $org.

[Now Released] vCloud User Training

 

A challenge for a lot of vCloud Providers is training the vCloud users. The current training available is for running your own vCloud Director environment, and what little there is  for the vCloud users is mixed with provider level stuff as well. This can be very confusing for the users, as you can imagine.

 

 

I am thrilled to say that TrainSignal now has will soon be coming out with a course specifically tailored to vCloud Users!

The course focuses on using vCloud from an Organization perspective. It includes real world examples, best practices, and pitfalls that I’ve fallen in so you don’t have to! In addition, you’ll learn about some cool tools available to users that integrate with a vCloud Organization.

If you are a provider: Contact the TrainSignal team to inquire how you can provide this course to your vCloud users.

If you are a user: Pass this info along to your vCloud Provider, or you can simply purchase the course!

Shout outs to Sean and the rest of the TrainSignal team for all the hard work making my first authored course a reality!

vCloud Director Snapshots with PowerCLI

If you or your provider are already running vCloud Director 5.1, you now have access to take a single snapshot of your VMs!

If you want to play with snapshots in PowerCLI, you’re going to need the module in this blog. PowerCLI 5.1 currently only supports the vCloud API 1.5, which means there are no 5.1 API capabilities yet. (I stress “yet.” :D)

The module uses the current PowerCLI for it’s session and API access, and then bolts on some 5.1 API snapshot magic.

To use the module, simply download the .psm1 file and:

Import-Module vCloudSnapshots.psm1

After that, you’ll have access to 4 cmdlets:

Get-CISnapshot
New-CISnapshot
Remove-CISnapshot
Set-CISnapshot

To get snapshots:

# Get vApps with Snapshots
Get-CIVapp | Get-CISnapshot

# Get VMs with Snapshots
Get-CIVM | Get-CISnapshot

# Get Snapshot info for a particular VM
Get-CIVM "MyVM" | GetCISnapshot

It should be noted that all the cmdlets apply to a VM or vApp, so always make sure you’re doing a Get-CIVM or Get-CIVapp first.

For example…here’s creating, reverting, and removing:

# Create a snapshot for a vApp (or all VMs in the vApp, really.)
Get-CIVApp "MyVApp" | New-CISnapshot

# Revert to our snapshot
Get-CIVApp "MyVApp" | Set-CISnapshot -revertToCurrent

# Remove our snapshot
Get-CIVApp "MyVApp" | Remove-CISnapshot

For more help, just use Get-Help and the command!

vCloud Snapshots Powershell Module