The Problem:

In Windows Powershell, deleting items with Remove-Item causes a confirmation prompt to stop a script from functioning. The prompt says:

Confirm

The item at [path] has children and the Recurse parameter was not specified. If you continue, all children will be removed with the item. Areyou sure you want to continue?

[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is “Y”)

The Solution:

Run Remove-Item with the -recurse switch.

Remove-Item C:pathtofile -recurse

Now your script will run with no intervention necessary.

The Long Story:

There is considerable confusion about how to supress confirmation prompts with Remove-Item. It’s rather silly since the solution is right in the confirmation wording (I am guilty of being silly since I didn’t see that at first either). Here are some false ways of performing this task along with why they are false:

Using the -Confirm parameter.

Some people will suggest that you use the following line:

Remove-Item c:pathtofile -Confirm:$false

However, -confirm is set to $false by default and furthermore it has nothing to do with the warning above. The -confirm parameter “prompts you for confirmation before executing the command.” In the above scenario, I’m not being prompted before running the command, I’m being prompted to confirm the deletion of a file that has child objects.

For more information abuot the -confirm parameter, run the following command in a PowerShell prompt:

get-help about_CommonParameters

Using the -force switch

This does not “force” the Remove-Item cmdlet to delete files in the face of a confirmation prompt. This forces the deletion of hidden and read only items.

I’m a closet storage geek. I don’t have a lot of clients that need really cool storage products, but I wish I did. When I can, I try to read up on as much information about the storage world as possible. I’ve come across three fascinating players in the flash storage world that any SysAdmin should keep their eye on.

Pure Storage

Pure Storage has a rather daring claim. Their arrays are claimed to be ten times faster, smaller and more power efficient than disks.

The product touts its software as much as (if not more than) its hardware. The software is called the Purity Operating Environment and performs global deduplication, compression and thin provisioning to make data storage more efficient. From their site:

Purity is a fully-virtualized storage operating environment, which abstracts individual flash devices into a single unified storage pool and optimizes data placement across the pool. Purity’s data layout is aligned with the erase block size of the flash, reducing flash write amplification to extend flash life and improve performance. Moreover, Purity’s data structures are “append only”, meaning that all writes (new data, updates, parity re-builds, recoveries) are coalesced into write segments that are always placed somewhere new, improving performance and extending flash life. Finally, Purity implements a set of active background flash management services (wear leveling, deletion management, performance optimization, integrity/health checking and automatic healing) across the global pool to ensure the reliability of both the data and the underlying flash.

As of this post they only offer two appliances. A 2 controller / 2 storage shelf version and a 1 controller / 1 storage shelf version. Pictured below is the single controller / shelf model:

Concerning the hardware, the site has this to say:

The Pure Storage FlashArray is built on a flexible, scalable, redundant, highly-available hardware architecture, designed to allow Pure Storage solutions to scale from single application to consolidated cross-data center deployments. The Pure Storage FlashArray implements a node-based design, with clustered controllers and storage shelves. This allows for the independent scaling of storage performance (controllers) and storage capacity (storage shelves). Configurations can range from 10s to 100s of usable TBs of flash storage, for both HA and non-HA configurations.

The Pure Storage CEO is none other than Scott Dietzen former president and CEO of Zimbra. The CIO is John Colgrove who worked for Veritas. Also on board is Michael Cornwell who worked on flash chips for the iPod and iPhone while at Apple. He has also worked on flash-based products at Sun.

Pure has attracted some important investors including Mendel Rosenblum and Diane Greene, the couple who founded VMware. Also funding Pure Storage is Greylock Partners which brings Pure Storage’s total amount of VC funding to about 50 million dollars.

I foresee big things for Pure Storage and hope to work with of their equipment some day.

Fusion-io

Fusion-io doesn’t actually make server appliances. They make flash devices that interface with a computer through the PCI bus. They make three major hardware products, the ioDrive, ioDrive Duo and ioDrive Octal (the latter shown below).

The Fusion ioDrive can supply 160GB (at 123,000 Mixed IOPS [75/25/r/w]) to 640GB (at 74,000 Mixed IOPS [75/25/r/w]). The ioDrive Duo can supply 320GB (at 238,000 Mixed IOPS [75/25/r/w]) to 1.28TB (at 150,000 Mixed IOPS [75/25/r/w]). Finally, the ioDrive Octal can supply 5.12TB at 729,000 75/25 Mixed IOPS (512 B).

Fusion-io went public in the summer of 2011. They have been invested into by Samsung and are working with the major flash chip manufacturer Toshiba.

They make a software platform called ioSphere that allows you the following features (taken from their website):

• Manage all ioMemory modules across all servers from a single interface
• Real-time
• Historical performance monitoring
• Warranty forecasting

Also available is the software tool known as direct cache. From the Fusion-io website:

Fusion’s directCache transforms ioMemory into a transparent, auto-tiering, acceleration device to cache any block-based storage medium whether it is a disk array, SAN, direct attached storage or iSCSI target. directCache places the caching software in the server to deliver lower cache latency. With directCache, Fusion-io customers can have Terabytes of cache acceleration at their fingertips to speed performance of any backing store. This add-on module for ioSphere integrates tightly with Fusion’s Virtual Storage Layer, a flash-optimized OS subsystem, to deliver immediate application workload performance improvements.

They look like a great tool for anyone doing CAD work or video editing. I’m not sure about building out a server full of them though. Perhaps I’m spoiled by the pretty boxxen that most vendors will supply.

Violin Memory

The former CEO of Fusion-io was Donald Basile. Why is that important to Violin Memory? Because he became the former Fusion-io CEO when he left to head up Violin Memory. Unlike Fusion-io, Violin Memory is focused on datacenter products. With an impressive catalog of appliances, I won’t go into depth on each of them here, but will give a brief  overview (using some or all of the text from their product website):

3200 Flash Memory Array - A redundant, modular 3U memory array that scales from 500GB to 10TB SLC NAND Flash

vCACHE NFS Caching - The NFS caching system is built on flash memory arrays in conjunction with their vCACHE software. These vCACHE NFS Caching systems increase the size of the caching available to applications, but also reduce the cost per GB of the cache by more than 70%. Unlike an internal cache, the vCACHE system can also support 200K operations per second. vCACHE systems enable the entire active data set to be stored in cache. This may be 5%, 10% or 20% of the total data stored in the filers. By caching the entire data set, the full application speed-up of 5x to 30x is enabled for both IOPS and latency. With caches of 1% or less, the speed-up is typically a small increase over the standard disk speed.

3140 Capacity Flash Memory Array - A redundant, modular 3U memory array that scales to 40TB of Capacity Flash. It scales to more than 500TB in a rack with performance over 1.5 Million IOPS. The Violin 3140 includes hardware-based flash RAID across hot-swappable memory modules to provide data protection and high-sustained IOPS.

SAN Attached - The Violin Memory Arrays can be clustered via PCIe with one or more Memory Gateways that provide connections via an combination of Fibre Channel (8Gb/s or 4Gb/s), 10 GbE (iSCSI or FCoE), or InfiniBand. A single Memory Gateway can support up to 4 Violin Memory Arrays, 400K IOPS and over 3GB/s. Through striping, each LUN on the system can get the full bandwidth and IOPS capability of the cluster. LUNs can range in size from 1GB to 120TB!

DRAM Array - Mmemory appliances made to provide a platform for provisioning DRAM as a large scale Tier-Ø storage infrastructure.

Violin provides “vCLUSTER Management” which is their storage management software to keep an eye on their products in your environment.

Flash storage is going to inevitably replace spinning disks, and it appears that with the above offerings, especially Pure Storage, it’s going to happen in this decade. Do you have any experience with the above systems? Or perhaps another upstart flash storage vendor? Let me know in the comments below.

Today I’m at the Phoenix, Arizona leg of Interface 2011. Once again, the event is billed as being in Phoenix, but it’s really somewhere rather far away from “Phoenix” proper. Technically it’s at Ft. Mcdowell at a Radisson Hotel that is adjacent to the Ft. Mcdowell Casino. For more information on the conference, check out the Event Details page.

The Plan

There are three theaters each with their own track. So far, I’m liking the following selections:

• 9:30AM Enabling Mobile Unified Communications. I’m facing the prospect of helping a friend out with his telephony business and would like to absorb some more information in this arena.
• 10:20AM PCI Myths and Mistakes. I’m currently helping a client become PCI compliant and am in need of every shred of PCI DSS information that I can find.
• 11:10AM SIEM 2.0 – See What Youre Missing. I have hopes of implement a small network monitor for a client at some point soon. I’d like to use a Fit-PC running CentOS and a few choice security tools. Any information that I can gain in the field of SIEM will be handy.
• 12:15PM Food. Mayhaps I’ll stick around to see what the Arizona chapter of InfraGuard has to say.
• 1:00PM Meh. Nothing seems to capture my attention. Maybe I’ll go to “Cloud Computing: Is Your Network Prepared?” just to see how little of substance is said. Perhaps I’ll sit in on “Desktop Transformation: Agility, Security & Improved Management” if the thought of a cloud talk makes me want to start cutting on myself.
• 1:50PM See What You’ve Been Missing. Similarly named session to the SIEM 2.0 one earlier in the day. This one is on applications that are supposedly not detectable by traditional firewalls. Of course, SSL tunnels is a topic.
• 2:45PM Keynote: Thinking Outside of the Box. The presenter, Chris Roberts, is someone whom I’ve never heard of before, but apparently has some kind of profile in the InfoSec realm. Anyone out there know about him?

After the keynote, I believe there will be the obligatory swag giveaways. I hope to score something this time to make up for bitter defeat at the AZVMUG event two weeks ago.

Live Blog:

This post is auto-posting at 7:25AM Arizona time. Registration for the event starts at 9PM, but I’m arriving fashionably early at around 8:30 or so. Perhaps I can stake out the wifi and plan my attack on the vendors. More news as events warrant.

9:32PM – Yes, that’s PM. The conference was an absolute whirlwind. Sessions were jammed end to end with only ten minutes in between. Often speakers went over time and I had barely enough time to visit the restroom or get a quick chat in with a vendor of interest. There was a ton of vendors, and they were actually pertinent and interesting. At least the ones that I saw. I’ll attempt to recap what I learned. Vagueness is to be expected.

Enabling Mobile Unified Communications: Don’t go away! It’s not as crumby what it sounds. The first half was a marketing spiel about blah, some blah and even a bit of blah. However, the second half was given by a systems engineer that specifically spoke about Meru Networks’ wireless technology. I was not expecting a wireless network talk, but I was impressed with what I saw.

Apparently Meru Networks deploys WAPs all on the same channel, however they circumvent noise by some kind of timing mechanism. With a single channel and I believe cloned MAC addresses, among possibly other things,  a deployment appears to clients as one access point and only one access point no matter where they go. This takes the burden of client migration among WAPs off of the client and onto the wireless infrastructure. It allows clients to be handed off to new WAPs when signal starts to merely degrade, rather than when the signal drops. All in all, it looked like a technology worthy of looking deeper into.

Of interest is this bragging video of their showing 500 wireless clients streaming 100Mbps and then reassociating with the network after a total WAP reboot in 3 minutes. The engineer was quick to point out that the latest product can do it in 90 seconds.

PCI Myths and Mistakes. The talk was well done by a PCI QSA from Accuvant. I was impressed by his continual insistence and proving that PCI DSS is not an IT issue. It is a processes issue that uses technology at certain points. If the IT department is who has been handed the PCI compliance project at your company, then something is terribly wrong. We were introduced to the IT Unified Compliance Framework which is a conglomerate compliance framework that hits many of the different standards, PCI being one of them. ISACA was also given a nod for having different “feature matrices” that compare the different standards to each other so that you know that if you’ve got PCI covered, what few things will remain to gain compliance in a different standard.

SIEM 2.0 – See What Youre Missing. Jim Schaeffer, president of JCS and Associates (which is basically a system integrator), spoke. The talk was nebulous, and I wasn’t sure what was being sold or what was being talked about. This wasn’t a talk about SIEM as a concept. This was what appeared to be a product rundown of several, seemingly unrelated products, that were offered as an integrated bundle on an appliance. The appliance then collected, stored and indexed all of the logs that those products used. The SIEM part of the talk was about how all of the logs that each of the major tools created were cross referenced in a superior way so that you could have better insight into your network. Basically, they take software products that they prefer and have proven and then make sure that the various security products have common threads that allow them to be tied together with the infamous “single pane of glass.”

Their hardware appliance was advertised to take all logs and correlate the data, rather than simple only store the small percentage of events that seem to be security related. They take different products and integrate them onto the appliance and add their own high-level view.

The first product that is used in the usage of their appliance is SafeEnd. Safeend is a suite of software that works on Macs, Windows that allows you to monitor physical ports and data ingress and egress as well as encrypt information on media. It also can show who is on the network with which device an on which wireless network. It’s quite an array of monitoring and controlling features. It’s one agent that is installed and the various flavors and features are licensed with a key.

Ctera is next on their list of integrated products on their appliance. It’s a cloud storage system that is based on heavy, end to end encryption. You have a local store and then a cloud store. You may have heard of the Ctera CloudPlug. They also make larger devices for SMBs.

Next, the appliance tackles the issue of desktop management. It uses Panologic devices as a nextgen thin client. The Buzzword is that it’s a “no client”. There’s no moving parts, no storage, no CPU, no OS. It’s not a traditional PC in that a thin client is just a PC that’s stripped down. There little cubes that are simply a place to plug in a video cable and four USB devices. It’s best use in networks that are sub 10ms. For offsite use of datecenter VDIs, there’s a serialized USB thumbdrive that you can plug into any PC, no matter how hosed it is with virii, and it will connect to the VDI instance as a secure session. It relies on a hypervisor like VMWare, Citrix and/or HyperV. Panologic uses it’s own connection broker, of VMWare View / Citrix XenDesktop. The DVM provisioning and hypervisor are the same as any other VDI implementation.

The last product is called MailScape that is a monitoring solution for Exchange, Active Directory, ActiveSync, BES. Purportedly better than SCOM.

Basically, this session wasn’t about SIEM as much as it was a pitch for JCS and Associates’ own hand-rolled appliance mash-up. It looks decent. It really does. It takes these products, ties them together and allows for some seemingly in-depth reporting in a single place to facilitate SIEM so that you can make sure everything has been working to your specifications. However, this wasn’t anything that was precisely groundbreaking. It’s a mash-up. A good looking mashup, but still… it wasn’t about SIEM as much as it was about an appliance that provided a ton of features and then had a SIEM component in it.

12:15 to 2:45 FOOOOD! Gabbing. Vendor surfing. I ate a hearty meal, robbed the goodie bar and then sat around and talked with some other participants. I spoke to two in particular that had different companies with symbiotic relationships between them. They were local IT, security and etc. providers. We shared stories (some really, really wild ones that shall not be reprinted) and business tips. I learned a lot from them. I then vendor surfed, dropped business cards, asked for info and again learned a ton about the local business climate.

Keynote: Chris Roberts of One World Labs gave a security / hacker themed keynote. He was witty and a good presenter. The presentation was ho-hum with plenty of the same “Oooo” and “Ahhhh” stories of “hacking” exploits that could have crippled businesses, enterprises, prisons, power plants, military installations and the like. I’m not taking anything away from Chris, however, these kinds of talks are a dime a dozen among security conferences. Then again, nothing original was promised, so that’s fine. It causes one to stop and think about their own security practices. It was a good refresher talk.

You can see a few minutes of the same talk given at an earlier conference here:

You can see him in a drama filled video complete with crying women and violins here:

Afterwards there was a raffle for tons of vendor prizes. I won a Nintendo Wii that included an extra wand, an extra “Nunchuk” and a copy of AMF Bowling. I was going to sell it on eBay for a few dollars, but decided to give it to my mom to play with. She’s been eyeing one for a while now.

All in all it was a fine vendor driven conference. Nothing groundbreaking, nothing terribly disappointing. I learned about new vendors, was re-acquainted with some old ones and got a Wii. What more can one ask for?

Want to know how to find your Windows Product ID? I won’t tell you right away. Keep reading and I’ll clear up some common misconceptions that you might not know you have.

Recently I dove into the topic of how to discern what installation media was used to install Windows. It’s possible to find that information out using the Product ID number. The search engine results for anything Windows Product ID related were disconcerting. There is a lot of confusion around what the Product ID is. Before I show you how to find the Product ID, let me tell you what it is and what it isn’t.

What the Windows Product ID is Not

The Windows Product ID is not your Product Key (also known as your License Key). The Product ID is not the code that you type in to install Windows. A Product / License Key looks like the following:

XXXXX-XXXXX-XXXXX-XXXXX-XXXXX

That’s five sets of five numbers separated by dashes. To reiterate, the Product ID is not the above number. The above number is the Product Key (aka License Key).

What the Windows Product ID Is

The Windows Product ID is a 20-character number that follows this form:

12345-123-1234567-12345

That’s a five digit number, followed by a three digit number, followed by a seven digit number and finally a five digit number. The Product ID is a number that is generated based on the Product Key (the thing that you pay money for and can install Windows with). The Product ID is then combined with a “Hardware ID” that is generated based on the types of hardware that you have in your PC. Those two things combine to form the Installation ID. When you activate Windows, the Installation ID is associated with the Product ID.

What the Windows Product ID is Good For

Apart from being an internal number that Microsoft uses to make sure your copy of Windows is genuine, it does have a few surprising uses. You can determine what installation media was used to install Windows from it. You can also figure things out like the Microsoft Product Code (MPC) for the installation which tells you the locale and even if it was an upgrade or not.

How to Find Your Windows Product ID

Finally we come to what you probably wanted to see all along. How to find the Windows Product ID. Ultimately it’s located at the following registry key:

HKLMSOFTWAREMicrosoftWindows NTCurrentVersionProductId

And you can navigate there through RegEdit or via the command line using the “reg” command like this:

reg query "HKLMsoftwaremicrosoftwindows ntcurrentversion" /v ProductId

Notice that you must use quotes since there is a space in the key’s name and you have to run the command in an elevated command prompt.  When you have your Product ID, you can then do some interesting things with it like learn what media your installation came from and if it was an upgrade or not.

Aside from that, the Product ID will probably never be something that you have to write down or keep track of.

EDIT: As commentor Brian points out, the media type for all Vista and beyond installations is the same. Media type was only different in XP and prior versions of Windows. The license key that was used to install Windows is what will now determine the channel ID. I’ve had a hard time tracking down documentation on this subject, so it’s a bit fuzzy. However, I still remain skeptical that the media files between OEM, TechNet and MSDN are completely identical in Vista and beyond. I have no proof of this though, and it remains to be tested if my suspicions are true.

Far too many times, I’ve troubleshot a Windows PC and come to find out that the image was made from media that did not match the license that I was trying to work with. Unfortunately, I know many IT Professionals that use MSDN or TechNet images in a pinch for production machines, and rationalize that “It’s the same bits, and I really do have the license for it, I just don’t have the right media at this moment.” That’s true, to an extent, but it’s still completely illegal and seems to have a technical detriment at times as well.

While Vista and beyond theoretically use the same media regardless of TechNet, OEM, Retail and etc, I still have my doubts. Nonetheless, the license key used to install Windows is still very important. Many times I have suspected that a TechNet or MSDN license was used to activate Windows in a production environment, but had no knowledge of how to discern the truth of the matter.

Was this PC installed from the MSDN image or license? Maybe an OEM disc that someone had laying around? Perhaps a Volume License image? I suspected that there was a way to tell, because in many instances certain Windows features didn’t behave like I thought they should when the image was from TechNet or MSDN. There seemed to be a way that Microsoft “just knew” that the image wasn’t from the media type that it should have been.

While I don’t know about any tell-tale signs deep in the Windows bits, I now know that there is a high level way of discerning a Windows image’s origins. Thanks to this ServerFault question “Which media was used to install Windows 7“

I saw it and decided to launch into an investigation. I had had that very question running through my mind many times, but could never get to the bottom of it. In fact, after sifting through a mountain of search engine results to try and answer the ServerFault question, I still couldn’t find an answer. I favorited the question with the hopes that someone would answer it in the coming weeks or months. Fortunately, I didn’t have to wait that long. The question’s author found the answer just a little while later.

The crux of the matter is within the Windows Product ID and how one interprets the numbers. A Windows Product ID looks like this: 12345-123-1234567-12345. Notice that the Product ID is not the Product Key, the latter being what you are essentially paying for when you buy Windows. Searching for information on how to find the Product ID comes back with plenty of misguided articles that confuse the two. Here’s Microsoft way of finding the Product ID for some of the most popular iterations of Windows.

You can also find the Windows Product ID at the following registry key: HKLMSOFTWAREMicrosoftWindows NTCurrentVersionProductId

Oddly, I found the Windows Product ID at this seemingly unrelated key: HKLMSOFTWAREMicrosoftInternet ExplorerRegistrationProductId

The major source of information for how to interpret the Product ID number is from a free tech support community (that I had not heard of before this topic came up) called LunarSoft at their Windows Product IDs page. Searching around for other sources of Windows Product ID information finds that everyone seems to be gathering their information from them, even answers on Microsoft’s own support forums will link back to their Product ID page. If anyone knows where the official Microsoft information can be found, let me know.

The key part of the Product ID that is important for discovering what image was used to install Windows is the “Channel ID” – the three digit number that is the second number in the four number PID. In my case, my Channel ID is 292, however that number isn’t on the list at LunarSoft. Apparently, while LunarSoft’s list is great, it is a bit dated. You can see this forum thread that makes mention of the outdated nature of the list.

There is still some confusion, but apparently 292 stands for Windows Ultimate Retail, which stands to reason since my installation is Windows Ultimate installed from a disc I scored for free at an official Windows 7 launch party in Pittsburgh. I think the list of Channel IDs is in need of some confirmation, but I can’t find any official documentation on the subject. However, between LunarSoft’s Windows Product ID page and the forum thread over at MyDigitalLife I think you should be mostly taken care of.

Once you have your Channel ID, compare it to either LunarSoft’s list or the MyDigitalLife forum post and you’ll have a pretty good idea of what media was used to install Windows. I’ll be on the look out for any official and up-to-date documentation on the Channel ID in the mean time.

Do you know of a better way? Have any insights on official documentation? Let me know in the comments below.

If as an IT professional you find yourself consistently using Microsoft Windows, you may be interested in the 2011 TechMentor conference happening in Las Vegas on October 10th through the 14th. There are 8 tracks and over fifty sessions to choose from. There’s even a VMware vSphere track tucked in there as well.

However, the reason I’m writing this post and rushing it out the door is because the Early Bird Special is about to expire. You can save $200 off the charge if you register before September 16th. That’s only a day away. Alas, I could have posted this sooner, but allowed other things to get in the way.

Here are the available tracks with links to the full session list for each:

• An All New Focus on Windows Desktop Administration and Application Delivery
• Build a Production Ready Windows 7 Deployment Solution
• Discover Real World Security Tactics for Servers and Wireless Networks
• Expand your Skills with VMware vSphere
• Get Certified with an Expanded 3 Day MCITP Boot Camp
• Keynotes
• Learn the Secrets of How Microsoft Does IT
• Learn Windows PowerShell in 2 Days
• Master Windows Server 2008 R2 and Exchange 2010
• Tutorials

 (Apologies to Hyperbole and a Half)

Today I’m at the VMUG user conference in the greater Phoenix Arizona area. It’s actually held on the west side of Phoenix in the city of Glendale. For more information on the conference, check out the official agenda page.

The Plan

I’ll have my choice of four different talks in five separate breakout sessions throughout the day. Here’s how I’ll fill them up:

Breakout Session 1: The first breakout session starts at 10:30 and I’m probably going to take the one titled ”vCenter Operations – What’s New, What’s Cool?”

Breakout Session 2: I’m going to take the session titled “Veeam A look under the hood: Veeam Backup & Replication”

Breakout Session 3: I’m torn between two sessions. I’m either going to the “Nexsan A Case Study in Simplifying Management & Reducing Expense by 50%” session or “Xsigo Under the Hood with Virtual I/O Technology, and How VMware Uses It to Do More” I don’t grok virtual storage (beyond simple LVM concepts) or what problems it solves so the latter would probably be more informative.

Breakout Session 4: Again, it’s a toss up between two sessions: “Teradici Corporation An inside look into the PCoIP® protocol and zero clients” and “DataCore Architecting Your Storage Infrastructure to Yield Virtualization Objectives” I’m starting to prefer the storage sessions though. I mean, really… am I going to be implementing a PCoIP system anytime soon? Likely not.

Breakout Session 5: “HA 5.0 Deep Dive” is the probable winner, but “Performance Best Practices for vSphere 5″ is a slight possibility. Who am I kidding? After destroying HA myths just recently I’m ready for a rumble.

I’m tweeting on the #AZVMUG hashtag and will likely include some pictures. I may or may not live stream on this Justin.tv channel. I’ve scheduled this post to auto publish about an hour or so before the live blogging will actually begin.

Live Blog:

Registration

Registration is between 8:00 and 8:30AM and I’m currently at -8UTC (the other half of the year I’m in -7UTC, but that’s Arizona’s fault, not mine). You should hear from me sometime between 8:00 and 9:00AM. EDIT: I totally messed up the time zone information. Arizona is always -7UTC. We don’t change for daylight savings. So right now, I’m -8DST, AKA Pacific Time.

I arrived early and was able to check out the vendor area as people scrambled to set up their booths. A fruit-n-muffins breakfast is available including some pretty darn good coffee. Sadly, the only free wifi that is available is the hotel’s lobby wireless network which I’m having some trouble with. I’ll only be able to update the blog between sessions. Pardon any spelling mistakes and choppy writing. Pretty standard live blogging disclaimers.

Thankfully it’s not a massive conference, so it looks like there’s no mascots prowling around or booth babes being exploited (the latter of which should please Matt Simmons).

It’s a very informal get-together in spite of the size of the meeting. I was intimidated by the size of the venue and the amount of vendors, but it seems very close knit and “homey” to me. They had a few minutes of “housekeeping” that made it very apparent that it’s a friendly group. My last VMWare event was a rather large one that was corporate sponsored. It was loud, lots of music, flashing lights and seemed like it was trying to hard to get people excited. It’s computer virtualization, not a Linkin Park concert.

8:40AM

I learned that VMUG is a completely indie organization. It used to fall under VMWare. For about a year now it’s been its own entity. That was all news to me since I’m not intimately close to VMWare.

There’s apparently some kind of hands on lab available here that’s new to the VMUG. It’s sponsored by EMC. I hope to get my hands on it. Apparently done at VMWorld last week to great acclaim.

There’s maybe only 150 people in the main hall. Seems pretty sparse for the size of the room that was rented out. I was expecting it to be packed. That’s nice to see – better chances for me to win raffles!! I want to have to rent a cube truck to make it back home with prizes.

8:50AM

Tony Welsh, VMWare Systems Engineer takes the stage.

He’s going to talk about the lab environment. 148,000 VMs were created at VMWorld in the lab environment over 4 days. All the datacenter equipment was offsite. 480 lab stations were on site and going to a vSwitch environment. There were three datacenters used to create the environment on the backend. Terremark in Florida, one in Europe and one right there in Las Vegas.

Talking about vSphere 5 / ESXi 5.0. Autodeploy is baked in now, so no more custom scripts. The storage engine is profile driven, so you can simply deploy a set of VMs and tell it what kind of performance you want from the storage system and it will auto deploy to your storage backend appropriately. The 2TB limit has been lifted. HA has been rewritten. Split clusters are finally true, rather than being unrecoverable if storage went offline.

View 5. PCoIP is enhanced reduced bandwidth requirements. Upwards of 75% reduction. 3D graphics driver is available.

Blah blah, missed a lot of stuff about Android and iPhone apps as I typed. *sad face* Why am I checking ServerFault on my iPhone? Oh yeah, I’m trying to solve a problem to get a 200 point bounty. Forget this VMWare stuff, I need arbitrary points that mean nothing in the real world!!

AppBlast is a product that can bring up remote applications in “any” browser. Air quotes are mine. Apparently it worked to bring up Excel in an iPad. I now have crazy dreams of my own VMWare application server in my closet so that I can get certain applications to run on my iPhone.

9:08AM

Tony leaves the stage. Michael Krutikov from Symantec in the BackupExec product team takes the stage. Wearing a very dark suit, emaculately pressed. Looks like either an FBI agent or a preacher.

Right away he positions BackupExec as the #1 backup solution hands down for VMWare. Takes a swipe at “niche solutions”. Considering that Veeam is in the house, that’s… awkward. IBM is actually #2. EMC a very close #3. CommVault, CA and HP straggle in after that. Seriously? CA? *gags* He says that half of the world’s data is backed up by Symantec.

He says that BackupExec and NetBackup are 95% identical so it really doesn’t matter which one you use most of the time. So… why have two product lines? *shrug* I’m no VM backup expert, so I hope there’s some valid difference. But still, seems silly. Ahh, later he says that NetBackup does better global deduplication.

Michael is leading the group with questions – good questions – but no one is raising their hand or really interacting. Tough crowd. “How many people restore a full VMDK when you need to restore just a few files from a backup?” No one responds. C’mon people, you know you do that more than you’re willing to admit.

NetBackup uses vStorage APIs. No agent in the virtual machine. Nice (not revolutionary by any means, of course). Only need one backup to be able to recover files from a snapshot. Not a VMDK base and then snapshots on top of it. Restores are quicker.

V-Ray is what they call their snazzy backup technology. I’m hearing the word “patented” a lot. Global deduplication across VM Guests, ESX servers, virtual, physical and NDMP sets. That seems cool.

It took 15 minutes before I heard the term “Single pane of glass!” I was expecting it sooner. Mad props.

There’s a basic deduplication lesson. Target, source, blah blah. You all know about that.

Apparently the agent for backup installed on a media server accelerates backups enough to where clients buy the software just to use the agent to then spool the data onto a deduplication appliance that itself is a backup device. Something like that, anyway. I was a little foggy on that whole discussion.

There is a tab within vCenter / vSphere for Symantec backups, so no separate console is needed to look at your backups.

Symantec has backup exec appliances now that were announced at VMWorld last week. I wasn’t aware of that. They’re a big hit. The appliances do look nice. They have agents and etc. already in it. It would be good for remote sites. I’m interested enough to look into it for the future. Symantec will be offering its own cloud backup service so you can go Disk to Disk to Cloud if you don’t want your own DR site. Clouds. Huzzah.

10AM

Closed laptop. Gotta go find Wifi to past this into the blog.

10:18AM

Charging laptop and got close enough to the lobby to be able to post this. Surprised that such a swanky hotel doesn’t have full, free wifi coverage. Someone remind me to order a new battery for my aging XPS 1530. Great laptop, probably has another 3 years in it. The battery, however, does not. Even so, Fedora 14 is good to it. If I remember to turn off my wireless adapter I might be able to eek out 2 hours from it. Quite a difference from the 4 or so hours it used to get.

Is there anyone reading this that’s at the event?

11:15 AM

Presentation over. It was just announced that the labs are available!

11:25 AM

Hotel lobby wireless is flaky. Can’t get online.

I didn’t know about the labs so I’m going to see if I can grab a spot and do that instead of the breakout session. There’s about 15 workstations that are available to sit at.

11:35 AM

The lab was packed. I decided to go to the Veeam talk it’s. All about backup and replication.

The presneter states that images aren’t enough. Applications within the VMs are often not consistent when restores are performed. The next way some places backup virtual machines is perform a virtual machine image (backup the VHD) and then have an agent on the inside.

A feature called instant restore: They wrote their own NFS service that “rehydrates” the backed up VM on the fly so you can have a downed VM back into production in seconds as it starts to be restored. The NFS Veeam datastore is a VMWare datastore within ESXi. The NFS service is just like a proxy. It’s going through the NFS service to the ESX host. You can use whatever storage you want. They don’t use a NFS datastore. They use any volume, and then publish it with the NFS service. Restores like this, of course, don’t edit the original backup file. The changes to the VM are captured and the backup is never touched.

I just heard the word “agnostic.” “Rehydrate” and “agnostic” within a few minutes of eachother. Oh yeah, this is a sales demonstration. The sad thing? I use those words too sometimes. D=

But what if your live servers are on a SAN, and the backup storage is a slower NAS? How do you move the VM to the faster storage? It relies on storage vMotion to move from your NFS server to your SAN if you want. You could do a cold migration as well if you’re not licensed for vMotion.

Next feature. “SureBackup” After backups are done, backups are loaded using InstantRestore and it then tests things like pinging network adapters, makes sure that everything is running, can run custom scripts to make sure that applications inside the VM are running. You could run scripts to make sure that Exchange is running right or SQL Server or whatever. There is a virtual lab which is a vSwitch with no network adapters that the VMs are loaded into and tested. That feature sounds awesome.

Next feature: On demand sandbox. Allows you to power them up for QA / patch testing / etc. It’s a lot like SureBackup, but used in a less automated way. There are dependency groups, so if you want to test Exchange, it needs a DC of course, so if you make a dependency group including the DC, you’ll automatically have the DC when you spin up Exchange to test on. Application aware restorations means that the DC will restore in AD restore mode so that you’re not replicating bad data. Application restores work in SQL Server, Exchange and even Oracle. Look into U-AIR for more info on that.

Nice thing about Veeam is it’s agentless so they don’t charge per agent. They don’t care how much data your backing up. It’s just feature based pricing. Standard and Enterprise.

Application backups are done without agents, which is interesting. Not sure how that’s done. There was a demo about a change being made in AD on a production DC, and then the Veeam backup server was looked at for the DC backup. The single LDAP property that was changed was restored, all without rebooting the server or causing disruptions to the DC.

The latest version now separates the roles out, so you have backup servers (basically the schedulers), proxy servers for dedupe CPU and repository servers as storage points. Apparently before this latest version was released each backup servers had each of the three roles on it.

1:00PM

Just finished eating. I’ve covered half of the vendor floor. I’ve actually found some interest products I’ve never heard of before. VirtualWisdom’s SAN monitoring solutions and Quest Software’s VM management products.

I’m ignoring the Lunch Keynote by Arista networks which goes from 12:15 to 1:30. I think I’ll go to the Xsigo breakout at 1:45, but take the opportunity to tinker in the VMWare / EMC lab instead of a breakout session at slot #4. The fourth session was the least pertinent to me anyway.

With how many times my badge has been scanned, I’m expecting a flood of sales calls and emails. I just realized, I didn’t use my obfuscated email address. I have an email address that I use for vendors so my main company account doesn’t get hammered. I fail.

Time to go troll the vendors. Once more into the breach!

1:45 PM

After a rejeuvinating lunch (fully catered Mexican style food that was amazing), I walked into a talk by virtual storage company Xsigo. Pronounced “SEA-go”.

Xsigo provides a large appliance known as an I/O director. It gets all adapters out of servers, puts them in one large box and aggregates bandwidth into 1.5TB total. You carve up your datacenter into as many vNICs  or vHBAs.

You have blade servers consolidating your servers. VMWare consolidates your OSs. Your I/O infrastructure is consolidated by Xsigo. They use InfiniBand, a 40Gb connection between each servers and the virtual I/O. InfiniBand cards in your servers connect to the Xsigo director. The director is physically divided into two parts. The top half has what looks like 24 Infiniband 40Gb ports. The lower half has line cards for 1Gb / 10Gb Ethernet and 4Gb / 8Gb Fibre Channel. You then provision virtual NICs for FC or Ethernet to your servers.

The savings on fabric for servers is substantial. Instead of tons of fabric in each blade housing, you have a few Infiniband fabric cards and then use the Xsigo to provision bandwidth.

Using Xsigo’s management console, you can manage your physical servers and you can set your peak and comitted rates for each vNIC so you manage your congestion tolerance. You can place servers in groups and provision network interfaces for them all at once.

There is a tab that can be installed in vCenter so you can have a view into your Xsigo network from within vCenter.

I’m not 100% sure how much visibility there is to manage the bandwidth to see what and where your bandwidth is going. The management console looks okay, but I wasn’t clear on how I could see multiple Xsigo boxes’ performance metrics. There is a performance monitor built into the product to view quite a few different metrics. The largest deployment they have is 4 sold to Disney Interactive. Apparently it saved Disney about 2 million dollars in HBA and cabling costs.

He mentions that they’re working on getting more into better insight into what is the IO paths so you can know better how to provision your network cards.

2:45 PM

I was planning on doing a VMWare lab, but whem I went by the lab room, most of the workstations were broken down and only four or five were left. I figured that it would end up being a 1:1 sales pitch so I decided to go to a breakout session instead. I took a surprise course. I usually avoid purely cloud themed sessions, however the private cloud session put on by EMC / VMWare intrigued me. I’m somewhat interested in an “internal cloud” for some ideas I have. I just wanted to see what this was all about.

Get ready for a definition of the term cloud! “It’s your datacenter, virtualized. Not just your servers, but the whole datacenter. Networks. Storage. CPU. Everything.” That’s actually a fair definition. The idea is to pool resources, CPU, Network, Storage, etc. We all know this, but sometimes it’s hard to pull out of the older view of thinking about virtualization as purely “I make one physical server hold many virtual servers.”

An interesting idea is that the concept of “chargeback” not ever being used within a company, however you can use those numbers to becoming a “costback” to show how much a project is costing the IT infrastructure but also how IT might be saving the company. I thought that was interesting since very few companies actually use chargeback internally. I always wondered if those metrics could be used in a different way though.

The emphasis is that it’s not just about virtual servers, it’s a virtual datacenter. The first quarter of the talk was pretty much as nebulous as one could expect.

Next quarter was some interesting high level views into how EMC and VMWare integrate and how each uses the others APIs so that you can see which VM is using which LUN, etc. I was impressed with that part. It doesn’t seem to just be marketing spiel. It seems legit that EMC and VMWare really interplay with eachother in good ways

It was all over my head though. I’m not a storage admin nor do I use a lot of VMWare in the context of this breakout session. I was impressed with the sub-LUN tiering of storage. Data within a LUN can be spread across multiple tiers of storage based on what parts are used more. That seemed pretty smart.

Mention was made of vBlocks. I’ve liked the idea of vBlocks for a little while now, but like it even more now. I hope to get to use it someday.

3:45 PM

HA 5.0 has been rewritten and is changed much from 4.1. There is no longer a primary and secondary server. There’s no dependency on DNS (not sure how that was elaborated on). While HA supports IPv6, some other products do not, like the VC appliance. You can use IPv6 for HA but you won’t be able to manage it through the VC appliance.

Multiple hosts can act as a failover host. The limit to the number of hosts failures has been raised to 31.

The HA module that is installed on the host (the agent) is referred to as the Fault Domain Manager (FDM). There is now a master or slave. There is no longer a primary or secondary. That helps in blade environments where you used to have to keep very close track of primary hosts.

The master is the central point of communication. It monitors hosts and virtual machines and reports all of that up to vCenter. The slaves monitor the VMs running on them alone. Slaves forwards any type of state change, like a power on of a VM or a crash, to the master. It will perform actions that the master dictates, for example restarting a particular VM. It also monitors application within the VM if that is turned on.

HA also has application level components. You can protect applications within the virtual machines such as Exchange. I’d love to write more, however my battery is drained down to the last few minutes. I’ll have to finish this up when I get out of the session and near a power outlet.

Later that night:

I sat through the HA breakout session and it was by far the most technical of the talks. So much so that I zoned out through most of it since I am not familiar with VMWare HA. I needed a base of understanding that I did not have. All in all, it looks like it has had quite an overhaul from previous versions with a lot of special case intelligence to make sure you don’t end up with

Most of the talk was on the logic behind failover and master election and how bad network design won’t necessarily cause problems in fringe-case failures. Application aware HA wasn’t gone into as deep as I would have liked since we went over time.

It was a good day! The fact that it is a user group and not a corporate event made it very friendly. A ton of iPads were raffled off (none of which I won) as well as a few other goodies.

So how was it? Was the live blog actually useful for you or a little blah? Are there any conferences in the southwest coming up that you know of? Let me know in the comments below.

Back in August I blogged about the ServerFault Scalability Conference that was happening this October 2011. Sad to say, the conference has been called off. Check out Joel Spolsky’s post about the cancellation. In short, both DevDays and the Scalaibility Conference were called off due to lack of sign-ups.

DevDays ’09 was a $99 one-day event in ten cities. Affordable, close-by and easy to get time off for. This latest iteration was a $499 two day event in only four cities. All the things you loved, with none of the things you loved. Okay, perhaps that’s overstating things a bit, but it was a bit harder to get time off to go to something like that.

Here’s Joel’s quick FAQ, but make sure to read the full blog post for details:

Q: I registered anyway. Will I get a refund?

A: Yes, this will happen automatically. If you have any trouble or questions email Alex & Alison at [email protected] for help.

Q: What about the ServerFault Scalability Conference?

A: That has been canceled, also.

Q: What about the hackathon in Washington, DC?
A: We’ll let you know. We are still planning to hold the Stack Exchange company meeting in Washington, so we will try to organize some public event at the same time.

Q: Why don’t you just scale back to $99, one-day conferences?

A: Unfortunately, the four conferences we planned this year were going to be held at much larger venues and would have cost way too much to put on, so we can’t just trim them back to one day, $99 events.

Q: What are you going to do in the future?

A: We want to work on a much larger number of much smaller events in far more cities, such as meet-ups and individual talks sponsored by Stack Overflow.

According to a recent Infosec Island article titled “The Urban Legend of Multipass Hard Disk Overwrite” something that most of us have taken for fact is in fact, not fact.

As IT professionals, we’re often told that data that was previously stored on magnetic media can be extracted even if it’s been overwritten. Methods like magnetic force microscopy and scanning tunneling microscopy may be cited. However, modern hard drives no longer use the same storage methods or hardware that allowed those techniques to be successful.

Noticeably absent in many of the most modern government standards documents for information security is the requirement that drives have multi-pass formats performed on them. Most standards call for degaussing or outright pulverizing. The one standard that does mention formatting is NIST Special Publication 800-88 which says on page 27:

Writing patterns of data on top of the data stored on a magnetic medium. NSA has researched that one overwrite is good enough to sanitize most drives. See comments on clear/purge convergence.

Infosec Island reports that in a recent paper titled “Overwriting Hard Drive Data: The Great Wiping Controversy” single pass wiping is considered good enough to prevent data retrieval:

…a single overwrite using an arbitrary data value will render the original data irretrievable even if MFM and STM techniques are employed.

Of course, the question remains in my mind: Are there newer and more advanced techniques that are available to recover erased data? Apparently, even if they do exist, they are currently more advanced and expensive than many people are willing to worry about.

Ultimately, if you’re concerned about data security, your best bet will always be to physically grind the media to powder. Drilling and hammering isn’t enough. However, if you don’t have that option and you’re not hiding some of the world’s most valuable secrets, then a single pass wipe on modern drives is apparently good enough. I suppose that means that I’ll no longer need to let DBAN sit and wipe a retired mail server’s drives over a weekend.

What do you do for data security? Do you have a degausser? Do you contract with a physical destruction company? Or were you formerly only satisfied with a week-long DBAN session?