Archive for 'Uncategorized'

In a recent post named The Making of a Meta Server or “Why I Bought a Mac Mini as a NMS” I explained why I had chosen a brand new, 2012 Mac Mini as my NMS hardware. After two weeks of mind numbing work, I have officially declared the Mac-Mini-as-a-NMS project a failure.

The main problem surrounded Apple’s custom EFI. Apple hardware does not use a BIOS, but instead uses EFI (note: not, specifically speaking, UEFI). Or rather, it uses an ancient, bastardized version of EFI 1.1. There is a BIOS compatibility layer that allows OSs that can only communicate with a BIOS to operate on the hardware. Most notably Windows. Apple’s OS also runs on a hard disk that has been partitioned using the GPT partitioning scheme, which isn’t itself a huge deal, but you might be surprised at the anemic support for GPT boot disks in even modern operating systems.

To use the Mac Mini to boot an OS that needs BIOS compatibility and a MBR disk should be relatively easy. Right? Right!

Unless Apple is involved.

There are several things that Apple has mutated away from the EFI standard, one of them being not using the EFI system partition for anything except firmware updates. Their custom EFI implementation has the boot process (as well as some extra filesystem drivers) baked in. The whole EFI experience just never worked like I expected it to. The other trouble is that Boot Camp has been changed in OS X Lion. If you wanted to be hand held through the partitioning process and the creation of a hybrid GPT/MBR disk, you’re invited to use Boot Camp. However the latest alterations only allow media with Windows images to be accepted. You can no longer (from my ability to understand) use Boot Camp to install non-Windows OSs. Of course, it was always unsupported, but at least it was doable.

During the whole process, I used the EFI boot manager rEFIt which apparently only recently works with OS X Lion. I read more about the GPT partitioning scheme than I ever have previously. I learned more about EFI than I ever wanted to know (although all of that information will come in very handy in the near future). I hand-rolled bootable USB thumbdrives. I tweaked partition tables. I did very nearly everything I could think of except rolling my own EFI boot partition. After the hours had steadily ticked away I decided it was no longer worth it.

After countless errors concerning boot media, partition problems, and blinking cursors, I concede that the latest Mac Mini has defeated me. It has been shipped back to Amazon and I can go back to my Apple-less existence. Speaking of Amazon, I believe that they deserve some praise in this.

Amazon made the returns process easier than any return I have ever made. Anywhere. I stated that the reason I returned it was because software I had intended to use with it was not compatible. As a result of the return not being their fault, I had to pay return shipping. Within just a few clicks, Amazon created a return label. I printed it out, boxed the mini up, taped the label to the box and handed it over to the man behind the UPS Store counter. Within 15 seconds I was walking out of the store. I have the fortune of living just a few hundred miles from an Amazon return center located in the Las Vegas area so the return was processed and money credited back within two days. Thank you, Amazon. You were the only bright spot in this debacle.

I am now investigating other pieces of hardware for this project based on the recommendations of several colleagues. If you have a recommendation, share it with me and the rest of my readers in the comments below. I’ll certainly write about my second attempt at this project as it happens.

In the end, I’m not mad. The Apple wasn’t designed to do what I was asking it to do. It was my fault. My only lingering frustration is that the Mac seems to take any standard technology that it uses and twists it in new and different ways so that your familiairty with a standard becomes more of a liability than an asset. Sound like another familiar company that SysAdmins like to pick on? Then again, Apple isn’t intended to be in the business market. Let us pause and mourn the passing of the Xserve (I handed my G5 Xserve over to Best Buy for free recycling last year. So, so sad…).

Any similar experiences with an Apple product? Have you managed to wedge an alternate OS on 2012 Apple hardware? Let me know in the comments below.

I recently had a moment involving a CentOS server that caused me to circle the wagons and ask “Who just did what in their shell?!”

After quickly checking to see who was currently logged in (as well as those that had just recently been logged in), I wanted to see the command history for each user on the server.

Before I go any further, let me say a few important things:

There are more shells than bash

Each shell has its own history options and files. Don’t assume that because you found all the .bash_history files on a machine that you have all shell histories.

And all the zsh proselytes said “Amen.”

.bash_history is a suggestion not a rule

Bash’s history file (that’s the $HISTFILE variable) can be changed. Just because you found all the .bash_history files on a machine doesn’t mean you have all of bash’s history.

Bash history is a convenience not a reporting tool

Bash history can easily be altered for both good and bad purposes. It is not to be relied on as a a way of seriously auditing what has been done on a server. For that kind of thing, look at auditd.

Scan all .bash_history files

The above notwithstanding, if you want to quickly scan your machine’s .bash_history files consider the following options.

The first is dead simple, and I thank @etrever, @evilchili and Gilles over at unix.StackExchange.com for this method (I’m still getting my *nix chops).

grep -e "stuff goes here" /home/*/.bash_history

Yep, simple as that. This is of course assuming that 1) All user folders are standardized, and 2) your history files all share a common name. If the previous two things are true, this is a great, quick way to see things like… oh… say… “Who just went all chmod -R 777 on the httpdocs folder?!“

However, if you want a slightly more robust way of searching through all bash history on a machine that takes the home folder ambiguity out of the equation, Gilles from the Unix & Linux Stack Exchange had an awesome solution.

getent passwd |
cut -d : -f 6 |
sed 's:$:/.bash_history:' |
xargs -d '\n' grep -H -e "$pattern"

I had never seen the ‘getent’ tool before which gets entries from the following administrative databases: ahosts, ahostsv4, ahostsv6, aliases, ethers, group, gshadow, hosts, netgroup, networks, passwd, protocols, rpc, services, and shadow. ‘Cut’ segments the input by a colon and then selects the sixth field which is each user’s home directory. Sed works its magic to take the input and append it with the probable location of the .bash_history file. Finally grep is fed each path and searches for our pattern.

Certainly, if there is a question about the existence of other shells or if you want to be certain that your history file really is called .bash_history, you’ll need to add some extra logic in. However, for my scenario, this was enough to get me going.

Unfortunately, I was made painfully aware of how bash history is a mere user level convenience and not an auditing tool. Nothing malicious was done to the server and nothing terribly bad was done, however as I looked deeper into what could have happened, I realized that a much more thorough auditing trail might be needed in the future.

How do you handle shell history? Do you implement any special tricks to make it more reliable or do you use an entirely different system to keep track of commands that have been run?

Back in January of 2011 I announced a ServerFault 10K challenge. The premise was to gain ten thousand points on ServerFault (or, alternatively, the StackExchange site of your choosing). Many joined, only a select few achieved the goal.

Prizes were awarded to those who achieved the 10,000 point goal as well as for those who reached the halfway point of 5,000 points. Ten-thousand point earners won a Buds Luxury Duck and five-thousand point earners won a Buds Mini Duck.

Some of the winners were able to submit a photo of their duckies.

Moirai must be giggling because Philip “Chopper3″ Buckley-Mellor already had quite a collection of Buds ducks. This contest allowed him to add to it as well as bask in his Britishness with a fancy Royal Guard duck.

Rob Moir received a pretty polka dotted duck. It matches perfectly with his pretty, pretty pony avatar.

I’m not really sure what’s going on here, but I’m a little weirded out:

Tom O’Connor is a rock star SysAdmin who is not afraid to be a bit edgy, and thus he won a rock star duck:

Here’s to you, ServerFault winners! May the upvotes continue to pour onto your worthy contributions.

My Problem

I have sometimes had to copy the permissions on one directory over to another directory. Sometimes it’s simply due to the migration of files from one server to another. Other times it’s for the purpose of backing up ACL entries before an ACE edit. When frobbing around with permissions, it’s often nice to make a dummy folder or file and copy the pre-frobbed permissions over. That way any post-frobbing disasters can be rectified rather quickly.

At first, I thought copying permissions would be a simple matter of using icacls to perform some kind of permission dump. Sadly, and somewhat surprisingly, I was not able to find an easy way to do that. Certainly you can simply pipe the output of icacls to a text file, however I could not find an easy way to consume that text-based permission record. That’s where the wonder of RoboCopy comes in.

My Solution

Reading Microsoft KB323275 reveals yet another interesting use of RoboCopy.

robocopy [source] [destination] /secfix [include appropriate exclusion filters here]

Yes, once again RoboCopy comes to the rescue for things other than copying files and folders. If you’ve been an admin on Windows boxxen for terribly long, I hope you’re at least moderately familiar with RoboCopy.

In this usage, have a peek at the /secfix switch. The official TechNet help for the switch simply says:

Fixes file security on all files, even skipped ones.

However there is a larger note at the bottom of the help document that states:

When using the /SECFIX copy option, specify the type of security information you want to copy by also using one of these additional copy options:

• /COPYALL
• /COPY:O
• /COPY:S
• /COPY:U
• /SEC

If you’re simply performing a permissions copy, make sure that you use the proper file selection options (/XO, /XN, /XF, etc.) to get only the files and/or folders you want. No use in copying an entire directory structure if you only need the permissions on one folder to be copied. Also, if you’re only interested in copying permissions and not the files themselves, keep in mind the /CREATE switch:

/CREATE - Creates a directory tree and zero-length files only.

Perhaps I should start collecting various uses for RoboCopy and compile them into a series of “Stupid RoboCopy Tricks” posts. =)

Happy SysAdmin Day! This week’s blog posts have been full of SysAdmin Day contests, so I’ll round out the week with one final contest. Except, this is slightly different. First, it’s not a SysAdmin Day contest. Second, it has a huge payout. Ten thousand dollars, to be exact.

MegaPath, a privately owned data services provider, is hosting a “Tech Talk Video Contest” that has a $30,000 USD prize purse. The contest is a video based one, with some other opportunities to win a few hundred dollars in prizes along the way.

From the contest website:

For Filmmakers—Create a video, up to 60 seconds long, that somehow features techies using obscure high-tech jargon to discuss the finer points of communications technology.

Your video could be about techies arguing different points, trying to convince a boss about an obscure point, or it could just be techies waxing poetic about their high-tech muses.

There is a lot of room for humor in this one, but the tech talk needs to come through loud and clear. It may help to use graphics or text to spell out more complex terms. Check the jargon dictionary to perfect your usage.

The contest started on June 10, 2011 and the last day to upload a video is August 12, 2011. The winners will be announced on August 26, 2011. The grand prize for the winning video is $10,000USD with stair-stepped cash prizes for second, third and all the way down to 12th place.

Want to see what kind of videos are being submitted? Check out the competition at the videos page. Like, for example, this really, really campy (really campy) music video geek-rap called “I’m IT.”

If you don’t have any videographer skills (or can’t find someone to humiliate in front of a camera), you can still win some dollars. Check out the “Awards Page” to see the different prizes that are available. For example, you can be entered to win $500 for simply sharing the links (the sharing contest closes on August 26, 2011).

Admittedly, the myriad ways one can win a prize is bewildering and I don’t quite get it all. Check out the “Competition Details” page for full details and maybe you can find other ways to enter for some prizes.

Take a look at the contest and see if you’ve got what it takes to enter. If you make a video, make sure to let me know and I’ll link to it. Happy SysAdmin Day! Now go brush off your Flip camera and start shooting some video!

Earlier this January I was taking the Christmas lights off of the front cactus (I heart Phoenix) when I came across an interesting newspaper article. Before you ask, my household rolls individual strands of Christmas lights up in newspaper and then stuffs the bundles into boxes. That keeps the strands from getting tangled in eachother. Moving on.

An article from the Asbury Park Press dated Dec 12, 1999 caught my eye. It was so novel that I scanned it for long term keeping and decided to share it here. The article is a 30,000 foot view of what websites have to be put through to make sure they don’t go offline under heavy load. In fact, it’s more than just a 30,000 foot view. It’s more like looking at earth from Mars using binoculars.

The very concept of eCommerce web sites was so new that the article doesn’t even shorten the term, instead using the uncompressed form “Electronic Commerce”. It’s interesting to note that testing web sites for load problems was a novel concept worthy of a newspaper article just a few years ago.

I will not reproduce the text of the article, but will link to an image of the scanned article. Pardon the wrinkles. It’s been wrapped and re-wrapped around Christmas lights for a decade. Let me know what parts of the article tickle you in particular.

I assume almost everyone is aware of what the United States’ is doing to airline passengers in the name of (faux) security. If anyone dares to fly through an American airport, I hope you’re wearing underwear. But not just any underwear. Fourth Amendment underwear.

These are undergarments that have the US Fourth Ammendment printed on them with metallic ink. They show up quite nicely on X-rays.

Ladies, you’re not left out:

Let all of your friends know about these patriotic underthings. And no, the solution to this current political nonsense is not in the wrongheaded retort “If you don’t like it, don’t fly.”

And now back to our regularly scheduled broadcasting.

I made an error in judgmental when rushing to publish last Monday’s “Freebie Monday” giveaway. Giving people barely two days to respond was not terribly gracious. The entries have been extended to 11:59PM (-5PST) Sunday the 12th of September 2010 (yes, I like to delimit things very, very specifically).

All other rules remain the same. Only entries that include a picture along with a reason for not rebooting the server or appliance will be accepted.

Remember, a free Transcender practice exam of your choice is up for grabs!

According to last week’s polls, I am an arrogant jerk! This news is probably least shocking to family and friends.

As a result of the polls’ findings, I will be removing all social bookmarking links in my RSS feed. I will also be seriously pairing down my links at the bottom of each post.

The poll concerning how many social media links to inlcude at the bottom of each post was a close call with the “1 – 5 links” category winning by only one vote. As a result I’ll only be including Reddit (so far my best traffic source), Digg (I can dream can’t I?!), Delicious and Stumble Upon. That’s four out of five. I’ll save one spot in case I missed someone’s favorite. I may change them around in the future as well.

I wasn’t terribly happy with my original theme. After hunting around for a few days, I think I found one that I’ll stick with for a while. I was hoping to find a good three column theme, but the text column was never wide enough for me.

Even this theme’s text column is a bit too skinny for me (and the text is too small). The theme is the free Typebased by WooThemes.

What do you think? Is is pleasant enough or is there an element that is too bothersome for you? Do you have any theme suggestions? I’m really looking for a wide, three column personal blog theme that has some space for ads (gotta pay the hosting fees) but gives a good wide column to the content.

It doesn’t have to be free, it just has to be good. Methinks I’ll peruse RocketThemes.com…