thinking sysadmin

On the ESX console, do the following:

• Read the documentation for each patch.
• Group patches that can be installed together into a directory, possibly an NFS mount available on all your ESX hosts.
• Cd into the patch directory and untar the patches:

  for i in `ls *.tgz`; do
tar -xvzf $i
done

• Install the patches:

  for i in `ls`; do
if [ -d $i ]; then
cd $i
esxupdate --noreboot update
cd ..
fi
done

• Reboot.

• Amazon S3 Availability Event: July 20, 2008 – Amazon’s post-mortem on the 7/20 S3 outage. Excerpt: “We’ve now determined that message corruption was the cause of the server-to-server communication problems. More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted such that the message was still intelligible, but the system state information was incorrect.” (Seen first at Ars Technica.)
• VMware has released Update 2 for VMware Infrastructure 3 version 3.5 (I think that’s the Full Official Name That Only A Committee Could Love…). Scott Lowe has a good summary; release notes are here. Most notable among the updates is the ability to use VSS to quiesce Windows VMs prior to snapshotting.

So you thought you fixed the DNS spoofing vulnerability that was all over the news this month? You applied the patches and moved on to the other fifty-seven things crowded on your to-do list, thinking that you were safe? If your resolvers are behind a NAT, you might want to think again, smart guy.
Read the rest of this entry »

• File Level Recovery from within a VMDK backup – Nick Triantos of NetApp covers file-level recovery from VMware (Windows) VMDK files again, in more depth than he has before. Nick notes that it can be a “a point-and-click process.”

• VMware to release ESX 3i for free next week – Yeah, the title pretty much says it all; I guess that’s one way to compete with the price of Hyper-V. Now what were those 3i vs 3.5 limitations?

• Flash Forward – Jay Kidd, CTO of NetApp blogs that “NetApp is in the process of certifying enterprise-grade SSDs that you can use in our existing storage shelves.” No dates or pricing announced yet, of course, but he does make an excellent point about SSDs in storage arrays: “For the next few years, you won’t be using a lot of flash capacity in your systems, not just because of the costs. At 10x or more the IOP rate of hard disks, it only takes a small number of SSDs in disk slots to saturate the performance of the array controller. It’s like trying to fly a model airplane in your living room – you’ll run into a system performance wall long before you hit capacity limits. This is another reason that flash as cache is economically efficient – it puts the necessarily small amount of very fast storage at a point in the architecture where you can best exploit the performance.” Not unlike how Sun suggests using SSDs with ZFS. (Seen at Blocks and Files.)

• Storage virtualization doesn’t exist – This needed to be said: “Virtualization.info doesn’t cover the so called storage virtualization because at today this term doesn’t mean anything. Unlike what happens for hardware virtualization, OS virtualization and application virtualization, the storage vendors seems unable to find an agreement on the definition. The term is abused in almost every press announcement and it can refer to at least ten different approaches.”

• Elektronkind: OpenSolaris 2008.11 – A Preview For The Storage Admin – A look at upcoming storage technologies in OpenSolaris 2008.11, including ZFS, iSCSI, NDMP, COMSTAR, AVS and SAM-QFS. These products really set OpenSolaris apart from Linux distributions, although I wonder how official this list is, and have some doubts about the status of some of the projects. For example, there doesn’t appear to be much activity on the SAM-QFS OpenSolaris project, although maybe I’m just looking in the wrong place. (Seen at c0t0d0s0.org.)
• Ruling: SCO owes Novell $2.54 million from SCO-Sun SVRX deal – Interesting excerpt: “Judge Kimball also reviewed SCO’s agreement with Sun and found that some of the terms exceeded SCO’s licensing authority. Through the agreement, SCO lifted the confidentiality provisions of Sun’s 1994 SVRX deal with Novell even though SCO was not permitted to do so without Novell’s explicit consent. The judge concluded that lifting of the SVRX confidentiality provisions was not incidental to a UnixWare license and was consequently not permissible. This raises some intriguing legal questions about OpenSolaris, which includes SVRX code that we now know SCO clearly had no right to let Sun open.” I wonder if we’ll be hearing more about this in the coming months.
• Interview: IT consumerization and the future of higher ed – Another interesting piece on Ars Technica from today, an interview with Oren Sreebny of the University of Washington, whose best bits obliquely refer to the challenges of miasma computing and information security. Quotes: “Lately we’ve been looking at Google and Microsoft offerings for commodity stuff, and one of the things we deal with in some of our research [departments] is government regulations about ‘exporting munitions.’ So one of the manifestations of those government regulations is that you cannot store your data outside the US if you’re working on some types of government-funded projects. Google has said, ‘We can’t guarantee that anybody’s stuff in particular won’t be in a datacenter that’s located outside the US, so don’t bring that stuff to us,’ which is exactly what I’d be saying if I was them. So we have to figure out, as we start to move in those directions, what we do about that.” Also: “[Separate identity principals for people who are working on sensitive data] is an interesting conversation because, in many ways we’ve spent the last decade trying to integrate people’s identity, and do single-sign-on, and not make them have lots of separate accounts in separate places. And in many ways it really goes against the grain to step back from that, but maybe it’s time to do that.”

• VM HA – service console networking, isolation behavior – and other “under the covers stuff” – An overview of how VMware ESX’s High Availability works under the hood – making it much more apparent to me how important file locking is to HA’s functioning. (I’d love to see an overview of how file locking does – or doesn’t – differ on VMFS versus NFS datastores.)
• Why Upgrade? – DanT on what’s new in Sun Grid Engine 6.0 through 6.2.
• Part II: Since NFSv4 is Stateful It Must Be Less Robust, Right? – “Just because CIFS is old and busted, that doesn’t mean NFSv4 is.” Just kidding, that’s not an actual quote. But I think it’s a reasonable summary of the piece.
• HP – Performance-Optimized Data Center – Yet another vendor produces a data center-in-a-box product – which isn’t to say that there isn’t good technology inside of HP’s product. I wonder if container data centers will come down-market to the point where they become a reasonable alternative for new office building construction instead of building a conventional server room. (Seen at Data Center Knowledge.)
• Understanding NIC Utilization in VMware ESX – Scott Lowe comes through again with another practical piece on networking and VMware ESX.

How to find the version of BIND that you’re running:

> dig @localhost version.bind txt chaos

; <<>> DiG 9.3.2 <<>> @localhost version.bind txt chaos
; (2 servers found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7775
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;version.bind. CH TXT

;; ANSWER SECTION:
version.bind. 0 CH TXT "9.3.5-P1"

;; AUTHORITY SECTION:
version.bind. 0 CH NS version.bind.

;; Query time: 57 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Jul 14 11:45:14 2008
;; MSG SIZE rcvd: 65