thinking sysadmin

As a sysadmin, I’ve had the opportunity to interview candidates for software developer positions. While I have done some software development, the coding abilities of these candidates generally surpass my own – or at least that’s what their resumes claim – so it’s somewhat difficult for me to accurately assess their skills. A solution I’ve found is to ask the candidates questions relating to application security; this does a remarkably good job of separating the wheat from the chaff – and a similar approach is good for evaluating software products.

For a recent web developer position, I asked each applicant about cross-site scripting and SQL injection. The best candidates knew what each was and recognized the similarities between the two, the poor candidates said “my off-the-shelf framework takes care of keeping my app secure, so I don’t think about security,” and the worst said, “Huh?”

Along the same vein, in evaluating software, I’ve found it useful – particularly for web applications – to look at the requirements for file system permissions. Two big red flags for me are if the app needs wide-open (chmod 777) permissions, or requires that all its files be owned by the user the web server is running as. To me, these say that one or more of the following is true:

• The developer is lazy. They slapped their application together, and nothing says slapdash like chmod -R 777 when you don’t care to get the permissions right. Don’t kid yourself: lazy developers also don’t care if their users get compromised.
• The developer failed to plan. It’s reasonable to have some files owned or writable by the user the web server is running as in some situations (although often you just need to have the web server user be the only unprivileged user that can read a given file). However, it seems many developers don’t give much thought to isolating where the application needs to write to on the file system, and just sprinkle these writable files throughout their installation – or, worse, they put blanket writable permissions on everything, and can’t identify to the administrator what the real permission requirements are.
• The developer doesn’t understand the operating system they’re running on. I think this is one of the reasons why so many Java web apps tell you to make the UNIX user running Tomcat own everything – or why so many Tomcat instances run as root. Many of these applications were developed on Windows and then brought over to UNIX, and the developer couldn’t figure out how to make them work without resorting to this. (Note to developers: man chmod, man chown and man setfacl are your friends.)
• The environment the application is running in is flawed. This is unfortunately common when running under some shared hosting setups. My advice? Find a new webhost – you shouldn’t have to contend with exposing your application and data in a likely hostile environment.

Now consider – if one or more of the above is true, what overall level of quality do you expect from the application or installation?

One final bit of my advice, and then it’s </soapbox> for me. If you’re working with an in-house web developer that doesn’t understand the above, try getting them to grasp the following: Ask him who should be able to edit their application – he’ll probably respond that only he, or his team, should be able to. Then explain that the file system permissions should reflect this reality – hopefully he’ll get it then.