Computing Infrastructure

My system administration work for the last year has focused on doing what I can to eliminate as many of the non-humanities concerns as possible from the front end of a digital humanities project. We can worry about getting a large server when the project requires more CPU or memory or disk than we can offer. If we need to have a dedicated server for a project, then there are enough people who find that project useful that they generate that much traffic. At that point, the project should be important enough to garner grants or other financial support.

We have several goals in mind when engineering the support systems. We want to provide secure systems without the security getting in the way of what faculty need to do. We want to be able to rapidly build out a service without having to buy new hardware. We want to continue offering services if we lose a system. We want to do all of this with free, open source software.

The result is that we have two identical systems that serve as backup for each other. We use puppet to manage overall configuration so we can easily set up a new system, add users, or modify services. Services are contained within FreeBSD jails so that they don't interfere with each other, though memory and CPU are not limited and could be exhausted by an errant program.

We are working on high availability / failover capabilities with wackamole. This will allow us to balance services across the machines in the pool (currently two) as well as move services when a system fails.

We currently have a number of services configured:

• this website (http://dh.tamu.edu),
• UNIX shell services with a wide range of languages, web frameworks, and data/text processing tools,
• Simple project hosting (no server side scripting yet),
• Remote file access to home directories and hosted projects,
• PostgreSQL and MySQL database services,
• subversion and trac

I'm sure I'm forgetting something. I'll use future posts to cover each aspect of the configuration in depth. If there's something you want to know sooner rather than later, let me know. If you know of an open source package that I might have missed, post a link and comment on your experience with it.