My first major "crisis" happened today on the web server. A customer notified me that their web form was giving an error. I looked into it, and soon discovered that form mail scripts across my entire server were failing with nothing more than a "premature end of script headers" logged in the error.log file.
My first test was to create a simple "hello world" perl file and try and call that from the cgi-bin directory. It didn't work.
My next test was to move that simple test.pl file to the webroot (httpdocs), and it worked. Now I isolated the problem. Turns out that suexec.log was throwing an error every time something was called from cgi-bin. It logged "commad not found in docroot".
Now, to retrace my steps and see what caused the problem. A night or two before I had upgraded Apache via Redhat's up2date utility. It was a security related fix, so I'm usually somewhat quick to apply those. My webserver also runs Plesk 7, and this is where the problem came in. Plesk runs a non-standard document/virtual host directory structure, and in order to allow suexec to work properly, there's a Plesk-modified suexec binary in the /usr/sbin directory. When I ran the httpd update, it apparently overwrote this custom binary with the standard Apache binary.
A simple re-copy of the binary fixed my problem. All is well in server-land once again.
Posted under the influence of [[John Lydon :: A No And A Yes]]