Nov 10, 2018

How to fix: Error with cgi-bin 500 internal server error

If you get a message saying "Internal Server Error" while excuting a cgi script inside the cgi-bin folder, then it is better you check the Apache error log, you will probably find that it says "Premature end of script headers", possibly along with an error message generated by your CGI program. In this case, you will want to check each of the below sections to see what might be preventing your CGI program from emitting the proper HTTP headers.

File permissions

Remember that the server does not run as you. That is, when the server starts up, it is running with the permissions of an unprivileged user - usually nobody, or www - and so it will need extra permissions to execute files that are owned by you. Usually, the way to give a file sufficient permissions to be executed by nobody is to give everyone execute permission on the file:
chmod a+x first.pl
Also, if your program reads from, or writes to, any other files, those files will need to have the correct permissions to permit this.

Path information and environment

When you run a program from your command line, you have certain information that is passed to the shell without you thinking about it. For example, you have a PATH, which tells the shell where it can look for files that you reference.
When a program runs through the web server as a CGI program, it may not have the same PATH. Any programs that you invoke in your CGI program (like sendmail, for example) will need to be specified by a full path, so that the shell can find them when it attempts to execute your CGI program.
A common manifestation of this is the path to the script interpreter (often perl) indicated in the first line of your CGI program, which will look something like:
#!/usr/bin/perl
Make sure that this is in fact the path to the interpreter.
When editing CGI scripts on Windows, end-of-line characters may be appended to the interpreter path. Ensure that files are then transferred to the server in ASCII mode. Failure to do so may result in "Command not found" warnings from the OS, due to the unrecognized end-of-line character being interpreted as a part of the interpreter filename.

Missing environment variables

If your CGI program depends on non-standard environment variables, you will need to assure that those variables are passed by Apache.
When you miss HTTP headers from the environment, make sure they are formatted according to RFC 2616, section 4.2: Header names must start with a letter, followed only by letters, numbers or hyphen. Any header violating this rule will be dropped silently.

Program errors

Most of the time when a CGI program fails, it's because of a problem with the program itself. This is particularly true once you get the hang of this CGI stuff, and no longer make the above two mistakes. The first thing to do is to make sure that your program runs from the command line before testing it via the web server. For example, try:
cd /usr/local/apache2/cgi-bin
./first.pl
(Do not call the perl interpreter. The shell and Apache should find the interpreter using the path information on the first line of the script.)
The first thing you see written by your program should be a set of HTTP headers, including the Content-Type, followed by a blank line. If you see anything else, Apache will return the Premature end of script headers error if you try to run it through the server. 
Your minimum CGI program requirement:
#!/usr/bin/perl
print "Content-type: text/html\n\n";
print "Hello, World.";

Error logs

The error logs are your friend. Anything that goes wrong generates message in the error log. You should always look there first. If the place where you are hosting your web site does not permit you access to the error log, you should probably host your site somewhere else. Learn to read the error logs, and you'll find that almost all of your problems are quickly identified, and quickly solved.