The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator,
More information about this error may be available in the server error log.
Checking Apache error.log file revealed following error:
[Thu Oct 01 13:14:24 2011] [alert] [client 192.168.1.154] /var/web/test/site1/.htaccess: Invalid command 'RewriteEngine', perhaps misspelled or defined by a module not included in the server configuration
This made it clear that issue is caused by not enabled mod_rewrite module.
Following 2 commands solved the problem:
sudo a2enmod rewrite
sudo /etc/init.d/apache2 restart
Joomla 1.5
Joomla 1.6
Joomla 1.7