I found this link, but it is for Snow Leopard, and I found the first part did not apply to me: in Lion, to enable Apache you must go to System Preference -> Sharing -> Web Sharing.
Enable web sharing, which will enable Apache, and you can also see there the IP to access your computer (I found out that 127.0.0.1 or localhost did not work).

And the rest, just follow the instruction there. I copy here:

PHP

In /etc/apache2/httpd.conf, uncomment this line:

LoadModule php5_module libexec/apache2/libphp5.so

Restart Apache

sudo apachectl restart

Fix a warning appearing in phpinfo()

Create /etc/php.ini and make it writable

cd /etc
sudo cp php.ini.default php.ini
sudo chmod 666 php.ini

In php.ini, find this line:

;date.timezone =

Uncomment it and insert your time zone (http://php.net/manual/en/timezones.php)

date.timezone =America/Vancouver

Restart Apache

sudo apachectl restart

MySQL

Download the MySQL package for Mac OS X.5 (32 or 64 bits depending on your machine)
Install everything in the package in this order: mysql, the startup item, the preference pane.
Start MySQL in the preference pane.
Test it's working:

/usr/local/mysql/bin/mysql

Fix mysql.sock location in php.ini

In /etc/php.ini, replace the three occurences of /var/mysql/mysql.sock by /tmp/mysql.sock

pdo_mysql.default_socket=/tmp/mysql.sock
mysql.default_socket = /tmp/mysql.sock
mysqli.default_socket = /tmp/mysql.sock

Restart Apache

sudo apachectl restart

Extra
Activate PHP short tags

In /etc/php.ini, under Language Options, change

short_open_tag = Off

to

short_open_tag = On

Restart Apache

sudo apachectl restart

So I was developing a small php app which used DOM manipulation classes on my local Ubuntu machine. Everything went fine. Then I uploaded my app to the server which ran Fedora. My app would not work. The DOM classes were not recognized. It turned out that out of the box, php installation in Fedora did not include php-xml package which was needed for DOM manipulation. So, I had to install it.

su
yum install php-xml
yum install php-xmlrpc

Somehow I screwed my PHP installation in Ubuntu so when I ran code to access mysql database through PHP, it spewed trash back to me. I then decided to start from a clean slate so I did this.

sudo apt-get --purge remove apache2 apache2-mpm-prefork apache2-utils apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libdbd-mysql-perl libdbi-perl libmysqlclient15off libnet-daemon-perl libplrpc-perl libpq5 mysql-client-5.0 mysql-common mysql-server mysql-server-5.0 php5-common php5-mysql

That will remove the whole LAMP stack. The keyword here is purge. There is a big difference between simply apt-get remove and apt-get --purge remove. The first will not delete any config file, so the next time you install the same package, the old config file will be used. The latter deletes everything. Just watch closely the console when you issue that command because sometimes it will give warning that says that it can not delete certain directories because they are not empty. In that case, you need to manually delete those directories.

After that, you need to reinstall apache php and mysql by issuing the following commands, in the given order:

sudo apt-get update

This updates your Ubuntu with current package information so when we type the next commands, it gets the most recent data.

sudo apt-get install php5 apache2 libapache2-mod-php5 mysql-server


sudo apt-get install php5-mysql

sudo apt-get install phpmyadmin

I did all that. And then I start my apache, I went into directory in which I have my CI installation. At this point, my DocumentRoot is pointing to /var/www while my CI installation is in /var/www/ci.

All went fine so far. Apache worked well. I then entered a dir in which I had PHP files. PHP files were downloaded by my Chrome browser, instead of being translated by Apache. So enable it:

sudo a2enmod php5

Try opening the PHP file again. Still fail! Then I opened Firefox and tried it again, and it worked. It seemes that Chrome cached this somehow.

All went fine. And then, I copied  a .htaccess file into the /var/www/ci directory. Bust, apache returned 500 internal server error! The fix: enabling mod rewrite by issuing this in command line:

a2enmod rewrite

And then see that you have mod_rewrite enabled when you run phpinfo(). But then, after I enabled the mod_rewrite, the server spat another error. This time it was 404 not found. I got it fixed by setting the DocumentRoot to /var/www/ci.

This is my .htaccess file:

RewriteEngine on
RewriteCond $1 !^(index\.php|images|robots\.txt)
RewriteRule ^(.*)$ /index.php/$1 [L]

I wondered what was going on so I went online and talked with a guy in #linux at dal.net. Here's the conversation:

(06:33:56 PM) arekmalang: Hi all
(06:34:31 PM) arekmalang: I have enabled the rewrite module in apache, but why does it fail when I have .htaccess file in a directory?
(06:35:44 PM) boredboy: ubuntu82: what happens? a 500 error returned, incorrect rules followed, etc etc. Need more info to help you out
(06:36:22 PM) arekmalang: 500 internal server error returned
(06:36:37 PM) arekmalang: oh wait
(06:36:52 PM) arekmalang: that was before i enabled the mod rewrite
(06:37:22 PM) arekmalang: after i enabled it, it returned 404 not found
(06:37:54 PM) arekmalang: if i remove the .htaccess file, then everything goes back to normal
(06:38:05 PM) boredboy: pastebin your .htaccess file
(06:38:11 PM) arekmalang: ok
(06:39:14 PM) arekmalang: here it is: http://linux.pastey.net/137178
(06:39:56 PM) arekmalang: It's a short .htaccess file in Code Igniter tutorial...
(06:41:06 PM) boredboy: so you're getting a 404?
(06:41:09 PM) arekmalang: yes
(06:41:25 PM) boredboy: I assume you have an index.php file?
(06:41:46 PM) arekmalang: yes
(06:41:50 PM) arekmalang: i do
(06:42:03 PM) arekmalang: It says it can't find the index.php
(06:42:32 PM) Tdot__ [~chatzilla@dsl-209-183-27-78.static.tor.primus.ca] entered the room.
(06:43:01 PM) arekmalang: Ha!
(06:43:03 PM) arekmalang: Solved!
(06:43:42 PM) ***boredboy goes back to sleep
(06:43:43 PM) arekmalang: The document root in apache config file needs to be set to the directory where the index.php resides...
(06:43:52 PM) arekmalang: But why so?
(06:44:05 PM) arekmalang: boredboy...
(06:44:10 PM) arekmalang: wake up :)
(06:44:14 PM) arekmalang: enlighten me
(06:46:08 PM) boredboy: huh? which bit don't you get? Your rewrite rule says that if it doesn't start with index.php, images or robots.txt rewrite everything to /index.php/$i
(06:46:24 PM) boredboy: er, $1 not $i
(06:46:40 PM) arekmalang: What I don't get is why I need to set the DocumentRoot to /var/www/ci to make it to work
(06:46:50 PM) arekmalang: Previously DocumentRoot was /var/www
(06:47:07 PM) arekmalang: the .htaccess and index.php are in /var/www/ci
(06:47:47 PM) boredboy: so to hit index.php you had to do http://mysite.com/ci/index.php ?
(06:48:18 PM) arekmalang: that is when docroot set to /var/www and without .htaccess
(06:48:35 PM) boredboy: you can try either one of these
(06:48:48 PM) arekmalang: now, to get to index php i have to go to http://localhost/ only
(06:49:10 PM) boredboy: change your rewrite to RewriteRule ^(.*)$ /ci/index.php/$1 [L]
(06:49:27 PM) boredboy: or RewriteRule ^(.*)$ index.php/$1 [L]
(06:49:38 PM) Tdot_ left the room (quit: Read error: Operation timed out).
(06:49:49 PM) boredboy: if you have the preceeding / you will always try get the index.php in your doc root
(06:50:03 PM) _null is now known as null
(06:50:33 PM) arekmalang: Ahhhh that's it
(06:50:40 PM) arekmalang: Thanks a lot, master!
(06:51:17 PM) boredboy: no problem....., back to sleep

So.... the problem lies in my .htaccess file :) Thanks, boredboy. Well, yeah, that's my adventure for the day :))

Session functions, including session_start() should come first in an PHP page. It can not be preceded by any other thing, not HTML tags, not even a new line!