Thanks to kind folks at mensk, the setup was a breeze.
See Perfect Setup: Ubuntu Hardy+Nginx+MySQL5+PHP5+Wordress

If you are following the instructions on that post though, be careful. The first instruction for

sudo ln -s /usr/local/nginx/sites-available/mydomain.com /usr/local/nginx/sites-enabled/mydomain.com

should be

sudo ln -s /usr/local/nginx/sites-available/default /usr/local/nginx/sites-enabled/default

Otherwise, you will get a connection refused error when you try to test just after completing the Nginx setup. Other than that, I was able to get everything up and running with minimal fuss.

This is the first time I am playing with Nginx, that was pretty cool. As soon as I have some time, I should see how well it works with Tomcat.

And, thanks to good people at jestro for writing the Vigilance WordPress theme. It is a minimalistic theme that lets you control many aspects of the blog. If you are going to customize any of the styles in the theme, make sure to read Easy Upgrading With Child Themes.