1. Caching Your WordPress Site
Caching your WordPress site is highly recommended. Caching will bring the load down on the server and HTTP request which will be less increasing your page load.
Recommended caching plugin is W3 Total cache.
You can also look at server side caching like memcached. http://code.google.com/p/memcached/
2. Using S3 and CDN to increase your site speed.
CDN - Content Delivery Network, using this will spread your files geographically across network of servers. When a visitor comes to your site all your files stored on those servers will load from the closest server to your visitor. I’ve seen site performance increase for sites at least 40% in load speed.
Recommended for S3 and CDN is Amazon, you can use the W3 Total Cache plugin to help with this as it will handle all this for you when you have your accounts setup.
By minimizing your files by removing white space will decrease the file sizes and combining your files into one file. For example combine css files into one css file and move all inline css to your external css file.
Here is a url that will try to minimize your css for you. http://www.lotterypost.com/css-compress.aspx
4. Using image sprites.
Combining images on to one image file and then using css to display the image you want by adjusting the background position will also help increasing the site page load and decrease HTTP requests.
This helps to minimize the round trips to the server to download files, imagine having 10 image files loading on your page, now that is 10 http requests that needs to be made. Having all those images on one file means only loading the image once and then using it where ever you need it.
More article on CSS Sprites
5. Indexing SQL queries
Using indexes on your queries to the database can help reduce the querying time for all sql queries. This will help to take the load of your MySql server and make sql use less server memory thus increasing the overall data speed.
Here is a good article about MySql indexes http://www.informit.com/articles/article.aspx?p=377652
Do you have any more to add to the list? Please comment and let me know what helped you increase your sites performance?