Web site projects:

Web sites are a little simpler if you are doing inline code instead of code behind. Web sites also reflect changes in code files without needing to be manually compiled. That means you can edit a file and just refresh the browser.

If you need to explicitly "build", so you can ensure your code doesn’t have errors for example, you can still do so. However, the "build" command doesn’t really compile the project… it just verifies it using the dynamic compiler. While 99% of the time this is fine, I have come across a couple of minor cases where the verification compiler didn’t find an error, but attempting to run the site for real did.

Major advantages of web sites:

• Everything in the project’s folder is part of the project. This makes it easy to use other editors or tools with web sites. If you add files outside Visual Studio, they will still be part of the project. If you edit a file outside VS it will still be compiled and the changes visible when the site is viewed in a browser.
• You can deploy without having to compile… just XCOPY and go. Web sites do support pre-compilation if you choose to use it.
• Files don’t have to be written in the same language. VS will support having a mix of VB and C# code on a file-by-file basis. Sounds good, but I’ve never found this useful personally. Maintaining a site is much easier if you stick with one language.
• The add "item" dialogs in Visual Studio are more intuitive for web sites. I’m not sure why both projects don’t use the same dialogs, but they certainly don’t.
• Profile’s design time compilation is automatic. The ProfileCommon class is created dynamically making it easy to work with the profile provider in a strongly typed way.

Major Disadvantages of web sites:

• No way to really "exclude" a file without renaming it. Refactoring tools and the "compiler" have to crawl through every file in your application. This can get slow if you have a lot of files. For example, I often use FCKEditor, which has a dump-truck load of files. Most of them are not asp.net files. But just having to scan through them when I build or refactor can really slow things down. This has gotten a little better in VS 2008, but not fast enough for my tastes.
• No control over your namespaces. Sure, you can manually add namespaces to pretty much anything, but visual studio will fight you every step of the way. With generated code such as ADO.NET Datasets and such, this gets very hard to control. Eventually you will give up and just let VS put everything in the default namespace. In large applications this gets very annoying, especially if you like a well structured application.
• It is hard (read, nearly impossible) to reference pages, user controls, etc from custom classes in the app_code folder. This produces some interesting problems if you are doing anything fancy like dynamically loading pages or controls and such.
• The application compiles to the asp.net temporary internet files folder. This is a drop location for all that dynamically compiled code that the asp.net compiler will produce. This is a fine mechanism until it breaks. When it breaks you can get really weird errors from the compiler that doesn’t make obvious sense. These are pretty easy to cause by accident. For example, if you tell VS to "build" then refresh a browser pointed at the site at the same time…. the two compiles often conflict in some bizarre manner corrupting the temp asp.net files. When this happens, assuming you figure out that this is the cause of the problem, you have to shut down VS and the web server, manually remove the files from the temp folder, and then restart everything.
• No ability to product XML comment output files.
• Not much control over build outputs. In most projects you can set whether a file is compiled, copied to the output directory, and such. But not with web sites. If a file is in the project’s folder structure, it is part of the project.
• Team Build hates web sites. Lacking a project file, you can use the web deployment project add-on to help out, but even still I’ve found that trying to automate a build for any significantly complex web site is a disaster and time-sink.
• Disconnected Source Control. VS supports working disconnected from source control these days, but I find that it often has problems keeping web sites in sync when you reconnect. This is a sporadic problem, and hard to reproduce, but seems to be more common with delete, rename, and add operations.

The web application project:

The web application project is a little more formal than web sites. You get an actual project file by which Visual Studio tracks the files that are in your project. Web applications do generate "designer" files for your pages that link the code-behind to the controls you’ve put in the markup, but unlike old VS 2003 projects these are much simpler and leverage partial classes and such.

Drawbacks are:

• The site has to be compiled / built before it will run.
• Your project is specific to only one language.
• No automatic support for a Profile class. You have to use a separate tool to generate Profile Common or write one manually.

Advantages are:

• You can split the site into multiple projects.
• Include pre and post build steps to compilation.
• Disconnected source control seems to work more consistently with web application projects.
• Compile and refactoring is much faster since VS has a way to track what is in the project and doesn’t have to scan everything in every folder. Also, you can have stuff in the folders that aren’t part of the project (I find this useful sometimes). 
• You can control namespaces, assembly names, and build behavior for various files in the project. Namespaces are also automatically managed by VS based on the application’s folder structure. This includes a real "project properties" editor too with all those familiar things like build options, references, settings, etc…
• You can generate XML comment output files.
• You can exclude files from the project without having to rename them.
• Custom code files don’t have to be in a specific folder, you can put them anywhere and organize them however you see fit.
• Classes can reference pages and controls.