Code-Based "Build" Optimizations
When you need to optimize application performance and scalability, the first strategies to consider are caching, minimizing page size, and distributing resources among more Web servers. ASP.NET offers options for addressing all of these areas, but while code-based approaches can offer viable solutions, they also create new challenges.

Caching
Caching offers one of the most common and effective means of enhancing performance. After all, it's easy to see how an application that requires servers to dynamically generate a new page for every request received - despite the fact that most of the content in the page is static and redundant - can waste a huge amount of resources. The main caching strategies to consider include output caching (storing and reusing static content in server memory), browser caching (storing static content in the user's browser), and data caching (storing data in the server's memory for use by any code in the application).

ASP.NET offers rich features for implementing caching, and you can use these features to create flexible caching mechanisms that conform to configurable rules and expiration policies, and are triggered by the specific data in the request for the page. In principle, these strategies reduce the amount of work required by the server to process each request, increasing the number of requests each server can handle in a given period of time. Unfortunately, manually implementing any of these caching strategies has drawbacks as well.

Coding browser caching or output caching mechanisms is complex and time-consuming, since you have to incorporate caching instructions (such as declarative directives) into all pages and controls that use them. Beyond coding the caching mechanisms, practical time and cost constraints dictate that you can't cache everything. So you have to invest time and resources into identifying those areas of the application where caching will provide the greatest value.

The biggest challenge with code-based caching strategies, however, is maintaining cache currency. For caching to work correctly, you have to develop dynamic mechanisms to recognize when the cache is out of date and respond appropriately. ASP.NET 2.0 does offer features to help simplify this process. (Cache expirations can occur automatically when the requesting server recognizes changes to a query result.) But again, actually implementing these mechanisms is not a trivial task.

Minimizing Page Size
One of the biggest culprits for poor application performance - and one that programmers often don't account for in the development process - is the size of pages. Even when you're caching static resources like JavaScript files and images on the browser, just the HTML content of a page can become significant. (I knew one developer who built and tested an entire site internally, only to find on the day it went live that each page was 1.5MB.) To clamp down on bloated page sizes, your first target should be ViewState data.

ViewState, a block of data in a hidden form control on the page, plays an important role in almost all ASP.NET applications by storing data vital to the controls on the page. However, depending on the number and type of controls used, ViewState data can get extremely large. Theoretically, you could just disable ViewState, but unfortunately many controls can't function properly without it. Your options are either to examine closely every single page for opportunities to limit ViewState without affecting functionality, or find ways to code around it.

ASP.NET 2.0 introduced one option for addressing this problem with ControlState, a new form of hidden control storage that controls can use even when ViewState is disabled. However, this approach depends on controls that actually take advantage of ControlState, and most still only use ViewState. Of course, all of these solutions increase your coding requirements as well.

Another technique you can employ to reduce the performance hit from large pages is AJAX-based controls that can refresh sections of a page without requiring a full postback of the page. AJAX has its own shortcomings, however, most notably, the complex, extensive programming required to make it work well. You can use new Microsoft-created controls and JavaScript frameworks aimed at reducing the coding requirements for AJAX, or use runtime optimizations such as Silverlight to improve the perceived performance of the application. As Web-based software techniques, however, all of these strategies put an additional burden on your network and servers.

Distribution
If you're fortunate enough to create an application that is wildly successful (if you wake up one morning and your Web servers are getting bombarded by hundreds of requests per second), and you haven't planned for that possibility from early on in your development process, you can expect some serious scalability issues. Cached objects, session state, and transient .NET objects that were not an issue under a relatively small load can rapidly accumulate to the point where they overload the memory in your servers.

The traditional solution for scaling an application has been to simply throw more hardware at the problem to distribute the load among more Web servers. The problem with this strategy is that unless you've architected your application from the start to support load-balanced, parallel processing of requests, it doesn't really work. Resource affinities in the code - hidden dependencies that require code to run on a particular thread, CPU, component instance, or server - can lock processes into a specific resource and impede effective load balancing. Ultimately, you can double the Web servers in your server farm only to find that you now have twice as many servers overloaded, and very few additional users getting data.

The most common resource affinity problem is in-process session state, which assumes that all requests from a single user session will be processed on the same Web server. To deal with the session state problem, you can take session management out of process - distributing it to a separate shared resource that all Web servers in the farm can access. ASP.NET offers built-in support for storing session state in a separate database or designated state server, but neither strategy provides an ideal solution.