Comments
paul.nowak wrote: Matt, thanks for the comments. I made an error on the version of Plone. It's 2.5 Plone running on Zope 2.9x. In regards to the additional products, we have a skin installed and we have a product that we had custom developed for us that connects to a PostgreSQL database. We've looked at slow PostgreSQL queries causing problems and have not been able to find an issue. We've also tested for the case where the PostgreSQL server is down and have not been able to create an issue. We therefor...
Cloud Computing
Conference & Expo
November 2-4, 2009 NYC
Register Today and SAVE !..

SYS-CON.TV
Today's Top SOA Links


Ben Forta on ColdFusion
Ben Forta on ColdFusion

In my last column (CFDJ Vol. 1, Issue 2) I discussed database query caching - how to improve application performance by eliminating unnecessary database access. While caching query results will improve application performance, optimizing your application doesn't stop there.

In this column I'd like to continue with the subject of improving performance by improving the efficiency of database operations, but from another perspective. Specifically, I'd like to take a look at how databases should (and shouldn't) be used, and at database features that can be used to enhance application performance.

It's worth noting that some of the features and technologies discussed here (particularly items 3-5) are available only in client/server databases - which is all the more reason you should be using one.

1. Don't Do the Database's Job
This should be obvious, but I see this rule violated over and over. Database software is designed to do one thing only - manipulate data. All databases support a set of basic operations to manipulate data with. And almost always, database-level data manipulation code will outperform manipulation code written in ColdFusion (or any other client language for that matter).

Let's look at a simple example - you need to retrieve a list of all customers who have placed more than one order. You could write a SQL statement to retrieve all orders, and then loop the results to count orders per customer (you'd also need some sort of array or list to store the customers in). Then you'd filter out the ones with only one order. Or you could use a SQL statement that looks something like this:

SELECT cust_id, COUNT(*) AS num_orders
FROM orders
GROUP BY cust_id
HAVING COUNT(*) >= 2

The above SQL returns a list of customers and the number of orders made, and filters out only the customers with two or more orders.

If you want to count records that match a specific criterion, total items in an order or find the highest value in a set, you can perform these operations with simple SQL SELECT statements. Sure, you can retrieve the records with a <CFQUERY> and then loop through the results to perform the calculations yourself. But why would you want to? All you're doing is making ColdFusion work harder than it should, while forcing your database server to sit idle.

Any time you find yourself looping through result sets to perform calculations, passing the results of one query to a second query, or maintaining long lists of variables that contain running totals or other calculations - take a step back, and determine if you're doing the database's job. That's something you never want to do.

It's worth noting that more often than not, ColdFusion developers violate this rule because they're having a hard time with the SQL needed to perform the specific operations (be it the aggregate functions, joins or the correct use of subqueries). And while re-creating the wheel in ColdFusion works well as an interim solution, it's going to create performance and scalability problems down the road. My advice is to grab a good SQL book and master the language (try Sams Teach Yourself SQL in 24 Hours, or wait for my own soon-to-be-released Sams Teach Yourself SQL in 10 Minutes).

2. Enforce Data Constraints
An extension of the above is employing database-level constraints. Constraints are simply rules that govern the validity of data at the database level. Common uses for constraints include:

  • Making sure required fields have values when records are inserted or updated
  • Ensuring that a customer ID used in an order record actually points to a valid customer record
  • Forcing minimum or maximum values on specific fields
  • Defining default values for specific fields

    Data constraints are used to enforce data integrity, and employing constraints of some kind is an important part of application design. Many ColdFusion developers write code to enforce integrity - they validate form fields against databases (using a <CFQUERY> tag) before inserting them into tables; they write complex <CFIF> statements to check for valid values, and then convert data on the fly using lists of <CFSET> statements.

    Once again, this is a task best left to the database. Every decent database allows you to define constraints. From simple rules that establish default values or restrict values to those in another table (foreign keys) to more complex rules that define sets of valid values under specific conditions.

    Here's a practical example, and one I see asked repeatedly on the Allaire Developer's Forum. You're inserting a record into a table, and you want to save the current date and time along with the record in a timestamp field. Sure, you could play with the ColdFusion Now() and CreateODBCDateTime() functions, but why would you want to? All you need to do is instruct the database to use the current date and time if no explicit value is provided, and then provide no value!

    So why is this validation better suited for the database level rather than at the application level? There are two primary reasons:
    1. Performance - The database can process these rules far faster than your application will be able to.
    2. Data integrity - More often than not, your code will not be the only application to access the data. Embedding rules at the database level ensures that all applications use the same set of rules rather than requiring you to re-create the rules in each application and run the risk of missing them in one, or making a mistake in them.
    3. Use Stored Procedures

    Stored procedures are blocks of SQL code that are stored on the database server - similar to custom functions (or ColdFusion Custom Tags). Using stored procedures, you can group sets of related SQL statements together as a single unit. Instead of calling each SQL statement individually and passing values retrieved in one statement down to the next, you can simply call a stored procedure that contains all the needed statements.

    The fine points of stored-procedure use probably require their own column, but for now it's worth noting the primary advantages of using stored procedures:

  • Performance - Stored procedures execute faster than passed SQL.
  • Enhanced capabilities - There are SQL language elements that can't be used in sequential SQL calls (e.g., creating variables, working with temporary tables).
  • Security - Requiring that tables be accessed only via stored procedures (and never directly) can provide added security where needed.

    ColdFusion fully supports the use of stored procedures. In ColdFusion 4.0 you now have two different methods with which to invoke stored procedures: as simple SQL passed to a <CFQUERY> tag, and via the new <CFSTOREDPROC> tag family. The detailed differences between the two are beyond the scope of this column, but Table 1 lists the basic advantages of each.

    There is no hard and fast rule about where to use stored procedures and where not to, but here's one place to start: if you ever find yourself calling a <CFQUERY> and then passing the results of that query to another <CFQUERY> (perhaps using the ValueList() or QuotedValueList() functions), that code should probably be a stored procedure.

    4. Be Trigger-Happy
    All client/server databases support triggers. A trigger is simply a block of SQL code that is executed when specific events occur. Triggers are associated with specific operations on specific tables. For example, you might create a trigger that is called whenever a SQL INSERT or UPDATE operation is performed on a specific table.

    An ideal use of triggers is data cleanup. For instance, if you want state abbreviations always inserted in uppercase you can write a simple trigger for INSERT and UPDATE operations that will replace the passed state abbreviation with an uppercase version.

    Of course, not only will this be faster than doing data conversion at the ColdFusion level, but it also will allow all client applications (including ColdFusion) to share the same code - and it all happens transparently.

    5. Schedule Tasks
    Most databases allow you to schedule the execution of SQL statements. ColdFusion also allows you to schedule tasks by using <CFSCHEDULE> or the ColdFusion Administrator. The ColdFusion scheduling engine is very useful for scheduling execution of ColdFusion code, but it should never be used to schedule the execution of SQL statements themselves unless you need to do processing on the query results.

    For example, if you need to execute daily SQL cleanup code (to perform database cleanup), or if you need to move data from a live table to an archive table on a regular basis, or if you need to run nightly calculations on one table and write the results to another - all of these are best handled by the database's own scheduling engine.

    The rule here is to use ColdFusion scheduling to schedule ColdFusion processing and use database scheduling to schedule database processing. It makes absolutely no sense to tie up ColdFusion when it's the database doing the actual work.

    Conclusion
    The bottom line here is that database developers put a lot of time and effort into optimizing their products. Rather than reinvent the wheel, you should strive to take advantage of their work whenever possible. Most ColdFusion applications are running against powerful database servers that are not working as hard as they should. Push your database server - make it work as hard as it was designed to. Doing so will dramatically improve application performance - and it'll likely save you time too.

    About Ben Forta
    Ben Forta is Adobe's Senior Technical Evangelist. In that capacity he spends a considerable amount of time talking and writing about Adobe products (with an emphasis on ColdFusion and Flex), and providing feedback to help shape the future direction of the products. By the way, if you are not yet a ColdFusion user, you should be. It is an incredible product, and is truly deserving of all the praise it has been receiving. In a prior life he was a ColdFusion customer (he wrote one of the first large high visibility web sites using the product) and was so impressed he ended up working for the company that created it (Allaire). Ben is also the author of books on ColdFusion, SQL, Windows 2000, JSP, WAP, Regular Expressions, and more. Before joining Adobe (well, Allaire actually, and then Macromedia and Allaire merged, and then Adobe bought Macromedia) he helped found a company called Car.com which provides automotive services (buy a car, sell a car, etc) over the Web. Car.com (including Stoneage) is one of the largest automotive web sites out there, was written entirely in ColdFusion, and is now owned by Auto-By-Tel.

  • In order to post a comment you need to be registered and logged in.

    Register | Sign-in

    Reader Feedback: Page 1 of 1

    Subscribe to the World's Most Powerful Newsletters
    Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
    Click to Add our RSS Feeds to the Service of Your Choice:
    Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Subscribe in NewsGator Online
    myFeedster Add to My AOL Subscribe in Rojo Add 'Hugg' to Newsburst from CNET News.com Kinja Digest View Additional SYS-CON Feeds
    Publish Your Article! Please send it to editorial(at)sys-con.com!

    Advertise on this site! Contact advertising(at)sys-con.com! 201 802-3021

    SYS-CON Featured Whitepapers
    ADS BY GOOGLE