Complex Data Entry Systems Meet Complex Databases

NextLevelObjects are experts at building complex web based applications that interface with complex databases. The issues revolve around end user ease of use, database integrity, database security, web page security, form validation, database normalization, and command and control systems. Applicable technologies include AJAX, server side code, database abstraction modules, and data encryption and hashing routines.

Creating simple forms to update simple databases is easy, especially when the system isn't mission critical. However, when the system is mission critical, contains sensitive information, or contemplates multiple and simultaneous table updates, the complexity of the system rises exponentially.

The first challenge in developing these systems is making complex systems easy to use while maintaining data integrity. From a programmer's perspective, doing multiple and simultaneous table updates would be a lot easier if it weren't for the ease of use requirements of the customer. Customers naturally want an easy and logical form and/or web page, but that often creates complexity for the programmer. What makes sense from an end user perspective rarely makes sense from a database update perspective, especially if the database is normalized, as it always should be.

A second challenge is segregating business logic code from database update code and from code that controls navigation. As the number of tables, web pages and data entry forms increase, so does the need to keep the code organized (keeping code organized is a primary goal of object oriented programming -- see the article Business Overview of Object Orientation and Design Patterns). When code isn't organized the effort required to make changes and additions increases rapidly. By thinking through early the proper organization of code, a significant increase in speed can be achieved during the development process. Just as importantly, organized code tends to be more re-useable, less buggy, and easier to maintain.

A third challenge is database design. Medium to large databases require a significant effort during the design phase (see the article on Data Modeling). If this phase is rushed the consequences are difficult to use systems that are hard to change and have data integrity problems. A respectable design effort usually takes place when there is a business sponsor and clear, well-written, business objectives. The problem with poorly designed databases is that even the best programmers can't work around them. The database acts as a constraint on every aspect of system development, even affecting ease of use, navigation, and system performance.

A forth challenge is navigation. As the number of tables, web pages and data entry forms increase, so do the number of possibilities as to where the user can go next. Good navigation systems always provide a context for the user. For instance, if the user hasn't signed in and is on Page A, they should see links 1, 2, and 3. However, if a different user is signed in and also on Page A the user should see links 4, 5, and 6. This context makes the system easy to use, but as the number of forms and web pages increase, so do the variety of contexts that exist for the user. One way to meet this challenge is to design a command and control system combined with a roles and responsibilities matrix. Doing so addresses the issue of user context within a rationalized navigation system. While not required for small systems, this type of robustness is extremely useful for reducing the complexity of large systems.

The next challenge is standardization. As the number of tables, web pages and data entry forms increase so does the need for standardization. Standardization includes how pages and forms look, where forms display on the page, how forms are validated, how data is passed to command and control, how erroneous data is displayed back to the user, and how valid data updates the database. In many respects, when it comes to forms, tables and web pages, the more that can be standardized, the better. Standardization not only increases ease of use, but also makes for simpler, more reusable code.

The sixth challenge is security. Medium and large systems not only tend to store sensitive information, but also tend to be targeted by hackers, perhaps only because the hacker knows that a larger system is more likely to store sensitive information. Not only are larger systems a more appealing target for hackers, but a larger system has more programs that can be hacked. This is particularly true for robust shopping cart systems that hackers know contain sensitive information.

Complex interactions between web pages and databases not only exists in large systems, but also in small systems. I designed a 'complex' system that contains only one form and 3 database tables. The ease of use requirements of the client made the form and the database update very difficult. This system was developed for a very small company. Complexity, then, is not reserved to large companies. Small companies also want to go the extra mile to make their web based systems easy to use for their customers.