CoNatural Blogging

Better the devil you know - Part I.

Roger Torres - December 1st, 2008

INTRODUCTION

A common situation found in software development projects is “technical” managers trying to introduce, and sometimes enforce, “the latest” technologies without considering if developers have the time or willingness to learn new stuff… and yet expect they will quickly adapt to finish the job within schedule and budget.

I have seen entire projects go down the drain because the proposed architecture imposed a steep learning curve; furthermore it was almost impossible to find new developers with enough experience to fill the gaps. Today, when presented with such scenarios, I basically know what to expect: moving deadlines, frustration, and poorly written code that’s hard to follow and maintain.

Don’t get me wrong, I believe it’s very important to learn and apply new and “proven” technologies in order to succeed and sometimes survive in our careers; we just need to be very careful with the timing, particularly when others are involved and the completion of a project is on the line.

Software architects who understand their team’s abilities and motivations will eventually recognize that in this business sometimes it’s “better the devil you know than the devil you don’t”, and while pushing for the adoption of best practices and proven design principles, we should also strive for simple and pragmatic solutions.

THE CASE FOR IN-HOUSE SOLUTIONS

Mastering a sophisticated technology like LINQ for Entities or NHibernate requires the time and dedication that when extrapolated to an entire development team represents an investment that many companies are not ready to accept, particularly when there are simpler tools like ADO.NET that most developers understand and “can do the job”.

Some would argue that “the more sophisticated the technology you use, the more productive you are”… and as convincing as it sounds in theory, this shouldn’t be the only reason to embark ourselves into the tricky waters of third party solutions.

Let’s consider the following questions for a moment:

1. Are we really going to use all the features the solution is offering?
2. Does the solution support all the features we need?
3. How much time is it going to take for the “entire” development team to learn and adopt this technology without compromising the project?
4. How easy is to find developers with enough experience using this solution?

There will be other questions to ponder for sure, but when we don’t know or like the answers, it’s time to explore other options.

Many designers will love the idea of developing in-house, and while this is fine when we know exactly what’s needed, the most important precondition is having a team with enough “commitment and proficiency” to make it happen. Ultimately, the right decision will be influenced by each specific situation, and the one who makes it will definitely find a fair share of supporters and detractors.

GATHERING REQUIREMENTS

I’m going to present a .NET data access layer that was built with a few conflicting goals in mind:

1. Serve the most common data scenarios found in modern .NET applications following software best practices.
2. Simple enough so that it could be adopted with just a few hours of training.
3. Powerful enough so that users would feel compelled to adopt it.

The motivation to assemble something like this was not the shortage of solutions, because there are many excellent frameworks out there that can perform miracles. What I’ve found in my consulting career is that the majority of .NET developers are still using plain ADO.NET, and they are not really interested in learning other ways to handle their data. It’s not hard to understand their reasons, especially when you see the managers breathing behind their necks all the time, but I’m not going to get there. The problem is that when ADO.NET is used at will, it can lead to messy and sometimes unmaintainable code that will eventually call for a complete rewrite. I though that the best way to get them back on track was to develop a “simple layer” on top of ADO.NET that would basically “enforce” a systematic methodology without adding “unwanted noise” to the process.

I started by presenting the following list of functional and non-functional requirements to satisfy:

1. Use simple abstractions and technologies that are familiar to most .NET developers. Because ADO.NET was already in use, the design should follow ADO.NET as much as possible.
2. Follow best practices. Encapsulate using the open close principle, design in layers, and build a testable system.
3. Use a provider pattern to support multiple database management systems. Leverage the providers found in ADO.NET.
4. All the source code (including SQL scripts) should be under the same source control system.
5. SQL scripts should be organized in namespaces.
6. Enforce strong typing and systematic handling of NULL values.
7. Avoid ADO.NET DataSets. Encourage the use of “plain” object models.
8. Limit the proliferation of stored procedures during the development phase.
9. Provide a mechanism to automatically deploy and invoke stored procedures in a production environment where direct access to tables and views is disabled. This is a commonly used security pattern. Any syntax or dependency errors in the SQL scripts should be found at deployment time.
10. The system (not developers) should enforce a consistent naming convention for stored procedures.
11. Module designers should have the ability to wrap the SQL scripts with custom code to streamline security and business rule validations.
12. Keep the business objects unaware of the framework. All CRUD operations should be invoked by a higher layer in the system (Controller Layer).
13. Rationalize the coding required to materialize data records into business objects. Likewise, develop a simple mechanism to bind object property values to SQL scripts to support CRUD operations.
14. Provide the ability to execute multiple commands in batch (with a single call to the database).
15. Favor the use of transaction scopes. Share ADO.NET connections when possible to avoid the promotion of lightweight transactions to distributed transactions.
16. Develop with Visual Studio 2008, SQL Server 2005, and the .NET framework 3.5.
17. Deliver a prototype based on the AdventureWorks database shipped with SQL Server 2005 within 2 months.

Stay tuned for part II of this article where I’m going to start describing the solution and how it’s applied to real world scenarios.