This is part 2 in a series on the new updates to the Miado API.  You can read the first part in the series here.  In this post, I am going to discuss some of the new features available when configuring Miado.

Query Registration

I’ve had a couple of clients who’ve had a requirement where they wanted to store their SQL in a single location rather than spread across the Repository objects as embedded SQL.  In fact, one client put all their SQL in .sql files and stored them on the file system.  I had never seen this before, but after working with it for a while I definitely warmed up on the idea.  I definitely saw some advantages for doing this:

• Versioning – it’s very easy to isolate changes to individual SQL statements, and you can take advantage of all the benefits of using a version control system, like labeling, diffs, branching, etc.
• Testing - you can open the SQL files directly in your DB management software to test the queries (as opposed to cutting and pasting out your source code, where you have to remove code formatting like string concatenations)
• Collaboration – you can easily share the queries with a DBA, who can help tweak them or run profiling tools against them
• Isolation – all your queries are stored in a common well-known place

So I added the capability in Miado to “register” queries by name.  You can either do this manually or by pointing to files or a directory on the file system.  The IQueryRegistry interface controls the registration and retrieval of stored queries.  To get an access to this interface, you need to declare an Action<IQueryRegistry> delegate to configure itself.  The delegate is passed in via a method on the IDatabase interface.

Registering Queries Manually

The easiest way to register queries is by setting name/value pairs:

  1: var db = new SqlServerDatabase(connString);
  2: db.ConfigureQueryRegistry(queryRegistry => 
  3:     {
  4:         string name = "SelectStates";
  5:         string sql = "select StateCode, Name from States";
  6:         queryRegistry.RegisterQuery(name, sql);
  7: 
  8:         string name1 = "SelectUserByID";
  9:         string sql1 = "select UserID, Email " +
 10:                       "from Users " + 
 11:                       "where UserID = @ID";
 12:         // NamedQuery is a simple name/value pair object
 13:         var namedQuery = new NamedQuery(name1, sql1);
 14:         queryRegistry.RegisterQuery(namedQuery);
 15:     });

Registering Queries From Files

Alternately, you can register queries from files on the file system.  There are multiple ways to do this.  First of all, you can start at a selected directory and optionally pass a Predicate to select only certain files.  From the resulting files, you need to build a NamedQuery list from each individual file.

  1: var db = new SqlServerDatabase(connString);
  2: db.ConfigureQueryRegistry(queryRegistry =>
  3:     {
  4:         // start looking in the "sql" directory for files named *.sql
  5:         queryRegistry.RegisterQueriesFromPath(
  6:             "sql", 
  7:             fileInfo => String.Compare(fileInfo.Extension, ".sql") == 0,
  8:             fileInfo => 
  9:             {
 10:                 string queryName = fileInfo.Name.Substring(0, fileInfo.Name.IndexOf('.'));
 11:                 string sql = File.ReadAllText(fileInfo.FullName);
 12:                 return new NamedQuery[] { new NamedQuery(queryName, sql) };
 13:             });
 14:     });

You can also pass in the specific files you want processed:

  1: var db = new SqlServerDatabase(connString);
  2: db.ConfigureQueryRegistry(queryRegistry => 
  3:     {
  4:         // find the files yourself
  5:         string[] files = Directory.GetFiles("sql", "*.sql", SearchOption.AllDirectories);
  6:         queryRegistry.RegisterQueriesFromFiles(
  7:             files, 
  8:             fileInfo =>
  9:             {
 10:                 string queryName = fileInfo.Name.Substring(0, fileInfo.Name.IndexOf('.'));
 11:                 string sql = File.ReadAllText(fileInfo.FullName);
 12:                 return new NamedQuery[] { new NamedQuery(queryName, sql) };
 13:             });
 14:     });

Be Smart

Obviously, you need to be careful about exposing the raw SQL in any directory that can be compromised.  If you are using a web app, make sure it is not in a publicly visible folder.  If you are using a rich client app, you might want to think about using Isolated Storage as your underlying query repository.

In my next post, I will show how to use the IDbStatement to run queries, including those registered in the IQueryRegistry.

Monday, February 16, 2009 1:00:13 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  Miado

I released a new version of Miado on CodePlex last night, so I thought I’d spend a couple of blog posts explaining how to use the API.  Consider this post as the first in a multi-part series on how to use the newest version.

Basics – Creating an IDatabase

The first thing you need to do is get a handle on an IDatabase.  The interface (and its corresponding implementation) serves as the entry point of the API.  Obviously, the easiest way to do so would be to directly new one up.  The Database constructor takes in a DbProviderFactory object and a connection string.

  1: string connString = 
  2:     "Data Source=localhost;Initial Catalog=MyDatabase; User ID=MyUser; Password=Password1";
  3: IDatabase db = new Database(SqlClientFactory.Instance, connString);

I’ve created a few sub-classes of the Database class that take care of the provider initialization (using the corresponding DbProviderFactory implementations provided in the .Net Framework) and just take in a connection string:

• OdbcDatabase
• OleDatabase
• OracleDatabase
• SqlServerDatabase

e.g.

  1: string connString = 
  2:     "Data Source=localhost;Initial Catalog=MyDatabase; User ID=MyUser; Password=Password1";
  3: IDatabase db = new SqlServerDatabase(connString);

Using Miado with a Depedency Injection Container

I’m a big fan of Dependency Injection, so I normally create a reference to the IDatabase in my application start event and store it in the DI container. 

  1: public class Global : System.Web.HttpApplication, IContainerAccessor
  2: {
  3:     public IUnityContainer Container 
  4:     {
  5:         get;
  6:         protected set;
  7:     }
  8: 
  9:     protected void Application_Start()
 10:     {
 11:         this.InitializeContainer();
 12:     }
 13: 
 14:     protected void InitializeContainer() 
 15:     {
 16:         var container = new UnityContainer();
 17:         
 18:         string connString = ConfigurationManager.ConnectionStrings["MyDatabase"].ConnectionString;
 19:         // create IDatabase instance and store it in the container
 20:         var db = new SqlServerDatabase(connString);
 21: 
 22:         container.RegisterInstance<IDatabase>(db);
 23: 
 24:         // create a Repository object - IDatabase will injected in it
 25:         container.RegisterType<IUserRepository, UserRepository>();
 26: 
 27:         // rest of the container initialization omitted ...
 28: 
 29:         Container = container;
 30:     }
 31: }

When I create my repository objects, I design them to take a reference to the IDatabase in the constructor.

  1: public class UserRepository : IUserRepository
  2: {
  3:     public UserRepository(IDatabase db)
  4:     {
  5:         this.Database = db;
  6:     }
  7: 
  8:     public IDatabase Database 
  9:     {
 10:         get;
 11:         private set;
 12:     }
 13: 
 14:     // CRUD methods omitted
 15: 
 16: }
 17: 

The nice thing about the DI container is that it will automatically inject a reference to the IDatabase instance I registered when it resolves the Repository reference.

  1: IContainerAccessor accessor = HttpContext.Current.ApplicationInstance as IContainerAccessor;
  2: 
  3: // IDatabase is resolved and injected in UserRepository constructor
  4: IUserRepository userRepository = accessor.Container.Resolve<IUserRepository>();

If you don’t want to use a DI container, you can accomplish a similar de-coupling by hiding the IDatabase creation behind a Factory object.

  1: public static class DatabaseFactory 
  2: {
  3:     private static readonly IDatabase _db = 
  4:         new SqlServerDatabase(ConfigurationManager.ConnectionStrings["MyDatabase"].ConnectionString);
  5: 
  6:     public static IDatabase CreateDatabase() 
  7:     {
  8:         return _db;
  9:     }
 10: }

In my next post, I will discuss how to use the IDatabase interface to register queries (a new feature in Miado).

Sunday, February 15, 2009 11:23:59 AM (Eastern Standard Time, UTC-05:00)      Comments [0]  Miado

When I first jumped into C# programming, one of the biggest points of confusion for me was in understanding delegates.  You see, Java had no real similar concept.  So the idea of function pointers didn’t really register with me.  I saw some really weird syntax for event handlers that looked like this:

   1: myButton.Clicked += MyButton_Clicked;

   2:  

   3: // ...

   4:  

   5: private MyButton_Clicked(object sender, EventArgs e) 

   6: {

   7:     // do something

   8: }

Eventually, I caught on that because the function called MyButton_Clicked satisfied the signature required by the OnClick event, it could be added as an event handler to the event.

Aha!  Delegates are nothing more than “interfaces” at the method (as opposed to class) level.  So even though I understood what delegates were, I still didn’t see a widespread use for them outside event handlers.  Sure, I saw examples where they were used, but they all seemed very contrived to me. 

Enter in lambda expressions

Lambda expressions were introduced in C# 3.0.  I’ve said it before, while lambda expressions are new to .Net, they are not even close to being a new concept in programming languages.  They have been around forever.  Basically, it’s a terse syntax for defining anonymous delegates.  In .Net, they were really a simpler way to interact with LINQ.  Now you no longer had to go through the ceremony of creating an explicit method just to satisfy the compiler.  This “inline” function call was especially useful in simple delegate calls, as in a Predicate:

   1: var myList = new List<Person>();

   2: // ...

   3: Person chris = myList.Find(person => person.FirstName == "Chris");

I don’t know why, but I really latched on to lambda expressions in a  way I never did with delegates.  Which is actually a little weird, since I’ve heard numerous complaints that the lambda expression syntax is too confusing for the average developer. 

Changing the way I code

Now, lambda expressions have really changed the way I approach programming.  I’ll give an example from Miado.  I wanted to add support to Miado for creating typed DataSet objects (in addition to regular DataSet objects).  I added support for a querying for a DataSet using generics, where the generic type had to be a class derived from DataSet:

   1: public virtual T QueryForDataSet<T>() where T : DataSet

I wanted to have one common method that handles creating both types of DataSets (the ol’ DRY – i.e. “Don’t Repeat Yourself”).  Here’s the rub - when using a DataAdapter to fill the DataSet, the method actually is just a little different from using a regular DataSet versus a typed one.  It’s not that different, but enough to make you have to handle it two different ways.  Previously, I probably would’ve created a template method or maybe handled it through an interface.  But to use interfaces would require an interface to define the method and two separate classes to implement the varying implementations.  That’s a lot of work to accomplish this seemingly simple task.

Hmm … if I could just pass in the method  that I want to call, I can solve this problem much easier.  Enter in my good friend Action<T> to the rescue! 

Action<T>

Action<T> is a delegate method.  It takes in an instance of the generic T type defined in its signature and has no return value.  So it’s a callback method that performs some action (and thus cleverly named).  I love this delegate.  It’s really great for implementing simple one line functions via a lambda expression.  The syntax becomes very terse without having to build up all the ceremony just to satisfy the compiler.

Implementation

Actually, Action is overloaded so you can pass in multiple arguments.  In my case, I am using two arguments to the method (i.e. Action<T,U>).  Here is the way I implemented it:

   1: private T QueryForDataSetUsingCustomDataAdapterFillFunction<T>(

   2:             Action<DbDataAdapter, T> adapterFillFunction) where T : DataSet

   3: {

   4:     T ds = CreateEmptyDataSet<T>();

   5:  

   6:     using ( DbConnection conn = this.Database.CreateConnection() )

   7:     {

   8:         conn.Open();

   9:  

  10:         using ( DbCommand cmd = conn.CreateCommand() )

  11:         {

  12:             cmd.CommandText = this.CommandText;

  13:             cmd.CommandType = this.CommandType;

  14:  

  15:             if ( this.Parameters != null )

  16:             {

  17:                 cmd.Parameters.AddEnumeration(this.Parameters);

  18:             }

  19:  

  20:             using ( DbDataAdapter adapter = this.Database.CreateDataAdapter() )

  21:             {

  22:                 adapter.SelectCommand = cmd;

  23:                 adapterFillFunction(adapter, ds);

  24:             }

  25:         }

  26:     }

  27:  

  28:     return ds;

  29: }

  30:  

  31: public virtual DataSet QueryForDataSet()

  32: {

  33:     return QueryForDataSetUsingCustomDataAdapterFillFunction<DataSet>(

  34:             (adapter, ds) => adapter.Fill(ds));

  35: }

  36:  

  37: public virtual T QueryForDataSet<T>() where T : DataSet

  38: {

  39:     return QueryForDataSetUsingCustomDataAdapterFillFunction<T>(

  40:             (adapter, ds) => adapter.Fill(ds, ds.Tables[0].TableName));

  41: }

Basically, I made it so that the common method takes in a function that, given a DataAdapter and a DataSet, does some work on those objects.  So the two public methods pass in a different implementation of the Fill() method on the DataAdapter.  Like I said before, it’s kind of like using interfaces for methods.  The common method (in line 23) calls the Action method to perform its specific function by passing back the current DataAdapter and DataSet.

Conclusion

Again, lambda expressions have changed the way I approach programming.  Once you get a handle on the syntax, it’s a great tool to have in your toolbox!

Sunday, February 01, 2009 3:46:49 PM (Eastern Standard Time, UTC-05:00)      Comments [1]  .Net | C# | Miado

Tonight we are going to have another meeting of the Atlanta alt.net user group.  We are meeting at Manuel’s Tavern again, but we are supposed to be in a different room this time.  I know the last meeting at Manuel’s was a little hard to find (if you weren’t there, last time they put us in a side room off another room that was holding a meeting, which had closed its outer doors to the rest of the restaurant). 

This week were are going to be in the “Eagle’s Nest” room, which is located on the other side of the restaurant off the main dining area on its southern side.  Look for the small stairs in the corner of the room.

We are going to be discussing the book, “Agile Estimating and Planning” by Mike Cohn.  If you haven’t read the book, it’s no big deal.  We will be discussing lots of other things, too.  

Hope to see you there!

Wednesday, January 28, 2009 10:01:32 AM (Eastern Standard Time, UTC-05:00)      Comments [0]  .Net | alt.net | User Groups

In case you’ve been living in a cave on Mars with your fingers in your ears for the last year, you’ve probably heard we just ended a little election here in the United States.  I’m not big on the whole politics scene, so I’ll spare you much further discussion on the subject, except that I just don’t *get* why people blindly pledge their allegiance to extremist political views (no matter which side they’re on).   Yet people by the millions are quick to eat up the junk spooned out by people like Rush Limbaugh, (fellow Cornell alum) Keith Olberman, (fellow Cornell alum) Ann Coulter, Sean Hannity, Bill O’Reilly, etc.  If you ask me, they’re all equally wackos.

The same phenomena happens in the software development community.  We as an industry are very divided on what we hold to be the best technology, whether it be operating systems, programming languages, IDEs, frameworks, etc. 

Maybe it’s the middle child in me or the fact that I’m a Libra, but I tend to take a more centrist view on things.

I’ll share two quick items about my past:

1. At one time, I was a snobby Java programmer at the height of the dot-com boom.  If you told me during the early part of this decade that I would be making my living doing (egad!) Microsoft development for a living, I would have called you bat-shit crazy.  But a funny thing happened … I got exposed to C# on a project I was on (i.e. it was mandated we use it), and it turned out to be (IMO) a better version of Java.  And .Net was a wonderful development environment!  It wasn’t at all like going to “the dark side” as I would have been led to believe by my Java brethren.
2. I’ve known my friend Brad, another software developer, since college.  Though neither of us were CS majors, somehow we both ended up in the software development field.  Here we are close to 15 years removed from school and both have taken much different routes to get to where we are in our careers.  While I have toed the corporate line and always stuck to “enterprise” software development (i.e. Java and .Net), Brad has been fascinated with open source technologies, functional programming languages, alternative databases, and highly concurrent systems (way before any of these gathered mainstream momentum).  In short, about 180 degrees from my development background.  Now that he has moved to Atlanta, in the last few years he has been a great influence on me and my view of the software development world.  In fact, part of the reason I so quickly latched on to the new dynamic-language-type features introduced in .Net 3.0 (e.g. lambda expressions, closures, extension methods) was directly because of his influence since he had been telling me about them in other languages for so long.  Once I finally was exposed to them first hand, I quickly understood where he was coming from.

The lesson I’ve learned is to keep an open mind when it comes to software development.  Too many times, we let pre-conceived notions and technology prejudices influence our decisions.  (Want to know a dirty little secret?  I like Vista … a lot, actually.)

I’m finally reading the Pragmatic Programmer (I know, I know, lay off me!), and though I’ve heard it re-told many times by many people, there is a classic adage in the book about learning a new programming language every year.  There’s a reason it’s re-told so often.  It’s because it’s really excellent advice.  If nothing else, it will help broaden your horizon when it comes to tackling problems in your day-to-day programming language.   It can open your mind to an entirely different way of thinking.  As humans, we’re creatures of habit.  We need something to shake things up occasionally.

The reason I’m writing this post now is that within the last few weeks, I’ve been exposed to technologists wielding “Golden Hammers”.  If you’re not familiar with the term, it refers to people who get so focused on a particular technology, they think it will solve every problem thrown at them.  “Sharepoint can do that!”  “You need to employ a 100% SOA so your website can scale, no matter how many users you are projecting!”  “Let’s write an iPhone app for this niche blue-collar customer base!”  When you have this Golden Hammer, everything looks like a nail.

It’s perfectly fine to have your convictions.  Being passionate about something, whatever that thing may be, is one of the highest highs you can get in life.  But don’t stick your head in the sand when it comes to technology.  There is no “one technology to rule them all”.  Try to be objective in your decisions.  Use common sense.  Weigh trade-offs for choosing one technology over another.  Practice pragmatism.  And use the right tool for the job.

Wednesday, January 14, 2009 12:23:06 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  Agile | Programming

There is a great book on the Java programming language called Effective Java by Joshua Bloch.  Even if you don’t use Java as your primary coding language, this book contains innumerable software engineering tips and design patterns that will help you improve in no matter what language you code.  It’s a great reference guide that I continuously go back to, even though I haven’t done much Java coding in the last few years.  The fact that I bought the second edition when it came out despite the fact I no longer really do Java should tell you something.  Josh, if you’re reading this post, you can contact me to see how you can send me the referral fees.

The first item in the “Classes And Interfaces” section of the book is: “Minimize the accessibility of classes and interfaces”. The idea is to not just think of encapsulation in terms of class internals, but to also take it to the next level - meaning up to your API in general.  As Bloch says,

"The single most important factor that distinguishes a well-designed module from a poorly designed one is the degree to which the module hides its internal data and other implementation details from other modules.  A well-designed module hides all of its implementation details, cleanly separating its API from its implementation.  Modules then communicate only through their APIs and are oblivious to each others’ inner workings."

It’s a principle I strive for in all my software design.  By making your API strong and weakening your implementations, you are effectively decoupling your module from the rest of the system.  This decoupling allows you to easily develop, test, and refactor your module in isolation without affecting the rest of the system.  It’s a concept that falls right in line with “The Open/Closed Principle”.  This principal states you should make your software modules “open” for extension, but “closed” for modification.

One of the easiest ways to accomplish encapsulation in an object-oriented language is through the use of access modifiers (e.g. public, protected, internal, private).  Anything you declare as public is open to the whole world.  Therefore, Bloch says that when design your software, “make each class or member as inaccessible as possible.”

I am the author of an open-source project called Miado.  It’s an API that sits on top of ADO.Net and attempts to simplify data access and remove the repetitive, error-prone, boiler plate code we write over and over again in our data access modules.  In building my API, I’ve tried to adhere to good OO design principles.  I’m a big fan of coding to interfaces, using dependency injection, unit testing, and mocking.  Therefore, I’ve exposed most of the API as interfaces rather than concrete classes.  Since I want to abide by the tenants I’ve talked about already, I’ve restricted the corresponding class implementations of these interfaces.  They are declared as “internal”, which in C# terms means they are only exposed to classes in the same assembly in which they are defined.  Since no external assembly can reference my implementations, the “black box” of my interface is upheld, yet allows the consumer to override their own implementations (even if it is only in a mock).

Now this is all well and good, but there’s one more thing of which you need to be aware.  Remember that when you mark a class as “internal” in C#, only classes in that same assembly can reference that class.  Like I said, I’m a big fan of unit testing.  How do you test a class in a test assembly when you can’t even see it???  You could move your test classes to the declaring class assembly, but that’s really just a bad idea.  It adds extra overhead to an assembly and provides no real value to its consumer.  It’s just extra noise taking up space.  And jumping through hoops to remove those test classes as part of a build step reeks of wasted effort.

Java has a nice way to get around this gotcha.  In Java, “protected” scope is somewhat similar to “internal” scope in C#, with one important distinction.  Protected accessibility in Java allows any subclass to access that member, as well as any other class in the same “package” (a close cousin to a “namespace” in C#).  So when you write your unit tests in Java, you merely have to declare your unit test class as belonging to the same package as the class being tested, which then allows you to access any of that class’s internal state and behavior, even if it gets deployed as a separate jar file.  Unfortunately, C# has no similar “namespace” accessibility modifier.

But .Net does have some aces up its sleeve.  There is a way to declare another assembly as a sort of “friend” assembly so that it can see the original’s internal members.  In the original assembly (the one being tested), add the following line to your AssemblyInfo.cs class:

   1:  // let the unit tests see internals of this assembly

   2:  [assembly: InternalsVisibleTo("Miado.Test")]

Monday, January 12, 2009 4:46:33 PM (Eastern Standard Time, UTC-05:00)      Comments [0]  .Net | C# | Java | Miado