Welcome to Professional ASP.NET - Chris Love's Official Blog Sign in | Join | Help

Chris Love's Official ASP.NET Blog

Chris Love's Helpful tips, tricks and pragmatic development knowledge for the ASP.NET world.
Add to Technorati Favorites


ASP Insider Follow Me On Twitter
I Love Code Generation – Why Don’t You?

Several times over the past week I have had some conversations with folks about code generation, some pro (like me), some con and some that find it scary. My first real exposure to a serious code generation tool was at TechEd 2003 in Dallas. I remember an 8 AM session on the last day given by a Microsoft Engineer from Grenoble France. He showed a tool he called Oly Mars to a packed room. Right then and there I knew I was seeing something I wanted in my daily routine.

The demonstration showed how to point the tool at a database with the application’s tables, indexes, keys, etc defined and it would generate stored procedures, DAL, BLL and several user interfaces. The UIs included ASP.NET, WinForms, Mobile and Web Services if I remember it all correctly. All that was left to do was customize the UI’s final look and feel and add any customization to the application’s code that fell outside of the standard CRUD operations. Think about it, that means you could generate about 85% of the code and focus just on the user experience and custom features.

This experience opened my eyes to something very powerful. Eric J. Smith must have been in the room with me because he soon went on to create CodeSmith, my current code generation tool of choice. I think it has the largest user base and community as well.

I also like Miguel Castro’s CodeBreeze and plan on working with it more over the coming months. You can view Miguel demonstrating CodeBreeze on two different episodes of DNR TV, 77, 133.

Visual Studio also has the T-4 generator that is starting to get some traction. I am not a huge fan of it yet and have found it to be rather cumbersome to use. It is the way Visual Studio creates new content for projects, etc. Also do not forget about Visual Studio’s Code Snippet library or just adding common textual patterns to a custom toolbox tab.

The Figure above shows how to use Code Snippets in Visual Studio from either a contextual menu (Right Mouse Click). The example shows how to insert a Try Catch Block. It can also be added by typing ‘TryC’ and pressing the TAB key. More info on using Code Snippets is available on the MSDN site. A Visual Basic Code Snippet editor is also available to help create your own snippets.

Then there are third party add-ins like CodeRush, Refactor and ReSharper (C# tool) that also add code generation and management facilities at our fingertips. If you create either an Entity Framework or Linq to SQL model, those are also code generated. Finally Visual Studio allows you to export entire web sites/applications as project templates or just a form, class or other project file.

All of these tools add value to the development process. And as far as I know all of them allow you to customize templates for code. Most can also be used to generate just about anything that is textual in nature, not just code.

I don’t want to get into how various Code Generators work or pros and cons of each one. I do want to discuss why Code Generation is important to an efficient developer and why EVERY developer should be using Code Generation as a standard part of their development routine.

Reduced Time to Market

This should be obvious, but is not necessarily a truth. On just about every application I develop I generate around 80-85% of the final code in one way or another. This obviously means I can typically create a brand new site from scratch much faster than if I did everything by hand. But, as I will discuss a little later, it leads to adding more functionality or improving the UI itself.

Here is where many developers get scared. They assume that because 80% of their application can be generated in literally seconds they will soon be out of a job. This is far from the truth, in fact it should help them keep their job. Any employer that does not see the value in developers using code generation in their standard routine is probably not one you really want to work for in the first place.

Think about how much more valuable you are using Code Generation over say outsourcing it to cut and paste developers for $10 an hour. You are also adding value by following (your) standard design patterns for your code. And ultimately you do not waste time on routine coding work, like a Data Layer for example. You spend your time working on the real business code or the parts that really make the difference in the application. Anyone can make a standard CRUD application, that code is the first thing that should be code generated. That means a developer using code generation does not waste their time and thoughts on mundane code.

Working with as many small businesses as I have over the last decade I learned early on they want something they can touch and play with within hours of signing a contract with me. Technology scares them and they need to be comforted you are producing a site for them. Having something for them to touch the next day is a huge confidence factor. I have also found this fact to hold true for larger businesses as well. This is where writing a bunch of unit tests before writing any application code will cause a lot of tension with customers, and that is my opinion based on my experience. So having a solid set of code generation scripts is a huge gain.

Reduction of Errors

I was having a conversation with one of my peers on unit testing a few weeks ago. In that conversation I asked if I should code gen unit tests for my generated code? A true TDD conformists would say never, but my friend and I were on the same page that if I had already tested the pattern that I generate code from there was little to no value in writing a unit test from scratch for that code. In reality it would generally be an integration test and not a true unit test in most cases. So having some tests to verify everything is working is a good idea, so I do not want to discount the value of having the tests.

But unit tests are there to help developers reduce development time (at least in theory) and errors sent to Q&A and production. To me code generation serves the same purpose, but in reality, not theory. To me unit testing and code generation have a symbiotic nature in the code development process. When I create a new coding pattern I have to test it in one way or another to make sure it works. Once I am satisfied with the pattern I then I create or add the pattern to my code generation scripts.

By using this proven pattern that is replicated through code generation templates I reduce the chance I make typos and other bugs while writing code. For example I do not forget to add caching code in a get method of a repository. Or maybe a try catch block or data validation routine, etc.

The following code is a standard method pattern I use to get a list of entities from a database. There are two features it has that are ‘optional’; caching and filtering for active records. This is real code generated from a Code Smith template, which is in the screen shot above. I did not have to write one single character in this method by hand.

Public Function Getinforeqs(ByVal vActive As Boolean) As List(Of ContactRequest)

    Dim key As String = CacheKey & "_List" & "_Active_" & vActive

    If EnableCaching AndAlso Not IsNothing(Cache(key)) Then
        Return CType(Cache(key), List(Of ContactRequest))
    End If

    Dim lContactRequests As List(Of ContactRequest)
    inforeqctx.ContactRequests.MergeOption = Objects.MergeOption.NoTracking

    Dim tempList As IQueryable(Of ContactRequest) = (From lContactRequest In inforeqctx.ContactRequests)

    If vActive Then
        tempList = (From lContactRequest In tempList Where lContactRequest.Reponded = False)
    End If

    lContactRequests = tempList.ToList

    If EnableCaching Then
        CacheData(key, lContactRequests, CacheDuration)
    End If

    Return lContactRequests

End Function
 
I like to show this method when talking about Code Generation because it has evolved over the years. At its core it was essentially a ‘Select * from X’ type of wrapper. As I added functionality to the method and made sure the patterns worked, I updated the code generation template. I can have full confidence the generated method will work without testing.
 
You can still create integration tests for this method, but I would simply code gen those as well. If I think of a new way to test the method I will add it to my testing script too. So ultimately using code generation drastically increases the quality of my code and ultimately my applications.
 
There is a down side and that is replication of bad code. Yes there it is, if you have a script that generates a method with buggy or just smelly code you will replicate it everywhere, right? Yes you will. But I have found this to not be a real issue because I try my best to put my generated code into partial classes. By leveraging a partial class I can create one file that is fully maintained by code generation and one that is custom code and maintained by me. If I find I need to regenerate the code I can do it, feel confident in it and just replace the entire file or method from the regenerated code without breaking my customizations. Remember it takes only a few seconds to generate an entire DAL and BLL layer for my applications. Where I do have a problem is in the user interface code, and I have learned to live with it since it is so highly customized once it is generated in the first place.

Increased Product Functionality

Now that I have talked about reduced development time and increased quality I can talk about the product of the first two, increased product functionality. If you think about it, a 6 week project where you can generate 85% of the application’s code in a few minutes leaves you with a lot of time to improve functionality.

There are always custom queries and methods that every application needs to make it unique. How well those features are developed and matured makes a big difference as to how well the application is accepted by users.

There is also a lot more time to sculpt and refine the user interface and experience. This is becoming so important these days as users are expecting more and more richness in applications. If you had to spend that time writing the plumbing and wiring code the application you will have a hard time achieving that rich interface.

Summary

These are my primary observations of using Code Generation in my daily development practice for over 5 years. I know there are many more points I could have made, but I want to hear from you. So please comment either way with your opinions of code generation.

Posted: Saturday, May 23, 2009 11:33 AM

by Chris Love

Comments

Matthias said:

Hi Chris, you are really speaking my mind with your well written article here ... Some years ago I had a project, where they used the CodeSmith code generator for the stored procedures, bll and dal. It was so simple to have these layers generated, that I really could invest my time to create nice and functional user interfaces ... and the users loved it. After switching to a different company, which does not use code generators, you really start missing it!!! So I have to say that all your arguments are very valid and I also love code generators!!! By the way, I´m back to a new project, where they use code generation again .... Cheers Matthias
# June 5, 2009 4:27 AM

rtpHarry said:

Great! Where do I start? :)
# June 18, 2009 6:55 AM

reboltutorial said:

I find the code generators today too "fat". I prefer an agile approach for code generation with rebol which besides is totally free, see a tiny php sample here : http://reboltutorial.com/blog/create-wordpress-plugin/ More complex examples coming soon and for other languages in Java, C# etc.
# June 19, 2009 2:56 PM

Mike said:

Welcome to the club. I've been using Clarion since 1988, now SoftVelocity. It's not just code generation, but a template language with reusable design objects. Then you graduate to application generation.
# June 19, 2009 10:55 PM

noway said:

I moved from vb enviroment to php enviroment because able to manipulate to maximum code and speed.If i do in vb everything is hidden in active x or .net classes.All source majorly have been compiled and not for editing for performance purpose.Yes it was easy even i myself to generate php code as basic crud.But sometimes business logic very diffirent from auto generate code.Dreameweaver also have own php generator code but no way i would used it.Even Zend oop i wouldn't used.I used what i really need only.That's the power of php.If .net can compiled with not all .net dll it would the best .net solution.
# June 20, 2009 1:42 AM

Dmitri said:

I love code generation too, but have found that promoting the approach is very difficult - seems like there's too much FUD surrounding the issue. Plus, not everyone wants to learn how code generation even works.
# June 21, 2009 3:54 AM

Paco said:

Code generation is useful only when the code you want to generate is very similar in multiple cases, like the dataaccess and caching code you show. Instead of generating the same code again and again, you can also think of a writing code for more generic uses manually. Applications with duplicated (or very similar) code are very hard to maintain. That's why people hate to inherit an application with a lot of generated code. When you would have to maintain an application with 85% generated code, you can probably reduce the number of lines of code by 80% by writing a little bit smarter code. Code generation should be used to save you time typing in the code, not to save you time to think about good solutions. That doesn't mean I'm against all code generation, I do generate my database schema and some other things.
# June 21, 2009 6:47 AM

Andrei said:

How about code duplication?
# June 21, 2009 2:11 PM

Chris Love said:

Paco, I do generate at least 70% of all apps and that is just a WAG really, but as much as 90% in some cases. I generate the initial markup, javascripts, CSS, etc and go from there. I am constantly refactoring code and ultimately reducing my footprints, but that has nothing to do with Code Gen. per se. Once I establish a pattern works, I create corresponding Code Gen templates in the oppropriate tool, CodeSmith, VS, etc.

# June 21, 2009 5:00 PM

Rob said:

You're obviously an idiot. And it's contagious. Shut up.
# June 21, 2009 10:12 PM

Mike Hadlow said:

I'm with Paco here. Code generation is usually a poor choice. If you can generate the code, you can just as easily write a framework that does the same thing at runtime. Code generation is a blatent abuse of the programmer's core imperative: Don't Repeat Yourself. You are mechanically repeating yourself :) In your example above you are mixing multiple concerns in the same method: caching, query definition and data context merge options. All of these concerns could be separated out and defined in one place only, all it requires is a bit of thought. Code generation also implies that you have arrived at a chosen pattern and now intend to stick with that pattern for the rest of the time that you use that generator. It's at odds with the idea of relentless refactoring, where you are always ready to try a better way of doing things.
# June 22, 2009 8:58 AM

andyclap said:

IMHO Code generation == legacy application generation. I initially drafted a very level-headed reply, but I realise I wasn't be being true to myself, so ... how do I hate it, let me count the ways: It's a technique from the early 1990s, usually set up to generate vast swathes of VB3 style procedural goo; a maintenance headache that creates applications that are almost impossible to enhance. The generated code lies un-refactored and unloved as new techniques make it obsolete. A push-button code-debt generator. Not DRY (do not repeat yourself) but WET (Written every time). Cut and Paste coding. Not SOLID but ... SHI... oh you get my point of view. If you're going to use it, I'd suggest the golden rule is that the generated code is not part of the codebase, it's merely a build artifact: It must not be modifiable post-generation in any way. It should be automatically generated as part of the build from a "single source of truth". But getting this right is hugely difficult as you have a distance between the templates and the code. For me there are far better ways of encapsualting common behaviour than cut&paste. Generics, OO, even dynamic code. Or just stick it in a library for crying outloud!
# June 22, 2009 10:34 AM

Chris Brandsma said:

I will say I agree 50% with you -- because it is more fun to disagree anyway. Code Gen as a way to get the general code out faster, code you are then going to add to/modify heavily is perfectly fine. Creating general classes in standard ways is a very good thing. But, I feel you have fallen into a trap that I see all to often. You are generating your DAL. That is the one piece of code that you should do yourself. Note: you are either going to waist time creating your database, or waist time on the DAL -- and sometimes both. Your DAL ends up becoming your main api. Especially in a databound style application, you will spend far more time interacting with the DAL code that any other code in your entire project. If it is simply gen'ed from the database, that means you have waisted valuable time creating the database BEFORE you actually knew how the database was going to be used. Now you become afraid to change the database because then you would have to re-gen the DAL and break all of your code. Create your DAL the way you need it, then gen the database off of that. This is what NHibernate, and the next version of Entity Framework (and possbly SubSonic 3) do. That said, if you enter a project with an existing database with existing applications -- have at it. So again, I'm not against CodeGen. I love code gen. I just do not like the way I see most people using code gen.
# June 22, 2009 1:35 PM

Dan Crowell said:

Well said, Mr. Love. I love code generation for the reasons you mentioned. My preferred tool is My Generation (www.mygenerationsoftware.com). I can't compare it to CodeSmith since I did not keep CodeSmith beyond the free trial. My Generation does everything I want and it is free. Cheers, Dan
# June 22, 2009 1:52 PM

Chris Love said:

Chris - I think we are on the same page. Since moving to Entity framework last summer it serves as my DAL and the model generates a foundation of the BLL for me. Before EF I relied on the DAAB in the Enterprise library for the DAL layer and generated my entities and repository classes based on my DB schemas. Now, I extend the code generated by the EF model wizard with my code generation and then customize the 'last mile' of the BLL.

# June 22, 2009 3:31 PM

Chris Love said:

Mike - No I refactor all the time. Update my generation templates and move forward. Which is great because I can regen code that relies on the less refactored pattern to the new pattern without breaking anything. You have to love partial classes :)

# June 22, 2009 3:34 PM

Thomas Reinberger said:

I'm using the Olivanova Model execution system for several years now and we're quite satisfied with it because you can use a really powerful code generation framework and you so you don't have to develop it on your own (perfect for a small company like mine). Nevertheless, this method also has it's drawback, but if you want, you can take a look into my blog article and the running application to see how that stuff works. Thomas
# June 22, 2009 4:25 PM

Ian said:

Code generation haters are pseudo-pragmatists. If you rule out a tool, you are just as bad, or worse than those who over use a certain tool. Code Generation has it's place, just like unit tests, and patterns. By arguing against it, you are effectively stating jigs are dumb, and all carpenters who use jigs are dumb. Generally the template or pattern for code generation is based on an effective, efficient and established code base, and therefore it does not have the need for constant re-factoring of your data layer, and it can keep you from abstracting yourself into a corner.
# June 22, 2009 5:22 PM

ZagNut said:

@andyclap - just..wow... @Chris Love - CodeSmith = great, SubSonic = fantastic. Give it a try (nod to Brandsma) @Paco / Mike - Do you hear what you're saying? "Why generate code when you can write the framework"? And Mike in particular; CodeSmith has caching and all that. It uses the Enterprise libraries, which are extremely powerful.
# June 23, 2009 9:05 AM

andyclap said:

Chris, I realise by your further comment that you make sure your code can be regenerated from amended templates, this is hugely important. Ian - yes a consistent, data layer is fine, even code-generated; the thing I'm ranting against is when you are then expected to modify the generated code to customise behaviour: You *will* need to change the code, it *will* need refactoring. Even database technology continually develops (think nvarchar(max)). Alteration after generation makes it hard to regenerate customised code, and thus hard to improve. And I'd say carpenters generally don't need to support their output in the way that we do. But to extend the metaphor: What happens if a fault in the jig means all their bookcases collapse? Or more likely the customers love the bookcases but actually decide they really wanted them to hold bigger books ... The carpenter has to recall every single bookcase and manually spend time rejigging them. Any customisation to the bookcases has to be undone, and repeated after retooling. The non-dumb carpenter makes sure rejigging bookcases is easy (as he knows what his customers are like) and keeps dependencies between the components in his bookcases to a minimum. The clever carpenter creates a meta-jig machine that allows all bookcases to be rejigged automatically. But the genius carpenter creates bookcases that rejig themselves. Anyway: forget bookcases, just use a library (preferably one that supports line breaks).
# June 23, 2009 10:02 AM

ICW said:

One thing left out of the discussion is the role of having a centralized repository of metadata about your domain. If this is maintained religiously, the question of upgrading, enhancing, etc. becomes a cinch. If you're using CSLA 2.x and you've an XML document defining the metadata about every domain object, it becomes relatively easy to upgrade as new versions ship. (The need to hand code certain portions of code is recognized and accounted for by tools like CodeSmith.) It becomes theoretically possible to switch frameworks entirely as well. If you have isolated and described your business according to a detailed repository of info, you are no longer attached to any runtime/framework/etc. Easier said than done :) I agree that one should perform code generation as part of a build process by merging the templates with the metadata (partial classes/methods allow for preserving hand coded sections between builds in the event your tool doesn't support this). Kathleen Dollard wrote a fantastic book describing some of the processes here and I knew it would never take off in the real world. The vast majority of shops do very little formal modeling or design (I say this as a developer with 15 years experience working in Fortune 500 companies to startups. How many shops have licenses for UML tools or even Visio for their devs?). I've had developers flip out at the suggestion that an activity diagram be written for a module. Expecting professional developers to maintain a metadata repository is unrealistic. I've not much hope that DSLs will fair any better in the future. Software is about get-it-out-the-door yesterday because the management chain is waiting for the bonus checks. Methodical planning, even if it yields greater cost benefits over time, is hated by executive planners because it requires greater initial investment (thinking) up front. We live in a period of very, very short time preferences (think business quarters). In addition, only 1 or 2% of all developers have the discipline and talent to design efficient processes for maintaining code over the long term.
# June 23, 2009 12:53 PM

yteslax said:

As far as I know there was a good article about this kind of approach by Martin Fowler. In that article there was a comparison of code generation and reflection. IMHO it was very good article and the ideas proved several times in my past experiences. Basic rule is : if you have static data model which will not change, then code generation is ok otherwise it is very dangerous approach. Because you have to generate code again and again regarding the changes in your data model. Besides, code generation (creating DAL) is data centric. I'd prefer to go with model centric approach. Moreover you lose your control over the code. If you would like to change something about DAL, it is hard and if you change DAL somehow and then you changed the data model later as well, you will regenerate code and you will lose your extensions (or modifications, or whatever)...
# June 26, 2009 6:07 AM
Leave a Comment

(required) 

(required) 

(optional)

(required) 

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS