Summary of the year 2008

Another quite year of blogging, though the last quarter this blog was somewhat resurrected.

It has been 2.5 years since I started working on Validata product. The last year was moderately interesting one,. I worked on various staff, the most notable of which are an HTTP load testing tool (pretty challenging task) and a language parser using ANTLR (just for a week). I also continued to work on our web application and a couple of desktop applications. I feel good that some my ideas finally were implemented and went in production. Also I think my efforts to improve our development process start to pay off...

Unfortunately my pet project with ActiveRecord, Monorail, MySql, JQuery that I got started was put on hold in february, but still it was a good learning experience. After getting to know Monorail, now I quickly got was ASP.NET MVC all about.

Next year, I plan to become an MCTS (I think my company finds more value in those certifications, than I do), so I am starting with 70-536. As a matter of fact, this morning I finally took a practice test from the training kit. I expected to top it without reading a word from the training book, but wishful thinking did not suffice :) I realized that there are areas of the .NET Framework that I don't know enough about i.e. .NET Security, WMI, Interoperability. I wonder if I concentrate only on those areas, will this be enough to pass the exam. Having solid 4 years experience in the .NET world should be a good starting point, so reading a few chapters for a couple of days might be exactly what I need.

Helper for ANTLR Grammar

So I mentioned I am working on a ANTLR grammar (though not for long). And the experience with it was not always smooth.

The problem

1. We are trying to define a grammar for an existing language which is pretty complex (several hundreds of statements). Every new parser rule had the potential to break the grammar. After inputting several rules, we had difficulty identifying the problematic one. It required a lot of discipline to compile the grammar after each change.
2. Two developers are preparing the grammar together. Once or twice a day, we are merging the .g file. The merging proved to be quite error-prone.
3. The ANTLRWorks IDE is not always stable. In some scenarios it freezes and has to be restarted.
4. The errors and warnings were displayed together, so we had no way of filtering only errors (and we had many warnings).
5. The C# code generation is buggy in some scenarios - we got an error of a duplicate identifier.
6. After each compilations we had to overwrite the lexer and parser CS file in our C# project.

The solution

So I wrote an desktop application that monitors the ANTLR grammar file and on each change, it compiles it and does some post-processing to fix the C# code using a simple regex.

Other features

· Specify settings in .config file (grammar file, working folder, antllr install dir)

· View compilation output - complete output, and only errors

· View history of compilation attempts and results

· View notifications as balloon tips (displayed only on significant changes – broken grammar and fixed grammar)

· Automatically save the last successful build in a subfolder of the working folder

Was it worth it?

Yes, automating error-prone operations is always a good idea. We had problems with merging a few times and this continuous compilation enabled us to quickly find the problematic rules. The tool took me initially 3 hours to develop and additional 50% to maintain and bug fix.

How could we extend it?

Generating the C# files from the grammar is not enough. Its output should be included in the project. Then compile successfully. Then pass some unit tests. We have not reached this stage yet.

Points of interest

There is a good discussion here about how to change the .g file, save, hit F5 in Visual Studio and have it use the most recent version of the generated files. Apparently you need a custom tool to do so. A custom tool as a managed assembly that contains a class that implements the IvsSingleFileGenerator interface. Here is a link to an article how to create such a custom tool.

P.S. If you are interested in the code, let me know. I would have uploaded it on CodePlex, but currently it has a dependency to a third-party grid, so I should better use a normal grid, instead.

My Meta Parser

This is fun... I am working now on a task related to ANTLR and its grammar files. Since the grammar file got big and a bit messy, I wrote a small utility to parse the grammar file (.g file) and to sort the parser rules by name. So there it is: a parser of the parser... my meta parser

NOTE: Well, it was not a full-blown parser, but having the parser rules more strictly structured, made it easy enough to implement.

Ways to Compare .NET Assemblies

Sometimes comparing source code is not a viable option, and one has to resort to comparing assemblies (hopefully not obfuscated) to find out what is new, what has changed and what is removed. This kind of activity can be useful in many ways:

1. Determine what amount of work was done between two releases
2. Estimate the impact of the changes (e.g. what is needed to be retested).
3. Spend some work time doing playing with such amusing stuff :)

But how could you do it:

Commercial Tools

NDepend - I have only watched their video and it is impressive. Unfortunately the trial version does not support build comparison, so you should buy the product to try it for yourself. It is a pretty feature rich product that you can use for a variety of reasons.

Bitdiffer - This sole purpose of this tool is comparing assemblies. It lacks some of the NDepend visualizations goodness, but is nevertheless an excellent tool. You might use its GUI application as well as its command line version to automate your work.

Poor Man's Tools

Reflector Diff AddIn - We all love Reflector and this is one of its useful addins. You can either view the differences or you can have a XML report.

Framework Design Studio - There is a short review here. Unfortunately the tool does not work with all assemblies and I can't report good results with it.

VS.NET Macro To Display All File References in a VS.NET Solution

The problem: You have a large solution (mine is with 154 projects) and you want to quick find if you use file references and to what assemblies are they).

The solution: see the files below. Add the following module and class and execute ShowAllFileReferencesGroupedByProject()

Module:

Imports System

Imports EnvDTE

Imports EnvDTE80

Imports System.Collections.Generic

Imports System.Diagnostics

Imports System.Text

Imports System.Windows.Forms

 

Public Module VsProjectsManagement

 

    Public Sub ShowAllFileReferencesGroupedByProject()

        Dim sb As New StringBuilder()

 

        Dim finder As New FileReferencesFinder()

        Dim fileReferences As Dictionary(Of VSLangProj.VSProject, List(Of String)) = finder.GetAllFileReferencesInSolution(True)

        Dim sortedProject As New List(Of VSLangProj.VSProject)(fileReferences.Keys)

        sortedProject.Sort(AddressOf FileReferencesFinder.CompareProjectsByName)

 

        For Each proj As VSLangProj.VSProject In fileReferences.Keys

            sb.Append(proj.Project.Name).Append(" : ").AppendLine()

            For Each reference As String In fileReferences.Item(proj)

                sb.AppendLine(reference)

            Next

            sb.AppendLine()

        Next

 

        Debug.Write(sb.ToString)

        MessageBox.Show(sb.ToString, "File references")

 

    End Sub

Class FileReferencesFinder:

Imports System

Imports EnvDTE

Imports EnvDTE80

Imports System.Collections.Generic

Imports System.Diagnostics

Imports System.Text

Imports System.Windows.Forms

 

Public Class FileReferencesFinder

 

    Private Function GetAllSolutionVsProjects()

        Dim vsProjects As New List(Of VSLangProj.VSProject)

        For Each proj As Project In DTE.Solution.Projects

            If TypeOf proj.Object Is VSLangProj.VSProject Then

                vsProjects.Add(proj.Object)

            End If

        Next

 

        Return vsProjects

    End Function

 

    Public Shared Function CompareProjectsByName(ByVal prj1 As VSLangProj.VSProject, ByVal prj2 As VSLangProj.VSProject) As Integer

        Dim name1 As String = prj1.Project.Name

        Dim name2 As String = prj2.Project.Name

        Return String.Compare(name1, name2)

 

    End Function

 

    Public Function GetAllFileReferencesInSolution(ByVal ignoreNetFrameworkAssemgblies As Boolean)

 

        Dim fileReferences As New Dictionary(Of VSLangProj.VSProject, List(Of String))

 

        For Each vsProj As VSLangProj.VSProject In GetAllSolutionVsProjects()

            Dim references As List(Of String) = GetProjectFileReferences(vsProj, ignoreNetFrameworkAssemgblies)

            If references.Count > 0 Then

                fileReferences.Add(vsProj, references)

            End If

        Next

 

        Return fileReferences

    End Function

 

    Private Function GetProjectFileReferences(ByVal vsProj As VSLangProj.VSProject, ByVal ignoreNetFrameworkAssemgblies As Boolean)

        Dim fileReferences As New List(Of String)

 

        For Each objReference As VSLangProj.Reference In vsProj.References

            If objReference.Type = VSLangProj.prjReferenceType.prjReferenceTypeAssembly Then

                If objReference.SourceProject Is Nothing Then

                    If ignoreNetFrameworkAssemgblies Then

                        If objReference.Path.StartsWith("C:\Windows\Microsoft.NET\Framework\") Then

                            Continue For

                        End If

 

                        If objReference.Path.StartsWith("C:\Program Files\Reference Assemblies\Microsoft\Framework\") Then

                            Continue For

                        End If

                    End If

 

                    fileReferences.Add(objReference.Path)

                End If

            End If

        Next

 

        fileReferences.Sort()

        Return fileReferences

    End Function

 

End Class