ClickOnce Deployment + SGen Problem and The Workaround

In the post Slow Startup Performance with Spring.Net WebServiceProxyFactory, I discussed how important it is to have serialization assemblies pre-generated. In the process of achieving this goal, we found a something wrong with the ClickOnce Deployment handling the pre-generated serialization assembly. I believe this is a bug of ClickOnce Deployment.

In this post, I'm going to demonstrate how to re-produce the problem and implement a workaround for it. You can follow it step by step or if can download the source code.

Source code

All example solution source code I used in this post can be downloaded from here.

Setup the test environment

1. Start with a new ClickOnceBug solution with a Web Service Project called ClickOnceBug.WebSerivce.
2. Run the default Default Service1.asmx and make sure we can run HelloWorld.
3. Add a Console Application Project called ClickOnceBug.Client to the solution.
4. Add a Web Reference to the ClickOnceBug.Client referencing to the Service1 in the ClickOnceBug.WebSerivce project.
5. Add a line below to the Main method of the Program class.

   Console.WriteLine(new localhost.Service1().HelloWorld());

6. Run the client. Text "Hello World" should be printed.

Reproduce the problem

1. This is not required to reproduce the bug, but it helps to show the impact of this bug to your ClickOnce deployed application (see this post for details). Add the XML section below to the app.config file in the Client project.

     <!--

      Never deliver your application with this.

      It fills up your temp folder in no time.

     -->

     <system.diagnostics>

       <switches>

         <add name="XmlSerialization.Compilation" value="4"/>

       </switches>

     </system.diagnostics>

2. Double click (not expend) on the Properties folder of the Client project. In the Build tab, set the "Generate serialization assembly" option to "On".
   
3. Go to Publish tab, select "The application is available online only" option (again this is not necessary to reproduce the bug, it only helps to avoid the installation process when we test later). Then click on Publish Now button. The application will be built and the publish folder will open.
   
4. Compare the publish folder and the bin\Debug folder. The ClickOnceBug.Client.XmlSerializers.dll is missing in the publish folder. This is a problem of ClickOnce deployment.
    
5. If we go back to the publish tab in step 3 and click the "Application Files..." button. We'll find that the serialization assembly is listed as "Include(Auto)" although it actually failed to include. It doesn't help if we change it to "Include". If we do so, we may get build error or warning depends on other settings.
   

The impact

So what's the big deal of this? When we set the "Generate serialization assembly" to "On", we ask SGen utility to pre-generate XmlSerializers for our Web Service clients. Without the pre-generated serialization assembly, the XmlSerializer will need to compile a temporary assemble for each type to be XML serialized in runtime. It impacts the runtime performance. Grant Drake has a blog discusses this in depth.

Want to see it in action? Let's open the Temp folder (one easy way is go to Start->Run, enter "%Temp%" without quotation mark and click OK). Put it in detail view and sort files by date, scroll to the end of the list. Run the client application in Visual Studio and there should be no new files created in the Temp folder. Then run the "ClickOnceBug.Client.application" in the publish folder, 7 new files will show up in the Temp folder.

Workaround

Fortunately, we found that this problem only affect the application project that is being published. ClickOnce does deploy the pre-generated serialization assemblies of the dependent class library projects. Let me illustrate it step by step.

1. Add class library project called ClickOnceBug.Workaround.
2. Add a Web Reference to the ClickOnceBug.Workaround referencing to the Service1 in the ClickOnceBug.WebSerivce project. Similar to what we did in step 4 of the "Setup the test environment" section.
3. Go to the Built tab of ClickOnceBug.Workaround project properties. Turn "On" the "Generate serialization assembly" option. See step 2 of the "Reproduce the problem" section for a screenshot.
4. Add a project reference to the ClickOnceBug.Client referencing to the ClickOnceBug.Workaround project.
5. The Web Reference in the ClickOnceBug.Client is no longer useful. Let's remove it but this is not necessary to demonstrate the workaround.
6. Set the "Generate serialization assembly" option to "Auto" for the ClickOnceBug.Client project.
7. Add the namespace Workaround (bolded) to the line below that we have added to the Program class earlier.
   Console.WriteLine(new Workaround.localhost.Service1().HelloWorld());
8. Delete the ClickOnceBug.Client.XmlSerializers.dll file in the ClickOnceBug.Client\bin\Debug folder and delete everything in the publish folder.
9. Rebuild and publish the solution. We can see the ClickOnceBug.Workaround.XmlSerializers.dll is there in a sub folder of the publish location. 
      

We can verify the effect by running the ClickOnceBug.Client.application when monitoring the Temp folder. No temporary files is generated.

Integrate Web Service Client Generation into Build Process

In my last post, I presented a source code generator. It creates Web Service Client source code that implements our own domain interface. In this post, I'm going to talk about how we can integrate the generator into VS.Net.

Important Update: please also read ClickOnce Deployment + SGen Problem and The Workaround if the application is deployed through ClickOnce.

The interface between human and the generator

First of all I need a way to tell the generator what clients to generate. One idea from my colleague was to use the configuration file of Spring.Net. This is a great idea if we can get it to work but here are two major problems here.

• It's not trivial to do this when property placeholder, abstract definition, sub-context and etc come into picture.
• This strongly couples our generator to Spring.Net. I believe the generator is useful with or without Spring.Net.

So what I need is some sort of configuration file that let me specify the clients I want to generate. My generator must be able to easily read the configuration. Being both human and machine readable, XML is naturally the choice.

Below is an example of such an XML file (You can download all source code in this blog from http://www.codeplex.com/WSCodeGen).

<?xml version="1.0" encoding="utf-8" ?>

<web-service xmlns="urn:web-service-client-configuration-1.0">

  <client-group namespace="Example.Client.WebService" 

                xml-namespace="http://tempuri.org/">

    <client class-name="MyTestClient" xml-namespace="http://tempuri.org/">

      <interface>Example.Model.IComplex, Example.Model</interface>

      <interface>Example.Model.ISimple, Example.Model</interface>

    </client>

    <client class-name="HelloWorldClient">

      <interface>Example.Model.IHelloWorld, Example.Model</interface>

    </client>

  </client-group>

</web-service>

And yes, we have schema for it.

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"

           xmlns="urn:web-service-client-configuration-1.0"

           targetNamespace="urn:web-service-client-configuration-1.0"

           elementFormDefault="qualified" attributeFormDefault="unqualified">

  <xs:element name="web-service">

    <xs:complexType>

      <xs:sequence maxOccurs="unbounded">

        <xs:element name="client-group">

          <xs:complexType>

            <xs:sequence maxOccurs="unbounded">

              <xs:element name="client">

                <xs:complexType>

                  <xs:sequence maxOccurs="unbounded">

                    <xs:element name="interface" type="xs:string"

                                maxOccurs="unbounded"/>

                  </xs:sequence>

                  <xs:attribute name="class-name" type="xs:Name" use="required"/>

                  <xs:attribute name="xml-namespace" type="xs:anyURI"/>

                </xs:complexType>

              </xs:element>

            </xs:sequence>

            <xs:attribute name="namespace" type="xs:Name" use="required"/>

            <xs:attribute name="xml-namespace" type="xs:anyURI"

                          default="http://tempuri.org/"/>

          </xs:complexType>

        </xs:element>

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Now our generator has grown from a single class to a console application project that takes an XML file and generates the Web Service client source code. With this enhancement, we can start to integrate with VS.Net.

VS.Net integration

I wish I could implement a VS.Net plug-in and/or template that it generates a Xyz.Designer.cs file based on an Xyz.xml (or may be a fancier one, Xyz.xwsc) file. Paulo Reichert's has a nice blog on this. I think I can implement what he described relatively quick. But the manual installation process of the plug-in kept me away from this solution. We have developers come and go and I don't want to be the one to answer questions like why this is not working for me. I guess I'll leave it aside until somebody is kind enough to tell me how to create an installer shell for it.

An easier, and actually also flexible approach is to make use of the VS.Net's build events. We can call the generator to generate the code in one of those build events to create the source file and then continue the build process. This worked pretty well for me and it also give a way to integrate with Spring.Net's XML configuration file. The generator command line takes an XSLT file that can be used to transform the XML configuration file to our Web Service client definition file. I'll cover the command line option in later section. Let's look at how it is setup in the example solution that you can download from http://www.codeplex.com/WSCodeGen. (Update 9/4: The example solution is updated with an additional project to workaround the ClickOnce Deployment problem)

The example solution consists of three projects illustrated in the following pictures. I think this is a simplified solution setup for most of the real world applications. If the application has only one project, I believe it is small enough to just write the clients manually.

    

We have the domain objects and interfaces defined in the Example.Model project. Both Example.Client and Example.WebService reference to Example.Model.

At the server side, you can find the HelloWorld Web Service that implements IHelloWord interface, and MyTestSerivce that implements both IComplex and ISimple interfaces.

At the client side, we created the WebServiceClient.xml file and a dummy WebServiceClient.cs file to start with. Creating the dummy file and have it included in the project is important so that the Example.Client will compile the generated clients. The XML is exactly was what I posted above.

Here comes the work horse. In the Build Events tab of the Example.Model project properties. We added command line below (in one line) to the post-build event.

$(SolutionDir)build\WebServiceClientGenerator\Debug\WebServiceClientGenerator.exe $(SolutionDir)example\Example.Client\WebService\WebServiceClient.xml

Rebuild the application and we should see that the WebServiceClient.cs file now contains the generated source code. Add code below to Program.cs and enable XmlSerializer diagnose in App.config.

static void Main(string[] args)

{

    IHelloWorld helloWorld = GetHelloWorldClient();

    Console.WriteLine(helloWorld.SayHello("WSCodeGen"));

    Console.ReadLine();

}

static IHelloWorld GetHelloWorldClient()

{

    HelloWorldClient helloWorld = new HelloWorldClient();

    helloWorld.Url = "http://localhost:3586/HelloWorld.asmx";

    return helloWorld;

}

Let's start the Example.WebService followed by the Example.Client. We can see that the temporary assembly along with other temporary files are created by XmlSerializer in your %TEMP% folder. This is because we haven't tell VS.Net to pre-compile the XmlSerializer.

The last step is to open the Build tab of the Example.Client project's properties. Change the "Configuration" to "All Configurations" and then change the "Generate serialization assembly" option to "On". Delete those generated files and run Example.Client again. In the mean time, monitor the %TEMP% folder, we should see no files are generated. Great!

Command line parameters

For the folks want to go beyond what was demonstrated in the Example solution, there are the command line parameters that you can use with the generator. Run the generator without any parameter will give you the usage help below.

Usage: WebServiceClientGenerator DefinitionFile [SourceFile] [XslFile]
    DefinitionFile:
        The XML definition file describes what Web Service clients to generate.
        If the XslFile is provide, the definition  file  is  transformed  using
        XSLT.
    SourceFile:
        Specifies the full path of the source file to be generated. If the file
        extension is not given, the default  source  file  extension  is  used.
        If this parameter is omitted,  default  to  the  definition  file  with
        extension replaced by the default source file extension.
    XslFile:
        If present, the definition file is transformed using this XSLT file.

I haven't gotten a chance to test it but theoretically, you should be able to pass a Spring.Net XML configuration file as the "DefinitionFile" and provide an XSL file to create the final definition file.

Future enhancement

I have started this as an open source project at http://www.codeplex.com/WSCodeGen. More API documentation is needed and Unit Test is still missing. If anybody is willing to help, thank you and please add to the comments.

Welcome suggestions and ideas, or just encouragements of creating a plug-in and template. I'll consider if there is enough demand.

I realized that we can do the same at the server side as well. It will make the next two enhancements possible if we have control at the server side.

I would like to be able to have the client throw the real exception instead of the meaningless SOAP exception that gets thrown universally by .Net framework. This is another architecture problem I would like to resolve.

I can also integrate the ability of using IList<T> and IDictionary<K,V> in web methods, we already have a solution for this using Spring.Net AOP with our home grown carrier classes. But I believe it would be much cleaner if we can do this here.