SOAP Compression

Image:header_technote.gif

Introduction

NOTE: Applies to Microsoft .NET Framework 1.1

SOAP messages generated by both the sforce API client and the sforce API service can become very large. This can have adverse performance implications due to increased transmission and reception periods over the network. In the case of mobile devices on slow connections, this issue is magnified. Because increasing client performance capabilities is often much cheaper and easier than increasing network bandwidth, performing pre- and post-processing of SOAP messages to reduce their size is a reasonable solution.

The sforce service fully supports the 1.1 content encoding specification for use of compression. To date, the .Net framework generated proxy clients to not fully support this at the HTTP transport level. This requires us to use extensions to implement compression/decompression in .Net.

By using compression on the client and the server, the size of large SOAP messages transmitted across the wire can be reduced by up to 90%. This means that a 100K SOAP message can be compressed to 10K. A call that may have spent 10 seconds on the network will spend nearer 2 seconds on the network.

Image:Compression_serialization.gif

For the majority of SOAP messages that you generate to the sforce service, you will probably not need to use compression. For batch operations in particular, compressing the out-bound message can provide very good performance gains. Requesting that the response be compressed will provide significant gains for queries that return large amounts of data.

Using Compression is contingent on the ability to control the HTTP headers (HTTP protocol @ w3c) that are generated with your SOAP messages. Most web services support some kind of compression and will send compressed data to the client if the client has indicated that it can handle compressed data. The client indicates this to the service by setting an HTTP header called Accept-Encoding. This header tells the service what kind of message encoding the client can accept. You typically see this value set to "gzip". If you use this setting when sending you SOAP request to sforce, the message response will be compressed. If the server does compress the data, it will add an HTTP header called Content-Encoding with a value of "gzip" depending on the Accept-Encoding value sent with the request.

If you have set the Accept-Encoding header on your request, you should examine the HTTP header on the response to verify the encoding. Look for "gzip" in the Content-Encoding http header.

Now that we know how to let the client and the service know whether or not we want and can handle compression, we need to know how and when to compress and decompress the messages. Regardless of what language you are using for your implementation (VB .Net or C#) the process is the same.

When you create the parameters that represent the data being sent to a web service you are creating objects that are typed using your language of choice. Your message parameters are then serialized in to a SOAP message. The SOAP message is XML that can be represented as a string and reliably transmitted over the web. You will want to compress your message after serialization.

Upon receipt of the SOAP response from the service, you will have a stream of compressed data. Your application will attempt to deserialize it and convert it to the appropriate DOM objects that are represented by the XML contained in the response. Before this can happen you need de-compress the data back to XML. You will want to decompress the stream before deserialization.

The illustration below shows the process flow for client and server side compression. GetRequestStream and GetResponseStream are overridable methods on the WebRequest object that the generated SOAP proxy derives from.

The rest of this tech note will address the .Net implementation for compressing SOAP messages using wrappers for the System.Net.WebRequest class, System.Net.WebResponse class and the generated proxy class.

This technote previously espoused using Soap extensions to implement compression. Although that method works, it had some pretty serious drawbacks, the worst being that any changes made to the generated proxy were lost if you had to refresh your WSDL file.

Instead of using the Soap extensions, we will explain a significantly more reliable and stable method that takes advantage of object oriented practices.

Wrapping the Generated Proxy Client

The generated proxy in .Net is an extension of the System.Web.Services.Protocols.SoapHttpClientProtocol class. This class exposes the GetWebRequest method. This method is our opportunity to add compression functionality to the Soap client. To allow refreshing of the generated soap client, we will extend the soap client using a new class called GzipSforceService. In this sample we are using soapGzip as the namespace and have added a web reference to the salesforce.com wsdl and called it sforce. The result is that the generated proxy client has a fully qualified name of soapGzip.sforce.SforceService. The declaration of our wrapper class is shown below:

using System;

namespace soapGzip
{

    class GzipSforceService : sforce.SforceService
    {
  }
}

Instead of declaring our proxy client as

sforce.SforceService binding = new sforce.SforceService();

we will declare the client as

soapGzip.GzipSforceService binding = new soapGzip.GzipSforceService();

This gives us a level of indirection on the generated proxy client that will allow us to replace or refresh the WSDL file without having to rewrite code changes to the generated client.

Adding a couple of properties to our wrapper class will give us control over compression on a request by request basis.

private bool _acceptZippedResponse;
private bool _sendZippedRequest;
// Property to toggle compression on and off, set to true to enable
   sending compressed requests, false to disable.
     public bool SendZippedRequest
        {
            set { _sendZippedRequest = value;}
            get { return _sendZippedRequest;}
        }
        /// Property to toggle compression on and off, set to true to enable
        receiving compressed responses, false to disable.
        public bool AcceptZippedResponse
        {
            set { _acceptZippedResponse = value;}
            get { return _acceptZippedResponse;}
        }

To indicate that we would like to have our next request compressed we would set the property on the binding object

Binding.SendZippedRequest = true;

Binding.AcceptZippedResponse = true;

The default constructor for our wrapper class takes no parameters. This will cause the default behavior of no inbound or outbound compression to occur. For convenience, I have included a second constructor that accepts two boolean values to initialize the compression behavior at instantiation.