A new Oba pattern ? Asynchronous Business Task Pattern

The only (efficient) way to have an asynchronous communication between a Sharepoint workflow (see the “Cooperating Document Workflow “OBA pattern) and a LOB (Line of Business Application) is to use what I call the "Asynchronous Business Task Pattern". It is a pattern that I frequently use for my customers, because there is no other alternative in Sharepoint:

1. a workflow create a business task (task is identified by an Task Id)

2. a workflow calls the Lob (provides the LOB with the Task Id)

3. the application (workflow) can go to the next step and in parallel can listen to the LOB by checking if the task is updated.

This pattern is illustrated in the following PDC session (after 5'38'') by Alex Malek (Sharepoint team)

http://channel9.msdn.com/pdc2008/BB47/ ; it was not presented as a pattern, but I’m realizing that is a real & nice pattern.

This pattern makes sense

· On the business point of View:

We create a task for the LOB and the LOB replies to this task. Tasks are not just for humans, they can also be used for business applications.

· On the technical point of view:

To be efficient when a workflow is idle (in our case, it is waiting for the LOB), it must automatically be persisted in the persistence store (the Sharepoint database here), and reanimated when the event comes up. If we don't use the "Asynchronous Business Task Pattern", we have to provide some kind of polling to listen to the reply and Sharepoint, polling doesn't allow the workflow to be persisted; a workflow will automatically be persisted if the running activity is “Listening” to an event (not any kind of event, there are a limited number of events an Sharepoint activity can listen to): in the picture below , the ListenToLOB activity is listening to an Out of the Box Sharepoint event: the OnTaskChanged event. The type of this activity is an OnTaskChanged activity and it is an out of the box activity.

Enjoy my new pattern :-)

As a MOSS consultant, I face many situations where Sharepoint is not an island and needs to communicate or to integrate data coming from different sources, different applications (Line of Business Applications) like SAP and others.

Also as a .Net consultant I use to keep my mind structured by thinking in term of patterns, and there are many of them.

I recently talk to a confirmed MOSS consultant about OBA (Office Business Applications), and it quickly appeared that for him OBA was just using VSTO (Visual Studio Tools for Office), which is a tool I appreciate anyway; but Oba is also about MOSS and the client side is often the cherry on the cake.

The following picture illustrates the main OBA patterns, but there are also sub patterns

I will quickly describe the main Oba patterns; for more details you can read this book.

Pattern 1 : Office Applications as a Reach Channel

Goal: we want to make sure people can easily use the Line of Business Applications (LOB) through the Office System (Rich Client or Web Client).

Sub pattern 1 : direct integration

In this sub pattern, we will call the Line of business application directly, typically via Web services; however we can do this from Rich Client (Office 2003 or Office 2007) by writing .Net add ins with Visual Studio Tools for Office. This is professional development, no VBA anymore and the add-in can be updated thanks to the click once technology (improved in Visual Studio 2008).

We can also apply this pattern by calling web services from Sharepoint (like from a web part), but the main issue we will face is that the data won’t be indexed and users won’t be able to retrieve it in the Sharepoint search.

Another issue will be that if the interface with the Line of business application changes, we need to update every proxy (high coupling).

Sub pattern 2. Mediated Integration

Instead of invoking the LOB directly, we will register it into a mediator and we will invoke the mediator from a rich client or from a web client (low coupling).

The typical mediator is the Business Data Catalog (BDC) in MOSS Enterprise; the BDC allows MOSS to interact with Web Services and relational databases (and not just SQl server).

Using the BDC is quite simple, you just have to define the abstractions of your LOB (entities, likes Customer and Orders,…) and the links between the abstractions (like a Customer can have several Orders); you can use tools like the BDC Editor or BDC Meta Man to generate such abstractions in the form of a xml file; you import the xml file in the MOSS Shared Service provider and voilà, you can display your objects in Out Of the Box web parts, in Custom Sharepoint lists or from your custom code (custom web part,…). This is quite easy and well documented.

Another benefit of this pattern is the the LOB entities can be indexed and thus retrieved through the Sharepoint search textboxes of MOSS.

Pattern 2. Document Generation

This pattern is one of the most popular.

The idea is that the data coming from the LOB can be presented as “Documents” like Microsoft Office documents or others.This is possible thanks to new file formats like Open XML or Open Document used mainly in Open Office.

The Lob can 1°generate this documents  and 2° Move this documents to MOSS/WSS document libraries.

Generating OpenXML document is very easy, you can use the Package and PackagePart classes of .Net 3.0 ; or much easier the new OpenXML sdk (version 2.0 is still in CTP at the moment). You don’t have to install Office on the server to achieve this pattern (anyway, this was not supported and was very instable).

This was also doable with Office 2003 file formats, but tricky.

If we use Excel Services (provided with MOSS Enterprise), we can generate Spreadsheets (with or without diagrams) and display them in Web Parts like the Excel Web Access web part.

Pattern 3. Document Integration

Sub pattern 1. Embedded LOB Template pattern

Here we insert LOB data in the document and we display the data in specific fields of the document

Sub pattern 2. LOB recognizer pattern

On the client, the application might present context-sensitive information, such as the details of a customer whose name is recognized in a Word document. when a user change any data, then the LOB will be updated appropriately.

Pattern 4. Composite User Interfaces

We define different components in the user interface as illustrated in the picture above where we define our own menu (Custom Ribbon) with VSTO, a custom Task Pane (which is actually a .Net Form, which can also host a WPF control if necessary) ; this custom Task Pane communicates directly (“direct integration pattern”) or indirectly (“mediated integration”) with the LOB to fetch data or to update the LOB.

We can also use this pattern on web user interfaces (see below) when we have several parts (web parts in Sharepoint) communicating together.

Pattern 5.Collaborative Site patterns

The idea is to create a web site to allow a team to work temporally on business objects; for instance when sales department has a new important lead, then a new collaborative web site can be created around this lead, just to figure out out approach the lead; and later on, when everything is clear, we can delete the web site.

Another example is a list of incidents or issues; anytime an issue is created, a new temporally web site can be generated for a team.

Pattern 6. Complementary document workflow

2 sub patterns :

LOB Initiated Document Workflow :

here the LOB will generate a document (“Generate Document pattern”), will place it into a Sharepoint list which will trigger a workflow

In the Cooperating Document Workflow, the LOB will communicate with a MOSS workflow along the way.

Pattern 7. Application generated tasks

I’ve noticed that many MOSS (and non MOSS) consultants often underestimate the power of the concept of tasks as a great way to have a communication channel between a LOB and users.

Sharepoint can be a mediator between the LOB and the user: the LOB can generate tasks in a Sharepoint task list and the user will be automatically notified.

Introduction

One of the most important web part provided with MOSS 2007 is the Content Query Web Part which allows to “rollup” list items.This web part is very flexible and extremely fast

You will find a lot of descriptions of  this Web Part in web sites and blogs like the Sharepoint team blog, ,Heather Solomon and Paul Galvin. I recently evaluated the Enhanced Query Web part code from Codeplex that displays data in html tables, so I will quickly describe what I’ve learnt plus new stuff.

My step by step code can be downloaded here.

Step1. Create the contacts list

First off, let’s create a list of contact named “devoteamcontacts” by using the demo-contacts.stp template file provided in the folder step1 of the solution.zip file (import the .stp file in the list templates and create a list instance).

Just to make sure everything works correctly, you can add a Content Query Web part to display the data :

In the Query property :

In the Presentation property :

I’ve renamed the Title step1 :

So far so good!

The Content Query Web part cannot fetch data across site collections ; if you really need it, you can take a look at the Lightning Conductor Web Part.

Step 2.Retrieving raw data

Open you web site with Sharepoint Designer and go the XSL Style Sheets under the Style Library, the CQWP uses 2 important stylesheets :

The Content Query Web part parses the data from the data source and send the data to an xsl template. The “bootstrapper” is the template OuterTemplate in ContentQueryMain.xsl , which among others, defines several interesting variables like the number of rows (“Rows”) and the current row) .

Just to illustrate the raw data the CQWP generates, I suggest that make a copy of ContentQueyMain.xsl, you open up ContentQueryMain.xsl, and find the root template :

Replace it with xslt code that parses everything :

Also, switch the method attribute of the <xsl:output …> from html to “xml” and indent from “no” to “yes” like this :

Then the CQWP will display this :

The link between the raw data and the variables defined in ContentQuerymain.xsl is obvious

You will find a raw version of ContentQueryMain.xsl in the step2 folder of the solution.zip file.

Replace the modified version of ContentQueryMain.xsl with the backuped version.

Step 3. Display and additional field :  Company

Company is provided with the Contacts list, no need to add it !

Each web part instance in the web part page has a set of properties coming from  the Web part gallery and customized at the instance level. As described the Sharepoint sdk, if we want to display an additional field, we have to declare it in the CommonViewFields property like this :

<property name="CommonViewFields" type="string">Company, Text</property>

The way to achieve this is quite simple : export  the .webpart file and name it step2.webpart:

Modify the file by locating CommonViewFields and change it accordingly (if you want to display several fields, they must be separated with semi columns).

Import the new WebPart file  and add a new web part to the page to take the change into account; to achieve this : move the page to edit mode->Add a web part to a web part zone->and in the web part gallery, click on “Advanced Web Part gallery and options” (a bit tricky to find ) :

Then in the next window (“Add Web Parts”), click on the “Browse” option :

Select “Import” :

Import you .web part file and drag and drop the associated web part just below the previous web part; change its title to “Step3”.

As you noticed, nothing seems to be changed. Indeed we still need to put our fingers into some xslt code to display the “Company” field.

Sharepoint Designer is our friend here (at least in the “development” phase); 

At a certain point, OuterTemplate.CallItemTemplate is invoked

By default, “CallItemTemplate” is resolved to fetch the file ItemStyle.xsl (this can be changed we’ll see that later).

If (as we did), you choose The style “Title Only”, the following in “ItemStyle.xml” will be invoked :

Just to verify this, add the following message:

If you refresh you page, you will get this :

Let’s clean up the previous code '(“Hi There”) and add our own Custom template just below the template “TitleOnly”; Just copy and paste the template “TitleOnly” and rename it as “devoteamstyle”.

This is not the good practice as you’ll see later, there will be an alternative; Refresh the page, and reload the item style list in the web part property page; our custom style will show up :

Now we need to make sure the Company field is parsed by the web part; to display the value of all fields fetched by the web part, add the next xslt (in bold) fragment in your custom style style.

(you can also you the raw xslt code used in step2)

<xsl:template name="devoteamstyle" match="Row[@Style='devoteamstyle]" mode="itemstyle">
            <xsl:for-each select="@*">
                <br><xsl:value-of select="name()" /></br>
            </xsl:for-each>
<xsl:variable name="SafeLinkUrl">

….

If you refresh the page, you will notice the Company field in the Web part “Step3”, but not in “Step1”.

Let’s remove the code, and  try to display the Company field :

If we take a look at the rendered page, we will get this :

Step 4. Displaying Content Query Web Part fields in a table

Go back contentQueryMain.xsl and find the template <xsl:template name="OuterTemplate.Body">; as mentioned before, this template invokes another template “OuterTemplate.CallItem” for each row found. Only one parameter is provided to this template (CurPosition), we need to supplement it with an indication about  the index of the last row; also go to the definition of the template ”OuterTemplate.CallItemtemplate”  and define the new LastRow parameter:

Just below provide the paameters to the “devoteamstyle” item :

Now, we need to create our table and this will happen in our devoteamstyle in itemStyle.xsl; the table will only be created one time; however “devoteamstyle” will be invoked for each row.

In the devoteamstyle, provide 2 new params to host the current index and the number of rows:

Now, we can create the table by declaring 2 variables : “tableStart” and “tableEnd”

    <xsl:variable name="tableStart">

      <xsl:if test="$CurPos = 1">

        <![CDATA[

        <table>

            <tr>
                       <td align="left">

                    <b>Title</b>

                </td>

                <td align="left">

                    <b>Company</b>

                </td>

            </tr>]]>

      </xsl:if>

    </xsl:variable>
    <xsl:variable name="tableEnd">

      <xsl:if test="$CurPos = $LastRow">

        <![CDATA[</table>]]>

      </xsl:if>

    </xsl:variable>

Now, let’s make use of tableStart, tableEnd  and display our data in rows and columns as described below :

You will get something like this :

If you want a more Sharepoint like look and feel, you need to modify your table definition by creating a child table for each column (plus an extra column as a separator) and use appropriate css styles:

<![CDATA[
<table width="100%" class="ms-listviewtable" border=0 cellspacing=0 cellpadding=1 dir="None">

<!--HEADER-->
<tr class="ms-viewheadertr" VALIGN=TOP>

<th nowrap scope="col" class="ms-vh2"><div style="width:100%;position:relative;left:0;top:0;">
<table style="width:100%;" CtxNum="1" height="100%" cellspacing=1 cellpadding=0 class="ms-unselectedtitle">
<tr>
  <td width="100%" Class="ms-vb" nowrap>
  Name
  </td>
  <td style="position:absolute;">
  </td>
</tr>
</table>
</th>

<th nowrap scope="col" class="ms-vh2"><div style="width:100%;position:relative;left:0;top:0;">
<table style="width:100%;" CtxNum="1" height="100%" cellspacing=1 cellpadding=0 class="ms-unselectedtitle">

<tr>
  <td width="100%" Class="ms-vb" nowrap>
  Company
  </td>
  <td style="position:absolute;">
  </td>
 
</tr>
</table>
</th>

</tr>        ]]>

You should therefore get something like this :

The problem: Custom stylesheets

One of the common problem with the basic Content Query web part is that developers usually modify existing files xslt that may be used by other web parts, sites, site collections..

We can define our own stylesheets by setting up the properties HeaderXslLink, ItemXslLink and MainXslLink:

HeaderXslLink = "/Style Library/XSL Style Sheets/devoteamHeader.xsl"
ItemXslLink = "/Style Library/XSL Style Sheets/devoteamItem.xsl"
MainXslLink ="/Style Library/XSL Style Sheets/devoteamMain.xsl"

But then, it may be easier to use the Extended Content Query Web part.

Crash of the CQWP when one site column has no display name

My team recently noticed that in some situation the CQWP crashes with the following error:

“Key cannot be null.
Parameter name: key”

This happens when one site column has no display name; more details here.

Step 5. The Extended Content Query Web part (ECQWP)

We can override the Content Query Web part to specify the fields we want to display and also the style sheets we want to use.

That’s basically what the Extended Content Query Web Part (ECQWP) does :

Download the solution from Codeplex , add it to the solution store and activate the site collection feature.

Add an instance of the Enhanced Content Query Web part to your Web part page , and set the Query property as we did before. We can enjoy additional properties like Common View Fields (let’s specify the Company Column here, no need to export/re-import a .webpart file anymore), and also Show Context Menus for items; most customers want their objects to be actionable.

If you apply these settings, you will get this :

The objects are actionable; but even if extra fields have been declared, they won’t show up immediately, we still have to add some xsl code.

Custom style sheets

One of the common problem with the basic Content Query web part is that developers usually modify existing files, that may be used by other web parts, sites, site collections and application.

We can easily make use of a custom stylesheet by defining it in  the property ItemXslLink and we can also provide our own bootstrapper sylesheet (the main).

This is quite easy with the ECQWP.

We can reuse our previous  item style code and paste it in a custom .xslt file, or in an existing one.

Here is a Folder view of the Extended Content Query Web part style sheets folders :

For instance, I’ve copied and pasted my devoteamstyle in ECQWPItems.xslt :

The new style will show up again in the web property window :

Don’t forget to pass the “CurPos” and “Last” parameters to the devoteamstyle template from the “main” stylesheet:

We get this :

But we don’t have our contextual menu anymore.

So I’ve replaced and customized my “devoteamstyle” with a copy of the template that displays the context menus “Context_Menus_With_Icons”

Step 6. Extending the style sheets with your own custom function (in .Net)

You can find more details about this here.

Several people have asked me recently how to focus a search on a content type.

In this post, I will describe the different steps to follow in order to achieve this.

1.Go to your Shared Services Provider Administration page:

2.Click on “Search settings” :

3.Select “Metadata property mappings” :

4.In the Metadata property mappings windows, select the Content type property :

Check the option “Allow this property to be used in scopes” :

5.Define the scope by clicking on the Add rules hyperlink :

6.Make sure Property Query is selected; in the Add property restrictions textbox, select “Content type” and provide the value of the Content type, like “Document” for instance :

7.Go back to… and select the “Start update now” menu option :

8.Go to your portal Administration Settings, select Search scopes :

You scope will be part of a list of unused scopes :

9.Select the Search Dropdown display group and tick the “documents scope” scope; click ok.

10.Go back to your portal, refresh the browser (F5 in IE) and the new display group linked to the new scope will show up.

Recently, somebody asked me to add version history to word documents stored in Sharepoint. The word documents were docx files, based on the OpenXML format. This was a great opportunity to try the new OpenXML sdk 2.0 CTP (I’ve used a lot the Package & PackagePart classes in the past).

I decided to create an event handler overriding ItemCheckingIn method of the SPItemEventReceiver class and to a table in my OpenXml document (fig 1).

Fig 1.Overriding ItemCheckingIn and modifying the document

Then, after having associated my event handler with a document library, checking in a document generated an exception (fig 2)

Fig 2.Save Conflict exception when we update a list item during a Checkingin

To check my code, I’ve tried the following tests (without success):

1. I’ve commented the AddTable() function –>same error
2. Instead of Update, I’ve tried SystemUpdate() –>same error

Then I decided to override the CheckedIn event instead of the Checkingin (fig 3)

Fig 3. Trapping the ItemCheckedIn event

When I checked in, I got the usual window asking to to select the version (fig 4) ; here let’s select 0.2

Fig 4. Version selection , selecting 0.2

When I clicked on the document, indeed the version history shows up (fig 5).

Fig 5.Version history in the word document

But when I checked the version history, I’ve noticed that the system had created an additional draft version (fig 6).

Fig 6. version history: an additional draft version created.

The real culprit is the call to the SPFile::SaveBinary() function (see fig 3); it seems that there is no way to bypass it ! I’ve tried several things like removing the call to SaveBinary() and calling Flush() and Close() on the stream the file was not modified.

I hope that we will find a elegant way to achieve this in Sharepoint vNext.