7 tips and tricks about the User Profile Replication Engine

 

The SharePoint User Profile Replication Engine (part of the SharePoint Administration Toolkit) is a powerful beast that you can use to:

 

 

 

• Migrate user profiles from a different farm to your farm (MOSS 2007 to SPS 2010 IS supported)
• Synchronize user profiles from a master a farm to a different farm (geographical replication or disaster recovery farm replication)

I have been playing with this tool a bit lately and here are a few things I noticed that might interest you. First of I would advise you to start with the official documentation at: http://technet.microsoft.com/en-us/library/cc663011.aspx . You will most likely only use the full and incremental commands.

1 -Full replication DOES NOT use any windows service

The Full synchronization ( Start-SPProfileServiceFullReplication )does not uses the Windows Service (aptly named “Replication Engine”) used by the incremental command (Start-SPProfileServiceIncrementalReplication). This is important as it means that the full sync is done with the account you launched the command line with WHEREAS the incremental sync is done with the “Replication Engine” service account. Knowing which account is running is important to know which account needs the permissions on the User Profile Service Applications.

2 - Local Admin permissions IS NOT needed for incremental sync

Running the incremental sync (i.e. the windows service) is possible with a non-local administrator account, but it needs:

• Full permissions on the local folder where the replication engine is installed (C:\Program Files\Microsoft\SharePoint 2010 Administration Toolkit\Replication Engine)
• Full permissions on the registry hive
• HKLM:\Software\Microsoft\Replication Engine
• HKLM:\SYSTEM\CurrentControlSet\services\eventlog
• HKLM:\Software\Microsoft\EnterpriseCertificates

This is not based on any official documentation; I had to use Process Explorer to actually see what was preventing the Replication Engine from starting when running with a non-local admin account. The last two permissions might be optional but I added these anyway to be on the safe side. So far I haven’t seen any downside to this workaround.

3 - Full replication IS NOT needed before starting incremental sync

At least it is not always needed. If the User Profiles are already in sync because you did a fresh install and imported the profiles on source and destination with no other changes, you might skip the full sync and start with the incremental sync.

But if there is any difference between your two User Profiles SAs, start with the full as these differences will not be propagated.

In fact the incremental sync only propagates information that has been changed while it is running.

4 - Account changing option (credential parameter) behaves badly

The credential parameter of the Start-SPProfileServiceIncrementalReplication should be used to specify the windows service credential on the first run according to the documentation, well I don’t because it works erratically giving inconsistent results. Instead I set the windows service account itself. You can even do it in scripts.

5 - Social sync IS supported BUT

Social sync doesn’t included managed metadata (because it isn’t part of the User Profiles Service Application obviously) event though tagging is based on managed metadata. In fact tagging is stored in the User Profile DB as reference to the managed metadata entry.

Thus if you need to replicate tags you MUST replicate the managed metadata SA data as well.

6 - MySite URL CAN be replaced with the central administration Url

The documentation tells you to use the MySite Host Url for Source and destination but you can use the central administration URL as well.

In DRP (Disaster Recovery Plan) configuration it might be cumbersome to do so as URLs are shared on the source and destination farm. But the central administration URL being based on machine names is different on both farms.

Beware: this might depend on whether or not you User Profile SA is in the default Service Application Proxy group.

7 - Incremental sync is as fast as light

Yup it is seriously fast, so fast I thought my test protocol was broken when I started to test the replication :)

Beware of SharePoint 2010 Web Analytics on SQL Server Enterprise

You might have missed this information (we did :( ) but when running your SharePoint 2010 farm on SQL Server 2008 Enterprise (or 2008 R2), the Web Analytics reporting database uses the table partitioning feature provided in SQL Server.

 

 

 

For large-scale environments, we recommend that you run the Web Analytics service application Reporting database on a server that is running SQL Server 2008 Enterprise Edition so that the Web Analytics service application can take advantage of table partitioning.

           

         From: Databases Types and Descriptions (http://technet.microsoft.com/en-us/library/cc678868.aspx)

What this means is that your reporting databases will span one additional “NDF” file (and file group, named with the Aggregation DATE suffix) every week as illustrated below:

We realized this while trying to setup the logshipping of this base (which is supported according to this page by the way).

Hope it helps !

SharePoint 2010 – Resizing the Ribbon bar

Recently I was asked to add some element in the ribbon upper part just before the Site Actions Menu. Thus I needed to increase the height of the upper part of the ribbon to fit our content (blue part in the screenshot above).

Problem

After spending an insane amount of time on what seemed so simple, I managed to get it kinda working. I say “kinda” because no matter what I did I couldn’t get the ribbon to display correctly when it was opened.

After checking this article from the SharePoint team: http://sharepoint.microsoft.com/blog/Pages/BlogPost.aspx?PageType=4&ListId={72C1C85B-1D2D-4A4A-90DE-CA74A7808184}&pID=426

I understood that it came from a FixRibbonAndWorkspaceDimensions() javascript function from init.js that just kept setting the height at 44px (ribbon collapsed) or 135px (ribbon opened) no matter what my CSS said. The culprit line is:

var baseRibbonHeight=RibbonIsMinimized() ? 44 : 135;

No need to tell you that after hearing Microsoft boast about their compliance with the latest Web Design best practices, I was expecting more than to find hard-coded height values in a JavaScript file…

Solution

Yet everything isn’t lost as the actual ribbonHeight is the aforementioned value plus a padding value

var ribbonHeight=baseRibbonHeight+g_wpadderHeight;

Thus if you want to increase the ribbon height, you just need to set the javascript variable g_wpadderHeight to the value you want to increment. You might do that in your masterpage as such :

<script type="text/javascript">

g_wpadderHeight = 30;

</script>

Something else to note, the WebPart adding part of the ribbon stopped displaying the “Add”, “Cancel” button after resizing the ribbon bar as they were out of the box. I solved this problem with a bit of css :

.ms-wpadder-buttonArea{

padding-bottom:30px;

}

Custom site logo not appearing on WebPart page

As you probably all know, SharePoint 2010 allows site administrators to change the sites logos just like 2007 did. This is done on the “Title, description, and icon” page

The picture then appears in the top left of the site pages, just before the page title/breadcrumb

 

The trouble is that this doesn’t work on WebPart pages, no matter what you do; users will still see the default yellow icon.

 

Cause

WebPart pages use the TitleBarWebPart, this WebPart allows you to specify a title and logo for the page using the button in the ribbon bar

The problem here is that whatever you do, this Web Part always overrides your site logo setting. If you specify a page specific logo along with the title, it will use it. If you don’t specify a page specific logo, it will use the default SharePoint logo.

This overriding is done in javascript by the code below that you will find at the end of your web part page :

var logoImg = documentGetElementsByName('onetidHeadbnnr0');
if (logoImg != undefined && logoImg[0] != undefined)
logoImg[0].src='/_layouts/images/siteIcon.png';

This javascript code is generated by the TitleBarWebPart, the “/_layouts/images/siteIcon.png” is replaced with the page specific logo if you set one. Thus this WebPart systematically overrides the site wide logo settings.
By the way you will notice that it doesn’t work under Chrome 8 because of a Javascript error.

Solution

Aaron Han suggested removing the Web part from the pages impacted on a MSDN forum:
http://social.msdn.microsoft.com/Forums/en-US/sharepoint2010customization/thread/9706669e-0488-45cd-9cf6-2edfef91f77f
If you don’t want to edit every single WebPart page and/or if you want to retain the ability to change the page title I have another solution for you.
My solution is to change the SiteLogoImage node name in the master page. Thus the javascript generated by the PageTitleWebPart is no longer able to change the logo.

Details

Open your master page and find the line starting with:

<SharePoint:SiteLogoImage name="onetidHeadbnnr0"

And replace the “onetidHeadbnnr0” name with something else; “onetidHeadbnnr1” will do the trick.

Conclusion

I haven’t seen any drawback with this technique, yet :). Tell me if you do !

Percentile in PowerPivot DAX

	When analyzing data, I often find myself computing the percentile value. As I am a big fan of PowerPivot, I went with it lately to analyze some data and tried to get the 90th percentile of my series. So I chose to create a DAX measure and went looking for the PERCENTILE function only to discover that:
Normal Distribution courtesy of Peter Strandmark @ Wikipedia

PowerPivot DAX doesn't support PERCENTILE! Even more shocking, neither is the much more common MEDIAN measure supported.

Don't panic folks; Marco Russo was kind enough to share his way of computing the median. With a tiny tweak it allows us to compute the 90^th percentile as such:

MINX(FILTER(VALUES(MyDataSerie1[Col1]), CALCULATE(COUNTROWS(MyDataSerie1), MyDataSerie1[Col1] <= EARLIER(MyDataSerie1[Col1]) ) > COUNTROWS(MyDataSerie1)*0.9), MyDataSerie1[Col1])

The important part is 0.9. Just replace it with 0.5 for the median, 0.25 for the first quartile and so forth.

I bet the Excel and SQL folks had good reasons not to include these functions, if you have any idea why, leaves me a message. I also posted a line on the MSDN forum(http://social.msdn.microsoft.com/Forums/en/sqlkjpowerpivotforexcel/thread/b1ccf0e6-d5d3-4405-9748-4381a310b57b ).

Accuracy of this measure

I am no statistician, the formula looked ok but I wanted more assurance about its accuracy. Thus I fixed a little test to compare the percentile computed with this DAX measure to regular Excel PERCENTILE.INC and PERCENTILE.EXC formula. I computed the 5,25,50,75,90,95 percentiles using these 3 means for 3 different kind of number series and then averaged the difference between those :

DiffINC is the difference in percentage between PERCENTILE.INC and the DAX measure. Same for DiffEXC.

As you can see, this DAX measure is giving results pretty close (less than 0,05% error) to the PERCENTILE.INC Excel formula. This is close enough for my needs anyway. Yet be careful, with smaller number series (less than 50 items), the error went up to reach values as high as 5% on some cases.

Percentile Explanation

By the way, for those unfamiliar with percentile, the Nth percentile is the value below which N% of the value fall. More details on Wikipedia.

Ok, so why it is useful you might ask, well it is common to analyze your data series without the extreme values that might indicate problem with your data source. An example is in the software performance testing field where you will usually consider the 90^th or 95^th values of the response time in order to know what the reasonable highest response time is without considering test aberrations or errors.

N.B: Some of you might have noticed the new addition to the title of this blog: Jonathan's IT blog – SharingPointers, tell me what you think in the comments !

Using tabs in SharePoint 2010 central administration, a solution

I don’t know about you but tabbed navigation has become a must have for me in the past years. I mean who wants to open a different window for each and every web page.

Well my friends, it seems like Microsoft remembers too fondly those pre-tab browsing days and wants us to return to this prehistoric era. I am talking about the infamous Javascript links one can find in the SharePoint 2010 Central Administration – User Profile Service Application management page:

If you mouse over those links, you will see in the status bar that they are not actual links but instead Javascript calls:

If you add to that, the fact that the navigation is completely useless in this site, you have the recipe for a disastrous user experience as you can’t even open tab to compensate.

Having to work with the User Profile Service Application for a client I got fed up with the situation and decided to act. As javascript is the swiss knife of SharePointer, I wrote a script to replace those javascripts link with actual links so that I can open these links in tabs if needed :

Voila, here are your tab-compliant links.

As FireFox is now fully supported by SharePoint, I use this code in a GreaseMonkey user script. This way the script is applied on this page whatever the farm I am administering. Without even having to modify anything on the farms.

To add the user script, nothing too fancy, just add “*/_layouts/ManageUserProfileServiceApplication.aspx*” in the include. And add the following as the code :

// Jonathan Roussel - http://blog.jonathanroussel.com

// Make sure to Include */_layouts/ManageUserProfileServiceApplication.aspx*



var cUrl=document.location.href;

var aId = cUrl.substring(cUrl.indexOf("ApplicationID"));

var links,link;



links=document.getElementsByTagName("a");

for(var i=0;i<links.length;i++)

{

	var link = links[i];

	if(link.href.indexOf("javascript:AppendApplicationIdAndNavigate(")>=0)

	{

		link.href = link.href.substring(45,link.href.length-2) +aId;

	}

};

If you want a more portable solution, I also put this code together as a Bookmarklet. Just drag and drop this link to your favorite bar. Thus whenever you are on the “Manage User Profile” page, just hit the link in the favorite bar to transform these nasty Javascript links into actual links.

SP2010_SCA_TabAllower

Tested with IE8, FireFox 3.6, Chrome 4.1.

Awesomizing your SharePoint picture libraries in no time

Still using those plain olds SharePoint picture libraries? You know the libraries where a click on a thumbnail brings ups the DispaForm.aspx pages with details about the pics that none of your users care about. What about having your picture in your libraries shown this way instead :

Guess what, it’s as easy as ABC to enhance the way your picture libraries look like with very nice inpage popup.

The basics

To do that you need:

• jQuery, a javascript library to ease development
• FancyBox, a jQuery plugin to handle picture zoom/magnification automatically
• A Content Editor Web Part, I use this editor web part to push my JavaScript code that will
  • Change the thumbnail so that they link to the full size picture instead of the Picture detail
  • Tell fancybox to do its magic so that these regular links launches a zoom on the full size picture

A detailed walkthrough

• Get jQuery from (http://jquery.com/ )
• Get FancyBox from: (http://fancybox.net/ )
• Extract the fancybox folder from the zip you got
• Put the jQuery file and fancybox in someplace your user can access. For the sake of simplicity I am going to put these files in a document library for this walkthrough. In a real world scenario I am putting these files in the 12 using a WSP.
  • Create or open an existing document library
  • Create a JS folder (or not, I just like my stuff to be organized)
  • Push the jquery-1.4.2.min.js and fancybox folder in.

• Now you can go to your Picture Library and Edit the page (screen grabbed from a French WSS Install)

• Add a Content Editor Web Part and edit it
• Open the “Source Editor”

Paste the following code

<script type="text/javascript" src="/Documents/JS/jquery-1.4.2.min.js"></script>
<script type="text/javascript" src="/Documents/JS/fancybox/jquery.fancybox-1.3.1.pack.js"></script>
<link rel="stylesheet" href="/Documents/JS/fancybox/jquery.fancybox-1.3.1.css" type="text/css" media="screen" />
<script type="text/javascript">
// Jonathan Roussel
$(document).ready(function() {
//for each IMG node children of a A node in a .thumbnail class section
$(".thumbnail a > IMG").each(function(){
if($(this).attr('alt') == 'Image')
{
// Replace thumbnail link from item page to the pic itself
var imageUrl = $(this).attr('src');
// Change http://moss/pics/_t/MyPics_jpg.jpg
// In - http://moss/pics/MyPics.jpg
// using a regular expression
imageUrl = imageUrl.replace(/((.)+)\/_t\/((.)+)_([a-z]{3})\.[a-z]{3}/gi,'$1/$3.$5');
$(this).parent().attr('href',imageUrl );
$(this).parent().attr('rel','Gallerie' );
$(this).attr('alt','');
//plugin fancybox
$(this).parent().fancybox({'cyclic':true});
}
});
});
</script>

• Press Ok and enjoy the magic of Fancybox.

The result might not yet be perfect; the two following notes shall set everything straight.

Notes 1,of missing GUI pics

The path to the pictures in the CSS might need to be modified for the pictures used by fancybox to work .
So if you don’t see any Loading, Close, Prev, Next button, open the CSS in the document library (/documents/js/fancybox/jquery.fancybox-1.3.1.css)
And replace

(src='/fancybox 

with

(src='/documents/js/fancybox

Reload the Image Library in Internet Explorer and the pics shall be there.

Note 2, of using this with IE8:

This component seems to have trouble with IE8 and SharePoint. Without a strict XHTML doctype the picture frame are messed up as you can see on the right:

As adding an XHTML strict doctype was a no go, I tried to circumvent the problem by removing the frame and setting the white frame right. Here is the result:

Not as nice as what fancybox can do (lost the gradient border) but good enough for my purpose. To get this result you will need to do these modifications in the css file (the same on you modified in note 1):

Original & Instructions	Target
#fancybox-outer replace line background: #FFF;	top:-10px !important; left:-10px !important; /*background: #FFF; */
#fancybox-inner replace line top: 0; left: 0;	top: 0 !important; left: 0 !important;
.fancy-bg add line	display: none !important;
Add at the end of the file	.fancybox-ie { background-color:#fff; }

After saving the CSS and reloading the page you should be all set and ready to receive your end user congratulations :)

Of course, this is only a start. The next step would be to have the picture retrieved by a Content Query Web Part with the fancybox magic added. It would mean losing the option to download multiple picture and it would allow for an even better rendering.

PowerPivot - Choosing a calculated column type

Lately I have been using PowerPivot (more details on this at the end of this post) and I came across a problem whose solution might interest others.

The Problem

As I created a calculated column based on a date column, the result was always displayed as a number instead of a date. Worst the drop down list where you can choose the type of the column was greyed out.

Here is an example; CalculatedColumn1 is created using some number function (ROUNDDOWN). Yet I am working with Dates, while PowerPivot understandably consider the results to be a number, one can’t override this automatic setting of the Data Type. The “Data Type” DropDown list is just greyed out with a value of Number (Nombre in French in the screenshot below) :

The obvious drawback is that my column isn’t very readable, I mean no normal human beings can understand that 40255 mean 18/03/2010 00:00, else leave me a comment, I would be interested to meet you :)

My solution

Here is my workaround, just throw a date function in your function. In my case I choose to add TIME(0,0,0) which is a neutral operation :

Thus PowerPivot will understand that your column is a date, and you will be able to select a Date format:

Some Information on PowerPivot (aka Gemini)

For those who don’t know what PowerPivot is, you might know by its codename, Gemini. It is a Microsoft Addin for Excel and SharePoint 2010 that allows you to analyze tons of data very easily. It’s supposedly really fast due to its in In Memory Analysis. For BI noobs (a group I am sadly part of) you might see it as a personal version of Anaysis Service (PowerPivot for Excel) or as a Team flavored BI (PowerPivot for SharePoint), on this subject I found this blog post by the PowerPivot Team very interesting (http://blogs.msdn.com/powerpivot/archive/2010/03/12/comparing-analysis-services-and-powerpivot.aspx).

Microsoft released a series of nice videos to explain PowerPivot, here is the first one :

Here is what PowerPivot promises and, as far as my experience went, delivers:

• Analysis (Pivot Table and graph) of Data from multiples sources (DataBase, Excel files, flat files)
• Analysis of huge volume Data (I am talking millions here)
• Integration of simple relational constraint between your data sources
• And maybe the coolest part is that this is available to any Information Worker already familiar with Excel. I would take this one with a grain of salt; your Information Worker needs to be quite technical savvy to fully benefit from this tool.

Additionally the Integration with SharePoint is supposed to bring added performance and to deliver your reports more easily to multiple persons. I haven’t tested the integration, yet!

After spending some quality time with PowerPivot I have got to admit being very enthusiastic about this product. I can see numerous scenarii where it would have saved me hours in the past (More details and cool links at the end of the post). I just can’t wait to set my hands on the RTM version of Excel 2010 and PowerPivot to enjoy this tool without worrying so much about the too frequent crashes (in my experience double clicking on a graph horizontal axis freezes quite often).

To go further I would advise you to start with the official PowerPivot site (http://www.powerpivot.com ), especially the demo part.

To my French reader I would also recommend the hilarious BI video by Têtes à Claques that you might have enjoyed at the 2010 Techdays in Paris :(http://www.microsoft.com/france/serveur/sql/secretedouard/ )

Replacing delegate controls with custom actions

When using jQuery in a SharePoint application, you obviously need to deploy the jQuery javascript file. The deploy part is no big deal, the referencing is a bit more complex. In this post is the detail of a new way to reference the resources, it involves using Custom Action to load external resources such as javascript or css.

Current Solution

As explained by Jan Tielens (http://weblogs.asp.net/jan/archive/2008/11/20/sharepoint-2007-and-jquery-1.aspx there are usually three solutions considered to be available to do that:

• Add a <script src> tag in a Content Editor Web Part
• Add a <script src> tag in the Master Page itself
• Add a <script src> tag dynamically in the <head> using a DelegateControl placed in the standard master pages

In my case, only the third solution was usable. So I went with it, created my feature, added it to my solution.

The problem

The story would have ended here if it wasn’t for a detail I discovered quickly enough; the standard publishing pages (BlueGlassBand and so forth) do not have the AdditionalPageHead delegate control defined. Thus my solution worked really well with collaboration pages but failed on publishing pages.

The obvious solution was to modify the master page used by my client to add the Delegate Control tag but the client didn’t want any modification to the master page. Thus I was at this point :

How do I add a reference to some javascript and css files in the HTML <head> section of a page generated by a master page that doesn’t include any DelegateControl and that I can’t modify.

The solution

After digging a bit I found a very surprising (to me at least) but fully working solution. Use a Custom Action.

Before you start thinking I am nuts, let me explain, If you take a close look at the MSDN you will see that you can specify a ControlAssembly and a ControlClass, guess what, the class specified will do the actual rendering. Custom Actions aren’t just for Site Action Menu entries, ECB and so forth.

So here is more detail:

• Create a class inheriting from CompositeControl
• Override the constructor so that I can plug a method in the Load event.
• In my method triggered on load, access the page header and add a LiteralControl to declare the scripts and css I need

And here is a stripped down but fully working version of my code :

using System; using System.Collections.Generic; using System.Text; using System.Web.UI.WebControls; using System.Web.UI.HtmlControls; using System.Web.UI; namespace ItemsInTheSafeControl { class SiteActionMenu : CompositeControl { public const string mStrIncludedRessources = "<script type=\"text/javascript\" src=\"/_layouts/jquery-1.3.2.min.js\">"; public SiteActionMenu() { this.Load += new EventHandler(SiteActionMenu_Load); } protected void SiteActionMenu_Load(object Sender, EventArgs E) { LiteralControl vLiteralControl = new LiteralControl(SiteActionMenu.mStrIncludedRessources); this.Page.Header.Controls.Add(vLiteralControl); } protected override void Render(HtmlTextWriter Writer) { } } }

To plug this code as a custom action, I use the following ElementManifest

<?xml version="1.0" encoding="utf-8" ?> <Elements xmlns="http://schemas.microsoft.com/sharepoint/"> <CustomAction Id="SAG_CR_Ressources_Deploy_Action" Location="Microsoft.SharePoint.StandardMenu" GroupId="SiteActions" ControlAssembly="SAG.CommentsRating, Version=1.0.0.0, Culture=neutral, PublicKeyToken=406cabeaa3d2932a" ControlClass="SAG.CommentsRating.WebParts.SiteActionMenu"> </CustomAction> </Elements>

Now you just package the ElementManifest in a feature, add your dll and create a solution out of it (wspbuilder is my tool of choice). Your solution manifest will need to specify a SafeControl tag to be added for your class/namespace in the web.config for your CompositeControl.

Here you go, a non-intrusive quite generic solution to plug additional resources into SharePoint.

JSONP your ASP.Net Web Service – Direction to your destination

I was recently developing a Comments & Rating SharePoint solution. As postbacks were forbidden (for good reasons, who loves postbacks ?), we used jQuery with a custom Web Service hosted in a SharePoint web application to try offering a nice experience to end-users.

The custom WebService was to be invoked from pages located on foreign domains. As we had no control over those foreign sites (they weren’t even ASP.Net sites), we had to use JSONP to make it works.

While there is no major difficulty in JSONPing your ASP.net web services even if it is hosted in a SharePoint Web Application, I stumbled upon a couple of problem I wanted to share with you. So let me take you on the road to JSONP in your SharePoint solutions.

Freeway opened - a few turn on the road but no trouble ahead

I will not cover the basics of making your web service JSONP compliant, other bloggers have done a great job at that ( I would recommend Adel Khalil’s and Jason Grundy’s (check the comments) posts among others). Neither will I talk about the basis of JSONP, just let me tell you it is an hack of the HTML script tag intended use that will let you call Web Services from other domains. Other ressources will let you understand why we need this hack (Jonathan Snook cross domain ajax : a quick summary) and what this hack does (Raymond Camden’s article does a great job at this).

There are really very few specifics to SharePoint as far as the Web Service is concerned. My SharePoint solution pushes the asmx file in a subfolder of the Layouts folder along with a web.config specific to this folder that contains the necessary configuration to support JSON (thus JSONP) in the Web Service. The Web Service code behind is deployed in the GAC.

Roadblock #1 – Does size matters ?

Once everything was it in place I started to test the Web Service and everything was going fine. But as I started to push more and data, at one point the web service started not to work anymore. As soon as the Web Service had less data to return it would works again.

To check what was going wrong, I fired up fiddler to check the web service response and saw this :

Jsonp1260899050254({"d":{"__type":"……. );jsonp1260899050254(…ErrorMessage":null}});

As you can see it seemed like my response was cut at one point (always between 16 000 and 17 000 characters) with a ‘)’, then the name of the callback method was added again and finally it was closed correctly.

I don’t know what the problem was exactly. I suspect the the streambuffer used in the HttpHandler suggested by Adel Khalil’s was somewhat full at about 16 000 characters. Thus the compliance method would be run twice.

Being onsite and needing to fix the problem ASAP I didn’t spent time investigating and rushed into finding a solution. I tried to simplify the compliance HttpModule as much as possible and here is what I come up with:

public class JsonHttpModule : IHttpModule { private const string JSON_CONTENT_TYPE = "application/json; charset=utf-8"; public void Dispose() { } public void Init(HttpApplication app) { app.BeginRequest += OnBeginRequest; app.EndRequest += new EventHandler(OnEndRequest); } public void OnBeginRequest(object sender, EventArgs e) { HttpApplication app = (HttpApplication)sender; HttpRequest request = app.Request; //Make sure we only apply to our Web Service if (request.Url.AbsolutePath.ToLower().Contains("MyWS.asmx")) { if (string.IsNullOrEmpty(app.Context.Request.ContentType)) { app.Context.Request.ContentType = JSON_CONTENT_TYPE; } app.Context.Response.Write(app.Context.Request.Params["callback"] + "("); } } void OnEndRequest(object sender, EventArgs e) { HttpApplication app = (HttpApplication)sender; HttpRequest request = app.Request; if (request.Url.AbsolutePath.ToLower().Contains("MyWS.asmx")) { app.Context.Response.Write(")"); } } }

As you can see my HttpModule is doing as little as possible:

• Before generating the response : Add the name of callback from the QueryString parameters and a ‘(‘ to the response if the HttpModule is running for my web Service
• After generating the response : Add ‘);’ at the end of the response if the HttpModule is running for my WebService

Simple enough for my problem to go away, moreover it meant less code to maintain in the future.

If needed, it is possible to add a switch to make it SOAP and straight JSON compliant, if no callback parameter is given it wouldn’t do anything to the response. It would only take a couple more lines.

Roadblocks #2 : Definitely a matter of size

It could have ended like that, a running ASP.Net JSONP compliant Web Service, but it didn’t. Keeping testing the web service with more and more data, I hit another point where the Web Service seemed not to work.

Once again, I called my good friend fiddler for help. It soon became apparent that the server would only return 500 responses.

Http 500 status code means that an internal server error happened. So I went searching through my application logs, SharePoint logs, the Windows logs, … but couldn’t the slightest beginning of a reason for my problem.
Using the debugger I found out that my Web Service methods were called correctly and that no exceptions were thrown by my code.

It was getting weirder and weirder. The response data were correctly generated and returned by my Web Service. I didn’t know exactly what happened but figured that it had to do with the way ASP.Net serialized (‘converted’ if you wish) my response object. As it happened when the size of object increased I started to look for a parameter that would cap the JSONP response size. Such a parameter exists. By default this parameter specify that no response longer than 102 400 character can be serialized to JSONP. For reference you can consult: http://msdn.microsoft.com/en-us/library/bb763183.aspx

It takes place in the web.config as illustrated below:

<configuration> <system.web.extensions> <scripting> <webServices> <jsonSerialization maxJsonLength="102400"/> </webServices> </scripting> </system.web.extensions> </configuration>

And here it is, problem solved.

Yet I have to admit being upset when I found out the solution to my problem, the fact no error messages where logged anywhere I could found them. How are we supposed to diagnose our problem when the only error message is a 500 Http Error code?

Finally arriving

Here I am, I now have a working WSP that deploys my JSONP web service in my SharePoint Web Application. Using jQuery to consume the Web Service it is even possible to use my application from non SharePoint web pages hosted on other web servers.