Thursday 24 October 2013

Build and Package/Publish Behaviour Changes for SharePoint 2013 Projects from Visual Studio 2012 to Visual Studio 2013

I recently moved to a fresh SharePoint Development Farm with Visual Studio 2013 (from an install with VS 2012 and VS 2013 RC) - and noticed that my projects would build but not publish or deploy correctly. When attempting to Publish or deploy certain projects (i.e. to build SharePoint Solution or .wsp files) in Visual Studio 2013, I found that they would fail on the publish step after rebuilding other projects with your usual build failure exceptions:

The type or namespace name "Interfaces" does not exist in the namespace "ProjectName.Repository" (are you missing an assembly reference?)

I tried to see if there were differences in the Project (csproj) structure in Team Foundation Server (TFS)  - as my colleague still had Visual Studio 2012 and could build the project without issue. There were no differences.

To troubleshoot the issue, I turned build logging way up to the Diagnostics Level from the default of Minimum (in Tools > Options > Projects and Solutions > Build and Run > MSBuild project build output verbosity) as seen below:


Once you do this, you will have tens of thousands of lines of errors and information to sift through. A lot of the errors are spurious and are actually part of the normal functioning of MSBuild. One of the first errors that came up related to the ToolSet Version Installed. It was apparently looking for the Visual Studio 2010 Tools (not even the 2012 Toolset from the Project File (csproj) Definition):

1>------ Build started: Project: MyProject.Model, Configuration: Debug Any CPU ------
1>Build started 23/10/2013 6:32:46 PM.
1>Building with tools version "12.0".
1>Project file contains ToolsVersion="4.0". This toolset may be unknown or missing, in which case you may be able to resolve this by installing the appropriate version of MSBuild, or the build may have been forced to a particular ToolsVersion for policy reasons. Treating the project as if it had ToolsVersion="12.0". For more information, please see http://go.microsoft.com/fwlink/?LinkId=293424.
1>Target "_CheckForInvalidConfigurationAndPlatform" in file "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets" from project "C:\src\DDK\BIT\MyProject\trunk\MyProject.Models\MyProject.Model.csproj" (entry point):
1>Task "Error" skipped, due to false condition; ( '$(_InvalidConfigurationError)' == 'true' ) was evaluated as ( '' == 'true' ).
1>Task "Warning" skipped, due to false condition; ( '$(_InvalidConfigurationWarning)' == 'true' ) was evaluated as ( '' == 'true' ).
1>Using "Message" task from assembly "Microsoft.Build.Tasks.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
1>Task "Message"
1>  Configuration=Debug
1>Done executing task "Message".
1>Task "Message"
1>  Platform=AnyCPU
1>Done executing task "Message".
1>Task "Error" skipped, due to false condition; ('$(OutDir)' != '' and !HasTrailingSlash('$(OutDir)')) was evaluated as ('bin\Debug\' != '' and !HasTrailingSlash('bin\Debug\')).
1>Task "Error" skipped, due to false condition; ('$(BaseIntermediateOutputPath)' != '' and !HasTrailingSlash('$(BaseIntermediateOutputPath)')) was evaluated as ('obj\' != '' and !HasTrailingSlash('obj\')).
1>Task "Error" skipped, due to false condition; ('$(IntermediateOutputPath)' != '' and !HasTrailingSlash('$(IntermediateOutputPath)')) was evaluated as ('obj\Debug\' != '' and !HasTrailingSlash('obj\Debug\')).

I tried to replace all the ToolsVersions to the Visual Studio 2013 version (which is ToolsVersion="12.0" - but to no avail.

I also had exceptions pointing to files that didn't exist - like so:
8>Output file "__NonExistentSubDir__\__NonExistentFile__" does not exist.

I checked with the Fusion Log Viewer (fuslogvw from a Visual Studio Command Prompt) - but there were no errors loading assemblies found. I also checked the GAC with gacutil /l and the files definitely got installed there - so that wasn't the issue. I also considered that it may be one of the perennial problems involving permissions - but Visual Studio was running as an Administrator and the Diagnostics Build log showed no permissions errors like this.

Thinking about how the error only occurred during a Publish or Deploy in Visual Studio, I considered that it may be a problem with the SharePoint Package (Package.package) file. Turns out this was right on the money.

Looking at the very end of the Diagnostic Build log:

8>Done executing task "Exec".
8>Done building target "PostBuildEvent" in project "MyProject.Common.UI.csproj".
8>Target "CoreBuild" in file "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets" from project "C:\src\DDK\BIT\MyProject\trunk\MyProject.Common.UI\MyProject.Common.UI.csproj" (target "Build" depends on it):
8>Done building target "CoreBuild" in project "MyProject.Common.UI.csproj".
8>Target "CreateTfsBuildInfoResource" skipped, due to false condition; ( $(AddBuildInfoToAssembly)==true ) was evaluated as ( false==true ).
8>Target "AfterBuild" in file "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets" from project "C:\src\DDK\BIT\MyProject\trunk\MyProject.Common.UI\MyProject.Common.UI.csproj" (target "Build" depends on it):
8>Done building target "AfterBuild" in project "MyProject.Common.UI.csproj".
8>Target "Build" in file "C:\Program Files (x86)\MSBuild\12.0\bin\Microsoft.Common.CurrentVersion.targets" from project "C:\src\DDK\BIT\MyProject\trunk\MyProject.Common.UI\MyProject.Common.UI.csproj" (entry point):
8>Done building target "Build" in project "MyProject.Common.UI.csproj".
8>Done executing task "MSBuild" -- FAILED.
8>Done building target "BuildSharePointProjectReferences" in project "MyProject.ApplicationPages.csproj" -- FAILED.
8>Done executing task "CallTarget" -- FAILED.
8>Done building target "ConditionalPackage" in project "MyProject.ApplicationPages.csproj" -- FAILED.
8>
8>Build FAILED.
8>

When I removed all project assemblies from the Package.Package File (under "Additional Assemblies") that were in my project output (i.e. any non-3rd Party Components), the package would work correctly. Turns out the main difference between Visual Studio 2012 and 2013 is that the order of "Additional Assemblies" defined in a SharePoint package actually makes a difference - and they must be in order of your project dependencies. If they are out of order, MSBuild process (during creation of the wsp only) will not find them in its working directory and spit the dummy.

The fix to resolve the issue then is to make sure all your "Additional Assemblies" are created in correct dependency order - this didn't make a difference during the build process in Visual Studio 2012 but it makes a big difference in Visual Studio 2013.




I suspect this difference in behavior is related to the changes in architecture to MSBuild and its migration from .NET into Visual Studio itself as discussed here -




DDK

Tuesday 8 October 2013

SharePoint 2013 - Bug - Breaking Role Inheritance on Document Library Items within a Synchronous ItemAdded Item Event Receiver will always generate an exception when not a Site Collection Administrator (e.g. a Site Owner) - Fix

This particular issue has caused me a fair amount of grief in the past when developing List Item Event Receivers against Document Libraries (there is no such problem with other List types that don't have Checkouts). I believe it is a bug (or at least a sub-optimal design) in the SaveButton code in SharePoint itself.

It essentially means you cannot change permissions on an item without getting an exception in certain scenarios/use cases like so:



In short, if you (as a non-site collection admin user such as a site owner) perform a BreakRoleInheritance() or any overloads on an item in an ItemAdded Synchronous Receiver against a Document Library in SharePoint 2013, you will always receive the following exception in your SharePoint ULS Logs and the user will see a spurious exception:

 Application error when access /Secured Document Library/Forms/EditForm.aspx, Error=Specified argument was out of the range of valid values. 
  at Microsoft.SharePoint.SPListItemCollection.get_Item(Int32 iIndex)   
  at Microsoft.SharePoint.SPListItem.EnsureItemIsValid()   
  at Microsoft.SharePoint.SPListItem.GetValue(SPField fld, Int32 columnNumber, Boolean bRaw, Boolean bThrowException)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName, Boolean bThrowIfValueMissing, Boolean bThrowIfFieldMissing)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName)   
  at Microsoft.SharePoint.SPListItem.get_File()   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem(SPContext itemContext, Boolean uploadMode, String checkInComment)   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem()   
  at Microsoft.SharePoint.WebControls.SaveButton.OnBubbleEvent(Object source, EventArgs e)   
  at System.Web.UI.Control.RaiseBubbleEvent(Object source, EventArgs args)   

 at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

  Followed by this:

 System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.  
  at Microsoft.SharePoint.SPListItemCollection.get_Item(Int32 iIndex)   
  at Microsoft.SharePoint.SPListItem.EnsureItemIsValid()   
  at Microsoft.SharePoint.SPListItem.GetValue(SPField fld, Int32 columnNumber, Boolean bRaw, Boolean bThrowException)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName, Boolean bThrowIfValueMissing, Boolean bThrowIfFieldMissing)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName)   
  at Microsoft.SharePoint.SPListItem.get_File()   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem(SPContext itemContext, Boolean uploadMode, String checkInComment)   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem()   
  at Microsoft.SharePoint.WebControls.SaveButton.OnBubbleEvent(Object source, EventArgs e)   
  at System.Web.UI.Control.RaiseBubbleEvent(Object source, EventArgs args)   
  at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)


Then this equally entertaining Exception:

Getting Error Message for Exception System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.   
  at Microsoft.SharePoint.SPListItemCollection.get_Item(Int32 iIndex)   
  at Microsoft.SharePoint.SPListItem.EnsureItemIsValid()   
  at Microsoft.SharePoint.SPListItem.GetValue(SPField fld, Int32 columnNumber, Boolean bRaw, Boolean bThrowException)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName, Boolean bThrowIfValueMissing, Boolean bThrowIfFieldMissing)   
  at Microsoft.SharePoint.SPListItem.GetValue(String strName)   
  at Microsoft.SharePoint.SPListItem.get_File()   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem(SPContext itemContext, Boolean uploadMode, String checkInComment)   
  at Microsoft.SharePoint.WebControls.SaveButton.SaveItem()   
  at Microsoft.SharePoint.WebControls.SaveButton.OnBubbleEvent(Object source, EventArgs e)   
  at System.Web.UI.Control.RaiseBubbleEvent(Object source, EventArgs args)   
  at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
  at System.Web.UI.Page.HandleError(Exception e)   
  at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
  at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
  at System.Web.UI.Page.ProcessRequest()   
  at System.Web.UI.Page.ProcessRequest(HttpContext context)   
  at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()   
  at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Let's Debug Some Microsoft Code!

Using various tools such as Redgate's Reflector VS Pro and SQL Profiler to step through SharePoint's code, I managed to find why an exception always occurs after you break inheritance on an item. Reflector was particularly useful as I could step through Microsoft's code and see all the variables as if it was my own.

To get this debugging working, I followed this article on the Redgate site: http://documentation.red-gate.com/display/REF8/Debugging+into+SharePoint+and+seeing+the+locals. Essentially, you need to make 2 changes to debug the SharePoint code:

1) Run regedit from the Run menu HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment and add a string entry of COMPLUS_ZAPDISABLE, with a value of 1


2) To stop the code being optimized away and to see the local variables in Visual Studio, you also need to add an ini file in the SharePoint assembly directory (you can find this out via the modules window when in debug mode) e.g.C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.SharePoint\v4.0_15.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.dll

The ini file should be called Microsoft.SharePoint.ini with the following contents:
[.NET Framework Debugging Control]
GenerateTrackingInfo=1
AllowOptimize=0



Why does this happen?


You will always get an exception (with no way to handle it) because the properties.ListItem and properties.ListItem.File are invalidated by the BreakRoleInheritance() call. This occurs in the Update() call within the SaveButton.cs code in SharePoint 2013. I assume the code was also the same in 2010.

Because a CheckIn is attempted after the Update() code, against the file, the properties.ListItem.File will always throw the exceptions you see above.

You can see this in the code screenshot of the Microsoft SaveButton code below. Because BreakRoleInheritance invalidates the item during the update (because the SharePoint callback SQL code can't find the item), the CheckIn line will always get an exception.



The screenshot below shows the call (which goes to an external COM component, and then to SQL). This callback returns zero records.


The External Call to GetListItemDataWithCallback2():

If you run SQL Profiler at the same time, you will see that the SQL query is too specific and excludes the item on which you just Broke Role Inheritance.


After the call to SQL (which failed to retrieve the item and returns an array of zero length), an index is used on an array assuming that there is a record there. The index is out of the bounds of the array, and so fails with an exception. This is where the invalid index call occurs (during the Save Button
CheckIn() call):



The SQL that returns no records (and why the array is empty) is as below:

exec sp_executesql N'          SELECT t3.[nvarchar12] AS c15c10, t1.[SortBehavior] AS c0, UserData.[nvarchar10], UserData.[tp_ItemOrder], UserData.[nvarchar1], t1.[FolderChildCount] AS c31, t1.[ParentLeafName] AS c36, t2.[nvarchar4] AS c3c6, UserData.[ntext1], UserData.[nvarchar14], t4.[nvarchar6] AS c22c7, UserData.[tp_AppAuthor], t2.[tp_Created] AS c3c11, t3.[nvarchar6] AS c15c7, t1.[ProgId] AS c18, UserData.[nvarchar19], UserData.[tp_AppEditor], t1.[Type] AS c13, UserData.[tp_ID], t1.[ScopeId] AS c21, UserData.[nvarchar5], UserData.[bit1], t1.[ClientId] AS c26, UserData.[tp_GUID], t1.[TimeCreated] AS c1, UserData.[tp_Editor], t2.[nvarchar11] AS c3c9, UserData.[tp_Author], t4.[tp_ID] AS c22c5, t4.[nvarchar3] AS c22c12, t2.[nvarchar1] AS c3c4, t3.[tp_Created] AS c15c11, UserData.[nvarchar13], UserData.[nvarchar18], t1.[CheckinComment] AS c29, t3.[tp_ID] AS c15c5, CASE WHEN DATALENGTH(t1.DirName) = 0 THEN t1.LeafName WHEN DATALENGTH(t1.LeafName) = 0 THEN t1.DirName ELSE t1.DirName + N''/'' + t1.LeafName END  AS c16, UserData.[tp_ContentTypeId], t1.[Size] AS c24, UserData.[tp_WorkflowVersion], t1.[ETagVersion] AS c37, UserData.[nvarchar4], UserData.[tp_CheckoutUserId], UserData.[tp_Version], UserData.[nvarchar9], t4.[nvarchar9] AS c22c8, t5.[nvarchar1] AS c4, UserData.[tp_IsCurrentVersion], t2.[nvarchar6] AS c3c7, UserData.[tp_HasCopyDestinations], UserData.[tp_Level], UserData.[nvarchar12], UserData.[nvarchar17], t4.[nvarchar12] AS c22c10, t2.[nvarchar3] AS c3c12, t1.[TimeLastModified] AS c14, t3.[nvarchar9] AS c15c8, t1.[MetaInfo] AS c19, t1.[Size] AS c27, t1.[ParentVersionString] AS c35, t1.[LeafName] AS c2, UserData.[nvarchar3], UserData.[tp_Modified], UserData.[nvarchar8], t4.[nvarchar4] AS c22c6, UserData.[tp_UIVersion], t1.[ItemChildCount] AS c30, t2.[tp_ID] AS c3c5, t3.[nvarchar3] AS c15c12, UserData.[tp_CopySource], UserData.[nvarchar11], UserData.[nvarchar16], t7.[Title] AS c34c33, UserData.[tp_InstanceID], t2.[nvarchar12] AS c3c10, t3.[nvarchar4] AS c15c6, t1.[IsCheckoutToLocal] AS c17, t1.[LTCheckoutUserId] AS c25, t6.[Title] AS c32c33, UserData.[tp_UIVersionString], t1.[Id] AS c20, UserData.[nvarchar2], UserData.[nvarchar7], t4.[nvarchar11] AS c22c9, t2.[nvarchar9] AS c3c8, UserData.[nvarchar15], t4.[nvarchar1] AS c22c4, t4.[tp_Created] AS c22c11, t3.[nvarchar11] AS c15c9, t3.[nvarchar1] AS c15c4, UserData.[tp_ModerationStatus], UserData.[nvarchar6], UserData.[tp_Created], t1.[DirName] AS c23, UserData.[tp_WorkflowInstanceID] FROM AllUserData AS UserData WITH(FORCESEEK(AllUserData_PK(tp_SiteId,tp_ListID,tp_DeleteTransactionId,tp_IsCurrentVersion))) INNER LOOP JOIN Docs AS t1 WITH(NOLOCK) ON (UserData.[tp_RowOrdinal] = 0) AND (t1.SiteId=UserData.tp_SiteId) AND (t1.SiteId = @SITEID) AND (t1.ParentId = UserData.tp_ParentId) AND (t1.Id = UserData.tp_DocId) AND ( (UserData.tp_Level = 1 OR  UserData.tp_Level =255) ) AND (t1.Level = UserData.tp_Level) AND ((UserData.tp_Level = 255 AND t1.LTCheckoutUserId =@IU OR (UserData.tp_Level = 1 AND (UserData.tp_DraftOwnerId IS NULL) OR UserData.tp_Level = 2)AND (t1.LTCheckoutUserId IS NULL OR t1.LTCheckoutUserId <> @IU ))) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) AND (UserData.[tp_ListID] =@LISTID) AND (UserData.[tp_SiteId] =@SITEID) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_DeleteTransactionId] = 0x ) INNER LOOP JOIN (SELECT CAST(val AS uniqueidentifier) AS InValues FROM dbo.fn_UnpackCsvString(@L4TXP) ) AS Scopes ON (t1.ScopeId = Scopes.InValues) LEFT OUTER LOOP JOIN AllUserData AS t2 WITH(FORCESEEK(AllUserData_PK(tp_SiteId,tp_ListId,tp_DeleteTransactionId,tp_IsCurrentVersion,tp_ID,tp_CalculatedVersion)),NOLOCK) ON (UserData.[tp_Editor]=t2.[tp_ID]) AND (UserData.[tp_RowOrdinal] = 0) AND (t2.[tp_RowOrdinal] = 0) AND ( (t2.tp_Level = 1) ) AND (t2.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (t2.[tp_CalculatedVersion] = 0 ) AND (t2.[tp_DeleteTransactionId] = 0x ) AND (t2.tp_ListId = @L5 AND t2.tp_SiteId = @SITEID) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) LEFT OUTER LOOP JOIN AllUserData AS t3 WITH(FORCESEEK(AllUserData_PK(tp_SiteId,tp_ListId,tp_DeleteTransactionId,tp_IsCurrentVersion,tp_ID,tp_CalculatedVersion)),NOLOCK) ON (UserData.[tp_CheckoutUserId]=t3.[tp_ID]) AND (UserData.[tp_RowOrdinal] = 0) AND (t3.[tp_RowOrdinal] = 0) AND ( (t3.tp_Level = 1) ) AND (t3.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (t3.[tp_CalculatedVersion] = 0 ) AND (t3.[tp_DeleteTransactionId] = 0x ) AND (t3.tp_ListId = @L5 AND t3.tp_SiteId = @SITEID) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) LEFT OUTER LOOP JOIN AllUserData AS t4 WITH(FORCESEEK(AllUserData_PK(tp_SiteId,tp_ListId,tp_DeleteTransactionId,tp_IsCurrentVersion,tp_ID,tp_CalculatedVersion)),NOLOCK) ON (UserData.[tp_Author]=t4.[tp_ID]) AND (UserData.[tp_RowOrdinal] = 0) AND (t4.[tp_RowOrdinal] = 0) AND ( (t4.tp_Level = 1) ) AND (t4.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (t4.[tp_CalculatedVersion] = 0 ) AND (t4.[tp_DeleteTransactionId] = 0x ) AND (t4.tp_ListId = @L5 AND t4.tp_SiteId = @SITEID) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) LEFT OUTER LOOP JOIN AllUserData AS t5 WITH(FORCESEEK(AllUserData_PK(tp_SiteId,tp_ListId,tp_DeleteTransactionId,tp_IsCurrentVersion,tp_ID,tp_CalculatedVersion)),NOLOCK) ON (t1.[LTCheckoutUserId]=t5.[tp_ID]) AND (t5.[tp_RowOrdinal] = 0) AND ( (t5.tp_Level = 1) ) AND (t5.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (t5.[tp_CalculatedVersion] = 0 ) AND (t5.[tp_DeleteTransactionId] = 0x ) AND (t5.tp_ListId = @L5 AND t5.tp_SiteId = @SITEID) LEFT OUTER LOOP JOIN AppPrincipals AS t6 WITH(NOLOCK) ON (UserData.[tp_AppAuthor]=t6.[Id]) AND (UserData.[tp_RowOrdinal] = 0) AND (t6.SiteId = @SITEID) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) LEFT OUTER LOOP JOIN AppPrincipals AS t7 WITH(NOLOCK) ON (UserData.[tp_AppEditor]=t7.[Id]) AND (UserData.[tp_RowOrdinal] = 0) AND (t7.SiteId = @SITEID) AND (UserData.tp_ListId = @L3 AND UserData.tp_SiteId = @SITEID) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_DeleteTransactionId] = 0x ) WHERE (UserData.[tp_CalculatedVersion] = 0 ) AND (UserData.[tp_IsCurrentVersion] = CONVERT(bit,1) ) AND (UserData.[tp_DeleteTransactionId] = 0x ) AND (UserData.tp_ListID=@LISTID) AND (UserData.tp_SiteId=@SITEID) AND ( (UserData.tp_Level = 1 OR  UserData.tp_Level =255)  AND ( UserData.tp_Level= 255 AND UserData.tp_CheckoutUserId = @IU OR  ( UserData.tp_Level  = 2 AND UserData.tp_DraftOwnerId IS NOT NULL OR UserData.tp_Level  = 1 AND UserData.tp_DraftOwnerId IS  NULL  ) AND ( UserData.tp_CheckoutUserId IS  NULL  OR UserData.tp_CheckoutUserId <> @IU))) AND (UserData.tp_RowOrdinal=0) AND ((UserData.[tp_ID] = @II)) ORDER BY t1.[SortBehavior]  DESC ,UserData.[tp_ID]  ASC  OPTION (FORCE ORDER, MAXDOP 1)',N'@LFFP uniqueidentifier,@SITEID uniqueidentifier,@IU int,@L3 uniqueidentifier,@L4TXP nvarchar(4000),@L5 uniqueidentifier,@II int,@LISTID uniqueidentifier,@RequestGuid uniqueidentifier',@LFFP='00000000-0000-0000-0000-000000000000',@SITEID='DC59B5ED-EF2E-4701-89A5-88057D449776',@IU=10,@L3='5E68F1D8-4EF0-4375-9433-246368FC6E2C',@L4TXP=N'{32C47E39-1A7E-4949-9A71-CEBA54BE5AC7},{882F0725-834F-48F0-95F4-FDA490367003},{4AE1520E-ACF9-4014-BDB8-4F0C897ECD6A},{7D93E0FA-FF19-4E3A-A344-378A5E8DFD4E},{5DC69BCF-2778-4AFD-B4E9-FCAAC84AA40F},{5FCA1571-0643-4467-8067-9E24CA271EBE}',@L5='70002672-8F36-4F40-BB22-7AD9D75923F9',@II=332,@LISTID='5E68F1D8-4EF0-4375-9433-246368FC6E2C',@RequestGuid='8F43499C-ADFE-1030-7AAF-EE92CFD86881'

The Problem In Summary:
1) Your BreakRoleInheritance() call in code makes another call to the SharePoint stored procedure proc_SecRemovePrincipalFromScope
2) proc_SecRemovePrincipalFromScope updates values that are specifically filtered for in the SQL generated by SaveButton.cs for the properties.ListItem.File call
3) Because the query returns an empty array, you will get an out of index exception - even if you try
the properties.InvalidateListItem() or properties.InvalidateWeb() to force a reload.


The Fix:

If you look at the decompiled SharePoint Save Button code, you may notice that there is a Non-fatal error handler at the end. You can essentially short-circuit the CheckIn() call (that is causing the exceptions against the "missing" object in the empty array) by setting the SPItemEventProperties Event Receiver to SPEventReceiverStatus.CancelNoError in the CheckingIn() override like so:
        /// 
        /// An item is being checked in.
        /// 
        public override void ItemCheckingIn(SPItemEventProperties properties)
        {
            //For document libraries, need to do Role Inheritance on Checking in.
            ProcessMetadataChange(properties, SPEventReceiverType.ItemCheckingIn);
            properties.Status = SPEventReceiverStatus.CancelNoError;
            //base.ItemCheckingIn(properties);
        } 

You then do the CheckIn() call in the code yourself rather than rely on SharePoint to do it for you. You can see our entry point in the screenshot below - which means we can Cancel the CheckIn Event and effectively prevent the code that causes exceptions against the expired/invalidated objects in SaveButton from being processed:


When we set the properties.Status to CancelNoError, the code jumps to this soft goto handler and doesn't throw an exception like so:



DDK

Friday 21 June 2013

SharePoint 2013 - Custom Field Type Controls - How can I use the JSLink Property for Rendering in the List View, but still Use Server Controls (e.g. Telerik Controls) for Edit and Display View?

You may know that in SharePoint 2013, you can now control how a field renders completely by using JavaScript via the new JSLink Property on field controls. An example of this can be found on MSDN at "How to: Extend the Geolocation field type using client-side rendering". What if you want to do JavaScript rendering for just the list view - and use fully featured controls for editing and item display?

This is not documented anywhere (I found this by trial and error and some help from .NET Reflector - http://www.red-gate.com/products/dotnet-development/reflector/) - but you can actually have both the JSLink render your field in your SharePoint list view and take full advantage of server rendering of controls (e.g. using Telerik AutoComplete for Token Selection) at the same time.

You can do this by overriding the JSLink property in your Field class (inheriting from SPField) as below so that it renders your full controls for Display/Edit and renders your JSLink for viewing in a list:

   public override string JSLink
        {
            get
            {
                if (SPContext.Current != null && SPContext.Current.FormContext.FormMode == SPControlMode.Invalid)
                    return "../_layouts/15/MyProjectName/CustomFieldControl.js";
                else
                {
                    return string.Empty;
                }
            }
            set
            {
                base.JSLink = value;
            }
        }

As above, this looks at the current SPContext to determine whether it is currently rendering in the list view (and renders the JSLink property) - otherwise, it just uses the default server control rendering behaviour.

DDK

SharePoint 2013 - Why do my Custom Field Type Controls Seem to be Cached and keep Loading Up An Old Version? Some Gotchas for SharePoint Custom Field Types

When creating custom Field Types in SharePoint 2013 (see http://msdn.microsoft.com/en-us/library/jj901637.aspx for examples), there are several steps you need to take to register it with SharePoint. These are all convention-based.

  1. You must create a file called fldtypes_MYFIELDNAMEField.xml e.g. fldTypes_ParentField if you want to add a field called "Parent". SharePoint will pick this field up based on this naming convention for your file.
  2. You need to deploy a user control (e.g. ParentFieldControl.ascx) to the SharePoint HIVE Control Templates directory (it will not work in a subdirectory).
  3. Within the ascx files, you should Specify a rendering Template like so:
<SharePoint:RenderingTemplate ID="ParentFieldControl" runat="server">
    <Template>
        <div class="customFieldType">
        <telerik:RadAutoCompleteBox runat="server" ID="autoCompleteTextBox" CssClass="customFieldControlParentGroupList" Filter="Contains" DropDownHeight="400" DropDownWidth="375" InputType="Token" Delimiter='<%# Constants.DefaultDelimiter %>' DataTextField="Text" ToolTip='<%# DataBinder.Eval(Container.DataItem, "Description")%>' DataValueField="Value">
            <DropDownItemTemplate>

Problem is, I did this and deployed my solution and my changes were not being propagated when my updated custom field was rendered within SharePoint. I tried a few things:

  1. Deleting the field from all lists and the Site fields collection in SharePoint with SharePoint Designer and/or the free tool SharePoint Manager 2013
  2. Removing the Custom Field
  3. Rebooting
Yet my changes weren't being reflected when I deployed the new version. What was going on? Was it some kind of weird caching that persisted between reboots?

What is not clear from the documentation is that the Rendering Template Id MUST be unique between all controls that you deploy as a custom field type. I was REUSING the same DefaultTemplateName and DisplayTemplateName between all controls (following the DRY principle).

I originally thought that it was loading up my control based on the NAME of the ascx control e.g. fieldTypes_ParentField would (by convention) load up the ParentFieldControl.ascx file to perform rendering for that field. This is NOT the case. Instead, the when you override the "DefaultTemplateName" or "DisplayTemplateName" properties on "Microsoft.SharePoint.WebControls.BaseFieldControl", you must ensure that these values have not been used elsewhere by custom fields in your project - otherwise SharePoint will keep picking up the first control it finds with a matching "DefaultTemplateName" or "DisplayTemplateName".

Now because I was reusing the same Template Name between all controls within my ascx files, it was rendering the first control it happened to find - without throwing an exception!

Be warned.

DDK

Thursday 13 June 2013

Outlook 2013 Bug when using IMAP - Beware the Bandwidth Blowout!

We recently had issues with our work internet connection that meant sizeable amounts of data were being chewed up without our users doing anything in particular or untoward.

Using the bandwidth monitor utility Networx (http://www.softperfect.com/products/networx/) (which breaks accumulated traffic down by process), we specifically found that Outlook 2013 was downloading large amounts of data (40-50MB) - many (but not all) times that a "Send and Receive" synch was performed. This was using IMAP running against the open source DoveCot (http://www.dovecot.org/) as a mail server.

Now by default, this is set to every 30 minutes. If a machine is on 24 hours day, this can add up to 2.4GB per user - which clogs up your network and really adds up on anything but an unlimited plan!

Cracking open Wireshark (http://www.wireshark.org/), I turned off SSL to see what was going on in terms of the IMAP commands going down the wire and the full extent of the download. Nothing stuck out apart from the fact that setting the "Mail to keep offline setting" to 24 months as below performs an IMAP "UID SEARCH SINCE 13-Jun-2011" to filter results. This is as expected.


To my surprise, with the default "All" Setting for "Mail to keep offline", you can also see that on regular but apparently random occasions every single email (including ALL content and ALL binaries) was being downloaded every time from the mail server - regardless of whether Outlook already had a locally cached version or not. Sometimes, the download will only be 5MB per click of "Send and Receive All". I tried to reproduce it and it tends to happen when you save Drafts or send/receive large items in your inbox - but I wasn't able to isolate the issue through a process of elimination.

This may well be part of the OST corruption issues mentioned here - the IMAP issues in Outlook 2013 are apparently high on the MS internal support agenda:
http://productforums.google.com/forum/#!topic/gmail/9alMlaOkMC4

In terms of the mail server, the installed version of Dovecot also seems to support LIST-EXTENDED for special folder support introduced in Outlook 2013 - so that doesn't appear to be the problem (unless there is a variation on the standard between Outlook/the IMAP server - which is not impossible).

So no clear conclusions I'm afraid. The one thing that is clear is that there are problems with Outlook 2013 and IMAP - based on the large number of problems I've seen floating around the web. The above problem is just one example of these. I've also seen that Outlook 2013 sometimes doesn't download new emails at all (even though other clients such as Android download from the same mail server correctly.). In addition, when working with drafts and saving the draft email repeatedly, Outlook will not be able to save it and say that it must be saved as a new copy within your Drafts folder.

Apparently, a fix to some of these problems should be released around August 2013 (as per http://productforums.google.com/forum/#!topic/gmail/9alMlaOkMC4)

In the meantime, to fix the heavy bandwidth usage issue, I can only suggest:
  1. Roll back to Outlook 2010 if you're using IMAP . This seems to be a popular option - although you cannot have Outlook 2013 and 2010 running simultaneously (unlike other MS Office Products).  [Recommended - this worked in my situation and reduced synch bandwidth usage from 50MB to 850KB]
  2. Perform a full initial synchronisation with the default "All" setting. Then reduce your "Mail to keep offline" time window to a smaller figure after this initial synchronization (e.g. 1 month) 
  3. Delete your local OST file and set up email again. This may give you temporary relief from the problem - but it wasn't a permanent fix for me.
  4. Use yet another client like Thunderbird - although this apparently also has issues with non-appearing folders with some IMAP providers as well.
  5. Reduce your synchronisation frequency (to something less than 30 minute full synchs). 
  6. Update your mail server if possible to ensure full compliance with standards. This isn't an option for me. You should also confirm this with a pilot/Proof of Concept to confirm that it fixes your specific issue.
  7. Clean out/auto-archive your emails!

DDK

Saturday 1 June 2013

DDK's Trip to Singapore and Malaysia 2013 - "The All-You-Can-Eat Chilli Crab and Stingray Event"

We just returned from our trip to Singapore and Malaysia after a particularly long and arduous flight back. There wasn't time for rest at all and we spent most of the time on our feet! Some of the more memorable events of the trip:
  1. Swimming on top of the Marina Bay Sands Hotel ("aka The Sands SkyPark") in their Infinity Pool - 59 Levels above the city. The feeling of floating above the city was a little disorienting at first. The spa also had spectacular views out to the 100+ ships at sea off the coast of Singapore.





  2. The light shows at Gardens By the Bay were quite spectacular (i.e. the kids loved it!).



  3. Visiting The Sentosa Island (Singapore) Adventure Cove Water Park - which has a river run that takes you around the park and through tanks of Stingrays and Sharks along the way. The obstacle course, the snorkel pools and wave park were also quite good value for the kids.

  4. Getting hungry after that and eating some Stingrays in the food market in Kuala Lumpur in Malaysia.
  5. Zach being sick on the floor of KL airport twice just before boarding closed - and then having the 300 people on the plane wait for a quarantine inspector at Sydney Airport so they could inspect him for avian flu.
  6. Trying some real Singapore Chilli crab and "Monster Fish" - one of the spiciest fish dishes I've ever had (and I lather tabasco on most meals normally). Kids were also impressed with my ability to suck out fish eyeballs - what a delicacy! The sweet Coffee Tapioca (covered in Coconut) and Taro breakfast treats were also an excellent start to a day on the road.
  7. Walking up the 272 steps of the Batu Caves and seeing the massive Gold Murugan Statue at the base of the steps. The light filtering through the roof of the caves creates a wonderfully magical atmosphere when standing inside.

  8. Singapore Slings @ Raffles Hotel that gave Lisa a big hangover that she is still recovering from!

  9. Going up the Petronas Towers - towering 452 metres above Kuala Lumpur.
  10. Heidi going on the Genting Highlands Horror ride in Malaysia. She said it was "so boring" at the start and was crying by the end of it. Zach loved the whole ride and wanted to shoot more werewolves and zombies by the end of it. This distracted him from the fact that he had just received a massive bump in the middle of his forehead that (even after copious amounts of ice) made him look like a mini-unicorn.
  11. The crazy bathrooms in the Star Hill Shopping Centre in Kuala Lumpur - all made of piled slate. There was a man who would pump an old waterwheel when you needed to wash your hands. A bit creepy I have to say. Zach got a fright and insisted that the man only had one arm. Crrreeeeepy!

For more pictures of our holiday, see http://www.ddkonline.com/Gallery/2013_05_May_Singapore_Malaysia/gallery.html
DDK

Friday 3 May 2013

SharePoint 2013 - Delegate Controls - Beware of throwbacks to the SharePoint 2010 14 Hive

Delegate controls are particularly useful in SharePoint when you want to add functionality to all pages within your site without changing the master page. For example, if you want to add an external JavaScript library (such as JQuery) to every page, you can do this easily by registering a delegate control. An example of this approach can be seen here: http://blogs.msdn.com/b/kaevans/archive/2011/04/06/adding-jquery-to-every-page-in-sharepoint-with-delegate-controls.aspx

A delegate control is registered through an Elements.xml with a ControlSrc property pointing to your custom delegate control.
<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Control Id="AdditionalPageHead"
           Sequence="100"
           ControlSrc="~/_controltemplates/MyAppName.Core/MyDelegateControl.ascx" />
</Elements>
Problem is, the above code will work in SharePoint 2010 but NOT in SharePoint 2013. You will get an exception like the following:
DelegateControl: Exception thrown while building custom control 'Microsoft.SharePoint.SPControlElement': System.Web.HttpException (0x80004005): The file '/_controltemplates/MyAppName.Core/MyDelegateControl.ascx' does not exist.    
 at System.Web.UI.Util.CheckVirtualFileExists(VirtualPath virtualPath)    
 at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)    
 at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate)    
 at System.Web.Compilation.BuildManager.GetVPathBuildResult(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean ensureIsUpToDate)    
 at System.Web.UI.TemplateControl.LoadControl(VirtualPath virtualPath)    
 at Microsoft.SharePoint.Utilities.SPUtility.CreateUserControlFromVirtualPath(TemplateControl tctlPage, String sControlPath)    
 at Microsoft.SharePoint.SPControlElement.BuildCustomControl(TemplateControl tctlPage, String sControlAssembly, String sControlClass, String sControlSrc, XmlNode xnElementDefinition, SPFeatureDefinition featdefElement, String sElementId)    
 at Microsoft.SharePoint.WebControls.DelegateControl.BuildCustomControlResilient(SPControlElement ctlelemDefinition)

The control itself DOES exist - so this is a somewhat misleading exception. Looking at this issue more closely, SysInternals ProcessMon will show you the way:



Looking closely, it is actually referencing the 14 Hive rather than the 15 Hive as a throwback to SharePoint 2010. So to fix this, you simply need to update your ControlSrc to include 15 in the path:
<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <Control Id="AdditionalPageHead"
           Sequence="100"
           ControlSrc="~/_controltemplates/15/MyAppName.Core/MyDelegateControl.ascx" />
</Elements>
As an alternative, you could also use a Module and a custom action for registering a ScriptLink to something like jQuery or knockout.js.

DDK

Monday 25 March 2013

SharePoint 2013 - Fix for Exception - Microsoft.SharePoint.SPException: Exception occurred. (Exception from HRESULT: 0x80020009 (DISP_E_EXCEPTION))

I received the following exception when creating new custom field types in SharePoint 2013. I couldn't even open the List Settings page (via the Web UI, SharePoint Designer or SharePoint Manager).

To resolve, you simply have to rename any potentially problematic custom field types e.g. rename fldtypes_MyCustomField.xml to zzfldtypes_MyCustomField.xml". You can find your field types located in %Program Files%Common Files\microsoft shared\Web Server Extensions\15\TEMPLATE\XML. Restart IIS after each rename.

This will get you back up and running - and you will now know what custom SharePoint field type is causing the issue.

Full Exception Details as below:

Getting Error Message for Exception System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> Microsoft.SharePoint.SPException: Exception occurred. (Exception from HRESULT: 0x80020009 (DISP_E_EXCEPTION)) ---> System.Runtime.InteropServices.COMException: Exception occurred. (Exception from HRESULT: 0x80020009 (DISP_E_EXCEPTION))   
at Microsoft.SharePoint.Library.SPRequestInternalClass.GetFieldTypeInfo(String bstrUrl, Boolean bInternal, Object& pvarTypeNames, Object& pvarTypeDisplayNames, Object& pvarTypeShortDescriptions, Object& pvarBaseRenderingTypeNames, Object& pvarFieldTypeClasses, Object& pvarFieldEditorUserControls, Object& pvarPropertySchema, Object& pvarTypeInfoFlags)   
at Microsoft.SharePoint.Library.SPRequest.GetFieldTypeInfo(String bstrUrl, Boolean bInternal, Object& pvarTypeNames, Object& pvarTypeDisplayNames, Object& pvarTypeShortDescriptions, Object& pvarBaseRenderingTypeNames, Object& pvarFieldTypeClasses, Object& pvarFieldEditorUserControls, Object& pvarPropertySchema, Object& pvarTypeInfoFlags) -
-- End of inner exception stack trace ---   
at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)   
at Microsoft.SharePoint.Library.SPRequest.GetFieldTypeInfo(String bstrUrl, Boolean bInternal, Object& pvarTypeNames, Object& pvarTypeDisplayNames, Object& pvarTypeShortDescriptions, Object& pvarBaseRenderingTypeNames, Object& pvarFieldTypeClasses, Object& pvarFieldEditorUserControls, Object& pvarPropertySchema, Object& pvarTypeInfoFlags)   
at Microsoft.SharePoint.SPFieldTypeDefinitionCollection..ctor(SPWeb web, Boolean internalCall)   
at Microsoft.SharePoint.SPField.GetFieldTypeDefinitionForType(SPFieldCollection fields, String strType)   
at Microsoft.SharePoint.SPField.get_FieldTypeDefinition()   
at Microsoft.SharePoint.SPField.get_TypeDisplayName()   
at ASP._layouts_15_listedit_aspx.__Render__control13(HtmlTextWriter __w, Control parameterContainer)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at Microsoft.SharePoint.WebControls.AjaxDelta.RenderChildren(HtmlTextWriter output)   
at System.Web.UI.WebControls.WebControl.RenderContents(HtmlTextWriter writer)   
at System.Web.UI.WebControls.WebControl.Render(HtmlTextWriter writer)   
at Microsoft.SharePoint.WebControls.AjaxDelta.Render(HtmlTextWriter writer)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)   
at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)   
at Microsoft.SharePoint.WebControls.SharePointForm.Render(HtmlTextWriter output)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)   
at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.RenderChildren(HtmlTextWriter writer)   
at System.Web.UI.Page.Render(HtmlTextWriter writer)   
at Microsoft.SharePoint.WebControls.DeltaPage.RenderToBase(HtmlTextWriter writer)   
at Microsoft.SharePoint.WebControls.DeltaPage.Render(HtmlTextWriter writer)   
at Microsoft.SharePoint.WebControls.UnsecuredLayoutsPageBase.Render(HtmlTextWriter writer)   
at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)   
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
at System.Web.UI.Page.HandleError(Exception e)   
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)   
at System.Web.UI.Page.ProcessRequest()   
at System.Web.UI.Page.ProcessRequest(HttpContext context)   
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()   
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

DDK

Saturday 23 March 2013

Microsoft Licensing - Can I Purchase Microsoft Software from overseas and use it here in Australia?

With the Federal Government inquiry into the inflated prices of software in Australia, it has become quite clear (officially) that there is a price differential between Australia and most of the world (especially the US). When the CEOs (such as Pip Marlow from Microsoft) were asked why the prices are different, it basically comes down to the fact "because we can". When I did commerce at UNSW, this was known in palatable terms as "extracting the consumer surplus through price discrimination".

The enormity of the price difference became clear when we decided to purchase Microsoft Visual Studio 2012 with MSDN here in Oz.

As per http://msdn.microsoft.com/en-au/subscriptions/buy.aspx

Australia - $21,202 (AUD)
USA - $13,299 (USD)



It has been suggested by some (including Choice Magazine no less) that it would be much cheaper to send someone over to the USA in luxury, purchase it and then bring it back home (if that were neccessary).

However, this would then not be in compliance with the licensing agreement as outlined at http://www.microsoft.com/licensing/licensing-options/open-regional.aspx. The line that specifically prohibits you purchasing from another region (such as the USA) is "Customers and resellers must place agreements and orders with partners within the same defined region."
 
So if you even attempt to purchase from a buyer outside your region and ship it back in, you would be outside the terms of the agreement and have unlicensed and illegal software. This got me thinking laterally within the bounds of our licensing region. Looking at other options within the APAC (Asia Pacific) region, the prices are as follows:
Country of Purchase Amount (Local Currency) Amount in AUD
Korea ₩15,730,600.00 $13,461.00
Brunei
$17,814.00
$13,653.00
Singapore $17,814.00 $13,653.00
Malaysia RM46,081.00 $14,196.00
Thailand ฿451,605.00 $14,763.00
Phillipines Php643,970.00 $15,113.00
New Zealand $26,369.00 $21,040.00
New Caledonia $13,299.00 $12,754.00
Australia
$21,202.00
$21,202.00
Indonesia
$15,294.00
NA - Rupiah Not Provided
Vietnam $15,294.00 NA - VND not provided

So you can save $7,741 by purchasing the same product from New Caledonia as opposed to Australia.

Looking into the License terms and conditions for MSDN with Visual Studio Ultimate specifically, there is a list of "General Terms and Conditions" that can be found here:
http://www.microsoftvolumelicensing.com/ProductPage.aspx?pid=422. More specific product use rights can be found here: http://www.microsoftvolumelicensing.com/userights/Downloader.aspx?DocumentId=6131

Conclusion:
1) There is nothing specifically in the January 2013 Microsoft Product Usage Rights document about purchasing a product from another country (inside or outside your licensing region) and using it elsewhere.
2) There is only a subtle hint on the "Microsoft Open Programs Affiliate Participation Policy" page that you shouldn't be contacting partners outside the APAC licensing region (say an Australian going to the US) to purchase your software.

To be completely in the clear, you can purchase from somewhere like New Caledonia (or Korea) and save yourself in excess of $8K. This is provided it's OK that:
1) You have someone at the other end who can receive your invoice for you (or you will fly/kayak/swim over there to receive it) - or you don't care about the invoice at all.
2) You don't need the DVDs.

For $8K, many would consider these to be small sacrifices indeed!

[Update 18/07/2013] - I've had a few emails about this post. We ended up purchasing the AU licenses as it was seen as too much of a risk on a large purchase to have a potentially unsupported product. Microsoft also called us up to confirm we were who we said we were - so if you desperately want to save money and still have a legitimate MSDN license, I recommend you set up both postal and phone redirection!

DDK