Fix - Duet Enterprise/SharePoint 2010 Exception - "System.ServiceModel.QuotaExceededException: The size necessary to buffer the XML content exceeded the buffer quota."

When attempting to establish communications between SAP and SharePoint (using the Duet Enterprise Claims provider) today at a new client, I encountered the following exception in SharePoint (as per the ULS logs):

InnerException 1: System.ServiceModel.QuotaExceededException: The size necessary to buffer the XML content exceeded the buffer quota. Server stack trace:

at System.ServiceModel.Channels.BufferedOutputStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
at System.Xml.XmlStreamNodeWriter.FlushBuffer()
at System.Xml.XmlBinaryNodeWriter.FlushBuffer()
at System.Xml.XmlStreamNodeWriter.GetBuffer(Int32 count, Int32& offset)
at System.Xml.XmlBinaryNodeWriter.UnsafeWriteText(Char* chars, Int32 charCount)
at System.Xml.XmlBinaryNodeWriter.WriteText(Char[] chars, Int32 offset, Int32 count)
at System.Xml.XmlBaseWriter.WriteChars(Char[] chars, Int32 offset, Int32 count)
at System.Xml.XmlBinaryWriter.WriteTextNode(XmlDictionaryReader reader, Boolean attribute)
at System.Xml.XmlDictionaryWriter.WriteNode(XmlDictionaryReader reader, Boolean defattr)
at System.ServiceModel.Channels.ReceivedFault.CreateFault12Driver(XmlDictionaryReader reader, Int32 maxBufferSize, EnvelopeVersion version)
at System.ServiceModel.Channels.MessageFault.CreateFault(Message message, Int32 maxBufferSize)
at System.ServiceModel.Channels.SecurityChannelFactory`1.ClientSecurityChannel`1.TryGetSecurityFaultException(Message faultMessage, Exception& faultException)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown
at [0]:
at Microsoft.SharePoint.BusinessData.SystemSpecific.Wcf.WcfSystemUtility.Execute(Object[] args)
at Microsoft.SharePoint.BusinessData.SystemSpecific.Wcf.WcfSystemUtility.ExecuteStatic(IMethodInstance methodInstance, ILobSystemInstance lobSystemInstance, Object[] args, IExecutionContext context)
at Microsoft.SharePoint.BusinessData.Runtime.DataClassRuntime.ExecuteInternalWithAuthNFailureRetry(ISystemUtility systemUtility, IMethodInstance methodInstanceToExecute, IMethod methodToExecute, ILobSystemInstance lobSystemInstance, ILobSystem lobSystem, IParameterCollection nonReturnParameters, Object[] overrideArgs)
at Microsoft.SharePoint.BusinessData.Runtime.DataClassRuntime.ExecuteInternal(IDataClass thisDataClass, ILobSystemInstance lobSystemInstance, ILobSystem lobSystem, IMethodInstance methodInstanceToExecute, IMethod methodToExecute, IParameterCollection nonReturnParameters, Object[]& overrideArgs)

I also received the following exception immediately after the above exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ServiceModel.QuotaExceededException: The size necessary to buffer the XML content exceeded the buffer quota. Server stack trace:

at System.ServiceModel.Channels.BufferedOutputStream.WriteCore(Byte[] buffer, Int32 offset, Int32 size)
at System.Xml.XmlStreamNodeWriter.FlushBuffer()
at System.Xml.XmlBinaryNodeWriter.FlushBuffer()
at System.Xml.XmlStreamNodeWriter.GetBuffer(Int32 count, Int32& offset)
at System.Xml.XmlBinaryNodeWriter.UnsafeWriteText(Char* chars, Int32 charCount)
at System.Xml.XmlBinaryNodeWriter.WriteText(Char[] chars, Int32 offset, Int32 count)
at System.Xml.XmlBaseWriter.WriteChars(Char[] chars, Int32 offset, Int32 count)
at System.Xml.XmlBinaryWriter.WriteTextNode(XmlDictionaryReader reader, Boolean attribute)
at System.Xml.XmlDictionaryWriter.WriteNode(XmlDictionaryReader reader, Boolean defattr)
at System.ServiceModel.Channels.ReceivedFault.CreateFault12Driver(XmlDictionaryReader reader, Int32 maxBufferSize, EnvelopeVersion version)
at System.ServiceModel.Channels.MessageFault.CreateFault(Message message, Int32 maxBufferSize)
at System.ServiceModel.Channels.SecurityChannelFactory`1.ClientSecurityChannel`1.TryGetSecurityFaultException(Message faultMessage, Exception& faultException)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)
at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown
at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at BCSServiceProxy.IWXManageCustomerIn.FindCustomerByElements(FindCustomerByElementsRequest request)
at BCSServiceProxy.WXManageCustomerInClient.BCSServiceProxy.IWXManageCustomerIn.FindCustomerByElements(FindCustomerByElementsRequest request)
at BCSServiceProxy.WXManageCustomerInClient.FindCustomerByElements(BPCCustGetAll BPCCustGetAll) -
-- End of inner exception stack trace ---
at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Microsoft.SharePoint.BusinessData.SystemSpecific.Wcf.WcfSystemUtility.Execute(Object[] args)

With my experiences with SAP, buffer overflows normally indicate that a large Java stack trace is coming back from SAP and it overloads the WCF client buffer which is expecting a normal SOAP response.

To see the real exception, I tried to use Wireshark (http://www.wireshark.org/)- but our setup had (not by choice) Netweaver and SharePoint on the same box - and Wireshark cannot listen to localhost traffic.

As usual, the Fiddler Tool HTTP proxy (http://www.fiddler2.com/fiddler2/) came to the rescue- as it can actually listen to traffic occurring between applications on the same machine. When running Fiddler and listening to the https traffic, it popped up with an exception regarding certificate errors:

Session #24: The remote server (ausyd-a-sh1) presented a certificate that did not validate, due to RemoteCertificateChainErrors.

SUBJECT: CN=ausyd-a-sh1, OU=I0020310622, OU=SAP Web AS, O=SAP Trust Community, C=DE
ISSUER: CN=ausyd-a-sh1, OU=I0020310622, OU=SAP Web AS, O=SAP Trust Community, C=DE
EXPIRES: 1/01/2038 11:00:01 AM
I went to the server url in Internet Explorer and sure enough there was a certificate error occurring when going to the URL of the SAP server - at https://ausyd-a-sh1:8001/

Basically this exception was occurring because the SAP SSL certificate was not in the Trusted Root Certification Authorities store in Windows. To import it, I just opened mmc at a command prompt and added the certificates Snap-In (via File - Add/Remove Snap In... - for the Local Computer.

In Certificates (Local Computer), I went to Trusted Root Certification Authorities - Certificates and Imported the SAP Certificate in.

However, I kept getting the same exception. On closer inspection, the thumbprint of the SharePoint Security Token Service (STS) certificate was not the same as what had been imported into SAP - this was becuase SharePoint had been reinstalled - which changes the SharePoint STS certificate (i.e. SharePoint STS certificates are install specific).

I then received the following Exception - which was related to user mappings in SAP:

An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail

To resolve this last exception:

1) We made sure SharePoint:: was used to prefix the name in SAP (in the VUSEREXITID table) - this prefix would normally be added through the Duet Active Directory user import job (specific to Duet Enterprise) - but we don't have AD in our environment.
2) We then updated the STS certificate in the SAP Certificate store (using SAP transaction /nstrust)

3) Used the /nsaml2 transaction to update the certificate used there as well.

This resolved all our SAP to SharePoint communication issues.

DDK

Duet Enterprise Released and Available for Download on MSDN!

After a long wait, Duet Enterprise 1.0, Microsoft and SAP's premier framework for SAP integration with SharePoint 2010 and Office 2010 has finally been released on MSDN. At Oakton, I've been doing SAP to SharePoint integration for the last 18 months so a framework and guidance like this is quite exciting.

If you have an MSDN subscription, you can get it from here:
https://msdn.microsoft.com/en-us/subscriptions/securedownloads/default.aspx
Also note that this is not the ONLY way to integrate and alternatives like just using out of the box SharePoint 2010 BCS (Business Connectivity Services), or the Biztalk SAP Adapters Packs are the way to go. It really depends on your requirements. Duet Enterprise provides a few sample scenarios as part of the framework to get you started as a model for integration guidance.

Here is a high level architecture diagram in case you're completely unfamiliar with the product (it requires and utilizes SharePoint 2010 BCS under the covers) :

Basically, Duet Enterprise gives you a head-start in terms of plumbing when you start a new SAP-SharePoint Integration project. It includes the following:

1. A framework for building SAP-SharePoint-Office Integration Scenarios
2. SAP Workflow Integration from SharePoint (though this is not using SAP BPM)
3. Some Pre-canned Site Templates for certain scenarios like retrieving Customer Information through SharePoint
4. Components to allow retrieval of SAP reports from SharePoint
5. Duet Enterprise Document collaboration functionality
6. Communication Monitoring and Support Components to make troubleshooting easier.
7. A security framework for authorization purposes.

There are a couple of known issues with the release (not absolute showstoppers for my clients, but you should be aware of them):

Known Issue (1):


Aggregator Web Part will not work across the Web Apps having different Authentication
Description: Aggregator Web Part is deliverable of the Duet Workflow Feature. This Web Part will provide a support to view the Task for particular user across the Site Collection and Web App.


Current Limitation to the web Part - This Web Part cannot support the Scenario of Web Part is created in one type of authenticated Web App and the Rootsite Parameter pointing to the different Web App which if different authentication. Eg: Web Part placed in Win Claims based Web App can pull/show the task assigned to the User under another rootsite created in another win claim Web App but it will not work for the rootsite created in Win Legacy Web App and then the Page on which Web Part is placed will not accessible.


Viewing the Tasks from the Cross Auth Web App in aggregator web part is not Supported


Workaround: To Fix the Page which is caused by cross authentication issue in Aggregator Web Part in the Page is to remove the web Part through SPD and then add the Aggregator web Part and point only to the rootsite accessible and same authenticated.


Known Issue (2):
Task with attachments cannot be created and assigned to users in Turkish environment.


The following languages are included in this product: Arabic, Chinese-Simplified, Chinese-Traditional, English, French, German, Hebrew, Italian, Japanese, Portuguese-Brazil, Russian, Spanish, and Turkish.


Microsoft will be having some official launch announcements soon and I'll be covering Duet in more detail in later blog posts.

DDK

List of Permissions Required for Deployment and Activation of Solutions/Features on a SharePoint Site

Today I had some issues today with deployment of SharePoint Solutions/Features to one of the development servers. I knew you need both database and site permissions - but wasn't clear on exactly what databases I needed permissions to. Here are the permissions that you need to deploy a solution/feature to SharePoint:

1. Farm Administrator (add this through Central Admin)
2. Site Collection Administrator (Done through the Individual Site Collection itself)
3. db_owner and db_securityadmin permissions on the Central Admin Content database (note that this is normally generated with a Guid in the name e.g. SharePoint_AdminContent_XXXX-XXX-XXX-XXXX.
4. db_owner permissions on the Content database for your site e.g. DEV_MachineName_WSS_Content_Portal_5000_0
5. Local admin rights on the server itself.

Steps for Setting up an SAP Enterprise Service to be consumed from an external application such as SharePoint

1. Go to SAP GUI
2. Run transaction /nsproxy. This will show what services are available to expose as endpoints, including your own custom created services (otherwise you have to download or create it).

3. In transaction /nsproxy, search for your service by clicking the binoculars icon e.g. the EmployeeBasicDataByEmployeeQueryResponse_In Service in the Enterprise Service repository. Alternatively you can browse for the Service you are looking for (as pictured above).
   

4. Note that there are no service endpoints defined if you go to the WSDL tab when viewing the service details.

5. Go to transaction /nsoamanager in SAP GUI
6. Go to the “Application And Scenario Communication” Tab ->; “Single Service Administration”.

7. Search for the name of the service as found in sproxy (e.g. EmployeeBasicDataByEmployeeQueryResponse_In)
8. Select the Service in the search results and click “Apply Selection”. Notice that there are no Bindings Listed if you click on “Display Selected Binding’s WSDL”
9. Click the "Configurations" tab and Click the "Create Service" button. Enter a service name e.g. EmployeeBasicDataByEmployeeQueryResponse_In_Service, and a description
   
   
10. Also enter a binding name e.g. EmployeeBasicDataByEmployeeQueryResponse_In_Binding, and Click “Apply Settings”

11. A new set of tabs should appear at the bottom of your screen. Enter details for the provider security
12. Click “Save”

13. The service is not activated yet. Even if you try and open the service with SOAP UI at this stage, you will just get errors such as "“Logon Error Message. What has happened? URL http://servername:serverport/sap/bc/srt/xip/sap/ecc_employeebasicdbyemployeeqr/120/employeebasicdatabyemployeequery/employeebasicdatabyemployeequeryresponse_in_binding call was terminated because the corresponding service is not available."
14. To activate the service, go to transaction /nsicf in SAP GUI

15. Search for the service e.g. put “EMPLOYEE*” in the service name. This should find your service e.g. “EmployeeBasicDataByEmployeeQuery”
    
16. Right click on the service at the bottom level and click Activate, and click on the “Yes” button (yes it is odd that there are 2 – I don’t know the difference).
    

17. You should be able to call the web service through SOAP UI or any other web services client or tool (e.g. Visual Studio .NET). Make sure you enter a username and password under the “Auth” tab of the request (if using SOAP UI 3.0).
    

18. DONE!
    

Exposing the SAP Netweaver CE 7.1 BPM Universal Worklist (UWL) through an iView and the Object Tag Viewer Web Part in SharePoint 2007

SAP iViews are one of many mechanisms by which you can surface SAP content into a SharePoint site. This post describes how you can show the SAP Universal Worklist (UWL) in SharePoint through a custom web part called the Object Tag Page Viewer.

As desribed in an MS white paper ( http://download.microsoft.com/download/B/F/2/BF2C3AAD-BC64-496C-B3BC-72ADD9617E2C/iView%20Integration%20with%20Microsoft%20Office%20SharePoint%20Server.pdf):

"There are scenarios where an end user would require logging in to SAP GUI or SAP Portal to perform some of their commonly performed operations. It would be convenient if these operations were performed from a portal like Microsoft Office SharePoint Server Portal (MOSS) which an end user might be using more frequently. In such scenarios the best point of integration is to display SAP iViews or Web Dynpro iViews in MOSS. MOSS out of the box provides two web parts that can be used for displaying pages from portals/web sites outside MOSS. These are Page Viewer Web Part and iView Web Part. When configuring SAP iView or Web Dynpro iViews, these web parts can go in to an infinite refresh loop with some iViews. In order to overcome this issue, there is a need to develop a custom web part that would display the SAP iView. The Object Tag Page Viewer Web Part is an approach that allows MOSS portals to display these iViews. "

While the document above covers the creation of the custom web part, there are a few issues with it:

• a) It doesn't cover how you set up or consume something like the SAP UWL (The UWL is basically the same thing as the Task list in SharePoint)
  
• b) The code in the whitepaper is not complete.

The Object Tag Page Viewer Web Part (Code)

using System;
using System.Runtime.InteropServices;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Xml.Serialization;
using Microsoft.SharePoint;
using Microsoft.SharePoint.WebControls;
using Microsoft.SharePoint.WebPartPages;
using System.ComponentModel;

namespace Microsoft
{
    /// 

    /// Created as per pdf Document "SAP iView Integration with Microsoft Office SharePoint 2007"
    /// 
    [Guid("a9c26185-1970-4f29-b45c-edad343938c1")]
    public class ObjectTagPageViewer : System.Web.UI.WebControls.WebParts.WebPart
    {
        const int heightWidth = 150;

        #region Properties
        /// 
 /// The URL to display in the Object Tag /// 
        private string pageUrl = "http://";

        [Personalizable(), WebBrowsable(true), Browsable(true), Category("Custom Property"), DefaultValue("http://"), WebPartStorage(Storage.Personal),
        FriendlyName("Page Url"), Description("Default page URL such as http://www.live.com")]
        public string PageUrl { get { return pageUrl; } set { pageUrl = value; } }

        #endregion

        /// 

        ///
        /// 
        /// 
        protected override void OnPreRender(EventArgs e)
        {
            if (base.Height.IsEmpty)
            {
                base.Height = heightWidth;
            }
            base.OnPreRender(e);
        }

        protected override void Render(HtmlTextWriter output)
        {
            if (String.Compare(pageUrl, "http://") == 0  String.IsNullOrEmpty(pageUrl))
            {
                output.WriteBreak();
                output.WriteLine(@"To link to content, <a id="'ObjectTagPageViewer_" href="javascript:MSOTlPn_ShowToolPane2Wrapper(">                output.WriteBreak();
            }
            else
            {
                if (Uri.IsWellFormedUriString(pageUrl, UriKind.Absolute))
                {
                    string objectTagID = "outputObject_" + base.UniqueID;
                    output.AddAttribute(HtmlTextWriterAttribute.Id, objectTagID, false);
                    output.AddAttribute(HtmlTextWriterAttribute.Name, objectTagID, false);
                    output.AddAttribute(HtmlTextWriterAttribute.Width, "100%", false);
                    output.AddAttribute(HtmlTextWriterAttribute.Height, "100%", false);
                    output.AddAttribute(HtmlTextWriterAttribute.Type, "text/html", false);
                    output.AddAttribute("data", pageUrl, false);
                    output.RenderBeginTag(HtmlTextWriterTag.Object);
                    output.RenderEndTag();
                }
                else
                {
                    output.WriteBreak();
                    output.WriteLine("Please enter a valid Absolute  Page Url ");
                    output.WriteBreak();

                }

            }
        }

        public ObjectTagPageViewer()
        {
        }

    }
}

Setup of iView Web Part to show the SAP UWL

1. Go to http://insert_ce_server_name_here/irj/portal and log in.
   
2. Go to Content Administration -> Portal Content -> Content Provided by SAP -> End User Content -> Standard Portal Users -> iViews -> com.sap.netweaver.bc.uwl.iviews -> Universal Worklist
3. Right Click - > Preview
   
4. Use the url that shows in the new preview window and put it into an Object Tag Page Viewer Web Part that you’ve added to a SharePoint Web Part page (as per the Microsoft Whitepaper)
   
5. Et voila! SAP UWL now shows in SharePoint. Clicking on tasks opens them up into another window - from where the user can action the task.
6. Tasks now open up in a new window for actioning: