SYMPTOM
Symptom 1: SharePoint is showing unexpected response (403 error) in Edge or Chrome Browsers but not in Internet Explorer whenever a call to client.svc/ProcessQuery is sent to the server as an incoming request.
For example, after adding a people column to a document library and typing in a username, test
Symptom 2: SharePoint is showing unexpected response (403 error) in Edge or Chrome Browsers but not in Internet Explorer running JavaScript from a content editor web part.
CAUSE
SharePoint 2016 has a security feature that will compare the actual request URL with the request origin header. If they don't match, the request will be rejected with status 403.
In order to verify if this is the problem, add a hosts file entry to your local client machine that resolves the SharePoint web site URL to a SharePoint Web front end server IP address to bypass the Network Load Balancer or Reverse Proxy.
RESOLUTION
Microsoft recommends configuring a rule in your Reverse Proxy or Network Load Balancer to adjust the origin to match the original request.
In case you don't have access to this, you can create a re-write rule in IIS. Implement the following IIS inbound rewrite rules to overcome the 403 error for JavaScript/CSOM calls not working/loading when accessing site through Reverse Proxy or Network Load Balancer URL.
Before trying out anything you find on the internet, make sure you are in a testing environment and have known good backups.
1. Make sure URL Rewrite is available
Download and install the IIS rewrite module: https://www.IIS.net/downloads/microsoft/URL-rewrite
Close and reopen IIS
2. Configure Rewrite Rules and add Server Variables:
Go to your SharePoint site.
Click on rewrite URL:
On the Right under Actions, click on View Server Variables
- Add this to allowed server variables:
HTTP_Origin
HTTP_HOST
Click on Back to Rules under Actions menu on the right. Then, click on Create an inbound rule:
- Create a new inbound rule
- Add this as regular expression filter:
.svc.+
- In Server Variables, click Add
- Use this information:
Name: HTTP_Origin
Value: http://{HTTP_HOST}
- For action choose 'None'
- Save the rule
- Create another new inbound rule to allow rewrite for the java scripts
- Add this as regular expression filter:
_api.+
- In Server Variables, click Add
- Use this information:
Name: HTTP_Origin
Value: http://{HTTP_HOST}
- For action choose 'None'
- Save the rule
In application.config you would see something like this (there may be other variables for other rules but leave them alone, make sure that these two are included)
<rewrite>
<allowedServerVariables>
<add name="HTTP_Origin" />
<add name="HTTP_HOST" />
</allowedServerVariables>
</rewrite>
In web.config, you should see this:
<rewrite>
<rules>
<clear />
<rule name="Origin">
<match URL=".svc.+" />
<serverVariables>
<set name="HTTP_Origin" value="http://{HTTP_HOST}" />
</serverVariables>
<action type="None" />
</rule>
<rule name="Origin2">
<match URL="api.+" />
<serverVariables>
<set name="HTTP_Origin" value="http://{HTTP_HOST}" />
</serverVariables>
<action type="None" />
</rule>
MORE INFORMATION
DATA ANALYSIS:
From the client machine where you just configured the fiddler, browse to the site with the public URL for the zone.
From Fiddler: Note the origin and header in the Headers tab in the upper right and find the correlation id to search for in your SharePoint logs in the miscellaneous section in the lower right listed as request-id or SPRequestGuid:
Symptom 1: Fiddler
05/29/2018 18:45:08.23 w3wp.exe (0x1DF4) 0x2618 SharePoint Foundation Logging Correlation Data xmnv Medium Name=Request (POST:http://sp.contoso.com/sites/corstest/_vti_bin/client.svc/ProcessQuery) 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x1804 SharePoint Foundation CSOM agw10 Medium Begin CSOM Request ManagedThreadId=6, NativeThreadId=6148 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x1804 SharePoint Foundation CSOM azvn3 Medium Request is a Cross-Origin request. Origin is : 'http://melissa.contoso.com'. Host is : http://sp.contoso.com/_vti_bin/client.svc/ProcessQuery 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x1804 SharePoint Foundation CSOM azvn4 Medium Request is a Cross-Origin request for a user that was not authenticated using OAuth. Returning 403 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x1804 SharePoint Foundation CSOM aiv4g Medium OnBeginRequest returns false, do not need to continue process the request. 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x0934 SharePoint Foundation Runtime aoxsq Medium Sending HTTP response 403 for HTTP request POST to http://sp.contoso.com/_vti_bin/client.svc/ProcessQuery 06046c9e-92a0-40c7-b2ae-2165c547d61c
05/29/2018 18:45:08.24 w3wp.exe (0x1DF4) 0x0934 SharePoint Foundation Monitoring b4ly Medium Leaving Monitored Scope: (Request (POST:http://sp.contoso.com/sites/corstest/_vti_bin/client.svc/ProcessQuery)) Execution Time=14.3752; CPU Milliseconds=10; SQL Query Count=0;Parent=None 06046c9e-92a0-40c7-b2ae-2165c547d61c
Symptom 2:
Right click on the SPRequestGuid in the right hand lower left section, copy value only, then go open up the SP ULS and search for the correlation id:
Remember, we browsed to http://melissa.contoso.com. Here is an excerpt from the correlation id in this instance:
05/21/2018 18:49:04.38 w3wp.exe (0x163C) 0x1E10 SharePoint Foundation Logging Correlation Data xmnv Medium Name=Request (POST:http://sp.contoso.com/sites/corstest/_api/contextinfo) 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.38 w3wp.exe (0x163C) 0x1E10 SharePoint Foundation General adyrv High Cannot find site lookup info for request Uri http://sp.contoso.com/sites/corstest/_api/contextinfo. 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.38 w3wp.exe (0x163C) 0x1E10 SharePoint Foundation Audience Validation a9fy7 Medium The audience uri loads a web application matches. AudienceUri: 'http://melissa.contoso.com/', InputWebApplicationId: '8e26ceaa-446b-45bc-ba30-4fc65baeec0f', InputURLZone: 'Default'. 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x1D24 SharePoint Foundation CSOM agw10 Medium Begin CSOM Request ManagedThreadId=42, NativeThreadId=7460 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x1D24 SharePoint Foundation CSOM azvn3 Medium Request is a Cross-Origin request. Origin is : 'http://melissa.contoso.com'. Host is : http://sp.contoso.com/_vti_bin/client.svc/contextinfo 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x1D24 SharePoint Foundation CSOM azvn4 Medium Request is a Cross-Origin request for a user that was not authenticated using OAuth. Returning 403 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x2368 SharePoint Foundation General adyrv High Cannot find site lookup info for request Uri http://sp.contoso.com/sites/corstest/_api/contextinfo. 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x2368 SharePoint Foundation Runtime aoxsq Medium Sending HTTP response 403 for HTTP request POST to http://sp.contoso.com/_vti_bin/client.svc/contextinfo 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x2368 SharePoint Foundation General azrx9 Medium LookupHostHeaderSite: Using site lookup provider Microsoft.SharePoint.Administration.SPConfigurationDatabaseSiteLookupProvider for host-header site-based multi-URL lookup string http://sp.contoso.com/sites/corstest for request Uri http://sp.contoso.com/sites/corstest/_api/contextinfo. 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x2368 SharePoint Foundation General adyrv High Cannot find site lookup info for request Uri http://sp.contoso.com/sites/corstest/_api/contextinfo. 1271699e-0247-40c7-b2ae-2e61ad704f51
05/21/2018 18:49:04.39 w3wp.exe (0x163C) 0x2368 SharePoint Foundation Monitoring b4ly Medium Leaving Monitored Scope: (Request (POST:http://sp.contoso.com/sites/corstest/_api/contextinfo)) Execution Time=34.3893; CPU Milliseconds=21; SQL Query Count=14; Parent=None 1271699e-0247-40c7-b2ae-2e61ad704f51
Here in the network trace, we can see the request is coming from 192.168.2.51 – this is where the reverse proxy is running and we can see SharePoint (192.168.2.53) reply with the 403 Forbidden error message. Note the host and origin are highlighted and they are not matching resulting in the 403 error message.
SETUP/SCENARIO
Symptom 1:
- Configure environment with a path based site collection.
- Create a document library
- Add a person column to the library
- In Chrome, browse the library with reverse proxy URL
Symptom 2:
- Configure environment with a path based site collection.
-
Configure the site collection to run JavaScript from a content editor web part
- Have a content editor web part configured on a page, for example: http://sp.contoso.com/sites/corstest/SitePages/example.aspx
- Upon editing the content editor web part, there is a content link set to /sites/corstest/SiteAssets/example.js
- Have a content editor web part configured on a page, for example: http://sp.contoso.com/sites/corstest/SitePages/example.aspx
- Find the example.js code at the end of the post.
-
Starting config - Alternate Access Mappings
Prior to configuring the SharePoint to use a different URL and configure the reverse proxy:
The web app URL: http://sp
The alternate access mapping:
-
modified config – AAM's. FYI, AAM's are deprecated in SP 2016.
Configured AAM for the "new" URL:
Which automatically updates the web application URL:
Add a DNS entry or hosts file for melissa.contoso.com.
Irrespective of browser, there is no issue or 403 error browsing to http://melissa.contoso.com or loading the example.aspx page referencing the JavaScript.
*Please note if the public URL for the zone is added as https, then on the SP servers in IIS it will be necessary to add a binding for https port 443 and an SSL certificate.
-
Configure Fiddler as a reverse proxy. There's lots of documentation and videos on this, but here is the short of it.
Tools, Options, Connections tab: check "Allow remote computers to connect"
Then, back at the menu bar, select Rules, Customize rules, and the Fiddler ScriptEditor window should open. From its menu, click on Go, click to OnBeginRequest.
Add the following (with your own URL's, of course) after the comments in the section:
static function OnBeforeRequest(oSession: Session) {
if (oSession.HostnameIs("melissa.contoso.com"))
{
oSession.hostname="sp.contoso.com";
}
JAVASCRIPT CODE EXAMPLE
The sample.js contents are:
<html>
<head>
<title>Cross-domain sample</title>
</head>
<body>
<!-- This is the placeholder for the announcements -->
<div id="renderAnnouncements"></div>
<script
type="text/javascript"
src="//ajax.aspnetcdn.com/ajax/jQuery/jquery-1.7.2.min.js">
</script>
<script type="text/javascript" src="//ajax.aspnetcdn.com/ajax/4.0/1/MicrosoftAjax.js"></script>
<script type="text/javascript" src="/_layouts/15/sp.runtime.js"></script>
<script type="text/javascript" src="/_layouts/15/sp.js"></script>
<script type="text/javascript">
//var hostwebURL;
//var appwebURL;
// Load the required SharePoint libraries
$(document).ready(function () {
SP.SOD.executeFunc('sp.js', 'SP.ClientContext', getProjectURL);
// //Get the URI decoded URLs.
// hostwebURL =
// decodeURIComponent(
// getQueryStringParameter("SPHostURL")
// );
// appwebURL =
// decodeURIComponent(
// getQueryStringParameter("SPAppWebURL")
// );
// resources are in URLs in the form:
// web_URL/_layouts/15/resource
var scriptbase = getProjectURL() + "/_layouts/15/";
// Load the js files and continue to the successHandler
$.getScript(scriptbase + "SP.RequestExecutor.js", execCrossDomainRequestA);
});
// Function to prepare and issue the request to get
// SharePoint data
function execCrossDomainRequest() {
// executor: The RequestExecutor object
// Initialize the RequestExecutor with the add-in web URL.
var executor = new SP.RequestExecutor(appwebURL);
// Issue the call against the add-in web.
// To get the title using REST we can hit the endpoint:
// appwebURL/_api/web/lists/getbytitle('listname')/items
// The response formats the data in the JSON format.
// The functions successHandler and errorHandler attend the
// sucess and error events respectively.
executor.executeAsync(
{
URL:
appwebURL +
"/_api/web/lists/getbytitle('Announcements')/items",
method: "POST",
headers: { "Accept": "application/json; odata=verbose" },
success: successHandler,
error: errorHandler,
crossDomain: true
}
);
}
function successHandlerA(data, req) {
var announcementsHTML = "";
var enumerator = allAnnouncements.getEnumerator();
while (enumerator.moveNext()) {
var announcement = enumerator.get_current();
announcementsHTML = announcementsHTML +
"<p><h1>" + announcement.get_item("Title") +
"</h1>" + announcement.get_item("Body") +
"</p><hr>";
}
document.getElementById("renderAnnouncements").innerHTML =
announcementsHTML;
}
// Function to handle the success event.
// Prints the data to the page.
function successHandler(data) {
var jsonObject = JSON.parse(data.body);
var announcementsHTML = "";
var results = jsonObject.d.results;
for (var i = 0; i < results.length; i++) {
announcementsHTML = announcementsHTML +
"<p><h1>" + results[i].Title +
"</h1>" + results[i].Body +
"</p><hr>";
}
document.getElementById("renderAnnouncements").innerHTML =
announcementsHTML;
}
// Function to handle the error event.
// Prints the error message to the page.
function errorHandler(data, errorCode, errorMessage) {
document.getElementById("renderAnnouncements").innerText =
"Could not complete cross-domain call: " + errorMessage;
}
function execCrossDomainRequestA() {
// context: The ClientContext object provides access to
// the web and lists objects.
// factory: Initialize the factory object with the
// app web URL.
var addinwebURL = getProjectURL();
var context = new SP.ClientContext(addinwebURL);
var factory =
new SP.ProxyWebRequestExecutorFactory(
addinwebURL
);
context.set_webRequestExecutorFactory(factory);
//Get the web and list objects
// and prepare the query
var web = context.get_web();
var list = web.get_lists().getByTitle("Announcements");
var camlString =
"<View><ViewFields>" +
"<FieldRef Name='Title' />" +
"<FieldRef Name='Body' />" +
"</ViewFields></View>";
var camlQuery = new SP.CamlQuery();
camlQuery.set_viewXml(camlString);
allAnnouncements = list.getItems(camlQuery);
context.load(allAnnouncements, "Include(Title, Body)");
//Execute the query with all the previous
// options and parameters
context.executeQueryAsync(
successHandlerA, errorHandler
);
}
function getProjectURL() {
var URLToReturn = "";
var baseURL = document.URL.split("/");
//URLToReturn = baseURL[0] + "//" + baseURL[2] + _spPageContextInfo.siteServerRelativeURL + pageURL + "?" + queryStringKey + "=" + queryStringValue + "&" + categoryString + "&ViewMode=1";
URLToReturn = baseURL[0] + "//" + baseURL[2] + _spPageContextInfo.siteServerRelativeURL;
return (URLToReturn);
}
// Function to retrieve a query string value.
// For production purposes you may want to use
// a library to handle the query string.
function getQueryStringParameter(paramToRetrieve) {
var params =
document.URL.split("?")[1].split("&");
var strParams = "";
for (var i = 0; i < params.length; i = i + 1) {
var singleParam = params[i].split("=");
if (singleParam[0] == paramToRetrieve)
return singleParam[1];
}
}
</script>
</body>
</html>
The End
Thanks for reading, thanks for you and thanks for all those who came before us. Share your experience and submit your suggestions for SharePoint here: https://sharepoint.uservoice.com