Provider-hosted apps and SharePoint memory

After a period of building and testing a full-blown business application (a provider-hosted app on a SharePoint 2013 farm) it was time to release it to Production.

After an hour or so after installation on Production, I received a call from the IT department telling me memory usage of the SharePoint front-ends has increased significantly. More than 500MB per server and growing… Up to 2GB+…

Very naively, during building, I never really looked at memory usage because I assumed, since I wasn’t building a farm solution, SharePoint would clean up its own mess and the provider-hosted app would be kept under control by the garbage collector.

After the initial call from IT, I immediately started looking at our provider-hosted app. Was there anything there that could leak memory? I did see memory growing so obviously there was an issue.

Because I couldn’t find anything wrong in our code, I decided to start a new provider-hosted app, generated from Visual Studio.

Freshly started, the default SharePoint App pool uses between 500-650MB of memory, depending on the actions by users in SharePoint. This does not include Search, which runs under a separate application pool.

So, let’s send a request to SharePoint from our provider-hosted app:

User spUser = null;
var spContext = SharePointContextProvider.Current.GetSharePointContext(HttpContext);
using (var clientContext = spContext.CreateUserClientContextForSPHost()
{
  if (clientContext != null)
  {
    spUser = clientContext.Web.CurrentUser;
    clientContext.Load(spUser, user => user.Title);
    clientContext.ExecuteQuery();
    ViewBag.UserName = spUser.Title;
  }
}

This piece of code is what Visual Studio generates when you create a new provider-hosted app project. No custom code on my part.

What I noticed: the first time such a request is sent to SharePoint, memory used by the default SharePoint App pool increases almost instantly to more than 900MB, an increase of more than 300MB. The memory usage of the provider-hosted app itself meanwhile seems pretty stable.

Why is SharePoint suddenly using more memory? I have no idea. As I said, I’m no expert on memory or the way .NET and SharePoint use memory.

As you know:  clientContext.ExecuteQuery() will result in a call to _vti_bin/client.svc/ProcessQuery

So, maybe SharePoint creates some kind of memory pool as soon as the client.svc is hit?

Anyway, after the initial increase in memory I also noticed the memory keeps increasing with every subsequent call to the client.svc.

So I modified the code by adding a loop so the call is executed a thousand times. After running this code, resulting in a thousand requests, memory increased by approx. 340MB. After running this test several times, I guess that a request will run up memory anywhere between 250 and 400KB per request. I admit, all measured very unscientifically… but still…

After running this test multiple times, memory usage run up to 2GB, then leveled off and stayed between 1.6GB and 1.9GB. My development environment has 16GB of memory.
I also tried this on another machine with 12GB of memory and on that machine, memory leveled off at approx. 1.4GB.
I’m guessing SharePoint is disposing objects and releasing memory when it hits a certain threshold, depending on available memory.

Btw: during all this testing, there was no application pool recycle.

Provider-hosted apps are chatty

Now, you could say: “all very nice but a thousand requests in a short span of time is kinda much. That represents a very busy site”.
Unfortunately, I have to disagree on that one. Because we’re talking provider-hosted apps, everything you do in code requires a request to SharePoint. The very nature of provider-hosted apps is chatty.

You want to get a list, you need to request it… You want to get the base permissions on a list, which requires another request… Setting permissions or removing them from a list item, another request… etc., etc.

Even if you try to batch requests as much as possible, when you’re building a full-blown business application as a provider-hosted app, you will see that the amount of requests to SharePoint will increase fast with every piece of functionality you add.

In my particular case, when we create a dossier-like structure with documents, we need to create a document set, set permissions, create tasks, log actions, etc. This relatively simple scenario fires about 40 requests to set everything up.

By design?

I have no idea if this behavior (the memory increase) is by design. It could very well be since memory usage levels off without an app pool recycle.

I do know that this behavior is not restricted to my development laptop (built by me). I also noticed this behavior on our test environment (built by colleague) and our acceptance and production environment (both built by IT Pro’s, under advisement by Microsoft consultants).

Why did I write this blogpost?
So you can be aware of this behavior if you’re going to build complex provider-hosted apps. So you can make sure you batch as much requests as possible. So you can reassure the IT department monitoring SharePoint that your application doesn’t cause memory leaks… (as far as I know)

So what do you SharePoint experts out there think? Am I missing something obvious here? Am I making wrong or right assumptions?
If you have any suggestions, please feel free to respond!

Please notice: I’m not an IT Pro and I have very little in-depth knowledge of memory usage of SharePoint or memory usage in general. In this blogpost I describe my findings and assumptions on what’s happening under the covers of SharePoint.

Leave a Reply

Your email address will not be published. Required fields are marked *