Why Remote Provisioning Helps Testing in Sharepoint

Introduction to Remote Provisioning

Before continuing with our series on testing in sharepoint we need to do a pit stop and introduce the concept of Remote Provisioning because it is quite useful when testing your application, in particular for the integration tests.

Remote provisioning is a concept introduced in Sharepoint 2013 and fundamental when working on Add-ins. It’s the suggested way to provision all your SHP artifacts (like Content Types, Fields, or Lists). It replaces everything that was deployed thought and Elements.xml and the Feature Framework.

OK, not everything, unfortunately the API isn’t on par on some things, but usually there are workarounds.

While remote provisioning an artifact you use the Client Side Object Model (CSOM) to programmatically create the artifact you need. This gives you the flexibility to build it imperatively and not declaratively (if you wish), but more importantly this prevents all the problems related to the feature framework like what happens when you have to update an existing object. If you have worked in SHP I have probably just summoned a few nightmares for you tonight, for something that should actually be quite simple.

If you want more information on doing remote provisioning I highly suggest you check out Office Dev PnP and it’s related provisioning engine, they provided a plethora of samples and helper methods to simplify doing remote provisioning and working with CSOM in general.

Why it helps

Let’s do a thought experiment:

Let’s say that you are testing a class in sharepoint, your test fails and you fix the class. Now you want to run the test… What should you do?

If you are used to doing TDD you might answer: “Just run the test”. Well doing it the traditional way you would have deployed your application’s DLLs with the same WSP that provision your artifacts*. This means that the DLLs are in the GAC and that means that the old version will be used instead of the one you just fixed.

So your test will keep failing and if you don’t realize it you could be wasting a lot of time trying to figure out why that happened.

To fix this you either have to deploy the new version (which is slow) or you retract the WSP so that your test framework can run from the local (and recent) copy. The problem is that if you retract the WSP all your artifact are removed from the sitecollection and you might get failing tests.

By doing remote provisioning all your artifacts deployed separately to the WSP and so you can safely retract it and keep working in a fast RED-GREEN-REFACTOR cycle. Also this way the deployment of the WSP is way faster.

*For those that don’t know SHP development this is like deploying your database in the same package that deploys your application’s DLLS

Testing your logic

Well, this article will be quite short, because when you isolate yourself from sharepoint, or any other hard to test framework, you can write test the same way you would do in any other environment. Now it doesn’t matter if you are a sharepoint developer or a mvc developer, you have plain old business object everywhere and any boundary can be replaced with a test double.

Let’s see a concrete example of what I’m saying. Picking a recent example from a project I’m working on let’s implement the following requirement:

Given that a user can be in multiple groups, and that a company is related to 2 groups, find all the companies that are related to the current user

To do this we need two data sources:

  • One provides the list of groups the current user is in
  • The other provides the list of all the companies and its related groups

This data will be provided by two different repositories with the following interfaces:

public class Group
{
  public string Name { get; set; }
}

public interface IGroupRepository
{
  IEnumerable<Group> GetCurrentUserGroups();
}

public class Company
{
  public string Name { get; set; }
  public Group Reader { get; set; }
  public Group Editor { get; set; }
}

public interface ICompanyRepository
{
  IEnumerable<Company> All();
}

OK now we can start TDDing our logic. We start with the most basic test and work our way up till we are done.

public class Tests
{
  [Test]
  public void NoCompanyNoGroups_NoCompany
  {
    var sut = new UserManager(new NoCompanyRepository(), new NoGroupRepository());
    var companies = sut.GetRelatedCompanies();
    Assert.That(companies.Count(), Is.EqualTo(0));
  }
}

This obviously fails, it doesn’t even compile. Let’s pretend we have the No___Repository stubs (if you have trouble implementing them ask in the comments). Let’s write the simplest manager that can pass that test

public class UserManager
{
  private readonly ICompanyRepository companyRepo;
  private readonly IGroupRepository groupRepo;

  public UserManager(ICompanyRepository companyRepo, IGroupRepository groupRepo)
  {
    this.companyRepo = companyRepo;
    this.groupRepo = groupRepo;
  }

  public IEnumerable<Company> GetRelatedCompanies()
  {
    return new List<Company>();
  }
}

Great this passes, but obviously isn’t what we were after. Let’s write another test

 [Test]
 public void RightCompanyAndGroup_OneCompany()
 {
   var sut = new UserManager(new OneCompanyRepository(), new OneEditorGroupRepository());
   var companies = sut.GetRelatedCompanies();
   Assert.That(companies.Count(), Is.EqualTo(1));
   Assert.That(companies.First().Name, Is.EqualTo("Right"));
 }

  public class OneCompanyRepository : ICompanyRepository
  {
    public IEnumerable<Company> All()
    {
      return new List<Company>()
            {
                new Company
                {
                  Editor = new Group {Name = "Editor"}, 
                  Reader = new Group {Name = "Reader"}, 
                  Name = "Right"
                }
            };
    }
  }

  public class OneEditorGroupRepository : IGroupRepository
  {
    public IEnumerable<Group> GetCurrentUserGroups()
    {
      return new List<Group>() { new Group { Name = "Editor" } };
    }
  }

Given the new test the simplest manager can return all the companies from the repository without filter, so we need to test a negative result. The following test checks that we don’t return any company given a set of unrelated groups.

[Test]
public void CompanyAndWrongGroup_NoCompany()
{
  var sut = new UserManager(new OneCompanyRepository(), new OneUnreleatedGroupRepository());
  var companies = sut.GetRelatedCompanies();
  Assert.That(companies.Count(), Is.EqualTo(0));
}
public class OneUnreleatedGroupRepository : IGroupRepository
{
  public IEnumerable<Group> GetCurrentUserGroups()
  {
    return new List<Group>() { new Group { Name = "Unreleated" } };
  }
}

Now our implementation can finally be compleated

public IEnumerable<Company> GetRelatedCompanies()
{
  var names = groupRepo.GetCurrentUserGroups().Select(g => g.Name).ToList();
  return companyRepo.All()
    .Where(company =>
      names.Contains(company.Editor.Name) ||
      names.Contains(company.Reader.Name));
}

As you can see we have implemented a (simple) bit of business logic that in production will be dealing with objects coming from sharepoint, but that thanks to the layer of repositories, we were able to isolate and build using TDD

Testing your sharepoint repositories

In this article we will see how to test your sharepoint repositories, as we said before writing unit test for them isn’t easy. I don’t think you can test drive them or that it would help, at the very least it would be too slow.

Therefore SHP repositories are a weak point in your testing strategy, but at the very same time they are the base on which the rest of your application sits on. So it is very important that you do two things:

  1. Write KISS repositories
  2. Write integration tests for them

KISS: Keep It Simple Stupid

Your repositories need to be simple, strip them of any logic that has a reason to be somewhere else. Any constraint that isn’t strictly related to the persistent layer should be checked by another layer.

They should receive an entity from above and persist it like it is, if the entity comes with errors those errors will be persisted (or the layer bellow will throw an exception).

Expose a basic API. Instead of a method Save that check if the entities exists and then decides whether to update it or create a new entity expose three basic methods (Exists, Create, Update) and let the layer above make that decision.

Once you have a KISS repository you need to secure it the best you can. This is done by writing tests integrated with sharepoint. Usually to do this you will have to do the following things:

  1. Create a test site collection
  2. Provision your artifacts (Fields, ContentTypes, Lists…)
  3. For each test
    1. Populate one or more list
    2. Run a test
    3. Clean up
  4. Delete the site collection

This is obviously a very slow process so you won’t run this tests in your normal TDD cycle, but you can run them frequently enough that you will catch most bugs before they can cause any major problem.

Now let’s put aside the theory and write some code, starting from the repository of the previous article

    private const string WebAppUrl = "http://localhost";
    private const string SiteUrl = "http://localhost/sites/Test";
    private IPersonRepository repository;
    private SPSite site;

    [SetUp]
    public void SetUp()
    {
      site = CreateSite();
      repository = new PersonRepository(site.RootWeb);
    }

    [Test]
    public void WhenAddingAPerson__ItShouldAddAPerson()
    {
      var list = site.RootWeb.Lists["Person"];
      CleanList(list);

      var person = AddPerson();

      AssertCreation(list, person);
    }

    private SPSite CreateSite()
    {
      var webapp = SPWebApplication.Lookup(new Uri(WebAppUrl));

      if (SPSite.Exists(new Uri(SiteUrl)))
        webapp.Sites.Delete(SiteUrl);

      var site = webapp.Sites.Add(SiteUrl, "contoso\\admin", "admin@contoso.com");
      site.Features.Add(new Guid("<GUID>"));
      return site;
    }

    private void CleanList(SPList list)
    {
      while (list.Items.Count > 0)
        list.Items[0].Delete();
    }

    private Person AddPerson()
    {
      var person = new Person
      {
        Address = "Address",
        BirthDate = DateTime.Now,
        Name = "Name"
      };
      repository.Create(person);
      return person;
    }

    private void AssertCreation(SPList list, Person person)
    {
      Assert.That(list.Items.Count, Is.EqualsTo(1));
      var item = list.Items[0];
      Assert.That(item["Address"], Is.EqualsTo(person.Address));
      Assert.That(item["Title"], Is.EqualsTo(person.Name));
      Assert.That(item["DateOfBirth"], Is.EqualsTo(person.BirthDate));
    }

Here we see a setup method that creates (or recreates) a site collection at every test run. To speed up the tests a bit, you can move that operation to a Fixture Setup, but that might mean that some tests pass or fail depending on the order of execution, so proceed with care. During the site creation we also enable one (or more) features that provision our sharepoint artifacts.

We then have the test method that checks that the Create method of the repository works fine by:

  1. Removing any item present on the list
  2. Calling the Create method with a valid person
  3. Checking (via standard object model) that was added an item to the expected list with the required properties.

In the same way one might test a Get method. First you clean the list, than you add the expected item (via SSOM), than you get it with your repository and assert that everything is fine. Try it doing it yourself and let me know if you have any problem.

Repository Pattern

If you have ever heard of software design patterns, you have heard of the Gang of Four and their book. It’s the bible on this matter and it would be a disservice not to reference it, but the pattern we will talk about is missing from this book. You instead can find more information here.

In short the repository is an object that sits between your application data and your business logic. It helps (among other things) remove duplicated code, improve testability and provide a strong typed business entity that helps catch errors at compile time.

Repository are used to isolate from data source like databases, file system or a Sharepoint list, and from other difficult to integrate objects like web services. When all your business logic deals with data retrieval or persistence through a repository, you can test your business logic in isolation replacing the real repository with a test double.

To have a better understanding of what’s going on let’s have a look at some code

    class Person
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public DateTime BirthDate { get; set; }
        public string Address { get; set; }
    }

    interface IPersonRepository
    {
        Person Get(int id);
        void Create(Person person);
        void Update(Person person);
        void Delete(int id);
    }

    class PersonRepository : IPersonRepository
    {
        private readonly SPList list;
        private readonly PersonSPListItemMapper mapper;

        public PersonRepository(SPWeb web)
        {
            list = web.Lists["Person"];
            mapper = new PersonSPListItemMapper();
        }

        public Person Get(int id)
        {
            SPListItem item = list.GetItemById(id);
            Person person = mapper.From(item);
            return person;
        }

        public void Create(Person person)
        {
            SPListItem item = list.AddItem();
            mapper.Populate(item, person);
            item.Update();
        }

        public void Update(Person person)
        {
            SPListItem item = list.GetItemById(person.Id);
            mapper.Populate(item, person);
            item.Update();
        }

        public void Delete(int id)
        {
            SPListItem item = list.GetItemById(id);
            item.Delete();
        }
    }

    class PersonSPListItemMapper
    {
        public Person From(SPListItem item)
        {
            return new Person
            {
                Name = item.Title,
                Id = item.ID,
                BirthDate = (DateTime)item["DateOfBirth"],
                Address = (string)item["Address"]
            };
        }

        public void Populate(SPListItem item, Person person)
        {
            item["Title"] = person.Name;
            item["DateOfBirth"] = person.BirthDate;
            item["Address"] = person.Address;
        }
    }

Here we see, in oder, the business entity we are interested on (Person), with a few basic properties, then the interface of our repository with a few basic CRUD operations, then we implement said interface against a sharepoint list, at the end we have a utility class that’s meant only to map to and from a sharepoint item.

The repository pattern is quite simple to comprehend, what’s important is that:

  • You keep following it
  • Every object in the business layer interacts with the entities through a repository
  • Every interaction is isolated by an interface

The following is a business layer object that expects a repository in its contructor and uses it to create a Person.

    internal class PersonManager
	{
		IPersonRepository repository;
		public PersonManager(IPersonRepository repository)
		{
			this.repository = repository;
		}
		public void Create(string name, string address, DateTime birthDate)
		{
			Person person = new Person
			{
				Name = name,
				BirthDate = birthDate,
				Address = address
			}
			repository.Create(person);
		}
	}

As you can see this class has no knowledge of what is Sharepoint and can be tested quite easily by passing a spy to the constructor instead of the real implementation.

All operations to persist a business entity pass through a repository, and any data retrieved by a query comes from it. In my example you can see a simple get by id query, but more complex queries are allowed. You can implement them in two ways:

  • The repository has a method for each query
  • The repository receives a query object that contains the required information to build a query

Another important advantage of using the repository pattern is that it can provide a centralized caching layer to for the rest of the application. This way if a query has already been served the repository can return a cached result set without having to muck the details of the business layer. This is quite useful when dealing with repositories that mask web services.

In the end, repository help isolate data access code, from the rest of your application, it provides an insertion point to test your business layer but adds another abstraction layer so it can be difficult to understand for developers in your team whom are unfamiliar with this pattern. Also keep in mind that repositories tend to be hard to unit test, so it is better to write integration test for them.

How to test in sharepoint

We have talked about how to set up your test environment, how to write tests and what to do with the freedom that the test give you (write code that is clean). Now we start to get into the nitty gritty of being a code cleaner, we need to start to clean some legacy code. I’m young, so probably my experience is not the same as most professional code cleaners, but when I think of legacy code, I think of sharepoint programming.

Programming with sharepoint is hard, you need to know how to work your way around many weird behaviours is sharepoint like the fact that in a field XML schema  False != FALSE.

But you get used to that stuff. The biggest problem with sharepoint is that it wasn’t built with the idea of enabling testability. No, most interesting classes are sealed (can’t be extended), and a value that can be approximated with 100% of the classes doesn’t inherit from an interface. You have to pass around concrete classes all around your code. This is a problem because you can’t replace an SPListItem with a test double. And to make matters worst, the biggest advantage of using sharepoint is also its biggest testability problem. Sharepoint is a monster, it has every functionality you could think of: CMS, Document management, Search, OCR, Cache, line of business integration, social netnork, blog, forum, ecc. It wants you to use its features, it wants you to integrate with it, but if you do in a naive way you are lost. There’s no way you will be able to TDD your application.

Luckily there’s a better way, its hard work, specially in the beginning when you need to change your frame of mind and the way you work, but there’s a better way, and I’m here to guide you.

We will start with a brief detour talking about a building block, the repository pattern, then we will see:

  • How to test your repositories
  • How to test your business logic
  • Sharepoint tdd tips and tricks
  • Why remote provisioning helps
  • How to do remote provisioning
  • Building repositories with SPMetal
  • More ways to isolate from the framework