December 13, 2017 // By Ralph Rivas
A key decision point with SharePoint projects involves the choice of which development patterns to select when designing, creating, packaging and deploying the solution to satisfy business. A major assumption for this post is that the decision making process has already narrowed the medium to the SharePoint space likely because the requirement seeks to have a significant level of integration with their SharePoint implementation on premises or in the cloud.
The options have definitely changed over time but it is interesting that though there are more current recommendations, nearly all the patterns going back to the first customizable version is still doable and, perhaps unfortunately, used in the spirit of expediency because that is what the developers and administrators know or understand. We will discuss these patterns in context of when they came out and what they solved as well as where their use has or may have become obsolete or ill-advised. New is not necessarily better as new abilities opens Pandora's box which means many problems that tend to obscure the hope of better things to come.
We will look at these solution types historically and discuss them in context as SharePoint improved over time. We will not cover solution types from before SharePoint 2003 since those attempts where more like "hack" customizations that may only have been permitted unofficially by Microsoft. Keep in mind, again, that all these solution types are all still viable with the exception coming around the introduction of BPOS and SharePoint online as well as SharePoint 2016.
2003 and the birth of the Farm solution
A Farm Solution has the following key characteristics:
- Server Side (Trusted) execution Model
- Uses the Server API for the product
- Works only in On Premises Farms and not at Microsoft's Publicly Hosted offerings (exceptions for private implementations and for non-Microsoft hosted farms are known)
- Programs typically built with Visual Studio and not by hand (Special Project Types to create Farm Solution packages came around the time of SharePoint 2007 and have improved since)
- Solutions scoped to the entire Farm (which means their deployment effectively brings down the whole farm for the duration)
- Used to build Web Parts, Event Receivers, Feature Receivers, timer jobs, workflows, and other artifacts available through the SharePoint or Windows Server APIs
Farm solutions were powerful once Microsoft started documenting how all the server side pieces could be used, what frameworks worked (e.g. Windows Workflow Foundation) and what the new crop of SharePoint MVP's had to contribute and say about implementing solutions in this manner.
Initially, in 2003, changes to the farm were manual or scripted (x-copy commands, etc.) and a main issue that has made it the bane of Production system administrators is the fact that changes usually required an update to perhaps the most important file in a web application: Web.Config. Change that and IIS resets. And that may not sound like a big deal but it is the equivalent of jumping out of a plane for the first time … each and every time. No one gets used to it and it always seems like someone HAS to use the system at the time no one should be on it. Naturally a cottage industry of solutions to mitigate this has also cropped up but at some one, someone asked "why" this had to happen at all especially when the change may have been something as benign as changing the fonts or colors of the page headers.
Indeed, many competing web based systems had solved that problem for quite some time and without the need of special third party help. Even with current Container technology in the cloud to simplify things, the cost of making changes for a large swath of projects certainly exceeded what the alternatives of late (and in some cases back then as well) have.
Perhaps the more obvious CON of a server side solution what that it had the ability to "bring down the farm" if it ran into a serious error and even though the best efforts are made by developers and contractors to create stable systems by way of best practices and so forth, integration situations and even changes that Microsoft makes to the product put the installation at risk every time they are introduced.
So we have the major CON of a heavy change management aspect for these solutions but there is one PRO that is still in play even as currently as SharePoint 2016 and that is the fact that it is the only development model that can handle Event Receivers and Feature Receivers which are sometimes needed for backward compatibility or because a requirement has been deemed as needing that level of customization. This PRO however may be considered weak in light of the newest development framework and in light of the advances in Web Application Development in General which has changed the industry as a whole while changing user expectations significantly along with it.
JavaScript Injection
JavaScript Injection solutions have the following main characteristics:
- Client Side (untrusted) Execution model
- Client Side API usage (varied as the product progressed)
- Can be built with any development tool kit (Notepad or JavaScript/HTML aware IDE)
- Scoped specifically to the current SharePoint page only
- Manually deployed with no publicly shared or known automation model for deployment
As soon as there was such a thing as the content editor web part, clever developers have attempted to circumvent the Web Part and Feature Deployment limitations by putting in manual script and using early web page debugging tools, including the ubiquitous "View Source", to reverse engineer pieces and quickly make customizations that did not require the Farm to go down.
Microsoft's attempt to block these types of scripts in the early iterations gave way to a more formal "framework" for using them properly or rather more appropriately. In 2007, there was a command directive you could invoke in your script code to "allow" your script to not just run, but so other things like communicate to the "outside" or to another script package. It solved problems like making lists automatically update other lists or change focus when some item in one list is selected as well as updating multi selection choices based on previous selections straight forward and common Web application abilities that were otherwise not possible at all with the out of box list web part short of purchasing a third party version.
More commonly, using hidden Content Editor and then Script Editor web parts, javascript could literally change the behavior of the entire SharePoint page, one page at a time of course, such that SharePoint administrators could hide edit and selective settings features without permission changes that normally turn off or on, the entire set.
It eventually got very advanced by 2013 and even usable on SharePoint Online in the use of the JSLink functionality which effectively added a script to run when a page ran so the need for a potentially removable Content Editor or Script Editor Web Part was removed. These advancements also came with the rise of the Client Side Object Model (CSOM) abilities originally targeted for Add-ins but also useful as REST calls form page based scripts calling the API or more directly from the active page resources.
The Add-In model discussed later came in between various iterations of JavaScript Injection patterns though it almost seemed at the later stage that the could be the answer to dealing with a large domain of SharePoint solutions, it still lacked some very important pieces that are addressed later. First and foremost was the lack of a manageable deployment story … there pretty much was none and there were very few ways to automate copy pasting code into a content editor or script editor web part and there were no methods until only just recently for posting JSLINK style page updates and keep in mind that those scripts are NOT compatible with the new "Modern Pages" feature of SharePoint Online. With zero access to the server side even via REST (because you could not effectively "hijack" the user session to change context to an "administrator" to do any useful work), the solutions here were limited.
The Sandbox Solutions
Sandbox solutions debuted with SharePoint 2010 to mitigate the "IIS RESET" problem and have the following characteristics:
- Server Side (Partially Trusted) execution model ("Declarative" solutions are the only ones of this type that work on SharePoint Online)
- Programs built with Visual Studio (there was no good way to do these by hand as there are a slew of steps done by the IDE to package it as a Sandbox solution as well as deploy it automatically)
- Scopes to the Site Collection (not the whole Farm but to more than a page)
- Used to build Features and Declarative Solutions (Declarative solutions are XML files that "declare" what should happen to, say, a list or page or site … configuration changes, effectively)
The Sandbox solution appeared to fulfill the promise but the limitations compared to many other solutions being designed in that time probably held back what would have been a wave of new development with a new system. Sandbox solutions did not force the servers to reset nor could they bring down the farm if they run into integration issues (although they would still bring down the site.) Sandbox solutions could be enabled and disabled by non-technical administrators and the concept of the sandbox solution was something way more manageable compared to JavaScript injection and could even benefit from Continuous integration techniques and clever DevOps Patterns. They were still, however, brittle enough to be affected when Microsoft made changes (hence why only Declarative solutions are allowed in the cloud) and they still had no way of truly leveraging the power of the built in components or perform any pure client side customizations that still require JavaScript Injection.
Microsoft announced that Sandbox solutions were to be deprecated from SharePoint as soon as SharePoint 2013 came out which made it one of the shortest lived solution attempts around but although deprecated, it does not mean it is not supported at least up until the next version of SharePoint or when the last of the "declarative" features are migrated over to the newer SPFx framework being discussed shortly.
Provider Hosted Add-ins (originally Apps)
SharePoint 2013 introduced a model that appeared to not just solve the IIS RESET Problem but literally allow any kind of solution to be a SharePoint application or add-in. The Characteristics of an Add-in are as follows:
- Client or Server side execution models - Trust handled in a very different way in that the REST Calls to the server work in the context they require while the client side work is concerned only with itself
- Client Side API Available - CSOM calls and other techniques allow for an Add-in to appear and behave very much like a native SharePoint feature
- The Execution context is externalized from SharePoint because it runs within an IFRAME
- Scopes to a Site or Tenant in the case of SharePoint Online (and not the Farm!)
- Used to build Web Parts, Event Receivers, Feature Receivers, timer jobs, workflows, or literally anything the developer wants to do that does not have to have anything to do with SharePoint
In essence, Microsoft created a framework where through the deployment of a small declarative file, an application built to run on a completely external system that could run in any operating system or be located anywhere in the world could integrate with SharePoint such that it can leverage important aspects such as Context Permissions and perform actions upon the SharePoint system using the more advanced APIs for the Server and the Client side.
The Sky is the limit here but with all that high flying, left behind below are some important abilities which still brought in the need for JavaScript Injection or even traditional web Part Development because in the end, the application or Add in is still EXTERNAL and if we take something important to web development as, say, responsive design, the issues of being in an Iframe and the logic behind handing responsiveness in that situation alone can make the solution stick out like a sore thumb or fail to meet requirements by taking away some of the reasons SharePoint was selected in the first place. I personally have heard many requests to "mimic" the SharePoint basic list functionality in a Provider hosted add in which would require completely re-inventing that wheel at the least since you cannot take something from SharePoint and put it inside your External Application REST calls will not render a List or its menus or even the page beyond the ability to Provision is before it renders.
Enter the SharePoint Framework (SPFx)
The SharePoint development story may not end here but the latest and perhaps insightful move by Microsoft clearly driven by their full embrace of cloud technologies and the web application development that supports it is known as SPFx and is an open source initiative from the get go spawned right beside a major Patterns and Practice Initiative to ease deployment and change management. The Characteristics of the SPFx are as follows:
- Client Side execution model - with trust purely IN CONTEXT and discussed shortly
- Executed and Rendered in the context of the page - it is not a separate independent piece or IFRAME
- Developed by design using Open Source Tooling versus Visual Studio
- The Defacto Development for the Microsoft Development team and for third party (us) teams
- Responsive and Mobile Friendly out of the box because it is in the same page context using the same directives and frameworks
- Limited Compatibility with on premise version of SharePoint starting with 2016. On going updates will slowly bring in features that appropriately align but other abilities will wait until the next version (tentatively SharePoint 2019)
- Scoped to the Tenant or Site Collection using a flexible deployment model that allows for assets to be hosted within the SharePoint online tenant (either in the App Catalog, a default Office 365 CDN storage space or SharePoint folder) or on an external resource (e.g. Azure or AXS CDN)
- For Creating Client Side Web Parts and Page/Site Extensions
- Ability to leverage the exact same component models of most SharePoint Online and Office 365 applications via the Office UI Fabric
Although seemingly important abilities such as workflow or event receiver abilities are, by design, not part of the model, the entire concept has forced a true re-evaluation of what a solution for SharePoint should look like. If the Event receiver or workflow was created to apply business logic to a form entry or other event, why aren't more powerful and more Out of Box systems from MS Flow to Azure Jobs and Functions in user to deal with those in the proper and more appropriate layer which is fully independent of the underlying system whose own change control is not at all available? What reasons would a client side web part every have to change context from the user to an administrator in the same potentially hijack-able process if the concept of services running in their appropriate domains can apply using modern cloud applications? Perhaps one might find that nuanced case especially, perhaps, in the big data area where full integration with Power BI is not fully delivered but even in those cases you may question whether the rush is worth the effort over the wait.
Those concerns aside, the newest development model takes a step towards global standards and is bringing more traditional (if tradition goes back just a few years, that is) web developers into the resource pool because the web tool chain is something ubiquitous to Windows, MAC and Linux systems to name a few. Examples and Tutorials for the new model are actually available in those various platforms and the choice of Node, Gulp and React in addition to the other supporting pieces means that the SharePoint Developer as we know it really becomes an Office Web Developer where they have just another slot in a large utility belt.
Decision factors
This posting has been concerned with showing the various development patterns that are all still in use and potentially applicable along with information that should help guide discussions and choices. Though I am aware that it appears to skew towards the latest and greatest pattern, the concept of "your mileage will vary" applies to your specific situation since unless you are the final decision maker, the best that can be expected is that you make the case with as comprehensive a set of facts as possible and to that end I hope you have a chance to dive into these topics much deeper especially case studies and samples widely available in the public domain.
Note – this post was updated on April 25, 2018 to reflect changes introduced to the framework that may have significant impact in planning and decision making coinciding with important releases ahead of the SharePoint North American Conference coming soon. Please consult the definitive documentation site at https://docs.microsoft.com/ for the latest information before proceeding on any related initiatives.
Links: