SharePoint Development - SharePoint 2013

This post is the fourth in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look at the SharePoint 2013 release.

More posts in the series:

• Part 0 - SharePoint Development - Past, Present and Future (the series introduction)
• Part 1 - SharePoint Origins (including 2001 and 2003 releases)
• Part 2 - MOSS 2007
• Part 3 - SharePoint 2010
• Part 4 - SharePoint 2013 (this post)
• Part 5 - SharePoint 2016
• Part 6 - The Future of SharePoint Development

 

SharePoint 2013

The SharePoint 2013 release was considerably less ambitious than the 2010 release. It was relatively minor upgrade from an IT perspective, with no major architectural changes. This wasn’t a major surprise. SharePoint was by this stage a mature and robust product that had been around for over 12 years, and was, by 2013, used by some 100 million users and 80% of Fortune 500 companies. The 2013 release was simply an incremental change on the existing software that focused on improving usability for end users, particularly around the social experience.

The different versions available for SharePoint 2103 remained unchanged from SharePoint 2010 (Foundation, Standard and Enterprise). The first technical preview of SharePoint 2013 was released in early 2012, and the public beta was released in July 2012. The public release of the finalized version was on 11th October, 2012.

The main features in this release were:

• A major redesign of the user interface, including:
  • Improved cross browser support
  • Drag and drop documents from your desktop into SharePoint libraries, and also to move content within and between SharePoint libraries (also cross browser)
  • Ability to create and edit lists without leaving the page you’re currently editing
• Sync document libraries to your computer using OneDrive for Business (previously known as SkyDrive Pro, it was simply an updated version of the Groove tool).
• Additionally, the MySites feature was rebranded as ‘OneDrive for Business’. In fact, the existing MySite feature was kept, and each MySite now came with a Document Library specially built for the OneDrive for Business tool. As end users unsurprisingly associated it with Microsoft’s OneDrive cloud storage solution, this caused endless confusion. Chalk up yet another win for Microsoft Marketing.
• Improved task management features, such as:
  • Updated Project Site template, and an improved Tasks list that now included a timeline and Gantt views.
  • Ability to assign multiple resources to tasks, and to create subtasks
  • Improved integration with Microsoft Project
  • Option to view all tasks (across the entire SharePoint farm)  assigned to you in your personal site, without the need for custom or third party solutions.
  • Tasks could also be synchronized with Outlook, and could also be pulled in from other providers (such as a CRM or TFS) using the extendable framework provided.
• The search features in SharePoint 2013 were considerably improved, with an eye to the search features used in Google that users were already familiar with. These new features included:
  • Type ahead (auto-complete) when entering search terms
  • Previous search terms are now displayed as query suggestions at the top of the search results page
  • Use of Boolean operators in search queries
  • Wildcard searches
  • Quick preview of documents within the search results, allowing the user to preview the document without leaving the results page.
  • Additionally, users could now see how many times an item has been viewed from within the search results
  • A new one click option to view search item in their parent library 
  • Users could follow, and be updated about changes made to, a document from the search results page
  • The new continuous crawl meant that there was no lag period following the addition of new documents to the documents appearing in the search results
  • Improved discussion list, with a more ‘Facebook’-like activity feed.
• The most important search related improvement was the ability for a piece of content to be surfaced in different locations, across site collection boundaries. This included a number of new category of web parts called ‘Search-driven content’, where the web parts were all pre-configured variants of the Content by Search Web Part (e.g. “Items Matching a Tag”). Similarly, any list/library in SharePoint could be nominated as a ‘catalog’ and then shared across site collections as part of the ‘Cross-Site Collection Publishing’ Feature.
• Improved social features. The social networking features in SharePoint had been poor, and organisations were forced to use third party add-ons, like Social Sites from Newsgator. The 2013 release had a number of improved social features:
  • The introduction of micro-blogging with (searchable) #hashtags and @mentions
  • Ability to follow and be updated about changes to people, sites and documents, as opposed to the older style alerts
  • New Community site template focused on conversations between users. This was very different to the existing team sites - users could observe conversations and then “join” a community when they want to post. The user’s photo was shown beside all posts they had made, and there was gamification (ratings, badges) to encourage participation
  • An interactive newsfeed to aggregate and follow a user’s activities across all their sites, and to view the activities of users and content that they follow
  • One click sharing to easily share documents and sites with others
• Mobile
  • In SharePoint 2010, it was possible to create a mobile views for sites, but it was very constrained, and didn’t allow for any site customization for specific mobile devices or tablets. With the SharePoint 2013 release, the mobile browser experience was optimized, and allowed the creation of device channels to render SharePoint content in a design for a specific device type
  • SharePoint Newsfeed apps were released for Windows Phone and iOS devices, and later for Windows 8
  • SkyDrive Pro (later re-branded as OneDrive for Business) applications were also available for Windows Phone, and later for Windows 8 and iOS, allowing for offline viewing of SharePoint lists and document libraries
• Business Intelligence
  • The BI options in SharePoint 2013 enhanced the existing features, such as Excel integration and the PerformancePoint service (including a new Dashboard designer)
  • Improved support for rendering dashboards on iPads and Windows 8 tablets
  • A new Business Intelligence Center site template to manage reports, scorecards, dashboards, and data sources in a single location
• Miscellaneous
  • OneNote integration with team sites (when configured with Office 365)
  • Improved integration with Microsoft Outlook
  • Improved support for playing video (HTML5 and Silverlight players) and searching for videos
  • New eDiscovery (electronic discovery and audit) site template
  • Use of Metro styling for the UI

As always, this is an incomplete list, but it covers the major improvements in user features for SharePoint 2013.

Developing for SharePoint 2013

As already noted above, there were no major architectural changes in SharePoint 2013. Despite this, the upgrade process from SharePoint 2010 to SharePoint 2013 was still problematic, due to deprecated features and new or modified features (such as the updated discussion forums) which could cause existing SP2010 solutions to stop working following the upgrade. Additionally, for this release, there was only one supported path, from 2010 to 2013; you could no longer skip a release and go from 2007 to 2013.

The visual and in-place upgrade options were discontinued in this release, with only the database attach upgrade method being now supported. However, there was now an option to retain a content database as a 2010 content database and to perform a “deferred site collection upgrade”. In practice, this means that the 2010 site collection continues to be a 2010 site collection, just running on a 2013 farm. The solutions for the 2010 site collection continued to be deployed to the 14 hive, while solutions for 2013 site collections were deployed to the new 15 hive. In theory, the existing 2010 sites would continue to work when migrated to the SP2013 farm, including your customizations. When you performed the deferred site collection upgrade, the site collection now uses the 15 hive, which is when the custom solutions might break. As someone who has recently upgraded a major SharePoint farm to 2013, this was a nice feature that allowed the migration to proceed in phases as the existing 2010 customizations were made 2013 compatible.

At the time, Microsoft stated that SharePoint 2013 would be the last version released on a 3 year upgrade cycle. In future, both the on-premise and online versions will be updated much more frequently, with SharePoint Online to be upgraded every 90-120 days. Future releases of the on-premise release would simply be snapshots of current SharePoint Online.

The main changes to the administration of SharePoint were:

• Improvements in the ability to manage multi-tenancy deployments (again, based on Microsoft’s experience of running the SharePoint Online service)
• Significant improvements in the User Profile Synchronization Service with the re-introduction of the AD Import (faster than the User Profile Sync), faster performance for full synchronizations, and the ability to sync with more directory services (such as the Java System Directory Server, Novell eDirectory and IBM Tivoli). The User Profile Sync was still difficult to setup, but it was considerably improved on the 2010 release.
• Introduction of AppFabric to give a distributed, in-memory object cache to scale out high-performance .NET applications. AppFabric was installed as a prerequisite for SharePoint 2013 and could only be configured via PowerShell. While an important new feature, it was difficult to configure SharePoint’s Distributed Cache Service (a wrapper around AppFabric) to work across a high availability SharePoint farm with redundant servers.
• Request Management to control how the SharePoint farm handles incoming requests by evaluating logic rules against them in order to determine which action to take, and which machine(s) in the farm (if any) should handle the request. This wasn’t to replace load balancers, but to provide more flexible configuration options for the SharePoint farm.
• Internet Explorer 7 was no longer supported
• Various improvements in the Web Content Management (WCM) features, including:
  • Ability to copy and paste directly content from Word, without having to strip out the formatting
  • Improved support to upload and embed videos in pages, including a new video content type, and the auto-generation of thumbnail preview images
  • Ability to insert iframe elements into a page, allowing external content to be embedded
  • Improved support for multilingual sites
  • The introduction of cross-site publishing, allowing specified site collections to be published to other publishing site collections. Also, the new Product Catalog (simply terrible naming) site template allowed you to create lists and libraries whose content could be published to other sites
  • Introduction of managed navigation, new feature that allowed site admins to define and maintain the navigation on a site collection by using managed metadata term sets
  • Improved SEO for public facing websites, such as user friendly URLs, allowing verification of site ownership, the ability to edit sitemap settings (such as the robots.txt) for search engines, and the ability to create SEO meta tags.
  • Improved branding options, allowing the use of standard HTML, CSS and JavaScript designs and workflows to brand SharePoint, rather than constraining designers to using only Visual studio or SharePoint Designer
  • As mentioned above, the introduction of device specific channels
• Related to the above improvements in WCM, SharePoint Internet sites no longer required an expensive licence. The use of cheaper standard server licence for public facing sites meant that organisations could host their internet and intranet on the same platform, reducing costs and providing more flexibility.

The new features for developers included:

• The new App Model (see below)
• Major improvements in SharePoint search, with the core SharePoint 2010 Search product and Fast Search for SharePoint 2010 being merged and extensively overhauled into a single product called SharePoint 2013 search that was common between the Foundation and Standard versions.
• Introduction of Shredded Storage to store large files in the database as small shreds instead of a single BLOB (binary large object). The file shreds are combined into a single document when retrieved from the database. This aimed to reduce the I/O of updates to existing files (only the file shreds that are modified are uploaded). So if only a file’s metadata is updated, the file shreds are not updated (in SharePoint 2010, a copy of the document’s BLOB was created). Unfortunately, this feature was poorly implemented, and could result in increasing rather than decreasing file storage in many common scenarios, as well as increasing the time required to retrieve documents from SharePoint. Additionally, this also caused some issues for admins, as they could no longer retrieve documents directly from content database backups as was possible in SharePoint 2010.
• Addition of OAuth authorization (used in the new App Model)
• As already mentioned above, the introduction of Device Channels for mobile/tablet specific page rendering.
• The JavaScript Link property (JSLink) that allows the user interface of fields and lists to be modified using a JavaScript file, as opposed to using modified XSLT views as in previous versions. A small but very useful change that allowed easy customization of the UI.
• Introduction of the Minimal Download Strategy (MDS) to download only the differences between pages when navigating between them. This improves rendering performance when browsing content where large parts of the page do not change. MDS is enabled by default on team and community site collections.
• Support for HTML5
• Significant improvements to Workflow. As well as being backwards compatible with SharePoint 2010 workflows (based on Windows Workflow Foundation 3.5), SharePoint 2013 used the new Workflow Manager service, based on the Windows Workflow Foundation in .NET Framework 4.5. The new Workflow Manager is external to SharePoint, and can run on completely separate server(s), allowing for greater performance and scale. The Workflow Manager and SharePoint WFEs communicate using a REST API secured by OAuth. Note, it isn’t possible to automatically upgrade from older SharePoint 2010 workflow to the new workflow types.

App Model

For developers, the main change with the release of SharePoint 2013 was the introduction of the new App Model (later renamed the Add-in Model). Previously, developers wrote custom server-side code that was run within the SharePoint process (either as a fully trusted solution or within the sandboxed user code service). SharePoint Apps, however, are run external to the SharePoint process, running within the browser as a client-side solution, or remotely on some other server or service external to SharePoint.

What did this mean in effect? Developers would no longer develop SharePoint solutions using C# to run on top of SharePoint, but would instead develop apps (essentially web applications) using HTML5, CSS and JavaScript using current web development best practices. The apps would be loosely coupled with SharePoint, consuming data using the SharePoint CSOM, JSOM and REST APIs, rather than using the server side API. These new apps would be compatible across both on-premise SharePoint and SharePoint online instances, and would provide users with ‘No Code’ (as in no server side SharePoint code) customization solutions.

The new App Model had 3 separate deployment models:

• SharePoint Hosted App – the solution components are hosted on either an on-premise or Office 365 SharePoint farm. SharePoint hosted apps are installed on a SharePoint 2013 website, called the host web. They have their resources hosted on an isolated subsite of a host web, called the app web.
• Provider Hosted App -  also known as a self-hosted app, this solution include components that are deployed and hosted outside the SharePoint farm. They are installed to the host web, but their remote components are hosted on another server. The separately hosted components are typically an ASP.NET application, but they can be developed in any server side technology (such as Node.js, PHP) that can use OAuth and consume the SharePoint REST API.
• Automatically Provisioned Azure App – also known as auto-hosted apps, these were cloud-hosted apps whose remote components are provisioned and deployed on Windows Azure. As with the provider hosted app, it could interact with a SharePoint website but also uses resources and services that are located on a remote site that is hosted by Windows Azure. This deployment option was only available to SharePoint Online customers.

As app stores were all the rage (Apple’s App Store, Google Play for Android, the Windows Phone Store, Windows 8 Store), SharePoint also got one (the Office Store). Corporations got an internal app store, the internal app catalog where only apps approved for use within the organization were available.

SharePoint Online/Office 365

The SharePoint Online service was updated to use the SharePoint 2013 version at the end of February 2013. Initially, it was identical to the on premise version. However, Microsoft quickly started to add new features to SharePoint Online/Office 365 that were not available with the on-premise installations. These unique features included Yammer integration (for Enterprise subscriptions), and then, later, Delve, a personal search and discovery tool (think Cortana/Sir for work). The quick iterative nature of the updates to Office 365/SharePoint Online reflected the new 90-120 day release cycle that Microsoft was using. Other changes for SharePoint Online included the new App Model and the Office Store mentioned above.

The use of a hybrid architecture, with some SharePoint sites and services running in Office 365/SharePoint Online, and some on-premises, gained popularity after the release of SharePoint 2013. In particular, many organisations opted to host their MySites with SharePoint Online, while retaining the majority of their data in an on-premise farm. For those organisations, this removed the need to police the contents of an employee’s MySite and meant they could simply offshore the problem to Microsoft’s data centres.

Conclusions

For end users, SharePoint 2013 was a major improvement with its improved UI and focus on usability. For those administering SharePoint, there were significant improvements,  particularly the new Web Content Management features. For developers, however, the release was more of a mixed bag.

The notable improvements were the introduction of JSLink, and the new Search and Workflow engines. Unfortunately, there were quite a few ‘meh’ features, including the new Shredded Storage which had a sloppy implementation. Another was Device Channels – why base this feature solution on User Agent strings, and completely ignore CSS3 media queries? Similarly, HTML5 support was great, but being able to build components for SharePoint with ASP.NET MVC rather than legacy Web Forms would have been even better.

In addition, I felt in this release that there were several major failings. The first, and the most surprising given that the trouble that Microsoft had went to to improve the UI and to introduce Device Channels for mobiles and tablets, was that there was no official Microsoft SharePoint app for iOS or Android (or even Microsoft Phone, not that anyone actually owned one…). There was the newsfeed app, but if you want a native app to browse and view content on a SharePoint site, you had to use third party offerings.

The second major fail was the new App Model. It sounds like a brilliant idea. No longer would developers be forced to use clunky old C# full trust solutions to customize SharePoint that would risk the stability and performance of the SharePoint server (and more importantly, Microsoft’s SharePoint Online service). Instead, they could deliver solutions across both on-premise and hosted SharePoint sites using the latest and greatest web development practices. Developers obviously rushed to use the new App Model to develop their SharePoint solutions, right?

Well, actually, no. In fact, practically no one used the new app model, as SharePoint developers could see it for what it was – a desperate attempt by Microsoft to solve the no server code customization issue for their SharePoint Online service. They had been burnt by Microsoft pushing Sandboxed solutions in SharePoint 2010, in a previous attempt to solve the same issue, and were understandably wary of jumping on another development model that could be killed off with the next release.

The App Model wasn’t helped by the fact that it was more complex to implement and deploy than full trust solutions, or that it didn’t provide the same range of customization options as either full trust or the more restrictive Sandbox solutions. In particular, it was unable to:

• Deploy resources to the SharePoint hive, such as master pages, application pages, CSS, JavaScript or image files
• Provide any delegate control elements or user controls (*.ascx files)
• Deploy custom Site Definitions
• Perform Custom Action Groups & Hiding Actions
• Deploy Custom themes
• Create a Timer job

Effectively, if it wasn’t possible to do something with the CSOM API, it wasn’t possible to do it with SharePoint Apps.

In time, with the help of the excellent Patterns and Practices project (PnP), some of these major gaps were (partially) filled in, but for some problems, you would still have to use full trust solutions. This meant that when customising for an on-premises solution, you were being asked to support 3 different development models: full trust code, Sandboxed solutions and the App Model.

Of the 3 different deployment types for the App Model, the Provider Hosted App was considered the most useful. Very few people used the SharePoint Hosted App due to issues with authentication and upgrading, and so few used the Azure Hosted Apps that Microsoft removed the deployment option within 2 years of release, stating scalability and manageability concerns.

In the end, most SharePoint developers simply ignored the App Model, despite all of Microsoft’s marketing. The ideas behind the App Model were correct. The SharePoint architecture and development methods were still based on ASP.NET 2.0 Web Forms, which was a decade old. Customers wanted SharePoint solutions that made use of web development best practices and frameworks, and a UI that was responsive. Unfortunately, the App Model simply wasn’t going to deliver that.

A third failing was the Yammer integration. While it was possible to integrate it with an on-premise SharePoint 2013 farm, there was never a clear reason to do so. Yammer never offered enough value to business to be used instead of the existing SharePoint social features.

The Hybrid approach was heavily promoted by Microsoft, as a way of introducing organisations to Office 365. But actually implementing a hybrid architecture was difficult, and was the most complex option (compared to simple on-premise or SharePoint Online solutions). Hybrid really only offered:

• Search across both environments (but with a very poor user experience, as the search results from the 2 environments weren’t merged together)
• BCS allowing access to data in on-premise and Office 365
• Duet integration allowing access to data in on-premises SAP from Office 365

There was no shared global navigation across the 2 environments, or a global site directory. The branding across the 2 environments could not be synchronized, and likewise, configuration settings for one environment were not replicated across to the other. The link between the 2 environments was really quite minimal.

A Hybrid architecture was, however, attractive to Microsoft, as you were effectively paying twice for the same solution – the licencing costs of your on-premise servers, and the monthly subscription for Office 365.

Overall, SharePoint 2013 was worth upgrading to, due to the real improvements for end users, and a number of notable new features such as the WCM improvements. Would the next SharePoint release, which Microsoft hinted would be available in under 2 years due to their new release cycle, be as useful? I’ll look at this in my next post in this series.