SharePoint Development – SharePoint 2010

This post is the third in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look the SharePoint 2010 release.

More posts in the series:

• Part 0 - SharePoint Development - Past, Present and Future (the series introduction)
• Part 1 - SharePoint Origins (including 2001 and 2003 releases)
• Part 2 - MOSS 2007
• Part 3 - SharePoint 2010 (this post)
• Part 4 - SharePoint 2013
• Part 5 - SharePoint 2016
• Part 6 - The Future of SharePoint Development

 

SharePoint 2010

The 2010 release built on the growing popularity of SharePoint following MOSS 2007, and it was clear that Microsoft had ambitious goals for SharePoint. Notably, the Microsoft CEO Steve Ballmer referred to it as "like an operating system" in the keynote presentation at the Microsoft SharePoint Conference in 2009. This release was seen as positioning SharePoint as a business collaboration platform that would fill all the needs of an information worker. As a result, there was huge marketing drive for the SharePoint 2010 that targeted end users. These changes were reflected in the updated functional pie chart, from the emphasis on IT Solutions in SharePoint 2007, to the emphasis on end user and business solutions in SharePoint 2010.

Comparison of SharePoint 2007 and 2010 features (taken from https://nikpatel.net/2010/02/14/sharepoint-2010-end-users-improvements-and-new-features-overview/)

In this release, SharePoint was positioned separately to both Office and Windows brands, with references to both removed from the version names. There were 3 versions:

• Foundation (free, corresponding to WSS in the earlier releases)
• Standard (with additional search functionality)
• Enterprise (additional content management capability)

SharePoint 2010 was launched globally in May 2010. There were a large number of new features in this release:

• An enhanced user interface:
  • Introduction of the fluent ribbon to mimic the changes to the Office UI, minimizing the number of mouse clicks required
  • Multi-browser support
  • Use of Wiki pages rather than web-part pages in templates
  • New themes to customise a site's appearance
  • Mobile Device Access (to render mobile views of the default SharePoint templates)
• Improved ways of integrating with existing business data
  • The InfoPath Forms service was updated to allow customization of the default SharePoint list forms
  • Introduction of PowerPivot, a special Excel workbook that can be shared and updated in a SharePoint site, as part of a drive to add Business Intelligence capability to SharePoint
  • Two way Business Connectivity Service (BCS , formerly the Business Data Catalog in MOSS 2007) to read and write data from external systems
    • Use of External Content Types to define external data sources, and to allow creation of external lists that can be used in the same way as SharePoint's own lists
• Improved workflow options
• Updated user profile and introduction of social networking features:
  • User profiles now included photos and presence indicator, status updates
  • Tagging of both user profiles and content
  • Users could now create their own blogs
  • Colleague suggestions
  • An Organisation chart to show a user's place in the organisation hierarchy
• Records Management (retention and auditing policies, records repository, email content as records, legal holds)
• Improved integration with Microsoft Office, including the ability to open content and edit in SharePoint directly in Word and Excel
  • Office Web Apps (OWA) was introduced as an add-on to SharePoint as the online companion to Word, Excel, PowerPoint and OneNote applications that enables users to access and edit documents from anywhere. This allowed for the replacement of the traditional desktop Microsoft Office clients.
• Improved governance and IT management tools, to control what end users can do with SharePoint. Notably, this was to prevent duplication of content and to also prevent users creating sites without any oversight, as these were major problems with the MOSS 2007 release.
• Improved Content Management
  • Document Sets
  • Shared Content Types
  • A new Managed Metadata Service was introduced to allow for tagging of content, and the use of taxonomy hierarchies for implementing logical data architectures
  • Rich Media management

This is by no means a comprehensive list of the new features in SharePoint 2010 (but you can find a full list of the new features and comparisons to previous versions here, here and also here).

Developing for SharePoint 2010

The SharePoint 2010 release involved massive changes for developers. This wasn’t just an incremental release on top of the previous 2007 release. There were significant architectural changes, mainly aimed at removing the scalability and performance issues that had limited MOSS 2007 as an enterprise platform. Notably, Microsoft had experienced these problems when managing their online SharePoint service, Business Productivity Online Services (BPOS). These architectural changes included:

• Moving from a 32-bit to 64-bit architecture, giving a significant performance boost. This also meant that upgrading from MOSS 2007 involved significant changes to the hosting infrastructure.
• The replacement of Shared Service Providers by Service Applications to share resources across sites in different web applications and farms. The new service application architecture allowed for the use of dedicated application servers, and also lent itself to multi-tenancy scenarios, which was particularly important for Microsoft’s online offering.
• Improved the boundaries and thresholds for scalability, including:
  • Increased the list view threshold from 2000 (in MOSS 2007) to 5000 (20,000 for administrators)
  • Increased the number of content databases supported per web application from 100 to 300
  • Maximum content database size increased from 100 GB to 200 GB
  • The full list of boundaries and limits for SharePoint 2007 can be found here, and for 2010, here.
• The introduction of the Sandbox Solutions to allow site collection administrators to monitor and control the resources consumed by web parts and other  solutions in a site collection. Sandboxed solutions were hosted in the new SharePoint user code solution worker process (SPUCWorkerProcess.exe), and the code could only affect the performance of the site collection it was deployed to. As the code was not run in the IIS worker process (as in the full trust farm solutions that had previously been used), the sandboxed code could not affect the performance or stability of solutions in other site collections. This had been a common issue for Microsoft when running their multi-tenancy BPOS solution – poorly coded web parts in one tenant’s site collection would affect the availability of other tenants solutions running on the same web application. The new Sandbox Solution framework were created to solve this problem. Microsoft was so insistent on Sandbox Solutions being used by developers that they made them the default solution type when creating a new SharePoint solution in Visual Studio, and SharePoint Online (the BPOS successor, see below) would only accept Sandbox Solutions.

The changes in the architecture are highlighted by below between the two diagrams:

SharePoint 2007 Architecture

SharePoint 2010 Architecture

In addition to the Sandbox Solutions, Microsoft had a major push to educate developers on common programming pitfalls and performance issues when developing SharePoint solutions, and released the SPDisposeCheck tool (a static code analyser) to help identify poorly coded solutions.

There were a number of important changes to improve the administration of SharePoint:

• The Central Administration application interface was updated, including the introduction of the ribbon and wizards to step through common configuration tasks (including the initial configuration of a farm).
• In addition to the STSADM executable, PowerShell was introduced to provide command-line administration. As an early adopter of PowerShell, I particularly welcomed the addition of PowerShell to the SharePoint developer’s toolbox.
• Managed Accounts were added, allowing for the management of service accounts directly in Central Administration, instead of having to manage these accounts directly in the AD.

The tooling and best practices for developers changed radically with the release of SharePoint 2010. Notably, there were now 3 separate APIs available to develop against:

1. Server-side Object Model – the existing .NET API, updated to use .NET Framework 3.5.
2. Client Object Model – this was a new API designed to allow developers to produce solutions for use in remote client-side solutions that run on computers where SharePoint 2010 is not installed. This included a Client Class Library (for use in  .NET managed or Silverlight applications) and a JavaScript class library (an unmanaged ECMAScript object model for use in the browser).
3. SharePoint 2010 Web Services- updated to have a RESTful interface.

The changes to the API model reflected the improvements in web development techniques and a new emphasis on creating rich Internet applications (RIAs) in the browser for an improved user experiences using Ajax and Silverlight. It also reflected the new emphasis on SharePoint as an enterprise platform – the new APIs allowed integration with other enterprise systems.

In addition to the new APIs, the Business Connectivity Services (BCS), the successor to the Business Data Catalog, had significant new features to connect to external data sources outside of SharePoint and literally treat them as a SharePoint List. These connections were now two way (read/write) connections. Out of the box, the BCS could connect to SQL databases and web services, and also provided a pluggable Connector Framework to allow connectors for new external systems to be developed.

Similarly, the User Profile service was revamped in SharePoint 2010, with the associated synchronization service also made a two way connection, so that changes made in a SharePoint user profile could be written back to an organisation’s AD. Personally, I came to dread dealing with it (and the associated Microsoft Forefront Identity Manager (FIM) engine). The initial documentation on TechNet was very poor, so I came to rely on Spencer Harbar’s excellent Rational Guide to implementing SharePoint Server 2010 User Profile Synchronization.

For developers, there were significant improvements in the tooling for SharePoint 2010, including the introduction of a number of different SharePoint project templates in Visual Studio 2010. It was now possible to deploy and upgrade features and solutions from within Visual Studio, instead of having to rely on third party tools and extensions.

Developers were encouraged to start using the new LINQ to SharePoint provider, building on the popularity of the LINQ to SQL ORM that was released in 2007. LINQ (Language Integrated Query) is a .NET Framework component that adds native data querying capabilities that was introduced with C# 3.0. The LINQ to SharePoint provider promised to abstract away the complexity of writing CAML to query SharePoint lists. Sadly, the implementation of the first release was poor – you can read the details in Bjørn Furuknap's article. There were also a number of other open source libraries that provided similar functionality, such as Camlex.NET.

Accessibility was improved in SharePoint 2010, after being famously dire in MOSS 2007. This had been a major issue in the take-up of SharePoint solutions by UK public sector organisations who had a duty to provide accessible websites to their users. SharePoint 2010 was marketed as being WCAG 2.0 AA compliant, and also standards (W3C XHTML) and cross browser complaint. In reality, while still a major step forward compared to MOSS 2007, SharePoint 2010 was not fully compliant to the WCAG 2.0 AA standard. I detailed the issues in a past article (the article is no longer online but is still available via the Internet Archive), but to develop a fully accessible web application using SharePoint 2010 required a significant developer effort, using custom control adaptors and JavaScript.  SharePoint 2010 could now support Firefox, Chrome and Safari in addition to Internet Explorer, but the differences in browser support for various features and web parts (mainly due to the use of ActiveX) caused a lot of work for developers. Many organisations were also forced to work around the lack of (official) support for Internet Explorer 6.

Other notable changes for developers included:

• In addition to the change to a 64-bit architecture already mentioned, SharePoint could also be installed on developer workstations (using Windows 7 or Vista OSs), instead of only on servers as previously.
• Improved search functionality with an emphasis on promoting information discovery across both structured and unstructured data. Additionally, the FAST search application was integrated into SharePoint only 18 months after Microsoft had acquired the company. Unfortunately, there were 3 separate search options available in SharePoint 2010 (Basic Search Center, Enterprise Search Center and FAST Search) which was needlessly confusing. This was finally resolved in the later SharePoint 2013, when the FAST search become the standard search engine in SharePoint 2013 and the SharePoint Online.
• Addition of web templates and provisioning providers, as well as updates to the existing SharePoint deployment approaches, site definitions and site templates (detailed here).
• The introduction of claims based authentication, using the Microsoft Identity framework (code name Geneva), allowing non-AD users to be authenticated (such as users using Hotmail and Gmail accounts).
• Major improvements in the information architecture, with improved ways of adding metadata to documents with managed terms and keywords.
• Introduction of Remote Blob Storage (RBS) to store large binary files such as video.

I don’t have space or time to do justice to these new features, or the many others that I have failed to list. But even from this brief overview, you can see that SharePoint was fundamentally rebuilt and extended with the 2010 release.

BPOS/SharePoint Online

As already discussed, many of the changes made in SharePoint 2010 were made to improve Microsoft’s BPOS offering. This led to to an identity crisis that Microsoft still hasn’t fully resolved - is SharePoint an online service or on-premise software? While Microsoft’s direction of travel is now clear (more on this later in the series), the contradictions between these two views of SharePoint have certainly affected how developers see the platform. However, it has been evident that SharePoint Online has contributed significantly to driving new features in SharePoint and in also reducing the product release cycle time.

With the development of Sandbox solutions, custom code could now be deployed to SharePoint Online. However, the feature set for SharePoint Online lagged considerably behind SharePoint 2010 on-premise. Following the release of Office and SharePoint 2010, the BPOS services were refreshed and re-launched as Office 365 in October 2010. Microsoft chose the name ‘Office 365’ to reflect "dependability every day of the year", due to BPOS’s very poor record of uptime.

Existing BPOS customers were given 12 months to upgrade to the new online service, with possibly complex migrations involving updating the client Office applications. This was an issue for many existing BPOS customers who were predominately small or medium-sized businesses. The demand for BPOS/Office 365 was seen as low, and the uptake for SharePoint Online at the time was even lower.

Conclusion

As described above, SharePoint 2010 was a massive improvement from MOSS 2007, both in terms of features for end users and organisations, and also for developers. But there were some notable failures. SharePoint attempted to do everything, but did very few things well. The new blogging and wiki features were underwhelming. Similarly, the accessibility story fell considerably short of the marketing hype. I recall repeatedly having to spend considerable amounts of time developing SharePoint solutions that were fully WCAG 2.0 AA compliant for UK public sector clients.

SharePoint 2010 was widely marketed as a suitable platform for Enterprise development, and I worked on a number of large scale projects (with 20+ development teams) to develop various platforms. Some were successful, most were not. One notable failure I recall was an attempt to develop an ecommerce platform on top of SharePoint 2010. This was a clear case of believing marketing hype over reality, as SharePoint was never suitable for developing this type of solution.

With the huge growth in SharePoint usage (detailed in my previous post), a large number of ASP.NET developers started to develop solutions in SharePoint, with mixed results. At one point in my career, I was employed in a large SharePoint team in a major consulting firm that was responsible for cleaning up SharePoint solutions gone wrong. Without fail, the root cause of these failures were architects and developers attempting to develop SharePoint solutions without prior experience. Reading one of the many developer books published on SharePoint 2010 didn’t make you a SharePoint expert, though I came across many who believed it did.

One common anti-pattern was attempting to use SharePoint as a relational database, with developers using SharePoint lists as data tables for massive numbers of records. Another was attempting to install and configure SharePoint using only 1-2 service accounts, instead of the 15-20 accounts actually required. While fine for creating a developer environment, this would cause no end of problems in Production, particularly with the User Profile Synchronization Service. And yet another common failure was re-using service accounts used in an existing MOSS 2007 farm for your new SharePoint 2010 installation – this would always result in constantly locked accounts when password changes failed to synchronize across the two farms.

Despite the major improvements in SharePoint 2010, I now see the release as damaging Microsoft’s relationship with developers. While the new APIs were significant, Microsoft’s push to have developers build Silverlight clients for SharePoint would rebound when the company would side-line Silverlight in favour of HTML5 towards the end of 2010. Likewise, the LINQ to SharePoint provider was killed off along with LINQ to SQL, when Microsoft settled on Entity Framework as their chosen ORM.

However, probably the biggest setback was the introduction of Sandbox solutions. Introduced to overcome performance and instability issues introduced by badly coded farm solutions, particularly in Microsoft’s BPOS service, they were made the default solution type in Visual Studio. However, these solutions offered fewer features but were significantly more complex to implement, especially when attempting to use APIs that were not by default supported within the sandbox (full-trust proxies and the dreaded service locator, anyone?). As a result, very few developers bothered to use them.

The introduction of Sandbox solutions was clear case of Microsoft’s SharePoint identity crisis – was it online or on-premise, service or software? Some very quickly came to realise that the future of SharePoint was to be subsumed in the Office 365 brand. At the time, despite being aware of the major architectural changes going on, I tended to sneer at SharePoint Online as a toy offering, given the very low take-up of the service.

These and other issues (like the completely unnecessary debate over the future of .NET in Windows 8)  would lead to SharePoint developers becoming reluctant to adopt new frameworks and features being touted by Microsoft - more on this trend later in the series.

In the next post in the series, I’ll look at SharePoint 2013. Apologies for the delay in publishing this series – the combination of the Christmas holidays and the baby has meant that I’m running significantly behind in my blogging schedule. Also, thank you to all those who have sent emails about this blog series. All comments or feedback are gratefully received.