SharePoint Development - MOSS 2007

This post is the second in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look the SharePoint 2007 release.

Series Outline:

• Part 0 - SharePoint Development - Past, Present and Future (the series introduction)
• Part 1 - SharePoint Origins (including 2001 and 2003 releases)
• Part 2 - MOSS 2007 (this post)
• Part 3 - SharePoint 2010
• Part 4 - SharePoint 2013
• Part 5 - SharePoint 2016
• Part 6 - The Future of SharePoint Development

 

MOSS 2007

The 2007 release (actually released in 2006) attempted to build on SharePoint 2003, and to integrate it more closely with the Microsoft Office suite. This version fixed many of the shortcomings of the 2003 product but also vastly expanded the platform’s core functionality. A major focus was placed on improving document and records management, alongside web content and portals. This version had a fully functional and integrated content management system, a result of the feedback on Content Management in the 2003 release. This gave rise to the infamous pie diagram for the SharePoint 2007 release:

End users were given the option to navigate the platform to create team sites and manage workflows. As a result, SharePoint moved to become a more collaborative platform, and started to resemble the product we know today.

As in 2003, the product came in 2 different versions:

• Windows SharePoint Services v3 (WSS 3.0), a basic free version for Window Server.
• Microsoft Office SharePoint Services (MOSS 2007), the premium version built on top of WSS, with the new MOSS designation indicating the greater level of integration with the Office suite. This version was bundled with the Office 2007 suite and added enterprise search and people search, document repositories, additional Web parts, workflow, and content syndication. MOSS targeted organizations storing more than 500,000 documents (large enterprise customers).

The new features included with MOSS 2007 were:

• Business Data Catalog
• InfoPath Form Services
• Excel Services
• Content types
• Use of SharePoint Designer to develop portals (replacing FrontPage)
• Rights Management
• Shared Service providers
• Updated security model
• Improved administration (now create and configure IIS web sites via Central Admin)
• Improved authoring and site management experience
• Updated MySite UI
• Introduced minor versioning and document check out/in in document libraries for editing

 

Developing for SharePoint 2007

WSS 3.0 was built on top of ASP.NET 2.0 and IIS 6.0. Unlike WSS 2.0, requests to WSS 3.0 were routed through the ASP.NET runtime before reaching the WSS layer. This allowed the use of AJAX with SharePoint web parts and pages in WSS 3.0, as well as wrapping custom ASP.NET user controls (.ascx files) in web parts for deployment. Additionally, it allowed the use of  ASP.NET authentication (Forms Based Authentication). As portal sites no longer had to be created at the root of the IIS web site, hundreds of portal sites could be hosted inside a single IIS web site (in current SharePoint terminology, at the web application level).

SharePoint 2007 Stack Architecture (taken from http://dotnetslackers.com/articles/net/MOSS-2007-Architecture-and-Requests.aspx)

The stacked architecture was meant to encourage existing ASP.NET developers to start developing for SharePoint, as it meant that their existing skills and tools could be re-used for SharePoint development. However, developers new to the SharePoint platform still struggled to cope with SharePoint’s particular development methodology (such as features) and gotchas.

As a result of these fundamental architectural changes in the SharePoint architecture, upgrading from 2003 to 2007 was a painful  and laborious process. Additionally, updating custom site definitions (.stp files) when upgrading to 2007 was to be avoided - something that would be repeated in the next release as well. The use of custom site templates for application development, although available since the 2003 release, was heavily promoted in MOSS 2007 by Microsoft. This gave developers a means of creating custom templates for creating new sites in SharePoint with specific site or web features automatically activated. This led to a lot of messing around with GUIDs for features in onet.xml files, and using the Visual Studio Extensions for Windows SharePoint Services (VSeWSS) to package and deploy the custom templates.  In general, you could add custom site definitions easily enough, but it was very difficult to edit or delete an existing site definition.

Additionally, in MOSS 2007, an existing site could be saved as a site template (.stp file) for new sites. It is a package containing a set of differences and changes from a base site definition. It wasn’t as performant as the site definitions, as it was stored in the content database, while the site definition files were cached on IIS on the web front end servers. Site templates also weren’t as portable as site definitions, as to move them between farms, the farms had to be at the same version and patch level. This was because the site templates were dependent on the original base site definitions they were created from.

In WSS 3.0, features and farm (full trust) solutions were introduced. A feature is simply an XML definition for a component deployed to the SharePoint farm, with a specified scope (Farm, Web Application, Site and Web). Features were introduced in WSS 3.0 to address the failings in WSS 2.0 with regard to defining functionality used across multiple sites. Feature Receivers and Feature Stapling were also introduced, allowing modification of features on activation, and allowing features to be automatically enabled on specific site templates. Features were deployed in the new Windows SharePoint Services Solution Package (WSP) files that replaced the .cab file format used previously, and were deployed either using the SharePoint UI, or more commonly using the STSADM.exe tool (and a plethora of .bat files).  Again, I seem to recall a lot of messing around with DDF files to package and deploy web parts correctly.

In addition to all this, developers could now use:

• Content Types: metadata used to describe the attributes and actions applied to a particular category of information, and used  to develop a logical information architecture. Content types are in practice the schema definition of a site or list, and allow us to encapsulate a data schema and make it independent of a location on a SharePoint site.
• Integrated workflow and out of the box workflow templates
• InfoPath Forms Services and Excel Services to integrate with existing business processes
• Business Data Catalog
• Integration with SQL Server Reporting Services

With the release of MOSS 2007, there was major growth in the use of SharePoint, particularly across the enterprise. In 2008, Microsoft reported more than 100 million SharePoint users across over 17,000 client organisations, and total SharePoint sales of more than $1 billion. The product and associated tools had matured enough to be a realistic platform for internet and intranet usage in large organisations. As a developer, it was better than the 2003 release to work with (a very low bar), but it was still an extremely frustrating product. The tooling in Visual Studio 2007, whilst improved since Visual Studio 2003, was still clunky and slow. Developers instead looked at third party tools to help with packaging and deploying solutions, such as:

• WSPBuilder
• SharePoint Content Deployment Wizard
• SmartPart for SharePoint (allowing you to host ASP.NET user controls)

There are probably many other tools that I can’t recall – feel free to call out any in the comments below.

SharePoint Online (BPOS)

In October 2007, the Business Productivity Online Suite (BPOS) was released. This was Microsoft's first attempt at establishing a SaaS offering. BPOS included SharePoint Online, Exchange Online, Office Communications Online and Live Meeting. The SharePoint Online offering was based on the SharePoint 2007 release. Depending on the subscription chosen, this would be either a Dedicated (single tenant) or a Standard (multi-tenant) instance.

While the BPOS offering was successful, Microsoft encountered a number of issues running high performance multi-tenanted instances of SharePoint Online using MOSS 2007, and this experience fed directly into the development of SharePoint 2010.

Next week, I’ll look at the release of SharePoint 2010 and how the developer story changed significantly.

SharePoint Development - Origins

This post is the first in a series looking at how SharePoint development has changed over the years, and how it is likely to change in the future. In this post, we will look back at SharePoint's origins, up to and including the release of SharePoint 2003.

Series Outline:

• Part 0 - SharePoint Development - Past, Present and Future
• Part 1 - SharePoint Origins (including 2001 and 2003 releases) (this post)
• Part 2 - MOSS 2007
• Part 3 - SharePoint 2010
• Part 4 - SharePoint 2013
• Part 5 - SharePoint 2016
• Part 6 - The Future of SharePoint Development

 

SharePoint's Origins

SharePoint evolved from two separate Microsoft projects, named "Office Server" and "Tahoe", that were developed during the development of Office XP. It was also influenced by the Platinum project that introduced the concept of digital dashboards, and the TeamPages project that allowed users to create and edit simple web-based lists (and the origin of the CAML markup language). These projects were all aimed at accessing and sharing information across organisations. The initial development on these products started back in March 1998, prior to the release of the .NET framework (the initial beta versions of the .NET framework were released in late 2000).

SharePoint 2001

The initial SharePoint release was actually two separate products, the Microsoft SharePoint Portal Server (SPS, released 2001) and the Microsoft SharePoint Team Services (STS, released in 2002). The SPS product was based on "Tahoe", offering top-down portals, search and document management. Tahoe was a collection of different technologies from various companies acquired by Microsoft that was built on top of the Exchange data store. STS was based on "Office Server", and was a bottom up team collaboration product.

Thankfully, I have never had to work with this early version of SharePoint, which was very basic.

SharePoint 2003

In 2003, SharePoint as we would recognise it today was released. It combined STS and SPS together to offer collaboration, search, content management and portal capabilities in a single product. It had an improved user interface, better personalization, and a collaboration store.

A lot of the criticism aimed at the initial SharePoint 2001 release was around the under-powered web store that limited the functionality of the product and a digital dashboard that was outside Microsoft’s core development platform, limiting support options for users. As a result, significant work was put into making SharePoint 2003 more reliable and scalable, and to improve support options by making use of the same developer tools as other Microsoft products.

The product came in 2 different versions:

• Windows SharePoint Services v2 (WSS 2.0), a basic free version that was included with Windows Server 2003.
• SharePoint Portal services 2003 (SPS 2003), a premium version was built on top of WSS that included additional functionality for document management and search.

The main features included:

• Alerts
• Site templates
• Calendars
• Surveys
• Document Libraries
• Lists
• MySite
• Site Directory
• User Profile
• Improved search and indexing (SPS)
• Improved user interface and personalization options
• Taxonomy
• Document collaboration and versioning features
• Single Sign-on
• Audiences
• Web Parts (.cab)

 

Developing for SharePoint 2003

The major developer features in SharePoint 2003 were the introduction of server-side API, and the replacement of the Exchange data store with SQL Server. The server-side API made use of the new integration with the .NET framework, and allowed the development of custom code solutions, including web parts. In addition to the default web parts (Content Editor, Image, Form and Contacts), developers could develop custom web parts using Visual Studio 2003 (using the .cab file format).

Note that in SharePoint 2003, and all subsequent versions of SharePoint, COM components (unmanaged code) are used for core features, with a (smaller) managed layer wrapped around it for the .NET API. This is the cause of the dispose issues that affects fundamental SharePoint objects such as SPSite and SPWeb (here and also here).

I did have to work with SharePoint 2003, and it was a pretty horrible experience. Even in it's second release, it was a very immature product, and the developer tooling (based around Visual Studio) was lacking.

In the next post, I’ll look at at the release of MOSS 2007.

SharePoint Development - Past, Present and Future

With the recent release of SharePoint 2016, and with details of the latest development framework (the SharePoint Framework) for SharePoint becoming available, I thought it would be useful to produce a series of blog posts looking at SharePoint development over the years, and how it is likely to change in the future, based on trends from past releases and the current focus on cloud based solutions from Microsoft. The series will consist of 6 articles, published weekly:

• Part 0 - SharePoint Development - Past, Present and Future (this post)
• Part 1 - SharePoint Origins (including 2001 and 2003 releases)
• Part 2 - MOSS 2007
• Part 3 - SharePoint 2010
• Part 4 - SharePoint 2013
• Part 5 - SharePoint 2016
• Part 6 - The Future SharePoint Development

This series will be focused primarily on on-premise SharePoint development. While Office 365/SharePoint Online are growing rapidly and are influencing the development of the SharePoint platform greatly, the majority of large organisations are still using on-premise, and are (rightly) wary of having all their data in a third party data centre, even if that third party is Microsoft. I'll point out trends in Office 365 that I think will affect future SharePoint development, but it isn't the main focus for these articles.

I'll be drawing heavily on my own experiences of developing for SharePoint. I first started developed solutions for SharePoint back in 2006, when as a recent graduate I was part of a large team building integration products with SharePoint 2003 (a hideous project). I then spent over 3 years working full time on SharePoint solutions (MOSS 2007 and SharePoint 2010), at which point I took a break to spend several years lost in rewriting large .NET legacy applications. Last year, I started working with SharePoint once more. In this new role, I am currently part of a small team working on a phased SharePoint migration (2010 to 2013) that will be completed in the new year (January 2017). Over all this time, I've worked on a wide variety of SharePoint solutions (intranets, various organisational platforms, eCommerce sites, HR applications and currently a Virtual Learning Environment).

So, lets start by looking at the origins of the SharePoint platform.

My Son

My son, Matthew Oiyan Parkhill, was born on Monday 26th September at 04:18, at the Royal Jubilee Maternity Hospital, Belfast. He weighed in at 5 lbs 9 oz. Mum and baby are doing well.

Mei and I would like to pass on our thanks to everyone at the Royal Jubilee Maternity Hospital for all their help during Mei’s & Matthew’s stay, especially the midwives Patricia, Deirdre and Heidi. We won’t forget their many kindnesses to our little family.

Matthew comes from the Hebrew name Matityahu, meaning ‘Gift of God’, and Oiyan comes from the Cantonese for ‘Love and Grace’.

Forgotten Your Restriction PIN for IPhone?

Recently, I forgot the restrictions PIN on my iPhone (fat thumbs when initially setting it). I eventually recovered it by using the very useful pinfinder utility. This console program, written in Go, was kindly developed and open sourced by Gareth Watts. His utility recovers the restrictions (parental) passcode on iPhones, iPads and iPod touches from an (unencrypted) iTunes backup.

I’m blogging about this to help anyone with the same problem, and also to recommend pinfinder, so people use it rather than one of the rather dubious other commercial products that are available.

Download the latest release of pinfinder from here:

https://github.com/gwatts/pinfinder/releases

What I’ve Been Reading…

I’ve always loved reading. I can remember (aged 6-7?) looking forward to every Thursday afternoon, when my sister and I would be taken to a local newsagent for a comic. My first reading material consisted of The Beezer, The Dandy and The Beano.

At the age of 9, my mum left me for an hour in the local library. By the time she returned, I was hooked on books. I can still remember the first library book I took out on my new library card - Gorilla Adventure by Willard Price. I was enthralled by this story of 2 teenagers who travelled the world capturing wild animals. I ended up reading most of the titles in the excellent Adventure book series (I seem to recall it being called ‘The Adventures of Hal & Roger’ – a UK printing possibly?).

With the recent migration of my data from Shelfari, I wondered – what have I actually been reading?  After importing the data into a local database, I decided to take a look.

Since November 2011 (when I signed up to Shelfari), I’ve read 332 books. 29 of these were second readings, so this brings the total down to 303 unique books. So, how many books do I read per year?

So I vary from 56 to 91 books per year (over the 4 full years of data, 2012 to 2015), averaging at around 74 books. If we look at the type of books (fiction/non-fiction), does this help explain the variability in the number of books read? I would expect the number of books to decrease if I’m reading more non-fiction in a given year. My impression is that it takes me longer to complete a non-fiction book, and I’m also less inclined to start a new book after I finish.

And looking at non-fiction books as a percentage of the total number of books read each year:

There doesn't appear to be a clear relationship between the amount of non-fiction I read and the total number of books. What other trends can we see in my reading? Looking at my fiction reading:

Author	Number of Highly Rated Books
Patrick O'Brian	18
J.K. Rowling	8
George R. R. Martin	6
Paul McAuley	6
Stephen Baxter	6
James Clavell	5
Robert Harris	5
Stephen King	5
William Gibson	5
Neil Gaiman	4

Over the past few years, I’ve enjoyed reading Patrick O’Brian’s amazing Aubrey–Maturin series (think Jane Austin on the high seas), the Harry Potter books, A Song of Ice and Fire (the book series on which the TV series Game of Thrones is based on), as well as several science fiction series from the talented UK writers Paul Mcauley and Stephen Baxter. The Asian Saga from James Clavell is a series I return to time and again.

As you can probably tell from the list above, I like reading complex novels, preferably in a series, with intricate plots and multi-facetted characters. One author who is not featured above is Frank Herbert – I still consider his Dune series the best writing I have ever read. I plan to read it again soon. I especially enjoy science fiction (Macauley, Baxter, Gibson - Neal Stephenson just fell outside the top 10) and Fantasy (Martin, Gaiman and Rowling).

In my non-fiction, I’m tend to read more widely – the topic interests me more than the author. There are a few exceptions:

Author	Number of Highly Rated Books
Fergal Keane	2
Jared Diamond	2
John Briffa	2
Max Hastings	2
Michael Lewis	2

I’ve particularly enjoy reading Fergal Keane’s books, in particular his epic story Road of Bones, about the battle of Kohima in 1944. I’ve also recently enjoyed Jared Diamond's books for their unique mix of anthropology, ecology, geography and evolutionary biology

What about books I didn’t like? Here is a list of the worst from the past 4 years:

Title	Author	My Review
Antifragile	Nassim Nicholas Taleb	‘Vainglorious and pompous nonsense. Read about 15 pages and gave up, not on the ideas being conveyed, but on the arrogance of the author.’
American Sniper	Chris Kyle, Scott McEwen, Jim DeFelice	Very poor, jingoistic fare.  Tells a completely different, and much grubbier, story than the film of the same name.  I found this book very depressing reading indeed.
The Bone Clocks	David Mitchell	Very disappointing. After enjoying the wonderful Cloud Atlas so much, very let down by this novel. It read like a terrible 1950's pulp Sci-Fi novel. Yet there was the odd glimpse of some great writing.  Such a pity.
Slow-Tech	Andrew Price	Rubbish.  The author has a very poor understanding of sound engineering principles.  The book reads more as a left wing diatribe, as opposed to a serious discussion of how to improve design choices.
A Discovery of Witches	Deborah E. Harkness	A truly dreadful book.  I threw the book away after approx. 30 pages. Dire stuff.

I regret putting David Mitchell in the above list – he is a fantastic author (Cloud Atlas, Black Swan Green), but Bone Clocks was a great let-down.

At some point, I hope to spend some time analysing the hashtags I associated with each book in my reading list. I’ll update this post with the results. Until then, happy reading!

Shelfari

Recently, the Shelfari book cataloguing service was shut down by Amazon. Officially, the site was merged with GoodReads, another Amazon purchase, but effectively it was a shutdown. I've used Shelfari to track my reading habits and to maintain a list of 'must read books' for over 4 years, and I was a bit put out by the announcement, but not particularly surprised. The Shelfari site hasn't been upgraded in years, and was slow, unresponsive (both in terms of performance and mobile usage) and prone to outages.

I looked at alternative sites that I could migrate my data to (GoodReads, LibraryThing, Open Library), and decided that none of them was what I was after. Given that I only had 2 lists (books I've read, and books I want to read), a simple spreadsheet would do. So I migrated my data out from Shelfari using the data export functionality, and looked at the contents of the export file.

A cursory examination of the file (unusually, a .tsv file rather than the more popular .csv format) showed:

1. The reading list was missing key data. Primarily, if a book was recorded twice (when I had re-read it), that book's details was completely missing from the list.
2. The file format was invalid - it inconsistently quoted string fields (i.e. empty/null values didn't have quotes, while other values in the same column were quoted). This meant it was extremely difficult to import the data into a database/Excel to manipulate. 

In order to import my Shelfari data into an Excel file, I had to write a small command line application to carry out the conversion.  While it was an interesting exercise (I came across the excellent EPPlus and CsvHelper libraries for manipulating Excel and .csv files respectively), it took up time I could have used on other projects. Also, how is a non-programmer meant to deal with invalid data?  It is pretty poor for Amazon to ignore a site like Shelfari for years and then on shutting it down, to hamper users from accessing their own data due to an incompently coded and untested data export functionality. It also stands in stark contrast to the excellent work from both Google and Twitter in allowing users to export their data in a usable format.

Thankfully, I had taken screenshots of my book history and was able to recreate the missing book data. After managing to get all of my data into an Excel spreadsheet, I've now generated an updated list of the books I've read since joining Shelfari back in November 2011. You can view here:

• http://www.andyparkhill.co.uk/p/reading-list.html
  

Social Media

This weekend, I’ve deleted my Twitter and LinkedIn accounts. I deleted my Facebook account years ago. It looks like I’m done on social media for good.

LinkedIn is a cesspit of recruiter slime and unsolicited offers to connect about jobs I wouldn’t offer to my cat, so it was no hardship to delete that account. But Twitter?

I used to love Twitter. I’m made some great friends there, been part of some fascinating conversations, and followed some really interesting people. But the fact is, Twitter has become a nasty place to be. I’ve seen too many people dog-piling onto others for not sharing their views, and people being abused for little or no reason. Too often lately, I’ve also found myself getting snarky with others when trying to argue a point in those rare discussions that still occasionally occur. I don’t want to spend my time in a place where people are mean to each other. I don’t want to be one of those being mean.

Apologies to all my friends on Twitter – I’ll miss catching up with you there! Please drop me a line. We can grab a coffee and stay in touch the old-fashioned, analogue way. I’ll still be posting to my blog, hopefully a bit more often than in the past year.

Goals for 2016

As it now mid-February, it is now the time of year when I finally update you with my goals for this year. 

Recap of 2015

But before I set out my goals for the year ahead, how did I do with my goals for last year?  Well, I set out 4 goals:

1. Start running again
2. Get physically fit again
3. To develop a non-trivial side project using JavaScript.
4. Marry my girlfriend Mei.

So, tackling each in turn:

1. Running – a partial success. I did build up my running, just not as much as I had hoped.  Looking at the sub-goals:
   1. As you can see from the graph of my monthly mileage below, trying to run 80 miles a month was probably a bit too ambitious.  I averaged just over 30 miles a month over 2015, and was hitting over 45 miles per month once I got going.
   2. I ran in one Parkrun, the Ecos run in Ballymena on Saturday 10th January 2015. Frankly, this was just too big a goal, and would have meant giving up every other Saturday morning. I needed to make this goal a lot more manageable.
2. Get fit – a definite fail.
   1. My end of year score in the Five Minute Fitnes Challenge was 182, actually down on my personal best of  194.
   2. My bodyweight at the end of 2015 was 78.4 kg at around 23% body fat – very poor. I started 2015 weighing 80.7 kg.
3. Develop a side project in JavaScript – Another fail! I did have a couple of side projects on the go, specifically one to export OneNote documents into markdown format, but I never got round to completing them or releasing them, due to a lack of time and interest. I’m not too concerned about this, as I have been heavily committed to my new role since April, and have been learning a lot. But I do think this something I need to look at again this year.
4. Marry my girlfriend Mei – Yes!! Mei and I married in the summer at St. Patrick’s church in Coleraine. The massive dip in my monthly running mileage was caused by the August wedding (and associated stress) and our honeymoon cruise to Norway.

So, based on a very mixed bag of results for last year’s goals, what about this year?  The main lessons I learnt from last year was to focus on a single big goal, and to be realistic about what you can achieve. Funnily enough, it is not the first time I’ve learnt the same lessons. So for this year:

1. Continue to focus on my running
   1. Run 12 Parkruns by end of 2016 – a slightly more realistic target!
   2. To be running 60 miles per month by the end of the year
2. Focus on my blog
   1. To blog a new article monthly, and to start focusing on generating more technical content for my blog.
   2. Migrate my blog away to a new blogging platform. Blogger was a good platform to start on back in March 2010, but it is definitely showing its age.  It is time for a blog makeover!!
   3. Make more time for blogging by giving up Twitter for the year. I enjoy Twitter and I have some good conversations there – but I also waste a lot of time, and occasionally I get dragged down into exchanging bile and snark. I would rather spend the time on something more meaningful and long term, like my blog and other side projects. I’ll occasionally post links to my own content on Twitter, but otherwise I’ll be avoiding it.

As usual, even if I don’t achieve my goals, I’ll hopefully learn or achieve something worthwhile.