C3 Associates Inc.

The hype cycle has started for the upcoming release of SharePoint 2010 and I’m certainly not the only one to get caught up in it. I’m excited about anything that can help my clients better manage their information and I’ve always seen SharePoint as a potentially transformative platform bridging structured content, unstructured content and social computing in one flexible package. The current release of SharePoint does a decent job of this but in my opinion and the opinions of others (here and here) it has some shortcomings when it comes to its capabilities as a true Enterprise Content Management platform.

While this post is all about SharePoint as a technology, I want to be clear that technology is about the very last thing that should be considered when implementing ECM. It should start with a good business case, appropriate sponsorship, choosing the right areas of focus then building capacity within the organization to truly succeed with ECM. Technology is only the last piece of this puzzle. All of that said, there has been an incredible amount of interest in SharePoint and many of my clients have questions about where (or whether) this tool should fit into their ECM strategy.

For SharePoint 2010 to become the ECM category killer and truly threaten the market share of Open Text, EMC Documentum, IBM FileNet and others, the new version should have the following eight things:

1. Persistent links – I’ve told anyone who would listen over the past two years (and many people who wouldn’t…hi mom!) that the single biggest shortcoming of SharePoint 2007 is the inability to link directly to a unique object ID. One of the greatest benefits of ECM systems is the ability to send content via a link rather than relying on email attachments. In traditional ECM applications this isn’t a problem; each content object has its own unique ID that doesn’t change regardless of where it lives in the repository. In SharePoint links will break if you rename or move a file. The other benefit of persistent linking is that it enables the management of compound documents (a container that stores multiple documents like the chapters of a book) and the ability to link directly to an older version of a document. Rumor has it that SharePoint 2010 will include persistent linking and if it does the tool will have taken a big step forward.

2. Store once, use many – SharePoint has a nasty habit of copying content throughout the system rather than using pointers to a single source of the truth (because content links might break as noted above). Yes, yes, I know that you can “Send to…Other Location” and link that new doc back to the original but this linkage is easy to break and experience tells me that the content falls out of synch very quickly. Perhaps the best example of the misguided use of “copy” capabilities in SharePoint is the “Send to…Records Center” feature where a copy of a document is sent to the Records Center while leaving the original in place rather than either moving the document and leaving a pointer or changing the state of the document to indicate its changed status (see the next point for more on the RM capabilities of SharePoint). The propagation of copies of documents throughout a repository is very bad mojo from an ECM and records management perspective and it is something that Microsoft must fix if SharePoint is going to replace traditional ECM applications.

3. Honest-to-goodness Records Management – I recognize that that SharePoint 2007 is DoD 5015.2 certified but the statement from the product development team that the DoD 5015.2 components are “not intended for customers…who would like to enhance the records management functionality of MOSS 2007 with particular 5015.2 oriented features but are not required to run their system in a certified configuration” doesn’t exactly inspire confidence. Add to this the fact that SharePoint doesn’t allow users to manage physical objects out of the box and it is clear that Microsoft needs to decide if they are really serious about the records management space or if they will leave it to partners.

4. Better metadata management – Metadata in SharePoint 2007 took a quantum leap forward with the introduction of Content Types. However, managing metadata in SharePoint 2007 can be difficult especially when dealing with multiple site collections. An ECM system should be able to easily manage relationships between data in the form of cascading attributes and parent-child relationships throughout the entire repository and should also support inheritance of metadata from the container level (eg. folder) to the content within that container. I’m hopeful that we’ll see improvements to the SharePoint Business Data Catalogue (BDC) that make this possible. The other option appears to be Microsoft’s Master Data Management tool (codenamed “Bulldog”) which is rumoured to be included in SharePoint 2010.

5. Reusable search templates and stored search results – There is no question that search is a focus for Microsoft based on their acquisition of FAST and their push into public internet search with the recent launch of Bing. Search in SharePoint 2007 is reasonably good but the tool does not have the ability to either store a “snapshot” of search results for future reference nor does SharePoint 2007 allow users to create reusable search templates. This feature would save users time by allowing them to create a search query then either re-execute that query in the future or add new criteria without having to rebuild the entire search.

6. More granular security – This is one area where SharePoint was already reasonably strong but truly deep ECM systems include advanced security features like the ability to deny permission to certain objects on an as-needed basis. The current process for managing security is a bit cumbersome but I expect this is something Microsoft is working on. It will be interesting to see if what changes, if any, make it into the final release of the product.

7. Surface the audit trail – One of the things I like the best about established ECM applications is the ability to see who has opened my documents. I find this particularly handy on status report day when I inevitably discover that I’ve made a mistake in the document I’ve just sent out (as an unbreakable persistent link of course). I can check the audit trail to see if anyone has opened the document and if not, make my changes without anyone knowing I’d messed up in the first place. While SharePoint tracks most major audit events, the list of events is not as extensive as in a traditional ECM application nor is this information surfaced through the function menu of the content object.

8. More and more mature line-of-business integrations – This should be a strength of SharePoint given the sizeable .NET developer community as well as the extensive Microsoft partner ecosystem, but SharePoint still has a lot of catching up to do in this area. Organizations deploying SharePoint won’t be able to hold a single vendor to account for a series of modules (or Content Enabled Vertical Applications, as Gartner likes to call them). This may or may not be a bad thing depending on your perspective but established ECM vendors have offerings that satisfy a variety of industry verticals and business functions. To achieve the same thing with SharePoint customers will need to research, purchase and deploy modules from a variety of Microsoft partners. CMS Watch offers a good summary of the issues associated with third party add-ons for SharePoint.

It will be interesting to see whether SharePoint 2010 includes some or all of these features. Microsoft has done a good job of capturing a new market without significantly eroding the market share of traditional ECM vendors; as SharePoint adoption has increased the overall market has grown and, as they say on Wall Street, “a rising tide floats all boats.” Whether Microsoft’s ship will sail away from the rest remains to be seen (as does my ability to stretch a bad metaphor) but they were clearly not able to do so with SharePoint 2007.

As I learn more about SharePoint 2010 I will share my thoughts here and on Twitter (GregClarkC3). I also plan to attend the 2009 SharePoint Conference this coming October where I’m sure I’ll find out more. In the meantime I’d appreciate any feedback, additional information or opinions you have to share about SharePoint 2010 or ECM in general.

A special thanks to C3 Principal John Meilleur and C3 Associates Ankur Gupta and Lam Huynh for sharing their expertise in the creation of this article.

Just back from my presentation and it seemed to be well received by those in attendance. I certainly had a good time. Thanks to everyone who showed up at 4:45 on a wet Wednesday afternoon to hear me talk about my experience “rescuing” failed ECM implementations. I was particularly impressed by the excellent questions throughout; most of the attendees seemed to have had varying levels of success implementing ECM (from total failure to a reasonable amount of success: I’m very pleased to report that the organization that seemed to be most successful was the Nuclear Regulatory Commission which will certainly help me sleep at night).

You can find a copy of the presentation here.

Day 2 Summary

The day started with a series of keynote speeches. The most interesting was from Andrew Lippman, founding director of the MIT Media Laboratory. He talked about the “cracks” forming in institutions (he used banks as an example, but says the example can apply to many / most organizations) as a result of four things:

1) Inability to scale – Things that work well when an organization is small don’t usually work well when they get too big. Ethics comes to mind here.

2) Monocultures – Tribes of political beasts roam through most large organizations limiting its ability to thrive.

3) Opacity – Because of the first two points, it becomes difficult to understand exactly what’s going and innovation tends to be stifled.

4) Blurred mission – Organizations forget why they existed in the first place and start to exist only to sustain themselves.

His solution to these issues is “living content” generated by networks of people working from a common platform to build what they need. He suggested that building applications that constrain users to work a certain way is outdated and will be eclipsed by (social) networks and common platforms.

Extending this to an ECM example, I see this broadly as the approach Microsoft has taken with MOSS; although it is a proprietary server-based application its extensibility has attracted a large developer community who are busily building custom Web Parts that integrate nicely into the SharePoint platform. Although traditional ECM vendors have a similar model, theirs is closed and limited only to official partners whereas anyone with some .NET skills can spin up a web part and get to work.

The open source folks may disagree with me that SharePoint is a true platform but it is certainly more so than the traditional ECM applications on offer. This is not the death knell for traditional ECM but more of a shot across the bow that they too will likely need to open up to survive in the brave new networked world.

Enough thinking and more drinking! I’m off to the pub to talk ECM (or not) with my conference colleagues. As always, comments are welcome.

There’s been an interesting convergence of opinion pieces and hard news over the past 24 hours that speak to the future of Enterprise Content Management as we know it. I know that sounds a bit melodramatic but I sense that we’re in the midst of a seismic shift in the way organizations view ECM applications and the way vendors are providing those applications.

Yesterday, Matt Asay of CNet compared the recenty hype about Google’s Chrome browser to the quiet (or maybe not so quiet if you’ve been paying attention to the ECM blogosphere) reality of SharePoint’s massive expansion within the enterprise. His suggestion that Google has a long way to go to grab the attention of corporate IT groups and displace Microsoft is an understatement to say the least. He goes on to say that the trick for Google and others is to focus on interoperability if they have any hope of challenging SharePoint.

Matt is either incredibly astute or just plain lucky because it looks like Microsoft and friends have beaten Google to that punch as well. Today, Microsoft, EMC and IBM, supported by other key ECM vendors including Open Text, Alfresco, SAP and Oracle announced the Content Management Interoperability Services (CMIS) standard (see also the Yahoo Finance article on this).

At this point in the history of ECM, I think Google Chrome and Google Sites are just noise from an enterprise point of view. Until we see significant adoption in the real world and some acknowledgement from Google that content has a lifecycle, I’m not going to consider Google tools as part of the ECM equation.

And that equation, in my mind, includes both collaboration and control. All true ECM tools enable both but each tool does some things better than others. Traditional ECM applications like Livelink, Documentum and FileNet are very strong when it comes to controlling content but have not been noted for their ease-of-use when it comes to collaboration. SharePoint, on the other hand, is not the first choice amongst the records management community but provides arguably the best enterprise-ready collaboration platform on the market.

Which brings me back to the CMIS Standard.

My first impression about the impact CMIS will have in the ECM market is that it validates what I’ve been saying for a while. SharePoint will not be the video that kills ECM’s radio stars (hands up all you Buggles fans; be honest). CMIS allows organizations to legitimately pursue an “all of the above” ECM strategy and removes any impediments they may have in deploying SharePoint to supplement an existing ECM application. At the same time, traditional ECM vendors will continue to survive and thrive as they support content that requires more rigorous controls.

Ultimately, as the news release announcing the CMIS standard says, this should be all about the customer. Of course we all know that vendors don’t do things out of the goodness of their hearts, there’s clearly something in it for them as well. Here’s hoping that the CMIS standard will help break down barriers to successful content management by providing the best of both collaboration and control to the benefit of everyone.

Open Text Chairman and Chief Strategy Officer Tom Jenkins was featured in an extensive article in the July 21 edition of the Globe and Mail. You can find the article along with an audio interview here.

In addition to several hockey references that underscore the company’s Canadian roots, Jenkins made a couple of interesting comments about the “acquire or be acquired” nature of the software business. Let’s just say he didn’t do anything to dispel the speculation that Open Text will someday be acquired. At the same time, I think he didn’t take enough credit for the good work Open Text has done to become (and remain) Canada’s largest software company. He put a lot of it down to luck and also the ability to operate in relative obscurity due to the overwhelming success of Research in Motion (the makers of the ubiquitous BlackBerry, in case you didn’t know).

In my mind, Open Text has done a good job of positioning its technology to address a growing need while at the same time marketing well. The supposed “threat” to Open Text posed by SharePoint has yet to materialize and as the market comes to better understand how each of the products are positioned it seems even less likely that SharePoint will erode Open Text’s market share. Open Text’s “keep your friends close and your enemies closer” strategy seems to be working. We’re certainly seeing a lot of interest in hybrid solutions (using both Livelink and SharePoint) here in Calgary and I imagine that is likely the case elsewhere as well.

*so long as I build out the required functionality myself.

Let me explain.

I attended a customer demonstration session today given by Elliot Gerard of General Mills, the good people who bring you Count Chocula cereal and the Pillsbury Dough Boy (getting hungry just thinking about it). They are long time SharePoint users dating back to the original “Tahoe” version in 2001 and have developed a significant amount of in-house expertise with the tool since that time. In addition, they have a good relationship with Microsoft and regularly share knowledge about SharePoint between their technical team and Microsoft’s experts.

The work that General Mills has done in SharePoint is nothing less than impressive. They have created pretty much everything I think is needed to fill some of the gaps in the application. For example, they’ve built in a unique SharePoint documnent ID structure that allows for persistent linking to documents in SharePoint libraries, created a really slick way of adding metadata when uploading multiple documents at one time and have built a custom records dispostion process that allows users to either delete or renew any documents that have passed their retention periods (the way they’ve handled the application of retention information in the first place is cool too, but too detailed to go into here). I suspect this is exactly the kind of work Microsoft was hoping would happen given the flexible development capacity of the tool. If the reaction from the reps ranging from large technology companies and airlines is any indication, Microsoft is going to face a fair bit of pressure to include many of the features General Mills has created in the next version of SharePoint.

The likelihood that some or all of this functionality will be baked into SharePoint’s vNext seems even higher given the intersting discussion I had with a couple of members of the SharePoint development team at the ‘Ask the Experts’ lunch. I felt a bit sorry for everyone wearing the bright white “Ask Me a Question, I’m an Expert!” t-shirts but I guess that falls into the category of “other duties as assigned” in the standard Microsoft employee agreement. But I digress. The guys I spoke with confirmed that some of the finer points of ECM are a bit new to the more senior people at Microsoft (see my last couple of blog posts for more examples of this) but they said that the level of understanding was growing. Their plan seemed to be to allow partners and customers to build out missing functionality to see what really is required by the end user community and that the best examples of these customizations will be included in future versions of the tool. No one was willing to commit to anything formally of course but I was left with a good indication that at least a few of the functions General Mills has built would be included in the next release (if not exactly the same, the basic concept will be there).

Now, before you go and accuse me of being anti-SharePoint (or pro-SharePoint, or pro- / anti-anything else) let me say that the approach Microsoft has taken is a sound one; allowing the user and developer communities to really drive the requirements for a tool in a real-world setting is a great idea. Some may look at this and say that the tool is half-baked, others would say it’s extremely flexible. Neither perspective is wrong. As I think I say at least every other post, it all depends on what problem you’re trying to solve. If it’s collaboration you’re after and you’ve got a couple of .NET rock stars in your stable, fill your boots with SharePoint. If you need to meet stringent regulatory requirements or get sued a lot, consider Documentum or Livelink. If you’ve got both scenarios at your organization, consider implementing both. Tools to connect SharePoint and other ECM apps are also maturing rapidly and hold the potential to create a ‘best of both worlds’ scenario, so hopefully everyone wins in the end.

Looking forward to seeing my family again tomorrow after what has been a long four days. It was definitely worthwhile to attend this event and Microsoft did their usual good job of keeping us well-fed and entertained. It will be interesting to see how things evolve with the product come this time next year.