In an earlier post I listed Eight Things SharePoint 2010 Needs to be a True ECM System, and, at first glance the new version looks very encouraging from an ECM perspective. As I’ve said before, I get excited by anything that can help my clients better manage their information and SharePoint has the potential to be a transformative platform bridging structured content, unstructured content and social computing in one flexible package. SharePoint 2007 does a decent job of this but it has some deficiencies when it comes to managing all content in the enterprise.

I’ll also give you the same caveat I gave last time; While this post focuses on SharePoint as a technology, technology is about the very last thing that should be considered when an organization sets out to manage its content more efficiently. Information management should start with a good business case, appropriate sponsorship, choosing the right areas of focus then building capacity within the organization to truly succeed. Technology is only the last piece of this puzzle. All of that said, there has been an incredible amount of interest in SharePoint (as illustrated by the 7,500 people who attended the SharePoint conference from October 19 to 22, 2009) and many of my clients have questions about where (or whether) this tool should fit into their ECM strategy.

Finally, the updates below are based on my attendance at the SharePoint conference where I went to as many breakout sessions as possible and chased down beleaguered Microsoft staffers to ask questions in what must have felt like a trip to the old Roman Coliseum (with the lions, not with Caesar). I tried to focus on attending ECM-specific sessions and have done as much reading as I can but as a vendor-neutral consultant Microsoft hasn’t seen fit to furnish C3 Associates with a pre-beta version of SharePoint 2010, so I haven’t actually used the system myself. As always we will continue to learn as much as we can about all of the ECM tools and technologies that are of interest to our clients but in the absence of actually working with SharePoint some of our understanding will be incomplete or possibly incorrect. I will provide updates in future posts as I learn more.

I have used a five point scale to evaluate the how well I think SharePoint 2010 meets my “Eight Things” criteria for inclusion into the ECM club. Remember that these are based on only my first look at the tool and are subject to revision as I learn more about how the new features and functions actually work.

Initial Ranking Scale

With all of that out of the way here are the eight reasons I think Microsoft has moved towards a more complete ECM solution.

1. Persistent links – The single biggest shortcoming of SharePoint 2007 is the inability to link directly to a unique object ID. One of the greatest benefits of ECM systems is the ability to send content via a link rather than relying on email attachments. In traditional ECM applications this isn’t a problem; each content object has its own unique ID that doesn’t change regardless of where it lives in the repository. In SharePoint 2007, links break if you rename or move a file. The other benefit of persistent linking is that it enables the management of compound documents (a container that stores multiple documents like the chapters of a book) and the ability to link directly to an older version of a document.

2. Store once, use many – SharePoint 2007 had a nasty habit of copying content throughout the system rather than using pointers to a single source of the truth (because content links might break as noted above). Perhaps the best example of the misguided use of “copy” capabilities in SharePoint is the “Send to…Records Center” feature where a copy of a document is sent to the Records Center while leaving the original in place rather than either moving the document and leaving a pointer or changing the state of the document to indicate its changed status (see point 3 for more on the RM capabilities of SharePoint).

3. Honest-to-goodness Records Management – I recognize that that SharePoint 2007 is DoD 5015.2 certified but the statement from the product development team that the DoD 5015.2 components are “not intended for customers…who would like to enhance the records management functionality of MOSS 2007 with particular 5015.2 oriented features but are not required to run their system in a certified configuration” doesn’t exactly inspire confidence. Add to this the fact that SharePoint doesn’t allow users to efficiently manage physical objects out of the box and it is clear that Microsoft needs to decide if they are really serious about the records management space or if they will leave it to partners.

4. Better metadata management – Metadata in SharePoint 2007 took a quantum leap forward with the introduction of Content Types. However, managing metadata in SharePoint 2007 can be difficult especially when dealing with multiple site collections.

5. Reusable search templates and stored search results – There is no question that search is a focus for Microsoft based on their acquisition of FAST and their push into public internet search with the recent launch of Bing. Search in SharePoint 2007 is reasonably good but the tool does not have the ability to either store a “snapshot” of search results for future reference nor does SharePoint 2007 allow users to create reusable search templates. This feature would save users time by allowing them to create a search query then either re-execute that query in the future or add new criteria without having to rebuild the entire search.

6. More granular security – This is one area where SharePoint was already reasonably strong but truly deep ECM systems include advanced security features like the ability to deny permission to certain objects on an as-needed basis. The current process for managing security is a bit cumbersome but I expect this is something Microsoft is working on. It will be interesting to see if what changes, if any, make it into the final release of the product.

7. Surface the audit trail – One of the things I like the best about established ECM applications is the ability to see who has opened my documents. I find this particularly handy on status report day when I inevitably discover that I’ve made a mistake in the document I’ve just sent out (as an unbreakable persistent link of course). I can check the audit trail to see if anyone has opened the document and if not, make my changes without anyone knowing I’d messed up in the first place. While SharePoint tracks most major audit events, the list of events is not as extensive as in a traditional ECM application nor is this information surfaced through the function menu of the content object.

8. More and more mature line-of-business integrations – This should be a strength of SharePoint given the sizeable .NET developer community as well as the extensive Microsoft partner ecosystem, but SharePoint still has a lot of catching up to do in this area. Organizations deploying SharePoint won’t be able to hold a single vendor to account for a series of modules (or Content Enabled Vertical Applications, as Gartner likes to call them). This may or may not be a bad thing depending on your perspective but established ECM vendors have offerings that satisfy a variety of industry verticals and business functions. To achieve the same thing with SharePoint customers will need to research, purchase and deploy modules from a variety of Microsoft partners. CMS Watch offers a good summary of the issues associated with third party add-ons for SharePoint.

To sum up, it is clear that SharePoint will continue to have a significant impact on the ECM landscape. The question is whether the functional improvements evident in SharePoint 2010 mean that organizations with significant commitments to other ECM platforms have to start all over again with SharePoint? In the short term, I think the answer is no. In many cases, the true benefit from the investments made in traditional ECM can be realized by surfacing some of this content though SharePoint interfaces; done well this can significantly enhance the user experience while still ensuring that the strong compliance engine in your existing ECM system keeps your content safe and your CEO out of jail.

I suspect that any changes in the ECM world will be evolutionary rather than revolutionary. SharePoint is a disruptive technology to be sure, but given it’s breadth, relative lack of maturity and a widely varied partner community Microsoft will help the overall ECM market grow and likely take established ECM vendors with it. As they (used to) say on Wall Street, a rising tide floats all boats.

This is not to say that things will be easy for the makes of FileNet, Documentum, Livelink and others; they have a significant challenge ahead in trying to position their products not relative to one another, but relative to SharePoint (whether they like it or not). The vendors that do this well will continue to thrive and any that choose to ignore SharePoint or do not recognize the significance of the changes in SharePoint 2010 could be in trouble.

As always I appreciate your feedback on anything you read here. Feel free to leave a comment here or drop me a note via my Twitter account at GregClarkC3.

I’m working on an update to my SharePoint 2010 post based on what I’ve learned since it was released in beta this fall. Look for that post early in new year. Until then, I wish all of my readers the best of the season and a fantastic 2010!

ECM is about managing information rather than just storing it. Done properly, it helps your organization take advantage of the vast amounts of information created every day by placing content in an appropriate context for your user community. This can mean pushing content out through a web portal or providing intuitive tools to allow users to create, browse or search for content when needed. The end result is that your organization will be both more efficient and more compliant with the rules that govern your industry because you will be able to retain information for as long as needed but no longer.

That strays a bit from the traditional definitions of ECM but I think that is kind of the point. It got me thinking about what I like about my job and why I got started on this crazy adventure in the first place. First off, I like ECM because I like people. The successful ECM implementations I have seen have not been implementations of a technology but have instead focused on efficiency, compliance, sharing and know-how; all of these things are influenced by the human beings in the mix and the technology is only a supporting player. Conversely, the ECM implementations I have seen that have not been as successful have been seen as technology rollouts.

The other thing I enjoy about ECM is that it allows me to work in a variety of different business areas. Although the focus of C3 Associates is on the energy, oil and gas industry there is an incredible diversity amongst the various business units in each of our clients. The opportunity to work with engineers in the morning, lawyers at lunch (because they’re busy the rest of the time) and accountants in the afternoon doesn’t come along very often. Each has their own unique perspective on the business of their organization and I see ECM as an opportunity to unite all of them to achieve more than they would have otherwise.

If helping people take their work to another level doesn’t get you out of bed in the morning, I don’t know what does.

You can download both slide decks in the Presentations section of our site.

Recently I helped a client work through some challenges they were having in aligning an established business process with a new piece of technology. To cut a (very) long story short, the projects group in this organization had started using a document generation tool that included the ability to route documents for review and approval. The tool was slick; because it is built on top of a database, a change that applied to multiple documents could be completed in a single step. Further, the updated documents could be routed for review and approval then automatically updated in the repository. Fast, simple and very cool. It helped the projects group cut down on the time to create new operating procedures before handing them over to the operations team. What more could we ask for?

Good question.

The operations group uses a document review and approval workflow embedded in their Enterprise Content Management system to update existing operating procedures. The workflow was built some years ago and wasn’t fully automated; there is a need for a “real-world review” before routing an updated document for review and approval (ie. send a link to the document around via email to get comments) and if the same change applied to multiple documents that change would have to be made multiple times. The process was not so fast, not so simple and not exactly cool. However, everyone in the operations group knew how to use the process and critical documents were kept up to date.

The challenge came when trying to marry up these two processes. Even though the projects group was using a “cool” technology that worked for the creation of net new documents, the operations team would have to drastically change their process (not to mention train several hundred users) to adopt this tool for ongoing updates and changes to their documents.

At the end of the day the operating group decided they would keep their old process and old tool. Why? The cost and effort to deploy the latest technology outweighed the benefit for doing so. Everyone at the table agreed the new technology was fantastic and that it was something that should be on the radar but it didn’t address a pressing business need. The old process wasn’t broken so there was no point in fixing it.

It is very easy to get caught up in the wonders of a particular technology but it’s critical to remember that even if it’s the ideal technical solution, it may not be the ideal business solution.

While this post is all about SharePoint as a technology, I want to be clear that technology is about the very last thing that should be considered when implementing ECM. It should start with a good business case, appropriate sponsorship, choosing the right areas of focus then building capacity within the organization to truly succeed with ECM. Technology is only the last piece of this puzzle. All of that said, there has been an incredible amount of interest in SharePoint and many of my clients have questions about where (or whether) this tool should fit into their ECM strategy.

For SharePoint 2010 to become the ECM category killer and truly threaten the market share of Open Text, EMC Documentum, IBM FileNet and others, the new version should have the following eight things:

1. Persistent links – I’ve told anyone who would listen over the past two years (and many people who wouldn’t…hi mom!) that the single biggest shortcoming of SharePoint 2007 is the inability to link directly to a unique object ID. One of the greatest benefits of ECM systems is the ability to send content via a link rather than relying on email attachments. In traditional ECM applications this isn’t a problem; each content object has its own unique ID that doesn’t change regardless of where it lives in the repository. In SharePoint links will break if you rename or move a file. The other benefit of persistent linking is that it enables the management of compound documents (a container that stores multiple documents like the chapters of a book) and the ability to link directly to an older version of a document. Rumor has it that SharePoint 2010 will include persistent linking and if it does the tool will have taken a big step forward.

2. Store once, use many – SharePoint has a nasty habit of copying content throughout the system rather than using pointers to a single source of the truth (because content links might break as noted above). Yes, yes, I know that you can “Send to…Other Location” and link that new doc back to the original but this linkage is easy to break and experience tells me that the content falls out of synch very quickly. Perhaps the best example of the misguided use of “copy” capabilities in SharePoint is the “Send to…Records Center” feature where a copy of a document is sent to the Records Center while leaving the original in place rather than either moving the document and leaving a pointer or changing the state of the document to indicate its changed status (see the next point for more on the RM capabilities of SharePoint). The propagation of copies of documents throughout a repository is very bad mojo from an ECM and records management perspective and it is something that Microsoft must fix if SharePoint is going to replace traditional ECM applications.

3. Honest-to-goodness Records Management – I recognize that that SharePoint 2007 is DoD 5015.2 certified but the statement from the product development team that the DoD 5015.2 components are “not intended for customers…who would like to enhance the records management functionality of MOSS 2007 with particular 5015.2 oriented features but are not required to run their system in a certified configuration” doesn’t exactly inspire confidence. Add to this the fact that SharePoint doesn’t allow users to manage physical objects out of the box and it is clear that Microsoft needs to decide if they are really serious about the records management space or if they will leave it to partners.

4. Better metadata management – Metadata in SharePoint 2007 took a quantum leap forward with the introduction of Content Types. However, managing metadata in SharePoint 2007 can be difficult especially when dealing with multiple site collections. An ECM system should be able to easily manage relationships between data in the form of cascading attributes and parent-child relationships throughout the entire repository and should also support inheritance of metadata from the container level (eg. folder) to the content within that container. I’m hopeful that we’ll see improvements to the SharePoint Business Data Catalogue (BDC) that make this possible. The other option appears to be Microsoft’s Master Data Management tool (codenamed “Bulldog”) which is rumoured to be included in SharePoint 2010.

5. Reusable search templates and stored search results – There is no question that search is a focus for Microsoft based on their acquisition of FAST and their push into public internet search with the recent launch of Bing. Search in SharePoint 2007 is reasonably good but the tool does not have the ability to either store a “snapshot” of search results for future reference nor does SharePoint 2007 allow users to create reusable search templates. This feature would save users time by allowing them to create a search query then either re-execute that query in the future or add new criteria without having to rebuild the entire search.

6. More granular security – This is one area where SharePoint was already reasonably strong but truly deep ECM systems include advanced security features like the ability to deny permission to certain objects on an as-needed basis. The current process for managing security is a bit cumbersome but I expect this is something Microsoft is working on. It will be interesting to see if what changes, if any, make it into the final release of the product.

7. Surface the audit trail – One of the things I like the best about established ECM applications is the ability to see who has opened my documents. I find this particularly handy on status report day when I inevitably discover that I’ve made a mistake in the document I’ve just sent out (as an unbreakable persistent link of course). I can check the audit trail to see if anyone has opened the document and if not, make my changes without anyone knowing I’d messed up in the first place. While SharePoint tracks most major audit events, the list of events is not as extensive as in a traditional ECM application nor is this information surfaced through the function menu of the content object.

8. More and more mature line-of-business integrations – This should be a strength of SharePoint given the sizeable .NET developer community as well as the extensive Microsoft partner ecosystem, but SharePoint still has a lot of catching up to do in this area. Organizations deploying SharePoint won’t be able to hold a single vendor to account for a series of modules (or Content Enabled Vertical Applications, as Gartner likes to call them). This may or may not be a bad thing depending on your perspective but established ECM vendors have offerings that satisfy a variety of industry verticals and business functions. To achieve the same thing with SharePoint customers will need to research, purchase and deploy modules from a variety of Microsoft partners. CMS Watch offers a good summary of the issues associated with third party add-ons for SharePoint.

It will be interesting to see whether SharePoint 2010 includes some or all of these features. Microsoft has done a good job of capturing a new market without significantly eroding the market share of traditional ECM vendors; as SharePoint adoption has increased the overall market has grown and, as they say on Wall Street, “a rising tide floats all boats.” Whether Microsoft’s ship will sail away from the rest remains to be seen (as does my ability to stretch a bad metaphor) but they were clearly not able to do so with SharePoint 2007.

As I learn more about SharePoint 2010 I will share my thoughts here and on Twitter (GregClarkC3). I also plan to attend the 2009 SharePoint Conference this coming October where I’m sure I’ll find out more. In the meantime I’d appreciate any feedback, additional information or opinions you have to share about SharePoint 2010 or ECM in general.

A special thanks to C3 Principal John Meilleur and C3 Associates Ankur Gupta and Lam Huynh for sharing their expertise in the creation of this article.

The discussion started from a comment Simon made on my retweet of Mary Abraham’s comments on Open Text Enterprise 2.0 expert Cheryl McKinnon’s webinar on Enterprise 2.0 in the Legal world (and no, my mother would have no idea what that all means but I hope you can follow along at home).

To paraphrase, Simon’s point is that restricting social networking tools like del.icio.us “isn’t very social”. In general terms I agree with him; openness should be the primary modus operandi of any ECM or Enterprise 2.0 implementation (or as I tweeted earlier today, ECM should be “open by design, closed by exception”).

I don’t think organizations should prevent people from using social networking tools inside the firewall and frankly can’t prevent them from doing so in their personal lives. In either case it is critical to educate them to be smart about it.

An example: What if the CEO’s del.icio.us links were all about mergers and acquisitions, potential target companies and research about management consultants who can integrate two big companies? That would be a bit of a tip off that something might be up.

In the end I think it’s easy to forget that social computing is in its infancy and that many organizations are struggling to find the right balance between openness and (appropriate) controls. A good start is to educate your workforce about some of the potential pitfalls of social computing and to put formal controls in place as necessary. My advice; don’t be afraid of the openness required for social computing to be successful in your organization. The good will outweigh the bad in almost every case.

My opinion: the core concepts of KM apply now as much as they ever did regardless of how they are applied in the enterprise. The term KM may be going out of style but the concepts are valid.

To that end I’ve been thinking about how knowledge embedded in business processes is perhaps the best way to realize the seemingly limitless promise of KM. To me, a successful “process knowledge” scenario might look something like this:

Organization X has a resonably well defined set of engineering standards but not everyone knows about them (and therefore not everyone follows them) nor is the process for providing feedback on the standards well understood. This has led to variations in engineering design processes which have impacted how quickly the organization is able to complete engineering projects. In this economy this is obviously a situation they would like to correct.

The organization agrees to undertake a project to surface these standards through their intranet linked back to their source location in their enterprise document management repository. The goal is to ensure that everyone in the company knows where to find the standards by either browsing or searching. Once in the intranet workspace, users find clear instructions about how to provide feedback on current standards, request updates or ask questions. The business owner of the standards (or her team) may either respond to questions on a threaded discussion and/or she might blog about ongoing updates and revisions to the standards.

Anyone new to the company can find the workspace on the intranet just by poking around but if they can’t (or choose not to) their colleagues will point them there because “that’s the way we do things around here”. The owner of the standards and her team will also hold regular face to face meetings with key stakeholders to discuss change requests and to communicate best practices. Meeting schedules, agendas and outcomes would be posted to the site for all to see, comment on and possibly even update (perhaps in a wiki). This community would be related to other communities that deal with other aspects of the engineering and operations function within the organization.

To this point, none of what I’ve talked about is particularly new or radical. It is the definition of a classic Community of Practice that many, if not most, KM initiatives will implement. The extension of the CoP is that the business process for creating, maintaining and distributing standards no longer relies on the knowledge with one person (or group) to succeed. When the owner of the engineering standard moves on the process not only survives, it thrives. When new people join the company they are brought up to speed quicker. When new ideas occur or when new requirements are identified the organization is able to respond quickly. And all of this capacity is embedded in the business process, not the person.

It is important to understand that implementing such a process will be an evolutionary change and that these projects will have successes and failures along the way. The tools choosen to enable the development and propagation of process knowledge are largely irrelevant but will guide what the organization is able to do and their areas of focus. If the application isn’t “ATM simple” people simply won’t use it.

Here’s a picture from the big night. Congrats again Pankaj, we’re lucky to have such talented people on the C3 team!

Sheila Bryant, Pankaj Bhawnani and Greg Clark celebrate Pankaj's PMI-SAC award