Thursday, October 29, 2009

Updated: Building a SharePoint 2010 Service Application #sp2010


A few weeks back, I wrote a series of three articles on the topic of building SharePoint 2010 Service Applications. Today, I have updated the source code for these articles to include Powershell cmdlets for provisioning, retrieving and updating the demo service application from the command line.

The three articles and a link to download the source code solution can be found on the SharePoint Solutions' community site:
Part One – The Service Application
Part Two – The Service Application’s Admin Interface
Part Three – The WCF Service Endpoint
Part Four – The User Application Page

Tuesday, October 27, 2009

Take 2: Lists as Native SQL Server Tables in SharePoint 2010? #spc09 #SP2010


Last week, while at the SharePoint Conference in Las Vegas, I wrote a blog post on being disappointed that SharePoint 2010 does not seem to include a critical feature promised by Bill Gates and Tom Rizzo in their keynote speech at last year's (2008) SharePoint Conference.

The feature promised was a one-to-one relationship (as needed for specific applications) between a SharePoint List and a native database table (SQL Server, Oracle, etc.). I explain in my first blog post why I think this is an extremely important feature to have.

Turns out that this feature IS included in SharePoint 2010! It is just that it did not receive the level of coverage in this year's keynote speech that I was expecting based on the expectation that Microsoft built up in last year's keynote speech. So, I missed it even though I was sitting in the audience during last week's keynote.

I discovered that the new feature exists (and truly looks outstanding) after getting back home and starting to watch some of the recorded breakout sessions that I didn't get a chance to attend.

The specific new features that enable this are named External Content Types and External Lists. These two new features fall into the overall category of the new Business Connectivity Services feature-set of SharePoint 2010.

The way this works is that you can now create (using SharePoint Designer 2010 or Visual Studio 2010) External Content Types which are really just a definition of an external database table (or web service). Once an External Content Type is defined for the external database table, an External List can be created (using the SharePoint 2010 browser UI, SharePoint Designer 2010 or Visual Studio 2010). The External List behaves just like any first-class List in SharePoint 2010. Anything you can do with a native List in SharePoint 2010 (workflows, permissions, views, etc.) can be done with External Lists.

But, the data that is displayed and stored for an External List is not stored in the SharePoint 2010 content database. The data in the External Lists lives in the external database table - and, only there. Therefore, if it is updated by a non-SharePoint application (think an ERP system or CRM system for instance), the data that is displayed in the SharePoint External List will update immediately.

This is totally different from the way the MOSS 2007 Business Data Catalog worked. In the MOSS 2007 BDC, the business data brought into SharePoint was stored in a List column(s) in the SharePoint Content Database. So, if the data changed in the external database, the values in the corresponding SharePoint List did not get updated immediately.

Futhermore, in the MOSS 2007 BDC it was not really possible to update data in the external database from within SharePoint. With External Content Types and External Lists in SharePoint 2010, all of that has changed. If you change a value of a column in a SharePoint 2010 External List, the new value is immediately written to the external database.

The last, but certainly not the least, important announcement is that External Content Types and External Lists are "baked-in" to SharePoint 2010 Foundation (the new name for WSS 4.0). This means that all of this goodness is now available to any organization that chooses to use SharePoint 2010, regardless of whether you are using a free or premium version of SharePoint.

Very exciting!

Tuesday, October 20, 2009

Lists as Native SQL Tables in SharePoint 2010? #spc09


Even after today's keynote speech by Steve Ballmer and Tom Rizzo, I am wondering if this got included in SharePoint 2010?

I expected it to be one of the big announcements, but heard nothing about it. I also have seen nothing to indicate that it got added as I have played around with the Technical Preview version that we have had and were under NDA until today.

Why was I expecting this to be one of the big announcements? Only because at the SharePoint Conference in 2008 in Seattle it was promised at the keynote by Bill Gates and Tom Rizzo (the Director of SharePoint at Microsoft).

Why am I so anxious about this feature being included? The reason is that the guts of SharePoint (including document libraries) is built 100% on SharePoint List technology. And, it is the limitations of SharePoint List technology that has held back the possibility of porting thousands of applications around the world to SharePoint. What Gates and Rizzo talked about in 2008 seemed to open the door for the possibility for the bread-and-butter line of business (or vertical market) applications that are prevalent in industry after industry, to be ported to SharePoint.

Why would anyone want this to happen? Well, from a user perspective, in many organizations SharePoint and the Office client applications have been slowly but surely becoming the business application operating system - the way that Windows became the business application operating system many years ago. A big chunk of daily tasks now get done by rank and file users that are primarily using Office and SharePoint to get them done. Isn't the next logical step to port the line-of-business applications that they use to be native SharePoint applications in order to provide the best user experience?

The problem to-date is that there is not a one-to-one relationship between a SharePoint List and a native SQL Server table. Instead, there is a single (as in one) native SQL Server table that contains all of the list data for all lists in a SharePoint site collection. (In organizations that have SharePoint implemented in a single site collection, that means all field, row and list data is contained in a single SQL Server table.)

Most database-oriented people that have worked with SharePoint for a few years know that this is the case, but it is not something that is widely discussed and many newcomers to SharePoint don't know about this fact. So, just to drive this home, if you are in an relatively small organization that has say 75 sites in a site collection and an average of 6 lists in each site (task lists, announcement lists, links lists, etc, etc.) and an average of 60 rows in each list, that means that all of this data, a total of 27,000 records are all stored in a single SQL server table. Therefore if you assumed that the 450 lists in your site collection represent 450 SQL Server tables, you would have assumed wrong. All the rows for all 450 of these lists are stored in a single SQL Server table that just continues to grow and grow as you add more lists and rows.

Imagine what this SQL Server table looks like at a large multi-national corporation for one of their site collections? There might be millions and millions of records in this SQL Server table.

The bottom line to all of this is that SharePoint Lists are abstracted from the native SQL Server database. In other words, no matter how many lists you create and how many columns and records you add to them, all of that data is stored in a single SQL Server table that only SharePoint has the permission to write to directly.

Of course, this defies all of the rules of architecting fully normalized database structures. But, when SharePoint was conceived, Lists were not envisioned as being used just like relational database tables. The thought was that SharePoint Lists were just a better way to do everyday data tracking activities that users had previously relied on Excel spreadsheets for. Therefore a single table that is not normalized would be OK for SharePoint List data.

But now, the user interface of SharePoint has become so popular and the richness of the common SharePoint feature-set so powerful, that users are wanting more. Why shouldn't application developers be able to develop native relational database applications using SharePoint as the platform? Only one major reason that I can think of - SharePoint Lists are not available as native SQL Server tables that can support complex relationships, triggers and a host of other important functions that are needed to develop powerful database applications.

That gets me back to where I started. Is this feature (option to have native SQL Server table for a SharePoint List) really in SharePoint 2010 and I have just missed it? So far, I can't find it - and Bill Gates is now retired :( .

Important Update: New information related to this feature can be found in this new blog post. It definitely looks like the feature has been included in SharePoint 2010 - it just wasn't very apparent!

Wednesday, October 14, 2009

SharePoint Conference: Theater Sessions at Our Booth + Ipod Nanos


We will have our team and a large booth at the SharePoint Conference next week in Las Vegas. We've designed the booth this year with a mini-theater area (50" plasma, sound system, chairs, etc.) so that our experts can conduct short sessions on topics that we feel we have a special perspective on.

We plan for these sessions to be short (approximately 15 minutes) and on a very specific topic. So, if you are in the exhibit hall and want to take a break from swag hunting for a few minutes and see a quick presentation delivered by an expert on the topic, we invite you to stop by. Like I said, this is a "mini" theater, so don't expect a big area that will accommodate hundreds. We'll have a half-dozen seats available and then standing room for another 10 - 12 people.

You should be able to easily find us as we will be right next to the Microsoft Pavilion where Microsoft will have lots of individual stands demonstrating various features in SharePoint 2010.

Here is the schedule we are planning for right now:

Sunday Evening, October 18:

6:15 - 6:30, "Top 3 Best Practices for Deploying SharePoint on the Extranet"

6:45 - 7:00, "Easy SharePoint Extranet with Extranet Collaboration Manager"

7:15 - 7:30, "Outlook Email and SharePoint Integration with WISDOM Message"

7:45 - 8:00, "SharePoint Designer Workflow Series: Looping Through SharePoint Lists"

8:15 - 8:30, "Automating and Controlling Your Site Request, Creation and Permission Process"

Monday Afternoon, October 19:

4:00 - 4:15, "Souping up SharePoint for Document Management: WISDOM DMF"

Tuesday Afternoon, October 20:

4:00 - 4:15, "The New SharePoint Help Community Website"

Wednesday Afternoon, October 21:

4:00 - 4:15, "After One Year with MOSS for the Internet: Lessons Learned From sharepointsolutions.com"

Wednesday Evening, October 21:

5:45 - 6:00, "Expert Opinion: The New Workflow Features in SharePoint 2010"

6:15 - 6:30, "Expert Opinion: Impact of SharePoint 2010 on Extranet Deployments"

6:45 - 7:00, "Expert Opinion: Usability Improvements in SharePoint 2010"

In addition to these sessions, we will always be available (when the exhibit hall is open - which, unlike previous years, is most of the time) to talk and do one-on-one demos.

Finally, we will be giving away one of the new 5th Generation Ipod Nanos (the new one that comes with an integrated video camera) each day. All you have to do to get in on the drawing is come by the booth and let us scan your badge to enter you.

If you are attending, make sure to stop by and say hello! Looking forward to seeing everyone at the conference!

SharePoint 2010 Training is Coming!



In preparation for the SharePoint Conference next week, on our website we've outlined our plans for offering SharePoint 2010 training.

The short version of the story is that we have already started working on a full lineup of SharePoint 2010 courses that will be targeted at a variety of audiences such as: IT Administrators, IT Developers, IT Managers, non-IT Managers and non-IT Users.

Furthermore, one of the courses, "SharePoint 2010 Skills Upgrade and Planning" will begin to be offered in December (assuming the public beta of SharePoint 2010 is released sometime in November.)

This new SharePoint 2010 course will be offered 1 - 2 times per month at different locations around the US and online. It is being designed by, and will be taught by, our most seasoned SharePoint instructors/consultants. It will be targeted at IT Professionals and non-IT Project Leaders who will be responsible for planning out the SharePoint 2010 adoption strategy for their organization.

If you would like to be notified when we release new information about our SharePoint 2010 training classes (such as course outlines, dates, locations, etc.), I encourage you to visit this page.

We're getting excited about SharePoint 2010 and are looking forward to offering a new slate of classes!

Friday, October 09, 2009

WSS as a platform for public-facing, anonymous-access Internet websites


As I mentioned in a post last week, our public website is a MOSS-based site and we are celebrating its first birthday this month (We launched it on October 1, 2008. Prior to that we had a standard HTML website since 2003.). For the most part, we are very happy with our decision to launch our new site on MOSS.

Reality is that there aren't that many companies that can afford to pay the licensing costs to have their public Internet running on MOSS. The stated retail price for MOSS for Internet Sites is $41,134 per server.

So, if you are running a couple of load-balanced servers for your Internet site, you are looking at around 80k to license MOSS for Internet Sites for them. The only piece of good news here (financially speaking) is that this gives you unlimited anonymous user access - so there are no CALs to buy. (In case you are wondering, since we are Microsoft Gold Partner, we get our licenses as part of the Gold Partner program and pay much, much less than this - otherwise, we wouldn't be able to afford MOSS for our Internet site.)

So, what are you to do if you want a SharePoint-based Internet site? Well, it is not all that widely publicized, but several businesses around the world have figured out how to do this quite nicely with WSS only. And, more importantly, there are no (zero) SharePoint licensing costs if you do it this way.

You say, hold on now, there is no way that can be true! First, I don't believe that you can develop a really polished public site with WSS!

Well, I am not going to argue about this using words. Instead, I am going to point you to some examples that I think prove differently. All of these sites are built using WSS 3.0:

http://6sc.com/default.aspx (check out this page for how they did it)

http://www.zevenseas.com/en/default.aspx

http://www.nestle.si/default.aspx

http://www.acision.com/default.aspx

There are more out there, but these are four that I particularly like.

OK, if I have convinced you about the viability of developing a public website using WSS 3.0, what about whether or not I have my facts straight about the no-cost for licenses?

First, in case you don't already know, WSS 3.0 is a free add-on to Windows Server 2003 and Windows Server 2008. If you have Windows Server licensed properly, you don't have to pay anything else for WSS - period. This is a pretty commonly known fact so I am not goint to bother hyperlinking to any proof of it.

Second, there is a very popular and well-respected Microsoft (as in, employee) architect in New Zealand, Ian Morrish, who blogged about this in March of this year. Here is the link to his blog post: http://www.wssdemo.com/Blog/archive/2009/03/06/Web-Content-Management-with-Windows-SharePoint-Services.aspx. Make sure and read the comments at the bottom of the post, because Ian says the following in response to a question about whether this is "legal" or not:

Ian says:

"It is legal. If you are using full SQL server then SQL Server must be per-proc licensed but the built in DBthat comes with single server WSS is free. If you want to authenticate Internet users then you only need the Windows Internet licence which is cheap compared to MOSS for Internet license."

The Windows Internet license he mentioned (its formal name is Windows Server External Connector license) is required (but, according to Ian only if you want to authenticate external users - which most Internet sites don't), but it is a Windows Server licensing add-on, not a SharePoint license. Again, as long as you have Windows Server properly licensed, there are NO additional licenses required for WSS in any scenario. WSS is truly a free add-on to Windows Servers.

I know at this point you are wondering what is the cost of the Windows Server External Connector license if it turns out I need it? Well, according to this page on Microsoft's site it retails for $1999 for Windows Server 2008 (scroll down towards the bottom to find it). This license is purchased per server. So, if you have a two server farm for running your public website and you want to use WSS as the platform, the retail license cost to get you there would be $3998. Furthermore, this will allow unlimited external users to access your website.

Now, I am not saying that WSS is as good as a platform for public websites as MOSS is. PLEASE, don't hear me say that! MOSS has some great features that make it very, very nice as a web content management system. WSS does not include these features at all. One of the biggest of these features is the publishing subsystem that allows you to decentralize content authorship for different portions of you website. It includes a robust approval system so that approver's can be assigned to review authored content before it goes live. WSS does not have this same level of functionality.

But, for those companies where MOSS is simply not an option due to the cost, it certainly appears that there is evidence that WSS 3.0 can be used successfully as an extremely low-cost platform for building Internet sites.

I'd say that right now WordPress is probably the most popular open source platform for building high-quality websites. If you thought WordPress is only for blogging, you thought wrong. There are thousands of websites around the world running on WordPress (here is a good example). I like WordPress a lot, but I think WSS 3.0 is a very capable website platform as well and would definitely evaluate both of them if I were setting out to build a new website.

Monday, October 05, 2009

Updating SharePoint Profiles from ASP.NET Profile Data


Becky Bertram has published a nice developer guide on how to update SharePoint profile data from ASP.NET profile data. I wanted to make sure our customer's saw this post, so here's a link:

Updating SharePoint Profiles from ASP.NET Profile Data

Nice article Becky. Thanks for posting it.

Thursday, October 01, 2009

MOSS for Internet and 301 Redirects


We've been using MOSS for our public Internet site for SharePoint Solutions for almost a year now. We have a very broad range of products and services, and therefore our site is what I would categorize as "moderately" complex. It takes work every day to keep all of the content fresh, do a good job marketing our products and services, and enhance usability. If you are reading this and your title is Webmaster and that is your full-time job, you know exactly what I am talking about (regardless of whether you are using MOSS or Drupal or something else).

As you may know, MOSS 2007 is version 1 of the web content management features of SharePoint that really make SharePoint a viable platform for a public-facing Internet site. Microsoft had a previous product designed for this purpose, Microsoft Content Management Server, which was retired with the release of MOSS 2007.

We like MOSS 2007 as a content management platform for a public Internet site, but the fact that it is a version 1 product continues to be evident to us every day.

One example is its inability to natively handle 301 redirects. This may sound like something that is no big deal, but it is a very big deal if you are using MOSS for a public Internet site and care about searching engine optimization (SEO).

The reason is that if you change the URL to an existing page on your site, the major search engines (Google, Bing, Yahoo) will still have your existing page in their indexes and will therefore still display your old URL in their listings until they find and index the new page. If you want them to find the new page and index it properly, the best practice way to do this is by creating a 301 redirect from the old URL to the new URL. This tells the search crawler that you have replaced Page A with Page B and that you would like for it to do the same in its index.

It is critical that the redirect is a type 301, not a type 302. A 301 is a "permanent" redirect, while a type 302 is a "temporary" redirect. According to the experts on search engine optimization (SEO), the search engines honor 301 redirects and will update their indexes accordingly, but they do not honor 302 redirects.

Getting back to MOSS and 301 redirects - there is no way to natively redirect one page in MOSS to another page and return a 301 permanent redirect status to the requestor (be it a human or a bot). Several people have offered up custom developed code to do this, but there is no consensus on whether any of these is really a good solution or not. Even if one of them is, it is still a hack to native MOSS to get a feature that really needs to be supported out-of-the-box in some user-friendly way because changing URLs is something that the average content manager is going to need to do quickly and easily from time to time over the life of a web site.

Interestingly, MOSS does offer a native and easy way to achieve a 302 temporary redirect. If you are using the Publishing Sites feature (which is what most MOSS public Internet sites use), you can enable a "Redirect Page" content type in your publishing pages libraries. This is an out-of-the-box content type, that will allow you to create a page that has a property that allows you to enter a URL to redirect to. From a live visitor standpoint, this certainly gives you a way to easily redirect from an old page to a new page. But, from a search bot standpoint, it doesn't do anything for you at all.

So, what is the solution? Well, first, I am hoping that something has been added to SharePoint 2010 to allow a content manager to create a 301 redirect from the user interface. Guess I need to add that to our list of what the community is hoping will be in SharePoint 2010.

In the meantime, the two different categories of solutions that I have been able to find are:
  1. Use custom code to enhance your MOSS for Internet implementation to be able to return 301 redirects (here is one example), or
  2. Use "URL re-writing" technology with IIS to accomplish this (for IIS6 here is an example, for IIS7 there is now native capability to do this). This technology will allow you to set up a map of old URLs to new URLs and then it will return 301 permanent redirects to requestors.
In my mind, neither of these is a great solution given the fact that the whole idea behind a web content management system is to put the power to maintain the website in the hands of non-technical content authors and content managers. Option 1 requires programming, which these folks can't do. Option 2 requires access to the web front-ends and configuration of IIS-related settings, which these folks probably can't do and even if they could would not be given access too.

So, really what is needed here is an out-of-the-box feature to support page-level 301 redirects. Will it be in SharePoint 2010?.........................