Saturday, February 21, 2009

Exploring Live Framework Activities

Live Framework has an interesting undocumented feature called Activities.  You can think of Activities as a more general-purpose, transient alternative to News in the Mesh.  Currently the only resource for learning more about Activities is John Macintyre’s PDC session Live Services: Notifications, Awareness, and Communications.  Some of the information in this post is from John’s presentation, but much of it comes from my own exploration.

What are Activities good for?

Unlike News, Activities aren’t necessarily meant to be displayed or used in a predefined way.  Live Mesh uses Activities to track transient state such as which users are online, which users are currently in a folder, and which users are currently using a particular app (more details on this later).  You can use Activities in your own apps to build features such as chat, remote control of apps, and near-real-time transmission of small messages (when used with notifications).

Where can you use Activities?

Only cloud LOE MeshObjects expose Activities feeds.  There is no Activities link from client LOE MeshObjects.  Technically ApplicationInstance also has an Activities feed, but in practice the ApplicationInstance Activities feed points to the Activities feed of the app’s MeshObject.

An interesting side note is that ApplicationInstance is essentially the same resource as its MeshObject.  It shares the same entry id and contains a copy of all the MeshObject’s elements, plus a few more app-specific elements.  I assume these entries map to a single CoreObject under the hood.

What do Activities look like?

An Activity has the following interesting data elements:

  • MemberLink (Uri)
  • ActivityTime (DateTimeOffset)
  • MaxAge (short int)
  • Type (string)
  • UserData (serialized object)

MemberLink is created for you automatically and points to the parent MeshObject’s Member entry corresponding to the user who created the Activity.  You can’t create a MemberLink that points to a different user’s Member entry.

ActivityTime is optional and is separate from the entry’s published and updated times.  You can specify any ActivityTime you want in the past or in the future, although you would typically set it to something like DateTimeOffset.UtcNow.

MaxAge is the number of seconds until the Activity expires.  MaxAge is automatically generated for you.

Type can contain anything you want.  Later we’ll see an example of how Live Mesh encodes user presence in Type.

UserData is the same GetUserData/SetUserData extensibility point you find on MeshObject and DataEntry.

The Title of an Activity is always blank, even if you try to supply one.

How do Activities behave?

The most interesting Activity behavior is the automatic expiration based on MaxAge.  When an Activity is created, a random MaxAge between 600 and 900 (between 10 and 15 minutes) is assigned.  This randomness is intended to smooth out the server load in the datacenter.  The same technique is used for Subscriptions.  See slide 11 of John’s presentation for an illustration of what happened before they started using random TTLs:

PatchTuesdayTtlBadness

Once an Activity has been created, its MaxAge value counts down to zero.  You can see the countdown happen as you poll the Activity or its feed.  When MaxAge reaches zero, the Activity entry automatically disappears from the feed.  The Activity actually hangs around for a few seconds while MaxAge is zero, and during this time the MaxAge element is missing from the entry.

You can update an Activity entry which will reset its MaxAge to a new random value between 10 and 15 minutes.  Later we will see how Live Mesh user presence uses this technique.

You can also delete an Activity immediately without waiting for MaxAge to expire if you so choose.

Activity feeds are subscribable, so you can register to receive notifications as entries are added or removed.  Entries that are removed due to MaxAge reaching zero also trigger a notification.  I’m not sure if you will receive a notification if Activity state is lost due to unexpected server state loss (remember, Activities are in-memory only).

Activity feeds are not extensible, meaning you can’t use ElementExtensions and AttributeExtensions.  As mentioned earlier they do support UserData which is hopefully sufficient for most scenarios.

Activities support Create and Update triggers but not Delete triggers.  This is the same partial trigger support exhibited by Contacts.

Earlier I mentioned you can’t create a MemberLink that points to a different user’s Member entry.  It turns out that you also can’t delete an Activity created by another user, even though you can delete your own Activities in the same feed.  That’s right, in this case it is possible to have different permissions on each item within a single feed!  So much for MeshObject being the most granular unit of permissioning… ;-)

The OPTIONS verb and $metadata do not work for Activities.

What is the programming model?

Today, Activities are only exposed through the resource model as MeshObjectActivityResource.  There is only an ActivitiesLink property on MeshObject, not an Activities collection, and there is no LiveItem-based Activity object.

Actually, that’s only true for the .NET and Silverlight SDKs.  The JavaScript SDK has a MeshObjectActivity class and MeshObject has an Activities property, but I haven’t used the JavaScript SDK.

For now, using Activities from .NET or Silverlight requires using AtomPubClient, Resource Scripts, or raw HTTP.

I’m guessing Activities aren’t yet exposed through the high-level programming model because of the additional complexity of MaxAge expiration, automatically handling MaxAge resets, and the possibility of wrapping Activities in a scenario-specific programming model for user presence.

Behind the scenes

The Mesh is composed of a variety of services spread across many servers in the data center.  Some of these services use reliable state stores and others use soft state stores.

Reliable state includes the Accounts store (accounts.developer.mesh-ctp.com), user-data structured storage (storage.developer.mesh-ctp.com), and user-data blob storage (enclosure.developer.mesh-ctp.com).

Soft state includes storage for device presence, notification queues, subscriptions, activities, and dictionary state for the Live Desktop.  You can see most of these on slide 21 from Abolade’s PDC session:

MeshServicesDeployment

The design of the Activity Service is similar to the notification and subscription services.  This means the Activity Service uses only in-memory tables for high performance, so state loss is a possibility.  Therefore you shouldn’t use Activities for any state that you can’t afford to lose.  On the bright side, the high performance in-memory design of Activities fits together quite nicely with notifications and subscriptions.

The client LOE doesn’t appear to implement any of the soft state stores at this time.  This is probably why client LOE MeshObjects don’t have an Activities link. Update: The client LOE implements Notifications and Subscriptions, although they behave slightly differently from the cloud LOE. However, the client LOE still doesn't implement Activities.

How does Live Mesh use Activities?

One way Live Mesh uses Activities is to track which users are currently in a Live Folder.  Here’s an example of such an activity:

<entry>
<id>...</id>
<title type="text">
</title>
<published>2009-02-21T08:27:15Z</published>
<updated>2009-02-21T08:27:15Z</updated>
<link rel="LiveFX/Member" title="LiveFX/Member" href="Mesh/MeshObjects/.../Members/..." />
<link rel="self" title="self" href="Mesh/MeshObjects/.../Activities/...-..." />
<link rel="edit" title="edit" href="Mesh/MeshObjects/.../Activities/...-..." />
<category term="Activity" label="Activity" scheme="http://user.windows.net/Resource" />
<content type="application/xml">
<ActivityContent>
<UserDataBuffer>
</UserDataBuffer>
<ActivityTime>2009-02-21T08:27:13Z</ActivityTime>
<MaxAge>539</MaxAge>
<Type>UserActivity:Type[Presence];LiveFolderId[::{4ba12b8a-865f-4f52-99a4-12901be64d54}];</Type>
</ActivityContent>
</content>
</entry>

The important pieces of information are the link to the user who is in the folder, the ActivityTime when they opened the folder, and UserActivity encoded in the Type field.  LiveFolderId is the entry id of the folder’s MeshObject.  Type[Presence] implies that other user activity types are probably tracked using the same format.

If you open a folder on the Live Desktop and watch the corresponding Activity entry, you will see that the Live Desktop does a PUT to reset the MaxAge when 15 seconds are left.  All of the other entry details such as ActivityTime remain unchanged.

If you close the folder, the Activity entry is deleted immediately without waiting for MaxAge to expire.

It turns out that anything that has a Mesh companion bar tracks user presence activity.  This means that Mesh-Enabled Web Applications also get this same behavior for free.  In this case the UserActivity’s LiveFolderId is the entry id of the app’s MeshObject.

The Activity’s Member link is used by the Mesh companion bar to display an orange box next to users who are currently in the folder or app.

MeshBarPresence

In this case you can see that I am playing Collaborative Crossword all by myself.  Apparently Ray doesn’t have time for me anymore. ;-)

There is nothing stopping you from creating an Activity that makes another user believe you are in a folder or app when you’re not, although you would have to periodically update the Activity to continue to appear present.  So if you must be present to spoof your presence, how much are you really spoofing?  Hmm…  It may also be possible to delete Activities to hide the fact that you’re in a folder or app, although I assume the Activity will eventually be recreated.

You would think that user presence should be determined by the combination of the Member link and the specially formatted Type field’s Presence and LiveFolderId, but it turns out that all you need to do to appear present is create any Activity whatsoever in a MeshObject’s Activities feed.

Source Code

I have source code that demonstrates much of what I’ve described, but it is part of a larger project that also demonstrates notifications and subscriptions, so I will publish the source with my notifications and subscriptions blog post.  If you were surprised by how much there is to know about activities, just wait until you see notifications and subscriptions!

Monday, February 16, 2009

Mesh4Linux

Last week my eyes lit up when I saw the following tweet from Miguel de Icaza, leader of the Mono project.

Avatar_biggermigueldeicaza

Miguel de Icaza

RT: @bradyanderson:is trying to authenticate my Linux Live Operating Environment using Windows Live Delegated authentication (Mesh4Linux)

4:21 PM Feb 11th from web

I immediately tried to find out more, but this is the only mention of “Mesh4Linux” I could find on the web.  Brady is a distinguished engineer at Novell who works on the Mono project.  I checked out his tweets, and he’s been talking about building a Live Mesh / Live Framework implementation for Linux ever since PDC.

Photo_16_biggerbradyanderson

is attending the "What I learned building My first live mesh app" session at #pdc2008

11:35 AM Oct 28th, 2008 from twitterrific

is excited to start hacking on a Live Operating Environment for the Linux Desktop. Opens the door for some amazing applications

2:04 PM Oct 29th, 2008 from twitterrific

created his first MeshObject on Linux! Now DataFeed, DataEntry, Membership.... bows his head and goes away quietly

1:02 PM Dec 10th, 2008 from twitterrific

just implemented DataFeed creation and Mesh/MeshObject enumeration

4:06 PM Dec 11th, 2008 from twitterrific

"is hoping this is the first twitter message pushed from the sync framework (2)"

2:18 PM Jan 27th from web

is researching Differential Synchronization algorithms

10:04 AM Jan 28th from twitterrific

is catching up on the latest Live Mesh developments. I can't seem to find the January tools update :-(

8:11 AM Feb 11th from web

is trying to authenticate my Linux Live Operating Environment using Windows Live Delegated authentication - getting closer.

2:55 PM Feb 11th from web

the CTP versions of Live Mesh and Azure Services are so slow they're barely usable. *screams profanities*

3:10 PM Feb 11th from web

I’m guessing Brady is the guy Ori Amiga is referring to in the following quote from this PDC session (15:50 onwards):

The whole point I wanted to make, it's just plain good old HTTP, and if you can talk that, every device, programming language, stack is welcome to the party.

Some guy walked up to me after the stage yesterday, if you're here I'd love to keep chatting with you, said man I want to write Live Operating Environment for Linux. Can I do that?  I was like, hell yeah, we'll hire you, come write it even in-house if you want to.

But really the idea is the Mesh will never be, I can't imagine we'll be successful in making people's lives better if we only stick to a Microsoft stack.  That makes no sense.  let's say, I'd admit, most of my devices at home and my receivers, my TVs, all the media stuff we have, the car, they don't run Windows, and that's ok, there's nothing wrong with that. It's great that my Windows devices are gonna behave really well in the Mesh, but I'd love for everything else that's sort of net connected to behave that way as well.

It is worth noting that Ori Amiga, a Principal Group Program Manager on the Live Mesh team, has built multiple carputers, one using Linux and another using Live Mesh.

I pinged Brady and Miguel for details on Mesh4Linux but haven’t heard back yet.  There appears to be no connection to Mesh4x, another open source project with many similarities to Live Mesh, including FeedSync support.

Mesh4Linux is in the early stages at this point, but I’m already dreaming of the possibilities it will enable not just on Linux desktops but on the iPhone, Google Android phones, and embedded devices such as carputers.  Go Brady!

2/28/09 Update:

Scott Hanselman used kyte to live stream Miguel’s Mono on iPhone session at the Alt.NET conference in Seattle.  Via the online comments, I asked about Mesh4Linux (as well as Miguel’s communist C# Turkish flag t-shirt).  Scott got a chance to ask Miguel about Mesh4Linux at the end (50:08) right after Miguel mentioned the benefits of sync for disconnected scenarios:

SH: What about Mesh4Linux?

MdI: There is no Mesh4Linux as far as I know.  I know there is an engineer at Novell who wants Mesh4Linux.

SH: So it’s a dream, not a project.

MdI: Yes.