MeshScript Ideas for the Future

In my last post where I added LiveItem syntax to MeshScripts, I said I had some more ideas for MeshScripts.  Some of these ideas are very small, some are very big, and some are in between.  The reason I’m listing them here is that there’s no way I could get to even a fraction of them (I’d like to move on and explore other areas of Live Mesh), so hopefully they spark your imagination.

Enhancements for existing MeshScripts

These enhancements could be applied to MeshScripts without taking a dependency on the library I wrote.

Chunking and chaining

There is an upper limit on the number of statements the server will process in a single script.  It would be nice to implement automatic script chunking and chaining based on a configurable statement batch size.  For cross-batch bindings, outputs from one batch can be fed into inputs for the next batch.  Special care needs to be taken at CompoundStatement boundaries, especially with ConditionalStatement.  This could also be used to implement a progress indicator for large jobs while still preserving most of the performance benefits of batching.

Automatic parallelization option

I’m guessing most people are going to write their scripts using SequenceStatement as their CompoundStatement of choice.  It would be cool to have the option of automatically transforming and optimizing scripts by wrapping sections in InterleaveStatements where possible based on analysis of binding dependencies and URLs.  This ought to be a feature the user explicitly opts into.

Enhancements to Fluent MeshScripts library

These enhancements are specific to my Fluent MeshScripts library.

Use control flow MeshScript features

Note that ScriptContext’s record-replay model doesn’t need any “programmatic” script control flow features such as ConditionalStatement or the coming-soon LoopStatement.  Perhaps there’s an opportunity to add If/Else logic to the SLiveItem syntax.  This would require an expression tree visitor to touch up references to script statements since currently you must hard-code the statement ID/Name in the condition.

A switch statement could be added that uses multiple ConditionalStatements under the hood.

A similar sub-ScriptContext scoping solution could be used for generating InterleaveStatement sections.

Until we get a real LoopStatement, a fake Loop statement could be created that unwraps the loop a specified number of times.  This could be used for scenarios such as copying the last 10 items from a Twitter feed into a Mesh feed.

Miscellaneous enhancements

Create a helper method that enables single-round-trip Live Folder creation that returns LiveItems.

I haven’t investigated expansions yet, but it seems like there’s an opportunity to take advantage of them in MeshScripts, preferably with helper methods to simplify common scenarios, whatever those might be.

Taking MeshScripts in bold new directions

Here are some wild and crazy ideas that probably won’t come to pass, but wouldn’t they be cool?

Add batching to LiveItems

It would be nice if LiveItems had the option of operating either in batched mode (like my SLiveItem implementation) or real-time mode (as they currently do).  Perhaps this could be enabled through a new Batched property on LiveItemAccessOptions.  This means LiveItems would be able to speak both AtomPub/FeedSync and MeshScripts.

Escaping the MEWA sandbox

This next one is more of a “see if it’s possible” item than a new thing to implement, but it could open up some interesting new scenarios that would be ripe for additional library development.  It would be interesting to see if MEWAs can use MeshScripts to escape the MEWA sandbox, either by calling them as sprocs or as triggers.  Of course if this is possible, it’s likely it will be quickly disabled, but who knows, perhaps it’s acceptable.  I tried running a simple ReadCollection script from a Silverlight MEWA and got an exception trying to deserialize the result (no public default constructor), so I haven’t pursued it further.

Yahoo Pipes for AtomPub

Yes, building Yahoo Pipes for AtomPub and FeedSync would involve much more than just MeshScripts, but I think MeshScripts could play an important role in its implementation, especially with the forthcoming visual script designer.

This idea first came to me as I was experimenting with pulling in external Atom feeds using MeshScripts (it’s also possible using LiveItems).  Most feeds had formatting that broke the script, but a few external Atom feeds magically worked.  I thought, wouldn’t it be nice if there were a MeshAtomTidy service that touched up external feeds with the appropriate data to ensure they load nicely into the Mesh?

That would be great for read-only feeds.  Wouldn’t it be even nicer if you could map LiveID credentials to external credentials and access other AtomPub APIs such as Google Calendar, Google Spreadsheets, Picasa, and more?  It would also be cool if you could write a little bit of glue code or script to wrap an AtomPub API around non-AtomPub APIs such as Twitter.  Or even better, just select from a list of pre-existing AtomPub wrappers for popular services.

Next, I’d like to enable automatic synchronization between Mesh feeds and external feeds.  It’s not very exciting to send tweets while you’re offline, but offline access to the Google APIs is more compelling.  There may be a need for additional transformations and business logic in between which is where the full-featured suite of Yahoo Pipes modules comes in handy.

Popfly integration

The closest Microsoft equivalent to Yahoo Pipes is Popfly.  It has a similar set of modules and a drag-and-drop design experience.  Perhaps there is an opportunity to integrate Popfly mashups with Mesh feeds.  It would also be cool if you could package Popfly games as MEWAs that can run on your desktop or on your phone and maybe even sell them through a Live Mesh App Store, but that’s probably enough crazy talk for one blog post. :-)

MeshScript Queries, LiveItems, and Magic

I’ve continued to extend my Fluent Resource Scripts library in several interesting new ways.  I’ve added:

• Strongly typed LINQ queries
• Turning script results into LiveItems (MeshObject, DataFeed, DataEntry, etc.)
• LiveItem syntax for scripts

I’ve had to resort to more significant hacks to implement these, and I don’t feel that these features are as solid or as complete as the features in my original Fluent MeshScripts post.  But I think these ones are far more interesting, so I hope you will look past the rough edges and imagine the potential if these features were done properly.  The LiveItem syntax for scripts is especially cool, if I do say so myself.

Queries

Both LiveQuery and ResourceQuery let you generate query strings from strongly-typed LINQ queries.  ResourceQuery is broken, which is unfortunate because its only generic parameter is of type Resource, which also happens to be the only generic parameter for most script statements.  This would have allowed us to implicitly pass along the statement’s generic parameter to our helper method without having to write any generic angle brackets.

So we have to use LiveQuery instead, which takes a generic parameter of type LiveItem (the non-generic LiveItem, not LiveItem<TResource>).  This makes it so that instead of calling my helper method like this:

S.ReadResourceCollection<MeshObjectResource>(ScriptHelper.MeshObjectsUrl)

    .WithQuery(
   q => q.Where(o => o.Resource.Title.StartsWith("my")))

You must instead call it like this, specifying both the Resource type and the LiveItem type:

S.ReadResourceCollection<MeshObjectResource>(ScriptHelper.MeshObjectsUrl)

    .WithQuery<MeshObjectResource, MeshObject>(
   q => q.Where(o => o.Resource.Title.StartsWith("my")))

Oh well, it’s still useful, and once ResourceQuery is fixed we can switch to the shorter syntax.

I should note that these queries are immediately turned into query strings under the hood.  In other words, they are evaluated at script design-time, not at script runtime.  I have put in a feature request to support query generation at runtime.

Turning script results into LiveItems

So you’ve written a resource script, you run it, and you get some results back.  Then you think, “I’d like to do further work with the results I’ve gotten back.”  You dig into the individual statements in the script’s Result property, cast them to the appropriate statement type so you can then dig into the Resource property and examine the actual data that was returned.

But what if instead of working with results MeshScript-style, you wanted to work with them LiveItem-style, without having to do a bunch of tedious digging and casting?

DataFeed outFeed = null;

using (new ScriptContext(loe))

{

    S.Sequence(

        ..., // create moStatement

        S.CreateResource(scriptFeed)

            .AtUrl(moStatement, mo => mo.Response.DataFeedsLink)

            .SaveResult(ref outFeed)

    ).Compile().Run();

}

DataEntry de = new DataEntry("new entry");

outFeed.DataEntries.Add(ref de);

I’m not quite sure what programming idiom to compare SaveResult() to since the result isn’t actually saved until the script is run.  The closest thing I can think of is a future.

SaveResult() creates a new LiveItem of the appropriate type, and after the script is run, this LiveItem is filled in with all of the necessary response information to give you a “live” LiveItem that you can start partying on right away without requiring another server round-trip to flesh it out.  This required a fair bit of reflection magic because by default LiveItems are essentially “DeadItems” until they’re associated with a LOE and Reloaded.

Notice I’ve added the use of a ScriptContext to enable the generated LiveItems to be automatically associated with an existing LiveOperatingEnvironment.

SaveResult() has been implemented for CreateResourceStatement, ReadResourceStatement, and ReadResourceCollectionStatement.  I have also added Statement extension methods ToMeshObject(), ToDataFeed(), and ToDataEntry() if you prefer to work that way.

When you combine ReadResourceCollection with SaveResult() and the query support above, you can now batch multiple LiveItem queries.

LiveItem syntax for MeshScripts

So we’re starting to get a decent bridge between the world of MeshScripts and the world of LiveItems.  But man, that MeshScript syntax is still pretty nasty, even with the fluent stuff I’ve added.  Wouldn’t it be nice if you could write a MeshScript the same way you write LiveItem code?

using (new ScriptContext(loe))

{

    var mo = new SMeshObject("original title");

    var feed = new SDataFeed("first feed");

    var feed2 = new SDataFeed("second feed");

    var feed3 = new SDataFeed("third feed");

    mo.Resource.Title = "script-generated title";

    mo.Resource.Type = "LiveMeshFolder";

    feed.Resource.Type = "LiveMeshFiles";

    feed2.Resource.Title = feed.Resource.Title;

    feed3.Resource = feed.Resource;

    mo.DataFeeds.Add(feed);

    mo.DataFeeds.Add(feed2);

    mo.DataFeeds.Add(feed3);

    var entry = new SDataEntry("my entry");

    feed.DataEntries.Add(entry);

}

The only syntactic differences in the code above are the S-prefixes on the various SLiveItems, the absence of ref parameters, and there are no explicit calls to loe.Mesh.MeshObjects.Add() for the SMeshObject, although that extra syntactic hoop could easily be enabled.

Yes, that code results in just one round-trip to the server.  “You’re kidding,” you say?  “Where’s the man behind the curtain?”  Let’s see that again, annotated with comments.

using (new ScriptContext(loe))

{

    // CreateResourceStatements

    var mo = new SMeshObject("original title");

    var feed = new SDataFeed("first feed");

    var feed2 = new SDataFeed("second feed");

    var feed3 = new SDataFeed("third feed");

    // set properties at runtime using ExpressionBindings

    mo.Resource.Title = "script-generated title";

    mo.Resource.Type = "LiveMeshFolder";

    feed.Resource.Type = "LiveMeshFiles";

    // bind sFeed's Response.Title to sFeed2's Request.Title

    feed2.Resource.Title = feed.Resource.Title;

    // bind sFeed's Response to sFeed3's Request

    feed3.Resource = feed.Resource;

    // bind DataFeedsLinks to CollectionUrls

    mo.DataFeeds.Add(feed);

    mo.DataFeeds.Add(feed2);

    mo.DataFeeds.Add(feed3);

    // CreateResourceStatement

    var entry = new SDataEntry("my entry");

    // bind DataEntriesLink to CollectionUrl

    feed.DataEntries.Add(entry);

} // automatically run the script

In ORM terms, the ScriptContext now functions as a UnitOfWork that automatically saves all items created or modified within its scope.  You can think of it as a script recorder that replays what it has recorded when it’s done.  In addition to the LOE parameter, ScriptContext also takes an optional RunLocality parameter that determines whether the script is executed by the client or by the server.  It defaults to running at the server.

The CreateResourceStatements were surprisingly straight-forward to implement.  So were the Add() methods.

The property getters and setters required a bit more magic.  String-based property getters all return a magic string that specifies the binding.  String-based property setters generate a PropertyBinding if they are passed a magic string, otherwise they generate a constant ExpressionBinding with the string they are given.  SResource-based setters generate a PropertyBinding.  The magic string approach could also be used for Uri-based properties, although I didn’t implement those in this prototype.

At first I generated script statements in the order that the SLiveItems were created, but later I added dependency-tracking so that property assignments and Add() calls re-order the statements if necessary.

Another advantage of late SLiveItem statement execution is that you won’t run into NullReferenceExceptions if you Add() SLiveItems to other SLiveItems in the “wrong” order.  That forum thread was actually part of the inspiration for imitating LiveItem syntax.

In that same forum thread I also detailed how LiveItems lead a dual life as a request and as a response.  A similar duality exists with SLiveItems.  When assigning properties, the object on the left-hand-side sets properties on its request, while the object on the right-hand-side gets properties on its response.

Each SLiveItem has a Result property that returns a “live” LiveItem once the script has been run by explicitly calling ScriptContext.Current.CreateScript().Run().  This uses the SaveResult() technique described earlier.

Each SLiveItem exposes a WrappedStatement property that lets you make changes to the underlying Statement if you need that escape hatch into script-land to touch things like the request resource.

Using SLiveItem syntax with triggers

With these tools in our bag, we are now ready to make triggers more accessible.

First, I created a new type, SResourceParameter<TResource> and a factory method, S.ResourceParameter<TResource>().  This enables a more strongly-typed Bind() call than what I had in V1.

Then I updated the Bind() methods to tell the ScriptContext about any ParameterBindings so they can be automatically added to the auto-generated root script statement.

Finally, I decided to try out yet another syntax for Bind() that looks like Set().EqualTo().  It exists as a method on SLiveItem instead of as an extension method for Statements and I only implemented it for Resource ParameterBindings, but it could just as easily be applied to all of the other binding types.

This lets us write:

using (new ScriptContext(loe))

{

    var originalObject = new MeshObject("Original object");

    var triggerParam = S.ResourceParameter<MeshObjectResource>();

    var triggerCreatedObject = new SMeshObject("trigger-created object");

    triggerCreatedObject.Set(s => s.Request.Title).EqualTo(triggerParam, p => p.Title);

    originalObject.Resource.Triggers.PostCreateTrigger = ScriptContext.Current.CreateScript();

    loe.Mesh.MeshObjects.Add(ref originalObject);

}

The Set().EqualTo() syntax isn’t as nice as the LiveItem property getter/setter syntax, but Set().EqualTo() was quick to write and I didn’t feel like taking more time to write the wrapper that would enable plain old property support.  I’m sure it could be done.

What remains to be done?

The LiveItem syntax turned out to be quite a bit more work than I was expecting, and it’s still nowhere near being done.  In fact it may have some major flaws that necessitate a rewrite, I’m not quite sure yet.  One major issue is that it currently only supports CreateResourceStatements.  Adding support for other statement types could have a huge ripple effect.  The use of magic strings is another design decision that may cause headaches down the road, but so far I’m getting away with it, and the alternatives aren’t nearly as nice to use.

I started out without any generics in the SLiveItem and SResource base classes but later introduced the rather insane explosion of generics to consolidate repetitive functionality from the derived classes.  It may be desirable to first undo the base class generics to create some mental breathing room and then add more functionality.

SLiveItem wrappers need to be created for Contact, Mapping, Member, Device, News, and Profile.  This may require creating additional wrapper classes for the helper classes they depend on such as NewsItemContext.

As mentioned in the previous section, Uri properties need magic string support, and ParameterBindings need LiveItem-style property assignment support.

Once the ResourceQuery bug is fixed, WithQuery should use it instead of LiveQuery.  If the runtime query feature is implemented, this should also be supported.

SLiveItem needs to implement Update() and CreateQuery().

The current implementation freely reorders statements.  It also assumes that when you’re doing property assignments, everything on the left-hand-side is a request and everything on the right-hand-side is a response.  If you set the same property more than once, only the last set sticks, and getting the same property at different points always returns the same value.  This may result in unexpected behavior if the user is depending on the same property having different values at different points in the script.  This could be dealt with by generating more than one statement per SLiveItem and/or by generating AssignStatements.

Per the comments in the code, my usage of ETags when generating LiveItems from script results may not be correct.  There are also a number of other TODOs in the code comments.

Last but not least, this needs unit tests.  More interfaces such as IScriptContext probably need to be created to enhance testability.

I’m sure there’s more I’m forgetting.  I told you, it’s a lot of work! :-)

Download

You can download V2 of the Fluent MeshScripts library here.  I have enhanced the console app with additional examples that demonstrate most of the new features in this post.

To run the sample you will need to change the username and password.  If the project references to Microsoft.LiveFX.Client.dll, Microsoft.LiveFX.ResourceModel.dll, and Microsoft.Web.dll are broken, you will need to remove and recreate them in both projects.

Wrap-up

I was going to discuss future directions for this library and for MeshScripts in general but this post has gone on for long enough so I’ll save that for my next post.

Hopefully my enhancements help you visualize more possibilities for MeshScripts.  At the very least, they should make MeshScripts much easier for you to write and work with.

As always, I’d love to hear any feedback.

Fluent LiveFX Resource Scripts

After writing my Resource Script demo post, I’ve been digging deeper into Live Framework Resource Scripts.   Along the way I’ve written a helper library to make them easier to work with.  My enhancements focus primarily on keeping your scripts strongly typed and enabling a more concise fluent interface syntax.  These enhancements let Intellisense help you out quite a bit more, resulting in greater discoverability and productivity.

I must warn you that the following discussion won’t make much sense unless you’re already somewhat familiar with Resource Scripts.  I apologize and promise to follow up in future posts with material that’s more suitable as an introduction, using my library of course. :-)

Strongly typed bindings

If you’ve played with Resource Scripts at all, you’ve almost certainly run into Bindings.  These creatures consume magic strings such as “EntryUrl”, “CollectionUrl”, “Request.Title”, “Response.SelfLink”, “Response.DataFeedsLink”, and “Response.DataEntriesLink” to name a few.  Yuck!  To make matters worse, the types of Request and Response are usually (but not always!) generic parameters to a statement, meaning that their available sub-properties will vary based on the generic type.  Also, some statement types don’t have Request, and others don’t have either Request or Response.  It would sure be nice if I don’t have to consult MSDN documentation or Reflector each time I write a binding statement.  Bindings are strongly-typed at runtime, so why not at design-time too?

Then this post popped up in Google Reader and reminded me that I can generate those icky dirty strings from nice shiny expression trees, just like LINQ to SQL generates SQL from strongly-typed C# statements.  So my Bindings can go from this:

Statement.CreateResource("feedStatement", null, dataFeed,

    Statement.Bind("CollectionUrl", 

        "folderStatement", "Response.DataFeedsLink"));

to this:

S.CreateResource(dataFeed)

    .Bind(df => df.CollectionUrl,

        folderStatement, fs => fs.Response.DataFeedsLink);

Due to the way I’ve defined the generic parameters on the lambda expressions, the types of the source property and the target property have to match.  If they don’t, you get immediate red squiggly feedback in Visual Studio.  I’m not sure if that’s a Visual Studio thing or a Resharper thing, but that’s how it works on my box.  At the very least you will find out at compile-time instead of at runtime.

Besides the lambda expressions and the ability to call Bind() on the statement after it has been created, it’s worth noting that the Statement.Name “folderStatement” string has been replaced with a reference to the source Statement itself.  No more remembering statement names (until we get to ConditionalStatements that is…).

Simpler statement construction

So what was that “S” thing in the previous example?  That’s my static utility class that offers methods equivalent to most of the static factory methods on the Statement class.  Methods in “S” have the same names, but they typically have fewer parameters, resulting in a more concise syntax when you’re data binding.  They also give you the option of using strings instead of Uri objects.

Yes I know, it’s not fair that my utility class gets the short, easy to type name while “Statement” makes you type twice as much before Intellisense kicks in and wastes a bunch of horizontal space.  So put “using S = Microsoft.LiveFX.ResourceModel.Scripting.Statement;” at the top of your code if that makes you feel better. :-)

You may also have noticed that I didn’t supply a name for the statement.  All of the factory methods in “S” automatically generate a random statement name so that every statement is inherently bindable.  If you need a well-known name for inspecting script results or for use in a ConditionalStatement, you can use the NameStatement() extension method:

S.CreateResource(dataFeed).NameStatement("myName")

NameStatement() also checks for valid statement names so you find out at design-time rather than at runtime that statements can’t start with a number and can’t contain spaces.

Functional statement construction

Once you start writing utility methods to generate groups of statements that you string together into a script, you quickly run into the issue that you’re always having to write little bits of shim code to repackage your statements into a single Statement[] before feeding them into your CompoundStatement of choice (Sequence, Interleave, or Conditional).  Wouldn’t it be nice if you could throw anything you wanted into a CompoundStatement and it would all be taken care of, similar to the XElement constructor in LINQ to XML?

If you’re not familiar with the XElement constructor, it looks like this:

public XElement(XName name, params object[] content)

It’s a bit loosey-goosey with the object[] parameter, but according to the documentation it allows you to pass in objects that are (or can be converted to) XML nodes, as well as IEnumerable<T> of such objects.  Null content is silently ignored.  Anything else results in an exception at runtime.

When this style is applied to CompoundStatement construction, it enables the following code:

S.Sequence(

    readMeshObjects,

    conditionallyCreateFolders,

    createAnotherFolder,

    createFiles)

That doesn’t look very interesting without the type declarations, but imagine the first two parameters are different types of Statements, the third parameter is a custom object that implements IEnumerable<Statement>, and the last parameter is a Statement array.  You can see this code in action in the IfElseSample in the download.

Binding to URLs and Requests

One of the most common uses of bindings is to perform CRUD operations on a URL that comes from the result of a previous statement.  Specifying the target URL in the binding can become quite repetitive.  The property name for the target URL also varies by statement type.  Sometimes it’s EntryUrl, sometimes it’s CollectionUrl, and sometimes it’s MediaResourceUrl.

I address this with the AtUrl() extension method which eliminates the need to specify the target property and lets you write:

S.CreateResource(dataFeed)

    .AtUrl(folderStatement, fs => fs.Response.DataFeedsLink);

A similar WithRequest() extension method exists for binding to the Request property on CreateResource, UpdateResource, and SynchronizeResourceCollection.

Conditional statements

The ConditionalStatement is worth an entire blog post.  Until then, here’s an example of the syntax I’ve enabled:

S.If(statement =>

     (((ReadResourceCollectionStatement<MeshObjectResource>)

       statement.FindStatement("ReadObjects")).Response.Entries

       .Where(mo => mo.Title == "My Folder").Count() == 0))

    .Then(ScriptHelper.CreateFolder("Folder didn't exist"))

    .Else(ScriptHelper.CreateFolder("Folder DID exist")

I should note that ConditionalStatement already exposed the ability to use lambda expressions.  All I did was enable the If().Then().Else() syntax.  The Else() is optional.

Strongly typed statement groups

Notice the CreateFolder() helper method in the previous example?  Originally this method returned a Statement[] containing two statements.  The first statement created the MeshObject that represents the folder and the second statement created a DataFeed at the DataFeedsLink of the folder.  This Statement[] was sufficient for creating a folder with a given name, but if I wanted to do something interesting with it such as put files in the folder or use a binding to change its title, it quickly became a pain to grab the appropriate entry from the array and cast it to the correct type.

So I created a helper class named CreateFolderStatementPair that exposes strongly typed properties named FolderStatement and FilesFeedStatement.  This lets you write:

S.CreateResource<DataEntryResource>()

    .AtUrl(folder.FilesFeedStatement, f => f.Response.DataEntriesLink)

and

folder.FolderStatement.Bind(mo => mo.Request.Title, "new title");

CreateFolderStatementPair inherits from an abstract class named StatementGroup which implements IEnumerable<Statement> and also has an implicit operator conversion to Statement[].  Implementing IEnumerable<Statement> means you can pass a StatementGroup into S.Sequence(), S.Interleave(), and the Then()/Else() methods.  The implicit conversion to Statement[] means you can pass a StatementGroup into methods that expect a Statement[] such as the original Statement.Sequence() method.  You can use StatementGroup to create your own strongly typed group of statements that play well with bindings and with the S.*/Statement.* factory methods.

Miscellaneous helpers

Besides CreateFolder(), The ScriptHelper static class has a few other useful methods.  CreateMedia() takes a CreateResourceStatement<DataFeedResource> and an external media URL and does a CreateMedia at the MediaResourcesLink of the CreateResourceStatement.  There is also a CreateMedias() method that takes multiple media URLs.  ScriptHelper has a few other convenience properties and methods, but nothing noteworthy.

There are a few other extension methods I haven’t mentioned yet.

ToSequence() and ToInterleave() turn a Statement array into a SequenceStatement or an InterleaveStatement respectively.

AddBindings() and AddParameters() let you add bindings and parameters to statements after they have been created.

FindStatement<TStatement>() recursively finds the first statement of the specified type in an IEnumerable<Statement>.

A Compile() method has been added to all statement types, not just CompoundStatements.  Run() and RunAtServer() with an implicit Compile() have also been added to any Statement type.

Run() and RunAtServer() no longer require any parameters if you first call ScriptHelper.SetCredential(username, password).

Download

You can download the code here.  The solution contains a console app that demonstrates some of the features with a few sample scripts.

To run the sample you will of course need to change the username and password.  If the project references to Microsoft.LiveFX.Client.dll, Microsoft.LiveFX.ResourceModel.dll, and Microsoft.Web.dll are broken, you will need to remove and recreate them in both projects.

Conclusion

If you’re wondering why Resource Scripts didn’t have these features already, you need to remember that Resource Scripts were designed to be written using a visual designer tool similar to the Windows Workflow designer.  The team was also under intense pressure to make the CTP available in time for PDC.

Think of this library as an experiment to see what a more code-centric API might look like and whether it could co-exist with a visual designer tool.  Who knows, maybe some of the concepts such as strongly-typed StatementGroups might find their way into such a visual designer.

Hopefully this library enables and encourages more people to play with Resource Scripts.  If you have any feedback, I’d love to hear it.  Have fun scripting your Mesh!

L1v3 M35H L337 H4x0rZ

In case you aren’t already persuaded that the Live Mesh team are a bunch of L337 H4x0rZ, check out the ids of entries in the Profiles feed.

id	title
G3N3RaL	GeneralProfile
480u7Y0U	AboutYouProfile
k0n74C7Inf0	ContactProfile
wORK1nfo	WorkProfile
1n7eRE572	InterestsProfile

 

I believe this also demonstrates their far-reaching commitment to open web standards and the new generation of social apps.  Or perhaps the hidden message is “so easy, even script kiddies can hack it!”

You can see this for yourself by firing up the Live Framework Resource Browser (LivefxResourceBrowser.exe from the SDK tools) and drilling down into Cloud LOE > Profiles.

On a slightly related note, as I was digging around with the Resource Browser I discovered that the following two URL styles appear to be interchangeable.

https://user-ctp.windows.net/V0.1/cid-1234567890123456789/Profiles

https://user-ctp.windows.net/V0.1/email-abc@live.com/Profiles

I’m not sure if the email identifier format is stable enough to bank on, but it’s convenient for typing or tweaking URLs by hand.  Does anyone know of other equivalent identifier types in Mesh resources?

Live Mesh Resource Script Demo

In the Live Framework Programming Model Architecture and Insights session, Ori Amiga (standing in for Dharma Shukla, previously a WF architect) demos a Live Mesh resource script that runs in the cloud.  The script creates a folder on the Live Mesh desktop and downloads two images from external resources, placing them in the newly created folder.

I couldn’t find this sample on the web, so I recreated it from the session video.  You can download the demo project here.

You may need to touch up the references to Microsoft.LiveFX.Client.dll, Microsoft.LiveFX.ResourceModel.dll, and Microsoft.Web.dll since they live under C:\Program Files (x86)\ on my 64-bit box and are probably under C:\Program Files\ if you’re running 32-bit.

At first, my demo threw an error trying to run the following line:

script.RunAtServer(creds);

After investigating with Reflector, I discovered that RunAtServer() hard-codes a default script URL of https://user.windows.net/V0.1/Script/ which needs to be changed to https://user-ctp.windows.net/V0.1/Script/ .  You can override this either by calling an overload of RunAtServer() that takes a URI, or by creating an App.config file and adding the following line in the <appSettings> section.

<add key="ScriptUrl" value="https://user-ctp.windows.net/V0.1/Script/"/>

I chose to use the appSettings solution since that is what Ori must have used in the demo.

I really would prefer ScriptUrl to be exposed as a property on ResourceScript<> that appSettings/ScriptUrl maps into rather than having to specify the URL either in config or in each method call.  My philosophy is that you should always be able to do in code what you can do in config.

I’m looking forward to playing more with Live Mesh resource scripts, documented here.  Right now they feel a bit convoluted to create programmatically, but they appear to be designed to put a friendlier layer on top such as an Oslo DSL or a “resource workflow designer”.

Dissecting Live Mesh App Packages

After bundling Flash inside a Mesh app, I took a closer look at what Visual Studio is doing behind the scenes.  The Mesh-enabled Web Application template creates a project with a .meshproj extension.  A .meshproj file has several important properties.  <OutputName> is the prefix used to name the resulting zip file.  <DebuggerCodeType> is set to either JavaScript or Silverlight, depending on whether you create an HTML/JavaScript or a Silverlight Mesh app.  <ApplicationUri> is the Application Self-Link that you are supposed to copy-and-paste from the Developer Portal after you upload the zip file, as instructed below:

A .meshproj file also imports $(MSBuildExtensionsPath)\Microsoft\Live Framework\v1.0\Microsoft.LiveFramework.targets which first ensures that your project has an index.html and then zips up the output directory, naming the zip file using the OutputName you specified.

You don’t need to use Visual Studio to do this.  You can easily create your own Mesh app package by hand.  At a minimum, your zip file must contain:

• index.html
• Manifest.xml
• Logo.png

Index.html is the entry point for your app.  Logo.png is the icon that will be displayed on your desktop and should be a 32-bit 256 x 256 png.  Manifest.xml is your app manifest. A detailed description of the manifest configuration options is documented here.

I believe you can bundle anything you want in the zip file, although Microsoft supposedly runs an antivirus scan on the contents, and there may be additional checks for inappropriate content.  Anything in the zip file gets downloaded to your computer when the app is installed on your local desktop.  This is why my Flash app was able to run offline.

You might be able to use this to “install” an XBAP application that can run offline.  To make it cross-platform, you could bundle the XBAP together with a “down-level” Silverlight version and choose which one to display based on what the client supports.  If download size is a concern, it might be possible to put the executables in a DataFeed instead of in the app zip file and selectively sync only the version you want to display, but I haven’t dug into DataFeeds enough yet to see if this kind of per-client sync filtering is possible.  Of course you would be working against the built-in versioning management if you did this (updates should only occur when the user closes and re-opens the app).

Ok, so uploading a zip file sounds nice and simple, right?  Then why does Visual Studio want me to copy-and-paste the Application Self-Link URI?  It turns out that if you use Visual Studio you only upload the zip file once per app.  Once you’ve uploaded the zip and told Visual Studio about the Self-Link URI, Visual Studio will use that URI for subsequent deployments to upload the individual files directly.

If you watch Visual Studio using Fiddler (you’ll need to configure HTTPS support) you will see it query the Mesh for the resource feeds of your app, do HTTP DELETEs for each resource that was inside your zip file, and then do a bunch of POSTs to upload each item in your project.  That seems a bit risky.  What if Visual Studio dies before reposting all the resources it deleted?  It seems like updating an app by manually uploading a zip file is a safer, slightly more atomic operation.  It’s no big deal right now, but once real production apps are being upgraded, something more robust would be nice.  I’m guessing we will see more explicit versioning, giving the user the choice of whether or not to upgrade.  If such a feature is added, the direct app resource update trick might be useful for bypassing an explicit upgrade prompt.

The next time your Live Mesh client (MOE.exe) talks to the cloud, it will download the new versions of the files into your local app cache (AppData\Local\Microsoft\Live Framework Client\Bin\Moe2\MR\).  For some reason I was unable to pinpoint the download traffic with Fiddler, so I can’t say for certain whether individual files are downloaded or if they are zipped up first.  It appears older versions of files aren’t removed.  This is probably to support the explicit user upgrade scenario in the future, but it seems like they could still be doing more cleanup.

I’m really curious why Visual Studio updates individual app resources rather than following the documented workflow of uploading a zip file with the updates.  Anyone know?

Update: I posed this question in the comments on Danny Thorpe’s blog and he responded:

On your second question, the reason we upload files individually instead of uploading the zip file is because the REST API we’re uploading to doesn’t handle zip files. The dev portal that you manually upload your zip file to unzips the file and uploads the individual bits to the production storage. The Live Services REST APIs that the VS tools use to upload files goes (as far as I know) straight into the production storage.

In a nutshell, the dev portal that you see in your web browser is just a front end to the actual cloud service. VS doesn’t upload to the dev portal UI, it uploads to the cloud itself.

Keep in mind that the long side trip of manual steps that you currently have to go through to get a new app created and uploaded to the cloud will all be going away as soon as the cloud APIs to create and provision a new application are implemented.

He also explains the debugging versioning scheme in the comments, and I suggest you go read it for more great details.

One other related insight from Danny comes from this forum thread:

Our goal for the VS tools is to do all development against the local LOE and let the local LOE deal with sync'ing things back to the cloud.  All the parts needed to do that aren't ready yet, so for the PDC CTP we redirected the VS tools to upload and debug mesh apps in the cloud.

This makes the current chattiness (and the “glue” dialog box) much more acceptable to me since the end goal is to use the local REST API rather than the cloud API.

Update 2: Danny has posted a thorough response to this post.  There’s lots of great information there, so I won’t quote it all here.  One “aha” moment for me was the concept of separate debug application resources.  He also confirms that the “glue” dialog will be going away soon.  Go read it for details.  Thanks, Danny!

Live Mesh + Flash == Adobe AIR

Yes, that’s an Adobe Flash app running as a Live Mesh app, and it was easy.  Feel free to install the app and try it out for yourself.

First I snagged a pre-existing .swf file since I am not a Flash developer.  Then I created a new Visual Studio project using the Mesh-enabled Web Application template that comes with the Live Framework Tools for Visual Studio.  I added the .swf file to the project with the default build action of Content and copied-and-pasted the object embed tag into the body of index.html.  Then I ran through the usual Ctrl-F5 steps to upload and deploy the resulting zip package and boom, it just worked!  I was able to use the app in the browser in my Live Desktop, and an icon for the app magically appeared on my Windows desktop that let me run the app offline, “outside the browser” (I believe MeshAppHost.exe actually hosts a chromeless IE browser control).  I’m guessing you would also get the same desktop experience using the Mac Tech Preview.

If I were an actual Flash developer, I would take it to the next step and call the Mesh APIs using the Microsoft.LiveFramework.js library.  That should “just work” as ActionScript, right?

If Flash for Windows Mobile appears before Silverlight for Windows Mobile, this could make for a very interesting deployment option when combined with the Live Mesh client for Windows Mobile once it supports Mesh apps.  Three days ago Amit Chopra announced that a public CTP of Silverlight 2 for Mobile will be available in Q1 of 2009.  I’m guessing this will coincide with MIX09 which starts March 18.

The skeptics might say hosting Flash in a Live Mesh app is an unsupported hack that Microsoft will quickly disable, but I don’t think so.  David Chappell’s whitepaper Introducing the Azure Services Platform specifically states:

“A mesh-enabled Web application must be implemented using a multi-platform technology, such as Microsoft Silverlight, DHTML, or Adobe Flash. These technologies are supported on all of the operating systems that can run the Live Framework: Windows Vista/XP, Macintosh OS X, and Windows Mobile 6.”

I think this is a very cool option that highlights the fact that Microsoft designed Mesh to be an open platform with the broadest possible reach.

Update:

A whitepaper just published by a Program Manager and an Architect on the Live Framework team contains the following quote that confirms Flash support:

What application types are supported by the Live Framework?

The Live Framework supports client side applications of all types including the following application
types on Windows to interact with Client or Cloud versions of Live Operating Environment:

1. Browser based apps (Javascript, Flash and Silverlight) on IE, Firefox and Safari
2. Managed desktop applications written using WPF, WinForms, or other languages like Python,
   Ruby, or Perl. All you need is an HTTP Client stack in your programming environment of choice.
3. Traditional native Win32 applications (all you need is WinInet/IXmlHttpRequest and MSXML)

Additionally, on the server side, you can use PHP, WCF, ASP.Net or any other server-side language or
technology to interact with the cloud version of the Live Operating Environment.