MeshScript Queries, LiveItems, and Magic

I’ve continued to extend my Fluent Resource Scripts library in several interesting new ways.  I’ve added:

• Strongly typed LINQ queries
• Turning script results into LiveItems (MeshObject, DataFeed, DataEntry, etc.)
• LiveItem syntax for scripts

I’ve had to resort to more significant hacks to implement these, and I don’t feel that these features are as solid or as complete as the features in my original Fluent MeshScripts post.  But I think these ones are far more interesting, so I hope you will look past the rough edges and imagine the potential if these features were done properly.  The LiveItem syntax for scripts is especially cool, if I do say so myself.

Queries

Both LiveQuery and ResourceQuery let you generate query strings from strongly-typed LINQ queries.  ResourceQuery is broken, which is unfortunate because its only generic parameter is of type Resource, which also happens to be the only generic parameter for most script statements.  This would have allowed us to implicitly pass along the statement’s generic parameter to our helper method without having to write any generic angle brackets.

So we have to use LiveQuery instead, which takes a generic parameter of type LiveItem (the non-generic LiveItem, not LiveItem<TResource>).  This makes it so that instead of calling my helper method like this:

S.ReadResourceCollection<MeshObjectResource>(ScriptHelper.MeshObjectsUrl)

    .WithQuery(
   q => q.Where(o => o.Resource.Title.StartsWith("my")))

You must instead call it like this, specifying both the Resource type and the LiveItem type:

S.ReadResourceCollection<MeshObjectResource>(ScriptHelper.MeshObjectsUrl)

    .WithQuery<MeshObjectResource, MeshObject>(
   q => q.Where(o => o.Resource.Title.StartsWith("my")))

Oh well, it’s still useful, and once ResourceQuery is fixed we can switch to the shorter syntax.

I should note that these queries are immediately turned into query strings under the hood.  In other words, they are evaluated at script design-time, not at script runtime.  I have put in a feature request to support query generation at runtime.

Turning script results into LiveItems

So you’ve written a resource script, you run it, and you get some results back.  Then you think, “I’d like to do further work with the results I’ve gotten back.”  You dig into the individual statements in the script’s Result property, cast them to the appropriate statement type so you can then dig into the Resource property and examine the actual data that was returned.

But what if instead of working with results MeshScript-style, you wanted to work with them LiveItem-style, without having to do a bunch of tedious digging and casting?

DataFeed outFeed = null;

using (new ScriptContext(loe))

{

    S.Sequence(

        ..., // create moStatement

        S.CreateResource(scriptFeed)

            .AtUrl(moStatement, mo => mo.Response.DataFeedsLink)

            .SaveResult(ref outFeed)

    ).Compile().Run();

}

DataEntry de = new DataEntry("new entry");

outFeed.DataEntries.Add(ref de);

I’m not quite sure what programming idiom to compare SaveResult() to since the result isn’t actually saved until the script is run.  The closest thing I can think of is a future.

SaveResult() creates a new LiveItem of the appropriate type, and after the script is run, this LiveItem is filled in with all of the necessary response information to give you a “live” LiveItem that you can start partying on right away without requiring another server round-trip to flesh it out.  This required a fair bit of reflection magic because by default LiveItems are essentially “DeadItems” until they’re associated with a LOE and Reloaded.

Notice I’ve added the use of a ScriptContext to enable the generated LiveItems to be automatically associated with an existing LiveOperatingEnvironment.

SaveResult() has been implemented for CreateResourceStatement, ReadResourceStatement, and ReadResourceCollectionStatement.  I have also added Statement extension methods ToMeshObject(), ToDataFeed(), and ToDataEntry() if you prefer to work that way.

When you combine ReadResourceCollection with SaveResult() and the query support above, you can now batch multiple LiveItem queries.

LiveItem syntax for MeshScripts

So we’re starting to get a decent bridge between the world of MeshScripts and the world of LiveItems.  But man, that MeshScript syntax is still pretty nasty, even with the fluent stuff I’ve added.  Wouldn’t it be nice if you could write a MeshScript the same way you write LiveItem code?

using (new ScriptContext(loe))

{

    var mo = new SMeshObject("original title");

    var feed = new SDataFeed("first feed");

    var feed2 = new SDataFeed("second feed");

    var feed3 = new SDataFeed("third feed");

    mo.Resource.Title = "script-generated title";

    mo.Resource.Type = "LiveMeshFolder";

    feed.Resource.Type = "LiveMeshFiles";

    feed2.Resource.Title = feed.Resource.Title;

    feed3.Resource = feed.Resource;

    mo.DataFeeds.Add(feed);

    mo.DataFeeds.Add(feed2);

    mo.DataFeeds.Add(feed3);

    var entry = new SDataEntry("my entry");

    feed.DataEntries.Add(entry);

}

The only syntactic differences in the code above are the S-prefixes on the various SLiveItems, the absence of ref parameters, and there are no explicit calls to loe.Mesh.MeshObjects.Add() for the SMeshObject, although that extra syntactic hoop could easily be enabled.

Yes, that code results in just one round-trip to the server.  “You’re kidding,” you say?  “Where’s the man behind the curtain?”  Let’s see that again, annotated with comments.

using (new ScriptContext(loe))

{

    // CreateResourceStatements

    var mo = new SMeshObject("original title");

    var feed = new SDataFeed("first feed");

    var feed2 = new SDataFeed("second feed");

    var feed3 = new SDataFeed("third feed");

    // set properties at runtime using ExpressionBindings

    mo.Resource.Title = "script-generated title";

    mo.Resource.Type = "LiveMeshFolder";

    feed.Resource.Type = "LiveMeshFiles";

    // bind sFeed's Response.Title to sFeed2's Request.Title

    feed2.Resource.Title = feed.Resource.Title;

    // bind sFeed's Response to sFeed3's Request

    feed3.Resource = feed.Resource;

    // bind DataFeedsLinks to CollectionUrls

    mo.DataFeeds.Add(feed);

    mo.DataFeeds.Add(feed2);

    mo.DataFeeds.Add(feed3);

    // CreateResourceStatement

    var entry = new SDataEntry("my entry");

    // bind DataEntriesLink to CollectionUrl

    feed.DataEntries.Add(entry);

} // automatically run the script

In ORM terms, the ScriptContext now functions as a UnitOfWork that automatically saves all items created or modified within its scope.  You can think of it as a script recorder that replays what it has recorded when it’s done.  In addition to the LOE parameter, ScriptContext also takes an optional RunLocality parameter that determines whether the script is executed by the client or by the server.  It defaults to running at the server.

The CreateResourceStatements were surprisingly straight-forward to implement.  So were the Add() methods.

The property getters and setters required a bit more magic.  String-based property getters all return a magic string that specifies the binding.  String-based property setters generate a PropertyBinding if they are passed a magic string, otherwise they generate a constant ExpressionBinding with the string they are given.  SResource-based setters generate a PropertyBinding.  The magic string approach could also be used for Uri-based properties, although I didn’t implement those in this prototype.

At first I generated script statements in the order that the SLiveItems were created, but later I added dependency-tracking so that property assignments and Add() calls re-order the statements if necessary.

Another advantage of late SLiveItem statement execution is that you won’t run into NullReferenceExceptions if you Add() SLiveItems to other SLiveItems in the “wrong” order.  That forum thread was actually part of the inspiration for imitating LiveItem syntax.

In that same forum thread I also detailed how LiveItems lead a dual life as a request and as a response.  A similar duality exists with SLiveItems.  When assigning properties, the object on the left-hand-side sets properties on its request, while the object on the right-hand-side gets properties on its response.

Each SLiveItem has a Result property that returns a “live” LiveItem once the script has been run by explicitly calling ScriptContext.Current.CreateScript().Run().  This uses the SaveResult() technique described earlier.

Each SLiveItem exposes a WrappedStatement property that lets you make changes to the underlying Statement if you need that escape hatch into script-land to touch things like the request resource.

Using SLiveItem syntax with triggers

With these tools in our bag, we are now ready to make triggers more accessible.

First, I created a new type, SResourceParameter<TResource> and a factory method, S.ResourceParameter<TResource>().  This enables a more strongly-typed Bind() call than what I had in V1.

Then I updated the Bind() methods to tell the ScriptContext about any ParameterBindings so they can be automatically added to the auto-generated root script statement.

Finally, I decided to try out yet another syntax for Bind() that looks like Set().EqualTo().  It exists as a method on SLiveItem instead of as an extension method for Statements and I only implemented it for Resource ParameterBindings, but it could just as easily be applied to all of the other binding types.

This lets us write:

using (new ScriptContext(loe))

{

    var originalObject = new MeshObject("Original object");

    var triggerParam = S.ResourceParameter<MeshObjectResource>();

    var triggerCreatedObject = new SMeshObject("trigger-created object");

    triggerCreatedObject.Set(s => s.Request.Title).EqualTo(triggerParam, p => p.Title);

    originalObject.Resource.Triggers.PostCreateTrigger = ScriptContext.Current.CreateScript();

    loe.Mesh.MeshObjects.Add(ref originalObject);

}

The Set().EqualTo() syntax isn’t as nice as the LiveItem property getter/setter syntax, but Set().EqualTo() was quick to write and I didn’t feel like taking more time to write the wrapper that would enable plain old property support.  I’m sure it could be done.

What remains to be done?

The LiveItem syntax turned out to be quite a bit more work than I was expecting, and it’s still nowhere near being done.  In fact it may have some major flaws that necessitate a rewrite, I’m not quite sure yet.  One major issue is that it currently only supports CreateResourceStatements.  Adding support for other statement types could have a huge ripple effect.  The use of magic strings is another design decision that may cause headaches down the road, but so far I’m getting away with it, and the alternatives aren’t nearly as nice to use.

I started out without any generics in the SLiveItem and SResource base classes but later introduced the rather insane explosion of generics to consolidate repetitive functionality from the derived classes.  It may be desirable to first undo the base class generics to create some mental breathing room and then add more functionality.

SLiveItem wrappers need to be created for Contact, Mapping, Member, Device, News, and Profile.  This may require creating additional wrapper classes for the helper classes they depend on such as NewsItemContext.

As mentioned in the previous section, Uri properties need magic string support, and ParameterBindings need LiveItem-style property assignment support.

Once the ResourceQuery bug is fixed, WithQuery should use it instead of LiveQuery.  If the runtime query feature is implemented, this should also be supported.

SLiveItem needs to implement Update() and CreateQuery().

The current implementation freely reorders statements.  It also assumes that when you’re doing property assignments, everything on the left-hand-side is a request and everything on the right-hand-side is a response.  If you set the same property more than once, only the last set sticks, and getting the same property at different points always returns the same value.  This may result in unexpected behavior if the user is depending on the same property having different values at different points in the script.  This could be dealt with by generating more than one statement per SLiveItem and/or by generating AssignStatements.

Per the comments in the code, my usage of ETags when generating LiveItems from script results may not be correct.  There are also a number of other TODOs in the code comments.

Last but not least, this needs unit tests.  More interfaces such as IScriptContext probably need to be created to enhance testability.

I’m sure there’s more I’m forgetting.  I told you, it’s a lot of work! :-)

Download

You can download V2 of the Fluent MeshScripts library here.  I have enhanced the console app with additional examples that demonstrate most of the new features in this post.

To run the sample you will need to change the username and password.  If the project references to Microsoft.LiveFX.Client.dll, Microsoft.LiveFX.ResourceModel.dll, and Microsoft.Web.dll are broken, you will need to remove and recreate them in both projects.

Wrap-up

I was going to discuss future directions for this library and for MeshScripts in general but this post has gone on for long enough so I’ll save that for my next post.

Hopefully my enhancements help you visualize more possibilities for MeshScripts.  At the very least, they should make MeshScripts much easier for you to write and work with.

As always, I’d love to hear any feedback.