Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts

Thursday, January 31, 2013

How not to write a subclass

As a programmer coming to JavaScript from the Java world (arguably a bad influence) and the ActionScript world (undoubtedly an even worse one), learning how JavaScript works has been a gradual and error-prone process. I've recently been working on a Coding conventions page on the Orion wiki, and a big chunk of that page is devoted to avoiding the kind of common JavaScript traps that I'm always falling into.

Today I wrote a section about creating classes, taking care to point out a mistake that I'd made dozens of times in the past. This post is adapted from there, so go read the original writeup if you're not interested in the extra verbiage.

First, here's how we tend to create classes in Orion's JavaScript code. I won't claim this is the One True Way, just an easy and straightforward convention that we've settled on.

function Duck(name) {
    this.name = name;
}
Duck.prototype.greet = function() {
    console.log("Quack quack, I'm a duck named " + this.name);
};

What new really does

Let's look at this piece of code, which is familiar to any JavaScript programmer:
new Duck("Robert");

It obviously creates a new instance of Duck. But what is the new operator actually doing here? Well, it performs an algorithm that we can break into 4 distinct steps:

  1. Create a brand-new object (call it O).
  2. Set O's prototype equal to Duck.prototype.
  3. Invoke the Duck function with this equal to O, and the name parameter equal to "Robert".
  4. Return O.

Subclasses

A problem arises when we want to extend an existing class with new behavior. How do we create a prototype for the subclass such that it extends the superclass's prototype?

Here's the wrong way:

function SeaDuck(name, diveDepth) {
    Duck.call(this, name);      // call the super constructor
    this.diveDepth = diveDepth;
}
SeaDuck.prototype = new Duck(); // XXX wrong
SeaDuck.prototype.dive = function() {
    console.log(this.name + " dived to a depth of " + this.diveDepth);
};

Everything here is reasonable except the XXX'd line, SeaDuck.prototype = new Duck(). This part is wrong. (Unfortunately, many occurrences of this pattern remain in the Orion source code, lots of them written by me, which still need to be cleaned up.) So what's wrong with it?

  • It's inefficient, since SeaDuck.prototype has fields created by Duck that are never used (like name, for example). Any work done in the Duck constructor is useless to us here.
  • It's fragile, since the proper operation of this code relies on Duck not validating its input parameters. If anyone ever changes Duck to assert that it receives a valid name, our SeaDuck code will blow up. (And sure: we could pass in a fake name to satisfy it, but that clutters up our code with even more useless data).

For our SeaDuck.prototype, what we really want is not to call the Duck constructor, but just to create a new object whose prototype is set to Duck.prototype. In other words, we only want steps #1, #2, and #4 of the new algorithm, not #3.

Object.create

For a long time JavaScript provided no way to decouple object creation and prototype-setting from initialization. ECMAScript 5 finally fixed this problem by introducing Object.create. Among other things, Object.create allows you to build a new object and tell the JS engine exactly what its prototype should be. Just pass the desired prototype object as the first argument. Easy!
SeaDuck.protoype = Object.create(Duck.prototype);

And that's exactly what we wanted. In fact, we can use Object.create to replace even legitimate uses of the new operator. Instead of this:

var robert = new Duck("Robert");

…We can write this, effectively re-implementing the new algorithm by hand:

var robert = Object.create(Duck.prototype);
Duck.call(robert, "Robert");

While this might be conceptually clearer, it's wildly verbose, so I'd recommend sticking with new in these cases.

But what if you're coding for a crap browser like Internet Explorer 8, which doesn't support Object.create? Then you need to write your own utility, typically called "beget":

function beget(obj) {
  function BogusConstructor() {}
  BogusConstructor.prototype = obj;
  return new BogusConstructor();
}

SeaDuck.prototype = beget(Duck.prototype);

beget avoids the problem I pointed out in the previous section by creating a new BogusConstructor every time it's called, which does no initialization work and only exists to achieve point #3 of the new algorithm.

You could also turn beget into a partial shim for Object.create. I say partial because Object.create also deals with property descriptors, which are impossible to shim.

The prototype, "prototype", [[Prototype]], and __proto__ mess

Looking back, a big source of my confusion as a learner was in understanding how the prototype property of functions relates to an "object's prototype" and how that in turn affects property lookup. You can easily see that regular objects don't have a prototype property: try evaluating ({ }).prototype in your debugger. So what's this "prototype" thing everyone keeps talking about on objects? Well, here's my attempt to clarify things, in point form:
  1. When we say "an object's prototype", we're referring to an internal property of the object, which the ECMAScript spec calls [[Prototype]]. Being an internal property, [[Property]] is not observable from regular ECMAScript code.
  2. An object's [[Prototype]] is consulted to resolve property names when the dot . and array index [] operators are used on the object. If the desired property name is not found in [[Prototype]], then the [[Prototype]]'s [[Prototype]] is consulted, and so on. When people talk about the prototype chain, this is what they mean.
  3. The prototype property can be set on a function. When some function F is invoked as a constructor through the new operator, the value of F.prototype becomes the [[Prototype]] property of the newly-constructed object.
  4. To create a new object whose [[Prototype]] is some existing object P, use Object.create(P).
  5. I lied a bit in point (i). Most JavaScript engines provide a non-standard alias for the internal [[Prototype]] property. It's called __proto__. (ES5 has defined a standardized Object.getPrototypeOf, so use that instead of __proto__!)

While you can't rely on __proto__ in production code, it's great for debugging, and for fixing your mental model of how the JS engine works. Here's what a SeaDuck's __proto__ looks like in the JS console:

Note how you can expand the __proto__ chain all the way up to Object.prototype.

Thursday, April 5, 2012

Connecting Amazon S3 to Orion

Amazon S3 is a popular cloud file hosting service. For months people have been complaining that there was no way to store your Orion work — that is to say, your files and folders — in S3 so you'd have a fast, reliable, place to store your data in the cloud. (Well OK, these complaints were all from one particular Orion committer, but whatever.)

In any case, I finally opted to write a plugin for Orion that lets you use S3 as a file store. This post talks about my experiences. If you'd rather skip right to the code, go here:

The S3 Worldview

In the S3 storage model, an object is the basic unit of storage. Objects are resources that you can refer to and manipulate using a REST API. Objects live in buckets, which are flat containers that can hold many objects (sometimes millions). As an S3 user, you'll generally create one or more buckets to hold your data. S3 also supports powerful security policies for granting bucket access to other users, but that doesn't concern us here.

Some S3 concepts.

The Orion File Model

Orion's file support is based on the concept of filesystems. The model here is decidedly more traditional than S3's:

Orion file concepts.

As the diagram tries to show, a filesystem is a place accessible to Orion where files can live. At the top level of a filesystem is the workspace. Within a workspace are folders and files. Naturally, folders can contain files and other folders, nested to an arbitrary depth.

NOTE: The file API is still evolving: see the Client API documentation for details.

From S3 to Orion

The important thing to note about S3 is that the bucket is the only unit of containment. Every object is contained by one and only one bucket. There is no containment relationship between objects. Initially this might seem like it limits us to a filesystem that is simply a huge, flat list of files. But that's not the case: a hierarchical view can be imposed on a properly-structured bucket. I won't go into detail about how this is done (read the AWS documentation if you're interested). The upshot is that we can indeed implement the Orion filesystem concepts on top of S3 — although some operations will necessarily be more complex (and likely slower) than if we were connecting to a true hierarchical back end.

Extending Orion's File Support

To connect S3 as an Orion filesystem, we need to write a plugin that contributes an implementation of a service named orion.core.file. Various parts of the Orion client UI delegate to implementors of this service. For example, the Orion navigator relies on the fetchChildren() operation, so implementing this function immediately allows the Orion navigator to descend into your filesystem and display child items as nodes in the navigation tree. Implementing read() and write() allows the Orion editor to open and save files that live in your file system. There's also file and folder creation, move, copy, rename, delete, etc. The more of these features you implement, the smoother the integration into Orion will be.

fetchChildren: function(location) loadWorkspace: function(location) createFolder: function(parentLocation, folderName) createFile: function(parentLocation, fileName) deleteFile: function(location) moveFile: function(sourceLocation, targetLocation, name) copyFile: function(sourceLocation, targetLocation) read: function(location, isMetadata) write: function(location, contents, args) remoteImport: function(targetLocation, options) remoteExport: function(sourceLocation, options) search: function(location, query)
List of orion.core.file API methods.

Domain Topology

For maximum portability, we want our implementation of the FileClient API to live purely on the client side: no server-side proxies or other hacks allowed. This means all our logic for talking to S3 will run as client-side JavaScript in a browser. Our code will make Ajax requests using the S3 REST API to manipulate S3 objects.

This imposes one important limitation: the web browser's same origin policy. Essentially the plugin (web page) hosting our S3 file service must have the same origin (protocol + host + port) as the target of all its Ajax requests. So does S3 let us configure a bucket to satisfy this limitation? The answer appears to be no. You can't have, say, a particular path prefix in your bucket configured to host a static website, while other prefixes respond to S3 API calls. A bucket can be configured either to host a static website, or as an S3 endpoint, but not both.

OK. Can we instead use two separate buckets to overcome this limitation? Yes! Conveniently, S3 allows you to access objects using 3 different URL styles:

  1. http://s3.amazonaws.com/bucket/key
  2. http://bucket.s3.amazonaws.com/key
  3. http://bucket/key
    (where bucket is a DNS CNAME record pointing to bucket.s3.amazonaws.com)

Notice that while URLs #2 and #3 have the bucket name as part of the host name (hence tying the origin to a single bucket), URL #1 does not. Using the style of URL #1, our plugin can be hosted from one bucket (configured as a publicly-readable static website), and still be free to blast Ajax REST requests at a second bucket (configured as a restricted-access S3 endpoint). Both buckets are accessed from the same domain, s3.amazonaws.com, so the browser won't complain about same-origin violations. Below is a diagram showing how this arrangement works out:

High-level view of the architecture.

In the diagram above, Bucket 1 is publicly-readable. Thus the Orion plugin registry can access and load the plugin without worrying about how to authenticate itself. By contrast, we've restricted access to Bucket 2 to just our S3 user account, thus the Ajax requests that are sent its way must be authenticated. The next section explains further. (Also note: I've drawn the Orion client UI in red since access it usually requires some kind of authentication to access. That's not a requirement, however: you can deploy Orion in all sorts of less-secure ways.)

Security

Keeping all the code on the client is convenient, but presents some security challenges. To modify the contents of Bucket 2, all our REST requests must be properly authenticated. Authentication requires three pieces of information: the request, our Access Key (public key), and our Secret Access Key (private key).

So the plugin needs to know how to authenticate requests, and to authenticate a request it needs your private key. This exposes a limitation in the existing Orion plugin/service framework, which is that there's no secure way to do this. (More generally, there's no way for a plugin to request access to a user setting from the Orion core… but that's a larger issue.)

My somewhat-inadvisable temporary solution is this: when the plugin requires your keys, it simply opens a JavaScript prompt asking you to paste them in. It also asks for a "passphrase", which it uses to encrypt the keys. The encrypted keys are kept in localStorage, which saves you the trouble of typing them in every time. The passphrase is not persisted, and gets squirreled away inside a closure variable, which vanishes when you close the browser tab.

The upshot is this: if you close and reopen your browser, the plugin will find the cached keys in local storage, and prompt only for the passphrase so it can decrypt the keys and carry on.

Now, obviously asking people to paste secret information into a web page is quite dubious. On top of that, storing sensitive information (albeit encrypted) in a localStorage database for a domain that you don't fully control (here, s3.amazonaws.com) is also a very bad idea. Finally, the usability suffers because the keys are tied to your browser, not your Orion user settings. (Hence, if you use a different web browser, you'll have to paste in your keys all over again). So please regard this as a starting point for future work around security and authentication in Orion, not a realistic long-term solution!

The End Result

If the disclaimer in the previous section hasn't scared you off, you can set up the plugin (see its GitHub readme for installation instructions). It lets you do basic file operations against your S3 bucket, and performs pretty well.

Left: Orion showing our plugin in action. Right: The S3 console showing the same contents.

Future Directions

Storing any sensitive data in localStorage is not a good approach. The naive solution might be to move your private credentials into the Orion preference store, where access to them could at least be managed somehow (perhaps on a per-plugin basis). But while security is one of Orion's concerns, becoming a secure-storage provider is not. In fact, it's probably better to compartmentalize sensitive information so that the Orion core never actually sees your private credentials.

So how do we accomplish that? One approach would be for the Orion core to simply establish a channel between an signing requester and a signing provider. The requester in this case would be our S3 plugin, and the provider could be any Orion-connected entity who possesses your S3 credentials and knows how to use them to sign a given request. (We can imagine a provider linked to a secure, trusted third-party service running a cloud deployment of something like KeePass, or even a private provider that is only accessible from your local network.)

Once the provider–requester channel is established, these pieces talk only to each other. Most importantly, your private key never has to leave the signing provider. This all sounds a bit hand-wavy right now, but remember Orion lives on the web where all kind of crazy ideas are possible.

See Also

Tuesday, July 12, 2011

Contributing syntax highlighting in Orion

NOTE: The API described in this post is deprecated. Don't use it. Instead, check out this blog post at PlanetOrion, which explains a new & improved highlighting API.

I'm keeping this post around for reference only.

In this post, I'll explain how to write a plugin for Orion that contributes syntax highlighting for JSON files using a TextMate grammar. It will let us go from this:

…to this:

Which is much nicer on the eyes/brain.

What you need

An Orionhub account, or a build of Orion 0.2 running on your local server.

Language Grammars

Orion implements a subset of the language grammar format used by TextMate, a popular text editor for Mac OS X. A grammar gives a declarative description of the structure of files of a certain type (for example, a .java or .php file). This "structure" can be limited to simple keyword recognition using regular expressions, but the format provides enough power to recognize more complex language constructs like classes or function definitions. Sometimes a carefully-written grammar can even detect common syntax errors, such as a missing comma in a list of items.

The grammar provides rules that tell the underlying TextView what CSS class a portion of text should have. The mapping of CSS classes to rules (font-color, font-weight, etc.) is ultimately determined by the stylesheet applied to the TextView.

Finding a Grammar

We're going to use a JSON grammar, which its authors have made available under a BSD license. Not every grammar you come across in the wild will allow redistribution, so take note.

Download the JSON grammar file from here:

http://svn.textmate.org/trunk/Bundles/JSON.tmbundle/Syntaxes/JSON.tmLanguage

From PList to JavaScript

If you open the JSON.tmLanguage file you'll see that it's written in a strange XML dialect. It's actually a Property List file, which is a common data storage format used in OS X. TextMate's grammars are written as property lists. This poses a problem for all faithful JavaScript programmers, who avoid XML like the plague.

But don't whip out your validator just yet. It turns out that property lists convert fairly well to JavaScript object literals, and we've written a tool that will do it automatically:

Orion grammar converter

Running the converter

Go to the grammar converter in your web browser. Find the folder on your computer where you saved the JSON.tmLanguage and drag it onto the textarea shown in the converter. Immediately, the result should appear below it. Click the Clean up button and the converted JavaScript object will be formatted a little more nicely.

Copy and paste the JavaScript object from the text field into a new file somewhere. This is our grammar.

Packaging the Grammar as a Plugin

Now we have to write some code that registers our grammar with the Orion editor. Orion allows clients to contribute language grammars using the orion.edit.highlighter service.

First grab a copy of plugin.js, which provides the Orion plugin API. Put it in a new folder called JsonProject in your Orion workspace. In the same folder, create a new file called jsonPlugin.html with the following content:

<!DOCTYPE html> <html> <head> <title>JSON highlighting plugin</title> <script src="plugin.js"></script> <script> var jsonGrammar = /* Paste grammar object here -- omitted to save space */ ; window.onload = function() { var provider = new eclipse.PluginProvider(); provider.registerServiceProvider("orion.edit.highlighter", {}, { type: "grammar", fileTypes: ["json"], grammar: jsonGrammar }); provider.registerServiceProvider("orion.navigate.openWith", {}, { name: "Orion web editor", href: "/edit/edit.html#${Location}", validationProperties: {Name: "*.(json)"} }); provider.connect(); }; </script> </head> <body></body> </html>

The first registerServiceProvider call is the crucial part: it contributes our grammar and tells the Orion editor to use it for .json files.

The second registerServiceProvider call associates the Orion editor with .json files so you can click them from the Navigator view to open them in the editor. This is not strictly necessary to get syntax highlighting, but it's usually what you want when you're adding support for a new file type.

Installing and Testing

To install your plugin, you'll need to host it at a web URL. You can do this by launching the JsonProject folder as a standalone site from within Orion — see this page for instructions. Or if you have a personal web server, you can copy the project contents there instead.

Once you have a URL to jsonPlugin.html, visit the Plugins page in Orion. Type the URL into the box and click Install.

Reload the Navigator page so that the plugin we just installed takes effect. Create a new .json file. Click on it and the editor should open. At this point you should be able to see it recognizing JSON styling as you type:

Conclusion

We've seen how to take an existing .tmLanguage file and use it to provide syntax highlighting in Orion.

See Also