An Extensible Approach to Browser Security Policy

Alex Russell posted some thoughts today about how he wishes the W3C would architect the next version of the Content Security Policy.

I agree with Alex that designing CSP as a “library” that uses other browser primitives would increase its long-term utility and make it compose better with other platform features.

Alex is advocating the use of extensible web principles in the design of this API, and I wholeheartedly support his approach.

Background

You can skip this section if you already understand CSP.

For the uninitiated, Content Security Policy is a feature that allows web sites to opt into stricter security than what the web platform offers by default. For example, it can restrict which domains to execute scripts from, prevent inline scripts from running altogether, and control which domains the network stack is allowed to make HTTP requests to.

To opt into stricter security using the current version of CSP, a website includes a new header (Content-Security-Policy) which can contain a number of directives.

For example, in order to prevent the browser from making any network requests to cross-domain resources, a server can return this header:

Content-Security-Policy: default-src 'self'

This instructs the browser to restrict all network requests to the current domain. This includes images, stylesheets, and fonts. Essentially, this means that scripts run on your page will be unable to send data to third-party domains, which is a common source of security vulnerabilities.

If you want to allow the browser to make requests to its own domain, plus the Google Ajax CDN, your server can do this:

Content-Security-Policy: default-src 'self' ajax.googleapis.com

Factoring the Network Layer

If you look at what CSP is doing, it’s essentially a syntax for controlling what the network stack is allowed to do.

There are other parts of the web platform that likewise control the network stack, and more all the time. What you’d like is for all of these features to be defined in terms of some lower-level primitive—ideally, one that was also exposed to JavaScript itself for more fine-grained, programmatic tweaks.

Imagine that you had the ability to intercept network requests programmatically, and decide whether to allow the request to continue. You might have an API something like this:

var origin = window.location.origin;
 
page.addEventListener('fetch', function(e) {
  var url = e.request.url;
  if (origin !== url.origin) {
    // block the network request
    e.preventDefault();
  }
 
  // otherwise, allow the network request through
});

You would then be able to describe how the browser interprets CSP in terms of this primitive API.

You could even imagine writing a CSP library purely in JavaScript!

page.addEventListener('fetch', function(e) {
  if (e.type === 'navigate') {
    e.respondWith(networkFetch(url).then(function(response) {
      // extract CSP headers and associate them with e.window.id
      // this is a pseudo-API to keep the implementation simple
      CSP.setup(e.window.id, response);
 
      return response;
    });
  } else {
    if (!CSP.isAllowed(e.window.id, e.request)) {
      e.preventDefault();
    }
  }
});

The semantics of CSP itself can be expressed in pure JavaScript, so these primitives are enough to build the entire system ourselves!

I have to confess, I’ve been hiding something from you. There is already a proposal to provide exactly these network layer hooks. It even has exactly the API I showed above.

The Extensible Web

Extensible web principles give us a very simple path forward.

Continue iterating on the declarative form of the Content Security Policy, but describe it in terms of the same primitives that power the Navigation Controller proposal.

When web developers want to tweak or extend the built-in security features, they can write a library that intercepts requests and applies tweaks to the policy by extending the existing header syntax.

If all goes well, those extensions will feed into the next iteration of CSP, giving us a clean way to let platform users inform the next generation of the platform.

This approach also improves the likelihood that other features that involve the network stack will compose well with CSP, since they will also be written in terms of this lower level primitive.

Many of the benefits that Dave Herman outlined in the closing of my last post are brought into concrete terms in this example.

I hope to write more posts that explore how extensible web principles apply to platform APIs, both new and old.

Fellow web developers, let’s persuade Adam Barth, Dan Veditz, Mike West (the CSP specification editors) to factor the next version of CSP in terms of the new Navigation Controller specification.

Then, we will have the tools we need to extend the web’s security model forward.

An Extensible Approach to Browser Security Policy

Alex Russell posted some thoughts today about how he wishes the W3C would architect the next version of the Content Security Policy.

I agree with Alex that designing CSP as a “library” that uses other browser primitives would increase its long-term utility and make it compose better with other platform features.

Alex is advocating the use of extensible web principles in the design of this API, and I wholeheartedly support his approach.

Background

You can skip this section if you already understand CSP.

For the uninitiated, Content Security Policy is a feature that allows web sites to opt into stricter security than what the web platform offers by default. For example, it can restrict which domains to execute scripts from, prevent inline scripts from running altogether, and control which domains the network stack is allowed to make HTTP requests to.

To opt into stricter security using the current version of CSP, a website includes a new header (Content-Security-Policy) which can contain a number of directives.

For example, in order to prevent the browser from making any network requests to cross-domain resources, a server can return this header:

Content-Security-Policy: default-src 'self'

This instructs the browser to restrict all network requests to the current domain. This includes images, stylesheets, and fonts. Essentially, this means that scripts run on your page will be unable to send data to third-party domains, which is a common source of security vulnerabilities.

If you want to allow the browser to make requests to its own domain, plus the Google Ajax CDN, your server can do this:

Content-Security-Policy: default-src 'self' ajax.googleapis.com

Factoring the Network Layer

If you look at what CSP is doing, it’s essentially a syntax for controlling what the network stack is allowed to do.

There are other parts of the web platform that likewise control the network stack, and more all the time. What you’d like is for all of these features to be defined in terms of some lower-level primitive—ideally, one that was also exposed to JavaScript itself for more fine-grained, programmatic tweaks.

Imagine that you had the ability to intercept network requests programmatically, and decide whether to allow the request to continue. You might have an API something like this:

var origin = window.location.origin;
 
page.addEventListener('fetch', function(e) {
  var url = e.request.url;
  if (origin !== url.origin) {
    // block the network request
    e.preventDefault();
  }
 
  // otherwise, allow the network request through
});

You would then be able to describe how the browser interprets CSP in terms of this primitive API.

You could even imagine writing a CSP library purely in JavaScript!

page.addEventListener('fetch', function(e) {
  if (e.type === 'navigate') {
    e.respondWith(networkFetch(url).then(function(response) {
      // extract CSP headers and associate them with e.window.id
      // this is a pseudo-API to keep the implementation simple
      CSP.setup(e.window.id, response);
 
      return response;
    });
  } else {
    if (!CSP.isAllowed(e.window.id, e.request)) {
      e.preventDefault();
    }
  }
});

The semantics of CSP itself can be expressed in pure JavaScript, so these primitives are enough to build the entire system ourselves!

I have to confess, I’ve been hiding something from you. There is already a proposal to provide exactly these network layer hooks. It even has exactly the API I showed above.

The Extensible Web

Extensible web principles give us a very simple path forward.

Continue iterating on the declarative form of the Content Security Policy, but describe it in terms of the same primitives that power the Navigation Controller proposal.

When web developers want to tweak or extend the built-in security features, they can write a library that intercepts requests and applies tweaks to the policy by extending the existing header syntax.

If all goes well, those extensions will feed into the next iteration of CSP, giving us a clean way to let platform users inform the next generation of the platform.

This approach also improves the likelihood that other features that involve the network stack will compose well with CSP, since they will also be written in terms of this lower level primitive.

Many of the benefits that Dave Herman outlined in the closing of my last post are brought into concrete terms in this example.

I hope to write more posts that explore how extensible web principles apply to platform APIs, both new and old.

Fellow web developers, let’s persuade Adam Barth, Dan Veditz, Mike West (the CSP specification editors) to factor the next version of CSP in terms of the new Navigation Controller specification.

Then, we will have the tools we need to extend the web’s security model forward.

Google Authorship on Giant Robots

You may have noticed a recent addition to our blog posts. We concluded an experiment to add a “Written by [Author Name]” byline and link the author’s name to their Google+ profile.

Google calls this Authorship. It is a feature that lets you verify that you are the author of some content, such as a blog post. It is kind of like a microformat that Google uses in its search algorithms. It also causes an author’s face to show in Google search results:

We take the following one-time steps to set up Google Authorship for each author on our blog:

1. Get a Google+ account.
2. Verify our email address for the domain, which will automatically add you as a collaborator of that website on Google+.

Now, whenever we author a post for one of those websites, we take the following steps:

1. Add a byline that has a link like Written by <a rel='author' href='http://plus.google.com/GOOGLE_PLUS_ID'>AUTHOR NAME</a>..
2. Link back to the article from our Google+ profile.

The links in both directions are help to confirm authorship of the article.

We can then see statistics on impressions, clicks, click-through rates, and average position in search results for pages for which we are the verified author:

Google Authorship is a nice way for us to better understand how our content appears in Google searches as well as expose our content to a wider audience.

Written by Caleb Thompson.

Extend the Web Forward

If we want to move the web forward, we must increase our ability as web developers to extend it with new features.

For years, we’ve grabbed the browsers extension points with two hands, not waiting for the browser vendors to gift us with new features. We built selector engines, a better DOM API, cross-domain requests, cross-frame APIs.

When the browser has good extension points (or any extension points, really), we live in a virtuous cycle:

• Web developers build new APIs ourselves, based on use-cases we have
• We compete with each other, refining our libraries to meet use cases we didn’t think of
• The process of competition makes the libraries converge towards each other, focusing the competition on sharp use-case distinctions
• Common primitives emerge, which browser vendors can implement. This improves performance and shrinks the amount of library code necessary.
• Rinse, repeat.

We’ve seen this time and time again. When it works, it brings us querySelectorAll, the template element, and Object.observe.

The Sad Truth

The sad truth is that while some areas of the browser are extremely extensible, other areas are nearly impossible to extend.

Some examples include the behavior and lifecycle of custom element in HTML, the CSS syntax, and the way that the browser loads an HTML document in the first place. This makes it hard to extend HTML, CSS, or build libraries that support interesting offline capabilities.

And even in some places that support extensibility, library developers have to completely rewrite systems that already exist. For example, John Resig had to rewrite the selector engine from scratch just to add a few additional pseudo-properties, and there is still no way add custom pseudo-properties to querySelectorAll.

Declarative vs. Imperative

A lot of people see this as a desire to write everything using low-level JavaScript APIs, forever.

No.

If things are working well, JavaScript library authors write new declarative APIs that the browser can roll in. Nobody wants to write everything using low-level calls to canvas, but we’re happy that canvas lets us express low-level things that we can evolve and refine.

The alternative, that web authors are stuck with only the declarative APIs that standards bodies have invented, is too limiting, and breaks the virtuous cycle that allows web developers to invent and iterate on new high-level features for the browser.

In short, we want to extend the web forward with new high-level APIs, but that means we need extension points we can use.

Explaining the Magic

If we want to let web authors extend the web forward, the best way to do that is to explain existing and new high-level forms in terms of low-level APIs.

A good example of in-progress work along these lines in Web Components, which explains how elements work in terms of APIs that are exposed to JavaScript. This means that if a new custom element becomes popular, it’s a short hop to implementing it natively, because the JavaScript implementation is not a parallel universe; it’s implemented in terms of the same concepts as native elements.

That doesn’t necessarily mean that browsers will simply rubber-stamp popular components, but by giving library authors the the tools to make components with native-like interfaces, it will be easy for vendors to synthesize web developer zeitgeist into something standard.

Another example is offline support. Right now, we have the much-derided AppCache, which is a declarative-only API that makes it possible to display an HTML page, along with its assets, even if the browser is offline.

AppCache is not implemented in terms of a low-level JavaScript API, so when web developers discovered problems with it, we had no way to extend or modify it to meet our needs. This also meant that we had no way to show the browser vendors what kinds of solutions would work for us.

Instead, we ended up with years of stagnation, philosophical disagreements and deadlock between web developers and specification editors, and no way to move forward.

What we need instead is something like Alex Russell’s proposal that allows applications to install JavaScript code in the cache that intercepts HTTP requests from the page and can fulfill them, even when the app is offline. With an API like this, the current AppCache could be written as a library!

Something like Jonas Sicking’s app cache manifest is a great companion proposal, giving us a nice starting point for a high-level API. But this time if the high-level API doesn’t work, we can fix it by using the low-level API to tweak and improve the manifest.

We can extend the web forward.

Extensions != Rewriting

It’s important to note that web developers don’t want a high level API and then a cliff into the low-level API.

Today, while you can implement custom elements or extend the selector engine, you can only do this by rewriting large chunks of the stack alongside the feature you want.

Real extensibility means an architecture that lets you tweak, not rewrite. For example, it would be possible to add custom rules to CSS by writing a full selector engine and application engine, and apply rules via .style as the DOM changes. With mutation observers, this might even be feasible. In fact, this is how some of the most devious hacks in the platform today (like the Polymer Shadow DOM polyfill) actually work.

That kind of “extensibility” doesn’t fit the bill. It doesn’t compose well with other extensions, defeats the browser’s ability to do performance work on unrelated parts of the stack (because the entire stack had to be rewritten), and is too hard to provide meaningful iteration.

Browser implementers are often wary of providing extension points that can be performance footguns. The biggest footgun is using libraries that rewrite the entire stack in JavaScript, and whole-stack-rewriting strategies are the tactic du jour today. For performance, we have little to lose and much to gain by making extensions more granular.

Extend the Web Forward

So what do we gain from a more extensible web? I’ll let Dave Herman, a member of TC39, answer that for me.

• When you design new APIs, you are forced to think about how the existing system can express most of the semantics. This cleanly separates what new power is genuinely needed and what isn’t. This prevents cluttering the semantics with unnecessary new magic
• Avoiding new magic avoids new security surface area
• Avoiding new magic avoids new complexity (and therefore bugs) in implementation
• Avoiding new magic makes more of the new APIs polyfillable
• Being more polyfillable means people can ramp up faster, leading to faster adoption and evolution of the platform
• Avoiding new magic means that optimizations in the engines can focus on the stable core, which affects more of new APIs as they are added. This leads to better performance with less implementation effort
• Avoiding new magic means less developer education required; people can understand new APIs more easily when they come out, because they build off of known concepts
• This means that the underlying platform gets fleshed out to be expressive enough to prototype new ideas. Library authors can experiment with new features and create more cowpaths to fill the Web API pipeline

All this, and more! There’s something for everybody!

Implementors and web developers: let’s work together to extend the web forward!

Extend the Web Forward

If we want to move the web forward, we must increase our ability as web developers to extend it with new features.

For years, we’ve grabbed the browsers extension points with two hands, not waiting for the browser vendors to gift us with new features. We built selector engines, a better DOM API, cross-domain requests, cross-frame APIs.

When the browser has good extension points (or any extension points, really), we live in a virtuous cycle:

• Web developers build new APIs ourselves, based on use-cases we have
• We compete with each other, refining our libraries to meet use cases we didn’t think of
• The process of competition makes the libraries converge towards each other, focusing the competition on sharp use-case distinctions
• Common primitives emerge, which browser vendors can implement. This improves performance and shrinks the amount of library code necessary.
• Rinse, repeat.

We’ve seen this time and time again. When it works, it brings us querySelectorAll, the template element, and Object.observe.

The Sad Truth

The sad truth is that while some areas of the browser are extremely extensible, other areas are nearly impossible to extend.

Some examples include the behavior and lifecycle of custom element in HTML, the CSS syntax, and the way that the browser loads an HTML document in the first place. This makes it hard to extend HTML, CSS, or build libraries that support interesting offline capabilities.

And even in some places that support extensibility, library developers have to completely rewrite systems that already exist. For example, John Resig had to rewrite the selector engine from scratch just to add a few additional pseudo-properties, and there is still no way add custom pseudo-properties to querySelectorAll.

Declarative vs. Imperative

A lot of people see this as a desire to write everything using low-level JavaScript APIs, forever.

No.

If things are working well, JavaScript library authors write new declarative APIs that the browser can roll in. Nobody wants to write everything using low-level calls to canvas, but we’re happy that canvas lets us express low-level things that we can evolve and refine.

The alternative, that web authors are stuck with only the declarative APIs that standards bodies have invented, is too limiting, and breaks the virtuous cycle that allows web developers to invent and iterate on new high-level features for the browser.

In short, we want to extend the web forward with new high-level APIs, but that means we need extension points we can use.

Explaining the Magic

If we want to let web authors extend the web forward, the best way to do that is to explain existing and new high-level forms in terms of low-level APIs.

A good example of in-progress work along these lines in Web Components, which explains how elements work in terms of APIs that are exposed to JavaScript. This means that if a new custom element becomes popular, it’s a short hop to implementing it natively, because the JavaScript implementation is not a parallel universe; it’s implemented in terms of the same concepts as native elements.

That doesn’t necessarily mean that browsers will simply rubber-stamp popular components, but by giving library authors the the tools to make components with native-like interfaces, it will be easy for vendors to synthesize web developer zeitgeist into something standard.

Another example is offline support. Right now, we have the much-derided AppCache, which is a declarative-only API that makes it possible to display an HTML page, along with its assets, even if the browser is offline.

AppCache is not implemented in terms of a low-level JavaScript API, so when web developers discovered problems with it, we had no way to extend or modify it to meet our needs. This also meant that we had no way to show the browser vendors what kinds of solutions would work for us.

Instead, we ended up with years of stagnation, philosophical disagreements and deadlock between web developers and specification editors, and no way to move forward.

What we need instead is something like Alex Russell’s proposal that allows applications to install JavaScript code in the cache that intercepts HTTP requests from the page and can fulfill them, even when the app is offline. With an API like this, the current AppCache could be written as a library!

Something like Jonas Sicking’s app cache manifest is a great companion proposal, giving us a nice starting point for a high-level API. But this time if the high-level API doesn’t work, we can fix it by using the low-level API to tweak and improve the manifest.

We can extend the web forward.

Extensions != Rewriting

It’s important to note that web developers don’t want a high level API and then a cliff into the low-level API.

Today, while you can implement custom elements or extend the selector engine, you can only do this by rewriting large chunks of the stack alongside the feature you want.

Real extensibility means an architecture that lets you tweak, not rewrite. For example, it would be possible to add custom rules to CSS by writing a full selector engine and application engine, and apply rules via .style as the DOM changes. With mutation observers, this might even be feasible. In fact, this is how some of the most devious hacks in the platform today (like the Polymer Shadow DOM polyfill) actually work.

That kind of “extensibility” doesn’t fit the bill. It doesn’t compose well with other extensions, defeats the browser’s ability to do performance work on unrelated parts of the stack (because the entire stack had to be rewritten), and is too hard to provide meaningful iteration.

Browser implementers are often wary of providing extension points that can be performance footguns. The biggest footgun is using libraries that rewrite the entire stack in JavaScript, and whole-stack-rewriting strategies are the tactic du jour today. For performance, we have little to lose and much to gain by making extensions more granular.

Extend the Web Forward

So what do we gain from a more extensible web? I’ll let Dave Herman, a member of TC39, answer that for me.

• When you design new APIs, you are forced to think about how the existing system can express most of the semantics. This cleanly separates what new power is genuinely needed and what isn’t. This prevents cluttering the semantics with unnecessary new magic
• Avoiding new magic avoids new security surface area
• Avoiding new magic avoids new complexity (and therefore bugs) in implementation
• Avoiding new magic makes more of the new APIs polyfillable
• Being more polyfillable means people can ramp up faster, leading to faster adoption and evolution of the platform
• Avoiding new magic means that optimizations in the engines can focus on the stable core, which affects more of new APIs as they are added. This leads to better performance with less implementation effort
• Avoiding new magic means less developer education required; people can understand new APIs more easily when they come out, because they build off of known concepts
• This means that the underlying platform gets fleshed out to be expressive enough to prototype new ideas. Library authors can experiment with new features and create more cowpaths to fill the Web API pipeline

All this, and more! There’s something for everybody!

Implementors and web developers: let’s work together to extend the web forward!

Two gotchas every CoffeeScript and Backbone.js developer should know

If you’re a Ruby developer working with Rails, at some point you’re going to need to work with JavaScript. While the two languages have many similarities, the fundamental differences in their object models can be quite jarring. CoffeeScript helps to provide a more Ruby-like syntax, but if you’re not careful you can introduce bugs to your code in surprising ways.

Let’s look at an example. Let’s say we have an expensive operation, and need to cache the result. In CoffeeScript, we might write the this code as:

class GiantRobot
  smashCache: {}

  smashInto: (other) =>
    @smashCache[other] ||= @expensiveCalculations(other)

However, this code leads to a surprising problem when we start smashing things together.

ralph = new GiantRobot()
voltron = new GiantRobot()

ralph.smashInto(optimusPrime)   # => not cached
ralph.smashInto(optimusPrime)   # => cached
voltron.smashInto(optimusPrime) # => cached?!

To see why this occurs, we need to take a look at how objects work in JavaScript. Unlike Ruby, JavaScript has no concept of classes. Instead it constructs its objects using prototypes. If you’re writing your code in CoffeeScript, you can almost always ignore this fact, and write your code as if you were in Ruby or Python. In this case, however, CoffeeScript is making our problem less apparent, so let’s take a look at what this code looks like using plain JavaScript.

function GiantRobot() {}

GiantRobot.prototype.smashCache = {};

GiantRobot.prototype.smashInto = function(other) {
  if (!this.smashCache[other]) {
    this.smashCache[other] = this.expensiveCalculations(other);
  }
  return this.smashCache[other];
};

Specifically, our problem stems from the fact that setting smashCache on the prototype may not work the way you’d think.

How prototypes work

When we create ralph with, ralph technically has no smashCache property. When we do this.smashCache, JavaScript looks for that property on ralph, followed by ralph’s prototype, then ralph’s prototype’s prototype, and so forth until it finds something that has the smashCache property.

JavaScript provides us a way to see this in action, using the hasOwnProperty method.

ralph.hasOwnProperty('smashCache')                # => false
GiantRobot.prototype.hasOwnProperty('smashCache') # => true

So when we call this.smashCache, we are always getting back GiantRobot.prototype.smashCache, which means when we mutate it, we are mutating the same instance of smashCache that is used by every instance of GiantRobot. This is only an issue when we’re talking about arrays and objects.

ralph.smashCache = 'some value'
# Variable assignment sets the property directly on the instance
ralph.hasOwnProperty('smashCache') == true

Avoiding mutating the prototype

So how do we get around this? Ironically, the answer is simply to make our original example look more like our Ruby code.

class GiantRobot
  constructor: ->
    @smashCache = {}

Now each instance will have it’s own separate smashCache, and all is well with the world.

Backbone.js gotcha: Mutating the defaults object

A similar problem exists in Backbone.js when using the defaults object on your models.

class RobotProfile extends Backbone.Model
  defaults:
    images: []

  addImage: (newImage) ->
    @get('images').push(newImage) # => Mutates a shared instance

The solution for this problem is even simpler. If we change the defaults property to be a function that returns the same hash, Backbone will call it every time a new instance is created.

class RobotProfile extends Backbone.Model
  defaults: ->
    images: []

Now our images array will no longer be shared across multiple instances, and we can mutate to our heart’s content.

Written by Sean Griffin

Episode 49: The psychology of work

Episode 49: The psychology of work:

In this week’s episode, recorded at RailsConf 2013, Ben Orenstein is joined by Gregg Pollack and Nathaniel Bibler from EnvyLabs and codeschool.com. Gregg shares what he’s learned running his business, when not to be transparent, how to deal with compensation, and how the EnvyLabs compensation structure has changed over the years. Nathan, Gregg, and Ben also discuss Code School, yearly payments to a subscription, making courses effective, effective marketing, the effectiveness of mailing lists, community events, shared ownership, and much more.

• Episode Notes and Links
• Subscribe via iTunes
• Subscribe via RSS
• Direct Download