Category Archives : Web Development

For generic web development


Remembering Terry Pratchett

“Tech-savvy admirers of the late Terry Pratchett have hit upon an idea for a particularly appropriate memorial. It will be everywhere and nowhere, hiding in the code of the internet.

Pratchett’s 33rd Discworld novel, Going Postal, tells of the creation of an internet-like system of communication towers called “the clacks”. When John Dearheart, the son of its inventor, is murdered, a piece of code is written called “GNU John Dearheart” to echo his name up and down the lines. “G” means that the message must be passed on, “N” means “not logged”, and “U” means the message should be turned around at the end of a line. (This was also a realworld tech joke: GNU is a free operating system, and its name stands, with recursive geek humour, for “GNU’s not Unix”.) The code causes Dearheart’s name to be repeated indefinitely throughout the system, because: “A man is not dead while his name is still spoken.”

What better way to remember the beloved inventor of this fictional system, then, than “GNU Terry Pratchett”?”

Source: The Guardian

This, of course, means simply adding a HEADER value for HTTP responses in your favourite Web Server.  The header name = “X-Clacks-Overhead”, value = “GNU Terry Pratchett”.

image


IIS 8.5 Dynamic Compression Issue 1

Windows Server 2012 R2 comes with IIS 8.5, and in this release an issue has been found in relation to the Dynamic Compression module.  The module sets the “Vary” header which is used to specify caching properties that the browser uses to determine whether the response should be cached or not. 

In IIS 8.0 and earlier, the Dynamic Compression module was overwriting the Vary header with the value “Accept-Encoding”, and as it happens this is the correct value to ensure that dynamic content is correctly cached – but – according to IIS it should be appending this value to the existing value and not overwriting it.

As it happens, this was supposed to be fixed in IIS 8.5 but the fix appears to be broken.   In IIS 8.5 (which ships with Windows Server 2012 R2) the Vary header is being set to “*” and the “Accept-Encoding” from the Dynamic Compression module is not appended.  The result of this is that no dynamic content is being cached by the browser.

Workaround

Thankfully there is an easy workaround in IIS 8.5 for this:

1. Select an IIS site, and go to Configuration Editor.

image

2. Select system.web/caching/outputCache section, then set the omitVaryStar property to true

image

image

Setting this value results in the Vary header being returned with a value of “Accept-Encoding” and the browser then caches the dynamic content.


Getting to know Cross-Origin Resource Sharing (CORS)

Hello there.  I’ve been spending a lot of time of late trying to develop a solution to a very obscure problem scenario.  The entire problem itself is outside the scope of this article – and to be honest, probably wouldn’t be terribly relevant to many – however, I felt there was value in articulating my recent experiences with Cross-Origin Resource Sharing, or CORS for short.

So what is CORS? 

 

To quote from Wikipedia:

Cross-origin resource sharing (CORS) is a mechanism that allows many resources (e.g., fonts, JavaScript, etc.) on a web page to be requested from another domain outside the domain from which the resource originated.[1] In particular, JavaScript’s AJAX calls can use the XMLHttpRequest mechanism.

Such “cross-domain” requests would otherwise be forbidden by web browsers, per the same-origin security policy. CORS defines a way in which the browser and the server can interact to determine whether or not to allow the cross-origin request.[2] It is more useful than only allowing same-origin requests, but it is more secure than simply allowing all such cross-origin requests.”

Now that we’ve cleared that up…  my take – a web site makes use of resources which are hosted on another site outside of its domain.  This is an important distinction owing primarily due to cross-site scripting (XSS) and cross-site request forgery (CSRF) vulnerabilities.

 

What does something like CORS address exactly?

Well, the concept is fairly straightforward.  What CORS aims to do is have two websites (sites, pages, APIs etc.) agree on what kind of resources and types of requests one website will provide to another.  Both must agree exactly on what is being shared and how.

 

How is this accomplished?

There’s a few parties who need to participate to enable CORS – the two parties involved, of course, and the user’s browser.  Both sites need to request and respond to each other in an expected manner, and browsers need to be aware of, and in some cases make special requests to ensure CORS works correctly.

In essence, what happens is that both websites agree on how resources will be shared.  The requesting site must be known as an “allowed origin” by the site providing the resources.  The response also must contain headers which contain scope for acceptable resource sharing, e.g. naming allowable methods (e.g. GET, PUT) and whether credentials are supported.  Browsers themselves are the last key – they must respect the restrictions established by the requesting site and the resource site.

 

What is a “pre-flight request”?

In some cases, a browser might make a special type of request known as an OPTIONS request, which is sort of like an initial handshake before performing the actual request specified (e.g. a GET request). 

In essence, an OPTIONS request attempts to determine what supported methods and other information is available from a resource sharing server.  In browser terms, this is known as a “pre-flight” request and is often attempted automatically by the browser.

The first time a cross-site request might fail (and in subsequent attempts) the browser’s JavaScript console might log something similar to the following error:

XMLHttpRequest cannot load https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json. The request was redirected to ‘https://<ANOTHERSERVER>’, which is disallowed for cross-origin requests that require preflight.

Here’s an example of a browser (Chrome) attempting an OPTIONS pre-flight request and failing:

failed

Let’s take a look at a pre-flight HTTP(S) request example.

Remote Address:<IPADDRESS>:443

Request Headers URL:https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json

Request Method:OPTIONS

Status Code:200 OK

Request Headers

OPTIONS /p?ReadViewEntries&outputformat=json
HTTP/1.1

Host: <TARGETSERVER>

Connection: keep-alive

Access-Control-Request-Method: GET

Origin: https://<ORIGINSERVER>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36

Access-Control-Request-Headers: accept, content-type

Accept: */*

Referer: https://<ORIGINSERVER>/Home/Test

Accept-Encoding: gzip,deflate,sdch

Accept-Language: en-US,en;q=0.8

Query String Parametersview sourceview URL encoded

ReadViewEntries:

outputformat:json

count:4000

r:0.88333622

charset:utf-8

 

Under normal circumstances, a target server which honours this request would respond with something similar to this:

HTTP/1.1 200 OK

Date: Tue, 11 Nov 2014 03:35:05 GMT

Access-Control-Allow-Origin: https://<ORIGINSERVER>

Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept,Access-Control-Request-Headers,Access-Control-Allow-Methods,Access-Control-Allow-Origin,Access-Control-Allow-Credentials

Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT, HEAD

Access-Control-Allow-Credentials: true

Content-Length: 495

Keep-Alive: timeout=10, max=100

Connection: Keep-Alive

Content-Type: text/html; charset=iso-8859-1

 

The most important thing in this response isn’t probably what you’d expect it to be.  It’s actually the HTTP STATUS CODE (i.e. 200).  For CORS pre-flight to work, the resource target must respond with a Status Code of 200 to a HTTP OPTIONS request – and it must do so for unauthenticated requests!

 

Why must the pre-flight/OPTIONS requests be unauthenticated?

It’s actually a requirement direct from the respective W3C specification regarding pre-flight requests:

Otherwise, make a preflight request. Fetch the request URL from origin source origin with the manual redirect flag and the block cookies flag set, using the method OPTIONS, and with the following additional constraints:

[1] https://dvcs.w3.org/hg/cors/raw-file/tip/Overview.html#preflight-request

Therefore, a response status code of 302 (Found – Redirect, usually to authenticate) or 401 (unauthorised) will clearly fail pre-flight.  Note that the resource (target) server doesn’t have to honour all OPTIONS requests, you could lock down the server’s security to only respond (Status 200) to requests on certain paths, for example.

In the example screenshot earlier, the pre-flight was failing because the resource server had authentication enabled for the OPTIONS request, and it was redirecting to a security token service (STS) for authentication.

 

Now that that’s all cleared up, how about some sample code? 

Here’s a pretty straightforward request which sets the appropriate header values for cross-domain and with credentials for a GET request, which is expecting a JSON response:

var aQuery = function()

{

     var url = ‘https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json';

    
$.ajax(url, {

            type: “GET”,

            contentType: “application/json; charset=utf-8″,

            success: function(data, status, xhr) {

                alert(data);

            },

            xhrFields: {

                withCredentials: true

            },

            crossDomain: true

        });

}

In many cases though, this is not enough.  Some browsers, particularly older versions for example Internet Explorer 8 and 9 simply don’t support CORS well enough.  A more version sensitive approach might be something like this:

var basicQuery = function ()

{    

      var url = ‘https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json';

      var method = ‘GET';

      var xhr = new XMLHttpRequest();

       

      if (“withCredentials” in xhr) {           

           // Most browsers.       

            xhr.open(method, url, true);

            xhr.setRequestHeader(“Access-Control-Allow-Origin”, “https://<ORIGINSERVER>)”);

            xhr.setRequestHeader(“Access-Control-Allow-Credentials”, “true”);

            xhr.setRequestHeader(“Access-Control-Allow-Methods”, “GET”);

      } else if (typeof XDomainRequest != “undefined”) {

                 // IE8 & IE9      

           xhr = new XDomainRequest();

           xhr.open(method, url);

      } else {

           // CORS not supported.

           xhr = null;

      }

 

      if (xhr != null)

      {                

            xhr.followsRedirect=false;          

            xhr.withCredentials=true;

 

            xhr.onreadystatechange  = function() {

                  if(xhr.readyState==4)

                  {

                        if(xhr.status == 200)

                        {

                              alert(xhr.responseText);                 

                        }

                        else

                        {

                              alert(“error”);

                        }

                  }

            }

            xhr.send();

      }

}

 

Finally, let’s see that gratifying successful CORS request:

success

 

Some notes about setting headers

  • CORS header values in a request must be supported on the resource server for a given request to succeed, i.e. for a request which specifies Access-Control-Allow-Credentials, the target server must respond with Access-Control-Allow-Credentials listed in the Access-Control-Allow-Headers. 
  • The same applies to methods, e.g. a GET method must be listed in the response Access-Control-Allow-Methods
  • When you set the Access-Control-Allow-Origin value, it should be the name of the origin server (the origin of the request) it really shouldn’t be asterisk (*) which is often used in sample code.  The target resource server also must recognise/honour the origin.