Author Archives : Rob Sanders


About Rob Sanders

IT Professional and TOGAF 9 certified architect with over 16 years experience, 14 in commercial software development and 8 years in IT consulting. Check out the "About Rob" page for more information.


Fight secret copyright deals: a global TPP teach-in

via Maira Sutton, Global Policy Analyst, Electronic Frontier Foundation

Electronic Frontier Foundation - www.eff.org

Hi there,

A lot has happened on the Trans-Pacific Partnership (TPP) front these past couple months. There was another major leak of a recent draft of the TPP’s Intellectual Property chapter that confirmed that many of the most extreme copyright provisions are still there, and in some cases, have even gotten worse. It also showed that it included some new harmful language on “trade secrets” that could be used to crack down on journalists.

The TPP could be finalized in the coming months, and things are moving fast. That’s why we’d like to invite you to a public, online teach-in to hear representatives from several digital rights groups based in TPP countries share their analysis of the latest leaked text, as well as to lay out the current state of play of the negotiations. We’ll make sure there’s time at the end for users to ask us questions.

When the time comes, we’ll need to be ready to fight this agreement on a global front. So please join us on Nov 19, and help us spread the word about how TPP poses a threat to users’ rights.

When: November 19, 2014, 12pm Pacific / 3pm Eastern / 5pm Chile / 7am Australia EST / 9am New Zealand

Where: Google Hangout and streamed on Youtube (link to livestream will be provided here on the day of the event)

Who: Members of the TPP Fair Deal Coalition, including Electronic Frontier Foundation (US), Derechos Digitales (Chile), Consumer NZ (New Zealand), Public Citizen (US)

If you have any questions, please don’t hesitate to contact me.

Cheers,
Maira

P.S. Make sure to check out some of our Fair Deal Coalition partners’ new TPP materials:


Getting to know Cross-Origin Resource Sharing (CORS)

Hello there.  I’ve been spending a lot of time of late trying to develop a solution to a very obscure problem scenario.  The entire problem itself is outside the scope of this article – and to be honest, probably wouldn’t be terribly relevant to many – however, I felt there was value in articulating my recent experiences with Cross-Origin Resource Sharing, or CORS for short.

So what is CORS? 

 

To quote from Wikipedia:

Cross-origin resource sharing (CORS) is a mechanism that allows many resources (e.g., fonts, JavaScript, etc.) on a web page to be requested from another domain outside the domain from which the resource originated.[1] In particular, JavaScript’s AJAX calls can use the XMLHttpRequest mechanism.

Such “cross-domain” requests would otherwise be forbidden by web browsers, per the same-origin security policy. CORS defines a way in which the browser and the server can interact to determine whether or not to allow the cross-origin request.[2] It is more useful than only allowing same-origin requests, but it is more secure than simply allowing all such cross-origin requests.”

Now that we’ve cleared that up…  my take – a web site makes use of resources which are hosted on another site outside of its domain.  This is an important distinction owing primarily due to cross-site scripting (XSS) and cross-site request forgery (CSRF) vulnerabilities.

 

What does something like CORS address exactly?

Well, the concept is fairly straightforward.  What CORS aims to do is have two websites (sites, pages, APIs etc.) agree on what kind of resources and types of requests one website will provide to another.  Both must agree exactly on what is being shared and how.

 

How is this accomplished?

There’s a few parties who need to participate to enable CORS – the two parties involved, of course, and the user’s browser.  Both sites need to request and respond to each other in an expected manner, and browsers need to be aware of, and in some cases make special requests to ensure CORS works correctly.

In essence, what happens is that both websites agree on how resources will be shared.  The requesting site must be known as an “allowed origin” by the site providing the resources.  The response also must contain headers which contain scope for acceptable resource sharing, e.g. naming allowable methods (e.g. GET, PUT) and whether credentials are supported.  Browsers themselves are the last key – they must respect the restrictions established by the requesting site and the resource site.

 

What is a “pre-flight request”?

In some cases, a browser might make a special type of request known as an OPTIONS request, which is sort of like an initial handshake before performing the actual request specified (e.g. a GET request). 

In essence, an OPTIONS request attempts to determine what supported methods and other information is available from a resource sharing server.  In browser terms, this is known as a “pre-flight” request and is often attempted automatically by the browser.

The first time a cross-site request might fail (and in subsequent attempts) the browser’s JavaScript console might log something similar to the following error:

XMLHttpRequest cannot load https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json. The request was redirected to ‘https://<ANOTHERSERVER>’, which is disallowed for cross-origin requests that require preflight.

Here’s an example of a browser (Chrome) attempting an OPTIONS pre-flight request and failing:

failed

Let’s take a look at a pre-flight HTTP(S) request example.

Remote Address:<IPADDRESS>:443

Request Headers URL:https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json

Request Method:OPTIONS

Status Code:200 OK

Request Headers

OPTIONS /p?ReadViewEntries&outputformat=json
HTTP/1.1

Host: <TARGETSERVER>

Connection: keep-alive

Access-Control-Request-Method: GET

Origin: https://<ORIGINSERVER>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36

Access-Control-Request-Headers: accept, content-type

Accept: */*

Referer: https://<ORIGINSERVER>/Home/Test

Accept-Encoding: gzip,deflate,sdch

Accept-Language: en-US,en;q=0.8

Query String Parametersview sourceview URL encoded

ReadViewEntries:

outputformat:json

count:4000

r:0.88333622

charset:utf-8

 

Under normal circumstances, a target server which honours this request would respond with something similar to this:

HTTP/1.1 200 OK

Date: Tue, 11 Nov 2014 03:35:05 GMT

Access-Control-Allow-Origin: https://<ORIGINSERVER>

Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept,Access-Control-Request-Headers,Access-Control-Allow-Methods,Access-Control-Allow-Origin,Access-Control-Allow-Credentials

Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT, HEAD

Access-Control-Allow-Credentials: true

Content-Length: 495

Keep-Alive: timeout=10, max=100

Connection: Keep-Alive

Content-Type: text/html; charset=iso-8859-1

 

The most important thing in this response isn’t probably what you’d expect it to be.  It’s actually the HTTP STATUS CODE (i.e. 200).  For CORS pre-flight to work, the resource target must respond with a Status Code of 200 to a HTTP OPTIONS request – and it must do so for unauthenticated requests!

 

Why must the pre-flight/OPTIONS requests be unauthenticated?

It’s actually a requirement direct from the respective W3C specification regarding pre-flight requests:

Otherwise, make a preflight request. Fetch the request URL from origin source origin with the manual redirect flag and the block cookies flag set, using the method OPTIONS, and with the following additional constraints:

[1] https://dvcs.w3.org/hg/cors/raw-file/tip/Overview.html#preflight-request

Therefore, a response status code of 302 (Found – Redirect, usually to authenticate) or 401 (unauthorised) will clearly fail pre-flight.  Note that the resource (target) server doesn’t have to honour all OPTIONS requests, you could lock down the server’s security to only respond (Status 200) to requests on certain paths, for example.

In the example screenshot earlier, the pre-flight was failing because the resource server had authentication enabled for the OPTIONS request, and it was redirecting to a security token service (STS) for authentication.

 

Now that that’s all cleared up, how about some sample code? 

Here’s a pretty straightforward request which sets the appropriate header values for cross-domain and with credentials for a GET request, which is expecting a JSON response:

var aQuery = function()

{

     var url = ‘https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json';

    
$.ajax(url, {

            type: “GET”,

            contentType: “application/json; charset=utf-8″,

            success: function(data, status, xhr) {

                alert(data);

            },

            xhrFields: {

                withCredentials: true

            },

            crossDomain: true

        });

}

In many cases though, this is not enough.  Some browsers, particularly older versions for example Internet Explorer 8 and 9 simply don’t support CORS well enough.  A more version sensitive approach might be something like this:

var basicQuery = function ()

{    

      var url = ‘https://<TARGETSERVER>/p?ReadViewEntries&outputformat=json';

      var method = ‘GET';

      var xhr = new XMLHttpRequest();

       

      if (“withCredentials” in xhr) {           

           // Most browsers.       

            xhr.open(method, url, true);

            xhr.setRequestHeader(“Access-Control-Allow-Origin”, “https://<ORIGINSERVER>)”);

            xhr.setRequestHeader(“Access-Control-Allow-Credentials”, “true”);

            xhr.setRequestHeader(“Access-Control-Allow-Methods”, “GET”);

      } else if (typeof XDomainRequest != “undefined”) {

                 // IE8 & IE9      

           xhr = new XDomainRequest();

           xhr.open(method, url);

      } else {

           // CORS not supported.

           xhr = null;

      }

 

      if (xhr != null)

      {                

            xhr.followsRedirect=false;          

            xhr.withCredentials=true;

 

            xhr.onreadystatechange  = function() {

                  if(xhr.readyState==4)

                  {

                        if(xhr.status == 200)

                        {

                              alert(xhr.responseText);                 

                        }

                        else

                        {

                              alert(“error”);

                        }

                  }

            }

            xhr.send();

      }

}

 

Finally, let’s see that gratifying successful CORS request:

success

 

Some notes about setting headers

  • CORS header values in a request must be supported on the resource server for a given request to succeed, i.e. for a request which specifies Access-Control-Allow-Credentials, the target server must respond with Access-Control-Allow-Credentials listed in the Access-Control-Allow-Headers. 
  • The same applies to methods, e.g. a GET method must be listed in the response Access-Control-Allow-Methods
  • When you set the Access-Control-Allow-Origin value, it should be the name of the origin server (the origin of the request) it really shouldn’t be asterisk (*) which is often used in sample code.  The target resource server also must recognise/honour the origin.

Telstra BigPond 4G – Bypassing Device Registration

image

If you use a Telstra 4G Mobile Broadband dongle, you might occasionally have issues with it, as I have found out from time to time.  Something seems to corrupt the local settings forcing a complete uninstall and reinstall of the device driver(s) and Mobile Broadband Manager software.

When re-installing, when the Mobile Broadband Manager (MBM) loads initially after the installation it prompts you to register the device.  My experience this morning was that the registration screen kept loading “beyond” the page which required you to enter your BigPond credentials, i.e. the fields were rendered and then you were wept to a “Please wait” screen which times out eventually.

I was rather ticked off (to say the least), and tried a bunch of ways around it, including patching Internet Explorer.  Nothing worked, and I couldn’t register my device or access the Internet.  Then, on a separate (Internet accessible) computer, I spied the following on Whirlpool:

“Well for future reference if you hold down shift when opening the options it’ll give you a few extra ‘things’ one of which is a check box that tells it the device has already been registered. Just tick that, save and away you go.”

Brilliant – and it works!  Fire up MTM and as you click to open the options menu, hold down the shift key.  On this dialog, go to the Diagnosis tab and check the “Signup has been completed for this device” option:

image

Make sure you enter your username and password on the Account tab if you haven’t already done so.  Now MTM will connect and not insist that you register the device first.  Problem solved.

By the looks of this tab, there’s some excellent additional options too. I wouldn’t advise that you mess with any of the other settings unless you’re sure you know what you’re doing!