Jedv

Learning D3

This week I have been getting up to speed on one of the most powerful ways to visualize any datasets in any way you could imagine on the browser – using D3.

D3 describes itself as

“D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG, and CSS.”

https://d3js.org/ – Homepage

This might seem on the face of it just another Javascript jQuery style graphing library like highcharts, Chart.js etc. I thought this too but this couldn’t be further from the truth. D3 focuses on pure data driven development, leaving you to create the entire visual side from scratch using the power of SVG.

This allows you to create graphs from the very ground up, giving unlimited customisation options and never leaving you trying to work around the limitations of the library. There is a steep learning curve if like myself, you haven’t done much with SVG before. I recommend getting cheatsheets for both the library & SVG and get as in-depth material to learn from that you can find/afford.

There is multiple sites including the original D3 site and bl.ocks.org that offer hundreds and hundreds of detailed & amazing examples of what you can accomplish if you just start with D3 & the browser, it doesn’t even have any dependencies whatsoever – allowing you to get straight into the fight.

Data sourced from: http://data.un.org/

This is a chart I created using just HTML, CSS, SVG & D3 in only a modest amount of lines. There is a tooltip highlight that shows the actual figures of the data point you hover.

Tooltip highlight on scatter points in D3

Here is the code from the app.js file, containing all the D3 for plotting & interaction.

document.addEventListener('DOMContentLoaded', function(){ // Just make sure DOM is ready.
    // Padding, width & height for SVG element - used to calculate everything else.
    var padding = 40;
    var width = 700;
    var height = 700;

    // Setup scales for X, Y position, colour & radius of points.
    var yScale = d3.scaleLinear()
                  .domain(d3.extent(regionData, d => d.subscribersPer100))
                  .range([height - padding, padding]);

    var xScale = d3.scaleLinear()
                  .domain(d3.extent(regionData, d => d.medianAge))
                  .range([padding, width - padding]);

    var colourScale = d3.scaleLinear()
                  .domain(d3.extent(regionData, d => d.medianAge))
                  .range(['lightgreen', 'black']);

    var radiusScale = d3.scaleLinear()
                  .domain(d3.extent(regionData, d => d.growthRate))
                  .range([2, 20]);

    // Register the axis & individual ticks for the grid.
    var xAxis = d3.axisBottom(xScale)
                  .tickSize(-height + 2 * padding)
                  .tickSizeOuter(0);

    var yAxis = d3.axisLeft(yScale)
                  .tickSize(-width + 2 * padding)
                  .tickSizeOuter(0);

    // Draw each axis.
    d3.select('svg')
      .append('g')
      .attr('transform', 'translate(0, '+ (height - padding)+')')
      .call(xAxis);

    d3.select('svg')
      .append('g')
      .attr('transform', 'translate('+padding+', 0)')
      .call(yAxis);

    // Plotting the data.
    d3.select('svg')
        .attr('width', width)
        .attr('height', height)
      .selectAll('circle')
      .data(regionData)
      .enter()
      .append('circle')
        .attr('cx', d => xScale(d.medianAge))
        .attr('cy', d => yScale(d.subscribersPer100))
        .attr('fill', d => colourScale(d.medianAge))
        .attr('r', d => radiusScale(d.growthRate))
        .attr('stroke', 'black');

    // Axis labels & Title
    d3.select('svg')
      .append('text')
        .attr('x', width / 2)
        .attr('y', height - padding)
        .attr('dy', '1.5em')
        .style('text-anchor', 'middle')
        .text('Median Age');

    d3.select('svg')
      .append('text')
      .attr('transform', 'rotate(-90)')
      .attr('x', - height / 2)
      .attr('y', padding)
      .attr('dy', '-1.5em')
      .style('text-anchor', 'middle')
      .text('Subscribers per 100');

    d3.select('svg')
      .append('text')
      .attr('x', width / 2)
      .attr('y', padding - 20)
      .attr('font-size', '1.5em')
      .style('text-anchor', 'middle')
      .text('Regional Statistical Data');

    // Transition hover & Tooltip for each plotted point.
    var circle = d3.selectAll("circle");
    circle.on('mouseover', function(d) {
        let r = d3.select(this).attr('r');
        d3.select(this).attr('r', r*1.1);

        let html  = '&nbsp;<strong>Region:</strong> ' + d.region + '<br />' +
                    '&nbsp;<strong>Median Age:</strong> ' + d.medianAge + '<br/>' +
                    '&nbsp;<strong>Subscribers per 100:</strong> ' + d.subscribersPer100 + '<br />' +
                    '&nbsp;<strong>Growth Rate:</strong> ' + d.growthRate + '<br />';

        tooltip.html(html)
            .style('left', (d3.event.pageX + 15) + 'px')
            .style('top', (d3.event.pageY - 28) + 'px')
          .transition()
            .duration(200)
            .style('opacity', .9)
    }).on('mouseout', function(d) {
        let r = d3.select(this).attr('r');
        d3.select(this).attr('r', r/1.1);
        tooltip.transition()
            .duration(300)
            .style('opacity', 0);
    });

    // Append tooltip div to body for use later.
    var tooltip = d3.select("body").append("div")
      .attr("class", "tooltip")
      .style("opacity", 0);

}, false);

As you can see, the level of detail on drawing the graph from the axis to the points and the text labelling them is all thought of and catered for by D3 – even if it can seem alien to someone who is used to a world of high-level Javascript plotting libraries.

If you want to get the full code for each file to run/modify this yourself, you can get it here: https://gist.github.com/Jedtek/1ab13979312cd3ed812ce23af59cb2b2

I am continuing on to learn the advanced D3 module of my current studying but I think its important people realise that although it can be challenging to get into advanced SVG drawing (opposed to generating automatically) that D3 is wholly worthwhile!

Jedv

Leaving behind ES5 for ES20xx – The (r)evolution of Javascript

The evolution of Javascript as a language is speeding up more than ever before. It’s debated humble beginnings on Netscape now hidden behind a formidable community spirit that seems to be growing out of the recent big changes in how Javascript is seen & used by different developers. Now harnessed for more uses than thought possible by the average developer just around 10 years ago*.

Lets just re-cap on a brief history of how this all came to be:

  • 2008 – ECMA specification 4 is due to be released, after already being worked on for years and causing many a disagreement between parties trying to have it swing more to their needs, it ends up a blog-based sparring match between Brendan Eich (Mozilla) & Chris Wilson (Microsoft). This is all based around the argument of incompatibility that would occur with the proposed block of changes among general open/closed rivalry.
  • 2009 – After many a specification revision, incompatibility argument and unhappy campers on all sides of the Javascript camp, ECMA TC39 publish ES5 in December; the final edition being agreed by all parties but as we all find out loosely interpreted.
  • 2015 – Just when you thought ECMA would never unite, quite like the factions in the film Braveheart – the release of ES2015 is finalised. It comes packed with new features and new ways of doing faster cleaner and more efficient code, especially for the class-based programming camp. (2015 Specification)

Now up to this point, the updates and periods between each had been drawn out and more about semantic cross-compatibility than advancing the language . ES2015 showed a more decisive specification, with features for all and a brighter future. All they had to do now was keep it up.

Meanwhile you & everyone else were crawling articles and whitepapers detailing the new way of getting by on the client or server-side of Javascript. The clean pretty syntax of arrow functions still fresh every time you view your newly ES2015 refactored files.

There is a new air to the whole world of Javascript, Node.js booming with popularity by this point is starting to become the over-packed npm nightmare we now deal with daily; reminiscent of the fun you could have breaking the rpm package management system in the mid 2000s.

The client side sector is now dominated by frameworks such as React & Angular, making sure the abstraction between different parts of the age of ‘single page apps’ is set out in stone. The goalposts for your front-end Javascript interview processes are now based upon who can master the most of these 3rd party UI libraries that pop up like whack-o-moles.

While you have been distracted with all the shining lights and bells & whistles of Javascript taking on the challenge to re-invent itself as hip and down with those class-based kids, ECMA were beginning to get the hang of a release cycle.

ES2016/2017 come out on the actual year they were supposed to! Not only that, they are focused and contain only well thought-out additions such as the [].includes, native promises with generators, async functions and the lovely await keyword. (2016 Specification) (2017 Specification)

ES2018 promises even more ways to make that code async-as-you-like while looking forward to a new, revolutionised world of Javascript. Not just that but additional rest/spread operators and additions to the long neglected RegExp. (2018 Specification)

This united front has also been taken on by browsers working for full compatibility more than ever before, you could say this is inevitable but anything could go the other way.

Thank fuck for Javascript.

* I am referring to 2009 & the creation of Node.js, I am aware however that server-side Javascript did exist before V8 really put it on the map.


Also my articles convey both technical fact AND opinions on technology, programming and software both closed & open source. Please keep this in mind! 🙂