Germany vs the world

Here’s something I did back in March, but never shared or tweeted a link to.

Jorge Camoes, the legend behind ExcelCharts.com tweeted this an hour before I was about to leave for a vacation back in March. I was all packed, and twiddling my thumbs, waiting for my ride.

This was a cool chart and Jorge had made the data available. In an attempt to show how quickly something that can be built in Tableau I wanted to see if I could do this in the time before I left for my vacation. Below is the result. Click it for the interactive version.

I’m a big fan of connected scatterplots so this was a fun challenge. In order to compare Germany to the rest of the world, I duplicated the data source and blended the duplicate with the original. That’s just another example of one of the amazing things you can do with blending, beyond just joining two different datasets together.

The above is what I managed in about 50 minutes. There’s a couple of advanced tricks in the workbook. What advantage do I have over Jorge? Practice. None of the tricks in the workbook are “hard” – they are things anyone can learn. In that one hour I didn’t manage to turn it into a trellis chart, but given more time I would have used the work in a previous blog post to do that.

A tribute to the Economist Graphic Department

I have always been a fan of The Ecomonist and the amazing charts they create. Their Graphic Detail blog is a must-read. They are simple and have a visual style that is consistent week after week. This is good because each time I view an Economist chart, I don’t have to waste energy working out the layout and style. Instead, I can focus on the content.

This got me thinking: can Tableau create Economist-style charts?

Why yes.

I chose a recent chart from the Graphic Detail blog. It charts the changing speed and size of the Tour de France. It’s a great little chart. I downloaded the data from Bike Race Info and got working. The results are at the top.

Go check out the original post. How close do you think I got? I am pretty pleased with how close it is possible to get. And of course, with Tableau-interactivity, it is easy to enhance the chart. I added some nice tooltips and a URL action to go visit the Wikipedia page for each year of the race. It would be easy to go further with this and add more views to a rich-analytical dashboard.

What other publications have a consistent style that we could try and emulate in Tableau?

Which is which?
Which is which?

Making the Kraken. Part 2: In Tableau

kraken finished

This is the second part of the story about how I made the Kraken, the winning entry in Tableau’s internal art competition. In part 1, we got the tools to make circle charts, and a fantastic Kraken image. In this part, we’ll look at how I took the data and made the final image. We’re going to learn about the internals of a Tableau file (XML), how to hack those internals, and some of the limitations to the way Tableau works.

First tries

Here’s the data, alongside the final image I used. I must have made 15 or 20 different configurations of the kraken image before I was happy.

rawdata
My packed circle Kraken data source

From there, you might imagine it was simply a case of connecting and creating a scatter plot in Tableau, right? Wrong. First off I had to create a calculation to calculate the area of the circle. Below is the first sight of my data:

FirstTry
The first look at the data in Tableau

Not bad, eh? But far from what I wanted. There are many problems to solve.

Colour

I have colour in my data source and I need to encode each circle according to the colour value in the data. This was my first sinking moment – Tableau can’t do that. But I do have something of a reputation as someone who’s not afraid to break open a TWB file and edit the XML directly. It was time to crack open the text editors.

Hacking the TWB files is fun but do it at your own risk! Tableau don’t specify the format of the TWB files, so you are on your own when attempting to hack it directly. That said, the structure’s pretty easy to work for most things. In this case, I created a file that set some custom Colours on a dimension, saved it, and opened it in a text editor. Here’s what I saw:

XMLHackColour
How Tableau encodes custom colours with XML

Every colour can be encoded with a simple XML pattern. Excel is my friend, along with some data and the ever trusty CONCATENATE function:

XMLMaker
Using Excel to quickly encode 5,000+ colours

That XML column can be copied directly into the TWB file. Save it, open it up in Tableau and now put Colour onto the Colour shelf. Progress:

SecondTry
The Kraken takes shape!

Circles.

Take a look at the circles above. They don’t scale quite as you’d expect. Getting this bit right was the hardest and made me realise there’s something unusual going on when you do circles on a scatterplot. Surely, if I’ve calculated Area based on [Radius], then Tableau should size the circle according to that value?

Well, not quite. It’s also influenced by the slider:

SizeSlider

And the size ranges dialog:

SizeDialog

And the available space on the screen.

In fact, getting this bit right was the most manual process. In the end, I had to put the view onto a fixed sized dashboard, and tinker until it felt right. One thing I discovered is that it’s simply not possible to get Tableau to scale the circles “accurately” by the area calculation. My image was never going to have no overlaps in the way the Percolator output does. But you know what? Tableau’s wacky drawing algorithms was creating a different kind of aesthetic that I found equally appealing.

Finally, I am getting really close. My Kraken is looking pretty good:

AlmostThere
Almost there – just sorting the circles to go

Finishing touches: the z-order of the circles.

What’s wrong in the picture above? Well, the eye’s aren’t visible, and there’re lots of blue circles overlayed on top of the Kraken’s tentacles and the submarine. Tableau doesn’t will draw circles in a scatterplot in the order that they are sorted. But here I have 5,000 of the damn things. I’m not going to battle Tableau’s manual sort dialog box:

SortOrder

That way lies madness! Instead, it was back to the XML editor. I selected the circles I wanted bought forward, exported the data, and created some XML to encode the sort order inside the TWB file. It was a similar process to the colour encoding, outlined above.

It’s done! Kraken image ready for entry

At some stage, I knew I’d done enough sorting of the circles. Maybe it was when the clock struck 3am. And I was done. I’d gone from a PC Viz Challenge, and grown the seed of an idea into a fully matured final picture. I was really pleased when I sent the entry for our internal art competition. And overjoyed to hear I won!

This process was great fun. What should you take away from it? Doing something like this will teach you more about Tableau. Knowing how it works under the hood, or discovering how packed circles behave will help you make better pictures. Hunting around the web will also inspire you to make new contacts and discover new ways of seeing data.

I hope you’ve been inspired by these posts – if you’re going to make some art out of Tableau, let me know on twitter (@acotgreave)

 

Making the Kraken. Part 1: Getting the image and data

kraken finished

This is a story of how a small seed in my head became the winning entry in Tableau’s internal “Make some are for our walls” competition. The creation of the Kraken entry was one of those experiences where I was driven by an idea. Along the way, I made new friends, discovered new stuff about Tableau, and was helped by serendipitous Google searches.

The seed

At Tableau, on our Product Consultant mailing alias, people occasionally mail out a “PC Challenge.” It will be some sort of edge case that’s been encountered and solved by the sender. The solution comes later, once people have engaged. I sowed my own seed for the Kraken with my “Pies on Scatter Plots” challenge. Can you put a pie chart on a scatter plot? Don’t worry about the question of whether you ever should put a pie on a scatter plot!

scatter pies

Feel free to work out how to do this. If not, just go download the workbook. It involves Table Calcs on the X/Y axis or blending. Why we can do pies on maps but not scatterplots is one of those great Tableau mysteries.

I sent the mail and the thread was a great one. Lots of people attempted the challenge. Some got it right, and eventually it descended into chaos. For example, this was one solution:

ReuterScatter

It was Aaron Clancy’s Cut and Paste hack job that was the seed for this whole project. He sent this to the email group:

VanGoghPies

Wow! That’s Starry Night by Van Gogh encoded by into pie charts.

And so the seed was sown. “I will make that chart” was now in my mental to-do list.

The germination

That seed lay dormant until watered by an email a couple of weeks later. “Hey Tableau, we have new offices with empty walls. They need art on them.” That was all I needed to get going.

Calling all viz-cassos

The hunt

But how would I begin? Well – Google the Van Gogh circles image in order find its source. This led me to Quasimondo’s page on Flickr. This was the origin. I sent Quasimondo an email and got a vague promise to share the data, but, alas, this never materialised.

Undeterred, I dug around to find the community of people who were into this packed circle drawing stuff. All roads led to flickr user TinRocket. And then to his site, TinRocket.com, run by John Baliestrieri. You need to go visit his site. He does better image processing than anyone or anything I have ever seen. His instant comic book effects are breathtaking. But I was after his packed circle data. This treasure lay in Percolator, John’s amazing iPad image processing app.

A percolated image

I paid up and downloaded the app, ready to get data and make a viz.

But, oh no: John’s app didn’t allow me to export the data. I was stumped. By now, though, I was driven. This project was going to work, come what may. I contacted John via flickr, and within an hour, I’m Skyping him in New York (I’m in the UK). We had a chat, and John’s enthused. So much so, he offers to build me a custom version of Percolator that will export the data. What? John B, contacted by some random geek in the UK, enjoys an idea so much he’ll dedicate time to creating a one-off version of the app just to allow me to complete this project? Amazing.

Three days later, I have Andy’s Special Percolator exclusively made for my iPad. The image in and, on my iPad, the data comes out. But which images? I processed Van Gogh’s starry night, plus a whole bunch of others. These included Rothko’s Red On Maroon (incidentally, I proposed to my wife in front of that painting), some Tableau-related photos, and photos of my village’s windmill. Van Goghs’s image would give me the kudos I sought in the PC Challenge, but none of these were good enough to win the art competition (by now I knew I was onto something cool, and I wanted a great entry for the competition).

What image would engage my colleagues? Why, a kraken, of course. Our most recent version of Tableau, version 8, had the internal code name Kraken. Off to Google again to find an image I could use. Go and Google Kraken images – lots of amazing, fantastical pictures reveal themselves. But they weren’t appropriate. I scrolled through page after page before finding this:

Perfect! And now you need to take a break to visit Macula.TV. It turns out Chris Bonnette is a great artist who loves to draw illustrations of mythical monsters. But I can’t just steal someone’s image for the competition.I mailed Chris and he was generous enough to let me use the Kraken for our competition. I am very grateful to him for this.

That’s the end of part one. I have a tool for making data. And I have some great images, including the kraken. The next part of the story is all about Tableau and how to draw 5,000 circles in a pleasing way. Click here to read part 2.

Wrapped bars: a Tableau implementation

Click here to see the interactive version of the wrapped bar chart.

wrapped with colour

Click here to see the interactive version of the wrapped bar chart.

When I saw the wrapped bars paper, I thought it would be good to see if Tableau could cope with these. Below is my best attempt. I must say at the outset that the implementation is complex and probably too complex to be considered a practical implementation of wrapped bars. My motivation was just to see how far I could push Tableau.

I have also posted my thought on the Wrapped Bars in a separate post.

I initially figured I could reuse the principles I used to make Panel Charts a couple of years back. This requires use of the INDEX() calculation along with some modulus arithmetic. It didn’t take long to implement that, but there’s a big problem:

wrapped bars using columns

What’s the problem? Tableau’s columns are all the same size. You can’t dynamically set the size of each column differently.

That had me stumped and, pretty much defeated. So I went for a walk: that’s where I do my best thinking. A solution came to me: Gantt charts. Using Gantt charts I could position each bar anywhere along an x axis. But at a price: huge calculated field complexity! In principle I needed to calculate three things (see the figure below)

wrapped bar calcs

1. The number of rows in each column

This is pretty straightforward: Size of each column

2. The length of the first (the biggest) bar in each column

Max in this column

3. The position on the x-axis to start each column

This one was a stinker!

x

Once the calcs were built, making the chart itself was a breeze.

Is this practical? 

I’ll let you decide that – let me know what you think in the comments or on twitter (@acotgreave)

 

Stephen Few’s wrapped bars: thoughts

Stephen Few has this week come up with another innovative new ways of visualising data: wrapped bars. Check out his article for an explanation of what they are and why they are useful.In this post, I want to discuss my thoughts on them. I have also implemented wrapped bars in Tableau: that’s a separate post you can view here.

Let me know what you think in the comments, or on twitter (@acotgreave).

1. The fear of the new

Stephen has created wrapped bars to fix problems in treemaps. I confess I initially looked at wrapped bars and disliked them. But was this the fear of the new? They clearly fix a problem in treemaps, but I initially thought, “Well, treemaps are good enough, why do we need something else to fix a smallish problem with them?”

But what if Ben Schneiderman had invented Wrapped Bars, not treemaps, back in the 1990s? We’d be comfortable with those, by now, I suspect. And then imagine Stephen Few comes along and fixes a problem with Wrapped Bars by giving us the treemap. Would my reaction still be that the new thing was inferior? I suspect yes. This is something one should always be mindful of when assessing new visualisation types.

With that in mind. what is the sweet spot that wrapped bars fix? In essence it’s finding the exact relative magnitude of dimension members at lower ranks. For example, what is the rank of the red mark in the treemap below?

treemap with colour

It’s very hard to answer that. Now check out a Wrapped Bar version with 60 rows in each column (incidentally, you can change the rows/column count in my interactive version)

wrapped bar with colour

It’s much easier, isn’t it? I am interested to know how big an issue this is for users of visual analysis tools. I don’t think I have ever felt a need for a middle ground between the treemap and bar chart. Switching between those two has sufficed for me so far.

2. Moire pattern

The bars on the first few pages of his article create a  moire pattern that disturbs my eyes as I scan the graphic. I would recommend using lollipops for wrapped bars: they reduce the amount of ink while preserving the link from the dot back to the axis origin. I created the lollipop version in Tableau, too, which you can see in my “How-to” blog post. Incidentally, lollipops should only be used when the scale begins at zero. If you are showing another value, then the line should extend right across the plot area.

wrapped  with colour (lollipop)

 

3. Negative values?

Stephen addresses negative values and draws an example where a small number of values are negative. This works well enough and is an improvement on treemaps. It would be an interesting exercise to see how well they work if, say, 50% of values are negative. I tried it in Tableau but my implementation did not work with negative values.

5. My conclusions

The Wrapped Bar has definite advantages over existing visualisations, but I fear that whatever these are, this visualisation will be slow to be adopted. Slower than bullet charts, at any rate, I think this is because the problem being solved is not as pressing as Stephen’s most successful innovation, the excellent bullet chart. If it was a chart type that could be switched to and from very quickly, I expect there would be more success. There are times during a visual analytic cycle that you do want to understand the lower ranked members of a dimension and I believe the wrapped bar might have more value there. Click – bar chart – click – wrapped bar – click -treemap: that kind of cycle would visually answer different questions very quickly.

What do you think? I am always very wary of my first reactions to new visualisation types. People are naturally resistant to change. Are my concerns part of a natural fear of the new, or are they correct? Let me know below or on twitter (@acotgreave).

And don’t forget – if you want to see how you can do this in Tableau, click here.