Effectiveness of Chart Design: See for Yourself

by Neal Levene on Wednesday, August 19, 2009 · 16 comments

in Visualization

The original posting had pagination issues that created “404 errors”. I had to change the pagination method. Sorry about any issues. Neal

Let’s try an experiment together. . .

generic pie chart Effectiveness of Chart Design: See for Yourself

Can you rank the size of the wedges from largest to smallest? Take a second to write down your answer. (We will reveal the answer later in the article.) When you are finished, click to the next page of this post.

Click below for page 2.

[spoiler]
generic dot plot Effectiveness of Chart Design: See for Yourself

Here is a different graph. Can you rank the sizevalues of the data points from largest to smallest? Take a second to write down your answer. (We will reveal the answer later in the article.) When you are finished, click to the next page of this post.

[/spoiler]

Click below for page 3.

[spoiler]
generic pie chart 300x180 Effectiveness of Chart Design: See for Yourselfgeneric dot plot 300x182 Effectiveness of Chart Design: See for Yourself

These two graphs display the same data, located below in the table. The correct ranking is: A, C, D, B, E, F (or A, C, D, B, F, E, as E and F are the same size).

generic table Effectiveness of Chart Design: See for Yourself

Which was the easiest with which to rank the data points?

Most people will answer, the table or the dot plot was easiest. Most people probably had some difficulty ranking the wedges in the pie chart. For most people, it is actually frustrating. The human vision system is not particularly good at that exercise. Even those of you who had no trouble ranking the wedges in the pie chart will probably feel that it was way easier to rank the points with the dot plot (and very easy with the table).

This is a very simple demonstration of the problems pie charts have in delivering information. How did you do? I welcome you to comment below.
[/spoiler]

Social Media Links:
  • RSS
  • email
  • Twitter
  • Facebook
  • del.icio.us
  • StumbleUpon
  • Digg
  • Google Bookmarks
  • Technorati
  • Sphinn
  • SphereIt
  • Mixx
  • LinkedIn
  • Reddit
  • Live
  • Netvibes
  • Yahoo! Bookmarks
  • Yahoo! Buzz
  • PDF
  • Print

We welcome your feedback. Please leave us a comment below. If you haven't already, there is no time like the present to subscribe to the RSS feed.

Category and Tags

This post filed in the following categories:

  • Visualization - Visualization is any technique for creating images, diagrams, or animations to communicate a message.

About the Author

This post was written by Neal Levene, CEO of InnovaTech, Inc., who blogs about data and business issues here at Simple Complexity and about a variety of other topics at NealLevene.com. Find Neal on LinkedIn or follow him on Twitter. Neal is available to speak to your organization on a variety of topics. You may also use Simple Complexity's Contact Form.

Comments

{ 1 trackback }

The Big Apple Pie Chart — InnovaTech, Inc.'s Simple Complexity
Thursday, August 20, 2009 at 5:28 am

{ 15 comments… read them below or add one }

1 Tentotwo Wednesday, August 19, 2009 at 5:52 am

Hey there,

the links to pages two and three of the article lead to a “404 – page not found” error for me… Which, in this case, only adds to the excitement (are A and C identical in size?). You might still want to look into fixing the error, I believe.

While I’m at it: Very nice blog, I have learnt a lot from you guys. Keep it up!

Reply

2 Neal Levene Wednesday, August 19, 2009 at 6:31 am

Grin – WOW – I didn’t mean to create a cliff hanger. For some reason, normal pagination is resulting in a 404 error. Sigh.

I used a different method to expose the two other pages.

I’m very sorry for any problems.

Reply

3 Tentotwo Wednesday, August 19, 2009 at 8:02 am

You’re right, the scatter plot and table are much easier to extract quantitative data from, although I found that looking at the scatter plot without an annotated axis made me feel a bit uncomfortable…

Edward Tufte said in one of his books (I believe it was ‘The Visual Display of Quantitative Information’) that there is no situation in which a pie chart should be preferred to a bar chart or a table.

Reply

4 Neal Levene Wednesday, August 19, 2009 at 9:56 am

The main reason that the annotated axis was excluded was to make the comparisons between charts “more fair”. Even without the axis labels (which really are mandatory), the dot chart is still easier from which to extract information. All of this said, the pie chart is one of the most common visual displays used to describe parts-to-whole relationships, regardless of its ineffectiveness.

Thanks so much for the comment.

Reply

5 Chris Wednesday, August 19, 2009 at 10:53 am

I think you cheated slightly by sorting the table. But even without the sorting the table makes it easier to see the values. Only with many more data points would the plot make it easier to find the extreme values.

Reply

6 DivinoAG Wednesday, August 19, 2009 at 11:25 am

I have read before how pie charts are not a recommended method for displaying information because of the difficulty it gives the viewer to figure exact quantitative values from it, but from a graphic design standpoint, I believe this can be a limited point-of-view because it fails to acknowledge the pie chart strength: provide instant size discrepancy reference for large values.

I agree that if you are to use a pie chart to compare more than 4 or 5 elements with values all over the spectrum, the information presented carries little to no value to the viewer since the precise value will probably be, as it were on your example, critical for the comprehension of the graph and this is not something the pie chart displays very easily. I imagine a chart displaying the Internet usage for the top 10 countries would be such a case.

But there are cases were the precise value is not the most important information to read on the graph, but the discrepancy between the larger and smaller values (or lack of it), and on those cases the pie chart is not only acceptable but probably better than all the alternatives.

Take the example of an article talking about the strength of Microsoft on the OS market, featuring a chart that displays the top 5 Operating Systems. The objective here is not to see the exact numbers of each part, but to visualize how bigger one piece is than the others. Being that an article specifically about Windows, the difference between the remaining slices is irrelevant for the chart’s main purpose.

The matter of being harder to identify the relative sizes when the graph is somewhat scrambled is also in my view unimportant, since proper charts would be drawn sorted on descending order clockwise. The system is not self-correcting, but bad technique should not be considered as an inherent fault of the chart type.

In conclusion, my understanding is that pie charts can be useful, but only in cases where the numerical value of each slice is not absolutely relevant to the comprehension of the chart (a good pie chart should be understandable even if devoid of precise accompanying data), and can be used to display either very large or very small discrepancies within specific contexts.

Reply

7 Tentotwo Wednesday, August 19, 2009 at 11:49 am

You’re certainly right that the point of pie charts is not to extract numerical data, but to compare relative amounts. However, if you were to use a simple stacked bar chart, you would have the visual comparison a pie chart offers, but in addition, you can put it next to other bar charts to allow for a direct comparison to other data sets, and you can much more easily compare the absolute amount by making the bar chart taller. With pie charts, you get that length vs. area problem that is so frequently encountered in infographics.

Reply

8 Tentotwo Wednesday, August 19, 2009 at 11:52 am

Oh, and yes, you would probably order the slices in descending order clockwise from the top. But again, what if you wanted to compare the market shares between two years, and the order has changed? Do you stick with the original order, to allow the reader to find the corresponding share more quickly, or do you reorder them so you’ve got the descending order again?

Reply

9 DivinoAG Wednesday, August 19, 2009 at 12:19 pm

You’re absolutely right. In such cases that require you to compare multiple datasets, pie charts are not adequate at all, specially since the round format is not visually appropriate for side-by-side positioning.

But for percentage-based single datasets with low amount of elements, I still think you can use pie charts when you want to emphasize the discrepancy between the values.

The problem I see with using stacked bar charts for this is that having one element between others can make this comparison not as obvious as having all elements have at least one contiguous point to visually connect them. It’s a graphic design concern (which is my area of interest), not as much a information design problem; the more visual elements you have between two determined objects, the harder it is to visually compare them. Pie charts are used exactly because they are very, very easy to read, even if that means they can lose legibility equally as easy.

So I do realize the weaknesses pie charts present, but I find that if used with those in mind, one can present information very clearly and directly on specific cases, and switch to other chart types when more detailed data is necessary. I just don’t like seeing them being cast as “useless” because most people don’t know how or when to use them. :)

Reply

10 Tentotwo Wednesday, August 19, 2009 at 12:26 pm

Hehe, okay, point taken. You’re right, having at least one common point between all areas would definitely be the advantage pie charts have. I did not think about that.

I was a bit skeptical of the “you never need pie charts” point made by Tufte, but then again, I also really liked that he has such a strong opinion on the matter, and didn’t resort to the usual “it depends on the context and you can’t generalise things like this” that you always get in textbooks.

Reply

11 Neal Levene Wednesday, August 19, 2009 at 1:07 pm

The conversation is great. I’ve come around to the opinion that if your goal is communication of information, there is never a time when pie charts are as effective as an alternate method.

If your goals are something other than effective communication of information, pie charts can have an ornamental or aesthetic value.

I disagree with some of the things that Divino said above. Although, I have to admit I was hoping that someone would make those exact points.

The statement I disagree with most is:

Pie charts are used exactly because they are very, very easy to read, even if that means they can lose legibility equally as easy.

The study data actually conflicts with the fact that pie charts are easy to read or that they even do a good job showing percentage based differences. The study data shows that most people have trouble either estimating the amount of volume in a wedge or the differences in the angles at the center of the pie. The first comment above from TentoTwo stated that wedge A and wedge C were the same size. A 4% difference in size is, in my mind, usually meaningful. If showing clearly the differences in your data set is not important, why present a visualization at all? Outside of artistic reasons, I can’t think of one.

Frequently, pie charts are marked with legends that require your eyes to go back and forth to figure out what is what. When the pie wedges are actually labeled, you are using the text and not the graphic for communication of the information.

While I was not an original believer and felt that saying pie charts have little to no communicative value was overstated, the weight of evidence that they are an ineffective communication model is just overwhelming.

I feel the studies have demonstrated that from a communicative perspective there is always a better medium. Of course, on the front of a USA Today, to catch a reader’s eye or as an unimportant margin decoration in an annual report, they may have some aesthetic value. People definitely like them. That doesn’t mean they are effective.

Reply

12 Tentotwo Wednesday, August 19, 2009 at 2:26 pm

Okay, I’m a bit torn here. I think Divino made a point that cannot simply be dismissed by stating that in the overwhelming majority of applications, other forms of visualisations are the better choice. While looking for studies to shed some scientific light on this discussion, I first came across a document called “Save the Pies for Dessert” by Stephen Few of Perceptual Intelligence (). The title already gives away the conclusions of the (highly enjoyable) paper: The author agrees with the common opinion that pie charts should be avoided. However, on page 13, he refers to a study titled “Displaying Proportions and Percentages” (the link points to the article on Wiley Interscience, in case you have access to that through a university or subscription) by Ian Spence of the University of Toronto, in which it has been found that pie charts can be superior for one task: Comparing the compound sizes of several slices. In our case: Which is greater, A+B or C+D, for example. The pictures shown in the “Dessert” paper only compare pie charts to non-stacked bar charts and tables; in the original study, (horizontal) stacked bar charts are also included in the experiments. That’s an extremely interesting result, I believe.
In the example Divino suggested (comparing the market share of different operating systems) this could be a case where it is indeed better to use a pie chart than a (stacked) bar chart, as the data for the different OS could be subdivided according to the market share of different Windows or Linux versions, while still maintaining the comparability of All Windows vs. All Linux.

Reply

13 DivinoAG Wednesday, August 19, 2009 at 2:31 pm

Neal, I think you are welding two different concepts together, which is why you don’t agree with that statement. You’ll forgive me if my terminology might be unclear, english is not my mother language and sometimes some advanced terms escape me (that’s why I compensate with long explanations). There are probably better words to describe what I mean. :)

One concept is that of “ease to read”. What I understand (and mean) by that is: a chart is easy to read if it requires no explanation to the meaning of the data. A pie chart displays a closed, subdivided shape that can be easily understood as parts of a whole, as percentages.

A stacked bar chart on the other hand is too similar to linear bar charts, which are limited to the scale determined numerically on the graph, instead of a percentual. You cannot visually determine what type of graph it is until you check for additional data, which makes this type of chart harder to read, almost meaningless without this extra reference material.

The other concept is of “tool for precise data analysis”, and here I absolutely agree with you. Pie charts are not adequate for evaluating fine or even medium level data, but they are good for demonstrating high contrast data, because they are highly visual.

On the other hand, I would argue that other types of graphs are also not good for visually understanding fine data; try to display your example dataset from the article as a stacked bar and you’ll realize that is just as hard to read as the pie chart, or to clearly differentiate anything below that 5% margin. What makes those charts better is their ease to be complemented with additional data such as scales; pie charts offer no clear way to add extra information to help clear up its meaning.

Sometimes people ask me why on this day and age I keep using a analog wrist watch (everybody I know seem to prefer using their cellphones to check the time). While my answer usually depends on who’s asking, the real answer is “pie charts”.

I don’t think anyone ever checks the watch to know what time it is right now (unless you need to fill that on a form, there’s hardly any use for that information), but to see how much time they have until something else happens. Because of that, using a digital watch is a pain for me because I need to mentally calculate the difference between the current time and the time I actually am interested, and that only gives me another number which I find harder to quantify than having an image.

But with a analog watch I can look at the current hand position, the number for whatever time I’m waiting for, and create a mental image that is nothing but a pie chart, which I can then, almost instantly, understand as “a lot” or “a little” time.

That’s how pie charts are useful: they provide very clear contrast between few entities (two in this case, “time left” and “other time”), and allow the viewer to gather instant information when granular detail is not needed. It’s usually not important if I have 10 or 11 minutes to lunch time, only if it’s 10 or 30 minutes.

I think that no chart type is useless. But none of them should never be seeing as the “best way to present this particular data”. The best, most precise way to view any dataset is with a table. Anything else is a controlled simplification. If I can simplify the data enough to present it as a pie chart and still retain the message I’m trying to convey, why wouldn’t I?

Reply

14 Neal Levene Wednesday, August 19, 2009 at 2:53 pm

Your English is amazing. I cannot believe it is your second language.

I could not engage in this kind of discussion in any of the second languages I know. Your response contains a lot of interesting points. I’d like to discuss your points as part of a fuller discussion on graph effectiveness in a later post, probably Friday.

BTW: Read tomorrow for the backstory on the data contained in today’s post. It came from an actual publication.

Reply

15 Tentotwo Wednesday, August 19, 2009 at 2:34 pm

Oh, to finish the point I was making at the beginning of my last post, about me “being a bit torn”: I still believe that the advice “If you want to display data and you can choose between a pie chart and something else, something else is better” is a pretty safe bet, however this discussion has softened my position somewhat. As Divino said, if you keep the weaknesses of pie charts in mind, there might be some (very) specific applications where pie charts make sense. However, I also agree with Neal: Almost all pie charts you see in day to day publications were most likely not used because they truly are the best visualisation tool for the task at hand, but out of habit and for their visual appeal.

Reply

Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Previous post:

Next post:

WordPress Admin