Compost!

Recycling waste in the garden and on the internet

My blog is supposed to be about technical and management issues, but today I'm going to write about composting. There are obvious 'humorous' comparisons with the technical world, most obviously about recycling ideas and rotting content, but beyond the obvious, there are lessons about material on the internet.

How it works for me

I have what's called a tumbling composter. It has two chambers. The idea is you fill one chamber with material to compost and while that's decomposing, you fill the other chamber. Complete composting takes a few weeks in summertime, a little longer in spring and fall, and stops almost completely in winter. You're supposed to rotate the drum every few days to aerate the compost. Each chamber gives about a wheelbarrow load of compost and you get several loads per chamber per year.

(My garden compost tumbler.)

Garden waste: a waste of time

The first lesson I learned is that it's hard or impossible to compost garden waste. In principle, garden waste is ideal, but in practice, there's so much of it that it overwhelms the compost mix and stops the decomposition process. You need a mix of materials for successful composting and garden waste is just too much of one thing. 

Of course, the first thing I tried to compost was leaves and I learned they break down extremely slowly. A friend suggested I shred them first, but even then, the rotting process is slow. Leaves just aren't good for compost and you should dispose of them separately.

Sticks and branches decompose slowly too. If you're going to put woody material into the compost heap, you need to chop it up into small pieces first. Even then, they don't tend to rot completely.

If you look at the Amazon reviews of composting bins, you'll find multiple reviews from people who've stuffed their bins full of grass clippings, leaves, or other garden waste and they're complaining that it doesn't compost. They're publicly blaming the product instead of figuring out why they made a mistake. (First internet lesson: reviews and comments from people on the internet can be wrong and/or misinformed. The customer isn't always right, especially when they're writing reviews.) To make composters work, you have to mix your content.

Greens and browns: the golden ratio

Almost all composting websites talk about greens and browns and the correct ratio. Here's what they consider browns (the list varies from source to source):

• Dried grass clippings
• Woody plant material
• Pine needles
• Oats, grains, and feedstock
• Autumn leaves
• Oak leaves
• Sawdust
• Wood chips
• Straw and hay
• Uncooked pasta
• Shredded paper 

Here's what they consider greens (again the list varies):

• Grass clippings
• Coffee grounds/tea bags
• Vegetable and fruit scraps
• Trimmings from perennial and annual plants
• Annual weeds that haven't set seed
• Eggshells
• Animal manure
• Seaweed

The correct ratio is something like 3 brown to 1 green, but the ratio varies from site to site and I've even seen it stated as 1 to 1. I try to stick roughly to a 3 brown to 1 green ratio, but it's never exact.

Initially, I found my composter gave balls or clumps of material. This is a well-known problem with tumbling composters like mine and is caused by the mixture being too wet and/or an insufficient amount of brown material. If your mix is clumping into balls, add more shredded paper, but mix it in thoroughly.

I've visited lots of sites to find details of the mix and what I should do. Strikingly, the writers all made similar statements about greens and browns and the ratio, but they never backed their assertions with science and they never linked to other resources. After a while, I realized I was seeing the same content over and over again, and even though it wasn't an exact copy, it was so close it might as well have been. Many of the sites didn't read that well and contained a lot of repetition, which leads me to think they were being SEO'd to death, it also explains the lack of links; they want to keep people on the site. Overall, I visited a lot of low-quality sites that didn't say very much. There are a couple of internet lessons here:

• Wily marketers are out-smarting search engines and getting low-quality pages to score highly.
• Content is recycled from site to site with almost nothing informative added.
• Many sites with information on the home and garden are just junk sites with low-quality copied content.
• I'll still read the low-quality content because I'm looking for advice; the marketers' tactics are working.

Am I guilty of the same thing? I hope not. I'm trying to say something new, but then this is a hobby site and I'm not making any money out of it.

Paper and kitchen waste: a working combo

The thing that works wonders for me is kitchen waste coupled with shredded paper; this gives me the best compost and it decays quickly. There are some rules though.

• No meat or dairy. Rotting meat or dairy attracts animals. No one wants rats dining in their backyard. Don't do it.
• Rules for paper:
• Whole pages take ages to decompose so shred paper or tear it up into small strips. 
• Shredded paper from a shredder works well, but don't add it all at once as it tends to clump. 
• Don't include paper with bright metallic inks, waxed paper, or glossy or shiny paper. 
• Kitchen paper and similar paper will compost, but you have to tear it up into small strips.
• No pizza boxes with meat waste on them (it's the animal thing again).
• Cardboard will decompose well if you tear it up into small strips. It helps to soak it thoroughly first for several days. Adding too much cardboard can stop the decomposition process, so be careful about your mix.
• Coffee grounds and tea bags are great, but tear tea bags to speed decomposition.
• Add kitchen waste little and often rather than a lot at once. Chop up larger pieces (e.g. brocolli stems). Banana skins rot very quickly!

Blood and maggots

I used some kitchen paper to mop up blood from meat and threw the kitchen paper into the composter. A few days later, I saw maggots eating the blood-stained paper; but only the spots where the blood was. Gross, but fascinating. Maggots usually indicate you have animal products in your compost.

Starter mix

The composting process is mainly bacterial and the bacteria has to come from somewhere. To get started, I threw in several handfuls of soil from different parts of my garden. When I empty my composter, I don't remove all the compost, I leave some in so the decomposition process for the next load can get started.

I also added worms to my bins too. I hope they like the paper and cardboard I'm putting in. I don't want to be cruel, even to worms.

How much waste?

Once food and paper rot, it takes up a lot less space. I've found that a nearly full compost chamber has a lot more space after a week or two when the contents have decayed a bit. The lesson here is that even when a chamber looks full, if you leave it a while, you can fit more waste in.

Wasps and rats

I'd heard blood-curdling stories of wasps setting up home inside rotating compost bins. In practice, that didn't happen to me, maybe because I rotated the bins every few days during the warmer weather. I can see if you left the bins alone for a week or so, it might be an attractive place for wasps to nest, after all, it's warm and dry. The moral is: don't neglect your compost!

Because I don't compost any animal products, I've never had a rat or raccoon problem.

Winter is coming - even for the compost heap

I found that decomposition stops in winter. Once my chambers were full up in late November, that was it until March. The advice I read was not to rotate the drum once the weather gets cold, the idea is that rotating the drum causes the compost to lose heat; if you keep the drum still, decomposition can go on a bit longer. Of course, once winter really set in, the chamber contents froze solid and after a while, the sliding chamber cover froze in place so I couldn't view the chamber contents anyway. 

To keep my recycling going during winter, I filled up cardboard boxes with food and paper waste and waited for the spring to restart composting; of course, I composted the cardboard box too. Because I didn't throw out meat products, I didn't have any problems with animals.

The secret composting benefit: garbage reduction

I bought the composter to get rid of garden waste and found out that it wasn't good for that. What I found in practice was it was great for disposing of kitchen waste and paper. Using my composter, I've managed to reduce the amount of waste I throw out by several trash bags per year. Of course, I also get several wheelbarrow loads of compost per year. Overall, composting is both better for the planet and better for my garden. 

Sadly, I found that it wasn't just my composter that was full of recycled material, it turns out, that a lot of internet sites are too. Who knew.

All about pens

Handwriting is the new typing

After many years of terrible handwriting (think spiders on LSD), I recently decided to improve it. I bought a book on handwriting and practiced, practiced, practiced. Along the way, I learned something about the writing experience; the choice of pen and ink matters. I'm going to share what I learned with you.

This post is all about ball pens within a reasonable price range, fountain pens are just too advanced for me and I'm cheap.

What makes a good handwriting experience?

Early on, I discovered that the pen and ink you use make a big difference, not only to the quality of the result (legible handwriting) but also to the tactile pleasure of writing. I found the smoothness of the pen moving across the paper was important; some pens just glide across the page and are wonderful to use, while others skip and drag like taking a pet to the vet. Some otherwise great pens gave smooth and thick lines that bled through to the other side of the paper, while other pens gave precise narrowness at the expense of scratchiness. After some experimentation, I concluded that the thrill of the writing experience is governed by two things: the pen barrel and the refill. 

For the pen barrel, its weight and the feel of the pen in my hand were the most important factors. As we'll see later, the weight of pens varies by almost an order of magnitude and I had very different writing experiences at either end of the scale. After many trials, I found I like heavier pens. The feel of the pen is harder to describe; I like pens with some form of special 'grip' or finger guide, but my favorite pen is all metal and smooth (I'm obviously not consistent). In the picture below, only the Pilot G-2 (2nd from top) and the Zebra Sarasa (3rd from top) have guides.

(Muji 0.38mm, Pilot G-2 0.7mm, Zebra Sarasa 0.7mm, AliExpress 0.5mm)

Refills for ball pens are of two general types, ballpoint ink, and gel ink. Ballpoint ink is thicker and heavier but lasts longer, while gel ink is smoother on the paper but doesn't last as long. For a better writing experience, the choice for me is clear: gel ink. As a bonus, gel ink pens come in a rainbow of colors.

Gel ink refills (and pen refills in general) are like dogs, they come in a range of different sizes. There are international standards, but even within standards, the variation is great. The image below shows some refills which are all about the same length (110mm) and all about the same width (6mm). As you've probably guessed, some of these refills fit some pens and not others. Is there any way of knowing what size refill a pen will take? No. You just have to guess or buy the same refill that went into your pen.

The size of the ball on the refill is hugely important. Typically, gel refills have the following ball sizes:

• 1.0 mm bold
• 0.7 mm medium
• 0.5 mm fine
• 0.4 mm extra fine

The thicker the ball, the better the pen glides across the paper, but the cost is thicker lines which may lead to ink bleeding through to the other side of the paper. Thinner balls give more writing precision but can feel a bit scratchy and you have to be careful about the angle you use to write.

The other obvious factor to consider is the manufacturer. I tried M&G, Zebra, Muji, and Pilot. I found I liked the Muji 0.38mm refill for precision at the cost of a little scratchiness. Sadly, all of the Muji refills froze partway through and I couldn't revive them (see below). I ended up using the Zebra and M&J refills but I'll probably move to Zebra permanently soon (see below for why).

Frozen balls

A few times, I've had the experience where a new refill stops working partway through. There are two closely related symptoms: it just stops writing altogether or it only writes in one direction. I've tried cleaning the type with alcohol, putting the refill in hot water, and removing the nib and cleaning the insides with alcohol. Nothing worked. On the internet, I've heard stories of people using heat guns or using naked flames to heat the refill nib, however, I've also heard stories of refills exploding when people do this kind of thing, so perhaps it's not a good idea.

It's annoying, but typically refills cost around $1, so I just buy another refill and move on.

Different weights

I thought I liked heavier pens, but I wanted to be sure, and what better way for a nerd to be sure than weigh his pens? I weighed all my pens without their refills to avoid differences due to the refills themselves. Here are the results.

Pen	Weight
Muji Gel Ink Ball Point Pen	6g
Pilot G-2	8g
Zebra Sarasa	23g
AliExpress solid brass pen #1	42g
AliExpress solid brass pen #2	43g

There's a 7x weight difference between the Muji and the AliExpress pens. I knew the Muji was light, but I didn't think it was that light.

Interchangeable refills - or not

My favorite pen was my $2 solid brass AliExpress pen which takes M&G refills. M&G is a Chinese brand and unfortunately, it's recently become harder to get their refills in the US. I wondered if I could use the Zebra refills in my AliExpress pen. Sadly not. The M&G refills are slightly narrower than the Muji refill and have a slightly different end. These differences are small, less than 1mm, but pens are precision instruments, and when something won't fit, something won't fit. I couldn't find a non-M&G refill that fit, so when I finish my last M&G refill, my $2 brass pen becomes a $2 brass stick.

But all is not lost. I actually bought two seemingly identical brass pens from AliExpress a few weeks apart. It turns out, the second one is ever so slightly different. Different enough that the Zebra refill fits. 

I'm lost

Before the pandemic, I mislaid my $2 (actually $1.99) AliExpress brass pen at work. The office manager asked me what I was looking for and I told her "My one ninety-nine pen". She dropped everything to help me find it, which we did after a thorough search. Once we'd found it, she said it didn't look expensive and I said it was $1.99, not $199. She gave me a look that said "you're an idiot" and of course, she was right.

Propaganda and public relations

Different name, same thing

I've just read a book that's both inspiring from a business perspective and at the same time, deeply worrying from a society perspective. It's about public relations and propaganda. The kicker is that the book was published in 1928.

(Propaganda, Edward Bernays, 1928)

The author was Edward Bernays who's generally regarded as the father of public relations and was and is a controversial figure. He was born in Vienna in 1891 and was Sigmund Freud's nephew - another example of the huge influence of the Frued family. In the 1890's, the family moved to the US, where he lived for the rest of his long life, he died in 1995 at the age of 103. During the first world war, he worked for a US government propaganda unit where he learned most of the tools of his trade. In 1929, he successfully promoted smoking to women, and in the 1950's, he worked with the United Fruit Company and the CIA to topple the democratically elected government of Nicaragua. 

His 1928 book, Propaganda, outlines the theory behind public relations and gives details of how successful PR campaigns work. Although Bernays draws a distinction between propaganda and public relations, the line is very, very thing (if it's there at all). The book provides a psychological and sociological background for how PR works and even suggests that it's morally necessary for society to function. He then dives into the use of PR for commerce, politics, and education etc., providing examples of successful campaigns and how they were orchestrated. He very clearly explains, in terms of psychology and sociology, why some influence approaches work and some don't.  What's striking is how politicians and companies are still using these techniques today; it helps explain why some of our media are the way they are.

The book isn't an easy read. In my view, it's repetitive, overwritten, and lacks detail in many places. Bernays' moral justification for propaganda feels paper thin. But despite this, I recommend reading it, or at least reading a more recent book on propaganda, it's eye-opening.

The highlights

I'm not going to review the book in detail, instead, I'm going to give you some key quotes so you get a sense of what it says. You can decide for yourself if it's worth a trip to the library (or a click to download).

"In theory, every citizen makes up his mind on public questions and matters of private conduct. In practice, if all men had to study for themselves the abstruse economic, political, and ethical data involved in every question, they would find it impossible to come to a conclusion about anything."

In other words, people need PR to understand the world and form opinions about things.

"It has been found possible so to mold the mind of the masses that they will throw their newly gained strength in the desired direction. In the present structure of society, this practice is inevitable. Whatever of social importance is done to-day, whether in politics, finance, manufacture, agriculture, charity, education, or other fields, must be done with the help of propaganda. Propaganda is the executive arm of the invisible government."

Bernays talks a lot about the invisible government, these are the people who shape the thoughts and opinions of the masses.

"The mechanism by which ideas are disseminated on a large scale is propaganda, in the broad sense of an organized effort to spread a particular belief or doctrine."

"Small groups of persons can, and do, make the rest of us think what they please about a given subject."

"There are invisible rulers who control the destinies of millions. It is not generally realized to what extent the words and actions of our most influential public men are dictated by shrewd persons operating behind the scenes."

"The invisible government tends to be concentrated in the hands of the few because of the expense of manipulating the social machinery which controls the opinions and habits of the masses."

"Trotter and Le Bon concluded that the group mind does not think in the strict sense of the word. In place of thoughts it has impulses, habits and emotions. In making up its mind its first impulse is usually to follow the example of a trusted leader."

"The newer salesmanship, understanding the group structure of society and the principles of mass psychology, would first ask: "Who is it that influences the eating habits of the public?" The answer, obviously, is: "The physicians." The new salesman will then suggest to physicians to say publicly that it is wholesome to eat bacon. He knows as a mathematical certainty, that large numbers of persons will follow the advice of their doctors, because he understands the psychological relation of dependence of men upon their physicians."

"This point is most important in successful propaganda work. The leaders who lend their authority to any propaganda campaign will do so only if it can be made to touch their own interests. There must be a disinterested aspect of the propagandist's activities. In other words, it is one of the functions of the public relations counsel to discover at what points his client's interests coincide with those of other individuals or groups."

"Just as the production manager must be familiar with every element and detail concerning the materials with which he is working, so the man in charge of a firm's public relations must be familiar with the structure, the prejudices, and the whims of the general public, and must handle his problems with the utmost care. The public has its own standards and demands and habits. You may modify them, but you dare not run counter to them."

"The public is not an amorphous mass which can be molded at will, or dictated to. Both business and the public have their own personalities which must somehow be brought into friendly agreement."

"A sound public relations policy will not attempt to stampede the public with exaggerated claims and false pretenses, but to interpret the individual business vividly and truly through every avenue that leads to public opinion"

"Continuous interpretation is achieved by trying to control every approach to the public mind in such a manner that the public receives the desired impression, often without being conscious of it. High-spotting, on the other hand, vividly seizes the attention of the public and fixes it upon some detail or aspect which is typical of the entire enterprise."

"Present-day politics places emphasis on personality. An entire party, a platform, an international policy is sold to the public, or is not sold, on the basis of the intangible element of personality. A charming candidate is the alchemist's secret that can transmute a prosaic platform into the gold of votes."

"Propaganda will never die out. Intelligent men must realize that propaganda is the modern instrument by which they can fight for productive ends and help to bring order out of chaos."

Final thoughts

I can clearly see companies pursuing Bernays' PR strategy even today and what's more, I can see why they're doing it and why they're successful. I can see the role of newspapers and magazines in shaping public preferences and I can see how organizations are using social media in the same way. The same goes for politics. 

It's nice to be idealistic about the future, but reading Bernays' book, I get the feeling people have been trying to manipulate me my entire life and that it's not going to stop.

Plagiarism and blog posts

Imitation is not the sincerest form of flattery

Prior to the pandemic, I wrote a thought piece on data science. It compared the work of data science to building Lego models and called back to some of my childhood memories of building Lego models with my brothers. I deliberately wrote it to have a slightly dreamy and nostalgic quality. I was very pleased with the finished piece and I referenced it from my LinkedIn profile. You can read it here: https://www.truefit.com/blog/Data-is-the-New-Lego.

The other day, I was thinking about this piece and did a Google search on it. I found someone had plagiarized it. They'd taken the whole article and replaced a few sentences with their 'own' work. They'd even used the same type of images I did. It was pretty much a word-for-word copy (to be clear: it's blindingly obvious this is a direct copy of my work). Of course, they didn't acknowledge my piece at all. What was truly galling was a comment someone had made calling the piece insightful. The plagiarist replied commenting that they were glad they liked it. 

(Hariadhi, CC BY-SA 3.0, via Wikimedia Commons)

The plagiarist has several other pieces on Medium. I have no idea if they copied the other pieces too. They're studying data science and on their profile, they say they want to tell stories with data. Perhaps the biggest story they're telling is that they cheat and take credit for other people's work.

The borders of originality

In this case, the copying was a blatant lift of my work, but other cases are more difficult. There's a nuanced question of what's plagiarism and what's not, for example, many people have written stories about time machines after H.G. Wells, are they all guilty of plagiarism? 

For me, the line is the story arc and ideas. If you're telling the same story as someone else and using the same ideas, you're on very thin ice. If you're using the same metaphors, similies, or allegories then you've crossed the line. If you must tell the same story as someone else (and you really shouldn't), at least use your own imagery.

What have I done?

On the person's Medium post, I have called out their plagiarism and I've reported the piece as violating Medium's terms and conditions. It was posted in the "Towards Data Science" publication so I complained to them too. The Towards Data Science team removed the author from their publication and reported the plagiarism to Medium. I reported the author for plagiarism to Medium again.

It also set me thinking about the interview process. I've looked at people's Github pages and their portfolios. Up to now, it didn't occur to me that people might blatantly cheat. After this experience, I'm going to up my checks.

What brown M&Ms can tell you about a company

Small things reveal deeper truths

I was reading an old story on the internet and it struck me that there's something I could learn from it about diagnosing company culture. I'll tell you the story and show you how small things can be very revealing.

The Van Halen story

Here's a quote from David Lee Roth’s autobiography, Crazy from the Heat, that tells the story. 

"Van Halen was the first band to take huge productions into tertiary, third-level markets. We’d pull up with nine eighteen-wheeler trucks, full of gear, where the standard was three trucks, max. And there were many, many technical errors — whether it was the girders couldn’t support the weight, or the flooring would sink in, or the doors weren’t big enough to move the gear through. The contract rider read like a version of the Chinese Yellow Pages because there was so much equipment, and so many human beings to make it function. So just as a little test, in the technical aspect of the rider, it would say “Article 148: There will be fifteen amperage voltage sockets at twenty-foot spaces, evenly, providing nineteen amperes . . .” This kind of thing. And article number 126, in the middle of nowhere, was: “There will be no brown M&M’s in the backstage area, upon pain of forfeiture of the show, with full compensation.”

So, when I would walk backstage, if I saw a brown M&M in that bowl . . . well, line-check the entire production. Guaranteed you’re going to arrive at a technical error. They didn’t read the contract. Guaranteed you’d run into a problem. Sometimes it would threaten to just destroy the whole show. Something like, literally, life-threatening."

In other words, the no brown M&Ms clause was a simple compliance check that the venue had read the contract and taken it seriously. It was an easy test of much deeper problems.

(This would fail the test - there are brown M&Ms! Evan-Amos, Public domain, via Wikimedia Commons)

Tells

The brown M&Ms story shows that something simple can be used to uncover a fundamental and harder-to-check problem. The same idea appears in Poker too - it's the ideas that players have "tells" that reveal something about their hands. It occurred to me that over the years, I'd seen something similar in business. I've seen cases where companies have made sweeping statements about culture but small actions have given the game away. Unlike the Van Halen story, the tells are usually unintentional, but nonetheless, they're there. Here are some examples.

Our onboarding is the best, but we won't pay you

Years ago, I worked for a company that made a big deal of how great its onboarding was; the CEO and other executives claimed it was "industry-leading" and praised the process. 

When I was onboarded, the company messed up its payroll and didn't pay me for a while; way past the legal deadline. I asked when it was going to be resolved and I was told I should "manage my finances better". I later learned this was a common experience and many new employees weren't paid on time, the "manage your finances better" was the stock response. In one extreme case, I know someone who wasn't paid for over two months.

As it turned out, this was a brown M&Ms case. It indicated profound issues at the company and in particular with the executive team; they were too remote from what was going on and they really weren't interested in hearing anything except praise. It took me and others a long time to discover these issues. The brown M&Ms should have warned us very early that something was quite broken. 

I'm too important to talk to you

At another company, a new C-level executive joined the organization and there was a long announcement about how great they were and how they exhibited the company values, one of which was being people-centric. I reported to the new person's organization. 

One day, early on in their tenure, the new C-level person visited the office I was working at. They walked straight by me and my team without stopping to say hello. During the week they were with us, they didn't meet or talk with any of us. They even managed to avoid being in the break room at the same time as the little people (and people tried very hard to meet the new executive). On that visit, the new C-level person didn't meet or say hello to anyone below vice-president level. Later on, they gave a talk to their organization that included a discussion of the necessity of connecting with people and how it was important to them.

I didn't see many of their other actions, but this was very definitely a brown M&M moment for me. I saw trouble ahead and left the company not long after, and I wasn't the only one.

Candies: going, going, gone

My last example is actually about candy. 

I worked for a company that provided candy and snacks. It was very proud that what it provided was top quality, and I agreed; it really did provide great treats. The company presented top-quality candy and snacks as a way of showing how much it valued its employees; we were told that we got the best because we were valued. 

You can probably guess what happened next. The snack and candy brands went from well-known brands to own-label brands, while the company insisted that nothing had changed. After a few months of own-label brands, the candy and snacks stopped altogether, and the company never said a word. A number of other things happened too, including worse terms and conditions for new employees (less leave etc.), more restrictions on travel, and fewer corporate lunches, but these were harder to see. The company started valuing employees less and the treats and candies were only the most visible of several actions that took place at the same time; they were the canary in the coal mine.

What can you do?

Small issues can give you a clue that things are deeply broken in hard-to-detect ways. You should be on the lookout for brown M&M moments that give you advance warning of problems.

As an employee, these moments provide insight into what the company really is. If the M&M moment is serious enough, it's time to think about employment elsewhere, even if you've just started.

As an executive, you need to be aware that you're treated differently from other people. You might not experience the brown M&M moment yourself, but people in your organization might. Listen to people carefully and hear these moments; use them to diagnose deeper issues in your organization and fix the root cause. Be aware that this is one of the few moments in your life you might get to be like David Lee Roth.

W.E.B. Du Bois - data scientist

Changing the world through charts

Two of the key skills of a data scientist are informing and persuading others through data. I'm going to show you how one man, and his team, used novel visualizations to illustrate the lives of African-Americans in the United States at the start of the 20th century. Even though they created their visualizations by hand, these visualizations still have something to teach us over 100 years later. The team's lack of computers freed them to try different forms of data visualizations; sometimes their experimentation was successful, sometimes less so, but they all have something to say and there's a lesson here on communication for today's data scientists.

I'm going to talk about W.E.B. Du Bois and the astounding charts his team created for the 1900 Paris exhibition.

(W.E.B. Du Bois in 1904 and one of his 1900 data visualizations.)

Who was W.E.B. Du Bois?

My summary isn't going to do his amazing life justice so I urge you to read any of these short descriptions of who he was and what he did:

• Wikipedia
• NAACP
• Britannica
• History.com

To set the scene here's just a very brief list of some of the things he did. Frankly, summarizing his life in a few lines is ridiculous.

• Born 1868, Great Barrington, Massachusetts
• Graduate of Fisk University and Harvard - the first African-American to gain a Ph.D. from Harvard
• Conducted ground-breaking sociological work in Philadelphia, Virginia, Alabama, and Georgia
• His son died in 1899 because no white doctor would treat him and black doctors were unavailable
• Was the primary organizer of "The Exhibit of American Negroes" at the Exposition Universelle held in Paris between April and November 1900
• NAACP director and editor of the NAACP magazine The Crisis
• Debated Lothrop Stoddard, a "scientific racist" in 1929 and thoroughly bested him.
• Opposed US involvement in World War I and II.
• Life-long peace activist and campaigner, which led to the FBI investigating him in the 1950s as a suspected communist. They withheld his passport for 8 years.
• Died in Ghana in 1963.

Visualizing Black America at the start of the twentieth century

In 1897, Du Bois was a history professor at Atlanta University. His former classmate and friend, Thomas Junius Calloway, asked him to produce a study of African-Americans for the 1900 Paris world fair, the "Exposition Universelle". With the help of a large team of Atlanta University students and alumni, Du Bois gathered statistics on African-American life over the years and produced a series of infographics to bring the data to life. Most of the names of the people who worked on the project are unknown, and it's a mystery who originated the form of the plots, but the driving force behind the project was undoubtedly Du Bois. Here are some of my favorite infographics from the Paris exhibition.

The chart below shows where African-Americans lived in Georgia in 1890. There are four categories: 

• Red - country and villages
• Yellow - cities 2,500-5,000
• Blue - cities 5,000-10,000
• Green - cities over 10,000

the length of the lines is proportional to the population and obviously, the chart shows the huge majority of the population lived in the country and villages. I find the chart striking for three reasons: it doesn't follow any of the modern charting conventions, it clearly represents the data, and it's visually very striking. My criticism is that the design makes it hard to visually quantify the differences, for example, how many more people live in the country and villages compared to cities 5,000-10,000? If I were drawing a chart with the same data today, I might use an area chart to represent the same data; it would quantify things better, but it would be far less visually interesting.

The next infographic is two choropleth charts that show the African-American population of Georgia counties in 1870 and 1880. Remember that the US civil war ended in 1865, and with the Union victory came freedom for the slaves. As you might expect, there was a significant movement of the now-free people. Looking at the charts in detail raises several questions, for example, why did some areas see a growth in the African-American population while other areas did not? Why did the highest populated areas remain the highest populated? The role of any good visualization is to prompt meaningful questions.

This infographic shows the income and expenditure of 150 African-American families in Georgia. The income bands are on the left-hand side, and the bar chart breaks down the families' expenses by category:

• Black - rent
• Purple - food
• Pink - clothes
• Dark blue - direct taxes
• Light blue - other expenses and savings

There are several notable observations from this chart: the disappearance of rent above a certain income level, the rise in other expenses and savings with rising income, and the declining fraction spent on clothing. There's a lot on this chart and it's worthy of greater study; Du Bois' team crammed a great deal of meaning into a single page. For me, the way the key is configured at the top of the chart doesn't quite work, but I'm willing to give the team a pass on this because it was created in the 19th century. A chart like this wouldn't look out of place in a 2022 report - which of itself is startling.

My final example is a comparison of the occupations of African-Americans and the white population in Georgia. It's a sort-of pie chart, with the upper quadrant showing African Americans and the bottom quadrant showing the white population. Occupations are color-coded:

• Red - agriculture, fishery, and mining
• Yellow - domestic and personal service
• Blue - manufacturing and mechanical industries
• Grey - trade and transportation
• Brown - professions

The fraction of the population in these employment categories is written on the slices, though it's hard to read because the contrast isn't great. Notably, the order of the occupations is reversed from the top to the bottom quadrant, which has the effect of making the sizes of the slices easier to compare - this can't be an accident. I'm not a fan of pie charts, but I do like this presentation.

Influences on later European movements - or not?

Two things struck me about Du Bois' charts: how modern they looked and how similar they were to later art movements like the Italian Futurists and Bauhaus. 

At first glance, his charts look to me like they'd been made in the 1960s. The typography and coloring were obviously pre-computerization, but everything else about them suggests modernity, from the typography to the choice of colors to the layout. The experimentation with form is striking and is another reason why this looks very 1960s to me; perhaps the use of computers to visualize data has constrained us too much. Remember, Du Bois's mission was to explain and convince and he chose his charts and their layout to do so, hence the experimentation with form. It's quite astonishing how far ahead of his time he was.  

Italian Futurism started in 1909 and largely fizzled out at the end of the second world war due to its close association with fascism. The movement emphasized the abstract representation of dynamism and technology among other things. Many futurist paintings used a restricted color palette and have obvious similarities with Du Bois' charts, here are just a few examples (below). I couldn't find any reliable articles that examined the links between Du Bois' work and futurism.

Numbers In Love - Giacomo Balla Image from WikiArt	Music - Luigi Russolo Image from WikiArt

The Bauhaus design school (1919-1933) sought to bring modernity and artistry into mass production and had a profound and lasting effect on the design of everyday things, even into the present day. Bauhaus designs tend to be minimal ("less is more") and focus on functionality ("form follows function") but can look a little stark. I searched, but I couldn't find any scholarly study of the links between Du Bois and Bauhaus, however, the fact the Paris exposition charts and the Bauhaus work use a common visual language is striking. Here's just one example, a poster for the Bauhaus school from 1923.

(Joost Schmidt, Public domain, via Wikimedia Commons)

Du Bois' place in data visualization

I've read a number of books on data visualization. Most of them include Nightingale's coxcomb plots and Playfair's bar and pie charts, but none of them included Du Bois charts.  Du Bois didn't originate any new chart types, which is maybe why the books ignore him, but his charts are worth studying because of their experimentation with form, their use of color, and most important of all, their ability to communicate meaning clearly. Ultimately, of course, this is the only purpose of data visualization.

Reading more

W. E. B. Du Bois's Data Portraits: Visualizing Black America, Whitney Battle-Baptiste, Britt Rusert. This is the book that brought these superb visualizations to a broader audience. It includes a number of full-color plates showing the infographics in their full glory.

The Library of Congress has many more infographics from the Paris exhibition, it also has photos too. Take a look at it for yourself here https://www.loc.gov/collections/african-american-photographs-1900-paris-exposition/?c=150&sp=1&st=list - but note the charts are towards the end of the list. I took all my charts in this article from the Library of Congress site. 

"W.E.B. Du Bois’ Visionary Infographics Come Together for the First Time in Full Color" article in the Smithsonian magazine that reviews the Battle-Baptiste book (above).

"W. E. B. Du Bois' Hand-Drawn Infographics of African-American Life (1900)" article in Public Domain Review that reviews the Battle-Baptiste book (above).

RCT bingo!

A vocabulary of causal inference testing

I was having a clear-out and I came across a printout of some notes I made a while back. It was a list of terms used in causal inference testing. At the time, I used it as a checklist or dictionary to ensure I knew what I was talking about - a kind of RCT bingo if you like.

(Myriam Thomas, CC BY-SA 4.0, via Wikimedia Commons)

I thought I would post it here in case anyone wants to play the same game. Do you know what all these terms mean? Are there key terms I've missed off my list?

• ATE - Average Treatment Effect
• CATE - Conditional Average Treatment Effect
• Counterfactual
• DAG - Directed Acyclic Graph
• Dynamic Treatment Effect
• Epsilon greedy
• Estimands
• External and internal validity
• Heterogeneity (treatment effect heterogeneity) 
• Homophily
• Instrumental Variable (IV)
• LATE - Local Average Treatment Effect
• Logit model
• RCT - Randomized Control Trial
• Regret
• Salience
• Spillover
• Stationary effect (and it's opposite non-stationary effect)
• Surrogate
• SUTVA - Stable Unit Treatment Value Assumption
• Thompson sampling
• Treatment effect heterogeneity
• Wald estimator