What is a social technology?

[This post is part of a series on social technologies.]

In this post, I’ll introduce the concept of a social technology, by means of a couple of examples.

The first comes from a post by Sarah Constantin about the social norm of “don’t shoot the messenger”. Back in the old days, when armies communicated by sending messengers back and forth, this was an extremely important norm to have. If you’re fighting an enemy army, and you notice that your messengers keep getting killed, then you’re going to stop sending messengers. But if both sides have “don’t shoot the messenger” as a norm, then this guarantees that they’ll be able to communicate. Sarah Constantin points out that, without this norm, it becomes much harder for two armies to negotiate for peace.

This example gets at the basic idea behind social technologies, which is that social norms aren’t arbitrary. They often serve important functions, and we can learn a lot by looking at social norms and asking “What is the problem this social norm is trying to solve?” In fact, we can think of social norms as technologies that societies develop in order to help solve the problems they’re facing. These problems can include maintaining internal stability, communicating with other societies, distributing resources, and many more.

Once we start thinking of social norms as technologies, it becomes easier to ask certain questions. For instance, when encountering a new social norm, we might immediately ask “What is this social norm for?” And when instituting a new social norm (or removing an old one), it becomes natural to ask what the effects might be. The more social technologies we study, the better we’ll get at reasoning about these effects, and the better we’ll get at building social technologies for ourselves.

This series (and the link bibliography at the end) should give you plenty of examples of social technologies, so by the end you should hopefully get a pretty good sense for what the term means. But it’s important to note up front that not every social activity is a social technology. For instance, as a friend of mine points out, sex is merely a social activity, but sex rituals are a social technology. Watching a movie is merely a social activity, but watching a movie as a common first date activity is a social technology.

In particular, we tend to use “social technology” to refer to social activities that serve some function beyond the immediate goal. Not shooting the messenger does more than just preserve the messenger’s life; it keeps a channel of communication open. And the Amish custom of barn-raising, where the whole village gathers to help a family put up a barn, does more than just build a barn; it provides a system for the fair distribution of labor (since the family that’s being helped will return the favor by raising other people’s barns later).

Now let’s consider another example: food taboos. They’re extremely common cross-culturally (they may even be a human universal; I’m not sure). Judaism and Islam both have strict food taboos (Kosher laws in Judaism, and halal vs. haram foods in Islam). The Hindu custom of not eating beef is another food taboo. We often think of those as silly or extreme, but our culture has food taboos too: we don’t eat dogs or cats, and we look with disgust and horror on cultures that do.

There’s probably many reasons that food taboos exist, and I don’t claim to understand all of them. But one reason I’ve encountered is that food taboos exist to keep cultures isolated from one another.

It turns out that sharing a meal has primal significance to humans. When we eat together, it brings us closer together, in a way that just having a conversation or taking a walk together wouldn’t. For instance, according to this paper, when two people eat the same foods, it increases the level of trust they have for each other.

So cultures can keep themselves isolated by creating food taboos, which make it much harder to eat with people outside the group. If your culture forbids eating pork, and the people in the tribe next door eat pork all the time, then it’ll be difficult to share a meal with them. And if your culture says you need separate pots for cooking milk and meat dishes, it’ll be almost impossible to eat the food that any other culture has cooked.

In general, the more restrictive a culture’s food taboos, the more isolating that culture becomes. This creates extra bonds of solidarity within the group, while preventing those bonds of solidarity from forming with outsiders.

This example gets at a few things. The first is that, when you’re looking at a social norm, it’s not always obvious what that norm is for. Like, this explanation of food taboos seems pretty non-obvious. You don’t look at food taboos and immediately think “ah, that’s why this norm exists”. We’re able to grasp this function of food taboos because, via anthropology and psychology, we’ve learned a lot about how eating together creates solidarity. But we still have much to learn about the psychological mechanisms underlying human behavior, and there are many human customs that remain mysterious to us.

The second is that a given social norm may serve many functions. I’ve suggested one function for food taboos, but there’s lots of others I’ve heard proposed. For instance, the ban on pork may have arisen because pigs were not suited for Middle Eastern environments and would have caused ecological damage. And the Hindu ban on eating cows might be because cows produce more sustenance through dairy than meat, so it’s better not to kill your cows for meat.

The third is that food taboos, as a social technology, were probably not consciously invented by someone to serve any of these functions. It’s more likely that they developed over time, and were selected for by evolutionary pressures acting on the society. In the case of “don’t shoot the messenger”, you can imagine some bronze age warrior sitting down and saying “huh, if we kill this messenger, they’re not going to send us any more messages, so let’s let him go”. But it’s much harder to imagine primitive people sitting down and saying “well, we want to make that other tribe feel more like outsiders, so let’s eat a different thing than they do”.

All of this combines to suggest: most of the social technologies we encounter are going to be mysterious and hard to understand. Which is part of what makes them interesting to study, but is also an important caveat to keep in mind: this stuff is not obvious. It’s easy to miss social technologies that are right in front of our eyes, and it’s probably easy to guess wrong, and draw incorrect conclusions about how various social norms work.

One other thing that I’d like you to take away from this example: social technologies do not always serve the individual people in the society (at least not directly). They may instead serve the emergent structure of society, which we can view as a sort of organism in its own right. Keeping cultures isolated from one another does not necessarily help the people in the cultures, but it helps the culture, as a memetic entity, maintain itself and propagate itself to the next generation.

So anyway, that should give you an idea of what social technologies are, and some of the considerations that arise when studying them. I’ve already started to touch on the theme that social technologies can be strange and mysterious; the next post will discuss this in more detail.

Posted in Uncategorized | Leave a comment

Social Tech Series: Table of Contents

In this part of the internet, people often talk about social technologies. But I haven’t seen any overview posts that explain what a social technology is. So I’m going to try to write some.

In this series of posts, I’ll present the basic concept, and also describe some of the themes that have pervaded the discussion of social technologies. At the end, I’ll link to a bunch of the relevant articles and blog posts I’ve encountered.

Here’s what the series will look like (links to be added as the posts are published):

  1. What is a social technology?
  2. Theme 1: Social technologies are strange and mysterious
  3. Themes 2 and 3: A postapocalyptic world without social tech?
  4. Additional themes and notes
  5. Social tech link bibliography

I’m actually going to finish this series, unlike every other series I’ve posted on this blog. The posts are mostly written already, and you can expect a new one to appear every couple days.

A quick note, before the series begins: it’s always a little dangerous to write posts like this, which try to explain and summarize what’s been going on in this part of the internet. It’s inevitable that I’m going to miss things. I’m sure some people will read my list of themes and think of obvious things that it’s missing, and I’m sure my link bibliography will leave out posts or articles that other people consider essential.

Even worse, there’s a decent chance that my use of “social technologies” will differ from the way that phrase is used elsewhere on the internet, and will thereby complicate discussions for people who are using the other definitions.

I think these posts are worth writing regardless, because the idea of social technologies (as I’ve encountered it) is important enough to be worth sharing more widely. But I hope that people who disagree with my definition, or who think that I’m missing something important, will leave some comments and let me know.

Posted in Uncategorized | Leave a comment

How to Deal with Self-Fulfilling Prophecies


You probably know the story of Oedipus. He was the son of Laius and Jocasta, the king and queen of Thebes. The oracle at Delphi foretold that he would kill his father and marry his mother, and it was their attempt to avert the prophecy that caused it to come true.

The story goes like this: When Jocasta was still pregnant with Oedipus, Laius went to consult with the oracle at Delphi, who told him the prophecy. So Laius and Jocasta determined that their son should die. When the baby was born, they handed him over to a shepherd, telling the shepherd to abandon the child on a mountainside. But the shepherd took pity on the baby, and brought him to King Polybus of Corinth, who raised him as his own child.

When Oedipus had grown up, he too consulted the oracle of Delphi, who told him the prophecy. And he too tried to avert it. Thinking that King Polybus was his real father, he resolved to leave Corinth and never see King Polybus again.

Leaving Corinth, he headed north to Thebes. At that time, a sphinx was wreaking havoc in Thebes, and Oedipus hoped to meet it and answer its riddle. But on the way to Thebes, Oedipus met a stranger, who challenged his right to walk on that road. They fought, and Oedipus killed the stranger (who, unbeknownst to him, was his real father, King Laius).

Then Oedipus met the sphinx, and successfully answered its riddle, at which point the sphinx committed suicide. The people of Thebes were so grateful that they made Oedipus their king. He married the newly-widowed queen, Jocasta, and thus the prophecy was fulfilled. (When they discovered the truth of what had happened, Oedipus “tore his eyes out, one by one“, and Jocasta committed suicide.)


Myth and fairy tale are full of these sorts of self-fulfilling prophecies. The story basically always goes like this: the characters consult some prophet or oracle, who warns them that something terrible will happen. So the characters take some action to try to avert the prophecy, but it’s that very action which ultimately causes the prophecy to come true.

The story of Sleeping Beauty follows this pattern (though with a curse rather than a prophecy). When Sleeping Beauty is born, she is given a curse, saying that one day, she will prick her finger on a spindle, and fall into a deep sleep that lasts 100 years. So her father, the king, sends out a decree, saying that all spindles are to be banned from the kingdom, on pain of death. Thus Sleeping Beauty grows up without ever having seen a spindle. Then, one day when she is a teenager, she’s exploring an old attic, and finds a spindle which was missed by the purge. Having never seen one before, she is curious, and goes to examine it. While doing so, she pricks her finger, and falls into the sleep that was foretold.

In another story, called “The Devil with the Three Golden Hairs”, it is prophesied that a certain boy (currently a baby) will grow up to marry the king’s daughter. The king hears about this prophecy, and is very unhappy, and tries his best to have the boy killed. First, he takes the boy from his parents’ house, and puts him in a box, and sends the box floating down the river. But a miller and his wife find the box, and raise the baby themselves. Years later, the king discovers that the boy has survived. And so he visits the boy, and asks him boy to deliver a letter to the queen. Unbeknownst to the boy, the letter contains the following instructions: “When you open this letter, kill the messenger who brought it to you.” But while traveling to deliver the letter, the boy gets lost in the forest, and ends up spending the night at the house of some strangers. While he is sleeping, the strangers open the king’s letter; seeing what’s written in it, they forge a new letter, which says “The messenger who brought this letter should marry your daughter.” So the boy delivers the letter, and marries the king’s daughter. Thus, once again, the king’s attempts to avert the prophecy are what cause it to come true.

There are many, many stories of self-fulfilling prophecies. (Wikipedia lists a bunch of them, and TVTropes does as well.) In one common theme, the king hears a prophecy, which says that a child born during some time period, or from some group of people, will eventually kill him. So the king gathers all the babies who fit that description, and has them killed… except he inevitably misses one of them, and that baby grows up to kill him (possibly to avenge the slaughter of all the other babies). (The story of Exodus fits this theme.)

Of all the stories about self-fulfilling prophecies, I think my favorite is the folk song “Crazy Man Michael” (youtube link, lyrics). It was written by Fairport Convention in the 1960s, but it has a timeless quality about it, and if you have any interest in folk music, I highly recommend listening. I’ll summarize the lyrics: One day, Crazy Man Michael is out walking, when he meets a prophetic raven. The raven tells him, “Your true love will die by your own right hand, and Crazy Man Michael will cursed be.” Michael is furious, and he stabs the raven to death. As the raven falls to the ground, it transforms into his true love; he realizes that he has stabbed her, and the prophecy has come true.

The reason I like “Crazy Man Michael” so much is that it seems to distill the self-fulfilling prophecy trope down to its very essence. It strips away all unnecessary parts of the narrative. Michael tries to avert the prophecy in the most direct way possible — by killing the prophet. And rather than this killing his true love by a long and convoluted series of events, it kills her directly, via the same sword-stroke. It’s a perfect summary of the principle behind every self-fulfilling prophecy story: that the very action undertaken to avert the prophecy is the one which makes it come true.


I’ve been thinking about these stories a lot lately. They seem to give a very clear message: trying to avert the prophecy will get you nowhere. No matter what you do, the prophecy is going to come true.

So what should you do if you receive such a prophecy? Suppose you’re walking along, and the raven tells you “Your true love will die by your own right hand.” Or you visit the oracle at Delphi, and she tells you, “You’re going to kill your brother in a car crash.” What do you do with that information?

You can’t escape the prophecy; that much is clear from the stories. But maybe, by trying to fight it, you can delay it. Maybe, if you do nothing, and you make no changes to your life, you’ll kill your brother in a car crash next week. But if you take the prophecy to heart, and you give up driving, you’ll prolong your brother’s life for several more years. (Until that fateful day when your brother breaks his leg slipping on some ice, and there’s no one who can drive him to the hospital except for you. So you pick up his car keys and get in the car, but you haven’t driven in twenty years, and you barely remember how the car works, and the roads are very icy…)

But you can’t know for sure. The opposite might be true. Maybe, if you fight the prophecy, you’ll kill your brother twenty years from now driving on the ice. But if you ignore the prophecy, and just let nature take its course, then maybe the car crash won’t occur until you’re both in your eighties.

The point is, you can’t know whether fighting the prophecy will help. So ultimately, I don’t think there’s any use in trying. Fighting the prophecy just gives you a false sense of control, which will eventually be betrayed.

So what should you do? Should you just give up in despair? Should you spend every day waiting in dread for the inevitable to occur? Should you let depression overtake you, since there’s nothing you can do to prevent tragedy?

I think the answer is exactly the opposite. I think the answer is to savor every moment, because you know that good times will only last for so long, and so you want to make the most of them while they last.

I think Crazy Man Michael should have gone back to his wife, and treasured every day he had with her. (And then maybe one day he kills her by accident, or maybe, when she’s very old and on her deathbed, he’s the one who holds her hand and disconnects the life support.) I think that, in the car crash hypothetical, you should spend lots of time with your brother, and appreciate every minute of it, because you know that your time with him is finite, and therefore precious.

I think maybe we should all do this, with all of our loved ones, all the time. Because we’ve all received prophecies of a sort; we all know that someday, all of us will die. There’s nothing we can do to avoid it, so we might as well enjoy our time on this earth, and try to make the best out of every moment.


Thank you Betawolf, SomeBrashAtom, and housecarpenter for helping me come up with examples of self-fulfilling prophecies!

Posted in Uncategorized | 3 Comments

Literal Yearly Cider

About a month ago, I had the pleasure of attending a literal yearly cider pressing. It seemed like something I should write about on this blog.

In terms of equipment, this cider pressing was very different than the last one I’d attended. That one was back in fall 2016, when I was adventuring in the woods of North Carolina. I was visiting some friends at Wild Roots, an off-the-grid community with no electricity or indoor plumbing. They get their food from hunting, foraging, and gardening; they also dumpster-dive, and collect food and materials that the rest of society is throwing away.

On the day of the cider pressing, we all climbed into the back of Todd’s truck and drove to a nearby organic apple farm. (Todd is the person who’s been at Wild Roots the longest; you’ll see him in the video at that link.) Wild Roots had an arrangement with the apple farm, where after the season was over, they’d come and clean up the apples that had fallen on the ground, taking home as many as they wanted. So we spent the morning salvaging apples, and then drove over to a friend’s house to use his big wooden cider press. We sorted through apples, cutting out the especially rotten or wormy bits, and cutting them into chunks so we could press them more easily. Then we threw them into the wooden press and crushed them into juice.

The cider pressing I attended this year, in Longmont, Colorado, had much more modern machinery, but a similar sense of community. People drove in from nearby towns, bringing crates of apples that they had picked. Some had gotten them from trees in their own yards; others had gone out to public spaces and collected apples there. One woman told me about a volunteer organization that picked apples in order to save the bears. Apparently, there’s a lot of apples trees in town, and many of the people who own the properties don’t care about harvesting the apples. So they just ripen and fall on the ground, and the bears come into town to eat them. Apparently, if a bear is sighted a certain number of times in town (I think it was three), then the bear gets shot. So these volunteers drive around, picking the unwanted apples in hopes of keeping the bears away.

So everyone arrived, and we finished cleaning the equipment, and then the cider pressing began. Everyone who was there pitched in to help. People seemed to settle into jobs. Someone needed to load the apples into the washer, and then someone else needed to help guide them into the grinder after they were washed. And once they were ground, someone needed to carry the buckets of ground apples over to the tables where people were filling them in to the pressing cloths. I ended up working with two boys who were putting buckets under the grinder, and making sure that, when one bucket got filled up, a new bucket could be efficiently slid into place. The younger boy, who was maybe eight years old, seemed like a future engineer. He designed an assembly line process for getting the buckets slid into place, and then stayed there for hours, making sure that everyone was doing the exact job they were supposed to. But he wasn’t strong enough to slide the full buckets himself, so I helped with that part.

Here’s some pictures of the process. These are some of the apples that people brought to be pressed:


This is the apple washer and grinder. The part on the left spins the apples and sprays water on them, in order to rinse them. (The water drips down into the bathtub and then recirculates.) Then a chute opens and the apples spill out into the red part, which is the apple grinder.

The second picture shows the inside of the apple spinner, so you can get a sense for how it works. It wasn’t spinning, so the two gentlemen in the photo had opened it to fix it. They’re the ones own and built the cider press; they used to host this cider pressing every year, until a fire destroyed their barn and all of their equipment. They’ve spent the last few years rebuilding, and this year was the first cider pressing with their new equipment. The concrete slab we were standing on was the place where the old barn used to be.

Here’s some photos of the grinding process. The first two show the apples rolling into the grinder. The last photo shows the grinder itself; that cylinder spins, and the bits of metal (maybe pieces of nails?) shred the apples.

Once the apples were ground, they were put in cloths to be pressed. The wood frames were used to get the filled cloths to be the right size. A cloth was placed in the frame, and then filled with apples, and then the cloth was folded over the top, and the wood frame was removed.

The finished cloths full of apples were stacked in between sheets of plastic to press them in this hydraulic press.

Here’s the cider coming out of the press! It was filtered through a mesh bag to remove any solids, and then it flowed down a pipe (the cider-colored one) to a refrigerated tank.

When the cider pressing was finished, my friend Trevor and I took about 12 gallons home to brew into alcoholic beverages. (Trevor is the one who invited me to the cider pressing; he’s also been teaching me, over the last year or so, how to brew various things.)

So the next day, I went over to Trevor’s house to actually do the brewing. Trevor wanted to make a cyzer (which is a combination of cider and mead), while I wanted to see what the cider would taste like if fermented with its natural yeasts. So we did both.

It turns out it’s extremely easy to make hard cider. In fact, it practically makes itself. If you’ve ever picked a wild apple, you’ll know that it’s not shiny; instead, it looks kind of matte, because it’s covered in a thin white film. That film is the yeasts. The yeasts want to eat the apple, but they can’t, because they can’t get through its skin. (That’s the whole point of the apple skin — to protect the fruit from microorganisms that might want to eat it.) But even though the yeasts can’t eat the apple yet, they stay on the skin anyway, waiting for the day when the apple will fall off the tree and the skin will get broken open.

So when you take a whole apple, skin and all, and you crush it for cider, you’re doing exactly what the yeasts were waiting for: you’re giving them access to the delicious apple flesh within. And the cider you press will be full of yeasts as well. So basically, all you need to do in order to make wild cider (that is, cider made with wild yeasts) is to put the freshly-pressed cider in a nice clean container, and let the yeasts do their work. (The alcohol they produce will keep any other microorganisms from growing in the beverage.)

Here’s some pictures of that process.

The first (and most time-consuming) step in any brewing project is cleaning and santizing all the equipment. For cleaning, we just wash stuff. For sanitizing, we use something called StarSan, which sanitizes the equipment without affecting the flavor of the beverage. Here’s a picture of Trevor washing the equipment, and me spraying the fermenting bucket with StarSan.

We then poured some cider from the tank where we were storing it, into the bucket. At this point, the cider had been sitting in the tank for almost 24 hours, and it had already started to ferment. It was slightly fizzy.

Once the cider was in the bucket, we aerated it. Aerating introduces oxygen, which makes the yeasts extra excited about fermenting. We used Trevor’s immersion blender for this step.

Then we transferred the cider from the bucket to the carboy, and left it to ferment. (We could have fermented the cider in the bucket, but we wanted to use the bucket for the cyzer project. So we moved the cider to the carboy. The reason we didn’t just pour it into the carboy to begin with is that you can’t aerate it when it’s in the carboy.)


Then Trevor took a specific gravity reading. I don’t quite understand how this works, but somehow it measures the amount of sugar in the cider, which gives a sense for how much alcohol it might eventually produce.


The last step was putting the airlock on the carboy and then leaving it to ferment. The airlock solves the following problem: Fermentation produces a lot of carbon dioxide, and you need to let the carbon dioxide out of the carboy, or else it will explode. But if you just leave the carboy open, then all sorts of bugs and microorganisms can get in. So you use an airlock, which lets air out but not in (at least assuming that the pressure inside the carboy is higher than the pressure outside it).

So we put on the airlock, and we filled it with liquid (we used StarSan, but water would have worked), and then we left it there for nature to take its course.

So that’s our cider! It’s been sitting there for about a month now. It’s probably done fermenting, or almost done, but Trevor and I have both been busy (including with other homebrewing projects), and haven’t had time to rack or bottle it. (Racking is when you transfer it from one fermenting vessel to another. That stirs up the cider, which causes it to start fermenting more; it also helps get rid of some of the sediment, which is the solid gunk that sinks to the bottom of the carboy.)

So I can’t tell you how the cider tastes yet; I’ll have to report back later. But I can tell you that I’m looking forward to tasting some real Colorado cider, made from apples that we picked and pressed ourselves, and using the yeasts that grow naturally in this part of the world.


Posted in Uncategorized | Leave a comment

The Fragility of Knowledge

A couple years ago, I was taking an outdoor class in West Virginia. I had brought a book to read, and I had put it down on a tree stump as I worked on my project. And while I wasn’t looking, one of the resident goats came by and ate the book’s cover.

This got me thinking about the fragility of knowledge. A book is such a fragile object, and the knowledge it contains is fragile as well. If our society collapsed, how much of our knowledge would be saved, and how much would be lost? What would it take to preserve that knowledge?


We have such an abundance of knowledge these days that we forget how easy it is to destroy. We store our knowledge in physical media (books, hard drives, etc.) and those physical objects are remarkably fragile.

Digital media are fragile in a few ways. For one thing, they’re physically fragile; if you’ve ever dropped a hard drive and broken it, you know how easy it is to lose hundreds of gigabytes of data. They’re also fragile in that they rely on electricity; if the electric grid goes down, we’ll lose access to all the knowledge on our DVDs, hard drives, and the internet. And they’re fragile in the sense that digital technologies quickly become obsolete; I remember watching my dad laboriously copying our family’s home videos from VCR to DVD when that technology shifted.

We can work to preserve our digital data. We can guard against physical damage by making backups, and if the electric grid goes down, we can bring it back up. And there are data librarians, carefully transferring knowledge into new storage media and file formats as the old technologies become obsolete.

But it takes a lot of work to maintain our digital data, and already, so much of it has been lost. Essential data from the Apollo missions is inaccessible because we’ve lost the ability to read the tapes it’s stored on. And the last programmer who really understood the code for the Voyager spacecraft has now retired. As time goes on, more and more scientific data and instruments will be lost in the same way.


We don’t think about books as fragile, because we live in a carefully climate-controlled environment where they’re easy to preserve. But imagine trying to preserve a library full of books in a more primitive dwelling place, say, a humble log cabin with a dirt floor.

Animals will eat your books if you’re not careful. It should be easy to keep out the larger ones, like goats, as long as your cabin has a door. But mice and rats are notoriously good at sneaking through the cracks. They’re a problem in modern dwellings, so I can only imagine they’d cause that much more trouble in a primitive house.

And if you somehow manage to keep out the mice and rats, then you still have bugs to contend with: silverfish and all the other creepy-crawlies that like chewing on paper.

Now suppose you somehow manage to eliminate the rats, mice, and bugs. You still have to worry about dampness, which will rot your books if you’re not careful.

And you also need to worry about humans. Presumably, you’re going to have people handling these books (since otherwise, why bother storing them?). And people are going to cause wear and tear, ripping pages and bending covers and so on. The oils from people’s fingers will damage the books. And if you’re living in a less civilized society, where people work outside and don’t wash their hands as often, they’re going to end up smearing dirt, grease, and animal residue on the pages.

(To give you a more concrete picture: a couple years ago, I spent a week visiting some friends who had run away to live in the woods. When you spend all day outdoors, your hands get dirty. There was no easy access to soap, so you’d use the bathroom and then just not wash your hands (presumably following the left-hand right-hand rule). If you wanted to cook meat, you’d just grab some raw meat that was sitting out (they didn’t have a refrigerator), throw it in the pot or pan, and then wipe your hands on your pants. I helped them chop up some bear fat to render, and then I wiped my greasy, smelly, meaty hands on my pants, and picked up my book and started reading it again. By the time I got back from that trip, the cover had fallen off. That’s also part of what made me want to write this post.)

Anyway, even if you solve all these problems, even if you are perfectly and inhumanly carefully with your books, they still only have a finite lifespan. Eventually, the pages will get brittle, and will crumble to dust in your hand. No matter how carefully you tend to your books, you can’t hold onto them forever.

If you want to preserve books for hundreds of years, you have to copy them. If you have a printing press, you can use that. If you don’t, then someone has to sit down and copy the books by hand.


“So what,” you say to all this. “It’s an interesting thought experiment, but we’ve got people archiving all the important digital knowledge. And if something happens to our books, we can just print new ones. It’s all well and good to think about previous civilizations, but this is 2018. We’ve got climate control. We’ve got very technologically sophisticated libraries.”

And yes, we’re fine for now. As long as we can preserve our current levels of technology, then we can also preserve our knowledge. But with one little blip, all of that could be destroyed (nuclear apocalypse, destruction of the electrical grid, major plague that kills 90% of people, you name it).

The thing is, knowledge and technology are mutually reinforcing. As we’ve discussed, you can’t preserve knowledge without the technology to read, store, and reproduce the media that contain that knowledge. And without the knowledge, you can’t recreate the technology… because the instructions for building a printing press were (you guessed it) written in one of the books that got destroyed.

So suppose you have a nuclear war that kills large numbers of people and throws society into chaos for a couple hundred years. People are too desperate for food and survival to bother preserving books, or building printing presses, or doing much of else besides struggling to survive. And by the time things have stabilized enough for people to start wanting to print books again, all the instructions on building a printing press (or a computer, or the electrical grid) have been lost, and all the people who once knew how to do it are dead.

If you fail to preserve knowledge, if you fail to save the printing presses and copy the books as they start to fall apart, then you only have a short window — maybe 40 or 50 years — before anyone who might have been able to recreate the technologies will have died off.


Let’s think about a non-industrial, non-electric society (either pre-modern or post-apocalyptic), and let’s ask: what would it take for such a society to preserve knowledge?

If you don’t have the printing press, then the only way to pass written knowledge down through the ages is to have scribes laboriously copying books by hand.

And to make the books at all, you need certain technologies. You need something to write on: paper or parchment, or maybe clay tablets. If you’re using paper or parchment, you need ink. And you need something to write with: a pen or stylus. You need the raw materials to make these things from, and you need the knowledge of how to make them. For the raw materials, there may only be so much to go around — only so many animal skins to use as parchment, only so many berries from which to get ink. So you’ll be limited in the number of books you can store, since you can only copy a finite number as they start to disintegrate.

And let’s not forget the most important technology of them all — written language. We take it for granted, but writing was only invented around 3200 BC. Without writing, all of your knowledge must be transmitted orally (or through non-verbal pictures). Oral histories can be surprisingly high-fidelity, but they’re still very limited in the amount of knowledge they can store. When all you have is the oral tradition, you’re limited to the amount that your tribe can memorize and pass on.

Could we lose our writing system? Could we lose the concept of writing altogether, and forget that our ancestors had ever stored words in a physical object? I think that’s unlikely — even in the most dire post-apocalyptic setting, scraps of writing will remain: words inscribed into monuments and gravestones, ancient fading street signs. It will be a long time before all traces of our writing are gone.

But if we did lose it, could we recover it? How long would it take? It took most of our evolutionary history to develop the concept in the first place. How long would it take us to figure it out again?


So those are some of the technological requirements for storing and copying knowledge. But what about the social requirements?

Well, first of all, you need a society that values the preservation of knowledge. If all your society cares about is warfare and cattle herding, then it’s not going to devote its resources to creating and preserving books. Why waste human resources training scribes when you could send those people out to wage war or raise cattle?

And even if your society cares about preserving knowledge, it has to be physically structured in such a way that creating and preserving books is possible. For instance, if you belong to a nomadic tribe, and you have to carry all your possessions on your back or in small wagons, a large library just wouldn’t be feasible.

And preserving knowledge takes time and resources. Someone has to create the paper or parchment, and someone has to make the ink to write with. Then someone has to laboriously copy the books, letter by letter.

If you belong to a small tribe of only 50 or 100 people, it might not make sense to train someone as a scribe (or an ink-maker, or a paper-maker). And even if you do have someone skilled in these roles, they might not have much time to devote to it in addition to all the everyday tasks around the village.

I may be wrong, but the way I see it is, book-making (at least at a scale larger than one or two books per tribe) only starts to become viable when your society gets large enough to support division of labor. You need enough people taking care of the basic subsistence needs that a few others can devote their time to specialized crafts like being a scribe.


So what might we expect to see, in a society that values the preservation of knowledge?

Well, for one thing, we’re likely to find scholars: people whose job is it to produce and preserve knowledge. And it’s likely that these people will receive high status, or some other kind of special recognition, in the society.

And this is indeed what we have in our society today! We have academics, a whole group of people whose entire job is to study the knowledge that other people have generated, and to add to that body of knowledge.

Think how weird academics might seem to a stone-age civilization. “You mean there are people who just sit and read books all day? And they never go outside and work with their hands? They do nothing but read and write, and yet society still gives them food and shelter? What freeloaders!”

If you don’t understand the fragility of knowledge, and the importance of knowledge to society, it’s easy to see “academic” as a worthless profession. There are historians that do no nothing but read and understand ancient texts and add a little commentary to their interpretation. There are mathematicians who climb through ever-higher realms of abstraction with less and less connection to the actual world.

And it’s unclear whether the knowledge produced by any specific historian or mathematician will be useful down the line (or even what “useful” means to a given society). But if a society has reverence for knowledge in general, and gives a special class of people the freedom to pursue knowledge, then you end up with a large body of information and a group of people that understand what it means and how to use it.

Another thing I’d expect to see, in a society that valued knowledge, is a cult of sacredness surrounding books. The need for this cult arises directly from the fragility of the physical media: if books are fragile, and books are important, then it’s essential to treat them with reverence and care, in order to preserve them as long as possible.

Books are easily destroyed, so you store them in a special room or container to keep them from being damaged. People are dirty, so you make them undergo a ritual purification before they are allowed to touch the book. Scribes treat their jobs as holy, reverently copying character after character from the old to the new parchment.

And indeed, you can find this in Judaism, a religion whose worship centers around a holy book, the Torah, which makes it necessary to preserve that book carefully. The Torah is stored in the Ark, and only taken out during special religious occasions. You don’t touch it directly with your hands, but with a special metal pointer. The Torah can only be copied by a qualified scribe.

In general, if you want your culture to treat books with respect and care, it makes sense to teach people that books are sacred. The individual rituals might do nothing to preserve the books, but they teach people that books are important and worthy of reverence. For instance, in Judaism, if you accidentally drop a holy book or set it on the floor, it is customary to kiss it. When the Torah is taken out of the Ark and carried around, the whole congregation stands. See this article for more details of Jewish ideas and practices surrounding holy books.


So you can make a society that values knowledge. You can make a society that treats books with reverence, and which has the technology and manpower to copy its books over as the old ones decay.

But knowledge isn’t just a physical object. You need a human to interact with what’s stored in that object. You need a reader to look at those written words and understand what they mean. Fun thought exercise: if you have a book written in a dead language that no one can read, can it still be said to be storing knowledge?

Even if you manage to preserve the books, even if you have the technology to copy them as needed, and your society wants to devote its resources to doing so… if no one can understand what the books mean, then they’re worthless.

Here are some issues that make knowledge fragile even if there’s no apocalypse or loss of technology:

Language change. Anyone who’s tried to study Greek or Roman texts knows this one. If you want to access the knowledge written by Ovid and Cicero, you either need to learn Latin well enough to read the originals, or you have to rely on a translation (which will necessarily introduce some inaccuracies).

If you rely on translations, eventually the language is going to change enough that you need a new translation. So if you have Cicero in 21st-century English and the original Latin, and no one in 2400 AD understands Latin, then they’re going to have to translate Cicero from the English, introducing even more inaccuracies.

So in practice, if you really want to preserve knowledge, then you’re going to need scholars of ancient languages, who are willing to devote their time to studying the meaning of these texts.

Loss of context. Even if you have scholars who are fluent in some ancient tongue, they won’t necessarily be able to understand everything that’s written in it. Life has changed so much over the last few thousand years (both culturally and in terms of material existence) that it’s hard to understand what people were writing about back then.

For example, I have a friend who’s a historian. At one point, he and some colleagues were studying fight scenes in ancient literature. The fights were described as very short, and my friend’s colleagues were speculating about why the authors had chosen this stylized representation. My friend (who has done a fair bit of fighting) had to point out that no, these fight scenes weren’t stylized; real fights tend to be quite short.

This seemed like a perfect example of how, without the lived experience, it’s impossible to tell what’s an accurate description and what’s artistic flair; it’s impossible to tell what’s normal behavior (on the part of the characters or environment) and what’s out of the ordinary.

As another example, I recently read a translation of The Voyage of St. Brendan, a 10th-century Latin text written by Irish monks. I found a lot of it difficult to understand, because the narrative is organized around Christian holidays and the different prayers recited at different parts of the day, and I’m not familiar enough with Christianity to understand their significance. It’s clear that these times and holidays had symbolic meaning in the text, which the reader was expected to understand.

(Note that modern writers omit just as much context as ancient ones. For a silly example of what stories might look like if we included more of the context, see “If All Stories Were Written Like Science Fiction Stories” by Mark Rosenfelder.)

In general, in order to understand a piece of ancient writing, you need to understand the material life that people led back then, and you also need to understand the culture. You also need to understand the canon of previous literature that the writer was drawing on. If you’re not familiar with the Bible, you’re going to have a lot of trouble with European literature from the last 1000 years, since most of it is sprinkled with biblical allusions.


One of the biggest threats to knowledge is the people who actively want to destroy it.

I’ve said that knowledge and technology are mutually reinforcing, but knowledge and society reinforce each other too. The society preserves the books, and the books contain the ideals that form the foundational principles of the society.

If you want to destroy a society, and make sure it never returns, then you need to destroy everything that society has ever written. If you don’t, then people in your culture might discover those books and be influenced by them.

This is why acts of destruction so often focus on the intellectual content of a society. The library at Alexandria was burned by Muslim conquerors who wanted their ideology to be the only one. Legend states that:

“John the Grammarian” asked Amr to spare the library, and Amr contacted the caliph Umar for authorization. Umar replied that if the books agreed with the Quran they were redundant, and if they did not, then they were forbidden. Amr handed the books over to Alexandria’s heated bath houses, where they were burned as fuel for six months.

And there have been many occasions, throughout history, where conquering societies have burned the temples or religious relics of the people they conquered. That, along with burning books, is an effective way of destroying a culture.

Even in our own society, there are those who want to destroy knowledge. Environmentalists, who think our technologies are wasteful and destructive, might be happy to get rid of the knowledge that we need to construct industrial machines. Those who hate warfare might be happy if we lost the knowledge for making nuclear weapons. Those who are afraid of a future of designer babies might be glad if we lost the knowledge for genetic engineering.

It’s a common theme in post-apocalyptic fiction that knowledge was destroyed on purpose, because it was that knowledge that people used to create the weapons that wiped out society.

So if you want to preserve knowledge, it’s not merely a matter of protecting the physical objects, and making sure you can still understand the knowledge they contain. You also must defend your books or inscriptions against those who will want to destroy them.


I’ve written this all as a hypothetical, but we really have lost a lot of knowledge over the ages. Some of it has disappeared “naturally”, the books simply getting lost or disintegrating as the years went by. Some of it has been deliberately destroyed.

We will never know how much has been lost. But we can catch glimpses of the past’s intellectual wealth from the few hints that survived through the ages. In ancient literature, we find stranded references to books that have been destroyed: lost pieces of mythology, lost works by Archimedes and Plato, books of the Bible, and non-canonical gospels from the early days of Christianity.

We also know about lost knowledge from ancient technologies that, even in the modern world, we struggle to recreate. No one knows how the Egyptian obelisks were raised onto their bases. The secret of Roman concrete was only recently rediscovered. Scientists are trying to recreate the recipe for garum, a fish sauce that ancient Romans used as a condiment. No one knows exactly what Greek fire (an incendiary that could float on water) was made of, and nobody knows how they produced Damascus steel.

Some technologies got lost during the dark ages, and weren’t rediscovered until a thousand years later during the Renaissance. The ancient Greek Antikythera mechanism predicted astronomical events with a sophistication unrivaled until the 1300s, and the Romans built their aqueducts to a level of precision that wasn’t equalled until the modern age.

And there are some technologies that we’ve lost simply because they no longer seem useful. Canning was only invented in 1810, after Napoleon offered a large financial reward to anyone who could more effectively feed his troops. Before that, people preserved meat by drying it, smoking it, or home-curing it with nitrates. People preserved vegetables using lacto-fermentation, a process which may be more nutritionally beneficial than canning is. Who knows what other food preservation techniques we’ve lost, simply because they seemed useless compared to modern technology?

We will never know how much has been lost to the vaults of history. We only have these few scraps that have come through to us through the archaeological or literary record. How much has simply disappeared, never to be seen again? If you think that nothing of real substance could be lost so easily, consider that for thousands of years, everyone simply forgot that the Sumerians existed.


So if we value knowledge, what can we do to prevent its loss in the event of an apocalypse? How can we ensure that as many books as possible are carried into the future, and that people retain enough understanding to use the knowledge that they store?

Well, for one thing, we can teach people that knowledge is sacred, and that without knowledge, we could not have the way of life that we currently enjoy. We can teach people that knowledge is what lifted us up out of past ages, what allowed us to create civilization. We can teach people to hold onto knowledge even in the face of calamity.

But we can also write books with the fragility of knowledge in mind. We can write books that store the essential knowledge of our civilization, and we can write them in a way that makes them accessible to the people of the future.

In Finland, there is a book called the Taitokirja (“skill book”), whose pages contain detailed instructions on how to build various tools and perform various tasks. If I remember correctly, the tools range from simple wood and stone contraptions to modern industrial machinery. The book is intended to allow people to teach themselves these skills, in the event that they are forgotten.

I hear that there is a similar book in English, The Knowledge: How to Rebuild Civilization in the Aftermath of a Cataclysm by Lewis Dartnell. But I haven’t seen this book, so I can’t tell you exactly what it contains.

This is not a book, but the Global Village Construction Set contains instructions for building machines that can be used to bootstrap our way back to modern society in the event that our technology is lost.

On a different note, it is eerie to contemplate this inscription that was written for people 10,000 years in the future, warning them of radioactive waste. There’s no reason to believe that people in that time period would understand what radiation is, so they hired anthropologists to word the message in a way that would ward off people from various cultures. And in case the inscription itself was unreadable, they designed the landscape to be foreboding and to contain ominous forms that would threaten people away from the area.

I recently read the classic novel A Canticle for Leibowitz, which centers around an order of monks, living in a post-apocalyptic future, whose holy task is to collect and preserve any textual remnant of the past. That’s the sort of reverence that you need for preserving knowledge — meticulously copying ancient texts, even if you don’t understand what they mean, in hopes that they’ll be valuable to people of the future. Or simply because the texts themselves, as relics of the past, are sacred.


I have no real conclusion to this post. I simply wanted to convey the awe I have felt ever since I realized how precious our knowledge is, and how easy it is to destroy. I hold each book with reverence now, and I look at our world with a new sense of wonder. We are living in an era of unprecedented technology. Will our society continue to grow and expand? Or will it collapse, the way so many societies have before, leaving us to pick up the broken pieces from the ruins of our world? And if our society does collapse, how much of what we hold dear — our technologies, our philosophies, our novels and legends and stories — will be forgotten so entirely that future societies will never even know they existed? It makes me want to treasure the knowledge we have, for as long as we have it.


I’ve been obsessed with this topic for over a year now. Thanks to everyone at work who has put up with my incessant conversations about it. Particular thanks to my friend Kirby, who sent me many of the links included here.

I haven’t read any of these yet, so I can’t tell you whether they’re any good, but people have recommended the following books about the fragility of knowledge:

  • How the Irish Saved Civilization, by Thomas Cahill
  • 1491: New Revelations of the Americas before Columbus, by Charles C. Mann
  • Bread, Wine, and Chocolate: The Slow Loss of Foods We Love, by Simran Sethi
  • The Wake, by Paul Kingsnorth

For a visceral understanding of how much a culture can change in a few thousand years, and how much knowledge can be lost, I highly recommend the novels Riddley Walker by Russell Hoban and Engine Summer by John Crowley. Both are set in the post-apocalyptic future, and both contain a lot of very reasonable, but completely incorrect, misinterpretations of the artifacts of our time.

(If you have other books to recommend, please leave a comment! I would love to add more books to this list.)

Posted in Uncategorized | Leave a comment

Shattered Dichotomies

A couple years ago, I was browsing Facebook, and I stumbled across this quote from Marvin Minsky:

Could Computers Be Creative? I plan to answer “no” by showing that there’s no such thing as “creativity” in the first place. I don’t believe there’s a substantial difference between ordinary thought and creative thought.

I found this very interesting, because at the time, I was reading The Way We Think by Gilles Fauconnier and Mark Turner, and they also claimed that we use the same cognitive mechanisms for ordinary thought as we do for creative thought. Except instead of concluding that creativity doesn’t exist, they used this to argue that all thought is creative.

So which is it?

I think the answer is: the question is meaningless.

As a society, we rely on this conceptual distinction between ordinary and creative thought. Ordinary thought is the kind we do all the time, when we’re thinking about our shopping lists or putting gas in our cars. Creative thought is rarer; it appears in moments of inspiration, and results, perhaps, in art or poetry.

Or at least, that’s what our folk models lead us to believe. Our culture has an almost mystical view of creativity, where only a few gifted souls are able to produce art. And we look on them with a sort of reverence, thinking of them as different, lost in their own rarefied realm of shape and color and rhyme. Artists are sensitive, our folk models tell us. They think in a different way than other people.

And we use this model when we’re making inferences. When an artist forgets to pay the rent, we assume she’s so lost in creative thought that she can’t be bothered with earthly concerns. We view her disorganized nature as intimately tied to her artistic abilities, and we forgive her accordingly. But when a dental hygienist forgets to pay the rent, we just assume she’s lazy, and we wonder why she can’t keep track of what day it is.

I could go on for pages talking about the difference between “ordinary people” and “creative people” in our culture’s folk model of the world. But the important thing is, we view them as two separate groups of people. We view ordinary and creative thought as two separate phenomena. And we use this distinction when we’re reasoning about the world.

So when someone comes in and says “ordinary thought is the same as creative thought”, what do we do? How does this affect our reasoning process?

Well, if we really believe that these two things are the same, then we should throw out the dichotomy altogether, since it means our whole framework is wrong. It means there’s no such thing as “ordinary thought” or “creative thought”, there’s just “thought”, and so we can’t trust either of the original categories for inference.

But in practice, that’s not what happens. In practice, we keep the dichotomy, but resolve everything to one side of it or the other. We either decide that “all thought is creative”, in which case all thought is special and all people are artists in some sense, or we decide that “no thought is creative”, in which case all thought is ordinary and artists aren’t any different from the rest of us after all.

And this happens all the time, with all sorts of shattered dichotomies.

For instance, back in high school philosophy class, I used to argue that “all people are selfish”. If you’re hurt, and I go to help you, it’s not because I’m altruistic. It’s because the sight of you in pain causes me to feel pain, and I, selfishly, want to relieve my own pain (or I want to avoid the guilt I’d feel for not helping). Similarly, if I give you a gift, it’s not because I’m altruistic; it’s because I selfishly want the pleasure and satisfaction that comes from gift-giving.

In high school, I thought this was a great argument. As an adult, I roll my eyes. It’s not that the argument is wrong, per se; based on the definition of “selfish”, it really is possible to classify all actions as selfish. I just don’t think it’s useful. Our folk concepts of “selfless” and “selfish” might be fuzzy and imprecise (as all concepts are), but they help us navigate a complicated world. When you realize that your friend Mike is selfish, you might decide to hang out with him less, or to avoid doing him favors because you know they won’t be reciprocated. And when you’re deciding whether to give your friend Steve a ride to the airport, you might agree to do it, because you don’t want him to think you’re selfish.

(Though maybe the shattering of this dichotomy can be useful for some people! If someone suffers from scrupulosity, and is wracked with unnecessary guilt that they’ve chosen the selfish option too often, then completely removing the distinction between “selfish” and “selfless” could be exactly what they need.)

Another shattered dichotomy I’ve encountered is the people who argue “A city is just as natural as a pristine forest, because cities were made by humans, and humans are part of nature. A city is just as much a natural structure as a bird’s nest or an anthill.” I neither agree nor disagree with this argument; it’s really just a matter of what you want the concepts to mean. And that, in turn, will depend on what you’re using them for. A lot of people (myself included) find natural landscapes beautiful, but also find industrial complexes ugly. And, while I’ll always probably find refineries ugly on a visceral level, this argument helps me appreciate them as part of the ecosystem of human activity, which I do in fact find beautiful. So in that particular instance, I appreciate the shattering of the dichotomy. But when someone says “pollution in the Shenandoah river isn’t a big deal, because industrial waste is just as natural as fish poop”, then I’m going to object, because they’re relying on the standard inference “natural => harmless”, and they’re trying to get you to classify pollution as natural so you’ll think of it as harmless as well.

Anyway, there’s no bigger point to this post. I just find this to be a really interesting cognitive phenomenon, both in terms of how human concepts work, and how we use them in framing and rhetoric.

Posted in Uncategorized | 1 Comment

You can change people’s minds without changing their beliefs.

(This is not part of the Postrationality series. It’s just an isolated thought that I wanted to share.)

In order to change someone’s mind, you don’t have to change their beliefs. You just have to change their associations.

Let me unpack that a bit. By “change someone’s mind”, I mean change it in a way that affects their actions. In the rationalist community, we tend to see beliefs as the be-all and end-all of decision making. Based on our beliefs, we should choose our actions to maximize expected utility, and that’s all there is to decision-making. But in practice, beliefs are only part of our reasoning and decision-making processes.

Let me give you a (pretty obvious) example of how we can fail at decision-making despite having correct beliefs. Suppose your friend is coming over tonight, and you’re planning to make dinner. You know that your friend is a vegetarian, but when you’re at the supermarket, you forget this and buy chicken. This mistake leads to a loss of utility, since either you serve your friend chicken for dinner, or you eventually remember and have to run back out to the store. In either case, a failure of reasoning occurred, and it led to a loss of utility. But the problem here wasn’t with your beliefs; you knew about your friend’s dietary preferences. The problem was with your memory, and which beliefs you actually used when you were making the decision.

So that’s the point I’m trying to get at here: your decision doesn’t just depend on your beliefs; it also depends on which specific beliefs you actually use when you’re deciding. And we can’t just use every belief, because there’s too many to reason with efficiently.  So we have to do approximate inference, and restrict ourselves to a subset of our beliefs.

Fortunately, most beliefs will be totally irrelevant to a given decision. If you’re choosing what to buy for dinner, then it doesn’t really matter that DNA is stored in the nucleus of the cell. This means that in order to make good decisions, you need to figure out which of your beliefs are most relevant, and use those and only those when reasoning. In the example above, where the person bought chicken to serve to a vegetarian friend, that was a failure at retrieving and using a relevant belief. In general, when you forget something important, you are failing to retrieve a relevant belief.

This is where associations come in, because we use them in our belief-retrieval systems. That is, let’s say your vegetarian friend’s name is Steve, and he plays in a metal band, he frequently wears mismatched socks, and one time he went bungee jumping off the Eiffel Tower. These are all things you associate with Steve; that is, when you think of Steve, these are the facts that might spring to mind. If we want to get slightly more formal, we can imagine that the mind contains a network of facts/entities/ideas, and that every pair of these has a strength of association between them. These strengths will be mediated by context, so that in the context of cooking Steve dinner, your association score might grow stronger to facts about his food preferences, and weaker to facts about his socks. This means that in order to remember that Steve is a vegetarian, you either need a strong base association score between “Steve” and “vegetarian”, or you need the context of cooking Steve dinner to increase the score enough that it passes some threshold of relevance.

This is why, if you want to change how someone acts, you don’t need to change their beliefs. You just need to change their associations. Change which facts they (subconsciously) decide are relevant to the situation. Change what comes to mind when they think of a person or organization. The media does this all the time; it doesn’t even have to lie. It just has to broadcast information selectively. Suppose there’s a politician running for office, Senator Dick Head. You know that Senator Head once donated $5,000 dollars to protecting the short-snouted snail, a cause that is dear to your heart. But he also cheated on his wife, and you find this morally repugnant. The media doesn’t care about the short-snouted snail, so it never reports on his donation. But the news channel you watch is constantly telling you what a horrible awful cheater Senator Head is. So your association between “Senator Head” and “cheated on his wife” gets stronger, while your association to “cares about the short-snouted snail” remains weak. This means that by voting time, you are truly disgusted with Senator Head, and you vote for his competitor, Congressman Mike Rotch, instead.

So, in conclusion, reasoning is not just about which beliefs you possess. It’s also about which beliefs you actually use during a specific reasoning task.  Thus, if you want to change someone’s mind, you don’t have to change their beliefs.  You just need to change which beliefs they’re likely to use when reasoning.

Note: this post is not science. I cannot cite research that supports anything I just said (though I do think it’s reasonable, or I wouldn’t have written it). And I don’t know any mathematical models that reason by choosing beliefs according to strengths of association. I do know some researchers are working on how to choose which beliefs to use, but I have no idea how they’re going about it. So please don’t take anything I’ve said here as scientific fact. This is just informed speculation.

Posted in Uncategorized | 3 Comments

Postrationality, Table of Contents

A couple of weeks ago, Scott Alexander posted a map of the rationalist community, and much to my delight, I’m on it! Specifically, I’ve been placed in the country of Postrationality, alongside Meaningness, Melting Asphalt, Ribbonfarm, and A Wizard’s Word. This is truly an illustrious country, and I’m honored to be a member of it.

But anyway, as a result of this map, a lot of people have been asking: what is postrationality? I think Will Newsome or Steve Rayhawk invented the term, but I sort of redefined it, and it’s probably my fault that it’s come to refer to this cluster in blogspace. So I figured I would do a series of posts explaining my definition.

As you might imagine, postrationality has a lot in common with rationality. For instance, they share an epistemological core: both agree that the map is not the territory, and that concepts are part of the map and not part of the territory, and so on. Also, the two movements share some goals: both groups want to get better at thinking, and at achieving their object-level goals.

But the movements diverge in the way that they pursue these goals. In particular, rationality tends to give advice like “ignore your intuitions/feelings, and rely on conscious reasoning and explicit calculation”. Postrationality, on the other hand, says “actually, intuitions and feelings are really important, let’s see if we can work with them instead of against them”.

For instance, rationalists really like Kahneman’s System 1/System 2 model of the mind. In this model, System 1 is basically intuition, and System 2 is basically analytical reasoning. Furthermore, System 1 is fast, while System 2 is slow. I’ll describe this model in more detail in the next post, but basically, rationalists tend to see System 1 as a necessary evil: it’s inaccurate and biased, but it’s fast, and if you want to get all your reasoning done in time, you’ll just have to use the fast but crappy system. But for really important decisions, you should always use System 2. Actually, you should try to write out your probabilities explicitly and use those in your calculations; that is the best strategy for decision-making.

Postrationality recognizes that System 1 and System 2 (if they even exist) have different strengths and weaknesses, and what we need is an appropriate interplay between the two. Postrationality understands that emotions and intuitions are often better at decision-making than explicit conscious reasoning (I’ll discuss this in more detail in the second post). Therefore, postrationality tends to favor solutions (magick, ritual, meditation) that make System 1 more effective, instead of trying to make System 2 do all the work.

Here are some other things that seem to be true of postrationalists:

  • Postrationalists are more likely to reject scientific realism.
  • Postrationalists tend to enjoy exploring new worldviews and conceptual frameworks (I am thinking here of Ribbonfarm’s “refactoring perception”).
  • Postrationalists don’t think that death, suffering, and the forces of nature are cosmic evils that need to be destroyed.
  • Postrationalists tend to be spiritual, or at least very interested in spirituality.
  • Postrationalists like (and often participate in) rituals and magick.
  • When postrationalists are trying to improve their lives/the world, they tend to focus less on easily quantified measures like income, amount of food, amount of disease, etc., and instead focus on more subjective struggles like existential angst.
  • Postrationalists enjoy surrealist art and fiction.

This may seem like a rather disjointed list, so one of the purposes of this series will be to show how these tendencies all fit together, and in particular how they all derive from the basic postrationalist attitude towards life.

My current plan is to include three posts in this series (which I’ll link to as they become available):

  • A post explaining the rationalist perspective, including the System 1/System 2 model of the mind, the need to overcome bias using our analytic reasoning skills, and a strange form of Bayesianism where people actually try to do explicit calculations with their subjective probabilities.
  • A post explaining why the rationalist perspective is misguided.
  • A post examining the attitudes held by the two communities. This will be the most important post, since at the heart of it, rationality vs. postrationality is not a factual disagreement, but a disagreement of attitude. I will try to show how the postrationalist attitude (one of accepting the world and our own humanity) gives rise to the bullet-pointed list of tendencies that I showed above.

As a final note, I should probably mention: this definition of postrationality is purely my own. In particular, it does not necessarily represent the viewpoint of the other Postrationalists on Scott’s map. So if you’re on that map, and you think the definition of postrationality should be different than the one I’m giving here, then I hope you will leave a comment and let me know!

Posted in Uncategorized | 22 Comments

Even the Ugliness of the Universe Is Beautiful

The universe is a chasm of inconceivable space, surrounding us dizzily from all directions. We are afraid of the distance between adjacent stars and we are afraid of the distance between adjacent atoms; any open space is a breeding ground for phantoms. When confronted with the unknown and the wild, we have two choices: to build strength enough to join the wilderness, to revel in its fathomless wonders; or to hide within our fear, to tear down everything we can’t control and build walls to insulate ourselves against the sky.

I am trying to follow the path of strength, but it’s such a steep and narrow road. I want to look at the universe unflinchingly, to meet the eyes of God and hold his gaze. But the two ravens, fear and desire, circle above me; they try to push me off the pathway into the endless black abyss.

Our eyes were not meant for the universe in its rawness. Cognitive science reduces the human mind to mechanical computation. Evolutionary biology shows us that everything we do is rooted in selfishness. Quantum physics is maddeningly impossible to interpret. If we dwell on these things too long, we may find ourselves swallowed by insanity.

Cognitive science was my own personal bane. I got caught in the trap of watching each of my thoughts unfold, seeing how the analogical links I made were shaping my understanding of the world. It became impossible to believe in any thought or reason I concocted, because I could easily see how each thought arose and how many alternatives were possible.

And so I was almost ready to turn back, to retreat to the ancestral forest and abandon my quest for knowledge. But now I understand: if the discoveries of science seem ugly, if they warp our minds into madness, it is only because this knowledge was not meant for Man. We are digging deeper into these questions than evolution has prepared us for, and we’re finding that the universe is stark and alien and Other. If I’m disheartened by the knowledge that I’ve gained, it’s because I have started to pierce through the veil of human illusions; I am starting to see the universe as it truly is.

And so I will continue on my quest, armed with this understanding: even the ugliness of the universe is beautiful; even my descent into madness is beautiful. We are dealing with cosmic mysteries that were not meant for the eyes of Man. I will climb this steep and narrow road, even though the abyss still yawns before me. For now, when I look into its depths, I see that it is full of stars.

Posted in Uncategorized | 9 Comments

Identity and Bureaucracy


Lately, the internet has been awash with new gender and sexual identities. On the gender side, the strict dichotomy of male and female has given way to a proliferation of possibilities, including agender, transgender and gender fluid; these categories have entered the public consciousness to the point where Facebook recently changed the way it handles gender, allowing users to pick from fifty-six different options instead of just the usual two. As for sexuality, the choices are no longer limited to heterosexual and homosexual; the list has grown to include asexual, sapiosexual, and demisexual as well.

As usual when society changes, we see a lot of people lauding this trend as the next big step towards freedom, equality, and acceptance, and we also see a lot of people condemning this trend as a sure sign that society is headed straight to hell. Both views have their merits, and I don’t want to argue about which one is right. Instead, I want to ask a different question: why is society changing in the first place? What sorts of cultural and environmental pressures are causing people to be dissatisfied with their default genders?

All sorts of explanations have been proposed. One is that people have always longed for this freedom to choose their own gender, but up until now, society has been too bigoted and close-minded to allow it. Another explanation blames plastics and other industrial chemicals for interfering with our hormones, causing many people to feel at odds with their biological gender.

In this post I’d like to put forth another potential explanation, which is that our cultural obsession with fine-grained gender identities is a natural consequence of living in a rigid bureaucratic society.


As Ribbonfarm explains, in order to function, bureaucracies require the world to be legible.

The idea of legibility is rooted in our human need for order. The neater and more organized the world is, the easier it is for us to process and interact with it. From Ribbonfarm:

In Mind Wide Open, Steven Johnson’s entertaining story of his experiences subjecting himself to all sorts of medical scanning technologies, he describes his experience with getting an fMRI scan. Johnson tells the researcher that perhaps they should start by examining his brain’s baseline reaction to meaningless stimuli. He naively suggests a white-noise pattern as the right starter image. The researcher patiently informs him that subjects’ brains tend to go crazy when a white noise (high Shannon entropy) pattern is presented. The brain goes nuts trying to find order in the chaos. Instead, the researcher says, they usually start with something like a black-and-white checkerboard pattern.

The idea of legibility is as follows: when a system is so complex that we can’t process it, we change that system to make it simpler. Ribbonfarm gives the example of “scientific” forestry:

The early modern state, Germany in this case, was only interested in maximizing tax revenues from forestry. This meant that the acreage, yield and market value of a forest had to be measured, and only these obviously relevant variables were comprehended by the statist mental model. Traditional wild and unruly forests were literally illegible to the state surveyor’s eyes, and this gave birth to “scientific” forestry: the gradual transformation of forests with a rich diversity of species growing wildly and randomly into orderly stands of the highest-yielding varieties. The resulting catastrophes — better recognized these days as the problems of monoculture — were inevitable.

Bureaucracies are known for being rigid, dehumanizing, soul-sucking things, and it’s easy to see how legibility is responsible for this. Every piece of paperwork makes the world more legible, by distilling the complexity of our lives down to a few discrete fields. When you apply for a job, your application will be reviewed by some guy hunched over his desk, reading through 500 similar applications while drinking his third cup of coffee that morning. He doesn’t care about you as a person, in all your glorious uniqueness and complexity. He just wants to get through your application as quickly as possible. The job application form makes his life easier because he can see at a glance where you went to school, what your previous work experience is, and so on. It lets him decide very quickly whether you’re qualified for the job. The paperwork makes you legible to him.

But it also means there’s no room for individual differences and special cases. If you never went to college and you have no work experience in that field, the guy at the desk might throw away your application, even if you’re self-taught and brilliant and you really would be the best person for the job. And thus all of us have learned: the system does not reward people who are brilliant and capable. The system rewards people who are brilliant and capable and willing to play by its rules. The system is dehumanizing because it reduces a whole, complicated, intricate human being down to a handful of statistics.

As this example hopefully makes clear, bureaucracies don’t do this because they’re evil. They do it because they’re in a hurry and they don’t have time to consider the complicated details of everyone’s individual lives. Bureaucracies are like assembly lines. Before assembly lines, you had a whole bunch of craftsmen each making just a few items at a time. If a woodworker made chairs with two different shapes of legs, that was fine, because he could build two different chair-seats, one that went with each pair of legs. But in a factory, it’s essential that all parts be identical; that’s what makes the assembly line run quickly.

Analogously, when the world was smaller and less centralized, it used to be that a few individuals could gather together and work problems out for themselves. Because these people were operating on a small scale (three or four people rather than hundreds, thousands, or millions), it was possible to deal with each person at high resolution; it was possible to take everyone’s complex personalities into account when devising a solution. But a bureaucracy is dealing with thousands and thousands of cases at a time; it doesn’t have time to devise a unique solution for every individual problem. So instead, it pattern-matches each problem to some general class of problems, and applies a one-size-fits-all solution.


Now let’s return to the original topic of the post. Why on earth would rigid bureaucracies cause people to develop new gender identities?

To answer that, let’s consider the following situation. Suppose you’re a grad student, and your life has been pretty stressful lately. Maybe your girlfriend just broke up with you, or maybe your mom is in the hospital. Whatever the reason is, you’re having trouble focusing on your schoolwork, and you decide you want to take a semester off. Well, if your school is anything like mine, then in order to take a semester off, you have to apply for a leave of absence, which they’ll give you if you have a medical condition, a family hardship, or you need to do military service. Family hardship covers the “mom is in the hospital” case, but what about the guy whose girlfriend just broke up with him? Well, it turns out there’s a medical condition that corresponds to his problems, and it’s called “depression”. So all he needs to do is go to a doctor, explain what’s going on in his life, and get a diagnosis.

I have a lot of problems with how our culture views “mental health issues”, but that’s not the point I’m trying to get at right now. I don’t want to debate whether this hypothetical student is actually depressed, or whether depression is actually a medical condition. Instead, I want to point out that pasting a label of “depression” onto this guy’s life didn’t change his situation in any way. He was just as depressed about his breakup before a psychologist filled out an official-looking form as he was afterwards. And yet, prior to receiving that label, this guy was not qualified for a leave of absence. After receiving the label, he was. The label didn’t change his problem; it just made it visible to the bureaucracy.

The point I’m trying to get at is that our bureaucratic society is sending us a powerful message: until your problem has a name, it doesn’t exist.


And this is grad school, where people are treated as individuals to the point where every student is personally mentored by a successful researcher in the field. Grad students have it easy compared to the elementary, middle, and high school students at your average public school. If a 10-year-old with Asperger’s gets overwhelmed by all the noise and commotion in gym class, the gym teacher can’t just notice this and allow the student to sit out. The parents need to take their child to a psychologist, procure a diagnosis of autism, and bring this to the school; only then can any action be taken.

I don’t mean to say that this system is all bad. If the gym teacher is ignoring the problem, a note from the doctor can force her to take it seriously. And on the other hand, requiring a doctor’s note keeps the kid from faking an illness, or the teacher from playing favorites.

But this system does have its consequences. Once the child is labeled as “autistic”, the diagnosis can never be taken back. It will color how the parents view their child’s behavior, and ultimately influence how the child views himself. This makes a diagnosis of autism different from, say, a diagnosis of diabetes. Both are permanent conditions, and knowing about either of them will change how the child interfaces with the world. But the symptoms of autism, unlike those of diabetes, cover aspects of one’s personality and preferences that have traditionally been included as part of the self. This makes autism compelling as an identity label in the way that diabetes is not.


Psychiatric diagnoses are everywhere these days. We are faced with a generation of children and young adults who have received these diagnoses, and who see them as a fundamental part of their identities. And it was the act of getting diagnosed, the act of having these identities recognized officially, that allowed these students’ individual differences to finally be taken seriously.

In our label-driven society, receiving the right classification is essential for ensuring that you are treated in a manner that befits you as a person. That’s why it’s important to find a set of labels that fit you well, and to make sure that those labels are accepted by society at large.

So is it any surprise that people are seeking out finer-grained gender identities, ones which describe their personalities better than “male” or “female” could? It is a surprise that people consider these labels so incredibly important?

My prediction is that the new gender identities will be embraced most strongly by people who have also strongly embraced one or more psychiatric diagnoses. And this prediction seems to be borne out by the number of people on the internet who introduce themselves by some combination of gender and psychiatric identities. “Hi, I’m a non-binary asexual submissive with anxiety and depression.”


So that’s my answer. Why do we create these identity labels, these ever-finer-grained descriptions of who we are? Maybe it’s because we live in a rigidly bureaucratic society, where our individual differences will only be noticed if they have a label attached. Maybe it’s because we’re used to having our dimensionality reduced down to a few searchable keywords. Maybe we’re trying to make ourselves visible by making ourselves legible. Maybe we, as a culture, have internalized the idea that if something doesn’t have an official label, it might as well not exist.

Posted in Uncategorized | 7 Comments