Statistical analyses of literature: let’s see what happens

I got some pushback to the link on what heretical things statistics can tell us about fiction, and I’ve read pushback like it before: the objections tend to say that great literature can’t be reduced to statistics; big data will never replicate the reading experience; a novel is more than the sum of the words chosen. That sort of thing. All of which is likely true, but the more interesting question is, “What kinds of things is nobody doing in the study of fiction?” (Or words, or sentences, of writers’ oeuvres). Lots and lots of people, including me, closely study individual works and connect them to a smallish body of other works and ideas.

Over centuries, if not longer, thousands, if not millions, of people have engaged this practice. Not very many people have attempted to systematically examine thousands if not millions of works simultaneously. So that may tell us something the usual methods haven’t. It’s worth exploring that domain. And just because that domain is being explored, the more usual paths via close reading aren’t closed off.

In other words, don’t think that an argument along the lines of “x is interesting” means “we should always and only do x.”

At the moment, we also appear to be at the very start of the field. Maybe it’ll become extremely important and maybe it won’t. The potential is there. People have (arguably) been doing some form of close reading and analysis, even if the practice didn’t use those specific words, for millennia. Certainly for centuries. So I’d be pretty surprised to see statistical analyses produce whatever good material they’re likely to produce in just a decade or two.

Part of what art and analysis should do is be novel. Another part is “be interesting.” We’re looking for the intersection of those two zones.

Briefly noted: The Magicians, re-read, and the TV show

The Magicians holds up well (and the link goes to my original review). What stands out still is the relentless focus of Quentin on happiness: I’d guess that the word appears at last a dozen times, and maybe more, in the novel—too often for anyone who is actually happy to think about it. Quentin’s melancholia is a sort that, if it can be cured, cannot be cured in the ways in which he is attempting to cure it. Don’t be fooled by the magical trappings: the novel is still primarily psychological.

Between now and then The Magicians has been made into a disappointing TV show; that show has high points and funny moments but it cannot overcome a fundamental problem that is illustrative for other writers: it advances all of the characters’ ages by five to ten years, which defeats much of the point and pleasure of the book. The book is about coming of age. It is stuffed with references like this one, from late in it, when (I don’t think this gives anything away) most of the main characters make it to Fillory: “For all the glory of their high and noble purpose, it felt like they were going on a summer-camp nature hike, or a junior high field trip, with the kids goofing on and the two counselors looking dour and superior and grown-up and glaring them back into life when they strayed too far” (one decent definition of being grown-up is that you are no longer concerned with appearing grown up (or not)). It is hard to feel glorious and “noble” when you are being supervised by adults who’ve really seen the world, as Dint and Fen (their guides) have, or apparently have.

Characters who are in the 22 – 30 age range are less likely to analogize their lives to summer camps or junior high field trips. This may seem like a minor point at first. In the show, the characters are still angsty, but at their age their style of angst no longer makes any sense, as they ought to have decently developed, decently resilient personalities by then. That they do not is the flaw the show never manages to overcome.

To be sure, The Magicians tv show does have excellent individual moments, but they don’t add up to much. The actor who plays Penny in particular is a standout (unfortunately, there is something off about the one who plays Quentin). Mostly, the show is an exercise in showing why HBO is so good at its shows and the SyFy channel is so not good at its shows. The Magicians as a TV show is a weak show with a strong one lurking obviously within it, which may be the most frustrating kind. The ones that are transparently bad are just passing phenomena. The ones that are transparently good offer their pleasures. The ones that could be good pain.

Briefly noted: Kindle Voyage

For a while I’ve had a Kindle Voyage. It’s functional and the screen is nice. Not much has changed since this 2010 post. Amazon still has no good system for organizing and sorting books, and Amazon doesn’t want you to use desktop computers and that shows in their whole ecosystem design.

The Voyage hardware is, at best, slightly better than the last Kindle iteration I used. Really, though, the improvements are so marginal that I can’t imagine anyone buying the new version unless their old one dies or is lost, as happened to me: Amazon will often knock some money off the new version if you ask them to “repair” the old version. To get the discount, Amazon requires that you send the broken Kindle to them. I don’t know what happens after that. Probably Amazon trashes it, but I’d like to imagine that it’s refurbished.

A lot about the Kindle Voyage is okay. There’s little to love. If you’re going to bother buy a Kindle the Voyage is a better choice than the regular Kindle Paperwhites because it has buttons, albeit buttons that aren’t as prominent or tactile as I’d like.

I don’t use the Kindle for books much, because I still prefer paper and Instapaper is my killer app. At the margins, I now read more nonfiction and fewer books in general, including novels. You’ve probably read or noticed that too many popular nonfiction books are just unsatisfactorily elongated articles. Preferring to read those rather than just clicking the “buy book” button is easier with Instapaper.

This review is thorough and says most of what I’d say. I don’t know how people produce many thousands of words in Kindle reviews. It’s a device without a personality. Which isn’t bad: It just is. There are good use cases for it, but not for me using it.

I still find button presses annoyingly too easy.

 

The Right Stuff — Tom Wolfe

How Tom Wolfe Became … Tom Wolfe” inspired me to re-read The Right Stuff, which is still excellent today and still worth dropping everything to read, today. In the foreward to my edition Wolfe writes that “This book grew out of some ordinary curiosity.” That “ordinary curiosity,” however, didn’t have ordinary results. He notices things that others don’t; few people noticed the possibility for the “Serious treatment of the drama and psychology of this new pursuit, flying high-performance aircraft in battle…” How many people don’t notice fields that today call for serious treatment yet don’t get them?

the_right_StuffIn the book Wolfe recounts, numerous times, the square footage of houses, and, sometimes explicitly and sometimes implicitly, what that square footage means. For one test-pilot couple—the wife essentially assumes her husband’s position in this world—an 1,100 square-foot-house is made bigger by the way the couple “designed it themselves.” The story is often about men who feel they are doing it themselves, though they aren’t: they’re part of a vast human network, and they’re made the figureheads of the network. The Right Stuff can be read well with Kelly: More than My Share of It All, since Kelly is about the engineer and engineering behind the flying machine and The Right Stuff is about the pilots and the lives of those strapped into the nose. Wolfe is a much better writer—there’s no way to ignore that—but while the perspectives differ the romance remains. Wolfe is fond of denigrating technocrats, or having his characters denigrate them—he does, repeatedly, in A Man in Full, for example—but that denigration may spring from the steady elevation of technocrats. Lewis notes as much:

The world needed them to be heroic pilots, and so they played the part, but no one (except for one American writer) thought to look more deeply into the matter. No one noticed the best story. Process had replaced courage. Engineers had replaced warriors. A great romantic way of life, a chivalric code, had been trampled by modernity. Not for the first time! (As Wolfe might write.) It’s the story of the American South in the 20th century—or at least the story a lot of white southern men told themselves.

Was there ever a real chivalric code? I’d guess not: a chivalric code is most useful as a way of waving one’s hand in one direction while the other hand picks a pocket or preps itself for a punch. But hierarchy! That exists and probably always will. Wolfe is towards the top of the hierarchy of writers: he notes, in an almost throwaway moment, how flying does things to “the gyroscope of the soul.” He writes, from the fighter pilots’s perspective, how in flying test craft the very top steadily leave others behind. And, moreover, the test works because it works on belief in masculinity itself:

Why, it seemed to be nothing less than manhood itself. Naturally, this was never mentioned, either. Yet there it was. Manliness, manhood, manly courage . . . there was something ancient, primordial, irresistible about the challenge of this stuff, no matter what a sophisticated and rational age one might think he lived in.

The romance and death are linked. Wolfe notes that “More fighter pilots died in automobiles than in airplanes.” But death in airplanes is news; death in cars is distressingly prosaic. Today, countless billions are spent fighting statistically unlikely terrorism—the snapping hand—while the other hand—the punch hand—is increasing the likely number of people who’ll die on the road. Romance seizes attention and attention is today the scarcest resource in existence. Wolfe gets that, I think, and got it long before most of the rest of us.

Wolfe is unafraid, too, to be enthusiastic:

My God!—to be part of Edwards in the late forties and early fifties!—even to be on the ground and hear one of those incredible explosions from 35,000 feet somewhere up there in the blue over the desert and know that some True Brother had commenced his rocket launch . . . in the X-1, the X-1A, the X-2, the D-558–1, the horrible XF-92A, the beautiful D-558–2 . . .

The sentence rambles on, itself feeling rocketlike. Edwards then is like Silicon Valley today. The center of the world may shift at times, but the keen listeners and seers attend not to where it’s been, but where it’s going. A pity that short-sighted noisy NIMBYs have made it nearly impossible for normal people to visit the center of the universe. Instead, that center has to spawn extra branches in Seattle, Austin, and even New York—New York!—New York is now cheaper than San Francisco. It’s a madness Wolfe would get, with his attention to housing and the status implied by housing.

One more moment from The Right Stuff. Wolfe writes:

To fighter jocks it was bad enough to have doctors of any sort as your final judges. To find psychologists and psychiatrists positioned above you in this manner was irritating in the extreme. Military pilots, almost to a man, perceived psychiatry as a pseudo-science. They regarded the military psychiatrist as the modern and unusually bat-brained version of the chaplain.

The fighter jocks were and are right. Maybe romance isn’t dead.

Why can’t we solve poverty, or solve it through schools?

I’m not that old, and I’ve already seen a lot of proposals for solving “poverty” come and go. Many—think Head Start—are tied up in education. The current debate around education tends to run in two directions: one group wants to improve parenting, or ameliorate poverty, or something along those lines, having seen innumerable correlative studies demonstrating that rich kids on average do better than poor kids at school. The other group—the one I belong to—tends to think that we could do a lot for schools, and especially big urban schools, through some combination of charters, vouchers, and/or weakening the power of teachers’s unions. For more on why the latter group thinks as we do, see the many links in this post.

The first group—the one that wants to attack poverty and what not—tends to say things like cjensen’s: “Statistical studies have long shown that (1) education outcomes strongly correlate with parenting,” to which I replied:

Citations are needed on this: “Statistical studies have long shown that…”

“We”—schools, society, etc.—can’t really control parenting. But we can control schools, and it is probably possible to get substantially better outcomes than the ones we’re getting now, chiefly through better teachers. At the moment, most public school teachers are paid in lockstep based on seniority—CS teachers and PE teachers get the same pay—and can’t be fired after their second or third year of teaching, and that creates a lot of perverse incentives.

Ceras replied with another fairly common sentiment: “Programs exist for this with some positive results. Here’s one from a quick Google search,” and he linked to “Nurse-Family Partnership – Top Tier.”

But innumerable small-scale programs that show limited positive results, but almost none of them scale up, for the reasons Megan McArdle describes at the link:

That pilot program has a huge administrative staff whose sole incentive is to ensure that it is meticulously carried out. In the real world, that curriculum will be put into place by an administrator whose priority list is crowded with everything from mollifying the latest lunatic on the school board[. . . ]

That pilot program is staffed with a narrow band of extremely highly qualified teachers, sifted from the best the environment has to offer. In the real world, whoever happens to be standing in front of the classroom come September 5th has to do it, even if they flunked Remedial Math four times and only got this job because the school board needed a body.

McArdle’s book The Up Side of Down is also good on this subject. Lots of small-scale Head Start programs show promise too, but the program’s effects fade out after a couple years, and on a large scale it hasn’t done anything except provide daycare and jobs. Despite the 40-year failure of Head Start to do what it was intended to do—improve life outcomes for poor, minority kids—there’s a press for it in liberal cities, only now it goes by the phrase “Universal Pre-Kindergarten” (UPK). New York City has a UPK program. Seattle mayor Ed Murray wants one, and he wants to spend a lot of money creating it.

OLYMPUS DIGITAL CAMERALet me return to Ceras’s example. Programs like “Nurse-Family Partnership – Top Tier” (NFP) already operate. I know because I’ve written numerous Health Resources and Services Administration (HRSA) Healthy Start Initiative (HSI) proposals that attempt to do just what NFP proposes. For my real, work-for-money job, I do grant writing for nonprofit and public agencies, so I see citations like the one to NFP all the time. Next time I write an HSI or similar program, I might cite NFP. Doing so isn’t going to make the specific program any better—HSI has been operating for a couple decades, under different names, and hasn’t accomplished much on a large scale, in part because of the scale-up problems described a few paragraphs ago.

Ideas like NFP sound good in the abstract, but the gap between the real world and the proposal world is quite wide. Virtually every idea for improving health, welfare, and education has been funded through some grant program or another, but most people proposing new programs aren’t aware of the old ones—and they aren’t aware of the gap between the real and proposal world. After his $100 million donation to the Newark Public Schools, Zuckerberg has evidently learned this.

So what can “we” do? The people who want to keep the existing structure of education in place usually say they want to fight poverty first. On some level who doesn’t? There are some challenges, however. Poverty is a moving target. It’s usually calculated as a percentage of income, which means that it will always be with us (barring some unforeseen technology, or extinction). In addition, from the perspective of someone in 1800 or 1700 or really anytime before about 1950, we have solved poverty, at least in a material sense. Virtually no one in the United States lacks running water, plumbing, or refrigeration. Almost no one starves to death, and the real problem among the poor is obesity. TV penetration is hovering around 98% of households, and the households without TVs are more likely to be like mine—that is, relatively well-off people who choose not to have a TV.

I’m not saying it’s great to be poor in the U.S., but it’s still better to be poor in the U.S. than to be poor in, say, Nigeria, or Brazil. Globally, there have been innumerable people trying to improve life in the developing world, and many books about why those efforts haven’t been totally effective: Why Nations Fail is good. Dead Aid is good. There are others; you’ll see them at the Amazon links. Developmental economics is an entire field devoted to this question. There aren’t easy answers, because if there were, they already would’ve been found and implemented. To quote Megan McArdle again, “The very existence of a policy issue tells you that it is difficult to solve, either politically or technically.”

Beyond measurement and definitional issues around what one means by “poverty,” consider the history of fighting it. Johnson launched the “War on Poverty” 50 years ago, and even the New York Times (at the link) calls it “a mixed bag,” which sounds charitable to me. There is a large poverty-fighting infrastructure that does some really good things (like Food Stamps, now called TANF), and some less good things. Nonetheless, if poverty could be “fought” successfully, I think it would have already been defeated. That it hasn’t should make us question our approach.

There has also been some regression in terms of culture and behaviors: that’s one important message of Charles Murray’s book Coming Apart: The State of White America, 1960-2010. Most women, for example, are better off having children with a dedicated and ideally married partner, but around 40% of all births are currently to unmarried women. There’s a political argument about why that is and what if anything should be done about it, but the behavioral and sociological changes of the last 50 years are still real.

This has a lot to do with education because, as I noted in the first paragraph, people who are relatively okay with the educational status quo tend to want to address things outside of school first. Diane Ravitch is a great leader for this group. I’ve read two of Ravitch’s books on education—Left Back: A Century of Battles over School Reform and The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education—and to read her work is to respect her knowledge and erudition. She moved from a strong educational reformer who favored charter schools to someone who… I don’t know how to characterize her current position other than to say she doesn’t favor charters or vouchers. She does observe the many ways particular charter schools haven’t done very well, but in my view they haven’t been worse than the urban schools they competed with, and some have done much better.

Overall, Ravitch wants to reduce poverty, but as noted above I’m skeptical of social or government forces to do so. In Reign of Error, her most recent book—I’m not all the way through it—she says that public schools are better than they’re commonly depicted. She’s somewhat right: relatively wealthy suburban schools are okay. But that pretty much leaves urban schools (L.A., Chicago, New York, Newark) to languish, and those are the areas and schools that are most promising for vouchers.

The final thing I’ll note is that a lot of people favor “more” money for schools. Overall, inflation-adjusted funding has roughly doubled on a per-pupil basis, per the New Yorker article, and overall funding is quite high—including in screwed up districts like Washington D.C.’s. The Great Stagnation also discusses this dynamic. So while “more” money for school districts may or may not be a good thing, it’s apparent that more money does not automatically lead to better results.

This has turned into a much longer post than I meant it to be, but, to reiterate a point made above, there are no simple answers. Though this post is long it is shorter than many of the books it cites, and it is much shorter and more fun to read than many of the proposals I’ve written. The number of people who are genuinely interested in this kind of social policy minutia is probably small, as the popular support for programs like UPK shows.

If this is what his admirers think, what do his detractors think?

“Like Austen’s plots, [Henry] James’s lack adventure and suspense. His novels progress at a very slow pace: his characters waver and postpone action interminably, and their conversations revolve awkwardly around unclear goals without ever seeming to reach them.”

That’s from Thomas Pavel’s The Lives of the Novel: A History, which is unlikely to be of interest to non-specialists but is much more interesting than most of its peers in the genre. There are a surprisingly small number of direct quotes and a surprisingly number of plot summaries but I’m going to read to the end. One paragraph also gave me an idea for a novel, which relatively few books do.

I would probably be less even less charitable than Pavel to Henry James, but a lot of old and well-read people say my view of him is likely to change in the future. Nonetheless I am struck by how few non-academics read him.

Rereading Nick Hornby’s High Fidelity

I still laughed aloud many times at High Fidelity, although the jokes are almost all context-dependent and so can’t be quoted without causing a quizzical look that says, “You really think that’s funny?” Flipping through it doesn’t yield anything obvious, but I kept smiling at many moments. This is the closest I can get:

There were some nights with Laura when I’d kind of nestle into her back in bed when she was asleep, and I’d be filled with this enormous, nameless terror, except now I have a name for it: Brian. Ha, ha. OK, not really a name, but I can see where it came from, and why I wanted to sleep with Rosie the pain-in-the-arse simultaneous orgasm woman, and if that sounds feeble and self-serving at the same time—oh, right! He sleeps with other women because he has a fear of death!—well, I’m sorry, but that’s the way things are.

Rob’s voice and attitude carry the book, as does the writing, which is largely about nothing yet still moves rapidly from incident to incident, creating plot, which is easily overlooked in novels like this, such as Wilson’s Flatscreen. The continuous happening in the plot contrasts with the non-happening in many of the characters’ lives.

There are moments of astute observation too, as when Laura says “sometimes you need someone to lob into the middle of a bad relationship like a hand grenade and blow it all apart.” Which is true, even if the hand grenade is often made out to be the bad guy (or girl) in the relationship. Often the grenade is the bad guy. But sometimes he (or she) is the catalyst for doing what should have been done long before. When big, life-changing transitions stop happening on a regular basis, (from high school to college, college to grad school and/or work), it becomes distressingly easy to slip into a single path and lose the willingness necessary to make radical changes, whether in work, the mind, or love.

Rob basically knows as much:

None of us is young anymore, but what has just taken place could have happened when I was sixteen, or twenty, or twenty-five. We got to adolescence and just stopped dead; we drew up the map then and left the boundaries exactly as they were.

Life changes even if you don’t. This should be obvious. It takes Laura to tell him what he should already know; when Rob asks “So what should I be doing?”, she replies:

I don’t know. Something. Working. Seeing people. Running a scout troop, or running a club even. Something more than waiting for life to change and keeping your options open. You’d keep your options open for the rest of your life, if you could. You’d be lying on your deathbed, dying of some smoking-related disease, and you’ll be thinking, ‘Well, at least I’ve kept my options open.’

She’s right. Whatever else you’re doing, you should be doing something. But Rob doesn’t, mostly, and as a result his problems are largely self-imposed. He says:

It’s only beginning to occur to me that it’s important to have something going on somewhere, at work or at home, otherwise you’re just clinging on. [. . .] You need as much ballast as possible to stop you from floating away; you need people around you, things going on, otherwise life is like some film where the money ran out, and it’s just one bloke on his own staring into the camera with nothing to do and nobody to speak to, and who’d believe in this character then?

Rob lacks that intellectual ballast. He only listens to music and doesn’t play; at his level of obsession, connoisseurship and taste should pale compared to making (Rob hooks up with an American singer named Marie and says of her place, “thrillingly, there are two guitars leaning against the wall.” He could have two guitars leaning against his wall, although I think one would suffice). Still, I am struck by the extent to which many YouTube videos can be reduced to “one block on his own staring into the camera with nothing to do,” except talk to an audience that isn’t present. Jenna Marbles is a useful approximation of this idea.

There are moments of poignance and useful articulations of the obvious, as when Rob says:

You run the risk of losing anyone who is worth spending time with, unless you are so paranoid about loss that you choose someone unlosable, somebody who could not possibly appeal to anyone else at all.

Being overly fearful of loss increases the likelihood of loss, and Rob is disproportionately anxious. As a college student dating Charlie Rob is “fretful about my abilities as a lover,” and fifteen or so years later he is still fretful about his abilities as a lover. Eventually shouldn’t he just let the anxiety go and figure out what he’s doing? Though he apparently hasn’t in his economic life so perhaps his love and economic lives reflect each other. Rob is a sort of what-not-to-do when it comes to women. He even says, “There are still enough of the old-style, big-mouthed, self-opinionated egomaniacs around to make someone like me appear refreshingly different.” That might work for him, but the big-mouthed egomaniacs are the way they are because what they do tends to work (link is text but potentially NSFW).

For a guy who thinks a lot about his love life, and pop songs that are almost entirely about love, sex, and romance, Rob appears to know very little about actual women. Most pop culture, however, appears to be highly misleading on this score, which may explain why a pop-culture junkie like Rob is or has been highly misled. People who don’t make a concerted effort to learn about actual women. But this is true of much narrative art, especially American narrative art.

In my reading over the last few days, I’m struck by how much more pathetic Rob seems: as I said before, his problems are largely self-imposed, or imposed by his personality, and the solutions also must come from within. Rob fears the women he’s attracted to, like a fifteen-year-old; he goes to a small gig where Marie plays and afterwords she sells CDs: “We all buy one from her, and to our horror she speaks to us.” Most guys are happy to be talking to the people they’re attracted to, and the same obviously applies to women.

In addition, High Fidelity feels like a period piece: Rob owns a record store in an era when CDs and records are mainstream, and people who want to hear a particular song must track down a physical copy of it. Though I was born into that era it feels very long ago and foreign. So does the difficulty of getting ahold of people through the phone. The default state of more people as “alone” then. Computers are almost totally absent. It also feels highly PC, as when Rob recounts “a terribly unsound joke” that is only mildly funny and not really offensive. Why qualify it by saying that it’s “terribly unsound” when it’s not and when interesting humor by its nature is “unsound,” using Rob’s definition?

%d bloggers like this: