Parsing trends in Congressional speech

I just saw something thought provoking: “The rapid drop in intelligent Congressional speech”. It’s based on this analysis from the Sunlight Foundation, drawing out some comparisons between famous speeches & documents from years gone by and this chart of trends in the language complexity of congresspeoples’ speeches over the past 15 years or so:

Trends in speech complexity chart

Much as I want to read this chart as part “political discourse today has gone down the toilet” and part “those damn Republicans debasing political discourse”, I see several really important caveats:

  1. Is this statistically significant? My gut tells me it must be because it’s a consistent enough pattern, but it’s also a pretty small difference in scores overall and gut checks are no substitute for significance testing.
  2. It’s based on Flesch-Kincaid reading level scores – is this a useful measure? One gets a higher score by using longer words and longer sentences – not particularly good public speaking technique, nor a reliable benchmark of the complexity of ideas, and I would expect pre-20th Century prose to score higher just because that’s how the language has evolved (for an example, read Conrad, Greene & Grisham in sequence, and remember that each was writing popular thrillers in their time). For that reason I’m not at all surprised that MLK scored lower than Lincoln: each was using the language of their time, pitched right for a broad audience.
  3. I don’t think it makes sense to compare speeches with written documents – a successful speech is written using simpler language than a good constitutional document, precisely because of the ways in which it’s expected to be used. So the only comparison with pre-20th Century touchstones that makes sense is the Gettysburg Address, which at 11.2 is apparently on a par with the simpler speeches in 96-2005.
  4. Related to the previous two points: remember that before widespread radio, even addresses that were given as speeches would have reached most people as a transcript, whereas these days far more people will listen to a speech than read it. A good speech writer will be very conscious of this, and a good off-the-cuff speaker will have trained themselves differently in this light.
  5. Finally, I don’t remember the source, but there was a study in the 90s of Canada’s parliament that showed opposition parties consistently giving the simpler speeches. The authors’ contention was that in opposition it’s much easier to simply point out criticisms, while the governing party has to defend its actual record and put forward plans that won’t embarrass it in the coming months. The party trend in that chart above certainly fits this model, and I wonder if controlling for it would explain the whole difference between parties?

This entry was posted in Meeja and tagged , , , , . Bookmark the permalink.