Disciplines Meet: A Methodological Aside

Up to this point I've largely limited my postings to lengthy articles directly pertaining to the foundation, the data and the results of the LEO test.  This post represents the first significant departure first of all because I believe it worthy of note, and secondly because it must be said.

There's something truly encouraging about this project:  The most responsive readers I have read so far include a man generally steeped in statistical analysis (jimdesu), and another generally well-versed in the theoretical foundations of political thought (Mark Zuniga).  Granted, Jim has been involved in this project for a few months now, and we've had some pretty arcane discussions via email pertaining to the model itself, especially considering whether it's best represented as 3-space or a 3D spatial model.  Mark, whom I've known since my freshman year in college and who, if memory serves me well, has pursued careers both in journalism and law, has only lately joined the discussion (whether bounced from Carl or Zathras, I still am left to wonder--I'm thinking Zathras), and he's already had some insightful commentary, especially given his experience in journalism.

Why is it encouraging?  For years the American Political Science Association, of which I count myself a member, has been wracked with division over the state and indeed the very nature of the discipline; specifically, whether to focus on methods of empirical observation and analysis or to focus instead on 'first principles' and the analysis of ideas about political things.  The last two APSA conferences I have attended demonstrated that these two approaches have drifted so far apart from each other that each side either impugns the other or else forgets that the other even exists.  Their presentations and panel discussions are often in different buildings, their research is often mutually incomprehensible, their respective scholars appear unknown to each other, even the publishers hawking their wares set up in different rooms, usually on different floors.  All things considered, I often felt as if I were attending two quite different yet simultaneous conferences at APSA every Labor Day weekend.

That such people as Jim and Mark demonstrate enough interest in this project to comment constructively on it suggests that there is hope after all for theorists and empiricists to come together, that they need not view each other as coming from different worlds.   As I explained to Mark when he observed that this project was "so radically different than anything (he) did for (his) politics degree from UD"  the LEO model is actually a hybrid of political theory and empirical method, indirectly suggesting not only that theorists and empiricists can come together, but that they can actually benefit from each other's insights.

Comments

[Anna]

As someone who did interdisciplinary subject/method research for ::gasp:: nine years before this current, albeit forced, stint totally outside academia, I can only applaud your efforts. Glad to see that the approach is crossing more and more boundaries.

[Mark Zuniga]

Speaking of disciplines meeting, I have to wonder if something akin to the LEO test could be used in vetting Supreme Court nominees. It's not exactly the same thing. However, Republican presidents try very hard to pick SCOTUS nominees who have not taken a public stand on Roe v. Wade but will rule the way they want them to. People are trying to parse prior thoughts of Roberts to predict what kind of judge he'll be. He will have to face the same conflict that Kennedy, O'Connor and Breyer have faced; namely, to what extent to prior court rulings and decades of general jurispurdence are superior to one's own intepretation of the constitution. Roberts will never tell anyone until the case comes up how he'll decide.

Yeah, this is outside the scope of this blog. Wanna make something of it?!!

[rufel]

Yes, the LEO test seems to be the perfect bridge between political science and political philosophy. The strict empiricists, from what I remember from your UTA days slogging through quantitative methodology and statistics, tend to measure current political behavior that is frankly "well, DUH," and doesn't really mean anything important because they don't look to larger, political trends, especially historical trends, and are suspicious of anything that can't be put into a strict mathematical model. The strict theorists tend to wax nostalgic about "the good old days" of some Golden Age and hardly deal with current political issues and seem to be afraid of math.

[jonathon]

Mark,
By no means is your question outside the scope of this blog. I originally built the test table to be blind to the political actor's office or role in the body politic. I still have a methodology article that I havent put up yet, and it suggests source material for legislators, executives, judges, candidates, pundits (even bloggers). I even did an informal spot-test of CNN and Fox News' websites, and it seemed to work there too (One was generally high-E communitarian, the other low-E, and either conservative or libertarian, depending on the article). It also worked for the first chapter of Mark Palmer's book, _Breaking the Real Axis of Evil_(High-L libertarian ideologue) The original LEO test is a text-analysis tool suitable for any political actor who has a written record.

Since Judges make speeches, as well as publish opinions, a LEO analysis is certainly possible to determine general ideological preference. Specific policy positions, while not exactly revealed by the LEO test, nevertheless could be predicted in terms of probability as a function of one's LEO score. For example, high-O with low-L may predict an anti-abortion position, while High-L with high-E may predict a pro-abortion position. Communitarian positions (low L, nearly equal O and E) are not so simple. We'll have to test it. Do you have any source material from Roberts? :)

[Mark Zuniga]

The first thing to look at is not Roe v. Wade, but Planned Parenthood of Southeastern Pa. v. Casey, 505 U.S. 833 (1992).
http://caselaw.lp.findlaw.com/scripts/getcase.pl?court=us&vol=505&invol=833

If you're not used to reading opinions, it can be a little confusing. The first thing it does is summarize the opinions. This language is not the judges', so I would not pay attention to it for your test. The analysis only matters once you get to section I of the joint opinion.

There are several opinions. There is the joint opinion of O'Connor, Kennedy and Souter (all Republican appointments [Kennedy is a Catholic]); the opinion of Stevens; the opinion of Blackmun; the Rehnquist dissent, with whom White, Scalia and Thomas join; and the Scalia dissent.

Essentially, you have 3 opinions with regard to the right to abortion: (1) uphold the right to abortion [at a minimum] because of the previous court rulings; (2) uphold the right to abortion on substantive issues; and, (3) rejection of a constitutionally protected right to privacy.

I feel that at least one of the three justices of the joint opinion, maybe even all of them, would have rejected Roe had it been presented to them.

The question is where Roberts will go. I'm going to look for opinions by Roberts.

[Mark Zuniga]

If you trust it, there is always Wikeapedia
http://en.wikipedia.org/wiki/John_G._Roberts_Jr.#On_Roe_v._Wade

[Mark Zuniga]

There is Roberts' testimony in front of the judiciary committee.
http://www.access.gpo.gov/congress/senate/pdf/108hrg/89324.pdf

[Mark Zuniga]

Of course, Roberts' testimony is compromised to some extent because there is a great difference between being an appellate judge and a SCOTUS justice.

[Mark Zuniga]

I have heard that there are documents Roberts wrote when he was young and clerked at the supreme court and worked at the state department. To some extent, those opinions would be less well formed. You would think that an appreciation for precedence could grow over time. Still, those opinions would have been written when he did not speak for a client, so he might have been less constrained by the needs of that client.

I haven't found those documents, but I will keep looking.

[Mark Zuniga]

Here is an opinion:
http://laws.lp.findlaw.com/dc/021283a.html
What I find interesting about this case is the concern that NTSB had not followed its own precedent.

[jonathon]

These all sound like good sources, and the roadmap for court opinions will doubtless be helpful when refining the sampling frame for judges. I'll have to gather samples into a text file to test. (Looks like I'm going to need a 'bot to strip webpages and build text files--there are so many of these, although it looks like judicial samples might be a little less tedious than Congressional sponsorship records)

On another note about the test's flexibility, Last night I applied four different test tables to some online pundit named Andrew Sullivan. This was after Kevin at Technogypsy expressed doubt over James Taranto's comment at OpinionJournal about Sullivan being 'conservative', so I put him to the test. For the sample of his work that I tested, not one of four separate test tables yielded a conservative position. Instead he tested as a moderate left libertarian. Could the LEO test be wrong? I thought. Or could Taranto? Earlier this year I had compared presidential candidate LEO scores with LEO-IP scores (an individual inventory based on the same model) and was able to predict voting behavior in virtually every case (I haven't published these results on the webpage yet) suggests that there is clearly something to the LEO model, so I decided to see how Taranto backed up his claim.

So I logged to Taranto's article on Opinion Journal and found that he was using the Political Spectrum model on Sullivan. All it allows for is 'left' and 'right', and since the Reagan Coalition, the source of the Republican majority, is made up of conservatives and moderate libertarians, of course Taranto would put Sullivan on the 'right'.

Hmm. maybe that should be another post. Argh. There is so much to do. I've barely scratched the Democrats, and here I'm already looking at pundits. I really need a 'bot to do this.

[jonathon]

Ugh. I just looked at the Seante Judiciary Committee report from the GPO. I'll have to extract Roberts' testimony. Looks like that's the project for tomorrow, since I'm certainly interested in Roberts. Unless of course somebody would like to build a text file with only Roberts' words?

...awsu, it was worth a shot, anyway...

[jonathon]

Mark, I just ran the Ramaprakash case through the test. Here's the results:
First table:
L=6
E=4
O=29

Second table:
L=11
E=5
O=44

Third table:
L=10
E=5
O=44

Fourth table:
L=4
E=2
O=38

Now, I don't know if these numbers are typical of Roberts' court opinions (the Casey decision was high-L with every measure) But if these numbers are correct, John Roberts makes Rick Santorum look like a bleeding-heart pothead. What's his record regarding precedent?

[jonathon]

Mark,
I've been sifting through the Judiciary Committee report you linked above, and found this from Roberts:

"There is no role for advocacy with respect to personal beliefs or views on the part of a judge. The judge is bound to follow the Supreme Court precedent, whether he agrees with it or disagrees with it, and bound to apply the rule of law in cases whether there's applicable Supreme Court precedent or not. Personal views, personal ideology, those have no role to play whatever."

And this:
"There's no doubt there are difficult cases. There are cases at the margin where text gets difficult to interpret. But, yes, I do think what lawyers do is at the end of the day what judges do, which is read Constitutions, read statutes to determine what the Framers or that legislative body meant. Those words have meaning. There are statutes--rules of construction that give guidance to the meaning of those words. And judges have an obligation to follow those rules and to follow the text of the statute or in some cases the text of the Constitution in cases before them."

Based on the above, Roberts sounds like a literalist who defers to his reading of legislative intent.

[Mark Zuniga]

It's my understanding that he wrote an amicus brief in favor of gay rights (or at least anti anti-gay rights) in the Roemer v. Evans case. I wonder if the legal nature of all of this is distorting the "o" readings.

[Jeremy]

I agree with Mark. Legal opinions are probably a bad thing to analyse because judges consciously (in rhetoric if not always in practice) try to make it look as if they're doing nothing but applying precedent. You'll only rarely find broad assertations of philosophical principles, most of the time judge's will be trying to obscure what they're doing by making it look like they're doing nothing.

[jonathon]

Mark,
It probably is. I've been thinking for a while, and conventional wisdom in PoliSci circles reinforces it, that a basic difficulty arises when trying to analyze the judiciary. The test results for Roberts' decision suggests we may have run into precisely that difficulty. Still, the Casey decision has a far different signature from the Ramaprakash case, and that difference will have to be explained. I am inclined to think that the best way we get a sense of where a judge stands is by looking at other public statements, or else, having them take the inventory.

Jeremy,
Just an idle thought, and brought about by your comment about precedent, but perhaps a variation of this analysis could be used to measure the strength of an "O" signature as an indicator of reliance on precedent. Maybe that's the captured difference between Casey and Ramaprakash.

[Mark Zuniga]

BTW, the quote that a judge is bound by Supreme Court precedent is uneventful for a hearing on a person nominated for the court of appeals. No one would disagree with that statement. However, no one would also say that Supreme Court precedents can't be overturned by the sitting supreme court. SCOTUS overrules itself with some frequency. We wouldn't want the Dredd Scott case (or its equivelant) to be the law of the land forever.

I'm going to do an analysis of Ramaprakash to see if I can explain the "o."

[jonathon]

Thanks. After all, that "O" score is _awfully_ high. It could be an application of existing law that requires significant articulation of the language of said law.

[Jeremy]

If you want to check, you could try running the test on some notoriously liberal opinions such as Brown v Board of Education, Miranda, Roe and see what you get. I suspect it'll still be pretty high on the O.

[jonathon]

Good idea Jeremy.

I followed your advice and tested Brown v Board just now. Here are the results (in l.e.o. order):
1st: 8.33.15
2nd: 17.33.14
3rd: 18.35.21
4th: 19.30.19
Comes up strong egalitarian every time.

[jonathon]

Miranda v AZ was a 5-4, with a split dissent: Clark on one dissent; Harlan, Stewart and White on the others.
The following scores are also in l.e.o. order.

First the Majority:
#1: 132.26.34
#2: 202.49.64
#3: 206.51.111
#4: 159.42.87
Comes up strong libertarian. I'm not surprised, given that this is a landmark civil liberties/restraint of government case.

Here's Clark's dissent:
#1: 3.2.2
#2: 5.4.5
#3: 6.4.10
#4: 3.3.9
A Libertarian-leaning conservative signature.

Here's Harlan, Stewart and White:
#1: 39.44.12
#2: 57.46.30
#3: 64.52.64
#4: 37.34.53
A much more moderate signature than either the majority or Clark. Moderate left libertarian in #1 and #2, Center Libertarian/Conservative in #3, Libertarian-leaning conservative in #4. Probably a very interesting dissent, this one. Likely arguing on the side of restraint.

[jonathon]

In the last dissent, I mean restraint of the judiciary, rather than government generally. Mark, got anything on the Ramaprakash case yet?

[jonathon]

Roe v. Wade gave me very strange numbers. Here they are:
Majority:
#1: 38.33.87 lbt estab
#2: 50.35.38 right lbt
#3: 55.37.78 lbt estab
#4: 37.18.63 lbt estab
Egalitarian scores are always lowest, but establishmentarian scores are generally high. This might be the legalist skew towards O that you were talking about, Mark. But it didn't show up in either Miranda or Brown. Why is it in Roe and Ramaprakash?

Is Roe v. Wade was not actually a liberal decision? Or does the LEO model measure something else in Judiciary decisions?

Here's Stewart's concurrence in Roe:
#1: 26.4.9 right lbt
#2: 24.17.9 left lbt
#3: 25.17.13 left lbt
#4: 23.17.9 left lbt
So Stewart's concurrence appears to be on libertarian grounds?

OK, here's the strange one:
Rehnquist's dissent:
#1: 4.1.2 right lbt
#2: 6.8.3 lbt egal
#3: 7.8.8 centrist communitarian
#4: 4.8.7 egal
Is Rehnquist is speaking in the negative in this dissent, or is he genuinely using a communitarian argument? I'll have to go look.

[jonathon]

Oh dear. This is going to have to be a post.