5.02.2007

Let Them Frill Their Inner Robes



Now that I've dragged my fingers out of the air duct and gotten on with my day, it's time to build on events I've nearly missed. First thing first: Stephen Jackson's ejection. When Baron Davis got tossed, I saw nothing but "upbeat sarcasm." Though in that case, I know he was up in the ref's sockets, and had clearly made said official's actions the object of his disdain. Sarcasm might be less aggressive than spitting of cursing but if the standard is undermining authority, then I guess it's a line-crosser. I'm also told by people who regularly step on basketball courts that clapping in someone's face is a dick move, whether it's mocking or resolute. I am willing to accept that, on some level, the refs' emotions are the litmus test for outrage. That's why we have a right to expect for them to be level-headed, socialized people, and it's why we sometimes have to trust their contextual conclusions.

Turn with me now to last night's disposal of Stephen Jackson. This was what happens when a referee thinks in terms of precedents, instead of letting the moment wash over them in a productive manner. I firmly believe that Jackson was ended because he clapped, and clapping was something Davis had been t'ed up for in game two. Unfortunately, human expression is never so simplistic, so formulaic. To get what Jackson meant, whoever made the call should have paid attention to the game around him, rather than knee-jerked like he did. Here's Matt Barnes on the subject: "If you can't clap to get your team motivated and you get thrown out for that, it's kind of tough." Obviously Barnes is would defend Jackson after the fact, but couple his explanation with how it all went down. S-Jax wasn't looking at the ref, the Warriors were still in the thick of competition, and Jackson has been the team's emotional leader throughout this series.



But the ref saw another Warrior clapping, and assumed the worst. Or, more accurately, assumed that history was repeating itself, which is truly bizarre considering that the call on Davis was a purely subjective determination. A ref takes offense and pronounces the technical; the former yields the latter, making the moment into a technical. What makes it thus is the official's gut awareness, not a set of criteria. While Joey Crawford is bad for the profession, he's at least good refereeing gone bad, rather than a misguided attempt to turn the technical foul into something, well, technical. Actually, analytical works better. Save that shit for regular fouls, which can never be excused by appealing to the sixth sense of the zebra. "Felt like a hack to me" is nonsense unless you're in on the play.

As backward as this sounds, I feel this stems from a mutant form of political correctness. Not sure if you'd call it well-meaning, but I honestly think that refs would rather be given clear, eternally-applicable principles than delve into the intricacy of the way players express themselves. Some of this came up in reference to zero tolerance, but here I think it's even more pronounced. It's one thing to sanction away yelling and throwing; it's another altogether to make rule-of-thumb policy with regard to complex situations. Neither instance of clapping was just clapping, but to get into how they differed—or why they differed—one would really have to grasp a lot about player attitudes. I can only imagine that some refs, be it because of age, race, or class, don't feel comfortable wandering out that deep. And thusly, the terror of this series.



Let's end this on a positive note: being a referee is like being police, which is to say allegiance to an abstract principle that calls on subjective instincts. Cops toil for justice, refs for the order of the game. They are expected to put themselves into the equation, but remember to never take things to personally. Joey Crawford is like that old guy who remembers the days of the neighborhood police presence but has lot the ability to relate. That call on Jackson was the war on drugs.

32 Comments:

At 5/02/2007 7:58 PM, Anonymous Anonymous said...

Damn, a Wire tie in. I always feel that even with trying to size up a charge vs. break on a lightning fast break,the emotional part of being a ref is the hardest. The cop analogy fits. I wonder if a second after the call that ref was like "oh shit i fucked that one up" like Prez in the alley in season 3.

 
At 5/02/2007 8:10 PM, Anonymous Anonymous said...

Damn Bethlehem, sometimes I need a decoder ring to understand your posts. Is this a basketball blog or shall I wax poetic?

 
At 5/02/2007 8:16 PM, Blogger salt_bagel said...

When are we getting some teeth into the More Fouls Against Blacks study? It's popping up on every news and sports show; Barkley just called the authors jackasses (and demonstrated a complete misunderstanding of basic mathematics and the scientific method while doing so). Stern is having an absolute fit and is spouting claims that he can't back up. I haven't heard a single sound argument that refutes the study, but there's no shortage of utterly emotional arguments by people that obviously haven't paid attention to the statements the authors were actually making.

Maybe scientific review isn't this site's cup of tea, but I'm dying to see you weave it in.

 
At 5/02/2007 8:35 PM, Anonymous Anonymous said...

Salt--Here's the sound argument that refutes the study:

1) The authors "classified each N.B.A. player and referee as either black or not black by assessing photographs and speaking with an anonymous former referee." So--the basis for the study was utterly subjective? One anonymous former ref determines how everyone was categorized?

2) 100% of the Stephen Jacksons who got booted last night are black. Brother can't even get his applause on? Shee-it, String.

 
At 5/02/2007 8:49 PM, Blogger Shane Murphy said...

They wanted to look at ref's/peoples perceptions, so subjectivity was going to play a role any way.

Whats funny is that the Cornell guy, Joe Price, isn't much on sports. He's been trying to play things down I heard. Hopefully people on the radio take it easy on him, he had a question, he tried to answer it, and his answer is pretty provocative. Too bad the NBA's counter study is such bull shit and they won't release their data, so we can get another opinion.

 
At 5/02/2007 8:53 PM, Anonymous Anonymous said...

Good observation about Barkley's unintelligible rantings. I am yet to read the paper but I am sure that even the most elementary of regression analyses would control for biases in sample size. That said the econometrics knowledge I gleaned from my liberal arts undergrad is pretty paltry. I have hazy memories of majoring in Economics with a minor in pot smoking and teen-to-adult transition angst.

As for my feelings on the paper- the armchair economist in me says that assuming the analysis is robust, it probably reveals underlying differences in style that are positively correlated to race. Probably the influence of social perceptions that influence the development of white vs. black players. For example the only white guard I can think of that relentlessly attacks the rim is manu, likewise the only white guy that I would call a lock down defender is kirk hinrich. I think the differences in style are reflected foul disparity.

On another note, the Nuggets have the body language of a beaten team which does not bode well for their chances in the game.

 
At 5/02/2007 9:04 PM, Anonymous Anonymous said...

salt bagel: From what I understand the professors who did the study got their raw data solely from box scores without watching any game tape, which brings up a number of problems (in addition to the ones maxwelldemon already mentioned) with the validity of the study.

1) how can they determine which calls were fair and which were garbage? To be sure there's a varying level of subjectivity in every foul call but someone with a working knowledge of basketball should be able to generally judge good from bad calls but only if they watch the tape in slow-motion to do so.

2) how do they know which refs in interracial crews gave out which fouls? If a crew had two whites and a black ref the box score wouldn't clarify who each call was attributed to.

3) they don't know who each call helps. did a white player commit a foul on a black player? vice versa? the box score doesn't say.

4) maxwelldemon already mentioned this but how is anyone going to make arbitrary decisions about a player's race? What about biracial players like Boozer, Darrent Williams, european Tony Parker etc.? South Americans? How did they classify Eduardo Najera and Leandro Barbossa? What about Yao? This last point is in my eyes the most troubling b/c it makes a lot of assumptions about definitions of race that go beyond a simplistic black/white dichotomy.

I'm not trying to deny that racism is still an integral part of American society or that it may influence NBA refs but this study really doesn't seem to prove anything one way or the other.

The NBA decided not to punish Stephen Jackson which kind of validates the view that at least the latest if not both of his ejections were bullshit. Clearly his reputation preceded him which is no one's fault but his own. I do think Tim Duncan has been getting screwed by the officials as a clear Joey Crawford fallout and that it will continue for however long the Spurs remain in the playoffs.

 
At 5/02/2007 9:15 PM, Blogger Unknown said...

My prescription for the Nuggets is after Horry sinks the inevitable dagger and Denver is sent home. Get on the phones with the T-wolves and offer BOTH Camby and Nene for KG- a bit expensive but it is the only kind offer that can get McHale to free Garnett. Next move- offer KMart to the Knicks for any mix of 3-4 players that can make the 9-man rotation.

With KG, AI and Melo, 2007/08 Nuggets can finally embrace their destiny as the embodiment of over-animated competitiveness superstar futility. Who knows along the way they may just steal a redemptive championship.

 
At 5/02/2007 9:35 PM, Anonymous Anonymous said...

Excuse me while I get my math on, but I had a look at the actual text of the study (http://bpp.wharton.upenn.edu/jwolfers/Papers/NBARace.pdf) and I think I can answer some of these questions:

Anon853PM: In the study, they find that white players commit more fouls overall, regardless of who the ref is -- probably because whites are more likely to be centers and PFs, but that's my guess, not part of the original analysis. The key thing is, white players commit even MORE fouls when the crew is majority black (pg 36) then they do when it's majority white. The difference is small - 5.02 fouls per 48 minutes, versus 4.95. But it's "statistically significant," meaning that it's probably not caused by random chance (since the sample size is huge). Possible explanations: either white refs are giving white players a break, or black refs are giving white players a hard time. It's impossible to tell from the data which explanation is right, or if it's a little of both.

Padraig:

1) It doesn't matter which calls were "fair" and which were garbage. -- that evens out in the end. If you add up enough games, and you find that white refs are calling fewer fouls on white players than black refs are, that suggests bias (unless white players really do commit more fouls when there are white refs... intuitively unlikely, but impossible to rule out).

2) "How do they know which refs in interracial crews gave out which fouls?" Answer: They don't. They just compare the number of fouls given by the whole crew with the racial makeup of the crew. Again, majority-white crews gave fewer fouls to white players. They don't say who actually gave the foul.

3) Q: "They don't say who each call helps." A: True. While that's an interesting question to ask, I don't see how not answering it makes the study invalid. The basic fact that mostly white crews give fewer fouls to mostly white players is still interesting and surprising, even if we don't know who got fouled.

4) Q: "What about biracial / latino / international players" Yeah, that's a good question. Probably the right thing to do would just be to not count those guys one way or another. I didn't read it closely enough to see whether or not they did that. But society draws racial boundaries such that 90-95% of NBA players can be classified pretty much unambiguously, so I don't see this as a huge problem.

Overall it seemed like a pretty solid, reasonable study to me. The differences they're talking about are very, very small -- something you might not notice even if you watched 100 games. So we're not talking about Joey Crawford burning crosses on Tim Duncan's lawn or anything. It wouldn't surprise me at all if well-meaning refs still were a little biased in this way -- other psychology studies about race find that people have all kinds of unconscious biases.

 
At 5/02/2007 9:53 PM, Blogger salt_bagel said...

maxwell's point 1: They determined who was white and who was black by looking at them. Then they asked a ref whether he agreed. They also admitted that there were a small number of players that could go either way. They fit this into their stats and conclude that the answers would still hold true no matter how you classify those players.

padraig point 1: it doesn't matter which calls were fair or not. If you're saying that they get called at a higher rate for a certain subgroup, and you control for other factors as best you can, it means one of two things: that group actually commits more real fouls, or there's a bias. The study must operate on the assumption that black players commit fouls at an equal rate. This may not be true, but see what kind of controversy you get if you assert the other alternative.

point 2: They couldn't know this information. This is Stern's big beef. He says the league study takes this into account, but he won't release the data due to confidentiality issues. Either way, the Ivy study lumps the crews into 3 white, 3 black, and both combinations of 2 and 1. It sounded like they drew some of their statistical conclusions by comparing all-black versus all-white crews, but they tell you when they do this. It is a limitation, but there's no reason we should give the league any extra credit on this. They just make claims with nothing on paper to back it up, and I guarantee they will never produce any data for the public, whereas the Ivy guys will put their paper under peer review like everyone else.

Point 3: Again, this doesn't matter for the type of thing they claim to be measuring. Whether blacks are fouling blacks or whites, if you call more fouls on a certain color player, that indicates some sort of bias.

Point 4: See above. Everything I've seen about the study sounds rigorous, and sounds like they tried to control for as many grey areas as possible.

These guys were trying to do something serious to contribute to the ongoing study of prejudice and conscious/unconscious bias. They weren't trying to rile people up. They never said that the refs are consciously biased; in fact, they are mostly apologetic for the refs, and they say it's probably completely unconscious, and it's not nearly as bad as a lot of other real life instances of unconscious prejudice. The league immediately took it personal, and they don't realize that by doing so, they're hindering well meaning people who are actually on their side.

They did make a bad choice in publicizing the study in the middle of the playoffs, and they didn't realize it would land them in a shitstorm. But they don't deserve it. Everything they are doing is in the interest of understanding how people act and think. If people would step back and pay attention instead of reacting emotionally, we wouldn't have to do these kinds of studies.

 
At 5/02/2007 10:03 PM, Blogger Louie Bones said...

I really see Joey Crawford as the Bill Clinton of referees. That would make Dick Bavetta our Gerald Ford, and Bennet Slavatore our Richard Nixon.

 
At 5/02/2007 10:04 PM, Anonymous Anonymous said...

Salvatore.

 
At 5/02/2007 10:12 PM, Anonymous Anonymous said...

Finally AI has decided to snatch the Denver leadership mantle. He is channeling his die-hard Philly 1-on-5 persona. Lets see if he still has it in him to single-handedly pull out a victory.Now if only Denver could play defense!

 
At 5/02/2007 10:29 PM, Anonymous Anonymous said...

Anon810: "Or"? You're missing something elemental, Holmes.

The Suns ride like Avon tonight: "I'm just a gangsta, I suppose."

 
At 5/02/2007 10:39 PM, Anonymous Anonymous said...

Comparisons after tonight's game:

Spurs = cold, elemental fury
Horry = sublime, subtle grace
A.I. = one-eyed, cornered hellcat
Parker = quicksilver
Popovich = Palpatine

As a 76ers fan, I'm disappointed at my boy...

 
At 5/02/2007 11:12 PM, Blogger Colonel D. Williams (Ret.) said...

Robert Horry calmly ripping out hearts. The Tao of Big Shot Bob. I think it trumps PJ as the Zen Master. What has PJ won BSB left the team. I'm not saying, I'm just saying...

Re: Refs as pc Cops. What about a simple reading? They're just dicks.

 
At 5/02/2007 11:49 PM, Blogger Nathaniel Z said...

Stephen Jackson is easily the player I dislike most in the league, and that was a hack call, but Jax has to assume responsibility for his rep.

 
At 5/03/2007 12:20 AM, Anonymous Anonymous said...

Salt--"They weren't trying to rile people up"? Probably could have foreseen a little riling as a consequence, I think. And if the basis for this whole study is that they looked at pictures and decided how to reduce someone's racial identity into a single word, it would help to know who gets what label. Boozer, for example--does he get the white treatment from black refs and vice versa? Yao? Bibby? Matt Barnes?

Also--if racial distribution is fairly even among all teams, wouldn't such a bias cancel itself out? It would explain the championship in Hoosiers though.

 
At 5/03/2007 12:59 AM, Blogger Antid Oto said...

The differences they're talking about are very, very small -- something you might not notice even if you watched 100 games.

Not something you would notice, but the authors argue that the difference in foul calls also extends a small but measurable effect into almost all areas of player performance: points, rebounds, assists, and so on. Over the course of a season, they argue, these differences translate into something like one or two extra wins for a team with an extra white player, all other things being equal. It's a fascinating illustration of how very small biases that certainly don't rise to the level of what we'd call racism in ordinary life build up over not a huge amount of time to cause relatively big differences in outcomes.

 
At 5/03/2007 1:14 AM, Anonymous Anonymous said...

md -- i think you're really arguing in the margins... for 90-95% of players, it's an easy call. I don't see why quibbling about the remaining 5-10% would invalidate the study. Maybe in the future when half of the league is Chinese and we're all interracial anyway, they'll have to redo the study.

As for the second part, about bias cancelling itself out: on a team level, i guess it usually does in the NBA, although I'm gonna watch Stanford games a whole lot more closely.

p.s. 4th quarter, 2:05 left: 2004-5 Amare would have made that dunk.

 
At 5/03/2007 1:35 AM, Anonymous Anonymous said...

Antid Oto -- that's a really good point, although presumably that assumes you always got white refs, right?

Oh, and one more excellent point, from my second favorite blog, by Andrew Gelman:

"Which reminds me that the data show a familiar pattern also noted in historical baseball data by Bill James: the black players are, by most measures, better than the whites (more points scored, more points per minute, more minutes played, more likely to be starters), which is consistent with discrimination in hiring (picking good-but-not-great whites over good-but-not-great blacks)."

I had read something similar from Jason Whitlock, who had quoted some assistant GM about how NBA teams always take a white 12th man if they can, to be a fan favorite.

 
At 5/03/2007 1:36 AM, Anonymous Anonymous said...

Whenever you challenge the religion of colorblindness, especially in sports, you're going to encounter some hostility.

After Dan LeBatard suggested that a few MVP voters might have been swayed by Nash's whiteness, sportswriters didn't merely disagree with him; many called him a "racist idiot" for even suggesting the possibility.

Rex Chapman, a friend and supporter of Nash, said that LeBatard may have a point because during his own college career, he encountered racists and white-pride types who privileged him over his black teammates. The response of the LeBatard bashers?.....crickets.

 
At 5/03/2007 1:39 AM, Anonymous Anonymous said...

One point that's being missed is that the study is saying that players who are preceived as Black where not getting the calls. There are very few players in the NBA for whom this is truly an issue. I would say it's pretty fair to say officials perceive boozer and williams as black and the interview that the people who did the study conducted probably either confirmed or refuted this. Also the fact that there are one or two players in the league that could possibly skew the 13 year study is pretty unlikely unless Boozer is going to the line 10 million times per season.
The truth is that this study might not be accurate but you can't bring up little narrow subjectivities to refute it. In economics no meaningful study has ever been completely black and white. The best you can hope for is sound statistical analysis and enough of a corelation to take away the perception of randomness.
I'm pretty interested to see what happens when this study is given to the wolves aka academia to feast on. Sterns reponse wasn't really that surprising either.

 
At 5/03/2007 1:54 AM, Blogger Antid Oto said...

although presumably that assumes you always got white refs, right?

I don't think so. I think it applies to the situation as it currently is: mostly black players, mostly white refs.

To quote the study:

Throughout our sample, the refereeing crew was, on average, 68% white, while the teams were 83% black (weighting by playing time). A different thought experiment considers the consequences of race-norming the referee pool so that it matches the racial composition of the player pool. In our sample, the team with a greater share of playing time accounted for by black players won 48.6% of games, which is close to our regression-predicted value of 48.7%. Our estimates suggest that a race-normed refereeing panel would lead this number to rise by 1.8 percentage points, to 50.5%.

A 1.8 percentage-point difference over the course of 82 games is between one and two games.

The authors explain:

Equally, these apparently small impacts of own-race bias have big effects on game-level outcomes in a league in which around 3½ percent of games go to overtime, and around 4 percent of game outcomes are determined by only one point. Indeed, given that the winning margin has a standard deviation of about 12 points and is approximately normally distributed, it is not surprising that only a half-point shift in average winning margins would be sufficient to yield the substantial changes in the winning chances of one or other team winning.

 
At 5/03/2007 1:58 AM, Anonymous Anonymous said...

What about intentional fouls at the end of close games? Wouldn't those skew the results?

 
At 5/03/2007 2:11 AM, Anonymous Anonymous said...

salt bagel, fix_the_knicks: Once again I wasn't trying to assert that NBA refs are completely unbiased or that racial prejudice (or at least racial considerations) doesn't affect not only offciating but also marketing, teams' roster choices, etc. because it clearly does.

As to your refutations, well, they mostly make a lot of sense and I admittedly don't really know a hell of a lot about economics. It seems like the study's limitations have to do with its' goals and what information the scholars were able to access.

1) In this study the race of the referee giving out each individual call seems important. If Stern won't give out the info for 'confidentiality' reasons then I can't fault the guys who did the study. I have no doubt that the scholars are infinitely more open to review than the NBA. It's just an unfortunate limitation.

2) The complete lack of context really removes a lot of the study's teeth even if it doesn't disprove anything they're saying.

Really, it just seems to indicate that there is a small but noticeable and likely mostly unconscious racial bias at work among officials. If someone told me this without having done any research at all I'd still be inclined to belive them. Also if the researchers didn't realize that they would create a shitstorm by releasing their study in the middle of the playoffs then they're completely oblivious to the reality of the situation. It's unfortunate that the timing of their report might lead to unfair questions about its' validity but they have no one to blame for that but themselves. Of course the lig and its' diehard supportes were going to overreact. This is David Stern we're talking about after all.

I don't disagree with you guys. I just think the study isn't telling me anything I don't already know. Admittedly that's not a reason not to do it to confirm that. If someone could gain access to large amounts of game footage and do a more thorough study to take all that context stuff into account it would be great but somehow I can't see the NBA enthusiastically handing game tapes out.

 
At 5/03/2007 3:56 AM, Blogger Reno said...

"That call on Jackson was the war on drugs."

ETHER.

 
At 5/03/2007 9:52 AM, Blogger salt_bagel said...

Maxwell and padraig, you both say they should have foreseen the consequences. I'm sure they knew it was going to be controversial. However, the authors did everything they could to defuse it and try to explain to people what they were really up to. They told anyone who would listen that they weren't holding anything against the league or refs.

Now, in my opinion, any scientist putting a study out there in the media, especially before peer review, is asking for trouble. But preliminary reports of scientific studies are made available to the public all the time, and every once in a while one of them is deemed newsworthy and gets blown out of proportion. If you haven't heard any explanations from the authors, that's the media's fault. The Times article is the only place I've seen them get a shot at explaining things.

The best discussion the average sports fan will get is the inevitable Sports Reporters, or maybe half of Outside the Lines. If it gets on OTL, then they might have time to ask an independent researcher (i.e., someone who read past the first page of the math book) whether the stats make sense.

Regarding whether Boozer counts as black or white, I would assume black, but it doesn't matter for the purposes of the study. Since they took over 10 years of data, Boozer wasn't present for half of it anyway.

Padraig, you almost hit it on the head. There is a small but noticeable bias, probably unconscious. We knew this already. What the authors add is that it probably affects game outcomes in some situations. As far as the lack of context, if you want some, ask the league for those hidden stats.

One more thing: you said it was "an unfortunate limitation" that the league could not release those data. Actually, it's an intentional limitation.

I guess I'm just tired of people who don't know what they're talking about molding public opinion. That's my big gripe. If I sound pissed at individuals on this board, I'm actually just trying to explain the study, albeit through gritted teeth. But why would you have the real answers to those questions (the ones that the authors made a lot of effort to answer) when the media and the league are feeding us answers they just decided to make up?

wv: kilhaf: one solution to gas prices.

 
At 5/03/2007 2:02 PM, Anonymous Anonymous said...

Ultimately, I think the study is a waste of ink because it attempts to quantify a set of assumptions, and also because its focus is unrealistically narrow. Foul calls affect a team, not just a player. Even if you accept the assumptions that there are "black" and "white" players and refs, and that a "black" ref looks at all "black" players the same, the research ignores the race of the player who gets fouled, and that of the fouler's teammates.

 
At 5/03/2007 3:04 PM, Anonymous Anonymous said...

Most of the points that have been raised to argue the legitamacy of this study can be refuted statistically. Intentional fouls at the end of the game, the raise of the referee, the race of the teammates...these things are all statistically insignificant on a sample so large...pretty much the point of having a sample this huge is to bury the other variables...the majority of fould are in the flow of the game, the majority of the refs are white, and the majority of a player's teammates are black in the big picture.

I think the most interesting thing is how people are reactig to this study. Everybody seems to be trying to dismiss the results by saying the study isn't sound...

Isn't it possible that black players simply foul more often than white players?

Certainly there is no way to rule out an inherent underlying and subconscious racial bias. But to act as though white players and black players on the large scale play basketball the same way is naive. They are a differnet demographic within the scope of basketball, and that most certainly could account for part or all of that stastical significance.

If a study came out and said that point guards are chosen to shoot technical free throws more often than forwards, you would immediately conclude that point guards are just better free throw shooters than forwards. Obviously this is an extreme example of the point, but that makes it no less valid. To attribute the entire statistical significance to subconscious racis, THAT is where subjectivity really comes into play.

 
At 5/03/2007 3:28 PM, Blogger Antid Oto said...


Isn't it possible that black players simply foul more often than white players?

Certainly there is no way to rule out an inherent underlying and subconscious racial bias. But to act as though white players and black players on the large scale play basketball the same way is naive. They are a differnet demographic within the scope of basketball, and that most certainly could account for part or all of that stastical significance.


No.

The article is about differences in rates of foul calls by white and black refs within and across race. If white refs call fouls on white players less often than black refs do, the question of whether black and white players have similar or different playing styles is totally, utterly irrelevant.

 
At 5/03/2007 5:23 PM, Anonymous Anonymous said...

"Isn't it possible that black players simply foul more often than white players?"

No, in fact the study shows that exactly the opposite is true: white players foul more often than black players. Anyway, like Antid says, that's totally not the point of the study.

The MSM reporting on this study has been beyond horrible -- you should just look at the tables at the end, and ignore everything else you read about it.

 

Post a Comment

<< Home