0

The Lawyers and the Statistician

By day three of the Michael Ellis termination hearing, the Ellis women had moved up to the first row of the hearing room. They wore apparel appropriate for Valentine's Day—a red hat, a red coat, a red sweater, a red dress.

The hearing resumed with Ellis attorney Lisa Ross recalling Charlie Bonds, Jackson Public Schools executive director of internal audits. After Bonds' testimony yesterday, Ross requested the originals of two weekly time sheets with changed entries initialed by Ellis. Ross questioned Bonds about apparent differences between the copies introduced as evidence by JPS and the original documents. JPS attorney Armin Moeller would not allow Bonds to speculate about the differences because Bonds said he had never seen the originals, only the copies.

After Ross concluded her redirect, Moeller called Dr. Willie Johnson, JPS executive director of accountability and research, as his next witness. Johnson's testimony centered around two primary-color bar charts: the first showing Chastain discipline offenses from the '02-'03 school year through the first half of the '06-'07 school years, and the second showing summaries of Chastain's Mississippi Curriculum Test scores for the same period. Consistent with JPS strategy thus far, Moeller concentrated only on data from Chastain, and only on the years of Ellis' incumbency as principal, and the school year immediately preceding his hiring. In such isolation, the data seemed to support the allegations that Ellis had "failed to achieve significant progress," that Chastain had "steadily declined" and was in a "state of chaos and disruption," and that Ellis failed to "maintain a safe environment."

But as Mark Twain once said, there are "lies, damn lies and statistics." Under Ross' cross-examination, Johnson put the Chastain statistics into larger context, which calls Moeller's cherry-picked conclusions into question. (Cherry picking is a term used by statisticians for data presented out of context.)

Under Moeller's questioning, Johnson testified that Chastain's number of student disciplinary offenses increased from a total of 21 in the '05-'06 school year to 52 for the first half of the current year, an increase of more than 250 percent. Without any other context, the numbers didn't look good, especially in the bright red, yellow and green bar chart.

Moeller seemed pleased with Johnson's testimony, and distributed copies of JPS evidence, topped off with the multicolored Chastain charts, to reporters.

Under cross-examination, Johnson testified that the data did not show the specific type of offense, which could range from relatively trivial offenses like creating disturbances in the classroom to rape and murder. Ross also had Johnson testify to the number of offenses at all other JPS middle schools for the '05-'06 year, and had him estimate the number of students in each. Ross also introduced the proportional size of each school as a factor. Rowan Middle School, for example, with a student population roughly one-third that of Chastain, had 18 offenses to Chastain's 21, showing that proportionally, Rowan actually had more offenses for the year than Chastain. Once placed in a wider context, the cherry-picked Chastain disciplinary data lost its glamour, moving Chastain from the bottom of the barrel to the third-best middle school in the district for that year.

"You want to paint a true picture of Chastain, don't you?" Ross asked Johnson at one point, to which Johnson replied that he did.

On the chart showing MCT scores at Chastain, Moeller concluded that scores had been on a downward trend during Ellis' tenure at the school. Ross' cross-examination of Johnson concentrated first on the important effect household income has on test scores. Johnson admitted that household income was a key factor not represented in the chart. He estimated that 70 percent of Chastain students received free or reduced lunches, indicating that the majority of Chastain students are from economically challenged or impoverished households. Johnson also testified that Chastain's test scores had actually improved for the same period in four of nine areas.

In further testimony, Johnson put Chastain's entry into the federal "No Child Left Behind" school improvement program into context with the ten other Jackson middle schools. Including Chastain, nine of the 11 JPS middle schools are in the program, with four—Hardy, Peeples, Siwell and Whitten—in corrective action, meaning that they have failed to meet certain standards for three years or more, although they have received additional federal funds specifically with the goal of raising those scores. None of the principals of the other eight middle schools have been fired, Johnson testified.

Ross also questioned the timing of Johnson's data compilations. Johnson testified that he had pulled the Chastain data on Friday, Feb. 9, 2007, at the request of JPS district counsel JoAnne Shepherd, and that no one had requested the data previously.

Due to scheduling conflicts, the Ellis hearing will reconvene on or around March 19, when Ross will conclude her cross-examination of Johnson.

Previous Comments

ID
91321
Comment

This is sad and I guess the long and short of this situation is: Was Ellis sexually harassed and was this the REAL reason he was terminated. One question: Why was Ellis offered another job and his wife offered a promotion if there is no truth to his story? The lawyer made an interesting point when she questioned Johnson's house-hold information and its effects on performance. WOW!!!!!!!!!1

Author
justjess
Date
2007-02-16T16:37:12-06:00
ID
91322
Comment

Justjess, to clarify, Ross didn't question Johnson's household income. She questioned whether Chastain students' household incomes--a key factor in standardized test scores--were considered when compiling the data presented. Johnson testified that it was not.

Author
Ronni_Mott
Date
2007-02-16T17:25:43-06:00
ID
91323
Comment

Ok, I’ll have to step in and point some things out that are glaringly left out. Yes, Chastain is a high poverty school with 73% of students eligible for the Free and Reduced Lunch program (a common definition of "low income"). But guess what? It's actually the lowest of all the JPS middle schools. The second lowest school (Siwell Middle School) has 79% eligible for the F/RL program. The average F/RL percentage for all JPS middle schools is 77.8% So how does that figure into the test scores issue? You also need to consider this, with the lowest number of low-income students, Chastain is the only middle school in the district to actually drop in ratings over a three-year period. All others have remained steady or rose slightly. So, least poor students with dropping test scores. Go figure.

Author
Rex
Date
2007-02-16T17:44:54-06:00
ID
91324
Comment

Rex, the point that Ross was making is that Moeller's conclusions were based on incomplete data. Your info may be correct, but it's still incomplete. Check with some statisticians: it's easy to draw conclusions when you don't look at all the facts, but only some of them.

Author
Ronni_Mott
Date
2007-02-16T18:04:36-06:00
ID
91325
Comment

Ronni M and Rex, thanks for the clairity and where I'm from, most of the kids were poor and there was absolutely no free lunch programs Principals and teachers too knew who needed to be fed and whose parents could not afford to pay for lunches. Too bad my later point was not considered in your responses. I'm not taking sides with the issue; however, I don't like to see good people hurt for the sake of just simply making points.

Author
justjess
Date
2007-02-16T18:12:35-06:00
ID
91326
Comment

Justjess, you asked some excellent questions, and I didn't mean to gloss over them. I agree completely that no one should get hurt just to make a point. It's frustrating and unfortunate, but this hearing isn't going to answer your questions, and I'm not in a position to speculate, which is why I didn't address them. This hearing is only about one question: was JPS justified in firing Ellis for the reasons they gave him. Secondarily, it may answer whether Ellis was singled out for scrutiny and firing, but that requires showing that JPS didn't use the same standards for Ellis as they did with the other school principals. Like you, I'm doing my best not to take sides. With a background in marketing, though, I have some knowledge about how statistics work. What I've seen so far in this hearing is that JPS isn't presenting the complete picture. To use a hypothetical analogy, it's like writing a headline that says "RAPES DOUBLE IN ONE YEAR" without disclosing that the numbers went from one to two, which probably makes the doubling statistically insignificant. From a research standpoint, if you only look for data that agree with your position, it's usually pretty easy to find. It doesn't, however, make it an accurate conclusion to begin with.

Author
Ronni_Mott
Date
2007-02-16T20:27:26-06:00
ID
91327
Comment

Ronni, my point was that Ellis, his lawyer, and your article are also not giving the full picture. The fact remains that of all 8 middle schools, Chastain is the only one to lose ground on achievement ratings. Further, trying to blame low scores on poverty is moot when you consider that Chastain is the least poverty stricken of the 8 middle schools. To accuse me of cherry picking in the context of your article is not only hypocritical, it is farcical. JustJess, Ellis was offered another position away from Chastain only for the remaining term of his contract, which would not be renewed, as opposed to immediate dismissal. This is not exactly what I would call offering him another job. His wife was not offered a promotion. She became interim principal at Wilkins Elementary due to a vacancy in the principal's position. To my knowledge she has not been offered the position of principal and will retain her assistant principal position when a new principal is hired.

Author
Rex
Date
2007-02-19T11:51:24-06:00
ID
91328
Comment

Rex, as a reporter, I did my best to present what happened in the hearing room. Whether either side presented a complete picture of the data, well, that's for the hearing officer to determine. Ellis' attorney's point in bringing up the whole poverty issue was, it looked to me, only to point out one of the many, many data variables that were not factored into the JPS evidence. I have made no claim that all of the relevant data was presented by the Ellis team, either, though they attempted to present a more complete picture. I also make no claim to being completely unbiased. That would just be delusional. What I said is that your conclusions aren't based on complete data, which they aren't. If you understood how statistics work, you'd know that yourself. I did not accuse you of anything, including cherry picking, but if you prefer to see it that way, you're entitled. Calling me a hypocrite doesn't change the fact that JPS cherry-picked the data. Oh, and by the way, there are 10 middle schools in the JPS system, not eight. We both had that number wrong... See how easy it is? Without looking at each individual school's scores, I can't agree or disagree with your statement that "Chastain is the only middle school in the district to actually drop in ratings over a three-year period. All others have remained steady or rose slightly," although I'd love to know your source. Acccording to the JPS Web site, in all of the JPS middle schools, only 38 percent of 8th graders were proficient in reading last year, 36 percent in language skills and 41 percent in math. I can see that compared with the averages of all other middle schools in the district, Chastain scored as well or better in most categories, and that they are a Level 3, or successful school, while three are Level 2, or under performing. I also know that seven of the other middle schools have been receiving federal funds to improve test scores for two to three years while Chastain has not, and that one of the other schools entered AYP school improvement at the same time as Chastain. The point is, in and of themselves, NONE of these facts are statistically complete, which brings me back to what I said originally: it's easy to draw conclusions when you don't look at all the facts, but only some of them.

Author
Ronni_Mott
Date
2007-02-19T18:04:08-06:00
ID
91329
Comment

Actually there are 11 if you count the special academic complex at Morrison. I mistakenly said 8, which is the number of high schools in the district. And as for my sources, it’s quite easy. Try the following publicly accessible (DUH!) website: http://www.jackson.k12.ms.us/about/2006_testscores/accreditation_AYP.htm As for looking at facts, silly me, I thought a reporter's job was to do just that, even do some research when the facts are not readily available from your sources. By saying you did your "best to present what happened in the hearing room" is a cop out for not digging when you knew neither side was presenting complete facts. Heck, I can read transcripts of the hearing all day long. An investigating journalist is supposed to go beyond the transcripts and press statements and dig for the more than observation and surface facts-- the truth. There is a WHOLE lot more going on with Ellis. This hearing is mere surface. Look behind the scenes if you want the real story. But perhaps I am wrong in assuming that you are an investigative reporter. You can understand my confusions since JFP is supposedly this bastion of “real” reporting, scooping the shoddy mainstream press. (That’s called “heavy sarcasm” in case it went over your reporter-head.)

Author
Rex
Date
2007-02-20T10:35:08-06:00
ID
91330
Comment

Rex, You better check that sarcasm or you're going to get your walking papers. Ronni is making a good faith effort to discuss the issue with you and you're just talking trash. You seem to have some pretty strong ideas about what reporters do, but you are clearly not a reporter yourself. A reporter can't write checks his butt can't cash. In other words, he can't just make sneering comments about "looking behind the real story." He can't just say "there is a WHOLE lot more going on with Ellis." That's great, Rex, but do you have something more than innuendo? Put up or shut up. Your clarification of the poverty numbers and the decline in scores is useful, but you don't make any comment on Chastain's overall place within the middle schools. You don't address the disciplinary numbers. You don't address the fact that all of these numbers supposedly had something to do with firing Ellis but were only compiled by Johnson in February. Ronni did not accuse you of cherry picking, and she has made no comment on whether Ellis was a good principal or a bad one. She has said that JPS did all it could to make rather uncompelling numbers look like compelling grounds for Ellis' termination. This effort was not particularly convincing, and nothing you've written changes that. The notion that all she did was look at a transcript is a ridiculous insult. Your comments on poverty and test scores are useful. Your tone is useless. Continue this discussion in good faith or get lost.

Author
Brian C Johnson
Date
2007-02-20T11:00:47-06:00
ID
91331
Comment

Trying to figure this one out is akin to a "chicken - egg" story. Why would you try to figure out the statistical evidence relative to Ellis' firing first. Why not examine the information presented by him relative to the reason he was fired. My sincere hope is that the case will not fall into a political war and that the rights of all involved be protected. I don't know why I'm such a stickler for justice: perhaps it's simply because I'm JUSTJESS.

Author
justjess
Date
2007-02-20T11:26:16-06:00
ID
91332
Comment

Justjess, I'm with you 100% regarding examining the information relative to the reason he was fired. That's exactly what the hearing is supposed to determine. As to whether it will become a political war, my fear is that it's already there. As for you, Rex, the link you gave provides absolutely and unequivacably no support, none whatsoever, for your assertions. As a matter of fact, the pages from that link have been in my research binder dedicated to this case for weeks, and I've quoted the information from those pages in this thread. Nowhere does it say, for example, that "Chastain is the only one to lose ground on achievement ratings." Not even close, dude. It also is completely silent regarding your assertion that "Chastain is the least poverty stricken of the 8 middle schools." So, once again, what are your sources?!? I'm still willing to give you the benefit of the doubt, so if you're as knowlegable about the case as you claim, educate the rest of us. But quit with the sarcasm, innuendo and insults. If the only thing way you can address this issue is by attacking me personally, give it up. I don't care if you don't like me or my reporting style, and frankly, neither does anyone else. Put up or shut up.

Author
Ronni_Mott
Date
2007-02-20T13:32:44-06:00
ID
91333
Comment

Ronni, you must not be reading the same stats at the link I posted. Take a look again. Chastain is the ONLY middle school of the ten (Morrison is not rated due to its special status) that met AYP in all areas in 2003-04 but went to “school improvement” status in 2005-06. Others were already in school improvement or maintained their AYP compliance. As for the poverty data, it’s not published on the website but it is available from the Office of State and Federal Programs at JPS or if that’s not good enough the Mississippi Department of Education can give you the same information with a simple phone call. But for you, Ronni, here’s the data on JPS middle school poverty for 2006-07 as reported by the Office of State and Federal Programs: Blackburn MS 92% Low-Income Brinkley MS 95% Low-Income Chastain MS 73% Low-Income Hardy MS 91% Low-Income Northwest MS 84% Low-Income Peeples MS 84% Low-Income Powell MS 84% Low-Income Rowan MS 92% Low-Income Siwell MS 79% Low-Income Whitten MS 88% Low-Income I repeat, Chastain is the least poverty stricken of the JPS middle schools.

Author
Rex
Date
2007-02-20T14:22:17-06:00
ID
91334
Comment

Mr. Johnson, yes Ronni DID accuse me of cherry picking with the statement, “Your info may be correct, but it's still incomplete. Check with some statisticians: it's easy to draw conclusions when you don't look at all the facts, but only some of them.” That is virtually the definition of “cherry picking.” And just to let your “marketing major” know, I have long made my living in statistics and research. I self-banned months ago so your threat really means diddly. It’s been months since my last post (also about incorrect statements by your “staff”) and it will probably be months before I get so nauseated as to feel compelled to post again. As for my tone, I’ve read far worse from your regulars on this site (“put up or shut up”? good tone) and never see them threatened with walking papers. But let a non-clique member question a reporter’s tactics or an opinion from the cabal and suddenly he’s a troll headed for the infernal regions. If you want me to pull my punches, I’ll be glad to issue an apology: I did not say or mean to be interpreted as saying that all your reporter did was look at a transcript and repeat it. I have the utmost regret if that is the impression I gave. I sincerely apologize to your paper, your reporter, and to your readers for any misunderstanding or unwarranted denigration my comments might have caused. What I meant to say is that the article had all the information of a transcript and nothing more.

Author
Rex
Date
2007-02-20T14:24:05-06:00
ID
91335
Comment

OK, this has devolved into silliness. I will trust that the na-na-na-boo-boo crap ends here. Agree or disagree, discuss issues or move on, Rex. If you continue the berating stuff in one more post, you will be moved to trollblog. And for the record, this isn't an "investigative" piece; it is a report of what happened during a hearing proceeding. Learn the difference before you start trying to tell people how to do their jobs.

Author
DonnaLadd
Date
2007-02-20T14:49:48-06:00

Comments

Use the comment form below to begin a discussion about this content.

Sign in to comment