Multiple Choice Testing Failure

(best read using IE, FireFox seems to be loosing the equations)

It is hard to define what an education actually means, and in the current atmosphere of liberal versus conservative with respect to teaching “values” in the academy, it is amusing that virtually no one disentangles scientific from non-scientific learning. Political Science must be a priori controversial since it attempts a dispassionate examination of a passionate subject. On the other hand, calculus is value free. No one, to my knowledge, thinks of it as controversial.

So the discussion of education should bifurcate into two discussions, one concerning politically correct (or otherwise) topics, and the other addressing the engineering, physics, biology, and, of course, chemistry education which the nation needs.

Of course, this last thought lies at the heart of the problem, since we constantly confuse training with education, just as we confuse an interest in how things are (and why) with a desire to change the world (for the better usually).

We desperately need large scale, country wide, testing in technical subjects.

I recognize how controversial this is, but ascertaining what our students are learning in algebra, calculus, physics and chemistry (my areas of quasi-expertise) is essential to sustaining the effort our predecessors made possible through their efforts on our behalf. We need to know that what we expect to be learned has, in fact, been learned (and retained).

That’s different than knowing what was taught. All of us will concede that what’s taught is rarely learned. What we need is a measure of what is learned. This instantaneous measure, coupled with a time shifted measure of what has been retained, is a true measurement of effective teaching/learning.

Since the material we deal with in technical work is hierarchical in nature, there is no way that students can continue to level “b” if their mastery of precursor material taught in level “a” is
insufficient to the new task at hand. Learned foundational material which has been forgotten is useless, and the “learning” was in vain, i.e., non-existent! Let me give you an example:

Consider the following:

Show that

This identity came up in a theoretical chemistry problem, and we expect any high school graduate to be able to show that the statement is true.

Now, before continuing, I want to make sure you understand what I mean by “be able to show that the statement is true”. I do not mean plucking something out of a multiple choice list. I mean actually writing lines of (what? ) algebra/mathematics/arithmetic/whatever and arriving at a proof that the statement is true. A multiple choice question of the type:

The following equations do not show up properly in FireFox for some unknown reason.
Please try another browser. Thanks.

1.

2.

3.

4.

5. etc.

tells us almost nothing when the student gets it wrong, especially if we’ve been extraordinarily devious in constructing “distractors”. Even getting the “right” answer assures us of nothing, since guesses are permissible in this environment! Worse yet, the calculator equipped student can evade the question completely, evaluating

(to have the value 1.618033989...) and then evaluating each of the choices until s/he finds the “correct” one ¹ (choice 2 has the value 0.437016024.... while choice 4 is indeterminant, i.e. complex). Multiple choice examinations do not test what we think they are testing, and such test’s results are useless from the point of view of knowing what a student knows!

The entire motivation for Computer Assisted Testing was to make it possible to understand whether or not the examinee could actually do the work without the hints of a set of choices. The argument was made that the so-called real world does not give multiple choices to
scientific/mathematical queries. More important, even listing choices warps the intellectual environment of the measurement!

Finally, students need to have a “I don’t know” button or a “I can’t do this problem” button which forces them to come to grips with their lack of preparation, or lack of knowledge, or lack of , whatever.

It occurs to me that you may not know how to do this problem. Here's a small indication of how its done:

Start with

and expand the square under the radical. We get
which is almost what's requested. The point in the above discussion is that if one is interested in seeing whether or not an examinee can actually carry out the processes indicated above, then one needs an examination instrument which actually asks the right questions in the right way, not something which is "machine gradeable" and not something which is artificially impartial.

I can not leave this explanation without laughing about the first, outer, inner, last, a
mnemonic (FOIL) memorized by students who really are just passing through mathematics on the way to a life in words. Even recognizing the FOIL applies here indicates something about your earlier learning. Be that as it may, memorizing schemes for solving set-piece problems is not learning mathematics nor is it even educationally valid. Its just hoop jumping (through). Phooey.

1Testmanship in the extreme

Math Illiteracy Persists, Increases Unabated, and we Yawn

The Seattle Post-Intelligencer writes that Freshman at the University of Washington can't do middle school algebra. After 45 years of teaching, I can tell you that this situation, lamented by me in writing in various places during the years, persists. Worse, schools are catering to the decay by devaluing their degrees, lowering their requirements, and generally prostituting themselves. The civilization is in deep doo-doo.

Don't worry, be happy, the world will not end, just our children's (or children's children's, etc.) share will.

In the same article, the author notes the distinction between faculty of schools of ed on the one hand, and faculty of the technical community on the other (including as part, mathematicians). We know that the mathematicians have been given their chance at education reform, and that they bungled it. We also know that the other part of the technical community has no interest in proposing solutions to the problems, knowing their ineptitude or just plain lack of knowledge about teaching versus learning. But the chorus of disapproval of the status quo ante is deafening. There can be no doubt that the system is broken, and the people who are supposed to come up with solutions can't.

Here is a quote from a recent New York Times blog which epitomizes the situation:

Until you have taught in an inner city school you cannot understand how little the lack of achievement has to do with curriculum and teachers and how much it has to do with disruptions and fights, both physical and verbal. An average teacher with limited supplies could change the world if only their voice could be heard above the din of profanity and constant hum of apathy.

What this means is that closeted academics are not going to be the source of solutions.

Further, it is to be remembered that faculty in schools or education, as well as teachers in the field, do not actually use mathematics. They teach it, or teach how to teach it (?) but they are not actual practitioners. On the other hand, mathematicians are not equipped to understand difficulties in learning of materials which they think of as "mother's milk". So the situation is without a solution, something antithetical to all American beliefs.

Critical Thinking, an oxymoron

"Facts are quick to teach: The capital of New York state is Albany. Critical thinking is time-consuming to teach and involves provoking students to wonder why, for example, the capital of New York state is Albany" is quoted from a blog in the New York Times, after a "Dead Prez’s tirade" anonymously.

It is curious that asking why(?) Albany is New York's capital is regarded as critical thinking. What's so critical about it? Without knowing how to look up some history, without knowing how to spell, without knowing how to use the internet and library, why would someone think that the problem can be "solved" using thinking? Critical thinking in mathematics makes sense in the solitude of one's brain, but "critical thinking" about an historical oddity doesn't make sense.

Sometimes "critical thinking" in a social context is thought to be somehow thinking outside the box about social problems to suggest solutions which are "off the wall" different. In some sense or other, this is truly thinking, although falling in love with one's own suggestions implies that it is difficult to consider more than one option for a solution model.

But thinking remains the hardest thing we do. It takes focus to keep ourselves involved in thinking, critical or not. It takes discipline. It's wearying, and distractions abound to seduce one into day dreaming or some other mental activity other than real thinking.

We teachers are faulted for "teaching to the test", but, the tools of thinking are necessary but not sufficient for "critical thinking". Thus mathematics, physics, and chemistry thinking requires that one not think about arithmetic! Thus the tool of arithmetic is a necessary but not sufficient precursor of thinking about engineering or biology or any other quantitative subject.

Now when one thinks about truly elementary arithmetic, one easily concludes that either one learns it through practice, practice, practice, or, one never gets to Carnegie Hall (arithmetically speaking). Today, I had a student divide 10^{10} by 10^{16} and decide that the answer was negative. That's serious; this is a student who was maleducated in grade school and/or high school, who is now floundering in a course which assumes that this kind of arithmetic is beyond elementary. It's trivial and shouldn't cause a college student any trouble what-so-ever. Worse, the damage that's been done to her is irreversible. It can't be undone. She may be happy with herself, but I'm not happy with her. And I'm sure when she's a mother, her children will end up maleducated also.