Thursday, October 30, 2008

Multiple Choice Testing Failure

(best read using IE, FireFox seems to be loosing the equations)

It is hard to define what an education actually means, and in the current atmosphere of liberal versus conservative with respect to teaching “values” in the academy, it is amusing that virtually no one disentangles scientific from non-scientific learning. Political Science must be a priori controversial since it attempts a dispassionate examination of a passionate subject. On the other hand, calculus is value free. No one, to my knowledge, thinks of it as controversial.




So the discussion of education should bifurcate into two discussions, one concerning politically correct (or otherwise) topics, and the other addressing the engineering, physics, biology, and, of course, chemistry education which the nation needs.



Of course, this last thought lies at the heart of the problem, since we constantly confuse training with education, just as we confuse an interest in how things are (and why) with a desire to change the world (for the better usually).




We desperately need large scale, country wide, testing in technical subjects.




I recognize how controversial this is, but ascertaining what our students are learning in algebra, calculus, physics and chemistry (my areas of quasi-expertise) is essential to sustaining the effort our predecessors made possible through their efforts on our behalf. We need to know that what we expect to be learned has, in fact, been learned (and retained).

That’s different than knowing what was taught. All of us will concede that what’s taught is rarely learned. What we need is a measure of what is learned. This instantaneous measure, coupled with a time shifted measure of what has been retained, is a true measurement of effective teaching/learning.




Since the material we deal with in technical work is hierarchical in nature, there is no way that students can continue to level “b” if their mastery of precursor material taught in level “a” is
insufficient to the new task at hand. Learned foundational material which has been forgotten is useless, and the “learning” was in vain, i.e., non-existent! Let me give you an example:



Consider the following:


Show that

This identity came up in a theoretical chemistry problem, and we expect any high school graduate to be able to show that the statement is true.

Now, before continuing, I want to make sure you understand what I mean by “be able to show that the statement is true”. I do not mean plucking something out of a multiple choice list. I mean actually writing lines of (what? ) algebra/mathematics/arithmetic/whatever and arriving at a proof that the statement is true. A multiple choice question of the type:











The following equations do not show up properly in FireFox for some unknown reason.
Please try another browser. Thanks.










1.



2.



3.



4.



5. etc.




tells us almost nothing when the student gets it wrong, especially if we’ve been extraordinarily devious in constructing “distractors”. Even getting the “right” answer assures us of nothing, since guesses are permissible in this environment! Worse yet, the calculator equipped student can evade the question completely, evaluating







(to have the value 1.618033989...) and then evaluating each of the choices until s/he finds the “correct” one 1 (choice 2 has the value 0.437016024.... while choice 4 is indeterminant, i.e. complex). Multiple choice examinations do not test what we think they are testing, and such test’s results are useless from the point of view of knowing what a student knows!



The entire motivation for Computer Assisted Testing was to make it possible to understand whether or not the examinee could actually do the work without the hints of a set of choices. The argument was made that the so-called real world does not give multiple choices to
scientific/mathematical queries. More important, even listing choices warps the intellectual environment of the measurement!



Finally, students need to have a “I don’t know” button or a “I can’t do this problem” button which forces them to come to grips with their lack of preparation, or lack of knowledge, or lack of , whatever.





It occurs to me that you may not know how to do this problem. Here's a small indication of how its done:


Start with

and expand the square under the radical. We get
which is almost what's requested. The point in the above discussion is that if one is interested in seeing whether or not an examinee can actually carry out the processes indicated above, then one needs an examination instrument which actually asks the right questions in the right way, not something which is "machine gradeable" and not something which is artificially impartial.

I can not leave this explanation without laughing about the first, outer, inner, last, a
mnemonic (FOIL) memorized by students who really are just passing through mathematics on the way to a life in words. Even recognizing the FOIL applies here indicates something about your earlier learning. Be that as it may, memorizing schemes for solving set-piece problems is not learning mathematics nor is it even educationally valid. Its just hoop jumping (through). Phooey.




1Testmanship in the extreme

No comments: