Sunday, January 6, 2013

Writing Part 3: What Reading Comprehension Questions Tell Us About Writing

There are three fundamental types of reading comprehension questions currently utilized in testing students:
  1. Literal Comprehension Questions
  2. Inference Comprehension Questions
  3. Evaluative Comprehension Questions
(http://www.project-read.com/types-of-comprehension-questions/, http://www.litart.com/weblog/three-types-of-questions-to-build-comprehension, http://academic.cuesta.edu/acasupp/as/303.htm, I have not found the seminal work introducing these questions, but the types are commonly listed in educational books and websites, and they essentially follow Bloom's Taxonomy).

Of note, these questions are commonly applied to all sorts of different texts--from fiction to nonfiction--across all grade levels--from kindergarten to college.  Therefore, these types of questions ought to reveal to us writers what expert readers (i.e., test-makers) determine that the average person ought to be able to discern from any given text.

Literal comprehension questions are generally, "Can you read?" type questions, literally.  The Iowa Test of Basic Skills reading comprehension for grade 1 literally asks students to match the word to the picture.  A literal SAT question, taken from the College Board's own example, would look like:

In line 5, "surveying" most nearly means

What do literal questions teach us about writing?

Literal questions teach us the importance of facts and evidence.  In most academic writing, this evidence is going to take the form of quotations, data, results, statistics, dates, acts/laws, and other similar facts.  In news writing, you'll often be looking at quotations, numbers, pictures, etc.  In business writing, this evidence will usually take the form of procedure, analysis, and statistics.  In technical writing, you're looking at repeatability, procedure, specificity, and clarity.  In creative writing, evidence most often takes the form of scenes.  It will matter very much whether you conform to your genre's idea regarding evidence when you write and also whether you conform to your audience's idea of evidence.  Note, for example, how pop-psychology and motivational books often incorporate novel-like scenes.  That inclusion stems from publishing companies' understanding that the people who buy these books most often read novels before they turned to this genre and think of evidence in terms of scenes.  Therefore, they have asked authors to include scenes in their work.  Ta-DAH!  The work is suddenly accessible.

Literal questions also show us the importance of directing the reader implicitly through straightforward words to the relationship between what came before and what will happen next.  As the Project Read website link clearly explains, recalling a series of events is a literal question, but it's one that leads to an inference question.  In other words, the ability of a reader to infer is directly related to how well an author literally lexically directly portrays the logic and sequence of what is happening now.


Inference questions ask the reader, in the context of what is known (i.e., what has been literal), what might also be known.  Inference questions include questions like, "Given that we know Mom hates dirt in the house, how do we think she will feel when Ruby and Sue, who we know have been playing in mud puddles, walk through the front door?" and "Since you know that it's dark outside, during what time of day do you think that the action is taking place?" 

Again from the College Board's own example of an inference question
The narrator would most likely describe Mr. Pontellier's conduct during the evening as
 We know this is an inference question not only from the phrase "The narrator would most likely describe...", but we can also be assured of the College Board's feeling on the matter when, in the explanation, they tell us literally, "This question asks you to consider a large portion of the passage and to make an inference about the narrator's view of 'Mr. Pontellier's conduct during the evening.'"

Inference question are not complete guesses, but neither are their conclusions without doubt.  We tend to forget that doubt because we score these questions regularly.  Inference questions often follow syllogistic logic.  For a thorough but not exhaustive discussion of syllogistic logic, see "Syllogistic Reasoning" at http://www.logicinaction.org/docs/ch3.pdf.  The problem with syllogistic reasoning is that it assumes that no other conditions exist.  As we see in life, this situation is hardly ever the case.  Some circumstance or condition is always happening which you don't know about or can't qualify.  The acknowledgment of the existence of these unknown conditions are the basis behind controls in experimental design and the idea of extenuating circumstances in law.

Lest we think that the problems of fallibility in inference fall solely outside the field of words and literature, let's consider mysteries.  Tales of mystery are a staple genre that's been active for centuries and that operates almost entirely on the fallibility of inference.  We draw erroneous conclusions because not all of the facts are in.

Similarly, on the non-literary side of linguistics, the fallibility of inferences provides much of the background to the field of sociolinguistics which studies why difference groups of people use language differently in different situations--and do so with a fair (but not total) degree of miscommunication.

What do inference questions teach us as writers?

Inference questions, in and of themselves, teach us two things.  First, they teach us that inference/implication is expected, and thus a general disregard for extenuating circumstances (or a suspension of disbelief) is necessary for comprehending (but not thinking critically about!  Don't forget that, and don't think for a minute that our general questions on critical thinking teaching inference are critical thinking at all!--But more on that later).  From a writer's perspective, you need to take away two lessons from this point.  First, you need to recognize your audiences beliefs and doubts and work with them to make sure your points are not too far off the beaten path that the audience will no longer suspend disbelief and then to realize that you may need to simplify the presentation first and then address the arguments later (i.e., give them the main thrust of your arguments while belief is suspended and then work in the intricacies of  your points further on in the piece).

Secondly, returning back to syllogistic logic, inference questions teach writers that readers need a certain amount of literal information (i.e., the premises of the syllogism) to be clearly presented in order to expect the reader to draw an inference (i.e., syllogistic conclusion).  The writer will need to figure out what those premises are before she makes her argument in earnest.

Furthermore, and this I haven't proven here but you will see it as you look for examples in inference questions, the existence and types of these questions on tests teach us that the level of implication, from which we are able to make our inferences, varies greatly depending on the type of writing you are doing.  News stories without spin generally do not imply much; they state it directly.  The spin is the implication and the tone.  Likewise, most technical writing pieces do not leave implications open to the imagination.  They are straightforward in their assertions.  Academic writing does imply some concepts, but to a greater degree than a technical paper and to a lesser degree than a story with spin, and generally with much evidence presentation.  Creative writing, however, relies heavily, almost entirely, on implication.  Writers would do well to remember these differences.

Also, the art of writing a piece that implies something well requires a certain lexicon and grammar which differs between genres.  I will get to this grammar in a later post.  But, suffice it to say, writers need to be aware of and comfortable with using the appropriate level of implication and the diction and syntax necessary to convey it in the genre within which they are writing.


Evaluative questions ask readers to speculate on what the author might think.  These questions often refer to topics like tone, underlying belief, and possible group membership.  In other words, evaluative questions often ask us to draw conclusions based on facts not in evidence.   For example:

The College Board's example:
In the final sentence of Passage 2 ("I thought... in me"), the author expresses

 The College Board further explains this question, by stating:
Explanation

Difficulty: Hard

Even though this question focuses on a single sentence, you must understand the context in which the statement occurs in order to determine the feeling expressed by the author.
In this case, the phrase "in context" is extremely vague.  A student may stumble upon the answer without knowing cultural values around the time of the piece's writing or without understanding how statements of regret are commonly written, but it is unlikely that all such students would (or could) do so.  Therefore, this question asks not merely comprehension of a line in the passage or comprehension within the context of the passage, but comprehension of a line in the passage taken in both the context of the passage and in the context of the passage's culture.  Such questions require greater knowledge of the literature than the passage itself offers to the reader.  It requires speculation on the part of the reader, the consideration of other connections that the reader is expected to know.

In and of itself, speculation isn't necessarily bad.  In real life, speculation is often required.  Speculation provides the catalyst for experimentation and inquiry.  Speculations form hypotheses.  They are the beginning not the end.

That last differentiation, the understanding that the speculation is not the same as a definitive conclusion, is where we deceive ourselves as teachers of reading and writing.  Only in the classroom is an evaluation so easily codified.

Outside of teaching, the circumstances in which we actually see evaluative questions in action, these questions lead to experimental design, data, and results.  When we reach the professional level, we have many ways of judging the validity of the evidence presented:  levels of confidence, standard deviations, p-values, rules of evidence/admissibility, market research, etc.  In creative writing, we see the same types of validity presented other ways:  alternate viewpoints, foils to characters, "historic" facts woven throughout.  In other words, when we get to the higher levels of thought, we recognize that the correctness of any given point of view is relative.  Doubt is both acknowledged and quantified.

But when we begin teaching speculation and evidence gathering, we don't qualify our evidence much beyond the labels "fact" and "opinion."  And this practice is very important to recognize.  Yes, one needs to learn to do something before one can quantify and qualify it.  Basics do come first.  But something very important is missing from this equation:  the fact that these speculations are not without doubt.  They are based on facts not in evidence, on assumptions that might be made but don't necessarily have to be made and may not be correct--assumptions which are, in fact, assumptions.

In other words, when we begin teaching speculation (which continues into the upper levels of writing because the quantification of doubt generally happens within disciplines), we forget that we are no longer teaching facts and truths.  We treat speculation as truth even though it isn't so.  We are asking students to recognize their audience (us), do the best they can to read our minds, and then present our speculations to us as though they are facts.

What do evaluative questions teach us as writers and as teachers?

Evaluative questions teach us three essential concepts:

First, writers will be asked to recognize the logic of the culture at large and to be able to follow it to its conclusion and to present it to another body who is evaluating their work.  Furthermore, writers need to recognize the lexicon (choice of nouns--this concept vs. this misconception--adjectives, and adverbs) and syntax (succinct writing is generally used to convey the point you believe while convoluted sentences are often given to present the points of your opponents) used to convey the writer's true feelings and beliefs while not stating them outright.

Secondly, teachers need to recognize that students may come to class with other forms of logic or culture that will lead them to different conclusions that are as valid as the ones that are culturally accepted.

Third, in order to learn to succeed at writing and be true to themselves, writers need to recognize when they must parrot back the party line and when and how they can challenge it.  Challenging the accepted interpretation will require more various and adept rhetoric, more numerous and more specifically on-topic examples/evidence than generally accepted, and also an acknowledgment of audience.  You don't take your genre-smashing film to MGM immediately.  You start at an independent film company.  The same holds true of most writing.  While you might get an exceptional SAT essay-reader, chances are that the essay is not the time to protest the method.  BUT if you wanted to write a well-crafted letter or essay to the makers of the test, you might actually get a response, and a thoughtful one at that.  The people who design these tests are thoughtful and well-versed.  They do consider other points of view, especially when they are thoughtfully and thoroughly presented.  And because of the relation between the feeling of powerlessness and depression and violence, it is in the best interest of every teacher to make certain that every student who writes learns to express his voice in ways that matter both to him and to society at large and could actually evoke change.

No comments:

Post a Comment