One of the questions unique to the High-Adventure Science project is what we call the explanation-certainty item set. These item sets consist of four separate questions:

1. Claim
2. Explanation
3. Rating of certainty
4. Certainty rationale

In the first High-Adventure Science project, we developed these items as a reliable way to assess student argumentation and developed rubrics to score the items, which I’ll explain below. (You can also look at Exploring the Unknown, our first publication in The Science Teacher. Check out our Publications for links to all of the publications generated from High-Adventure Science.)

I’ll work through an example from the High-Adventure Science water module.

Scoring the Claim item:

The scoring for this portion of the explanation-certainty item set is fairly straightforward. Where there is a correct answer, the correct answer gets a point, and the incorrect answer gets zero points. Where there is no correct answer (because the problem is so nuanced and/or there is not enough information to make a definitively correct answer), we score into categories. For instance, in this question from the water module, there is no definitively correct answer:

No-one knows if the answer is yes or no until the wells run dry!

Scoring the Explanation item:

The scoring for this portion of the explanation-certainty item set follows the generic rubric seen below. Basically, we’re assessing whether (and to what extent) the student is able to make scientific claims.

What’s a scientific claim?

Scientific claims are backed by evidence. The more links a student is able to make between the evidence and the argument, the higher on the scale s/he scores.

It’s helpful to look at a couple of examples to really understand how this works.

Here are some “student” responses to the explanation portion following the claim question about irrigation of fields.  (Note that I made these up to be illustrative; they are not actually from students.)

• Student A: I don’t know.
• Student B: The well could supply irrigation easily for many years.
• Student C: The farmer might be drilling into a confined aquifer so the well wouldn’t last forever.
• Student D: After the water is used, it will sink back into the ground and be ready to pump up again.
• Student E: If the well is pumping from a confined aquifer, it won’t be recharged by precipitation. That means that the well won’t last forever.
• Student F: If the farmer had a limited amount of crops to irrigate, and the well was drilled into an unconfined aquifer so that it could be recharged by the rainfall, then the well might last forever. But if the well went into a confined aquifer, it would eventually run out.
• Student G: If the farmer drilled into an unconfined aquifer, the well might last forever. But that depends on how much water is being pumped out vs. how much can be recharged by precipitation. If the sediments above the aquifer are very permeable, then the aquifer will recharge quickly, but if they are not super-permeable, the aquifer will take some time to recharge, so it’s possible to pump the well dry, if only temporarily. If the farmer drills into a confined aquifer, the water might last a really long time (fi the aquifer is huge), but since it can’t be recharged because the sediments above it are impermeable, it would eventually run out of water.

How would you score these responses?

The first thing to think about is what the “best answer” looks like. Some of the sample answers are pretty good. But how do you distinguish between good, pretty good, and excellent?

The answer lies in the number of ideas in the answer and whether those ideas are linked. For instance, the main ideas to consider in the “best answer” are:

• wells pump from aquifers
• aquifers can be confined or unconfined
• unconfined aquifers can be recharged by precipitation
• confined aquifers are not recharged by precipitation
• recharge happens more quickly when the sediments overlying the aquifer are more permeable (and more slowly when sediments are less permeable)
• the amount of water in the aquifer is a limiting constraint (you can’t pump more than exists!)

Making links between these ideas is the key to a good scientific argument. The ideas for the “best answer” vary between explanation items, but the scoring idea is the same across all High-Adventure Science explanation items.

• Score 0: no links, no scientific claim present
• Score 1: no links (if a claim is present)
• Score 2: any one idea
• Score 3: one link between ideas
• Score 4: two or more links between ideas

So, scoring the student responses:

• Student A: This one is easy. The student did not make any claim or provide any evidence. This response scores a 0.
• Student B: There’s a claim, but does it contain any key ideas? No, so this scores a 1.
• Student C: This student brings up the idea of a confined aquifer. That’s one idea, so it’s a score of 2.
• Student D: This student recognizes that water flows in a cycle and that water sprayed on the crops will percolate down through the soil. That’s only one of the main ideas, so this response also gets a score of 2.
• Student E: This student makes the link between recharge and unconfined aquifers. This response scores a 3.
• Student F: This student brings in three links: unconfined aquifers are recharged; confined aquifers are not recharged; and rainfall provides recharge. This response scores a 4.
• Student G: This student brings in all of the ideas. There is a discussion of why unconfined aquifers are recharged by precipitation while confined aquifers are not. There is a discussion about how the permeability of sediments affects the rate of recharge. There is a discussion about the size of the aquifer. This response also scores a 4.

Scoring the Certainty Rating item:

We use this as an indication of how strongly a student is confident in his/her argument. There are no right or wrong answers here.

Scoring the Certainty Rationale item:

Like the Explanation item, the scoring for this item follows a rubric. Unlike the rubric for the Explanation item, however, this follows a rubric that’s very easy to generalize across all topic areas.

Students use many different reasons for their uncertainty, but they can be broadly categorized as personal, scientific within the investigation, and scientific beyond the investigation.

Personal reasons include:

• My teacher told me.
• I learned it from a science show.
• I read it in a magazine.
• I’m not really good at this topic. (Or conversely, I’m really good at this topic.)
• I haven’t learned this yet. (Or conversely, we learned this last year.)

Scientific within the investigation reasons include evidence from within the question or specific knowledge directly related to the question.
Scientific beyond the investigation reasons include:

• questioning the data or evidence presented in the question
• recognizing limitations in scientific knowledge about the topic
• recognizing the inherent uncertainty in the phenomenon (in this case, the uncertainty of knowing the type of aquifer from which the well is pulling its water)

So there you have it – a nutshell view of how we score the certainty-explanation items in High-Adventure Science.

NEW!! We’ve developed a user-friendly generic rubric structure for all High-Adventure Science argumentation items. Use it in conjunction with the exemplar answers and teacher tips in the Teacher Edition in the High-Adventure Science Curriculum Packages. You’ll need to have a free teacher account on the STEM Resource Finder to access the teacher materials.

NEW!! Two of the High-Adventure Science modules (climate and water) offer automated scoring for selected argumentation items. Learn more about how HASBot automatically scores students’ responses and get specific rubrics in the High-Adventure Science Curriculum Packages for the climate and water modules.

If you want to use parts of this rubric to score your students’ responses for your own grading, that’s great! Feel free to ask questions as they come up. The scoring is not always easy!  🙂