One of my favorite things about Twitter is the copious among of sharing that occurs among education users. In addition to the great conversation, teachers are often sharing resources they've developed or come across. Today, John Calvert (@jcalvert4) shared a rubric he'd developed based on another educator's rubric. I thought it would be interesting to explore how a rubric can evolve in our efforts to make our expectations more explicit and to help students to understand what degrees of quality look like.
First, many thanks to John for letting me wander through his rubric and share my thinking. Second, it's my philosophy that while there's no such thing as a "perfect" rubric, there's also no such thing as a "bad" rubric. (What makes a rubric a rubric?). Moving towards quality rubrics means clarifying our language and better aligning the assessment tool with our intended outcomes. Finally, my standard disclaimer. A quality rubric is a tool that can assist students with self-assessing their own work during the design or development process. In other words, grading and rubrics do not go hand in hand. If a teacher is looking for specific things (they're mentally counting as they go through student's work), they probably have a scoring chart in mind. Even though some may call these "scoring rubrics", I maintain that a rubric is more about self-assessment and quality than it is about scoring and grading.
The original rubric appears to be designed to help professors communicate with their students about what they're looking for on a discussion board. Again, it's not to say that rubrics and grading should never be married, but rather it's helpful to a learner to be able to separate out the quantitative from the qualitative; the tangible from the intangible. Below, I've walked through the evolution of a rubric, using attributes from a Checklist for Quality Rubrics. The statements in bold are taken from that checklist.
Evolution of an Online Discussion Rubric
|
Category
|
1
|
2
|
3
|
4
|
|
Promptness and Initiative
|
Does not respond to most postings; rarely participates freely
|
Responds to most postings several days after initial discussion; limited initiative
|
Responds to most postings within a 24 hour period; requires occasional prompting to post
|
Consistently responds to postings in less than 24 hours; demonstrates good self-initiative
|
|
Delivery of Post
|
Unitizes poor spelling and grammar in most posts; posts appear “hasty”
|
Errors in spelling and grammar evidenced in several posts
|
Few grammatical or spelling errors are noted in posts
|
Consistently uses grammatically correct posts with rare misspellings
|
|
Relevance of Post
|
Posts topics which do not relate to the discussion content; makes short or irrelevant remarks
|
Occasionally posts off topic; most posts are short in length and offer no further insight into the topic
|
Frequently posts topics that are related to discussion content; prompts further discussion of topic
|
Consistently posts topics related to discussion topic; cites additional references related to topic.
|
|
Expression within the post
|
Does not express opinions or ideas clearly, no connection to topic
|
Unclear connection to topic evidenced in minimal express of opinions or ideas
|
Opinions and ideas are stated clearly with occasional lack of connection to topic.
|
Expresses opinions and ideas in a clear and concise manner with obvious connection to topic
|
|
Contribution to the Learning Community
|
Des not make effort to participate in learning community as it develops; seems indifferent
|
Occasionally makes meaningful reflection on group’s efforts; marginal effort to become involved with group
|
Frequently attempts to direct the discussion and to present relevant viewpoints for consideration by group; interacts freely
|
Aware of needs of community; frequently attempts to motivate the group discussion; presents creative approaches to topic.
|
|
Original from Middle Tennessee State University: http://frank.mtsu.edu/~webctsup/faculty/discussionboards/fac_db_samplerubrics.htm
|
Challenges with version #1:
The low levels of the rubric focus on what is not present, rather than what is present: If a rubric is a tool for a student to self-assess, a tool with lower levels that stress a student's failings might backfire. Few adults, much less children, have reached a point in our self-esteem when we can say, "yeah, I really sucked at that. I didn't even try." Instead, we rationalize to protect ourselves (I highly recommend Mistakes were Made for more about self-protection). We may not even read that low level as it's difficult to conceive of ourselves as "being hasty" or not "making an effort".
The dimensions of the rubric do not appear to be prioritized or placed in a purposeful order. Generally speaking, when we look at a series, we know that the most important thing is at the top. If someone were to infer priorities according the rubric above, compliance and mechanics appear to be more important than the contribution to the community. My hunch, however, is if we talked to the professors, they'd say the opposite. They care more about the postings content and the impact on the community than on if all the words are spelled correctly.
Revisions that John made:
John's rubric is below. Right off the bat, you can see he's revised it to include language that students will understand. By changing the language to "I", John has returned the rubric to its core purpose: a self-assessment tool for learners. This is not to say that quality rubrics need always be written from the students' perspective. Using "I" can be a powerful way to ensure the rubric is accessible to students (providing the students review the language and agree it makes sense to them).
|
Category
|
1
|
2
|
3
|
4
|
|
Promptness and Initiative
|
I did not respond to the question until my teacher reminded me.
|
I answered the question, but did not reply to anyone else.
|
I responded to the discussion board question in a timely manner. I also answered anyone who commented on my post or I replied on another person’s post.
|
I responded to the discussion board question in a timely manner. I also answered anyone who commented on my post and freely replied on other posts.
|
|
Mechanics
|
I did not proofread. I had more than 7 spelling and grammar mistakes.
|
I had between 4 and 6 spelling and grammar mistakes.
|
I had up to 3 spelling and grammar mistakes.
|
I did not have spelling and grammar mistakes.
|
|
Relevance of Post
|
I did not post topics that are related to the discussion content.
|
I posts topics that are related to discussion, but my posts are short and lack detail.
|
I posts topics that are related to discussion content and include supporting details.
|
I always make my posts related to discussion topic. I include supporting details and tell people where I got my information by citing my sources.
|
|
Contribution to the Learning Community
|
I do not make an effort to participate in the learning community. My tone was disrespectful.
|
I tried to read other posts with an open mind and think about their perspective before I replied. My responses occasionally contributed to a positive discussion.
|
I read other posts with an open mind and think about their perspective before I reply. I either make personal connections to the posts of other people or I defend my position with respect.
|
I am supportive of the discussion board community. I make personal connections to the posts of other people. I defend my position with respect.
|
|
Modified version by John Calvert: https://docs.google.com/document/d/1oY8krtLMRmVB39MEwpcdOv5CrcCN7kYo42gUpkdIzFQ/edit?hl=en&pli=1#
|
Challenges with Rubric #2:
There does not appear to be consistency in the language used. The written word can fail us as human beings. We know what we want to convey, but sometimes end up with the reader getting a totally a different message. The challenge in rubrics is to avoid verbosity but at the same time communicate what's important. In these examples, I think I know what John means. If we wrote it together, I'd probably know what he means but if I'm a student trying to self-assess at home while sitting in front of my computer, I may forget what I heard during the discussion and struggle to find myself if I have only the written word to rely on. If we look at the top dimension, Promptness and Initiative, the concept of timeliness doesn't enter until the third level. While Level 1 reflects an absence of timeliness, what about the student who replied to others' postings but waited until right before the posting window closed? The goal isn't to address every "what if" but rather answer the question "what does a student do when they don't have a handle on promptness?"
When quantitative terms are used, they aren't supported with quality attributes. I've made the general statement on Twitter and in this wiki that if you see numbers, you're probably looking at a checklist. That's not always true as John's rubric shows. However, looking at the mechanics dimension, the rubric appears to be focusing on quantity of errors, rather than the quality of their writing. Having the number in the rubric means the the student - and the assessor - has to physically count how many spelling errors there are. This poses some challenges, especially for struggling writers who may be experimenting with new words. If I misspell "ambivalent" six times in my posting, does that count as one misspelling or six (or just highlight my need for a thesaurus?). What is it that makes a posting with 7 errors worse than one with 5? If you want to be explicit with your students, if you want them to attend to the quantitative features of their work, then give them a checklist. Rubrics are tools for focusing on quality - spelling errors don't make or break quality. They may interfere but that's part of articulating quality.
Revisions that I made:
My revisions are captured in the rubric below. I made a few changes to focus the rubric on the quality of postings. My changes include:
- moving mechanics to the bottom (The dimensions of the rubric are prioritized or placed in a purposeful order.)
- renamed the levels to assist the learner with understanding how the rubric is to be used (The levels are appropriately named, given the audience and nature of the rubric’s use.)
- fleshed out the top level so that it shows the students what is beyond our expectations (The top level of the rubric is above the expected standard – it would be a stretch for the highest achieving students.)
- added a level on Webiquette to help the reader better understand what it means to be "disrespectful" in a posting
|
|
Novice Poster
|
Amateur Poster
|
Proficient Poster
|
Mentor Poster
|
|
Promptness and Initiative: How often do I post? How often do I think about the discussion board?
|
I have to be reminded to respond to the question by my teacher or a friend. The discussion board just isn’t something I think about regularly and visit it to do only what I’m told to do.
|
I remember to respond to the question when I see a reminder in my planner. I visit the board to answer the question I’m told to. I might read other responses, but I don’t reply.
|
Posting on the discussion board is a regular part of my week. I respond to the question and answered anyone who commented on my post. I may reply to another person’s post if I really agree or really disagree.
|
Posting on the discussion board is a regular part of my day. I am often the first or second poster and my posts appear all over the board. I respond to people when I see they don’t have many responses and I try and get others talking by asking open-ended questions.
|
|
Relevance of Post: How relevant is my response to the question or previous post?
|
I am not sure how my post connects to the question or previous post. I had something I wanted to say and was more focused on saying it than on connecting to others' comments.
|
My post starts by connecting to the previous question or post but I head off in another direction, making it difficult for someone to comment on my posting and remain connected to the original posting.
|
My post connects to the previous question or comment through the use of examples, specific details, or by restating the original question. A reader can easily follow the topic from the first post to mine. |
My post connects to the previous question or comment and makes connections between seemingly unrelated questions or posts. I serve as de facto moderator by making connections for fellow posters who appear to go off in another direction.
|
|
Contribution to the Learning Community: To what degree did my post add to the community?
|
I post for the sake of hearing my own voice and seeing my own words. My focus is not on connecting to my community but rather on making sure I'm the loudest on the board or simply to get the task done.
|
I post for the sake of posting. I consider others' postings as a way to generate ideas for my own posts. It doesn't matter to me if my posting contributes to the discussion.
|
I post for the purpose of making personal connections with others in my community. I consider others' opinions and postings as a way to make connections and think differently about my ideas.
|
I post for the purpose of making personal connections with others in my local and world community. I seek out others' opinions and when appropriate, bring in people from outside my physical community who I know can add to the discussion.
|
| Webiquette: To what degree did I follow the rules of engaging with classmates on the internet? |
My posting is tone-deaf. For example, I use offensive language, personal attacks, or otherwise ignore the fact that the other posts are made by actual people.
|
The tone of my posting is irregular.When a reader comes across my postings, they won't know if I'll be polite, use calm language and respect my readers' feelings or, especially when I am emotional about a topic, use offensive language, personal attacks, or forget that I am writing to actual people.
|
The tone of my postings is calm and reasonable. When a reader comes across one of my postings, they'll know I'll be polite, use respectful language, and treat my readers as people with their own opinions and feelings.
|
The tone of my postings is calm, reasonable and welcoming. When a reader comes across one of my postings, they'll know I'll be polite, respectful, and seek out the reader's opinion to expand my thinking.
|
|
Clarity: How easily can the reader interpret my words?
|
A reader’s first impression of my post is how difficult it is to understand me, rather than the content of my message. The reader has to fill in words, interpret my spelling, or decode my abbreviations. I did not proofread before hitting submit.
|
A reader's first impression of my post is that I have something to say, it's just hard to interpret. They can see my big idea but have to infer my meaning or try and find my meaning under too many words. I skimmed my post before hitting submit and caught glaring errors but did not attend to grammar and spelling errors.
|
A reader's first impression is of the content of my post. If I have misspelling or grammatical mistakes, they don't detract from my message. I proofread my post before hitting submit and looked for mechanical errors.
|
A reader's first impression is of the content and clarity of my post. Any spelling or grammatical mistakes do not detract from content. I proofread my post before hitting submit, looking for spelling, grammatical errors and word choice. |
|
Revised by Jennifer Borgioli, Learner-Centered Initiatives, Ltd.
A note regarding copyright - these resources are licensed under a Creative Commons Attribution - ShareAlike 3.0 Unported License
|
Challenges with my rubric:
I have no idea if this rubric will work. I wrote it based on my own experiences on posting boards and it may fail miserably when students use it. While it may work for college students, it may fail for High School or vice versa.
I don't have examplars or anchors to point students to. As a I wrote, I looked at some boards I'm on and thought about different types of posters but if a student asked, I wouldn't be able to give them an example right now.
Feel free to email me at jenniferb@lciltd.org or Tweet me @datadiva if you have any questions about the rubrics.
Comments (0)
You don't have permission to comment on this page.