Assessing Standards by Depth Of Knowledge

As we’re approaching spring, many states (including Michigan!) are anticipating the arrival of state tests, possibly with much fear and trepidation.

In Michigan, we use the M-STEP, which is derived from the Smarter Balance Assessment. Michigan was a governing state in the development of that assessment, and though Michigan chose to change the name of the test in our state before launching it, we still assesses the same four claim areas in mathematics.

The question we have been hearing from teachers as we have been in classrooms the past few months has been, “What should we be targeting?”

That’s a tough question to answer. Typically, we turn to our district-purchased math books to find the questions to ask the students, but as I work with eight different math programs in our different districts, I’m shocked at the differing levels of complexity contained in books intended for the same grade level.

In trying to make sense of the drastic variation between books, I started to question our understanding of Webb’s Depth of Knowledge (DOK) within the world of elementary education. We talk a lot about Bloom’s Taxonomy and its levels of understanding, and we want to continually spur students on towards higher levels of knowledge. In math, the 8 Mathematical Practices are the entry point for larger level of DOK questions.

DOK Discrepancies in the Classroom

We see the impact of DOK levels when some classrooms take the end-of-module tests and only 37% of the kids pass with proficient. Teacher are baffled that the scores could be so low after they’ve just taught the content. So, we started analyzing the kinds of questions being asked in various programs to see if they were a fair measure of students’ progress towards mastery of a particular standard. Some programs such as Eureka Math/Engage NY have very high level DOK questions on their exit tickets and various tests. So, when kids aren’t successful on these assessments, it might just be that they don’t have a DOK level 3 or 4 grasp of the standard because the instruction was on a DOK level 1 or 2. Other programs, like Math Expressions, ask questions on lower DOK levels, but then our state tests ask questions on a DOK level 3 or 4.

Within our instruction, we must ensure we map out a great variety of questions at various levels to gauge our students’ understanding of the concept. It’s not about taking a 4th grader who is excelling and advancing him to a 5th grade level, but rather taking him to a higher level DOK to advance his knowledge of the content. The only way we can truly prepare our students for any assessment, including a state test, is to increase their depth of knowledge.

This year, we’re really working with our schools to establish essential standards. Some states call them power standards, priority standards, essential standards – there are lots of different names for them – but these are the standards within a grade level that, when taught to mastery, solidify a student’s understanding and create a solid foundation for further knowledge. But, if we establish essential standards for a school, we must also determine how those standards will be assessed. How will we go about looking at where kids are within their level?

Understanding the DOK Levels

One tool that is very helpful for teachers seeking to understand DOK is CCSS Math Activities (https://ccssmathactivities.com/). Julie, one of our math leaders, shared this site with me and I’ve been amazed at the resources that are available! Though the site is integrated with Eureka Math/Engage NY, the K-5 support page allows you to select your own grade level, and provides a compacted pacing guide that looks at where students are supposed to be able do after they’ve been introduced to a certain topic. It helps you know what standards precede the current standards, what lessons you could omit, what lessons are identified as remediation and which are accelerated. The idea as you’re looking at this series of lessons is not so much whether you got through every single lesson, but did that child master that particular standard based on what we taught? And at what DOK level does the child understand the complexity of that standard?

When you click on the snapshot assessments – it’s kind of a glorious find!  This particular program organizes by module, where you can download the snapshot assessment for all of a grade level, by module, and it will give it to you by the standards. For other schools that aren’t using this math program, you can still download the assessments and store them in folders based on standards (I have a Fractions folder, a Numbers/Base Ten folder, etc.).

Teachers can find their grade level, let’s say 3rd grade, and the concept, let’s say numbers and fractions, and then identify the standard they are working on, let’s say standard 1. Then, voila!  There’s the snapshot. It has 2 or 3 (4 at the most) questions that are actually labeled by DOK.

Some of the questions actually are labeled “ceiling,” because there are ceilings within the DOK for certain grade levels. For example, in 2nd grade, they only need to know some of the standards at a DOK level 2, so they wouldn’t have to answer higher level application questions because obviously that will come back when they get older.

As we’ve been analyzing the level of these questions, I wonder: Do we, as teachers, really understand these DOK levels? Looking at these snapshots really opened my eyes to how much more we really need to focus on DOK in schools. We need to look at the complexity of questions to be able to increase the rigor for students who are above grade level. But also, for students who are at-risk or behind grade level, we need to be able to identify exactly where they in their DOK and understanding.

 

DOK Dash

Investigating DOK levels can be fun! We pulled questions from CSSS Math Activities and removed the DOK label, and teachers practice sorting by DOK level. I was at Central Park Elementary in Midland, Michigan. Each grade level has a race to see if they can lay out their questions and match them to the correct DOK level. The last part of the activity is to create a task they think would be appropriate for DOK level 4 to see if they can connect all those pieces.

They soon discovered that there weren’t very many DOK 4 questions – those are really high-level tasks.Some of them needed hints – they noticed that there weren’t any DOK level 3 questions in Kindergarten. A lot of them had trouble deciding if the questions was a 2 or a 3. It depends on if it wants you to explain your thinking, which could still be a DOK level 1, based on only having to explain your thinking around that procedure, which is at a basic knowledge level.

Common Assessment

Increasing awareness of DOK levels in districts and math committees, and really encouraging discussion around these levels is so helpful. You’ve got to be able to compare apples to apples, so identifying a common assessment, especially when common power standards have been identified, is vital. Each school is in a different situation when it comes to power standards and common assessments.  I have one school that has a really great set of power standards already in place, and they even have a great color-coded system of determining where students are in working towards mastering that standard (red, yellow, or green), but when I ask what tool they’re using to assess mastery, they’re all using something different – TPT, Pinterest, or other teacher-created things – so there is no continuity in the DOK level assessment, which weakens the impact of having the power standards.

Another school I worked with recently was working on DOK level questions because they have their priority standards identified, but they were working with unclear definitions of the DOK levels. One teacher decided to label all story problems that required model drawings as DOK level 2 or 3. In reality, the story problem might be a part-whole or additive comparison problem, which actually makes it actually just a DOK level 1. Yes it was within a story problem, but it was only asking them to do a simple computation. Multi-step problems, where you have to answer the first part in order to answer the second part, or problems that have students draw conclusions, require students apply information, so those types of story problems would move into a DOK level 2 or 3.

It doesn’t need to be a formal assessment to determine a students’ DOK. The concept of a Jot Lot could be used as a way to gather reflective feedback from students to see where they’re going. You could have exit ticket-type questions that are at different DOK levels to see where kids are doing it. If a kid can do a simple fraction with uncommon denominators and solve it, can they do the application of it at higher levels?

Additional Resources for Understanding Depth of Knowledge

The Common Core Companion, which we’ve talked about before, is another great resource to use in conjunction with DOK levels. As we look at the DOK levels and the standards, we start to ask if we really actually understand the standard…when teachers are overwhelmed by all the different resources, as they often are, that they can look at the Common Core Companion and see, What does the standard say? What is the teacher doing? What is the student doing? What is the misconception?

This all circles back around to the concept of backwards design, that we’ve also talked about before. Many teachers feel scattered in elementary school because they are teaching all of their subjects and to trying to keep the end in mind with all of them, asking themselves where they want to go with their instruction as they look at their standards. Teachers today also have so many assessments available to them – chapter test, unit tests, state tests, the PSAT in 8th grade – we’re inundated with all these different things. However, if my district establishes a common assessment by DOK level, and I have a good understanding of DOK levels, I will be able to look at a standard, know what level my students should be on, and I can be more effective and focused in my instruction.

Illuminate (https://www.illuminateed.com/) is a bank of test questions that is popular in Michigan. Many of our ISDs have purchased licenses for our districts. The are many resources that are like this, but this is one that I’ve personally used while working with a school district recently and I was happy with the ability to pull questions by DOK level and by standard. The questions are given to students online, which is good practice for some of the state testing, but you can also print it out and customize your own test.

One more great resource for Eureka Math/Engage NY users is Affirm (https://eurekamath.greatminds.org/digital-assessments), which is in association with GreatMinds, the sister/housing company of Eureka Math/Engage NY. It is a paid resource, but, in a few of the webinars that I’ve seen, they’ve recognized that the Eureka Math/Engage NY tests are at a higher level, and so they’ve created this bank of questions through Affirm that go all the way up through middle school (3000+ per grade level), and are all from different DOK levels.