MAA Books Beat: Knowing vs. Measuring: Doing the Scholarship

Written by Steve Kennedy, MAA Acquisitions Editor, Knowing vs. Measuring: Doing the Scholarship appears in the December 2014/January 2015 issue of MAA FOCUS.

Last winter I participated in the tenure review of a junior colleague, watched with interest as Miguel Cabrera defeated Mike Trout in a close American League Most Valuable Player contest, and read a draft version of Curtis Bennett’s and Jackie Dewar’s Doingthe Scholarship of Teaching and Learning in Mathematics. It could just have been the temporal proximity of these experiences that led me to see analogies between them, but I think there are real connections.

Measuring

First, and most frivolously, the baseball: There is a huge, fascinating, and contentious debate going on in baseball these days that we might characterize as traditionalists versus statheads. Even if you are not a baseball fan, you might know about this from the movie and book Moneyball.

Baseball players have traditionally been measured by their batting averages, home runs, and runs batted in. All of these are easy to count, have obvious meaning, and have value that is clear to even the most casual baseball fan. All are limited as a measure of what they ostensibly represent: the player’s contribution to his team winning the game.

For example, batting average is a rough proxy for the frequency at which a batter gets on base, but it does not count walks, or reaching base on an error; it values a single and a triple identically; and it ignores certain outs that achieve other (good) outcomes, such as advancing another baserunner.

None of these three stats cited even tries to measure baserunning skill, or the ability to reach base by walk or error, or defensive prowess.

A number of more advanced statistical measures of a baseball player’s performance are in use today. WAR, Wins Above Replacement, is one such. It attempts to measure the total (batting, defensive, baserunning) contribution of a player to his team’s success.

In 2012 Mike Trout, the young centerfielder for the Los Angeles Angels, posted a WAR score that was among the two dozen highest in modern baseball history. That is out of tens of thousands of individual seasons.

In the same season, Miguel Cabrera led the league in all three of the traditional statistics: batting average, home runs, and runs batted in. This is, in baseball lingo, called winning the Triple Crown. It had happened only 15 times previously in baseball history and not since 1967. Cabrera had a batting season among the best of all time, but he is a slow runner and not a very good fielder.

WAR (and other advanced statistical metrics that include baserunning and defense) rated Trout’s season as very strongly more valuable than Cabrera’s. The debate over the MVP award was widely portrayed as a battle between crusty, tradition-bound baseball old-timers against basement-dwelling, smart-aleck nerds.

Essentially the same thing happened in 2013. Cabrera dominated the old-fashioned offensive categories; Trout lead the league in WAR, again by a wide margin.

Recognizing

On campus, as I watched my young colleague teach and evaluated her performance, I thought about this baseball controversy and wondered about the wisdom of the exercise I was engaged in. I teach at an institution where quality of instruction is the primary criterion upon which faculty are evaluated. And our measurement instrument consists of sending in grizzled, gray-bearded veterans to watch and decide if they like what they see.

I wanted something I could measure. (Oh sure, we do a student opinion survey, but I wanted to directly measure learning and teaching.) Essentially we were a bunch of Potter Stewarts (“I know it when I see it.”), and I yearned for a bunch of Nate Silvers (who, famously, knows it only when he can measure it).

Miguel Cabrera won consecutive MVP awards, which, to a scientist, is the wrong result reach by listening to experts instead of data. Baseball awards are not so important. But getting my colleague’s tenure decision right has real human and institutional consequences.

Researching the Difference

Bennett and Dewar’s book on the scholarship of teaching and learning (SoTL) came as a revelation to me as I wrestled with the difference between knowing good teaching (or baseball) and measuring good teaching. It is designed as a primer for mathematics faculty interested in doing research in teaching and learning. It describes how to ask a good question, how to design a study, how to analyze the results, and how to get your results into print in the literature. It is a unique, and uniquely valuable, resource for mathematics faculty who wish to make a transition from traditional mathematics research to pedagogical research.

In the first four chapters Bennett and Dewar describe the principles and procedures of engaging in SoTL. These chapters are informative and enlightening.

The next 13 chapters are inspiring. They describe individual studies by mathematics faculty who tried an innovation in a course and then tried to quantify and measure its effect on learning.

One very nice feature of all 13 of these essays is that the authors are all cognizant of their volume’s intended purpose as instructional manual for neophyte scholars of pedagogical research. Thus, each author shares information about motivation, stumbling blocks encountered, resources discovered, how to find colleagues and collaborators, and how they got their findings into print. That is, these essays are not just reports on how to improve teaching and learning, but they are also case studies on how to conduct and report your own experiments.

Despite the volume being designed for faculty embarking on a professional turn toward SoTL, it has great value for anyone teaching mathematics.

In one of the case studies, Rann Bar-On, Jack Bookman, Benjamin Cooke, Donna Hall, and Sarah Schott describe their transitions from good to reflective to scholarly teachers. Roughly, a good teacher seeks to motivate, challenge, and support his students; a reflective teacher thinks about what went wrong and tinkers with her approach; a scholarly teacher designs scientifically valid experiments of her pedagogical ideas and implements them.

Wherever you are in this taxonomy, there is much that you will earn from this volume. There are dozens of implementable pedagogical ideas that will inspire you here. You will learn much about how to measure their effectiveness in your own classes. You will be energized to go into your class tomorrow and try something exciting.

Books like this are the reason the MAA has a books program—so that MAA members can come together and participate in a national conversation about improving the teaching and learning of mathematics.