As a thought experiment I propose to rate universities using this metric: amount of learning per dollar per year per student. This measure presumes that the primary role of a university is “improving students’ minds” and that efficiency matters. I will assume that I can measure anything in any detail I want. Let’s follow these thoughts down the rabbit hole.
There are contexts here. President Obama wants to develop some sort of system for ranking colleges and universities and giving or withholding financial aid based on such rankings. U.S. News and World Report has a profitable (I think) business ranking schools based on several different types of data and is very influential. Here in Oregon there is a push to measure a university’s worth by the number of graduates and the number of graduates who get jobs. These measurements are based on input data, number of faculty per student for example, indirect and manipulable data, number of applications per number accepted for example, or outcomes that a university can not control, the number of jobs depends on the state of the economy for example. They are for the most part proxies for the statement: “The university’s students must have learned something.” Because, say, its graduates had small classes, or high technology learning tools, or smart classmates, or whatever. The other context is of course that the cost of a college education is escalating, student debt is increasing and students are taking longer to graduate. In contrast this thought experiment is about directly measuring efficient learning and clear definitions of cost and time.
Let’s first attempt to define what “per student” could mean. Student come in many varieties – full time, half time, part time, residential, commuter, nineteen years old, fifty-five years old, under-prepared, over-prepared, honors, first generation, single mother, veteran, recovering addict, incipient alcoholic, different sexes, races, self-identified genders, etc. Are two half time students equal to one full time student for instance? A well-designed curriculum has certain synchronicities that come from a full time load. Universities may be good with one population of students and not another. Somehow already the data will have to be disambiguated and down the rabbit hole we go.
Let look at possible meanings for “per year”. There is the freshman year, the senior year, the stop-out year, the foreign travel term, etc, Students are on a six year plan or a three year plan. The amount of learning in a typical senior year should, all other variables constant, be more than in a typical freshman year since by then students will have learned how to learn. Another measure could be years until graduation but that would lose students who transfer in and/or out or become part-time for awhile or take a year off to earn enough money to go back to the university. Maybe a smaller unit could work. Certainly not “per day” since learning curves are not linear. “Per quarter or semester” might be reasonable though this too does not capture the difference between freshman learning and senior learning. Another rabbit hole.
Let’s try to define “per dollar”. Do we mean just what a student spends on tuition and fees? Do we include scholarships and financial aid which vary widely among institutions? Do we include state subsidies? We certainly should include the cost of books and supplies which vary greatly by major. What about private tutors or internet access? Do we include room and board? On campus or off campus? Do we include the present value of the difference between student loan interest to be paid adjusted for future inflation? Transportation costs? Child care costs? Maybe it is better to look at the expenditure side. How much money does a university spend providing undergraduate education? I might not be too hard to separate faculty salary into categories of undergraduate teaching, graduate student support and research. But should we include capital costs? How will we include costs of books and supplies? How much administrative costs do we attribute to undergraduate education? Do we include advisers’ salaries and benefits? Health workers’ salaries? Lots of complications (See the periodic discussion, say, of the California Community College Fifty Percent Law). All this is very complex – another suburb of Wonderland.
The difficulties of defining typical students, time units and expenditures pale in comparison to measuring learning. Since learning is fundamentally a physiological change in the brain, we will have to do before and after measurements of neural chemistry or connectivity or something. Since this is a thought experiment let’s for the moment assume we can do so. We just need to choose a start time and an end time. People are learning all the time whether they are in college or not so we will need baseline measurements. We will have to measure brain changes in non-college students in otherwise similar circumstances.
What change in the brain should we be looking for? On surface this fairly easy. Most accredited universities have mission statements and consequent goals. Let’s look there. Our mission here at SOU is fairly typical: “SOU is an inclusive campus community dedicated to student success, intellectual growth, and responsible global citizenship.” We just have to define our terms and then find the parts of the brain that predict success and responsible global citizenship. Measure before and after and we are done. Another rabbit hole.
Universities have general education goals. Maybe this is a better place to look. SOU, for example, has a quantitative reasoning strand, “Effectively formulate and use mathematical models and procedures to address abstract and applied problems”. Now all we have to do is identify the part of the brain that does this and measure before and after neural complexity or chemistry or something. Or we could give a test (two tests – before and after) that matter to students, i.e. affects their grades, that ask them to effectively formulate etc. It really should be a project since modeling is usually a time-consuming iterative process. We will need to verify that the students did the work themselves. The Collegiate Learning Assessment does something like this with no before and after and no consequences for the student. By the way our accreditors want us to be assessing this. They also want us to be assessing our Citizenship and Social Responsibility strand: “Understand and apply moral standards to individual conduct and citizenship through ethical inquiry.”
I think I have made my point. Measuring the true task of a university – student learning – is incredibly hard though required (accreditation). Ranking universities based on their primary mission is hard. I will leave it to others to criticize and justify proxies for such learning. I come back to the argument I made a few years ago. The true measure of a university lies in what it requires students to do, the standards it sets, and the integrity of their processes.