Rubric data is commonly shown with a bar graph like this.

Note the gaps. I prefer a histogram-like bar graph like this.

I included the normal curve to make following point. If the scale on the horizontal axis has ordered meaning, then the bars on the histogram, if the same width, represent parts of a whole – think placing the bars end to end. This is what we are used to looking at with histograms – area = probability. The shape can be deceptive (see this post) but the idea gets across.

Yet overall I prefer a standard horizontal graph like so.

Here the redder the “badder” and the greener the “gooder” or for the color blind, “lefter” is “badder” and “righter” is “gooder.” One loses the ability to read frequencies directly but their relative size is easily seen. Comparisons work smoothly as can be seen in this graph from a forthcoming assessment committee report.

The problem with any of these choices is how to show prediction intervals (PI) This is a clunky graph from a previous post.

Now the rest of this saga.

The beauty of the Bayesian method as I gleaned from Statistical Rethinking by Richard McElreath was that one can get a distribution for each rubric category by repeated sampling using posterior probabilities. The ability to get means and PI’s and HPDI’s (prediction intervals with minimum width) and also densities comes naturally. I was proud of this graph for 15 minutes.

So proud I showed it to my wife. It is easy to read from afar – nice thick lines and color-coded. But it is deceptive (and mistitled). The overlap of the density plots have no meaning. The four graphs are just mushed together. The overlaps will be large or small depending on the sharpness of the density plots and also on their position. For instance two categories may have the same estimate for the mean.

I was also proud of this graph for a while.

This is similar in style to those in Statistical Rethinking except for larger plotted points and no shading extending between the bars for the PI’s. I particularly liked the contrast between posterior sampling mean and data using large open and filled disks. Yet the information seems to float in space and there is no sense that this is frequency data.

Maybe I could modify the horizontal stacked bar graph. I tried several ways of showing the “fuzz”, the uncertainty between the categories using transparent coloring and shades of gray. Not much success. Here is an example with the grey sections presenting uncertainty (95%).

The grey sections dominate the graph since the sample size was so small and are a bit deceptive since a smaller percentage for one category will necessarily cause a larger value in another category.

A cool feature of the Bayesian tools in Statistical Rethinking is the ability to sample from the posterior distribution. I decided to plot four thousand samples on one graph. I used forty narrow stacked graphs one above the other and iterated one hundred times with decreasing transparency. This gave a fair picture of the fuzz but no ability to visually quantify the sampling error. The last two graphs show how the uncertainty decreases with sample (original data) size.

Gradually changing the transparency one hundred times is overkill but I am out of ideas. So, at this point I will use the latter graphs to show fuzz and revert to a table to compare the various methods.

]]>

Index to Rethinking – Formatted

]]>

]]>

“Here’s what you don’t get to do. When you have children, you don’t get to say, “I was bad at math.” Because you are good at math. You could be better at math. You know how. Just work harder. Many people like to ‘brag’ that they are bad at math. Don’t do that to your kids. Don’t give them any excuses. You are good at math. I know. I grade your work and answer your good questions.”

]]>

One group, looking for perfect square factors, had gotten this far,

One person asked the other, “Now what?” The other replied, “Just * deroot* the

What a great word! Dare I add “*deroot*” to the conversation next time we work such a problem? I think so..

]]>

I woke up a 1:30 am the next morning knowing what Steven meant. At the beginning of the next class I warned my students to work left to right when combining logarithms and meet with Steven after class with the following explanation.

First let’s revisit the original issues remembering this codicil to PEMDAS :

Addition/subtraction are on the same hierarchical level of the order of operations and must be worked left to right. The same holds for multiplication/division.

Here is the issue for addition/subtraction.

The final numbers differ. Our agreement is that the first process, moving left to right, gives the “correct” answer. Correct in this context means what we have agreed such expressions should be evaluated left to right since order makes a difference.

Such expressions can be made unambiguous by remembering the definition of subtraction is just adding the opposite. Thus,

Now any order of evaluation gives the same result since addition is associative.

The same argument works for multiplication/division.

The first evaluation is “correct” and we can “fix” the issue by remembering that division is just multiplying by the reciprocal. Thus,

Now any order of evaluation gives the same result since multiplication is associative.

Now let’s use the properties of logarithms to simplify the following.

Again left to right gives the desired answer. This can be “fixed” as follows.

This simplification uses a property of logarithms that we didn’t make explicit in class, namely,

We have an interesting application of PEMDAS for simplifying logarithm expressions addressing about both subtraction and division.

All of the above to fix (explain) a defect in standard algebraic expression notation. Thanks to Steven for the excellent question.

]]>

]]>

A first question would be “How well does the sample data represent the population?” – a question of confidence intervals. The R function, *MultinomialCI*, based on a paper by Cristina P. Sison and Joseph Glaz for this sample data,

gives these confidence intervals,

which are depicted on this chart.

The function treats the data as simply multinomial without using the ordinal aspect of the data. For such a small sample, n=39, the error is quite large, for instance the rubric data estimates that the student population in the developing category is between 18 and 50 percent.

A second question that could be asked is “How do these students compare to other students?” First I would like to compare before and after data. It might be possible to obtain data on the same students at an earlier time. Here at SOU we compare end of freshman year writing to capstone writing for specific students. To do this we can use the Wilcoxon Rank Sum test for paired data. This is a non-parametric statistical test that takes advantage of the data’s ordinal character. This is the data.

Note the added column the improvement, has After score minus Before score calculated. Using this R command: *wilcox.test(badata$Before,badata$After, paired = TRUE,alternative = “less”)* I got a p-value of .00002 which indicates that there was improvement.

Finally, it is possible to compare two populations with sample rubric data. This can be done with the Wilcoxon Rank-Sum test. The method essentially ranks all the data and sees if one population has more ranks higher than the other. This is the R command: *wilcox.test(badata$Y2014,badata$Y2015, paired = FALSE,alternative = “less”,na.action = na.omit) *Using this data,

R gave an approximate p-value of .48. There was no change from 2014 to 2015.

All this is fairly basic and pro forma and leaves out how to discover effects of other variables like gpa or major for example and there are better ways of doing all of it. I have spent my winter break immersed in Rethinking Statistics by Richard McElreath. This is a wonderful book and opened my eyes to the world of Bayesian modeling. I am attempting to build reliable models for modeling rubric data using the software that comes with the text. The process has been exciting and fulfilling. I will report my progress in Part III of this series.

]]>

My death will break the more direct relations between my present experiences and future experiences, but it will not break various other relations.

When we die our immediate relationships are broken yet our influence on other people and for that matter the material universe lives on through memories people have of us and the artifacts we created, such artifacts, for example, as the words in this blog, the furniture I have made, the stone house I built.

It occurred to me that as teachers we are creating memories of ourselves in others whether we like it or not. These memories will become part of our students’ persons, generally a very small part and be passed on marginally to their children. This is consoling in one way but should also imbue us, teachers, with a sense of responsibility. I recall this sentence by Gilbert Highet,

It is a serious thing to infer with another man’s life.

]]>

In 1932 John Von Neumann, the famous and formidable mathematician, “proved” that a result of Bohmian mechanics was impossible thus putting the theory to bed for, it turns out, 30 or so years at which time John Stewart Bell found errors in Von Neumann’s proof. A few more details to this story can be found here.

The title of this post expresses in some way a triviality. Any action in the present changes the course of events in the future – see every time-traveling science fiction movie. It’s just that we cannot know the magnitude of the resultant ripples in time. Yet, if the alternate theory of quantum behavior has been studied and deepened over that gap of more than 30 years might we not have other wonderful inventions like electromagnetic drive (if it works?)

]]>