I write about applications of data and analytical techniques like statistical modelling and simulation to real-world situations. I show how to access and use data, and provide examples of analytical products and the code that produced them.
I compare some different ways of forcing the coefficients in a regression to form a simplex, all greater than zero and adding up to exactly one. Two methods - quadratic programming, and explicit modelling of the coefficients from a Dirichlet distribution - give essentially identical results that match the data generating process well.
I was honoured to give the third and final Ihaka Lecture in the 2024 series. My talk had the theme "Making R work in government".
I explore the number of ways to make a prime number as the sum of squares of three positive integers.
Stepwise variable selection is bad and dangerous, and you shouldn't do it. It increases false positives. It drops variables that should be in the model. It gives biased estimates for regression coefficients. The problems are worse for smaller samples; higher correlation between the X variables; and models with weaker explanatory power for the y (i.e. lower R-squared).
I familiarise myself with the Australian Bureau of Statistics' statistical Standard on sex and gender, and play around with some data from the Australian General Social Survey that has outputs reported by persons' sexual orientation.
I play around with sampling from finite populations with unequal probabilities, where the R sample() function turns out not to work the way I had expected it to.
I draw some quick charts of Australian economic indicators and ponder the implications. A ratio of two indexes can be a useful part of exploratory analysis.
I draw maps of the largest settlements closest to the north pole and to the south pole, based on an idea by 'Brilliant Maps'.
Inspired by a Toot from Thomas Lumley, I explore a situation where adding random noise to a distribution changes the median but not the mean.
When playing Snakes and Ladders with the common rules actually used, it is more complex than a simple mathematical model; I simulate games and put forward some findings that could be useful in a future high stakes Snakes and Ladders game.