Commensurability

A great many things in life can and should be measured, as I believe I've previously written. Measuring different things is crucial to be able to track progress and speed and without metrics, we have to rely on gut instinct. In this respect a certain degree of formality is very welcome, because this relates to the problem of commensurability and comparing one point of data with another one.

Very often, however, it seems that people either on purpose or by accident devise and change metrics at whim. This makes longitudinal analyses of the data pointless; establishing trends becomes very difficult if you are encountered with a data set in which each point has been measured using different methods and perhaps even with a different logic behind. With heterogenous data sets transparency and formal descriptions of how the data has come about are needed if the data set is to be cleaned, and even then it might be tricky.

With this consideration, then, the obvious question is why are people so horribly bad at measurements and tracking different variables? A pessimistic view might be that this is done on purpose. Companies change their metrics on purpose to through stakeholders off by ensuring that a clear picture of trends cannot be established and if someone does attempt to clean the data, the analysis and results can still be questioned. Another reason could be that the people developing the metrics just need something to do to legitimize their existence in the organization, hence new metrics are devised every so often.

An equally worrying notion is that metrics are changed and tracking is executed in a very ad hoc fashion because the people responsible do not at the end of the day understand what they are doing and why. Sheer incompetence. Related to this, perhaps another annoying bit is that people tend to measure what is easy to measure, not what would be useful to measure. Proxy measurements are of course fine as long as you spend enough effort to establish that the logic by which you argue the proxy to be good is rigorous enough. This, sadly, is not the case too often either.

Overall, this is a topic which seems to pop up and cause immense headaches for me ever so often, and I believe the only solution will be to start whacking people with the Clue Stick every time I run across some offender in the future. Sheesh...