In today's chapter we will move on to the topic opened in the previous article of Coffee with IOT: Coffee with IOT. Chapter 4: Descriptive statistical parameters I.
Special forces
Given the possibility that the negotiation process may fail, the decision has been taken to rely on the support of the special forces. This unit is specialised in action, working in areas ranging from dispersal of concentrations to the reduction of dangerous subjects.
To give an example, among his latest achievements is the resolution of a case in a restaurant-bar-cafeteria-taca. On that occasion, a skewer of omelette was eaten with bread by four customers and a waiter-bartender-who-at-the-same-time-is-an-employee, and after his refusal to leave with the potatoes up, it was executed with the use of a microwave, a mini blender and two turmixers.
A not-so-pretty graphic
As we saw a couple of articles ago, trying to understand a set of data is not an easy task. That is why, in the previous article, we introduced descriptive statistical parameters, a series of numerical values that help us to understand and summarise the set of values that our data take. We also saw a typical classification of them, and developed the first of these types: the measures of centralisation. Today, we will look at a second group of this classification: the dispersion parameters. A set of statistics that allow us to know how wide the values of the data are.
The deviation
The first approach to calculating dispersion is usually to measure the difference between each of the data and a centrality statistic. We call this deviation, and it allows us to know how concentrated or dispersed our values are with respect to the centre.
Let's look at a very simple example of deviation from the mean:
| Value | Media | Deviation |
|---|---|---|
| 1 | 4 | 1 – 4 = -3 |
| 2 | 4 | 2 – 4 = -2 |
| 2 | 4 | 2 – 4 = -2 |
| 5 | 4 | 5 – 4 = 1 |
| 5 | 4 | 5 – 4 = 1 |
| 6 | 4 | 6 – 4 = 2 |
| 7 | 4 | 7 – 4 = 3 |
The problem with calculating the deviation of the values from the mean is that if we add up the deviations, the result is always zero. Let's look at our example:
(-3) + (-2) + (-2) + 1 + 1 + 2 + 3 = 0
- Variance: 47482.14 watts2
A typical solution to avoid this zero is to square the values before adding them up. And if we divide this sum by the number of data we have, we get the variance.
| Value | Media | Deviation | Squared deviation |
|---|---|---|---|
| 1 | 4 | -3 | 9 |
| 2 | 4 | -2 | 4 |
| 2 | 4 | -2 | 4 |
| 5 | 4 | 1 | 1 |
| 5 | 4 | 1 | 1 |
| 6 | 4 | 2 | 4 |
| 7 | 4 | 3 | 9 |
If we add up the values in the last column and divide by 7 (which is the amount of data we have), we get that the variance of our example is 4.57, and with this we seem to have solved the zero problem.
- Standard deviation: 217,904 Watts
Now, if our mean is 4, and we have values that deviate at most 3 units from the mean, it is unintuitive to understand what it means that the statistic chosen to measure its dispersion is 4.57.
On the other hand, there is a problem that I have been avoiding addressing so far. The data we are dealing with are not just numbers out of a workbook. In our case, they are measures, i.e. numerical values accompanied by some kind of unit. So squaring our data squares its unit, and this makes it even more difficult to interpret the variance.
Suppose that in our example, what we measure are humans. So we have 1 human, 2 humans, ... 7 humans, with a mean of 4 humans and a variance of 4.57 humans squared.
One possible solution is to undo the operation of squaring the values by applying a square root to the variance. We call this the standard deviation, and it is the most commonly used dispersion statistic. In our case it would be 2.13 humans, which we could interpret as typically having 4 humans and deviating a couple of humans up or down (from 2 to 6 humans).
La Différance
On 27 January 1968 Jacques Derrida gave his speech on différance, a neologism that plays with the ambiguity of its sound in French and evokes the polysemic verb différer. In Spanish we would find that this term is related both to difference, in the sense of different or distinct, and to deferred, in the sense of something that occurs with a time lag, something postponed.
There is much to comment on the term, but what I am interested in emphasising is how these kinds of ideas embody the difference that exists between meaning and signifier, between what we want to say and what is actually said, between the idea and the word, or between what is said and what is heard, between what is written and what is read.
If I say "house", what do you mean? Home, building, maybe shanty, or mansion, maybe a boarding house, maybe a lodging house, maybe you mean Gryffindor or Stark, or maybe you mean someone officiating at a wedding, or, if you're from a region where you're from, maybe you mean hunting witches. I like to think of the park bench as 'home' when I play tag.
Did you know that, as Eduardo Sáenz de Cabezón says, I have more eyes than the average (and possibly you do too)? If we take it for granted that you can have 0, 1 or 2 eyes, the average must be less than 2. What interests me is the discrepancy between what is represented and its representation. Between 0 and 1 there is the same difference as between 1 and 2. But between 0 and 1 eye, there is not the same difference as between 1 and 2 eyes.
After reading Derrida, you have to read Saramago.
Dispersions of descriptive statistics parameters
There are other dispersion statistics that we can calculate depending on what we need to know about the data or the use we are going to make of them: winsorised standard deviation, mean adjusted percentage variance, mean bipound variance, mean absolute difference, distance standard deviation, mean relative difference, Allan variance, Hadamard variance, quartile dispersion coefficient... But let's look at just a few basic cases:
- Median absolute deviation: 0.167699 watts
Another way of dealing with the problem of deviations is to use the median as a reference of centrality. This is the case of the median absolute deviation, which is the median of the absolute values of the differences between the data values and their median. This gives a robust dispersion statistic that is less affected by outliers or extremes.
For our example, we obtain that the median absolute deviation is 2, i.e. our data is centred on 5 humans and deviates 2 humans up or down (between 3 and 7 humans).
- Range: 1772,588 watts
The range belongs to a group of measures known as scales, which tell us how far apart our data can be. Its calculation is simple. We subtract the minimum value from the maximum value, so in the example we are following, the range would be 6 humans.
- Interquartile range: 0,335398 watts
The interquartile range is one of the robust versions of the scales. If we assimilate the minimum value to 0% of our data, and the maximum value to 100%, the interquartile range is calculated by subtracting the value corresponding to 75% of the data minus the value corresponding to 25% (we will see this better in the next article in the series). Thus, in our example, we can say that the central half of the data differs by at most 3.5 humans.
- Coefficient of variation: 307.0466
All of the above statistics have a problem: they do not allow us to compare how dispersed the data are between variables of different measures or between variables of the same measure with large differences in magnitude. For example, with the above statistics we cannot know whether humans are more homogeneous in terms of weight or height. Or we cannot know whether the weight of humans is more heterogeneous than that of elephants. Therefore, it is sometimes necessary to use relative dispersion statistics, which are scale-independent and unitless measures.
This is the case for the coefficient of variation, which is calculated by dividing the standard deviation by the mean, and multiplying by 100. Thus, for our human example, the coefficient of variation is 57.73503. In other words, our example varies about 5 times less than the electricity consumption of the Future Space coffee machine.
And what do we still need to know about the statistics that explain the electricity consumption of the coffee machine?
We will see this in the following deliveries of IOT coffee
