Time series data may be “transformed” in a variety of ways, depending on the type of data (e.g. numeric or boolean).
Folding
Folding is probably the most common transformation used. Folding time series data is the process of taking many values within a specific time range and “folding” them into a single representative value for the entire time range. The fold function determines how a raw history is “folded” when the history is being rolled up. Consider the following raw history, recorded at 1minute intervals:
ts | My Point |
2022-01-01 00:00 | 1 kWh |
2022-01-01 00:01 | 2 kWh |
2022-01-01 00:02 | 3 kWh |
2022-01-01 00:03 | 2 kWh |
2022-01-01 00:04 | 1 kWh |
2022-01-01 00:05 | 2 kWh |
2022-01-01 00:06 | 3 kWh |
2022-01-01 00:07 | 4 kWh |
2022-01-01 00:08 | 3 kWh |
2022-01-01 00:09 | 2 kWh |
If we were to roll this history up into 5minute intervals, we would get different results depending on how we fold the raw history. The table below shows the resulting values for fold functions of “sum”, “avg”, “min” and “max” respectively.
ts | My Point foldFunc: sum | My Point foldFunc: avg | My Point foldFunc: min | My Point foldFunc: max |
2022-01-01 00:00 | 9 kWh | 1.8 kWh | 1 kWh | 3 kWh |
2022-01-01 00:05 | 14 kW | 2.8 kWh | 2 kWh | 4 kWh |
The available fold functions depend on the data type.
Interpolation
Interpolating time series data could be considered the inverse of folding time series data. With folding, many values are transformed into a single value. With interpolation, two or more values are transformed into many values.
Interpolation is one method used for filling gaps in data.
Consider the following history, which has data every 5 minutes but nothing in between:
ts | My Point |
2022-01-01 00:00 | 5 kWh |
2022-01-01 00:01 |
|
2022-01-01 00:02 |
|
2022-01-01 00:03 |
|
2022-01-01 00:04 |
|
2022-01-01 00:05 | 10 kWh |
2022-01-01 00:06 |
|
2022-01-01 00:07 |
|
2022-01-01 00:08 |
|
2022-01-01 00:09 |
|
2022-01-01 00:10 | 5 kWh |
There are three approaches to interpolating data. The correct approach to use depends entirely on the data in question:
Linear - the data is simply linearly interpolated between known values. Usually applied to sampled data such as temperature trends.
Change of Value - all gaps in the data are filled with the last known value. Usually applied to set point trends or boolean data
Apportion - the size of the gap is determined and all known data is evenly apportioned across the gap. usually applied to consumption data such as energy usage
The table below demonstrates the three approaches:
ts | Orignal | Linear | COV | Apportion |
2022-01-01 00:00 | 5 | 5 | 5 | 1 |
2022-01-01 00:01 |
| 6 | 5 | 1 |
2022-01-01 00:02 |
| 7 | 5 | 1 |
2022-01-01 00:03 |
| 8 | 5 | 1 |
2022-01-01 00:04 |
| 9 | 5 | 1 |
2022-01-01 00:05 | 10 | 10 | 10 | 2 |
2022-01-01 00:06 |
| 9 | 10 | 2 |
2022-01-01 00:07 |
| 8 | 10 | 2 |
2022-01-01 00:08 |
| 7 | 10 | 2 |
2022-01-01 00:09 |
| 6 | 10 | 2 |
2022-01-01 00:10 | 5 | 5 | 5 | 5 |
Default Values
Using default values is another approach for filling gaps in data. Very simply, the default value provided is substituted into the history whenever there is a gap.
The history below demonstrates the use of a defaultIfNull value: (notice the missing value in the raw history)
ts | My Point raw history | My Point defaultIfNull: 0kWh |
2022-01-01 00:00 | 1 kWh | 1 kWh |
2022-01-01 00:01 | 2 kWh | 2 kWh |
2022-01-01 00:02 |
| 0 kWh |
2022-01-01 00:03 | 2 kWh | 2 kWh |
Unit Conversion
Unit conversion can be applied to numeric data, once the data has a unit, to begin with. It must be possible to convert from the original unit to the desired unit, otherwise, an error will occur. This is usually enforced by ensuring the two units belong to the same quantity.
The history below demonstrates the use of a toUnit to convert from kWh to Wh:
ts | My Point raw history | My Point toUnit: Wh |
2022-01-01 00:00 | 1 kWh | 1000 Wh |
2022-01-01 00:01 | 2 kWh | 2000 Wh |
2022-01-01 00:02 | 3 kWh | 3000 Wh |
Quantity Conversion
With time-series data, it is sometimes possible to convert data from one quantity to another, when the only difference between the two quantities is time.
Consider mass and mass flow. The only difference between these two quantities is time:
Quantity | Unit |
Mass | kg |
Mass Flow | kg/s |
In a history grid, each row represents a set duration of time. It is the time between this row’s timestamp and the next row’s timestamp.
Consider the example history grid below, where each row represents 1 minute of time:
ts | Mass Flow Point | Mass Point |
2022-01-01 00:00 | 1 kg/s | 60 kg |
2022-01-01 00:01 | 2 kg/s | 120 kg |
2022-01-01 00:02 | 1.5 kg/s | 90 kg |
If we know the mass flow and the time interval, we can determine the mass and vice versa. Other examples of quantity conversions are as follows:
Quantity A | Quantity B |
Mass | Mass Flow |
Volume | Volume Flow |
Power | Energy |
Distance | Velocity |