About the correlation and not only

Sometimes, having data on hand, you feel the lack of standard tools. This is especially the case when the numbers are a dynamic process that constantly strives to change the internal state.
Under the cut, the author will try to show how, using a simple trick, from ordinary data you can pull out mountains of various information. In these mountains, you can find the most intimate details of the process under study, the only question is in curiosity and a certain amount of luck.

We take two simple curves as the initial data - the average speeds for some categories of roads in a couple of cities of our vast Motherland.

Just for information, we see a cut of data for the week in 15-minute increments.

What could be done with standard tools? Go through the moving average, build the distribution, find the variance, co-variation / correlation, make the convolution. This will give us a few numbers, a few new curves. And if this is not enough to understand what is actually happening there?
')
Recall that we have two intervals on our hands, each of which can give a bunch of sub-intervals. Comparing the two sub-intervals in turn can give us a number (or numbers). Choosing ways to generate the sub-intervals of the two original curves, we begin to generate at least two-dimensional data. For example:

• Fix the size of the sub-intervals
• On the abscissa axis we will postpone the distance from the beginning of the interval to the beginning of the curve.
• The ordinate is the same, but for another curve
• The value will be considered the correlation of two sub-intervals

For example, here is a picture for sub-intervals with a width of 45 points:


And here is the same, but for a width of 127 points:


Now add some noise to the source data,
We fix the size of the sub-intervals of 127 and we will mix noise with increasing amplitude to the data. For example, a picture for noise with an amplitude of 9


But with an amplitude of 69:


Finally, we will look at the ratio of the amplitudes of the main harmonic (corresponding to the day cycle) after Fourier with windows of 256 points.


and on the phase difference of the main harmonics after Fourier with windows in 512 points


Morality. Hmm ... yes, morality.
Do not be lazy to look for a black cat in a dark room. There and in addition to the cat there may be something interesting.