About CMTime

Hi, Habr! I propose to get acquainted with the translation of the article by Warren Moore on the understanding and application of CMTime. In my previous post related to working with video , I promised to tell you more about these wonderful and so important structures. The main focus will be on the importance of looking at the time, describing the structure of CMTime, but without the simplest examples will not do.

About the importance and accuracy of time evaluation

Most people do not need to think about the accuracy of time. Take the extreme case, perhaps you are an Olympic runner who cares about the difference in milliseconds between your speed and the world record on the hundred-meter mark. These results are almost the same. However, when it comes to timely information, we take care of very short intervals (tens of microseconds), but sometimes these segments are several days or weeks.
Suppose you need to set the exact frame in the film, for example, at 35:06 . It will naively represent time as a double-precision floating-point number, for example:

NSTimeInterval t = 2106.0; 

Quite often, such a representation fits well with the conditions of the problem, but is inapplicable when it comes to very long intervals with fragmentation into small components. Without going into the specifics of floating-point numbers, a double can contain 16 (in decimal) significant digits in eight bytes:

 sizeof(NSTimeInterval) == sizeof(Float64) == sizeof(double) == 8 

Floating point numbers are one problem: repeated operations (addition, multiplication, etc.) lead to an increase in inaccuracy, which can lead to a noticeable discrepancy after a long period of time. And this, in turn, can lead to errors in the synchronization of several streams of information.


')
This is what prompted us to find a way to more accurately represent the time, ending with the use of a floating point number and this inaccuracy (not to mention the hard behavior when rounding).

Time as a rational number

There are many data structures created by Apple for representing the time for both the MAC OS and iOS. With the advent of MAC OS X 10.7 and iOS 4, two more have been added:

 CMTime CMTimeRange 

Having dealt with the first, you will easily understand the second, so the article will focus the reader’s attention on CMTime .

In fact, CMTime is no more than a four-member sash structure, and looks like this:

 typedef struct { CMTimeValue value; CMTimeScale timescale; CMTimeFlags flags; CMTimeEpoch epoch; } CMTime; 

Next, consider in detail the value and timescale , but the flags deserve a mention. Different values ​​of the flags show us that the timestamp ( hereinafter, the time interval ) can be equal to the plus or minus of infinity, or it has been rounded as a result of some calculations. Thus, the structure is more expressive in comparison with NSTimeInterval and has a number of advantages. What is actually expressed in CMTime ?

It is very important to understand that value and timescale are stored as integers: 64 and 32 bits, respectively. This should be obvious from the text above, since value is presented in such a way as to avoid errors identical to those given in the example. In addition, by allocating only 64 bits to the denominator, we can use up to 19 decimal digits for each of the possible timescale values.

What is timescale ? It represents the number of “slices” into which each second is divided. This is important because the accuracy of the CMTime object as a whole is limited to this value.



How to choose a reasonable timescale , so as not to get a cut piece? Apple recommends 600 for video (explaining that 600 is universal for most videos with a frequency of 24, 25, and 30 frames per second). You may want to set the value to 60000 if you need precise indexing for the audio file. What else is cool about this 64-bit value is that you can imagine 5.8 million years in steps of 1/60000 seconds along the way.

The number of seconds in a time span is nothing more than t.value / t.timescale . You can use CMTimeGetSeconds to easily convert CMTime to float64 in such a way as to ensure maximum accuracy.

Not-so-and-convenient convenient methods.

CMTimeGetSeconds may be a friendly and useful method, but its wicked twin brother, CMTimeMakeWithSeconds is far from being so friendly to our brother:

 CMTime CMTimeMakeWithSeconds ( Float64 seconds, int32_t preferredTimeScale ); 

The first couple of times when using this function, I could not get her to do what I wanted ( comment of the translator , damn vital). Now that we have gone through all the insides of CMTime , I hope that you will understand my mistakes. I tried performing the submission in 0.5 seconds in order to receive periodic callbacks from an AVPlayer object (streaming MP3).
I tried to do the following:

 CMTime interval = CMTimeMakeWithSeconds(0.5, 1); 

If you've read this far, you know that the interval will actually be zero, not 0.5. It does not make sense to request half of the unit in the sequence of integers, but this is exactly what I tried to do. Our value has been truncated because timescale is not accurate enough here.

The Core Media API includes a complete set of functions for building, comparing, and performing CMTime- related arithmetic operations. Although it is not easy to perform arithmetic operations on CMTime using these functions, it is important to use them if you want to maintain accuracy and integrity. Each of these functions performs special overflow and rounding checks and sets the appropriate flags for CMTime objects when necessary.

To add two CMTimes , use the CMTimeAdd function. For comparison, use CMTimeCompare (the return value of this function follows the convention of the function comparator, which is used in the standard C library and is known by the name qsort ). To learn more from CMTime , use CMTimeMaximum . Familiarize yourself with the documentation for learning the other methods, but this set of functions is usually sufficient.

Uncertainties and infinity: set your “weird” flag

The final part of the CMTime related article is the presentation of flags of infinity, undefined values, and rounding. An “indefinite” time is a time whose value is unknown. You can get such a time period of an object if you request its elapsed time before it is initialized. Flag values ​​can be checked for each exception:

 kCMTimeFlags_PositiveInfinity kCMTimeFlags_NegativeInfinity kCMTimeFlags_Indefinite 

Since the bitwise OR (in the original, OR'ing) is not used by everyone, there are useful macros to check for the presence of these flags:

 CMTIME_IS_POSITIVE_INFINITY CMTIME_IS_NEGATIVE_INFINITY CMTIME_IS_INDEFINITE 

By the way, they can also be used to check the validity of the time interval. The result of such an operation will be NO if the timestamp is not valid.

In addition, here is a long list of how the comparison of different time categories works:

• Invalid time will be equal to another invalid time, and more than any other time.

• Plus infinity (Positive infinity) will be less than any invalid time, equal to another plus infinity and more than any other time.

• An indefinite time (An indefinite time) will be less than any invalid time, less than plus infinity, equal to the same indefinite time, more than any other time.

• Negative infinity will be the same and less than any other.

Now you know.

Note:
Of course, the topic under consideration is rather narrow, but still it may seem interesting to some. Moreover, the information in runet is slightly less than zero. To obtain the approval of the author, as well as clarify some of the points had to contact him by mail, and eventually got what happened. Please do not write comments and errors in the comments, but send to internal mail, thank you.