************************************************************************ Media Pro 4000 Brief April 1st 1999 File: ICM_SMPTE_SYNC_BR040199.TXT Module: ICM4020, IMC4020 Title: CUE & SCENE SYNCHRONIZATION WITH SMPTE CODE USING TCM ************************************************************************ PLAYING MP4000 CUES AGAINST SMPTE CODE MP4KSMPT.DOC, 4/30/98 3/29/99 At the configuration pass, all cues are set to INTERNAL source in the ICM's internal tables, regardless of the defined source. However, cues that are going to be set to a SMPTE source should, in fact, be marked in the source box as being SMPTE. This permits the firmware to gather some additional information about the cue that will be needed for later attempts to maintain cue synchronism [more later]. Cues may be moved among rates, or assigned to SMPTE sources. These changes can only be accomplished from within the affected cue. Any SMPTE source in any TCM in the rack may be specified, rx or tx, by referring to its rack, slot, port (TCM port 0 is rx, TCM port 1 is tx; eventually, we will likely permit connection to TCM port 2, the video sync port on the TCM). Since a cue must belong to SOME rate, illegal requests arriving at the relink_cue function will be converted to the default rate and internal source. RESET @q restores the default rate from the cue definition, and to internal source. STOP @q and PLAY @q don't. Note that there may be some interesting details concerning STOP and PLAY of cues while they are attached to a SMPTE source, and discussions of various scenarios may be needed. SMPTE CUE SYNCHRONIZATION Typical control statements in SMPTE-synchronized cues are as follows. The effects of these statements are described subsequently. To 'attach' a cue to a SMPTE source (takes both these statements): SET @q3 FRAME 01:01:00.00 ; SET @q3 RATE @r0,10,0 TIMEOUT 300 GOTO error ; To reattach the cue to an internal rate (may also be used to switch ordinary cues among rates): SET @q3 RATE 24 ; To control a SMPTE cue AFTER attaching it to a rate source: WAIT 01:03:30.00 ; DELAY 150 ; The two SET statements must be used together to 'connect' the cue to a SMPTE port. The SET RATE TIMEOUT GOTO statement causes the cue to begin to attempt to run in synchronism with the arriving SMPTE time code, and playing of the cue is actually suspended at this statement waiting for the SMPTE code value to satisfy the just-previous SET FRAME. Typically, the cue's author plans the show sequencing (for example, by seeking and playing a video player with the source SMPTE before issuing these SET statements) so that the incoming SMPTE will be less than the specified frame, and the cue will enter a 'SMPTE hold' state. The SET RATE TIMEOUT GOTO statement forces specification of a timeout and an error label (abbreviated forms are not accepted by MP4). If the cue does not achieve synchronization within the timeout specified (always measured at a fixed 30 frames per second), then the cue will be restored to its default rate and control passed to the label specified. Note that if the timeout is specified as 0 (zero), there is NO TIMEOUT and the cue will wait forever for SMPTE code to arrive and satisfy the start condition. A reset from another cue will restore the cue operation in that case. In the least problematical example, the author has succeeded in establishing correct startup conditions and SMPTE begins to arrive at the attached port, and its value reaches the specified frame before the timeout expires. At the frame in which the match occurs, the cue is played from the statement following the SET RATE TIMEOUT GOTO, and all statements are executed (as would be the case in a 'normal' cue) until the end of the cue is reached or a statement is encountered that suspends the cue activity. (The condition for match is that the arriving timecode is equal to or greater than the specifier, but not by more than four frames.) The statements that may 'suspend' the cue (actually, return it to the 'SMPTE hold' state) are WAIT and DELAY. The WAIT and DELAY statements exactly parallel the way an internally paced cue would be controlled. The reason the SET FRAME command must be used in the attachment sequence is because, at that moment, the cue is on an internal clock, and the WAIT has a different effect than when the cue is attached to SMPTE. After the arriving SMPTE code has reached the specified start frame, and the cue is 'playing', the WAIT and the DELAY statements are used to establish new 'SMPTE hold' waits for later frames in the arriving SMPTE. [Note that in 'internal' cues, DELAY pays no attention to the frame number, but it is treated differently in a cue attached to SMPTE.] The WAIT sets a new pickup frame and puts the cue back into the hold state. If a DELAY is used, the new pickup frame is calculated by adding the delay to the current SMPTE frame. By working from hold to hold, then, a cue running on SMPTE code will maintain synchronism with the incoming timecode. The timeout is reset at each successful SMPTE frame, so that if SMPTE fails, the timeout will proceed and the error label will receive control. The term 'successful' just used refers to a monotonic, not-too-jumpy continuation of frames. This is enforced, not in the cue handling, but in the port polling. The cue handler is simply not called if SMPTE is not arriving or is faulty. The ramifications to maintenance of the timeout may or may not be simple. Here is a discussion of a few points. In the initial wait for SMPTE in a cue just switched, the SMPTE is expected to be earlier than the specified frame. If not, error handling (perhaps with a 'catchup' algorithm) is needed. Otherwise, the cue idles, and the timeout runs down to its end, forcing the error exit; or, the arriving SMPTE passes the specified frame and the cue plays. If the cue plays in this case, the timeout is reset to its initial value. During subsequent waits, it is conceivable that the wait to continue will be far in excess of the specified wait, and the purpose of the timeout becomes principally to detect SMPTE failure while waiting. So, a 15-minute hold might be specified; if SMPTE stops arriving after 5 minutes, the error transfer will happen timeout frames later. The method stated two paragraphs earlier will accomplish this, if the cue handler always restores the timeout value each time the cue is processed while in one of these subsequent (not initial) hold states. WORKING TEST CUE: (with TCM in slot 13 transmitting to TCM in slot 10) top: DISPLAY "" ROW 0 COLUMN 0 ; DISPLAY "" ROW 1 COLUMN 0 ; DISPLAY "" ROW 2 COLUMN 0 ; DISPLAY "Starting q003" ROW 0 COLUMN 0 ; DELAY 30 ; DISPLAY "Waiting SMPTE sync" ROW 0 COLUMN 0 ; DELAY 10 ; SEEK @r0,13,1 TO 01:00:55.00 ; SET @q3 FRAME 01:01:00.00 ; SET @q3 RATE @r0,10,0 TIMEOUT 300 GOTO error ; DISPLAY "" ROW 0 COLUMN 0 ; DISPLAY "got SMPTE sync" ROW 0 COLUMN 0 ; WAIT 01:01:20.00 ; DISPLAY "after wait 1" ROW 1 COLUMN 0 ; DELAY 30 ; DISPLAY "after delay" ROW 2 COLUMN 0 ; SET @q3 RATE 30 ; DELAY 300 ; GOTO top ; error: DISPLAY "" ROW 0 COLUMN 0 ; DISPLAY "TIMEOUT" ROW 3 COLUMN 0 ; DELAY 5 ; DISPLAY "" ROW 3 COLUMN 0 ; GOTO top ; Note that the intention is that the SMPTE time code be controlled first, so that its source is reset and restarted at some value earlier than the times at which a cue is expected to lock. PLAYING ANIMATION DATA AGAINST SMPTE CODE MP4KSMPT.DOC, 4/29/98 3/29/99 'Animation scenes' in the ICM are referenced as @A0 through @511 and can be played against the internal clock at various rates, or against SMPTE code arriving at (or being generated by) a TCM. A form of the SET RATE command is used to attach and detach the scene from the SMPTE time code. CONFIGURATION During the configuration pass, scenes are sorted by rate, according to the rate value given in the animation download's data. Scenes may be marked as 'SMPTE_SCENE_SYNC' by the animation program, this is treated by configuration as a separate 'rate' and such scenes have their 'default' rate set to 30 fps. Scenes may be attached to SMPTE regardless of how the animation program set their rate and source. Although cues can be moved among the various legal rates, this is not be possible for scenes, which will operate either at their default rate, or at a SMPTE rate if 'attached' to a TCM port, as described below. Legal scene rates are currently 12, 15, 24, 25, and 30 (there is no video rate for scenes, as there is for cues). GLOSSARIC NOTES The scene data is an animation file containing repeated sets of data values for a channel set. Each set of data is a 'frame' of animation data, and in the scene these are numbered from 0 through the size (in frames) of the show. These frames in the animation data are called here 'physical' frames, to emphasize their distinction from SMPTE time code, in which the individual code packets are often called 'SMPTE frames'. SEEKs are to 'physical' frames, regardless of scene rate. A SMPTE frame is a time hh:mm:ss.ff. The synchronization procedure will produce an offset from the current SMPTE time to a corresponding 'physical' frame in the animation file. As the file is played 'synchronously' to the arriving SMPTE, this offset will be constant, and is determined automatically from what 'physical' frame the scene is at when a PLAY command is issued, and the SMPTE passes the specified SMPTETIME that was set up for the scene. ATTACHMENT TO SMPTE The attachment procedure is: SET @Aaaa RATE @R0,t,r SMPTETIME hh:mm:ss.ff ; [ aaa is the 'scene', 0-511 ; t is the slot of the TCM, r the TCM port (0 = Rx, 1 = Tx)] This command does not immediately change any current operation of the scene, merely 'primes' it for a subsequent PLAY command. If the scene is currently playing against an internal source, or a different time code source, it will continue to play until a STOP, STEP, SEEK, or RESET command is given to it, or it runs to its normal termination (a scene reaching its end STOPs with the final frame of data). Any of these commands remove the scene from any previous SMPTE attachment, but continue to have the expected effect on the animation. i.e.: STOP holds the scene at the current 'physical' frame STEP stops the scene, but advances to the next 'physical' frame and holds SEEK stops the scene, but moves to the specified 'physical' frame and holds RESET stops the scene, but moves to the first frame and holds. A user familiar with the procedure attaching a cue to SMPTE will note that this scene attachment provides no way to specify a timeout while waiting for the SMPTE to reach the SMPTETIME, nor for cessation of SMPTE after synchronization is started. If this check on the SMPTE source is desired, the user will need to lock a cue to the same source, or monitor the status bits of the scene (to be determined). INITIATING THE SCENE AND SMPTE LOCK Once attached, the scene will continue operating as it was (as described above) and subject to ordinary control. To place the pending SET RATE SMPTETIME command into action, the scene must be set to the desired 'physical' frame and a PLAY command issued. SEEK @Aaaa TO physframe ; setup the scene frame for lock point PLAY @Aaaa ; The SEEK command is not absolutely required. Whatever 'frame' the scene is at when the PLAY command is issued will determine the 'offset' within the scene for synchronization against the SMPTE, and the scene will play from that point when the SMPTE lock is established. The scene's physical frame may be set before or after the SET RATE SMPTETIME command. Note that if a RESET command is used after the attachment to a SMPTE source, the attachment will be cancelled. There is a consideration relating to SEEKing the cue to a desired starting point, about whether the sought frame is output immediately, or is not output until the SMPTE sync point is passed. A SEEK, RESET, or STEP command must have at least a single frame delay following it, before the PLAY command is issued, in order to have immediate effect. If the scene is PLAYed with no intervening delay after one of these commands, the new frame will not be distributed until SMPTE is satisfactory. After the PLAY command is issued, the scene is placed in a 'waiting SMPTE match' state, and then, when the approprate SMPTE frame arrives, enters a 'playing SMPTE sync' state without further intervention. Any scene control commands (except another PLAY) that are issued before synchronization is achieved remove the scene from the attempted lock with the SMPTE source. After lock is achieved, any commands except PLAY remove the scene from the SMPTE lock and have their ordinary effect. DETACHMENT FROM SMPTE As described above, any ordinary scene control statement PLAY, STOP, STEP, SEEK, or RESET, will remove the connection to SMPTE and restore the scene to its normal rate, with the commands all having the effect you would expect at the time. To remove the scene from SMPTE and reconnect to its normal rate, use one of the cue control statements listed in the preceding paragraph or use: SET @Aaaa RATE s ; restore default rate, scene continues 'as was' with s=0, typically, since the default rate will not generally be known to the user. In any case, for a scene, the actual specified rate will be ignored. If the scene was waiting for a SMPTE lock, the scene will be left in the STOPPED state. If it was already playing in SMPTE sync, it will be placed into the CUE_PLAYING state and therefore will continue playing against the internal clock at its own default rate. SMPTE VERIFICATION The SET RATE SMPTETIME command that attaches the scene to a SMPTE source does not permit setting a timeout for successful achievement of synchronization. It is of course possible to verify SMPTE with the cue TIMEOUT check, provided the cue need not be doing other things, or if the subversion of the cue (to the GOTO label if SMPTE times out) can be tolerated: [... in cue #20 ...] SET @A0 RATE @R0,2,1 SMPTETIME 01:00:00.00 ; RESET @A0 ; PLAY @A0 ; SET @Q20 FRAME 01:00:00.00 ; SET @Q20 RATE @R0,2,1 TIMEOUT 450 GOTO NOSCENE ; WAITHERE: GOTO WAITHERE ; NOSCENE: DISPLAY "No SMPTE, ICM scene 0" ; PLAYBACK AGAINST SMPTE The 'playing' of the scene is literally connected to the TCM/SMPTE data. The 'play-scene' operation is invoked ONLY when the TCM signals the ICM that data has changed in the port, and this happens at the rate that SMPTE arrives. This makes the SMPTE-locked scenes asynchronous with respect to the data distribution and update operations of the net and local ICM, and so it is possible that not EVERY frame of animation data will be transmitted, and that a single frame may be transmitted more than once. The granularity is 30Hz, which means that some of the artifacts so induced may be noticeable to the eye if watching LEDs on an IOM, but not likely noticeable if observing effects in the show set pieces and animated figures. If, when the PLAY command is issued, the arriving SMPTE frame is already past the specified lock frame by more than four frames, the scene will not start. After the PLAY command is issued, the scene handler will wait until the arriving SMPTE frame matches (or is not more than four ahead of) the specified SMPTETIME. At this time, the scene will be taken out of the 'wait for SMPTE' state, and placed in the 'playing SMPTE' state. The 'physical' frame will be adjusted (if necessary), and the data will be output (i.e., moved to the output block for distribution to net and modules). Once started, a cessation of SMPTE data will cause the scene to stop playing for as long as satisfactory new frames are not available. If a new frame is less than the previous frame, the scene will not update. If a new frame is more than the previous frame, but not more than four frames distant, the scene will be instantly positioned to the corresponding physical frame, the data will be output. If the new frame is more than the previous frame, but more than four frames distant, the scene will advance four frames, be instantly positioned to the corresponding physical frame, the data will be output. This 'chase' will continue until the scene is recovered to synchronism with the arriving SMPTE time code. TIME CODE RATE CONSIDERATIONS & ERRORS SMPTE CODE IS MEANT TO BE CONTROLLED BEFORE LOCK IS ATTEMPTED: Note that the intention is that the SMPTE time code be controlled first, so that its source is reset and restarted at some value earlier than the times at which a scene is expected to lock. SMPTE RATE IS MULTIPLE OF RECORDING RATE (1x or 2x) This is the appropriate assignment, and other combinations described below represent errors in configuration by the show programmer. There is an animation frame for every SMPTE frame at the 24, 25, and 30 frame rates; or for every second SMPTE frame at the 12 and 15 frame rates. SMPTE RATE IS DIFFERENT THAN RECORDING RATE (NOT 1x NOR 2x) The TCM converts the arriving SMPTE frame information into a 32-bit integer frame number based on the rate that it determines for the SMPTE channel. Thus, the ICM can only assume that the rate matches correctly. Hence, a scene recorded at 30 frames, but played against a 24 frame SMPTE will play slow. ------------------------------------------------------------------------ Specifications subject to change at any time. The data in this document incorporates proprietary rights of Anitech Systems Inc., 25021 Anza Drive, Valencia, CA 91355 USA. Phone (661) 257-2184 Fax (661) 257-2025 email Mail@Anitech-Systems.com Web http://www.Anitech-Systems.com Any party accepting this document does so in confidence and agrees that it shall not be duplicated in whold or in part, without the consent of ANITECH SYSTEMS.