This page is a basic walkthrough of how to use VidSync for 3-D measurement. There are some other special considerations when simply using VidSync for 2-D measurement or when using VidSync as an event logger that records the times and counts of observations without measuring anything spatial.
The best way to begin actually learning VidSync may be the 2-hour video tutorial, a screencast recorded during a live lesson on Vidsync in a fish ecology lab.
The basic idea
Humans have 3-D vision because our two eyes observe the same scene from slightly different perspectives, and our brains combine the information from the two perspectives to estimate the distance to an object. VidSync does the same basic thing, using two video cameras. To make accurate 3-D measurements, we have to 1) synchronize the footage from the two cameras, 2) correct for lens distortion, 3) calibrate the video clips to establish the perspective of each camera, and 4) click on the point we intend to measure on each video clip to establish two lines-of-sight that approximately intersect at the point we're measuring.
VidSync does all this math for you using highly accurate methods, so the process is simple and user-friendly. Familiarizing yourself with the math (described in Chapter 1 of Jason Neuswanger's dissertation) might help you understand the reasoning behind some of the advice in this guide, but it is not mandatory.
Recording footage
It's useful to consider some basic principles of cinematography to assure that your video is as clear as possible, and not too degraded by motion blur or grainness or glare.
Cameras
VidSync can do 2-D analysis with one video camera, or 3-D analysis with a pair of cameras. Most video cameras will work (including popular GoPros), but there are many factors to consider to choose the best ones for your project — see the video cameras and mounts page for details.
Distortion correction
Almost all video cameras, especially when operated underwater through a housing, have some amount of non-linear radial distortion, usually the "barrel" distortion familiar to many as the bulging appearance of a photo from a fisheye lens. Even high-quality rectilinear lenses usually exhibit subtle (not readily apparent to the naked eye) distortion that should be corrected to ensure accurate calculations.
To enable calculation of the chessboard parameters, VidSync users film a chessboard-like pattern large enough to fill the screen and fine enough that many squares of the board are visible both horizontally and vertically. It should be filmed in each camera separately with an approximately straight-on, flat view filling the screen in that camera. VidSync has an automatic algorithm — Automatic plumbline detection — to detect straight lines in this chessboard pattern ("plumblines") and calculate the correction factors required to convert between the distorted image and an undistorted coordinate system in which straight lines in the real world map to straight lines in the calculations.
3-D calibration frame
To give VidSync the information it needs for 3-D measurements, you have to film a grid of dots on two parallel surfaces called the calibration frame. See the Hardware page for details on its construction, which can be done in a variety of ways — most easily with the help of a professional printer and a couple hundred dollars.
Synchronization
Synchronizing playback of multiple clips requires identifying some event on the video brief and distinctive enough that you can identify the same exact moment in all clips (to the nearest video frame). Briefly flashing an LED light easily provides this synchronization point. If you forget to bring an LED in the field, you can also drop a pebble into smooth water and synchronize around the moment the pebble hits the water. If you forget to do an obvious synchronization, it's possible (just less convenient) to find a substitute among the naturally occurring events on video.
Loading and navigating videos in VidSync
Loading
On the "Project" tab in a new VidSync document, use the "Add New Clip" button twice to add a pair of videos recorded at the same time. If the location of a video files changes on disk, the "Relocate Selected Clip" button lets you find it.
VidSync can load and use any video that Quicktime Player will play on the same computer. Third-party applications such as Wondershare Video Converter can batch convert incompatible videos into Quicktime movies usable by VidSync.
The "Master Clip"
One of your videos in each project (by default, the first one you load) is the "master clip." It determines the timecodes for all your measurements, which are recorded relative to time 0:00:00 on the master clip. The other clip is the "non-master clip." It usually doesn't matter which clip is the master, as long as you stick with your choice after you've started taking measurements.
Before you synchronize the videos, each one is individually overlaid with the usual Quicktime Player controls. After they are synchronized, those controls disappear and are replaced by a custom VidSync control bar described below. If you want to access these advanced controls while analyzing a single video, go to the "VidSync Preferences" menu, "Misc. Visual Settings" tab, and check the box for "Show advanced playback controls with only one video clip loaded."
Synchronizing the clips in VidSync
Mouse over each video and drag the slider in its individual controls to find the point just before you flashed an LED light in front of the cameras. Then hit "Play" and run the video until the light flashes on and off, and pause it. Then, back up to the first frame in which the light turned off, using either the frame stepping buttons, the left and right arrow keys, or the mouse wheel to step frame-by-frame. Do this for both videos, and when they're both at the same frame, check the "Lock Sync" box above the non-master clip. The process is analogous if you used a different synchronization event, such as a dropped pebble.
When you lock sync, the controls overlaying each video disappear and are replaced by a control window at the top of your screen. Every action in the top control bar navigates both video clips together. The layout is intuitive, but you can hover over each button for a tooltip explaining its function and its shortcut key (if any). Click the tiny, circled "+" button in the top left of the control panel to stretch it all the way across your screen. If you don't like it, you can drag from the right edge to shrink it to your desired size. The Advanced Playback Panel page contains more detailed information, tips, and tricks.
VidSync's custom video controls
In addition to standard playback buttons (play/pause, play backward, fast forward/rewind, step one frame forward/back), VidSync has several customizable playback controls. These include buttons to step forward or backward by a specified number of frames, buttons to play forward or backward at a customizable speed, and two pairs of customizable-speed “play while pressed” buttons. By default, one pair is set for fast playback, and the other for slow motion. The “play while pressed” buttons are particularly useful, because they facilitate sudden pausing and easy, repeated review when an event of interest is observed. Other controls instruct VidSync to play the video for a fixed or random duration and then pause, facilitating systematic or random sampling schemes.
Navigating to measurements and annotations
You can quickly jump to any past data you recorded. For measurements, under the "Measurements" tab, select the relevant object, event, and point, and click the "Go To Point" button above the points list. On the "Annotations" tab, a "Go to Annotation" button is below the list.
Plumblines can be accessed in the "Calibration" tab, "Lens Distortion" sub-tab, by selecting the relevant plumbline and clicking the "Go" button above the list. On the "3-D Calibration Frame Input" sub-tab, the "Go to Calibration Frame" button will jump the video to the moment used for calibration.
Navigation is further enhanced by allowing users to select any previous measurement and press a “go to” button to view the video at the frame where that measurement was taken. The program separately allows unmeasured text annotations with the same “go to” functionality, serving as bookmarks for video frames of interest.
Saving your work
Each VidSync project should contain a single synchronized set of video clips (or a single clip if only using one camera), and VidSync saves all information about this project in a VidSync document (.vsd) file. This file contains all the information about a single synchronized set of video clips, including its calibration, measurements, annotations, and metadata. VidSync rarely crashes, but users should still save early and often to avoid data loss.
VidSync also allows users to export certain data, such as calibration frame specifications and object/event type specifications, and import it into other projects.
Calibration
Distortion correction with a checkerboard
To find the 8 distortion parameters for each video in VidSync, locate a clear, full-screen view of the chessboard pattern and presses a button to automatically detect the corners and arrange them into plumblines. Detected plumblines are overlaid on the video, where the user can click on any erroneous points to edit them. After that, the program calculates the distortion parameters. If no chessboard image was filmed, the user can manually digitize plumblines by clicking other straight objects in the image (such as the edge of the calibration frame), but this time-consuming approach is less accurate than the automatic, chessboard method. Users can import distortion parameters from other VidSync documents that used the same camera system; however, any optical adjustment (such as removing a camera from its underwater housing to change a battery) may affect the parameters and warrant re-calculation.
3-D calibration with the calibration frame
Before calibrating a video clip, the user enters the distance between the calibration frame faces and a list of physical 2-D coordinates of the nodes on each face of the calibration frame. The length units in which these node coordinates are provided become the measurement and output units for VidSync. The user can optionally customize the coordinate axes (x, y, and z) corresponding to each face of the frame, for example, to tell the program if a top-view camera and a side-view camera are looking through different, perpendicular faces of the frame. These frame descriptions can be saved as a separate file and reloaded for any other video filmed with the same calibration frame.
The user begins calibration by finding a synchronized timecode at which all cameras have a clear view of the calibration frame, and the frame and cameras are both as motionless as possible. A button click loads the frame node coordinates into a table matching them with (so far, blank) screen coordinates. Screen coordinates are recorded by clicking on the center of each node. If a node center is unclear due to poor visibility, it can and should be deleted from the list, because visual guesswork increases error. Throughout this process, each point’s matched 3-D coordinates are overlaid on the video, so mistaken correspondences are easy to detect. Once the user completes this process for the front and back faces of the frame in one camera, VidSync calculates the projection matrices, and the calibration of that camera is complete. Each camera is calibrated separately in the same manner. Using two cameras and a 5-by-4 node calibration frame, a typical calibration takes 5 to 10 minutes. Completed calibrations can be saved as separate files and reused for other videos shot with the cameras in the same relative orientation.
Measurements
VidSync organizes all measurements in a two-level hierarchy of "objects" and "events," which belong to various user-defined types. For example, in a study of juvenile Chinook salmon feeding behavior, one object type was "Chinook Salmon," and individual objects of that type referred to individual fish. An event type was "Prey Capture." Objects will usually have several associated events (several Prey Capture events for each Chinook Salmon object), and events also be associated with multiple objects (for example, two Chinook Salmon objects may be involved in one Conflict event). VidSync lets you set the visual appearance of your measurements based on their types, associate multiple objects with one event, move an event to a different object, or split an object into two — see measuring objects and events in VidSync for details.
There can be multiple measurements associated with a single event. For example, Prey Capture events consisted of two 3-D measurements: the positions of the fish's head when it began and when it ended the capture maneuvers.
The "notes" field for each event and object is also a powerful tool. Notes can be added by clicking in the table listing events. These can serve as another level of organization, for example, by using a code in the notes to denote whether a Prey Capture was successful or not.
Tips for working with VidSync Documents
Make use of shortcut keys.
See the Keyboard Shortcuts page for details.
Re-use files when appropriate.
Calibration frame and distortion corrections can be re-used across multiple videos in various situations.
The calibration frame should be re-filmed anytime the optical properties of the cameras or their orientations relative to each other might have changed. For example, changes to focus or zoom can affect the optical properties captured by a calibration. Any deliberate change of the angle between the cameras requires a new calibration, but there can also be small, incidental changes caused by removing and replacing a camera in its housing to change a battery, bumping the system around in a shaky boat, etc. Whether or not a new calibration is required after such events is very situation-specific and trial-and-error, or erring on the side of caution and calibrating frequently, is the best policy.
The distortion correction need only be re-filmed when the optical properties of the system (i.e. focus, zoom, lens, housing port, etc) have changed. It is often possible to go through several recording sessions using the same settings in the field, but again it's best to err on the side of the caution and film the distortion checkerboard frequently.
An example situation in which frequent calibrations are not really necessary is a laboratory study in which the cameras are fixed on sturdy benches and powered/recorded externally, or permanently mounted in very firm frames.
If you do choose to use the same calibration or distortion corrections across multiple recordings in the field, make sure to keep good notes on your camera settings for each recording site. That way you'll know which other videos had the most compatible settings to borrow a calibration.