Mathematical methods in VidSync

The novel blend of mathematical methods used by VidSync enables excellent precision and accuracy as well as a more flexible range of applications than any other published method of fisheries videogrammetry. VidSync performs all the calculations for 3-D measurement automatically, and one can use it proficiently without understanding the details. However, basic familiarity with the mathematics helps to understand the reasoning behind our hardware and software guidelines, and to better interpret program output.

The mathematics were initially described and evaluated for accuracy in the author’s Ph.D. dissertation:

Neuswanger J. 2014. New 3-D video methods reveal novel territorial drift-feeding behaviors that help explain environmental correlates of Chena River Chinook salmon productivity. Ph.D. Dissertation, University of Alaska Fairbanks, 181 pp.

This chapter, with some improvements demonstrating additional applications of VidSync (but no changes to the underlying math), was published in 2016 in the Canadian Journal of Fisheries and Aquatic Sciences.

Neuswanger, J. R., Wipfli, M. S., Rosenberger, A. E., & Hughes, N. F. (2016). Measuring fish and their physical habitats: versatile 2D and 3D video techniques with user-friendly software. Canadian Journal of Fisheries and Aquatic Sciences, 73(12), 1861-1873.

If you don’t have official access to CJFAS, you can see the fully reviewed and accepted, pre-print version of this paper on the author’s website.

A brief summary

Each 3-D position is calculated from points on two or more video screens, which the user digitizes by clicking on the same object (e.g., a fish’s snout) in each view. The input point is adjusted to compensate for non-linear (radial and decentering) distortion caused by optical imperfections in the lens and housing system. Then, a linear (matrix) method projects the distortion-corrected 2-D coordinates of each point into a line of sight in 3-D space. When at least two lines of sight have been obtained from different camera views, their approximate intersection is triangulated to find the 3-D world coordinates (e.g., in meters) of the measured point. Length measurements can then be obtained by computing the distance between the head and tail points in 3-D space, timecodes can be incorporated to compute velocities, and so on.