background
The following sequence of pages (and dynamic sketches) describe a problem we've been investigating in the Banks Lab and present some data from a report in Vision Research (Backus, BT, Banks, MS, van Ee, R, & Crowell, JA (1999) Horizontal and vertical disparity, eye position, and stereoscopic slant perception). We've been investigating visual space perception which is the problem of how we recover the location, size, shape, and orientation of objects in the environment from the pattern of light reaching the eyes. The visual system uses small differences in the two eyes' retinal images to glean information about the 3-dimensional layout of the environment; this is called stereoscopic vision. Stereoscopic vision is the basis of the Magic Eye Stereograms and many other 3-d visual displays. In the following sequence, we will examine how stereoscopic information is used to recover the orientation of a flat surface. Many of the pages contain "sketches" created using the Geometer's Sketchpad, a product of Key Curriculum Press. When you load the first page, Applets required to run Sketchpad on your machine must be loaded. Be patient; this may take a few minutes. After the Applet is loaded, each subsequent page will load quickly.
The left side of the sketch is a plan view (top view) of a binocular observer fixating a flat surface straight ahead. You can vary the surface's slant by dragging the small dot on its right side. The right side of the sketch is a stereogram that is linked dynamically to the plan view. To view the stereogram, position yourself ~30 cm from the display. Cross your eyes such that the left eye is directed toward the small dot in the middle of the concentric boxes on the right and the right eye is directed toward the small dot in the middle of the boxes on the left. When you've done it right, you'll see three sets of concentric boxes; we're interested in the middle one. (If you can't cross fuse, but can divergently fuse, it will work, but the depth will be reversed.) When you think you've got it, manipulate the surface's slant by dragging the dot labeled "Vary Slant". You will see a compelling sensation of 3-dimensionality as you vary the slant. The sensation is created by a variety of cues that we examine in the following sequence.
Horizontal Disparity
An important signal for estimating the slant and curvature of a surface is the horizontal differences between the two eyes' images. These are Horizontal Disparities. Such disparities are the primary cue to perceived depth in stereograms such as the Magic Eye Stereograms. In the following sequence of sketches, we consider planar surfaces that have been rotated about a vertical axis. For such surfaces, the horizontal disparity pattern can be represented locally as a horizontal size ratio (HSR; Rogers & Bradshaw, 1993), the ratio of horizontal angles subtended in the left and right eyes; those angles are respectively bL and bR in the sketch below. HSR is bL/bR. The sketch is a plan view of a binocular observer fixating a point (Fixation Point) in the middle of a small plane. The thin line represents the Objective Gaze Normal Plane, a hypothetical plane perpendicular to the line of sight (between the Fixation Point and the midpoint between the eyes). The viewed surface is represented by the thick line. Its slant is the angle between it and the Objective Gaze Normal Plane; you can vary the slant by moving the small dot labeled Surface Slant. Notice that HSR varies systematically as you change the slant; for example, it increases in value as the surface is rotated clockwise.
eye position affects horizontal disparities
Here we demonstrate how changes in the position of an object relative to the head alter horizontal disparities. Position yourself less than 30 cm in front of the screen, forehead parallel to the screen. Cross your eyes such that the left eye is directed toward the cross above the green half-square and the right eye toward the cross below the red square. (This is cross-fusing.) You should see three crosses and we're interested in the middle one and particularly the red and green half-squares above and below it. Notice that the widths of the green and red half-squares are similar. Now slowly rotate the head leftward (keeping distance and left-right position constant), so that you have to turn the eyes rightward to cross-fuse the cross. As you rotate leftward, you'll notice that the red square begins to look wider than the green one. This occurs because the head rotation brings the right eye closer to the screen and the left eye farther from the screen. Thus, the object seen by the right eye creates a larger retinal image and the object seen by the left eye a smaller image. This creates a horizontal disparity, which we express in terms of the horizontal size ratio (HSR); with leftward head rotation, HSR decreases. Rotate your head rightward and you'll see the opposite; HSR increases. The change in HSR occurred because you rotated the head, not because the object changed.
horizontal disparity is ambiguous
Although changes in HSR produce obvious and immediate changes in perceived slant, horizontal disparity by itself is an ambiguous indicator of surface slant. The sketch below demonstrates the ambiguity. Two surfaces, A and B, are shown in plan view; A is the one with the dot labeled "Fixation Point" and B is the one with the dot labeled "Vary Distance". You can vary the position of Surface A by dragging the Fixation Point and vary its slant by dragging the point labeled Vary Slant of A. You can vary the azimuth and distance of Surface B by dragging the labeled points. In the sketch, the angles bL and bR are created by Surface A and we find a solution for Surface B that preserves those angles. If bL and bR are equal for the two surfaces, then HSR must be equal for the two as well. Notice that for nearly every position and slant of Surface A, there is a solution for Surface B. Indeed, it is easy to show that given a pattern of horizontal disparities, there is an infinite numbers of surfaces that could have given rise to them. (Precisely the same ambiguity exists for other measures of horizontal disparity). The point of the sketch is that HSR (or other measures of horizontal disparity) cannot by itself specify surface slant.
binocular eye position
Given that horizontal disparity alone is an ambiguous indicator of slant, how might the visual system estimate surface slant? One means is illustrated in the next two sketches. Consider eye movements in the Visual Plane, the plane containing the two eyes and the Fixation Point. Ignoring torsion (rotation of an eye about the line of sight), each eye has one degree of freedom in the Visual Plane. We can, therefore, represent eye position by two numbers, Version and Vergence. The rotations of the left and right eyes are given by the angles aL and aR in the sketch. Version is the average of the two rotations and Vergence is the difference. The circles in the sketch represent positions in the Visual Plane where Vergence is constant; they are Vieth-Muller Circles. Double click on the button labeled Hide and then on the button labeled Isoversion. The lines that appear are positions where Version is constant; they are called Hyperbolae of Hillebrand (yes, they're hyperbolae, but well-approximated by lines).
using HSR and eye-position signals to estimate slant
If the visual system knew the eyes' Vergence and Version (by referring to eye-muscle signals), the position of the Fixation Point could be estimated. The surface slant could then be estimated by using eye-position signals and HSR. In the sketch below, you can vary the position of the surface by dragging the Fixation Point about and you can vary HSR by dragging the point labeled Vary HSR up and down SLIGHTLY. Notice that there is a unique solution - a particular surface slant - for each position and HSR. Notice too that HSR specifies different surface slants depending on azimuth and distance.
Although the eyes are horizontally displaced, vertical differences in the two eyes' images are created whenever the stimulus is closer to one eye. These vertical disparities could in principle be used along with horizontal disparities to estimate surface slant. Like the horizontal counterpart, vertical disparity can be quantified as vertical size ratio (VSR), the vertical angle a surface patch subtends in the left eye divided by the vertical angle in the right eye. The sketch below illustrates vertical disparities for different viewing situations. On the left is a plan view. You can move the position of the surface and fixation point by dragging the midpoint; you can alter the surface's slant by dragging the end point. One the right is what the stimulus would look like if its texture were three vertical lines. The green and red lines represent projections of the lines to the left and right eyes, respectively. The VSR of the central line is presented below. VSR is >1 for all stimulus positions to the left of the head's median plane because all such stimuli are closer to the left eye. VSR <1 for all positions to the right of the head's median plane.
eye position affects vertical disparities
Here we demonstrate how changes in the position of an object relative to the head alter vertical disparities. Position yourself less than 30 cm in front of the screen, forehead parallel to the screen. Cross your eyes such that the left eye is directed toward the cross left of the green half-square and the right eye toward the cross right of the red square. You should see three crosses and we're interested in the middle one and particularly the red and green half-squares left and right of it. Notice that the heights of the green and red half-squares are similar. Now slowly rotate the head leftward so that you have to turn the eyes rightward to cross-fuse the cross. As you rotate leftward, the red square begins to look taller than the green one because the head rotation brings the right eye closer to the screen and the left eye farther from the screen. Thus, the object seen by the right eye creates a larger retinal image and the object seen by the left eye a smaller image. This creates a vertical disparity, which we express in terms of the vertical size ratio (VSR); with leftward head rotation, VSR decreases. Rotate your head rightward and you'll see the opposite; VSR increases. VSR is determined by object position relative to the head.
VSR is determined by position relative to the head and not by local surface slant. Thus, VSR provides another critical piece of information from which slant can be determined (along with HSR). The sketch below shows a series if contours for which VSR is constant; they are isoVSR contours. Each contour is labeled with its VSR value.
haploscope
To examine how the visual system uses horizontal and vertical disparities and eye-position signals to estimate surface slant, we need to be able to manipulate those signals independently. We did so by using a haploscope. The sketch below is a plan view of this instrument. The observer's eyes are placed directly above the two pivot points. The CRTs and mirrors are rigidly attached to armatures that pivot about those points. Thus, the left eye views the left CRT in the left mirror and so forth. The virtual image of the Fixation Point is shown in the sketch. You can simulated conditions of the experiment by dragging the fixation point left and right or up and down. When you drag left and right, you're simulating a version eye movement; up and down simulates a vergence eye movement. Now consider given images on the left and right CRTs. Note that the retinal images created always remain the same for all positions of the two armatures. Thus, we can dissociate changes in retinal images from changes in eye position.
natural viewing experiment
We first wanted to determine whether people can estimate slant accurately when the only reliable cues are horizontal and vertical disparity and eye-muscle signals. We presented 35x35 deg planes of small dots in random positions in otherwise total darkness. Monocular cues to slant were minimized using a technique developed in our lab (Banks & Backus, 1998). Dot positions on the two CRTs specified planes of different slants. Observers fixated a central dot. Observers adjusted the plane's slant until it appeared perpendicular to the line of sight (represented by the thin black line). In this natural-viewing experiment, disparities presented to the eyes were geometrically correct for the simulated viewing situation and were consistent with the positions of the eyes. The sketch illustrates the experimental conditions. Distance to the stimulus was 57 cm. Azimuth was -15, -7.5, 0, 7.5, or 15 deg (vary azimuth by dragging the appropriate point). Slant was adjusted (about a vertical axis) until the plane appeared perpendicular to the line of sight or "gaze normal". Adjust slant by dragging the appropriate point. Gaze normal planes have different HSRs depending on azimuth, so observers cannot perform this task veridically by setting HSR to one value for all conditions. The results showed that people could perform the task veridically (Backus, et al, 1999, Figure 10). The standard deviations of settings were 1-2 deg and bias was typically less than 2 deg. This shows that people can estimate slants veridically using the signals identified above, but it doesn't show whether all of them are used.
conflict experiment
We next examined what happens when the position of the stimulus plane specified by eye-position signals differs from position specified by vertical disparities. The sketch below schematizes the experiment. The arms of the haploscope were positioned such that the observers looked left, right, or straight ahead; this direction is shown in the sketch with the plane labeled "Stimulus (re real eye azimuth)". The vertical disparities in the stimulus specified another direction which is represented by the plane labeled "Stimulus (re VSR azimuth)". The real and VSR azimuths were -15, -7.5, 0, 7.5, and 15 deg, but they were manipulated independently. The experimental task was again to adjust the slant of the stimulus plane until it appeared gaze normal. If the visual system estimates slant from HSR and eye position, observers should set slant according to that means of slant estimation; this is represented by the quantities in the bottom right of the sketch. If the system estimates slant from HSR and VSR, then observers should set slant according to the means represented by the quantities in the bottom left. If they use both methods equally weighted, then they should adopt a compromise strategy. You can simulate doing the experiment by altering the positions of the real-eye-azimuth stimulus and the VSR-azimuth stimulus and then by varying the slant of the former. The text in the lower left and right corners present values relative to the VSR-specified and version-specified azimuths, respectively. When the azimuths of the two stimuli are quite different, setting the slant to zero relative to real-eye azimuth does not lead to zero slant relative to VSR azimuth. The quantitative predictions and results are depicted in the next page.
results of conflict experiment
The figure below shows predictions and results for the conflict experiment. The natural logarithm of HSR is plotted as a function of the eyes' version (leftward gaze on the left). The predictions are represented by the dashed lines. The horizontal yellow, pink, and blue lines are predictions for slant estimation by HSR and VSR. Yellow is the predictions when VSR in the stimulus was consistent with a version of 15 deg (leftward gaze). Pink is the predictions when VSR was consistent with version = 0 deg (forward gaze) and blue is the predictions when VSR was consistent with version = -15 deg (rightward gaze). If settings were based only on HSR and VSR, observers should set the slant of the stimulus such that lnHSR would vary with azimuth specified by VSR (specifically, such that lnHSR = lnVSR). Moreover, there should be no effect of the eyes' actual version, so slopes should be zero. The dashed gray line is prediction for slant estimation by HSR and eye position. If settings were based only on that means of estimation, observers should set slant such that lnHSR would not vary with VSR and such that it would vary with the eyes' version. The results for one observer are also shown (the symbols and solid lines). Observers' actual settings were quite consistent with the predictions of slant estimation by HSR and VSR. Even when actual eye position was very different from azimuth specified by VSR, observers based slant judgments on HSR and VSR. Thus, with large textured surfaces, vertical disparities are used to correct changes in horizontal disparities due to eccentric gaze. There is a slight downward trend in the data that suggests a small effect of eye position, but the dominant effect is that of vertical disparity variation. We next examined the eye-position signal more closely.
stimulus height experiment
In the conflict experiment, we found a small effect of eye position (version) on the horizontal disparities that are perceived as gaze normal. To look further for eye-position effects, we conducted an experiment in which stimulus height was varied. The magnitude of vertical disparity created by a point in the stimulus is dependent on the point's position relative to the head and on the point's elevation above the visual plane; vertical disparity magnitude is roughly proportional to elevation about the visual plane. We used this property to render vertical disparities more difficult to measure. Specifically, stimulus height was 30, 6.5, 1.3, or 0 deg. The experiment was otherwise the same as the conflict experiment described earlier. The results for one observer are shown below. As before, observers' settings were quite consistent with predictions of slant estimation by HSR and VSR when stimulus height was 30 deg. At shorter heights, however, settings became more consistent with slant estimation by HSR and eye position. Thus, the visual system uses felt eye position to correct for changes in horizontal disparities due to eccentric gaze; it does so when vertical disparities are difficult to measure.
