The Landy Lab

Research

A recent video Introduction to the Lab that surveys a few recent projects (circa Spring, 2021) is available here.

Research Areas

How does vision determine the size, shape and boundaries of objects in
our environment? Research in my laboratory centers on various aspects of visual perception and the visual control of action. Recent research on the visual system has grown into an exciting collaboration among psychologists, physiologists, computer scientists, and mathematicians. My research continues to blur the lines between these fields in two ways. First, traditional psychophysical methods are enhanced using advanced computer graphics and image processing techniques for stimulus generation and analysis. Second, both mathematical methods and computer simulations are used to model the psychophysical results. As much as possible, the simulation models attempt to reflect a feasible physiological implementation, as I have a strong interest in neural network models of vision. Next, I describe each broad area of my research in turn.

This is an animation of our “overt-criterion” task. Observers discriminate between two noisy categories of ellipses differing in their mean orientation. On each trial, observers rotate a line to indicate their decision criterion, and are scored on a subsequently presented ellipse based on whether it fell on the correct side (clockwise or counterclockwise) of the line’s orientation based on the ellipse’s category membership.

Sensory decision-making. I study how observers make perceptual decisions under uncertainty. Sensory signals are noisy and an ideal observer will combine such signals with knowledge of their uncertainty, prior expectations and knowledge of potential outcome- and decision-contingent rewards to guide decisions. We ask whether humans act as ideal decision-makers and, if not, where are compromises made or heuristics used. We have shown that orientation estimation appears to be consistent with the ideal-observer model and that humans use a prior distribution of orientations that matches environmental statistics. We examine sequential effects as the observer changes her decision criterion from trial-to-trial to track changing perceptual categories or prior probabilities. We also examine how payoffs and priors affect both the (type I) judgment and a subsequent confidence (type II) response. We have developed a new model of how sensory evidence is accumulated over time, which has implications for modeling reaction-time and cued-response tasks.

Sample publications:

This is a frame from a stimulus movie of a pair of rotating cylinders. The images include the depth cues of binocular disparity (stereo), motion, texture perspective and occluding contour.
This is a frame from a stimulus movie of a pair of rotating cylinders. The images include the depth cues of binocular disparity (stereo), motion, texture perspective and occluding contour.

Sensory cue integration. Perhaps I am best known for my work on the integration of multiple sources of sensory information. This work began in collaboration with my NYU colleague, Larry Maloney. We developed an ideal-observer model of cue integration as well as a psychophysical approach to estimating cue reliability and the cue-combination rule used by observers. We often find that cues are combined linearly with near-optimal weights that respond to changes in cue reliability on a trial-by-trial basis. We have found cue integration to be mandatory in some circumstances, so that observers appear not to have access to information from individual cues, even when it would improve performance.

See my interview for NPR’s Science Friday, October, 2012, on this topic.

Sample publications:

Texture-defined edge based on a difference in texture element orientation.
Texture-defined edge based on a difference in texture element orientation.

Spatial vision and texture perception. Sometimes when one texture pattern is placed on a background of another, the two segregate quickly and seemingly effortlessly into foreground and background. Other times not. Why is this? What sequence of linear and nonlinear image transformations leads to this variation in texture segregation performance? Our research in this area consists of both psychophysical experiments and computational modeling to determine the details of the visual machinery used to code, interpret and segregate texture patterns. We have also looked at the identification of shapes defined by texture (e.g., letters) and the estimation of texture properties (e.g., surface roughness) in 3-d rendered scenes. We have examined the cortical coding of 2nd-order patterns by looking for orientation-selective adaptation of cortical responses to 1st- and 2nd-order patterns using functional magnetic resonance imaging (fMRI) in collaboration with David Heeger (NYU). We have also developed and explored a new model of cortical pattern adaptation, which has implications for computational theory, physiology and perception.

Sample publications:

This is one of our subjects (Julia Trommershäuser) set up for the collection of movement data while performing a pointing task. Infra-red emitting diodes are strapped to her finger and arm. Their motion path is tracked in 3D by the three infrared-sensitive cameras of an Optotrak system, visible in the background.
This is one of our subjects (Julia Trommershäuser) set up for the collection of movement data while performing a pointing task. Infra-red emitting diodes are strapped to her finger and arm. Their motion path is tracked in 3D by the three infrared-sensitive cameras of an Optotrak system, visible in the background.

Visually guided action. We have also applied statistical decision theory to modeling visuo-motor control. This began as an extension of my collaboration with Larry Maloney. We were concerned with the notion of perception as optimal, when optimality itself depends on the choice of cost function. We wondered whether behavior would remain optimal when the cost function was known, i.e., was imposed by the experimenter. In this research, subjects perform pointing or other tasks under tight time constraints. Subjects earn points (and eventually, money) for fast, accurate performance of the task (pointing at a target region), but lose points if they respond late or point towards penalty regions. By measuring outcome uncertainty (the variance in motor outcome), we can compute the optimal aim point for any configuration of payoff and penalty regions and values. In a variety of situations, subjects are optimal or near-optimal in this task. That is, they earn as many points as would have been earned by an ideal movement planner having the same movement variability as the subject. Subjects appear to have available an estimate of their movement variability and take it into account in movement planning, even in situations in which that variability has been increased (artificially) by the experimenter. More recently we have studied movement planning for reaches and saccadic eye movement using both learning and adaptation experiments to study the coordinate systems in which movements are planned.

Animation of a 3-d shape rocking back and forth.
This is an animation of a 3-d shape rocking back and forth, thus cued by relative motion. The dots that carry the motion flicker occasionally so as to eliminate the possible cue of changing local dot density. Despite the elimination of that cue and the flicker, it is relatively easy to perceive (and to judge) the 3-d shape.

Depth perception. I am interested in the details of how the visual
system determines depth and object shape using a variety of visual cues. I have done psychophysics coupled with modeling on several individual depth cues (as well as their combination, discussed below), including structure from motion, binocular stereopsis and pictorial depth cues (texture, contour). The work on binocular stereopsis looks at the entire process, from the low-level computation of binocular disparity through the scaling of disparities to estimate absolute depth and interpolation of local measurements to form perceptual contours and surfaces. Studies on the pictorial cues center on the prior constraints (i.e., Bayesian priors) used to resolve perceptual ambiguities.

Sample publications: