Université d'Auvergne Clermont1 | CNRS


Tutorial on Computer Vision in a Non-Rigid World at SCIA'11

Tutorial on Computer Vision in a Non-Rigid World

SCIA'11, Ystad, Sweden, may 23, 2011 -- 1:30pm - 5:30pm


This half-day tutorial is the third edition in a series of tutorials on the theme of deformable 3D reconstruction in computer vision. The recent approaches to Non-Rigid Structure-from-Motion and template-based reconstruction form a basis for the contents that also address the key problem of deformable image registration.


The tutorial presents modern computer vision techniques to deal with the problem of 3D reconstruction of deformable shapes from images. The aim of this course is to present a principled procedure for dealing with images where each oject can arbitrarily change its shape – a common occurrence in the human motion analysis, medical imaging and video-surveillance scenarios. The tutorial will discuss a set of techniques general in their formulation which can then be customized given the user’s specific imaging problem. A particular emphasis will be made over the use of physical or statistical priors to aid the solution of such an ill-posed problem. Real examples in human motion analysis and surfaces such as paper and cloth will show the effectiveness of the approaches in dealing with different deforming shapes.

Our slides in pdf format can be downloaded for each part from the syllabus.


  • Introduction
    • Computer vision
    • 3D computer vision
    • Depth ambiguity, prior knowledge
    • SfX, SfM, rigid and deformable SfM
  • Deformable Image Registration
    • Inter-image warp, piecewise continuity
    • Linear Basis Expansion warps
    • Difficulties in registration
    • Cost functional: data + regularization terms
    • Regularization term: warp unsmoothness
    • Data term: feature-based
    • Warp from keypoint matches
    • Robust estimation
    • Self-occlusions: the oversmoothing approach
    • Data term: pixel-based
    • Brightness Constancy Assumption
    • Photometric transformation, light-invariance
    • Robust estimation
    • Self-occlusions: the shrinker approach
  • Non-Rigid Structure-from-Motion (NRSfM)
    • Problem formulation
    • Use of priors: physical/statistical priors or template
    • Low-rank shape model: linear combination of basis shapes
    • Matrix formulation
    • Original factorization formulation
    • Two stage approaches: rank constraint + metric upgrade.
    • Closed-form solutions
    • NRSfM as a single optimization
    • Classification of current approaches (piecewise, trajectory space, manifold learning, ...)
  • Alternation approaches to NRSfM
    • Metric Projections algorithm: implicit metric upgrade
    • Missing data
    • Bilinear factorization via Augmented Lagrange Multipliers
  • Low-Rank NRSfM: PPCA and C2F
    • PPCA (Probabilistic PCA): principle and algorithm
    • C2F (Coarse-to-Fine): principle, algorithm and stopping criterion     
  • Alternative models
    • Piecewise approaches: planar, quadratic, locally-rigid
    • Quadratic deformation model
    • Energy based multiple model fitting
    • Trajectory Space representation
  • Template-Based Deformable Shape-from-Motion
    • Surface continuity is not enough: physical shape priors
    • Template-based developable reconstruction
    • Maximum depth heuristic and SOCP solution
    • Analytical non-convex developable solution
    • Application to laparoscopy
    • Non-developable and non-isometric surfaces
    • 3D template reconstruction with rigid SfM
    • Isometric versus extensible conformal 3D reconstruction
  • Conclusion
    • Deformable image registration
    • Deformable 3D reconstruction
    • Closure


Lourdes Agapito Queen Mary University of London

Dr Agapito is a Senior Lecturer at the School of Electronic Engineering and Computer Science of Queen Mary, University of London where she leads a research group of seven PhD and Postdoctoral researchers. Previously, she had been an EU Marie Curie Postdoctoral fellow at the Robotics Research Group of the University of Oxford. Since 2004 her research has focused in the area of 3D reconstruction of non-rigid structure from image sequences. In 2008 she was awarded a prestigious ERC Starting Independent Researcher Grant to work on the recovery of detailed 3D models of deformable and articulated objects purely from image sequences focusing on the exciting scenario of human motion analysis.

Adrien Bartoli Université d'Auvergne, Clermont-Ferrand

Prof. Bartoli runs the ALCOV (Advanced Laparoscopy and Computer Vision) research team at ISIT with Prof. Canis since november 2009. Previously, he was a permanent CNRS research scientist at LASMEA since October 2004 and a visiting professor at DIKU Copenhagen in 2006-2009. He obtained an Habilitation Degree in 2008. He was a postdoctoral fellow at the University of Oxford in 2004. He did his PhD at INRIA and received the 2004 Grenoble INP PhD thesis prize. He received the best paper award at CORESA’07 and the 2008 CNRS Bronze Medal. He led the ComSee research team (with Dr Chateau) between 2006 and 2009.

Alessio Del Bue Istituto Italiano di Tecnologia, Genoa

Dr Del Bue is a Senior Post-doc researcher at the facility of Computer Imaging of the Italian Institute of Technology (IIT). Previously, he was a researcher in the Institute for Systems and Robotics at the Instituto Superior Tecnico (IST) in Lisbon, Portugal. He obtained his Ph.D. under the supervision of Dr. Agapito in the Department of Computer Science at Queen Mary University of London.