User:RichW/Research proposal

Tasks

Pending

1111

• Joan to write HOD statement (doing this now)
• JL changed title, but not sure if it should be changed back :)
• Lay report (first draft -- JL made some changes 22/10/06)
• Personal statment (drafted -- JL made some changes 22/10/06)
• Technical report (now on second draft: remember to add some references if we think we need them)
• Send draft to Duncan for passing through the 'lets not make this suck' process (depends on preceeding tasks)

Done

• Rich to send CV to Duncan.
• Rich to fill in FEC part of eGAP form.

Executive summary

Title

Using computer graphics and computer vision for real-time performance feedback.

20 word-ish summary

Pull together and innovate upon existing expertise, the SESAME project, and DTI-supported projects toward an integrated system for increasing performance quality and rapidity of feedback for athletic and medical rehabilitation applications.

Publications

Journals

Computer Graphics using Conformal Geometric Algebra, Joan Lasenby and Anthony Lasenby, To appear in Phil. Trans. Roy. Soc. A special issue. http://www-sigproc.eng.cam.ac.uk/ga/static/c/c7/Rjw_roysoc.pdf

Peer reviewed symposia contributions

Applications of Conformal Geometric Algebra in Computer Vision and Graphics, Rich Wareham, Jonathan Cameron and Joan Lasenby, Lecture Notes in Computer Science - Computer Algebra and Geometric Algebra with Applications. http://www.springerlink.com/content/k60hg6pryarvxwh2/

Personal statement (3500 chars)

My long term career aspiration is to push the envelope of computer graphics research and, crucially, find compelling applications which can help real people. Toward this goal I wish to investigate and extend work in the related fields of motion capture, computer vision and data visualisation. I view computer vision as a discipline tightly coupled to computer graphics in that one is, in a sense, the inverse of the other. In striving to attain a degree of excellence in both fields I aim to develop a fundamental understanding of the nature of human vision and how useful information may both be inferred from, and presented to, a lay-person.

Since October 2005 I have been employed by Geomerics Ltd. performing postdoctoral research as a Senior R&D Engineer. At Geomerics I invented and developed a revolutionary new computer graphics lighting algorithm which allows a radiosity solution, the shading of objects both from direct illumination and light bouncing around the environment, to be estimated around a hundred times per second. The research now forms much of the core 'intellectual property' of the company and has generated significant excitement in the lighting community, press interest and investment opportunities.

During my time with the company I have also developed novel research into techniques for sampling using the latent power of modern graphics cards. I created, in addition to the radiosity, a significant innovation on 'Precomputed Radiance Transfer', another popular lighting algorithm. I advanced, to a proof-of-concept stage, some new methods in electromagnetic modelling arising from work performed by academics involved with the company. Examples of the work, including videos, are available from http://www.geomerics.com/index.php?page=lighting and http://www.geomerics.com/index.php?page=emm.

My position within Geomerics has given me an unrivaled grounding in the needs of industry and how they mesh with those of academia. In seeking to return to academia I hope to bring experience gained from Geomerics back with me in order to better focus the application and target audience of my work.

Prior to my position at Geomerics I was working towards my PhD as part of the Signal Processing and Communications Group (http://www-sigproc.eng.cam.ac.uk/ga/) in the Department of Engineering, University of Cambridge. As part of my PhD I investigated how a potentially revolutionary new interpretation of Clifford algebra, termed Geometric Algebra (GA), may help the animation industry with particular relevance to pose interpolation. My most significant advance was finding a natural linear parametisation of rotations and translations allowing any exisitng linear point-based interpolation scheme to be extended naturally to pose and position in arbitrary dimension spaces and multiple geometries. In addition to this I investigated how GA may be extended to handle non-Euclidean geometries and along the way generalised the popular escape time Mandelbrot and Julia fractals to hyperbolic geometries.

Lay proposal (3500 chars)

We propose a project designed to pull together and build upon existing research and expertise to provide a compelling body of work targetted at enhancing the performance of UK athletes and providing useful tools for medical rehabilitation using computer vision, motion capture and computer graphics techniques.

The fundamental mechanism whereby humans learn, improve and progress is feedback; we observe the results of our actions, identify errors and then attempt to correct them. The rate, and level, of improvement is strongly related to the delay between analysing the performance and presenting appropriate, high quality, feedback. We hope that our proposal will lead to improvements in both athletic and medical feedback through advanced computer graphics and computer vision techniques.

As an example of what we hope to achieve, we consider the training of promising athletes with a view to increasing the overall standard of UK competitors in the 2012 Olympics. In this case the feedback loop consists of three stages: measurement, analysis and presentation.

In the measurement phase the performance of an athelete is measured using a sensor array. For example the SESAME project, which we hope to work with, is tasked with measuring performance via wireless sensors. Another useful measurement process is 'motion capture' where video images of the athlete are analysed by a computer and limb positions are calculated.

In the analysis phase the captured performance is processed to form useful data such as joint orientations and stresses. The final phase which closes the feedback loop is the presentation of this data to the athlete in an easily digested manner. For example a graphical representation of the athelete with a 'false colour' skin showing the underlying joint stresses would allow the athelete to rapidly visualise aspects of the performance which waste effort and hence need changing.

We hope to provide novel research contributions in fields related to all three phases. For the measurement phase we can build upon some work done within the institution as part of the SESAME project which will provide a set of sensor data for a particular athelete. In addition we hope to develop new algorithms and techniques for speeding up the motion capture process.

For the analysis phase we can make use of some recent advances in a sophisticated modelling technique known as 'Geometric Algebra' which will generate medically useful data about joint poisitions, strains and motion from the raw motion capture data.

We expect the greatest scope for novel research to be in the final presentation stage where expertise in real time computer graphics can be used to provide visual feedback for the target audience rapidly after performance capture. For example, a deliverable might be the ability for the subject to see realistic, useful graphical presentations of their performance on a 'trackside' laptop or for a doctor to monitor the motion of a prosthetic patient and make real-time equipment adjustments.

In conclusion we hope that this project will lead to a significant reduction in the length of the feedback loop for athletes, coaches and medical professionals when attempting to improve performance or accelerate rehabilitation. We propose a mechanism whereby we improve not only the measurement and analysis of performance but, crucially, the presentation of both errors and desirable properties to the subject, be it doctor, coach or athlete.

Technical proposal (5000 chars)

The main aim of the research described below is to apply and extend state of the art techniques in computer graphics and computer vision to the field of real time, or close to real time, visual feedback. Such instant feedback has the potential to accelerate progress in the fields of medical rehabilitation, injury analysis and sporting technique improvement.

The research will concentrate on three main areas: motion capture, data analysis and visualisation. Each of these areas will be discussed in more detail below.

Motion Capture

Capture of 3d motion using multiple camera systems may be achieved using a marker system, passive or active, or extracting motion directly from synchronised video streams. To reduce time between subject performance and feedback to them, it is vital that any system is portable, easy to use and has rapid set-up time.

The institution has a 16 camera PhaseSpace motion capture system (www.phasespace.com) and belongs to the EPSRC-funded SESAME consortium (www.sesame.ucl.ac.uk) whose aim is to investigate sensor-based systems to enhance the performance of elite athletes. We would work closely with PhaseSpace to develop rapid re-calibration algorithms allowing the cameras to be placed at any convenient capture location and be capture-ready almost instantly. A system in which the cameras can be moved during capture would be the ultimate goal. This re-calibration work would require extremely fast computation, particularly as the cameras we are dealing with unusually consist of 1d arrays rather than 2d camera image planes. Our aim is to work in the mathematical framework of geometric algebra (GA) and to utilise fast algorithms developed for computer graphics for this work (e.g. tinyurl.com/ya3clh).

Data Analysis

Providing rapid feedback inevitably requires efficient data analysis. Motion capture systems only give information on marker positions or, in the case of capture from video streams, movement of the body surface. From this we must infer limb segmentation and hierarchical linking, as well as joint positions and rotations. To date there is one prototype system (tinyurl.com/ybhqav) which gives simple real-time feedback; this system uses GA to infer joint characteristics. We will extend the ideas behind this promising initial research to include a detailed investigation into the noise characteristics in order to increase the quality of the inferred data and to reduce its variance. Again, fast techniques from graphics will be utilised.

Real time or interactive medically accurate skeletal data will then be fused with data from other sensors (inertial, strain gauge, etc) to form a complete picture of the internal dynamics and external force characteristics to provide a tool which will enable useful conclusions to be drawn, e.g. athlete's performance or patient rehabilitation progression.

Visualisation

Data visualisation is crucial for any genuinely useful system. We hope to progress the state of the art in visualisation of markered motion capture data and characteristics inferred therefrom. To do this we would provide real-time representations of joint motion, forces and other relevant parameters and use these in an interactive diagnostic tool for detecting medical problems and flaws in athletic performance.

A first stage in the SESAME project above will be to use simple fixed-camera tracking techniques to guide high-speed cameras on pan-tilt heads. This will produce a deliverable of multiple view high-speed video playback of athletic performance. Model building from this data using techniques such as silhouette carving will be investigated. In particular, our hope is to provide a system which rapidly 'scans' an athlete into the visualisation system. The huge amounts of data involved will require implementation of the most recent developments in data processing and will draw heavily from graphics research.

To date, auto-rigging and skinning algorithms to allow visualisation of a realistic model which faithfully represents the motion given the data, are not feasible in real-time feedback systems. We hope to advance the state of the art in this field and will draw on results from a current DTI/EPSRC-funded project (J1522D) in this area whose aims are to use GA techniques for advanced mesh deformation. This research will proceed together with the investigation of new methods of visualising performance-related data directly onto an animated model.

Timescales and Milestones

• Summer 2007: Multiple view high-speed video streams will be available from the SESAME project.
• Oct 2007-8: Model-building. Visualisation and auto-rigging work, will proceed in in parallel over this year.
• Oct 2008-9: Concentrate on the auto-calibration phase and test out the utility of such a system.
• Oct 2009-10: Realisation of the data analyis techniques.
• Oct 2010-12: The various strands of the research project are brought together and tested in applied sitations.

Justification of resources (1000 chars)

• Salary is based upon institutional pay scales for staff with PhD and 2 years of industrial experience.
• Initial equipment budget includes purchase of laptop computer with
  • A high-end graphics card since it is envisioned that the project will be graphics heavy and research experience is in the field of computer graphics.
  • A fast data capture rate since it is envisioned that the project will include capturing high-framerate video streams in real time.
• Ongoing equipment budget includes purchase of camera systems for computer vision tasks, sensors for application to body and maintainance/upgrade of the group's motion capture rig.
• Consumables include
  • Initial outlay for software to analyse and process data (e.g. MATLAB, Motion Builder, etc).
  • Ongoing upgrade of software, stationary, etc.
• Travel costs are based upon institution provided typical figures for measurement-based projects.