Professional Activities
Adobe Digital Imaging group
I recently joined the Adobe Digital Imaging team, the organization that develops, among other things, Photoshop. It's a bit early to disclose what I'm working on. :-)
Adobe Image Foundation
Microsoft was a great place to work and the DirectX team was and is an awesome group. But an opportunity to work at Adobe on a cool project came up right as Windows Vista was shipping. I was ready to make a change and Adobe came along at the exact correct time. Adobe Image Foundation (AIF) is a technology for improving the use of GPU resources within the Adobe product line. Pixel Bender is the graphics langauge that is part of AIF and I'm one of the two compiler developers on the project. We have an active 3rd party developer community writing Pixel Bender and can be used in After Effects, Photoshop, and Flash (via byte code translator). You can learn more about AIF here and on wikipedia here.
I gave a talk about our language and our use of LLVM for code generation here.
As a side benefit for leaving Microsoft, since I now live and work in Seattle proper, I rarely have to sufer the notorious i520 bridge. While I don't do it every day, I love riding my bike to work now.
DirectX
My last work which appeared in Windows Vista were bug fixes to the HLSL compiler in addition to improvements to the mid-level optimization and value range propagation mechanisms within the compiler. My time in DX helped me to page graphics back into my head and to learn more about the shader side of graphics. The team is composed of some really great folks and I could have easily envisioned staying there for a long time, but I was wooed by a very intersting project at Adobe.
Elliptic-Curve Cryptography Smartcards
Once the late-term decision to add elliptic curve crypto support to Vista was made, long term projects were put on hold in order to fully plumb ECC into Vista and the enterprise support infrastructure that makes up a Windows-based installation. My little part of the fire was writing the key storage provider for dual-mode ECC & RSA smartcards. Along with an wonderfully talented and pleasant developer named Dan Sledz, I helped write the code to support products like this one from Spyrus.
XrML 2.0 & Windows Vista Software Licensing
When Windows Rights Management 1.0 wrapped up, my friend Parama and I took a look at the authorization side of what we had worked on and realized that it was a topic we wanted to pursue further. While his modesty would probably keep him from saying so, Parama is one of the big brains who had a huge impact on the XrML 2.0 standard, especially helping to ensure that you could model security declaratively with it. Together with Matt Stipes, we formed the XrML team within Windows Security under John Manferdelli. Our team's first offering was to write an XrML library in managed and native code which was used by the Windows Vista software licensing server and client teams respectively to peform license authorization check. In addition to coding, I managed the two other developers on the team: Nicolas Payette and Sachin Deshpande both formerly from the WRM team.
Our project was well along in its integration into Vista early on and this left us towards the end of the Vista cycle to begin to prototype declarative-based models for security administration. This work, while still in the incubation stage, was well regarded by external CSO's to whom it was exhibited. I think it would also be useful for making for simpler administration of the security features of home systems.
Microsoft Research Beijing
When WRM 1.0 was wrapping up, I had the opportunity to spend three crazy weeks at MSR Beijing helping to whip some papers into shape. One I had enough of an impact on that I became an author. Here it is:
Example-Based Composite Sketching of Human Portraits
Creating a portrait in the style of a particular artistic tradition or in the personal look of a particular artist is a difficult problem. Elusive to codify algorithmically, the nebulous qualities which combine to form artwork are often well captured using example-based approaches. These methods place the artist in the process, often during system training, in the hope that their talents may be tapped.
This paper demonstrates an example-based system which is used to turn images of East Asian females into the style of a Manga sketch while still maintaining the look of the subject.
Appeared in NPAR. PDF
Windows Rights Management
I had been doing animation related work since 1993 and in 2000 I decided to try my hand at something new and to see what life was like in the world of shipping software. I left MSRG to work on Windows Rights Management under John Manferdelli and became a principal developer for the WRM client API. You are most likely to have encountered this work with rights protected email or documents in Microsoft Office, but many folks have been using the platform I helped develop. You can learn more about this work by clicking here.
Microsoft Research Graphics
I began my career at Microsoft in the graphics group of MSR/Redmond. This was an awesome place to be. I had unfettered access to many of the biggest brains in graphics. It was a social group which took lunches together at 11:30 (we were a bit retentive about the promptness of it) and in the summers interns from all over would vie to join us. At MSRG I worked on a number of projects there, mostly related to human figure animation or shape modelling. Here are the papers which resulted from that work:
Artist-Directed Inverse Kinematics Using Radial Basis Function Interpolation
One of the most common tasks in computer animation is inverse-kinematics, or determining a joint configuration required to place a particular part of an articulated character at a particular location in global space. Inverse-kinematics is required at design-time to assist artists using commercial 3D animation packages, for motion capture analysis, and for run-time applications such as games.
We present an efficient inverse-kinematics methodology based on the interpolation of example motions and positions. The technique is demonstrated on a number of inverse-kinematic positioning tasks for a human figure. In addition to simple positioning tasks, the method provides complete motion sequences that satisfy an inverse-kinematic goal.
The interpolation at the heart of the algorithm allows an artist's influence to play a major role in ensuring that the system always generated plausible results. Due to the lightweight nature of the algorithm, we can position a character at extremely high frame rates, making the technique useful for time-critical run-time applications such as games.
Appeared in Eurographics 2001. PDF. Supplementary video segments.
Shape by Example
Modern modeling systems enable artists to create high quality content, but provide limited support for interactive applications. Although complex forms can be constructed either by hand or with geometry capture technologies, once they are created, they are difficult to modify, particularly at runtime.
Interpolation provides a way to leverage artist-generated source material. We present a methodology for efficient runtime interpolation between multiple forms. Linear plus radial basis functions provide the key mathematical support for the interpolation. Once our system is provided with example forms, it generates a continuous range of forms we call a shape. We also apply the shape interpolation methodology to articulated figures and human face models to create smoothly skinned figures that deform in natural ways. Unlike previous formulations, the one presented here is efficient enough to support interactive design of the abstract interpolation space as well as support runtime interpolation of the forms in interactive applications such as games.
Appeared in Interactive 3D Symposium 2001. PDF. Longer tech report version.
Verbs and Adverbs: Multidimensional Motion Interpolation Using Radial Basis Functions
Ph.D. Thesis
This thesis describes a new technology dubbed Verbs & Adverbs, the goal of which is to create controllable animation through interpolation of example motions. Use of motion capture or hand-animated source material was a key requirement for this work since artists and capture systems currently produce the most compelling animations. Leveraging the artist's talent for interactive animation is an important goal for 3D human figure animation research.
Interactive, non-scripted animations require control mechanisms. With control, a system can be designed to react to a user's wishes, a simulation's unfolding state, or both. Many control techniques are being pursued by the research community with dynamic, procedural, and interpolated methods the three primary groupings. Dynamic animations require a complex controller and a dynamics simulator. Procedural animations use code-like scripts. Both of these methods are alien to classically-trained animators. Likewise, these techniques do not easily incorporate motion capture. Thus we have developed a technique that supports, rather than supplants, the artist or motion capture system. Verbs & Adverbs is a system which seeks to empower the artist, to provide a new way to construct animations for use in the interactive realm.
This thesis describes a process for turning source motions into rich controllable animation segments called "verbs," parameterized along a number of control axes called "adverbs.'' The core mathematics for this technique uses multi-dimensional function interpolation with radial basis functions. Once created, verbs are placed into a "verb-graph,'' an object detailing the appropriate times to transition from one verb to another. The verb-graph is the entity controlled by the interactive system.
Additionally, a number of ancillary topics are detailed in this thesis. Robust motion capture analysis, torque-minimal transitioning between motions, and a standard motion interface, or formalism, are all developed here. These, when combined with the Verbs & Adverbs technique, form a powerful animation system.
PDF.
Verbs & Adverbs: Multidimensional Motion Interpolation
This paper describes methods and data structures used to leverage motion sequences of complex linked figures. We present a technique for interpolating between example motions derived from live motion capture or produced through traditional animation tools. These motions can be characterized by emotional expressiveness or control behaviors such as turning or going uphill or downhill. We call such parameterized motions "verbs" and the parameters that control them "adverbs." Verbs can be combined with other verbs to form a "verb graph," with smooth transitions between them, allowing an animated figure to exhibit a substantial repertoire of expressive behaviors.
A combination of radial basis functions and low order polynomials is used to create the interpolation space between example motions. Inverse kinematic constraints are used to augment the interpolations in order to avoid, for example, the feet slipping during a support phase of a walk cycle.
Once the verbs and verb graph have been constructed, adverbs can be modified in real-time providing interactive or programmatic control over the characters' actions. This allows the creation of autonomous characters in a virtual environment that exhibit complex and subtle behavior.
Appeared in IEEE Computer Graphics and Applications, Volume 18. PDF. Supplementary video segment.
The Process of Motion Capture: Dealing with the Data
This paper presents a detailed description of the process of motion capture, whereby sensor information from a performer is transformed into an articulated, hierarchical rigid-body object. We describe the gathering of the data, the real-time construction of a virtual skeleton which a director can use for immediate feedback, and the offline processing which produces the articulated object. The offline process involves a robust statistical estimation of the size of the skeleton and an inverse kinematic optimization to produce the desired joint angle trajectories. Additionally, we discuss a variation on the inverse kinematic optimization which can be used when the standard approach does not yield satisfactory results for the special cases when joint angle consistency is desired between a group of motions. These procedures work well and have been used to produce motions for a number of commercial games.
Appeared in Computer Animation and Simulation 1997. PDF.
Efficient Generation of Motion Transitions Using Spacetime Constraints
This paper describes the application of space time constraints to creating transitions between segments of human body motion. The motion transition generation uses a combination of spacetime constraints and inverse kinematic constraints to generate seamless and dynamically plausible transitions between motion segments. We use a fast recursive dynamics formulation which makes it possible to use spacetime constraints on systems with many degrees of freedom, such as human figures. The system uses an interpreter of a motion expression language to allow the user to manipulate motion data, break it into pieces, and reassemble it into new, more complex, motions. We have successfully used the system to create basis motions, cyclic data, and seamless motion transitions on a human body model with 44 degrees of freedom.
Appeared in Siggraph 1996. PDF. Tech report of dynamics formulation derivatives.
Charles F. Rose, III
