A couple of weeks ago, the video Clouds premiered at the Wired Frames exhibit at Eyebeam in NYC. I had the good fortune of being involved in this project, which was led by documentary filmmaker Jonathan Minard and creative coder James George. The three of us met at the Art&&Code conference in Pittsburgh, in October ’11. In addition to attending the conference, we also participated as “labbers” for the four days preceding it. Minard had intended to interview each of the labbers before the conference, and was interested in trying to add Kinect data into the process. It turned out that George had been planning to do a similar mixing of Kinect and DLSR data, so they combined forces.
By the end of the labber session, the pointcloud remix project was born. By mapping real-world color information (through the digital camera) onto points in space (provided by the Kinect), Minard and George had developed a new documentary medium. Realistically coloring a pointcloud was one thing, but this merging of datasets offered the possibility of customizable, remixable video playback. For starters, the camera view did not have to be dictated by the real-life camera position during filming; video playback could include all sorts of pans and zooms. Even more exciting is the possibility of writing code to act on top of the pointcloud, opening up possibilities such as image processing, mesh distortions, or particle systems.
In the beginning of 2012, George got back in touch with the labbers, announcing a video editing program that he had written on top of openFrameworks. Each person was given their interview data, and had the opportunity to remix it in their own way. For my experimentations, I focussed on distorting the mesh of points, as well as tweaking colors. The final compilation of my interview was an exploration of these forms, beginning at a relatively stable form, and then becoming more distorted and crazy as the interview progressed. Below are stills from different sections of the interview, as well as a video of one of the more wild distortions.
