Augmenting with optical flow

From Wikipedia,

Optical flow or optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene.

The video above augments a water drop with a fluid simulation activated by the water’s optical flow. Meta, if you will. In a way, the augmentation shows something that is present but invisible, specifically the derivatives of the motion, the forces acting on the water. The water drop source video is on YouTube.

In mathematics and physics, we’re often concerned with derivatives, the rates of change. If we represent x as the position of a particle, d’(x) represents its velocity and d”(x) represents the particle’s acceleration. In the physical world, at any point of time, we see x, the instantaneous position of any object. But the world is not static; things are constantly moving, whether affected by natural forces or a consequence of our actions and decisions. This project makes visible the d’(x), the change of position, with a fluid simulation. There’s something beautiful about continuity.

Other Videos


The fluid simulation is provided by the openFramework library ofxMSAFluid.

The program uses the cvCalcOpticalFlowBM function of openCv, which provides two CvMat’s of derivatives (dx and dy) for a set number of points. For input, the function takes two CvImage’s, one for the previous frame and one for the current frame.

The following code extracts the derivatives from the CvMat into double arrays.

void ofxCvOpticalFlowBM::getField(double * dx, double * dy) {
    for (int y = 0; y < rows; y++) {
        for(int x = 0; x < cols; x++) {
            dx[y * cols + x] = cvGetReal2D (vel_x, y, x);
            dy[y * cols + x] = cvGetReal2D (vel_y, y, x);