Flying memes

Following a suggestion by a VJ friend of mine I worked on creating a real time HTML5 video mosaic that can be used to aggregate videos from your hard disk in a custom-size grid.

To accomplish that I set-up a two-page structure: a dashboard, used to choose which files to play and to control the grid size (other controls may came next), and the mosaic page itself: in this way one page can be projected in full-screen while the other one can be viewed on the second monitor.

In the dashboard lays a drop-area where video files can be dropped, dropping one or more file cause afunction ‘caricaIlVideo’ to be called:

caricaIlVideo = function(evento){
  var files = evento.dataTransfer.files;
  for(var x=0; x < files.length; x++){
    console.log("wooo");
    worker.port.postMessage("comando-verso-mosaic:carica-video:" + 
      window.webkitURL.createObjectURL(files[x]));
  }
}

The function uses Drag And Drop API and File API to recover the dropped files, then a message is sent to a SharedWebWorker for each of the dropped files containing a string like :

"comando-verso-mosaic:carica-video:blob:http://sandropaganotti.com/ecba59ec-d1cc-435c-9bd3-ba50e2bf0eab"

the blob* part is the most interesting: it contains the URL of the File dropped. The SharedWebWorker then unfold the message removing the ‘comando-verso-mosaic’ part and sends the rest to the mosaic page.

When the mosaic page receives a message like this it launch a function called ‘caricaVideo’:

caricaVideo = function(blob_url){
  var video  = document.createElement('video');
  var source = document.createElement('source');
  video.muted = true;
  video.volume = 0.0;
  video.loop  = true;
  source.src = blob_url;
  source.type= "video/webm";
  video.appendChild(source);
  document.querySelector("#videos").appendChild(video);
  video.addEventListener('canplaythrough', iniziaLaProiezione, false);
}

The function basically creates a new video element using the blob URL as a source and then attach to the element a listener for the ‘canplaythrough’ event, which occurs when the video buffer reach enough capacity to guarantee a full-play performance.

The mosaic has a looping function, called ‘aggregaVideo’ which takes care of the drawing stuff: it iterates over all the video elements created and it draws them over a canvas according with the tessellation configuration:

aggregaVideo = function(canvas_context, videoboxes){
    canvas_context.canvas.width = canvas_context.canvas.width; 
    for(var i=0; i < tassello['tassellazione-y']; i++){
      for(var k=0; k < tassello['tassellazione-x']; k++){
        if(videoboxes[k + i*tassello['tassellazione-y']] != undefined){
          canvas_context.drawImage(videoboxes[k + i*tassello['tassellazione-y']], 
            tassello.width * k, tassello.height * i,
            tassello.width , tassello.height
          );
        }
      }
    }
    setTimeout(function(){aggregaVideo(canvas_context,  document.querySelectorAll('#videos video'))},0);
}

Here’s a video showing the whole thing running:

The code of this project is, as usual, on my github account.

Finally you can test the video mosaic by yourself launching both the dashboard and the mosaic; be sure to have some webm video files on your computer, you can find some here: http://www.webmfiles.org/demo-files/ or you can convert some using miro video converter. Please note that everything should work, at the moment, only on Chrome 10.

Thanks!

Tags: html5, video mosaic