Creative Coding: How to Build Your Own VJ Engine in JavaScript
VJing, or live visual performance, is a popular way for digital artists to display animated graphics, especially at concerts, clubs and festivals. While there are many existing software packages for VJing, building your own custom engine using web technologies offers several advantages:
- Tailor the controls and features to your needs
- Tighter integration between your visuals and audio inputs
- Use the full power of HTML5 canvas and JavaScript for the visuals
- Easily share your VJ sets online or collaborate with other artists
In this guide, we‘ll walk through how to code a basic VJ engine in vanilla JavaScript. By the end, you‘ll have a working system for organizing your visual content and dynamically loading effects based on keyboard commands. Let‘s get started!
Organizing the Visual Content
The first step is deciding how to structure the folders and files containing your visual effects. I like to organize things into sets, banks, and individual content files:
- A set is a folder containing related banks of effects (e.g. for a particular performance or project)
- A bank is a subfolder holding up to 26 content files
- Content files are the individual JavaScript files with the code for each effect
With this structure, you can use the following keyboard controls:
- Number keys 0-9 to change sets
- Shift + number keys to change banks within the current set
- Letter keys a-z to trigger content files in the current bank
Here‘s what the folder hierarchy looks like for this setup:
/art
/0
/0
0.js
1.js
...
25.js
/1
...
/9
/1
...
/9
Each set folder contains 10 bank subfolders, and each bank has up to 26 JavaScript files, for a maximum of 2600 total effects. You can add a second keyboard layer with Shift + letters to double this if needed.
Loading Content with JavaScript
Now that we have our content organized, we need a way to dynamically load the JS files based on keyboard input. The core of this is a function to inject a <script> tag into the HTML page:
function loadJS(filename) {
// Remove previously loaded script if any
if (typeof(fileref) != ‘undefined‘) {
document.getElementsByTagName("head")[0].removeChild(fileref);
}
// Create new script element and set its src
fileref = document.createElement(‘script‘);
fileref.setAttribute("type","text/javascript");
fileref.setAttribute("src", filename);
// Inject script into document head
document.getElementsByTagName("head")[0].appendChild(fileref);
}We‘ll call this loadJS function whenever we want to load a new content file. It first removes any previously loaded script, creates a new <script> element, sets its src attribute to the provided filename, and injects it into the document <head>.
To trigger loading files, we‘ll listen for keypress events and determine which set, bank or file to load based on the key that was pressed:
var current_set = 0;
var current_bank = 0;
var current_file = 0;
function onKeyDown(event) {
var keyCode = event.keyCode;
if (keyCode >= 65 && keyCode <= 90) {
// Letter key pressed (a-z)
current_file = keyCode - 65;
loadContent();
}
else if (keyCode >= 48 && keyCode <= 57) {
// Number key pressed (0-9)
if (event.shiftKey) {
current_bank = keyCode - 48;
} else {
current_set = keyCode - 48;
current_bank = 0;
}
current_file = 0;
loadContent();
}
}
function loadContent() {
var filename = `/art/${current_set}/${current_bank}/${current_file}.js`;
loadJS(filename);
}
window.addEventListener(‘keydown‘, onKeyDown);Here we store the currently selected set, bank and file numbers in variables. The onKeyDown function checks if a letter or number key was pressed and updates the appropriate variable:
- If a letter key was pressed, we set
current_fileto 0-25 based on the key - If a number key was pressed, we set either
current_bank(if shift key is down) orcurrent_set - We also reset
current_fileto 0 when changing banks andcurrent_bankto 0 when changing sets
After updating the current selection, we call loadContent which generates the filename for the content to load and passes it to our loadJS injection function.
Creating the Visual Effects
So what goes inside those loaded content files? Each file should contain a self-executing function that draws a frame of the visual effect to an HTML canvas. Here‘s a simple template:
(function () {
function draw() {
// Your creative code goes here
}
window.draw = draw;
})();Wrapping the draw function in a self-executing anonymous function ensures any variables declared don‘t pollute the global scope and lets us redeclare the function each time a new file is loaded.
Assigning the draw function to window.draw allows us to call it from outside the self-executing function. We‘ll set up a main animation loop that checks if window.draw is defined and calls it at the target framerate if so:
var frameCount = 0;
var targetFPS = 60;
var lastDraw = Date.now();
function loop() {
var elapsed = Date.now() - lastDraw;
if (elapsed >= (1000 / targetFPS)) {
frameCount++;
lastDraw = Date.now();
if (typeof(window.draw) == ‘function‘) {
window.draw();
}
}
requestAnimationFrame(loop);
}
loop();Now that we have the core engine working, the creative fun begins! Inside your draw functions you can use any JavaScript drawing APIs – 2D canvas, WebGL, SVG, CSS, etc.
Here are a few ideas and techniques to try:
- Periodic functions like sine and cosine waves to create repeating patterns
- Simplex or Perlin noise to generate organic textures
- Fractals like the Mandelbrot set
- Reaction-diffusion algorithms to simulate chemical processes
- Lorenz attractors and other chaotic systems
- Cellular automata like Conway‘s Game of Life
- Particle systems and flocking behaviors
- Procedural generation of 3D meshes and terrains
The possibilities are endless! Playing with different algorithms, color palettes and compositions, you can create an infinite variety of dynamic visuals.
Adding Audio Reactivity
To make the visuals respond to music, we need to analyze the audio signal in real-time. The Web Audio API makes this fairly straightforward. First we create an audio context and hook up the computer‘s line input or microphone:
var audioCtx = new AudioContext();
function connectInput() {
navigator.mediaDevices.getUserMedia({ audio: true, video: false })
.then(function(stream) {
microphone = audioCtx.createMediaStreamSource(stream);
analyser = audioCtx.createAnalyser();
microphone.connect(analyser);
})
.catch(function(err) {
console.log(‘Error getting microphone input: ‘ + err);
});
}
connectInput();This sets up an audio graph with an input source and an analyser node which lets us access the raw PCM audio data or frequency spectrum. We can call analyser.getByteTimeDomainData() or analyser.getByteFrequencyData() each frame to grab the latest audio information and use it to influence our drawing code:
var bufferLength = analyser.frequencyBinCount;
var dataArray = new Uint8Array(bufferLength);
function draw() {
analyser.getByteFrequencyData(dataArray);
var bass = dataArray[0];
var mid = dataArray[511];
var treble = dataArray[1023];
context.fillStyle = `rgb(${bass}, ${mid}, ${treble})`;
context.fillRect(0, 0, width, height);
}This retrieves the frequency spectrum data, grabs the first, middle and last elements as measures of the bass, midrange and treble levels, and sets the canvas color based on them. More sophisticated audio analysis like beat detection can also be done to trigger animations in sync with the music.
You‘ll likely want to provide some way to adjust the input gain from within the VJ engine. I map the +/- keys to increase or decrease the input volume:
function onKeyDown(event) {
...
else if (event.keyCode == 189) {
inputGain -= 0.1;
}
else if (event.keyCode == 187) {
inputGain += 0.1;
}
inputGain = Math.min(Math.max(inputGain, 0), 1);
microphone.gain.setValueAtTime(inputGain, audioCtx.currentTime);
}Expanding Your VJ Engine
There are many ways you can extend and enhance your engine. Some ideas to consider:
- Use MIDI or OSC messages to control the visuals, letting you use external hardware interfaces
- Add post-processing effects like blurs, distortions, feedback, etc.
- Provide controls for audio effects like delays, filters, flanging, etc.
- Mix multiple content layers together with transparency, blending modes, masking, etc.
- Set up a remote interface on a separate device for changing sets and controlling effects
- Stream the canvas output to a remote display or across the network using WebRTC
I hope this tutorial has given you a solid foundation for building your own browser-based VJ software. The combination of JavaScript graphics APIs and real-time audio input opens up a world of creative possibilities. I‘m excited to see what you create!
Remember, you can find all the code for this project at [https://github.com/username/javascript-vj-engine](). If you make something cool, please share it with me [on Twitter]() or [Instagram](). Happy live visual coding!