Canvas – Scaling

Adapting a canvas application to display on multiple devices with different resolutions and aspect ratios is a gnarly but necessary task. Before diving further into creating canvas art, this post designs a framework that handles scaling issues and can be found on codepen as a template.

Presenting canvas art on multiple devices usually involves one of two strategies:

  1. Scale the real dimensions of the canvas (may be hard, depending on the design.) The physical dimensions of the canvas change; the position and size of the canvas elements must adapt programmatically based on the dynamic resolution and aspect ratio of the device.
  2. Scale the Canvas element (challenging, depending on the range of devices.) This approach starts with a fixed canvas size; the position and size of canvas elements target this size and the canvas dimensions do no appear to change to the program, however, the canvas element itself is visually scaled used CSS to match the device resolution and aspect ratio.

Approach #1: Code for Resolution Independence

Generally this is much trickier to code for. Starting with a simple example:

var canvas = document.getElementById('canvas'); 
var ctx = canvas.getContext('2d'); 
var img = new Image();

img.src = '200ax200a.png'; 
img.onload = draw;

function draw(){

    ctx.fillStyle = "#48160D";
    ctx.fillRect(0,0,300,300);

    ctx.fillStyle = "#FEBF28";
    ctx.fillRect(0,0, 50,50); // top left corner
    ctx.fillRect(0,250, 50, 50); // bottom left
    ctx.fillRect(250,0, 50, 50); // top right
    ctx.fillRect(250, 250, 50, 50); // bottom left

    // put a 200x200 image in the middle
    ctx.drawImage(img, 50, 50);
}

on a device with screen dimensions of 1024 x 768 this code will result in this:

2015-09-26_192120

Note the page background color is a lighter red than the canvas background to demarcate what is happening. The canvas can be repositioned with CSS but here it defaults to the standard flow in HTML where elements are written from the top, left corner to the right until they wrap or extend off-screen.

To own the space on the page with a resolution independent approach every element drawn on the canvas needs to adapt:

function draw() {
    ctx.fillStyle = "#48160D";
    ctx.fillRect(0,0,canvas.width,canvas.height);

    ctx.fillStyle = "#FEBF28";
    var newRW = 0.166 * canvas.width;
    var newRH = 0.166 * canvas.height;
    var newIW = 0.666 * canvas.width;
    var newIH = 0.666 * canvas.height;

    ctx.fillRect(0,0, newRW,newRH); // top left corner
    ctx.fillRect(0,canvas.height-newRH,newRW,newRH); // bottom left
    ctx.fillRect(canvas.width-newRW,0, newRW,newRH); // top right
    ctx.fillRect(canvas.width-newRH, canvas.height-newRH,newRW,newRH); // bottom left

    // put a 200x200 image in the middle and scale up
    ctx.drawImage(img, canvas.width/2-newIW/2, canvas.height/2 - newIH/2, newIW, newIH);
}

Now on the 1024 x 768 layout it looks like this:

2015-09-26_195214

It’s elements are skewed because it ignores the aspect ratio of the original (300×300 or 1:1)  It just took whatever ratio was needed to fill the display area using the following for its adaption:

function resizeCanvas() {
    var viewport = { // Get the dimensions of the viewport
        width: window.innerWidth,
        height: window.innerHeight
    };
    // adjust canvas
    canvas.width = viewport.width; 
    canvas.height = viewport.height;
    draw();
}

For a best fit, preserving the original aspect ratio but filling as much of the viewport as possible a different approach is required:

function resizeCanvasAspect() {

    var viewport = { // Get the dimensions of the viewport
        width: window.innerWidth,
        height: window.innerHeight
    };

    if (canvas.height / canvas.width > viewport.height / viewport.width) {
        newHeight = viewport.height;
        newWidth = newHeight * canvas.width / canvas.height;
    } else {
        newWidth = viewport.width;
        newHeight = newWidth * canvas.height / canvas.width;
    }

    // adjust canvas
    canvas.width = newWidth;
    canvas.height = newHeight;
    draw();
}

Now it stays proportional and fills the max available space with those proportions:

2015-09-26_201556

adding a little tweak to center the canvas on the page:

// center
var newX = (viewport.width - newWidth) / 2;
var newY = (viewport.height - newHeight) / 2;

// Center by setting the padding of the game
document.getElementById("canvas").style.padding = newY + "px " + newX + "px";

2015-09-26_202339

Note the CSS background color now brackets each side of the centered canvas. This is called a “letter-box” Some ways of mitigating this padding are discussed a bit later.

Approach #2: Scale the Canvas Element

The second technique, scaling the canvas, uses CSS to resize the canvas element and doesn’t require adapting any of the canvas code or recalculating coordinates. As far as the code is concerned it’s still operating in a 300×300 pixel world:

var canvas = document.getElementById('canvas'); 
var ctx = canvas.getContext('2d'); 
var img = new Image();

img.src = '200ax200a.png'; 
img.onload = resizeCanvas();


function draw(){

    ctx.fillStyle = "#48160D";
    ctx.fillRect(0,0,300,300);

    ctx.fillStyle = "#FEBF28";
    ctx.fillRect(0,0, 50,50); // top left corner
    ctx.fillRect(0,250, 50, 50); // bottom left
    ctx.fillRect(250,0, 50, 50); // top right
    ctx.fillRect(250, 250, 50, 50); // bottom left

    // put a 200x200 image in the middle
    ctx.drawImage(img, 50, 50);
}


function resizeCanvas () {
    // Get the dimensions of the viewport
    var viewport = {
        width: window.innerWidth,
        height: window.innerHeight
    };

    // Determine canvas size
    if (canvas.height / canvas.width > viewport.height / viewport.width) {
        newHeight = viewport.height;
        newWidth = newHeight * canvas.width / canvas.height;
    } else {
        newWidth = viewport.width;
        newHeight = newWidth * canvas.height / canvas.width;
    }


    // Resize canvas
    document.getElementById("canvas").style.width = newWidth + "px";
    document.getElementById("canvas").style.height = newHeight + "px";

    // center
    newX = (viewport.width - newWidth) / 2;
    newY = (viewport.height - newHeight) / 2;

    // Center by setting the padding of the game
    document.getElementById("canvas").style.padding = newY + "px " + newX + "px";

    draw();
}

window.addEventListener("resize", resizeCanvas);

A CSS Transform could be use in place of modifying the style width and height above, and may prove faster as transforms are often optimized in the GPU of modern devices:

function resizeCanvas () {

    var scaleX = window.innerWidth / canvas.width; 
    var scaleY = window.innerHeight / canvas.height;

    var scaleToFit = Math.min(scaleX, scaleY);

    //var scaleToCover = Math.max(scaleX, scaleY); 
    document.getElementById("canvas").style.transformOrigin = "0 0"; //scale from top left 
    document.getElementById("canvas").style.transform = "scale(" + scaleToFit + ")";


    // Center by setting the padding of the game
    var newX = Math.floor(((window.innerWidth - scaleToFit * canvas.width) / 2) / scaleToFit);
    var newY = Math.floor(((window.innerHeight - scaleToFit * canvas.height) / 2) / scaleToFit);
    document.getElementById("canvas").style.padding = newY + "px " + newX + "px";

    draw();
}

Some notes on managing letter-boxing:

  • Use a background image on the page that blends with the canvas.
    https://css-tricks.com/perfect-full-page-background-image/
  • Extend the canvas a little more than needed, center it and keep all the critical operations with a safe-area
    http://www.williammalone.com/articles/html5-game-scaling/

Choosing a good aspect ratio helps mask the letter-box effect. Unfortunately there is no universal ratio that works best, it all depends on your target audience/devices. Popular ratios for HTML5 game development are:

  • 800×500 (1.6)
  • 960×540 (1.77..)
  • 960/640 (1.5)

Viewports

On desktop browsers it’s pretty simple, the size of the viewport is the size of the browser window. On mobiles it gets squirrelly.

A mobile device, with physical screen of 480 x 640 css pixels, for example, runs into a typical website, not optimized for mobile or responsive, which generally has widths between 800 and 1024. In order to try to display this site without zooming into a corner, the mobile will claim it has a default width of 980 (usually, though it varies by mobile device) This is the layout viewport, used to accommodate sites designed for desktops. A second viewport, the visual viewport is different and defines the area of the layout currently visible. This viewport zooms in and out on the content without affecting the layout viewport. A third viewport, the ideal viewport, is the dimensions that the mobile device reports as optimal for a display area that doesn’t need to zoom. This viewport is defined by the mobile’s browser (i.e. different vendor’s browsers on the same mobile may have different ideal viewports) and usually matches, more or less, the physical CSS pixels (not the device pixels) that the mobile is capable of.

When a meta tag in the HTML states:

<meta name="viewport" content="width=device-width">

it tells the mobile to use the ideal viewport size instead of the default layout size and basically means the site is mobile ready.

This will be used for the canvas html as well, but it needs some additional tweaks:

<meta name="viewport" content="user-scalable=no, initial-scale=1, maximum-scale=1, minimum-scale=1, width=device-width, height=device-height, target-densitydpi=device-dpi" />

While heretical to responsive design, where disabling user zoom is regarded as evil, it is essential for our canvas art where a double touch can result in zooming parts of the canvas off the screen.

Those are the hacks to-date, with more experiments this article may undergo further revision.

There is also a set of strategies for optimizing for high pixel densities, where 1 CSS pixel may map to 2 or more actual device pixels. Topic for a future article though.

Some additional reading and sources:

Canvas – Drawing on Images

This project adds an additional canvas function, some basic animation and does a little sculpting of randomness.

The new canvas function is

drawImage(image, x, y)
which draws an image on the canvas at coordinates x,y

to draw an image it needs to be loaded. One way to do this is load it from the same directory as the HTML and JavaScript:

var canvas, ctx;
var img = new Image();
img.src = 'skyline.jpg'; 
img.onload = init;

function init(){
    canvas = document.getElementById("canvas");
    ctx = canvas.getContext("2d");
    ctx.drawImage(img, 0, 0);
}

Images loaded this way load asynchronously, which means they may not be completely loaded when drawImage is called. Telling the img object to call a function after the load is completed avoids that problem.

It’s also possible to pre-load the image with a data url, which is the raw image bits encoded in a string, e.g.:

img.src = 'data:image/jpeg;base64,/9j/4QAYRXhpZgAA....';

Places like DataURL.net allow uploading images and offer a conversion service. However, this can also be done easily using the canvas. After loading the image using the first example:

var dataURL = canvas.toDataURL('image/jpeg', 0.2); // override default png and
// specify jpg quality 0..1
tb = document.createElement("textarea");
document.body.appendChild(tb);
tb.value = dataURL;

The encoded string for a jpg image can then be copied from the textarea on the page and pasted into a img.src=”; statement.

The downsides are that embedding images this way can result in a quite a large chunk of text in the code and they aren’t cached by the browser. The advantages are no waiting for images to load, they can be used immediately, and the images don’t need to be hosted externally.

Using an image as a backdrop, effects can be created by drawing rectangles with colors and opacity settings on top.

The CityScape

One idea for using images and rectangles was to find an image of a skyline, with silhouettes of buildings, and then add blinking windows to them using small rectangles.

I brought the image into Photoshop and used the rectangular selection marquee to plot out the areas where the “windows” should go.

2015-09-23_162504

By drawing the marquee and then selecting Transform (Ctl or Cmd T) the info box populates both the height and width of the rectangle as well as the x,y origin point of the top left.

The areas can then be mapped as entries in a JavaScript array:

areas = [
    {x:0,y:219, w:17, h:38}, {x:39, y:227, w:30, h:30},
    {x:80, y:233, w:11, h:47}, {x:101, y:163, w:21, h:111} 
];

A function can be written that will take this area and for each area, draw however many rectangles will fit with coordinates inside the area.

The next task is to make the rectangles blink on an off and this introduces a very important tool for generative art: animation.

We need a loop that will run continuously and update the canvas each time it is called. The modern approach to this is via:

RequestAnimationFrame(callback)
tell the browser to call the function “callback” before the next canvas repaint

So setting up a basic loop that will continuously call the draw function to update the canvas:

window.requestAnimationFrame(draw); // call draw before next canvas repaint
function draw() {
     // do canvas draw stuff  
    window.requestAnimationFrame(draw); // call draw again before next repaint
}

If things are running optimally the draw() function will generally be called about 60 times a second.

The final construct needed to implement CityScape was a way to control the random toggling of lights on and off. If random() was used each time the draw() function was called, 60 times a second, it would just be a spastic jitter. To slow it down, the random behavior needs to be tweaked to be a little less uniform. Shaping randomness in various ways is very useful in tuning the aesthetics of its contribution to a piece. In this case, it toggles a light on or off only a small fraction of the time:

if (Math.random() < 0.005) {
    rects[i].d = !rects[i].d;
}

If something needed to happen only 1 every 10 times:

if (Math.random() < 0.1) {
    rects[i].d = !rects[i].d;
}

Since random() returns floating point numbers fairly uniformly between 0 and 1, the above would be true only 10% of the time.

Putting these new constructs together into a finished piece:

See the Pen Canvas – Drawing on Images by Kentskyo (@kentskyo) on CodePen.0

Canvas – The Beginning

This is an expedition into generative art using JavaScript and the HTML canvas. It is not, however, an HTML or JavaScript tutorial. It’s a diary of a deep dive into both the nature of generative art and the use of canvas for its expression. Putting it in writing, and words, helps me bring clarity to my journey. But a collateral audience might be someone who knows a little JavaScript, or is willing to pick up the basics, and who wants to explore the possibilities of creating art with code with me. If that’s you, welcome!

To me, generative art is a form that plays in the borders between intent and discovery. I will try to articulate my thoughts on this in time. There is something very special, I think, in the disciplines and boundaries that play between rules and chaos, between control and freedom.

The approach for this series is old school. There are a gazillion libraries, engines and frameworks that offer to make working with canvas easier. But the canvas api is not all that difficult and working with it directly imparts a certain economy and precision.

Creating visual designs this way requires only a simple page of minimal HTML with a canvas tag:

<!DOCTYPE html>
<html lang="en">
<head>
<title>Canvas Example</title>
</head>
<body>
<canvas id="canvas" width="150" height="150"></canvas>
</body>
</html

The <canvas> tag really only has two attributes, width and height, and they are optional, defaulting to 300 and 150 pixels respectively. There’s a third attribute: “id”, which is not really a canvas property per se but a handle HTML uses to reference it from JavaScript or CSS.

To draw something on the canvas with code requires a little JavaScript and it can be included from a file:

<!DOCTYPE html>
<html lang="en">
<head>
<title>Canvas Example</title>
</head>
<body>
<canvas id="canvas" width="150" height="150"></canvas>
<script src="draw.js"></script>
</body>
</html

The <script> tag loads the file “draw.js” from the same directory as the html page and executes it.

Since this isn’t a tutorial on HTML, CSS or JavaScript, here are a few external links if you are just getting up to speed:

For Programmers: Speaking JavaScript
Non-programmers: A Smarter Way to Learn JavaScript

&  A Smarter Way to Learn HTML and CSS

You won’t need too much to start and it may be better to learn what you need as you go rather than over prepare.

There are many ways to work with JavaScript code via editors, development environments and the like. These projects in canvas, however, can be done entirely inside the chrome browser using the Google Dev Editor for Chrome.

They can also be done in online sandboxes. For example, I’ll be using CodePen to publish the code and results and embed them here in the blog. The only reason I don’t do the whole thing in CodePen is I like the ability to doodle with code art in coffee shops and odd places that don’t always have internet connections. I know, heresy, but such places actually exist especially as I do most of my work/play out of the country (USA.) Instant access also rewards working in small time slices when the opportunity arises, although it’s almost guaranteed that you’ll get pulled into something deep at some point or another if you engage in this activity that will in turn create flow states of hours of fascinated absorption.

With the preamble out of the way, it’s time to dive in.

The canvas tag defines a rectangular surface with dimensions of height and width in a browser that code can then scribble on. The canvas coordinate system starts with x=0 and y=0 in the top left corner of the canvas. So a canvas with a height and width of 150 pixels would have these x,y coordinates:

coords
To start drawing on the canvas JavaScript needs a reference to the canvas tag using its “id” attribute. There can be multiple canvases on a page, each with different ids. Obtaining the reference is illustrated in the following code, which first grabs the  canvas id and then uses it to create a drawing context:

var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");

ctx is assigned a canvas context; a context returns different tools based on whether it is writing to a canvas in 2d mode, which is the most widely supported, or 3d, also known as webgl which later projects will delve into.

Using the ctx object returned from the request, we can access functions which will write directly on the canvas.

And this is enough of a start to make something interesting. In generative art, it’s often fun to start with just a handful of elements and constraints and see what happens. This project will use only:

  • rectangles and the 3 functions that draw rectangles in the canvas api
  • colors – for both the strokes (the outline) and the fills (inside area) of the rectangles
  • random numbers – injecting the essential “discovery” element into the generative art piece
  • and nothing else, canvas-wise

A rectangle is the “hello world” of canvas drawing; there are 3 functions that create them:

fillRect(x, y, width, height)
draws a rectangle filled with the current fillStyle, defaults to black if no fill was specified.

strokeRect(x, y, width, height)
draws a rectangular outline with the current strokeStyle.

clearRect(x, y, width, height)
clears the specified rectangular area, making it fully transparent.

The two color styles mentioned above can be assigned colors in the format used by CSS.

fillStyle = “#ff0000”; // set to red
fillStyle = “red”; // also sets to red using css predefined color names (see link above for all the valid names)

strokeStyle = rgb(255,0,0); // same deal, sets to red using another css color construct
strokeStyle = rgba(255,0,0, 0.5); // sets to a red with an opacity of 50%
lineWidth = 2; // how many pixels wide the stroke is, defaults to 1

The ctx variable pulled from canvas can now be used to access all the functions in the canvas api. To draw a red rectangle at coordinates 50, 50 on the canvas:

ctx.fillStyle = "red"; 
ctx.fillRect(75,75,60,60);

And before running this, adding this css in the <head> section in the HTML will show the boundaries of the canvas more clearly to help see where in the canvas objects are being drawn:

<style type="text/css">
body {margin:20px; background-color: #202020}
canvas { border: 1px solid white; }
</style>

Which creates this:

red box

Notice the the x,y coordinates of the box starts at 75, 75 which is the exact center of the 150×150 canvas. The box isn’t centered though because the x,y coordinates is where the top, left of the box starts, not the middle. Now to punch a transparent hole in the middle of that red box:

ctx.clearRect(105,150,20,20);

punch hole

and finally, draw a rectangular outline:

ctx.strokeStyle = "red";
ctx.lineWidth = 3;
ctx.strokeRect(95, 80, 20, 50);

jail

Putting it all together, the final example above was created with the following HTML:

<!DOCTYPE html>
<html lang="en">
<head>
<title>Canvas Example</title>
<style type="text/css">
body {margin:20px; background-color: #202020}
canvas { border: 1px solid white; }
</style>
</head>
<body>
<canvas id="canvas" width="150" height="150"></canvas>
<script src="draw.js">
</script>
</body>
</html>

and the following JavaScript (in draw.js):

var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
ctx.fillStyle = "red"; 
ctx.fillRect(75,75,60,60);
ctx.clearRect(105,150,20,20);
ctx.strokeStyle = "red";
ctx.lineWidth = 3; 
ctx.strokeRect(95, 80, 20, 50);

The last piece needed before cutting loose and making some cool stuff is a little helper function added to the Javascript that will return a random integer between x and y inclusively:

function randomIntBetween(min,max) {
   return Math.floor(Math.random()*(max-min+1)+min);
}

Enough to get started… let’s make some art with rectangles using fills, strokes, colors and coordinates and random properties. Here’s what I built. Comments in the code should explain what was done. If you (i.e. that one guy or gal that tripped across this page while searching for blue arctic foxes) decide to make something, please post a link in the comments, which goes for all the following parts of this journey as well.

Playing with Rectangles

Left click on the canvas to generate a new pattern. Right click and save to save the image. Click Open Controls to change the parameters. (Don’t click the Rerun button that appears from CodePen unless you want to reset all the parameters.)

See the Pen Canvas 1.0 by Kentskyo (@kentskyo) on CodePen.0

the strangstein

or “skein rock” in German.

found in an excavation. I’m not at liberty to say where, but it was NOT the Vordarian Beltway. Looks to be slightly used. The glyphs are still running.

25 Bitcoins or best offer.

Cannot ship due to localized temporal distortions created by the artifact. Must arrange for pick-up. Earth sector: 37°14’31.1″N 115°49’08.9″W

Click on image for real time video feed of device in operation.

alien artifact

Temple Bells

and sines of madness.

In the last two projects sound was “sculpted” with subtractive synthesis —using some basic patterns and tools for filtering frequencies and modulating amplitudes. It was more like working with clay. Molding and pinching. This next approach constructs sound from individual sine waves.

Sine waves are the “atoms” of sound. They are pure frequencies, simple vibrations.

If you want to find the secrets of the universe, think in terms of energy, frequency and vibration. -Nikola Tesla

Sines hold an intimate relationship with right triangles and circles. This makes some intuitive sense as circles are the quintessential symbol of periodic time. The seasons, a watch hand, the cycles of life the rotation of the planets around the sun. A sine itself is the relation between two sides of a right triangle (the hypotenuse and opposite angles), and when this relationship is plotted around the circumference of a circle it produces a wave, beautifully illustrated here.

The cosine in the illustration is the relation between another two sides of the triangle:  the hypotenuse and adjacent angles. The final formula completing the foundations of trignometry is the tangent, or relation between the last unpaired sides of the triangle, the adjacent and the opposite.

Sine and Cosine can be used to plot a rotation around a circle (Sine for the y value and Cosine for the x) and they also underlie important building blocks of nature, like the spiral, which can be created by plotting these same points around an expanding or contracting radius. (So, what’s a tangent good for? With a clinometer app or apparatus, it can determine the height of something that you can see the top of, given the angle to the top and the distance from the base of the thing.)

In the 19th century, Joseph Fourier demonstrated that sines and cosines formed building blocks of all periodic phenomenon. A map for everything that happens in waves or vibrations can be expressed as a composition of these simple relationships. Even gnarly waves. His theorem was crucial to the development of a number of sciences: optics, signal processing, information theory, quantum mechanics and more. If all matter can be expressed as particles and/or waves, as is currently believed, it makes sense such a theorem would have broad application.   1

To get a better feel for how simple sines and cosines can under-wire more complex phenomenon, this project uses a handful of pure sine waves to compose the sound of a bell.

The first pass at this was taking an existing bell sound and looking at its spectrogram, which uses Fourier analysis to break down the frequencies of the sound:

Spectral Display of a Bell Sound

A spectrogram reveals frequencies that are more active over time. It’s indispensable for understanding sound and can be found as a tool in the open-source Audacity program, Adobe’s Audition or the standalone and multiplatform Sonic Visualizer.

From first glance at view above, the bell sound appears to be broken down into a handful of dominant frequencies. In reconstructing sound, the lowest frequency on the bottom of the graph is called the fundamental frequency and the frequencies above it are called overtones and relate to the fundamental frequency in ratios, either harmonically (multiples of the fundamental) or in more complex and fractional ways.

The fundamental frequency of the bell appears to be in the range of 500-700hz, and the overtones at the 1400 and 2100 and 2600 spectra. The 1400 overtone is about the same duration as the fundamental but with slightly less amplitude and the next two, 2100 and 2600 are a bit shorter in duration with quite a bit less amplitude.

The first, simple tool constructed to experiment with additive synthesis generates 4 simultaneous sine waves of selectable frequencies and each has envelope controls for duration and amplitude (as well as attack and decay for a little more control over the shape of the sound volume.) Playing with values within the fundamental and overtone spectra, and emulating their amplitudes with envelopes, can simulate a variety of sounds. Playing with similar ratios to the sample can even create a “family” of bell sounds.

The bell tool can be found here. A short video on how to use it and some results is here. The tool was built with HTML5, Javascript and Web Audio using pixi.js to write the graphics and tone.js to put together the audio graphs.

“Profound study of nature is the most fertile source of mathematical discoveries” -Joseph Fourier

References

Trigonometric Delights by Eli Maor

Notes:

  1. We don’t even have time to get into another amazing application of sines and cosines called Euler’s formula, which physicist Richard Feynman called the “jewel” and “the most remarkable formula in mathematics.”