Let it snow!

Square Cash makes gift giving a little more festive.

Reddit
LinkedIn

Written by Bob Lee.

If you sent or received Square Cash recently, you might have noticed slight precipitation backdropping our email headers. It all started last week when we decided to do something special for the holidays. I reckon my teammates thought I was joking when I shouted out with glee, “falling snowflakes!” But then I followed up with a working prototype. What says “Happy Holidays!” better than parallax-scrolling, alpha-composited, bokeh snowflakes, amirite?

While the effect is simple, generating it was anything but. Hacking together a proof-of-concept took moments. Profiling and optimizing the rendering speed and file size required orders of magnitude more effort. We didn’t draw this animation in Photoshop. I wrote a couple hundred lines of Java code instead. The challenges were many, but we Square engineers pride ourselves on our ability to push the envelope at every layer of the stack, up to and including email animations.

Most email clients don’t support HTML5. We’re stuck with 90s era technologies, or in this case, animated GIFs. Obviously the animations should look crisp and beautiful, especially on today’s high resolution displays. We render them at 2X resolution — more than 300k pixels per frame. The animations include custom text. This requires us to render them on the fly and poses some fun challenges. Once rendered, the animation needs to download and play quickly, even over slow mobile networks.

To start, I pored over the 25-year-old GIF spec, reading it backwards and forwards, looking for opportunities to cut down the file size without compromising the design. I used pngquant’s best-of-breed Median Cut quantization algorithm to precompute an optimal 16-color palette. Restricting the animation to 16 colors enabled us to encode the images using 4 bits-per-pixel1. It also resulted in slight posterization, promoting color repetition and further improving compression, particularly in the gradient at the bottom of the image. I even tried plotting only the pixels that changed between each frame, but to my surprise, the diffs resulted in more complexity, less repetition, and therefore worse compression than the full frames.

The biggest win came from an optical illusion aimed at reducing the total number of frames. The animation looks like it necessitates 240 frames, but it really only requires 60 (a 75% savings). The animation is composed of four layers of snowflakes, each meant to appear a different distance from the observer. As the layers get further away, the snowflakes get smaller, lighter, slower, and denser, resulting in the illusion of depth. Here are the separate layers ordered closest to furthest:

The trick is only the closest layer travels the full height of the frame. The lower layers travel only a fraction of the height, but they tile and repeat, giving the illusion of continuity. For example, during the course of the animation, the second layer travels only half way down the height of the image, but it repeats twice and looks like it scrolls continuously. The third layer travels one third of the way and repeats three times, and the fourth layer travels one fourth of the way and repeats four times. As you can see, this repetition becomes less evident when you layer the snowflakes on top of each other and animate them at different speeds.

The final animation clocks in at less than 650KB — 0.25 bits/pixel! To see it in action, send some Cash to a loved one. And check out the final code below. Consider it my gift to you! Happy holidays.

1 */**
  2 * * Copyright 2013 Square Inc.*
  3 * **
  4 * * Licensed under the Apache License, Version 2.0 (the "License");*
  5 * * you may not use this file except in compliance with the License.*
  6 * * You may obtain a copy of the License at*
  7 * **
  8 * *     [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)*
  9 * **
 10 * * Unless required by applicable law or agreed to in writing, software*
 11 * * distributed under the License is distributed on an "AS IS" BASIS,*
 12 * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.*
 13 * * See the License for the specific language governing permissions and*
 14 * * limitations under the License.*
 15 * */*
 16 package com**.**squareup**.**franklin**.**email**.**image**;**
 17 
 18 import com.google.common.cache.CacheBuilder**;**
 19 import com.google.common.cache.CacheLoader**;**
 20 import com.google.common.cache.LoadingCache**;**
 21 import com.google.common.io.ByteSource**;**
 22 import com.google.common.io.ByteStreams**;**
 23 import com.google.common.io.Files**;**
 24 import com.squareup.common.locale.ISOCurrency**;**
 25 import com.squareup.common.values.Money**;**
 26 import java.awt.Color**;**
 27 import java.awt.GradientPaint**;**
 28 import java.awt.Graphics2D**;**
 29 import java.awt.Paint**;**
 30 import java.awt.RenderingHints**;**
 31 import java.awt.image.BufferedImage**;**
 32 import java.io.File**;**
 33 import java.io.IOException**;**
 34 import java.util.ArrayList**;**
 35 import java.util.List**;**
 36 import java.util.Random**;**
 37 import javax.imageio.IIOImage**;**
 38 import javax.imageio.ImageIO**;**
 39 import javax.imageio.ImageTypeSpecifier**;**
 40 import javax.imageio.ImageWriteParam**;**
 41 import javax.imageio.ImageWriter**;**
 42 import javax.imageio.metadata.IIOInvalidTreeException**;**
 43 import javax.imageio.metadata.IIOMetadata**;**
 44 import javax.imageio.metadata.IIOMetadataNode**;**
 45 import javax.imageio.stream.ImageOutputStream**;**
 46 import javax.imageio.stream.MemoryCacheImageOutputStream**;**
 47 import org.apache.commons.io.output.ByteArrayOutputStream**;**
 48 import org.w3c.dom.Node**;**
 49 
 50 */***
 51 * * Generates animated header images for the holidays.*
 52 * **
 53 * * @author Bob Lee*
 54 * */*
 55 **public** **class** **HolidayHeader** **{**
 56 
 57   */**
 58 *   * Docs for Java's GIF encoder:*
 59 *   **
 60 *   * - Plugin notes: [http://docs.oracle.com/javase/6/docs/api/javax/imageio/package-summary.html#gif_plugin_notes](http://docs.oracle.com/javase/6/docs/api/javax/imageio/package-summary.html#gif_plugin_notes)*
 61 *   * - Metadata spec: [http://docs.oracle.com/javase/6/docs/api/javax/imageio/metadata/doc-files/gif_metadata.html#gif_stream_metadata_format](http://docs.oracle.com/javase/6/docs/api/javax/imageio/metadata/doc-files/gif_metadata.html#gif_stream_metadata_format)*
 62 *   **
 63 *   * To inspect GIF file structure:*
 64 *   **
 65 *   * - Daktari: [http://interglacial.com/pub/daktari_gif.html](http://interglacial.com/pub/daktari_gif.html)*
 66 *   */*
 67 
 68   **private** **static** **final** **int** FRAMES_PER_SECOND **=** 24**;**
 69 
 70   */** Random seed used when positioning flakes. */*
 71   **private** **static** **final** **int** SEED **=** 6**;**
 72 
 73   */** Number of flake layers. */*
 74   **private** **static** **final** **int** LAYERS **=** 4**;**
 75 
 76   */** Number of flakes in the top layer. */*
 77   **private** **static** **final** **int** FLAKES_IN_TOP_LAYER **=** 8**;**
 78 
 79   **private** **static** **final** **int** MAX_FLAKE_RADIUS **=** radiusFor**(**LAYERS **-** 1**);**
 80   **private** **static** **final** **int** HEIGHT_WITH_PADDING **=** HeaderImage**.**HEIGHT **+** MAX_FLAKE_RADIUS ***** 2**;**
 81 
 82   **private** **static** **final** HeaderImage**.**Color COLOR **=** HeaderImage**.**Color**.**RED**;**
 83 
 84   */** Constants used for gradient at the bottom of the header. */*
 85   **private** **static** **final** Color TRANSPARENT_BACKGROUND_COLOR **=**
 86       **new** **Color(**COLOR**.**backgroundColor**().**getRGB**()** **&** 0x00ffffff**,** **true);**
 87   **private** **static** **final** **int** GRADIENT_HEIGHT **=** 200**;**
 88   **private** **static** **final** GradientPaint GRADIENT **=** **new** **GradientPaint(**
 89       0**,** HeaderImage**.**HEIGHT **-** GRADIENT_HEIGHT**,** TRANSPARENT_BACKGROUND_COLOR**,** 0**,**
 90       HeaderImage**.**HEIGHT**,** COLOR**.**backgroundColor**());**
 91 
 92   */** Change in y per frame for top layer of flakes. */*
 93   **private** **static** **final** **int** DY **=** 10**;**
 94 
 95   */** The number of frames required for the highest layer of flakes to cycle once. */*
 96   **private** **static** **final** **int** FRAME_COUNT **=** HEIGHT_WITH_PADDING **/** DY**;**
 97 
 98   */** Frames without the amount overlaid. */*
 99   **private** **static** **final** List**<**BufferedImage**>** FRAMES **=** **new** ArrayList**<>();**
100 
101   **static** **{**
102     **for** **(int** frameIndex **=** 0**;** frameIndex **<** FRAME_COUNT**;** frameIndex**++)** **{**
103       Random random **=** **new** **Random(**SEED**);**
104 
105       *// Start with a blank red frame.*
106       BufferedImage frame **=** newBuffer**();**
107       Graphics2D g **=** frame**.**createGraphics**();**
108       g**.**setRenderingHint**(**RenderingHints**.**KEY_ANTIALIASING**,** RenderingHints**.**VALUE_ANTIALIAS_ON**);**
109       g**.**setColor**(**COLOR**.**backgroundColor**());**
110       g**.**fillRect**(**0**,** 0**,** HeaderImage**.**WIDTH**,** HeaderImage**.**HEIGHT**);**
111 
112       *// Draw snowflakes in 3D! We draw the snowflakes in layers, starting with the most distant*
113       *// layer. Nearer layers are bigger, faster, brighter, and more sparse, just like real life!*
114       *// To reduce the number of frames–and in turn the file size–lower layers don't scroll the*
115       *// entire height of the image. We tile them vertically so they still scroll continuously.*
116       *// For example, layer 3 moves only 1/3 of the way down during the animation, but we tile it*
117       *// three times so the last frame looks just like the first frame.*
118       **for** **(int** layerIndex **=** 0**;** layerIndex **<** LAYERS**;** layerIndex**++)** **{**
119         *// We add some padding to the top and bottom and draw outside the bounds of the physical*
120         *// image so snowflakes don't magically appear at the top and disappear when they touch the*
121         *// bottom.*
122         g**.**setColor**(new** **Color(**255**,** 255**,** 255**,** **(**layerIndex **+** 2**)** ***** 25**));**
123         **int** tiles **=** LAYERS **-** layerIndex**;**
124         **int** tileHeight **=** HEIGHT_WITH_PADDING **/** tiles**;**
125         **int** radius **=** radiusFor**(**layerIndex**);**
126         **int** diameter **=** radius ***** 2**;**
127         **for** **(int** flakeIndex **=** 0**;** flakeIndex **<** FLAKES_IN_TOP_LAYER**;** flakeIndex**++)** **{**
128           **int** x **=** random**.**nextInt**(**HeaderImage**.**WIDTH**);**
129           **int** y **=** random**.**nextInt**(**tileHeight**);**
130           y **=** y **+** frameIndex ***** DY **/** tiles**;**
131           **for** **(int** copy **=** 0**;** copy **<** tiles**;** copy**++)** **{**
132             g**.**fillOval**(**x **-** radius**,** **(**y **+** copy ***** tileHeight**)** **%** HEIGHT_WITH_PADDING **-** MAX_FLAKE_RADIUS**,**
133                 diameter**,** diameter**);**
134           **}**
135         **}**
136       **}**
137 
138       *// Draw Square logo.*
139       g**.**drawImage**(**HeaderImage**.**LOGO**,** **null,** HeaderImage**.**LOGO_X**,** HeaderImage**.**LOGO_Y**);**
140 
141       *// Fade to red at the bottom.*
142       Paint paint **=** g**.**getPaint**();**
143       g**.**setPaint**(**GRADIENT**);**
144       g**.**fillRect**(**0**,** HeaderImage**.**HEIGHT **-** GRADIENT_HEIGHT**,** HeaderImage**.**WIDTH**,** HeaderImage**.**HEIGHT**);**
145       g**.**setPaint**(**paint**);**
146 
147       FRAMES**.**add**(**frame**);**
148     **}**
149 
150     */**
151 *     * Note: I tried storing the differences between frames instead of the entire frames, but*
152 *     * the resulting file was actually bigger. This is because the diff images are more complex*
153 *     * than the full images and are therefore less compressible.*
154 *     */*
155   **}**
156 
157   **private** **static** BufferedImage **newBuffer()** **{**
158     *// Use 4 bits per pixel (i.e. BINARY, not INDEXED).*
159     **return** **new** **BufferedImage(**HeaderImage**.**WIDTH**,** HeaderImage**.**HEIGHT**,** BufferedImage**.**TYPE_BYTE_BINARY**,**
160         COLOR**.**model**());**
161   **}**
162 
163   **private** **static** **final** IIOMetadata FIRST_FRAME_METADATA **=** newFrameMetadata**(true);**
164   **private** **static** **final** IIOMetadata FRAME_METADATA **=** newFrameMetadata**(false);**
165 
166   **private** **static** ImageWriter **newGifWriter()** **{**
167     **return** ImageIO**.**getImageWritersByFormatName**(**"gif"**).**next**();**
168   **}**
169 
170   **private** **enum** DisposalMethod **{**
171     none**,** doNotDispose**,** restoreToBackgroundColor**,** restoreToPrevious
172   **}**
173 
174   **private** **static** IIOMetadata **newFrameMetadata(boolean** first**)** **{**
175     ImageWriter writer **=** newGifWriter**();**
176     ImageWriteParam iwp **=** writer**.**getDefaultWriteParam**();**
177     IIOMetadata metadata **=** writer**.**getDefaultImageMetadata**(**
178         **new** **ImageTypeSpecifier(**newBuffer**()),** iwp**);**
179     String metaFormat **=** metadata**.**getNativeMetadataFormatName**();**
180     Node root **=** metadata**.**getAsTree**(**metaFormat**);**
181     IIOMetadataNode gce **=** **(**IIOMetadataNode**)** findChild**(**root**,** "GraphicControlExtension"**);**
182     gce**.**setAttribute**(**"userDelay"**,** "FALSE"**);**
183     gce**.**setAttribute**(**"delayTime"**,** String**.**valueOf**(**100 **/** FRAMES_PER_SECOND**));** *// hundredths of a sec.*
184     gce**.**setAttribute**(**"disposalMethod"**,** DisposalMethod**.**doNotDispose**.**name**());**
185     **if** **(**first**)** **{**
186       *// Use the Netscape application extension to enable looping.*
187       IIOMetadataNode aes **=** **new** **IIOMetadataNode(**"ApplicationExtensions"**);**
188       IIOMetadataNode ae **=** **new** **IIOMetadataNode(**"ApplicationExtension"**);**
189       ae**.**setAttribute**(**"applicationID"**,** "NETSCAPE"**);**
190       ae**.**setAttribute**(**"authenticationCode"**,** "2.0"**);**
191       *// Loop infinitely.*
192       ae**.**setUserObject**(new** **byte[]** **{** 0x1**,** 0**,** 0 **});**
193       aes**.**appendChild**(**ae**);**
194       root**.**appendChild**(**aes**);**
195     **}**
196     **try** **{**
197       metadata**.**setFromTree**(**metaFormat**,** root**);**
198     **}** **catch** **(**IIOInvalidTreeException e**)** **{**
199       **throw** **new** **AssertionError(**e**);**
200     **}**
201     **return** metadata**;**
202   **}**
203 
204   **private** **static** Node **findChild(**Node root**,** String name**)** **{**
205     Node child **=** root**.**getFirstChild**();**
206     **while** **(**child **!=** **null)** **{**
207       **if** **(**name**.**equals**(**child**.**getNodeName**()))** **return** child**;**
208       child **=** child**.**getNextSibling**();**
209     **}**
210     **throw** **new** **AssertionError();**
211   **}**
212 
213   **private** **static** **int** **radiusFor(int** layer**)** **{**
214     **return** 8 **+** **(**layer ***** 4**);**
215   **}**
216 
217   **private** **static** **final** LoadingCache**<**Icon**,** ByteSource**>** headersWithIcons **=**
218       CacheBuilder**.**newBuilder**().**build**(new** CacheLoader**<**Icon**,** ByteSource**>()** **{**
219         @Override **public** ByteSource **load(**Icon icon**)** **throws** Exception **{**
220           **return** **withOverlay(**HeaderImage**.**readImage**(**icon**.**fileName**()));**
221         **}**
222       **});**
223 
224   */** Creates a header image with the given icon. */*
225   **public** **static** ByteSource **withIcon(**Icon icon**)** **{**
226     **return** headersWithIcons**.**getUnchecked**(**icon**);**
227   **}**
228 
229   */** This cache will use ~700MB of memory. */*
230   **private** **static** **final** LoadingCache**<**Money**,** ByteSource**>** headersWithAmounts **=**
231       CacheBuilder**.**newBuilder**().**maximumSize**(**1000**).**build**(new** CacheLoader**<**Money**,** ByteSource**>()** **{**
232         @Override **public** ByteSource **load(**Money amount**)** **throws** Exception **{**
233           **return** **withOverlay(**newAmountImage**(**amount**));**
234         **}**
235       **});**
236 
237   */** Creates a header image with the given amount. */*
238   **public** **static** ByteSource **withAmount(**Money amount**)** **{**
239     **return** headersWithAmounts**.**getUnchecked**(**amount**);**
240   **}**
241 
242   */** Creates a header image with the given image overlaid. */*
243   **private** **static** ByteSource **withOverlay(**BufferedImage overlay**)** **{**
244     **try** **{**
245       **int** x **=** **(**HeaderImage**.**WIDTH **/** 2**)** **-** **(**overlay**.**getWidth**()** **/** 2**);**
246       **int** y **=** 200**;**
247 
248       ByteArrayOutputStream bout **=** **new** **ByteArrayOutputStream();**
249       ImageOutputStream iout **=** **new** **MemoryCacheImageOutputStream(**bout**);**
250       ImageWriter writer **=** newGifWriter**();**
251       writer**.**setOutput**(**iout**);**
252       writer**.**prepareWriteSequence**(null);**
253 
254       **boolean** first **=** **true;**
255       BufferedImage buffer **=** newBuffer**();**
256       Graphics2D g **=** buffer**.**createGraphics**();**
257       g**.**setRenderingHint**(**RenderingHints**.**KEY_ANTIALIASING**,** RenderingHints**.**VALUE_ANTIALIAS_ON**);**
258       **for** **(**BufferedImage frame **:** FRAMES**)** **{**
259         g**.**drawImage**(**frame**,** **null,** 0**,** 0**);**
260         g**.**drawImage**(**overlay**,** **null,** x**,** y**);**
261 
262         *// All of our frames share the same color table. Because we explicitly set per-frame*
263         *// metadata without including a local color table, Java won't emit a local color table.*
264         *// Java will automatically use the first frame's color table as the global color table.*
265         IIOImage ii **=** **new** **IIOImage(**buffer**,** **null,** first **?** FIRST_FRAME_METADATA **:** FRAME_METADATA**);**
266         writer**.**writeToSequence**(**ii**,** **null);**
267         first **=** **false;**
268       **}**
269 
270       writer**.**endWriteSequence**();**
271       iout**.**close**();**
272 
273       **return** ByteStreams**.**asByteSource**(**bout**.**toByteArray**());**
274     **}** **catch** **(**IOException e**)** **{**
275       **throw** **new** **AssertionError(**e**);**
276     **}**
277   **}**
278 
279   **private** **static** BufferedImage **newAmountImage(**Money amount**)** **{**
280     Dimensions textDimensions **=** **new** **Dimensions(**440**,** 218**);**
281     BufferedImage amountImage **=** **new** **BufferedImage(**
282         textDimensions**.**width**(),** textDimensions**.**height**(),** BufferedImage**.**TYPE_INT_ARGB**);**
283     Graphics2D g **=** amountImage**.**createGraphics**();**
284     AmountImage**.**drawAmount**(**g**,** 0**,** 0**,** amount**,** textDimensions**,** Color**.**WHITE**);**
285     **return** amountImage**;**
286   **}**
287 **}**

Bob Lee - Profile Reinventing something as fundamental as paths was hard. The solution we came up with may seem obvious, but there were…medium.com


  1. It may sound counterintuitive, but using 4 bit-per-pixel can result in a larger file size than using 8 bits-per-pixel. Which encoding is better depends on which bit width produces more repetition in the bytes, and that depends on the nature of the image. The easiest strategy is to try both and see which results in a smaller file.