Scaling Christmas — An Illustrated Adventure

The lemurs are back! This time they’ve come for a special Christmas-themed adventure about how Christmas scales, using practical lessons in holiday operations.

Now let’s see what they’re up to today.

Meet Red

You may have already met Red some time in the past from other talks and events. For those of you who haven’t had the pleasure, Red is a magical lemur. He’s also particularly fond of Christmas, especially presents.

Every year he sends Santa a letter, like clockwork. Every year, Santa delivers.

This raises a question though: how exactly does Santa do it? How does he scale Christmas?

Let’s journey to the North Pole to see if we can find some answers.

The North Pole

Santa started small with his operation. He’d only need to visit a handful of villages a year, and he was more than able to keep up with that.

As time went by, the demand for Christmas grew, and it was abundantly clear that the current setup just wasn’t going to cut it.

Fast forward a few years, to the modern-day North Pole.

The North Pole of Today

Christmas has gone global, and it shows no sign of stopping. It’s scaled up to incredible numbers, and sustained growth like Santa would have never imagined.

This raises a good question: what changed? How in the world did the lemurs go from barely being able to service a small region to delivering to the entire world?

You see, Santa met someone. Mrs. Claus.

Mrs. Claus: The True Mastermind

Mrs. Claus was the genius behind Christmas growing to its current size. She helped to initiate multi-regional delivery systems, dispatches, and other technologies the likes of which Santa had never seen before.

Her brilliance knew no bound in terms of planning, and Santa’s speed had become a thing of legend. Together they were Christmas.

That is, until one fateful night when Santa got sick.

Santa is Sick!

There are only two days until Christmas, and Santa and his technical leads have come down with nasty, story-convenient colds! With them out of commission, no amount of planning can save Christmas.

Stuck without their leads and without Santa, Mrs. Claus made a call.

Reinforcement

It was time to call them. A legendary band of merry lemurs known throughout the land for their silliness and love for whimsy, the Rainbow Brigade.

Explanations

So Mrs. Claus called Violet, their leader, and explained the situation. She told Violet about Santa getting sick, how the leads were out, and how they desperately needed help to maintain the various Christmas tasks.

“Not to worry,” Violet responded, “I know just the lemurs for the job.”

The Rainbow Brigade

So Violet summoned the members of the Rainbow Brigade, and explained what had happened. They were stunned! No Christmas? That could never come to pass.

The lemurs agreed to help, and so Violet set them to their tasks: Scaling Christmas.

Code Freezes

Mrs. Claus was in charge of code freezes, making sure that no naughty lemur could potentially release a new toy version while they were still making new toys with its template.

This type of effort has to be done at the top of the top to make sure all the lemurs are well aware that there are to be no deploys or changes made across the entire North Pole.

Logging and Monitoring

Violet took on monitoring and logging.

Regionally-aware statistics helped them decide who was naughty and nice, predict capacity requirements, and be able to see data on past delivery rates and requirements. Without monitoring and logging the lemurs would be going in blind, so this was an especially critical item to get right.

With scaling and capacity planning, data is key as it informs decisions and gives basis for comparisons to see what worked and what didn’t.

On Call Rotations

Saffron was in charge of On Call rotations.

He made sure the right lemurs were paged when something caught fire as things always do during the holidays. He also decided whether or not a certain event really needed paging, and helped to perform basic maintenance on services to keep them running smoothly.

He keeps an eye on logs and abnormalities to be the first line of defense against a downed network.

Load Balancing

Vermilion takes care of Elastic Letter Balancing.

She makes sure that all the letters coming in get distributed to the right toy factories. She takes extra care to make sure letters from one region find their way to a toy factory in the same region for the fastest possible turnaround.

If a lemur in the US-West region happens to send in a letter, she makes sure that letter gets to a toy factory in the US-West region. Likewise with Asia, Europe, and other major regions.

Auto Scaling

Red is in charge of Auto Scaling.

Right now the demand for cats is up significantly, meaning more toy makers are needed to find and adopt the right cats from local shelters for all the new and loving families.

Likewise, the demand for footballs isn’t too high at the moment, so Red scales down the amount of toy makers working on footballs.

He makes these decisions based on information he gets from Violet about how many requests are being made.

DDoS Prevention and Points of Presence

Chartreuse is in charge of making sure trolls can’t ruin Christmas using a system called Points of Presents.

The trolls particularly enjoy DDoS attacks, or rather Distributed Denial of Santa attacks.

Each request goes through a regional point first, and if too many trolls are sending letters from that region, Chartreuse can shut it down to make sure that the rest of the delivery system can keep running like normal.

She makes sure to note any good children who may be impacted to give extra special presents to after the other regions are delivered.

Disaster Recovery

Because all the lemurs work together, if things do happen to go down, they have the ability and the information to build everything up from scratch.

You see, they assume anything could go down at any time, so they prepare a way to bring things back up just in case.

The most interesting of times are those in which the entire deployment pipeline goes down. It’s rather hard to deploy fixes when your method of deployment is down.

During the holiday season, it’s safe to assume something will always go down, so it’s best to have a plan of action on how to recover any and every service.

Santa Returns!

But wait! Santa is better! (conveniently right after our lemurs manage to get Christmas running) …and what should he find but a merry band of lemurs making Christmas happen?

Thanks Given

Santa thanks all the lemurs for their hard work in making Christmas happen, because without a good crew taking care of operations, the holidays simply wouldn’t happen.

So this holiday season, give your Ops friends a high five, a hug, or something else they might enjoy because scaling during holidays is a heroic task. The holidays need more heroes, and lucky for us they’re ready to go this season!

But there was one Lemur missing, a particularly chaotic Lemur at that! So Santa had to ask…

“Whatever happened to Indigo and Escher C.A.T.? …and how did you manage to get all the presents delivered on time? I don’t remember hearing that part.”

Delivery and Deus Ex Machina

Who in the world is fast enough to get around the entire world? Why, Indigo and his friend Escher Cat! Escher has the ability to bend space and time, and Indigo managed to rig up a rocket sleigh just for the occasion.

“AHAHAHAHAHAHAHAHA!” screamed Indigo, while flying through the sky with Escher Cat. “We’ll make Christmas happen yet dear friend! Second star to the right and straight on…! Ah, right, wrong fairytale.”

So the duo went, warping in and out of time to make sure all the deliveries were made right on time.

As an aside, Indigo would like you all to consider top hats, he swears Santa will be wearing one next year, and it’ll be all the rage.

Wrapping Up

As always, I hope you’ve enjoyed this magical little Christmas tale.

It should be noted that all of Santa’s helper lemurs are paid fair and equitable wages, and work sustainable hours during the holiday season. No lemurs or rocket sleighs were harmed or injured during this story, but there is a particularly motion sick C.A.T. who’s not keen on doing that again.

To one and to all, happy holidays, whatever your holiday may be, and a happy New Year!