Let’s Encrypt & App Engine, pt. 2

*For more background on using *Let’s Encrypt with *Google’s App Engine, read the original post *Setting up https for your website

When it came time to renew my Let’s Encrypt certificates for Google’s App Engine, I wanted to see whether I could improve the process. For my first attempt, I tried out the DNS verification method. I updated the appropriate TXT records for my domain, but either wasn’t able to do it correctly, or wasn’t patient enough for the caches to refresh fully, so I went back to good ol’ https verification instead. This ended up being a good choice when I decided to add more automation to the process, and now you can benefit my from my work 🙂.

Configuration file

If you have several different projects on the same machine, the first thing you might want to do is get a handle on creating a separation of concerns for each project. An easy start is adding a folder to your project to help maintain all the relevant items for each.

project/
--...
--letsencrypt/
----logs/
----config/
----work/

Each folder can be used as a way to check in different aspects of the Let’s Encrypt process into my repository. We also want to start associating our projects with a configuration file, instead of remembering all of the different flags and commands needed. Initially my configuration file looked like this:

# Flags that we are using certbot on a different computer than the host.
manual 

# Tells certbot where we want to store the ouput of respective processes
config-dir = ./config
logs-dir = ./logs
work-dir = ./work

#Specifies the domains that we want to create certificates for.
domains = example.com,www.example.com

#Uses the http request-response authentication method.
preferred-challenges = http

#create a new certificate, even if this one isn't expired already
force-renewal 

#The following options are mainly to prevent unessecary interactions. 
email = [email protected]
agree-tos 
manual-public-ip-logging-ok

Now we can runcertbot --config ./cli.ini certonly and go straight to the step to upload the challenge responses to my server. Unfortunately, all that work doesn’t quite make the process of renewing certificates any easier just makes the command a little shorter. Luckily certbot has some new tricks up its sleeves for that.

Automation with validation hooks

Certbot has a couple new features that help automate the manual certificate request process on a machine different than the one serving your website. Those features are the --manual-auth-hook and --manual-cleanup-hook flags. They allow you to specify shell scripts that will be run with pertinent details included as environment variables. You can read all about the Certbot hooks in the official user guide.

Because we’re set on using a config file for everything let’s add a couple lines for my new pre & post validation hooks.

#pre & post validation hook:
manual-auth-hook ../scripts/auth.sh
manual-cleanup-hook ../scripts/cleanup.sh

auth.sh

Each of those scripts is pretty simple. The auth script will add the appropriate file to serve to meet the url challenge (& deploy my application), and the cleanup script will delete the files that get made. Let’s look at auth.sh first:

echo "writing file to ../letsencrypt/$CERTBOT_TOKEN.txt"
echo $CERTBOT_VALIDATION > ../letsencrypt/${CERTBOT_TOKEN}.txt
gcloud app deploy ../app.yaml --project=xxx --quiet;

The first line just creates a little output that tells me where the file is being created, and the second line places the content of the CERTBOT_VALIDATION environment variable into a file named after the CERBOT_TOKEN environment variable. Then deploy the App Engine application specifying the project. The --quiet flag prevents the need for any confirmations during the process for maximum automation.

app.yaml

It is easy to meet the URL challenges if you have the right file handler set in your app.yaml for the project:

- url: /.well-known/acme-challenge/(.*)
  static_files: letsencrypt/\1.txt
  upload: letsencrypt/(.*\.txt)

For any request that gets sent under yourdomain/.well-known/acme-challenge, App Engine will look for an appropriately named text file and return the contents.

cleanup.sh

`rm ../letsencrypt/*`

After the validation, this script just cleans up the files only so that they don’t end up crufting up your repo.

Uploading the certificate

Once successful, you’ll get a certificate in a .pem file. You’ll need to add that to your certificates page for App Engine PEM encoded X.509 public key certificate. For the Unencrypted PEM encoded RSA private key you’ll first have to modify the private key file. You likely have into an RSA private key which is pretty simple with openssl:

sudo openssl rsa -inform pem -in ./config/live/yourdomain.com/privkey.pem -outform pem > ./config/live/yourdomain.com/rsaprivatekey.pem

Now you can upload your new rsaprivatekey.pem and patently wait another three months until you need to renew again. 😉

I hope you enjoyed reading the second part of my App Engine SSL adventures with Let’s Encrypt! Let me know if you have any thoughts, or suggestions for improvements in the comments.