Advent of Code Day 20: a little bit of (re)vision

Coder’s log, AoC-stardate 2019.40. With help of the tractor beam we have managed to escape the security perimeter of the planet called Neptune. We have landed on Sol IX, nearby a donut-shaped structure. It appears to have a maze-like interior that we need to cross….

Except that I am terribly behind on collecting stars for the previous days, not to mention that our financial controller just mailed: “enter your expenses regarding 2019 as soon as possible“ . Time for… more procrastination, and a small look back and forth.

As an evening person, for me Advent of Code never has been about solving the daily puzzles as fast as possible. Without speed as main factor there’s more room for exploration and learning. And for some self-imposed constraints for extra fun.

When I first joined AoC in 2016 I came to the (embarassing?) conclusion that I no longer had a “real” go-to language: in my daily platform and DevOps engineering work I mainly used Bash. But that wouldn’t work, would it? Well AoC proved me wrong! Almost all puzzles were solvable and I learned some neat tricks along the way. See my earlier post from 2017 for a summary. Henceforth I am known as “Bashtiaan Bakker” (on our Xebia Slack at least).

Other languages like Haskell and Go followed, and to tease some of my collegueas, a bit of Brainf*ck and Whitespace.

Since my youngest son started programming in Scratch, I had to give that a try as well.

Also the awesome open-source game engine Godot was a nice alternative to ASCII art for visualization.

But best of all were the discussions with colleagues, friends and seeing their solutions and those of worldwide participants.

A new year, a new opportunity to use AoC to learn and explore, this time the Google Cloud Platform. First I played with Google Cloud DataFlow / Apache Beam. E.g., star 2:

with beam.Pipeline(options=pipeline_options) as p:
    def fuel(f):
        x = int(f)
        while x > 0:
            x =  max(0, math.floor(x / 3) - 2)
            yield x
    texts = (p
        | 'Read'  >> ReadFromText(known_args.input)
        | 'Fuel'  >> beam.FlatMap(fuel)
        | 'Sum'   >> beam.CombineGlobally(sum)
        | 'Write' >> WriteToText(known_args.output)
    )

A nice property is that you can run the same code locally, but also highly parallel on automatically provisioned and scaled nodes managed by Cloud DataFlow, neat!

After that, the sequential nature of most of the IntCode based puzzles made parallelizations more difficult.

Then, Day 11 featured a different kind of challenge, which already featured in Edo’s blog: the code generates a bitmap depicting some text. Then we apply a wetware OCR, our brain, to obtain the actual solution.

Now wouldn’t it be nice if our code would perform the OCR itself? As it happens the GCP Machine Learning Vision API provides exactly what we need. First let’s create an actual image, instead of ASCII art. With the Python Pillow library this is easy:

# dict panels contains all painted panels...
from PIL import Image
img = Image.new('RGB', (x_size, y_size), 'black')
pixels = img.load()
for p in panels.keys():
    if panels[p] == 1:
        pixels[p[0],p[1]] = (255, 255, 255)
img.save("star23.png")

Now all we have to do is to submit it to GCP for text detection. As a ‘Basher’ I choose to do this via the gcloud command line tool:

$ gcloud ml vision detect-text star23.png

Wow, this returns a lot of information!

{
  "responses": [
    {
      "fullTextAnnotation": {
        "pages": [
...
        ]
      },
      "textAnnotations": [
        {
...
        },
        {
          "boundingPoly": {
            "vertices": [
              {
                "x": 1
              },
              {
                "x": 39
              },
              {
                "x": 39,
                "y": 5
              },
              {
                "x": 1,
                "y": 5
              }
            ]
          },
          "description": "BLULZJLZ"
        }
      ]
    }
  ]
}

We are only interested in the “description”. With a little bit of jq we get to our answer:

$ gcloud ml vision detect-text star23.png | \
   jq -r '.responses[0].textAnnotations[].description' | sort -u
BLULZJLZ

Great, that’s solved! Back to work now, all those parking receipt are waiting to be entered in the expense system 🙁

Hmmm… I just need the date and amount from them. Well, the right date and the right amount. Ah, the newest date and the highest amount should be OK. Back to Bash again!

So, here it is, my highly sophisticated receipt2json.sh script. Let’s wait for another time to turn it into a nice Google Cloud Function. First, where are all those missing stars…?

#!/usr/bin/env bash
textfile=$(mktemp)
gcloud ml vision detect-text "$1" | jq -r '.responses[0].textAnnotations[].description' > $textfile
amount=$(egrep -o '\b[0-9]+,[0-9]{2}\b' $textfile | sort -n | tail -1 | tr , .)
date=$(egrep -o '[0-9]{2}-[0-9]{2}-(20)?(19|20)' $textfile | sort | tail -1)
rm $textfile
cat <<EOF
{ 'amount': $amount, 'date': '$date' }
EOF