How To Forward GitHub Actions Runner Logs

You’re using ephemeral GitHub Actions Runners and an issue occurs. Time to troubleshoot. But, where to look? All evidence is gone already.. Let’s prevent this situation by persisting the runner logs to Cloud Logging.

Prerequisites

To collect and ingest logs, you must install the Ops Agent or the Legacy Logging agent.

Ops Agent Configuration

The Ops Agent is configured through the /etc/google-cloud-ops-agent/config.yaml-file. Merge the logging configuration below to start collecting GitHub Actions Runner logs.

logging:
  receivers:
    github_runner:
      type: files
      include_paths:
      - /your-runner-installation-directory/_diag/Runner_*.log
    github_worker:
      type: files
      include_paths:
      - /your-runner-installation-directory/_diag/Worker_*.log

  processors:
    github:
      type: parse_regex
      regex: "^\[(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2})Z\s+(?<severity>[^\s]+)\s+(?<thread>.*)\] (?<message>.*)"
      time_key: time
      time_format: "%Y-%m-%d %H:%M:%S"

  service:
    pipelines:
      github_runner:
        receivers:
        - github_runner
        processors:
        - github
      github_runner:
        receivers:
        - github_worker
        processors:
        - github

Legacy Logging Agent Configuration

The Legacy Logging Agent is configured through files in the /etc/google-fluentd/config.d/-directory. Create a github.conf-file with the configuration below to start collecting GitHub Actions runner logs.

<source>
  @type tail
  format multiline
  format_firstline /\[\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}Z/
  format1 /^\[(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2})Z\s+(?<severity>[^\s]+)\s+(?<thread>.*)\] (?<message>.*)/

  path /your-runner-installation-directory/_diag/Runner_*.log
  pos_file /var/lib/google-fluentd/pos/github-runner.pos
  read_from_head true
  tag github-runner
</source>

<source>
  @type tail
  format multiline
  format_firstline /\[\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}Z/
  format1 /^\[(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2})Z\s+(?<severity>[^\s]+)\s+(?<thread>.*)\] (?<message>.*)/

  path /your-runner-installation-directory/_diag/Worker_*.log
  pos_file /var/lib/google-fluentd/pos/github-worker.pos
  read_from_head true
  tag github-worker
</source>

Discussion

One might wonder why we still need an Agent to forward logs. Can’t you just direct the logs to stdout? Currenty, no. A feature request is reported on GitHub.

Conclusion

Observability is key in running your ephemeral workloads. With Cloud Logging, you can persist arbitrary log files using a simple configuration file. Happy GitHub Actions runner logging!

Image by PollyDot from Pixabay