CVE-2022-29810: SSH Key Disclosure in go-getter (Or Is It?)

Published

According to an April 15 advisory from Rancher, a vulnerability in go-getter may “expose the configured SSH private key secret if Fleet fails to download the git repo.” The advisory rates the severity as Low, despite the potential of a complete takeover of any cluster managed by the affected repository if an exposure leaks the key to a malicious user. More interesting, though, is Rancher’s finger pointing at go-getter to the tune of going to MITRE as a CNA of Last Resort to assign a CVE. In my opinion, the vulnerability is not in go-getter but in how Rancher Fleet handles errors returned by go-getter. I am the researcher that disclosed the original vulnerability to Rancher, and this is the story of that disclosure.

The Discovery

While configuring Fleet for a collection of clusters, I fat-fingered a repository path and got an error in the Fleet UI letting me know that I had screwed up. The error was something like, “error downloading ssh://git@example.com/path/to/repo.git?sshkey=ABCDEF0...,” and it was so long that the UI truncated it. As unsurprised as I was that I had typed the wrong path, I was a bit shocked to be looking at what appeared to be a base64-encoded SSH key—presumably the private key configured for Fleet to access the intended git repository. If my concern was correct and Fleet was exposing this key, I needed to remove it from the repository and notify the security team immediately. Unfortunately, the UI had cleared the notification. After a little searching, though, I found the error message in the logs of the fleet agent running on the target cluster and quickly verified that fleet had indeed logged the private SSH key used to authenticate with the git repository hosting gitops deployments.

The Investigation

Having reported the issue internally, I wanted to understand what was happening under the hood, how the SSH key ending up in the agent logs. After some digging, I found where exactly Fleet was constructing the go-getter URL for the repository:

pkg/bundle/resources.go 

340
341
342
343
344
345
346
347

if auth.SSHPrivateKey != nil {
    if strings.IndexAny(c.Src, "?") == -1 {
        c.Src += "?"
    } else {
        c.Src += "&"
    }
    c.Src += fmt.Sprintf("sshkey=%s", base64.StdEncoding.EncodeToString(auth.SSHPrivateKey))
}

Which is quickly shortly followed by the start of the error handling that leads to the disclosure:

pkg/bundle/resources.go 

351
352
353

if err := c.Get(); err != nil {
    return nil, err
}

Here’s what’s happening: go-getter’s API is centered around a source URL to get, with protocol-specific parameters defined as queries in the URL. When fetching a repository using SSH, Fleet appends the private SSH key to the URL (line 346) and then executes Get() (line 351). When go-getter fails to download the git repository, it returns an error string in the format “error downloading URL: ERROR”, and the Fleet code then starts returning the error up the callstack.

The Fleet code has surprisingly little error-handling. It basically just passes the error back through the callstack about a dozen times until it reaches the agent’s entry point, which is actually called by another Rancher codebase called wrangler-cli. The wrangler-cli doesn’t do much other than execute the command context provided by Fleet, and critically, log any error returned to stdout.

In short, this disclosure vulnerability is the result of Fleet passing an error generated by one library up its callstack to a second library that promptly logs it to stdout and exits. The Fleet controller then sees the failed exit and forwards the final log message along until it is ultimately displayed in the Web UI. So why this vulnerability attributed to go-getter?

The Disclosure

I worked with the Rancher security team late last year on CVE-2021-36779 and CVE-2021-36780, and honestly they were great. Awkwardly, though, this time it was different. The Rancher team quickly passed the blame to go-getter for generating the error with the SSH key embedded. But here’s the thing, go-getter isn’t generating the error from the SSH key specifically, it’s generating it from the source URL provided by Fleet, which includes the SSH key. The basic flow is this:

  1. Fleet generates a URL string and embeds the SSH key as a query parameter, something like: ssh://git@example.com/repo.git?sshkey=LS0tLS1C...
  2. Fleet passes string that it generated to go-getter to fetch.
  3. When an error occurs, go-getter generates and returns an error that includes the provided URL string.

As the Rancher team tells it, that flow makes go-getter liable for the disclosure. I had several email exchanges with the Rancher team on the matter, including providing a very detailed breakdown of their own function calls to counter their initial narrative that it was go-getter logging the key and not Fleet (it was actually wrangler-cli) and encourage them to resolve the issue. However, the team remained adamant that the issue was with go-getter and there was nothing that Rancher could do to but wait for go-getter to be updated.

Pointing the Finger

Following my disclosure, Rancher worked with the go-getter developers to update go-getter to redact the sshkey query parameter from errors, which is definitely a good idea, and then updated Fleet. Personally, while I was pleased to see go-getter updated, I still felt that Fleet needed better error handling. Knowing that the raw URL included the SSH key, Fleet should have used its own context to generate an error message known to be free of sensitive data and returned that, which would have the added benefits of being safer and a providing more useful information to administrators. I also continued to push for a Rancher to issue a security advisory.

Ultimately, Rancher took the issue to MITRE as a CNA of Last Resort to publish CVE-2022-29810 as a vulnerability in go-getter. Notably, the CVE references CWE-532: Insertion of Sensitive Information into a Log File, which go-getter does not do. I’m happy that issue was resolved and advisories were issued, but I remain perplexed by Rancher’s behavior and disappointed that the published CVE is based on inaccurate information.