creativecommons/quantifying

Swallowing Critical GitHub API Errors (KeyError) in github_fetch.py

Open

#300 opened on Jul 5, 2026

View on GitHub
 (0 comments) (1 reaction) (0 assignees)Python (73 forks)auto 404
help wanted🏁 status: ready for work💻 aspect: code🛠 goal: fix🟩 priority: low

Repository metrics

Stars
 (48 stars)
PR merge metrics
 (PR metrics pending)

Description

Description

In scripts/1-fetch/github_fetch.py, the script fetches public repository counts for various Creative Commons tools. When parsing the GitHub API search response, the code expects to find "total_count" in the returned JSON object: count = search_data["total_count"]

However, if the GitHub API returns an error response (such as "API Rate Limit Exceeded" or "Invalid Token" / "Bad credentials"), "total_count" will not exist in the response payload. As a result, the script encounters a KeyError on "total_count", which is caught by:

        except KeyError as e:
            raise shared.QuantifyingException(f"KeyError: {e}", 1)

This swallows the actual error details returned by the GitHub API (e.g., in the "message" field), making debugging extremely difficult without manually outputting the API response payload.

Reproduction

  1. Run python scripts/1-fetch/github_fetch.py using a rate-limited network or an invalid/expired GH_TOKEN.
  2. The GitHub API returns a JSON response lacking the "total_count" key (e.g., containing "message": "API rate limit exceeded..." or "message": "Bad credentials").
  3. The script throws KeyError: 'total_count', which is converted into QuantifyingException("KeyError: 'total_count'", 1).
  4. Observe the script exiting with a generic KeyError: 'total_count' exception, concealing the actual API rate limiting or authentication message.

Expectation

The script should check for GitHub API error messages (like "message" or other error indicators in the JSON response) before attempting to access "total_count", logging the specific error payload from GitHub to provide clarity on why the request failed.

Contributor guide