gleam-lang/gleam

Make a tool for adding canonical links to historical hexdocs

Open

#3,865 opened on Nov 20, 2024

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Rust (960 forks)batch import
help wanted

Repository metrics

Stars
 (21,417 stars)
PR merge metrics
 (Avg merge 10d 19h) (69 merged PRs in 30d)

Description

Search engines are surfing links to outdated versions of package documentation. This is a problem as it misleads users, especially for core libraries such as the stdlib.

If the documentation had canonical links in the head element then search engines would hopefully suggest the current version instead.

<link rel="canonical" href="https://hexdocs.pm/package_name/path/to/file.html" />

https://github.com/gleam-lang/gleam/issues/3864 covers adding the link to future releases, but we need to add it to historical releases too.

Make a Gleam program which given the name of a package:

  • Queries the hex API to get all the release version numbers
  • For each release
    • Downloads the docs tarball for each one (e.g. https://repo.hex.pm/docs/gleam_stdlib-0.17.1.tar.gz)
    • For each HTML page
      • Checks to see if the canonical link is in the HTML head
        • Leaves the HTML as-is if the link is present
        • Adds the link if it was absent
    • Creates a new tarball with the updated content
    • Uploads the new tarball to update the release's documentation

Making a tar library is probably the first step. It should protect against the security vulnerabilities of path traversal attacks and compression bombs.

Contributor guide