bazel-contrib/rules_python

Cannot cross-compile Gazelle extension

Open

#1,913 opened on May 22, 2024

View on GitHub
ย (11 comments)ย (3 reactions)ย (0 assignees)Starlarkย (705 forks)auto 404
gazellehelp wanted

Repository metrics

Stars
ย (679 stars)
PR merge metrics
ย (PR metrics pending)

Description

๐Ÿž bug report

Affected Rule

The issue is caused by the rule: Gazelle extension

Is this a regression?

Yes, before #1895, we were able to cross-compile Gazelle extension by disabling cc toolchain resolution with --noincompatible_enable_cc_toolchain_resolution (#1825).

Description

The cross-compilation of Gazelle extension is broken after #1895, due to the introduction of dependency on a C library .

The issue can be partly mitigated by using hermetic_cc_toolchain, which supports cross-compiling into Linux, but we still cannot cross-compile from Linux into any macOS, or from darwin_arm64 into darwin_amd64.

This is technically not a problem specific to the Gazelle extension, but a problem of the lack of C toolchain support. However, are we better off by replacing a Python dependency with a C dependency?

๐Ÿ”ฌ Minimal Reproduction

From the gazelle directory of this rules_python repo on a Linux or Apple M1 machine:

bazel build --platforms=@io_bazel_rules_go//go/toolchain:darwin_amd64_cgo //python:gazelle_binary

๐Ÿ”ฅ Exception or Error

๐ŸŒ Your Environment

Operating System:

  • Linux
  • macOS

Output of bazel version:

7.1.2

Rules_python version:

730a2e39bd2702910f28629d4583b3ec49f4ee5e

Anything else relevant? We can register a hermetic_cc_toolchain by adding this to MODULE.bazel file:

bazel_dep(name = "hermetic_cc_toolchain", version = "3.1.0")

toolchains = use_extension("@hermetic_cc_toolchain//toolchain:ext.bzl", "toolchains")
use_repo(toolchains, "zig_sdk")

register_toolchains(
    "@zig_sdk//toolchain:linux_amd64_gnu.2.31",
    "@zig_sdk//toolchain:linux_arm64_gnu.2.31",
    "@zig_sdk//toolchain:darwin_arm64",
    "@zig_sdk//toolchain:darwin_amd64",
)

Then from a Linux machine:

$ bazel build --platforms=@zig_sdk//platform:darwin_arm64 //python:gazelle_binary
INFO: Analyzed target //python:gazelle_binary (133 packages loaded, 15547 targets configured).
ERROR: /home/user/go-repos/src/github.com/bazelbuild/rules_python/gazelle/python/BUILD.bazel:92:15: GoLink python/gazelle_binary_/gazelle_binary failed: (Exit 1): builder failed: error executing GoLink command (from target //python:gazelle_binary) bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/go_sdk/builder_reset/builder link -sdk external/go_sdk -installsuffix darwin_arm64 -arc ... (remaining 95 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
external/go_sdk/pkg/tool/linux_amd64/link: running external/hermetic_cc_toolchain~~toolchains~zig_sdk/tools/aarch64-macos-none/c++ failed: exit status 1
error: unable to find framework 'CoreFoundation'. searched paths:  none
error: unable to find framework 'Security'. searched paths:  none

link: error running subcommand external/go_sdk/pkg/tool/linux_amd64/link: exit status 2
Target //python:gazelle_binary failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 16.068s, Critical Path: 14.64s
INFO: 52 processes: 2 internal, 50 processwrapper-sandbox.
ERROR: Build did NOT complete successfully

Contributor guide