diff options
Diffstat (limited to 'stackoverflow-posts')
| -rw-r--r-- | stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md | 320 |
1 files changed, 320 insertions, 0 deletions
diff --git a/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md b/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md new file mode 100644 index 0000000..aca7af4 --- /dev/null +++ b/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md @@ -0,0 +1,320 @@ +I can't get bazel to build an android app when everything is setup in my local environment with `nix-shell`. + +When I try to build, I get the following error: + +```sh +$ bazel build //app/java:dnsproxy +ERROR: [...]/WORKSPACE:12:23: fetching android_sdk_repository rule //external:androidsdk: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager. +ERROR: Analysis of target '//app/java:dnsproxy' failed; build aborted: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager. +INFO: Elapsed time: 0.169s +INFO: 0 processes. +FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured) +``` + +The `sdkmanager` tells me, that the correct versions are installed. + +```sh +$ sdkmanager --list_installed +[=======================================] 100% Fetch remote repository... +Installed packages: + Path | Version | Description | Location + ------- | ------- | ------- | ------- + build-tools;33.0.1 | 33.0.1 | Android SDK Build-Tools 33.0.1 | build-tools/33.0.1 + cmdline-tools;8.0 | 8.0 | Android SDK Command-line Tools | cmdline-tools/8.0 + patcher;v4 | 1 | SDK Patch Applier v4 | patcher/v4 + platform-tools | 33.0.3 | Android SDK Platform-Tools | platform-tools + platforms;android-31 | 1 | Android SDK Platform 31 | platforms/android-31 +``` + +The necessary environment variables are set and everything seems fine. + +```sh +$ echo $ANDROID_HOME +/nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk + +$ which sdkmanager +/nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/bin/sdkmanager +``` + +The paths to everything are there. + +```sh +$ ls -lah /nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk/build-tools +Permissions Size User Date Modified Name +lrwxrwxrwx 101 root 1 1月 1970 33.0.1 -> /nix/store/ycc7k5p19ih0ky3pj0zviqk6mkghzgn7-build-tools-33.0.1/libexec/android-sdk/build-tools/33.0.1 + +$ ls -lah /nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk/platforms +Permissions Size User Date Modified Name +lrwxrwxrwx 97 root 1 1月 1970 android-31 -> /nix/store/20bc5fca6vkgyddhjm7l68ams9ccqr6y-platforms-31/libexec/android-sdk/platforms/android-31 +``` + +I found this Pull Request: https://github.com/bazelbuild/bazel/pull/12626 +There was an issue with symlink detection in bazel causing the exact problem I'm having right now, +but that has been fixed for some time now. + +Following are the `.nix`, and bazel files I try to build with. + +**shell.nix** +```nix +with (import <nixpkgs> { + config.allowUnfree = true; + config.android_sdk.accept_license = true; +}); +let + jdk = jdk11; + android-composition = androidenv.composeAndroidPackages { + platformVersions = ["31"]; + }; +in +mkShell rec { + name = "android-shell"; + + buildInputs = [ + jdk + android-composition.androidsdk + bazel + ]; + + JAVA_HOME = jdk.home; + ANDROID_SDK_ROOT = "${android-composition.androidsdk}/libexec/android-sdk"; + ANDROID_HOME = "${ANDROID_SDK_ROOT}"; +} +``` + +**WORKSPACE** +```starlark +# Load the Android build rules +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +http_archive( + name = "build_bazel_rules_android", + urls = ["https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip"], + sha256 = "cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806", + strip_prefix = "rules_android-0.1.1", +) + +# Configure Android SDK Path +load("@build_bazel_rules_android//android:rules.bzl", "android_sdk_repository") +android_sdk_repository( + name = "androidsdk", +) +``` + +**app/java/BUILD** +```starlark +load("@build_bazel_rules_android//android:rules.bzl", "android_library", "android_binary") + +android_library( + name = "main", + srcs = glob(["src/*.java"]), + manifest = "AndroidManifest.xml", + resource_files = glob(["res/**"]), +) + +android_binary( + name = "dnsproxy", + manifest = "AndroidManifest.xml", + deps = [":main"], +) +``` + +**versions** +```sh +$ nix --version +nix (Nix) 2.15.0 + +$ nix-channel --list +nixpkgs https://nixos.org/channels/nixpkgs-unstable + +$ bazel --version +bazel 6.1.2- (@non-git) + +$ uname -a +Linux archbook 6.2.13-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 26 Apr 2023 20:50:14 +0000 x86_64 GNU/Linux +``` + +I tried a host of different approaches I found scattered around the internet, like [`buildFHSUserEnv`](https://nixos.wiki/wiki/Android#Building_Android_on_NixOS), but I get the exact same error there. + +I don't know whether it's still a bazel issue, or a nix package issue or something else entirely and I can't think of anything else that I could check or try to get this to work. + + +PS: + +Response to [comment](https://stackoverflow.com/questions/76142628/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment#comment134299657_76142628): + +`strace` returns the same environment variables with the same paths as above. + + +Response to [comment](https://stackoverflow.com/questions/76142628/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment#comment134311204_76142628): + +I examined the mentioned links and came to the conclusion that I was using an [old guide](https://developer.android.com/codelabs/bazel-android-intro) from https://developer.android.com. Loading the android build rules from https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip doesn't seem to be necessary anymore. +[This](https://bazel.build/start/android-app) seems to be a more recent guide from https://bazel.build, so the new file contents are + +**WORKSPACE** +```starlark +android_sdk_repository( + name = "androidsdk", +) +``` + +**app/java/BUILD** +```starlark +android_library( + name = "main", + srcs = glob(["src/*.java"]), + manifest = "AndroidManifest.xml", + resource_files = glob(["res/**"]), +) + +android_binary( + name = "dnsproxy", + manifest = "AndroidManifest.xml", + deps = [":main"], +) +``` + +But I'm still getting the same error, that no build tools are installed. + +```sh +ERROR: [...]/WORKSPACE:1:23: fetching android_sdk_repository rule //external:androidsdk: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager. +``` + +I'm still not sure whether this is more a nix or a bazel issue. + + +PPS: + +I tried copying and symlinking a `30.0.3` build-tools folder to the `/nix/store` path that my `$ANDROID_HOME` reports. Copying the directory there resulted in a new error message, so it seems like it found the build-tools finally. Symlinking it produced the same error about the missing build-tools as before. + +The new error with `bazel v6` is: + +```sh +$ bazel build //app/java:dnsproxy +Starting local Bazel server and connecting to it... +INFO: Analyzed target //app/java:dnsproxy (57 packages loaded, 942 targets configured). +INFO: Found 1 target... +ERROR: [...]/app/java/BUILD:1:16: Desugaring app/java/libmain.jar for Android failed: Worker process returned an unparseable WorkResponse! + +Did you try to print something to stdout? Workers arent allowed to do this, as it breaks the protocol between Bazel and the worker process. + +---8<---8<--- Start of response ---8<---8<--- +# +# There is insufficient memory for the Java Runtime Environment to continue. +# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory. +# An error report file with more information is saved as: +# [...]/.cache/bazel/_bazel_dweipert/139e0de675811190a7e9267b6c7ce3c8/execroot/__main__/hs_err_pid128444.log +---8<---8<--- End of response ---8<---8<--- + +---8<---8<--- Exception details ---8<---8<--- +com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. + at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:107) + at com.google.protobuf.CodedInputStream$StreamDecoder.readRawBytesSlowPathOneChunk(CodedInputStream.java:2970) + at com.google.protobuf.CodedInputStream$StreamDecoder.readBytesSlowPath(CodedInputStream.java:3021) + at com.google.protobuf.CodedInputStream$StreamDecoder.readBytes(CodedInputStream.java:2432) + at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:514) + at com.google.protobuf.GeneratedMessageV3$Builder.parseUnknownField(GeneratedMessageV3.java:863) + at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$Builder.mergeFrom(WorkerProtocol.java:3037) + at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$1.parsePartialFrom(WorkerProtocol.java:3333) + at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$1.parsePartialFrom(WorkerProtocol.java:3325) + at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:215) + at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:255) + at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:267) + at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:272) + at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:48) + at com.google.protobuf.GeneratedMessageV3.parseDelimitedWithIOException(GeneratedMessageV3.java:382) + at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse.parseDelimitedFrom(WorkerProtocol.java:2810) + at com.google.devtools.build.lib.worker.ProtoWorkerProtocol.getResponse(ProtoWorkerProtocol.java:46) + at com.google.devtools.build.lib.worker.SingleplexWorker.getResponse(SingleplexWorker.java:141) + at com.google.devtools.build.lib.worker.WorkerSpawnRunner.executeRequest(WorkerSpawnRunner.java:715) + at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorkerClassic(WorkerSpawnRunner.java:575) + at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:359) + at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:205) + at com.google.devtools.build.lib.exec.SpawnRunner.execAsync(SpawnRunner.java:301) + at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:152) + at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:112) + at com.google.devtools.build.lib.actions.SpawnStrategy.beginExecution(SpawnStrategy.java:47) + at com.google.devtools.build.lib.exec.SpawnStrategyResolver.beginExecution(SpawnStrategyResolver.java:64) + at com.google.devtools.build.lib.analysis.actions.SpawnAction.beginExecution(SpawnAction.java:352) + at com.google.devtools.build.lib.actions.Action.execute(Action.java:133) + at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$5.execute(SkyframeActionExecutor.java:957) + at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:1124) + at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:1082) + at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:160) + at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:93) + at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:516) + at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:827) + at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.computeInternal(ActionExecutionFunction.java:323) + at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:161) + at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:571) + at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382) + at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) + at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) + at java.base/java.lang.Thread.run(Thread.java:829) +---8<---8<--- End of exception details ---8<---8<--- + +---8<---8<--- Start of log, file at [...]/.cache/bazel/_bazel_dweipert/139e0de675811190a7e9267b6c7ce3c8/bazel-workers/worker-1-Desugar.log ---8<---8<--- +OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000600000000, 8589934592, 0) failed; error='Not enough space' (errno=12) +---8<---8<--- End of log ---8<---8<--- +Target //app/java:dnsproxy failed to build +Use --verbose_failures to see the command lines of failed build steps. +INFO: Elapsed time: 12.429s, Critical Path: 2.73s +INFO: 17 processes: 12 internal, 5 linux-sandbox. +FAILED: Build did NOT complete successfully +``` + +I tried older bazel versions in-between when trying to figure out what works and what not, +so I tried that here as well. With `bazel v5` I get a different error: + +```sh +$ bazel build //app/java:dnsproxy +Starting local Bazel server and connecting to it... +INFO: Analyzed target //app/java:dnsproxy (57 packages loaded, 1164 targets configured). +INFO: Found 1 target... +ERROR: [...]/app/java/BUILD:8:15: Building deploy jar app/java/dnsproxy_deploy.jar failed: (Exit 1): singlejar_cc_bin failed: error executing command bazel-out/k8-opt-exec-2B5CBBC6/bin/external/remote_java_tools/singlejar_cc_bin @bazel-out/k8-fastbuild/bin/app/java/dnsproxy_deploy.jar-0.params + +Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging +singlejar_cc_bin: external/remote_java_tools/java_tools/src/tools/singlejar/singlejar_main.cc:27: Desugar checking not currently supported in Bazel. +Target //app/java:dnsproxy failed to build +Use --verbose_failures to see the command lines of failed build steps. +INFO: Elapsed time: 17.899s, Critical Path: 7.33s +INFO: 40 processes: 13 internal, 26 linux-sandbox, 1 worker. +FAILED: Build did NOT complete successfully +``` + +With `bazel v4` I actually get a working environment and I could run `bazel build` without errors. + +With `bazel v4` I get a similar error to before, it's just now complaining about the platforms not being installed. +Here it's the same with the build-tools. The platforms are there, but they are symlinks. +So I tried copying instead of symlinking a directory there and that then actually ran without errors. + +**bazel v4 with error** +```sh +$ bazel build //app/java:dnsproxy +ERROR: Error fetching repository: [...]/WORKSPACE:1:23: android_sdk_repository requires that at least one Android SDK Platform is installed in the Android SDK. Please install an Android SDK Platform through the Android SDK manager. +ERROR: Analysis of target '//app/java:dnsproxy' failed; build aborted: android_sdk_repository requires that at least one Android SDK Platform is installed in the Android SDK. Please install an Android SDK Platform through the Android SDK manager. +INFO: Elapsed time: 0.183s +INFO: 0 processes. +FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured) +``` + +**bazel v4 without error** +```sh +$ bazel build //app/java:dnsproxy +Starting local Bazel server and connecting to it... +INFO: Analyzed target //app/java:dnsproxy (35 packages loaded, 631 targets configured). +INFO: Found 1 target... +Target //app/java:dnsproxy up-to-date: + bazel-bin/app/java/dnsproxy_deploy.jar + bazel-bin/app/java/dnsproxy_unsigned.apk + bazel-bin/app/java/dnsproxy.apk +INFO: Elapsed time: 9.581s, Critical Path: 1.51s +INFO: 16 processes: 13 internal, 3 linux-sandbox. +INFO: Build completed successfully, 16 total actions +``` + +I tried to investigate the version differences and noticed for the pull request I mentioned above (https://github.com/bazelbuild/bazel/pull/12626), that it only concerns the `platforms` and not the `build-tools`. +Under tag `6.1.2` I found this line: https://github.com/bazelbuild/bazel/blob/6.1.2/src/main/java/com/google/devtools/build/lib/bazel/rules/android/AndroidSdkRepositoryFunction.java#LL376C68-L376C68 +Here the `Dirent.Type.SYMLINK` check is missing, so it makes sense that I got that error. + +The OOM issue with `v6` I found here: https://github.com/bazelbuild/bazel/issues/17665 + +So it looks like I have to bring that up in the [bazel issues](https://github.com/bazelbuild/bazel/issues). |