summaryrefslogtreecommitdiff
path: root/stackoverflow-posts
diff options
context:
space:
mode:
authorDaniel Weipert <git@mail.dweipert.de>2024-10-19 10:04:14 +0200
committerDaniel Weipert <git@mail.dweipert.de>2024-10-19 10:04:14 +0200
commitac93ed4d29dd85409fb4c0cd9c2af266e90777c1 (patch)
tree6a0b4487439bfe068d4b7d412be70d4f90faa2a5 /stackoverflow-posts
initial commitHEADmain
Diffstat (limited to 'stackoverflow-posts')
-rw-r--r--stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md320
1 files changed, 320 insertions, 0 deletions
diff --git a/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md b/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md
new file mode 100644
index 0000000..aca7af4
--- /dev/null
+++ b/stackoverflow-posts/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment.md
@@ -0,0 +1,320 @@
+I can't get bazel to build an android app when everything is setup in my local environment with `nix-shell`.
+
+When I try to build, I get the following error:
+
+```sh
+$ bazel build //app/java:dnsproxy
+ERROR: [...]/WORKSPACE:12:23: fetching android_sdk_repository rule //external:androidsdk: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager.
+ERROR: Analysis of target '//app/java:dnsproxy' failed; build aborted: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager.
+INFO: Elapsed time: 0.169s
+INFO: 0 processes.
+FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
+```
+
+The `sdkmanager` tells me, that the correct versions are installed.
+
+```sh
+$ sdkmanager --list_installed
+[=======================================] 100% Fetch remote repository...
+Installed packages:
+ Path | Version | Description | Location
+ ------- | ------- | ------- | -------
+ build-tools;33.0.1 | 33.0.1 | Android SDK Build-Tools 33.0.1 | build-tools/33.0.1
+ cmdline-tools;8.0 | 8.0 | Android SDK Command-line Tools | cmdline-tools/8.0
+ patcher;v4 | 1 | SDK Patch Applier v4 | patcher/v4
+ platform-tools | 33.0.3 | Android SDK Platform-Tools | platform-tools
+ platforms;android-31 | 1 | Android SDK Platform 31 | platforms/android-31
+```
+
+The necessary environment variables are set and everything seems fine.
+
+```sh
+$ echo $ANDROID_HOME
+/nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk
+
+$ which sdkmanager
+/nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/bin/sdkmanager
+```
+
+The paths to everything are there.
+
+```sh
+$ ls -lah /nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk/build-tools
+Permissions Size User Date Modified Name
+lrwxrwxrwx 101 root 1 1月 1970 33.0.1 -> /nix/store/ycc7k5p19ih0ky3pj0zviqk6mkghzgn7-build-tools-33.0.1/libexec/android-sdk/build-tools/33.0.1
+
+$ ls -lah /nix/store/vgcmyvfyqb7kz505r1x1n14lbma2f7ri-androidsdk/libexec/android-sdk/platforms
+Permissions Size User Date Modified Name
+lrwxrwxrwx 97 root 1 1月 1970 android-31 -> /nix/store/20bc5fca6vkgyddhjm7l68ams9ccqr6y-platforms-31/libexec/android-sdk/platforms/android-31
+```
+
+I found this Pull Request: https://github.com/bazelbuild/bazel/pull/12626
+There was an issue with symlink detection in bazel causing the exact problem I'm having right now,
+but that has been fixed for some time now.
+
+Following are the `.nix`, and bazel files I try to build with.
+
+**shell.nix**
+```nix
+with (import <nixpkgs> {
+ config.allowUnfree = true;
+ config.android_sdk.accept_license = true;
+});
+let
+ jdk = jdk11;
+ android-composition = androidenv.composeAndroidPackages {
+ platformVersions = ["31"];
+ };
+in
+mkShell rec {
+ name = "android-shell";
+
+ buildInputs = [
+ jdk
+ android-composition.androidsdk
+ bazel
+ ];
+
+ JAVA_HOME = jdk.home;
+ ANDROID_SDK_ROOT = "${android-composition.androidsdk}/libexec/android-sdk";
+ ANDROID_HOME = "${ANDROID_SDK_ROOT}";
+}
+```
+
+**WORKSPACE**
+```starlark
+# Load the Android build rules
+load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
+http_archive(
+ name = "build_bazel_rules_android",
+ urls = ["https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip"],
+ sha256 = "cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806",
+ strip_prefix = "rules_android-0.1.1",
+)
+
+# Configure Android SDK Path
+load("@build_bazel_rules_android//android:rules.bzl", "android_sdk_repository")
+android_sdk_repository(
+ name = "androidsdk",
+)
+```
+
+**app/java/BUILD**
+```starlark
+load("@build_bazel_rules_android//android:rules.bzl", "android_library", "android_binary")
+
+android_library(
+ name = "main",
+ srcs = glob(["src/*.java"]),
+ manifest = "AndroidManifest.xml",
+ resource_files = glob(["res/**"]),
+)
+
+android_binary(
+ name = "dnsproxy",
+ manifest = "AndroidManifest.xml",
+ deps = [":main"],
+)
+```
+
+**versions**
+```sh
+$ nix --version
+nix (Nix) 2.15.0
+
+$ nix-channel --list
+nixpkgs https://nixos.org/channels/nixpkgs-unstable
+
+$ bazel --version
+bazel 6.1.2- (@non-git)
+
+$ uname -a
+Linux archbook 6.2.13-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 26 Apr 2023 20:50:14 +0000 x86_64 GNU/Linux
+```
+
+I tried a host of different approaches I found scattered around the internet, like [`buildFHSUserEnv`](https://nixos.wiki/wiki/Android#Building_Android_on_NixOS), but I get the exact same error there.
+
+I don't know whether it's still a bazel issue, or a nix package issue or something else entirely and I can't think of anything else that I could check or try to get this to work.
+
+
+PS:
+
+Response to [comment](https://stackoverflow.com/questions/76142628/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment#comment134299657_76142628):
+
+`strace` returns the same environment variables with the same paths as above.
+
+
+Response to [comment](https://stackoverflow.com/questions/76142628/building-an-android-app-with-bazel-setup-in-a-nix-shell-environment#comment134311204_76142628):
+
+I examined the mentioned links and came to the conclusion that I was using an [old guide](https://developer.android.com/codelabs/bazel-android-intro) from https://developer.android.com. Loading the android build rules from https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip doesn't seem to be necessary anymore.
+[This](https://bazel.build/start/android-app) seems to be a more recent guide from https://bazel.build, so the new file contents are
+
+**WORKSPACE**
+```starlark
+android_sdk_repository(
+ name = "androidsdk",
+)
+```
+
+**app/java/BUILD**
+```starlark
+android_library(
+ name = "main",
+ srcs = glob(["src/*.java"]),
+ manifest = "AndroidManifest.xml",
+ resource_files = glob(["res/**"]),
+)
+
+android_binary(
+ name = "dnsproxy",
+ manifest = "AndroidManifest.xml",
+ deps = [":main"],
+)
+```
+
+But I'm still getting the same error, that no build tools are installed.
+
+```sh
+ERROR: [...]/WORKSPACE:1:23: fetching android_sdk_repository rule //external:androidsdk: Bazel requires Android build tools version 30.0.0 or newer but none are installed. Please install a recent version through the Android SDK manager.
+```
+
+I'm still not sure whether this is more a nix or a bazel issue.
+
+
+PPS:
+
+I tried copying and symlinking a `30.0.3` build-tools folder to the `/nix/store` path that my `$ANDROID_HOME` reports. Copying the directory there resulted in a new error message, so it seems like it found the build-tools finally. Symlinking it produced the same error about the missing build-tools as before.
+
+The new error with `bazel v6` is:
+
+```sh
+$ bazel build //app/java:dnsproxy
+Starting local Bazel server and connecting to it...
+INFO: Analyzed target //app/java:dnsproxy (57 packages loaded, 942 targets configured).
+INFO: Found 1 target...
+ERROR: [...]/app/java/BUILD:1:16: Desugaring app/java/libmain.jar for Android failed: Worker process returned an unparseable WorkResponse!
+
+Did you try to print something to stdout? Workers arent allowed to do this, as it breaks the protocol between Bazel and the worker process.
+
+---8<---8<--- Start of response ---8<---8<---
+#
+# There is insufficient memory for the Java Runtime Environment to continue.
+# Native memory allocation (mmap) failed to map 8589934592 bytes for committing reserved memory.
+# An error report file with more information is saved as:
+# [...]/.cache/bazel/_bazel_dweipert/139e0de675811190a7e9267b6c7ce3c8/execroot/__main__/hs_err_pid128444.log
+---8<---8<--- End of response ---8<---8<---
+
+---8<---8<--- Exception details ---8<---8<---
+com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length.
+ at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:107)
+ at com.google.protobuf.CodedInputStream$StreamDecoder.readRawBytesSlowPathOneChunk(CodedInputStream.java:2970)
+ at com.google.protobuf.CodedInputStream$StreamDecoder.readBytesSlowPath(CodedInputStream.java:3021)
+ at com.google.protobuf.CodedInputStream$StreamDecoder.readBytes(CodedInputStream.java:2432)
+ at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:514)
+ at com.google.protobuf.GeneratedMessageV3$Builder.parseUnknownField(GeneratedMessageV3.java:863)
+ at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$Builder.mergeFrom(WorkerProtocol.java:3037)
+ at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$1.parsePartialFrom(WorkerProtocol.java:3333)
+ at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse$1.parsePartialFrom(WorkerProtocol.java:3325)
+ at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:215)
+ at com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:255)
+ at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:267)
+ at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:272)
+ at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:48)
+ at com.google.protobuf.GeneratedMessageV3.parseDelimitedWithIOException(GeneratedMessageV3.java:382)
+ at com.google.devtools.build.lib.worker.WorkerProtocol$WorkResponse.parseDelimitedFrom(WorkerProtocol.java:2810)
+ at com.google.devtools.build.lib.worker.ProtoWorkerProtocol.getResponse(ProtoWorkerProtocol.java:46)
+ at com.google.devtools.build.lib.worker.SingleplexWorker.getResponse(SingleplexWorker.java:141)
+ at com.google.devtools.build.lib.worker.WorkerSpawnRunner.executeRequest(WorkerSpawnRunner.java:715)
+ at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorkerClassic(WorkerSpawnRunner.java:575)
+ at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:359)
+ at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:205)
+ at com.google.devtools.build.lib.exec.SpawnRunner.execAsync(SpawnRunner.java:301)
+ at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:152)
+ at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:112)
+ at com.google.devtools.build.lib.actions.SpawnStrategy.beginExecution(SpawnStrategy.java:47)
+ at com.google.devtools.build.lib.exec.SpawnStrategyResolver.beginExecution(SpawnStrategyResolver.java:64)
+ at com.google.devtools.build.lib.analysis.actions.SpawnAction.beginExecution(SpawnAction.java:352)
+ at com.google.devtools.build.lib.actions.Action.execute(Action.java:133)
+ at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$5.execute(SkyframeActionExecutor.java:957)
+ at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:1124)
+ at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:1082)
+ at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:160)
+ at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:93)
+ at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:516)
+ at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:827)
+ at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.computeInternal(ActionExecutionFunction.java:323)
+ at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:161)
+ at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:571)
+ at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:382)
+ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
+ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
+ at java.base/java.lang.Thread.run(Thread.java:829)
+---8<---8<--- End of exception details ---8<---8<---
+
+---8<---8<--- Start of log, file at [...]/.cache/bazel/_bazel_dweipert/139e0de675811190a7e9267b6c7ce3c8/bazel-workers/worker-1-Desugar.log ---8<---8<---
+OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000600000000, 8589934592, 0) failed; error='Not enough space' (errno=12)
+---8<---8<--- End of log ---8<---8<---
+Target //app/java:dnsproxy failed to build
+Use --verbose_failures to see the command lines of failed build steps.
+INFO: Elapsed time: 12.429s, Critical Path: 2.73s
+INFO: 17 processes: 12 internal, 5 linux-sandbox.
+FAILED: Build did NOT complete successfully
+```
+
+I tried older bazel versions in-between when trying to figure out what works and what not,
+so I tried that here as well. With `bazel v5` I get a different error:
+
+```sh
+$ bazel build //app/java:dnsproxy
+Starting local Bazel server and connecting to it...
+INFO: Analyzed target //app/java:dnsproxy (57 packages loaded, 1164 targets configured).
+INFO: Found 1 target...
+ERROR: [...]/app/java/BUILD:8:15: Building deploy jar app/java/dnsproxy_deploy.jar failed: (Exit 1): singlejar_cc_bin failed: error executing command bazel-out/k8-opt-exec-2B5CBBC6/bin/external/remote_java_tools/singlejar_cc_bin @bazel-out/k8-fastbuild/bin/app/java/dnsproxy_deploy.jar-0.params
+
+Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
+singlejar_cc_bin: external/remote_java_tools/java_tools/src/tools/singlejar/singlejar_main.cc:27: Desugar checking not currently supported in Bazel.
+Target //app/java:dnsproxy failed to build
+Use --verbose_failures to see the command lines of failed build steps.
+INFO: Elapsed time: 17.899s, Critical Path: 7.33s
+INFO: 40 processes: 13 internal, 26 linux-sandbox, 1 worker.
+FAILED: Build did NOT complete successfully
+```
+
+With `bazel v4` I actually get a working environment and I could run `bazel build` without errors.
+
+With `bazel v4` I get a similar error to before, it's just now complaining about the platforms not being installed.
+Here it's the same with the build-tools. The platforms are there, but they are symlinks.
+So I tried copying instead of symlinking a directory there and that then actually ran without errors.
+
+**bazel v4 with error**
+```sh
+$ bazel build //app/java:dnsproxy
+ERROR: Error fetching repository: [...]/WORKSPACE:1:23: android_sdk_repository requires that at least one Android SDK Platform is installed in the Android SDK. Please install an Android SDK Platform through the Android SDK manager.
+ERROR: Analysis of target '//app/java:dnsproxy' failed; build aborted: android_sdk_repository requires that at least one Android SDK Platform is installed in the Android SDK. Please install an Android SDK Platform through the Android SDK manager.
+INFO: Elapsed time: 0.183s
+INFO: 0 processes.
+FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
+```
+
+**bazel v4 without error**
+```sh
+$ bazel build //app/java:dnsproxy
+Starting local Bazel server and connecting to it...
+INFO: Analyzed target //app/java:dnsproxy (35 packages loaded, 631 targets configured).
+INFO: Found 1 target...
+Target //app/java:dnsproxy up-to-date:
+ bazel-bin/app/java/dnsproxy_deploy.jar
+ bazel-bin/app/java/dnsproxy_unsigned.apk
+ bazel-bin/app/java/dnsproxy.apk
+INFO: Elapsed time: 9.581s, Critical Path: 1.51s
+INFO: 16 processes: 13 internal, 3 linux-sandbox.
+INFO: Build completed successfully, 16 total actions
+```
+
+I tried to investigate the version differences and noticed for the pull request I mentioned above (https://github.com/bazelbuild/bazel/pull/12626), that it only concerns the `platforms` and not the `build-tools`.
+Under tag `6.1.2` I found this line: https://github.com/bazelbuild/bazel/blob/6.1.2/src/main/java/com/google/devtools/build/lib/bazel/rules/android/AndroidSdkRepositoryFunction.java#LL376C68-L376C68
+Here the `Dirent.Type.SYMLINK` check is missing, so it makes sense that I got that error.
+
+The OOM issue with `v6` I found here: https://github.com/bazelbuild/bazel/issues/17665
+
+So it looks like I have to bring that up in the [bazel issues](https://github.com/bazelbuild/bazel/issues).