[SR-1287] Ubuntu 15.10: lldb packaging setup fails to install on rebuild

[trfiala]

Previous ID	SR-1287
Radar	None
Original Reporter	@trfiala
Type	Bug

Attachment: Download

Environment

Ubuntu 15.10, identical setup to this builder:
https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/

See attached build script I'm using.

Additional Detail from JIRA

Votes	2
Component/s	LLDB for Swift
Labels	Bug, arm, armv7
Assignee	None
Priority	Medium

md5: e934b50837044efd92f26ebf1a02c1c9

relates to:

• SR-1109 Cannot run swift REPL in 2016-03-24-a on Ubuntu 15.10. Missing SwiftGlibc?

Issue Description:

The first build of LLDB via the build steps used by the Ubuntu 15.10 builder will make it through fine. But, if something about any of the build steps fails after the initial lldb build (e.g. missing a package needed in some other part of swift that shows up after the lldb build), then lldb will go through its build steps again. During the rebuild using the packaging flow, lldb will fail to install with a message like this:

CMake Error at scripts/Python/modules/readline/cmake_install.cmake:44 (file):
file INSTALL cannot find
"/home/tfiala/src/lldb-github/build/buildbot_linux/lldb-linux-x86_64/scripts/Python/modules/readline/CMakeFiles/CMakeRelink.dir/readline.so".
Call Stack (most recent call first):
scripts/Python/modules/cmake_install.cmake:37 (include)
scripts/cmake_install.cmake:41 (include)
cmake_install.cmake:42 (include)

Inspection of the related directory tree shows there is no CMakeRelink.dir file:

$ find /home/tfiala/src/lldb-github/build/buildbot_linux/lldb-linux-x86_64/scripts/Python/modules/readline/CMakeFiles
/home/tfiala/src/lldb-github/build/buildbot_linux/lldb-linux-x86_64/scripts/Python/modules/readline/CMakeFiles
/home/tfiala/src/lldb-github/build/buildbot_linux/lldb-linux-x86_64/scripts/Python/modules/readline/CMakeFiles/readline.dir
/home/tfiala/src/lldb-github/build/buildbot_linux/lldb-linux-x86_64/scripts/Python/modules/readline/CMakeFiles/readline.dir/readline.cpp.o

Googling around seems to indicate that relink dir shows up when doing cross compiles, which I don't think should be taking place here. This might be related to the same change that is breaking SR-1109, as it did some changing around of host vs. target details.

Note I only hit this on a rebuild of lldb. (I get a very similar result if llbuild has to rebuild and reinstall).

[swift-ci]

Comment by Joe (JIRA)

@trfiala: Confirmed that I can reproduce this on Ubuntu 14.04. Failure under SR-1109 will result in SR-1287 unless I do a "distclean".

[trfiala]

Thanks for reproducing that, Joe. Glad to see it is at least deterministic across machines.

[trfiala]

This looks to be a potentially known issue with the Ninja generator on cmake. See this bug:
https://cmake.org/Bug/print_bug_page.php?bug_id=13934

[trfiala]

That bug also mentions cross compiling as being an underlying cause. The first thing I'd want to do here is print out whether cmake thinks it is running in a cross-compiler mode, which it should not be in this case. (Building for Ubuntu on Ubuntu).

[swift-ci]

Comment by Rob Allen (JIRA)

Same issue here. Ubuntu 15.10 in a VirtualBox VM (via vagrant), running this build command:

$ swift/utils/build-script --preset=buildbot_linux_1510 installable_package=/home/rob/swift.tar.gz install_destdir=/home/rob/swift-install

I can also confirm that rm -rf build/buildbot_linux and then rerunning the build command worked for me too.

[trfiala]

Looks like other projects have hit something like this as well:
https://bugs.webkit.org/show_bug.cgi?id=144902

[swift-ci]

Comment by Joe (JIRA)

Reproducible as of 5/3:

CMake Error at scripts/Python/modules/readline/cmake_install.cmake:44 (FILE):
  file INSTALL cannot find
  "/mnt/usbms/workspace/Swift-3.0-ARM-Incremental/build/Ninja-ReleaseAssert/lldb-linux-armv7/scripts/Python/modules/readline/CMakeFiles/CMakeRelink.dir/readline.so".
Call Stack (most recent call first):
  scripts/Python/modules/cmake_install.cmake:37 (INCLUDE)
  scripts/cmake_install.cmake:41 (INCLUDE)
  cmake_install.cmake:42 (INCLUDE)

I hate to have to {{rm rf build}} as this is a native ARM compilation. :(

[swift-ci]

Comment by Ryan Lovelett (JIRA)

I've also attempted all of the work arounds in https://cmake.org/Bug/print_bug_page.php?bug_id=13934 and none of them are working for me. Perhaps this is confirmation that CMake does not think it is in cross compile mode.

[rintaro]

It seems, the immediate cause is:
https://cmake.org/Bug/view.php?id=14820

But, the root cause is the fact I didn't know this bug 🙁
I deeply apologize for causing the inconvenience.

Posted a PR swiftlang/swift#2393
that reverts swiftlang/swift#2226

[trfiala]

rlovelett (JIRA User) One way to tell if CMake is in cross-compiling mode is to print out the CMake CMAKE_CROSSCOMPILING flag, which will be set to TRUE when it is cross compiling. (The best concise reference I found for this was the Mastering CMake book, chapter 8, although all the bits are online).

But as I mentioned above, and Rintaro also looks like he discovered, is that there is something in the Ninja generator that is not following an expected contract for cross-compiling, relinking scenarios. The cross compiling might be valid per CMake's definition (contrary to what I mentioned earlier) if we're essentially specifying a different toolchain to use, even if it's targeting the same machine. I just haven't had a chance yet to dissect it and figure out what it is doing.

[trfiala]

Abandoning as I don't really have insight into the cmake/ninja interactions occurring here. Somebody else can feel free to pick this up.

[keith]

We can probably close this as this Ubuntu version is no longer supported (I don't have access to)