Building open source Swift on ARMv7
Apple recently open sourced their Swift language and compiler. I'm pretty excited about this (I'm a huge Swift fan), and I wanted a project to help motivate me to dig in. I also do a lot of work with embedded linux at work, so I'm eager to use this great language there, too. I found an existing feature request in the issue tracker that Apple set up for swift. I was pleased that they had already given it 'medium' priority, but I know that if you want something to happen sooner than later, you should just start working on it.
I've done enough linux work that I know that if you start deviating from the officially supported distro. you start making work for yourself pretty quickly. Therefore, I decided to limit myself to only ARMv7 systems that have an Ubuntu 14.04 image. In my case that was a BeagleBone Black (BBB) and an Nvidia Tegra TK1. If you can, go with the Nvidia every time (The TX1 has 3GB of RAM). It's much faster, and has many times the BBB's 512MB of RAM. Whatever amount of RAM you have, you'll need to setup some swap space. Also, depending on whether you're using a SD card, you might need to set aside at least 8GB of space to store the source and build products.
utils/build-scrip
To begin with, we need to understand the build process. Apple's documentation makes it plain that you should use the utils/build-script script to build, and that any processes outside of that system aren't supported:
For all automated build environments, this tool is regarded as the only way to build Swift. This is not a technical limitation of the Swift build system. It is a policy decision aimed at making the builds uniform across all environments and easily reproducible by engineers who are not familiar with the details of the setups of other systems or automated environments.
This script is responsible for parsing the arguments presented on the command line, printing help text, making build directories, and a prepares things for utils/build-script-impl. A read through yields no information about architectures at all, so we move on to build-script-impl.
utils/build-script-impl
Most of the real work occurs in build-script-impl. Right away, we're presented with architecture specific parts.
LLVM_TARGETS_TO_BUILD="X86;ARM;AArch64"
This line sets a variable that will eventually tell the LLVM build process (several packages need to build to support Swift; one of those is LLVM, another is Clang) which architectures it should be able to generate machine code for.
This script is a little hard to follow because it's well over 1000 LOC, and there are functions mixed in with top level code. In this section, I'll trace the execution of the file rather than file order. I'll also omit anything that doesn't have a bearing on target/host architecture.
# A list of deployment targets to compile the Swift host tools for, in cases
# where we can run the resulting binaries natively on the build machine.
NATIVE_TOOLS_DEPLOYMENT_TARGETS=()
# A list of deployment targets to cross-compile the Swift host tools for.
# We can't run the resulting binaries on the build machine.
CROSS_TOOLS_DEPLOYMENT_TARGETS=()
# Determine the native deployment target for the build machine, that will be
# used to jumpstart the standard library build when cross-compiling.
This section of code determines what the native platform is, both in terms of OS and architecture. The native tools deployment targets is an array of native targets. This could be something like building for multiple arm targets that (may) be binary compatible, like different hard(or soft)-float versions. The case block matches on the results of uname -s -m, which on most ARM platforms will return something like this: Linux armv7l. You'll almost always see the l after armv7. We don't want to be that specific, so we'll match just the armv7 part. The cross tools deployment targets variable is used to specify the cross compilers that should be built. However, we can't just put linux-armv7 into this list to build a cross compiler, not yet anyway:
# Sanitize the list of cross-compilation targets.
for ; do
done
This code will trap on any entry in that array that isn't in the list above. It would probably be possible to add linux targets there eventually, but for now let's move on. I want a native compiler for ARM, not just a cross compiler. The next thing that we come across is the determination of the platform for which we want the stdlib built for. This case, like the prior one, matches on armv7, and ignores the suffix.
# A list of deployment targets that we compile or cross-compile the
# Swift standard library for.
STDLIB_DEPLOYMENT_TARGETS=()
After skipping lots more functions, we come across the meat of the process.
#
# Configure and build each product
#
# Start with native deployment targets because the resulting tools
# are used during cross-compilation.
for; do
# ~~~ skipped compile option flag management ~~~
# Build.
done
The important thing here is that the build process runs in full for each native target and cross target. At the beginning of the process the set_deployment_target_based_options function runs, and it sets the host variant for Swift. Remember at this point, its target variant, not host variant.
The rest of the script does the work of testing, packaging, and installing.
CMakeLists.txt
The CMakeLists.txt file provides CMake instructions on how to actually build the tools using the flags and options determined by build-script and build-script-impl. I am far less than a novice in all things CMake, so please correct me if I make mistakes in this area, but this is what I 'm able to infer from what's happening.
Within CMakeLists.txt , we have to manually set CMAKE_SYSTEM_PROCESSOR unless the build is cross-compiled, or otherwise already set. In linux (andFreeBSD) it's set to the value of uname -m; it's just set to i386.
# Reset CMAKE_SYSTEM_PROCESSOR if not cross-compiling.
# CMake refuses to use `uname -m` on OS X
# http://public.kitware.com/Bug/view.phpid=10326
Later, we use the value of CMAKE_SYSTEM_PROCESSOR to determine what architecture to build. Remember that if we're not cross-compiling, its value is the host architecture. Also, note that cross-compiling is only available on Darwin (macosx):
# FIXME: separate the notions of SDKs used for compiler tools and target
# binaries.
To make sense of this we really need to dig into the implementation of the configure_sdk_unix macro in SwiftConfigureSDK.cmake, otherwise the arguments are just a bunch of "linux" strings with an arch. and a triple.
This macro sets a number of CMake variables that will be used in both the generation of the swift compiler as well as the standard lib. An interesting thing to note is that the 4th linux should probably be linux-x86_64 and linux-arm.
While building swift files, these variables are ultimately consumed in the cmake/modules/AddSwift file for a variety of tasks, such as finding the path to the libraries, setting sdk and target flags, etc. Not that in this module, what was ${prefix} becomes ${sdk}.
In cases where it's not obvious where control flow leaves you, especially with how CMake seems to make liberal use of globals, I find it helpful to grep for variable names. Doing so for SWIFT_SDK_${ led me to...
stdlib/public/runtime/CMakeLists.txt
Where I found this gem!
This FIXME is a big deal, because I hadn't found it before. I was having linker issues with my binary, and this holds some promise helping address that issue. I'll go ahead and add a check here, copy the 64-bit script and edit it for 32 bit.
The 64-bit ld script is pretty small, and doesn't seem too intimidating (I've never seen an ld script before):
SECTIONS
{
.swift2_protocol_conformances :
{
.swift2_protocol_conformances_start = . ;
QUAD(SIZEOF(.swift2_protocol_conformances) - 8) ;
*(.swift2_protocol_conformances) ;
}
}
INSERT AFTER .dtors
The only problem is that it's far from obvious what needs to change when porting that to a 32-bit system!! My only guess is that 8*8 is 64-bits, so maybe that -8 needs to be a -4! :)
SECTIONS
{
.swift2_protocol_conformances :
{
.swift2_protocol_conformances_start = . ;
QUAD(SIZEOF(.swift2_protocol_conformances) - 4) ;
*(.swift2_protocol_conformances) ;
}
}
INSERT AFTER .dtors
Unfortunately, that got me nowhere. But, I'm getting ahead of myself. Moving on...
stdlib/public/SwiftShims/LibcShims.h
This one is an easy one… All we have to do is make a new typedef for __swift_ssize_t for the 32-bit arm:
typedef int __swift_ssize_t;
typedef long int __swift_ssize_t;
stdlib/public/stubs/Stubs.cpp
Next, we have to deal with the fact that a multiply with overflow is missing from what libgcc provides us. I'll leave this out for brevity, but basically I just copied an implementation from the compiler-rt project.
lib/Driver/ToolChains.cpp
ToolChains.cpp is an interesting one because it is responsible for linking the compiler pieces together when building swift applications. In the original version, the x86_64 path to the swift stdlib was hard-coded in there. Now, we just have to query the target triple, and select the correct path to the library.
- Arguments.;
// ~ becomes ~
+ Arguments.;
Testing files
There were several files that affect the testing suite. This post is already way to long, so I'll leave those out. If you want to see all the changes I've made, checkout the commit:
Summary
This port is getting pretty close. My 32-bit linker script doesn't fix anything, and I'm still looking for other potential problems. But, for now I'm happy with the progress. I hope this helps anyone else exploring other ports. The 32-bit arm port is potentially the simplest possible one, because the older iPhones and iWatch use 32-bit arm chips.
Update 12/16/2015
I was able to fix the linking problem by looking at another exciting swift port (SwiftAndroid), and noticing that -Bsymbolc is being used in the AddSwift cmake module.
I added that to my copy of AddSwift, and it seems to work beautifully. I will do some more testing and clean up, and submit a new pull request.
)