DFHack on the Mac 1: Build Process

I have decided to see if I can get the incredibly useful dfhack utilities for Dwarf Fortress ported to the Mac. Currently, they only exist for Windows and Linux. I created my own fork of the dfhack project on github, and prepared to follow the building instructions.

Downloading cmake was pretty easy; they have a convenient-to-install binary package for the Mac.

Unfortunately, I ran into errors before I even started compiling. Cmake was giving errors about my gcc’s hash map being broken. I’m working on my laptop, which is running OS X 10.6.8 (“Snow Leopard”). I know it’s a version behind (10.7 a.k.a. “Lion” is out now), but with 10.8 (“Mountain Lion”) just around the corner, I’m not going to bother with the Lion upgrade now. The version of gcc included with XCode 3.2 is:

yuzu:~/src/dfhack jonathan$ gcc –version
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I installed a new gcc from source. The process was quite straightforward, and I am not going to document it here.

yuzu:~/src/dfhack jonathan$ /usr/local/bin/gcc –version
gcc (GCC) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

And I told cmake to use that. At that point, I was able to get the Makefiles generated.

From a previous Mac port attempt, there are some `_DARWIN` defines that control which headers are included. The cmake scripts are setting `_LINUX` by default, which is breaking on some Linux-specific includes. The first place I’m hitting that is here:

Scanning dependencies of target clsocket
[ 47%] Building CXX object depends/clsocket/CMakeFiles/clsocket.dir/src/SimpleSocket.cpp.o
In file included from /Users/jonathan/src/dfhack/depends/clsocket/src/SimpleSocket.cpp:43:0:
/Users/jonathan/src/dfhack/depends/clsocket/src/SimpleSocket.h:61:29: fatal error: linux/if_packet.h: No such file or directory
compilation terminated.

I edited the `CMakeLists.txt` files in the dfhack root and the `depends/clsocket` directory so that if `APPLE` is defined, to add the `-D_DARWIN` option instead of `-D_LINUX` (diff). I have not committed the changes to `clsocket` anywhere, as I didn’t feel like forking the project just for that one change, plus I’m not entirely confident about how git submodules work.

The next error was the lack of a perl module:

Scanning dependencies of target generate_headers
[ 52%] Generating ../../library/include/df/codegen.out.xml
Can’t locate XML/LibXSLT.pm in @INC (@INC contains: xml /Library/Perl/Updates/5.10.0 /System/Library/Perl/5.10.0/darwin-thread-multi-2level /System/Library/Perl/5.10.0 /Library/Perl/5.10.0/darwin-thread-multi-2level /Library/Perl/5.10.0 /Network/Library/Perl/5.10.0/darwin-thread-multi-2level /Network/Library/Perl/5.10.0 /Network/Library/Perl /System/Library/Perl/Extras/5.10.0/darwin-thread-multi-2level /System/Library/Perl/Extras/5.10.0 .) at xml/codegen.pl line 15.
BEGIN failed–compilation aborted at xml/codegen.pl line 15.

I used CPAN to install it.

Next, a missing C library function:

[ 56%] Building CXX object library/CMakeFiles/dfhack.dir/LuaTypes.cpp.o
/Users/jonathan/src/dfhack/library/LuaTypes.cpp: In function ‘void read_field(lua_State*, const DFHack::struct_field_info*, void*)’:
/Users/jonathan/src/dfhack/library/LuaTypes.cpp:463:55: error: ‘strnlen’ was not declared in this scope

This function exists on Windows and Linux, but not OS X. It is basically the same as `strlen()`, but takes a maximum length parameter, and will not return a value larger than that. I replaced it with a call to `strlen()` and a manual comparison with the max value (diff). The only way this could backfire is if the string being measured here isn’t NULL-terminated in the case of max length.

Next up comes a template error. I had forgotten how annoying template errors were.

[ 60%] Building CXX object library/CMakeFiles/dfhack.dir/modules/Gui.cpp.o
/Users/jonathan/src/dfhack/library/modules/Gui.cpp: In function ‘void DFHack::Gui::showAnnouncement(std::string, int, bool)’:
/Users/jonathan/src/dfhack/library/modules/Gui.cpp:451:48: error: no matching function for call to ‘min(std::basic_string::size_type, unsigned int)’
/Users/jonathan/src/dfhack/library/modules/Gui.cpp:451:48: note: candidates are:
/usr/local/lib/gcc/x86_64-apple-darwin10.8.0/4.6.3/../../../../include/c++/4.6.3/bits/stl_algobase.h:187:5: note: template const _Tp& std::min(const _Tp&, const _Tp&)
/usr/local/lib/gcc/x86_64-apple-darwin10.8.0/4.6.3/../../../../include/c++/4.6.3/bits/stl_algobase.h:233:5: note: template const _Tp& std::min(const _Tp&, const _Tp&, _Compare)
/usr/local/lib/gcc/x86_64-apple-darwin10.8.0/4.6.3/../../../../include/c++/4.6.3/bits/stl_algo.h:4184:5: note: template _Tp std::min(std::initializer_list<_Tp>)
/usr/local/lib/gcc/x86_64-apple-darwin10.8.0/4.6.3/../../../../include/c++/4.6.3/bits/stl_algo.h:4189:5: note: template _Tp std::min(std::initializer_list<_Tp>, _Compare)

This is the offending line:

int size = std::min(message.size(), 73U);

I changed it to `73UL` (diff), but I will possibly break it on another system as a result. I will need to revisit this.

There was a missing macro:

[ 58%] Building CXX object library/CMakeFiles/dfhack.dir/Console-linux.cpp.o
/Users/jonathan/src/dfhack/library/Console-linux.cpp: In member function ‘bool DFHack::Private::read_char(unsigned char&)’:
/Users/jonathan/src/dfhack/library/Console-linux.cpp:151:13: error: ‘TEMP_FAILURE_RETRY’ was not declared in this scope

This macro is used under glibc to automatically retry system calls that were interrupted and returned `EINTR`. The BSD behaviour is to automatically retry, and I believe that this includes Mac OS X. As such, I added an `#ifdef _DARWIN` and defined a macro that just executes the desired function once and does nothing funny (diff).

Then I ran into an error of the linker including an unnecessary library that did not exist:

[ 52%] Building CXX object library/CMakeFiles/dfhack.dir/Console-linux.cpp.o
Linking CXX shared library libdfhack.dylib
ld: library not found for -lrt

This is the library that, on Linux, provides access to various “realtime” APIs. If I exclude this library, I am not getting any undefined symbol errors, so for the time being I will assume it is either unused, or the relevant functions are already included in another library on OS X. The dependency is defined in `library/CMakeLists.txt`; I added an `IF(APPLE)` section where it defines the `PROJECT_LIBS` variable (diff).

Another somewhat inscrutable C++ error:

/Users/jonathan/src/dfhack/plugins/liquids.cpp:81:15: error: ‘std::string setmode’ redeclared as different kind of symbol
/usr/include/unistd.h:575:7: error: previous declaration of ‘void* setmode(const char*)’
/Users/jonathan/src/dfhack/plugins/liquids.cpp: In function ‘DFHack::command_result df_liquids(DFHack::color_ostream&, std::vector >&)’:
/Users/jonathan/src/dfhack/plugins/liquids.cpp:257:23: error: assignment of function ‘void* setmode(const char*)’
/Users/jonathan/src/dfhack/plugins/liquids.cpp:257:23: error: cannot convert ‘const char [3]’ to ‘void*(const char*)’ in assignment
/Users/jonathan/src/dfhack/plugins/liquids.cpp:261:23: error: assignment of function ‘void* setmode(const char*)’
/Users/jonathan/src/dfhack/plugins/liquids.cpp:261:23: error: cannot convert ‘const char [3]’ to ‘void*(const char*)’ in assignment
/Users/jonathan/src/dfhack/plugins/liquids.cpp:265:23: error: assignment of function ‘void* setmode(const char*)’
/Users/jonathan/src/dfhack/plugins/liquids.cpp:265:23: error: cannot convert ‘const char [3]’ to ‘void*(const char*)’ in assignment
/Users/jonathan/src/dfhack/plugins/liquids.cpp: In function ‘DFHack::command_result df_liquids_execute(DFHack::color_ostream&)’:
/Users/jonathan/src/dfhack/plugins/liquids.cpp:460:35: error: comparison between distinct pointer types ‘void* (*)(const char*)’ and ‘const char*’ lacks a cast [-fpermissive]
/Users/jonathan/src/dfhack/plugins/liquids.cpp:464:40: error: comparison between distinct pointer types ‘void* (*)(const char*)’ and ‘const char*’ lacks a cast [-fpermissive]
/Users/jonathan/src/dfhack/plugins/liquids.cpp:469:40: error: comparison between distinct pointer types ‘void* (*)(const char*)’ and ‘const char*’ lacks a cast [-fpermissive]

It appears that a file-local static variable named `setmode` is interfering with the function `std::string::setmode()`, thanks to the `using std::string` declaration at the head of the file. Unfortunately, I see no way around this other than not `using` that namespace (and having to adjust every intentional usage of a string function) or renaming the `setmode` variable. I opted for the latter, replacing it with `set_mode` (diff).

And with that, the build was complete.

I am, however, slightly worried about these warnings, which I received many times:

In file included from /Users/jonathan/src/dfhack/library/include/df/map_block.h:17:0,
from /Users/jonathan/src/dfhack/library/include/modules/Maps.h:42,
from /Users/jonathan/src/dfhack/plugins/lair.cpp:9:
/Users/jonathan/src/dfhack/library/include/df/tile_designation.h:19:38: warning: ‘df::tile_designation::::dig’ is too small to hold all values of ‘enum df::enums::tile_dig_designation::tile_dig_designation’ [enabled by default]
/Users/jonathan/src/dfhack/library/include/df/tile_designation.h:27:37: warning: ‘df::tile_designation::::liquid_type’ is too small to hold all values of ‘enum df::enums::tile_liquid::tile_liquid’ [enabled by default]
/Users/jonathan/src/dfhack/library/include/df/tile_designation.h:30:34: warning: ‘df::tile_designation::::traffic’ is too small to hold all values of ‘enum df::enums::tile_traffic::tile_traffic’ [enabled by default]

At this point, though, I will try and get the library preloading working and see what happens. I expect there will be a variety of interesting debugging to do.