distcc: a fast, free distributed C/C++ compiler
Speed, it seems to me, provides the one genuinely modern pleasure.
— Aldous Huxley
distcc is a program to distribute builds of C, C++, Objective C or Objective C++ code across several machines on a network. distcc should always generate the same results as a local build, is simple to install and use, and is usually much faster than a local compile.
distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed. They can even have different processors or operating systems, if cross-compilers are installed.
Just wanted to drop you a line to say that we are now using distcc at work and it is excellent. We have a rack of opteron machines that we use for computational tasks and we are now using them as a compile farm to compile our ~1MLOC C++ tree (which can take an hour on a single CPU to recompile if we change certain header files). We tried using Sun's grid engine to do this (we already use it to schedule our computational jobs), but the combination of its polled operation and the overhead of NFS led to little improvement (and greatly stressed our network). With distcc compile times are way down and my productivity has greatly improved; the two best features for me are its low impact on the network and its simplicity.
Thanks a lot for a great tool! — Jeremy Barnes
60-second instructions:
- For each machine, download distcc
unpack, and do
./configure && make && sudo make install - On each of the servers, run distccd --daemon, with --allow options to restrict access.
- Put the names of the servers in your environment:
export DISTCC_POTENTIAL_HOSTS='localhost red green blue'
- Build!
Wrap your build command in the "pump" script and use "distcc" as your C compiler:
cd ~/work/myproject; pump make -j8 CC=distcc
For more detailed installation instructions, see the INSTALL file.
distcc is developed on GNU/Linux, but has been reported to work on other systems including FreeBSD, NetBSD, Darwin, Solaris, HP-UX, IRIX, Cygwin and BSD/OS.
distcc sends the complete preprocessed source code across the network for each job, so all it requires of the volunteer machines is that they be running the distccd daemon, and that they have an appropriate compiler installed.
distcc is not itself a compiler, but rather a front-end to the GNU C/C++ compiler (gcc) and LLVM compiler (clang). (There is preliminary support for some other compilers but the main focus is gcc.) Almost all gcc options and features work as normal.
distcc is designed to be used with the -j parallel-build feature in GNU Make or SCons, or other build tools. Shipping files across the network takes time, but few cycles on the client machine. Any files that can be built remotely are essentially "for free" in terms of client CPU.
distcc has been under development since early 2002. It reliably and successfully compiles large, complex free and proprietary software systems. Programs known to build correctly with distcc include the Linux kernel, rsync, KDE, GNOME (via GARNOME), Samba and Wireshark.
distcc is nearly linearly scalable for small numbers of machines: Building Linux 2.4.19 on a single 1700MHz Pentium IV machine with distcc 0.15 takes 6 minutes, 45 seconds. Using distcc across three such machines on a 100Mbps switch takes only 2 minutes, 30 seconds: 2.6x faster. The (unreachable) theoretical maximum speedup is 3.0x, so in this case distcc scales with 89% efficiency.
You don't need a lot of machines to benefit: a laptop and a single desktop is much faster than a laptop alone.
There are several tools available separately that enhance or complement distcc:
- ccache caches compiler output to accelerate builds.
- dmucs helps distcc select appropriate servers.
- ccontrol gives centralized control of parallelism, caching and distribution even while the build is running.
- crosstool automates building cross-compilation toolchains.
- distccWebView is a CGI to show which of your servers are up and running.
- tcpbalance is a load-balancing TCP proxy written in Erlang that works well with distcc.