distcc testimonialsC lets you shoot yourself in the foot, and distcc gives you a chaingun. For speeding up our builds we use and recommend distcc to allow us to distribute compilation across two or more machines (i.e. a laptop and a desktop). distcc can be set up in no time and if like us you're continuously and repeatedly building lots of code it'll save you hours if not days of waiting around for compiles to finish. Also check out ccache. — The Rosegarden music software developers. Paul Daniels: I'd just like to extend my thanks to the creators (and maintainers ;-) of distcc and associated projects. My projects here are getting rather large in size (relative) such that I have to wait too long between the process of tweak-build-test-*groan* (repeat). 5 mins of distcc across a few unused boxes in this office and everything was much better ... only trouble now is that I no longer have an excuse to walk the hallways and make coffee *curse*. Seriously though, great project, simple to use, generates real results — hallmarks of a indispensable utility. Jens Mayer: distcc rocks. Thank you very much for your excellent work! Alex Landa: It's a scorcher! I originally also had a rather low-memory (24MB) Pentium S-90 in the chain. I removed this because it was swapping and its old hard drive was making a total racket. I'm not sure if it slowed the whole cluster, but it seemed to. Learning which order to put the machines in was something that I got the hang of rather quickly. The Xbox did most of the hard work. Richard Bown: Previously building on desktop or laptop - times around 25-30mins for complete build. Really getting me down, decided I needed to upgrade my hardware. Then after consideration thought why should I be spending more money on hardware when I've already got so much lying around? Quick search failed to find distcc but instead found old (defunct) project called "doozer" - tried, talked to the author, he recommended distcc. Downloaded yesterday, up and running in no time. First attempts:
With main desktop and laptop: 15mins (make -j 4)
Excellent results after some prodding about. 6. Any other observations: Significant improvements with just the two machines (as noted in the distcc web pages) and good enough to stop me from buying anything else for a while. Will continue to play around with configuration settings. Considering putting another (older) machine into the cluster but wonder whether it'll make enough of a difference. Current hard limit to improving times much further is the link stage which can take almost two mins just by itself.
Great piece of software. Many thanks,
Thank you very much for this great tool. Our only regret is not to have used it before. — Guillaume LAURENT, telegraph-road.org Stephane Eranian, hp (linux ia64 maintainer): Yes, distcc is really nice. I have been using it for a month or so for my kernel compiles using a cluster of Itanium 2 machines. What I like is that it is really easy to setup. The danger with the approach is that you can easily overload your local (master) host. As for kernel compiles for 2.5, there are a lot of ld steps which are still all done on the master node. Besides that you can really feel the speed difference. I need to take a look at ccache now that you mention it. Grant Grundler, hp: (Endorsement): Stephane and I played around with just distcc for 2.4 ia64 kernels on a smaller cluster of rx2600 (dual CPU). The "master" runs out of CPU around -j16 (8 hosts). But that resulted in 66 second kernel builds. And distcc really is trivial to setup. Hey Martin- I've only been using distcc for a little while, on and off, and have only been on this mailing list for a month or so... but I've gotta say, you're doing one hell of a nice job. It seems extremeley rare to see one-man projects with devs so open about what they're doing, why they're doing it, and what potential problems might be these days... You're doing a bang up job and distcc totally rocks. Keep up the good work! Rob distcc just rocks for doing regression tests! J.T. Conklin: I'm in the midst of converting the build infrastructure at work from using recursive gmake to Scons, and thought it might be useful to share my experiences. The gmake build scheme we have been using is much like many others I have encountered over the years. A list of subdirectories are built sequentially by iterating over the list and spawning a new make for each. One problem with this approach is that the end of each module/ subdirectory becomes a synchronization point. For example, when there are fewer files to compile (in a given module) than you have processes in your pool, you're not taking full advantage of your build farm. This deficiency can be addressed by rewriting your makefiles to avoid recursion (Cf Peter Miller's paper "Recursive Make Considered Harmful" or by using another tool (like scons, jam, etc.) that constructs a full dependency graph before building. Here's a real-world example that demonstrates the advantages. Our source code is about 600,000 lines of C++, with a small amount of C and assembly thrown in. There are sixteen libraries, a dozen or so executables (including four very large ones), etc.
On a AMD Athlon 2000+, using our old recursive gmake with -j4 (mainly
to hide i/o latency):
On a distcc build farm including the above AMD machine, plus 2 1.0GHz
PIII machines and 2 Dual 1.0GHz PIII machines with gmake -j20:
Our scons build on the single AMD machine with -j4:
And finally, scons -j20 on the distcc build farm:
While distcc alone more than doubled the performance, distcc plus a build scheme that takes full advantage of build parallelism really lets it shine by speeding up the build more than 5 times. distcc was too easy to setup. You should add some obscure dependency so its a bit more difficult to get running. Nice job on a great util!! Only downside is that I am tempted to buy more boxes now ;) — Justin Kirby, openaether.org: distcc is awesome: easy install, no maintenance, works right. It's saved us a lot of time, and the price sure beats that of Trolltech's Teambuilder! I have a Sunblade 100 on my desktop and it takes 4 minutes to compile one project of mine. Using 3 public Sunblade 1000's it now only takes 1 minute. Thank You, Thank You, Thank You. -- Shane McDaniel, MIT |
Send comments to distcc(at)lists.samba.org