RE: Speeding up ATLAS build time

["Staelin, Carl" <staelin@exch.hpl.hp.com>]

Clint,

W/r to the speedup, and why 2*ncpu instead of cpu:
The speedup generally comes solely from overlapping
I/O with processing.  On older machines, compilation
generally caused lots of I/O, due both to reading 
the source code and (especially) headers, and to the
reading and writing of temporary files.  Compiler
writers have generally moved away from using temp 
files to using pipes to move data between compiler
modules, and file system buffers are usually large
enough to contain at least most of the source files
in memory.  So, the speedup due to overlapping I/O
and CPU has been slowly diminishing over the years.
The flip side is the potential slowdown due to 
increased resource contention, context switching,
and cache pollution.  In my experience, these
costs are usually minimal, and are often outweighed
by the gain from overlapped I/O and CPU utilization.

W/r to the dependencies, you are right, I didn't
think hard enough about that aspect of the problem.
I will see if I can rectify the problem with a
slightly different solution.

Cheers,

Carl


> -----Original Message-----
> From: R Clint Whaley [mailto:rwhaley@cs.utk.edu]
> Sent: Thursday, March 08, 2001 6:51 PM
> To: staelin@exch.hpl.hp.com
> Cc: atlas-comm@cs.utk.edu
> Subject: RE: Speeding up ATLAS build time
> 
> 
> Carl,
> 
> Just a few boneheaded questions . . .
> 
> First, it looks like -j does not work with SunOS or AIX or 
> IRIX make.  Can
> anyone with some knowledge of these guys comment (I don't 
> have access to
> their man pages at present)?  Since gnu make is often used on 
> these systems,
> it's not a killer, but surely they have an equivalent?
> 
> >"make -j N" it is set to "make -j n" where "n"
> >is 2 * ncpu.
> 
> Why is it 2*ncpu rather than ncpu?  Is -j 2 more efficient 
> than no -j on
> a serial machine?
> 
> >Then I went through all the makefiles in "makes"
> >and edited them to use $(PMAKE) to build object
> >files, usually by moving the object files from
> >the dependency line to a $(PMAKE) line in the
> >target build section.
> 
> But the dependency information is not lost, right?  For 
> instance, it is
> still true that already-built libs will not be rearchived, 
> and that missing
> files will be compiled, yes?
> 
> Sorry to send questions rather than just scoping the stuff, 
> but I'm afraid
> time constrainsts mean that I will probably want to scope it 
> out once you
> have the whole thing working and debugged, just to avoid 
> spending the time
> several times, even though I realize the process would be 
> helped by doing
> it iteratively . . .
> 
> Thanks,
> Clint
>