Wednesday, May 21, 2008

California here we come

Mono Lake, 2001, webduke pics

Indirection #2

There is a quotation from a British Computer Scientist, David John Wheeler, I do really like and where I feel a strong need to post this again: “Any problem in computer science can be solved with another layer of indirection. But that usually will create another problem." It is not new and we all know about the meaning behind it but it is still relevant today and tomorrow, and it describes the dilemma of toooo many “genericcontainerwrappingadaptergatewayframeworkolalasolutions” perfectly.

Sunday, May 18, 2008

Some Clouds over Dresden

River Elbe Valley, Mid of May 2008

Thursday, May 15, 2008

Limitations of today’s Parallelization Approaches

As already stated in detail, physics has stopped the race for more CPU clock speed. It’s about the heat, period. Hardware (Chip) manufactures response is to put more and more cores on a die – we all call this multi-core. I also tried to list and explain a couple of technologies (and also No-Go’s) to exploit those architectures; just skim through the blog. But most of the approaches are based on the process of determining sections of code which can be executed in parallel, loops for instance. Saying “sections of codes” does imply that we can expect serial remnants in the source code. And these parts [The percentage of code resisting tenaciously to be parallelized! :-)] determine the speedup factor defined by Amdahl’s Law (see post on Amdahl's Law). The dilemma looks a little bit similar to the expectations from the increase of clock speed in the past. It will not work out to lean back and wait until there are 16 or 32 cores on a single die. This will not fix the problem sufficiently because most of the existing approaches for parallelization does not scale. Other technologies and strategies have to be developed and applied. And before this will happen, it’s always a great approach to optimize the source code and to think about an effective usage of the cash (memory).

Thursday, May 08, 2008

Beautiful Dresden in May

Amdahl’s Law

There is not just Moore’s Law and Murphy’s Law, both are well known to IT and Non-IT folks, but also Amdahl’s Law which is about the maximum of expected improvement to a system when only parts of the systems are improved. It is applied in Parallel Computing commonly, and specifies the theoretical speedup of a program running on multiple CPU’s in parallel. The maximum speedup is limited by the sequential parts of the code. A boundary condition is the assumptions that the problem size is the same when running in parallel. Here comes the formula (with N = number of processors and P the percentage of code that can be parallelized; hence, (1-P) is the serial code):

Speedup = 1 / (1-P) + P/N

You can play around with this formula. In essence, the sequential part of the code must be minimized to gain speedup. And, even a high number of processors does not have that impact when the percentage of parallel code is low. Anyhow, it’s a formula. In practice, the speedup depends also on other conditions, the communication (mechanisms) between the processors and the cache [Because Cache is king! ;-)].

Monday, May 05, 2008