A practical comparision of multitasking libraries

The following code-base attempts to compare multiple libraries on a simple experiment of Algebraic operations.

The results are in favour of certain libraries because they are more natural in he application of Compute bound tasks, whereas the others are better at I/O bound tasks. Threading in Python is considered broken... I've also refrained from using the MPI library of C.

I've made a driver.py program which create 5 sets of test-cases of increasing size to test the excution times of each version of each program.

The comparison is across different libraries and languages, with the following programs:

C:

optimal serial code
openMP directive based parallelism
pthread library

Python:

optimal serial
pyMP
Python multiprocessing module

Note that the gcc compiler automatically vectorizes the addition of arrays x and y using SIMD compliant hardware and appropriate data-types.

I'll be adding the GPU comparison as soon as I can! Currently I don't have a GPGPU compatible graphics card on my device...

Open source Code:

https://github.com/varun-manjunath/ParallelComputing/tree/master/library_comparison

Comments

Scilab matrix norm optimization

I had used Scilab for learning Digital Image Processing. It is a good open source alternative to Matlab. Although it has a good foundation, it was very clear that it was improvement was still possible. In a recent image processing workshop, I learnt that the Scilab community had open sourced the source code on GitHub. Keeping in mind that the software is continually evolving, I knew I could put my optimization skills to good use. So, I dug into the code-base and found a few optimizable programs. So, I decided to inculcate parallelism into matrix norm calculation program, which is quite simple yet used by many other subprograms and external function calls. https://github.com/opencollab/scilab/blob/master/scilab/modules/linear_algebra/src/c/norm.c There were many potential areas for optimization: Elimination of redundant comparisons Use else if construct to remove redundant checks Remove floating point equality comparison! Embarrassingly parallel/perfect ly parallel for l...

Parallel computing jargon

Parallel is always better than Serial... right? NO Well, in the most general case, parallel computing is better than serial computing in terms of speed and throughput. Sometimes, we have to make other considerations too. As a comparison , consider computer networking where serial transmissions are straight-forward and faster than their parallel SCSI counterparts! Some processes are inherently not parallelizable, due to presence of data dependency. (Two bank account withdrawals from different locations which may lead to negative account balance if done simultaneously! Anyway, such a pair of withdrawals with critical section management using semaphores/mutexes conceptually and momentarily reduces to serial execution...) On a lighter note, the process of Sneezing and keeping your eyes open is not parallelizable for example! Before jumping into the concepts and principles of parallelizing a given task, let us go through some interesting set of ( controversial ...

Welcome to a magical world...

A magical world where clever algorithms meet elegant multitasking models! Suppose you were given the following open-ended problem: Given an array of n bits , perform a negation operation on each of the bits. The most obvious brute force approach would be to go from index 0 to index n-1 and invert the i th bit on the way. Well, this works in a clean way and is the simplest formulation which delivers correct results. Nice! Now suppose you have an array of about a million digits; our algorithm directs us to go through each of the elements serially, one by one, giving each bit a feeling of self-importance! (The CPU dedicates few clock cycles exclusively on accessing, processing and finally writing out the result). Can we make this faster? Can we use the fact that an operation on the i th bit is independent of the operation on the ( i+1) th bit (or any other bit in general)? Of course we can! Imagine a switchbo...

Magical World of Parallel Computing

Search This Blog