Magical World of Parallel Computing

I had used Scilab for learning Digital Image Processing. It is a good open source alternative to Matlab. Although it has a good foundation, it was very clear that it was improvement was still possible. In a recent image processing workshop, I learnt that the Scilab community had open sourced the source code on GitHub. Keeping in mind that the software is continually evolving, I knew I could put my optimization skills to good use. So, I dug into the code-base and found a few optimizable programs. So, I decided to inculcate parallelism into matrix norm calculation program, which is quite simple yet used by many other subprograms and external function calls. https://github.com/opencollab/scilab/blob/master/scilab/modules/linear_algebra/src/c/norm.c There were many potential areas for optimization: Elimination of redundant comparisons Use else if construct to remove redundant checks Remove floating point equality comparison! Embarrassingly parallel/perfect ly parallel for l...

I would like to show you a very simple, yet subtle example on how programs can seem to produce unreasonable outputs. Recently, I was glancing through certain programs in Python, searching for places to optimize code and induce parallelism. I started thinking of threads immediately, and how independent contexts of computation can speed up code. Although I program frequently with Python, I hadn't been using any kind of explicit parallelism in my code. So, using my C knowledge, I went towards the Threading library of Python. Long story short, that was a mistake! Turns out that the Python implementation which is distributed by default (CPython) and Pypy, both have a process-wide mutex called the Global Interpreter Lock. This lock is necessary mainly because CPython's memory management is not thread-safe. The GIL locks up any kind of concurrent access to any objects in the Python run-time to prevent any race conditions or corruption of states. This is effectively a synchroniz...