OpenMP: Reduction

Hey guys! Welcome to HPC Education and today we are looking at the Reduction Clause.
Let’s go back to our example in the parallel for video. We are trying to find the sum of first 100 natural numbers parallelly. Here’s the code. Basically, we are using the parallel for construct to add up 25 numbers in each of the four threads and then add these partial sums. Thread_sum[ID] contains each of the individual thread sums while sum contains the gathered thread sums. We know this method causes false-sharing.
To tackle this issue, OpenMP introduced a concept called reduction which solves our problem more efficiently and avoids false sharing. Now lets look at the same problem with reduction incorporated the code. Reduction is a clause that adds specific functionality to the for loop here. ‘ +’ is the type of operation we want to perform and sum is the reduction variable. Reduction basically performs all the steps of initiating and calculating individual thread sums to local variable and then combining these local variables into a single global variable in just a couple of lines of code. Internally, reduction is implemented by creating private copies of each list item for every implicit task, as if the private clause has been used.
The sum += i we saw in the code is basically a shorthand assignment operation. Reduction supports various short hand assignment operations. Operations like sum += a[i]; sum = sum + a[i]; sum = a[i] + sum; can be written as shorthand assignment operations. The same goes for subtraction, multiplication, division and other operations. When operators have the same precedence, the associativity of shorthand operators is always right to left just like assignment.
In mathematics, addition and multiplication of real numbers is associative. However in computer science, the addition and multiplication of floating point numbers is not associative due to rounding errors. Reduction in OpenMP does not have mechanisms in place to account for this and leaves it to the programmer. This why you might see a difference in results between a serial and parallel implementation of the same calculation.
Reduction is useful when we need to perform some operation on a large number of operands and saving it in a single variable. Here are some reduction use cases. We can use reduction to add, subtract or multiply a large number of operands. For multiplication, the reduction variable is automatically initialized as 1. Reduction can also be applied to Logical and bitwise operations. We can also calculate minimum and maximum where the reduction variable is initialized as the largest positive number and most negative number respectively.
That’s all for this video. See you again in the next one!

[@mohammedimran1280]

after watching reduction, my syllabus also got reduced:))

[@dummyaccount8328]

Thank you for this series, really helping alot

[@anlmaral1762]

Thank you!