Coarsen the base case of P-MERGE
In order to coarsen the base case of P-MERGE, switch to the ordinary serial sort when the size of the array is small. In other words use quick sort if size of the array is significantly small.
In practice, the recursion is coarsened to minimize overheads. P-MERGE could be coarsened by using function calls and in-lining near the leaves of recursion, rather than spawning. It is used to make algorithm more efficient.
Algorithm:
The following procedure merges two sub arrays that lie within the same array.
The procedure P-MERGE sorts an array T and stores the output in array A. The array would be frequently divided into sub arrays. The lower and upper bounds of the concerned sub arrays are p1, r1 and p2, r2. The lower bound of the output array is p3.
// Parallel merging
P-MERGE
//calculate the size of sub arrays and stores them in n1 and n2
//check the combined size of two arrays. If they are very small then do the quick sort
// coarsen
if is very small
//apply the quick sort (refer the text book)
Quick-Sort
//to insure that is larger than .If not then swap.
else if
//check if the size of the first sub array is lesser than the size of the second sub
//array then exchange the arrays bounds
exchangewith
exchangewith
exchangewith
//check if array is empty
else if
return
else
//get the mid of the first sub array
//apply the binary search to find the mid location of the second sub array
BINARY_SEARCH
//get the mid-point for the output array so that the element must be placed at
// proper location in sorted array
//store the elements in the output array
//parallel recursive call to merge the entire arrays in parallel
Spawn P-MERGE
P-MERGE
sync
P-Merge uses the procedure BINARY_SEARCH to find the mid of the array. The location of the mid element x is found in the array T that has the lower bound p and upper bound r.
BINARY_SEARCH
//set the pointer low to the startindex of thearray
//set high to p or r+1; whichever is greater
//check whether elements are there in the array
while
//set the mid to the middle of the array
//check if the parameterized element is smaller than the mid element
if
//update value of high to mid
else
//update value of low to one more than mid
return
In the above algorithm of P-MERGE, coarsen start at first “if” clause and close after the Quick-sort. Remaining part of the algorithm is same as old version of P-Merge.
Here the base case form coarsenis “the size of the array is small” and to operate this condition, quick-sort concept comes in picture.
Illustrate the example of P-MERGE:
When size of two arrays is large, then the array get merged by using the P-MERGE(). If size of two arrays is not too large then it gets sorted by quick sort.
Suppose T1 and T2 are two sorted array which has to be merge by P-MERGE() function defined above to get a single sorted array.
In above sorted arrays T1 and T2, the value of p1and p2 is 0 and value of r1 and r2 is 4.
In P-MERGE( ) function the statement inside else part get execute. First calculate the value of which is the mid element of array T1.
Therefore, the mid element of T1 array is at index 2 that is 40.
After finding the , find the first element from T2 array which is greater than T1(q1) that is . Call BINARY_SEARCH() function to find a .
So call BINARY_SEARCH function with the parameters BINARY_SEARCH. In BINARY_SEARCH() first set the low and high.
After this use while loops to check whether low is lesser than high or not. In this case condition is true therefore calculating mid value of T2.
This implies that mid element of T2 array is at index 2 that is 34. Then check that is
As if condition is false, else block of BINARY_SEARCH() function executes. Now value of low is updated to mid+1 that is 3. Again while loop iterate and check whether low is lesser than high or not that is. So again mid value is calculated.
This implies that new mid element of T2 array is at index 4 that is 78. Then check that is
As if condition is true, if block of BINARY_SEARCH() function executes. Now value of high is updated to mid that is 4. Again while loop iterate and check whether low is lesser than high or not that is. So again mid value is calculated.
This implies that new mid element of T2 array is at index 3 that is 65. Then, checkthat is
The if condition is true. Therefore if block of BINARY_SEARCH() function executes. Now value of high is updated to mid that is 3.Then again while loop iterate and check whether low is lesser than high or not that is . This time the condition is false. Therefore value of high return to P-MERGE() which is .
Now, calculate the position of
Now, store index elements of T1 array into A array at index.
Now again call Spawn P-MERGE and P-MERGE. This time, in Spawn P-MERGEelse if condition gets true because therefore both these sub-array exchange. The P-MERGEget sorted by the Quick-sort() because is very small that is 4. Hence at last, the sorted array A is as follow:
Pseudocode for an efficient multithreaded merging procedure that uses
median finding procedure
Work is defined as total time required for completing the entire multithreaded computation on a single processor.
Span is defined as the maximum time required for completing the strands along any path in the directed acyclic graph (DAG). It is an expensive path which contains maximum number of strands.
Work is the execution time of a computation on a single processor. Consider it as.
Now suppose there are unlimited numbers of processors. Then the span is denoted by
The ratio of and, that is gives the parallelism of the multithreaded computation.Median is defined as a value which is present or lie at the mid of any series. Example as shown below: The series shown above is the series of seven elements. 23 is the first element of this series and 98 is the median of this series.Algorithm:
// Parallel merging
P-MERGE
//calculate the size of sub arrays and stores them in n1 and n2
//check the combined size of two arrays. If they are very small then do the quick sort
// coarsen
if is very small
//apply the quick sort (refer the text book)
Quick-Sort
//to insure that is larger than .If not then swap.
else if
return
else
//call function to find the median of two arrays
Val=TWO-ARRAY-MEDIAN
for j=0 to n1
if val==T[j]
j
//apply the binary search to find the mid location of the second sub array
BINARY_SEARCH
//get the mid-point for the output array so that the element must be place at proper
//locationin sorted array
//storethe elements in theoutput array
//parallel recursive call to merge the entire arrays in parallel
Spawn P-MERGE
P-MERGE
sync
// binary search to find the first element larger than x in second sub-array.
BINARY_SEARCH
//set the pointer low to the startindex of thearray
//set high to p or r+1; whichever is greater
//check whether elements are still remaining to be checked in the array
while
//set the mid to the middle of the array
//check if the parameterized element is smaller than the mid element
if
//update value of high to mid
else
//update value of low to one more than mid
//return index of first element in second sub array which is greater than x
return
Consider the result of the exercise 9.3-8 in textbook,
// function which find two array median
TWO-ARRAY-MEDIAN
//Call FIND-MEDIAN() function to find the median.
median=FIND-MEDIAN
//check median found or not, if not exchange both sub arrays
if median == NOT-FOUND
//Again Call FIND-MEDIAN() function to find the median.
median=FIND-MEDIAN
//return median to calling function
return median
// finding median
FIND-MEDIAN
//check low is smaller than high or not
if
//return to calling function
return NOT-FOUND
else
//find mid value index
//compare value of k, n and value of Tarray at different index
ifand
return
else if
and
return
else if
//again call function to find median in lower half
return FIND-MEDIAN
else
//again call function to find median in upper half
return FIND-MEDIAN
Analyzing the above algorithm:
Here three methods which are work, span and parallelism are used for analyzing the algorithm which is as follow:
Work:
Work of the above Mt-FLOYD-WARSHALL (W) will be same as that of the execution time of its serialization. So in order to compute work, parallel for loop must be replaced with ordinary for loop.
TWO-ARRAY-MEDIUM will cost additional work ofbecause this forms a binary tree structure and a binary tree structure takes that much of time.
Consider the analysis of multithreaded merge sort in the textbook,
And,
Now in analyzing part the only difference is that the addition work of needed due to the TWO-ARRAY-MEDIUM.
Therefore total work will be,
Hence total work of modified P-MERGE is
Span:
TWO-ARRAY-MEDIUM will cost additional span of.
Therefore total span will be,
Hence span of modified P-MERGE is.
Parallelism:
Parallelism is the ratio of work by span that is ratio of by.
Therefore,
Parallelism=
Hence parallelism of modified P-MERGE is.
An efficient multithreaded algorithm for partitioning an array around a pivot
Work is defined as total time required for completing the entire multithreaded computation on a single processor.
Span is defined as the maximum time required for completing the strands along any path in the directed acyclic graph (DAG). It is an expensive path which contains maximum number of strands.
Work is the execution time of a computation on a single processor. Consider it as.
Now suppose there are unlimited numbers of processors. Then the span is denoted by
The ratio of and, that is gives the parallelism of the multithreaded computation. Pivot element in the array is that element in array where actual calculation, operation like sorting in quick sort starts. In quick sort pivot element is the element which is placed at their proper correct position (in sorted order) in array.Consider the procedure of Partition, section 7.1 of the text book,
The algorithm below is going to use the concept of partition and multithreaded algorithm. In partition procedure there is two ‘for’ loops. So here those loops are going to be parallel to make it efficient multithreaded algorithm.
Algorithm:
// partition in parallel fashion
Parallel_partition
//check whether array having only one element
if
return
//taking auxiliary arrays array ,,
//use for loop to copy elements in auxiliary array
parallel for to
if
else
if
else
//call function to apply sum operation
parallel_prefix_sum
parallel_prefix_sum
//loop to compare all element of auxiliary array X[] with x.
parallel for to
//check ith index element of X [ ] array is lesser than x.
if
//check ith index element of X [ ] array is greater than x.
else if
return
The example below shows the concept and implementation of parallel prefix sum.
There is a set of elements say and be another set which is defined on element set using binary associative operation.
Set will be defined as,
for .
For example, the set is given below,
=binary addition
Then set will be given by,
// applying sum operation on parallel partition arrays
Parallel_prefix_sum
//Check if number of element in array is 1
if
else
parallel for to
parallel_prefix_sum
parallel forto
if
else if is even
else
return
Analyzing the above algorithm:
Three methods work, span and parallelism are used for analyzing the algorithm.
Work:
Work of the above algorithm can be calculated by computing the execution time of its serialization.
As parallel for loop of the above algorithm are not nested,
… … (1)
Work of parallel_prefix algorithm can be calculated by replacing the parallel for loop with ordinary for loop and then computing the running time of algorithm.
Since parallel for loop of the parallel_prefix algorithm are not nested,
Applying master theorem,
… … (2)
Putting value of equation (2) in (1)
… … (3)
Hence total work of Parallel_partition algorithm is
Span:
Span of the above algorithm will be given by,
… … (4)
Span of the parallel_prefix_algorithm will be given by,
Applying master theorem (In the analysis of the algorithm, master theorem gives the solution in asymptotic term for the recurrence relation),
… … (5)
From equation (4) and (5),
Hence span of Parallel_partition algorithm is
Parallelism:
Parallelism is the ratio of work by span that is ratio of by.
Parallelism=
=
=
Hence parallelism of Parallel_partition algorithm is
Multithreaded version of RECURSIVE- FFT
Fast Fourier transform is an algorithm to calculate discrete Fourier transform (DFT) and also to calculate inverse discrete Fourier transform.
Fourier transformation changes time to frequency and frequency to time. It is one of the important numerical algorithms.
In Discrete Fourier Transformation (DFT), the polynomial that should be evaluated is given as:
Here, the polynomial has a degree bound of n and the roots of unity in the complex form will be. If the result is represented as, then DFT for the given polynomial can be given as:
The DFT given above is for 1-dimension only. If the polynomial has more than 1 dimension, then discrete Fourier transform for d-dimensions can be given as:
The above expression requires operation for evaluation. This means that it requires execution time.
The FFT can do the same work in time. In other words, can say that FFT algorithm only require operations. The FFT work on divide and conquer rule which divide polynomial into polynomial of odd and even indexes coefficients of .
Work is defined as total time required for completing the entire multithreaded computation on a single processor.
Span is defined as the maximum time required for completing the strands along any path in the directed acyclic graph (DAG). It is an expensive path which contains maximum number of strands.
Work is the execution time of a computation on a single processor. Consider it as.
Now suppose there are unlimited numbers of processors. Then the span is denoted by
The ratio of and, that is gives the parallelism of the multithreaded computation.Multithreaded version of RECURSIVE-FFT:
Consider the algorithm Recursive-FFT on section 30.2 from the textbook. Given below is the multithreaded version of this algorithm. To make it multithreaded, spawn keyword used before the recursive function to make its implementation in parallel.
// multithreaded version of recursive FFT
RECURSIVE-FFT
//Find length of polynomial which must be a power of 2
// check length is equal to 1 or not.
if
return
//divide polynomial into polynomial of odd and even indexes coefficients
// using the concept of parallelization to making algorithm multithreaded
Spawn RECURSIVE-FFT()
RECURSIVE-FFT()
//use loop to combine the output of even and odd index terms subscript.
forto
// y is assumed to be a column vector
return
Analyzing the above algorithm:
Here three methods which are work, span and parallelism are used for analyzing the algorithm which is as follow:
Work:
Work of the above algorithm can be calculated by computing the running time of its serialization.
That is the other than recursive call, there is work of because of the ‘for’ loop.
Therefore,
Applying master theorem (In the analysis of the algorithm, master theorem gives the solution in asymptotic term for the recurrence relation),
So,
Total work will be as follow (in above recursion equation the first part make the tree structure having length (lgn))
… … (1)
Hence total work of Multithread RECURSIVE- FFT isSpan:
Span of the above algorithm will be because each of the call will has to do a minimum work of executing the ‘for’ loop in term of.
Therefore,
… … (2)
Hence span of Multithread RECURSIVE- FFT isParallelism:
Parallelism is the ratio of work by span that is ratio of by.
Parallelism=
By EQUATION (1) and (2)
=
=
=
Hence span of Multithread RECURSIVE- FFT is.
Multithreaded version of Randomized selection
Randomized Selection:
It is a process used to find minimum element in an array which has execution time. It also depends on divide-and-conquer algorithms, similar to quick sort. But Randomized selection processes only one side of partition unlike quick sort which processes both sides of partition. That is why the execution time of Randomized select is instead of.
Work is defined as total time required for completing the entire multithreaded computation on a single processor.
Span is defined as the maximum time required for completing the strands along any path in the directed acyclic graph (DAG). It is an expensive path which contains maximum number of strands.
Work is the execution time of a computation on a single processor. Consider it as.
Now suppose there are unlimited numbers of processors. Then the span is denoted by
The ratio of and, that is gives the parallelism of the multithreaded computation.Algorithm:
//consider the algorithm Randomized-Select on section 9.2 of the textbook
// randomize selection using parallel algorithmt
Parallel_randomized_select
//find number of element in array
//check size of array
if is very less say
sort using any sorting algorithm
else
//select x element from array
Select a random element from
//call function to divide the array
Parallel_partition
if
return A[k]
else if
return
spawn Parallel_randomized_select
else
return Parallel_randomized_select
sync
// parallel partitioning
Parallel_partition
//check whether array having only one element
if
return
//taking auxiliary arrays
array ,,
//use for loop to copy elements in auxiliary array
parallel for to
if
else
if
else
//call function to apply sum operation
parallel_prefix_sum
=parallel_prefix_sum
parallel for to
//check if ith index element of X [ ] array is lesser than x.
if
//check if ith index element of X [ ] array is greater than x.
else if
return
The example below shows the concept and implementation of parallel prefix sum.
There is a set of elements say andbe another set which is defined on element set using binary associative operation.
Set will be defined as,
for .
For example,
Consider the set is given below,
=binary addition
Then set will be given by,
// function calculation sum operation in parallel fashion
Parallel_prefix_sum
//check if array contains only one element.
if
else
parallel for to
parallel_prefix_sum
parallel for to
if
else if is even
else
returnAnalyzing the above algorithm:
Three methods work, span and parallelism are used for analyzing the algorithm.
Work: Work can be calculated by computing the execution time of its serialization.
Work of parallel randomized algorithm can be following recurrence,
… … (X)
Work of above Parallel partition can be calculated by replacing the parallel for loop with ordinary for loop and the computing of the running time of algorithm.
As parallel for loop of the above algorithm are not nested ,
… … (1)
Work of parallel_prefix algorithm can be calculated by replacing the parallel for loop with ordinary for loop and the computing of the running time of algorithm.
As parallel for loop of the parallel_prefix_algorithm algorithm are not nested,
Applying master theorem (In the analysis of the algorithm, master theorem gives the solution in asymptotic term for the recurrence relation),
… … (2)
Putting value of equation (2) in (1),
… … (3)
From (X) and (3),
Applying master theorem (In the analysis of the algorithm, master theorem gives the solution in asymptotic term for the recurrence relation),
Hence total work of Parallel_Randomized_Select algorithm is.
Span:
Span of the Parallel_randomized_select algorithm are given by following recurrences.
… … (Y)
Span of the Parallel_partition algorithm will be given by,
… … (4)
Span of the parallel_prefix_algorithm will be given by,
Applying master theorem,
… … (5)
From equation (4) and (5),
… … (6)
From equation (Y) and (6),
Hence span of Parallel_Randomized_Select algorithm is.
Parallelism:
Parallelism is the ratio of work by span that is ratio of by.
Parallelism=
=
=
Hence parallelism of Parallel_Randomized_Select algorithm is.
Multithread SELECT Algorithm
The serial algorithms are executed on the uniprocessor system in which only one instruction is executed at a time. In order to reduce the execution time of algorithm, a multithread algorithm was evolved. A multithread algorithm executes on the multiprocessor system in which more than one instruction can be executed at a time.
Multithreading of the SELECT:
Work is defined as total time required for completing the entire multithreaded computation on a single processor.
Span is defined as the maximum time required for completing the strands along any path in the directed acyclic graph (DAG). It is an expensive path which contains maximum number of strands.
Work is the running time of a computation on a single processor. Consider it as.
Now suppose there are unlimited numbers of processors. Then the span is denoted by
The ratio of and, that is gives the parallelism of the multithreaded computation.
Algorithm:
// consider the section 9.3 of the textbook.
// SELECT procedure using parallel algorithm concept.
Parallel_Selection
if
//Return the first element as smallest element because array contains only one
//element
return
else
Divide elements of into groups, each group having 5
elements. Last group have remaining n mod 5 elements.
parallel forto
//Consider section 9.3 for selection algorithm
first sort each group using merge sort and then find median of each group
and also store it simultaneously into
//Consider section 27.3 for multithreaded merge sort
Calculate the median among all median of
=Par-Partition
if
return
else if
return Par-Selection
else
return Par-Selection
Example:
Assume an array A contains 23 elements which is represented as small circles in the above figure. According to parallel_Selection algorithm first divide the elements into groups, each group containing 5 elements each. The last group contains n mod 5 elements.
Then sort each group using merge sort algorithm. The arrows shown that moves from larger element to smaller element. The white circle represents the median of that group. The white circle which is denoted by m is the median of the median. The elements in the shaded portion are greater than m.Analyzing the above algorithm
Three methods work, span and parallelism are used for analyzing the algorithm.
Work:
Work can be calculated by replacing the parallel for loop with ordinary for loop and calculating the running time of the algorithm.
According to the section 9.3,
The generalizations form of the above algorithm:
Hence, work of Parallel_Selection algorithm is.
Span:
Consider the analysis of multithreaded merge sort on page 803,
Span of the above algorithm can be given by,
=
Hence work of Parallel_Selection algorithm is.
Parallelism:
Parallelism is the ratio of work by span that is ratio of by.
Therefore, Parallelism=
Hence work of Parallel_Selection algorithm is.


TFOptimizer
TFOptimizer
TFOptimizer is used for optimizing a TensorFlow model with respect to its training variables on Spark/BigDL.
Remarks:
- You need to install tensorflow==1.15.0 on your driver node.
- Your operating system (OS) is required to be one of the following 64-bit systems: Ubuntu 16.04 or later and macOS 10.12.6 or later.
- To run on other systems, you need to manually compile the TensorFlow source code. Instructions can be found here.
Create a TFOptimizer:
import tensorflow as tf
from zoo.tfpark import TFOptimizer
from bigdl.optim.optimizer import *
loss = ...
optimizer = TFOptimizer.from_loss(loss, Adam(1e-3))
optimizer.optimize(end_trigger=MaxEpoch(5))
For Keras model:
from zoo.tfpark import TFOptimizer
from bigdl.optim.optimizer import *
from tensorflow.keras.models import Model
model = Model(inputs=..., outputs=...)
model.compile(optimizer='rmsprop',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
optimizer = TFOptimizer.from_keras(model, dataset)
optimizer.optimize(end_trigger=MaxEpoch(5))
Methods
from_loss (factory method)
Create a TFOptimizer from a TensorFlow loss tensor.
The loss tensor must come from a TensorFlow graph that only takes TFDataset.tensors and
the tensors in tensor_with_value
as inputs.
from_loss(loss, optim_method, session=None, val_outputs=None,
val_labels=None, val_method=None,
clip_norm=None, clip_value=None, metrics=None,
tensor_with_value=None, **kwargs)
Arguments
- loss: The loss tensor of the TensorFlow model, should be a scalar.
The loss tensor must come from a TensorFlow graph that only takes TFDataset.tensors and
the tensors in
tensor_with_value
as inputs. - optim_method: the optimization method to be used, such as bigdl.optim.optimizer.Adam
- session: the current TensorFlow Session, if you want to used a pre-trained model, you should use the Session to load the pre-trained variables and pass it to TFOptimizer.
- val_outputs: the validation output TensorFlow tensor to be used by val_methods
- val_labels: the validation label TensorFlow tensor to be used by val_methods
- val_method: the BigDL val_method(s) to be used.
- clip_norm: float >= 0. Gradients will be clipped when their L2 norm exceeds this value.
- clip_value: float >= 0. Gradients will be clipped when their absolute value exceeds this value.
- metrics: a dictionary. The key should be a string representing the metric's name and the value should be the corresponding TensorFlow tensor, which should be a scalar.
- tensor_with_value: a dictionary. The key is TensorFlow tensor, usually a placeholder, the value of the dictionary is a tuple of two elements. The first one of the tuple is the value to feed to the tensor in training phase and the second one is the value to feed to the tensor in validation phase.
from_keras (factory method)
Create a TFOptimizer from a tensorflow.keras model. The model must be compiled.
from_keras(keras_model, dataset, optim_method=None, **kwargs)
Arguments
- keras_model: the tensorflow.keras model, which must be compiled.
- dataset: a TFDataset
- optim_method: the optimization method to be used, such as bigdl.optim.optimizer.Adam
set_train_summary
set_train_summary(summary)
Arguments
- summary: The train summary to be set. A TrainSummary object contains information necessary for the optimizer to know how often the logs are recorded, where to store the logs and how to retrieve them, etc. For details, refer to the docs of TrainSummary.
set_val_summary
set_val_summary(summary)
Arguments
- summary: The validation summary to be set. A ValidationSummary object contains information necessary for the optimizer to know how often the logs are recorded, where to store the logs and how to retrieve them, etc. For details, refer to the docs of ValidationSummary.
set_constant_gradient_clipping
set_constant_gradient_clipping(min_value, max_value)
Arguments
- min_value: the minimum value to clip by
- max_value: the maxmimum value to clip by
set_gradient_clipping_by_l2_norm
set_gradient_clipping_by_l2_norm(self, clip_norm)
Arguments
- clip_norm: gradient L2-Norm threshold
optimize
optimize(self, end_trigger=None)
Arguments
- end_trigger: BigDL's Trigger to indicate when to stop the training. If none, defaults to train for one epoch.


By casting gem-inlaid sticks, rolling dragon bones, laying out ornate cards, or employing some other divining tool, you receive an omen from an otherworldly entity about the results of a specific course of action that you plan to take within the next 30 minutes. The DM chooses from the following possible omens: • Weal, for good results • Woe, for bad results • Weal and woe, for both good and bad results • Nothing, for results that aren't especially good or bad The spell doesn't take into account any possible circumstances that might change the outcome, such as the casting of additional spells or the loss or gain of a companion.
If you cast the spell two or more times before completing your next long rest, there is a cumulative 25 percent chance for each casting after the first that you get a random reading. The DM makes this roll in secret.


{# 1. Extend the relevant layout #} {# --------------------------------------- #} {% extends 'sidebar-standard.html' %} {# --------------------------------------- #} {# 2. Assign Settings and variables #} {# --------------------------------------- #} {% set id = "standard" %} {% set section = "standard" %} {% set title = "" %} {% set hero_copy = "Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua." %} {# --------------------------------------- #} {# 3. Add models and macros #} {# --------------------------------------- #} {% import 'macros/button.html' as buttons %} {% import 'macros/icon.html' as icons %} {% import 'macros/alert.html' as alerts %} {# --------------------------------------- #} {# 4. Add page content into relevant areas Add/Delete blocks as necessary #} {# --------------------------------------- #} {% block hero %} {% include 'heroes/hero-standard.html' %} {% endblock %} {% block aside %}
The IATI standard allows for the recording of information on all organisations that participate in any part of the lifecycle of an aid activity: inter alia donors, beneficiaries, extending and implementing agencies.
Crucial to this, is a common process through which to identify and declare identifiers for these organisations.
Organisational identifiers have two uses in IATI.
In the IATI organisation standard:
iati-organisation
.iati-identifier
- used to declare a globally unique identifier for the specific iati-organisation
In the IATI activity standard:
iati-activities
(using participating-org)iati-activity
- composed of the reporting organisation Organisation ID, and an Activity ID.The two components of IATI Organisation Identifiers
The IATI Organisational Identifier scheme is based on re-using existing known identifiers for an organisation.
Identifier strings consist of two components:
1 and 2 are separated by a dash -.
The namespace code (1) may contain a dash -.
GB-COH is the namespace code to the UK Companies Registrar which issues numerical company identifiers to legally constituted limited companies in the United Kingdom of Great Britain and Northern Ireland:
6368740 is the company number (base identifier) issued to Development Initiatives Poverty Research Limited.
Combining these we get an IATI Organisational Identifier of: GB-COH-06368740
To assist with creating and de-referencing IATI Organisation Identifiers, the codelist Organisation Registration Agency is available.
This codelist provides the namespace codes for a number of Registration Agencies. This list is non-embedded in the IATI Standard, which means changes and additions may be made at short notice, outside of published decimal and integer upgrade schedules. If a Registation Agency that you wish to use is not on this list, please contact support@iatistandard.org.


ICEDOVE(1) Linux User's Manual ICEDOVE(1) NAME icedove - Mail User Agent (MUA) and newsgroup/RSS client for X11 derived from the Mozilla Thunderbird. SYNOPSIS icedove [OPTIONS] [URL] /usr/lib/icedove/icedove-bin [OPTIONS] [URL] DESCRIPTION Icedove provides IMAP/POP support, a built-in RSS reader, support for HTML mail, powerful quick search, saved search folders, advanced message filtering, junk mail controls, message grouping, labels, return receipts, smart address book, LDAP address completion, import tools and the ability to manage multiple identities in email and newsgroup accounts. Icedove provides enterprise and government grade security such as S/MIME, digital signing, message encryption, and support for certificates and security devices. USAGE icedove is a executable file that will set up the environment for the starting executable, icedove-bin. If there is an Icedove mail client already running, icedove will arrange for it to create a new mail client window; otherwise it will start the Icedove application. OPTIONS A summary of the options supported by icedove is included below. X11 options --display=DISPLAY X display to use --sync Make X calls synchronous --g-fatal-warnings Make all warnings fatal Mozilla options -h, -help Show summary of options. -v, -version Print Icedove version. -P [profile] Start with profile. When no profile is given, displays the Profile Manager. May require -no-remote, see below. -migration Start with migration wizard. May require -no-remote, see below. -ProfileManager Start with profile manager. May require -no-remote, see below. -no-remote Don't connect to a running Icedove instance. Don't accept or send remote commands. This option can be necessary in conjunction to several of the options above, that won't have any effect when an Icedove instance is running unless -no-remote is used at the same time. -new-instance Open a new instance instead of a new windows in the running instance. -UILocale locale Start with locale resources as User Interface locale. By default, it is guessed from environment and available locales for Icedove. -safe-mode Starts Icedove in safe mode, i.e. disabling all extensions and showing a bit more debugging messages. -jsconsole Start with Javascript Console -addressbook Open the address book at startup. -compose Compose a mail or news message. -mail Open the mail folder view. -mail URL Open the message specified by this URL. -news Open the news client. -setDefaultMail Set Icedove as the default mail client. -options Open the options dialog. -file Open the specified email file. DEBUGGING To debug Icedove, you need to run run-mozilla.sh with the absolute path of icedove. /usr/lib/icedove/run-mozilla.sh [DEBUGOPTIONS] /usr/lib/icedove/icedove- bin DEBUGOPTIONS A summary of the options for debugging icedove is included below. -g, -debug Starts Icedove through a debugger (gdb by default). --debugger program Use program (instead the default) as a debugger for Icedove. Implies -debug. ENVIRONMENT MOZILLA_DISABLE_PLUGINS - when set, totally disables loading plugins. FILES /usr/bin/icedove - shell script wrapping icedove-bin /usr/lib/icedove/icedove-bin - icedove executable EXAMPLES Some typical use case for starting Icedove from a console Starting Icedove without any extra options, useful to any messages from icedove in case something went not o.k.: icedove Starting Icedove without any extensions or themes, useful if extensions may make some trouble: icedove -safe-mode Starting Icedove with a composing window: icedove -compose Starting Icedove with the default debugger: /usr/lib/icedove/run-mozilla.sh -debug /usr/lib/icedove/icedove-bin Starting Icedove with the specific debugger: /usr/lib/icedove/run-mozilla.sh --debugger /foo/bar/debugger /usr/lib/icedove/icedove-bin BUGS To report a bug, please visit http://bugzilla.mozilla.org/ and/or report bugs to the Debian Bug Tracking System, as usual. AUTHORS The Mozilla Organization http://www.mozilla.org/about.html Christoph GÃ¶hre February 27, 2010 ICEDOVE(1)


layout: true class: center, middle, inverse --- # .ps-green[Pythonic Interfaces] ## The secret to building maintainable, quality code! ## .ps-purple[A mini version :)] .invisible[-] ## Praveen G Shirali ### .ps-yellow[BangPypers Meetup on Code Quality] ### .ps-yellow[18th July 2020] --- # What is an interface? .invisible[-] ### .ps-red[.underline[shared boundary]] across which .ps-yellow[.underline[two or more components]] .ps-green[.underline[share information]] .invisible[-] ##### Source: .underline[en.wikipedia.org/wiki/Interface_(computing)] --- layout: false class: inverse, middle # Example application 1. An application does some data processing through a `DataProcessor` component 2. `DataProcessor` performs `read` and `write` operations on file-like objects supplied to it 3. `DataProcessor` does not know/care about the _type_ of file-like object ``` ____________ ___________ | | depends on | | | Data | _ _ _ _ _ _ _ _ _ _ _ _ \ | File-like | | Processor | / | object | |____________| |___________| ``` --- layout: false class: inverse, middle # Interface for File-like Object ```python def read(self, size: int = -1) -> bytes: # read `size` (int) number of bytes. -1 means EOF # return data (bytes) def write(self, data: bytes) -> int: # write data (bytes) # return the number of bytes written (int) # --** # File-like objects generally have more methods # We are keeping it simple in this example ``` --- layout: false class: inverse, middle # There may be many implementations ... --- layout: false class: inverse, middle ```python class FileIO: '''Performs I/O on an underlying file''' # # ------------------------------------------------------ def read(self, size: int = -1) -> bytes: ... def write(self, data: bytes) -> int: ... # ------------------------------------------------------ # # ... other methods ``` --- layout: false class: inverse, middle ```python class SocketIO: '''Performs I/O on a socket''' # # ------------------------------------------------------ def read(self, size: int = -1) -> bytes: ... def write(self, data: bytes) -> int: ... # ------------------------------------------------------ # # ... other methods ``` --- layout: false class: inverse, middle ```python class MemoryIO: '''Performs I/O on an buffer in memory''' # # ------------------------------------------------------ def read(self, size: int = -1) -> bytes: ... def write(self, data: bytes) -> int: ... # ------------------------------------------------------ # # ... other methods ``` --- layout: false class: inverse, middle # Observation - .ps-yellow[Method definitions are same] - .ps-yellow[Intended behaviour is the same] - .ps-red[Implementation is different] --- layout: false class: inverse, middle # Today, we'll explore this ... - .ps-yellow[Method definitions are same] .invisible[- Intended behaviour is the same] .invisible[- Implementation is different] --- layout: false class: inverse, middle # Why? ### .ps-blue[(1) Many ways to implement methods] ### .ps-blue[(2) How you interact.. depends on how the methods are defined] --- layout: false class: inverse, middle # I pity the _foo_ ... ```python def foo(self): # name matters def foo(self, a: int, b: float, c=None): # signature matters @staticmethod def foo(): # no 'self' @classmethod def foo(cls): # class as 1st arg @property def foo(self): # not callable @foo.setter def foo(self, bar: bool): # assign. Don't call def foo(self): yield # generator function async def foo(self): # coroutine function ... ``` --- layout: false class: inverse, middle # So, - .ps-yellow[Method definitions are same] .invisible[- Intended behaviour is the same] .invisible[- Implementation is different] --- layout: false class: inverse, middle # So, - .ps-yellow[Method definitions are same] .ps-green[<--- can this be enforced?] .invisible[- Intended behaviour is the same] .invisible[- Implementation is different] --- layout: false class: inverse, middle # Quest: A mechanism which ensures/enforces ... 1. All .ps-yellow[required methods] are implemented with correct .ps-yellow[names] 2. Method .ps-yellow[signatures] are correct (arguments, order, type annotation etc) 3. .ps-yellow[Descriptors/decorators] like `@staticmethod`, `@classmethod`, `@property` etc are honoured. 4. generator, coroutine functions etc 5. ...more?? --- layout: false class: inverse, middle # Ways to go about it ... 1. Passive 2. Active --- layout: false class: inverse, middle # Passive 1. Checks are `not` part of the application 2. Checks are invoked externally and independently ### Example: - Static code analysis --- linters, mypy etc. - Runtime code checks --- unittests --- layout: false class: inverse, middle # Active 1. Checks are `part of` the application 2. They execute when the application runs 3. They raise errors from within the application ### Example: - Python's abstract base classes (abc) --- layout: false class: inverse, middle # Abstract Base Class ```python # ABCMeta is the enforcement mechanism # available as a **metaclass** # from abc import ABCMeta # decorate interface methods with `abstractmethod` decorator # from abc import abstractmethod # +more # Can define `__subclasshook__` which controls behaviour of `issubclass(..)` # # `collections.abc` has ABCs for common collections ``` --- layout: false class: inverse, middle # Abstract Base Class - Interface definition ```python from abc import ABCMeta, abstractmethod # IOMeta is the interface definition # class IOMeta(metaclass=ABCMeta): # <--- Class can have any name @abstractmethod def read(self, size: int = -1) -> bytes: raise NotImplemented @abstractmethod def write(self, data: bytes) -> int: raise NotImplemented ``` --- layout: false class: inverse, middle # Abstract Base Class - Usage ```python # Concrete implementations derive from abstract base classes # class MemoryIO(IOMeta): def read(self, size: int = -1) -> bytes: # actual implementation def write(self, data: bytes) -> int: # actual implementation if __name __ == "__main__": m = MemoryIO() # <--- Compliance is checked on instantiation ``` --- layout: false class: inverse, middle # Abstract Base Class - Enforcement ```python class MemoryIO(IOMeta): # missing `write` method implementation def read(self, size: int = -1) -> bytes: # actual implementation if __name __ == "__main__": m = MemoryIO() # <--- Compliance is checked on instantiation ``` ``` # ---- [output] ---- Traceback (most recent call last): File "{filename}", line {number}, in
m = MemoryIO() TypeError: Can't instantiate abstract class MemoryIO with abstract methods write ``` --- layout: false class: inverse, middle # Abstract Base Class - Limitations ```python # ABCMeta does not enforce method signatures # class MemoryIO(IOMeta): def read(self): # <--- incorrect signature # actual implementation def write(self): # <--- incorrect signature # actual implementation if __name __ == "__main__": m = MemoryIO() # <--- Does not raise errors ``` --- layout: false class: inverse, middle ## .. on abstract base classes: - enforce presence of methods and names - implemented using metaclasses; enforced on derived classes too ## .. but - abc has existed since Python2 - does not enforce signatures, or anything beyond just method names - goals are/were different (refer to a talk by Raymond Hettinger*) ###### Build powerful new datastructures with Python's abstract base classes https://www.youtube.com/watch?v=S_ipdVNSFlo by Raymond Hettinger, at PyCon Russia 2019. --- layout: false class: inverse, middle ## Today we have (in Python3)... - type annotations - inspect library ###### _inspect_ can be used to implement poweful enforcement code --- layout: false class: inverse, middle # Some 3rd party libraries ... ### zope.interface - version 5 now, lots of features - https://pypi.org/project/zope.interface/ ### python-interfaces - simpler, modern implementation, supports py2 & py3, uses metaclasses - https://pypi.org/project/python-interface/ ### implmements - very simple, supports py3.6+ only, composition over inheritance - https://pypi.org/project/implements/ --- layout: false class: inverse, middle # Some 3rd party libraries ... ### zope.interface - version 5 now, lots of features - https://pypi.org/project/zope.interface/ ### python-interfaces - simpler, modern implementation, supports py2 & py3, uses metaclasses - https://pypi.org/project/python-interface/ ### implmements .ps-green[<--- lets demo this] - very simple, supports py3.6+ only, composition over inheritance - https://pypi.org/project/implements/ --- layout: false class: inverse, middle # Demo ... --- layout: false class: inverse, middle # Open for Q&A Feel free to reach out with any questions or feedback. Thanks. ## Praveen G Shirali -<EMAIL_ADDRESS>- `pshirali` on BangPypers Slack - https://linkedin.com/in/praveenshirali - https://github.com/pshirali - I'm .underline[not] on Twitter ## Link to this talk - https://pshirali.github.io/pyifc-demo


On 6 April 2020, 6 new articles were published in 5 journals.
Divergent Perceptions of Muslim Americans on Being Stereotyped As Terrorists
Bringing Jails Into the Twenty-First Century: Evaluating Jail Tv As A Method For Programming
Childhood Trauma, Problematic Drug Use and Coping
Femme Fatales: Girl Gangsters and Violent Street Culture in Cape Town
Intended and Unintended Benefits of Specialty Courts: Results From A Texas Dwi Court
Keeping Connected While Living Apart: An Exploration of A Prison Contact Maintenance Program


UNIX> cat -n rec2.cpp 1 #includeTo use gdb, we compile the program with the -g flag. That makes the program run slower, but it also stores information that helps you debug.2 using namespace std; 3 4 5 void a(int i) 6 { 7 int j; 8 9 j = i*5; 10 printf("In procedure a: i = %d, j = %d\n", i, j); 11 if (i > 0) a(i-1); 12 printf("Later In procedure a: i = %d, j = %d\n", i, j); 13 } 14 15 main() 16 { 17 int i; 18 19 i = 16; 20 a(3); 21 printf("main: %d\n", i); 22 } UNIX>
We will run the program under the control of gdb. Each time we hit a breakpoint, we will run the where command and the print command so that you can see the values that are currently on the stack:
Try it out. You should see how the output of gdb matches the lecture note example of running rec2.cpp:
UNIX> gdb rec2 GNU gdb 6.3.50-20050815 (Apple version gdb-1461.2) (Fri Mar 5 04:43:10 UTC 2010) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries ... done (gdb) break 11 Breakpoint 1 at 0x100000c9a: file rec2.cpp, line 11. (gdb) break 20 Breakpoint 2 at 0x100000cd2: file rec2.cpp, line 20. (gdb) run Starting program: /Users/plank/tmp-class/Recursion/rec2 Reading symbols for shared libraries ++. done Breakpoint 2, main () at rec2.cpp:20 20 a(3); (gdb) where #0 main () at rec2.cpp:20 (gdb) cont Continuing. In procedure a: i = 3, j = 15 Breakpoint 1, a (i=3) at rec2.cpp:11 11 if (i > 0) a(i-1); (gdb) where #0 a (i=3) at rec2.cpp:11 #1 0x0000000100000cdc in main () at rec2.cpp:20 (gdb) print i $1 = 3 (gdb) print j $2 = 15 (gdb) cont Continuing. In procedure a: i = 2, j = 10 Breakpoint 1, a (i=2) at rec2.cpp:11 11 if (i > 0) a(i-1); (gdb) where #0 a (i=2) at rec2.cpp:11 #1 0x0000000100000caa in a (i=3) at rec2.cpp:11 #2 0x0000000100000cdc in main () at rec2.cpp:20 (gdb) print i $3 = 2 (gdb) print j $4 = 10 (gdb) cont Continuing. In procedure a: i = 1, j = 5 Breakpoint 1, a (i=1) at rec2.cpp:11 11 if (i > 0) a(i-1); (gdb) where #0 a (i=1) at rec2.cpp:11 #1 0x0000000100000caa in a (i=2) at rec2.cpp:11 #2 0x0000000100000caa in a (i=3) at rec2.cpp:11 #3 0x0000000100000cdc in main () at rec2.cpp:20 (gdb) print i $5 = 1 (gdb) print j $6 = 5 (gdb) cont Continuing. In procedure a: i = 0, j = 0 Breakpoint 1, a (i=0) at rec2.cpp:11 11 if (i > 0) a(i-1); (gdb) where #0 a (i=0) at rec2.cpp:11 #1 0x0000000100000caa in a (i=1) at rec2.cpp:11 #2 0x0000000100000caa in a (i=2) at rec2.cpp:11 #3 0x0000000100000caa in a (i=3) at rec2.cpp:11 #4 0x0000000100000cdc in main () at rec2.cpp:20 (gdb) print i $7 = 0 (gdb) print j $8 = 0 (gdb) cont Continuing. Later In procedure a: i = 0, j = 0 Later In procedure a: i = 1, j = 5 Later In procedure a: i = 2, j = 10 Later In procedure a: i = 3, j = 15 main: 16 Program exited normally. (gdb) quit UNIX>