Table of Contents¶
1 An example of a small Multi-Player simulation, with Centralized Algorithms
1.1 Creating the problem
1.1.1 Parameters for the simulation
1.1.2 Three MAB problems with Bernoulli arms
1.1.3 Some RL algorithms
1.2 Creating the EvaluatorMultiPlayers objects
1.3 Solving the problem
1.4 Plotting the results
1.4.1 First problem
1.4.2 Second problem
1.4.3 Third problem
1.4.4 Comparing their performances
An example of a small Multi-Player simulation, with Centralized Algorithms¶
First, be sure to be in the main folder, or to have SMPyBandits installed, and import EvaluatorMultiPlayers
from Environment
package:
[1]:
!pip install SMPyBandits watermark
%load_ext watermark
%watermark -v -m -p SMPyBandits -a "Lilian Besson"
Requirement already satisfied: SMPyBandits in ./venv3/lib/python3.6/site-packages (0.9.4)
Requirement already satisfied: watermark in ./venv3/lib/python3.6/site-packages (1.7.0)
Requirement already satisfied: numpy in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (1.15.4)
Requirement already satisfied: scikit-learn in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (0.20.0)
Requirement already satisfied: scikit-optimize in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (0.5.2)
Requirement already satisfied: matplotlib>=2 in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (3.0.2)
Requirement already satisfied: joblib in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (0.13.0)
Requirement already satisfied: scipy>0.9 in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (1.1.0)
Requirement already satisfied: seaborn in ./venv3/lib/python3.6/site-packages (from SMPyBandits) (0.9.0)
Requirement already satisfied: ipython in ./venv3/lib/python3.6/site-packages (from watermark) (7.1.1)
Requirement already satisfied: python-dateutil>=2.1 in ./venv3/lib/python3.6/site-packages (from matplotlib>=2->SMPyBandits) (2.7.5)
Requirement already satisfied: kiwisolver>=1.0.1 in ./venv3/lib/python3.6/site-packages (from matplotlib>=2->SMPyBandits) (1.0.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in ./venv3/lib/python3.6/site-packages (from matplotlib>=2->SMPyBandits) (2.3.0)
Requirement already satisfied: cycler>=0.10 in ./venv3/lib/python3.6/site-packages (from matplotlib>=2->SMPyBandits) (0.10.0)
Requirement already satisfied: pandas>=0.15.2 in ./venv3/lib/python3.6/site-packages (from seaborn->SMPyBandits) (0.23.4)
Requirement already satisfied: prompt-toolkit<2.1.0,>=2.0.0 in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (2.0.7)
Requirement already satisfied: decorator in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (4.3.0)
Requirement already satisfied: pickleshare in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (0.7.5)
Requirement already satisfied: jedi>=0.10 in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (0.13.1)
Requirement already satisfied: setuptools>=18.5 in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (40.6.2)
Requirement already satisfied: traitlets>=4.2 in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (4.3.2)
Requirement already satisfied: pexpect; sys_platform != "win32" in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (4.6.0)
Requirement already satisfied: pygments in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (2.2.0)
Requirement already satisfied: backcall in ./venv3/lib/python3.6/site-packages (from ipython->watermark) (0.1.0)
Requirement already satisfied: six>=1.5 in ./venv3/lib/python3.6/site-packages (from python-dateutil>=2.1->matplotlib>=2->SMPyBandits) (1.11.0)
Requirement already satisfied: pytz>=2011k in ./venv3/lib/python3.6/site-packages (from pandas>=0.15.2->seaborn->SMPyBandits) (2018.7)
Requirement already satisfied: wcwidth in ./venv3/lib/python3.6/site-packages (from prompt-toolkit<2.1.0,>=2.0.0->ipython->watermark) (0.1.7)
Requirement already satisfied: parso>=0.3.0 in ./venv3/lib/python3.6/site-packages (from jedi>=0.10->ipython->watermark) (0.3.1)
Requirement already satisfied: ipython-genutils in ./venv3/lib/python3.6/site-packages (from traitlets>=4.2->ipython->watermark) (0.2.0)
Requirement already satisfied: ptyprocess>=0.5 in ./venv3/lib/python3.6/site-packages (from pexpect; sys_platform != "win32"->ipython->watermark) (0.6.0)
Info: Using the Jupyter notebook version of the tqdm() decorator, tqdm_notebook() ...
Lilian Besson
CPython 3.6.6
IPython 7.1.1
SMPyBandits 0.9.4
compiler : GCC 8.0.1 20180414 (experimental) [trunk revision 259383
system : Linux
release : 4.15.0-38-generic
machine : x86_64
processor : x86_64
CPU cores : 4
interpreter: 64bit
[2]:
# Local imports
from SMPyBandits.Environment import EvaluatorMultiPlayers, tqdm
We also need arms, for instance Bernoulli
-distributed arm:
[3]:
# Import arms
from SMPyBandits.Arms import Bernoulli
And finally we need some single-player and multi-player Reinforcement Learning algorithms:
[4]:
# Import algorithms
from SMPyBandits.Policies import *
from SMPyBandits.PoliciesMultiPlayers import *
[5]:
# Just improving the ?? in Jupyter. Thanks to https://nbviewer.jupyter.org/gist/minrk/7715212
from __future__ import print_function
from IPython.core import page
def myprint(s):
try:
print(s['text/plain'])
except (KeyError, TypeError):
print(s)
page.page = myprint
For instance, this imported the UCB
algorithm:
[6]:
UCBalpha?
Init signature: UCBalpha(nbArms, alpha=4, lower=0.0, amplitude=1.0)
Docstring:
The UCB1 (UCB-alpha) index policy, modified to take a random permutation order for the initial exploration of each arm (reduce collisions in the multi-players setting).
Reference: [Auer et al. 02].
Init docstring:
New generic index policy.
- nbArms: the number of arms,
- lower, amplitude: lower value and known amplitude of the rewards.
File: /tmp/SMPyBandits/notebooks/venv3/lib/python3.6/site-packages/SMPyBandits/Policies/UCBalpha.py
Type: type
As well as the CentralizedMultiplePlay
multi-player policy:
[7]:
CentralizedMultiplePlay?
Init signature: CentralizedMultiplePlay(nbPlayers, nbArms, playerAlgo, uniformAllocation=False, *args, **kwargs)
Docstring:
CentralizedMultiplePlay: a multi-player policy where ONE policy is used by a centralized agent; asking the policy to select nbPlayers arms at each step.
Init docstring:
- nbPlayers: number of players to create (in self._players).
- playerAlgo: class to use for every players.
- nbArms: number of arms, given as first argument to playerAlgo.
- uniformAllocation: Should the affectations of users always be uniform, or fixed when UCB indexes have converged? First choice is more fair, but linear nb of switches, second choice is not fair, but cst nb of switches.
- `*args`, `**kwargs`: arguments, named arguments, given to playerAlgo.
Examples:
>>> import sys; sys.path.insert(0, '..'); from Policies import *
>>> s = CentralizedMultiplePlay(2, 3, UCB)
>>> [ child.choice() for child in s.children ]
[2, 0]
- To get a list of usable players, use ``s.children``.
- Warning: ``s._players`` is for internal use ONLY!
File: /tmp/SMPyBandits/notebooks/venv3/lib/python3.6/site-packages/SMPyBandits/PoliciesMultiPlayers/CentralizedMultiplePlay.py
Type: type
We also need a collision model. The usual ones are defined in the CollisionModels
package, and the only one we need is the classical one, where two or more colliding users don’t receive any rewards.
[8]:
# Collision Models
from SMPyBandits.Environment.CollisionModels import onlyUniqUserGetsReward
onlyUniqUserGetsReward?
Signature: onlyUniqUserGetsReward(t, arms, players, choices, rewards, pulls, collisions)
Docstring:
Simple collision model where only the players alone on one arm samples it and receives the reward.
- This is the default collision model, cf. [[Multi-Player Bandits Revisited, Lilian Besson and Emilie Kaufmann, 2017]](https://hal.inria.fr/hal-01629733).
- The numpy array 'choices' is increased according to the number of users who collided (it is NOT binary).
File: /tmp/SMPyBandits/notebooks/venv3/lib/python3.6/site-packages/SMPyBandits/Environment/CollisionModels.py
Type: function
Creating the problem¶
Parameters for the simulation¶
\(T = 10000\) is the time horizon,
\(N = 100\) is the number of repetitions (should be larger to have consistent results),
\(M = 2\) is the number of players,
N_JOBS = 4
is the number of cores used to parallelize the code.
[9]:
HORIZON = 10000
REPETITIONS = 100
NB_PLAYERS = 2
N_JOBS = 4
collisionModel = onlyUniqUserGetsReward
Three MAB problems with Bernoulli arms¶
We consider in this example \(3\) problems, with Bernoulli
arms, of different means.
The first problem is very easy, with two good arms and three arms, with a fixed gap \(\Delta = \max_{\mu_i \neq \mu_j}(\mu_{i} - \mu_{j}) = 0.1\).
The second problem is as easier, with a larger gap.
Third problem is harder, with a smaller gap, and a very large difference between the two optimal arms and the suboptimal arms.
Note: right now, the multi-environments evaluator does not work well for MP policies, if there is a number different of arms in the scenarios. So I use the same number of arms in all the problems.
[10]:
ENVIRONMENTS = [ # 1) Bernoulli arms
{ # Scenario 1 from [Komiyama, Honda, Nakagawa, 2016, arXiv 1506.00779]
"arm_type": Bernoulli,
"params": [0.3, 0.4, 0.5, 0.6, 0.7]
},
{ # Classical scenario
"arm_type": Bernoulli,
"params": [0.1, 0.3, 0.5, 0.7, 0.9]
},
{ # Harder scenario
"arm_type": Bernoulli,
"params": [0.005, 0.01, 0.015, 0.84, 0.85]
}
]
Some RL algorithms¶
We will compare Thompson Sampling against \(\mathrm{UCB}_1\), using two different centralized policy:
CentralizedMultiplePlay
is the naive use of a Bandit algorithm for Multi-Player decision making: at every step, the internal decision making process is used to determine not \(1\) arm but \(M\) to sample. For UCB-like algorithm, the decision making is based on a \(\arg\max\) on UCB-like indexes, usually of the form \(I_j(t) = X_j(t) + B_j(t)\), where \(X_j(t) = \hat{\mu_j}(t) = \sum_{\tau \leq t} r_j(\tau) / N_j(t)\) is the empirical mean of arm \(j\), and \(B_j(t)\) is a bias term, of the form \(B_j(t) = \sqrt{\frac{\alpha \log(t)}{2 N_j(t)}}\).CentralizedIMP
is very similar, but instead of following the internal decision making for all the decisions, the system uses just the empirical means \(X_j(t)\) to determine \(M-1\) arms to sample, and the bias-corrected term (i.e., the internal decision making, can be sampling from a Bayesian posterior for instance) is used just for one decision. It is an heuristic, proposed in [Komiyama, Honda, Nakagawa, 2016].
[12]:
nbArms = len(ENVIRONMENTS[0]['params'])
assert all(len(env['params']) == nbArms for env in ENVIRONMENTS), "Error: not yet support if different environments have different nb of arms"
nbArms
SUCCESSIVE_PLAYERS = [
CentralizedMultiplePlay(NB_PLAYERS, nbArms, UCBalpha, alpha=1).children,
CentralizedIMP(NB_PLAYERS, nbArms, UCBalpha, alpha=1).children,
CentralizedMultiplePlay(NB_PLAYERS, nbArms, Thompson).children,
CentralizedIMP(NB_PLAYERS, nbArms, Thompson).children
]
SUCCESSIVE_PLAYERS
[12]:
5
- One new child, of index 0, and class #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
- One new child, of index 1, and class #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
- One new child, of index 0, and class #1<CentralizedIMP(UCB($\alpha=1$))> ...
- One new child, of index 1, and class #2<CentralizedIMP(UCB($\alpha=1$))> ...
- One new child, of index 0, and class #1<CentralizedMultiplePlay(Thompson)> ...
- One new child, of index 1, and class #2<CentralizedMultiplePlay(Thompson)> ...
- One new child, of index 0, and class #1<CentralizedIMP(Thompson)> ...
- One new child, of index 1, and class #2<CentralizedIMP(Thompson)> ...
[12]:
[[CentralizedMultiplePlay(UCB($\alpha=1$)),
CentralizedMultiplePlay(UCB($\alpha=1$))],
[CentralizedIMP(UCB($\alpha=1$)), CentralizedIMP(UCB($\alpha=1$))],
[CentralizedMultiplePlay(Thompson), CentralizedMultiplePlay(Thompson)],
[CentralizedIMP(Thompson), CentralizedIMP(Thompson)]]
The mother class in this case does all the job here, as we use centralized learning.
[13]:
OnePlayer = SUCCESSIVE_PLAYERS[0][0]
OnePlayer.nbArms
OneMother = OnePlayer.mother
OneMother
OneMother.nbArms
[13]:
5
[13]:
<SMPyBandits.PoliciesMultiPlayers.CentralizedMultiplePlay.CentralizedMultiplePlay at 0x7fc67216f908>
[13]:
5
Complete configuration for the problem:
[14]:
configuration = {
# --- Duration of the experiment
"horizon": HORIZON,
# --- Number of repetition of the experiment (to have an average)
"repetitions": REPETITIONS,
# --- Parameters for the use of joblib.Parallel
"n_jobs": N_JOBS, # = nb of CPU cores
"verbosity": 6, # Max joblib verbosity
# --- Collision model
"collisionModel": onlyUniqUserGetsReward,
# --- Arms
"environment": ENVIRONMENTS,
# --- Algorithms
"successive_players": SUCCESSIVE_PLAYERS,
}
Creating the EvaluatorMultiPlayers
objects¶
We will need to create several objects, as the simulation first runs one policy against each environment, and then aggregate them to compare them.
[15]:
%%time
N_players = len(configuration["successive_players"])
# List to keep all the EvaluatorMultiPlayers objects
evs = [None] * N_players
evaluators = [[None] * N_players] * len(configuration["environment"])
for playersId, players in tqdm(enumerate(configuration["successive_players"]), desc="Creating"):
print("\n\nConsidering the list of players :\n", players)
conf = configuration.copy()
conf['players'] = players
evs[playersId] = EvaluatorMultiPlayers(conf)
Considering the list of players :
[CentralizedMultiplePlay(UCB($\alpha=1$)), CentralizedMultiplePlay(UCB($\alpha=1$))]
Number of players in the multi-players game: 2
Time horizon: 10000
Number of repetitions: 100
Sampling rate for plotting, delta_t_plot: 1
Number of jobs for parallelization: 4
Using collision model onlyUniqUserGetsReward (function <function onlyUniqUserGetsReward at 0x7fc672469bf8>).
More details:
Simple collision model where only the players alone on one arm samples it and receives the reward.
- This is the default collision model, cf. [[Multi-Player Bandits Revisited, Lilian Besson and Emilie Kaufmann, 2017]](https://hal.inria.fr/hal-01629733).
- The numpy array 'choices' is increased according to the number of users who collided (it is NOT binary).
Using accurate regrets and last regrets ? True
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.3, 0.4, 0.5, 0.6, 0.7]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.3, 0.4, 0.5, 0.6, 0.7]
- with 'arms' = [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)]
- with 'means' = [0.3 0.4 0.5 0.6 0.7]
- with 'nbArms' = 5
- with 'maxArm' = 0.7
- with 'minArm' = 0.3
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- with 'arms' represented as: $[B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.1, 0.3, 0.5, 0.7, 0.9]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.1, 0.3, 0.5, 0.7, 0.9]
- with 'arms' = [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)]
- with 'means' = [0.1 0.3 0.5 0.7 0.9]
- with 'nbArms' = 5
- with 'maxArm' = 0.9
- with 'minArm' = 0.1
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- with 'arms' represented as: $[B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.005, 0.01, 0.015, 0.84, 0.85]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.005, 0.01, 0.015, 0.84, 0.85]
- with 'arms' = [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)]
- with 'means' = [0.005 0.01 0.015 0.84 0.85 ]
- with 'nbArms' = 5
- with 'maxArm' = 0.85
- with 'minArm' = 0.005
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- with 'arms' represented as: $[B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)^*]$
Number of environments to try: 3
Considering the list of players :
[CentralizedIMP(UCB($\alpha=1$)), CentralizedIMP(UCB($\alpha=1$))]
Number of players in the multi-players game: 2
Time horizon: 10000
Number of repetitions: 100
Sampling rate for plotting, delta_t_plot: 1
Number of jobs for parallelization: 4
Using collision model onlyUniqUserGetsReward (function <function onlyUniqUserGetsReward at 0x7fc672469bf8>).
More details:
Simple collision model where only the players alone on one arm samples it and receives the reward.
- This is the default collision model, cf. [[Multi-Player Bandits Revisited, Lilian Besson and Emilie Kaufmann, 2017]](https://hal.inria.fr/hal-01629733).
- The numpy array 'choices' is increased according to the number of users who collided (it is NOT binary).
Using accurate regrets and last regrets ? True
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.3, 0.4, 0.5, 0.6, 0.7]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.3, 0.4, 0.5, 0.6, 0.7]
- with 'arms' = [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)]
- with 'means' = [0.3 0.4 0.5 0.6 0.7]
- with 'nbArms' = 5
- with 'maxArm' = 0.7
- with 'minArm' = 0.3
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- with 'arms' represented as: $[B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.1, 0.3, 0.5, 0.7, 0.9]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.1, 0.3, 0.5, 0.7, 0.9]
- with 'arms' = [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)]
- with 'means' = [0.1 0.3 0.5 0.7 0.9]
- with 'nbArms' = 5
- with 'maxArm' = 0.9
- with 'minArm' = 0.1
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- with 'arms' represented as: $[B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.005, 0.01, 0.015, 0.84, 0.85]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.005, 0.01, 0.015, 0.84, 0.85]
- with 'arms' = [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)]
- with 'means' = [0.005 0.01 0.015 0.84 0.85 ]
- with 'nbArms' = 5
- with 'maxArm' = 0.85
- with 'minArm' = 0.005
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- with 'arms' represented as: $[B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)^*]$
Number of environments to try: 3
Considering the list of players :
[CentralizedMultiplePlay(Thompson), CentralizedMultiplePlay(Thompson)]
Number of players in the multi-players game: 2
Time horizon: 10000
Number of repetitions: 100
Sampling rate for plotting, delta_t_plot: 1
Number of jobs for parallelization: 4
Using collision model onlyUniqUserGetsReward (function <function onlyUniqUserGetsReward at 0x7fc672469bf8>).
More details:
Simple collision model where only the players alone on one arm samples it and receives the reward.
- This is the default collision model, cf. [[Multi-Player Bandits Revisited, Lilian Besson and Emilie Kaufmann, 2017]](https://hal.inria.fr/hal-01629733).
- The numpy array 'choices' is increased according to the number of users who collided (it is NOT binary).
Using accurate regrets and last regrets ? True
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.3, 0.4, 0.5, 0.6, 0.7]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.3, 0.4, 0.5, 0.6, 0.7]
- with 'arms' = [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)]
- with 'means' = [0.3 0.4 0.5 0.6 0.7]
- with 'nbArms' = 5
- with 'maxArm' = 0.7
- with 'minArm' = 0.3
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- with 'arms' represented as: $[B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.1, 0.3, 0.5, 0.7, 0.9]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.1, 0.3, 0.5, 0.7, 0.9]
- with 'arms' = [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)]
- with 'means' = [0.1 0.3 0.5 0.7 0.9]
- with 'nbArms' = 5
- with 'maxArm' = 0.9
- with 'minArm' = 0.1
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- with 'arms' represented as: $[B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.005, 0.01, 0.015, 0.84, 0.85]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.005, 0.01, 0.015, 0.84, 0.85]
- with 'arms' = [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)]
- with 'means' = [0.005 0.01 0.015 0.84 0.85 ]
- with 'nbArms' = 5
- with 'maxArm' = 0.85
- with 'minArm' = 0.005
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- with 'arms' represented as: $[B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)^*]$
Number of environments to try: 3
Considering the list of players :
[CentralizedIMP(Thompson), CentralizedIMP(Thompson)]
Number of players in the multi-players game: 2
Time horizon: 10000
Number of repetitions: 100
Sampling rate for plotting, delta_t_plot: 1
Number of jobs for parallelization: 4
Using collision model onlyUniqUserGetsReward (function <function onlyUniqUserGetsReward at 0x7fc672469bf8>).
More details:
Simple collision model where only the players alone on one arm samples it and receives the reward.
- This is the default collision model, cf. [[Multi-Player Bandits Revisited, Lilian Besson and Emilie Kaufmann, 2017]](https://hal.inria.fr/hal-01629733).
- The numpy array 'choices' is increased according to the number of users who collided (it is NOT binary).
Using accurate regrets and last regrets ? True
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.3, 0.4, 0.5, 0.6, 0.7]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.3, 0.4, 0.5, 0.6, 0.7]
- with 'arms' = [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)]
- with 'means' = [0.3 0.4 0.5 0.6 0.7]
- with 'nbArms' = 5
- with 'maxArm' = 0.7
- with 'minArm' = 0.3
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- with 'arms' represented as: $[B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.1, 0.3, 0.5, 0.7, 0.9]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.1, 0.3, 0.5, 0.7, 0.9]
- with 'arms' = [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)]
- with 'means' = [0.1 0.3 0.5 0.7 0.9]
- with 'nbArms' = 5
- with 'maxArm' = 0.9
- with 'minArm' = 0.1
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- with 'arms' represented as: $[B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)^*]$
Creating a new MAB problem ...
Reading arms of this MAB problem from a dictionnary 'configuration' = {'arm_type': <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>, 'params': [0.005, 0.01, 0.015, 0.84, 0.85]} ...
- with 'arm_type' = <class 'SMPyBandits.Arms.Bernoulli.Bernoulli'>
- with 'params' = [0.005, 0.01, 0.015, 0.84, 0.85]
- with 'arms' = [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)]
- with 'means' = [0.005 0.01 0.015 0.84 0.85 ]
- with 'nbArms' = 5
- with 'maxArm' = 0.85
- with 'minArm' = 0.005
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- with 'arms' represented as: $[B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)^*]$
Number of environments to try: 3
CPU times: user 170 ms, sys: 20.5 ms, total: 191 ms
Wall time: 194 ms
Solving the problem¶
Now we can simulate the \(2\) environments, for the successive policies. That part can take some time.
[16]:
%%time
for playersId, evaluation in tqdm(enumerate(evs), desc="Policies"):
for envId, env in tqdm(enumerate(evaluation.envs), desc="Problems"):
# Evaluate just that env
evaluation.startOneEnv(envId, env)
# Storing it after simulation is done
evaluators[envId][playersId] = evaluation
Evaluating environment: MAB(nbArms: 5, arms: [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)], minArm: 0.3, maxArm: 0.7)
- Adding player # 1 = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 7.2s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 27.5s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.4min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)], minArm: 0.1, maxArm: 0.9)
- Adding player # 1 = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 10.9s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 47.0s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.7min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)], minArm: 0.005, maxArm: 0.85)
- Adding player # 1 = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 7.7s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 55.0s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.8min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)], minArm: 0.3, maxArm: 0.7)
- Adding player # 1 = #1<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedIMP(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedIMP(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 19.6s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 1.5min
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 3.6min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)], minArm: 0.1, maxArm: 0.9)
- Adding player # 1 = #1<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedIMP(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedIMP(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 23.3s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 1.6min
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 3.7min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)], minArm: 0.005, maxArm: 0.85)
- Adding player # 1 = #1<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #1<CentralizedIMP(UCB($\alpha=1$))> ...
- Adding player # 2 = #2<CentralizedIMP(UCB($\alpha=1$))> ...
Using this already created player 'player' = #2<CentralizedIMP(UCB($\alpha=1$))> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 16.0s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 1.6min
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 3.8min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)], minArm: 0.3, maxArm: 0.7)
- Adding player # 1 = #1<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(Thompson)> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 7.2s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 47.9s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.7min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)], minArm: 0.1, maxArm: 0.9)
- Adding player # 1 = #1<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(Thompson)> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 7.9s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 45.6s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.7min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)], minArm: 0.005, maxArm: 0.85)
- Adding player # 1 = #1<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #1<CentralizedMultiplePlay(Thompson)> ...
- Adding player # 2 = #2<CentralizedMultiplePlay(Thompson)> ...
Using this already created player 'player' = #2<CentralizedMultiplePlay(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 8.1s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 46.2s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.7min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.3), B(0.4), B(0.5), B(0.6), B(0.7)], minArm: 0.3, maxArm: 0.7)
- Adding player # 1 = #1<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #1<CentralizedIMP(Thompson)> ...
- Adding player # 2 = #2<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #2<CentralizedIMP(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 7.7s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 46.7s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.8min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.1), B(0.3), B(0.5), B(0.7), B(0.9)], minArm: 0.1, maxArm: 0.9)
- Adding player # 1 = #1<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #1<CentralizedIMP(Thompson)> ...
- Adding player # 2 = #2<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #2<CentralizedIMP(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 6.4s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 32.1s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 1.0min finished
Evaluating environment: MAB(nbArms: 5, arms: [B(0.005), B(0.01), B(0.015), B(0.84), B(0.85)], minArm: 0.005, maxArm: 0.85)
- Adding player # 1 = #1<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #1<CentralizedIMP(Thompson)> ...
- Adding player # 2 = #2<CentralizedIMP(Thompson)> ...
Using this already created player 'player' = #2<CentralizedIMP(Thompson)> ...
[Parallel(n_jobs=4)]: Using backend LokyBackend with 4 concurrent workers.
[Parallel(n_jobs=4)]: Done 5 tasks | elapsed: 3.8s
[Parallel(n_jobs=4)]: Done 42 tasks | elapsed: 26.9s
[Parallel(n_jobs=4)]: Done 100 out of 100 | elapsed: 57.2s finished
CPU times: user 2min 45s, sys: 913 ms, total: 2min 46s
Wall time: 27min 44s
Plotting the results¶
And finally, visualize them, with the plotting method of a EvaluatorMultiPlayers
object:
[24]:
def plotAll(evaluation, envId):
evaluation.printFinalRanking(envId)
# Rewards
evaluation.plotRewards(envId)
# Fairness
#evaluation.plotFairness(envId, fairness="STD")
# Centralized regret
evaluation.plotRegretCentralized(envId, subTerms=True)
#evaluation.plotRegretCentralized(envId, semilogx=True, subTerms=True)
# Number of switches
#evaluation.plotNbSwitchs(envId, cumulated=False)
evaluation.plotNbSwitchs(envId, cumulated=True)
# Frequency of selection of the best arms
evaluation.plotBestArmPulls(envId)
# Number of collisions - not for Centralized* policies
#evaluation.plotNbCollisions(envId, cumulated=False)
#evaluation.plotNbCollisions(envId, cumulated=True)
# Frequency of collision in each arm
#evaluation.plotFrequencyCollisions(envId, piechart=True)
[25]:
import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (12.4, 7)
First problem¶
\(\mu = [0.3, 0.4, 0.5, 0.6, 0.7]\) was an easy Bernoulli problem.
[26]:
for playersId in tqdm(range(len(evs)), desc="Policies"):
evaluation = evaluators[0][playersId]
plotAll(evaluation, 0)
Final ranking for this environment # 0 : CentralizedMultiplePlay(UCB($\alpha=1$)) ...
- Player # 2 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 6516.6).
- Player # 1 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 6346.1).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 9 ...
- For 2 players, our lower bound gave = 18 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 12.1 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- [Anandtharam et al] centralized lower-bound = 9,
- [Anandkumar et al] decentralized lower-bound = 12.1
- Our better (larger) decentralized lower-bound = 18,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 0 : CentralizedIMP(UCB($\alpha=1$)) ...
- Player # 2 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 6464.3).
- Player # 1 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 6400.6).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 9 ...
- For 2 players, our lower bound gave = 18 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 12.1 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- [Anandtharam et al] centralized lower-bound = 9,
- [Anandkumar et al] decentralized lower-bound = 12.1
- Our better (larger) decentralized lower-bound = 18,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 0 : CentralizedMultiplePlay(Thompson) ...
- Player # 1 / 2, CentralizedMultiplePlay(Thompson) was ranked 1 / 2 for this simulation (last rewards = 6495.3).
- Player # 2 / 2, CentralizedMultiplePlay(Thompson) was ranked 2 / 2 for this simulation (last rewards = 6394).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 9 ...
- For 2 players, our lower bound gave = 18 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 12.1 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- [Anandtharam et al] centralized lower-bound = 9,
- [Anandkumar et al] decentralized lower-bound = 12.1
- Our better (larger) decentralized lower-bound = 18,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 0 : CentralizedIMP(Thompson) ...
- Player # 2 / 2, CentralizedIMP(Thompson) was ranked 1 / 2 for this simulation (last rewards = 6550.7).
- Player # 1 / 2, CentralizedIMP(Thompson) was ranked 2 / 2 for this simulation (last rewards = 6346.3).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 9 ...
- For 2 players, our lower bound gave = 18 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 12.1 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- [Anandtharam et al] centralized lower-bound = 9,
- [Anandkumar et al] decentralized lower-bound = 12.1
- Our better (larger) decentralized lower-bound = 18,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Second problem¶
\(\mu = [0.1, 0.3, 0.5, 0.7, 0.9]\) was an easier Bernoulli problem, with larger gap \(\Delta = 0.2\).
[27]:
for playersId in tqdm(range(len(evs)), desc="Policies"):
evaluation = evaluators[1][playersId]
plotAll(evaluation, 1)
Final ranking for this environment # 1 : CentralizedMultiplePlay(UCB($\alpha=1$)) ...
- Player # 1 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 8059.3).
- Player # 2 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 7817.7).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 4.23 ...
- For 2 players, our lower bound gave = 8.46 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 5.35 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- [Anandtharam et al] centralized lower-bound = 4.23,
- [Anandkumar et al] decentralized lower-bound = 5.35
- Our better (larger) decentralized lower-bound = 8.46,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 1 : CentralizedIMP(UCB($\alpha=1$)) ...
- Player # 1 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 8184.8).
- Player # 2 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 7700).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 4.23 ...
- For 2 players, our lower bound gave = 8.46 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 5.35 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- [Anandtharam et al] centralized lower-bound = 4.23,
- [Anandkumar et al] decentralized lower-bound = 5.35
- Our better (larger) decentralized lower-bound = 8.46,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 1 : CentralizedMultiplePlay(Thompson) ...
- Player # 1 / 2, CentralizedMultiplePlay(Thompson) was ranked 1 / 2 for this simulation (last rewards = 8084.4).
- Player # 2 / 2, CentralizedMultiplePlay(Thompson) was ranked 2 / 2 for this simulation (last rewards = 7815.1).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 4.23 ...
- For 2 players, our lower bound gave = 8.46 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 5.35 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- [Anandtharam et al] centralized lower-bound = 4.23,
- [Anandkumar et al] decentralized lower-bound = 5.35
- Our better (larger) decentralized lower-bound = 8.46,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 1 : CentralizedIMP(Thompson) ...
- Player # 1 / 2, CentralizedIMP(Thompson) was ranked 1 / 2 for this simulation (last rewards = 8008.4).
- Player # 2 / 2, CentralizedIMP(Thompson) was ranked 2 / 2 for this simulation (last rewards = 7885.5).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 4.23 ...
- For 2 players, our lower bound gave = 8.46 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 5.35 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- [Anandtharam et al] centralized lower-bound = 4.23,
- [Anandkumar et al] decentralized lower-bound = 5.35
- Our better (larger) decentralized lower-bound = 8.46,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Third problem¶
\(\mu = [0.005, 0.01, 0.015, 0.84, 0.85]\) is an harder Bernoulli problem, as there is a huge gap between suboptimal and optimal arms.
[28]:
for playersId in tqdm(range(len(evs)), desc="Policies"):
evaluation = evaluators[2][playersId]
plotAll(evaluation, 2)
Final ranking for this environment # 2 : CentralizedMultiplePlay(UCB($\alpha=1$)) ...
- Player # 2 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 8399.9).
- Player # 1 / 2, CentralizedMultiplePlay(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 8392.3).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 1.41 ...
- For 2 players, our lower bound gave = 2.83 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 2.78 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- [Anandtharam et al] centralized lower-bound = 1.41,
- [Anandkumar et al] decentralized lower-bound = 2.78
- Our better (larger) decentralized lower-bound = 2.83,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 2 : CentralizedIMP(UCB($\alpha=1$)) ...
- Player # 2 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 1 / 2 for this simulation (last rewards = 8403.6).
- Player # 1 / 2, CentralizedIMP(UCB($\alpha=1$)) was ranked 2 / 2 for this simulation (last rewards = 8387.9).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 1.41 ...
- For 2 players, our lower bound gave = 2.83 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 2.78 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- [Anandtharam et al] centralized lower-bound = 1.41,
- [Anandkumar et al] decentralized lower-bound = 2.78
- Our better (larger) decentralized lower-bound = 2.83,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 2 : CentralizedMultiplePlay(Thompson) ...
- Player # 2 / 2, CentralizedMultiplePlay(Thompson) was ranked 1 / 2 for this simulation (last rewards = 8414.9).
- Player # 1 / 2, CentralizedMultiplePlay(Thompson) was ranked 2 / 2 for this simulation (last rewards = 8391).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 1.41 ...
- For 2 players, our lower bound gave = 2.83 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 2.78 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- [Anandtharam et al] centralized lower-bound = 1.41,
- [Anandkumar et al] decentralized lower-bound = 2.78
- Our better (larger) decentralized lower-bound = 2.83,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Final ranking for this environment # 2 : CentralizedIMP(Thompson) ...
- Player # 2 / 2, CentralizedIMP(Thompson) was ranked 1 / 2 for this simulation (last rewards = 8404.6).
- Player # 1 / 2, CentralizedIMP(Thompson) was ranked 2 / 2 for this simulation (last rewards = 8400.8).
Warning: forcing to use putatright = False because there is 2 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 1.41 ...
- For 2 players, our lower bound gave = 2.83 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 2.78 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- [Anandtharam et al] centralized lower-bound = 1.41,
- [Anandkumar et al] decentralized lower-bound = 2.78
- Our better (larger) decentralized lower-bound = 2.83,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Warning: forcing to use putatright = False because there is 2 items in the legend.
Comparing their performances¶
[29]:
def plotCombined(e0, eothers, envId):
# Centralized regret
e0.plotRegretCentralized(envId, evaluators=eothers)
# Fairness
e0.plotFairness(envId, fairness="STD", evaluators=eothers)
# Number of switches
e0.plotNbSwitchsCentralized(envId, cumulated=True, evaluators=eothers)
# Number of collisions - not for Centralized* policies
#e0.plotNbCollisions(envId, cumulated=True, evaluators=eothers)
[30]:
N = len(configuration["environment"])
for envId, env in enumerate(configuration["environment"]):
e0, eothers = evaluators[envId][0], evaluators[envId][1:]
plotCombined(e0, eothers, envId)
- For 2 players, Anandtharam et al. centralized lower-bound gave = 9 ...
- For 2 players, our lower bound gave = 18 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 12.1 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 9.46 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 60.00% ...
- [Anandtharam et al] centralized lower-bound = 9,
- [Anandkumar et al] decentralized lower-bound = 12.1
- Our better (larger) decentralized lower-bound = 18,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 4.23 ...
- For 2 players, our lower bound gave = 8.46 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 5.35 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 3.12 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 40.00% ...
- [Anandtharam et al] centralized lower-bound = 4.23,
- [Anandkumar et al] decentralized lower-bound = 5.35
- Our better (larger) decentralized lower-bound = 8.46,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
- For 2 players, Anandtharam et al. centralized lower-bound gave = 1.41 ...
- For 2 players, our lower bound gave = 2.83 ...
- For 2 players, the initial lower bound in Theorem 6 from [Anandkumar et al., 2010] gave = 2.78 ...
This MAB problem has:
- a [Lai & Robbins] complexity constant C(mu) = 27.3 for 1-player problem ...
- a Optimal Arm Identification factor H_OI(mu) = 29.40% ...
- [Anandtharam et al] centralized lower-bound = 1.41,
- [Anandkumar et al] decentralized lower-bound = 2.78
- Our better (larger) decentralized lower-bound = 2.83,
Warning: forcing to use putatright = False because there is 7 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
Warning: forcing to use putatright = False because there is 4 items in the legend.
That’s it for this demo!


CSPICE_KCLEAR clears the KEEPER system: unload all kernels, clears
the kernel pool, and re-initialize the system.
The call:
cspice_kclear
Re-initialize the KEEPER system.
Any numerical results shown for this example may differ between
platforms as the results depend on the SPICE kernels used as input
and the machine specific arithmetic implementation.
%
% Load the standard meta kernel, retrieve the number of
% loaded kernels.
%
cspice_furnsh( 'standard.tm' )
n = cspice_ktotal( 'ALL' );
txt = sprintf('Count of loaded kernels before cspice_kclear call: %d', n);
disp( txt )
MATLAB outputs:
Count of loaded kernels before cspice_kclear call: 4
The expected result counting standard.tm and the three kernels
named in the meta kernel.
%
% Clear the KEEPER system, retrieve the number of loaded
% after the clear.
%
cspice_kclear
n = cspice_ktotal( 'ALL' );
txt = sprintf('Count of loaded kernels after cspice_kclear call: %d', n);
disp( txt )
MATLAB outputs:
Count of loaded kernels after cspice_kclear call: 0
This routine allows you re-initialize the KEEPER system with
a single call. The KEEPER system is the kernel management system
underlying the set of Mice APIs
cspice_furnsh
cspice_ktotal
cspice_kdata
cspice_kinfo
cspice_kclear
cspice_unload
This routine unloads all kernels from their kernel-type-specific
kernel management subsystems (SPKBSR, CKBSR, etc.), clears the
kernel pool, clears KEEPER's internal file database, and re-sets
the watch status for the kernel variables used to load kernels
via meta-kernels.
This capability, though implemented in Fortran, is particularly
relevant to SPICE implementations such as Mice, for which the
state of the KEEPER system persists after any Mice-based MATLAB
script is run. Successive runs of Mice-based scripts may perform
in unexpected ways when scripts access data loaded during runs of
previous scripts.
Cleaning up after such programs using explicit unload_c commands is
tedious and error-prone. One call to this routine sets the
KEEPER system to its initial state, preventing unintentional
interaction between scripts via KEEPER's state.
For important details concerning this module's function, please refer to
the CSPICE routine kclear_c.
MICE.REQ
KERNEL.REQ
-Mice Version 1.0.0, 01-DEC-2006, EDW (JPL)
Re-initialize the keeper system
Clear the keeper system
Unload all kernels
|


We modified the earlier code slightly to make it buildable by r.js.
This is basically to check if things work fine after modifications.
/**
* If the file contains a requirejs local config
* the whole file is shimmed with an outer define.
* Hence the inner defined will not be executed
*/
// define(function(require) {
// 'use strict';
require([
"prism-remove-whitespace", // This automatically loads prism as well.
"css!node_modules/prismjs/themes/prism-coy",
], function() {
Prism.highlightAll(); // Known bug for prism: https://github.com/PrismJS/prism/issues/903
});
// });
// This package does not have a local requirejs.config
// Hence, it will not be shimmed. Define is required here or
// else the dependencies are not loaded in r.js build
define(function (require) {
"use strict";
require("./initSidebar");
});
// Only the global configs are given here. The rest are distributed as on need.
requirejs.config({
// paths are loaction independent. Use them to alias global dir/mod instead of map
// Using relative paths as absolute create problems in build
paths: {
"jquery": "../../../node_modules/jquery/dist/jquery",
"bootstrap": "../../../node_modules/bootstrap/dist/js/bootstrap",
"prism": "../../../node_modules/prismjs/prism",
"prism-remove-whitespace": "../../../node_modules/prismjs/plugins/normalize-whitespace/prism-normalize-whitespace",
"node_modules": "../../../node_modules", // added to fix CSS build issue. See **1**
},
shim: {
"bootstrap": {
deps: ["jquery"],
exports: "$",
},
},
map: {
'*': {
"stylesheets": "../../stylesheets", // alias path name. Better give in paths.
},
// SEE: You can give configurations for specific modules too
"sidebar": {
"template": "../../../template", // alias path name. Better give in paths.
},
// **1** Sounds good. Doesn't work. Some issue with CSS loader plugin.
// R.js generates different ID's in built
// "coder": {
// "node_modules": "../../../../node_modules"
// }
},
// it will look for -> baseUrl/package-name/main.js
// moved css and text loader plugin to packages to resolve various build issues
packages: ["blocks", "coder", "sidebar", {
name: 'css',
location: '../../../node_modules/require-css/',
main: 'css.min'
}, {
name: 'text',
location: '../../../node_modules/text/',
main: 'text'
}],
// Enabling this a great way to check for correct shims
// enforceDefine: true,
});
// Global imports
require([
"css!stylesheets/simple-sidebar",
"css!node_modules/bootstrap/dist/css/bootstrap.min",
]);
// Load the packages
require(["blocks", "coder", "sidebar"]);
Toggle Menu


FEATURED STORIES
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
TOP STORY
Democratic Defectors Unable to Prevent Impeachment of Trump by House
The defectors weren't enough to stop the House from impeaching, as Democrats corralled support from all corners of their caucus, including vulnerable members from Trump districts.
OPINION
BY WAI WAI NU
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
Chinese Media: Trump Impeachment Shows 'Flaws of Western-Style Democracy'
"No politicians would be ashamed of making false statements for their own interests," an op-ed by The Global Times argued.
If Iran Falls, ISIS May Rise Again
When Young Girls Are Forced to Go to War
Failing to Make the Grade: Would More Money Save Students And Schools?
Jennifer Hudson on Playing Grizabella in 'Cats'
FLIP SIDE
After Sarah Sanders Mocks Joe Biden, Stuttering Association Hits Back
The National Stuttering Association took to social media to condemn the former White House press secretary for taking jabs at Joe Biden's imitation of a young boy's stutter during a debate.
Record-Breaking Heat Melts Roads in Parts of Australia
A heatwave across Australia has exacerbated bush fires and South Australia is enduring a fourth successive day of record temperatures.
Ring Camera Hack: How to Know If Your Home Camera Is Compromised
"Two-factor authentication is more than needed," a security researcher told Newsweek. "People have to understand that they will not have any privacy with this kind of product."
Nancy Pelosi Says Trump 'Impeached Forever' and Shown 'Power of the Gavel'
House Speaker Nancy Pelosi told the Associated Press that President Donald Trump was impeached "no matter what" happens in the Senate.
Megyn Kelly Says Trump Impeachement Trial Is 'Rigged' in Documentary
"I don't see enough Republicans in that body turning on a man who's about to face re-election anyway," she said.
Ebola Virus Vaccine 'A Triumph of American Global Health Leadership'
The virus killed more than 11,000 people during the 2014 outbreak in West Africa.
Tiny Replica of Michelangelo's David Created by Scientists Using 3D Printer
The smallest of the statues measures less than one millimeter in height.
Trump's GOP 2020 Challengers Respond to Impeachment: 'Not at All Surprised'
"We knew he was going to get impeached and he deserved to be impeached."


France's Le Maire favors car industry professional as next Renault CEO
French Finance Minister Bruno Le Maire said on Sunday that the search for a new CEO for Renault was still on and that he favored selecting a professional from the automobile sector.
PARIS: French Finance Minister Bruno Le Maire said on Sunday that the search for a new CEO for Renault was still on and that he favored selecting a professional from the automobile sector.
"I can confirm the next CEO has not yet been chosen. The only indication I can give you is that I wish it is a professional of the car industry because amid the radical transformation of the sector, one needs professionals," Le Maire told BFM television.
The French carmaker, chaired by Jean-Dominique Senard, a former executive at tire maker Michelin who was parachuted in following the Carlos Ghosn scandal, is expected to choose a new CEO by the end of the year so that the group can try and fully refocus on its operations.
The French state has a 15per cent stake in Renault.
Renault's interim chief executive Clotilde Delbos has applied to take the job on a permanent basis, two sources familiar with the matter have said, as the carmaker edges toward a shortlist likely to also feature several external candidates.
One of the sources had said that Patrick Koller, the Franco-German CEO of car parts maker Faurecia , and Luca de Meo, the Italian boss of Volkswagen-owned SEAT, also ticked many of the boxes for recruiters, including that they both spoke French.
(Reporting by Dominique Vidalon; Editing by Susan Fenton)


Fraudulent schemes cost businesses in Canada millions of dollars per year. Although larger corporations have a greater potential of recurring fraud, it is puzzling to discover that smaller businesses still remain more susceptible. Small businesses, moreso known to be insufficient in major precautionary measures, experience more fraudulent claims due to employees looking to obtain unfair or unlawful personal gain. As these businesses begin seeking assistance, private investigation agencies are contacted to come to their rescue.
Surveillance is used in investigations for fraud as a practice of tracking suspicious activity. It is the most effective method of fastening a business’ security and determining the roots of its fraudulent claims. A considerable benefit of allowing a private investigation agency to direct the surveillance in a business is the level of professional training that their investigators have received. These highly qualified individuals are not only able to gather evidence that tends to stand out in court, but are also capable of doing so without being detected. A wide range of approaches can be taken to implement surveillance in the workplace. Common methods generally involve the use of electronic and/or physical observance and are known to be the most efficient in gathering significant data about suspect behaviour.
An essential procedure that can be implemented in the recruitment phase to prevent fraud in its entirety is the conduction of background checks.Thorough background checks go far beyond simply analyzing a potential candidate’s criminal record. You want to be certain that you hire someone who can thrive in their role. Thus, private investigators are often hired to conduct preliminary research on potential candidates looking to join a business. A certified private investigator is knowledgeable of the specificities he/she needs to seek and is able to properly verify an applicant's personal history without posing any violations.
Undercover investigations are generally unnecessary in most cases of fraud. However, when they are required, allowing private investigation agencies to mediate these operations is always the most desirable option. These types of investigations are broadly utilized under strict circumstances where businesses encounter any large, unexpected losses (i.e. inventory loss). Attempting to unravel what triggered these situations is made possible primarily through the use of undercover operations, where specially trained private investigators are able to mask their identity (to avoid detection) in an effort to observe the suspicious activity of those accused/suspected.
Today’s society belongs to an ever so rapidly advancing technological world. As a result, a growing quantity of electronically operated fraudulent schemes are being utilized daily. Thus, building expertise to bridge the gap between law and computer science is a very beneficial skill to have. Devices including DVDs, USBs, phones, computers, and even printers are all being used to store unauthorized information capable of harming the reputation of a business. Although IT departments can be very valuable in such situations, they are often not specialized enough to analyze heavily encrypted information and transform it appropriately for use in court. Private investigation companies offer computer forensic specialists who are highly trained to detect and extract “hidden” information. These individuals are able to combat fraud in the workplace by retrieving information from devices without changing their state (or harming them) and recovering deleted (or damaged) files.
Fraud often leads to the discussion of litigation support. Defendants often take legal action against businesses that choose to accuse them of participating in fraudulent activities. Private investigation agencies have long-term connections with law firms and are able to act as mediators that serve the relationship between businesses and legal support. Most of these agencies also offer experienced paralegal staff that are able to assist businesses with complex investigations and provide legal assistance with frauds of all types.
 It is one thing to recognize these schemes, and another to overcome them. With the help of private investigation agencies, businesses benefit from the guarantee of having their entire investigative process conducted in compliance with current laws and codes of conduct; further legal complications are, thereby, prevented.
“Background Checks.” Legal Line, Legal Line, https://www.legalline.ca/legal-answers/background-checks/.
“Investigative Services.” CHRIS MATHERS INC., http://www.chrismathers.com/investigative.
“Litigation Support.” Litigation Support through Private Investigation, Fast Track Investigations, http://www.fasttrackinvestigations.ca/litigation-support/.
Sulwski, Tom, and David James. “Small Businesses Are More Vulnerable to Fraud.” Puget Sound: Business Journal, American City Business Journals, 23 May 2004, https://www.bizjournals.com/seattle/stories/2004/05/24/smallb3.html.
“Surveillance Investigation.” PInow.com - Find Local Investigators, PInow, https://www.pinow.com/investigations/surveillance.


Luckily for developers, there are MANY Vim plugins available for installation that will make your Vim experience much more enjoyable. In fact, the various plugins available for Vim is one of the reasons that Vim is so well-loved by developers. No matter what you're looking for, you can probably find a plugin that will help you. These plugins are excellent for solving workflow issues, customization, and more. Of course, one thing to keep in mind when using plugins is that plugins are very good at preventing your thorough understanding of Vim and it's functionality. It's easy to get lost in the ease that many of these plugins provide for you, so just be weary which plugins you decide to use.
Here is a list of some plugins that users have found useful. Of course, there are many, many more plugins out there, so be sure to search for more that suit your needs!
First and foremost, you may need a plugin to manage your plugins. That's what Vundle can do for you. Short for "Vim Bundle", Vundle helps you keep track of all the plugins you install, configure these plugins, and install the configured plugins. Additionally, it enables you to search for Vim scripts by name and sort out which of your plugins are consistently unused.
Syntastic is a simple syntax checker for Vim. It highlights any syntax problems and shows errors without the need for compiling the code or executing the script. The linting works both on demand and automatically upon saving.
Ctrl-P is a Vim plugin that helps you find files even if you don't remember the name or full file path. All you have to do is enter some characters that you can remember from the file path. From there, the plugin will find any matches and sort them by best-guess.
Although Vim has a built-in directory browser, NERDTree allows you to navigate through files much more easily. The plugin will open your current directory, and from there you can browse through your projects and files. It also enables you to create files, delete files, and perform other basic file management tasks.
Tagbar is a plugin that helps you browse through any tags in your source code files. For example, if you are developing Java code, Tagbar displays a sidebar that lists the various classes in your code and what methods are defined under these classes.
EasyMotion is a plugin that allows you to traverse through your files
in a much quicker way. By using the
Multiple Cursors is a Vim plugin that enables you to edit multiple sections of code at once. For instance, if you want to change the same word in multiple parts of your file, Multiple Cursors can be used by leaving your cursor over a word in normal mode, typing Ctrl-n, and highlighting the instances of that word that you wish to change. From there, whatever you type will be edited in the instances you selected.
Additionally, the site Vim Awesome provides links to downloads of many useful plugins. In addition to plugins like those listed above there a plenty for customization (like color scheme plugins), syntax highlighting/assistance with various programming languages, and of course, much more.


Back
Dibdaa Product Details Report Item
Item no: {{data_item?.PROSKU}}
Seller: {{data_item?.SellerName}}
The listing has wording that questions the authenticity or legality of an item.
Before reporting an item, please review the Dibdaa Listings Policies.


fun registerOption(option: Option): Unit
Register an option with this command.
This is called automatically for the built in options, but you need to call this if you want to add a custom option.


Click on any location in the map, or move the marker around, in order to get the latest social media content in that area.


NOTE: The Instrumentation features are new in Evothings Studio 2.1.0, currently available as an early release for testers and enthusiasts - download Evothings Studio 2.1.0-alpha.
What we’re rolling out as part of this alpha release is two new features that we hope will make your debugging simpler and your development quicker; instrumentation and file injections.
To access both features, you now need to open a new tool window, by clicking on the ‘Viewers’ button at the far right in the toolbar of Evothings studio. You will also need at least version 1.3.0 of the Evothings Viewer App and a stable internet cnnection to fully utilize all new features!
The window will list all connected viewers and the make of the phone they’re running on. Feel free to click around exploring what you can do!
The tricky part of developing mobile apps is that your code by definition runs on another device than the one you’re sitting at. Logging, for one thing. Since the beginning we’ve had support for sending logs back to the studio, by using the hyper.log command.
What we’re rolling out as part of this alpha release is two new features that we hope will make your debugging simpler and your development quicker; instrumentation and file injections.
To access both features, you now need to open a new tool window, by clicking on the ‘Viewers’ button.
The window will list all connected viewers and the make of the phone they’re running on.
If you have several phones of the same make, you can identify which one you are working with by clicking on the phone icon, which will trigger a vibration on the device (if supported).
Let’s start by looking at injection of files. This feature have two different modes. Either files can be transferred as regular JavaScript files, and be evaluated/executed in the phone’s cordova webview environment directly after transfer, or the file (which in this case can be of any type) is encoded in base64, transferred and then a callback function will be called in the phone with the file data, if present.
For the second option, you can for example load an app that defines the array window.evo.fileCallbacks, and adds a function to the array, like this;
window.evo.fileCallbacks = []
Var cb = function(file)
{
hyper.log(‘Yohoo! Got file callback fired for file ‘+file.name)
}
window.evo.fileCallbacks.push(cb)
Then you can enter the Viewers tools window, open the ‘Inject File(s)’ menu and inject a file with Base64 encoding, which will trigger your callback function after the file has been uploaded.
We envision that this feature might be useful if you are using a web app on a phone to deliver OTA (over The Air) updates for devices over BLE, and if you already have a mechanism for getting updates to the devices from the phone.
You work with this feature by adding a folder somwhere under your project directiry named '_filesync'. You must then make sure that the project is run (loaded) on your mobile phone viewers. After that, any changed file under that _filesync directory will be send to the phone is the same manner as described above, so you can register listeners for file transfers for non-js files (or make sure that any javaScript files sent are able to call any custom registration functions of your themselves.
A special case is when you are developing a ES6 project, where you have a source and a build directory. Currently you must place the _filesync directory under the folder named 'app' in your project directory. We aim in the future to make this placement more flexible.
What happen is that a number of files that resides in the directory ‘hyper/injectables/’ was transferred to the phone and evaluated. When a phone has instrumentation loaded, the circle above its icon will change from grey to green, and the registered instrumentation modules will be shown.
Included in this release are three modules.
Cordova - A sample instrumentation module which can be used as a starter kit for your own, custom instrumentation. It shows how to build a small hierarchy that can be selected and subscribed to. Check the file ‘hyper/injectables/cordova-instrumentation.js’ in the Evothings studio installation directory for more details.
Cordova exposes two subscribable services; accelerometer and compass. Selecting either will send a stream of updates from the phone to the studio, which will be interpreted as a timeline graph.
All subscription of module services data streams sends any changes every 300 ms and will stop after 10 minutes, after which you will have to restart it. By clicking on the output of a subscription you immediately unsubscribe from it.
Bluetooth - This module is a simple Bluetooth scanner which let you see which BLE devices your phone can see. If a device have exposed service characteristics, these can be further drilled into.
Watches - Sometimes you need to watch for property changes in your app, but not as a graph. The watch does this for you, and essentially monitors a named property of an object and send any changes to the studio.
You can add watches to your app by having instrumentation loaded and then execute;
window.foo = 17
window.evo.watcher.watch('myWatch', window, 'foo', 'noplot')
Which will result in a new watch called ‘myWatch’ showing under the ‘watches’ module
If you change the value of window.foo to something else, this will immediately be reflected, for example if you execute in your app
window.foo = 4711
You will see
Another useful feature is the select branch of the watch module. If you click that, you can drill down into all properties under the cordova webview, and choose which you want to add as watch.
If you select window.orientation, you will get a new watch called ‘orientation’, showing the current value of that property.
If you now turn your phone sideways, the value will change to reflect the orientation of the phone.
If you would like to add your own watches, but as graphical plots, you can do that too. If instrumentation is loaded, you can execute;
window.xyzzy = 1
timer = setInterval(function() { window.xyzzy = Math.random()*10 }, 500)
window.evo.watcher.watch('myPlot', window, 'xyzzy', 'plot')
If you want to try it out, you can paste the above text into the Tools window editor and evaluate it on the client, which will add a new watch called ‘myPlot’ under your watches.
'This is the alpha-release of instrumentation, so we expect some bugs and possible API changes in the future. We do not recommend that you use instrumentation for production use at the moment.
However, we strongly encourage you to play with it as much as you can, tweak it, break it and make up new uses for it.
Please let us know what you are doing with it and what features you’d like to see next.
A great way to get questions answered is our gitter forum, which can be found here: https://gitter.im/evothings/evothings
Download Evothings Studio 2.1.0-alpha and get started within minutes. It is fun and easy!
Ask questions and discuss IoT app development on Gitter: gitter.im/evothings/evothings


HierDmap is designed to visualize the biodiversity (especially for genetic diversity) in hierarchies and plot the geo-diversity distribution/density map. In the hierarchy plot, the circle (node) represents a aggregate at a certain hierarchical level and the circles inside of it (sub-nodes) represents sub-aggregate at a low level. The size of each aggregate is proportional to the diversity value.
Install the most recent version of the HierDmapC package using devtools:
library("devtools")
devtools::install_github("xinghuq/HierDmap")
Alternatively, you can install from the source files, run the following commands in the shell:
R CMD build HierDmap
R CMD check --as-cran HierDmap_0.5.0.tar.gz
R CMD INSTALL HierDmap_0.5.0.tar.gz
Before install or during installation, make sure the below dependences are installed.
if (!requireNamespace("devtools", quietly = TRUE))
utils::install.packages("devtools")
if (!requireNamespace("circlepackeR", quietly = TRUE))
devtools::install_github("jeromefroe/circlepackeR")
Qin. X. 2020. HierDmap: Visualizing the geo-biodiversity profiles and the hierarchical structure of biodiversity. R package version 0.5.0.
Qin, X. (2019). HierDpart: partitioning hierarchical diversity and differentiation across metrics and scales, from genes to ecosystems. R package version 0.5. 0 https://cran.r-project.org/package=HierDpart.


SapphireDb requires knowledge about the primary keys a model has. By default it is assumed that the primary always is called
id
. You can configure it by defining a custom model class on client side and configure the collection to use this class type.


--- layout: page title: Contact date: 2010-03-01 00:00:04.000000000 +00:00 type: page published: true status: publish categories: [] tags: [] meta: _wp_page_template: default _edit_last: '1' author: David Simpson excerpt_separator: ---
You can contact me via:
Download my vcard.
Or use this form:


{% extends "base.html" %} {% block content %}
“PAGE NOT FOUND” — is probably the most annoying thing to look at, on a screen. This is a page nobody wants to land into. For the longest time this page echoed the words ‘disappointment’ and had always been notoriously referred to as the ‘last page of the internet’. The idea of 404 came about from, you guessed it, a room numbered 404 that housed the web’s very first servers at CERN.


Welcome to Cycle2U
Cycle2U is a new, innovative, premium mobile bicycle servicing, repairs and maintenance workshop, recognising the importance of customers needs. Avoid the hassle involved in transporting your bike to and from bike shops in business hours and waiting days to retrieve at a later date. WE COME TO YOU! We specialise in road bicycles and will visit most suburbs around Adelaide, to carry out servicing, custom build ups and minor repairs. At home or office, WHEN IT SUITS YOU!
Complete Servicing Package
Specialising in road bicycles, I will visit most suburbs around Adelaide to carry out servicing, custom build ups and minor repairs. At home or office, when it suits you! A self sufficient set up allows me to remain on site all day if needed.
We offer a group booking discount when three or more bikes are serviced together.
Stocking a basic range of genuine top quality parts:
- Tyres
- Tubes
- Inner and Outer cables
- Brake Pads
- Handle Bar Tapes
- and much more!