| | |
- __builtin__.object
-
- Scanner
class Scanner(__builtin__.object) |
| |
A simple class to aid in lexical analysis of a string.
Styled after, but not entirely the same as, Ruby's StringScanner class.
Basic philosophy is simple: Scanner traverses a string left to right,
consuming the string as it goes. The current position is the string pointer
Scanner.pos. At each iteration, the caller uses the scanning methods to
determine what the current piece of string actually is.
Scanning methods:
With the exception of get and peek, all scanning methods take a pattern and
(optionally) flags (e.g re.X). The patterns are assumed to be either
strings or compiled regular expression objects (i.e. the result of
re.compile, or equivalent). If a pattern is not a string but does not
implement match or search (whichever is being used), a ValueError is raised.
String patterns are compiled and cached internally.
The check, scan and skip methods all try to match *at* the current scan
pointer. check_to, scan_to and skip_to all try to find a match somewhere
beyond the scan pointer and jump *to* that position. check_until, scan_until,
and skip_until are like *_to, but also consume the match (so the jump to
the *end* of that position)
Lookahead:
check()
check_to()
check_until()
peek()
Consume:
get()
scan()
scan_to()
scan_until()
skip()
skip_to()
skip_until()
skip_bytes() (convenience wrapper)
skip_lines() (convenience wrapper)
skip_whitespace() (convenience wrapper)
Note that scan* and check* both return either a string, in the case of a
match, or None, in the case of no match. If the match exists but is zero
length, the empty string is returned. Be careful handling this as both
None and the empty string evaluate to False, but mean very different things.
peek and get also return the empty string when the end of the stream is
reached.
Most recent match data:
matched() -- True/False - was the most recent match a success?
The following methods all throw Exception if not matched()
match() -- matched string
match_len() -- matched string length
match_pos() -- offset of match
Wrappers around re.*
match_info() -- the re.MatchObject
match_group()
match_groups()
match_groupdict()
pre_match() -- string preceding the match
post_match() -- string following the match
Misc:
pos -- get/set current scan pointer position
bol() -- beginning of line? (DOS/Unix/Mac aware)
eol() -- end of line? (DOS/Unix/Mac aware)
eos() -- end of string?
rest() -- remaining (unconsumed) string
rest_len() -- length of remaining string
unscan() -- revert to previous state
Setup:
string -- get/set current source string
reset() -- reset the scanner ready to start again
terminate() -- trigger premature finish |
| |
Methods defined here:
- __init__(self, src=None)
- Constructor
Arguments:
src -- a string to scan. This can be set later by string()
- bol(self)
- Return whether or not the scan pointer is immediately after a newline
character (DOS/Unix/Mac aware), or at the start of the string.
- check(self, pattern, flags=0)
- Return a match for the pattern (or None) at the scan pointer without
actually consuming the string
If the pattern matched but was zero length, the empty string is returned
If the pattern did not match, None is returned
- check_to(self, pattern, flags=0)
- Return all text up until the beginning of the first match for the pattern
after the scan pointer without consuming the string
If the pattern matched but was zero length, the empty string is returned
If the pattern did not match, None is returned
- check_until(self, pattern, flags=0)
- Return all text up until the end of the first match for the pattern
after the scan pointer without consuming the string
If the pattern matched but was zero length, the empty string is returned
If the pattern did not match, None is returned
- eol(self)
- Return whether or not the scan pointer is immediately before a newline
character (DOS/Unix/Mac aware) or at the end of the string.
- eos(self)
- Return True iff we are at the end of the string, else False.
- exists(self, pattern, flags=0)
- Return True if the given pattern matches ANYWHERE after the scan
pointer. Don't advance the scan pointer
- get(self, length=1)
- Return the given number of characters from the current string pointer
and consume them
If we reach the end of the stream, the empty string is returned
- location(self)
- Return as a tuple: (linenumber, bytenumber)
- match(self)
- Return the last matching string
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- match_group(self, *args)
- Return the contents of the given group in the most recent match.
This is a wrapper to re.MatchObject.group()
raise IndexError if the match exists but the group does not
raise Exception if no match attempts have been recorded
raise Exception if most recent match failed
- match_groupdict(self, default=None)
- Return a dict containing group_name => match. This is a wrapper to
re.MatchObject.groupdict() and as such it only works for named groups
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- match_groups(self, default=None)
- Return the most recent's match's groups, this is a wrapper to
re.MatchObject.groups()
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- match_info(self)
- Return the most recent match's MatchObject. This is what's returned by
the re module. Use this if the other methods here don't expose what you
need.
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- match_len(self)
- Return the length of the last matching string
This is equivalent to len(scanner.match()).
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- match_pos(self)
- Return the offset into the string of the last match
Raise Exception if no match attempts have been recorded.
Raise Exception if most recent match failed
- matched(self)
- Return True if the last match was successful, else False.
Raise Exception if no match attempts have been recorded.
- peek(self, length=1)
- Return the given number of characters from the current string pointer
without consuming them.
If we reach the end of the stream, the empty string is returned
- post_match(self)
- Return the string following the last match or None. This is equivalent
to: scanner.string[scanner.match_pos() + scanner.match_len() : ]
raise Exception if no match attempts have been recorded
- pre_match(self)
- Return the string preceding the last match or None. This is equivalent
to: scanner.string[:scanner.match_pos()]
raise Exception if no match attempts have been recorded
- reset(self)
- Reset the scanner's state including string pointer and match history.
- rest(self)
- Return the string from the current pointer onwards, i.e. the segment of
string which has not yet been consumed.
- rest_len(self)
- Return the length of string remaining.
This is equivalent to len(rest())
- scan(self, pattern, flags=0)
- Return a match for the pattern at the scan pointer and consume the
string.
Return None if not match is found
- scan_to(self, pattern, flags=0)
- Return all text up until the beginning of the first match for the pattern
after the scan pointer.
The pattern is not included in the match.
The scan pointer will be moved such that it immediately precedes the pattern
Return None if no match is found
- scan_until(self, pattern, flags=0)
- Return the first match for the pattern after the scan pointer and
consumes the string up until the end of the match.
Return None if no match is found
- skip(self, pattern, flags=0)
- Scan ahead over the given pattern and return how many characters were
consumed, or None.
Similar to scan, but does not return the string or record the match
- skip_bytes(self, n)
- Skip the given number of bytes and return the number of bytes consumed
- skip_lines(self, n=1)
- Skip the given number of lines and return the number of lines consumed
- skip_to(self, pattern, flags=0)
- Scan ahead until the beginning of first occurrance of the given pattern
and return how many characters were skipped, or None if the match
failed
The match is not recorded.
- skip_until(self, pattern, flags=0)
- Scan ahead until the end of first occurrance of the given pattern and
return how many characters were consumed, or None if the match failed
The match is not recorded
- skip_whitespace(self, n=None, multiline=True)
- Skip over whitespace characters and return the number of characters
consumed
Arguments:
n -- maximum number of characters to cosume (default None)
multiline -- whether or not to consume newline characters (default True)
- terminate(self)
- Set the string pointer to the end of the input and clear the match
history.
- unscan(self)
- Revert the scanner's state to that of the previous match. Only one
previous state is remembered
Throw Exception if there is no previous known state to restore
Data descriptors defined here:
- __dict__
- dictionary for instance variables (if defined)
- __weakref__
- list of weak references to the object (if defined)
- pos
- The current string pointer position.
- string
- The source string
| |


Documentation¶
Graphs¶
A set graph (collection of unconnected vertices). |
|
A hypercube lattice of side L in d dimensions. |
|
A generic lattice built translating a unit cell and adding edges between nearest neighbours sites. |
Hilbert¶
Hilbert space composed of bosonic states. |
|
A custom hilbert space. |
|
Hilbert space composed of spin states. |
Operators¶
A Bose Hubbard model Hamiltonian operator. |
|
A custom graph based operator. |
|
A custom local operator. |
|
Constructs a new |
|
Constructs a new |
Optimizer¶
AdaDelta Optimizer. |
|
AdaGrad Optimizer. |
|
AdaMax Optimizer. |
|
Momentum-based Optimizer. |
|
RMSProp optimizer. |
|
Simple Stochastic Gradient Descent Optimizer. |
Machines¶
A fully connected Restricted Boltzmann Machine (RBM). |
|
A fully connected Restricted Boltzmann Machine with lattice symmetries. |
|
A fully connected Restricted Boltzmann Machine (RBM) with real-valued parameters. |
|
A fully connected Restricted Boltzmann Machine (RBM) with real-valued parameters. |
|
A fully connected Restricted Boltzmann Machine for handling larger local Hilbert spaces. |
|
A feedforward neural network (FFNN) Machine. |
|
A Jastrow wavefunction Machine. |
|
A Jastrow wavefunction Machine with lattice symmetries.This machine defines the wavefunction as follows: |
|
Samplers¶
Sampler acting on one local degree of freedom. |
|


YOU
{{ userHealth }}
MONSTER
{{ monsterHealth }}
START NEW GAME
ATTACK
SPECIAL ATTACK
HEAL
GIVE UP
{{ log.text }}


Enum AccountActivityType
Types of account activities
Namespace: Alpaca.Markets
Assembly: Alpaca.Markets.dll
Syntax
[JsonConverter(typeof(StringEnumConverter))]
public enum AccountActivityType
Fields
| Name | Description |
|---|---|
| ACATCash | ACATS IN/OUT (Cash) |
| ACATSecurities | ACATS IN/OUT (Securities) |
| CashDisbursement | Cash disbursement(+) |
| CashReceipt | Cash receipt(-) |
| Dividend | Dividends |
| DividendCapitalGainsLongTerm | Dividend (capital gains long term) |
| DividendCapitalGainsShortTerm | Dividend (capital gain short term) |
| DividendFee | Dividend fee |
| DividendForeignTaxWithheld | Dividend adjusted (Foreign Tax Withheld) |
| DividendNRAWithheld | Dividend adjusted (NRA Withheld) |
| DividendReturnOfCapital | Dividend return of capital |
| DividendTaxExempt | Dividend (tax exempt) |
| DividendTefraWithheld | Dividend adjusted (Tefra Withheld) |
| Fill | Order fills (both partial and full fills) |
| Interest | Interest (credit/margin) |
| InterestNRAWithheld | Interest adjusted (NRA Withheld) |
| InterestTefraWithheld | Interest adjusted (Tefra Withheld) |
| JournalEntry | Journal entry |
| JournalEntryCash | Journal entry (cash) |
| JournalEntryStock | Journal entry (stock) |
| MergerAcquisition | Merger/Acquisition |
| Miscellaneous | Miscellaneous or rarely used activity types (All types except those in TRANS, DIV, or FILL) |
| NameChange | Name change |
| PassThruCharge | Pass Thru Charge |
| PassThruRebate | Pass Thru Rebate |
| Reorg | Reorganization |
| StockSpinoff | Stock spinoff |
| StockSplit | Stock split |
| SymbolChange | Symbol change |
| Transaction | Cash transactions (both CSD and CSR) |


Celtics' Kemba Walker leaves game on stretcher after colliding with teammate
by Tom GattoCeltics point guard Kemba Walker is suffering from concussion-like symptoms after colliding with teammate Semi Ojeleye on Friday night in Denver.
The team announced Walker's condition some time after Walker was taken off the court on a stretcher.
ESPN's Adrian Wojnarowski, citing an anonymous league source, reported that it is not believed Walker suffered a serious injury.
Celtics coach Brad Stevens gave good news after the game when speaking with reporters.
MORE: NBA players offer support, well-wishes for Walker
Walker appeared to stumble headfirst into Ojeleye's midsection while trying to secure a steal in the second quarter vs. the Nuggets.
Walker, 29, came into Friday's game averaging 22.6 points, 4.8 assists and 4.9 rebounds for the Celtics. Boston lost 96-92 to fall to 11-4 for the season.
This article has been updated.


--- title: fibreODF sidebar: function_reference_sidebar permalink: fibreODF.html folder: function_reference toc: false ---
defines an fibre symmetric ODF
fibreODF defines a fibre symmetric ODF with respect to a crystal direction h
and a specimen directions r
. The shape of the ODF is defined by a S2Kernel function.
| h | Miller / vector3d crystal direction |
| r | vector3d specimen direction |
| CS, SS | crystal, specimen symmetry |
| hw | halfwidth of the kernel (default -- 10°) |
| psi | S2Kernel function (default -- S2 de la Vallee Poussin) |
| odf | SO3Fun |


Don't Lose Yourself
So, before I went to Coding Bootcamp there was a long road that my wife and I had to traverse to set ourselves up financially to go down to one income while paying bootcamp tuition. We decided that the best way to accomplish this goal was to have me take a night management job at a new hotel to increase our income. The differential was nice and definitely made a big impact on how quickly we were able to reach our financial goals. But it also meant we were working opposite schedules.
So, a year working as a night manager, three months of bootcamp, and now almost three months doing everything I can to find a job.
A year and a half, eighteen months, seventy-seven weeks, five hundred and thirty-nine days.
We have been relentlessly focused on this goal the entire time. But, what about us? What about our individuality? Our hobbies? The things we did before we switched gears to achieve this goal. Everything fell by the wayside. I used to write, a lot. I was working on the second draft of my first novel and I was working on the first draft of my second novel. I used to draw, write and illustrate comic strips with my wife, fabricate cosplay armor, painting, photography excursions, we even dabbled in Special Effects Makeup. She was all about DIY stuff, crafting, and creating all kinds of things. But, we set all of that aside to achieve this goal.
I have found myself CRAVING an opportunity to draw. But, by the time I finish my schedule for the day, I’m just kinda wiped out. Coding can be tedious, slamming your head against a wall for hours trying to fix a bug only to realize it was your indentation or you put a return statement in the wrong place. It can be mentally taxing. Finding artistic inspiration when your brain is fried is not easy.
So, I think I might add something to my schedule in the coming weeks. Setting aside time to enjoy the other things that I love. To spend more time with my wife doing the things that we used to do before we put our blinders on.
I’m also going to be updating my About Me page. I want to create some additional pages to house some of my other artistic ventures. A page dedicated to paintings and drawings I have laying around. A page dedicated to SFX makeup my wife and I have done together. A page set up for the comic strips we wrote and illustrated. And, I want to put a page together for my photography.
We are so close to achieving every aspect of the goal we set for ourselves. It is time to step out of that bubble and start enjoying life again.
How about you? What goals do you have for yourself? What are you currently working on? Have you set yourself up for success? What do you need to do to reach those goals? If we can get ourselves this far, so can you.
So get out there. Get going. If you need motivation, click the link below.
Click Here For Motivation


--- layout: post title: 'Eric Evans: What I''ve learned about DDD since the book' date: 2009-06-24 10:37:02.000000000 -06:00 type: post published: true status: publish categories: - Building Blocks - Event Sourcing - Learning DDD - Library - Strategic Design - Video tags: [] meta: _gcid: '151' _gcuser: Vladimir Gitlevich content_type: flashnode Summary: '2' _bitly_trim: http://dddcommunity.org/?p=73 _edit_last: '3110' custom_permalink: library/evans_2009_1/ _dependencies: s:6:"a:0:{}"; _custom_background: a:5:{s:3:"url";s:0:"";s:16:"background-color";s:0:"";s:17:"background-repeat";s:6:"repeat";s:21:"background-attachment";s:6:"scroll";s:19:"background-position";s:8:"left top";} _intro_text: default _intro_custom_banner: a:1:{s:3:"url";s:0:"";} U_S_C_message: '' _thumbnail_id: '602' author: login: Vladimir Gitlevich email<EMAIL_ADDRESS>display_name: Vladimir Gitlevich first_name: Vladimir Gitlevich last_name: '' ---
In this talk, first presented at QCon London 2009, Eric Evans describes what he'd learned about DDD since the book was published. The talk was recorded at the May meeting of DDD-NYC SIG.
Because the talk goes over what has changed since the book, it assumes some knowledge of DDD. If you need a quick refresher, please watch the introduction part first.
Here is the summary from QCon site:
"In the 5 years since the book was published, I've practiced DDD on
various client projects, and I've continued to learn about what works,
what doesn't work, and how to conceptualize and describe it all. Also,
I've gained perspective and learned a great deal from the increasing
number of expert practitioners of DDD who have emerged.
The fundamentals have held up well, as well as most patterns, but there
are differences in how I do things and look at things now. I will try to
describe them, very informally, in this talk.
Over this time, I have folded in a couple of additional patterns, and
essentially come to ignore a few, but the biggest change has been a
subtle shift of emphasis. Ubiquitous Language and Context Mapping and
Core Domain are at the center, with aggregates in close orbit. Why, I
ask myself, did I put context mapping in Chapter 14? Core domain in
Chapter 15?! Before the book, it seemed self-evident to me that SOA fit
well with DDD, but five years of questions on that topic have made it
clear that my early explanations were inadequate and helped me clarify
how it fits. Increased emphasis on events and distributed processing
have crystallized the significance of aggregates and refined the
building blocks.
The talk cannot go into depth on all these topics, but the goal will be
to give a quick look at where my view of DDD has been heading."


--- layout: page title: Design date: 2011-10-06 04:10:26.000000000 -04:00 type: page published: true status: publish categories: [] tags: [] meta: _edit_last: '1' author: login: sophie email<EMAIL_ADDRESS>display_name: Pika first_name: '' last_name: '' ---
When I was little, there was only one thing I wanted to be when I grew up: an artist.
That didn't quite happen, but I still love to doodle and dream in my spare time!
Here are some simple little designs and posters I've done for various campus events (most of them for the student catering group I'm in, Nom3 Foods) and just for fun.
For more, check out the Doodles sidebar on the right!
I've just recently started dabbling in Photoshop/GIMP. All images were drawn freehand on my MacBook pro touchpad.
NOM^3 POSTERS


The formula which is taken here to calculate the surface is the following:
Radius:
Height:
Surface:
Back


metadata_ondisc_matrix
and multimodal_ondisc_matrix
tutorial_other_classes.Rmd
Tutorial 1 covered ondisc_matrix
, the core class implemented by ondisc
. Thus tutorial covers metadata_ondisc_matrix
and multimodal_ondisc_matrix
, two additional classes provided by the package. metadata_ondisc_matrix
stores cell-specific and feature-specific covariate matrices alongside the expression matrix, and multimodal_ondisc_matrix
stores multiple metadata_ondisc_matrices
representing different cellular modalities. Together, metadata_ondisc_matrix
and multimodal_ondisc_matrix
facilitate feature selection, quality control, and other common single-cell data preprocessing tasks.
We begin by loading the package.
metadata_ondisc_matrix
classA metadata_ondisc_matrix
object consists of three components: (i) an ondisc_matrix
representing the expression data, (ii) a data frame storing the cell-specific covariates, and (iii) a data frame storing the feature-specific covariates. The easiest way to initialize a metadata_ondisc_matrix
is by calling create_ondisc_matrix_from_mtx
on an mtx file and associated metadata files, setting the optional parameter return_metadata_ondisc_matrix
to TRUE
. Below, we reproduce the example from Tutorial 1, this time returning a metadata_ondisc_matrix
instead of a list.
# Set paths to the .mtx and .tsv files
raw_data_dir <- system.file("extdata", package = "ondisc")
mtx_fp <- paste0(raw_data_dir, "/gene_expression.mtx")
barcodes_fp <- paste0(raw_data_dir, "/cell_barcodes.tsv")
features_fp <- paste0(raw_data_dir, "/genes.tsv")
# Specify directory in which to store the .h5 file
temp_dir <- tempdir()
# Initialize metadata_ondisc_matrix
expressions <- create_ondisc_matrix_from_mtx(mtx_fp = mtx_fp,
barcodes_fp = barcodes_fp,
features_fp = features_fp,
on_disk_dir = temp_dir,
return_metadata_ondisc_matrix = TRUE)
#>
|======== | 11%
|================= | 23%
|========================== | 36%
|==================================== | 48%
|============================================= | 61%
|====================================================== | 73%
|=============================================================== | 86%
|=========================================================================| 98%
|=========================================================================| 100%
#>
|======== | 11%
|================= | 23%
|========================== | 36%
|==================================== | 48%
|============================================= | 61%
|====================================================== | 73%
|=============================================================== | 86%
|=========================================================================| 98%
|=========================================================================| 100%
#> Writing CSC data.
#> Writing CSR data.
The variable expressions
is an object of class metadata_ondisc_matrix
; expressions
contains the fields ondisc_matrix
, cell_covariates
, and feature_covariates
.
# Print the variable
expressions
#> A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 300 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#> A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
We alternately can initialize a metadata_ondisc_matrix
by calling the constructor function of the metadata_ondisc_matrix
class; see documentation (via ?metadata_ondisc_matrix) for details.
multimodal_ondisc_matrix
classThe multimodal_ondisc_matrix
class is used to represent multimodal data. multimodal_ondisc_matrix
objects have two fields: (i) a named list of metadata_ondisc_matrices
representing different modalities, and (ii) a global (i.e., cross-modality) cell-specific covariate matrix. The ondisc
package ships with example CRISPR perturbation data, which we use to initialize a new perturbation modality via a call to create_ondisc_matrix_from_mtx
.
# Set paths to the perturbation .mtx and .tsv files
mtx_fp <- paste0(raw_data_dir, "/perturbation.mtx")
barcodes_fp <- paste0(raw_data_dir, "/cell_barcodes.tsv")
features_fp <- paste0(raw_data_dir, "/guides.tsv")
# Initialize metadata_ondisc_matrix
perturbations <- create_ondisc_matrix_from_mtx(mtx_fp = mtx_fp,
barcodes_fp = barcodes_fp,
features_fp = features_fp,
on_disk_dir = temp_dir,
return_metadata_ondisc_matrix = TRUE)
#>
#> Writing CSC data.
#> Writing CSR data.
Like expressions
, the variable perturbations
is an object of class metadata_ondisc_matrix
. However, because perturbations
represents logical perturabtion data instead of integer gene expression data, the cell-specific and feature-specific covariates of perturbations
differ from those of expressions
.
# These matrices have different columns
head(expressions@cell_covariates)
#> n_nonzero n_umis p_mito
#> 1 43 214 0.04672897
#> 2 26 169 0.00000000
#> 3 22 116 0.05172414
#> 4 37 258 0.08139535
#> 5 36 224 0.08035714
#> 6 31 147 0.07482993
head(perturbations@cell_covariates)
#> n_nonzero
#> 1 21
#> 2 11
#> 3 17
#> 4 14
#> 5 7
#> 6 11
The expressions
and perturbations
data are multimodal – they are collected from the same set of cells. We can create a multimodal_ondisc_matrix
by passing a named list of metadata_ondisc_matrix
objects – in this case, expressions
and perturbations
– to the constructor function of the multimodal_ondisc_matrix
class.
modality_list <- list(expressions = expressions, perturbations = perturbations)
crispr_experiment <- multimodal_ondisc_matrix(modality_list)
The variable crispr_experiment
is an object of class multimodal_ondisc_matrix
. The column names of the global covariate matrix are derived from the names of the modalities.
# print variable
crispr_experiment
#> A multimodal_ondisc_matrix with the following modalities:
#> 1. expressions A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 300 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#> A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
#> 2. perturbations A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 250 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero.
#> A feature covariate matrix with columns n_nonzero.
#> Plus, a global cell-covariate matrix containing 4 covariates and 900 cells.
# show the global covariate matrix
head(crispr_experiment@global_cell_covariates)
#> expressions_n_nonzero expressions_n_umis expressions_p_mito
#> 1 43 214 0.04672897
#> 2 26 169 0.00000000
#> 3 22 116 0.05172414
#> 4 37 258 0.08139535
#> 5 36 224 0.08035714
#> 6 31 147 0.07482993
#> perturbations_n_nonzero
#> 1 21
#> 2 11
#> 3 17
#> 4 14
#> 5 7
#> 6 11
The figure below summarizes the relationship between ondisc_matrix
, metadata_ondisc_matrix
, and multimodal_ondisc_matrix
.
We can use the functions get_feature_ids
, get_feature_names
, and get_cell_barcodes
to obtain the feature IDs, feature names (if applicable), and cell barcodes, respectively, of a metadata_ondisc_matrix
or a multimodal_ondisc_matrix
. get_feature_ids
and get_feature_names
return a list when called on a multimodal_ondisc_matrix
, as the different modalities contain different features.
# metadata_ondisc_matrix
cell_barcodes <- get_cell_barcodes(expressions)
feature_ids <- get_feature_ids(expressions)
feature_names <- get_feature_names(expressions)
# multimodal_ondisc_matrix
cell_barcodes <- get_cell_barcodes(crispr_experiment)
feature_ids <- get_feature_ids(crispr_experiment)
We likewise can use dim
, nrow
, and ncol
to query the dimension, number of rows, and number of columns of a metadata_ondisc_matrix
or multimodal_ondisc_matrix
. dim
and nrow
again return lists when called on a multimodal_ondisc_matrix
.
# metadata_ondisc_matrix
dim(expressions)
#> [1] 300 900
nrow(expressions)
#> [1] 300
ncol(expressions)
#> [1] 900
# multimodal_ondisc_matrix
dim(crispr_experiment)
#> $expressions
#> [1] 300 900
#>
#> $perturbations
#> [1] 250 900
nrow(crispr_experiment)
#> $expressions
#> [1] 300
#>
#> $perturbations
#> [1] 250
ncol(crispr_experiment)
#> [1] 900
Similar to ondisc_matrices
, metadata_ondisc_matrices
and multimodal_ondisc_matrices
can be subset using the [
operator. metadata_ondisc_matrices
can be subset either by feature or cell, while multimodal_ondisc_matrices
can be subset by cell only.
# metadata_ondisc_matrix
# keep cells 100 - 150
expressions_sub <- expressions[,100:150]
# keep genes ENSG00000188305, ENSG00000257284, ENSG00000251655
expressions_sub <- expressions[c("ENSG00000188305", "ENSG00000257284", "ENSG00000251655"),]
# multimodal_ondisc_matrix
# keep all cells except 1 - 100
crispr_experiment_sub <- crispr_experiment[,-c(1:100)]
As with ondisc_matrices
, the original objects remain unchanged.
expressions
#> A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 300 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#> A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
crispr_experiment
#> A multimodal_ondisc_matrix with the following modalities:
#> 1. expressions A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 300 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero, n_umis, p_mito.
#> A feature covariate matrix with columns mean_expression, coef_of_variation, n_nonzero.
#> 2. perturbations A metadata_ondisc_matrix with the following components:
#> An ondisc_matrix with 250 features and 900 cells.
#> A cell covariate matrix with columns n_nonzero.
#> A feature covariate matrix with columns n_nonzero.
#> Plus, a global cell-covariate matrix containing 4 covariates and 900 cells.
To save and load a metadata_ondisc_matrix
or multimodal_ondisc_matrix
, simply use the functions saveRDS
and readRDS
, respectively.
It is not possible to apply the double-bracket “pull submatrix into memory” operator to a metadata_ondisc_matrix
or multimodal_ondisc_matrix
. To apply the double-bracket operator to an ondisc_matrix
stored inside a metadata_ondisc_matrix
or multimodal_ondisc_matrix
, simply access the desired ondisc_matrix
first.
The metadata_ondisc_matrix
and multimodal_ondisc_matrix
classes are inspired by analogous classes in the SingleCellExperiment
and Seurat
packages. However, unlike SingleCellExperiment
and Seurat
, ondisc
does not (currently) support storing the dense matrix of normalized UMI counts. We view this as a feature: storing the normalized expression matrix is almost never necessary and is hugely expensive (in space and time) for large-scale data.


import {MissionControlClient} from 'mission-control-client/src/index.js'
MissionControlClient
The mission control client class.
You can easily build your own client implementation, this one is just easy to use and has everything you might need.
The client hides the authentication implementation details from the user, by hijacking the connect
event.
The connect event only gets called after the authentication scheme is successful.
Constructor Summary
| Public Constructor | ||
| public |
constructor(url: string, authToken: string) The MissionControlClient constructor. |
Member Summary
| Public Members | ||
| public |
The JWT authentication token that is used to authenticate. |
|
| public |
eventBus: * The event bus used to communicate events within the client. |
since 1.0.0 |
| public |
socket: * The socket.io socket used for the communication. |
since 1.0.0 |
| Private Members | ||
| private |
_subscribeTo: Array<eventKey<string>> |
|
| private |
_subscriptions: Object<eventKey<string>, listenerCount<Number>> |
|
| private |
_unsubscribeFrom: Array<eventKey<string>> |
Method Summary
| Public Methods | ||
| public |
Execute an action on the mission control server. |
since 1.0.0 |
| public |
Listen to a socket event. |
since 1.0.0 |
| public |
Listen to a socket event, and clear it after it's been called once. Please note, if you want to subscribe to action or state events, please use the subscribe method. The returned function can be used to unsubscribe from the event listener again. This makes it possible to for example remove inline listeners. |
since 1.0.0 |
| public |
Subscribe to a server event (actions, state updates). |
since 1.0.0 |
| Private Methods | ||
| private |
This function sets up all the listeners for the socket (connect, disconnect, error, reconnect, etc). |
Public Constructors
Public Members
public eventBus: * since 1.0.0 source
The event bus used to communicate events within the client.
While it is possible it is recommended not to use this variable directly and to use the exposed MissionControlClient#on and MissionControlClient#subscribe methods instead.
Emit:
SocketEvent |
public socket: * since 1.0.0 source
The socket.io socket used for the communication.
While it is possible it is recommended not to use this variable directly and to use the exposed MissionControlClient#action and MissionControlClient#subscribe methods instead.
Private Members
Public Methods
public action(action: string, data: object) since 1.0.0 source
Execute an action on the mission control server.
This method sends an 'action' event, which the server will use to execute the action and modify the state accordingly.
Example:
client.action('EXAMPLE:DO', { parameter: 'example' })
client.action('VIDEO-QUEUE:PUSH', { video: { url: '...', format: 'mp4' }})
public on(event: string, listener: function(data: object)): function since 1.0.0 source
Listen to a socket event.
Please note, if you want to subscribe to action or state events, please use the subscribe method. The returned function can be used to unsubscribe from the event listener again. This makes it possible to for example remove inline listeners.
Example:
on('connect', () => {})
on('error', (errorType, errorObject) => {})
public once(event: string, listener: function(data: object)): function since 1.0.0 source
Listen to a socket event, and clear it after it's been called once.
Please note, if you want to subscribe to action or state events, please use the subscribe method. The returned function can be used to unsubscribe from the event listener again. This makes it possible to for example remove inline listeners.
Example:
once('connect', (data) => {})
public subscribe(serverEvent: string, listener: function(data: object)): function since 1.0.0 source
Subscribe to a server event (actions, state updates).
To subscribe to a server event, we need to emit a 'subscribe' event to the server so it knows to broadcast the right events to us. This function automatically handles these 'subscribe' and 'unsubscribe' events so you can simply use this method to do it. When we disconnect from the server, this function also handles resubscribing to the events. It returns a function that can you can use to remove the event listener again and unsubscribe from the server.
Example:
client.subscribe('action:EXAMPLE:DO', (data) => {})
client.subscribe('update:stateObject', (data) => {});
Private Methods
private _setupSocketHandlers() source
This function sets up all the listeners for the socket (connect, disconnect, error, reconnect, etc).
Job of this function is to unify all error events into a shape that makes more sense. See SOCKET_ERROR for the possible errors.


 Unicorn
la croix actually pop-up subway tile
taxidermy food truck.


If you're new to Mesos
See the getting started page for more information about downloading, building, and deploying Mesos.
If you'd like to get involved or you're looking for support
See our community page for more details.
Framework Development Guide
In this document we refer to Mesos applications as “frameworks”.
See one of the example framework schedulers in MESOS_HOME/src/examples/
to
get an idea of what a Mesos framework scheduler and executor in the language
of your choice looks like. RENDLER
provides example framework implementations in C++, Go, Haskell, Java, Python
and Scala.
Create your Framework Scheduler
If you are writing a scheduler against Mesos 1.0 or newer, it is recommended to use the new HTTP API to talk to Mesos.
If your framework needs to talk to Mesos 0.28.0 or older, you can write the
scheduler in C++, Java/Scala, or Python. Your framework scheduler should inherit
from the Scheduler
class (see API below). Your scheduler should create a SchedulerDriver
(which will mediate communication between your scheduler and the Mesos master) and then call
SchedulerDriver.run()
.
Scheduler API
Callback interface to be implemented by framework schedulers.
Declared in MESOS_HOME/include/mesos/scheduler.hpp
/*
* Invoked when the scheduler successfully registers with a Mesos
* master. A unique ID (generated by the master) used for
* distinguishing this framework from others and `MasterInfo`
* with the ip and port of the current master are provided as arguments.
*/
virtual void registered(
SchedulerDriver* driver,
const FrameworkID& frameworkId,
const MasterInfo& masterInfo);
/*
* Invoked when the scheduler re-registers with a newly elected Mesos master.
* This is only called when the scheduler has previously been registered.
* `MasterInfo` containing the updated information about the elected master
* is provided as an argument.
*/
virtual void reregistered(
SchedulerDriver* driver,
const MasterInfo& masterInfo);
/*
* Invoked when the scheduler becomes "disconnected" from the master
* (e.g., the master fails and another is taking over).
*/
virtual void disconnected(SchedulerDriver* driver);
/*
* Invoked when resources have been offered to this framework. A
* single offer will only contain resources from a single slave.
* Resources associated with an offer will not be re-offered to
* _this_ framework until either (a) this framework has rejected
* those resources (see SchedulerDriver::launchTasks) or (b) those
* resources have been rescinded (see Scheduler::offerRescinded).
* Note that resources may be concurrently offered to more than one
* framework at a time (depending on the allocator being used). In
* that case, the first framework to launch tasks using those
* resources will be able to use them while the other frameworks
* will have those resources rescinded (or if a framework has
* already launched tasks with those resources then those tasks will
* fail with a TASK_LOST status and a message saying as much).
*/
virtual void resourceOffers(
SchedulerDriver* driver,
const std::vector<Offer>& offers);
/*
* Invoked when an offer is no longer valid (e.g., the slave was
* lost or another framework used resources in the offer). If for
* whatever reason an offer is never rescinded (e.g., dropped
* message, failing over framework, etc.), a framework that attempts
* to launch tasks using an invalid offer will receive TASK_LOST
* status updates for those tasks (see Scheduler::resourceOffers).
*/
virtual void offerRescinded(SchedulerDriver* driver, const OfferID& offerId);
/*
* Invoked when the status of a task has changed (e.g., a slave is
* lost and so the task is lost, a task finishes and an executor
* sends a status update saying so, etc). If implicit
* acknowledgements are being used, then returning from this
* callback _acknowledges_ receipt of this status update! If for
* whatever reason the scheduler aborts during this callback (or
* the process exits) another status update will be delivered (note,
* however, that this is currently not true if the slave sending the
* status update is lost/fails during that time). If explicit
* acknowledgements are in use, the scheduler must acknowledge this
* status on the driver.
*/
virtual void statusUpdate(SchedulerDriver* driver, const TaskStatus& status);
/*
* Invoked when an executor sends a message. These messages are best
* effort; do not expect a framework message to be retransmitted in
* any reliable fashion.
*/
virtual void frameworkMessage(
SchedulerDriver* driver,
const ExecutorID& executorId,
const SlaveID& slaveId,
const std::string& data);
/*
* Invoked when a slave has been determined unreachable (e.g.,
* machine failure, network partition). Most frameworks will need to
* reschedule any tasks launched on this slave on a new slave.
*
* NOTE: This callback is not reliably delivered. If a host or
* network failure causes messages between the master and the
* scheduler to be dropped, this callback may not be invoked.
*/
virtual void slaveLost(SchedulerDriver* driver, const SlaveID& slaveId);
/*
* Invoked when an executor has exited/terminated. Note that any
* tasks running will have TASK_LOST status updates automagically
* generated.
*
* NOTE: This callback is not reliably delivered. If a host or
* network failure causes messages between the master and the
* scheduler to be dropped, this callback may not be invoked.
*/
virtual void executorLost(
SchedulerDriver* driver,
const ExecutorID& executorId,
const SlaveID& slaveId,
int status);
/*
* Invoked when there is an unrecoverable error in the scheduler or
* scheduler driver. The driver will be aborted BEFORE invoking this
* callback.
*/
virtual void error(SchedulerDriver* driver, const std::string& message);
Scheduler Driver API
The Scheduler Driver is responsible for managing the scheduler’s lifecycle (e.g., start, stop, or wait to finish) and interacting with Mesos Master (e.g., launch tasks, kill tasks, etc.).
Note that this interface is usually not implemented by a framework itself, but it describes the possible calls a framework scheduler can make to interact with the Mesos Master.
Please note that usage of this interface requires an instantiated
MesosSchedulerDiver.
See src/examples/test_framework.cpp
for an example of using the
MesosSchedulerDriver.
Declared in MESOS_HOME/include/mesos/scheduler.hpp
// Starts the scheduler driver. This needs to be called before any
// other driver calls are made.
virtual Status start();
// Stops the scheduler driver. If the 'failover' flag is set to
// false then it is expected that this framework will never
// reconnect to Mesos. So Mesos will unregister the framework and
// shutdown all its tasks and executors. If 'failover' is true, all
// executors and tasks will remain running (for some framework
// specific failover timeout) allowing the scheduler to reconnect
// (possibly in the same process, or from a different process, for
// example, on a different machine).
virtual Status stop(bool failover = false);
// Aborts the driver so that no more callbacks can be made to the
// scheduler. The semantics of abort and stop have deliberately been
// separated so that code can detect an aborted driver (i.e., via
// the return status of SchedulerDriver::join, see below), and
// instantiate and start another driver if desired (from within the
// same process). Note that 'stop()' is not automatically called
// inside 'abort()'.
virtual Status abort();
// Waits for the driver to be stopped or aborted, possibly
// _blocking_ the current thread indefinitely. The return status of
// this function can be used to determine if the driver was aborted
// (see mesos.proto for a description of Status).
virtual Status join();
// Starts and immediately joins (i.e., blocks on) the driver.
virtual Status run();
// Requests resources from Mesos (see mesos.proto for a description
// of Request and how, for example, to request resources from
// specific slaves). Any resources available are offered to the
// framework via Scheduler::resourceOffers callback, asynchronously.
virtual Status requestResources(const std::vector<Request>& requests);
// Launches the given set of tasks. Any remaining resources (i.e.,
// those that are not used by the launched tasks or their executors)
// will be considered declined. Note that this includes resources
// used by tasks that the framework attempted to launch but failed
// (with `TASK_ERROR`) due to a malformed task description. The
// specified filters are applied on all unused resources (see
// mesos.proto for a description of Filters). Available resources
// are aggregated when multiple offers are provided. Note that all
// offers must belong to the same slave. Invoking this function with
// an empty collection of tasks declines offers in their entirety
// (see Scheduler::declineOffer).
virtual Status launchTasks(
const std::vector<OfferID>& offerIds,
const std::vector<TaskInfo>& tasks,
const Filters& filters = Filters());
// Kills the specified task. Note that attempting to kill a task is
// currently not reliable. If, for example, a scheduler fails over
// while it was attempting to kill a task it will need to retry in
// the future. Likewise, if unregistered / disconnected, the request
// will be dropped (these semantics may be changed in the future).
virtual Status killTask(const TaskID& taskId);
// Accepts the given offers and performs a sequence of operations on
// those accepted offers. See Offer.Operation in mesos.proto for the
// set of available operations. Any remaining resources (i.e., those
// that are not used by the launched tasks or their executors) will
// be considered declined. Note that this includes resources used by
// tasks that the framework attempted to launch but failed (with
// `TASK_ERROR`) due to a malformed task description. The specified
// filters are applied on all unused resources (see mesos.proto for
// a description of Filters). Available resources are aggregated
// when multiple offers are provided. Note that all offers must
// belong to the same slave.
virtual Status acceptOffers(
const std::vector<OfferID>& offerIds,
const std::vector<Offer::Operation>& operations,
const Filters& filters = Filters());
// Declines an offer in its entirety and applies the specified
// filters on the resources (see mesos.proto for a description of
// Filters). Note that this can be done at any time, it is not
// necessary to do this within the Scheduler::resourceOffers
// callback.
virtual Status declineOffer(
const OfferID& offerId,
const Filters& filters = Filters());
// Removes all filters previously set by the framework (via
// launchTasks()). This enables the framework to receive offers from
// those filtered slaves.
virtual Status reviveOffers();
// Inform Mesos master to stop sending offers to the framework. The
// scheduler should call reviveOffers() to resume getting offers.
virtual Status suppressOffers();
// Acknowledges the status update. This should only be called
// once the status update is processed durably by the scheduler.
// Not that explicit acknowledgements must be requested via the
// constructor argument, otherwise a call to this method will
// cause the driver to crash.
virtual Status acknowledgeStatusUpdate(const TaskStatus& status);
// Sends a message from the framework to one of its executors. These
// messages are best effort; do not expect a framework message to be
// retransmitted in any reliable fashion.
virtual Status sendFrameworkMessage(
const ExecutorID& executorId,
const SlaveID& slaveId,
const std::string& data);
// Allows the framework to query the status for non-terminal tasks.
// This causes the master to send back the latest task status for
// each task in 'statuses', if possible. Tasks that are no longer
// known will result in a TASK_LOST update. If statuses is empty,
// then the master will send the latest status for each task
// currently known.
virtual Status reconcileTasks(const std::vector<TaskStatus>& statuses);
Handling Failures
How to build Mesos frameworks that remain available in the face of failures is discussed in a separate document.
Working with Executors
Using the Mesos Command Executor
Mesos provides a simple executor that can execute shell commands and Docker containers on behalf of the framework scheduler; enough functionality for a wide variety of framework requirements.
Any scheduler can make use of the Mesos command executor by filling in the
optional CommandInfo
member of the TaskInfo
protobuf message.
message TaskInfo {
...
optional CommandInfo command = 7;
...
}
The Mesos slave will fill in the rest of the ExecutorInfo
for you when tasks
are specified this way.
Note that the agent will derive an ExecutorInfo
from the TaskInfo
and
additionally copy fields (e.g., Labels
) from TaskInfo
into the new
ExecutorInfo
. This ExecutorInfo
is only visible on the agent.
Using the Mesos Default Executor
Since Mesos 1.1, a new built-in default executor (experimental) is available that can execute a group of tasks. Just like the command executor the tasks can be shell commands or Docker containers.
The current semantics of the default executor are as folows:
– Tasks are launched as nested containers underneath the executor container.
– Task containers and executor container share resources like cpu, memory, network and volumes.
– There is no resource isolation between different tasks within an executor. Tasks' resources are added to the executor container.
– If any of the tasks exits with a non-zero exit code, all the tasks in the task group are killed and the executor shuts down.
– Multiple task groups are not supported.
Once the default executor is considered stable, the command executor will be deprecated in favor of it.
Any scheduler can make use of the Mesos default executor by setting ExecutorInfo.type
to DEFAULT
when launching a group of tasks using the LAUNCH_GROUP
offer operation.
If DEFAULT
executor is explicitly specified when using LAUNCH
offer operation, command
executor is used instead of the default executor. This might change in the future when the default
executor gets support for handling LAUNCH
operation.
message ExecutorInfo {
...
optional Type type = 15;
...
}
Creating a custom Framework Executor
If your framework has special requirements, you might want to provide your own Executor implementation. For example, you may not want a 1:1 relationship between tasks and processes.
If you are writing an executor against Mesos 1.0 or newer, it is recommended to use the new HTTP API to talk to Mesos.
If writing against Mesos 0.28.0 or older, your framework executor must inherit from the Executor class. It must override the launchTask() method. You can use the $MESOS_HOME environment variable inside of your executor to determine where Mesos is running from.
Executor API
Declared in MESOS_HOME/include/mesos/executor.hpp
/*
* Invoked once the executor driver has been able to successfully
* connect with Mesos. In particular, a scheduler can pass some
* data to its executors through the `FrameworkInfo.ExecutorInfo`'s
* data field.
*/
virtual void registered(
ExecutorDriver* driver,
const ExecutorInfo& executorInfo,
const FrameworkInfo& frameworkInfo,
const SlaveInfo& slaveInfo);
/*
* Invoked when the executor re-registers with a restarted slave.
*/
virtual void reregistered(ExecutorDriver* driver, const SlaveInfo& slaveInfo);
/*
* Invoked when the executor becomes "disconnected" from the slave
* (e.g., the slave is being restarted due to an upgrade).
*/
virtual void disconnected(ExecutorDriver* driver);
/*
* Invoked when a task has been launched on this executor (initiated
* via Scheduler::launchTasks). Note that this task can be realized
* with a thread, a process, or some simple computation, however, no
* other callbacks will be invoked on this executor until this
* callback has returned.
*/
virtual void launchTask(ExecutorDriver* driver, const TaskInfo& task);
/*
* Invoked when a task running within this executor has been killed
* (via SchedulerDriver::killTask). Note that no status update will
* be sent on behalf of the executor, the executor is responsible
* for creating a new TaskStatus (i.e., with TASK_KILLED) and
* invoking ExecutorDriver::sendStatusUpdate.
*/
virtual void killTask(ExecutorDriver* driver, const TaskID& taskId);
/*
* Invoked when a framework message has arrived for this
* executor. These messages are best effort; do not expect a
* framework message to be retransmitted in any reliable fashion.
*/
virtual void frameworkMessage(ExecutorDriver* driver, const std::string& data);
/*
* Invoked when the executor should terminate all of its currently
* running tasks. Note that after a Mesos has determined that an
* executor has terminated any tasks that the executor did not send
* terminal status updates for (e.g., TASK_KILLED, TASK_FINISHED,
* TASK_FAILED, etc) a TASK_LOST status update will be created.
*/
virtual void shutdown(ExecutorDriver* driver);
/*
* Invoked when a fatal error has occurred with the executor and/or
* executor driver. The driver will be aborted BEFORE invoking this
* callback.
*/
virtual void error(ExecutorDriver* driver, const std::string& message);
Install your custom Framework Executor
After creating your custom executor, you need to make it available to all slaves in the cluster.
One way to distribute your framework executor is to let the
Mesos fetcher download it on-demand when your scheduler launches
tasks on that slave. ExecutorInfo
is a Protocol Buffer Message class (defined
in include/mesos/mesos.proto
), and it contains a field of type CommandInfo
.
CommandInfo
allows schedulers to specify, among other things, a number of
resources as URIs. These resources are fetched to a sandbox directory on the
slave before attempting to execute the ExecutorInfo
command. Several URI
schemes are supported, including HTTP, FTP, HDFS, and S3 (e.g. see
src/examples/java/TestFramework.java for an example of this).
Alternatively, you can pass the frameworks_home
configuration option
(defaults to: MESOS_HOME/frameworks
) to your mesos-slave
daemons when you
launch them to specify where your framework executors are stored (e.g. on an
NFS mount that is available to all slaves), then use a relative path in
CommandInfo.uris
, and the slave will prepend the value of frameworks_home
to the relative path provided.
Once you are sure that your executors are available to the mesos-slaves, you should be able to run your scheduler, which will register with the Mesos master, and start receiving resource offers!
Labels
Labels
can be found in the FrameworkInfo
, TaskInfo
, DiscoveryInfo
and
TaskStatus
messages; framework and module writers can use Labels to tag and
pass unstructured information around Mesos. Labels are free-form key-value pairs
supplied by the framework scheduler or label decorator hooks. Below is the
protobuf definitions of labels:
optional Labels labels = 11;
/**
* Collection of labels.
*/
message Labels {
repeated Label labels = 1;
}
/**
* Key, value pair used to store free form user-data.
*/
message Label {
required string key = 1;
optional string value = 2;
}
Labels are not interpreted by Mesos itself, but will be made available over
master and slave state endpoints. Further more, the executor and scheduler can
introspect labels on the TaskInfo
and TaskStatus
programmatically.
Below is an example of how two label pairs ("environment": "prod"
and
"bananas": "apples"
) can be fetched from the master state endpoint.
$ curl http://master/state.json
...
{
"executor_id": "default",
"framework_id": "20150312-120017-16777343-5050-39028-0000",
"id": "3",
"labels": [
{
"key": "environment",
"value": "prod"
},
{
"key": "bananas",
"value": "apples"
}
],
"name": "Task 3",
"slave_id": "20150312-115625-16777343-5050-38751-S0",
"state": "TASK_FINISHED",
...
},
Service discovery
When your framework registers an executor or launches a task, it can provide additional information for service discovery. This information is stored by the Mesos master along with other imporant information such as the slave currently running the task. A service discovery system can programmatically retrieve this information in order to set up DNS entries, configure proxies, or update any consistent store used for service discovery in a Mesos cluster that runs multiple frameworks and multiple tasks.
The optional DiscoveryInfo
message for TaskInfo
and ExecutorInfo
is
declared in MESOS_HOME/include/mesos/mesos.proto
message DiscoveryInfo {
enum Visibility {
FRAMEWORK = 0;
CLUSTER = 1;
EXTERNAL = 2;
}
required Visibility visibility = 1;
optional string name = 2;
optional string environment = 3;
optional string location = 4;
optional string version = 5;
optional Ports ports = 6;
optional Labels labels = 7;
}
Visibility
is the key parameter that instructs the service discovery system
whether a service should be discoverable. We currently differentiate between
three cases:
- a task should not be discoverable for anyone but its framework.
- a task should be discoverable for all frameworks running on the Mesos cluster but not externally.
- a task should be made discoverable broadly.
Many service discovery systems provide additional features that manage the
visibility of services (e.g., ACLs in proxy based systems, security extensions
to DNS, VLAN or subnet selection). It is not the intended use of the visibility
field to manage such features. When a service discovery system retrieves the
task or executor information from the master, it can decide how to handle tasks
without DiscoveryInfo
. For instance, tasks may be made non discoverable to
other frameworks (equivalent to visibility=FRAMEWORK
) or discoverable to all
frameworks (equivalent to visibility=CLUSTER
).
The name
field is a string that provides the service discovery system
with the name under which the task is discoverable. The typical use of the name
field will be to provide a valid hostname. If name is not provided, it is up to
the service discovery system to create a name for the task based on the name
field in taskInfo
or other information.
The environment
, location
, and version
fields provide first class support
for common attributes used to differentiate between similar services in large
deployments. The environment
may receive values such as PROD/QA/DEV
, the
location
field may receive values like EAST-US/WEST-US/EUROPE/AMEA
, and the
version
field may receive values like v2.0/v0.9. The exact use of these fields
is up to the service discovery system.
The ports
field allows the framework to identify the ports a task listens to
and explicitly name the functionality they represent and the layer-4 protocol
they use (TCP, UDP, or other). For example, a Cassandra task will define ports
like "7000,Cluster,TCP"
, "7001,SSL,TCP"
, "9160,Thrift,TCP"
,
"9042,Native,TCP"
, and "7199,JMX,TCP"
. It is up to the service discovery
system to use these names and protocol in appropriate ways, potentially
combining them with the name
field in DiscoveryInfo
.
The labels
field allows a framework to pass arbitrary labels to the service
discovery system in the form of key/value pairs. Note that anything passed
through this field is not guaranteed to be supported moving forward.
Nevertheless, this field provides extensibility. Common uses of this field will
allow us to identify use cases that require first class support.


Where are my manners? Hello, Jp at your service!
Background
- Born and raised in Chicago, relocated to Seattle in 1991
- Father of two surprisingly good children
- Former merchant marine officer
- Fullstack Software Developer
- Apprentice Software Engineer at Microsoft (through the Apprenti program)
Top 10 things that really damage my calm!
- Seattle Drivers
- New Seattle residents being allowed to drive
- Intellectual bullies
- stretchy pants
- Massive road construction projects downtown
- Lack of j-walking tickets being written
- Cats
- Purse dogs (just an undignified existance for the poor dog)
- Plenty of money for campaining but none for better education...hmmm.
- Did I mention Seattle drivers? Non-merging !@%#&$*$#%ers!


PebbleNotes
Account
Log in
Log out
Options
Move completed tasks to the end of list
Off
On
Sort tasks by date updated
Off
Oldest first
Newest first
Sort tasks alphabetically
(ignore user-selected order)
Off
On
Save
Cancel


public abstract class AdvisableOperation
extends java.lang.Object
IOperationAdvisor
) and
post-operation participation (@see IOperationParticipant
).
Advisable operations must be declared via extension so that clients can view
the list of available operations to configure when authoring their process.
Advisable operations are declared on the client as elements of the extension
point com.ibm.team.process.client.configurationPoints
.
Operations are declared on the server as elements of the extension point
com.ibm.team.process.service.configurationPoints
.
For example:
<extension point="com.ibm.team.process.client.configurationPoints"> <operation id="com.ibm.example.operation" name="Example Operation"> <action id="foo"> <action id="bar"/> </action> <action id="foo2"> <action id="bar2"/> </action> </operation> </extension>
Advisable operations may not be executed by clients directly. They should instead be passed to the advise and execute methods on the client or server process (@see com.ibm.team.process.client.IClientProcess and com.ibm.team.process.service.IServerProcess).
| Constructor and Description |
|---|
AdvisableOperation(java.lang.String operationId,
java.lang.Object operationData,
IProcessArea processArea)
Creates a new operation with the given identifier, which must be a valid
operation identifier declared via the configurationPoints extension-point
(see class comment for details).
|
AdvisableOperation(java.lang.String operationId,
java.lang.Object operationData,
IProcessArea processArea,
IDevelopmentLine line)
Creates a new operation with the given identifier, which must be a
valid operation identifier declared via the configurationPoints
extension-point (see class comment for details).
|
| Modifier and Type | Method and Description |
|---|---|
IOperationReport |
advise(IBehaviorConfiguration configuration,
org.eclipse.core.runtime.IProgressMonitor monitor)
Runs the given operation in "advise" mode.
|
boolean |
canReRun()
Returns whether or not this operation can be rerun.
|
boolean |
equals(java.lang.Object obj)
By default, advisable operations are considered equal if their operation
data are equal.
|
java.lang.String[] |
getActions()
Returns the set of permission-requiring actions which this operation will
perform when run which require permissions.
|
IDevelopmentLine |
getDevelopmentLine()
Returns the development line associated with this operation or
null if this operation's process area is a team area and the
operation is using the team area's dev line. |
IOperationConfigurationInfo |
getOperationConfigurationInfo()
Returns the operation configuration info for this operation or
null if the info has not yet been set. |
java.lang.Object |
getOperationData()
Returns the operation-specific data associated with this operation or
null if this operation carries no domain-specific data. |
java.lang.String |
getOperationIdentifier()
Returns this operation's unique identifier.
|
IProcessArea |
getProcessArea()
Returns the process area associated with this operation.
|
java.lang.String[] |
getReportData()
Returns this operation's report data or
null if none. |
java.lang.String[] |
getSecondaryActions(IProcessArea secondaryArea)
Returns the actions that this operation is performing relative to the
given secondary area.
|
IDevelopmentLine |
getSecondaryDevelopmentLine(IProcessArea secondaryArea)
Returns the development line relative to which the given secondary area's
actions will be performed or
null if the given area is a
team area. |
IProcessArea[] |
getSecondaryProcessAreas()
Returns a collection of process areas which are secondarily affected by
this operation.
|
int |
hashCode() |
IOperationReport |
reRun(IBehaviorConfiguration configuration,
org.eclipse.core.runtime.IProgressMonitor monitor)
Re-runs the operation and returns any advice which was computed in the
course of executing the operation or
null if no additional
advice was computed. |
abstract IOperationReport |
run(IBehaviorConfiguration configuration,
org.eclipse.core.runtime.IProgressMonitor monitor)
Runs the operation and returns any advice which was computed in the
course of executing the operation or
null if no additional
advice was computed. |
void |
setDevelopmentLine(IDevelopmentLine line)
This method is called by the process framework when a development line
is dynamically computed for an operation.
|
void |
setOperationConfigurationInfo(IOperationConfigurationInfo info)
This method is called by the process framework before the run method
is invoke.
|
void |
setReportData(java.lang.String[] reportData)
Sets this operation's report data or
null if none. |
boolean |
shouldThrowPermissionDeniedException()
Returns whether the process framework should throw a
PermissionDeniedException if the user does not have the
permissions necessary to execute this operation. |
public AdvisableOperation(java.lang.String operationId, java.lang.Object operationData, IProcessArea processArea)
The given process area determines the process which will govern this operation.
operationId
- the operation identifieroperationData
- the operation data or null
processArea
- the affected process areapublic AdvisableOperation(java.lang.String operationId, java.lang.Object operationData, IProcessArea processArea, IDevelopmentLine line)
Taken together, the given process area and development line determine the process which will govern this operation.
operationId
- the operation identifieroperationData
- the operation data or null
processArea
- the affected process arealine
- the affected development line or null
public abstract IOperationReport run(IBehaviorConfiguration configuration, org.eclipse.core.runtime.IProgressMonitor monitor) throws TeamRepositoryException
null
if no additional
advice was computed. Typically, this is used to return advice which is
run as part of the server-side implementation of a client-side operation.
Implementors of this method should check the progress monitor for cancellation when it is safe and appropriate to do so.
This method should not be called directly by clients. Clients wishing to run an operation should do so by via the methods on the client or server process.
configuration
- the configuration which configures this operation or
null
monitor
- the monitor used to report progress and receive
cancellation requestsnull
TeamOperationCanceledException
- in the event that this method
caused another operation to run and that operation was
canceled by processStaleConfigurationDataException
- if stale configuration data was
used to advise the operationTeamRepositoryException
- if an error occurspublic boolean canReRun()
true
should also
override reRun(IBehaviorConfiguration, IProgressMonitor).public IOperationReport reRun(IBehaviorConfiguration configuration, org.eclipse.core.runtime.IProgressMonitor monitor) throws TeamRepositoryException
null
if no additional
advice was computed. Typically, this is used to return advice which is
run as part of the server-side implementation of an operation.
This method should not be called directly by clients. Clients wishing to re-run an operation should do so via IClientProcess#reAdviseAndExecute(AdvisableOperation).
configuration
- the configuration which configures this operation or
null
monitor
- the monitor used to report progress and receive
cancellation requestsnull
TeamOperationCanceledException
- in the event that this method
caused another operation to run and that operation was
canceled by processStaleConfigurationDataException
- if stale configuration data was
used to advise the operationTeamRepositoryException
- if an error occurspublic IOperationReport advise(IBehaviorConfiguration configuration, org.eclipse.core.runtime.IProgressMonitor monitor) throws TeamRepositoryException
This method should not be called directly by clients. Clients wishing to advise an operation should do so via IClientProcess#advise(AdvisableOperation).
configuration
- the configuration which configures this operation or
null
monitor
- the monitor used to report progress and receive
cancellation requestsnull
StaleConfigurationDataException
- if stale configuration data was
used to advise the operationTeamRepositoryException
- if an error occurspublic java.lang.String[] getActions() throws TeamRepositoryException
Subclasses should override this method to enable process permissions for their operations. The returned actions are valid paths declared in the operation's extension. For example, if the operation extension declares the actions:
<action id="modify"> <action id="fieldA"/> <action id="fieldB"/> </action>The following would be valid element values in the returned array:
TeamRepositoryException
- if an error occurspublic IProcessArea getProcessArea()
public IProcessArea[] getSecondaryProcessAreas()
This is important, for example, for an operation which moves an item from the context of one process area to another. In order to enforce a notion of ownership which is enforced by process permissions, such an action would specify the source area as a secondary process area and return an appropriate action such as "remove".
Secondary areas are consulted only for permissions. Advisors and participants configured for secondary areas are not invoked.
Secondary areas and their actions do not replace the primary area and its actions, returned from getProcessArea() and getActions().
public java.lang.String[] getSecondaryActions(IProcessArea secondaryArea) throws TeamRepositoryException
secondaryArea
- the areaTeamRepositoryException
- if an error occursfor more details
public IDevelopmentLine getSecondaryDevelopmentLine(IProcessArea secondaryArea)
null
if the given area is a
team area. If the given area is a team area and this method returns
null
, the team area's development line will be automatically
computed.
Subclasses which return only team areas as secondary process areas need not override this method.
This method must only be overridden by operations which potentially
return project areas from getSecondaryProcessAreas()
.
secondaryArea
- the areapublic IDevelopmentLine getDevelopmentLine()
null
if this operation's process area is a team area and the
operation is using the team area's dev line.null
public java.lang.Object getOperationData()
null
if this operation carries no domain-specific data.null
public java.lang.String getOperationIdentifier()
public boolean shouldThrowPermissionDeniedException()
PermissionDeniedException
if the user does not have the
permissions necessary to execute this operation. By default, the
framework throws a TeamOperationCanceledException
.
This is experimental support for components who wish to explore an alternate programming model.
PermissionDeniedException
when asked to execute this operationpublic boolean equals(java.lang.Object obj)
A meaningful equality check allows the framework to clear old operation reports when a new report is created for the same operation. For example, if the user attempts to save an item and is blocked by process, the next time they try to save the same item this should be identified as the same operation (allowing the framework to remove the previous report).
equals
in class java.lang.Object
Object.equals(java.lang.Object)
public int hashCode()
hashCode
in class java.lang.Object
public IOperationConfigurationInfo getOperationConfigurationInfo()
null
if the info has not yet been set. The operation
configuration info will be set by the process framework before the run,
reRun, or advise methods are invoked. Extenders may rely on this method
returning non-null inside their implementations of the run, rerun, and
advise methods.
Components can use this info to validate that client-side process data is up to date by passing the info returned from a client-side operation to the server and then invoking IServerProcess#validateSource(...).
Clients may not override this method.
null
public void setOperationConfigurationInfo(IOperationConfigurationInfo info)
Clients may not call this method.
Clients may not override this method.
public void setDevelopmentLine(IDevelopmentLine line)
Clients may not call this method.
Clients may not override this method.
public void setReportData(java.lang.String[] reportData)
null
if none.
When this operation is passed to the client or server process, the given
report data will be set on the IOperationReport
that is generated
for the operation. Because this data is an operation-specific string
which will be transmitted with the report even over the wire, this field
is useful for implementors who wish to encode server-side data for a UI
presentation.
public java.lang.String[] getReportData()
null
if none. This
method is called by the process framework when creating a report for this
operation. Clients generally need not call this method.null


|
ovs all
|
Namespaces | |
| qemu-wrap | |
Functions | |
| def | qemu-wrap.kill_qemu_process (signum, stack) |
| Signal haldler to kill Qemu subprocess. More... | |
| def | qemu-wrap.find_huge_mount () |
| Find the system hugefile mount point. More... | |
| def | qemu-wrap.get_vhost_fd () |
| Get a userspace Vhost file descriptor. More... | |
| def | qemu-wrap.modify_netdev_arg (arg) |
| Check for vhostfd. More... | |
| def | qemu-wrap.main () |
| Main. More... | |
Variables | |
| string | qemu-wrap.emul_path = "/usr/local/bin/qemu-system-x86_64" |
| This script is designed to modify the call to the QEMU emulator to support userspace vhost when starting a guest machine through libvirt with vhost enabled. More... | |
| string | qemu-wrap.us_vhost_path = "/dev/usvhost-1" |
| list | qemu-wrap.emul_opts_user = [] |
| list | qemu-wrap.emul_opts_user_vhost = [] |
| int | qemu-wrap.use_huge_all = 0 |
| string | qemu-wrap.hugetlbfs_dir = "" |
| list | qemu-wrap.fd_list = [] |
| list | qemu-wrap.vhost_flags |
| string | qemu-wrap.qemu_pid = "/tmp/%d-qemu.pid" |


Welcome to Asasa.com!
Choose from our selection of verified properties.
Make sure to check key features, virtual tours, photos and videos mentioned in each listing to gain a better insight about the property.
Search for the right property by filtering through the listings being offered.
Contact the agent of the property selected, using the contact details mentioned on the listing.
Wait for the agent to contact and guide you regarding the property selected.


© Cogeco Peer 1.
Design: Steamclock Software and Cogeco Peer 1.
Development: Steamclock Software.
Data: CAIDA and Jeff Johnston.
Dev Team: Nigel Brooke, Justin Alm, Chani Armitage, Alexander von Franqué, Angelina Fabbro, Shayla Sawchenko, Jot Kali, Robert MacEachern, and Allen Pike.
Special thanks to conributors Kyle Dickau, Victor Swarovski, Skyler Richter, and Rajan Sodhi.
Special thanks to open source projects: MBProgressHUD, vectormath, TTTAttributedLabel, Boost, jsoncpp, ASIHTTPRequest, ipify.org, and iptoasn.com.
Map of Internet includes GeoLite2 data created by MaxMind, available from maxmind.com


# Publishing a Lint Check
Lint will look for jar files with a service registry key for issue
registries.
You can manually point it to your custom lint checks jar files by using
the environment variable `ANDROID_LINT_JARS`:
```shell
$ export ANDROID_LINT_JARS=/path/to/first.jar:/path/to/second.jar
```
(On Windows, use `;` instead of `:` as the path separator)
However, that is only intended for development and as a workaround for
build systems that do not have direct support for lint or embedded lint
libraries, such as the internal Google build system.
## Android
### AAR Support
Android libraries are shipped as `.aar` files instead of `.jar` files.
This means that they can carry more than just the code payload. Under
the hood, `.aar` files are just zip files which contain many other
nested files, including api and implementation jars, resources,
proguard/r8 rules, and yes, lint jars.
For example, if we look at the contents of the timber logging library's
AAR file, we can see the lint.jar with several lint checks within as
part of the payload:
```shell
$ jar tvf ~/.gradle/caches/.../jakewharton.timber/timber/4.5.1/?/timber-4.5.1.aar
216 Fri Jan 20 14:45:28 PST 2017 AndroidManifest.xml
8533 Fri Jan 20 14:45:28 PST 2017 classes.jar
10111 Fri Jan 20 14:45:28 PST 2017 lint.jar
39 Fri Jan 20 14:45:28 PST 2017 proguard.txt
0 Fri Jan 20 14:45:24 PST 2017 aidl/
0 Fri Jan 20 14:45:28 PST 2017 assets/
0 Fri Jan 20 14:45:28 PST 2017 jni/
0 Fri Jan 20 14:45:28 PST 2017 res/
0 Fri Jan 20 14:45:28 PST 2017 libs/
```
The advantage of this approach is that when lint notices that you
depend on a library, and that library contains custom lint checks, then
lint will pull in those checks and apply them. This gives library
authors a way to provide their own additional checks enforcing usage.
### lintPublish Configuration
The Android Gradle library plugin provides some special configurations,
`lintConfig` and `lintPublish`.
The `lintPublish` configuration lets you reference another project, and
it will take that project's output jar and package it as a `lint.jar`
inside the AAR file.
The [](https://github.com/googlesamples/android-custom-lint-rules)
sample project demonstrates this setup.
The `:checks` project is a pure Kotlin library which depends on the
Lint APIs, implements a `Detector`, and provides an `IssueRegistry`
which is linked from `META-INF/services`.
Then in the Android library, the `:library` project applies the Android
Gradle library plugin. It then specifies a `lintPublish` configuration
referencing the checks lint project:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apply plugin: 'com.android.library'
dependencies {
lintPublish project(':checks')
// other dependencies
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Finally, the sample `:app` project is an example of an Android app
which depends on the library, and the source code in the app contains a
violation of the lint check defined in the `:checks` project. If you
run `./gradlew :app:lint` to analyze the app, the build will fail
emitting the custom lint check.
### Local Checks
What if you aren't publishing a library, but you'd like to apply
some checks locally for your own codebase?
You can use a similar approach to `lintPublish`: In your app
module, specify
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
apply plugin: 'com.android.application'
dependencies {
lintConfig project(':checks')
// other dependencies
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now, when lint runs on this application, it will apply the checks
provided from the given project.
!!! Warning
This mechanism works well on the CI server for enforcing local code
conventions, and it also works for developers on your team; the
errors should be flagged in the IDE (providing they are analyzing
single-file scopes). However, there have been various bugs and
difficulties around the lint checks getting rebuilt after changes or
clean builds. There are some bugs in the Android Gradle Plugin issue
tracker for this.
### Unpublishing
If you end up “deleting” a lint check, perhaps because the original
conditions for the lint check are not true, don't just stop
distributing lint checks with your library. Instead, you'll want to
update your `IssueRegistry` to override the `deletedIssues` property to
return your deleted issue id or ids:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/**
* The issue id's from any issues that have been deleted from this
* registry. This is here such that when an issue no longer applies
* and is no longer registered, any existing mentions of the issue
* id in baselines, lint.xml files etc are gracefully handled.
*/
open val deletedIssues: List