Class MailComposer
- Defined in: mailcomposer.js
| Constructor Attributes | Constructor Name and Description |
|---|---|
|
MailComposer(options)
Costructs a MailComposer object. |
Method Summary
Class Detail
Costructs a MailComposer object. This is a Stream instance so you could pipe the output to a file or send it to network.
Possible options properties are:
- escapeSMTP - convert dots in the beginning of line to double dots
- encoding - forced transport encoding (quoted-printable, base64, 7bit or 8bit)
- keepBcc - include Bcc: field in the message headers (default is false)
Events
- 'envelope' - emits an envelope object with
from
andto
(array) addresses. - 'data' - emits a chunk of data
- 'end' - composing the message has ended
- Parameters:
- {Object} options Optional
- Optional options object
Field Detail
Contains a list of attachments
Contains e-mail addresses for the SMTP
Counter for generating unique mime boundaries etc.
Contains all header values
Contains message related values
Contains a list of attachments that are related to HTML body
Method Detail
-
<private> _buildMessageHeaders()
Uses data from the
this._message
object to build headers -
<private> _composeBody()
Composes the e-mail body based on the previously generated mime tree
Assumes that the linebreak separating headers and contents is already sent
Emits 'data' events
-
<private> _composeHeader()
Composes a header for the message and emits it with a
'data'
eventAlso checks and build a structure for the message (is it a multipart message and does it need a boundary etc.)
By default the message is not a multipart. If the message containes both plaintext and html contents, an alternative block is used. it it containes attachments, a mixed block is used. If both alternative and mixed exist, then alternative resides inside mixed.
-
<private> _composeMessage()
Generate an e-mail from the described info
-
<private> {String} _convertAddresses(value)
Handles a list of parsed e-mail addresses, checks encoding etc.
- Parameters:
- {Array} value
- A list or single e-mail address
{address:'...', name:'...'}
- Returns:
- {String} Comma separated and encoded list of addresses
-
<private> {Object} _createAttachmentComponent(attachment)
Creates a mime tree node for a text component (plaintext, HTML)
- Parameters:
- {Object} attachment
- Attachment info for the component
- Returns:
- {Object} Mime tree node
-
<private> {Object} _createMimeNode()
Creates an empty mime tree node
- Returns:
- {Object} Mime tree node
-
<private> {Object} _createTextComponent(text, contentType)
Creates a mime tree node for a text component (plaintext, HTML)
- Parameters:
- {String} text
- Text contents for the component
- {String} contentType Optional, Default: "text/plain"
- Content type for the text component
- Returns:
- {Object} Mime tree node
-
<private> _emitDataElement(element, callback)
Emits a data event for a text or html body and attachments. If it is a file, stream it
If
this.options.escapeSMTP
is true, replace dots in the beginning of a line with double dots - only valid for QP encoding- Parameters:
- {Object} element
- Data element descriptor
- {Function} callback
- Callback function to run when completed
-
<private> {String} _encodeMimeWord(str, encoding, maxlen)
Converts a string to mime word format. If the length is longer than
maxlen
, split itIf the string doesn't have any unicode characters return the original string instead
- Parameters:
- {String} str
- String to be encoded
- {String} encoding
- Either Q for Quoted-Printable or B for Base64
- {Number} maxlen Optional
- Optional length of the resulting string, whitespace will be inserted if needed
- Returns:
- {String} Mime-word encoded string (if needed)
-
<private> _flattenMimeTree()
Converts a structured mimetree into an one dimensional array of components. This includes headers and multipart boundaries as strings, textual and attachment contents are.
-
<private> _generateBodyStructure()
Generates the structure (mime tree) of the body. This sets up multipart structure, individual part headers, boundaries etc.
The headers of the root element will be appended to the message headers
-
<private> {String} _generateBoundary()
Generates a boundary for multipart bodies
- Returns:
- {String} Boundary String
-
<private> {String|Array} _getHeader(key)
Gets a header field
- Parameters:
- {String} key
- Key name
- Returns:
- {String|Array} Header field - if several values, then it's an array
-
<private> {String} _getMimeType(filename)
Resolves a mime type for a filename
- Parameters:
- {String} filename
- Filename to check
- Returns:
- {String} Corresponding mime type
-
<private> {String} _handleValue(key, value)
Handles a message object value, converts addresses etc.
- Parameters:
- {String} key
- Message options key
- {String} value
- Message options value
- Returns:
- {String} converted value
-
<private> {Boolean} _hasUTFChars(str)
Tests if a string has high bit (UTF-8) symbols
- Parameters:
- {String} str
- String to be tested for high bit symbols
- Returns:
- {Boolean} true if high bit symbols were found
-
<private> _init()
Resets and initializes MailComposer
-
<private> {String} _normalizeKey(key)
Normalizes a key name by cpitalizing first chars of words, except for custom keys (starting with "X-") that have only uppercase letters, which will not be modified.
x-mailer
will becomeX-Mailer
Needed to avoid duplicate header keys
- Parameters:
- {String} key
- Key name
- Returns:
- {String} First chars uppercased
-
<private> _serveFile(filePath, callback)
Pipes a file to the e-mail stream
- Parameters:
- {String} filePath
- Path to the file
- {Function} callback
- Callback function to run after completion
-
<private> _serveStream(stream, callback)
Pipes a stream source to the e-mail stream
This function resumes the stream and starts sending 76 bytes long base64 encoded lines. To achieve this, the incoming stream is divded into chunks of 57 bytes (57/3*4=76) to achieve exactly 76 byte long base64
- Parameters:
- {Object} stream
- Stream to be piped
- {Function} callback
- Callback function to run after completion
-
addAttachment(attachment)
Adds an attachment to the list
Following options are allowed:
- fileName - filename for the attachment
- contentType - content type for the attachmetn (default will be derived from the filename)
- cid - Content ID value for inline images
- contents - String or Buffer attachment contents
- filePath - Path to a file for streaming
- streamSource - Stream object for arbitrary streams
One of
contents
orfilePath
orstream
must be specified, otherwise the attachment is not included- Parameters:
- {Object} attachment
- Attachment info
-
addHeader(key, value)
Adds a header field to the headers object
- Parameters:
- {String} key
- Key name
- {String} value
- Header value
-
{Array} compileHeaders(headers)
Compiles headers object into an array of header lines. If needed, the lines are folded
- Parameters:
- {Object|Array} headers
- An object with headers in the form of
{key:value}
or[[key, value]] or [{key:key, value: value}]
- Returns:
- {Array} A list of header lines. Can be joined with \r\n
-
{Object} getEnvelope()
Composes and returns an envelope from the
this._envelope
object. Needed for the SMTP clientGenerated envelope is int hte following structure:
{ to: "address", from: ["list", "of", "addresses"] }
Both properties (
from
andto
) are optional and may not exist- Returns:
- {Object} envelope object with "from" and "to" params
-
setMessageOption(options)
Resets and initializes MailComposer
Setting an option overwrites an earlier setup for the same keys
Possible options:
- from - The e-mail address of the sender. All e-mail addresses can be plain
<EMAIL_ADDRESS>or formattedSender Name<EMAIL_ADDRESS>- to - Comma separated list of recipients e-mail addresses that will appear on the
To:
field - cc - Comma separated list of recipients e-mail addresses that will appear on the
Cc:
field - bcc - Comma separated list of recipients e-mail addresses that will appear on the
Bcc:
field - replyTo - An e-mail address that will appear on the
Reply-To:
field - subject - The subject of the e-mail
- body - The plaintext version of the message
- html - The HTML version of the message
- Parameters:
- {Object} options
- Message related options
- from - The e-mail address of the sender. All e-mail addresses can be plain
-
streamMessage()
Starts streaming the message


Welcome to spark_sklearn’s documentation!¶
Contents:
-
class
spark_sklearn.
Converter
(sc)¶ Bases:
object
Class for converting between scikit-learn and Spark ML models
Parameters: sc – SparkContext -
toPandas
(df)¶ This is similar to the Spark DataFrame built-in toPandas() method, but it handles MLlib Vector columns differently. It converts MLlib Vectors into rows of scipy.sparse.csr_matrix, which is generally friendlier for PyData tools like scikit-learn.
Note
Experimental: This will likely be replaced in later releases with improved APIs.
Parameters: df – Spark DataFrame Returns: Pandas dataframe
-
toSKLearn
(model)¶ Convert a Spark MLlib model from the Pipelines API (spark.ml) to a scikit-learn model. Currently supported models: - pyspark.ml.classification.LogisticRegressionModel - pyspark.ml.regression.LinearRegressionModel
Parameters: model – Spark ML model Returns: scikit-learn model with equivalent predictive behavior. Currently, parameters or arguments for training are not copied.
-
toSpark
(model)¶ Convert a scikit-learn model to a Spark ML model from the Pipelines API (spark.ml). Currently supported models: - sklearn.linear_model.LogisticRegression (binary classification only, not multiclass) - sklearn.linear_model.LinearRegression
Parameters: model – scikit-learn model Returns: Spark ML model with equivalent predictive behavior. Currently, parameters or arguments for training are not copied.
-
-
class
spark_sklearn.
CSRVectorUDT
¶ Bases:
pyspark.sql.types.UserDefinedType
SQL user-defined type (UDT) for scipy.sparse.csr_matrix (vectors only, not matrices).
Note
Experimental
-
class
spark_sklearn.
GridSearchCV
(sc, estimator, param_grid, scoring=None, fit_params=None, n_jobs=1, iid=True, refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score='raise', return_train_score=True)¶ Bases:
sklearn.model_selection._search.BaseSearchCV
Exhaustive search over specified parameter values for an estimator.
Important members are fit, predict.
GridSearchCV implements a “fit” and a “score” method. It also implements “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.
The parameters of the estimator used to apply these methods are optimized by cross-validated grid-search over a parameter grid. Read more in the User Guide.
- estimator : estimator object.
- This is assumed to implement the scikit-learn estimator interface.
Either estimator needs to provide a
score
function, orscoring
must be passed. - param_grid : dict or list of dictionaries
- Dictionary with parameters names (string) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.
- scoring : string, callable or None, default=None
- A string (see model evaluation documentation) or
a scorer callable object / function with signature
scorer(estimator, X, y)
. IfNone
, thescore
method of the estimator is used. - fit_params : dict, optional
- Parameters to pass to the fit method.
- n_jobs : int, default=1
- Number of jobs to run in parallel.
- pre_dispatch : int, or string, optional
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
- None, in which case all the jobs are immediately created and spawned. Use this for lightweight and fast-running jobs, to avoid delays due to on-demand spawning of the jobs
- An int, giving the exact number of total jobs that are spawned
- A string, giving an expression as a function of n_jobs, as in ‘2*n_jobs’
- iid : boolean, default=True
- If True, the data is assumed to be identically distributed across the folds, and the loss minimized is the total loss per sample, and not the mean loss across the folds.
- cv : int, cross-validation generator or an iterable, optional
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- None, to use the default 3-fold cross validation,
- integer, to specify the number of folds in a (Stratified)KFold,
- An object to be used as a cross-validation generator.
- An iterable yielding train, test splits.
For integer/None inputs, if the estimator is a classifier and
y
is either binary or multiclass,StratifiedKFold
is used. In all other cases,KFold
is used. Refer User Guide for the various cross-validation strategies that can be used here.- refit : boolean, default=True
- Refit the best estimator with the entire dataset. If “False”, it is impossible to make predictions using this GridSearchCV instance after fitting.
- verbose : integer
- Controls the verbosity: the higher, the more messages.
- error_score : ‘raise’ (default) or numeric
- Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.
- return_train_score : boolean, default=True
- If
'False'
, thecv_results_
attribute will not include training scores.
>>> from sklearn import svm, datasets >>> from spark_sklearn.grid_search import GridSearchCV >>> from spark_sklearn.util import createLocalSparkSession >>> sc = createLocalSparkSession().sparkContext >>> iris = datasets.load_iris() >>> parameters = {'kernel':('linear', 'rbf'), 'C':[1, 10]} >>> svr = svm.SVC() >>> clf = GridSearchCV(sc, svr, parameters) >>> clf.fit(iris.data, iris.target) ... GridSearchCV(cv=None, error_score=..., estimator=SVC(C=1.0, cache_size=..., class_weight=..., coef0=..., decision_function_shape=..., degree=..., gamma=..., kernel='rbf', max_iter=-1, probability=False, random_state=None, shrinking=True, tol=..., verbose=False), fit_params={}, iid=..., n_jobs=1, param_grid=..., pre_dispatch=..., refit=..., scoring=..., verbose=...) >>> sorted(clf.cv_results_.keys()) ... ['mean_fit_time', 'mean_score_time', 'mean_test_score',... 'mean_train_score', 'param_C', 'param_kernel', 'params',... 'rank_test_score', 'split0_test_score',... 'split0_train_score', 'split1_test_score', 'split1_train_score',... 'split2_test_score', 'split2_train_score',... 'std_fit_time', 'std_score_time', 'std_test_score', 'std_train_score'...]
- cv_results_ : dict of numpy (masked) ndarrays
A dict with keys as column headers and values as columns, that can be imported into a pandas
DataFrame
. For instance the below given tableparam_kernel param_gamma param_degree split0_test_score … rank_…. ‘poly’ – 2 0.8 … 2 ‘poly’ – 3 0.7 … 4 ‘rbf’ 0.1 – 0.8 … 3 ‘rbf’ 0.2 – 0.9 … 1 - will be represented by a
cv_results_
dict of:: { ‘param_kernel’: masked_array(data = [‘poly’, ‘poly’, ‘rbf’, ‘rbf’],
mask = [False False False False]…)- ‘param_gamma’: masked_array(data = [– – 0.1 0.2],
- mask = [ True True False False]…),
- ‘param_degree’: masked_array(data = [2.0 3.0 – –],
- mask = [False False True True]…),
‘split0_test_score’ : [0.8, 0.7, 0.8, 0.9], ‘split1_test_score’ : [0.82, 0.5, 0.7, 0.78], ‘mean_test_score’ : [0.81, 0.60, 0.75, 0.82], ‘std_test_score’ : [0.02, 0.01, 0.03, 0.03], ‘rank_test_score’ : [2, 4, 3, 1], ‘split0_train_score’ : [0.8, 0.9, 0.7], ‘split1_train_score’ : [0.82, 0.5, 0.7], ‘mean_train_score’ : [0.81, 0.7, 0.7], ‘std_train_score’ : [0.03, 0.03, 0.04], ‘mean_fit_time’ : [0.73, 0.63, 0.43, 0.49], ‘std_fit_time’ : [0.01, 0.02, 0.01, 0.01], ‘mean_score_time’ : [0.007, 0.06, 0.04, 0.04], ‘std_score_time’ : [0.001, 0.002, 0.003, 0.005], ‘params’ : [{‘kernel’: ‘poly’, ‘degree’: 2}, …], }
NOTE that the key
'params'
is used to store a list of parameter settings dict for all the parameter candidates. Themean_fit_time
,std_fit_time
,mean_score_time
andstd_score_time
are all in seconds.- will be represented by a
- best_estimator_ : estimator
- Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
- best_score_ : float
- Score of best_estimator on the left out data.
- best_params_ : dict
- Parameter setting that gave the best results on the hold out data.
- best_index_ : int
- The index (of the
cv_results_
arrays) which corresponds to the best candidate parameter setting. The dict atsearch.cv_results_['params'][search.best_index_]
gives the parameter setting for the best model, that gives the highest mean score (search.best_score_
). - scorer_ : function
- Scorer function used on the held out data to choose the best parameters for the model.
- n_splits_ : int
- The number of cross-validation splits (folds/iterations).
The parameters selected are those that maximize the score of the left out data, unless an explicit score is passed in which case it is used instead. If n_jobs was set to a value higher than one, the data is copied for each point in the grid (and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 * n_jobs.
ParameterGrid
:- generates all the combinations of a hyperparameter grid.
sklearn.model_selection.train_test_split()
:- utility function to split the data into a development set usable for fitting a GridSearchCV instance and an evaluation set for its final evaluation.
sklearn.metrics.make_scorer()
:- Make a scorer from a performance metric or loss function.
-
fit
(X, y=None, groups=None)¶ Run fit with all sets of parameters.
- X : array-like, shape = [n_samples, n_features]
- Training vector, where n_samples is the number of samples and n_features is the number of features.
- y : array-like, shape = [n_samples] or [n_samples, n_output], optional
- Target relative to X for classification or regression; None for unsupervised learning.
- groups : array-like, with shape (n_samples,), optional
- Group labels for the samples used while splitting the dataset into train/test set.
-
spark_sklearn.
gapply
(grouped_data, func, schema, *cols)¶ Applies the function
func
to data grouped by key. In particular, given a dataframe grouped by some set of key columns key1, key2, …, keyn, this method groups all the values for each row with the same key columns into a single Pandas dataframe and by default invokesfunc((key1, key2, ..., keyn), values)
where the number and order of the key arguments is determined by columns on which this instance’s parentDataFrame
was grouped andvalues
is apandas.DataFrame
of columns selected bycols
, in that order.If there is only one key then the key tuple is automatically unpacked, with
func(key, values)
called.func
is expected to return apandas.DataFrame
of the specified schemaschema
, which should be of typeStructType
(output columns are of this name and order).If
spark.conf.get("spark.sql.retainGroupColumns")
is notu'true'
, thenfunc
is called with an empty key tuple (note it is set tou'true'
by default).If no
cols
are specified, then all grouped columns will be offered, in the order of the columns in the original dataframe. In either case, the Pandas columns will be named according to the DataFrame column names.The order of the rows passed in as Pandas rows is not guaranteed to be stable relative to the original row order.
Note: Users must ensure that the grouped values for every group must fit entirely in memory.
Note: This method is only available if Pandas is installed.
Parameters: - func – a two argument function, which may be either a lambda or named function
- schema – the return schema for
func
, aStructType
- cols – list of column names (string only)
Raises: - ValueError – if
"*"
is incols
- ValueError – if
cols
contains duplicates - ValueError – if
schema
is not aStructType
- ImportError – if
pandas
module is not installed - ImportError – if
pandas
version is too old (less than 0.7.1)
Returns: the new
DataFrame
with the original key columns replicated for each returned value in each group’s resulting pandas dataframe, the schema being the original key schema prepended toschema
, where all the resulting groups’ rows are concatenated. Of course, if retaining group columns is disabled, then the output will exactly matchschema
since no keys can be prepended.>>> import pandas as pd >>> from pyspark.sql import SparkSession >>> from spark_sklearn.group_apply import gapply >>> from spark_sklearn.util import createLocalSparkSession >>> spark = createLocalSparkSession() >>> df = (spark ... .createDataFrame([Row(course="dotNET", year=2012, earnings=10000), ... Row(course="Java", year=2012, earnings=20000), ... Row(course="dotNET", year=2012, earnings=5000), ... Row(course="dotNET", year=2013, earnings=48000), ... Row(course="Java", year=2013, earnings=30000)]) ... .select("course", "year", "earnings")) >>> def yearlyMedian(_, vals): ... all_years = set(vals['year']) ... # Note that interpolation is performed, so we need to cast back to int. ... yearly_median = [(year, int(vals['earnings'][vals['year'] == year].median())) ... for year in all_years] ... return pd.DataFrame.from_records(yearly_median) >>> newSchema = StructType().add("year", LongType()).add("median_earnings", LongType()) >>> gapply(df.groupBy("course"), yearlyMedian, newSchema).orderBy("median_earnings").show() +------+----+---------------+ |course|year|median_earnings| +------+----+---------------+ |dotNET|2012| 7500| | Java|2012| 20000| | Java|2013| 30000| |dotNET|2013| 48000| +------+----+---------------+ >>> def twoKeyYearlyMedian(_, vals): ... return pd.DataFrame.from_records([(int(vals["earnings"].median()),)]) >>> newSchema = StructType([df.schema["earnings"]]) >>> gapply(df.groupBy("course", "year"), twoKeyYearlyMedian, newSchema, "earnings").orderBy( ... "earnings").show() +------+----+--------+ |course|year|earnings| +------+----+--------+ |dotNET|2012| 7500| | Java|2012| 20000| | Java|2013| 30000| |dotNET|2013| 48000| +------+----+--------+ >>> spark.stop(); SparkSession._instantiatedContext = None
Keyed Models¶
The use case that this addresses is where a client has a dataset with many keys - the distribution of which is such that the total number of rows for with a shared key value can be contained completely in memory on a single machine.
This assumption is particularly enabling because clients may wish to apply more intricate single-machine models (such as a scikit-learn estimator) to every user.
The API provided here generalizes the scikit-learn estimator interface to the Spark ML one; in particular, it allows clients to train their scikit-learn estimators in parallel over a grouped and aggregated dataframe.
>>> from sklearn.linear_model import LinearRegression
>>> from sklearn.cluster import KMeans
>>> from pyspark.ml.linalg import Vectors, Matrices, MatrixUDT
>>> from pyspark.sql.functions import udf
>>> from pyspark.sql import SparkSession
>>> from spark_sklearn.util import createLocalSparkSession
>>> from spark_sklearn.keyed_models import KeyedEstimator
>>> spark = createLocalSparkSession()
>>> df = spark.createDataFrame([(user,
... Vectors.dense([i, i ** 2, i ** 3]),
... 0.0 + user + i + 2 * i ** 2 + 3 * i ** 3)
... for user in range(3) for i in range(5)])
>>> df = df.toDF("key", "features", "y")
>>> df.where("5 < y and y < 10").sort("key", "y").show()
+---+-------------+---+
|key| features| y|
+---+-------------+---+
| 0|[1.0,1.0,1.0]|6.0|
| 1|[1.0,1.0,1.0]|7.0|
| 2|[1.0,1.0,1.0]|8.0|
+---+-------------+---+
>>> km = KeyedEstimator(sklearnEstimator=LinearRegression(), yCol="y").fit(df)
>>> def printFloat(x):
... return "{:.2f}".format(round(x, 2))
...
>>> def printModel(model):
... coef = "[" + ", ".join(map(printFloat, model.coef_)) + "]"
... intercept = printFloat(model.intercept_)
... return "intercept: {} coef: {}".format(intercept, coef)
...
>>> km.keyedModels.columns
['key', 'estimator']
>>> printedModels = udf(printModel)("estimator").alias("linear fit")
>>> km.keyedModels.select("key", printedModels).sort("key").show(truncate=False)
+---+----------------------------------------+
|key|linear fit |
+---+----------------------------------------+
|0 |intercept: 0.00 coef: [1.00, 2.00, 3.00]|
|1 |intercept: 1.00 coef: [1.00, 2.00, 3.00]|
|2 |intercept: 2.00 coef: [1.00, 2.00, 3.00]|
+---+----------------------------------------+
Now that we have generated a linear model for each key, we can apply it to keyed test data. In the following, we only show one point for simplicity, but the test data can contain multiple points for multiple different keys.
>>> input = spark.createDataFrame([(0, Vectors.dense(3, 1, -1))]).toDF("key", "features")
>>> km.transform(input).withColumn("output", udf(printFloat)("output")).show()
+---+--------------+------+
|key| features|output|
+---+--------------+------+
| 0|[3.0,1.0,-1.0]| 2.00|
+---+--------------+------+
Suppose we wanted to perform key-based clustering. The most common use case would require just fitting our model.
>>> km = KeyedEstimator(sklearnEstimator=KMeans(random_state=0, n_clusters=2)).fit(df)
>>> def getCentroids(kmeans):
... n_clusters, n_features = kmeans.cluster_centers_.shape
... return Matrices.dense(n_clusters, n_features, kmeans.cluster_centers_.reshape(-1))
...
>>> centroids = udf(getCentroids, MatrixUDT())("estimator").alias("centroids")
>>> km.keyedModels.select("key", centroids).sort("key").show()
+---+--------------------+
|key| centroids|
+---+--------------------+
| 0|4.0 64.0 3.5 ...|
| 1|4.0 64.0 3.5 ...|
| 2|4.0 64.0 3.5 ...|
+---+--------------------+
Usually, this is all we want. In the case of KMeans
, we can also predict cluster labels,
since the scikit-learn estimator provides this functionality. Note this is not the case for
some other clusterers, such as DBSCAN
.
>>> km.transform(input).withColumnRenamed("output", "cluster label").show()
+---+--------------+-------------+
|key| features|cluster label|
+---+--------------+-------------+
| 0|[3.0,1.0,-1.0]| 1|
+---+--------------+-------------+
>>> spark.stop(); SparkSession._instantiatedContext = None # clear hidden SparkContext for reuse
-
class
spark_sklearn.keyed_models.
KeyedEstimator
(*args, **kwargs)¶ Bases:
pyspark.ml.base.Estimator
A
KeyedEstimator
provides an interface for training per-key scikit-learn estimators.The
KeyedEstimator
can be part of any Spark ML pipeline provided the columns are appropriately matched.Currently, the class provides a generalization for scikit-learn transformers, clusterers, and predictors. Because these scikit-learn estimators all derive from the same base type (yielding the same API), yet have different expectations for what methods should be called and with what arguments, this class enumerates three different types of behavior:
"transformer"
Examples:
sklearn.decomposition.PCA
,sklearn.cluster.KMeans
In this case, the estimator will aggregate the all input features for a given key into a NxD data matrix, where N is the number of rows with the given key and D is the feature space dimensionality; let this matrix be
X
.For each such key and data matrix pair, a clone of the parameter estimator is fitted with
estimator.fit(X)
, inducing a mapping between keys and fitted estimators: this produces a fitted transformerKeyedModel
, whose Spark MLtransform()
method generates an output column by applying each key’s fitted scikit-learn estimator’s owntransform
method.The output column type for transformers will always be a
DenseVector
."clusterer"
Examples:
sklearn.cluster.DBSCAN
,sklearn.cluster.KMeans
As before, the data will be aggregated into a design matrix
X
, andestimator.fit(X)
will be called for each key group.The difference between a
"transformer"
and"clusterer"
lies in their prediction behavior: a clusterer will callestimator.predict()
whereas a transformer refers to thetransform
method.The output column type for clusterers will always be of
LongType
."predictor"
Examples:
sklearn.svm.LinearSVC
,sklearn.linear_model.ElasticNet
Here, the estimator will likewise aggregate input features into the data matrix
X
. In addition, the label column will be aggregated in a collated manner, generating a vectory
for each key. The estimator clone will be fitted withestimator.fit(X, y)
.A predictor
KeyedModel
transforms its input dataframe by generating an output column with the output of the estimator’spredict
method.The output column type for predictors will be the same as the label column (which must be an
AtomicType
(else aTypeError
will be thrown atfit()
-time).
The input column should be numeric or a vector (else a
TypeError
will be thrown atfit()
-time). Don’t use “estimator” as a column name.- In certain cases, a scikit-learn estimator may support both
"transformer"
and"clusterer"
interfaces.sklearn.cluster.KMeans
, for instance, supports both the cluster-labelling operationpredict()
and a transformation into cluster-mean-distance space. Such ambiguity is resolved by prefering clustering. It may be overriden by manually specifying theestimatorType
totransformer
in theKeyedEstimator
constructor. - Key-based grouping only occurs during training. During the transformation/prediction phase of computation, the output is unaggregated: the number of rows inputted as test data will be equal to the number of rows outputted.
spark.conf.get("spark.sql.retainGroupColumns")
assumed to beu"true"
. This is the case by default for Spark 1.4+. This is necessary for both the keyed estimator and the keyed model.- Estimators trained, persisted, and loaded across different scikit-learn versions are not guaranteed to work.
For all instances, the ordered list of
keyCols
determine the set of groups which eachsklearnEstimator
is applied to.For every unique
keyCols
value, the remaining columns are aggregated and used to train the scikit-learn estimator.estimatorType
inference is conducted as follows: ifyCol
is specified, then this is assumed to be of"predictor"
type, else a"transformer"
or a"clusterer"
, depending on the estimator having thetransform()
orfit_predict()
attributes, with"clusterer"
being chosen in case both attributes are present.Parameters: - sklearnEstimator – An instance of a scikit-learn estimator, with parameters configured as desired for each user.
- keyCols – Key column names list used to group data to which models are applied, where order implies lexicographical importance.
- xCol – Name of column of input features used for training and transformation/prediction.
- yCol – Specifies name of label column for regression or classification pipelines.
Required for predictors, must be unspecified or
None
for transformers. - estimatorType – Identifies the type of scikit-learn estimator being used, which
changes the interface the
sklearnEstimator
is expected to have. This parameter’s value is inferred using reflection by default, but may be manually overriden.
Raises: - ValueError – if
sklearnEstimator
isNone
. - ValueError – if
sklearnEstimator
does not derive fromsklearn.base.BaseEstimator
. - ValueError – if
keyCols
is empty. - ValueError – if any column has the name
"estimator"
- AttributeError – if reflection checks indicate that parameter estimator is not equipped
with a
fit()
method.
-
fit
(dataset, params=None)¶ Fits a model to the input dataset with optional parameters.
Parameters: - dataset – input dataset, which is an instance of
pyspark.sql.DataFrame
- params – an optional param map that overrides embedded params. If a list/tuple of param maps is given, this calls fit on each param map and returns a list of models.
Returns: fitted model(s)
New in version 1.3.0.
- dataset – input dataset, which is an instance of
-
sklearnEstimatorType
¶ Returns: the estimator type of this keyed estimator
-
class
spark_sklearn.keyed_models.
KeyedModel
(*args, **kwargs)¶ Bases:
pyspark.ml.base.Model
Represents a Spark ML Model, generated by a fitted
KeyedEstimator
.Wraps fitted scikit-learn estimators - at transformation time transforms the input for each key using a key-specific model. See
KeyedEstimator
documentation for details.If no estimator is present for a given key at transformation time, the prediction is null.
The constructor is used by
KeyedEstimator
to generate aKeyedModel
; it is not intended for external use.-
keyedModels
¶ Returns: Returns the keyedSklearnEstimators
param, aDataFrame
with columnskeyCols
(where each key is unique) and the column"estimator"
containing the fitted scikit-learn estimator as aSparkSklearnEstimator
.
-
sklearnEstimatorType
¶ Returns: the estimator type of this keyed model
-
transform
(dataset, params=None)¶ Transforms the input dataset with optional parameters.
Parameters: - dataset – input dataset, which is an instance of
pyspark.sql.DataFrame
- params – an optional param map that overrides embedded params.
Returns: transformed dataset
New in version 1.3.0.
- dataset – input dataset, which is an instance of
-
-
class
spark_sklearn.keyed_models.
SparkSklearnEstimator
(estimator)¶ Bases:
object
SparkSklearnEstimator
is a wrapper for containing scikit-learn estimators in dataframes - any estimators need to be stored inside the wrapper class to be properly serialized/deserialized in dataframe operations.Note any method called on the estimator this object wraps may be called on the wrapper instead.
Initializes with the parameter estimator.
Param: estimator: scikit-learn estimator to contain. -
estimator
¶ Returns: the underlying estimator
-


eLCS package¶
Submodules¶
eLCS.Algorithm module¶
Name: eLCS_Algorithm.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: The major controlling module of eLCS. Includes the major run loop which controls learning over a specified number of iterations. Also includes
periodic tracking of estimated performance, and checkpoints where complete evaluations of the eLCS rule population is performed.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
-
class
eLCS.Algorithm.
eLCS
[source]¶ Bases:
object
-
doContPopEvaluation
(isTrain)[source]¶ Performs evaluation of population via the copied environment. Specifically developed for continuous phenotype evaulation. The population is maintained unchanging throughout the evaluation. Works on both training and testing data.
-
doPopEvaluation
(isTrain)[source]¶ Performs a complete evaluation of the current rule population. The population is unchanged throughout this evaluation. Works on both training and testing data.
-
eLCS.ClassAccuracy module¶
Name: eLCS_ClassAccuracy.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Manages the logistical aspects of balance accuracy calculations which can handle unbalanced datasets, and/or datasets with multiple discrete classes.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
eLCS.Classifier module¶
Name: eLCS_Classifier.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: This module defines an individual classifier within the rule population, along with all respective parameters.
Also included are classifier-level methods, including constructors(covering, copy, reboot) matching, subsumption, crossover, and mutation. Parameter update methods are also included.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
-
class
eLCS.Classifier.
Classifier
(a=None, b=None, c=None, d=None)[source]¶ Bases:
object
-
Mutation
(state, phenotype)[source]¶ Mutates the condition of the classifier. Also handles phenotype mutation. This is a niche mutation, which means that the resulting classifier will still match the current instance.
-
classifierCopy
(clOld, exploreIter)[source]¶ Constructs an identical Classifier. However, the experience of the copy is set to 0 and the numerosity is set to 1 since this is indeed a new individual in a population. Used by the genetic algorithm to generate offspring based on parent classifiers.
-
classifierCovering
(setSize, exploreIter, state, phenotype)[source]¶ Makes a new classifier when the covering mechanism is triggered. The new classifier will match the current training instance. Covering will NOT produce a default rule (i.e. a rule with a completely general condition).
-
equals
(cl)[source]¶ Returns if the two classifiers are identical in condition and phenotype. This works for discrete or continuous attributes or phenotypes.
-
isMoreGeneral
(cl)[source]¶ Returns if the classifier (self) is more general than cl. Check that all attributes specified in self are also specified in cl.
-
isSubsumer
()[source]¶ Returns if the classifier (self) is a possible subsumer. A classifier must be as or more accurate than the classifier it is trying to subsume.
-
rebootClassifier
(classifierList)[source]¶ Rebuilds a saved classifier as part of the population Reboot
-
uniformCrossover
(cl)[source]¶ Applies uniform crossover and returns if the classifiers changed. Handles both discrete and continuous attributes. #SWARTZ: self. is where for the better attributes are more likely to be specified #DEVITO: cl. is where less useful attribute are more likely to be specified
-
updateCorrect
()[source]¶ Increases the correct phenotype tracking by one. Once an epoch has completed, rule accuracy can’t change.
-
updateExperience
()[source]¶ Increases the experience of the classifier by one. Once an epoch has completed, rule accuracy can’t change.
-
eLCS.ClassifierSet module¶
Name: eLCS_ClassifierSet.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: This module handles all classifier sets (population, match set, correct set) along with mechanisms and heuristics that act on these sets.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
-
class
eLCS.ClassifierSet.
ClassifierSet
(a=None)[source]¶ Bases:
object
-
addClassifierToPopulation
(cl, covering)[source]¶ Adds a classifier to the set and increases the microPopSize value accordingly.
-
clearSets
()[source]¶ Clears out references in the match and correct sets for the next learning iteration.
-
deleteFromCorrectSet
(deleteRef)[source]¶ Delete reference to classifier in population, contained in self.corectSet.
-
deleteFromMatchSet
(deleteRef)[source]¶ Delete reference to classifier in population, contained in self.matchSet.
-
deleteFromPopulation
()[source]¶ Deletes one classifier in the population. The classifier that will be deleted is chosen by roulette wheel selection considering the deletion vote. Returns the macro-classifier which got decreased by one micro-classifier.
-
deletion
(exploreIter)[source]¶ Returns the population size back to the maximum set by the user by deleting rules.
-
doCorrectSetSubsumption
()[source]¶ Executes correct set subsumption. The correct set subsumption looks for the most general subsumer classifier in the correct set and subsumes all classifiers that are more specific than the selected one.
-
getPopTrack
(accuracy, exploreIter, trackingFrequency)[source]¶ Returns a formated output string to be printed to the Learn Track output file.
-
insertDiscoveredClassifiers
(cl1, cl2, clP1, clP2, exploreIter)[source]¶ Inserts both discovered classifiers and activates GA subsumption if turned on. Also checks for default rule (i.e. rule with completely general condition) and prevents such rules from being added to the population, as it offers no predictive value within eLCS.
-
makeEvalMatchSet
(state)[source]¶ Constructs a match set for evaluation purposes which does not activate either covering or deletion.
-
makeMatchSet
(state_phenotype, exploreIter)[source]¶ Constructs a match set from the population. Covering is initiated if the match set is empty or a rule with the current correct phenotype is absent.
-
runAttGeneralitySum
(isEvaluationSummary)[source]¶ Determine the population-wide frequency of attribute specification, and accuracy weighted specification. Used in complete rule population evaluations.
-
runGA
(exploreIter, state, phenotype)[source]¶ The genetic discovery mechanism in eLCS is controlled here.
-
runPopAveEval
(exploreIter)[source]¶ Calculates some summary evaluations across the rule population including average generality.
-
selectClassifierRW
()[source]¶ Selects parents using roulette wheel selection according to the fitness of the classifiers.
-
selectClassifierT
()[source]¶ Selects parents using tournament selection according to the fitness of the classifiers.
-
setIterStamps
(exploreIter)[source]¶ Sets the time stamp of all classifiers in the set to the current time. The current time is the number of exploration steps executed so far.
-
subsumeClassifier
(cl=None, cl1P=None, cl2P=None)[source]¶ Tries to subsume a classifier in the parents. If no subsumption is possible it tries to subsume it in the current set.
-
eLCS.ConfigParser module¶
Name: eLCS_ConfigPars.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Manages the configuration file, by loading it, parsing it and passing values to the ‘Constants’ module.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
-
class
eLCS.ConfigParser.
ConfigParser
(filename)[source]¶ Bases:
object
This class docstring shows how to use sphinx and rst syntax
The first line is brief explanation, which may be completed with a longer one. For instance to discuss about its methods. The only method here is
function1()
’s. The main idea is to document the class and methods’s arguments withparameters, types, return and return types:
:param arg1: description :param arg2: description :type arg1: type description :type arg1: type description :return: return description :rtype: the return type description
and to provide sections such as Example using the double commas syntax:
:Example: followed by a blank line !
which appears as follow:
Example: followed by a blank line
Finally special sections such as See Also, Warnings, Notes use the sphinx syntax (paragraph directives):
.. seealso:: blabla .. warnings also:: blabla .. note:: blabla .. todo:: blabla
Note
- There are many other Info fields but they may be redundant:
- param, parameter, arg, argument, key, keyword: Description of a parameter.
- type: Type of a parameter.
- raises, raise, except, exception: That (and when) a specific exception is raised.
- var, ivar, cvar: Description of a variable.
- returns, return: Description of the return value.
- rtype: Return type.
Note
There are many other directives such as versionadded, versionchanged, rubric, centered, … See the sphinx documentation for more details.
Here below is the results of the
function1()
docstring.
eLCS.Constants module¶
Name: eLCS_Constants.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Stores and manages all algorithm run parameters, making them accessible anywhere in the rest of the algorithm code by (cons.) .
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
eLCS.DataManagement module¶
Name: eLCS_DataManagement.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Able to manage both training and testing data. This module loads the dataset, detects and characterizes all attributes in the dataset,
handles missing data, and finally formats the data so that it may be conveniently utilized by eLCS.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
-
class
eLCS.DataManagement.
DataManagement
(trainFile, testFile, infoList=None)[source]¶ Bases:
object
-
characterizeAttributes
(rawData)[source]¶ Determine range (if continuous) or states (if discrete) for each attribute and saves this information
-
compareDataset
(rawTestData)[source]¶ Ensures that the attributes in the testing data match those in the training data. Also stores some information about the testing data.
-
discriminateAttributes
(rawData)[source]¶ Determine whether attributes in dataset are discrete or continuous and saves this information.
-
discriminateClasses
(rawData)[source]¶ Determines number of classes and their identifiers. Only used if phenotype is discrete.
-
discriminatePhenotype
(rawData)[source]¶ Determine whether the phenotype is Discrete(class-based) or Continuous
-
eLCS.OfflineEnvironment module¶
Name: eLCS_Offline_Environment.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: In the context of data mining and classification tasks, the environment is a data set with a limited number of instances with X attributes
and some endpoint (typically a discrete phenotype or class) of interest. This module loads the data set, automatically detects features of the data.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
eLCS.OutputFileManager module¶
Name: eLCS_OutputFileManager.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: This module contains the methods for generating the different output files generated by eLCS.
These files are generated at each learning checkpoint, and the last iteration. These include… writePopStats: Summary of the population statistics writePop: Outputs a snapshot of the entire rule population including classifier conditions, classes, and parameters.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
eLCS.Prediction module¶
Name: eLCS_Prediction.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Given a match set, this module uses a voting scheme to select the phenotype prediction. Set up to handle both discrete and continuous phenotypes.
Also set up to try and handle prediction ties if possible.
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————
eLCS.Timer module¶
Name: eLCS_Timer.py Authors: Ryan Urbanowicz - Written at Dartmouth College, Hanover, NH, USA Contact<EMAIL_ADDRESS>Created: November 1, 2013 Description: Tracks and stores the run time of algorithm and some of it’s major components
algorithm published by Stewart Wilson (1995).
Copyright (C) 2013 Ryan Urbanowicz This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABLILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA ———————————————————————————————————————————————————


Ocean [aq50], ocean, in The Inner Sea
Last updated Turn #104
Ocean [aq50], ocean, in The Inner Sea
------------------------------------------------------------------------
22: Whalebone [fe4m], ship, 6 hulls, 1 rowing port, 18 sails, 1% damaged,
22: arrived from Ocean [aq51], owner:
22: Toulouse [h1q], Pen, with 20 sailors, 15 lt. xbowmen,
22: accompanied by:
22: Jonathan [h9k], Pen, duke, with three skirmishers, 16 pikemen,
22: 20 soldiers
22: Halmary [f9m], Pen, with 15 lt. xbowmen
23: Whalebone [fe4m], ship, 6 hulls, 1 rowing port, 18 sails, 1% damaged,
23: departed for Ocean [aq49].
Routes leaving Ocean:
North, to Ocean [ap50], 6 days
East, to Ocean [aq51], 5 days
South, to Ocean [ar50], 6 days
West, to Ocean [aq49], 5 days


Welcome
Start using the Roulette Predictor in 3 easy steps.
Step 1
Using the number pad and the Enter button, please select the last 3 rolls.
Step 2
The selected numbers appear in the roll stack and as you enter the 3rd roll,
the probabilities appear on the board.
Step 3
Based on the algorithm, you will see the number of times a particular roll appeared after the enetered combination of last 3 rolls.
Get Started


On 14 February 2018, 6 new articles were published in 6 journals.
Imprisoned Mothers in Victorian England, 1853–1900: Motherhood, Identity and the Convict Prison
Donne, Emigrazione E Prostituzione in Europa: Non Si Tratta Di “Sex Work”
Understanding the Role of Marriage in Black Women’s Offending Over the Life Course
Looking Into the Day-to-Day Process of Victim Safety Planning
Assessment of Stress and Resiliency in Emergency Dispatchers


Twin-Turbo AWD Dodge Charger Has 1,525 HP, Carbon Widebody Kit
SpeedKore Demon levels of power in combination with an all-wheel drive and four doors.
by Anthony KarrThis year’s SEMA Show is shifting into a higher gear with the debut of this twin-turbo AWD Dodge Charger. Teased for the first time a month ago and dubbed as the “world’s first all-wheel-drive, twin-turbocharged, carbon-fiber widebody” Charger, the vehicle has been developed by SpeedKore in cooperation with MagnaFlow and HP Tuners.
The project uses a 2019 Dodge Charger Pursuit as a starting point to which SpeedKore installed a complete widebody carbon-fiber kit. The tuners received a widebody Charger from the automaker itself to 3D scan it and create the necessary molds. The result is a kit, consisting of a carbon-fiber front bumper, fender flare set, factory rocker set, hood, rear spoiler, rear bumper, and rear diffuser.
Gallery: Twin-turbo AWD Dodge Charger by SpeedKore
We know you are more interested in what’s under the hood so let’s get straight to the point. There’s a 6.2-liter twin-turbocharged V8 in the engine bay, very similar to the one found in the SpeedKore Demon. It has been heavily modified and now features two Precision 6466 ball-bearing billet turbochargers instead of the factory supercharger. A triple-pump fuel system with Injector Dynamics 1700cc injectors is delivering E85 to the motor.
Some Of SpeedKore's Other Projects:
The initial dyno tests showed 1,525 horsepower (1,137 kilowatts) and the crank with a “conservative 26 pounds of boost.” SpeedKore will continue to work on the engine before and after the 2019 SEMA Show, so we can expect this number to increase slightly with time. Power is channeled to all four wheels through a Hellraiser Performance-built transmission with a custom billet transfer case created by Traction Products, an FTI torque converter, and a carbon-fiber driveshaft from The Driveshaft Shop.
The car displayed in Las Vegas is a one-off project commissioned by a returning SpeedKore customer. He ordered the speedy sedan for his brother’s 65th birthday. We are eager to see it in action and especially in a direct drag race with the SpeedKore Demon, which is believed to be the fastest Demon on the planet.
Source: MagnaFlow
Show press release
MagnaFlow Unveils World’s First All-Wheel-Drive, Twin-Turbocharged, Carbon-Fiber Widebody Dodge Charger by SpeedKore at the 2019 SEMA Show
The MagnaFlow-equipped carbon-fiber widebody Charger utilizes SpeedKore’s world record-setting turbocharged Demon platform in combination with an all-wheel-drive drivetrain.
OCEANSIDE, Calif. (November 2019) — MagnaFlow, the industry leader in catalytic converters and performance exhaust systems, in collaboration with SpeedKore Performance Group, unveiled the first carbon-fiber widebody, all-wheel-drive Dodge Charger, powered by a twin-turbo 6.2-liter Dodge Demon V-8, at the largest annual automotive specialty products trade show, The SEMA Show, in Las Vegas.
The starting platform for the drag-focused Charger was a 2019 Dodge Charger Pursuit. To create the carbon-fiber widebody components, SpeedKore received a widebody Charger from Dodge to perform a 3D scan and create the necessary molds. The vehicle was then equipped with a pre-preg carbon-fiber front bumper, fender flare set, factory rocker set, hood, rear spoiler, rear bumper and rear diffuser.
The Charger features a custom-valve exhaust system that exits out of the front bumper for track use and can be easily switched to flow through a MagnaFlow Competition Series exhaust for street driving. The Competition Series exhaust features 3” stainless steel throughout and exits through dual split rear exiting polished tips, providing an aggressive exterior sound while maintaining a moderate interior sound. HP Tuners provided the calibration software for the vehicle, which made 1,525 horsepower to the crank on a conservative 26 pounds of boost during the initial dyno-tuning session. SpeedKore will continue to tune for performance before and after the SEMA Show.
Under the hood is the same turbocharged 6.2-liter V-8 engine found in the record-setting SpeedKore Demon, which has been modified by Gearhead Fabrications to replace the factory supercharger with two Precision 6466 ball-bearing billet turbochargers and a custom upper plenum designed and engineered by SpeedKore. To handle the increased boost levels, the engine has been outfitted with custom Thitek heads and a Boost Leash C02 progressive boost controller. Delivering E85 to the engine is a Fore Innovations triple-pump fuel system through Injector Dynamics 1700cc injectors.
Unlike the Demon, the Charger features all-wheel drive and delivers power to the ground through a Hellraiser Performance-built transmission with a custom billet transfer case created by Traction Products, an FTI torque converter and carbon-fiber driveshaft from The Driveshaft Shop, all being cooled by a Gearhead Fabrications transmission cooler. The final component putting power to the ground is a set of Bogart Competition Series wheels wrapped in Mickey Thompson ET Street tires on all four corners.
The one-off Dodge Charger was built for a returning SpeedKore customer, Alex Palermo, who previously commissioned the Wisconsin-based performance shop to build the twin-turbo Dodge Demon that holds the Demon quarter-mile record. After seeing the twin-turbo Demon, his older brother, whom Palermo credits for his love of all things Mopar, said he liked the car but was more of a four-door guy. Palermo took those words to heart and enlisted SpeedKore to build a four-door version of his Demon as a surprise for his brother's 65th birthday.


$def with ()
About
Accessible reliable and fully described experimental data is critically lacking in the materials/mechanics community to validate accurate predictive models. Our project proposes a platform for researchers to present their experimental results as standardized datasets. Experimental researchers can then obtain a DOI for their datasets, making them citable by others.


https://github.com/Cybereason/Invoke-WMILM
Invoke-WMILM
This is a PoC script for various methods to acheive authenticated remote code execution via WMI, without (at least directly) using the Win32_Process class. The type of technique is determined by the "Type" parameter.
Parameters- Target - Name or IP of target machine
- Type - The type of technique to use
- Name - For techniques creating named objects (services, tasks etc.)
- Command - Executable to run
- CommandArgs - Arguments to the executable
- CleanUp - an optional phase to remove artifacts created by the various techniques
- Username
- Password
Supported Techniques- DerivedProcess - Creates a class deriving from Win32_Process, and calls the Create method of that class
- Service - Creates a service and runs it using WMI. Basically PSEXEC with different network traffic
- Job - Creates an at.exe style scheduled task to run in 30 seconds. Does not work on Win8+, unless at.exe is enabled
- Task - Creates an schtasks.exe style scheduled task and runs it. Works only on Win8+
- Product - Runs an arbitrary MSI file from a given path (given by the Command parameter)
- Provider - Creates a new provider with the command and arguments as the underlying COM object, and loads it


Text Above
Why nuts are shaped the way they are
They started out most round but a few had spikes
A nut cracker enjoy crushing the round nuts
The spikey nuts survive to produce more spikey nuts
They don't worry about nut crackers anymore
Text Below


What is Maptime?
- A time for learning about maps
- Hands-on
- Beginner-focused
- Emphasis on open source, programming, and web-mapping
Tonight:
The Basics of Geodata, Geodesy, and Geocoding within the context of Open Street Map
If following along, use left and right arrows to navigate.
PLEASE interrupt if you have questions!
OSM is an editable database with global coverage
OSM is licensed under the Open Database License:
ODbL
Anyone can edit OSM
So, let's sign up
now!
While everyone is doing that...here are some resources for later:
Let's get started!
Two ways to view/edit OSM:
- web (basic): iD
- desktop (advanced): JOSM
Go to
OSM and, assuming you've signed in, click
edit in top left corner
Are you able to scroll around and zoom in (i.e., is our wifi fast enough here)?
If so, find the
Carnegie Museum in Oakland
What are we looking at here?
Geodata
Geographic data, or Geodata, represents things that have a location.
This can include
defined physical or abstract features like
- roads
- buildings
- census boundaries
BUILDINGS
It can also include
temporal or ephemeral events, like
- 911 calls
- cloud cover
- trending, geo-located tweets
There are numerous types of geodata.
Let's talk about the data types you can see right in front of you, in OSM.
Geodata typically is stored and managed in one of two formats:
RASTER
Raster data stores its geographic information in pixels
PIXELS
Pixels can represent color, height, slope, direction...and many other classifications or gradients.
CONTINENTS
Raster data is commonly encountered in:
- satellite imagery
- weather data
- digital elevation model
Elevation
Rasters can be derived from other rasters.
e.g., a digital elevation model can be analyzed to create...
Slope
"Hillshade"
Raster analysis is
very powerful. The format is conducive to complex spatial analyses and modeling:
- kernel density (a.k.a."heat-mapping")
- overlay (e.g., site suitability)
- flow (e.g., surficial hydrology)
In OSM, raster data is provided in the form of satellite imagery.
Try changing the sources and transparencies.
Layers provide sources for
validation.
Vector
Vector data stores geometry, attribute, and location information
No matter how much you 'zoom in', you won't see pixels.
Vector data is dynamically rendered.
Zoom out to see all of
Allegheny County on
OSM.
BUILDINGS
Market Square!
Vector comes (primarily) in three geometric flavors:
The
OSM translation:
- point = point
- line = line or "way"
- polygon = "area"
Editing tools are at the top!
The editing tooltips suggest certain kinds of things are best represented by certain vector types.
Select the
Map Data button and then
Map Features drop-down
Turn some
layers on and off!
Select a feature on the map.
Look at its
attributes!
Attributes are an important component of vector data.
Each individual vector feature is like a row in a table. Each table column is an attribute.
| unique feature id  |
name |
operator |
address |
website |
| 0001  |
Carnegie Museum of Natural History |
Carnegie Institute |
Unknown |
carnegiemnh.org |
| 0002  |
another building |
another operator |
another address |
anotherlink.com |
A vector dataset usually stores lots of individual features.
You'll notice that if you have selected a building or street, you'll see on the map the feature is shown as series of of connected points and lines.
Geometry and geographic referencing!
Remember our Market Square buildings?
Coordinates!
These numbers tell us where the vectors are
located on the globe...more or less!
This brings us to Part 2:
Geodesy
(or, the part where we talk about coordinate systems and projections)
Where do these numbers come from? How do we place geodata on the globe?

Locations are identified using latitude / longitude (lat/lon) decimal degrees.
| LATITUDE |
north-south |
horizontal lines, different circumferences |
-90 to +90 decimal degrees |
| LONGITUDE |
east-west |
vertical lines, same circumferences |
0 to 180 decimal degrees |
You've probably seen depictions of the globe as a map, both with a lat/lon grid.
So...lumpy, sphere-like 3-D object becomes smooth, planar, 2-D object...?
How do we go from this
to this?
Short answer:
Map Projections
mathematical equations for stretching and warping the
lat/long grid to fit the globe to a planar map.
Robinson
Two Point Equidistant
Equidistant Cylindrical
Mercator
Suffice to say, there is no way to project the globe onto a map without distorting it.
Distortion is typically worst at the edges of the map, affecting relative scale.
OSM and other web maps use a common projection:
This distortion at the map edge is why map projections are typically specific to a certain geographic extent.
Pennsylvania State Plane (South)
EPSG: 2272
LOOKS RIDICULOUS
Looks (and functions) well here!
These buildings use EPSG 2272.
But the coordinates shown aren't decimal degrees!
A map projection often also has a built-in conversion to a local unit of distance.
Decimal degrees are not useful for communicating: distances between things, the width of a building facade, or calculating the area of a parcel, etc.
Data in the local PA State Plane projection is stored using
feet.
WHAT DOES IT ALL MEAN
If you have some
geodata from a local source (say the Allegheny County GIS data available on
PASDA) that you want to put on a
web map, or use to contribute to
OSM...
Re-project the data first.
(to Web Mercator or whichever system the web map is using).
Use desktop software like
QGIS - it's a simple "save as"!
Unless of course, you are starting without geodata.
Which you may very well not have yet.
This brings us to Part 3:
Geocoding
So, maybe you don't have any geodata yet. But you have some addresses, or maybe some street intersections.
Geocoding is the process of associating an address, which is only a bit of text, with geodata that references an associated address(es).
Geocoding can be done using a number of different tools.
The APIs and QGIS plugins allow you to geocode a lot of data at once.
For geocoding to work quickly and accurately, the reference data must have accurate
geometry, an accurate
location, and accurate
address attributes.
Addressing around
Pittsburgh and in
Allegheny County is fairly
convoluted, for a lot of reasons.
If you want to your data to be gecoded (or reverse-geocoded) properly, you must choose a reliable data source.
The benefit of using an OSM-based geocoding tool is that if you find a discrepancy in geocoding results, it is within your power to fix the base data!
Not as easy with Google.
OSM FTW
What else can I do with OSM? Suggestions:
Trace some buildings
Add some POIs (points of interest)
Get outside!
Export some data for use in desktop software
What not to do:
When contributing, don't copy from Google!
I've got some data, can I load it into OSM?
What did we learn today?
OSM is an open-source, global, and editable geodata source that anyone can use and improve.
Geodata are the elements that make up a map.
Raster data (pixels) and Vector data (points, lines, polygons) are the basic geodata types.
The world is a lumpy 3-D ball, maps are flat 2-D objects, and map projections take care of the hard work of converting from one to the other.
Web Mercator (EPSG 3857) is the standard map projection for data used in a web map (lat/lon)
Local data is often published in the PA State Plane South (EPSG 2272) projection (feet).
Geocoding is the process of converting addresses to geodata.
There many tools out there for Geocoding.
OSM-based tools allow you some control over the quality of the geocoding.
Thanks!
Christian Gass | @GassChristianB
Matt Mercurio | @Geologistatlarg
With special guests
Patrick Hammons | @hamhandedly of Code for America and formerly of Maptime Philadelphia
and Bob Gradeck of Open Pittsburgh
Thanks to
Maptime HQ for providing some great basic presentation material (from which much of this has been drawn)
Presentation made with
big


Create some Scrawl-canvas entitys which we can use with our Emitter entity
Create a particle Emitter entity
Check to see that the Emitter entity can lock to the mouse cursor
Check the effect of changing the world object's tickMultiplier (world speed) attribute
Check that particle color, and opacity appearance, can be controlled via the user-defined attributes in the world object (particle stroke color, minimum fill color, maximum fill color, opacity)
Check the effects of changing the following emitter/particle attributes: historyLength; generationRate
Confirm that particles comply with changes in the emitter's kill radius attributes
Check that the canvas backgroundColor attribute can be updated (background color)
Change the emitter's artefact; check that the update is immediate and impacts on all existing particles as well as new ones
Navigate away from the web page (open another tab in the browser) and, a few seconds later, navigate back to the page - check that the animation resets and starts, and does not freeze the page


| Interface | Description |
|---|---|
| Callback<T> | |
| IAsyncResult |
Represents the stauts of Asynchronous operation.
|
| IFunction<T,TResult> |
The Interface FuncInterface.
|
| ITraceListener |
ITraceListener handles message tracing.
|
| Class | Description |
|---|---|
| AbstractAsyncCallback | |
| AbstractFolderIdWrapper |
Represents the abstraction of a folder Id.
|
| AbstractItemIdWrapper |
Represents the abstraction of an item Id.
|
| AsyncCallback | |
| AsyncCallbackImplementation | |
| AsyncExecutor | |
| AsyncRequestResult | |
| CalendarActionResults |
Represents the results of an action performed on a calendar item or meeting
message, such as accepting, tentatively accepting or declining a meeting
request.
|
| CallableMethod | |
| ConversationAction |
ConversationAction class that represents
ConversationActionType in the request XML.
|
| DelegateInformation |
Represents the results of a GetDelegates operation.
|
| EwsTraceListener |
EwsTraceListener logs request/response.
|
| ExpandGroupResults |
Represents the results of an ExpandGroup operation.
|
| FolderIdWrapper |
Represents a folder Id provided by a FolderId object.
|
| FolderIdWrapperList |
Represents a list a abstracted folder Ids.
|
| FolderWrapper |
Represents a folder Id provided by a Folder object.
|
| HangingTraceStream |
A stream that traces everything it returns from its Read() call.
|
| IFunctions |
Class with re-usable function implementations.
|
| IFunctions.Base64Decoder | |
| IFunctions.Base64Encoder | |
| IFunctions.DateTimeToXSDateTime | |
| IFunctions.StringToObject | |
| IFunctions.ToBoolean | |
| IFunctions.ToLowerCase | |
| IFunctions.ToString | |
| IFunctions.ToUUID | |
| IFunctionsTest | |
| ImpersonatedUserId |
Represents an impersonated user Id.
|
| ItemIdWrapper |
Represents an item Id provided by a ItemId object.
|
| ItemIdWrapperList |
Represents a list a abstracted item Ids.
|
| ItemWrapper |
Represents an item Id provided by a ItemBase object.
|
| MapiTypeConverter |
Utility class to convert between MAPI Property type values and strings.
|
| MapiTypeConverterMap |
The Class MapiTypeConverterMap.
|
| MapiTypeConverterMapEntry |
Represents an entry in the MapiTypeConverter map.
|
| MobilePhone |
Represents a mobile phone.
|
| NameResolution |
Represents a suggested name resolution.
|
| NameResolutionCollection |
Represents a list of suggested name resolutions.
|
| OutParam<T> |
The Class OutParam.
|
| Param<T> |
The Class Param.
|
| RefParam<T> |
The Class RefParam.
|
| SoapFaultDetails |
Represents SoapFault details.
|
| Time |
Represents time.
|
| TimeSpan |
The Class TimeSpan.
|
| TimeSpanTest |
The Class TimeSpanTest.
|
| UserConfiguration |
Represents an object that can be used to store user-defined configuration
settings.
|


i occasionally contribute to (primarily GNOME-adjacent) open source software with UI/UX, as well as graphic design.
here are some things i had a hand in.
Spot - native, beautiful and responsive Spotify client for GNOME.
Bottles - manager for Wine prefixes.


Home
Service
Book Online
Assembly
Contacts
Customize Your Personal Hair Style
We Provide
Booking Online
Customize Your Personal Hair Style
We Provide
Booking Online
Previous
Next
Here is Title
Here is description
Here is Title
Here is description
Here is Title
Here is description


We graduated over 80 people in Belgium and more than 200 globally.In Belgium, over 70% found and retained a high-quality job as a developer.
Students graduated since HackYourFuture Belgium was founded in 2018
Of our graduates find an internship, a job or start university.
Alumni in the Netherlands, Denmark, Belgium and Canada.
We’re always happy to welcome people like you to our HYF community.Drop us a line and become part of it!
We’re always happy to welcome people like you to our HYF community.Drop us a line and become part of it!


Feel free to get in touch with us if you need anything.
Alias: {{committee[i*3+j].alias}}
College Id: {{committee[i*3+j].college_id}}


--- layout: post title: "My Week in Review" date: 2016-12-18 05:05:00 +0000 tags: image: https://s3.amazonaws.com/audreywatters/2016-12-18-pelican.jpg imagecredits: https://www.flickr.com/photos/bertknot/8121289235 ---
Icon credits: The Noun Project


| boot.strata {BIOsurvey} | R Documentation |
Returns an object of class boot that contains bootstrap estimates of the stratified mean.
boot.strata(data, nless = 0, nresamp = 1, method = c("RESCALE", "MIRROR-MATCH", "BWR", "NAIVE"))
data |
strata object created by Stratify . |
nless |
parameter to set resample sample size for RESCALE method (nless=1,2,3). |
nresamp |
the number of bootstrap resamples to be calculated. |
method |
choose one of four methods of resampling from survey data which incorporate the survey design. RESCALE: Method of Rao and Wu (1988) which resamples with replacement either 1,2 or 3 less than the number of samples within each stratum. MIRROR-MATCH: Method of Sitter (1992). Not implemented yet. BWR (Bootstrap with replacement): Special case of MIRROR-MATCH when the number of population sample units is much larger than the number of sampled units. NAIVE: Simple method of resampling the n sample units with replacement from the n available in each stratum. This method is known to provide biased estimates of the variance of the stratified mean (Rao and Wu 1988). |
WARNING! Although, the bootstrap algorithms has been written as efficiently as possible (using modified versions of the sapply() function) they are still written in R code and may take a few minutes. An implementation using C code with a dynamic load will be considered for a future version.
A list object with class boot containing
orig.mean |
original stratified mean. |
orig.var |
original variance of stratified mean. |
boot.means |
vector of length=nresamp containing bootstrap estimates of stratified mean. |
boot.vars |
vector of length=nresamp containing estimate of variance of stratified mean from each bootstrap resample. |
accel |
acceleration constant needed for BCa confidence interval (see Smith 1997, for details). accel=NA when any of the sample sizes within strata are less than or equal to 1. |
call |
copy of the function call to document what methods were chosen. |
Kovar, J.G., Rao, J.N.K. and C.F.J. Wu. (1988). Bootstrap and other methods to measure errors in survey estimates. Can. J. Stat. 16 (supplement): 25–45.
Rao, J.N.K. and C.F.J. Wu. (1988). Resampling inference with complex survey data. J. Amer. Stat. Assoc. 83: 231–241.
Sitter, R.R. (1992). A resampling procedure for complex survey data. J. Amer. Stat. Assoc. 87: 755–765.
Smith, S.J. (1997). Bootstrap confidence limits for groundfish trawl survey estimates of mean abundance. Can. J. Fish. Aquat. Sci. 54: 616–630.
Scallop.numbers.2008.boot<-boot.strata(Scallop.numbers.2008, nresamp=1000, method="BWR")


Can We Repair the Brain in Parkinson's Disease?
by Roger BarkerWho's speaking at this Webinar?
Parkinson's Disease (PD) is a common neurodegenerative disorder of the brain that is characterised pathologically by the loss of nigral dopaminergic neurons. Whilst motor features of the disease respond well to current dopaminergic treatments, side effects such as dyskinesia and neuropsychiatric problems are common over time. These problems could, in theory, be avoided through targeted delivery of dopamine to the dorsal striatum as is possible with neuronal replacement approaches or growth factor (GDNF) delivery.
These strategies have been tried over the last 20-30 years and shown long-term clinical benefit in some patients. But efficacy is inconsistent, and these new therapies often have their own side effects.
In this webinar, the University of Cambridge’s Professor Roger Barker will discuss the history, current status and potential of growth factor- and cell-based therapies for PD.
Attend this webinar to hear:
- The latest advances in cell-based treatments for PD
- The potential of stem cell therapies for brain repair


Download and Get Started with Apicurio Registry
The easiest way to get started with Apicurio Registry is by using Docker to quickly start up an in-memory server. This server is non-persistent, so don't use this for production!
Prerequisites
- You have Docker installed.
Pull the Apicurio Registry image
Pull the apicurio-registry-mem Docker image from Docker Hub as follows:
docker pull apicurio/apicurio-registry-mem:{{ version }}
Run the Apicurio Registry image
Apicurio Registry supports different storage options, but the quickest way to get started is to use the non-persistent in-memory image with no external dependencies. Each option has its own Docker image. For more options, see the Apicurio Registry documentation.
Run the apicurio-registry-mem
Docker image as follows:
docker run -it -p 8080:8080 apicurio/apicurio-registry-mem:{{ version }}
Open the Apicurio Registry web console
Open the Apicurio Registry web console in your browser to add API and schema artifacts to Apicurio Registry.


ARMAGEDDON (1998)
Rating: 2.0 stars (out of 4.0) ******************************** Key to rating system: 2.0 stars - Debatable 2.5 stars - Some people may like it 3.0 stars - I liked it 3.5 stars - I am biased in favor of the movie 4.0 stars - I felt the movie's impact personally or it stood out *********************************
Directed by: Michael Bay
Screenplay: Jonathan Hensleigh and J.J. Abrams
Starring: Bruce Willis, Liv Tyler, Ben Affleck, Billy Bob Thornton, Peter Stormare, Steve Buscemi, Seiko Matsuda
Ingredients: Large asteroid, crew of misfit oil drillers who become astronauts, countdown timer with big green lights
Synopsis: In the beginning of ARMAGEDDON the voice of Charlton Heston explains that long ago in the Cretaceous Period, a 6-mile chunk of asteroid exterminated the world of the dinosaurs. When the rock struck, its instant vaporization threw so much dust and gas into the atmosphere that the sun was blocked long enough to kill off most plants and animals. In ARMAGEDDON a new rock - - this time 'the size of Texas' - - is poised to collide with the Earth. American heroes must save the world from certain disaster while the helpless international community follows the American heroes' exploits with awe and adoration.
Harry Stamper (Bruce Willis) and his hell-raising crew of eccentric oil drillers are considered the best drillers in the world. Dan Truman (Billy Bob Thornton) as head of NASA recruits Stamper and the oilers for a mission to save humanity from extinction. NASA divides the oilers into two mission teams: one led by Stamper and the other led by Stamper's daughter's boyfriend AJ (Ben Affleck), a rival who doesn't get along with Stamper. Each team gets a mobile drilling rig, a space shuttle, and a nuclear bomb. Using Earthbound shuttles as space rockets, each team must slingshot around the moon, reach the giant asteroid, and drill 800 feet into it the asteroid in order to plant and explode a nuclear device - - so that hopefully the asteroid will be split into pieces and avoid colliding with Earth. Only one nuclear device is needed for success but the two rival teams are sent simultaneously to preclude failure.
On the mission, Stamper and the boys face perils such as the asteroid's selective gravity. Two-ton space vehicles containing crew members are in danger of floating off into free space. At the same time, two-ton rocks might crush the men as the boulders fall with resounding thuds on the airless asteroid's surface. Also, somebody has thoughtlessly equipped the mobile drilling rigs with Gatling machine guns.
Liv Tyler plays Stamper's daughter (and AJ's girlfriend) Grace, a pretty face who waits helplessly on Earth. There are also bits involving stereotyped minorities such as the chatterbox African American who shrieks in surprise, the overweight Samoan hit by a meteor, and the Asian couple who applaud Stamper when their faces get splattered with black oil. Japanese superstar entertainer Seiko Matsuda gets all of two seconds as an oblivious tourist who declares that she wants to go shopping during a meteor shower.
Can Stamper and the oil drilling guys save humanity from certain destruction?
Opinion: Speaking of doom, a New Age friend told me her theory about why DEEP IMPACT and ARMAGEDDON came out this summer. She says a 16th century doom-and-gloom prophet named Nostradamus wrote that a terror would fall from the sky next year "in the seventh month of 1999." Which could mean anything from a slingshot crash of NASA's plutonium-laden Cassini Probe to a terrorist on a parachute, if you believe in that sort of thing. But in Hollywood, it amounts to little more than coffee shop inspiration for summer asteroid movies such as DEEP IMPACT and ARMAGEDDON.
Getting back to ARMAGEDDON, it seems like there's a scene missing. In the movie, as time runs short, the crew led by AJ has a perfect opportunity to wake up, unpack their gear, start digging, plant their nuclear bomb, and save the Earth (and its 6 billion people). Instead, they go driving around the asteroid with the mobile drilling rig - - inexplicably wasting precious time sightseeing when the Earth (including AJ's girlfriend) is in immediate and dire jeopardy. Is AJ's nuke completely forgotten as a viable option in his mission?
At its heart ARMAGEDDON is mostly a rehash of stuff we've already seen before in other uninspiring movies. But at least the mediocre doomsday script is spiced up with implausible events (such as Gatling guns) to increase the number of booms and bangs.
The cinematography in ARMAGEDDON is noteworthy in that for two hours and 30 minutes the audience is hit with a dizzying mishmash of photographic angles nearly every second. Meanwhile, in the background music, suspenseful drumming goes on and on and on. All this changing scenery and ominous drumming is illusory fluff - - an obvious attempt to make it seem like something interesting is always about to happen. But the continuous images hold interest for only about an hour, then diminish in impact as nothing substantial really happens. On the other hand, being distracted from ARMAGEDDON's implausible plot while waiting for the ending to roll around might not be a bad thing.
In short, ARMAGEDDON consists of a few booms, a few bangs, lots of distracting images, and a period of waiting around for a possibly happy ending.
Reviewed July 1, 1998
Trivia (Sci Fi fans only): After the movie I wondered whether in reality we Americans would be able to use nukes to fend off a space rock the size of Texas (i.e. over a quarter million square miles), and found out that astronomers have actually done simulations. In the computer simulations using the rough equivalent of an atom bomb planted in an asteroid only a mile in size, the asteroid cracked up, but the pieces stayed on course to wipe out the Earth.
One thing that's always bugged me about sci fi movies is the use of rockets, which somehow seem kind of low tech, since rockets waste vast amounts of fuel and then run out. But in order to travel such vast distances, interstellar travel would require establishing a propulsion force that, unlike rocketry, doesn't use up much fuel.
For example, when you drop a penny, it experiences a force which pulls it in the direction of earth. This is because the penny is within a gravitational field which propels the penny towards Earth, without having to burn fuel. If the gravitational field pointed sideways, the penny would fall sideways. If the field pointed up, the penny would fall up. The penny always falls in the direction of the field, without having to burn fuel. Field is the key.
Field effect propulsion is perfect for sci fi space travel. Basically, in order to create a propulsive force to drive a spaceship in any direction without the use of fuel, all you have to do is establish a ship with its own field. Point the field in the direction you want to go, and you continue to "fall" in that direction forever. Point it in a different direction, and there you go.
Has the principle of field effect propulsion been demonstrated before in real life? Certainly. It's known as the Biefeld Brown effect, a weak phenomenon first observed by physicist/astronomer Dr. Paul Alfred Biefeld as early as 1923. Biefeld created a strong electrostatic field in a simple capacitor (a capacitor is a sandwich made of two metal plates and gook in between) - - and the capacitor always levitated weakly towards the direction stipulated by its generated field. That was over seventy five years ago, in 1923, before big budgets and modern materials. Just imagine the kind of field effect propulsion we could achieve in a 1998 science fiction movie (or in real life using the movie's budget).
Most recently, in 1992 Dr Eugene Podkletnov of Tampere University, Finland published a paper describing research in which he using a special ceramic disk to produce a field in the direction opposing gravity. When non-metal and non-magnetic materials were placed within the field directly above the flat disk, he recorded up to a 2% cancellation of gravity. Presently Dr Ning Li of the University of Alabama, Huntsville, along with NASA's Marshall Manned Space Flight Center in Huntsville, Alabama, are working on a similar device. Go, NASA.
It seems to me, that if science fiction is supposed to portray imaginary high tech space propulsion, field effect is the way to go.
Copyright © 1998 by David Sunga This review and others like it can be found at THE CRITIC ZOO: http://www.criticzoo.com email<EMAIL_ADDRESS>The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


|
|||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object dxf2svg.util.Dxf2SvgConstants
This class represents all of the constants that are required for the Dxf2Svg conversion process. The class is not instantialable, all variables are called via calls to static values. This class may not be sub-classed.
| Field Summary | |
static int |
ANIMATE_FILL
This is a list of attributes that typically can be targets for animation. |
static int |
ANIMATE_FONT_SIZE
|
static int |
ANIMATE_STROKE
|
static int |
ANIMATE_STROKE_WIDTH
|
static int |
ANIMATE_VISIBILITY
|
static java.lang.String |
APPLICATION
The name of the application. |
static int |
BEST_GUESS
Default if no resolution specified. |
static int |
CSS_ONLY
Make a Cascading Style Sheet (CSS) called svg.css and exit application See command line arguments in Dxf2Svg.java documentation. |
static int |
CURRENT_RESOLUTION
Auto detect current display settings. |
static int |
CUSTOM_CSS
Make a custom page by reading in the styling information from an external file. |
static int |
CUSTOM_MASK
This mask will, when anded with MAKE_CSS, will mask out the lower placement bits. |
static int |
DECLARED_CSS
Declare styles at the beginning of the Svg (default). |
static int |
EIGHT_HUNDRED_X_SIX_HUNDRED
Typical setting and IETM default resolution specification |
static int |
ENGLISH
This is a set of languages currently understood by the conversion process. |
static int |
EXTERNAL_CSS
Place attributes in external style sheet |
static int |
EXTERNAL_SCRIPT
External placement |
static int |
FRENCH
|
static int |
HEAD
|
static int |
INCHES
Inch measure conversion (default). |
static int |
INDENT
Indent internal style sheet entries. |
static int |
INLINE_STYLES
Each element looks after its own attributes |
static int |
INTERNAL_AND_EXTERNAL_SCRIPT
You can also include both internal and external JavaScript in the same file. |
static int |
INTERNAL_SCRIPT
Internal placement i.e. with in the <script> tags in <head> |
static int |
MULTI_LINGUAL
|
static int |
NO_INDENT
Do not indent for external style sheet entries. |
static int |
NO_SCRIPT
Script placement settings |
static java.lang.String |
PART_NUMBER_ID
This value is the id name of any text if it is a part number. |
static int |
PLACEMENT_MASK
This mask will mask off any custom setting bit set in MAKE_CSS. |
static int |
POINTS
Points measure conversion (72 per inch). |
static int |
SIXFORTY_X_FOUREIGHTY
VGA type resolution |
static int |
SPAR_C130
Make an internal Spar style sheet that matches line weights and layer colours. |
static java.lang.String |
STYLE_SHEET_NAME
All external css are refered to by this name. |
static int |
TAIL
These control where any supplied partial list should appear. |
static int |
TENTWENTYFOUR_X_SEVENSIXTYEIGHT
1024 x 768 resolution typically large monitors. |
static int |
UNDEFINED
|
static java.lang.String |
VERSION
This value needs to be updated with each release of Dxf2Svg. |
static java.lang.String |
WIRE_RUN_ID_VALUE
This is the default id value for all groups of wires that are created dynamically during the conversion process. |
| Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
public static final int CSS_ONLY
public static final int INLINE_STYLES
public static final int DECLARED_CSS
public static final int EXTERNAL_CSS
public static final int CUSTOM_CSS
public static final int SPAR_C130
public static final int CUSTOM_MASK
public static final int PLACEMENT_MASK
public static final java.lang.String STYLE_SHEET_NAME
public static final int NO_INDENT
public static final int INDENT
public static final java.lang.String VERSION
public static final java.lang.String APPLICATION
public static final int UNDEFINED
public static final int CURRENT_RESOLUTION
public static final int SIXFORTY_X_FOUREIGHTY
public static final int EIGHT_HUNDRED_X_SIX_HUNDRED
public static final int TENTWENTYFOUR_X_SEVENSIXTYEIGHT
public static final int BEST_GUESS
public static final int INCHES
public static final int POINTS
public static final int NO_SCRIPT
public static final int INTERNAL_SCRIPT
public static final int EXTERNAL_SCRIPT
public static final int INTERNAL_AND_EXTERNAL_SCRIPT
public static final int TAIL
public static final int HEAD
public static final int ANIMATE_FILL
public static final int ANIMATE_STROKE
public static final int ANIMATE_STROKE_WIDTH
public static final int ANIMATE_VISIBILITY
public static final int ANIMATE_FONT_SIZE
public static final int ENGLISH
public static final int FRENCH
public static final int MULTI_LINGUAL
public static final java.lang.String WIRE_RUN_ID_VALUE
public static final java.lang.String PART_NUMBER_ID
|
|||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |