First: the deorbit phase starts.
The Dragon coasts away from the station to prevent damage to the station from its thrusters. Once it has reached a safe distance, it fires its Draco thrusters, each capable of 400 newtons of thrust,
in a sequence to ensure the Dragon enters the atmosphere at the correct angle to reenter safely with the least stress on the PICA heat shield. This deorbit burn takes seven minutes to execute.
Second: the reentry phase begins.
The Dragon's trunk is jettisoned; it's not designed to withstand the heat and forces of reentry and so burns up and is destroyed in the upper atmosphere. The Dragon then orients itself to point
its heatshield forwards, but at a slight angle; since the Dragon is loaded asymmetrically prior to its release from the ISS, the capsule can be rotated along its axis, acting like a weak wing, to adjust its trajectory and fine-tune
its landing position.
Third: the landing phase begins.
When the Dragon reaches an altitude of 13.7 km, two small parachutes called "drogue chutes" are deployed. These slow down the Dragon capsule and ensure full deployment of the three main parachutes.
After the three main parachutes deployed, they slow the Dragon to about 5 m/s at sea level for splashdown.


Home
Biography
Research
Projects
Professional Service
Publications
Teaching
Students
Photos
Links
My Academic Genealogy
Xinbing's Academic Genealogy @ MGP


Java Dev | Discord Bot Dev | Android Dev
From Discord bot development to writing an Android app for an electromiography watch, I do a lot of programming and algorithm design. Below are just some of the projects that I maintain.
A mutli-purpose, for-fun and for-utility Discord bot. Spectra has a complex tag system for a plethora of "custom command" capabilities. Additionally, it sports utilities like reminders, announcements, self-assignable roles, afk notifications, and a full moderation suite including logs.
WikiStraightforward moderation bot that requires no setup for basic commands. It also has automoderation capabilities that stop common spam techniques and keep the server safe.
InviteA bot designed to connect as many Discord servers as possible, bringing fun and entertainment. It features several fun "game" commands, a soundboard, and cross-server text and voice chat.
InviteA fast and efficient Discord music bot with a clean interface that is easy to host yourself. It supports many online media formats, as well as support for basic permissions.
SetupA basic Discord selfbot to enhance functionality of your account. Use this bot to quote other members' messages, perform Google searches, or create custom text "emoji" replacements.
SetupA 💯% emoji bot that's just for fun. It has a few interesting commands, like an emoji cleverbot and some simple games.
Invite


5. THE GOVERNMENT OF GOD.
The government of the Almighty, has always been very dissimilar to the
government of men; whether we refer to his religious government, or to the
government of nations. The government of God has always tended to promote
peace, unity, harmony, strength and happiness; while that of man has been
productive of confusion, disorder, weakness and misery. The greatest acts of
the mighty men have been to depopulate nations, and to overthrow kingdoms; and
whilst they have exalted themselves and become glorious, it has been at the
expense of the lives of the innocent-the blood of the oppressed-the moans of
the widow, and the tears of the orphan. Egypt, Babylon, Greece, Persia,
Carthage, Rome-each were raised to dignity amid the clash of arms, and the din
of war; and whilst their triumphant leaders led forth their victorious armies
to glory and victory, their ears were saluted with the groans of the dying,
and the misery and distress of the human family;--before them the earth was a
paradise, and behind them a desolate wilderness; their kingdoms were founded
in carnage and bloodshed, and sustained by oppression, tyranny, and despotism.
The designs of God, on the other hand, have been to promote the universal
good, of the universal world;-to establish peace and good will among men;-to
promote the principles of eternal truth;-to bring about a state of things that
shall unite man to his fellow man-cause the world to "..."-make the nations of the
earth dwell in peace; and to bring about the millenial [millennial] glory-when
"..."
The great and wise of ancient days have failed in all their attempts to
promote eternal power, peace, and happiness. Their nations have crumbled to
pieces; their thrones have been cast down in their turn; and their cities, and
their mightiest works of art, have been annihilated; or their dilapidated
towers, or time worn monuments have left us but feeble traits of their former
magnificence, and ancient grandeur. They proclaim as with a voice of thunder,
those imperishable truths-that man's strength is weakness, his wisdom is
folly, his glory is his shame.
Monarchical, aristocratic, and republican forms of government, of their
various kinds and grades, have in their turn been raised to dignity and
prostrated in the dust. The plans of the greatest politicians, the wisest
senators, and most profound statesmen have been exploded; and the proceedings
of the greatest chieftains, the bravest generals, and the wisest kings have
fallen to the ground. Nation has succeeded nation, and we have inherited
nothing but their folly. History records their puerile plans, their short
lived glory, their feeble intellect, and their ignoble deeds.
Have we increased in knowledge or intelligence? where is there a man that can
step forth and alter the destiny of nations, and promote the happiness of the
world? Or where is there a kingdom or nation, that can promote the universal
happiness of its own subjects, or even their general well being? Our nation,
which possesses greater resources than any other, is rent from center to
circumference, with party strife, political intrigue, and sectional interest;
our counsellors [counselors] are panic struck, our legislators are astonished,
and our senators are confounded; our merchants are paralized [paralyzed], our
tradesmen are disheartened, our mechanics out of employ, our farmers
distressed, and our poor crying for bread. Our banks are broken, our credit
ruined, and our states overwhelmed in debt;-yet we are, and have been in
peace.-What is the matter? Are we alone in this thing? Verily, no. With all
our evils we are better situated than any other nation. Let Egypt, Turkey,
Spain, France, Italy, Portugal, Germany, England, China, or any other nation
speak, and tell the tale of their trouble-their perplexity, and distress, and
we should find that their cup was full, and that they were preparing to drink
the dregs of sorrow. England, that boasts of her literature, her science,
commerce, &c., has her hands reeking with the blood of the innocent, abroad;
and she is saluted with the cries of the oppressed, at home.-Chartism,
O'Connelism, and Radicalism are gnawing her vitals at home; and Ireland,
Scotland, Canada, and the East, are threatening her destruction abroad. France
is rent to the core-intrigue, treachery, and treason lurk in the dark; and
murder, and assassination stalk forth at noon-day. Turkey, once the glory of
European nations, has been shorn of her strength-has dwindled into her dotage,
and has been obliged to ask her allies to propose to her tributary terms of
peace: and Russia, and Egypt are each of [t]hem opening their jaws to devour
her. Spain has been the theatre [theater] of bloodshed, of misery and woe, for
years past. Syria is now convulsed with war and bloodshed. The great and
powerful empire of China, which has for centuries resisted the attacks of
barbarians, has become tributary to a foreign foe; her batteries thrown down;
many of her cities destroyed, and her villages deserted. We might mention the
Eastern rajahs; the miseries and oppressions of the Irish; the convulsed state
of Central America; the situation of Texas and Mexico; the state of Greece,
Switzerland, and Poland-nay, the world itself presents one great theatre
[theater] of misery, woe, and "..." All, all
speak with a voice of thunder, that man is not able to govern himself-to
legislate for himself-to protect himself-to promote his own good, nor the good
of the world.
It has been the design of Jehovah, from the commencement of the world, and is
his purpose now, to regulate the affairs of the world in his own time; to
stand as head of the universe, and take the reigns of government into his own
hand. When that is done judgment will be administered in righteousness;
anarchy and confusion will be destroyed, and "..."
It is for want of this great governing principle that all this confusion has
existed; "..." this we have
fully shewn [shown].
If there was any thing great or good in the world it came from God. The
construction of the first vessel was given to Noah, by revelation. The design
of the ark was given by God, "..." The learning of
the Egyptians, and their knowledge of astronomy was no doubt taught them by
Abraham and Joseph, as their records testify, who received it from the Lord.
The art of working in brass, silver, gold, and precious stones, was taught by
revelation, in the wilderness. The architectural designs of the Temple at
Jerusalem, together with its ornament and beauty was given of God. Wisdom to
govern the house of Israel was given to Solomon, and to the judges of Israel;
and if he had always been their king, and they subject to his mandate, and
obedient of his laws, they would still have been a great and mighty people;
the rulers of the universe-and the wonder of the world. If Nebuchadnezzar, or
Darius, or Cyrus, or any other king possessed knowledge or power it was from
the same source, as the scriptures abundantly testify. If then, God puts up
one, and sets down another, at his pleasure-and made instruments of kings,
unknown to themselves, to fulfill his prophesies, how much more was he able,
if man would have been subject to his mandate, to regulate the affairs of this
world, and promote peace and happiness among the human family.
The Lord has at various times commenced this kind of government, and tendered
his services vo [to] the human family. He selected Enoch, whom he directed,
and gave his law unto, and to the people who were with him; and when the world
in general would not obey the commands of God, he translated Enoch and his
church, and the priesthood or government of heaven, was taken away.
Abraham was guided in all his family affairs by the Lord; was told where to
go, and when to stop; was conversed with by angels, and by the Lord; and
prospered exceedingly in all that he put his hand unto; it was because he and
his family obeyed the counsel of the Lord.-When Egypt was under the
superintendence of Joseph, it prospered, because he was taught of God; when
they oppressed the Israelites destruction came upon them. When the children of
Israel were chosen with Moses at their head, they were to be a peculiar
people, among whom God should place his name: their motto was "..." While in this state they might truly say, "..." Their government was a theocracy; they had God to make their
laws, and men chosen by him to administer them; he was their God, and they
were his people. Moses received the word of the Lord from God himself; he was
the mouth of God to Aaron, and Aaron taught the people in both civil and
ecclesiastical affairs; they were both one; there was no distinction; so will
it be when the purposes of God shall be accomplished; when "..." and "..." "..."
This is the only thing that can bring about the "..."- "..." Other attempts to promote universal peace and happiness in the human
family have proven abortive; every effort has failed; every plan and design
has fallen to the ground; it needs the wisdom of God, the intelligence of God,
and the power of God to accomplish this. The world has had a fair trial for
six thousand years; the Lord will try the seventh thousand himself; "..." iniquity will hide its hoary head, Satan will be bound, and
the works of darkness destroyed; righteousness will be put to the line, and
judgment to the plummet, and "..." To bring about this state of things there must of necessity be
great confusion among the nations of the earth; "..."-Am I asked what is the cause of the present distress? I would
answer: "..." The
earth is groaning under corruption, oppression, tyranny, and bloodshed; and
God is coming out of his hiding place, as he said that he would do, to vex the
nations of the earth. Daniel, in his vision, saw convulsion upon convulsion;
he "..." It is for us to be righteous
that we may be wise and understand, for "..." As a church, and a people it behoves [behooves]
us to be wise, and to seek to know the will of God, and then be willing to do
it; for "..." says
the scriptures. "..." says our Savior, "..." If Enoch, Abraham, Moses, the children of
Israel, and all God's people were saved by keeping the commandments of God,
we, if saved at all, shall be saved upon the same principle. As God governed
Abraham, Isaac and Jacob, as families, and the children of Israel as a nation,
so we, as a church, must be under his guidance if we are prospered, proserved
[preserved], and sustained. Our only confidence can be in God; our only wisdom
obtained from him; and he alone must be our protector and safeguard,
spiritually and temporally, or we fall.
We have been chastened by the hand of God heretofore for not obeying his
commands, although we never violated any human law, or transgressed any human
precept: yet we have treated lightly his commands, and departed from his
ordinances, and the Lord has chastened us sore, and we have felt his arm, and
kissed the rod: let us we [be?] wise in time to come, and ever remember that
"..." The
Lord has told us to build the temple, and the Nauvoo House, and that command
is as binding upon us as any other; and that man who engages not in these
things is as much a transgressor as though he broke any other command-he is
not a doer of God's will, nor a fulfiller of his laws.
In regard to the building up of Zion it has to be done by the counsel of
Jehovah; by the revelations of heaven, and we should feel to say "..." We would say to the saints that come
here, we have laid the foundation for the gathering of God's people to this
place, and expect that when the saints do come they will be under the counsel
of those that God has appointed. The Twelve are set apart to counsel the
saints pertaining to this matter: and we expect that those who come here will
send before them their wise men according to revelation; or if not
practicable, be subject to the counsel that God has given or they cannot
receive an inheritance among the saints, or be considered as God's people; and
they will be dealt with as transgressors of the laws of God; we are trying
here to gird up our loins, and purge from our midst the workers of iniquity;
and we hope that when our brethren arrive from abroad, they will assist us to
roll forth this good work, and to accomplish this great design; that "..." that as
God's people, under his direction, and obedient to his law, we may grow up in
righteousness, and truth; that when his purposes shall be accomplished, we may
receive an inheritance among those that are sanctified.-ED.


Loading may take a moment or two if you are the first person to ever request a visualization of a particular dataset.
Datasets


--- ---
[Internal] Gets the decoded representation of a value if the value is defined, otherwise, returns the default value, which may be the result of an injectable function.
Checks if two param value objects are equal (for a set of Param objects)
The list of Param objects to check
The first set of param values
The second set of param values
true if the param values in values1 and values2 are equal
Generated using TypeDoc
Cache the default value if it is a static value


This document attempts to gather requirements for Linked Data in JSON (JSON-LD) in order to create an objective measure with which to evaluate the JSON-LD Specification.
This document is an experimental work in progress.
Use URIs as names for things
Use HTTP URIs so that people can look up those names.
When someone looks up a URI, provide useful information.
Include links to other URIs so that they can discover more things.
An object structure is represented as a pair of curly brackets surrounding zero or more name/value pairs (or members). A name is a string. A single colon comes after each name, separating the name from the value. A single comma separates a value from a following name. The names within an object SHOULD be unique.
The following are taken to be assertions about the meaning of
This section is intended to abstractly describe the concept of
Definitions for Linked Data are often stricter than is required for many applications, where it is not feasible
or desirable to identify every object with an IRI which resolves to a
The following are taken to be requirements and principles for creating


Check the events you want to sign up for and then click submit at the bottom!
User: {{ name }}
Currently Signed up for:
{{activities }}


Description
Limits (velocity) Rule defines a logical rule that allows or prohibits the transaction from being executed when the conditions are met.
Important:
Be very careful with the rules you define as certain combinations may prohibit the execution of all transactions!
How to add new Velocity Rule:
- Click on „Plus“ icon on top right corner
- A modal window will open to add new velocity rule
How to define new Velocity Rule:
When you see Modal window for add/edit Velocity risk rule, you can define new rule by choosing the required Time group, time count, attributes, operator, limit and result.
The definition logic is as follows:
For X parts ("Times" field) of Time ("Time Group") count of the transactions per "Attribute 2" and per "Attribute 3" (optional) must meet the limited("limit" field) condition selected in the "Operator", the "Result" will be executed.
Example
"I want for 12 hours count of the transactions per Customer Id and per Billing Address Country cannot be more than 5."
How to define Example Velocity Rule:
- Select "Hour" option in "Time group" dropbox
- Fill "12" in "Times" input field
- Select "Count" option in "Count group" dropbox
- "transactions.count" option will be selected in "Attribute 1" dropbox
- Select "customer.id" option in "Attribute 2" dropbox
After you select the option in "Attribute 2", this option will disappear from "Attribute 3" dropbox.
- Select "billing.address.country" option in "Attribute 3" dropbox
- Select ">=" option in "Operator" dropbox
- Fill "5" in "Limit" input field
- Select "Red" option in "Result" dropbox
- Click on "Save" button on the bottom of the modal window
Important:
After you save the items in modal window, you must click the "Save" button at the bottom of the page. Otherwise, your changes will not be saved.
How to edit existing Velocity Rule:
- Click on „Edit“ icon on the row with selected item
- A modal window will open to edit selected rule
- At this point "Rule type" dropbox can not be changed
- Change the options that you want
Important:
After you save the items in modal window, you must click the "Save" button at the bottom of the page. Otherwise, your changes will not be saved.
How to delete existing Velocity Rule:
- Click on „Delete“ icon on the row with selected item
- Risk Rule will be deleted from the rules list
Important:
After you delete the items from list, you must click the "Save" button at the bottom of the page. Otherwise, your changes will not be saved.
Description of the options:
"Time group"
"14 days" --> 14 days
"Hour" --> Hour
"Week" --> Week
"Day" --> Day
If "14 days" option is selected, "Times" field will will not be visible.
"Times" - positive number, used in combination with ""Time group""
Represents the count of the "Time group" option.
In the above mentioned example, this means that the 12 hours before the request hit the rule engine and the rule check was initiated.
Max allowed period is 14 days.
Examples:
Period is 14 days:
Period is 3 days:
Option 1
Option 2
"Count group"
"Count" --> Count
"Attribute 1"
"transactions.count" --> Count of the transactions (DEBIT and CAPTURE transactions) in status SUCCESS.
"Attribute 2" and "Attribute 3"
"billing.address.city" --> Billing Address City
"billing.address.country" --> Billing Address Country
"billing.address.firstName" --> Billing Address FirstName
"billing.address.houseNumber" --> Billing Address HouseNumber
"billing.address.id" --> Billing Address Id
"billing.address.lastName" --> Billing Address LastName
"billing.address.state" --> Billing Address State
"billing.address.street" --> Billing Address Street
"billing.address.zip" --> Billing Address Zip Code
"customer.email" --> Customer Email
"customer.id" --> Customer ID
"order.ipAddress" --> IP Address of Order
"persona.id" --> Persona ID
"persona.phone" --> Persona Phone
"persona.mobile" --> Persona Mobile
"paymentInstrument.iban" --> IBAN of Payment Instrument
"paymentInstrument.id" --> ID of Payment Instrument
"shipping.address.city" --> Shipping Address City
"shipping.address.country" --> Shipping Address Country
"shipping.address.firstName" --> Shipping Address FirstName
"shipping.address.houseNumber" --> Shipping Address HouseNumber
"shipping.address.id" --> Shipping Address Id
"shipping.address.lastName" --> Shipping Address LastName
"shipping.address.state" --> Shipping Address State
"shipping.address.street" --> Shipping Address Street
"shipping.address.zip" --> Shipping Address Zip Code
"Operator":
">=" - Greater than or equal
"<=" - Less than or equal
"<" - Less than
"==" - Equality
"!=" - Inequality
">" - Greater than
"Limit" - positive number represents limit of the transactions count.
Result:
"Red" - Transaction will be declined
"Green" - Transaction will pass.


There are two ways to invoke a script xxx.
Execution as a function is the default. If these is no function with name xxx then execution as a script occurs.
The scriptOnly option will cause execution as a script even when there is a function with name xxx. The funcOnly option will cause execution as a function. Execution will fail if there is no xxx function.
When running @x:Act under a server as a CGI or servlet, the funcOnly option is normally specified for increased security.


Deploying QATrack+¶
A note about security¶
Before deploying QATrack+ in the clinic, please take the time to read about security & QATrack+ on the security notes page.
Deployment Options¶
Django applications like QATrack+ are typically deployed on a Linux server using PostgreSQL or MySQL as the database. If your department has Linux support/experience this is the recommended way to deploy QATrack+.
That said, at TOHCC we are primarily a Microsoft based clinic and therefore have deployed QATrack+ using Windows Server 2008R2 and SQLServer2008 and have been generally happy with this setup.
Deployment on a Mac server is also relatively straightforward.
Below you can find a number of guides (works in progress) for a variety of platforms.
- Linux/Apache/PostgreSQL(or MySQL) (recommended)
- Windows Server/IIS/CherryPy/SQL Server (TOHCC’s current setup serving 2 campuses & 100+ users)
- Windows Server/IIS/SQL Server (Full MS implementation with SSO/Integrated Authentication)
- Windows Desktop/Apache/PostgreSQL (Run QATrack+ from a Windows desktop)
- Windows Desktop/Apache/MySQL (Run QATrack+ from a Windows desktop)
- OS X Server(ML)/Apache/PostgreSQL
Python Environment (virtualenv)¶
The deployment guides above make use of Python’s http://www.virtualenv.org/ to isolate the QATrack+ Python environment from the system Python version. While not strictly required it is considered a “best practice” in the Python community and allows multiple applications requiring different library versions to run side by side on the same server. You will want to always remember to activate the QATrack+ virtualenv (explained in the deployment guides) before installing, upgrading or managing your QATrack+ installation.
Hosting at a non root url¶
(new in v0.2.6)
If you want to host your QATrack+ installation at a non root directory of your server than you need to set FORCE_SCRIPT_NAME in your local_settings.py file to the location of your QATrack+ installation. If for example your QATrack+ installation is configured to live at http://yourservername/qatrack then you need to have a line like:
#!python
FORCE_SCRIPT_NAME = "/qatrack"
Note that the leading slash is required but no trailing slash should be included.
You may also need to modify your server configuration slightly (url rewrites etc.) to account for the difference.
Configuring QATrack+ for your needs¶
There are lots of settings in QATrack+ that can be tweaked to meet the specific needs of your clinic. Some of the most common ones are described below (in progress!). Generally the qatrack/settings.py file should be left alone and any customizations for your clinic should be placed in a qatrack/local_settings.py file.
Migrating from the standalone version¶
If you would like to migrate your trial test configuration from the standalone demo version of QATrack+, you can find instructions here: moving test configuration from standalone QATrack+ to production.


Rust has a variety of familiar types:
bool
for representing true/falseu8
u32
u64
u128
for representing positive whole numbersi8
i32
i64
i128
for representing positive/negative whole numbersusize
isize
for representing indexes
and sizes of things in memoryf32
f64
(value, value, ...)
for passing fixed sequences of values on the stackstr
(string slice) - text with a length known at runtimeText might be more complex than you are used to in other languages, since Rust is a system programming language, it's cares about memory issues you might not be used to. We will be going into this in detail later.
Numeric types can be explicitly specified by appending the type to the end of the number (e.g. 13u32
, 2u8
)


UI5con 2018 - News from Control Development
Example 1: Hello World
Example 2: a simple SearchField control
Example 3: a container control, which internally creates a Button for each Item given into the aggregation
Example 4: ManagedObjectModel, basic usage
Example 5: ManagedObjectModel, with aggregation binding
Example 6: Aggregation Forwarding
Example 7: Aggregation Forwarding (with getter)
Example 8: Aggregation Forwarding in XMLComposite
Example 9: Event Handler Parameters, basic
Example 10: Event Handler Parameters in a Table
Example 11: a Paginator XMLComposite using event handler parameters


The ‘ggnomics’ package aims to provide tools for integrating Bioconductor S4 classes inheriting from the S4Vectors::Vector
-class with the popular plotting library ‘ggplot2’. It does so by formulating scale and coordinate systems that are Vector aware, such that Vector classes can be shaperoned through the internals of ‘ggplot2’. Furthermore, it provides smooth transitions from Bioconductor infrastructure such as the S4Vectors::DataFrame
-class to ‘ggplot2’.
Suppose we would want to make of some genomic ranges and colour them by strand. We’ll use the DataFrame
class to organise our data and feed this directly into ‘ggplot2’ with their familiar syntax. Since ‘ggnomics’ operates mostly on the scale and coordinate level, all ggplot2’s layer functionality (geoms and stats), such as geom_tile()
, are available to use with Vector
classes.
library(ggnomics) library(GenomicRanges) df <- DataFrame( x = GRanges(c("chr1:100-200:+", "chr1:150-250:-", "chr2:200-300:*")), y = 1:3 ) g <- ggplot(df, aes(x, y, fill = strand(x))) + geom_tile(width = 0, height = 0.8) g
A few things are happening here under the hood. Because we provide a Vector
class as ggplot()
’s data
argument, an S4-compatible cartesian coordinate system is added to the plot automatically.
g <- ggplot_build(g) print(class(g$plot$coordinates)) #> [1] "CoordS4" "CoordCartesian" "Coord" "ggproto" #> [5] "gg"
Since we’ve provided a GRanges
class as the x-position, the decision is made to use scale_x_genomic()
as the default scale for that aesthetic. This scale type comes with an guide that is aware of the different sequences the ranges are located to and displays the sequences next to one another.
print(g$layout$panel_scales_x) #> [[1]] #> <ScaleGenomic> #> Range: chr1:100-250 -- chr2:200-300 #> Limits: chr1:100-250 -- chr2:200-300
An effort is made to retain the S4 classes of data columns as long is feasible. This means that, unless a layer needs to transform values, the S4 classes are stable across the majority of plot building.
knitr::kable(g$data[[1]][,1:5])
| fill | x | y | PANEL | group |
|---|---|---|---|---|
| #F8766D | chr1:100-200:+ | 1 | 1 | -1 |
| #00BA38 | chr1:150-250:- | 2 | 1 | -1 |
| #619CFF | chr2:200-300:* | 3 | 1 | -1 |


Apache Spark is not a one-trick pony : Going beyond in-memory processing
At DataMantra, whenever we suggest Apache Spark to our customers, they respond “it just in-memory thing right?”. There is a misconception about Spark, that it’s just about in-memory computation. Even creators of spark acknowledge it. In one of their recent blog post they say
This has always been one of the most common misconceptions about Spark, especially for people new to the community. Spark is well known for its in-memory performance, but from its inception Spark was designed to be a general execution engine that works both in-memory and on-disk. Almost all Spark operators perform external operations when data does not fit in memory. More generally, Spark’s operators are a strict superset of MapReduce.
Though in-memory is an important feature, it’s not only thing that makes Spark special. So in this blog post, I am going to discuss about few of the other important features that make Spark stand out.
(Note : This post originally published in DataMantra blog)
1. One platform for all Big data loads
Big data analysis has different data loads. Different use cases need different kind of analysis. Most of the big data analysis need both batch processing and real time capabilities. Not only that, they have structured, unstructured, graph analysis and other advanced processing needs too. So a big data processing framework has to cater to all of this needs.
Hadoop M/R, current leading big data processing framework, is only optimized for batch processing. So in hadoop, you need to go to other frameworks like Storm for real time and Apache Giraph for graph processing capabilities. Having multiple different frameworks is a pain for development and maintenance. Though introduction of YARN has solved few issues, YARN is too low level to be used in applications.
In case of Spark, it was designed to cater to multiple data loads from day one. Batch processing and real time capabilities are built in to the core of Spark. Not only that many advanced graph and machine learning libraries are built in so that it can cater to wide variety of data analysis needs.
2. One abstraction to rule them all
In Spark, all APIs and libraries talk same abstraction called RDD. RDD stands for Resilient Distributed Dataset. It’s just a big collection of immutable data which is sitting in some storage. But what’s the advantage if all libraries talk in RDD?
The advantage is that you can mix and match different kind of processing in same application. You can take a stream of data from spark real time system and can run sql queries using SparkSQL. Then take the output of SparkSQL and feed it to machine learning algorithm using MLLib. All of this is done without ever have to convert or store intermediate result. This is very powerful to build complex data flow systems.
3. Runs everywhere
Spark runs everywhere. It is designed to run on different platforms and different distributed systems. You can run Spark on Hadoop 1.0, Hadoop 2.0, Apache Mesos or stand alone spark cluster. This flexibility of deployment is very attractive to customers, as it allows them to harness power of spark using their existing infrastructure investments.
4. Small and Simple
The original release of Spark contained only 1600 lines of Scala code. It was designed to be extremely modular. So today you can add or remove capabilities to your spark application simply by changing your build file. Also having a small code base makes extend the framework easy.
Spark API thrives for simplicity. If you ever seen a spark word count vs Map/Reduce word count you will understand how simple API is. There is a lot of thought gone in to the API to make it more approachable and consistent. Contrast to that Java API of Map/Reduce is a mess. There is too much verbosity.
5. Prospering Ecosystem
In Hadoop, ecosystem was an after thought. The ecosystem projects like Hive and Pig dint have access to Map/Reduce abstractions. All they can do was generate Map/Reduce programs on the fly and run it on cluster. This severely effected their performance.
Where as in Spark, there was a plan for ecosystem from day one. The ecosystem libraries Pregel and MLLib were developed side by side with core spark. So in Spark, all libraries have access to same level of abstraction as main API. This makes them first class citizen on the platform.
6. Multi language API
Spark officially supports Scala, Java and Python API’s. This makes it more attractable to many developers compared to java only M/R. Though M/R supported C++ and other API’s they are not up to date like Java API. So Spark is great option for developers with different backgrounds.
Now you know spark is not just about in-memory processing. Let’s break this myth by sharing this article with everyone.


We adopt a very broad definition of “sensor,” taking it to mean any signal that can be unobtrusively captured from the smartphone device. A full list of the sensors we currently support are available on the menu bar. These include:
Pull Sensors: All sensors that the Android OS does not capture data from until requested to do so by an application. They currently include the accelerometer, location (both coarse, from the nearest cell-tower, and fine, from the GPS), and microphone amplitude levels. They further support querying for Wi-Fi, Bluetooth, and active application scan results, as well as querying the SMS and call logs.
Push Sensors. The Android OS publishes data about particular events that applications can receive: this set of sensors receives this information on behalf of an application, and includes: the battery (energy levels), connection state (whether the phone is connected to a Wi-Fi or mobile network and if it is in roaming mode), the proximity sensor (near/far events), screen on/off events, phone calls starting/ending, and SMS sent/received events.
All JSON formatted data includes the following keys:
{
"sensorType": "string",
"senseStartTime":"HH:mm:ss:SSS dd MM yyyy Z z",
"senseStartTimeMillis":0,
"userid":"string",
"deviceid":"string",
}
Where:
sensorType
: is the string identifier for the sensor
senseStartTime
: is the string-formatted time that sensing from this sensor began (measured on the device).
senseStartTimeMillis
: is the time in milliseconds that sensing from this sensor began (measured on the device).
userid
: is the string user-id that you have given the data manager.
deviceid
: is string device-id that you have given the data manager.
All JSON formatted data includes the following keys:
{
"postSenseSleepMillis":0,
"isAdaptivelySensed":false,
}
Where:
postSenseSleepMillis
: if the data was sensed by subscribing to a sensor, this is the time in milliseconds that the sensing sleeps after collecting this sample.
isAdaptivelySensed
: a boolean value denoting whether the adaptive sensing algorithm was used.


{% extends "base.html" %}
{% load custom_filters %}
{% block titlebar %}Server Error{% endblock %}
{% block title %}Server Error{% endblock %}
{% block content %}
We're sorry, but the page you requested could not be loaded
at this time due to an internal server error. Please try again later. You may
wish to contact the system administrator if the problem persists.
{% endblock %}


Chapter 15 Develop an operator
https://github.com/tercen/rtercen
Install rtercen package
devtools::install_github("tercen/rtercen", ref = "2.10")
Optional configuration save the following in
.Rprofile
file
options("tercen.username"="myusername")
options("tercen.password"="mypassword")
- Running Tercen R operator locally
Access tercen.com and upload some data, create a workflow and a data step. Configure your data step, and retreive the workflowId and stepId from the url. Example : https://tercen.com/core/index.html#ds/{workflowId}/{stepId}
library(rtercen)
library(plyr)
# your Tercen workflowId and stepId
workflowId="ec25c9aa533ad9f655c2e5e7c4004460"
stepId="47e61df0-4c30-11e6-ad8a-795d6db3b40c"
# create a Tercen client object using your username and password
# client = TercenClient$new(myusername, mypassword)
# create a Tercen client object using your username and password store in .Rprofile file
# serviceUri points to local vagrant for testing
# client = TercenClient$new(serviceUri="https://vagrant.tercen.com/service")
# serviceUri points to tercen.com
client = TercenClient$new()
# get the cube query defined by your workflow
query = client$getCubeQuery(workflowId, stepId)
# inspect query as json
# query$toJson()
# execute the query and get the data
cube = query$execute()
# execute some computation
computed.df <- ddply(cube$sourceTable$as.data.frame(), c(".ids"), summarize, mean = mean(.values))
# send the result to Tercen
query$setResult(computed.df)


 The basic point-to-point communication operations are send and receive. Their use is illustrated in the example below.
In this example, process zero ( myrank = 0) sends a message to process one using the send operation MPI_SEND. The operation specifies a send buffer in the sender memory from which the message data is taken. In the example above, the send buffer consists of the storage containing the variable message in the memory of process zero. The location, size and type of the send buffer are specified by the first three parameters of the send operation. The message sent will contain the 13 characters of this variable. In addition, the send operation associates an envelope with the message. This envelope specifies the message destination and contains distinguishing information that can be used by the receive operation to select a particular message. The last three parameters of the send operation, along with the rank of the sender, specify the envelope for the message sent. Process one ( myrank = 1) receives this message with the receive operation MPI_RECV. The message to be received is selected according to the value of its envelope, and the message data is stored into the receive buffer. In the example above, the receive buffer consists of the storage containing the string message in the memory of process one. The first three parameters of the receive operation specify the location, size and type of the receive buffer. The next three parameters are used for selecting the incoming message. The last parameter is used to return information on the message just received.
The next sections describe the blocking send and receive operations. We discuss send, receive, blocking communication semantics, type matching requirements, type conversion in heterogeneous environments, and more general communication modes. Nonblocking communication is addressed next, followed by channel-like constructs and send-receive operations, Nonblocking communication is addressed next, followed by channel-like constructs and send-receive operations, ending with a description of the ``dummy'' process, MPI_PROC_NULL.


{% from "macros.html" import experiment_list %} {% extends "outline.html" %} {% block title %}Experiment List{% endblock %} {% block body %}
Click on an experiment to see its dashboard.
Download backup of experiments selected above.
Retire the experiments selected above.
Note: Retiring an experiment only affects the dashboard; the experiment is still active.
Select a database backup file for restoring.
Warning: Restoring is an "insert only" action meaning that no action is taken for those experiments that already exist in the system.
Uploading...


{% extends 'dashboard/base.html' %} {% load static %} {% block extrahead %} {% endblock %} {% block body %}
Welcome, {{ user.first_name }}!
Ready to shop, and send payments easily and more securely? Link a card and ensure it has been enabled for international transactions
Buyer Protection - Shop with peace of mind. PayU Buyer Protection safeguards your eligible purchases.
More secure - Pay at million of online stores globally without exposing your financial information.
Reward points - Earn reward points when you pay with card*.
* Reward schemes may differ when using PayU. Please refer to your card issusers.


During the export, a graph of the entities is created in the database. Three tables are created for this purpose:
-
JAILER_ENTITYGRAPH the graph
-
JAILER_ENTITY entity in the graph
-
JAILER_DEPENDENCY dependencies between
entities
For each entity in the graph the primary-key and the
type (as table name) is stored. The birthday of
an entity is the day of the insertion into the
graph.
The graph will be
constructed day after day (step-by-step) starting at day 1:
The progression set of a day is the set of
all tables from which entities are inserted at that day.
The progession set of day 1 is the subject set. The
process stops if the progression set of a day is
empty.
Algorithm
1. insert subject into
graph
2.
progression-set := { subject
}
3. while progression-set !=
{}
1. progression-of-today := {}
2.
for all (restricted) associations A between tables
T1 and
T2, ifT1 or T2 in
progression-set:
1. if T2
(T1) depends on
T1
(T2):
add dependencies from T2(T 1)-entities born yesterday
to the entities they are associated with according to
A
2. insert all entities of T1 (T2), which are associated
according to A
with an entity of T2 (T1) born yesterday
3. if at least one entity of T1 (T2) is inserted then progression-of-today := progression-of-today
+ { T1 (T2) }
3. progression-set :=
progression-of-today
Example
Subject = {
A1,
A2
}
|
day |
progression set |
entities |
dependencies |
| 1 |
A |
A1, A2 |
|
| 2 |
B, C |
+ B1, B2, C3, C4 |
A2->C3, A2->C4 |
| 3 |
B, C |
+ C1, C2, B3, B5 |
+ B1->C1, B1->C2 |
| 4 |
B |
+ B6 |
no change |
| 5 |
|
no change |
no change |
Line 1, insert subject
INSERT INTO JAILER_ENTITY(PK, birthday, type)
SELECT
<subject>.PK, 1, '<subject>'
FROM
<subject>
WHERE (<condition> AND 103
NOT EXISTS (SELECT * FROM JAILER_ENTITY WHERE
<subject>.PK=Entity.PK)
[fetch first <limit> rows only]
Line 3.2.1, adding dependencies
INSERT INTO
JAILER_DEPENDENCY(from_PK, to_PK)
SELECT
<T2>.PK, <T1>.PK
FROM JAILER_ENTITY
E1, JAILER_ENTITY E2,
<T2> JOIN <T1> ON
<join-condition>
WHERE
E1.type='<T1>' AND
E2.type='<T2>'
AND E1.PK=<T1>.PK
AND E2.PK=<T2>.PK
Line 3.2.2, adding referenced entities
INSERT INTO
JAILER_ENTITY(PK, birthday, type)
SELECT
DISTINCT <T2>.PK, <today>,
'<T2>'
FROM
<T1> JOIN <T2> ON
<join-condition>,
Entity
WHERE
<T1>.PK=JAILER_ENTITY.PK AND JAILER_ENTITY.birthday =
<yesterday>
AND NOT EXISTS
(SELECT * FROM JAILER_ENTITY WHERE
<T2>.PK=JAILER_ENTITY.PK)
|


Ironclad
Ironclad is a movie starring Paul Giamatti, Jason Flemyng, and Brian Cox. In thirteenth century England, a Knights Templar and few of the Barons men, fight to defend Rochester Castle against the tyrannical King John.


Badge for the promotion “Seid nett zueinander” of the German newspaper “Hamburger Abendblatt”.
Use the jsDelivr CDN to easily include the badge:
<script src="https://cdn.jsdelivr.net/npm/seid-nett-zueinander" defer></script>


{% extends "base.html" %} {% load i18n %} {% comment %} This template can be used whenever the user doesn't have permission to access a page. In other words, this is a custom 403 template - except that it won't be shown in every instance of a 403. This template will only be shown when our code uses it (our custom view decorator or something). Customizing Django's general 403 handler requires some digging with middleware, like this: http://mitchfournier.com/2010/07/12/show-a-custom-403-forbidden-error-page-in-django/ For the record, having a custom 403 template would be much easier in a Django version later than 1.3: https://code.djangoproject.com/changeset/16606 https://docs.djangoproject.com/en/dev/topics/http/views/#the-403-http-forbidden-view {% endcomment %} {% block content_title %}
{{ message|linebreaks }}
{% else %}{% trans "Sorry, you don't have permission to access this page."%}
{% endif %} {% endblock %}


--- layout: post title: Lifehacker - Firebug 1.5 Updates with New Features, Still a Web Developer's Dream [Downloads] date: '2010-01-21T01:38:00.001Z' author: solrevdev tags: modified_time: '2010-01-21T01:38:45.899Z' blogger_id: tag:blogger.com,1999:blog-31446756.post-8167047372550041441 blogger_orig_url: http://www.solrevdev.com/2010/01/lifehacker-firebug-15-updates-with-new_21.html ---


VideoPlayer Example using an static picture
The VideoPlayer object is capable of creating an online video using an static picture
Contents
Create the VideoPlayer object
In order to create a VideoPlayer object using an static picture we have to define the VideoPlayer object using the optional parameter 'UseStaticPicture', where we have define a projective transform, which will be applied to the image for each frame. See next source code:
pictureName = 'peppers.png'; % Static picture transX = 10; % X Translation transY = 5; % Y Translation rotX = 0; % Rotation in the X axis (in degrees) rotY = 0; % Rotation in the Y axis (in degrees) rotZ = 10; % Rotation in the Z axis (in degrees) scale = 1.08; % Scale factor imageSize = [320, 240]; % Cutted region of the static picture numberFrames = 10; % Number of frames of the synthetic video. vp = VideoPlayer(pictureName, ... 'UseStaticPicture', [transX transY rotX rotY rotZ scale], ... 'ValidRectangle', imageSize, 'MaxFrames', numberFrames);
Reproduce the synthetic video
In order to reproduce the synthetic video you have to create the loop like in the example_VideoPlayer.m.
while(true) plot(vp); disp( mat2str(vp.Tgp) ); drawnow; if (~vp.nextFrame) break; end end
[1 0 0;0 1 0;0 0 1] [1.06359237325318 0.187540031880285 0;-0.187540031880285 1.06359237325318 0;10 5 1] [1.09605747288468 0.39893229517506 0;-0.39893229517506 1.09605747288468 0;20 10 1] [1.0909425934521 0.629856 0;-0.629856 1.0909425934521 0;30 15 1] [1.04219500773272 0.874505446603326 0;-0.874505446603326 1.04219500773272 0;40 20 1] [0.944465882331592 1.12557060835134 0;-1.12557060835134 0.944465882331592 0;50 25 1] [0.793437161472 1.37427347628274 0;-1.37427347628274 0.793437161472 0;60 30 1] [0.586162422042981 1.61046801869592 0;-1.61046801869592 0.586162422042981 0;70 35 1] [0.321410658004117 1.82281042137011 0;-1.82281042137011 0.321410658004117 0;80 40 1] [1.2240373090321e-16 1.99900462710443 0;-1.99900462710443 1.2240373090321e-16 0;90 45 1]
Release the VideoPlayer object
Finally you have to release the object.
clear vp;


Press release
Parents to get greater say in the school admissions process
New rules will also stop complaints against faith schools from secularist campaign groups.
The government will sweep away admissions bureaucracy and ensure parents have more of a say in the school admissions process so they can be confident the system works for them, Education Secretary Nicky Morgan announced today (25 January 2016).
The
school admissions code
is now simpler and easier for parents to understand than ever before. Under new plans the government wants to ensure every child has access to the good school place their parents rightly expect and is taking further action to simplify the school admission system and ensure it works for parents by:
-
unclogging the admissions system by stopping objections to a school or local authority’s admissions arrangements from outside the local area - this means only local parents will have a say on admissions and helps local authorities to ensure they are fair
-
stopping vexatious complaints against faith schools from secularist campaign groups
-
giving parents and communities a greater voice in local admissions by requiring admissions authorities to consult on their admission arrangements every 4 years rather than the current 7
Education Secretary Nicky Morgan said:
As part of our mission to deliver educational excellence everywhere, we want every child to have the opportunity to go to a good local school by making it easier for parents to have a say in their local school’s admission process.
So that parents can be confident that the school admission process is working for them - we are ensuring only local parents and councils can object to admission arrangements, which will also put a stop to vexatious complaints against faith schools by secularist campaign groups. At the same time we will be giving parents a greater voice by requiring admission authorities to consult every 4 years.
Despite rising pupil numbers, 95% of parents received an offer at 1 of their top 3 preferred schools last year. Local authorities are responsible for making sure that there are enough places in local areas, and the government will consider intervening where local authorities are not playing their part in ensuring these places are available.
Nicky Morgan added:
Our manifesto commitment to create 500 new free schools will significantly broaden the choice available to local parents, and improve the supply of good quality places.
Notes to editors
-
All admission authorities must, when drawing up their admission arrangements, ensure that the practices and criteria used to decide the allocation of places are fair, clear and objective and that they comply with the school admissions code. The school admissions code is now simpler and easier for parents to understand than ever before.
-
In some cases, parents from outside the local area are making objections purely in the hope that their child might get a place at their preferred school, not because of a problem with the admission arrangements themselves. Additionally secular campaign groups have targeted faith school admissions as part of a particular agenda. We plan to put a stop to this by restricting who can object to school admissions arrangements to local parents and the local authority.
-
Under the current rules admission authorities must set (‘determine’) admission arrangements annually. Where changes are proposed to admission arrangements, the admission authority must first publicly consult on those arrangements. If no changes are made to admission arrangements, they must be consulted on at least once every 7 years, to ensure admissions arrangements continue to meet local needs. We intend to change the rules to require admissions authorities to consult on their admission arrangements every 4 years rather than the current 7.
-
We will be conducting a full public consultation on the proposed changes in due course.
-
Local authorities are responsible for ensuring there are sufficient school places in their area. They can use basic need funding to provide places in new schools or expansions of existing schools, and can work with any school in their local area, including academies and free schools.
-
We doubled basic need funding to local authorities for new school places to £5 billion between 2011 and 2015, which helped to create half a million new school places in maintained schools and academies since May 2010. This helped to correct the decline of over 200,000 places between 2004 and 2010. We have also committed to spend £7 billion on new school places in this parliament.
-
In 2015, 95.9% received an offer at one of their top 3 preferred primary schools. (87.8% were offered their top preference primary school).


The Diceware method can be used to generate strong passphrases. In short, you roll a 6-faced dice 5 times in a row, the number obtained is matched against a dictionary of easily remembered words. By combining together 7 words thus generated, you obtain a password that is relatively easy to remember, but would take several millions years (on average) for a powerful computer to guess.
| Version: | 0.4 |
| Depends: | R (≥ 3.1.3) |
| Imports: | random (≥ 0.2.4) |
| Suggests: | testthat |
| Published: | 2015-05-21 |
| Author: | Francois Michonneau [aut, cre], Arnold G. Reinhold [cph] |
| Maintainer: | Francois Michonneau <francois.michonneau at gmail.com> |
| BugReports: | https://github.com/fmichonneau/riceware/issues |
| License: | MIT + file LICENSE |
| URL: | https://github.com/fmichonneau/riceware |
| NeedsCompilation: | no |
| CRAN checks: | riceware results |
| Reference manual: | riceware.pdf |
| Package source: | riceware_0.4.tar.gz |
| Windows binaries: | r-devel: riceware_0.4.zip, r-release: riceware_0.4.zip, r-oldrel: riceware_0.4.zip |
| OS X binaries: | r-release: riceware_0.4.tgz, r-oldrel: riceware_0.4.tgz |
| Old sources: | riceware archive |
Please use the canonical form https://CRAN.R-project.org/package=riceware to link to this page.


Welcome to IsaNet ML Lib’s documentation!¶
Indices and tables¶
Tutorial¶
An example with the low level api (keras-like):
# ...
from isanet.model import Mlp
from isanet.optimizer import SGD, EarlyStopping
from isanet.datasets.monk import load_monk
import numpy as np
X_train, Y_train = load_monk("1", "train")
X_test, Y_test = load_monk("1", "test")
#create the model
model = Mlp()
# Specify the range for the weights and lambda for regularization
# Of course can be different for each layer
kernel_initializer = 0.003
kernel_regularizer = 0.001
# Add many layers with different number of units
model.add(4, input= 17, kernel_initializer, kernel_regularizer)
model.add(1, kernel_initializer, kernel_regularizer)
es = EarlyStopping(0.00009, 20) # eps_GL and s_UP
#fix which optimizer you want to use in the learning phase
model.setOptimizer(
SGD(lr = 0.83, # learning rate
momentum = 0.9, # alpha for the momentum
nesterov = True, # Specify if you want to use Nesterov
sigma = None # sigma for the Acc. Nesterov
))
#start the learning phase
model.fit(X_train,
Y_train,
epochs=600,
#batch_size=31,
validation_data = [X_test, Y_test],
es = es,
verbose=0)
# after trained the model the prediction operation can be
# perform with the predict method
outputNet = model.predict(X_test)
An example with the low level api (keras-like) with NCG or LBFGS optimizer:
from isanet.model import Mlp
from isanet.optimizer import NCG, LBFGS
from isanet.optimizer.utils import l_norm
from isanet.datasets.monk import load_monk
from isanet.utils.model_utils import printMSE, printAcc, plotHistory
import isanet.metrics as metrics
import numpy as np
X_train, Y_train = load_monk("1", "train")
X_test, Y_test = load_monk("1", "test")
#create the model
model = Mlp()
# Specify the range for the weights and lambda for regularization
# Of course can be different for each layer
kernel_initializer = 0.003
kernel_regularizer = 0.001
# Add many layers with different number of units
model.add(4, input= 17, kernel_initializer, kernel_regularizer)
model.add(1, kernel_initializer, kernel_regularizer)
es = EarlyStopping(0.00009, 20) # eps_GL and s_UP
optimizer = NCG(beta_method="hs+", c1=1e-4, c2=0.1, restart=None, ln_maxiter = 100, norm_g_eps = 1e-9, l_eps = 1e-9)
# or you can choose the LBFGS optimizer
#optimizer = LBFGS(m = 30, c1=1e-4, c2=0.9, ln_maxiter = 100, norm_g_eps = 1e-9, l_eps = 1e-9)
#start the learning phase
# no batch with NCG or LBFGS optimizer
model.fit(X_train,
Y_train,
epochs=600,
validation_data = [X_test, Y_test],
es = es,
verbose=0)
# after trained the model the prediction operation can be
# perform with the predict method
outputNet = model.predict(X_test)
An example with the high level API (sklearn like):
from isanet.neural_network import MLPClassifier
from isanet.datasets.monk import load_monk
X_train, Y_train = load_monk("1", "train")
X_test, Y_test = load_monk("1", "test")
mlp_c = MLPClassifier(X_train.shape[1], # input dim
Y_train.shape[1], # out dim
n_layer_units=[4], # topology
activation="sigmoid", # activation hidden layer
kernel_regularizer=0.001, # l2 regularization term
max_epoch=600, # Max number of Epoch
learning_rate=0.83, # learning rate
momentum=0.9, # momentum term
nesterov=True, # if Nesterov
sigma=None, # sigma Acc. Nesterov
early_stop=None, # define the early stop
verbose=0) # verbosity
mlp_c.fit(X_train, Y_train, X_test, Y_test)
outputNet = mlp_c.predict(X_test)
An example with the Model Selection Module:
from isanet.model_selection import Kfold, GridSearchCV
from isanet.neural_network import MLPRegressor
dataset = np.genfromtxt('CUP/ML-CUP19-TR_tr_vl.csv',delimiter=',')
X_train, Y_train = dataset[:,:-2], dataset[:,-2:]
grid = {"n_layer_units": [[38], [20, 32]], #[20, 32] means two hidden layer
"learning_rate": [0.014, 0.017],
"max_epoch": [13000, 1000],
"momentum": [0.8, 0.6],
"nesterov": [True, False],
"sigma": [None, 0.8, 0.6, 2, 4]
"kernel_regularizer": [0.0001],
"activation": ["sigmoid"],
"early_stop": [EarlyStopping(0.00009, 20), EarlyStopping(0.09, 200)]}
mlp_r = MLPRegressor(X_train.shape[1], Y_train.shape[1])
kf = Kfold(n_splits=5, shuffle=True, random_state=1)
gs = GridSearchCV(estimator=mlp_r, param_grid = grid, cv = kf, verbose=2)
result = gs.fit(X, Y) # dict with keys as column headers and values as columns


Chapter 8 Plotting spatial data
Together with timelines, maps belong to the most powerful graphs, perhaps because we can immediately relate where we are, or have been, on the space of the plot. Two recent books on visualisation (Healy 2018, Wilke (2019)) contain chapters on visualising geospatial data or maps. Here, we will not try to preach the do’s and don’ts of maps, but rather point out a number of possibilities how to do things, point out challenges along the way and ways to mitigate them.
8.1 Every plot is a projection
The world is round, but plotting devices are flat. As mentioned in section 7.3.1, any time we visualise, in any way, the world on a flat device, we project: we convert angular, geodetic coordinates into Cartesian coordinates. This includes the cases where we think we “do nothing” (figure 8.1, left), or where show the world “as it is”, e.g. as seen from space (figure 8.1, right).
The left plot of figure 8.1 was obtained by
library(sf)
library(rnaturalearth)
w <- ne_countries(scale = "medium", returnclass = "sf")
plot(st_geometry(w))
and we see that this is the default projection for data with geodetic coordinates, as indicated by
st_crs(w)
#> Coordinate Reference System:
#> User input: EPSG:4326
#> wkt:
#> GEOGCRS["WGS 84",
#> DATUM["World Geodetic System 1984",
#> ELLIPSOID["WGS 84",6378137,298.257223563,
#> LENGTHUNIT["metre",1]]],
#> PRIMEM["Greenwich",0,
#> ANGLEUNIT["degree",0.0174532925199433]],
#> CS[ellipsoidal,2],
#> AXIS["geodetic latitude (Lat)",north,
#> ORDER[1],
#> ANGLEUNIT["degree",0.0174532925199433]],
#> AXIS["geodetic longitude (Lon)",east,
#> ORDER[2],
#> ANGLEUNIT["degree",0.0174532925199433]],
#> USAGE[
#> SCOPE["unknown"],
#> AREA["World"],
#> BBOX[-90,-180,90,180]],
#> ID["EPSG",4326]]
The projection taken here is the equirectangular (or equidistant cylindrical) projection, which maps longitude and latitude linear to the x and y axis, keeping an aspect ratio of 1. Were we to do this for smaller areas not on the equator, it makes sense to choose a plot ratio such that one distance unit E-W equals one distance unit N-S on the center of the plotted area.
We can also carry out this projection before plotting. Say we want to do this for Germany, then after loading the (rough) country outline, we use st_transform
to project:
DE = st_geometry(ne_countries(country = "germany", returnclass = "sf"))
DE.eqc = st_transform(DE, "+proj=eqc +lat_ts=51.14 +lon_0=90w")
st_transform
takes an sf
or sfc
object, and as second argument the projection. This can either be a number of a known EPSG projection, e.g. listed at http://spatialreference.org/ref/epsg/ , or a string describing the projection (+proj=...
) with further parameters. The parameter here is lat_ts
, the latitude of true scale (i.e., one length unit N-S equals one length unit E-W), which was here chosen as the middle of the bounding box latitudes
mean(st_bbox(DE)[c("ymin", "ymax")])
#> [1] 51.1
When we now plot both maps (figure 8.2), they look the same up to their values along the axes: degrees for geodetic (left), and metres for Cartesian coordinates.
par(mfrow = c(1, 2))
plot(DE, axes = TRUE)
plot(DE.eqc, axes = TRUE)
8.1.1 What is a good projection for my data?
There is unfortunately no silver bullet here. Projections that maintain all distances do not exist; only globes do. The most used projections try to preserve
- areas (equal area),
- directions (conformal, e.g. Mercator),
- some properties of distances (e.g. equirectangular preserves distances along meridians, azimuthal equidistant preserves distances to a central point)
or some compromise of these. Parameters of projections decide what is shown in the center of a map and what on the fringes, which areas are up and which are down, and which areas are most enlarged. All these choices are in the end political decisions.
It is often entertaining and at times educational to play around with the different projections and understand their consequences. When the primary purpose of the map however is not to entertain or educate projection varieties, it may be preferrable to choose a well-known or less surprising projection, and move the discussion which projection should be preferred to a decision process on its own.
8.1.2 Does projection always work?
No. Look for instance at the figure 8.1, right. Countries like the USA are half out-of-sight. Where is the California coast line drawn?
The PROJ string used here was "+proj=ortho +lat_0=30 +lon_0=-10"
and we can easily check what happens to a polygon that crosses the visible area by setting both parameters to 0:
sq = rbind(c(-89,0), c(-89,1), c(-91,1), c(-91,0), c(-89,0))
pol = st_sfc(st_polygon(list(sq)), crs = 4326)
(pol.o = st_transform(pol, "+proj=ortho +lat_0=0 +lon_0=0"))[[1]]
#> POLYGON ((-6377166 0, -6376194 111314, -6377166 0))
st_is_valid(pol.o, NA_on_exception=FALSE)
#> Error in CPL_geos_is_valid(st_geometry(x), as.logical(NA_on_exception)): Evaluation error: IllegalArgumentException: Invalid number of points in LinearRing found 3 - must be 0 or >= 4.
where we see that the polygon is not nicely cut along the visibility line, but that the invisible points are simply dropped. This leads in this case to an invalid geometry, and may in the case of 8.1 lead to straight lines that do not follow the map border circle.
How was figure 8.1 created? By using a rather ugly script that used a projected half-sphere circle to first cooky-cut the part of the countries that would remain visible on this projection. The script is available, and so is its output.
8.2 Plotting points, lines, polygons, grid cells
Since maps are just a special form of plots of statistical data, the usual rules hold. Frequently occuring challenges include:
- polygons may be very small, and vanish when plotted
- depending on the data, polygons for different features may well overlap, and be visible only partially; using transparent fill colors may help indentify them
- when points are plotted with symbols, they may easily overlap and be hidden; density maps (chapter 13) may be more helpful
- lines may be hard to read when coloured and may overlap regardless line width
When plotting polygons filled with colors, one has the choice to plot polygon boundaries, or to suppress these. If polygon boundaries draw too much attention, an alternative is to colour them in a grey tone, or another color that doesn’t interfere with the fill colors. When suppressing boundaries entirely, polygons with (nearly) identical colors will melt together. If the property indicating the fill color is constant over the region, such as land cover type, this is OK. If the property is an aggregation, the region over which it was aggregated gets lost. Especially for extensive variables, e.g. the amount of people living in a polygon, this strongly misleads. But even with polygon boundaries, using filled polygons for such variables may not be a good idea.
The use of continuous color scales for continuously varying variables may look attractive, but is often more fancy than useful:
- it impracticle to match a color on the map with a legend value
- colors ramps often stretch non-linearly over the value range
Only for cases where the identification of values is less important than the continuity of the map, such as the coloring of a high resolution digital terrain model, it does serve its goal.
8.3 Class intervals
When plotting continuous geometry attributes using a limited set of colors (or symbols), classes need to be made from the data. The R package classInt
(R. Bivand 2019a) provides a number of methods to do so. Using it is quite simple:
library(classInt)
# set.seed(1) needed ?
r = rnorm(100)
(cI <- classIntervals(r))
#> style: quantile
#> one of 1.49e+10 possible partitions of this variable into 8 classes
#> [-2.61,-1.26) [-1.26,-0.356) [-0.356,-0.131) [-0.131,0.091) [0.091,0.433)
#> 13 12 13 12 12
#> [0.433,0.623) [0.623,1.11) [1.11,2.76]
#> 13 12 13
cI$brks
#> [1] -2.612 -1.257 -0.356 -0.131 0.091 0.433 0.623 1.113 2.755
and it takes argument n
for the number of intervals, and a style
that can be one of “fixed”, “sd”, “equal”, “pretty”, “quantile”, “kmeans”, “hclust”, “bclust”, “fisher” or “jenks”. Style “pretty” may not obey n
; if n
is missing, ‘nclass.Sturges’ is used; two other methods are available for choosing n
automatically. If the number of observations is greater than 3000, a 10% sample is used to create the breaks for “fisher” and “jenks”.
8.4 Poles and datelines
Given the linestring
(ls = st_sfc(st_linestring(rbind(c(-179.5, 52), c(179.5, 52))), crs = 4326))
#> Geometry set for 1 feature
#> geometry type: LINESTRING
#> dimension: XY
#> bbox: xmin: -180 ymin: 52 xmax: 180 ymax: 52
#> geographic CRS: WGS 84
#> LINESTRING (-180 52, 180 52)
How long a distance does it span? Let’s see:
st_length(ls)
#> 68677 [m]
which seems sensible. But does ls
actually intersect with the dateline?
dateline = st_sfc(st_linestring(rbind(c(180, 51), c(180, 53))), crs = 4326)
st_intersects(ls, dateline)
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> Sparse geometry binary predicate list of length 1, where the predicate was `intersects'
#> 1: (empty)
… it seems not? How can this be? The warning said it all: if ls
is not in spherical coordinates, it means it follows the 52th parallel, crossing (0, 52)
half way. This is as if we drew a straight line between the two points on the left figure of 8.1, almost across the complete 52N parallel.
Where do these inconsistencies come from? The software sf
is built upon (see the C/C++ libraries box in figure 1.4) is a GIS software stack that originally targeted flat, 2D geometries. The simple feature standard assumes straight lines between points, but great circle segments are not straight.
The functions that deal with spherical geometry, such as st_length
, use PostGIS extensions in liblwgeom that were added later on to PostGIS, without rewriting the entire geometry core for geodetic coordinates. More recent systems, including Google’s S21, BigQuery GIS2 and Ubers H33 were written from scratch with global data in mind, and work exclusively with geodetic coordinates.
8.4.1 st_wrap_dateline
The st_wrap_dateline
function can be convenient,
(ls.w = st_wrap_dateline(ls))[[1]]
#> MULTILINESTRING ((-180 52, -180 52), (180 52, 180 52))
as it cuts any geometry crossing the dateline into MULTI-geometries of which the sub-geometries touch on, but no longer cross the dateline. This is in particular convenient for plotting geodetic coordinates using naive approaches such as that of figure 8.1 left, where they would have crossed the entire plotting area. Note that by cutting the line at (180,52)
, st_wrap_dateline
does not follow a great circle; for this, it should be preceded by st_segmentize
, as e.g. in
(ls.w2 = st_wrap_dateline(st_segmentize(ls, units::set_units(30, km))))[[1]]
#> MULTILINESTRING ((-180 52, -180 52, -180 52, -180 52), (180 52, 180 52, 180 52))
Also note that bounding boxes like
st_bbox(ls.w)
#> xmin ymin xmax ymax
#> -180 52 180 52
simply take the coordinate ranges, and are pretty much meaningless as descriptors of the extent of a geometry for geometries that cross the dateline.
Similar notions hold for the poles; a polygon enclosing the North pole
pole = st_sfc(st_polygon(list(rbind(c(0,80), c(120,80), c(240,80), c(0,80)))), crs = 4326)
does not include the pole
st_intersects(pole, st_sfc(st_point(c(0,90)), crs = 4326))
#> although coordinates are longitude/latitude, st_intersects assumes that they are planar
#> Sparse geometry binary predicate list of length 1, where the predicate was `intersects'
#> 1: (empty)
(Cartesian interpretation) but has a positive area
st_area(pole)
#> 1.63e+12 [m^2]
indicating again a geodetic interpretation.
8.5 Graticules and other navigation aids
Graticules are lines on a map that follow constant latitude or longitude values. On figure 8.1 left they are drawn in grey. Graticules are often drawn in maps to give reference where something is. In our first map in figure 1.1 we can read that the area plotted is near 35\(^o\) North and 80\(^o\) West. Had we plotted the lines in the projected coordinate system, they would have been straight and their actual numbers would not have been very informative, apart from giving an interpretation of size or distances when the unit is known, and familiar to the map reader. Graticules, by that, also shed light on which projection was used: equirectangular or Mercator projections will have straight vertical and horizontal lines, conic projections have straight but diverging meridians, equal area may have curved meridians
The real navigation aid on figure 8.1 and most other maps are geographical features like the state outline, country outlines, coast lines, rivers, roads, railways and so on. If these are added sparsely and sufficiently, graticules can as well be omitted. In such cases, maps look good without axes, tics, and labels, leaving up a lot of plotting space to be filled with actual map data.
References
Bivand, Roger. 2019a. ClassInt: Choose Univariate Class Intervals. https://github.com/r-spatial/classInt/.
Healy, Kieran. 2018. Data Visualization, a Practical Introduction. Princeton University Press. http://socviz.co/index.html.
Wilke, Claus O. 2019. Fundamentals of Data Visualization. O’Reilly Media, Inc. https://serialmentor.com/dataviz/.


com.johnsnowlabs.nlp.util
This represents an external source which contains information into how an external resource shall be read by Spark-NLP's Resource Helper
Helper one-place for IO management.
Helper one-place for IO management. Streams, source and external input should be handled from here
Helper one-place for IO management. Streams, source and external input should be handled from here


Olson Kundig Imagines a Center for Human Composting in Seattle
- Written by Christele Harrouk
- about 21 hours
The international design firm Olson Kundig has designed a new sustainable option for after-death care. In fact, the architects created the world’s first facility for converting human remains into soil, a flagship building for Recompose in Seattle.
Following the legalization of the disposal of human remains through composting in Washington State, Recompose|SEATTLE, set to open in the spring of 2021, will be the first project to offer this burial alternative to people. Led by design principal Alan Maskin, and designed in collaboration with Katrina Spade, founder, and CEO of the Recompose public benefit corporation, the innovative facility introduces the original service of “natural organic reduction”.
We asked ourselves how we could use nature – which has perfected the life/death cycle – as a model for human death care, […] We saw an opportunity for this profound moment to both give back to the earth and reconnect us with these natural cycles. -- Katrina Spade, founder and CEO of the Recompose public benefit corporation.
The process initiated can convert human remains into soil in about 30 days, helping nourish new life after death. An alternative to traditional techniques, this method uses one-eighth of the energy of cremation, allowing, therefore, to save over one metric ton of carbon dioxide per person. Working in close collaboration, the Recompose team designed the first prototype of the Recompose vessel, in 2016, while fulfilling a residency at the Olson Kundig’s offices.
Located in Seattle’s SODO neighborhood, the Recompose|SEATTLE proposal puts in place an 18,500-square-foot facility that helps induce the process. With a modular system at the core of the conception, the project that comprises of 75 vessels, is “centered around individual natural organic reduction vessels that transform human remains into clean, usable soil”.
Stacked and organized in a way to generate a central gathering space, the vessels are not the only entity of the composition. In fact, Alan Maskin, owner and design principal at Olson Kundig, states that “this facility hosts the Recompose vessels, but it is also an important space for ritual and public gathering. The project will ultimately foster a more direct, participatory experience and dialogue around death and the celebration of life". The Recompose|SEATTLE space includes also spaces for the storage and preparation of bodies, administrative back-of-house areas, and an interpretive public lobby which describes the Recompose process. Finally, transparency between the exterior and interior spaces emphasizes furthermore the collapse of boundaries between the human experience and natural processes.
As a studio, Olson Kundig has always thrived on close collaboration with some of the world’s most imaginative and innovative problem-solvers, like Katrina and her team. We’re honored to be involved with this project, and excited for the first Recompose facility in the world to open its doors in Seattle. -- Alan Maskin, owner and design principal at Olson Kundig.
- Olson Kundig Design Team: Alan Maskin, Design Principal; Blair Payson, AIA, LEED® AP, Project Manager; Nick Ladd, LEED® AP, Project Architect
- Recompose Team: Katrina Spade, Founder and CEO; Lynne Carpenter-Boggs, PhD, Research Advisor; Tanya Marsh, Legal Strategy; Alan Maskin, Architecture and Design; Angela Hennessy, Ritual and Cultural Practices; Caitlin Doughty, Alternative Funeral Practices; Leslie Christian, CFA, Organizational Strategy; Jim Boldt, Public Affairs; Alua Arthur, End of Life Planning; Erik Bondo, Project Management
- Consultant Team: Veksel Consulting & Construction, Owner’s Representative


This talk will offer some anecdotes and opinions on three topics:
- Data Analysis/Data Science at Bell Labs
- R and Data Science
- Data Science for the Future
Date: 10/30/2018
Additional ReadingsThere is no additional readings for this lecture.
John Chambers is a professor in the department of Statistics at Stanford and one the leaders of Stanford Data Science Inititive. He was a member of Bell Labs research from 1966 until his retirement in 2005. In 1997, he became the first statistician to be named a Bell Labs Fellow, cited for ‘‘pioneering contributions to the field of statistical computing’’. His research has touched on nearly all aspects of computing with data but he is best known for the design (and continuing re-design) of the S language and its successor, R. Since 2008, he has been Consulting Professor, Department of Statistics, Stanford University.
In May of 1999, the Association for Computing Machinery presented him its Software System Award for the design of the S system. The ACM citation stated that “S has forever altered the way people analyze, visualize, and manipulate data”. This is the only time the award has been made for statistical software (previous citations included Unix, TeX, and the World-Wide Web). The money from the award was donated to the American Statistical Association to establish an annual student prize for software.
The University of Waterloo awarded John Chambers an honorary Doctor of Mathematics degree in 2004. The citation for the award began: ‘‘John M. Chambers is a name synonymous, worldwide, with statistical computing; his leadership and impact are unequalled.’’
Since retiring from Bell Labs, Dr. Chambers has visited and taught at several universities, including University of Auckland, UCLA, and Stanford.
He is the author or co-author of eight books; his most recent book is ‘‘Software for Data Analysis: Programming with R’’ (Springer, 2008). His first book, ‘‘Computational Methods for Data Analysis’’ (Wiley, 1977), was the first general treatment of statistical computing. He was co-author of ‘‘Graphical Methods for Data Analysis’’, the first exposition of data visualization. Other books introduced versions of the S language and related topics, including ‘‘Statistical Models in S’’.
Dr. Chambers’s professional activities have included being president of the International Association for Statistical Computing and various offices in the ISI, ASA and AAAS. He is a fellow of the ASA, the IMS, and the AAAS, and an elected member of the International Statistical Institute. He is a member of the board of the R Foundation.
At Bell Labs, he served as head of the advanced software department (1981-1983) and the statistics and data analysis research department (1983-1989), before returning to full-time research in 1989. He continues active research on computing with data, with many plans for the future of computing with data.
Dr. Chambers obtained his Ph. D. in statistics in 1966 from Harvard University, after receiving a B.Sc. degree in 1963 from the University of Toronto.


Data Driven Detroit has requested our team to conduct an initial analysis and research on the usability of the City of Detroit’s Public Service Map tool. This tool serves residents in providing various information regarding recycling and waste management. Additionally, the interface where this tool resides contains other helpful resources and information that the Public Works Department offers its Detroit residents.
I served in a team of 2 other student designers to evaluate the then-current state of the Public Service Map tool, and made design suggestions to enhance its usability.
To gauge the then-current status of the Public Service Map tool in terms of general usability, assess issues, and propose design recommendations. Our team undertook a few initial UX research methods that deemed to fit the amount of time allotted.
We began the analysis by creating an in-depth analysis of the tool through heuristic evaluation (as per Nielsen's Heuristics). The following are the two lowest-evaluated heuristics.
The system failed to support users in taking back their mistakes and recover from errors. The tool often left users to deal with their own mistakes instead of helping them undo their actions.
Generally, the tool lacked flexibility and required users to click many times to navigate through. For example, the map had an auto-focus feature whereby the map would focus into the exact location when clicked on. The users had no way of zooming back out.
The team designed three user scenarios with varying difficulties to test whether the web interface can support users in accomplishing information-search tasks. Three Detroit residents were recruited to participate in the tests. Each were compensated with $20 gift cards thereafter completion.
The map tool did not support conventional online map interactions, such as dragging and/or pinch-zooming, which confused users on how to navigate through the map tool.
The map tool failed to offer users a simple and easy way to take back on their actions, do not leave users stranded after making a mistake!
The website in which the map tool existed contained a collection of helpful links on the right-hand side in a form of stacked-vertical navigation. This tool confused users of its disorganized hierarchy and relevance.
Our team found that the color-coding of the map failed to pass the color blindness test, where the color coding may confuse and mislead color blind users. The image on the left-side is the original interface, and the right represents a view from a color blind user.
After conductign the heuristics analysis on the tool, the team created a list of major functionalities of the tool in its entirety. One of which was the use of the right-hand side navigation for various information retrieval. The following is the list of our major findings gathered from the card sorting exercises.
Card sorters had no idea what most of the links meant. If the only method of communicating with users is through text, it best speak the users' language for starters.
Think about why users would use this feeature, then focus on the actions and primary topics users would relate to this feature. This tool was made to make garbage disposal information easily accessible, so that information should become the focus.
The right-hand navigation menu nested various information within each other, confusing users as to why that was. When nesting relevant information together, users must find them relevant!
After compiling the insights gathered, the team drafted a final summary of findings and design recommendations supporting the goal of the project. The clients and stakeholders of this system were professionals who aren't familiar with technical UX-terminology, which was closely considered whilst documenting our findings.
Users tend to assume grouped items to be related
Users were confused as to who the links were meant for, construction workers, business owners, or the residents? When naming helpful links, keep a primary action in mind and who might be visiting the link.
Many users are familiar with the interaction functionalities of Google Maps, whether it be on the Web or on Mobile devices. The interactions supported by Google Maps API are easy to adopt for a wide-range of audiences!
Using colors consistently and appropriately will make it easier for users to navigate through the tool. Always keep in mind that users may be color blind and will need additional methods of distinguishing between elements! Patterns and numbers are easy alternatives.
What's a design project without a final mockup? All hands were on deck!
Whilst interacting with residents, our team learned that not everyone interacting with digital devices and interfaces are visually capable. When engaging with civic-technology that supports a wider age-range of users, designers must always keep accessibility in mind and craft an inclusive and accessible interface.
The card sorting exercise for this project was the most intensive card sorting exercise I've ever practiced. Through it all, I learned that users tend to have a single primary action in mind while sifting through navigation text. They imagine one final action they wish to achieve and find relevant text on the way.
I felt a real need for UX professionals to really dive deep into the public works and civic technologies. The need for usability improvements and tests exist! Residents are crying out for better digital experiences. Once deployed into the workforce, I promised myself to also dedicate my time and skills to create lasting usability impact for civic-technologies.


Please contact Scot P, Amy B or Ellen M with any questions, suggestions or complaints*.
*Remember, complaints that qualify as whining will be penalized 1,000,000 points for your team.


| Invoice number: |
{{invoice.pk}} |
| Invoice status: |
{%if invoice.ispaid %}
{%if invoice.refund %}
{%if invoice.refund.completed%}Refunded{%elif invoice.refund.issued%}Refund submitted to payment provider and will complete soon.{%else%}Pending refund{%endif%}
{%else%}
Paid
{%endif%}
{%else%}
Awaiting payment
{%endif%}
|
| Invoice title: |
{{invoice.title}} |
| Total amount: |
{{invoice.total_amount|format_currency}} |
| Due date: |
{{invoice.duedate|datetimeformat("%Y-%m-%d")}} |
{%if invoice.isexpired and invoice.canceltime%}
| Auto cancel: |
If not paid before {{invoice.canceltime|datetimeformat("%Y-%m-%d %H:%M")}}, this invoice will automatically be cancelled. |
{%endif%}
| Invoice: |
View PDF |
{%if invoice.ispaid%}
| Receipt: |
View receipt |
{%else%}
| Pay invoice: |
{%if invoice.extradescription%}
{{invoice.extradescription}}
{%endif%}
This invoice can be paid using one of the methods listed below. Note
that only payments made using these methods are accepted.
{%for method in invoice.allowedmethodwrappers%}
{{method.name}}
{%if method.ok%}
{{method.description}}
{%if method.available%}
Proceed to pay with {{method.name}}
{%else%}
This method is currently not available. {{method.unavailable_reason}}
{%endif%}
{%else%}
An error occurred when initializing this payment method. Unfortunately, it is not currently available. Please contact the {{org_name}} treasurer to report this.
{%endif%}
{%else%}
There are no payment methods allowed for this invoice. Please contact
{{org_name}} <{{treasurer_email}}> reporting
your invoice number to get one or more methods assigned to it.
{%endfor%}
|
{%endif%}
{%if invoice.refund and invoice.refund.refund_pdf%}
| Refund: |
View refund note |
{%endif%}
{%if not fromsecret%}
| External link: |
Using this link you can give somebody else access to
view or pay this invoice, without being logged in to your account. |
{%endif%}


The DataHub Development Operations Process¶
This page details the practices used by the Data 8 tech team in order to maintain and upgrade their JupyterHub, called the DataHub. The DataHub is a JupyterHub deployment using Kubernetes that provides the environment in which all Data 8 students work.
The DataHub deployment repository¶
The current deployment configuration for the DataHub is hosted as a GitHub repository. Any changes made to this repository should immediately be reflected in the live DataHub.
There are two branches for the DataHub repository - one for staging
, and
another for prod
(short for “production”). Each branch controls a
different JupyterHub deployment.
staging
lives atstaging.datahub.berkeley.edu
. It is used by the Data 8 development team to test modifications before they’re deployed to the production environment. Students do not use this JupyterHub.prod
lives atdatahub.berkeley.edu
. It is the JupyterHub that students use, and as such has higher standards for stability.
The DataHub uses Travis CI (short for
“continuous integration”) to automatically run tests and code every
time a change is made to the DataHub master repository. This is
also used to automatically deploy these changes to the staging
and prod
DataHub deployments when new changes are made.
Modifying the DataHub¶
The following steps are followed by the Data 8 tech team any time a modification must be made to the DataHub.
Create a fork of the DataHub repository and work from this fork.
Pull the latest changes from the DataHub repository into your fork.
Create a new branch for your changes.
Make the modifications necessary to this branch (this is usually something like modifying the DataHub helm chart or editing the DataHub Docker Image).
Commit your changes and create a pull request. The pull request should be against
staging
, not againstprod
.The pull request will trigger a Travis CI process, and potentially a rebuild of docker images depending on what modifications have been proposed.
Once the CI process is complete and if there are no problems, the committer requests that someone review the PR before merging.
If all looks well, merge the PR into
staging
. This will trigger another Travis step which automatically upgrades the DataHub deployment atstaging.datahub.berkeley.edu
.After this upgrade finishes (per the logs in Travis CI), the team tests the changes by performing actions on
staging.datahub.berkeley.edu
(e.g., trying to import a package that was added)If
staging
tests fail, or if there are any problems during the live user testing, never updateprod
. Revert the change via the GitHub UI and debug before submitting another PR tostaging
.If the change passes tests and looks successful on
staging
, create a new PR to mergestaging
intoprod
. This will trigger a similar Travis process to deploy toprod
atdatahub.berkeley.edu
. Test your change on production for good measure.Celebrate!


Error with request, please try again.
Open Panel to Remove Groups
HSA contracts on November 9th in FY 2016
Columns...
Visualize Location
Clear
Total:
$0
×
© 2015 GSA.gov


Posts
- » More Airbnb Articles
- » How To Find Newsletter Subscriber Counts (if they use MailChimp)
- » Hacker Newsletter Book List - Volume One
- » Hello Jekyll
- » Exporting Data From MongoHQ
- » Killing Windows Services that hang on "STOPPING"
- » Why SharePoint Share?
- » VMware Logo - 12 Signs You Are Too Big
- » Links - "Some Favorite Sites, Vol 1"
- » Incorrect syntax near 'WSSList' in dbSPSReporting.sql
- » The exception unknown software exception (a.k.a. SysFader)
- » Using CRM with a SQL named instance
Projects
- Hacker Newsletter A weekly newsletter of the best articles on startups, technology, programming, and more, curated by hand from the popular Hacker News site.
- SharePoint Share A simple community driven SharePoint links site.


College is too much for me. I can't keep up with my classes. I just barely make it on time. I don't finish copying all the notes. I'm missing some assignments in most of my classes. And now I have finals coming up soon. I study and study, but it just doesn't help. I've asked my friends for help, but they're all busy.
On top of that, I'm working late everyday just to pay for tuition. I don't get much sleep anymore. I won't pass my finals. I won't pass the classes. I won't graduate. I......I don't know what to do. I don't want to drop out, but at this rate I just might have to. I want to keep trying. I want to get better, but I might not have the chance to do so. I....I'm scared.


My Dog Skip (2000) Reviewed by Eugene Novikov http://www.ultimate-movie.com/ Member: Online Film Critics Society
Starring Frankie Muniz, Luke Wilson, Kevin Bacon, Diane Lane. Directed by Jay Russell. Rated PG.
Boy-and-his-dog tales are one of the oldest staples in Hollywood. From Lassie to Old Yeller to Shiloh, they've defined the "family movie" and captured the hearts of generations. My Dog Skip, the latest in the genre, is one of the best films of its kind. It's remarkable because it doesn't patronize or condescend -- this is truly a family movie, not a kids' movie. With its distinctive, unsentimental tone, it's a moving experience that never descends into melodrama and hokum. My dog's name is Phil. He's a cocker spaniel.  This movie made me realize how much I value my pet -- my friend, my companion. Before he found fame on the popular new tv show Malcolm in the Middle, young Frankie Muniz played Willie, a boy growing up in 1940's Missouri. He's not very social; all the kids at school make fun of him and the bullies cruelly taunt him. He doesn't have any friends his age; his birthday party invitation list consists of his parents and grandparents. His only companion is neighbor Dick Jenkins (Luke Wilson), an athlete whom all the kids in the town look up to.
That all changes when, for his birthday, Willie's mom (Diane Lane) gives him a puppy. Willie names the terrier Skip but his dad (Kevin Bacon) thinks that he is not ready for a dog and tries to take Skip away. Mom and Skip win out in the end, however, and Willie gets to keep the dog. The two quickly become best companions and with Skip's help, Willie wins the friendship of the people who made fun of him most vigorously as well as the affections of the prettiest girl in school.
There is also a subplot about moonshiners who, for some reason, hide their booze in an old cemetery crypt. It does little except create vital conflicts for the dynamic duo of Willie and Skip to deal with. There's so little lip service paid to these scenes that the movie all but acknowledges the subplot as a mere plot device, thereby making it especially convenient for us to ignore it. My Dog Skip tackles the age- old concept of a dog being a boy's best friend but it takes it one step further: Skip isn't merely Willie's friend, he's his key to society. The dog opens up new possibilities for him, helps him make friends, enjoy childhood. This isn't necessarily a case of a boy moving up in the world simply because of possessions either: once he's in he stays in with character, heart and compassion. While the dog is given its fair due, by the end of the film most of the emphasis has been placed on the human. We really feel that Willie has been changed forever. And we see why.
Muniz is a wonderful actor mainly because of his penchant for downplaying crucial scenes. He's a real kid rather than a movie kid and anyone who's ever been in his situation can identify with him. He doesn't come off as a phony. Kevin Bacon does come off as a bit of a phony -- too thoughtless at first (sometimes, puffing his cigar, he seems genuinely malevolent) and then becoming a caring father too quickly. But I suppose this is what the character would seem like to a grown man remembering his childhood.
The end of the film brought tears to my eyes because it hits the nail squarely on the head. If a pet has the more of an influence on a person's life than most of the humans, it does not mean that he is antisocial. As anyone who has ever had a dog knows, a canine is often the most steadfast, loyal friend you'll have.  My Dog Skip is about a dog that taught a troubled young boy how to live.
Grade: A-
©2000 Eugene Novikov
Sent via Deja.com http://www.deja.com/ Before you buy.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Set of functions for processing forest inventory data with methods such as simple random sampling, stratified random sampling and systematic sampling. There are also functions for yield and growth predictions and model fitting, linear and non linear grouped data fitting functions, and statistical tests.
| Version: | 0.9.0 |
| Depends: | R (≥ 3.3) |
| Imports: | dplyr (≥ 0.7.0), ggplot2 (≥ 2.0), ggthemes, tidyr, broom, scales (≥ 0.4.1), psych, purrr, plyr, tibble, systemfit, htmltools, ggpmisc, rlang, graphics, utils, covr, rmarkdown, curl, car, stats, methods, magrittr, minpack.lm |
| Published: | 2018-10-26 |
| Author: | Sollano Rabelo Braga [aut, cre, cph], Marcio Leles Romarco de Oliveira [aut], Eric Bastos Gorgens [aut] |
| Maintainer: | Sollano Rabelo Braga <sollanorb at gmail.com> |
| License: | GPL-2 |
| NeedsCompilation: | no |
| Materials: | README NEWS |
| CRAN checks: | forestmangr results |
| Reference manual: | forestmangr.pdf |
| Package source: | forestmangr_0.9.0.tar.gz |
| Windows binaries: | r-devel: forestmangr_0.9.0.zip, r-release: forestmangr_0.9.0.zip, r-oldrel: forestmangr_0.9.0.zip |
| OS X binaries: | r-release: forestmangr_0.9.0.tgz, r-oldrel: forestmangr_0.9.0.tgz |
Please use the canonical form https://CRAN.R-project.org/package=forestmangr to link to this page.


{% extends 'layout.html' %} {% block pageTitle %} Manage your preferences for breast screening {% endblock %} {% block content %} {{ backLink({ "href": "index", "text": "Back to managing your preferences" }) }}
To help us plan properly for your appointment, tell us if there are any words or phrases you would prefer us not to use


Posted on Tuesday, July 14, 2015
Coming soon...
Read MorePosted on Saturday, July 18, 2015
Alright, coming into the second to last week of Phase 0 and it's time to talk about conflict! Or, you know, we could just, maybe, talk about anything else and just pretend it doesn't exist. You're cool with that, right? Unfortunately, as one who tries to avoid conflict at all costs, this is often how I think. It doesn't help, and it isn't healthy for myself or my relationship with the other person.
Read MorePosted on Friday, July 10, 2015
When you think of the times in your life where you’ve been the happiest, the proudest, or the most satisfied, which of the following values come to mind?
Read MoreAny sufficiently advanced technology is indistinguishable from magic.


--- layout: default title: Kapowell location: Castine, Maine ---
Colin Powell is a reporter, web developer and farmer in {{ page.location }}. Sometimes his work shows up on github. Sometimes he hoards it so that he can release it all at once and hope nobody notices his terrible coding style. Though mostly he's just working to get by.
What you'll find is that he's following/creating django apps and various other tools to make websites that interact with clients less painful to create and maintain.
This is his personal blog. You can find out more about what he does at: One Cardinal or Timberwyck Farm
.


createMetaplaylist
createMetaplaylist
is a function that allows to build a [metaplaylist]
(./metaplaylist.md) object from given contents information.
You may need to use this function because not all information about contents are known by the user when wanting to create a metaplaylist. For example, the end of a content will be found thanks to the content duration, that can be parsed from the content manifest.
How to import it
createMetaplaylist
is for now considered an “experimental” tool. This means
that its API could change at any new version of the RxPlayer (don’t worry, we
would still document all changes made to it in the corresponding release note).
As an experimental tool, it is imported as such:
import { createMetaplaylist } from "rx-player/experimental/tools";
You can then begin to use it right away.
How to use it
createMetaplaylist
takes two arguments :
-
contentInfos (
Array<Object>
) : The list of content information, in the playback order they should have in the metaplaylist.The list is an array of objects with this attributes :
- url (
string
): The URL of the source content - transport (
string
): The transport type of the content (dash
,smooth
or evenmetaplaylist
)
- url (
-
timeOffset (
number|undefined
) : the optionnal time offset that applies to the metaplaylist start time (default is 0).
Example:
createMetaplaylist(
[
// dash content, 10mn long
{
url: "https://somedashcontent.mpd",
transport: "dash",
},
// smooth content, 35s long
{
url: "https://somesmoothcontent.ism",
transport: "smooth",
},
// metaplaylist content, 100mn long
{
url: "https://somemetaplaylistcontent",
transport: "metaplaylist",
}
],
1000
)
The returned metaplaylist will look like :
{
type: "MPL",
version: "0.1",
dynamic: false,
contents: [
{
url: "https://somedashcontent.mpd",
transport: "dash",
startTime: 1000,
endTime: 1600,
},
{
url: "https://somesmoothcontent.ism",
transport: "smooth",
startTime: 1600,
endTime: 1635,
},
{
url: "https://somemetaplaylistcontent",
transport: "metaplaylist",
startTime: 1635,
endTime: 7635,
},
],
}