About me
Carlos Justiniano, HC.JS founder and a long-time programmer
A fullstack developer using NodeJS since 2011
Developed Node applications in production use
Developer at MediaMath, note: we're hiring!
Node = NodeJS
---
In this presentation I'll refer to NodeJS simply as Node.
This is part two
Part one is here: http://cjus.github.io/nodejs-presentation
---
In our last presentation we looked at NodeJS as a JavaScript execution engine and platform for building everything
from command line tools and desktop applications - to full-blown web applications.
In addition we looked at installing Node and adding packages such as the Express web application framework.
If you missed that talk you can find it here… http://cjus.github.io/nodejs-presentation
Quick recap
Node is an application which contains a JavaScript execution engine
Specifically, that engine is Google's Chrome V8 engine
However, Node is also much more
---
Let’s do a quick recap before diving into more advanced Node topics.
So we discussed node as a JavaScript execution engine because node is an application - which contains an embedded
JavaScript interpreter for running JavaScript programs.
Node is actually much more, but this simple definition serves as a foundation for learning more about Node.
Node as a JS execution engine
Start node at the terminal
Node then enters a mode called a REPL (Read Eval Print Loop)
Type JS and watch it execute
Tip: Press the TAB key to view a list of internal commands
Press CTRL-C twice to exit
---
Let’s see how Node works as a JavaScript execution engine.
Here’s a quick demo. Let’s start Node at the terminal prompt. We won’t go into installing node, that’s covered in
the last presentation.
$ node
That starts the Node and enters an interactive mode known as a REPL which is an acronym for Read Eval(uate) Print
Loop. We can enter JavaScript in this console and see the results.
Tip: Press the TAB key to view a list of internal commands
To exit simply press CTRL-C twice.
Node as a tool to build command line programs
Code sample here: https://github.com/cjus/nodejs-presentation-advanced/tree/master/NodeJSProjects/commandline
---
Node can also be used to build programs that execute on the command line.
Here’s our example of a command line program which displays system information.
Node as a web server
Code sample here: https://github.com/cjus/nodejs-presentation-advanced/tree/master/NodeJSProjects/web/login
---
We also saw how Node can be used to serve web pages and even act as an API server as part of a larger web
applications.
Here’s that demo.
Node as an API server
---
In the last demo Node is functioning as both a web server and an API server. To demonstrate this, let’s login using
a desktop client called PAW. PAW is a mac only tool for testing APIs that is popular among API developers.
Using PAW we can see how our API request is constructed and what Node returns in response.
We can also see this on the command line as the calls happen.
We’ll get back to this example a bit later, for now let’s take a slight detour and learn about web frameworks for
Node.
Node web frameworks
The last demo used a web framework, an add-on which consists of code that simplifies the creation of web
applications
There are lots of frameworks to choose from: ExpressJS, HapiJS, Koa and KrakenJS to name a few.
These frameworks differ in how they're configured, create servers, handle routing and middleware
---
The last demo shows Node as an API server. To accomplish we made use of a node package called a web framework.
A web framework consist of code which simplifies the creation of web applications. In this regard, there are a
number of frameworks available for Node. Which framework you should use depends on your projects needs.
The most popular are Express.js, Hapi.js and Koa.js, KrakenJS
These frameworks differ in how they create servers, handle routing and middleware. These are topics we haven’t
addressed in this presentation but you can learn more about them online.
Framework popularity
The following data is from an article entitled "Node.js Framework Comparison: Express vs. Koa vs. Hapi"
https://www.airpair.com/node.js/posts/nodejs-framework-comparison-express-koa-hapi
---
The following data is from an article entitled "Node.js Framework Comparison: Express vs. Koa vs. Hapi"
https://www.airpair.com/node.js/posts/nodejs-framework-comparison-express-koa-hapi
Framework popularity
---
The data shows that the Express.js framework is by far the most popular based on a number of important metrics.
| Metric | Express.js | Koa.js | Hapi.js
| —————— | ——————: | ——: | ——:
| Github Stars | 16,158 | 4,846 | 3,283
| Contributors | 163 | 49 | 95
| Packages that depend on: | 3,828 | 99 | 102
| StackOverflow Questions | 11,419 | 72 | 82
Available books
---
Another important metric is how many books and online tutorials you can find while trying to learn a framework.
I did a quick search on Google Shopping for books on Express, Koa and Hapi:
|Framework | Query | Hits
|—————— | :——: ——:
| Express.js | “express.js” +book | dozens
| Hapi.js | “hapi.js” +book | 1
| Koa.js | “koa.js” +book | 0
| Kraken.js | “kraken.js” +book | 0
Framework age
---
That said, Express.js is also the oldest Node framework. So it’s conceivable that another framework may someday
exceed it in popularity among developers.
For now at least, Express would be a reasonable place to start.
| Framework | First version
|—————— | ——:
| Express.js | June 2009
| Hapi.js | August 2011
| Koa.js | August 2013
In our earlier talk we looked at the ExpressJS framework for Node. That’s the framework behind our prior demo.
Code review of a Node / Express app
Code can be found in this presentation's repo on GitHub: https://github.com/cjus/nodejs-presentation-advanced
Look inside of the NodeJSProjects/web/login folder
---
Let’s revisit our login sample application. We’ll take a closer look at how the application is constructed. Our
example uses the Express framework, but keep in mind that what you’ll see is comparable to what you’ll find in other
frameworks.
The source for our application can be found this presentation’s repo on GitHub:
https://github.com/cjus/nodejs-presentation-advanced
You’ll find the login application in the NodeJSProjects/web/login folder.
Our application’s JavaScript can be found in the project’s root, app.js and in the routes and bin folder. The
node_modules folder is where our package dependencies are stored. We’ll learn more about node packages shortly. The
views folder contains web templates which are not really used in this project.
The routes folder contains handlers for the URLs our application handles. More about that shortly.
The public folder holds are website files, which Node will statically serve when users access our site.
Let’s see how this all comes together.
Application Routes
Site loads from: http://localhost:3000
Site calls http://localhost:3000/api/v1/login from the login page
---
At a high level when the user visits our site, in our case at http://localhost:3000, our Express based app will
handle that URL and respond by sending back the sites index.html from the public folder. That will then trigger the
user’s web browser to make requests for the associated javascript and CSS files.
Once our site loads in the user’s browser they’re able to login to our application by sending their user name and
password. That is essentially a jQuery AJAX call to http://localhost:3000/api/v1/login
So in this basic example, our application is both serving a website and responding to an API request.
Note the format of our API URL. It contains a path to ‘api’ followed by a ‘v1’ segment and a ‘login’ segment. This
structuring allows us to differentiate API requests from other requests, such as loading the site’s CSS.
Additionally the v1 portion allows us to support multiple API versions. The last portion is the login which is the
name of the individual API. We might, for example, have a logout API.
---
Let’s look at some code.
The first thing we do to wire up our login handler is to add a route entry to our app.js file. Here we’re just
saying that when express see’s a request for /api/v1/login it should send that request to our login.js handler.
The separates a login path from the code which actually handles it.
At the very top of our login.js handler we see a warning about this code fragment not being secure. Don’t do this at
work!
We create an express object which uses the requires function to load the express module. We then create a router
object from express.Router.
We’ll look at node modules a bit later in this presentation.
So we essentially provide an anonymous function as our route handler. Note the function signature contains three
parameters: req, res, next. The request object allows us to see the contents of a request. This allows us to examine
HTTP headers and review body responses such as a JSON object that might have been sent from our website. The
response object allows us to package up a response which will then get sent back to the user’s web browser or
calling client, such as PAW.
The last parameter, next, is used to chain request processing - but we won’t go into that in this presentation.
---
In public/javascripts/app.js we see our API request which will get sent to api/v1/login. This uses the $.post jQuery
method to send a JSON object to our server.
Note who the request object has handlers for done and fail. Because our response won’t be handled immediately we
have to create callback handler for use when a response arrives. This pattern is known as promises and it’s
something you should be using in your own applications. In this case we’re using jQuery promises and you can learn
more about them online.
Another point worth noting is that our request is to ‘api/v1/login’ because the API handling is on the same server
as the one where the website is being served from. You could make requests to other servers by using something like
‘http://someserver.com/api/v1/login'. This also allows you to have one node server to serve web content and another
to process API requests. This core concept will later lead us to micro services the new web hotness.
So in addition to handling requests, Node applications can also make requests in the same manner listed in this
example. This allows Node applications to communicate with one another and to become larger than the sum of their
parts.
Using modules in our app
npm install module_name
for example: npm install express
Then require('module_name') in our app
---
In the last example we saw how our login handler requested the express module and called the Router constructor to
create a new route.
That allows us to build on an existing express component. In a similar fashion you might need to create an Excel
spreadsheet or access a remote database. Node handles this by allowing you to load modules which address your
particular need.
In the first presentation we saw how we could add a module, such as express.
Then we can require the module for use in our app.
NPM - The Node Package Manager
The Node Package Manager is a tool which ships with Node and assist with the searching, packaging, publishing
and updating of node modules
---
Modules offer a huge boon to developers building Node-based applications. By building modules you’re able to package
reusable functionality to share across projects within your company or with the global open source community at
large. And better yet you can locate modules which perhaps address your project's specific concerns.
The Node Package Manager is a tool which ships with Node and assist with the searching, packaging, publishing and
updating of node modules.
Because every package module you add to your project becomes a dependency of your project, NPM is often considered a
dependency manager, however it's much more.
We saw an npm example when we install ExpressJS:
```
$ npm install express
```
If you look under the hood you’ll find that the NPM tool is in-fact a JavaScript file which runs at the command line
using Node.
In this example snap shot from my machine we see that npm is just a symlink to the npm-cli.js program.
```
lrwxr-xr-x 1 root admin 38B Jul 21 23:45 npm -> ../lib/node_modules/npm/bin/npm-cli.js
```
The NPM repository
npmjs.org contains a massive repository of node packages
Over 178,000 packages exist!
There are no shortage of packages and likely dozens to fit your apps specific needs
---
Visit npmjs.org and you'll find a massive repository of node packages. When I last visited the site, while writing
this presentation, I found that on that day over 178,000 total packages existed with over 111 million downloads in
the last day alone! npm has become the largest source package repository ever created. It has far surpassed other
leading repositories such as RubyGems and CSPAN.
For an interactive presentation see: https://unpm.nodesource.com/
The npmjs.org site allows you to search for packages, so the problem isn't searching - it's finding which of many
packages to use to address your app's concern. Let's just say that there is not shortage of packages!
Building and publishing an NPM package
You too can contribute Node modules!
Let's see how
---
Part of the reason that there are so many Node packages is because anyone can contribute a package. You won't find
an acceptance committee such as those who police mobile app stores. In fact, you too can contribute node modules!
We'll see how to do that next.
About a year ago I contributed two open source packages to the NPM repository.
Let me step through the process with you.
* The I configured NPM for publishing
$ npm set init.author.name "Carlos Justiniano"
$ npm set init.author.email<EMAIL_ADDRESS>$ npm set init.author.url "http://cjus.me"
$ npm adduser
The last command wrote my credentials to the ~/.npmrc file, where it will be available to subsequent npm commands.
* Next I created my project on github.
* Then I update my projects package.json file to include information which NPM publish would need. Important fields
include "name", "contributors", "repository", "keywords" and "license"
* The npm publish command reads the package.json to determine how to configure and publish your project in the npm
repository.
* To publish we just use: npm publish
For complete step by step tutorial see: https://quickleft.com/blog/creating-and-publishing-a-node-js-module/
NVM - The Node Version Manager
NVM allows you to install and switch between different versions of Node
This is useful for trying new features and testing existing code
---
Node is updated periodically in response to enhancements and new features. It's a good idea to keep up with updates
by trying your programs against newer version of Node. This can, however, become problematic if you only have one
version of node installed on your machine at one time.
To address this issue there is the Node Version Manager, a tool which allows you to install and switch between
multiple versions of Node.
Installing a new version of node is as simple as:
$ nvm install 0.12.7
And switching beween version is:
$ nvm use 0.10.25
to find out more about NVM visit: https://github.com/creationix/nvm
Uses of Node
---
Node is being used to address many types of concerns.
When people first learn about node they see it as a backend server technology, primarily used to build web facing
applications.
Uses of Node
---
Later they learn that Node is also used to power the build tools they use to package their web applications. Tools
like Grunt and Gulp.
Along the way we learn that node can be used to build command line tools. This is quickly becoming an alternative to
shell / batch scripting.
What many developers don’t realize is that Node is used to build desktop applications and power Robotics and the
Internet of Things.
You’ll find Node in use from tiny devices (see Node on Raspberry Pi) to large scale clusters running on cloud
infrastructure.
For JavaScript developers this is awesome. You get to take your JS and Node experience and apply it to an increasing
field of opportunities. The same can’t be said for many other options.
Node and Robotics
Checkout CylonJS and NodeBots!
---
Oh, and did I mention you can build robots using Node?
Checkout CylonJS and NodeBots!
Node internals
---
Node consist of two main parts. One part is concerned with handling requests in a single threaded loop. This is
known as the event loop. This is also where the V8 JavaScript engine is used.
Node takes I/O tasks and creates threads to handle them. This means that those threads are able to run efficiently
on multi-processor and multi-core machines. In fact you can scale the single event loop process by running multiple
version of node on a single machine. This is handled using Node clustering.
You can think of Node’s Event loop as a staging area for I/O tasks.
So why is this effective? Because a majority of time in web applications is spent waiting for file and network
communication to take place. By not waiting for those requests to complete, Node can focus on handling more requests
and running the rest of your code.
Node Add-ons: Extending Node, using C++
Modules written as native C/C++ components call shareable objects
Extends Node and JavaScript by allowing progams to call into the native module
Let's see an actual example. Code available here: NodeJSProjects/node-addon
---
Node can be extended using C/C++ native modules. There are a variety of reasons for wanting to do this, but a common
one has to do we getting absolute performance. Another reason is to hook into lower level Node functionality which
might not be accessible from JavaScript. At least one reason to avoid doing this is that addons have to be compiled
for each platform they're expected to run on. So you lose the portability inherit in most JavaScript node
applications.
Node itself is written in the C++ programming language. As such, it can be extended using C and/or C++. Node add-ons
are packaged as dynamically loadable modules which is common in the *nix camp and similar to DLLs on the Microsoft
Windows platform. The basic idea is that a module can exists outside of the main program as a component that can be
loaded at runtime. This is an important approach because otherwise you'd have to obtain the NodeJS source code and
extend it prior to rebuilding your own custom version. This is of course a bad idea on so many levels. But hey, this
is open source - so you could do that if you really wanted too.
Ok, let’s see an example!
To do this I’ll need to log onto my Linux box on Amazon.
Preparing our build environment
$ npm install nan@latest --save
$ sudo npm install node-gyp -g
---
For this demo we need to install NAN, which allows us to which makes it easier to work with C++, Node and the V8
API. It also allows us to more easily target multiple versions of Node
$ npm install nan@latest --save
Next we need to make sure that Node-gyp is installed globally. Node-gyp is a Node.js native addon build tool.
$ sudo npm install node-gyp -g
Building our project
$ node-gyp configure
$ node-gyp build
Then we use in Node as:
var addon = require('./build/Release/hello.node');
console.log(addon.hello());
---
Each Node addon consist of at least three files:
* binding.gyp - binding information for our source files
* One or more C/C++ files which actually make the code of our addon
* A package.json file which describes our project and its dependencies
We use node-gyp to configure our project
$ clear
$ node-gyp configure
Now we can actually build our project
$ node-gyp build
The output of the last two operations is the creation of a hello.node module in the ~/node-addon/build/Release
folder. This file is actually a Dynamic Shared Object which usally end in a .so extension. Here, node-gyp names our
addon with a .node extension because Node looks for such files when a module is required.
We can now use the new node addon in our JS projects:
var addon = require('./build/Release/hello.node');
console.log(addon.hello());
For more information about this demo checkout:
https://github.com/nodejs/node-addon-examples/tree/master/1_hello_world
### Useful links for understanding and building Node add-ons
So much of this is outside the scope of this talk, but if you're interested in learning more see these great
articles:
* https://nodejs.org/api/addons.html
* https://www.npmjs.com/package/node-gyp
* https://www.youtube.com/watch?v=q1ri36UI5GA
Microservices and Node
Micro Services is "An approach to developing a single application as a suite of small services". - Martin Fowler
Node has emerged as an ideal way of building Micro Services. Coupled with a framework such as Express and the vastness that is NPM, Node-based microservices can be quickly built and tested.
---
Micro Services is "An approach to developing a single application as a suite of small services". - Martin Fowler
There a number of reason for doing this:
* Bodies of code serve specific functionality, such as email, order processing or image processing.
* Code is easier to maintain for each service.
* Each service can be build using the technologies that given team feels comfortable with.
* Large applications can scale based on the load of individual services.
Node has emerged as an ideal way of building Micro Services. Coupled with a framework such as Express and the
vastness that is NPM, Node-based microservices can be quickly built and tested.
Good links on Microservices:
* https://www.youtube.com/watch?v=2yko4TbC8cI
* https://www.youtube.com/watch?v=wgdBVIX9ifA
* https://www.youtube.com/watch?v=nMTaS07i3jk
* https://www.youtube.com/watch?v=X0_d0ci8UcY
* http://exponential.io/blog/nodeday-2014-you-only-need-100-lines-of-javascript/
William Blankenship's talk next month on "Rapidly developing microservices using Node.JS and Docker" will cover this
topic in greater depth.
Good links on Microservices:
https://www.youtube.com/watch?v=2yko4TbC8cI
https://www.youtube.com/watch?v=wgdBVIX9ifA
https://www.youtube.com/watch?v=nMTaS07i3jk
https://www.youtube.com/watch?v=X0_d0ci8UcY
http://exponential.io/blog/nodeday-2014-you-only-need-100-lines-of-javascript/
---
Good links on Microservices:
* https://www.youtube.com/watch?v=2yko4TbC8cI
* https://www.youtube.com/watch?v=wgdBVIX9ifA
* https://www.youtube.com/watch?v=nMTaS07i3jk
* https://www.youtube.com/watch?v=X0_d0ci8UcY
* http://exponential.io/blog/nodeday-2014-you-only-need-100-lines-of-javascript/
William Blankenship's talk next month on "Rapidly developing microservices using Node.JS and Docker" will cover this
topic in greater depth.
Don't miss next month's talk:
Rapidly developing microservices using Node.JS and Docker by William Blankenship
---
William Blankenship's talk next month on "Rapidly developing microservices using Node.JS and Docker" will cover this
topic in greater depth.
Contact
cjus on Twitter and Github
Email<EMAIL_ADDRESS>About: http://cjus.me


TypeScript is an open-source language which builds on JavaScript, one of the world’s most used tools, by adding static type definitions. Types provide a way to describe the shape of an object, providing better documentation, and allowing TypeScript to validate that your code is working correctly. Writing types can be optional in TypeScript, because type inference allows you to get a lot of power without writing additional code.
All valid JavaScript code is also TypeScript code. You might get type-checking errors, but that won't stop you from running the resulting JavaScript.  TypeScript code is transformed into JavaScript code via the TypeScript compiler or Babel. This JavaScript is clean, simple code which runs anywhere JavaScript runs: In a browser, on Node.JS or in your apps.
Adopting TypeScript is not a binary choice, you can start by annotating existing JavaScript with JSDoc, then switch a few files to be checked by TypeScript and over time prepare your codebase to convert completely. TypeScript’s type inference means that you don’t have to annotate your code until you want more safety.
Most of the world's JavaScript is un-typed, and inference can only go so far. To address this, the TypeScript team helps maintain Definitely Typed - a community project to provide types and inline documentation to existing JavaScript. This project allows the community to maintain type definitions for JavaScript libraries without putting extra pressure on their maintainers.
By handling a lot of the editor integration inside TypeScript, you can get a consistent experience working in many editors. This lets you easily jump between editors like Visual Studio, Visual Studio Code, Nova, Atom, Sublime Text, Emacs, Vim, WebStorm and Eclipse. TypeScript's editor integration supports JavaScript, so it's quite likely you're already using TypeScript under the hood.
The TypeScript team contributes to the TC39 committees which help guide the evolution of the JavaScript language. When new features have reached stage 3, then they are ready for inclusion in TypeScript. For example the TypeScript team championed proposals like Optional Chaining, Nullish coalescing Operator, Throw Expressions and RegExp Match Indices.
Via npm:
> npm install -g typescript
By installing TypeScript’s Visual Studio plugins
Like JavaScript and any other language, TypeScript also provides basic data types to handle numbers, strings, etc. Some common data types in TypeScript are: number, string, boolean, enum, void, null, undefined, any, never, Array and tuple.
In TypeScript, numbers are floating point values that have a type of number. You can assign any numeric values including decimals, hexadecimals, binary, and octal literals. But to use binary and octal literals, you must use a TypeScript version which follows ECMAScript 2015 or higher.
let decimalValue: number = 10;
When you want to use textual data, string types are used and get denoted by the keyword string. Like JavaScript, TypeScript also uses double quotes (") and single quotes (') to surround the string value.
let lastName: string = 'Chowdhury';
To use boolean data types in TypeScript, for declaring variables, use the boolean keyword. Here's a simple code to declare a boolean type variable:
let isPrimaryAccount: boolean = true; let hasCards: boolean = false;
Enumerated data types (enums) are a set of numeric values with more friendly names. It's an addition on top of JavaScript that TypeScript offers. The variables of enumerated data types are declared with the keyword enum. Here's how you can declare an enum variable and use it in TypeScript:
enum CardTypes { Debit, Credit, Virtual } let card: CardTypes = CardTypes.Debit;
In general, this type of data types are used in functions that do not return any value.
For example,
function showMessage(): void { ... }
. In TypeScript, you can also declare a variable of type void,
but can only assign undefined or null to them. We will discuss undefined and null types in the following sections.
You can declare a variable of type null using the null keyword and can assign only null value to it. As null is a subtype of all other types, you can assign it to a number or a boolean value.
let nullValue: null = null; let numericValue: number = null;
You can use the undefined keyword as a data type to store the value undefined to it. As undefined is a subtype of all other types, just like null, you can assign it to a number or a boolean value.
let undefinedValue: undefined = undefined; let numericValue: number = undefined;
While writing code for which you are unsure of the data type of a value, due to its dynamic content, you can use the keyword any to declare said variable. This is often useful when you are seeking input from users or at third-party library/service. This is also useful when you are declaring an array which has a mixed data type. It's just like the dynamic keyword available in C#. For example:
let dynamicValue: any = "Kunal Chowdhury"; dynamicValue = 100; dynamicValue = 0b1100101; dynamicValue = true;
The never type represents the data type of values that never occur. For example, the following function that always throws an exception can never return a value. Thus, the return type of the function can be set as never. Here's the function declaration:
function throwError(message: string): never { throw new Error(message); }
Just like JavaScript, you can work with arrays in TypeScript and can define them in either of the following two ways; the second approach, however, is the more generic way to declare arrays:
let marks: number[] = [80, 85, 75];
A tuple is a data type that allows you to create an array where the type of a fixed number of elements are known but need not be same. While accessing an element of a Tuple with a valid known index, the data of correct type will be returned. In case you access an element outside the set of known indices, a union type will be used. Consider the following code snippet as an example:
let person: [string, number] = ["Hello world", 2021];


If you use this tool, please cite the following work:
- G. Pizzi, S. Milana, A. C. Ferrari, N. Marzari, M. Gibertini,
Shear and Breathing Modes of Layered Materials, ACS Nano (2021), doi:10.1021/acsnano.0c10672.
- You might also want to cite the spglib,
ASE and
pymatgen libraries that are used internally by the tool.
You can access the source code of this tool on
its GitHub project page.


In order to connect a graph database to Blueprints, the Blueprints property graph interfaces must be implemented. For non-native Blueprints graph databases, the raw graph database API and the Blueprints API need to be coupled. For native Blueprints graph databases, no such coupling is needed as only the interfaces need to be implemented. Each subsection discusses the various implementations of the Blueprints interfaces maintained by the main Blueprints distribution.
DEPENDENCY NOTE: It is important to note that including a <dependency/>
to Blueprints does not entail all the respective graph database/framework dependencies distributed with Blueprints. The reason for this is that it allows users to include only those graph implementations they wish for their particular projects (e.g. based on licensing, size, performance, etc.).
This section defines the general rules and standards by which Blueprints implementations are expected to work. By following these implementation details, developers implementing the Blueprints API can expect consistent operations throughout the TinkerPop stack. Many of these details are enforced by the Blueprints test suite, so there will be little choice in those areas when it comes to implementation, however others are merely convention and not explicitly enforced. Choosing not to following convention will introduce risk that the implementation will not work as expected elsewhere in the stack.
The getVertex
and getEdge
methods on the Graph
class accept an object
type as the parameter for the respective vertex or edge identifier. Always coerce that identifier to its expected type. If the underlying graph, only expects a long
as an identifier, then ensure that the parameter is converted to a long (as it may come in as a String
or other such value). If the identifier cannot be coerced to its expected type then simply return null
.
The identifier parameter cannot be null
and if it is an exception should immediately be thrown:
if (null == id)
throw ExceptionFactory.vertexIdCanNotBeNull();


--- layout: page title: Agenda subtitle: CTFs first semester ---
| Event Name | Type | Date |
|---|---|---|
| RCTF 2021 | On-line (Jeopardy) | 11 Set., 01:00 UTC — 13 Set. 2021, 01:00 UTC |
| N1CTF 2021 | On-line (Jeopardy) | 16 Out., 00:00 UTC — 18 Out. 2021, 00:00 UTC |
| ASIS CTF Quals 2021 | On-line (Jeopardy) | 22 Out., 15:00 UTC — 24 Out. 2021, 15:00 UTC |
| Balsn CTF 2021 | On-line (Jeopardy) | 20 Nov., 02:00 UTC — 22 Nov. 2021, 02:00 UTC |


October 31ST at 5:15 pm
9/17/2016
Calling all kids and those young at heart! The Annual Halloween Costume parade will be held October 31st at 5:30 pm. Participants should meet at the parking lot at Prairie Fields Park at 5:15 for the Costume Contest.
The parade route will leave the park and go west on Gentian St., north on Tickseed Ave., east on Dropseed Dr., and south at Prairie Rose Lane where they will return to the Park. Residents are also encouraged to line the parade route and cheer the participants along.


BadTwitterClone Terms of Use
Home
- DON'T HACK US! That is not cool.
- Don't tamper with website cookies or our MySql database where we store all user information.
- Use a strong password when you register an account. There are lots of hackers out there!
- If you forget your account password, don't worry, we can email it to you. Just email joe@badtwitterclone.com.


[1]
with either unit the street names If Garmin nuvi 255W I would n't use.
[2]
with either unit the street names If Garmin nuvi 255W I would n't compare but of 'extras ' I ca n't use.


gdalcompare.py
Compare two images.
Synopsis
gdalcompare.py [-sds] golden_file new_file
Description
The gdalcompare.py script compares two GDAL supported datasets and reports the differences. In addition to reporting differences to the standard output the script will also return the difference count in its exit value.
Image pixels, and various metadata are checked. There is also a byte by byte comparison done which will count as one difference. So if it is only important that the GDAL visible data is identical a difference count of 1 (the binary difference) should be considered acceptable.
- -sds
If this flag is passed the script will compare all subdatasets that are part of the dataset, otherwise subdatasets are ignored.
- <golden_file>
The file that is considered correct, referred to as the golden file.
- <new_file>
The file being compared to the golden file, referred to as the new file.
Note that the gdalcompare.py script (like all the other scripts) can also be called as a library from python code: from osgeo_utils import gdalcompare. The primary entry point is gdalcompare.compare_db() which takes a golden gdal.Dataset and a new gdal.Dataset as arguments and returns a difference count (excluding the binary comparison). The gdalcompare.compare_sds() entry point can be used to compare subdatasets.


To become a Warcryer your level must be over 40 and you must collect the Mark of the Pilgrim, the Mark of Glory, and the Mark of the War Spirit.
Obviously you lack the marks and the skills to advance. Come back when you have collected all 3 marks and completed the necessary training and I will change your occupation to Warcryer.


SRFS and Period (X and Y Directions)
Force Distribution (X Direction)
According to the Location, Importance category, Site Class, Height of the building and NBC 2012 Articles <IP_ADDRESS> and <IP_ADDRESS>, the selected SFRA is:
When S(0.2) is less than or equal to 0.12, earthquake loads and effects need not be considered.
| Level | h (m) | W (kN) | F(kN) | V (kN) |
|---|---|---|---|---|
| Level | M(kN) | J () * | Mreduced(kN-m) * |
|---|---|---|---|
* Reduction coefficient for base overturning moment is not applied.
Force Multiplier For Deflections Calculation (X Direction)
Force Distribution (Y Direction)
According to the Location, Importance category, Site Class, Height of the building and NBC 2012 Articles <IP_ADDRESS> and <IP_ADDRESS>, the selected SFRA is:
When S(0.2) is less than or equal to 0.12, earthquake loads and effects need not be considered.
| Level | h (m) | W (kN) | F(kN) | V (kN) |
|---|---|---|---|---|
| Level | M(kN) | J () * | Mreduced(kN-m) * |
|---|---|---|---|
* Reduction coefficient for base overturning moment is not applied.


Melbourne Zoo welcomes first baby koala in eight years
Joey, whose gender is not yet known, is starting to explore the world beyond its mother's pouch.
Australia's Melbourne Zoo is celebrating the arrival of its newest koala - an unnamed five-month-old joey - the first to be born at the zoo in more than eight years.
The baby has just started to peek out from its mother's pouch where it has been growing for the past few months.
The zoo's Australian Bush keeper Maddy Jamieson said the joey and first-time koala mum Karri were doing well.
"Karri has been a fantastic mum so far," Jamieson said. "She is feeding the joey really, really well and she is looking after her and doing everything that she is supposed to be doing."
Karri was the last koala to be born at the zoo.
Her joey's gender is not yet known, and it is not expected to emerge fully from her pouch for about a month, when it will start to eat the gum leaves that make up the adult koala's diet.
Koalas have been listed as a vulnerable species in many parts of Australia with their population under threat from habitat destruction and disease.
They prefer to live in coastal forests, which are being cleared for suburban expansion, and are thought to have been victims of the ferocious bushfires that have swept through the country's eastern coast in recent weeks.


WARNING: Below post contains vulgar and potentially offensive language
I have been wanting to learn more about webscraping and working with text data in R, so I thought a great way to do this would be through visualizing data from the online space where I spend a significant amount of time on Sundays during football season: the Green Bay Packers subreddit. During each game of the season, there is a “game thread” where anonymous users post comments about the game as the game is happening (see here for last week’s game thread).
Luckily, there is an R package called RedditExtractoR that can help us scrape comments from Reddit. Along with this package, we will also load the tidyverse package.
library(tidyverse)
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──
## ✓ ggplot2 3.3.5 ✓ purrr 0.3.4
## ✓ tibble 3.1.5 ✓ dplyr 1.0.7
## ✓ tidyr 1.1.4 ✓ stringr 1.4.0
## ✓ readr 2.0.0 ✓ forcats 0.5.1
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()
library(RedditExtractoR)
Rather than manually collecting all of the URLs from each of the game threads, I will use the find_thread_urls function to search for threads that contain the phrase “Game Thread: Green Bay Packers vs”, as the title of all game threads contain this phrase.
post_urls <- find_thread_urls(keywords = 'Game Thread: Green Bay Packers vs',
subreddit = 'GreenBayPackers', period = "year")
We can then take a look at the titles of the post that the function pulled for us:
head(post_urls$title)
## [1] "2021 OTAs: May 25th"
## [2] "[Silver] The @packers are trying to negotiate a restructured contract with Aaron Rodgers that could free up cap space... the deal (if it happens) might also inform us about how they view his future in Green Bay. @nflnetwork @PatrickClaybon"
## [3] "Green Bay Packers stock sale tops $36 million, 110,000 shares sold in first two days"
## [4] "[Bukowski] @RapSheet said on @PatMcAfeeShow's show the #Packers wanted to convert Aaron Rodgers' base into signing bonus. Rodgers wanted an extension. Neither happened, so Green Bay moved money w/ other guys."
## [5] "Reasons to be Optimistic about the Postseason"
## [6] "2021 Training Camp: July 29th"
As you can see, something clearly went wrong when searching for just threads that contained the specified keyword, so we will have to clean up this data ourself. Luckily, tidyverse gives us the tools to do this in a few lines of code!
The below code does the following:
-
Renames the column date_utc to just date, and pulls out only the date, title, and url of each post
-
Changes the date column from a character to a date class
-
Selects threads that only happened this season (which started on 9/12/21)
-
Using regular expressions, I select threads that have the phrase “Week [1-18]” somewhere in the title (this is common amongst all game threads). I noticed that the week 4 thread had “Week[4]” so I had to account for the lack of a space between Week and the first bracket.
-
Remove pre-and-post game threads (threads that people comment on before or after the games)
-
Pull out the URLs for all of the game threads
game_thread_urls <-
post_urls %>%
select(date = date_utc, title, url) %>% #rename date_utc to date
mutate(date = as.Date(date, format = "%Y-%m-%d")) %>% # format date column
filter(date > "2021-09-11") %>% # Only select games from 2021 season
filter(str_detect(title, regex("\\[Week.*[(\\d+)]"))) %>% # Select threads with [Week 1-18] in name
filter(!str_detect(title, regex("Pre"))) %>% # Remove pre-game threads
filter(!str_detect(title, regex("Post"))) %>% # Remove post-game threads
pull(url)
We can then use the get_thread_content to get a data frame with comments for the game threads. Due to the reddit API, unfortunately we do not get all of the comments in each thread, but we still get close to 500 for each thread.
game_thread_comments <- get_thread_content(game_thread_urls)[[2]]
We can take a look at the first several comments
head(game_thread_comments$comment)
## [1] "Packers fan here. I'm not going to brag. Chicago may have lost, but the looked good doing it. Also, GB got hoes out of a TD and a pick."
## [2] "Well, at least they got hoes."
 I own you.  THE BEARS STILL SUCK! THE BEARS STILL SUCK!"

What I decided to do in this post is visualize the most common 200 words in a word cloud. To do this, I will use the unnest_tokens and anti_join functions from the tidytext package. The unnest_tokens function “flattens” each comment, so that I have a data frame where each row is a word from a comment. The anti_join function is then used to remove stop words. I then removed the word “game” since this was not interesting to me, and I also noticed that there was a weird character to remove. Finally, I removed all numbers from the word list.
library(tidytext)
comments_words <-
game_thread_comments %>%
select(comment) %>%
unnest_tokens(word, comment) %>% # Create data frame where each row is a word
anti_join(stop_words) %>% # Remove stop words
filter(!(word %in% c("game", "à"))) %>% # Remove the word "game" and also weird character
filter(!str_detect(word, regex("(\\d+)"))) # Remove numbers
After creating this data frame, I used the count function to count the number of times each word appears in all of the comments, and I selected the 200 most commonly used words.
### Create data frame where each row is a word and the number of times it appears
### And then select top 200 most commonly used words
comments_words_freq <-
comments_words %>%
count(word, name = "freq") %>%
slice_max(freq, n = 200)
Finally, I used the wordcloud2 package to create the word cloud, using a color pallette corresponding to the Packers green and gold.
library(wordcloud2)
color_palette <- c("#203731", "#FFB612")
wordcloud2(comments_words_freq,
color=rep_len(c(color_palette), nrow(comments_words_freq)),
size = .75)
There are some future potential directions to go with this. One would be to dig into the reddit API to get all of the comments, rather than just 500 from each thread. Another would be to train a neural network to generate game thread comments. If you have any ideas, I’d love to hear them!


Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Learning
Title: cleverhans v2.0.0: an adversarial machine learning library
(Submitted on 3 Oct 2016 (v1), last revised 5 Oct 2017 (this version, v4))
Abstract: \texttt{cleverhans} is a software library that provides standardized reference implementations of \emph{adversarial example} construction techniques and \emph{adversarial training}. The library may be used to develop more robust machine learning models and to provide standardized benchmarks of models' performance in the adversarial setting. Benchmarks constructed without a standardized implementation of adversarial example construction are not comparable to each other, because a good result may indicate a robust model or it may merely indicate a weak implementation of the adversarial example construction procedure.
This technical report is structured as follows. Section~\ref{sec:introduction} provides an overview of adversarial examples in machine learning and of the \texttt{cleverhans} software. Section~\ref{sec:core} presents the core functionalities of the library: namely the attacks based on adversarial examples and defenses to improve the robustness of machine learning models to these attacks. Section~\ref{sec:benchmark} describes how to report benchmark results using the library. Section~\ref{sec:version} describes the versioning system.
Submission history
From: Ian Goodfellow [view email][v1] Mon, 3 Oct 2016 22:04:07 GMT (5kb)
[v2] Wed, 5 Oct 2016 13:54:04 GMT (5kb)
[v3] Wed, 14 Dec 2016 10:47:15 GMT (7kb)
[v4] Thu, 5 Oct 2017 17:27:32 GMT (7kb)


public class MouseEvent extends PenEvent
Event.Type<H>
| Modifier and Type | Field and Description |
|---|---|
static int |
MOUSE_IN
The event type for a mouse moving into a control.
|
static int |
MOUSE_MOVE
The event type for a mouse moving over a control.
|
static int |
MOUSE_OUT
The event type for a mouse moving outside a control.
|
static int |
MOUSE_WHEEL
The event type for a mouse wheel.
|
int |
wheelDirection |
absoluteX, absoluteY, EVENT_NAME, modifiers, PEN_DOWN, PEN_DRAG, PEN_DRAG_END, PEN_DRAG_START, PEN_UP, x, y
| Constructor and Description |
|---|
MouseEvent() |
clone, dispatch, getEventName, toString, update
clearQueue, getNextAvailableEventId, isAvailable, touch
public int wheelDirection
DragEvent
public static final int MOUSE_MOVE
public static final int MOUSE_IN
public static final int MOUSE_OUT
public static final int MOUSE_WHEEL


Who is ordering?
How can we contact you?
where do i place the order to?


A lighting parameter is of one of five types: color, position, direction, real, or boolean. A color parameter consists of four floating-point elements, one for each of R, G, B, and A, in that order. There are no restrictions on the allowable values for these parameters. A position parameter consists of four floating-point coordinates (x, y, z, and w) that specify a position in object coordinates (w may, in some cases, be zero, indicating a point at infinity in the direction given by x, y, and z). A direction parameter consists of three floating-point coordinates (x, y, and z) that specify a direction in object coordinates. A real parameter is one floating-point value. The various values and their types are summarized in Table 2.5. The result of a lighting computation is undefined if a value for a parameter is specified that is outside the range given for that parameter in the table.
Table 2.5: Summary of lighting parameters.
The range of individual color components is .
There are n light sources, indexed by . (n is an implementation dependent maximum that must be at least 8.) Note that the default values for and differ for i=0 and i>0.
Before specifying the way that lighting computes colors, we introduce operators and notation that simplify the expressions involved. If and are colors without alpha where and , then define . Addition of colors is accomplished by addition of the components. Multiplication of colors by a scalar means multiplying each component by that scalar. If and are directions, then define
(Directions are taken to have three coordinates.) If and are (homogeneous, with four coordinates) points then let be the unit vector that points from to . Note that if has a zero w coordinate and has non-zero w coordinate, then is the unit vector corresponding to the direction specified by the x, y, and z coordinates of ; if has a zero w coordinate and has a non-zero w coordinate then is the unit vector that is the negative of that corresponding to the direction specified by . If both and have zero w coordinates, then is the unit vector obtained by normalizing the direction corresponding to .
If is an arbitrary direction, then let be the unit vector in 's direction. Let be the distance between and .  Let be the eyepoint ( in eye coordinates).
The color produced by lighting a vertex is given by
where
All computations are carried out in eye coordinates.
The value of A produced by lighting is the alpha value associated with . Results of lighting are undefined if the coordinate (w in eye coordinates) of is zero.
Lighting may operate in two-sided mode (), in which a front color is computed with one set of material parameters (the front material) and a back color is computed with a second set of material parameters (the back material). This second computation replaces with . If , then the back color and front color are both assigned the color computed using the front material with .
The selection between back color and front color depends on the primitive of which the vertex being lit is a part. If the primitive is a point or a line segment, the front color is always selected. If it is a polygon, then the selection is based on the sign of the (clipped or unclipped) polygon's signed area computed in window coordinates. One way to compute this area is
where and are the x and y window coordinates of the ith vertex of the n-vertex polygon (vertices are numbered starting at zero for purposes of this computation) and is . The interpretation of the sign of this value is controlled with
Setting dir to CCW (corresponding to counter-clockwise orientation of the projected polygon in window coordinates) indicates that if , then the color of each vertex of the polygon becomes the back color computed for that vertex while if a > 0, then the front color is selected. If dir is CW, then a is replaced by -a in the above inequalities. This requires one bit of state; initially, it indicates CCW.
COPYRIGHT © 1994, 1995 Silicon Graphics, Inc. All Rights Reserved. Trademark Information


IN THE HIGH COURT OF JHARKHAND AT RANCHI A.B. A. No. 4332 of 2016 Pancham Mahto ...... Petitioner Versus The State of Jharkhand ...... Opposite Party ---------
CORAM: HON'BLE MR. JUSTICE ANANT BIJAY SINGH
---------
For the Petitioner : Mr. Dhirendra Kr. Prasad, Advocate For the State : A.P.P. --------- 03/Dated: 27/10/2016
Learned counsel for the petitioner seeks permission to make necessary correction in para 13 of the bail application.
Permission accorded.
Heard learned counsel for the petitioner and learned counsel for the State.
The petitioner is apprehending his arrest in connection with the case registered under Sections 414/34 of the Indian Penal Code, Section 30(ii) of the Coal Mines Act and Section 33 of the Indian Forest Act and Section 21 of the M.M.D.R. Act and Section 21/37 of the Air Pollution Act and Section 25/26/44 of the Water Pollution and Control Act.
Learned counsel for the petitioner has submitted that petitioner is quite innocent and have committed no offence.
Learned A.P.P has opposed the prayer for anticipatory bail. In the fact and circumstances of the case, the above named petitioner is directed to surrender in the Court below within four weeks from the date of this order and in the event of his arrest or surrender the Court below shall enlarge the above named petitioner on bail on furnishing bail bond of Rs. 10,000/- (Rupees ten thousand) with two sureties of the like amount each to the satisfaction of Judicial Magistrate, 1st Class, Hazaribag in connection with Barkagaon P.S. Case No. 67 of 2016 corresponding to G.R. No. 675 of 2016, subject to the conditions as laid down under Section 438(2) of the Cr.P.C, and subject to the further condition that on the date of surrender petitioner shall deposit Rs. 1,500/- in the trial Court.
The aforesaid deposit is subject to the result of the case.
(Anant Bijay Singh, J.) Satayendra/


AWS Services
Overview
Get familiar with AWS services
Description
This assingment is going to cover some of the important Amazon Web Services and get you started with elastic cloud and server-less scripting. You will be need to fix errors related to Node application and find your way around an actual Linux server.
Instructions
- Watch the lectures about Cloud
- Create an AWS account ( You can cancel your account after class, you would not be charged for free tier services)
- Go to "Compute" and Launch an EC2 Free Tier instance.
- Select a Free Tier Ubuntu Server 18.04 Amazon Linux Image
- Select t2.micro Free Tier
- Click to Configure Instance Details
- Configure Instance Details
- Change "Auto-assign Public IP" to enable
- Click on add storage and leave the default values
- Click on Review and Launch
- Click on Launch
- Change option to "Create new key pair"
- Type the name of the keypair (anything)
- Click on "Download" (if you loose these you can access to that server and need to start over)
- Click on Launch Instance
- Click on Launch View Instances
- Once the EC2 is changed from "Pending" to "Running" continue
- Find the IPv4 Public IP address and take a screen-shot
- open terminal and access to this ec2
>> ssh -i youkey.pem ec2-user@ipnumber
You will get an error about the wrong permission. Look at the screen-shots to fix this issue.
You will than be able to login to your ec2 after that execute the commands and take scree-shots.
- run command "whoami" and take a screen-shot ( this should be ec2-user)
- run command "pwd" and take a screen-shot (this should be /home/ec2-user)
- run command "sudo yum update type y"
- Follow this page to setup node https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-up-node-on-ec2-instance.html
- Screen-shot the output of the "node -e "console.log('Running Node.js ' + process.version)"
Let's open ports to our Node Server
- Click on "EC2" (Home page)
- Click on "Security Groups"
- Select "launch-wizard-1"
- Click on "Actions"
- Click on "Inbound"
- Click on "Add Rule" and enter Port Range (8080)
- Click on "Save"
Go back to your SSH window in your server
- Create a app.js file
In your terminal window execute vi app.js and hit i (to insert) and write the code given below. Once you are done writing.
To save "shift + :")
Type w! to write the file
again shift + :"
type q to quit
You can repeat this process if you have errors and/or need to edit the document.
"use strict"
var express= require("express");
var request = require("request");
var app=express();
var router = express.Router();
router.get("/",function(req,res){
console.log("Here");
res.send("Hello by Your Name");
})
app.use("/",router);
app.listen("8080");
- run node app.js
- Fix errors if any
- Go to your browser and use the public ip address of the server with the port number http://youpublicip:8080
- Take a screen shot
Rubric
| EC2 Created | 10.00% |
| Succesffull SSH connection | 10.00% |
| Node is running on ec2 | 20.00% |
| Ports are opened correctly | 10.00% |
| Node app functions on ec2 | 20.00% |
| App is availabe on public IP | 20.00% |
| Delivery 6 Video URL of all submissions | 10.00% |
Deliverable
- Create a demo video of all the steps and post it in the text area on FSO Assingment Submission


After self-reflecting a lot, speaking to family members and friends I can say I'm a reserved person, who values the family, friendship, integrity and perseverence. There's one particular situation that really marked me.
When I started working as a receptionist in a Backpacker in Queenstown, life was awesome because I was always meeting different people from all over the world, doing bungee jumps, skydiving, white water rafting, etc, all for free, besides living in one of the most beautiful cities in the world. I wasn't making enough money, though, to do anything else other than pay rent and food. One day, on the way back home after work, I decided to take the "scenic route" and instead of walking on the street I took the track that goes along Lake Wakatipu. It was almost night time and I didn't see anyone else all the way. When I was getting close home I found a wallet on the ground with a lot of money on it (enough to hold me on for a couple of months). I took the wallet and won't lie I thought about keeping it without saying anything, but I didn't. By the time I got home all I could think of was how the person who had lost it was feeling and what that money could be for so I walked all the way back to town and handed in the wallet at the police station. Walking back home again, all I could feel and think of was peace of mind.
I consider myself a perseverant person so when I want something I give everything to get it, doesn't matter how hard it is. I would say that's the main strength I've got to keep on going during this course even though I can easily get distracted when reading or studying,


Money Play. How should Red play 51?
pips Red=101 White=135
(CorrectAnswer= 0/0, ErrorScore= 0)
Woodenly playing 16/10, maintaining the five-point prime, is not the right idea here.
Red wants to pick up the White checker on the 12-pt and he also wants to distribute his checkers to hit any entering White checker, particularly on the ace-point.
16/15, 7/2; 8/2 and 7/6, 7/2 all play to that plan. The rollouts have 16/15, 7/2 shading it by a whisker but all three plays are fine.
Red wants to pick up the White checker on the 12-pt and he also wants to distribute his checkers to hit any entering White checker, particularly on the ace-point.
16/15, 7/2; 8/2 and 7/6, 7/2 all play to that plan. The rollouts have 16/15, 7/2 shading it by a whisker but all three plays are fine.
1. Cubeful 2-ply 16/15 7/2 Eq.: +1.194 0.874 0.483 0.024 - 0.126 0.018 0.000 2. Cubeful 2-ply 8/2 Eq.: +1.190 (-0.004) 0.872 0.484 0.024 - 0.128 0.018 0.000 3. Cubeful 2-ply 7/6 7/2 Eq.: +1.183 (-0.011) 0.870 0.476 0.025 - 0.130 0.015 0.000 4. Cubeful 2-ply 6/1 2/1 Eq.: +1.176 (-0.017) 0.879 0.459 0.020 - 0.121 0.019 0.000 5. Cubeful 2-ply 9/8 7/2 Eq.: +1.123 (-0.071) 0.852 0.475 0.025 - 0.148 0.027 0.001


Description
This demo will showcase how a jPlayer video can be integrated into Popcorn.
jPlayer autommatically selects an HTML5 solution, if available, or uses a Flash solution on browsers that do not natively support MP3, M4A or M4V. (MP4: H.264 / AAC)
Due to the Flash security model, this demo must be run from a web server. Otherwise Firefox, Opera and IE8 will fail.
The jPlayer player is chromeless (hidden controls), but the controls may be added using the standard jPlayer mechanism described at
jplayer.org.
Basically, the skin would be done in the normal manner, but the Popcorn jPlayer player instances jPlayer on your jPlayer DIV instead of the usual commands to instance jPlayer.
Clicking play/pause or seeking in either the video or via custom controls will cause the other to update.
The video/audio can be specified in the HTML source by giving the media url (http://www.domain.com/media/video.m4v) to Popcorn.jplayer.
Alternatively, the video/audio can be specified in the HTML source by giving the SRC Object to Popcorn.jplayer, which contains a media and options Object.
Expected Events
- From 5 to 40 seconds, the footnote 'The video is "Big Buck Bunny", © 2008 Blender Foundation' will appear below 'Footnote Area'.
- From 20 to 40 seconds, the a flickr stream of 8 images tagged 'edinburgh' will appear below 'Flickr Area'.
- From 20 to 45 seconds, tweets tagged @jPlayerOrg will appear below 'Twitter Area'.
- From 5 to 15 seconds, the Mozilla Drumbeat logo will appear below 'Image Area'.
- From 5 to 15 seconds, the subtitle 'This is overlaid on top of the video. You can hightlight it!' will appear in front of the video.
- From 0 to 20 seconds, blogs from 'http://happyworm.com/blog/feed/rss/' will appear below 'Google Feed Area'.
- From 0 to 15 seconds, the site 'http://webmademovies.org/' will appear below 'Web Page Area'.


Fit
View
2D
3D
Import
Upload from disk
Get from URL
Export
Lat/Long:
Ruler:
m
°
Alert
Select the file to upload:
Select the URL to import:
Import
Choose the format to save in:
json
svg
csv
Download
Reference point:
Lat
Lon
X
Y
Scaling:


Welcome to coding quiz!
The button on the left will start a new quiz after a count-down
Answer questions by clicking the small button


class Bundler::Persistent::Net::HTTP::Persistent
Persistent
connections for Net::HTTP
Bundler::Persistent::Net::HTTP::Persistent
maintains persistent connections across all the servers you wish to talk to. For each host:port you communicate with a single persistent connection is created.
Connections will be shared across threads through a connection pool to increase reuse of connections.
You can shut down any remaining HTTP connections when done by calling shutdown
.
Example:
require 'bundler/vendor/net-http-persistent/lib/net/http/persistent' uri = Bundler::URI 'http://example.com/awesome/web/service' http = Bundler::Persistent::Net::HTTP::Persistent.new # perform a GET response = http.request uri # or get = Net::HTTP::Get.new uri.request_uri response = http.request get # create a POST post_uri = uri + 'create' post = Net::HTTP::Post.new post_uri.path post.set_form_data 'some' => 'cool data' # perform the POST, the Bundler::URI is always required response http.request post_uri, post
Note that for GET, HEAD and other requests that do not have a body you want to use Bundler::URI#request_uri not Bundler::URI#path. The request_uri contains the query params which are sent in the body for other requests.
TLS/SSL¶ ↑
TLS connections are automatically created depending upon the scheme of the Bundler::URI
. TLS connections are automatically verified against the default certificate store for your computer. You can override this by changing verify_mode
or by specifying an alternate cert_store.
Here are the TLS settings, see the individual methods for documentation:
certificate
-
This client's certificate
ca_file
-
The certificate-authorities
ca_path
-
Directory with certificate-authorities
cert_store
-
An SSL certificate store
ciphers
-
List of SSl ciphers allowed
private_key
-
The client's SSL private key
reuse_ssl_sessions
-
Reuse a previously opened SSL session for a new connection
ssl_timeout
-
Session lifetime
ssl_version
-
Which specific SSL version to use
verify_callback
-
For server certificate verification
verify_depth
-
Depth of certificate verification
verify_mode
-
How connections should be verified
Proxies¶ ↑
A proxy can be set through proxy=
or at initialization time by providing a second argument to ::new
. The proxy may be the Bundler::URI
of the proxy server or :ENV
which will consult environment variables.
See proxy=
and proxy_from_env
for details.
Headers¶ ↑
Headers may be specified for use in every request. headers
are appended to any headers on the request. override_headers
replace existing headers on the request.
The difference between the two can be seen in setting the User-Agent. Using http.headers['User-Agent'] = 'MyUserAgent'
will send “Ruby, MyUserAgent” while http.override_headers['User-Agent'] = 'MyUserAgent'
will send “MyUserAgent”.
Tuning¶ ↑
Segregation¶ ↑
Each Bundler::Persistent::Net::HTTP::Persistent
instance has its own pool of connections. There is no sharing with other instances (as was true in earlier versions).
Idle Timeout¶ ↑
If a connection hasn't been used for this number of seconds it will automatically be reset upon the next use to avoid attempting to send to a closed connection. The default value is 5 seconds. nil means no timeout. Set through idle_timeout
.
Reducing this value may help avoid the “too many connection resets” error when sending non-idempotent requests while increasing this value will cause fewer round-trips.
Read Timeout¶ ↑
The amount of time allowed between reading two chunks from the socket. Set through read_timeout
Max Requests¶ ↑
The number of requests that should be made before opening a new connection. Typically many keep-alive capable servers tune this to 100 or less, so the 101st request will fail with ECONNRESET. If unset (default), this value has no effect, if set, connections will be reset on the request after max_requests.
Open Timeout¶ ↑
The amount of time to wait for a connection to be opened. Set through open_timeout
.
Socket Options¶ ↑
Socket options may be set on newly-created connections. See socket_options
for details.
Connection
Termination¶ ↑
If you are done using the Bundler::Persistent::Net::HTTP::Persistent
instance you may shut down all the connections in the current thread with shutdown
. This is not recommended for normal use, it should only be used when it will be several minutes before you make another HTTP request.
If you are using multiple threads, call shutdown
in each thread when the thread is done making requests. If you don't call shutdown, that's OK. Ruby will automatically garbage collect and shutdown your HTTP connections when the thread terminates.
Constants
- DEFAULT_POOL_SIZE
- VERSION
The version of
Bundler::Persistent::Net::HTTP::Persistent
you are using
Attributes
An SSL certificate authority. Setting this will set verify_mode
to VERIFY_PEER.
A directory of SSL certificates to be used as certificate authorities. Setting this will set verify_mode
to VERIFY_PEER.
This client's OpenSSL::X509::Certificate
An SSL certificate store. Setting this will override the default certificate store. See verify_mode
for more information.
This client's OpenSSL::X509::Certificate
The ciphers allowed for SSL connections
Sends debug_output
to this IO via Net::HTTP#set_debug_output.
Never use this method in production code, it causes a serious security hole.
Headers that are added to every request using Net::HTTP#add_field
Maps host:port to an HTTP version. This allows us to enable version specific features.
Maximum time an unused connection can remain idle before being automatically closed.
The value sent in the Keep-Alive header. Defaults to 30. Not needed for HTTP/1.1 servers.
This may not work correctly for HTTP/1.0 servers
This method may be removed in a future version as RFC 2616 does not require this header.
This client's SSL private key
Maximum number of requests on a connection before it is considered expired and automatically closed.
Number of retries to perform if a request fails.
See also max_retries=
, Net::HTTP#max_retries=.
Maximum SSL version to use, e.g. :TLS1_2
By default, the version will be negotiated automatically between client and server. Ruby 2.5 and newer only.
Minimum SSL version to use, e.g. :TLS1_1
By default, the version will be negotiated automatically between client and server. Ruby 2.5 and newer only.
The name for this collection of persistent connections.
List of host suffixes which will not be proxied
Seconds to wait until a connection is opened. See Net::HTTP#open_timeout
Headers that are added to every request using Net::HTTP#[]=
This client's SSL private key
The URL through which requests will be proxied
Seconds to wait until reading one block. See Net::HTTP#read_timeout
By default SSL sessions are reused to avoid extra SSL handshakes. Set this to false if you have problems communicating with an HTTPS server like:
SSL_connect [...] read finished A: unexpected message (OpenSSL::SSL::SSLError)
An array of options for Socket#setsockopt.
By default the TCP_NODELAY option is set on sockets.
To set additional options append them to this array:
http.socket_options << [Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1]
SSL session lifetime
SSL version to use.
By default, the version will be negotiated automatically between client and server. Ruby 1.9 and newer only. Deprecated since Ruby 2.5.
Sets the depth of SSL certificate verification
HTTPS verify mode. Defaults to OpenSSL::SSL::VERIFY_PEER which verifies the server certificate.
If no ca_file
, ca_path
or cert_store
is set the default system certificate store is used.
You can use verify_mode
to override any default values.
Seconds to wait until writing one block. See Net::HTTP#write_timeout
Public Class Methods
Use this method to detect the idle timeout of the host at uri
. The value returned can be used to configure idle_timeout
. max
controls the maximum idle timeout to detect.
After
Idle timeout detection is performed by creating a connection then performing a HEAD request in a loop until the connection terminates waiting one additional second per loop.
NOTE: This may not work on ruby > 1.9.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 199 def self.detect_idle_timeout uri, max = 10 uri = Bundler::URI uri unless Bundler::URI::Generic === uri uri += '/' req = Net::HTTP::Head.new uri.request_uri http = new 'net-http-persistent detect_idle_timeout' http.connection_for uri do |connection| sleep_time = 0 http = connection.http loop do response = http.request req $stderr.puts "HEAD #{uri} => #{response.code}" if $DEBUG unless Net::HTTPOK === response then raise Error, "bad response code #{response.code} detecting idle timeout" end break if sleep_time >= max sleep_time += 1 $stderr.puts "sleeping #{sleep_time}" if $DEBUG sleep sleep_time end end rescue # ignore StandardErrors, we've probably found the idle timeout. ensure return sleep_time unless $! end
Creates a new Bundler::Persistent::Net::HTTP::Persistent
.
 Your library name should be good enough, but this otherwise has no purpose.
proxy
may be set to a Bundler::URI::HTTP
or :ENV to pick up proxy options from the environment. See proxy_from_env
for details.
In order to use a Bundler::URI
for the proxy you may need to do some extra work beyond Bundler::URI
parsing if the proxy requires a password:
proxy = Bundler::URI 'http://proxy.example' proxy.user = 'AzureDiamond' proxy.password = 'hunter2'
Set pool_size
to limit the maximum number of connections allowed. Defaults to 1/4 the number of allowed file handles or 256 if your OS does not support a limit on allowed file handles. You can have no more than this many threads with active HTTP transactions.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 473 def initialize name: nil, proxy: nil, pool_size: DEFAULT_POOL_SIZE @name = name @debug_output = nil @proxy_uri = nil @no_proxy = [] @headers = {} @override_headers = {} @http_versions = {} @keep_alive = 30 @open_timeout = nil @read_timeout = nil @write_timeout = nil @idle_timeout = 5 @max_requests = nil @max_retries = 1 @socket_options = [] @ssl_generation = 0 # incremented when SSL session variables change @socket_options << [Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1] if Socket.const_defined? :TCP_NODELAY @pool = Bundler::Persistent::Net::HTTP::Persistent::Pool.new size: pool_size do |http_args| Bundler::Persistent::Net::HTTP::Persistent::Connection.new Net::HTTP, http_args, @ssl_generation end @certificate = nil @ca_file = nil @ca_path = nil @ciphers = nil @private_key = nil @ssl_timeout = nil @ssl_version = nil @min_version = nil @max_version = nil @verify_callback = nil @verify_depth = nil @verify_mode = nil @cert_store = nil @generation = 0 # incremented when proxy Bundler::URI changes if HAVE_OPENSSL then @verify_mode = OpenSSL::SSL::VERIFY_PEER @reuse_ssl_sessions = OpenSSL::SSL.const_defined? :Session end self.proxy = proxy if proxy end
Public Instance Methods
Sets the SSL certificate authority file.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 538 def ca_file= file @ca_file = file reconnect_ssl end
Sets the SSL certificate authority path.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 547 def ca_path= path @ca_path = path reconnect_ssl end
Overrides the default SSL certificate store used for verifying connections.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 557 def cert_store= store @cert_store = store reconnect_ssl end
Sets this client's OpenSSL::X509::Certificate
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 526 def certificate= certificate @certificate = certificate reconnect_ssl end
The ciphers allowed for SSL connections
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 566 def ciphers= ciphers @ciphers = ciphers reconnect_ssl end
Creates a new connection for uri
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 575 def connection_for uri use_ssl = uri.scheme.downcase == 'https' net_http_args = [uri.hostname, uri.port] # I'm unsure if uri.host or uri.hostname should be checked against # the proxy bypass list. if @proxy_uri and not proxy_bypass? uri.host, uri.port then net_http_args.concat @proxy_args else net_http_args.concat [nil, nil, nil, nil] end connection = @pool.checkout net_http_args http = connection.http connection.ressl @ssl_generation if connection.ssl_generation != @ssl_generation if not http.started? then ssl http if use_ssl start http elsif expired? connection then reset connection end http.keep_alive_timeout = @idle_timeout if @idle_timeout http.max_retries = @max_retries if http.respond_to?(:max_retries=) http.read_timeout = @read_timeout if @read_timeout http.write_timeout = @write_timeout if @write_timeout && http.respond_to?(:write_timeout=) return yield connection rescue Errno::ECONNREFUSED address = http.proxy_address || http.address port = http.proxy_port || http.port raise Error, "connection refused: #{address}:#{port}" rescue Errno::EHOSTDOWN address = http.proxy_address || http.address port = http.proxy_port || http.port raise Error, "host down: #{address}:#{port}" ensure @pool.checkin net_http_args end
CGI::escape wrapper
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 626 def escape str CGI.escape str if str end
Returns true if the connection should be reset due to an idle timeout, or maximum request count, false otherwise.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 642 def expired? connection return true if @max_requests && connection.requests >= @max_requests return false unless @idle_timeout return true if @idle_timeout.zero? Time.now - connection.last_use > @idle_timeout end
Finishes the Net::HTTP connection
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 671 def finish connection connection.finish connection.http.instance_variable_set :@last_communicated, nil connection.http.instance_variable_set :@ssl_session, nil unless @reuse_ssl_sessions end
Returns the HTTP protocol version for uri
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 682 def http_version uri @http_versions["#{uri.hostname}:#{uri.port}"] end
Set the maximum number of retries for a request.
Defaults to one retry.
Set this to 0 to disable retries.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 700 def max_retries= retries retries = retries.to_int raise ArgumentError, "max_retries must be positive" if retries < 0 @max_retries = retries reconnect end
maximum SSL version to use
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1033 def max_version= max_version @max_version = max_version reconnect_ssl end
Minimum SSL version to use
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1024 def min_version= min_version @min_version = min_version reconnect_ssl end
Adds “http://” to the String uri
if it is missing.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 689 def normalize_uri uri (uri =~ /^https?:/) ? uri : "http://#{uri}" end
Sets this client's SSL private key
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 713 def private_key= key @private_key = key reconnect_ssl end
Sets the proxy server. The proxy
may be the Bundler::URI
of the proxy server, the symbol :ENV
which will read the proxy from the environment or nil to disable use of a proxy. See proxy_from_env
for details on setting the proxy from the environment.
If the proxy Bundler::URI
is set after requests have been made, the next request will shut-down and re-open all connections.
The no_proxy
query parameter can be used to specify hosts which shouldn't be reached via proxy; if set it should be a comma separated list of hostname suffixes, optionally with :port
appended, for example example.com,some.host:8080
.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 736 def proxy= proxy @proxy_uri = case proxy when :ENV then proxy_from_env when Bundler::URI::HTTP then proxy when nil then # ignore else raise ArgumentError, 'proxy must be :ENV or a Bundler::URI::HTTP' end @no_proxy.clear if @proxy_uri then @proxy_args = [ @proxy_uri.hostname, @proxy_uri.port, unescape(@proxy_uri.user), unescape(@proxy_uri.password), ] @proxy_connection_id = [nil<EMAIL_ADDRESS>':' if @proxy_uri.query then @no_proxy = CGI.parse(@proxy_uri.query)['no_proxy'].join(',').downcase.split(',').map { |x| x.strip }.reject { |x| x.empty? } end end reconnect reconnect_ssl end
Returns true when proxy should by bypassed for host.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 810 def proxy_bypass? host, port host = host.downcase host_port = [host, port].join ':' @no_proxy.each do |name| return true if host[-name.length, name.length] == name or host_port[-name.length, name.length] == name end false end
Creates a Bundler::URI
for an HTTP proxy server from ENV variables.
If HTTP_PROXY
is set a proxy will be returned.
If HTTP_PROXY_USER
or HTTP_PROXY_PASS
are set the Bundler::URI
is given the indicated user and password unless HTTP_PROXY contains either of these in the Bundler::URI
.
The NO_PROXY
ENV variable can be used to specify hosts which shouldn't be reached via proxy; if set it should be a comma separated list of hostname suffixes, optionally with :port
appended, for example example.com,some.host:8080
. When set to *
no proxy will be returned.
For Windows users, lowercase ENV variables are preferred over uppercase ENV variables.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 783 def proxy_from_env env_proxy = ENV['http_proxy'] || ENV['HTTP_PROXY'] return nil if env_proxy.nil? or env_proxy.empty? uri = Bundler::URI normalize_uri env_proxy env_no_proxy = ENV['no_proxy'] || ENV['NO_PROXY'] # '*' is special case for always bypass return nil if env_no_proxy == '*' if env_no_proxy then uri.query = "no_proxy=#{escape(env_no_proxy)}" end unless uri.user or uri.password then uri.user = escape ENV['http_proxy_user'] || ENV['HTTP_PROXY_USER'] uri.password = escape ENV['http_proxy_pass'] || ENV['HTTP_PROXY_PASS'] end uri end
Forces reconnection of all HTTP connections, including TLS/SSL connections.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 826 def reconnect @generation += 1 end
Forces reconnection of only TLS/SSL connections.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 833 def reconnect_ssl @ssl_generation += 1 end
Makes a request on uri
. If req
is nil a Net::HTTP::Get is performed against uri
.
If a block is passed request
behaves like Net::HTTP#request (the body of the response will not have been read).
req
must be a Net::HTTPGenericRequest subclass (see Net::HTTP for a list).
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 865 def request uri, req = nil, &block uri = Bundler::URI uri req = request_setup req || uri response = nil connection_for uri do |connection| http = connection.http begin connection.requests += 1 response = http.request req, &block if req.connection_close? or (response.http_version <= '1.0' and not response.connection_keep_alive?) or response.connection_close? then finish connection end rescue Exception # make sure to close the connection when it was interrupted finish connection raise ensure connection.last_use = Time.now end end @http_versions["#{uri.hostname}:#{uri.port}"] ||= response.http_version response end
Finishes then restarts the Net::HTTP connection
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 840 def reset connection http = connection.http finish connection start http rescue Errno::ECONNREFUSED e = Error.new "connection refused: #{http.address}:#{http.port}" e.set_backtrace $@ raise e rescue Errno::EHOSTDOWN e = Error.new "host down: #{http.address}:#{http.port}" e.set_backtrace $@ raise e end
Shuts down all connections
NOTE: Calling shutdown for can be dangerous!
If any thread is still using a connection it may cause an error! Call shutdown
when you are completely done making requests!
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 935 def shutdown @pool.shutdown { |http| http.finish } end
Enables SSL on connection
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 942 def ssl connection connection.use_ssl = true connection.ciphers = @ciphers if @ciphers connection.ssl_timeout = @ssl_timeout if @ssl_timeout connection.ssl_version = @ssl_version if @ssl_version connection.min_version = @min_version if @min_version connection.max_version = @max_version if @max_version connection.verify_depth = @verify_depth connection.verify_mode = @verify_mode if OpenSSL::SSL::VERIFY_PEER == OpenSSL::SSL::VERIFY_NONE and not Object.const_defined?(:I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG) then warn <<-WARNING !!!SECURITY WARNING!!! The SSL HTTP connection to: #{connection.address}:#{connection.port} !!!MAY NOT BE VERIFIED!!! On your platform your OpenSSL implementation is broken. There is no difference between the values of VERIFY_NONE and VERIFY_PEER. This means that attempting to verify the security of SSL connections may not work. This exposes you to man-in-the-middle exploits, snooping on the contents of your connection and other dangers to the security of your data. To disable this warning define the following constant at top-level in your application: I_KNOW_THAT_OPENSSL_VERIFY_PEER_EQUALS_VERIFY_NONE_IS_WRONG = nil WARNING end connection.ca_file = @ca_file if @ca_file connection.ca_path = @ca_path if @ca_path if @ca_file or @ca_path then connection.verify_mode = OpenSSL::SSL::VERIFY_PEER connection.verify_callback = @verify_callback if @verify_callback end if @certificate and @private_key then connection.cert = @certificate connection.key = @private_key end connection.cert_store = if @cert_store then @cert_store else store = OpenSSL::X509::Store.new store.set_default_paths store end end
SSL session lifetime
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1006 def ssl_timeout= ssl_timeout @ssl_timeout = ssl_timeout reconnect_ssl end
SSL version to use
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1015 def ssl_version= ssl_version @ssl_version = ssl_version reconnect_ssl end
Starts the Net::HTTP connection
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 653 def start http http.set_debug_output @debug_output if @debug_output http.open_timeout = @open_timeout if @open_timeout http.start socket = http.instance_variable_get :@socket if socket then # for fakeweb @socket_options.each do |option| socket.io.setsockopt(*option) end end end
CGI::unescape wrapper
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 633 def unescape str CGI.unescape str if str end
SSL verification callback.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1064 def verify_callback= callback @verify_callback = callback reconnect_ssl end
Sets the depth of SSL certificate verification
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1042 def verify_depth= verify_depth @verify_depth = verify_depth reconnect_ssl end
Sets the HTTPS verify mode. Defaults to OpenSSL::SSL::VERIFY_PEER.
Setting this to VERIFY_NONE is a VERY BAD IDEA and should NEVER be used. Securely transfer the correct certificate and update the default certificate store or set the ca file instead.
# File lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb, line 1055 def verify_mode= verify_mode @verify_mode = verify_mode reconnect_ssl end


Tidy the results of a Latent Dirichlet Allocation or Correlated Topic Model.
# S3 method for LDA tidy(x, matrix = c("beta", "gamma"), log = FALSE, ...) # S3 method for CTM tidy(x, matrix = c("beta", "gamma"), log = FALSE, ...) # S3 method for LDA augment(x, data, ...) # S3 method for CTM augment(x, data, ...) # S3 method for LDA glance(x, ...) # S3 method for CTM glance(x, ...)
| x | An LDA or CTM (or LDA_VEM/CTA_VEM) object from the topicmodels package |
|---|---|
| matrix | Whether to tidy the beta (per-term-per-topic, default) or gamma (per-document-per-topic) matrix |
| log | Whether beta/gamma should be on a log scale, default FALSE |
| ... | Extra arguments, not used |
| data | For |
tidy
returns a tidied version of either the beta or gamma matrix.
If matrix == "beta"
(default), returns a table with one row per topic and term,
with columns
Topic, as an integer
Term
Probability of a term generated from a topic according to the multinomial model
If matrix == "gamma"
, returns a table with one row per topic and document,
with columns
Topic, as an integer
Document name or ID
Probability of topic given document
augment
returns a table with one row per original
document-term pair, such as is returned by tdm_tidiers:
Name of document (if present), or index
Term
Topic assignment
If the data
argument is provided, any columns in the original
data are included, combined based on the document
and term
columns.
glance
always returns a one-row table, with columns
Number of iterations used
Number of terms in the model
If an LDA_VEM, the parameter of the Dirichlet distribution for topics over documents
if (requireNamespace("topicmodels", quietly = TRUE)) { set.seed(2016) library(dplyr) library(topicmodels) data("AssociatedPress", package = "topicmodels") ap <- AssociatedPress[1:100, ] lda <- LDA(ap, control = list(alpha = 0.1), k = 4) # get term distribution within each topic td_lda <- tidy(lda) td_lda library(ggplot2) # visualize the top terms within each topic td_lda_filtered <- td_lda %>% filter(beta > .004) %>% mutate(term = reorder(term, beta)) ggplot(td_lda_filtered, aes(term, beta)) + geom_bar(stat = "identity") + facet_wrap(~ topic, scales = "free") + theme(axis.text.x = element_text(angle = 90, size = 15)) # get classification of each document td_lda_docs <- tidy(lda, matrix = "gamma") td_lda_docs doc_classes <- td_lda_docs %>% group_by(document) %>% top_n(1) %>% ungroup() doc_classes # which were we most uncertain about? doc_classes %>% arrange(gamma) }#>#> # A tibble: 100 x 3 #> document topic gamma #> <int> <int> <dbl> #> 1 40 2 0.546 #> 2 14 1 0.553 #> 3 89 3 0.579 #> 4 32 1 0.591 #> 5 9 1 0.633 #> 6 42 4 0.647 #> 7 56 2 0.660 #> 8 28 2 0.676 #> 9 75 2 0.717 #> 10 98 4 0.725 #> # … with 90 more rows


Start
[no data]
Red
Green
Blue
Start capturing
Stop capturing
| Build time:
@GIT_DATE@
| Commit hash:
@GIT_SHA1@
| Commit subject:
@GIT_COMMIT_SUBJECT@
|
Source Code
|


We like to consult, research, prototype and develop. If you have a weird idea - let's work together!
We like to donate our code to open source community as much as we can. Take a look at our portfolio below - maybe you will find something fascinating.
Unlimited free file hosting using GitHub repo storage (yes, we are already banned).
We still have a lot of new weird ideas for our old and new projects. We need you to join our team of dreamers and professionals.
I'm Quite Interested


heat mitigation temperature stress wet-bulb global INDC exposure days delayed Figure emission number regions scenario continuous South climate scenarios population {border-style Asia, px;} #ffffff;} humidity scenario, models Paris human increase target temperatures approximately change exposed hours period warming West eastern efforts scenarios, used China, Climate areas high year Agreement emissions values China Global different exceed extreme significantly (the East based level people predicted reduction CMIP Group Heat United data maximum mean px; range scenario) targets {font-family (under average calculated changes compared font-size future health order reduced regional response risk study threshold °C. Asia Clim. GCMs Health Table Twmax, collab: dangerous double; ensemble grid heat-stress hotspot humidity, level. lines pathways quantile; regions, stress. (compared .–. Amazon Australia, Chang. INDCs Model Nat. North Public Report States, air annual border-color border-width current daily end estimated expected gas greenhouse intensity labor magnitude median national region scenario; scenarios. solid; studies total {font-style {font-weight °C, ./nclimate Africa, Amazon, Amazon. Change China; Compared Environ. However, IPCC India, No. Rev. Sci. Science Supplementary UNFCCC Wang actions affect affected baseline body bold caused century, continued control courier; duration effect effects em; followed highest humid impact increases lead low-latitude models. multiple northern obtained person-days pledged predict production pt;} reach relative represent scenario. simulated stress, time using various year,