Wikipedia releases clickstream data:
This allows us to see how people move betweeen Wikipedia pages.
Wikipedia also has a project specifically designed to improve Medical and Health content on Wikipedia WikiProject Medicine
We can see the lists of pages that WikiProject Medicine concerns itself with by using the Wikipedia API Category to Page API
Note that there are multiple different categories. I think it is possible to replicate the WikiProject Medicine priority grid which is the last table on this page: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Assessment
I also believe that it is possible to replicate the structure found here: https://en.wikipedia.org/wiki/Wikipedia:Vital_articles/Level/5/Biological_and_health_sciences/Health
We can also reverse the search and show all of the categories of the pages that are inside the Wikiproject Medicine project, using the Wikipedia Page API
The datasets of pages, categories and clickstream are massive datasets. This dataset is a narrowing of these dataset to solve a very specific problem.
Wikiproject medicine has had huge successes disseminating its information into other language in an impactful manner
https://www.nytimes.com/2014/10/27/business/media/wikipedia-is-emerging-as-trusted-internet-source-for-information-on-ebola-.html
One of this strategies is to use Wikipedias "internet in a box" program https://meta.wikimedia.org/wiki/Internet-in-a-Box So provide copy of the entire corpus of health information, in local languages all over the world, even in places where Internet is absent or very expensive.
But to do this, they need to know presicely what content to provide.
Using this dataset, we can answer the question "What pages should wikiproject medicine include" a question that can be answered by actual usage patterns. This might help WikiProject medicine to know, for instance, which Biology articles are vital to include. Further, it might be possible to advise the WikiProject Medicine editors on methods to improve their understanding of articles that are "central" from users priority but are not a high priority for the project.
Of course, I also think it is important to simply "see" the wikipedia wikiproject medicine article as a flow-graph. It is entirely possible that there might be insights that a wikipedian might be immediately be able to see in the graph that might be very difficult for someone else to understand.
At a smaller scale I think it might be interesting to create interactive graphs that reveal the "neighborhood" around a given article, especially and article that has already reached FA status.
We did not include it, or pre-filter the data, but I suspect that the page-view data (which shows how often a page was viewed) might be a good overlay for this. https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews
Also, this might be a good job for Gephi.. https://gephi.org/


Apprentice David Jenkins was deleted
Apprentices added
All your ULNs were successfully added
View your cohort
25
Apprentices
10
Incomplete records
£29,002
Total cost
Employer:
Acme Coventry and Birmingham Ltd
Cohort reference:
98HGS3F
Message sent:
There is space here for a short note. The user can enter a message for the other side to look at.
25 Mechatronics Engineers
Level: 4
Training code: 3246
| Name | ULN | Date of birth | Training dates | Cost | |
|---|---|---|---|---|---|
| Rob Edwards |<PHONE_NUMBER> | 03 Jun 1997 | 11/18 to -- | £2,000 | View |
| Taylor Lucas |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Prahlad Naik |<PHONE_NUMBER> | 03 Jun 1997 | 11/18 to 11/20 | £2,000 | View |
| Jeff Bloggs |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Mahershalalhashbaz Tollemache-Tollemache de Orellana Plantagenet |<PHONE_NUMBER>3 | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Melissa Shaw |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Brendan Hamazaki |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Gary Barton |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| Tommy Barger |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |
| William Dewitt |<PHONE_NUMBER> | 10 Jun 1999 | 11/18 to 11/20 | £2,000 | View |


The HTML source of this page may be viewed at index.html
, and the JavaScript that drives the following demos can be found in demos.js
.
Each element that you place inside of the component will be wrapped in its own card of appropriate width. The height if the cards should be constant, and defined by how you style the elements you put in the component. By default, the <vue-swipe-cards>
component assigns card widths to be the same as their parent element.
If you would like your full-width slider to start on a card other than index 0, you may set the :startIndex="1"
prop to a positive, non-zero number. The slider will not animate to this card position on mount, but will start in that state.
Partial card widths are practical for illustrating to users that there is more content in this category to see if they scroll or swipe horizontally.
If writing your component template in Markup, you would use the attribute :num-cards-visible="2.5"
to scale the cards such that 2.5 cards are visible within the width of the parent element.
If writing your component template in String Templates, you would use the attribute :numCardsVisible="3.5"
to scale the cards such that 3.5 cards are visible within the width of the parent element.
This component makes use of a touch handling mixin which other components may benefit from using. You may add it to your component by means of mixins: [touchHandlingMixin]
, then adding @touchstart="handleTouchStart" @touchmove="handleTouchMove" @touchend="handleTouchEnd"
event listeners to the desired object.
The above event handlers will call locally defined methods of dragStart, dragMove, and dragEnd
. Parameters provided are point
and event
. Coordinates of the point are localized to the coordinates of the interior of the element the listeners are bound to.


URL-Shortener-Microservice
User Storie:
1) I can pass a URL as a parameter and I will receive a shortened URL in the JSON response.
2) If I pass an invalid URL that doesn't follow the valid http://www.example.com format, the JSON response will contain an error instead.
3) When I visit that shortened URL, it will redirect me to my original link.
Place your URL above and press Shorten!
Example output:
{"originalUrl":"www.google.com","shorterUrl":"35848","_id":"59ac7f358da71c395dced436"}
Usage:
https://urlshortjh.glitch.me/70861
Will redirect to:


Behemoth Leader's Corpse:
(A glittering, bead-shaped object was found while searching the corpse of the Behemoth Leader. This must be the Behemoth Leader's Essence!)


Here is a simple example of how error gets magnified that also illustrates rigor: Suppose I want to calculate the volume of a large cubic concrete tank. Well, that’s pretty easy from a math perspective. The volume of a cube is the length of one side cubed, or \(V = L^3\text{,}\) where V is volume and L is length. So, I just have to measure the length of a side. I take a tape measure and measure the side. Let’s say I measure it at 36 inches. There is an inherent inaccuracy to the tape measure; let’s say it is \(/pm 1/8\) in. Oh, and did I make sure that the tape measure was completely straight and rigid from one side to the other? Maybe not. Maybe there’s a 1-inch error because of that. Or maybe I stretched my arms so far, I really only “eyeballed” the length.
in-context


This is some text inside of a paragraph.
This is some other text in a paragraph that is not special :)


This checks that the code is properly initialized when it sets up. Three different models are computed twice. The punch results output goes to file1.txt (the first set) and file2.txt (the second). The main output goes to comp4a.out and comp4b.out. These pairs of files should be exactly the same. If they are not then a variable is not properly initialized when the code is set up.
last modified 19 July, 2008


Before adding a new server, please ensure that it has been provisioned
with a suitable username for Raxak Protect to use, and the public key for
Raxak Protect has been installed.
For more information about the server enrollment process, see
Help link


Droptune
Sign in with Twitter
Latest Albums
Upcoming
Latest Videos
Artists
Charts
Users
Latest Releases
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist
Album Title
Artist


Rock, Paper, Scissors
Your Name
Rock
Paper
Scissors
0 Wins
Rock beats Scissors
Name Wins!
Opponent Name
Rock
Paper
Scissors
0 Wins
Smack talk goes here
Line after line
Line after line
Chat entry field and button


Welcome To HRM!
Hi [User Name],
To get started, you need to verify your email address. Please click on the link below to activate your account:
|
Confirm your email address →
|
|
Didn't create an account?
You can safely ignore this message, as your email was likely entered by mistake.
Thanks,
The Team at HRM.
|


Hello {{name}},
Your employer processed your commute using {{applicationName}}! We found a variety of ways for you to get to work and matched you with {{matches}} other carpoolers.
Click here to get started with your personalized trip plan using {{applicationName}}.


Just use the glyphicon
class followed by the specific icon class.

 Check on the map

| Icon | Class |
|---|---|
glyphicon glyphicon-asterisk |
|
glyphicon glyphicon-plus |
|
glyphicon glyphicon-euro |
|
glyphicon glyphicon-minus |
|
glyphicon glyphicon-cloud |
|
glyphicon glyphicon-envelope |
|
glyphicon glyphicon-pencil |
|
glyphicon glyphicon-glass |
|
glyphicon glyphicon-music |
|
glyphicon glyphicon-search |
|
glyphicon glyphicon-heart |
|
glyphicon glyphicon-star |
|
glyphicon glyphicon-star-empty |
|
glyphicon glyphicon-user |
|
glyphicon glyphicon-film |
|
glyphicon glyphicon-th-large |
|
glyphicon glyphicon-th |
|
glyphicon glyphicon-th-list |
|
glyphicon glyphicon-ok |
|
glyphicon glyphicon-remove |
|
glyphicon glyphicon-zoom-in |
|
glyphicon glyphicon-zoom-out |
|
glyphicon glyphicon-off |
|
glyphicon glyphicon-signal |
|
glyphicon glyphicon-cog |
|
glyphicon glyphicon-trash |
|
glyphicon glyphicon-home |
|
glyphicon glyphicon-file |
|
glyphicon glyphicon-time |
|
glyphicon glyphicon-road |
|
glyphicon glyphicon-download-alt |
|
glyphicon glyphicon-download |
|
glyphicon glyphicon-upload |
|
glyphicon glyphicon-inbox |
|
glyphicon glyphicon-play-circle |
|
glyphicon glyphicon-repeat |
|
glyphicon glyphicon-refresh |
|
glyphicon glyphicon-list-alt |
|
glyphicon glyphicon-lock |
|
glyphicon glyphicon-flag |
|
glyphicon glyphicon-headphones |
|
glyphicon glyphicon-volume-off |
|
glyphicon glyphicon-volume-down |
|
glyphicon glyphicon-volume-up |
|
glyphicon glyphicon-qrcode |
|
glyphicon glyphicon-barcode |
|
glyphicon glyphicon-tag |
|
glyphicon glyphicon-tags |
|
glyphicon glyphicon-book |
|
glyphicon glyphicon-bookmark |
|
glyphicon glyphicon-print |
|
glyphicon glyphicon-camera |
|
glyphicon glyphicon-font |
|
glyphicon glyphicon-bold |
|
glyphicon glyphicon-italic |
|
glyphicon glyphicon-text-height |
|
glyphicon glyphicon-text-width |
|
glyphicon glyphicon-align-left |
|
glyphicon glyphicon-align-center |
|
glyphicon glyphicon-align-right |
|
glyphicon glyphicon-align-justify |
|
glyphicon glyphicon-list |
|
glyphicon glyphicon-indent-left |
|
glyphicon glyphicon-indent-right |
|
glyphicon glyphicon-facetime-video |
|
glyphicon glyphicon-picture |
|
glyphicon glyphicon-map-marker |
|
glyphicon glyphicon-adjust |
|
glyphicon glyphicon-tint |
|
glyphicon glyphicon-edit |
|
glyphicon glyphicon-share |
|
glyphicon glyphicon-check |
|
glyphicon glyphicon-move |
|
glyphicon glyphicon-step-backward |
|
glyphicon glyphicon-fast-backward |
|
glyphicon glyphicon-backward |
|
glyphicon glyphicon-play |
|
glyphicon glyphicon-pause |
|
glyphicon glyphicon-stop |
|
glyphicon glyphicon-forward |
|
glyphicon glyphicon-fast-forward |
|
glyphicon glyphicon-step-forward |
|
glyphicon glyphicon-eject |
|
glyphicon glyphicon-chevron-left |
|
glyphicon glyphicon-chevron-right |
|
glyphicon glyphicon-plus-sign |
|
glyphicon glyphicon-minus-sign |
|
glyphicon glyphicon-remove-sign |
|
glyphicon glyphicon-ok-sign |
|
glyphicon glyphicon-question-sign |
|
glyphicon glyphicon-info-sign |
|
glyphicon glyphicon-screenshot |
|
glyphicon glyphicon-remove-circle |
|
glyphicon glyphicon-ok-circle |
|
glyphicon glyphicon-ban-circle |
|
glyphicon glyphicon-arrow-left |
|
glyphicon glyphicon-arrow-right |
|
glyphicon glyphicon-arrow-up |
|
glyphicon glyphicon-arrow-down |
|
glyphicon glyphicon-share-alt |
|
glyphicon glyphicon-resize-full |
|
glyphicon glyphicon-resize-small |
|
glyphicon glyphicon-exclamation-sign |
|
glyphicon glyphicon-gift |
|
glyphicon glyphicon-leaf |
|
glyphicon glyphicon-fire |
|
glyphicon glyphicon-eye-open |
|
glyphicon glyphicon-eye-close |
|
glyphicon glyphicon-warning-sign |
|
glyphicon glyphicon-plane |
|
glyphicon glyphicon-calendar |
|
glyphicon glyphicon-random |
|
glyphicon glyphicon-comment |
|
glyphicon glyphicon-magnet |
|
glyphicon glyphicon-chevron-up |
|
glyphicon glyphicon-chevron-down |
|
glyphicon glyphicon-retweet |
|
glyphicon glyphicon-shopping-cart |
|
glyphicon glyphicon-folder-close |
|
glyphicon glyphicon-folder-open |
|
glyphicon glyphicon-resize-vertical |
|
glyphicon glyphicon-resize-horizontal |
|
glyphicon glyphicon-hdd |
|
glyphicon glyphicon-bullhorn |
|
glyphicon glyphicon-bell |
|
glyphicon glyphicon-certificate |
|
glyphicon glyphicon-thumbs-up |
|
glyphicon glyphicon-thumbs-down |
|
glyphicon glyphicon-hand-right |
|
glyphicon glyphicon-hand-left |
|
glyphicon glyphicon-hand-up |
|
glyphicon glyphicon-hand-down |
|
glyphicon glyphicon-circle-arrow-right |
|
glyphicon glyphicon-circle-arrow-left |
|
glyphicon glyphicon-circle-arrow-up |
|
glyphicon glyphicon-circle-arrow-down |
|
glyphicon glyphicon-globe |
|
glyphicon glyphicon-wrench |
|
glyphicon glyphicon-tasks |
|
glyphicon glyphicon-filter |
|
glyphicon glyphicon-briefcase |
|
glyphicon glyphicon-fullscreen |
|
glyphicon glyphicon-dashboard |
|
glyphicon glyphicon-paperclip |
|
glyphicon glyphicon-heart-empty |
|
glyphicon glyphicon-link |
|
glyphicon glyphicon-phone |
|
glyphicon glyphicon-pushpin |
|
glyphicon glyphicon-usd |
|
glyphicon glyphicon-gbp |
|
glyphicon glyphicon-sort |
|
glyphicon glyphicon-sort-by-alphabet |
|
glyphicon glyphicon-sort-by-alphabet-alt |
|
glyphicon glyphicon-sort-by-order |
|
glyphicon glyphicon-sort-by-order-alt |
|
glyphicon glyphicon-sort-by-attributes |
|
glyphicon glyphicon-sort-by-attributes-alt |
|
glyphicon glyphicon-unchecked |
|
glyphicon glyphicon-expand |
|
glyphicon glyphicon-collapse-down |
|
glyphicon glyphicon-collapse-up |
|
glyphicon glyphicon-log-in |
|
glyphicon glyphicon-flash |
|
glyphicon glyphicon-log-out |
|
glyphicon glyphicon-new-window |
|
glyphicon glyphicon-record |
|
glyphicon glyphicon-save |
|
glyphicon glyphicon-open |
|
glyphicon glyphicon-saved |
|
glyphicon glyphicon-import |
|
glyphicon glyphicon-export |
|
glyphicon glyphicon-send |
|
glyphicon glyphicon-floppy-disk |
|
glyphicon glyphicon-floppy-saved |
|
glyphicon glyphicon-floppy-remove |
|
glyphicon glyphicon-floppy-save |
|
glyphicon glyphicon-floppy-open |
|
glyphicon glyphicon-credit-card |
|
glyphicon glyphicon-transfer |
|
glyphicon glyphicon-cutlery |
|
glyphicon glyphicon-header |
|
glyphicon glyphicon-compressed |
|
glyphicon glyphicon-earphone |
|
glyphicon glyphicon-phone-alt |
|
glyphicon glyphicon-tower |
|
glyphicon glyphicon-stats |
|
glyphicon glyphicon-sd-video |
|
glyphicon glyphicon-hd-video |
|
glyphicon glyphicon-subtitles |
|
glyphicon glyphicon-sound-stereo |
|
glyphicon glyphicon-sound-dolby |
|
glyphicon glyphicon-sound-5-1 |
|
glyphicon glyphicon-sound-6-1 |
|
glyphicon glyphicon-sound-7-1 |
|
glyphicon glyphicon-copyright-mark |
|
glyphicon glyphicon-registration-mark |
|
glyphicon glyphicon-cloud-download |
|
glyphicon glyphicon-cloud-upload |
|
glyphicon glyphicon-tree-conifer |
|
glyphicon glyphicon-tree-deciduous |


Type Definitions
-
Iterator(key, value, map)
-
A callback invoked during iteration with module:Map.Map#each.
Parameters:
Name Type Description key
object The key of the mapping being iterated upon.
value
* The value of the mapping being iterated upon.
map
Map The map being iterated upon.
- Source:
-
Predicate(key, value, map) → {boolean}
-
A callback invoked during filtering with module:Map.Map#filter. It should returns true if the current entry should be included in the filtered map, false otherwise.
Parameters:
Name Type Description key
object The key of the mapping being iterated upon.
value
* The value of the mapping being iterated upon.
map
Map The map being iterated upon.
- Source:
Returns:
True if the current entry should be included in the filtered map, false otherwise.
- Type
- boolean


In these sections you will see examples of setups for different platforms.
Concourse Pipeline (Cloud Foundry)
| In this chapter, we assume that you deploy your application to Cloud Foundry PaaS. |
The Cloud Pipelines repository contains opinionated Concourse pipeline definitions. Those jobs form an empty pipeline and an opinionated sample pipeline one that you can use in your company.
There following projects take part in the microservice setup
for this demo.
-
Github Analytics: The app that has a REST endpoint and uses messaging — part of our business application.
-
Github Webhook: Project that emits messages that are used by Github Analytics — part of our business application.
-
Eureka: Simple Eureka Server. This is an infrastructure application.
-
Github Analytics Stub Runner Boot: Stub Runner Boot server to be used for tests with Github Analytics and uses Eureka and Messaging. This is an infrastructure application.
Step-by-step
If you want only to run the demo as far as possible by using PCF Dev and Docker Compose, do the following:
Fork Repositories
Four applications compose the pipeline
You need to fork only the following repositories, because only then can you tag and push the tag to the repository:
Start Concourse and Artifactory
You can run Concourse + Artifactory locally. To do so, run the
start.sh
script from this repository. The following listing shows the script:
git clone https://github.com/CloudPipelines/concourse
cd concourse/demo
./setup_docker_compose.sh
./start.sh <IP_ADDRESS>
The setup_docker_compose.sh
script should be run only once, to allow
generation of keys.
The <IP_ADDRESS>
param is an example of an external URL of Concourse
(equal to the Docker-Machine IP in this example).
Then Concourse runs on port 8080
, and Artifactory runs on 8081
.
Deploy the infra JARs to Artifactory
When Artifactory is running, run the tools/deploy-infra.sh
script from this repo.
The following listing shows the script:
git clone https://github.com/CloudPipelines/concourse
cd concourse
./tools/deploy-infra.sh
As a result, both eureka
and stub runner
repos are cloned, built,
and uploaded to Artifactory.
Start PCF Dev
| You can skip this step if you have CF installed and do not want to use PCF Dev. The only thing you have to do is to set up spaces. |
| Servers often run run out of resources at the stage step. If that happens, clear some apps from PCF Dev and continue. |
You have to download and start PCF Dev, as described here.
The default credentials for PCF Dev are as follows:
username: user
password: pass
email: user
org: pcfdev-org
space: pcfdev-space
api: api.local.pcfdev.io
You can start the PCF Dev as follows:
cf dev start
You must create three separate spaces, as follows:
cf login -a https://api.local.pcfdev.io --skip-ssl-validation -u admin -p admin -o pcfdev-org
cf create-space pcfdev-test
cf set-space-role user pcfdev-org pcfdev-test SpaceDeveloper
cf create-space pcfdev-stage
cf set-space-role user pcfdev-org pcfdev-stage SpaceDeveloper
cf create-space pcfdev-prod
cf set-space-role user pcfdev-org pcfdev-prod SpaceDeveloper
You can also run the ./tools/cf-helper.sh setup-spaces
script to create the spaces.
Setup the fly
CLI
If you go to the Concourse website, you should see something resembling the following image:
You can click one of the icons (depending on your OS) to download fly
, which is the Concourse CLI. Once you download that (and maybe, depending on your OS, add it to your PATH) you can run the following command:
fly --version
If fly
is properly installed, it should print out the version.
Set up Your credentials.yml
File
The repository comes with credentials-sample-cf.yml
, which is set up with sample data (mostly credentials) that are applicable for PCF Dev. Copy this file to a new file called credentials.yml
(the file is added to .gitignore
so that you cannot push it with your passwords) and edit it as you wish. For our demo, set up the following:
-
app-url
: URL pointing to your forkedgithub-webhook
repository. -
github-private-key
: Your private key to clone and tag GitHub repositorys. -
repo-with-binaries
: The IP is set to the defaults for Docker Machine. You should update it to point to your setup.
If you do not have a Docker Machine, run th ./whats_my_ip.sh
script to
get an external IP that you can pass to your repo-with-binaries
, instead of the default
Docker Machine IP.
The following table describes the environment variables required by the scripts:
| Property Name | Property Description | Default value |
|---|---|---|
|
The URL to the CF Api for TEST env |
|
|
The URL to the CF Api for STAGE env |
|
|
The URL to the CF Api for PROD env |
|
|
Name of the org for the test env |
|
|
Prefix of the name of the CF space for the test env to which the app name will be appended |
|
|
Name of the org for the stage env |
|
|
Name of the space for the stage env |
|
|
Name of the org for the prod env |
|
|
Name of the space for the prod env |
|
|
URL to repo with the deployed jars |
|
|
The id of server from Maven settings.xml |
|
|
Additional suffix for the route. In a shared environment the default routes can be already taken |
|
|
Additional options you would like to pass to the Maven / Gradle build |
The right column shows the default values for PCF Dev that we set in the credentials-sample-cf.yml
. PAAS_HOSTNAME_UUID
and BUILD_OPTIONS
have no default values.
Build the Pipeline
Log in (for example, for a Concourse instance running at <IP_ADDRESS>
— if you do not provide any value, localhost
is assumed). If you run the login script, it assumes that either fly
is on your PATH
or it is in the same folder as the script. The following example shows how to specify an IP address for the login script:
./login.sh <IP_ADDRESS>
Next, run the command to create the pipeline, as follows:
./set_pipeline.sh
Then you can create a github-webhook
pipeline under the docker
alias, using the provided credentials.yml
file.
You can override these values in exactly that order (for example ./set-pipeline.sh some-project another-target some-other-credentials.yml
)
Run the github-webhook
Pipeline
The following images show the various steps involved in running the github-webhook
pipeline:
Login
main
team
concourse
user and changeme
password
play
button
+
sign to start a new build
Concourse Pipeline (Kubernetes)
| In this chapter, we assume that you deploy your application to Kubernetes PaaS |
The Cloud Pipelines repository contains opinionated Concourse pipeline definitions. Those jobs form an empty pipeline and an opinionated sample pipeline that you can use in your company.
The following projects take part in the microservice setup
for this demo:
-
Github Analytics: The application that has a REST endpoint and uses messaging — part of our business application.
-
Github Webhook: Project that emits messages that are used by Github Analytics — part of our business application.
-
Eureka: Simple Eureka Server. This is an infrastructure application.
-
Github Analytics Stub Runner Boot: Stub Runner Boot server to be used for tests with Github Analytics and uses Eureka and Messaging. This is an infrastructure application.
Step-by-step
If you want only to run the demo as far as possible by using PCF Dev and Docker Compose, do the following:
Fork Repositories
Four applications compose the pipeline:
You need to fork only the following repositories, because only then can you tag and push the tag to the repository:
Concourse in K8S (Kubernetes)
The simplest way to deploy Concourse to K8S is to use Helm.
Once you have Helm installed and your kubectl
is pointing to the
cluster, run the following command to install the Concourse cluster in your K8S cluster:
$ helm install stable/concourse --name concourse
Once the script is done, you should see the following output
1. Concourse can be accessed:
* Within your cluster, at the following DNS name at port 8080:
concourse-web.default.svc.cluster.local
* From outside the cluster, run these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=concourse-web" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://<IP_ADDRESS>:8080 to use Concourse"
kubectl port-forward --namespace default $POD_NAME 8080:8080
2. Login with the following credentials
Username: concourse
Password: concourse
Follow the steps and log in to Concourse under http://<IP_ADDRESS>:8080.
Deploying Artifactory to K8S
You can use Helm also to deploy Artifactory to K8S, as follows:
$ helm install --name artifactory --set artifactory.image.repository=docker.bintray.io/jfrog/artifactory-oss stable/artifactory
After you run this command, you should see the following output:
NOTES:
Congratulations. You have just deployed JFrog Artifactory Pro!
1. Get the Artifactory URL by running these commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of the service by running 'kubectl get svc -w nginx'
export SERVICE_IP=$(kubectl get svc --namespace default nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo http://$SERVICE_IP/
2. Open Artifactory in your browser
Default credential for Artifactory:
user: admin
password: password
Next, you need to set up the repositories.
First, access the Artifactory URL and log in with
a user name of admin
and a password of password
.
Then, click on Maven setup and click Create
.
Maven
RepositorySetup the fly
CLI
You can click one of the icons (depending on your OS) to download fly
, which is the Concourse CLI. Once you download that (and maybe added it to your PATH, depending on your OS) you can run the following command:
fly --version
If fly
is properly installed, it should print out the version.
Setup your credentials.yml
We made a sample credentials file called credentials-sample-k8s.yml
prepared for k8s
. You can use it as a base for your credentials.yml
.
To allow the Concourse worker’s spawned container to connect to the Kubernetes cluster, you must pass the CA contents and the auth token.
To get the contents of CA for GCE, run the following command:
$ kubectl get secret $(kubectl get secret | grep default-token | awk '{print $1}') -o jsonpath='{.data.ca\.crt}' | base64 --decode
To get the auth token, run the following command:
$ kubectl get secret $(kubectl get secret | grep default-token | awk '{print $1}') -o jsonpath='{.data.token}' | base64 --decode
Set that value under paas-test-client-token
, paas-stage-client-token
, and paas-prod-client-token
Build the pipeline
After running Concourse, you should get the following output in your terminal:
$ export POD_NAME=$(kubectl get pods --namespace default -l "app=concourse-web" -o jsonpath="{.items[0].metadata.name}")
$ echo "Visit http://<IP_ADDRESS>:8080 to use Concourse"
$ kubectl port-forward --namespace default $POD_NAME 8080:8080
Visit http://<IP_ADDRESS>:8080 to use Concourse
Log in (for example, for Concourse running at <IP_ADDRESS>
— if you do not provide any value, localhost
is assumed). If you run this script, it assumes that either fly
is on your PATH
or that it is in the same folder as the script:
$ fly -t k8s login -c http://localhost:8080 -u concourse -p concourse
Next, run the following command to create the pipeline:
$ ./set_pipeline.sh github-webhook k8s credentials-k8s.yml
Run the github-webhook
Pipeline
The following images show the various steps involved in runnig the github-webhook
pipeline:
Login
main
team
concourse
user and concourse
password
play
button
+
sign to start a new build
Kubernetes Setup
This section describes how to set up Kubernetes.
Kubernetes CLI Installation
First, you need to install the kubectl
command-line interface (CLI).
Script Installation
You can use the tools/k8s-helper.sh
script to install kubectl
. To do so, run the following script:
$ ./tools/minikube-helper download-kubectl
Then the kubectl
gets downloaded.
Manual Installation
You can perform a manual installation for either OSX or Linux.
Example for OSX
The following listing shows how to manually install on OSX:
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin/kubectl
----
Example for Linux
The following listing shows how to manually install on Linux:
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin/kubectl
See this page for more information.
Kubernetes Cluster Setup
We need a cluster of Kubernetes. The best choice is Minikube.
| You can skip this step if you have a Kubernetes cluster installed and do not want to use Minikube. In that case, the only thing you have to do is to set up spaces. |
| Servers often run run out of resources at the stage step. If that happens, clear some apps from PCF Dev and continue. |
Script Installation
You can use the tools/k8s-helper.sh
script to install Minikube
. To do so, run the following script:
$ ./tools/minikube-helper download-minikube
Then the Minikube
cluster gets downloaded.
Manual Installation
You can perform a manual installation for either OSX or Linux.
Example for OSX
The following listing shows how to manually install on OSX:
$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.20.0/minikube-darwin-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
Feel free to skip running sudo mv minikube /usr/local/bin
if you want to add minikube to your path manually.
Example for Linux
The following listing shows how to manually install on Linux:
$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.20.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
You can skip running sudo mv minikube /usr/local/bin
if you want to add minikube to your path manually.
See this page for more information on the installation.
Run Minikube
To start Kubernetes on your local box, run minikube start
.
To add the dashboard, run minikube dashboard
.
Certificates and Workers
Minikube Certificates and Workers
By default, if you install Minikube, all the certificates get installed in your
~/.minikube
folder. Your kubectl
configuration under ~/.kube/config
also
gets updated to use Minikube.
Manual Certificates and Workers Setup
| If you want to run the default demo setup, you can skip this section. |
To target a given Kubernetes instance, you need to pass around Certificate Authority key and also user keys.
You can read more about the instructions on how to generate those keys here.
Generally speaking, if you have a Kubernetes installation (such as minikube
), this step
has already been done for you. Now you can reuse those keys on the workers.
The following inormation has been extracted from the Kubernetes official documentation.
Configure kubectl
to connect to the target cluster using the following commands, replacing the following values:
-
Replace
${MASTER_HOST}
with the master node address or name used in previous steps. -
Replace
${CA_CERT}
with the absolute path to theca.pem
created in previous steps. -
Replace
${ADMIN_KEY}
with the absolute path to theadmin-key.pem
created in previous steps. -
Replace
${ADMIN_CERT}
with the absolute path to theadmin.pem
created in previous steps.
The following commands show how to perform these steps:
$ kubectl config set-cluster default-cluster --server=https://${MASTER_HOST} --certificate-authority=${CA_CERT}
$ kubectl config set-credentials default-admin --certificate-authority=${CA_CERT} --client-key=${ADMIN_KEY} --client-certificate=${ADMIN_CERT}
$ kubectl config set-context default-system --cluster=default-cluster --user=default-admin
$ kubectl config use-context default-system
Generate Minikube Namespaces
With the Minikube cluster running, we need to generate namespaces. To do so, run the
./tools/k8s-helper.sh setup-namespaces
.
The demo setup (Cloud Foundry)
The demo uses two applications: Github Webhook and Github analytics code. The following image shows how these application communicate with each other:
For the demo scenario we have two applications. Github Analytics
and Github Webhook
.
Let’s imagine a case where Github is emitting events via HTTP. Github Webhook
has
an API that could register to such hooks and receive those messages. Once this happens
Github Webhook
sends a message by RabbitMQ to a channel. Github Analytics
is
listening to those messages and stores them in a MySQL database.
Github Analytics
has its KPIs (Key Performance Indicators) monitored. In the case
of that application the KPI is number of issues.
Let’s assume that if we go below the threshold of X issues then an alert should be sent to Slack.
Deploying Production Applications to PCF Dev
In a real-world scenario, we would not want to automatically provision services such as
RabbitMQ, MySQL, or Eureka each time we deploy a new application to production. Typically,
production is provisioned manually (often by using automated solutions). In our case, before
you deploy to production, you can provision the pcfdev-prod
space by using the
cf-helper.sh
. To do so, call the following script:
$ ./cf-helper.sh setup-prod-infra
The CF CLI:
-
Logs in to PCF Dev,
-
Targets the
pcfdev-prod
space -
Sets up:
-
RabbitMQ (under the
rabbitmq-github
name) -
MySQL (under
mysql-github-analytics
name) -
Eureka (under
github-eureka
name)
-
Running Prometheus on CF
You can check out Toshiaki Maki’s code on how to automate Prometheus installation on CF.
Go to https://prometheus.io/download/ and download the Linux binary. Then run the following command:
cf push cloud-pipelines-prometheus -b binary_buildpack -c './prometheus -web.listen-address=:8080' -m 64m
Also, localhost:9090
in prometheus.yml
should be localhost:8080
.
The file should resemble the following listing to work with the demo setup (change github-analytics-cloud-pipelines.cfapps.io
to your github-analytics
installation).
# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'codelab-monitor'
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first.rules"
# - "second.rules"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ['localhost:8080']
- job_name: 'demo-app'
# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 5s
metrics_path: '/prometheus'
# scheme defaults to 'http'.
static_configs:
- targets: ['github-analytics-cloud-pipelines.cfapps.io']
A deployed version for the Cloud Pipelines demo is available here.
Running Grafana on CF
You can check out Toshiaki Maki’s code to see how to automate Prometheus installation on CF.
Download the tarball from https://grafana.com/grafana/download?platform=linux
and set http_port = 8080
in conf/default.ini
. Then run the following the command:
cf push cloud-pipelines-grafana -b binary_buildpack -c './bin/grafana-server web' -m 64m
The demo uses Grafana Dashboard with an ID of 2471
.
A deployed version for the Cloud Pipelines demo is available here
The demo setup (Kubernetes)
The demo uses two applications: Github Webhook and Github analytics code. The following image shows how these application communicate with each other:
For the demo scenario we have two applications. Github Analytics
and Github Webhook
.
Let’s imagine a case where Github is emitting events via HTTP. Github Webhook
has
an API that could register to such hooks and receive those messages. Once this happens
Github Webhook
sends a message by RabbitMQ to a channel. Github Analytics
is
listening to those messages and stores them in a MySQL database.
Github Analytics
has its KPIs (Key Performance Indicators) monitored. In the case
of that application the KPI is number of issues.
Let’s assume that if we go below the threshold of X issues then an alert should be sent to Slack.
Deploying Production Applications to Minikube
In a real-world scenario, we would not want to automatically provision services such as
RabbitMQ, MySQL, or Eureka each time we deploy a new application to production. Typically,
production is provisioned manually (often by using automated solutions). In our case, before
you deploy to production, you can provision the cloud-pipelines-prod
namespace by using the
k8s-helper.sh
. To do so, call the following script:
$ ./k8s-helper.sh setup-prod-infra
Running Prometheus on Kubernetes
Use Helm to install Prometheus. Later in this demo, we point it to the services deployed to our cluster.
Create a file called values.yaml
with the following content:
rbac:
create: false
alertmanager:
## If false, alertmanager will not be installed
##
enabled: true
# Defines the serviceAccountName to use when `rbac.create=false`
serviceAccountName: default
## alertmanager container name
##
name: alertmanager
## alertmanager container image
##
image:
repository: prom/alertmanager
tag: v0.9.1
pullPolicy: IfNotPresent
## Additional alertmanager container arguments
##
extraArgs: {}
## The URL prefix at which the container can be accessed. Useful in the case the '-web.external-url' includes a slug
## so that the various internal URLs are still able to access as they are in the default case.
## (Optional)
baseURL: ""
## Additional alertmanager container environment variable
## For instance to add a http_proxy
##
extraEnv: {}
## ConfigMap override where fullname is {{.Release.Name}}-{{.Values.alertmanager.configMapOverrideName}}
## Defining configMapOverrideName will cause templates/alertmanager-configmap.yaml
## to NOT generate a ConfigMap resource
##
configMapOverrideName: ""
ingress:
## If true, alertmanager Ingress will be created
##
enabled: false
## alertmanager Ingress annotations
##
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: 'true'
## alertmanager Ingress hostnames
## Must be provided if Ingress is enabled
##
hosts: []
# - alertmanager.domain.com
## alertmanager Ingress TLS configuration
## Secrets must be manually created in the namespace
##
tls: []
# - secretName: prometheus-alerts-tls
# hosts:
# - alertmanager.domain.com
## Alertmanager Deployment Strategy type
# strategy:
# type: Recreate
## Node labels for alertmanager pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
persistentVolume:
## If true, alertmanager will create/use a Persistent Volume Claim
## If false, use emptyDir
##
enabled: true
## alertmanager data Persistent Volume access modes
## Must match those of existing PV or dynamic provisioner
## Ref: https://kubernetes.io/docs/user-guide/persistent-volumes/
##
accessModes:
- ReadWriteOnce
## alertmanager data Persistent Volume Claim annotations
##
annotations: {}
## alertmanager data Persistent Volume existing claim name
## Requires alertmanager.persistentVolume.enabled: true
## If defined, PVC must be created manually before volume will be bound
existingClaim: ""
## alertmanager data Persistent Volume mount root path
##
mountPath: /data
## alertmanager data Persistent Volume size
##
size: 2Gi
## alertmanager data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: "-"
## Subdirectory of alertmanager data Persistent Volume to mount
## Useful if the volume's root directory is not empty
##
subPath: ""
## Annotations to be added to alertmanager pods
##
podAnnotations: {}
replicaCount: 1
## alertmanager resource requests and limits
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# cpu: 10m
# memory: 32Mi
# requests:
# cpu: 10m
# memory: 32Mi
service:
annotations: {}
labels: {}
clusterIP: ""
## List of IP addresses at which the alertmanager service is available
## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
##
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
servicePort: 80
# nodePort: 30000
type: ClusterIP
## Monitors ConfigMap changes and POSTs to a URL
## Ref: https://github.com/jimmidyson/configmap-reload
##
configmapReload:
## configmap-reload container name
##
name: configmap-reload
## configmap-reload container image
##
image:
repository: jimmidyson/configmap-reload
tag: v0.1
pullPolicy: IfNotPresent
## configmap-reload resource requests and limits
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
kubeStateMetrics:
## If false, kube-state-metrics will not be installed
##
enabled: true
# Defines the serviceAccountName to use when `rbac.create=false`
serviceAccountName: default
## kube-state-metrics container name
##
name: kube-state-metrics
## kube-state-metrics container image
##
image:
repository: gcr.io/google_containers/kube-state-metrics
tag: v1.1.0-rc.0
pullPolicy: IfNotPresent
## Node labels for kube-state-metrics pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
## Annotations to be added to kube-state-metrics pods
##
podAnnotations: {}
replicaCount: 1
## kube-state-metrics resource requests and limits
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# cpu: 10m
# memory: 16Mi
# requests:
# cpu: 10m
# memory: 16Mi
service:
annotations:
prometheus.io/scrape: "true"
labels: {}
clusterIP: None
## List of IP addresses at which the kube-state-metrics service is available
## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
##
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
servicePort: 80
type: ClusterIP
nodeExporter:
## If false, node-exporter will not be installed
##
enabled: true
# Defines the serviceAccountName to use when `rbac.create=false`
serviceAccountName: default
## node-exporter container name
##
name: node-exporter
## node-exporter container image
##
image:
repository: prom/node-exporter
tag: v0.15.0
pullPolicy: IfNotPresent
## Additional node-exporter container arguments
##
extraArgs: {}
## Additional node-exporter hostPath mounts
##
extraHostPathMounts: []
# - name: textfile-dir
# mountPath: /srv/txt_collector
# hostPath: /var/lib/node-exporter
# readOnly: true
## Node tolerations for node-exporter scheduling to nodes with taints
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
tolerations: []
# - key: "key"
# operator: "Equal|Exists"
# value: "value"
# effect: "NoSchedule|PreferNoSchedule|NoExecute(1.6 only)"
## Node labels for node-exporter pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
## Annotations to be added to node-exporter pods
##
podAnnotations: {}
## node-exporter resource limits & requests
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# cpu: 200m
# memory: 50Mi
# requests:
# cpu: 100m
# memory: 30Mi
service:
annotations:
prometheus.io/scrape: "true"
labels: {}
clusterIP: None
## List of IP addresses at which the node-exporter service is available
## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
##
externalIPs: []
hostPort: 9100
loadBalancerIP: ""
loadBalancerSourceRanges: []
servicePort: 9100
type: ClusterIP
server:
## Prometheus server container name
##
name: server
# Defines the serviceAccountName to use when `rbac.create=false`
serviceAccountName: default
## Prometheus server container image
##
image:
repository: prom/prometheus
tag: v1.8.0
pullPolicy: IfNotPresent
## (optional) alertmanager URL
## only used if alertmanager.enabled = false
alertmanagerURL: ""
## The URL prefix at which the container can be accessed. Useful in the case the '-web.external-url' includes a slug
## so that the various internal URLs are still able to access as they are in the default case.
## (Optional)
baseURL: ""
## Additional Prometheus server container arguments
##
extraArgs: {}
## Additional Prometheus server hostPath mounts
##
extraHostPathMounts: []
# - name: certs-dir
# mountPath: /etc/kubernetes/certs
# hostPath: /etc/kubernetes/certs
# readOnly: true
## ConfigMap override where fullname is {{.Release.Name}}-{{.Values.server.configMapOverrideName}}
## Defining configMapOverrideName will cause templates/server-configmap.yaml
## to NOT generate a ConfigMap resource
##
configMapOverrideName: ""
ingress:
## If true, Prometheus server Ingress will be created
##
enabled: false
## Prometheus server Ingress annotations
##
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: 'true'
## Prometheus server Ingress hostnames
## Must be provided if Ingress is enabled
##
hosts: []
# - prometheus.domain.com
## Prometheus server Ingress TLS configuration
## Secrets must be manually created in the namespace
##
tls: []
# - secretName: prometheus-server-tls
# hosts:
# - prometheus.domain.com
## Server Deployment Strategy type
# strategy:
# type: Recreate
## Node tolerations for server scheduling to nodes with taints
## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
##
tolerations: []
# - key: "key"
# operator: "Equal|Exists"
# value: "value"
# effect: "NoSchedule|PreferNoSchedule|NoExecute(1.6 only)"
## Node labels for Prometheus server pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}
persistentVolume:
## If true, Prometheus server will create/use a Persistent Volume Claim
## If false, use emptyDir
##
enabled: true
## Prometheus server data Persistent Volume access modes
## Must match those of existing PV or dynamic provisioner
## Ref: https://kubernetes.io/docs/user-guide/persistent-volumes/
##
accessModes:
- ReadWriteOnce
## Prometheus server data Persistent Volume annotations
##
annotations: {}
## Prometheus server data Persistent Volume existing claim name
## Requires server.persistentVolume.enabled: true
## If defined, PVC must be created manually before volume will be bound
existingClaim: ""
## Prometheus server data Persistent Volume mount root path
##
mountPath: /data
## Prometheus server data Persistent Volume size
##
size: 8Gi
## Prometheus server data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: "-"
## Subdirectory of Prometheus server data Persistent Volume to mount
## Useful if the volume's root directory is not empty
##
subPath: ""
## Annotations to be added to Prometheus server pods
##
podAnnotations: {}
# iam.amazonaws.com/role: prometheus
replicaCount: 1
## Prometheus server resource requests and limits
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# cpu: 500m
# memory: 512Mi
# requests:
# cpu: 500m
# memory: 512Mi
service:
annotations: {}
labels: {}
clusterIP: ""
## List of IP addresses at which the Prometheus server service is available
## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
##
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
servicePort: 80
type: ClusterIP
## Prometheus server pod termination grace period
##
terminationGracePeriodSeconds: 300
## Prometheus data retention period (i.e 360h)
##
retention: ""
pushgateway:
## If false, pushgateway will not be installed
##
enabled: true
## pushgateway container name
##
name: pushgateway
## pushgateway container image
##
image:
repository: prom/pushgateway
tag: v0.4.0
pullPolicy: IfNotPresent
## Additional pushgateway container arguments
##
extraArgs: {}
ingress:
## If true, pushgateway Ingress will be created
##
enabled: false
## pushgateway Ingress annotations
##
annotations:
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: 'true'
## pushgateway Ingress hostnames
## Must be provided if Ingress is enabled
##
hosts: []
# - pushgateway.domain.com
## pushgateway Ingress TLS configuration
## Secrets must be manually created in the namespace
##
tls: []
# - secretName: prometheus-alerts-tls
# hosts:
# - pushgateway.domain.com
## Node labels for pushgateway pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector: {}
## Annotations to be added to pushgateway pods
##
podAnnotations: {}
replicaCount: 1
## pushgateway resource requests and limits
## Ref: https://kubernetes.io/docs/user-guide/compute-resources/
##
resources: {}
# limits:
# cpu: 10m
# memory: 32Mi
# requests:
# cpu: 10m
# memory: 32Mi
service:
annotations:
prometheus.io/probe: pushgateway
labels: {}
clusterIP: ""
## List of IP addresses at which the pushgateway service is available
## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
##
externalIPs: []
loadBalancerIP: ""
loadBalancerSourceRanges: []
servicePort: 9091
type: ClusterIP
## alertmanager ConfigMap entries
##
alertmanagerFiles:
alertmanager.yml: |-
global:
# slack_api_url: ''
receivers:
- name: default-receiver
# slack_configs:
# - channel: '@you'
# send_resolved: true
route:
group_wait: 10s
group_interval: 5m
receiver: default-receiver
repeat_interval: 3h
## Prometheus server ConfigMap entries
##
serverFiles:
alerts: ""
rules: ""
prometheus.yml: |-
rule_files:
- /etc/config/rules
- /etc/config/alerts
scrape_configs:
- job_name: 'demo-app'
scrape_interval: 5s
metrics_path: '/prometheus'
static_configs:
- targets:
- github-analytics.cloud-pipelines-prod.svc.cluster.local:8080
- job_name: prometheus
static_configs:
- targets:
- localhost:9090
# A scrape configuration for running Prometheus on a Kubernetes cluster.
# This uses separate scrape configs for cluster components (i.e. API server, node)
# and services to allow each to use different authentication configs.
#
# Kubernetes labels will be added as Prometheus labels on metrics via the
# `labelmap` relabeling action.
# Scrape config for API servers.
#
# Kubernetes exposes API servers as endpoints to the default/kubernetes
# service so this uses `endpoints` role and uses relabelling to only keep
# the endpoints associated with the default/kubernetes service using the
# default named port `https`. This works for single API server deployments as
# well as HA API server deployments.
- job_name: 'kubernetes-apiservers'
kubernetes_sd_configs:
- role: endpoints
# Default to scraping over https. If required, just disable this or change to
# `http`.
scheme: https
# This TLS & bearer token file config is used to connect to the actual scrape
# endpoints for cluster components. This is separate to discovery auth
# configuration because discovery & scraping are two separate concerns in
# Prometheus. The discovery auth config is automatic if Prometheus runs inside
# the cluster. Otherwise, more config options have to be provided within the
# <kubernetes_sd_config>.
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
# If your node certificates are self-signed or use a different CA to the
# master CA, then disable certificate verification below. Note that
# certificate verification is an integral part of a secure infrastructure
# so this should only be disabled in a controlled environment. You can
# disable certificate verification by uncommenting the line below.
#
insecure_skip_verify: true
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
# Keep only the default/kubernetes service endpoints for the https port. This
# will add targets for each API server which Kubernetes adds an endpoint to
# the default/kubernetes service.
relabel_configs:
- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
action: keep
regex: default;kubernetes;https
- job_name: 'kubernetes-nodes'
# Default to scraping over https. If required, just disable this or change to
# `http`.
scheme: https
# This TLS & bearer token file config is used to connect to the actual scrape
# endpoints for cluster components. This is separate to discovery auth
# configuration because discovery & scraping are two separate concerns in
# Prometheus. The discovery auth config is automatic if Prometheus runs inside
# the cluster. Otherwise, more config options have to be provided within the
# <kubernetes_sd_config>.
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
# If your node certificates are self-signed or use a different CA to the
# master CA, then disable certificate verification below. Note that
# certificate verification is an integral part of a secure infrastructure
# so this should only be disabled in a controlled environment. You can
# disable certificate verification by uncommenting the line below.
#
insecure_skip_verify: true
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/${1}/proxy/metrics
# Scrape config for service endpoints.
#
# The relabeling allows the actual service scrape endpoint to be configured
# via the following annotations:
#
# * `prometheus.io/scrape`: Only scrape services that have a value of `true`
# * `prometheus.io/scheme`: If the metrics endpoint is secured then you will need
# to set this to `https` & most likely set the `tls_config` of the scrape config.
# * `prometheus.io/path`: If the metrics path is not `/metrics` override this.
# * `prometheus.io/port`: If the metrics are exposed on a different port to the
# service then set this appropriately.
- job_name: 'kubernetes-service-endpoints'
kubernetes_sd_configs:
- role: endpoints
relabel_configs:
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
action: replace
target_label: __scheme__
regex: (https?)
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
action: replace
target_label: __address__
regex: (.+)(?::\d+);(\d+)
replacement: $1:$2
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
action: replace
target_label: kubernetes_name
- job_name: 'prometheus-pushgateway'
honor_labels: true
kubernetes_sd_configs:
- role: service
relabel_configs:
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
action: keep
regex: pushgateway
# Example scrape config for probing services via the Blackbox Exporter.
#
# The relabeling allows the actual service scrape endpoint to be configured
# via the following annotations:
#
# * `prometheus.io/probe`: Only probe services that have a value of `true`
- job_name: 'kubernetes-services'
metrics_path: /probe
params:
module: [http_2xx]
kubernetes_sd_configs:
- role: service
relabel_configs:
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
action: keep
regex: true
- source_labels: [__address__]
target_label: __param_target
- target_label: __address__
replacement: blackbox
- source_labels: [__param_target]
target_label: instance
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
target_label: kubernetes_name
# Example scrape config for pods
#
# The relabeling allows the actual pod scrape endpoint to be configured via the
# following annotations:
#
# * `prometheus.io/scrape`: Only scrape pods that have a value of `true`
# * `prometheus.io/path`: If the metrics path is not `/metrics` override this.
# * `prometheus.io/port`: Scrape the pod on the indicated port instead of the default of `9102`.
- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: (.+):(?:\d+);(\d+)
replacement: ${1}:${2}
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
networkPolicy:
## Enable creation of NetworkPolicy resources.
##
enabled: false
Next, create the prometheus installation with the predefined values. To do so, run the following command:
$ helm install --name cloud-pipelines-prometheus stable/prometheus -f values.yaml
Then you should see the following output:
NOTES:
The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
cloud-pipelines-prometheus-prometheus-server.default.svc.cluster.local
Get the Prometheus server URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9090
The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
cloud-pipelines-prometheus-prometheus-alertmanager.default.svc.cluster.local
Get the Alertmanager URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=alertmanager" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9093
The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
cloud-pipelines-prometheus-prometheus-pushgateway.default.svc.cluster.local
Get the PushGateway URL by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 9093
For more information on running Prometheus, visit:
https://prometheus.io/
Running Grafana on Kubernetes
Use Helm to install Grafana, by running the following command:
$ helm install --name cloud-pipelines-grafana stable/grafana
You should see the following output:
NOTES:
1. Get your 'admin' user password by running:
kubectl get secret --namespace default cloud-pipelines-grafana-grafana -o jsonpath="{.data.grafana-admin-password}" | base64 --decode ; echo
2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:
cloud-pipelines-grafana-grafana.default.svc.cluster.local
Get the Grafana URL to visit by running these commands in the same shell:
export POD_NAME=$(kubectl get pods --namespace default -l "app=cloud-pipelines-grafana-grafana,component=grafana" -o jsonpath="{.items[0].metadata.name}")
kubectl --namespace default port-forward $POD_NAME 3000
3. Login with the password from step 1 and the username: admin
Perform the steps listed in the preceding output and add the Grafana’s datasource
as Prometheus with the following URL: https://cloud-pipelines-prometheus-prometheus-server.default.svc.cluster.local
You can pick up the dashboard with the Grafana ID (2471). This is the default dashboard for the Cloud Pipelines demo apps.
If you have both apps (github-webhook
and github-analytics
) running on production,
you can now trigger the messages. Download the JSON with a sample request
from the github-webhook repository.
Next, pick one of the github-webhook
pods and forward its port
locally to a port 9876
, as follows:
$ kubectl port-forward --namespace=cloud-pipelines-prod $( kubectl get pods --namespace=cloud-pipelines-prod | grep github-webhook | head -1 | awk '{print $1}' ) 9876:8080
Next, send a couple of requests (more than four), by using cURL as follows:
$ curl -X POST http://localhost:9876/ -d @path/to/issue-created.json \
--header "Content-Type: application/json"
Then, if you use Grafana, you can see that you went above the threshold.


Mobile application created for KL Rapid bus commuters
KUALA LUMPUR, Nov 13 — A mobile application has been developed to help plan travelling using KL Rapid Bus on the actual time of arrival of the public transport at a certain location.
The Transport Ministry in a written reply displayed on Parliament website said the usage of technology will enable commuters know the arrival and departure of buses to facilitate them plan their journeys.
“Prasarana (Prasarana Malaysia Berhad) will also ensure the recruitment of drivers is conducted more frequently to overcome the shortage of drivers which affected the bus operation,” the ministry said.
The ministry was replying to a question by P. Prabakaran (PH-Batu) who asked the Transport Minister to explain how the ministry intends to ensure public buses such as KL Rapid Bus provide their service according to schedule.
Meanwhile, the ministry also stated that the perception of bus service monopoly in the Klang Valley was inaccurate as there are other bus companies apart from those under Prasarana.
“The government will study and implement the bus contracting model which is more efficient and encourages competition in future,” said the ministry.
This was in reply to a question by Prabakaran who wanted an clarification from the ministry on whether there is a monopoly system by the bus company in Kuala Lumpur. — Bernama


This was my first experience (and last) with Sinicuichi:
Mindset:
I expected this herb to change the proportion of the room in appearance, to dim my vision, cause me to be cold, give auditory hallucinations, give me vivid memories of the past, and a calm euphoric feeling. There is much concurrent validity among sources on these effects, so I expected this would be at least a mild 'trip.'
Setting: A quiet apartment, alone
Preparation: The herb was bought and already a bit wilted. 6 grams of sinicuichi were infused in 2 cups of water for 15 minutes. This was laid out in the sun to ferment for eight daylit hours with Saran wrap over the two cups.
6:50pm: Drank the two cups over 23 minutes. It had no taste, but the bitterness was barely bearable.  But the point is that I got it down regardless. Many feelings of nausea afflicted me, but only during the drinking process. I took two breaks to let those feelings calm down before continuing. Lemon juice was added and helped a little.
No effects whatsoever.
3:30am
I was woken in the middle of the night by the sound of the train going by far off blowing its horn (as it normally does and occasionally I awaken to it). But the train horn sound had digital delay on it, a musical effect where the sound echoes over and over. I was in a pleasant slightly dazed state, but that I accredit mostly to being half awake and half asleep at 3:30am. I stayed in this half-and-half state for a half an hour. I did feel good emotionally though, very comfortable, but maybe I can attest this to simply warm covers on a cold night and laying down when sleepy.
I heard someone start their car fifty times (which I doubt actually occurred more than once), and drive it off five times, but it was very low in audibility.
Finally, I heard the train tracks behind/beside me, when the train is way down the street in front of me; over and over the train seemed to go by the tracks, even after the horn stopped (meaning the train was no longer there). These are not auditory hallucinations to me, but are simply the repeated hearing of the same sound. Or, rather, they are not psychogenic, the sounds were born of the environment. I was in such a state of tiredness, combined with the suggestiveness of maybe waking up and hearing something like another account stated, combined with WANTING the herb to work, that I cannot say any of the above is anything more than a placebo effect, in other words the same as if I hadnít taken the herb but thought I had taken it.
Two days later:
Looking back, I am VERY sure the herb caused the above effects of auditory repetitions and maybe part of the comforting pleasantness I felt in bed. Furthermore, I have slept very well the last two nights (slightly better than usual it seems).
There werenít many sounds at all at that time of the night. I remember speaking to try to induce the effect, but that didnít work. It seemed only far off sounds were duplicated. Nonetheless, I can thank the herb for almost throwing up, a stuck-record train horn, seven dollars less in my wallet, and a comfortable wake in the middle of the night.
Conclusion: The bitterness makes it not worth it. I sat with crazed expression saying, 'Itís good! Itís so good, I like it!' and I could convince myself for one second until I took another sip and my back taste buds blown with horror. Then for some insane reason I took another sip, and another, and it happened again. If it tasted like orange juice, then it would be decent. Adding honey and drinking it warmer may help, and the lemon juice helped a little. Interesting, but only for a one-time thing.
Note: If I was in an auditory stimulating environment and wide awake when the effects set in, I think I would have enjoyed it more. I feel that since I was in a quiet room, nothing happened.
Note: There seems to be little known about the chemical make-up of this plant, although it certainly seems safe to use.
My list of what has been easiest to drink to worst to drink:
Damiana (pleasant almost)
Yohimbe (not bad)
Hops (weak taste of unpleasantness)
Calea (strong unpleasantness in taste, and hard to get down)
Sinicuichi (no real unpleasantness in taste, but very hard to get down, nausea).
Coffee (unpalatable, extremely bad, how do people drink this shtuff?)
--Malawi
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Mark Nguyen
Code Artisan
Reach Me
You can reach me at<EMAIL_ADDRESS>Experience Me
June 2015 – PresentSoftware Engineer
– AppleTop Secret, bounded by the laws of causality. Nov 2013 – March 2015Android Engineer
– ProcoreWorked hard on making things fast and reliable! April 2013 – June 2015Web Master
– SexInfoOnlineDid anything and everything involved with keeping this website running while adding new things constantly. March 2013 – June 2014Classroom Support Tech
– Instructional Development, UCSB-
STAR Helper –
Github
Google Chrome extension that modifies the CSS and HTML for usability on the STAR internal website that manages student information as well as provides common shortcuts and preemptively loads data for quick access.
-
Laptop Updates –
Github
Wrote scripts for windows and mac to automate the cleaning and updating process including individual programs as well as system updates.
-
STAR Helper –
Github
Build With Me
June 2015 – PresentBisque for Lobsters
– GithubAn Android app that embraces Material Design and Android's best practices with the end goal of providing a comprehensive experience with https://lobste.rs. April 2015 – June 2015http://nguyenmp.com
– Link GithubPersonal website to experiment with HTML and CSS. Primary goals include fast loading times and responsive design. This website was made entirely by hand with zero libraries. No javascript. Feb 2014 – Dec 2014CSIL Status
– GithubAnalytics engine for UCSB CSIL computers composed of three parts:- Android Client – Mobile interface for checking usage information as well as posted notifications when followers came online
- Web App – Frontend for viewing historical data of users logged in as well as backend for crunching and analyzing through the big data
- Desktop Client/Scraper – Highly resilliant desktop application that regularly sent the server current usage data for individual computers
Feb 2014 – Dec 2014/r/eader
A reboot of Reddit Reader using the best practices of Android. Jan 2013 – Mar 2013AudioSync
A group project to synchronize music playback on multiple Android phones over WiFi. – Google Play Sep 2012 – March 2013Gaucho Wifi
Automatically logs you into the UCSB wireless web. – Google Play Sep 2012 – Dec 2012Gaucho Homework
An unofficial Android frontend for GauchoSpace, UCSB's course management system. – Google Play Sep 2012 – Dec 2012Puush Image Manager
An unofficial Android frontend for puush.me. This was primarily a learning experience and due to unofficial support and no official API, this project was scrapped. I also stopped using Puush. Sep 2011 – June 2012Reddit Reader
My first app ever! I spent a year learning how to make an Android app. Eventually scrapped the project because a better app finally was released.


Some Simple Tests
Simple tests of js-easy
Here is a second paragraph of text
This is the second section of the page


--- title: Keras array object type: docs repo: https://github.com/rstudio/keras menu: main: name: Keras array object - keras parent: keras-reference aliases: - /reference/keras/keras_array.html - /keras/reference/keras_array.html - /guide/keras/reference/keras_array.html - /tools/tools/keras/reference/keras_array.html - /installation/keras/reference/keras_array.html - /tutorials/keras/reference/keras_array.html - /guide/tools/keras/reference/keras_array.html - /deploy/keras/reference/keras_array.html - /tools/keras/reference/keras_array.html - /tutorials/tools/keras/reference/keras_array.html ---
Convert an R vector, matrix, or array object to an array that has the optimal in-memory layout and floating point data type for the current Keras backend.
keras_array(x, dtype = NULL)
| x | Object or list of objects to convert |
| dtype | NumPy data type (e.g. float32, float64). If this is unspecified then R doubles will be converted to the default floating point type for the current Keras backend. |
NumPy array with the specified dtype
(or list of NumPy arrays if a
list was passed for x
).
Keras does frequent row-oriented access to arrays (for shuffling and drawing batches) so the order of arrays created by this function is always row-oriented ("C" as opposed to "Fortran" ordering, which is the default for R arrays).
If the passed array is already a NumPy array with the desired dtype
and "C"
order then it is returned unmodified (no additional copies are made).


Intro to TOML
Aug 16, 2014TOML stands for Tom’s Own Minimal Language. It is a configuration language vaguely similar to YAML or property lists, but far, far better. But before we get into it in detail, let’s look back at what came before.
Long Ago, In A Galaxy Far, Far Away
Since the beginning of computing, people have needed a way to configure their software. On Linux, this generally is done in text files. For simple configurations, good old foo = bar works pretty well. One setting per line, name on the left, value on the right, separated by an equals. Great. But when your configuration gets more complicated, this quickly breaks down. What if you need a value that is more than one line? How do you indicate a value should be parsed as a number instead of a string? How do you namespace related configuration values so you don’t need ridiculously long names to prevent collisions?
The Dark Ages
In the 90’s, we used XML. And it sucked. XML is verbose, it’s hard for humans to read and write, and it still doesn’t solve a lot of the problems above (like how to specify the type of a value). In addition, the XML spec is huge, processing is very complicated, and all the extra features invite abuse and overcomplication.
Enlightenment
In the mid 2000’s, JSON came to popularity as a data exchange format, and it was so much better than XML. It had real types, it was easy for programs to process, and you didn’t have to write a spec on what values should get processed in what way (well, mostly). It was sigificantly less verbose than XML. But it is a format intended for computers to read and write, not humans. It is a pain to write by hand, and even pretty-printed, it can be hard to read and the compact data format turns into a nested mess of curly braces. Also, JSON is not without its problems… for example, there’s no date type, there’s no support for comments, and all numbers are floats.
A False Start
YAML came to popularity some time after JSON as a more human-readable format,
and its key: value
syntax and pretty indentation is definitely a lot easier on
the eyes than JSON’s nested curly-braces. However, YAML trades ease of reading
for difficulty in writing. Indentation as delimiters is fraught with error…
figuring out how to get multiple lines of data into any random value is an
exercise in googling and trial & error.
The YAML spec is also ridiculously long. 100% compatible parsers are very difficult to write. Writing YAML by hand is a ridden with landmines of corner cases where your choice of names or values happens to hit a reserved word or special marker. It does support comments, though.
The Savior
On February 23, 2013, Tom Preston-Werner (former CEO of GitHub) made his first commit to https://github.com/toml-lang/toml. TOML stands for Tom’s Obvious, Minimal Language. It is a language designed for configuring software. Finally.
TOML takes inspiration from all of the above (well, except XML) and even gets some of its syntax from Microsoft’s INI files. It is easy to write by hand and easy to read. The spec is short and understandable by mere humans, and it’s fairly easy for computers to parse. It supports comments, has first class dates, and supports both integers and floats. It is generally insensitive to whitespace, without requiring a ton of delimiters.
Let’s dive in.
The Basics
The basic form is key = value
# Comments start with hash
foo = "strings are in quotes and are always UTF8 with escape codes: \n \u00E9"
bar = """multi-line strings
use three quotes"""
baz = 'literal\strings\use\single\quotes'
bat = '''multiline\literals\use
three\quotes'''
int = 5 # integers are just numbers
float = 5.0 # floats have a decimal point with numbers on both sides
date = 2006-05-27T07:32:00Z # dates are ISO 8601 full zulu form
bool = true # good old true and false
One cool point: If the first line of a multiline string (either literal or not) is a line return, it will be trimmed. So you can make your big blocks of text start on the line after the name of the value and not need to worry about the extraneous newline at the beginning of your text:
preabmle = """
We the people of the United States, in order to form a more perfect union,
establish justice, insure domestic tranquility, provide for the common defense,
promote the general welfare, and secure the blessings of liberty to ourselves
and our posterity, do ordain and establish this Constitution for the United
States of America."""
Lists
Lists (arrays) are signified with brackets and delimited with commas. Only primitives are allowed in this form, though you may have nested lists. The format is forgiving, ignoring whitespace and newlines, and yes, the last comma is optional (thank you!):
foo = [ "bar", "baz"
"bat"
]
nums = [ 1, 2, ]
nested = [[ "a", "b"], [1, 2]]
I love that the format is forgiving of whitespace and that last comma. I like that the arrays are all of a single type, but allowing mixed types of sub-arrays bugs the heck out of me.
Now we get crazy
What’s left? In JSON there are objects, in YAML there are associative arrays… in common parlance they are maps or dictionaries or hash tables. Named collections of key/value pairs.
In TOML they are called tables and look like this:
# some config above
[table_name]
foo = 1
bar = 2
Foo and bar are keys in the table called table_name. Tables have to be at the end of the config file. Why? because there’s no end delimiter. All keys under a table declaration are associated with that table, until a new table is declared or the end of the file. So declaring two tables looks like this:
# some config above
[table1]
foo = 1
bar = 2
[table2]
foo = 1
baz = 2
The declaration of table2 defines where table1 ends. Note that you can indent the values if you want, or not. TOML doesn’t care.
If you want nested tables, you can do that, too. It looks like this:
[table1]
foo = "bar"
[table1.nested_table]
baz = "bat"
nested_table
is defined as a value in table1
because its name starts with
table1.
. Again, the table goes until the next table definition, so baz="bat"
is a value in table1.nested_table
. You can indent the nested table to make it
more obvious, but again, all whitespace is optional:
[table1]
foo = "bar"
[table1.nested_table]
baz = "bat"
This is equivalent to the JSON:
{
"table1" : {
"foo" : "bar",
"nested_table" : {
"baz" : "bat"
}
}
}
Having to retype the parent table name for each sub-table is kind of annoying, but I do like that it is very explicit. It also means that ordering and indenting and delimiters don’t matter. You don’t have to declare parent tables if they’re empty, so you can do something like this:
[foo.bar.baz]
bat = "hi"
Which is the equivalent to this JSON:
{
"foo" : {
"bar" : {
"baz" : {
"bat" : "hi"
}
}
}
}
Last but not least
The last thing is arrays of tables, which are declared with double brackets thusly:
[[comments]]
author = "Nate"
text = "Great Article!"
[[comments]]
author = "Anonymous"
text = "Love it!"
This is equivalent to the JSON:
{
"comments" : [
{
"author" : "Nate",
"text" : Great Article!"
},
{
"author" : "Anonymous",
"text" : Love It!"
}
]
}
Arrays of tables inside another table get combined in the way you’d expect, like [[table1.array]].
TOML is very permissive here. Because all tables have very explicitly defined parentage, the order they’re defined in doesn’t matter. You can have tables (and entries in an array of tables) in whatever order you want. This is totally acceptable:
[[comments]]
author = "Anonymous"
text = "Love it!"
[foo.bar.baz]
bat = "hi"
[foo.bar]
howdy = "neighbor"
[[comments]]
author = "Anonymous"
text = "Love it!"
Of course, it generally makes sense to actually order things in a more organized fashion, but it’s nice that you can’t shoot yourself in the foot if you reorder things “incorrectly”.
Conclusion
That’s TOML. It’s pretty awesome.
There’s a list of parsers on the TOML page on github for pretty much whatever language you want. I recommend BurntSushi’s for Go, since it works just like the built-in parsers.
It is now my default configuration language for all the applications I write.
The next time you write an application that needs some configuration, take a look at TOML. I think your users will thank you.


About The Event
The first edition of MoroccoAI annual conference (MoroccoAI 2021) will be held on
December 21st and 22nd, 2021. It will bring advanced knowledge and recent
developments in AI for students, researchers and industry professionals.
In addition to the scientific program, the conference will also include plenary
sessions, Workshops and Data challenges. Attending the conference is a great
opportunity for participants to network and debate about different fields of AI.
Main objectives
- Advance the state of R&D in the field of AI in Morocco
- Build a dynamic Moroccan AI community with the aim to educate, connect and support AI practitioners
- Foster more collaboration between academia and industry
- Increase the number of Moroccan representation in top AI conferences


Jang seoksoon
Send a mail
Here is my
Resume
This is my
CV
I am
@boraseoksoon
on Github.
I am
@boraseoksoon
on StackOverflow
I am also a founder of
'HLVM'
.
Here is my
Linkedin
page.


Setting up Gmail
Setting up Code Co-op to work with Gmail is quick and easy.
First you have to obtain a free Gmail account and configure it to accept POP connections.
- From your browser, login to your Gmail Account.
- Select Settings in the upper right corner of your Gmail homepage and make the adjustments as shown in the screenshot.
- Select Forwarding and POP from the settings menu.
- Click the radio button "Enable POP only for mail that arrives from now on".
< Back Next: Email Diagnostics >


{% load static %}
Please fill the following details to access your account.
Please fill the information below to create your account.
Pranshu is a brilliant tactician, strategist, scientist, tracker and a master of all forms of unarmed combat whose unique hybrid fighting style incorporates acrobatics and aspects of animal mimicry. He is a master planner who always thinks several steps ahead and will go to extreme measures to achieve his goals and protect the kingdom of Wakanda.
Pratik is an accomplished scientist, inventor and photographer. He can cling to most surfaces, has superhuman strength and is roughly 15 times more agile than a regular human. The combination of his acrobatic leaps and web-slinging enables him to travel rapidly from place to place. His spider-sense provides an early warning detection system linked with his superhuman kinesthetics.
Rajat is an eccentric self-described genius, billionaire, playboy and philanthropist and the former head of Rajat Industries. Using his own great wealth and exceptional technical knowledge, Rajat enjoyed the playboy lifestyle for many years until he joined DTU.


|
|
Oh no! We're sorry you can't make it.
|
|
We understand, things can get busy. While we're disappointed we won't get to celebrate with you in December, we hope to see you real soon.
|
|
| |
| You've also rsvp'd for the following people: |
| |
|
|
Terra Gallo cannot attend.
|
|
|
Terra Gallo can attend, and will be having the fish for dinner.
|
|
|
|
|
If you have any questions, or if you need to update your RSVP, feel free to reply directly to this email or call Nick or Beth. We'll be happy to make sure you've got all the info you need.
|
|
|


Get back to nature with all the tools and toys you need to enjoy the great outdoors.
Nothing beats food ocooked over a fire. We have all you need for good camping eats.
Geat a good night's rest in the wild in a home away from home that travels well.
From flannel shirts to toques, look the part while roughing it in the great outdoors.


Advanced search
Claim status
Mode of approval
Type of claim
Visit rules
Date of visit
Date claim submitted
Date claim approved
Date claim rejected
- Search results 7 Export search results (CSV file)
| Ref No. | Claimant name | Submitted | Status | |
|---|---|---|---|---|
| 4VTE9ZM | Joseph Bloggs | 29/09/16 - 12:24 | First time claim | View |
| 4DTGEHI | Paul Smith | 29/09/16 - 12:17 | Eligibility changed | View |
| 6WKMCHS | Chris Sisk | 29/09/16 - 11:59 | First time claim | View |
| 9PLATGD | Jane Smith | 29/09/16 - 11:25 | First time claim | View |
| 2YDINGC | Susan Jones | 29/09/16 - 11:24 | Eligibility changed | View |
| 5SLMMOP | Jim Brown | 29/09/16 - 10:01 | First time claim | View |
| 7DNJSPP | Sarah Smith | 29/09/16 - 10:00 | First time claim | View |


Constructor
new PdfTk(src)
PdfTk constructor.
Parameters:
| Name | Type | Description |
|---|---|---|
src |
Array | Input source file(s). |
Members
args :Array
Type:
- Array
command :String
Type:
- String
postArgs :Array
Type:
- Array
src :Array
Type:
- Array
tmpFiles :Array
Type:
- Array
Methods
(static) generateFdfFromJSON(data) → {Buffer}
Creates fdf file from JSON input.
Converts input values to binary buffer, which seems to allow PdfTk to render utf-8 characters.
Parameters:
| Name | Type | Description |
|---|---|---|
data |
Object | JSON data to transform to fdf. |
Returns:
Fdf data as a buffer.
- Type
- Buffer
(static) generateInfoFromJSON(data) → {Buffer}
Creates pdf info text file from JSON input.
Parameters:
| Name | Type | Description |
|---|---|---|
data |
Object | JSON data to transform to info file. |
Returns:
Info text file as a buffer.
- Type
- Buffer
(static) input(src) → {Object}
Input files and initialize plugin.
Parameters:
| Name | Type | Description |
|---|---|---|
src |
String | Array | Source files to input. |
Returns:
PdfTk class instance.
- Type
- Object
(static) isObject(item) → {Boolean}
Simple object check. Arrays not included.
Parameters:
| Name | Type | Description |
|---|---|---|
item |
Item to check. |
Returns:
Is object.
- Type
- Boolean
(static) isString(item) → {Boolean}
Simple string check.
Parameters:
| Name | Type | Description |
|---|---|---|
item |
Item to check. |
Returns:
Is string.
- Type
- Boolean
(static) toBuffer(file) → {Buffer}
Returns a buffer from a file.
Parameters:
| Name | Type | Description |
|---|---|---|
file |
String | Buffer | File to buffer. |
Returns:
Buffered file.
- Type
- Buffer
allow(perms) → {Object}
Set permissions for a PDF. By not passing in the "perms" parameter, you are disabling all features.
Parameters:
| Name | Type | Description |
|---|---|---|
perms |
Array | String | Permissions to set. Choices are: Printing, DegradedPrinting, ModifyContents, Assembly, CopyContents, ScreenReaders, ModifyAnnotations, FillIn, AllFeatures. |
Returns:
PdfTk class instance.
- Type
- Object
attachFiles(files) → {Object}
Attach files to PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
files |
String | Array | Files to attach. |
- Source:
- See:
-
- https://www.pdflabs.com/docs/pdftk-man-page/#dest-op-attach for more information.
Returns:
PdfTk class instance.
- Type
- Object
background(file) → {Object}
Applies a PDF watermark to the background of a single PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
file |
String | Buffer | PDF file that contains the background to be applied. |
Returns:
PdfTk class instance.
- Type
- Object
burst(outputOptions) → {Object}
Splits a single PDF into individual pages.
Parameters:
| Name | Type | Description |
|---|---|---|
outputOptions |
String | Array | Burst output options for naming conventions. |
Returns:
PdfTk class instance.
- Type
- Object
cat(catCommand) → {Object}
Assembles ("catenates") pages from input PDFs to create a new PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
catCommand |
String | Array | Page ranges for cat method. |
Returns:
PdfTk class instance.
- Type
- Object
compress() → {Object}
Restore page sream compression.
Returns:
PdfTk class instance.
- Type
- Object
doAsk() → {Object}
Always prompt when errors occur.
Returns:
PdfTk class instance.
- Type
- Object
dontAsk() → {Object}
Never prompt when errors occur.
Returns:
PdfTk class instance.
- Type
- Object
dropXfa() → {Object}
Drop all XFA data.
Returns:
PdfTk class instance.
- Type
- Object
dumpData() → {Object}
Outputs PDF bookmarks and metadata.
Returns:
PdfTk class instance.
- Type
- Object
dumpDataAnnots() → {Object}
Outputs PDF annotation information.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
dumpDataFields() → {Object}
Outputs form field statistics.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
dumpDataFieldsUtf8() → {Object}
Outputs form field statistics with utf-8 encoding.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
dumpDataUtf8() → {Object}
Outputs PDF bookmarks and metadata with utf-8 encoding.
Returns:
PdfTk class instance.
- Type
- Object
encrypt40Bit() → {Object}
Set 40 bit encryption.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
encrypt128Bit() → {Object}
Set 128 bit encryption.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
fillForm(data) → {Object}
Fill a PDF form from JSON data.
Parameters:
| Name | Type | Description |
|---|---|---|
data |
Object | Form fill data. |
Returns:
PdfTk class instance.
- Type
- Object
flatten() → {Object}
Merge PDF form fields and their data.
Returns:
PdfTk class instance.
- Type
- Object
generateFdf() → {Object}
Generate fdf file from input PDF.
Returns:
PdfTk class instance.
- Type
- Object
inputPw(password) → {Object}
Set the input password.
Parameters:
| Name | Type | Description |
|---|---|---|
password |
String | Password to set. |
Returns:
PdfTk class instance.
- Type
- Object
keepFinalId() → {Object}
Keep final ID when combining pages.
Returns:
PdfTk class instance.
- Type
- Object
keepFirstId() → {Object}
Keep first ID when combining files.
Returns:
PdfTk class instance.
- Type
- Object
multiBackground(file) → {Object}
Same as the background operation, but applies each page of the background PDF to the corresponding page of the input PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
file |
String | Buffer | PDF file that contains the background to be applied. |
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
multiStamp(file) → {Object}
Same as the stamp operation, but applies each page of the stamp PDF to the corresponding page of the input PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
file |
String | Buffer | PDF file that contains the content to be stamped. |
Returns:
PdfTk class instance.
- Type
- Object
needAppearances() → {Object}
Set Adobe Reader to generate new field appearances.
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
output(writeFile, outputDest) → {Promise}
Run the command.
Parameters:
| Name | Type | Description |
|---|---|---|
writeFile |
String | Path to the output file to write from stdout. If used with the "outputDest" parameter, two files will be written. |
outputDest |
String | The output file to write without stdout. When present, the returning promise will not contain the output buffer. If used with the "writeFile" parameter, two files will be written. |
Returns:
Promise that resolves the output buffer, if "outputDest" is not given.
- Type
- Promise
ownerPw(password) → {Object}
Set the owner password.
Parameters:
| Name | Type | Description |
|---|---|---|
password |
String | Password to set. |
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
rotate(rotateCommand) → {Object}
Takes a single input PDF and rotates just the specified pages.
Parameters:
| Name | Type | Description |
|---|---|---|
rotateCommand |
String | Array | Page ranges for rotate command. |
Returns:
PdfTk class instance.
- Type
- Object
shuffle(shuffleCommand) → {Object}
Collates pages from input PDF to create new PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
shuffleCommand |
String | Array | Page ranges for shuffle method. |
Returns:
PdfTk class instance.
- Type
- Object
stamp(file) → {Object}
This behaves just like the background operation except it overlays the stamp PDF page on top of the input PDF document’s pages.
Parameters:
| Name | Type | Description |
|---|---|---|
file |
String | Buffer | PDF file that contains the content to be stamped. |
Returns:
PdfTk class instance.
- Type
- Object
toPage(pageNo) → {Object}
Used with the attachFiles method to attach to a specific page.
Parameters:
| Name | Type | Description |
|---|---|---|
pageNo |
Number | Page number in which to attach. |
Returns:
PdfTk class instance.
- Type
- Object
uncompress() → {Object}
Remove page stream compression.
Returns:
PdfTk class instance.
- Type
- Object
unpackFiles(outputDir) → {Promise}
Unpack files into an output directory. This method is not chainable, and hereby does not require
the output method afterwards.
Parameters:
| Name | Type | Description |
|---|---|---|
outputDir |
String | Output directory for files. |
- Source:
- See:
-
- https://www.pdflabs.com/docs/pdftk-man-page/#dest-op-unpack for more information.
Returns:
Promise callback
- Type
- Promise
updateInfo(data) → {Object}
Update the bookmarks and metadata of a PDF with utf-8 encoding.
Parameters:
| Name | Type | Description |
|---|---|---|
data |
Object | Update data. |
Returns:
PdfTk class instance.
- Type
- Object
updateInfoUtf8(data) → {Object}
Update the bookmarks and metadata of a PDF.
Parameters:
| Name | Type | Description |
|---|---|---|
data |
Object | Update data. |
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
userPw(password) → {Object}
Set the user password.
Parameters:
| Name | Type | Description |
|---|---|---|
password |
String | Password to set. |
- Source:
- See:
Returns:
PdfTk class instance.
- Type
- Object
verbose() → {Object}
Set the verbose option.
Returns:
PdfTk class instance.
- Type
- Object


QUALITY COFFEE, QUALITY CAFFEINE
At Starbuzz Coffee, we are dedicated to filling all your caffeine needs through our quality coffees and teas. Sure, we want you to have a great cup of coffee and a great coffee experience as well, but we're the only company that actively monitors and optimizes caffeine levels. So stop by and fill your cup, or order online with our new Bean Machine online order form, and get that quality Starbuzz coffee that you know will meet your caffeine standards.


Why do you need such broad access to my GitHub account?
RetroGit requests access for a couple of kinds of data from your GitHub account:
Personal user data: Needed in order to determine which email address to send your digest to (I did not want to build my own email address validation infrastructure).
Repositories: Needed to get at historical commits used to generate your digest. The authentication scopes that GitHub's API offers are quite coarse-grained, so there is no narrower option. This means that RetroGit also has access to the read-write contents of your source files, even though it does not need (or use) it. The one mitigating option is to only request this level of access for public repositories — this can be done in RetroGit by unchecking the "Include private repositories" checkbox when signing in.
How much data can you see about my account?
RetroGit has access to the following data about your GitHub account and repositories:
Email addresses
Commit history
Source code
Issues
Pull requests
Wikis
Settings
Webhooks
Deploy keys
However it only uses the data in bold, everything else is provided as a side effect of the scope that it uses with the GitHub API.
What is is stored in your servers?
RetroGit does not persist any commit messages or source code from your repositories on its servers (GitHub API responses may be cached in memory for a short period). Digests are generated dynamically when they need to be sent out. What ends up being stored is (see the Account struct for details):
OAuth token enabling RetroGit to query data for your account.
Which email address to receive your digests at.
Timezone, digest frequency and other settings.
There is also a per-user map of the timestamp of the oldest commit for each repository, since this is expensive to compute.
Can I run my own instance?
RetroGit's source is available and it runs on the App Engine Go Runtime, so you can easily start your own instance. It is not very resource intensive -- single user accounts should definitely fit within the free daily quota.
Can I delete my account?
Yes, this can be done via the settings page. You can also revoke RetroGit's access to your account via the GitHub authorized applications page.


Hello, Welcome to My Page.
These links are for demonstrating My Page:
Please Comment here !! Type your secret here:
You can also load up a more advanced page.


You're entering a site under construction. You can see, edit and feedback the code of the REST API and Angular.io app. Estimated initial release: Autumn 2017.
Until then, things just may not work. Wear a helmet!
Warning: the data you enter won't be kept in the final version. Write down your creations.Play. Hack. Feedback.


This certificate is issued on request of without any financial obligation /liability on the part of institution ; netither this certificate should be treated as any commitment from the institution in whatsover matter.


3 columns | 2 columns | Samplebutton | Sub-page 1 | Sub-page 2 | Sub-page 3
Blue is one of the most common main colors in webdesign. It is so common that it is considered very non-trendy today. But it is still a soft and nice color that does no harm - and also my favorite color. So andreas09, which is the last design in the andreasXX series in a while, is aqua blue by default. But there is a twist:
If you don't like the blue colors you can easily change to another color using any of the attached alternative background image pairs: green, orange, purple, red or black! The alternative images are located in a separate folder, and changing color is done by replacing the bodybg.jpg and menuhover.jpg colors in the "img" folder with any pair of images from the "alt-img" folder. Pick a color, move the files and make sure that they have the same filename as the original files - and you're done! As a little bonus, I have also included an alternative layout with 2 columns.
This template is released as a completely free XHTML/CSS template, which means that you are allowed to use it in any way you may want to without any limitations or obligations at all. I kindly ask you to leave the credit link in the footer (the line that says "Original design by Andreas Viklund") since that is a great way of supporting my work. But it is not a requirement, there are several other ways you can support my work if you find this template useful.
Custom, professional XHTML/CSS template design services are available. Contact me for information about rates and availability.