Documentacion DAP

Documentacion DAP

[line-through]#[[flujo-general-BitDubai-V1]]

= Flujo General de la Plataform DAP

= Flujo Desarrollo

El flujo de desarrollo del Asset describe lo necesario para generar un Digital Asset.
El objeto DigitalAsset de la plataforma, contará con las siguientes propiedades:

 * Identity Asset Issuer: Identity del Actor AssetIssuer
 * String name: nombre del Asset definido por el issuer.
 * String description: descripción del asset definido por el issuer.
 * List<Resource>: lista de recursos posibles del asset, como por ejemplo imágenes, videos, etc.
 * String GenesisTransaction: hash de la transacciín bitcoin donde el asset está incluido.
 * Address GenesisAddress: dirección Genesis generada por la Asset Vault
 * long GenesisAmount: valor crypto del digital asset en Satoshis.

Como contrato básico para el Digital Asset para poder cerrar el ciclo en el desarrollo inicial de la plataforma,
especificaremos que:

 * Redimible
 * ExpirationDate: fecha de expiración que puede ser nunca.

Una vez completadas estas propiedades por parte del Issuer, y cerrado y decidido a crear el Digital Asset sin más cambios,
la Asset Issuer subApp, inicia una Digital Asset Transaction de Asset Issuing en la que lo siguiente ocurre:

* Se solicita a la Crypto Vault la creación de una transaction bitcoin la cual devuelve un hash. Este hash se define como la GenesisTransaction
en el Digital Asset.
* Se solicita a la Asset Vault una dirección bitcoin que será la GenesisAddress
* Se completa la crypto transacción especificando el GenesisAmount.
* Se ejecuta un hash del Digital Asset y el mismo se coloca en el OP_OUTPUT de la crypto transacción.
* Se commitea la transacción y la misma es publicada a la red de bitcoin a través de la Crypto Network.
* El Digital Asset es considerado creado y cerrado.

[Manuel completar más info de la transacción]

Realizadas estas operaciones, y al momento de recibir el GenesisAmount en la Asset Vault, las operaciones de crédito dentro de la
Asset Wallet deben ocurrir.

 * Registro en Asset Issuer Wallet

Es importante mencionar que este proceso realizara una transaccion contable al Asset del Issuer, aumentando la cantidad de Asset originado por el Asset en su creacion,
para que cuando sea enviado a un Asset User, le sea debitado dicho asset a la wallet Issuet y acreditados a la walletAsset User, de esta forma podriamos conocer cuantos Asset de ese tipo
han sido creados y estan disponible para su envio, todo esta operacion debe ser originado mediante una transaccion de la capa transaction.

* Registro en Asset User Wallet

Este proceso realizara una transaccion contable a la wallet Asset del User, aumentando la cantidad de Asset User enviado por el Asset Issuer, y acreditando dicho asset a la wallet Issuet,
de esta forma podriamos conocer cuantos Asset de ese tipo tiene disponible el Aset User para su intercambio, todo esta operacion debe ser originado mediante una transaccion
de la capa transaction.

* Registro en Redeem Point Wallet

Este proceso determina el intercambio de la wallet Asset User donde mediante un metodo donde se pase las direcciones correspondiente se realizara una transaccion contable a la Asset del User, disminuyendo la cantidad de la wallet Asset User,
y acreditando o retornando dicho asset a la Asset Issuer en punto Reddem Point o en alguna wallet que determine la Redemcion.
Aclarar que pasa con este Asset si se puede volver a enviar.
Una vez verificada la genesis transaction por parte del Redeem Point, se debe notificar al plugin Outgoing Issuer que debe realizar la transferencia del monto crypto asociado
a la genesis transaction hacía el dispositivo que va a recibir el digital asset para su "redemción". Una vez que se produzca la transferencia del monto del Digital Asset
el dispositivo del beneficiero del asset debe proceder a registrar el consumo del mencionado Asset y proceder a su inhabilitación (¿?) del dispositivo y registrarlo como consumido.

= Funciones del Asset Issuing

El plugin Asset Issuing será el encargado de la creación de los Assets, incluyendo metadata y crypto monto, que posteriormente serán enviados a los posibles destinatarios de los Digital Assets. Este plugin
deberá solicitar la genesis transaction, la cual se define como la transacción origen de los crypto montos a ser transferidos junto al asset, así mismo, debe solicitar
la genesis address, la cual se define como la dirección que emitió el Asset. Una vez obtenida la información antes mencionada, este plugin debe crear las transacciones que
permitan la transferencia de los Assets a los destinatarios. Este plugin debe ser responsable de cada uno de los Assets generados hasta que todos hayan sido entregados a los
destinatarios. Este plugin debe tener la capacidad de monitorear que Asset ha sido entregado y registrarlo (aun no veo donde) para su posterior consumo. Si un asset no ha
sido registrado como entregado por este plugin, ese Asset no debería ser consumido por el beneficiario.
Estos Assets pueden ser enviados a cualquier tipo de wallet, es decir, debe ser transparente para este plugin los destinatarios de los assets, la metadata que se entrega a
 este plugin debe incluir la información que permita discriminar a que tipo de wallet/destinatario va dirigido el Asset, este proceso de filtrado podría tomarse de las wallets
 instaladas actualmente, este plugin debe recibir de una capa superior el criterio de selección de los destinatarios, de tal forma que este pueda crear la transación que permita
 enviar el asset al destinatario.
Los Assets que no se hayan enviado deben permanecer en el dispositivo emisor hasta que el beneficiario haya autorizado la instalación/descarga del digital asset
a su dispositivo. Al momento de instalación del Asset en el dispositivo destino es que se debe realizar la trasferencia del crypto monto al dispositivo destino.
Los crypto montos del emisor asociados a cada assets deben estar reservados en el dispositivo emisor hasta que sean transferidos a los beneficiarios. Estos montos no deben
cambiar una vez reservados.

*Preguntar al team:*
¿Es posible que el emisor elimine/cancele un asset a discreción?

= Transferencia y confirmación de Asset a beneficiario

= Notificación en Incoming Issuer

El plugin Incoming Issuer en el lado del consumidor o beneficiario será el encargado de monitorear la existencia de un Digital Asset (de alguna manera, por definir) que esté dirigido para la Asset Wallet del usuario,
en caso de recibir la notificación de la existencia de un Digital Asset, este plugin deberá verificar la metadata que debe estar incluida en el Asset,  esta comprobación debe
incluir la verificación de la genesis Transaction, con lo cual se puede verificar el origen del Asset.
Una vez verificado el origen del Asset, el plugin debe lanzará una notificación a una capa superior para que el usuario esté en cuenta de la existencia de este Asset.
Es de hacer notar que hasta el momento, se piensa que el usuario va a autorizar si el Asset va a ser descargado en el dispositivo, esto podría cambiar en un futuro,
ya que se plantea que este Asset puede estar "pre-instalado" en el dispositivo. Se podría definir varias posibles condiciones de descarga de un Asset:
 * descargable automáticamente (el Issuer la instala automáticamente en las Asset Wallets destino)
 * descargable por autorización del usuario.
 * preinstalada en la Asset Wallet.
Una vez que la Asset Wallet autoriza la instalación o descarga del Digital Asset, se procede a crear la transacción que va a transferir el monto crypto asociado a la metadata
de la genesis transaction a la Asset Vault del destinatario. Una vez realizada correctamente la transferencia de fondos, se debe notificar a las capas que controlan la UI la
correcta instalación/descarga del Asset.

*Preguntar al team*:
¿Un asset puede ser descargado o instalado por algún otro medio distinto a la red de fermat? ¿Se podría recibir vía código QR?

#

[line-through]#[[flujo-general-BitDubai-V1]]

= Flujo General de la Plataform DAP

El actor ISSUER procede a crear el Asset donde colocara la informacion relevante:

  * Tienen un Emisor, que es alguno de los actores del sistema.
  * Tienen una fecha de expiración. (aunque en algunos casos puede ser que no expiren nunca)
  * Tienen uno o mas elementos multi-media (imagen, sonido, video, animación, etc.) destinados a
    representar ante el usuario final dicho asset.
  * Son de un determinado tipo.
  * Pueden ser redimido.
  * Tienen una lista de condiciones que dependen de su tipo. (Las entienden cualquier wallet que maneje
    ese tipo de Assets)
  * Son de un determinado sub-tipo.
  * Tienen una lista de condiciones que dependen de su sub-tipo. (Las entienden cualquier wallet que
    maneje ese sub-tipo de Assets)
  * Tienen una lista de condiciones que dependen de su emisor. (Solo las entienden las wallets del emisor)

El Issuer determinara en que momento el Asset estara disponible para su envio o publicacion a las Wallets
instaladas en otros dispositivos y de manera asincrona comenzara la instalacion del Asset, siempre y
cuando se haya validado algunas informacion tecnica del Asset, por ejemplo que no haya caducado,
el actor User, debera de una manera aceptar
el Asset recibido.

Deberiamos de buscar la manera de saber el codigo de la billetera para proceder a enviar el asset.

Interacion y Flujo de pluggin, subapp, explicacion.

= Flujo Desarrollo

El flujo de desarrollo del Asset describe lo necesario para generar un Digital Asset.
El objeto DigitalAsset de la plataforma, contará con las siguientes propiedades:

 * Identity Asset Issuer: Identity del Actor AssetIssuer
 * String name: nombre del Asset definido por el issuer.
 * String description: descripción del asset definido por el issuer.
 * List<Resource>: lista de recursos posibles del asset, como por ejemplo imágenes, videos, etc.
 * String GenesisTransaction: hash de la transacciín bitcoin donde el asset está incluido.
 * Address GenesisAddress: dirección Genesis generada por la Asset Vault
 * long GenesisAmount: valor crypto del digital asset en Satoshis.

Como contrato básico para el Digital Asset para poder cerrar el ciclo en el desarrollo inicial de la plataforma,
especificaremos que:

 * Redimible
 * ExpirationDate: fecha de expiración que puede ser nunca.

Una vez completadas estas propiedades por parte del Issuer, y cerrado y decidido a crear el Digital Asset sin más cambios,
la Asset Issuer subApp, inicia una Digital Asset Transaction de Asset Issuing en la que lo siguiente ocurre:

* Se solicita a la Crypto Vault la creación de una transaction bitcoin la cual devuelve un hash. Este hash se define como la GenesisTransaction
en el Digital Asset.
* Se solicita a la Asset Vault una dirección bitcoin que será la GenesisAddress
* Se completa la crypto transacción especificando el GenesisAmount.
* Se ejecuta un hash del Digital Asset y el mismo se coloca en el OP_OUTPUT de la crypto transacción.
* Se commitea la transacción y la misma es publicada a la red de bitcoin a través de la Crypto Network.
* El Digital Asset es considerado creado y cerrado.

[Manuel completar más info de la transacción]

Realizadas estas operaciones, y al momento de recibir el GenesisAmount en la Asset Vault, las operaciones de crédito dentro de la
Asset Wallet deben ocurrir.

 * Registro en Asset Issuer Wallet

Es importante mencionar que este proceso realizara una transaccion contable al Asset del Issuer, aumentando la cantidad de Asset originado por el Asset en su creacion,
para que cuando sea enviado a un Asset User, le sea debitado dicho asset a la wallet Issuet y acreditados a la walletAsset User, de esta forma podriamos conocer cuantos Asset de ese tipo
han sido creados y estan disponible para su envio, todo esta operacion debe ser originado mediante una transaccion de la capa transaction.

* Registro en Asset User Wallet

Este proceso realizara una transaccion contable a la wallet Asset del User, aumentando la cantidad de Asset User enviado por el Asset Issuer, y acreditando dicho asset a la wallet Issuet,
de esta forma podriamos conocer cuantos Asset de ese tipo tiene disponible el Aset User para su intercambio, todo esta operacion debe ser originado mediante una transaccion
de la capa transaction.

* Registro en Redeem Point Wallet

Este proceso determina el intercambio de la wallet Asset User donde mediante un metodo donde se pase las direcciones correspondiente se realizara una transaccion contable a la Asset del User, disminuyendo la cantidad de la wallet Asset User,
y acreditando o retornando dicho asset a la Asset Issuer en punto Reddem Point o en alguna wallet que determine la Redemcion.
Aclarar que pasa con este Asset si se puede volver a enviar.
Una vez verificada la genesis transaction por parte del Redeem Point, se debe notificar al plugin Outgoing Issuer que debe realizar la transferencia del monto crypto asociado
a la genesis transaction hacía el dispositivo que va a recibir el digital asset para su "redemción". Una vez que se produzca la transferencia del monto del Digital Asset
el dispositivo del beneficiero del asset debe proceder a registrar el consumo del mencionado Asset y proceder a su inhabilitación (¿?) del dispositivo y registrarlo como consumido.

= Funciones del Asset Issuing

El plugin Asset Issuing será el encargado de la creación de los Assets, incluyendo metadata y crypto monto, que posteriormente serán enviados a los posibles destinatarios de los Digital Assets. Este plugin
deberá solicitar la genesis transaction, la cual se define como la transacción origen de los crypto montos a ser transferidos junto al asset, así mismo, debe solicitar
la genesis address, la cual se define como la dirección que emitió el Asset. Una vez obtenida la información antes mencionada, este plugin debe crear las transacciones que
permitan la transferencia de los Assets a los destinatarios. Este plugin debe ser responsable de cada uno de los Assets generados hasta que todos hayan sido entregados a los
destinatarios. Este plugin debe tener la capacidad de monitorear que Asset ha sido entregado y registrarlo (aun no veo donde) para su posterior consumo. Si un asset no ha
sido registrado como entregado por este plugin, ese Asset no debería ser consumido por el beneficiario.
Estos Assets pueden ser enviados a cualquier tipo de wallet, es decir, debe ser transparente para este plugin los destinatarios de los assets, la metadata que se entrega a
 este plugin debe incluir la información que permita discriminar a que tipo de wallet/destinatario va dirigido el Asset, este proceso de filtrado podría tomarse de las wallets
 instaladas actualmente, este plugin debe recibir de una capa superior el criterio de selección de los destinatarios, de tal forma que este pueda crear la transación que permita
 enviar el asset al destinatario.
Los Assets que no se hayan enviado deben permanecer en el dispositivo emisor hasta que el beneficiario haya autorizado la instalación/descarga del digital asset
a su dispositivo. Al momento de instalación del Asset en el dispositivo destino es que se debe realizar la trasferencia del crypto monto al dispositivo destino.
Los crypto montos del emisor asociados a cada assets deben estar reservados en el dispositivo emisor hasta que sean transferidos a los beneficiarios. Estos montos no deben
cambiar una vez reservados.

*Preguntar al team:*
¿Es posible que el emisor elimine/cancele un asset a discreción?

= Transferencia y confirmación de Asset a beneficiario

= Notificación en Incoming Issuer

El plugin Incoming Issuer en el lado del consumidor o beneficiario será el encargado de monitorear la existencia de un Digital Asset (de alguna manera, por definir) que esté dirigido para la Asset Wallet del usuario,
en caso de recibir la notificación de la existencia de un Digital Asset, este plugin deberá verificar la metadata que debe estar incluida en el Asset,  esta comprobación debe
incluir la verificación de la genesis Transaction, con lo cual se puede verificar el origen del Asset.
Una vez verificado el origen del Asset, el plugin debe lanzará una notificación a una capa superior para que el usuario esté en cuenta de la existencia de este Asset.
Es de hacer notar que hasta el momento, se piensa que el usuario va a autorizar si el Asset va a ser descargado en el dispositivo, esto podría cambiar en un futuro,
ya que se plantea que este Asset puede estar "pre-instalado" en el dispositivo. Se podría definir varias posibles condiciones de descarga de un Asset:
 * descargable automáticamente (el Issuer la instala automáticamente en las Asset Wallets destino)
 * descargable por autorización del usuario.
 * preinstalada en la Asset Wallet.
Una vez que la Asset Wallet autoriza la instalación o descarga del Digital Asset, se procede a crear la transacción que va a transferir el monto crypto asociado a la metadata
de la genesis transaction a la Asset Vault del destinatario. Una vez realizada correctamente la transferencia de fondos, se debe notificar a las capas que controlan la UI la
correcta instalación/descarga del Asset.

*Preguntar al team*:
¿Un asset puede ser descargado o instalado por algún otro medio distinto a la red de fermat? ¿Se podría recibir vía código QR?

#


Update 2016-05-27-My-First-Post-with-Hub-Press.adoc

= My First Post with HubPress
:hp-tags: hubpress, test, new
:hp-image: students-703001_1920.jpg

I kindly ask you to shoot an email to help@about.me with the subject "Namecheap +About.me". Please add your about.me username and zorgan.me domain to the email.

= My First Post with HubPress
:hp-tags: hubpress
:hp-image: notebook-405755_1280.jpg

I kindly ask you to shoot an email to help@about.me with the subject "Namecheap +About.me". Please add your about.me username and zorgan.me domain to the email.


Update 2016-09-04-Hugo-No-Go-Felicitous-Update.adoc

= Hugo: No Go (Felicitous Update)
:hp-tags: front-end, github, hugo, static site generator, hosting, push, workflow, automation, themes, deployment

My Hugo experience has been really frustrating. I guess https://discuss.gohugo.io/t/i-am-really-confused-as-to-how-to-deploy-hugo-with-github/3669/7[I am not the only one].

**(You can skip all the whining and scroll down to where Surge turns out to be a very satisfactory partial solution. --Editor)**

With Hugo, I am able to build and serve locally all manner of remarkable sites with ease. It rocks. It rolls. It has an great big and extremely creative community.

But I just cannot seem to deploy these sites, to GitHub Pages or anywhere else.

I have followed http://codethejason.github.io/blog/setupghpages/[these instructions], and variants for deploying to a personal or org page.

I have tested Netflify and Surge and am testing https://gohugo.io/tutorials/automated-deployments/[Wercker] and https://travis-ci.org/gringolalia/gringolalia[Travis] and others. I am making a list. I am checking it twice.

I have checked out a couple of https://github.com/spencerlyon2/hugo_gh_blog/blob/master/deploy.sh[deploy scripts] as well, including http://heiber.im/post/switching-to-hugo/[this one] for deploying to personal or org pages rather than project pages.

image::netlifyfailurescale.png[]

=== Google Search & Deploy

In the case of Netflify and http://harmonious-advertisement.surge.sh/[Surge], the deploy succeeds, but the theme is missing. Just raw HTML, though 	there are folders and files for CCS, JS and SASS.

When I build the site with `hugo --theme=ghostwriter` and upload it to my local Web server, same problem.

I should not need to specify the stylesheet as it is in my `config.toml`

Stylesheets and other elements of style are present in the deployed `public/` folder but the theme does not load.

A fellow sufferer suggests removing the `.git` folder from the `themes/ghostwriter` folder.

To no effect. I follow the Netflify tutorial and get the `no configuration file found` error.

> 12:27:23 PM: Error running command: Build script returned non-zero exit code: 255

I copy the `gringolalia` folder into one called `glossalia`, remove the `.git` folder, create a new project with that name on gringolalia@github  and repeat `git init` and all that stuff.

I really have to stop floundering and draw up a plan of attack. Hugo is a great tool with a wealth of themes to adapt and study, but I need to have time to actually write in https://brasilianas.github.io/[these writing spaces I keep setting up for myself].

Ghostwriter in Hugo is not *so* fantastic that I just have to have it in my portfolio. But I do like it.

I want to write criticism and essays in the genre of Camus and a cheap spy novel about money laundering and Uruguay and Paraguay. I have https://brasilianas.github.io/[a thing about Paraguay], this is true.  I explain it https://gringolalia.surge.sh/2016/09/07/porque-sambodia/[here, in Portuguese]. I have decided to write all my front end development and design notes in Portuguese, for the practice.

=== Update: Surge and My Urge to Merge

image::surgetogringolalia.png[Urge to Surge]

Okay, the second time through the list of third-party deploy options I discover that http://griongolalia.surge.sh[Surge] is the answer to my need to post sites to URLs that I can show to clients.

Surge totally rocks. `npm install --global surge` and then just plain `surge` in your project directory.

Prompts you for the folder to push and the subdomain and **bazunga**!

What is their business model, I wonder?

But this solution does not let me clone and push from a clone repo another computer, like the one in the https://en.wikipedia.org/wiki/Cancer_Ward[Cancer Ward] (Раковый Корпус). So I cannot push things like theme changes with version control. Handy to be able to check out your homework during your rehab down time, even if the Window XP machine -- you heard right -- is agonizingly slow.

http://prose.io/#bretonio[Prose.io] is one way to work from remote locations: Brazilian LAN houses or a laptop in a café or a chic *padaria* with WiFi or such like. But then you get those problems when you `git pull`. I really need to study `git` more systematically.

Is there any online courseware on this subject?

https://github.com/github/training-kit[Of course there is]!

Forking to https://github.com/bretonio[bretonio] ...

I audited https://www.coursetalk.com/providers/coursera/courses/social-network-analysis[a great Coursera course] last year on social network analysis and visualization with https://gephi.org/[Gephi].

I was running my own crawls using http://www.cwr.cl/projects/WIRE/doc/[WIRE], https://tupiwire.wordpress.com/2014/05/01/spidermen-wire-pajek/[churning the data in Pajek], and visualizing the results.

A big waste of my potentially productive -- money-earning --time, but I enjoyed it a lot.  One of these days I will look into link analysis with http://wiki.apache.org/nutch/NewScoring[Nutch], but for the time being I am busy enough with rehab, writing and studying the fabrication of sites.

= Hugo: No Go (Felicitous Update)
:hp-tags: front-end, github, hugo, static site generator, hosting, push, workflow, automation, themes, deployment

My Hugo experience has been really frustrating. I guess https://discuss.gohugo.io/t/i-am-really-confused-as-to-how-to-deploy-hugo-with-github/3669/7[I am not the only one].

>>You can skip all the whining and scroll down to where Surge turns out to be a very satisfactory partial solution. --Editor

With Hugo, I am able to build and serve locally all manner of remarkable sites with ease. It rocks. It rolls.

But I just cannot seem to deploy these sites, to GitHub Pages or anywhere else.

I have followed http://codethejason.github.io/blog/setupghpages/[these instructions], and variants for deploying to a personal or org page.

I have tested Netflify and Surge and am testing https://gohugo.io/tutorials/automated-deployments/[Wercker] and https://travis-ci.org/gringolalia/gringolalia[Travis] and others. I am making a list. I am checking it twice.

I have checked out a couple of https://github.com/spencerlyon2/hugo_gh_blog/blob/master/deploy.sh[deploy scripts] as well, including http://heiber.im/post/switching-to-hugo/[this one] for deploying to personal or org pages rather than project pages.

image::netlifyfailurescale.png[]

=== Google Search & Deploy

In the case of Netflify and http://harmonious-advertisement.surge.sh/[Surge], the deploy succeeds, but the theme is missing. Just raw HTML, though 	there are folders and files for CCS, JS and SASS.

When I build the site with `hugo --theme=ghostwriter` and upload it to my local Web server, same problem.

I should not need to specify the stylesheet as it is in my `config.toml`

Stylesheets and other elements of style are present in the deployed `public/` folder but the theme does not load.

A fellow sufferer suggests removing the `.git` folder from the `themes/ghostwriter` folder.

To no effect. I follow the Netflify tutorial and get the `no configuration file found` error.

> 12:27:23 PM: Error running command: Build script returned non-zero exit code: 255

I copy the `gringolalia` folder into one called `glossalia`, remove the `.git` folder, create a new project with that name on gringolalia@github  and repeat `git init` and all that stuff.

I really have to stop floundering and draw up a plan of attack. Hugo is a great tool with a wealth of themes to adapt and study, but I need to have time to actually write in https://brasilianas.github.io/[these writing spaces I keep setting up for myself].

Ghostwriter in Hugo is not *so* fantastic that I just have to have it in my portfolio. But I do like it.

I want to write criticism and essays in the genre of Camus and a cheap spy novel about money laundering and Uruguay and Paraguay. I have https://brasilianas.github.io/[a thing about Paraguay], this is true.  I explain it https://gringolalia.surge.sh/2016/09/07/porque-sambodia/[here, in Portuguese]. I have decided to write all my front end development and design notes in Portuguese, for the practice.

=== Update: Surge and My Urge to Merge

image::surgetogringolalia.png[Urge to Surge]

Okay, the second time through the list of third-party deploy options I discover that http://griongolalia.surge.sh[Surge] is the answer to my need to post sites to URLs that I can show to clients.

Surge totally rocks. `npm install --global surge` and then just plain `surge` in your project directory.

Prompts you for the folder to push and the subdomain and **bazunga**!

What is their business model, I wonder?

But this solution does not let me clone and push from a clone repo another computer, like the one in the https://en.wikipedia.org/wiki/Cancer_Ward[Cancer Ward] (Раковый Корпус). So I cannot push things like theme changes with version control. Handy to be able to check out your homework during your rehab down time, even if the Window XP machine -- you heard right -- is agonizingly slow.

http://prose.io/#bretonio[Prose.io] is one way to work from remote locations: Brazilian LAN houses or a laptop in a café or a chic *padaria* with WiFi or such like. But then you get those problems when you `git pull`. I really need to study `git` more systematically.

Is there any online courseware on this subject?

https://github.com/github/training-kit[Of course there is]!

Forking to https://github.com/bretonio[bretonio] ...

I audited https://www.coursetalk.com/providers/coursera/courses/social-network-analysis[a great Coursera course] last year on social network analysis and visualization with https://gephi.org/[Gephi].

I was running my own crawls using http://www.cwr.cl/projects/WIRE/doc/[WIRE], https://tupiwire.wordpress.com/2014/05/01/spidermen-wire-pajek/[churning the data in Pajek], and visualizing the results.

A big waste of my potentially productive -- money-earning --time, but I enjoyed it a lot.  One of these days I will look into link analysis with http://wiki.apache.org/nutch/NewScoring[Nutch], but for the time being I am busy enough with rehab, writing and studying the fabrication of sites.


Work Ansible and rolling

Work Ansible and rolling

= Ansible and rolling upgrades

:hp-tags: ansible, elasticsearch, kafka, cassandra
:hp-image: /images/logos/ansible.png
:source_dir: ../sources/2017-07-05-Ansible-and-rolling-upgrades
:image_dir: 2017-07-05-Ansible-and-rolling-upgrades
:published_at: 2017-01-15

Ansible is a nice tool to deploy distributed systems like Elasticsearch, Kafka, Cassandra and the like.
These distributed systems are built to avoid downtime and allow partial failures.
Upgrading these softwares, or updating their configuration, requires restarting each member of the cluster.

The aim of the this article is to describe an Ansible pattern I discovered
trying to improve deployment speed
and yet being able to guarantee availability during the upgrade process.
To improve deployment, you need to deploy all hosts in parallel.
To guarantee availability, you can't stop all host at the same time: you must deploy each host one after the other.

== The problem

Let's take the Elasticsearch example.

The tasks required to deploy Elasticsearch on each host of the cluster are gathered in an `elasticsearch` role.
Then this role is applied to all hosts belonging to the `elasticsearch` group in the Ansible playbook.

.playbook.yml
[source,yaml]
----
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
----

At some point, the `elasticsearch` role probably contains something to stop Elasticsearch in order to apply changes:

.roles/elasticsearch/tasks/main.yml
[source,yaml]
----
...
- name: Stop service
  become: true
  service:
    name: elasticsearch
    state: stopped
    enabled: true
...
----
If we do that, all Elasticsearch nodes will be stopped at the same time, making the whole cluster unavailable.
To fix this problem, the role can be ran in serial instead of parallel:

.deploy.yml
[source,yaml]
----
- hosts: es
  serial: 1 # Force serial execution
  roles:
    - role: elasticsearch
----
But deploying every host in serial takes a very long time and makes deployment endless.

== Refactoring the role

The main idea is to split the role into multiple steps, and do as much as possible in parallel:

1. *In Parallel*: Deploy system settings, software settings, software binaries.
  But don't stop anything and don't remove anything.
2. *In Serial*:
  * Stop the node
  * Install or upgrade the node as quickly as possible
  * Start the node
3. *In Parallel*: finish applying settings on running cluster and remove old files and binaries.

Then descibe each step in its own YAML file: `before.yml`, `stop_start.yml` and `after.yml`.
Finally, the role entry point `main.yml` will include the appropriate step using a variable named `elasticsearch_step`:

.roles/elasticsearch/tasks/main.yml
[source,yaml]
----
- name: "Running step {{ elasticsearch_step }}"
  include: "{{ elasticsearch_step }}.yml"
----

We can also create a fake step including all the steps (more on that later):

.roles/elasticsearch/tasks/all.yml
[source,yaml]
----
- include: "before.yml"
- include: "stop_start.yml"
- include: "after.yml"
----

== Refactoring the playbook

Now in the playbook we will call the role 3 times, each individual step being called independently.

.playbook.yml
[source,yaml]
----
# In parallel
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "before"

# In serial
- hosts: elasticsearch
  any_errors_fatal: true
  serial: 1
  roles:
    - role: elasticsearch
      elasticsearch_step: "stop_start"

# In parallel
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "after"
----

To install a brand new cluster, there is nothing to stop, and we don't fear anything.
In this particular case, this role can still be used, and the `stop_start` step can be called in parallel:

.playbook.yml
[source,yaml]
----
# New cluster
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "before"
    - role: elasticsearch
      elasticsearch_step: "stop_start"
    - role: elasticsearch
      elasticsearch_step: "after"
----

Or even simpler, we can use `all` step:

.playbook.yml
[source,yaml]
----
# New cluster
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "all"
----

== Hot configuration

Most of the time, I am running my Ansible playbook only to change settings that don't need nodes to be restarted.
The trick is here to detect in the `before` step whether nodes should be restarted or not:
Version upgrade? Configuration that can not be hot reloaded?
On lucky days, you can skip the expensive `stop_start` step and have a quick deploy.

= Ansible and rolling upgrades

:hp-tags: ansible, elasticsearch, kafka, cassandra
:hp-image: /images/logos/ansible.png
:source_dir: ../sources/2017-07-05-Ansible-and-rolling-upgrades
:image_dir: 2017-07-05-Ansible-and-rolling-upgrades
:published_at: 2017-01-15

Ansible is a nice tool to deploy software associated configuration for distributed systems like Elasticsearch, Kafka, Cassandra and the like.
These distributed systems are built to avoid downtime and allow partial failures.
Upgrading these softwares or updating their configuration requires restarting each member of the cluster.

The aim of the this article is to explain an Ansible pattern I discovered deploying trying to improve deployment speed
and yet being able to guarantee availability during the upgrade process.
To improve deployment, you need to deploy all hosts in parallel.
To guarantee availability, you can't stop all host at the same: you must deploy each host one after the other.

== The problem

Let's take the Elasticsearch example.
The tasks required to deploy Elasticsearch on each host of the cluster are gather in an `elasticsearch` role.
Then this role is applied to all hosts belonging to the `elasticsearch` group in the Ansible playbook.

.playbook.yml
[source,yaml]
----
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
----

The `elasticsearch` role probably contains something to stop Elasticsearch in order to apply changes:

.roles/elasticsearch/tasks/main.yml
[source,yaml]
----
...
- name: Stop service
  become: true
  service:
    name: elasticsearch
    state: stopped
    enabled: true
...
----
If we do that, all Elasticsearch nodes will be stopped at the same time and users will cry all their tears.
To fix this problem, the role can be ran in serial instead of parallel:

.deploy.yml
[source,yaml]
----
- hosts: es
  serial: 1 # Force serial execution
  roles:
    - role: elasticsearch
----
But deploying every host in serial makes deployment endless.

== Refactoring the role

The main idea is to split the role into multiple steps:

1. *In Parrallel*: deploy as much things as possible: system settings, software settings, software binaries.
  But don't stop anything and don't remove anything.
2. *In Serial*:
  * Stop the node
  * Install or upgrade the node as quickly as possible
  * Start the node
3. *In Parallel*: finish applying settings on running cluster and remove old files and binaries.

Each step is described in its own YAML file: `before.yml`, `stop_start.yml` and `after.yml`.
The role entry point routes to the appropriate step using role variable named `elasticsearch_step`:

.roles/elasticsearch/tasks/main.yml
[source,yaml]
----
  - name: "Running step {{ elasticsearch_step }}"
    include: "{{ elasticsearch_step }}.yml"
----

== Refactoring the playbook

Now in the playbook we will call the role 3 times, each step being called independently.

.playbook.yml
[source,yaml]
----
# In parallel
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "before"

# In serial
- hosts: elasticsearch
  any_errors_fatal: true
  serial: 1
  roles:
    - role: elasticsearch
      elasticsearch_step: "stop_start"

# In parallel
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "after"
----

To install a brand new cluster, there is nothing to stop, and we don't fear anything.
This role can still be used, and the `stop_start` step can be called in parallel:

.playbook.yml
[source,yaml]
----
# New cluster
- hosts: elasticsearch
  any_errors_fatal: true
  roles:
    - role: elasticsearch
      elasticsearch_step: "before"
    - role: elasticsearch
      elasticsearch_step: "stop_start"
    - role: elasticsearch
      elasticsearch_step: "after"
----


Update README.adoc (#1189)

Update README.adoc (#1189)

change the get messages mapping

= Google Cloud Reactive Pub/Sub Code Sample

This code sample demonstrates consuming data from a GCP Cloud Pub/Sub subscription as a reactive stream.

Acquiring and processing the reactive stream is done in link:src/main/java/com/example/ReactiveController.java/[ReactiveController.java].
Each message in the stream is acknowledged with `doOnNext()` operator.
Then the `map()` operator converts the message's payload into a proper `String` and returns it to the subscriber:

[source,java]
----
return flux
  .doOnNext(message -> {
    System.out.println("Received a message: " + message.getPubsubMessage().getMessageId());
    message.ack();
  })
  .map(message -> new String(
	  message.getPubsubMessage().getData().toByteArray(), Charset.defaultCharset()));
----

The other two classes (link:src/main/java/com/example/ReactiveReceiverApplication.java/[ReactiveReceiverApplication.java] and link:src/main/java/com/example/MessageSenderController.java/[MessageSenderController.java]) boostrap the application and simplify publishing test data to a topic.

== Setup & Configuration

image:http://gstatic.com/cloudssh/images/open-btn.svg[link=https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspring-cloud-gcp&cloudshell_open_in_editor=spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-reactive-sample/README.adoc]

1. Configure your GCP project ID and credentials by following link:../../docs/src/main/asciidoc/core.adoc#project-id[these instructions].
+
Alternatively, if you have the https://cloud.google.com/sdk/[Google Cloud SDK] installed and initialized, and are logged in with https://developers.google.com/identity/protocols/application-default-credentials[application default credentials], Spring will auto-discover those parameters for you.

2. Go to the https://console.cloud.google.com/cloudpubsub/topicList[Google Cloud Console Pub/Sub topics page] and create a topic called `exampleTopic`.

3. Still in the same page, locate the newly created topic, click the button with the three vertical dots at the end of the topic's line and click "New subscription".
Create a new subscription called `exampleSubscription` with all default parameters.

4. Run `$ mvn clean install` from the root directory of the project.

5. In a terminal window, move into this directory (`spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-reactive-sample`) and run:

  mvn spring-boot:run

6. In a browser, open http://localhost:8080, or use the `Web Preview` button in Cloud Shell to preview the app on
port 8080, and send some (`N`) messages to the GCP Pub/Sub topic.
The messages sent will all start with the message text provided, and end with a number from `0` to `N-1`.
Observe that nothing is received by the sample application, since there are no subscribers yet.

7. In another terminal, open a connection to `/getMessages` streaming endpoint.
This stream will remain open until you stop it with `Ctrl+C`.

  curl localhost:8080/getMessages

8. Verify that the `curl` terminal received the messages, while the application itself logged their IDs.

9. Publish some more messages from http://localhost:8080 or from Cloud Shell `Web Preview`;
observe that the additional data is streamed to the `curl` client.

= Google Cloud Reactive Pub/Sub Code Sample

This code sample demonstrates consuming data from a GCP Cloud Pub/Sub subscription as a reactive stream.

Acquiring and processing the reactive stream is done in link:src/main/java/com/example/ReactiveController.java/[ReactiveController.java].
Each message in the stream is acknowledged with `doOnNext()` operator.
Then the `map()` operator converts the message's payload into a proper `String` and returns it to the subscriber:

[source,java]
----
return flux
  .doOnNext(message -> {
    System.out.println("Received a message: " + message.getPubsubMessage().getMessageId());
    message.ack();
  })
  .map(message -> new String(
	  message.getPubsubMessage().getData().toByteArray(), Charset.defaultCharset()));
----

The other two classes (link:src/main/java/com/example/ReactiveReceiverApplication.java/[ReactiveReceiverApplication.java] and link:src/main/java/com/example/MessageSenderController.java/[MessageSenderController.java]) boostrap the application and simplify publishing test data to a topic.

== Setup & Configuration

image:http://gstatic.com/cloudssh/images/open-btn.svg[link=https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fspring-cloud-gcp&cloudshell_open_in_editor=spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-reactive-sample/README.adoc]

1. Configure your GCP project ID and credentials by following link:../../docs/src/main/asciidoc/core.adoc#project-id[these instructions].
+
Alternatively, if you have the https://cloud.google.com/sdk/[Google Cloud SDK] installed and initialized, and are logged in with https://developers.google.com/identity/protocols/application-default-credentials[application default credentials], Spring will auto-discover those parameters for you.

2. Go to the https://console.cloud.google.com/cloudpubsub/topicList[Google Cloud Console Pub/Sub topics page] and create a topic called `exampleTopic`.

3. Still in the same page, locate the newly created topic, click the button with the three vertical dots at the end of the topic's line and click "New subscription".
Create a new subscription called `exampleSubscription` with all default parameters.

4. Run `$ mvn clean install` from the root directory of the project.

5. In a terminal window, move into this directory (`spring-cloud-gcp-samples/spring-cloud-gcp-pubsub-reactive-sample`) and run:

  mvn spring-boot:run

6. In a browser, open http://localhost:8080, or use the `Web Preview` button in Cloud Shell to preview the app on
port 8080, and send some (`N`) messages to the GCP Pub/Sub topic.
The messages sent will all start with the message text provided, and end with a number from `0` to `N-1`.
Observe that nothing is received by the sample application, since there are no subscribers yet.

7. In another terminal, open a connection to `/getmessages` streaming endpoint.
This stream will remain open until you stop it with `Ctrl+C`.

  curl localhost:8080/getmessages

8. Verify that the `curl` terminal received the messages, while the application itself logged their IDs.

9. Publish some more messages from http://localhost:8080 or from Cloud Shell `Web Preview`;
observe that the additional data is streamed to the `curl` client.