Delete the file at '_posts/2017-08-18-or-how-Ansible-Galaxy-can-short-your-way-to-awesomeness.adoc'

= How a developer could short their way on the Ansible learning
// See https://hubpress.gitbooks.io/hubpress-knowledgebase/content/ for information about the parameters.
// :hp-image: /covers/galaxy_cover.jpg
:published_at: 2017-08-18
:hp-tags: Ansible, Ansible_Galaxy
:hp-alt-title: ...or how Ansible Galaxy can short your way to awesomeness

You're a developer and gets starting feeling anxious to learn about Ansible (everybody is talking about it), but you barely know how to list a directory on a Linux machine! Don't worry, you could use your developer skills to learn Ansible and starting doing some magic to build your way towards DevOps in a short time with a little help of Ansible Galaxy.

So you've learned http://docs.ansible.com/ansible/latest/intro.html[all the basics of Ansible] and now are arguing yourself: where's the patterns? How do I start using Ansible on a regular basis? How can I do this or that? I'm newbie on OS and I barely know where to start..

I don't know if I'll provide the answers for all these questions, but I may help guide your way through it.

Quoting from the https://galaxy.ansible.com/intro[official Ansible Galaxy's documentation]:
____
Ansible Galaxy is Ansible’s official community hub for sharing Ansible roles. A role is the Ansible way of bundling automation content and making it reusable.
____

**Reusable**. That's the word IT people always (or should) care about. Roles on Ansible is a great way to reuse a piece of task that you always have to do when provisioning some environment. I won't extend myself here on how to create a role, for that please take a look on the http://docs.ansible.com/ansible/latest/playbooks_roles.html[official docs] that you should have your answers.

Instead, I'd like to bring your attention on Ansible Galaxy and how you can reuse someone else's work or contribute to it while learning something great. Galaxy is the https://hub.docker.com/[Docker Hub] for docker images or the Github for open source projects. There you'll find a lot of roles people already did so you could take them and orchestrate your playbook on a way that it could do barely anything you need.

So Ansible Galaxy can help you with: (a) learn by example by looking on the code other people already provided and; (b) create great playbooks by orchestrating with roles that have been done by the community.

Let's take a look on a https://bitbucket.org/tecnobizz/vagrant-alm[project I'm working on] that's provisioning a environment to set up a sort of an https://en.wikipedia.org/wiki/Application_lifecycle_management[ALM process]. For that, I need to provision three servers: https://jenkins.io/[Jenkins] for CI/CD process, https://www.sonarsource.com/[SonarSource] for quality analysis and http://www.sonatype.org/nexus/[Nexus] for artifact repository.

To provision all the three servers requires a lot of vanilla work that I, as an only enthusiast of Ansible and OS in general surely will take *a lot* of time to do so. And we don't have that time, right? Take a deep breath, a couple of coffee and go along with me.

=== 1) Define what you need to do

First things first. Draw and write down everything you need to provision: a CentOS machine? Needs NodeJs? Java? PHP? What else? What are the application's requirements? Try to be as detailed as you need to. Take your time to plan your way down the hill, so you'll save time ahead.

For this project, I just need the aforementioned 3 servers: Jenkins, SonarSource and Nexus on the latest version possible and, of course, everything that implies on it: a database server, JVM, web server and etc. Everything that I know that Ansible may provision for me.

=== 2) Prepare your environment

Start by http://docs.ansible.com/ansible/latest/intro_installation.html[installing Ansible]. After that, be sure that Ansible Galaxy is on your path by typing this on your console:

```
$ ansible-galaxy --version
```

You should see an output like this:

```
ansible-galaxy 2.3.1.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
python version = 2.7.13 (default, Jun 26 2017, 10:20:05) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
```

Now configure your `ANSIBLE_ROLES_PATH`. This path is where Ansible Galaxy will save every role you install and where's Ansible will look at when resolving the imports from your playbook. The http://docs.ansible.com/ansible/latest/galaxy.html#roles-path[default path] is `/etc/ansible/roles`. Make sure that the user who's installing the roles have write permissions. If you are running on your own machine, make it yours:

```
$ sudo chown -R myuser:myuser /etc/ansible/roles
```

Then you should work on a virtual machine environment. For that you could use http://docs.ansible.com/ansible/latest/guide_vagrant.html[Vagrant on top of libvirt].

And that's it! You should have your environment set up. For more detailed information about the installation and configuration of Ansible Galaxy, please http://docs.ansible.com/ansible/latest/galaxy.html[check the docs].

=== 3) Looking for roles that you need

So now you have the plan for what you need to do and the Ansible Galaxy configured on your environment. Let's search for the roles that we need to **compose** our playbook so we can provision all the three servers at once.

Remember that you just need to bring all these roles from the Galaxy repository and run them on a orchestrate fashion to do what you need.

You have at least three ways to search for roles:

1 - **Ansible Galaxy command line**:

```
$ ansible-galaxy search *jenkins*
```

You'll see a list of roles with the name you provide as an argument. It's a simple and fast way to look for a role, but not my preferable way. This list only provides the name and a short description for the role. This must be suffice if what you are looking for is not so popular. But like the example above, the result may return a lot of options. What to choose?

2 - **Ansible Galaxy Web UI Search**.

Go to https://galaxy.ansible.com/[the Ansible Galaxy web site] and start exploring or browsing for a role. Let's use the same example as before and just search for Jenkins. Click on "Browse Roles" and type "Jenkins" on keyword (or just https://galaxy.ansible.com/list#/roles?page=1&page_size=10&autocomplete=jenkins[click here]). You'll see a nice interface and a sort function that will make all the difference: `Stargazers`. That means we can filter by the most relevant and upvoted roles by the community. Prefer to take that one with the most starts. For this example, the chosen role will be the https://galaxy.ansible.com/geerlingguy/jenkins/[geerlingguy one].

3 - **Google it**.

The mindset here is the same as #2, but now you may hit on the Github repository instead of on the Ansible Galaxy web site. Hint: it's almost the same thing. Every Ansible Galaxy project has a Github backed project. So, guess what? It's open and you can (and should) contribute to them.

Now that we have our role(s) we can move forward and start using it.

=== 4) Read the docs, explore their code

First things first. Read all available documentation on the project you choose and pay attention on the extension points. These normally are variables you should set to get the role do what you need. Take for example the https://github.com/geerlingguy/ansible-role-java[Geerling Guy's Java role].

On this role you could set the Java version you want to be installed just by setting the `java_packages` variable.

Go ahead and install the role on your machine:

```
$ ansible-galaxy install geerlingguy.jenkins
```

You have to have internet connection so Ansible could connect on the repository and bring the role and all the dependencies it needs (maven, anyone?). After the installation process, check for the role installed on your system:

```
$ ansible-galaxy list
```

You should see the Jenkins role installed on your system provided with the version and all the other dependencies it needs. Ansible Galaxy works like many others package managers over there: it downloads all other required dependencies so you can work with it without any worries. Remember the Ansible Role's path set on step #2? Try to list the directory and you'll see all the roles there too.

Explore the role's creator work by looking for what the role really do. Start by exploring the `$ANSIBLE_ROLES_PATH/roles/geerlingguy.jenkins/tasks/main.yml`. Normally this is the starting point of the role and you really could see what's going on. Try to understand their code and how things are been doing there. It's a very nice exercise to learn by example and to absorb concepts harder to get on books or manuals.

After that, be comfortable with their code and explore everything else, trying to understand how the pieces are working together to make the role that awesome.

=== 5) Test it

It's time to enter in the arena and put some bytes to work for you. You have to create your project, so start small and on a directory, create a tree like this one:

```
├── project.yml
├── group_vars
│   ├── all.yml
├── inventory.ini
├── meta
│   └── main.yml
├── requirements.yml
```

Let's describe one by one:

1 - `project.yml`: this will be the starting point of your project, so rename it as you pleased. To start using the role we've just installed, open the file and put this on it:

```
- name: Deploy Jenkins CI
  hosts: jenkins_server
  remote_user: the_user
  become: yes

  roles:
    - geerlingguy.jenkins
```

We are creating a task to just run the role given it a name, defining a host and the user who will run the process on the target machine. Please note that is beyond of the scope of this post to create a container or a virtual machine where the Ansible will run. I suggest https://www.ansible.com/docker[Docker] or https://www.vagrantup.com/docs/provisioning/ansible.html[Vagrant] to do so.

2 - `inventori.ini` is the file to list the machines Ansible will provision. Read more about it http://docs.ansible.com/ansible/latest/intro_inventory.html[here]. Remember that the `hosts` property described on #1 should match the host created on this file, for example:

```
[jenkins_server]
192.168.0.32
```

3 - `group_vars/all.yml`. Remember about the "role's extension points" I mentioned on step #4? This file contains all the variables that we need to set to configure the role to do what we want. So, go ahead and define the java version for the Jenkins:

```
java_packages:
  - java-1.8.0-openjdk
```

This variable is mentioned on `geerlingguy.java` role that is a dependency from `geerlingguy.jenkins`. Isn't cool how roles can stick together to do great things?

4 - `meta/main.yml`. On this file we are going to list all the dependencies we need from the roles:

```
dependencies:
  - geerlingguy.jenkins
```

This way we can tell Ansible that we need this role to perform our tasks.

5 - `requirements.yml`. Besides the name, this isn't a required file for your project, but it's very handy because we can list all Ansible Galaxy's roles that we need and download then with just one command line:

```
$ ansible-galaxy -r requirements.xml
```

Go ahead and start to provision. Your Jenkins server should be provisioned on your environment and you can start making CI/CD jobs.

=== The Bottom Line

Now it's time to shine. Gather every role you could search for that you are going to use on your project and start chaining them together for awesomeness. In a short time you'll be creating your own roles, contributing for the ones already on the community and do a great job on your own playbooks. Just to help you out, take a look https://bitbucket.org/tecnobizz/vagrant-alm[at this project] that I did to provision all the three servers for a ALM process I was talking about at the begining of this post. There I orchestrate a simple playbook to create all the three machines using just three tasks: provision Jenkins, Sonar Source and Nexus.

See you next time.


Update 2016-09-12-Ensemble-analysis-of-Differential-Expression-from-RNA-Seq-data-sets.adoc

= Ensemble analysis of Differential Expression from RNA-Seq data sets

Recently I've seen discussions among bioinformatians of which is 'the best' method for DE analysis of RNA-Seq data.  It's a recurring theme, and mostly the answer is 'it depends on context and aims' although there is also evidence that some methods are just better than others.

For the past couple of years, my answer has been that the best method is 'all of them'.  I've come round to thinking that an ensemble of methods probably gives the best result most of the time.

This isn't really a new problem or a new question.  Before RNA-Seq was _de rigeur_ for expression experiments, we noticed that in microarray experiments, the list of DE genes varied depending on the method used to estimate DE.  In our projects at the time, the 'best' method for DE analysis was a source of regular debate in project meetings among bioinformaticians, or statisticians, or modellers.

We came to the conclusion that the most robust approach was to use multiple methods, and to correlate them.  For example, in this timeseries analysis http://www.plantcell.org/content/24/9/3530.full, we used two different methods for estimating whether genes were DE.  We then did a manual (or is that visual?) review of the expression profiles of the anomalous cases where only one of the methods score the gene as significantly DE.  This review led us to a conclusion about which method was more reliable in the edge cases for this particular data set.

Fast forward a bit and the same issue is live in RNA-Seq expression analysis.  You've aligned your reads or analysed your kmer content, and have a score or count for each gene or transcript, and you need to decide which tool to use to estimate DE.

This Venn diagram from Chen et al, 2015 was recently used to give a nice example of the problem:

image::http://static-content.springer.com/image/art%3A10.1186%2F1471-2164-16-S7-S14/MediaObjects/12864_2015_Article_7208_Fig6_HTML.jpg[]

Each method comes up with a different list of genes that are DE. There is some overlap between methods, but it's not immediately clear which method is 'right' or 'wrong' about which particular genes.

My answer is to believe all the methods, to some extent, and to take an ensemble of methods as my estimate of DE.

*The problem is how to take an ensemble of DE analysis methods.*

My insight, and my gut feeling, was that when we take the Venn diagram/p-value cut-off approach, then we throw away most of the information about DE.  We demand that genes are DE or not, but actually, there is a whole spectrum from 'almost definitely not DE at all' to 'almost definitely DE'.  The analysis tools even make it easy for us by giving an output of q-values, or FDR values for each gene or transcript, that are essentially a metric of 'DE-ness'.

I decided to combine all these separate metrics into a single, ensemble, metric of DE-ness.  Being a fettler at heart, I took the obvious approach.

	I used each tool to generate a ranked order list of all genes,
        sorted by q-value or FDR.
	For each gene, I calculated its sum of ranks,
        by adding up the ranks each individual method had given the gene.
	I sorted the gene list by this rank-sum metric.

This gave me a list of genes, sorted from most likely to be DE, as estimated by all the methods, to least likely.  Bingo, I have an ensemble DE metric.

The only thing that remains is to answer that dreaded biologist question "So which genes are DE?".  The choice of cut-off in the list.

My view on cut-offs is you cut your list to meet your needs.

If you want a strict cut-off, because you are going to spend money on follow-up qPCR experiments, then take a cutoff where all genes above the cut-off are assessed as significant by all methods.  If you want a lax cut-off, because you want sensitivity, and you are going to follow up with a cheap bioinformatic method like functional enrichment testing, then choose the cutoff accordingly - cut-off at the point where all genes above the cutoff were found significant by at least one method, or at least two.

This ensemble approach is flexible and extensible to as many different methods of DE assessment as you can choose.  My initial choices might be DESeq2, EdgeR, BaySeq, cuffdiff, and limma-voom, because they all have such different models of what it means to be DE, so are likely to form an interesting ensemble.  You might prefer your own or different methods.

I think there is a right answer to "Which methods should I use in my ensemble?" and I think it is "All of them".

The point is, all methods tend to have a sweet spot where they are particularly sensitive and/or specific, so the more methods you use in your ensemble, the more potential you have for a generally good ensemble.

In our first public outing for this method (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3915549/) we used three different methods of DE assessment, and we cutoff at the point where at least two methods classified a gene as significant.  Our approach was queried by one of our anonymous reviewers.  They asked whether this was an accepted method, and whether we could point to evidence that it would work.  We responded with logic, rather than direct evidence.  We showed that we were using a number of well-cited accepted methods, and correlating their results for robustness.  We also showed that the burgeoning fields of Data Science, and competitive mathematical modelling, were rich with examples of ensemble methods outperforming a single method.  Our argument was accepted.

The obvious extension of the approach is to start weighting the methods in the ensemble, either based on prior information about how good they are, or where their sweet spots are, or using an iterative approach based on the data in hand.  I might get back to that later.

Have you used an ensemble method for estimating differential gene expression?  Do you think this is a worthwhile approach?

= Ensemble analysis of Differential Expression from RNA-Seq data sets

Recently I've seen discussions among bioinformatians of which is 'the best' method for DE analysis of RNA-Seq data.  It's a recurring theme, and mostly the answer is 'it depends on context and aims' although there is also evidence that some methods are just better than others.

For the past couple of years, my answer has been that the best method is 'all of them'.  I've come round to thinking that an ensemble of methods probably gives the best result most of the time.

This isn't really a new problem or a new question.  Before RNA-Seq was _de rigeur_ for expression experiments, we noticed that in microarray experiments, the list of DE genes varied depending on the method used to estimate DE.  In our projects at the time, the 'best' method for DE analysis was a source of regular debate in project meetings among bioinformaticians, or statisticians, or modellers.

We came to the conclusion that the most robust approach was to use multiple methods, and to correlate them.  For example, in this timeseries analysis (www.plantcell.org/content/24/9/3530.full), we used two different methods for estimating whether genes were DE.  We then did a manual (or is that visual?) review of the expression profiles of the anomalous cases where only one of the methods score the gene as significantly DE.  This review led us to a conclusion about which method was more reliable in the edge cases for this particular data set.

Fast forward a bit and the same issue is live in RNA-Seq expression analysis.  You've aligned your reads or analysed your kmer content, and have a score or count for each gene or transcript, and you need to decide which tool to use to estimate DE.

This Venn diagram from Chen et al, 2015 was recently used to give a nice example of the issue: image::http://static-content.springer.com/image/art%3A10.1186%2F1471-2164-16-S7-S14/MediaObjects/12864_2015_Article_7208_Fig6_HTML.jpg[]

*The problem is how to take an ensemble of DE analysis methods.*

My insight, and my gut feeling, was that when we take the Venn diagram/p-value cut-off approach, then we throw away most of the information about DE.  We demand that genes are DE or not, but actually, there is a whole spectrum from 'almost definitely not DE at all' to 'almost definitely DE'.  The analysis tools even make it easy for us by giving an output of q-values, or FDR values for each gene or transcript, that are essentially a metric of 'DE-ness'.

I decided to combine all these separate metrics into a single, ensemble, metric of DE-ness.  Being a fettler at heart, I took the obvious approach.

	I used each tool to generate a ranked order list of all genes,
        sorted by q-value or FDR.
	For each gene, I calculated its sum of ranks,
        by adding up the ranks each individual method had given the gene.
	I sorted the gene list by this rank-sum metric.

This gave me a list of genes, sorted from most likely to be DE, as estimated by all the methods, to least likely.  Bingo, I have an ensemble DE metric.

The only thing that remains is to answer that dreaded biologist question "So which genes are DE?".  The choice of cut-off in the list.

My view on cut-offs is you cut your list to meet your needs.

If you want a strict cut-off, because you are going to spend money on follow-up qPCR experiments, then take a cutoff where all genes above the cut-off are assessed as significant by all methods.  If you want a lax cut-off, because you want sensitivity, and you are going to follow up with a cheap bioinformatic method like functional enrichment testing, then choose the cutoff accordingly - cut-off at the point where all genes above the cutoff were found significant by at least one method, or at least two.

This ensemble approach is flexible and extensible to as many different methods of DE assessment as you can choose.  My initial choices might be DESeq2, EdgeR, BaySeq, cuffdiff, and limma-voom, because they all have such different models of what it means to be DE, so are likely to form an interesting ensemble.  You might prefer your own or different methods.

I think there is a right answer to "Which methods should I use in my ensemble?" and I think it is "All of them".

The point is, all methods tend to have a sweet spot where they are particularly sensitive and/or specific, so the more methods you use in your ensemble, the more potential you have for a generally good ensemble.

In our first public outing for this method (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3915549/) we used three different methods of DE assessment, and we cutoff at the point where at least two methods classified a gene as significant.  Our approach was queried by one of our anonymous reviewers.  They asked whether this was an accepted method, and whether we could point to evidence that it would work.  We responded with logic, rather than direct evidence.  We showed that we were using a number of well-cited accepted methods, and correlating their results for robustness.  We also showed that the burgeoning fields of Data Science, and competitive mathematical modelling, were rich with examples of ensemble methods outperforming a single method.  Our argument was accepted.

The obvious extension of the approach is to start weighting the methods in the ensemble, either based on prior information about how good they are, or where their sweet spots are, or using an iterative approach based on the data in hand.  I might get back to that later.

Have you used an ensemble method for estimating differential gene expression?  Do you think this is a worthwhile approach?


Correct method signature in code example

Closes gh-1887

[[webflux-fn]]
= Functional Endpoints

Spring WebFlux includes a lightweight, functional programming model in which functions
are used to route and handle requests and contracts are designed for immutability.
It is an alternative to the annotated-based programming model but otherwise running on
the same <<web-reactive.adoc#webflux-reactive-spring-web>> foundation

[[webflux-fn-overview]]
== Overview

An HTTP request is handled with a **`HandlerFunction`** that takes `ServerRequest` and
returns `Mono<ServerResponse>`, both of which are immutable contracts that offer JDK-8
friendly access to the HTTP request and response. `HandlerFunction` is the equivalent of
an `@RequestMapping` method in the annotation-based programming model.

Requests are routed to a `HandlerFunction` with a **`RouterFunction`** that takes
`ServerRequest` and returns `Mono<HandlerFunction>`. When a request is matched to a
particular route, the `HandlerFunction` mapped to the route is used. `RouterFunction` is
the equivalent of an `@RequestMapping` annotation.

`RouterFunctions.route(RequestPredicate, HandlerFunction)` provides a router function
default implementation that can be used with a number of built-in request predicates.
For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.server.RequestPredicates.*;
import static org.springframework.web.reactive.function.server.RouterFunctions.route;

PersonRepository repository = ...
PersonHandler handler = new PersonHandler(repository);

RouterFunction<ServerResponse> route =
	route(GET("/person/{id}").and(accept(APPLICATION_JSON)), handler::getPerson)
		.andRoute(GET("/person").and(accept(APPLICATION_JSON)), handler::listPeople)
		.andRoute(POST("/person"), handler::createPerson);

public class PersonHandler {

	// ...

	public Mono<ServerResponse> listPeople(ServerRequest request) {
		// ...
	}

	public Mono<ServerResponse> createPerson(ServerRequest request) {
		// ...
	}

	public Mono<ServerResponse> getPerson(ServerRequest request) {
		// ...
	}
}
----

One way to run a `RouterFunction` is to turn it into an `HttpHandler` and install it
through one of the built-in <<web-reactive.adoc#webflux-httphandler,server adapters>>:

* `RouterFunctions.toHttpHandler(RouterFunction)`
* `RouterFunctions.toHttpHandler(RouterFunction, HandlerStrategies)`

Most applications will run through the WebFlux Java config, see <<webflux-fn-running>>.

[[webflux-fn-handler-functions]]
== HandlerFunction

`ServerRequest` and `ServerResponse` are immutable interfaces that offer JDK-8 friendly
access to the HTTP request and response with
http://www.reactive-streams.org[Reactive Streams] back pressure against the request
and response body stream. The request body is represented with a Reactor `Flux` or `Mono`.
The response body is represented with any Reactive Streams `Publisher`, including `Flux`
and `Mono`. For more on that see
<<web-reactive.adoc#webflux-reactive-libraries,Reactive Libraries>>.

[[webflux-fn-request]]
=== ServerRequest

`ServerRequest` provides access to the HTTP method, URI, headers, and query parameters
while access to the body is provided through the `body` methods.

To extract the request body to a `Mono<String>`:

 Mono<String> string = request.bodyToMono(String.class);

To extract the body to a `Flux<Person>`, where `Person` objects are decoded from some
serialized form, such as JSON or XML:

 Flux<Person> people = request.bodyToFlux(Person.class);

The above are shortcuts that use the more general `ServerRequest.body(BodyExtractor)`
which accepts the `BodyExtractor` functional, strategy interface. The utility class
`BodyExtractors` provides access to a number of instances. For example, the above can
also be written as follows:

 Mono<String> string = request.body(BodyExtractors.toMono(String.class));
 Flux<Person> people = request.body(BodyExtractors.toFlux(Person.class));

To access form data:

 Mono<MultiValueMap<String, String> map = request.body(BodyExtractors.toFormData());

To access multipart data as a map:

 Mono<MultiValueMap<String, Part> map = request.body(BodyExtractors.toMultipartData());

To access multiparts, one at a time, in streaming fashion:

 Flux<Part> parts = request.body(BodyExtractos.toParts());

[[webflux-fn-response]]
=== ServerResponse

`ServerResponse` provides access to the HTTP response and since it is immutable, you use
a build to create it. The builder can be used to set the response status, to add response
headers, or to provide a body. Below is an example with a 200 (OK) response with JSON
content:

 Mono<Person> person = ...
 ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).body(person, Person.class);

This is how to build a 201 (CREATED) response with `"Location"` header, and no body:

 URI location = ...
 ServerResponse.created(location).build();

[[webflux-fn-handler-classes]]
=== Handler Classes

We can write a handler function as a lambda. For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
HandlerFunction<ServerResponse> helloWorld =
  request -> ServerResponse.ok().body(fromObject("Hello World"));
----

That is convenient but in an application we need multiple functions and useful to group
related handler functions together into a handler (like an `@Controller`). For example,
here is a class that exposes a reactive `Person` repository:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.ServerResponse.ok;
import static org.springframework.web.reactive.function.BodyInserters.fromObject;

public class PersonHandler {

	private final PersonRepository repository;

	public PersonHandler(PersonRepository repository) {
		this.repository = repository;
	}

	public Mono<ServerResponse> listPeople(ServerRequest request) { // <1>
		Flux<Person> people = repository.allPeople();
		return ok().contentType(APPLICATION_JSON).body(people, Person.class);
	}

	public Mono<ServerResponse> createPerson(ServerRequest request) { // <2>
		Mono<Person> person = request.bodyToMono(Person.class);
		return ok().build(repository.savePerson(person));
	}

	public Mono<ServerResponse> getPerson(ServerRequest request) { // <3>
		int personId = Integer.valueOf(request.pathVariable("id"));
		return repository.getPerson(personId)
			.flatMap(person -> ok().contentType(APPLICATION_JSON).body(fromObject(person)))
			.switchIfEmpty(ServerResponse.notFound().build());
	}
}
----
<1> `listPeople` is a handler function that returns all `Person` objects found in the repository as
JSON.
<2> `createPerson` is a handler function that stores a new `Person` contained in the request body.
Note that `PersonRepository.savePerson(Person)` returns `Mono<Void>`: an empty Mono that emits
a completion signal when the person has been read from the request and stored. So we use the
`build(Publisher<Void>)` method to send a response when that completion signal is received, i.e.
when the `Person` has been saved.
<3> `getPerson` is a handler function that returns a single person, identified via the path
variable `id`. We retrieve that `Person` via the repository, and create a JSON response if it is
found. If it is not found, we use `switchIfEmpty(Mono<T>)` to return a 404 Not Found response.

[[webflux-fn-router-functions]]
== RouterFunction

`RouterFunction` is used to route requests to a `HandlerFunction`. Typically, you do not
write router functions yourself, but rather use
`RouterFunctions.route(RequestPredicate, HandlerFunction)`. If the predicate applies, the
request is routed to the given `HandlerFunction`, or otherwise no routing is performed,
and that would translate to a 404 (Not Found) response.

[[webflux-fn-predicates]]
=== Predicates

You can write your own `RequestPredicate`, but the `RequestPredicates` utility class
offers commonly implementations, based on the request path, HTTP method, content-type,
and so on. For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
RouterFunction<ServerResponse> route =
	RouterFunctions.route(RequestPredicates.path("/hello-world"),
	request -> Response.ok().body(fromObject("Hello World")));
----

You can compose multiple request predicates together via:

* `RequestPredicate.and(RequestPredicate)` -- both must match.
* `RequestPredicate.or(RequestPredicate)` -- either may match.

Many of the predicates from `RequestPredicates` are composed. For example
`RequestPredicates.GET(String)` is composed from `RequestPredicates.method(HttpMethod)`
and `RequestPredicates.path(String)`.

You can compose multiple router functions into one, such that they're evaluated in order,
and if the first route doesn't match, the second is evaluated. You can declare more
specific routes before more general ones.

[[webflux-fn-routes]]
=== Routes

You can compose multiple router functions together via:

* `RouterFunction.and(RouterFunction)`
* `RouterFunction.andRoute(RequestPredicate, HandlerFunction)` -- shortcut for
`RouterFunction.and()` with nested `RouterFunctions.route()`.

Using composed routes and predicates, we can then declare the following routes, referring
to methods in the `PersonHandler`, shown in <<webflux-fn-handler-class>>, through
https://docs.oracle.com/javase/tutorial/java/javaOO/methodreferences.html[method-references]:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.server.RequestPredicates.*;

PersonRepository repository = ...
PersonHandler handler = new PersonHandler(repository);

RouterFunction<ServerResponse> personRoute =
	route(GET("/person/{id}").and(accept(APPLICATION_JSON)), handler::getPerson)
		.andRoute(GET("/person").and(accept(APPLICATION_JSON)), handler::listPeople)
		.andRoute(POST("/person"), handler::createPerson);
----

[[webflux-fn-running]]
== Running a server

How do you run a router function in an HTTP server? A simple option is to convert a router
function to an `HttpHandler` using one of the following:

* `RouterFunctions.toHttpHandler(RouterFunction)`
* `RouterFunctions.toHttpHandler(RouterFunction, HandlerStrategies)`

The returned `HttpHandler` can then be used with a number of servers adapters by following
<<web-reactive.adoc#webflux-httphandler,HttpHandler>> for server-specific instructions.

A more advanced option is to run with a
<<web-reactive.adoc#webflux-dispatcher-handler,DispatcherHandler>>-based setup through the
<<web-reactive.adoc#webflux-config>> which uses Spring configuration to declare the
components quired to process requests. The WebFlux Java config declares the following
infrastructure components to support functional endpoints:

* `RouterFunctionMapping` -- detects one or more `RouterFunction<?>` beans in the Spring
configuration, combines them via `RouterFunction.andOther`, and routes requests to the
resulting composed `RouterFunction`.
* `HandlerFunctionAdapter` -- simple adapter that allows the `DispatcherHandler` to invoke
a `HandlerFunction` that was mapped to a request.
* `ServerResponseResultHandler` -- handles the result from the invocation of a
`HandlerFunction` by invoking the `writeTo` method of the `ServerResponse`.

The above components allow functional endpoints to fit within the `DispatcherHandler` request
processing lifecycle, and also potentially run side by side with annotated controllers, if
any are declared. It is also how functional endpoints are enabled the Spring Boot WebFlux
starter.

Below is example WebFlux Java config (see
<<web-reactive.adoc#webflux-dispatcher-handler,DispatcherHandler>> for how to run):

[source,java,indent=0]
[subs="verbatim,quotes"]
----
@Configuration
@EnableWebFlux
public class WebConfig implements WebFluxConfigurer {

	@Bean
	public RouterFunction<?> routerFunctionA() {
		// ...
	}

	@Bean
	public RouterFunction<?> routerFunctionB() {
		// ...
	}

	// ...

	@Override
	public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
		// configure message conversion...
	}

	@Override
	public void addCorsMappings(CorsRegistry registry) {
		// configure CORS...
	}

	@Override
	public void configureViewResolvers(ViewResolverRegistry registry) {
		// configure view resolution for HTML rendering...
	}
}
----

[[webflux-fn-handler-filter-function]]
== HandlerFilterFunction

Routes mapped by a router function can be filtered by calling
`RouterFunction.filter(HandlerFilterFunction)`, where `HandlerFilterFunction` is essentially a
function that takes a `ServerRequest` and `HandlerFunction`, and returns a `ServerResponse`.
The handler function parameter represents the next element in the chain: this is typically the
`HandlerFunction` that is routed to, but can also be another `FilterFunction` if multiple filters
are applied.
With annotations, similar functionality can be achieved using `@ControllerAdvice` and/or a `ServletFilter`.
Let's add a simple security filter to our route, assuming that we have a `SecurityManager` that
can determine whether a particular path is allowed:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.HttpStatus.UNAUTHORIZED;

SecurityManager securityManager = ...
RouterFunction<ServerResponse> route = ...

RouterFunction<ServerResponse> filteredRoute =
	route.filter((request, next) -> {
		if (securityManager.allowAccessTo(request.path())) {
			return next.handle(request);
		}
		else {
			return ServerResponse.status(UNAUTHORIZED).build();
		}
  });
----

You can see in this example that invoking the `next.handle(ServerRequest)` is optional: we only
allow the handler function to be executed when access is allowed.

[NOTE]
====
CORS support for functional endpoints is provided via a dedicated <<webflux-cors-webfilter,`CorsWebFilter`>>.
====

[[webflux-fn]]
= Functional Endpoints

Spring WebFlux includes a lightweight, functional programming model in which functions
are used to route and handle requests and contracts are designed for immutability.
It is an alternative to the annotated-based programming model but otherwise running on
the same <<web-reactive.adoc#webflux-reactive-spring-web>> foundation

[[webflux-fn-overview]]
== Overview

An HTTP request is handled with a **`HandlerFunction`** that takes `ServerRequest` and
returns `Mono<ServerResponse>`, both of which are immutable contracts that offer JDK-8
friendly access to the HTTP request and response. `HandlerFunction` is the equivalent of
an `@RequestMapping` method in the annotation-based programming model.

Requests are routed to a `HandlerFunction` with a **`RouterFunction`** that takes
`ServerRequest` and returns `Mono<HandlerFunction>`. When a request is matched to a
particular route, the `HandlerFunction` mapped to the route is used. `RouterFunction` is
the equivalent of an `@RequestMapping` annotation.

`RouterFunctions.route(RequestPredicate, HandlerFunction)` provides a router function
default implementation that can be used with a number of built-in request predicates.
For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.server.RequestPredicates.*;
import static org.springframework.web.reactive.function.server.RouterFunctions.route;

PersonRepository repository = ...
PersonHandler handler = new PersonHandler(repository);

RouterFunction<ServerResponse> route =
	route(GET("/person/{id}").and(accept(APPLICATION_JSON)), handler::getPerson)
		.andRoute(GET("/person").and(accept(APPLICATION_JSON)), handler::listPeople)
		.andRoute(POST("/person"), handler::createPerson);

public class PersonHandler {

	// ...

	public Mono<ServerResponse> listPeople(ServerRequest request) {
		// ...
	}

	public Mono<ServerResponse> createPerson(ServerRequest request) {
		// ...
	}

	public Mono<ServerResponse> getPerson(ServerRequest request) {
		// ...
	}
}
----

One way to run a `RouterFunction` is to turn it into an `HttpHandler` and install it
through one of the built-in <<web-reactive.adoc#webflux-httphandler,server adapters>>:

* `RouterFunctions.toHttpHandler(RouterFunction)`
* `RouterFunctions.toHttpHandler(RouterFunction, HandlerStrategies)`

Most applications will run through the WebFlux Java config, see <<webflux-fn-running>>.

[[webflux-fn-handler-functions]]
== HandlerFunction

`ServerRequest` and `ServerResponse` are immutable interfaces that offer JDK-8 friendly
access to the HTTP request and response with
http://www.reactive-streams.org[Reactive Streams] back pressure against the request
and response body stream. The request body is represented with a Reactor `Flux` or `Mono`.
The response body is represented with any Reactive Streams `Publisher`, including `Flux`
and `Mono`. For more on that see
<<web-reactive.adoc#webflux-reactive-libraries,Reactive Libraries>>.

[[webflux-fn-request]]
=== ServerRequest

`ServerRequest` provides access to the HTTP method, URI, headers, and query parameters
while access to the body is provided through the `body` methods.

To extract the request body to a `Mono<String>`:

 Mono<String> string = request.bodyToMono(String.class);

To extract the body to a `Flux<Person>`, where `Person` objects are decoded from some
serialized form, such as JSON or XML:

 Flux<Person> people = request.bodyToFlux(Person.class);

The above are shortcuts that use the more general `ServerRequest.body(BodyExtractor)`
which accepts the `BodyExtractor` functional, strategy interface. The utility class
`BodyExtractors` provides access to a number of instances. For example, the above can
also be written as follows:

 Mono<String> string = request.body(BodyExtractors.toMono(String.class));
 Flux<Person> people = request.body(BodyExtractors.toFlux(Person.class));

To access form data:

 Mono<MultiValueMap<String, String> map = request.body(BodyExtractors.toFormData());

To access multipart data as a map:

 Mono<MultiValueMap<String, Part> map = request.body(BodyExtractors.toMultipartData());

To access multiparts, one at a time, in streaming fashion:

 Flux<Part> parts = request.body(BodyExtractos.toParts());

[[webflux-fn-response]]
=== ServerResponse

`ServerResponse` provides access to the HTTP response and since it is immutable, you use
a build to create it. The builder can be used to set the response status, to add response
headers, or to provide a body. Below is an example with a 200 (OK) response with JSON
content:

 Mono<Person> person = ...
 ServerResponse.ok().contentType(MediaType.APPLICATION_JSON).body(person, Person.class);

This is how to build a 201 (CREATED) response with `"Location"` header, and no body:

 URI location = ...
 ServerResponse.created(location).build();

[[webflux-fn-handler-classes]]
=== Handler Classes

We can write a handler function as a lambda. For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
HandlerFunction<ServerResponse> helloWorld =
  request -> ServerResponse.ok().body(fromObject("Hello World"));
----

That is convenient but in an application we need multiple functions and useful to group
related handler functions together into a handler (like an `@Controller`). For example,
here is a class that exposes a reactive `Person` repository:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.ServerResponse.ok;
import static org.springframework.web.reactive.function.BodyInserters.fromObject;

public class PersonHandler {

	private final PersonRepository repository;

	public PersonHandler(PersonRepository repository) {
		this.repository = repository;
	}

	public Mono<ServerResponse> listPeople(ServerRequest request) { // <1>
		Flux<Person> people = repository.allPeople();
		return ok().contentType(APPLICATION_JSON).body(people, Person.class);
	}

	public Mono<ServerResponse> createPerson(ServerRequest request) { // <2>
		Mono<Person> person = request.bodyToMono(Person.class);
		return ok().build(repository.savePerson(person));
	}

	public Mono<ServerResponse> getPerson(ServerRequest request) { // <3>
		int personId = Integer.valueOf(request.pathVariable("id"));
		return repository.getPerson(personId)
			.flatMap(person -> ok().contentType(APPLICATION_JSON).body(fromObject(person)))
			.switchIfEmpty(ServerResponse.notFound().build());
	}
}
----
<1> `listPeople` is a handler function that returns all `Person` objects found in the repository as
JSON.
<2> `createPerson` is a handler function that stores a new `Person` contained in the request body.
Note that `PersonRepository.savePerson(Person)` returns `Mono<Void>`: an empty Mono that emits
a completion signal when the person has been read from the request and stored. So we use the
`build(Publisher<Void>)` method to send a response when that completion signal is received, i.e.
when the `Person` has been saved.
<3> `getPerson` is a handler function that returns a single person, identified via the path
variable `id`. We retrieve that `Person` via the repository, and create a JSON response if it is
found. If it is not found, we use `switchIfEmpty(Mono<T>)` to return a 404 Not Found response.

[[webflux-fn-router-functions]]
== RouterFunction

`RouterFunction` is used to route requests to a `HandlerFunction`. Typically, you do not
write router functions yourself, but rather use
`RouterFunctions.route(RequestPredicate, HandlerFunction)`. If the predicate applies, the
request is routed to the given `HandlerFunction`, or otherwise no routing is performed,
and that would translate to a 404 (Not Found) response.

[[webflux-fn-predicates]]
=== Predicates

You can write your own `RequestPredicate`, but the `RequestPredicates` utility class
offers commonly implementations, based on the request path, HTTP method, content-type,
and so on. For example:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
RouterFunction<ServerResponse> route =
	RouterFunctions.route(RequestPredicates.path("/hello-world"),
	request -> Response.ok().body(fromObject("Hello World")));
----

You can compose multiple request predicates together via:

* `RequestPredicate.and(RequestPredicate)` -- both must match.
* `RequestPredicate.or(RequestPredicate)` -- either may match.

Many of the predicates from `RequestPredicates` are composed. For example
`RequestPredicates.GET(String)` is composed from `RequestPredicates.method(HttpMethod)`
and `RequestPredicates.path(String)`.

You can compose multiple router functions into one, such that they're evaluated in order,
and if the first route doesn't match, the second is evaluated. You can declare more
specific routes before more general ones.

[[webflux-fn-routes]]
=== Routes

You can compose multiple router functions together via:

* `RouterFunction.and(RouterFunction)`
* `RouterFunction.andRoute(RequestPredicate, HandlerFunction)` -- shortcut for
`RouterFunction.and()` with nested `RouterFunctions.route()`.

Using composed routes and predicates, we can then declare the following routes, referring
to methods in the `PersonHandler`, shown in <<webflux-fn-handler-class>>, through
https://docs.oracle.com/javase/tutorial/java/javaOO/methodreferences.html[method-references]:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.MediaType.APPLICATION_JSON;
import static org.springframework.web.reactive.function.server.RequestPredicates.*;

PersonRepository repository = ...
PersonHandler handler = new PersonHandler(repository);

RouterFunction<ServerResponse> personRoute =
	route(GET("/person/{id}").and(accept(APPLICATION_JSON)), handler::getPerson)
		.andRoute(GET("/person").and(accept(APPLICATION_JSON)), handler::listPeople)
		.andRoute(POST("/person"), handler::createPerson);
----

[[webflux-fn-running]]
== Running a server

How do you run a router function in an HTTP server? A simple option is to convert a router
function to an `HttpHandler` using one of the following:

* `RouterFunctions.toHttpHandler(RouterFunction)`
* `RouterFunctions.toHttpHandler(RouterFunction, HandlerStrategies)`

The returned `HttpHandler` can then be used with a number of servers adapters by following
<<web-reactive.adoc#webflux-httphandler,HttpHandler>> for server-specific instructions.

A more advanced option is to run with a
<<web-reactive.adoc#webflux-dispatcher-handler,DispatcherHandler>>-based setup through the
<<web-reactive.adoc#webflux-config>> which uses Spring configuration to declare the
components quired to process requests. The WebFlux Java config declares the following
infrastructure components to support functional endpoints:

* `RouterFunctionMapping` -- detects one or more `RouterFunction<?>` beans in the Spring
configuration, combines them via `RouterFunction.andOther`, and routes requests to the
resulting composed `RouterFunction`.
* `HandlerFunctionAdapter` -- simple adapter that allows the `DispatcherHandler` to invoke
a `HandlerFunction` that was mapped to a request.
* `ServerResponseResultHandler` -- handles the result from the invocation of a
`HandlerFunction` by invoking the `writeTo` method of the `ServerResponse`.

The above components allow functional endpoints to fit within the `DispatcherHandler` request
processing lifecycle, and also potentially run side by side with annotated controllers, if
any are declared. It is also how functional endpoints are enabled the Spring Boot WebFlux
starter.

Below is example WebFlux Java config (see
<<web-reactive.adoc#webflux-dispatcher-handler,DispatcherHandler>> for how to run):

[source,java,indent=0]
[subs="verbatim,quotes"]
----
@Configuration
@EnableWebFlux
public class WebConfig implements WebFluxConfigurer {

	@Bean
	public RouterFunction<?> routerFunctionA() {
		// ...
	}

	@Bean
	public RouterFunction<?> routerFunctionB() {
		// ...
	}

	// ...

	@Override
	public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
		// configure message conversion...
	}

	@Override
	default void addCorsMappings(CorsRegistry registry) {
		// configure CORS...
	}

	@Override
	public void configureViewResolvers(ViewResolverRegistry registry) {
		// configure view resolution for HTML rendering...
	}
}
----

[[webflux-fn-handler-filter-function]]
== HandlerFilterFunction

Routes mapped by a router function can be filtered by calling
`RouterFunction.filter(HandlerFilterFunction)`, where `HandlerFilterFunction` is essentially a
function that takes a `ServerRequest` and `HandlerFunction`, and returns a `ServerResponse`.
The handler function parameter represents the next element in the chain: this is typically the
`HandlerFunction` that is routed to, but can also be another `FilterFunction` if multiple filters
are applied.
With annotations, similar functionality can be achieved using `@ControllerAdvice` and/or a `ServletFilter`.
Let's add a simple security filter to our route, assuming that we have a `SecurityManager` that
can determine whether a particular path is allowed:

[source,java,indent=0]
[subs="verbatim,quotes"]
----
import static org.springframework.http.HttpStatus.UNAUTHORIZED;

SecurityManager securityManager = ...
RouterFunction<ServerResponse> route = ...

RouterFunction<ServerResponse> filteredRoute =
	route.filter((request, next) -> {
		if (securityManager.allowAccessTo(request.path())) {
			return next.handle(request);
		}
		else {
			return ServerResponse.status(UNAUTHORIZED).build();
		}
  });
----

You can see in this example that invoking the `next.handle(ServerRequest)` is optional: we only
allow the handler function to be executed when access is allowed.

[NOTE]
====
CORS support for functional endpoints is provided via a dedicated <<webflux-cors-webfilter,`CorsWebFilter`>>.
====