Fixed error in Stork guide

Fixed error in Stork guide

Signed-off-by: Helber Belmiro <acae567605c3b6ad01310ceb8780b22694b0e829@gmail.com>

////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Getting Started with SmallRye Stork
:extension-status: preview

include::_attributes.adoc[]

The essence of distributed systems resides in the interaction between services.
In modern architecture, you often have multiple instances of your service to share the load or improve the resilience by redundancy.
But how do you select the best instance of your service?
That's where https://smallrye.io/smallrye-stork[SmallRye Stork] helps.
Stork is going to choose the most appropriate instance.
It offers:

* Extensible service discovery mechanisms
* Built-in support for Consul and Kubernetes
* Customizable client load-balancing strategies

include::{includes}/extension-status.adoc[]

== Prerequisites

:prerequisites-docker:
include::{includes}/prerequisites.adoc[]

== Architecture

In this guide, we will build an application composed of:

* A simple blue service exposed on port 9000
* A simple red service exposed on port 9001
* A REST Client calling the blue or red service (the selection is delegated to Stork)
* A REST endpoint using the REST client and calling the services
* The blue and red services are registered in https://www.consul.io/[Consul].

image::stork-getting-started-architecture.png[Architecture of the application,width=50%, align=center]

For the sake of simplicity, everything (except Consul) will be running in the same Quarkus application.
Of course, each component will run in its own process in the real world.

== Solution

We recommend that you follow the instructions in the next sections and create the applications step by step.
However, you can go right to the completed example.

Clone the Git repository: `git clone {quickstarts-clone-url}`, or download an {quickstarts-archive-url}[archive].

The solution is located in the `stork-quickstart` {quickstarts-tree-url}/stork-quickstart[directory].

== Discovery and selection

Before going further, we need to discuss discovery vs. selection.

- Service discovery is the process of locating service instances.
It produces a list of service instances that is potentially empty (if no service matches the request) or contains multiple service instances.

- Service selection, also called load-balancing, chooses the best instance from the list returned by the discovery process.
The result is a single service instance or an exception when no suitable instance can be found.

Stork handles both discovery and selection.
However, it does not handle the communication with the service but only provides a service instance.
The various integrations in Quarkus extract the location of the service from that service instance.

image::stork-process.png[Discovery and Selection of services,width=50%, align=center]

== Bootstrapping the project

Create a Quarkus project importing the quarkus-rest-client-reactive and quarkus-resteasy-reactive extensions using your favorite approach:

:create-app-artifact-id: stork-quickstart
:create-app-extensions: quarkus-rest-client-reactive,quarkus-resteasy-reactive
include::{includes}/devtools/create-app.adoc[]

In the generated project, also add the following dependencies:

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
  <groupId>io.smallrye.stork</groupId>
  <artifactId>stork-service-discovery-consul</artifactId>
</dependency>
<dependency>
  <groupId>io.smallrye.reactive</groupId>
  <artifactId>smallrye-mutiny-vertx-consul-client</artifactId>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
implementation("io.smallrye.stork:stork-service-discovery-consul")
implementation("io.smallrye.reactive:smallrye-mutiny-vertx-consul-client")
----

`stork-service-discovery-consul` provides an implementation of service discovery for Consul.
`smallrye-mutiny-vertx-consul-client` is a Consul client which we will use to register our services in Consul.

== The Blue and Red services

Let's start with the very beginning: the service we will discover, select and call.

Create the `src/main/java/org/acme/services/BlueService.java` with the following content:

[source, java]
----
package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class BlueService {

    @ConfigProperty(name = "blue-service-port", defaultValue = "9000") int port;

    /**
     * Start an HTTP server for the blue service.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        vertx.createHttpServer()
                .requestHandler(req -> req.response().endAndForget("Hello from Blue!"))
                .listenAndAwait(port);
    }
}
----

It creates a new HTTP server (using Vert.x) and implements our simple service when the application starts.
For each HTTP request, it sends a response with "Hello from Blue!" as the body.

Following the same logic, create the `src/main/java/org/acme/services/RedService.java` with the following content:

[source, java]
----

package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class RedService {
    @ConfigProperty(name = "red-service-port", defaultValue = "9001") int port;

    /**
     * Start an HTTP server for the red service.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        vertx.createHttpServer()
                .requestHandler(req -> req.response().endAndForget("Hello from Red!"))
                .listenAndAwait(port);
    }

}
----

This time, it writes "Hello from Red!".

== Service registration in Consul

Now that we have implemented our services, we need to register them into Consul.

NOTE: Stork is not limited to Consul and integrates with other service discovery mechanisms.

Create the `src/main/java/org/acme/services/Registration.java` file with the following content:

[source, java]
----
package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.ext.consul.ServiceOptions;
import io.vertx.mutiny.ext.consul.ConsulClient;
import io.vertx.ext.consul.ConsulClientOptions;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class Registration {

    @ConfigProperty(name = "consul.host") String host;
    @ConfigProperty(name = "consul.port") int port;

    @ConfigProperty(name = "red-service-port", defaultValue = "9000") int red;
    @ConfigProperty(name = "blue-service-port", defaultValue = "9001") int blue;

    /**
     * Register our two services in Consul.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        ConsulClient client = ConsulClient.create(vertx, new ConsulClientOptions().setHost(host).setPort(port));

        client.registerServiceAndAwait(
                new ServiceOptions().setPort(red).setAddress("localhost").setName("my-service").setId("red"));
        client.registerServiceAndAwait(
                new ServiceOptions().setPort(blue).setAddress("localhost").setName("my-service").setId("blue"));
    }
}
----

When the application starts, it connects to Consul using the Vert.x Consul Client and registers our two instances.
Both registration uses the same name (`my-service`), but different ids to indicate that it's two instances of the same _service_.

== The REST Client interface and the front end API

So far, we didn't use Stork; we just scaffolded the services we will be discovering, selecting, and calling.

We will call the services using the Reactive REST Client.
Create the `src/main/java/org/acme/MyService.java` file with the following content:

[source, java]
----
package org.acme;

import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;

import javax.ws.rs.GET;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

/**
 * The REST Client interface.
 *
 * Notice the `baseUri`. It uses `stork://` as URL scheme indicating that the called service uses Stork to locate and
 * select the service instance. The `my-service` part is the service name. This is used to configure Stork discovery
 * and selection in the `application.properties` file.
 */
@RegisterRestClient(baseUri = "stork://my-service")
public interface MyService {

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    String get();
}
----

It's a straightforward REST client interface containing a single method. However, note the `baseUri` attribute.
It starts with `stork://`.
It instructs the REST client to delegate the discovery and selection of the service instances to Stork.
Notice the `my-service` part in the URL.
It is the service name we will be using in the application configuration.

It does not change how the REST client is used.
Create the `src/main/java/org/acme/FrontendApi.java` file with the following content:

[source, java]
----
package org.acme;

import org.eclipse.microprofile.rest.client.inject.RestClient;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

/**
 * A frontend API using our REST Client (which uses Stork to locate and select the service instance on each call).
 */
@Path("/api")
public class FrontendApi {

    @RestClient MyService service;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String invoke() {
        return service.get();
    }

}
----

It injects and uses the REST client as usual.

== Stork Filter
The `baseUri` configured in the REST client will be processed by `StorkClientRequestFilter` class, this is a https://quarkus.io/specs/jaxrs/2.1/index.html#filters[JAX-RS filter].
If you need to process the metadata associated with a message: HTTP headers, query parameters, media type, and other metadata, you can implement another filter to configure what you need.
Let’s implement a custom filter to add logging capability to our service. We create `CustomLoggingFilter` and annotating it with the @Provider annotation:

[source, java]
----
package org.acme;

import io.vertx.core.http.HttpServerRequest;
import org.jboss.logging.Logger;
import org.jboss.resteasy.reactive.client.spi.ResteasyReactiveClientRequestContext;
import org.jboss.resteasy.reactive.client.spi.ResteasyReactiveClientRequestFilter;

import javax.ws.rs.ext.Provider;

@Provider
public class CustomLoggingFilter implements ResteasyReactiveClientRequestFilter {

    private static final Logger LOG = Logger.getLogger(CustomLoggingFilter.class);

    @Override
    public void filter(ResteasyReactiveClientRequestContext requestContext) {
        LOG.infof("Resolved address by Stork: %s",requestContext.getUri().toString());
    }
}
----

The order in which filters are executed is defined by https://quarkus.io/specs/jaxrs/2.1/index.html#priorities[Priorities].
Note that `CustomLoggingFilter` is using a default value, so the user-level priority and the `StorkClientRequestFilter` uses the security authentication filter priority. This means that `StorkClientRequestFilter` will be executed before our `CustomLoggingFilter`.
Use `@Priority` annotation to change this behaviour.

== Stork configuration

The system is almost complete. We only need to configure Stork and the `Registration` bean.

In the `src/main/resources/application.properties`, add:

[source, properties]
----
consul.host=localhost
consul.port=8500

quarkus.stork.my-service.service-discovery.type=consul
quarkus.stork.my-service.service-discovery.consul-host=localhost
quarkus.stork.my-service.service-discovery.consul-port=8500
quarkus.stork.my-service.load-balancer.type=round-robin
----

The first two lines provide the Consul location used by the `Registration` bean.

The other properties are related to Stork.
`stork.my-service.service-discovery` indicates which type of service discovery we will be using to locate the `my-service` service.
In our case, it's `consul`.
`quarkus.stork.my-service.service-discovery.consul-host` and `quarkus.stork.my-service.service-discovery.consul-port` configures the access to Consul.
Finally, `quarkus.stork.my-service.load-balancer.type` configures the service selection.
In our case, we use a `round-robin`.

== Running the application

We're done!
So, let's see if it works.

First, start Consul:

[source, shell script]
----
docker run --rm --name consul -p 8500:8500 -p 8501:8501 consul:1.7 agent -dev -ui -client=0.0.0.0 -bind=0.0.0.0 --https-port=8501
----

If you start Consul differently, do not forget to edit the application configuration.

Then, package the application:

include::{includes}/devtools/build.adoc[]

And run it:

[source, shell script]
----
> java -jar target/quarkus-app/quarkus-run.jar
----

In another terminal, run:

[source, shell script]
----
> curl http://localhost:8080/api
...
> curl http://localhost:8080/api
...
> curl http://localhost:8080/api
...
----

The responses alternate between `Hello from Red!` and `Hello from Blue!`.

You can compile this application into a native executable:

include::{includes}/devtools/build-native.adoc[]

And start it with:

[source, shell script]
----
> ./target/stork-getting-started-1.0.0-SNAPSHOT-runner
----

== Going further

This guide has shown how to use SmallRye Stork to discover and select your services.
You can find more about Stork in:

- the xref:stork-reference.adoc[Stork reference guide],
- the https://smallrye.io/smallrye-stork[SmallRye Stork website].

////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Getting Started with SmallRye Stork
:extension-status: preview

include::_attributes.adoc[]

The essence of distributed systems resides in the interaction between services.
In modern architecture, you often have multiple instances of your service to share the load or improve the resilience by redundancy.
But how do you select the best instance of your service?
That's where https://smallrye.io/smallrye-stork[SmallRye Stork] helps.
Stork is going to choose the most appropriate instance.
It offers:

* Extensible service discovery mechanisms
* Built-in support for Consul and Kubernetes
* Customizable client load-balancing strategies

include::{includes}/extension-status.adoc[]

== Prerequisites

:prerequisites-docker:
include::{includes}/prerequisites.adoc[]

== Architecture

In this guide, we will build an application composed of:

* A simple blue service exposed on port 9000
* A simple red service exposed on port 9001
* A REST Client calling the blue or red service (the selection is delegated to Stork)
* A REST endpoint using the REST client and calling the services
* The blue and red services are registered in https://www.consul.io/[Consul].

image::stork-getting-started-architecture.png[Architecture of the application,width=50%, align=center]

For the sake of simplicity, everything (except Consul) will be running in the same Quarkus application.
Of course, each component will run in its own process in the real world.

== Solution

We recommend that you follow the instructions in the next sections and create the applications step by step.
However, you can go right to the completed example.

Clone the Git repository: `git clone {quickstarts-clone-url}`, or download an {quickstarts-archive-url}[archive].

The solution is located in the `stork-quickstart` {quickstarts-tree-url}/stork-quickstart[directory].

== Discovery and selection

Before going further, we need to discuss discovery vs. selection.

- Service discovery is the process of locating service instances.
It produces a list of service instances that is potentially empty (if no service matches the request) or contains multiple service instances.

- Service selection, also called load-balancing, chooses the best instance from the list returned by the discovery process.
The result is a single service instance or an exception when no suitable instance can be found.

Stork handles both discovery and selection.
However, it does not handle the communication with the service but only provides a service instance.
The various integrations in Quarkus extract the location of the service from that service instance.

image::stork-process.png[Discovery and Selection of services,width=50%, align=center]

== Bootstrapping the project

Create a Quarkus project importing the quarkus-rest-client-reactive and quarkus-resteasy-reactive extensions using your favorite approach:

:create-app-artifact-id: stork-quickstart
:create-app-extensions: quarkus-rest-client-reactive,quarkus-resteasy-reactive
include::{includes}/devtools/create-app.adoc[]

In the generated project, also add the following dependencies:

[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
.pom.xml
----
<dependency>
  <groupId>io.smallrye.stork</groupId>
  <artifactId>stork-service-discovery-consul</artifactId>
</dependency>
<dependency>
  <groupId>io.smallrye.reactive</groupId>
  <artifactId>smallrye-mutiny-vertx-consul-client</artifactId>
</dependency>
----

[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
.build.gradle
----
implementation("io.smallrye.stork:stork-service-discovery-consul")
implementation("io.smallrye.reactive:smallrye-mutiny-vertx-consul-client")
----

`stork-service-discovery-consul` provides an implementation of service discovery for Consul.
`smallrye-mutiny-vertx-consul-client` is a Consul client which we will use to register our services in Consul.

== The Blue and Red services

Let's start with the very beginning: the service we will discover, select and call.

Create the `src/main/java/org/acme/services/BlueService.java` with the following content:

[source, java]
----
package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class BlueService {

    @ConfigProperty(name = "blue-service-port", defaultValue = "9000") int port;

    /**
     * Start an HTTP server for the blue service.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        vertx.createHttpServer()
                .requestHandler(req -> req.response().endAndForget("Hello from Blue!"))
                .listenAndAwait(port);
    }
}
----

It creates a new HTTP server (using Vert.x) and implements our simple service when the application starts.
For each HTTP request, it sends a response with "Hello from Blue!" as the body.

Following the same logic, create the `src/main/java/org/acme/services/RedService.java` with the following content:

[source, java]
----

package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class RedService {
    @ConfigProperty(name = "red-service-port", defaultValue = "9001") int port;

    /**
     * Start an HTTP server for the red service.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        vertx.createHttpServer()
                .requestHandler(req -> req.response().endAndForget("Hello from Red!"))
                .listenAndAwait(port);
    }

}
----

This time, it writes "Hello from Red!".

== Service registration in Consul

Now that we have implemented our services, we need to register them into Consul.

NOTE: Stork is not limited to Consul and integrates with other service discovery mechanisms.

Create the `src/main/java/org/acme/services/Registration.java` file with the following content:

[source, java]
----
package org.acme.services;

import io.quarkus.runtime.StartupEvent;
import io.vertx.ext.consul.ServiceOptions;
import io.vertx.mutiny.ext.consul.ConsulClient;
import io.vertx.ext.consul.ConsulClientOptions;
import io.vertx.mutiny.core.Vertx;
import org.eclipse.microprofile.config.inject.ConfigProperty;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.event.Observes;

@ApplicationScoped
public class Registration {

    @ConfigProperty(name = "consul.host") String host;
    @ConfigProperty(name = "consul.port") int port;

    @ConfigProperty(name = "blue-service-port", defaultValue = "9000") int red;
    @ConfigProperty(name = "red-service-port", defaultValue = "9001") int blue;

    /**
     * Register our two services in Consul.
     *
     * Note: this method is called on a worker thread, and so it is allowed to block.
     */
    public void init(@Observes StartupEvent ev, Vertx vertx) {
        ConsulClient client = ConsulClient.create(vertx, new ConsulClientOptions().setHost(host).setPort(port));

        client.registerServiceAndAwait(
                new ServiceOptions().setPort(blue).setAddress("localhost").setName("my-service").setId("blue"));
        client.registerServiceAndAwait(
                new ServiceOptions().setPort(red).setAddress("localhost").setName("my-service").setId("red"));

    }
}
----

When the application starts, it connects to Consul using the Vert.x Consul Client and registers our two instances.
Both registration uses the same name (`my-service`), but different ids to indicate that it's two instances of the same _service_.

== The REST Client interface and the front end API

So far, we didn't use Stork; we just scaffolded the services we will be discovering, selecting, and calling.

We will call the services using the Reactive REST Client.
Create the `src/main/java/org/acme/MyService.java` file with the following content:

[source, java]
----
package org.acme;

import org.eclipse.microprofile.rest.client.inject.RegisterRestClient;

import javax.ws.rs.GET;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

/**
 * The REST Client interface.
 *
 * Notice the `baseUri`. It uses `stork://` as URL scheme indicating that the called service uses Stork to locate and
 * select the service instance. The `my-service` part is the service name. This is used to configure Stork discovery
 * and selection in the `application.properties` file.
 */
@RegisterRestClient(baseUri = "stork://my-service")
public interface MyService {

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    String get();
}
----

It's a straightforward REST client interface containing a single method. However, note the `baseUri` attribute.
It starts with `stork://`.
It instructs the REST client to delegate the discovery and selection of the service instances to Stork.
Notice the `my-service` part in the URL.
It is the service name we will be using in the application configuration.

It does not change how the REST client is used.
Create the `src/main/java/org/acme/FrontendApi.java` file with the following content:

[source, java]
----
package org.acme;

import org.eclipse.microprofile.rest.client.inject.RestClient;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

/**
 * A frontend API using our REST Client (which uses Stork to locate and select the service instance on each call).
 */
@Path("/api")
public class FrontendApi {

    @RestClient MyService service;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String invoke() {
        return service.get();
    }

}
----

It injects and uses the REST client as usual.

== Stork Filter
The `baseUri` configured in the REST client will be processed by `StorkClientRequestFilter` class, this is a https://quarkus.io/specs/jaxrs/2.1/index.html#filters[JAX-RS filter].
If you need to process the metadata associated with a message: HTTP headers, query parameters, media type, and other metadata, you can implement another filter to configure what you need.
Let’s implement a custom filter to add logging capability to our service. We create `CustomLoggingFilter` and annotating it with the @Provider annotation:

[source, java]
----
package org.acme;

import io.vertx.core.http.HttpServerRequest;
import org.jboss.logging.Logger;
import org.jboss.resteasy.reactive.client.spi.ResteasyReactiveClientRequestContext;
import org.jboss.resteasy.reactive.client.spi.ResteasyReactiveClientRequestFilter;

import javax.ws.rs.ext.Provider;

@Provider
public class CustomLoggingFilter implements ResteasyReactiveClientRequestFilter {

    private static final Logger LOG = Logger.getLogger(CustomLoggingFilter.class);

    @Override
    public void filter(ResteasyReactiveClientRequestContext requestContext) {
        LOG.infof("Resolved address by Stork: %s",requestContext.getUri().toString());
    }
}
----

The order in which filters are executed is defined by https://quarkus.io/specs/jaxrs/2.1/index.html#priorities[Priorities].
Note that `CustomLoggingFilter` is using a default value, so the user-level priority and the `StorkClientRequestFilter` uses the security authentication filter priority. This means that `StorkClientRequestFilter` will be executed before our `CustomLoggingFilter`.
Use `@Priority` annotation to change this behaviour.

== Stork configuration

The system is almost complete. We only need to configure Stork and the `Registration` bean.

In the `src/main/resources/application.properties`, add:

[source, properties]
----
consul.host=localhost
consul.port=8500

quarkus.stork.my-service.service-discovery.type=consul
quarkus.stork.my-service.service-discovery.consul-host=localhost
quarkus.stork.my-service.service-discovery.consul-port=8500
quarkus.stork.my-service.load-balancer.type=round-robin
----

The first two lines provide the Consul location used by the `Registration` bean.

The other properties are related to Stork.
`stork.my-service.service-discovery` indicates which type of service discovery we will be using to locate the `my-service` service.
In our case, it's `consul`.
`quarkus.stork.my-service.service-discovery.consul-host` and `quarkus.stork.my-service.service-discovery.consul-port` configures the access to Consul.
Finally, `quarkus.stork.my-service.load-balancer.type` configures the service selection.
In our case, we use a `round-robin`.

== Running the application

We're done!
So, let's see if it works.

First, start Consul:

[source, shell script]
----
docker run --rm --name consul -p 8500:8500 -p 8501:8501 consul:1.7 agent -dev -ui -client=0.0.0.0 -bind=0.0.0.0 --https-port=8501
----

If you start Consul differently, do not forget to edit the application configuration.

Then, package the application:

include::{includes}/devtools/build.adoc[]

And run it:

[source, shell script]
----
> java -jar target/quarkus-app/quarkus-run.jar
----

In another terminal, run:

[source, shell script]
----
> curl http://localhost:8080/api
...
> curl http://localhost:8080/api
...
> curl http://localhost:8080/api
...
----

The responses alternate between `Hello from Red!` and `Hello from Blue!`.

You can compile this application into a native executable:

include::{includes}/devtools/build-native.adoc[]

And start it with:

[source, shell script]
----
> ./target/stork-getting-started-1.0.0-SNAPSHOT-runner
----

== Going further

This guide has shown how to use SmallRye Stork to discover and select your services.
You can find more about Stork in:

- the xref:stork-reference.adoc[Stork reference guide],
- the https://smallrye.io/smallrye-stork[SmallRye Stork website].


Documentation for private struct members

== Structs ==

Golo allows the definition of simple structures using the `struct` keyword. They resemble structures
in procedural languages such as C `struct` or Pascal *records*. They are useful to store data when
the set of named entries is fixed.

=== Definition ===

Structures are defined at the module-level:

----
module sample

struct Person = { name, age, email }

function main = |args| {
  let p1 = Person("Mr Bean", 54, "bean@gmail.com")
  println(p1: name())
  let p2 = Person(): name("John"): age(32): email("john@b-root.com")
  println(p2: age())
}
----

When declaring a structure, it also defines two factory functions: one with no arguments, and one
with all arguments in their order of declaration in the `struct` statement. When not initialized,
member values are `null`.

Each member yields a *getter* and a *setter* method: given a member `a`, the getter is method `a()`
while the setter is method `a(newValue)`. It should be noted that setter methods return the
structure instance which makes it possible to chain calls as illustrated in the previous example
while building `p2`.

=== JVM existence ===

Each `struct` is compiled to a self-contained JVM class.

Given:

----
module sample

struct Point = { x, y }
----

a class `sample.types.Point` is being generated.

It is important to note that:

1. each `struct` class is `final`,
2. each `struct` class inherits from `gololang.GoloStruct`,
3. proper definitions of `toString()`, `hashCode()` and `equals()` are being provided.

=== `toString()` behavior ===

The `toString()` method is being overridden to provide a meaningful description of a structure
content.

Given the following program:

----
module test

struct Point = { x, y }

function main = |args| {
  println(Point(1, 2))
}
----

running it prints the following console output:

----
struct Point{x=1, y=2}
----

=== Immutable structs ===

Structure instances are mutable by default. Golo generates a factory function with the `Immutable`
prefix to directly build immutable instances:

----
module test

struct Point = { x, y }

function main = |args| {

  let p = ImmutablePoint(1, 2)
  println(p)

  try {
    # Fails! (p is immutable)
    p: x(100)
  } catch (expected) {
    println(expected: getMessage())
  }
}
----

=== Copying ===

Instances of a structure provide copying methods:

- `copy()` returns a *shallow* copy of the structure instance, and
- `frozenCopy()` returns a read-only *shallow* copy.

Trying to invoke any setter methods on an instance obtained through `frozenCopy()` raises a
`java.lang.IllegalStateException`.

IMPORTANT: The result of calling `copy()` on a frozen instance **is a mutable** copy, not a frozen
copy.

=== `equals()` and `hashCode()` semantics ===

Golo structures honor the contract of Java objects regarding equality and hash codes.

By default, `equals()` and `hashCode()` are the ones of `java.lang.Object`. Indeed, structure
members can be changed, so they cannot be used to compute stable values.

Nevertheless, structure instances returned by `frozenCopy()` have stable members, and members are
being used.

Consider the following program:

----
module test

struct Point = { x, y }

function main = |args| {

  let p1 = Point(1, 2)
  let p2 = Point(1, 2)
  let p3 = p1: frozenCopy()
  let p4 = p1: frozenCopy()

  println("p1 == p2 " + (p1 == p2))
  println("p1 == p3 " + (p1 == p3))
  println("p3 == p4 " + (p3 == p4))

  println("#p1 " + p1: hashCode())
  println("#p2 " + p2: hashCode())
  println("#p3 " + p3: hashCode())
  println("#p4 " + p4: hashCode())
}
----

the console output is the following:

----
p1 == p2 false
p1 == p3 false
p3 == p4 true
#p1 1555845260
#p2 104739310
#p3 994
#p4 994
----

TIP: It is recommended that you use `Immutable<name of struct>(...)` or `frozenCopy()` when you can,
especially when storing values into collections.

=== Helper methods ===

A number of helper methods are being generated:

- `members()` returns a tuple of the member names,
- `values()` returns a tuple with the current member values,
- `isFrozen()` returns a boolean to check for frozen structure instances,
- `iterator()` provides an iterator over a structure where each element is a tuple `[member, value]`,
- `get(name)` returns the value of a member by its name,
- `set(name, value)` updates the value of a member by its name, and returns the same structure.

=== Private members ===

By default, all members in a struct can be accessed. It is possible to make some elements private by
prefixing them with `_`, as in:

----
struct Foo = { a, _b, c }

# (...)

let foo = Foo(1, 2, 3)
----

In this case, `_b` is a private struct member. This means that `foo: _b()` and `foo: _b(666)` are
valid calls only if made from:

- a function from the declaring module, or
- an augmentation defined in the declaring module.

Any call to, say, `foo: _b()` from another module will yield a `NoSuchMethodError` exception.

Private struct members also have the following impact:

- they do not appear in `members()` and `values()` calls, and
- they are not iterated through `iterator()`-provided iterators, and
- they are being used like other members in `equals()` and `hashCode()`, and
- they do not appear in `toString()` representations.


Remove garbage in PID documentation

Remove garbage in PID documentation

[[time-series-measures]]
= Time Series Measures

The original purpose of *Inform* was to analyze time series data. This explains why most of
*Inform*'s functionality resides in functions specifically optimized for analyzing time
series. The API was designed to be easy to use in C or {cpp}, or to be wrapped in a
higher-level language, e.g. https://elife-asu.github.io/PyInform[Python]. This means that we
avoided some of the "niceties" of C, such as extensive use of macros and other generic
atrocities, in favor of wrappability. Keep this in mind as you learn the API.

Many information measures have "local" variants which compute a time series of point-wise
values. These local variants have names similar to their averaged or global counterparts,
e.g <<inform_active_info,inform_active_info>> and
<<inform_local_active_info,inform_local_active_info>>. We have been meticulous in ensuring
that function and parameter names are consistent across measures. If you notice some
inconsistency, please https://github.com/elife-asu/inform/issue[report it as an issue].

[[time-series-notation]]
== Notation

Throughout the discussion of time series measures, we will try to use a consistent notation.
We will denote random variables as stem:[X,Y,\ldots], and let stem:[x_i,y_i,\ldots]
represent the stem:[i]-th time step of a time series drawn from the associated random
variable. Many of the measures consider stem:[k]-histories (a.k.a stem:[k]-blocks) of the
time series, e.g. stem:[x_i^{(k)} = \left\{x_{i-k+1}, x_{i-k+2},\ldots,x_i\right\}].

When denoting probability distributions, we will only make the random variable explicit in
situations where the notation is ambiguous. We will typically write stem:[p(x_i)],
stem:[p(x_i^{(k)})], and stem:[p(x_i^{(k)}, x_{i+1})] to denote the empirical probability
of observing the stem:[x_i] state, the stem:[x_i^{(k)}] stem:[k]-history, and the joint
probability of observing stem:[\left(x_i^{(k)}, x_{i+1}\right)].

*Please report any notational ambiguities as an
https://github.com/elife-asu/inform/issue[issue].*

[[time-series-detail]]
== Implementation Details

=== The Base: States and Logarithms
The word "base" has two different meanings in the context of information measures on time
series. It could refer to the base of the time series itself, that is the number of unique
states in the time series. For example, the time series stem:[\{0,2,1,0,0\}] is a base-3
time series. On the other hand, it could refer to the base of the logarithm used in
computing the information content of the inferred probability distributions. The problem is
that these two meanings clash. The base of the time series affects the range of values the
measure can produce, and the base of the logarithm represents a rescaling of those values.

In this library we deal with this by *always* using base-2 logarithms, and having the user
specify the base of the time series — don't worry, we <<error-handling, set an error>> if
the provided base doesn't make sense. All of this ensures that the library is a simple as
reasonably possible.

=== Multiple Initial Conditions
You generally need *a lot* of data to infer a probability distribution.  An experimentalist
or simulator might then collect data over multiple trials or initial conditions. Most of
*Inform*'s time series measures allow the user to pass in a two-dimensional, rectangular
array which each row representing a time series from a different initial condition. From
this the probability distributions are inferred, and the desired value is calculated. This
has the downside of requiring that the user store all of the data in memory, but it has the
advantage of being fast and simple. Trade-offs, man...

A subsequent release, https://github.com/elife-asu/inform/milestone/3[likely v1.1.0], will
allows the initial conditions to have time series of different lengths and will provide
accumulator implementations of all of these measures that will let the incrementally
construct the distributions. This will lift some of the memory burden at the expense of
runtime performance.

=== Calling Conventions
All of the of the time series functions described in this section use the same basic calling
conventions and use the same (or similar) argument names were possible.

|===
| Argument Name | Description

| `series`
| A 2-D or 3-D, finite-state time series in contiguous, row-major form

| `l`
| The number of "sources" or variables in a 3-D time series

| `n`
| The number of initial conditions per "source"

| `m`
| The number of time steps per "source"

| `b`
| The base of the time series

| `k`
| The length history length

| `err`
| An error argument
|===

Average measures generally return a double-precision, floating-point value while local
variants return a pointer to an appropriately shaped, contiguous array. Local measures
accept an argument, often named after the function (e.g. local active information takes an
`ai` argument), which is used to store the computed local values. If that argument is NULL,
then the function allocates an array.

We will try to note any deviations from these conventions.

[[active-info]]
== Active Information

Active information (AI) was introduced in <<Lizier2012>> to quantify information storage in
distributed computations. Active information is defined in terms of a temporally local
variant

[stem]
++++
a_{X,i}(k) = \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})p(x_{i+1})}}.
++++

where the probabilities are constructed empirically from the _entire_ time series. From the
local variant, the temporally global active information is defined as

[stem]
++++
A_X(k) = \langle a_{X,i}(k) \rangle_i
       = \sum_{x_i^{(k)},x_{i+1}} p(x_i^{(k)},x_{i+1}) \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})p(x_{i+1})}}.
++++

Strictly speaking, the local and average active information are defined as

[stem]
++++
a_{X,i} = \lim_{k\rightarrow \infty}a_{X,i}(k)
\qquad \textrm{and} \qquad
A_X = \lim_{k\rightarrow \infty}A_X(k),
++++

but we do not provide limiting functionality in this library
(https://github.com/elife-asu/issues/24[yet]!).

****
[[inform_active_info]]
[source,c]
----
double inform_active_info(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average active information with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double ai = inform_active_info(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// ai ~ 0.305958
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double ai = inform_active_info(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// ai ~ 0.359879
----
****

****
[[inform_local_active_info]]
[source,c]
----
double *inform_local_active_info(int const *series, size_t n, size_t m,
        int b, size_t k, double *ai, inform_error *err);
----
Compute the local active information with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double *ai = inform_local_active_info(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// ai ~ {-0.193, 0.807, 0.222, 0.222, -0.363, 1.222, 0.222}
free(ai);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double ai[14];
inform_local_active_info(series, 2, 9, 2, 2, ai, &err);
assert(inform_succeeded(&err));
// ai ~ { 0.807, -0.363, 0.637, 0.637, -0.778, 0.807, -1.193,
//        0.807,  0.807, 0.222, 0.807,  0.807, 0.222,  0.807 }

// no need to free since `ai` was statically allocated in this scope
// free(ai);
----
****

[[block-entropy]]
== Block Entropy
Block entropy, also known as stem:[N]-gram entropy <<Shannon1948>>, is the standard Shannon
entropy of the stem:[k]-histories of a time series:
[stem]
++++
H(X^{(k)}) = -\sum_{x_i^{(k)}} p(x_i^{(k)}) \log_2{p(x_i^{(k)})}
++++
which reduces to the traditional Shannon entropy for stem:[k=1].

****
[[inform_block_entropy]]
[source,c]
----
double inform_block_entropy(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average block entropy of a time series with block size `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};

// k = 1
double h = inform_block_entropy(series, 1, 9, 2, 1, &err);
assert(inform_succeeded(&err));
// h ~ 0.991076

// k = 2
h = inform_block_entropy(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// h ~ 1.811278
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double h = inform_active_info(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// h ~ 1.936278
----
****

****
[[inform_local_block_entropy]]
[source,c]
----
double *inform_local_block_entropy(int const *series, size_t n,
        size_t m, int b, size_t k, double *ent, inform_error *err);
----
Compute the local block entropy of a time series with block size `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};

// k == 1
double *h = inform_local_block_entropy(series, 1, 9, 2, 1, NULL, &err);
assert(inform_succeeded(&err));
// h ~ { 0.848, 0.848, 1.170, 1.170, 1.170, 1.170, 0.848, 0.848, 0.848 }

// k == 2
double *h = inform_local_block_entropy(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// h ~ { 1.415, 3.000, 1.415, 1.415, 1.415, 3.000, 1.415, 1.415 }

free(ai);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double h[16];
inform_local_block_entropy(series, 2, 9, 2, 2, h, &err);
assert(inform_succeeded(&err));
// h ~ { 1.415, 2.415, 2.415, 2.415, 2.415, 2.000, 1.415, 1.415,
//       2.000, 1.415, 2.415, 2.000, 1.415, 2.415, 2.000, 1.415 }

// no need to free since `h` was statically allocated in this scope
// free(h);
----
****

[[conditional-entropy]]
== Conditional Entropy
https://en.wikipedia.org/wiki/Conditional_entropy[Conditional entropy] is a measure of the
amount of information required to describe a random variable stem:[Y] given knowledge of
another random variable stem:[X]. When applied to time series, two time series are used to
construct the empirical distributions, and <<inform_shannon_ce,inform_shannon_ce>> can be
applied to yield
[stem]
++++
H(Y|X) = - \sum_{x_i,y_i} p(x_i,y_i) \log_2{p(y_i|x_i)}.
++++
This can be viewed as the time-average of the local conditional entropy
[stem]
++++
h_i(Y|X) = -\log_2{p(y_i|x_i)}.
++++
See <<Cover1991>> for more information.

****
[[inform_conditional_entropy]]
[source,c]
----
double inform_conditional_entropy(int const *xs, int const *ys,
        size_t n, int bx, int by, inform_error *err);
----
Compute the conditional entropy between two time series.

This function expects the *condition* to be the first argument, `xs`. It is expected that
each time series be the same length `n`, but may have different bases `bx` and `by`.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1};
int const ys[20] = {0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1};

double ce = inform_conditional_entropy(xs, ys, 20, 2, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.597107

ce = inform_conditional_entropy(ys, xs, 20, 2, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.507757
----
****

****
[[inform_local_conditional_entropy]]
[source,c]
----
double *inform_local_conditional_entropy(int const *xs, int const *ys,
        size_t n, int bx, int by, double *mi, inform_error *err);
----
Compute the local conditional entropy between two time series.

This function expects the *condition* to be the first argument, `xs`. It is expected that
each time series be the same length `n`, but may have different bases `bx` and `by`.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1};
int const ys[20] = {0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1};

double *ce = inform_local_conditional_entropy(xs, ys, 20, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// ce == { 3.00, 3.00, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19,
//         0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.42, 0.42, 0.42, 2.00 }

inform_local_conditional_entropy(ys, xs, 20, 2, 2, ce, &err);
assert(inform_succeeded(&err));
// ce == { 1.32, 1.32, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10,
//         0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.74, 0.74, 0.74, 3.91 }

free(ce);
----
****

[[cross-entropy]]
== Cross Entropy
https://en.wikipedia.org/wiki/Cross_entropy[Cross entropy] between two distributions
stem:[p_X] and stem:[q_X] measures the amount of information needed to identify events
using a coding scheme optimized for stem:[q_X] when stem:[p_X] is the "real" distributions
over stem:[X].
[stem]
++++
H(p,q) = -\sum_{x} p(x) \log_2{q(x)}
++++
Cross entropy's local variant is equivalent to the self-information of stem:[q_X], and as
such is implemented by <<inform_local_block_entropy,inform_local_block_entropy>>.

See <<Cover1991>> for more details.
****
[[inform_cross_entropy]]
[source,c]
----
double inform_cross_entropy(int const *ps, int const *qs, size_t n,
        int b, inform_error *err);
----
Compute the cross entropy between the "true" and "unnatural" distributions stem:[p_X] and
stem:[q_X] from associated time series `ps` and `qs`, respectively.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const ps[10] = {0,1,1,0,1,0,0,1,0,0};
int const qs[10] = {0,0,0,0,0,1,1,0,0,1};

double ce = inform_cross_entropy(ps, qs, 10, 2, &err);
assert(inform_succeeded(&err));
// ce == 1.003530

ce = inform_cross_entropy(qs, ps, 10, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.912454
----
****

[[effective-information]]
== Effective Information

****
[[inform_effective_info]]
[source,c]
----
double inform_effective_info(double const *tpm, double const *inter,
        size_t n, inform_error *err);
----
****

[[entropy-rate]]
== Entropy Rate
https://en.wikipedia.org/wiki/Entropy_rate[Entropy rate] quantifies the amount of
information needed to describe the next state of stem:[X] given observations of
stem:[X^{(k)}].  In other wrods, it is the entropy of the time series conditioned on the
stem:[k]-histories.  The local entropy rate
[stem]
++++
h_{X,i}(k) = \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})}}.
++++
can be averaged to obtain the global entropy rate
[stem]
++++
H_X(k) = \langle h_{X,i}(k) \rangle_i
       = \sum_{x_i^{(k)},x_{i+1}} p(x_i^{(k)},x_{i+1}) \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})}}.
++++
Much as with <<active-info, active information>>, the local and average entropy rates are
formally obtained in the limit
[stem]
++++
h_{X,i} = \lim_{k\rightarrow \infty}h_{X,i}(k)
\qquad \textrm{and} \qquad
H_X = \lim_{k\rightarrow \infty}H_X(k),
++++

but we do not provide limiting functionality in this library
(https://github.com/elife-asu/issues/24[yet]!).

See <<Cover1991>> for more details.

****
[[inform_entropy_rate]]
[source,c]
----
double inform_entropy_rate(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average entropy rate with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double er = inform_entropy_rate(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// er ~ 0.679270
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double er = inform_entropy_rate(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// er ~ 0.625349
----
****

****
[[inform_local_entropy_rate]]
[source,c]
----
double *inform_local_entropy_rate(int const *series, size_t n,
        size_t m, int b, size_t k, double *er, inform_error *err);
----
Compute the local entropy rate with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double *er = inform_local_entropy_rate(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// er ~ { 1.000, 0.000, 0.585, 0.585, 1.585, 0.000, 1.000 }
free(er);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double er[14];
inform_local_entropy_rate(series, 2, 9, 2, 2, er, &err);
assert(inform_succeeded(&err));
// er ~ { 0.415, 1.585, 0.585, 0.585, 1.585, 0.000, 2.000,
//        0.000, 0.415, 0.585, 0.000, 0.415, 0.585, 0.000 }

// no need to free since `er` was statically allocated in this scope
// free(er);
----
****

[[excess-entropy]]
== Excess Entropy

****
[[inform_excess_entropy]]
[source,c]
----
double inform_excess_entropy(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
****

****
[[inform_local_excess_entropy]]
[source,c]
----
double *inform_local_excess_entropy(int const *series, size_t n,
        size_t m, int b, size_t k, double *ee, inform_error *err);
----
****

[[information-flow]]
== Information Flow

****
[[inform_information_flow]]
[source,c]
----
double inform_information_flow(int const *src, int const *dst,
        int const *back, size_t l_src, size_t l_dst, size_t l_back,
        size_t n, size_t m, int b, inform_error *err);
----
****

[[evidence-of-integration]]
== Evidence Of Integration

****
[[inform_integration_evidence]]
[source,c]
----
double *inform_integration_evidence(int const *series, size_t l,
        size_t n, int const *b, double *evidence, inform_error *err);
----
****

****
[[inform_integration_evidence_part]]
[source,c]
----
double *inform_integration_evidence_part(int const *series, size_t l,
        size_t n, int const *b, size_t const *parts, size_t nparts,
        double *evidence, inform_error *err);
----
****

[[mutual-information]]
== Mutual Information
https://en.wikipedia.org/wiki/Mutual_information[Mutual information] (MI) is a measure of
the amount of mutual dependence between at least two random variables. Locally, MI is
defined as
[stem]
++++
i_i(X_1,\ldots,X_l) = \frac{p(x_{1,i},\ldots,x_{l,i})}{p(x_{1,i})\ldots p(x_{l,i})}.
++++
The mutual information is then just the time average of stem:[i_i(X_1,\ldots,X_l)]:
[stem]
++++
I(X_1,\ldots,X_l) =
    \sum_{x_{1,i},\ldots,x_{l,i}} p(x_{1,i},\ldots,x_{l,i}) \frac{p(x_{1,i},\ldots,x_{l,i})}{p(x_{1,i})\ldots p(x_{l,i})}.
++++
See <<Cover1991>> for more details.

****
[[inform_mutual_info]]
[source,c]
----
double inform_mutual_info(int const *series, size_t l, size_t n,
        int const *b, inform_error *err);
----
Compute the mutual information between two or more time series.

For this function, `l` is the number of random variables, and `n` is the length of each
variable's time series. Each variable can have a different base, so `b` is an array of
length `l`.

*Examples:*

Two variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[40] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1  // var 2};

double mi = inform_mutual_info(xs, 2, 20, (int[2]){2,2}, &err);
assert(inform_succeeded(&err));
// mi == 0.214171
----

Three variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[60] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1, // var 2
                    1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 3};

double mi = inform_mutual_info(xs, 3, 20, (int[3]){2,2,2}, &err);
assert(inform_succeeded(&err));
// mi == 1.095462
----
****

****
[[inform_local_mutual_info]]
[source,c]
----
double *inform_local_mutual_info(int const *series, size_t l, size_t n,
        int const *b, double *mi, inform_error *err);
----
Compute the local mutual information between two or more time series.

For this function, `l` is the number of random variables, and `n` is the length of each
variable's time series. Each variable can have a different base, so `b` is an array of
length `l`.

*Examples:*

Two variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[40] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1  // var 2};

double *mi = inform_local_mutual_info(xs, 2, 20, (int[2]){2,2}, NULL, &err);
assert(inform_succeeded(&err));
// mi ~ { -1.000, -1.000, 0.222,  0.222, 0.222, 0.222, 0.222, 0.222,
//         0.222,  0.222, 0.222,  0.222, 0.222, 0.222, 0.222, 0.222,
//         1.585,  1.585, 1.585, -1.585 }
free(mi);
----

Three variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[60] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1, // var 2
                    1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 3};

double *mi = inform_local_mutual_info(xs, 3, 20, (int[3]){2,2,2}, NULL, &err);
assert(inform_succeeded(&err));
// mi ~ { 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737,
//        0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737,
//        3.322, 3.322, 0.322, 0.152 }
free(mi);
----
****

[[partial-information-decomposition]]
== Partial Information Decomposition

****
[[inform_pid_source]]
[source,c]
----
typedef struct inform_pid_source
{
    size_t *name;
    struct inform_pid_source **above;
    struct inform_pid_source **below;
    size_t size, n_above, n_below;
    double imin;
    double pi;
} inform_pid_source;
----
****

****
[[inform_pid_lattice]]
[source,c]
----
typedef struct inform_pid_lattice
{
    inform_pid_source **sources;
    inform_pid_source *top;
    inform_pid_source *bottom;
    size_t size;
} inform_pid_lattice;
----
****

****
[[inform_pid_lattice_free]]
[source,c]
----
void inform_pid_lattice_free(inform_pid_lattice *l);
----
****

****
[[inform_pid]]
[source,c]
----
inform_pid_lattice *inform_pid(int const *stimulus,
        int const *responses, size_t l, size_t n, int bs,
        int const *br, inform_error *err);
----
****

[[predictive-information]]
== Predictive Information

****
[[inform_predictive_info]]
[source,c]
----
double inform_predictive_info(int const *series, size_t n, size_t m,
        int b, size_t kpast, size_t kfuture, inform_error *err);
----
****

****
[[inform_local_predictive_info]]
[source,c]
----
double *inform_local_predictive_info(int const *series, size_t n,
        size_t m, int b, size_t kpast, size_t kfuture, double *pi,
        inform_error *err);
----
****

[[relative-entropy]]
== Relative Entropy
https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence[Relative entropy], also
known as the Kullback-Leibler divergence, measures the amount of information gained in
switching from a prior distribution stem:[q_X] to a posterior distribution stem:[p_X] over
_the same support_:
[stem]
++++
D_{KL}(p||q) = \sum_{x_i} p(x_i) \log_2{\frac{p(x_i)}{q(x_i)}}.
++++
The local counterpart is
[stem]
++++
d_{KL,i}(p||q) = log_2{\frac{p(x_i)}{q(x_i)}}.
++++
Note that the average in moving from the local to the non-local relative entropy is taken
over the posterior distribution.

See <<Kullback1951>> and <<Cover1991>> for more information.

****
[[inform_relative_entropy]]
[source,c]
----
double inform_relative_entropy(int const *xs, int const *ys, size_t n,
        int b, inform_error *err);
----
Compute the relative entropy between time series drawn from posterior and prior
distributions, here `xs` and `ys` respectively.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[10] = {0,1,0,0,0,0,0,0,0,1};
int const ys[10] = {0,1,1,1,1,0,0,1,0,0};

double re = inform_relative_entropy(xs, ys, 10, 2, &err);
assert(inform_succeeded(&err));
// re == 0.278072

re = inform_relative_entropy(ys, xs, 10, 2, &err);
assert(inform_succeeded(&err));
// re == 0.321928
----
****

****
[[inform_local_relative_entropy]]
[source,c]
----
double *inform_local_relative_entropy(int const *xs, int const *ys,
        size_t n, int b, double *re, inform_error *err);
----

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[10] = {0,1,0,0,0,0,0,0,0,1};
int const ys[10] = {0,1,1,1,1,0,0,1,0,0};

double *re = inform_local_relative_entropy(xs, ys, 10, 2, NULL, &err);
assert(inform_succeeded(&err));
// re ~ { 0.678, -1.322, 0.678, 0.678, 0.678, 0.678, 0.678,
//        0.678, 0.678, -1.322 };

inform_local_relative_entropy(ys, xs, 10, 2, re, &err);
assert(inform_succeeded(&err));
// re ~ { -0.678, 1.322, 1.322, 1.322, 1.322, -0.678, -0.678, 1.322,
//        -0.678, -0.678 }

free(re);
----
****

[[separable-information]]
== Separable Information

****
[[inform_separable_info]]
[source,c]
----
double inform_separable_info(int const *srcs, int const *dest,
        size_t l, size_t n, size_t m, int b, size_t k,
        inform_error *err);
----
****

****
[[inform_local_separable_info]]
[source,c]
----
double *inform_local_separable_info(int const *srcs, int const *dest,
        size_t l, size_t n, size_t m, int b, size_t k, double *si,
        inform_error *err);
----
****

[[transfer-entropy]]
== Transfer Entropy

****
[[inform_transfer_entropy]]
[source,c]
----
double inform_transfer_entropy(int const *src, int const *dst,
        int const *back, size_t l, size_t n, size_t m, int b, size_t k,
        inform_error *err);
----
****

****
[[inform_local_transfer_entropy]]
[source,c]
----
double *inform_local_transfer_entropy(int const *src, int const *dst,
        int const *back, size_t l, size_t n, size_t m, int b, size_t k,
        double *te, inform_error *err);
----
****

[[time-series-measures]]
= Time Series Measures

The original purpose of *Inform* was to analyze time series data. This explains why most of
*Inform*'s functionality resides in functions specifically optimized for analyzing time
series. The API was designed to be easy to use in C or {cpp}, or to be wrapped in a
higher-level language, e.g. https://elife-asu.github.io/PyInform[Python]. This means that we
avoided some of the "niceties" of C, such as extensive use of macros and other generic
atrocities, in favor of wrappability. Keep this in mind as you learn the API.

Many information measures have "local" variants which compute a time series of point-wise
values. These local variants have names similar to their averaged or global counterparts,
e.g <<inform_active_info,inform_active_info>> and
<<inform_local_active_info,inform_local_active_info>>. We have been meticulous in ensuring
that function and parameter names are consistent across measures. If you notice some
inconsistency, please https://github.com/elife-asu/inform/issue[report it as an issue].

[[time-series-notation]]
== Notation

Throughout the discussion of time series measures, we will try to use a consistent notation.
We will denote random variables as stem:[X,Y,\ldots], and let stem:[x_i,y_i,\ldots]
represent the stem:[i]-th time step of a time series drawn from the associated random
variable. Many of the measures consider stem:[k]-histories (a.k.a stem:[k]-blocks) of the
time series, e.g. stem:[x_i^{(k)} = \left\{x_{i-k+1}, x_{i-k+2},\ldots,x_i\right\}].

When denoting probability distributions, we will only make the random variable explicit in
situations where the notation is ambiguous. We will typically write stem:[p(x_i)],
stem:[p(x_i^{(k)})], and stem:[p(x_i^{(k)}, x_{i+1})] to denote the empirical probability
of observing the stem:[x_i] state, the stem:[x_i^{(k)}] stem:[k]-history, and the joint
probability of observing stem:[\left(x_i^{(k)}, x_{i+1}\right)].

*Please report any notational ambiguities as an
https://github.com/elife-asu/inform/issue[issue].*

[[time-series-detail]]
== Implementation Details

=== The Base: States and Logarithms
The word "base" has two different meanings in the context of information measures on time
series. It could refer to the base of the time series itself, that is the number of unique
states in the time series. For example, the time series stem:[\{0,2,1,0,0\}] is a base-3
time series. On the other hand, it could refer to the base of the logarithm used in
computing the information content of the inferred probability distributions. The problem is
that these two meanings clash. The base of the time series affects the range of values the
measure can produce, and the base of the logarithm represents a rescaling of those values.

In this library we deal with this by *always* using base-2 logarithms, and having the user
specify the base of the time series — don't worry, we <<error-handling, set an error>> if
the provided base doesn't make sense. All of this ensures that the library is a simple as
reasonably possible.

=== Multiple Initial Conditions
You generally need *a lot* of data to infer a probability distribution.  An experimentalist
or simulator might then collect data over multiple trials or initial conditions. Most of
*Inform*'s time series measures allow the user to pass in a two-dimensional, rectangular
array which each row representing a time series from a different initial condition. From
this the probability distributions are inferred, and the desired value is calculated. This
has the downside of requiring that the user store all of the data in memory, but it has the
advantage of being fast and simple. Trade-offs, man...

A subsequent release, https://github.com/elife-asu/inform/milestone/3[likely v1.1.0], will
allows the initial conditions to have time series of different lengths and will provide
accumulator implementations of all of these measures that will let the incrementally
construct the distributions. This will lift some of the memory burden at the expense of
runtime performance.

=== Calling Conventions
All of the of the time series functions described in this section use the same basic calling
conventions and use the same (or similar) argument names were possible.

|===
| Argument Name | Description

| `series`
| A 2-D or 3-D, finite-state time series in contiguous, row-major form

| `l`
| The number of "sources" or variables in a 3-D time series

| `n`
| The number of initial conditions per "source"

| `m`
| The number of time steps per "source"

| `b`
| The base of the time series

| `k`
| The length history length

| `err`
| An error argument
|===

Average measures generally return a double-precision, floating-point value while local
variants return a pointer to an appropriately shaped, contiguous array. Local measures
accept an argument, often named after the function (e.g. local active information takes an
`ai` argument), which is used to store the computed local values. If that argument is NULL,
then the function allocates an array.

We will try to note any deviations from these conventions.

[[active-info]]
== Active Information

Active information (AI) was introduced in <<Lizier2012>> to quantify information storage in
distributed computations. Active information is defined in terms of a temporally local
variant

[stem]
++++
a_{X,i}(k) = \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})p(x_{i+1})}}.
++++

where the probabilities are constructed empirically from the _entire_ time series. From the
local variant, the temporally global active information is defined as

[stem]
++++
A_X(k) = \langle a_{X,i}(k) \rangle_i
       = \sum_{x_i^{(k)},x_{i+1}} p(x_i^{(k)},x_{i+1}) \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})p(x_{i+1})}}.
++++

Strictly speaking, the local and average active information are defined as

[stem]
++++
a_{X,i} = \lim_{k\rightarrow \infty}a_{X,i}(k)
\qquad \textrm{and} \qquad
A_X = \lim_{k\rightarrow \infty}A_X(k),
++++

but we do not provide limiting functionality in this library
(https://github.com/elife-asu/issues/24[yet]!).

****
[[inform_active_info]]
[source,c]
----
double inform_active_info(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average active information with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double ai = inform_active_info(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// ai ~ 0.305958
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double ai = inform_active_info(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// ai ~ 0.359879
----
****

****
[[inform_local_active_info]]
[source,c]
----
double *inform_local_active_info(int const *series, size_t n, size_t m,
        int b, size_t k, double *ai, inform_error *err);
----
Compute the local active information with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double *ai = inform_local_active_info(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// ai ~ {-0.193, 0.807, 0.222, 0.222, -0.363, 1.222, 0.222}
free(ai);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double ai[14];
inform_local_active_info(series, 2, 9, 2, 2, ai, &err);
assert(inform_succeeded(&err));
// ai ~ { 0.807, -0.363, 0.637, 0.637, -0.778, 0.807, -1.193,
//        0.807,  0.807, 0.222, 0.807,  0.807, 0.222,  0.807 }

// no need to free since `ai` was statically allocated in this scope
// free(ai);
----
****

[[block-entropy]]
== Block Entropy
Block entropy, also known as stem:[N]-gram entropy <<Shannon1948>>, is the standard Shannon
entropy of the stem:[k]-histories of a time series:
[stem]
++++
H(X^{(k)}) = -\sum_{x_i^{(k)}} p(x_i^{(k)}) \log_2{p(x_i^{(k)})}
++++
which reduces to the traditional Shannon entropy for stem:[k=1].

****
[[inform_block_entropy]]
[source,c]
----
double inform_block_entropy(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average block entropy of a time series with block size `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};

// k = 1
double h = inform_block_entropy(series, 1, 9, 2, 1, &err);
assert(inform_succeeded(&err));
// h ~ 0.991076

// k = 2
h = inform_block_entropy(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// h ~ 1.811278
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double h = inform_active_info(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// h ~ 1.936278
----
****

****
[[inform_local_block_entropy]]
[source,c]
----
double *inform_local_block_entropy(int const *series, size_t n,
        size_t m, int b, size_t k, double *ent, inform_error *err);
----
Compute the local block entropy of a time series with block size `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};

// k == 1
double *h = inform_local_block_entropy(series, 1, 9, 2, 1, NULL, &err);
assert(inform_succeeded(&err));
// h ~ { 0.848, 0.848, 1.170, 1.170, 1.170, 1.170, 0.848, 0.848, 0.848 }

// k == 2
double *h = inform_local_block_entropy(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// h ~ { 1.415, 3.000, 1.415, 1.415, 1.415, 3.000, 1.415, 1.415 }

free(ai);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double h[16];
inform_local_block_entropy(series, 2, 9, 2, 2, h, &err);
assert(inform_succeeded(&err));
// h ~ { 1.415, 2.415, 2.415, 2.415, 2.415, 2.000, 1.415, 1.415,
//       2.000, 1.415, 2.415, 2.000, 1.415, 2.415, 2.000, 1.415 }

// no need to free since `h` was statically allocated in this scope
// free(h);
----
****

[[conditional-entropy]]
== Conditional Entropy
https://en.wikipedia.org/wiki/Conditional_entropy[Conditional entropy] is a measure of the
amount of information required to describe a random variable stem:[Y] given knowledge of
another random variable stem:[X]. When applied to time series, two time series are used to
construct the empirical distributions, and <<inform_shannon_ce,inform_shannon_ce>> can be
applied to yield
[stem]
++++
H(Y|X) = - \sum_{x_i,y_i} p(x_i,y_i) \log_2{p(y_i|x_i)}.
++++
This can be viewed as the time-average of the local conditional entropy
[stem]
++++
h_i(Y|X) = -\log_2{p(y_i|x_i)}.
++++
See <<Cover1991>> for more information.

****
[[inform_conditional_entropy]]
[source,c]
----
double inform_conditional_entropy(int const *xs, int const *ys,
        size_t n, int bx, int by, inform_error *err);
----
Compute the conditional entropy between two time series.

This function expects the *condition* to be the first argument, `xs`. It is expected that
each time series be the same length `n`, but may have different bases `bx` and `by`.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1};
int const ys[20] = {0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1};

double ce = inform_conditional_entropy(xs, ys, 20, 2, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.597107

ce = inform_conditional_entropy(ys, xs, 20, 2, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.507757
----
****

****
[[inform_local_conditional_entropy]]
[source,c]
----
double *inform_local_conditional_entropy(int const *xs, int const *ys,
        size_t n, int bx, int by, double *mi, inform_error *err);
----
Compute the local conditional entropy between two time series.

This function expects the *condition* to be the first argument, `xs`. It is expected that
each time series be the same length `n`, but may have different bases `bx` and `by`.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[20] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1};
int const ys[20] = {0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1};

double *ce = inform_local_conditional_entropy(xs, ys, 20, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// ce == { 3.00, 3.00, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.19,
//         0.19, 0.19, 0.19, 0.19, 0.19, 0.19, 0.42, 0.42, 0.42, 2.00 }

inform_local_conditional_entropy(ys, xs, 20, 2, 2, ce, &err);
assert(inform_succeeded(&err));
// ce == { 1.32, 1.32, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.10,
//         0.10, 0.10, 0.10, 0.10, 0.10, 0.10, 0.74, 0.74, 0.74, 3.91 }

free(ce);
----
****

[[cross-entropy]]
== Cross Entropy
https://en.wikipedia.org/wiki/Cross_entropy[Cross entropy] between two distributions
stem:[p_X] and stem:[q_X] measures the amount of information needed to identify events
using a coding scheme optimized for stem:[q_X] when stem:[p_X] is the "real" distributions
over stem:[X].
[stem]
++++
H(p,q) = -\sum_{x} p(x) \log_2{q(x)}
++++
Cross entropy's local variant is equivalent to the self-information of stem:[q_X], and as
such is implemented by <<inform_local_block_entropy,inform_local_block_entropy>>.

See <<Cover1991>> for more details.
****
[[inform_cross_entropy]]
[source,c]
----
double inform_cross_entropy(int const *ps, int const *qs, size_t n,
        int b, inform_error *err);
----
Compute the cross entropy between the "true" and "unnatural" distributions stem:[p_X] and
stem:[q_X] from associated time series `ps` and `qs`, respectively.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const ps[10] = {0,1,1,0,1,0,0,1,0,0};
int const qs[10] = {0,0,0,0,0,1,1,0,0,1};

double ce = inform_cross_entropy(ps, qs, 10, 2, &err);
assert(inform_succeeded(&err));
// ce == 1.003530

ce = inform_cross_entropy(qs, ps, 10, 2, &err);
assert(inform_succeeded(&err));
// ce == 0.912454
----
****

[[effective-information]]
== Effective Information

****
[[inform_effective_info]]
[source,c]
----
double inform_effective_info(double const *tpm, double const *inter,
        size_t n, inform_error *err);
----
****

[[entropy-rate]]
== Entropy Rate
https://en.wikipedia.org/wiki/Entropy_rate[Entropy rate] quantifies the amount of
information needed to describe the next state of stem:[X] given observations of
stem:[X^{(k)}].  In other wrods, it is the entropy of the time series conditioned on the
stem:[k]-histories.  The local entropy rate
[stem]
++++
h_{X,i}(k) = \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})}}.
++++
can be averaged to obtain the global entropy rate
[stem]
++++
H_X(k) = \langle h_{X,i}(k) \rangle_i
       = \sum_{x_i^{(k)},x_{i+1}} p(x_i^{(k)},x_{i+1}) \log_2{\frac{p(x_i^{(k)}, x_{i+1})}{p(x_i^{(k)})}}.
++++
Much as with <<active-info, active information>>, the local and average entropy rates are
formally obtained in the limit
[stem]
++++
h_{X,i} = \lim_{k\rightarrow \infty}h_{X,i}(k)
\qquad \textrm{and} \qquad
H_X = \lim_{k\rightarrow \infty}H_X(k),
++++

but we do not provide limiting functionality in this library
(https://github.com/elife-asu/issues/24[yet]!).

See <<Cover1991>> for more details.

****
[[inform_entropy_rate]]
[source,c]
----
double inform_entropy_rate(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
Compute the average entropy rate with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double er = inform_entropy_rate(series, 1, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// er ~ 0.679270
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double er = inform_entropy_rate(series, 2, 9, 2, 2, &err);
assert(inform_succeeded(&err));
// er ~ 0.625349
----
****

****
[[inform_local_entropy_rate]]
[source,c]
----
double *inform_local_entropy_rate(int const *series, size_t n,
        size_t m, int b, size_t k, double *er, inform_error *err);
----
Compute the local entropy rate with a history length `k`.

*Examples:*

One initial condition:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[9] = {0,0,1,1,1,1,0,0,0};
double *er = inform_local_entropy_rate(series, 1, 9, 2, 2, NULL, &err);
assert(inform_succeeded(&err));
// er ~ { 1.000, 0.000, 0.585, 0.585, 1.585, 0.000, 1.000 }
free(er);
----

Two initial conditions:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const series[18] = {0,0,1,1,1,1,0,0,0,
                        1,0,0,1,0,0,1,0,0};
double er[14];
inform_local_entropy_rate(series, 2, 9, 2, 2, er, &err);
assert(inform_succeeded(&err));
// er ~ { 0.415, 1.585, 0.585, 0.585, 1.585, 0.000, 2.000,
//        0.000, 0.415, 0.585, 0.000, 0.415, 0.585, 0.000 }

// no need to free since `er` was statically allocated in this scope
// free(er);
----
****

[[excess-entropy]]
== Excess Entropy

****
[[inform_excess_entropy]]
[source,c]
----
double inform_excess_entropy(int const *series, size_t n, size_t m,
        int b, size_t k, inform_error *err);
----
****

****
[[inform_local_excess_entropy]]
[source,c]
----
double *inform_local_excess_entropy(int const *series, size_t n,
        size_t m, int b, size_t k, double *ee, inform_error *err);
----
****

[[information-flow]]
== Information Flow

****
[[inform_information_flow]]
[source,c]
----
double inform_information_flow(int const *src, int const *dst,
        int const *back, size_t l_src, size_t l_dst, size_t l_back,
        size_t n, size_t m, int b, inform_error *err);
----
****

[[evidence-of-integration]]
== Evidence Of Integration

****
[[inform_integration_evidence]]
[source,c]
----
double *inform_integration_evidence(int const *series, size_t l,
        size_t n, int const *b, double *evidence, inform_error *err);
----
****

****
[[inform_integration_evidence_part]]
[source,c]
----
double *inform_integration_evidence_part(int const *series, size_t l,
        size_t n, int const *b, size_t const *parts, size_t nparts,
        double *evidence, inform_error *err);
----
****

[[mutual-information]]
== Mutual Information
https://en.wikipedia.org/wiki/Mutual_information[Mutual information] (MI) is a measure of
the amount of mutual dependence between at least two random variables. Locally, MI is
defined as
[stem]
++++
i_i(X_1,\ldots,X_l) = \frac{p(x_{1,i},\ldots,x_{l,i})}{p(x_{1,i})\ldots p(x_{l,i})}.
++++
The mutual information is then just the time average of stem:[i_i(X_1,\ldots,X_l)]:
[stem]
++++
I(X_1,\ldots,X_l) =
    \sum_{x_{1,i},\ldots,x_{l,i}} p(x_{1,i},\ldots,x_{l,i}) \frac{p(x_{1,i},\ldots,x_{l,i})}{p(x_{1,i})\ldots p(x_{l,i})}.
++++
See <<Cover1991>> for more details.

****
[[inform_mutual_info]]
[source,c]
----
double inform_mutual_info(int const *series, size_t l, size_t n,
        int const *b, inform_error *err);
----
Compute the mutual information between two or more time series.

For this function, `l` is the number of random variables, and `n` is the length of each
variable's time series. Each variable can have a different base, so `b` is an array of
length `l`.

*Examples:*

Two variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[40] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1  // var 2};

double mi = inform_mutual_info(xs, 2, 20, (int[2]){2,2}, &err);
assert(inform_succeeded(&err));
// mi == 0.214171
----

Three variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[60] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1, // var 2
                    1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 3};

double mi = inform_mutual_info(xs, 3, 20, (int[3]){2,2,2}, &err);
assert(inform_succeeded(&err));
// mi == 1.095462
----
****

****
[[inform_local_mutual_info]]
[source,c]
----
double *inform_local_mutual_info(int const *series, size_t l, size_t n,
        int const *b, double *mi, inform_error *err);
----
Compute the local mutual information between two or more time series.

For this function, `l` is the number of random variables, and `n` is the length of each
variable's time series. Each variable can have a different base, so `b` is an array of
length `l`.

*Examples:*

Two variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[40] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1  // var 2};

double *mi = inform_local_mutual_info(xs, 2, 20, (int[2]){2,2}, NULL, &err);
assert(inform_succeeded(&err));
// mi ~ { -1.000, -1.000, 0.222,  0.222, 0.222, 0.222, 0.222, 0.222,
//         0.222,  0.222, 0.222,  0.222, 0.222, 0.222, 0.222, 0.222,
//         1.585,  1.585, 1.585, -1.585 }
free(mi);
----

Three variables:
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[60] = {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 1
                    0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,1, // var 2
                    1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1, // var 3};

double *mi = inform_local_mutual_info(xs, 3, 20, (int[3]){2,2,2}, NULL, &err);
assert(inform_succeeded(&err));
// mi ~ { 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737,
//        0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737, 0.737,
//        3.322, 3.322, 0.322, 0.152 }
free(mi);
----
****

[[partial-information-decomposition]]
== Partial Information Decomposition

****
[[inform_pid_source]]
[source,c]
----
typedef struct inform_pid_source
{
    size_t *name;
    struct inform_pid_source **above;
    struct inform_pid_source **below;
    size_t size, n_above, n_below;
    double imin;
    double pi;
    double info;
} inform_pid_source;
----
****

inform_pid_lattice *inform_pid(int const *stimulus, int const *responses,
        size_t l, size_t n, int bs, int const *br, inform_error *err);

#ifdef __cplusplus
}
#endif

****
[[inform_pid_lattice]]
[source,c]
----
typedef struct inform_pid_lattice
{
    inform_pid_source **sources;
    inform_pid_source *top;
    inform_pid_source *bottom;
    size_t size;
} inform_pid_lattice;
----
****

****
[[inform_pid_lattice_free]]
[source,c]
----
void inform_pid_lattice_free(inform_pid_lattice *l);
----
****

****
[[inform_pid]]
[source,c]
----
inform_pid_lattice *inform_pid(int const *stimulus,
        int const *responses, size_t l, size_t n, int bs,
        int const *br, inform_error *err);
----
****

[[predictive-information]]
== Predictive Information

****
[[inform_predictive_info]]
[source,c]
----
double inform_predictive_info(int const *series, size_t n, size_t m,
        int b, size_t kpast, size_t kfuture, inform_error *err);
----
****

****
[[inform_local_predictive_info]]
[source,c]
----
double *inform_local_predictive_info(int const *series, size_t n,
        size_t m, int b, size_t kpast, size_t kfuture, double *pi,
        inform_error *err);
----
****

[[relative-entropy]]
== Relative Entropy
https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence[Relative entropy], also
known as the Kullback-Leibler divergence, measures the amount of information gained in
switching from a prior distribution stem:[q_X] to a posterior distribution stem:[p_X] over
_the same support_:
[stem]
++++
D_{KL}(p||q) = \sum_{x_i} p(x_i) \log_2{\frac{p(x_i)}{q(x_i)}}.
++++
The local counterpart is
[stem]
++++
d_{KL,i}(p||q) = log_2{\frac{p(x_i)}{q(x_i)}}.
++++
Note that the average in moving from the local to the non-local relative entropy is taken
over the posterior distribution.

See <<Kullback1951>> and <<Cover1991>> for more information.

****
[[inform_relative_entropy]]
[source,c]
----
double inform_relative_entropy(int const *xs, int const *ys, size_t n,
        int b, inform_error *err);
----
Compute the relative entropy between time series drawn from posterior and prior
distributions, here `xs` and `ys` respectively.

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[10] = {0,1,0,0,0,0,0,0,0,1};
int const ys[10] = {0,1,1,1,1,0,0,1,0,0};

double re = inform_relative_entropy(xs, ys, 10, 2, &err);
assert(inform_succeeded(&err));
// re == 0.278072

re = inform_relative_entropy(ys, xs, 10, 2, &err);
assert(inform_succeeded(&err));
// re == 0.321928
----
****

****
[[inform_local_relative_entropy]]
[source,c]
----
double *inform_local_relative_entropy(int const *xs, int const *ys,
        size_t n, int b, double *re, inform_error *err);
----

*Examples:*
[source,c]
----
inform_error err = INFORM_SUCCESS;
int const xs[10] = {0,1,0,0,0,0,0,0,0,1};
int const ys[10] = {0,1,1,1,1,0,0,1,0,0};

double *re = inform_local_relative_entropy(xs, ys, 10, 2, NULL, &err);
assert(inform_succeeded(&err));
// re ~ { 0.678, -1.322, 0.678, 0.678, 0.678, 0.678, 0.678,
//        0.678, 0.678, -1.322 };

inform_local_relative_entropy(ys, xs, 10, 2, re, &err);
assert(inform_succeeded(&err));
// re ~ { -0.678, 1.322, 1.322, 1.322, 1.322, -0.678, -0.678, 1.322,
//        -0.678, -0.678 }

free(re);
----
****

[[separable-information]]
== Separable Information

****
[[inform_separable_info]]
[source,c]
----
double inform_separable_info(int const *srcs, int const *dest,
        size_t l, size_t n, size_t m, int b, size_t k,
        inform_error *err);
----
****

****
[[inform_local_separable_info]]
[source,c]
----
double *inform_local_separable_info(int const *srcs, int const *dest,
        size_t l, size_t n, size_t m, int b, size_t k, double *si,
        inform_error *err);
----
****

[[transfer-entropy]]
== Transfer Entropy

****
[[inform_transfer_entropy]]
[source,c]
----
double inform_transfer_entropy(int const *src, int const *dst,
        int const *back, size_t l, size_t n, size_t m, int b, size_t k,
        inform_error *err);
----
****

****
[[inform_local_transfer_entropy]]
[source,c]
----
double *inform_local_transfer_entropy(int const *src, int const *dst,
        int const *back, size_t l, size_t n, size_t m, int b, size_t k,
        double *te, inform_error *err);
----
****