Re-worked and edited

Re-worked and edited

= Replication (Enterprise Edition Feature)
Neil Clifford
:toc: macro
:toclevels: 1
:css-signature: demo
:toc-placement: macro
:icons: font

toc::[]

Chronicle Queue replicates Chronicle Queue messages over TCP/IP, from the `PRIMARY` to the `SECONDARY` to provide real-time back-up of your Chronicle queues.

Chronicle Queue automatically performs replication if the `chronicle-queue-enterprise.jar` is available in your class path, and you have set up a cluster in the `cluster.yaml` configuration file.

NOTE: For more information on obtaining the `chronicle-queue-enterprise.jar` please contact mailto:sales@chronicle.software[sales@chronicle.software].

Chronicle Queue replication works by replicating all messages from the `PRIMARY` queue to the `SECONDARY` queue.

image::Chronicle-Queue-Replication_diagram_02.jpg[Replication]

IMPORTANT: Chronicle requires exclusive write access to the `SECONDARY`. You should treat the `SECONDARY` as read-only, and not attempt to manually write messages.

== Queue header
The Chronical Queue header is structured as follows:

[source, java]
----

--- !!meta-data #binary
header: !SCQStore {
  wireType: !WireType BINARY_LIGHT,
  writePosition: 524744,
  roll: !SCQSRoll {
    length: !short 1000,
    format: yyyyMMdd-HHmmss,
    epoch: 0
  },
  indexing: !SCQSIndexing {
    indexCount: !int 32768,
    indexSpacing: 4,
    index2Index: 385,
    lastIndex: 4
  },
  lastAcknowledgedIndexReplicated: -1,
  recovery: !TimedStoreRecovery {
    timeStamp: 0
  },
  deltaCheckpointInterval: 0
}
----
`lastAcknowledgedIndexReplicated` is set to `-1` when

- replication is not being used, or

- no replication acknowledgment has yet been received from the other host.

`lastAcknowledgedIndexReplicated` will be automatically set to the index position of the latest acknowledged message.

== Tiered Indexing
Chronicle Queue uses tiered multi-level indexing to provide a fast and efficient method for searching for messages in a large queue.

=== Primary index
----
# position: 385, header: -1 #  <1>
--- !!meta-data #binary
index2index: [ #  <2>
  # length: 32768, used: 1
  262568, #  <3>
  0, 0, 0, 0, 0,
]
----

<1> `position` specifies the starting address of the data that follows, within the the queue (*.cq4) file.

<2> `index2Index` defines a pointer to the next lower level of index.

<3> specifies the point to the next level index.

NOTE: In this way, indexes can be tiered (primary, secondary, tertiary, etc.) as required to enhance indexing speed and efficiency.

=== Final level index

----
# position: 262568, header: -1 #  <1>
--- !!meta-data #binary
index: [ #  <2>
  # length: 32768, used: 1
  524744, #  <3>
  0, 0, 0, 0, 0, 0, 0, 0,
]
----
<1> `position` specifies the point in the queue.

<2> `index` defines a pointer to the data.

<3> specifies the point in the queue where the data begins.

=== Data location

----
# position: 524744, header: 0 #  <1>
--- !!data #binary #  <2>
"": some more text
...
# 785952 bytes remaining

----
<1> `position` specifies the point in the queue where the data begins.

<2> `data` defines the information that follows is data (`--- !!data #binary #`), rather than than meta data (`--- !!meta-data #binary`).

== Configuration File

The host in a cluster are defined in a YAML configuration file. This configuration file below, has just two hosts, but there is no fixed limit to the number of hosts you could include, ultimately it is limited by your hardware resources.

Each host is a running an instance of Chronicle. When using Chronicle Queue Enterprise Edition, these engines are likely to each contain a Chronicle queue. In this example below there are two hosts, one running on `localhost:8080`, and the other on `localhost:8081`. The hosts are on the same machine; in a real life example, these hosts would usually be on different machines.

NOTE: The examples given below perform queue replication using established interfaces that are based on the open source Chronicle Engine API.

[source, yaml]
----

cluster: {
  context: !EngineClusterContext {
  }
  host1: {
     hostId: 1,
     tcpBufferSize: 65536,
     connectUri: localhost:8080,
     timeoutMs: 5000,
  },
  host2: {
     hostId: 2,
     tcpBufferSize: 65536,
     connectUri:  localhost:8081,
     timeoutMs: 5000,
  }
}
----

== Host ID

When adding a host, you must provide a unique ID for each host; we refer to this ID as the `hostId`. The `hostId` must be a integer, from `1` to `MAX_INTEGER`.

[source, yaml]
----
  host: {
     hostId: <unique id>,
  }
----

It is important that this `hostId` is unique as it is used by Chronicle Engine clustering. For Queue replication, by default, the host with `hostID` of `1` is assumed to be the PRIMARY, and the other hostids are assumed to be the SECONDARYs.

== Creating an instance of the `ChronicleQueueView`

The following code is an example of how to create an instance of a host.

[source, java]
----
final AssetTree tree = new VanillaAssetTree((byte) hostId)
                .forTesting()
                .withConfig(resourcesDir() + "/config", OS.TARGET + "/" + hostId);

final Asset queue = tree.root().acquireAsset("the/uri");

queue.addLeafRule(QueueView.class, LAST + "chronicle queue", (context, asset) -> {
    try {
        return new ChronicleQueueView(context.wireType(writeType).cluster(clusterName)
                .elementType(context.elementType()).messageType(context.messageType()), asset);
    } catch (IOException e) {
        throw Jvm.rethrow(e);
    }
});

// change the host/port to either localhost:8080 or localhost:8081 depending on which host you are running
ServerEndpoint serverEndpoint = new ServerEndpoint("localhost:8080", tree);
----

The code above:

- sets up Chronicle on port `localhost:8080`.
- uses the configuration file shown. This configuration file should be stored at `config/etc/clusters.yaml`.
- configures an asset on the Chronicle Engine asset tree at `/the/uri` with a `leafRule`. This `leafRule` is setup to provide a `ChronicleQueueView` when asked for a `ChronicleQueue`. The `ChronicleQueueView` is an implementation of the `ChronicleQueue` interface.

Assuming that both `host1` and `host2` instances of Chronicle Engine were run with the same java code above, then messages added to the chronicle queue on `host1` would be replicated to `host2`.

The interface of the `ChronicleQueueView`, is as follows:

[source, java]
----
public interface QueueView<T, M> extends TopicPublisher<T, M>, KeyedView {

    /**
     * returns a {@link Excerpt} at a given index
     *
     * @param index the location of the except
     */
    @Nullable
    Excerpt<T, M> get(long index);

    /**
     * the next message from the current tailer which has this {@code topic}
     *
     * @param topic next excerpt that has this topic
     * @return the except
     */
    Excerpt<T, M> get(T topic);

    /**
     * Publish to a provided topic.
     *
     * @param topic   to publish to
     * @param message to publish.
     * @return the index in the chronicle queue the excerpt
     */
    long publishAndIndex(@NotNull T topic, @NotNull M message);

    interface Excerpt<T, M> {
        T topic();

        M message();

        long index();

        void clear();
    }

    interface Tailer<T, M> {
        /**
         * @return the next message from the current tailer
         */
        @Nullable
        Excerpt<T, M> read();
    }
}
----

== Definitions

|=======
|`Excerpt`| In Chronicle we refer to messages as excerpts.
|`PRIMARY`   | The master of the messages; messages are replicated from the PRIMARY to the SECONDARY.
|`SECONDARY`    | The receiver of the messages; holds an real-time up-to-date copy of the PRIMARY's data.
|=======

'''

<<../README.adoc#,Back to Chronicle Queue>>

= Replication (Enterprise Edition Feature)
Neil Clifford
:toc: macro
:toclevels: 1
:css-signature: demo
:toc-placement: macro
:icons: font

toc::[]

Chronicle Queue replicates Chronicle Queue messages over TCP/IP, from the `PRIMARY` to the `SECONDARY` to provide real-time back-up of your Chronicle queues.

Chronicle Queue automatically performs replication if the `chronicle-queue-enterprise.jar` is available in your class path, and you have set up a cluster in the `cluster.yaml` configuration file.

NOTE: For more information on obtaining the `chronicle-queue-enterprise.jar` please contact mailto:sales@chronicle.software[sales@chronicle.software].

Chronicle Queue replication works by replicating all messages from the `PRIMARY` queue to the `SECONDARY` queue.

image::Chronicle-Queue-Replication_diagram_02.jpg[Replication]

IMPORTANT: Chronicle requires exclusive write access to the `SECONDARY`. You should treat the `SECONDARY` as read-only, and not attempt to manually write messages.

== Queue header
The Chronical Queue header is structured as follows:

[source, java]
----

--- !!meta-data #binary
header: !SCQStore {
  wireType: !WireType BINARY_LIGHT,
  writePosition: 524744,
  roll: !SCQSRoll {
    length: !short 1000,
    format: yyyyMMdd-HHmmss,
    epoch: 0
  },
  indexing: !SCQSIndexing {
    indexCount: !int 32768,
    indexSpacing: 4,
    index2Index: 385,
    lastIndex: 4
  },
  lastAcknowledgedIndexReplicated: -1,
  recovery: !TimedStoreRecovery {
    timeStamp: 0
  },
  deltaCheckpointInterval: 0
}
----
`lastAcknowledgedIndexReplicated` is set to `-1` when

- replication is not being used, or

- no replication acknowledgment has yet been received from the other host.

`lastAcknowledgedIndexReplicated` will be automatically set to the index position of the latest acknowledged message.

== Tiered Indexing
Chronicle Queue uses tiered multi-level indexing to provide a fast and efficient method for searching for messages in a large queue.

=== Primary index
----
# position: 385, header: -1 #  <1>
--- !!meta-data #binary
index2index: [ #  <2>
  # length: 32768, used: 1
  262568, #  <3>
  0, 0, 0, 0, 0,
]
----

<1> `position` specifies the starting address of the data that follows, within the the queue (*.cq4) file.

<2> `index2Index` defines a pointer to the next lower level of index.

<3> specifies the point to the next level index.

NOTE: In this way, indexes can be tiered (primary, secondary, tertiary, etc.) as required to enhance indexing speed and efficiency.

=== Final level index

----
# position: 262568, header: -1 #  <1>
--- !!meta-data #binary
index: [ #  <2>
  # length: 32768, used: 1
  524744, #  <3>
  0, 0, 0, 0, 0, 0, 0, 0,
]
----
<1> `position` specifies the point in the queue.

<2> `index` defines a pointer to the data.

<3> specifies the point in the queue where the data begins.

=== Data location

----
# position: 524744, header: 0 #  <1>
--- !!data #binary #  <2>
"": some more text
...
# 785952 bytes remaining

----
<1> `position` specifies the point in the queue where the data begins.

<2> `data` defines the following information is data (`--- !!data #binary #`) rather than than meta data (`--- !!meta-data #binary`).

== Configuration File

The host in a cluster are defined in a YAML configuration file. This configuration file below, has just two hosts, but there is no fixed limit to the number of hosts you could include, ultimately it is limited by your hardware resources.

Each host is a running an instance of Chronicle. When using Chronicle Queue Enterprise Edition, these engines are likely to each contain a Chronicle queue. In this example below there are two hosts, one running on `localhost:8080`, and the other on `localhost:8081`. The hosts are on the same machine; in a real life example, these hosts would usually be on different machines.

NOTE: The examples given below perform queue replication using established interfaces that are based on the open source Chronicle Engine API.

[source, yaml]
----

cluster: {
  context: !EngineClusterContext {
  }
  host1: {
     hostId: 1,
     tcpBufferSize: 65536,
     connectUri: localhost:8080,
     timeoutMs: 5000,
  },
  host2: {
     hostId: 2,
     tcpBufferSize: 65536,
     connectUri:  localhost:8081,
     timeoutMs: 5000,
  }
}
----

== Host ID

When adding a host, you must provide a unique ID for each host; we refer to this ID as the `hostId`. The `hostId` must be a integer, from `1` to `MAX_INTEGER`.

[source, yaml]
----
  host: {
     hostId: <unique id>,
  }
----

It is important that this `hostId` is unique as it is used by Chronicle Engine clustering. For Queue replication, by default, the host with `hostID` of `1` is assumed to be the PRIMARY, and the other hostids are assumed to be the SECONDARYs.

== Creating an instance of the `ChronicleQueueView`

The following code is an example of how to create an instance of a host.

[source, java]
----
final AssetTree tree = new VanillaAssetTree((byte) hostId)
                .forTesting()
                .withConfig(resourcesDir() + "/config", OS.TARGET + "/" + hostId);

final Asset queue = tree.root().acquireAsset("the/uri");

queue.addLeafRule(QueueView.class, LAST + "chronicle queue", (context, asset) -> {
    try {
        return new ChronicleQueueView(context.wireType(writeType).cluster(clusterName)
                .elementType(context.elementType()).messageType(context.messageType()), asset);
    } catch (IOException e) {
        throw Jvm.rethrow(e);
    }
});

// change the host/port to either localhost:8080 or localhost:8081 depending on which host you are running
ServerEndpoint serverEndpoint = new ServerEndpoint("localhost:8080", tree);
----

The code above:

- sets up Chronicle on port `localhost:8080`.
- uses the configuration file shown. This configuration file should be stored at `config/etc/clusters.yaml`.
- configures an asset on the Chronicle Engine asset tree at `/the/uri` with a `leafRule`. This `leafRule` is setup to provide a `ChronicleQueueView` when asked for a `ChronicleQueue`. The `ChronicleQueueView` is an implementation of the `ChronicleQueue` interface.

Assuming that both `host1` and `host2` instances of Chronicle Engine were run with the same java code above, then messages added to the chronicle queue on `host1` would be replicated to `host2`.

The interface of the `ChronicleQueueView`, is as follows:

[source, java]
----
public interface QueueView<T, M> extends TopicPublisher<T, M>, KeyedView {

    /**
     * returns a {@link Excerpt} at a given index
     *
     * @param index the location of the except
     */
    @Nullable
    Excerpt<T, M> get(long index);

    /**
     * the next message from the current tailer which has this {@code topic}
     *
     * @param topic next excerpt that has this topic
     * @return the except
     */
    Excerpt<T, M> get(T topic);

    /**
     * Publish to a provided topic.
     *
     * @param topic   to publish to
     * @param message to publish.
     * @return the index in the chronicle queue the excerpt
     */
    long publishAndIndex(@NotNull T topic, @NotNull M message);

    interface Excerpt<T, M> {
        T topic();

        M message();

        long index();

        void clear();
    }

    interface Tailer<T, M> {
        /**
         * @return the next message from the current tailer
         */
        @Nullable
        Excerpt<T, M> read();
    }
}
----

== Definitions

|=======
|`Excerpt`| In Chronicle we refer to messages as excerpts.
|`PRIMARY`   | The master of the messages; messages are replicated from the PRIMARY to the SECONDARY.
|`SECONDARY`    | The receiver of the messages; holds an real-time up-to-date copy of the PRIMARY's data.
|=======

'''

<<../README.adoc#,Back to Chronicle Queue>>


Update rule descriptions to improve language

= The Cypher Style Guide

Style guide for writing Cypher code.

NOTE: Cypher is a language.
Languages have their own rules, and are not eager to adapt to circumstance.
Remember not to be too eager to shiv a Cypher expression into the conventions of a different language.

== Why Is Style Important?

Consider this dadaist work of art from Nigel Small and Nicole White:

.Insane query
[source, cypher]
----
MATCH (null)-[:merge]->(true)
with null.delete as foreach, `true`.false as null
return 2 + foreach, coalesce(null, 3.1415)
limit 10;
----

Then compare it to this classical piece by Mark Needham:

.Sane query
[source, cypher]
----
MATCH (member:Member {name: 'Mark Needham'})
      -[:HAS_MEMBERSHIP]->()-[:OF_GROUP]->(:Group)-[:HAS_TOPIC]->(topic)
WITH member, topic, count(*) AS score
MATCH (topic)<-[:HAS_TOPIC]-(otherGroup:Group)
WHERE NOT (member)-[:HAS_MEMBERSHIP]->(:Membership)-[:OF_GROUP]->(otherGroup)
RETURN otherGroup.name, collect(topic.name), sum(score) AS score
ORDER BY score DESC
----

== Rules

Whenever two rules are in conflict, use the order to decide which one trumps.

// Template:
//. rule
//+
//.Bad
//[source, cypher]
//----
//MATCH (person:Person {property: value})
//----
//.. No padding space for parameters.
//+
//.Good
//[source, cypher]
//----
//WITH { param } AS value
//----

. Start a new clause on a new line.
+
.Bad
[source, cypher]
----
MATCH (n) WHERE n.name CONTAINS 's' RETURN n.name
----
+
.Good
[source, cypher]
----
MATCH (n)
WHERE n.name CONTAINS 's'
RETURN n.name
----

.. Indent subclauses `ON CREATE` and `ON MATCH` with two spaces.
+
.Bad
[source, cypher]
----
MERGE (n) ON CREATE SET n.prop = 0
MERGE (a:A)-[:T]-(b:B)
ON CREATE SET a.name = 'me'
ON MATCH SET b.name = 'you'
----
+
.Good
[source, cypher]
----
MERGE (n)
  ON CREATE SET n.prop = 0
MERGE (a:A)-[:T]-(b:B)
  ON CREATE SET a.name = 'me'
  ON MATCH SET b.name = 'you'
----

.. Put `ON CREATE` before `ON MATCH` if both are present.
// no example necessary

. Write keywords in upper case.
+
.Bad
[source, cypher]
----
match (p:Person)
where p.name starts with 'Ma'
return p.name
----
+
.Good
[source, cypher]
----
MATCH (p:Person)
WHERE p.name STARTS WITH 'Ma'
RETURN p.name
----

. Write labels in camel case, starting with an upper case character.
+
.Bad
[source, cypher]
----
MATCH (e:editor-in-chief)-->(:employee)
RETURN e.name
----
+
.Good
[source, cypher]
----
MATCH (e:EditorInChief)-->(:Employee)
RETURN e.name
----

. Prefer single nouns for labels.
+
.Bad
[source, cypher]
----
MATCH (e:IsEmployed)
RETURN e.name
----
+
.Good
[source, cypher]
----
MATCH (e:Employee)
RETURN e.name
----

. Write relationship types in upper case, using underscore to separate words.
+
.Bad
[source, cypher]
----
MATCH (:Person)-[own:owns-vehicle]->(:Car)
RETURN own.since
----
+
.Good
[source, cypher]
----
MATCH (:Person)-[own:OWNS_VEHICLE]->(:Car)
RETURN own.since
----

. Use camel case, starting with a lower case character, for:
.. functions
.. properties
.. variables
.. parameters
+
.Bad
[source, cypher]
----
CREATE (N {Prop: 0})
WITH RAND() AS Rand, {param} AS MAP
RETURN Rand, MAP.property_key, Count(N)
----
+
.Good
[source, cypher]
----
CREATE (n {prop: 0})
WITH rand() AS rand, {param} AS map
RETURN rand, map.propertyKey, count(n)
----

. Use single quotes (Unicode character U+0027: ') for literal string values.
+
.Bad
[source, cypher]
----
RETURN "Cypher"
----
+
.Good
[source, cypher]
----
RETURN 'Cypher'
----

. *Spacing*
.. For literal maps:
... No space between the opening brace and the first key
... No space between key and colon
... One space between colon and value
... No space between value and comma
... One space between comma and next key
... No space between the last value and the closing brace
+
.Bad
[source, cypher]
----
WITH { key1 :'value' ,key2  :  42 } AS map
RETURN map
----
+
.Good
[source, cypher]
----
WITH {key1: 'value', key2: 42} AS map
RETURN map
----

.. No padding space for parameters.
+
.Bad
[source, cypher]
----
RETURN { param }
----
+
.Good
[source, cypher]
----
RETURN {param}
----

.. One space between label/type predicates and property predicates in patterns.
+
.Bad
[source, cypher]
----
MATCH (p:Person{property: -1})-[:KNOWS   {since: 2016}]->()
RETURN p.name
----
+
.Good
[source, cypher]
----
MATCH (p:Person {property: -1})-[:KNOWS {since: 2016}]->()
RETURN p.name
----

.. No space in patterns.
+
.Bad
[source, cypher]
----
MATCH (:Person) --> (:Vehicle)
RETURN count(*)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(:Vehicle)
RETURN count(*)
----

.. Use a wrapping space around operators.
+
.Bad
[source, cypher]
----
MATCH p=(s)-->(e)
WHERE s.name<>e.name
RETURN length(p)
----
+
.Good
[source, cypher]
----
MATCH p = (s)-->(e)
WHERE s.name <> e.name
RETURN length(p)
----

.. No space in label predicates.
+
.Bad
[source, cypher]
----
MATCH (person    : Person  :  Owner  )
RETURN person.name
----
+
.Good
[source, cypher]
----
MATCH (person:Person:Owner)
RETURN person.name
----

.. Use a space after each comma in lists and enumerations.
+
.Bad
[source, cypher]
----
MATCH (),()
WITH ['a','b',3.14] AS list
RETURN list,2,3,4
----
+
.Good
[source, cypher]
----
MATCH (), ()
WITH ['a', 'b', 3.14] AS list
RETURN list, 2, 3, 4
----

. When patterns wrap lines, break after arrows, not before.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)
      <--(:Country)
RETURN count(vehicle)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)<--
      (:Country)
RETURN count(vehicle)
----

. Surround node patterns with parentheses.
.. This only concerns old Cypher code, as omitting parentheses is no longer legal in Cypher.
+
.Bad
[source, cypher]
----
MATCH person-->vehicle
RETURN person, vehicle
----
+
.Good
[source, cypher]
----
MATCH (person)-->(vehicle)
RETURN person, vehicle
----

. Avoid having to use back-ticks to escape characters and keywords.
+
.Bad
[source, cypher]
----
MATCH (`odd-ch@racter$`:`Spaced Label` {`&property`: 42})
RETURN labels(`odd-ch@racter$`)
----
+
.Good
[source, cypher]
----
MATCH (node:NonSpacedLabel {property: 42})
RETURN labels(node)
----

. *Patterns*:
.. Use anonymous nodes and relationships when the variable would not be used.
+
.Bad
[source, cypher]
----
CREATE (a:End {prop: 42}),
       (b:End {prop: 3}),
       (c:Begin {prop: id(a)})
----
+
.Good
[source, cypher]
----
CREATE (a:End {prop: 42}),
       (:End {prop: 3}),
       (:Begin {prop: id(a)})
----

.. Chain patterns together to avoid repeating variables.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car), (vehicle:Car)-->(:Company)
RETURN count(vehicle)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)
RETURN count(vehicle)
----

.. Put named nodes before anonymous nodes.
+
.Bad
[source, cypher]
----
MATCH ()-->(vehicle:Car)-->(manufacturer:Company)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (manufacturer:Company)<--(vehicle:Car)<--()
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----

.. Keep anchor nodes at the beginning of the `MATCH` clause.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(manufacturer:Company)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (manufacturer:Company)<--(vehicle:Car)<--(:Person)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----

.. Prefer outgoing (left to right) pattern relationships to incoming pattern relationships.
+
.Bad
[source, cypher]
----
MATCH (:Country)-->(:Company)<--(vehicle:Car)<--(:Person)
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)<--(:Country)
RETURN vehicle.mileage
----

= The Cypher Style Guide

Style guide for writing Cypher code.

NOTE: Cypher is a language.
Languages have their own rules, and are not eager to adapt to circumstance.
Remember not to be too eager to shiv a Cypher expression into the conventions of a different language.

== Why Is Style Important?

Consider this dadaist work of art from Nigel Small and Nicole White:

.Insane query
[source, cypher]
----
MATCH (null)-[:merge]->(true)
with null.delete as foreach, `true`.false as null
return 2 + foreach, coalesce(null, 3.1415)
limit 10;
----

Then compare it to this classical piece by Mark Needham:

.Sane query
[source, cypher]
----
MATCH (member:Member {name: 'Mark Needham'})
      -[:HAS_MEMBERSHIP]->()-[:OF_GROUP]->(:Group)-[:HAS_TOPIC]->(topic)
WITH member, topic, count(*) AS score
MATCH (topic)<-[:HAS_TOPIC]-(otherGroup:Group)
WHERE NOT (member)-[:HAS_MEMBERSHIP]->(:Membership)-[:OF_GROUP]->(otherGroup)
RETURN otherGroup.name, collect(topic.name), sum(score) AS score
ORDER BY score DESC
----

== Rules

Whenever two rules are in conflict, use the order to decide which one trumps.

// Template:
//. rule
//+
//.Bad
//[source, cypher]
//----
//MATCH (person:Person {property: value})
//----
//.. No padding space for parameters.
//+
//.Good
//[source, cypher]
//----
//WITH { param } AS value
//----

. Start a new clause on a new line.
+
.Bad
[source, cypher]
----
MATCH (n) WHERE n.name CONTAINS 's' RETURN n.name
----
+
.Good
[source, cypher]
----
MATCH (n)
WHERE n.name CONTAINS 's'
RETURN n.name
----

.. Indent subclauses `ON CREATE` and `ON MATCH` with two spaces.
+
.Bad
[source, cypher]
----
MERGE (n) ON CREATE SET n.prop = 0
MERGE (a:A)-[:T]-(b:B)
ON CREATE SET a.name = 'me'
ON MATCH SET b.name = 'you'
----
+
.Good
[source, cypher]
----
MERGE (n)
  ON CREATE SET n.prop = 0
MERGE (a:A)-[:T]-(b:B)
  ON CREATE SET a.name = 'me'
  ON MATCH SET b.name = 'you'
----

.. Put `ON CREATE` before `ON MATCH` if both are present.
// no example necessary

. Write keywords in all caps.
+
.Bad
[source, cypher]
----
match (p:Person)
where p.name starts with 'Ma'
return p.name
----
+
.Good
[source, cypher]
----
MATCH (p:Person)
WHERE p.name STARTS WITH 'Ma'
RETURN p.name
----

. Write labels in camel case, starting with an upper case character.
+
.Bad
[source, cypher]
----
MATCH (e:editor-in-chief)-->(:employee)
RETURN e.name
----
+
.Good
[source, cypher]
----
MATCH (e:EditorInChief)-->(:Employee)
RETURN e.name
----

. Prefer single nouns for labels.
+
.Bad
[source, cypher]
----
MATCH (e:IsEmployed)
RETURN e.name
----
+
.Good
[source, cypher]
----
MATCH (e:Employee)
RETURN e.name
----

. Write relationship types in all caps, using underscore to separate words.
+
.Bad
[source, cypher]
----
MATCH (:Person)-[own:owns-vehicle]->(:Car)
RETURN own.since
----
+
.Good
[source, cypher]
----
MATCH (:Person)-[own:OWNS_VEHICLE]->(:Car)
RETURN own.since
----

. Use camel case, starting with a lower case character, for:
.. functions
.. properties
.. variables
.. parameters
+
.Bad
[source, cypher]
----
CREATE (N {Prop: 0})
WITH RAND() AS Rand, {param} AS MAP
RETURN Rand, MAP.property_key, Count(N)
----
+
.Good
[source, cypher]
----
CREATE (n {prop: 0})
WITH rand() AS rand, {param} AS map
RETURN rand, map.propertyKey, count(n)
----

. Use single quotes (Unicode character U+0027: ') for literal string values.
+
.Bad
[source, cypher]
----
RETURN "Cypher"
----
+
.Good
[source, cypher]
----
RETURN 'Cypher'
----

. *Spacing*
.. For literal maps:
... No space between opening brace and first key
... No space between key and colon
... One space between colon and value
... No space between value and comma
... One space between comma and next key
... No space between last value and closing brace
+
.Bad
[source, cypher]
----
WITH { key1 :'value' ,key2  :  42 } AS map
RETURN map
----
+
.Good
[source, cypher]
----
WITH {key1: 'value', key2: 42} AS map
RETURN map
----

.. No padding space for parameters.
+
.Bad
[source, cypher]
----
RETURN { param }
----
+
.Good
[source, cypher]
----
RETURN {param}
----

.. One space between label/type predicates and property predicates in patterns.
+
.Bad
[source, cypher]
----
MATCH (p:Person{property: -1})-[:KNOWS   {since: 2016}]->()
RETURN p.name
----
+
.Good
[source, cypher]
----
MATCH (p:Person {property: -1})-[:KNOWS {since: 2016}]->()
RETURN p.name
----

.. No space in patterns.
+
.Bad
[source, cypher]
----
MATCH (:Person) --> (:Vehicle)
RETURN count(*)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(:Vehicle)
RETURN count(*)
----

.. Use wrapping space around operators.
+
.Bad
[source, cypher]
----
MATCH p=(s)-->(e)
WHERE s.name<>e.name
RETURN length(p)
----
+
.Good
[source, cypher]
----
MATCH p = (s)-->(e)
WHERE s.name <> e.name
RETURN length(p)
----

.. No space in label predicates.
+
.Bad
[source, cypher]
----
MATCH (person    : Person  :  Owner  )
RETURN person.name
----
+
.Good
[source, cypher]
----
MATCH (person:Person:Owner)
RETURN person.name
----

.. Use space after each comma in lists and enumerations.
+
.Bad
[source, cypher]
----
MATCH (),()
WITH ['a','b',3.14] AS list
RETURN list,2,3,4
----
+
.Good
[source, cypher]
----
MATCH (), ()
WITH ['a', 'b', 3.14] AS list
RETURN list, 2, 3, 4
----

. When patterns wrap lines, break after arrows, not before.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)
      <--(:Country)
RETURN count(vehicle)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)<--
      (:Country)
RETURN count(vehicle)
----

. Surround node patterns with parentheses.
.. This only concerns old Cypher code, as it is no longer legal Cypher to omit parentheses.
+
.Bad
[source, cypher]
----
MATCH person-->vehicle
RETURN person, vehicle
----
+
.Good
[source, cypher]
----
MATCH (person)-->(vehicle)
RETURN person, vehicle
----

. Avoid having to use back-ticks to escape characters and keywords.
+
.Bad
[source, cypher]
----
MATCH (`odd-ch@racter$`:`Spaced Label` {`&property`: 42})
RETURN labels(`odd-ch@racter$`)
----
+
.Good
[source, cypher]
----
MATCH (node:NonSpacedLabel {property: 42})
RETURN labels(node)
----

. *Patterns*:
.. Leave nodes and relationships anonymous when the variable would not be used.
+
.Bad
[source, cypher]
----
CREATE (a:End {prop: 42}),
       (b:End {prop: 3}),
       (c:Begin {prop: id(a)})
----
+
.Good
[source, cypher]
----
CREATE (a:End {prop: 42}),
       (:End {prop: 3}),
       (:Begin {prop: id(a)})
----

.. Chain patterns together to avoid repeating variables.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car), (vehicle:Car)-->(:Company)
RETURN count(vehicle)
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)
RETURN count(vehicle)
----

.. Put named nodes before anonymous nodes.
+
.Bad
[source, cypher]
----
MATCH ()-->(vehicle:Car)-->(manufacturer:Company)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (manufacturer:Company)<--(vehicle:Car)<--()
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----

.. Keep anchor nodes at the beginning of the `MATCH` clause.
+
.Bad
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(manufacturer:Company)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (manufacturer:Company)<--(vehicle:Car)<--(:Person)
WHERE manufacturer.founded_year < 2000
RETURN vehicle.mileage
----

.. Prefer outgoing (left to right) pattern relationships to incoming.
+
.Bad
[source, cypher]
----
MATCH (:Country)-->(:Company)<--(vehicle:Car)<--(:Person)
RETURN vehicle.mileage
----
+
.Good
[source, cypher]
----
MATCH (:Person)-->(vehicle:Car)-->(:Company)<--(:Country)
RETURN vehicle.mileage
----


Update 2015-07-18-Low-Cost-Quadcopter-Build-for-DIY-Mapping.adoc

= Low Cost Quadcopter Build for DIY Mapping

grassrootsmapping

=== What I want to do

I want to build a programmable aerial mapping vehicle that I can use as an interim testbed to refine various DIY/Open Source methods that will eventually be included in a related project for an [affordable mapping drone](http://publiclab.org/notes/code4maine/08-05-2014/aerial-mapping-drone-for-under-60).

image::https://lh3.googleusercontent.com/-VuFdGoKo3UE/VcouDCctNRI/AAAAAAAAWDU/XHdkf7FCV7A/s640-Ic42/Quadannotated.png[]

In addition to testing flight components, the vehicle will also be a testbed for a wide variety of aerial sensors including a low-cost Camera's, LIDAR sensors and GPS.

=== My attempt and results

==== The -$100 Flying Wing
This project is an outgrowth of, and a complement to an earlier project that sought to build a fully programmable UAV for under $100. To keep costs down, project parameters stipulated that as much of the aircraft as possible should to be constructed out of standardized, off-the-shelf materials including the electronics which were designed around an Arduino Microcontroller and a Gyro-acceleromoter module extracted from a Ninetendo Wii remote. A flying-wing configuration using recycled foam allowed for further efficiency as it required only one motor and speed controller to remain in the air for longer periods.
  That project is still very much in progress and remains the ultimate goal.  However, while I was amazed at how much could be achieved with nothing more than an Arduino pro Mini and a Wiimote, the project reached a critical juncture when it became clear that the 8bit Arduino would not be up to the task of following preprogrammed flight patterns by GPS waypoint. As this capability is central to the task of aerial mapping, the entire project has been forced to go "back to the drawing board"...

==== Quadcopter
As counter-intuitive as it may seem, the simpler design in this case (the flying wing) turned out to be far more difficult than a more technologically complex design like a quadcopter.  In the world of DIY/Open Source, everything depends on there being enough documentation and support for any particular project.  As the quadcopter configuration has become the most popular over the last decade, it is far easier to start from scratch as you can depend on there being ample support from other users whenever a problem is encountered.

(In Progress)

= Low Cost Quadcopter Build for DIY Mapping

grassrootsmapping

===What I want to do

I want to build a programmable aerial mapping vehicle that I can use as an interim testbed to refine various DIY/Open Source methods that will eventually be included in a related project for an [affordable mapping drone](http://publiclab.org/notes/code4maine/08-05-2014/aerial-mapping-drone-for-under-60).

image::https://lh3.googleusercontent.com/-VuFdGoKo3UE/VcouDCctNRI/AAAAAAAAWDU/XHdkf7FCV7A/s640-Ic42/Quadannotated.png[]

In addition to testing flight components, the vehicle will also be a testbed for a wide variety of aerial sensors including a low-cost Camera's, LIDAR sensors and GPS.

===My attempt and results

====The -$100 Flying Wing
  This project is an outgrowth of, and a complement to an earlier project that sought to build a fully programmable UAV for under $100. To keep costs down, project parameters stipulated that as much of the aircraft as possible should to be constructed out of standardized, off-the-shelf materials including the electronics which were designed around an Arduino Microcontroller and a Gyro-acceleromoter module extracted from a Ninetendo Wii remote. A flying-wing configuration using recycled foam allowed for further efficiency as it required only one motor and speed controller to remain in the air for longer periods.
  That project is still very much in progress and remains the ultimate goal.  However, while I was amazed at how much could be achieved with nothing more than an Arduino pro Mini and a Wiimote, the project reached a critical juncture when it became clear that the 8bit Arduino would not be up to the task of following preprogrammed flight patterns by GPS waypoint. As this capability is central to the task of aerial mapping, the entire project has been forced to go "back to the drawing board"...

Quadcopter
   As counter-intuitive as it may seem, the simpler design in this case (the flying wing) turned out to be far more difficult than a more technologically complex design like a quadcopter.  In the world of DIY/Open Source, everything depends on there being enough documentation and support for any particular project.  As the quadcopter configuration has become the most popular over the last decade, it is far easier to start from scratch as you can depend on there being ample support from other users whenever a problem is encountered.

####ALL-IN-ONE


Update _ugfun_how-tos_class-structure_properties.adoc

[[_ugfun_how-tos_class-structure_properties]]
= Property
:Notice: Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at. http://www.apache.org/licenses/LICENSE-2.0 . Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR  CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
:_basedir: ../
:_imagesdir: images/

A property is an instance variable of a domain object, of a scalar type, that holds some state about either a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_entities[domain entity] or a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_view-models[view model].

For example, a ``Customer``'s `firstName` would be a property, as would their `accountCreationDate` that they created their account.
All properties have at least a "getter" method, and most properties have also a "setter" method (meaning that they are mutable).
Properties that do _not_ have a setter method are derived properties, and so are not persisted.

Formally speaking, a property is simply a regular JavaBean getter, returning a scalar value recognized by the framework.
Most properties (those that are editable/modifiable) will also have a setter and (if persisted) a backing instance field.
And most properties will also have a number of annotations:

* Apache Isis defines its own set own `@Property` annotation for capturing domain semantics.
It also provides a `@PropertyLayout` for UI hints (though the information in this annotation may instead be provided by a supplementary xref:ugfun.adoc#_ugfun_object-layout[`.layout.xml`] file

* the properties of domain entities are often annotated with the JDO/DataNucleus `@javax.jdo.annotations.Column` annotation.
For property references, there may be other annotations to indicate whether the reference is bidirectional.
It's also possible (using annotations) to define a link table to hold foreign key columns.

* for the properties of view models, then JAXB annotations such as `@javax.xml.bind.annotation.XmlElement` will be present

Apache Isis recognises some of these annotations for JDO/DataNucleus and JAXB and infers some domain semantics from them (for example, the maximum allowable length of a string property).

Since writing getter and setter methods adds quite a bit of boilerplate, it's common to use link:https://projectlombok.org/[Project Lombok] to code generate these methods at compile time (using Java's annotation processor) simply by adding the `@lombok.Getter` and `@lombok.Setter` annotations to the field.
The xref:guides/ugfun.adoc#_ugfun_getting-started_simpleapp-archetype[SimpleApp archetype] uses this approach.

[[__ugfun_how-tos_class-structure_properties_value-vs-reference-types]]
== Value vs Reference Types

Properties can be either a value types (strings, int, date and so on) or be a reference to another object (for example, an `Order` referencing the `Customer` that placed it).

For example, to map a string value type:

[source,java]
----
@lombok.Getter @lombok.Setter       // <1>
private String notes;
----
<1> using link:https://projectlombok.org/[Project Lombok] annotations to reduce boilerplate

You could also add the `@Property` annotation if you wished:

[source,java]
----
@Property
@lombok.Getter @lombok.Setter
private String notes;
----

Although in this case it is not required in this case (none of its attributes have been set).

Or to map a reference type:

[source,java]
----
@lombok.Getter @lombok.Setter
private Customer customer;
----

It's ok for a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_entities[domain entity] to reference another domain entity, and for a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_view-models[view model] to reference both view model and domain entities.
However, it isn't valid for a domain entity to hold a persisted reference to view model (DataNucleus will not know how to persist that view model).

[NOTE]
====
For further details on mapping associations, see the JDO/DataNucleus documentation for link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/one_to_many.html[one-to-many] associations, link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/many_to_one.html[many-to-one] associations, link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/many_to_many.html[many-to-many] associations, and so on.
====

For domain entities, the annotations for mapping value types tend to be different for properties vs action parameters, because JDO annotations are only valid on properties.
The table in the xref:ugfun.adoc#_ugfun_how-tos_class-structure_properties-vs-parameters[Properties vs Parameters] section provides a handy reference of each.

[[__ugfun_how-tos_class-structure_properties_optional-properties]]
== Optional Properties

(For domain entities) JDO/DataNucleus' default is that a property is assumed to be mandatory if it is a primitive type (eg `int`, `boolean`), but optional if a reference type (eg `String`, `BigDecimal` etc).
To override optionality in JDO/DataNucleus the `@Column(allowsNull="...")` annotations is used.

Apache Isis on the other hand assumes that all properties (and action parameters, for that matter) are mandatory, not optional.
These defaults can also be overridden using Apache Isis' own annotations, specifically `@Property(optionality=...)`, or (because it's much less verbose) using `@javax.annotation.Nullable`.

These different defaults can lead to incompatibilities between the two frameworks.
To counteract that, Apache Isis also recognizes and honours JDO's `@Column(allowsNull=...)`.

For example, you can write:

[source,java]
----
@javax.jdo.annotations.Column(allowsNull="true")
@lombok.Getter @lombok.Setter
private LocalDate date;
----

rather than the more verbose:

[source,java]
----
@javax.jdo.annotations.Column(allowsNull="true")
@Property(optionality=Optionality.OPTIONAL)
@lombok.Getter @lombok.Setter
private LocalDate date;
----

The framework will search for any incompatibilities in optionality (whether specified explicitly or defaulted implicitly) between Isis' defaults and DataNucleus, and refuse to boot if any are found (fail fast).

[[__ugfun_how-tos_class-structure_properties_editable-properties]]
== Editable Properties

Apache Isis provides the capability to allow individual properties to be modified.
This is specified using the `@Property(editing=...)` attribute.

For example:

[source,java]
----
@Property(editing = Editing.ENABLED)
@lombok.Getter @lombok.Setter
private String notes;
----

If this is omitted then whether editing is enabled or disabled is defined globally, in the `isis.properties` configuration file; see xref:rgcfg.adoc#__rgcfg_configuring-core_isis-objects-editing[reference configuration guide] for further details.

[[__ugfun_how-tos_class-structure_properties_ignoring-properties]]
== Ignoring Properties

By default Apache Isis will automatically render all properties in the xref:ugvw.adoc[UI] or in the xref:ugvro.adoc[REST API].
To get Apache Isis to ignore a property (exclude it from its metamodel), annotate the getter using `@Programmatic`.

Similarly, you can tell JDO/DataNucleus to ignore a property using the `@javax.jdo.annotations.NotPersistent` annotation.
This is independent of Apache Isis; in other words that property will still be rendered in the UI (unless also annotated with `@Programmatic`).

For view models, you can tell JAXB to ignore a property using the `@javax.xml.bind.annotation.XmlTransient` annotation.
Again, this is independent of Apache Isis.

[[__ugfun_how-tos_class-structure_properties_derived-properties]]
== Derived Properties

Derived properties are those with a getter but no setter.
Provided that the property has not been annotated with `@Programmatic`, these will still be rendered in the UI, but they will be read-only (not editable) and their state will not be persisted.

Subtly different, it is also possible to have non-persisted but still editable properties.
In this case you will need a getter and a setter, but with the getter annotated using `@NotPersistent`.
The implementation of these getters and setters will most likely persist state using other properties (which might be hidden from view using `@Programmatic`).

For example:

[source,java]
----
@javax.jdo.annotations.NotPersistent
@Property(editing=Editing.ENABLED)
public String getAddress() { return addressService.toAddress( getLatLong() ); }             // <1>
public void setAddress(String address) { setLatLong(addressService.toLatLong(address)); }

@javax.jdo.annotations.Column
private String latLong;
@Programmatic
public String getLatLong() { return latLong; }                                              // <2>
public void setLatLong(String latLong) { this.latLong = latLong; }

@javax.inject.Inject
AddressService addressService;                                                              // <3>
----
<1> the representation of the address, in human readable form, eg "10 Downing Street, London, UK"
<2> the lat/long representation of the address, eg "51.503363;-0.127625"
<3> an injected service that can convert to/from address and latLong.

[[__ugfun_how-tos_class-structure_properties_mapping-strings]]
== Mapping ``String``s (Length)

By default JDO/DataNucleus will map string properties to a `VARCHAR(255)`.
To limit the length, use the `@Column(length=...)` annotation.

For example:

[source,java]
----
@javax.jdo.annotations.Column(length=50)
@lombok.Getter @lombok.Setter
private String firstName
----

This is a good example of a case where Apache Isis infers domain semantics from the JDO annotation.

[[__ugfun_how-tos_class-structure_properties_mapping-joda-dates]]
== Mapping JODA Date

Isis' JDO objectstore bundles DataNucleus' http://www.datanucleus.org/documentation/products/plugins.html[built-in support] for Joda `LocalDate` and `LocalDateTime` datatypes, meaning that entity properties of these types will be persisted as appropriate data types in the database tables.

It is, however, necessary to annotate your properties with `@javax.jdo.annotations.Persistent`, otherwise the data won't actually be persisted.
See the link:http://db.apache.org/jdo/field_types.html[JDO docs] for more details on this.

Moreover, these datatypes are _not_ in the default fetch group, meaning that JDO/DataNucleus will perform an additional `SELECT` query for each attribute.
To avoid this extra query, the annotation should indicate that the property is in the default fetch group.

For example, the `ToDoItem` (in the https://github.com/isisaddons/isis-app-todoapp[todoapp example app] (not ASF)) defines the `dueBy` property as follows:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="true")
@javax.jdo.annotations.Column(allowsNull="true")
@Getter @Setter
private LocalDate dueBy;
----

[[__ugfun_how-tos_class-structure_properties_mapping-bigdecimals]]
== Mapping ``BigDecimal``s (Precision)

Working with `java.math.BigDecimal` properties takes a little care due to scale/precision issues.

For example, suppose we have:

[source,java]
----
@lombok.Getter @lombok.Setter
private BigDecimal impact;
----

JDO/DataNucleus creates, at least with HSQL, the table with the field type as NUMERIC(19). No decimal digits are admitted. (Further details http://hsqldb.org/doc/2.0/guide/sqlgeneral-chapt.html#sgc_numeric_types[here]).

What this implies is that, when a record is inserted, a log entry similar to this one appears:

[source,java]
----
INSERT INTO ENTITY(..., IMPACT, ....) VALUES (...., 0.5, ....)
----

But when that same record is retrieved, the log will show that a value of "0" is returned, instead of 0.5.

The solution is to explicitly add the scale to the field like this:

[source,java]
----
@javax.jdo.annotations.Column(scale=2)
@lombok.Getter @lombok.Setter
private BigDecimal impact;
----

In addition, you should also set the scale of the `BigDecimal`, using `setScale(scale, roundingMode)`.

More information can be found http://www.opentaps.org/docs/index.php/How_to_Use_Java_BigDecimal:_A_Tutorial[here] and http://www.tutorialspoint.com/java/math/bigdecimal_setscale_rm_roundingmode.htm[here].

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs]]
== Mapping ``Blob``s and ``Clob``s

Apache Isis configures JDO/DataNucleus so that the properties of type `org.apache.isis.applib.value.Blob` and `org.apache.isis.applib.value.Clob` can also be persisted.

As for xref:ugfun.adoc#__ugfun_how-tos_class-structure_properties_mapping-joda-dates[Joda dates], this requires the `@javax.jdo.annotations.Persistent` annotation.
However, whereas for dates one would always expect this value to be retrieved eagerly, for blobs and clobs it is not so clear cut.

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-blobs]]
=== Mapping ``Blob``s

For example, in the `ToDoItem` class (of the https://github.com/isisaddons/isis-app-todoapp/blob/0333852ddd18ad67e3356fccf805aa442246790d/dom/src/main/java/todoapp/dom/todoitem/ToDoItem.java#L442[todoapp example app] (non-ASF) the `attachment` property is as follows:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype"),
    @javax.jdo.annotations.Column(name = "attachment_bytes", jdbcType="BLOB", sqlType = "LONGVARBINARY")
})
@Property(
        optionality = Optionality.OPTIONAL
)
@lombok.Getter @lombok.Setter
private Blob attachment;
----

The three `@javax.jdo.annotations.Column` annotations are required because the mapping classes that Apache Isis provides (https://github.com/apache/isis/blob/isis-1.4.0/component/objectstore/jdo/jdo-datanucleus/src/main/java/org/apache/isis/objectstore/jdo/datanucleus/valuetypes/IsisBlobMapping.java#L59[IsisBlobMapping] and https://github.com/apache/isis/blob/isis-1.4.0/component/objectstore/jdo/jdo-datanucleus/src/main/java/org/apache/isis/objectstore/jdo/datanucleus/valuetypes/IsisClobMapping.java#L59[IsisClobMapping]) map to 3 columns.
(It is not an error to omit these `@Column` annotations, but without them the names of the table columns are simply suffixed `_0`, `_1`, `_2` etc.

If the `Blob` is mandatory, then use:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name", allowsNull="false"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype", allowsNull="false"),
    @javax.jdo.annotations.Column(name = "attachment_bytes",
                                  jdbcType="BLOB", sqlType = "LONGVARBINARY",
                                  allowsNull="false")
})
@Property(
    optionality = Optionality.MANDATORY
)
@lombok.Getter @lombok.Setter
private Blob attachment;
----

[NOTE]
====
If specifying a `sqlType` of "LONGVARBINARY" does not work, try instead "BLOB".
There can be differences in behaviour between JDBC drivers.
====

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-clobs]]
=== Mapping ``Clob``s

Mapping `Clob`s works in a very similar way, but the `jdbcType` and `sqlType` attributes will, respectively, be `CLOB` and `LONGVARCHAR`:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype"),
    @javax.jdo.annotations.Column(name = "attachment_chars",
                                  jdbcType="CLOB", sqlType = "LONGVARCHAR")
})
private Clob doc;
@Property(
    optionality = Optionality.OPTIONAL
)
public Clob getDoc() {
    return doc;
}
public void setDoc(final Clob doc) {
    this.doc = doc;
}
----

[NOTE]
====
If specifying a `sqlType` of "LONGVARCHAR" does not work, try instead "CLOB".  There can be differences in behaviour between JDBC drivers.
====

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-to-varbinary-or-varchar]]
=== Mapping to VARBINARY or VARCHAR

Instead of mapping to a sqlType of `LONGVARBINARY` (or perhaps `BLOB`), you might instead decide to map to a `VARBINARY`.
The difference is whether the binary data is held "on-row" or as a pointer "off-row"; with a `VARBINARY` the data is held on-row and so you will need to specify a length.

For example:

[source,java]
----
@javax.jdo.annotations.Column(name = "attachment_bytes", jdbcTypr="BLOB", sqlType = "VARBINARY", length=2048)
----

The same argument applies to `LONGVARCHAR` (or `CLOB`); you could instead map to a regular `VARCHAR`:

[source,java]
----
@javax.jdo.annotations.Column(name = "attachment_chars", sqlType = "VARCHAR", length=2048)
----
Support and maximum allowed length will vary by database vendor.

[[__ugfun_how-tos_class-structure_properties_handling-mandatory-properties-in-subtypes]]
== Handling Mandatory Properties in Subtypes

If you have a hierarchy of classes then you need to decide which inheritance strategy to use.

* "table per hierarchy", or "rollup" (`InheritanceStrategy.SUPERCLASS_TABLE`) +
+
whereby a single table corresponds to the superclass, and also holds the properties of the subtype (or subtypes) being rolled up

* "table per class" (`InheritanceStrategy.NEW_TABLE`) +
+
whereby is a table for both superclass and subclass, in 1:1 correspondence

* "rolldown" (`InheritanceStrategy.SUBCLASS_TABLE`) +
+
whereby a single table holds the properties of the subtype, and also holds the properties of its supertype

In the first "rollup" case, we can have a situation where - logically speaking - the property is mandatory in the subtype - but it must be mapped as nullable in the database because it is n/a for any other subtypes that are rolled up.

In this situation we must tell JDO that the column is optional, but to Apache Isis we want to enforce it being mandatory. This can be done using the `@Property(optionality=Optionality.MANDATORY)` annotation.

For example:

[source,java]
----
@javax.jdo.annotations.Inheritance(strategy = InheritanceStrategy.SUPER_TABLE)
public class SomeSubtype extends SomeSuperType {
    @javax.jdo.annotations.Column(allowsNull="true")
    @Property(optionality=Optionality.MANDATORY)
    @lombok.Getter @lombok.Setter
    private LocalDate date;
}
----

[TIP]
====
The `@Property(optionality=...)` annotation is equivalent to the older but still supported `@Optional` annotation and `@Mandatory` annotations.
====

[[_ugfun_how-tos_class-structure_properties]]
= Property
:Notice: Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at. http://www.apache.org/licenses/LICENSE-2.0 . Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR  CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
:_basedir: ../
:_imagesdir: images/

A property is an instance variable of a domain object, of a scalar type, that holds some state about either a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_entities[domain entity] or a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_view-models[view model].

For example, a ``Customer``'s `firstName` would be a property, as would their `accountCreationDate` that they created their account.
All properties have at least a "getter" method, and most properties have also a "setter" method (meaning that they are immutable).
Properties that do _not_ have a setter method are derived properties, and so are not persisted.

Formally speaking, a property is simply a regular JavaBean getter, returning a scalar value recognized by the framework.
Most properties (those that are editable/modifiable) will also have a setter and (if persisted) a backing instance field.
And most properties will also have a number of annotations:

* Apache Isis defines its own set own `@Property` annotation for capturing domain semantics.
It also provides a `@PropertyLayout` for UI hints (though the information in this annotation may instead be provided by a supplementary xref:ugfun.adoc#_ugfun_object-layout[`.layout.xml`] file

* the properties of domain entities are often annotated with the JDO/DataNucleus `@javax.jdo.annotations.Column` annotation.
For property references, there may be other annotations to indicate whether the reference is bidirectional.
It's also possible (using annotations) to define a link table to hold foreign key columns.

* for the properties of view models, then JAXB annotations such as `@javax.xml.bind.annotation.XmlElement` will be present

Apache Isis recognises some of these annotations for JDO/DataNucleus and JAXB and infers some domain semantics from them (for example, the maximum allowable length of a string property).

Since writing getter and setter methods adds quite a bit of boilerplate, it's common to use link:https://projectlombok.org/[Project Lombok] to code generate these methods at compile time (using Java's annotation processor) simply by adding the `@lombok.Getter` and `@lombok.Setter` annotations to the field.
The xref:guides/ugfun.adoc#_ugfun_getting-started_simpleapp-archetype[SimpleApp archetype] uses this approach.

[[__ugfun_how-tos_class-structure_properties_value-vs-reference-types]]
== Value vs Reference Types

Properties can be either a value types (strings, int, date and so on) or be a reference to another object (for example, an `Order` referencing the `Customer` that placed it).

For example, to map a string value type:

[source,java]
----
@lombok.Getter @lombok.Setter       // <1>
private String notes;
----
<1> using link:https://projectlombok.org/[Project Lombok] annotations to reduce boilerplate

You could also add the `@Property` annotation if you wished:

[source,java]
----
@Property
@lombok.Getter @lombok.Setter
private String notes;
----

Although in this case it is not required in this case (none of its attributes have been set).

Or to map a reference type:

[source,java]
----
@lombok.Getter @lombok.Setter
private Customer customer;
----

It's ok for a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_entities[domain entity] to reference another domain entity, and for a xref:ugfun.adoc#__ugfun_how-tos_class-structure_class-definition_view-models[view model] to reference both view model and domain entities.
However, it isn't valid for a domain entity to hold a persisted reference to view model (DataNucleus will not know how to persist that view model).

[NOTE]
====
For further details on mapping associations, see the JDO/DataNucleus documentation for link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/one_to_many.html[one-to-many] associations, link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/many_to_one.html[many-to-one] associations, link:http://www.datanucleus.org/products/accessplatform_4_1/jdo/orm/many_to_many.html[many-to-many] associations, and so on.
====

For domain entities, the annotations for mapping value types tend to be different for properties vs action parameters, because JDO annotations are only valid on properties.
The table in the xref:ugfun.adoc#_ugfun_how-tos_class-structure_properties-vs-parameters[Properties vs Parameters] section provides a handy reference of each.

[[__ugfun_how-tos_class-structure_properties_optional-properties]]
== Optional Properties

(For domain entities) JDO/DataNucleus' default is that a property is assumed to be mandatory if it is a primitive type (eg `int`, `boolean`), but optional if a reference type (eg `String`, `BigDecimal` etc).
To override optionality in JDO/DataNucleus the `@Column(allowsNull="...")` annotations is used.

Apache Isis on the other hand assumes that all properties (and action parameters, for that matter) are mandatory, not optional.
These defaults can also be overridden using Apache Isis' own annotations, specifically `@Property(optionality=...)`, or (because it's much less verbose) using `@javax.annotation.Nullable`.

These different defaults can lead to incompatibilities between the two frameworks.
To counteract that, Apache Isis also recognizes and honours JDO's `@Column(allowsNull=...)`.

For example, you can write:

[source,java]
----
@javax.jdo.annotations.Column(allowsNull="true")
@lombok.Getter @lombok.Setter
private LocalDate date;
----

rather than the more verbose:

[source,java]
----
@javax.jdo.annotations.Column(allowsNull="true")
@Property(optionality=Optionality.OPTIONAL)
@lombok.Getter @lombok.Setter
private LocalDate date;
----

The framework will search for any incompatibilities in optionality (whether specified explicitly or defaulted implicitly) between Isis' defaults and DataNucleus, and refuse to boot if any are found (fail fast).

[[__ugfun_how-tos_class-structure_properties_editable-properties]]
== Editable Properties

Apache Isis provides the capability to allow individual properties to be modified.
This is specified using the `@Property(editing=...)` attribute.

For example:

[source,java]
----
@Property(editing = Editing.ENABLED)
@lombok.Getter @lombok.Setter
private String notes;
----

If this is omitted then whether editing is enabled or disabled is defined globally, in the `isis.properties` configuration file; see xref:rgcfg.adoc#__rgcfg_configuring-core_isis-objects-editing[reference configuration guide] for further details.

[[__ugfun_how-tos_class-structure_properties_ignoring-properties]]
== Ignoring Properties

By default Apache Isis will automatically render all properties in the xref:ugvw.adoc[UI] or in the xref:ugvro.adoc[REST API].
To get Apache Isis to ignore a property (exclude it from its metamodel), annotate the getter using `@Programmatic`.

Similarly, you can tell JDO/DataNucleus to ignore a property using the `@javax.jdo.annotations.NotPersistent` annotation.
This is independent of Apache Isis; in other words that property will still be rendered in the UI (unless also annotated with `@Programmatic`).

For view models, you can tell JAXB to ignore a property using the `@javax.xml.bind.annotation.XmlTransient` annotation.
Again, this is independent of Apache Isis.

[[__ugfun_how-tos_class-structure_properties_derived-properties]]
== Derived Properties

Derived properties are those with a getter but no setter.
Provided that the property has not been annotated with `@Programmatic`, these will still be rendered in the UI, but they will be read-only (not editable) and their state will not be persisted.

Subtly different, it is also possible to have non-persisted but still editable properties.
In this case you will need a getter and a setter, but with the getter annotated using `@NotPersistent`.
The implementation of these getters and setters will most likely persist state using other properties (which might be hidden from view using `@Programmatic`).

For example:

[source,java]
----
@javax.jdo.annotations.NotPersistent
@Property(editing=Editing.ENABLED)
public String getAddress() { return addressService.toAddress( getLatLong() ); }             // <1>
public void setAddress(String address) { setLatLong(addressService.toLatLong(address)); }

@javax.jdo.annotations.Column
private String latLong;
@Programmatic
public String getLatLong() { return latLong; }                                              // <2>
public void setLatLong(String latLong) { this.latLong = latLong; }

@javax.inject.Inject
AddressService addressService;                                                              // <3>
----
<1> the representation of the address, in human readable form, eg "10 Downing Street, London, UK"
<2> the lat/long representation of the address, eg "51.503363;-0.127625"
<3> an injected service that can convert to/from address and latLong.

[[__ugfun_how-tos_class-structure_properties_mapping-strings]]
== Mapping ``String``s (Length)

By default JDO/DataNucleus will map string properties to a `VARCHAR(255)`.
To limit the length, use the `@Column(length=...)` annotation.

For example:

[source,java]
----
@javax.jdo.annotations.Column(length=50)
@lombok.Getter @lombok.Setter
private String firstName
----

This is a good example of a case where Apache Isis infers domain semantics from the JDO annotation.

[[__ugfun_how-tos_class-structure_properties_mapping-joda-dates]]
== Mapping JODA Date

Isis' JDO objectstore bundles DataNucleus' http://www.datanucleus.org/documentation/products/plugins.html[built-in support] for Joda `LocalDate` and `LocalDateTime` datatypes, meaning that entity properties of these types will be persisted as appropriate data types in the database tables.

It is, however, necessary to annotate your properties with `@javax.jdo.annotations.Persistent`, otherwise the data won't actually be persisted.
See the link:http://db.apache.org/jdo/field_types.html[JDO docs] for more details on this.

Moreover, these datatypes are _not_ in the default fetch group, meaning that JDO/DataNucleus will perform an additional `SELECT` query for each attribute.
To avoid this extra query, the annotation should indicate that the property is in the default fetch group.

For example, the `ToDoItem` (in the https://github.com/isisaddons/isis-app-todoapp[todoapp example app] (not ASF)) defines the `dueBy` property as follows:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="true")
@javax.jdo.annotations.Column(allowsNull="true")
@Getter @Setter
private LocalDate dueBy;
----

[[__ugfun_how-tos_class-structure_properties_mapping-bigdecimals]]
== Mapping ``BigDecimal``s (Precision)

Working with `java.math.BigDecimal` properties takes a little care due to scale/precision issues.

For example, suppose we have:

[source,java]
----
@lombok.Getter @lombok.Setter
private BigDecimal impact;
----

JDO/DataNucleus creates, at least with HSQL, the table with the field type as NUMERIC(19). No decimal digits are admitted. (Further details http://hsqldb.org/doc/2.0/guide/sqlgeneral-chapt.html#sgc_numeric_types[here]).

What this implies is that, when a record is inserted, a log entry similar to this one appears:

[source,java]
----
INSERT INTO ENTITY(..., IMPACT, ....) VALUES (...., 0.5, ....)
----

But when that same record is retrieved, the log will show that a value of "0" is returned, instead of 0.5.

The solution is to explicitly add the scale to the field like this:

[source,java]
----
@javax.jdo.annotations.Column(scale=2)
@lombok.Getter @lombok.Setter
private BigDecimal impact;
----

In addition, you should also set the scale of the `BigDecimal`, using `setScale(scale, roundingMode)`.

More information can be found http://www.opentaps.org/docs/index.php/How_to_Use_Java_BigDecimal:_A_Tutorial[here] and http://www.tutorialspoint.com/java/math/bigdecimal_setscale_rm_roundingmode.htm[here].

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs]]
== Mapping ``Blob``s and ``Clob``s

Apache Isis configures JDO/DataNucleus so that the properties of type `org.apache.isis.applib.value.Blob` and `org.apache.isis.applib.value.Clob` can also be persisted.

As for xref:ugfun.adoc#__ugfun_how-tos_class-structure_properties_mapping-joda-dates[Joda dates], this requires the `@javax.jdo.annotations.Persistent` annotation.
However, whereas for dates one would always expect this value to be retrieved eagerly, for blobs and clobs it is not so clear cut.

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-blobs]]
=== Mapping ``Blob``s

For example, in the `ToDoItem` class (of the https://github.com/isisaddons/isis-app-todoapp/blob/0333852ddd18ad67e3356fccf805aa442246790d/dom/src/main/java/todoapp/dom/todoitem/ToDoItem.java#L442[todoapp example app] (non-ASF) the `attachment` property is as follows:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype"),
    @javax.jdo.annotations.Column(name = "attachment_bytes", jdbcType="BLOB", sqlType = "LONGVARBINARY")
})
@Property(
        optionality = Optionality.OPTIONAL
)
@lombok.Getter @lombok.Setter
private Blob attachment;
----

The three `@javax.jdo.annotations.Column` annotations are required because the mapping classes that Apache Isis provides (https://github.com/apache/isis/blob/isis-1.4.0/component/objectstore/jdo/jdo-datanucleus/src/main/java/org/apache/isis/objectstore/jdo/datanucleus/valuetypes/IsisBlobMapping.java#L59[IsisBlobMapping] and https://github.com/apache/isis/blob/isis-1.4.0/component/objectstore/jdo/jdo-datanucleus/src/main/java/org/apache/isis/objectstore/jdo/datanucleus/valuetypes/IsisClobMapping.java#L59[IsisClobMapping]) map to 3 columns.
(It is not an error to omit these `@Column` annotations, but without them the names of the table columns are simply suffixed `_0`, `_1`, `_2` etc.

If the `Blob` is mandatory, then use:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name", allowsNull="false"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype", allowsNull="false"),
    @javax.jdo.annotations.Column(name = "attachment_bytes",
                                  jdbcType="BLOB", sqlType = "LONGVARBINARY",
                                  allowsNull="false")
})
@Property(
    optionality = Optionality.MANDATORY
)
@lombok.Getter @lombok.Setter
private Blob attachment;
----

[NOTE]
====
If specifying a `sqlType` of "LONGVARBINARY" does not work, try instead "BLOB".
There can be differences in behaviour between JDBC drivers.
====

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-clobs]]
=== Mapping ``Clob``s

Mapping `Clob`s works in a very similar way, but the `jdbcType` and `sqlType` attributes will, respectively, be `CLOB` and `LONGVARCHAR`:

[source,java]
----
@javax.jdo.annotations.Persistent(defaultFetchGroup="false", columns = {
    @javax.jdo.annotations.Column(name = "attachment_name"),
    @javax.jdo.annotations.Column(name = "attachment_mimetype"),
    @javax.jdo.annotations.Column(name = "attachment_chars",
                                  jdbcType="CLOB", sqlType = "LONGVARCHAR")
})
private Clob doc;
@Property(
    optionality = Optionality.OPTIONAL
)
public Clob getDoc() {
    return doc;
}
public void setDoc(final Clob doc) {
    this.doc = doc;
}
----

[NOTE]
====
If specifying a `sqlType` of "LONGVARCHAR" does not work, try instead "CLOB".  There can be differences in behaviour between JDBC drivers.
====

[[__ugfun_how-tos_class-structure_properties_mapping-blobs-and-clobs_mapping-to-varbinary-or-varchar]]
=== Mapping to VARBINARY or VARCHAR

Instead of mapping to a sqlType of `LONGVARBINARY` (or perhaps `BLOB`), you might instead decide to map to a `VARBINARY`.
The difference is whether the binary data is held "on-row" or as a pointer "off-row"; with a `VARBINARY` the data is held on-row and so you will need to specify a length.

For example:

[source,java]
----
@javax.jdo.annotations.Column(name = "attachment_bytes", jdbcTypr="BLOB", sqlType = "VARBINARY", length=2048)
----

The same argument applies to `LONGVARCHAR` (or `CLOB`); you could instead map to a regular `VARCHAR`:

[source,java]
----
@javax.jdo.annotations.Column(name = "attachment_chars", sqlType = "VARCHAR", length=2048)
----
Support and maximum allowed length will vary by database vendor.

[[__ugfun_how-tos_class-structure_properties_handling-mandatory-properties-in-subtypes]]
== Handling Mandatory Properties in Subtypes

If you have a hierarchy of classes then you need to decide which inheritance strategy to use.

* "table per hierarchy", or "rollup" (`InheritanceStrategy.SUPERCLASS_TABLE`) +
+
whereby a single table corresponds to the superclass, and also holds the properties of the subtype (or subtypes) being rolled up

* "table per class" (`InheritanceStrategy.NEW_TABLE`) +
+
whereby is a table for both superclass and subclass, in 1:1 correspondence

* "rolldown" (`InheritanceStrategy.SUBCLASS_TABLE`) +
+
whereby a single table holds the properties of the subtype, and also holds the properties of its supertype

In the first "rollup" case, we can have a situation where - logically speaking - the property is mandatory in the subtype - but it must be mapped as nullable in the database because it is n/a for any other subtypes that are rolled up.

In this situation we must tell JDO that the column is optional, but to Apache Isis we want to enforce it being mandatory. This can be done using the `@Property(optionality=Optionality.MANDATORY)` annotation.

For example:

[source,java]
----
@javax.jdo.annotations.Inheritance(strategy = InheritanceStrategy.SUPER_TABLE)
public class SomeSubtype extends SomeSuperType {
    @javax.jdo.annotations.Column(allowsNull="true")
    @Property(optionality=Optionality.MANDATORY)
    @lombok.Getter @lombok.Setter
    private LocalDate date;
}
----

[TIP]
====
The `@Property(optionality=...)` annotation is equivalent to the older but still supported `@Optional` annotation and `@Mandatory` annotations.
====


Update traverse_scenegraph.adoc

Update traverse_scenegraph.adoc

Fixed broken new lines.

= Traverse the SceneGraph
:author:
:revnumber:
:revdate: 2016/03/17 20:48
:keywords: spatial, node, mesh, geometry, scenegraph
:relfileprefix: ../../
:imagesdir: ../..
ifdef::env-github,env-browser[:outfilesuffix: .adoc]

You can run a search across the whole scene graph and search for individual Spatials (`Nodes` and `Geometry`s) by custom criteria, such as the Spatial's name, or the Spatial's class, or the Spatial's user data, or Spatial's Controls. You do this when you want modify  the found nodes (move them, call a method, etc) but you don't have a local variable for them.

== Example Use Cases

*Example 1:*

.  You have created a procedural scene with lots of dynamically generated elements.
.  You want to find individual Spatials under ever-changing conditions and modify their state.

*Example 2:*

.  You created a mostly static scene in the jMonkeyEngine SDK and exported it as .j3o file. +
The scene also contains interactive objects, for example a particle emitter, spatials with user data, or spatials with custom controls.
.  You load the .j3o scene using the assetManager.
.  You want to interact with one of the loaded interactive scene elements in your Java code. +
For example, you want to call `emitAllParticles()` on the particle emitter. Or you want to find all NPC's Geometries with a custom CivilianControl, and call the CivilianControl method that makes them start acting their role.

In this case, you can use a SceneGraphVisitorAdapter to identify and access the Spatials in question.

== Code Sample

For each search, you create a `com.jme3.scene.SceneGraphVisitorAdapter` that defines your search criteria and what you want to do with the found Spatials. Then you call the `depthFirstTraversal(visitor)` or `breadthFirstTraversal(visitor)` method of the Spatial (e.g. the rootNode, or better, a subnode) to start the search.

[source,java]
----

SceneGraphVisitor visitor = new SceneGraphVisitor() {

  @Override
  public void visit(Spatial spat) {
    // search criterion can be control class:
    MyControl control = spatial.getControl(MyControl.class);
    if (control != null) {
      // you have access to any method, e.g. name.
      System.out.println("Instance of " + control.getClass().getName()
                       + " found for " + spatial.getName());
    }
  }

};

// Now scan the tree either depth first...
rootNode.depthFirstTraversal(visitor);
// ... or scan it breadth first.
rootNode.breadthFirstTraversal(visitor);

----

Which of the two methods is faster depends on how you designed the scengraph, and what tree element you are looking for. If you are searching for one single Geometry that is a “leaf of the tree, and then stop searching, depth-first may be faster. If you search for a high-level grouping Node, breadth-first may be faster.

The choice of depth- vs breadth-first also influences the order in which found elements are returned (children first or parents first). If you want to modify user data that is inherited from the parent node (e.g. transformations), the order of application is important, because the side-effects add up.

You can use the SceneGraphVisitorAdapter class to scan separately for Geometry and Nodes.

== See Also

*  <<jme3/the_scene_graph#,The Scene Graph>>
*  <<jme3/advanced/spatial#,Spatial>>

= Traverse the SceneGraph
:author:
:revnumber:
:revdate: 2016/03/17 20:48
:keywords: spatial, node, mesh, geometry, scenegraph
:relfileprefix: ../../
:imagesdir: ../..
ifdef::env-github,env-browser[:outfilesuffix: .adoc]

You can run a search across the whole scene graph and search for individual Spatials (`Nodes` and `Geometry`s) by custom criteria, such as the Spatial's name, or the Spatial's class, or the Spatial's user data, or Spatial's Controls. You do this when you want modify  the found nodes (move them, call a method, etc) but you don't have a local variable for them.

== Example Use Cases

*Example 1:*

.  You have created a procedural scene with lots of dynamically generated elements.
.  You want to find individual Spatials under ever-changing conditions and modify their state.

*Example 2:*

.  You created a mostly static scene in the jMonkeyEngine SDK and exported it as .j3o file. +The scene also contains interactive objects, for example a particle emitter, spatials with user data, or spatials with custom controls.
.  You load the .j3o scene using the assetManager.
.  You want to interact with one of the loaded interactive scene elements in your Java code. +For example, you want to call `emitAllParticles()` on the particle emitter. Or you want to find all NPC's Geometries with a custom CivilianControl, and call the CivilianControl method that makes them start acting their role.

In this case, you can use a SceneGraphVisitorAdapter to identify and access the Spatials in question.

== Code Sample

For each search, you create a `com.jme3.scene.SceneGraphVisitorAdapter` that defines your search criteria and what you want to do with the found Spatials. Then you call the `depthFirstTraversal(visitor)` or `breadthFirstTraversal(visitor)` method of the Spatial (e.g. the rootNode, or better, a subnode) to start the search.

[source,java]
----

SceneGraphVisitor visitor = new SceneGraphVisitor() {

  @Override
  public void visit(Spatial spat) {
    // search criterion can be control class:
    MyControl control = spatial.getControl(MyControl.class);
    if (control != null) {
      // you have access to any method, e.g. name.
      System.out.println("Instance of " + control.getClass().getName()
                       + " found for " + spatial.getName());
    }
  }

};

// Now scan the tree either depth first...
rootNode.depthFirstTraversal(visitor);
// ... or scan it breadth first.
rootNode.breadthFirstTraversal(visitor);

----

Which of the two methods is faster depends on how you designed the scengraph, and what tree element you are looking for. If you are searching for one single Geometry that is a “leaf of the tree, and then stop searching, depth-first may be faster. If you search for a high-level grouping Node, breadth-first may be faster.

The choice of depth- vs breadth-first also influences the order in which found elements are returned (children first or parents first). If you want to modify user data that is inherited from the parent node (e.g. transformations), the order of application is important, because the side-effects add up.

You can use the SceneGraphVisitorAdapter class to scan separately for Geometry and Nodes.

== See Also

*  <<jme3/the_scene_graph#,The Scene Graph>>
*  <<jme3/advanced/spatial#,Spatial>>


Update 2018-04-16-When-is-using-a-Blockchain-compelling.adoc

= When is using a Blockchain compelling
Peter Lawrey
:published_at: 2018-04-16
:hp-tags: Block Chain, Use Case

I was recently asked, when is using a Blockchain really needed?

IMHO the main benefit is **ease of provider integration** Blockchain allows providers to join a system, with a much lower barrier to entry. If you are looking to build a network of new and innovative providers, which don't have to trust each other, instead they trust the protocol, a blockchain is a compelling solution.

=== When is Blockchain not a compelling reason?

First I feel I should debunk some reasons often stated.

==== Blockchain doesn't make everything better

In many cases, when a blockchain is proposed, it is used as little more than a log of changes in a database. In some cases, a distributed ledger could be used when a centralised database isn't enough. Blockchain projects are easier to get approved these days as companies and individuals want to improve their blockchain skill sets but this doesn't make it compelling from a technical point of view IMHO.

==== Blockchain is unlikely to make things faster

In most cases you will still need a database for reporting services, so adding a blockchain is highly likely to make it more complicated and slower, not simpler and faster than a database alone. https://www2.deloitte.com/nl/nl/pages/financial-services/articles/5-blockchain-use-cases-in-financial-services.html[See this article which mention speeding up payments]

Blockchain can make a system faster if it can replace enough of an existing system which already implements the features a blockchain offers, but in a less efficient manner.

==== Blockchain doesn't address all security concerns

To compromise a cluster of nodes you can employ a https://learncryptography.com/cryptocurrency/51-attack[51% attack]. If you control a majority of nodes, you control the whole system.  This could happen by adding or hacking enough nodes. A state level organisation could do this, potentially even with the most popular crypto currencies, however most governments have easier/cheaper ways of exercising policy or influence.

Many cryptocurrencies are public which is a real privacy concern when some of the bank balances are very high (or at least high to the individuals holding them).

At the time of writing https://bitinfocharts.com/bitcoin/address/3D2oetdNuZUqQHPJmcMDDHYoqkyNVsFk9r[This bitcoin address] has $1.5 bn, and this https://etherscan.io/address/0x281055afc982d96fab65b3a49cac8b878184cb16[ethereum address] has around $770 m which makes them targets for hackers and organised crime.

Making this information private is likely to be highly desirable. However it could still be obtained through a hack though that is harder.

=== When is Blockchain compelling?

When participants trust a protocol, they don't have to trust each other as much. This makes inclusion of new participants easier and allow existing ones to be more innovative.

==== Security

Reduced security impacts via a https://www.forbes.com/sites/bernardmarr/2017/08/10/practical-examples-of-how-blockchains-are-used-in-banking-and-the-financial-services-sector/[immutable ledger] This means a new provider can join without all the existing providers needing to be fully comfortable with their security setup.

NOTE: A high percentage of hacks involve disgruntled former employees. If you trust every node from every provider you have to trust every person operating it.

==== Innovative solutions

Using sub-chains, different providers can offer innovative solutions, without everyone else in the system needing to be comfortable with those solutions. By the time every participant is comfortable with a new solution, it's probably not that innovative/disruptive. If every node needs to be trusted, you can't offer a solution/transaction type unless every provider is comfortable with the risk associated with it.

==== Built in fee support

As the system controls the flow of value, it can also include settlement of fees and cut off participants as a result of exceeding some limit. e.g. if a participant, for whatever reason, starts spamming the system, this will cost them and result in them being cut off.

=== Does a blockchain need to have its own currency?

A blockchain offers a virtualisation of currencies.  These can be backed by fiat or another digital currencies, or a new currency or both.  If the system has fees to cover the cost of running the service, it needs to be in a supported currency.  This could be in USD or EUR rather a new currency, however a crypto currency has the advantage that the actual cost can have a floating exchanging rate with fiat currencies. e.g. each transaction could cost 1 TXN and the value of this can be determined by an exchange.

New currencies can have novel features such as only being usable after a future date. You could have a 1000 USD token which is only usable after 1st Jan 2020, but can be bought now at a discount.

=== Conclusion

While I feel than many blockchain solutions could have used a database instead, there are some compelling use cases which allow multiple providers to offer innovating solutions to customers.

= When is using a Blockchain compelling
Peter Lawrey
:published_at: 2018-04-16
:hp-tags: Block Chain, Use Case

I was recently asked; when is using a Blockchain really needed?

IMHO the main benefit is **ease of provider integration** Blockchain allows providers to join a system, with a much lower barrier to entry. If you are looking to build a network of new and innovative providers, which don't have to trust each other, instead they trust the protocol, a blockchain is a compelling solution.

=== When is Blockchain not a compelling reason?

First I feel I should debunk some reasons often stated

==== Blockchain doesn't make everything better

In many cases when a blockchain is proposed, it is used as little more than a log of changes in a database. In some cases, a distributed ledger could be used when a centralised database isn't enough. Blockchain projects are easier to get approved these days as companies and individuals want to improve their blockchain skill sets but this doesn't make it compelling from a technical point of view IMHO.

==== Blockchain is unlikely to make things faster

In most cases you will still need a database for reporting services, so adding a blockchain is highly likely to make it more complicated and slower, not simpler and faster than a database alone. https://www2.deloitte.com/nl/nl/pages/financial-services/articles/5-blockchain-use-cases-in-financial-services.html[To speed up a process]

Blockchain can make a system faster if it can replace enough of an existing system which already implements the features a blockchain offers but in a less efficient manner.

==== Blockchain doesn't address all security concerns

To compromise a cluster of nodes you can employ a https://learncryptography.com/cryptocurrency/51-attack[51% attack]. If you control a majority of nodes, you control the whole system.  This could happen by adding or hacking enough nodes. A state level organisation could do this, potentially even with the most popular crypto currencies, however most govenments have easier/cheaper ways of exercising policy or influence.

Many cryptocurrencies are public which is a real privacy concern when some of the bank balances are very high (or at least high to the individuals holding them)

At the time of writing https://bitinfocharts.com/bitcoin/address/3D2oetdNuZUqQHPJmcMDDHYoqkyNVsFk9r[This bitcoin address] has $1.5 bn, and this https://etherscan.io/address/0x281055afc982d96fab65b3a49cac8b878184cb16[ethereum address] has around $770 m which makes them targets for hackers and organised crime.

Making this information private is likely to be highly desirable. However it could still be obtained through a hack though that is harder.

=== When is Blockchain compelling?

When participants trust a protocol, they don't have to trust each other as much. This makes inclusion of new participants easier and allow existing ones to be more innovative.

==== Security

Reduced security impacts via a https://www.forbes.com/sites/bernardmarr/2017/08/10/practical-examples-of-how-blockchains-are-used-in-banking-and-the-financial-services-sector/[immutable ledger] This means a new provider can joins without all the existing providers needing to be fully comfortable with their security setup.

NOTE: A high percentage of hacks involve disgruntled former employees. If you trust every node from every provider you have to trust every person operating it.

==== Innovative solutions

Using sub-chains, different providers can offer innovative solutions, without every one else in the system needing to be comfortable with those solutions. By the time every participant is comfortable with a new solution, it probably not that innovative/disruptive. If every node needs to be trusted, you can't offer a solution/transaction type unless every provider is comfortable with the risk associated with it.

==== Built in fee support

As the system controls the flow of value, it can also include settlement of fees and cut off participants as a result of exceeding some limit. e.g. if a participant, for what ever reason, starts spamming the system, this will cost them and result in them being cut off.

=== Does a blockchain need to have it's own currency?

A blockchain offers a virtualisation of currencies.  These can be backed by fiat or another digital currencies, or a new currency or both.  If the system has fees to cover the cost of running the service, it needs to be in a supported currency.  This could be in USD or EUR rather a new currency, however a crypto currency has the advantage that the actual cost can have a floating exchanging rate with fiat currencies. e.g. each transaction could cost 1 TXN and the value of this can be determined by an exchange.

New currencies can have novel features such as only being usable after a future date. You could have a 1000 USD token which is only usable after 1st Jan 2020, but can be bought now at a discount.

=== Conclusion

While I feel than many blockchain soltuions could have used a database instead, there are some compelling use cases which allow multiple providers to offer innovating solutions to customers.