Prepare for release version 4.1.4

Prepare for release version 4.1.4

= [.ebi-color]#Release notes#
:toc: auto

This pages contains links to release notes for BioSamples for version 4.0.0 and higher. This release represents a comprehensive overhaul and therefore previous release notes are no longer applicable.

[[section]]
4.1.4
----

* As part of curation pipeline attributes with the value "not_applicable" are removed
* Date titles on the sample page are now "Releases on" and "Updated on" rather than "Release" and "Update"
* An initial accession endpoint has been added to the REST API to enable ENA to get a list of accessions for a project
* A multi-step Docker build has been added to allow Docker images to be distributed on quay.io
* A fix has been made for an issue that caused the Zooma Pipeline to fail on wwwdev

[[section]]
4.1.3
----

* Additional sample attributes required by ENA are now available including a single, top-level taxId field
* The export box for a sample is now renamed download and contains a list of serialisations that always download as a file fixing a blocked popups issue in Safari
* The search results now have an updated look and feel based on feedback from ENA

[[section]]
4.1.2
----

* Sample JSON now contains a numeric taxId field at the top level
* IRI of ontology terms now resolve to the defining ontology when they are available in multiple ontologies
* Requests for a sample now contain a computed ETag header to identify changes
* When requesting a private sample an explanation message is now provided in addition to the 403 error code
* The search UI now contains a clear filters button

[[section]]
4.1.1
----

* Expose the BioSchemas markup with enhanced context and Sample ontology code
* SampleTab submission pipeline has been rewritten for better robustness
* In the samples results page, the sample name and the sample accession are now linking to the single sample page
* Fixed various broken hyperlinks on the home page and in documentation

[[section]]
4.1.0
----

### New features

* GDPR:
** SampleTab submissions enforce explicit acceptance of the terms of service and the privacy information
** GDPR notices added throughout
* SampleTab where targets of relationships are neither sample name nor sample accession are now rejected, providing user additional information on the problematic data
* *Bioschema.org* entities are exported in BioSamples and available both in the UI - embedded in a script tag - and through the API

### Bug fixes
* Solved issues with wrong header’s hyperlinks
* Solved issue with resolving relationship by name in SampleTab submissions
* Solved issue with converting DatabaseURI to external references in SampleTab submissions
* Improved special characters handling in SampleTab submissions

[[section]]
4.0.7
-----

This is a bugfix release that addresses the following issues:

* GDPR notices
* Update format of the Sitemap file

[[section]]
4.0.6
-----

This is a bugfix release that addresses the following issues:

* Improves search handling of special characters in facets
* Improves search handling of special characters in search terms
* Fix issue with curation link URLs
* Implemented DataCatalog, Dataset and DataRecord profiles on JSON+LD
* Add ability to control which curation domains are applied to a sample
* Updated and improved API documentation
* Updated and improved SampleTab documentation
* Fix links to XML and JSON serialisation in the UI
* Fix bug in handling special characters in SampleTab submission
* Add export pipeline
* Add copy down pipeline

[[section]]
4.0.5
-----

This is a bugfix release that addresses the following issues:

* Improved consistency of paged search results if any of the samples are added or modified whilst paging
* Improved search update throughput by using Solr transaction log
* Updated JSON+LD format to the latest version
* Correctly accept XML sample groups and their related samples
* Fix issue related to search query terms not being applied to legacy XML and legacy JSON endpoints.
* Fix incorrect HAL links on autocomplete endpoint
* Replace SampleTab submitted relationships by name with accessions. As a consequence, they can now be consistently cross referenced by accession in user interface and API
* Improved indexing of samples when they are rapidly updated or curated
* Updated Elixir Deposition Database banner URL
* Reduce number of Zooma calls by not attempting to map "unknown" or "other" attributes
* Reduce load on OLS by ensuring Zooma does not requery OLS as any results from OLS would not be used by BioSamples

[[section]]
4.0.4
-----

This is a bugfix release that addresses the following issues:

* Persistence of search terms and filters when using HAL paging links
* SameAs relation in the legacy JSON API works as intended
* Removed residual test endpoints from legacy JSON API
* Details relation in legacy JSON API now correctly resolves
* Added informative and specific title to webpages
* Added https://www.elixir-europe.org/platforms/data/elixir-deposition-databases[Elixir Deposition Database] banner

[[section]]
4.0.3
-----

This is a bugfix release that addresses the following issues:

* Forward legacy group URLs /biosamples/groups/SAMEGxxxx to /biosamples/samples/SAMEGxxxxx
* Missing or malformed update and release date on legacy XML group submission will default to current datetime. It is not recommended that users intentionally rely on this.
* Index legacy XML group submissions, which was not happening due to an unexpected consequence of the interaction of components.
* Redirect /biosamples/sample and /biosamples/group URLs in case of typo

[[section]]
4.0.2
-----

This is a bugfix release that addresses the following issues:

* Fix javascript on SampleTab submission and accession
* Handle load-balanced accessioning
* Fix for storage of relationships source on new samples

[[section]]
4.0.1
-----

This is a bugfix release that addresses the following issues:

* Fix submission of new unaccessioned samples with relationships by inserting an assigned accession into the source of any relationships that are missing it.
* Fix curation pipeline of numeric organism iri to "http://purl.obolibrary.org/obo/NCBITaxon_+taxId" when it should be "http://purl.obolibrary.org/obo/NCBITaxon_"+taxId e.g. http://purl.obolibrary.org/obo/NCBITaxon_9606
* Allow CORS requests for legacy XML APIs.
* Updated homepage project sample links to use a filter search rather than a text search.

[[section]]
4.0.0
-----

Version v4.0.0 represents a re-architecture and re-engineering of the
BioSamples software stack. It is now based on the Java
https://projects.spring.io/spring-boot[Spring-Boot] framework, utilising
https://www.mongodb.com[MongoDB] for storage and
https://lucene.apache.org/solr[Solr] for indexing and search. It tries
to follow up-to-date web standards and conventions, while remaining
backwards compatible. This will also give us a strong and stable
foundation to build more features and improvements from, more reliably
and more rapidly.

Highlights include:

* Submissions and updates will be available immediately via accession,
and will be available via search within a few minutes or less. There is
also improved handling of submissions and updates, with fewer errors and
better feedback about any problems.
* Integration with https://aap.tsi.ebi.ac.uk[EBI AAP] for login
management and access to pre-publication samples, including use of
https://www.elixir-europe.org/services/compute/aai[ELIXIR AAI] single
sign-on accounts.
* Separation of submitted sample information from curation of that
information, including the ability for 3rd party (re-)curation of
samples. Please contact us if you would be interested in more
information and/or to supply curation information.
* Improved handling of non-alphanumeric characters in attribute types
e.g. "geographic location (country and/or sea)"
* Improved faceting allowing selection of multiple values within same
facet, fixed re-use and re-distribution of search URLs. This will be
expanded in future with additional facet types where appropriate.
* Support and recommend the use
of https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation[content
negotiation] to accessing multiple formats at the same URIs. In addition
to the content (HTML vs XML vs JSON) this also supports
https://developer.mozilla.org/en-US/docs/Web/HTTP/Compression[compression]
and https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching[caching]
through standard mechanisms.
* Java client using Spring, and a Spring-Boot starter module for easy
use. This is used by BioSamples internally and other teams at EMBL-EBI,
so is high performance and battle tested.
* Containerisation using Docker and Docker-Compose, which makes it
easier to run a local version for client development or for local
storage of sample information.

[[data-content]]
Data content
~~~~~~~~~~~~

* Ontology terms Numeric tax IDs (e.g. 9606) and short ontology terms
(e.g. PATO:0000384) are being replaced with full IRIs (e.g.
http://purl.obolibrary.org/obo/NCBITaxon_9606 and http://purl.obolibrary.org/obo/PATO_0000384 )
in many places, eventually everywhere.
* Groups will continue to exist for backwards compatibility purposes.
However, we are investigating future development to reduce or remove
many of these in favour of alternatives such as filtering samples by
external link, or delegating grouping of samples to other EMBL-EBI
archives such as https://www.ebi.ac.uk/biostudies[BioStudies].

[[jsonbiosamples]]
JSON `/biosamples`
~~~~~~~~~~~~~~~~~~

This is the preferred API for use, and uses the same URIs as the HTML
pages, and utilising content negotiation to provide a JSON response.
This is designed as
a https://en.wikipedia.org/wiki/Hypertext_Application_Language[hypermedia
as the engine of application state (HATEOS) API] and therefore we
recommend users do not use specific URLs but rather follow relationships
between API endpoints, much like a user would use links between HTML
pages. It is similar to the `/biosamples/api` JSON format, with a few
critical differences:

* added __release__ in full ISO 8601 format including time. The
backwards-compatible __releaseDate__ exists but should be considered
deprecated and will be removed in a future release.
* added __update__ in full ISO 8601 format including time.
The backwards-compatible __updateDate__ exists but should be considered
deprecated and will be removed in a future release.
* removed __description__ as a separate field, is now available as
a __characteristic__.
* remove **relations** rel link; equivalent information is now embedded
in sample in __relationships__ and __externalReferences__ lists.
* remove **sample** rel link; with relations now embedded, this link
serves no purpose.
* added **curationLinks** rel link.
* ordering may be different.
* fields are not displayed if empty or null.
* characteristic names accurately reflect what was submitted and may now
be multiple words and may include non alphanumeric characters (e.g
brackets, greek letters, etc). In the `/biosamples/api` responses
characteristic names were always camelCased and with non-alphanumeric
characters removed.
* external references directly embedded in the samples and the groups.

[[xmlbiosamplesxml]]
XML `/biosamples/xml`
~~~~~~~~~~~~~~~~~~~~~

We are maintaining this for backwards compatibility. Later in 2018 we
will be consulting about future development of this API, particularly in
the context of the improved JSON `/biosamples` API using content
negotiation and several long-standing issues with limitations arising
from the XML schema in use.

* XML element *TermSourceREF* element *Name* and element *URI* are
removed.
* XML element *Property* attributes characteristic and comment always
false.
* elements and attributes may be in different order.
* allows only one IRI on attributes, so in rare cases of multiple IRIs
will not be complete.
* Query parameter `query` has now a default value of * if none is
provided.
* Query parameter `sort` is ignored for the search, due to undefined
behaviour and lack of usage.

[[json-biosamplesapi]]
JSON `/biosamples/api`
~~~~~~~~~~~~~~~~~~~~~~

This API should be considered **deprecated** and we will aim to remove
it by 2019. Any users of this should move to using the `/biosamples`
URIs to retrieve JSON representations with an improved schema via
content negotiation. Further announcements will be made in future for
specific updates and deadlines.

* ordering may be different from previous versions, and is not
guaranteed for future versions.
* fields are not displayed if empty or null.
* `/api/externallinksrelations/{id}/sample` and
`/api/externallinksrelations/{id}/group` are removed due to lack of
usage.
* fixed _externalReferences_ and _publications_ to be nested objects and
not JSON strings.

[[acknowledgements]]
Acknowledgements
~~~~~~~~~~~~~~~~

This release has been made possible with the support of our funders:

* EMBL-EBI Core Funds
* EC -ELIXIR-EXCELERATE
* WT- HIPSCI
* IMI - EBiSC
* ELIXIR – Meta Data Implementation Study
* WT-GA4GH

= [.ebi-color]#Release notes#
:toc: auto

This pages contains links to release notes for BioSamples for version 4.0.0 and higher. This release represents a comprehensive overhaul and therefore previous release notes are no longer applicable.

[[section]]
4.1.3
----

* Additional sample attributes required by ENA are now available including a single, top-level taxId field
* The export box for a sample is now renamed download and contains a list of serialisations that always download as a file fixing a blocked popups issue in Safari
* The search results now have an updated look and feel based on feedback from ENA

[[section]]
4.1.2
----

* Sample JSON now contains a numeric taxId field at the top level
* IRI of ontology terms now resolve to the defining ontology when they are available in multiple ontologies
* Requests for a sample now contain a computed ETag header to identify changes
* When requesting a private sample an explanation message is now provided in addition to the 403 error code
* The search UI now contains a clear filters button

[[section]]
4.1.1
----

* Expose the BioSchemas markup with enhanced context and Sample ontology code
* SampleTab submission pipeline has been rewritten for better robustness
* In the samples results page, the sample name and the sample accession are now linking to the single sample page
* Fixed various broken hyperlinks on the home page and in documentation

[[section]]
4.1.0
----

### New features

* GDPR:
** SampleTab submissions enforce explicit acceptance of the terms of service and the privacy information
** GDPR notices added throughout
* SampleTab where targets of relationships are neither sample name nor sample accession are now rejected, providing user additional information on the problematic data
* *Bioschema.org* entities are exported in BioSamples and available both in the UI - embedded in a script tag - and through the API

### Bug fixes
* Solved issues with wrong header’s hyperlinks
* Solved issue with resolving relationship by name in SampleTab submissions
* Solved issue with converting DatabaseURI to external references in SampleTab submissions
* Improved special characters handling in SampleTab submissions

[[section]]
4.0.7
-----

This is a bugfix release that addresses the following issues:

* GDPR notices
* Update format of the Sitemap file

[[section]]
4.0.6
-----

This is a bugfix release that addresses the following issues:

* Improves search handling of special characters in facets
* Improves search handling of special characters in search terms
* Fix issue with curation link URLs
* Implemented DataCatalog, Dataset and DataRecord profiles on JSON+LD
* Add ability to control which curation domains are applied to a sample
* Updated and improved API documentation
* Updated and improved SampleTab documentation
* Fix links to XML and JSON serialisation in the UI
* Fix bug in handling special characters in SampleTab submission
* Add export pipeline
* Add copy down pipeline

[[section]]
4.0.5
-----

This is a bugfix release that addresses the following issues:

* Improved consistency of paged search results if any of the samples are added or modified whilst paging
* Improved search update throughput by using Solr transaction log
* Updated JSON+LD format to the latest version
* Correctly accept XML sample groups and their related samples
* Fix issue related to search query terms not being applied to legacy XML and legacy JSON endpoints.
* Fix incorrect HAL links on autocomplete endpoint
* Replace SampleTab submitted relationships by name with accessions. As a consequence, they can now be consistently cross referenced by accession in user interface and API
* Improved indexing of samples when they are rapidly updated or curated
* Updated Elixir Deposition Database banner URL
* Reduce number of Zooma calls by not attempting to map "unknown" or "other" attributes
* Reduce load on OLS by ensuring Zooma does not requery OLS as any results from OLS would not be used by BioSamples

[[section]]
4.0.4
-----

This is a bugfix release that addresses the following issues:

* Persistence of search terms and filters when using HAL paging links
* SameAs relation in the legacy JSON API works as intended
* Removed residual test endpoints from legacy JSON API
* Details relation in legacy JSON API now correctly resolves
* Added informative and specific title to webpages
* Added https://www.elixir-europe.org/platforms/data/elixir-deposition-databases[Elixir Deposition Database] banner

[[section]]
4.0.3
-----

This is a bugfix release that addresses the following issues:

* Forward legacy group URLs /biosamples/groups/SAMEGxxxx to /biosamples/samples/SAMEGxxxxx
* Missing or malformed update and release date on legacy XML group submission will default to current datetime. It is not recommended that users intentionally rely on this.
* Index legacy XML group submissions, which was not happening due to an unexpected consequence of the interaction of components.
* Redirect /biosamples/sample and /biosamples/group URLs in case of typo

[[section]]
4.0.2
-----

This is a bugfix release that addresses the following issues:

* Fix javascript on SampleTab submission and accession
* Handle load-balanced accessioning
* Fix for storage of relationships source on new samples

[[section]]
4.0.1
-----

This is a bugfix release that addresses the following issues:

* Fix submission of new unaccessioned samples with relationships by inserting an assigned accession into the source of any relationships that are missing it.
* Fix curation pipeline of numeric organism iri to "http://purl.obolibrary.org/obo/NCBITaxon_+taxId" when it should be "http://purl.obolibrary.org/obo/NCBITaxon_"+taxId e.g. http://purl.obolibrary.org/obo/NCBITaxon_9606
* Allow CORS requests for legacy XML APIs.
* Updated homepage project sample links to use a filter search rather than a text search.

[[section]]
4.0.0
-----

Version v4.0.0 represents a re-architecture and re-engineering of the
BioSamples software stack. It is now based on the Java
https://projects.spring.io/spring-boot[Spring-Boot] framework, utilising
https://www.mongodb.com[MongoDB] for storage and
https://lucene.apache.org/solr[Solr] for indexing and search. It tries
to follow up-to-date web standards and conventions, while remaining
backwards compatible. This will also give us a strong and stable
foundation to build more features and improvements from, more reliably
and more rapidly.

Highlights include:

* Submissions and updates will be available immediately via accession,
and will be available via search within a few minutes or less. There is
also improved handling of submissions and updates, with fewer errors and
better feedback about any problems.
* Integration with https://aap.tsi.ebi.ac.uk[EBI AAP] for login
management and access to pre-publication samples, including use of
https://www.elixir-europe.org/services/compute/aai[ELIXIR AAI] single
sign-on accounts.
* Separation of submitted sample information from curation of that
information, including the ability for 3rd party (re-)curation of
samples. Please contact us if you would be interested in more
information and/or to supply curation information.
* Improved handling of non-alphanumeric characters in attribute types
e.g. "geographic location (country and/or sea)"
* Improved faceting allowing selection of multiple values within same
facet, fixed re-use and re-distribution of search URLs. This will be
expanded in future with additional facet types where appropriate.
* Support and recommend the use
of https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation[content
negotiation] to accessing multiple formats at the same URIs. In addition
to the content (HTML vs XML vs JSON) this also supports
https://developer.mozilla.org/en-US/docs/Web/HTTP/Compression[compression]
and https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching[caching]
through standard mechanisms.
* Java client using Spring, and a Spring-Boot starter module for easy
use. This is used by BioSamples internally and other teams at EMBL-EBI,
so is high performance and battle tested.
* Containerisation using Docker and Docker-Compose, which makes it
easier to run a local version for client development or for local
storage of sample information.

[[data-content]]
Data content
~~~~~~~~~~~~

* Ontology terms Numeric tax IDs (e.g. 9606) and short ontology terms
(e.g. PATO:0000384) are being replaced with full IRIs (e.g.
http://purl.obolibrary.org/obo/NCBITaxon_9606 and http://purl.obolibrary.org/obo/PATO_0000384 )
in many places, eventually everywhere.
* Groups will continue to exist for backwards compatibility purposes.
However, we are investigating future development to reduce or remove
many of these in favour of alternatives such as filtering samples by
external link, or delegating grouping of samples to other EMBL-EBI
archives such as https://www.ebi.ac.uk/biostudies[BioStudies].

[[jsonbiosamples]]
JSON `/biosamples`
~~~~~~~~~~~~~~~~~~

This is the preferred API for use, and uses the same URIs as the HTML
pages, and utilising content negotiation to provide a JSON response.
This is designed as
a https://en.wikipedia.org/wiki/Hypertext_Application_Language[hypermedia
as the engine of application state (HATEOS) API] and therefore we
recommend users do not use specific URLs but rather follow relationships
between API endpoints, much like a user would use links between HTML
pages. It is similar to the `/biosamples/api` JSON format, with a few
critical differences:

* added __release__ in full ISO 8601 format including time. The
backwards-compatible __releaseDate__ exists but should be considered
deprecated and will be removed in a future release.
* added __update__ in full ISO 8601 format including time.
The backwards-compatible __updateDate__ exists but should be considered
deprecated and will be removed in a future release.
* removed __description__ as a separate field, is now available as
a __characteristic__.
* remove **relations** rel link; equivalent information is now embedded
in sample in __relationships__ and __externalReferences__ lists.
* remove **sample** rel link; with relations now embedded, this link
serves no purpose.
* added **curationLinks** rel link.
* ordering may be different.
* fields are not displayed if empty or null.
* characteristic names accurately reflect what was submitted and may now
be multiple words and may include non alphanumeric characters (e.g
brackets, greek letters, etc). In the `/biosamples/api` responses
characteristic names were always camelCased and with non-alphanumeric
characters removed.
* external references directly embedded in the samples and the groups.

[[xmlbiosamplesxml]]
XML `/biosamples/xml`
~~~~~~~~~~~~~~~~~~~~~

We are maintaining this for backwards compatibility. Later in 2018 we
will be consulting about future development of this API, particularly in
the context of the improved JSON `/biosamples` API using content
negotiation and several long-standing issues with limitations arising
from the XML schema in use.

* XML element *TermSourceREF* element *Name* and element *URI* are
removed.
* XML element *Property* attributes characteristic and comment always
false.
* elements and attributes may be in different order.
* allows only one IRI on attributes, so in rare cases of multiple IRIs
will not be complete.
* Query parameter `query` has now a default value of * if none is
provided.
* Query parameter `sort` is ignored for the search, due to undefined
behaviour and lack of usage.

[[json-biosamplesapi]]
JSON `/biosamples/api`
~~~~~~~~~~~~~~~~~~~~~~

This API should be considered **deprecated** and we will aim to remove
it by 2019. Any users of this should move to using the `/biosamples`
URIs to retrieve JSON representations with an improved schema via
content negotiation. Further announcements will be made in future for
specific updates and deadlines.

* ordering may be different from previous versions, and is not
guaranteed for future versions.
* fields are not displayed if empty or null.
* `/api/externallinksrelations/{id}/sample` and
`/api/externallinksrelations/{id}/group` are removed due to lack of
usage.
* fixed _externalReferences_ and _publications_ to be nested objects and
not JSON strings.

[[acknowledgements]]
Acknowledgements
~~~~~~~~~~~~~~~~

This release has been made possible with the support of our funders:

* EMBL-EBI Core Funds
* EC -ELIXIR-EXCELERATE
* WT- HIPSCI
* IMI - EBiSC
* ELIXIR – Meta Data Implementation Study
* WT-GA4GH


changed wording on links (#1362)

changed wording on links (#1362)

[[quickstart-guide-intro]]
<<<

:linkattrs:

=== Bootstrap Scripts

[[quickstart-guide-intro]]
GeoWave currently supports the use of either Accumulo or HBase, so the version of the bootstrap script you
use when setting up your cluster will be dependent upon which system you want to use as your datastore. The bootstrap scripts help to set up geowave and your environment. Click on any of the links below to download the script.

- For Accumulo use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/accumulo/bootstrap-geowave.sh[s3://geowave/latest/scripts/emr/accumulo/bootstrap-geowave.sh]
- For HBase use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/hbase/bootstrap-geowave.sh[s3://geowave/latest/scripts/emr/hbase/bootstrap-geowave.sh]

These scripts will provide you with everything necessary to complete this guide, but will also set up Geowave if you would like to use it in other workflows. There are scripts available for each of the releases of Geowave going back to 0.9.3. These can be used by replacing ``/latest/`` with the desired release (i.e. ``/0.9.3/``)

[NOTE]
====
We have also provided quickstart scripts that will perform all of the steps in this guide automatically. This will allow you to verify your own steps, or test out other geowave commands and features on an already conditioned data set.

If you would prefer to have all of the steps run automatically, please use these bootstrap scripts instead of the
ones listed previously:

- For Accumulo use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/accumulo/bootstrap-geowave.sh[s3://geowave/latest/scripts/emr/quickstart/accumulo/bootstrap-geowave.sh]
- For HBase use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/hbase/bootstrap-geowave.sh[s3://geowave/latest/scripts/emr/quickstart/hbase/bootstrap-geowave.sh]
====

To use one of the scripts listed above, it must be accessible from an s3 bucket. Because buckets are region specific, you may not be able to use our geowave bucket if your cluster is not deployed in the us-east-1 region. In this case, you will need a personal bucket in your region that contains the desired script. Instructions on <<110-appendices.adoc#create-aws-s3-bucket, creating>> and <<110-appendices.adoc#upload-to-aws-s3-bucket, uploading>> to an s3 bucket can be found in the appendices.

[[quickstart-guide-intro]]
<<<

:linkattrs:

=== Bootstrap Scripts

[[quickstart-guide-intro]]
GeoWave currently supports the use of either Accumulo or HBase, so the version of the bootstrap script you
use when setting up your cluster will be dependent upon which system you want to use as your datastore. The bootstrap scripts help to set up geowave and your environment. Click on any of the links below to download the script.

- For Accumulo use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/accumulo/bootstrap-geowave.sh[http://s3.amazonaws.com/geowave/latest/scripts/emr/accumulo/bootstrap-geowave.sh, window="_blank"]
- For HBase use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/hbase/bootstrap-geowave.sh[http://s3.amazonaws.com/geowave/latest/scripts/emr/hbase/bootstrap-geowave.sh, window="_blank"]

These scripts will provide you with everything necessary to complete this guide, but will also set up Geowave if you would like to use it in other workflows. There are scripts available for each of the releases of Geowave going back to 0.9.3. These can be used by replacing ``/latest/`` with the desired release (i.e. ``/0.9.3/``)

[NOTE]
====
We have also provided quickstart scripts that will perform all of the steps in this guide automatically. This will allow you to verify your own steps, or test out other geowave commands and features on an already conditioned data set.

If you would prefer to have all of the steps run automatically, please use these bootstrap scripts instead of the
ones listed previously:

- For Accumulo use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/accumulo/bootstrap-geowave.sh[http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/accumulo/bootstrap-geowave.sh, window="_blank"]
- For HBase use: link:http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/hbase/bootstrap-geowave.sh[http://s3.amazonaws.com/geowave/latest/scripts/emr/quickstart/hbase/bootstrap-geowave.sh, window="_blank"]
====

To use one of the scripts listed above, it must be accessible from an s3 bucket. Because buckets are region specific, you may not be able to use our geowave bucket if your cluster is not deployed in the us-east-1 region. In this case, you will need a personal bucket in your region that contains the desired script. Instructions on <<110-appendices.adoc#create-aws-s3-bucket, creating>> and <<110-appendices.adoc#upload-to-aws-s3-bucket, uploading>> to an s3 bucket can be found in the appendices.


update WORKLOG

update WORKLOG

= WORKLOG

* use Groovy language specification as case study to find corner cases
  - perhaps progit2 as well; others?

== TODO

* backport resize method from prawn-svg and use it in converter (now in two places)
* document fit attribute for images in running content in theming guide (or README)
* rework resolve_image_path API so it's more logical; override based on type of first argument; document as option
* rename inherited_align to text_align? (and base_align to base_text_align?)
* rename text-alignment attribute to text-align? (change is within alpha.14, so still a chance to change)

* implement index of index terms (#386)
  - introduce model object
* don't crash if image in running content is an unsupported format
* support image URL (using resolve_image_path) in running content (what about data-uri?)
* numbering for appendix subsections is not correct; should be A.1, A.2 (#627)
  - seems like an issue in core too
* BUG: implicit width of inline image (SVG only?) calculated incorrectly (see sandbox/issue-494)
* BUG: http://asciidoctor.org[Asciidoctor] surrounded by smart quotes doesn't get translated to a link (as it should)
* FILE ISSUE draw border for quote/verse block on right if text is aligned to the right
* Prawn should not move cursor before placing image if image exceeds height of bounding box
  - workaround now by overriding move_text_position
* part title / number (#597)
  - upper roman numeral
  - add part-label, fallback to Part (e.g., Part I)
  - only use roman numeral in toc
* support equal column widths in header/footer as "columns: 3*" (with optional leading alignment)
* support padding for each column in running header/footer
* should we move files under asciidoctor/pdf and use asciidoctor-pdf as the alias?
* consider moving RomanNumeral into a gem named roman_numeral
* should vertical alignment of admonition icon/label should respect padding on content? have it's own padding?
* allow general settings for admonition icon to be set using admonition_icon key prefix (e.g., admonition_icon_size)
* allow alignment to be set on discrete heading using role
* SIMPLE: submit pastie theme for Rouge upstream
* SIMPLE: document autofit option on verbatim blocks (in README?)
* SIMPLE: fill in missing defaults in theming guide
  - title_page
  - toc
  - table_header_cell
  - admonition_icon_<name>_stroke_color
  - description_list_term_font_style
  - header/footer
* SIMPLE: mark required keys (assume keys are optional by default)
  - required keys can never have a null value; most are set by base theme
* SIMPLE: document vw/vh measurement units in theming guide (make it clear it only applies in certain cases)
* SIMPLE: in theming guide, recommend not using the fallback font when working with very large documents
  - explain that default theme uses a fallback font
* allow font properties to be set for normal paragraph separate from base (need to think about inheritence)
* allow alignment of list to be set separately from base align (and perhaps a hint in document) (#182)
* FIXME: stop using fallback fonts in default theme (instead, bundle a fuller font)
  - using fallback fonts significantly slows down Prawn because it checks every letter every time (see https://github.com/prawnpdf/prawn/blob/master/lib/prawn/text/formatted/box.rb#L427-L434)
  - add broader character range to monospace font so we can drop fallback font by default (#282)
  - consider having a fallback for prose and fallback for literal
* SIMPLE: recommend use of prawn-gmagick in README
  - https://github.com/packetmonkey/prawn-gmagick
  - must install GraphicsMagick-devel on Fedora
  - or use ImageMagick (or GraphicsMagick) to uncompress PNG images before reading them
* use  instead of  for anchor point in formatted text
  - benchmark to see if it's faster to use empty or non-empty element in parser
* text decoration should be supported as part of theme_font
* QUESTION: should we set pdf-anchor attribute on every node that has an id?
  - isn't it required for cross references to work?
* QUESTION should preface subsection be numbered? (although it is numbered in DocBook and dblatex)
* new design for keep together; necessary to get exact height accounting for gaps at page breaks
  . in dry run, set to stop when advancing to next page (override on_page_create to throw exception)
  . if less than one page, return calculation (similar to what we do now)
  . if greater than one page, clear on_page_create; move to y offset of original and start dry run again; fix calculation
  . (if not keeping together, we can skip 1 and 2)
* rename "convert_content_for_" since it can collide with existing blocks
* don't orphan block title (make sure anchor stays with start of block)
* QUESTION should we report full image path of gif in warning message when prawn-gmagick is not available?
* QUESTION should we add destination to top of imported PDF page?
  - import page should accept id as section, optional argument
* leading (line height) isn't applied when content is split across pages
* generate fonts without PS Glyph Names to reduce file size
  - create script that can generate fonts entirely from original font source
* if start_new_page is called at end of layout_chapter_title, and media=prepress, ghostscript reports an error
  - problem is no color space is set; can fix by calling update_colors before advancing to recto page in start_new_chapter
  - maybe introduce a skip_page helper to combine these operations?
  - upstream issues: https://github.com/prawnpdf/prawn/issues/951 and https://github.com/prawnpdf/prawn/issues/473
* document how to test / use a PR
  - see https://github.com/asciidoctor/asciidoctor-pdf/issues/462#issuecomment-246200953
  - clearer instructions for how to test local development version (using rake install)
* break printed URIs at each forward slash (and perhaps at ? and & too) (#563)
* allow font size of dot_leader to be specified (some risk if it exceeds size of entries)
* add empty? method to Page (instead of page_empty? on document)
* FILE ISSUE: add option to svg method to not move cursor (in prawn-svg)
* FILE ISSUE: in prawn: go_to_page should accept second argument that is cursor position
* UNRESOLVED: dry_run should start at cursor of main document (or as option); total height calculation would need to be revised
  - box_height isn't currently accurate when it spans more than one page
  - this should fix height calculation when content is split over a page break (leaving small amount of excess)
  - however, if cursor is advanced to fit content on page, then that excess will cause box_height to be too large
  - life would be simpler if Prawn allowed us to draw graphics at bottom layer
* support pdfwidth for inline image (#620)
* allow height of inline image to be set to line height (perhaps 1em?)
* space around inline anchors/index entries doesn't get collapsed by text formatter
* add support for format attribute on image macro to image-related attributes such as title-page-background-image
  - support explicit image format for cover page image and page background image
* introduce abstract-title attribute to complement preface-title?
* need some sort of post_construct method for converter that receives document
  - inline convert methods can get called before init_pdf
  - monkeypatch?
* document nonfacing option more clearly (in README or theming guide)
* create document that explains how built-in fonts are generated and what subsets are selected
  - I need instructions for myself so I know how to update/modify the fonts
  - document in theming guide what must be done to prepare fonts (old-style 'kern' table, optionally subset) (file issue!)
  - add note to README that Prawn will subset any fonts provided
* consider supporting icon tag in parser to simplify how inline icons are stored; simpler use of passthrough content
* cache stateless cell data resolved from theme (don't need to recalc each time; at least per table)
* FILE ISSUE: autowidth on table doesn't work for multi-line content (prawn-table bases width calculation on normalized value)
  - table ends up being stretched even though it doesn't need to be
* table logic: does the layout_table_caption have to be inside the table block? can we pre-calculate the actual width for the caption? does the table offer a callback we can use to keep the caption on the same page as the table?
* introduce object to store/organize running content data and specs
* QUESTION: should theme font handle hierarchical keys (either explicitly or implicitly)
* need to support .canvas role on image so it isn't shrunk to fit inside top/bottom margins
  - perhaps .canvas, .canvas-x, .canvas-y
  - allow image to span width of page (role=canvas, role=canvas-x or role=canvas-y); if role is canvas or canvas-y, then it does not consume height
  - partially addressed by vw units
* FILE ISSUE: when split source listing, add top padding to bounding box (or is it the line metrics top?)
  - perhaps this has to do with the a miscalculation in dry run when not starting from same y position?
  - separate theme control for listing vs literal block (and maybe source too)
* should we print alt text if an error is caught when attempting to embed an image?
* stroke and fill multi-page sidebar block (#259) and example block (#362)
* support URL images in running content (need to delegate to resolve_image_path)
* rewrite optimize-pdf using rghost (#535) (also see #21 and #22)
  - add Optimizer class; wire to cli (separate issue?)
* add feature to number bullets according to section number (needed for OpenDevise agreements)
* outline should link to title page if there's a cover page (skip cover page and ensuing blank page)
* might be better to organize fragments of source chunks by lines (and pass that around) to simplify post-processing
* support negative start value for list (#498)
  - need to count negative numbers in correct direction
* support zero-leading integers (use dedicated type like w/ roman numerals) in reversed order lists
* margins/paddings at page boundaries are strange, fragile
* implement margin collapsing (between blocks)
  - would eliminate need for negative padding for blockquote
* bw theme for CodeRay (or Rouge) to match output of Pygments bw
* the nested? method on list isn't checking if nested inside a block inside a list
  - need an example
* wrapped lines in source listing should be indented to account for line number gutter (#504)
* add sample SVG to content of chronicles-example.adoc (we do already use one for title page)
* inline images: allow built-in font family names for SVG to be remapped
* inline images: should we be passing absolute image path in tag or something relative (or even a lookup id?)?
* large image runs into running footer (doesn't bottom margin need to be aligned with running footer height?)
* should str_to_pt helper handle % and vw units?
* allow format of printed link to be controlled by theme (similar to what we do in the manpage converter)
* FILE ISSUE: should not wrap at formatting mark unless it's at a break opportunity
  - the problem here is that Prawn is allowing breaks at the boundaries of text fragments; it should only look at the contents
* allow top as alternative to margin_top for all elements on title page (#431)
* swallowing exceptions! (any use of e.message in a string is dangerous)
* allow ordered list marker to be prefixed by section number (a global setting?)
* conum not aligned vertically with callout text (perhaps too small?)
* conum should never wrap (push it into the text if necessary)
* decouple theme settings for section titles and discrete headings
* decouple listing/literal/source theme settings; currently all under code
* replace explicit char ranges with classes in regexp (e.g., [[:word:]] or \w)
* devise a way to specify a value as a string literal (variable replacement only) in theme
* apply calculated theme values after loading?
* allow "content" in place of recto_content & verso_content for running header/footer
  - still relevant after restructuring?
* be more specific in theming guide as to where prose_margin_top and prose_margin_bottom apply
* allow valign value to be a number (requires change to Prawn)
* layout SVG without using keep_together (since we know all the dimensions)
  - fix SVG to a single page (check height + caption height)
* allow background color to be set for chapter / heading
* allow border to be set around block image
* file issue in prawn to dispatch to image handler for images it doesn't know about
* add brief mention in theming guide that deeper customizations can be achieved by extending the converter
  - see sandbox/asciidoctor_pdf_extensions.rb
  - reference infoq-minibook repo & blog post
  - document how to extend the converter, use Prawn
  - document how to override the Ruby code to get custom styling in the theming guide
* is https://github.com/packetmonkey/prawn-pdfimage a safer way than prawn-templates to import PDF as image?
* rename ThemeLoader to ThemeReader (or ThemeManager)?
* normalize step leaves space after endline (i.e., `\n `) at a hard line break (doesn't seem to affect flow)
* *margin per heading level* (#176)
* need a single object to hold complete font properties; different from font family/style object
* font method should support a single argument that's a font object or font hash
  - in general, the way font properties are set needs to be cleaned up
* allow font size in theme to be specified in em or %
  - should multiply value being inherited
* can't put margin top on chapter (chapter_top?) (#576)
* convenience method to check if there's enough room for another line on page
* allow dynamic background image with page number in path
* running header/footer covers content (perhaps just a limitation that needs to be documented)
* document that palette-based transparent in PNGs is not supported in older version of Prawn
* FILE ISSUE: for prawn to preserve space (even w/ guards, spaces don't preserve over wrapped lines)
  - if this is fixed, we can remove all the guard indent code
* FILE ISSUE: for prawn to support spacer fragments with fixed width / height and no text (or text is ignored in calculations)
  - needed for arranging inline objects
* document limitations in README (such as no linear gradients in SVG, etc)
* document all permutations of image sizing
* set vposition on title page logo image explicitly to avoid page overrun?
* verse has problems with wrapping if line is long (in what way?)
* allow default kerning to be set using theme
* keep line comment in front of callout number to aid copying?
* rework pull request for source line numbers (combine with restore conum logic if conums are enabled)
  - also combine with the preserve_space logic
* allow front cover and back cover image to be defined in theme as fallback; document in theming-guide
* should we shorten the keys to front-cover and back-cover (since image is implied?)
* keep caption with table (check for sufficient space); only for top placement since bottom placement is much harder
* allow valign to be set on image block (vertical center in page for things like slides)
* allow title page image "bottom" to be set instead of "top" (mutually exclusive)
* rtl (see ./sandbox/rtl/ folder)
* pass macro doesn't work in source block when macro subs and highlighting are both enabled (#180)
* enable cache_images option for prawn-svg (#223)
* bind image_registry between scratch and main document so we don't process the same image more than once
  - need to do some testing
* show SVG warnings if debug (or trace) is on
* clean temporary files once per conversion instead of per node? (file issue)
* title is being rendered 3 times (maybe one for scratch?); explain why in comments if normal
  - block title?
* continue working on json schema for theme; try to generate keys section from it
* rethink how we're handling line heights for fonts, then document carefully
  - look closer at line_height and line_height_length and see if we need to document other details
  - allow line height to be set in more places (such as the prose for admonition, example, sidebar, etc)
* implement first-line indent for paragraphs (seems like conflict w/ our text formatter)
  - option to not indent first paragraph in section
  - add indent/noindent options
  - if you indent, perhaps drop the margin between paragraphs?
* add entry to TOC for preamble/preface
* can we create fragments directly in converter instead of using the formatted text parser?
  - would need to override how blocks join content; perhaps even how apply_subs works
* don't issue warnings on scratch document
  - perhaps introduce a helper method to abstract this away
* getting a line wrap break before comma if preceding word is emphasized (problem in Prawn wrapping)
  - no longer a problem? perhaps was due to #462; could also be when it does wrap by char
* toc
  - make dot leader style separate from title / number
* running content
  - side margins (allow override, default to content margins)
  - numbered and unnumbered chapter and section titles (file issue)
  - chapter and section number (easily solved by previous)
  - separate running content for chapter page (by default uses normal content)
* should we rename base_ to body_ to make it more familiar to CSS developers?
* support !include in theme file (#571)
* add cover page example to chronicles so people see how to use it
  - need to find a good cover page
* don't orphan a single line of paragraph (send it with a buddy line)
  - implement orphan sentences for paragraph
* implement stem support
  - see asciidoctor-mathematical and asciidoctor-mathoid
* fail gracefully if theme file cannot be found
  - report it can't be found (should we fallback to default theme?)
* expose theme variable on document (using attr_reader?)
* dedicated style for top/bottom margin of outline list
  - allow margin top and bottom to be set for lists (applies to outer-most list)
  - allow spacing between nested lists levels be configured in theme
* need dedicated theme styles for paragraph spacings, etc
* subtitles for parts and chapters (#623)
* part titles need their own styling
* add color calculation functions in theme file (like in SASS)
* create utility method to get % offset of page as y value (option to constrain to bounds)
* document why we have converter assignment in convert_content_for_block method
  - do we still need the converter hack in convert_content_for_block? (seems to be needed for admonitions)
* support transparency for colors (this is now supported by resolve_theme_color)
  - utility to coerce the color value transparent to nil (better handling in general)
* support generic color (or value) attribute in formatted text parser instead of specific color systems (rgb, cmyk)
* **allow theme_font to set line_height** (honor this setting from document)
  - theme setting for code line height (currently using base_line_height)
* should we put an entry for doctitle in the outline if notitle is set? (need to test these edge cases)
* add more theme control over toc (per-level font size, style, color, etc)
* strip formatted text (e.g., monospace) from headings and toc entries
* prevent title-logo-image from spilling to next page (same with title content)
* document what each keep_together is doing / expects
  - keep_together really needs to pick up the inherited horizontal bounds or else measurement is inaccurate; fixed?
* code cleanups (regexps to constants, nil? checks and such)
  - split prawn_ext/extensions into individual files based on function
* enable line above (or below?) title on title page (file issue)
  - perhaps 4-sided border?
* enable text transform for:
  - running content
  - admonition content
  - table foot row
  - table header cell
  - table (overall)
  - generic paragraph?
  - inline literal text
  - inline link
  - inline literal/monospaced
  - listing/literal/source block
  - global default
* file upstream issue for Prawn to warn if it can't resolve a glpyh (or monkeypatch it)
* support web fonts; use uri-cache to avoid redundant fetching
* align caption to match alignment of block image
* FILE ISSUE: attribute or role to control table shading (all, even, odd) (or call it striped like bootstrap?)
  - e.g., [.striped]
  - currently controlled by theme
* make conum glyphs configurable in theme (use reference table to resolve)
* CJK and/or multilingual support (see https://github.com/chloerei/asciidoctor-pdf-cjk)
* description list term should keep together with content (file issue)
* allow font properties to be set for lists (description_list, outline_list)
* hardbreak in table cell results in extra endline (likely not normalizing cell content)
* remove pdfmarks file after optimizing
* look into single_line + shrink_to_fit in listings, perhaps other places
* refactor as Prawn view to avoid method name conflicts (also see https://github.com/prawnpdf/prawn/issues/802)
* create proper default (Asciidoctor) theme (#60)
* document how the treetop parser is rebuilt
* rework font so we can set actual height, calculate x_height internally (use 1em for spacings)
* padding top and bottom on content affects height_of calculations (need to review)
* code font needs to support more than just ascii (Golo license block is an example)
* don't cutoff content in partintro
* admonition styles are one big hack; need to be organized and based on theme
* BUG: autofit logic not working with Courier (still overrunning line)
* honor safe mode rules
* print scratch.pdf file if verbose / trace mode is on in Asciidoctor
* introduce setting to indent section content
* rename default theme to docbook theme, make default the Asciidoctor theme (should we have a base theme?)
* allow relative font size for inline code to be set (perhaps a percentage or em value? there are problems with this in arranger)
* apply line height metrics for table content
  - figure out how to adjust line height for monospaced cell content
  - figure out how to layout regular cell content to adjust for line height
* document the typeset_text methods very clearly
* move check for node.title? inside layout_caption
* theme idea / tester: see sandbox/ebook-learn_version_control_with_git-SAMPLE.pdf
* make alternating page title position optional (via theme?)
* fix passthrough placeholders that get caught up in syntax highlighting (see https://github.com/asciidoctor/asciidoctor/blob/master/test/blocks_test.rb#L2258-L2277)
* FILE ISSUE list Preamble in TOC
  - perhaps only if it has a title? or should we use a default title if one isn't specified?
* honor font defs in SVG (to get M+ 1p); prawn-svg supports loading fonts; need to pass fonts to prawn-svg
* should we support % as a unit in theme (divides by 100 and sets float value)?
* disable monospace font color (and family?) in headings
* add source language to upper-left corner of listing block
* implement quote style from default Asciidoctor stylesheet
* reorganize Prawn extensions (see prawn-table for example)
* rename "theme" to "style"?
* restrict custom theme path to jail (or load from load_path)
* enforce jail on SVG option enable_file_requests_with_root
* implement convert_toc
* italic text in a line of text styled as bold in the theme loses its bold style
* introduce method for start_initial_page?
* make outline a document option (perhaps "outline" like "toc")
* add bench/ directory for the script to test the speed of the formatted text parser
* start page numbering on page 1 (use /PageLabels reference to make i the title page number)
  - add this feature upstream to Prawn
* *report image only page w/ stamps corruption issue to Prawn*
  - still true?
* add /PageMode /UseOutlines
* cli arguments
  - theme (pdf-style, pdf-stylesdir)
  - enable/disable writing pdfmark file
  - optimize-pdf
* implement footnotes correctly (#73, #85)
* flesh out outline more (in what way?)
* flesh out title page more
  - document subtitle (partially solved)
* don't create title page for article doctype (#95, #105)
  - only create title page if doctype=book
* allow character spacing to be controlled by theme
* might be able to avoid dry run for listing/literal in obvious cases; engineering estimate
* allow pdf-page-margin to be set in document
  - this is slighly more complicated now that we have mirror margins; perhaps can't set those from document?
* use `module Asciidoctor; module PDF; module FormattedText` convention to simplify indentation
* introduce code style guide (like in Jekyll AsciiDoc); perhaps make this a shared file in the Asciidoctor ecosystem?
* I'd like for theme to be able to set font scan path for Prawn SVG, but registry is global
* rename dot_leader to just leader or tab_leader?
* rename align to text_align?
* QUESTION should bullets be on right if list alignment is right (what about center?)
* QUESTION should we resolve font-based icons globally, in init_pdf?

* use treetop to parse and evaluate theme file
* use or don't use pad method? check performance
* switch wolpertinger to howling grasshopper mouse

== Major Efforts / Milestones

* add a test suite
* refactor as Prawn View
* add support for AsciiDoc table cell content by writing custom Table::Cell implementation
* add support for nested tables by writing custom Table::Cell implementation
* add support for footnotes (as article or chapter endnotes)
* rework text handling in Prawn to support line height natively
* margin collapsing (like CSS)

== Documentation

* "Incorrect number of arguments in 'SCN' command" happens when you add a stamp to an imported page
* be mindful that layout_prose adds margin to bottom of content by default (important when working in a bounding box)
* ttfunk does not support ligatures (e.g., fi -> ﬁ); we could do this manually in post_replacements

== Notes

* when using `single_line: true` on formatted_text, it's necessary to reapply our padding top/bottom from line metrics
* we always leave cursor on start of page we're about to write on; certain checks rely on this fact
* "section title" is the semantic element; "heading" is the structural element
* /PageLabels/Nums must have entry for every page in front matter, even if a blank page
  - in fact, must account for every page or else numbering lags behind when scrolling document
* if we set the vposition on image to a numeric value, it skips the overrun check that happens internally
* any instance variables referenced by converter methods for inline nodes could get accessed before the converter for document is called
* Evince throws warning when printing PDF if & is used in document title; but this is valid according to the PDF specification
* Prawn drops fragments with empty text (hence the need to use zero-width space)
  - analyze_glyphs_for_fallback_font_support drops fragments with empty text
  - later on, initialize_wrap drops fragment with empty text
* use term "page number label" to refer to the visible, printed page number (not the implicit page number)
* vertical alignment of text doesn't work properly in Prawn; better to calculate alignment manually, if possible

== Known Issues

* inline image at start of the line is slighly shifted to the right due to the fact that it's placed in the center of the reserved fragment width; perhaps we are adding this padding

== Potential Optimizations

* if autofit is set on a listing/literal block that has conums, we are splitting fragments by line twice
* comparing > 0 is slightly faster than == 0 (for cases when we can swap the logic)

== Usage Optimizations

* uncompress PNG files to avoid slow zlib inflating step in Prawn
* flatten PNGs (remove alpha channel) since it messes up font rendering on the page in Adobe Acrobat Reader (need to verify)
* avoid the fallback font if possible (use full fonts in your theme) because it checks for *every* glyph
* font families used in SVGs must match keys in the font catalog

== Open Questions

== Implementation

* should we read SVG file using UTF-8 encoding; or does REXML handle encoding?
* can we leverage before_rendering_page callback on table?
* should we use move_past_bottom in some places instead of start_new_page?

=== Design

* remove/reduce padding above heading when it appears at the start of a page?
* Default line height?
* Should the heading sizes be calculated according to the default font size?
* Page margins
* Body indentation?
  - recto / verso indentation?
* Size of masthead / footer
* Line separating masthead / footer?
* Separate title page
* Start chapter on new page?
* Special layout for chapter page?

=== Theme

* keep or drop base_ prefix in theme? I think we should keep it because it provides context elsewhere in the document (e.g. $base_font_size vs $font_size)

== Resources

* https://code.google.com/p/origami-pdf/[Origami PDF: A PDF inspection library]
* https://github.com/a1ee9b/PrintPretty[A theme for PDF designed for printing]
* http://randomtextgenerator.com[Random Text Generator, supports multiple languages]
* http://clagnut.com/blog/2380[List of pangrams]
  - http://www.camcc.org/_media/reading-group/qianziwen-en.pdf[1,000 character classic (Chinese)]
* pdf2svg can convert the PDF file into an SVG (one SVG per page)
* https://blog.codeship.com/build-math-evaluation-engine[How to Build a Simple Math Evaluation Engine]
* http://blog.typekit.com/2011/11/03/optimizing-fonts-for-the-web-unicode-values-glyph-set-underlines-and-strike-through/[Optimizing Fonts for the Web]

= WORKLOG

* use Groovy language specification as case study to find corner cases
  - perhaps progit2 as well; others?

== TODO

* remove support for link_color option in layout_prose as this is now handled by text formatter
* BUG: implicit width of inline image (SVG only?) calculated incorrectly (see sandbox/issue-494)
* FILE ISSUE: SIMPLE: use docdate attribute instead of Time.now to set modification date on document
* SIMPLE: fill in missing defaults in theming guide
  - title_page
  - toc
  - table_header_cell
  - admonition_icon_<name>_stroke_color
  - description_list_term_font_style
  - header/footer
* SIMPLE: mark required keys (assume keys are optional by default)
  - required keys can never have a null value; most are set by base theme
* add URL to chronicles example that contains a character reference, such as pass:[&]
* allow links to be underlined (theme setting) (#567)
* SIMPLE: document vw/vh measurement units in theming guide (make it clear it only applies in certain cases)
* SIMPLE: in theming guide, emphasize not using the fallback font when working with very large documents
  - explain that default theme uses a fallback font
* FIXME: stop using fallback fonts in default theme (instead, bundle a fuller font)
  - using fallback fonts significantly slows down Prawn because it checks every letter every time (see https://github.com/prawnpdf/prawn/blob/master/lib/prawn/text/formatted/box.rb#L427-L434)
* add broader character range to monospace font so we can drop fallback font by default (#282)
  - consider having a fallback for prose and fallback for literal
* SIMPLE: recommend use of prawn-gmagick in README
  - https://github.com/packetmonkey/prawn-gmagick
  - must install GraphicsMagick-devel on Fedora
* use ImageMagick (or GraphicsMagick) to uncompress PNG images before reading them (could also just document this)
  - mention where we talk about prawn-gmagick in README
* use  instead of  for anchor point in formatted text
  - benchmark to see if it's faster to use empty or non-empty element in parser
* QUESTION: should we set pdf-anchor attribute on every node that has an id?
  - isn't it required for cross references to work?
* new design for keep together; necessary to get exact height accounting for gaps at page breaks
  . in dry run, set to stop when advancing to next page (override on_page_create to throw exception)
  . if less than one page, return calculation (similar to what we do now)
  . if greater than one page, clear on_page_create; move to y offset of original and start dry run again; fix calculation
  . (if not keeping together, we can skip 1 and 2)
* rename "convert_content_for_" since it can collide with existing blocks
* rework resolve_image_path API so it's more logical; override based on type of first argument; document as option
* don't orphan block title (make sure anchor stays with start of block)
* QUESTION should we report full image path of gif in warning message when prawn-gmagick is not available?
* QUESTION should we add destination to top of imported PDF page?
  - import page should accept id as section, optional argument
* leading (line height) isn't applied when content is split across pages
* generate fonts without PS Glyph Names to reduce file size
  - create script that can generate fonts entirely from original font source
* if start_new_page is called at end of layout_chapter_title, and media=prepress, ghostscript reports an error
  - problem is no color space is set; can fix by calling update_colors before advancing to recto page in start_new_chapter
  - maybe introduce a skip_page helper to combine these operations?
  - upstream issues: https://github.com/prawnpdf/prawn/issues/951 and https://github.com/prawnpdf/prawn/issues/473
* FILE ISSUE: cells in table head are wrapping by char, but don't set width of column
  - most prominent in auto-width tables
* document how to test / use a PR
  - see https://github.com/asciidoctor/asciidoctor-pdf/issues/462#issuecomment-246200953
* break printed URIs at each forward slash (and perhaps at ? and & too) (#563)
* add empty? method to Page (instead of page_empty? on document)
* SIMPLE: add warning that AsciiDoc table cell content is not supported; print as raw source
* FILE ISSUE: add option to svg method to not move cursor (in prawn-svg)
* FILE ISSUE: prevent import_page from adding an extra page; leaves blank page when placing two inserts in a row
  - have import_page first delete current page if page is empty
* FILE ISSUE: in prawn: go_to_page should accept second argument that is cursor position
* UNRESOLVED: dry_run should start at cursor of main document (or as option); total height calculation would need to be revised
  - box_height isn't currently accurate when it spans more than one page
  - this should fix height calculation when content is split over a page break (leaving small amount of excess)
  - however, if cursor is advanced to fit content on page, then that excess will cause box_height to be too large
  - life would be simpler if Prawn allowed us to draw graphics at bottom layer
* allow height of inline image to be set to line height (perhaps 1em?)
* support pdfwidth for inline image
* space around inline anchors/index entries doesn't get collapsed by text formatter
* add support for format attribute on image macro to image-related attributes such as title-page-background-image
  - support explicit image format for cover page image and page background image
* introduce abstract-title attribute to complement preface-title?
* need some sort of post_construct method for converter that receives document
  - inline convert methods can get called before init_pdf
  - monkeypatch?
* backport resize method from prawn-svg and use it in converter
* create document that explains how built-in fonts are generated and what subsets are selected
  - I need instructions for myself so I know how to update/modify the fonts
  - document in theming guide what must be done to prepare fonts (old-style 'kern' table, optionally subset) (file issue!)
* consider supporting icon tag in parser to simplify how inline icons are stored; simpler use of passthrough content
* cache stateless cell data resolved from theme (don't need to recalc each time; at least per table)
* autowidth on table doesn't work for multi-line content (prawn-table bases width calculation on normalized value)
  - table ends up being stretched even though it doesn't need to be
* table logic: does the layout_table_caption have to be inside the table block? can we pre-calculate the actual width for the caption? does the table offer a callback we can use to keep the caption on the same page as the table?
* introduce object to store/organize running content data and specs
* QUESTION: should theme font handle hierarchical keys (either explicitly or implicitly)
* need to support .canvas role on image so it isn't shrunk to fit inside top/bottom margins
  - perhaps .canvas, .canvas-x, .canvas-y
* FILE ISSUE: when split source listing, add top padding to bounding box (or is it the line metrics top?)
  - perhaps this has to do with the a miscalculation in dry run when not starting from same y position?
* should we print alt text if an error is caught when attempting to embed an image?
* stroke and fill multi-page sidebar block (#259) and example block (#362)
* support URL images in running content (need to delegate to resolve_image_path)
* rewrite optimize-pdf using rghost (#535) (also see #21 and #22)
  - add Optimizer class; wire to cli (separate issue?)
* add feature to number bullets according to section number (needed for OpenDevise agreements)
* outline should link to title page if there's a cover page (skip cover page and ensuing blank page)
* might be better to organize fragments of source chunks by lines (and pass that around) to simplify post-processing
* support negative start value for list (#498)
  - need to count negative numbers in correct direction
* margins/paddings at page boundaries are strange, fragile
* implement margin collapsing (between blocks)
  - would eliminate need for negative padding for blockquote
* SIMPLE: submit pastie theme for Rouge upstream
* bw theme for CodeRay (or Rouge) to match output of Pygments bw
* the nested? method on list isn't checking if nested inside a block inside a list
* wrapped lines in source listing should be indented to account for line number gutter (#504)
* add sample SVG to content of chronicles-example.adoc (we do already use one for title page)
* inline images: allow built-in font family names for SVG to be remapped
* inline images: should we be passing absolute image path in tag or something relative (or even a lookup id?)?
* support zero-leading integers (use dedicated type like w/ roman numerals) in reversed order lists
* large image runs into running footer (doesn't bottom margin need to be aligned with running footer height?)
* should str_to_pt helper handle % and vw units?
* allow style of visible link to be controlled by theme (similar to what we do in the manpage converter)
* FILE ISSUE: should not wrap at formatting mark unless it is at a break opportunity
  - the problem here is that Prawn is allowing breaks at the boundaries of text fragments; it should only look at the contents
* be more consistent with how examples are shown in theming-guide.adoc (do we show last segment only?)
* allow top as alternative to margin_top for all elements on title page (#431)
* swallowing exceptions! (any use of e.message in a string is dangerous)
* allow ordered list marker to be prefixed by section number (a global setting?)
* conum not aligned vertically with callout text (perhaps too small?)
* conum should never wrap (push it into the text if necessary)
* decouple theme settings for section titles and discrete headings
* decouple listing/literal/source theme settings; currently all under code
* replace explicit char ranges with classes in regexp (e.g., [[:word:]] or \w)
* devise a way to specify a value as a string literal (variable replacement only) in theme
* apply calculated theme values after loading?
* allow "content" in place of recto_content & verso_content for running header/footer
  - still relevant after restructuring?
* be more specific in theming guide as to where prose_margin_top and prose_margin_bottom apply
* allow image to span width of page (role=canvas, role=canvas-x or role=canvas-y); if role is canvas or canvas-y, then it does not consume height
  - partially addressed by vw units
* allow valign value to be a number (requires change to Prawn)
* layout SVG without using keep_together (since we know all the dimensions)
  - fix SVG to a single page (check height + caption height)
* allow background color to be set for chapter / heading
* allow border to be set around block image
* switch wolpertinger to howling grasshopper mouse
* file issue in prawn to dispatch to image handler for images it doesn't know about
* add brief mention in theming guide that deeper customizations can be achieved by extending the converter
  - reference infoq-minibook repo & blog post
  - document how to extend the converter, use Prawn
* allow alignment of list to be set separately from base align (and perhaps a hint in document)
* is https://github.com/packetmonkey/prawn-pdfimage a safer way than prawn-templates to import PDF as image?
* rename ThemeLoader to ThemeReader (or ThemeManager)?
* SIMPLE: document autofit option on verbatim blocks (in README?)
* normalize step leaves space after endline (i.e., `\n `) at a hard line break (doesn't seem to affect flow)

* *margin per heading level* (#176)
* can't put margin top on chapter (chapter_top?) (#576)
* document how to override the Ruby code to get custom styling in the theming guide
* convenience method to check if there's enough room for another line on page
* allow font size in theme to be specified in em or %
* allow dynamic background image with page number in path
* running header/footer covers content (perhaps just a limitation that needs to be documented)
* document that palette-based transparent in PNGs is not supported
* file issue for prawn to preserve space (even w/ guards, spaces don't preserve over wrapped lines)
  - if this is fixed, we can remove all the guard indent code
* file issue for prawn to support spacer fragments with fixed width / height and no text (or text is ignored in calculations)
* file issue that prawn-svg messes with the cursor (need to explain how)
* document limitations in README (such as no linear gradients in SVG, embedded images in SVG must be URL or inlined, PNGs must be flattened, etc)
* document all permutations of image sizing
* set vposition on title page logo image explicitly to avoid page overrun?

* allow pdf-page-margin to be set in document
  - this is slighly more complicated now that we have mirror margins; perhaps can't set those from document?
* verse has problems with wrapping if line is long (in what way?)
* allow default kerning to be set using theme
* keep line comment in front of callout number to aid copying?
* rework pull request for source line numbers (combine with restore conum logic if conums are enabled)
  - also combine with the preserve_space logic
* allow front cover and back cover image to be defined in theme as fallback; document in theming-guide
* should we shorten the keys to front-cover and back-cover (since image is implied?)
* keep caption with table (check for sufficient space); only for top placement since bottom placement is much harder
* allow valign to be set on image block (vertical center in page for things like slides)
* separate theme control for listing vs literal block (and maybe source too)
* allow title page image "bottom" to be set instead of "top" (mutually exclusive)
* rtl (see ./sandbox/rtl/ folder)

* pass macro doesn't work in source block when macro subs and highlighting are both enabled (#180)
* enable cache_images option for prawn-svg (#223)
* bind image_registry between scratch and main document so we don't process the same image more than once
  - need to do some testing
* show SVG warnings if debug (or trace) is on
* clean temporary files once per conversion? (file issue)
* clear font paths in SVG interface so it doesn't scan system? (since it's not portable anyway)
* title is being rendered 3 times (maybe one for scratch?); explain why in comments if normal
* finish docs/theming-guide.adoc
  - continue working on json schema for theme; try to generate keys section from it
* rethink how we're handling line heights for fonts, then document carefully
  - look closer at line_height and line_height_length and see if we need to document other details
* implement first-line indent for paragraphs (seems like conflict w/ our text formatter)
  - option to not indent first paragraph in section
  - if you indent, perhaps drop the margin between paragraphs?
* add index support
* add entry to TOC for preamble/preface
* can we create fragments in converter instead of using the formatted text parser?
* allow text alignment of prose to be set in document
* don't issue warnings on scratch document
* rake release seems messed up (tagging the wrong commit)
* getting a line wrap break before comma if preceding word is emphasized (problem in Prawn wrapping)
* toc
  - make dot leader style separate from title / number
* running content
  - side margins (allow override, default to content margins)
  - numbered and unnumbered chapter and section titles (file issue)
  - chapter and section number (easily solved by previous)
  - separate running content for chapter page (by default uses normal content)
* should we rename base_ to body_ to make it more familiar to CSS developers?
* support !include in theme file (#571)
* add cover page example to chronicles so people see how to use it
* don't orphan a single line of paragraph (send it with a buddy line)
  - implement orphan sentences for paragraph
* implement stem support
  - see asciidoctor-mathematical and asciidoctor-mathoid
* fail gracefully if theme file cannot be found
* expose theme variable on document (attr_reader?)
* dedicated style for top/bottom margin of outline list
  - allow margin top and bottom to be set for lists (applies to outer-most list)
  - allow spacing between nested lists levels be configured in theme
* need dedicated theme styles for paragraph spacings, etc
* recto/verso indentation (on body?)
* don't indent and draw line next to quote block unless width > 0 or color != transparent
* subtitles for chapters
* part titles need their own page and styling
* add color calculation functions in theme file (like in SASS)
* create utility method to get % offset of page as y value (option to constrain to bounds)
* document converter assignment in convert_content_for_block method
* support transparency for colors (this is now supported by resolve_theme_color)
* support generic color (or value) attribute in formatted text parser instead of specific color systems (rgb, cmyk)
* **allow theme_font to set line_height** (honor this setting from document)
  - theme setting for code line height (currently using base_line_height)
* should we put an entry for doctitle in the outline if notitle is set? (need to test these edge cases)
* add more theme control over toc (font size, style, color per level)
* don't allow formatted text (e.g., monospace) in headings or toc entries
* prevent title-logo-image from spilling to next page (same with title content)
* document what each keep_together is doing / expects
  - keep_together really needs to pick up the inherited horizontal bounds or else measurement is inaccurate
* code cleanups (regexps to constants, nil? checks and such)
  - split prawn_ext/extensions into individual files based on function
* enable line above (or below?) title on title page (file issue)
  - perhaps 4-sided border?
* enable text transform for:
  - running content
  - admonition content
  - table foot row
  - table header cell
  - table (overall)
  - generic paragraph?
  - inline literal text
  - link
  - listing/literal/source block
  - global default
* file upstream issue for Prawn to warn if it can't resolve a glpyh (or monkeypatch it)
* support web fonts; use uri-cache to avoid redundant fetching
* align caption to match alignment of block image
* FILE ISSUE: attribute or role to control table shading (all, even, odd) (or call it striped like bootstrap?)
  - e.g., [.striped]
  - currently controlled by theme
* make conum glyphs configurable in theme (use reference table to resolve)
* do we still need the converter hack in convert_content_for_block? (seems to be needed for admonitions)
* utility to coerce the color value transparent to nil (better handling in general)
* CJK and/or multilingual support (see https://github.com/chloerei/asciidoctor-pdf-cjk)
* description list term should keep together with content (file issue)
* allow font properties to be set for lists (description_list, outline_list)
* hardbreak in table cell results in extra endline (likely not normalizing cell content)
* remove pdfmarks file after optimizing
* add note to README that Prawn will subset any fonts provided
* look into single_line + shrink_to_fit in listings, perhaps other places
* refactor as Prawn view to avoid method name conflicts (also see https://github.com/prawnpdf/prawn/issues/802)
* create proper default (Asciidoctor) theme
* document how the treetop parser is rebuilt
* rework font so we can set actual height, calculate x_height internally (use 1em for spacings)
* padding top and bottom on content affects height_of calculations (need to review)
* code font needs to support more than just ascii (Golo license block is an example)
* don't cutoff content in partintro
* admonition styles are one big hack; need to be organized and based on theme
* FILE ISSUE: can't change font properties of admonition block content
* SIMPLE: add admonition_label_font_color to theme
* BUG: autofit logic not working with Courier (still overrunning line)
* honor safe mode rules
* print scratch.pdf file if verbose / trace mode is on in Asciidoctor
* introduce setting to indent section content
* rename default theme to docbook theme, make default the Asciidoctor theme (should we have a base theme?)
* allow relative font size for inline code to be set (perhaps a percentage or em value? there are problems with this in arranger)
* apply line height metrics for table content
  - figure out how to adjust line height for monospaced cell content
  - figure out how to layout regular cell content to adjust for line height
* document the typeset_text methods very clearly
* move check for node.title? inside layout_caption
* theme idea / tester: see sandbox/ebook-learn_version_control_with_git-SAMPLE.pdf
* make alternating page title position optional (via theme?)
* fix passthrough placeholders that get caught up in syntax highlighting (see https://github.com/asciidoctor/asciidoctor/blob/master/test/blocks_test.rb#L2258-L2277)
* FILE ISSUE list Preamble in TOC
  - perhaps only if it has a title? or should we use a default title if one isn't specified?
* honor font defs in SVG (to get M+ 1p); prawn-svg supports loading fonts; need to pass fonts to prawn-svg
* should we support % as a unit in theme (divides by 100 and sets float value)?
* disable monospace font color (and family?) in headings
* add source language to upper-left corner of listing block

* implement quote style from default Asciidoctor stylesheet
* reorganize Prawn extensions (see prawn-table for example)
* rename "theme" to "style"?
* restrict custom theme path to jail (or load from load_path)
* implement convert_toc
* italic text in a line of text styled as bold in the theme loses its bold style

* introduce method for start_initial_page?
* make outline a document option (perhaps "outline" like "toc")
* add bench/ directory for the script to test the speed of the formatted text parser
* start page numbering on page 1 (use /PageLabels reference to make i the title page number)
  - add this feature upstream to Prawn
* *report image only page w/ stamps corruption issue to Prawn*
  - still true?
* add /PageMode /UseOutlines
* cli arguments
  - theme (pdf-style, pdf-stylesdir)
  - enable/disable writing pdfmarks file
  - optimize-pdf
* implement footnotes correctly (#73, #85)
* flesh out outline more (in what way?)
* flesh out title page more
  - document subtitle (partially solved)
* don't create title page for article doctype (#95, #105)
  - only create title page if doctype=book
* allow character spacing to be controlled by theme
* might be able to avoid dry run for listing/literal in obvious cases; engineering estimate
* implement index of index terms (#386)

* use treetop to parse and evaluate theme file
* use or don't use pad method? check performance

== Major Efforts / Milestones

* add a test suite
* refactor as Prawn View
* add support for AsciiDoc table cell content by writing custom Table::Cell implementation
* add support for nested tables by writing custom Table::Cell implementation
* add support for footnotes (as article or chapter endnotes)
* rework text handling in Prawn to support line height natively

== Documentation

* "Incorrect number of arguments in 'SCN' command" happens when you add a stamp to an imported page
* be mindful that layout_prose adds margin to bottom of content by default (important when working in a bounding box)
* ttfunk does not support ligatures (e.g., fi -> ﬁ); we could do this manually in post_replacements

== Notes

* when using `single_line: true` on formatted_text, it's necessary to reapply our padding top/bottom from line metrics
* we always leave cursor on start of page we're about to write on; certain checks rely on this fact
* "section title" is the semantic element; "heading" is the structural element
* /PageLabels/Nums must have entry for every page in front matter, even if a blank page
  - in fact, must account for every page or else numbering lags behind when scrolling document
* if we set the vposition on image to a numeric value, it skips the overrun check that happens internally
* any instance variables referenced by converter methods for inline nodes could get accessed before the converter for document is called
* Evince throws warning when printing PDF if & is used in document title; but this is valid according to the PDF specification
* Prawn drops fragments with empty text (hence the need to use zero-width space)
  - analyze_glyphs_for_fallback_font_support drops fragments with empty text
  - later on, initialize_wrap drops fragment with empty text
* use term "page number label" to refer to the visible, printed page number (not the implicit page number)

== Known Issues

* inline image at start of the line is slighly shifted to the right due to the fact that it's placed in the center of the reserved fragment width; perhaps we are adding this padding

== Potential Optimizations

* if autofit is set on a listing/literal block that has conums, we are splitting fragments by line twice
* comparing > 0 is slightly faster than == 0 (for cases when we can swap the logic)

== Usage Optimizations

* uncompress PNG files to avoid slow zlib inflating step in Prawn
* flatten PNGs (remove alpha channel) since it messes up font rendering on the page in Adobe Acrobat Reader (need to verify)
* avoid the fallback font if possible (use full fonts in your theme) because it checks for *every* glyph
* font families used in SVGs must match keys in the font catalog

== Open Questions

== Implementation

* should we read SVG file using UTF-8 encoding; or does REXML handle encoding?
* can we leverage before_rendering_page callback on table?
* should we use move_past_bottom in some places instead of start_new_page?

=== Design

* remove/reduce padding above heading when it appears at the start of a page?
* Default line height?
* Should the heading sizes be calculated according to the default font size?
* Page margins
* Body indentation?
  - recto / verso indentation?
* Size of masthead / footer
* Line separating masthead / footer?
* Separate title page
* Start chapter on new page?
* Special layout for chapter page?

=== Theme

* keep or drop base_ prefix in theme? I think we should keep it because it provides context elsewhere in the document (e.g. $base_font_size vs $font_size)

== Resources

* https://code.google.com/p/origami-pdf/[Origami PDF: A PDF inspection library]
* https://github.com/a1ee9b/PrintPretty[A theme for PDF designed for printing]
* http://randomtextgenerator.com[Random Text Generator, supports multiple languages]
* http://clagnut.com/blog/2380[List of pangrams]
  - http://www.camcc.org/_media/reading-group/qianziwen-en.pdf[1,000 character classic (Chinese)]
* pdf2svg can convert the PDF file into an SVG (one SVG per page)
* https://blog.codeship.com/build-math-evaluation-engine[How to Build a Simple Math Evaluation Engine]
* http://blog.typekit.com/2011/11/03/optimizing-fonts-for-the-web-unicode-values-glyph-set-underlines-and-strike-through/[Optimizing Fonts for the Web]


Doc on calling methods

Doc on calling methods

== Java interoperability ==

Golo aims at providing a seamless 2-way interoperability with the Java programming language.

=== Calling static methods ===

Golo can invoke public Java static methods by treating them as functions:

[source,text]
------------------------
module sample

import java.util.Arrays

function oneTwoThree = {
  return asList(1, 2, 3)
}
------------------------

In this example, `asList` is resolved from the `java.util.Arrays` import and called as a function.
Note that we could equivalently have written a qualified invocation as `Arrays.asList(1, 2, 3)`.

=== Calling instance methods ===

When you have an object, you may invoke its methods using the `:` operator.

The following would call the `toString` method of any kind, then print it:

[source,text]
----------------------------------------
println(">>> " + someObject: toString())
----------------------------------------

Of course, you may chain calls as long as a method is not of a `void` return type. Golo converts
Java `void` methods by making them return `null`. This is neither a bug or a feature: the
*invokedynamic* support on the JVM simply does so.

=== Creating objects ===

Golo doesn't have an instantiation operator like `new` in Java. Instead, creating an object and
calling its constructor is done as if it was just another function.

As an example, we may allocate a `java.util.LinkedList` as follows:

[source,text]
---------------------
module sample

import java.util

function aList = {
  return LinkedList()
}
---------------------

Another example would be using a `java.lang.StringBuilder`.

[source,text]
--------------------------------------
function str_build = {
  return java.lang.StringBuilder("h"):
    append("e"):
    append("l"):
    append("l"):
    append("o"):
    toString()
}
--------------------------------------

As one would expect, the `str_build` function above gives the `"hello"` string.

=== Static fields ===

Golo treats public static fields as function, so one could get the maximum value for an `Integer` as
follows:

[source,text]
--------------------------------------
module samples.MaxInt

local function max_int = {
  return java.lang.Integer.MAX_VALUE()
}

function main = |args| {
  println(max_int())
}
--------------------------------------

NOTE: Given than most static fields are used as constants in Java, Golo does not provide support to
change their values. This may change in the future if compelling general-interest use-cases emerge.

=== Inner classes and enumerations ===

We will illustrate both how to deal with public static inner classes and enumerations at once.

The rules to deal with them in Golo are as follows.

1. Inner classes are identified by their real name in the JVM, with nested classes being separated
   by a `$` sign. Hence, `Thread.State` in Java is written `Thread$State` in Golo.
2. Enumerations are just normal objects. They expose each entry as a static field, and each entry is
   an instance of the enumeration class.

Let us consider the following example:

[source,text]
--------------------------------------------------------------------------
module sample.EnumsThreadState

import java.lang.Thread$State

function main = |args| {

  # Call the enum entry like a function
  let new = Thread$State.NEW()
  println("name=" + new: name() + ", ordinal=" + new: ordinal())

  # Walk through all enum entries
  foreach (element in atoList(Thread$State.values())) {
    println("name=" + element: name() + ", ordinal=" + element: ordinal())
  }
}
--------------------------------------------------------------------------

Running it yields the following console output:

[source,console]
------------------------------------------
$ gologolo samples/enums-thread-state.golo
name=NEW, ordinal=0
name=NEW, ordinal=0
name=RUNNABLE, ordinal=1
name=BLOCKED, ordinal=2
name=WAITING, ordinal=3
name=TIMED_WAITING, ordinal=4
name=TERMINATED, ordinal=5
$
------------------------------------------

=== Clashes with Golo operators and escaping ===

Because Golo provides a few named operators such as `is`, `and` or `not`, they are recognized as
operator tokens.

However, you may find yourself in a situation where you need to invoke a Java method whose name is
a Golo operator, such as:

[source,text]
-------------------------------------
# Function call
is()

# Method call
someObject: foo(): is(): not(): bar()
-------------------------------------

This results in a parsing error, as `is` and `not` will be matched as operators instead of method
identifiers.

The solution is to use *escaping*, by prefixing identifiers with a backtick, as in:

[source,text]
---------------------------------------
# Function call
`is()

# Method call
someObject: foo(): `is(): `not(): bar()
---------------------------------------

== Java interoperability ==

Golo aims at providing a seamless 2-way interoperability with the Java programming language.

=== Calling static methods ===

Golo can invoke public Java static methods by treating them as functions:

[source,text]
------------------------
module sample

import java.util.Arrays

function oneTwoThree = {
  return asList(1, 2, 3)
}
------------------------

In this example, `asList` is resolved from the `java.util.Arrays` import and called as a function.
Note that we could equivalently have written a qualified invocation as `Arrays.asList(1, 2, 3)`.

=== Creating objects ===

Golo doesn't have an instantiation operator like `new` in Java. Instead, creating an object and
calling its constructor is done as if it was just another function.

As an example, we may allocate a `java.util.LinkedList` as follows:

[source,text]
---------------------
module sample

import java.util

function aList = {
  return LinkedList()
}
---------------------

Another example would be using a `java.lang.StringBuilder`.

[source,text]
--------------------------------------
function str_build = {
  return java.lang.StringBuilder("h"):
    append("e"):
    append("l"):
    append("l"):
    append("o"):
    toString()
}
--------------------------------------

As one would expect, the `str_build` function above gives the `"hello"` string.

=== Static fields ===

Golo treats public static fields as function, so one could get the maximum value for an `Integer` as
follows:

[source,text]
--------------------------------------
module samples.MaxInt

local function max_int = {
  return java.lang.Integer.MAX_VALUE()
}

function main = |args| {
  println(max_int())
}
--------------------------------------

NOTE: Given than most static fields are used as constants in Java, Golo does not provide support to
change their values. This may change in the future if compelling general-interest use-cases emerge.

=== Inner classes and enumerations ===

We will illustrate both how to deal with public static inner classes and enumerations at once.

The rules to deal with them in Golo are as follows.

1. Inner classes are identified by their real name in the JVM, with nested classes being separated
   by a `$` sign. Hence, `Thread.State` in Java is written `Thread$State` in Golo.
2. Enumerations are just normal objects. They expose each entry as a static field, and each entry is
   an instance of the enumeration class.

Let us consider the following example:

[source,text]
--------------------------------------------------------------------------
module sample.EnumsThreadState

import java.lang.Thread$State

function main = |args| {

  # Call the enum entry like a function
  let new = Thread$State.NEW()
  println("name=" + new: name() + ", ordinal=" + new: ordinal())

  # Walk through all enum entries
  foreach (element in atoList(Thread$State.values())) {
    println("name=" + element: name() + ", ordinal=" + element: ordinal())
  }
}
--------------------------------------------------------------------------

Running it yields the following console output:

[source,console]
------------------------------------------
$ gologolo samples/enums-thread-state.golo
name=NEW, ordinal=0
name=NEW, ordinal=0
name=RUNNABLE, ordinal=1
name=BLOCKED, ordinal=2
name=WAITING, ordinal=3
name=TIMED_WAITING, ordinal=4
name=TERMINATED, ordinal=5
$
------------------------------------------

=== Clashes with Golo operators and escaping ===

Because Golo provides a few named operators such as `is`, `and` or `not`, they are recognized as
operator tokens.

However, you may find yourself in a situation where you need to invoke a Java method whose name is
a Golo operator, such as:

[source,text]
-------------------------------------
# Function call
is()

# Method call
someObject: foo(): is(): not(): bar()
-------------------------------------

This results in a parsing error, as `is` and `not` will be matched as operators instead of method
identifiers.

The solution is to use *escaping*, by prefixing identifiers with a backtick, as in:

[source,text]
---------------------------------------
# Function call
`is()

# Method call
someObject: foo(): `is(): `not(): bar()
---------------------------------------