Configuration¶
Configuration can be specified in environment variables (a limited set of options), in a TOML config file or passed into monstache as program arguments on the command line.
Environment variables names can be suffixed with __FILE. In this case the value of the environment variable will be interpreted as a file path. Monstache will attempt to read the file at that path and use the contents of the file as the value of the variable.
Note
Command line arguments take precedance over environment variables which in turn take precedance over the
TOML config file. You can verify the final configuration used by Monstache by running monstache with -print-config
.
Warning
Keep simple one-line configs above any TOML table definitions in your config file. A TOML table is only ended by another TOML table or the end of the file. Anything below a TOML table will be interpreted to be part of the table by the parser unless it is ended. See the following Issue 58 for more information.
aws-connect¶
TOML table (default nil
)
Enable support for using a connection to Elasticsearch that uses AWS Signature Version 4
strategy¶
int (default 0)
The stategy used to configure the AWS credential provider. The 0 strategy is static and uses the values of access-key
and secret-key
. The 1 strategy is file based and loads the credentials from the credentials-file
setting or from value
of the standard AWS_SHARED_CREDENTIALS_FILE
, or ~/.aws/credentials
. The 2 strategy loads the credentials from the standard
AWS environment variables AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
. The 3 strategy loads the credentials from the default
remote endpoints such as EC2 or ECS roles. The 4 strategy chains together strategies 1-3 and uses the first strategy that returns
a credential.
credentials-file¶
string (default "~/.aws/credentials")
The credentials file to use. Normally, you need not set this as it will come from either AWS_SHARED_CREDENTIALS_FILE
or default
to ~/.aws/credentials
.
profile¶
string (default "")
The AWS profile to use from the credentials file. If not provided a profile named default
will be used.
watch-credentials¶
bool (default false)
Set to true to put a watch on the credentials-watch-dir
. When a file in the watch dir is changed monstache will invalidate the
credentials such that they will be re-established on the next request to Elasticsearch.
credentials-watch-dir¶
string (default "~/.aws")
The path to a directory to watch for changes if watch-credentials
is enabled.
force-expire¶
string (default "")
A golang duration string, e.g. 5m. If given, monstache will force expire the credentials on this interval.
access-key¶
string (default "") (env var name MONSTACHE_AWS_ACCESS_KEY
)
AWS Access Key
secret-key¶
string (default "") (env var name MONSTACHE_AWS_SECRET_KEY
)
AWS Secret Key
region¶
string (default "") (env var name MONSTACHE_AWS_REGION
)
AWS Region
change-stream-namespaces¶
[]string (default nil
) (env var name MONSTACHE_CHANGE_STREAM_NS
)
This option requires MongoDB 3.6 or above
This option allows you to opt in to using MongoDB change streams.
The namespaces included will be tailed using watch
API.
When this option is enabled the direct tailing of the oplog is disabled, therefore you do not need to specify additional
regular expressions to filter the set of collections to watch.
If you are using MongoDB 4 or greater you can open a change stream against entire databases or
even the entire deployment. To tail a database set the value of the namespace to the database name.
For example, instead of db.collection
the value would simply be db
. To tail the entire deployment
use an empty string as the namespace value. For example, change-stream-namespaces = [ '' ]
.
This option may be passed on the command line as ./monstache --change-stream-namespace test.foo
If specified as an environment variable the value should be namespaces separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_CHANGE_STREAM_NS=test.foo,test.bar
config-database-name¶
string (default monstache
)
The name of the MongoDB database that monstache will store metadata under. This metadata includes information to support resuming from a specific point in the oplog and managing cluster mode. This database is only written to for some configurations. Namely, if you specify cluster-name
, enable resume
or set direct-read-stateful
. WARNING: If you are listening to changes via change-stream-namespaces
, you cannot set the same database to both listen to changes & store the configs in.
cluster-name¶
string (default ""
) (env var name MONSTACHE_CLUSTER
)
When cluster-name is given monstache will enter a high availablity mode. Processes with cluster name set to the same value will coordinate. Only one of the processes in a cluster will sync changes. The other processes will be in a paused state. If the process which is syncing changes goes down for some reason one of the processes in paused state will take control and start syncing. See the section high availability for more information.
delete-index-pattern¶
string (default *
)
When using a stateless delete strategy, set this to a valid Elasticsearch index pattern to restrict the scope of possible indexes that a stateless delete
will consider. If monstache only indexes to index a, b, and c then you can set this to a,b,c
. If monstache only indexes to indexes starting with
mydb then you can set this to mydb*
.
delete-strategy¶
int (default 0
)
The strategy to use for handling document deletes when custom indexing is done in scripts.
Strategy 0 -default- will do a term query by document id across all Elasticsearch indexes in delete-index-pattern
.
Will only perform the delete if one single document is returned by the query.
Stategy 1 -deprecated- will store indexing metadata in MongoDB in the monstache.meta
collection and use this metadata to locate and delete the document.
Stategy 2 will completely ignore document deletes in MongoDB.
direct-read-bounded¶
boolean (default false
)
When this option is enabled monstache will ensure that all direct read queries have a min and max set on the query. This ensures that direct reads will complete and not chase new data that is being inserted while the cursor is being exhausted.
direct-read-concur¶
int (default 0
)
This option allows you to control the number of namespaces in direct-read-namespaces
which will be syncing concurrently.
By default monstache starts reading and syncing all namespaces concurrently. If this places too much stress on MongoDB then
you can set this option to an integer greater than 0. If you set it to 1
, for example, then monstache will sync the
collections serially. Numbers greater than 1 allow you to sync collections in batches of that size.
direct-read-dynamic-exclude-regex¶
string (default ""
) (env var name MONSTACHE_DIRECT_READ_NS_DYNAMIC_EXCLUDE_REGEX
)
This option is only available in monstache v5 and v6.
This options allows you to exclude any collections that match the given regex when monstache is directed to dynamically
register direct-read-namespaces
. When direct read namespaces are explicit it is not used.
direct-read-dynamic-include-regex¶
string (default ""
) (env var name MONSTACHE_DIRECT_READ_NS_DYNAMIC_INCLUDE_REGEX
)
This option is only available in monstache v5 and v6.
This options allows you to only include collections that match the given regex when monstache is directed to dynamically
register direct-read-namespaces
. When direct read namespaces are explicit it is not used.
direct-read-namespaces¶
[]string (default nil
) (env var name MONSTACHE_DIRECT_READ_NS
)
This option allows you to directly copy collections from MongoDB to Elasticsearch. Monstache allows filtering the data that is actually indexed to Elasticsearch, so you need not necessarily copy the entire collection.
Note
In monstache v5 and v6 you can use an array with a single empty string to direct monstache to dynamically discover your
collections and perform direct reads on them. When direct-read-dynamic-exclude-regex
is configured you can prune
from the list that is discovered. System collections will not be considered for inclusion in the discovery.
Since the oplog is a capped collection it may only contain a subset of all your data. In this case you can perform a direct sync of Mongodb to Elasticsearch. To do this, set direct-read-namespaces to an array of namespaces that you would like to copy. Monstache will perform reads directly from the given set of db.collection and sync them to Elasticsearch.
Note
This option may be passed on the command line as ./monstache --direct-read-namespace test.foo --direct-read-namespace test.bar
If specified as an environment variable the value should be namespaces separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_DIRECT_READ_NS=test.foo,test.bar
Warning
When direct reads are enabled Monstache still processes change events while the direct reads are being performed.
It does not wait until direct reads are completed to start listening for changes. This is to ensure that any
changes that occur during the direct read process get synchronized.
By default, Monstache maps a MongoDB collection named foo
in a database named test
to the test.foo
index in Elasticsearch.
For maximum indexing performance when doing alot of a direct reads you might want to adjust the refresh interval during indexing on the destination Elasticsearch indices. The refresh interval can be set at a global level in elasticsearch.yml or on a per index basis by using the Index Settings or Index Template APIs.
For more information see Update Indices Settings.
By default, Elasticsearch refreshes every second. You will want to increase this value or turn off refresh completely during the indexing phase by setting the refresh_interval to -1. Remember to reset the refresh_interval to a positive value and do a force merge after the indexing phase has completed if you decide to temporarily turn off refresh, otherwise you will not be able to see the new documents in queries.
Another way to speed up bulk indexing is to set the number_of_replicas to 0 while indexing and then later increase the number of replicas. The following index template shows how one might configure a target index for better indexing throughput by controlling replicas and the refresh interval. The index template needs to be installed before running monstache.
{
"index_patterns": ["test.*"],
"settings": {
"number_of_shards": 5,
"number_of_replicas": 0,
"refresh_interval": "30s"
}
}
direct-read-no-timeout¶
boolean (default false
)
When direct-read-no-timeout is true monstache will set the no cursor timeout flag on cursors opened for direct reads.
The default is not to do this since having cursors without timeouts is not generally a good practice. However, for
reading very large collections you may find it necessary to avoid cursor timeout errors. An alternative to enabling
this setting is to increase the cursor timeout on your MongoDB server or look into using the direct-read-split-max
and direct-read-concur
options to limit the number of cursors opened for direct reads.
direct-read-split-max¶
int (default 9
)
The maximum number of times to split a collection for direct reads. This setting greatly impacts the memory consumption of Monstache. When direct reads are performed, the collection is first broken up into ranges which are then read concurrently is separate go routines. If you increase this value you will notice the connection count increase in mongostat when direct reads are performed. You will also notice the memory consumption of Monstache grow. Increasing this value can increase the throughput for reading large collections, but you need to have enough memory available to Monstache to do so. You can decrease this value for a memory constrained Monstache process.
To disable collection splitting altogether, set this option to -1
. In this case monstache will not try to segment the
collection, but rather use a single cursor for the entire read.
direct-read-stateful¶
boolean (default false
)
When this setting is set to true monstache will mark direct read namespaces as complete after they have been fully read in
a collection named directreads
in the monstache config database.
On subsequent restarts monstache will check this collection and only start direct reads for the namespaces not in the completed list.
This allow you to keep the list the direct read namespaces in the configuration but manage the list that has completed and should
not be run again externally in MongoDB. Deleting the directreads
collection and restarting monstache will force a full sync.
disable-change-events¶
boolean (default false
)
When disable-change-events is true monstache will not listen to change events from the oplog or call watch on any collections. This option is only useful if you are using direct-read-namespaces to copy collections and would prefer not to sync change events.
disable-file-pipeline-put¶
boolean (default false
)
This setting only applies to monstache versions 5 and 6.
When this option is true monstache will not attempt to auto create an ingest pipeline named attachment
with a file
field
at startup when index-files
is enabled. In this case the user must create the pipeline before running monstache.
For example, the user must issue a command against Elasticsearch as follows prior to running monstache in order
to index GridFS files:
PUT _ingest/pipeline/attachment
{
"description" : "Extract file information",
"processors" : [
{
"attachment" : {
"field" : "file",
"indexed_chars" : -1
}
}
]
}
dropped-databases¶
boolean (default true
)
When dropped-databases is false monstache will not delete the mapped indexes in Elasticsearch if a MongoDB database is dropped
dropped-collections¶
boolean (default true
)
When dropped-collections is false monstache will not delete the mapped index in Elasticsearch if a MongoDB collection is dropped
elasticsearch-user¶
string (default ""
) (env var name MONSTACHE_ES_USER
)
Optional Elasticsearch username for basic auth
elasticsearch-password¶
string (default ""
) (env var name MONSTACHE_ES_PASS
)
Optional Elasticsearch password for basic auth
elasticsearch-urls¶
[]string (default [ "http://localhost:9200" ]
) (env var name MONSTACHE_ES_URLS
)
An array of URLs to connect to the Elasticsearch REST Interface
Note
This option may be passed on the command line as ./monstache --elasticsearch-url URL1 --elasticsearch-url URL2
If specified as an environment variable the value should be URLs separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_ES_URLS=http://es1:9200,http://es2:9200
elasticsearch-healthcheck-timeout-startup¶
int (default 15
)
The number of seconds to wait on the initial health check to Elasticsearch to responed before giving up and exiting.
elasticsearch-healthcheck-timeout¶
int (default 5
)
The number of seconds to wait for a post-initial health check to Elasticsearch to respond
elasticsearch-version¶
string (by default determined by connecting to the server
)
When elasticsearch-version is provided monstache will parse the given server version to determine how to interact with the Elasticsearch API. This is normally not recommended because monstache will connect to Elasticsearch to find out which version is being used. This option is provided for cases where connecting to the base URL of the Elasticsearch REST API to get the version is not possible or desired.
elasticsearch-max-conns¶
int (default 4
)
The size of the Elasticsearch HTTP connection pool. This determines the concurrency of bulk indexing requests to Elasticsearch. If you increase this value too high you may begin to see bulk indexing failures if the bulk index queue gets overloaded. To increase the size of the bulk indexing queue you can update the Elasticsearch config file:
thread_pool:
bulk:
queue_size: 200
For more information see Thread Pool.
You will want to tune this variable in sync with the elasticsearch-max-bytes
option.
elasticsearch-retry¶
boolean (default false
)
When elasticseach-retry is true a failed request to Elasticsearch will be retried with an exponential backoff policy. The policy is set with an initial timeout of 50 ms, an exponential factor of 2, and a max wait of 20 seconds. For more information on how this works see Back Off Strategy
elasticsearch-client-timeout¶
int (default 0
)
The number of seconds before a request to Elasticsearch times out. A setting of 0, the default, disables the timeout.
elasticsearch-max-docs¶
int (default -1
)
When elasticsearch-max-docs is given a bulk index request to Elasticsearch will be forced when the buffer reaches the given number of documents.
Warning
It is not recommended to change this option but rather use elasticsearch-max-bytes
instead since the document count is not a good gauge of when
to flush. The default value of -1 means to not use the number of docs as a flush indicator.
elasticsearch-max-bytes¶
int (default 8MB as bytes)
When elasticsearch-max-bytes is given a bulk index request to Elasticsearch will be forced when a connection buffer reaches the given number of bytes. This setting greatly impacts performance. A high value for this setting will cause high memory monstache memory usage as the documents are buffered in memory.
Each connection in elasticsearch-max-conns
will flush when its queue gets filled to this size.
elasticsearch-max-seconds¶
int (default 1
)
When elasticsearch-max-seconds is given a bulk index request to Elasticsearch will be forced when a request has not been made in the given number of seconds.
The default value is automatically increased to 5
when direct read namespaces are detected. This is to ensure that flushes do not happen too often in this
case which would cut performance.
elasticsearch-pem-file¶
string (default ""
) (env var name MONSTACHE_ES_PEM
)
When elasticsearch-pem-file is given monstache will use the given file path to add a local certificate to x509 cert pool when connecting to Elasticsearch. This should only be used when Elasticsearch is configured with SSL enabled.
elasticsearch-pki-auth¶
TOML table (default nil
)
Used to configure client to use PKI user auth for Elasticsearch
cert-file¶
string (default "") (env var name MONSTACHE_ES_PKI_CERT
)
Path to the cert file e.g. the --cert argument to curl
key-file¶
string (default "") (env var name MONSTACHE_ES_PKI_KEY
)
Path to the key file e.g. the --key argument to curl
elasticsearch-validate-pem-file¶
boolean (default true
) (env var name MONSTACHE_ES_VALIDATE_PEM
)
When elasticsearch-validate-pem-file is false TLS will be configured to skip verification
enable-easy-json¶
boolean (default false
)
When enable-easy-json is true monstache will the easy-json library to serialize requests to Elasticsearch
enable-http-server¶
boolean (default false
)
Add this flag to enable an embedded HTTP server at localhost:8080
enable-oplog¶
boolean (default false
)
This option only applies to monstache v5 and v6. Enabling it turns on change event emulation feature that tails the MongoDB oplog directly. It should only be turned on when pairing monstache v5 or v6 with a MongoDB server at a server compatibility version less than 3.6.
enable-patches¶
boolean (default false
)
Set to true to enable storing rfc7396 patches in your Elasticsearch documents
env-delimiter¶
string (default ,
)
This option is only supported on the command line. The value for this delimiter will be used to split environment variable values when the
environment variable is used in conjunction with an option of array type. E.g. with export MONSTACHE_DIRECT_READ_NS=test.test,foo.bar
.
exit-after-direct-reads¶
boolean (default false
)
The direct-read-namespaces option gives you a way to do a full sync on multiple collections. At times you may want to perform a full sync via the direct-read-namespaces option and then quit monstache. Set this option to true and monstache will exit after syncing the direct read collections instead of continuing to tail the oplog. This is useful if you would like to run monstache to run a full sync on a set of collections via a cron job.
fail-fast¶
boolean (default false
)
When fail-fast is true, if monstache receives a failed bulk indexing response from Elasticsearch, monstache will log the request that produced the response as an ERROR and then exit immediately with an error status. Normally, monstache just logs the error and continues processing events.
If monstache has been configured with elasticsearch-retry true, a failed request will be retried before being considered a failure.
file-downloaders¶
int (default 10
)
Number of go routines concurrently processing GridFS files when file index-files
is turned on.
file-highlighting¶
boolean (default false
)
When file-highlighting is true monstache will enable the ability to return highlighted keywords in the extracted text of files for queries on files which were indexed in Elasticsearch from gridfs.
file-namespaces¶
[]string (default nil
) (env var name MONSTACHE_FILE_NS
)
The file-namespaces config must be set when index-files is enabled. file-namespaces must be set to an array of MongoDB namespace strings. Files uploaded through gridfs to any of the namespaces in file-namespaces will be retrieved and their raw content indexed into Elasticsearch via either the mapper-attachments or ingest-attachment plugin.
Note
This option may be passed on the command line as ./monstache --file-namespace test.foo --file-namespace test.bar
If specified as an environment variable the value should be namespaces separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_FILE_NS=test.foo,test.bar
filter¶
[] array of TOML table (default nil
)
When filter is given monstache will pass the MongoDB document from an insert or update operation into the filter function immediately after it is read from the oplog. Return true from the function to continue processing the document or false to completely ignore the document. See the section Middleware for more information.
namespace¶
string (default "")
The MongoDB namespace, db.collection, to apply the script to. If you omit namespace the filter function will be applied to all documents.
script¶
string (default "")
An inline script. You can use TOML multiline syntax here. The function should take 2 arguments, a doc and a namespace, and return true/false to include or filter the document.
path¶
string (default "")
The file path to load a script from. Use this or an inline script but not both. Can be a path relative to the directory monstache is executed from or an absolute path.
graylog-addr¶
string (default "") (env var name MONSTACHE_GRAYLOG_ADDR
)
The address of a graylog server to redirect logs to in GELF
gtm-settings¶
TOML table (default nil
)
The following gtm configuration properties are available. See gtm for details
channel-size¶
int (default 512)
Controls the size of the go channels created for processing events. When many events are processed at once a larger channel size may prevent blocking in gtm.
buffer-size¶
int (default 32)
Determines how many documents are buffered by a gtm worker go routine before they are batch fetched from MongoDB. When many documents are inserted or updated at once it is better to fetch them together.
buffer-duration¶
string (default 75ms)
A string representation of a golang duration. Determines the maximum time a buffer is held before it is fetched in batch from MongoDB and flushed for indexing.
max-await-time¶
string (default "")
A string represetation of a golang duration, e.g. "10s". If set, will be converted and passed at the
maxAwaitTimeMS
option for change streams. This determines the maximum amount of time in milliseconds
the server waits for new data changes to report to the change stream cursor before returning an empty batch.
gzip¶
boolean (default false
)
When gzip is true, monstache will compress requests to Elasticsearch. If you enable gzip in monstache and are using Elasticsearch prior to version 5 you will also need to update the Elasticsearch config file to set http.compression: true. In Elasticsearch version 5 and above http.compression is enabled by default. Enabling gzip compression is recommended if you enable the index-files setting.
http-server-addr¶
string (default :8080
) (env var name MONSTACHE_HTTP_ADDR
)
The address to bind the embedded HTTP server on if enabled
index-as-update¶
boolean (default false
)
When index-as-update is set to true monstache will sync create and update operations in MongoDB as updates to Elasticsearch. This does not change the fact that Monstache always sends an entire copy of the data in MongoDB. It just means that any existing non-overlapping fields in Elasticsearch will be maintained.
By default, monstache will overwrite the entire document in Elasticsearch. This setting may be useful if you make updates to Elasticsearch to the documents monstache has previously synced out of band and would like to retain these updates when the document changes in MongoDB. You will only be able to retain fields in Elasticsearch that do not overlap with fields in MongoDB.
When this setting is turned on some guarantees about the order of operations applied in Elasticsearch are lost. The reason for this is that the version field cannot be set with this enabled. The version field by default is set to the timestamp of the event in MongoDB. Elasticsearch will only apply changes if the version number is greater or equal to the last value indexed maintaining serialization.
If you enable this setting and do not see serialized updates in MongoDB being indexed correctly then you can mitigate this problem with the following settings:
elasticsearch-max-conns = 1
[gtm-settings]
buffer-size = 2048
buffer-duration = 4s
index-files¶
boolean (default false
)
When index-files is true monstache will index the raw content of files stored in GridFS into Elasticsearch as an attachment type. By default index-files is false meaning that monstache will only index metadata associated with files stored in GridFS. In order for index-files to index the raw content of files stored in GridFS you must install a plugin for Elasticsearch. For versions of Elasticsearch prior to version 5, you should install the mapper-attachments plugin. In version 5 or greater of Elasticsearch the mapper-attachment plugin is deprecated and you should install the ingest-attachment plugin instead. For further information on how to configure monstache to index content from GridFS, see the section GridFS support.
index-oplog-time¶
boolean (default false
)
If this option is set to true monstache will include 2 automatic fields in the source document indexed into
Elasticsearch. The first is oplog_ts
which is the timestamp for the event copied directly from the MongoDB
oplog. The second is oplog_date
which is an Elasticsearch date field corresponding to the time of the same
event.
This information is generally useful in Elasticsearch giving the notion of last updated. However, it's also valuable information to have for failed indexing requests since it gives one the information to replay from a failure point. See the option resume-from-timestamp for information on how to replay oplog events since a given event occurred.
For data read via the direct read feature the oplog time will only be available if the id of the MongoDB document is an ObjectID. If the id of the MongoDB document is not an ObjectID and the document source is a direct read query then the oplog time will not be available.
index-stats¶
boolean (default false
)
When both stats and index-stats are true monstache will write statistics about its indexing progress in Elasticsearch instead of standard out.
The indexes used to store the statistics are time stamped by day and
prefixed monstache.stats.
. E.g. monstache.stats.2017-07-01 and so on.
As these indexes will accrue over time your can use a tool like curator to prune them with a Delete Indices action and an age filter.
logs¶
TOML table (default nil
) (env var name MONSTACHE_LOG_DIR
)
Allows writing logs to a file using a rolling appender instead of stdout. Supply a file path for each type of log you would like to send to a file.
When the MONSTACHE_LOG_DIR
environment variable is used then a log file for each log level will be generated in the given directory.
info¶
string (default "")
The file path to write info level logs to
warn¶
string (default "")
The file path to write warning level logs to
error¶
string (default "")
The file path to write error level logs to
trace¶
string (default "")
The file path to write trace level logs to. Trace logs are enabled via the verbose option.
stats¶
string (default "")
The file path to write indexing statistics to. Stats logs are enabled via the stats option.
log-rotate¶
TOML table (default nil
)
Use to configure how log files are rotated/managed when logging to files. These options are passed through to the lumberjack logger.
max-size¶
int (default 500) (env var name MONSTACHE_LOG_MAX_SIZE
)
MaxSize is the maximum size in megabytes of the log file before it gets rotated.
max-age¶
int (default 28) (env var name MONSTACHE_LOG_MAX_AGE
)
MaxAge is the maximum number of days to retain old log files based on the timestamp encoded in their filename. Note that a day is defined as 24 hours and may not exactly correspond to calendar days due to daylight savings, leap seconds, etc. Use a value of zero to ignore the age of files.
max-backups¶
int (default 5) (env var name MONSTACHE_LOG_MAX_BACKUPS
)
MaxBackups is the maximum number of old log files to retain. Use a value of zero to retain all old log files (though MaxAge may still cause them to get deleted.)
localtime¶
boolean (default false)
LocalTime determines if the time used for formatting the timestamps in backup files is the computer's local time. The default is to use UTC time.
compress¶
boolean (default false)
Compress determines if the rotated log files should be compressed using gzip. The default is not to perform compression.
mapper-plugin-path¶
string (default ""
)
The path to an .so file golang plugin.
mapping¶
[] array of TOML table (default nil
)
When mapping is given monstache will be directed to override the default index and type assigned to documents in Elasticsearch. See the section Index Mapping for more information.
namespace¶
string (default "")
The MongoDB namespace, db.collection, to apply the mapping to.
index¶
string (default "same as namespace including the dot. e.g. test.test")
Allows you to override the default index that monstache will send documents to. By default, the index is the same as the MongoDB namespace.
type¶
string (default "_doc for ES 6.2+ and the name of the MongoDB collection otherwise")
Allows you to override the default type that monstache will index documents with. Overriding the type is not recommended for Elasticsearch version 6.2+.
pipeline¶
string (default "")
The name of an existing Elasticsearch pipeline to index the data with. A pipeline is a series of Elasticsearch processors to be executed. An Elasticsearch pipeline is one way to transform MongoDB data before indexing.
max-file-size¶
int (default 0
)
When max-file-size is greater than 0 monstache will not index the content of GridFS files that exceed this limit in bytes.
merge-patch-attribute¶
string (default json-merge-patches
)
Customize the name of the property under which merge patches are stored
mongo-url¶
string (default localhost
) (env var name MONSTACHE_MONGO_URL
)
The URL to connect to MongoDB which must follow the Standard Connection String Format
For sharded clusters this URL should point to the mongos
router server and the mongo-config-url
option must be set to point to the config server.
mongo-config-url¶
string (default ""
) (env var name MONSTACHE_MONGO_CONFIG_URL
)
This config must only be set for sharded MongoDB clusters. Has the same syntax as mongo-url.
This URL must point to the MongoDB config
server.
Monstache will read the list of shards using this connection and then setup a listener to react to new shards being added to the cluster at a later time. It will then setup a new direct connection to each shard to listen for events.
Setting the mongo-config-url is not necessary if you are using change-stream-namespaces.
mongo-pem-file¶
string (default ""
) (env var name MONSTACHE_MONGO_PEM
)
This setting only applies to the mgo driver in monstache versions 3 and 4. The driver in monstache 5 and 6 uses the connection string for all settings.
When mongo-pem-file is given monstache will use the given file path to add a local certificate to x509 cert pool when connecting to MongoDB. This should only be used when MongoDB is configured with SSL enabled.
mongo-validate-pem-file¶
boolean (default true
) (env var name MONSTACHE_MONGO_VALIDATE_PEM
)
This setting only applies to the mgo driver in monstache versions 3 and 4. The driver in monstache 5 and 6 uses the connection string for all settings.
When mongo-validate-pem-file is false TLS will be configured to skip verification
mongo-oplog-database-name¶
string (default local
) (env var name MONSTACHE_MONGO_OPLOG_DB
)
When mongo-oplog-database-name is given monstache will look for the MongoDB oplog in the supplied database
mongo-oplog-collection-name¶
string (default oplog.rs
) (env var name MONSTACHE_MONGO_OPLOG_COL
)
When mongo-oplog-collection-name is given monstache will look for the MongoDB oplog in the supplied collection.
The collection defaults to oplog.rs
which is what will be produced when replica sets are enabled. If you are
using an old version of MongoDB with master
based replication instead of replica sets, then you will need to
configure this setting to oplog.$main
.
Warning
If this setting was not supplied monstache would previously search for the first collection prefixed oplog
in the local
database. However, starting in monstache v4.13.1
and v3.20.1
this behavior has changed.
Now, monstache will not do dynamic resolution. Since master
based replication in MongoDB is no longer
supported, monstache now defaults to oplog.rs
and will only use another collection (e.g. oplog.$main
) if
you explicitly config it to do so.
mongo-dial-settings¶
TOML table (default nil
)
This setting only applies to the mgo driver in monstache versions 3 and 4. The driver in monstache 5 and 6 uses the connection string for all settings.
The following MongoDB dial properties are available. Timeout values of 0 disable the timeout.
ssl¶
bool (default false)¶
Set to true to establish a connection using TLS.
timeout¶
int (default 15)¶
Seconds to wait when establishing an initial connection to MongoDB before giving up
read-timeout¶
int (default 30)¶
Seconds to wait when reading data from MongoDB before giving up. Must be greater than 0. This should be greater than 10 because Monstache waits 10s for new change events by retrying the query.
write-timeout¶
int (default 30)¶
Seconds to wait when writing data to MongoDB before giving up. Must be greated than 0. This should be greater than 10 because Monstache waits 10s for new change events by retrying the query.
mongo-session-settings¶
TOML table (default nil
)
This setting only applies to the mgo driver in monstache versions 3 and 4. The driver in monstache 5 and 6 uses the connection string for all settings.
The following MongoDB session properties are available. Timeout values of 0 disable the timeout.
socket-timeout¶
int (default 0)
Seconds to wait for a non-responding socket before it is forcefully closed
sync-timeout¶
int (default 30)
Amount of time in seconds an operation will wait before returning an error in case a connection to a usable server can't be established. Must be greater than 0.
mongo-x509-settings¶
TOML table (default nil
)
This setting only applies to the mgo driver in monstache versions 3 and 4. The driver in monstache 5 and 6 uses the connection string for all settings.
Allows one to configure x509 authentication with MongoDB. For more information see x509 auth.
Note
You must configure your mongo-url
with the request parameter authMechanism=MONGODB-X509
.
You must also supply both of the following file paths:
client-cert-pem-file¶
string (default "")
The path to a PEM encoded file containing the client cert
client-key-pem-file¶
string (default "")
The path to a PEM encoded file containing the client key
namespace-drop-exclude-regex¶
regex (default ""
) (env var name MONSTACHE_NS_DROP_EXCLUDE_REGEX
)
When namespace-drop-exclude-regex is given this regex is tested against the namespace, database.collection, of drops in MongoDB. For database drops the namespace will be database-name.$cmd. For collections drops the namespace will be database-name.collection-name. If the regex does not match the namespace then the operation will by synced.
namespace-drop-regex¶
regexp (default ""
) (env var name MONSTACHE_NS_DROP_REGEX
)
When namespace-drop-regex is given this regex is tested against the namespace, database.collection, of drops in MongoDB. For database drops the namespace will be database-name.$cmd. For collections drops the namespace will be database-name.collection-name. If the regex matches the namespace then the operation will by synced.
namespace-exclude-regex¶
regex (default ""
) (env var name MONSTACHE_NS_EXCLUDE_REGEX
)
When namespace-exclude-regex is given this regex is tested against the namespace, database.collection, of any insert, update, delete in MongoDB. If the regex matches monstache ignores the event, otherwise it continues processing event filters. By default monstache processes events in all databases and all collections with the exception of the reserved database monstache, any collections suffixed with .chunks, and the system collections. For more information see the section Namespaces.
namespace-regex¶
regexp (default ""
) (env var name MONSTACHE_NS_REGEX
)
When namespace-regex is given this regex is tested against the namespace, database.collection, of any insert, update, delete in MongoDB. If the regex matches monstache continues processing event filters, otherwise it drops the event. By default monstache processes events in all databases and all collections with the exception of the reserved database monstache, any collections suffixed with .chunks, and the system collections. For more information see the section Namespaces.
oplog-date-field-format¶
string (default 2006/01/02 15:04:05
)
Use this option to override the layout for formatting the oplog_date
field. Refer to
the Format function for the reference time
values to use in the layout.
oplog-date-field-name¶
string (default oplog_date
)
Use this option to override the name of the field used to store the oplog date string
oplog-ts-field-name¶
string (default oplog_ts
)
Use this option to override the name of the field used to store the oplog timestamp
patch-namespaces¶
[]string (default nil
) (env var name MONSTACHE_PATCH_NS
)
An array of MongoDB namespaces that you would like to enable rfc7396 patches on
Note
This option may be passed on the command line as ./monstache --patch-namespace test.foo --patch-namespace test.bar
If specified as an environment variable the value should be namespaces separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_PATCH_NS=test.foo,test.bar
pipeline¶
[] array of TOML table (default nil
)
When pipeline is given monstache will call the function specified to determine an array of aggregation pipeline stages to run. See the section Middleware for more information.
namespace¶
string (default "")
The MongoDB namespace, db.collection, to apply the script to. If you omit the namespace the pipeline function with be applied to all namespaces.
script¶
string (default "")
An inline script. You can use TOML multiline syntax here. The function should take 2 arguments, a namespace and a boolean indicating whether or not the data is a change stream.
The function should return an array of aggregation pipeline stages. Note, for change streams the root of the pipeline will be the change event with a field fullDocument
representing the
changed doc. You should alter your pipeline stages according to this boolean. Monstache needs the change event data so do not replace the root of the document in your pipeline for change
streams.
path¶
string (default "")
The file path to load a script from. Use this or an inline script but not both. Can be a path relative to the directory monstache is executed from or an absolute path.
pipe-allow-disk¶
boolean (default false
)
Add this flag to allow MongoDB to use the disk as a temporary store for data during aggregation pipelines
post-processors¶
int (default 10
)
Number of go routines concurrently calling the Process
method in any golang middleware plugins installed via mapper-plugin-path
.
pprof¶
boolean (default false
)
When pprof is true and the http server is enabled, monstache will make profiling information available.
See Profiling for Go for more information.
print-config¶
boolean (default false
)
When print-config is true monstache will print its configuration and then exit
prune-invalid-json¶
boolean (default false
)
If your MongoDB data contains values like +Infinity, -Infinity, NaN, or invalid dates you will want to set this option to true. The Golang json serializer is not able to handle these values and the indexer will get stuck in an infinite loop. When prune-invalid-json is set to true Monstache will drop those fields so that indexing errors do not occur.
relate¶
[] array of TOML table (default nil
)
Allows one to relate 2 namespaces together such that a change to one causes a sync of the associated namespace
namespace¶
string (default "")
The namespace of the collection that, when modified, triggers a sync of the with-namespace
with-namespace¶
string (default "")
The namespace of the collection or view that will be synced when namespace changes
src-field¶
string (default "_id")
The name of the field in namespace that will be extracted from the change doc and used as the value side of the query into with-namespace
match-field¶
string (default "_id")
The name of the field in with-namespace that will be used as the field name to match side of the query into with-namespace
match-field-type¶
string (default "")
Valid values for this property are objectId
, string
, int
, long
and decimal
. If the property is given the value of the src-field
will be converted into the type given (if possible) before being used to query against match-field
.
keep-src¶
bool (default "false")
Whether or not to sync the original change event in addition to the one looked up in with-namespace. By default the original change is ignored and only the document from with-namespace is synced.
max-depth¶
int (default 0)
If max-depth is greater than 0 then the relationship will only fire if the number of relationships between this relate and the originating event is less than or equal to the value given. By default monstache will continue following relationships until none are left.
dot-notation¶
bool (default "false")
If match-field
is a nested field like foo.bar
then setting dot-notation
to true
produces the query { "foo.bar": 1 }
to MongoDB. If dot-notation
is not enabled then the query will
be an exact match query sent as { foo { bar : 1 } }
.
relate-buffer¶
int (default 1000
)
Number of relate events allowed to queue up before skipping the event and reporting an error.
This setting was introduced to prevent scenarios where relate queries get queued up and stall
the pipeline. You can increase this value if you are hitting this limit, but monstache will take
more memory to hold the events and you may find that MongoDB experiences high CPU due to Monstache
performing many queries concurrently to try to clear the buffer. This limit is usually hit when
you have a relate
config and then do a mass insert or update against the relate namespace.
relate-threads¶
int (default 10
)
Number of go routines concurrently processing relationships when relate
is enabled. This dictates
the concurrency of queries trying to unload the relate queue.
replay¶
boolean (default false
)
Warning
Replay is currently deprecated in favor of direct-read-namespaces
. Replay may be removed in a future release.
When replay is true, monstache replays all events from the beginning of the MongoDB oplog and syncs them to Elasticsearch.
If you've previously synced Monstache to Elasticsearch you may see many WARN statments in the log indicating that there was a version conflict. This is normal during a replay and it just means that you already have data in Elasticsearch that is newer than the point in time data from the oplog.
When resume and replay are both true, monstache replays all events from the beginning of the MongoDB oplog, syncs them to Elasticsearch and also writes the timestamps of processed events to monstache.monstache.
When neither resume nor replay are true, monstache reads the last timestamp in the oplog and starts listening for events occurring after this timestamp (tails starting at the end). Timestamps are not written to monstache.monstache. This is the default behavior.
resume¶
boolean (default false
)
When resume is true, monstache writes the timestamp of MongoDB operations it has successfully synced to Elasticsearch to the collection monstache.monstache. It also reads that timestamp from that collection when it starts in order to replay events which it might have missed because monstache was stopped. If monstache is started with the cluster-nameoption set then resume is automatically turned on.
resume-from-timestamp¶
int64 (default 0
)
This option only applies when the resume-strategy
is 0 for timestamp based resume.
When resume-from-timestamp (a 64 bit timestamp where the high 32 bytes represent the number of seconds since epoch and the low 32 bits represent an offset within a second) is given, monstache will sync events starting immediately after the timestamp. This is useful if you have a specific timestamp from the oplog and would like to start syncing from after this event.
If you supply an integer that is greater than 0 but less than or equal to the max value of a 32-bit integer then monstache will interpret the value as seconds since the epoch and automatically shift the value 32 bits left.
resume-name¶
string (default default
)
monstache uses the value of resume-name as an id when storing and retrieving timestamps
to and from the MongoDB collection monstache.monstache
. The default value for this option is the string default
.
However, there are some exceptions. If monstache is started with the cluster-name option set then the
name of the cluster becomes the resume-name. This is to ensure that any process in the cluster is able to resume
from the last timestamp successfully processed. Another exception occurs when worker is enabled.
In that case the worker name becomes the resume-name.
resume-strategy¶
int (default 0
)
The strategy to use for resuming streams from previous runs of monstache. Only applies when resume
is enabled.
This strategy is also used when cluster-name
is set to ensure streams are resumed when the active process in the
cluster switches.
Strategy 0 -default- Timestamp based resume of change streams. Compatible with MongoDB API 4.0+.
Stategy 1 Token based resume of change streams. Compatible with MongoDB API 3.6+.
Timestamps and tokens are written periodically to the database configured in config-database-name
. Timestamps are
written to the collection named monstache
. Tokens are written to the collection named tokens
.
resume-write-unsafe¶
boolean (default false
)
When resume-write-unsafe is true monstache sets the safety mode of the MongoDB session such that writes are fire and forget. This speeds up writing of timestamps used to resume synching in a subsequent run of monstache. This speed up comes at the cost of no error checking on the write of the timestamp. Since errors writing the last synched timestamp are only logged by monstache and do not stop execution it's not unreasonable to set this to true to get a speedup.
routing-namespaces¶
[]string (default nil
)
You only need to set this configuration option if you use golang and javascript plugins are do custom routing: override parent or routing attributes. This array should be set to a list of all the namespaces that custom routing is done on. This ensures that deletes in MongoDB are routed correctly to Elasticsearch.
script¶
[] array of TOML table (default nil
)
When script is given monstache will pass the MongoDB document into the script before indexing into Elasticsearch. See the section Middleware for more information.
namespace¶
string (default "")
The MongoDB namespace, db.collection, to apply the script to. If you omit the namespace the mapping function with be applied to all documents.
routing¶
boolean (default false)
Set routing to true if you override the index, routing or parent metadata via _meta_monstache
script¶
string (default "")
An inline script. You can use TOML multiline syntax here. The function should take 2 arguments, a doc and a namespace, and return a modified doc. You can also return true to index the original document or false to ignore the document and schedule any previous documents with the same id for deletion.
path¶
string (default "")
The file path to load a script from. Use this or an inline script but not both. Can be a path relative to the directory monstache is executed from or an absolute path.
stats¶
boolean (default false
)
When stats is true monstache will periodically print statistics accumulated by the indexer
stats-duration¶
string (default 30s
)
Sets the duration after which statistics are printed if stats is enabled
stats-index-format¶
string (default monstache.stats.2006-01-02
)
The time.Time
supported index name format for stats indices. By default, stats indexes
are partitioned by day. To use less indices for stats you can shorten this format string
(e.g monstache.stats.2006-01) or remove the time component completely to use a single index.
time-machine-namespaces¶
[]string (default nil
) (env var name MONSTACHE_TIME_MACHINE_NS
)
Monstache is good at keeping your MongoDB collections and Elasticsearch indexes in sync. When a document is updated in MongoDB the corresponding document in Elasticsearch is updated too. Same goes for deleting documents in MongoDB. But what if you also wanted to keep a log of all the changes to a MongoDB document over its lifespan. That's what time-machine-namespaces are for. When you configure a list of namespaces in MongoDB to add to the time machine, in addition to keeping documents in sync, Monstache will index of copy of your MongoDB document at the time it changes in a separate timestamped index.
Say for example, you insert a document into the test.test
collection in MongoDB. Monstache will index by default into the test.test
index in Elasticsearch, but with time machines it will also index it into log.test.test.2018-02-19
. When it indexes it into the time machine index it does so without the id from MongoDB and lets Elasticsearch generate a unique id. But, it stores the id from MongoDB in the source field _source_id
. Also, it adds _oplog_ts and _oplog_date fields on the source document. These correspond to the timestamp from the oplog when the data changed in MongoDB. Finally, it routes the document by the MongoDB id so that you can speed up queries later to find changes to a doc.
This lets you do some cool things but mostly you'll want to sort by _oplog_date
and filter by _source_id
to see how documents have changed over time.
Because the indexes are timestamped you can drop then after a period of time so they don't take up space. If you just want the last couple of days of changes, delete the indexes with the old timestamps. Elastic curator is your friend here.
Note
This option may be passed on the command line as ./monstache --time-machine-namespace test.foo --time-machine-namespace test.bar
If specified as an environment variable the value should be namespaces separated only by the env-delimiter
which
defaults to a comma. E.g. MONSTACHE_TIME_MACHINE_NS=test.foo,test.bar
time-machine-index-prefix¶
string (default log
)
If you have enabled time machine namespaces and want to change the prefix assigned to the index names use this setting.
time-machine-index-suffix¶
string (default 2006-01-02
)
If you have enabled time machine namespaces and want to suffix the index names using a different date format use this setting. Consult the golang docs for how date formats work. By default this suffixes the index name with the year, month, and day.
time-machine-direct-reads¶
boolean (default false
)
This setting controls whether or not direct reads are added to the time machine log index. This is false by default so only changes read from the oplog are added.
tpl¶
boolean (default false
)
This option is only supported on the command line. When the tpl mode is turned on then any config file passed via -f will be interpreted and executed
as a golang template before being loaded. The template will have access to all environment variables. The environment variables will be passed as a map
to the template. The env map can be accessed as the dot .
symbol in the golang template and values from the map obtained using the index
function.
For example, the given an environment variable THRESHOLD, then with -tpl -f config.toml
the config.toml might contain...
[[script]]
namespace = "mydb.mycollection"
script = """
module.exports = function(doc) {
if ( doc.score > {{index . "THRESHOLD"}} ) {
doc.important = true;
}
return doc;
}
"""
verbose¶
boolean (default false
)
When verbose is true monstache with enable debug logging including a trace of requests to Elasticsearch
worker¶
string (default ""
) (env var name MONSTACHE_WORKER
)
When worker is given monstache will enter multi-worker mode and will require you to also provide the config option workers. Use this mode to run multiple monstache processes and distribute the work between them. In this mode monstache will ensure that each MongoDB document id always goes to the same worker and none of the other workers. See the Workers section for more information.
workers¶
[]string (default nil
)
An array of worker names to be used in conjunction with the worker option.
Note
This option may be passed on the command line as ./monstache --workers w1 --workers w2


Library: Net
Package: Reactor
Header: Poco/Net/SocketAcceptor.h
Description
This class implements the Acceptor part of the Acceptor-Connector design pattern.
The Acceptor-Connector pattern has been described in the book "Pattern Languages of Program Design 3", edited by Robert Martin, Frank Buschmann and Dirk Riehle (Addison Wesley, 1997).
The Acceptor-Connector design pattern decouples connection establishment and service initialization in a distributed system from the processing performed once a service is initialized. This decoupling is achieved with three components: Acceptors, Connectors and Service Handlers. The SocketAcceptor passively waits for connection requests (usually from a remote Connector) and establishes a connection upon arrival of a connection requests. Also, a Service Handler is initialized to process the data arriving via the connection in an application-specific way.
The SocketAcceptor sets up a ServerSocket and registers itself for a ReadableNotification, denoting an incoming connection request.
When the ServerSocket becomes readable the SocketAcceptor accepts the connection request and creates a ServiceHandler to service the connection.
The ServiceHandler class must provide a constructor that takes a StreamSocket and a SocketReactor as arguments, e.g.:
MyServiceHandler(const StreamSocket& socket, ServiceReactor& reactor)
When the ServiceHandler is done, it must destroy itself.
Subclasses can override the createServiceHandler() factory method if special steps are necessary to create a ServiceHandler object.
Member Summary
Member Functions: createServiceHandler, onAccept, reactor, registerAcceptor, setReactor, socket, unregisterAcceptor
Constructors
SocketAcceptor
explicit SocketAcceptor(
ServerSocket & socket
);
Creates a SocketAcceptor, using the given ServerSocket.
SocketAcceptor
SocketAcceptor(
ServerSocket & socket,
SocketReactor & reactor
);
Creates a SocketAcceptor, using the given ServerSocket. The SocketAcceptor registers itself with the given SocketReactor.
Destructor
~SocketAcceptor
virtual ~SocketAcceptor();
Destroys the SocketAcceptor.
Member Functions
onAccept
void onAccept(
ReadableNotification * pNotification
);
Accepts connection and creates event handler.
registerAcceptor
virtual void registerAcceptor(
SocketReactor & reactor
);
Registers the SocketAcceptor with a SocketReactor.
A subclass can override this function to e.g. register an event handler for timeout event.
If acceptor was constructed without providing reactor to it, the override of this method must either call the base class implementation or directly register the accept handler with the reactor.
setReactor
void setReactor(
SocketReactor & reactor
);
Sets the reactor for this acceptor.
unregisterAcceptor
virtual void unregisterAcceptor();
Unregisters the SocketAcceptor.
A subclass can override this function to e.g. unregister its event handler for a timeout event.
If the accept handler was registered with the reactor, the overriding method must either call the base class implementation or directly unregister the accept handler.
createServiceHandler
virtual ServiceHandler * createServiceHandler(
StreamSocket & socket
);
Create and initialize a new ServiceHandler instance.
Subclasses can override this method.
reactor
SocketReactor * reactor();
Returns a pointer to the SocketReactor where this SocketAcceptor is registered.
The pointer may be null.
socket
Socket & socket();
Returns a reference to the SocketAcceptor's socket.


UTC + 1, summer time UTC + 2 April to October.
Voltage: 230 V
Frequency: 50 Hz
Note: a C
style plug can be used with E
and F
receptacles. All double-insulated appliances are indeed fitted with a C
plug, and can be used in any compatible receptacle (C E F and narrow L). Type C receptacles are prohibited in Belgium
Euro
International dialing code for Belgium is +32.
rangeRoamer provides short-term cell phone service for international travelers. Buy, Rent, or use your own cell phone. Acquiring a OneSimCard global SIM card before arriving in Belgium is most efficient method.
Embassies in Belgium see link
Emergency telephone numbers
Medical and Fire Services - 100
Police - 101
European emergency number - 112 (calls are free from any mobile cellular of fixed-line telephone)
IMRAY AND ADLARD COLES PILOT BOOKS are available at a discounted price for Noonsite.com users via World Cruising Club
Inland Waterways of Belgium
By Jacqueline Jones
Publisher Imray, Laurie, Norie & Wilson Ltd (1st May 2005)
ISBN<PHONE_NUMBER>608
North Sea Harbours and Pilotage
By Brian Navin
Publisher Imray, Laurie, Norie & Wilson Ltd (5th Edition Dec 2008)
ISBN<PHONE_NUMBER>339 -<PHONE_NUMBER>339
The Cruising Companion to North France and Belgium
By Neville Featherstone
Publisher John Wiley & Sons (Dec 2001)
ISBN<PHONE_NUMBER>548
North Sea Passage Pilot
By Brian Navin
Publisher: Imray,Laurie,Norie & Wilson Ltd. (6th Revised edition (2013)
ISBN-13:<PHONE_NUMBER>234446
THE MINISTERE DES TRAVAUX PUBLICS has published a map of the inland waterways.
The Cruising Almanac 2015
By Cruising Association and Imray
Publisher Imray (2015 Edition)
ISBN<PHONE_NUMBER>771
Covers North West Europe from Shetland and southern Norway to Gibraltar and West Ireland to the Baltic. Fully revised and additional plans. Available as Imray Nautical Apps for iPad users. Free updates approximately every two months
Noonsite welcomes information and updates especially regarding clearance, customs and immigration procedures from cruisers visiting this country.
Please E-mail noonsite with any new information, updates or corrections. Even just a short email confirming that the current data is accurate would be most helpful.


---
url: /index
layout: index
title: An open-source append only ledger
description: Build tamper-evident services with Trillian and cryptographically prove that the data hasn’t been unexpectedly changed.
---
You need to trust the data you rely on. Your organisation handles more data every day,
and the data helps you make critical decisions. But you need to know that it hasn't been
modified or corrupted.
Building with tamper-evident logs means you can cryptographically prove that the data hasn’t been
unexpectedly changed. Knowing the provenance, integrity and sequence your data was created means
you’ll be able to monitor third-party ecosystems and simplify regulatory compliance. You can trust
the data you rely on.
A tamper-evident log stores an accurate, immutable and
verifiable history of activity.
This is an example of a verifiable system. You could use them to track credits
and debits in banking transactions, access logs to sensitive healthcare records, cryptographic hashes
of software packages, compliance artifacts of regulated activities, or modifications to a document.
Trust your data with a tamper-evident log
A tamper-evident log makes it impossible for a malicious insider to cover their tracks.
A tamper-evident log discourages malicious behaviour by increasing the chance of discovery.
}}">
Learn how you can permanently log all actions performed on your
Key Transparency protects users against malicious insiders by making key updates visible.
A tamper-evident log keeps you in control of your audit artifacts and gives auditors confidence.
Regulated industries require companies to collect and retain many types of compliance records. Auditors need to verify the integrity and nonrepudiation of those records.
By using a tamper-evident to store compliance records, you can keep them in one place and simplify presenting them to an auditor. You can cryptographically prove they haven't been tampered with.
A tamper-evident log is a more efficient way of presenting compliance records to an auditor who can easily verify their history and integrity.
}}">
Learn how you can permanently log all actions performed on your
A tamper-evident log enables multiple parties to monitor each others' actions.
Where organisations can be required to publish data about their actions, a tamper-evident log can hold a permanent record of this data.
Certificate Transparency discourages misbehaviour of Certificate Authorities by making their actions public.
Tamper-evident logs can frustrate software supply chain attacks by providing a shared, global view of the content of packages.
Often people just put everything in the log and stop there.
For a system to be valuable to you and those that rely on it, you need to identify what’s important to log and who will verify its contents.
We've designed two exercises to help you get started.
Trillian is a log that stores an accurate, immutable and verifiable history of activity. It is an open-source library and powers one of the world’s largest, most used and respected, production crypto ledger based ecosystems, certificate transparency.
Trillian is similar to a blockchain: it's an append-only log for storing records.
Trillian provides support for several }}">verifiable data structures, each offering different types of verifiability. In the case of a Merkle tree, Trillian can cryptographically prove:
As an example, Trillian could be used to create a tamper-evident financial transaction ledger.
The financial ledger app provides an API for storing, retrieving and verifying transactions. It uses Trillian to store the transactions and cryptographically prove the integrity of transactions.
}}">
Learn about Trillian's verifiable data
Trillian is designed to be integrated with existing systems, rather than requiring the ground-up rebuild that a blockchain ledger would need.
Trillian is the foundation for Certificate Transparency, one of the largest production ledgers on the internet. Performs at over 2,000 writes per second.
Released under the permissive Apache 2.0 licence. You have the freedom to use Trillian on your own terms. Benefit from the support of Google Cloud Platform, or deploy it yourself on-premise or cross cloud.
Google developed the most challenging parts of the technology and maths underpinning Trillian. They continue to make engineering improvements to ensure scalability and availability.


QUIET AMERICAN, THE (director/writer: Joseph L. Mankiewicz; screenwriter: from a book by Graham Greene; cinematographer: Robert Krasker; editor: William W. Hornbeck; cast: Audie Murphy (The American, Alden Pyle), Michael Redgrave (Thomas Fowler), Claude Dauphin (Inspector Vigot), Giorgia Moll (Phuong), Kerima (Phuong's Sister), Bruce Cabot (Bill Granger), Richard Loo (Mr. Heng), Fred Sadoff (Dominguez), Georges Bréhat (French Colonel); Runtime: 120; United Artists/Figaro; 1958)
Reviewed by Dennis Schwartz
The Quiet American is loosely adapted from Graham Greene's penetrating 1956 book about the Indo-China War, where he blames America for foolishly interfering in a war it doesn't know. He was proved right in later historical events. Director Joseph L. Mankiewicz switched Greene's anti-American premise around and made it an anti-Communism theme, which was to the detriment of the film as it seemed to lose its potent messages. Mankiewicz turns it into a feeble murder mystery and an uninvolving romantic love triangle, which is told in flashback. What kept the film lively, was Michael Redgrave as a middle-aged British war correspondent, Thomas Fowler, who is an embittered intellectual neurotic not believing in religion or in war or peace efforts but only in keeping his mistress for his personal pleasure. The film was shot in magnificent shades of black and white, and it was filmed on location in Vietnam giving it a look of authencity.
A quiet, enigmatic American, Alden Pyle (Murphy), arrived in Saigon in 1952, on a private economic mission to help provide aid to that region at war. He is a naive humanitarian, self-righteous, priggish, do-gooder, believing in his idea of a Third Force to be a neutral buffer between Communism and French colonialism forces. Vietnam had an emperor chosen by the French, and they were fighting as partners with the French the Communists in the north. Among the 22 million Vietnamese, there were 2 million who belonged to a new religion called Cao-Dai. Their religion was a mixture of Buddhism and Christianity; they had a pope; a Disneylike temple with pictures of the eye of God, snakes and dragons; and, they remained neutral in the fighting. There was a general who broke away from following them completely and committed terrorist acts, and he had an army of 25,000 at his disposal.
 She used to be Fowler's girlfriend.
In flashback, the film tells the quiet American's story through the eyes of the British reporter. The married Fowler is dining with his beautiful, much younger mistress, Phuong (Moll), who loves milk shakes and whose name means feelings. She formerly worked in the Rendez-vous Club as an escort to men, that is where Fowler met her. Like many of her countrymen, she does what she has to do to survive.
When Pyle comes into the restaurant, he meets the couple and they can't shake him. He immediately falls in love with Phuong and tells Pyle he will fairly compete for her. The cynical Fowler is soon sent to cover the war in the north, and Pyle who is always chaperoned by Phuong's older sister (Kerima), escorts her around Saigon.
Pyle surprisingly visits Fowler in the front to deliver him a cablegram at the command post of the French colonel (Bréhat), who reads poetry as an escape from his bitter reality of fighting a losing 20th century war with 19th century methods. Fowler learns that he's been promoted to be the foreign editor and must report back to London. He fears that he will lose Phuong, so he writes his pious Episcopalian wife for a divorce. When he receives her reply of no for religious reasons, he tells Phuong and Pyle the reverse. But when Phuong, who can't speak English, takes the letter to her sister for a translation and learns the truth she leaves him for Pyle--which breaks Fowler's heart.
Two Communists, Dominguez and Heng, work to manipulate the self-pitying Fowler against Pyle. They suggest that the plastics he says he is using for toys, is really being used to make bombs. On a visit to a religious temple, both their cars are tampered with and they are stuck on a road in which the Communists attack at night. Staying with two frightened Vietnamese soldiers in a watchtower they survive the attack by fleeing to the rice fields with Pyle acting as a hero, by carrying the injured Fowler to safety after the attack.
In Saigon, the tormented Pyle witnesses a terrorist bombing where a number of innocent people lose their lives, as Heng convinces him without substantial proof that Pyle's bombs caused the blast. Heng then convinces him to betray Pyle, as someone who is selling munitions to the enemies of the Communists.
Inspector Vigot (Dauphin) determines that Fowler didn't kill Pyle, but arrests Heng and his henchmen as belonging to a group of Communist assassins. He accuses Fowler of being used by that group because he was weak, afraid of losing his girlfriend to Pyle and being left alone. The inspector tells him that he will have to live with that guilt. He will also have to live without his mistress, as she returns to work as an escort at her former club and refuses to see him.
REVIEWED ON 8/26/2001 GRADE: C +
Dennis Schwartz: "Ozus' World Movie Reviews"
http://www.sover.net/~ozus
<EMAIL_ADDRESS>© ALL RIGHTS RESERVED DENNIS SCHWARTZ
========== X-RAMR-ID: 29344 X-Language: en X-RT-SourceID: 873 X-RT-AuthorID: 1315 X-RT-RatingText: C+
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Root Motion
By default a skeletal animation has its origin at the position of the game object on which it is played. Relative to that location animations will move the bones and the skinned mesh will move accordingly. The game object itself stays fixed at that location, though.
This is sufficient if either the game object shouldn't change its location anyway, or when any change in position is controlled through other means anyway. For example a player character might be moved around the world through custom code and a walking animation is only played to visualize the action. This approach can be the right solution, depending on the type of game.
Such a method is, however, very prone to foot sliding, meaning an artifact where the feet move, but don't stick to the ground. It the movement of a game object should generally be determined by the exact blend of animation clips, it is better to have the motion be part of each animation clip.
For example a walk animations would contain the information into which direction and at what speed a game object should be moved to fit the animation. When a forward and walk right animation get mixed together, their root motion information is equally mixed and the object would be moved diagonally.
Defining Root Motion
There are multiple ways how root motion could be defined for a clip. It could come from a dedicated bone for overall motion, or it could be extracted from how the feet touch the ground, etc.
For the time being EZ only implements the most simple method. An animation clip either has no root motion at all, or it has a constant motion that is used for the entire clip. This is sufficient to build basic locomotion animations.
Finally, for now only positional root motion is available. That means an animation can change the position of a game object, but not its rotation.
It is planned to add more sophisticated methods for root motion in the future.
Applying Root Motion
The simple animation component and the animation controller component get the root motion data from the played animation clips. There are these modes to apply it to their owner game object:
Ignore
: No root motion is applied, the game object will not be moved by the animation.ApplyToOwner
: Any available root motion is directly applied to the game object and thus moves it without restriction. This mode is useful for objects that have to follow a fixed path. For example moving platforms (which are kinematic physics actors), or objects that don't physically interact with the player. This mode is not suited for characters that should obey physical restrictions.SendMoveCharacterMsg
: If this mode is used, root motion is not applied to any object, instead the messageezMsgMoveCharacterController
is sent to the top most game object in the hierarchy. This way, if there is also a character controller or other component that accepts this type of message, it can apply the root motion as it sees fit.


Noé Roman
- Idea Engine
- Compliant
- Team Player
- Great Artist
- Natural Born Leader
- Computer Savy
- Proficient in Photoshop
- Proficient in illustrator
- Solid foundation to HTML/CSS
- Really good in teams
- Good Ground in IT
- Can Crimp Cables
- Take apart and reassemble computers
- Really good at Explaining
- Corovan IT/facilities Supervisor (August 2015 - Present)
- Lead a team of people
- Executed projects up to 1,200 user move
- Communicated with clients in order to get the job done
- Worked along side my team instead of bossing them around
- DevMission Debug (June 2017 - Present)
- Learned to break apart and reassemble computers
- Able to identify all the parts of a computer
- Learned to develop a website using HTML and CSS


I'm working with XWiki to create an importer for Dokuwiki projects to XWiki, ensuring consistency and flexibility. I'll be building upon and improving XWiki's FilterStream framework for this project.
Amazon Internet Services private limited is the Indian entity for Amazon Web Services in India. I've worked with Business development, sales and the solutions architect teams. I've worked with dozens of startups and enterprises to create scalable, cost effective, fault tolerant, and powerful systems. I've also completed all 5 AWS certifications during my internship.
We built an end-to-end platform for selling the 'Next' brand appliances. The startup handled the online business for Next appliances. Apart from the primary business, we focused more on creating interesting technology, most notable one was: A web service that brings apple's continuity features (including calling) to the masses.
I worked on creating a Point of Sale (POS) system, using some great javascript libraries, mongoDB, deployed it to a raspberry pi. The goal was to make it end to end, and light enough to support raspberry pi as the central server.
I tested and benchmarked various enterprise applications on variety of platforms. The test ensured validity of various SLAs.


Teacher Quiz Dashboard
New Quiz
Available Quizzes
Courses
Quiz Title
Course Code
Session
Start Time
End Time
Status
View
Controller
Database Quiz Exam 3
CSE333
2011-12
5-Sat-15
10 Sat-15
Unknown
View
Show
Course Title
Course Code
Credit
Semester
Session
Database
CSE333
3.0
3-1
2011


Fury in Spain as Gareth Bale holds 'inflammatory' flag after Wales qualify
The Real Madrid forward appeared to mock his club after holding a flag with the message "Wales. Golf. Madrid. In that order" following his country's win over Hungary
by Alex MilneThe Spanish media have reacted with anger after Gareth Bale was pictured holding a flag with the words "Wales. Golf. Madrid. In that order."
Bale helped his country qualify for Euro 2020 after Wales beat Hungary 2-0 to secure their passage through to next summer's tournament.
After the game Bale joined in the celebrations with his team and was seen holding the flag, which mocks the La Liga giants with the claim that they are only third in Bale's list of priorities.
Spanish publication Madrid Sports tweeted a video of the incident along with the caption: "We have always defended you, Gareth Bale.
"We have had patience with your injuries, we have celebrated your goals and assists, and we have defended your passion for golf and your limited Spanish.
"But enough is enough. As it says on the flag, you are laughing at Madrid."
Spanish journalist Ruben Martin , meanwhile, tweeted: "He is laughing at Madrid. At their fans and his teammates. He doesn't care. He doesn't deserve to wear the shirt.
"And he shouldn't wear it again. Ever."
And fellow journalist Manu Sainz added: "Bale knows he will only be able to leave Madrid in January if he forces a move, but he is getting out of hand."
Bale has only featured in six La Liga games all season, and was set to leave last summer, only for the club to deny him a move to China at the last minute.
He recently admitted he prefers playing for the national team than for Madrid, saying: "It's like playing with your mates down the park on a Sunday.
"It's normal- with Wales, I'm speaking my own language and feel more comfortable. I definitely have a bit more excitement playing with Wales."
It remains to be seen whether Bale is involved when Madrid take on Real Sociedad in La Liga on Saturday.


Arabella
Caption 1
Caption 2
Caption 3
I am looking for a gift for my
Father
Mother
Brother
Sister
Friend
Age is
Less than 10yrs
Between 10yrs to 18yrs
18yrs to 30yrs
30yrs to 55yrs
55+yrs
in price range
<100 INR
100-500 INR
500-1000 INR
1000+ INR
Priceless
Submit


MARS - Mips Assembly and Runtime Simulator
Release 4.5
August 2014
Operating Requirements
MARS is written in Java and
requires at least Release 1.5 of the J2SE Java Runtime Environment (JRE) to work.
The graphical IDE is implemented using Swing. It has been
tested on Windows XP, Vista and 7; Mac OS X; and is also being used under Linux.
Some MARS Assembler and Simulator Limitations
Releases 3.0 and later assemble and simulate nearly all the MIPS32 instructions documented in the
textbook Computer Organization and Design, Fourth Edition by Patterson and Hennessy,
Elsevier - Morgan Kaufmann, 2009. All basic and pseudo instructions, directives,
and system services described in Appendix B are implemented.
Limitations of MARS as of Release 4.5 include:
- Memory segments (text, data, stack, kernel text, kernel data) are limited to 4MB each starting at their
respective base addresses.
- There is no pipelined mode (but delayed branching is supported).
- If you open a file which is a link or shortcut to another file, MARS will not
open the target file. The file open dialog is implemented using Java Swing's JFileChooser,
which does not support links.
- Very few configuration changes, besides those in the Settings menu, are saved from one session to the next.
The editor settings, which include font settings and display of line numbers, are saved.
- The IDE will work only with the MARS assembler. It cannot be used
with any other compiler, assembler, or simulator. The MARS assembler and simulator
can be used either through the IDE or from a command prompt.
- Bug:The error message highlighter does not automatically select the code for the first assembly
error if the file containing the error is not open at the time of assembly (assemble-on-open, assemble-all).
- Bug: The Screen Magnifier screen capture feature does not appear to work properly under Windows Vista.
- Bug: There appears to be a memory leak in the Editor. Several different people have independently
reported the same behavior: severe slowdown in editor response during an extended interactive session.
If MARS is exited and restarted, this behavior disappears and the editor responds instantly to actions.
- Not a bug, but documented here anyway: MIPS Branch instruction target addresses are represented by
the relative number of words to branch. With Release 3.4, this value reflects delayed branching,
regardless of whether the Delayed Branching setting is enabled or not.
The generated binary code for branches will now match that of examples in the Computer Organization
and Design textbook. This is a change from the past, and was made after extensive discussions
with several faculty adopters of MARS. Previously, the branch offset was 1 lower if the Delayed Branching setting
was enabled -- the instruction label: beq $0,$0,label would generate 0x1000FFFF if
Delayed Branching was enabled and 0x10000000 if it was disabled. Now it will generate 0x1000FFFF in
either case. The simulator will always branch to the correct location; MARS does not allow assembly under one
setting and simulation under the other to occur.
This document is available for printing on the MARS home page
http://www.cs.missouristate.edu/MARS/.


Installation¶
Nemaktis
is a mixture of C++ and python codes, and have been successfully tested both on
Windows and Linux (note that it should be straightforward to adapt it for Mac, since it is
a Unix-based OS). The next two subsections present the two possible ways of installing this
package.
With Conda (windows or linux)¶
The simplest way of installing the Nemaktis
package is through the package manager
conda
(no compilation and no dependency to be installed by hand). I provide precompiled
packages for both Linux and Windows, which means the following method will work on these two
operating systems.
The first step is to install the software miniconda
, which contains a barebone version of
the package manager conda
. If you already have the full python distribution Anaconda
installed on your machine, this step is not necessary. The installation files for Windows/Linux
are available at this address: https://docs.conda.io/en/latest/miniconda.html
If you don’t want to copy-paste commands in a terminal, the next step is as simple as running the following installation script:
Windows: https://github.com/warthan07/Nemaktis/releases/download/v1.0/Install.Nemaktis.cmd
Linux: soon
Both scripts will create a special environment for Nemaktis
named nm and will install
everything needed in it. The windows script will also install the python editor Spyder
and create a shortcut named Spyder (Nemakis environment) for it on your Desktop (this is
necessary even if you already installed Spyder, since it has to be run from inside the conda
environment nm).
If you want to type the installation commands yourselves (they are not very complicated after all!), open a terminal (Windows: application “Conda terminal” installed with miniconda, Linux: any terminal) and type the following command:
conda create -n nm -c conda-forge -c warthan07 -y nemaktis
(Optional) If you want to use your favourite python editor when using Nemaktis
, you have
to install and run it from the same environment. You can search https://anaconda.org/ to find
the associated package and installation command. For example, to install Spyder
you just
need to type:
conda activate nm
conda install spyder
Developper method (only linux)¶
If you want to be able to modify the code, but still want to enjoy the simplicity of conda packages (no relative paths to manage, everything works as with a system package), you can build yourselves the nemaktis package for Linux:
Get the code of the
Nemaktis
repository:git clone git@github.com:warthan07/Nemaktis.git
And implements the changes that you want. For the C++ codes, compilation steps are provided in the subfolders bpm-solver and rt-solver if you want to test them locally (in which case you will have to install yourselves the dependencies listed in each CMakeLists).
In a terminal, go to the subfolder conda_recipe of the
Nemaktis
repository and activate any conda environment in which you have write access. If you don’t have any conda environment yet, you can type:conda create -n build conda activate build
If necessary, install the conda-build tools:
conda install conda-build conda-verify
Run the following command, which will create a sub-environment, install all dependencies listed in meta.yaml, and compile/package everything (it should take between 5 and 10 minutes):
conda-build . -c conda-forge
Once the package is built, you can install it in your current environment by typing:
conda install -c anaconda -c conda-forge -c ${CONDA_PREFIX}/conda-bld/ nemaktis


The interview is below:
A volunteer-led organization that has raised hundreds of thousands of dollars to buy and pay off medical debts as a gift to debtors says it intends to seek IRS recognition as a section 501(c)(4) tax-exempt organization.
The Rolling Jubilee Fund works to educate the public about consumer credit — how and when it is advanced, how it is bought and sold, and how it’s collected, three attorneys representing the fund on a pro bono basis told Tax Analysts. The fund also furthers social welfare by helping patients with outstanding medical debt, the attorneys said.
“For centuries, charitable institutions have attempted to relieve the burdens of the sick,” the attorneys said. “And hundreds of tax-exempt organizations educate the public about debt and how to relieve debt. We are not aware of any organization that has combined these two objectives in exactly the way that Rolling Jubilee does, but we don’t believe that there is any reason such a combination should not be considered an exempt purpose that promotes social welfare.”
The organization hasn’t yet submitted its exemption application, and no conversations about it have taken place with IRS officials, the attorneys said. Rolling Jubilee is run by volunteers, many of whom were involved in the Occupy Wall Street movement and face debt burdens themselves, the attorneys said.
The three attorneys advising the group are David S. Miller of Cadwalader, Wickersham & Taft LLP, Deborah Hrbek of Hrbek Law LLC, whose practice focuses on entertainment and small business law, and Margaret Ratner Kunstler, a civil rights attorney with Hrbek Law.
According to the fund’s website, more than $560,000 has been raised to retire more than $11 million in debt, but the attorneys said the organization has so far bought and forgiven $100,000 in debt. (See http://rollingjubilee.org/.)
Rolling Jubilee has discovered that although patients unable to pay their bills are subject to collection agency demands to pay them all off, that debt can be bought for pennies on the dollar, the lawyers said. The fund is doing that and forgiving the debt to educate the country about the debt market, the lawyers said, adding that the fund may later acquire nonmedical debt of debtors in impoverished areas.
“Rolling Jubilee plans only to forgive debt when that forgiveness promotes social welfare,” the attorneys said.
Exempt Purpose
Rolling Jubilee is “an interesting concept,” and the fund’s stated goal of educating the public about Americans going bankrupt when they get sick seems well targeted at boosting the country’s social welfare, said Ofer Lion of Hunton & Williams LLP. That, he said, is an exempt purpose for a 501(c)(4) organization.“Their actual methods don’t seem to run counter to charitable purposes either, as I understand that they wipe out burdensome medical-related debts — ‘relief of the distressed,’ in Treasury regulations parlance,” Lion said.Lloyd Hitoshi Mayer of the University of Notre Dame Law School said there’s “at least a reasonably strong argument” for why the Rolling Jubilee effort generally furthers an exempt purpose. There are two elements of the argument, and the first is that standards for 501(c)(4) organizations are looser than those for 501(c)(3) groups, Mayer said.
“The second thing is I think you could argue that while it’s true they don’t do means testing of the people that benefit from this because it’s practically impossible, the debt they’re buying is highly distressed debt,” Mayer said.
According to Mayer, that could mean that most people holding that debt are in a precarious financial position; otherwise, the debt wouldn’t be available for purchase at such discounts. The same may not be true with other types of debt, he said.
“Mortgages I think are actually a little bit of a harder case because you have someone who is pretty well off, but their house is underwater and maybe the foreclosure process is difficult, and so the debt is being discounted not because the individuals are particularly bad off financially, but just because your chance of actually collecting on the debt is relatively small,” Mayer said.
A Detached and Disinterested Gift?
Miller, Kunstler, and Hrbek told Tax Analysts that forgiveness of the debt does not result in income to the debtor if that forgiveness comes from a detached and disinterested generosity. Lion said these debtors seem to have a good case that they’re receiving an exempt gift rather than a taxable gain.Mayer agreed, saying the debt forgiveness counts as a gift under section 102. He said that when Rolling Jubilee buys the debt, it doesn’t know who the beneficiaries are but discharges the debt purely out of a desire to benefit those who incurred it.“It seems to be that’s about as detached and disinterested as you can get,” Mayer said. “The one weird thing is usually you don’t think of entities being detached and disinterested givers — you just think of individuals.”
Robert Willens of Robert Willens LLC said he believes the IRS would resist the characterization of these cancellations of debt as gifts but that they’ll “get lost” in the reporting process anyway.
“Since there’s no way for the IRS to verify all this, there won’t be much enforcement on their part,” Willens said. “That’s not the way we like to do tax planning, but I guess it could work.”
Mayer said he doesn’t expect immediate IRS movement regarding any aspect of the Rolling Jubilee program. Unless the dollar volume grows higher or there’s an indication of someone with Rolling Jubilee personally profiting, it won’t be a high priority for the IRS, he said.
Mayer said he doesn’t think the IRS would act “until at least an initial [Form] 990 is filed,” adding, “Even then, of course, the IRS has three years at minimum to audit the 990, and they may or may not bother to do so depending on their other priorities.”