bucardo - utility script for controlling the Bucardo program
This document describes version 5.6.0 of bucardo
bucardo [<options>] <command> [<action>] [<command-options>] [<command-params>]
The bucardo script is the main interaction to a running Bucardo instance. It can be used to start and stop Bucardo, add new items, kick syncs, and even install and upgrade Bucardo itself. For more complete documentation, please view the wiki.
Run bucardo help <command>
for additional details
install
Installs the Bucardo configuration database.
upgrade
Upgrades the Bucardo configuration database to the latest schema.
start [<start options>] [<reason>]
Starts Bucardo.
stop [<reason>]
Stops Bucardo.
restart [<start options>] [<reason>]
Stops and starts Bucardo.
list <type> [<regex>]
Lists objects managed by Bucardo.
add <type> <name> <parameters>
Adds a new object.
update <type> <name> <parameters>
Updates an object.
remove <type> <name> [<name>...]
Removes one or more objects.
kick <syncname> [<sync options>] [<syncname>...] [<timeout>]
Kicks off one or more syncs.
reload config
Sends a message to all CTL and KID processes asking them to reload the Bucardo configuration.
reopen
Sends a message to all Bucardo processes asking them to reopen any log files they may have open. Call this after you have rotated the log file(s).
show all|<setting> [<setting>...]
Shows the current Bucardo settings.
<set <setting=value
[<setting=value>...] >>Sets one or more configuration setting..
ping [<timeout>]
Sends a ping notice to the MCP process to see if it will respond.
status [<status options>] <syncname> [<syncname>...]
Shows the brief status of syncs in a tabular format.
activate <syncname> [<syncname>...] [<timeout>]
Activates one or more named syncs.
deactivate <syncname> [<syncname>...] [<timeout>]
Deactivates one or more named syncs.
message '<body>'
Sends a message to the running Bucardo logs.
reload [<syncname> [<syncname>...]]
Sends a message to one or more sync processes, instructing them to reload.
inspect <type> <name> [<name>...]
Inspects one or more objects of a particular type.
validate all|<syncname> [<syncname>...]
Validates one or more syncs.
purge all|<table> [<table>...]
Purges the delta and track tables for one or more tables, for one or more databases.
delta [<database(s)>]
Show the delta counts for each source target.
help [<command> [<action>]]
Shows help.
-d --db-name NAME Database name.
-U --db-user USER Database user name.
-P --db-pass PASS Database password.
-h --db-host HOST Database server host name.
-p --db-port PORT Database server port number.
--bucardorc FILE Use specified .bucardorc file.
--no-bucardorc Do not use .bucardorc file.
--batch Enable batch mode: don't ask questions or validate inputs
--quiet Incremental quiet.
--verbose Incremental verbose mode.
-? --help Output basic help and exit.
--version Print the version number and exit.
--dryrun Do not perform any actual actions.
--confirm Require direct confirmation before changes.
Most of the commands take parameters. These may be passed after the command name and, where appropriate, an object name. Parameters take the form of key/value pairs separated by an equal sign (=
). For example:
bucardo add db sea_widgets dbname=widgets host=db.example.com
Here dbname
and <host> are parameters.
Many of the commands also use command-line options, which are specified in the normal way. For example, the bucardo add db
command could also be written as:
bucardo add db sea_widgets --dbname widgets --dbhost db.example.com
However, parameters and options are not directly interchangeable in all cases. See the documentation for individual commands for their supported options.
bucardo install
Installs the Bucardo schema from the file bucardo.schema into an existing Postgres cluster. The user "bucardo" and database "bucardo" will be created first as needed. This is an interactive installer, but you can supply the following values from the command line:
--dbuser
defaults to postgres
--dbname
defaults to postgres
--dbport
defaults to 5432
--pid-dir
defaults to /var/run/bucardo/
bucardo upgrade
Upgrades an existing Bucardo installation to the current version of the bucardo database script. Requires that bucardo and the bucardo.schema file be the same version. All changes should be backwards compatible, but you may need to re-validate existing scripts to make sure changes get propagated to all databases.
bucardo start "Reason"
Starts Bucardo. Fails if the MCP process is running (determined if its PID file is present). Otherwise, starts cleanly by first issuing the equivalent of a stop to ask any existing Bucardo processes to exit, and then starting a new Bucardo MCP process. A short reason and name should be provided - these are written to the reason_file
file (./bucardo.restart.reason.txt by default) and sent in the email sent when Bucardo has been started up. It is also appended to the reason log, which has the same name as the the reason_file
but ends in .log.
The options for the start
command are:
--sendmail
Tells Bucardo whether or not to send mail on interesting events: startup, shutdown, and errors. Default is on.
--extra-name string
A short string that will be appended to the version string as output by the Bucardo process names. Mostly useful for debugging.
--log-destination destination
Determines the destination for logging output. The supported values are:
stderr
stdout
syslog
none
May be specified more than once, which is useful for, e.g., logging both to a directory and to syslog. If --log-destination
is not specified at all, the default is to log to files in /var/log/bucardo.
--log-separate
Forces creation of separate log files for each Bucardo process of the form "log.bucardo.X.Y", where X is the type of process (MCP, CTL, or KID), and Y is the process ID.
--log-extension string
Appends the given string to the end of the default log file name, log.bucardo. A dot is added before the name as well, so a log extension of "rootdb" would produce a log file named log.bucardo.rootdb.
--log-clean
Forces removal of all old log files before running.
--debug
--no-debug
Enable or disable debugging output. Disabled by default.
--exit-on-nosync
--no-exit-on-nosync
On startup, if Bucardo finds no active syncs, it normally will continue to run, requiring a restart once syncs are added. This is useful for startup scripts and whatnot.
If, however, you want it to exit when there are no active syncs, pass the --exit-on-nosync
option. You can also be explicit that it should not exit when there are no syncs by passing --no-exit-on-nosync
. This is the default value.
bucardo stop "Reason"
Forces Bucardo to quit by creating a stop file which all MCP, CTL, and KID processes should detect and cause them to exit. Note that active syncs will not exit right away, as they will not look for the stop file until they have finished their current run. Typically, you should scan the list of processes after running this program to make sure that all Bucardo processes have stopped. One should also provide a reason for issuing the stop - usually this is a short explanation and your name. This is written to the reason_file
file (./bucardo.restart.reason.txt by default) and is also used by Bucardo when it exits and sends out mail about its death. It is also appended to the reason log, which has the same name as the the reason_file
but ends in .log.
bucardo restart "Reason"
Stops bucardo, waits for the stop to complete, and then starts it again. Supports the same options as start
. Useful for start scripts. For getting just CTL and KID processes to recognize newly added, updated, or removed objects, use the reload
command, instead.
bucardo list <type> <regex>
Lists summary information about Bucardo objects. The supported types are:
database
dbgroup
relgroup
sync
table
sequence
customcode
customname
customcols
all
The all
option will list information about all object types.
The optional regex
option can be used to filter the list to only those matching a regular expression.
bucardo add <type> <name> <parameters>
Adds a new object to Bucardo. The type
specifies the type of object to add, while the name
should be the name of the object. The supported types include:
db
dbgroup
table
sequence
all tables
all sequences
relgroup
sync
customname
customcols
bucardo add db <name> dbname=actual_name port=xxx host=xxx user=xxx
Adds one or more new databases. The name
is the name by which the database will be known to Bucardo, and must be unique. This may vary from the actual database name, as multiple hosts might have databases with the same name. Multiple databases can be added by separating the names with commas. Options that differ between the databases should be separated by a matching commas. Example:
bucardo add db alpha,beta dbname=sales host=aa,bb user=bucardo
This command will attempt an immediate test connection to the added database(s). The supported named parameters are:
dbname
The actual name of the database. Required unless using a service file or setting it via dbdsn.
type
The type of the database. Defaults to postgres
. Currently supported values are:
postgres
drizzle
mongo
mysql
maria
oracle
redis
sqlite
dbdsn
A direct DSN to connect to a database. Will override all other connection options if set.
user
The username Bucardo should use when connecting to this database.
pass
The password Bucardo should use when connecting to this database. It is recommended that you use a .pgpass file rather than entering the password here.
host
The host Bucardo should use when connecting to this database. Defaults to the value of the $PGHOSTADDR
or $PGHOST
environment variables, if present.
port
The port Bucardo should use when connecting to this database. Defaults to the value of the $PGPORT
environment variable, if present.
conn
Additional connection parameters, e.g. sslmode=require
.
service
The service name Bucardo should use when connecting to this database.
status
Initial status of this database. Defaults to "active" but can be set to "inactive".
dbgroup
Name of the database group this database should belong to.
addalltables
Automatically add all tables from this database.
addallsequences
Automatically add all sequences from this database.
server_side_prepares
ssp
Set to 1 or 0 to enable or disable server-side prepares. Defaults to 1.
makedelta
Set to 1 or 0 to enable or disable makedelta. Defaults to 0.
Additional parameters:
--force
Forces the database to be added without running a connection test.
Note: As a convenience, if the dbuser
value is its default value, "bucardo", in the event that Bucardo cannot connect to the database, it will try connecting as "postgres" and create a superuser named "bucardo". This is to make things easier for folks getting started with Bucardo, but will not work if it cannot connect as "postgres", or if it the connection failed due to an authentication failure.
bucardo add dbgroup name db1:source db2:source db3:target ...
Adds one or more databases to the named dbgroup. If the dbgroup doesn't exist, it will be created. The database parameters should specify their roles, either "source" or "target".
bucardo add table [schema].table db=actual_db_name
Adds a table object. The table information will be read from the specified database. Supported parameters:
db
The name of the database from which to read the table information. Should be a name known to Bucardo, thanks to a previous call to add database
. Required.
autokick
Boolean indicating whether or not the table should automatically send kick messages when it's modified. Overrides the autokick
parameter of any syncs of which the table is a part.
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync. Off by default. Optional.
analyze_after_copy
Boolean indicating whether or not to analyze the table after every sync. Off by default. Optional.
vacuum_after_copy
Boolean indicating whether or not to vacuum the table after every sync. Off by default. Optional.
relgroup
Adds the table to the named relgroup. If the relgroup does not exist, it will be created. Optional.
makedelta
Turns makedelta magic on or off. Value is a list of databases which need makedelta for this table. Value can also be "on" to enable makedelta for all databases. Defaults to "off".
strict_checking
Boolean indicating whether or not to be strict when comparing the table between syncs. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable strict_checking
and column differences will result in warnings rather than failing the validation. Defaults to true.
bucardo add sequence [schema].sequence relgroup=xxx
db
The name of the database from which to read the sequence information. Should be a name known to Bucardo, thanks to a previous call to add database
. Required.
relgroup
Adds the sequence to the named relgroup. If the relgroup does not exist, it will be created. Optional.
bucardo add all tables [relgroup=xxx] [pkonly]
Adds all the tables in all known databases or in a specified database. Excludes tables in the pg_catalog
, information_schema
, and bucardo
schemas. (Yes, this means that you cannot replicate the Bucardo configuration database using Bucardo. Sorry about that.) Supported options and parameters:
db
--db
Name of the database from which to find all the tables to add. If not provided, tables will be added from all known databases.
schema
--schema
-n
Limit to the tables in the specified comma-delimited list of schemas. The options may be specified more than once.
exclude-schema
--exclude-schema
-N
Exclude tables in the specified comma-delimited list of schemas. The options may be specified more than once.
table
--table
-t
Limit to the specified tables. The options may be specified more than once.
exclude-table
--exclude-table
-T
Exclude the specified tables. The options may be specified more than once.
relgroup
--relgroup
Name of the relgroup to which to add new tables.
pkonly
Exclude tables without primary keys.
bucardo add all sequences relgroup=xxx
Adds all the sequences in all known databases or in a specified database. Excludes sequences in the pg_catalog
, information_schema
, and bucardo
schemas. (Yes, this means that you cannot replicate the Bucardo configuration database using Bucardo. Sorry about that.) Supported options and parameters:
db
--db
Name of the database from which to find all the sequences to add. If not provided, sequences will be added from all known databases.
schema
--schema
-n
Limit to the sequences in the specified comma-delimited list of schemas. The options may be specified more than once.
exclude-schema
--exclude-schema
-N
Exclude sequences in the specified comma-delimited list of schemas. The options may be specified more than once.
relgroup
--relgroup
Name of the relgroup to which to add new tables or sequences.
bucardo add relgroup name
bucardo add relgroup name table, sequence, ...
Adds a relgroup. After the name, pass in an optional list of tables and/or sequences and they will be added to the group.
bucardo add sync name relgroup=xxx dbs=xxx
Adds a sync, which is a named replication event containing information about what to replicate from where to where. The supported parameters are:
dbs
The name of a dbgroup or comma-delimited list of databases. All of the specified databases will be synchronized. Required.
dbgroup
The name of a dbgroup. All of the databases within this group will be part of the sync. If the dbgroup does not exists and a separate list of databases is given, the group will be created and populated.
relgroup
The name of a relgroup to synchronize. All of the tables and/or sequences in the relgroup will be synchronized. Required unless tables
is specified.
tables
List of tables to add to the sync. This implicitly creates a relgroup with the same name as the sync. Required unless relgroup
is specified.
status
Indicates whether or not the sync is active. Must be either "active" or "inactive". Defaults to "active".
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync. Defaults to off.
lifetime
Number of seconds a KID can live before being reaped. No limit by default.
maxkicks
Number of times a KID may be kicked before being reaped. No limit by default.
conflict_strategy
The conflict resolution strategy to use in the sync. Supported values:
bucardo_source
The rows on the "source" database always "win". In other words, in a conflict, Bucardo copies rows from source to target.
bucardo_target
The rows on the "target" database always win.
bucardo_skip
Any conflicting rows are simply not replicated. Not recommended for most cases.
bucardo_random
Each database has an equal chance of winning each time. This is the default.
bucardo_latest
The row that was most recently changed wins.
bucardo_abort
The sync is aborted on a conflict.
onetimecopy
Determines whether or not a sync should switch to a full copy mode for a single run. Supported values are:
stayalive
Boolean indicating whether or not the sync processes (CTL) should be persistent. Defaults to false.
kidsalive
Boolean indicating whether or not the sync child processes (KID) should be persistent. Defaults to false.
autokick
Boolean indicating whether or not tables in the sync should automatically send kick messages when they're modified. May be overridden by the autokick
parameter of individual tables.
checktime
An interval specifying the maximum time a sync should go before being kicked. Useful for busy systems where you don't want the overhead of notify triggers.
priority
An integer indicating the priority of the sync. Lower numbers are higher priority. Currently used only for display purposes.
analyze_after_copy
Boolean indicating whether or not to analyze tables after every sync. Off by default. Optional.
overdue
An interval specifying the amount of time after which the sync has not run that it should be considered overdue. check_bucardo_sync
issues a warning when a sync has not been run in this amount of time.
expired
An interval specifying the amount of time after which the sync has not run that it should be considered expired. check_bucardo_sync
issues a critical message when a sync has not been run in this amount of time.
track_rates
Boolean indicating whether or not to track synchronization rates.
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync. Off by default. Optional.
strict_checking
Boolean indicating whether or not to be strict when comparing tables in the sync. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable strict_checking
and column differences will result in warnings rather than failing the validation. Defaults to true.
bucardo add customname oldname newname [db=name] [sync=name]
Creates a new Bucardo custom name mapping. This allows the tables involved in replication to have different names on different databases. The oldname
must contain the schema as well as the table name (if the source database supports schemas). The optional parameters limit it to one or more databases, and/or to one or more syncs. Supported parameters:
sync
A sync to which to add the customname. May be specified multiple times.
database
db
A database for which to add the customname. May be specified multiple times.
bucardo add customcols tablename select_clause [sync=x db=x]
Specify the list of columns to select from when syncing. Rather than the default SELECT *
behavior, you can specify any columns you want, including the use of function call return values and things not in the source column list. The optional parameters limit it to one or more databases, and/or to one or more syncs. Some examples:
bucardo add customcols public.foobar "select a, b, c"
bucardo add customcols public.foobar "select a, upper(b) AS b, c" db=foo
bucardo add customcols public.foobar "select a, b, c" db=foo sync=abc
Supported parameters:
sync
A sync to which to add the customcols. May be specified multiple times.
database
db
A database for which to add the customcols. May be specified multiple times.
bucardo add customcode <name> <whenrun=value> <src_code=filename> [optional information]
Adds a customcode, which is a Perl subroutine that can be run at certain points in the sync process. It might handle exceptions, handle conflicts, or just run at certain times with no expectation of functionality (e.g., before Bucardo drops triggers). Metadata about that point will be passed to the subroutine as a hash reference.
Supported parameters:
name
The name of the custom code object.
about
A short description of the custom code.
whenrun
when_run
A string indicating when the custom code should be run. Supported values include:
before_txn
before_check_rows
before_trigger_drop
after_trigger_drop
after_table_sync
exception
conflict
before_trigger_enable
after_trigger_enable
after_txn
before_sync
after_sync
getdbh
Boolean indicating whether or not Perl DBI database handles should be provided to the custom code subroutine. If true, database handles will be provided under the dbh
key of the hash reference passed to the subroutine. The value under this key will be a hash reference mapping database names to their respective handles.
sync
Name of the sync with which to associate the custom code. Cannot be used in combination with relation
.
relation
Name of the table or sequence with which to associate the custom code. Cannot be used in combination with sync
.
status
The current status of this customcode. Anything other than "active" means the code is not run.
priority
Number indicating the priority in which order to execute custom codes. Lower numbers are higher priority. Useful for subroutines that set lastcode
in order to cancel the execution of subsequent custom codes for the same when_run
.
src_code
File from which to read the custom code Perl source.
The body of the Perl subroutine should be implemented in the src_code
file, and not inside a sub
declaration. When called, it will be passed a single hash reference with the following keys:
syncname
The name of the currently-executing sync.
version
The version of Bucardo executing the sync.
sourcename
The name of the source database.
targetname
The name of the target database.
sendmail
A code reference that can be used to send email messages.
sourcedbh
A DBI database handle to the sync source database. Provided only to custom code executed by the controller.
rellist
An array reference of hash references, each representing a relation in the sync. Provided only to custom code executed by the controller. The keys in the hash are the same as the parameters supported by "add table" and "add sequence", as appropriate.
schemaname
The schema for the table that triggered the exception. Provided only to "exception" custom codes.
tablename
The name of the table that triggered the exception. Provided only to "exception" custom codes.
error_string
The string containing the actual error message. Provided only to "exception" custom codes.
deltabin
A hash reference with the name of each source database as a key and a list of all primary keys joined together with "\0". Provided only to "exception" custom codes.
attempts
The number of times the sync has been attempted. Provided only to "exception" custom codes.
conflicts
A hash reference of conflicting rows. The keys are the primary key values, and the values are hash references with the names of the databases containing the conflicting rows and true values. Provided only to "conflict" custom codes.
The custom code subroutine may set any of these keys in the hash reference to change the behavior of the sync:
message
Message to send to the logs.
warning
A warning to emit after the subroutine has returned.
error
An error to be thrown after the subroutine has returned.
nextcode
Set to send execution to the next custom code of the same type. Mainly useful to exception custom codes, and supported only by custom codes executed by the controller.
lastcode
Set to true to have any subsequent custom codes of the same type to be skipped.
endsync
Cancels the sync altogether.
An example:
use strict;
use warnings;
use Data::Dumper;
my $info = shift;
# Let's open a file.
my $file = '/tmp/bucardo_dump.txt';
open my $fh, '>:encoding(UTF-8)', $file or do {
$info->{warning} = "Cannot open $file: $!\n";
return;
};
# Inspect $info for fun.
print $fh Dumper $info;
close $fh or $info->{warning} = "Error closing $file: $!\n";
# Log a message and return.
$info->{message} = 'IN UR DATABASEZ NORMALIZIN UR RELAYSHUNS';
return;
bucardo update <type> <name> <parameters>
Updates a Bucardo object. The type
specifies the type of object to update, while the name
should be the name of the object. The supported parameters for each type are the same as those for "add". The supported types are:
customcode
db
sync
table
sequence
bucardo update customcode <name> setting=value
Updates an existing customcode. Items that can be changed are:
about
A short description of the custom code.
getdbh
Boolean indicating whether or not Perl DBI database handles should be provided to the custom code subroutine. If true, database handles will be provided under the dbh
key of the hash reference passed to the subroutine. The value under this key will be a hash reference mapping database names to their respective handles.
name
The name of the custom code object.
priority
Number indicating the priority in which order to execute custom codes. Lower numbers are higher priority. Useful for subroutines that set lastcode
in order to cancel the execution of subsequent custom codes for the same when_run
.
src_code
File from which to read the custom code Perl source.
status
The current status of this customcode. Anything other than "active" means the code is not run.
whenrun
A string indicating when the custom code should be run. Supported values include:
before_txn
before_check_rows
before_trigger_drop
after_trigger_drop
after_table_sync
exception
conflict
before_trigger_enable
after_trigger_enable
after_txn
before_sync
after_sync
bucardo update db <name> port=xxx host=xxx user=xxx pass=xxx
Updates a database. The name
is the name by which the database is known to Bucardo. This may vary from the actual database name, as multiple hosts might have databases with the same name.
The supported named parameters are:
dbname
db
The actual name of the database.
type
dbtype
The type of the database. Currently supported values are:
postgres
drizzle
mongo
mysql
maria
oracle
redis
sqlite
username
dbuser
dbdsn
A direct DSN to connect to a database. Will override all other connection options if set.
user
The username Bucardo should use to connect to the database.
password
dbpass
pass
The password Bucardo should use when connecting to the database.
dbhost
pghost
host
The host name to which to connect.
dbport
pgport
port
The port to which to connect.
dbconn
pgconn
conn
Additional connection parameters, e.g., sslmode=require
. Optional.
status
Status of the database in Bucardo. Must be either "active" or "inactive".
dbgroup
server_side_prepares
ssp
Enable or disable server-side prepares. Pass 1 to enable them or 0 to disable them.
makedelta
Enable or disable makedelta for this database.
dbservice
service
The service name to use for a Postgres database.
dbgroup
A comma-separated list of dbgroups to which to add the database. The database will be removed from any other dbgroups of which it was previously a member.
bucardo update sync syncname relgroup=xxx dbs=xxx
Updates a sync, which is a named replication event containing information about what to replicate from where to where. The supported parameters are:
name
The name of the sync. Required.
dbs
The name of a dbgroup or comma-delimited list of databases.
relgroup
The name of a relgroup to synchronize.
status
Indicates whether or not the sync is active. Must be either "active" or "inactive". Note that this will not change the current run status of the sync, just mark whether it should be active or inactive on the next reload. Use the activate sync
and <deactivate sync> commands to actually activate or deactivate a sync.
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync.
lifetime
Number of seconds a KID can live before being reaped.
maxkicks
Number of times a KID may be kicked before being reaped.
isolation_level
The transaction isolation level this sync should use. Only choices are "serializable" and "repeatable read"
conflict_strategy
The conflict resolution strategy to use in the sync. Supported values:
bucardo_source
The rows on the "source" database always "win". In other words, in a conflict, Bucardo copies rows from source to target.
bucardo_target
The rows on the "target" database always win.
bucardo_latest
The row that was most recently changed wins.
bucardo_abort
The sync is aborted on a conflict.
onetimecopy
Determines whether or not a sync should switch to a full copy mode for a single run. Supported values are:
stayalive
Boolean indicating whether or not the sync processes (CTL) should be persistent.
kidsalive
Boolean indicating whether or not the sync child processes (KID) should be persistent.
autokick
Boolean indicating whether or not tables in the sync should automatically send kick messages when they're modified. May be overridden by the autokick
parameter of individual tables.
checktime
An interval specifying the maximum time a sync should go before being kicked. Useful for busy systems where you don't want the overhead of notify triggers.
priority
An integer indicating the priority of the sync. Lower numbers are higher priority. Currently used only for display purposes.
analyze_after_copy
Boolean indicating whether or not to analyze tables after every sync. Off by default.
overdue
An interval specifying the amount of time after which the sync has not run that it should be considered overdue. check_bucardo_sync
issues a warning when a sync has not been run in this amount of time.
expired
An interval specifying the amount of time after which the sync has not run that it should be considered expired. check_bucardo_sync
issues a critical message when a sync has not been run in this amount of time.
track_rates
Boolean indicating whether or not to track synchronization rates.
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync.
strict_checking
Boolean indicating whether or not to be strict when comparing tables in the sync. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable strict_checking
and column differences will result in warnings rather than failing the validation. Defaults to true.
bucardo update table [schema].table db=actual_db_name
Updates a table object. The table information will be read from the specified database. Supported parameters:
db
The name of the database from which to read the table information. Should be a name known to Bucardo.
schemaname
The name of the schema in which the table is found.
tablename
The actual name of the table.
autokick
Boolean indicating whether or not the table should automatically send kick messages when it's modified. Overrides the autokick
parameter of any syncs of which the table is a part.
rebuild_index
Boolean indicating whether or not to rebuild indexes after every sync.
analyze_after_copy
Boolean indicating whether or not to analyze the table after every sync.
vacuum_after_copy
Boolean indicating whether or not to vacuum the table after every sync.
relgroup
Adds the table to the named relgroup. May be specified more than once. The table will be removed from any other relgroups.
makedelta
Specifies which databases need makedelta enabled for this table.
strict_checking
Boolean indicating whether or not to be strict when comparing the table between syncs. If the columns have different names or data types, the validation will fail. But perhaps the columns are allowed to have different names or data types. If so, disable strict_checking
and column differences will result in warnings rather than failing the validation. Defaults to true.
bucardo update sequence [schema].sequence relgroup=xxx
db
The name of the database where the sequence lives.
schemaname
The name of the schema in which the sequence is found.
relgroup
Adds the sequence to the named relgroup. May be specified more than once. The sequence will be removed from any other relgroups.
bucardo remove <item_type> <item_name>
Removes one or more objects from Bucardo. Valid item types are;
db
or database
Use the --force
option to clear out related tables and groups instead of erroring out.
dbgroup
relgroup
sync
table
sequence
customcols
customname
customcode
bucardo kick <syncname(s)> [timeout]
Tells one or more named syncs to fire as soon as possible. Note that this simply sends a request that the sync fire: it may not start right away if the same sync is already running, or if the source or target database has exceeded the number of allowed Bucardo connections. If the final argument is a number, it is treated as a timeout. If this number is zero, the bucardo command will not return until the sync has finished. For any other number, the sync will wait at most that number of seconds. If any sync has not finished before the timeout, an exit value of 1 will be returned. Errors will cause exit values of 2 or 3. In all other cases, an exit value of 0 will be returned.
If a timeout is given, the total completion time in seconds is also displayed. If the sync is going to multiple targets, the time that each target takes from the start of the kick is also shown as each target finishes. Options:
--retry
The number of times to retry a sync if it fails. Defaults to 0.
--retry-sleep
How long to sleep, in seconds, between each retry attempt.
--notimer
By default, kicks with a timeout argument give a running real-time summary of time elapsed by using the backspace character. This may not be wanted if running a kick, for example, via a cronjob, so turning --notimer on will simply print the entire message without backspaces.
bucardo pause <syncname(s)>
bucardo pause all
bucardo resume <syncname(s)>
bucardo resume all
Tells one or more named syncs to temporarily pause, or to resume from a previous pause. This only applies to active syncs and only takes effect if Bucardo is currently running. The keyword 'all' can be used as well to pause or resume all known active syncs.
bucardo reload config
bucardo reload config 30
Sends a message to all CTL and KID processes asking them to reload the Bucardo configuration. This configuration is a series of key/value pairs that configure Bucardo's behavior, and not any of the objects managed by the add
, remove
, or update
commands.
By default, Bucardo will send the message and then exit. Pass an optional number and Bucardo will instead wait up to that length of time for all child processes to report completion.
bucardo set setting1=value [setting2=value]
Sets one or more configuration setting table. Setting names are case-insensitive. The available settings are:
autosync_ddl
Which DDL changing conditions do we try to remedy automatically? Default: newcol
.
bucardo_version
Current version of Bucardo. Default: 5.6.0
.
bucardo_vac
Do we want the automatic VAC daemon to run? Default: 1
.
bucardo_initial_version
Bucardo version this schema was created with. Default: 5.6.0
.
ctl_checkonkids_time
How often does the controller check on the kids health? Default: 10
.
ctl_createkid_time
How long do we sleep to allow kids-on-demand to get on their feet? Default: 0.5
.
ctl_sleep
How long does the controller loop sleep? Default: 0.2
.
default_conflict_strategy
Default conflict strategy for all syncs. Default: bucardo_latest
.
default_email_from
Who the alert emails are sent as. Default<EMAIL_ADDRESS>.
default_email_host
Which host to send email through. Default: localhost
.
default_email_to
Who to send alert emails to. Default<EMAIL_ADDRESS>.
email_debug_file
File to save a copy of all outgoing emails to. Default: None.
endsync_sleep
How long do we sleep when custom code requests an endsync? Default: 1.0
.
flatfile_dir
Directory to store the flatfile output inside of. Default: .
.
host_safety_check
Regex to make sure we don't accidentally run where we should not. Default: None.
isolation_level
The transaction isolation level all sync should use. Defaults to 'serializable'. The only other valid option is 'repeatable read'
kid_deadlock_sleep
How long to sleep in seconds if we hit a deadlock error. Default: 0.5
. Set to -1 to prevent the kid from retrying.
kid_nodeltarows_sleep
How long do kids sleep if no delta rows are found? Default: 0.5
.
kid_pingtime
How often do we ping check the KID? Default: 60
.
kid_restart_sleep
How long to sleep in seconds when restarting a kid? Default: 1
.
kid_serial_sleep
How long to sleep in seconds if we hit a serialization error. Default: 0.5
. Set to -1 to prevent the kid from retrying.
kid_sleep
How long does a kid loop sleep? Default: 0.5
.
log_conflict_file
Name of the conflict detail log file. Default: bucardo_conflict.log
.
log_level
How verbose to make the logging. Higher is more verbose. Default: normal
.
log_microsecond
Show microsecond output in the timestamps? Default: 0
.
log_showlevel
Show log level in the log output? Default: 0
.
log_showline
Show line number in the log output? Default: 0
.
log_showpid
Show PID in the log output? Default: 1
.
log_showtime
Show timestamp in the log output? 0=off 1=seconds since epoch 2=scalar gmtime 3=scalar localtime. Default: 3
.
log_timer_format
The strftime
format to use to format the log timestamp when log_showtime
is set to 2 or 3. Defaults to simply the scalar output of the time.
mcp_dbproblem_sleep
How many seconds to sleep before trying to respawn. Default: 15
.
mcp_loop_sleep
How long does the main MCP daemon sleep between loops? Default: 0.2
.
mcp_pingtime
How often do we ping check the MCP? Default: 60
.
mcp_vactime
How often in seconds do we check that a VAC is still running? Default: 60
.
piddir
Directory holding Bucardo PID files. Default: /var/run/bucardo
.
reason_file
File to hold reasons for stopping and starting. Default: bucardo.restart.reason.txt
.
reload_config_timeout
Number of seconds the reload_config
command should wait for the reload to complete. Default: 30
.
semaphore_table
Table to let apps know a sync is ongoing. Default: bucardo_status
.
statement_chunk_size
How many primary keys to shove into a single statement. Default: 10000
.
stats_script_url
Location of the stats script. Default: http://www.bucardo.org/
.
stopfile
Name of the semaphore file used to stop Bucardo processes. Default: fullstopbucardo
.
syslog_facility
Which syslog facility level to use. Default: log_local1
.
tcp_keepalives_count
How many probes to send. 0 indicates sticking with system defaults. Default: 0
.
tcp_keepalives_idle
How long to wait between each keepalive probe. Default: 0
.
tcp_keepalives_interval
How long to wait for a response to a keepalive probe. Default: 0
.
vac_run
How often does the VAC process run? Default: 30
.
vac_sleep
How long does VAC process sleep between runs? Default: 120
.
warning_file
File containing all log lines starting with "Warning". Default: bucardo.warning.log
.
bucardo show all|changed|<setting> [<setting>...]
Shows the current Bucardo settings. Use the keyword "all" to see all the settings, "changed" to see settings different than the installed defaults, or specify one or more search terms. See "set" for complete details on the configuration settings.
bucardo config show all|<setting> [<setting>...]
bucardo config set <setting=value> [<setting=value>...]
Deprecated interface for showing and setting configuration settings. Use the "show" and "set" commands, instead.
bucardo ping
bucardo ping 60
bucardo ping 0
Sends a ping notice to the MCP process to see if it will respond. By default, it will wait 15 seconds. A numeric argument will change this timeout. Using a 0 as the timeout indicates waiting forever. If a response was returned, the program will exit with a value of 0. If it times out, the value will be 1. Returns a Nagios like message starting with "OK" or "CRITICAL" for success or failure.
bucardo status [syncname(s)] [--sort=#] [--show-days] [--compress]
Shows the brief status of all known syncs in a tabular format. If given one or more sync names, shows detailed information for each one. To see detailed information for all syncs, simply use "status all"
When showing brief information, the columns are:
The name of the sync
The state of the sync. Can be 'Good', 'Bad', 'Empty', 'No records found', 'Unknown', or the run state for a currently-running sync.
When the sync last successfully ran.
How long it has been since the last sync success
The number of insert and deletes performed by the last successful sync. May also show the number of rows truncated (T) or conflicted (C), if applicable.
When the sync last failed.
How long it has been since the last sync failure
The options for status
are:
--show-days
Specifies whether or not do list the time interval with days, or simply show the hours. For example, "3d 12h 6m 3s" vs. "48h 6m 3s"
--compress
Specifies whether or not to compress the time interval by removing spaces. Mostly used to limit the width of the 'status' display.
--sort=#
Requests sorting of the 'status' output by one of the nine columns. Use a negative number to reverse the sort order.
bucardo activate syncname [syncname2 syncname3 ...] [timeout]
Activates one or more named syncs. If given a timeout argument, it will wait until it has received confirmation from Bucardo that each sync has been successfully activated.
bucardo deactivate syncname [syncname2 syncname3 ...] [timeout]
Deactivates one or more named syncs. If given a timeout argument, it will wait until it has received confirmation from Bucardo that the sync has been successfully deactivated.
bucardo message 'I WAS HERE'
Sends a message to the running Bucardo logs. This message will appear prefixed with "MESSAGE: ". If Bucardo is not running, the message will go to the logs the next time Bucardo runs and someone adds another message.
bucardo reload [syncname2 syncname3 ...]
Sends a message to one or more sync processes, instructing them to reload. Waits for each to reload before going on to the next. Reloading consists of deactivating a sync, reloading its information from the database, and activating it again.
bucardo inspect <type> <name> [<name2>...]
Inspects one or more objects of a particular type. The results are sent to STDOUT
. The supported types include:
table
sync
relgroup
bucardo validate all|<sync> [<sync>...]
Validates one or more syncs. Use the keyword "all" to validate all syncs, or specify one or more syncs to validate.
Note that this command executes a subset of all the validation done when a sync is started or activated.
bucardo purge all|<table> [<table>...]
Purges the delta and track tables for one or more tables, for one or more databases. Use the keyword "all" to validate all tables, or specify one or more tables to validate.
bucardo delta [total] [<database>...]
Show the current delta count for each source target. Provide a list of databases to limit it to just the given ones. Wildcards are allowed. Use the special name "totals" to show only the grand total.
bucardo help
bucardo help <command>
bucardo help <command> <action>
Get help. General help can be returned, as well as help for a single command or a command and its action. Some examples:
bucard help list
bucard help add table
It is usually easier to set most of these options at the top of the script, or make an alias for them, as they will not change very often if at all.
-d
--db-name
bucardo --db-name widgets
bucardo -d bricolage
Name of the Bucardo database to which to connect.
-U
--db-user
bucardo --db-user postgres
bucardo -U Mom
User name to use when connecting to the Bucardo database.
-P
--db-pass
bucardo --db-pass s3cr1t
bucardo -P lolz
Password to use when connecting to the Bucardo database.
-h
--db-host
bucardo --db-host db.example.com
bucardo -h db2.example.net
Host name to use when connecting to the Bucardo database.
-p
--db-port
bucardo --db-port 7654
Port number to connect to when connecting to the Bucardo database.
--bucardorc
bucardo --bucardorc myrcfile
Use the specified file for configuration instead of the default ./.bucardorc.
--no-bucardorc
Do not use the ./.bucardorc configuration file.
--verbose
Makes bucardo run verbosely. Default is off.
--quiet
Tells bucardo to be as quiet as possible. Default is off.
--help
Shows a brief summary of usage for bucardo.
In addition to command-line configurations, you can put any options inside of a file. The file .bucardorc in the current directory will be used if found. If not found, then the file ~/.bucardorc will be used. Finally, the file /etc/bucardorc will be used if available. The format of the file is option = value, one per line. Any line starting with a '#' will be skipped. Any values loaded from a bucardorc file will be overwritten by command-line options. All bucardorc files can be ignored by supplying a --no-bucardorc
argument. A specific file can be forced with the --bucardorc=file
option; if this option is set, bucardo will refuse to run unless that file can be read.
The bucardo script uses $ENV{HOME} to look for a .bucardorc file.
Bug reports and feature requests are always welcome, please visit bucardo.org, file GitHub Issues, or post to our email list.
Bucardo
Copyright 2006-2023 Greg Sabino Mullane<EMAIL_ADDRESS>This program is free to use, subject to the limitations in the LICENSE file.


Scandium
Element Name:
Scandium
Latin Name:
Scandium
Atomic Number:
21
Atomic Weight:
44.9559 u
Is a:
Transition-Metal
State of the element:
Solid
About:
Scandium is a chemical element with the symbol Sc and atomic number 21. A silvery-white metallic d-block element, it has historically been classified as a rare-earth element,[5] together with yttrium and the lanthanides. It was discovered in 1879 by spectral analysis of the minerals euxenite and gadolinite from Scandinavia. Scandium is present in most of the deposits of rare-earth and uranium compounds, but it is extracted from these ores in only a few mines worldwide. Because of the low availability and the difficulties in the preparation of metallic scandium, which was first done in 1937, applications for scandium were not developed until the 1970s. The positive effects of scandium on aluminium alloys were discovered in the 1970s, and its use in such alloys remains its only major application. The global trade of scandium oxide is about 10 tonnes per year.
Electrons
21
Protons
21
Neutrons
24
Discoverer:
Lars Fredrik Nilson
Discovered in:
1879


| MODE | CH1 | CH2 | CH3 | CH4 | VALUE | STATE | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Before | After | Before | After | Before | After | Before | After | Ch1 | Ch2 | Ch3 | Ch4 | |||
| cell1_1 | cell2_1 | cell3_1 | cell4_1 | cell5_1 | cell6_1 | cell7_1 | cell8_1 | cell8_1 | cell9_1 | cell10_1 | cell11_1 | cell12_1 | cell13_1 | cell14_1 |
| MODE | Channel | Before | After | AA | BB | PP | KK | STATE | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| cell1_1 | cell2_1 | cell3_1 | cell4_1 | cell5_1 | cell6_1 | cell7_1 | cell8_1 | cell9_1 | cell10_1 | cell11_1 | cell12_1 | cell13_1 | cell14_1 |
MEAN:
P-P:
MIN:
MAX:
Offset:
Gain:
MEAN:
P-P:
MIN:
MAX:
Offset:
Gain:
MEAN:
P-P:
MIN:
MAX:
Offset:
Gain:
MEAN:
P-P:
MIN:
MAX:
Offset:
Gain:
Decimation:
Decimation:
Hysteresis (V):


Reliable, efficient delivery
Powered by Technology
Our Artificial Intelligence powered tools use millions of project data points to ensure that your project is successful
Supervisor
Monitors activity to identify project roadblocks
Scans our talent network to create the optimal team for your project
Regularly evaluates our talent to ensure quality
Uses data from past projects to provide better delivery estimates


Get your Peace of Mind Score!
Check how effective your company is at raising awareness of mental health
issues and providing resources to those who may need help.
TEAM
We are a team of two Computer Science undergraduates from MIT and one Economics undergraduate from Yale
who are passionate about mental health awareness.
GOAL
Studies have shown that there is a direct correlation between workplaces that are
accepting of mental health issues and have happier employees with better productivity.
Our goal is to promote mental health awareness in the workplace and
provide information on how organizations can invest in the prevention of mental illnesses.
RESEARCH
Backed by data from the NIH, our model analyzes thousands of datapoints from employees
across the United States while considering factors such as:
Whether if the company is in the tech field
Whether if the company provides mental health benefits
Whether if the company has a wellness program
The number of employees in the company


| Methods' Details |
bootupProfile
- Description
- attempts to init xpcom runtime using given profile.
- Parameter product
- is the product to start up.
- Parameter profileName
- the profile name to be used.
- Returns
- the current refrence count for the given profile.
|
|
shutdownProfile
- Description
- attempts to shutdown the current profile.
- Returns
- the current refrence count for the current profile.
|
|
getCurrentProduct
- Description
- attempts to get the current product.
- Returns
- the current used product.
|
|
getCurrentProfile
| string |
| getCurrentProfile(); |
- Description
- attempts to get the current profile name.
- Returns
- the current used profile.
|
|
isCurrentProfileLocked
| boolean |
| isCurrentProfileLocked(); |
- Description
- attempts to check whether the current profile locked or not
- Returns
- return sal_True is current profile is locked
|
|
setCurrentProfile
- Description
- attempts to set the current used profile name for the given product.
- Parameter product
- is the product to be used.
- Parameter profileName
- the profile name to be used.
- Returns
- the current used profile name for the given product.
|
|
Top of Page
Copyright © 2013, The Apache Software Foundation, Licensed under the Apache License, Version 2.0. Apache, the Apache feather logo, Apache OpenOffice and OpenOffice.org are trademarks of The Apache Software Foundation. Other names may be trademarks of their respective owners.


Woodford 40 mile
Woodford ENG
Race Winners
(24 runnings)
30 Apr
1989
4:07:19
5:08:10
1988
---
---
26 Jul
1987
4:17:36
---
27 Jul
1986
4:16:46
6:25:02
27 Jul
1985
4:14:17
5:36:09
28 Jul
1984
4:02:07
---
30 Jul
1983
4:20:54
---
31 Jul
1982
4:08:22
---
18 Jul
1981
3:54:50
---
12 Jul
1980
4:00:24
---
1979
---
---
29 Jul
1978
3:50:14
---
30 Jul
1977
3:30:08
---
31 Jul 1976
3:42:59
---
26 Jul 1975
3:49:35
---
27 Jul 1974
3:42:45
---
28 Jul 1973
3:38:45
---
29 Jul 1972
3:50:05
---
31 Jul 1971
4:03:31
---
12 Sep 1970
4:07:21
---
13 Sep 1969
3:56:44
---
1968
---
---
09 Sep 1967
4:01:32
---
10 Sep 1966
3:52:43
---
11 Sep 1965
3:50:30
---
13 Sep 1964
3:56:54
---
14 Sep 1963
4:07:39
---
Notes: 1963 was the inaugural year. The 1968 race was cancelled due to a lack of entries. The distance prior to
1978 were well short of 40 miles. Reported distances: 37.8 miles for 1969, 37.4 miles for 1971, 37 miles for 1972,
and 37.4 miles for 1974-75. A new route was used from 1978 that was reported to have been 40 miles. Thru 1984,
the course was point-to-point from Woodford to Southend. The 1985 race was run from Woodford to Canvey Island.
Races from 1986 thru 1991 were run on an out-and-back course from Woodford.
Terry Tullett (ENG)
Hilary Walker-2
cancelled
cancelled
Paul Taylor (ENG)
n/a
Peter Sugden-2
Jill Cavanagh (ENG)
Cavin Woodward-5
Hilary Walker (ENG)
Peter Sugden (ENG)
none
Cavin Woodward-4
none
Mark Pickard (ENG)
none
Cavin Woodward-3
none
Martin Daykin (ENG)
none
not held
not held
Cavin Woodward-2
none
Norman Wilson (ENG)
none
Ian Burgess-2
none
Ian Burgess (ENG)
none
Cavin Woodward (ENG)
none
Joe Keating (ENG)
none
Ian MacIntosh (SCO)
none
Derek Horsup (ENG)
none
Eddie Nolan (ENG)
none
Mike Casse (ENG)
none
cancelled
cancelled
Eddie Elderfield-3
none
Manie Kuhn (RSA)
none
Eddie Elderfield-2
none
John Tarrant (ENG)
none
Eddie Elderfield (ENG)
none
Most Wins (1963/89)
Men
5
Cavin Woodward (ENG)
1974,1978,1981,1983,1985
Women
2
Hilary Walker (ENG)
1986,1989
Prize Money
Prize money was not been awarded at this race.
Race Time Bias
Insufficient data to calculate the race time bias.
Competitive Level
Insufficient data to calculate the competitive level.
Last Updated on 26 Oct 2004
Contributor: Ian Champion (50)


Support Vector Machine (SVM)
- 1. Linear SVM
- 2. Generalization properties
- 3. Non-linear decision boundary
- 4. Python implementation
- References
1. Linear SVM
The Support Vector Machine is a learning algorithm whose primary goal is to find the optimal decision boundary. In the separable case, when the decision boundary is a hyperplane, we can show that the solution only depends on a few data points, known as support vectors, and hence the name.
- Linearly separable case:
Let’s say we are in two dimensions, and we have a dataset with two labels. We want to find the line that achieves maximal separation between the two classes. That is, of all the separating lines, we want to find the one that maximizes the margin $\rho$, as depicted below.
A line has equation \(\omega^T\cdot x+b=0\) where $x=(x_1,x_2)$ are the 2d coordinates and $\omega$ is the normal vector. The distance between a point with coordinates $x$ and the line is given by $\omega^T(x-x_0)/{\lVert\omega\rVert}$, where $x_0$ is a point on the plane. Since $\omega^T\cdot x_0=-b$, the signed distance is \(d=\frac{\omega^T\cdot x+b}{\lVert\omega\rVert}\)
The margin is defined as a the minimum distance $C$ from the separating line, that is, $C=\text{Min }{y_i d_i}$. The optimal separating line maximizes this margin, that is, \(y_i\frac{\omega^T\cdot x_i+b}{\lVert\omega\rVert}\geq \text{Max }C=\rho,\;\forall (x_i,y_i)\) We have multiplied by the target $y_i\in{-1,1}$ to guarantee that each term is always positive on both sides of the separating line. Since the line equation is invariant under rescaling $(\omega,b)\rightarrow (\lambda \omega,\lambda b)$ we can choose $\lVert\omega\rVert=1/\rho$. This means that maximizing $\rho$ as above is equivalent to \(\text{Min }\lVert\omega\rVert,\;\;y_i(\omega^T\cdot x_i+b)\geq 1,\;\forall (x_i,y_i)\)
We can translate this minization problem to finding the minima of the loss function \(L=\frac{1}{2}\sum_{k=1}^d \omega_k^2 -\sum_{i=1}^N\alpha_i[ y_i(\omega^T\cdot x_i+b)-1],\;\alpha_i\geq 0\) where $\alpha_i$ are Lagrange multipliers, $d$ is the number of dimensions and $N$ is the number of datapoints. The local minima solves the equations
\[\begin{equation*}\begin{split}&\frac{\partial L}{\partial \omega_k}=\omega_k -\sum_{i=1}^N\alpha_i y_ix^k_i=0\\ &\frac{\partial L}{\partial b}=\sum_i\alpha_iy_i=0\\ &\frac{\partial L}{\partial \alpha_i}= y_i(\omega^T\cdot x_i+b)-1=0 \end{split}\end{equation*}\]Provided $\alpha_i>0$ for which the loss function is differentiable in $\alpha$. If $\alpha_i=0$ then the we only have the first two equations. This means that $\alpha_i>0$ corresponds to points that sit exactly on the margin, and the remaining equations depend only on these points. These are known as support vectors.
If we replace $\omega$ with its equation in the loss function $L$, we obtain the dual problem:
\[\hat{L}=\sum_{i=1}^N\alpha_i-\frac{1}{2}\sum_{i=1}^N\sum_{j=1}^N\alpha_i \alpha_j y_iy_j x_i^T\cdot x_j,\;\alpha_i\geq 0\]where we have used that $\sum_i\alpha_iy_i=0$. The problem with $-\hat{L}$ is actually a convex minimization problem and can be solved using traditional methods. The solution of the dual problem must suplemented with the additional conditions:
\[\begin{equation*}\begin{split}&y_i(\omega^T\cdot x_i+b)=1,\;\alpha_i>0\\ &y_i(\omega^T\cdot x_i+b)>1,\;\alpha_i=0\\ &\sum_i\alpha_i y_i=0\end{split}\end{equation*}\]Support vectors live on the margin and thus $y_i(\omega^T\cdot x_i+b)=1$. Given a support vector $x_s,y_s$, we can use this equation to determine $b$ \(b=y_s-\sum_{i=1}^m\alpha_p y_px_p^T\cdot x_s\) where $p=1\ldots m$ runs over the support vectors. $\hat{L}$ is maximized by the solution and as such \(\frac{\partial\hat{L}}{\partial \alpha_s}=1-y_s\sum_{p=1}^m\alpha_py_px_p^T\cdot x_s=0\) Multiplying this equation by $\alpha_s$ and summing over $s$ we obtain
\[\sum_{s=1}^m\alpha_s-\sum_{s=1}^m\sum_{p=1}^m\alpha_s\alpha_py_sy_px_p^T\cdot x_s=0\iff \lVert\omega\rVert^2=\sum_{s=1}^m\alpha_s=\lVert\alpha\rVert_1\]So the margin is inversely proportional to the linear norm of $\alpha$.
- Non-separable case:
In the non-separable case, one cannot find a hyperplane that separates the classes. That is, for any hyperplane, there exists $x_i,y_i$ such that
\[y_i(\omega^T\cdot x_i +b)\ngtr 1\]See picture above.
However, one can formulate a relaxed version of the constraints using slack variables $\xi_i\geq 0$
\[y_i(\omega^T\cdot x_i +b)\geq 1-\xi_i\]If we remove the points for which $0 < y_i(\omega^T\cdot x_i +b)<1$ then the data is linearly separable. With the remaining data, we can define a margin called a soft-margin instead of a hard-margin as in the separable case. The points for which $\xi_i$ is non-zero are the outliers.
As before, we want to minimize $\lVert\omega\rVert$, but at the same time we want to use the smallest possible number of $\xi$ with the smallest values possible. This can be written as
\[\frac{1}{2}\lVert\omega\rVert^2+\lambda \sum_i \xi_i^p-\sum_i\alpha_i[y_i(\omega^T\cdot x_i +b)- 1+\xi_i],\;\alpha_i\geq 0,\xi_i\geq 0\]The term $\lambda \sum_i \xi_i^p$ with $\lambda>0$ works as a regulator, which prevents $\xi_i$ from taking large values as well as having a large number of non-zero $\xi_i$. The exponent $p$ defines different types of regularization. For $p=1$ the loss function becomes the Hinge loss function
\[\frac{1}{2}\lVert\omega\rVert^2+\lambda\sum_i \text{max}(0,1-y_i(\omega^T\cdot x_i +b))\]The next steps are very similar to the separable case. One can build a dual problem and determine the support vectors and outliers.
2. Generalization properties
Given the separating hyperplane we can build the predictor
\[h(x)=\text{sign}(\omega\cdot x+b)\]We want to bound the generalization error
\[R(h_S)=\sum_{x,y} 1_{h(x)\neq y}D(x)\]To do this we can explore the leave-one-out error $R_{LOO}$. The $R_{LOO}(x)$ is the error on a point $x$ provided we train the algorithm on the remaining $S\setminus{x}$ dataset, that is, with the point $x$ excluded. The empirical $\hat{R}_{LLO}$ is obtained by averaging over all points of the dataset $S$, that is,
\[\hat{R}_{LLO}=\frac{1}{m}\sum_{x} R_{LLO}(x)\]One can show that the average of $\hat{R}_{LLO}$ is an unbiased estimate of the generalization error. That is,
\[\mathbb{E}_{S\sim D^m} \hat{R}_{LLO}=\mathbb{E}_{S'\sim D^{m-1}}(R(h_{S'}))\]In more detail,
\[\begin{aligned}\mathbb{E}_{S\sim D^m} \hat{R}_{LLO}&=\mathbb{E}_{S\sim D^m}\frac{1}{m}\sum_{x} R_{LLO}(x)\\ &=\mathbb{E}_{S\sim D^m}1_{h_{S'}(x)\neq y}\\ &=\mathbb{E}_{S'\sim D^{m-1},x\sim D}1_{h_{S'}(x)\neq y}\\ &=\mathbb{E}_{S'\sim D^{m-1}}R(h_{S'}) \end{aligned}\]Let’s estimate $R_{LLO}(x)$ for the SVM in the separable case. If $x$ is above the margin, then the error is zero because if we remove this point, the predictor will not change as it depends only on the support vectors. However, if $x$ is exactly on the margin, then the new predictor’s support vectors will change. This point $x$ may or may not be correctly classified, and so the maximum number of points that this procedure can misclassify is the same as the number of support vectors. This means that, \(\hat{R}_{LLO}\leq \frac{NV(S)}{m}\) where $NV$ is the number of support vectors for the dataset $S$. We may therefore conclude that the average generalization error is bounded by the average number of support vectors, that is, \(\mathbb{E}_{S'\sim D^{m-1}}R(h_{S'})\leq \frac{\mathbb{E}_{S\sim D^m}NV(S)}{m}\)
Assuming that $NV$ remains small for different datasets, the above result implies that the average error remains small.
3. Non-linear decision boundary
The dataset may be separable, but the decision boundary is not a hyperplane. In this case, there should exist a map $\phi: x\rightarrow x’$ that makes the problem linearly separable. All the previous steps follow except that we use $x’$ instead of $x$. The problem is in determining the map $\phi$, which is usually a difficult problem.
Kernel methods are used to address solutions of this type. Suppose we find such a map. Then we have a new set of features $\phi(x_1),\phi(x_2),\ldots \phi(x_n)$. The dual problem becomes
\[L_D=\sum_{i=1}^N\alpha_i-\frac{1}{2}\sum_{i=1}^N\sum_{j=1}^N\alpha_i \alpha_j y_iy_j \phi(x_i)^T\cdot \phi(x_j),\;\alpha_i\geq 0\]and the predictor
\[\begin{equation*}\begin{split}G(x)&=\text{sign} \big(\omega^T\phi(x)+b\big)\\ &=\text{sign} \big(\sum_{i=1}^N\alpha_i y_i\phi(x)^T\cdot\phi(x_i)+b\big) \end{split}\end{equation*}\]The constant $b$ can be determined from the location of a support vector. So we see that the solution only depends on the scalar $K(x,x’)=\phi(x)^T\cdot \phi(x’)$. The function $K(x,x’)$ is a Kernel function and obeys the following conditions
- It is symmetric: $K(x,x’)=K(x’,x)$
- It is positive semi-definite: $\sum_{i,j}K(x_i,x_j)\lambda_i\lambda_j\geq 0,\; \forall \lambda_i$
Instead of looking for the map $\phi$, we can search for the Kernel, a scalar function. For example, let’s consider the gaussian Kernel: \(K(x,y)=e^{-\frac{\lVert x-y\rVert^2}{2\sigma^2}}\) Since this is a positive and symmetric function, it is easy to see that the above conditions are satisfied. But can we find the map $\phi$ such that $K(x,x’)=\phi(x)^T\phi(x’)$? Note that
\[e^{-\frac{\lVert x-y\rVert^2}{2\sigma^2}}=e^{-\frac{x^2+y^2}{2\sigma^2}}\sum_{n=1}^{\infty}\frac{(xy)^n}{(2\sigma^2)^n n!}\]For each polynomial term in the expansion
\[\begin{equation*}\begin{split}(x y)^n=(\sum_{i=1}^d x_iy_i)^n&=\sum_{\sum_i k_i=n}\frac{n!}{k_1!k_2!\ldots k_d!}(x_1y_1)^{k_1}(x_2y_2)^{k_2}\ldots (x_dy_d)^{k_d}\\ &=\sum_{\sum_i k_i=n}\frac{n!}{k_1!k_2!\ldots k_d!} x_1^{k_1}x_2^{k_2}\ldots x_d^{k_d} y_1^{k_1}y_2^{k_2}\ldots y_d^{k_d}\\ &=h_n(x)^T\cdot h_n(y) \end{split}\end{equation*}\]where
\[h_n(x)=(x_1^n,\sqrt{n} x_1^{n-1}x_2,\sqrt{n} x_1^{n-1}x_3,\ldots,\sqrt{\frac{n!}{k_1!k_2!\ldots k_d!}} x_1^{k_1}x_2^{k_2}\ldots x_d^{k_d},\ldots x_d^n)\]This means that for the gaussian Kernel the map $\phi$ is infinite dimensional. This shows how powerful Kernel methods can be.
4. Python implementation
- The CVXOPT library can be used to solve quadratic optimization problems with constraints.
import cvxopt
class LinearSVM:
def __init__(self,num_iter=10**5):
self.coef_=None
self.intercept_={}
self.weight_={}
self.support_vectors_={}
self.classes=None
self.num_iter=num_iter
def fit(self,x,y):
y_aux=y.copy().reshape(-1,1)
y_aux=y_aux.astype('float64')
self.classes=set(y)
cvxopt.solvers.options['show_progress'] = False
pairs=[[(i,j) for i in self.classes for j in self.classes if j>i]]
for i,j in pairs:
idx=(y_aux==i) | (y_aux==j)
idx=idx.reshape(-1,)
x_temp=x[idx].copy()
y_temp=y_aux[idx].copy()
y_temp[y_temp==j]=-1.0
y_temp[y_temp==i]=1.0
z=y_temp*x_temp
Q=0.5*np.tensordot(z,z,axes=[(1),(1)])
Q=cvxopt.matrix(Q.tolist())
p=-1*np.ones(x_temp.shape[0])
p=cvxopt.matrix(p.tolist())
G=-1*np.identity(x_temp.shape[0])
G=cvxopt.matrix(G.tolist())
h=np.zeros(x_temp.shape[0])
h=cvxopt.matrix(h.tolist())
A=cvxopt.matrix(y_temp.tolist())
b=cvxopt.matrix(0.0)
sol=cvxopt.solvers.qp(Q, p, G, h, A, b)
sol=np.array(sol['x'])
#support vectors
sup_vec_loc=np.round(sol,2).reshape(-1,)
sol[sup_vec_loc==0]=0
sup_vec_loc=sup_vec_loc!=0
sup_vec=x_temp[sup_vec_loc]
self.support_vectors_[pair]=sup_vec
#margin
w=((sol*y_temp)*x_temp).sum(0)
y_s=y_temp[sup_vec_loc]
v=np.dot(sup_vec,w.reshape(-1,1))
coef=(y_s*v).mean()-y_s.mean()*v.mean()
coef=coef/(v.var())
intercept=y_s.mean()-coef*v.mean()
self.intercept_[pair]=intercept
self.weight_[pair]=coef*w
- SMO algorithm
In the SMO algorithm, or sequential minimal optimization, we solve the dual minimization problem iteratively. First we randomly initialize all $\alpha_i$. Then we choose a random pair of $\alpha$’s, say $\alpha_0,\alpha_1$ and solve for the minimum of $L(\alpha_0,\alpha_1)$ with the other $\alpha$ fixed. This is easy to do because the function $L(\alpha_0,\alpha_1)$ is actually one dimensional after using the constraint $\sum_i \alpha_i y_i=0$. Then we proceed with a different pair of $\alpha$’s and repeat until the solution converges.
"""Ensure that the target y has values -1,1
Step 1): generate all (i,j) to get access to alpha pairs
Step 2):
"""
alpha=np.random.normal(0,10,(y.shape[0],1))
alpha=np.abs(alpha)
l=[i for i in range(x.shape[0])]
samples=[(i,j) for i in l for j in l if i<j]
random.shuffle(samples)
num_iter=0
threshold=len(samples)
alpha_prev=alpha.copy()
T=(alpha*y).sum()
Z=(alpha*y*x).sum(0)
not_converged=True
while not_converged and num_iter<5*threshold:
for i,j in samples:
alpha0=alpha[i][0]
alpha1=alpha[j][0]
y0=y[i][0]
y1=y[j][0]
#If constraint is possible to be solved continue
k=-T+alpha0*y0+alpha1*y1
if y1*k<0 and y0*y1>0:
continue
# Solve 2-dimensional optimization problem with constraint
A=Z-alpha0*y0*x[i]-alpha1*y1*x[j]
B=x[i]-x[j]
a=1-y0*y1-y0*k*np.dot(B,x[j])-y0*np.dot(B,A)
a=a/(np.dot(B,B))
alpha0=max(0,a)
alpha1=y1*k-alpha0*y0*y1
if alpha1<0:
alpha1=0
alpha0=k*y0
#update alpha
alpha[i][0]=alpha0
alpha[j][0]=alpha1
T=-k+alpha0*y0+alpha1*y1
Z=A+alpha0*y0*x[i]+alpha1*y1*x[j]
# verify if alpha converges
error=alpha-alpha_prev
error=np.abs(error).max()
if error<10**(-8) and num_iter>3*threshold:
not_converged=False
print('converged')
break
alpha_prev=alpha.copy()
num_iter+=1
References
[1] Understanding Machine Learning: from Theory to Algorithms, Shai Ben-David and Shai Shalev-Shwartz
[2] The elements of statistical learning, T. Hastie, R. Tibshirani, J. Friedman
[3] Foundations of machine learning, M. Mohri, A. Rostamizadeh, A. Talwalkar


I'm Alex, Karate black-belt and Software Developer from Porto Alegre 🇧🇷, now living in Berlin 🇩🇪. Working with development since 2006, I already used a huge amount of different technologies. Technologies from JVM languages like Java, Kotlin, Scala, and Groovy to more script languages as Ruby and Python. Learned and taught TDD, Lean, Agile principles and many others through the years.
{% include /svg/wordcloud.svg %}
Today I work as a Software Engineer for Zalando where I can use all my knowledge about resilience and scalable Cloud-Native Applications using on a daily basis Python, Java, Scala and Kotlin with Kubernetes on the AWS stack.


Bootstrap components written in pure AngularJS by the AngularUI Team: angular-ui-bootstrap
Generate forms from a JSON schema, with AngularJS! angular-schema-form Note: Check Formly
AngularJS Toaster is a customized version of "toastr" non-blocking notification javascript library. AngularJS-Toaster
Free User Interface Kit Flat-UI main github page
The components Flat-UI Components Docs
fui-angular Native AngularJS (Angular) directives for FLAT UI


---
layout: default
title: "Home"
description: "My Personal Computer Repair and CompSci Blog"
---
Independent Bay Area Tech Support and Services
VicsTech offers laptop, desktop, tablet, and iPhone screen repairs at competitive prices with fast turnaround times.


gmrun(1) gmrun(1) NAME gmrun - Featureful CLI-like GTK+ application launcher SYNOPSIS gmrun [TEXT] DESCRIPTION This manual page documents briefly the gmrun command. This manual page was written for the Debian GNU/Linux distribution because the original program does not have a manual page. gmrun is a program that allows a user to use a CLI-like interface to launch applications from within the X11 windowing environment. It is based on the GTK+ toolkit. OPTIONS If gmrun is invoked with a command-line parameter, that will be the ini‐ tial content of the dialog box. gmrun can be customized using a configuration file, ~/.gmrunrc . The sys‐ tem-wide configuration file is /etc/gmrunrc . The format of the configuration file is simple; "variable = value". To refer to a variable as opposed to setting it, enclose the variable in ${}. Example: variable1 = foobar newvariable = ${variable1} foobaz Were this placed within the configuration file, the gmrun variable "new‐ variable" would contain the text "foobar foobaz". Some variables that are understood by gmrun, by default. These are: Terminal The command to run when Ctrl+Enter is pressed with no command entered; used to start a new terminal. TermExec The command to run when Ctrl+Enter is pressed with a command entered. The entered command is supplied as an argument to Ter‐ mExec. Width Initial width of gmrun window. Top, Left Initial placement of gmrun window - specify both Top and Left variables such that the gmrun window is placed at pixel (Top,Left) of your desktop. History Number of entered commands which should be kept in gmrun's his‐ tory(~/.gmrun_history). Using the Up and Down arrow keys within the gmrun window will cycle through the history. You can search backwards through the history with Ctrl+R or by writing ! (excla‐ mation mark), and forward with Ctrl+S. To cancel a search, hit Ctrl+G or the ESC key. Otherwise, after you have found the history item you wish to run, hit Enter. ShowLast Whether to show the last command as initial text, or an empty textarea (1 or 0). Selected Whether the initial text should be selected or not (1 or 0). URL HANDLING As opposed to entering a command, gmrun supports the entering of URLs. The included configuration file(/etc/gmrunrc) gives some examples. To set up your own url, create a variable in /etc/gmrunrc or ~/.gmrunrc with the format of "URL_protocol". For instance, URL_http. Here is an example: URL_http = /usr/bin/mozilla %u Using this, entering "http://www.slashdot.org" will run '/usr/bin/mozilla http://slashdot.org'. gmrun may also strip the protocol and colon away from the entered URL before passing it to the defined program. For example: URL_mailto = mutt %s Using this, entering<EMAIL_ADDRESS>will run the command 'mutt<EMAIL_ADDRESS>The included configuration files has a few examples which you can work with. You should also note that the protocol part of URL_protocol can be *anything*. You can set up "URL_foobarbazcustom", and so long as you enter in "foobarbazcustom:<whatever>", it will use the given program. Have fun :) SEE ALSO /usr/share/doc/gmrun/README.gz, /etc/gmrunrc. AUTHOR gmrun was written and is maintained by Mihai Bazon<EMAIL_ADDRESS>, the latest version is available from http://www.source‐ forge.net/projects/gmrun/ This manual page was written by David B Harris<EMAIL_ADDRESS>for the Debian GNU/Linux system (but may be used by others). August 8, 2001 gmrun(1)


public final class MergeSchedulerConfig
extends java.lang.Object
ConcurrentMergeScheduler
) controls the execution of
merge operations once they are needed (according to the merge policy). Merges
run in separate threads, and when the maximum number of threads is reached,
further merges will wait until a merge thread becomes available.
The merge scheduler supports the following dynamic settings:
index.merge.scheduler.max_thread_count
:
The maximum number of threads that may be merging at once. Defaults to
Math.max(1, Math.min(4, Runtime.getRuntime().availableProcessors() / 2))
which works well for a good solid-state-disk (SSD). If your index is on
spinning platter drives instead, decrease this to 1.
index.merge.scheduler.auto_throttle
:
If this is true (the default), then the merge scheduler will rate-limit IO
(writes) for merges to an adaptive value depending on how many merges are
requested over time. An application with a low indexing rate that
unluckily suddenly requires a large merge will see that merge aggressively
throttled, while an application doing heavy indexing will see the throttle
move higher to allow merges to keep up with ongoing indexing.
| 限定符和类型 | 字段和说明 |
|---|---|
static Setting<java.lang.Boolean> |
AUTO_THROTTLE_SETTING |
static Setting<java.lang.Integer> |
MAX_MERGE_COUNT_SETTING |
static Setting<java.lang.Integer> |
MAX_THREAD_COUNT_SETTING |
| 限定符和类型 | 方法和说明 |
|---|---|
int |
getMaxMergeCount()
Returns
maxMergeCount . |
int |
getMaxThreadCount()
Returns
maxThreadCount . |
boolean |
isAutoThrottle()
Returns
true iff auto throttle is enabled. |


Please log in with your admin account before searching.
{{'Select an area on the map by dragging/dropping' | t }}


During crisis response, resources are limited. In a short amount of time, first responders need to evaluate the situation they are in and decide on the most efficient plan of action to respond to crises. The Remote Sensing course discusses optimization problems and optimization algorithms to maximize the effect of data analysis and operations.
Topics Covered
- Decision making
- What decisions need to be made in HADR applications?
- Who makes those decisions?
- What information is necessary to support decisions?
- What costs/benefits/constraints are considered?
- What is optimization? What kind of optimization problems are there?
- How to sort data?
- What optimization problems can be done on graphs?
- What is optimization?
- Why is optimization difficult?
- Combinatorial difficulty, integer problems
- What kinds of optimization problems are there?
- Shortest path problems
- Routing problems
- Sensor placement/knapsack
- Traveling salesman problem (TSP)?
- What approaches are there to TSP?
- What variations are there of TSP?
- How to implement a solution to TSP (greedy) in python (networkx)
- How to improve on greedy solution
- Knapsack problem
- Formulation
- Applications in real world: budgets, scheduling
- Approaches: greedy, discuss edge cases where greedy fails
- What are job-shop scheduling problems (JSP)?
- How can they be represented on a graph?
- What can be modeled as JSP?
- What approaches can you take for JSP?
- How to implement a solution to JSP? How to improve?
- Exercise: implement greedy approach, see if they can improve
- Lecture + Teambuilding exercise: Sorting algorithms
- Teambuilding exercise, phase I: students are to line up by age, increasing. Timed.
- Lecture: sorting algorithms
- big O() complexity
- data structures
- simple sort algorithms, incl. heapsort
- Teambuilding part II: each student is given a random number, need to sort by number, increasing. Timed again.
- Discussion – were they able to improve on their time from part I?
- Part III: repeat one last time the sorting exercise, except the students get to discuss and plan for up to 5 minutes beforehand. They get new random numbers, and have to sort again.
- Discussion, were they able to improve given prep time? What things helped/didn't help?
- Goal of part I -> part II is to teach the benefit of doing things with more efficient algorithm. Goal of part II -> part III is the teach benefit of communication, planning, and setting responsibilities.


Website URL:
Website Name:
If you choose not to provide a name for the URL you are testing, the title field content from the tested website will be used as its name.
Scrape a specific Bioschemas type:
With the Submit tool you can test a specific URL for its compliance with the Bioschemas specifications and submit it to the Registry for public consultation. You can also choose to test all the children links for that page (links with the same root URL) as well as scraping your URL for one specific Bioschemas type (Event, Organization, Person, Training Material).
Scrape a specific Bioschemas type:


Guidance
Relates to learners completing an SFA-funded programme in one academic year, and progressing to a defined positive destination.
PDF , 335KB , 10 pages
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
The performance indicator for learner destinations is the proportion of learners completing a Skills Funding Agency funded programme in one academic year, and progressing in the next academic year to a defined positive destination. It includes all learners who completed at least one relevant qualification in the completion year with no other continuing learning aims funded by the Skills Funding Agency.
The results of the learner destination survey 2013 (for learners who completed learning in 2010 to 2011, and progressed into the 2011 to 2012 destination year) can be viewed on the FE Choices comparison site .
Outcome Based Success Measures are intended to better capture learner outcomes (learner destination into further learning or employment, learner progression to a higher level of qualification, and earnings). The measures are produced using matched data from the Department for Work and Pensions (DWP), HM Revenue and Customs (HMRC) the Individual Learner Record (ILR) and the National Pupil database (NPD). The measures are currently in development and the data is experimental. The aim is to introduce them in 2017.This will replace the Learner Destinations survey for those learners who are 19 years and older.
For 16-18 year old learners, where data matching into HMRC data is currently not legally possible, a destinations survey for learners who completed their learning in 2011 to 2012 is being undertaken to support the Outcome Measures. The survey is being run on behalf of the Agency by the research company GfK NOP. Where they are in-scope for the survey and where appropriate permissions have been given, learners will be contacted directly by GfK NOP. A copy of the destinations survey for 16- 18 year old learners is available above on this page.
Published: 8 April 2014
Updated: 25 February 2015
From: Skills Funding Agency
Part of: Further education and training


How much do you know about makeup and beauty awareness?
Signup today for free and be the first to get notified


Create new chat channel
Choose a name (max. 26 characters), add some initial members (user IDs), add an assistant to the channel
and make it public (anybody can join) if you like.
Join chat channel
Get the channel ID and access key from the channel owner, enter the data and hit 'join'.
If the channel is public or you are already a member but it is not yet in your list enter the ID and "open" as key.
Edit chat channel
Enter channel ID of a channel that you OWN then 'edit' members or hit 'delete'.
Please note that all content will be removed and all members will be kicked out of a channel when you delete it!


I’d like to describe a philosophy of software design that will ground my work. It will act as a simple guideline that I can follow when evaluating past, current and future software projects. Rather than an original effort, it’s a hybrid of two existing philosophies.
First, Paul Graham says in Six Principles for Making New Things:
Here it is: I like to find
- simple solutions
- to overlooked problems
- that actually need to be solved, and
- deliver them as informally as possible,
- starting with a very crude version 1, then
- iterating rapidly.
Principles one through three are spot on. I would, however, make a change to the second point. These must be overlooked problems that are meaningful to me. Create things you care about, first and foremost.
In iPhone-Likeness, John Gruber of Daring Fireball writes:
I’ll put forth one central, overriding guideline for iPhone UI design:
Figure out the absolute least you need to do to implement the idea, do just that, and then polish the hell out of the experience.
It’s difficult to argue with any of this advice. The separation between design and implementation (first the figuring out, then the doing), is especially insightful, if only because it is so often overlooked. The only quibble I have with John’s philosophy is that it should apply to the design of the entire application, not just the user interface.
And so we get to my own philosophy of software design:
I can’t claim that this philosophy will result in a commercially viable product. But I do believe it will help me create something that I can care about. And that is what matters.