Debezium connector for MySQL
MySQL has a binary log (binlog) that records all operations in the order in which they are committed to the database. This includes changes to table schemas as well as changes to the data in tables. MySQL uses the binlog for replication and recovery.
The Debezium MySQL connector reads the binlog, produces change events for row-level INSERT
, UPDATE
, and DELETE
operations, and emits the change events to Kafka topics. Client applications read those Kafka topics.
As MySQL is typically set up to purge binlogs after a specified period of time, the MySQL connector performs an initial consistent snapshot of each of your databases. The MySQL connector reads the binlog from the point at which the snapshot was made.
How the connector works
An overview of the MySQL topologies that the connector supports is useful for planning your application. To optimally configure and run a Debezium MySQL connector, it is helpful to understand how the connector tracks the structure of tables, exposes schema changes, performs snapshots, and determines Kafka topic names.
Supported MySQL topologies
The Debezium MySQL connector supports the following MySQL topologies:
- Standalone
-
When a single MySQL server is used, the server must have the binlog enabled (and optionally GTIDs enabled) so the Debezium MySQL connector can monitor the server. This is often acceptable, since the binary log can also be used as an incremental backup. In this case, the MySQL connector always connects to and follows this standalone MySQL server instance.
- Primary and replica
-
The Debezium MySQL connector can follow one of the primary servers or one of the replicas (if that replica has its binlog enabled), but the connector sees changes in only the cluster that is visible to that server. Generally, this is not a problem except for the multi-primary topologies.
The connector records its position in the server’s binlog, which is different on each server in the cluster. Therefore, the connector must follow just one MySQL server instance. If that server fails, that server must be restarted or recovered before the connector can continue.
- High available clusters
-
A variety of high availability solutions exist for MySQL, and they make it significantly easier to tolerate and almost immediately recover from problems and failures. Most HA MySQL clusters use GTIDs so that replicas are able to keep track of all changes on any of the primary servers.
- Multi-primary
-
Network Database (NDB) cluster replication uses one or more MySQL replica nodes that each replicate from multiple primary servers. This is a powerful way to aggregate the replication of multiple MySQL clusters. This topology requires the use of GTIDs.
A Debezium MySQL connector can use these multi-primary MySQL replicas as sources, and can fail over to different multi-primary MySQL replicas as long as the new replica is caught up to the old replica. That is, the new replica has all transactions that were seen on the first replica. This works even if the connector is using only a subset of databases and/or tables, as the connector can be configured to include or exclude specific GTID sources when attempting to reconnect to a new multi-primary MySQL replica and find the correct position in the binlog.
- Hosted
-
There is support for the Debezium MySQL connector to use hosted options such as Amazon RDS and Amazon Aurora.
Because these hosted options do not allow a global read lock, table-level locks are used to create the consistent snapshot.
Schema history topic
When a database client queries a database, the client uses the database’s current schema. However, the database schema can be changed at any time, which means that the connector must be able to identify what the schema was at the time each insert, update, or delete operation was recorded. Also, a connector cannot just use the current schema because the connector might be processing events that are relatively old and may have been recorded before the tables' schemas were changed.
To handle this, MySQL includes in the binlog not only the row-level changes to the data, but also the DDL statements that are applied to the database. As the connector reads the binlog and comes across these DDL statements, it parses them and updates an in-memory representation of each table’s schema. The connector uses this schema representation to identify the structure of the tables at the time of each insert, update, or delete operation and to produce the appropriate change event. In a separate database history Kafka topic, the connector records all DDL statements along with the position in the binlog where each DDL statement appeared.
When the connector restarts after having crashed or been stopped gracefully, the connector starts reading the binlog from a specific position, that is, from a specific point in time. The connector rebuilds the table structures that existed at this point in time by reading the database history Kafka topic and parsing all DDL statements up to the point in the binlog where the connector is starting.
This database history topic is for connector use only. The connector can optionally See emit schema change events to a different topic that is intended for consumer applications.
When the MySQL connector captures changes in a table to which a schema change tool such as gh-ost
or pt-online-schema-change
is applied there are helper tables created during the migration process. The connector needs to be configured to capture change to these helper tables. If consumers do not need the records generated for helper tables then a simple message transform can be applied to filter them out.
See default names for topics that receive Debezium event records.
Schema change topic
You can configure a Debezium MySQL connector to produce schema change events that include all DDL statements applied to databases in the MySQL server. The connector emits these events to a Kafka topic named serverName where serverName is the name of the connector as specified by the database.server.name
connector configuration property.
If you choose to use schema change events, ensure that you consume records from the schema change topic. The database history topic is for connector use only.
A global order for events emitted to the schema change topic is vital. Therefore, you must not partition the database history topic. This means that you must specify a partition count of 1 when creating the database history topic. When relying on auto topic creation, make sure that Kafka’s num.partitions configuration option, which specifies the default number of partitions, is set to 1 .
|
Each record that the connector emits to the schema change topic contains a message key that includes the name of the connected database when the DDL statement was applied, for example:
{
"schema": {
"type": "struct",
"name": "io.debezium.connector.mysql.SchemaChangeKey",
"optional": false,
"fields": [
{
"field": "databaseName",
"type": "string",
"optional": false
}
]
},
"payload": {
"databaseName": "inventory"
}
}
The schema change event record value contains a structure that includes the DDL statements, the name of the database to which the statements were applied, and the position in the binlog where the statements appeared, for example:
{
"schema": {
"type": "struct",
"name": "io.debezium.connector.mysql.SchemaChangeValue",
"optional": false,
"fields": [
{
"field": "databaseName",
"type": "string",
"optional": false
},
{
"field": "ddl",
"type": "string",
"optional": false
},
{
"field": "source",
"type": "struct",
"name": "io.debezium.connector.mysql.Source",
"optional": false,
"fields": [
{
"type": "string",
"optional": true,
"field": "version"
},
{
"type": "string",
"optional": false,
"field": "name"
},
{
"type": "int64",
"optional": false,
"field": "server_id"
},
{
"type": "int64",
"optional": false,
"field": "ts_ms"
},
{
"type": "string",
"optional": true,
"field": "gtid"
},
{
"type": "string",
"optional": false,
"field": "file"
},
{
"type": "int64",
"optional": false,
"field": "pos"
},
{
"type": "int32",
"optional": false,
"field": "row"
},
{
"type": "boolean",
"optional": true,
"default": false,
"field": "snapshot"
},
{
"type": "int64",
"optional": true,
"field": "thread"
},
{
"type": "string",
"optional": true,
"field": "db"
},
{
"type": "string",
"optional": true,
"field": "table"
},
{
"type": "string",
"optional": true,
"field": "query"
}
]
}
]
},
"payload": {
"databaseName": "inventory",
"ddl": "CREATE TABLE products ( id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY, name VARCHAR(255) NOT NULL, description VARCHAR(512), weight FLOAT ); ALTER TABLE products AUTO_INCREMENT = 101;",
"source" : {
"version": "1.4.0.Alpha2",
"name": "mysql-server-1",
"server_id": 0,
"ts_ms": 0,
"gtid": null,
"file": "mysql-bin.000003",
"pos": 154,
"row": 0,
"snapshot": true,
"thread": null,
"db": null,
"table": null,
"query": null
}
}
}
The ddl
field might contain multiple DDL statements. Each statement applies to the database in the databaseName
field. The statements appear in the order in which they were applied to the database. The source
field is structured exactly as a standard data change event written to table-specific topics. This field is useful to correlate events on different topics.
....
"payload": {
"databaseName": "inventory",
"ddl": "CREATE TABLE products ( id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,...)",
"source" : {
...
}
}
....
A client can submit multiple DDL statements to be applied to multiple databases. If MySQL applies them atomically, the connector takes the DDL statements in order, groups them by database, and creates a schema change event for each group. If MySQL applies them individually, the connector creates a separate schema change event for each statement.
See also: schema history topic.
Snapshots
When a Debezium MySQL connector is first started, it performs an initial consistent snapshot of your database. The following flow describes how the connector creates this snapshot. This flow is for the default snapshot mode, which is initial
. For information about other snapshot modes, see the MySQL connector snapshot.mode
configuration property.
| Step | Action |
|---|---|
1 |
Grabs a global read lock that blocks writes by other database clients. |
2 |
Starts a transaction with repeatable read semantics to ensure that all subsequent reads within the transaction are done against the consistent snapshot. |
3 |
Reads the current binlog position. |
4 |
Reads the schema of the databases and tables for which the connector is configured to capture changes. |
5 |
Releases the global read lock. Other database clients can now write to the database. |
6 |
If applicable, writes the DDL changes to the schema change topic, including all necessary |
7 |
Scans the database tables. For each row, the connector emits |
8 |
Commits the transaction. |
9 |
Records the completed snapshot in the connector offsets. |
- Connector restarts
-
If the connector fails, stops, or is rebalanced while performing the initial snapshot, then after the connector restarts, it performs a new snapshot. After that intial snapshot is completed, the Debezium MySQL connector restarts from the same position in the binlog so it does not miss any updates.
If the connector stops for long enough, MySQL could purge old binlog files and the connector’s position would be lost. If the position is lost, the connector reverts to the initial snapshot for its starting position. For more tips on troubleshooting the Debezium MySQL connector, see behavior when things go wrong.
- Global read locks not allowed
-
Some environments do not allow global read locks. If the Debezium MySQL connector detects that global read locks are not permitted, the connector uses table-level locks instead and performs a snapshot with this method. This requires the database user for the Debezium connector to have
LOCK_TABLES
privileges.Table 2. Workflow for performing an initial snapshot with table-level locks Step Action 1
Obtains table-level locks.
2
Starts a transaction with repeatable read semantics to ensure that all subsequent reads within the transaction are done against the consistent snapshot.
3
Reads and filters the names of the databases and tables.
4
Reads the current binlog position.
5
Reads the schema of the databases and tables for which the connector is configured to capture changes.
6
If applicable, writes the DDL changes to the schema change topic, including all necessary
DROP…
andCREATE…
DDL statements.7
Scans the database tables. For each row, the connector emits
CREATE
events to the relevant table-specific Kafka topics.8
Commits the transaction.
9
Releases the table-level locks.
10
Records the completed snapshot in the connector offsets.
Topic names
The default behavior is that a Debezium MySQL connector writes events for all INSERT
, UPDATE
, and DELETE
operations in one table to one Kafka topic. The Kafka topic naming convention is as follows:
serverName.databaseName.tableName
Suppose that fulfillment
is the server name, inventory
is the database name, and the database contains tables named orders
, customers
, and products
. The Debezium MySQL connector emits events to three Kafka topics, one for each table in the database:
fulfillment.inventory.orders fulfillment.inventory.customers fulfillment.inventory.products
Data change events
The Debezium MySQL connector generates a data change event for each row-level INSERT
, UPDATE
, and DELETE
operation. Each event contains a key and a value. The structure of the key and the value depends on the table that was changed.
Debezium and Kafka Connect are designed around continuous streams of event messages. However, the structure of these events may change over time, which can be difficult for consumers to handle. To address this, each event contains the schema for its content or, if you are using a schema registry, a schema ID that a consumer can use to obtain the schema from the registry. This makes each event self-contained.
The following skeleton JSON shows the basic four parts of a change event. However, how you configure the Kafka Connect converter that you choose to use in your application determines the representation of these four parts in change events. A schema
field is in a change event only when you configure the converter to produce it. Likewise, the event key and event payload are in a change event only if you configure a converter to produce it. If you use the JSON converver and you configure it to produce all four basic change event parts, change events have this structure:
{
"schema": { (1)
...
},
"payload": { (2)
...
},
"schema": { (3)
...
},
"payload": { (4)
...
},
}
| Item | Field name | Description |
|---|---|---|
1 |
|
The first |
2 |
|
The first |
3 |
|
The second |
4 |
|
The second |
By default, the connector streams change event records to topics with names that are the same as the event’s originating table. See topic names.
|
The MySQL connector ensures that all Kafka Connect schema names adhere to the Avro schema name format. This means that the logical server name must start with a Latin letter or an underscore, that is, a-z, A-Z, or _. Each remaining character in the logical server name and each character in the database and table names must be a Latin letter, a digit, or an underscore, that is, a-z, A-Z, 0-9, or _. If there is an invalid character it is replaced with an underscore character. This can lead to unexpected conflicts if the logical server name, a database name, or a table name contains invalid characters, and the only characters that distinguish names from one another are invalid and thus replaced with underscores. |
Change event keys
A change event’s key contains the schema for the changed table’s key and the changed row’s actual key. Both the schema and its corresponding payload contain a field for each column in the changed table’s PRIMARY KEY
(or unique constraint) at the time the connector created the event.
Consider the following customers
table, which is followed by an example of a change event key for this table.
CREATE TABLE customers (
id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
first_name VARCHAR(255) NOT NULL,
last_name VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL UNIQUE KEY
) AUTO_INCREMENT=1001;
Every change event that captures a change to the customers
table has the same event key schema. For as long as the customers
table has the previous definition, every change event that captures a change to the customers
table has the following key structure. In JSON, it looks like this:
{
"schema": { (1)
"type": "struct",
"name": "mysql-server-1.inventory.customers.Key", (2)
"optional": false, (3)
"fields": [ (4)
{
"field": "id",
"type": "int32",
"optional": false
}
]
},
"payload": { (5)
"id": 1001
}
}
| Item | Field name | Description |
|---|---|---|
1 |
|
The schema portion of the key specifies a Kafka Connect schema that describes what is in the key’s |
2 |
|
Name of the schema that defines the structure of the key’s payload. This schema describes the structure of the primary key for the table that was changed. Key schema names have the format connector-name.database-name.table-name.
|
3 |
|
Indicates whether the event key must contain a value in its |
4 |
|
Specifies each field that is expected in the |
5 |
|
Contains the key for the row for which this change event was generated. In this example, the key, contains a single |
Change event values
The value in a change event is a bit more complicated than the key. Like the key, the value has a schema
section and a payload
section. The schema
section contains the schema that describes the Envelope
structure of the payload
section, including its nested fields. Change events for operations that create, update or delete data all have a value payload with an envelope structure.
Consider the same sample table that was used to show an example of a change event key:
CREATE TABLE customers (
id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY,
first_name VARCHAR(255) NOT NULL,
last_name VARCHAR(255) NOT NULL,
email VARCHAR(255) NOT NULL UNIQUE KEY
) AUTO_INCREMENT=1001;
The value portion of a change event for a change to this table is described for:
create events
The following example shows the value portion of a change event that the connector generates for an operation that creates data in the customers
table:
{
"schema": { (1)
"type": "struct",
"fields": [
{
"type": "struct",
"fields": [
{
"type": "int32",
"optional": false,
"field": "id"
},
{
"type": "string",
"optional": false,
"field": "first_name"
},
{
"type": "string",
"optional": false,
"field": "last_name"
},
{
"type": "string",
"optional": false,
"field": "email"
}
],
"optional": true,
"name": "mysql-server-1.inventory.customers.Value", (2)
"field": "before"
},
{
"type": "struct",
"fields": [
{
"type": "int32",
"optional": false,
"field": "id"
},
{
"type": "string",
"optional": false,
"field": "first_name"
},
{
"type": "string",
"optional": false,
"field": "last_name"
},
{
"type": "string",
"optional": false,
"field": "email"
}
],
"optional": true,
"name": "mysql-server-1.inventory.customers.Value",
"field": "after"
},
{
"type": "struct",
"fields": [
{
"type": "string",
"optional": false,
"field": "version"
},
{
"type": "string",
"optional": false,
"field": "connector"
},
{
"type": "string",
"optional": false,
"field": "name"
},
{
"type": "int64",
"optional": false,
"field": "ts_ms"
},
{
"type": "boolean",
"optional": true,
"default": false,
"field": "snapshot"
},
{
"type": "string",
"optional": false,
"field": "db"
},
{
"type": "string",
"optional": true,
"field": "table"
},
{
"type": "int64",
"optional": false,
"field": "server_id"
},
{
"type": "string",
"optional": true,
"field": "gtid"
},
{
"type": "string",
"optional": false,
"field": "file"
},
{
"type": "int64",
"optional": false,
"field": "pos"
},
{
"type": "int32",
"optional": false,
"field": "row"
},
{
"type": "int64",
"optional": true,
"field": "thread"
},
{
"type": "string",
"optional": true,
"field": "query"
}
],
"optional": false,
"name": "io.debezium.connector.mysql.Source", (3)
"field": "source"
},
{
"type": "string",
"optional": false,
"field": "op"
},
{
"type": "int64",
"optional": true,
"field": "ts_ms"
}
],
"optional": false,
"name": "mysql-server-1.inventory.customers.Envelope" (4)
},
"payload": { (5)
"op": "c", (6)
"ts_ms":<PHONE_NUMBER>815, (7)
"before": null, (8)
"after": { (9)
"id": 1004,
"first_name": "Anne",
"last_name": "Kretchmar",
"email"<EMAIL_ADDRESS>},
"source": { (10)
"version": "1.4.0.Alpha2",
"connector": "mysql",
"name": "mysql-server-1",
"ts_ms": 0,
"snapshot": false,
"db": "inventory",
"table": "customers",
"server_id": 0,
"gtid": null,
"file": "mysql-bin.000003",
"pos": 154,
"row": 0,
"thread": 7,
"query": "INSERT INTO customers (first_name, last_name, email) VALUES ('Anne', 'Kretchmar', 'annek@noanswer.org')"
}
}
}
| Item | Field name | Description |
|---|---|---|
1 |
|
The value’s schema, which describes the structure of the value’s payload. A change event’s value schema is the same in every change event that the connector generates for a particular table. |
2 |
|
In the |
3 |
|
|
4 |
|
|
5 |
|
The value’s actual data. This is the information that the change event is providing. |
6 |
|
Mandatory string that describes the type of operation that caused the connector to generate the event. In this example,
|
7 |
|
Optional field that displays the time at which the connector processed the event. The time is based on the system clock in the JVM running the Kafka Connect task. |
8 |
|
An optional field that specifies the state of the row before the event occurred. When the |
9 |
|
An optional field that specifies the state of the row after the event occurred. In this example, the |
10 |
|
Mandatory field that describes the source metadata for the event. This field contains information that you can use to compare this event with other events, with regard to the origin of the events, the order in which the events occurred, and whether events were part of the same transaction. The source metadata includes:
If the |
update events
The value of a change event for an update in the sample customers
table has the same schema as a create event for that table. Likewise, the event value’s payload has the same structure. However, the event value payload contains different values in an update event. Here is an example of a change event value in an event that the connector generates for an update in the customers
table:
{
"schema": { ... },
"payload": {
"before": { (1)
"id": 1004,
"first_name": "Anne",
"last_name": "Kretchmar",
"email"<EMAIL_ADDRESS>},
"after": { (2)
"id": 1004,
"first_name": "Anne Marie",
"last_name": "Kretchmar",
"email"<EMAIL_ADDRESS>},
"source": { (3)
"version": "1.4.0.Alpha2",
"name": "mysql-server-1",
"connector": "mysql",
"name": "mysql-server-1",
"ts_ms":<PHONE_NUMBER>100,
"snapshot": false,
"db": "inventory",
"table": "customers",
"server_id": 223344,
"gtid": null,
"file": "mysql-bin.000003",
"pos": 484,
"row": 0,
"thread": 7,
"query": "UPDATE customers SET first_name='Anne Marie' WHERE id=1004"
},
"op": "u", (4)
"ts_ms":<PHONE_NUMBER>523 (5)
}
}
| Item | Field name | Description |
|---|---|---|
1 |
|
An optional field that specifies the state of the row before the event occurred. In an update event value, the |
2 |
|
An optional field that specifies the state of the row after the event occurred. You can compare the |
3 |
|
Mandatory field that describes the source metadata for the event. The
If the |
4 |
|
Mandatory string that describes the type of operation. In an update event value, the |
5 |
|
Optional field that displays the time at which the connector processed the event. The time is based on the system clock in the JVM running the Kafka Connect task. |
|
Updating the columns for a row’s primary/unique key changes the value of the row’s key. When a key changes, Debezium outputs three events: a |
Primary key updates
An UPDATE
operation that changes a row’s primary key field(s) is known
as a primary key change. For a primary key change, in place of an UPDATE
event record, the connector emits a DELETE
event record for the old key and a CREATE
event record for the new (updated) key. These events have the usual structure and content, and in addition, each one has a message header related to the primary key change:
-
The
DELETE
event record has__debezium.newkey
as a message header. The value of this header is the new primary key for the updated row. -
The
CREATE
event record has__debezium.oldkey
as a message header. The value of this header is the previous (old) primary key that the updated row had.
delete events
The value in a delete change event has the same schema
portion as create and update events for the same table. The payload
portion in a delete event for the sample customers
table looks like this:
{
"schema": { ... },
"payload": {
"before": { (1)
"id": 1004,
"first_name": "Anne Marie",
"last_name": "Kretchmar",
"email"<EMAIL_ADDRESS>},
"after": null, (2)
"source": { (3)
"version": "1.4.0.Alpha2",
"connector": "mysql",
"name": "mysql-server-1",
"ts_ms":<PHONE_NUMBER>300,
"snapshot": false,
"db": "inventory",
"table": "customers",
"server_id": 223344,
"gtid": null,
"file": "mysql-bin.000003",
"pos": 805,
"row": 0,
"thread": 7,
"query": "DELETE FROM customers WHERE id=1004"
},
"op": "d", (4)
"ts_ms":<PHONE_NUMBER>461 (5)
}
}
| Item | Field name | Description |
|---|---|---|
1 |
|
Optional field that specifies the state of the row before the event occurred. In a delete event value, the |
2 |
|
Optional field that specifies the state of the row after the event occurred. In a delete event value, the |
3 |
|
Mandatory field that describes the source metadata for the event. In a delete event value, the
If the |
4 |
|
Mandatory string that describes the type of operation. The |
5 |
|
Optional field that displays the time at which the connector processed the event. The time is based on the system clock in the JVM running the Kafka Connect task. |
A delete change event record provides a consumer with the information it needs to process the removal of this row. The old values are included because some consumers might require them in order to properly handle the removal.
MySQL connector events are designed to work with Kafka log compaction. Log compaction enables removal of some older messages as long as at least the most recent message for every key is kept. This lets Kafka reclaim storage space while ensuring that the topic contains a complete data set and can be used for reloading key-based state.
Tombstone events
When a row is deleted, the delete event value still works with log compaction, because Kafka can remove all earlier messages that have that same key. However, for Kafka to remove all messages that have that same key, the message value must be null
. To make this possible, after Debezium’s MySQL connector emits a delete event, the connector emits a special tombstone event that has the same key but a null
value.
Data type mappings
The Debezium MySQL connector represents changes to rows with events that are structured like the table in which the row exists. The event contains a field for each column value. The MySQL data type of that column dictates how Debezium represents the value in the event.
Columns that store strings are defined in MySQL with a character set and collation. The MySQL connector uses the column’s character set when reading the binary representation of the column values in the binlog events.
The connector can map MySQL data types to both literal and semantic types.
-
Literal type: how the value is represented using Kafka Connect schema types
-
Semantic type: how the Kafka Connect schema captures the meaning of the field (schema name)
Basic types
The following table shows how the connector maps basic MySQL data types.
| MySQL type | Literal type | Semantic type |
|---|---|---|
|
|
n/a |
|
|
n/a |
|
|
|
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
n/a |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Temporal types
Excluding the TIMESTAMP
data type, MySQL temporal types depend on the value of the time.precision.mode
connector configuration property. For TIMESTAMP
columns whose default value is specified as CURRENT_TIMESTAMP
or NOW
, the value 1970-01-01 00:00:00
is used as the default value in the Kafka Connect schema.
MySQL allows zero-values for DATE, `DATETIME
, and TIMESTAMP
columns because zero-values are sometimes preferred over null values. The MySQL connector represents zero-values as null values when the column definition allows null values, or as the epoch day when the column does not allow null values.
The DATETIME
type represents a local date and time such as "2018-01-13 09:48:27". As you can see, there is no time zone information. Such columns are converted into epoch milliseconds or microseconds based on the column’s precision by using UTC. The TIMESTAMP
type represents a timestamp without time zone information. It is converted by MySQL from the server (or session’s) current time zone into UTC when writing and from UTC into the server (or session’s) current time zone when reading back the value. For example:
-
DATETIME
with a value of2018-06-20 06:37:03
becomes1529476623000
. -
TIMESTAMP
with a value of2018-06-20 06:37:03
becomes2018-06-20T13:37:03Z
.
Such columns are converted into an equivalent io.debezium.time.ZonedTimestamp
in UTC based on the server (or session’s) current time zone. The time zone will be queried from the server by default. If this fails, it must be specified explicitly by the database serverTimezone
MySQL configuration option. For example, if the database’s time zone (either globally or configured for the connector by means of the serverTimezone
option) is "America/Los_Angeles", the TIMESTAMP value "2018-06-20 06:37:03" is represented by a ZonedTimestamp
with the value "2018-06-20T13:37:03Z".
The time zone of the JVM running Kafka Connect and Debezium does not affect these conversions.
More details about properties related to termporal values are in the documentation for MySQL connector configuration properties.
- time.precision.mode=adaptive_time_microseconds(default)
-
The MySQL connector determines the literal type and semantic type based on the column’s data type definition so that events represent exactly the values in the database. All time fields are in microseconds. Only positive
TIME
field values in the range of00:00:00.000000
to23:59:59.999999
can be captured correctly.Table 9. Mappings when time.precision.mode=adaptive_time_microseconds
MySQL type Literal type Semantic type DATE
INT32
io.debezium.time.Date
Represents the number of days since the epoch.TIME[(M)]
INT64
io.debezium.time.MicroTime
Represents the time value in microseconds and does not include time zone information. MySQL allowsM
to be in the range of0-6
.DATETIME, DATETIME(0), DATETIME(1), DATETIME(2), DATETIME(3)
INT64
io.debezium.time.Timestamp
Represents the number of milliseconds past the epoch and does not include time zone information.DATETIME(4), DATETIME(5), DATETIME(6)
INT64
io.debezium.time.MicroTimestamp
Represents the number of microseconds past the epoch and does not include time zone information. - time.precision.mode=connect
-
The MySQL connector uses defined Kafka Connect logical types. This approach is less precise than the default approach and the events could be less precise if the database column has a fractional second precision value of greater than
3
. Values in only the range of00:00:00.000
to23:59:59.999
can be handled. Settime.precision.mode=connect
only if you can ensure that theTIME
values in your tables never exceed the supported ranges. Theconnect
setting is expected to be removed in a future version of Debezium.Table 10. Mappings when time.precision.mode=connect
MySQL type Literal type Semantic type DATE
INT32
org.apache.kafka.connect.data.Date
Represents the number of days since the epoch.TIME[(M)]
INT64
org.apache.kafka.connect.data.Time
Represents the time value in microseconds since midnight and does not include time zone information.DATETIME[(M)]
INT64
org.apache.kafka.connect.data.Timestamp
Represents the number of milliseconds since the epoch, and does not include time zone information.
Decimal types
Debezium connectors handle decimals according to the setting of the decimal.handling.mode
connector configuration property.
- decimal.handling.mode=precise
-
Table 11. Mappings when decimal.handing.mode=precise
MySQL type Literal type Semantic type NUMERIC[(M[,D])]
BYTES
org.apache.kafka.connect.data.Decimal
Thescale
schema parameter contains an integer that represents how many digits the decimal point shifted.DECIMAL[(M[,D])]
BYTES
org.apache.kafka.connect.data.Decimal
Thescale
schema parameter contains an integer that represents how many digits the decimal point shifted. - decimal.handling.mode=double
-
Table 12. Mappings when decimal.handing.mode=double
MySQL type Literal type Semantic type NUMERIC[(M[,D])]
FLOAT64
n/a
DECIMAL[(M[,D])]
FLOAT64
n/a
- decimal.handling.mode=string
-
Table 13. Mappings when decimal.handing.mode=string
MySQL type Literal type Semantic type NUMERIC[(M[,D])]
STRING
n/a
DECIMAL[(M[,D])]
STRING
n/a
Boolean values
MySQL handles the BOOLEAN
value internally in a specific way.
The BOOLEAN
column is internally mapped to the TINYINT(1)
data type.
When the table is created during streaming then it uses proper BOOLEAN
mapping as Debezium receives the original DDL.
During snapshots, Debezium executes SHOW CREATE TABLE
to obtain table definitions that return TINYINT(1)
for both BOOLEAN
and TINYINT(1)
columns. Debezium then has no way to obtain the original type mapping and so maps to TINYINT(1)
.
The operator can configure the out-of-the-box TinyIntOneToBooleanConverter
custom converter that would either map all TINYINT(1)
columns to BOOLEAN
or if the selector
parameter is set then a subset of columns could be enumerated using comma-separated regular expressions.
Following is an example configuration:
converters=boolean boolean.type=io.debezium.connector.mysql.converters.TinyIntOneToBooleanConverter boolean.selector=db1.table1.*, db1.table2.column1
Spatial types
Currently, the Debezium MySQL connector supports the following spatial data types.
| MySQL type | Literal type | Semantic type |
|---|---|---|
|
|
|
Set up
Some MySQL setup tasks are required before you can install and run a Debezium connector.
Creating a user
A Debezium MySQL connector requires a MySQL user account. This MySQL user must have appropriate permissions on all databases for which the Debezium MySQL connector captures changes.
-
A MySQL server.
-
Basic knowledge of SQL commands.
-
Create the MySQL user:
mysql> CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';
-
Grant the required permissions to the user:
mysql> GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'user' IDENTIFIED BY 'password';
The table below describes the permissions.
If using a hosted option such as Amazon RDS or Amazon Aurora that does not allow a global read lock, table-level locks are used to create the consistent snapshot. In this case, you need to also grant LOCK_TABLES
permissions to the user that you create. See snapshots for more details. -
Finalize the user’s permissions:
mysql> FLUSH PRIVILEGES;
| Keyword | Description |
|---|---|
|
Enables the connector to select rows from tables in databases. This is used only when performing a snapshot. |
|
Enables the connector the use of the |
|
Enables the connector to see database names by issuing the |
|
Enables the connector to connect to and read the MySQL server binlog. |
|
Enables the connector the use of the following statements:
The connector always requires this. |
|
Identifies the database to which the permissions apply. |
|
Specifies the user to grant the permissions to. |
|
Specifies the user’s MySQL password. |
Enabling the binlog
You must enable binary logging for MySQL replication. The binary logs record transaction updates for replication tools to propagate changes.
-
A MySQL server.
-
Appropriate MySQL user privileges.
-
Check whether the
log-bin
option is already on:mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::" FROM information_schema.global_variables WHERE variable_name='log_bin';
-
If it is
OFF
, configure your MySQL server configuration file with the following properties, which are described in the table below:server-id = 223344 log_bin = mysql-bin binlog_format = ROW binlog_row_image = FULL expire_logs_days = 10
-
Confirm your changes by checking the binlog status once more:
mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::" FROM information_schema.global_variables WHERE variable_name='log_bin';
| Property | Description |
|---|---|
|
The value for the |
|
The value of |
|
The |
|
The |
|
This is the number of days for automatic binlog file removal. The default is |
Enabling GTIDs
Global transaction identifiers (GTIDs) uniquely identify transactions that occur on a server within a cluster. Though not required for a Debezium MySQL connector, using GTIDs simplifies replication and enables you to more easily confirm if primary and replica servers are consistent.
GTIDs are available in MySQL 5.6.5 and later. See the MySQL documentation for more details.
-
A MySQL server.
-
Basic knowledge of SQL commands.
-
Access to the MySQL configuration file.
-
Enable
gtid_mode
:mysql> gtid_mode=ON
-
Enable
enforce_gtid_consistency
:mysql> enforce_gtid_consistency=ON
-
Confirm the changes:
mysql> show global variables like '%GTID%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
| gtid_mode | ON |
+--------------------------+-------+
| Option | Description |
|---|---|
|
Boolean that specifies whether GTID mode of the MySQL server is enabled or not.
|
|
Boolean that specifies whether the server enforces GTID consistency by allowing the execution of statements that can be logged in a transactionally safe manner. Required when using GTIDs.
|
Configuring session timeouts
When an initial consistent snapshot is made for large databases, your established connection could timeout while the tables are being read. You can prevent this behavior by configuring interactive_timeout
and wait_timeout
in your MySQL configuration file.
-
A MySQL server.
-
Basic knowledge of SQL commands.
-
Access to the MySQL configuration file.
-
Configure
interactive_timeout
:mysql> interactive_timeout=<duration-in-seconds>
-
Configure
wait_timeout
:mysql> wait_timeout=<duration-in-seconds>
| Option | Description |
|---|---|
|
The number of seconds the server waits for activity on an interactive connection before closing it. See MySQL’s documentation for more details. |
|
The number of seconds the server waits for activity on a non-interactive connection before closing it. See MySQL’s documentation for more details. |
Enabling query log events
You might want to see the original SQL
statement for each binlog event. Enabling the binlog_rows_query_log_events
option in the MySQL configuration file allows you to do this.
This option is available in MySQL 5.6 and later.
-
A MySQL server.
-
Basic knowlede of SQL commands.
-
Access to the MySQL configuration file.
-
Enable
binlog_rows_query_log_events
:mysql> binlog_rows_query_log_events=ON
binlog_rows_query_log_events
is set to a value that enables/disables support for including the originalSQL
statement in the binlog entry.-
ON
= enabled -
OFF
= disabled
-
Deployment
To deploy a Debezium MySQL connector, install the Debezium MySQL connector archive, configure the connector, and start the connector by adding its configuration to Kafka Connect.
Installing
To install a Debezium MySQL connector, download the connector archive, extract it to your Kafka Connect environment, and ensure that the plug-ins parent directory is specified in your Kafka Connect environment.
-
Zookeeper, Kafka, and Kafka Connect are installed.
-
MySQL Server is installed and set up for Debezium.
-
Download the Debezium MySQL connector plug-in.
-
Extract the files into your Kafka Connect environment.
-
Add the plug-ins parent directory to your Kafka Connect
plugin.path
:plugin.path=/kafka/connect
The above example assumes that you extracted the Debezium MySQL connector into the
/kafka/connect/debezium-connector-mysql
path. -
Restart your Kafka Connect process. This ensures that the new JAR files are picked up.
Connector configuration example
Typically, you configure a Debezium MySQL connector in a .json
file that sets configuration properties for the connector. Following is an example configuration for a MySQL connector that connects to a MySQL server on port 3306 at <IP_ADDRESS>, whose logical name is fullfillment
.
For details, see MySQL connector configuration properties.
{
"name": "inventory-connector", (1)
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector", (2)
"database.hostname": "<IP_ADDRESS>", (3)
"database.port": "3306", (4)
"database.user": "debezium-user", (5)
"database.password": "debezium-user-pw", (6)
"database.server.id": "184054", (7)
"database.server.name": "fullfillment", (8)
"database.whitelist": "inventory", (9)
"database.history.kafka.bootstrap.servers": "kafka:9092", (10)
"database.history.kafka.topic": "dbhistory.fullfillment", (11)
"include.schema.changes": "true" (12)
}
}
| 1 | Connector’s name when registered with the Kafka Connect service. |
| 2 | Connector’s class name. |
| 3 | MySQL server address. |
| 4 | MySQL server port number. |
| 5 | MySQL user with the appropriate privileges. |
| 6 | MySQL user’s password. |
| 7 | Unique ID of the connector. |
| 8 | Logical name of the MySQL server or cluster. |
| 9 | List of databases hosted by the specified server. |
| 10 | List of Kafka brokers that the connector uses to write and recover DDL statements to the database history topic. |
| 11 | Name of the database history topic. This topic is for internal use only and should not be used by consumers. |
| 12 | Flag that specifies if the connector should generate events for DDL changes and emit them to the fulfillment schema change topic for use by consumers. |
Adding connector configuration
To start running a MySQL connector, configure a connector and add the configuration to your Kafka Connect cluster.
-
MySQL server is set up for a Debezium connector.
-
Debezium MySQL connector is installed.
-
Create a configuration for the MySQL connector.
-
Use the Kafka Connect REST API to add that connector configuration to your Kafka Connect cluster.
When the connector starts, it performs a consistent snapshot of the MySQL databases that the connector is configured for. The connector then starts generating data change events for row-level operations and streaming change event records to Kafka topics.
Connector properties
The Debezium MySQL connector has numerous configuration properties that you can use to achieve the right connector behavior for your application. Many properties have default values. Information about the properties is organized as follows:
The following configuration properties are required unless a default value is available.
| Property | Default | Description |
|---|---|---|
Unique name for the connector. Attempting to register again with the same name fails. This property is required by all Kafka Connect connectors. |
||
The name of the Java class for the connector. Always specify |
||
|
The maximum number of tasks that should be created for this connector. The MySQL connector always uses a single task and therefore does not use this value, so the default is always acceptable. |
|
IP address or host name of the MySQL database server. |
||
|
Integer port number of the MySQL database server. |
|
Name of the MySQL database to use when connecting to the MySQL database server. |
||
Password to use when connecting to the MySQL database server. |
||
Logical name that identifies and provides a namespace for the particular MySQL database server/cluster in which Debezium is capturing changes. The logical name should be unique across all other connectors, since it is used as a prefix for all Kafka topic names that receive events emitted by this connector. Only alphanumeric characters and underscores are allowed in this name. |
||
random |
A numeric ID of this database client, which must be unique across all currently-running database processes in the MySQL cluster. This connector joins the MySQL database cluster as another server (with this unique ID) so it can read the binlog. By default, a random number between 5400 and 6400 is generated, though the recommendation is to explicitly set a value. |
|
The full name of the Kafka topic where the connector stores the database schema history. |
||
A list of host/port pairs that the connector uses for establishing an initial connection to the Kafka cluster. This connection is used for retrieving database schema history previously stored by the connector, and for writing each DDL statement read from the source database. Each pair should point to the same Kafka cluster used by the Kafka Connect process. |
||
empty string |
An optional, comma-separated list of regular expressions that match the names of the databases for which to capture changes. The connector does not capture changes in any database whose name is not in |
|
empty string |
An optional, comma-separated list of regular expressions that match the names of databases for which you do not want to capture changes. The connector captures changes in any database whose name is not in the |
|
empty string |
An optional, comma-separated list of regular expressions that match fully-qualified table identifiers of tables whose changes you want to capture. The connector does not capture changes in any table not included in |
|
empty string |
An optional, comma-separated list of regular expressions that match fully-qualified table identifiers for tables whose changes you do not want to capture. The connector captures changes in any table not included in |
|
empty string |
An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to exclude from change event record values. Fully-qualified names for columns are of the form databaseName.tableName.columnName. |
|
empty string |
An optional, comma-separated list of regular expressions that match the fully-qualified names of columns to include in change event record values. Fully-qualified names for columns are of the form databaseName.tableName.columnName. |
|
n/a |
An optional, comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be truncated in the change event record values if the field values are longer than the specified number of characters. You can configure multiple properties with different lengths in a single configuration. The length must be a positive integer. Fully-qualified names for columns are of the form databaseName.tableName.columnName. |
|
n/a |
An optional, comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be replaced in the change event message values with a field value consisting of the specified number of asterisk ( |
|
n/a |
An optional, comma-separated list of regular expressions that match the fully-qualified names of character-based columns whose values should be pseudonyms in the change event record values. Pseudonyms consist of the hashed value obtained by applying the algorithm |
|
n/a |
An optional, comma-separated list of regular expressions that match the fully-qualified names of columns whose original type and length should be added as a parameter to the corresponding field schemas in the emitted change event records. These schema parameters:
are used to propagate the original type name and length for variable-width types, respectively. This is useful to properly size corresponding columns in sink databases. Fully-qualified names for columns are of one of these forms: databaseName.tableName.columnName databaseName.schemaName.tableName.columnName |
|
n/a |
An optional, comma-separated list of regular expressions that match the database-specific data type name of columns whose original type and length should be added as a parameter to the corresponding field schemas in the emitted change event records. These schema parameters:
are used to propagate the original type name and length for variable-width types, respectively. This is useful to properly size corresponding columns in sink databases. Fully-qualified data type names are of one of these forms: databaseName.tableName.typeName databaseName.schemaName.tableName.typeName See how MySQL connectors map data types for the list of MySQL-specific data type names. |
|
|
Time, date, and timestamps can be represented with different kinds of precision, including: |
|
|
Specifies how the connector should handle values for |
|
|
Specifies how BIGINT UNSIGNED columns should be represented in change events. Possible settings are: |
|
|
Boolean value that specifies whether the connector should publish changes in the database schema to a Kafka topic with the same name as the database server ID. Each schema change is recorded by using a key that contains the database name and whose value includes the DDL statement(s). This is independent of how the connector internally records database history. |
|
|
Boolean value that specifies whether the connector should include the original SQL query that generated the change event. |
|
|
Specifies how the connector should react to exceptions during deserialization of binlog events. |
|
|
Specifies how the connector should react to binlog events that relate to tables that are not present in internal schema representation. That is, the internal representation is not consistent with the database. |
|
|
Positive integer value that specifies the maximum size of the blocking queue into which change events read from the database log are placed before they are written to Kafka. This queue can provide backpressure to the binlog reader when, for example, writes to Kafka are slow or if Kafka is not available. Events that appear in the queue are not included in the offsets periodically recorded by this connector. Defaults to 8192, and should always be larger than the maximum batch size specified by the |
|
|
Positive integer value that specifies the maximum size of each batch of events that should be processed during each iteration of this connector. Defaults to 2048. |
|
|
Positive integer value that specifies the number of milliseconds the connector should wait for new change events to appear before it starts processing a batch of events. Defaults to 1000 milliseconds, or 1 second. |
|
|
A positive integer value that specifies the maximum time in milliseconds this connector should wait after trying to connect to the MySQL database server before timing out. Defaults to 30 seconds. |
|
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources that match one of these include patterns are used.
Do not also specify a setting for |
||
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources that do not match any of these exclude patterns are used. Do not also specify a value for |
||
|
|
When set to |
|
Controls whether a delete event is followed by a tombstone event. |
|
empty string |
A semicolon separated list of tables with regular expressions that match table column names. The connector maps values in matching columns to key fields in change event records that it sends to Kafka topics. This is useful when a table does not have a primary key, or when you want to order change event records in a Kafka topic according to a field that is not a primary key. |
|
bytes |
Specifies how binary columns, for example, |
The following table describes advanced MySQL connector properties. The default values for these properties rarely need to be changed. Therefore, you do not need to specify them in the connector configuration.
| Property | Default | Description |
|---|---|---|
|
A Boolean value that specifies whether a separate thread should be used to ensure that the connection to the MySQL server/cluster is kept alive. |
|
|
A Boolean value that specifies whether built-in system tables should be ignored. This applies regardless of the table include and exclude lists. By default, system tables are excluded from having their changes captured, and no events are generated when changes are made to any system tables. |
|
|
An integer value that specifies the maximum number of milliseconds the connector should wait during startup/recovery while polling for persisted data. The default is 100ms. |
|
|
The maximum number of times that the connector should try to read persisted history data before the connector recovery fails with an error. The maximum amount of time to wait after receiving no data is |
|
|
A Boolean value that specifies whether the connector should ignore malformed or unknown database statements or stop processing so a human can fix the issue.
The safe default is |
|
|
A Boolean value that specifies whether the connector should record all DDL statements |
|
|
Specifies whether to use an encrypted connection. Possible settings are: |
|
0 |
The size of a look-ahead buffer used by the binlog reader. The default setting of |
|
|
Specifies the criteria for running a snapshot when the connector starts. Possible settings are: |
|
|
Controls whether and how long the connector holds the global MySQL read lock, which prevents any updates to the database, while the connector is performing a snapshot. Possible settings are: |
|
All tables specified in |
An optional, comma-separated list of regular expressions that match names of schemas specified in |
|
Controls which table rows are included in snapshots. This property affects snapshots only. It does not affect events captured from the binlog. Specify a comma-separated list of fully-qualified table names in the form databaseName.tableName. |
||
|
During a snapshot, the connector queries each table for which the connector is configured to capture changes. The connector uses each query result to produce a read event that contains data for all rows in that table. This property determines whether the MySQL connector puts results for a table into memory, which is fast but requires large amounts of memory, or streams the results, which can be slower but work for very large tables. The setting of this property specifies the minimum number of rows a table must contain before the connector streams results. |
|
|
Controls how frequently the connector sends heartbeat messages to a Kafka topic. The default behavior is that the connector does not send heartbeat messages. |
|
|
Controls the name of the topic to which the connector sends heartbeat messages. The topic name has this pattern: |
|
A semicolon separated list of SQL statements to be executed when a JDBC connection, not the connection that is reading the transaction log, to the database is established.
To specify a semicolon as a character in a SQ statement and not as a delimiter, use two semicolons, ( |
||
An interval in milliseconds that the connector should wait before performing a snapshot when the connector starts. If you are starting multiple connectors in a cluster, this property is useful for avoiding snapshot interruptions, which might cause re-balancing of connectors. |
||
During a snapshot, the connector reads table content in batches of rows. This property specifies the maximum number of rows in a batch. |
||
|
Positive integer that specifies the maximum amount of time (in milliseconds) to wait to obtain table locks when performing a snapshot. If the connector cannot acquire table locks in this time interval, the snapshot fails. See how MySQL connectors perform database snapshots. |
|
|
Boolean value that indicates whether the connector converts a 2-digit year specification to 4 digits. Set to |
|
|
Schema version for the |
|
|
Indicates whether field names are sanitized to adhere to Avro naming requirements. |
|
Comma-separated list of oplog operations to skip during streaming. Values that you can specify are: |
The MySQL connector also supports pass-through configuration properties that are used when creating the Kafka producer and consumer. Specifically, all connector configuration properties that begin with the database.history.producer.
prefix are used (without the prefix) when creating the Kafka producer that writes to the database history. All properties that begin with the prefix database.history.consumer.
are used (without the prefix) when creating the Kafka consumer that reads the database history upon connector start-up.
For example, the following connector configuration properties can be used to secure connections to the Kafka broker:
database.history.producer.security.protocol=SSL database.history.producer.ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks database.history.producer.ssl.keystore.password=test1234 database.history.producer.ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks database.history.producer.ssl.truststore.password=test1234 database.history.producer.ssl.key.password=test1234 database.history.consumer.security.protocol=SSL database.history.consumer.ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks database.history.consumer.ssl.keystore.password=test1234 database.history.consumer.ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks database.history.consumer.ssl.truststore.password=test1234 database.history.consumer.ssl.key.password=test1234
See the Kafka documentation for more details about pass-through properties.
In addition to the pass-through properties for the Kafka producer and consumer, there are pass-through properties for database drivers. These properties have the database.
prefix. For example, database.tinyInt1isBit=false
is passed to the JDBC URL.
Monitoring
The Debezium MySQL connector provides three types of metrics that are in addition to the built-in support for JMX metrics that Zookeeper, Kafka, and Kafka Connect provide.
-
Snapshot metrics provide information about connector operation while performing a snapshot.
-
Binlog metrics provide information about connector operation when the connector is reading the binlog.
-
Schema history metrics provide information about the status of the connector’s schema history.
Debezium monitoring documentation provides details for how to expose these metrics by using JMX.
Snapshot metrics
The MBean is debezium.mysql:type=connector-metrics,context=snapshot,server=<database.server.name>
.
| Attributes | Type | Description |
|---|---|---|
|
The last snapshot event that the connector has read. |
|
|
The number of milliseconds since the connector has read and processed the most recent event. |
|
|
The total number of events that this connector has seen since last started or reset. |
|
|
The number of events that have been filtered by include/exclude list filtering rules configured on the connector. |
|
|
The list of tables that are monitored by the connector. |
|
|
The length the queue used to pass events between the snapshotter and the main Kafka Connect loop. |
|
|
The free capacity of the queue used to pass events between the snapshotter and the main Kafka Connect loop. |
|
|
The total number of tables that are being included in the snapshot. |
|
|
The number of tables that the snapshot has yet to copy. |
|
|
Whether the snapshot was started. |
|
|
Whether the snapshot was aborted. |
|
|
Whether the snapshot completed. |
|
|
The total number of seconds that the snapshot has taken so far, even if not complete. |
|
|
Map containing the number of rows scanned for each table in the snapshot. Tables are incrementally added to the Map during processing. Updates every 10,000 rows scanned and upon completing a table. |
|
|
The maximum buffer of the queue in bytes. |
|
|
The current data of records in the queue in bytes. |
The Debezium MySQL connector also provides the HoldingGlobalLock
custom snapshot metric. This metric is set to a Boolean value that indicates whether the connector currently holds a global or table write lock.
Binlog metrics
The MBean is debezium.mysql:type=connector-metrics,context=binlog,server=<database.server.name>
.
Transaction-related attributes are available only if binlog event buffering is enabled. See binlog.buffer.size
in the advanced connector configuration properties for more details.
| Attributes | Type | Description |
|---|---|---|
|
The last streaming event that the connector has read. |
|
|
The number of milliseconds since the connector has read and processed the most recent event. |
|
|
The total number of events that this connector has seen since last started or reset. |
|
|
The number of events that have been filtered by include/exclude list filtering rules configured on the connector. |
|
|
The list of tables that are monitored by the connector. |
|
|
The length the queue used to pass events between the streamer and the main Kafka Connect loop. |
|
|
The free capacity of the queue used to pass events between the streamer and the main Kafka Connect loop. |
|
|
Flag that denotes whether the connector is currently connected to the database server. |
|
|
The number of milliseconds between the last change event’s timestamp and the connector processing it. The values will incoporate any differences between the clocks on the machines where the database server and the connector are running. |
|
|
The number of processed transactions that were committed. |
|
|
The coordinates of the last received event. |
|
|
Transaction identifier of the last processed transaction. |
|
|
The maximum buffer of the queue in bytes. |
|
|
The current data of records in the queue in bytes. |
The Debezium MySQL connector also provides the following custom binlog metrics:
| Attribute | Type | Description |
|---|---|---|
|
The name of the binlog file that the connector has most recently read. |
|
|
The most recent position (in bytes) within the binlog that the connector has read. |
|
|
Flag that denotes whether the connector is currently tracking GTIDs from MySQL server. |
|
|
The string representation of the most recent GTID set processed by the connector when reading the binlog. |
|
|
The number of events that have been skipped by the MySQL connector. Typically events are skipped due to a malformed or unparseable event from MySQL’s binlog. |
|
|
The number of disconnects by the MySQL connector. |
|
|
The number of processed transactions that were rolled back and not streamed. |
|
|
The number of transactions that have not conformed to the expected protocol of |
|
|
The number of transactions that have not fit into the look-ahead buffer. For optimal performance, this value should be significantly smaller than |
Schema history metrics
The MBean is debezium.mysql:type=connector-metrics,context=schema-history,server=<database.server.name>
.
| Attributes | Type | Description |
|---|---|---|
|
One of |
|
|
The time in epoch seconds at what recovery has started. |
|
|
The number of changes that were read during recovery phase. |
|
|
the total number of schema changes applied during recovery and runtime. |
|
|
The number of milliseconds that elapsed since the last change was recovered from the history store. |
|
|
The number of milliseconds that elapsed since the last change was applied. |
|
|
The string representation of the last change recovered from the history store. |
|
|
The string representation of the last applied change. |
Behavior when things go wrong
Debezium is a distributed system that captures all changes in multiple upstream databases; it never misses or loses an event. When the system is operating normally or being managed carefully then Debezium provides exactly once delivery of every change event record.
If a fault does happen then the system does not lose any events. However, while it is recovering from the fault, it might repeat some change events. In these abnormal situations, Debezium, like Kafka, provides at least once delivery of change events.
The rest of this section describes how Debezium handles various kinds of faults and problems.
Configuration and startup errors
In the following situations, the connector fails when trying to start, reports an error or exception in the log, and stops running:
-
The connector’s configuration is invalid.
-
The connector cannot successfully connect to the MySQL server by using the specified connection parameters.
-
The connector is attempting to restart at a position in the binlog for which MySQL no longer has the history available.
In these cases, the error message has details about the problem and possibly a suggested workaround. After you correct the configuration or address the MySQL problem, restart the connector.
MySQL becomes unavailable
If your MySQL server becomes unavailable, the Debezium MySQL connector fails with an error and the connector stops. When the server is available again, restart the connector.
However, if GTIDs are enabled for a highly available MySQL cluster, you can restart the connector immediately. It will connect to a different MySQL server in the cluster, find the location in the server’s binlog that represents the last transaction, and begin reading the new server’s binlog from that specific location.
If GTIDs are not enabled, the connector records the binlog position of only the MySQL server to which it was connected. To restart from the correct binlog position, you must reconnect to that specific server.
Kafka Connect stops gracefully
When Kafka Connect stops gracefully, there is a short delay while the Debezium MySQL connector tasks are stopped and restarted on new Kafka Connect processes.
Kafka Connect process crashes
If Kafka Connect crashes, the process stops and any Debezium MySQL connector tasks terminate without their most recently-processed offsets being recorded. In distributed mode, Kafka Connect restarts the connector tasks on other processes. However, the MySQL connector resumes from the last offset recorded by the earlier processes. This means that the replacement tasks might generate some of the same events processed prior to the crash, creating duplicate events.
Each change event message includes source-specific information that you can use to identify duplicate events, for example:
-
Event origin
-
MySQL server’s event time
-
The binlog file name and position
-
GTIDs (if used)
Kafka becomes unavailable
The Kafka Connect framework records Debezium change events in Kafka by using the Kafka producer API. If the Kafka brokers become unavailable, the Debezium MySQL connector pauses until the connection is reestablished and the connector resumes where it left off.
MySQL purges binlog files
If the Debezium MySQL connector stops for too long, the MySQL server purges older binlog files and the connector’s last position may be lost. When the connector is restarted, the MySQL server no longer has the starting point and the connector performs another initial snapshot. If the snapshot is disabled, the connector fails with an error.
See snapshots for details about how MySQL connectors perform initial snapshots.


Year's Old
Mileage
Engine
Power
Fuel Type
Petrol
Diesel
Transmission Type
Manual
Automatic
Owner's Type
First
Second
Third
Company
Choose a Company Name
BMW
TATA
Model
Choose a Model Name
i10
Creta
Estimate Price


Modern Management for the Modern Man
We handle every part of the process all you need to do is create your event, and choose your tux shop and add your groomsmen's details.
We can the emails, send reminders and get your groomsman information to the store without any work for you so you can focus on other duties.

It is as easy as looking up your store by name, location, email or phone.
All we ask for is there name and email to get them started
We will email and text (if we have phone numbers) reminders weekly until your groomsman get their information in.
It is as easy as looking up your store by name, location, email or phone.
All we ask for is there name and email to get them started
"This was my second marriage, the first time around it was virtually impossible to get my groomsman to the tux shop for measurements. When they finally did I had to be so involved getting their information to the tux shop. With TuxName the process was easy."
"This was my second marriage, the first time around it was virtually impossible to get my groomsman to the tux shop for measurements. When they finally did I had to be so involved getting their information to the tux shop. With TuxName the process was easy."
"This was my second marriage, the first time around it was virtually impossible to get my groomsman to the tux shop for measurements. When they finally did I had to be so involved getting their information to the tux shop. With TuxName the process was easy."
Are you a store curious to learn about the hype?
TuxName is making the process of managing rentals even easier for you.
We are always looking to add new, check out our Company page for more details about how it works for your business.


--- layout: archive title: Zulip Chat Archive permalink: /stream/116395-maths/topic/subtraction.20on.20natural.20numbers.html ---
Is there a name and notation to distinguish between subtraction on natural numbers and subtraction on integers? This is for my dissertation rather than a Lean proof
Subtraction on natural numbers is sometimes called "monus" and denoted with a subtraction sign with a dot over it, like the top half of an american division sign
like so: ∸
Thanks! Is there a LaTeX command for that?
wiki says \dot -
you will probably have to play with spacing to get it to look nice
Alright then
Short answer: use unicode-math
and type ∸
Long answer: if for some reason, you don't want to use unicode-math
, use its documentation: http://mirrors.ctan.org/macros/latex/contrib/unicode-math/unimath-symbols.pdf search for "dot above" and get to page 17 where you see that ∸ is translated to \dotminus
. The problem is that you need to find which package defines this (an information which is irrelevant if you go with unicode-math). Very quick google latex dotminus brings: https://tex.stackexchange.com/questions/360069/what-symbol-for-primitive-recursive-cutoff-minus
Actually maybe using this symbol in Lean could make sense
It would help mathematicians notice this is not the right operation
I think there's an argument for using some sort of modification on all the total extensions of the usual functions as well ;-)
It's a thought, but I'd hate to have to explain to newcomers why 3 - 2 = 1
doesn't work
Yeah, this is not great either
Can we have both?
Yes but I have to explain why 2 - 3 gets evaluated as something wrong, which is worse
Like, the lib uses the funny symbol but users can still use -
And when they complain about 2 - 3 we tell them about ∸
I don't know
I am not seriously advocating changes to -
and /
, however it was the realisation that I could think about them in this way which made me realise that the "make it total" system would not lead to contradictions.
It's in core, so this isn't changing anyway
this would break too much of the "programming" part of lean I think
Maybe obscure unicode characters which look the same and can be redefined by users is the best option :-)
it would be hard to support a separate notation without a separate constant, and that would then mess up simp lemmas and such
It's plausible that you could make a local notation (for @has_sub.sub nat _
) work
My favorite way to reason about 2 - 3
is to consider that 2 - 3 : nat
(since -
is a binary op on nat
) so it can't possibly be the "right" answer. Now just consider, what's the next best option?
I think it suffices to just remind people that this isn't a partial function
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


Quan Anh Vo-Digital Engineering & Fabrication Index Page
This page is created for the purposes of storage and exhibition of weekly projects
About me
Rotation 1
Rotation 2
Rotation 3
Rotation 4
Rotation 5
Rotation 6
Rotation 7
Rotation 8
Rotation 9
Final Project


Before LR(1) parsing can take place, first an LR(1) parse table must be defined according to the operations defined here.
The interaction relevant to the parsing of strings in LR(1) parsing is identical to the parsing of strings in LL(1) parsing. See the notes on LL(1) parsing to learn about the interface.
The LR(1) parsing algorithm is, of course, very different from LL(1): the stack is used differently, and the parse is bottom-up rather than top-down as in LL(1) parsing. In addition, when reducing according to a production, the production is highlighted. However, the interface is identical.


See: Description
| Annotation Type | Description |
|---|---|
| AliasNames |
AliasNames allow the mapper to map a field to another that doesn't have the same name.
|
| CollectionType |
Collection type annotation allows you to specify the concrete class the mapper should instantiate
the mapping element to.
|
| Converter |
This annotation allows the mapper to map a field to another that doesn't have the same type using, if found, the most appropriate converter given.
|
| Default |
This annotation allows the mapper to supply a default value for the setters.
|
| IgnoreField |
Ignore field annotation lets you specify the fields that should be ignored during the mapping.
|
@AliasNames
: used to specify the alternative names of a field.@CollectionType
: used for field of a type that extends Collection
, it allow the user to specify the type of collection to have in the destination class.@Converter
: used to specify the converter to use for a field. This is a repeatable annotation.@Default
: used to specify a default value to use for a field.@IgnoreField
: the annotated field will be ignored during the mapping. If used on a class, it can take a list of string containing the field names or aliases to be ignored.AliasNames
,
CollectionType
,
Converter
,
Default
,
IgnoreField
Copyright © 2022. All rights reserved.


The best way to make your application run fast is to travel light. Everything depends on the size of your download and there are two ways to make your web application smaller: include less code and make fewer requests.
The best way to make your application load fast is to strip it to the bare essentials. Every line of code you include makes your application run slower. Spiffy UI reduces the size of your application in four ways:
Google's Closure Compiler strips out code you aren't using and compresses the code you are.
Reusable JavaScript minimizes the amount of GWT code you have to write.
Flexible library bundling let's you pick and choose just the components you want. For example, this application only includes half of JQuery UI.
GZIP compression makes the code you include substantially smaller GZIP reduces the size of JQuery core by 65%. Smaller files make faster applications.
Browser requests are expensive and combining files save a lot of download time. CSS sprites combine many images for much faster load time. Spiffy UI gives you to the tools to do the same with your JavaScript and CSS.
The Spiffy UI Framework includes Ant tasks to build, compile, and compress JavaScript, CSS, HTML, and some other formats. We use most of these tasks to make this sample application run faster by combining and compressing the files that make up this page. You can also see the uncompressed version of this page.
The Ant tasks are downloaded from the Nexus repository as a project dependency. Make sure the following entry is added to the chain your ivysettings.xml file:
<ibiblio name="sonatype-snapshots" m2compatible="true" usepoms="true" root="https://oss.sonatype.org/service/local/repositories/snapshots/content"/>
Then add the following dependencies to your ivy.xml file:
<dependency org="org.spiffyui" name="spiffyuitasks" rev="0.7.3"/> <dependency org="servletapi" name="servletapi" rev="2.4-20040521"/> <dependency org="com.yahoo.platform.yui" name="yuicompressor" rev="2.4.2"/> <dependency org="com.googlecode.jslint4java" name="jslint4java" rev="1.4.4"/> <dependency org="com.googlecode.jslint4java" name="jslint4java-ant" rev="1.4.4"/>
Ant has a tstamp
task to generate the build time into your project. We use it to fill in the time in our footer.
The tstamp
task supports many formats, but doesn't support epoch time. This is a big problem for localization and timezone issues. The epochtstamp
task just generates the time stamp in epoch time. Define it like this:
<taskdef resource="tasks.properties"> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </taskdef>
The usage is simple:
<epochtstamp property="builddate" />
Specify any property name and it will be filled in with the current build time in epoch time format.
The Spiffy UI Framework uses the Google Closure Compiler for checking and compressing JavaScript files. To compress the JavaScript files in your project you must first define the Closure task like this:
<taskdef resource="tasks.properties"> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </taskdef>
The Closure compiler is the only dependency that is not fully available from the Nexus server. You must download it manually from the Closure Compiler site.
Once this task is defined you can call it on all JavaScript files you want to compress like this:
<closure jsfile="myjs.min.js" closurejar="lib/closure-compiler-r916.jar" compilationLevel="SIMPLE_OPTIMIZATIONS" warninglevel="DEFAULT"> <fileset dir="/" includes="**/*.js" /> </closure>
This task supports the full Ant path and fileset syntax for specifying the input files unlike the task from the Google Closure project which only supports individual file references. Advanced Compilation and Externs has documentation on the meaning of the other options.
This task is combined with GZIP to compress the spiffyui.min.js
file, the only JavaScript you need for the Spiffy UI framework, down to 58 kilobytes.
Using GZIP compression can drastically reduce the size of the files in your web applicstion increasing application performance. Most web servers support GZIP compression on the fly, but precompressing files saves CPU time and further improves performance.
The GZipList task can zip a set of files and generate the compressed files with a .gz
file extension. We use this task to compress the HTML files generated by the GWT compiler, but you could use it with any type of file. First you defing the Spiffy UI Ant tasks:
<taskdef resource="tasks.properties"> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </taskdef>
Once this task is defined you can call it on all HTML and JavaScript files generated by the GWT compiler. Add a call to your task after you invoke the GWT compiler. Like this:
<gziplist dest="output"> <fileset dir="output" includes="*.html" /> <fileset dir="output" includes="*.js" /> </gziplist>
Include the resulting files in your WAR files and the web server will serve them to browsers which support GZIP.
The HTML properties task allows you to embed the contents of an HTML file into your GWT application. This is very useful for pages like this one which has large chunks of content in HTML. This task is not used in the Spiffy UI framework, but it powers many of the pages in the Spiffy UI application.
The first step is to define this task:
<taskdef resource="tasks.properties"> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </taskdef>
Once you define the task you can use it like this:
<htmlprops properties="MyStrings.properties"> <fileset dir="/" includes="**/*.html" /> </htmlprops>
This task supports the full Ant path and fileset syntax for specifying the input files. It will take all HTML files and format them into a properties file conforming with the java.util.Properties
format. Each property has the key of the file name and the value of the HTML file contents. The contents are reformatted to conform to the properties file standard.
Once this properties file is generate you can access it with a GWT Messages class.
For example, suppose you have an HTML file name myfile.html and it has the following contents:
<h1> This is my file </h1>
An entry will be added to your properties file with new-line characters escaped like this:
myfile_html=<h1>\r\nThis is my file\r\n</h1>
You can then access the information in this properties file with a Messages class like this:
import com.google.gwt.i18n.client.Messages; public interface MyStrings extends Messages { public String myfile_html(); }
The MyStrings
class can be accessed from any GWT class like this:
MyStrings strings = (MyStrings) GWT.create(MyStrings.class); String s = strings.myfile_html()
A common use of this task is to set this string as the content of an HTMLPanel. BuildPanel.java
from the Spiffy UI application has an example of using this technique.
You can use any text file as input for this task. In addition to HTML files it can be useful for JavaScript and JSON files for unit tests.
JSLint is a JavaScript static code analysis tool. It will check your JavaScript and find common errors and bad practices. All JavaScript in the Spiffy UI framework is checked with JSLint as part of the normal build.
To use JSLint start by defining the jslint task:
<taskdef name="jslint" classname="com.googlecode.jslint4java.ant.JSLintTask"> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </taskdef>
Once you define the task you can use it at any point in your build like this:
<jslint> <formatter type="plain" /> <fileset dir="/" includes="**/*.js" /> </jslint>
This task supports the full Ant path and fileset syntax for specifying the input files. More information about the JSLint task can be found at the jslint4java project page.
The YUI Compressor can compress CSS and JavaScript files. The Spiffy UI framework only uses the YUI Compressor for CSS since the Google Closure compiler is more efficient on compressing JavaScript.
The YUI compressor does not have a separate Ant task, but can be called using the Ant java
task. The first step is to concatenate the CSS files you want to compress since YUI can only compress single files. The order the files are concatenated is important since CSS cascades the style rules.
Ant provides a concat
task to combine the files. It can be used like this:
<concat destfile="mystyles.css" force="no"> <fileset dir="css" includes="**/*.css" /> </concat>
Once you have built the mystyles.css file you can use YUI Compressor to compress it like this:
<java classname="com.yahoo.platform.yui.compressor.Bootstrap" fork="true" failonerror="true" maxmemory="256m"> <arg value="-o"/> <arg value="mystyles.min.css"/> <arg value="mystyles.css"/> <classpath> <fileset dir="lib" includes="*.jar"/> </classpath> </java>
This will call the YUI Compressor to compress the mystyles.css file into mystyles.min.css. You can now include mystyles.min.css in your WAR file and reference it like a standard CSS file.


Education
National Technical University of Ukraine ‘Kyiv Polytechnic Institute’, Kyiv, Ukraine
Bachelor’s degree, Computer Science
2016-2020
Projects & Activities
Magic Mirror Smart mirror, that helps you get dressed in the morning and displays metric data (i.e. time, weather, plans)
- Clothes dress code recognition using deep learning; TensorFlow
- Designed application back-end to run on Raspberry PI; Python, Flask Framework, SQLite, Electron
- Designed hardware infrastructure;
Edumacator Simple LMS system for school
- Designed back-end system to store students data; PHP, MySQL
- Designed application authentication/security flow and data encryption; PHP
- Implemented AJAX communication between server and client; PHP and JS
- User interface design; HTML/CSS, Twitter bootstrap
- Created CKEditor plugin for mathematical input; JS, HTML5, CSS, LaTeX
Ludumdare Participated in 48-hour game development hackathon (game jam) 7 times in a row. Helped fix competitions site bugs on their Github repository
Projects for Ross Scott Utilities to help YouTuber produce videos
- Screen magnification utility that scales game capture using nearest neighbor scaling, for better image quality in older games; Java
- Created a fast motion blur script for video encoding; Avisynth
Series of educational games Created HTML5 games for Ukrainians small academy of science to help students learn physics
Technical Skills
- Tech skills: Algorithms, Database Design, Coding, Linux guru, SCRUM, Version Control systems, Troubleshooting, Tools development, Fast learner
- Programing languages: Python (Advanced), HTML5/CSS/JS (Advanced), Node.JS (Beginner), AngularJS (Intermediate), SQL Databases (Intermediate), C/C++ (Intermediate), PHP (Intermediate), MongoDB (Beginner)
Work Experience
Kyiv Tutors Association Kyiv, Ukraine
November 2015 - Present
Web developer
- Designed and created Wordpress template for the website; PHP
- Created Wordpress plugins for Tutor management; PHP, MySQL
- Designed logo; Adobe Illustrator, Photoshop
Additional Skills
Languages: English (Full professional proficiency), Ukrainian (Native or bilingual proficiency), Russian (Native or bilingual proficiency)
Achievements: 1st place on Kyiv schools ACM olympiad (2015, 2013)
Personal Interests: Photography, Video/Film production, Writing blog about technology and lifestyles, Mess around with electronics, Teaching, Helping people and trying to make everyones life easier


Loading Plex...
If you don't see Plex in a few seconds, please wait for the retry, then report this issue to the developer


--------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------
1
Human Resources Policies
Code of Conduct
FUNCTION: HUMAN RESOURCE MANAGEMENT
POLICY: CODE OF CONDUCT
PREPARED BY: GAGANDEEP SINGH
APPROVED BY: BHARAT KAPOOR
DATE: 1
st
December, 2015
VERSION NO: HR/ CC/ 01
–
A
Policy Number
HR/ CC/ 01
Policy
Distribution
All employees
Confidentiality
Confidential to DIPI employees only
Effective Date
December 2015
Applicability
All permanent employees and workers of the Company
Policy content
I. Compliance
The DIPI pledges to take whatever steps are necessary to fully comply with all
applicable laws, regulations, and all company policies and procedures.
You must maintain the highest standards of integrity, and professionalism in your
work for the DIPI. Any adverse behaviour may result in reputational risk or damage
to the DIPI.
1. Compliance with Laws and Regulations
You must take whatever steps are necessary to fully comply with all laws and
regulations that apply to conducting business activities.
In the event that you are
unsure as to whether any laws or regulations apply to your conduct, you must
consult with your local human resources department or your manager or the
Legal/IP Department before proceeding with any business activities.
2. Compliance with Internal Rules and Policies
You must fully comply with related internal company rules, regulations, and policies
when conducting business activities.
In the event that you are unsure as to
whether any internal company rules, regulations, or policies apply to your conduct,
you must consult with your local human resources department or your manager or
the Legal/IP Department to obtain a correct understanding.
What Is Expected of Everyone
1) Understand the Code.
2)
Comply with the Code and the applicable law and internal rules and policies.
When to Seek Guidance
When you are unsure of the right things to do, ask yourself the questions listed
below to help guide your decision
–
Is the action consistent with DIPI core values?
Will my decision affect the customer experience and the harmonious co-
existence between DIPI and the society?
Will my decision compromise any of DIPI
’s commitments to employees,
customers, business partners, and the society we serve?
Is the action fair and just?


2du
All Task
Active Task
Due Task
Completed Task
Incomplete Task
2du
© 2020. Designed By
Adeleye Jamiu
.
Your Number One Todo List App.
Add Todo
Add Todo
×
Title
Description
Priority
Choose Priority...
Very Low
Low
Average
High
Very High
Due Date
Project
Choose Project...
Work
Personal
Add Project
Add Project
×
Project Title
Projects
All Projects
work
personal
Oldest to Newest
Newest to Oldest
Priority (Lowest to Highest Priority)
Priority (Highest to Lowest Priority)
Projects
All Projects
work
personal
Todo Lists
todo row
Undo


The Penthouse
Use player to stream radio station
Back to Station Selection
Site Designed By Brady Kondek
webdesign.bradykondek.ga


Workshop Plan
- An overview of the web
- A light intro to HTML/CSS
- A deeper intro to Javascript
- Creating your own custom soundboard.
- Controlling a song's play/pause/looping
- Animation, maybe to the beat of the song
Basic Overview of a Website
- Content - HTML
- Styling - CSS
- Interactivity - JavaScript
- Logic - various languages, frameworks
What is HTML?
- Renders your content on the webpage in the form of tags and attributes
- Tags represent certain elements: images, paragraphs, headings, etc
- The most generic and most customizable tag is the <div> tag
What is CSS?
- HTML is to architect as CSS is to interior designer
- CSS specifies the style for our html elements
- Can style an element by tag, by id, by class
div {
width: 100px;
height: 100px;
border: 1px solid;
}
Motivation
- Websites used to be boring and static.
- Needed some way to animate things, have user interaction, make things more fun.
- Handle alerts, custom key and mouse clicks, etc.
- Powerful frameworks
Javascript - the language of the web
- Created by Brendan Eich in 10 days - UIUC alum
- EcmaScript Standard, Managed by ECMA International
- Latest version - not fully supported yet in browsers - EcmaScript 6
- Separate from Java - not related at all
- Interactivity is based on events - mouse click, key press, etc
- Can set colors, change styles of elements
var variableName = "some value";
function someFunction() {
alert(variableName);
}
Feelings about Javascript


AIR Bib Creator
Rider Name
AIR Number
Blood Group
Emergency Contact
Emergency Number
Design
Simple
Simple Portrait
Simple Square
With WCCG Logo
With MR Logo
With WCCG & MR Logo
Color Theme
BG
AIR#
Body
Label
Note: Please apply changes before downloading.
Download


Patterns of tabs to close when button is clicked (one per line)
Patterns follow
match patterns rules for WebExtensions
(
<scheme>://<host><path>
).


Numeric restrictions will apply in the input field.
Note: type="number" is not supported in IE9 and earlier.


Hello, world! This is a toast message.
Jira abstraction with markdown
Set jira issue code to build dependency graph
Issue code
Build
History


Review by Lars Lindahl
"Reindeer Games" (2000)
Directed by John Frankenheimer Written by Ehren Kruger
Starring Ben Affleck, Gary Sinise, Charlize Theron, Dennis Farina, Danny Trejo, Clarence Williams III, and Isaac Hayes.
Grade: * (out of four)
What makes Reindeer Games even more disappointing than just a predictable, lifeless action flick is the cast and crew who signed onto the project. Most of the people involved in Reindeer Games just came off terrible slumps with excellent comeback performances in their most recent films. John Frankenheimer, who made the unforgettable Cold War drama The Manchurian Candidate way back in the early 60's, recently showed critics and fans alike that he still had some magic left in his directorial skills when he created the brilliant car chase scenes in Ronin. Screenwriter Ehren Kruger gave a remarkable first impression when he wrote the script for the underrated Arlington Rd. Ben Affleck displayed his comedic talents in Dogma and Boiler Room, and even on Saturday Night Live directly after the horrible Forces of Nature and 200 Cigarettes. Even Charlize Theron, who I was convinced couldn't act, did a decent job in Cider House Rules. When all of these bright stars come together, one would assume that the movie will have great potential to be entertaining. After coming out of the theater showing Reindeer Games, disbelief was my first reaction…then frustration. The movie could have premiered on Cinemax at two in the morning starring an all-rookie cast and still would have been equally lacking in entertaining. The acting is so pitiful and the script so dull, I may have to rethink some of my opinions on how talented these actors actually are. It seemed like everyone was restricted to just the script and not given a fair amount of improvisational rights. We have all seen Ben Affleck have some fun on the set, it is so apparent in a number of scenes in Good Will Hunting when he exchanges jokes with his friends at a bar. The actors in that movie were having fun and, from that, the audience had fun. Reindeer Games is the complete opposite; the actors don't seem to want to be their characters (even Danny Trejo, who will do practically anything, doesn't look like he is enjoying himself), causing the audience to quickly lose interest. Evidence of public disapproval was noted when a woman behind me yelled "Boy this movie is horrible!" and when there were scattered giggles during the most dramatic moments.
The story follows a man, Rudy (Ben Affleck), who is leaving prison in a few days along with his best friend and cellmate Nick (James Frain). Nick has been exchanging love letters with a beautiful woman, Ashley (Charlize Theron), and cannot wait to leave prison and meet her in person. When Nick is killed in a prison riot, Rudy decides to pretend to be Nick and take Ashley out on a date for the Christmas holidays. Unfortunately Ashley's brother, Gabriel (Gary Sinise) interferes with their relationship.  Now Rudy must pretend to be Nick so he can protect Ashley from her brother Gabriel. Confused? The movie only gets more complex as it goes on, until it gets way, way too complex at its climax.
But Reindeer Games makes its first serious mistake during the first half an hour.  Some hero. The point of a hero is to do the morally correct thing, a person for the audience to relate to or admire. When Rudy lies and cheats his way into trouble, instead of gathering support from me for him to ultimately succeed in getting the bad guys, I felt more like he was getting what he deserved and even wanted to see him fail at times. The most irritating part of the film (besides the fact that this is a Christmas movie released in February) is the number of chances the bad guys get to kill Rudy but fail or delay for some idiotic reason. The routine in which the bad guy must explain his genius plan to the hero before attempting to kill him with some elaborate machine is getting so tired nowadays, especially when I eagerly want to see the hero dead.
The only saving grace keeping this movie from receiving zero stars is an amusing but pointless cameo from Isaac Hayes as a prisoner upset with his food. Why he was in this movie for such a pointless role beats me. Why any of these people had anything at all to do with this movie beats me.
Lars Attacks! http://www.angelfire.com/ny3/larsattacks
by Lars Lindahl © 2000 Lars Lindahl
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews