Royal Navy's new aircraft carriers have gone green
HMS Prince of Wales has tested its state-of-the-art pyrolysis plants
by Max ChannonEngineers are testing a green waste disposal system on the Royal Navy's newest aircraft carrier – reducing rubbish aboard one hundred fold.
A Royal Navy spokesperson said: "During the latest phase of HMS Prince of Wales’ sea trials in the North Sea, sailors are testing her state-of-the-art pyrolysis plants – already fitted to the ship’s older sister HMS Queen Elizabeth – to keep the tonnes of rubbish generated by the 600 sailors and 400 industry engineers and experts on board to a minimum.
"During her first two visits to Invergordon, the ship offloaded several tonnes of rubbish – ‘gash’ in Royal Navy parlance – a manual, labour-intensive process given that the 1,000 souls aboard produce upwards of nine tonnes of waste every day , stored in shiny metal drums.
"Enter the two pyrolysis plants – which cause material to decompose under extreme temperatures.
"Most waste – including food, sewage and waste oils, including those from the galley, but not metals or glass – can be processed by the plants, which can deal with 150kg of rubbish every hour.
"The only flame comes from a burner which initially heats the ‘oven’ to 1,100°C – as hot as a large bonfire.
"The waste then becomes the fuel; the burner switches off and the plant becomes self-sustaining using minimum fuel.
"So 150kg of waste becomes 1½kg – or one and a half standard bags of sugar. At the end of the process you’re left with some grey-blackish ash, known as char, which is stored in drums until the ship puts into harbour again and the gash can be offloaded.
As part of the carrier’s trials, the first of the ship’s two pyrolysis plants was fired up to test seals and the operating temperature.
"After that proved successful, the marine engineers are now putting solid waste through the plant before moving on to food, bio sludge (sewage) and sludge oil.
Marine Engineering Technician Niall Kelleher, one of the carrier’s junior sailors, is responsible for operating the rubbish machines.
He said: "I’m excited to start working with the pyrolysis plant which is something I have never seen before.”
“I am hoping to gain more knowledge on such equipment which will be extremely useful to me over my career in the RN.
“The plants’ waste reduction ratio will benefit the ship’s company with us having far less waste to dispose of during ‘out all gash’ – and enable us to store the waste far easier.”
Petty Officer Graeme Coventry added: "The plant should see us processing all waste with the exception of metals and glass – they will both be crushed to enable us to store them without taking up too much space.
LATEST ROYAL NAVY AND ROYAL MARINE NEWS HERE
"This is a fantastic new piece of machinery. It will produce a minimum amount of waste and allow us to reduce our waste by almost 100:1"


SELECT
Section: SQL Commands (7)Updated: 2012-08-14
Index Return to Main Contents
NAME
SELECT, TABLE, WITH - retrieve rows from a table or viewSYNOPSIS
[ WITH [ RECURSIVE ] with_query [, ...] ] SELECT [ ALL | DISTINCT [ ON ( expression [, ...] ) ] ] * | expression [ [ AS ] output_name ] [, ...] [ FROM from_item [, ...] ] [ WHERE condition ] [ GROUP BY expression [, ...] ] [ HAVING condition [, ...] ] [ WINDOW window_name AS ( window_definition ) [, ...] ] [ { UNION | INTERSECT | EXCEPT } [ ALL ] select ] [ ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...] ] [ LIMIT { count | ALL } ] [ OFFSET start [ ROW | ROWS ] ] [ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY ] [ FOR { UPDATE | SHARE } [ OF table_name [, ...] ] [ NOWAIT ] [...] ] where from_item can be one of: [ ONLY ] table_name [ * ] [ [ AS ] alias [ ( column_alias [, ...] ) ] ] ( select ) [ AS ] alias [ ( column_alias [, ...] ) ] with_query_name [ [ AS ] alias [ ( column_alias [, ...] ) ] ] function_name ( [ argument [, ...] ] ) [ AS ] alias [ ( column_alias [, ...] | column_definition [, ...] ) ] function_name ( [ argument [, ...] ] ) AS ( column_definition [, ...] ) from_item [ NATURAL ] join_type from_item [ ON join_condition | USING ( join_column [, ...] ) ] and with_query is: with_query_name [ ( column_name [, ...] ) ] AS ( select ) TABLE { [ ONLY ] table_name [ * ] | with_query_name }
DESCRIPTION
SELECT retrieves rows from zero or more tables. The general processing of SELECT is as follows:
- 1.
- All queries in the WITH list are computed. These effectively serve as temporary tables that can be referenced in the FROM list. A WITH query that is referenced more than once in FROM is computed only once. (See WITH Clause [select(7)] below.)
- 2.
- All elements in the FROM list are computed. (Each element in the FROM list is a real or virtual table.) If more than one element is specified in the FROM list, they are cross-joined together. (See FROM Clause [select(7)] below.)
- 3.
- If the WHERE clause is specified, all rows that do not satisfy the condition are eliminated from the output. (See WHERE Clause [select(7)] below.)
- 4.
- If the GROUP BY clause is specified, the output is divided into groups of rows that match on one or more values. If the HAVING clause is present, it eliminates groups that do not satisfy the given condition. (See GROUP BY Clause [select(7)] and HAVING Clause [select(7)] below.)
- 5.
- The actual output rows are computed using the SELECT output expressions for each selected row. (See SELECT List [select(7)] below.)
- 6.
- Using the operators UNION, INTERSECT, and EXCEPT, the output of more than one SELECT statement can be combined to form a single result set. The UNION operator returns all rows that are in one or both of the result sets. The INTERSECT operator returns all rows that are strictly in both result sets. The EXCEPT operator returns the rows that are in the first result set but not in the second. In all three cases, duplicate rows are eliminated unless ALL is specified. (See UNION Clause [select(7)], INTERSECT Clause [select(7)], and EXCEPT Clause [select(7)] below.)
- 7.
- If the ORDER BY clause is specified, the returned rows are sorted in the specified order. If ORDER BY is not given, the rows are returned in whatever order the system finds fastest to produce. (See ORDER BY Clause [select(7)] below.)
- 8.
- DISTINCT eliminates duplicate rows from the result. DISTINCT ON eliminates rows that match on all the specified expressions. ALL (the default) will return all candidate rows, including duplicates. (See DISTINCT Clause [select(7)] below.)
- 9.
- If the LIMIT (or FETCH FIRST) or OFFSET clause is specified, the SELECT statement only returns a subset of the result rows. (See LIMIT Clause [select(7)] below.)
- 10.
- If FOR UPDATE or FOR SHARE is specified, the SELECT statement locks the selected rows against concurrent updates. (See FOR UPDATE/FOR SHARE Clause [select(7)] below.)
You must have SELECT privilege on each column used in a SELECT command. The use of FOR UPDATE or FOR SHARE requires UPDATE privilege as well (for at least one column of each table so selected).
PARAMETERS
WITH CLAUSE
The WITH clause allows you to specify one or more subqueries that can be referenced by name in the primary query. The subqueries effectively act as temporary tables or views for the duration of the primary query.
A name (without schema qualification) must be specified for each WITH query. Optionally, a list of column names can be specified; if this is omitted, the column names are inferred from the subquery.
If RECURSIVE is specified, it allows a subquery to reference itself by name. Such a subquery must have the form
non_recursive_term UNION [ ALL ] recursive_termwhere the recursive self-reference must appear on the right-hand side of the UNION. Only one recursive self-reference is permitted per query.
Another effect of RECURSIVE is that WITH queries need not be ordered: a query can reference another one that is later in the list. (However, circular references, or mutual recursion, are not implemented.) Without RECURSIVE, WITH queries can only reference sibling WITH queries that are earlier in the WITH list.
A useful property of WITH queries is that they are evaluated only once per execution of the primary query, even if the primary query refers to them more than once.
See in the documentation for additional information.
FROM CLAUSE
The FROM clause specifies one or more source tables for the SELECT. If multiple sources are specified, the result is the Cartesian product (cross join) of all the sources. But usually qualification conditions are added to restrict the returned rows to a small subset of the Cartesian product.
The FROM clause can contain the following elements:
- table_name
- The name (optionally schema-qualified) of an existing table or view. If ONLY is specified, only that table is scanned. If ONLY is not specified, the table and any descendant tables are scanned.
- alias
- A substitute name for the FROM item containing the alias. An alias is used for brevity or to eliminate ambiguity for self-joins (where the same table is scanned multiple times). When an alias is provided, it completely hides the actual name of the table or function; for example given FROM foo AS f, the remainder of the SELECT must refer to this FROM item as f not foo. If an alias is written, a column alias list can also be written to provide substitute names for one or more columns of the table.
- select
- A sub-SELECT can appear in the FROM clause. This acts as though its output were created as a temporary table for the duration of this single SELECT command. Note that the sub-SELECT must be surrounded by parentheses, and an alias must be provided for it. A VALUES [values(7)] command can also be used here.
- with_query_name
- A WITH query is referenced by writing its name, just as though the query's name were a table name. (In fact, the WITH query hides any real table of the same name for the purposes of the primary query. If necessary, you can refer to a real table of the same name by schema-qualifying the table's name.) An alias can be provided in the same way as for a table.
- function_name
- Function calls can appear in the FROM clause. (This is especially useful for functions that return result sets, but any function can be used.) This acts as though its output were created as a temporary table for the duration of this single SELECT command. An alias can also be used. If an alias is written, a column alias list can also be written to provide substitute names for one or more attributes of the function's composite return type. If the function has been defined as returning the record data type, then an alias or the key word AS must be present, followed by a column definition list in the form ( column_name data_type [, ... ] ). The column definition list must match the actual number and types of columns returned by the function.
- join_type
-
One of
-
- *
- [ INNER ] JOIN
- *
- LEFT [ OUTER ] JOIN
- *
- RIGHT [ OUTER ] JOIN
- *
- FULL [ OUTER ] JOIN
- *
- CROSS JOIN
-
For the INNER and OUTER join types, a join condition must be specified, namely exactly one of NATURAL, ON join_condition, or USING (join_column [, ...]). See below for the meaning. For CROSS JOIN, none of these clauses can appear.
A JOIN clause combines two FROM items. Use parentheses if necessary to determine the order of nesting. In the absence of parentheses, JOINs nest left-to-right. In any case JOIN binds more tightly than the commas separating FROM items.
CROSS JOIN and INNER JOIN produce a simple Cartesian product, the same result as you get from listing the two items at the top level of FROM, but restricted by the join condition (if any). CROSS JOIN is equivalent to INNER JOIN ON (TRUE), that is, no rows are removed by qualification. These join types are just a notational convenience, since they do nothing you couldn't do with plain FROM and WHERE.
LEFT OUTER JOIN returns all rows in the qualified Cartesian product (i.e., all combined rows that pass its join condition), plus one copy of each row in the left-hand table for which there was no right-hand row that passed the join condition. This left-hand row is extended to the full width of the joined table by inserting null values for the right-hand columns. Note that only the JOIN clause's own condition is considered while deciding which rows have matches. Outer conditions are applied afterwards.
Conversely, RIGHT OUTER JOIN returns all the joined rows, plus one row for each unmatched right-hand row (extended with nulls on the left). This is just a notational convenience, since you could convert it to a LEFT OUTER JOIN by switching the left and right inputs.
FULL OUTER JOIN returns all the joined rows, plus one row for each unmatched left-hand row (extended with nulls on the right), plus one row for each unmatched right-hand row (extended with nulls on the left).
- ON join_condition
- join_condition is an expression resulting in a value of type boolean (similar to a WHERE clause) that specifies which rows in a join are considered to match.
- USING ( join_column [, ...] )
- A clause of the form USING ( a, b, ... ) is shorthand for ON left_table.a = right_table.a AND left_table.b = right_table.b .... Also, USING implies that only one of each pair of equivalent columns will be included in the join output, not both.
- NATURAL
- NATURAL is shorthand for a USING list that mentions all columns in the two tables that have the same names.
WHERE CLAUSE
The optional WHERE clause has the general form
WHERE conditionwhere condition is any expression that evaluates to a result of type boolean. Any row that does not satisfy this condition will be eliminated from the output. A row satisfies the condition if it returns true when the actual row values are substituted for any variable references.
GROUP BY CLAUSE
The optional GROUP BY clause has the general form
GROUP BY expression [, ...]
GROUP BY will condense into a single row all selected rows that share the same values for the grouped expressions. expression can be an input column name, or the name or ordinal number of an output column (SELECT list item), or an arbitrary expression formed from input-column values. In case of ambiguity, a GROUP BY name will be interpreted as an input-column name rather than an output column name.
Aggregate functions, if any are used, are computed across all rows making up each group, producing a separate value for each group (whereas without GROUP BY, an aggregate produces a single value computed across all the selected rows). When GROUP BY is present, it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions, since there would be more than one possible value to return for an ungrouped column.
HAVING CLAUSE
The optional HAVING clause has the general form
HAVING conditionwhere condition is the same as specified for the WHERE clause.
HAVING eliminates group rows that do not satisfy the condition. HAVING is different from WHERE: WHERE filters individual rows before the application of GROUP BY, while HAVING filters group rows created by GROUP BY. Each column referenced in condition must unambiguously reference a grouping column, unless the reference appears within an aggregate function.
The presence of HAVING turns a query into a grouped query even if there is no GROUP BY clause. This is the same as what happens when the query contains aggregate functions but no GROUP BY clause. All the selected rows are considered to form a single group, and the SELECT list and HAVING clause can only reference table columns from within aggregate functions. Such a query will emit a single row if the HAVING condition is true, zero rows if it is not true.
WINDOW CLAUSE
The optional WINDOW clause has the general form
WINDOW window_name AS ( window_definition ) [, ...]where window_name is a name that can be referenced from subsequent window definitions or OVER clauses, and window_definition is
[ existing_window_name ] [ PARTITION BY expression [, ...] ] [ ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...] ] [ frame_clause ]
If an existing_window_name is specified it must refer to an earlier entry in the WINDOW list; the new window copies its partitioning clause from that entry, as well as its ordering clause if any. In this case the new window cannot specify its own PARTITION BY clause, and it can specify ORDER BY only if the copied window does not have one. The new window always uses its own frame clause; the copied window must not specify a frame clause.
The elements of the PARTITION BY list are interpreted in much the same fashion as elements of a GROUP BY Clause [select(7)], except that they are always simple expressions and never the name or number of an output column. Another difference is that these expressions can contain aggregate function calls, which are not allowed in a regular GROUP BY clause. They are allowed here because windowing occurs after grouping and aggregation.
Similarly, the elements of the ORDER BY list are interpreted in much the same fashion as elements of an ORDER BY Clause [select(7)], except that the expressions are always taken as simple expressions and never the name or number of an output column.
The optional frame_clause defines the window frame for window functions that depend on the frame (not all do). It can be one of
RANGE UNBOUNDED PRECEDING RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING ROWS UNBOUNDED PRECEDING ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWINGThe first two are equivalent and are also the default: they set the frame to be all rows from the partition start up through the current row's last peer in the ORDER BY ordering (which means all rows if there is no ORDER BY). The options RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING and ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING are also equivalent: they always select all rows in the partition. Lastly, ROWS UNBOUNDED PRECEDING or its verbose equivalent ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW select all rows up through the current row (regardless of duplicates). Beware that this option can produce implementation-dependent results if the ORDER BY ordering does not order the rows uniquely.
The purpose of a WINDOW clause is to specify the behavior of window functions appearing in the query's SELECT List [select(7)] or ORDER BY Clause [select(7)]. These functions can reference the WINDOW clause entries by name in their OVER clauses. A WINDOW clause entry does not have to be referenced anywhere, however; if it is not used in the query it is simply ignored. It is possible to use window functions without any WINDOW clause at all, since a window function call can specify its window definition directly in its OVER clause. However, the WINDOW clause saves typing when the same window definition is needed for more than one window function.
Window functions are described in detail in in the documentation, in the documentation, and in the documentation.
SELECT LIST
The SELECT list (between the key words SELECT and FROM) specifies expressions that form the output rows of the SELECT statement. The expressions can (and usually do) refer to columns computed in the FROM clause.
Just as in a table, every output column of a SELECT has a name. In a simple SELECT this name is just used to label the column for display, but when the SELECT is a sub-query of a larger query, the name is seen by the larger query as the column name of the virtual table produced by the sub-query. To specify the name to use for an output column, write AS output_name after the column's expression. (You can omit AS, but only if the desired output name does not match any PostgreSQL keyword (see in the documentation). For protection against possible future keyword additions, it is recommended that you always either write AS or double-quote the output name.) If you do not specify a column name, a name is chosen automatically by PostgreSQL. If the column's expression is a simple column reference then the chosen name is the same as that column's name; in more complex cases a generated name looking like ?columnN? is usually chosen.
An output column's name can be used to refer to the column's value in ORDER BY and GROUP BY clauses, but not in the WHERE or HAVING clauses; there you must write out the expression instead.
Instead of an expression, * can be written in the output list as a shorthand for all the columns of the selected rows. Also, you can write table_name.* as a shorthand for the columns coming from just that table. In these cases it is not possible to specify new names with AS; the output column names will be the same as the table columns' names.
UNION CLAUSE
The UNION clause has this general form:
select_statement UNION [ ALL ] select_statementselect_statement is any SELECT statement without an ORDER BY, LIMIT, FOR UPDATE, or FOR SHARE clause. (ORDER BY and LIMIT can be attached to a subexpression if it is enclosed in parentheses. Without parentheses, these clauses will be taken to apply to the result of the UNION, not to its right-hand input expression.)
The UNION operator computes the set union of the rows returned by the involved SELECT statements. A row is in the set union of two result sets if it appears in at least one of the result sets. The two SELECT statements that represent the direct operands of the UNION must produce the same number of columns, and corresponding columns must be of compatible data types.
The result of UNION does not contain any duplicate rows unless the ALL option is specified. ALL prevents elimination of duplicates.
Multiple UNION operators in the same SELECT statement are evaluated left to right, unless otherwise indicated by parentheses.
Currently, FOR UPDATE and FOR SHARE cannot be specified either for a UNION result or for any input of a UNION.
INTERSECT CLAUSE
The INTERSECT clause has this general form:
select_statement INTERSECT [ ALL ] select_statementselect_statement is any SELECT statement without an ORDER BY, LIMIT, FOR UPDATE, or FOR SHARE clause.
The INTERSECT operator computes the set intersection of the rows returned by the involved SELECT statements. A row is in the intersection of two result sets if it appears in both result sets.
The result of INTERSECT does not contain any duplicate rows unless the ALL option is specified. With ALL, a row that has m duplicates in the left table and n duplicates in the right table will appear min(m,n) times in the result set.
Multiple INTERSECT operators in the same SELECT statement are evaluated left to right, unless parentheses dictate otherwise. INTERSECT binds more tightly than UNION. That is, A UNION B INTERSECT C will be read as A UNION (B INTERSECT C).
Currently, FOR UPDATE and FOR SHARE cannot be specified either for an INTERSECT result or for any input of an INTERSECT.
EXCEPT CLAUSE
The EXCEPT clause has this general form:
select_statement EXCEPT [ ALL ] select_statementselect_statement is any SELECT statement without an ORDER BY, LIMIT, FOR UPDATE, or FOR SHARE clause.
The EXCEPT operator computes the set of rows that are in the result of the left SELECT statement but not in the result of the right one.
The result of EXCEPT does not contain any duplicate rows unless the ALL option is specified. With ALL, a row that has m duplicates in the left table and n duplicates in the right table will appear max(m-n,0) times in the result set.
Multiple EXCEPT operators in the same SELECT statement are evaluated left to right, unless parentheses dictate otherwise. EXCEPT binds at the same level as UNION.
Currently, FOR UPDATE and FOR SHARE cannot be specified either for an EXCEPT result or for any input of an EXCEPT.
ORDER BY CLAUSE
The optional ORDER BY clause has this general form:
ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...]The ORDER BY clause causes the result rows to be sorted according to the specified expression(s). If two rows are equal according to the leftmost expression, they are compared according to the next expression and so on. If they are equal according to all specified expressions, they are returned in an implementation-dependent order.
Each expression can be the name or ordinal number of an output column (SELECT list item), or it can be an arbitrary expression formed from input-column values.
The ordinal number refers to the ordinal (left-to-right) position of the output column. This feature makes it possible to define an ordering on the basis of a column that does not have a unique name. This is never absolutely necessary because it is always possible to assign a name to an output column using the AS clause.
It is also possible to use arbitrary expressions in the ORDER BY clause, including columns that do not appear in the SELECT output list. Thus the following statement is valid:
SELECT name FROM distributors ORDER BY code;A limitation of this feature is that an ORDER BY clause applying to the result of a UNION, INTERSECT, or EXCEPT clause can only specify an output column name or number, not an expression.
If an ORDER BY expression is a simple name that matches both an output column name and an input column name, ORDER BY will interpret it as the output column name. This is the opposite of the choice that GROUP BY will make in the same situation. This inconsistency is made to be compatible with the SQL standard.
Optionally one can add the key word ASC (ascending) or DESC (descending) after any expression in the ORDER BY clause. If not specified, ASC is assumed by default. Alternatively, a specific ordering operator name can be specified in the USING clause. An ordering operator must be a less-than or greater-than member of some B-tree operator family. ASC is usually equivalent to USING < and DESC is usually equivalent to USING >. (But the creator of a user-defined data type can define exactly what the default sort ordering is, and it might correspond to operators with other names.)
If NULLS LAST is specified, null values sort after all non-null values; if NULLS FIRST is specified, null values sort before all non-null values. If neither is specified, the default behavior is NULLS LAST when ASC is specified or implied, and NULLS FIRST when DESC is specified (thus, the default is to act as though nulls are larger than non-nulls). When USING is specified, the default nulls ordering depends on whether the operator is a less-than or greater-than operator.
Note that ordering options apply only to the expression they follow; for example ORDER BY x, y DESC does not mean the same thing as ORDER BY x DESC, y DESC.
Character-string data is sorted according to the locale-specific collation order that was established when the database was created.
DISTINCT CLAUSE
If DISTINCT is specified, all duplicate rows are removed from the result set (one row is kept from each group of duplicates). ALL specifies the opposite: all rows are kept; that is the default.
DISTINCT ON ( expression [, ...] ) keeps only the first row of each set of rows where the given expressions evaluate to equal. The DISTINCT ON expressions are interpreted using the same rules as for ORDER BY (see above). Note that the ``first row'' of each set is unpredictable unless ORDER BY is used to ensure that the desired row appears first. For example:
SELECT DISTINCT ON (location) location, time, report FROM weather_reports ORDER BY location, time DESC;retrieves the most recent weather report for each location. But if we had not used ORDER BY to force descending order of time values for each location, we'd have gotten a report from an unpredictable time for each location.
The DISTINCT ON expression(s) must match the leftmost ORDER BY expression(s). The ORDER BY clause will normally contain additional expression(s) that determine the desired precedence of rows within each DISTINCT ON group.
LIMIT CLAUSE
The LIMIT clause consists of two independent sub-clauses:
LIMIT { count | ALL } OFFSET startcount specifies the maximum number of rows to return, while start specifies the number of rows to skip before starting to return rows. When both are specified, start rows are skipped before starting to count the count rows to be returned.
If the count expression evaluates to NULL, it is treated as LIMIT ALL, i.e., no limit. If start evaluates to NULL, it is treated the same as OFFSET 0.
SQL:2008 introduced a different syntax to achieve the same thing, which PostgreSQL also supports. It is:
OFFSET start { ROW | ROWS } FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLYBoth clauses are optional, but if present the OFFSET clause must come before the FETCH clause. ROW and ROWS as well as FIRST and NEXT are noise words that don't influence the effects of these clauses. In this syntax, when using expressions other than simple constants for start or count, parentheses will be necessary in most cases. If count is omitted in FETCH, it defaults to 1.
When using LIMIT, it is a good idea to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query's rows --- you might be asking for the tenth through twentieth rows, but tenth through twentieth in what ordering? You don't know what ordering unless you specify ORDER BY.
The query planner takes LIMIT into account when generating a query plan, so you are very likely to get different plans (yielding different row orders) depending on what you use for LIMIT and OFFSET. Thus, using different LIMIT/OFFSET values to select different subsets of a query result will give inconsistent results unless you enforce a predictable result ordering with ORDER BY. This is not a bug; it is an inherent consequence of the fact that SQL does not promise to deliver the results of a query in any particular order unless ORDER BY is used to constrain the order.
It is even possible for repeated executions of the same LIMIT query to return different subsets of the rows of a table, if there is not an ORDER BY to enforce selection of a deterministic subset. Again, this is not a bug; determinism of the results is simply not guaranteed in such a case.
FOR UPDATE/FOR SHARE CLAUSE
The FOR UPDATE clause has this form:
FOR UPDATE [ OF table_name [, ...] ] [ NOWAIT ]
The closely related FOR SHARE clause has this form:
FOR SHARE [ OF table_name [, ...] ] [ NOWAIT ]
FOR UPDATE causes the rows retrieved by the SELECT statement to be locked as though for update. This prevents them from being modified or deleted by other transactions until the current transaction ends. That is, other transactions that attempt UPDATE, DELETE, or SELECT FOR UPDATE of these rows will be blocked until the current transaction ends. Also, if an UPDATE, DELETE, or SELECT FOR UPDATE from another transaction has already locked a selected row or rows, SELECT FOR UPDATE will wait for the other transaction to complete, and will then lock and return the updated row (or no row, if the row was deleted). For further discussion see in the documentation.
To prevent the operation from waiting for other transactions to commit, use the NOWAIT option. SELECT FOR UPDATE NOWAIT reports an error, rather than waiting, if a selected row cannot be locked immediately. Note that NOWAIT applies only to the row-level lock(s) --- the required ROW SHARE table-level lock is still taken in the ordinary way (see in the documentation). You can use the NOWAIT option of LOCK [lock(7)] if you need to acquire the table-level lock without waiting.
FOR SHARE behaves similarly, except that it acquires a shared rather than exclusive lock on each retrieved row. A shared lock blocks other transactions from performing UPDATE, DELETE, or SELECT FOR UPDATE on these rows, but it does not prevent them from performing SELECT FOR SHARE.
If specific tables are named in FOR UPDATE or FOR SHARE, then only rows coming from those tables are locked; any other tables used in the SELECT are simply read as usual. A FOR UPDATE or FOR SHARE clause without a table list affects all tables used in the command. If FOR UPDATE or FOR SHARE is applied to a view or sub-query, it affects all tables used in the view or sub-query. However, FOR UPDATE/FOR SHARE do not apply to WITH queries referenced by the primary query. If you want row locking to occur within a WITH query, specify FOR UPDATE or FOR SHARE within the WITH query.
Multiple FOR UPDATE and FOR SHARE clauses can be written if it is necessary to specify different locking behavior for different tables. If the same table is mentioned (or implicitly affected) by both FOR UPDATE and FOR SHARE clauses, then it is processed as FOR UPDATE. Similarly, a table is processed as NOWAIT if that is specified in any of the clauses affecting it.
FOR UPDATE and FOR SHARE cannot be used in contexts where returned rows cannot be clearly identified with individual table rows; for example they cannot be used with aggregation.
-
Caution:
Avoid locking a row and then modifying it within a later savepoint or
PL/pgSQL exception block. A subsequent
rollback would cause the lock to be lost. For example:
BEGIN; SELECT * FROM mytable WHERE key = 1 FOR UPDATE; SAVEPOINT s; UPDATE mytable SET ... WHERE key = 1; ROLLBACK TO s;
After the ROLLBACK, the row is effectively unlocked, rather than returned to its pre-savepoint state of being locked but not modified. This hazard occurs if a row locked in the current transaction is updated or deleted, or if a shared lock is upgraded to exclusive: in all these cases, the former lock state is forgotten. If the transaction is then rolled back to a state between the original locking command and the subsequent change, the row will appear not to be locked at all. This is an implementation deficiency which will be addressed in a future release of PostgreSQL.
- Caution: It is possible for a SELECT command using both LIMIT and FOR UPDATE/SHARE clauses to return fewer rows than specified by LIMIT. This is because LIMIT is applied first. The command selects the specified number of rows, but might then block trying to obtain a lock on one or more of them. Once the SELECT unblocks, the row might have been deleted or updated so that it does not meet the query WHERE condition anymore, in which case it will not be returned.
- Caution: Similarly, it is possible for a SELECT command using ORDER BY and FOR UPDATE/SHARE to return rows out of order. This is because ORDER BY is applied first. The command orders the result, but might then block trying to obtain a lock on one or more of the rows. Once the SELECT unblocks, one of the ordered columns might have been modified and be returned out of order. A workaround is to perform SELECT ... FOR UPDATE/SHARE and then SELECT ... ORDER BY.
TABLE COMMAND
The command
TABLE nameis completely equivalent to
SELECT * FROM nameIt can be used as a top-level command or as a space-saving syntax variant in parts of complex queries.
EXAMPLES
To join the table films with the table distributors:
SELECT f.title, f.did, d.name, f.date_prod, f.kind FROM distributors d, films f WHERE f.did = d.did title | did | name | date_prod | kind -------------------+-----+--------------+------------+---------- The Third Man | 101 | British Lion | 1949-12-23 | Drama The African Queen | 101 | British Lion | 1951-08-11 | Romantic ...
To sum the column len of all films and group the results by kind:
SELECT kind, sum(len) AS total FROM films GROUP BY kind; kind | total ----------+------- Action | 07:34 Comedy | 02:58 Drama | 14:28 Musical | 06:42 Romantic | 04:38
To sum the column len of all films, group the results by kind and show those group totals that are less than 5 hours:
SELECT kind, sum(len) AS total FROM films GROUP BY kind HAVING sum(len) < interval '5 hours'; kind | total ----------+------- Comedy | 02:58 Romantic | 04:38
The following two examples are identical ways of sorting the individual results according to the contents of the second column (name):
SELECT * FROM distributors ORDER BY name; SELECT * FROM distributors ORDER BY 2; did | name -----+------------------ 109 | 20th Century Fox 110 | Bavaria Atelier 101 | British Lion 107 | Columbia 102 | Jean Luc Godard 113 | Luso films 104 | Mosfilm 103 | Paramount 106 | Toho 105 | United Artists 111 | Walt Disney 112 | Warner Bros. 108 | Westward
The next example shows how to obtain the union of the tables distributors and actors, restricting the results to those that begin with the letter W in each table. Only distinct rows are wanted, so the key word ALL is omitted.
distributors: actors: did | name id | name -----+-------------- ----+---------------- 108 | Westward 1 | Woody Allen 111 | Walt Disney 2 | Warren Beatty 112 | Warner Bros. 3 | Walter Matthau ... ... SELECT distributors.name FROM distributors WHERE distributors.name LIKE 'W%' UNION SELECT actors.name FROM actors WHERE actors.name LIKE 'W%'; name ---------------- Walt Disney Walter Matthau Warner Bros. Warren Beatty Westward Woody Allen
This example shows how to use a function in the FROM clause, both with and without a column definition list:
CREATE FUNCTION distributors(int) RETURNS SETOF distributors AS $$ SELECT * FROM distributors WHERE did = $1; $$ LANGUAGE SQL; SELECT * FROM distributors(111); did | name -----+------------- 111 | Walt Disney CREATE FUNCTION distributors_2(int) RETURNS SETOF record AS $$ SELECT * FROM distributors WHERE did = $1; $$ LANGUAGE SQL; SELECT * FROM distributors_2(111) AS (f1 int, f2 text); f1 | f2 -----+------------- 111 | Walt Disney
This example shows how to use a simple WITH clause:
WITH t AS ( SELECT random() as x FROM generate_series(1, 3) ) SELECT * FROM t UNION ALL SELECT * FROM t x -------------------- 0.534150459803641 0.520092216785997 0.0735620250925422 0.534150459803641 0.520092216785997 0.0735620250925422Notice that the WITH query was evaluated only once, so that we got two sets of the same three random values.
This example uses WITH RECURSIVE to find all subordinates (direct or indirect) of the employee Mary, and their level of indirectness, from a table that shows only direct subordinates:
WITH RECURSIVE employee_recursive(distance, employee_name, manager_name) AS ( SELECT 1, employee_name, manager_name FROM employee WHERE manager_name = 'Mary' UNION ALL SELECT er.distance + 1, e.employee_name, e.manager_name FROM employee_recursive er, employee e WHERE er.employee_name = e.manager_name ) SELECT distance, employee_name FROM employee_recursive;Notice the typical form of recursive queries: an initial condition, followed by UNION, followed by the recursive part of the query. Be sure that the recursive part of the query will eventually return no tuples, or else the query will loop indefinitely. (See in the documentation for more examples.)
COMPATIBILITY
Of course, the SELECT statement is compatible with the SQL standard. But there are some extensions and some missing features.
OMITTED FROM CLAUSES
PostgreSQL allows one to omit the FROM clause. It has a straightforward use to compute the results of simple expressions:
SELECT 2+2; ?column? ---------- 4Some other SQL databases cannot do this except by introducing a dummy one-row table from which to do the SELECT.
Note that if a FROM clause is not specified, the query cannot reference any database tables. For example, the following query is invalid:
SELECT distributors.* WHERE distributors.name = 'Westward';PostgreSQL releases prior to 8.1 would accept queries of this form, and add an implicit entry to the query's FROM clause for each table referenced by the query. This is no longer the default behavior, because it does not comply with the SQL standard, and is considered by many to be error-prone. For compatibility with applications that rely on this behavior the add_missing_from configuration variable can be enabled.
OMITTING THE AS KEY WORD
In the SQL standard, the optional key word AS can be omitted before an output column name whenever the new column name is a valid column name (that is, not the same as any reserved keyword). PostgreSQL is slightly more restrictive: AS is required if the new column name matches any keyword at all, reserved or not. Recommended practice is to use AS or double-quote output column names, to prevent any possible conflict against future keyword additions.
In FROM items, both the standard and PostgreSQL allow AS to be omitted before an alias that is an unreserved keyword. But this is impractical for output column names, because of syntactic ambiguities.
ONLY AND PARENTHESES
The SQL standard requires parentheses around the table name after ONLY, as in SELECT * FROM ONLY (tab1), ONLY (tab2) WHERE .... PostgreSQL supports that as well, but the parentheses are optional.
NAMESPACE AVAILABLE TO GROUP BY AND ORDER BY
In the SQL-92 standard, an ORDER BY clause can only use output column names or numbers, while a GROUP BY clause can only use expressions based on input column names. PostgreSQL extends each of these clauses to allow the other choice as well (but it uses the standard's interpretation if there is ambiguity). PostgreSQL also allows both clauses to specify arbitrary expressions. Note that names appearing in an expression will always be taken as input-column names, not as output-column names.
SQL:1999 and later use a slightly different definition which is not entirely upward compatible with SQL-92. In most cases, however, PostgreSQL will interpret an ORDER BY or GROUP BY expression the same way SQL:1999 does.
WINDOW CLAUSE RESTRICTIONS
The SQL standard provides additional options for the window frame_clause. PostgreSQL currently supports only the options listed above.
LIMIT AND OFFSET
The clauses LIMIT and OFFSET are PostgreSQL-specific syntax, also used by MySQL. The SQL:2008 standard has introduced the clauses OFFSET ... FETCH {FIRST|NEXT} ... for the same functionality, as shown above in LIMIT Clause [select(7)], and this syntax is also used by IBM DB2. (Applications written for Oracle frequently use a workaround involving the automatically generated rownum column, not available in PostgreSQL, to implement the effects of these clauses.)
NONSTANDARD CLAUSES
The clause DISTINCT ON is not defined in the SQL standard.
Index
This document was created by man2html, using the manual pages.
Time: 05:34:16 GMT, December 24, 2015


Volodymyr Petrytsya
About
Portfolio
Contact Me
Portfolio
Memory Maker
Staying In Tonight
Weather Dashboard
Daily Scheduler
Coding Quiz
Password Generator


Pediatric behavioral health care integration shows promise
| Source: | Boston University School of Medicine |
| Summary: | A new study finds that, in the first year and a half of the program, children with mental health diagnoses who were served by the TEAM UP sites went for more primary care visits than similar children served by nearby non-participating community health centers. |
Integrating behavioral health services into pediatric primary care in three Boston-area community health centers increased primary care visits by children with mental health diagnoses without raising Medicaid costs.
In mid-2016, the Dimock Center in Roxbury, Codman Square Health Center in Dorchester, and the Lowell Community Health Center began working with Boston University (BU) and Boston Medical Center (BMC) to implement the Transforming and Expanding Access to Mental Health Care in Urban Pediatrics (TEAM UP) model of fully-integrated pediatric behavioral health within primary care, with support from the Richard and Susan Smith Family Foundation.
Now, a new study published in Health Services Research and led by a Boston University School of Public Health (BUSPH) researcher finds that, in the first year and a half of the program, children with mental health diagnoses who were served by the TEAM UP sites went for more primary care visits than similar children served by nearby non-participating community health centers.
Despite this increase in visits, children who were part of TEAM UP did not see a change in avoidable health care utilization (such as emergency department visits or hospitalizations) or in healthcare costs.
"Notwithstanding the direct investment of implementing the intervention, integrating behavioral health into the pediatric medical home for low-income children has measurable value in as little as one and a half years, without further increasing patient spending," says lead study author Dr. Megan Cole, assistant professor of health law, policy & management at BUSPH. "If increased engagement in primary care leads to earlier and improved treatment for children with mental health conditions, longer-term cost savings could result."
An estimated one in five children in the US has a behavioral health issue, but there are major unmet needs and systemic barriers to access care. Progress to integrate behavioral health into pediatric primary care has been slow, especially at centers serving low-income families who already face the greatest barriers.
make a difference: sponsored opportunity
Story Source:
Materials provided by Boston University School of Medicine. Original written by Michelle Samuels.
Note: Content may be edited for style and length.
Journal Reference:
- Megan B. Cole, Qiuyuan Qin, Radley C. Sheldrick, Debra S. Morley, Megan H. Bair‐Merritt. The effects of integrating behavioral health into primary care for low‐income children. Health Services Research, 2019; 54 (6): 1203 DOI: 10.1111/1475-6773.13230
Cite This Page:
| MLA | Boston University School of Medicine. "Pediatric behavioral health care integration shows promise." ScienceDaily. ScienceDaily, 19 November 2019. <www.sciencedaily.com/releases/2019/11/191119161453.htm>. |
| APA | Boston University School of Medicine. (2019, November 19). Pediatric behavioral health care integration shows promise. ScienceDaily. Retrieved November 20, 2019 from www.sciencedaily.com/releases/2019/11/191119161453.htm |
| Chicago | Boston University School of Medicine. "Pediatric behavioral health care integration shows promise." ScienceDaily. www.sciencedaily.com/releases/2019/11/191119161453.htm (accessed November 20, 2019). |


{% extends "base.html" %} {% block title %}Centaurus - Home{% endblock %} {% block head %} {% endblock %} {% block body %}
This service analyzes data to determine the best number of clusters for it, using K-Means algorithm with Mahalanois distance and Bayesian Information Criterion.
Upload your data file here to start a new analysis:
Check the status of a previously submitted job here:
View the report for a previously submitted job here:
Demonstration of the usefullness of this analysis on synthetic datasets:


The aim of art is to represent not the outward appearance of things, but their inward significance.
Aristotle
Hello!
My name is Farcas Nicoleta, a beginner artist in search
of sharing experience with others.
My goal with this blog is to spread ideas of peace and head on a journey to discover ourselves.
All of this is possible with the help of art and spiritual guidence.
A young artist striving for perfection an seeking balance in life trough art and faith.
I put love, time and passion into my art.
I also belive that every day you got to be a better version of your previous self.
Phoenix AZ
If love is lost the fire of passion and energy will slowly fade away so ...
YES do it with love or not at all.
If you like one of my paintings do not hesitate to reach out.


SocketCluster
Design is not just what it looks like and feels like. Design is how it works. — Steve Jobs
Getting Started
·
Documentation


Register
First Name:
Last Name:
Female
Male
Other
Email
Password
Birthday:
Month
Jan
Aug
Day
1
21
Year
1991
1994
I agree to the terms and conditions


The Treemap Visualization presents two levels of detail:
State level : This is the first level where the groups of variables are depicted as rectangles.
The size of the groups depend on the number of variables within.
Variable level : This is the second level of detail, shown through an interactive visual
interface when the user places the mouse on a specific state. In such a case, the detail
of the group is depicted, showing the variables within the state. Also, the variables
are shown in any other state they belong to.
The variables belonging a specific state can be explored clicking on the state to get a zoom of the area of interest.
The original view can be accessed by clicking on the top rectangle with the title "STATES.followed by the number of the state we are exploring".
Both states and variables are shown with a color code. The color code of the treemap visualization is
based on user defined values we call weights. Weights are defined taking into account the
following:
The color of the state is equal to the color of the most relevant variable grouped in
it. This way, in the exploration phase we should first review the states whose color
is closer to red, since those would contain variables marked a-priori as relevant.
The color of the variables represents the relevance of each variable from white to
red going through yellow. The most relevant variables are closer to red and the less
relevant ones are closer to white. The definition of relevance is data specific.
1rs, you need to build a Matlab .mat file with the variables bellow:
"names" cell array of 1xN for N the number of variables. Element i in the cell array should contain a string with the name of variable i.
"states" cell array of 1xN. Element i in the cell array should contain an array with the states variable i belongs to, if any.
"weights" array of doubles 1xN. All the elements are between 0 and 1 and element i represents a level of relevance of the variable i.
You can find a .mat example here with the right variable names and structures. The weights represent the severity in terms of security. These data have been used in the paper J. Camacho, G. Maciá Fernández, J. E. Díaz Verdejo y P. García Teodoro. "Tackling the Big Data 4 Vs for Anomaly Detection". INFOCOM'2014 Workshop on Security and Privacy in Big Data. 2014.
2nd, select this .mat file with the "Choose File" button.
3rd, introduce a threshold for the visualization and press the "Upload" button. This threshold should be between 0 and 1. With this threshold the visualization will only show those variables in the .mat file whose weights are greater than the threshold. If 0 is selected the visualizations will show all the variables in the .mat file.


{% load static %}
Manasaram Architects
Manasaram Architects
Document Search
Document List
Book Search
Books List
LMS
Book Issue Form
Book Submit Form
Book Issue list
{{user.first_name}} {{user.last_name}}
Admin
Logout
BOOK SUBMIT FORM
LIBRARY MANAGEMENT SYSTEM
{{page_msg}}
Name:


the and that (). for are with {background COVID- The :â€". margin not coronavirus this {border-style human px;} #ffffff;} SARS-CoV- al.. animal respiratory This has clinical disease have from been can health humans between coronaviruses more virus However, acute development emerging Health Med One There Vet about animals but medicine new other patients px; severe which {font-family University Virol. against antiviral approach being collaboration feline font-size immune infected its much veterinary Healthâ€� SARS SARS-CoV-. article canine double; enteric highly pandemic research sequence some studies there transmission treatments was â€œOne ./jmv. COVID-. Canine Clin Remdesivir associated border-color border-width cats current disciplines fight hosts importance important knowledge novel pathogens. reservoir solid; spike syndrome than these treatment two viral {font-style {font-weight Clinical CoV- Emerging Jenner Medicine, Severe Veterinary Wuhan, [Epub agent ahead biomarkers bold cats, caused coronavirus. courier; currently did drug em; enhance further horseshoe infectious intermediate many may medicine, out potential print]. progress protein pt;} rapidly repurposed resulting scientific significant species spread strains taking trials use viruses were where will without years zoonotic


About Me
I’m attempting to become a unicorn. A product manager who can write code, design beautiful user experiences, and execute business strategies with aplomb.
Some stuff to know about me: I’m an avid runner, my greyhound’s name is Rocketflamingo, and I’m the only biologist/lawyer/product manager I know.
I live in Washington, DC with my partner and RF - and work at FiscalNote where I work with talented designers and engineers to ship products that drive business growth. I love a good prototype test, and nothing makes me happier than seeing usage stats on my new feature spike.


Ella is starting her own wedding decor rental business. She graduated with her bachelor's degree in business in 2020. Her goal has always been to start her own business that centered around design and weddings.
Ella James wants to hire a developer and website designer to begin marketing her wedding decor business. Her preference is to hire an individual who can both develop and design a perfectly curated website for her business. She wants it to be just as aesthetically pleasing as her instagram business page that she spent a long time and effort designing. Therefore, someone with a passion for design would be an ideal candidate. It's not necessary but she would like to work with someone who has a touch of wedding industry knowledge because they may better understand her needs and passion. My professional portfolio demonstrates my background in wedding design and marketing. It will display my work and certification as a fullstack developer. As well as demonstrating my love for all things design.
As recent college graduate and aspiring small business owner, Ella doesn't have a large budget for her website. She hasn't found a developer or website designer that will work within her budget. She attempted to create her own website on wix but it doesn't look anything like what she envisions. Her main frustration is that she posted a website launch reveal on her instagram page, 1 month from now and no one has that quick of a turnaround time. It has been a daunting task to find someone who can create her website, understands her design needs, all within a 1 month timeframe and within her budget.
Ella wants to visit a professional portfolio website that demonstrates their work with visuals. Visuals catch her eye more than text. She wants to be able to see their prior work background and get a sense of the candidate's personality in hopes it will mesh well with hers. Being able to save a resume for her records is ideal so that she can look back at it when selecting the perfect candidate.
This project will be a professional portfolio website with a focus on web design and digital marketing.
This is a personal website designed with a professional audience in mind. While appealing to those who just want to build their network, bloggers, and even content creators. My target audience includes but not limited to: hiring managers, startup founders, businesses seeking freelance work, recruiters, professional peers, and bloggers.
My overarching goal for this site is to share my skill set and passion with the public while visually portraying who I am on a personal level. I want to come across in both a professional but fun-loving manner. The site will allow potential employers to know who I am professionally and personally. My secondary goal is for users to contact me with any collaborations they have in mind, whether that be in the tech, design, or blogging space.


--- layout: archive title: Zulip Chat Archive permalink: /stream/113488-general/topic/parsing.20question.html ---
tl;dr: what are the binding powers of Π _ , _
and →
in Lean's dependent type theory? Does this even make sense?
I have around seven half-written blog posts and I'm trying to get some of them into shape (all part of my general moving back into more active Leaning now my course has finished). One of them is a very basic post about how you can use #print notation
to figure out how Lean will parse a + b * c
or a ^ b ^ c
(i.e. how to find out that *
has a higher precedence than +
, and that ^
is right associative).
But I can't work out how to figure out how ∀ n : ℕ, n = 1 → true
will parse, short of just tying it. The problem is that ∀
and →
are not notation, they are inbuilt. The comma is also playing a role here, I think; I guess it's part of the forall notation. Are there binding powers at play here that I cannot see? I know forall = Pi. I don't know how to work out whether the example is (∀ n : ℕ, n = 1) → true
or ∀ n : ℕ, (n = 1 → true)
(short of trying it and discovering that it's the latter). There is presumably some underlying logic here.
#print notation -> -- _ `->`:25 _:24 := #1 → #2
If →
is supposed to be exactly the same as ->
then this is evidence that the binding power of →
is 25 (and the 24 agrees with the fact that →
is right associative). But I don't know how to check this short of experimentation.
`Σ`:1024 binders `,`:0 (scoped 0) := #0` `Pi`:0 binders `,`:0 (scoped 0) := #0
Vast difference there between Pi
and Σ
.
From init/core: def std.prec.arrow : nat := 25
ha ha, that does look relevant doesn't, it -- although I don't know for sure that this is referring to →
it certainly looks like it is. So what about Π
and ∀
? Here's another one I found in #print notation
output:
`∃`:1024 binders `,`:0 (scoped 0) := #0`
which agrees with Σ
.
OK so the only difference between the 1024 and the 0 with Σ
and Pi
is that Σ
eats the binders more greedily, and it seems to me that binders are highly restricted in what they can look like, so all attempts so far by me to construct terms Σ X, Y
and Π X, Y
that parse seriously differently have failed. I think the key point is the ``,``:0
which just means in practice "evaluate everything after this first and then eat the lot" for both sigma and pi.
I don't think that first binding power matters at all. Binders are a special parse mode anyway, it's not clear how binding power would affect it. The important part is the `,`:0
, which means that everything after the comma is parsed with very low precedence, so that \all x, P op Q
will always parse as \all x, (P op Q)
for any choice of op
.
Precedence on tokens sets their left-binding power, i.e. influences things to their left. For example, the first precedence means that a × Π X, Y
does not parse while a × Σ X, Y
does, because 1024 is higher than the rbp 34
specified in the prod
notation. Similarly, the rbp 0
on the scoped
action is actually the part that influences how far a Pi or Sigma stretches after the comma. Precedences on tokens after the first one in a notation, like ,
here, shouldn't matter in general.
(Does it make sense that Pi and Sigma do not have the same precedences? Probably not)
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


What is Reticulum?¶
Reticulum is a cryptography-based networking stack for wide-area networks built on readily available hardware, and can operate even with very high latency and extremely low bandwidth. Reticulum allows you to build very wide-area networks with off-the-shelf tools, and offers end-to-end encryption, autoconfiguring cryptographically backed multi-hop transport, efficient addressing, unforgeable packet acknowledgements and more.
Reticulum is a complete networking stack, and does not use IP or higher layers, although it is easy to utilise IP (with TCP or UDP) as the underlying carrier for Reticulum. It is therefore trivial to tunnel Reticulum over the Internet or private IP networks. Reticulum is built directly on cryptographic principles, allowing resilience and stable functionality in open and trustless networks.
No kernel modules or drivers are required. Reticulum runs completely in userland, and can run on practically any system that runs Python 3.
Current Status¶
Reticulum should currently be considered beta software. All core protocol features are implemented and functioning, but additions will probably occur as real-world use is explored. There will be bugs. The API and wire-format can be considered relatively stable at the moment, but could change if warranted.
Caveat Emptor¶
Reticulum is an experimental networking stack, and should be considered as such. While it has been built with cryptography best-practices very foremost in mind, it has not been externally security audited, and there could very well be privacy-breaking bugs. To be considered even remotely secure, Reticulum needs a very thourough security review by independt cryptographers and security researchers. If you want to help out, or help sponsor an audit, please do get in touch.
What does Reticulum Offer?¶
Coordination-less globally unique adressing and identification
Fully self-configuring multi-hop routing
Asymmetric X25519 encryption and Ed25519 signatures as a basis for all communication
Forward Secrecy with ephemereal Elliptic Curve Diffie-Hellman keys on Curve25519
Reticulum uses the Fernet specification for encryption
AES-128 in CBC mode with PKCS7 padding
HMAC using SHA256 for authentication
IVs are generated through os.urandom()
Keys are ephemeral and derived from an ECDH key exchange on Curve25519
Unforgeable packet delivery confirmations
A variety of supported interface types
An intuitive and easy-to-use API
Reliable and efficient transfer of arbritrary amounts of data
Reticulum can handle a few bytes of data or files of many gigabytes
Sequencing, transfer coordination and checksumming is automatic
The API is very easy to use, and provides transfer progress
Efficient link establishment
Total bandwidth cost of setting up a link is only 3 packets, totalling 240 bytes
Low cost of keeping links open at only 0.62 bits per second
Where can Reticulum be Used?¶
Over practically any medium that can support at least a half-duplex channel with 1.000 bits per second throughput, and an MTU of 500 bytes. Data radios, modems, LoRa radios, serial lines, AX.25 TNCs, amateur radio digital modes, ad-hoc WiFi, free-space optical links and similar systems are all examples of the types of interfaces Reticulum was designed for.
An open-source LoRa-based interface called RNode has been designed specifically for use with Reticulum. It is possible to build yourself, or it can be purchased as a complete transceiver that just needs a USB connection to the host.
Reticulum can also be encapsulated over existing IP networks, so there’s nothing stopping you from using it over wired ethernet or your local WiFi network, where it’ll work just as well. In fact, one of the strengths of Reticulum is how easily it allows you to connect different mediums into a self-configuring, resilient and encrypted mesh.
As an example, it’s possible to set up a Raspberry Pi connected to both a LoRa radio, a packet radio TNC and a WiFi network. Once the interfaces are configured, Reticulum will take care of the rest, and any device on the WiFi network can communicate with nodes on the LoRa and packet radio sides of the network, and vice versa.
Supported Interface Types and Devices¶
Reticulum implements a range of generalised interface types that covers most of the communications hardware that Reticulum can run over. If your hardware is not supported, it’s relatively simple to implement an interface class. Currently, the following interfaces are supported:


client request rewriting library for nodejs and the browser
This is a work-in-progress project. For more info, see Roadmap
This was primarily inspired by Wombat, but it was built for recording (Pademelon can be used for recording, given custom (un)rewriteUrlIntercept functions that add a timestamp) and not proxying and had the following issues:
location.href
/location.pathname
is not rewritten, breaking websites that use react.localStorage
is not stored in respective sitesmodifiedFunc.toString
is not faked, so websites that validate functions are broken (or something, depends on the website)The goal of this project is to provide easy, understandable client rewriting tools to nodejs developers.
Primarily used as a URL proxy (like https://someproxysite.com/https://destinationwebsite.com
).
Rewrites XMLHttpRequest
, fetch
, history
, postMessage
, Worker
, and more (for the full list see Roadmap, all to make sure everything is to be served under https://someproxysite.com/https://destinationwebsite.com
and not https://destinationwebsite.com
.
This package can be used in nodejs (for reversing rewritten URLs and rewriting js/css files) or the browser.
const Pademelon = require('pademelon').Pademelon;
const proxyPrefix = 'https://someproxysite.com/prefix/';
const rewriter = new pademelon.Rewriter(proxyPrefix, options);
// "rewrite" is defined as rewriting the asset url/link from https://originalurl.com to https://someproxysite.com/https://originalurl.com
// rewriting behavior can be defined in options
// rewrite JavaScript files
let rewrittenJS = rewriter.rewriteJavaScript(javascriptText);
// rewrite CSS files
let rewrittenCSS = rewriter.rewriteCSS(cssText);
// rewrite HTML files
let rewrittenHTML = rewriter.rewriteHTML(htmlText);
// rewrite absolute URLs
let rewrittenAbsoluteUrl = rewriter.rewriteUrl(absoluteURL);
// reverse rewrite
let unrewrittenAbsoluteUrl = rewriter.unrewriteUrl(rewrittenAbsoluteUrl);
<!-- this isn't really necessary because injection is already taken care of by nodejs's rewriteHTML -->
<head>
<script src='pademelon.min.js'></script>
<script>
window.pademelonInstance = new Pademelon({
hostname: window.location.host,
pathnamePrefix: "/pademelonprefix/",
windowProp: "pademelonInstance"
});
pademelonInstance.init()
</script>
</head>
Following is the current functions that are implemented with their unit tests written.
Todo list:
add getter/setter rewriters
add more todo rewrite functions to todo rewrite list
todo rewrite list
Info about contributing can be found here.
Generated using TypeDoc


Capture
Status:
img:
video:
Cordova Capture API
Actions
Capture 10 secs of audio and play
Capture 1 image
Capture 10 secs of video
Back


The average actual time (in minutes) spent on each log entry type this month.
Enter some data in order to see the graph!
The average HR value per log entry (excluding rest days) this month.
Enter some data in order to see the graph!
The average Watts value per log entry (excluding rest days) this month.
Enter some data in order to see the graph!
The number of log entries a given RPE level relates to (excluding rest days).
Enter some data in order to see the graph!
The amount of time (in minutes) spent in each zone (excluding rest days) this month.
Enter some data in order to see the graph!
The number of log entries a given Sleep Quality relates to.
Enter some data in order to see the graph!


prosecode¶
by Don Sheehy
The prosecode Python library was written to support a literate programming workflow. It allows one to take markdown files with fenced code blocks and execute the code in the blocks. Then, the code blocks are tangled out into separate files of pure source code. Moreover, the markdown files are converted into LaTeX.
prosecode was written primarily as a way to build the book A First Course on Data Structures in Python. This open source textbook can be found at https://donsheehy.github.io/datastructures/.


|
|
Designing protection
By
Rachel Powell Norton
December 13, 1996, 6 p.m. PT
Digital watermarking is a popular new way to protect copyrighted images. Here's how it works:
|
|
Step 1:
Watermark created
The image creator "stamps" the image with a digital watermark that can only be viewed with special software. The watermark takes advantage of the variation between pixels to remain all but invisible in the finished product.
The watermark can still be detected with the special software even after the image is printed and rescanned.
|
Step 2:
Watermark
detected
Image editors such as Photoshop 4.0 are equipped with the special software. The application can detect the subtle pixel variation and interprets it as a watermark, alerting the viewer that the image has been copyrighted.
|
Step 3:
Artist identified
In addition to marking the image as a copyrighted work, the watermark contains information on how to contact the artist.
|
| Source: PictureMarc Software
|
|
|
|


Speaker: Alessandro Giorgetti / @a_giorgetti
Founder of SID s.r.l. - a company that works for the HealthCare system.
One of the founders of DotNetMarche.
Involved in DevMarche.
I've heard I can write Windows Store Apps using Javascript...
And someone mentioned WinJS...
A very good move to bring web develpers to the dark side of native apps development.
A toolkit to develop Windows Store (Universal) Apps using Javascript and Html:
The approach is very similar to what KnockoutJS does.
Everything is based on the Observer Pattern.
Plain Javascript objects need to be wrapped for the UI to react to objects' property changes ( WinJS.Binding.as({...}) ).
Arrays and lists need to be wrapped by WinJS.Binding objects ( WinJS.Binding.List([]) ).
If you want to bind to a function you need to wrap it using another object ( WinJS.Utilities.markSupportedForProcessing() ).
Native bindings are one way only! You need to extend the framework to support two way bindings.
It seems there are lots of things to learn before I can use it effectively.
var product = WinJS.Binding.define({
ProductName: "",
IsSelected: false
});
ui.Pages.define("/pages/ListViewBinding/listViewBinding.html", {
listData: null,
ready: function (element, options) {
this.listData = new WinJS.Binding.List([]); // list wrapper
this.createDummyData();
var listview = element.querySelector(".mainList").winControl;
// setup the listview
listview.itemDataSource = this.listData.dataSource;
listview.itemTemplate = element.querySelector(".itemtemplate");
listview.oniteminvoked = this.itemInvoked.bind(this);
listView.layout = new ui.GridLayout();
},
itemInvoked: function (args) {
var dataItem = this.listData.getAt(args.detail.itemIndex);
dataItem.IsSelected = !dataItem.IsSelected;
},
createDummyData: function () {
// Creating dummy data .
//Data format : Name, Quantity, SalesPrice, IsSelected
this.listData.push(WinJS.Binding.as(new product({ ProductName: "Product1", IsSelected: false })));
this.listData.push(WinJS.Binding.as(new product({ ProductName: "Product1", IsSelected: false })));
this.listData.push(WinJS.Binding.as(new product({ ProductName: "Product1", IsSelected: false })));
}
});
// somewhere in code ...
WinJS.Binding.processAll(htmlElement, viewModel);
Is it really required to use (All) WinJS to write a Windows Store App in Javascript?
...according to Microsoft documentation...
You can just use plain standard HTML controls and the DOM to interact with the UI.
Nonetheless WinJS has some very usefull features (like the standard controls we've talked about) that makes it pretty useful.
The idea is to keep what we consider useful and swap out / replace what we do not need.
| WinJS | AngularJs | KnockoutJs | |
|---|---|---|---|
| Language patterns (classes, namespaces, promises...) | yes | partial | no |
| App patterns (MV*, IoC/DI) | partial | yes | yes |
| Bindings / SPA | 1-way / custom | yes / custom | yes / custom |
| (Windows) Controls | yes | custom directives | custom bindings |
I'm lucky...
It gives me lots of tools I'm already familiar with.
How long will it takes to me to have WinJS and AngularJS cooperate in a way that I can be productive writing Windows Store (Universal) Apps?
Setting up a Universal App Javascript project with AngularJs
AngularJS Directives are way different than standard WinJS Controls
Using standards controls with the Angular-Winjs library
Of course you can also create your very own wrappers around standard UI controls and services.
You have loads of options:
Navigation frameworks, make them work together!
We have 2 options (Windows.Storage.ApplicationData):
Run, Suspend, Terminate...


JavaScript Arrays
By Manwinder Singh
Show Numbers
Language Names
Language Popularity
Below is the link to my homepage and theme site
Homepage Site
Theme site


Version: Final 20200118
September 2019
Authors
Christian Felde, Mohamed Elshami
Abstract
This paper introduces the AKA protocol and AKAP registry smart contract, deployed on Ethereum networks. The primary benefit of AKAP is the ability to externalize and manage shared references and data, in addition to the discovery of these resources through hierarchical structure.
On the AKAP registry contract users may register nodes as part of a directed acyclic graph and then associate values and data on these nodes. Nodes are identified using a hashing technique combining their structural context and user selected identifiers.
The ownership of these nodes can be transferred and managed according to the ERC-721 Non-Fungible Tokens standard, enabling reuse of existing tools and procedures.
Table of Contents
1 Introduction
2 The AKA protocol
2.1 Creating and updating a node and its values
2.2 Node attributes
2.3 Transfer ownership of nodes
3 Implementation considerations
3.1 Identifying a node
3.2 Considerations and limits excluded from the label requirements
4 Reclaiming dead entries
5 Minimizing contract risk
6 Summary
Only the simplest of software can function without interacting with other external components or holding shared global configuration values. We also see this within the realm of Ethereum smart contracts, with contracts, for example, holding an address reference to another contract.
This works fine in simple setups, but becomes difficult to manage when the number of references and locations of these references grow. It becomes time consuming, error prone, and difficult to manage.
The AKA protocol (AKAP) proposes a solution to this and other similar use cases by allowing anyone to register what we call a node. A node is simply an entry stored in the AKAP registry contract, following certain well-defined procedures, and is initially owned and control by whomever creates the node.
On a node, various types of attributes can be associated. This, for example, allows us to solve the above reference use case by now having these contracts look up the address rather than directly storing the address. Having this, we can now update this reference at a single location in the AKAP registry contract, rather than individually updating each and every direct reference.
Another use case could be for application specific configuration to be stored within the AKAP node. Assume for example that a certain dApp consists of a number of smart contracts. [1] For various reasons it might be beneficial for the administrator of this application to temporarily pause any interaction with these smart contracts. We can image each contract holding a flag value indicating this setting. Setting this flag to false causes any user interaction with the contract to stop from being allowed. If the administrator is required to update multiple flags across multiple contracts this causes maintenance issues from the complexity of interacting with multiple contracts. It also potentially causes transactional issues; What happens if some contracts have the flag set to false while other contracts still maintain the flag as true? Does it cause potential inconsistencies in contract state?
Instead of having separate flags across each of these dApp contracts we can imagine putting the configuration flag on a node in the AKAP registry contract. Each of the separate dApp contracts can then all reference this AKAP node when they need to check the flag value. This solves the complexity described by now only needing to update the flag in one location. And later, when new contracts are added whom also depend on this flag, these only need to reference the same existing node, rather than the value directly.
Thirdly, because nodes sit in a directed acyclic graph structure, we can foresee a use case where a contract is given a node reference from which it will navigate to other nodes using known relative relationships. From this we can build large and complex setups of any size, all of which can be atomically updated from a single node reference.
It is not hard to imagine additional use cases but we hope these are sufficient to illustrate the value of having such a centralized registry for managing such setups. We’ll also detail these with example source code.
While there are use cases and arguments we could imagine for why you’d want to deploy multiple instances of the registry, we think there are clear network benefits to having one shared AKAP registry. Just like the internet benefits from one shared IP address space or a global DNS, having one central registry allows anyone and everyone to build on a shared reference setup. The question then becomes: Why should anyone trust and build upon any such “centralized location,” as represented by the AKAP registry contract? We answer this question by arguing for no administrator functionality because it guides us towards guaranteed and consistent behaviour.
Firstly, for the registered nodes we mention above, these always have clear owners: Initially those that created a node become the owner. Ownership can later be transferred using the ERC-721 Non-Fungible Tokens standard. [2] Only current owners can modify a node.
More importantly however: There is no special user or owner of the AKAP registry contract itself. It is never possible for anyone with some kind of special access to override any aspect of the nodes, values on the nodes, or the hierarchical structure they sit within. There simply is not any such functionality in the AKAP registry contract. This can be verified by looking at the source code and byte code of the deployed contract.
This also implies that there is no functionality to “upgrade” the AKAP registry contract. The code that is deployed will forever remain. Naturally this does increase our risk for bugs which cannot be fixed, and we’ll discuss the approach to minimize such issues later in this paper.
The premissionless nature also guarantees that it will forever be free to use and interact with the AKAP registry contract beyond required transaction costs. [3] While a paid feature often motivates the removal of dead entries from a registry, we don’t have that luxury since it is free to use AKAP. As an alternative pruning mechanism, we’ll look at the reclaim functionality which allows dead entries to be taken out of service by anyone willing and capable to do so.
AKA, as you might have guessed, is an abbreviation for “also known as,” used when someone or something has another name. This is in essence what the AKA protocol, or AKAP, lets you do. By associating some identifier with another value of some kind you are creating a reference to this value. We call this reference a node, and a node consists of a parent node, a user defined identifier, in addition to various attributes. Each node can have an unlimited number of child nodes.
No node exists until it is first created. Creating a node involves calling a function on the AKAP registry contract. This process is known as claiming a node.
All nodes are created having another node as its parent. Child nodes can only be created by the owner of the parent node. The exception to this is the parentless root node, on which anyone can claim ownership of new child nodes. We illustrate such a structure next.
Here we find that Node A and Node B are children of the special root node. Someone claimed Node A and Node B on a first come first serve basis. They are now owners of these nodes unless they transfer ownership to someone else.
Node C was created by the then owner of Node A. While initially the owner of Node C was equal to that of Node A at the time Node C was created, the owner of newly created Node C can immediately transfer ownership to anyone else. The ownership of Node C does not change if the ownership of Node A is modified.
Only the owner of Node A could create Node C. As soon as ownership of Node C is transferred, it is no longer possible for the current or any future owner of Node A to influence any aspect of Node C. This does assume that Node C is kept alive as we will discuss later.
The structure you see above remains immutable at all times. It is never possible to change the parent of, say, Node C, or any other node for that matter. There is however an attribute on nodes called SeeAlso which can be used as pointers to other nodes. The SeeAlso attribute does not however change any structural relationship.
A few different attributes can be defined on each node. With the exception of the parent identifier and the self-identifier, the remaining attributes can be updated and set by the current owner of the node. Each attribute also contains metadata related to who set the value and when it was defined.
Nodes may hold the following attributes:
As mentioned, both parent and node identifiers are set at node creation. These remain immutable from there on. We’ll focus on how these are defined later in the implementation section of this paper.
The remaining attributes, Node owner, SeeAlso, SeeAddress and NodeBody, may all be set by the owner of the node. Besides being defined as a blob of bytes, NodeBody does not otherwise dictate any specific structure. It is left to the application of NodeBody to define the meaning of these.
Each node, as described above, is to be regarded as a Non-Fungible Token. These tokens can be transferred to new owners as defined by the ERC-721 interface.
At its core the AKAP registry is a mapping between a derived identifier and related attributes. Public functions exposed on the registry smart contract allow for anyone to interact with the registry according to the protocol definition.
Much of the functionality relating to ownership and transfer rights are directly taken from ERC-721, with only some registry functionality needed on top.
Maybe the most distinctive functionality in the registry is the claiming process. This is the method used to create new nodes and for owners of existing nodes to maintain their ownership.
We take inspiration from the namehash algorithm as found on EIP-137. [4] While not exactly the same it works on similar principles. Creating a node identifier follows these steps:
The process of claiming a node requires a caller to pass in the parent identifier, of type uint256, and a label sequence, of type bytes. The label sequence may contain any sequence of bytes assuming they have a length between 1 and 32 bytes.
The byte sequence is hashed using keccak256, and this produces a bytes32 output, which we call labelHash.
The hashed output of the byte sequence, labelHash, together with the parent identifier is again hashed using keccak256(abi.encode(parentId, labelHash)).
The output of step 3 is converted to uint256 and also used as the NFT id.
While the output from step 3 is of type bytes32, when converting this to uint256 on step 4 there is no loss of precision as a bytes32 will fit in uint256. We use uint256 as that is what identifies an ERC-721 token.
Interacting with a node, for example when setting or getting its attributes, will only require the uint256 identifier. Utility methods are provided on the registry contract to easily produce the identifier for your convenience.
In the previous section we only specified that the sequence of bytes used as a node label needs only to conform to a length between 1 and 32 bytes. No other requirements are given.
This is different from what you can find in EIP-137, which also takes into consideration standardization requirements like UTS46. [5] It is understandable why you’d need such a standardization of domain names as they are intended to be human readable.
However, there is no direct requirement for node labels in AKAP to be human readable. It is left to the applications building on top of AKAP to put additional requirements as they see fit for their domain and subset of the node structure.
While it is never possible to remove a node after it has been created, there is a potential issue with having “dead nodes” that no one can touch. It might be that the identifier holds value others could find useful but can no longer make use of because the node owner might have lost access to or interest in their own account.
If we had a mechanism of payments, a lack of future payments would automatically cause a node to expire. However, in the interest of maximising utility of the AKA protocol, no such payment is taken when claiming a node.
As a compromise, nodes will need to periodically be reclaimed in order for them to remain with the current owner. As long as nodes are reclaimed within 1 year, they are renewed for another year going forward. The owner is free to reclaim nodes as often as they want, with each reclaim causing the expiry date to be set 1 year into the future in relation to the invocation time of the reclaim. The function of reclaiming is equal to the function used to claim the node initially. A reclaim does not alter any other aspect of a node.
If a node has expired, i.e., the owner did not reclaim the node within a year, the node is again up for grabs by the parent node owner, just like we had initially when the node was first created. And if the parent happens to be the special root node, then anyone may claim such an expired node.
In the case of the parent node not being the special root node, this does allow for the flexibility to not reclaim child nodes, assuming the owner maintains the ownership of the parent. This is because an expired node remains fully functional, with its current values and attributes fully available.
Throughout this paper we have specified a minimal set of features for what can be stored on nodes in a directed acyclic graph. We have explicitly highlighted that we avoid complicated validation of node data, and rather leave this to the application. We reduce the scope to concern ourselves with simple structural integrity.
This has all been done on purpose because we think this is the simplest and safest method of eliminating potential bugs and issues. If functionality can be placed outside the registry contract we opt to do so.
Also, for the features required by ERC-721 we will opt to base this part of the implementation on existing source code, specifically that from OpenZeppelin, as we regard this as battle tested and well-reviewed code. [6]
This of course does not remove the need to perform reviews, testing and verifications of the concrete AKAP registry contract source code, which we invite everyone to participate in on our source code repository.
Allowing for more than sufficient time on this phase is important when we aim to not include any form of upgrade or administration features in the contract.
Throughout this paper we have presented the need for and functionality of the AKAP registry.
We argue that such a registry should be free to use in order to maximize utility. We also argue that there are network benefits with having one centralized registry, and we address the centralization risk by proposing that there should be no administrator feature on the contract.
We believe the AKA protocol is of genuine benefit to a lot of use cases and invite you to participate in the design, implementation and testing/verification process.
[1] https://www.coindesk.com/information/what-is-a-decentralized-application-dapp
[3] https://github.com/ethereum/wiki/wiki/White-Paper#messages-and-transactions
[4] https://eips.ethereum.org/EIPS/eip-137
[5] https://unicode.org/reports/tr46/
[6] https://github.com/OpenZeppelin/openzeppelin-contracts/tree/master/contracts/token/ERC721


The Keras Tutorial - Introduction
Keras is a high-level neural networks library written in Python and built on top of Theano or Tensorflow. That means you need one of them as a backend for Keras to work. I have been working with Neural Networks for a while, I have tried Caffe, Tensorflow and Torch and now I’m working with Keras. Its main advantage is the minimalism of the code as it allows the creation of big networks with a few lines of code. It allows multi-input and multi-ouput networks, convolutional and recurrent neural networks, embeddings, etc. I’m really comfortable working with it so I thought it would be nice to write some paragraphs to explain how it works and the tricks I found. I hope this tutorial is helpful for new users.
Index
- Download and Install Keras
- Backend
- Our first Neural Network
- The basic layers
Download and Install Keras
Before we actually install Keras let’s make our life easier and install the pip program.
sudo apt-get install python-pip
Now we can easily install the dependencies:
pip install numpy scipy scikit-learn pillow h5py
Next we install Keras:
pip install keras
You can check the version of Keras you are using by typing the following command in the terminal:
python -c "import keras; print keras.__version__"
Moreover, you can upgrade Keras with the following command:
sudo pip install --upgrade keras
Backend
As I mentioned earlier, Keras works on top of Tensorflow (by default) or Theano. I tend to use Theano as my backend. To specify which backend we want to use we have to edit the keras.json file:
nano ~/.keras/keras.json
Example of the content of my keras.json file using Theano:
{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "theano"
}
In the ‘backend’ variable we can specify ‘theano’ or ‘tensorflow’. The ‘image_dim_ordering’ (‘th’ for theano and ‘tf’ for tensorflow) is used to specify the arrangement of the dimensions in images, i.e. in Theano we use the order (channels, width, height) whereas in Tensorflow the order is (width, height, channels). In the case of theano, you have to create a file ‘~/.theanorc’:
touch ~/.theanorc
And include the following:
[global]
floatX = float32
device = gpu
optimizer = fast_run
[lib]
cnmem = 0.9
[nvcc]
fastmath = True
[blas]
ldflags = -llapack -lblas
If you want to use the CPU change ‘gpu0’ to ‘cpu’. With this final steps we should have keras ready to work with Theano.
Our first Neural Network
First I will say that Keras has two different structures to create Neural Networks: the Sequential Model and the Functional API. We will work with the second one through the tutorial as it allows more freedom. If you have heard about the Graph Model I have to say that it was removed (therefore it’s now deprecated) in the version 1.0 (link).
So let’s start coding! Our first neural network is going to have a single input and a final dense layer (which will be the output). The example code can also be shown in the Keras Models section but we will go through each of the lines to understand better what we are doing.
from keras.models import Model
from keras.layers import Input, Dense
a = Input(shape=(32,))
b = Dense(32)(a)
model = Model(input=a, output=b)
The first two lines are the imports. No need for explanation, we need to import all the layers, optimizers, functions for the initilisation of layers, etc. we want to use.
The Functional API forces you to include an input layer. Here you have to specify the shape of your input (without the batch size). In this example we only have a 1D input of 32 values. The comma after the value 32 is also mandatory in some cases to avoid errors. Your input layer is stored in the variable ‘a’, so you can use this variable as input to other variables to create links (this allows the network to branch out easily).
Now we include a Dense layer by calling the function and providing the number of neurons for that layer (in this case 32). The next step is to stack the output layer or dense layer on top of the input layer, i.e., we have to connect them. In this type of model we do this by providing the variable of the last layer at the end of the new layer (check the ‘a’ between parenthesis after the Dense layer).
Finally, we have to instantiate the Model or create a container for it. We can do this with the Model function, providing the input and output. In this case providing the variables of the input and output layers. We can also provide a Python list for multi-input and output.
a = Input(shape=(32,))
b = Input(shape=(32,))
...
z = merge([a,b], mode='concat', concat_axis = -1)
...
c = Dense(32)(z)
model = Model(input=[a, b], output=c)
This is an example of a multi-input neural network, in this case a and b. At some point in the code we have to fuse both of them into one stream, which we cal z. Finally, we will output c.
Anyway, now that the model is created we have to configure it. To do so we have to call the function ‘compile’ of the model:
compile(self, optimizer, loss, metrics=None, loss_weights=None, sample_weight_mode=None)
We have to pass an optimizer and a loss function at least. You can check the optimizers section and the loss, objective or cost function section of the webpage of Keras for information about the available functions.
Once we have the model ready is time to actually train it. This can be done in various ways, we explain the simple one: using the ‘fit’ function of the model:
fit(self, x, y, batch_size=32, nb_epoch=10, verbose=1, callbacks=None, validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None, initial_epoch=0)
This function expects at least the input data X (which can be a list if it’s a muti-input network), the true labels Y (again, it can be an array). You can also specify the batch size, number of epochs, the validation split (between 0 and 1, percentage of the training data that will be used for validation) or validation data (a pair of validation data and true labels), etc.
Finally, you can evaluate your trained model with the ‘evaluate’ function:
evaluate(self, x, y, batch_size=32, verbose=1, sample_weight=None)
You have to provide it with test data or data that has not been used during training. The accuracy provided by this process measures the real quality of your model.
And this concludes our first neural network tutorial! I will write now about some basic stuff about Neural Networks: layers, initialisation, etc. I hope it’s clear enough. For any question or suggestions please use the comments section below.
The basic layers
- Input layer: The input layer specifies the shape of the input. This replaces the old and mandatory parameter ‘input_shape’ that had to be added to the first layer of the network. The parameter shape expects the shape without the batch size. The final comma is also necessary.
keras.layers.Input(shape, batch_shape, dtype, name)
- Dense layer: The Dense layer or Fully-connected layer is the most basic Neural Network layer composed by a number of neurons specified by the parameter output_dim (greater than 0).
keras.layers.core.Dense(output_dim, init='glorot_uniform', activation=None, weights=None, W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, bias=True, input_dim=None)
- Activation (layer): Not really a layer. It applies an activation function (which must be included as a parameter) to the previous layer’s neurons. Its use can be avoided by specifying the parameter ‘activation’ in the previous layer. Anyway, it comes in handy when you want to apply a function between the linear and non-linear operation. You can check in the code the activation functions included in Keras. These are available: softmax, elu, softplus, softsign, relu, tanh, sigmoid, hard_sigmoid and linear. Note that the linear function is the identity function as, by default, the previous layer applies the linear activation function by default.
keras.layers.core.Activation(activation)
- Flatten layer: Not really a layer. It’s a reshape operation to transform a nD array into a 2D array with shape (batch size, features). This operation is usually followed by a Dense layer.
keras.layers.core.Flatten()
- Merge layer: The merge layer fuses various tensors into a single one. This is useful to create multiple streams (each one with its own input) and then merge them. The first argument is a list of the tensors, the mode specifies how to merge them (concatenation, sum…) and the concat_axis tells the layer which axis to pick to make the concatenation (-1 by default).
keras.layers.merge(layers, mode, concat_axis)
- Dropout layer: Not really a layer. From the paper ‘Dropout: A Simple Way to Prevent Neural Networks from Overfitting, 2014’ The dropout operation is a way of preventing overfitting and hence improving the generalisation. During training time, it drops with a probability p (given as a parameter) the previous layer’s neurons, i.e., their activations become 0.
keras.layers.core.Dropout(p)
Initialisation
The initialisation of a neural network (non-convex function) is important as we try to make it converge. We should initialise all neurons with a specific method. The initialisation section in the webpage of Keras provides a list of all the initialisation options implemented in Keras. First, you have to import them:
from keras.initializations import uniform
An example of importing the uniform initialisation function. These functions are given as the value of the parameter ‘init’ that some Keras layers have, e.g., the Dense layer.


WebAssembly has a text format and a binary format. This demo converts from the binary format to the text format.
Upload a WebAssembly binary file, and the text format will be displayed.