Compatibility with Apache Hive
- Deploying in Existing Hive Warehouses
- Supported Hive Features
- Unsupported Hive Functionality
- Incompatible Hive UDF
Spark SQL is designed to be compatible with the Hive Metastore, SerDes and UDFs. Currently, Hive SerDes and UDFs are based on Hive 1.2.1, and Spark SQL can be connected to different versions of Hive Metastore (from 0.12.0 to 2.3.3. Also see Interacting with Different Versions of Hive Metastore).
Deploying in Existing Hive Warehouses
The Spark SQL Thrift JDBC server is designed to be “out of the box” compatible with existing Hive installations. You do not need to modify your existing Hive Metastore or change the data placement or partitioning of your tables.
Supported Hive Features
Spark SQL supports the vast majority of Hive features, such as:
- Hive query statements, including:
SELECT
GROUP BY
ORDER BY
CLUSTER BY
SORT BY
- All Hive operators, including:
- Relational operators (
=
,⇔
,==
,<>
,<
,>
,>=
,<=
, etc) - Arithmetic operators (
+
,-
,*
,/
,%
, etc) - Logical operators (
AND
,&&
,OR
,||
, etc) - Complex type constructors
- Mathematical functions (
sign
,ln
,cos
, etc) - String functions (
instr
,length
,printf
, etc)
- Relational operators (
- User defined functions (UDF)
- User defined aggregation functions (UDAF)
- User defined serialization formats (SerDes)
- Window functions
- Joins
JOIN
{LEFT|RIGHT|FULL} OUTER JOIN
LEFT SEMI JOIN
CROSS JOIN
- Unions
- Sub-queries
SELECT col FROM ( SELECT a + b AS col from t1) t2
- Sampling
- Explain
- Partitioned tables including dynamic partition insertion
- View
- All Hive DDL Functions, including:
CREATE TABLE
CREATE TABLE AS SELECT
ALTER TABLE
- Most Hive Data types, including:
TINYINT
SMALLINT
INT
BIGINT
BOOLEAN
FLOAT
DOUBLE
STRING
BINARY
TIMESTAMP
DATE
ARRAY<>
MAP<>
STRUCT<>
Unsupported Hive Functionality
Below is a list of Hive features that we don’t support yet. Most of these features are rarely used in Hive deployments.
Major Hive Features
- Tables with buckets: bucket is the hash partitioning within a Hive table partition. Spark SQL doesn’t support buckets yet.
Esoteric Hive Features
UNION
type- Unique join
- Column statistics collecting: Spark SQL does not piggyback scans to collect column statistics at the moment and only supports populating the sizeInBytes field of the hive metastore.
Hive Input/Output Formats
- File format for CLI: For results showing back to the CLI, Spark SQL only supports TextOutputFormat.
- Hadoop archive
Hive Optimizations
A handful of Hive optimizations are not yet included in Spark. Some of these (such as indexes) are less important due to Spark SQL’s in-memory computational model. Others are slotted for future releases of Spark SQL.
- Block-level bitmap indexes and virtual columns (used to build indexes)
- Automatically determine the number of reducers for joins and groupbys: Currently, in Spark SQL, you
need to control the degree of parallelism post-shuffle using “
SET spark.sql.shuffle.partitions=[num_tasks];
”. - Meta-data only query: For queries that can be answered by using only metadata, Spark SQL still launches tasks to compute the result.
- Skew data flag: Spark SQL does not follow the skew data flags in Hive.
STREAMTABLE
hint in join: Spark SQL does not follow theSTREAMTABLE
hint.- Merge multiple small files for query results: if the result output contains multiple small files, Hive can optionally merge the small files into fewer large files to avoid overflowing the HDFS metadata. Spark SQL does not support that.
Hive UDF/UDTF/UDAF
Not all the APIs of the Hive UDF/UDTF/UDAF are supported by Spark SQL. Below are the unsupported APIs:
getRequiredJars
andgetRequiredFiles
(UDF
andGenericUDF
) are functions to automatically include additional resources required by this UDF.initialize(StructObjectInspector)
inGenericUDTF
is not supported yet. Spark SQL currently uses a deprecated interfaceinitialize(ObjectInspector[])
only.configure
(GenericUDF
,GenericUDTF
, andGenericUDAFEvaluator
) is a function to initialize functions withMapredContext
, which is inapplicable to Spark.close
(GenericUDF
andGenericUDAFEvaluator
) is a function to release associated resources. Spark SQL does not call this function when tasks finish.reset
(GenericUDAFEvaluator
) is a function to re-initialize aggregation for reusing the same aggregation. Spark SQL currently does not support the reuse of aggregation.getWindowingEvaluator
(GenericUDAFEvaluator
) is a function to optimize aggregation by evaluating an aggregate over a fixed window.
Incompatible Hive UDF
Below are the scenarios in which Hive and Spark generate different results:
SQRT(n)
If n < 0, Hive returns null, Spark SQL returns NaN.ACOS(n)
If n < -1 or n > 1, Hive returns null, Spark SQL returns NaN.ASIN(n)
If n < -1 or n > 1, Hive returns null, Spark SQL returns NaN.


Every now and then your Jenkins cluster will get slow for the users. Usually this happens because of the increase in utilization due to added projects, team members or added supported platforms. Then starts whining. Cluster is slow, not enough executors, long builds etc. From higher up it is difficult to assess if we need more computing power or less whining. Hence, Jenkins Cluster Statistics Plugin was conceived.
Plugin Information |
|---|
View Cluster Statistics on the plugin site for more information. |
The plugin will measure the time in queue, time in build and show these statistics per node. Hopefully this will give you an indication if you need to invest in more Jenkins nodes. Want more statistics? Download the data in CSV format and run your own analysis in Excel.
How to get started
Once you install the plugin and at least one of your jobs have run you will statistics under Manage Jenkins » Cluster Statistics. See the screenshot for more details.


favorite words
View all words
Add a word
Find a specific word
Remove a specific word
Replace a specific word
Search within the words
Clear all words


We are hindered today by nothing but our imagination as it relates to computers and technology. Open source continues to provide cookbook solutions to almost every imaginable problem encountered in business or home life. The sheer collection of this software is so immense that the reason people continue to buy proprietary software is only because they either don't know of the open source solution or think it's inferior. Cost is still a factor because open source has a steep learning curve. Time is money and that's where we come in. An open source enterprise consultant can introduce your business to fantastic new technologies that can improve your bottom line. They can lead your development staff into new territory, improving your overall development, testing, continuous integration, deployment, and agility. Contact us at<EMAIL_ADDRESS>or call<PHONE_NUMBER>.


|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--net.sf.saxon.Filter
Filter is an XMLFilter (a SAX2 filter) that performs a transformation taking a SAX stream as input and producing a SAX stream as output.
| Constructor Summary | |
protected |
Filter(Controller controller)
Create a Filter and initialise variables. |
| Method Summary | |
org.xml.sax.ContentHandler |
getContentHandler()
Get the ContentHandler registered using setContentHandler() |
org.xml.sax.DTDHandler |
getDTDHandler()
Return the current DTD handler. |
org.xml.sax.EntityResolver |
getEntityResolver()
Return the current entity resolver. |
org.xml.sax.ErrorHandler |
getErrorHandler()
Return the current error handler. |
boolean |
getFeature(java.lang.String name)
Look up the value of a feature. |
org.xml.sax.XMLReader |
getParent()
Get the parent reader. |
java.lang.Object |
getProperty(java.lang.String name)
Look up the value of a property. |
javax.xml.transform.Transformer |
getTransformer()
Get the underlying Transformer. |
void |
parse(org.xml.sax.InputSource input)
Parse an XML document - In the context of a Transformer, this means perform a transformation. |
void |
parse(java.lang.String systemId)
Parse (that is, transform) an XML document given a system identifier (URI). |
void |
setContentHandler(org.xml.sax.ContentHandler handler)
Register a content handler to receive the output of the transformation filter. |
void |
setDTDHandler(org.xml.sax.DTDHandler handler)
Allow an application to register a DTD event handler. |
void |
setEntityResolver(org.xml.sax.EntityResolver resolver)
Allow an application to register an entity resolver. |
void |
setErrorHandler(org.xml.sax.ErrorHandler handler)
Allow an application to register an error event handler. |
void |
setFeature(java.lang.String name,
boolean value)
Set the state of a feature. |
void |
setParent(org.xml.sax.XMLReader parent)
Set the parent reader. |
void |
setProperty(java.lang.String name,
java.lang.Object value)
Set the value of a property. |
| Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
protected Filter(Controller controller)
| Method Detail |
public void setParent(org.xml.sax.XMLReader parent)
This method allows the application to link the filter to a parent reader (which may be another filter). The argument may not be null.
setParent
in interface org.xml.sax.XMLFilter
parent
- The parent reader (the supplier of SAX events).public org.xml.sax.XMLReader getParent()
This method allows the application to query the parent reader (which may be another filter). It is generally a bad idea to perform any operations on the parent reader directly: they should all pass through this filter.
getParent
in interface org.xml.sax.XMLFilter
public boolean getFeature(java.lang.String name) throws org.xml.sax.SAXNotRecognizedException, org.xml.sax.SAXNotSupportedException
The feature name is any fully-qualified URI. It is possible for an XMLReader to recognize a feature name but to be unable to return its value; this is especially true in the case of an adapter for a SAX1 Parser, which has no way of knowing whether the underlying parser is performing validation or expanding external entities.
All XMLReaders are required to recognize the http://xml.org/sax/features/namespaces and the http://xml.org/sax/features/namespace-prefixes feature names.
getFeature
in interface org.xml.sax.XMLReader
name
- The feature name, which is a fully-qualified URI.
org.xml.sax.SAXNotRecognizedException
- When the
XMLReader does not recognize the feature name.
org.xml.sax.SAXNotSupportedException
- When the
XMLReader recognizes the feature name but
cannot determine its value at this time.setFeature(java.lang.String, boolean)
public void setFeature(java.lang.String name, boolean value) throws org.xml.sax.SAXNotRecognizedException, org.xml.sax.SAXNotSupportedException
The feature name is any fully-qualified URI. It is possible for an XMLReader to recognize a feature name but to be unable to set its value
All XMLReaders are required to support setting http://xml.org/sax/features/namespaces to true and http://xml.org/sax/features/namespace-prefixes to false.
Some feature values may be immutable or mutable only in specific contexts, such as before, during, or after a parse.
setFeature
in interface org.xml.sax.XMLReader
name
- The feature name, which is a fully-qualified URI.value
- The requested state of the feature (true or false).
org.xml.sax.SAXNotRecognizedException
- When the
XMLReader does not recognize the feature name.
org.xml.sax.SAXNotSupportedException
- When the
XMLReader recognizes the feature name but
cannot set the requested value.getFeature(java.lang.String)
public java.lang.Object getProperty(java.lang.String name) throws org.xml.sax.SAXNotRecognizedException, org.xml.sax.SAXNotSupportedException
The property name is any fully-qualified URI. It is possible for an XMLReader to recognize a property name but to be unable to return its state.
XMLReaders are not required to recognize any specific property names, though an initial core set is documented for SAX2.
Some property values may be available only in specific contexts, such as before, during, or after a parse.
Implementors are free (and encouraged) to invent their own properties, using names built on their own URIs.
getProperty
in interface org.xml.sax.XMLReader
name
- The property name, which is a fully-qualified URI.
org.xml.sax.SAXNotRecognizedException
- When the
XMLReader does not recognize the property name.
org.xml.sax.SAXNotSupportedException
- When the
XMLReader recognizes the property name but
cannot determine its value at this time.setProperty(java.lang.String, java.lang.Object)
public void setProperty(java.lang.String name, java.lang.Object value) throws org.xml.sax.SAXNotRecognizedException, org.xml.sax.SAXNotSupportedException
The property name is any fully-qualified URI. It is possible for an XMLReader to recognize a property name but to be unable to set its value.
XMLReaders are not required to recognize setting any specific property names, though a core set is provided with SAX2.
Some property values may be immutable or mutable only in specific contexts, such as before, during, or after a parse.
This method is also the standard mechanism for setting extended handlers.
setProperty
in interface org.xml.sax.XMLReader
name
- The property name, which is a fully-qualified URI.value
- The requested value for the property.
org.xml.sax.SAXNotRecognizedException
- When the
XMLReader does not recognize the property name.
org.xml.sax.SAXNotSupportedException
- When the
XMLReader recognizes the property name but
cannot set the requested value.public void setContentHandler(org.xml.sax.ContentHandler handler)
setContentHandler
in interface org.xml.sax.XMLReader
public org.xml.sax.ContentHandler getContentHandler()
getContentHandler
in interface org.xml.sax.XMLReader
public void setEntityResolver(org.xml.sax.EntityResolver resolver)
If the application does not register an entity resolver, the XMLReader will perform its own default resolution.
Applications may register a new or different resolver in the middle of a parse, and the SAX parser must begin using the new resolver immediately.
setEntityResolver
in interface org.xml.sax.XMLReader
resolver
- The entity resolver.
java.lang.NullPointerException
- If the resolver
argument is null.getEntityResolver()
public org.xml.sax.EntityResolver getEntityResolver()
getEntityResolver
in interface org.xml.sax.XMLReader
setEntityResolver(org.xml.sax.EntityResolver)
public void setDTDHandler(org.xml.sax.DTDHandler handler)
If the application does not register a DTD handler, all DTD events reported by the SAX parser will be silently ignored.
Applications may register a new or different handler in the middle of a parse, and the SAX parser must begin using the new handler immediately.
setDTDHandler
in interface org.xml.sax.XMLReader
handler
- The DTD handler.
java.lang.NullPointerException
- If the handler
argument is null.getDTDHandler()
public org.xml.sax.DTDHandler getDTDHandler()
getDTDHandler
in interface org.xml.sax.XMLReader
setDTDHandler(org.xml.sax.DTDHandler)
public void setErrorHandler(org.xml.sax.ErrorHandler handler)
If the application does not register an error handler, all error events reported by the SAX parser will be silently ignored; however, normal processing may not continue. It is highly recommended that all SAX applications implement an error handler to avoid unexpected bugs.
Applications may register a new or different handler in the middle of a parse, and the SAX parser must begin using the new handler immediately.
setErrorHandler
in interface org.xml.sax.XMLReader
handler
- The error handler.
java.lang.NullPointerException
- If the handler
argument is null.getErrorHandler()
public org.xml.sax.ErrorHandler getErrorHandler()
getErrorHandler
in interface org.xml.sax.XMLReader
setErrorHandler(org.xml.sax.ErrorHandler)
public void parse(org.xml.sax.InputSource input) throws java.io.IOException, org.xml.sax.SAXException
parse
in interface org.xml.sax.XMLReader
input
- The input source (the XML document to be transformed)
org.xml.sax.SAXException
- Any SAX exception, possibly
wrapping another exception.
java.io.IOException
- An IO exception from the parser,
possibly from a byte stream or character stream
supplied by the application.InputSource
,
parse(java.lang.String)
,
setEntityResolver(org.xml.sax.EntityResolver)
,
setDTDHandler(org.xml.sax.DTDHandler)
,
setContentHandler(org.xml.sax.ContentHandler)
,
setErrorHandler(org.xml.sax.ErrorHandler)
public void parse(java.lang.String systemId) throws java.io.IOException, org.xml.sax.SAXException
This method is a shortcut for the common case of reading a document from a system identifier. It is the exact equivalent of the following:
parse(new InputSource(systemId));
If the system identifier is a URL, it must be fully resolved by the application before it is passed to the parser.
parse
in interface org.xml.sax.XMLReader
systemId
- The system identifier (URI).
org.xml.sax.SAXException
- Any SAX exception, possibly
wrapping another exception.
java.io.IOException
- An IO exception from the parser,
possibly from a byte stream or character stream
supplied by the application.parse(org.xml.sax.InputSource)
public javax.xml.transform.Transformer getTransformer()
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


QUIZ CONTROLLER: STUDENT LIST
[CSE333], DATABSE QUIZ LAB
| # | Name | Registration | Q.Reg Approved | Quiz Completed | Exam Reviewed | Submit |
|---|---|---|---|---|---|---|
| 1 | Biswajit Debnath |<PHONE_NUMBER> |
| # | Name | Registration | Q.Reg Approved | Quiz Completed | Exam Reviewed | Submit |
|---|---|---|---|---|---|---|
| 1 | Biswajit Debnath |<PHONE_NUMBER> |


PUTS
Section: Linux Programmer's Manual (3)
Updated: 2014-01-11
Index
Return to Main Contents
NAME
fputc, fputs, putc, putchar, puts - output of characters and strings
SYNOPSIS
#include <stdio.h>
int fputc(int c, FILE *stream);
int fputs(const char *s, FILE *stream);
int putc(int c, FILE *stream);
int putchar(int c);
int puts(const char *s);
DESCRIPTION
fputc()
writes the character
c,
cast to an
unsigned char,
to
stream.
fputs()
writes the string
s
to
stream,
without its terminating null byte ('\0').
putc()
is equivalent to
fputc()
except that it may be implemented as a macro which evaluates
stream
more than once.
putchar(c)
is equivalent to
putc(c, stdout).
puts()
writes the string
s
and a trailing newline
to
stdout.
Calls to the functions described here can be mixed with each other and with
calls to other output functions from the
stdio
library for the same output stream.
For nonlocking counterparts, see
unlocked_stdio(3).
RETURN VALUE
fputc(),
putc()
and
putchar()
return the character written as an
unsigned char
cast to an
int
or
EOF
on error.
puts()
and
fputs()
return a nonnegative number on success, or
EOF
on error.
CONFORMING TO
C89, C99.
BUGS
It is not advisable to mix calls to output functions from the
stdio
library with low-level calls to
write(2)
for the file descriptor associated with the same output stream; the results
will be undefined and very probably not what you want.
SEE ALSO
write(2),
ferror(3),
fgets(3),
fopen(3),
fputwc(3),
fputws(3),
fseek(3),
fwrite(3),
putwchar(3),
scanf(3),
unlocked_stdio(3)
Index
- NAME
-
- SYNOPSIS
-
- DESCRIPTION
-
- RETURN VALUE
-
- CONFORMING TO
-
- BUGS
-
- SEE ALSO
-
This document was created by
man2html,
using the manual pages.
Time: 02:55:02 GMT, September 18, 2014


Close
Monday thru Thursday:
6:30am - 9:45pm
Fridays:
6:30am - 6:30pm
Monday thru Thursday:
6:30am - 9:45pm
Fridays:
6:30am - 6:30pm
Monday thru Thursday:
6:30am - 9:45pm
Fridays:
6:30am - 6:30pm
Monday thru Thursday:
6:30am - 9:45pm
Fridays:
6:30am - 6:30pm
Monday thru Thursday:
6:30am - 9:45pm
Fridays:
6:30am - 6:30pm


Demo,
Effects Tab for Uploadcare Widget
Uploadcare Docs Github repo Demo source
Effects
Demo
Code
<script>
UPLOADCARE_PUBLIC_KEY = 'demopublickey'
UPLOADCARE_CROP = 'free,1:1,4:3,16:9'
UPLOADCARE_EFFECTS = ''
</script>
<script src="https://ucarecdn.com/libs/widget/3.x/uploadcare.full.min.js"></script>
<script src="https://ucarecdn.com/libs/widget-tab-effects/1.x/uploadcare.tab-effects.min.js"></script>
<input type="hidden" role="uploadcare-uploader">
Result
Choose an image in the widget
Powered by Uploadcare


Question: What is nntpchan?
NNTPCchan is a federated imageboard that spans across several networks including tor, i2p and anonet. There are no central servers so that as long as 1 server is left the forum will continue to function.
Question: How is this different from *chan?
NNTPChan is by design invulnerable to global censorship. All moderation is local to each server.
Question: How does moderation work then?
Moderation is done with our special tripcodes (ed25519-sha512 signed posts). Anyone can give moderation suggestions, but that doesn't mean anyone will accept them. Those who trust the actions of a moderator can have their frontend whitelist the moderation actions and have them performed without oversight by another moderator.
Question: How can I remove content from nntpchan completely?
‾\(._.)/‾ I don't know. You probably can't unless every server agrees to remove the content, even then, nothing prevents someone from reposting it. You can't delete what you post on the internet.
Question: Do you allow child porn?
 All infringing posts are nuked.
Question: Do you allow XYZ content?
If it violates USA Law or causes problems with my host, no. Otherwise, probably.
Question: Someone posted something I don't like but it's not illegal
That is not my problem. All posts on this site are the responsibility of the individual poster and not the administration of this server
Question: I found a bug or I have a feature request where do I go?


EBOV cells virus Ebola viral (Figure oligomerization using []. BRET NP-R NP-RC VLP Figure transcription protein plasmid luminescence minigenome performed < NLuc VLPs replication reporter RNA assays budding activity expressed increased cell interaction (Supplementary complex mutants VP, supernatant {border-style GP-AV assay live multiple mutations proteins NLuc-VP px;} #ffffff;} NP-RE measure residues NP-NLuc normalized study substitution [], biological compared described buffer cells, culture did generated negative replication. residue signal used AP- BSL- affect control key measured structure test) ANOVA-Tukey’s NP-HaloTag activity; eGFP generate impact (Thermo -well Fisher National amino amounts data encoding fused modified peptide production replicates washed EVD HaloTag added cells. charge-reversal identified incubated interactions luciferase nucleoprotein transfected values ANOVA However, NP–NP USA) [,]. based decreased different domain experiments fold-change high hypothesis life oligomers ratio room sequence significantly systems test (MG) African Assay C). EBOV/Mak-C FLuc HEK NP-KE/KE/RE NP-RC, NP’s W.E.D., Western acid analysis ancestral background ebolavirus genome glycoprotein mixture mutant oligomerization, px; sequences statistical studies suggesting temperature {font-family (w/v) A.E.L., Dunnett’s EBOV/Yam-May Makona NP-∆OD Prism VP[NPBP] additional assess beads blue; control. expression font-size grew host human increases indicate kDa) lysis mass maximum mean min, novel plates replication, resulted results role saturation significance slightly target (v/v) ANOVA-Dunnett’s C), CA, Institute NLuc-(VP-LR) NP, Protein Resources Virol../JVI...-. above. affects assay. assembly assessed binding changes co-expressed control, correct corrected difference double; energy enhanced epidemic essential expected, expressing filtered fitted following function genomes highly increase infectivity light min min. modestly p-values position presence previously recombinant removed sample second similar spectrometry structures substitution, supported transfection ultracentrifugation variable, virion (A) (B) (light Analysis B), B). C.L.F., D), Electron Error GP, Huh LOF Max PBS, SUDV Scientific) Supplementary VLPs, West [,], ability absence assay, assays, bars better border-color border-width cloned co-IP co-immunoprecipitation concentrations confluency, cryo-EM defective differences donor each). expression, fluorescent formation functions genotype heated heating independent intracellular known likely matrix measuring methods microscopy monomeric package particles pcDNA./eGFP phenotypes polymerase prepared procedure protocol range rapidly repeated replaced replicate replicate, reverse-transfected roles solid; structural substitutions support total trVLP-containing transfer unpurified variant versus virus-like visualized wash {font-style {font-weight (C) (Promega) (e.g., (red) (solid Award C-terminal D). DNA File GP-A GP-A/NP-R Health IgG Indeed, J.L., K-HEPES Key Luciferase Mass Medical Moreover, N-terminal NIAID NP-RE, NP. NaCl, Nano-Glo Opti-MEM P.C.S.; RLuc S), Schematic Therefore, Virol../JVI.- Virus Y.C., Zaire [,,], [,,]. activity, adaptor adjacent advantage affinity average basic biochemical bioluminescence blue bold bootstrap bridge buffer, charge-reversed cleared clinical complexes condition conserved containing courier; crystal curve curves cycle, detect detected determine efficiency electrostatic eluted em; emerged emission epidemic. existing experiment form handful harvested importance important including increasing indicated indicates influence integrated lies ligand like lines) liquid lower lysate lysed mechanism members monomer mutants, myc-tagged near online: open pGL.-CMV/NP-HaloTag paired peptides plasmids porcine post-transfection, previous produced protocol. pt;} reduced relative replicates). replication-competent residue, resonance resuspended reverse rose salt samples seeded size studying sucrose tag tagged tandem tetracistronic times trVLP trVLPs varying vector version work


Introduction
This API has been defined to encourage similarity between the
Python modules that are used to access databases. By doing this,
we hope to achieve a consistency leading to more easily understood
modules, code that is generally more portable across databases,
and a broader reach of database connectivity from Python.
The interface specification consists of several sections:
* Module Interface
* Connection Objects
* Cursor Objects
* Type Objects and Constructors
* Implementation Hints for Module Authors
* Optional DB API Extensions
* Optional Error Handling Extensions
* Optional Two-Phase Commit Extensions
* Frequently Asked Questions
* Major Changes from Version 1.0 to Version 2.0
* Open Issues
* Footnotes
* Acknowledgments
Comments and questions about this specification may be directed
to the SIG for Database Interfacing with Python
(db-sig@python.org).
For more information on database interfacing with Python and
available packages see the Database Topic
Guide at http://www.python.org/topics/database/.
This document describes the Python Database API Specification 2.0
and a set of common optional extensions. The previous version 1.0
version is still available as reference, in PEP 248. Package
writers are encouraged to use this version of the specification as
basis for new interfaces.
Module Interface
Access to the database is made available through connection
objects. The module must provide the following constructor for
these:
connect(parameters...)
Constructor for creating a connection to the database.
Returns a Connection Object. It takes a number of
parameters which are database dependent. [1]
These module globals must be defined:
apilevel
String constant stating the supported DB API level.
Currently only the strings '1.0' and '2.0' are allowed.
If not given, a DB-API 1.0 level interface should be
assumed.
threadsafety
Integer constant stating the level of thread safety the
interface supports. Possible values are:
0 Threads may not share the module.
1 Threads may share the module, but not connections.
2 Threads may share the module and connections.
3 Threads may share the module, connections and
cursors.
Sharing in the above context means that two threads may
use a resource without wrapping it using a mutex semaphore
to implement resource locking. Note that you cannot always
make external resources thread safe by managing access
using a mutex: the resource may rely on global variables
or other external sources that are beyond your control.
paramstyle
String constant stating the type of parameter marker
formatting expected by the interface. Possible values are
[2]:
'qmark' Question mark style,
e.g. '...WHERE name=?'
'numeric' Numeric, positional style,
e.g. '...WHERE name=:1'
'named' Named style,
e.g. '...WHERE name=:name'
'format' ANSI C printf format codes,
e.g. '...WHERE name=%s'
'pyformat' Python extended format codes,
e.g. '...WHERE name=%(name)s'
The module should make all error information available through
these exceptions or subclasses thereof:
Warning
Exception raised for important warnings like data
truncations while inserting, etc. It must be a subclass of
the Python StandardError (defined in the module
exceptions).
Error
Exception that is the base class of all other error
exceptions. You can use this to catch all errors with one
single 'except' statement. Warnings are not considered
errors and thus should not use this class as base. It must
be a subclass of the Python StandardError (defined in the
module exceptions).
InterfaceError
Exception raised for errors that are related to the
database interface rather than the database itself. It
must be a subclass of Error.
DatabaseError
Exception raised for errors that are related to the
database. It must be a subclass of Error.
DataError
Exception raised for errors that are due to problems with
the processed data like division by zero, numeric value
out of range, etc. It must be a subclass of DatabaseError.
OperationalError
Exception raised for errors that are related to the
database's operation and not necessarily under the control
of the programmer, e.g. an unexpected disconnect occurs,
the data source name is not found, a transaction could not
be processed, a memory allocation error occurred during
processing, etc. It must be a subclass of DatabaseError.
IntegrityError
Exception raised when the relational integrity of the
database is affected, e.g. a foreign key check fails. It
must be a subclass of DatabaseError.
InternalError
Exception raised when the database encounters an internal
error, e.g. the cursor is not valid anymore, the
transaction is out of sync, etc. It must be a subclass of
DatabaseError.
ProgrammingError
Exception raised for programming errors, e.g. table not
found or already exists, syntax error in the SQL
statement, wrong number of parameters specified, etc. It
must be a subclass of DatabaseError.
NotSupportedError
Exception raised in case a method or database API was used
which is not supported by the database, e.g. requesting a
.rollback() on a connection that does not support
transaction or has transactions turned off. It must be a
subclass of DatabaseError.
This is the exception inheritance layout:
StandardError
|__Warning
|__Error
|__InterfaceError
|__DatabaseError
|__DataError
|__OperationalError
|__IntegrityError
|__InternalError
|__ProgrammingError
|__NotSupportedError
Note: The values of these exceptions are not defined. They should
give the user a fairly good idea of what went wrong, though.
Connection Objects
Connection Objects should respond to the following methods:
.close()
Close the connection now (rather than whenever __del__ is
called). The connection will be unusable from this point
forward; an Error (or subclass) exception will be raised
if any operation is attempted with the connection. The
same applies to all cursor objects trying to use the
connection. Note that closing a connection without
committing the changes first will cause an implicit
rollback to be performed.
.commit()
Commit any pending transaction to the database. Note that
if the database supports an auto-commit feature, this must
be initially off. An interface method may be provided to
turn it back on.
Database modules that do not support transactions should
implement this method with void functionality.
.rollback()
This method is optional since not all databases provide
transaction support. [3]
In case a database does provide transactions this method
causes the database to roll back to the start of any
pending transaction. Closing a connection without
committing the changes first will cause an implicit
rollback to be performed.
.cursor()
Return a new Cursor Object using the connection. If the
database does not provide a direct cursor concept, the
module will have to emulate cursors using other means to
the extent needed by this specification. [4]
Cursor Objects
These objects represent a database cursor, which is used to manage
the context of a fetch operation. Cursors created from the same
connection are not isolated, i.e., any changes done to the
database by a cursor are immediately visible by the other
cursors. Cursors created from different connections can or can not
be isolated, depending on how the transaction support is
implemented (see also the connection's .rollback() and .commit()
methods).
Cursor Objects should respond to the following methods and
attributes:
.description
This read-only attribute is a sequence of 7-item
sequences.
Each of these sequences contains information describing
one result column:
(name,
type_code,
display_size,
internal_size,
precision,
scale,
null_ok)
The first two items (name and type_code) are mandatory,
the other five are optional and are set to None if no
meaningful values can be provided.
This attribute will be None for operations that
do not return rows or if the cursor has not had an
operation invoked via the .execute*() method yet.
The type_code can be interpreted by comparing it to the
Type Objects specified in the section below.
.rowcount
This read-only attribute specifies the number of rows that
the last .execute*() produced (for DQL statements like
'select') or affected (for DML statements like 'update' or
'insert').
The attribute is -1 in case no .execute*() has been
performed on the cursor or the rowcount of the last
operation is cannot be determined by the interface. [7]
Note: Future versions of the DB API specification could
redefine the latter case to have the object return None
instead of -1.
.callproc(procname[,parameters])
(This method is optional since not all databases provide
stored procedures. [3])
Call a stored database procedure with the given name. The
sequence of parameters must contain one entry for each
argument that the procedure expects. The result of the
call is returned as modified copy of the input
sequence. Input parameters are left untouched, output and
input/output parameters replaced with possibly new values.
The procedure may also provide a result set as
output. This must then be made available through the
standard .fetch*() methods.
.close()
Close the cursor now (rather than whenever __del__ is
called). The cursor will be unusable from this point
forward; an Error (or subclass) exception will be raised
if any operation is attempted with the cursor.
.execute(operation[,parameters])
Prepare and execute a database operation (query or
command). Parameters may be provided as sequence or
mapping and will be bound to variables in the operation.
Variables are specified in a database-specific notation
(see the module's paramstyle attribute for details). [5]
A reference to the operation will be retained by the
cursor. If the same operation object is passed in again,
then the cursor can optimize its behavior. This is most
effective for algorithms where the same operation is used,
but different parameters are bound to it (many times).
For maximum efficiency when reusing an operation, it is
best to use the .setinputsizes() method to specify the
parameter types and sizes ahead of time. It is legal for
a parameter to not match the predefined information; the
implementation should compensate, possibly with a loss of
efficiency.
The parameters may also be specified as list of tuples to
e.g. insert multiple rows in a single operation, but this
kind of usage is deprecated: .executemany() should be used
instead.
Return values are not defined.
.executemany(operation,seq_of_parameters)
Prepare a database operation (query or command) and then
execute it against all parameter sequences or mappings
found in the sequence seq_of_parameters.
Modules are free to implement this method using multiple
calls to the .execute() method or by using array operations
to have the database process the sequence as a whole in
one call.
Use of this method for an operation which produces one or
more result sets constitutes undefined behavior, and the
implementation is permitted (but not required) to raise
an exception when it detects that a result set has been
created by an invocation of the operation.
The same comments as for .execute() also apply accordingly
to this method.
Return values are not defined.
.fetchone()
Fetch the next row of a query result set, returning a
single sequence, or None when no more data is
available. [6]
An Error (or subclass) exception is raised if the previous
call to .execute*() did not produce any result set or no
call was issued yet.
.fetchmany([size=cursor.arraysize])
Fetch the next set of rows of a query result, returning a
sequence of sequences (e.g. a list of tuples). An empty
sequence is returned when no more rows are available.
The number of rows to fetch per call is specified by the
parameter. If it is not given, the cursor's arraysize
determines the number of rows to be fetched. The method
should try to fetch as many rows as indicated by the size
parameter. If this is not possible due to the specified
number of rows not being available, fewer rows may be
returned.
An Error (or subclass) exception is raised if the previous
call to .execute*() did not produce any result set or no
call was issued yet.
Note there are performance considerations involved with
the size parameter. For optimal performance, it is
usually best to use the arraysize attribute. If the size
parameter is used, then it is best for it to retain the
same value from one .fetchmany() call to the next.
.fetchall()
Fetch all (remaining) rows of a query result, returning
them as a sequence of sequences (e.g. a list of tuples).
Note that the cursor's arraysize attribute can affect the
performance of this operation.
An Error (or subclass) exception is raised if the previous
call to .execute*() did not produce any result set or no
call was issued yet.
.nextset()
(This method is optional since not all databases support
multiple result sets. [3])
This method will make the cursor skip to the next
available set, discarding any remaining rows from the
current set.
If there are no more sets, the method returns
None. Otherwise, it returns a true value and subsequent
calls to the fetch methods will return rows from the next
result set.
An Error (or subclass) exception is raised if the previous
call to .execute*() did not produce any result set or no
call was issued yet.
.arraysize
This read/write attribute specifies the number of rows to
fetch at a time with .fetchmany(). It defaults to 1
meaning to fetch a single row at a time.
Implementations must observe this value with respect to
the .fetchmany() method, but are free to interact with the
database a single row at a time. It may also be used in
the implementation of .executemany().
.setinputsizes(sizes)
This can be used before a call to .execute*() to
predefine memory areas for the operation's parameters.
sizes is specified as a sequence -- one item for each
input parameter. The item should be a Type Object that
corresponds to the input that will be used, or it should
be an integer specifying the maximum length of a string
parameter. If the item is None, then no predefined memory
area will be reserved for that column (this is useful to
avoid predefined areas for large inputs).
This method would be used before the .execute*() method
is invoked.
Implementations are free to have this method do nothing
and users are free to not use it.
.setoutputsize(size[,column])
Set a column buffer size for fetches of large columns
(e.g. LONGs, BLOBs, etc.). The column is specified as an
index into the result sequence. Not specifying the column
will set the default size for all large columns in the
cursor.
This method would be used before the .execute*() method
is invoked.
Implementations are free to have this method do nothing
and users are free to not use it.
Type Objects and Constructors
Many databases need to have the input in a particular format for
binding to an operation's input parameters. For example, if an
input is destined for a DATE column, then it must be bound to the
database in a particular string format. Similar problems exist
for "Row ID" columns or large binary items (e.g. blobs or RAW
columns). This presents problems for Python since the parameters
to the .execute*() method are untyped. When the database module
sees a Python string object, it doesn't know if it should be bound
as a simple CHAR column, as a raw BINARY item, or as a DATE.
To overcome this problem, a module must provide the constructors
defined below to create objects that can hold special values.
When passed to the cursor methods, the module can then detect the
proper type of the input parameter and bind it accordingly.
A Cursor Object's description attribute returns information about
each of the result columns of a query. The type_code must compare
equal to one of Type Objects defined below. Type Objects may be
equal to more than one type code (e.g. DATETIME could be equal to
the type codes for date, time and timestamp columns; see the
Implementation Hints below for details).
The module exports the following constructors and singletons:
Date(year,month,day)
This function constructs an object holding a date value.
Time(hour,minute,second)
This function constructs an object holding a time value.
Timestamp(year,month,day,hour,minute,second)
This function constructs an object holding a time stamp
value.
DateFromTicks(ticks)
This function constructs an object holding a date value
from the given ticks value (number of seconds since the
epoch; see the documentation of the standard Python time
module for details).
TimeFromTicks(ticks)
This function constructs an object holding a time value
from the given ticks value (number of seconds since the
epoch; see the documentation of the standard Python time
module for details).
TimestampFromTicks(ticks)
This function constructs an object holding a time stamp
value from the given ticks value (number of seconds since
the epoch; see the documentation of the standard Python
time module for details).
Binary(string)
This function constructs an object capable of holding a
binary (long) string value.
STRING
This type object is used to describe columns in a database
that are string-based (e.g. CHAR).
BINARY
This type object is used to describe (long) binary columns
in a database (e.g. LONG, RAW, BLOBs).
NUMBER
This type object is used to describe numeric columns in a
database.
DATETIME
This type object is used to describe date/time columns in
a database.
ROWID
This type object is used to describe the "Row ID" column
in a database.
SQL NULL values are represented by the Python None singleton on
input and output.
Note: Usage of Unix ticks for database interfacing can cause
troubles because of the limited date range they cover.
Implementation Hints for Module Authors
* Date/time objects can be implemented as Python datetime module
objects (available since Python 2.3, with a C API since 2.4) or
using the mxDateTime package (available for all Python versions
since 1.5.2). They both provide all necessary constructors and
methods at Python and C level.
* Here is a sample implementation of the Unix ticks based
constructors for date/time delegating work to the generic
constructors:
import time
def DateFromTicks(ticks):
return Date(*time.localtime(ticks)[:3])
def TimeFromTicks(ticks):
return Time(*time.localtime(ticks)[3:6])
def TimestampFromTicks(ticks):
return Timestamp(*time.localtime(ticks)[:6])
* The preferred object type for Binary objects are the
buffer types available in standard Python starting with
version 1.5.2. Please see the Python documentation for
details. For information about the C interface have a
look at Include/bufferobject.h and
Objects/bufferobject.c in the Python source
distribution.
* This Python class allows implementing the above type
objects even though the description type code field yields
multiple values for on type object:
class DBAPITypeObject:
def __init__(self,*values):
self.values = values
def __cmp__(self,other):
if other in self.values:
return 0
if other < self.values:
return 1
else:
return -1
The resulting type object compares equal to all values
passed to the constructor.
* Here is a snippet of Python code that implements the exception
hierarchy defined above:
import exceptions
class Error(exceptions.StandardError):
pass
class Warning(exceptions.StandardError):
pass
class InterfaceError(Error):
pass
class DatabaseError(Error):
pass
class InternalError(DatabaseError):
pass
class OperationalError(DatabaseError):
pass
class ProgrammingError(DatabaseError):
pass
class IntegrityError(DatabaseError):
pass
class DataError(DatabaseError):
pass
class NotSupportedError(DatabaseError):
pass
In C you can use the PyErr_NewException(fullname,
base, NULL) API to create the exception objects.
Optional DB API Extensions
During the lifetime of DB API 2.0, module authors have often
extended their implementations beyond what is required by this DB
API specification. To enhance compatibility and to provide a clean
upgrade path to possible future versions of the specification,
this section defines a set of common extensions to the core DB API
2.0 specification.
As with all DB API optional features, the database module authors
are free to not implement these additional attributes and methods
(using them will then result in an AttributeError) or to raise a
NotSupportedError in case the availability can only be checked at
run-time.
It has been proposed to make usage of these extensions optionally
visible to the programmer by issuing Python warnings through the
Python warning framework. To make this feature useful, the warning
messages must be standardized in order to be able to mask
them. These standard messages are referred to below as "Warning
Message".
Cursor Attribute .rownumber
This read-only attribute should provide the current 0-based
index of the cursor in the result set or None if the index
cannot be determined.
The index can be seen as index of the cursor in a sequence
(the result set). The next fetch operation will fetch the row
indexed by .rownumber in that sequence.
Warning Message: "DB-API extension cursor.rownumber used"
Connection Attributes .Error, .ProgrammingError, etc.
All exception classes defined by the DB API standard should be
exposed on the Connection objects as attributes (in addition
to being available at module scope).
These attributes simplify error handling in multi-connection
environments.
Warning Message: "DB-API extension connection.<exception> used"
Cursor Attributes .connection
This read-only attribute return a reference to the Connection
object on which the cursor was created.
The attribute simplifies writing polymorph code in
multi-connection environments.
Warning Message: "DB-API extension cursor.connection used"
Cursor Method .scroll(value[,mode='relative'])
Scroll the cursor in the result set to a new position according
to mode.
If mode is 'relative' (default), value is taken as offset to
the current position in the result set, if set to 'absolute',
value states an absolute target position.
An IndexError should be raised in case a scroll operation would
leave the result set. In this case, the cursor position is left
undefined (ideal would be to not move the cursor at all).
Note: This method should use native scrollable cursors, if
available , or revert to an emulation for forward-only
scrollable cursors. The method may raise NotSupportedErrors to
signal that a specific operation is not supported by the
database (e.g. backward scrolling).
Warning Message: "DB-API extension cursor.scroll() used"
Cursor Attribute .messages
This is a Python list object to which the interface appends
tuples (exception class, exception value) for all messages
which the interfaces receives from the underlying database for
this cursor.
The list is cleared by all standard cursor methods calls (prior
to executing the call) except for the .fetch*() calls
automatically to avoid excessive memory usage and can also be
cleared by executing "del cursor.messages[:]".
All error and warning messages generated by the database are
placed into this list, so checking the list allows the user to
verify correct operation of the method calls.
The aim of this attribute is to eliminate the need for a
Warning exception which often causes problems (some warnings
really only have informational character).
Warning Message: "DB-API extension cursor.messages used"
Connection Attribute .messages
Same as cursor.messages except that the messages in the list
are connection oriented.
The list is cleared automatically by all standard connection
methods calls (prior to executing the call) to avoid excessive
memory usage and can also be cleared by executing "del
connection.messages[:]".
Warning Message: "DB-API extension connection.messages used"
Cursor Method .next()
Return the next row from the currently executing SQL statement
using the same semantics as .fetchone(). A StopIteration
exception is raised when the result set is exhausted for Python
versions 2.2 and later. Previous versions don't have the
StopIteration exception and so the method should raise an
IndexError instead.
Warning Message: "DB-API extension cursor.next() used"
Cursor Method .__iter__()
Return self to make cursors compatible to the iteration
protocol [8].
Warning Message: "DB-API extension cursor.__iter__() used"
Cursor Attribute .lastrowid
This read-only attribute provides the rowid of the last
modified row (most databases return a rowid only when a single
INSERT operation is performed). If the operation does not set
a rowid or if the database does not support rowids, this
attribute should be set to None.
The semantics of .lastrowid are undefined in case the last
executed statement modified more than one row, e.g. when
using INSERT with .executemany().
Warning Message: "DB-API extension cursor.lastrowid used"
Optional Error Handling Extensions
The core DB API specification only introduces a set of exceptions
which can be raised to report errors to the user. In some cases,
exceptions may be too disruptive for the flow of a program or even
render execution impossible.
For these cases and in order to simplify error handling when
dealing with databases, database module authors may choose to
implement user defineable error handlers. This section describes a
standard way of defining these error handlers.
Cursor/Connection Attribute .errorhandler
Read/write attribute which references an error handler to call
in case an error condition is met.
The handler must be a Python callable taking the following
arguments:
errorhandler(connection, cursor, errorclass, errorvalue)
where connection is a reference to the connection on which the
cursor operates, cursor a reference to the cursor (or None in
case the error does not apply to a cursor), errorclass is an
error class which to instantiate using errorvalue as
construction argument.
The standard error handler should add the error information to
the appropriate .messages attribute (connection.messages or
cursor.messages) and raise the exception defined by the given
errorclass and errorvalue parameters.
If no errorhandler is set (the attribute is None), the
standard error handling scheme as outlined above, should be
applied.
Warning Message: "DB-API extension .errorhandler used"
Cursors should inherit the .errorhandler setting from their
connection objects at cursor creation time.
Optional Two-Phase Commit Extensions
Many databases have support for two-phase commit (TPC) which
allows managing transactions across multiple database connections
and other resources.
If a database backend provides support for two-phase commit and
the database module author wishes to expose this support, the
following API should be implemented. NotSupportedError should be
raised, if the database backend support for two-phase commit
can only be checked at run-time.
TPC Transaction IDs
As many databases follow the XA specification, transaction IDs
are formed from three components:
* a format ID
* a global transaction ID
* a branch qualifier
For a particular global transaction, the first two components
should be the same for all resources. Each resource in the
global transaction should be assigned a different branch
qualifier.
The various components must satisfy the following criteria:
* format ID: a non-negative 32-bit integer.
* global transaction ID and branch qualifier: byte strings no
longer than 64 characters.
Transaction IDs are created with the .xid() connection method:
.xid(format_id, global_transaction_id, branch_qualifier)
Returns a transaction ID object suitable for passing to the
.tpc_*() methods of this connection.
If the database connection does not support TPC, a
NotSupportedError is raised.
The type of the object returned by .xid() is not defined, but
it must provide sequence behaviour, allowing access to the
three components. A conforming database module could choose
to represent transaction IDs with tuples rather than a custom
object.
TPC Connection Methods
.tpc_begin(xid)
Begins a TPC transaction with the given transaction ID xid.
This method should be called outside of a transaction
(i.e. nothing may have executed since the last .commit() or
.rollback()).
Furthermore, it is an error to call .commit() or .rollback()
within the TPC transaction. A ProgrammingError is raised, if
the application calls .commit() or .rollback() during an
active TPC transaction.
If the database connection does not support TPC, a
NotSupportedError is raised.
.tpc_prepare()
Performs the first phase of a transaction started with
.tpc_begin(). A ProgrammingError should be raised if this
method outside of a TPC transaction.
After calling .tpc_prepare(), no statements can be executed
until tpc_commit() or tpc_rollback() have been called.
.tpc_commit([xid])
When called with no arguments, .tpc_commit() commits a TPC
transaction previously prepared with .tpc_prepare().
If .tpc_commit() is called prior to .tpc_prepare(), a single
phase commit is performed. A transaction manager may choose
to do this if only a single resource is participating in the
global transaction.
When called with a transaction ID xid, the database commits
the given transaction. If an invalid transaction ID is
provided, a ProgrammingError will be raised. This form should
be called outside of a transaction, and is intended for use in
recovery.
On return, the TPC transaction is ended.
.tpc_rollback([xid])
When called with no arguments, .tpc_rollback() rolls back a
TPC transaction. It may be called before or after
.tpc_prepare().
When called with a transaction ID xid, it rolls back the given
transaction. If an invalid transaction ID is provided, a
ProgrammingError is raised. This form should be called
outside of a transaction, and is intended for use in recovery.
On return, the TPC transaction is ended.
.tpc_recover()
Returns a list of pending transaction IDs suitable for use
with .tpc_commit(xid) or .tpc_rollback(xid).
If the database does not support transaction recovery, it may
return an empty list or raise NotSupportedError.
Frequently Asked Questions
The database SIG often sees reoccurring questions about the DB API
specification. This section covers some of the issues people
sometimes have with the specification.
Question:
How can I construct a dictionary out of the tuples returned by
.fetch*():
Answer:
There are several existing tools available which provide
helpers for this task. Most of them use the approach of using
the column names defined in the cursor attribute .description
as basis for the keys in the row dictionary.
Note that the reason for not extending the DB API specification
to also support dictionary return values for the .fetch*()
methods is that this approach has several drawbacks:
* Some databases don't support case-sensitive column names or
auto-convert them to all lowercase or all uppercase
characters.
* Columns in the result set which are generated by the query
(e.g. using SQL functions) don't map to table column names
and databases usually generate names for these columns in a
very database specific way.
As a result, accessing the columns through dictionary keys
varies between databases and makes writing portable code
impossible.
Major Changes from Version 1.0 to Version 2.0
The Python Database API 2.0 introduces a few major changes
compared to the 1.0 version. Because some of these changes will
cause existing DB API 1.0 based scripts to break, the major
version number was adjusted to reflect this change.
These are the most important changes from 1.0 to 2.0:
* The need for a separate dbi module was dropped and the
functionality merged into the module interface itself.
* New constructors and Type Objects were added for date/time
values, the RAW Type Object was renamed to BINARY. The
resulting set should cover all basic data types commonly
found in modern SQL databases.
* New constants (apilevel, threadlevel, paramstyle) and
methods (.executemany(), .nextset()) were added to provide
better database bindings.
* The semantics of .callproc() needed to call stored
procedures are now clearly defined.
* The definition of the .execute() return value changed.
Previously, the return value was based on the SQL statement
type (which was hard to implement right) -- it is undefined
now; use the more flexible .rowcount attribute
instead. Modules are free to return the old style return
values, but these are no longer mandated by the
specification and should be considered database interface
dependent.
* Class based exceptions were incorporated into the
specification. Module implementors are free to extend the
exception layout defined in this specification by
subclassing the defined exception classes.
Post-publishing additions to the DB API 2.0 specification:
* Additional optional DB API extensions to the set of
core functionality were specified.
Open Issues
Although the version 2.0 specification clarifies a lot of
questions that were left open in the 1.0 version, there are still
some remaining issues which should be addressed in future
versions:
* Define a useful return value for .nextset() for the case where
a new result set is available.
* Integrate the decimal module Decimal object for use as
loss-less monetary and decimal interchange format.
Footnotes
[1] As a guideline the connection constructor parameters should be
implemented as keyword parameters for more intuitive use and
follow this order of parameters:
dsn Data source name as string
user User name as string (optional)
password Password as string (optional)
host Hostname (optional)
database Database name (optional)
E.g. a connect could look like this:
connect(dsn='myhost:MYDB',user='guido',password='234$')
[2] Module implementors should prefer 'numeric', 'named' or
'pyformat' over the other formats because these offer more
clarity and flexibility.
[3] If the database does not support the functionality required
by the method, the interface should throw an exception in
case the method is used.
The preferred approach is to not implement the method and
thus have Python generate an AttributeError in
case the method is requested. This allows the programmer to
check for database capabilities using the standard
hasattr() function.
For some dynamically configured interfaces it may not be
appropriate to require dynamically making the method
available. These interfaces should then raise a
NotSupportedError to indicate the non-ability
to perform the roll back when the method is invoked.
[4] a database interface may choose to support named cursors by
allowing a string argument to the method. This feature is
not part of the specification, since it complicates
semantics of the .fetch*() methods.
[5] The module will use the __getitem__ method of the parameters
object to map either positions (integers) or names (strings)
to parameter values. This allows for both sequences and
mappings to be used as input.
The term "bound" refers to the process of binding an input
value to a database execution buffer. In practical terms,
this means that the input value is directly used as a value
in the operation. The client should not be required to
"escape" the value so that it can be used -- the value
should be equal to the actual database value.
[6] Note that the interface may implement row fetching using
arrays and other optimizations. It is not
guaranteed that a call to this method will only move the
associated cursor forward by one row.
[7] The rowcount attribute may be coded in a way that updates
its value dynamically. This can be useful for databases that
return usable rowcount values only after the first call to
a .fetch*() method.
[8] Implementation Note: Python C extensions will have to
implement the tp_iter slot on the cursor object instead of the
.__iter__() method.
Acknowledgements
Many thanks go to Andrew Kuchling who converted the Python
Database API Specification 2.0 from the original HTML format into
the PEP format.
Many thanks to James Henstridge for leading the discussion which
led to the standardization of the two-phase commit API extensions.
Copyright
This document has been placed in the Public Domain.


Responsive Form
Remember me
Forget password?
LOGIN
Create New a account?
Signup
SIGN UP
Already have an account?
Login


Let's Rock with Adventure Club

Register Free
First Name:
Last Name:
E-mail:
Age:
Contact Us
|
Upcoming Tours


Twitoff lets you see which of two Twitter users is more likely to tweet a certain statement.
Two will enter the Twitoff Arena, and one will win. The victor--and the victim--are up to you alone.
Submit a twitter username to enter them as a Twitoff Contestant!
Current Twitoff Contestants:
{% for user in users %} {% endfor %}Now, select two Contestants and a tweet for them to battle over!


Declare an enum Currency.
It should include the following currency codes (according to ISO 4217):
You must include all of the above codes, others do not. The constant in the enum can be declared in any order.


pika activity pikas temperature model temperatures metabolic fur climate range American Niche observations heat biophysical data microclimate Ochotona sites time rate simulations used {border-style National field talus (Fig. Mapper active animal behavioral elevations minimum px;} #ffffff;} above‐talus behavior body change GNP Glacier high higher elevation environmental lapse observed summer July surface using Figure However, predicted produced Park, Table Wang below‐talus estimates modeled princeps) rates thermal thermoregulation Smith University allowable critical low maintain models relatively size warming (see empirical future measured species winter (Smith analyses conditions loggers properties BMR Ecol. Ecology Field chamber elevation, lower maximum mechanistic nocturnal px; refuge selection {font-family (e.g., Beever Department Moyer‐Horner Ray available especially example, font-size haying low‐elevation properties, relative remain responses role simulated temperatures, threshold understanding variation wind < Beever, Comparison Great Park Pikas USA. Weston ambient calls change. changes conducted cooler day double; factors foraging hours individual ln( mean molt north‐facing parameters pelage physiological potential predict provided reduce results seven short size, speed suggested support temperature, variables (MacArthur AIC Am. August Basin MacArthur Nat. Pika Porter Sierra Wisconsin‐Madison ability activity. analysis aspect border-color border-width considered daily direct distribution effects field‐based food generally impacts increasing mass measurements microclimates midday model, princeps ratio recorded regression response sites, slope slopes smaller solid; summers temperatures. tolerate variability {font-style {font-weight (:–:), .), Anderson Behavioral Brussard Can. Data FMR/ FMR/BMR GNP. Increasing Mammal. Mapper's Microclimate Model NicheMapper North Science Smith, Temperature Wildlife Wilkening [], above‐ accurately air aspect, balance behaviorally behaviors bold calculated capita complex consistent core correlative courier; day, demonstrated driving effect em; endotherm evening fur, greater hypotheses identified impact individuals inputs limit longer lowest mechanisms model. monthly organisms patterns pika. play predictions protected pt;} ranging relationship restrictions revealed scenario seasonality shift shifts site studies times varied varying weather work °C, –July


110
Sit at My Right Hand
A PSALM OF DAVID.
1 The LORD says to my Lord:
“Sit at my right hand,
until I make your enemies your footstool.”
2 The LORD sends forth from Zion
your mighty scepter.
Rule in the midst of your enemies!
3 Your people will offer themselves freely
on the day of your power,
in holy garments;
from the womb of the morning,
the dew of your youth will be yours.
4 The LORD has sworn
and will not change his mind,
“You are a priest forever
after the order of Melchizedek.”
5 The Lord is at your right hand;
he will shatter kings on the day of his wrath.
6 He will execute judgment among the nations,
filling them with corpses;
he will shatter chiefs
over the wide earth.
7 He will drink from the brook by the way;
therefore he will lift up his head.


{% extends "global/Page.html" %} {% load otree i18n%} {% block title %} {% trans "Results"%} {% endblock %} {% block content %} {% if player.id_in_group == 1 %}
{% blocktrans trimmed with sent_amount=group.sent_amount sent_back_amount=group.sent_back_amount %} You sent Participant B {{ sent_amount }}. Participant B returned {{ sent_back_amount}}. {% endblocktrans %}
{% else %}{% blocktrans trimmed with sent_amount=group.sent_amount sent_back_amount=group.sent_back_amount %} Participant A sent you {{ sent_amount }}. You returned {{sent_back_amount}}. {% endblocktrans %}
{% endif %}{% blocktrans trimmed with payoff=player.payoff %} Therefore, your total payoff is {{payoff}}. {% endblocktrans %}
{% include Constants.instructions_template %} {% endblock %}


I'm a data scientist and story teller with a passion for solving human-centered problems with rigorous hypothesis testing.
I am fascinated by the agility of NumPy and the no-nonsense grace of R. I have experience in Ruby on Rails and Javascript ES6, including React and Node.js. Take a look at some of my work below, or check out my GitHub, LinkedIn, or blog.
I'm a data scientist and story teller with a passion for solving human-centered problems with rigorous hypothesis testing.
I am fascinated by the agility of NumPy and the no-nonsense grace of R. I have experience in Ruby on Rails and Javascript ES6, including React and Node.js. Take a look at some of my work below, or check out my GitHub, LinkedIn, or blog.


Nyzo Chrome extension version 4 (commit on GitHub) adds support for automatic transactions.
This version does not place a rate limit on automatic transactions. Support for automatic transactions will not be submitted to the Chrome Web Store until such a limit is in place. To protect yourself if you choose to use this version of the extension, set a very small automatic transaction maximum (such as ∩0.000001) and use an account with a small balance.
In the content script, a listener was added to the document to respond when automatic transactions are generated. When a new transaction is received by the content script, the amount is set, if necessary, and the transaction is sent to the client.
The new sendTransaction() function is similar to the sendTransaction() function in the popup script. The parameters are checked to ensure a valid transaction can be built, and the amount is checked to ensure it does not exceed the maximum allowed amount for automatic transactions.
Events are dispatched to indicate success or failure of sending the transaction.
A failure event is also dispatched when invalid parameters prevent creation of the transaction.
In extensionUtil, constants were added for the number of micronyzos per nyzo and for the extension parameter names. The micronyzosPerNyzo constant is used in three places in this script to improve readability. The isValidMaximumAutomaticAmount() function was added to support the new maximum-amount parameter for automatic transactions.
The manifest was updated from version 2 to version 3. The change from browser_action to action was required by the new manifest version.
The uniqueReferenceKeyForMicropayConfiguration() function was moved to the micropayConfiguration script to allow reuse across more scripts.
The maximumAutomaticAmount field was added to the options menu. The notices were moved under the relevant fields.
In the options script, the new maximumAutomaticAmount field is handled in the same manner as the other fields: a listener is added for validation and storage in response to input, the value is loaded from local storage, and initial validation is performed.
The validateAndStoreMaximumAutomaticAmount() function was added to the options script. This function parallels the other validateAndStore... functions.
The width of 20rem of the popup was changed to a min-width, and a width of 100% was added. This does not change the appearance of the popup on desktop, but it can result in better presentation on mobile.
In the popup script, the maximumAutomaticAmount is now retrieved from local storage. The extensionConfigurationParameters are used for this. The chrome.tabs.getSelected() function call was replaced with a chrome.tabs.query() function call to support the new manifest version, and an additional block for configuring the popup for an error state was added. The remaining changes are indentation.
The changes to tip button configuration are all due to indentation.
Changes due to indentation continue. These are all a result of the additional error handling that was necessary when the chrome.tabs.getSelected() function call was replaced with a chrome.tabs.query() function call. The chrome.tabs.query() function call returns an empty result when the "Inspect popup" extension option is selected from the Chrome context menu, while chrome.tabs.getSelected() returned a valid tab.
The micronyzosPerNyzo constant and extensionConfigurationParameters constant were substituted for readability. A chrome.tabs.getSelected() function call was replaced with a chrome.tabs.query() function call.
Another chrome.tabs.getSelected() function call was replaced with a chrome.tabs.query() function call.
In isValidExtensionConfiguration(), the maximum automatic amount is now considered.
The uniqueReferenceKeyForMicropayConfiguration() function was moved to the micropayConfiguration script.
In the openOptionsPage() function, a chrome.extension.getURL() function call was changed to chrome.runtime.getURL() to support manifest version 3.
The attached-notice CSS class is like the notice class, but it uses a negative top margin to appear connected to the element above it.
The new attached-notice class is shown below in the options menu.
If you load this version of the extension and open the Micropay game example, you can see automatic transactions in action. The following table shows how the table on the game example should appear if the extension is configured correctly for automatic transactions. Note: this is just an example, and the extension is not used by these release notes.


Welcome back to another edition of Meme the News. This is my longest running series on the Steem blockchain. I’ve been doing this for over a year and a half now. It’s my favorite post I make and one I’m glad to keep working on. There are unfortunate side effects though. Warning- Meme the News may cause cogntitive disssonance, hair ejection, and freckled lungs.
Childish taking a shot at education
Weird new meme
badass lady.

After the election and….. IT’S GONE!
TROLOLOLOLOLOL
“Programming”
THEFT!!!
Yerp
Other than that the hyperviolent video is kinda neat
This page is synchronized from the post: ‘Meme the News- 5/16/18 (998k accounts on Steemit. Almost 1M!)’


Don Yang
http://uguu.org/
make
more generated1.c
more generated2.c
more generated3.c
make
./left < prog.c
./right < prog.c
./left < prog.c | ./left
./shift < prog.c
./msg0
./msg1
./msg2
Code should be readable in any direction! This tool will help you rotate your code so you don’t have to stare at it sideways or even upside down. Rotating this program left and recompiling will reveal other tools including a right rotate and a shift program.
Strange things happen when the world is upside down! It is entirely possible that this is remark is completely misleading.
What exactly does the shift program do?
Like a great sliding puzzle (hint) this entry has 6 more programs that will reveal messages and one more tool that can be used to reveal the final message hidden in the original source. All of these can be created using combinations of ./left, ./right and ./shift and the additionally generated programs.
The final message can be revealed using
./msg9 < prog.c
But what combinations will generate ./msg3, ./msg4, ./msg5, ./msg6, ./msg7, ./msg8 and finally ./msg9?
Nuko is a text rotator: given some text in stdin, Nuko will write the same text to stdout, but rotated 90 degrees counterclockwise.
gcc prog.c -o left
./left < input.txt > rotated_counterclockwise.txt
Due to static memory allocation, only the first 1K columns by 1K rows are rotated, the rest are silently ignored. But fear not, a separate tool with dynamic memory allocation is included:
./left < prog.c > r1.c
gcc r1.c -o right
./right < input.txt > rotated_clockwise.txt
For variety, this second tool rotates clockwise instead of counterclockwise. Also, it can handle files larger than 1024x1024, depending on how much memory you got.
Of course we wouldn’t stop with just two rotations, if we continue to rotate counterclockwise once more, we get another program. This program outputs a single message to stdout, which is the name of the series that Nuko came from.
./left < prog.c | ./left > r2.c
gcc r2.c -o msg0
./msg0
There is one final rotation, which produces a tool that removes leading whitespaces:
./left < prog.c | ./left | ./left > r3.c
gcc r3.c -o shift
./shift < input.txt > no_leading_space.txt
Where this might be useful, besides ruining the formatting of certain files, is that it completes the set of tools needed to solve the puzzle that is embedded in prog.c
Notice how the edges of prog.c contain two notches. By rotating prog.c and removing leading space, the code would be shifted one space toward one of those notches (and creating a new notch on the other side). This shifted code behaves slightly different from the original program. For example, here are two more messages that can be produced:
./shift < prog.c | ./right > msg1.c && gcc msg1.c -o msg1 && ./msg1
./shift < prog.c | ./left > msg2.c && gcc msg2.c -o msg2 && ./msg2
In total, there are 9 embedded strings that can be produced via a sequence of rotates and shifts, one of which can be used to extract the 10th final string from prog.c. The intent is to simulate those wooden puzzle boxes that can be opened by pushing and shifting various well-concealed seams. Thus, finding the correct sequence of rotates and shifts is left as an exercise to the reader (but if you are really lazy, just read the Makefile).
Source code for all the tools needed to solve this puzzle are embedded in prog.c, all you need is a C compiler.
Code compiles when rotated 4 ways. This required a bit of patience to achieve. Code still compiles even with one column of text shifted. This required even more patience.
All rotated and shifted variants compiles without warnings. This involves various tweaks to satisfy cases where compiler is overly protective, including but not limited to the “1125” at line 4 as opposed to “1025”, to satisfy -Waggresive-loop-optimizations.
CRC32 of the code is embedded in the code itself.
Process for writing prog.c is available in spoiler.html
Nuko and the rotated tools accepts only ASCII files where each character maps to exactly one byte. Also, end-of-line sequence is assumed to be LF only, as opposed to CR-LF. All control characters are treated like normal printable characters, so files containing tabs will look weird after rotation, for example.
Nuko has been verified to work with these compiler / OS combinations:
Nuko compiles without warnings with all compilers above, even with “-Wall -Wextra -pedantic” for gcc and clang.
© Copyright 1984-2018,
Leo Broukhis, Simon Cooper, Landon Curt Noll
- All rights reserved |


{% extends "blog/main.html" %} {% load i18n %} {% get_available_languages as languages %} {% block title %}{{ article.title }}{% endblock %} {% block content %}
{% trans "We found this story, but right now it not available because it is in review." %}
{% trans "If you are the author of this story, you can edit it in your profile." %}
{{ article.text }}
{% if tags %}


To include a field in your tag, click on the checkbox next to the field and either enter a value for the field in the text box (if the field is of type string, double or datetime) or choose a value for the field (if the field is of type enum or bool).
Mandatory template fields are marked by * and are already checked for you.


Finda by SMURT
We are bunch young aspiring techpreneurs’, who have been working together for several years.
Copyright © 2020 SMURT. All rights reserved.


Lorem Ipsum is simply dummy text of the industry.Lorem Ipsum is simply dummy text of the industry.Lorem Ipsum is simply dummy text of the industry.Lorem Ipsum is simply dummy text of the industry. Lorem Ipsum is simply dummy text of the industry.Lorem Ipsum is simply dummy text of the industry.Lorem Ipsum is simply dummy text of the industry.


Since I was tired of using the flash circuit hack (my version) to power my tube experiments I finally decided to build a isolation transformer using two identical step-down transformers as suggested by Mike in his nixie clock documentation. Here are some photos of it:
I’m planing to build a adjustable HV lab power supply in the future (which I may sell as a kit), so stay tuned.
As I’ve described in Unbelievable Prices, I bought a crappy chinese MP3 player to use as a “true” sound source (instead of the function generator I use while early-testing my audio projects) for testing my audio circuits without having to worry about accidentally shorting 12V into my iPod’s headphone jack or something like that while probing around.
Yesterday it arrived and as it was expected, it’s the best of Shenzhen. Horrible plastics and build quality, the buttons are super stiff, and overall a shitty product as it was expected to be, but since I’m only going to use it for testing, I don’t care. Here are some pictures of the “beautiful” thing:
As you can see it’s a typical chinese product. The LiPo battery has no markings, except for a weird XI logo, it doesn’t look like a protected pack and the flimsy wires that connect it to the main board can snap off at any second and short the thing out.
Right next to what looks like the main processor, which sadly I couldn’t find any information about it, there’s a very nicely heat-shrunk clock crystal. On the center of the board you can see a generic 4871 audio power amplifier, and on the left side there’s a BK1080 FM receiver IC.
On the other side of the board all you can see is the horrible LCD and the shittiest buttons you can buy in the Shenzhen market.
This week I’ve been experimenting with a very simple and cheap project for wireless transmissions, a lightwave AM transmitter and receiver based on Scott’s design, which was based on VK2ZAY’s design. In my final design I’ve increased the base biasing resistors to decrease the size of the coupling capacitor and also used a darlington transistor to get more current gain.
The transmitter is pretty straight forward, the input modulates the current passing by the LED, which modulates the intensity of light, if you’ve designed any class A amplifiers in the past you surely know how it works. The receiver is just a simple transimpedance amplifier, which is amplifying the signal quite a bit (~56x gain) since the transmitter will usually be a bit far from the receiver. You can do the same with a op-amp, but I much prefer a discrete circuit for these simple things.
You can put a buffer stage with a darlington emitter follower on the output of the receiver so you can drive a speaker directly. Something that I would recommend is to add a small (10x gain maybe?) pre-amplifier for the transmitter, that way you’ll get a bit more signal if you’re source isn’t very loud, specially if you want to drive some high power LEDs, since you have a lot of current headroom with those.
If you want to experiment with different values in a simulation, here is the LTspice schematic. The best way to choose the best LED + photodiode combination to maximize the range is to build some breakout boards that you can plug different LEDs and photodiodes until you have the perfect combination.
 The last thing I want is to short something or accidentally apply a (relatively) high voltage to one of the audio pins, or do anything that could damage my device.
So, I quickly went to MercadoLivre, the brazilian version of eBay, and searched for the cheapest ($3 multimeter quality) MP3 player available, and sure enough, this popped up.  That’s just insane, they are probably assembling millions of those to be able to produce them at that price.


If you see this page, the web server is up and running.
For online documentation and support please refer to nginx.org. Commercial support is available at nginx.com.


Binary Heaps: Min-Heaps and Max-Heaps
I owe much of my learning & implementation to Eloquent Javascript by Marijn Haverbeke.
Binary heaps are partially ordered data structures in the form of binary trees.

In a Binary Search Tree, the left child is less than the parent and the right child is greater than the parent. In a Binary Heap, there is no ordering of the children relative to the parent other than children being strictly less or strictly greater than the parent.
Big-O Breakdown
Binary Heaps have O(log n) complexity for insertion and deletion in the worst case, and O(1) complexity for removal (of the min or max element).
To insert an element, we add it to the very end of the tree and then swap its position with adjacent nodes until it satisfies the ordering property of the heap.
We can delete a specified element from the heap in O(log n) time by performing a binary search for the element and then removing it. However, min and max heaps are optimized for removal of the root element in O(1) time, rather than supporting arbitrary deletion.
Implementing Heaps
Heaps are often implemented using an array . When implementing a heap as an array, we’re creating what’s called an implicit data structure . Whereas in a traditional binary tree implementation
Given a tree array with index starting at 1, for any element with index idx, its parent is located at Math.floor(idx/2)
, and its two children are located at 2*idx
and 2*idx+1
.
min-heap:
1
/ \
2 3
/ \ / \
4 11
1-indexed array:
values: x 1 2 3 4 11
indices: 0 1 2 3 4 5
0-indexed array:
values: 1 2 3 4 11
indices: 0 1 2 3 4
We could start the array at index 0, but it would make the math just a little uglier.
If we did use a zero indexed array, the left and right children would be located at idx*2+1
and idx*2+2
respectively, and the parent would be located at Math.floor((idx-1)/2)
.
Simple BinaryHeap Object
Let’s implement a Binary Heap.
We’ll let the user supply a comparator function that will provide a “score” for each element that will be used to compare two elements. The heap will be a array with index starting at 1 to simplify subscripts for parents and children. The zero index, null, will never be accessed.
var BinaryHeap = function (comparator) {
this.heap = [null];
this.comparator = comparator;
};
If we’re ordering numbers or other elements with lexicographic ordering, the scoring function can just be the identity function: function(x) { return x; }
. The scoring function allows us to define either a minimum or a maximum heap (by changing the sign of the returned element).
To start, we’ll define a few helper methods. score
will just call the specified comparator function on a given value of an element at index idx
.
BinaryHeap.prototype.score = function (idx) {
return this.comparator.call(this, this.heap[idx]);
};
maxIdx
returns the current heap length-1
, which represents the largest array index we can match without going out of bounds.
BinaryHeap.prototype.maxIdx = function () {
return this.heap.length - 1;
};
swap
swaps two values in the heap at indices a and b.
BinaryHeap.prototype.swap = function (a, b) {
var temp = this.heap[b];
this.heap[b] = this.heap[a];
this.heap[a] = temp;
};
Insert / Push
BinaryHeap.prototype.push = function (value) {
this.heap.push(value);
this.bubbleUp(this.heap.length - 1);
};
To insert values, we simply push them onto the end of the array and then call bubbleUp
with the index of the newly inserted element. What does bubbleUp do?
bubbleUp
moves the newly inserted element up in the tree until its score is greater than or equal to that of its parent.
BinaryHeap.prototype.bubbleUp = function (idx) {
while (idx > 1) {
var eleScore = this.score(idx);
var parentIdx = Math.floor(idx / 2);
var parentScore = this.score(parentIdx);
if (eleScore < parentScore) {
this.swap(parentIdx, idx);
idx = Math.floor(idx / 2);
} else {
break;
}
}
};
If the score is less than that of the parent, we want to swap out the current index with the index of the parent. The parent index is always given by the floor of the current index divided by two, so long as idx>1
. Remember, we are never accessing idx=0, and when idx=1, Math.floor(idx/2) => Math.floor(1/2) => 0.
Delete / Pop
We must account for the fact that children are not necessarily ordered when we delete or “pop” from the heap. The algorithm for deletion is:
- Remove the element at index 1. This is the value that will be returned.
- Pop the last value from the heap. This is the value we’ll use to replace index 1.
- Make sure the index 1 element !== the last value from the heap.
- If it does, we’re removing the final element from the heap: simply return it.
- If the index 1 element isn’t the last value of the heap, put the last value at index 1.
- Call
siftDown
on the element at index 1.
siftDown
is a method that propagates a newly added element down the tree until it is in the proper place given the heap’s ordering.
BinaryHeap.prototype.siftDown = function (idx) {
const max = this.maxIdx();
while (idx < max) {
var eleScore = this.score(idx);
var leftChild = idx * 2;
var rightChild = leftChild + 1;
var swap = null;
if (leftChild <= max) {
var leftChildScore = this.score(leftChild);
if (leftChildScore < eleScore) {
swap = leftChild;
}
}
if (rightChild <= max) {
if (this.score(rightChild) < (swap ? leftChildScore : eleScore)) {
swap = rightChild;
}
}
if (!swap) {
break;
} else {
this.swap(idx, swap);
idx = swap;
}
}
};
What we’re doing is this:
- Calculating the index for leftChild and rightChild
- Ensuring the index is in bounds
- If the index is in bounds, comparing the element score with the given child’s score.
- If the element has a score greater than one of the children’s scores, we need to swap these two elements.
For each iteration of the loop in siftDown
, assuming that a left and right child exist, the inserted element could either be:
- In the correct place (
break
the loop) - Greater than the left child but less than the right child (swap the left child and the element)
- Greater than both the left and the right child (swap the right child and the element)
After performing the comparisons, we check to see if we are due to swap elements. If so, we perform the swap between the element and either the left or the right child.
The most confusing part is the ternary expression if (this.score(rightChild) < (swap ? leftChildScore : eleScore))
in the if (rightChild <= max)
block.
This is required because it could be that the value of the inserted element is greater than the leftChild
value but less than the rightChild
value. In this case, we would not want to swap the element with rightChild
.
Consider this tree that illustrates the case:
4 (A)
/ \
3 (B) 7 (C)
If we did swap the element with the right child, we know it would create an invalid heap: for three numbers A, B, C, if A > B < C, it follows that A < C, and C cannot become A’s parent. B can become the parent of A and C, however.
Without the ternary expression, the tree above would become this (if we just compared the inserted element’s score with each child individually):
7
/ \
3 4
For each parent (A) we ask: is A greater than B? If not, we’re done. If so, then we’re going to swap it at least with B, and potentially with C as well. Now is C less than B? If not, swap A and B. Otherwise, swap A and C. Now consider the new tree generated by the movement to the left (swapping A with B) or right (swapping A with C).
Repeat until either A is less than both its left (B) and right (C) child or until we reach the end of the tree.
Let’s visualize how this algorithm works for the following min-heap:
4
/ \
9 14
/ \ / \
14 12 30 10
Note again this is a valid min-heap but an invalid binary search tree.
We remove the element 4.
4 does not equal the last element in the tree (10), so we know the tree is not empty.
We swap 10 with 4’s position. The tree now looks like this:
10
/ \
9 14
/ \ / \
14 12 30
We then call siftDown
on 8. Let’s trace it through the tree. leftChild is 10, and rightChild is 8.
9 is less than 10, so swap is equal to 10. 14 is not less than 10, so we swap out 9 and 10.
9
/ \
10 14
/ \ / \
14 12 30
Now we continue comparing, since we are not yet at idx==max
.
leftChild is 14. 10 is not greater than 14, so we continue. rightChild is 12. swap is null, so we compare the rightChild score with the element score.
The right child is not less than the element, so we’re done.
Notice this is a complete binary tree (but not a binary search tree): all levels are filled to the maximum amount, and unfilled levels are populated from left-to-right.
Consider another shorter case where a deletion happens to place the element in the correct position from the start (best-case insertion):
4
/ \
10 7
/ \ / \
14 12 9 8
We remove the element 4.
4 does not equal the last element in the tree (8), so we know the tree is not empty.
We swap 8 with 4’s position. The tree now looks like this:
8
/ \
10 7
/ \ / \
14 12 9
We then call siftDown
on 8. Let’s trace it through the tree. leftChild is 10, and rightChild is 8.
10 is not less than 8, so swap is equal to 10. 8 is not less than 8, so we stop.
HeapSort
Organizing elements in a binary heap gives them the convenient property of being able to be sorted very quickly. Notice that each time we perform a pop
operation on a min-heap, we retrieve the next ascending sorted element of the heap’s values; similarly, when we pop
a max-heap, we retrieve the next descending sorted element.
This can be used to implement a priority queue by high or low priority.
We can also use this ordered property of heaps to implement HeapSort .
BinaryHeap.prototype.heapSort = function () {
const copy = this.heap.slice(0);
let res = [];
const max = this.maxIdx();
for (var i = 0; i < max; i++) {
res.push(this.pop());
}
this.heap = copy;
return res;
};
We don’t want to destroy the heap when we return its elements in a sorted manner, so we want to somehow copy and restore the heap after the sorting operation. There are a few ways that we could go wrong in implementing this.
- We might think that we could just say:
for (var i=0; i<this.maxIdx(); i++)
.
- This is a bad idea:
this.maxIdx()
changes with each invocation ofthis.pop()
since removing an element shortens an array by one each time. We’ll only get halfway through the array and then stop.
- We might think that we can copy the heap like this:
const copy = this.heap
.
- This would be incorrect:
copy
would hold a reference tothis.heap
, but not the value ofthis.heap
. Usingarray.slice()
(Docs) solves this problem, but with a caveat! Note from the documentation thatArray.slice
will copy strings, numbers, and booleans, but not objects.
If our heap contains an array of objects that we order by a given key, say priority
, heapSort will wipe out each of these elements, because copy
will contain an array of references, not a deep copy of the objects contained.
To implement this copying in a robust manner, we should really define a custom copy method for our array, so that we could simply say: copy = this.copyHeap()
.
If the objects in the heap are large enough, it would make far more sense to simply create a map between heap indices and the key upon which they are sorted, sort the map array, and return it. This array, when iterated through from start to finish, will be the order in which we access elements of the heap to read them in ascending or descending order.
BinaryHeap.prototype.orderMap = function (dir = 1) {
// 1 ascending, -1 descending
let priorities = this.heap
.map((e, idx) => (
idx !== 0 ? { key: this.comparator(e), idx: idx } : null
)
).filter((e) => e);
return priorities.sort((a, b) => a.key >= dir * b.key).map((e) => e.idx);
};
Since the elements are stored in an array for which we can look up the value of any cell in O(1) time, sorting is actually just an abstract operation of accessing cells in sorted order. Whether or not the cells are sorted when accessed in ascending or descending sequence of its index is incidental, and just an often convenient convention.
Indeed, it would perhaps make more sense to store a reference to each priority object in an array that is bound to a key on which we sort by and instead insert these priority->reference mappings into a heap. Making these decisions is implementation-specific, but it’s good to think about the different ways you could achieve the same ends.
It is of course trivial to sort a heap that already exists. If we had an unsorted array, say, [3, 4, 70, 13, 11, -4, 7], we could easily sort it using heapSort like so:
function heapSortArray(arr, dir = 1) {
let sortedArray = new BinaryHeap((x) => dir * x);
for (var i = 0; i < arr.length; i++) {
sortedArray.push(arr[i]);
}
return sortedArray.heapSort();
}
Converting a min heap into a max heap
It’s an interesting tidbit that we can convert a min heap into a max heap (or vice versa) by reversing the comparator function’s direction and then calling siftDown
on indices [Math.floor(heapLength/2), ..., 1]
.
let T = new BinaryHeap(function (x) {
return x;
});
T.push(4);
T.push(3);
T.push(5);
T.push(7);
T.push(13);
T.push(22);
console.log(T.heapSort());
// [3, 4, 5, 7, 13, 22]
T.comparator = function (x) {
return -x;
};
let len = T.maxIdx();
for (var i = Math.floor(len / 2); i > 0; i--) {
T.siftDown(i);
}
console.log(T.heapSort());
// [22, 13, 7, 5, 4, 3]
This is an application of Floyd's method of heap construction applied in-place to an existing heap structure. Kinda cool!
Wrapping Up
Congratulations! You’ve mastered yet another simple data structure. Try reviewing the structures you’ve already learned and, most important, coding them from scratch, perhaps even with a pen and paper. It’s one thing to read code and conceptually understand. It’s far different to wrestle with off-by-one errors and indexing foibles.
Practice hones intuition, and sufficient intuition is indiscernible from mastery.


#assign subtitle = "Clustering"> <#assign clustering = "true">
...(under construction)... needs reviewed
Use ContainerConfig Configuration for the container that holds the EbeanServer instances.
Provides configuration for cluster communication (if clustering is used). The cluster communication is used to invalidate appropriate parts of the L2 cache across the cluster.
```java private ContainerConfig createContainerConfig(String local, String threadPoolName) { ContainerConfig container = new ContainerConfig(); container.setMode(ContainerConfig.ClusterMode.SOCKET); // also CusterMode.MULTICAST ContainerConfig.SocketConfig socketConfig = new ContainerConfig.SocketConfig(); socketConfig.setLocalHostPort(local); socketConfig.setThreadPoolName(threadPoolName); socketConfig.setMembers(Arrays.asList("<IP_ADDRESS>:9876", "<IP_ADDRESS>:9866")); container.setSocketConfig(socketConfig); return container; } ``` ```java // Startup server 1 ContainerConfig container = createContainerConfig("<IP_ADDRESS>:9876", "pool0"); ClusterManager mgr = new ClusterManager(container); // Create your ebean server as you normally do EbeanServer server = ...; // Register the EbeanServer with the cluster manger mgr.registerServer(server); // Register the cluster manager with the cluster mgr.startup(); // Deregister the cluster manager from the cluster mgr.shutdown(); ``` Same code different port: ```java // Startup server 2 ContainerConfig container = createContainerConfig("<IP_ADDRESS>:9866", "pool1"); ClusterManager mgr = new ClusterManager(container); ServerConfig serverConfig = ...; EbeanServer server = EbeanServerFactory.create(config); mgr.registerServer(server); // Create your ebean server as you normally do EbeanServer server = ...; // Register the EbeanServer with the cluster manger mgr.registerServer(server); // Register the cluster manager with the cluster mgr.startup(); // Deregister the cluster manager from the cluster mgr.shutdown(); ```The BeanPersistListener listens for committed bean events.
These listen events occur after a successful commit. They also occur in a background thread rather than the thread used to perform the actual insert update or delete. In this way there is a delay between the commit and when the listener is notified of the event.
For a cluster these events may need to be broadcast. Each of the inserted(), updated() and deleted() methods return true if you want those events to be broadcast to the other members of a cluster (the id values are broadcast). If these methods return false then the events are not broadcast.
Ebean.externalModification(String tableName, boolean inserted, boolean updated, boolean deleted) processes committed changes from another framework or clustered server
Calling Ebean.externalModification notifies Ebean that beans have been committed externally, either by another framework or clustered server. Ebean uses this information to maintain its cache and lucene indexes appropriately.


Home
Travel
Gifts
RSVP
Home
Travel
Gifts
RSVP
menu
RSVP
We hope you can make it!
First Name
Last Name
Email
Choose your option
Option 1
Option 2
Option 3
Who will be attending?
Arrival Date
Arrival Time


Romanize Actual Numbers Converter
Enter Number you would like to convert to Roman Numerals:
Romanize