|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.simpleframework.xml.core.Persister
public class Persister
The Persister
object is used to provide an implementation
of a serializer. This implements the Serializer
interface
and enables objects to be persisted and loaded from various sources.
This implementation makes use of Filter
objects to
replace template variables within the source XML document. It is fully
thread safe and can be shared by multiple threads without concerns.
Deserialization is performed by passing an XML schema class into one
of the read
methods along with the source of an XML stream.
The read method then reads the contents of the XML stream and builds
the object using annotations within the XML schema class.
Serialization is performed by passing an object and an XML stream into
one of the write
methods. The serialization process will
use the class of the provided object as the schema class. The object
is traversed and all fields are marshalled to the result stream.
Serializer
| Constructor Summary | |
|---|---|
Persister()
Constructor for the Persister object. |
|
Persister(Filter filter)
Constructor for the Persister object. |
|
Persister(Filter filter,
Format format)
Constructor for the Persister object. |
|
Persister(Filter filter,
Matcher matcher)
Constructor for the Persister object. |
|
Persister(Filter filter,
Matcher matcher,
Format format)
Constructor for the Persister object. |
|
Persister(Format format)
Constructor for the Persister object. |
|
Persister(java.util.Map filter)
Constructor for the Persister object. |
|
Persister(java.util.Map filter,
Format format)
Constructor for the Persister object. |
|
Persister(Matcher matcher)
Constructor for the Persister object. |
|
Persister(Matcher matcher,
Format format)
Constructor for the Persister object. |
|
Persister(Strategy strategy)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Filter filter)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Filter filter,
Format format)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Filter filter,
Matcher matcher)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Filter filter,
Matcher matcher,
Format format)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Format format)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
java.util.Map data)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
java.util.Map data,
Format format)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Matcher matcher)
Constructor for the Persister object. |
|
Persister(Strategy strategy,
Matcher matcher,
Format format)
Constructor for the Persister object. |
| Method Summary | ||
|---|---|---|
|
read(java.lang.Class<? extends T> type,
java.io.File source)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.io.File source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
InputNode source)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
InputNode node,
boolean strict)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.io.InputStream source)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.io.InputStream source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.io.Reader source)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.io.Reader source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.lang.String source)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(java.lang.Class<? extends T> type,
java.lang.String source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. |
|
|
read(T value,
java.io.File source)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.io.File source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
InputNode source)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
InputNode node,
boolean strict)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.io.InputStream source)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.io.InputStream source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.io.Reader source)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.io.Reader source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.lang.String source)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
|
read(T value,
java.lang.String source,
boolean strict)
This read method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. |
|
boolean |
validate(java.lang.Class type,
java.io.File source)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.io.File source,
boolean strict)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
InputNode source)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
InputNode node,
boolean strict)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.io.InputStream source)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.io.InputStream source,
boolean strict)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.io.Reader source)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.io.Reader source,
boolean strict)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.lang.String source)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
boolean |
validate(java.lang.Class type,
java.lang.String source,
boolean strict)
This validate method will validate the contents of
the XML document against the specified XML class schema. |
|
void |
write(java.lang.Object source,
java.io.File out)
This write method will traverse the provided object
checking for field annotations in order to compose the XML data. |
|
void |
write(java.lang.Object source,
OutputNode root)
This write method will traverse the provided object
checking for field annotations in order to compose the XML data. |
|
void |
write(java.lang.Object source,
java.io.OutputStream out)
This write method will traverse the provided object
checking for field annotations in order to compose the XML data. |
|
void |
write(java.lang.Object source,
java.io.OutputStream out,
java.lang.String charset)
This write method will traverse the provided object
checking for field annotations in order to compose the XML data. |
|
void |
write(java.lang.Object source,
java.io.Writer out)
This write method will traverse the provided object
checking for field annotations in order to compose the XML data. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public Persister()
Persister
object. This is used
to create a serializer object that will use an empty filter.
This means that template variables will remain unchanged within
the XML document parsed when an object is deserialized.
public Persister(Format format)
Persister
object. This is used
to create a serializer object that will use the provided format
instructions. The persister uses the Format
object
to structure the generated XML. It determines the indent size
of the document and whether it should contain a prolog.
format
- this is used to structure the generated XMLpublic Persister(java.util.Map filter)
Persister
object. This is used
to create a serializer object that will use a platform filter
object using the overrides within the provided map. This means
that template variables will be replaced firstly with mappings
from within the provided map, followed by system properties.
filter
- this is the map that contains the overridespublic Persister(java.util.Map filter, Format format)
Persister
object. This is used
to create a serializer object that will use a platform filter
object using the overrides within the provided map. This means
that template variables will be replaced firstly with mappings
from within the provided map, followed by system properties.
filter
- this is the map that contains the overridesformat
- this is the format used to format the documentspublic Persister(Filter filter)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
filter
- the filter used to replace template variablespublic Persister(Filter filter, Format format)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
filter
- the filter used to replace template variablesformat
- this is used to structure the generated XMLpublic Persister(Matcher matcher)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
matcher
- this is used to customize the transformationspublic Persister(Matcher matcher, Format format)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
matcher
- this is used to customize the transformationsformat
- this is used to structure the generated XMLpublic Persister(Strategy strategy)
Persister
object. This is used
to create a serializer object that will use a strategy object.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classespublic Persister(Strategy strategy, Format format)
Persister
object. This is used
to create a serializer object that will use a strategy object.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesformat
- this is used to structure the generated XMLpublic Persister(Filter filter, Matcher matcher)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
filter
- the filter used to replace template variablesmatcher
- this is used to customize the transformationspublic Persister(Filter filter, Matcher matcher, Format format)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
filter
- the filter used to replace template variablesmatcher
- this is used to customize the transformationsformat
- this is used to structure the generated XMLpublic Persister(Strategy strategy, java.util.Map data)
Persister
object. This is used
to create a serializer object that will use a platform filter
object using the overrides within the provided map. This means
that template variables will be replaced firstly with mappings
from within the provided map, followed by system properties.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesdata
- this is the map that contains the overridespublic Persister(Strategy strategy, java.util.Map data, Format format)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesdata
- the filter data used to replace template variablesformat
- this is used to format the generated XML documentpublic Persister(Strategy strategy, Filter filter)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesfilter
- the filter used to replace template variablespublic Persister(Strategy strategy, Filter filter, Format format)
Persister
object. This is used
to create a serializer object that will use the provided filter.
This persister will replace all variables encountered when
deserializing an object with mappings found in the filter.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesfilter
- the filter used to replace template variablesformat
- this is used to format the generated XML documentpublic Persister(Strategy strategy, Matcher matcher)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesmatcher
- this is used to customize the transformationspublic Persister(Strategy strategy, Matcher matcher, Format format)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesmatcher
- this is used to customize the transformationsformat
- this is used to format the generated XML documentpublic Persister(Strategy strategy, Filter filter, Matcher matcher)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesmatcher
- this is used to customize the transformationsfilter
- the filter used to replace template variablespublic Persister(Strategy strategy, Filter filter, Matcher matcher, Format format)
Persister
object. This is used
to create a serializer object that will use the provided matcher
for customizable transformations. The Matcher
will
enable the persister to determine the correct way to transform
the types that are not annotated and considered primitives.
This persister will use the provided Strategy
to
intercept the XML elements in order to read and write persistent
data, such as the class name or version of the document.
strategy
- this is the strategy used to resolve classesmatcher
- this is used to customize the transformationsfilter
- the filter used to replace template variables| Method Detail |
|---|
public <T> T read(java.lang.Class<? extends T> type, java.lang.String source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.File source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.InputStream source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.Reader source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, InputNode source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.lang.String source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.File source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.InputStream source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, java.io.Reader source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(java.lang.Class<? extends T> type, InputNode node, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and convert it into an object
of the specified type. If the XML source cannot be deserialized
or there is a problem building the object graph an exception
is thrown. The instance deserialized is returned.
read
in interface Serializer
type
- this is the class type to be deserialized from XMLnode
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.lang.String source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.File source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.InputStream source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.Reader source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, InputNode source) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML document
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.lang.String source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.File source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.InputStream source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, java.io.Reader source, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tosource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic <T> T read(T value, InputNode node, boolean strict) throws java.lang.Exception
read
method will read the contents of the XML
document from the provided source and populate the object with
the values deserialized. This is used as a means of injecting an
object with values deserialized from an XML document. If the
XML source cannot be deserialized or there is a problem building
the object graph an exception is thrown.
read
in interface Serializer
value
- this is the object to deserialize the XML in tonode
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the object cannot be fully deserializedpublic boolean validate(java.lang.Class type, java.lang.String source) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.File source) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.InputStream source) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.Reader source) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, InputNode source) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML document
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.lang.String source, boolean strict) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.File source, boolean strict) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.InputStream source, boolean strict) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, java.io.Reader source, boolean strict) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLsource
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the class XML schema does not fully matchpublic boolean validate(java.lang.Class type, InputNode node, boolean strict) throws java.lang.Exception
validate
method will validate the contents of
the XML document against the specified XML class schema. This is
used to perform a read traversal of the class schema such that
the document can be tested against it. This is preferred to
reading the document as it does not instantiate the objects or
invoke any callback methods, thus making it a safe validation.
validate
in interface Serializer
type
- this is the class type to be validated against XMLnode
- this provides the source of the XML documentstrict
- this determines whether to read in strict mode
java.lang.Exception
- if the class XML schema does not fully matchpublic void write(java.lang.Object source, OutputNode root) throws java.lang.Exception
write
method will traverse the provided object
checking for field annotations in order to compose the XML data.
This uses the getClass
method on the object to
determine the class file that will be used to compose the schema.
If there is no Root
annotation for the class then
this will throw an exception. The root annotation is the only
annotation required for an object to be serialized.
write
in interface Serializer
source
- this is the object that is to be serializedroot
- this is where the serialized XML is written to
java.lang.Exception
- if the schema for the object is not validpublic void write(java.lang.Object source, java.io.File out) throws java.lang.Exception
write
method will traverse the provided object
checking for field annotations in order to compose the XML data.
This uses the getClass
method on the object to
determine the class file that will be used to compose the schema.
If there is no Root
annotation for the class then
this will throw an exception. The root annotation is the only
annotation required for an object to be serialized.
write
in interface Serializer
source
- this is the object that is to be serializedout
- this is where the serialized XML is written to
java.lang.Exception
- if the schema for the object is not validpublic void write(java.lang.Object source, java.io.OutputStream out) throws java.lang.Exception
write
method will traverse the provided object
checking for field annotations in order to compose the XML data.
This uses the getClass
method on the object to
determine the class file that will be used to compose the schema.
If there is no Root
annotation for the class then
this will throw an exception. The root annotation is the only
annotation required for an object to be serialized.
write
in interface Serializer
source
- this is the object that is to be serializedout
- this is where the serialized XML is written to
java.lang.Exception
- if the schema for the object is not validpublic void write(java.lang.Object source, java.io.OutputStream out, java.lang.String charset) throws java.lang.Exception
write
method will traverse the provided object
checking for field annotations in order to compose the XML data.
This uses the getClass
method on the object to
determine the class file that will be used to compose the schema.
If there is no Root
annotation for the class then
this will throw an exception. The root annotation is the only
annotation required for an object to be serialized.
source
- this is the object that is to be serializedout
- this is where the serialized XML is written tocharset
- this is the character encoding to be used
java.lang.Exception
- if the schema for the object is not validpublic void write(java.lang.Object source, java.io.Writer out) throws java.lang.Exception
write
method will traverse the provided object
checking for field annotations in order to compose the XML data.
This uses the getClass
method on the object to
determine the class file that will be used to compose the schema.
If there is no Root
annotation for the class then
this will throw an exception. The root annotation is the only
annotation required for an object to be serialized.
write
in interface Serializer
source
- this is the object that is to be serializedout
- this is where the serialized XML is written to
java.lang.Exception
- if the schema for the object is not valid
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


This is an example of fetching best times from speedrun.com for display on a wiki (ie twitch panels). Eventually will add other features (configurable output style?). The eventual goal would be to have uploads to speedrun.com update your twitch page automagically but I'm not sure that's possible given current twitch api.
It will output in the form:
where the time will link to the video and the game will link to the speedrun.com page.
Filter out places above value:
append world record holder:
Give this script credit. Put link to it at the end:
username:


OpenGL Interface Implementation¶
See OpenGL Interface for details on the publically-visible modules.
See ctypes Wrapper Generation for details on some of these modules are generated.
ctypes linkage¶
Most functions link to libGL.so (Linux), opengl32.dll (Windows) or OpenGL.framework (OS X). pyglet.gl.lib provides some helper types then imports linker functions for the appropriate platform: one of pyglet.gl.lib_agl, pyglet.gl.lib_glx, pyglet.gl.lib_wgl.
On any platform, the following steps are taken to link each function during import:
Look in the appropriate library (e.g. libGL.so, libGLU.so, opengl32.dll, etc.) using cdll or windll.
If not found, call wglGetProcAddress or glxGetProcAddress to try to resolve the function’s address dynamically. On OS X, skip this step.
On Windows, this will fail if the context hasn’t been created yet. Create and return a proxy object WGLFunctionProxy which will try the same resolution again when the object is __call__‘d.
The proxy object caches its result so that subsequent calls have only a single extra function-call overhead.
If the function is still not found (either during import or proxy call), the function is replaced with MissingFunction (defined in pyglet.gl.lib), which raises an exception. The exception message details the name of the function, and optionally the name of the extension it requires and any alternative functions that can be used.
The extension required is currently guessed by gengl.py based on nearby #ifndef declarations, it is occasionally wrong.
The suggestion list is not currently used, but is intended to be implemented such that calling, for example, glCreateShader on an older driver suggests glCreateShaderObjectARB, etc.
To access the linking function, import pyglet.gl.lib and use one of link_AGL, link_GLX, link_WGL, link_GL or link_GLU. This is what the generated modules do.
Missing extensions¶
The latest glext.h on opengl.org and nvidia does not include some recent extensions listed on the registry. These must be hand coded into pyglet.gl.glext_missing. They should be removed when glext.h is updated.


{% load static %}
Note: This is an incomplete JSON field that excludes key fields and should not be used for publication


This weekend I started experimenting with a filebrowser application. It's much more Ajax-y than my previous experiment with Ajax. Unfortunately I don't trust that it is secure yet, so I can't give you a demo, but I can give a screenshot.
I used the Prototype library from Rails. I found it quite pleasant. Despite its association with Rails, it has no noticeable server-side integration (which I strongly prefer). It introduces some new patterns to Javascript, but Javascript can use some new patterns. For example, you can use $(id_name) equivalent to document.getElementById(id_name).  If there were a dozen such shortcuts, that'd be a problem, but there's not and it's not a problem.
It could really use some documentation. Good old hand-written documentation, not fancy generated API documentation. I can understand what the code does well enough, but it's hard for me to tell what the code is intended to do, or how I'm intended to use it. But it's short, so it's not that hard to work my way through it.
As an example, here's how you call a URL, and insert whatever it returns into a container by the given id:

I'm not entirely sure I like this style; I find I often want the page to run Javascript in addition to updating some content. You can do this:

But I don't like putting so much logic in the link itself. But experimentation will tell better than my inexperience opinion (especially with respect to Javascript). The underlying functions are easy enough to use that I could implement other techniques if I want. The whole library is just 768 lines long, 567 if you leave out the visual effects, 283 if you leave out the form stuff and Insertions (both of which are probably interesting, but I don't yet understand).
I agree that it's unfortunate that prototype doesn't give you an eval- oriented updater out of the box, but it shouldn't be hard to add one.
The main point of "the form stuff" is the serialize method, which can be used to ease submitting a form asynchronously. Insertions are to insert or append the result to your container, instead of replacing it.
# Alexei
There is an excellent ajax file explorer here: http://www.fileexplorer.net
but! while the application is free, the components are not.
IE only and ASP based -- not very interesting to me. Obviously I'm not doing anything wildly novel with this; I think it's actually mostly interesting (at least to me) in that it's pluggable. Well, not really, but it has the potential at least.
There's more that you could probably knock out of Prototype if you wanted. I'm not sure why it defines Array.push and Function.apply if they don't exist; they both exist in IE 6. I don't have access to any older versions of IE, though, so maybe they are there for IE5.something compatibility?
Hum... Noia icons set and ajax : it seems Python world share some ideas : http://www.nimail.org/screenshots/screenshot3.png (</ads>).


About Author
Supakorn Laohasongkram's dream is to open a university where the goal of teaching is to inspire the students through real-practical work which purpose is aim at actually bettering their lives the lives of all people around them.- Learn more here -
-
{% if page.previous.url %}
- ← Previous Post {% endif %} {% if page.next.url %}
- Next Post → {% endif %}


|
Max API
8.2.0
|
The Max t_linklist data structure is useful for maintaining ordered lists of items where you want to be able to insert and delete items efficiently. More...
Data Structures | |
| struct | t_llelem |
| A linklist element. More... | |
| struct | t_linklist |
| The linklist object. More... | |
Functions | |
| BEGIN_USING_C_LINKAGE t_linklist * | linklist_new (void) |
| Create a new linklist object. More... | |
| void | linklist_chuck (t_linklist *x) |
| Free a linklist, but don't free the items it contains. More... | |
| t_atom_long | linklist_getsize (t_linklist *x) |
| Return the number of items in a linklist object. More... | |
| void * | linklist_getindex (t_linklist *x, long index) |
| Return the item stored in a linklist at a specified index. More... | |
| t_atom_long | linklist_objptr2index (t_linklist *x, void *p) |
| Return an item's index, given the item itself. More... | |
| t_atom_long | linklist_append (t_linklist *x, void *o) |
| Add an item to the end of the list. More... | |
| t_atom_long | linklist_insertindex (t_linklist *x, void *o, long index) |
| Insert an item into the list at the specified index. More... | |
| long | linklist_insert_sorted (t_linklist *x, void *o, long cmpfn(void *, void *)) |
| Insert an item into the list, keeping the list sorted according to a specified comparison function. More... | |
| t_llelem * | linklist_insertafterobjptr (t_linklist *x, void *o, void *objptr) |
| Insert an item into the list after another specified item. More... | |
| t_llelem * | linklist_insertbeforeobjptr (t_linklist *x, void *o, void *objptr) |
| Insert an item into the list before another specified item. More... | |
| t_llelem * | linklist_moveafterobjptr (t_linklist *x, void *o, void *objptr) |
| Move an existing item in the list to a position after another specified item in the list. More... | |
| t_llelem * | linklist_movebeforeobjptr (t_linklist *x, void *o, void *objptr) |
| Move an existing item in the list to a position before another specified item in the list. More... | |
| t_atom_long | linklist_deleteindex (t_linklist *x, long index) |
| Remove the item from the list at the specified index and free it. More... | |
| long | linklist_chuckindex (t_linklist *x, long index) |
| Remove the item from the list at the specified index. More... | |
| long | linklist_chuckobject (t_linklist *x, void *o) |
| Remove the specified item from the list. More... | |
| long | linklist_deleteobject (t_linklist *x, void *o) |
| Delete the specified item from the list. More... | |
| void | linklist_clear (t_linklist *x) |
| Remove and free all items in the list. More... | |
| t_atom_long | linklist_makearray (t_linklist *x, void **a, long max) |
| Retrieve linklist items as an array of pointers. More... | |
| void | linklist_reverse (t_linklist *x) |
| Reverse the order of items in the linked-list. More... | |
| void | linklist_rotate (t_linklist *x, long i) |
| Rotate items in the linked list in circular fashion. More... | |
| void | linklist_shuffle (t_linklist *x) |
| Randomize the order of items in the linked-list. More... | |
| void | linklist_swap (t_linklist *x, long a, long b) |
| Swap the position of two items in the linked-list, specified by index. More... | |
| t_atom_long | linklist_findfirst (t_linklist *x, void **o, long cmpfn(void *, void *), void *cmpdata) |
| Search the linked list for the first item meeting defined criteria. More... | |
| void | linklist_findall (t_linklist *x, t_linklist **out, long cmpfn(void *, void *), void *cmpdata) |
| Search the linked list for all items meeting defined criteria. More... | |
| void | linklist_methodall (t_linklist *x, t_symbol *s,...) |
| Call the named message on every object in the linklist. More... | |
| void * | linklist_methodindex (t_linklist *x, t_atom_long i, t_symbol *s,...) |
| Call the named message on an object specified by index. More... | |
| void | linklist_sort (t_linklist *x, long cmpfn(void *, void *)) |
| Sort the linked list. More... | |
| void | linklist_funall (t_linklist *x, method fun, void *arg) |
| Call the specified function for every item in the linklist. More... | |
| t_atom_long | linklist_funall_break (t_linklist *x, method fun, void *arg) |
| Call the specified function for every item in the linklist. More... | |
| void * | linklist_funindex (t_linklist *x, long i, method fun, void *arg) |
| Call the specified function for an item specified by index. More... | |
| void * | linklist_substitute (t_linklist *x, void *p, void *newp) |
| Given an item in the list, replace it with a different value. More... | |
| void * | linklist_next (t_linklist *x, void *p, void **next) |
| Given an item in the list, find the next item. More... | |
| void * | linklist_prev (t_linklist *x, void *p, void **prev) |
| Given an item in the list, find the previous item. More... | |
| void * | linklist_last (t_linklist *x, void **item) |
| Return the last item (the tail) in the linked-list. More... | |
| void | linklist_readonly (t_linklist *x, long readonly) |
| Set the linklist's readonly bit. More... | |
| void | linklist_flags (t_linklist *x, long flags) |
| Set the linklist's datastore flags. More... | |
| t_atom_long | linklist_getflags (t_linklist *x) |
| Get the linklist's datastore flags. More... | |
| long | linklist_match (void *a, void *b) |
| A linklist comparison method that determines if two item pointers are equal. More... | |
The Max t_linklist data structure is useful for maintaining ordered lists of items where you want to be able to insert and delete items efficiently.
Random access of individual items, however, gets appreciably slower as the list grows in size. The t_linklist is thread-safe by default, but thread safety can be turned off for performance benefits in single-threaded applications. However, ensure that your use of the linked list is truly single-threaded (based on an understanding of Max's Threading model) before turning off the thread safety features.
By default, the t_linklist holds pointers to Max objects. However, you can treat what the linklist holds as data rather than objects to be freed by using the linklist_flags() function.
Here is a simple example of the use of t_linklist. The code below stores five symbols, sorts them, searches for a specific item, deletes an item, prints all items, and then frees the entire structure. Since symbols in Max are never freed, linklist_flags() is used to specify that data, rather than object pointers, are being stored.
The sorting function compares two items in the list and returns non-zero if the first one should go before the second one.
The search function is passed the final argument to linklist_findfirst() and, in this case, just returns the symbol that matches, which is just testing for pointer equivalence since all Max symbols are unique. You could do more sophisticated searching if you store more complex data in a linklist.
The iteration function takes some action on all items in the list. The third argument to linklist_funall() is passed as the second argument to your iteration function. In this example, we don't do anything with it.
| t_atom_long linklist_append | ( | t_linklist * | x, |
| void * | o | ||
| ) |
Add an item to the end of the list.
| x | The linklist instance. |
| o | The item pointer to append to the linked-list. |
Referenced by jit_linklist_append().
| void linklist_chuck | ( | t_linklist * | x | ) |
Free a linklist, but don't free the items it contains.
The linklist can contain a variety of different types of data. By default, the linklist assumes that all items are max objects with a valid t_object header.
You can alter the linklist's notion of what it contains by using the linklist_flags() method.
When you free the linklist by calling object_free() it then tries to free all of the items it contains. If the linklist is storing a custom type of data, or should otherwise not free the data it contains, then call linklist_chuck() to free the object instead of object_free().
| x | The linklist object to be freed. |
Referenced by jit_linklist_chuck(), and jit_linklist_findcount().
| long linklist_chuckindex | ( | t_linklist * | x, |
| long | index | ||
| ) |
Remove the item from the list at the specified index.
You are responsible for freeing any memory associated with the item that is removed from the linklist.
| x | The linklist instance. |
| index | The index of the item to remove. |
Referenced by jit_linklist_chuckindex().
| long linklist_chuckobject | ( | t_linklist * | x, |
| void * | o | ||
| ) |
Remove the specified item from the list.
You are responsible for freeing any memory associated with the item that is removed from the linklist.
| x | The linklist instance. |
| o | The pointer to the item to remove. |
| void linklist_clear | ( | t_linklist * | x | ) |
Remove and free all items in the list.
Freeing items in the list is subject to the same rules as linklist_deleteindex(). You can alter the linklist's notion of what it contains, and thus how items are freed, by using the linklist_flags() method.
| x | The linklist instance. |
Referenced by jit_linklist_clear(), and jit_linklist_free().
| t_atom_long linklist_deleteindex | ( | t_linklist * | x, |
| long | index | ||
| ) |
Remove the item from the list at the specified index and free it.
The linklist can contain a variety of different types of data. By default, the linklist assumes that all items are max objects with a valid t_object header. Thus by default, it frees items by calling object_free() on them.
You can alter the linklist's notion of what it contains by using the linklist_flags() method.
If you wish to remove an item from the linklist and free it yourself, then you should use linklist_chuckptr().
| x | The linklist instance. |
| index | The index of the item to delete. |
Referenced by jit_linklist_deleteindex().
| long linklist_deleteobject | ( | t_linklist * | x, |
| void * | o | ||
| ) |
Delete the specified item from the list.
The object is removed from the list and deleted. The deletion is done with respect to any flags passed to linklist_flags.
| x | The linklist instance. |
| o | The pointer to the item to delete. |
| void linklist_findall | ( | t_linklist * | x, |
| t_linklist ** | out, | ||
| long | cmpfnvoid *, void *, | ||
| void * | cmpdata | ||
| ) |
Search the linked list for all items meeting defined criteria.
The items in the list are traversed, calling a specified comparison function on each, and returning the matches in another linklist.
| x | The linklist instance. |
| out | The address to a t_linklist pointer. You should initialize the pointer to NULL before calling linklist_findall(). A new linklist will be created internally by linklist_findall() and returned here. |
| cmpfn | The function used to determine a match in the list. |
| cmpdata | An argument to be passed to the t_cmpfn. This will be passed as the second of the two args to the t_cmpfn. The first arg will be the linklist item at each iteration in the list. |
Referenced by jit_linklist_findall(), and jit_linklist_findcount().
| t_atom_long linklist_findfirst | ( | t_linklist * | x, |
| void ** | o, | ||
| long | cmpfnvoid *, void *, | ||
| void * | cmpdata | ||
| ) |
Search the linked list for the first item meeting defined criteria.
The items in the list are traversed, calling a specified comparison function on each until the comparison function returns true.
| x | The linklist instance. |
| o | The address to pointer that will be set with the matching item. |
| cmpfn | The function used to determine a match in the list. |
| cmpdata | An argument to be passed to the t_cmpfn. This will be passed as the second of the two args to the t_cmpfn. The first arg will be the linklist item at each iteration in the list. |
Referenced by jit_linklist_findfirst().
| void linklist_flags | ( | t_linklist * | x, |
| long | flags | ||
| ) |
Set the linklist's datastore flags.
The available flags are enumerated in e_max_datastore_flags. These flags control the behavior of the linklist, particularly when removing items from the list using functions such as linklist_clear(), linklist_deleteindex(), or when freeing the linklist itself.
| x | The linklist instance. |
| flags | A valid value from the e_max_datastore_flags. The default is OBJ_FLAG_OBJ. |
| void linklist_funall | ( | t_linklist * | x, |
| method | fun, | ||
| void * | arg | ||
| ) |
Call the specified function for every item in the linklist.
| x | The linklist instance. |
| fun | The function to call, specified as function pointer cast to a Max method. |
| arg | An argument that you would like to pass to the function being called. |
| t_atom_long linklist_funall_break | ( | t_linklist * | x, |
| method | fun, | ||
| void * | arg | ||
| ) |
Call the specified function for every item in the linklist.
The iteration through the list will halt if the function returns a non-zero value.
| x | The linklist instance. |
| fun | The function to call, specified as function pointer cast to a Max method. |
| arg | An argument that you would like to pass to the function being called. |
| void* linklist_funindex | ( | t_linklist * | x, |
| long | i, | ||
| method | fun, | ||
| void * | arg | ||
| ) |
Call the specified function for an item specified by index.
| x | The linklist instance. |
| i | The index of the item to which to send the message. |
| fun | The function to call, specified as function pointer cast to a Max method. |
| arg | An argument that you would like to pass to the function being called. |
| t_atom_long linklist_getflags | ( | t_linklist * | x | ) |
Get the linklist's datastore flags.
| x | The linklist instance. |
| void* linklist_getindex | ( | t_linklist * | x, |
| long | index | ||
| ) |
Return the item stored in a linklist at a specified index.
| x | The linklist instance. |
| index | The index in the linklist to fetch. Indices are zero-based. |
NULL
is returned Referenced by jit_linklist_getindex().
| t_atom_long linklist_getsize | ( | t_linklist * | x | ) |
Return the number of items in a linklist object.
| x | The linklist instance. |
Referenced by jit_linklist_findcount(), and jit_linklist_getsize().
| long linklist_insert_sorted | ( | t_linklist * | x, |
| void * | o, | ||
| long | cmpfnvoid *, void * | ||
| ) |
Insert an item into the list, keeping the list sorted according to a specified comparison function.
| x | The linklist instance. |
| o | The item pointer to insert. |
| cmpfn | A comparison function by which the list should be sorted. |
| t_llelem* linklist_insertafterobjptr | ( | t_linklist * | x, |
| void * | o, | ||
| void * | objptr | ||
| ) |
Insert an item into the list after another specified item.
| x | The linklist instance. |
| o | The item pointer to insert. |
| objptr | The item pointer after which to insert in the list. |
| t_llelem* linklist_insertbeforeobjptr | ( | t_linklist * | x, |
| void * | o, | ||
| void * | objptr | ||
| ) |
Insert an item into the list before another specified item.
| x | The linklist instance. |
| o | The item pointer to insert. |
| objptr | The item pointer before which to insert in the list. |
| t_atom_long linklist_insertindex | ( | t_linklist * | x, |
| void * | o, | ||
| long | index | ||
| ) |
Insert an item into the list at the specified index.
| x | The linklist instance. |
| o | The item pointer to insert. |
| index | The index at which to insert. Index 0 is the head of the list. |
Referenced by jit_linklist_insertindex().
| void* linklist_last | ( | t_linklist * | x, |
| void ** | item | ||
| ) |
Return the last item (the tail) in the linked-list.
| x | The linklist instance. |
| item | The address of pointer in which to store the last item in the linked-list. |
| t_atom_long linklist_makearray | ( | t_linklist * | x, |
| void ** | a, | ||
| long | max | ||
| ) |
Retrieve linklist items as an array of pointers.
| x | The linklist instance. |
| a | The address of the first pointer in the array to fill. |
| max | The number of pointers in the array. |
Referenced by jit_linklist_makearray().
| long linklist_match | ( | void * | a, |
| void * | b | ||
| ) |
A linklist comparison method that determines if two item pointers are equal.
| a | The first item to compare. |
| b | The second item to compare. |
| void linklist_methodall | ( | t_linklist * | x, |
| t_symbol * | s, | ||
| ... | |||
| ) |
Call the named message on every object in the linklist.
The linklist_methodall() function requires that all items in the linklist are object instances with a valid t_object header.
| x | The linklist instance. |
| s | The name of the message to send to the objects. |
| ... | Any arguments to be sent with the message. |
Referenced by jit_linklist_methodall().
| void* linklist_methodindex | ( | t_linklist * | x, |
| t_atom_long | i, | ||
| t_symbol * | s, | ||
| ... | |||
| ) |
Call the named message on an object specified by index.
The item must be an object instance with a valid t_object header.
| x | The linklist instance. |
| i | The index of the item to which to send the message. |
| s | The name of the message to send to the objects. |
| ... | Any arguments to be sent with the message. |
Referenced by jit_linklist_methodindex().
| t_llelem* linklist_moveafterobjptr | ( | t_linklist * | x, |
| void * | o, | ||
| void * | objptr | ||
| ) |
Move an existing item in the list to a position after another specified item in the list.
| x | The linklist instance. |
| o | The item pointer to insert. |
| objptr | The item pointer after which to move o in the list. |
| t_llelem* linklist_movebeforeobjptr | ( | t_linklist * | x, |
| void * | o, | ||
| void * | objptr | ||
| ) |
Move an existing item in the list to a position before another specified item in the list.
| x | The linklist instance. |
| o | The item pointer to insert. |
| objptr | The item pointer before which to move o in the list. |
| BEGIN_USING_C_LINKAGE t_linklist* linklist_new | ( | void | ) |
Create a new linklist object.
You can free the linklist by calling object_free() on the linklist's pointer, or by using linklist_chuck().
Referenced by jit_linklist_new().
| void* linklist_next | ( | t_linklist * | x, |
| void * | p, | ||
| void ** | next | ||
| ) |
Given an item in the list, find the next item.
This provides an means for walking the list.
| x | The linklist instance. |
| p | An item in the list. |
| next | The address of a pointer to set with the next item in the list. |
| t_atom_long linklist_objptr2index | ( | t_linklist * | x, |
| void * | p | ||
| ) |
Return an item's index, given the item itself.
| x | The linklist instance. |
| p | The item pointer to search for in the linklist. |
Referenced by jit_linklist_objptr2index().
| void* linklist_prev | ( | t_linklist * | x, |
| void * | p, | ||
| void ** | prev | ||
| ) |
Given an item in the list, find the previous item.
This provides an means for walking the list.
| x | The linklist instance. |
| p | An item in the list. |
| prev | The address of a pointer to set with the previous item in the list. |
| void linklist_readonly | ( | t_linklist * | x, |
| long | readonly | ||
| ) |
Set the linklist's readonly bit.
By default the readonly bit is 0, indicating that it is threadsafe for both reading and writing. Setting the readonly bit to 1 will disable the linklist's theadsafety mechanism, increasing performance but at the expense of threadsafe operation. Unless you can guarantee the threading context for a linklist's use, you should leave this set to 0.
| x | The linklist instance. |
| readonly | A 1 or 0 for setting the readonly bit. |
| void linklist_reverse | ( | t_linklist * | x | ) |
Reverse the order of items in the linked-list.
| x | The linklist instance. |
Referenced by jit_linklist_reverse().
| void linklist_rotate | ( | t_linklist * | x, |
| long | i | ||
| ) |
Rotate items in the linked list in circular fashion.
| x | The linklist instance. |
| i | The number of positions in the list to shift items. |
Referenced by jit_linklist_rotate().
| void linklist_shuffle | ( | t_linklist * | x | ) |
Randomize the order of items in the linked-list.
| x | The linklist instance. |
Referenced by jit_linklist_shuffle().
| void linklist_sort | ( | t_linklist * | x, |
| long | cmpfnvoid *, void * | ||
| ) |
Sort the linked list.
The items in the list are ordered using a t_cmpfn function that is passed in as an argument.
| x | The linklist instance. |
| cmpfn | The function used to sort the list. |
Referenced by jit_linklist_sort().
| void* linklist_substitute | ( | t_linklist * | x, |
| void * | p, | ||
| void * | newp | ||
| ) |
Given an item in the list, replace it with a different value.
| x | The linklist instance. |
| p | An item in the list. |
| newp | The new value. |
| void linklist_swap | ( | t_linklist * | x, |
| long | a, | ||
| long | b | ||
| ) |
Swap the position of two items in the linked-list, specified by index.
| x | The linklist instance. |
| a | The index of the first item to swap. |
| b | The index of the second item to swap. |
Referenced by jit_linklist_swap().


This module allows you to use Google Guice dependency injection in your Cucumber tests. Guice comes as standard with
singleton scope and 'no scope'. This module adds Cucumber scenario scope to the scopes available for use in your
test code. The rest of this documentation assumes you have at least a basic understanding of Guice. Please refer to
the Google documentation if necessary, see https://code.google.com/p/google-guice/wiki/Motivation?tm=6
About scopes, injectors and migration from earlier versions
It's important to realise the differences in how this module functions when compared with earlier versions. The
changes are as follows.
<= 1.1.7 A Guice injector is created at the start of each test scenario and is destroyed at the end of each test
scenario. There is no scenario scope, just singleton and 'no scope'.
> 1.1.7 A Guice injector is created once before any tests are run and is destroyed after the last test has run.
Before each test scenario a new scenario scope is created. At the end of the test scenario the scenario
scope is destroyed. Singleton scope exists throughout all test scenarios.
Users wishing to migrate to this version should replace @Singleton
annotations with
@ScenarioScope
annotations. Guice modules should also have their singleton bindings updated. All
bindings in Scopes.SINGLETON
should be replaced with bindings in CucumberScopes.SCENARIO
.
Using the module
By including the cucumber-guice
jar on your CLASSPATH
your Step Definitions will be
instantiated by Guice. There are two main modes of using the module: with scope annotations and with module
bindings. The two modes can also be mixed. When mixing modes it is important to realise that binding a class in a
scope in a module takes precedence if the same class is also bound using a scope annotation.
Scoping your step definitions
Usually you will want to bind your step definition classes in either scenario scope or in singleton scope. It is not
recommended to leave your step definition classes with no scope as it means that Cucumber will instantiate a new
instance of the class for each step within a scenario that uses that step definition.
Scenario scope
Cucumber will create exactly one instance of a class bound in scenario scope for each scenario in which it is used.
You should use scenario scope when you want to store state during a scenario but do not want the state to interfere
with subsequent scenarios.
Singleton scope
Cucumber will create just one instance of a class bound in singleton scope that will last for the lifetime of all
test scenarios in the test run. You should use singleton scope if your classes are stateless. You can also use
singleton scope when your classes contain state but with caution. You should be absolutely sure that a state change
in one scenario could not possibly influence the success or failure of a subsequent scenario. As an example of when
you might use a singleton, imagine you have an http client that is expensive to create. By holding a reference to
the client in a class bound in singleton scope you can reuse the client in multiple scenarios.
Using scope annotations
This is the easy route if you're new to Guice. To bind a class in scenario scope add the
cucumber.runtime.java.guice.ScenarioScoped
annotation to the class definition. The class should have
a no-args constructor or one constructor that is annotated with javax.inject.Inject
. For example:
import cucumber.runtime.java.guice.ScenarioScoped;
import javax.inject.Inject;
@ScenarioScoped
public class ScenarioScopedSteps {
private final Object someInjectedDependency;
@Inject
public ScenarioScopedSteps(Object someInjectedDependency) {
this.someInjectedDependency = someInjectedDependency;
}
...
}
To bind a class in singleton scope add the javax.inject.Singleton
annotation to the class definition.
One strategy for using stateless step definitions is to use providers to share stateful scenario scoped instances
between stateless singleton step definition instances. For example:
import javax.inject.Inject;
import javax.inject.Singleton;
@Singleton
public class MyStatelessSteps {
private final Provider<MyStatefulObject> providerMyStatefulObject;
@Inject
public MyStatelessSteps(Provider<MyStatefulObject> providerMyStatefulObject) {
this.providerMyStatefulObject = providerMyStatefulObject;
}
@Given("^I have (\\d+) cukes in my belly$")
public void I_have_cukes_in_my_belly(int n) {
providerMyStatefulObject.get().iHaveCukesInMyBelly(n);
}
...
}
See https://code.google.com/p/google-guice/wiki/InjectingProviders "Providers for Mixing Scopes"
Using module bindings
As an alternative to using annotations you may prefer to declare Guice bindings in a class that implements
com.google.inject.Module
. To do this you should create a class that implements
cucumber.runtime.java.guice.InjectorSource
.  The injector must provide a binding for
cucumber.runtime.java.guice.ScenarioScope
. It should also provide a binding for the
cucumber.runtime.java.guice.ScenarioScoped
annotation if your classes are using the annotation. The
easiest way to do this it to use CucumberModules.SCENARIO
. For example:
import com.google.inject.Guice;
import com.google.inject.Injector;
import com.google.inject.Stage;
import cucumber.api.guice.CucumberModules;
import cucumber.runtime.java.guice.InjectorSource;
public class YourInjectorSource implements InjectorSource {
@Override
public Injector getInjector() {
return Guice.createInjector(Stage.PRODUCTION, CucumberModules.SCENARIO, new YourModule());
}
}
Cucumber needs to know where to find the cucumber.runtime.java.guice.InjectorSource
that it will use.
You should create a properties file called cucumber-guice.properties
and place it in the root of the
classpath. The file should contain a single property key called guice.injector-source
with a value
equal to the fully qualified name of the cucumber.runtime.java.guice.InjectorSource
. For example:
guice.injector-source=com.company.YourInjectorSource


Module: Doing::Prompt
- Extended by:
- Color, PromptChoose, PromptFZF, PromptInput, PromptSTD, PromptYN
- Defined in:
- lib/doing/prompt/prompt.rb
Overview
Terminal Prompt methods
Constant Summary
Constants included from Color
Color::ATTRIBUTES, Color::ATTRIBUTE_NAMES, Color::ESCAPE_REGEX
Class Attribute Summary collapse
-
.default_answer ⇒ Boolean
If true, always return the default answer without prompting.
-
.force_answer ⇒ Object
Value to return if prompt is skipped.
Method Summary
Methods included from PromptChoose
choose_from, choose_from_items
Methods included from PromptFZF
fzf, install_fzf, uninstall_fzf, which_fzf
Methods included from PromptYN
Methods included from PromptInput
enter_text, read_line, read_lines, request_lines
Methods included from PromptSTD
clear_screen, restore_std, silence_std
Methods included from Color
attributes, coloring?, rgb, support?, template, uncolor
Class Attribute Details
.default_answer ⇒ Boolean
If true, always return the default answer without prompting
.force_answer ⇒ Object
Value to return if prompt is skipped


I am a competent software developer with strong commitment to collaboration and solutions- oriented problem-solving. I am skilled in native desktop, mobile, and web application development. I am able to work well under pressure and consistently meet deadlines and targets while delivering high quality work.
B+: 71pts
I am an audiophile who likes to both listen and make music. I contribute to Open Source projects which help further this love for music. I also enjoy drawing using traditional media and also creating art using Inkscape and Blender.
To unwind, I like to watch a wide genre of movies and television shows. I also like to play video games. That said, I also like to make video games. This is a long term dream which I am helping realize by learning a variety of game engines like CryEngine, Unreal Engine and Godot Game Engine.


About Me: My Top 10 Series
1: The Office (US)
A mockumentary on a group of typical office workers, where the workday consists of ego clashes, inappropriate behavior, and tedium.
2: Breaking Bad
A high school chemistry teacher diagnosed with inoperable lung cancer turns to manufacturing and selling methamphetamine in order to secure his family's future.
3: One Piece
Follows the adventures of Monkey D. Luffy and his pirate crew in order to find the greatest treasure ever left by the legendary Pirate, Gold Roger. The famous mystery treasure named "One Piece".
4: Rick and Morty
An animated series that follows the exploits of a super scientist and his not-so-bright grandson.
5: Chyrnobyl
In April 1986, an explosion at the Chernobyl nuclear power plant in the Union of Soviet Socialist Republics becomes one of the world's worst man-made catastrophes.
6: Brooklyn 99
"Brooklyn Nine-Nine" follows the exploits of hilarious Det. Jake Peralta and his diverse, lovable colleagues as they police the NYPD's 99th Precinct.
7: The Boys
A group of vigilantes sets out to take down corrupt superheroes who abuse their superpowers.
8: Friends
Follows the personal and professional lives of six twenty to thirty-something-year-old friends living in Manhattan.
9: Prison Break
Due to a political conspiracy, an innocent man is sent to death row and his only hope is his brother, who makes it his mission to deliberately get himself sent to the same prison in order to break the both of them out, from the inside.
10: Game of Thrones
Nine noble families fight for control over the lands of Westeros, while an ancient enemy returns after being dormant for millennia.


CBSD control panel
Please visit
www.bsdstore.ru
for support.
CBSD
Dashboard
Jails
Bhyve VM
Resources
Nodes
Repository
Bases
System
Host Userlist...
Host Add User...
Settings...
Task Log
Your browser does not support iframes


Hue Groups
Hue Groups
Since you have some hue lights registered let's create some Groups.
Click on the button at the top bar.
If this is the first time that you run the app remember that the Hue bridge and this App have to be in the same LAN (Same network). If they are try the following:
Let's register this app with your Hue Bridge by pressing the bridge's button. Then you will have 30 seconds to click on the test button of this screen.


class: center, middle, inverse, title-slide # Dates and times with Lubridate ### Esteban Gracia & Lorenzo Gini ### Introduction to Data Science --- #Questions about time - How many days has a year? - How many hours has a day? - How many seconds has a minute? --

Source: knowyourmeme.com

 --- class: inverse, center, middle # Lubridate --- # Dates and times with Lubridate Lubridate is installed with the tidiverse: ```r install.packages("tidyverse") library(lubridate) ``` --- #Working with dates - Convert date-strings to a date-time object - Find and use the day, month or year from a Date-time object - Convert timezones --- class: inverse, center, middle # Main functions --- #DateTime objects - We have dates - POSIXct and POSIXlt classes -- (What?) -- ```r like_date <- as.Date("2021-11-04 10:15:20") str(like_date) ``` ``` ## Date[1:1], format: "2021-11-04" ``` -- ```r like_datetime_1 <- as.POSIXct("2021-11-04 10:15:20") str(like_datetime_1) ``` ``` ## POSIXct[1:1], format: "2021-11-04 10:15:20" ``` ```r like_datetime_2 <- as.POSIXlt("2021-11-04 10:15:20") str(like_datetime_2) ``` ``` ## POSIXlt[1:1], format: "2021-11-04 10:15:20" ``` --- #Parsing date-times - How often do we have the correct order in dates in our dataset? - January 31, 2017 - 31 January, 2017 - 2017, January 31 -- ```r "2017-01-31" ``` ``` ## [1] "2017-01-31" ``` -- ```r ymd("2017-01-31") # 1. year 2. month 3. day ``` ``` ## [1] "2017-01-31" ``` -- ```r mdy("January 31st, 2017") # 1. month 2. day 3. year ``` ``` ## [1] "2017-01-31" ``` --- #Parsing date-times ```r x <- c("2021-10-01", "02022010", "02-02-2010") parse_date_time(x, c("dmy", "ymd")) ``` ``` ## [1] "2021-10-01 UTC" "2010-02-02 UTC" "2010-02-02 UTC" ``` --- ###From individual components -- You can create a variable with datetime objects ``` ## names day month year hour minute ## 1 Lucas 21 03 1990 22 21 ## 2 Peter 10 05 1991 20 01 ## 3 Santiago 30 10 1992 19 15 ``` --- ###From individual components ```r birthdays_people_dates <- birthdays_people %>% mutate(date_time_birthday = make_datetime(year, month, day, hour, minute)) birthdays_people_dates ``` ``` ## names day month year hour minute date_time_birthday ## 1 Lucas 21 03 1990 22 21 1990-03-21 22:21:00 ## 2 Peter 10 05 1991 20 01 1991-05-10 20:01:00 ## 3 Santiago 30 10 1992 19 15 1992-10-30 19:15:00 ``` --- # Date time components ```r datetime <- ymd_hms("2016-07-08 12:34:56") ``` -- ##Getting -- ####Taking a single value of the date ```r year(datetime) #take the year ``` ``` ## [1] 2016 ``` -- ```r month(datetime) #take the month ``` ``` ## [1] 7 ``` -- ```r mday(datetime) #take the day ``` ``` ## [1] 8 ``` --- # Date time components ```r datetime <- ymd_hms("2016-07-08 12:34:56") ``` -- ##Setting -- ####Changing a single value of the date ```r day(datetime) <- 10 #changing 8 to 10 datetime ``` ``` ## [1] "2016-07-10 12:34:56 UTC" ``` -- ```r month(datetime) <- 12 #changing 7 to 12 datetime ``` ``` ## [1] "2016-12-10 12:34:56 UTC" ``` --- #Abrreviated name of month or week ```r datetime <- ymd_hms("2016-07-08 12:34:56") month(datetime, label = TRUE) #Taking the month name [1] Jul 12 Levels: Jan < Feb < Mar < Apr < May < Jun < Jul < Aug < Sep < ... < Dec ``` -- ```r wday(datetime, label = TRUE, abbr = FALSE) #Taking day name [1] Friday 7 Levels: Sunday < Monday < Tuesday < Wednesday < Thursday < ... < Saturday ``` --- #Round, floor and ceiling for dates ```r new_date <- ymd_hms("2009-08-03 12:01:59.23") ``` -- See how different is the value from each function ```r round_date(new_date, "day") #after mid-day ``` ``` ## [1] "2009-08-04 UTC" ``` -- ```r floor_date(new_date, "day") #before mid-day ``` ``` ## [1] "2009-08-03 UTC" ``` -- ```r ceiling_date(new_date, "day") #after mid-day ``` ``` ## [1] "2009-08-04 UTC" ``` --- #Round, floor and ceiling for dates See how it changes depending the function ```r new_date_2 <- ymd_hms("2009-08-22 12:01:59.23") ``` -- ```r round_date(new_date_2, "month") #more September than August ``` ``` ## [1] "2009-09-01 UTC" ``` -- ```r floor_date(new_date_2, "month") #Still September ``` ``` ## [1] "2009-08-01 UTC" ``` -- ```r ceiling_date(new_date_2, "month") #more September than August ``` ``` ## [1] "2009-09-01 UTC" ``` --- # Stamping a template to dates ```yaml date_by_Hertie <- stamp("Created by Hertie, Jan 17, 1999 3:34") date_by_Esteban <- stamp("Created by Esteban, Jan 21, 2021 3:34") date_by_Lorenzo <- stamp("Created by Lorenzo, Jan 21, 2021 3:34") ``` -- ```r date_by_Hertie(ymd("2010-04-05")) ``` ``` ## [1] "Created by Hertie, Apr 05, 2010 00:00" ``` -- ```r date_by_Esteban(ymd("2021-11-04")) ``` ``` ## [1] "Created by Esteban, Nov 04, 2021 00:00" ``` -- ```r date_by_Lorenzo(ymd("2021-11-05")) ``` ``` ## [1] "Created by Lorenzo, Nov 05, 2021 00:00" ``` --- #Timezones Lubridate uses the IANA Timezone Database named Olson Database because of his founder Arthur Olson. The nomenclature makes easier to work with timezones. ```r OlsonNames() %>% #List of Timezones sample(10) #Taking a sample of 30 tz ``` ``` ## [1] "Indian/Maldives" "Brazil/West" ## [3] "America/Argentina/Mendoza" "Asia/Hong_Kong" ## [5] "America/St_Thomas" "Atlantic/Canary" ## [7] "Asia/Istanbul" "Canada/Saskatchewan" ## [9] "America/Grand_Turk" "Pacific/Niue" ``` --- #Timezones ####Get the same data-time in different time zone Getting the current hour in my system: Berlin, Germany ```r x_now <- now(tzone = "Europe/Berlin") x_now ``` ``` ## [1] "2021-11-01 13:55:32 CET" ``` ```r x_sys <- now(tzone = Sys.timezone()) #Current hour in my pc x_sys ``` ``` ## [1] "2021-11-01 13:55:32 CET" ``` -- Getting the same hour in Bogotá, Colombia ```r berlin_to_bogota <- with_tz(x_sys, tzon="America/Bogota") #7 Hours diff berlin_to_bogota ``` ``` ## [1] "2021-11-01 07:55:32 -05" ``` --- class: inverse, center, middle # Time spans --- # Periods, durations and intervals ```r x_date<- ymd_hms("2021-02-27 12:00:00") y_date <- ymd_hms("2021-03-01 01:00:00") ``` ###Intervals Intervals are timespans that begin at a specific instant and end at a specific instant. ```r dates <- interval((x_date),(y_date)) dates #Dates now stores an interval of dates ``` ``` ## [1] 2021-02-27 12:00:00 UTC--2021-03-01 01:00:00 UTC ``` --- ###Periods Periods track the change in the "clock time" between two date-times. ```r period_dates <- as.period(dates) period_dates #Change in clock time of interval ``` ``` ## [1] "1d 13H 0M 0S" ``` ###Durations Durations record the exact number of seconds in a time span. ```r duration_dates <- as.duration(period_dates) duration_dates #Seconds between the interval ``` ``` ## [1] "133200s (~1.54 days)" ``` --- #References and more about Lubridate - Garrett Grolemund, Hadley Wickham (2011). Dates and Times Made Easy with lubridate. Journal of Statistical Software, 40(3), 1-25. https://www.jstatsoft.org/v40/i03/. - Lubridate by Vitalie Spinu, Garrett Grolemund, Hadley Wickham. https://lubridate.tidyverse.org/index.html - Lubridate cheatsheet. https://evoldyn.gitlab.io/evomics-2018/ref-sheets/R_lubridate.pdf --- class: center, middle # Thanks! Slides created via the R package [**xaringan**](https://github.com/yihui/xaringan).


In the first Lab (Lab 1: Ingestion with Glue) we did one transformation to the data; changed the format to parquet. Partitioning and bucketing are other common best practices. Partitioning, is a way to organizes tables into partitions by dividing tables into different parts based on partition keys. While Bucketing, buckets your data into groups within a single partition.
This lab discusses common best practices) that enable you to get the most out of Athena.
Before doing this let’s discuss how Athena pricing works. With Athena, you are charged for the number of bytes scanned by Amazon Athena, rounded up to the nearest megabyte, with a 10MB minimum per query. Thus, the aim is to run the query with least amount of data scanned.
We will run the same query before and after we do the optimisation to see how these recommendations significantly reduce the amount of data scanned, and thus, reducing cost and improving performance.
Apache Parquet is a popular columnar storage format that is optimised for fast retrieval of data and used in AWS analytical applications. Parquet and other popular columnar storage formats have three main benefits that make them suitable for using with Athena;
Now let’s compare the amount of data scanned when we run the same query on the raw and curated tables
SELECT * FROM {raw_table_name}
Wait the query to execute and note the amount of Data scanned by the query.
Let’s run the same query on the optimised data to compare the difference. Choose the {curated database} from the dropdown menu and execute the same query with the new {curated_table_name}.
Wait for the query to execute the note the amount of Data Scanned
You noticed in this lab by converting to Columnar storage format, you significantly reduced the amount of data scanned, and thus, reducing Athena costs and improving performance.
By partitioning your data, you can restrict the amount of data scanned by each query, thus improving performance and reducing cost. Athena leverages Hive for partitioning data. You can partition your data by any key. A common practice is to partition the data based on time, often leading to a multi-level partitioning scheme.
To see the benefit of partitioning, let’s run the same query on the raw (non partitioned) and curated (partitioned). This time the query should filter the results using the WHERE clause in the SQL statement. The aim is use the column used to partition the data to filter the results.
SELECT count(*) AS FilterCount
from {raw_table_name}
where {condition_on_partitioned_column}
Wait the query to execute and note the amount of Data scanned by the query.
Let’s run the same query on the optimised data to compare the difference. Choose the {curated database} from the dropdown menu and execute the same query with the new {table_name_curated}.
Wait for the query to execute the note the amount of Data Scanned
You learnt in this lab that if you have many queries that are filtered in the WHERE clause, then the column used to filter the query result would be a good partitioning candidate. By partitioning your data you will significantly reduce the amount of data scanned, and thus, reducing Athena costs and improving performance.
An additional optimisation supported by Athena is bucketing. Partitioning is used to group similar types of data based on a specific column. Bucketing is commonly used to combine data within a partition into a number of equal groups, or files. Therefore, partitioning is best suited for low cardinality columns and bucketing is best suited for high cardinality columns.
Bucketing your data, is another common technique that improves Athena performance and cost. For more information, see Bucketing vs Partitioning.
You can use CREATE TABLE AS SELECY
(CTAS) query to bucket the data within a partition. This lab, we will cover, how to use CTAS to physically create a subset table. Two identical subset tables will be created; the first one is partitioned and the second one is bucketed and partitioned. Finally, we will run the same query on both subset tables and compare the results.
Note: This lab will run on the Curated Table
[OPTIONAL]
A CREATE TABLE AS SELECT
(CTAS) query creates a new table in Athena from the results of a SELECT
statement from another query. Athena stores data files created by the CTAS statement in a specified location in Amazon S3
Note: You not need to create the S3 folder before running the CTAS query; Athena will do it for you. Just choose the S3 path to store the data. Feel free to use any path as long as you own the S3 bucket and it is in the same region you are using throughout this lab. If you choose to create the folder before running the query, make sure the folder is empty otherwise the query will fail.
Use CTAS queries to:
CREATE TABLE subset
WITH (
format = 'PARQUET',
external_location = 's3://{athena-s3-bucket}/{first_subset}/',
partitioned_by = ARRAY['{col1}','{col2}', etc ...],
)
AS SELECT *
FROM {curated_table_name}
where {CTAS_condition}
CREATE TABLE subset_bucketed
WITH (
format = 'PARQUET',
external_location = 's3://{athena-s3-bucket}/{first_subset}/',
partitioned_by = ARRAY['{col1}','{col2}',etc..],
bucketed_by = ARRAY['{primary_key}'],
bucket_count = 5)
AS SELECT *
FROM {curated_table_name}
WHERE {CTAS_condition}
Note: {primary_key} here should be the column used for bucketing. Columns with high cardinality (high number or unique values) that are spread evenly are good candidates for bucketing.
AS SELECT *
FROM subset
WHERE {primary_key} = {value}
Wait for the query to finish and note the amount of data scanned by the query.
In the Query Editor, execute the following query on the new table (subset_bucketed)
AS SELECT *
FROM subset_bucketed
WHERE {primary_key} = {value}
In this lab we learnt that bucketing is also a very useful optimisation. Columns with high cardinality (high number of unique values) that are evenly spread are good candidates for bucketing. Bucketing distribute the data to different files within a partition. So if we want to filter a partition, Athena will only scan the relevant bucket files not the whole partition.