Experimental Feature
S3 Select is a feature for Amazon S3 introduced in April 2018. It allows for SQL-like SELECT expressions to be applied to files in some structured formats, including CSV and JSON.
By performing the SELECT operation in the S3 storage infrastructure, the bandwidth requirements between S3 and the hosts making the request can be reduced. Along with latency, this bandwidth is often the limiting factor in processing data from S3, especially with larger CSV and JSON datasets.
Apache Hadoop’s S3A Client has experimental support for this feature, with the following warnings:
FileInputFormat
readers of text (LineRecordReader
etc) fail when the amount of data returned is less than they expect. For this reason, S3 Select MUST NOT BE USED IN PRODUCTION MAPREDUCE JOBS.S3 Select is enabled by default:
<property> <name>fs.s3a.select.enabled</name> <value>true</value> <description>Is S3 Select enabled?</description> </property>
To disable it, set the option fs.s3a.select.enabled
to false
.
To probe to see if a FileSystem instance implements it, StreamCapabilities.hasCapability("s3a:fs.s3a.select.sql")
will be true on an instance of the S3AFileSystem class if the version of Hadoop supports S3 Select, and it is enabled for that specific instance.
If this call returns false, then S3 Select calls will fail.
Rather than cast a filesystem to the S3AFileSystem
class, cast it to org.apache.hadoop.fs.StreamCapabilities
; a class which was added in Hadoop 2.9. This should result in less brittle code -and there is no need to have the hadoop-aws
JAR on the classpath at compile time.
/** * Probe for a filesystem instance supporting S3 Select. * @param fs filesystem * @return true if the filesystem supports S3 Select. */ public static boolean hasS3SelectCapability(final FileSystem fs) { return (fs instanceof StreamCapabilities) && ((StreamCapabilities)fs).hasCapability("s3a:fs.s3a.select.sql"); }
Applications can issue select queries through the Hadoop FileSystem/FileContext APIs via the asynchronous openFile()
call added in Hadoop 3.3.
Use the FileSystem.openFile(path)
or FileContext.openFile(path)
methods command to get a builder class for the open operations, then set the mandatory s3 select options though multiple must()
parameters.
FileSystem.FSDataInputStreamBuilder builder = filesystem.openFile("s3a://bucket/path-to-file.csv") .must("fs.s3a.select.sql", "SELECT * FROM S3OBJECT s WHERE s.\"odd\" = `TRUE`") .must("fs.s3a.select.input.format", "CSV") .must("fs.s3a.select.input.compression", "NONE") .must("fs.s3a.select.input.csv.header", "use") .must("fs.s3a.select.output.format", "CSV") .must("fs.s3a.select.output.csv.field.delimiter", "\t") .must("fs.s3a.select.output.csv.quote.character", "\"") .must("fs.s3a.select.output.csv.quote.fields", "asneeded") ; CompletableFuture<FSDataInputStream> future = builder.build(); try (FSDataInputStream select = future.get()) { // process the output byte[] bytes = new byte[8192]; int actual = select.read(bytes); }
When the Builder’s build()
call is made, if the FileSystem/FileContext instance does not recognize any of the mandatory options it will fail. The S3A connector does recognize them, and, if S3 Select has not been disabled, will issue the Select query against the object store.
If the S3A connector has S3 Select disabled, it will fail with an UnsupportedOperationException
.
The build()
call returns a CompletableFuture<FSDataInputStream>
. This future retrieves the result of the select call, which is executed asynchronously in the S3A FileSystem instance’s executor pool.
Errors in the SQL, missing file, permission failures and suchlike will surface when the future is evaluated, not the build call.
In the returned stream, seeking and positioned reads do not work as usual, because there are no absolute positions in the file to seek to.
seek()
calls will raise a PathIOException
.seek()
calls will succeed, but only by reading and discarding bytes. This will be slow.PathIOException
.getPos()
) will be updated. This is not compliant with the filesystem specification.This is all done as a best-effort attempt to support existing code which often uses seek()
to move forward in a split file after opening, or does a series of positioned read calls.
SelectInputStream
The returned stream, of type org.apache.hadoop.fs.s3a.select.SelectInputStream
, only supports forward seek()
operations.
A zero-byte seek operation is always valid:
stream.seek(stream.getPos());
A negative seek operation will always fail:
stream.seek(stream.getPos() - offset);
A forward seek operation will work, provided the final position is less than the total length of the stream:
stream.seek(stream.getPos() + offset);
If it is past the end of the file, an EOFException
is raised.
Important Forward seeks are implemented by reading and discarding the contents of the stream. The bigger the forward seek, the more data is thrown away, the longer the operation takes. And, if the data is being paid for over a long-haul S3 connection. the more the seek costs.
Calling seek()
on a SelectInputStream
should only be used with care.
The feature has been implemented for splittable queries across Selected data, where the initial read starts with a seek()
to the offset. However, for reasons covered below, a codec should be used to declare the input unsplittable.
Third party object stores do not, at the time of writing, support S3 Select. S3 Select operations against such stores will fail, presumably with a store-specific error code.
To avoid problems, disable S3 Select entirely:
<property> <name>fs.s3a.select.enabled</name> <value>false</value> </property>
This guarantees that the hasCapability()
check will fail immediately, rather than delaying the failure until an SQL query is attempted.
hadoop s3guard select
The s3guard select
command allows direct select statements to be made of a path.
Usage:
hadoop s3guard select [OPTIONS] \ [-limit rows] \ [-header (use|none|ignore)] \ [-out file] \ [-compression (gzip|none)] \ [-expected rows] [-inputformat csv] [-outputformat csv] <PATH> <SELECT QUERY>
The output is printed, followed by some summary statistics, unless the -out
option is used to declare a destination file. In this mode status will be logged to the console, but the output of the query will be saved directly to the output file.
Read the first 100 rows of the landsat dataset where cloud cover is zero:
hadoop s3guard select -header use -compression gzip -limit 100 \ s3a://landsat-pds/scene_list.gz \ "SELECT * FROM S3OBJECT s WHERE s.cloudCover = '0.0'"
Return the entityId
column for all rows in the dataset where the cloud cover was “0.0”, and save it to the file output.csv
:
hadoop s3guard select -header use -out s3a://mybucket/output.csv \ -compression gzip \ s3a://landsat-pds/scene_list.gz \ "SELECT s.entityId from S3OBJECT s WHERE s.cloudCover = '0.0'"
This file will:
The output can be saved to a file with the -out
option. Note also that -D key=value
settings can be used to control the operation, if placed after the s3guard
command and before select
hadoop s3guard \ -D s.s3a.select.output.csv.quote.fields=asneeded \ select \ -header use \ -compression gzip \ -limit 500 \ -inputformat csv \ -outputformat csv \ -out s3a://hwdev-steve-new/output.csv \ s3a://landsat-pds/scene_list.gz \ "SELECT s.entityId from S3OBJECT s WHERE s.cloudCover = '0.0'"
S3 Select support in analytics queries is a work in progress. It does not work reliably with large source files where the work is split up.
As a proof of concept only, S3 Select queries can be made through MapReduce jobs which use any Hadoop RecordReader
class which uses the new openFile()
API.
Currently this consists of the following MRv2 readers.
org.apache.hadoop.mapreduce.lib.input.LineRecordReader org.apache.hadoop.mapreduce.lib.input.FixedLengthRecordReader
And a limited number of the MRv1 record readers:
org.apache.hadoop.mapred.LineRecordReader
All of these readers use the new API and can be have its optional/mandatory options set via the JobConf
used when creating/configuring the reader.
These readers are instantiated within input formats; the following formats therefore support S3 Select.
org.apache.hadoop.mapreduce.lib.input.FixedLengthInputFormat org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat org.apache.hadoop.mapreduce.lib.input.NLineInputFormat org.apache.hadoop.mapreduce.lib.input.TextInputFormat org.apache.hadoop.mapred.KeyValueTextInputFormat org.apache.hadoop.mapred.TextInputFormat org.apache.hadoop.mapred.lib.NLineInputFormat
All JobConf
options which begin with the prefix mapreduce.job.input.file.option.
will have that prefix stripped and the remainder used as the name for an option when opening the file.
All JobConf
options which being with the prefix mapreduce.job.input.file.must.
will be converted into mandatory options.
To use an S3 Select call, set the following options
mapreduce.job.input.file.must.fs.s3a.select.sql = <SQL STATEMENT> mapreduce.job.input.file.must.fs.s3a.select.input.format = CSV mapreduce.job.input.file.must.fs.s3a.select.output.format = CSV
Further options may be set to tune the behaviour, for example:
jobConf.set("mapreduce.job.input.file.must.fs.s3a.select.input.csv.header", "use");
Note How to tell if a reader has migrated to the new openFile()
builder API:
Set a mandatory option which is not known; if the job does not fail then an old reader is being used.
jobConf.set("mapreduce.job.input.file.must.unknown.option", "anything");
S3 Select queries can be made against gzipped source files; the S3A input stream receives the output in text format, rather than as a (re)compressed stream.
To read a gzip file, set fs.s3a.select.input.compression
to gzip
.
jobConf.set("mapreduce.job.input.file.must.fs.s3a.select.input.compression", "gzip");
Most of the Hadoop RecordReader classes automatically choose a decompressor based on the extension of the source file. This causes problems when reading .gz
files, because S3 Select is automatically decompressing and returning csv-formatted text.
By default, a query across gzipped files will fail with the error “IOException: not a gzip file”
To avoid this problem, declare that the job should switch to the “Passthrough Codec” for all files with a “.gz” extension:
jobConf.set("io.compression.codecs", "org.apache.hadoop.io.compress.PassthroughCodec"); jobConf.set("io.compress.passthrough.extension", ".gz");
Obviously, this breaks normal .gz
decompression: only set it on S3 Select jobs.
Consult the javadocs for org.apache.hadoop.fs.s3a.select.SelectConstants
.
The listed options can be set in core-site.xml
, supported by S3A per-bucket configuration, and can be set programmatically on the Configuration
object use to configure a new filesystem instance.
Any of these options can be set in the builder returned by the openFile()
call —simply set them through a chain of builder.must()
operations.
<property> <name>fs.s3a.select.input.format</name> <value>csv</value> <description>Input format</description> </property> <property> <name>fs.s3a.select.output.format</name> <value>csv</value> <description>Output format</description> </property> <property> <name>fs.s3a.select.input.csv.comment.marker</name> <value>#</value> <description>In S3 Select queries: the marker for comment lines in CSV files</description> </property> <property> <name>fs.s3a.select.input.csv.record.delimiter</name> <value>\n</value> <description>In S3 Select queries over CSV files: the record delimiter. \t is remapped to the TAB character, \r to CR \n to newline. \\ to \ and \" to " </description> </property> <property> <name>fs.s3a.select.input.csv.field.delimiter</name> <value>,</value> <description>In S3 Select queries over CSV files: the field delimiter. \t is remapped to the TAB character, \r to CR \n to newline. \\ to \ and \" to " </description> </property> <property> <name>fs.s3a.select.input.csv.quote.character</name> <value>"</value> <description>In S3 Select queries over CSV files: quote character. \t is remapped to the TAB character, \r to CR \n to newline. \\ to \ and \" to " </description> </property> <property> <name>fs.s3a.select.input.csv.quote.escape.character</name> <value>\\</value> <description>In S3 Select queries over CSV files: quote escape character. \t is remapped to the TAB character, \r to CR \n to newline. \\ to \ and \" to " </description> </property> <property> <name>fs.s3a.select.input.csv.header</name> <value>none</value> <description>In S3 Select queries over CSV files: what is the role of the header? One of "none", "ignore" and "use"</description> </property> <property> <name>fs.s3a.select.input.compression</name> <value>none</value> <description>In S3 Select queries, the source compression algorithm. One of: "none" and "gzip"</description> </property> <property> <name>fs.s3a.select.output.csv.quote.fields</name> <value>always</value> <description> In S3 Select queries: should fields in generated CSV Files be quoted? One of: "always", "asneeded". </description> </property> <property> <name>fs.s3a.select.output.csv.quote.character</name> <value>"</value> <description> In S3 Select queries: the quote character for generated CSV Files. </description> </property> <property> <name>fs.s3a.select.output.csv.quote.escape.character</name> <value>\\</value> <description> In S3 Select queries: the quote escape character for generated CSV Files. </description> </property> <property> <name>fs.s3a.select.output.csv.record.delimiter</name> <value>\n</value> <description> In S3 Select queries: the record delimiter for generated CSV Files. </description> </property> <property> <name>fs.s3a.select.output.csv.field.delimiter</name> <value>,</value> <description> In S3 Select queries: the field delimiter for generated CSV Files. </description> </property> <property> <name>fs.s3a.select.errors.include.sql</name> <value>false</value> <description> Include the SQL statement in errors: this is useful for development but may leak security and Personally Identifying Information in production, so must be disabled there. </description> </property>
SQL Injection attacks are the classic attack on data. Because S3 Select is a read-only API, the classic “Bobby Tables” attack to gain write access isn’t going to work. Even so: sanitize your inputs.
CSV does have security issues of its own, specifically:
Excel and other spreadsheets may interpret some fields beginning with special characters as formula, and execute them
S3 Select does not appear vulnerable to this, but in workflows where untrusted data eventually ends up in a spreadsheet (including Google Document spreadsheets), the data should be sanitized/audited first. There is no support for such sanitization in S3 Select or in the S3A connector.
Logging Select statements may expose secrets if they are in the statement. Even if they are just logged, this may potentially leak Personally Identifying Information as covered in the EU GDPR legislation and equivalents.
For both privacy and security reasons, SQL statements are not included in exception strings by default, nor logged at INFO level.
To enable them, set fs.s3a.select.errors.include.sql
to true
, either in the site/application configuration, or as an option in the builder for a single request. When set, the request will also be logged at the INFO level of the log org.apache.hadoop.fs.s3a.select.SelectBinding
.
Personal Identifiable Information is not printed in the AWS S3 logs. Those logs contain only the SQL keywords from the query planner. All column names and literals are masked. Following is a sample log example:
Query:
SELECT * FROM S3OBJECT s;
Log:
select (project (list (project_all))) (from (as str0 (id str1 case_insensitive)))
Note also that:
org.apache.hadoop.fs.s3a
and other components’s debug logs may also log the SQL statements (e.g. aws-sdk HTTP logs).The best practise here is: only enable SQL in exceptions while developing SQL queries, especially in an application/notebook where the exception text is a lot easier to see than the application logs.
In production: don’t log or report. If you do, all logs and output must be considered sensitive from security and privacy perspectives.
The hadoop s3guard select
command does enable the logging, so can be used as an initial place to experiment with the SQL syntax. Rationale: if you are constructing SQL queries on the command line, your shell history is already tainted with the query.
The SQL Syntax directly supported by the AWS S3 Select API is documented by Amazon.
SELECT
calls, no JOIN
.“CSV” is less a format, more “a term meaning the data is in some nonstandard line-by-line” text file, and there are even “multiline CSV files”.
S3 Select only supports a subset of the loose “CSV” concept, as covered in the AWS documentation. There are also limits on how many columns and how large a single line may be.
The specific quotation character, field and record delimiters, comments and escape characters can be configured in the Hadoop configuration.
Consistency
Since November 2020, AWS S3 has been fully consistent. This also applies to S3 Select. We do not know what happens if an object is overwritten while a query is active.
Concurrency
The outcome of what happens when source file is overwritten while the result of a select call is overwritten is undefined.
The input stream returned by the operation is NOT THREAD SAFE.
Error Handling
If an attempt to issue an S3 select call fails, the S3A connector will reissue the request if-and-only-if it believes a retry may succeed. That is: it considers the operation to be idempotent and if the failure is considered to be a recoverable connectivity problem or a server-side rejection which can be retried (500, 503).
If an attempt to read data from an S3 select stream (org.apache.hadoop.fs.s3a.select.SelectInputStream)
fails partway through the read, no attempt is made to retry the operation
In contrast, the normal S3A input stream tries to recover from (possibly transient) failures by attempting to reopen the file.
The select operation is best when the least amount of data is returned by the query, as this reduces the amount of data downloaded.
LIMIT
to set an upper limit on the rows read, rather than implementing a row counter in application code and closing the stream when reached. This avoids having to abort the HTTPS connection and negotiate a new one on the next S3 request.The select call itself can be slow, especially when the source is a multi-MB compressed file with aggressive filtering in the WHERE
clause. Assumption: the select query starts at row 1 and scans through each row, and does not return data until it has matched one or more rows.
If the asynchronous nature of the openFile().build().get()
sequence can be taken advantage of, by performing other work before or in parallel to the get()
call: do it.
Getting S3 Select code to work is hard, though those knowledgeable in SQL will find it easier.
Problems can be split into:
The exceptions here are all based on the experience during writing tests; more may surface with broader use.
All failures other than network errors on request initialization are considered unrecoverable and will not be reattempted.
As parse-time errors always state the line and column of an error, you can simplify debugging by breaking a SQL statement across lines, e.g.
String sql = "SELECT\n" + "s.entityId \n" + "FROM " + "S3OBJECT s WHERE\n" + "s.\"cloudCover\" = '100.0'\n" + " LIMIT 100";
Now if the error is declared as “line 4”, it will be on the select conditions; the column offset will begin from the first character on that row.
The SQL Statements issued are only included in exceptions if fs.s3a.select.errors.include.sql
is explicitly set to true. This can be done in an application during development, or in a openFile()
option parameter. This should only be done during development, to reduce the risk of logging security or privacy information.
S3 Select returns paged results; the source file is not filtered in one go in the initial request.
This means that errors related to the content of the data (type casting, etc) may only surface partway through the read. The errors reported in such a case may be different than those raised on reading the first page of data, where it will happen earlier on in the read process.
See:
This surfaces when trying to read in data from a .gz
source file through an MR or other analytics query, and the gzip codec has tried to parse it.
java.io.IOException: not a gzip file at org.apache.hadoop.io.compress.zlib.BuiltInGzipDecompressor.processBasicHeader(BuiltInGzipDecompressor.java:496) at org.apache.hadoop.io.compress.zlib.BuiltInGzipDecompressor.executeHeaderState(BuiltInGzipDecompressor.java:257) at org.apache.hadoop.io.compress.zlib.BuiltInGzipDecompressor.decompress(BuiltInGzipDecompressor.java:186) at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:111) at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:105) at java.io.InputStream.read(InputStream.java:101) at org.apache.hadoop.util.LineReader.fillBuffer(LineReader.java:182) at org.apache.hadoop.util.LineReader.readCustomLine(LineReader.java:306) at org.apache.hadoop.util.LineReader.readLine(LineReader.java:174) at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.skipUtfByteOrderMark(LineRecordReader.java:158) at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:198)
The underlying problem is that the gzip decompressor is automatically enabled when the the source file ends with the “.gz” extension. Because S3 Select returns decompressed data, the codec fails.
The workaround here is to declare that the job should add the “Passthrough Codec” to its list of known decompressors, and that this codec should declare the file format it supports to be “.gz”.
io.compression.codecs = org.apache.hadoop.io.compress.PassthroughCodec io.compress.passthrough.extension = .gz
InvalidColumnIndex
Your SQL is wrong and the element at fault is considered an unknown column name.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT WHERE odd = true on test/testSelectOddLines.csv: com.amazonaws.services.s3.model.AmazonS3Exception: The column index at line 1, column 30 is invalid. Please check the service documentation and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidColumnIndex;
Here it’s the first line of the query, column 30. Paste the query into an editor and position yourself on the line and column at fault.
SELECT * FROM S3OBJECT WHERE odd = true ^ HERE
Another example:
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT s WHERE s._1 = "true" on test/testSelectOddLines.csv: com.amazonaws.services.s3.model.AmazonS3Exception: The column index at line 1, column 39 is invalid. Please check the service documentation and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidColumnIndex;
Here it is because strings must be single quoted, not double quoted.
SELECT * FROM S3OBJECT s WHERE s._1 = "true" ^ HERE
S3 select uses double quotes to wrap column names, interprets the string as column “true”, and fails with a non-intuitive message.
Tip: look for the element at fault and treat the InvalidColumnIndex
message as a parse-time message, rather than the definitive root cause of the problem.
ParseInvalidPathComponent
Your SQL is wrong.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT s WHERE s.'odd' is "true" on test/testSelectOddLines.csv : com.amazonaws.services.s3.model.AmazonS3Exception: Invalid Path component, expecting either an IDENTIFIER or STAR, got: LITERAL,at line 1, column 34. (Service: Amazon S3; Status Code: 400; Error Code: ParseInvalidPathComponent;
SELECT * FROM S3OBJECT s WHERE s.'odd' is "true" on test/testSelectOddLines.csv ^ HERE
ParseExpectedTypeName
Your SQL is still wrong.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT s WHERE s.odd = "true" on test/testSelectOddLines.csv: com.amazonaws.services.s3.model.AmazonS3Exception : Expected type name, found QUOTED_IDENTIFIER:'true' at line 1, column 41. (Service: Amazon S3; Status Code: 400; Error Code: ParseExpectedTypeName;
ParseUnexpectedToken
Your SQL is broken.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT s WHERE s.5 = `true` on test/testSelectOddLines.csv: com.amazonaws.services.s3.model.AmazonS3Exception: Unexpected token found LITERAL:5d-1 at line 1, column 33. (Service: Amazon S3; Status Code: 400; Error Code: ParseUnexpectedToken;
ParseUnexpectedOperator
Your SQL is broken.
com.amazonaws.services.s3.model.AmazonS3Exception: Unexpected operator OPERATOR:'%' at line 1, column 45. (Service: Amazon S3; Status Code: 400; Error Code: ParseUnexpectedOperator; Request ID: E87F30C57436B459; S3 Extended Request ID: UBFOIgkQxBBL+bcBFPaZaPBsjdnd8NRz3NFWAgcctqm3n6f7ib9FMOpR+Eu1Cy6cNMYHCpJbYEY =:ParseUnexpectedOperator: Unexpected operator OPERATOR:'%' at line 1, column 45. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
MissingHeaders
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: SELECT * FROM S3OBJECT s WHERE s."odd" = `true` on test/testSelectOddLines.csv: com.amazonaws.services.s3.model.AmazonS3Exception: Some headers in the query are missing from the file. Please check the file and try again. (Service: Amazon S3; Status Code: 400; Error Code: MissingHeaders;
This can happen if you are trying to use double quotes for constants in the SQL expression.
SELECT * FROM S3OBJECT s WHERE s."odd" = "true" on test/testSelectOddLines.csv: ^ HERE
Double quotes (") may only be used when naming columns; for constants single quotes are required.
org.apache.hadoop.fs.s3a.AWSS3IOException: Select on test/testSelectWholeFile: com.amazonaws.services.s3.model.AmazonS3Exception: The specified method is not allowed against this resource. (Service: Amazon S3; Status Code: 405; Error Code: MethodNotAllowed;
You are trying to use S3 Select to read data which for some reason you are not allowed to.
InvalidTextEncoding
The file couldn’t be parsed. This can happen if you try to read a .gz
file and forget to set the compression in the select request.
That can be done through the fs.s3a.select.compression
option.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: '" SELECT * FROM S3OBJECT s WHERE endstation_name = 'Bayswater Road: Hyde Park' " on s3a://example/dataset.csv.gz: com.amazonaws.services.s3.model.AmazonS3Exception: UTF-8 encoding is required. The text encoding error was found near byte 8,192. (Service: Amazon S3; Status Code: 400; Error Code: InvalidTextEncoding
InvalidCompressionFormat
“GZIP is not applicable to the queried object”A SELECT call has been made using a compression which doesn’t match that of the source object, such as it being a plain text file.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select: '" SELECT * FROM S3OBJECT s WHERE endstation_name = 'Bayswater Road: Hyde Park' " on s3a://example/dataset.csv: com.amazonaws.services.s3.model.AmazonS3Exception: GZIP is not applicable to the queried object. Please correct the request and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidCompressionFormat; at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:212) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111) ... Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: GZIP is not applicable to the queried object. Please correct the request and try again. Service: Amazon S3; Status Code: 400; Error Code: InvalidCompressionFormat; at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse ...
UnsupportedStorageClass
S3 Select doesn’t work with some storage classes like Glacier or Reduced Redundancy. Make sure you’ve set fs.s3a.create.storage.class
to a supported storage class for S3 Select.
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select on s3a://example/dataset.csv.gz: com.amazonaws.services.s3.model.AmazonS3Exception: We do not support REDUCED_REDUNDANCY storage class. Please check the service documentation and try again. (Service: Amazon S3; Status Code: 400; Error Code: UnsupportedStorageClass
PathIOException
: “seek() not supported”The input stream returned by the select call does not support seeking backwards in the stream.
Similarly, PositionedReadable
operations will fail when used to read data any offset other than that of getPos()
.
org.apache.hadoop.fs.PathIOException: `s3a://landsat-pds/landsat.csv.gz': seek() not supported at org.apache.hadoop.fs.s3a.select.SelectInputStream.unsupported(SelectInputStream.java:254) at org.apache.hadoop.fs.s3a.select.SelectInputStream.seek(SelectInputStream.java:243) at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:66)
There is no fix for this. You can move forward in a file using skip(offset)
; bear in mind that the return value indicates what offset was skipped -it may be less than expected.
IllegalArgumentException
: "Unknown mandatory key “fs.s3a.select.sql”The filesystem is not an S3A filesystem, and the s3a select option is not recognized.
java.lang.IllegalArgumentException: Unknown mandatory key "fs.s3a.select.sql" at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) at org.apache.hadoop.fs.AbstractFSBuilder.lambda$rejectUnknownMandatoryKeys$0(AbstractFSBuilder.java:331) at java.lang.Iterable.forEach(Iterable.java:75) at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080) at org.apache.hadoop.fs.AbstractFSBuilder.rejectUnknownMandatoryKeys(AbstractFSBuilder.java:330) at org.apache.hadoop.fs.filesystem.openFileWithOptions(FileSystem.java:3541) at org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4442)
IllegalArgumentException
: “Unknown mandatory key in non-select file I/O”The file options to tune an S3 select call are only valid when a SQL expression is set in the fs.s3a.select.sql
option. If not, any such option added as a must()
value will fail.
java.lang.IllegalArgumentException: Unknown mandatory key for s3a://example/test/testSelectOptionsOnlyOnSelectCalls.csv in non-select file I/O "fs.s3a.select.input.csv.header" at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115) at org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.lambda$rejectUnknownMandatoryKeys$0(AbstractFSBuilderImpl.java:352) at java.lang.Iterable.forEach(Iterable.java:75) at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080) at org.apache.hadoop.fs.impl.AbstractFSBuilderImpl.rejectUnknownMandatoryKeys(AbstractFSBuilderImpl.java:351) at org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:3736) at org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4471)
Requiring these options without providing a SQL query is invariably an error. Fix: add the SQL statement, or use opt()
calls to set the option.
If the fs.s3a.select.sql
option is set, and still a key is rejected, then either the spelling of the key is wrong, it has leading or trailing spaces, or it is an option not supported in that specific release of Hadoop.
Backwards seeks in an S3 Select SelectInputStream
are not supported.
org.apache.hadoop.fs.PathIOException: `s3a://landsat-pds/scene_list.gz': seek() backwards from 16387 to 0 not supported at org.apache.hadoop.fs.s3a.select.SelectInputStream.unsupported(SelectInputStream.java:288) at org.apache.hadoop.fs.s3a.select.SelectInputStream.seek(SelectInputStream.java:253) at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:66)
The SELECT refers to the name of a column which is not recognized
s.oddf
.s._1
, but headers have been enabled. Fix. disable, or use the header name.org.apache.hadoop.fs.s3a.AWSBadRequestException: SELECT * FROM S3OBJECT WHERE s."oddf" = 'true' on s3a://example/test/testParseBrokenCSVFile: com.amazonaws.services.s3.model.AmazonS3Exception: Invalid table alias is specified at line 1, column 30. Please check the file and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidTableAlias; Invalid table alias is specified at line 1, column 30. Please check the file and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidTableAlias; Request ID: 8693B86A52CFB91C; at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:225) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265) ... Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Invalid table alias is specified at line 1, column 30. Please check the file and try again. (Service: Amazon S3; Status Code: 400; Error Code: InvalidTableAlias; Request ID: 8693B86A52CFB91C; at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1640) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
AWSBadRequestException
“Attempt to convert from one data type to another failed: cast from STRING to TIMESTAMP.”A string field could not be converted to a timestamp because one or more of its entries were not parseable with the given timestamp.
Example, from a spreadsheet where “timestamp” is normally a well-formatted timestamp field, but in one column it is just “Tuesday”
SELECT CAST(s.date AS TIMESTAMP) FROM S3OBJECT s
org.apache.hadoop.fs.s3a.AWSBadRequestException: Select on s3a://example/test/testParseBrokenCSVFile: com.amazonaws.services.s3.model.AmazonS3Exception: Attempt to convert from one data type to another failed: cast from STRING to TIMESTAMP. (Service: Amazon S3; Status Code: 400; Error Code: CastFailed; Request ID: E2158FE45AF2049A; S3 Extended Request ID: iM40fzGuaPt6mQo0QxDDX+AY1bAgSVD1sKErFq6Y4GDJYHIAnmc00i0EvGGnH+0MFCFhKIivIrQ=), S3 Extended Request ID: iM40fzGuaPt6mQo0QxDDX+AY1bAgSVD1sKErFq6Y4GDJYHIAnmc00i0EvGGnH+0MFCFhKIivIrQ=:CastFailed: Attempt to convert from one data type to another failed: cast from STRING to TIMESTAMP. (Service: Amazon S3; Status Code: 400; Error Code: CastFailed; Request ID: E2158FE45AF2049A; S3 Extended Request ID: iM40fzGuaPt6mQo0QxDDX+AY1bAgSVD1sKErFq6Y4GDJYHIAnmc00i0EvGGnH+0MFCFhKIivIrQ=) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:225) at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111) at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$3(Invoker.java:265) Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Attempt to convert from one data type to another failed: cast from STRING to TIMESTAMP. (Service: Amazon S3; Status Code: 400; Error Code: CastFailed;)
There’s no way to recover from a bad record here; no option to skip invalid rows.
Note: This is an example stack trace without the SQL being printed.


This extension contains various modules for enhancing Burp's capabiities.
Warning: take care scanning untrusted sites. The SQLMapper component has had command injection flaws in the past.
The extension has its own configuration tab with sub-tabs for each Co2 module.
Modules that interact with other Burp tools can be disabled from within the Co2 configuration tab, so there is no need to disable the entire extension when using just part of the functionality.
O2 includes the following modules:
- SQLMapper, a sqlmap helper.
Simply right-click on any request in Burp and you will see a new menu option to send the request to SQLMapper.
The SQLMapper screen will appear pre-populated with the URL, POST data (if applicable) and Cookies (if applicable) from the request.
You can then set any other options you need and then copy/paste the SQLMap Command to sqlmap on your command line.
- User Generator -
For this one I collected publicly available census data from http://www.census.gov/genealogy/www/data/2000surnames/ (for surnames)
and popular baby names from the social security website (http://www.ssa.gov/OACT/babynames/) to make a username generator based on this statistical data.
The interface allows you to tinker with the data sets a little bit, specify if you want full names, initials, a delimiter between first and last names, etc.
The tool will approximate which name combinations are the most common and sort the list accordingly.
The result set is currently limited to the top 200,000 names to avoid performance issues.
- Name Mangler -
Given some names and domains it will mangle them to generate a list of potential usernames that can be dropped into
Intruder to test for valid logins.
- CeWLer -
Based on Digininja's command-line CeWL script for extracting a wordlist from HTML files, this version works with a list of responses
directly inside of Burp.
- Masher - Given a seed list of words and a password specification this tool will generate a fuzzy list of possible
passwords. Masher will start with combining words from the provided list, then append and replace characters to build
new passwords.
- BasicAuther - Given a list of usernames and a list of passwords it will output proper BasicAuth strings
that can then be dropped into Intruder.


Protect yourself, the patient, and your staff by reducing airborne particles with continuous suction
To protect the health care personnel from aerosols produced by Covid-19 patients, we aim to develop an indigenous micro-airflow management device.Aerosol generating procedures, pose a serious threat to all medical staff.10-20 % of healthcare staff is infected in various countries. .Virus in aerosolized form stay for 72 hours as per NIAID, USA .WHO requires 12 air changes /hour in negative pressure rooms however such rooms are short.Our solution is a micro-airflow management device, which provides an additional degree of protection to clinicians by using negative pressure that captures aerosols released by patients.
While we were working on aerosol containment we soon realized that one of the most affected field of medicine is dentistry, more than 90% of dental procedures involve aerosols generating procedures due to rotary instruments which have pressurized water spray. These bio aerosols might have pathogenic bacteria, viruses, blood, saliva, which disperse in the dental clinic and are responsible for respiratory diseases. With the advent of the pandemic, dentists have been forced to stop performing AGP’S or stop practice altogether effecting over 1.25lakhs dental clinics over the country.
The existing solutions, primarily cater to maintain a dry field in the mouth but are not effective in bio aerosol containment The Rubber Dam whose primary objective is to provide isolation is married with a High volume suction also known as HVE present in the dental chair this combination is currently considered as gold standard. However it is so cumbersome and time consuming to use that almost all dentist do not prefer to use it despite being part of SOP since decades. Not only that it is also uncomfortable for the patient to wear for long and do not have provision to suction out fluid from behind the rubber dam. Next product is a Extra oral suction system which as name suggest suctions the aerosols above the patient’s mouth, however these new systems price starts from 50k to as high as 90K based on make and country of origin Despite this Most of the dentist doubt their efficacy. And the last we have Isolite and similar products, these provide isolation, dry field, check and tongue retraction and aerosol containment almost equal to RD+HVE however these are not available in India and costs about 1.5 lacs which is too much for Indian Docs.
To understand the problem and the user we did two rounds of customer interactions. Based on these interactions we could clearly map out customers problems, goals, fear, preferences and influencers. One of the most important insight was the spending capacity of the customer for the problem and also Difference in preferences in the older and younger doctors
Understanding customer is not sufficient but we also need to deeply understand the task customer is trying to get done and how. With CJM we have pin pointed identified how the customer is trying to acheive that and what are the influential factors at each stage of buying process of the customer and also the the channels through which we would have to market, promote and sell our product in future.
Based on this information we have frozen the need statement as An Efficient and cost effective way to protect dentist from bioaerosols during procedures with need specs as reduce the spread of Bio Aerosols from Rotary instruments and patient’s throad and cost limited to 100 per procedure and upfront cost limited to 10000rs.


0:05:00
0:05:00
...
Moves:
Base time (min)
Increment (sec)
Skill Level (0-20)
I play
White
Black
Show score
New Game
Promote to
Queen
Rook
Bishop
Knight
Evaluation


{% extends "base.html" %}
{% block body_class %}journal{% endblock %}
{% block content %}
{% include "journal/includes/header.html" %}
{% include "journal/includes/levelMenu.html" %}
{% block main_content %}{% endblock %}
{% include "includes/footer.html" %}
{% include "journal/includes/alternative_header.html" %}
{% include "journal/includes/contact.html" %}
{% endblock %}
{% block extra_js %}
{% include "journal/includes/contact_script.html" %}
{% endblock%}


Everyday holds the possibility of a miracle.->
That moment when you have so much stuff to do but you decide to take a nap instead.
If you can look at the Sunset and Smile then you still have hope.


Saves a file onto the filesystem. If passed a string, uses default encoding.
rs2: obj = obj.fileSave(path="Path to save to."); levrJs: obj = fileSave.call(this,obj,"Path to save to.");
Method Summary
| Public Methods | |
|---|---|
| public |
fileSave(obj, path, [overwrite=true], [safe=true]):
|
Public Methods
ew.levr.base/src/main/java/com/eduworks/cruncher/file/CruncherFileSave.java:31
public fileSave(obj, path, [overwrite=true], [safe=true]):
Parameters:
| Name | Type | Attribute | Description |
|---|---|---|---|
| obj | Object |
|
(InMemoryFile|File|String) Data to save to a file. |
| path | Object |
|
(String) Path to save the file. |
| overwrite | Object |
|
Overwrite the file if it already exists. |
| safe | Boolean |
|
For security, will not obey paths that contain '..'or startsWIth('/') paths. |
Return:
(InMemoryFile|File|String) Value passed into obj.


The application would like the ability to access your basic information for and the following additional data:
Allow the application access?


--- title: Home sitemap: true ---
Kong is an open distributed platform for your APIs,
focused on high performance and reliability.
Learn how Kong can help manage and grow your private and public APIs
Manage API developers, from authentication to rate limiting.
Provide documentation, usage analytics and Issues to API developers.
You can add extra functionality to Kong by installing plugins in any flavor.
Kong sits in front of your API, on your servers or in the cloud
Accesible from any client, server, mobile or application
Horizontally scalable to handle hundreds of billion of requests
Supports any RESTful API, inside or outside your infrastructure
Don't panic, we have everything you need 1 click away
Learn how to install and use Kong by reading the official documentation
Learn how to contribute to Kong and get support from the community.
Having Kong running in production and looking for SLA-based support?


A simple app to swap items for trade with friends, may or may not be currently optimized for craft beer :)
Github RepoThis website! This site is open source as well, mainly because its a sandbox for me to experiment with new features, checkout the code sometime!
Github RepoPFPB was myself, and two collegues Senior Design Project while at Western Michigan University. We developed a auditory feedback workout system for the Psycology Department to run behavioral tests with. The system consisted of a microcontroller box, that would mount to the top of a punching bag stand, connected to an array of removable accelerometers on the punching bag itself. The microcontroller then communicted with a custom Java app we also wrote to track workout sessions over time, playback user selected music playlists, and change the frequency & speed of the playback based on the input from the microcontroller and the user defined goals for their workout. The device was so well recieved that the OVPR department at Western Michigan University filed for a patent on the device as well. Feel free to checkout the source code as it has now been open sourced.
Bitbucket Repo


| Time | Who | What |
| 4:00 | | Doors open + Social Time |
| 4:30 | | Food and beer served - sponsored by Neat and Sungard |
| 5:30 | Redsnake organizers | Welcome to RSP14 |
| | Peter Gebhard | What’s New In Python and Ruby |
| | Chris Conley | The Game of Hangman, Decision Theory and Delivering Software |
| | | a word from our sponsors |
| | Steve Dorazio | Blinker |
| | Alyssa Batula | Brain Controlled Robots with Python |
| | Stefan Natchev | Lapis - the Web Framework that Runs Inside Nginx |
| 6:50 | | Coffee Break - sponsored by PromptWorks |
| 7:10 | Dan McClory | Put a REPL on it! |
| | Ryan Deering | GPU Programming |
| | | a word from our sponsors |
| | Eric Windisch | Docker for Testing, Shipping, and Deploying |
| | Joe Winter | Torquebox - Like a JBoss |
| | Ben Plotnick | Salt Stack |
| | | Social |
| 9:00 | | Go home, you bunch of freeloaders. Or continue at a local bar. |


LIAM
A film review by Steve Rhodes
Copyright 2001 Steve Rhodes
RATING (0 TO ****): **
LIAM, a lackluster rendition of an ANGELA'S ASHES-type story by director Stephen Frears (HIGH FIDELITY), connects all of the usual dots. About a family facing unrelenting poverty due to "the yards" being closed, the 1930s British story concerns the usual themes of the rich vs. the poor, the haves vs. the have-nots and upstairs vs. downstairs.
The out-of-work dad (Ian Hart) vows never to accept a helping hand from the Catholic Church, lest he end up beholding to the local priest. That the dad won't accept help from the church is a key to the plot since it is the church in general and the Catholic Church in particular that serves as the movie's chief whipping boy. Just as secular religions like environmentalism are universally acclaimed in films without question, Christianity generally appears in movies only to present an easy villain. In LIAM, the Catholic teachers and priests are sadistic figures that make the devil look pretty attractive by comparison. One of the film's themes contrasts Christianity and Communism, with Communism coming out more compassionate. In the movie's least credible part, Jews enter the story only so that the movie can boldly beat its chest against both anti-Semitism and Fascism.
Most of the story, however, is more tedious and clichéd than political. It is in Jimmy McGovern's script that most of the problems lie. The acting, with one notable exception, is fine. With a single dour expression, young actor Anthony Borrows plays Liam, a lad of about eight. The over-the-top script feels it necessary to give Liam a horrendous stutter, as if his character weren't tragic enough already. When Liam begins to speak, he looks as if he is suffering through an especially difficult bowel movement. In contrast, Ian Hart, as the sullen and prideful father, is palpably poor.
The ending to LIAM is not to be believed.  If you're looking for a LIAM-type story, rent the genuine article, ANGELA'S ASHES, instead. The latter has its problems, but it's much preferable to LIAM.
LIAM runs 1:30. It is rated R for "some nudity and language" and would be acceptable for teenagers.
The film is playing now in nationwide release in the United States. In the Silicon Valley, it is showing at the Camera Cinemas and the Century Theaters.
Web: http://www.InternetReviews.com
Email<EMAIL_ADDRESS>Want free reviews and weekly movie and video recommendations via Email?
Just send me a letter with the word "subscribe" in the subject line.
========== X-RAMR-ID: 29775 X-Language: en X-RT-ReviewID: 254078 X-RT-TitleID: 1109975 X-RT-SourceID: 703 X-RT-AuthorID: 1271 X-RT-RatingText: 2/4
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


SETKEYCODES(8) Keyboard Support SETKEYCODES(8) NAME setkeycodes - load kernel scancode-to-keycode mapping table entries SYNOPSIS setkeycodes scancode keycode ... DESCRIPTION The setkeycodes command reads its arguments two at a time, each pair of arguments consisting of a scancode (given in hexadecimal) and a keycode (given in decimal). For each such pair, it tells the kernel keyboard driver to map the specified scancode to the specified keycode. This command is useful only for people with slightly unusual keyboards, that have a few keys which produce scancodes that the kernel does not recognize. THEORY The usual PC keyboard produces a series of scancodes for each key press and key release. (Scancodes are shown by showkey -s, see showkey(1).) The kernel parses this stream of scancodes, and converts it to a stream of keycodes (key press/release events). (Keycodes are shown by showkey.) Apart from a few scancodes with special meaning, and apart from the sequence produced by the Pause key, and apart from shiftstate related scancodes, and apart from the key up/down bit, the stream of scancodes consists of unescaped scancodes xx (7 bits) and escaped scancodes e0 xx (8+7 bits). To these scancodes or scancode pairs, a corresponding key‐ code can be assigned (in the range 1-127). For example, if you have a Macro key that produces e0 6f according to showkey(1), the command setkeycodes e06f 112 will assign the keycode 112 to it, and then loadkeys(1) can be used to define the function of this key. Some older kernels might hardwire a low scancode range to the equivalent keycodes; setkeycodes will fail when you try to remap these. 2.6 KERNELS In 2.6 kernels key codes lie in the range 1-255, instead of 1-127. (It might be best to confine oneself to the range 1-239.) In 2.6 kernels raw mode, or scancode mode, is not very raw at all. The code returned by showkey -s will change after use of setkeycodes. A ker‐ nel bug. See also showkey(1). OPTIONS None. BUGS The keycodes of X have nothing to do with those of Linux. Unusual keys can be made visible under Linux, but not under X. SEE ALSO dumpkeys (1), loadkeys (1), showkey (1), getkeycodes (8) Local 8 Nov 1994 SETKEYCODES(8)


Fill out form to see your galactic age and life expectancy stats:
Name
Age
Gender
Continent
Submit
Galactic Age
Life Expectancy
Enter your birthday:
Year
Month
Day
Submit
Your age:
Galactic Birthday
Dog Years
See how much time has passed between two dates:
Date 1
Date 2
Submit
Time between dates


studies HIV resistance integrase study viral {border-style drug px;} #ffffff;} patients data review systematic treatment use "the HIV- Sub-Saharan []. bias mutations dolutegravir regimens Health INSTI included level outcomes rate International according help inhibitors px; search {font-family Additional African Available [], antiretroviral assess effect eligible file font-size management regimens. risk studies. subtypes Africa. Conference DTG DTG-based Literature Microsoft [Internet]. alafenamide double; following from: include missing published quality resistance-associated reviews using "drug AIDS Cochrane Commons Creative Data Ngoufack available baseline border-color border-width conducted effects evidence genetic individual inhibitor-based living models randomized results reviewers solid; sources strand suppression tenofovir transfer {font-style {font-weight Africa Excel Ezechiel INSTI-based Jagni Joseph LINK Lancet Mar Medical Ndze, Ngum RAL SSA Semengue, Study Trials University Valantine [cited adherence adults adverse analyses article bold characteristics countries courier; data. duration em; emtricitabine, evidences explore grid.. https://linkinghub.elsevier.com/retrieve/pii/S. immune independently infection initial knowledge meta-analysis multicentre, non-randomized patient people present process protocol pt;} raltegravir refers relevant resolved suppression,�? tool viruses


We work with businessess and individuals to grow their online brand and reach the right audience.
We work with businessess and individuals to grow their online brand and reach the right audience.
We work with businessess and individuals to grow their online brand and reach the right audience.
We work with businessess and individuals to grow their online brand and reach the right audience.
We work with businessess and individuals to grow their online brand and reach the right audience.
 We are here to give you our time and expertise so you could implement the best into your business. We love what we do and we genuinly care about everyone we work with.
We work with businessess and individuals to grow their online brand and reach the right audience.
Designer and UX researcher
Could this be you?
Could this be you? more content


SMW
Arroyo Grande
Callender
Grover Beach
Miles
Nipomo
Oceano
Pismo Beach
San Luis Obispo
San Luis Obispo County
Santa Maria
Your City, CA
Solid Maybe Weather
Test Text
73
Humidity:
70
%
High:
85
Low:
45


Following are the implementations of the endpoint(s) used for the defect detection process.
It uses the ObjectDetection
& DefectDetection
models, trained on the Submersible Pump Impeller Defect Dataset, & available for use through
the implementation in models/ObjectModels.py & models/DefectModels.py respectively.
from fastapi import APIRouter, File, UploadFile
from starlette.responses import FileResponse
from pydantic import BaseModel
from models.DefectModels import DefectDetection
from models.ObjectModels import ObjectDetection
import utils.utils as utils
import shutil
import os
import tempfile
import base64
router = APIRouter()
defect_detection_model = DefectDetection()
object_detection_model = ObjectDetection()
class ImageResult(BaseModel):
mime: str
image: str
This function extracts the region of interest from the image (the area containing the part detected by our Object Detection model) using the function mentioned in utils/utils.py.
def extract_region_and_predict(img_name, b_box):
Once extracted, the Defect Detection model is used to predict the presence of any defect in the part.
saved_file = utils.extract_region(img_name, b_box)["filename"]
prediction = defect_detection_model.predict(saved_file)
os.remove(saved_file)
return prediction
The function first detects all objects present in an image & then predicts whether or not each of them is defective
Parameters:
img
: image file uploaded to the API endpoint using the POST request@router.post("/", response_model=ImageResult)
async def post_defect_detection(img: UploadFile =File(...)):
It first stores the uploaded image temporarily in the local database of the server runs it through our Object Detection pipeline.
img_name = "temp-img.png"
label_req = ["pump_impeller"]
with open(img_name, "wb") as buffer:
shutil.copyfileobj(img.file, buffer)
The bounding boxes of the detected parts are returned as a dictionary. For each detected part, the region of interest is selected using the bounding box & a prediction is made as to whether or not it contains any defect based on our Defect Detection Model.
detections = object_detection_model.detect_objects(img_name)
for i in range(len(detections)):
if detections[i]["label"] in label_req:
b_box = detections[i]["bounding_box"]
pred = extract_region_and_predict(img_name, b_box)
detections[i]["defect_pred"] = pred["prediction"]
These predictions are used to draw colored bounding boxes around the detected parts
final_img = utils.draw_bounding_boxes(img_name, detections)
The new image with colored bounding boxes is converted to base64 & returned as a string to the client for being displayed
os.remove(img_name)
with tempfile.NamedTemporaryFile(mode="w+b", suffix=".png", delete=False) as FOUT:
utils.save_img(final_img, filename=FOUT.name)
encoded_image_string = base64.b64encode(FOUT.read())
return {
"mime" : "image/png",
"image": encoded_image_string,
}


FoodTruck TrackR
Home
About Us
Favorites
Login
Sign-Up
Looking for food trucks in your area?
Try FoodTruck TrackR today, and enjoy the best food YOUR city has to offer.


Did you know that every single element on a web page is held within it’s own personal box? From the headline to the paragraphs to the pictures posted, each has it’s own personal box. An elements box is composed of 4 areas – content, padding, border, and margin.
Most of us have moved at least once in our life. We had to deal with the tedious task of packing up our current home in order to easily relocate our possessions to our new home. Most of the time, we pack similar items together, bathroom stuff with bathroom stuff, bedroom with bedroom, kitchen, etc. This is how a site is also set up, similar items are grouped together in order to maintain organization.
We are currently packing up the last room, the kitchen. In the kitchen, we have very fragile items. We can place them in a box and ship them out, however this would not be the most ideal move. We don’t want our dishes to break before we have a chance to use them in the new house. Let’s compare a box o dishes to an element bx on a website.
So let’s begin by thinking of the dishes as the content. The dishes are the actual important part that we are concerned about and will be utilized by a user later. We place the dishes in the box, just as a headline is placed inside it’s own box. Since we do not want the dishes to break, we place newspaper or bubble wrap around the dishes inside the box. This will keep the contents inside from being right on the side of the box. The newspaper/bubble wrap provides room around the content but inside the box just as with padding in a web element. Padding assist with the readability of the content on a web page, not so much with making sure the fragile contents don’t break.
Next we have the border, which is the actual box we used to put the dishes in. While the border is very obvious in our example, a brown cardboard box, this is not always the case on a web page. The border is what wraps around the content and padding however the border is usually invisible. A visible border is not required however the border is always there.
Lastly, we have the margin. The margin is the area outside the box. This is the space that separates one element box from the next so they do not bump into each other.
So lets review:
Now there is something very important to understand about the box model. The margin, border, and padding may either be added to the size of the element or included in the size of the element. This can be manipulated by the box-sizing property. If box sizing is set to border-box, any units of measurement added for padding, border, and margin will add to the actual size of the box. Lets say we set the width of the element to 200px. If we set the border is 1px and the padding and margin is set to 20px on the left and right sides, all these values are then added to the size of the element. Instead of just being 200px wide, the element is now 282px wide. On the other hand, if the box-sizing is set to content-box, the additional values will not increase the width of the box. The set width of 200px will remain 200px regardless of the added border, padding and margin widths.


|
App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
|
Public Member Functions | |
| def | __init__ |
| def | doc_id |
| def | fields |
| def | language |
| def | rank |
| def | field |
| def | __getitem__ |
| def | __iter__ |
| def | __repr__ |
| def | __eq__ |
| def | __ne__ |
| def | __hash__ |
| def | __str__ |
Public Attributes | |
| doc_id | |
| rank | |
| language | |
| fields | |
Represents a user generated document. The following example shows how to create a document consisting of a set of fields, some plain text and some in HTML. Document(doc_id='document_id', fields=[TextField(name='subject', value='going for dinner'), HtmlField(name='body', value='<html>I found a place.</html>', TextField(name='signature', value='brzydka pogoda', language='pl')], language='en')
| def google.appengine.api.search.search.Document.__init__ | ( | self, | |
doc_id = None , |
|||
fields = None , |
|||
language = 'en' , |
|||
rank = None |
|||
| ) |
Initializer. Args: doc_id: The visible printable ASCII string identifying the document which does not start with '!'. Whitespace is excluded from ids. If no id is provided, the search service will provide one. fields: An iterable of Field instances representing the content of the document. language: The code of the language used in the field values. rank: The rank of this document used to specify the order in which documents are returned by search. Rank must be a non-negative integer. If not specified, the number of seconds since 1st Jan 2011 is used. Documents are returned in descending order of their rank, in absence of sorting or scoring options. Raises: TypeError: If any of the parameters have invalid types, or an unknown attribute is passed. ValueError: If any of the parameters have invalid values.
| def google.appengine.api.search.search.Document.__getitem__ | ( | self, | |
| field_name | |||
| ) |
Returns a list of all fields with the provided field name. Args: field_name: The name of the field to return. Returns: All fields with the given name, or an empty list if no field with that name exists.
| def google.appengine.api.search.search.Document.__iter__ | ( | self | ) |
Documents do not support iteration. This is provided to raise an explicit exception.
| def google.appengine.api.search.search.Document.doc_id | ( | self | ) |
Returns the document identifier.
| def google.appengine.api.search.search.Document.field | ( | self, | |
| field_name | |||
| ) |
Returns the field with the provided field name. Args: field_name: The name of the field to return. Returns: A field with the given name. Raises: ValueError: There is not exactly one field with the given name.
| def google.appengine.api.search.search.Document.fields | ( | self | ) |
Returns a list of fields of the document.
| def google.appengine.api.search.search.Document.language | ( | self | ) |
Returns the code of the language the document fields are written in.
| def google.appengine.api.search.search.Document.rank | ( | self | ) |
Returns the rank of this document.


Dear customer. Your order has been confirmed. Please come on time. If you could not make it Please ring<PHONE_NUMBER> for reschedule or cancelation.Thank you very much!


--- layout: post title: Emacs coding system for correctly copy and paste Chinese characters categories: [productivity] tags: [emacs] ---
As I'm using emacs under Windows 7, I want to correctly copy and paste Chinese characters from or to emacs. The important settings are language environment variables. Following code snippet is used to set the default language environment in English and make use of utf-8 coding system as much as possible, while leaving the clipboard coding system as utf-16le, which is the coding system used in Windows system core:
Windows作業系統核心中的字元表示為UTF-16小尾序，可以正確處理、顯示以4位元組儲存的字元。1
(set-locale-environment "English")
(set-language-environment 'English)
(prefer-coding-system 'utf-8)
(set-file-name-coding-system 'gbk)
(set-buffer-file-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(set-clipboard-coding-system 'utf-16le)
(set-w32-system-coding-system 'utf-8)
One benefit of using utf-16e while not chinese-gbk or chinese-gb18030 for clipboard coding system is that the special characters from French and German are correctly copy/paste from/to emacs. While with chinese-gbk/chinese-gb18030, only Chinese characters are encoded correct, characters such as ß, ê are not displayed as expected when copy from web browser.
Also please pay attention that the file name coding system is using gbk which is used to display file names which Chinese characters. Windows weirdly use gbk for file name coding other than utf-16le. It is a painful experience to realize this issue.


Click and drag to launch particles
×
K
Friction
Damping
Restitution
Particles
Particle Lifespan
Reset
Show Connections
Show Vectors
SMBH
Walls
Random Spawn
Particle Attraction
Vertical Gravity
Kevin Zweerink


Bootstrap container
container one
container two
container three
container one
container two
container three


Peer Quick Start¶
The easiest and quickest way to get a Subutai Peer running on any platform is to use Vagrant with VirtualBox. Don’t use package managers to install them: their versions are too old. Download the latest vagrant 2.0.1 or higher and VirtualBox 5.1.0 or higher from their respective websites. Once Vagrant and VirtualBox are installed init the Vagrantfile then up the instance:
(Here we need to copy requirements from current user guide on website.)
1.1 This command creates a VM based on Ubuntu Server 16.04.3. (Recommended):
vagrant init –minimal subutai/xenial
1.2 This command creates a VM based on Debian 9.x. :
vagrant init –minimal subutai/stretch
- After you should install special plugin for VirtualBox:
vagrant plugin install vagrant-vbguest
- When plugin is installed you need to build your peer:
vagrant up¶
Congratulations you have a Peer!¶
After peer is created you need to register it on hub.subut.ai. Please use given to you before credentials: IP address, login and password of new peer (login and password always the same: admin/secret), you will be able to change password during the registration.
- Enter given IP address into address bar of browser.
- Input login and password, and change the last one.
- The Subutai Console will be opened, on the right corner of this page
- push “Register” button and input all necessary information: your login and password on hub.subut.ai, name of peer, peer’s scope (public/private). And push “register” button.
- Your peer will appear on hub.subut.ai peers list.
You can now use it!


WITNESS LIST Finance Committee February 18, 2003 - 8:00A Article I, Bond Review Board ON: Buie, Jim Executive Director (Texas Bond Review Board), Austin, TX Roberts, Wayne Budget Director (Bond Review Board), Austin, TX Smith, Zelma Senior Analyst (Legislative Budget Board), Austin, TX Article I, Fire Fighter's Pension ON: Sandefer, Morris Commissioner (Fire Fighter's Pension), Austin, TX Smith, Zelma Senior Analyst (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Deiters, Kevin Program Director (Fire Fighter's Pension Commission), Austin, TX Johnson, Helen Executive Director (State Firemen's & Fire Marshal's Association), Austin, TX Scopez, Allen Self Employed (Texas Statewide Emergency Services Pension Fund), Richmond, TX Torres, Francisco Willacy County EMS Administrator (Texas Statewide Emergency Services Pension Fund), Raymondville, TX Article I, Pension Review Board ON: Shivers, Shari Chair (Pension Review Board), Austin, TX Smith, Virginia Executive Director (Pension Review Board), Austin, TX Smith, Zelma Senior Analyst (Legislative Budget Board), Austin, TX Article I, TX Incentive & Productiv ON: Crawford, Denise Securities Commissioner (Texas Incentive & Productivity Commission), Austin, TX Garcia, Roel Analyst (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Davis, Susan Interim Executive Director (Texas Incentive & Productivity Commission), Austin, TX Jackson, Debra Staff Services Officer II (Texas Incentive & Productivity Commission), Austin, TX Article V, Adjutant General's Depar ON: Blalock, Michael Deputy Executive Director (Adjutant General's Department), Austin, TX Borgstedte, Amy Analyst (Legislative Budget Board), Austin, TX Goodwin, Bill (Texas Army National Guard) Registering, but not testifying: On: Dehnert, Allen Assistant Adjutant General (Adjutant General's Department), Austin, TX Duffy, Claire Budget Analyst (Adjutant General's Department) Marty, M.G. Wayne (Adjutant General's Department), Austin, TX Article V, Alcoholic Beverage Commi ON: Fox, Jeannene Acting Executive Director (Alcoholic Beverage Commission), Austin Hamilton, Gregory Chief of Enforcement (Texas Alcoholic Beverage Commission), Austin, TX Royal, Alfonso Analyst (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Steen, John Lawyer (Chairman, Texas Alcoholic Beverage Commission), San Antonio, TX Article V, Commission on Fire Prote ON: Gabbert, Lori Analyst (Legislative Budget Board), Austin, TX Stalder, Kelley Presiding Officer (Texas Commission on Fire Protection), Plano, TX Warren, Gary Executive Director (Commission on Fire Protection), Austin, TX Registering, but not testifying: On: Soteriou, Jake Division Director (Texas Commission on Fire Protection), Austin, TX Article V, Commission on Jail Stand ON: Julian, Terry Executive Director (Commission on Jail Standards), Austin, TX Sheppard, Val Manager (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Houston, Vincent Chief Fiscal Officer (Texas Commission on Jail Standards), Austin, TX Article V, Commission on Private Se ON: Craig, George (Texas Commission on Private Security) Grumbles, Cliff Executive Director (Texas Commission on Private Security), Austin, TX McDonald, Kim (State Auditor's Office), Austin, TX Royal, Alfonso Analyst (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Samulin, Michael Commission Secretary (Commission on Private Security) Vander Wal, Calvin Chief Financial Officer (Texas Commission on Private Security) Article V, Criminal Justice Policy ON: Fabelo, Tony Executive Director (Criminal Justice Policy Council), Austin, TX Shepperd, Val Manager (Legislative Budget Board), Austin, TX Article V, Law Enforcement Officer ON: Dozier, Jim Executive Director (Commission on Law Enforcement Officer Standards and Education), Austin, TX Gillham, Patsy Teacher Assistant (citizens that encounter law enforcement for mental health crisis), Cypress, TX Royal, Alfonso Analyst (Legislative Budget Board), Austin, TX Smith, Dan Sheriff (Commission on Law Enforcement Officer Standards and Education), Belton, TX Registering, but not testifying: On: Laine, Edward Chief of Operations (Commission on Law Enforcement), Austin, TX Lovelace, Joe Attorney (National alliance for the Mentally Ill), Dripping Springs, TX Article V, Texas Military Facilitie ON: Borgstedte, Amy Analyst (Legislative Budget Board), Austin, TX Wells, John Executive Director (Texas Military Facilities Commission), Austin, TX Registering, but not testifying: On: Giesbers, Lydia Director of Administration/Accounting (Texas Military Facilities Commission), Austin, TX Article VI, Council on Environmenta ON: Allen, David University Professor (Council on Environmental Technology), Austin, TX Lambert, Tom Senior Analyst (Legislative Budget Board), Austin, TX Registering, but not testifying: On: Matthews, Jerry Executive Director (Texas Council on Environmental Technology), Austin, TX Article VI, Soil & Water Conservati ON: Lambert, Tom Senior Analyst (Legislative Budget Board), Austin, TX Merz, Evelyn Systems - Training (Native Prairies Association of Texas, Treasurer) Moore, James Executive Director (Texas Soil & Water Conservation Board), Temple, TX Registering, but not testifying: On: Register, T. Wayne Chairman (State Soil & Water Conservation Board), Houston, TX Stephens, Bobbie Director of Administration (Texas Soil & Water Conservation Board), Temple, TX Article VI, Texas River Compact Com ON: Chenoweth, Todd Manager, Comm. on Environmental Quality (River Compact Commission), Austin, TX Gagnon, Gary Business Services (Sabine River Compact Commission), Orange, TX Lambert, Tom Senior Analyst (Legislative Budget Board), Austin, TX Thrasher, J.W. Commissioner (Pecos River Compact Commission), Monahans, TX


Settings of retry function.
A function that should be called.
An object that should be used as this
when calling the action function.
An array of parameters that should be passed into the action function.
An amount of milliseconds before first call of the action function. When the value is not specified or is negative, the action function will be called immediately first time.
An array specifying amount and timeouts between repeated calls of the action function.
Each item can be a number or a function (see retryTimeout
setting for details).
Has priority over retryQty
and retryTimeout
settings.
A boolean value or a function returning boolean value that specifies whether the action function should be called again when the action function throws an error or returned promise is rejected. When not specified the call of the action function will not be repeated on an error.
Maximum number of repeated calls of the action function. A negative value means no restriction.
Default value is -1
.
A boolean value or a function returning boolean value that specifies whether the action function should be called again after a made call. When not specified the action call will not be repeated.
A timeout between repeated calls of the action function, or a function that returns such timeout.
A negative or non-number value means the repeat call will be made without delay (this is applied by default).
If specified function returns false
then retry process will be finished and result promise
will be fulfilled or rejected depending on result of the last action's call.
Time in milliseconds specifying how long retry process can last
starting from call of retry
function.
Elapsed time is checked before each retry attempt and when the time exceeds the given limit
process will be finished and result promise will be fulfilled or rejected
depending on result of the last action's call.
0
or negative value means no limit.
Generated using TypeDoc
Settings of retry function.


TANGO LESSON, THE (director/writer: Sally Potter; cinematographer: Robby Müller; editor: Herve Schneid; cast: Sally Potter (Sally), Pablo Veron (Pablo), Gustavo Naveira (Gustavo), Fabian Salas (Fabian), David Toole (Fashion Designer), Peter Eyre (English Tango Fan), Olga Besio (Olga), Carlos Copello (Carlos), Carolina Iotti (Pablo's Partner); Runtime: 102; Sony Classics; 1997-France /Argentina / UK)
Reviewed by Dennis Schwartz
Somehow this quirky, egocentric, fantasy black-and-white film about a British filmmaker learning to dance the tango from a master, works in an obscure way by getting us hooked on what's going on. It is set in Paris, London, and Buenos Aires, and it might or might not be autobiographical of director-writer-songwriter-actress Sally Potter ("Orlando"). It works so well because the dancing was so natural and magnificently performed, even if the wistful romantic story never caught my imagination and the philosophizing about her being a Jew seemed more diverting than engrossing (in real-life she's not Jewish but an avowed atheist who was only influenced by her Jewish neighbors and their culture).
Potter was trained as a dancer and choreographer in London in the 1970s and in this film composes her original song compositions, which are quite good.
The 48-year-old Sally Potter (she plays herself), while filming in Paris a Technicolor thriller (it seemed like dreck and is probably some kind of personal statement Potter is making about Hollywood films) about a legless fashion designer in a wheelchair murdering models dressed in brightly colored costumes. She will take a break from filming and attend a theatrical dance exhibition by the great Argentine tango master Pablo Veron (he plays himself) and afterwards she decides to get tango lessons from him. "I've always wanted to be in films," says the thirtysomething Pablo. She counters, "I've always wanted to be a dancer." It sounds like a trade-off in self-indulgences, and all I can say, is why not! When Sally dances the tango in the film, that's really her and not a double. And if she can dance the part, why not play the part!
For whatever arcane reason (or most likely no reason at all), the story has twelve dance lessons which come to divide the film into chapters. The lessons are not necessarily lessons, but show her as she becomes obsessed with the dance, with the subtleties of the music, and with her teacher. She returns to Paris after being disappointed that her Hollywood backers complained that it would be better for the box office if the film was to be in English instead of French and that she should cast a different villain because it would be difficult to cast a legless person. She decides to abandon her film project and make a film instead about the tango. She takes further lessons in Buenos Aires from friends of Pablo's and becomes good enough to perform on stage with him when she returns to Paris.
Their romance and friendship becomes a battle of wills, which mirrors the way they dance. After they appear at a Paris theater, he critiques her dancing by stating that he wants her to follow, to completely surrender to him, complaining that she impedes his freedom.
Somehow I couldn't be convinced of their romance, in fact, I never cared about it. It seemed empty, and to the bargain she's no beauty. But...ah! those dancing scenes were tremendous, along with the penetrating cinematography of the great Robby Müller, giving the film such a stunning look. The film managed to look good and even though Pablo and Sally are no great shakes as actors, and their stiff romance was more pitiful than meaningful, the film had a bizarre strain to it, of this very uptight Sally learning how to loosen up and the sly Pablo jumping around like a fox at a hunt. It seemed to be a mix of reality and script, but hard to tell which was which.
 He has never been comfortable with his Jewish roots and considers himself just a dancer. In his spare time, he is seen reading a biography on Marlon Brando (get it--he was in "The Last Tango in Paris"). She is seen reading Martin Buber's 1923 philosophy book "I and Thou," which she believes unlocks the spiritual meanings of the tango, as Buber's scholarly work delves into relationships: "The soul is not really united unless all the bodily energies, all the limbs of the body, are united."
The film tries to make some connection between love and art and who is to be the follower and who the artist. It asks the spurious question, can two artists collaborate on any work, if they both must lead. I couldn't quite warm up to this debate about art.
The film is short on plot, short on acting, short on getting me involved in the romance, but most of the film had Potter and Veron dancing and that had me jumping with joy, even believing that there was something spiritual to the tango, a dance which is a mixture of Cuban and European styles, created by the lower-classes and elevated to the status of the ballroom, a dance created in this century, that maybe is even akin to the rapturous Eastern European klezmer music.
This is an inspiring dance film that covers some ground in exploring what the tango is and how that dance reflect's one's life and suffering, and to understand that is to begin to understand what the tango is about and what this film was trying to get at.
REVIEWED ON 11/28/2000 GRADE: B
Dennis Schwartz: "Ozus' World Movie Reviews"
http://www.sover.net/~ozus
<EMAIL_ADDRESS>© ALL RIGHTS RESERVED DENNIS SCHWARTZ
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


The groovy-dateutil
module supports numerous extensions for working with
Java’s classic Date
and Calendar
classes.
You can access the properties of a Date
or Calendar
using the normal array index notation
with the constant field numbers from the Calendar
class as shown in the following example:
import static java.util.Calendar.* (1)
def cal = Calendar.instance
cal[YEAR] = 2000 (2)
cal[MONTH] = JANUARY (2)
cal[DAY_OF_MONTH] = 1 (2)
assert cal[DAY_OF_WEEK] == SATURDAY (3)
| 1 | Import the constants |
| 2 | Setting the calendar’s year, month and day of month |
| 3 | Accessing the calendar’s day of week |
Groovy supports arithmetic on and iteration between Date
and Calendar
instances as shown in the following example:
def utc = TimeZone.getTimeZone('UTC')
Date date = Date.parse("yyyy-MM-dd HH:mm", "2010-05-23 09:01", utc)
def prev = date - 1
def next = date + 1
def diffInDays = next - prev
assert diffInDays == 2
int count = 0
prev.upto(next) { count++ }
assert count == 3
You can parse strings into dates and output dates into formatted strings:
def orig = '2000-01-01'
def newYear = Date.parse('yyyy-MM-dd', orig)
assert newYear[DAY_OF_WEEK] == SATURDAY
assert newYear.format('yyyy-MM-dd') == orig
assert newYear.format('dd/MM/yyyy') == '01/01/2000'
You can also create a new Date or Calendar based on an existing one:
def newYear = Date.parse('yyyy-MM-dd', '2000-01-01')
def newYearsEve = newYear.copyWith(
year: 1999,
month: DECEMBER,
dayOfMonth: 31
)
assert newYearsEve[DAY_OF_WEEK] == FRIDAY


The State of Javascript
Scott Sauyet
An Overview of Javascript
Where we've been
We're we are now
Where we're going
Where We've Been
Javascript Early History in a Single Slide
- Created in 1995 to offer a light-weight scripting language in Netscape 2.0
- The inventor was inspired by two very different languages, Self and Scheme
- Copied feature-for-feature, bug-for-bug by Microsoft
- Standardized before the kinks were worked out, named ECMAScript
- After two minor revisions, stood still for 10 years
- Looked down on for most of its existence as a toy language suitable only for hobbyists
- Got another look when Google started using it for impressive tools like GMail and Google Maps
- Gained more attention when someone coined "AJAX"
- Got some real respect when a respected Yahoo engineer started promoting it
Where We are Now
There are four major themes right now:
- The explosive growth of mobile
- The rapid adoption of server-side Javascript
- The number of tools using Javascript as a compilation target
- The huge variety of libraries being released for developing with Javascript
Javascript on the Go
Javascript on more devices than any other computing runtime... and has been for many years. It's only getting
more so with the growth of mobile devices.
Flash is disappearing. Applets are dead. JavaFX never got any traction. Silverlight's been
abandoned. There simply are no competitors standing except for the native device environments.
Not Just for The Client Anymore
Server-side Javascript has been around for years
- Bundled with early Netscape as a server component
- Included as part of .NET
- Rhino Java implementation
But none of that mattered much. Until...
Node.js
Created in 2009, based on the V8 Javscript engine behind Google Chrome, this has rapidly become a powerful
force in server-side development. Node features:
- non-blocking IO
- simple package and includes mechanism (no DLL/JAR hell)
- extremely high concurrency
- a very active community
Compiling to Javascript
A list maintained by altJS shows new languages that can be converted to JS
- Caffeine
- CoffeeScript
- CoffeeScript
- Coco
- Contracts.coffee
- GorillaScript
- Ham-Script
- heap.coffee
- IcedCoffeeScript
- Jack
- JMacro
- JS11
- Kaffeine
- LittleSmallscript
- LispyScript
- LiveScript
- LLJS
- Moescript
- Move
- pogoscript
- Sibilant
- ToffeeScript
- Parsec
- Uberscript
- Zedscript
Compiling to Javascript
The list of languages ported from existing languages is much longer
Ruby
- 8ball
- ColdRuby
- HotRuby
- Opal
- rb2js
- Red
- Quby
Ruby-related Libraries
Python
- Brython
- PYXC-PJ
- Pyjaco
- Pyjamas
- Pyjs
- PyCow
- PyvaScript
- RapydScript
- Skulpt
Perl
Java and JVM
- BicaVM
- Ceylon
- Doppio
- GrooScript
- GWT
- j2js
- Java2Script
- Kotlin
- Processing
- STJS
Scala
C#, F#, .NET related languages
- Apps in Motion
- Blade
- FunScript
- jsc
- JSIL
- NemerleWeb
- Pit
- Prefix
- Saltarelle
- SharpKit
- Script#
- WebSharper
Compiling to Javascript
Ported Languages (continued)
Lisp, Clojure & Scheme
- BiwaScheme
- Chlorinejs
- ClojureScript
- clojurejs
- EdgeLisp
- Fargo
- LispScript
- Moby
- nconc
- Oppo
- Outlet
- Parenscript
- Ralph
- scheme2js
- Scriptjure
- Spock
- Whalesong
OCaml/ML
- Ocamljs
- O'Browser
- Js_of_ocaml
- SMLtoJs
Haskell
- Fay
- ghcjs
- Haste
- jshaskell
- JSHC
- UHC
- YHC
Smalltalk
- Amber
- Clamato
- Lively Kernel
- Silver Smalltalk
C/C++
- Emscripten
- mala
- Mandreel
- Clue
BASIC
- NS Basic/App Studio
- qb.js
Pascal
- Smart Mobile Studio
- Elevate Web Builder
Go
Compiling to Javascript
Those lists are still only part of the story. altjs lists around 100 other
tools for
- Javascript Enhancements
- Security Enforcement
- Static Typing
- Synchronous to Asyncrhronous (CPS)
- Compiler Tools such as
- Parsers and Extenstions
- Optimizers
- Parser Generators
- Abstract Syntax Tree (AST) creators/manipulators
- Miscellaneous Other Related Projects
Library Development
There has been a recent change in how libraries are developed.
2000 - 2008: General-purpose libraries
- jQuery, Dojo, MooTools, YUI, etc
- Very large libraries
- Hard to get right (browsers are finicky)
- Slow to change
2009 - present: Smaller, more focused libraries
- For every conceivable purpose, but each one focused
- Nimble
- Easily interoperable on the server
- (Getting there on the client)
This has allowed for a huge number of alternative approaches, and a great deal of experimentation.
Where we're Headed
- Several
different
Javascript microcontrollers now available
- Several Javascript subsets for high-performance programming
- No slow-down in the performance improvements we've been seeing
- Isomorphic Javascript: same exact code client/server
- Multiple paradigms for programming: OO, functional, possibly even logic programming in one language
- Who knows?
←
→
/
#


Full Stack software developer with Python and Javascript languages. Knowledge using Django, REACT. Interested in working in the automotive and gaming industry


class Solution: def checkPartitioning(self, s: str) -> bool: n = len(s) dp = [[False for _ in range(n)] for _ in range(n)] for i in range(n): dp[i][i] = True for l in range(2, n+1): for i in range(n-l+1): j = i + l -1 if l == 2: dp[i][j] = s[i] == s[j] else: dp[i][j] = (s[i] == s[j]) and (dp[i+1][j-1]) for i in range(n-2): for j in range(i+1, n-1): if dp[0][i] and dp[i+1][j] and dp[j+1][n-1]: return True return False


Unified Research
Support Assistant
Support Assistant
Open Ursa
(Using Google)
@(title: String, head: Html, footer: Html)
(Using Google)


 I had been an admirer of Stanley Kubrick for twenty years, and in that time had only twice been able to experience a new Kubrick release, so it was good to hear of a new film starting production. It was nevertheless a bit of a disappointment that Kubrick's next film would not be his much anticipated project A.I., but would instead be a drama starring Tom Cruise and Nicole Kidman.
For three and a half years, the Internet has been awash with speculation and rumour about the cast and content of Eyes Wide Shut. Cast replacements and secretive and protracted principal photography fuelled the gossip, much of it outrageous, and nearly all of it incorrect. Finally the film itself here, and in the most tragic of circumstances. This is the last time we will ever see a new film from Stanly Kubrick. He was a legendary figure - one of the few truly great film makers - and a much misunderstood man. And now he's gone.
In Eyes Wide Shut, Tom Cruise plays New York doctor Bill Harford and Nicole Kidman plays his wife Alice. Their marriage seems perfect, but one night Alice admits to having had sexual fantasies about another man. Bill is deeply disturbed by this revelation and in his jealousy and desire for revenge enters a sexual underworld, the heart of which is a masked ritual orgy that takes place in a country mansion and at which things turn very dark for Bill and very weird for the viewer.
Is Eyes Wide Shut the fitting end to a great career that some reviewers have claimed, or the monumental turkey that others have reported? I think it depends on whether you get Kubrick films or not. Some are put off by the feeling of cold detachment in many Kubrick films - we often don't empathise with the characters or feel we are in the presence of real people, but instead we observe the characters as subjects on a microsope slide. This clinical formality can either be deeply off- putting or can be the price you willingly pay for letting Kubrick in to mess with your head. Personally, I'm willing to go along for the ride. After first seeing Eyes Wide Shut, I wasn't sure what I thought of it. Now it's had a chance to bed down a little, and I've seen it again, I think it is marvellous, but overlong. The film is full of subtleties of plot and theme that endlessly turn over in the mind. Questions are raised in the film and apparently answered, but ambiguities remain, and multiple interpretations are possible. A second viewing raises as many questions as it answers.
The centrepiece of the film is the masked orgy sequence. From the moment Bill arrives at the mansion gates to the moment he leaves, this 20-minute sequence is as good as anything Kubrick has ever done. The interplay of image and music to evoke mystery, eroticism and fear is spellbinding (although occasionally I had to wonder how satisfactory oral sex can be when the participants are wearing masks). The American release of Eyes Wide Shut was altered by placing computer-generated figures over certain of the more vigorously copulating orgy-goers. This was done to get the film an 'R' rating, rather than the more appropriate but commercially difficult 'NC-17' rating. This has become known as the "Austin Powers" version of Eyes Wide Shut. The digital cover-ups were, apparently, approved in principle by Kubrick before his death. In the U.S., hardcore porn is freely available, yet commercial considerations mean that adults cannot see a serious mainstream film featuring simulated sex in which no genitals are visible but which includes an unacceptably vigourous level of thrusting. What a strange country America is.
The film has an extraordinary visual quality.  The film was deliberately underexposed during shooting and overexposed during developing to compensate. This produces a dream-like washed-out grainy feel to the image. At times the scene seems alive with an orange glow, and during some close-ups, the grain is so prominent that the image seems to shimmer. (On a more practical level, low lighting levels also means a relatively small crew, so daily rates can be kept low, thereby allowing longer shooting schedules.) The look of the film is also influenced by a recurring motif of christmas tree lights and many other examples of similar tiny lights, such as in shop displays and cafes, and in curtains of beaded light.
Although there are few of the obvious trademarks, such as dramatic tracking shots through interior spaces, or the full-face shots of characters on the edge of insanity, the film still has the distinctive Kubrick feel. Interiors are framed and lit to provide not just a place where the action is set, but a psychological location for the characters. Colours scream out at you with significance. What that significance is it is hard to say, but perhaps any attempt to put it into words is to miss the point. Kubrick films work at a deep subconscious level, creating moods and reactions in the viewer that are not meant to be explained or understood.
As is usual with Kubrick films, the choice of music is eclectic and mixes original compositions with existing recordings. The original pieces are by Jocelyn Pook, whose sombre chamber music is central to several parts of the film, including the masked orgy sequence. Kubrick has again made use of the music of avant-garde composer Gyorgy Ligeti. The use of Ligeti in 2001 was spot on, but the use of his piano music in Eyes Wide Shut has come in for some criticism. To the untrained ear, such as mine, it sounds like little more than a child playing individual notes with little concern for melody or rhythm. Listening to the music in isolation, it is austere and foreboding, but in the context of the film, that damned piano music struck me at times as just plain ridiculous and nearly destroyed what it sought to augment.
Another noteworthy aspect of Kubrick films is the performances he gets from his actors. Tom Cruise is not an actor that seemed a likely choice for a Stanley Kubrick film, having neither the emotional intensity of a star like Jack Nicholson, nor the relative anonymity of the cast of 2001 or Full Metal Jacket. The casting of Cruise is reminiscent of that of Ryan O'Neal in Barry Lyndon. In defending his casting of O'Neal, Kubrick said in one of his rare interviews that "the personal qualities of an actor, as they relate to the role, are almost as important as his ability". Presumably similar considerations lay behind the casting of Cruise, who carries the main acting burden, appearing in virtually every scene. He is nevertheless hamstrung by his narrow range. Nicole Kidman is generally fine, but even she sometimes seemed to chew the scenery a bit. The minor roles are the usual collection of Kubrick eccentrics. None of this is meant to be critical - it's the peculiar nature of the performances, even those that would be deemed stilted or just plain bad in other films, that contribute to the disconcerting other-worldly appeal of Kubrick films. Sydney Pollack's performance as Bill's friend Ziegler does, however, seem slightly out-of-place, his being the sort of naturalistic performance we'd expect in a normal film.
The pacing of the film is often painfully slow. Scenes are dragged out and lines... are... often... delivered... with... great... big... gaps... between... words. Kubrick has been known to trim slow sequences after critical test audience reaction (he did it with 2001 and trimmed the European release of The Shining) and it is interesting to speculate whether he would have similarly tightened-up Eyes Wide Shut if he had had the chance. With his death, the film became untouchable.  It is magnificent, but occasionally a magnificent bore.
-- Gary Jones PGP public key available from servers (DH/DSS key ID: 0x11EAE903) Home page: www.bohr.demon.co.uk
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


The first while loop statement at line 4 contains two expressions, separated by the
short-circuit operator &&. As a rule, the short-circuit operator, &&,
will not evaluate
the second expression – if the first expression of a while loop statement
evaluates to false (as in this case).
Consequently, the first while loop (at lines 4 thru 5) fails to execute (i.e. it
never iterates).
The behavior of the expression thats associated with the second while loop statement (at line 6) modifies the value
of the variable i (via a
post-increment operator) after the expression is evaluated.
This repeats continually, until the value of i causes the while loop expression
to evaluate to false (halting the while loop's ability to repeat). At that point,
the code flow moves immediately to the print statement at line 7; which outputs the last value that was
incremented by the post-increment operator
(i.e. 14) before the while loop statement, at line 6, evaluated to false and stopped
repeating.
Following the output from the print statement, the program terminates.


Table Of Contents:
- Install Elasticsearch: Local & AWS
- Create Index and Mappings
- Use Elastichead
- Insert single log to elasticsearch
- Use JQ to do bulk insert
- Use MLT query perform Lexical Recommendations
- Next Steps
Install Elasticsearch
Install on Local: Docker
If you would like to install Elasticsearch locally you can follow these instructions Install: Docker + Elasticsearch
For this tutorial I will be using local instance of ES cluster with a single node, it saves us money as ES cluster on AWS is slightly
expensive.
Create ES Instance on AWS
If you would like to create an ES(Elasticsearch) cluster on AWS you should be able to follow below steps.
Search ES service on AWS.
Click on Create a new domain
Choose deployment type
Configure domain
Review all of the settings. (Screenshot is cut-off)
VPC endpoint below is the domain that we can use to perform ES actions. It is comparable to localhost:9200, when we run ES on localhost.
Get the status of our ES index.
If your cluster is deployed on AWS you will see different build types compare to below which says docker
.
Create Index and Mappings
We can create new index named analytics using below command.
Here is the Put request body:
If the command is successful, you should see response similar to below.
List all the indexes.
Use Elastichead
There are several plugins or apps you can use to explore Elasticsearch Data including Kibana and Elastichead etch. For the purpose of
this tutorial we will use Elastichead as it’s a simple Chrome Plugin.
We can see our newly created analytics
index show below. We will go through a lot more tips & tricks to search documents and do
aggregations etc.
Insert single item to Elasticsearch
Use JQ to do bulk insert
JQ is one of the most useful tools if you ever want to modify JSON objects. I stumbled upon a blog post
from Kevin Marsh. We will use JQ to bulk upload the
data to Elasticsearch.
Next Steps
In the next tutorial we will extend our similarity search to use Semantics to find similar documents in addition to just using Lexical
strategy.
comments powered by


Jack Kim is a Stanford student from Seattle who currently resides in Arroyo House, Wilbur Hall. He's a firm entrepreneur with clear visions and a drive to succeed.