This note largely documents the design and syntax of CRDS certify .tpn constraint definition files. But as an overview, CRDS certify performs these forms of checks:
1. .tpn defined constraint checks unique to CRDS 2. spec defined unique table row checks, missing mode checks 3. fitsverify checks, recategorized as needed 4. astropy.io.fits, ASDF, JSON, YAML format checks 5. JWST data models open() checks 6. JWST data models scraped value checks
.tpn files are used by CRDS certify to define checks on reference file header keywords and array properties.
For HST .tpn files define almost all CRDS checks (some table checks also exist) and were incorporated verbatim from CDBS certify to simplify porting a moving target.
For JWST, CRDS certify .tpn checks are used to extend and augment the checks performed by the data models, e.g. adding the notion of "required" when specified or implied, checking array dimensions, checking keyword interrelationships, etc. CRDS .tpn syntax was significantly extended from HST to support additional forms and organizations of checks as well as the original checks.
For HST, .tpn files were written for every combination of <instrument>_<type>.
There are two forms of .tpn, one which constraints reference file properties (.tpn) and one which constraints rmap properties (_ld.tpn). The _ld.tpn files originally constrained the expanded CDBS database matching parameter values. While limiting the file structure to <instrument>_<type> combinations was simple, there was also an extreme penalty for redundantly specified checks of common constraints like PEDIGREE or USEAFTER.
For JWST, additional broader classes of .tpn are also defined and all applicable forms are loaded for any given reference file:
all _ all .tpn (constraints on all instruments and types) <instrument> _ all .tpn (constraints on all types of one instrument) all _ <type> .tpn (constraints on one type of all instruments) <instrument> _ <type> .tpn (constraint on one instrument and type)
For JWST, the additional file classes permit generalization of constraints without added redundancy. PEDIGREE can defined once in all_all, etc.
Typical constraint directives result in a CRDS TpnInfo() object being defined in CRDS certify, which corresponds 1:1 to a CRDS Validator() object/subclass. The TpnInfo() is a bundle of constraint information, a Validator() is a class which checks the constraint defined by a TpnInfo. Ultimately the outcome of .tpn files is a list of TpnInfo() constraints to check, which are tuples of properties, interpreted via appropriate Validator() subclassses.
The CRDS certifier uses the JWST data models in two ways:
1. CRDS certify calls datamodels.open() on each reference to directly apply data models checks. This is very simple and fairly robust, but suffers from the dependency of open() on the DATAMODL keyword which is used to specify the model class of the reference file. The model class implies the exact schema to check. If DATAMODL is not specified, open() won't validate specialized schema.
2. Second, CRDS loads the data models core schema and converts them into synthetic TpnInfo() objects which don't appear in a .tpn file. These synthetic constraints are interpreted by CRDS normally to make that checking visible and also to permit automatically promoting the constraints from "optional" (check if present) to "definintely required" where keywords are needed for CRDS matching. The synthetic TpnInfo() objects generally scrape value enumerations from the core schema for use in CRDS.
.tpn files contain several forms of directives: include, replace, and constraint.
The semantics of 'include' are roughly model'ed after the C pre-processor's #include directives.
The include directive permits one .tpn file to include the text of another as if the directive was replaced by the contents of the included file. This permits factoring out common constraints into a single file and using them in multiple locations, e.g.:
include nir_sci_array.tpn
defines constraints on the SCI array for all NIR instruments except NIRSPEC.
For the sake of simplicity, the line of the directive should begin with 'include' (unquoted), a single space, and the name of the included .tpn file (also unquoted). Like many CRDS directives it is space delimited and consists of exactly two words, written on a single line.
An included file should also be located in the .tpn includes directory and generally will not follow the JWST classes of include files (all_all, fgs_all, etc.) or their naming conventions.
The replace directive causes CRDS to do substitutions on all subsequent lines in the .tpn file, even lines included from other .tpn files. The replace directive consists of 3 words in the general form of:
replace <original_pattern> <replacement>
where an example directive is:
replace SCI COEFFS
which means that all subsequent instances of the pattern SCI in a particular .tpn (or included .tpns) should be replaced by COEFFS. In this case, it effectively rewrites constraints on the SCI array as constraints on the COEFFS array.
An important limitation of 'replace' to note is that it only applies within the textual extent of on particular file class. At this time, it's not possible to e.g. define a replacement in all_all and have it apply everywhere.
By far the most common directive is the constraint directive, which defines one condition on a reference file and is of the form:
<name> <keytype> <datatype> <presence> <value constraint>
or sometimes with the values omitted:
<name> <keytype> <datatype> <presence>
Before explaining each field in detail, a typical (but abbreviated) example constraint taken from JWST is:
META.INSTRUMENT.PUPIL H C O CLEAR,CLEARP,F090W,F115W,F140M,F150W,F158M,\ F162M,F164N,F200W,F323N,F405N,F466N
The columns of the constraint are interpreted as follows:
Since whitespace is used to delimit fields within a constraint, spaces / whitespace MAY NOT appear within any single field, i.e. the value list, presence expressions, constraint expressions, etc. This can be slightly awkward at times but the addition of extra parentheses to existing punctuation is generally sufficient to write expressions containing no spaces.
For example, even expressions such as:
("IFU" not in EXP_TYPE)
which contain instrinsic whitespace can often be rephrased in a workable way as:
(not(("IFU")in(EXP_TYPE)))
A limitation of this approach is that literal strings containing white space are not permitted/straightforward in expressions. In that area, writing additional helper functions or custom validators might provide a way out.
While the idea of modernizing .tpn syntax is pretty obvious, the downsides of switching to more readable file formats like JSON or YAML are a combination of:
* Up-front work * Additional testing for multiple projects * Constraints which become more verbose and less dense.
Since constraints are easier to read and consider en masse when they're expressed in a dense format, not even readability is a complete slam dunk as a motivation for modernizing formats.
The name field specifies the name of a header keyword, table column, array, or expression constraint.
Header and table keyword names correspond roughly to FITS keywords or JWST CAL data models paths flattened into a single string, e.g.:
READPATT META.EXPOSURE.READPATT Array Names +++++++++++
Array names are specified as the bare HDU name in the <name> field, e.g. SCI. These are referenced within expressions as <name>_ARRAY. These are case insensitive and specified in all capital letters, numbers, or underscores much like FITS keywords. They should begin with a letter and be valid program identifiers.
There are two additional specification cases for array names:
Expression constraint names describe the check performed by the value expression, they do not describe any physical entity within the reference file. Note that expression here refers to a keytype=X constraint and have no relation to expressions used in the <presence> field described below.
The keytype field consists of one character corresponding to:
keytypes = { "H" : "HEADER", "C" : "COLUMN", "G" : "GROUP", "A" : "ARRAY_FORMAT", "D" : "ARRAY_DATA", "X" : "EXPRESSION", }
Header keyword names correspond to values taken from the union of all HDU headers.
Almost all of the HST constraints taken from CDBS are enumerations applying to a single FITS or GEIS keyword, e.g. READPATT.
Many JWST constraints are written using the format independent (FITS, ASDF, JSON...) data model hiearchical path names munged for CRDS purposes into all capital letters with periods replaced by underscores so that they can be evaluated as a single keyword name rather than as nested objects.
Column names generally apply to the name of a FITS table column and the corresponding constraint applies only to the values of that single column in isolation.
Array format constraints apply to lightweight array properties taken from FITS HDU data:
utils.Struct( SHAPE = hdu.data.shape, KIND = generic_class, DATA_TYPE = typespec, COLUMN_NAMES = column_names, EXTENSION = i, DATA = None, )
Most notably, the array data itself is not available for constraint checking but the lightweight properties are relatively fast and small to load.
Generally, array format and data keytypes have expression constraints rather than value enumerations, ranges, etc. Most commonly expressions limit the array shape and type.
Array expressions can be written in terms of all arrays for which constraints are defined. So an ERR array constraint might also refer to SCI if it was known to be loaded elsewhere.
Array data checks are heavy weight and entail loading the actual reference data so that constraints can be applied to it:
utils.Struct( SHAPE = hdu.data.shape, KIND = generic_class, DATA_TYPE = typespec, COLUMN_NAMES = column_names, EXTENSION = i, DATA = hdu.data # XXX the difference between 'A' and 'D' constraints! )
Generally, array format and data keytypes have expression constraints rather than value enumerations, ranges, etc. Most commonly expressions limit the array shape and type.
Expressions replace the typical value enumeration, range, etc. with a Python expression written in terms of the reference file header and array properties. While A and D array constraints are also generally written as as expressions, in contrast, an X constraint loads no new array properties and includes no arrays. The value expression should be written in terms of header keywords only. Arrays are pre-loaded and remain available to all expressions for the duration of a single reference file check.
Not implemented but parsed for the sake of HST CDBS backward compatibility.
The datatype field conceptually corresponds to the type of a FITS keyword defined in the reference file header or table. Similar properties are imposed on data models paths/keywords which may or may not correspond to a FITS keyword.
The datatype is written as a single character with these translations:
datatypes = { "C" : "CHARACTER", "I" : "INTEGER", "L" : "LOGICAL", "R" : "REAL", # float32 value(s) "D" : "DOUBLE", # float64 value(s) "X" : "EXPRESSION", # constraint expression expected }
The X datatype indicates that the constraint will be a boolean expression and hence has no data type; it is abstract, referring to no particular keyword or array by definition... although frequently expressions are used to check type.
The presence field determines the conditions under which a constraint applies and what should happen when it is omitted:
presences = { "E" : "EXCLUDED", "R" : "REQUIRED", "P" : "REQUIRED", "W" : "WARN", "O" : "OPTIONAL", "F" : "IF_FULL_FRAME", "S" : "IF_SUBARRAY", "A" : "ANY_SUBARRAY" }
Simple presence values are specified as a single character which correspond to these classifications:
REQUIRED or True results in an error if the keyword is not present in the file header or tables or is UNDEFINED or the constraint is not satisfied.
False means a constraint does not apply.
WARN results in a warning if the keyword is not present or is UNDEFINED.
OPTIONAL indicates that a constraint should be satisfied if the keyword is present and not UNDEFINED but is not an error when omitted.
IF_FULL_FRAME means that the constraint only applies when SUBARRAY keywords are defined (SUBARRAY,SUBSTRT1,SUBSTRT2,SUBSIZE1,SUBSIZE2) and SUBARRAY describes a full frame (FULL,GENERIC,N/A,ANY,*).
IF_SUBARRAY means that the constraint only applies when SUBARRAY keywords are defined and SUBARRAY does not describe a full frame.
ANY_SUBARRAY means that the constraint only applies when SUBARRAY keywords are defined.
EXCLUDED means that a keyword should not be specified and was supplied for backwards compatibility with HST CDBS and is generally unused.
For HST, every instrument and type specified the presence requirement for every keyword. This resulted in value enumerations repeated over and over throughout the .tpn files.
For JWST, CRDS support specifying keywords as optional... with one twist: if an optional keyword is used by an rmap to perform matching (appears in the 'parkey' header field), then every optional constraint on that keyword for that particular reftype becomes required.
This permits constraints to be specified once as optional at a relatively global level for easier maintenance, but then become "required" if a particular reftype uses the keyword directly within CRDS for matching. (This is a reflection of the "prime directive" of the CRDS certifier: while general checks can be implemented, the most crucial aspect of CRDS checking is to ensure that files work within CRDS. Although CRDS does strive to implement additional checks, the only real measure that references will work with the CAL code is running calibrations.)
For even more control, or for keywords not used by CRDS matching, additional constraints can be defined in more specialized .tpn's.
A Python expression can be specified to define when a constraint does or doesn't apply based on keyword values.
The expression should begin with ( and end with ) and should contain no spaces. (Sometimes extra parens are required to break up the expression into words using punctuation instead of spaces.)
An example of a presence expression is:
(EXP_TYPE!='FGS_ID-STACK')
which means that the constraint only applies when EXP_TYPE is not FGS_ID-STACK.
Keyword names used in presence expressions follow the usual rules and must be valid Python identifiers in all caps. Periods from data model paths are replaced by underscores to make the paths into simple identifiers suitable for Python's eval().
Presence helpers have been defined to convert the boolean result of a presence expression into a simple presence value. This enables conditional optional keywords, conditional warnings, conditional subarray expressions, etc:
optional(expr) --> False or 'O' full_frame(expr) --> False or 'F' subarray(expr) --> False or 'S' any_subarray(expr) --> False or 'A' required(expr) --> False or 'R' warn(expr) --> False or 'W'
For example, an expression further refined by the full_frame() helper:
(full_frame(EXP_TYPE!='FGS_ID-STACK'))
means that fundamentally, it only implies when EXP_TYPE is not FGS_ID-STACK, but in addition, it only applies when SUBARRAY keywords are defined and the SUBARRAY is some form of full frame, e.g.FULL or GENERIC. In effect, the helper arranges things so that the presence field is 'F' if the wrapped expression is satisfied.
Note that an expression return value of False indicates a constraint does not apply at all. An expression return value of True indicates the constraint is REQUIRED.
Helper functions in .tpn files are distinguished by being written in all lower case; this prevents collisions with keyword, column, or array names which are always written in upper case.
The <values> field of each constraint can define a number of things, including enumerations of literal values:
GUIDER1,GUIDER2
numerical ranges:
1:10
constraint expressions:
(not("IRS2")in(READPATT))
custom validator identifiers:
&PEDIGREE
or nothing at all.
Value enumerations list the possible literal values that can be assigned to a keyword, e.g.:
FGS,NIRCAM,NIRISS,NIRSPEC,MIRI,SYSTEM
Ranges specify inclusive numerical ranges which keyword values must lie within, e.g.:
1.0:10.0
means the value should be within 1 and 10 inclusive. An equivalent expression constraint would be:
(1.0<=KEYWORD<=10.0)
where KEYWORD is the name of the constrained keyword.
Custom constraint handlers define new classes of validators and are always specified by a value / validator name beginning with &, e.g.:
META.USEAFTER H C R &JWSTDATE
where validator values have meanings like:
&PEDIGREE -- implements algorithm to check various PEDIGREE value forms &USEAFTER -- implements HST USEAFTER date/time format checking &JWSTDATE -- implements JWST date/time format checking, e.g. JWST USEAFTER
Custom constraint validators can perform arbitrary processing to validate a single keyword value, i.e. specify precise date formats, etc. Custom constraint validators are defined in the crds.certify.validators module with classes named like e.g. PedigreeValidator, UseafterValidator, JwstdateValidator.
Unlike presence expressions which define when a constraint should or should not be applied, expressions constraints define the condition which should be satisfied when the constraint is applicable.
Someone might briefly wonder if both presence and constraint value expressions are needed. The answer is "yes" because a negative result of a value expression is limited to "constraint failed" while a negative result for the presence expression is limited to "do not evaluate", so the concerns truly are separate and two expressions are needed.
Constraint expressions always begin with '(' and end with ')' and should contain no spaces.
An example expression constraint is:
(1<=META_SUBARRAY_XSTART+META_SUBARRAY_XSIZE-1<=2048)
which asserts that XSTART + XSIZE - 1 should fall within the boundaries of the detector's 2048 X-dimension.
When specified within CRDS .tpn files, JWST CAL data models paths (ie. keyword names) are flattened to simple strings that resemble FITS keywords in all upper case:
meta.subarray.xstart --> META.SUBARRAY.XSTART
Within expressions, the periods are replaced with underscores:
META.SUBARRAY.XSTART --> META_SUBARRAY_XSIZE
so that when the name is eval()'ed it is a simple Python identifier instead of a e.g. three nested objects.
Array identifiers appear in expression constraints as e.g. SCI_ARRAY to refer to the SCI HDU properties. In this case SCI_ARRAY is a true utils.Struct() object so it refers to Struct() properties within the eval() expression using normal Python object attribute access, e.g. SCI_ARRAY.SHAPE not SCI_ARRAY_SHAPE.
Expression constraints have the unique property that they can be mutated to generate warnings instead of errors. In contrast, there is no such mechanism for value enumerations or ranges. Custom validators can be written to issue warnings as needed.
The example constraint expression above can be mutated to a warning like this:
(warn_only(1<=META_SUBARRAY_XSTART+META_SUBARRAY_XSIZE-1<=2048))
If the constraint fails, a log ERROR which would fail the file submission is replaced with a WARNING which can be investigated and/or ignored. Warnings truly are warnings, they can flag fatal conditions but may not be applicable in all cases.
Note that this is distinct from the Presence field "W" designator and related warning() mutator which only alter the "required" status of a constraint/keyword, not the result of a constraint failure.
Expression helper functions were added to check basic table properties based on the contents of HDUS. To some degree these are redundant to the HST "C" column style constraints... but have the advantage that they operate directly on HDU array properties and type information. In contrast, the "C" column constraints followed HST practices relying more on value string formatting, e.g. "if it looks like a FLOAT, it is a FLOAT." In practice, file developers actually do make the error of adding FLOAT repr()'s to references instead of actual FLOAT values so this minor extension was added to enable checking that.
Some of the table helpers:
(is_table(xxx_ARRAY)) (is_image(xxx_ARRAY)) (has_columns(DQ_DEF_ARRAY,['BIT','VALUE','NAME','DESCRIPTION'])) (has_column_type(DQ_DEF_ARRAY,'BIT','INT'))
The value list can be empty, in which case the constraint is limited to checking presence and type.
CRDS has an HST requirement to attempt to detect missing table modes. This is done by specifying table columns which should identify unique rows, and then comparing the unique rows of and old and new table to see if any unique rows are dropped. The same generic capability can also be used by JWST.
Because the table row checks are crude approximations, the net result is generally one of two kinds of warnings. First, a table may define more than once instance of a row which should be uniquely identified; these are referred to in a warning as "duplicate" rows. Second, the new version of a table may drop unique rows found in the original version; this is reported loosely as one or more missing "modes".
Unique rows are defined by combinations of column parameters. The parameter names used to select unique rows are defined in the "spec" file of each reference type as needed in a unique_rows header field, e.g.:
miri_cubepar.rmap: 'unique_rowkeys' : ('GRATING', 'FILTER'),
The spec files are located in the "specs" directory of each project directory, e.g. crds/jwst/specs/miri_cubepar.rmap. Spec files also define other static reference type properties like short and long form names, etc. To speed loading on slow file systems, specs for all instruments and types are combined into a single combined_specs.json file for each project.
Because one reference may define more than one table, unique row names are only used in the row selection combination if they're present in a particular table. Independent .tpn checks can verfy that all required columns are present.
In the above example, if one table defined unique rows by GRATING, and a second table defined unique rows by FILTER, CRDS would correctly support both table checks. In a different situation, unique table rows might be defined by combinations of both FILTER and GRATING. This dicey interpretation of unique rows turns out to be good enough in practice, it's relatively uncommon to check multiple tables in one reference.
When run without --verbose, CRDS certify is relatively quiet about what it is checking unless checks fail. (A current exception which may change is the regurgitation of the complete fitsverify output. But most .tpn checks are silent unless --verbose is set or they fail.)
Verifying changes to CRDS certify .tpn files can generally done by running certify over some context, imap, or rmap in --deep mode which will attempt to certify each reference file and/or sub-mapping. Further, turning on the debug messages with --verbose or --verbosity=60 or 70 or.. will generate output on what CRDS is checking, how, and why / why not.
An example of running CRDS this way would be:
$ export CRDS_SERVER_URL=https://jwst-crds.stsci.edu $ export CRDS_PATH=/grp/crds/cache $ crds certify jwst-nirspec-superbias-edit --deep --dump-unique-errors --verbose --dump-provenance
The output, which is copious, is relatively self-explanatory. Typically one greps through it for output from the constraint being added or modified.
For extensive changes to certify, it can be useful to run it on all the active reference files like this:
$ export CRDS_SERVER_URL=https://jwst-crds.stsci.edu $ export CRDS_PATH=/grp/crds/cache $ crds certify jwst-edit --deep --dump-unique-errors --verbose --dump-provenance
where the symbolic context name 'jwst-edit' is interpreted to something more literal like 'jwst_0442.pmap'. Likewise, exhaustive testing may require running certify on 'hst-edit' as well after setting:
$ export CRDS_SERVER_URL=https://hst-crds.stsci.edu
NIRSpec IRS2 readouts produce 3200 pixels in one image dimension. In the native detector readout orientation it's nx=3200, ny=2048 (i.e. it's a horizontal rectangle). But all science data and all reference data in CRDS always need to be in DMS (science) orientation, which for NIRSpec means the x/y axes get swapped, so that means IRS2 images have nx=2048, ny=3200 (i.e. a vertical rectangle). Taking a quick look at one of the NIRSpec MASK ref files in CRDS for IRS2 mode, it correctly shows that the image has dimensions of 2048x3200. So that's the correct orientation you're looking for. If anyone ever delivers a NIRSpec ref file to CRDS that has dimensions 3200x2048, it's wrong (it's still in native detector orientation) and needs to be rejected.
The complicating factor in all of this is that a conscious decision was made to still have the SUBSTRTn keywords (datamodel meta.subarray.[xy]size) retain their original values of 2048, rather than 3200, because the extra pixels in the image do not correspond to real pixels on the detector (they're virtual values inserted into the image). So the detector was still commanded to readout 2048x2048 pixels, hence the decision to make the size keywords still say 2048x2048. Even though ny=3200 in the actual image. So any comparison of subarray size keyword values against the actual image size needs to allow for this (i.e. it's OK to have meta.subarray.ysize=2048 when data.shape[-2] = 3200), as long as READPATT has the string "IRS2" somewhere in it.
For the JWST detectors all reference pixels are physical pixels that are counted as part of the detector dimensions (unlike virtual overscan regions in CCD's). So the 2048x2048 detector dimensions of the near-IR detectors already includes the reference pixels and the MIRI detectors are always referenced in the full 1032x1024 space that includes their reference pixels. The SUBSTRTn and SUBSIZEn values also always include the reference pixels (i.e. SUBSTRT1 = 1 means the subarray is starting on the first reference pixel. The first "live" pixel is at SUBSTRT1 = 5.) So for MIRI full-frame readouts SUBSIZE1 = 1032, not 1024 (the same as NAXIS1).
The only exception to this is the NIRSpec IRS2 readout mode that includes many more columns of reference pixels interspersed within the live pixels, resulting in total image dimensions that are greater than 2048 (at least along the y image axis). So this is the only case where SUBSIZEn != NAXISn, because NAXIS2 2048, while a decision was made to still set SUBSIZE2 = 2048.
Comments about array dimensions and array shape equivalence:
DARK: non-MIRI: SCI=ERR=3D, DQ=2D; MIRI: SCI=ERR=DQ=4D
LINEARITY: COEFFS=3D, DQ=2D
For NIRSpec data, the DARK step is run (in calwebb_detector1.py) after refpix, so the image at that point will be 2048 x 2048, and the dark file will have shape (N, 2048, 2048), where N has to do with the number of groups. So it is correct that the darks will be 2048x2048.
Similarly for READNOISE.
If memory serves, in a conversation we all had with NIRSpec folks about a year ago, they need to deliver some subarray ref files with SUBARRAY='GENERIC', because the exact placement of the subarray varies from exposure to exposure and is tied to the use of different gratings (different gratings result in spectra being located in slightly different places on the detector and they change the subarray location to match). So for example the "mystery stripe" 2048x256 subarray is probably used for fixed-slit exposures, where the 256 is large enough to cover all the slits. Science exposures taken using a subarray for a single slit (which is smaller yet) will use that 2048x256 reference file and extract (on the fly) the subarray that matches the smaller science subarray, matching both the location and size of the subarray used in the science exposure. So that's why SUBARRAY has to be set to 'GENERIC' in those ref files, so that CRDS knows to select it when a science exposure uses some other specific subarray like "SUBS200A1" or "SUBS400A1" and let the cal pipeline do the on-the-fly extraction thing, like it also does when full-frame ref files use SUBARRAY='GENERIC'.
The reason they can't just use a full-frame ref file with SUBARRAY='GENERIC' for these, is because NIRSpec subarrays are readout using a different gain than full-frame, so they have to use subarray-specific reference files (because the actual pixel values in the images are different than for full-frame).
The GainModel schema specifies a single FITS HDU with EXTNAME='SCI'. The jump and ramp_fit steps, which use the gain ref file, both load it into a GainModel data model, hence if there isn't a SCI extension present I would assume the load would fail. Therefore the SCI label should be mandatory.
The GAINFACT keyword is only used (and hence required) for NIRSpec gain ref files that are subarray (like jwst_nirspec_gain_0016.fits and jwst_nirspec_gain_0017.fits). So GAINFACT is only needed when the ref data have dimensions less than 2048x2048.
Huh, never noticed this before, which is due to the fact that all we do with the imaging-mode AREA reference files is attach the data array to an extra extension in the science product and that's it - we don't actually use it or apply it anywhere. It's information only, for possible use by the user while doing analysis. The reference pixels don't get stripped off until level-3 processing, which combines multiple images. Level-2 products, which is the stage where a user might need to use the AREA data, still have the reference pixels in the image. So theoretically I guess the AREA array should be the original 1032x1024 size that includes the reference pixels, just for ease in applying it to the science image, which will still be 1032x1024 at that point.
AREA ref files for the imaging mode of other instruments, like NIRCam, are the full 2048x2048 size, which means they contain reference pixels. Hence for consistency we should request that the MIRI IDT deliver theirs the same way.


Learnings
Advanced Git concepts and their workflows.
CI/CD tools used to manage software development.
Build knowledge on top of my existing one in python.
Test Driven Development.
Managing huge datasets efficiently using spark, HDFS.
Learned how to develop projects within team environment.
Technologies
GIT version control system.
Object Oriented Programming.
Python including various modules such as Pandas, PySpark, Pytest, PyHDFS etc.
Jira, Jenkins
Worked on file systems like NFS, HDFS.
Contributions
Overhauled many python scripts for reporting service as per the clients/internal requirements.
Implemented python scripts with complex SQL queries to performed various operations on huge datasets from ODS using pyspark.
Designed code to replicate data files to Large file store so that we don't lose data even if in case of any mishap.
Created and merged over 30 pull requests during my internship period to the Bloomberg's existing codebase.
All the scripts on which I worked are in production and are being used in the company.


AtomError
public enum AtomError : Error
List of all possible error cases thrown by Atom
framework.
-
Decoder failed to decode data.
Declaration
Swift
case decoder(Error)
-
Failed to initialize Data instance from
URL
.Declaration
Swift
case data(Error)
-
Failed to initialize
URLRequest
withRequestable
instance.Declaration
Swift
case requestable(RequestableError)
-
Service returned invalid response where the status code is not in
200...299
range.An optional response
data
will be set for further processing of thebody
. In the context of ACE Group,data
will contain the error message or the model object.For more information, see
AtomResponse
.Declaration
Swift
case response(AtomResponse)
-
URLSession failed with error.
Declaration
Swift
case session(Error)
-
Unexpected, logic error.
Declaration
Swift
case unexpected
-
Unknown error occurred.
As an example, this error is thrown when
URLSession
call fails without providing any meaningful data (ex: error, response, and data arenil
).Declaration
Swift
case unknown
-
Returns
Bool
indicating whether the error is due to an invalid or expired access token.Declaration
Swift
var isAuthorizationFailure: Bool { get }
-
Returns
Bool
indicating whether the error is due to a failed attempt to refresh an access token.A client might encouter this error if the authorization server determines that the refresh token used in a request is invalid or expired.
Unlike
AtomError
case -.response
- token refresh error is nested inside of the.session
error case to make differentiation between a standardBad Request
andBad Request
due to an invalid / or expired access token easier.Declaration
Swift
var isAccessTokenRefreshFailure: Bool { get }
-
Convenience method for decoding error object or message returned by the service.
JSON decoder with the default formatting settings and decoding strategies will be used.
Declaration
Swift
func decodeIfPresent<T>(as type: T.Type) throws -> T? where T : Decodable
Parameters
type
The type to decode.
Return Value
An optional instance of type
T
ifAtomError
is.response
, and contains valid data,nil
otherwise.


With the _attributes option, you can assign a class to each row, or set various settings
such as disabled, rowSpan, check and etc. You can see the options
here
The example code can be slower than your environment because the code is transpiled by babel-standalone in runtime.


---
layout: post
title: links for 2007-10-26
date: 2007-10-26 00:00:00
---
-
Agile fashion?
-
-
J2ME not dead after all :)
-
-
-
-
Sour grapes with a double helping of meh, but good to see some of the guys in this list getting recognition :)
-
-
I really must spend a bit more time learning about Groovy...
-
-


 Leave your struggle behind and enter into the peace of the true faith. Only then will you find refuge from the evil and chaos that surrounds us...
Learn skills.
Ask about Skill Transfer.
Ask about Skill Enchanting.
Quest


We created the Equity Bailout Listing Program to help homeowners in these uncertain times retain their equity by saving them thousands in commissions. With large corporations getting massive bailout programs from the government, why shouldn't our homeowners get a deal too? Using the Equity Bailout program, homeowners can list their homes today for just a 1% listing commission. To learn more simply Text Sell to 77948 to get started now!
Within 24 hours your listing can be live and on the market. From the moment we come in contact, Rick will reach out and gather all necessary information and documentation and discuss showing schedules and access information for your home.
Save Your Hard Earned Equity and Pay Just 1% to List Your Home
"Rick sold our home in just 2 days on the market for 10K over asking price! He was kind enough to sell our home and discount his commission to just 1% saving us thousands of dollars and allowing us to buy a new home!"
"Rick was amazing! We appreciated his professional and honest knowledge! He was quick to respond to any questions and/or requests. For this being our fist time selling our house, it was painless and enjoyable! I would recommend Rick to anyone that wants to have a memorable experience in the selling world! Thanks again Rick!
"Rick was an absolute pleasure to work with. He's a great communicator and is super flexible and eager to make a deal. I would recommend him to anyone I know!!


If you have not familiarized yourself with the build process, please start with {@tutorial dev}.
If you are not already familiar with how DASH and DASH content work, please see the {@tutorial intro} tutorial. We will not cover the encoding of DASH videos, but {@tutorial intro} links to external tools which could be used to encode and package content.
If you want to start streaming with HTML5 and DASH, the first thing you need is source material. For this tutorial, we'll be using DASH-packaged turtle videos to create the world's next big streaming site: TurtleTube. The DASH videos for this tutorial have already been uploaded to appspot, so the URLs used are real.
This library relies on polyfills for certain browser functionality. If you are not familiar with the concept, please browse the {@tutorial polyfills} tutorial before continuing this one.
The Shaka Player is composed of two parts: the player and the video source. To start, you instantiate a player object, then create and load a video source. A video source can be anything that implements {@link shaka.player.IVideoSource}. The library provides {@link shaka.player.DashVideoSource} for DASH content. We will be using this in the tutorial.
First, create a page with a {@link http://goo.gl/NwDFQ video tag}. Install the polyfills before you do anything else with the library. Then, instantiate your {@link shaka.player.Player} object to manage the video element. For simplicty, we will use the video tag's built-in controls in this example.
Next, construct a {@link shaka.player.DashVideoSource} object. This object will manage DASH streaming and adaptation. To pass the video source into the player, call {@link shaka.player.Player#load player.load}.
Here is a simple page which demonstrates basic DASH playback:
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Basic Test</title>

<video id="video"
width="640" height="480"
crossorigin="anonymous"
controls>
Your browser does not support HTML5 video.
</video>

There are two ways to start a video playing right away. The simplest is to use the {@link http://goo.gl/McU4Rh autoplay attribute} on the video tag.
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Autoplay</title>

<video id="video"
width="640" height="480"
crossorigin="anonymous"
controls
autoplay>
Your browser does not support HTML5 video.
</video>

If you want to do more than start the video playing, you can execute a function asynchronously after {@link shaka.player.Player#load player.load} completes. It returns a {@link http://goo.gl/8NHkC1 Promise} which is resolved once the video is loaded. You can execute any arbitrary code when load completes.
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Async Load</title>

<video id="video"
width="640" height="480"
crossorigin="anonymous"
controls>
Your browser does not support HTML5 video.
</video>

Autoplay on the video tag does have one distinct advantage. If autoplay is set, {@link shaka.player.Stats#playbackLatency playbackLatency stats} are collected. If this information is important to you, please use autoplay.
(The use of the autoplay attribute and the execution of asynchronous code after load are not mutually exclusive.)
(NOTE: Autoplay is disabled on some mobile browsers, including Chrome. Both the autoplay attribute and programmatic calling of video.play() from outside a user interaction event are disabled. This is something the library cannot fix. See http://crbug.com/159336 for background.)
Some of our TurtleTube content is worth a lot of money, so we are going to make use of Widevine to protect it.
The Shaka Player uses the {@link http://goo.gl/o9Guuu EME} APIs to get licenses and decrypt protected content. However, many details needed to do this are not part of the DASH spec. Although there is a ContentProtection element specified, its contents and interpretation are application-specific.
To bridge the gap between DASH and EME with application-specific details, Shaka makes a callback to the application to interpret ContentProtection elements from the DASH manifest. The application receives the ContentProtection's scheme ID URI (a string) and the entire ContentProtection element, and the app then returns one or more {@link shaka.player.DrmInfo.Config} objects, which each specify DRM configuration options (e.g., the key system and license server URL).
Here is how {@link shaka.player.DashVideoSource#ContentProtectionCallback} can be used to enable encrypted content:
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Encrypted Content</title>

<video id="video"
width="640" height="480"
crossorigin="anonymous"
controls autoplay>
Your browser does not support HTML5 video.
</video>

This example uses Widevine, but the Shaka Player library queries the browser to find out what key systems are supported. So you should be able to protect your content with multiple key systems for different browsers. The only requirement is that the browser has a standards compliant implementation of EME. If that's the case, you can put multiple ContentProtection tags in your MPD, and once the tags are resolved as DrmInfo.Config objects, the player will discard those which cannot be used by the browser. This way, there's no need to detect the browser vendor and serve different content based on that.
This is a sample demonstrating a complete TurtleTube site. Just click on one of the video thumbnails, and the appropriate video source will be constructed. The video will play back in an overlay on top of the thumbnails.
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Beta!</title>

TurtleTube!

Choose a video:

cute green sea turtle in Ko'olina Hawai'i

(MP4, WebM)

Endangered Ocean: Sea Turtles

(MP4, WebM)

sea turtles exercise: bent arms

(WebM only)

sea turtles exercise: straight arms

(WebM only)

Using robots to reveal secrets of walking baby sea turtles

(MP4, WebM)

kitten vs sea turtle

(MP4 only, encrypted)

<video id="video"
poster="https://turtle-tube.appspot.com/poster.jpg"
crossorigin="anonymous"
controls autoplay>
Your browser does not support HTML5 video.
</video>

<button id="closeButton" onclick="closeVideo()">Close Video</button>

There a few events which the Player will dispatch. This sample demonstrates how you can listen for the adaptation event and use to it indicate when an HD video is being played. It also shows an error dialog if/when error events occur.
<!DOCTYPE html>

<meta charset="utf-8">
<title>TurtleTube - Beta (HD)!</title>

TurtleTube!

Choose a video:

cute green sea turtle in Ko'olina Hawai'i

(MP4, WebM)

Endangered Ocean: Sea Turtles

(MP4, WebM)

sea turtles exercise: bent arms

(WebM only)

sea turtles exercise: straight arms

(WebM only)

Using robots to reveal secrets of walking baby sea turtles

(MP4, WebM)

kitten vs sea turtle

(MP4 only, encrypted)

<video id="video"
poster="https://turtle-tube.appspot.com/poster.jpg"
crossorigin="anonymous"
controls autoplay>
Your browser does not support HTML5 video.
</video>

<button id="closeButton" onclick="closeVideo()">Close Video</button>


Trove Convert
Download Selected Pictures
Goto Requests Administration
prev
1
next
Copyright © Crimson Work. All rights reserved.
123
123
123
123
123
123
123
123
123
123
123
123


I have always had a curious mind. I always strive to learn how things work from heavy machines, electronics, electrical appliances and even such simple things as lighters. I have no prior background in IT, but have studied and qualified in CPA (section II), of Arts degree in Gender and Development, and I am a certified mixologist with 4 years experience.
I took up the programming class so that I can learn how to talk to machines. IT is the next big thing in Africa, and I definitely getting a piece of that cake. Programming amuses me to the core. From the complex stuff such as robotics and AI, Machine learning to the much simpler stuff like creating and styling web-pages. I just love it!!!


Coronavirus Infection Prediction
This model predicts the probability of getting infected by 2019 nCov (aka. Coronavirus) using a logistic regression model trained on actual data from the patients in Wuhan.
{% if data.prediction %}
{{data.prediction}}
Gender: {{data.form.predict_sex}}
Age: {{data.form.predict_age}}
Fever: {{data.form.predict_fever}}
Cough: {{data.form.predict_cough}}
Sputum: {{data.form.predict_sputum}}
Headache: {{data.form.predict_headache}}
Been to Wuhan in past 40 days: {{data.form.predict_wuhan}}
Been to China in past 40 days: {{data.form.predict_china}}
Breathing Difficulty: {{data.form.predict_breath}}
{% endif %}


Full stack developer at "G&F inteligencia digital"
My name is Mariano, i am 32 years old and i live with my wife and pets in Villa de Mayo, Buenos Aires. In my free time i like going out with my family, watching series and playing chess online. As a developer, i perform with high responsability, aiming technical excellence.
I work since three years ago at G&F. It is a company specialized in financial products. My main activity was related with the development of a set of web applications that manage a bank customer database. During my experience in this project i worked close to the customer,being in charge of most deployment on productive enviroments. Finally i become the development team leader, and started coaching other members of the team, with technical excellence as a goal. Currently, i am performing as a technical leader. It is a new profile in the company, with the objective of improving products internal quality, and coaching about technical skills.
This was my first experience as a developer. DocSolutions was a start up company when i join them. There i worked with other five colleagues on several projects, most of those consisted on development and maintenance of web sites (e.g. Cinepolis), web services (e.g. Movistar) and web applications (e.g. GeDeCo).


Join the platform that already has tens of thousands of fellow expats posting photos of their adventures every single day!
Our platform "Expat Journal" has been called "A marvelous, user friendly experience" by The Amazing Tech Company and "Something I use every day" by Expat Monthly Magazine.


Comments
-
Greg Hurrell
Thanks for submitting the ticket, Mike.
Just to clarify a couple of points:
- Can you tell me what you mean by a "star rating system"? Synergy already allows you to set star ratings via hot keys, and it shows star ratings in the floater. Did you have something else in mind? There is a ticket open for clickable star ratings in the menu bar (ticket #59).
- The "ticker" functionality is on the "to do" list: the ticket for it is ticket #61.
-
anonymous
sorry. i know synergy has this functionality on the menu.. I would like to see it built in to the app so it apears next to the controls on the menu strip. Currently I am using youcontrol in conjunction synergy (synergy has much nicer control buttons, floating notifications and memory usage but youcontrol has a more functional drop down menu).
This gives me a star rating on the menu strip that becomes a track name/artist/album ticker during playback (roll over to get back to star rating). Im not a fan of hotkeys and really love the graphical display.
cant wait to see the ticker in a new version soon!
-
Greg Hurrell
Ok, thanks for the clarification. I'm going to close this ticket and make a note on those other ones (ticket #59 and ticket #61) noting your proposal that the ticker and the star rating bar should actually be combined.
-
Greg Hurrell
Status changed:
- From: New
- To: Closed
Add a comment
Comments are now closed for this issue.


|
SuanShu, a Java numerical and statistical library | |||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.numericalmethod.suanshu.matrix.doubles.factorization.eigen.Eigen
public class Eigen
This class implements the eigenvalue decomposition of a matrix.
Given a square matrix A
,
an eigenvalue λ
and its associated eigenvector v
are defined by
Av = λv
We first find the eigenvalue and then the eigenvector space by solving a system of homogeneous linear equations. That is,
(A - λ)v = 0
The eigen information is stored in the read-only structure Eigen.Property
.
The keys are the eigenvalue.
TODO: Complex eigenvectors is not yet supported. Note that, for now, we compute eigenvectors only for real eigenvalue. Although we compute the complex eigenvalue, we do not compute the eigenvectors associated with them.
The R equivalent function is eigen
.
| Nested Class Summary | |
|---|---|
static class |
Eigen.Method
the methods available to compute eigenvalue and eigenvectors |
static class |
Eigen.Property
Property contains the eigen information about a particular eigen value, such as its multiplicity and the associated eigen vectors. |
| Field Summary | |
|---|---|
double |
epsilon
a precision parameter: when a number |x| ≤ ε, it is considered 0 |
| Constructor Summary | |
|---|---|
Eigen(Matrix A)
Compute the eigenvalue and eigenvectors for a matrix. |
|
Eigen(Matrix A,
Eigen.Method method,
double epsilon)
Compute the eigenvalue and eigenvectors for a matrix. |
| Method Summary | |
|---|---|
DiagonalMatrix |
D()
Get a copy of the diagonal matrix D as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now. |
java.lang.Number |
eigenvalue(int i)
Get the i-th eigenvalue. |
NumberList |
eigenvalues()
Get all the eigenvalues. |
Eigen.Property |
getProperty(int i)
Get the i-th property. |
Eigen.Property |
property(java.lang.Number eigenvalue)
Get a Property object by its eigenvalue. |
Matrix |
Q()
Get a copy of Q as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now. |
Matrix |
Qt()
Get a copy of Q.t() as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now. |
double[] |
realEigenvalues()
Get the real eigenvalue. |
int |
size()
Get the number of distinct eigenvalue. |
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
public final double epsilon
| Constructor Detail |
|---|
public Eigen(Matrix A, Eigen.Method method, double epsilon)
For each eigenvalue, we compute for its properties as in Eigen.Property
.
There is no single eigenvector. There are infinitely many.
The set of eigenvectors forms a vector space.
Any linear combination of the eigen basis vectors qualifies as an eigenvector.
Note that, for now, this implementation computes the Eigen.Property
only for real eigenvalue.
Although we compute the complex eigenvalue, we do not compute the eigenvectors associated with them.
A
- a square matrixmethod
- the eigen decomposition algorithm, c.f., Eigen.Method
epsilon
- a precision parameter: when a number |x| ≤ ε, it is considered 0
java.lang.IllegalArgumentException
- if A
is not squarepublic Eigen(Matrix A)
A
- a square matrix| Method Detail |
|---|
public DiagonalMatrix D()
EigenDecomposition
D
as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now.
D
in interface EigenDecomposition
D
public Matrix Q()
EigenDecomposition
Q
as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now.
Q
in interface EigenDecomposition
Q
public Matrix Qt()
EigenDecomposition
Q.t()
as in
Q %*% D %*% Q' = A
Note that we only support real eigenvalues for now.
Qt
in interface EigenDecomposition
Q.t()
public int size()
public java.lang.Number eigenvalue(int i)
The eigenvalues are sorted in ascending order.
i
- the index, counting from 0
public NumberList eigenvalues()
public double[] realEigenvalues()
The eigenvalues are sorted in ascending order.
public Eigen.Property property(java.lang.Number eigenvalue)
Note that the number passed in must be exactly the same as the eigenvalue in binary representation. Passing in an approximate number (up to precision) will likely result in an unmatched error, i.e., a null object returned.
eigenvalue
- an eigenvalue
public Eigen.Property getProperty(int i)
The properties are sorted in ascending order by their eigenvalues.
i
- the index, counting from 0
|
SuanShu, a Java numerical and statistical library | |||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


index
|
up
|
prev
|
next
MemberParsingAction<Type> Class
Definition at line
22
of
Action.hpp
Constructors
MemberParsingAction
(Type* type_,
void
type_)
Member Functions
void
operator()
(
const
char32_t
* matchBegin,
const
char32_t
* matchEnd,
const
Span
& span,
const
std::string& fileName,
ParsingData
* parsingData,
bool
& pass)
const override
Member Variables
Type*
type
Constructor Details
MemberParsingAction Constructor
cmajor::parsing::MemberParsingAction::MemberParsingAction(Type* type_,
void
type_)
Definition at line
25
of
Action.hpp
:
25
:
26
type
(
type_
)
,
memfun
(
pm
)
27
{
28
}
Member Function Details
operator() Member Function
void
cmajor::parsing::MemberParsingAction::operator()(
const
char32_t
* matchBegin,
const
char32_t
* matchEnd,
const
Span
& span,
const
std::string& fileName,
ParsingData
* parsingData,
bool
& pass)
const override
Definition at line
29
of
Action.hpp
:
30
{
31
(
type
->*
memfun
)
(
matchBegin
,
matchEnd
,
span
,
fileName
,
parsingData
,
pass
)
;
32
}
Base class overridden functions:
cmajor::parsing::ParsingAction::operator()
index
|
up
|
prev
|
next


Student at Neumont College for Computer Science,
Event Coordinator at This is the Place Heritage Park
Greater Salt Lake City area | Contact info
Ads
Experience
Event Coordinator
Education
Neumont College of Computer Science
Volunteer Experience
Missionary
Skills
Cultural Heritage
Event Planning
Computer Science


Right now
{{wc.forecast.currently.temperature | number:0}}°
{{wc.forecast.currently.summary}}
Feels like {{wc.forecast.currently.apparentTemperature | number:0}}°
Humidity: {{wc.forecast.currently.humidity}} % Dew Point: {{wc.forecast.currently.dewPoint}} Pressure: {{wc.forecast.currently.pressure}}mb


Fake News Generator: Who starts viral misinformation?
- BBC NewsConspiracy theories and speculation about coronavirus have flooded social media. But who starts these rumours? And who spreads them? Specialist disinformation reporter Marianna Spring has investigated lots of misleading stories circulating online during the pandemic. It's given her an idea about who is behind this misinformation - and what motivates them. Laughs, power, clicks - or answers. Video produced by Suniti Singh
CIVIX Explains: Disinformation
- CTRL-FDisinformation is false or misleading information that is shared deliberately to cause harm. Learn about how disinformation producers use knowledge of human emotions and technology to spread disinformation.
"Misinfo" day for high school students
- KING 5As concerns rise about misinformation spread on social media, there is a new effort to educate a younger generation about how to sort facts from fiction. A team at UW gathered high school students for "mis-info" day. It's a workshop that trains teens to scrutinize what others are sharing. KING 5's Ted Land reports.
Disinformation vs Misinformation: 5 Ways To Detect Fake News Amid A Pandemic
- NewsweekNow more than ever we receive information from a range of sources, whether it be from TV, online or social media. There are different types of false info; misinformation and disinformation, most commonly known as fake news. Wrong or false info is easily spread now due to technology, so it’s vital to know what you’re consuming is reliable and accurate. Here are five ways you can detect fake news.
Why people fall for misinformation - Joseph Isaac
- TED-EdHow does a fact become a misconception? Dig into the world of misinformation to see how facts can become distorted and misleading.
The Conspiracy Chart TikTok Compilation
- Abbie RichardsCompilation of all my Conspiracy Charts videos from the original design to parts 1-5 breaking it down. Enjoy!
Useful Websites:
Checkology Virtual Classroom
News Literacy Project’s Checkology Virtual Classroom


| [ << ] | [ < ] | [ Up ] | [ > ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The shell can perform integer and floating point arithmetic, either using the builtin let, or via a substitution of the form $((...)). For integers, the shell is usually compiled to use 8-byte precision where this is available, otherwise precision is 4 bytes. This can be tested, for example, by giving the command ‘print - $((<PHONE_NUMBER>1 ))’; if the number appears unchanged, the precision is at least 8 bytes. Floating point arithmetic always uses the ‘double’ type with whatever corresponding precision is provided by the compiler and the library.
The let builtin command takes arithmetic expressions as arguments; each is evaluated separately. Since many of the arithmetic operators, as well as spaces, require quoting, an alternative form is provided: for any command which begins with a ‘((’, all the characters until a matching ‘))’ are treated as a quoted expression and arithmetic expansion performed as for an argument of let. More precisely, ‘((...))’ is equivalent to ‘let "..."’. The return status is 0 if the arithmetic value of the expression is non-zero, 1 if it is zero, and 2 if an error occurred.
For example, the following statement
(( val = 2 + 1 ))
is equivalent to
let "val = 2 + 1"
both assigning the value 3 to the shell variable val and returning a zero status.
Integers can be in bases other than 10. A leading ‘0x’ or ‘0X’ denotes hexadecimal and a leading ‘0b’ or ‘0B’ binary. Integers may also be of the form ‘base#n’, where base is a decimal number between two and thirty-six representing the arithmetic base and n is a number in that base (for example, ‘16#ff’ is 255 in hexadecimal). The base# may also be omitted, in which case base 10 is used. For backwards compatibility the form ‘[base]n’ is also accepted.
An integer expression or a base given in the form ‘base#n’ may contain underscores (‘_’) after the leading digit for visual guidance; these are ignored in computation. Examples are 1_000_000 or 0xffff_ffff which are equivalent to 1000000 and 0xffffffff respectively.
It is also possible to specify a base to be used for output in the form ‘[#base]’, for example ‘[#16]’. This is used when outputting arithmetical substitutions or when assigning to scalar parameters, but an explicitly defined integer or floating point parameter will not be affected. If an integer variable is implicitly defined by an arithmetic expression, any base specified in this way will be set as the variable’s output arithmetic base as if the option ‘-i base’ to the typeset builtin had been used. The expression has no precedence and if it occurs more than once in a mathematical expression, the last encountered is used. For clarity it is recommended that it appear at the beginning of an expression. As an example:
typeset -i 16 y print $(( [#8] x = 32, y = 32 )) print $x $y
outputs first ‘8#40’, the rightmost value in the given output base, and then ‘8#40 16#20’, because y has been explicitly declared to have output base 16, while x (assuming it does not already exist) is implicitly typed by the arithmetic evaluation, where it acquires the output base 8.
The base may be replaced or followed by an underscore, which may itself be followed by a positive integer (if it is missing the value 3 is used). This indicates that underscores should be inserted into the output string, grouping the number for visual clarity. The following integer specifies the number of digits to group together. For example:
setopt cbases print $(( [#16_4] 65536 ** 2 ))
outputs ‘0x1_0000_0000’.
The feature can be used with floating point numbers, in which case the base must be omitted; grouping is away from the decimal point. For example,
zmodload zsh/mathfunc print $(( [#_] sqrt(1e7) ))
outputs ‘3_162.277_660_168_379_5’ (the number of decimal places shown may vary).
If the C_BASES option is set, hexadecimal numbers are output in the standard C format, for example ‘0xFF’ instead of the usual ‘16#FF’. If the option OCTAL_ZEROES is also set (it is not by default), octal numbers will be treated similarly and hence appear as ‘077’ instead of ‘8#77’. This option has no effect on the output of bases other than hexadecimal and octal, and these formats are always understood on input.
When an output base is specified using the ‘[#base]’ syntax, an appropriate base prefix will be output if necessary, so that the value output is valid syntax for input. If the # is doubled, for example ‘[##16]’, then no base prefix is output.
Floating point constants are recognized by the presence of a decimal point or an exponent. The decimal point may be the first character of the constant, but the exponent character e or E may not, as it will be taken for a parameter name. All numeric parts (before and after the decimal point and in the exponent) may contain underscores after the leading digit for visual guidance; these are ignored in computation.
An arithmetic expression uses nearly the same syntax and associativity of expressions as in C.
In the native mode of operation, the following operators are supported (listed in decreasing order of precedence):
unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement
bitwise shift left, right
bitwise AND
bitwise XOR
bitwise OR
exponentiation
multiplication, division, modulus (remainder)
addition, subtraction
comparison
equality and inequality
logical AND
logical OR, XOR
ternary operator
assignment
comma operator
The operators ‘&&’, ‘||’, ‘&&=’, and ‘||=’ are short-circuiting, and only one of the latter two expressions in a ternary operator is evaluated. Note the precedence of the bitwise AND, OR, and XOR operators.
With the option C_PRECEDENCES the precedences (but no other properties) of the operators are altered to be the same as those in most other languages that support the relevant operators:
unary plus/minus, logical NOT, complement, {pre,post}{in,de}crement
exponentiation
multiplication, division, modulus (remainder)
addition, subtraction
bitwise shift left, right
comparison
equality and inequality
bitwise AND
bitwise XOR
bitwise OR
logical AND
logical XOR
logical OR
ternary operator
assignment
comma operator
Note the precedence of exponentiation in both cases is below that of unary operators, hence ‘-3**2’ evaluates as ‘9’, not ‘-9’. Use parentheses where necessary: ‘-(3**2)’. This is for compatibility with other shells.
Mathematical functions can be called with the syntax ‘func(args)’, where the function decides if the args is used as a string or a comma-separated list of arithmetic expressions. The shell currently defines no mathematical functions by default, but the module zsh/mathfunc may be loaded with the zmodload builtin to provide standard floating point mathematical functions.
An expression of the form ‘##x’ where x is any character sequence such as ‘a’, ‘^A’, or ‘\M-\C-x’ gives the value of this character and an expression of the form ‘#name’ gives the value of the first character of the contents of the parameter name. Character values are according to the character set used in the current locale; for multibyte character handling the option MULTIBYTE must be set. Note that this form is different from ‘$#name’, a standard parameter substitution which gives the length of the parameter name. ‘#\’ is accepted instead of ‘##’, but its use is deprecated.
Named parameters and subscripted arrays can be referenced by name within an arithmetic expression without using the parameter expansion syntax. For example,
((val2 = val1 * 2))
assigns twice the value of $val1 to the parameter named val2.
An internal integer representation of a named parameter can be specified with the integer builtin. Arithmetic evaluation is performed on the value of each assignment to a named parameter declared integer in this manner. Assigning a floating point number to an integer results in rounding towards zero.
Likewise, floating point numbers can be declared with the float builtin; there are two types, differing only in their output format, as described for the typeset builtin. The output format can be bypassed by using arithmetic substitution instead of the parameter substitution, i.e. ‘${float}’ uses the defined format, but ‘$((float))’ uses a generic floating point format.
Promotion of integer to floating point values is performed where necessary. In addition, if any operator which requires an integer (‘&’, ‘|’, ‘^’, ‘<<’, ‘>>’ and their equivalents with assignment) is given a floating point argument, it will be silently rounded towards zero except for ‘~’ which rounds down.
Users should beware that, in common with many other programming languages but not software designed for calculation, the evaluation of an expression in zsh is taken a term at a time and promotion of integers to floating point does not occur in terms only containing integers. A typical result of this is that a division such as 6/8 is truncated, in this being rounded towards 0. The FORCE_FLOAT shell option can be used in scripts or functions where floating point evaluation is required throughout.
Scalar variables can hold integer or floating point values at different times; there is no memory of the numeric type in this case.
If a variable is first assigned in a numeric context without previously being declared, it will be implicitly typed as integer or float and retain that type either until the type is explicitly changed or until the end of the scope. This can have unforeseen consequences. For example, in the loop
for (( f = 0; f < 1; f += 0.1 )); do # use $f done
if f has not already been declared, the first assignment will cause it to be created as an integer, and consequently the operation ‘f += 0.1’ will always cause the result to be truncated to zero, so that the loop will fail. A simple fix would be to turn the initialization into ‘f = 0.0’. It is therefore best to declare numeric variables with explicit types.
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This document was generated on April 26, 2019 using texi2html 5.0.
Zsh version 5.7.1, released on February 3, 2019.


| recode {dplyr} | R Documentation |
This is a vectorised version of switch()
: you can replace
numeric values based on their position or their name, and character or factor
values only by their name. This is an S3 generic: dplyr provides methods for
numeric, character, and factors. For logical vectors, use if_else()
. For
more complicated criteria, use case_when()
.
You can use recode()
directly with factors; it will preserve the existing
order of levels while changing the values. Alternatively, you can
use recode_factor()
, which will change the order of levels to match
the order of replacements. See the forcats
package for more tools for working with factors and their levels.
recode()
is questioning because the arguments are in the wrong order.
We have new <- old
, mutate(df, new = old)
, and rename(df, new = old)
but recode(x, old = new)
. We don't yet know how to fix this problem, but
it's likely to involve creating a new function then retiring or deprecating
recode()
.
recode(.x, ..., .default = NULL, .missing = NULL) recode_factor(.x, ..., .default = NULL, .missing = NULL, .ordered = FALSE)
.x |
A vector to modify |
... |
< When named, the argument names should be the current values to be replaced, and the argument values should be the new (replacement) values. All replacements must be the same type, and must have either
length one or the same length as |
.default |
If supplied, all values not otherwise matched will
be given this value. If not supplied and if the replacements are
the same type as the original values in
|
.missing |
If supplied, any missing values in |
.ordered |
If |
A vector the same length as .x
, and the same type as
the first of ...
, .default
, or .missing
.
recode_factor()
returns a factor whose levels are in the same order as
in ...
. The levels in .default
and .missing
come last.
na_if()
to replace specified values with a NA
.
coalesce()
to replace missing values with a specified value.
tidyr::replace_na()
to replace NA
with a value.
# For character values, recode values with named arguments only. Unmatched # values are unchanged. char_vec <- sample(c("a", "b", "c"), 10, replace = TRUE) recode(char_vec, a = "Apple") recode(char_vec, a = "Apple", b = "Banana") # Use .default as replacement for unmatched values. Note that NA and # replacement values need to be of the same type. For more information, see # https://adv-r.hadley.nz/vectors-chap.html#missing-values recode(char_vec, a = "Apple", b = "Banana", .default = NA_character_) # Throws an error as NA is logical, not character. ## Not run: recode(char_vec, a = "Apple", b = "Banana", .default = NA) ## End(Not run) # Use a named character vector for unquote splicing with !!! level_key <- c(a = "apple", b = "banana", c = "carrot") recode(char_vec, !!!level_key) # For numeric values, named arguments can also be used num_vec <- c(1:4, NA) recode(num_vec, `2` = 20L, `4` = 40L) # Or if you don't name the arguments, recode() matches by position. # (Only works for numeric vector) recode(num_vec, "a", "b", "c", "d") # .x (position given) looks in (...), then grabs (... value at position) # so if nothing at position (here 5), it uses .default or NA. recode(c(1,5,3), "a", "b", "c", "d", .default = "nothing") # Note that if the replacements are not compatible with .x, # unmatched values are replaced by NA and a warning is issued. recode(num_vec, `2` = "b", `4` = "d") # use .default to change the replacement value recode(num_vec, "a", "b", "c", .default = "other") # use .missing to replace missing values in .x recode(num_vec, "a", "b", "c", .default = "other", .missing = "missing") # For factor values, use only named replacements # and supply default with levels() factor_vec <- factor(c("a", "b", "c")) recode(factor_vec, a = "Apple", .default = levels(factor_vec)) # Use recode_factor() to create factors with levels ordered as they # appear in the recode call. The levels in .default and .missing # come last. recode_factor(num_vec, `1` = "z", `2` = "y", `3` = "x") recode_factor(num_vec, `1` = "z", `2` = "y", `3` = "x", .default = "D") recode_factor(num_vec, `1` = "z", `2` = "y", `3` = "x", .default = "D", .missing = "M") # When the input vector is a compatible vector (character vector or # factor), it is reused as default. recode_factor(letters[1:3], b = "z", c = "y") recode_factor(factor(letters[1:3]), b = "z", c = "y") # Use a named character vector to recode factors with unquote splicing. level_key <- c(a = "apple", b = "banana", c = "carrot") recode_factor(char_vec, !!!level_key)


augeasproviders documentation index
- README - overview, pre-reqs and project information
- Manifest examples - many examples of each type/provider
- MotW post - introductory Module of the Week blog post
Developer docs:
CSS is provided by Ryan Gray’s buttondown.


@import gospeak.core.services.meetup.domain.MeetupCredentials @import gospeak.web.pages.orga.settings.routes.SettingsCtrl @import gospeak.web.pages.partials.form.html._ @import gospeak.web.utils.GsForms @import gospeak.web.utils.OrgaReq @(id: String, meetupForm: Form[GsForms.GroupAccount.Meetup], credsOpt: Option[MeetupCredentials])(implicit req: OrgaReq[AnyContent])


Sites that may have content authored from various individuals can be accommodated by using data files.
To attribute an author to a post or page that is different from the site author specified in _config.yml
:
Step 1. Create _data/authors.yml
and add authors using the following format. Anything variables found under author
in _config.yml
can be used (e.g. name
, bio
, avatar
, author links
, etc.).
# /_data/authors.yml
Billy Rick:
name : "Billy Rick"
bio : "What do you want, jewels? I am a very extravagant man."
avatar : "/assets/images/bio-photo-2.jpg"
links:
- label: "Email"
icon: "fas fa-fw fa-envelope-square"
url<EMAIL_ADDRESS>- label: "Website"
icon: "fas fa-fw fa-link"
url: "https://thewhip.com"
- label: "Twitter"
icon: "fab fa-fw fa-twitter-square"
url: "https://twitter.com/extravagantman"
Cornelius Fiddlebone:
name : "Cornelius Fiddlebone"
bio : "I ordered what?"
avatar : "/assets/images/bio-photo.jpg"
links:
- label: "Email"
icon: "fas fa-fw fa-envelope-square"
url<EMAIL_ADDRESS>- label: "Twitter"
icon: "fab fa-fw fa-twitter-square"
url: "https://twitter.com/rhymeswithsackit"
Step 2. Assign one of the authors in authors.yml
to a post or page you wish to override the site.author
with.
Example: To assign Billy Rick
as an author for a post the following YAML Front Matter would be applied:
author: Billy Rick


Ask questions about the core tech here, but not questions about prices or personal questions or GUI and please don't DM JL777.
https://dexstats.info/richlistlookup.php?address=RLVzC4tr9cNKvuw2z4m8KuMfZURwCehx55
https://medium.com/@jameslee777
100 messages
Before 05-Dec-19 12:00 AM
acidbird21-Jul-19 10:06 AM
With apr
jl777c21-Jul-19 10:06 AM
is BTC a stablecoin?
people have BTC on exchanges to trade it for other things
this gives BTC a lot of value, as almost everything can be traded for BTC at the best spreads
and in crypto, trading is the only proven high volume activity
acidbird21-Jul-19 10:07 AM
Hmm, well, this sounds reasonable
But still a way for us to go
jl777c21-Jul-19 10:07 AM
yes, atomicDEX needs to get traction
we need the KMD liquidity
but with all the mandatory KYC, it could be that atomicDEX is the only place without the KYC...
so KMD would only be for people who like to trade without KYC and get 5% while waiting for good prices on the orderbooks
and the ecosystem currency
acidbird21-Jul-19 10:08 AM
But can’t they block the app if they decide to change the rules? Regulators(edited)
jl777c21-Jul-19 10:08 AM
and for whatever other benefits ecosystem projects give to KMD, we see a lot of projects give preference to KMD, even though it isnt required
they can block atomicDEX as well as they can block torrents
how is that working out for them?
will they start prosecuting grandma and their voters?
for bartering
its not even copyright infringement, which they couldnt stop
it is a common law right to barter what you own
acidbird21-Jul-19 10:10 AM
Noone knows but it is easier to block an app or website then a torrent
jl777c21-Jul-19 10:11 AM
at worst we need to host the download, maybe via torrent
in any case, atomicDEX is in the regulation safe harbor according to the latest guidance
they realized they cant make grandma have to get a money transmitter license for bartering
acidbird21-Jul-19 10:12 AM
Yeah, i saw the guidelines, we are good so far, hopefully will get out of beta soon
jl777c21-Jul-19 10:12 AM
also if the only problem is that atomicDEX gets too big that regulators are paying attention,then it is like bitmex
acidbird21-Jul-19 10:12 AM
And shake the market
jl777c21-Jul-19 10:12 AM
by then it is way too late
as atomicDEX grows, the first thing people will realize is that all the markets are in KMD
what will they do?
ERC20 all require ETH, so they are DOA as the base trading pair
what other coin will have the liquidity on atomicDEX AND pay 5%?
acidbird21-Jul-19 10:14 AM
This logic is good, i like it, when we get there KMD will be doing great
I love 5,1%, it is a great idea.
jl777c21-Jul-19 10:14 AM
also, for non-ecosystem coins, nSPV wont work
which means they need to rely on the electrum servers or normal headers only SPV
acidbird21-Jul-19 10:15 AM
Hmm, i didn’t know that, good point
jl777c21-Jul-19 10:15 AM
the reality is that there will be occasional maintenance issues with electrum servers and headers only SPV is actually quite slow, compared to nSPV
KMD will be the only top coin with nSPV
at least until other ecosystem coins become a top coin
acidbird21-Jul-19 10:16 AM
How far away are we from this?
jl777c21-Jul-19 10:16 AM
#dev-nspv
i already have a proof of concept working, the fullnode support is in jl777 branch already
it is a non-hardfork update
realistically, by end of year to get the atomicDEX with nSPV support, but by then it is iterated many times from all the feedback
trading within the ecosystem will then be able to scale basically without limits (other than blockchain capacity)
acidbird21-Jul-19 10:18 AM
Great, will check it out. ok James, it was nice chatting with you, thank you for answers and info, have a great rest of the day! Cheers!(edited)
willofleo21-Jul-19 04:50 PM
"atomicDEX listing $700 per year, atomicDEX Mobile Wallet Listing $1500 per year" what is the difference between atomicdex listings and atomicdex mobile , I mean if your listed on atomic dex api then you should be on mobile no? and I guess some project dont need to pay any fee like coins in the top 30 e.g as they can bring in volume you just added them already(edited)
jl777c21-Jul-19 04:58 PM
desktop vs desktop+mobile, i believe is the difference
willofleo21-Jul-19 04:59 PM
does the gui set the price ?
or this is at an api level meaning only payed project as allowed to list on the gui ?
jl777c21-Jul-19 05:00 PM
you are confusing general atomicDEX listings with composer
not all coins will be via composer
willofleo21-Jul-19 05:01 PM
oh I see this only applies to composer project using antara to make their blockchain
hence launching on DEX will incur a fee
makes sense if it a one time fee, I can not imagine delistings of a very good project that brings in volume(edited)
cant*
Neko21-Jul-19 07:24 PM
so what will happen after all the kmd is mined, the main security will be off the BTC network ?
Komodo. Contribute to jl777/komodo development by creating an account on GitHub.
1
vogelfreund8822-Jul-19 10:08 AM
Hi, I tried to run Agama on an raspi 3b.
Tried with ubuntu 18, suse, kubuntu, pi64 and so on... nothing works for me.
You have an idea?
It would be a big step in decentralization direction, for my understanding if every hardware can participate
A Small Form-Factor Single Board Computer Capable of 4K PlaybackROCK64 is a credit card sized Single Board Computer powered by Rockchip RK3328 Quad-Core ARM Cortex A53 64-Bit Processor and support …
Even I had said that doing atomic swaps on a blockchain with only private transactions is not possible. At first (and second and third)…
assodibrisco22-Jul-19 04:46 PM
Every coin and token will br supported?
jl777c22-Jul-19 04:46 PM
only those that have zaddr
assodibrisco22-Jul-19 04:48 PM
Example private swap btc-eth?
Only zaddr..
jl777c22-Jul-19 05:37 PM
BTC <-> PIRATE
LTC <-> PIRATE
ETH <-> PIRATE
KMD <-> PIRATE
1
1
1
1
1
1
1
11
игорь24-Jul-19 08:06 PM
Hello.
1)BLUR can become a pirate competitor?
2)What is your interest in BLUR if they this project will not migrate to KMD network?
hastrup24-Jul-19 08:08 PM
Hi do you need manually change all the fee from atomicdex to komodo? Right now there is like 20 coins in the DEX add but when there is more coins add that could be a very tedious job if it's done manually.
polycryptoblog25-Jul-19 04:34 AM
I’m sure our devs can make a way to automate the process , some custom tradebot or something , users won’t have to worry about it though
hastrup25-Jul-19 09:17 AM
@polycryptoblog i just thought that jl brain power could be used on something more productive then sitting manually and trading the coins to komodo XD.
crumb25-Jul-19 09:18 AM
rest assured, James has thought of that
CHMEX25-Jul-19 09:20 AM
With enough liquidity the fees can be sent to a BOB which will sell it automaticly if somebody wants it
I would think its just a matter of sending the collected fees to that bob which can be automated. So should not take him the whole day selling it


|
Volume 1, Article 20
I. Grzegory, M. Bockowski, B. Lucznik, S. Krukowski, M. Wroblewski, S. Porowski
High Pressure Research Center
We present recent results on bulk GaN crystallization. The best quality GaN crystals grown from the solution at high N2 pressure without an intentional seeding are single crystalline platelets of stable morphology reaching dimensions up to 10 mm. The fastest growth direction for such crystals is [1 0 0], perpendicular to the GaN c-axis. The maximum stable growth rate perpendicular to crystal c-axis is determined from the experiment and used for an estimate of the effective supersaturation for the {100} face assuming two dimensional layer growth. The heat of GaN disssolution, determined from experimental solubility data, is used for the estimation of the edge energy of 2-D nuclei on the growing {100} face. Bulk crystal growth seeded by a single hexagonal needle with well developed {100} faces is also reported. The crystallization mechanisms and morphological stability in seeded growth of GaN are discussed on the basis of experimental results. The physical properties of the GaN crystals and homoepitaxial layers grown on them are briefly reviewed.
GaN is a strongly bonded compound well suited for applications such as high power, high temperature electronics. The bonding energy of GaN is 8.9 eV/atom whereas for GaAs it is only 6.5 eV/atom [1]. Consequently, the free energy G(T) of the crystal is very low in relation to the reference state of free N and Ga atoms. On the other hand, the N2 molecule is also strongly bonded (4.9eV/atom). Therefore, the free energy of the constituent Ga and N2 becomes quite close and surpasses that of GaN as the temperature is raised. Figure 1 shows the free energy of GaN (1 mole) versus its constituents (Ga + 1/2N2) as a function of temperature.
The G(T) of the constituents decreases faster than that of the GaN crystal and at higher temperatures GaN become thermodynamically unstable. The crossing of G(T) curves determines the equilibrium temperature where GaN coexists with its constituents at a given N2 pressure. The application of pressure allows us to increase the free energy of the constituents much more than that of the crystal shifting the equilibrium point to higher temperature and extending the stability range of GaN. The equilibrium pN2 - T conditions for GaN determined experimentally by Karpinski et al. [2] [3] are shown in Figure 2.
Most of crystallization processes discussed in this paper have been carried-out in large volume gas pressure reactors at an N2 overpressure of up to 15 kbar, which corresponds to a GaN stability limit at 1850K. The maximum temperature of 1850 K is quite far from the melting temperature of GaN. According to the theoretical estimation of Van Vechten [4] the melting point of GaN is ~3000K. The N2 equilibrium pressure at the melting point is therefore much higher than we can obtain in our experimental system. The extension of the GaN stability range to 1850 K allows the growth of GaN crystals from a liquid Ga solution. In Figure 3 we show the N solubility data resulting from the annealing of Ga at the three phase equilibrium conditions of Figure 2.
Since the N concentrations are not high, the growth experiments have to be long to obtain high quality bulk GaN crystals with dimensions appropriate for measurements and applications. Therefore, stability over a long time (>100h) is an additional requirement for the growth system. The solid line in Figure 3 is the liquidus line for Ga-GaN system calculated using an ideal solution approximation and Van Vechten's melting temperature of 2790K. For an ideal solution, the solubility can be expressed as:
| (1) |
where Hsol is the heat of dissolution. For GaN, Hsol = 44.7 kcal/mole = 0.49 eV/bond and expresses the bonding energy in the crystal in relation to its mother phase: the solution.
Supersaturation in the growth solution has been created by the application of temperature gradients along the axis of the crucible. Both horizontal and vertical arrangements of the crucibles have been used. The typical duration of the experiments was 100 - 150 hours for crystals of 6-10 mm in size.
Gallium, when heated under high N2 pressure, does not wet the crucible walls (usually graphite or BN). Rather, it is covered with a thin GaN layer consisting of GaN single crystals. This is the first step of GaN crystallization at high N2 pressure. The size of the crystals covering the Ga depends on the sequence and rates of compression and heating of the system. A typical structure of the surface layer is shown in Figure 5.
The formation of the surface layer inhibits further synthesis of the nitride which allows longtime crystallization processes and the growth of relatively large GaN crystals. Since the solution is in a temperature gradient, the GaN layer at the hot end of the crucible dissolves in gallium and is transported, by convection and diffusion, to the cooler zone of the crucible where the solution become supersaturated and crystallization of GaN takes place. The crystals of the surface layer are the natural source of growth centers for further crystallization. Some of them start to grow faster hindering the growth of the neighboring ones. This selection leads to the formation of relatively large, well shaped GaN crystals on the internal surface of the surface layer.
The GaN crystals grown by this method are wurzite, mainly in the form of hexagonal platelets or hexagonal needles, depending on the growth conditions. A general tendency is that high supersaturation favors growth in the c-direction which produces the needle-like crystals. Supersaturation of the growth solution is determined mainly by the growth temperature, supercooling, mass transport mechanisms in Ga, and also the local surroundings for a particular crystal.
Crystals in the form of hexagonal platelets grow slowly, with a rate <0.1 mm/h into the {100} directions (perpendicular to the c-axis). They are usually single crystals having perfect morphology suggesting stable layer-by layer growth. They are transparent, colorless, with flat faces. In some cases, three dimensional, hexagonal growth which occurs during the post-growth system cooling is observed on the crystal surfaces. Some GaN platelets grown without intentional seeding are shown in Figure 6.
For crystallization of large GaN crystals by this method, it is crucial to control the supersaturation to avoid acceleration of the growth near the edges and corners of the growing crystal. This can occur if some of them reach dimensions significantly larger than the others. Then the edge nucleation on the hexagonal faces of the GaN platelets is often observed. This is the first step to unstable growth on these faces. The result of such a growth is shown in Figure 7a. In extreme cases of very high supersaturation or when the supersaturation is abruptly increased (i.e. by increasing the temperature gradient), the growth into c-direction, nucleated at the edges of the platelet, become very fast and leads to the formation of well developed {100} faces. This results in a hollow needle-like crystal such as is shown in Figure 7b. Therefore, for growing of large GaN platelets of stable morphology, the supersaturation must be decreased during the crystallization process inorder to prevent the runaway phenomena just described.
Therefore for the stable growth of large GaN crystals it is necessary to have sufficiently low and uniform supersaturation across the growing crystal faces. One of the most important factors influencing the uniformity of N-concentration in the growth zone is the volume of the solution in relation to the size of the growing crystal. In Figure 8 we show a crystal grown in a crucible with an internal diameter of 17 mm (as opposed to 14 mm previously). The increase in the crucible volume resulted in a significant increase of the size of crystals which could be grown with stable morphology.
We summarize the general tendencies of GaN crystal growth without an intentional seeding following from the experimental results:
For a crystal growing at 1800K, we have obtained a supersaturation of the order of 50%. For the growth of GaN on a seed crystal with well developed {100} faces, the supersaturation necessary for stable two dimensional growth can be significantly lower due to large surface area of the fast growing planes. The details of this estimation will be published elsewhere [5].
Prior to the growth, it was necessary to heat the system up to 1200oC at low N2 pressure to achieve wetting of the seed with liquid Ga. If the sysyem were be heated at high pressure, the polycrystalline surface layer would cover the Ga surface adjacent to the seed, and eliminate its usefulness for further crystallization.
The growth on the seed was performed under conditions where high quality platelets were obtained in previously described processes without intentional seeding. The resulting crystal, after 50 hours of growth, is shown in Figure 11.
The fast growth directions were the [1 0 0], as expected, and also the [1 0 1] one perpendicular to the pyramidal face. Since the growth rate in the c-direction was very small, a hexagonal face developed in the resulting crystal.
As we know from previous considerations, at low supersaturations, the growth rate in c-direction is almost zero whereas at higher supersaturations we fall into the "difficult" region of the abrupt increase of the nucleation rate. Therefore, the growth on the hexagonal face is difficult to control and probably is very sensitive to fluctuations of temperature, convectional flows and uniformity of the nitrogen concentration field in the crystal surrounding. This can easily lead to three dimensional nucleation or to edge instabilities of the growth.
The quality of the GaN crystals was estimated by X-ray rocking curves [6]. The full width at half maximum (FWHM) of the X-ray rocking curves ((0004) CuK reflection) depends on the size of the crystal. The value is 20-30 arcsec for 1mm crystals and 30-40 arcsec for 1-3 mm ones. For larger platelets the rocking curves split into a few 30-40 arcsec peaks showing the presence of low angle (1-3 arcmin) boundaries separating grains of 0.5-2 mm in size. Misorientation between grains increases monotonically from end to end of the crystal [7]. This indicates a "bending" of the crystal during growth, followed by strain relaxation leading to the formation of the low angle boundaries. Indeed, the lattice constantsmeasured for the two basal planes of the crystals differ by 0.001 », suggesting different growth mechanisms and defect structures for the two polar directions of the GaN platelet. It was shown by Liliental-Weber et al. [8] by TEM examination and confirmed by AFM studies [9], that one of the polar surfaces of the GaN crystals (especially for the smaller ones) is often atomically flat (2-3 monolayer steps present) and the crystal below is practically free of extended defects. However a high concentration of point defects, mainly N-vacancies and Ga-antisites, were deduced from Convergent Beam Electron Diffraction patterns [8]. On the opposite side, regular pyramids of 100 - 200 » in size terminate the surface [8] [9]. Below are many extended defects such as stacking faults, dislocation loops and Ga micro-precipitates. Such pecularities result from the different growth modes of the two polar directions of the crystal in the Ga-rich conditions of the process.
For larger (>3 mm) GaN plateletes, the presence of three dimensional hexagonal features on the surfaces is typical. This results from the system cooling after the growth in the temperature gradient.
The GaN crystals are highly conductive, showing metallic behavior in the whole temperature range of 4.2-300K. Free electron concentration of 1019 - 1020 cm-3 with mobilities of 30-90 cm2/Vs [10] are present regardless of temperature. No clear dependence of the electrical properties on the growth conditions has been observed. Both ab initio calculations of electronic structure of GaN native defects [11] [12] and experiments [13] confirm that the source of free electrons in this GaN is N-vacancies.
The optical absorption spectra of pressure-grown GaN crystals and epitaxial GaN layers with a free electron concentration of 6x1017 cm-3 have been measured by Teisseyre et al. [14]. A shift to high energy of the optical absorption edge for the bulk sample is observed and assigned to the Burstein-Moss effect [14]. Due to this shift, the bulk GaN substrate crystals are transparent to near edge luminescence of GaN.
The dominant luminescence features of GaN bulk crystals are near band edge emission at ~3.5 eV and a broad yellow luminescence band centered around 2.3 eV.
The first results of homoepitaxial growth of GaN MOCVD layers have been recently reported [15]. Pressure grown GaN crystals were used as substrates. The layers were grown in a horizontal atmospheric pressure Epigress system adapted to the growth of nitrides. TMG and NH3 were used as sources of Ga and N, and H2 was the carrier gas. The substrates did not require any mechanical or chemical surface treatment before the growth. Also the deposition of a low temperature buffer layer was omitted. The growth temperature was 900 - 1050°C. P-doping was realized using the MCp2Mg metalorganic and subsequent annealing of the layer at 700°C in the presence of N2 gas. Si was introduced to achieve n-doping of the layers.
Atomic Force Microscopy studies performed on homoepitaxially grown GaN layers indicate the presence of monatomic steps on the layer surfaces, often in the form of large diameter (100 m) spirals indicating two dimensional growth on screw dislocations.
The X-ray diffraction peaks from the homoepitaxial layer and from GaN substrate from [7] are shown in Figure 12. The lattice constant parallel to the interface is the same for both the substrate and homolayer, whereas the c constant of the epilayer is smaller by about 0.02%. The increase of the c lattice constant of the bulk GaN crystals is related to the presence of the same point defects responsible for the high concentration of free electrons.
The high quality of the homoepitaxial layers is confirmed by the photoluminescence measurements [15]. The spectrum is dominated by two strong and narrow lines XD (3.4719 eV) and XA (3.4666 eV) related to the excitons bound to neutral impurity centers [15]. The FWHM of these lines is about1.8 meV and 1.0 meV respectively. The higher energy XD line shows a double peak structure which fitted with Lorenzian curves can be described by energies: E1 = 3.4711 eV, E2 = 3.4720 eV. The photoluminescence and reflectivity of GaN homoepitaxial layers are analyzed in detail in [15]. Also the lifetime of the bound excitons has been measured [16] to be 650 and 700 ps for excitons bound to the neutral donor and acceptor, respectively. This is much higher than the corresponding values for GaN/sapphire (70 and 140 ps, respectively).
The PL spectra for p-type Mg doped GaN homoepitaxial layers have been reported [17]. In addition to the XD and XA exciton lines (XA line much stronger than for undoped layers), a series of very narrow lines attributed to donor-acceptor pair related transitions have been observed between 3.4 - 3.5 eV. The FWHM of these lines were < 0.5 meV. The PL lines observed for MOCVD grown homoepitaxial layers are the narrowest so far reported for GaN.
Recently very high quality homoepitaxial layer have been obtained by MBE [18]. The nearedge photoluminescence spectrum of this layer is dominated by XD and XA peaks and 3 lines corresponding to free excitons. The PL lines for this layer are even sharper than for best MOCVD samples. The halfwidth of the XA line is smaller than 0.5 meV
High quality GaN crystals can be grown at nitrogen pressure up to 15 kbar. The main obstacle to growing larger substrate sized GaN crystals is instability of the growth. The instability is related to the non-uniform distribution of nitrogen in the solution due to the small volume of the crucible in relation to the size of growing GaN crystals.
It is shown that high quality homoepitaxial layers can be obtained on the "as grown", single crystalline, pressure grown GaN substrates. Further optimization of both substrate and homoepitaxial growth techniques should lead to the fabrication of efficient optoelectronic devices. Special effort has to be paid to increasing of the size of high quality GaN substrates.
[1]W. A. Harrison, Electronic Structure and Properties of Solids, Freeman, San Francisco, (1980)
[2] J. Karpinski, J. Jun, S. Porowski, J. Cryst. Growth 66, 1 (1984).
[3] J. Karpinski, S. Porowski, J. Cryst. Growth 66, 11 (1984).
[4] J. A. Van Vechten, Phys. Rev. B 7, 1479 (1973).
[5]I. Grzegory and S. Porowski, unpublished.
[6] M Leszczynski, I Grzegory, M Bockowski, J. Cryst. Growth 126, 601-604 (1993).
[7] M. Leszczynski, I. Grzegory, H. Teisseyre, T. Suski, M. Bockowski, J. Jun, S. Porowski, J. Crystal Growth, unpublished (1996).
[8] Z. Liliental-Weber, C. Kisielowski, S. Ruvimov, Y. Chen, J. Washburn, I. Grzegory, M. Bockowski, J. Jun, S. Porowski, J. Electron. Mater. 25, 1545 (1996).
[9] G. Nowak, S. Krukowski, I. Grzegory, S. Porowski, J.M. Baranowski, K. Pakula, J. Zak, MRS Internet J. Nitride Semicond. Res. 1, 5 (1996).
[10] P. Perlin , J. Camassel, W. Knap, T. Taliercio , J. C. Chervin , T. Suski, I. Grzegory, S. Porowski , Appl. Phys. Lett. 67, 2524-2526 (1995).
[11] P. Boguslawski , E. L. Briggs, J. Bernholc , Phys. Rev. B 51, 17255-17258 (1995).
[12] Jorg Neugebauer, Chris G. Van de Walle , Phys. Rev. B 50, 8067-8070 (1994).
[13] P. Perlin, T. Suski, H. Teisseyre, M. Leszczynski, I. Grzegory, J. Jun, S. Porowski, P.Boguslawski, J. Bernholc, J. C. Chervin, A. Polian, T. D. Moustakas , Phys. Rev. Lett. 75, 296-299 (1995).
[14] H. Teisseyre, P. Perlin, T. Suski, I. Grzegory, S. Porowski, J. Jun , A. Pietraszko , T. D. Moustakas , J. Appl. Phys. 76, 2429-2434 (1994).
[15] K Pakula, A Wysmolek, KP Korona, JM Baranowski, R Stepniewski, I Grzegory, M Bockowski, J Jun, S Krukowski, M Wroblewski, S Porowski, Sol. St. Comm. 97, 919-922 (1996).
[16] M. Godlewski, A. Wysmolek, K. Pakula, J. M. Baranowski, I. Grzegory, J. Jun, S. Porowski, J. P. Bergman, B. Monemarin Proc. of Intl. Symp. on Blue Lasers and Light Emitting Diodes, , (Chiba, Japan, March 5-7, , 1996) 356.
[17] A. Wysmolek, J. M. Baranowski, K. Pakula, K. P. Korona, I. Grzegory, M.Wroblewski, S. Porowskiin Proc. of Intl. Symp. on Blue Lasers and Light Emitting Diodes, , (Chiba, Japan, March 5-7, , 1996) 492.
[18] H. Teisseyre, G. Nowak, M. Leszczynski, I. Grzegory, M. Bockowski, S. Krukowski, S. Porowski, M. Mayer, A. Pelzmann, Markus Kamp , K. J. Ebeling, G. Karczewski, MRS Internet J. Nitride Semicond. Res. 1, 13 (1996).
| Figure 1. Gibbs free energy of GaN and its constituents |
| Figure 2. Equilibrium curve for GaN [2] [3] |
| Figure 3. Liquidus line for the Ga-GaN system |
| Figure 4. Experimental system for the crystal growth of GaN at N2 pressure up to 15 kbar. |
|
Figure 5. Crystallized GaN surface layer at high N2 pressure |
|
Figure 6a. GaN single crystals grown without an intentional seeding. The underlying grid is 1mm x 1mm. |
|
Figure 6b. |
|
Figure 6c. |
|
Figure 6d. |
|
Figure 7a. Edge-nucleated growth on the {0001} faces at high supersaturation. |
|
Figure 7b. Initially plate-like growth changes into the needle-like type due to the increase of supersaturation during the growth. |
|
Figure 8. GaN crystal grown in a 150h process in the crucible having an ID of 17 mm. |
| Figure 9. GaN crystal growing quickly perpendicular to the c-axis |
|
Figure 10. GaN crystal used for the intentional seeding |
|
Figure 11. GaN crystal grown on a GaN single crystalline seed. |
| Figure 12. Diffraction peaks of (0004) CuK reflection from an MOCVD grown GaN heteroepitaxial layer and GaN substrate. |
© 1996-1999 The Materials Research Society
|