List of DFDL String Literals or DFDL Expression Defines the a whitespace separated list of single characters that will appear (individually) in the data as the decimal separator. This property is applicable, when dfdl:textNumberRep is 'standard' and dfdl:textStandardBase is 10. It must be set if dfdl:textNumberPattern contains a decimal separator symbol ("."), or the E or @ symbols. (it is a schema definition error otherwise.) Empty string is not an allowable value. This property can be computed by way of an expression which returns a character. The expression must not contain forward references to elements which have not yet been processed. Text Number Character Restrictions: The the string literal is restricted to allow only certain kinds of syntax: · DFDL character entities are allowed · The DFDL byte value entity ( %#r ) is not allowed. · DFDL Character classes NL, WSP, WSP+, WSP*, and ES are not allowed It is a schema definition error if the string literal contains any of the disallowed syntax constructs. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardGroupingSeparator |
DFDL String Literal or DFDL Expression Defines the single character that will appear in the data as the grouping separator. This property is applicable when dfdl:textNumberRep is 'standard' and dfdl:textStandardBase is 10. It must be set if dfdl:textNumberPattern contains a grouping separator symbol (it is a schema definition error otherwise.) Empty string is not an allowable value. This property can be computed by way of an expression which returns a character. The expression must not contain forward references to elements which have not yet been processed. The string literal value is restricted in the same way as described in "Text Number Character Restrictions" in the description of the dfdl:textStandardDecimalSeparator property. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardExponentRep |
DFDL String Literal or DFDL Expression Defines the actual character(s) that will appear in the data as the exponent indicator. If the empty string is specified then no exponent character will be used. This property is applicable when dfdl:textNumberRep is 'standard' and dfdl:textStandardBase is 10. Empty string is an allowable value, so that formats like NNN+M (meaning NNN x 10 with MM exponent) can be expressed. This property must be set even if the dfdl:textNumberPattern does not contain an 'E' (exponent) character. It is a schema definition error if this property is not set or in scope for any number with dfdl:representation 'text'. This property can be computed by way of an expression which returns a DFDL String Literal character. The expression must not contain forward references to elements which have not yet been processed. The string literal value is restricted in the same way as described in "Text Number Character Restrictions" in the description of the dfdl:textStandardDecimalSeparator property. If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardInfinityRep |
DFDL String Literal The value used to represent infinity. Infinity is represented as a string with the positive or negative prefixes and suffixes from the dfdl:textNumberPattern applied. This property is applicable when dfdl:textNumberRep is 'standard', dfdl:textStandardBase is 10 and the simple type is float or double. If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. The string literal value is restricted in the same way as described in "Text Number Character Restrictions" in the description of the dfdl:textStandardDecimalSeparator property. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardNaNRep |
DFDL String Literal The value used to represent NaN. NaN is represented as a string and the positive or negative prefixes and suffixes from the dfdl:textNumberPattern are not used. This property is applicable when dfdl:textNumberRep is 'standard', dfdl:textStandardBase is 10 and the simple type is float or double. If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. The string literal value is restricted in the same way as described in "Text Number Character Restrictions" in the description of the dfdl:textStandardDecimalSeparator property. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardZeroRep |
List of DFDL String Literals Valid values: empty string, any character string The whitespace separated list of alternative literal strings that are equivalent to zero, for example the characters 'zero'. The representation is examined for a match to one of the values of this property after padding has been trimmed away. On unparsing the first value is used. If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. The empty string means that there is no special literal string for zero. This property is applicable when dfdl:textNumberRep is 'standard' and dfdl:textStandardBase is 10. Each string literal in the list is restricted to allow only certain kinds of syntax: · DFDL character entities are allowed. · DFDL Byte Value entities ( %#r ) are not allowed. · DFDL Character class entities NL and ES are not allowed. · DFDL Character class entities WSP, WSP+, and WSP* are allowed. However, the WSP* entity cannot appear on its own as one of the string literals in the list. It must be used in combination with other text characters or entities so as to describe a representation that cannot ever be an empty string. It is a schema definition error if the string literal contains any of the disallowed syntax constructs. Annotation: dfdl:element, dfdl:simpleType |
|
textStandardBase |
Non-negative Integer Valid Values 2, 8, 10, 16 Indicates the number base. Only used when dfdl:textNumberRep is 'standard'. When base is not 10, xs:decimal, xs:float and xs:double are not supported. When dfdl:textNumberRep is 'zoned' dfdl:textNumberBase 10 is not used and base 10 is assumed. Annotation: dfdl:element, dfdl:simpleType |
|
textZonedSignStyle |
Enum Specifies the code points that are used to overpunch the sign nibble when the dfdl:encoding is an ASCII-derived character set encoding. The location of this sign nibble is indicated in the dfdl:textNumberPattern. This property is applicable when dfdl:textNumberRep is 'zoned'. Used only when dfdl:encoding is an ASCII-derived character set encoding. The encoding must provide the character to single byte code point mapping used by the specified value of dfdl:textZonedSignStyle, as stated below. Valid values 'asciiStandard', 'asciiTranslatedEBCDIC', 'asciiCARealiaModified', and 'asciiTandemModified' Which characters are used to represent 'overpunched' (included) positive and negative signs, varies by encoding, Cobol compiler and system. The code points are fixed for EBCDIC systems but not for ASCII. In EBCDIC-based encodings, code points 0xC0 to 0xC9 or 0xF0 to 0xF9 represent a positive sign and digits 0 to 9 (typically characters '{ABCDEFGHI' or '0123456789'), and code points 0xD0 to 0xD9 or 0xB0 to 0xB9 represent a negative sign and digits 0 to 9 (typically characters '}JKLMNOPQR' or '^£¥·©§¶¼½¾ ' ). On parsing both ranges will be accepted. On unparsing the range 0xC0 to 0xC9 will be produced for positive signs and the range 0xD0 to 0xD9 will be produced for negative signs. asciiStandard: ASCII characters '0123456789' represent a positive sign and the corresponding digit. (Sign nibble for '+' is 0x3, which is the high nibble of these code points unmodified.) ASCII characters 'pqrstuvwxy' represent negative sign and digits 0 to 9. (Code points 0x70 to 0x79) asciiTranslatedEBCDIC: The overpunched character is the ASCII equivalent of the typical EBCDIC above. So the characters '{ABCDEFGHI' still represent a positive sign and digits 0 to 9. (These are code points 0x7B, 0x41 through 0x49). The characters '}JKLMNOPQR' still represent negative sign and digits 0 to 9. (These are code points 0x7D, 0x4A through 0x52). This case comes up if EBCDIC zoned decimal data is translated to ASCII as if it were textual data.) asciiCARealiaModified[22]: In this style, the ASCII characters '0123456789' represent positive sign and digits 0 to 9 as in standard. However, ASCII characters from code points 0x20 to 0x29 are used for negative sign and the corresponding decimal digit. This doesn't translate well into printing characters. These characters include the space (' ') for zero, characters '!"#$%&' for 1 through 6, the single quote character "'" for 7, and the parenthesis '()' for 8 and 9. asciiTandemModified: In this style the ASCII characters '0123456789' represent positive sign and digits 0 to 9, but code points 0x80 to 0x89 are used to represent negative sign and a digit. There are no corresponding code points in the standard ASCII encoding since these values are all above 128 (decimal). This means the resultant bytes are not code points in standard ASCII, so the modeller must specify an encoding like ISO-8859-1 in order for such zoned decimals to parse without an encoding error. (Note that neither ISO-8859-1 encoding nor Unicode have assigned glyphs for these code points. They are considered control characters.) Annotation: dfdl:element, dfdl:simpleType |
Table 31 Properties Specific to Number with Text Representation
The dfdl:textStandardDecimalSeparator, dfdl:textStandardGroupingSeparator, dfdl:textStandardExponentRep, dfdl:textStandardInfinityRep, dfdl:textStandardNaNRep, and dfdl:textStandardZeroRep must all be distinct, and it is a schema definition error otherwise. Note that if dfdl:textStandardDecimalSeparator, dfdl:textStandardGroupingSeparator, or dfdl:textStandardExponentRep are expressions, this checking can only be carried out during processing (parsing or unparsing.)
Implementation note: This rule is in the interests of clarity, and is an extra constraint compared to ICU.
The dfdl:textNumberPattern describes how to parse and unparse text representations of number logical types with base 10.
The length of the representation of the number is determined first, and the number pattern is used only for conversion of the content text to and from a numeric logical infoset value.
The pattern described below is derived from the ICU DecimalFormat class described here: [ICUDecimal]
The pattern is an ICU-like syntax that defines where grouping separators, decimal separators, implied decimal points, exponents, positive signs and negative signs appear. It permits definition by either digits/fractions or significant digits.
When dfdl:textNumberRep is 'standard' this property only applies when dfdl:textStandardBase is 10.
The pattern comes in two parts separated by a semi-colon. The first is mandatory and applies to positive numbers, the second is optional and applies to negative numbers.
Examples: The first shows digits/fractions and positive/negative signs, the second shows exponent, the third shows virtual decimal point, the fourth shows scaling position.
+###,##0.00;(###,##0.00)
##0.0#E0
000V00
PPP0000
The 'V' symbol is used to indicate the location of an implied decimal point for fixed point number representations. (This is an extension to the ICU pattern language.)
The 'P' symbol is used to indicate that a decimal scaling factor needs to be applied. (This is an extension to the ICU pattern language.)
The actual grouping separator, decimal separator and exponent characters are defined independently of the pattern.
The actual positive sign and negative sign are defined within the pattern itself.
Many characters in a pattern are taken literally; they are matched during parsing and output unchanged during unparsing. Special characters, on the other hand, stand for other characters, strings, or classes of characters. For example, the '#' character is replaced by a digit.
To insert a special character in a pattern as a literal, that is, without any special meaning, the character must be quoted. There are some exceptions to this which are noted below.
|
Symbol |
Location |
Meaning |
|
0 |
Number |
Digit |
|
1-9 |
Number |
'1' through '9' indicates rounding. |
|
# |
Number |
Digit, zero shows as absent |
|
. |
Number |
Decimal separator or monetary decimal separator |
|
- |
Number |
Minus sign |
|
, |
Number |
Grouping separator |
|
E |
Number |
Separates mantissa and exponent in scientific notation. Need not be quoted in prefix or suffix. |
|
+ |
Exponent |
Prefix positive exponents with plus sign. Need not be quoted in prefix or suffix. |
|
; |
Subpattern boundary |
Separates positive and negative subpatterns |
|
' |
Prefix or suffix |
Used to quote special characters in a prefix or suffix, for example, "'#'#" formats 123 to "#123". To create a single quote itself, use two in a row: "# o''clock". |
|
* |
Prefix or suffix boundary |
Pad escape, precedes pad character |
|
V |
Number |
Virtual decimal point marker. Only used with decimal, float and double simple types. |
|
P |
Number |
Decimal scaling position. Only used with decimal, float and double simple types. |
|
@ |
Number |
Significant digits specifier. Only used with decimal simple type. Controls number of significant digits when used alone or in conjunction with the # character. |
Table 32 dfdl:textNumberPattern Special Characters
A pattern contains a positive and negative subpattern, for example, "#,##0.00;(#,##0.00)". Each subpattern has a prefix, a numeric part, and a suffix. If there is no explicit negative subpattern, the negative subpattern is the minus sign prefixed to the positive subpattern. That is, "0.00" alone is equivalent to "0.00;-0.00". If there is an explicit negative subpattern, it serves only to specify the negative prefix and suffix; the number of digits, minimal digits, and other characteristics are ignored in the negative subpattern. That means that "#,##0.0#;(#)" has precisely the same result as "#,##0.0#;(#,##0.0#)".
The prefixes, suffixes, and various symbols used for infinity, digits, grouping separators, decimal separators, etc. may be set to arbitrary values, and they will appear properly during unparsing. However, care must be taken that the symbols and strings do not conflict, or parsing will be unreliable. For example, either the positive and negative prefixes or the suffixes must be distinct for parse to be able to distinguish positive from negative values.
The grouping separator is a character that separates clusters of integer digits to make large numbers more legible. It commonly used for thousands, but in some locales it separates ten-thousands. The grouping size is the number of digits between the grouping separators, such as 3 for "100,000,000" or 4 for "1 0000 0000". There are actually two different grouping sizes: One used for the least significant integer digits, the primary grouping size, and one used for all others, the secondary grouping size. In most locales these are the same, but sometimes they are different. For example, if the primary grouping interval is 3, and the secondary is 2, then this corresponds to the pattern "#,##,##0", and the number 123456789 is formatted as "12,34,56,789". If a pattern contains multiple grouping separators, the interval between the last one and the end of the integer defines the primary grouping size, and the interval between the last two defines the secondary grouping size. All others are ignored, so "#,##,###,####" == "###,###,####" == "##,#,###,####".
The P symbol is used to derive the location of an assumed decimal point when the point is not within the number that appears in the data. It acts as a decimal scaling factor.
The symbol P can be specified only as a continuous string of Ps in the leftmost or rightmost digit positions in the vpinteger region of the pattern.
It is a schema definition error if any symbols other than "0", "1" through "9" or # are used in the vpinteger region of the pattern.
Examples
|
Data representation |
Pattern |
Value |
|
123 |
PP000 |
0.00123 |
|
123 |
000PP |
12300 |
Table 33 Examples of P Symbol in the dfdl:textNumberPattern Property
pattern := subpattern (';' subpattern)?
subpattern := prefix? ((number exponent?)| vpinteger) suffix?
number := (integer ('.' fraction)?) | sigdigits
vpinteger := pinteger | (vinteger exponent?)
pinteger := ('P'* integer) | (integer 'P'* )
vinteger := ('V'? integer) |
('#'* 'V'? integer)|
('#'* '0'* 'V'? '0'* '0')|
(integer 'V'?)
prefix := '\u0000'..'\uFFFD' - specialCharacters
suffix := '\u0000'..'\uFFFD' - specialCharacters
integer := '#'* '0'* '0'
fraction := '0'* '#'*
sigDigits := '#'* '@' '@'* '#'*
exponent := 'E'? '+'? '0'* '0'
padSpec := '*' padChar
padChar := '\u0000'..'\uFFFD' - quote
Notation:
X* 0 or more instances of X
X? 0 or 1 instances of X
X|Y either X or Y
C..D any character from C up to D, inclusive
S-T characters in S, except those in T
Figure 4 dfdl:textNumberPattern BNF syntax
The first subpattern is for positive numbers. The second (optional) subpattern is for negative numbers.
Not indicated in the BNF syntax above:
The grouping separator ',' can occur inside the integer region, between any two pattern characters of that region, as long as the number region is not followed by an exponent region.
Two grouping intervals are recognized: That between the decimal point and the first grouping symbol, and that between the first and second grouping symbols. These intervals are identical in most locales, but in some locales they differ. For example, the pattern "#,##,###" formats the number 123456789 as "12,34,56,789".
The pad specifier padSpec may appear before the prefix, after the prefix, before the suffix, after the suffix, or not at all.
In place of '0', the digits '1' through '9' in the number or vpinteger region may be used to indicate a rounding increment.
The term maximum fraction digits is the total number of '0' and '#' characters in the fraction sub-pattern above.
The term minimum fraction digits is the total number of '0' characters (only) in the fraction sub-pattern above.
The term maximum integer digits is a limit that is implementation-dependent, but must be at least 20 (which is the number of digits in a base 10 unsigned long).[23]
The term minimum integer digits is the total number of '0' characters (only) in the integer sub-pattern above.
Parsing
During parsing, grouping separators are removed from the data.
Unparsing
Unparsing is guided by several parameters all of which can be specified using a pattern. The following description applies to formats that do not use scientific notation.
If the number of actual integer digits exceeds the maximum integer digits, then only the least significant digits are shown. For example, 1997 is formatted as "97" if the maximum integer digits is 2.
If the number of actual integer digits is less than the minimum integer digits, then leading zeros are added. For example, 1997 is formatted as "01997" if the minimum integer digits is 5.
If the number of actual fraction digits exceeds the maximum fraction digits, then half-even rounding it performed to the maximum fraction digits. For example, 0.125 is formatted as "0.12" if the maximum fraction digits is 2. This behavior can be changed by specifying a rounding increment and a rounding mode.
If the number of actual fraction digits is less than the minimum fraction digits, then trailing zeros are added. For example, 0.125 is formatted as "0.1250" if the minimum fraction digits is 4.
Trailing fractional zeros are not displayed if they occur j positions after the decimal, where j is less than the maximum fraction digits. For example, 0.10004 is formatted as "0.1" if the maximum fraction digits is four or less.
Special Values
NaN is represented as a string determined by the dfdl:textStandardNaNRep property. This is the only value for which the prefixes and suffixes are not used.
Infinity is represented as a string with the positive or negative prefixes and suffixes applied. The infinity string is determined by the dfdl:textStandardInfinityRep property.
Numbers in scientific notation are expressed as the product of a mantissa and a power of ten, for example, 1234 can be expressed as 1.234 x 103. The mantissa is typically in the half-open interval [1.0, 10.0) or sometimes [0.0, 1.0), but it need not be. In a pattern, the exponent character immediately followed by one or more digit characters indicates scientific notation. Example: "0.###E0" formats the number 1234 as "1.234E3".
The number of digit characters after the exponent character gives the minimum exponent digit count. There is no maximum. Negative exponents are formatted using the minus sign, not the prefix and suffix from the pattern. This allows patterns such as "0.###E0 m/s". To prefix positive exponents with a plus sign, specify '+' between the exponent and the digits: "0.###E+0" will produce formats "1E+1", "1E+0", "1E-1", etc.
The minimum number of integer digits is achieved by adjusting the exponent. Example: 0.00123 formatted with "00.###E0" yields "12.3E-4". This only happens if there is no maximum number of integer digits. If there is a maximum, then the minimum number of integer digits is fixed at one.
The maximum number of integer digits, if present, specifies the exponent grouping. The most common use of this is to generate engineering notation, in which the exponent is a multiple of three, e.g., "##0.###E0". The number 12345 is formatted using "##0.####E0" as "12.345E3".
When using scientific notation, the formatter controls the digit counts using significant digits logic. The maximum number of significant digits limits the total number of integer and fraction digits that will be shown in the mantissa; it does not affect parsing. For example, 12345 formatted with "##0.##E0" is "12.3E3". .
Exponential patterns may not contain grouping separators.
Significant Digits
The '@' pattern character can be used with the '#' to control how many integer and fraction digits are needed to display the specified number of significant digits. The '@' only affects unparsing behavior. Examples:
|
Pattern |
Minimum significant digits |
Maximum significant digits |
Number |
Formatted Output |
|
@@@ |
3 |
3 |
12345 |
12300 |
|
@@@ |
3 |
3 |
0.12345 |
0.123 |
|
@@## |
2 |
4 |
3.14159 |
3.142 |
|
@@## |
2 |
4 |
1.23004 |
1.23 |
Table 34 Significant Digits '@' Symbol in the dfdl:textNumberPattern Property
Padding
Padding may be specified through the pattern syntax. In a pattern the pad escape character, followed by a single pad character, causes padding to be parsed and formatted. The pad escape character is '*'. For example, "*x#,##0.00" formats 123 to "xx123.00", and 1234 to "1,234.00".
When padding is in effect, the width of the positive subpattern, including prefix and suffix, determines the format width. For example, in the pattern "* #0 o''clock", the format width is 10.
The width is counted in 16-bit code units.
Some parameters which usually do not matter have meaning when padding is used, because the pattern width is significant with padding. In the pattern "* ##,##,#,##0.##", the format width is 14. The initial characters "##,##," do not affect the grouping size or maximum integer digits, but they do affect the format width.
Padding may be inserted at one of four locations: before the prefix, after the prefix, before the suffix, or after the suffix. If there is no prefix, before the prefix and after the prefix are equivalent, likewise for the suffix.
When specified in a pattern, the 32-bit codepoint immediately following the pad escape is the pad character. This may be any character, including a special pattern character. That is, the pad escape escapes the following character. If there is no character after the pad escape, then the pattern is illegal.
Note: Padding specified through the pattern syntax is distinct from, and in addition to, padding specified using dfdl:textPadKind.
Rounding
How rounding is controlled is given by dfdl:textNumberRounding. The rounding increment may be specified in the dfdl:textNumberPattern itself using digits '1' through '9' or using an explicit increment in dfdl:textNumberRoundingIncrement. For example, 1230 rounded to the nearest 50 is 1250. 1.234 rounded to the nearest 0.65 is 1.3.
Using an explicit rounding increment, dfdl:textNumberRoundingMode determines how values are rounded.
When dfdl:textNumberRep is 'zoned' a subset of the number pattern language described in Section <IP_ADDRESS> dfdl:textNumberPattern for dfdl:textNumberRep 'standard' is used.
Only the pattern for positive numbers is used. It is a schema definition error if the negative pattern is specified.
In addition, only the following pattern characters may be used:
Rounding occurs as described under Rounding in <IP_ADDRESS> dfdl:textNumberPattern for dfdl:textNumberRep 'standard'
· Signed numbers with dfdl:textNumberRep 'standard' and dfdl:textStandardBase 10 are mapped using the dfdl:textNumberPattern.
· Signed numbers with dfdl:textNumberRep 'standard' and dfdl:textStandardBase not 10 are mapped to an unsigned representation. On unparsing the minimum number of characters to represent the digits is output and it is a processing error if the value is negative.
These properties are applicable to simple type xs:decimal and its derived types which include all the signed and unsigned integer types. These properties are not applicable to types xs:float and xs: double. See section 13.8. Note that simple types derived from xs:decimal do not imply base-10 representations in the data stream.
|
Property Name |
Description |
||||||||
|
binaryNumberRep |
Enum Valid values are 'packed', 'bcd', 'binary', 'ibm4690Packed' Allowable values for each number type are:
'packed' means represented as an IBM 390 packed decimal. Each byte contains two decimal digits, except for the least significant byte, which contains a sign in the least significant nibble. 'bcd' means represented as a binary coded decimal with two digits per byte. 'binary' means represented as twos complement for signed types and unsigned base-2 binary for unsigned types. Note that the maximum allowed value for twos-complement and unsigned base-2 binary integers is implementation-dependent, but must be at least that of a xs:long type, which is the equivalent of an 8 byte/64-bit signed integer. 'ibm4690Packed' is a variant of a packed decimal having the following characteristics:
For all values, the dfdl:byteOrder property is used to determine the numeric significance of the bytes making up the representation, and the dfdl:bitOrder property is used to determine the numeric significance of the bits within a byte. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
binaryDecimalVirtualPoint |
Integer. Used when base simpleType is xs:decimal. An integer that represents the position of an implied decimal point within a number, or specify 0. If you specify 0 then there is no virtual decimal point If you specify a positive integer, the position of the decimal point is moved from the least-signficant side of the number toward the most-significant side of the number. For example, if 3 is specified then, the integer value 1234 represents 1.234. This is equivalent to dividing by 103. If you specify a negative integer, the position of the decimal point is moved from the least significant side of the number further in the less-significant direction. For example, if you specify -3, the integer value 1234 represents 1 234 000.This is equivalent to multiplying by 103. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
binaryPackedSignCodes |
List of Characters Used only when dfdl:binaryNumberRep or dfdl:binaryCalendarRep is 'packed' A whitespace separated string giving the hex sign nibbles to use for a positive value, a negative value, an unsigned value, and zero. Valid values for positive nibble: A, C, E, F Valid values for negative nibble: B, D Valid values for unsigned nibble: F Valid values for zero sign: A C E F 0 Example: 'C D F C' – typical S/390 usage Example: 'C D F 0' – handle special case for zero On parsing, whether to accept all valid values for a positive, negative or unsigned number, and for zero, is governed by the dfdl:binaryNumberCheckPolicy property. On unparsing, the specified values are always used. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
binaryNumberCheckPolicy |
Enum Values are 'strict' and 'lax'. Indicates how lenient to be when parsing binary numbers. If 'lax' then the parser tolerates all valid alternatives where such alternatives exist. Specifically, for dfdl:binaryNumberRep 'packed' the sign nibble for positive, negative, unsigned and zero is allowed to be any of the valid respective values. On unparsing, the specified value is always used Annotation: dfdl:element, dfdl:simpleType |
Table 35 Properties Specific to Number with Binary Representation
When unparsing a binary number (packed decimal or twos-complement) and excess precision is supplied in the Infoset no rounding occurs. It is a processing error.
For both parsing and unparsing, the bit string that represents the content region for a base-2 binary number is converted to/from an Infoset value by a calculation that involves the length and the dfdl:byteOrder property.
For unparsing, the dfdl:fillByte property can also be involved.
When parsing, DFDL specifies how an unsigned integer of unbounded magnitude is computed from a bit string based on its length, and the dfdl:byteOrder property. For signed types, this unbounded integer is converted into a signed value by way of the well-known twos-complement scheme, and for the xs:decimal type, the dfdl:binaryDecimalVirtualPoint property can be used to convert this integer into a decimal value with an integer and a fractional component, and for both xs:decimal and the integer types the dfdl:binaryVirtualDecimalPoint or to scale up the integer by some scale factor.
A DFDL implementation can use any conversion technique consistent with this description.
The dfdl:alignmentUnits of 'bits', and dfdl:alignment of '1' can be used to position a bit string anywhere in the data stream without regard for any other grouping of bits into bytes.
The numeric value of the unsigned integer represented by a bit string is unaffected by alignment.
When unparsing a bit string, alignment may cause the bits within the bit string to occupy only some of the bits within a byte of the data stream. The bits of data in the alignment fill region are unspecified by the elements of the DFDL schema, and when parsing, neither they, nor any data computed from them are put into the DFDL infoset. During unparsing, such unspecified bits are filled in using the value of the dfdl:fillByte property. Corresponding bits from the dfdl:fillByte value are used to fill in unspecified bits of the data stream. That is, if bit K (K will be 1 or greater, but less than or equal to 8) of a data stream byte is unspecified, its value will be taken from bit K of the dfdl:fillByte property value.
Since the value of any bit string element is unaffected by alignment, the logical unsigned integer value for a bit-string is always computed as if the first bit were at position 1 of the bit stream. If the dfdl:length for the bit-string evaluates to M, then the bit-string conceptually occupies bits 1 to M of a data stream for purposes of computing its value.
Any time the length in bits is < 8, then when set, the bit at position Z supplies value 2^(M-Z), and the value of the bit string as an integer is the sum of these values for each of its bits.
Call M the length of the bit string element in bits. In general, when M > 8 the contribution of a bit in position i to the numeric value of a bit string is given by a formula specific to the dfdl:byteOrder.
For dfdl:byteOrder of 'bigEndian' the value of bit i is given by 2^(M - i).
For dfdl:byteOrder of 'littleEndian' the value of bit i is given by a more complex formula. The following pseudo code computes the value of a bit in a littleEndian bit string. It is just a very big expression, but is spread out over many local variables to illustrate the various sub-calculations clearly. DFDL implementations may use any way of converting bit strings to the corresponding integer values that is consistent with this:
In the pseudo code below:
· '%' is modular division (division where remainder is returned)
· '/' is regular division (quotient is returned)
· the expression 'a ? b : c' means 'if a is true, then the value is b, otherwise the value is c'
littleEndianBitValue(bitPosition, bitStringLength)
assert bitPosition >= 1;
assert bitStringLength >= 1;
assert bitStringLength >= bitPosition;
numBitsInFinalPartialByte = bitStringLength % 8;
numBitsInWholeBytes = bitStringLength -
numBitsInFinalPartialByte;
bitPosInByte = ((bitPosition - 1) % 8) + 1;
widthOfActiveBitsInByte = (bitPosition <= numBitsInWholeBytes)
? 8 : numBitsInFinalPartialByte;
placeValueExponentOfBitInByte = widthOfActiveBitsInByte –
bitPosInByte;
bitValueInByte = 2^placeValueExponentOfBitInByte;
byteNumZeroBased = (bitPosition - 1)/8;
scaleFactorForBytePosition = 2^(8 * byteNumZeroBased);
bitValue = bitValueInByte * scaleFactorForBytePosition;
return bitValue;
Figure 5 Little Endian bit position and value
Consider the first three bytes of the data stream. Imagine their numeric values as 0x5A 0x92 0x00.
Positions:
00000000 01111111 11122222
12345678 90123456 78901234
Bits:
01011010 10010010 00000000
Hex values
5 A 9 2 0 0
Beginning at bit position 1, (the very first bit) if we consider the first two bytes as a bigEndian short, the value will be 0x5A92.
<xs:element name="num" type="unsignedShort"
dfdl:alignment="1"
dfdl:alignmentUnits="bytes"
dfdl:byteOrder="bigEndian"
dfdl:representation="binary"
dfdl:binaryNumberRep="binary"/>
As a littleEndian short, the value will be 0x925A.
<xs:element name="num" type="unsignedShort"
dfdl:alignment="1"
dfdl:alignmentUnits="bytes"
dfdl:byteOrder="littleEndian"
dfdl:representation="binary"
dfdl:binaryNumberRep="binary"/>
Now let us examine a bit string of length 13, beginning at position 2.
<xs:sequence>
<xs:element name="ignored" type="unsignedByte"
dfdl:alignment="1"
dfdl:alignmentUnits="bits"
dfdl:lengthUnits="bits"
dfdl:length="1"
dfdl:representation="binary"
dfdl:binaryNumberRep="binary"/>
<xs:element name="x" type="unsignedShort"
dfdl:alignment="1"
dfdl:alignmentUnits="bits"
dfdl:byteOrder="bigEndian"
dfdl:lengthUnits="bits"
dfdl:length="13"
dfdl:representation="binary"
dfdl:binaryNumberRep="binary"/>
...
</xs:sequence>
Let's examine the same data stream and consider the bit positions that make up element 'x', which are the bits at positions 2 through 14 inclusive.
Positions:
00000000 01111111 11122222
12345678 90123456 78901234
Bits:
1011010 100100
Since alignment does not affect logical value, we will obtain the same logical value as if we realigned the bits. That is, the value is the same as if we began the bits of the element's representation with bit position 1.
Realigned
Positions:
00000000 01111111 11122222
12345678 90123456 78901234
Bits:
10110101 00100
The DFDL schema fragment above gives element 'x' the dfdl:byteOrder 'bigEndian' property. In this case the place value of each position is given by 2^(M – i)
PlaceValue positions 2^(M - i)
...11110 00000000
...21098 76543210
Bit values
...10110 10100100
Hex values
1 6 A 4
The value of element 'x' is 0x16A4. Notice how it is the most-significant byte -- which is the first byte when big endian -- that becomes the partial byte (having fewer than 8 bits) in the case where the length of the bit string is not a multiple of 8 bits.
For dfdl:byteOrder of 'littleEndian'. The place values of the individual bits are not as easily visualized. However there is still a basic formula (given in the pseudo code in <IP_ADDRESS> Bits within Bit Strings of Length > 8) and value.
Looking again at our realigned positions:
Realigned
Positions:
00000000 01111111 11122222
12345678 90123456 78901234
Bits:
10110101 00100
The place values of each of these bits, for little endian byte order can be seen to be:
PlaceValue positions
00000000 ...11100
76543210 ...21098
Bit values
10110101 ...00100
Hex values
B 5 0 4
We must reorder the bytes for little endian byte order. The value of element 'x' is 0x04B5. In little endian form, the first 8 bits make up the first byte, and that contains the least-significant byte of the logical numeric unsignedShort value. The additional bits of the partial byte are once again the most significant byte; however, for little endian form, this is the second byte. The second byte contains only 5 bits, those make up the least significant 5 bits of that byte, but that logical 5-bit value makes up the most-significant byte of the unsignedShort integer.
Now let us examine the 13 bits beginning at position 2, in the context where dfdl:byteOrder is 'littleEndian' and dfdl:bitOrder is 'leastSignificantBitFirst' and dfdl:binaryNumberRep is 'binary'.
In this case, the bit positions are assigned differently. Below the bytes are shown left-to-right:
Positions:
00000000 11111110 22222111
87654321 65432109 43210987
Bits:
01011010 10010010
00000000
Hex values
5 A 9 2 0 0
The bits of interest are highlighted above. If we redisplay this same data, but reversing the order of the bytes to right-to-left, then we get:
Positions:
22222111 11111110 00000000
43210987 65432109 87654321
Bits:
00000000 10010010 01011010
Hex values
0 0 9 2 5 A
The above shows more clearly that we are looking at a contiguous region of bits containing
0 1001 0010 1101
or the value 0x092D.
Signed numbers with dfdl:binaryNumberRep 'packed' are parsed using a nibble to indicate the sign. The unsigned nibble is treated as positive. On unparsing the sign nibble is written according to dfdl:binaryPackedSignCodes. The unsigned nibble is never written.
Signed numbers with dfdl:binaryNumberRep 'bcd' are always positive. On unparsing it is a processing error if the Infoset data is negative.
Signed numbers with dfdl:binaryNumberRep 'ibm4690Packed' are parsed using the sign nibble to identify negative values. There is no sign nibble for positive values. On unparsing the nibble 0xD is written for negative values.
Unsigned numbers with dfdl:binaryNumberRep 'packed' are parsed if the nibble is positive or unsigned. It is a processing error if the data is negative. On unparsing the unsigned nibble is used.
Unsigned numbers with dfdl:binaryNumberRep 'bcd' are readily parsed as BCD data is always positive.
Unsigned numbers with dfdl:binaryNumberRep 'ibm4690Packed' are parsed if there is no sign nibble of 0xD to identify a negative value. It is a processing error if the data is negative. On unparsing no sign nibble is written.
|
Property Name |
Description |
|
binaryFloatRep |
Enum or DFDL Expression This specifies the encoding method for the float and double. Valid values are 'ieee', 'ibm390Hex',This property can be computed by way of an expression which returns a string of 'ieee' or ' ibm390Hex' . The expression must not contain forward references to elements which have not yet been processed. The enumeration value 'ieee' refers to the IEEE 754-1985 specification. For both 'ieee' and 'ibm390hex', an xs:float must have a physical length of 4 bytes. It is a schema definition error if there is a specified length not equivalent to 4 bytes. Similarly, for both 'ieee' and 'ibm390hex', an xs:double must have a physical length of 8 bytes. It is a schema definition error if there is a specified length not equivalent to 8 bytes. The dfdl:byteOrder property is used to construct a value from the bytes in the binary representation. Note: The DFDL Infoset float and double data types match the precision of the IEEE specification. There may be precision/rounding issues when converting IBM float/double to/from the DFDL infoset float/double types. Half-precision IEEE and quad-precision IEEE/IBM are not supported.[24] Annotation: dfdl:element, dfdl:simpleType |
Table 36 Properties Specific to Float/Double with Binary Representation
|
Property Name |
Description |
|
textBooleanTrueRep |
List of DFDL String Literals or DFDL Expression A whitespace separated list of representations to be used for 'true'. These are compared after trimming when parsing, and before padding when unparsing. If dfdl:lengthKind is 'explicit' or 'implicit' and either dfdl:textPadKind or dfdl:textTrimKind is 'none' then both dfdl:textBooleanTrueRep and dfdl:textBooleanFalseRep must have the same length else it is a schema definition error. This property can be computed by way of an expression which returns a string of whitespace separated list of values. The expression must not contain forward references to elements which have not yet been processed. On unparsing the first value is used If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. Text Boolean Character Restrictions: The string literal is restricted to allow only certain kinds of syntax: · DFDL character entities are allowed · The DFDL byte value entity ( %#r ) is not allowed. · DFDL Character classes NL, WSP, WSP+, WSP*, and ES are not allowed It is a schema definition error if the string literal contains any of the disallowed constructs. Annotation: dfdl:element, dfdl:simpleType |
|
textBooleanFalseRep |
List of DFDL String Literals or DFDL Expression A whitespace separated list of representations to be used for 'false' These are compared after trimming when parsing, and before padding when unparsing. If dfdl:lengthKind is 'explicit' or 'implicit' and either dfdl:textPadKind or dfdl:textTrimKind is 'none' then both dfdl:textBooleanTrueRep and dfdl:textBooleanFalseRep must have the same length else it is a schema definition error. This property can be computed by way of an expression which returns a string of whitespace separated list of values. The expression must not contain forward references to elements which have not yet been processed. On unparsing the first value is used If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. The string literal value is restricted in the same way as described in "Text Boolean Character Restrictions" in the description of the dfdl:textBooleanTrueRep property. Annotation: dfdl:element, dfdl:simpleType |
|
textBooleanJustification |
Enum Valid values 'left', 'right', 'center' Controls how the data is padded or trimmed on parsing and unparsing. Behavior as for dfdl:textStringJustification. Annotation: dfdl:element, dfdl:simpleType |
|
textBooleanPadCharacter |
DFDL String Literal The value that is used when padding or trimming boolean
elements. The value can be a single character or a single byte. If a byte, then it must be specified using a single byte value entity. If a pad character is specified when lengthUnits is 'bytes' then the pad character must be a single-byte character. If a pad byte is specified when lengthUnits is 'characters' then
The string literal value is restricted in the same way as described in "Pad Character Restrictions" in the description of the dfdl:textStringPadCharacter property. Annotation: dfdl:element, dfdl:simpleType |
Table 37 Properties Specific to Boolean with Text Representation
|
Property Name |
Description |
|
binaryBooleanTrueRep |
Non-negative Integer This value, treated as a binary xs:unsignedInt (See Section 13.7.1 Converting Logical Numbers to/from Binary Representation ), gives the representation to be used for 'true' If this property value is the empty string, when parsing it means dfdl:binaryBooleanTrueRep is any value other than dfdl:binaryBooleanFalseRep; when unparsing, the one's complement of the dfdl:binaryBooleanFalseRep will be used. The length of the data value of the element must be between 1 bit and 32 bits (4 bytes) as described in Section <IP_ADDRESS>. It is a schema definition error if the value (when provided) of dfdl:binaryBooleanTrueRep cannot fit as an unsigned binary integer in the specified length. Annotation: dfdl:element, dfdl:simpleType |
|
binaryBooleanFalseRep |
Non-negative Integer This value, treated as a binary xs:unsignedInt (See Section 13.7.1 Converting Logical Numbers to/from Binary Representation ), gives the representation to be used for 'false' The length of the data value of the element must be between 1 bit and 32 bits (4 bytes) as described in Section <IP_ADDRESS>. It is a schema definition error if the valuef dfdl:binaryBooleanFalseRep cannot fit as an unsigned binary integer in the specified length. Annotation: dfdl:element, dfdl:simpleType |
Table 38 Properties Specific to Boolean with Binary Representation
The properties describe how a calendar is to be interpreted including a unparsing pattern property plus properties that qualify the pattern.
These properties can be used when a calendar has dfdl:representation 'text' or dfdl:representation 'binary' and a packed decimal representation.
|
Property Name |
Description |
||||||||
|
calendarPattern |
String Defines the ICU pattern that describes the format of the calendar. The pattern defines where the year, month, day, hour, minute, second, fractional second and time zone components appear. See calendarPattern property section below. When the dfdl:representation is 'binary' and the representation is a packed decimal then the pattern can contain only characters and symbols that always result in the presentation of digits. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarPatternKind |
Enum Valid values 'explicit', 'implicit' 'explicit' means the pattern is given by dfdl:calendarPattern, 'implicit' means the pattern is derived from the XML schema date/time type.
Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarCheckPolicy |
Enum Valid values are 'strict', 'lax' Indicates how lenient to be when parsing against the pattern. See Section 13.11.2 The dfdl:calendarCheckPolicy Property below for details of the specific behaviors for 'strict' and 'lax'. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarTimeZone |
String This property provides the time zone that will be assumed if no time zone explicitly occurs in the data. Valid values specify a UTC time zone offset by matching the regular expression: (UTC)([+\-]([01]\d|\d)((([:][0-5]\d){1,2})?))?) In addition, empty string can be specified to indicate "no time zone", or the IANA time zone format (also known as the Olson time zone format) may be used. (e.g, America/New_York)) See [IANATimeZone]. Note that this property is used when parsing only. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarObserveDST |
Enum Valid values are 'yes', 'no' Whether the time zone given in dfdl:calendarTimeZone observes daylight savings time. Ignored if dfdl:calendarTimeZone is specified in UTC format, or if dfdl:calendarTimeZone is empty string. That is, this property is used only if the dfdl:calendarTimeZone is in IANA (also known as Olson) format [IANATimeZone]. This property applies to parsing only. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarFirstDayOfWeek |
Enum Valid values 'Monday' … 'Sunday' The day of the week upon which a new week is considered to start. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarDaysInFirstWeek |
Non-negative Integer Valid values 1 to 7 Specify the number of days of the new year that must fall within the first week. The start of a year usually falls in the middle of a week. If the number of days in that week is less than the value specified here, the week is considered to be the last week of the previous year; hence week 1 starts some days into the new year. Otherwise it is considered to be the first week of the new year; hence week 1 starts some days before the new year. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarCenturyStart |
Non-negative Integer Valid values 0 to 99. This property determines on parsing how two-digit years are interpreted. Specify the two digits that start a 100-year window that contains the current year. For example, if you specify 89, and the current year is 2006, all two-digit dates are interpreted as being in the range 1989 to 2088. A two-digit year less than 89 will be interpreted as 20nn and a two-digit year more than or equal to 89 will be treated as 19nn. Annotation: dfdl:element, dfdl:simpleType |
||||||||
|
calendarLanguage |
String or DFDL Expression The language that is used when the pattern produces a presentation in text. The value must match the regular expression: ([A-Za-z]{1,8}([\-_][A-Za-z0-9]{1,8})*) It is a schema definition error otherwise. All DFDL Implementations must support dfdl:calendarLanguage value "en". DFDL implementations may support additional values, however, the value of the dfdl:calendarLanguage property is always interpreted as a Unicode Language Indentifier as defined by [LDML], and [CLDR]. Annotation: dfdl:element, dfdl:simpleType |
Table 39 Properties specific to Calendar with Text or Binary Representation
The dfdl:calendarPattern describes how to parse and unparse text and binary representations of dateTime, date and time logical types. The pattern is primarily used on unparsing to define the format but is also used to aid parsing.
The pattern is derived from the ICU SimpleDatetimeFormat class described here: [ICUDateTime], which uses symbols defined by [LDML].
An extension is the formatting symbol I which means accept a subset of ISO 8601 [ISO8601] compliant calendars
|
Symbol |
Meaning |
Presentation |
Example |
|
|
G |
era designator |
Text |
G |
AD |
|
y |
year |
Number |
y, yyyy yy |
1996 96 |
|
u |
year (allows negative years) |
Number |
u |
1900, 0, -500 |
|
Y |
year (of the week of year) |
Number |
Y |
1997 |
|
M |
month in year |
Text & Number |
M, MM MMM MMMM MMMMM |
09 Sept September S |
|
d |
day in month |
Number |
d dd |
2 02 |
|
h |
hour in am/pm (1~12) |
Number |
h hh |
7 07 |
|
H |
hour in day (0~23) |
Number |
H HH |
0 00 |
|
m |
minute in hour |
Number |
m mm |
4 04 |
|
s |
second in minute |
Number |
s ss |
5 05 |
|
S |
fractional second (see note 1) |
Number |
S SS SSS |
2 24 235 |
|
E |
day of week |
Text |
E, EE,EEE EEEE EEEEE EEEEEE |
Tues Tuesday T Tu |
|
e |
day of week (local) |
Text & Number |
e, ee eee eeee eeeee eeeeee |
2 Tues Tuesday T Tu |
|
D |
day in year |
Number |
D |
189 |
|
F |
day of week in month |
Number |
F |
2 (2nd Wed in July) |
|
w |
week in year |
Number |
w, ww |
27 |
|
W |
week in month |
Number |
W |
2 |
|
a |
am/pm marker |
Text |
A |
pm |
|
k |
hour in day (0~24 ) |
Number |
k kk |
2, 24 02, 24 |
|
K |
hour in am/pm (0~11) |
Number |
K KK |
0 00 |
|
z |
time zone: specific non-location |
Text |
z, zz, zzz zzzz |
PDT Pacific Daylight Time |
|
Z |
time zone: ISO8601 basic format time zone: long localized GMT |
Text |
Z, ZZ, ZZZ ZZZZ |
-0800, +0000 GMT-08:00, GMT+00:00 |
|
O |
time zone: localized GMT |
Text |
O OOOO |
GMT-8 GMT-08:00 |
|
v |
time zone: generic non-location |
Text |
v vvvv |
PT Pacific Time |
|
V |
time zone: short time zone ID time zone: long time zone ID time zone: exemplar city time zone: generic location. |
Text |
V VV VVV VVVV |
uslax America/Los_Angeles Los Angeles Los Angeles Time |
|
x |
time zone: ISO8601 basic or extended format |
Text |
x xx xxx |
-08, +0530, +0000 -0800, +0000 -08:00, +00:00 |
|
X |
Time Zone: ISO8601 basic or extended format .The UTC indicator "Z" is used when local time offset is 0. |
Text |
X XX XXX |
-08, +0530, Z -0800, Z -08:00, Z |
|
I |
ISO8601 date/time |
Text |
I |
2006-10-07T12:06:56.568+01:00 |
|
' |
escape for text |
Delimiter |
' |
'Date=' |
|
'' |
single quote |
Literal |
'' |
'o''clock' |
Table 40 Symbols in the dfdl:calendarPattern Property
Any number of fractional seconds "S" may by specified in the pattern and accepted by implementations, but an implementation is free to represent a limited number of fractional seconds internally. Excess fractional seconds are truncated, not rounded up. At least millisecond accuracy must be implemented. Unlike other fields, fractional seconds are padded on the right with zero.
It is a processing error if seconds appear in that part of the SimpleContent region that represents a time zone.
The count of pattern letters determines the format as indicated in the table.
If dfdl:representation is text, any characters in the pattern that are not in the ranges of ['a'..'z'] and ['A'..'Z'] will be treated as quoted text. For instance, characters like ':', '.', ' ', '#' and '@' will appear in the formatted output even if they are not embraced within single quotes. The single quote is used to 'escape' letters. Two single quotes in a row, whether inside or outside a quoted sequence, represent a 'real' single quote.
If dfdl:representation is binary, then the pattern can contain only characters and symbols that always result in the presentation of digits.
The symbols 'z', 'zz', and 'zzz' have identical meaning, as do 'Z', 'ZZ', and 'ZZZ'.
The 'I' symbol must not be used with any other symbol with the exception of 'escape for text'. It represents calendar formats that match those defined in the restricted profile of the ISO 8601 standard proposed by the W3C at http://www.w3.org/TR/NOTE-datetime. The formats are referred to as 'granularities'.
When parsing, for any pattern that omits components the values for the omitted components are supplied from the Unix epoch 1970-01-01T00:00:00.000.[25]
When unparsing, and the pattern contains a formatting symbol that requires a component of the date/time and the infoset value does not contain that component, it is a processing error.
When parsing a calendar element with a packed decimal representation then the nibbles from the data are converted to text digits without any trimming of leading or trailing zeros, and the result is then matched against the pattern according to the usual rules.
When unparsing, if a time zone symbol is not available for a particular time zone, a fallback may be used as defined in [ICUDateTime].
The differences in behavior between 'strict' and 'lax' for this property can be subtle. Both are quite lenient in enforcement of many variations in format, with the 'lax' value adding additional tolerance of more format variations to those already allowed by the 'strict' value.
|
Property Name |
Description |
|
textCalendarJustification |
Enum Valid values 'left', 'right', 'center' Controls how the data is padded or trimmed on parsing and unparsing. Behavior as for dfdl:textStringJustification. Annotation: dfdl:element, dfdl:simpleType |
|
textCalendarPadCharacter |
DFDL String Literal The value that is used when padding or trimming calendar
elements. The value can be a single character or a single byte. If a byte, then it must be specified using a single byte value entity If a pad character is specified when dfdl:lengthUnits is 'bytes' then the pad character must be a single-byte character. If a pad byte is specified when dfdl:lengthUnits is 'characters' then
The string literal value is restricted in the same way as described in "Pad Character Restrictions" in the description of the dfdl:textStringPadCharacter property. Annotation: dfdl:element, dfdl:simpleType |
Table 41 Properties Specific to Calendar with Text Representation
|
Property Name |
Description |
|
binaryCalendarRep |
Enum Valid values are 'packed', 'bcd', 'ibm4690Packed', 'binarySeconds', 'binaryMilliseconds' For all values, the dfdl:byteOrder property is used to determine the numeric significance of the bytes making up the representation. 'packed' means represented as an IBM 390 packed decimal. Each byte contains two decimal digits, except for the rightmost byte, which contains a sign to the right of a decimal digit. The digits are interpreted according to the dfdl:calendarPattern property. Property dfdl:binaryPackedSignCodes is applicable. 'bcd' means represented as a binary coded decimal with two digits per byte. The digits are interpreted according to the dfdl:calendarPattern property 'ibm4690Packed' means represented as a variant of packed format as described in property dfdl:binaryNumberRep. The digits are interpreted according to the dfdl:calendarPattern property. For all packed decimals, property dfdl:binaryNumberCheckPolicy is applicable. For all these packed decimals, dfdl:calendarPattern can contain only characters and symbols that always result in the presentation of digits. It is a schema definition error otherwise. This implies that property dfdl:calendarPatternKind must be 'explicit' because the default patterns for 'implicit' contain non-numeric characters. It is a schema definition error otherwise. See Section 13.7 Properties Specific to Number with Binary Representation. Note also that a virtual decimal point for the boundary between seconds and fractional seconds is implied from the pattern at the boundary of 's' and 'S', i.e., where the substring 'sS' appears in the pattern. 'binarySeconds' means represented as binary xs:int, that is, as a 4 byte signed integer that is the number of seconds from the epoch (positive or negative). It is a schema definition error if there is a specified length not equivalent to 4 bytes. 'binaryMilliseconds' means represented as binary xs:long, that is, as an 8 byte signed integer that is the number of milliseconds from the epoch (positive or negative). It is a schema definition error if there is a specified length not equivalent to 8 bytes. binarySeconds and binaryMilliseconds may only be used when the type is xs:dateTime. (It is a schema definition error otherwise.) Annotation: dfdl:element, dfdl:simpleType |
|
binaryCalendarEpoch |
DateTime Used when dfdl:binaryCalendarRep is 'binarySeconds' or 'binaryMilliseconds' The epoch from which to calculate dates and times. If the time zone is omitted from the value, then UTC is used. Annotation: dfdl:element, dfdl:simpleType |
Table 42 Properties Specific to Calendar with Binary Representation
Examples of packed decimal format calendars for December 14, 1923 and dfdl:calendarPattern of 'MMddyy' would be:
packed: (hexadecimal) 01 21 42 3C,
bcd: (hexadecimal) 12 14 23
ibm4690Packed: (hexadecimal) 12 14 23
The 'C' nibble at the end of the 'packed' representation is a sign nibble, and the leading 0 nibble is just to align to a byte boundary..
There are no properties specific to opaque types
Sometimes it is desirable to represent an unused element, place-holder for unknown information, or inapplicable information explicitly with an element, rather than by the lack of an element.
For example, it may be desirable to represent a sparsely populated array of data using a distinguished nil element to fill the locations where data is absent, thereby preserving the position for the elements that are present.
As another example, it may be desirable to represent an unused simple element by a value which is not conformant to the logical type of the element.
Such cases can be represented using the DFDL nil mechanism which is based on the XML Schema nil mechanism. DFDL provides what are commonly called "in-band" nil values by way of dfdl:nilKind 'logicalValue', and also provides for two kinds of literal indicators of nil through dfdl:nilKind 'literalValue' and dfdl:nilKind 'literalCharacter'. Nil processing is used when the XSDL 'nillable' property of an element is true.
DFDL allows elements of complex type to be nillable. However, to avoid the concept of a complex element having a value, which does not exist in DFDL, the only permissible nil value is the empty string, represented by the DFDL %ES; entity and using dfdl:nilKind 'literalValue'.
On parsing, an element occurrence is nil if the element has XSDL nillable 'true' and the data is a nil representation as defined in section 9.2.1. Specifically:
For dfdl:nilKind 'literalValue' or 'literalCharacter':
On unparsing, an element is nil if XSDL nillable is 'true' AND the element information item in the augmented infoset has the [nilled] member as true, in which case what is output to the data stream is one of the following:
For dfdl:nilKind 'literalValue' or 'literalCharacter' then dfdl:nilValueDelimiterPolicy determines whether any initiator or terminator also appear surrounding the literal nil in the output data.
These properties are used when the XSDL 'nillable' property of an element is 'true', and they control when and how the representation data are interpreted as having the logical meaning 'nil'.
|
Property Name |
Description |
|
nilKind |
Enum Valid values 'literalValue', 'logicalValue', 'literalCharacter'. Used when XSDL nillable is 'true'. Specifies how dfdl:nilValue is interpreted to represent the nil value in the data stream. If 'literalCharacter' then dfdl:nilValue specifies a single character or a single byte that, when repeated to the length of the element, is the nil value. 'literalCharacter' may only be specified for fixed length elements, that is dfdl:lengthKind 'implicit' and 'explicit' when dfdl:length is not a DFDL expression, otherwise it is a schema definition error.. If 'literalValue' then dfdl:nilValue specifies a list of DFDL literal strings that are the possible representations for nil. If 'logicalValue' then dfdl:nilValue specifies a list of logical values that are the possible logical values for nil. Complex elements can be nillable, but dfdl:nilKind can only be 'literalValue' and dfdl:nilValue must be "%ES;". It is a schema definition error otherwise. Annotation: dfdl:element(simpleType) |
|
nilValue |
List of DFDL String Literals, List of Logical Values, DFDL String Literal Specifies the text strings that are the possible literal or logical nil values of the element. If dfdl:nilKind is 'literalValue' then dfdl:nilValue specifies a whitespace separated list of DFDL literal strings that are the possible representations for nil. On parsing the element value is nil if the trimmed data matches one of the string literals in the list. On unparsing if the element value is nil the first string literal in the list is output. If dfdl:nilKind is 'logicalValue' then dfdl:nilValue specifies a whitespace separated list of logical values that are the possible logical values for nil. On parsing the element value is nil if the data, converted to its logical type, matches any of the logical values in the list. On unparsing if the element value is nil, the first value from the list is converted to its physical representation and output. If dfdl:nilKind is 'literalCharacter' then dfdl:nilValue specifies a single character or byte that, when repeated to the length of the element, is the nil representation. If a character, then it can be specified using a literal character or using DFDL entities. If a character is specified when dfdl:lengthUnits is 'bytes' then the dfdl:nilValue must be a single-byte character. To specify a byte, it must be specified using a single "%#r;" entity. If a byte is specified when dfdl:lengthUnits is 'characters' then the dfdl:encoding must be a fixed-width encoding. On parsing, the element value is nil if all characters in the untrimmed data content match the dfdl:nilValue character . On unparsing, if the element value is nil the dfdl:nilValue character is output to the needed length. There are restrictions on the string literal syntax of dfdl:nilValue. When dfdl:nilKind is literalValue and text representation:
When dfdl:nilKind is literal value and binary representation:
When dfdl:nilKind is literalCharacter and text representation:
When dfdl:nilKind is literalCharacter and binary representation:
dfdl:nilValue is sensitive to dfdl:ignoreCase when dfdl:nilKind is 'literalValue' or 'logicalValue', but not when dfdl:nilKind is 'literalCharacter' Complex elements can be nillable, but dfdl:nilKind can only be 'literalValue' and dfdl:nilValue must be "%ES;". It is a schema definition error otherwise. Annotation: dfdl:element(simpleType) |
|
nilValueDelimiterPolicy |
Enum Valid values are 'none', 'initiator', 'terminator' or 'both'. Indicates that when the value nil is represented, an initiator (if one is defined), a terminator (if one is defined), both an initiator and a terminator (if defined) or neither must be present. Ignored if both dfdl:initiator and dfdl:terminator are "" (empty string). Ignored if dfdl:nilKind is set to 'logicalValue' In this case the DFDL processor treats a nil representation like any other representation of the element in that it expects delimiters when parsing, outputs them when unparsing. 'initiator' indicates that, on parsing, the dfdl:initiator followed by a dfdl:nilValue indicates that a nil representation is present. It also indicates that on unparsing when the logical value is nil that the dfdl:initiator will be output followed by the first dfdl:nilValue. 'terminator' indicates that, on parsing, a dfdl:nilValue followed by the dfdl:terminator indicates that a nil representation is present. It also indicates that on unparsing when the logical value is nil the first dfdl:nilValue followed by the dfdl:terminator will be output. 'both' indicates that, on parsing, both the dfdl:initiator and dfdl:terminator must be present with a dfdl:nilValue to indicate that a nil representation is present. On unparsing the dfdl:initiator followed by the first dfdl:nilValue, followed by the dfdl:terminator will be output. 'none' indicates that a dfdl:nilValue without any dfdl:initiator or dfdl:terminator indicates that a nil representation is present. On unparsing the first dfdl:nilValue is output without any dfdl:initiator or dfdl:terminator. It is a schema definition error if dfdl:nilValueDelimiterPolicy is set to 'none' or 'terminator' when the parent xs:sequence has dfdl:initiatedContent 'yes'. Annotation: dfdl:element(simpleType) |
|
useNilForDefault |
Enum Valid values are 'yes', 'no' When the conditions for applying a simple element default are satisfied, this property controls whether to set the Infoset item [nilled] boolean member, or to use the XSDL default or fixed properties to obtain a data value. This property has precedence over the XSDL default and XSDL fixed properties. It is only used, and must be defined, if the XSDL nillable property is 'true'. Defaulting occurs as described in section 9.4 Element Defaults with nil as the default value. The dfdl:nilValue property must specify at least one nil value otherwise it is a schema definition error. The dfdl:nilKind property may be any of its values. Annotation: dfdl:element (simpleType) |
Table 43 Properties for Nillable Elements
The DFDL element defaults processing uses XSDL default, XSDL fixed or dfdl:useNilForDefault to provide a default value. See section 9.4 Element Defaults for a full description.
The following properties are specific to sequences.
|
Property Name |
Description |
|
sequenceKind |
Enum Valid values are 'ordered', 'unordered' When 'ordered', this property means that the contained items of the sequence will be encountered in the same order that they appear in the schema, which is called schema-definition-order. When 'unordered', this property means that the items of the sequence will be encountered in any order. Repeating occurrences of the same element do not need to be contiguous. The children of an unordered sequence MUST be xs:element otherwise it is a schema definition error. Annotation: dfdl:sequence, dfdl:group (sequence) |
|
initiatedContent |
Enum Valid values are 'yes', 'no' When 'yes' indicates that all the children of the sequence
are initiated. It is a schema definition error if any children have their
dfdl:initiator property set to the empty string. When 'no', the children of the sequence may have their dfdl:initiator property set to the empty string. Annotation: dfdl:sequence, dfdl:choice, dfdl:group |
Table 44 Properties for Sequence Groups
A sequence can have a dfdl:initiator and/or a dfdl:terminator as described earlier.
A sequence having no children is syntactically legal in DFDL. In the data stream, such a sequence can have non-zero length LeftFraming and RightFraming regions, but the SequenceContent region in between must be empty. It is a processing error if the SequenceContent region of an empty sequence has non-zero length when parsing.
XML schema does not define an empty sequence that is the content model of a complex type definition as effective content so any DFDL annotations on such a construct would be ignored. It is a schema definition error if the empty sequence is the content model of a complex type, or if a complex type has nothing in its content model at all.
Additional properties apply to sequence groups that use text delimiters to separate one occurrence of a member of the group from the next. Such a delimiter is called a separator. DFDL provides several properties that control the parsing and writing of separators, and satisfy the requirement to model sequences where:
These requirements are addressed by the properties dfdl:separator, dfdl:separatorPosition and dfdl:separatorSuppressionPolicy, as described below.
These properties combine to define the syntax for a sequence group with dfdl:sequenceKind 'ordered'. Not all combinations of the properties will give rise to consistent syntax, so some combinations are disallowed and will give rise to a schema definition error.
In some sequences, the presence of separators alone is enough to establish occurrences within the sequence. Such a sequence is called a positional sequence.
Positional sequence - Each occurrence in the sequence can be identified by its position in the data. Typically the components of such a sequence do not have an initiator. In some such sequences, the separators for optional zero-length occurrences may or must be omitted when at the end of the group. A positional sequence can be modelled by setting dfdl:separatorSuppressionPolicy to 'never', 'trailingEmptyStrict' or 'trailingEmpty'.
Non-positional sequence - Occurrences in the sequence cannot be identified by their position in the data alone. Typically the components of such a sequence have an initiator. Such sequences allow the separator to be omitted for optional zero-length occurrences anywhere in the sequence. Speculative parsing is employed by the parser to identify each occurrence. A non-positional sequence can be modelled by setting dfdl:separatorSuppressionPolicy to 'anyEmpty'.
|
Property Name |
Description |
|
separator |
List of DFDL String Literals or DFDL Expression Specifies a whitespace separated list of alternative literal strings that are the possible separators for the sequence. Separators occur in the data either before, between or after all occurrences of the elements or groups that are the children of the sequence. This property can be computed by way of an expression which returns a string of whitespace separated values. The expression must not contain forward references to elements which have not yet been processed. It is a Schema Definition Error if the expression returns an empty string. This property can be used to determine the length of an element as described in Section 12.3.2 dfdl:lengthKind 'delimited'. Each string literal in the list, whether apparent in the schema, or returned as the value of an expression, is restricted to allow only certain kinds of syntax: · DFDL character entities are allowed. · DFDL Byte Value entities ( %#r ) are allowed. · DFDL Character Class ES is not allowed. · DFDL Character Classes NL, WSP, WSP+, and WSP* are allowed. However, the WSP* entity cannot appear on its own as one of the string literals in the list when determining the length of a component by scanning for delimiters, , and it is a schema definition error otherwise. delimiters The Separator, PrefixSeparator and PostfixSeparator regions contain one of the strings specified by the dfdl:separator property. When this property has "" (empty string) as its value then the separator region is of length zero. When parsing, the list of values is processed in a greedy manner, meaning it takes all the separators, that is, each of the string literals in the whitespace separated list, and matches them each against the data. The separator with the longest match is the one that is selected as having been 'found'. Once a matching separator is found, no other matches will be subsequently attempted (ie, there is no backtracking). On unparsing the first separator in the list is used as the separator. If a child element uses an escape scheme, then the escape scheme also applies to any separator; hence, if the separator appears within the element value, it will be escaped. If dfdl:ignoreCase is 'yes' then the case of the string is ignored by the parser. Annotation: dfdl:sequence, dfdl:group (sequence) |
|
separatorPosition |
Enum Valid values 'infix', 'prefix', 'postfix' 'infix' means the separator occurs between the elements in the Separator grammar region. 'prefix' means the separator occurs before each element in the Separator grammar region and the PrefixSeparator grammar region. 'postfix' means the separator occurs after each element in the Separator grammar region and the PostfixSeparator grammar region. Annotation: dfdl:sequence, dfdl:group (sequence). |
|
separatorSuppressionPolicy |
Enum Valid values 'never', 'anyEmpty', 'trailingEmpty', 'trailingEmptyStrict' Only applicable if dfdl:separator is not "" (empty string) and dfdl:sequenceKind is 'ordered'. Controls the circumstances when separators are expected in the data when parsing, or generated when unparsing, if an optional element occurrence or a group has a zero-length representation. See section 14.2.1 Separators and Suppression. When dfdl:sequenceKind is 'unordered' then 'anyEmpty' is implied. Annotation: dfdl:sequence, dfdl:group (sequence) |
Table 45 Properties for Sequence Groups with Separators
When parsing a sequence group that specifies a separator, the number of occurrences and separators that are expected in the data stream for a child element depends on several factors:
Where to expect a separator for optional content of zero-length is controlled by property dfdl:separatorSuppressionPolicy.
|
separatorSuppressionPolicy |
Implications |
|
never |
Positional sequence where all occurrences MUST be found in the data, along with their associated separator. |
|
trailingEmptyStrict |
Positional sequence where trailing occurrences that have zero length representation MUST be omitted from the data, along with their associated separator. |
|
trailingEmpty |
Positional sequence where trailing occurrences that have zero length representation MAY be omitted from the data, along with their associated separator. |
|
anyEmpty
|
Non-positional sequence where any occurrences that have zero length representation MAY be omitted from the data, along with their associated separator. It must be possible for speculative parsing to identify which elements are present. |
Table 46 Sequence groups and separator suppression
The following are definitions for terminology used in this section:
Potentially trailing element – An array or optional element describes an occurrence that is said to be potentially trailing if the element is capable of having a zero length representation and is followed in its enclosing group definition by only these kinds of schema components:
1. calculated elements (those having dfdl:inputValueCalc)
2. additional potentially trailing elements
3. potentially trailing groups
Intuitively, the array or optional element occurrence could be last.
Potentially trailing group – A group is said to be potentially trailing if the group has no framing and contains only potentially trailing element declarations/references, or recursively similar sequence or choice groups, and is followed in its enclosing group definition by only additional potentially trailing elements or potentially trailing groups.
Trailing or Actually Trailing – An element occurrence or group occurrence in the data is said to be actually trailing if it is potentially trailing and has zero-length representation and is not followed in the data by any other non-zero length element occurrence or group occurrence limited by the end of the enclosing sequence group.
In the sections that follow, it is important to remember that the dfdl:separatorSuppressionPolicy property is carried on the sequence, while the XSDL minOccurs, XSDL maxOccurs and dfdl:occursCountKind properties are is carried on an element in that sequence.
When an element is required and is not an array then one occurrence is always expected along with its separator. The dfdl:separatorSuppressionPolicy is not applicable and the implied behavior is 'never'.
Otherwise the behaviour is dependent on dfdl:occursCountKind.
When dfdl:occursCountKind is 'fixed' then XSDL minOccurs must equal maxOccurs and that many occurrences are always expected along with their separators. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'never'.
When dfdl:occursCountKind is 'expression' the number of occurrences is given by dfdl:occursCount and exactly that many occurrences are always expected along with their separators. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'never'.
When dfdl:occursCountKind is 'parsed' any number of occurrences and their separators are expected. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'anyEmpty'.
When dfdl:occursCountKind is 'stopValue', any number of occurrences and their separators are expected followed by the stop value and its separator. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'anyEmpty'.
When dfdl:occursCountKind is 'implicit', between XSDL minOccurs and maxOccurs (inclusive) occurrences and their separators are expected. The dfdl:separatorSuppressionPolicy is applicable and determines when separators are expected for optional zero-length occurrences.
The behaviour for 'implicit' is more fully expressed in matrix form. The cells in the matrix give the number of occurrences of element values that are expected in the data stream when parsing, for the different values of dfdl:separatorSuppressionPolicy. The number of occurrences also depends whether XSDL maxOccurs is unbounded or not, and the position of the element in the sequence. The number of separators can be inferred from this, taking into account dfdl:separatorPosition.
|
separatorSuppressionPolicy |
dfdl:occursCountKind 'implicit' |
|||||
|
Potentially Trailing |
Not Potentially Trailing |
|||||
|
maxOccurs unbounded |
maxOccurs bounded |
maxOccurs unbounded |
maxOccurs bounded |
|||
|
Element not declared last |
Element declared last |
Element declared last or occurrence followed by end-of-group |
Element not declared last and occurrence not followed by end-of-group |
|||
|
never |
Schema definition error |
|
Schema definition error |
RepDef(min) ~ Rep(max - min) |
||
|
trailingEmptyStrict |
|
RepDef(min) [ ~ Rep(M < INF) ~ RepNonZero(1) ] |
RepDef(min) [ ~ Rep(M < max - min) ~ RepNonZero(1) ] |
RepDef(min) ~ Rep(max - min) |
||
|
trailingEmpty |
RepDef(min) ~ Rep(M < INF) |
RepDef(min) ~ Rep(M <= max - min) |
||||
|
anyEmpty |
|
|
RepDef(min) ~ Rep(M < INF) |
RepDef(min) ~ Rep(M <= max - min) |
Table 47 Separator Suppression for dfdl:occursCountKind 'implicit' when Parsing
The notation in each cell uses the "~" symbol to mean "followed by" in the data stream. Square brackets surround things that are optional, as in they may or may not appear in the data stream.
The descriptions found in the cells of the matrix do not provide a parsing algorithm, but rather state declaratively a pattern that the data must match in order to be correctly parsed.
RepDef(min) is short for "representation" and "defaultable". It means XSDL minOccurs occurrences of nil, empty or normal representation[26]. These are required occurrences so default rules apply for empty representations. XSDL minOccurs may be 0, in which case there are no required occurrences.
RepNonZero(1) means an occurrence of a nil, empty or normal representation where such a representation does not have zero-length[27]. This is an optional occurrence so default rules do not apply.
A notation like Rep(M <= max – min) means that there are M occurrences, where M is some value between the values of the XSDL minOccurs and maxOccurs properties. When an unbounded number of occurrences is possible this is shown explicitly by Rep(M < INF) , INF meaning infinity or unbounded.
In the matrix above we see that there are some cells where the combination of properties doesn't make sense, and a schema definition error is raised. These occur when an element has dfdl:occursCountKind 'implicit' and XSDL maxOccurs 'unbounded', and dfdl:separatorSuppressionPolicy implies that the sequence is positional, specifically:
Consider the cell of the matrix above for the element in this DFDL schema fragment:
<xs:sequence dfdl:separator='|' dfdl:separatorPosition='infix'
dfdl:separatorSuppressionPolicy='trailingEmptyStrict'>
<xs:element name='a' type='xs:int' default='0'
maxOccurs='5' minOccurs='0'
dfdl:representation='text' dfdl:textNumberPattern='#0'
dfdl:occursCountKind='implicit'
dfdl:initiator='[' dfdl:terminator=']'
dfdl:emptyValueDelimiterPolicy='both' />
</xs:sequence>
Within the sequence this element 'a' is clearly potentially trailing as it is declared last. The corresponding cell in the matrix above contains this description:
RepDef(min) [~ Rep(M < max – min) ~ RepNonZero(1) ]
Since XSDL minOccurs='0', the first term, RepDef(min) vanishes. So we have left
Rep(M < max - min) ~ RepNonZero(1)
Note Rep(M) permits absent representations, and if encountered they will simply be omitted from the infoset.
So this data
[1]|[2]|[3]|[4]|[5]
parses and 5 items appear in the infoset.
This data
|||[4]
also parses because absent representations are accepted, but only one item appears in the infoset.(The fact that the occurrence was fourth in the array is not preserved into the infoset). However this data
|||[4]|
causes a processing error because there is a final trailing separator and dfdl:separatorSuppressionPolicy is 'trailingEmptyStrict'.
Now consider the same scenario but minOccurs of '2'. The first term reappears as RepDef(2). The data
|||[4]
which previously parsed successfully would now cause a processing error because the first two occurrences are required, so they have to be either a normal representation, that is, matching xs:int syntax with surrounding initiator and terminator, or the empty representation which is []. An example which will parse correctly with minOccurs of '2' is:
[1]|[]||[4]
In this case the Infoset will contain 3 items with values 1, 0, 4. The 0 value arises because the occurrence has the empty representation, the occurs index is 2 so it is required, and there is a default value 0.
If the scenario is changed so that dfdl:separatorSuppressionPolicy is 'trailingEmpty' then a different cell of the matrix above applies.
RepDef(min) [~ Rep(M < max – min) ]
This has a more lax behavior so that this data is also acceptable:
[1]|[]||[4]|
In this case the final trailing separator is tolerated, though when unparsing this final trailing separator would not be created. This is a case where what is parsed will not be exactly recreated on unparsing from the resulting infoset, but all the information content is preserved.
Now consider the same scenario but XSDL maxOccurs is 'unbounded'. In that case this data is acceptable:
[1]|[]||||||||||||||||||||||||[4]|||||||||||||||||||||
The infoset values are again 1, 0, 4. But all the excess separators are tolerated.
When an element is required and is not an array then one occurrence is always output along with its separator. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'never'.
Otherwise the behaviour is dependent on dfdl:occursCountKind.
When dfdl:occursCountKind is 'fixed' or 'expression' the occurrences in the augmented Infoset are always output along with their separators. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'never'.
When dfdl:occursCountKind is 'parsed' non zero-length occurrences in the augmented Infoset are output along with their separators. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'anyEmpty'.
When dfdl:occursCountKind is 'stopValue' non zero-length occurrences in the augmented Infoset are output along with their separators followed by the stop value and its separator. The dfdl:separatorSuppressionPolicy is not applicable and the implied behaviour is 'anyEmpty'.
When dfdl:occursCountKind is 'implicit' the occurrences in the augmented Infoset are output along with their separators. The dfdl:separatorSuppressionPolicy is applicable and helps determine whether optional zero length occurrences and their separators are output.
The behaviour for 'implicit' is more fully expressed in matrix form. The cells in the matrix give the number of occurrences of element values that are output to the data stream when unparsing, for the different values of dfdl:separatorSuppressionPolicy. The number of occurrences also depends whether XSDL maxOccurs is unbounded or not, and the position of the element in the sequence. The number of separators output can be inferred from this, taking into account dfdl:separatorPosition.
|
dfdl: separatorSuppressionPolicy
|
dfdl:occursCountKind 'implicit' |
|||||
|
Potentially Trailing |
Not Potentially Trailing |
|||||
|
maxOccurs unbounded |
maxOccurs bounded |
maxOccurs unbounded |
maxOccurs bounded |
|||
|
Element not declared last |
Element declared last |
Element declared last or occurrence followed by end-of-group |
Element not declared last and occurrence not followed by end-of-group |
|||
|
never |
Schema definition error |
Unparse N occurrences ~ unparse (maxOccurs -- N) trailing zero-length occurrences |
Schema definition error |
Unparse N occurrences ~ unparse (maxOccurs -- N) trailing zero-length occurrences |
||
|
trailingEmptyStrict |
|
Unparse N occurrences (suppressing trailing zero-length occurrences) |
|
|||
|
trailingEmpty |
||||||
|
anyEmpty |
Unparse N occurrences (suppressing any optional zero-length occurrences) |
|||||
Table 48 Separator Suppressions for dfdl:occursCountKind 'implicit'
The notation in each cell uses the "~" symbol to mean "followed by" in the output data stream.
N stands for the number of elements in the augmented Infoset, which includes any defaults.
unparse N occurrences means output N unparsed infoset items and associated separators.
unparse(M) trailing zero length occurrences means output M adjacent separators (according to dfdl:separatorPosition) as if separating M element occurrences.
(suppressing trailing zero-length reps) implies the unparser must look ahead into the infoset and determine when the representations will be zero-length, and then identify those in trailing position. No separators are output corresponding to the trailing zero-length representations.
Consider the cell of the matrix above for the element in this DFDL schema fragment:
<xs:sequence dfdl:separator='|' dfdl:separatorPosition='infix'
dfdl:separatorSuppressionPolicy='trailingEmpty'>
<xs:element name='a' type='xs:int'
maxOccurs='5' minOccurs='0'
nillable='true'
dfdl:representation='text' dfdl:textNumberPattern='#0'
dfdl:occursCountKind='implicit'
dfdl:initiator='[' dfdl:terminator=']'
dfdl:emptyValueDelimiterPolicy='none'
dfdl:nilKind='literalValue' dfdl:nilValue='%ES;'
dfdl:nilValueDelimiterPolicy='none' />
</xs:sequence>
This example is similar to the one used above in the discussion of parsing with separator suppression. However, the element has no default value, the dfdl:emptyValueDelimterPolicy has been removed, and the element is nillable. Element 'a' is clearly potentially trailing as it is declared last. The corresponding cell in the matrix above contains this description:
unparse N occurrences (suppressing trailing zero length reps)
Assume we are unparsing an infoset containing five values: 1, 0, nil[28], 4, nil. We will unparse the five occurrences; however, the last value is nil, which has a representation of '%ES;' meaning empty-string, and dfdl:nilvalueDelimiterPolicy is 'none' meaning no initiator or terminator is to appear in the data. Since we are suppressing trailing zero-length reps the unparse results in this output:
[1]|[0]||[4]
We see here an example where if we reparsed the data we would not get back the same infoset that we started from because the trailing empty value which is the representation of the nil value, is not represented in the output, and so will not cause an Infoset item with [nilled] true to be created in the infoset when this data is parsed. To preserve the nil we would have to change the dfdl:nilValueDelimiterPolicy to 'both', and in that case the output would be:
[1]|[0]|[]|[4]|[]
The nils now have explicit representation in the data, and will be recreated in the Infoset when parsing.
The occurrences of members of a sequence group with dfdl:sequenceKind of 'unordered' (hereafter referred to as an 'unordered sequence') may appear in the data in any order. Occurrences of the same member do not have to be contiguous. In the infoset, sequence groups are always in schema order, so a DFDL processor must sort the members of an unordered sequence into schema order when parsing. When unparsing, the infoset must already be in schema order, and the members of the sequence will be output in schema order.
It is a schema definition error if any member of the unordered sequence is not an element declaration or an element reference.
It is a schema definition error if a member of an unordered sequence is an optional element or an array element and its dfdl:occursCountKind property is not 'parsed'
It is a schema definition error if two or more members of the unordered sequence have the same name and the same namespace (see post-processing transformation below)
It is a schema definition error if an unordered sequence has no members.
When parsing, the semantics of an unordered sequence are expressed by way of:
An implementation may use any technique consistent with this semantic.
The source-to-source transformation turns the declaration of an unordered sequence into an ordered sequence group that contains a repeating choice. To ensure that the resulting schema is a valid DFDL schema, the choice group is wrapped in an array element.
The unordered sequence is transformed as follows:
Using the following example as an illustration:
<xs:sequence dfdl:sequenceKind="unordered" dfdl:separator=",">
<xs:element name="a" type="xs:string" dfdl:initiator="A:" />
<xs:element name="b" type="xs:int" minOccurs="0" dfdl:initiator="B:" />
<xs:element name="c" type="xs:string" minOccurs="0" maxOccurs="10"
dfdl:initiator="C:" />
</xs:sequence>
The above unordered sequence group is conceptually rewritten into the following ordered sequence group:
<xs:sequence dfdl:sequenceKind="ordered" dfdl:separator=",">
<xs:element name="choiceElement" minOccurs="0"
maxOccurs="unbounded"
dfdl:occursCountKind="parsed">
<xs:complexType>
<xs:choice dfdl:choiceLengthKind="implicit">
<xs:element name="a" type="xs:string" dfdl:initiator="A:" />
<xs:element name="b" type="xs:int" dfdl:initiator="B:" />
<xs:element name="c" type="xs:string" dfdl:initiator="C:" />
</xs:choice>
</xs:complexType>
</xs:element>
</xs:sequence>
Processing then constructs a temporary info set for this ordered sequence group by parsing the data.
If a member element is found to have the empty representation then the parsing of that element must use the original value of XSDL minOccurs. In this example, element "b" has minOccurs "0" and if it is found with the empty representation then it must not be defaulted.
Post-processing consists of the following steps:
Sort the Temporary Infoset
The temporary infoset is transformed into the infoset conforming to the original unordered sequence. All members of the temporary infoset having the same name and namespace as the first child of the unordered sequence are placed first, in the order in which they were parsed. This algorithm repeats for the second child of the unordered sequence and so on until all members of the temporary infoset have been sorted into the schema declaration order of the original unordered sequence.
For the example above, the temporary infoset is transformed into the infoset corresponding to:
<xs:sequence>
<xs:element name="a" type="xs:string" />
<xs:element name="b" type="xs:int" minOccurs="0" />
<xs:element name="c" type="xs:string" minOccurs="0" maxOccurs="10" />
</xs:sequence>
Check Scalar Elements and Validate
For each element in the unordered sequence having XSDL minOccurs "1" and maxOccurs "1", the number of occurrences is checked. Each such element must occur exactly once in the infoset, else it is a processing error.
If validation is enabled, the DFDL processor validates the number of occurrences of each member of the unordered sequence against XSDL minOccurs and maxOccurs.
These checks are the same as those performed for an ordered sequence group. However, in an unordered sequence the checking of XSDL minOccurs and maxOccurs must be performed after the entire group has been parsed.
When unparsing, the behavior is exactly as if dfdl:sequenceKind is 'ordered'. The infoset must be presented to the unparser in schema declaration order, and the members of the unordered sequence group are output in schema declaration order.
Elements within an ordered sequence can be designated as floating which means that they can appear in any position within the sequence.[29]
|
Property Name |
Description |
|
floating |
Enum Valid values are 'yes', 'no' Whether the occurrences of an element in an ordered sequence can appear out-of-order in the representation. When parsing, and dfdl:floating is 'yes', occurrences of the element may be encountered in the representation in many positions within its containing sequence. If present they are placed into the infoset in schema declaration order. If the element repeats, occurrences do not need to be contiguous in the representation. When parsing, and dfdl:floating is 'no', occurrences of the element must be in schema declaration order, and, if present, they are placed into the infoset in schema declaration order. It is a processing error if instances of the element are not encountered in schema declaration order. When unparsing, occurrences of the element are expected in the infoset in schema declaration order, and are output in the representation in schema declaration order. It is a processing error if occurrences of the element are not encountered in schema declaration order, It is a schema definition error if an unordered sequence or a choice contains any element with dfdl:floating 'yes'. It is a schema definition error if an ordered sequence contains any element with dfdl:floating 'yes' and also contains non-element component (such as a choice or sequence model group). It is a schema definition error if an element with dfdl:floating 'yes' is an optional element or an array element and its dfdl:occursCountKind property is not 'parsed' It is a schema definition error if two or more elements with dfdl:floating 'yes' in the same group have the same name and the same namespace. Annotation: dfdl:element |
Table 49 Properties for Floating Elements
An ordered sequence with floating components is similar to an unordered sequence except only the floating elements may be out of order.
Within an ordered sequence with floating components a non-floating array element must have its occurrences appearing contiguously, so any floating elements cannot appear in between occurrences of the array element.(In other words, property dfdl:floating 'yes' only makes a statement about the floating element, not about any other elements in the sequence.)
An ordered sequence of n element children with dfdl:floating 'yes' is equivalent to an unordered sequence with the same n element children with dfdl:floating 'no'.
A complex element with dfdl:floating 'yes' can have as its content model a sequence with elements that also have dfdl:floating 'yes'.
Every element in a sequence containing one or more floating elements is a point of uncertainty, similar to the way every element in an unordered sequence is a point of uncertainty.
In resolving this point of uncertainty, a parser MUST look for the element defined at that position in the schema first and only if unsuccessful with parsing that element, the parser should subsequently attempt to parse the floating elements in the order they are defined in the schema. As soon as any such parse is successful this resolves the point of uncertainty.
Some fields in the physical stream provide information about other fields in the stream and are not really part of the data. For example, a field could give the number of repeats in a following array. These fields may not be of interest to an application after the data has been parsed, and so may be removed from the Infoset on parsing by containing the element declarations for them within a hidden group. A hidden group allows elements to be defined that will not be added to the Infoset on parsing and will not be expected in the Infoset on unparsing.
<xs:element name="root">
<xs:complexType>
<xs:sequence>
<xs:sequence>
<xs:annotation><xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:sequence hiddenGroupRef="tns:hiddenRepeatCount">
</xs:appinfo></xs:annotation>
</xs:sequence>
<xs:element name="arrayElement" type="xs:int"
minOccurs="0" maxOccurs="unbounded"
dfdl:occursCountKind="expression"
dfdl:occurCount= "{../repeatCount}"
dfdl:representation="binary"
dfdl:lengthKind="implicit" />
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:group name="hiddenRepeatCount" >
<xs:sequence>
<xs:element name="repeatCount" type="xs:int"
dfdl:outputValueCalc="{count(../arrayElement)}"
dfdl:representation="binary"
dfdl:lengthKind="implicit" />
</xs:sequence>
</xs:group>
An element contained within the extent of a hidden group is commonly called a hidden element.
Hidden elements are referenced via path expressions using the same DFDL expression that would be used if they were not hidden.
Hidden elements can (typically will) contain the regular DFDL annotations to define their physical properties and on unparsing to set their value. They are processed using the same behavior as non-hidden elements.
When the dfdl:hiddenGroupRef property is specified on an xs:sequence schema component, the appearance of any other DFDL properties on that component is a schema definition error. It is also a schema definition error if the sequence is not empty.
It is a schema definition error if the sequence is the only thing in the content model of a complex type definition.
It is a schema definition error if dfdl:hiddenGroupRef appears on a xs:group reference, that is, unlike most format properties that apply to sequences, dfdl:hiddenGroupRef cannot be combined from a xs:group reference.
A hidden group may appear within another hidden group.
|
Property Name |
Description |
|
hiddenGroupRef |
QName Reference to a global model group definition. Elements within this model group will not be added to the Infoset, and are called hidden elements. The model group within the model group definition may be a xs:sequence or xs:choice It is a schema definition error if the value is the empty string. It is not possible to place this property in scope on a dfdl:format annotation. Annotation: dfdl:sequence |
Table 50 Properties for Hidden Groups
When unparsing a hidden group, the behaviour is the same as when elements are missing from the infoset; that is, the default-values algorithm applies. The only difference is that if a required element does not have a default value or a dfdl:outputValueCalc then it is a schema definition error instead of a processing error. Note that this can be checked statically.
When unparsing a hidden group, it is a processing error if an element information item is provided in the infoset for a hidden element.
The following properties are specific to xs:choice.
|
Property Name |
Description |
|
choiceLengthKind |
Enum Valid values are 'implicit', 'explicit' 'implicit' means the branches of the choice are not filled, so the ChoiceContent region is variable length depending on which branch appears. 'explicit' means that the branches of the choice are always filled to the fixed length specified by dfdl:choiceLength, so the ChoiceContent region is fixed length regardless of which branch appears. Annotation: dfdl:choice, dfdl:group (choice) |
|
choiceLength |
Integer Only used when dfdl:choiceLengthKind is 'explicit'. Specifies the length of the choice in bytes, so the ChoiceContent region is fixed length regardless of which branch appears. A ChoiceUnused region is therefore possible which when unparsing is filled with dfdl:fillByte. Annotation: dfdl:choice, dfdl:group (choice) |
|
initiatedContent |
Enum Valid values are 'yes', 'no' When 'yes' indicates that all the branches of the choice are initiated. It is a schema definition error if any children have their dfdl:initiator property set to the empty string. The branch is deemed to have been found when its initiator has been found. Any subsequent error parsing the branch will not cause the parser to backtrack. When 'no', the branches of the choice may have their dfdl:initiator property set to the empty string. Annotation: dfdl:sequence, dfdl:choice, dfdl:group |
|
choiceDispatchKey |
DFDL Expression The expression must evaluate to an xs:string which must not be the empty string. This property is used only when parsing. The resultant string must match the dfdl:choiceBranchKey property value of one of the branches of the choice. This match is case sensitive. If so, it discriminates to that branch. The parser then goes straight to that branch, ignoring consideration of any other choice branches. No backtracking of this decision occurs if there is a subsequent processing error. It is a processing error if the value of the expression does not match one of the dfdl:choiceBranchKey properties for the branches. It is a schema definition error if any choice branch does not specify a dfdl:choiceBranchKey in a choice that carries choiceDispatchKey. It is not possible to place this property in scope on a dfdl:format annotation. Annotation: dfdl:choice |
|
choiceBranchKey |
DFDL String Literal This literal provides an alternate way to discriminate a choice to a branch. When the dfdl:choiceDispatchKey expression evaluates to a string matching this property's value, the choice is discriminated to this branch. The match is case insensitive. It is a schema definition error if individual dfdl:choiceBranchKey values are not unique across all branches of a choice that carries dfdl:choiceDispatchKey. It is a schema definition error if dfdl:choiceBranchKey is specified on a global element, or on a sequence or choice that is the child of a global group definition. It is a schema definition error if any choice branch does not specify a dfdl:choiceBranchKey in a choice that carries choiceDispatchKey. Byte value entities are not allowed. Character classes are not allowed. This property is only used when parsing. It is not possible to place this property in scope on a dfdl:format annotation. Annotation: dfdl:element, dfdl:sequence, dfdl:choice, dfdl:group |
Table 51 Properties for Choice Groups
A choice can have a dfdl: initiator and/or a dfdl:terminator as described earlier.
We will use this terminology:
|
Branch |
A branch is one of the available alternatives within a choice. A branch can be an element of simple type or complex type, or it can be an embedded sequence, choice or group reference. |
|
Root of the Branch |
Each branch conceptually has a single schema component at its root which is an element, sequence, choice or group reference. This component is known as the Root of the Branch. |
Table 52 Choice group terminology
The Root of the Branch MUST NOT be optional. That is XSDL minOccurs MUST BE greater than 0.
A choice that declares no branches in the DFDL schema is a schema definition error.
When processing a choice group the parser validates any contained path expressions. If a path expression contained inside a choice branch refers to any other branch of the choice, then it is a schema definition error. Note that this rule handles nested choices also. A path that navigates outward from an inner choice to another alternative of an outer choice is violating this rule with respect to the outer choice.
A choice corresponds to concepts called variant records, multi-format records, discriminated unions, or tagged unions in various programming languages. In some contexts choices are referred to generally as 'unions'. However, this should not be confused with XSDL unions.
When processing a choice, there are two ways to resolve the intended branch. In one, speculative parsing is used. In the other, a constant-time direct dispatch to a branch is performed.
Speculative resolution works as follows:
1) Attempt to parse the first branch of the choice.
2) If this fails with a processing error
a) If a
dfdl:discriminator evaluated to true earlier on this branch
then the parser is 'bound' to this branch and parsing of the entire choice
construct fails with a processing error.
b) If the branch
has a dfdl:initiator and the choice has dfdl:initiatedContent ‘yes’
then the parser is 'bound' to this branch and parsing of the entire choice
construct fails with a processing error.
c) Otherwise we repeat from step 1 for the next branch of the choice.
3) It is a processing error if the branches of the choice are exhausted.
4) If a branch is successfully parsed without error, then that branch's infoset becomes the infoset for the parse of the choice construct.
It is not possible for variable settings to be communicated from the speculative attempt to parse a branch to any other parsing situation. The speculative effort is completely isolated. Whether it succeeds or fails, neither the parse position in the source data, nor anything in the variable memory, nor the Infoset is affected.
Nested choices can require unbounded look ahead into the data.
Direct dispatch provides a constant-time dispatch to a choice branch independent of how many choice branches there are.
Direct dispatch is indicated by the dfdl:choiceDispatchKey property. This expression is evaluated to compute the string matching (case sensitive) the dfdl:choiceBranchKey property of one of the choice branches,.
When a match is found, it is as if a dfdl:discriminator had evaluated to true on that branch. It is selected as resolution of the choice, and there is no backtracking to try other alternative selections if a processing error occurs.
The dfdl:choiceBranchKey property can be placed on element references, local element declarations, local sequences, local choices, or group references. It must be unique across all branches of a choice that carries a dfdl:choiceDispatchKey property and it is a schema definition error otherwise.
Note that it is a schema definition error if both dfdl:initiatedContent and dfdl:choiceDispatchKey are provided on the same choice. However, it is not an error if a discriminator exists on a choice branch along with a dfdl:choiceBranchKey.
On unparsing there is the question of how one identifies the appropriate schema choice branch corresponding to the data in the infoset. This is complicated by the fact that the children may not be elements. They may themselves be sequences or choices.The selection of the choice branch is as follows: The element in the infoset is used to search the choice branches in the schema, in schema definition order, but without looking inside any complex elements. If the element occurs in a branch, then that branch is selected and if subsequently a processing error occurs, this selection is not revisited (that is, there is no backtracking).
To avoid any unintended behavior, all the children of a choice can be modeled as elements.
These properties are for array elements (XSDL maxOccurs >1 or unbounded) or optional elements (XSDL minOccurs 0 and maxOccurs 1). The properties handle a logical one-dimensional array of any simple or complex type.
|
Property Name |
Description |
|
occursCountKind |
Enum Specifies how the actual number of occurrences is to be established. Valid values 'fixed', 'expression', 'parsed', 'implicit' ,'stopValue'. 'fixed' means use the XSDL maxOccurs property. 'expression' means use the dfdl:occursCount property. 'parsed' means that the number of occurrences is determined solely by speculative parsing. 'implicit' means that the number of occurrences is determined by speculative parsing in conjunction with the XSDL minOccurs and maxOccurs properties. 'stopValue' means look for a mandatory logical stop value which signifies the end of the occurrences. These values are described in detail in section 16.1. Annotation: dfdl:element |
|
occursCount |
DFDL Expression Specifies the number of occurrences of the element. Required only when dfdl:occursCountKind is 'expression'. This property is computed by way of an expression which returns a non-negative integer. The expression must not contain forward references to elements which have not yet been processed. Annotation: dfdl:element, |
|
occursStopValue |
List of DFDL Logical Values A whitespace separated list of logical values that specify the alternative logical stop values for the element. Required only when dfdl:occursCountKind is 'stopValue'. When parsing then if an occurrence of the element has a logical value that matches one of the values in this list then the parser must not expect any more occurrences of the element. On unparsing the first value will be inserted as an additional final occurrence in the array after all of the occurrences in the infoset have been output. Annotation: dfdl:element |
Table 53 Properties for Array Elements and Optional Elements
When XSDL minOccurs 1 and maxOccurs 1, the above properties are not used, and the behavior is as if dfdl:occursCountKind was 'fixed' as described in section 16.1.1.
The enum 'fixed' should be used when the number of occurrences is always the same. The number is provided by the XSDL maxOccurs property.
When parsing, maxOccurs occurrences are expected in the data. It is a processing error if less than maxOccurs occurrences are found or defaulted. The parser stops looking for occurrences when maxOccurs have been found or defaulted.
When unparsing, maxOccurs occurrences are expected in the infoset. It is a processing error if less than maxOccurs occurrences are found or defaulted, or if more than maxOccurs occurrences are found.
It is a schema definition error if minOccurs is not equal to maxOccurs.
The enum 'implicit' should be used when the number of occurrences is to be established using speculative parsing, and there are lower and upper bounds to control the speculation. The bounds are provided by the XSDL minOccurs and XSDL maxOccurs properties.
When parsing, up to maxOccurs occurrences are expected in the data. It is a processing error if less than minOccurs occurrences are found or defaulted. The parser stops looking for occurrences when either minOccurs have been found or defaulted and speculative parsing does not find another occurrence, or maxOccurs have been found or defaulted.
When unparsing, up to maxOccurs occurrences are expected in the infoset. It is a processing error if less than minOccurs occurrences are found or defaulted, or if more than maxOccurs occurrences are found.
The enum 'parsed' should be used when the number of occurrences is to be established solely using speculative parsing.
When parsing, any number of occurrences is expected in the data. The parser stops looking for occurrences when speculative parsing does not find another occurrence. If validation is enabled, it is a validation error if less than XSDL minOccurs occurrences are found or defaulted, or greater than XSDL maxOccurs occurrences are found.
When unparsing, any number of occurrences is expected in the infoset. If validation is enabled, it is a validation error if less than minOccurs occurrences are found or defaulted, or if more than maxOccurs occurrences are found.
The enum 'expression' should be used when the number of occurrences is calculated by evaluating a DFDL expression.
When parsing, the dfdl:occursCount expression is evaluated and provides the number of occurrences expected in the data. It is a processing error if less than dfdl:occursCount occurrences are found or defaulted. The parser stops looking for occurrences when dfdl:occursCount occurrences have been found. If validation is enabled, it is a validation error if less than XSDL minOccurs occurrences are found or defaulted, or more than XSDL maxOccurs occurrences are found.
When unparsing, any number of occurrences are expected in the infoset. If validation is enabled, it is a validation error if less than minOccurs occurrences are found or defaulted, or if more than maxOccurs occurrences are found. The dfdl:occurs expression is not evaluated, The ‘count’ is the number of occurrences in the augmented infoset.
It is a schema definition error if dfdl:occursCount is not provided or in scope.
The enum 'stopValue' should be used when the the end of the array is signaled by an occurrence having a logical value that is equal to one of the specified 'stop values'.
When parsing, any number of occurrences is expected in the data, followed by an occurrence which is a stop value as specified by dfdl:occursStopValue. It is a processing error if a stop value occurrence is not found in the data (including when there are zero other occurrences). The parser stops looking for occurrences once a stop value has been found. If validation is enabled, it is a validation error if less than XSDL minOccurs occurrences are found or defaulted, or more than XSDL maxOccurs occurrences are found, not including the stop value.
When unparsing, the behavior is the same as for 'parsed', with the addition that a stop value occurrence is output after the last infoset occurrence. If dfdl:occursStopValue provides multiple stop values then the first is used.
The stop value itself is not added to the infoset when parsing. It is a processing error if a stop value is found in the infoset when unparsing. (This insures that the array can be reparsed, as the stop value will be placed automatically and only at the end.)
It is a schema definition error if dfdl:occursStopValue is not provided or in scope.
It is a schema definition error if the type of the element is complex.
It is a schema definition error if any of the stop values provided by dfdl:occursStopValue do not conform to the simple type of the element.
When parsing, required occurrences that have empty representation may trigger the application of a default value, as described in Section 9.4.2 Element Defaults When Parsing.
When unparsing, required occurrences that are missing from the infoset may trigger the application of a default value, as described in Section 9.4.3 Element Defaults When Unparsing.
The application of default values is not dependent on dfdl:occursCountKind, only on whether the occurrence is required or optional, whether there is a default value specified, and whether the data contains the empty representation (parsing) or is missing (unparsing). Section 9.4 Element Defaults contains the details.
If the value of a DFDL property of an array element (other than dfdl:occursCount) is given by a DFDL Expression, then the expression must be re-evaluated for each occurrence of the element in case the value changes.
Arrays can have points of uncertainty depending on the value of dfdl:occursCountKind. See Section 9.3.3 Points of Uncertaintyfor details.
In some situations arrays of elements and sequence groups of elements seem to be similar; however, there is no notion of the array itself independent of its contained elements. Arrays are distinctly different from sequence groups in this way.
A sequence can have its own initiator, and a complex element having that sequence as its content can also have its own initiator, so you could express two different initiators.
Unlike a sequence group, an array does not have its own initiator, terminator, or alignment. Those properties apply to each element occurrence of the array. To give an alignment, initiator, separator or terminator to an entire array you must enclose the element declaration for the array in a sequence group and specify the alignment, separator, initiator and terminator on the sequence group.
To prevent an infinite loop, the parsing of an array that is potentially unbounded must terminate when the following are true:
· The occurrence is a point of uncertainty;
· The position in the data does not move during the parsing of the occurrence (including any associated Separator, PrefixSeparator or PostfixSeparator region);
·
The occurrence is known-to-exist with
empty representation or nil representation.
An array is potentially unbounded if any of the following are true:
· dfdl:occursCountKind is 'stopValue'
· dfdl:occursCountKind is 'parsed'
·
dfdl:occursCountKind is 'implicit' and
XSDL maxOccurs is unbounded
When dfdl:occursCountKind is
'stopValue' this results in a processing error because the stop value will
never be encountered.
Further, to prevent unnecessary consumption of resources for large bounded values of XSDL maxOccurs, the parsing of an array must terminate when the following are true:
· dfdl:occursCountKind is 'implicit';
· The occurrence is a point of uncertainty;
· The position in the data does not move during the parsing of the occurrence (including any associated Separator, PrefixSeparator or PostfixSeparator region);
· The occurrence is known-to-exist with empty representation.
When parsing a single array, it is possible to extract occurrences that have different representations (nil, empty, normal, absent) although with some values of dfdl:lengthKind certain combinations of representations are not possible.
Occurrences with nil representation are added to the infoset with [nilled] member true.
Occurrences with empty representation may or may not be added to the infoset, as described in Section 9.4. If a required occurrence is not added to the infoset, it may be a processing error, dependent on dfdl:occursCountKind as described in section 16.1.
Occurrences with absent representation are not added to the infoset. For a required occurrence it may be a processing error, dependent on dfdl:occursCountKind as described in section 16.1.
Consider parsing an array where optional occurrences with empty representation are present in the data, but there are also later optional occurrences present with normal representation. Such an array is called a 'sparse array'.
If the indices of the occurrences are significant and need to be preserved, then the array may be modelled using an element with XSDL nillable 'true', dfdl:nilKind 'literalValue', dfdl:nilValue '%ES;' and dfdl:nilValueDelimiterPolicy the same as dfdl:emptyValueDelimiterPolicy. The occurrences with empty representation now become occurrences with nil representation, and will produce nil values in the infoset, so the absolute positions of all occurrences are preserved.
If the indices of the occurrences are not significant, then the array should be modelled using an element with XSDL nillable 'false'. Optional occurrences with empty representation will not create items in the infoset, so the absolute position of any optional occurrences with normal representation is not preserved. Optional occurrences with empty representation are therefore skipped.
This section describes properties which allow the creation of calculated elements. When parsing, the value of a calculated element is derived using a DFDL Expression, and not by processing bytes from the data stream. When unparsing, the value of a calculated element is derived using a DFDL Expression, and is not obtained from the infoset in the usual way.
Calculated elements allow a technique that is commonly called layering. In this technique, some elements are said to be in the physical layer, and some in the logical layer. When parsing, the logical layer values are computed from physical layer values. When unparsing the opposite occurs, that is the physical layer values are computed from the logical layer values.
Calculated elements are commonly used with hidden elements so as to hide the physical layer elements so that they do not become part of the infoset.
When a DFDL Schema is used to both parse and unparse data, then a calculated element on parsing will normally have one or more calculated elements on unparsing.
These properties apply to elements of simple type.
|
Property Name |
Description |
|
inputValueCalc |
DFDL Expression An expression that calculates the value of the element when parsing. It is a schema definition error if the result type of the expression does not conform to the base type of the element. The element value created using dfdl:inputValueCalc is validated like any other element value (when validation is enabled). An element that specifies a dfdl:inputValueCalc expression has no representation of its own in the data stream. All other DFDL representation properties are ignored. When an element which carries this property appears in a sequence that has a separator, no separator is associated with the element. When parsing, no separator is expected in the input data. When unparsing, no separator is written to the output data. The element must not be optional nor an array nor be global. The DFDL Expression must not refer to this element nor cause a circular reference to this element. The expression must not contain forward references to elements which have not yet been processed. It is a schema definition error if this property is specified on an element which has an XSDL fixed or default property. It is a schema definition error if dfdl:inputValueCalc and dfdl:outputValueCalc are specified on the same element. It is not possible to place this property in scope on a dfdl:format annotation. This property is not allowed to appear on a local element or element reference that is the root of a choice branch. If this property appears on an element declaration or element reference schema component, the appearance of any other DFDL properties on that component is a schema definition error. If this property appears on an element reference, then DFDL properties expressed on the referenced global element declaration or its type are ignored. If this property appears on an element declaration, then DFDL properties expressed on its type are ignored. Annotation: dfdl:element |
|
outputValueCalc |
DFDL Expression An expression that calculates the value of the current element when unparsing. The element must not be optional nor an array nor be global. It is a schema definition error if the result type of the expression does not conform to the base type of the element. The value created using dfdl:outputValueCalc is validated like any other element value (when validation is enabled). The value for the element, if any, in the infoset is ignored. The DFDL expression must not refer to this element nor cause a circular reference to this element. The expression may contain forward references to elements which have not yet been processed. It is a schema definition error if dfdl:outputValueCalc is specified on an element which has an XSDL fixed or default property. It is a schema definition error if dfdl:inputValueCalc and dfdl:outputValueCalc are specified on the same element. It is not possible to place this property in scope on a dfdl:format annotation. Annotation: dfdl:element |
Table 54 Calculated Value Properties
Consider this simple example. The data stream contains two elements giving the number of rows and number of columns of an array of numbers. The representation of the array is stored after these two elements.
<xs:complexType name="array">
<xs:sequence dfdl:initiator="" >
<xs:sequence dfdl:hiddenGroupRef="tns:hiddenArrayCounts"/>
<xs:element name="rows" maxOccurs="unbounded"
dfdl:occursCountKind="expression"
dfdl:occursCount="{ ../nrows }">
<xs:complexType>
<xs:sequence>
<xs:element name="cols" type="xs:float" maxOccurs="unbounded"
dfdl:occursCountKind="expression"
dfdl:occursCount=" { ../../ncols } " />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
<xs:group name="hiddenArrayCounts" >
<xs:sequence>
<xs:element name="nrows" type="xs:unsignedInt"
dfdl:representation="binary"
dfdl:lengthKind="implicit"
dfdl:outputValueCalc="{ count(../rows) }"/>
<xs:element name="ncols" type="xs:unsignedInt"
dfdl:representation="binary"
dfdl:lengthKind="implicit"
dfdl:outputValueCalc=
"{ if ( count(../rows) ge 1 )
then
count(../rows[1]/cols)
else
0
}"/>
</xs:sequence>
</xs:group>
In the example above we see that there are two hidden elements named 'nrows' and 'ncols'. These hidden elements' values are computed when unparsing from the number of occurrences in the 'rows' and 'cols' repeating elements. The 'rows' and 'cols' repeating elements number of occurrences are computed when parsing from the hidden elements 'nrows' and 'ncols'.
Logically, the data is a date.
<xs:element name="d" type="date"/>
Physically, it is stored as 3 single byte integers.
The format of this data is expressed as this schema:
<xs:sequence dfdl:representation="binary">
<xs:element name="mm" type="byte" />
<xs:element name="dd" type="byte" />
<xs:element name="yy" type="byte"/>
</xs:sequence>
This physical representation can be hidden so that it does not become part of the infoset:
<xs:sequence>
<xs:sequence dfdl:hiddenGroupRef="tns:hiddenpDate"/>
<xs:element name="d" type="date">
…
</xs:element>
</xs:sequence>
<xs:group name="hiddenpDate" >
<xs:sequence>
<xs:element name="pdate">
<xs:complexType>
<xs:sequence dfdl:representation="binary">
<xs:element name="mm" type="byte" />
<xs:element name="dd" type="byte" />
<xs:element name="yy" type="byte"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:group>
A calculation can be used to compute the logical date element 'd' from the physical 'pdate' when parsing:
<xs:sequence>
... hidden pdate here ...
<xs:element name="d" type="date">
<xs:annotation><xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:element>
<dfdl:property name="inputValueCalc">
{
fn:date(fn:concat(if(../pdate/yy gt 50 )then "19" else "20",
if ( ../pdate/yy gt 9 )
then fn:string(../pdate/yy)
else fn:concat("0",
fn:string(../pdate/yy)),
"-",
fn:string(../pdate/mm),
"-",
fn:string(../pdate/dd)))
}
</dfdl:property>
</dfdl:element>
</xs:appinfo></xs:annotation>
</xs:element>
...
</xs:sequence>
The expression above assembles a string resembling, for example, "2005-12-17" or "1957-3-9" which is the string representation of a date that is acceptable to the fn:date constructor function. The hidden element 'pdate' is referenced by relative paths. The expression '../pdate/yy' accesses an element of type 'int', and the fn:string constructor function turns it into an integer.
Finally, we must handle the unparse case where the physical layer is computed from the logical layer:
<xs:sequence dfdl:representation="binary"
<xs:element name="mm" type="byte"
dfdl:outputValueCalc="{ fn:month-from-date(../d) }" />
<xs:element name="dd" type="byte"
dfdl:outputValueCalc="{ fn:day-from-date(../d) }" />
<xs:element name="yy" type="byte"
dfdl:outputValueCalc="{ fn:year-from-date(../d) idivmod 100 }"/>
</xs:sequence>
The entire example in one place:
<xs:sequence>
<xs:sequence dfdl:hiddenGroupRef="tns:hiddenpDate"/>
<xs:element name="d" type="date">
<xs:annotation><xs:appinfo source="http://www.ogf.org/dfdl/">
<dfdl:element>
<dfdl:property name="inputValueCalc">
{
fn:date(fn:concat(if(../pdate/yy gt 50) then "19" else "20",
if ( ../pdate/yy gt 9 )
then fn:string(../pdate/yy)
else fn:concat("0",
fn:string(../pdate/yy)),
"-",
fn:string(../pdate/mm),
"-",
fn:string(../pdate/dd)))
}
</dfdl:property>
</dfdl:element>
</xs:appinfo></xs:annotation>
</xs:element>
...
</xs:sequence>
<xs:group name="hiddenpDate" >
<xs:sequence>
<xs:element name="pdate">
<xs:complexType>