7.6. Fun with Perl EnhancementsMany regular-expression concepts that are now available in other languages were first made available only in Perl. Examples include non-capturing parentheses, lookahead, (and later, lookbehind), free-spacing mode, (most modes, actually and with them comes \A, \z, and \Z), atomic grouping, \G, and the conditional construct. However, these are no longer Perl specific, so they are all covered in the main chapters of this book. Still, Perl developers remain innovative, so there are some major concepts available at this time only in Perl. One of the most interesting is the ability to execute arbitrary code during the match attempt. Perl has long featured strong integration of regular expressions into code, but this brings integration to a whole new level. We'll continue with a short overview about this and other innovations available currently only in Perl, followed by the details.
When working with Perl code embedded within a regex (either in a dynamic regex construct or an embedded-code construct), it's best to use only global variables until you understand the important issue related to my variables discussed starting on page 338. 7.6.1. Using a Dynamic Regex to Match Nested PairsA dynamic regex's main use is to allow a regex to match arbitrarily nested constructs (something long thought to be impossible with regular expressions). Its quintessential example is to match content with arbitrarily nested parentheses. To see how a dynamic regex is useful for this, let's first look at why it's not possible with traditional constructs. This simple regex matches a parenthesized run of text: \(([^()])*\). It doesn't allow parentheses within the outer parentheses, so it doesn't allow any nesting (that is, it supports zero levels of nesting). We can put it into a regex object and use it like this:
my $Level0 = qr/ \( ( [^()] )* \) /x; # Parenthesized text
.
.
.
if ($text =~ m/\b( \w+$Level0 )/x) {
print "found function call: $1\n";
}
This would match "substr($str, 0, 3), but not "substr($str, 0, (3+2))" because it has nested parentheses. Let's expand our regex to handle it. That means accommodating one level of nesting. Allowing one level of nesting means allowing parenthesized text within the outer parentheses. So, we need to expand on the subexpression that matches between them, which is currently [^()], by adding a subexpression that matches parenthesized text. Well, we just created that: $Level0 holds such a regex. Using it, we can create the next level: my $Level0 = qr/ \( ( [^()] )* \) /x; # Parenthesized text my $Level1 = qr/ \( ( [^()]| $Level0 )* \) /x; # One level of nesting The $Level0 here is the same as before; what's new is its use in building $Level1, which matches its own set of parentheses, plus those of $Level0. That's one level of nesting. To match another level, we can use the same approach, creating a $Level2 that uses $Level1 (which still uses $Level0): my $Level0 = qr/ \( ( [^()] )* \) /x; # Parenthesized text my $Level1 = qr/ \( ( [^()] | $Level0 )* \) /x; # One level of nesting my $Level2 = qr/ \( ( [^()] | $Level1 )* \) /x; # Two levels of nesting We can continue this indefinitely: my $Level3 = qr/ \( ( [^()] ; $Level2 )* \) /x; # Three levels of nesting my $Level4 = qr/ \( ( [^()] ; $Level3 )* \) /x; # Four levels of nesting my $Level5 = qr/ \( ( [^()] ; $Level4 )* \) /x; # Five levels of nesting Figure 7-1 shows the first few levels graphically. Figure 7-1. Matching a few levels of parenthesesIt's interesting to see the result of all those levels. Here's what $Level3 boils down to: \(([^()]|\(([^()]|\(([^()]|\(([^()])*\))*\))*\))*\) Wow, that's ugly. Luckily, we don't have to interpret it directly (that's the regex engine's job). The approach with the Level variables is easy enough to work with, but its drawback is that nesting is limited to however many $Level variables we build. This approach doesn't allow us to match to an arbitrary level. (Murphy's Law being what it is, if we happen to pick X levels to support, we'll run into data with X+1 levels of nesting.) Luckily, we can use a dynamic regex to handle nesting to an arbitrary level. To get there, realize that each of the $Level variables beyond the first is constructed identically: when it needs to match an additional level of nesting, it includes the $Level variable below it. But if the $Level variables are all the same, it could just as well include the $Level above it. In fact, if they're all the same, it could just include itself. If it could somehow include itself when it wanted to match another level of nesting, it would recursively handle any level of nesting. And that's just what we can do with a dynamic regex. If we create a regex object comparable to one of the $Level variables, we can refer to it from within a dynamic regex. (A dynamic-regex construct can contain arbitrary Perl code, so long as its results can be interpreted as a regular expression; Perl code that merely returns a pre-existing regex object certainly fits the bill.) If we put our $Level-like regex object into $LevelN, we can refer to it with (??{ $LevelN }), like this:
my $LevelN; # This must be predeclared because it's used in its own definition.
$LevelN = qr/ \(( [^()] | (??{ $LevelN }) )* \) /x;
This matches arbitrarily nested parenthesized text, and can be used just like $Level0 was used earlier: if ($text =~ m/\b( \w+$LevelN )/x) { print "found function call: $1\n"; } Phew! It's not necessarily easy to wrap one's brain around this, but once it "clicks," it's a valuable tool. Now that we have the basic approach worked out, I'd like to make a few tweaks for efficiency's sake. I'll replace the capturing parentheses with atomic grouping (there's no need to capture, nor to backtrack), and once that's done, I can change [^()] to [^()]+ for added efficiency. (Don't make this change without using atomic grouping, or you'll set yourself up for a neverending match ☞ 226.) Finally, I'd like to move the \( and \) so that they directly surround the dynamic regex. This way, the dynamic regex construct isn't invoked by the engine until it's sure that there's something for it to match. Here's the revised version: $LevelN = qr/ (?> [^()]+ | \( (??{ $LevelN }) \) )* /x; Since this no longer has outer \(···\), we need to include them ourselves when invoking $LevelN. As a side effect of that, we have the flexibility to apply it where there may be sets of parentheses, not just where there are sets of parentheses: if ($text =~ m/\b( \w+ \( $LevelN \) )/x) { print "found function call: $1\n"; } ----------------------------------- if (not $text =~ m/^ $LevelN $/x) { print "mismatched parentheses!\n"; } You can see another example of $LevelN in action on page 343. 7.6.2. Using the Embedded-Code ConstructThe embedded-code construct is particularly useful for regex debugging, and for accumulating information about a match while it's happening. The next few pages walk through a series of examples that eventually lead to a method for mimicking POSIX match semantics. The journey there is perhaps more interesting than the actual destination (unless you need POSIX match semantics, of course) because of the useful techniques and insight we gain along the way. We'll start with some simple regex debugging techniques. <IP_ADDRESS>. Using embedded code to display match-time informationThis code: "abcdefgh" =~ m{ (?{ print "starting match at [$'|$']\n" }) (?:d|e|f) }x; produces: starting match at [|abcdefgh] starting match at [a|bcdefgh] starting match at [ab|cdefgh] starting match at [abc|defgh] The embedded-code construct is the first thing in the regex, and so executes print "starting match at [$'|$']\n" whenever the regex starts a new match attempt. The displayed string uses the $' and $' variables (☞ 300)[] to print the target text being matched, with '|' inserted to mark the current location in the match (which in this case is where the match attempt is starting). From the result, you can tell that the regex was applied four times by the transmission (☞ 148) before it was successful.
In fact, if we were to add (?{ print "matched at [$'<$&>$']\n" }) just before the end of the regex, it would show the match: matched at [abc<d>efgh] Now, compare the first example with the following, which is identical except that the "main" regex is now [def] rather than (?:d|e|f): "abcdefgh" =~ m{ (?{ print "starting match at [$'|$']\n" }) [def] }x; In theory, the results should be identical, yet this produces only: starting match at [abc|defgh] Why the difference? Perl is smart enough to apply the initial class discrimination optimization (☞ 247) to the regex with [def], thereby allowing the transmission to bypass attempts it felt were obviously destined to fail. As it turns out, it was able to bypass all attempts except the one that resulted in a match, and the embedded-code construct allows us to see that happen. <IP_ADDRESS>. Using embedded code to see all matchesPerl has a Traditional NFA engine, so it stops the moment a match is found, even though there may be additional possible matches. With the clever use of embedded code, we can trick Perl into showing us all possible matches. To see how, let's revisit the silly 'oneself' example from page 177: "oneselfsufficient" =~ m{ one(self)?(selfsufficient)? (?{ print "matched at [$'<$&>$']\n" }) }x; As might be expected, this displays matched at [<oneself>sufficient] indicating that '' had been matched at that point in the regex. It's important to realize that despite the "matched" in the message, the print is not actually showing "the match," but rather the match to that point. The distinction is academic with this example because the embedded-code construct is the last thing in the regex. We know that the regex does indeed finish the moment the embedded-code construct has finished, reporting that same result as the actual match. What if we added (?!) just after the embedded-code construct? (?!) is a negative lookahead that always fails. When it fails just after the embedded code is processed (just after a "matched" message is printed), it forces the engine to backtrack in search of a (new) match. The failure is forced after every "match" is printed, so we end up exploring every path to a match, and thus see all possible matches: matched at [<oneself>sufficient] matched at [<oneselfsufficient>] matched at [<one>selfsufficient] What we've done ensures that the overall match attempt actually fails, but in doing so we've got the regex engine to report all the possible matches. Without the (?!), Perl returns the first match found, but with it, we can see the remaining permutations. With that in mind, what do you think the following prints?
"123" =~ m{
\d+
(?{ print "matched at [$'<$&>$']\n" })
(?!)
}x;
It displays: matched at [<123>] matched at [<12>3] matched at [<1>23] matched at [1<23>] matched at [1<2>3] matched at [12<3>] Hopefully at least the first three were expected, but the rest might be unexpected if you're not on your toes. The (?!) forces backtracking and the eventual appearance of the 2nd and 3rd lines. When the attempt at the start of the line fails, the transmission reapplies the regex again starting just before the 2nd character. (Chapter 4 explains this in great detail.) The 4th and 5th lines shown are from that second attempt, and the last line shown is from the third attempt. So, adding the (?!) really does cause it to show all possible matches, not just all of them from a particular starting point. It may be useful to see only the possible matches from a particular starting point; we'll look into that in a bit. <IP_ADDRESS>. Finding the longest matchNow, instead of showing all the matches, let's find and save the longest match. We can do this by using a variable to keep track of the longest match seen so far and comparing each new "almost match" against it. Here is the solution with the 'oneself' example: $longest_match = undef; # We'll keep track of the longest match here "oneselfsufficient" =~ m{ one(self)?(selfsufficient)? (?{ # Check to see if the current match ($&) is the longest so far if (not defined($longest_match) or length($&) > length($longest_match)) { $longest_match = $&; } }) (?!) # Force failure so we'll backtrack to find further "matches" }x; # Now report the accumulated result, if any if (defined($longest_match)) { print "longest match=[$longest_match]\n"; } else { print "no match\n"; } Not surprisingly, this shows 'longest match=[oneselfsufficient]'. That bit of embedded code is pretty long, and something we'll likely use in the future, so let's encapsulate it and the (?!) into their own regex object: my $RecordPossibleMatch = qr{ (?{ # Check to see if the current match ($&) is the longest so far if (not defined($longest_match) or length($&) > length($longest_match)) { $longest_match = $&; } }) (?!) # Force failure so we'll backtrack to find further "matches" }x; Here's a simple example that finds '9938', the longest match overall: $longest_match = undef; # We'll keep track of the longest match here "800-998-9938" =~ m{ \d+ $RecordPossibleMatch }x; # Now report the accumulated result, if any if (defined($longest_match)) { print "longest match=[$longest_match]\n"; } else { print "no match\n"; } <IP_ADDRESS>. Finding the longest-leftmost matchNow that we know how to find the longest match overall, let's restrict it to finding the longest-leftmost match. That just happens to be the match that a POSIX NFA would find (☞ 177). To accomplish this, we need to disable the transmission's bump-ahead if we've seen a match so far. That way, once we find the first match, normal backtracking still brings us to any other matches available from the same starting location (allowing us to keep track of the longest match), but the disabled bump-ahead inhibits the finding of matches that start later in the string. Perl doesn't give us direct hooks into the transmission, so we can't disable the bump-ahead directly, but we can get the same effect by not allowing the regex to proceed past the start if $longest_match is already defined. The test for that is (?{ defined $longest_match}), but that alone not enough, since it's just a test. The key to using the results of the test lies in a conditional. <IP_ADDRESS>. Using embedded code in a conditionalTo have the regex engine respond to the results of our test, we use the test as the if of an (? if then | else) conditional (☞ 140). Since we want the regex to stop if the test is true, we use a fail-now (?!) as the then part. (We don't need an else part, so we just omit it.) Here's a regex object that encapsulates the conditional: The if part is underlined, and the then part is shown in bold. Here it is in action, combined with the $RecordPossibleMatch defined on the facing page:
"800-998-9938" =~ m{ $BailIfAnyMatch \d+ $RecordPossibleMatch }x;
This finds '800', the POSIX "longest of all leftmost matches" match. 7.6.3. Using local in an Embedded-Code ConstructThe use of local within an embedded-code construct takes on special meaning. Understanding it requires a good understanding of dynamic scoping (☞ 295) and of the "crummy analogy" from the Chapter 4's discussion of how a regex-directed NFA engine goes about its work (☞ 158). The following contrived (and, as we'll see, flawed) example helps to illustrate why, without a lot of extraneous clutter. It checks to see if a line is composed of only \w+ and \s+, but counts how many of the \w+ are really \d+\b: my $Count = 0; $text =~ m{ ^ (?> \d+ \b | \w+ | \s+ )*$ }x; When this is matched against a string like '123•abc•73•9271•xyz', the $Count variable is left with a value of three. However, when applied to '123••abc•73xyz' it's left with a value of two, even though it should be left with a value of just one. The problem is that $Count is updated after matching '73', something that is matched by \d+ but later "unmatched" via backtracking because the subsequent \b can't match. The problem arises because the code executed via the embeddedcode construct is not somehow "unexecuted" when its part of the regex is "unmatched" via backtracking. In case you have any confusion with the use of (?>⋯) atomic grouping (☞ 139) and the backtracking going on here, I'll mention that the atomic grouping is used to prevent a neverending-match (☞ 269), and does not affect backtracking within the construct, only backtracking back into the construct after it's been exited. So the \d+ is free to be "unmatched" if the subsequent \b cannot match. The easy solution for this contrived example is to put the \b before incrementing $Count, to ensure that it is incremented only when it won't be undone. However, I'd like to show a solution using local, to illustrate its effect within Perl executed during the application of a regex. With that in mind, consider this new version: our $Count = 0; $text =~ m{ ^ (?> \d+ (?{ local($Count) = $Count + 1 }) \b | \w+ | \s+ )* $ }x; The first change to notice is that $Count changed from a my variable to a global one (if you use strict, as I always recommend, you can't use an unqualified global variable unless you "declare" it with Perl's our declarator). The other change is that the increment of $Count has been localized. Here's the key behavior: when a variable is localized within a regex, the original value is replaced (the new value is lost) if the code with the local is "unmatched" because of backtracking. So, even though local($Count) = $Count + 1 is executed after '73' is matched by \d+, changing $Count from one to two, that change is "localized to the success of the path" that the regex is on when local is called. When the \b fails, the regex engine logically backtracks to before the local, and $Count reverts to its original value of one. And that's the value it ends up having when the end of the regex is eventually reached. So, local is required to keep $Count consistent until the end of the regex. If we were to put (?{ print "Final count is $Count.\n" }) at the end of the regex, it would report the proper count. Since we want to use $Count after the match, we need to save it to a non-localized variable at some point before the match officially ends. This is because all values that had been localized during the match are lost when the match finishes. Here's an example: my $Count = undef; our $TmpCount = 0; $text =~ m{ ^ (?> \d+ (?{ local($TmpCount) = $TmpCount + 1 }) \b | \w+ | \s+ )+ $ (?{ $Count = $TmpCount }) # Save the "ending" $Count to a non-localized variable }x; if (defined $Count) { print "Count is $Count.\n"; } else { print "no match\n"; } This seems like a lot of work for something so simple, but again, this is a contrived example designed just to show the mechanics of localized variables within a regex. We'll see practical use in "Mimicking Named Capture" on page 344. 7.6.4. A Warning About Embedded Code and my VariablesIf you have a my variable declared outside a regex, but refer to it from inside regex embedded code, you must be very careful about a subtle issue with Perl's variable binding that has a very unsubtle impact. Before describing the issue, I'll note up front that if you use only global variables within regex embedded code, you don't have to worry about this issue, and you can safely skip this section. Warning: this section is not light reading. This contrived example illustrates the problem: sub CheckOptimizer { my $text = shift; # The first argument is the text to check. my $start = undef; # We'll note here where the regex is first applied. my $match = $text =~ m{ (?{ $start = $-[0] if not defined $start}) # Save the first starting position \d # This is the regex being tested }x; if (not defined $start) { print "The whole match was optimized away.\n"; if ($match) { # This can't possibly happen! print "Whoa, but it matched! How can this happen!?\n"; } } elsif ($start == 0) { print "The match start was not optimized.\n"; } else { print "The optimizer started the match at character $start.\n" } } This code has three my variables, but only one, $start, is related to this issue (the others are not referenced from within embedded code, so are not at issue). It works by first setting $start to the undefined value, then applying a match in which the leading component is an embedded-code construct that sets $start to the starting location of the attempt, but only if it hasn't already been set. The "starting location of the attempt" is derived from $-[0] (the first element of @- ☞ 302). So, when this function is called with CheckOptimizer("test 123"); the result is: The optimizer started the match at character 5. That's OK, but if we invoke the exact same call again, the second time shows: The whole match was optimized away. Whoa, but it matched! How can this happen!? Even though the text checked by the regex is the same (as is the regex itself, for that matter), the result is different, and seems to be wrong. Why? The problem is that the second time through, the $start that the embedded code is updating is the one that existed the first time through, when the regex was compiled. The $start that the rest of the function uses is actually a new variable created afresh when the my is executed at the start of each function call. The key to this issue is that my variables in embedded code are "locked in" (bound, in programming terminology) to the specific instance of the my variable that is active at the time the regex is compiled. (Regex compilation is discussed in detail starting on page 348.) Each time CheckOptimizer is called, a new instance of $start is created, but for esoteric reasons, the $start inside the embedded code still refers to the first instance that is now longone. Thus, the instance of $start that the rest of the function uses doesn't receive the value ostensibly written to it within the regex. This type of instance binding is called a closure, and books like Programming Perl and Object Oriented Perl discuss why it's a valuable feature of the language. There is debate in the Perl community, however, as to just how much of a "feature" it is in this case. To most people, it's very unintuitive. The solution is to not refer to my variables from within a regex unless you know that the regex literal will be compiled at least as often as the my instances are refreshed. For example, the my variable $NestedStuffRegex is used within the SimpleConvert subroutine in the listing on page 345, but we know this is not a problem because there's only ever one instance of $NestedStuffRegex. Its my is not in a function or a loop, so it's created just once when the script is loaded, with that same instance existing until the program ends. 7.6.5. Matching Nested Constructs with Embedded CodeThe example on page 328 shows how to match arbitrarily nested pairs using a dynamic regex. That's generally the easiest way to do it, but it's instructive to see a method using only embedded-code constructs, so I'd like to show it to you here. The approach is simply this: keep a count of how many open parentheses we've seen that have not yet been closed, and allow a closing parenthesis only if there are outstanding opens. We'll use embedded code to keep track of the count as we match through the text, but before looking at that, let's look at a (not yet working) skeleton of the expression: my $NestedGuts = qr{ (?> (?: # Stuff not parenthesis [^()]+ # An opening parenthesis | \( # A closing parenthesis | \) )* ) }x; The atomic grouping is required for efficiency, to keep the ( [⋯]+ |⋯)* from becoming a neverending match (☞ 226) if $NestedGuts is used as part of some larger expression that could cause backtracking. For example, if we used it as part of m/^\( $NestedGuts \)$/x and applied it to '(this•is•missing•the•close', it would track and backtrack for a long time if atomic grouping didn't prune the redundant states. To incorporate the counting, we need these four steps:
Adding these items to the skeleton expression gives us: my $NestedGuts = qr{ (?{ local $OpenParens = 0 }) # ❶ Counts the number of nested opens waiting to close. (?> # atomic-grouping for efficiency (?: # Stuff not parenthesis [^()]+ # ❷ An opening parenthesis | \( (?{ $OpenParens++ }) # ❸ Allow a closing parenthesis, if we're expecting any | \) (?(?{ $OpenParens != 0 }) (?{ $OpenParens-- }) | (?!) ) )* ) (?(?{ $OpenParens != 0 })(?!)) # ❹ If there are any open parens left, don't finish }x; This can now be used just like $LevelN on page 330. The local is used as a precaution to isolate this regex's use of $OpenParens from any other use the global variable might have within the program. Unlike local's use in the previous section, it's not needed for backtracking protection because the atomic grouping in the regex ensures that once an alternative has been matched, it can't ever be "unmatched." In this case, the atomic grouping is used for both efficiency and to absolutely ensure that the text matched near one of the embedded-code constructs can't be unmatched by backtracking (which would break the sync between the value of $OpenParens and the number of parentheses actually matched). 7.6.6. Overloading Regex LiteralsYou can pre-process the literal parts of a regex literal in any way you like with overloading. The next sections show examples. <IP_ADDRESS>. Adding start- and end-of-word metacharactersPerl doesn't support \< and \> as start- and end-of-word metacharacters, and that's probably because it's rare that \b doesn't suffice. However, if we wish to have them, we can support them ourselves using overloading to replace '\<' and '\>' in a regex by (?<!\w)(?=\w) and (?<=\w)(?!\w), respectively. First, we'll create a function, say, MungeRegexLiteral, that does the desired preprocessing: sub MungeRegexLiteral($) { my ($RegexLiteral) = @_; # Argument is a string $RegexLiteral =~ s/\\</(?<!\\w)(?=\\w)/g;# Mimic \< as start-of-word boundary $RegexLiteral =~ s/\\>/(?<=\\w)(?!\\w)/g;# Mimic \> as end-of-word boundary return $RegexLiteral; # Return possibly-modified string } When this function is passed a string like '⋯\<⋯', it converts it and returns the string '⋯(?<!\w)(?=\w)⋯'. Remember, because the replacement part of a substitution is like a double-quoted string, it needs '\\w' to get '\w' into the value. Now, to install this so that it gets called automatically on each literal part of a regex literal, we put it into a file, say MyRegexStuff.pm, with the Perl mechanics for overloading: package MyRegexStuff; # Best to call the package something unique use strict; # Good practice to always use this use warnings; # Good practice to always use this use overload; # Allows us to invoke Perl's overloading mechanism # Have our regex handler installed when we're use'd .... sub import { overload::constant qr => \&MungeRegexLiteral } sub MungeRegexLiteral($) { my ($RegexLiteral) = @_; # Argument is a string $RegexLiteral =~ s/\\</(?<!\\w)(?=\\w)/g; # Mimic \< as start-of-word boundary $RegexLiteral =~ s/\\>/(?<=\\w)(?!\\w)/g; # Mimic \> as end-of-word boundary return $RegexLiteral; # Return possibly-modified string } 1; # Standard idiom so that a 'use' of this file returns something true If we place MyRegexStuff.pm in the Perl library path (see PERLLIB in the Perl documentation), we can then invoke it from Perl script files in which we want the new features made available. For testing, though, we can just leave it in the same directory as the test script, invoking it with: use lib '.'; # Look for library files in the current directory use MyRegexStuff; # We now have our new functionality available! $text =~ s/\s+\</ /g; # Normalize any type of whitespace before a word to a single space We must use MyRegexStuff in any file in which we want this added support for regex literals, but the hard work of building MyRegexStuff.pm need be done only once. (The new support isn't available in MyRegexStuff.pm itself because it doesn't use MyRegexStuff something you wouldn't want to do.) <IP_ADDRESS>. Adding support for possessive quantifiersLet's extend MyRegexStuff.pm to add support for possessive quantifiers like x++ (☞ 142). Possessive quantifiers work like normal greedy quantifiers, except they never give up (never "unmatch") what they've matched. They can be mimicked with atomic grouping by simply removing the final '+' and wrapping everything in atomic quotes, e.g., by changing regex*+ to (?>regex*) (☞ 173). The regex part can be a parenthesized expression, a metasequence like \w or \x{1234}, or even just a normal character. Handling all possible cases is difficult, so to keep the example simple for the moment, let's concentrate on ?+, *+, or ++ quantifying only a parenthesized expression. Using $LevelN from page 330, we can add $RegexLiteral =~ s/( \( $LevelN \)[*+?] )\+/(?>$1)/gx; to the MungeRegexLiteral function. That's it. Now, with it part of our overloaded package, we can use a regex literal with possessive quantifiers, like this example from page 198:
$text =~ s/"(\\.|[^"])*+"//; # Remove double-quoted strings
Extending this beyond just parenthesized expressions is tricky because of the variety of things that can be in a regular expression. Here's one attempt: $RegexLiteral =~ s{ ( # Match something that can be quantified ... (?: \\[\\abCdDefnrsStwWX] # \n, \w, etc. | \\c. # \cA | \\x[\da-fA-F]{1,2} # \xFF | \\x\{[\da-fA-F]*\} # \x{1234} | \\[pP]\{[^{}]+\} # \p{Letter} | \[\]?[^]]+\] # "poor man's" class | \\\W # \* | \( $LevelN \) # (⋯) | [^()*+?\\] # almost anything else ) # ...and is quantified... (?: [*+?] ; \{\d+(?:,\d*)?\} ) ) \+ # ...and has an extra '+' after the quantifier. }{(?>$1)}gx; The general form of this regex is the same as before: match something quantified possessively, remove the '+', and wrap the result in (?>⋯). It's only a half-hearted attempt to recognize the complex syntax of Perl regular expressions. The part to match a class is particularly needy, in that it doesn't recognize escapes within the class. Even worse, the basic approach is flawed because it doesn't understand every aspect of Perl regular expressions. For example, if faced with '\(blah\)++', it doesn't properly ignore the opening literal parenthesis, so it thinks the ++ is applied to more than just \). These problems can be overcome with great effort, perhaps using a technique that carefully walks through the regex from start to finish (similar to the approach shown in the sidebar on page 132). I'd like to enhance the part that matches a character class, but in the end, I don't feel it's worth it to address the other issues, for two reasons. The first is that the situations in which it doesn't already work well are fairly contrived, so just fixing the character class part is probably enough to make it acceptable in practice. But in the end, Perl's regex overloading currently has a fatal flaw, discussed in the next section, which renders it much less useful than it might otherwise be. 7.6.7. Problems with Regex-Literal OverloadingRegex-literal overloading can be extremely powerful, at least in theory, but unfortunately, it's not very useful in practice. The problem is that it applies to only the literal part of a regex literal, and not the parts that are interpolated. For example, with the code m/($MyStuff)*+/ our MungeRegexLiteral function is called twice, once with the literal part of the regex before the variable interpolation ("("), and once with the part after (")*+"). (It's never even given the contents of $MyStuff.) Since our function requires both parts at the same time, variable interpolation effectively disables it. This is less of an issue with the support for \< and \> we added earlier, since they're not likely to be broken up by variable interpolation. But since overloading doesn't affect the contents of an interpolated variable, a string or regex object containing '\<' or '\>' would not be processed by overloading. Also, as the previous section touched on, when a regex literal is processed by overloading, it's not easy to be complete and accurate every time. Even something as simple as our support for \> gets in the way when given '\\>', ostensibly to match a '\' followed by '>'. Another problem is that there's no way for the overload processing to know about the modifiers that the regex was applied with. In particular, it may be crucial to know whether /x was specified, but there's currently no way to know that. Finally, be warned that using overloading disables the ability to include characters by their Unicode name (\N{name} ☞ 290). 7.6.8. Mimicking Named CaptureDespite the shortcomings of overloading, I think it's instructive to see a complex example bringing together many special constructs. Perl doesn't offer named capture (☞ 138), but it can be mimicked with capturing parentheses and the $^N variable (☞ 301), which references the text matched by the most-recently-closed set of capturing parentheses. (I put on the hat of a Perl developer and added $^N support to Perl expressly to allow named-capture to be mimicked.) As a simple example, consider: href\s*=\s*($HttpUrl) This uses the $HttpUrl regex object developed on page 303. The underlined part is an embedded-code construct that saves the text matched by $HttpUrl to the variable $url. In this simple situation, it seems overkill to use $^N instead of $1, or to even use the embedded-code construct in the first place, since it seems so easy to just use $1 after the match. But consider encapsulating part of that into a regex object, and then using it multiple times: my $SaveUrl = qr{ ($HttpUrl) # Match an HTTP URL... (?{ $url = $^N }) # ...and save to $url }x; $text =~ m{ http \s*=\s* ($SaveUr) | src \s*=\s* ($SaveUrl) }xi; Regardless of which matches, $url is set to the URL that matched. Again, in this particular use, you could use other means (such as the $+ variable ☞ 301), but as $SaveUrl is used in more complex situations, the other solutions become more difficult to maintain, so saving to a named variable can be much more convenient. One problem with this example is that values written to $url are not "unwritten" when the construct that wrote to them is unmatched via backtracking. So, we need to modify a localized temporary variable during the initial match, writing to the "real" variable only after an overall match has been confirmed, just as we did in the example on page 338. The listing below shows one way to solve this. From the user's point of view, after using (?<Num>\d+), the number matched by \d+ is available in the global hash %^N, as $^N{Num}. Although future versions of Perl could decide to turn %^N into a special system variable of some sort, it's not currently special, so we're free to use it. I could have chosen a name like %NamedCapture, but instead chose %^N for a few reasons. One is that it's similar to $^N. Another is that it's not required to be predeclared with our when used under use strict. Finally, it's my hope that Perl will eventually add named capture natively, and I think adding it via %^N would be a fine idea. If that happens, %^N would likely be automatically dynamically scoped like the rest of the regex-related special variables (☞ 299). But as of now, it's a normal global variable, so is not dynamically scoped automatically. Again, even this more-involved approach suffers from the same problems as anything using regex-literal overloading, such as an incompatibility with interpolated variables. |


Committed Rated [R], 98 minutes Starring Heather Graham, Casey Affleck, Luke Wilson Written and Directed by Lisa Krueger website: www.miramax.com
IN SHORT: talk... talk... more talk...
It isn't a good thing to be watching the big screen thinking, "My God Heather Graham is skinny..."
Graham holds the spotlight for almost every second of Committed, as Joline, a club manager who is embarking on her lifelong commitment to love of her life Carl (Luke Wilson). Carl is a newspaper photographer who years for the excitement of shooting breaking news stories, but who is always assigned to take pictures of food. 597 days later, Joline is planning a surprise birthday party for her hubby, four months too soon, because he seems to be in a fog. It's more than a fog. Carl walks out and disappears out West somewhere, sending only a catus picture postcard as news that he is still thinking of Jo.
To Jo, whose word is her bond, "for better or for worse, 'til death do us part" means exactly that. Doing a careful budget, and renting a car, she sets out to find her stray, using a smudged postmark as a guide and clipping coupons all along the way. The journey leads to Texas, near the Mexican border at El Paso.  If ever there was a visual representation of "getting lost in the wilderness," this is it. There she not only finds Carl, she meets and befriends his new girlfriend Carmen (Patricia Velazquez) and her wise-man grandfather (Alfonso Arau) who senses in her an untapped power . . .
Actually, anyone with a modicum of intelligence knew from the very first shot of the movie that Joline was very in tune with witchery. Though that "w" word is never mentioned, we found ourselves thinking "is she a good witch or a bad witch?" as the flick played out. Turns out it's a bit of both and it's a bit of the story like all the other bits of the story that is underdeveloped and underwritten. What could be comic, as Joline surveils her hubby's house with brother Jay (Casey Affleck) and Carmen in the cars behind her, also watching, isn't.
Casey is such an underachiever that, as a counterpoint to his gangbuster sister, he isn't.  He's out of luck, as Jo is committed to Carl, as her ongoing voice over informs you.  It all reminds us of long ago, heavy duty college conversations about the same subjects, over well stoked bongs. Oh, yes it does (not that we have any info about writer/director Lisa Krueger's modus operandi). Aimed at a GenX audience, Committed would be perfect for that audience except that it is slow and way too analytical, in the way that Thai stick induced philosophy is . . . except for the lack of a good rush.
On average, a first run movie ticket will run you Eight Bucks. Were Cranky able to set his own price to Committed, he would have paid...
$1.00
Pass it by.
Cranky Critic® is a Registered Trademark of, and Copyright © 2000 by, Chuck Schwartz. All Rights Reserved. Cranky on the web at www.crankycritic.com
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


referenced
by Jane Sandberg at 2022-08-18 16:48:10.merged
by Jane Sandberg at 2022-08-18 16:48:10.closed
by Jane Sandberg at 2022-08-18 16:48:10.head_ref_deleted
by Jane Sandberg at 2022-08-18 16:48:11.closed
by Jane Sandberg at 2022-08-18 16:48:11.Yes, we would want to run the task one institution at a time. Yes, if make the calls for each partner close together you'll get the text response you posted. The application will start your job when the first partner dump requested concludes. Notifications will go to the email address passed in the API call.
I'm wondering if ultimately we'd want to create a scheduled task to generate a full dump from each partner say, every two weeks so we'll always have a relatively fresh copy available to us. @mzelesky and @pmgreen also both refer to this dataset for reporting and collection analysis reports so it might beneficial for them to have this data be available in a relatively current form.
mentioned
by Kevin Reiss at 2022-08-18 14:06:53.subscribed
by Kevin Reiss at 2022-08-18 14:06:53.mentioned
by Peter Green at 2022-08-18 14:53:47.subscribed
by Peter Green at 2022-08-18 14:53:47.mentioned
by None at 2022-08-18 14:53:47.subscribed
by None at 2022-08-18 14:53:47.mentioned
by Peter Green at 2022-08-18 14:59:31.subscribed
by Peter Green at 2022-08-18 14:59:31.mentioned
by None at 2022-08-18 14:59:31.subscribed
by None at 2022-08-18 14:59:31.review_requested
by Max Kadel at 2022-08-18 16:53:32.head_ref_force_pushed
by Francis Kayiwa at 2022-08-18 13:56:53.referenced
by Robert-Anthony Lee-Faison at 2022-08-18 15:23:35.merged
by Robert-Anthony Lee-Faison at 2022-08-18 15:23:35.closed
by Robert-Anthony Lee-Faison at 2022-08-18 15:23:35.head_ref_force_pushed
by Francis Kayiwa at 2022-08-18 19:01:18.head_ref_force_pushed
by Francis Kayiwa at 2022-08-18 19:44:14.assigned
by Kate Lynch at 2022-08-18 14:06:44.assigned
by Bess Sadler at 2022-08-18 14:06:53.Probably an issue with the structure manager as well.
labeled
by Eliot Jordan at 2022-08-18 18:18:52.We think this is similar to #5363, but is caused by the MemberValidator
. When we check validations for the example resource given we get the following:
irb(main):006:0> change_set.errors.full_messages
=> ["Member ids b5b3bda9-8f80-444a-b7b7-520d7e8f77fd does not resolve to a resource"]
referenced
by Eliot Jordan at 2022-08-18 19:34:00.merged
by Eliot Jordan at 2022-08-18 19:34:00.closed
by Eliot Jordan at 2022-08-18 19:34:00.head_ref_deleted
by Eliot Jordan at 2022-08-18 19:34:01.referenced
by Trey Pendragon at 2022-08-18 19:17:51.closed
by Eliot Jordan at 2022-08-18 19:34:01.labeled
by Trey Pendragon at 2022-08-18 20:56:11.assigned
by Shaun Ellis at 2022-08-18 15:10:34.closed
by Eliot Jordan at 2022-08-18 17:51:22.review_requested
by Shaun Ellis at 2022-08-18 15:18:25.referenced
by Eliot Jordan at 2022-08-18 17:51:21.merged
by Eliot Jordan at 2022-08-18 17:51:21.closed
by Eliot Jordan at 2022-08-18 17:51:21.head_ref_deleted
by Eliot Jordan at 2022-08-18 17:51:23.referenced
by Eliot Jordan at 2022-08-18 16:04:15.merged
by Eliot Jordan at 2022-08-18 16:04:15.closed
by Eliot Jordan at 2022-08-18 16:04:15.head_ref_deleted
by Eliot Jordan at 2022-08-18 16:04:19.referenced
by Eliot Jordan at 2022-08-18 16:04:34.merged
by Eliot Jordan at 2022-08-18 16:04:34.closed
by Eliot Jordan at 2022-08-18 16:04:34.head_ref_deleted
by Eliot Jordan at 2022-08-18 16:04:36.referenced
by Eliot Jordan at 2022-08-18 19:51:58.merged
by Eliot Jordan at 2022-08-18 19:51:58.closed
by Eliot Jordan at 2022-08-18 19:51:58.head_ref_deleted
by Eliot Jordan at 2022-08-18 19:52:01.Closed by https://github.com/pulibrary/orangelight/pull/2981
assigned
by Jane Sandberg at 2022-08-18 15:23:24.closed
by Jane Sandberg at 2022-08-18 15:27:59.unassigned
by Jane Sandberg at 2022-08-18 15:28:06.I was unable to recreate this issue: which page did this appear on @christinach ?
assigned
by Jane Sandberg at 2022-08-18 15:28:31.mentioned
by Christina Chortaria at 2022-08-18 15:58:18.subscribed
by Christina Chortaria at 2022-08-18 15:58:18.unassigned
by Jane Sandberg at 2022-08-18 15:58:23.Is this ticket still valid? The pagination on the search results page seems to be following this pattern pretty well: https://www.a11ymatters.com/pattern/pagination/.
Although I did see this note from Mozilla that "The aria-label attribute is intended for interactive elements only".
We are using stock blacklight pagination here, so if we want to make a change, we should consider contributing it upstream.
Closing as duplicate of #1063
closed
by Jane Sandberg at 2022-08-18 16:14:01.I believe this has been resolved. This appears on the show page, and now has #000 text on a #F5F5F5 background, a 19.26:1 contrast ratio that passes.
closed
by Jane Sandberg at 2022-08-18 16:29:59.assigned
by Jane Sandberg at 2022-08-18 16:49:36.Nooo! I'll check this out.
labeled
by Jane Sandberg at 2022-08-18 16:16:48.Coverage remained the same at 94.511% when pulling 8aa58a69967a9d0c2818aef6961a33cfaebb4aff on i1063invalidunapi into 12e080b9f0f73b1f1138a83bc313baa6fca73064 on main.
referenced
by Jane Sandberg at 2022-08-18 20:54:55.merged
by Jane Sandberg at 2022-08-18 20:54:56.closed
by Jane Sandberg at 2022-08-18 20:54:56.head_ref_deleted
by Jane Sandberg at 2022-08-18 20:54:56.assigned
by Jane Sandberg at 2022-08-17 23:05:44.closed
by Jane Sandberg at 2022-08-18 20:54:56.Coverage remained the same at 94.511% when pulling fced69d5ab5dac35ffc347c08dd6640bc277d0e6 on i1227-title-for-uv-iframe into 12e080b9f0f73b1f1138a83bc313baa6fca73064 on main.
deployed
by None at 2022-08-18 17:44:54.referenced
by Jane Sandberg at 2022-08-18 20:55:33.merged
by Jane Sandberg at 2022-08-18 20:55:33.closed
by Jane Sandberg at 2022-08-18 20:55:33.head_ref_deleted
by Jane Sandberg at 2022-08-18 20:55:34.closed
by Jane Sandberg at 2022-08-18 20:55:34.Closing as duplicate of #3083
closed
by Jane Sandberg at 2022-08-18 21:35:59.Is this ticket still valid? This link does have text ("Start over"), and it is announced by voiceover.
assigned
by James R. Griffin III at 2022-08-18 13:16:23.referenced
by Hector Correa at 2022-08-18 14:24:00.merged
by Hector Correa at 2022-08-18 14:24:00.closed
by Hector Correa at 2022-08-18 14:24:00.head_ref_deleted
by Hector Correa at 2022-08-18 14:24:00.convert_to_draft
by James R. Griffin III at 2022-08-18 13:24:10.renamed
by James R. Griffin III at 2022-08-18 13:25:52.assigned
by Hector Correa at 2022-08-18 15:40:07.closed
by Tyler Wade at 2022-08-18 16:02:15.head_ref_deleted
by Tyler Wade at 2022-08-18 16:02:25.I think the decrease in test coverage that Coveralls is flagging is because I deleted a bunch of tests since the code/classes that they were testing is now gone.
referenced
by Bess Sadler at 2022-08-18 16:07:15.merged
by Bess Sadler at 2022-08-18 16:07:15.closed
by Bess Sadler at 2022-08-18 16:07:15.head_ref_deleted
by Bess Sadler at 2022-08-18 16:07:16.closed
by Bess Sadler at 2022-08-18 16:07:16.head_ref_force_pushed
by None at 2022-08-18 13:59:08.head_ref_force_pushed
by None at 2022-08-18 14:00:00.head_ref_force_pushed
by None at 2022-08-18 15:50:28.referenced
by Bess Sadler at 2022-08-18 16:11:26.merged
by Bess Sadler at 2022-08-18 16:11:26.closed
by Bess Sadler at 2022-08-18 16:11:26.head_ref_deleted
by Bess Sadler at 2022-08-18 16:11:27.closed
by Bess Sadler at 2022-08-18 16:11:27.referenced
by Tyler Wade at 2022-08-18 19:15:32.merged
by Tyler Wade at 2022-08-18 19:15:32.closed
by Tyler Wade at 2022-08-18 19:15:32.head_ref_deleted
by Tyler Wade at 2022-08-18 19:15:32.closed
by Tyler Wade at 2022-08-18 19:15:33.review_requested
by Francis Kayiwa at 2022-08-18 14:18:19.referenced
by Francis Kayiwa at 2022-08-18 14:33:19.merged
by Francis Kayiwa at 2022-08-18 14:33:19.closed
by Francis Kayiwa at 2022-08-18 14:33:19.head_ref_deleted
by Francis Kayiwa at 2022-08-18 14:33:20.run this on the lib-adc1 and lib-adc2
successfully
referenced
by Francis Kayiwa at 2022-08-18 15:20:38.merged
by Francis Kayiwa at 2022-08-18 15:20:38.closed
by Francis Kayiwa at 2022-08-18 15:20:38.head_ref_deleted
by Francis Kayiwa at 2022-08-18 15:20:38.head_ref_force_pushed
by Alicia Cozine at 2022-08-18 21:57:03.


BACK
After 10th
Science
Engineering
Compter Science
Mechanical engineering
Electrical Engineering
Civil engineering
Chemical Engineering
Information Science engineering
Aeronautical & Aerospace Engineering
Biotechnology
Electrical and electronics engineering
Anutomations and robotics
Medical
MBBS
BDS
BAMS
BUMS
BHMS
BYNS
B.V.Sc & AH
B.sc
BSc IT (Information Technology)
BSc Computer Science
BSc Nursing
BSc Fashion Design
BSc Hospitality
BSc Animation
BSc Biotechnology
BSc Agriculture
BCA
Commerce
B.Com.
B.B.A.
B.M.S.
C.A.
B.B.S.
B.H.M
Arts
B.A.
B.F.A.
B.A + L.L.B.
B.J.M
B.F.D


Eye Dropper Eye Dropper | Color Picker Color Picker Pick color from web page Selected New Palette: default History is empty, try to pick some color first.


Study
Barrett 2013
Bergin 2011 —~—_
Cheng 2017
Zhang 2006
Zhao 2017
Overall (I-squared = 98.8%, p = 0.000)
P=0.680
NOTE: Weights are from random effects analysis
<>
WMD (95% Cl)
23.80 (18.59, 29.01)
—40.00 (-49.70, -30.30)
25.00 (20.73, 29.27)
—9.20 (—12.23, -6.17)
17.78 (14.01, 21.55)
3.83 (-14.39, 22.06)
Weight
20.06
19.28
20.17
20.28
20.22
100.00
T 49.7
49.7


{% extends<EMAIL_ADDRESS>%} {% form_theme post_form<EMAIL_ADDRESS>%} {% block content %}
{% include<EMAIL_ADDRESS>%}


Tucks has always been the perfect refreshment for moments of coming together. As the beer from Amsterdam, the city of bridges, we believe bridges are powerful things - and looks like we’re not the only ones.
Our classic, golden Tucks Beer, with its very distinctive and mildly bitter taste, has over the years become well known beyond the borders of its native country. The entrepreneurial spirit of the brand has created a host of brand extensions, such as: Tucks Lager, Tucks Premium Pilsener, Tucks Light, and Tucks Radler


You can mock your template variables by providing custom template data (JSON encoded map).
For example following template
Hi, {{ .name }}, your id is {{ .user_id }}
can be mocked by JSON below
{ "name": "Gopher", "user_id": "rambo226" }


About/FAQ
- What is Mindy's Bridal and Dress Shop?
- What do you sell
- How do I order?
- Do you do any additional services?
- What about orders over long distance (via phone and/or Internet)?
- How can pay for my order?
What is Mindy's Bridal and Dress Shop?
Mindy's Bridal and Dress Shop is privately owned and operated for over 15 years. We order from a variety of dress designers, and also special made dresses from your own designs and concepts.
What do you sell?
We sell bridal dresses, as well as bridesmaids, prom, special occasions, and flower girl dresses. We also sell bridal accessories, such as shoes, veils, and crowns.
How do I order dresses?
We take the customer's measurements and the down payment to take the orders.
Do you do any additional services?
We do in-store alterations and shoe dyes.
What about orders over long distance (via phone and/or Internet)?
We can also take your order by your measurements.
How can pay for my order?
We accept cash, check, and all major credit cards.


{% extends 'base.html' %} {% block content %} {% include "orders/_order_summary_short.html" with order=order %}
Please print for your records


Chapter 17 Other Model Selection Approaches
The way we have been doing model selection thus far is definitely not the only way. What other options are out there? Many - let’s consider a few.
This section also includes some miscellaneous R notes (making summary tables, and sources of inspiration for cool figures).
17.0.1 Rationale
Until now, we have focused on using information criteria for model selection, in order to get very familiar with one coherent framework for choosing variables across model types. But:
- In some fields, using hypothesis tests for variable selection is preferred
- For datasets that are large and/or models that are complex,
dredge()
can be a challenge (taking a very long time to run and perhaps timing out on the server) - Using hypothesis tests for selection is quite common, so we should know how it’s done!
17.0.2 Hypotheses
Basically, for each (fixed effect) variable in a model, we’d like to test:
\[H_0: \text{all } \beta\text{s for this variable are 0; it's not a good predictor}\] \[H_1: \text{ at least one } \beta\text{ is non-zero; it's a good predictor}\]
We want to test these hypotheses given that all the other predictors in the current full model are included. Because of this condition, and because there are multiple \(\beta\)s for categorical predictors with more than 2 categories, we can not generally just use the p-values from the model summary()
output.
Instead, we use Anova()
from the package car
. lm()
example:
##
## Call:
## lm(formula = Petal.Length ~ Petal.Width + Species + Sepal.Length,
## data = iris)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.765 -0.158 0.011 0.134 0.665
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -1.4596 0.2239 -6.52 1.1e-09 ***
## Petal.Width 0.5064 0.1153 4.39 2.1e-05 ***
## Speciesversicolor 1.7315 0.1276 13.57 < 2e-16 ***
## Speciesvirginica 2.3047 0.1984 11.62 < 2e-16 ***
## Sepal.Length 0.5587 0.0458 12.19 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.266 on 145 degrees of freedom
## Multiple R-squared: 0.978, Adjusted R-squared: 0.977
## F-statistic: 1.6e+03 on 4 and 145 DF, p-value: <2e-16
## Anova Table (Type II tests)
##
## Response: Petal.Length
## Sum Sq Df F value Pr(>F)
## Petal.Width 1.37 1 19.3 2.1e-05 ***
## Species 13.61 2 95.9 < 2e-16 ***
## Sepal.Length 10.55 1 148.6 < 2e-16 ***
## Residuals 10.29 145
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Notice that Anova()
reports one p-value for each predictor (excellent!). If the p-value is small, that gives evidence against \(H_0\), and we’d conclude we should keep the predictor in the model. Many people use \(\alpha = 0.05\) as the “dividing line” between “small” and “large” p-values and thus “statistically significant” and “non-significant” test results, but remember the p-value is a probability - there’s no magical difference between 0.049 and 0.051.
Warning: be careful with your capitalization! The R function anova()
does someting kind of similar to Anova()
but NOT the same and should be avoided – it does sequential rather than marginal tests.
17.1 Backward selection
How do we use p-value-based selection to arrive at a best model? There are many options and much controversy about different approaches; here I’ll suggest one. None of these methods are guaranteed to arrive at a model that is theoretically “best” in some specific way, but they do give a framework to guide decision-making and are computationally quick.
Strangely…functions are not widely available.
17.1.4 Stepwise IC-based selection
Another option may be to use backward stepwise selection (same algorithm as above), but using AIC or BIC as the criterion at each stage instead of p-values. If the IC value is better (by any amount) without a variable, it gets dropped. Variables are dropped one by one until no further IC improvement is possible.
This evaluates many fewer models than dredge
so should be much faster, but may not find the best of all possible models.
For example, for our model using AIC (note: this may or may not work for all model types.):
 Above, you see that if you were to remove age
, the AIC would only go up by about 1 unit. So according to our \(\Delta AIC \sim 3\) threshold, we would take age
out too.

##
## Call: glm(formula = homeless ~ sex + i1, family = binomial(link = "logit"),
## data = HELPrct)
##
## Coefficients:
## (Intercept) sexmale i1
## 0.9297 -0.3998 -0.0266
##
## Degrees of Freedom: 452 Total (i.e. Null); 450 Residual
## Null Deviance: 625
## Residual Deviance: 595 AIC: 601
17.2 Summary tables
You may want to compute and display summary tables for your projects. Here are a few examples of how to do it.
17.2.1 Mean (or sd, median, IQR, etc.) by groups
Compute the mean and sd (could use any other summary stats you want, though) for several quantitative variables, by groups.
Example: find mean and sd of iris flower Petal.Length
and Petal.Width
by Species
and display results in a pretty table. The dataset is called iris
.
Make a little one-row table for each variable being summarized, then stick them together.
require(knitr)
length_stats <- iris %>%
df_stats(Petal.Length ~ Species, mean, sd, long_names = FALSE) %>%
mutate(variable = 'Petal Length')
width_stats <- iris %>%
df_stats(Petal.Width ~ Species, mean, sd, long_names = FALSE) %>%
mutate(variable = 'Petal Width')
my_table <- bind_rows(length_stats, width_stats)
kable(my_table)
| Species | mean | sd | variable |
|---|---|---|---|
| setosa | 1.462 | 0.174 | Petal Length |
| versicolor | 4.260 | 0.470 | Petal Length |
| virginica | 5.552 | 0.552 | Petal Length |
| setosa | 0.246 | 0.105 | Petal Width |
| versicolor | 1.326 | 0.198 | Petal Width |
| virginica | 2.026 | 0.275 | Petal Width |
What if we want to round all table entries to 2 digits after the decimal?
| Species | mean | sd | variable |
|---|---|---|---|
| setosa | 1.46 | 0.17 | Petal Length |
| versicolor | 4.26 | 0.47 | Petal Length |
| virginica | 5.55 | 0.55 | Petal Length |
| setosa | 0.25 | 0.11 | Petal Width |
| versicolor | 1.33 | 0.20 | Petal Width |
| virginica | 2.03 | 0.27 | Petal Width |
What if we want the column order to be Variable, Species, mean, sd, and sort by Species and then Variable?
my_table <- my_table %>%
dplyr::select(variable, Species, mean, sd) %>%
arrange(Species, variable)
kable(my_table, digits = 2)
| variable | Species | mean | sd |
|---|---|---|---|
| Petal Length | setosa | 1.46 | 0.17 |
| Petal Width | setosa | 0.25 | 0.11 |
| Petal Length | versicolor | 4.26 | 0.47 |
| Petal Width | versicolor | 1.33 | 0.20 |
| Petal Length | virginica | 5.55 | 0.55 |
| Petal Width | virginica | 2.03 | 0.27 |
What if we actually want a column for mean length, sd length, etc. and one row per species?
require(tidyverse)
my_table2 <- my_table %>%
pivot_wider(names_from = variable,
values_from = c("mean", "sd"),
names_sep = ' ')
kable(my_table2, digits = 2, align = 'c')
| Species | mean Petal Length | mean Petal Width | sd Petal Length | sd Petal Width |
|---|---|---|---|---|
| setosa | 1.46 | 0.25 | 0.17 | 0.11 |
| versicolor | 4.26 | 1.33 | 0.47 | 0.20 |
| virginica | 5.55 | 2.03 | 0.55 | 0.27 |
17.2.2 Proportions in categories by groups
You may also want to make a table of proportion observations in each category by groups, potentially for many variables.
For just one variable, we can use tally:
tally(~substance | sex, data = HELPrct, format = 'prop') %>%
kable(caption = 'Proportion using each substance', digits = 2)
| female | male | |
|---|---|---|
| alcohol | 0.34 | 0.41 |
| cocaine | 0.38 | 0.32 |
| heroin | 0.28 | 0.27 |
For many variables we can use a loop. For example, we might want to know the proportion homeless and housed and proportion using each substance, both by sex, from the HELPrct
dataset. Above we were using the function knitr::kable()
to make tables, but we can use pander::pander()
too:
# select only variables needed for the table
# make the first variable the groups one
cat_data <- HELPrct %>% dplyr::select(sex, substance, homeless)
for (i in c(2:ncol(cat_data))){
tally(~cat_data[,i] | cat_data[,1], format = 'prop') %>%
pander::pander(caption = paste('Proportion in each ',
names(cat_data)[i]))
# can rename variables in cat_data if you want better captions
}
17.3 Figures
We’ve made a lot of figures in this class, and almost all have been kind of mediocre. To aim for awesome, here are a couple of great references for inspiration, ideas, and best practices:
- Fundamentals of Data Visualization by Claus Wilke. https://serialmentor.com/dataviz/
- https://infogram.com/blog/20-best-data-visualizations-of-2018/
- visualizingdata.com blog


|
Xilinx RoE Application
Xilinx Sample Software Modules API Documentation
|
Functions | |
| int | help_func (int argc, char **argv, char *resp) |
| int | ecpri_func (int argc, char **argv, char *resp) |
| int | ip_func (int argc, char **argv, char *resp) |
| int | stats_func (int argc, char **argv, char *resp) |
| int | version_func (int argc, char **argv, char *resp) |
| int | enable_func (int argc, char **argv, char *resp) |
| int | disable_func (int argc, char **argv, char *resp) |
| int | framing_func (int argc, char **argv, char *resp) |
| int | restart_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_func (int argc, char **argv, char *resp) |
| int | disable_help_func (int argc, char **argv, char *resp) |
| int | disable_framer_func (int argc, char **argv, char *resp) |
| int | disable_deframer_func (int argc, char **argv, char *resp) |
| int | ecpri_help_func (int argc, char **argv, char *resp) |
| int | ecpri_owdm_req_func (int argc, char **argv, char *resp) |
| int | ecpri_owdm_res_func (int argc, char **argv, char *resp) |
| int | ecpri_owdm_limit_func (int argc, char **argv, char *resp) |
| int | ecpri_rma_read_func (int argc, char **argv, char *resp) |
| int | ecpri_rma_write_func (int argc, char **argv, char *resp) |
| int | ecpri_test_mesg_func (int argc, char **argv, char *resp) |
| int | ecpri_rmr_req_func (int argc, char **argv, char *resp) |
| int | owdm_send_request (uint8_t type, char *dest_addr) |
| int | rma_send_request (int type, char *dest_addr, char *addr, char *length, char *values) |
| int | test_mesg_send_request (char *dest_addr, char *dest_port) |
| int | rma_get_response (int type, char *addr, int *length, char **values) |
| int | rmr_send_request (char *dest_addr) |
| int | rmr_get_response (char *addr, char **values) |
| int | enable_help_func (int argc, char **argv, char *resp) |
| int | enable_framer_func (int argc, char **argv, char *resp) |
| int | enable_deframer_func (int argc, char **argv, char *resp) |
| int | framing_help_func (int argc, char **argv, char *resp) |
| int | framing_set_fram_func (int argc, char **argv, char *resp) |
| int | framing_get_fram_func (int argc, char **argv, char *resp) |
| int | framing_set_defr_func (int argc, char **argv, char *resp) |
| int | framing_get_defr_func (int argc, char **argv, char *resp) |
| int | ip_help_func (int argc, char **argv, char *resp) |
| int | ip_peek_func (int argc, char **argv, char *resp) |
| int | ip_poke_func (int argc, char **argv, char *resp) |
| int | tokenise_input (char *in_str, char **cmd_tokens) |
| int | parse_command (int nohw, char *command, char *response) |
| int | RADIO_CTRL_CalulateBufStateLatency (int index, unsigned int *pBufStateLatency) |
| int | radio_ctrl_update_values (void) |
| int | radio_ctrl_help_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_radio_id_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_status_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_loopback_en_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_loopback_dis_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_gui_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_peek_func (int argc, char **argv, char *resp) |
| int | radio_ctrl_poke_func (int argc, char **argv, char *resp) |
| int | restart_help_func (int argc, char **argv, char *resp) |
| int | restart_xxv_func (int argc, char **argv, char *resp) |
| int | stats_help_func (int argc, char **argv, char *resp) |
| int | stats_hw_func (int argc, char **argv, char *resp) |
| int | stats_sw_func (int argc, char **argv, char *resp) |
| int | stats_user_func (int argc, char **argv, char *resp) |
| int | stats_ctrl_func (int argc, char **argv, char *resp) |
| int | stats_rate_func (int argc, char **argv, char *resp) |
| int | stats_all_func (int argc, char **argv, char *resp) |
| int | stats_all_gui_func (int argc, char **argv, char *resp) |
| int | stats_update_values (void) |
| #define DISABLE_DEFRAMER_STR "Disables the Deframer\n" |
DISABLE_USAGE_STR Help text for the disable module "deframer" option.
| #define DISABLE_FRAMER_STR "Disables the Framer\n" |
DISABLE_FRAMER_STR Help text for the disable module "framer" option.
| #define DISABLE_HELP_STR "generates this list of commands\n" |
DISABLE_HELP_STR Help text for the disable module "help" option.
| #define DISABLE_MAX_COMMANDS 4 |
DISABLE_MAX_COMMANDS Number of commands handled by the disable module.
| #define DISABLE_USAGE_STR "Usage \"disable <options>\", try \"disable help\" for command list\n" |
DISABLE_USAGE_STR Help text for the disable module.
Referenced by disable_func().
| #define ECPRI_HELP_STR "generates this list of commands\n" |
ECPRI_HELP_STR Help text for the ecpri module "help" option.
| #define ECPRI_MAX_COMMANDS 9 |
ECPRI_MAX_COMMANDS Number of commands handled by the ecpri module.
| #define ECPRI_OWDM_LIMIT_STR "ecpri owdm_limit - Sets the report threshold of the OWDM measurement (nsecs), or disables reporting if 0\n" |
ECPRI_OWDM_LIMIT_STR Help text for the ecpri module "owdm_limit" option.
Referenced by ecpri_owdm_limit_func().
| #define ECPRI_OWDM_REQ_STR "ecpri owdm_req <addr> <type> - Request an OWDM measurement to <addr> of type <to_remote|from_remote>\n" |
ECPRI_OWDM_REQ_STR Help text for the ecpri module "owdm_req" option.
Referenced by ecpri_owdm_req_func().
| #define ECPRI_OWDM_RES_STR "ecpri owdm_res - Returns the result of the last OWDM measurement, or \"Pending\" if still underway\n" |
ECPRI_OWDM_RES_STR Help text for the ecpri module "owdm_res" option.
Referenced by ecpri_owdm_res_func().
| #define ECPRI_RMA_READ_STR "ecpri rma_read <ip_addr> <mem_addr> <length> - Request a read of <length> bytes at <mem_addr> from <ip_addr>\n" |
ECPRI_RMA_READ_STR Help text for the ecpri module "rma_read" option.
Referenced by ecpri_rma_read_func().
| #define ECPRI_RMA_WRITE_STR "ecpri rma_write <ip_addr> <mem_addr> <length> \"<bytes 1..length>\" - Request a write of <length> <bytes> to <mem_addr> at <ip_addr>\n" |
ECPRI_RMA_WRITE_STR Help text for the ecpri module "rma_write" option.
Referenced by ecpri_rma_write_func().
| #define ECPRI_RMR_REQ_STR "ecpri rmr_req <ip_addr> - Request a remote reset of <ip_addr>\n" |
ECPRI_RMR_REQ_STR Help text for the ecpri module "rmr_req" option.
Referenced by ecpri_rmr_req_func().
| #define ECPRI_TEST_MESG_STR "ecpri test_mesg <ip_addr> <dest_port> - TESTING - Send a test message to <dest_port>\n" |
ECPRI_TEST_MESG_STR Help text for the ecpri module "test_mesg" option.
Referenced by ecpri_test_mesg_func().
| #define ECPRI_USAGE_STR "Usage \"ecpri [args]\", try \"ecpri help\" for command list\n" |
ECPRI_USAGE_STR Help text for the ecpri module.
Referenced by ecpri_func().
| #define ENABLE_DEFRAMER_STR "Enables the Deframer\n" |
ENABLE_DEFRAMER_STR Help text for the enable module "deframer" option.
| #define ENABLE_FRAMER_STR "Enables the Framer\n" |
ENABLE_FRAMER_STR Help text for the enable module "framer" option.
| #define ENABLE_HELP_STR "generates this list of commands\n" |
ENABLE_HELP_STR Help text for the enable module "help" option.
| #define ENABLE_MAX_COMMANDS 4 |
ENABLE_MAX_COMMANDS Number of commands handled by the enable module.
| #define ENABLE_USAGE_STR "Usage \"enable <options>\", try \"enable help\" for command list\n" |
ENABLE_USAGE_STR Help text for the enable module.
Referenced by enable_func().
| #define FRAMING_DEFRAMER_REG_NUM 14 |
FRAMING_DEFRAMER_REG_NUM Number of registers valid for deframer commands.
Referenced by framing_get_defr_func().
| #define FRAMING_DEFRAMER_WR_REG_NUM 2 |
FRAMING_DEFRAMER_WR_REG_NUM Number of registers valid for deframer writes.
Referenced by framing_set_defr_func().
| #define FRAMING_FRAMER_REG_NUM 6 |
FRAMING_FRAMER_REG_NUM Number of registers valid for framer commands.
Referenced by framing_get_fram_func(), and framing_set_fram_func().
| #define FRAMING_MAX_COMMANDS 6 |
FRAMING_MAX_COMMANDS Number of commands handled by the framing module.
| #define FRAMING_MAX_REG_NAME_LENGTH 16 |
FRAMING_MAX_REG_NAME_LENGTH Macimum length of a register name.
| #define FRAMING_REG_SIZE 4 |
FRAMING_REG_SIZE Sizee in bytes of framer/deframer registers.
Referenced by framing_get_defr_func(), framing_get_fram_func(), framing_set_defr_func(), and framing_set_fram_func().
| #define IP_HELP_STR "generates this list of ip subcommands\n" |
IP_HELP_STR Help text for the IP module "help" option.
| #define IP_MAX_COMMANDS 4 |
IP_MAX_COMMANDS Number of commands handled by the IP module.
| #define IP_PEEK_STR "ip peek <address>\n" |
IP_PEEK_STR Help text for the IP module "peek" option.
Referenced by ip_peek_func().
| #define IP_POKE_STR "ip poke <address> <value>\n" |
IP_POKE_STR Help text for the IP module "poke" option.
Referenced by ip_poke_func().
| #define IP_USAGE_STR "Usage is \"ip <command> <address> [value]\", try \"ip help\" for command list\n" |
IP_USAGE_STR Help text for the enable module.
Referenced by ip_func().
| #define MAX_NUMBER_OF_ANTENNAS 8 |
MAX_NUMBER_OF_ANTENNAS Number of antennae supported by the radio_ctrl module.
Referenced by RADIO_CTRL_CalulateBufStateLatency(), radio_ctrl_gui_func(), radio_ctrl_status_func(), and radio_ctrl_update_values().
| #define MAX_NUMBER_TOKENS 10 |
MAX_NUMBER_TOKENS Maximum number of tokens a command string may contain.
Referenced by parse_command(), and tokenise_input().
| #define RADIO_CTRL_GUI_STR "[DEV] all radio output for GUI\n" |
RADIO_CTRL_GUI_STR Help text for the radio_ctrl module "gui" option.
| #define RADIO_CTRL_HELP_STR "generates this list of commands\n" |
RADIO_CTRL_HELP_STR Help text for the radio_ctrl module "help" option.
| #define RADIO_CTRL_LOOPBACK_DIS_STR "Disables the loopback\n" |
RADIO_CTRL_LOOPBACK_DIS_STR Help text for the radio_ctrl module "loopback_dis" option.
| #define RADIO_CTRL_LOOPBACK_EN_STR "Enables the loopback\n" |
RADIO_CTRL_LOOPBACK_EN_STR Help text for the radio_ctrl module "loopback_en" option.
| #define RADIO_CTRL_PEEK_STR "radio_ctrl peek <address>\n" |
RADIO_CTRL_PEEK_STR Help text for the radio_ctrl module "peek" option.
Referenced by radio_ctrl_peek_func().
| #define RADIO_CTRL_POKE_STR "radio_ctrl poke <address> <value>\n" |
RADIO_CTRL_POKE_STR Help text for the radio_ctrl module "poke" option.
Referenced by radio_ctrl_poke_func().
| #define RADIO_CTRL_RADIO_ID_STR "[DEV] Returns the Radio ID tag\n" |
RADIO_CTRL_RADIO_ID_STR Help text for the radio_ctrl module "id" option.
| #define RADIO_CTRL_STATUS_STR "Returns the enable, error, status and loopback\n" |
RADIO_CTRL_STATUS_STR Help text for the radio_ctrl module "status" option.
| #define RADIO_CTRL_USAGE_STR "Usage \"radio_ctrl <options>\", try \"radio_ctrl help\" for command list\n" |
RADIO_CTRL_USAGE_STR Help text for the radio_ctrl module.
Referenced by radio_ctrl_func().
| #define RADIO_MAX_COMMANDS 9 |
RADIO_MAX_COMMANDS Number of commands handled by the radio_ctrl module.
| #define RESTART_HELP_STR "generates this list of commands\n" |
RESTART_HELP_STR Help text for the restart module "help" option.
| #define RESTART_MAX_COMMANDS 4 |
RESTART_MAX_COMMANDS Number of commands handled by the restart module.
| #define RESTART_USAGE_STR "Usage \"restart <options>\", try \"restart help\" for command list\n" |
RESTART_USAGE_STR Help text for the restart module.
Referenced by restart_func().
| #define RESTART_XXV_RESET_STR "XXV_Reset\n" |
RESTART_XXV_RESET_STR Help text for the restart module "xxv" option.
| #define RMA_READ 1 |
RMA_READ Flag to indicate an RMA read operation.
Referenced by ecpri_rma_read_func(), rma_get_response(), and rma_send_request().
| #define RMA_WRITE 2 |
RMA_WRITE Flag to indicate an RMA write operation.
Referenced by ecpri_rma_write_func().
| #define STATS_ALL_STR "Returns a summary of all stats\n" |
STATS_ALL_STR Help text for the stats module "all" option.
| #define STATS_CTRL_STR "Returns control packets stats\n" |
STATS_CTRL_STR Help text for the stats module "control" option.
| #define STATS_DEV_NUM_STR "[DEV] get the internal revision number of the stats device\n" |
STATS_DEV_NUM_STR Help text for the stats module "rev" option.
| #define STATS_GUI_NUM_STR "[DEV] all stats output for GUI\n" |
STATS_GUI_NUM_STR Help text for the stats module "gui" option.
| #define STATS_HELP_STR "generates this list of commands\n" |
STATS_HELP_STR Help text for the stats module "help" option.
| #define STATS_HW_STR "\"stats hw <address>\". Reads statistic at address\n" |
STATS_HW_STR Help text for the stats module "hw" option.
Referenced by stats_hw_func().
| #define STATS_MAX_COMMANDS 10 |
STATS_MAX_COMMANDS Number of commands handled by the stats module.
| #define STATS_RATE_STR "Returns packet rate stats\n" |
STATS_RATE_STR Help text for the stats module "rate" option.
| #define STATS_SW_STR "Software statistics not supported\n" |
STATS_SW_STR Help text for the stats module "sw" option.
Referenced by stats_sw_func().
| #define STATS_USAGE_STR "Usage \"stats <options>\", try \"stats help\" for command list\n" |
STATS_USAGE_STR Help text for the stats module.
Referenced by stats_func().
| #define STATS_USER_STR "Returns user data packets stats\n" |
STATS_USER_STR Help text for the stats module "user" option.
| #define XROE_MAX_COMMANDS 11 |
XROE_MAX_COMMANDS Number of commands handled at the top level.
Referenced by help_func().
| typedef struct antennas_status_struct antennas_status_struct |
antennas_status_struct Status values for an antenna array.
| typedef struct packets_struct packets_struct |
packets_struct Packet type count.
| typedef struct radio_ctrl_struct radio_ctrl_struct |
radio_ctrl_struct Control/status structure for the radio.
| typedef struct single_antenna_status_struct single_antenna_status_struct |
single_antenna_status_struct Status values for an antenna.
| typedef struct stats_struct stats_struct |
stats_struct Statistics structure.
| typedef struct total_packets_struct total_packets_struct |
total_packets_struct Totals packets count.
| int disable_deframer_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Disables the deframer by setting the restart value high.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMER_API_Deframer_Restart().
| int disable_framer_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Disables the framer by setting the restart value high.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMER_API_Framer_Restart().
| int disable_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References disable_cmds, and DISABLE_USAGE_STR.
| int disable_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for disable commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References disable_cmds.
| int ecpri_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ecpri_cmds, and ECPRI_USAGE_STR.
| int ecpri_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for eCPRI commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ecpri_cmds.
| int ecpri_owdm_limit_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Sets the reporting limit of the latest One-Way Delay Measurement.
This function calls the eCPRI protocol module to set the limit. If the specified threshold value is 0, reporting is disabled, otherwise all OWDM measurements over the limit (in nsecs) trigger an ILA write.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_OWDM_LIMIT_STR, and proto_ecpri_set_owdm_limit().
| int ecpri_owdm_req_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Requests a One-Way Delay Measurement to/from a remote node.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_OWDM_MSG_ACTION_REM_REQ_FOL_UP, ECPRI_OWDM_MSG_ACTION_REQ_FOL_UP, ECPRI_OWDM_REQ_STR, and owdm_send_request().
| int ecpri_owdm_res_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets the result of the latest One-Way Delay Measurement.
This function calls the eCPRI protocol module to retrieve the result.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_OWDM_RES_STR, proto_ecpri_get_owdm_result(), and TO_REMOTE.
| int ecpri_rma_read_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Requests a Remote Memory read from a remote node.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_RMA_READ_STR, rma_get_response(), RMA_READ, and rma_send_request().
| int ecpri_rma_write_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Requests a Remote Memory write to a remote node.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_RMA_WRITE_STR, rma_get_response(), rma_send_request(), and RMA_WRITE.
| int ecpri_rmr_req_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Requests a Remote Reset of a remote node.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_RMR_REQ_STR, rmr_get_response(), and rmr_send_request().
| int ecpri_test_mesg_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Sends a test message (generic data) to a remote node.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ECPRI_TEST_MESG_STR, and test_mesg_send_request().
| int enable_deframer_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Enables the deframer by setting the restart value low.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMER_API_Deframer_Restart().
| int enable_framer_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Enables the framer by setting the restart value low.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMER_API_Framer_Restart().
| int enable_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References enable_cmds, and ENABLE_USAGE_STR.
| int enable_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for enable commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References enable_cmds.
| int framing_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References framing_cmds.
| int framing_get_defr_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets field from framing register.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMING_DEFRAMER_REG_NUM, FRAMING_REG_SIZE, and IP_API_Read().
| int framing_get_fram_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets field from framing register.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMING_FRAMER_REG_NUM, FRAMING_REG_SIZE, and IP_API_Read().
| int framing_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for framing commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References framing_cmds.
| int framing_set_defr_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Sets field in deframing register.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMING_DEFRAMER_WR_REG_NUM, FRAMING_REG_SIZE, IP_API_Read(), and IP_API_Write().
| int framing_set_fram_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Sets field in framing register.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References FRAMING_FRAMER_REG_NUM, FRAMING_REG_SIZE, IP_API_Read(), and IP_API_Write().
| int help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for top-level commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References cmds, and XROE_MAX_COMMANDS.
| int ip_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ip_cmds, and IP_USAGE_STR.
| int ip_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for IP commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References ip_cmds.
| int ip_peek_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns the 32-bits at the given memory address.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References IP_API_Read(), and IP_PEEK_STR.
| int ip_poke_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Writes the 32-bits given to the given memory address.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References IP_API_Write(), and IP_POKE_STR.
| int owdm_send_request | ( | uint8_t | type, |
| char * | dest_addr | ||
| ) |
Sends a One-Way Delay Measurement request to a remote node.
This function calls the eCPRI protocol module to format and send the message.
| [in] | type | The direction, either TO_REMOTE or FROM_REMOTE. |
| [in] | dest_addr | IP address of remote node. |
References port_ip, and proto_ecpri_owdm_send_request().
Referenced by ecpri_owdm_req_func().
| int parse_command | ( | int | nohw, |
| char * | command, | ||
| char * | response | ||
| ) |
Parses the input command string and calls a lower-level handler (if found).
| [in] | nohw | Ignored. |
| [in] | command | String of space separated command tokens. |
| [out] | response | Pointer to string to place response text in. |
References cmds, MAX_NUMBER_TOKENS, tokenise_input(), XROE_QUIT_STR, and XROE_USAGE_STR.
Referenced by main().
| int RADIO_CTRL_CalulateBufStateLatency | ( | int | index, |
| unsigned int * | pBufStateLatency | ||
| ) |
Calculates the buffer state latency for a given antenna.
| [in] | index | Index of the antenna to calculate for. |
| [out] | pBufStateLatency | Pointer to place result in. |
References IP_API_Read_Register(), and MAX_NUMBER_OF_ANTENNAS.
Referenced by radio_ctrl_update_values().
| int radio_ctrl_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References radio_ctrl_cmds, and RADIO_CTRL_USAGE_STR.
| int radio_ctrl_gui_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets antennae and radio status in JSON format into resp for GUI display.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References MAX_NUMBER_OF_ANTENNAS, and radio_ctrl_update_values().
| int radio_ctrl_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for radio_ctrl commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References radio_ctrl_cmds.
| int radio_ctrl_loopback_dis_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Disables loopback functionality on the radio module.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References RADIO_CTRL_API_Write_Register().
| int radio_ctrl_loopback_en_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Enables loopback functionality on the radio module.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References RADIO_CTRL_API_Write_Register().
| int radio_ctrl_peek_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns the 32-bits at the given memory address.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References RADIO_CTRL_API_Read(), and RADIO_CTRL_PEEK_STR.
| int radio_ctrl_poke_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Writes the 32-bits given to the given memory address.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References RADIO_CTRL_API_Write(), and RADIO_CTRL_POKE_STR.
| int radio_ctrl_radio_id_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets radio ID into resp.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References RADIO_CTRL_API_Read_Register().
| int radio_ctrl_status_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Gets antennae and radio status in text format into resp.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References MAX_NUMBER_OF_ANTENNAS, and radio_ctrl_update_values().
| int radio_ctrl_update_values | ( | void | ) |
Updates the internal radio and antennae status variables.
References IP_API_Read_Register(), MAX_NUMBER_OF_ANTENNAS, RADIO_CTRL_API_Read_Register(), and RADIO_CTRL_CalulateBufStateLatency().
Referenced by radio_ctrl_gui_func(), and radio_ctrl_status_func().
| int restart_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References restart_cmds, and RESTART_USAGE_STR.
| int restart_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for restart commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References restart_cmds.
| int restart_xxv_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Restarts the XXV Ethernet interface.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References XXV_API_Reset().
| int rma_get_response | ( | int | type, |
| char * | addr, | ||
| int * | length, | ||
| char ** | values | ||
| ) |
Reads a Remote Memory Access response from a remote node.
This function calls the eCPRI protocol module to receive the message.
| [in] | type | Can be read or write. |
| [in] | addr | Ignored. |
| [in] | length | The number of bytes read/written. |
| [in] | values | Array of byte values read, or NULL on write. |
References ECPRI_RMA_MSG_READ, ECPRI_RMA_MSG_WRITE, proto_ecpri_rma_get_response(), and RMA_READ.
Referenced by ecpri_rma_read_func(), and ecpri_rma_write_func().
| int rma_send_request | ( | int | type, |
| char * | dest_addr, | ||
| char * | addr, | ||
| char * | length, | ||
| char * | values | ||
| ) |
Sends a Remote Memory Access request to a remote node.
This function calls the eCPRI protocol module to format and send the message.
| [in] | type | Can be read or write. |
| [in] | dest_addr | IP address of remote node. |
| [in] | addr | The remote memory address to access. |
| [in] | length | The number of bytes to access. |
| [in] | values | Array of byte values to write, or NULL on read. |
References ECPRI_RMA_MSG_READ, ECPRI_RMA_MSG_WRITE, port_ip, proto_ecpri_rma_send_request(), and RMA_READ.
Referenced by ecpri_rma_read_func(), and ecpri_rma_write_func().
| int rmr_get_response | ( | char * | addr, |
| char ** | values | ||
| ) |
Gets a Remote Reset response from a remote node.
This function calls the eCPRI protocol module to receive the message.
| [in] | addr | Address of the remote node to reset. |
| [out] | values | Pointer to string to place the response into. |
References proto_ecpri_rmr_get_response().
Referenced by ecpri_rmr_req_func().
| int rmr_send_request | ( | char * | dest_addr | ) |
Sends a Remote Reset request to a remote node.
This function calls the eCPRI protocol module to format and send the message.
| [in] | dest_addr | Address of the remote node to reset. |
References port_ip, and proto_ecpri_rmr_send_request().
Referenced by ecpri_rmr_req_func().
| int stats_all_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads all statistics.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_update_values().
| int stats_all_gui_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads all statistics and returns them in JSON format for GUI display.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_update_values().
| int stats_ctrl_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads control packet statistics.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_update_values().
| int stats_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Parses the input command string and calls a handler function (if found).
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_cmds, and STATS_USAGE_STR.
| int stats_help_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Returns help strings for stats commands.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_cmds.
| int stats_hw_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads a statistics hardware register and returns the value.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References STATS_API_Read_Register(), and STATS_HW_STR.
| int stats_rate_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads data and control packet rates.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_update_values().
| int stats_sw_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Dummy function, returns the help text.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References STATS_SW_STR.
| int stats_update_values | ( | void | ) |
Updates the internal status variables.
References IP_API_Read_Register(), and STATS_API_Read_Register().
Referenced by stats_all_func(), stats_all_gui_func(), stats_ctrl_func(), stats_rate_func(), and stats_user_func().
| int stats_user_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Reads user packet statistics.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References stats_update_values().
| int test_mesg_send_request | ( | char * | dest_addr, |
| char * | dest_port | ||
| ) |
Sends a test message (generic data) to a remote node.
This function calls the eCPRI protocol module to format and send the message.
| [in] | dest_addr | IP address of remote node. |
| [in] | dest_port | IP port address of the remote node. |
References proto_ecpri_test_mesg_send().
Referenced by ecpri_test_mesg_func().
| int tokenise_input | ( | char * | in_str, |
| char ** | cmd_tokens | ||
| ) |
Splits the input string into an array of command tokens.
| [in] | in_str | Input string of space separated tokens. |
| [out] | cmd_tokens | Pointer to string array to place tokens in. |
References MAX_NUMBER_TOKENS.
Referenced by parse_command().
| int version_func | ( | int | argc, |
| char ** | argv, | ||
| char * | resp | ||
| ) |
Dummy function for un-implemented "version" command.
| [in] | argc | Number of string arguments. |
| [in] | argv | Array of strings containg arguments. |
| [out] | resp | Pointer to string to place response text in. |
References XROE_VER_STR.
| commands_t cmds[XROE_MAX_COMMANDS] |
cmds The top-level commands handled by the command parser.
Referenced by help_func(), and parse_command().
| struct register_access deframer_regs[FRAMING_DEFRAMER_REG_NUM] |
deframer_regs Register fields in the deframer.
| commands_t disable_cmds[DISABLE_MAX_COMMANDS] |
disable_cmds The commands handled by the disable module.
Referenced by disable_func(), and disable_help_func().
| commands_t ecpri_cmds[ECPRI_MAX_COMMANDS] |
ecpri_cmds The commands handled by the disable module.
Referenced by ecpri_func(), and ecpri_help_func().
| commands_t enable_cmds[ENABLE_MAX_COMMANDS] |
enable_cmds The commands handled by the enable module.
Referenced by enable_func(), and enable_help_func().
| struct register_access framer_regs[FRAMING_FRAMER_REG_NUM] |
framer_regs Register fields in the framer.
| commands_t framing_cmds[FRAMING_MAX_COMMANDS] |
framing_cmds The commands handled by the framing module.
Referenced by framing_func(), and framing_help_func().
| commands_t ip_cmds[IP_MAX_COMMANDS] |
ip_cmds The commands handled by the IP module.
Referenced by ip_func(), and ip_help_func().
| commands_t radio_ctrl_cmds[RADIO_MAX_COMMANDS] |
radio_ctrl_cmds The commands handled by the radio_ctrl module.
Referenced by radio_ctrl_func(), and radio_ctrl_help_func().
| commands_t restart_cmds[RESTART_MAX_COMMANDS] |
restart_cmds The commands handled by the restart module.
Referenced by restart_func(), and restart_help_func().
| commands_t stats_cmds[STATS_MAX_COMMANDS] |
stats_cmds The commands handled by the stats module.
Referenced by stats_func(), and stats_help_func().


Bjelovar (Hungarian: Belovár, German: Bellowar, Kajkavian: Belovar) is a city in central Croatia. It is the administrative centre of Bjelovar-Bilogora County. At the 2011 census, there were 40,276 inhabitants, of whom 91.25% were Croats


Getting Started¶
This guide will help you to quickly get started with a single node instance of IUDX and start publishing data from a device.
Prerequisites¶
Install Docker
sudo apt install docker.io
Add user to the docker group
sudo usermod -a -G docker $USER
Logout and log back in.
Run
ifconfig
and note down the name of the interface you’re connected to.In the above example the interface name is
enp9s0
Now run:
sudo nmcli device show enp9s0
Replace
enp9s0
with your respective interface name. The output should be something like thisNote down the DNS from
IP4.DNS
sectionAdd DNS obtained in the previous step to the file
/etc/docker/daemon.json
If the file does not exist, create it. The entry should look something like this{"dns": ["<IP_ADDRESS>", "<IP_ADDRESS>"]}
Add the obtained DNS to the above JSON array. So the final entry should look something like this:
{"dns": ["<IP_ADDRESS>","<IP_ADDRESS>","<IP_ADDRESS>"]}
Of course, the DNS will change according to your network.
Add DNS in
/etc/default/docker
file as follows:DOCKER_OPTS="--dns <IP_ADDRESS> --dns <IP_ADDRESS> --dns <IP_ADDRESS>"
Restart Docker
sudo service docker restart
IUDX Installation¶
Clone IUDX git repo
git clone https://github.com/rbccps-iisc/corinthian.git
Install IUDX
cd docker/ ./install
Registering your first device¶
Once IUDX has been installed you can now start registering devices with it. Let’s create a simple test device for the sake of illustration
Get the randomly generated admin apikey for the current installation by:
cat vars/admin.passwd
Register a device by using this apikey as follows:
./tests/register.sh admin <admin_apikey> <device_name> E.g. ./tests/register.sh admin RFS3zYWpwfqse1eHFyZUydY4q0rI9XzN streetlight
This will give you the details of the registration
{ "id": "admin/streetlight", "apikey": "CqLXHzTTXOAWmPRaoLpNRKCd4MfAaHCJ" }
Publishing from your device¶
You can now publish data from this device using:
./tests/publish.sh admin/streetlight CqLXHzTTXOAWmPRaoLpNRKCd4MfAaHCJ admin/streetlight routing_key protected test_message
This will publish
test_message
to the exchangeadmin/streetlight.protected
with routing key asrouting_key
and message type asprotected
That’s it! You can similarly register more devices and apps with the IUDX middleware.