--- layout: post title: "Functors" description: "A functor is a common abstraction. While typically associated with functional programming, functors exist in C# as well." date: 2018-03-22 16:57 UTC tags: [Software Design] image: "/content/binary/functor-diagram.png" image_alt: "Functor diagram." --- {% include JB/setup %}
{{ page.description }}
This article series is part of a larger series of articles about functors, applicatives, and other mappable containers.
Programming is about abstraction, since you can't manipulate individual sub-atomic particles on your circuit boards. Some abstractions are well-known because they're rooted in mathematics. Particularly, category theory has proven to be fertile ground for functional programming. Some of the concepts from category theory apply to object-oriented programming as well; all you need is generics, which is a feature of both C# and Java.
In previous articles, you got an introduction to the specific Test Data Builder and Test Data Generator functors. Functors are more common than you may realise, although in programming, we usually work with a subset of functors called endofunctors. In daily speak, however, we just call them functors.
In the next series of articles, you'll see plenty of examples of functors, with code examples in both C#, F#, and Haskell. These articles are mostly aimed at object-oriented programmers curious about the concept.
IEnumerable<T>
. Since the combination of IEnumerable<T>
and query syntax is already well-described, I'm not going to cover it explicitly here.
If you understand how LINQ, IEnumerable<T>
, and C# query syntax works, however, all other functors should feel intuitive. That's the power of abstractions.
The purpose of this article isn't to give you a comprehensive introduction to the category theory of functors. Rather, the purpose is to give you an opportunity to learn how it translates to object-oriented code like C#. For a great introduction to functors, see Bartosz Milewski's explanation with illustrations.
In short, a functor is a mapping between two categories. A functor maps not only objects, but also functions (called morphisms) between objects. For instance, a functor F may be a mapping between the categories C and D:
Not only does F map a from C to F a in D (and likewise for b), it also maps the function f to F f. Functors preserve the structure between objects. You'll often hear the phrase that a functor is a structure-preserving map. One example of this regards lists. You can translate a List<int>
to a List<string>
, but the translation preserves the structure. This means that the resulting object is also a list, and the order of values within the lists doesn't change.
In category theory, categories are often named C, D, and so on, but an example of a category could be IEnumerable<T>
. If you have a function that translates integers to strings, the source object (that's what it's called, but it's not the same as an OOP object) could be IEnumerable<int>
, and the destination object could be IEnumerable<string>
. A functor, then, represents the ability to go from IEnumerable<int>
to IEnumerable<string>
, and since the Select method gives you that ability, IEnumerable<T>.Select
is a functor. In this case, you sort of 'stay within' the category of IEnumerable<T>
, only you change the generic type argument, so this functor is really an endofunctor (the endo prefix is from Greek, meaning within).
As a rule of thumb, if you have a type with a generic type argument, it's a candidate to be a functor. Such a type is not always a functor, because it also depends on where the generic type argument appears, and some other rules.
Fundamentally, you must be able to implement a method for your generic type that looks like this:
public Functor<TResult> Select<TResult>(Func<T, TResult> selector)
Here, I've defined the Select
method as an instance method on a class called Functor<T>
, but often, as is the case with IEnumerable<T>
, the method is instead implemented as an extension method. You don't have to name it Select
, but doing so enables query syntax in C#:
var dest = from x in source select x.ToString();
Here, source
is a Functor<int>
object.
If you don't name the method Select
, it could still be a functor, but then query syntax wouldn't work. Instead, normal method-call syntax would be your only option. This is, however, a specific C# language feature. F#, for example, has no particular built-in awareness of functors, although most libraries name the central function map
. In Haskell, Functor
is a typeclass that defines a function called fmap
.
The common trait is that there's an input value (Functor<T>
in the above C# code snippet), which, when combined with a mapping function (Func<T, TResult>
), returns an output value of the same generic type, but with a different generic type argument (Functor<TResult>
).
Defining a Select
method isn't enough. The method must also obey the so-called functor laws. These are quite intuitive laws that govern that a functor behaves correctly.
The first law is that mapping the identity function returns the functor unchanged. The identity function is a function that returns all input unchanged. (It's called the identity function because it's the identity for the endomorphism monoid.) In F# and Haskell, this is simply a built-in function called id
.
In C#, you can write a demonstration of the law as a unit test:
[Theory] [InlineData(-101)] [InlineData(-1)] [InlineData(0)] [InlineData(1)] [InlineData(42)] [InlineData(1337)] public void FunctorObeysFirstFunctorLaw(int value) { Func<int, int> id = x => x; var sut = new Functor<int>(value); Assert.Equal(sut, sut.Select(id)); }
While this doesn't prove that the first law holds for all values and all generic type arguments, it illustrates what's going on.
Since C# doesn't have a built-in identity function, the test creates a specialised identity function for integers, and calls it id
. It simply returns all input values unchanged. Since id
doesn't change the value, then Select(id)
shouldn't change the functor, either. There's nothing more to the first law than this.
The second law states that if you have two functions, f
and g
, then mapping over one after the other should be the same as mapping over the composition of f
and g
. In C#, you can illustrate it like this:
[Theory] [InlineData(-101)] [InlineData(-1)] [InlineData(0)] [InlineData(1)] [InlineData(42)] [InlineData(1337)] public void FunctorObeysSecondFunctorLaw(int value) { Func<int, string> g = i => i.ToString(); Func<string, string> f = s => new string(s.Reverse().ToArray()); var sut = new Functor<int>(value); Assert.Equal(sut.Select(g).Select(f), sut.Select(i => f(g(i)))); }
Here, g
is a function that translates an int
to a string
, and f
reverses a string. Since g
returns string
, you can compose it with f
, which takes string
as input.
As the assertion points out, it shouldn't matter if you call Select
piecemeal, first with g
and then with f
, or if you call Select
with the composed function f(g(i))
.
This is not a monad tutorial; it's a functor tutorial. Functors are commonplace, so it's worth keeping an eye out for them. If you already understand how LINQ (or similar concepts in Java) work, then functors should be intuitive, because they are all based on the same underlying maths.
While this article is an overview article, it's also a part of a larger series of articles that explore what object-oriented programmers can learn from category theory.
Next: The Maybe functor.
Comments


|
How To Dissect a TopCoder Problem Statement
By antimatter
TopCoder Member
How many times has this happened to you: you register for the SRM, go into your assigned room when the system tells you to, and when the match starts, you open the 250... and find it incomprehensible.
Maybe it's never happened to you. You may be lucky, or you may already be extremely skilled. However, many experienced competitors (yes, reds too) might end up just staring at a problem for a long time. This is a pretty serious issue. How can you solve the problem if you have no idea what it's asking you to do?
Fortunately, TopCoder problem statements are formatted in a very specific way.
Knowing your way around the various pieces will go a long way towards helping you understand what the problem is saying.
The Parts of a Problem Statement
Let's look at the composition of a typical TopCoder problem statement. First off is the introduction. Usually, a problem will be led off with a high-level description of a situation. This description may tie into real-life ideas and topics or it may just be a completely fictional story, serving only as some sort of context. For many problems the back-story itself is not particularly important in understanding the actual problem at hand.
Next comes the definition section. It gives you the skeleton of the solution you need to write: class name, method name, arguments, and return type, followed by the complete method signature. At minimum, you will need to declare a class with the given name, containing a method which conforms to the given method signature. The syntax given will always be correct for your chosen language.
Sometimes notes follow the method definition. They tend to be important reminders of things that you should pay attention to but might have missed, or they can also be things that are helpful background knowledge that you might not know beforehand. If the notes section appears, you should make sure to read it - usually the information contained within is extremely important.
The constraints section is arguably the most important. It lists specific constraints on the input variables. This lets you know crucial details such as how much memory to allocate or how efficient your algorithm will have to be.
Finally, a set of examples is provided. These give sample inputs against which you can test your program. The given parameters will be in the correct order, followed by the expected return value and, optionally, an explanation of the test case.
-
Introduction
The problem statement usually begins by motivating the problem. It gives a situation or context for the problem, before diving into the gory details. This is usually irrelevant to solving the problem, so ignore it if necessary. In some cases, the motivation can cause serious ambiguities if it is treated as binding - see MatchMaking (SRM 203 Div I Easy / Div II Medium). Also note that for some simple problems, the initial context may be left out.
The ordering of the rest of this section varies greatly from problem to problem, based on the writing style of the problem author.
There will be a description of what you need to do, in high-level terms. Take, for example, UserName (SRM 203, Div 2 easy). What the problem is asking for you to do is to find the first variant of a given username that is not already taken. Note that the problem has not yet said anything about variable names or types, or input formats.
There will also be a low-level description of the input. At the bare minimum, the types and variable names of the inputs will be given to you, as well as what they correspond to and what they mean. Sometimes much more information about input formats will be given; this typically occurs in more complicated problems.
Sometimes, even more detailed background information needs to be provided. That is also typically given here, or sometimes in the Notes section.
-
The Definition
This is a very barebones description of what TopCoder wants you to submit. It gives the class name, the method name to create inside that class, the parameters it should take, the return value, and a method signature. As mentioned before, the basic form of a submitted solution is to create a class containing a method with the required signature. Make sure that the class is declared public if not using C++, and make sure to declare the method public also.
-
Notes and Constraints
Notes don't always appear. If they do, READ THEM! Typically they will highlight issues that may have come up during testing, or they may provide background information that you may not have known beforehand. The constraints section gives a list of constraints on the input variables. These include constraints on sizes of strings and arrays, or allowed characters, or values of numbers. These will be checked automatically, so there is no need to worry about writing code to check for these cases.
Be careful of the constraints. Sometimes they may rule out certain algorithms, or make it possible for simpler but less efficient algorithms to run in time. There can be a very big difference between an input of 50 numbers and an input of 5, both in terms of solutions that will end up passing, and in terms of ease of coding.
-
Examples
These are a list of sample test cases to test your program against. It gives the inputs (in the correct order) and then the expected return value, and sometimes an annotation below, to explain the case further if necessary.
It goes without saying that you should test your code against all of the examples, at the very least. There may be tricky cases, large cases, or corner cases that you have not considered when writing the solution; fixing issues before you submit is infinitely preferable to having your solution challenged or having it fail during system testing.
The examples are not always comprehensive! Be aware of this. For some problems, passing the examples is almost the same as passing every test case.
For others, however, they may intentionally (or not) leave out some test case that you should be aware of. If you are not completely sure that your code is correct, test extensively, and try to come up with your own test cases as well. You may even be able to use them in the challenge phase.
Solving a problem
Now we'll walk through a simple problem and dissect it, bit by bit.
Have a look at BettingMoney, the SRM 191 Division 2 Easy. First we identify the parts of this problem. In the statement itself, we first have the situation behind the problem - gambling. Then we have a little bit of background information about the betting itself. Then, we have a description of the input - data types, variable names, and what they represent. After this we have the task: to determine what the net gain is for the day and return the amount in cents.
Also note the two explanatory paragraphs at the end; the first provides an example of the input format and types, and the second gives a completely worked example, which should be extremely helpful to your understanding.
The definition section is uninteresting, but it is there for completeness' sake.
The notes for this problem are fairly comprehensive. In terms of background information, you might not know that there are 100 cents in a dollar. And in terms of clarification, there is explicit confirmation that the return value may in fact be negative, and that the margin of victory (the variable finalResult) is all that matters when deciding which payoff to make.
The constraints are fairly straightforward. The input arrays will contain the same number of elements, between 1 and 50, inclusive. (50 is a long-standing TopCoder tradition for input sizes). finalResult will be between 0 and that same size minus one (which means, if you give it a little thought, that someone will win their bet). Each element of each array will be between 0 and 5000, inclusive. This is most likely to make sure that integer arithmetic will do the job just fine.
Finally, there's the examples section. Often, the problem statement section will contain an annotated example case, which will become example case 0. Then there are a couple of other example cases, some with explanation and some without. Also note that one of the examples tests for negative return values, to supplement the notes.
A More Complicated Example
Now have a look at Poetry, the SRM 170 Div 2 Hard. In this case, you may not be able to actually solve this in the time allotted. That's ok - the emphasis should first be on understanding what the problem says, even if you can't code it in time.
The first section tells you immediately what you want to do - you'll be given a poem, and you will have to determine what its rhyme scheme is. The rest of the section clarifies what this actually means, in bottom-up fashion (from simpler concepts to more complicated ones). It defines what a legal word is and how to extract words from a poem, and then it defines what it means when two words rhyme - that their ending patterns are equal. The concept of ending pattern is then defined. After all this, we find out what it means to have two lines of the poem rhyme: their last words have to rhyme. Finally, (whew!) we are told how to actually construct the rhyme scheme and in what format to return it.
This is a problem where a lot of terms need to be defined to get to the heart of things, and so all the definitions deserve at least a couple of read-throughs, especially if you're not sure how they all fit together.
The next section is the problem definition section, just for reference. Then there is a single note that clarifies a point that may have been overlooked when it was stated in the problem statement itself: that blank lines will be labeled with a corresponding space in the rhyme scheme.
The constraints are fairly standard for TopCoder problems: there will be between 1 and 50 lines in the poem, and each line will contain between 0 and 50 characters. The only allowable characters in the poem will be spaces and letters, and there will be only legal words in poem.
Finally, there are a number of examples. Usually, problems which are trickier or which have more complex problem statements will have more examples, to clarify at least some of the finer points of the problem statement. Again, this doesn't mean that passing the example cases given is equivalent to having a completely correct solution, but there is a higher chance that you can catch any bugs or trivial mistakes if there are more examples that you know the answers to.
Try it Yourself
Listed below are a number of additional problems, grouped roughly by difficulty of comprehension. Try them for yourself in the TopCoder Arena Practice Rooms. Even if you can't solve them, at least work on figuring out what the problem wants by breaking it down in this manner.
Mentioned in this writeup:
SRM 203 Div 2 Easy - UserName
SRM 191 Div 2 Easy - BettingMoney
SRM 203 Div 1 Easy - MatchMaking
SRM 170 Div 2 Hard - Poetry
Similar tasks:
SRM 146 Div 2 Easy - Yahtzee
SRM 200 Div 2 Easy - NoOrderOfOperations
SRM 185 Div 2 Easy - PassingGrade
SRM 155 Div 2 Easy - Quipu
SRM 147 Div 2 Easy - CCipher
SRM 208 Div 1 Easy - TallPeople
SRM 173 Div 1 Easy - WordForm
SRM 162 Div 1 Easy - PaperFold
More challenging tasks:
SRM 197 Div 2 Hard - QuickSums
SRM 158 Div 1 Hard - Jumper
SRM 170 Div 1 Easy - RecurrenceRelation
SRM 177 Div 1 Easy - TickTick
SRM 169 Div 2 Hard - Twain
SRM 155 Div 1 Med - QuipuReader
|
|
|


-->
We provide variety of skill set classes for kids and Students. Our primary aim is to
give genuine learning experience to our youngsters. There's a high demand of skillful people, whether it is Coding, Robotics , Digital Marketing, Entrepreneurship or Trading.
We want India to be the next super power and for that our future must be competent
All these courses are well planned from Beginner Level till the Advance Level.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed doer ket eismod tempor incididunt ut labore et dolores


?6
7
8
9
10
'
XT
v
.
,
The
New
York
copyright
reserved
botanical
Garden
SMITHSONIAN
INSTITUTION
From
THE
UNITED
STATES
NATIONAL
HERBARIUM
PLANTS
OF
CUBA
SAPOTACEAE
NO:
6294
Sideroxylon
foetidissimum
Jacq.
LOCALITY:
Provincia
de
Matanzas:
ca.
3
km
from
‘Caleta
Buena’
to
"Bermeja’
HABITAT:
Semi-deciduous,
esclerophytic
to
mesic
forest.
DESCRIPTION:
Tree,
15
m
tall,
with
scanty
milky
sap;
fruit
ellipsoid,
orange-yellow;
seed
brown-black.
COLL:
P.
Acevedo-Rdgz.,
M.
Fernandez,
K.
de
Queiroz,
R.
Oviedo,
&
L.
Rodriguez
DATE:
9
Apr.
1994
Collected
under
the
auspices
of
the
Smithsonian
Institution
NEW
YORK
BOTANICAL
GARDEN
o-
443515
BOTANICAL
v®ARDE^>
01443515
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


Chapter 2
2. INTRODUCTION TO GUI BASED OPERATING SYSTEM
2.4 Operating system simple setting
2.4.1 Changing System Date and Time
Steps for changing Date and Time:
1. Open Control panel.
2. Double click on Date and Time.
3. Click on Change Date and Time.
4. Set the Date and Time as you desire.
5. Then click OK.


Glasgow Warriors 13-7 Sale Sharks: DTH van der Merwe try sets up Champions Cup win
DTH van der Merwe's try gave Glasgow a winning start to their Champions Cup campaign as they edged out Sale 13-7.
Dave Rennie's team dominated the first half but should have taken more than a solitary try and eight points from the boot of Adam Hastings.
Having badly lacked tempo during the opening 40 minutes, Sale boss Steve Diamond rang the changes early after the interval and got a response as Coenie Oosthuizen crossed over.
Warriors failed to build on their tally but were relieved as they held on for victory.
Sale were missing World Cup winner Faf de Klerk as well as England's beaten finalists Tom Curry and Mark Wilson.
The hosts on the other hand were almost at full strength, with Scotland lock Jonny Gray the only one of the 19 men they sent to Japan yet to return to club duty and it showed as the Warriors bossed the early exchanges.
Hastings nudged Warriors ahead on the quarter-hour mark but it was a shaky kicking display from the Scotland fly-half as he skewed past the poles with another penalty after Jake Cooper-Woolley was penalised for handling in the maul.
Glasgow were well on top but a three-point advantage was little reward for their efforts, so Hastings made certain to split the posts when he was handed another chance after an illegal Byron McGuigan tackle on 27 minutes.
Glasgow broke through for the opening try four minutes before the break. Fraser Brown's lineout was plucked by Ryan Wilson before Sam Johnson turned up the speed levels as he flicked a George Horne pass round the corner for Van der Merwe and there was no catching the Canada international as he darted for the corner.
Sale were hoping the change of ends would produce a change in fortunes.
They got a stroke of luck when a cruel bounce denied Van der Merwe a second try as he chased after a clever Hastings kick to the corner.
And they finally built up the momentum that got them on the scoreboard on 63 minutes.
Diamond waited just a couple of minutes after half-time before replacing his front-row and it was substitute prop Oosthuizen who squeezed over the line for the score that drew Sharks back within six points of Rennie's team.
The Sale replacements had made a huge difference, giving them the upper hand at the set-piece and Glasgow's nerves appeared to be jangling as the game moved into the final 10 minutes.
Sale continued to press for a late winner with the clock gone red, but a Robert du Preez knock-on saw a relieved Glasgow hold on for victory.


Each function
Syntax:
def function_name(argument1, argument2):
statement_0
statement_1
...
return
Similar to variables names, function names:
def add_them(x, y, z):
print('total is:', x + y + z)
add_them(1, 2) # error! must provide 3 arguments when calling the function
add_them(1, 2, 10) # Ok, print out total is: 13
# x is 1, y is 2, z is 10
add_them(2, 1, 10) # x is 2, y is 1, z is 10, print out total is: 13
def add_them(x, y, z=10):
print('total is:', x + y + z)
add_them(1, 2) # No error, prints out: total is: 13
add_them(1, 2, 3) # No error, prints out: total is: 6
add_them(1, 2, 3, 4)# Error!
def add_them(*arguments):
# arguments is a named tuple of all the arguments
# use len(arguments) to decide how many arguments are passed
total = 0
for n in arguments:
total = total + n
print('Total is:', total)
add_them(1)
add_them(1, 2, 3)
add_them(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
def add_them(x, y):
print('result is:', x + y*2)
add_them(1, 2) # result is 5
add_them(x=2, y = 3) # result is 8
add_them(y=3, x=2) # result is 8
def name(**keyword_arguments):
# keyword_arguments is a dictionary of arguments
for k in keyword_arguments:
print(k, '->', keyword_arguments[k], type(keyword_arguments[k]))
name(x=1, y=2.0, z='three')
args = {'x': 1, 'y': 2.0, 'z': 'three'}
name(args) # ERROR !
name(arguments=args) # Ok, print keyword_arguments becomes 'arguments': dictionary
def add_them(x, y, z):
print('result is:', x + y + z*2)
add_them(1, 2, 3) # Ok
add_them(1, 2, z=3) # Ok
add_them(1, y=2, z=3) # Ok
add_them(1, z=3, y=2) # Ok
add_them(x=1, y=2, z=3) # Ok
add_them(y=2, z=3, 1) # ERROR
Example: Write a function to print out a greeting
def print_greeting(name):
print('This is my fancy greeting:')
print('---------------------------------')
print('>>> Hello,', name)
print('---------------------------------')
print_greeting('Nancy')
x = print_greeting('Tom')
print(x)
n = input('Enter your name: ')
print_greeting(n)
Example: Write a function to check if a user's age is valid for high school a student's age
def is_valid_highschool_age(age):
if age > 18 or age < 14:
return False
return True
is_valid_highschool_age(10) # nothing happens, although function run
is_valid = is_valid_highschool_age(17)
print(is_valid) # print True
if is_valid_highschool_age(21):
print('You are in highschool')
else:
print('You are not in highschool') # This one gets printed
Example: Write a function to calculate the sum of all numbers between 1 and the specified number
def calculate_total(maxnumber):
if maxnumber <= 1:
return None # Indicate an error
total = 0
for n in range(1, maxnumber):
total = total + n
return total
bad_total = calculate_total(-2)
print(bad_total) # print None
total_upto_10 = calculate_total(10)
print('Total of 1 ... 10 is', total_upto_10) # print 45
print('Total of 1 ... 25 is', calculate_total(25)) # print 300
For example:
def func(x):
y = x + 5 # y is "local" to func
print(y) # y is "visible"
func(10) # prints 15, which is y
print(y) # error: y is not "visible" to the caller
def foo():
f = 10
bar()
def bar():
print(f) # Error, I don't know what f is, even if my caller knows
foo()
def foo():
x = 10
bar()
print('in foo, x is', x)
def bar():
x = 11
print('in bar, x is', x)
foo()
def foo():
print('x is', x)
x = 10
foo() # print x is 10
def foo():
print('x is', x)
foo() # Error! Don't know what x is
x = 10
def foo():
x = 11
x = 10
foo()
print('x is', x) # What is the value of x?
def foo():
global x # Python! x is a global variable
x = 11
x = 10
foo()
print('x is', x) # This time, prints 11
def print_reverse_from(num):
print(num)
prev_num = num - 1
if prev_num > 0: # What if this check is not here?
print_reverse_from(prev_num)
print_reverse_from(10)
|
|
|
f0(1) # print 2
f0(2) # print 3
f0(-1) # print -3
f0(-2) # print -4
f0(0) # infinite loop


HTML My First jQuery Website : Card 1
Make some flash cards
0 First head over to codecademy.com and start on the JavaScript/jQuery track. Once you have the basic idea move on to the next step.
1 On JSFiddle change the drop down menu to the to be the latest version of jQuery (2.1 +). This allows to use the jQuery function ($) and its minions to effect the DOM.
2 On JSFiddle you can hit fork on this page: http://jsfiddle.net/rrbm823/3trf0ru5/
3 Recall from card 8 that we identify HTML elements from the CSS by their class attribute. You'll notice in flash-cards.html file there are many class attributes.
[Pic1]
They are there because we've put in the CSS template Bootstrap.
[Pic2]
We can spare the details of Bootstrap for another card.
4 Using JavaScript and jQuery, we want to get each flashcard question and answer from the array of objects called "allQuestions". This should be done by completing the callback function with the comment that says "finish me".
[Pic3]
It is the nameless function inside the .click() method.
5 The back and next button has been created already. Can you make a "show me" button to display the flash card answer? How about a card counter / shuffle option.
my-first-website/worksheets/03-make-the-home-page.pdf aPage


Input/Output
Comparison of two embeddings.
Decription
Quantification of local similarity across two maps or embeddings, such as PCA and t-SNE. To compare the embedding of samples in two different maps using a scale dependent similarity measure. For a pair of maps X and Y, we compare the sets of the, respectively, kx and ky nearest neighbours of each sample.
- param mapX
Mapping of first embedding.
- type mapX
numpy array
- param data2
Mapping of second embedding.
- type data2
numpy array
- param nn
number of neirest neighbor to compare between the two maps. This can be set based on the smalles class size or the aveage class size. The default is 250.
- type nn
integer, optional
- param n_steps
The number of evaluation steps until reaching nn, optional. If higher, the resolution becomes lower and vice versa. The default is 5.
- type n_steps
integer
- param verbose
print messages. The default is 3.
- type verbose
integer, optional
- returns
scores : array with the scores across various nearest neighbors (nn).
nn : nearest neighbors
n_steps : The number of evaluation steps until reaching nn.
- rtype
dict()
Examples
>>> # Load data
>>> X, y = flameplot.import_example()
>>>
>>> # Compute embeddings
>>> embed_pca = decomposition.TruncatedSVD(n_components=50).fit_transform(X)
>>> embed_tsne = manifold.TSNE(n_components=2, init='pca').fit_transform(X)
>>>
>>> # Compare PCA vs. tSNE
>>> scores = flameplot.compare(embed_pca, embed_tsne, n_steps=25)
>>>
>>> # plot PCA vs. tSNE
>>> fig = flameplot.plot(scores, xlabel='PCA', ylabel='tSNE')
>>>
References
Plot
Make plot.
- param out
output of the compare() function.
- type out
dict
- param cmap
colormap. The default is ‘jet’.
- type cmap
string, optional
- rtype
fig.


All Packages Class Hierarchy This Package Previous Next Index
Class com.jclark.xml.parse.NotWellFormedException
java.lang.Object
|
+----java.lang.Throwable
|
+----java.lang.Exception
|
+----java.io.IOException
|
+----com.jclark.xml.parse.NotWellFormedException
- public class NotWellFormedException
- extends IOException
- implements ParseLocation
Thrown when an XML document is not well-formed.
-
getByteIndex()
- Returns the index of the byte in the entity where the error occurred.
-
getColumnNumber()
- Returns the column number where the error occurred.
-
getEntityBase()
- Returns the URL used as the base URL for resolving relative URLs
contained in the entity where the error occurred.
-
getEntityLocation()
- Returns the location of the external entity where the
the error occurred in a form suitable for use in an error message.
-
getLineNumber()
- Returns the line number where the error occured.
-
getMessageWithoutLocation()
- Returns a description of the error that does not include
the location of the error.
getEntityLocation
public final String getEntityLocation()
- Returns the location of the external entity where the
the error occurred in a form suitable for use in an error message.
This is typically a URI or a filename.
getEntityBase
public final URL getEntityBase()
- Returns the URL used as the base URL for resolving relative URLs
contained in the entity where the error occurred.
getLineNumber
public final int getLineNumber()
- Returns the line number where the error occured.
The first line has number 1.
getColumnNumber
public final int getColumnNumber()
- Returns the column number where the error occurred.
The first column has number 0.
getByteIndex
public final long getByteIndex()
- Returns the index of the byte in the entity where the error occurred.
The first byte has offset 0.
getMessageWithoutLocation
public final String getMessageWithoutLocation()
- Returns a description of the error that does not include
the location of the error.
getMessage
returns a description of the error
that does include the location.
All Packages Class Hierarchy This Package Previous Next Index


Connect, configure and control a GigE Vision device. More...
Public Member Functions | |
| PvDeviceGEV () | |
| Constructor. | |
| virtual | ~PvDeviceGEV () |
| Destructor. | |
| PvResult | Connect (const PvDeviceInfo *aDeviceInfo) |
| Connect to a GigE Vision device. More... | |
| PvResult | Connect (const PvDeviceInfo *aDeviceInfo, PvAccessType aAccessType) |
| Connect to a GigE Vision device (found with PvSystem, PvInterface, PvDeviceInfo). More... | |
| PvResult | Connect (const PvString &aInfo) |
| Connect to a GigE Vision device with the default access control mode. More... | |
| PvResult | Connect (const PvString &aInfo, PvAccessType aAccessType) |
| Connect to a GigE Vision device. More... | |
| PvResult | SetStreamDestination (const PvString &aIPAddress, uint16_t aDataPort, uint32_t aChannel=0) |
| Set the UDP/IP destination to which the GigE Vision device transmits data. More... | |
| PvResult | ResetStreamDestination (uint32_t aChannel=0) |
| Resets the UDP/IP streaming destination of the GigE Vision device. More... | |
| PvResult | SetPacketSize (uint32_t aPacketSize, uint32_t aChannel=0) |
| Sets the packet size for a streaming channel. More... | |
| PvResult | NegotiatePacketSize (uint32_t aChannel=0, uint32_t aDefaultPacketSize=0) |
| Automatically finds and sets the optimal packet size. More... | |
| PvResult | ReadRegister (int64_t aAddress, uint32_t &aValue) |
| Read a 32-bit value from a specific register address on the GigE Vision device. More... | |
| PvResult | WriteRegister (int64_t aAddress, uint32_t aValue, bool aAcknowledge=true) |
| Write a 32-bit value to a specific register address on the GigE Vision device. More... | |
| Public Member Functions inherited from PvDevice | |
| virtual | ~PvDevice () |
| Destructor. | |
| PvDeviceType | GetType () const |
| Returns the type of the instantiated device. More... | |
| PvResult | Disconnect () |
| Disconnect the PvDevice object from the GigE Vision or USB3 Vision device to which it is currently connected. More... | |
| PvResult | StreamEnable (uint32_t aChannel=0) |
| Enables streaming on the device. More... | |
| PvResult | StreamDisable (uint32_t aChannel=0) |
| Disables streaming on the device. More... | |
| bool | IsConnected () const |
| Test if this PvDevice is currently connected to a GigE Vision or USB3 Vision device. More... | |
| bool | IsPleoraPowered () const |
| Test if this PvDevice is currently connected to a Pleora powered GigE Vision or USB3 Vision device. More... | |
| PvResult | DumpGenICamXML (const PvString &aFilename) |
| Save the GigE Vision or USB3 Visio device's GenICam XML file to disk. More... | |
| PvResult | GetDefaultGenICamXMLFilename (PvString &aFilename) |
| Returns a good default filename to use with DumpGenICamXML. More... | |
| PvGenParameterArray * | GetParameters () |
| Get the parameters used to control the GigE Vision or USB3 Vision device to which you are connected. More... | |
| PvGenParameterArray * | GetCommunicationParameters () |
| Get the parameters used to define how PvDevice connects to and communicates with the device. More... | |
| PvResult | ReadMemory (int64_t aAddress, unsigned char *aDestination, int64_t aByteCount) |
| Reads bytes starting at a specific register address on the GigE Vision or USB3 Vision device. More... | |
| PvResult | WriteMemory (int64_t aAddress, const unsigned char *aSource, int64_t aByteCount) |
| Writes bytes starting at a specific register address on the GigE Vision or USB3 Vision device. More... | |
| PvResult | WaitForMessagingChannelIdle (uint32_t aTimeout) |
| Waits for the messaging channel to become idle. More... | |
| PvResult | RegisterEventSink (PvDeviceEventSink *aEventSink) |
| Register an event sink used for callbacks. More... | |
| PvResult | UnregisterEventSink (PvDeviceEventSink *aEventSink) |
| Unregister an event sink. More... | |
| uint32_t | GetHeartbeatThreadPriority () const |
| Get the priority of the heartbeat thread. More... | |
| PvResult | SetHeartbeatThreadPriority (uint32_t aPriority) |
| Set the priority of the heartbeat thread. More... | |
| uint32_t | GetInterruptLinkThreadPriority () const |
| Get the priority of the interrupt link thread. More... | |
| PvResult | SetInterruptLinkThreadPriority (uint32_t aPriority) |
| Set the priority of the interrupt link thread. More... | |
| uint32_t | GetInterruptQueueThreadPriority () const |
| Get the priority of the interrupt queue thread. More... | |
| PvResult | SetInterruptQueueThreadPriority (uint32_t aPriority) |
| Set the priority of the interrupt queue thread. More... | |
| uint32_t | GetRecoveryThreadPriority () const |
| Get the priority of the recovery thread. More... | |
| PvResult | SetRecoveryThreadPriority (uint32_t aPriority) |
| Set the priority of the recovery thread. More... | |
| uint32_t | GetPayloadSize () |
| Returns the payload size the device is currently configured to stream with. More... | |
Static Public Member Functions | |
| static PvResult | GetAccessType (const PvString &aDeviceIPAddress, PvAccessType &aAccessType) |
| Returns the current access type of a device identified by its IP address. More... | |
| static PvResult | SetIPConfiguration (const PvString &aMACAddress, const PvString &aIP, const PvString &aSubnetMask=PvString("<IP_ADDRESS>"), const PvString &aGateway=PvString("<IP_ADDRESS>")) |
| Sets the GigE Vision device's IP address. More... | |
| Static Public Member Functions inherited from PvDevice | |
| static PvDevice * | CreateAndConnect (const PvDeviceInfo *aDeviceInfo, PvResult *aResult) |
| Dynamically allocates a PvDevice of the right type. More... | |
| static PvDevice * | CreateAndConnect (const PvString &aInfo, PvResult *aResult) |
| Dynamically allocates a PvDevice of the right type. More... | |
| static void | Free (PvDevice *aDevice) |
| Frees an object allocated with CreateAndConnect. More... | |
Additional Inherited Members | |
| Protected Member Functions inherited from PvDevice | |
| PvDevice () | |
| Constructor. | |
Connect, configure and control a GigE Vision device.
To connect and configure a GigE Vision device to begin transmitting data:
For a complete example that illustrates the steps above, refer to the following samples:
The SDK can let you automatically recover from problems such as accidental disconnects, power interrupts, and so on. To learn more, see PvDeviceEventSink.
|
virtual |
Connect to a GigE Vision device.
The Connect method connects to a GigE Vision device found using a PvSystem or device finder dialog object. Only one device can be connected to at a time.
| [in] | aDeviceInfo | The GigE Vision device's network connectivity information; a PvDeviceInfo object. |
Reimplemented from PvDevice.
| PvResult PvDeviceGEV::Connect | ( | const PvDeviceInfo * | aDeviceInfo, |
| PvAccessType | aAccessType | ||
| ) |
Connect to a GigE Vision device (found with PvSystem, PvInterface, PvDeviceInfo).
The Connect method connects to a GigE Vision device found using a PvSystem or device finder dialog object. Only one device can be connected to at a time.
| [in] | aDeviceInfo | The GigE Vision device's network connectivity information; a PvDeviceInfo object. |
| [in] | aAccessType | The GigE Vision device's availability for connections by multiple controllers. See PvAccessType. |
Connect to a GigE Vision device with the default access control mode.
If you know the GigE Vision device's IP address, MAC address or DeviceUserID, you can use this method to connect to it.
| [in] | aInfo | Information that uniquely identifies the GigE Vision device. You may use one of the following:
|
Reimplemented from PvDevice.
| PvResult PvDeviceGEV::Connect | ( | const PvString & | aInfo, |
| PvAccessType | aAccessType | ||
| ) |
Connect to a GigE Vision device.
If you know the GigE Vision device's IP address, MAC address or DeviceUserID, you can use this method to connect to it.
| [in] | aInfo | Information that uniquely identifies the GigE Vision device. You may use one of the following:
|
| [in] | aAccessType | The access control privilege to set at the time of connection. See PvAccessType. |
|
static |
Returns the current access type of a device identified by its IP address.
This function is static. It does not work on an specific object but will return the access type of a GigE Vision device identified by its IP address.
| [in] | aDeviceIPAddress | The IP address of the GigE Vision device. |
| [out] | aAccessType | The access type of the GigE Vision device on success. PvAccessUnknown on failure. |
| PvResult PvDeviceGEV::NegotiatePacketSize | ( | uint32_t | aChannel = 0 , |
| uint32_t | aDefaultPacketSize = 0 |
||
| ) |
Automatically finds and sets the optimal packet size.
The NegotiatePacketSize method tests the network connection for the largest possible packet size that the network can support between the GigE Vision transmitter and the controlling PC.
Calling this method resets the streaming destination for aChannel. You should always call this method AFTER connecting, BEFORE setting the GigE Vision streaming destination using SetStreamDestination.
| [in] | aChannel | The streaming channel. Use 0 for most GigE Vision devices only supporting a single streaming channel. |
| [in] | aDefaultPacketSize | The packet size to use, in bytes, should the negotiation fail. Use 1476 for a safe packet size; 0 to continue using the GigE Vision device's original setting in case of failure. |
| PvResult PvDeviceGEV::ReadRegister | ( | int64_t | aAddress, |
| uint32_t & | aValue | ||
| ) |
Read a 32-bit value from a specific register address on the GigE Vision device.
aValue is returned in the platform's expected Endianess. GigE Vision single register operations are always performed as Big Endian and on Intel platforms are corrected to/from Little Endian.
| [in] | aAddress | The GigE Vision device's register address (mod 4). |
| [out] | aValue | The value read from the GigE Vision device's register. |
| PvResult PvDeviceGEV::ResetStreamDestination | ( | uint32_t | aChannel = 0 | ) |
Resets the UDP/IP streaming destination of the GigE Vision device.
This method is equivalent to calling SetStreamDestination with values ("<IP_ADDRESS>", 0, aChannel).
| [in] | aChannel | See the aChannel parameter for the SetStreamDestination method. |
|
static |
Sets the GigE Vision device's IP address.
This static method lets you configure the GigE Vision device's IP address. You cannot change the IP address when a PvDevice is connected.
Can be used to assign a valid IP configuration to a device before connecting a PvDevice to a GigE Vision device (using the Connect method). The GigE Vision device to be configured is identified using its MAC address.
| [in] | aMACAddress | The GigE Vision device's MAC address. |
| [in] | aIP | The GigE Vision device's new IP address. |
| [in] | aSubnetMask | The GigE Vision device's new subnet mask. |
| [in] | aGateway | The GigE Vision device's new gateway. |
| PvResult PvDeviceGEV::SetPacketSize | ( | uint32_t | aPacketSize, |
| uint32_t | aChannel = 0 |
||
| ) |
Sets the packet size for a streaming channel.
Applies a specific packet size to a streaming channel. Unlike the NegotiatePacketSize method, it does not attempt to find the best packet size, it just uses the one provided as a parameter.
| [in] | aPacketSize | The packet size to use for aChannel. |
| [in] | aChannel | The streaming channel. Use 0 for most GigE Vision devices only supporting a single streaming channel. |
| PvResult PvDeviceGEV::SetStreamDestination | ( | const PvString & | aIPAddress, |
| uint16_t | aDataPort, | ||
| uint32_t | aChannel = 0 |
||
| ) |
Set the UDP/IP destination to which the GigE Vision device transmits data.
The destination constists of an IP address (unicast or multicast) and a port number.
| [in] | aIPAddress | The receiving PC's IP address (GenICam feature name: GevSCDA) or a multicast IP address. |
| [in] | aDataPort | The receiving PC's data port (feature name: GevSCPHostPort) or the multicast port. |
| [in] | aChannel | The GigE Vision stream channel; the channel over which the GigE Vision device transmits images. Use 0 for device having supporting a single streaming channel. |
| PvResult PvDeviceGEV::WriteRegister | ( | int64_t | aAddress, |
| uint32_t | aValue, | ||
| bool | aAcknowledge = true |
||
| ) |
Write a 32-bit value to a specific register address on the GigE Vision device.
See warnings about accessing the register map directly in WriteMemory.
aValue is provided in the platform's expected Endianess. GigE Vision single register operations are always performed as Big Endian and on Intel platforms are corrected to/from Little Endian.
| [in] | aAddress | The GigE Vision device's register address (mod 4). |
| [in] | aValue | The value to be written to the register. |
| [in] | aAcknowledge | When sending the command, sets the acknowledge requested bit and waits for that acknowledge. If set to false, the acknowledge is not requested and the method returns as soon as the command has been pushed in the UDP socket. If not specified true is assumed. |
Copyright (c) 2002-2015 Pleora Technologies Inc.
www.pleora.com


---
layout: post
title: Handling archives in linux (ranger file manager)
date: 2012-09-09 13:35:30 UTC
updated: 2012-09-09 13:35:30 UTC
comments: true
category: files
tags: [file-manager,archive,linux,ranger]
---
Today I was looking for a some plugin for unpacking files in Ranger file manager. I found great tool ( atool :) ). It is nice and simple command for packing and unpacking many types of archives.
First install atool, in ubuntu you do it with apt-get:
sudo apt-get install atool
ranger --copy-config all
). Now edit your bindings in ~/.config/ranger/rc.conf
map Pu shell atool -x %s map Pa shell atool -a %s


# Operations
## Checking availability and offers
This specification assumes that a *broker* has already obtained opportunity
data from a *booking system* that includes descriptions of **events**,
**facilities**, **slots** etc. For example by harvesting data via an [[RPDE]]
feed.
The availability of an **event**, or `subEvent`, or **facility** (either as a
`FacilityUse` or `Slot`) may change at any time.
The **offers** available to a customer might also change over time and the data
should be refreshed so that the most up-to-date representation of the live data
is presented to the **purchaser** before booking.
A **booking system** MUST allow a **broker** to check the current state of
**events** and/or **facilities** depending on what decide they expose from their
platform.
### Definition of a 'bookable' Event
An **event** is deemed to be 'bookable' if:
- The **event** has not already occurred, i.e. the `startDate` (when expressed
as a timedatestamp) and the `endDate` of the **event** has not been exceeded
- The `eventStatus` of the **event** does not indicate that it has been
cancelled or postponed
- The `remainingAttendeeCapacity` of the **event** is greater than the number
of places required
- The `potentialAction` of the **event** contains one or more appropriate
ReserveAction objects
- There are one or more valid bookable **offers** for the **event**
- The temporary lease on the space in the **event** (defined in
`orderLeaseDuration`) has not expired before the **payment** is taken
by the **broker** and the **seller** is notified by the **broker** that payment
has taken place.
The **customer** may be ineligible to attend a 'bookable' **event**, but may
book a space for someone else (e.g. a child) if:
- The gender of the **customer** taking up the opportunity is incompatible with
the `genderRestriction` of the **event** but the attendee is compatible
- The age of the **customer** taking up the opportunity is incompatible with the
`ageRange` of the **event** but the attendee is compatible
The **event** may also be deemed to be bookable if the **event** contains a deep
link to a URL where the user can book the **event** independently, however
this is out of scope for this API
The above set of restrictions are at the level of the **event** and the data
contained within the content of the API returns.
On a **booking system** level, an **event** is 'bookable', mediated by a
specific **broker**, if that **broker** has machine readable permission to
access the relevant instance of the Booking API provided by the **booking**
system via an API key or similar token.
For the purposes of this specification, the **event** is only bookable if the
data describing the the **event** is published via a **booking system** which
is compliant with and supports the Open Active Booking API.
### Definition of a 'bookable' slot/facility use
A slot is essentially an event, and many of the above conditions apply, however
some of the properties for a slot are different and as a consequence there are
different criteria for it being bookable.
A **slot** is deemed to be 'bookable' if:
- The **slot** has not already occurred, i.e. the `startDate` (when expressed
as a timedatestamp) and `endDate` of the **slot** has not been exceeded
- The `remainingUses` of the **slot** is greater than or equal to the number
of slots required. For example if an independently organised badminton league
requires four courts, the number of `remainingUses` should be 4 or higher.
- There are one or more valid **offers** for the slot
- The `potentialAction` of the **slot** contains one or more appropriate
ReserveAction objects
There may be more restrictions placed by a **seller** on the booking of a
**slot** for use of a **facility**. For example, it may be that one of the party
must be over a certain age as a supervising adult if some of the party are
minors. It is the responsibility of the **booking platform** to make the terms
and conditions of **booking** available to the **broker** and it is the
responsibility of the **broker** to ensure that the **customer** is aware of
these conditions before entering the **order** workflow.
### Checking availability and offers for an event or facility use slot
To check the availability of an opportunity a **broker** will issue a GET
request to the URI specified in the `id` of the opportunity, having
discovered it from the RDPE feed.
It is the responsibility of **brokers** to ensure that spaces within the
opportunity are still available before proceeding with booking.
GET /events/452 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
HTTP/1.1 200 OK
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Content-Length: ...
{
"@context": "https://openactive.io/ns/oa.jsonld",
"type": "Event",
"identifier": 452,
"id": "https://example.com/events/452",
"name": "Speedball",
"offers": [
{
"type": "Offer",
"acceptedPaymentMethod": [
"http://purl.org/goodrelations/v1#PaymentMethodCreditCard"
],
"availabilityEnds": "2018-04-29T12:14:35Z",
"availabilityStarts": "2018-04-01T12:14:35Z",
"description": "Winger space for Speedball.",
"id": "https://example.com/offers/1234",
"name": "Speedball winger position",
"price": "33.00",
"priceCurrency": "GBP",
}
],
"eventStatus": "https://schema.org/EventScheduled",
"maximumAttendeeCapacity": 30,
"remainingAttendeeCapacity": 20,
"startDate": "2018-01-27T11:00:00Z",
"endDate": "2018-01-27T12:00:00Z",
"duration": "PT1H",
"location": {
...
}
}
The **booking system** MUST ensure that the data presented to the **broker** in
the `remainingAttendeeCapacity` property of the **event** is up to date and includes a
combination of places already booked and places currently reserved by leases
from in-progress **orders**.
GET /slots/324 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
HTTP/1.1 200 OK
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Content-Length: ...
{
"@context": "https://openactive.io/ns/oa.jsonld",
"type": "Slot",
"id": "http://www.example.org/slots/324",
"identifier": 324,
"facilityUse": "http://www.example.org/facility-use/1",
"startDate": "2018-03-01T11:00:00Z",
"duration": "PT30M",
"remainingUses": 3,
"maximumUses": 4,
"offers": [
{
"type": "Offer",
"name": "30 minute hire",
"price": "15",
"priceCurrency": "GBP"
"potentialAction": [
{
"type": "ReserveAction",
"name": "Book",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "POST"
}
}
]
}
]
}
The **booking system** MUST ensure that the data presented to the **broker** in
the `remainingUses` property of the **slot** is up to date and includes a
combination of places already booked and places currently reserved by leases
from in-progress **orders**.
GET /facility-use/1 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
HTTP/1.1 200 OK
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Content-Length: ...
{
"@context": "https://www.openactive.io/ns/oa.jsonld",
"type": "FacilityUse",
"url": "http://www.example.org/facility-use/1",
"name": "Example Leisure Centre Table Tennis",
"description": "Table tennis tables are available to hire for thirty minute slots",
"activity": "Table Tennis",
"location": {
"type": "Place",
"name": "Example Leisure Centre",
"address": {
"type": "PostalAddress",
"streetAddress": "1 High Street",
"addressLocality": "Bristol",
"postalCode": "BS1 4SD"
}
},
"offers": [
{
"type": "Offer",
"name": "30 minute hire",
"price": "10",
"priceCurrency": "GBP",
"potentialAction": [
{
"type": "ReserveAction",
"name": "Book",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "POST"
}
}
]
}
],
"event": [
{
"type": "Slot",
"id": "http://www.example.org/slots/123",
"facilityUse": "http://www.example.org/facility-use/1",
"startDate": "2018-03-01T10:00:00Z",
"duration": "PT30M",
"remainingUses": 0,
"maximumUses": 4
},
{
"type": "Slot",
"id": "http://www.example.org/slots/234",
"facilityUse": "http://www.example.org/facility-use/1",
"startDate": "2018-03-01T11:00:00Z",
"duration": "PT30M",
"remainingUses": 3,
"maximumUses": 4,
"offers": [
{
"name": "30 minute hire",
"price": "15",
"priceCurrency": "GBP"
}
]
}
]
}
### Assembling a set of offers and presenting them to a customer
The **broker** MUST assemble a set of all applicable **offers** for an
**opportunity** from the information supplied by the **booking system**.
An applicable **offer** is an **offer** which is available for an
**opportunity** which is bookable and for which the **customer** is eligible.
#### Collating offers for an event
In the case of the **event** having no `subEvent` the set of all applicable
**offers** is simply the array of **offers** contained within the `offers`
property.
In the case of an **event** containing sub-events within the `subEvent` property
the set of all applicable offers should be compiled from the **offers** from the
sub-event selected by the customer, augmented with any additional different
offers contained in the parent super-event.
If the sub-event selected by the **customer** has no **offers** the set of all
applicable offers will be compiled from the **offers** contained withinin the
super-event.
The **broker** should then filter this set to provide a set of current **offers**
which apply to the **customer**. This specification provides no guidance as to
how the filtering should be performed.
The **broker** SHOULD however show all applicable **offers** to a customer,
including information on pricing restrictions, unless the **customer** has only
requested a subset in some form of preferences (e.g. only show adult classes
and offers).
The **customer** selects one of these **offers** and when all pre-conditions
have been met, the **broker** enters the **order** creation workflow on behalf
of the customer.
#### Collating offers for the use of a facility
The process for building a set of all applicable offers is similar for
**facilities** with the **offers** which are part of a specific **slot** having
primacy over the more generic **offers** contained within a`FacilityUse`.
The **booking system** MUST ensure that the data presented to the **broker** in
the `remainingUses` property in the **slot** is up to date and includes a
combination of places already booked and places currently reserved by leases
from in-progress **orders**. The **booking system** MUST also ensure that
the `hoursAvailable` property on the `FacilityUse` is kept updated.
### Obtaining a lease on a space within an opportunity
**Orders** SHOULD contain a `orderLeaseDuration` property. This
property MUST contain an ISO 8601 formatted duration for the lease. The lease
duration may also be advertised somewhere else to make it clear in advance,
e.g. in docs/setup config, or as part of a `potentialAction` property.
(Note: In previous iterations of the Booking API specification, the lease was
a property of an event. This has now been deprecated.)
The `orderLeaseDuration` property notifies the **broker** about the
length of time for which a space will be reserved by the **booking system**
whilst payment for the **event** is taken by the third-party payment system.
The default suggested lease duration is 15 minutes, but it is up to each
**booking plaftorm** to define and advertise their own lease duration.
This lease is NOT designed as a pause for a shopping cart system (in addition
shopping cart functionality is out of scope for v1.0 of this specification).
The **broker** MUST ensure that it has obtained all of the relevant details
about a **customer** before getting to the point where it attempts to place an
**order**. The lease is not designed as a pause in the process whilst a
**customer** is registered by a **broker**.
The client may then present the updated availability and suitable **offers** to
the customer.
## Creating and updating orders
**Booking systems** MUST expose an **orders** endpoint, e.g. `/orders` to allow
**brokers** to initiate the **booking** workflow.
**Orders** initiated by a specific **broker** SHOULD only be able to be updated
by the same **broker**. For the purpose of creating and updating an **order**
the only reliable mechanism for ensuring that the **broker** is the same across
requests is via authentication/identification by API keys or Authoriztion
headers. The **broker** property within the **order** is for information only.
It is the responsibility of the **booking system** to ensure that only the
**broker** which created the **order** can update, complete and cancel an
order.
### Discovering the order endpoint
**Booking systems** MUST expose an orders endpoint, e.g. /orders to allow
**brokers** to initiate and enter the **booking** workflow.
To initiate the **booking workflow**, the **booking system** must include a
`potentialAction` field within each **bookable** **opportunity**. The
`potentialAction` field MUST contain one or more objects of type ReserveAction.
These ReserveAction objects must contain a `target` field containing an
EntryPoint object.
The EntryPoint object MUST state the `url` of the orders endpoint, and SHOULD
contain the `encodingType` and the `httpMethod` which must be POST as a new
**order** object will be created on successful invocation.
An example is below:
"potentialAction": [
{
"type": "ReserveAction",
"name": "Book",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "POST"
}
}
]
### Pre-conditions for creating an order
Before creating an **order** the **broker** MUST:
- have obtained **customer** information
- have determined that the **opportunity** is bookable (see sections on the
definition of bookable for events and slots/facility uses)
- have identified the selected **offer**
- have prompted **customer** that they will enter a **payment** flow
(no pre-emptive leasing)
- have indicates that the **customer's** personal information is being submitted
to third-party (the **booking system** and the **seller**) and made the customer
aware of and assent to any legitimate privacy policy and terms and conditions
- have already obtained permission to use API
Before creating an **order** the **broker** SHOULD:
- have registered the **customer** (or have the **customer's** details to hand)
- have identified the **customer's** preferred payment options
### Creating a new order (`/orders`)
A POST request to the `/orders` endpoint of the **booking system** will result
in server creating an new **Order**. A time limited lease will be created for
spaces in the **event** referenced in the **Order**.
The `broker` property in the request body is to help the **booking platform** to
clarify who is making request and to help them in situations such as queries
to allow reconciliation of payments. It is not designed as any part of an
authentication which should be performed outside of the JSON body of the
request. No authentication tokens should be present within the `broker` property
of the submitted JSON.
The `broker` property should be a well formed schema:Organisation object.
In the situation where a middleware to be used to handle access control for
various **brokers**, independently of API key provisioning at the
**booking system** level (which in some systems is a lengthly and time-consuming
process), the `broker` property should contain the details of the end-user
**broker** rather than the middleware.
POST /orders HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
{
"@context": "https://openactive.io/ns/oa.jsonld",
"type": "Order",
"customer": {
"type": "Person",
"email"<EMAIL_ADDRESS>"givenName": "Geoff",
"familyName": "Capes"
},
"broker": {
"type": "Organization",
"name": "MyFitnessApp",
"url": "https://myfitnessapp.example.com"
},
"orderedItem": [{
"type": "OrderItem",
"orderQuantity": 1,
"acceptedOffer": {
"type": "Offer",
"acceptedPaymentMethod": [
"http://purl.org/goodrelations/v1#PaymentMethodCreditCard"
],
"availabilityEnds": "2018-04-29T12:14:35Z",
"availabilityStarts": "2018-04-01T12:14:35Z",
"description": "Winger space for Speedball.",
"id": "https://example.com/offers/1234",
"name": "Speedball winger position",
"price": "33.00",
"priceCurrency": "GBP"
}
}]
}
If successful the server will response with the location of a newly created
`Order` resource:
HTTP/1.1 201 Created
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Location: /orders/123
{
"@context": "https://openactive.io/ns/oa.jsonld",
"type": "Order",
"id": "https://example.com/orders/123",
"customer": {
"type": "Person",
"email"<EMAIL_ADDRESS>"givenName": "Geoff",
"familyName": "Capes"
},
"orderLeaseDuration": "PT15M",
"broker": {
"type": "Organization",
"name": "MyFitnessApp",
"url": "https://myfitnessapp.example.com"
},
"orderStatus": "https://schema.org/OrderPaymentDue",
"potentialAction": [
{
"type": "PayAction",
"name": "Pay",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders/{order_id}",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "PATCH"
}
}
],
"orderedItem": [{
"type": "OrderItem",
"orderQuantity": 1,
"acceptedOffer": {
"type": "Offer",
"acceptedPaymentMethod": [
"http://purl.org/goodrelations/v1#PaymentMethodCreditCard"
],
"availabilityEnds": "2018-04-29T12:14:35Z",
"availabilityStarts": "2018-04-01T12:14:35Z",
"description": "Winger space for Speedball.",
"id": "https://example.com/offers/1234",
"name": "Speedball winger position",
"price": "33.00",
"priceCurrency": "GBP"
}
}]
}
### Viewing an order (`/orders/{orderId}`)
To get the latest state of an order, perform a GET request on its URI.
GET /orders/123 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
HTTP/1.1 200 Created
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Content-Length: ...
{
"type": "Order",
"id": "https://example.com/orders/123",
"identifier": 123,
"customer": {
"type": "Person",
"email"<EMAIL_ADDRESS>"givenName": "Geoff",
"familyName": "Capes"
},
"orderedItem": [
{
"type": "OrderItem",
"acceptedOffer": {
"type": "Offer",
"acceptedPaymentMethod": [
"http://purl.org/goodrelations/v1#PaymentMethodCreditCard"
],
"availabilityEnds": "2018-04-29T12:14:35Z",
"availabilityStarts": "2018-04-01T12:14:35Z",
"description": "Winger space for Speedball.",
"id": "https://example.com/offers/1234",
"name": "Speedball winger position",
"price": "33.00",
"priceCurrency": "GBP",
},
"orderedItem": "https://example.com/events/452",
"amount": {
"type": "MonetaryAmount",
"value": "10.00",
"currency": "GBP"
},
}
],
"orderStatus": "https://schema.org/OrderPaymentDue",
"paymentDueDate": "2018-02-20T11:00:00Z",
"orderDate": "2018-02-20T11:00:00Z",
"partOfInvoice": {
"type": "Invoice",
"paymentStatus": "PaymentDue",
"totalPaymentDue": {
"type": "MonetaryAmount",
"value": "10.00",
"currency": "GBP"
}
},
"potentialAction": [
{
"type": "PayAction",
"name": "Pay",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders/{order_id}",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "PATCH"
}
},
{
"type": "CancelAction",
"name": "Pay",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders/{order_id}",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "PATCH"
}
}
]
}
### Deleting an order (`/orders/{orderId}`)
A `DELETE` request to an **Order** URI will delete the order. Deletion may only
be used in the specific case for cancelling an **order** and the associated
lease on a space in an **event** if the **customer** no longer wishes to take
up the opportunity.
A `DELETE` request MUST NOT be used to cancel an existing completed **order**
as part of a refund/cancellation workflow. An appropriate approach for that is
defined elsewhere.
### Completing an order (`/orders/{orderId}`)
An **order** is completed by the **broker** submitting notification that the
**customer** has made a payment via a third-party payment gateway.
An **order** is not considered complete until payment has occurred.
**Brokers** MUST update the **booking system** to indicate that **payment** has
been taken.
Once the **booking system** is content with the payments, it then sets the
`orderStatus` property of the `Order` to https://schema.org/OrderDelivered and
returns a representation of the completed **order** with a 200 status code.
PATCH /orders/123 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
{
"type": "Order",
"payments": [
{
"type": "Payment",
"paymentMethod": "https://purl.org/goodrelations/v1#PaymentMethodCreditCard",
"paymentMethodId": "3204",
"totalPaidToProvider": {
"type": "MonetaryAmount",
"value": "8.00",
"currency": "GBP"
}
}
]
}
HTTP/1.1 200 OK
Date: Tue, 07 Jun 2018 20:51:36 GMT
Content-Type: application/vnd.openactive.v0.7+json
Content-Length: ...
{
"type": "Order",
"id": "https://example.com/orders/535",
"identifier": 535,
"customer": {
"type": "Person",
"email"<EMAIL_ADDRESS>"givenName": "Geoff",
"familyName": "Capes"
},
"orderedItem": [{
"type": "OrderItem",
"acceptedOffer": {
"type": "Offer",
"acceptedPaymentMethod": [
"http://purl.org/goodrelations/v1#PaymentMethodCreditCard"
],
"availabilityEnds": "2018-04-29T12:14:35Z",
"availabilityStarts": "2018-04-01T12:14:35Z",
"description": "Winger space for Speedball.",
"id": "https://example.com/offers/1234",
"name": "Speedball winger position",
"price": "33.00",
"priceCurrency": "GBP"
},
"orderedItem": "https://example.com/events/452",
"amount": {
"type": "MonetaryAmount",
"value": "10.00",
"currency": "GBP"
}
}],
"orderStatus": "https://schema.org/OrderDelivered",
"paymentDueDate": "2018-02-20T11:00:00Z",
"orderDate": "2018-02-20T11:00:00Z",
"partOfInvoice": {
"type": "Invoice",
"paymentStatus": "PaymentDue",
"totalPaymentDue": {
"type": "MonetaryAmount",
"value": "10.00",
"currency": "GBP"
}
},
"payments": [{
"type": "Payment",
"paymentMethod": "https://purl.org/goodrelations/v1#PaymentMethodCreditCard",
"paymentMethodId": "3204",
"totalPaidToProvider": {
"type": "MonetaryAmount",
"value": "8.00",
"currency": "GBP"
},
"confirmationNumber": "T12371284"
}],
"potentialAction": [{
"type": "CancelAction",
"name": "Pay",
"target": {
"type": "EntryPoint",
"urlTemplate": "https://example.com/orders/{order_id}",
"encodingType": "application/vnd.openactive.v0.7+json",
"httpMethod": "PATCH"
}
}]
}
## Ordering a place at a free event/ordering a free slot
Free **opportunities** MUST follow the same workflow as paid **opportunities**.
A free **event** or **slot** is technically a free **offer** on a space for an
opportunity. There MUST be at least one **offer** with a `price` property of
"0.00". The `priceCurrency` SHOULD still be defined, where possible and
appropriate, for data consistency and completeness.
The **order** workflow MUST be followed as described for **offers** with a
payment. The PATCH to the URL defined in `PayAction` should follow the example
shown below.
**Orders** for a free place at an **opportunity** are the only current scenario
in the Booking API specification where a `Payment` object has `paymentMethod`
and `paymentMethodId` as optional properties.
PATCH /orders/123 HTTP/1.1
Host: example.com
Date: Tue, 07 Jun 2018 20:51:35 GMT
Accept: application/vnd.openactive.v0.7+json
{
"type": "Order",
"payments": [{
"type": "Payment",
"totalPaidToProvider": {
"type": "MonetaryAmount",
"value": "0.00",
"currency": "GBP"
}
}]
}
## Leases and the orderLeaseDuration property
If the lease expires during **payment**, the **broker** MAY delete the **order**
(if the **booking system** has not already done so, returning a response with a
410 status code).
The **broker** should then create a new **order** for the
**customer** in order to generate a new lease.
Before creating a new order, the **broker** MUST check the **opportunity** and
**offers** to ensure that a space still exists within the **opportunity**
and that the **offer** selected by the **customer** is still valid.
If no space exists or the **offer** is now invalid, the **customer**
should be refunded if payment has been taken.
## The state transitions during the order process
1. An **order** object is created by a **broker** invoking the orders endpoint
with a POST request containing details of the **broker** (represented as a
schema:Organisation), **customer** (represented as a schema:Person) and details
of the selected **event** and **offer** as components of a schema:OrderItem
object.
The **booking system** returns a representation of the **order** object to the
**broker** in a response with a 201 status and the URL for the object in the
`Location` header. The `orderStatus` of the **order** at this point in the flow
should be https://schema.org/OrderPaymentDue
2. Concurrently the **booking system** also creates a lease for a space in the
**event** for the **customer** for the duration specified in the
`orderLeaseDuration` property of the **event**.
3. The **broker** directs the customer into the flow of the third-party
**payment** provider.
4. If the **customer** successfully completes the payment within the duration
of the lease, the **broker** sends notification of the payment with a PATCH
request to the specific order end point (/orders/123 in the examples) with
details of the **payment** in the `paymentMethod` and `paymentMethodId`
properties. At this point the **order** is deemed to be completed. The
`orderStatus` property should be currently set to
`https://schema.org/OrderDelivered` and the `confirmationNumber`
property set to an appropriate unique value by the **booking system**. At this
point in the flow, the **order** is deemed to be complete.
If the lease expires before the payment is made, the **order** should either be
deleted by the **broker** or be deleted by the **booking system** at some point
after expiry. The `orderStatus` property should be set to
https://schema.org/OrderCancelled to indicate that the **order** cannot proceed
to completion.
## Actions which occur after order completion: confirmations and refunds
Both the **broker** and **booking system** may need to take further actions after
completing an order. This section describes some potential interactions.
### Supplying a confirmation to the customer
The **broker** SHOULD provide the **customer** with confirmation of a successful
payment having been made through the third-party system and that the **order**
has been completed successfully. The **broker** will have access to a completed
**order** which will contain a `confirmationCode` property.
The **booking system** may also provide the **customer** with a confirmation that
the space at the **opportunity** has been booked on their behalf.
### Refunding a customer and cancelling an order
Refunds and cancellations are a responsibility of the **broker** who is the
party in the system in a position to process a refund for the **customer** with
the third-party payments system.
The **broker** should initiate a refund with the third-party payment provider.
The refund process should be considered to be asynchronous for the purposes of
this specification.
After initiating a refund, the **broker** should cancel the
**order** by submitting a PATCH request to the URL defined in the `CancelAction`
prescribed in the `potentialAction` property of the **order** response.
The process of cancellation is documented in Section 5.5.2
## Customers
### Creation of customers on the booking system
In this current iteration of the Booking API specification, the programmatic
creation of **customers** on the **booking system** by the **broker** is
considered to be out of scope.
### Viewing a customer's previously placed orders
A server MUST provide endpoints to allow **brokers** to retrieve information
about previously placed **orders**. In future versions of the specification,
endpoints may be provided which permit a **broker** to see all **orders**
created by a **customer**. In this current implementation, if the **broker**
wishes to present a view of all **orders** for a **customer**, it should keep
an internal record.
### Deletion of personal data of the customer if the order is not completed
If an **order** is not completed, the **booking system** SHOULD delete all
personal information of the **customer** provided to it as part of the **order**
creation process.


Monitors Java Virtual Machine (JVM) statistics. This command is experimental and unsupported.
jstat [ generalOption | outputOptions vmid [ interval[s|ms] [ count ] ]
A single general command-line option -help
or -options
. See General Options.
One or more output options that consist of a single statOption
, plus any of the -t
, -h
, and -J
options. See Output Options.
Virtual machine identifier, which is a string that indicates the target JVM. The general syntax is the following:
[protocol:][//]lvmid[@hostname[:port]/servername]
The syntax of the vmid
string corresponds to the syntax of a URI. The vmid
string can vary from a simple integer that represents a local JVM to a more complex construction that specifies a communications protocol, port number, and other implementation-specific values. See Virtual Machine Identifier.
Sampling interval in the specified units, seconds (s) or milliseconds (ms). Default units are milliseconds. Must be a positive integer. When specified, the jstat
command produces its output at each interval.
Number of samples to display. The default value is infinity which causes the jstat
command to display statistics until the target JVM terminates or the jstat
command is terminated. This value must be a positive integer.
The jstat
command displays performance statistics for an instrumented Java HotSpot VM. The target JVM is identified by its virtual machine identifier, or vmid
option.
The syntax of the vmid
string corresponds to the syntax of a URI:
[protocol:][//]lvmid[@hostname[:port]/servername]
The communications protocol. If the protocol value is omitted and a host name is not specified, then the default protocol is a platform-specific optimized local protocol. If the protocol value is omitted and a host name is specified, then the default protocol is rmi
.
The local virtual machine identifier for the target JVM. The lvmid
is a platform-specific value that uniquely identifies a JVM on a system. The lvmid
is the only required component of a virtual machine identifier. The lvmid
is typically, but not necessarily, the operating system's process identifier for the target JVM process. You can use the jps
command to determine the lvmid
. Also, you can determine the lvmid
on Solaris, Linux, and OS X platforms with the ps
command, and on Windows with the Windows Task Manager.
A hostname or IP address that indicates the target host. If the hostname value is omitted, then the target host is the local host.
The default port for communicating with the remote server. If the hostname value is omitted or the protocol value specifies an optimized, local protocol, then the port value is ignored. Otherwise, treatment of the port
parameter is implementation-specific. For the default rmi
protocol, the port value indicates the port number for the rmiregistry on the remote host. If the port value is omitted and the protocol value indicates rmi
, then the default rmiregistry port (1099) is used.
The treatment of the servername
parameter depends on implementation. For the optimized local protocol, this field is ignored. For the rmi
protocol, it represents the name of the RMI remote object on the remote host.
The jstat
command supports two types of options, general options and output options. General options cause the jstat
command to display simple usage and version information. Output options determine the content and format of the statistical output.
All options and their functionality are subject to change or removal in future releases.
If you specify one of the general options, then you cannot specify any other option or parameter.
Displays a help message.
Displays a list of static options. See Output Options.
If you do not specify a general option, then you can specify output options. Output options determine the content and format of the jstat
command's output, and consist of a single statOption
, plus any of the other output options (-h
, -t
, and -J
). The statOption
must come first.
Output is formatted as a table, with columns that are separated by spaces. A header row with titles describes the columns. Use the -h
option to set the frequency at which the header is displayed. Column header names are consistent among the different options. In general, if two options provide a column with the same name, then the data source for the two columns is the same.
Use the -t
option to display a time stamp column, labeled Timestamp as the first column of output. The Timestamp column contains the elapsed time, in seconds, since the target JVM started. The resolution of the time stamp is dependent on various factors and is subject to variation due to delayed thread scheduling on heavily loaded systems.
Use the interval and count parameters to determine how frequently and how many times, respectively, the jstat
command displays its output.
Note: Do not to write scripts to parse the jstat
command's output because the format might change in future releases. If you write scripts that parse jstat
command output, then expect to modify them for future releases of this tool.
Determines the statistics information the jstat
command displays. The following lists the available options. Use the -options
general option to display the list of options for a particular platform installation. See Stat Options and Output.
class
: Displays statistics about the behavior of the class loader.
compiler
: Displays statistics about the behavior of the Java HotSpot VM Just-in-Time compiler.
gc
: Displays statistics about the behavior of the garbage collected heap.
gccapacity
: Displays statistics about the capacities of the generations and their corresponding spaces.
gccause
: Displays a summary about garbage collection statistics (same as -gcutil
), with the cause of the last and current (when applicable) garbage collection events.
gcnew
: Displays statistics of the behavior of the new generation.
gcnewcapacity
: Displays statistics about the sizes of the new generations and its corresponding spaces.
gcold
: Displays statistics about the behavior of the old generation and metaspace statistics.
gcoldcapacity
: Displays statistics about the sizes of the old generation.
gcmetacapacity
: Displays statistics about the sizes of the metaspace.
gcutil
: Displays a summary about garbage collection statistics.
printcompilation
: Displays Java HotSpot VM compilation method statistics.
Displays a column header every n samples (output rows), where n is a positive integer. Default value is 0, which displays the column header the first row of data.
Displays a timestamp column as the first column of output. The time stamp is the time since the start time of the target JVM.
Passes javaOption
to the Java application launcher. For example, -J-Xms48m
sets the startup memory to 48 MB. For a complete list of options, see java
(1).
The following information summarizes the columns that the jstat
command outputs for each statOption.
Class loader statistics.
Loaded
: Number of classes loaded.
Bytes
: Number of kBs loaded.
Unloaded
: Number of classes unloaded.
Bytes
: Number of Kbytes unloaded.
Time
: Time spent performing class loading and unloading operations.
Java HotSpot VM Just-in-Time compiler statistics.
Compiled
: Number of compilation tasks performed.
Failed
: Number of compilations tasks failed.
Invalid
: Number of compilation tasks that were invalidated.
Time
: Time spent performing compilation tasks.
FailedType
: Compile type of the last failed compilation.
FailedMethod
: Class name and method of the last failed compilation.
Garbage-collected heap statistics.
S0C
: Current survivor space 0 capacity (kB).
S1C
: Current survivor space 1 capacity (kB).
S0U
: Survivor space 0 utilization (kB).
S1U
: Survivor space 1 utilization (kB).
EC
: Current eden space capacity (kB).
EU
: Eden space utilization (kB).
OC
: Current old space capacity (kB).
OU
: Old space utilization (kB).
MC
: Metaspace capacity (kB).
MU
: Metacspace utilization (kB).
CCSC
: Compressed class space capacity (kB).
CCSU
: Compressed class space used (kB).
YGC
: Number of young generation garbage collection events.

FGC
: Number of full GC events.

GCT
: Total garbage collection time.
Memory pool generation and space capacities.
NGCMN
: Minimum new generation capacity (kB).
NGCMX
: Maximum new generation capacity (kB).
NGC
: Current new generation capacity (kB).
S0C
: Current survivor space 0 capacity (kB).
S1C
: Current survivor space 1 capacity (kB).
EC
: Current eden space capacity (kB).
OGCMN
: Minimum old generation capacity (kB).
OGCMX
: Maximum old generation capacity (kB).
OGC
: Current old generation capacity (kB).
OC
: Current old space capacity (kB).
MCMN
: Minimum metaspace capacity (kB).
MCMX
: Maximum metaspace capacity (kB).
MC
: Metaspace capacity (kB).
CCSMN
: Compressed class space minimum capacity (kB).
CCSMX
: Compressed class space maximum capacity (kB).
CCSC
: Compressed class space capacity (kB).
YGC
: Number of young generation GC events.
FGC
: Number of full GC events.
This option displays the same summary of garbage collection statistics as the -gcutil
option, but includes the causes of the last garbage collection event and (when applicable) the current garbage collection event. In addition to the columns listed for -gcutil
, this option adds the following columns.

S0C
: Current survivor space 0 capacity (kB).
S1C
: Current survivor space 1 capacity (kB).
S0U
: Survivor space 0 utilization (kB).
S1U
: Survivor space 1 utilization (kB).
TT
: Tenuring threshold.
MTT
: Maximum tenuring threshold.
DSS
: Desired survivor size (kB).
EC
: Current eden space capacity (kB).
EU
: Eden space utilization (kB).
YGC
: Number of young generation GC events.

New generation space size statistics.
NGCMN
: Minimum new generation capacity (kB).
NGCMX
: Maximum new generation capacity (kB).
NGC
: Current new generation capacity (kB).
S0CMX
: Maximum survivor space 0 capacity (kB).
S0C
: Current survivor space 0 capacity (kB).
S1CMX
: Maximum survivor space 1 capacity (kB).
S1C
: Current survivor space 1 capacity (kB).
ECMX
: Maximum eden space capacity (kB).
EC
: Current eden space capacity (kB).
YGC
: Number of young generation GC events.
FGC
: Number of full GC events.
Old generation and metaspace behavior statistics.
MC
: Metaspace capacity (kB).
MU
: Metaspace utilization (kB).
CCSC
: Compressed class space capacity (kB).
CCSU
: Compressed class space used (kB).
OC
: Current old space capacity (kB).
OU
: Old space utilization (kB).
YGC
: Number of young generation GC events.
FGC
: Number of full GC events.

GCT
: Total garbage collection time.
Old generation size statistics.
OGCMN
: Minimum old generation capacity (kB).
OGCMX
: Maximum old generation capacity (kB).
OGC
: Current old generation capacity (kB).
OC
: Current old space capacity (kB).
YGC
: Number of young generation GC events.
FGC
: Number of full GC events.

GCT
: Total garbage collection time.
Metaspace size statistics.
MCMN
: Minimum metaspace capacity (kB).
MCMX
: Maximum metaspace capacity (kB).
MC
: Metaspace capacity (kB).
CCSMN
: Compressed class space minimum capacity (kB).
CCSMX
: Compressed class space maximum capacity (kB).
YGC
: Number of young generation GC events.
FGC
: Number of full GC events.

GCT
: Total garbage collection time.

S0
: Survivor space 0 utilization as a percentage of the space's current capacity.
S1
: Survivor space 1 utilization as a percentage of the space's current capacity.
E
: Eden space utilization as a percentage of the space's current capacity.
O
: Old space utilization as a percentage of the space's current capacity.
M
: Metaspace utilization as a percentage of the space's current capacity.
CCS
: Compressed class space utilization as a percentage.
YGC
: Number of young generation GC events.

FGC
: Number of full GC events.

GCT
: Total garbage collection time.
Java HotSpot VM compiler method statistics.
Compiled
: Number of compilation tasks performed by the most recently compiled method.
Size
: Number of bytes of byte code of the most recently compiled method.
Type
: Compilation type of the most recently compiled method.
Method
: Class name and method name identifying the most recently compiled method. Class name uses slash (/) instead of dot (.) as a name space separator. Method name is the method within the specified class. The format for these two fields is consistent with the HotSpot -XX:+PrintCompilation
option.
This section presents some examples of monitoring a local JVM with an lvmid of 21891.
This example attaches to lvmid 21891 and takes 7 samples at 250 millisecond intervals and displays the output as specified by the -gcutil
option.
The output of this example shows that a young generation collection occurred between the third and fourth sample. The collection took 0.078 seconds and promoted objects from the eden space (E) to the old space (O), resulting in an increase of old space utilization from 66.80% to 68.19%. Before the collection, the survivor space was 97.02% utilized, but after this collection it is 91.03% utilized.
jstat -gcutil 21891 250 7 S0 S1 E O M CCS YGC YGCT FGC FGCT GCT 0.00 97.02 70.31 66.80 95.52 89.14 7 0.300 0 0.000 0.300 0.00 97.02 86.23 66.80 95.52 89.14 7 0.300 0 0.000 0.300 0.00 97.02 96.53 66.80 95.52 89.14 7 0.300 0 0.000 0.300 91.03 0.00 1.98 68.19 95.89 91.24 8 0.378 0 0.000 0.378 91.03 0.00 15.82 68.19 95.89 91.24 8 0.378 0 0.000 0.378 91.03 0.00 17.80 68.19 95.89 91.24 8 0.378 0 0.000 0.378 91.03 0.00 17.80 68.19 95.89 91.24 8 0.378 0 0.000 0.378
This example attaches to lvmid 21891 and takes samples at 250 millisecond intervals and displays the output as specified by -gcnew
option. In addition, it uses the -h3
option to output the column header after every 3 lines of data.
In addition to showing the repeating header string, this example shows that between the second and third samples, a young GC occurred. Its duration was 0.001 seconds. The collection found enough active data that the survivor space 0 utilization (S0U) would have exceeded the desired survivor Size (DSS). As a result, objects were promoted to the old generation (not visible in this output), and the tenuring threshold (TT) was lowered from 31 to 2.
Another collection occurs between the fifth and sixth samples. This collection found very few survivors and returned the tenuring threshold to 31.
jstat -gcnew -h3 21891 250 S0C S1C S0U S1U TT MTT DSS EC EU YGC YGCT 64.0 64.0 0.0 31.7 31 31 32.0 512.0 178.6 249 0.203 64.0 64.0 0.0 31.7 31 31 32.0 512.0 355.5 249 0.203 64.0 64.0 35.4 0.0 2 31 32.0 512.0 21.9 250 0.204 S0C S1C S0U S1U TT MTT DSS EC EU YGC YGCT 64.0 64.0 35.4 0.0 2 31 32.0 512.0 245.9 250 0.204 64.0 64.0 35.4 0.0 2 31 32.0 512.0 421.1 250 0.204 64.0 64.0 0.0 19.0 31 31 32.0 512.0 84.4 251 0.204 S0C S1C S0U S1U TT MTT DSS EC EU YGC YGCT 64.0 64.0 0.0 19.0 31 31 32.0 512.0 306.7 251 0.204
This example attaches to lvmid 21891 and takes 3 samples at 250 millisecond intervals. The -t
option is used to generate a time stamp for each sample in the first column.
The Timestamp column reports the elapsed time in seconds since the start of the target JVM. In addition, the -gcoldcapacity
output shows the old generation capacity (OGC) and the old space capacity (OC) increasing as the heap expands to meet allocation or promotion demands. The old generation capacity (OGC) has grown from 11,696 kB to 13,820 kB after the eighty-first full garbage collection (FGC). The maximum capacity of the generation (and space) is 60,544 kB (OGCMX), so it still has room to expand.
Timestamp OGCMN OGCMX OGC OC YGC FGC FGCT GCT 150.1 1408.0 60544.0 11696.0 11696.0 194 80 2.874 3.799 150.4 1408.0 60544.0 13820.0 13820.0 194 81 2.938 3.863 150.7 1408.0 60544.0 13820.0 13820.0 194 81 2.938 3.863
This example attaches to lvmid 40496 on the system named remote.domain using the -gcutil
option, with samples taken every second indefinitely.
The lvmid is combined with the name of the remote host to construct a vmid of<EMAIL_ADDRESS>. This vmid results in the use of the rmi
protocol to communicate to the default jstatd
server on the remote host. The jstatd
server is located using the rmiregistry
command on remote.domain
that is bound to the default port of the rmiregistry
command (port 1099).
jstat -gcutil<EMAIL_ADDRESS>1000 ... output omitted


Source code for delphin_6_automation.database_interactions.simulation_interactions
__author__ = "Christian Kongsgaard"
__license__ = "MIT"
# -------------------------------------------------------------------------------------------------------------------- #
# IMPORTS
# Modules:
import os
import datetime
import time
import shutil
import typing
# RiBuild Modules:
import delphin_6_automation.database_interactions.db_templates.delphin_entry as delphin_db
from delphin_6_automation.database_interactions import general_interactions as general_interact
from delphin_6_automation.logging.ribuild_logger import ribuild_logger
# Logger
logger = ribuild_logger(__name__)
# -------------------------------------------------------------------------------------------------------------------- #
# RIBUILD SIMULATION FUNCTIONS AND CLASSES
[docs]def download_simulation_result(sim_id: str, download_path: str, raw_or_processed='raw') -> None:
"""
Downloads Delphin simulation results from the database.
:param sim_id: Delphin project ID
:param download_path: Path to download to
:param raw_or_processed: Whether to download the raw results or the processed ones
:return: None
"""
object_ = delphin_db.Delphin.objects(id=sim_id).first()
download_extended_path = download_path + '/' + str(sim_id)
os.mkdir(download_extended_path)
if raw_or_processed == 'raw':
result_id = object_.results_raw
logger.info(f'Downloads raw result with ID: {result_id} from Delphin project with ID: {sim_id}')
general_interact.download_raw_result(result_id.id, download_extended_path)
elif raw_or_processed == 'processed':
pass
# TODO - Download processed results from database
else:
raise ValueError('raw_or_processed has to be raw or processed. Value given was: ' + str(raw_or_processed))
return None
[docs]def find_next_sim_in_queue() -> typing.Optional[str]:
"""
Finds the next entry in the simulation queue, which is not yet simulated and has the highest queue priority.
:return: If a entry is found the id will be returned otherwise None.
"""
try:
id_ = delphin_db.Delphin.objects(simulating=False, simulated=None).order_by('-queue_priority').first().id
set_simulating(str(id_), True)
logger.debug(f'Found unsimulated Delphin project with ID: {id_}')
return str(id_)
except AttributeError:
logger.info('All Delphin Projects in the queue are simulated!')
time.sleep(60)
return None
[docs]def set_simulating(id_: str, set_to: bool) -> str:
"""
Set the simulating flag of an entry.
:param id_: ID of the entry
:param set_to: What to set simulating to. Should be either True or False.
:return: ID of the entry
"""
simulation = delphin_db.Delphin.objects(id=id_).first()
simulation.update(set__simulating=set_to)
logger.debug(f'For Delphin project with ID: {id_}, simulation was change to: {set_to}')
return simulation.id
[docs]def set_simulated(id_: str) -> str:
"""
Flags an entry for finishing the simulation.
:param id_: ID of the entry
:return: ID of the entry
"""
simulation = delphin_db.Delphin.objects(id=id_).first()
simulation.update(set__simulated=datetime.datetime.now())
set_simulating(id_, False)
logger.debug(f'For Delphin project with ID: {id_}, simulated was changed to: {datetime.datetime.now()}')
return simulation.id
[docs]def clean_simulation_folder(path: str) -> bool:
"""
Cleans the simulation folder for content
:param path: Path to the simulation folder
:return: True on success
"""
shutil.rmtree(path)
logger.debug(f'Deleted {path}')
return True
[docs]def set_simulation_time(sim_id: str, computation_time: datetime.timedelta) -> str:
"""Sets the time it took to simulate Delphin project"""
delphin_entry = delphin_db.Delphin.objects(id=sim_id).first()
delphin_entry.update(set__simulation_time=computation_time.total_seconds())
logger.debug(f'For Delphin project with ID: {sim_id}, '
f'simulation time was changed to: {computation_time.total_seconds()}')
return sim_id
[docs]def wait_until_simulated(delphin_ids: list) -> bool:
"""
Wait until all simulations in the given list is simulated.
:param delphin_ids: List with Delphin database ids
:return: True
"""
simulated = [False] * len(delphin_ids)
while not all(simulated):
for index, id_ in enumerate(delphin_ids):
entry = delphin_db.Delphin.objects(id=id_).first()
if entry.simulated:
simulated[index] = True
else:
logger.debug('Waiting until all projects are simulated')
time.sleep(180)
logger.debug('All projects are simulated')
return True