About The Info Session
Flashstarts is an accelerator and micro-VC firm devoted to building the next generator of Cleveland companies. We focus on startups in blockchain healthcare IT, enterprise SaaS, data and analytics, AR/VR, lot, and FinTech. We believe that the most profitable startup opportunities occur at the intersection of software and other, very different disciplines. We look for teams that combine domain expertise and passion with a deep understanding of agile development.
Flashstarts is pleased to announce our first Learn About Flashstarts Blockchain Startup Fund event, for accredited investors on November 5th at StartMart. Here, we will explore investment and the emerging blockchain technology. The event will provide the opportunity to learn more about the momentum behind the Blockland initiative to establish Cleveland as a leading technology center for blockchain solutions.
Details
Doors open at 5:30PM. The event starts at 6PM and is expected to run until 730PM.
-
5.30 PM
Doors open.
-
6.00 PM
Presentation begins.
FlashStarts CEO, Charles Stack. -
7.30 PM
Event ends.
Our Speaker
Charles Stack is a successful serial entrepreneur. After graduation from Case Western Reserve University Law School, he formed Parallax, Inc., which developed and marketed a PC-based networking case management software system for asbestos litigation to national law firms and corporate law departments.
In 1992, Charles started Books.com, the first online bookstore. Charles pioneered many of the features we now take for granted in today’s internet retailers. Books.com offered millions of books, comprehensive searching, collaborative recommendations (patented), shopping carts, price comparisons (patented), real-time credit card processing, affiliate programs, and introduced many other now standard Internet retailing concepts. Books.com was acquired by CUC, Inc. in early 1996 and subsequently by Barnes and Noble. At CUC, Charles designed and launched groundbreaking sites for online music sales, movies, gifts, computer software, and games.
In 1998, he founded Flashline to address the new opportunity in software components and web services. After navigating Flashline through the technology meltdown of 2001-2003, Charles focused Flashline on the expanding web services management market and created the leading web services repository solution with multiple patents. BEA Systems, Inc. acquired Flashline in 2006. BEA was acquired by Oracle in 2008 and the Flashline product remains the core offering in Oracle’s web services suite. Charles has mentored, advised, and/or invested the first money in several notable startups including OnShift, DecisionDesk, eFuneral, Skritter, and iGuiders.
In 2000, Charles won the Infoworld E-Business Innovator Award for his contributions to Internet Retail along with Tim-Berners-Lee (WWW), Dr. James Gosling (Java), Jeff Hawkins (Palm Pilot), Richard Stallman (GNU), Times Howes (LDAP), and Phil Zimmerman (PGP).
Charles won the Ohio Venture Association (OVA) Venture of the Year award in 2007 and served on the OVA board until 2010. He has mentored and/or been a judge at numerous business plan and startup competitions including the recent Startup Weekend Cleveland and the 11 regional University iDeaLabs competition. He is a very frequent speaker at investment, startup, and entrepreneurship conferences as well as regularly sharing his experiences with classes of students at nearly every university in the region.
In 2013 Stack co-founded Flashstarts – a startup accelerator and micro-venture fund. In 2015 Stack co-founded 35,000 sq.ft. StartMart – one of the largest co-working facilities in the world.
Charles Stack
CEO
Contact Us
Questions? Contact us.


| Metric | Tag | Overall | Per Function |
| Lines of Code | LOC | 0 | ****** |
| McCabe's Cyclomatic Number | MVG | 0 | ****** |
| Lines of Comment | COM | 0 | ******** |
| LOC/COM | L_C | ------ | |
| MVG/COM | M_C | ------ | |
| Weighted Methods per Class ( weighting = unity ) | WMC1 | 0 | |
| Weighted Methods per Class ( weighting = visible ) | WMCv | 0 | |
| Depth of Inheritance Tree | DIT | 0 | |
| Number of Children | NOC | 0 | |
| Coupling between objects | CBO | 14 | |
| Information Flow measure ( inclusive ) | IF4 | 0 | ******** |
| Information Flow measure ( visible ) | IF4v | 0 | ******** |
| Information Flow measure ( concrete ) | IF4c | 0 | ******** |


Free And Open Source Software In Europe Policies And Implementations, Gijs Hillenius
This talk will give an overview of the use of free and open source software solutions by public administrations in the European Union. It will outline the major trends and give plenty of examples of public administrations that jointly develop, share and re-use such software solutions.
Gijs Hillenius is news editor for the European Commission's Open Source Observatory & Repository (OSOR.eu), part of Joinup.ec.europa.eu
OSOR and Joinup are projects by the European Commission on the sharing and reuse of software, by public administrations across Europe.


Preparation time
less than 30 mins
Cooking time
10 to 30 mins
Serves
Makes 6 scotch eggs
We cooked these in Manchester as our answer to the famous Manchester egg. We had the idea to do this when in bars in Scotland, drinkers would take a pickled egg, mash it into a bag of cheese and onion crisps and wash this mess down with a pint… terrible as it sounds we have turned it into culinary keeper.
Take a pickled egg and roll it in celery salt, then seasoned flour.
Divide the sausagemeat into six portions. Roll each portion into a patty big enough to wrap around an egg. Wrap the sausagemeat around the eggs and roll into a ball shape.
Mix the breadcrumbs with the crisps. Dust the sausage-coated egg in flour, dip in beaten egg and then roll in the breadcrumb mixture.
Heat the oil in a deep-fat fryer or a deep-sided, heavy-bottomed pan to 160C, or until a breadcrumb sizzles and turns golden-brown when dropped into it. (CAUTION: hot oil can be dangerous. Do not over-fill the pan or leave unattended).
Fry the eggs for about 8-10 minutes until golden-brown. Alternatively, shallow fry, turning occasionally, until cooked through.


Keep up to date with our latest news and special offers.
© 2020, Almayas Website. Made with passion
All Rights Reserved.


Memory leak test
Enable memory profiling in the browser, then run one of these tests:
Component
Backbone
Backbone.Picky
Backbone.Select
Backbone.Cycle
Repetition
models per set
sets generated *
collections generated *
* reference released immediately, only one instance kept around
Collection behaviour
initial selection
none
first
call close before releasing the reference
Run script for profiling
Script output


Compose
Compose
Settings
Logout
Folders
Inbox
Sent
Drafts
Contact list
Attached files
file.pdf
song.mp3
document.doc
Add file
Delete
Contact list
Add to recipients
Compose
{% csrf_token %}
To:
CC:
Subject:


This chapter describes the Chez Scheme extensions to the syntax-case syntactic abstraction mechanism now standardized in the Revised6 Report. These extensions include the module system (Section 11.5), meta definitions (Section 11.8), conditional expansion (Section 11.9) syntax-rules fenders, fluid-let-syntax, and include.
Keyword bindings established via the Revised6 Report define-syntax, let-syntax, or letrec-syntax forms may be rebound temporarily with fluid-let-syntax.
syntax: (fluid-let-syntax ((keyword expr) ...) form1 form2 ...)
returns: see explanation
libraries: (chezscheme)
Each expr must evaluate to a transformer. fluid-let-syntax is similar to the standard let-syntax, except that instead of introducing new bindings for the keywords keyword ..., fluid-let-syntax temporarily alters the existing bindings for the keywords during the expansion of its body. That is, during the expansion of form1 form2 ..., the visible lexical (or top-level) binding for each keyword is temporarily replaced by a new association between the keyword and the corresponding transformer. This affects any references to the keyword that resolve to the same lexical (or top-level) binding whether the references occur in the text of the body or are introduced during its expansion. In contrast, let-syntax captures only those references that occur within the text of its body.
The following example shows how fluid-let-syntax differs from let-syntax.
(let ([f (lambda (x) (+ x 1))])
(let-syntax ([g (syntax-rules ()
[(_ x) (f x)])])
(let-syntax ([f (syntax-rules ()
[(_ x) x])])
(g 1)))) 2
(let ([f (lambda (x) (+ x 1))])
(let-syntax ([g (syntax-rules ()
[(_ x) (f x)])])
(fluid-let-syntax ([f (syntax-rules ()
[(_ x) x])])
(g 1)))) 1
The two expressions are identical except that the inner let-syntax form in the first expression is a fluid-let-syntax form in the second. In the first expression, the f occurring in the expansion of (g 1) refers to the let-bound variable f, whereas in the second it refers to the keyword f by virtue of the fluid syntax binding for f.
The following code employs fluid-let-syntax in the definition of a define-integrable form that is similar to define for procedure definitions except that it causes the code for the procedure to be integrated, or inserted, wherever a direct call to the procedure is found. No semantic difference is visible between procedures defined with define-integrable and those defined with define, except that a top-level define-integrable form must appear before the first reference to the defined identifier. Lexical scoping is preserved, the actual parameters in an integrated call are evaluated once and at the proper time, integrable procedures may be used as first-class values, and recursive procedures do not cause indefinite recursive expansion.
(define-syntax define-integrable
(syntax-rules (lambda)
[(_ name (lambda formals form1 form2 ...))
(begin
(define xname
(fluid-let-syntax ([name (identifier-syntax xname)])
(lambda formals form1 form2 ...)))
(define-syntax name
(lambda (x)
(syntax-case x ()
[_ (identifier? x) #'xname]
[(_ arg (... ...))
#'((fluid-let-syntax ([name (identifier-syntax xname)])
(lambda formals form1 form2 ...))
arg
(... ...))]))))]))
A define-integrable has the following form.
(define-integrable name lambda-expression)
A define-integrable form expands into a pair of definitions: a syntax definition of name and a variable definition of xname. The transformer for name converts apparent calls to name into direct calls to lambda-expression. Since the resulting forms are merely direct lambda applications (the equivalent of let expressions), the actual parameters are evaluated exactly once and before evaluation of the procedure's body, as required. All other references to name are replaced with references to xname. The definition of xname binds it to the value of lambda-expression. This allows the procedure to be used as a first-class value. Because xname is introduced by the transformer, the binding for xname is not visible anywhere except where references to it are introduced by the transformer for name.
Within lambda-expression, wherever it appears, name is rebound to a transformer that expands all references into references to xname. The use of fluid-let-syntax for this purpose prevents indefinite expansion from indirect recursion among integrable procedures. This allows the procedure to be recursive without causing indefinite expansion. Nothing special is done by define-integrable to maintain lexical scoping, since lexical scoping is maintained automatically by the expander.
Chez Scheme integrate locally defined procedures automatically when it is appropriate to do so. It cannot integrate procedures defined at top-level, however, since code that assigns top-level variables can be introduced into the system (via eval or load) at any time. define-integrable can be used to force the integration of procedures bound at top-level, even if the integration of locally bound procedures is left to the compiler. It can also be used to force the integration of large procedures that the compiler would not normally integrate. (The expand/optimize procedure is useful for determining when integration does or does not take place.)
Chez Scheme extends syntax-rules to permit clause to include fenders just like those allowed within syntax-case clauses.
syntax: (syntax-rules (literal ...) clause ...)
returns: a transformer
libraries: (chezscheme)
Each literal must be an identifier other than an underscore ( _ ) or ellipsis ( ... ). Each clause must take the form below.
(pattern template)
(pattern fender template)
The first form is the only form supported by the Revised6 Report.
Chez Scheme provides several procedures and syntactic forms that may be used to simplify the coding of certain syntactic abstractions.
procedure: (syntax->list syntax-object)
returns: a list of syntax objects
libraries: (chezscheme)
This procedure takes a syntax object representing a list-structured form and returns a list of syntax objects, each representing the corresponding subform of the input form.
syntax->list may be defined as follows.
(define syntax->list
(lambda (ls)
(syntax-case ls ()
[() '()]
[(x . r) (cons #'x (syntax->list #'r))])))
#'(a b c) #<syntax (a b c)>
(syntax->list #'(a b c)) (#<syntax a> #<syntax b> #<syntax c>)
syntax->list is not required for list structures constructed from individual pattern variable values or sequences of pattern-variable values, since such structures are already lists. For example:
(list? (with-syntax ([x #'a] [y #'b] [z #'c]) #'(x y z)))) #t
(list? (with-syntax ([(x ...) #'(a b c)]) #'(x ...))) #t
procedure: (syntax->vector syntax-object)
returns: a list of syntax objects
libraries: (chezscheme)
This procedure takes a syntax object representing a vector-structured form and returns a list of syntax objects, each representing the corresponding subform of the input form.
syntax->vector may be defined as follows.
(define syntax->vector
(lambda (v)
(syntax-case v ()
[#(x ...) (apply vector (syntax->list #'(x ...)))])))
#'#(a b c) #<syntax #(a b c)>
(syntax->vector #'#(a b c)) #(#<syntax a> #<syntax b> #<syntax c>)
syntax->vector is not required for vector structures constructed from individual pattern variable values or sequences of pattern-variable values, since such structures are already vectors. For example:
(vector? (with-syntax ([x #'a] [y #'b] [z #'c]) #'#(x y z)))) #t
(vector? (with-syntax ([(x ...) #'(a b c)]) #'#(x ...))) #t
procedure: (syntax-object->datum obj)
returns: obj stripped of syntactic information
libraries: (chezscheme)
syntax-object->datum is identical to the Revised6 Report syntax->datum.
syntax: (datum template)
returns: see below
libraries: (chezscheme)
(datum template) is a convenient shorthand syntax for
(syntax->datum (syntax template))
datum may be defined simply as follows.
(define-syntax datum
(syntax-rules ()
[(_ t) (syntax->datum #'t)]))
(with-syntax ((a #'(a b c))) (datum a)) (a b c)
procedure: (datum->syntax-object template-identifier obj)
returns: a syntax object
libraries: (chezscheme)
datum->syntax-object is identical to the Revised6 Report datum->syntax.
syntax: (with-implicit (id0 id1 ...) body1 body2 ...)
returns: see below
libraries: (chezscheme)
This form abstracts over the common usage of datum->syntax for creating implicit identifiers (see above). The form
(with-implicit (id0 id1 ...)
body1 body2 ...)
is equivalent to
(with-syntax ([id1 (datum->syntax #'id0 'id1)] ...)
body1 body2 ...)
with-implicit can be defined simply as follows.
(define-syntax with-implicit
(syntax-rules ()
[(_ (tid id ...) b1 b2 ...)
(with-syntax ([id (datum->syntax #'tid 'id)] ...)
b1 b2 ...)]))
We can use with-implicit to simplify the (correct version of) loop above.
(define-syntax loop
(lambda (x)
(syntax-case x ()
[(k e ...)
(with-implicit (k break)
#'(call-with-current-continuation
(lambda (break)
(let f () e ... (f)))))])))
syntax: (include path)
returns: unspecified
libraries: (chezscheme)
path must be a string. include expands into a begin expression containing the forms found in the file named by path. For example, if the file f-def.ss contains (define f (lambda () x)), the expression
(let ([x "okay"])
(include "f-def.ss")
(f))
evaluates to "okay". An include form is treated as a definition if it appears within a sequence of definitions and the forms on the file named by path are all definitions, as in the above example. If the file contains expressions instead, the include form is treated as an expression.
include may be defined portably as follows, although Chez Scheme uses an implementation-dependent definition that allows it to capture and maintain source information for included code.
(define-syntax include
(lambda (x)
(define read-file
(lambda (fn k)
(let ([p (open-input-file fn)])
(let f ([x (read p)])
(if (eof-object? x)
(begin (close-input-port p) '())
(cons (datum->syntax k x)
(f (read p))))))))
(syntax-case x ()
[(k filename)
(let ([fn (datum filename)])
(with-syntax ([(exp ...) (read-file fn #'k)])
#'(begin exp ...)))])))
The definition of include uses datum->syntax to convert the objects read from the file into syntax objects in the proper lexical context, so that identifier references and definitions within those expressions are scoped where the include form appears.
In Chez Scheme's implementation of include, the parameter source-directories (Section 12.5) determines the set of directories searched for source files not identified by absolute path names.
procedure: (syntax-error obj string ...)
returns: does not return
libraries: (chezscheme)
Syntax errors may be reported with syntax-error, which produces a message by concatenating string ... and a printed representation of obj. If no string arguments are provided, the string "invalid syntax" is used instead. When obj is a syntax object, the syntax-object wrapper is stripped (as with syntax->datum) before the printed representation is created. If source file information is present in the syntax-object wrapper, syntax-error incorporates this information into the error message.
syntax-case and syntax-rules call syntax-error automatically if the input fails to match one of the clauses.
We can use syntax-error to precisely report the cause of the errors detected in the following definition of (unnamed) let.
(define-syntax let
(lambda (x)
(define check-ids!
(lambda (ls)
(unless (null? ls)
(unless (identifier? (car ls))
(syntax-error (car ls) "let cannot bind non-identifier"))
(check-ids! (cdr ls)))))
(define check-unique!
(lambda (ls)
(unless (null? ls)
(let ([x (car ls)])
(when (let mem? ([ls (cdr ls)])
(and (not (null? ls))
(or (bound-identifier=? x (car ls))
(mem? (cdr ls)))))
(syntax-error x "let cannot bind two occurrences of")))
(check-unique! (cdr ls)))))
(syntax-case x ()
[(_ ((i e) ...) b1 b2 ...)
(begin
(check-ids! #'(i ...))
(check-unique! #'(i ...))
#'((lambda (i ...) b1 b2 ...) e ...))])))
With this change, the expression
(let ([a 3] [a 4]) (+ a a))
produces the error message "let cannot bind two occurrences of a."
procedure: (literal-identifier=? identifier1 identifier2)
returns: see below
libraries: (chezscheme)
This procedure is identical to the Revised6 Report free-identifier=?, and is provided for backward compatibility only.
When defining sets of dependent macros, it is often convenient to attach information to identifiers in the same compile time environment that the expander uses to record information about variables, keywords, module names, etc. For example, a record-type definition macro, like define-record-type, might need to attach information to the record-type name in the compile-time environment for use in handling child record-type definitions.
Chez Scheme provides two mechanisms for attaching information to identifiers in the compile-time environment: compile-time values and compile-time properties. A compile-time value is a kind of transformer that can be associated with an identifier via define-syntax, let-syntax, letrec-syntax, and fluid-let-syntax. When an identifier is associated with a compile-time value, it cannot also have any other meaning, and an attempt to reference it as an ordinary identifier results in a syntax error. A compile-time property, on the other hand, is maintained alongside an existing binding, providing additional information about the binding. Properties are ignored when ordinary references to an identifier occur.
The mechanisms used by a macro to obtain compile-time values and properties are similar. In both cases, the macro's transformer returns a procedure p rather than a syntax object. The expander invokes p with one argument, an environment-lookup procedure lookup, which p can then use to obtain compile-time values and properties for one or more identifiers before it constructs the macro's final output. lookup accepts one or two identifier arguments. With one argument, id, lookup returns the compile-time value of id, or #f if id has no compile-time value. With two arguments, id and key, lookup returns the value of id's key property, or #f if id has no key property.
procedure: (make-compile-time-value obj)
returns: a compile-time value
libraries: (chezscheme)
A compile time value is a kind of transformer with which a keyword may be associated by any of the keyword binding constructs, e.g., define-syntax or let-syntax. The transformer encapsulates the supplied obj. The encapsulated object may be retrieved as described above.
The following example illustrates how this feature might be used to define a simple syntactic record-definition mechanism where the record type descriptor is generated at expansion time.
(define-syntax drt
(lambda (x)
(define construct-name
(lambda (template-identifier . args)
(datum->syntax template-identifier
(string->symbol
(apply string-append
(map (lambda (x)
(if (string?  r) #t
(prec-x r) 1
(crec-z r) 3
prec exception: invalid syntax prec
syntax: (define-property id key expr)
returns: unspecified
libraries: (chezscheme)
A define-property form attaches a property to an existing identifier binding without disturbing the existing meaning of the identifier in the scope of that binding. It is typically used by one macro to record information about a binding for use by another macro. Both id and key must be identifiers. The expression expr is evaluated when the define-property form is expanded, and a new property associating key with the value of expr is attached to the existing binding of id, which must have a visible local or top-level binding.
define-property is a definition and can appear anywhere other definitions can appear. The scope of a property introduced by define-property is the entire body in which the define-property form appears or global if it appears at top level, except where it is replaced by a property for the same id and key or where the binding to which it is attached is shadowed. Any number of properties can be attached to the same binding with different keys. Attaching a new property with the same name as an property already attached to a binding shadows the existing property with the new property.
The following example defines a macro, get-info, that retrieves the info property of a binding, defines the variable x, attaches an info property to the binding of x, retrieves the property via get-info, references x to show that its normal binding is still intact, and uses get-info again within the scope of a different binding of x to show that the properties are shadowed as well as the outer binding of x.
(define info)
(define-syntax get-info
(lambda (x)
(lambda (lookup)
(syntax-case x ()
[(_ q)
(let ([info-value (lookup #'q #'info)])
#`'#,(datum->syntax #'* info-value))]))))
(define x "x-value")
(define-property x info "x-info")
(get-info x) "x-info"
x "x-value"
(let ([x "inner-x-value"]) (get-info x)) #f
For debugging, it is often useful to have a form that retrieves an arbitrary property, given an identifier and a key. The get-property macro below does just that.
(define-syntax get-property
(lambda (x)
(lambda (r)
(syntax-case x ()
[(_ id key)
#`'#,(datum->syntax #'* (r #'id #'key))]))))
(get-property x info) "x-info"
The bindings for both identifiers must be visible where get-property is used.
The version of drt defined below is like the one defined using make-compile-time-value above, except that it defines the record name as a macro that raises an exception with a more descriptive message, while attaching the record type descriptor to the binding as a separate property. The variable drt-key defined along with drt is used only as the key for the property that drt attaches to a record name. Both drt-key and drt are defined within a module that exports only the latter, ensuring that the properties used by drt cannot be accessed or forged.
(library (drt) (export drt) (import (chezscheme))
(define drt-key)
(define-syntax drt
(lambda (x)
(define construct-name
(lambda (template-identifier . args)
(datum->syntax template-identifier
(string->symbol
(apply string-append
(map (lambda (x)
(if (string?  r) #t
(prec-x r) 1
(crec-z r) 3
prec exception: invalid use of record name prec
Modules are used to help organize programs into separate parts that interact cleanly via declared interfaces. Although modular programming is typically used to facilitate the development of large programs possibly written by many individuals, it may also be used in Chez Scheme at a "micro-modular" level, since Chez Scheme module and import forms are definitions and may appear anywhere any other kind of definition may appear, including within a lambda body or other local scope.
Modules control visibility of bindings and can be viewed as extending lexical scoping to allow more precise control over where bindings are or are not visible. Modules export identifier bindings, i.e., variable bindings, keyword bindings, or module name bindings. Modules may be named or anonymous. Bindings exported from a named module may be made visible via an import form wherever the module's name is visible. Bindings exported from an anonymous module are implicitly imported where the module form appears. Anonymous modules are useful for hiding some of a set of bindings while allowing the remaining bindings in the set to be visible.
Some of the text and examples given in this section are adapted from the paper "Extending the scope of syntactic abstraction" [31], which describes modules and their implementation in more detail.
syntax: (module name interface defn ... init ...)
syntax: (module interface defn ... init ...)
returns: unspecified
libraries: (chezscheme)
name is an identifier, defn ... are definitions, and init ... are expressions. interface is a list of exports (export ...), where each export is either an identifier identifier or of the form (identifier export ...).
The first syntax for module establishes a named scope that encapsulates a set of identifier bindings. The exported bindings may be made visible via import or import-only (Section 10.4) anywhere the module name is visible. The second syntax for module introduces an anonymous module whose bindings are implicitly imported (as if by import of a hidden module name) where the module form appears.
A module consists of a (possibly empty) set of definitions and a (possibly empty) sequence of initialization expressions. The identifiers defined within a module are visible within the body of the module and, if exported, within the scope of an import for the module. Each identifier listed in a module's interface must be defined within or imported into that module. A module form is a definition and can appear anywhere other definitions can appear, including at the top level of a program, nested within the bodies of lambda expressions, nested within library and top-level program forms, and nested within other modules. Also, because module names are scoped like other identifiers, modules and libraries may export module names as well as variables and keywords.
When an interface contains an export of the form (identifier export ...), only identifier is visible in the importing context. The identifiers within export ... are indirect imports, as if declared via an indirect-export form (Section 10.4).
Module names occupy the same namespace as other identifiers and follow the same scoping rules. Unless exported, identifiers defined within a module are visible only within that module.
Expressions within a module can reference identifiers bound outside of the module.
(let ([x 3])
(module m (plusx)
(define plusx (lambda (y) (+ x y))))
(import m)
(let ([x 4])
(plusx 5))) 8
Similarly, import does not prevent access to identifiers that are visible where the import form appears, except for those variables shadowed by the imported identifiers.
(module m (y) (define y 'm-y))
(let ([x 'local-x] [y 'local-y])
(import m)
(list x y)) (local-x m-y)
On the other hand, use of import-only within a module establishes an isolated scope in which the only visible identifiers are those exported by the imported module.
(module m (y) (define y 'm-y))
(let ([x 'local-x] [y 'local-y])
(import-only m)
x) Error: x is not visible
This is sometimes desirable for static verification that no identifiers are used except those explicitly imported into a module or local scope.
Unless a module imported via import-only exports import or import-only and the name of at least one module, subsequent imports within the scope of the import-only form are not possible. To create an isolated scope containing the exports of more than one module without making import or import-only visible, all of the modules to be imported must be listed in the same import-only form.
Another solution is to create a single module that contains the exports of each of the other modules.
(module m2 (y) (define y 'y))
(module m1 (x) (define x 'x))
(module mega-module (cons x y)
(import m1)
(import m2)
(import scheme))
(let ([y 3])
(import-only mega-module)
(cons x y)) (x . y)
Before it is compiled, a source program is translated into a core language program containing no syntactic abstractions, syntactic definitions, library definitions, module definitions, or import forms. Translation is performed by a syntax expander that processes the forms in the source program via recursive descent.
A define-syntax form associates a keyword with a transformer in a translation-time environment. When the expander encounters a keyword, it invokes the associated transformer and reprocesses the resulting form. A module form associates a module name with an interface. When the expander encounters an import form, it extracts the corresponding module interface from the translation-time environment and makes the exported bindings visible in the scope where the import form appears.
Internal definitions and definitions within a module body are processed from left to right so that a module's definition and import may appear within the same sequence of definitions. Expressions appearing within a body and the right-hand sides of variable definitions, however, are translated only after the entire set of definitions has been processed, allowing full mutual recursion among variable and syntactic definitions.
Module and import forms affect only the visibility of identifiers in the source program, not their meanings. In particular, variables are bound to locations whether defined within or outside of a module, and import does not introduce new locations. Local variables are renamed as necessary to preserve the scoping relationships established by both modules and syntactic abstractions. Thus, the expression:
(let ([x 1])
(module m (x setter)
(define-syntax x (identifier-syntax z))
(define setter (lambda (x) (set! z x)))
(define z 5))
(let ([y x] [z 0])
(import m)
(setter 3)
(+ x y z))) 4
is equivalent to the following program in which identifiers have been consistently renamed as indicated by subscripts.
(let ([x0 1])
(define-syntax x1 (identifier-syntax z1))
(define setter1 (lambda (x2) (set! z1 x2)))
(define z1 5)
(let ([y3 x0] [z3 0])
(setter1 3)
(+ x1 y3 z3)))
Definitions within a top-level begin, lambda, top-level program, library, or module body are processed from left to right by the expander at expand time, and the variable definitions are evaluated from left-to-right at run time. Initialization expressions appearing within a module body are evaluated in sequence after the evaluation of the variable definitions.
Mutually recursive modules can be defined in several ways. In the following program, a and b are mutually recursive modules exported by an anonymous module whose local scope is used to statically link the two. For example, the free variable y within module a refers to the binding for y, provided by importing b, in the enclosing module.
(module (a b)
(module a (x) (define x (lambda () y)))
(module b (y) (define y (lambda () x)))
(import a)
(import b))
The following syntactic abstraction generalizes this pattern to permit the definition of multiple mutually recursive modules.
(define-syntax rec-modules
(syntax-rules (module)
[(_ (module m (id ...) form ...) ...)
(module (m ...)
(module m (id ...) form ...) ...
(import m) ...)]))
Because a module can re-export imported bindings, it is quite easy to provide multiple views on a single module, as s and t provide for r below, or to combine several modules into a compound, as r does.
(module p (x y)
(define x 1) (define y 2))
(module q (y z)
(define y 3) (define z 4))
(module r (a b c d)
(import* p (a x) (b y))
(import* q (c y) (d z)))
(module s (a c) (import r))
(module t (b d) (import r))
To allow interfaces to be separated from implementations, the following syntactic abstractions support the definition and use of named interfaces.
(define-syntax define-interface
(syntax-rules ()
[(_ name (export ...))
(define-syntax name
(lambda (x)
(syntax-case x ()
[(_ n defs)
(with-implicit (n export ...)
#'(module n (export ...) .
defs))])))]))
(define-syntax define-module
(syntax-rules ()
[(_ name interface defn ...)
(interface name (defn ...))]))
define-interface creates an interface macro that, given a module name and a list of definitions, expands into a module definition with a concrete interface.
with-implicit is used to ensure that the introduced export identifiers are visible in the same scope as the name of the module in the define-module form.
define-interface and define-module can be used as follows.
(define-interface simple (a b))
(define-module m simple
(define-syntax a (identifier-syntax 1))
(define b (lambda () c))
(define c 2))
(let () (import m) (+ a (b))) 3
The abstract module facility defined below allows a module interface to be satisfied incrementally when module forms are evaluated. This permits flexibility in the separation between the interface and implementation, supports separate compilation of mutually recursive modules, and permits redefinition of module implementations.
(define-syntax abstract-module
(syntax-rules ()
[(_ name (ex ...) (kwd ...) defn ...)
(module name (ex ... kwd ...)
(declare ex) ...
defn ...)]))
(define-syntax implement
(syntax-rules ()
[(_ name form ...)
(module () (import name) form ...)]))
Within an abstract-module form, each of the exports in the list ex ... must be variables. The values of these variables are supplied by one or more separate implement forms. Since keyword bindings must be present at compile time, they cannot be satisfied incrementally and are instead listed as separate exports and defined within the abstract module.
Within an implement form, the sequence of forms form ... is a sequence of zero or more definitions followed by a sequence of zero or more expressions. Since the module used in the expansion of implement does not export anything, the definitions are all local to the implement form. The expressions may be arbitrary expressions, but should include one satisfy form for each variable whose definition is supplied by the implement form. A satisfy form has the syntax
(satisfy variable expr)
declare and satisfy may simply be the equivalents of define and set!.
(define-syntax declare (identifier-syntax define))
(define-syntax satisfy (identifier-syntax set!))
Alternatively, declare can initialize the declared variable to the value of a flag known only to declare and satisfy, and satisfy can verify that this flag is still present to insure that only one attempt to satisfy the value of a given identifier is made.
(module ((declare cookie) (satisfy cookie))
(define cookie "chocolate chip")
(define-syntax declare
(syntax-rules () [(_ var) (define var cookie)]))
(define-syntax satisfy
(syntax-rules ()
[(_ var exp)
(if (eq? var cookie)
(set! var exp)
(assertion-violationf 'satisfy
"value of variable ~s has already been satisfied"
'var))])))
Using abstract-module and implement, we can define mutually recursive and separately compilable modules as follows.
(abstract-module e (even?) (pred)
(define-syntax pred
(syntax-rules () [(_ exp) (- exp 1)])))
(abstract-module o (odd?) ())
(implement e
(import o)
(satisfy even?
(lambda (x)
(or (zero? x) (odd? (pred x))))))
(implement o
(import e)
(satisfy odd?
(lambda (x) (not (even? x)))))
(let () (import-only e) (even? 38)) #t
syntax: only
syntax: except
syntax: add-prefix
syntax: drop-prefix
syntax: rename
syntax: alias
libraries: (chezscheme)
These identifiers are auxiliary keywords for import and import-only. It is a syntax violation to reference these identifiers except in contexts where they are recognized as auxiliary keywords.
The local import and export forms described in Section 10.4 can be used equally well for and within modules.
Five modules are built-in to Chez Scheme: scheme, r5rs, r5rs-syntax, ieee, and $system. Each module is immutable, i.e., the exported bindings cannot be altered.
module: scheme
libraries: (chezscheme)
scheme contains all user-visible top-level bindings (variables, keywords, and module names) built into Chez Scheme.
module: r5rs
libraries: (chezscheme)
r5rs contains all top-level bindings (variables and keywords) defined in the Revised5 Report on Scheme. The bindings exported from r5rs are precisely those that are available within an expression evaluated via eval with the environment specifier returned by scheme-report-environment.
module: r5rs-syntax
libraries: (chezscheme)
r5rs-syntax contains all top-level keyword bindings defined in the Revised5 Report on Scheme. The bindings exported from r5rs-syntax are precisely those that are available within an expression evaluated via eval with the environment specifier returned by null-environment.
module: ieee
libraries: (chezscheme)
ieee contains all top-level bindings (variables and keywords) defined in the ANSI/IEEE standard for Scheme. The bindings exported from ieee are precisely those that are available within an expression evaluated via eval with the environment specifier returned by ieee-environment.
module: $system
libraries: (chezscheme)
$system contains all user-visible top-level bindings built into Chez Scheme along with various undocumented system bindings.
syntax: (meta . definition)
returns: unspecified
libraries: (chezscheme)
The meta keyword is actually a prefix that can be placed in front of any definition keyword, e.g.,
(meta define x 3)
It tells the expander that any variable definition resulting from the definition is to be an expand-time definition available only to the right-hand sides of other meta definitions and, most importantly, transformer expressions. It is used to define expand-time helpers and other information for use by one or more syntax-case transformers.
(module M (helper1 a b)
(meta define helper1
(lambda (---)
---))
(meta define helper2
(lambda (---)
--- (helper2 ---) ---))
(define-syntax a
(lambda (x)
--- (helper1 ---) ---))
(define-syntax b
(lambda (x)
--- (helper1 ---) ---
--- (helper2 ---) ---)))
The right-hand-side expressions of a syntax definition or meta definition can refer only to identifiers whose values are already available in the compile-time environment. Because of the left-to-right expansion order for library, module, lambda, and similar bodies, this implies a semantics similar to let* for a sequence of meta definitions, in which each right-hand side can refer only to the variables defined earlier in the sequence. An exception is that the right-hand side of a meta definition can refer to its own name as long as the reference is not evaluated until after the value of the expression has been computed. This permits meta definitions to be self-recursive but not mutually recursive. The right-hand side of a meta definition can, however, build syntax objects containing occurrences of any identifiers defined in the body in which the meta definition appears.
Meta definitions propagate through macro expansion, so one can write, for example:
(module (a)
(meta define-record foo (x))
(define-syntax a
(let ([q (make-foo #''q)])
(lambda (x) (foo-x q)))))
a q
where define-record is a macro that expands into a set of defines.
It is also sometimes convenient to write
(meta begin defn ...)
or
(meta module {exports} defn ...)
or
(meta include "path")
to create groups of meta bindings.
Expansion-time decisions can be made via meta-cond, which is similar to cond but evaluates the test expressions at expansion time and can be used in contexts where definitions are expected as well as in expression contexts.
syntax: (meta-cond clause1 clause2 ...)
returns: see below
libraries: (chezscheme)
Each clause but the last must take the form:
(test expr1 expr2 ...)
The last may take the same form or be an else clause of the form:
(else expr1 expr2 ...)
During expansion, the test expressions are evaluated in order until one evaluates to a true value or until all of the tests have been evaluated. If a test evaluates to a true value, the meta-cond form expands to a begin form containing the corresponding expressions expr1 expr2 .... If no test evaluates to a true value and an else clause is present, the meta-cond form expands to a begin form containing the expressions expr1 expr2 ... from the else clause. Otherwise the meta-cond expression expands into a call to the void procedure.
meta-cond might be defined as follows.
(define-syntax meta-cond
(syntax-rules ()
[(_ [a0 a1 a2 ...] [b0 b1 b2 ...] ...)
(let-syntax ([expr (cond
[a0 (identifier-syntax (begin a1 a2 ...))]
[b0 (identifier-syntax (begin b1 b2 ...))]
...)])
expr)]))
meta-cond is used to choose, at expansion time, from among a set of possible forms. For example, one might have safe (error-checking) and unsafe (non-error-checking) versions of a procedure and decide which to call based on the compile-time optimization level, as shown below.
(meta-cond
[(= (optimize-level) 3) (unsafe-frob x)]
[else (safe-frob x)])
syntax: (alias id1 id2)
returns: unspecified
libraries: (chezscheme)
alias is a definition and can appear anywhere other definitions can appear. It is used to transfer the binding from one identifier to another.
(let ([x 3]) (alias y x) (set! y 4) (list x y)) (4 4)
(module lisp (if)
(module (scheme:if)
(import scheme)
(alias scheme:if if))
(define-syntax if
(syntax-rules ()
[(_ e_1 e_2 e_3)
(scheme:if (not (memq e_1 '(#f ()))) e_2 e_3)])))
(define (length ls)
(import lisp)
(if ls (+ (length (cdr ls)) 1) 0))
(length '(a b c)) 3
Because of left-to-right expansion order, aliases should appear after the definition of the right-hand-side identifier, e.g.:
(let ()
(import-only (chezscheme))
(define y 3)
(alias x y)
x) 3
rather than:
(let ()
(import-only (chezscheme))
(alias x y)
(define y 3)
x) exception: unbound identifier
When source code is read from a file by load, compile-file, or variants of these, such as load-library, the reader attaches annotations to each object read from the file. These annotations identify the file and the position of the object within the file. Annotations are tracked through the compilation process and associated with compiled code at run time. The expander and compiler use the annotations to produce syntax errors and compiler warnings that identify the location of the offending form, and the inspector uses them to identify the locations of calls and procedure definitions. The compiler and run time also use annotations to associate source positions with profile counts.
While these annotations are usually maintained "behind the scenes," the programmer can manipulate them directly via a set of routines for creating and accessing annotations.
Annotations are values of a type distinct from other types and have four components: an expression, possibly with annotated subexpressions, a source object, a stripped version of the expression, and usage options. Annotations can be created via make-annotation, which has three required arguments corresponding to the first three components and an optional fourth argument corresponding to the fourth component. The second argument must be a source object, and the third argument should be a stripped version of the first argument, i.e., equivalent to the first argument with each annotation replaced by its expression component. An annotation is essentially equivalent to its stripped component as a representation of source code, with the source information attached and available to the expander or evaluator. The optional fourth argument, if present, must be an enumeration set over the symbols debug and profile and defaults to an enumeration set containing both debug and profile.
Annotations marked debug are used for compile-time error reporting and run-time error reporting and inspection; annotations marked profile are used for profiling. Annotations created by the Scheme reader are always marked both debug and profile, but other readers and parsers might choose to mark some annotations only debug or only profile. In particular, it might be useful to annotate multiple expressions in the output of a parser with the same source object for debugging purposes and mark only one of them profile to avoid duplicate counts. It might also be useful to mark no expressions profile and instead introduce explicit profile forms (Section 12.7) to identify the set of source locations to be profiled.
Source objects are also values of a type distinct from other types and also have three components: a source-file descriptor (sfd), a beginning file position (bfp), and an ending file position (efp). The sfd identifies the file from which an expression is read and the bfp identify the range of character positions occupied by the object in the file, with the bfp being inclusive and the efp being exclusive. A source object can be created via make-source-object, which takes three arguments corresponding to these components. The first argument must be a source-file descriptor, the second and third must be nonnegative exact integers, and the second must not be greater than the third.
Source-file descriptors are also values of a type distinct from all other types and have two components: the file's path, represented by a string, and a checksum, represented by a number. The path might or might not be an absolute path depending on how the file's path was specified when the source-file descriptor was created. The checksum is computed based on the file's length and contents when the file is created and checked by tools that look for the source file to make sure that the proper file has been found and has not been modified. Source-file descriptors can be created with make-source-file-descriptor, which accepts two arguments: a string naming the path and a binary input port, along with an optional third boolean argument, reset?, which defaults to false. make-source-file-descriptor computes a checksum based on the contents of the port, starting at its current position. It resets the port, using set-port-position!, after computing the checksum if reset? is true; otherwise, it leaves the port at end-of-file.
The procedures that create, check for, and access annotations, source objects, and source-file descriptors are summarized below and described in more detail later in this section.
(make-annotation obj source-object obj) annotation
(annotation? obj) boolean
(annotation-expression annotation) obj
(annotation-source annotation) source-object
(annotation-stripped annotation) obj
(make-source-object sfd uint uint) source-object
(source-object? obj) boolean
(source-object-bfp source-object) uint
(source-object-efp source-object) uint
(source-object-sfd source-object) sfd
(make-source-file-descriptor string binary-input-port) sfd
(make-source-file-descriptor string binary-input-port reset?) sfd
(source-file-descriptor? obj) boolean
(source-file-descriptor-checksum sfd) obj
(source-file-descriptor-path sfd) obj
A program might open a source file with open-file-input-port, create an sfd using make-source-file-descriptor, create a textual port from the binary port using transcoded-port, and create source objects and annotations for each of the objects it reads from the file. If a custom reader is not required, the Scheme reader can be used to read annotations via the get-datum/annotations procedure:
(get-datum/annotations textual-input-port sfd uint) obj, uint
get-datum/annotations is like get-datum but instead of returning a plain datum, it returns an annotation encapsulating a datum (possibly with nested annotations), a source object, and the plain (stripped) datum. It also returns a second value, the position of the first character beyond the object in the file. Character positions are accepted and returned by get-datum/annotations so that the textual port need not support port-position and need not report positions in characters if it does support port-position. (Positions are usually reported in bytes.) The bfp and efp positions recorded in the annotations returned by get-datum/annotations are correct only if the positions supplied to it are correct.
Once read, an annotation can be passed to the expander, interpreter, or compiler. The procedures eval, expand, interpret, and compile all accept annotated or unannotated input.
Two additional procedures complete the set of annotation-related primitives:
(open-source-file sfd) #f or port
(syntax->annotation obj) #f or annotation
open-source-file attempts to locate and open the source file identified by sfd. It returns a textual input port, positioned at the beginning of the file, if successful, and #f otherwise.
syntax->annotation accepts a syntax object. If the syntax object's expression is annotated, it returns the annotation; otherwise, it returns #f. It can be used by a macro to extract source information, when available, from an input form.
The procedure datum->syntax accepts either an annotated or unannotated input datum.
procedure: (make-annotation obj source-object stripped-obj)
procedure: (make-annotation obj source-object stripped-obj options)
returns: an annotation
libraries: (chezscheme)
The annotation is formed with obj as its expression component, source-object as its source-object component, and stripped-obj as its stripped component. obj should represent an expression, possibly with embedded annotations. stripped-obj should be a stripped version of obj, i.e., equivalent to obj with each annotation replaced by its expression component. options, if present must be an enumeration set over the symbols debug and profile, and defaults to an enumeration set containing both debug and profile. Annotations marked debug are used for compile-time error reporting and run-time error reporting and inspection; annotations marked profile are used for profiling.
procedure: (annotation? obj)
returns: #t if obj is an annotation, otherwise #f
libraries: (chezscheme)
procedure: (annotation-expression annotation)
returns: the expression component of annotation
libraries: (chezscheme)
procedure: (annotation-source annotation)
returns: the source-object component of annotation
libraries: (chezscheme)
procedure: (annotation-stripped annotation)
returns: the stripped component of annotation
libraries: (chezscheme)
procedure: (annotation-options annotation)
returns: the options enumeration set of annotation
libraries: (chezscheme)
procedure: (make-source-object sfd bfp efp)
returns: a source-object
libraries: (chezscheme)
sfd must be a source-file descriptor. bfp and efp must be exact nonnegative integers, and bfp should not be greater than efp.
procedure: (source-object? obj)
returns: #t if obj is a source object, otherwise #f
libraries: (chezscheme)
procedure: (source-object-bfp source-object)
returns: the bfp component of source-object
libraries: (chezscheme)
procedure: (source-object-efp source-object)
returns: the efp component of source-object
libraries: (chezscheme)
procedure: (source-object-sfd source-object)
returns: the sfd component of source-object
libraries: (chezscheme)
procedure: (make-source-file-descriptor string binary-input-port)
procedure: (make-source-file-descriptor string binary-input-port reset?)
returns: a source-file descriptor
libraries: (chezscheme)
To compute the checksum encapsulated in the source-file descriptor, this procedure must read all of the data from binary-input-port. If reset? is present and #t, the port is reset to its original position, as if via port-position. Otherwise, it is left pointing at end-of-file.
procedure: (source-file-descriptor? obj)
returns: #t if obj is a source-file descriptor, otherwise #f
libraries: (chezscheme)
procedure: (source-file-descriptor-checksum sfd)
returns: the checksum component of sfd
libraries: (chezscheme)
procedure: (source-file-descriptor-path sfd)
returns: the path component of sfd
libraries: (chezscheme)
sfd must be a source-file descriptor.
procedure: (source-file-descriptor path checksum)
returns: a new source-file-descriptor
libraries: (chezscheme)
path must be a string, and checksum must be an exact nonnegative integer. This procedure can be used to construct custom source-file descriptors or to reconstitute source-file descriptors from the path and checksum components.
syntax: (annotation-option-set symbol ...)
returns: an annotation-options enumeration set
libraries: (chezscheme)
Annotation-options enumeration sets may be passed to make-annotation to control whether the annotation is used for debugging, profiling, both, or neither. Accordingly, each symbol must be either debug or profile.
procedure: (syntax->annotation obj)
returns: an annotation or #f
libraries: (chezscheme)
If obj is an annotation or syntax-object encapsulating an annotation, the annotation is returned.
procedure: (get-datum/annotations textual-input-port sfd bfp)
returns: see below
libraries: (chezscheme)
sfd must be a source-file descriptor. bfd must be an exact nonnegative integer and should be the character position of the next character to be read from textual-input-port.
This procedure returns two values: an annotated object and an ending file position. In most cases, bfp should be 0 for the first call to get-datum/annotation at the start of a file, and it should be the second return value of the preceding call to get-datum/annotation for each subsequent call. This protocol is necessary to handle files containing multiple-byte characters, since file positions do not necessarily correspond to character positions.
procedure: (open-source-file sfd)
returns: a port or #f
libraries: (chezscheme)
sfd must be a source-file descriptor. This procedure attempts to locate and open the source file identified by sfd. It returns a textual input port, positioned at the beginning of the file, if successful, and #f otherwise. It can fail even if a file with the correct name exists in one of the source directories when the file's checksum does not match the checksum recorded in sfd.
procedure: (locate-source sfd pos)
returns: see below
libraries: (chezscheme)
sfd must be a source-file descriptor, and pos must be an exact nonnegative integer.
This procedure attempts to locate and open the source file identified by sfd. If successful, it returns three values: a string path, an exact nonnegative integer line, and an exact nonnegative integer char representing the absolute pathname, line, and character position within the line represented by the specified source-file descriptor and file position. If unsuccessful, it returns zero values. It can fail even if a file with the correct name exists in one of the source directories when the file's checksum does not match the checksum recorded in sfd.
Chez Scheme Version 9 User's Guide
Copyright © 2016 Cisco Systems, Inc.
Licensed under the Apache License Version 2.0
(full copyright notice.).
Revised June 2016 for Chez Scheme Version 9.4
about this book


Recieve communication from Nexus International?
Yes
No
Title
First Name
Last Name
Suffix
Street
City
State
Postal Code
Phone
Submit
{{AC.updateForm}}


Preload the electricity generation stats by the country:
Albania
Algeria
Angola
Argentina
Armenia
Australia
Austria
Azerbaijan
Bahrain
Bangladesh
Belarus
Belgium
Benin
Bolivia
Bosnia and Herzegovina
Botswana
Brazil
Brunei Darussalam
Bulgaria
Cambodia
Cameroon
Canada
Chile
China
Hong Kong SAR, China
Colombia
Congo, Dem. Rep.
Congo, Rep.
Costa Rica
Cote d'Ivoire
Croatia
Cuba
Curacao
Cyprus
Czech Republic
Denmark
Dominican Republic
Ecuador
Egypt, Arab Rep.
El Salvador
Eritrea
Estonia
Ethiopia
Finland
France
Gabon
Georgia
Germany
Ghana
Greece
Guatemala
Haiti
Honduras
Hungary
Iceland
India
Indonesia
Iran, Islamic Rep.
Iraq
Ireland
Israel
Italy
Jamaica
Japan
Jordan
Kazakhstan
Kenya
Korea, Dem. People’s Rep.
Korea, Rep.
Kosovo
Kuwait
Kyrgyz Republic
Latvia
Lebanon
Libya
Lithuania
Luxembourg
Macedonia, FYR
Malaysia
Malta
Mauritius
Mexico
Moldova
Mongolia
Montenegro
Morocco
Mozambique
Myanmar
Namibia
Nepal
Netherlands
New Zealand
Nicaragua
Niger
Nigeria
Norway
Oman
Pakistan
Panama
Paraguay
Peru
Philippines
Poland
Portugal
Qatar
Romania
Russian Federation
Saudi Arabia
Senegal
Serbia
Singapore
Slovak Republic
Slovenia
South Africa
South Sudan
Spain
Sri Lanka
Sudan
Suriname
Sweden
Switzerland
Syrian Arab Republic
Tajikistan
Tanzania
Thailand
Togo
Trinidad and Tobago
Tunisia
Turkey
Turkmenistan
Ukraine
United Arab
United Kingdom
United States
Uruguay
Uzbekistan
Venezuela, RB
Vietnam
Yemen, Rep.
Zambia
Zimbabwe
Coal
%
gCO
2
/kWh
Natural gas
%
gCO
2
/kWh
Oil
%
gCO
2
/kWh
Hydropower*
%
gCO
2
/kWh
Renewable**
%
gCO
2
/kWh
Nuclear
%
gCO
2
/kWh
Custom
%
gCO
2
/kWh
Total percentage:
How much power do you use (continuously)?
W
kW
MW
GW
TW
You produce
kg of CO
2
emissions per year.
You would need to plant
trees to eliminate your carbon footprint on our planet.
You can donate
$
to
trees.org
to eliminate your carbon footprint.
*gCO
2
can vary widely, read Wiki page in Sources
**using gCO2 values for Solar PV rooftop, read Wiki page in Sources for other values
Sources and notes
Electricity sources:
The World Bank statistics
(Data from 2015)
CO
2
/kWh values: Using median values from IPCC: Global warming potential of selected electricity sources (2014)
CO
2
/kWh values may differ a lot, especially when power plants use newest (better values), or older (worse values) technologies. Check the
Wikipedia page
for Min/Max values.
This calculations assume that you use electricity proportionally from all power plants of your country.
If you know better where your electricity comes from (for example, you use your own solar power), edit the settings accordingly.
One tree can eliminate 15.7 kg of CO
2
per year. Source:
https://www.tfaforms.com/4666774
If you have questions or suggestions about this calculator, contact me at
github.com/MarosMacko
.


Maksim, developer of ClickHouse.
1. ClickHouse query execution.
2. JIT compilation.
3. ClickHouse compilation of expressions.
4. ClickHouse compilation of operator GROUP BY.
5. ClickHouse compilation of ORDER BY comparator.
Column-oriented storage — data is physically stored by columns. Only necessary columns are read from disk during query.
Vectorized query execution — data is processed in blocks. Block contains multiple columns with max_block_size rows (65536 by default).
Each column is stored as a vector of primitive data types or their combination:
1. Better utilization for CPU caches and pipeline.
2. Data is processed using SIMD instructions.
Numeric columns — PODArray. Almost the same as std::vector.
1. Use our Allocator with support of realloc.
2. No additional memset during resize.
3. Padding with 15 bytes at the end.
Nullable columns contain data column and UInt8 column bitmask is element null.
Array columns contain data column and UInt64 column with offsets.
Const column contain 1 constant value.
Takes arguments as columns and returns function result as column.
class IFunction
{
virtual ~IFunction() = default;
virtual ColumnPtr executeImpl(
const ColumnsWithTypeAndName & arguments,
const DataTypePtr & result_type,
size_t input_rows_count) const = 0;
...
}
Specializations using templates for different types. Example plus, multiply for different types combinations.
Specializations for constant columns. Example plus, multiply with constant column.
Function plus has:
UInt8 UInt16 UInt32 UInt64 UInt8 UInt16 UInt32 UInt64
Int8 Int16 Int32 Int64 ✕ Int8 Int16 Int32 Int64
Float32 Float64 Float32 Float64
specialization.
In addition specialization if one of the columns is a constant column
20 x 20 = 400 specializations for single plus function.
Advantages:
1. Code isolation.
2. High efficiency (Necessary specializations can be generated).
3. Compiler can generate SIMD instructions.
Disadvantages:
1. Heavy template usage (Templates can be complex).
2. Binary code bloat.
3. No way to fuse multiple expressions into one. Example: x * y + 5.
1. Parse query into AST.
2. Make AST optimizations (Most of them need to be moved into logical, physical query plans or expression DAG optimizations).
3. Build logical query plan + make logical query plan optimizations.
4. Build physical query plan + make physical query plan optimizations.
5. Execution.
EXPLAIN AST value * 2 + 1 FROM test_table
WHERE value > 10 ORDER BY value;
┌─explain─────────────────────────────────────┐
│ SelectWithUnionQuery (children 1) │
│ ExpressionList (children 1) │
│ SelectQuery (children 4) │
│ ExpressionList (children 1) │
│ Function plus (children 1) │
│ ExpressionList (children 2) │
│ Function multiply (children 1) │
│ ExpressionList (children 2) │
│ Identifier value │
│ Literal UInt64_2 │
│ Literal UInt64_1 │
│ TablesInSelectQuery (children 1) │
│ TablesInSelectQueryElement (children 1) │
│ TableExpression (children 1) │
│ TableIdentifier test_table │
│ Function greater (children 1) │
│ ExpressionList (children 2) │
│ Identifier value │
│ Literal UInt64_10 │
│ ExpressionList (children 1) │
│ OrderByElement (children 1) │
│ Identifier value │
└─────────────────────────────────────────────┘
EXPLAIN SELECT value * 2 + 1 FROM test_table
WHERE value > 10 ORDER BY value;
┌─explain──────────────────────────────────────────────────────────┐
│ Expression ((Projection + Before ORDER BY [lifted up part])) │
│ Sorting (Sorting for ORDER BY) │
│ Expression (Before ORDER BY) │
│ Filter (WHERE) │
│ SettingQuotaAndLimits │
│ ReadFromMergeTree (default.test_table) │
└──────────────────────────────────────────────────────────────────┘
EXPLAIN PIPELINE SELECT value * 2 + 1 FROM test_table
WHERE value > 10 ORDER BY value;
┌─explain────────────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Sorting) │
│ MergingSortedTransform 16 → 1 │
│ MergeSortingTransform × 16 │
│ LimitsCheckingTransform × 16 │
│ PartialSortingTransform × 16 │
│ (Expression) │
│ ExpressionTransform × 16 │
│ (Filter) │
│ FilterTransform × 16 │
│ (SettingQuotaAndLimits) │
│ (ReadFromMergeTree) │
│ MergeTreeThread × 16 0 → 1 │
└────────────────────────────────────────────┘
JIT - Just in time compilation
Generate machine code and execute in runtime.
Examples: JVM Hotspot, V8.
Use LLVM for JIT compilation.
Use our own wrapper around LLVM dynamic linker and compiler.
2018 - First implementation for JIT compilation of expressions. Developer Denis Skorobogatov.
2019 - Improvements in initial implementation. Enable by default in production. Developers Alexey Milovidov, Alexander Sapin.
2019 - Disable JIT by default. Available as an experimental feature.
2021 - Fix issues with JIT compilation of expressions. Enable JIT compilation by default. Developer Maksim Kita.
2021 - Implement JIT compilation for GROUP BY operator.
2022 - Implement JIT compilation for ORDER BY comparator.
EXPLAIN PIPELINE SELECT value * 2 + 1 FROM test_table
WHERE value > 10 ORDER BY value;
┌─explain────────────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Sorting) │
│ MergingSortedTransform 16 → 1 │
│ MergeSortingTransform × 16 │
│ LimitsCheckingTransform × 16 │
│ PartialSortingTransform × 16 │
│ (Expression) │
│ ExpressionTransform × 16 │
│ (Filter) │
│ FilterTransform × 16 │
│ (SettingQuotaAndLimits) │
│ (ReadFromMergeTree) │
│ MergeTreeThread × 16 0 → 1 │
└────────────────────────────────────────────┘
SELECT a + b + c FROM test_table;
Expressions are represented in DAG that has input, function, constant types of node. Example:
1. During DAG interpretation, data is moving between functions. Operations are not fused.
Example plus(plus(a, b), c)) first plus on columns a and b is performed and the result stored in a temporary column.. Then plus with temporary column and column c is performed.
2. Code bloat for different function specializations using templates, increases binary size.
3. For AVX256, AVX512 instructions need to use runtime instructions specialization using CPUID. Necessary for distributing portable binary.
1. For each node in ActionsDAG get compilable_children_size
2. Sort nodes in descending order of compilable_children_size, to first compile nodes with most children.
3. Check if node can be compiled using heuristics. Currently we require node to contain at least 1 compilable children.
4. Compile node functions together into a function that takes raw columns data pointers and returns expression result.
4. Replace node in DAG with special LLVMFunction node. LLVMFunction execute method converts columns into raw data and calls compiled function
SELECT a + b + c FROM test_table;
SET compile_expressions = 1;
SELECT a + b + c FROM test_table;
1. Improve L1, L2 cache usages.
2. Less code to execute. It is placed on 1 page. Better usage of CPU branch predictor.
3. Eliminate indirections.
4. Multiple operations are fused in one function. More optimizations can be performed by the compiler.
5. Using target CPU instructions (AVX256, AVX512) if necessary.
Any function potentially can be compiled, but generating IR is inconvenient. Currently supported:
1. Binary operators. Example: plus, minus, multiply, xor.
2. Unary operators. Example: abs.
3. Logical functions. Example: and, or, not.
4. Branch functions. Example: if, multiIf.
5. Bit shift functions. Example: bitShiftLeft.
SELECT a + b * c + 5 FROM test_jit_merge_tree;
loop: ; preds = %loop, %entry
%19 = phi i64 [ 0, %entry ], [ %34, %loop ] /// Get counter phi node. If started from entry it is 0.
%20 = phi i64* [ %5, %entry ], [ %30, %loop ] /// Argument column phi node
%21 = phi i64* [ %9, %entry ], [ %31, %loop ] /// Argument column phi node
%22 = phi i64* [ %13, %entry ], [ %32, %loop ] /// Argument column phi node
%23 = phi i64* [ %17, %entry ], [ %33, %loop ] /// Result column phi node
%24 = load i64, i64* %20, align 8 /// Load argument current row
%25 = load i64, i64* %21, align 8 /// Load argument current row
%26 = load i64, i64* %22, align 8 /// Load argument current row
%27 = mul i64 %25, %26 /// Multiply b * c
%28 = add i64 %24, %27 /// Add a + result of multiply b * c
%29 = add i64 %28, 5 /// Add with constant 5
store i64 %29, i64* %23, align 8 /// Store value in result column
%30 = getelementptr inbounds i64, i64* %20, i64 1 /// Adjust pointer to next element
%31 = getelementptr inbounds i64, i64* %21, i64 1 /// Adjust pointer to next element
%32 = getelementptr inbounds i64, i64* %22, i64 1 /// Adjust pointer to next element
%33 = getelementptr inbounds i64, i64* %23, i64 1 /// Adjust pointer to next element
%34 = add i64 %19, 1 /// Increase counter
%35 = icmp eq i64 %34, %0br i1 %35, label %end, label %loop /// Check loop predicate
void aPlusBMulitplyCPlusConstant(
int64_t * a,
int64_t * b,
int64_t * c,
int64_t constant,
int64_t * result,
size_t size)
{
for (size_t i = 0; i < size; ++i)
{
*result = (*a) + (*b) * (*c) + constant;
++a;
++b;
++c;
++result;
}
}
.LBB0_8: # %vector.body
vmovdqu (%r11,%rax,8), %ymm1
vmovdqu (%r9,%rax,8), %ymm3
vmovdqu 32(%r11,%rax,8), %ymm2
vmovdqu 32(%r9,%rax,8), %ymm4
vpsrlq $32, %ymm3, %ymm5
vpsrlq $32, %ymm1, %ymm6
vpmuludq %ymm1, %ymm5, %ymm5
vpmuludq %ymm6, %ymm3, %ymm6
vpmuludq %ymm1, %ymm3, %ymm1
vpsrlq $32, %ymm4, %ymm3
vpmuludq %ymm2, %ymm3, %ymm3
vpaddq %ymm5, %ymm6, %ymm5
vpsllq $32, %ymm5, %ymm5
vpaddq %ymm5, %ymm1, %ymm1
vpsrlq $32, %ymm2, %ymm5
vpmuludq %ymm2, %ymm4, %ymm2
vpaddq (%r14,%rax,8), %ymm1, %ymm1
vpmuludq %ymm5, %ymm4, %ymm5
vpaddq %ymm3, %ymm5, %ymm3
vpsllq $32, %ymm3, %ymm3
vpaddq %ymm3, %ymm2, %ymm2
vpaddq 32(%r14,%rax,8), %ymm2, %ymm2
vpaddq %ymm0, %ymm1, %ymm1 /// in ymm0 there is constant 5. vpbroadcastq (%rbp), %ymm0
vmovdqu %ymm1, (%r10,%rax,8)
vpaddq %ymm0, %ymm2, %ymm2
vmovdqu %ymm2, 32(%r10,%rax,8)
addq $8, %rax
cmpq %rax, %r8
JIT standard expression compilation time is around 15 ms. Grows linearly with code size.
Typical compiled expressions use 1 page for the code section and 1 page for the data section. 4096 * 2 = 8192 bytes on most configurations.
Introspection works inside ClickHouse using `CompileExpressionsMicroseconds`, `CompileExpressionsBytes` metrics.
SELECT
ProfileEvents['CompileExpressionsMicroseconds'] AS compiled_time,
ProfileEvents['CompileExpressionsBytes'] AS compiled_bytes
FROM system.query_log
WHERE compiled_time > 0;
┌─compiled_time─┬─compiled_bytes─┐
│ 16258 │ 8192 │
│ 26792 │ 8192 │
│ 15280 │ 8192 │
│ 11594 │ 8192 │
│ 14989 │ 8192 │
└───────────────┴────────────────┘
SELECT count() FROM hits
WHERE
((EventDate >= '2018-08-01')
AND (EventDate <= '2018-08-03')
AND (CounterID >= 34))
OR ((EventDate >= '2018-08-04')
AND (EventDate <= '2018-08-05')
AND (CounterID <= 101500))
— 649 533 033 rows per second.
SET compile_expressions = 1;
SELECT count() FROM hits
WHERE
((EventDate >= '2018-08-01')
AND (EventDate <= '2018-08-03')
AND (CounterID >= 34))
OR ((EventDate >= '2018-08-04')
AND (EventDate <= '2018-08-05')
AND (CounterID <= 101500))
— 865 491 052 rows per second.
— +33% performance improvement!
SELECT
number * 2 +
number * 3 +
number * 4 +
number * 5
FROM system.numbers
FORMAT Null
— 0 rows in set. Elapsed: 0.903 sec. Processed 329.23 million rows, 2.63 GB (364.78 million rows/s., 2.92 GB/s.
SET compile_expressions = 1;
SELECT
number * 2 +
number * 3 +
number * 4 +
number * 5
FROM system.numbers
FORMAT Null
— 0 rows in set. Elapsed: 1.602 sec. Processed 1.89 billion rows, 15.15 GB (1.18 billion rows/s., 9.46 GB/s.)
—
+323% performance improvement!
SET compile_expressions = 1;
WITH number AS x, if(x = 1, 1, if(x = 2, 2,
if(x = 3, 3, if(x = 4, 4, if(x = 5, 5,
if(x = 6, 6, if(x = 7, 7, if(x = 8, 8,
if(x = 9, 9, if(x = 10, 10,
if(x = 11, 11, 12))))))))))) AS res
SELECT sum(res)
FROM numbers(10000000)
— 0 rows in set. Elapsed: 0.150 sec. Processed 10.02 million rows, 80.18 MB (66.95 million rows/s., 535.56 MB/s.).
SET compile_expressions = 1;
WITH number AS x, if(x = 1, 1, if(x = 2, 2,
if(x = 3, 3, if(x = 4, 4, if(x = 5, 5,
if(x = 6, 6, if(x = 7, 7, if(x = 8, 8,
if(x = 9, 9, if(x = 10, 10,
if(x = 11, 11, 12))))))))))) AS res
SELECT sum(res)
FROM numbers(10000000)
— 0 rows in set. Elapsed: 0.061 sec. Processed 10.02 million rows, 80.18 MB (163.20 million rows/s., 1.31 GB/s.)
—
+244% performance improvement!
EXPLAIN PIPELINE SELECT sum(UserID)
FROM default.hits_100m_single GROUP BY WatchID;
┌─explain────────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Aggregating) │
│ Resize 16 → 1 │
│ AggregatingTransform × 15 │
│ StrictResize 16 → 16 │
│ (Expression) │
│ ExpressionTransform × 16 │
│ (SettingQuotaAndLimits) │
│ (ReadFromMergeTree) │
│ MergeTreeThread × 16 0 → 1 │
└────────────────────────────────────────┘
class IAggregateFunction
{
...
virtual ~IAggregateFunction() = default;
/// AggregateDataPtr pointer to aggregate data for unique key during GROUP BY
/// Create empty data for aggregation with `placement new` at the specified location.
virtual void create(AggregateDataPtr place) const = 0;
/** Adds a value into aggregation data on which place points to.
* columns points to columns containing arguments of aggregation function.
* row_num is number of row which should be added.
*/
virtual void add(
AggregateDataPtr place,
const IColumn ** columns,
size_t row_num,
Arena * arena) const = 0;
/// Merges state (on which place points to) with other state of current aggregation function.
virtual void merge(AggregateDataPtr place, ConstAggregateDataPtr rhs, Arena * arena) const = 0;
/// Inserts results into a column. This method might modify the state (e.g.
/// sort an array), so must be called once, from single thread.
virtual void insertResultInto(AggregateDataPtr place, IColumn & to, Arena * arena) const = 0;
...
}
SELECT
sum(UserID),
sum(ClientIP),
sum(CounterClass),
sum(CounterID),
sum(WatchID)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
1. For each unique key, create an aggregate state for each function.
2. For each key update aggregate state for each.
3. For each unique key merge aggregate state for each function, in case multiple threads are used.
4. For each unique key insert result from aggregate state into final columns for each function.
1. A lot of virtual function calls.
2. For Nullable columns we have Nullable wrapper, additional indirection layer.
3. Aggregation combinators. -If, -Array, additional indirection layer.
Fuse multiple aggregate functions actions into one.
Functions require 4 actions: create, add, merge, insert. Fuse them and compile them into 4 functions.
1. Create function for multiple aggregate functions.
2. Add function for multiple aggregate functions.
3. Merge function for multiple aggregate functions.
4. Insert final result function for multiple aggregate functions.
1. Most common aggregate functions sum, count, min, max, avg, avgWeighted.
2. Combinators -If.
3. Nullable aggregate function adaptor.
SELECT
sum(UserID),
avg(ClientIP),
sum(CounterClass),
min(CounterID),
max(WatchID)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
SELECT
sum_avg_sum_min_max(
UserID,
ClientIP,
CounterClass,
CounterID,
WatchID)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
SELECT
sum(UserID),
sum(ClientIP),
sum(CounterClass),
sum(CounterID),
sum(WatchID)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
— 0 rows in set. Elapsed: 5.504 sec. Processed 100.00 million rows, 2.50 GB (18.17 million rows/s., 454.21 MB/s.)
SET compile_aggregate_expression = 1;
SELECT
sum(UserID),
sum(ClientIP),
sum(CounterClass),
sum(CounterID),
sum(WatchID)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
— 0 rows in set. Elapsed: 4.146 sec. Processed 100.00 million rows, 2.50 GB (24.12 million rows/s., 603.06 MB/s.)
— +34% performance improvement!
WITH (WatchID % 2 == 0) AS predicate
SELECT
minIf(UserID, predicate),
minIf(ClientIP, predicate),
minIf(CounterClass, predicate),
minIf(CounterID, predicate),
minIf(WatchID, predicate)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
— 0 rows in set. Elapsed: 6.234 sec. Processed 100.00 million rows, 2.50 GB (16.04 million rows/s., 401.05 MB/s.)
SET compile_aggregate_expression = 1;
WITH (WatchID % 2 == 0) AS predicate
SELECT
minIf(UserID, predicate),
minIf(ClientIP, predicate),
minIf(CounterClass, predicate),
minIf(CounterID, predicate),
minIf(WatchID, predicate)
FROM default.hits_100m_obfuscated
GROUP BY WatchID
— 0 rows in set. Elapsed: 4.146 sec. Processed 100.00 million rows, 2.50 GB (24.12 million rows/s., 603.06 MB/s.)
— +71% performance improvement!
EXPLAIN PIPELINE SELECT WatchID FROM hits_100m_single
ORDER BY WatchID, CounterID;
┌─explain──────────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Sorting) │
│ MergingSortedTransform 16 → 1 │
│ MergeSortingTransform × 16 │
│ LimitsCheckingTransform × 16 │
│ PartialSortingTransform × 16 │
│ (Expression) │
│ ExpressionTransform × 16 │
│ (SettingQuotaAndLimits) │
│ (ReadFromMergeTree) │
│ MergeTreeThread × 16 0 → 1 │
└──────────────────────────────────────────┘
PartialSortingTransform — sort block, apply special optimization for LIMIT if specified.
MergeSortingTransform — sort blocks using k-way-merge algorithm, output stream of sorted blocks.
MergingSortedTransform — sort streams of sorted blocks using k-way-merge algorithm.
ORDER BY WatchID, CounterID
We will apply comparator to each row almost N * log(N) * 2 times during our MergeSortingTransform, MergingSortedTransform.
We can compile it in a single function, to avoid unnecessary indirections.
For Nullable columns performance could be even better.
SET compile_sort_description=0;
SELECT WatchID FROM hits_100m_single
ORDER BY WatchID, CounterID
— 0 rows in set. Elapsed: 6.408 sec. Processed 100.00 million rows, 1.20 GB (15.60 million rows/s., 187.26 MB/s.)
SET compile_sort_description=1;
SELECT WatchID FROM hits_100m_single
ORDER BY WatchID, CounterID
— 0 rows in set. Elapsed: 5.300 sec. Processed 100.00 million rows, 1.20 GB (18.87 million rows/s., 226.40 MB/s.)
— +20% performance improvement!
SET compile_sort_description=1;
SELECT * FROM test_nullable_table
ORDER BY value
— 0 rows in set. Elapsed: 5.299 sec. Processed 100.00 million rows, 1.70 GB (18.87 million rows/s., 320.81 MB/s.)
SET compile_sort_description=1;
SELECT * FROM test_nullable_table
ORDER BY value
— 0 rows in set. Elapsed: 4.131 sec. Processed 100.00 million rows, 1.70 GB (24.21 million rows/s., 411.50 MB/s.)
— +30% performance improvement!
JIT compilation improves performance of expression evaluation and aggregation.
For expression evaluation JIT compilation improves performance in 1.5-3 times (for some cases more than 20 times).
For aggregation JIT compilation improves performance in 1.15-2 times.
For order by comparator JIT compilation improves performance in 1.15-1.5 times


This demo has been adapted from the updating line chart in the docs.
Update
There's information on this HOWTO page about creating dual color Line
charts using
gradients. It's very simple and can be done using either the
RGraph Gradient()
shortcut or more directly with the canvas linear gradient functionality.
<script src="RGraph.common.core.js"></script> <script src="RGraph.common.dynamic.js"></script> <script src="RGraph.line.js"></script>Put this where you want the chart to show up:
<canvas id="cvs" width="600" height="250"> [No canvas support] </canvas>This is the code that generates the chart - it should be placed AFTER the canvas tag(s):


~> rm -rf /var rm: /var: Operation not permitted Exception: rm exited with 1 Traceback: [interactive], line 1: rm -rf /var ~> xiaq@xiaqsmbp LASTCMD M-1 rm -rf /var 0 rm 1 -rf 2 /var


Bases: sigvisa.models.templates.template_model.TemplateModel
Bases: object
Abstract class defining a signal template model.
A phase template is defined by some number of parameters (e.g. onset period, height, and decay rate for a paired-exponential template). A signal consists of a number of phase arrivals.
The methods in the class deal with matrices, each row of which gives the parameters for a specific phase arrival. That is, we allow modeling the joint distribution of template parameters over multiple phases, though it’s also possible for a particular implementation to treat them independently.
Currently we assume that each channel and frequency band are independent. This should probably change.


In line with what I reported a few days ago, another major leak has just occurred with details of invites being leaked (via Videocardz) of an NVIDIA event that will be happening around Gamescom and should be the event we have all been waiting for: the launch of their next-gen...


h5pp¶
h5pp
is a C++17 wrapper for HDF5 with focus on simplicity.
In just a few lines of code, h5pp
lets users read and write to HDF5 files, a portable binary format.
h5pp
supports common data types and common containers, such as std::vector.
In particular, h5pp
makes it easy to read and write Eigen matrices and tensors.
Introduction¶
HDF5 is a popular format for portable binary storage of large datasets. With bindings to languages such as Python, Julia, Matlab and many others, it is straightforward to export, import and analyze data in a collaborative setting.
In C/C++ using HDF5 directly is not straightforward. Beginners are met with a steep learning curve to the vast API of HDF5. There are many C/C++ libraries already that simplify the user experience, but as a matter of opinion, things could be even simpler.
The goal of h5pp
is to make HDF5 simple to use in the following sense:
Users should be able to read/write common C++ data-types in a single line of code.
Users should not need prior knowledge of HDF5 for simple tasks.
Sensible defaults should let simple tasks stay simple, e.g., specifying storage layout, chunk dimensions or compression.
Advanced tasks should stay possible, e.g. MPI parallelism.
Logs and error messages should be meaningful to beginners.
Installation should be simple and scalable with opt-in automation.
Features¶
Header-only C++17 template library
High-level front-end to the C API of HDF5
Support for common data types:
short
,int
,long
,long long
(+ unsigned versions),float
,double
,long double
any of the above in C-style arrays
any of the above in
std::complex<>
formany of the above in POD-structs with x,y or x,y,z data members. In
h5pp
these go by the nameScalar2
andScalar3
. These work well together with types such asdouble2
orfloat3
found in CUDA.
std::string
andchar
arrays.Any container such as std::vector with
.data()
member for accessing a contiguous buffer (without conversion to/from row major).Eigen
types such asMatrix
,Array
andTensor
, with automatic conversion to/from row major storage layout.Support for user-defined compound HDF5 types
Support for creating HDF5 tables from user-defined compound HDF5 types.
Modern CMake installation providing targets for simple linking to your projects.
Installation with package managers: conan, conda (and apt using .deb installation file)
(Opt-in) Automatically find or download dependencies using either the Conan package manager or “CMake-only” methods.
Multi-platform: Linux, Windows, OSX. (Developed under Linux)
Usage¶
Using h5pp
is intended to be simple. After initializing a file,
most of the work can be achieved using just two member functions .writeDataset(...)
and .readDataset(...)
.
Example: Writing an std::vector
¶
#include <h5pp/h5pp.h>
int main() {
// Initialize a file
h5pp::File file("myDir/someFile.h5");
// Initialize a vector with 10 doubles
std::vector<double> v (10, 3.14);
// Write the vector to file.
// Inside the file, the data will be stored in a dataset named "myStdVector"
file.writeDataset(v, "myStdVector");
return 0;
}
Find more code examples in the examples directory or in the Wiki.
File permissions¶
h5pp
offers more flags for file access permissions than HDF5. The new flags are primarily intended to
prevent accidental loss of data, but also to clarify intent and avoid mutually exclusive options.
The flags are listed in the order of increasing “danger” that they pose to previously existing files.
| Flag | File exists | No file exists | Comment |
| —- | —- | —- | —- |
| READONLY
| Open with read-only permission | Throw error | Never writes to disk, fails if the file is not found |
| COLLISION_FAIL
| Throw error | Create new file | Never deletes existing files and fails if it already exists |
| RENAME
(default) | Create renamed file | Create new file | Never deletes existing files. Invents a new filename to avoid collision by appending “-#” (#=1,2,3…) to the stem of the filename |
| READWRITE
| Open with read-write permission | Create new file | Never deletes existing files, but is allowed to open/modify |
| BACKUP
| Rename existing file and create new | Create new file | Avoids collision by backing up the existing file, appending “.bak_#” (#=1,2,3…) to the filename |
| REPLACE
| Truncate (overwrite) | Create new file | Deletes the existing file and creates a new one in place |
When a new file is created, the intermediate directories are always created automatically.
When a new file is created,
READWRITE
permission to it is implied.
To give a concrete example, the syntax works as follows
h5pp::File file("myDir/someFile.h5", h5pp::FilePermission::REPLACE);
Storage Layout¶
Unless specified, h5pp
will automatically decide the best storage layout for each dataset. The possible layouts are
H5D_COMPACT
: For scalar or small datasets which can fit in the metadata header. Default on datasets smaller than 32 KB.H5D_CONTIGUOUS
: For medium size datasets. Default on datasets smaller than 512 KB.H5D_CHUNKED
: For large datasets. Default on datasets larger than 512 KB. This layout has some additional features:Chunking, portioning of the data to improve IO performance by caching more efficiently. Chunk dimensions are calculated by
h5pp
if not given specifically.Compression, disabled by default, and only available if HDF5 was built with zlib enabled.
Overwrite with different size (note that the file size never decreases, for instance after overwriting with a smaller dataset).
To specify the layout, pass it as a third argument when writing a new dataset, for instance:
file.writeDataset(myData, "science/myChunkedData", H5D_CHUNKED); // Creates a chunked dataset
Compression¶
Extendable (or chunked) datasets can also be compressed if HDF5 was built with zlib support. Use these functions to set or check the compression level:
file.setCompressionLevel(9); // 0 to 9: 0 to disable compression, 9 for maximum compression.
file.getCompressionLevel(); // Gets the current compression level
h5pp::checkIfCompressionIsAvailable(); // True if your installation of HDF5 has zlib support
or pass a temporary compression level as the fifth argument when writing a dataset:
file.writeDataset(myData, "science/myCompressedData", H5D_CHUNKED, std::nullopt, 8); // Creates a chunked dataset with compression level 8.
Debug and logging¶
Spdlog can be used to emit debugging information efficiently.
The amount of console output (verbosity) can be set to any level between 0
and 5
:
0: trace
(highest verbosity)1: debug
2: info
(default)3: warn
4: error
5: critical
(lowest verbosity)
Set the level when constructing a h5pp::File or by calling the function .setLogLevel(int)
:
int logLevel = 0; // Highest verbosity
// This way...
h5pp::File file("myDir/someFile.h5", h5pp::FilePermission::REPLACE, logLevel);
// or this way
file.setLogLevel(logLevel);
NOTE: Logging works the same with or without Spdlog enabled. When Spdlog is not found, a hand-crafted logger is used in its place to give identical output but without any performance considerations (implemented with STL lists, strings and streams).
Load data into Python¶
HDF5 data is easy to load into Python. Loading integer and floating point data is straightforward. compound data is almost as simple.
HDF5 does not support complex types specifically, but h5pp
enables this through compound HDF5 types. Here is a python example which uses h5py
to load 1D arrays from an HDF5 file generated with h5pp
:
import h5py
import numpy as np
file = h5py.File('myFile.h5', 'r')
# Originally written as std::vector<double> in h5pp
myDoubleArray = np.asarray(file['double-array-dataset'])
# Originally written as std::vector<std::complex<double>> in h5pp
myComplexArray = np.asarray(file['complex-double-array-dataset']).view(dtype=np.complex128)
Notice the cast to dtype=np.complex128
which interprets each element of the array as two doubles
, i.e. the real and imaginary parts are 2 * 64 = 128
bits.
Installation¶
There are currently 4 ways to obtain h5pp
:
git clone https://github.com/DavidAce/h5pp.git
and install (see below)From conda:
conda install -c davidace h5pp
From conan bintray repo
(Debian only) Download the latest release and install with apt:
sudo apt install ./h5pp_<version>_amd64.deb
Requirements¶
C++17 capable compiler. GCC version >= 7 or Clang version >= 7.0
CMake version >= 3.12
HDF5 library, version >= 1.8
Optional dependencies:¶
Eigen: Write Eigen matrices and tensors directly. Tested with version >= 3.3.4
spdlog: Enables logging for debug purposes. Tested with version >= 1.3.1
ghc::filesystem: This drop-in replacement for
std::filesystem
is downloaded and installed automatically when needed, but only ifH5PP_DOWNLOAD_METHOD=<fetch|conan>.
Install methods¶
For full working examples see the directory quickstart
. Find a summary below.
Option 1: Copy the headers¶
Copy the files under h5pp/source/include
and add #include<h5pp/h5pp.h>
.
Make sure to compile with -std=c++17 -lstdc++fs
and link the dependencies hdf5
, Eigen3
and spdlog
. The actual linking
is a non-trivial step, see linking below.
Option 2: Install with CMake¶
Build the library just as any CMake project. For instance, from the project’s root in command-line:
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=<install-dir> ../
make
make install
Headers will be installed under <install-dir>/include
and config files under <install-dir>/share/h5pp/cmake
.
These config files allow you to usefind_package(h5pp)
in your own projects, which in turn defines the target h5pp::h5pp
with everything you need to link h5pp
correctly (including dependencies, if you so choose).
If not set, CMAKE_INSTALL_PREFIX
defaults to ${CMAKE_BINARY_DIR}/install
, where ${CMAKE_BINARY_DIR}
is the directory you are building from.
Option 3: Install with Conan¶
Make sure to install and configure Conan first. Then, either use the cmake-conan integration by passing
-DH5PP_DOWNLOAD_METHOD=conan
as an argument to CMake (see below) or use Conan directly, for instance by running the following command:
$ conan install h5pp/1.8.0@davidace/stable --profile default
This is by far the simplest method and will also make sure to install dependencies HDF5, Eigen3 and spdlog.
Opt-in automatic dependency installation with CMake¶
The CMake flag H5PP_DOWNLOAD_METHOD
controls the automated behavior for finding or installing dependencies. It can take one of three valid strings:
| Option | Description |
| —- | —- |
| none
(default) | No handling of dependencies and linking is left to the user |
| find
| Use CMake’s find_package
to find dependencies pre-installed on your system |
| fetch
(!) | Use CMake-only features to download and install dependencies automatically. Disregards pre-installed dependencies on your system |
| native
| Deprecated. Use fetch
|
| find-or-fetch
| Start with find
and then go to fetch
if not found |
| conan
(!!) | Use the Conan package manager to download and install dependencies automatically. Disregards pre-installed dependencies on your system |
There are several variables you can pass to CMake to guide find_package
calls, see CMake build options below.
(!) Dependencies are installed into CMAKE_INSTALL_PREFIX
. Pass the CMake variable H5PP_DEPS_IN_SUBDIR
to install into separate directories under CMAKE_INSTALL_PREFIX/<libname>
.
(!!) Conan is guided by conanfile.txt
found in this project’s root directory. This method requires conan to be installed prior (for instance through pip
, conda
, apt
, etc). To let CMake find conan you have three options:
Add Conan install (or bin) directory to the environment variable
PATH
.Export Conan install (or bin) directory in the environment variable
CONAN_PREFIX
, i.e. from command line:export CONAN_PREFIX=<path-to-conan>
Give the variable
CONAN_PREFIX
directly to CMake, i.e. from command line:cmake -DCONAN_PREFIX:PATH=<path-to-conan> ...
CMake build options¶
The cmake
step above takes several options, cmake [-DOPTIONS=var] ../
:
| Var | Default | Description |
| —- | —- | —- |
| CMAKE_INSTALL_PREFIX
| ${CMAKE_BINARY_DIR}/install
| Specify h5pp
install directory |
| BUILD_SHARED_LIBS
| OFF
| Link dependencies with static or shared libraries |
| H5PP_ENABLE_TESTS
| OFF
| Build tests (recommended!) |
| H5PP_BUILD_EXAMPLES
| OFF
| Build example programs |
| H5PP_DOWNLOAD_METHOD
| none
| Download method for dependencies, select none
, find
, fetch
, find-or-fetch
or conan
. Fetch
downloads and builds from sources |
| H5PP_PRINT_INFO
| OFF
| Use h5pp with add_subdirectory() |
| H5PP_IS_SUBPROJECT
| OFF
| Print extra CMake info about the host and generated targets during configure |
| H5PP_ENABLE_EIGEN3
| OFF
| Enables Eigen3 linear algebra library support |
| H5PP_ENABLE_SPDLOG
| OFF
| Enables Spdlog support for logging h5pp
internal info to stdout |
| H5PP_DEPS_IN_SUBDIR
| OFF
| Appends <libname>
to install location of dependencies, i.e. CMAKE_INSTALL_PREFIX/<libname>
. This allows simple removal |
| H5PP_PREFER_CONDA_LIBS
| OFF
| Prioritize finding dependencies hdf5
, Eigen3
and spdlog
installed through conda. No effect when H5PP_DOWNLOAD_METHOD=conan
|
The following variables can be set to help guide CMake’s find_package
to your pre-installed software (no defaults):
| Var | Path to |
| —- | —- |
| Eigen3_DIR
| Eigen3Config.cmake |
| Eigen3_ROOT
| Eigen3 install directory |
| EIGEN3_INCLUDE_DIR
| Eigen3 include directory |
| spdlog_DIR
| spdlogConfig.cmake |
| spdlog_ROOT
| Spdlog install directory |
| HDF5_DIR
| HDF5Config.cmake |
| HDF5_ROOT
| HDF5 install directory |
| CONAN_PREFIX
| conan install directory |
Linking¶
Link using CMake targets (easy)¶
h5pp
is easily imported into your project using CMake’s find_package
. Just point it to the h5pp
install directory.
When found, targets are made available to compile and link to dependencies correctly.
A minimal CMakeLists.txt
to use h5pp
would look like:
cmake_minimum_required(VERSION 3.12)
project(myProject)
add_executable(myExecutable main.cpp)
find_package(h5pp PATHS <path-to-h5pp-install-dir> REQUIRED) # If h5pp is installed through conda the path may be $ENV{CONDA_PREFIX}
target_link_libraries(myExecutable PRIVATE h5pp::h5pp)
Targets explained¶
h5pp::h5pp
is the main target including “everything” and should normally be the only target that you need – headers,flags and (if enabled) the found/downloaded dependencies.h5pp::headers
links theh5pp
headers only.h5pp::deps
collects library targets to link all the dependencies that were found/downloaded whenh5pp
was built. These can of course be used independently.If
H5PP_DOWNLOAD_METHOD==find|find-or-fetch|fetch
the targets areEigen3::Eigen
,spdlog::spdlog
andhdf5::hdf5
,If
H5PP_DOWNLOAD_METHOD==conan
the targets areCONAN_PKG::Eigen3
,CONAN_PKG::spdlog
andCONAN_PKG::HDF5
.If
H5PP_DOWNLOAD_METHOD==none
thenh5pp::deps
is empty.
h5pp::flags
sets compile and linker flags to enable C++17 and std::filesystem library, i.e.-std=c++17
and-lstdc++fs
. OnMSVC
it sets/permissive-
to enable logicaland
/or
in C++.
Link manually (not as easy)¶
From the command-line you can of course link using linker flags such as -std=c++17 -lstdc++fs -leigen3 -lspdlog -lhdf5_hl -lhdf5
provided these flags make sense on your system.
You could also use CMake’s find_package(...)
mechanism. A minimal CMakeLists.txt
could be:
cmake_minimum_required(VERSION 3.12)
project(myProject)
add_executable(myExecutable main.cpp)
target_include_directories(myExecutable PRIVATE <path-to-h5pp-headers>)
# Setup h5pp
target_compile_features(myExecutable PRIVATE cxx_std_17)
target_link_libraries(myExecutable PRIVATE stdc++fs)
# Possibly use find_package() here
# Link dependencies (this is the tricky part)
target_include_directories(myExecutable PRIVATE <path-to-Eigen3-include-dir>)
target_include_directories(myExecutable PRIVATE <path-to-spdlog-include-dir>)
target_include_directories(myExecutable PRIVATE <path-to-hdf5-include-dir>)
# Link dependencies (this is the difficult part). Note that you only need the C libs for HDF5.
target_link_libraries(myExecutable PRIVATE hdf5_hl hdf5 rt dl m z pthread) # Possibly more libs, such as aec, dependending on your HDF5 installation
The difficult part is linking to HDF5 libraries and its dependencies.
Use the custom FindHDF5.cmake bundled with h5pp
¶
When installing h5pp
, finding HDF5 and setting up the CMake target hdf5::hdf5
for linking is handled by a custom module for finding HDF5, defined in cmake/FindHDF5.cmake
.
This module wraps the default FindHDF5.cmake
which comes with CMake and uses the same call signature, but fixes some annoyances with naming conventions in different versions of CMake and HDF5 executables.
It reads hints passed through CMake flags to find HDF5 somewhere on your system (e.g. installed via conda
,apt
, brew
, Easybuild
,etc) and defines a CMake target hdf5::hdf5
with everything you need to link correctly.
Most importantly, it avoids injecting shared versions of libraries (dl, zlib, szip, aec) during static builds on older platforms.
You can use the custom module too. Add the path pointing to FindHDF5.cmake
to the variable CMAKE_MODULE_PATH
from within your own project, e.g.:
list(APPEND CMAKE_MODULE_PATH path/to/h5pp/cmake/FindHDF5.cmake)
find_package(HDF5 1.10 COMPONENTS C HL REQUIRED)
if(TARGET hdf5::hdf5)
target_link_libraries(myExecutable PRIVATE hdf5::hdf5)
endif()
These are variables that can be used to guide the custom FindHDF5.cmake
module:
| Var | Where | Description |
| —- | —- | —- |
| CMAKE_MODULE_PATH
| CMake | List of directories where CMake
should search for find-modules |
| CMAKE_PREFIX_PATH
| CMake | List of directories where find_package
should look for dependencies|
| HDF5_ROOT
| CMake/ENV | Path to HDF5 root install directory |
| HDF5_FIND_VERBOSE
| CMake | Prints more information about the search for HDF5. See also HDF5_FIND_DEBUG
in the original module |
| EBROOTHDF5
| ENV | Variable defined by Easybuild with module load HDF5
|
Uninstall¶
The target uninstall
is defined by h5pp
which removes installed headers and dependencies using their respective install manifests.
From the build directory, run the following in the command-line to uninstall:
cmake --build . --target uninstall
To-do¶
In no particular order
Expand documentation. Perhaps a doxygen/sphinx webpage
Expand testing for more edge-cases in
filesystem permissions
user-defined types
tables
Expose more of the C-API:
Support for packed user-defined types. Read more: H5TPack
True support for parallel read/write with MPI
Support row-major <-> col-major transformation for types other than Eigen3 matrices and tensors. For instance, when raw pointers are passed together with dimension initializer list {x,y,z..}. (Although, this can be done by wrapping the data in an Eigen Map object).


P2P by design means that it does not have a centralized server to maintain or that could fail (NoDB)..
Decentralized and distributed Metaverse where non-fungible assets (NFTs) are managed. Incremental coordinates (infinite lands), based on WebXR cross-platform technology.
Supported by Mark Nadal's Gun Community