XLink implementation and the API for XLink processing in Scheme This software is in Public Domain. IT IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND. Please send bug reports and comments to<EMAIL_ADDRESS>Kirill Lisovsky<EMAIL_ADDRESS>Dmitry Lizorkin doc ::= '(*TOP* (@@ (sxlink (declared-here <sxlink-arc>* ) (embedded)? (outgoing (node <sxlink-arc>+ )* ) ) ... ; more aux list members ) ...)
They test whether an SXML node has a definite XLink type ATTENTION: 1. A non-prefixed XLink namespace uri is used for these node tests. If a prefix is used, these functions behave incorrectly. 2. These predicates should be used carefully - element's XLink-related meaning depends not only on its xlink:type attribute, but also on its position among other XLink element. For example, an element with an xlink:type="arc" attribute is not an arc element if it has anything other then an extended-link element as a parent
Helper for predicates type - a string, is supposed to have one of the following values: "extended", "simple", "locator", "resource", "arc", "title". A lambda is returned. When applied to an SXML node, it determines whether the node's xlink:type attribute has a 'type' value.
Node tests for different XLink elements
The following functions moved to "xlink-parser.scm" xlink:get-uri xlink:set-uri-for-sxlink-arcs
Sets the URI for the SXML document
Returns the id-index of the SXML document #f is returned is there is no "@@/id-index" subtree in the document
Returns (listof sxlink-arc) located in "@@/sxlink/declared-here" These are SXLink arcs that are declared in this document
Whether outgoing SXLink arcs are embedded into the document. This is denoted by the presense of "@@/sxlink/embedded" empty element.
Returns the content of "@@/sxlink/outgoing" The result is the associative list between nodes of the document and SXLink arcs that start from the corresponding node
Handler for error messages
Id+XLink parser parameterized
Returns the SXML representation for the resource specified by REQ-URI. Resource types supported: XML and HTML. XML is parsed into SXML with SSAX, HTML is parsed with HTML Prag. Additionally, linking information is parsed. For XML, linking information is assumed to be specified with XLink. For HTML,  elements are treated as simple links. In case of an error (resource doesn't exist or its type is unsupported), an error is signalled with 'xlink:api-error' and #f is returned.
Returns URIs of resources that participate in SXLink arcs sxlink-arcs ::= (listof sxlink-arc) Result: (listof string) The result may contain duplicates
Returns URIs of all linkbases encountered among SXLink arcs Result: (listof string) The result may contain duplicates
doc-set ::= (listof document)
Returns the list of URIs of the documents in the doc-set
Removes equal duplicates from the list
procedure xlink:find-doc :: URI-STRING (listof SXML-TREE) -> SXML-TREE Finding a document in 'doc-set' by its 'uri-string'. If there is no such document, #f is returned. doc-set ::= (listof SXML-TREE)
Returns a list of URIs which are refered by XLink markup Result: (listof string) The list may contain duplicates.
Returns a list of linkbase URIs which are refered by XLink markup Result: (listof string) The list may contain duplicates.
A helped low-level function for extending the doc-set with more documents Is parameterized with referenced-uris ::= (lambda (doc-set) ...) that would return URIs refered by XLink markup in the doc-set When parameterized, returns (lambda (doc-set . max-steps) ...) max-steps - maximal number of recursive steps The lambda returns the expanded doc-set
Two most common parameterized functions. The first one recursively loads linkbases. The second one recursively loads all refered documents
Parameterized with options, returns (lambda (uri . uris) ...) which is the lambda for getting documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied
The most common parameterized case. Loads documents and all linkbases
linked-docs ::= (listof document)
alist ::= (listof (cons key (listof item))) For equal keys in the alist, the function unites the corresponding key values Returns the new alist
Documents exchange their SXLink arcs, such as each arc is moved to the "@@/sxlink/outgoing" branch of the document where the arc's starting resource is Additional SXLink arcs may be specified in the optional argument.
The element node with embedded XLink arcs looks as follows element-node ::= (name (@ ...) (@@ (sxlink <sxlink-arc>+ ) ...) ; other members of the aux list ...) attribute-node ::= (name "value" (@@ (sxlink <sxlink-arc>+ ) ...) ; other members of the aux list )
Takes SXLink arcs outgoing from the document and embeds these arcs into element and attribute nodes of the document. The modified document is returned The function doesn't make a copy of nodes that remain unchanged
Returns all embedded SXLink arcs in the document Result: (listof sxlink-arc)
Parameterized with options, returns (lambda (linked-docs uri . uris) ...) which is the lambda for getting more documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied 'embed - embed SXLink arcs into nodes that are starting resources for that arcs 'no-embed - don't embed SXLink arcs into documents loaded
The most common case of parametrization
TODO: to be implemented later
Parameterized with options, returns (lambda (uri . uris) ...) which is the lambda for getting documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied 'embed - embed SXLink arcs into nodes that are starting resources for that arcs
procedure xlink:documents :: {REQ-URI}+ -> (listof SXML-TREE) procedure xlink:documents-embed :: {REQ-URI}+ -> (listof SXML-TREE) Both `xlink:documents' and `xlink:documents-embed' accept one or more strings as their arguments. Each string supplied denotes the URI of the requested document to be loaded. The requested document(s) are loaded and are represented in SXML. All XLink links declared in these document(s) are represented as a set of SXLink arcs. If any XLink links refer to XLink linkbases [XLink], these linkbases are additionally loaded, for additional SXLink arcs declared there. The starting resource for each SXLink arc is determined: 1. For each SXML document loaded, the function `xlink:document' adds all SXLink arcs whose starting resource is located within this document, to the auxiliary list of its document node (*TOP*). 2. The function 'xlink:documents-embed' embeds each SXLink arc into its starting resource-node, via auxiliary list of that node. For text nodes serving for starting resources, their SXLink arcs are stored in the auxiliary list of the document node (*TOP*), since SXML text nodes do not support their own auxiliary lists. Supported URI formats: + local file + http:// schema Supported document formats: XML and HTML. In the case of HTML,  hyperlinks are considered as XLink simple links. Result: (listof SXML-TREE) A particular SXML document can be located in this list using the function `xlink:find-doc'.
procedure sxml:document :: REQ-URI [NAMESPACE-PREFIX-ASSIG] -> -> SXML-TREE Obtain a [possibly, remote] document by its URI Supported URI formats: local file and HTTP schema Supported document formats: XML and HTML REQ-URI - a string that contains the URI of the requested document NAMESPACE-PREFIX-ASSIG - is passed as-is to the SSAX parser: there it is used for assigning certain user prefixes to certain namespaces. NAMESPACE-PREFIX-ASSIG is an optional argument and has an effect for an XML resource only. For an HTML resource requested, NAMESPACE-PREFIX-ASSIG is silently ignored. Result: the SXML representation for the requested document
Whether an SXLink arc
Returns the value of the administrative SXPath variable '*docs* This variable stores linked-docs
Extends var-bindings with administative information about linked-docs node - a single node or a nodeset
Returns SXLink arcs that are embedded into the node as aux list members Result: (listof sxlink-arc)
Returns SXLink arcs that are specified at the top-level of the document and start from node
Returns all SXLink arcs (both embedded and specified at the top-level) that start from ther node The union of the two previous functions
Traverse all SXLink arcs to their ending resources sxlink-arcs ::= (listof sxlink-arc) linked-docs ::= (listof document) num-ancestors - number of ancestors required for ending resources
XPath+XLink arc axis This axis returns all SXLink arcs that start from the context node num-ancestors is dummy here, since SXLink arcs don't have ancestors
XPath+XLink traverse axis This axis traverses from the context node The lambda produced additionally takes the var-binding. In var-binding, the linked-docs can be stored in the administrative variable '*docs*
XPath+XLink traverse-arc axis The axis traverses from the context node that is an SXLink arc The lambda produced additionally takes the var-binding. In var-binding, the linked-docs can be stored in the administrative variable '*docs*
Helper for predicates type - a string, is supposed to have one of the following values: "extended", "simple", "locator", "resource", "arc", "title". A lambda is returned. When applied to an SXML node, it determines whether the node's xlink:type attribute has a 'type' value.
(define (xlink:ntype?? type) (lambda (node) (let ((attval ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'http://www.w3.org/1999/xlink:type)) ((select-kids (ntype?? '@)) node))))) (if (null? attval) ; there is no xlink:type attribute #f (string=? (car attval) type)))))
Node tests for different XLink elements
(define xlink:elem-extended? (xlink:ntype?? "extended"))
(define xlink:elem-simple? (xlink:ntype?? "simple"))
(define xlink:elem-locator? (xlink:ntype?? "locator"))
(define xlink:elem-resource? (xlink:ntype?? "resource"))
(define xlink:elem-arc? (xlink:ntype?? "arc"))
(define xlink:elem-title? (xlink:ntype?? "title"))
Handler for error messages
(define (xlink:api-error . text) (cerr "XLink API error: ") (apply cerr text) (cerr nl))
Id+XLink parser parameterized
(define xlink:parser (ssax:multi-parser 'id 'xlink))
Returns the SXML representation for the resource specified by REQ-URI. Resource types supported: XML and HTML. XML is parsed into SXML with SSAX, HTML is parsed with HTML Prag. Additionally, linking information is parsed. For XML, linking information is assumed to be specified with XLink. For HTML,  elements are treated as simple links. In case of an error (resource doesn't exist or its type is unsupported), an error is signalled with 'xlink:api-error' and #f is returned.
(define (xlink:get-document-by-uri req-uri) (case (ar:resource-type req-uri) ((#f) ; resource doesn't exist (xlink:api-error "resource doesn't exist: " req-uri) #f) ((xml plain unknown) (let* ((port (open-input-resource req-uri)) (doc (xlink:parser port))) (close-input-port port) (xlink:set-uri req-uri doc))) ((html) (let* ((port (open-input-resource req-uri)) (doc (html->sxml port))) (close-input-port port) (SHTML->SHTML+xlink (xlink:set-uri req-uri doc)))) (else ; unknown resource type (xlink:api-error "resource type not supported: " req-uri) #f)))
alist ::= (listof (cons key (listof item))) For equal keys in the alist, the function unites the corresponding key values Returns the new alist
(define (xlink:unite-duplicate-keys-in-alist alist) (let loop ((src alist) (res '())) (if (null? src) res (let ((curr-key (caar src))) (let rpt ((scan (cdr src)) (content (cdar src)) (other '())) (cond ((null? scan) (loop other (cons (cons curr-key content) res))) ((equal? (caar scan) curr-key) (rpt (cdr scan) (append content (cdar scan)) other)) (else ; a different key (rpt (cdr scan) content (cons (car scan) other)))))))))
Documents exchange their SXLink arcs, such as each arc is moved to the "@@/sxlink/outgoing" branch of the document where the arc's starting resource is Additional SXLink arcs may be specified in the optional argument.
(define (xlink:docs-exchange-arcs doc-set . sxlink-arcs) (let ((doc-set-uris (xlink:uris doc-set)) (sxlink-arcs (if (null? sxlink-arcs) '() (car sxlink-arcs)))) ; outgoing-alist ::= (listof ; (cons uri ; (listof (cons node (listof sxlink-arc))))) ; declared-here-alist ::= (listof ; (cons uri (listof sxlink-arc))) (let loop ((outgoing-alist (map (lambda (doc) (cons (xlink:get-uri doc) (xlink:arcs-outgoing doc))) doc-set)) (declared-here-alist (map list doc-set-uris)) (arcs-to-scan (append sxlink-arcs (apply append (map xlink:arcs-declared-here doc-set))))) (if (null? arcs-to-scan) ; all arcs processed (let ((outgoing-alist (xlink:unite-duplicate-keys-in-alist outgoing-alist)) (declared-here-alist (xlink:unite-duplicate-keys-in-alist declared-here-alist))) (map (lambda (doc) (let ((uri (xlink:get-uri doc))) (xlink:replace-branch doc '(@@ sxlink) `((declared-here ,@(cdr (assoc uri declared-here-alist))) ,@(if (xlink:arcs-embedded? doc) '((embedded)) '()) (outgoing ,@(xlink:unite-duplicate-keys-in-alist (cdr (assoc uri outgoing-alist)))))))) doc-set)) (let* ((curr-arc (car arcs-to-scan)) (uri-from (car ; URI must be presented ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) ((select-kids (ntype?? 'from)) curr-arc))))) (uri-decl (car ; URI must be presented ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) ((select-kids (ntype?? 'declaration)) curr-arc)))))) (if (not (member uri-from doc-set-uris)) ; This arc starts from none of the documents from doc-set (loop outgoing-alist (cons (list uri-decl curr-arc) declared-here-alist) (cdr arcs-to-scan)) (let ((nodes ; nodes that are the starting resource (let ((nodes-nset ((select-kids (ntype?? 'nodes)) ((select-kids (ntype?? 'from)) curr-arc)))) (if (not (null? nodes-nset)) (cdar nodes-nset) (let ((xpointer-nset ((select-kids (ntype?? 'xpointer)) ((select-kids (ntype?? 'from)) curr-arc))) (starting-doc (xlink:find-doc uri-from doc-set))) (if (null? xpointer-nset) ; no XPointer ((select-kids (ntype?? '*)) ; document element starting-doc) (let ((func (sxml:xpointer (cadar xpointer-nset)))) (if (not func) ; parser error #f (let ((starting-nset (func starting-doc))) (if (nodeset? starting-nset) starting-nset #f)))))))))) (if nodes ; starting resource selects some nodes (loop (cons (cons uri-from (map (lambda (node) (list node curr-arc)) nodes)) outgoing-alist) declared-here-alist (cdr arcs-to-scan)) (loop outgoing-alist (cons (list uri-decl curr-arc) declared-here-alist) (cdr arcs-to-scan))))))))))
Parameterized with options, returns (lambda (linked-docs uri . uris) ...) which is the lambda for getting more documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied 'embed - embed SXLink arcs into nodes that are starting resources for that arcs 'no-embed - don't embed SXLink arcs into documents loaded
(define (xlink:parameterized-load-with-respect-documents . options) (let ((doc-getter (apply xlink:get-documents-with-params options)) (embed? (memq 'embed options)) (no-embed? (memq 'no-embed options))) (lambda (linked-docs . uris) (let* ((loaded-uris (xlink:uris linked-docs)) (req-docs (xlink:docs-exchange-arcs (filter (lambda (x) x) (map (lambda (uri) (if (member uri loaded-uris) ; document already loaded (xlink:find-doc uri linked-docs) (xlink:get-document-by-uri uri))) (xlink:remove-equal-duplicates uris))) (apply append (map xlink:arcs-declared-here linked-docs))))) (cond (no-embed? req-docs) ((or embed? ; embed arcs (member #t (map xlink:arcs-embedded? linked-docs))) (map xlink:embed-arcs-into-document req-docs)) (else req-docs))))))
The most common case of parametrization
(define xlink:get-docs-with-respect-to-loaded (xlink:parameterized-load-with-respect-documents 'linkbase))
Parameterized with options, returns (lambda (uri . uris) ...) which is the lambda for getting documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied 'embed - embed SXLink arcs into nodes that are starting resources for that arcs
(define (xlink:load-linked-docs-with-params . options) (let ((doc-getter (apply xlink:get-documents-with-params options))) (if (memq 'embed options) ; embed (lambda (uri . uris) (map xlink:embed-arcs-into-document (xlink:docs-exchange-arcs (apply doc-getter (cons uri uris))))) (lambda (uri . uris) (xlink:docs-exchange-arcs (apply doc-getter (cons uri uris)))))))
procedure xlink:documents :: {REQ-URI}+ -> (listof SXML-TREE) procedure xlink:documents-embed :: {REQ-URI}+ -> (listof SXML-TREE) Both `xlink:documents' and `xlink:documents-embed' accept one or more strings as their arguments. Each string supplied denotes the URI of the requested document to be loaded. The requested document(s) are loaded and are represented in SXML. All XLink links declared in these document(s) are represented as a set of SXLink arcs. If any XLink links refer to XLink linkbases [XLink], these linkbases are additionally loaded, for additional SXLink arcs declared there. The starting resource for each SXLink arc is determined: 1. For each SXML document loaded, the function `xlink:document' adds all SXLink arcs whose starting resource is located within this document, to the auxiliary list of its document node (*TOP*). 2. The function 'xlink:documents-embed' embeds each SXLink arc into its starting resource-node, via auxiliary list of that node. For text nodes serving for starting resources, their SXLink arcs are stored in the auxiliary list of the document node (*TOP*), since SXML text nodes do not support their own auxiliary lists. Supported URI formats: + local file + http:// schema Supported document formats: XML and HTML. In the case of HTML,  hyperlinks are considered as XLink simple links. Result: (listof SXML-TREE) A particular SXML document can be located in this list using the function `xlink:find-doc'.
(define xlink:documents (xlink:load-linked-docs-with-params 'linkbases))
(define xlink:documents-embed (xlink:load-linked-docs-with-params 'linkbases 'embed))
Whether an SXLink arc
(define xlink:arc? (ntype-names?? '(linkbase simple outbound inbound third-party local-to-local)))
Sets the URI for the SXML document
(define (xlink:set-uri uri doc) (let ((aux-nset ((select-kids (ntype?? '@@)) doc))) (if (or (null? aux-nset) ; no aux node at all yet ; no sxlink/declared-here subnode (null? ((select-kids (ntype?? 'declared-here)) ((select-kids (ntype?? 'sxlink)) (car aux-nset))))) (xlink:replace-branch ; inserts the @@/uri node in the document doc '(@@ uri) (list uri)) (xlink:replace-branch doc '(@@) (cdr ((xlink:branch-helper ; inserts URI to sxlink-arcs (lambda (declared-here-node dummy) (cons (car declared-here-node) (xlink:set-uri-for-sxlink-arcs uri (cdr declared-here-node))))) (xlink:replace-branch ; inserts (modified) URI (car aux-nset) '(uri) (list uri)) '(sxlink declared-here) '() ; dummy ))))))
Returns the id-index of the SXML document #f is returned is there is no "@@/id-index" subtree in the document
(define (xlink:id-index doc) (let ((nodeset ((select-kids (ntype?? 'id-index)) ((select-kids (ntype?? '@@)) doc)))) (if (null? nodeset) ; there is no "@@/id-index" subtree #f (cdar nodeset))))
Returns (listof sxlink-arc) located in "@@/sxlink/declared-here" These are SXLink arcs that are declared in this document
(define (xlink:arcs-declared-here doc) ((select-kids (ntype?? '*any*)) ((select-kids (ntype?? 'declared-here)) ((select-kids (ntype?? 'sxlink)) ((select-kids (ntype?? '@@)) doc)))))
Whether outgoing SXLink arcs are embedded into the document. This is denoted by the presense of "@@/sxlink/embedded" empty element.
(define (xlink:arcs-embedded? doc) (not (null? ((select-kids (ntype?? 'embedded)) ((select-kids (ntype?? 'sxlink)) ((select-kids (ntype?? '@@)) doc))))))
Returns the content of "@@/sxlink/outgoing" The result is the associative list between nodes of the document and SXLink arcs that start from the corresponding node
(define (xlink:arcs-outgoing doc) ((select-kids (ntype?? '*any*)) ((select-kids (ntype?? 'outgoing)) ((select-kids (ntype?? 'sxlink)) ((select-kids (ntype?? '@@)) doc)))))
Returns URIs of resources that participate in SXLink arcs sxlink-arcs ::= (listof sxlink-arc) Result: (listof string) The result may contain duplicates
(define (xlink:arcs-uris sxlink-arcs) ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) ((select-kids (ntype-names?? '(from to))) sxlink-arcs))))
Returns URIs of all linkbases encountered among SXLink arcs Result: (listof string) The result may contain duplicates
(define (xlink:arcs-linkbase-uris sxlink-arcs) ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) ((select-kids (ntype?? 'to)) (filter (ntype?? 'linkbase) sxlink-arcs)))))
Returns the list of URIs of the documents in the doc-set
(define (xlink:uris doc-set) (filter (lambda (x) x) (map xlink:get-uri doc-set)))
Removes equal duplicates from the list
(define (xlink:remove-equal-duplicates lst) (cond ((null? lst) lst) ((member (car lst) (cdr lst)) (xlink:remove-equal-duplicates (cdr lst))) (else (cons (car lst) (xlink:remove-equal-duplicates (cdr lst))))))
procedure xlink:find-doc :: URI-STRING (listof SXML-TREE) -> SXML-TREE Finding a document in 'doc-set' by its 'uri-string'. If there is no such document, #f is returned. doc-set ::= (listof SXML-TREE)
(define (xlink:find-doc uri-string doc-set) (let loop ((doc-set doc-set)) (cond ((null? doc-set) #f) ((equal? (xlink:get-uri (car doc-set)) uri-string) (car doc-set)) (else (loop (cdr doc-set))))))
Returns a list of URIs which are refered by XLink markup Result: (listof string) The list may contain duplicates.
(define (xlink:referenced-uris doc-set) (apply append (map (lambda (doc) (xlink:arcs-uris (xlink:arcs-declared-here doc))) doc-set)))
Returns a list of linkbase URIs which are refered by XLink markup Result: (listof string) The list may contain duplicates.
(define (xlink:referenced-linkbase-uris doc-set) (apply append (map (lambda (doc) (xlink:arcs-linkbase-uris (xlink:arcs-declared-here doc))) doc-set)))
A helped low-level function for extending the doc-set with more documents Is parameterized with referenced-uris ::= (lambda (doc-set) ...) that would return URIs refered by XLink markup in the doc-set When parameterized, returns (lambda (doc-set . max-steps) ...) max-steps - maximal number of recursive steps The lambda returns the expanded doc-set
(define (xlink:add-documents-helper referenced-uris) (lambda (doc-set . max-steps) (let ((max-steps (if (null? max-steps) -1 (car max-steps)))) (let loop ((doc-set doc-set) (loaded-uris (xlink:uris doc-set)) (to-load (referenced-uris doc-set)) (step 0)) (if (or (null? to-load) (= step max-steps)) doc-set (let rpt ((loaded-uris loaded-uris) (to-load to-load) (added-docs '())) (cond ((null? to-load) (loop (append added-docs doc-set) loaded-uris (referenced-uris added-docs) (+ step 1))) ((member (car to-load) loaded-uris) (rpt loaded-uris (cdr to-load) added-docs)) (else ; we load the linkbase (let ((doc (xlink:get-document-by-uri (car to-load)))) (rpt (cons (car to-load) loaded-uris) (cdr to-load) (if doc (cons doc added-docs) added-docs)))))))))))
Two most common parameterized functions. The first one recursively loads linkbases. The second one recursively loads all refered documents
(define xlink:add-linkbases-recursively (xlink:add-documents-helper xlink:referenced-linkbase-uris))
(define xlink:add-documents-recursively (xlink:add-documents-helper xlink:referenced-uris))
Parameterized with options, returns (lambda (uri . uris) ...) which is the lambda for getting documents by their URIs Options include the following: 'linkbases - load linkbases recursively '(linkbases <number> ) - load linkbases recursively, with the maximal number of recursive steps defined by the <number> supplied 'docs - load documents recursively '(docs <number> ) - load documents recursively, with the maximal number of recursive steps defined by the <number> supplied
(define (xlink:get-documents-with-params . options) (let ((get-initial-docs ; Returns documents by their URIs (lambda (uris) (filter ; keeps only correctly loaded documents (lambda (x) x) (map xlink:get-document-by-uri (xlink:remove-equal-duplicates uris))))) (linkbases-pairs (filter (lambda (option) (and (pair? option) (eq? (car option) 'linkbases))) options)) (docs-pairs (filter (lambda (option) (and (pair? option) (eq? (car option) 'docs))) options))) (let ((linkbases? (or (memq 'linkbases options) (not (null? linkbases-pairs)))) (max-steps-linkbases (if (null? linkbases-pairs) -1 (cadar linkbases-pairs))) (documents? (or (memq 'docs options) (not (null? docs-pairs)))) (max-steps-documents (if (null? docs-pairs) -1 (cadar docs-pairs)))) (cond ((and linkbases? documents?) (lambda (uri . uris) (xlink:add-linkbases-recursively (xlink:add-documents-recursively (get-initial-docs (cons uri uris)) max-steps-documents) max-steps-linkbases))) (linkbases? (lambda (uri . uris) (xlink:add-linkbases-recursively (get-initial-docs (cons uri uris)) max-steps-linkbases))) (documents? (lambda (uri . uris) (xlink:add-documents-recursively (get-initial-docs (cons uri uris)) max-steps-documents))) (else ; nothing extra to be loaded (lambda (uri . uris) (get-initial-docs (cons uri uris))))))))
The most common parameterized case. Loads documents and all linkbases
(define xlink:get-documents+linkbases (xlink:get-documents-with-params 'linkbases))
Takes SXLink arcs outgoing from the document and embeds these arcs into element and attribute nodes of the document. The modified document is returned The function doesn't make a copy of nodes that remain unchanged
(define (xlink:embed-arcs-into-document document) (letrec (; These helper functions return ; (values node outgoing-alist changed?) ; node - the (modified) node ; outgoing-alist ::= (listof (cons node (listof sxlink-arc))) ; changed? - whether the node was changed (process-element-node (lambda (node outgoing-alist) (cond ((or (not (pair? node)) (eq? (car node) '@@)) ; Text node or aux node (values node outgoing-alist #f)) ((eq? (car node) '@) (call-with-values (lambda () ((process-nodeset process-attribute-node) (cdr node) outgoing-alist)) (lambda (content new-out-alist changed?) (if changed? (values (cons '@ content) new-out-alist changed?) (values node outgoing-alist changed?))))) (else ; this is the element node (call-with-values (lambda () (cond ((assq node outgoing-alist) => (lambda (alist-member) (values (cdr alist-member) (filter (lambda (memb) (not (eq? memb alist-member))) outgoing-alist)))) (else ; the node is not the starting resource (values #f outgoing-alist)))) (lambda (outgoing-arcs new-out-alist) (call-with-values (lambda () ((process-nodeset process-element-node) (cdr node) new-out-alist)) (lambda (content new-out-alist changed?) (cond ((not (or outgoing-arcs changed?)) ; node remains unchanged (values node outgoing-alist changed?)) ((not outgoing-arcs) ; no arcs from that node (values (cons (car node) content) new-out-alist changed?)) (else ; the node is the starting resource (let ((new-content (if changed? content (cdr node)))) (values (cond ((not (null? ; aux list presented ((select-kids (ntype?? '@@)) new-content))) (xlink:append-branch (cons (car node) new-content) '(@@ sxlink) outgoing-arcs)) (((ntype?? '@) ; attribute node presented (car new-content)) `(,(car node) ,(car content) ; attribute node (@@ (sxlink ,@outgoing-arcs)) ,@(cdr content))) (else ; no attribute node `(,(car node) (@) (@@ (sxlink ,@outgoing-arcs)) ,@content))) new-out-alist #t)))))))))))) (process-attribute-node (lambda (node outgoing-alist) (cond ((assq node outgoing-alist) => (lambda (alist-member) (values (if (null? ; no aux node in the attribute ((select-kids (ntype?? '@@)) node)) (append node `((@@ (sxlink ,@(cdr alist-member))))) (xlink:append-branch node '(@@ sxlink) (cdr alist-member))) (filter (lambda (memb) (not (eq? memb alist-member))) outgoing-alist) #t))) (else ; the attribute node is not a starting resource (values node outgoing-alist #f))))) ; Is parameterized with one of the previous functions and ; processes the nodeset (process-nodeset (lambda (processing-func) (lambda (nodeset outgoing-alist) (let loop ((nset nodeset) (out-alist outgoing-alist) (changed? #f) (res '())) (if (null? nset) ; nodeset processed (values (reverse res) out-alist changed?) (call-with-values (lambda () (processing-func (car nset) out-alist)) (lambda (new-node new-out-alist ch?) (loop (cdr nset) new-out-alist (or changed? ch?) (cons new-node res)))))))))) (call-with-values (lambda () ((process-nodeset process-element-node) (cdr document) (xlink:arcs-outgoing document))) (lambda (content new-out-alist changed?) (if (not changed?) ; the document remains unchanged (xlink:replace-branch document '(@@ sxlink embedded) '()) (xlink:replace-branch (cons '*TOP* content) '(@@ sxlink) `((declared-here ,@(xlink:arcs-declared-here document)) (embedded) (outgoing ,@new-out-alist))))))))
Returns all embedded SXLink arcs in the document Result: (listof sxlink-arc)
(define (xlink:arcs-embedded doc) (let ((get-kids (select-kids (lambda (node) (and (pair? node) (not (eq? '@@ (car node)))))))) (let loop ((nodes-to-scan (get-kids doc)) (res '())) (if (null? nodes-to-scan) ; everyone processed (draft:remove-eq-duplicates res) (loop (append (get-kids (car nodes-to-scan)) (cdr nodes-to-scan)) (append ((select-kids (ntype?? '*any*)) ((select-kids (ntype?? 'sxlink)) ((select-kids (ntype?? '@@)) (car nodes-to-scan)))) res))))))
procedure sxml:document :: REQ-URI [NAMESPACE-PREFIX-ASSIG] -> -> SXML-TREE Obtain a [possibly, remote] document by its URI Supported URI formats: local file and HTTP schema Supported document formats: XML and HTML REQ-URI - a string that contains the URI of the requested document NAMESPACE-PREFIX-ASSIG - is passed as-is to the SSAX parser: there it is used for assigning certain user prefixes to certain namespaces. NAMESPACE-PREFIX-ASSIG is an optional argument and has an effect for an XML resource only. For an HTML resource requested, NAMESPACE-PREFIX-ASSIG is silently ignored. Result: the SXML representation for the requested document
(define (sxml:document req-uri . namespace-prefix-assig) (if (string? req-uri) (case (ar:resource-type req-uri) ((#f) ; resource doesn't exist (xlink:api-error "resource doesn't exist: " req-uri) #f) ((xml plain unknown) (let* ((port (open-input-resource req-uri)) (doc (ssax:xml->sxml port (if (null? namespace-prefix-assig) namespace-prefix-assig (car namespace-prefix-assig))))) (close-input-port port) doc ; DL: can also add URI: (xlink:set-uri req-uri doc) )) ((html) (let* ((port (open-input-resource req-uri)) (doc (html->sxml port))) (close-input-port port) doc ; DL: can also add URI: (xlink:set-uri req-uri doc) )) (else ; unknown resource type (xlink:api-error "resource type not supported: " req-uri) #f)) ; Otherwise: REQ-URI is not a string - producing an exception (exc:signal ; relies on SRFI-12 (make-property-condition 'exn 'message "sxml:document: expects type <string> as 1st argument"))))
Returns the value of the administrative SXPath variable '*docs* This variable stores linked-docs
(define (xlink:docs-variable var-binding) (cond ((assq '*docs* var-binding) => cdr) (else '())))
Extends var-bindings with administative information about linked-docs node - a single node or a nodeset
(define (xlink:add-docs-to-vars node var-binding) (if (assq '*docs* var-binding) ; variable already exists var-binding (cons (cons '*docs* (filter (lambda (doc) (and (draft:top? doc) (xlink:get-uri doc))) (draft:reach-root (as-nodeset node)))) var-binding)))
Returns SXLink arcs that are embedded into the node as aux list members Result: (listof sxlink-arc)
(define (xlink:node-embedded-arcs node) (if (draft:top? node) ; the root node '() ; no embedded arcs ((select-kids (ntype?? '*any*)) ((select-kids (ntype?? 'sxlink)) ((select-kids (ntype?? '@@)) node)))))
Returns SXLink arcs that are specified at the top-level of the document and start from node
(define (xlink:node-arcs-on-top node document) (cond ((assq node (xlink:arcs-outgoing document)) => cdr) (else '())))
Returns all SXLink arcs (both embedded and specified at the top-level) that start from ther node The union of the two previous functions
(define (xlink:node-arcs node document) (append (xlink:node-embedded-arcs node) (xlink:node-arcs-on-top node document)))
Traverse all SXLink arcs to their ending resources sxlink-arcs ::= (listof sxlink-arc) linked-docs ::= (listof document) num-ancestors - number of ancestors required for ending resources
(define (xlink:traverse-arcs sxlink-arcs linked-docs num-ancestors) (let* ((arcs-to ((select-kids (ntype?? 'to)) sxlink-arcs)) (req-docs (apply xlink:get-docs-with-respect-to-loaded (cons linked-docs (if (and num-ancestors (zero? num-ancestors)) ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) (filter ; elements that have a <nodes> subelement (lambda (arc-to) (null? ((select-kids (ntype?? 'nodes)) arc-to))) arcs-to))) ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) arcs-to))))))) ;(pp req-docs) (map-union (lambda (arc-to) (let ((nodes-nset ((select-kids (ntype?? 'nodes)) arc-to))) (if (and num-ancestors (zero? num-ancestors) (not (null? nodes-nset))) (cadar nodes-nset) ; otherwise we need the document and the XPointer node (let ((doc (xlink:find-doc (car ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'uri)) arc-to))) req-docs)) (xpointer-nset ((select-kids (ntype?? '*text*)) ((select-kids (ntype?? 'xpointer)) arc-to)))) ;(pp doc) ;(display xpointer-nset) ;(newline) (cond ((not doc) ; document couldn't be loaded '()) ((null? xpointer-nset) ; no XPointer part => addresses the document element ((draft:child (ntype?? '*) num-ancestors) doc)) (else (let ((impl (draft:xpointer (car xpointer-nset) (if num-ancestors num-ancestors -1)))) (if (not impl) ; parser error '() (let ((res (impl doc))) (if (nodeset? res) res (begin (xlink:api-error "XPointer fragment identifier doesn't " "select any nodeset: " (car xpointer-nset)) '()))))))))))) arcs-to)))
XPath+XLink arc axis This axis returns all SXLink arcs that start from the context node num-ancestors is dummy here, since SXLink arcs don't have ancestors
(define (xlink:axis-arc test-pred? . num-ancestors) (let ((this-axis (lambda (node) ; not a nodeset (let ((root-node (if (sxml:context? node) (draft:list-last (sxml:context->ancestors-u node)) node))) (if (draft:top? root-node) (xlink:node-arcs (sxml:context->node node) root-node) (xlink:node-embedded-arcs (sxml:context->node node))))))) (lambda (node) ; node or nodeset (filter test-pred? (if (nodeset? node) (map-union this-axis node) (this-axis node))))))
XPath+XLink traverse axis This axis traverses from the context node The lambda produced additionally takes the var-binding. In var-binding, the linked-docs can be stored in the administrative variable '*docs*
(define (xlink:axis-traverse test-pred? . num-ancestors) (let* ((num-anc (if (null? num-ancestors) 0 (car num-ancestors))) (get-arcs ; returns SXLink arcs that start from a given node (lambda (node) ; not a nodeset (let ((root-node (if (sxml:context? node) (draft:list-last (sxml:context->ancestors-u node)) node))) (if (draft:top? root-node) (xlink:node-arcs (sxml:context->node node) root-node) (xlink:node-embedded-arcs (sxml:context->node node))))))) ; node can be both a single node and a nodeset here (lambda (node var-binding) (filter (lambda (node) (test-pred? (sxml:context->node node))) (xlink:traverse-arcs (if (nodeset? node) (map-union get-arcs node) (get-arcs node)) (xlink:docs-variable var-binding) num-anc)))))
XPath+XLink traverse-arc axis The axis traverses from the context node that is an SXLink arc The lambda produced additionally takes the var-binding. In var-binding, the linked-docs can be stored in the administrative variable '*docs*
(define (xlink:axis-traverse-arc test-pred? . num-ancestors) (let ((num-anc (if (null? num-ancestors) 0 (car num-ancestors)))) (lambda (node var-binding) (filter (lambda (node) (test-pred? (sxml:context->node node))) (xlink:traverse-arcs (filter xlink:arc? (draft:reach-root (as-nodeset node))) (xlink:docs-variable var-binding) num-anc)))))


on
Tidymodels & palmer-penguins
Oddly enough, I’ve been hearing a lot about tidymodels
this week, from reddit to youtube. Plus, the
tidytuesday
dataset this week looks fun.I figure I’ll spend a tidytuesday trying to figure out this new and
in-development machine-learning package.
For the newcomer, tidymodels
is the planned replacement for the caret
package. Complicated
machine learning (ml) models will continue using caret
for a while as tidymodels
is very new to
the field. I’m taking the word of others more experienced in the matter, but it does have the same
abilities as caret
. This does not speak to tidymodels
potential, for it has successfully tied a
large number of ml models under one roof. It’s use of tidy implementation means it could also make
ml an easier field to break into for the non-computer scientist like me (I’m the classic biologist
turned programmer).
This is my exploration into the models, some of my notes, and my implementation of tidymodels
. I’ve
referred to a number of sites to get a broad idea. I’ve put those references at the very
bottom. This will be useful to those new to the package or machine learning.
the tidymodel packages
tidymodels
isn’t just one package. It’s a family of packages, each serving a distinct
purpose. I focus in this post on getting familiar with the essentials; rsample
, parsnip
, and yardstick
.
rsample
: Processes data to prepare it for models.recipes
: used to create ‘recipes’ for models. Can be used to design preprocessing matrixs, and useful for reproducible research.parsnip
: For running the models. Attempts to provide a singular interface to multiple packages. If you haven’t realized, the name is also a pun on the originalcaret
package!tune
: For fine tuning the models. Can be helpful in getting a model the way you need it. not attempted here.yardstick
: helps in quantifiying models.
I began my machine learning journey with caret
, which is wonderfully well-built. Any advanced user
could easily develop sophisticated scripts. Tidymodels
has an advantage over caret
that I appreciate
as someone still learning ml; it is refreshingly easy to change models, metrics, or other details
during experimentation.
palmer penguins
When I came across tidymodels
, this week’s tidytuesday dataset were the palmer penguins. It is
proposed as an alternative to the classic iris dataset.
Below, I show a simple ml model to classify penguins based on body features. I hope it is of help to someone.
data exploration
pacman::p_load(magrittr,
tidyverse,
ggplot2,
tidymodels
)
I’m grabbing the data directly from github, but you can also get it from the CRAN Palmer Penguins package.
There are two types of files. The smaller and larger penguins dataset. I’ll make use of
the smaller penguins to minimize the variables while playing with tidymodels
.
penguins <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv')
## Parsed with column specification:
## cols(
## species = col_character(),
## island = col_character(),
## bill_length_mm = col_double(),
## bill_depth_mm = col_double(),
## flipper_length_mm = col_double(),
## body_mass_g = col_double(),
## sex = col_character(),
## year = col_double()
## )
penguins %>% dim
## [1] 344 8
summary(penguins)
## species island bill_length_mm bill_depth_mm
## Length:344 Length:344 Min. :32.10 Min. :13.10
## Class :character Class :character 1st Qu.:39.23 1st Qu.:15.60
## Mode :character Mode :character Median :44.45 Median :17.30
## Mean :43.92 Mean :17.15
## 3rd Qu.:48.50 3rd Qu.:18.70
## Max. :59.60 Max. :21.50
## NA's :2 NA's :2
## flipper_length_mm body_mass_g sex year
## Min. :172.0 Min. :2700 Length:344 Min. :2007
## 1st Qu.:190.0 1st Qu.:3550 Class :character 1st Qu.:2007
## Median :197.0 Median :4050 Mode :character Median :2008
## Mean :200.9 Mean :4202 Mean :2008
## 3rd Qu.:213.0 3rd Qu.:4750 3rd Qu.:2009
## Max. :231.0 Max. :6300 Max. :2009
## NA's :2 NA's :2
penguins %>%
skimr::skim()
| Name | Piped data |
| Number of rows | 344 |
| Number of columns | 8 |
| _______________________ | |
| Column type frequency: | |
| character | 3 |
| numeric | 5 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| species | 0 | 1.00 | 6 | 9 | 0 | 3 | 0 |
| island | 0 | 1.00 | 5 | 9 | 0 | 3 | 0 |
| sex | 11 | 0.97 | 4 | 6 | 0 | 2 | 0 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| bill_length_mm | 2 | 0.99 | 43.92 | 5.46 | 32.1 | 39.23 | 44.45 | 48.5 | 59.6 | ▃▇▇▆▁ |
| bill_depth_mm | 2 | 0.99 | 17.15 | 1.97 | 13.1 | 15.60 | 17.30 | 18.7 | 21.5 | ▅▅▇▇▂ |
| flipper_length_mm | 2 | 0.99 | 200.92 | 14.06 | 172.0 | 190.00 | 197.00 | 213.0 | 231.0 | ▂▇▃▅▂ |
| body_mass_g | 2 | 0.99 | 4201.75 | 801.95 | 2700.0 | 3550.00 | 4050.00 | 4750.0 | 6300.0 | ▃▇▆▃▂ |
| year | 0 | 1.00 | 2008.03 | 0.82 | 2007.0 | 2007.00 | 2008.00 | 2009.0 | 2009.0 | ▇▁▇▁▇ |
I need species as a factor. All rows with nas will be removed to enable downstream analysis.
penguins$species <- penguins$species %>%
as.factor()
penguins <- penguins[-(penguins$bill_length_mm %>% is.na() %>% which()),]
Some rapid fire exploration. The graphs below suggest that each of the metrics can help identify a
specific penguin. All four metrics measuring the body have a multi-modal distribution. It’s obvious
that in every case, a peak is representative of one species. For example, bill_depth_mm
can
identify Gentoo, bill_length_mm
for Adelie, while Chinstrap has a strong overlap with either
species in all metrics. I imagine that this overlap will not interfere with the identification.
penguins.pivot <- penguins %>%
pivot_longer(cols = bill_length_mm:body_mass_g,
names_to = "metrics",
values_to = "values")
penguins.pivot %>%
ggplot(aes(values, fill = species)) +
geom_histogram(bins = 20) +
facet_wrap(~ metrics, scales = "free_x")
I don’t make use of the pivot functions often enough. They can be incredibly handy.
machine learning
Following chunk makes use of rsample
to split the dataset into train/test. As this is a small
dataset, lets split 65/35. The vfold_cv is necessary downstream when tuning the model.
penguins.split <- penguins %>%
rsample::initial_split(prop = 0.65, strata = species)
penguins.train <- penguins.split %>%
rsample::training()
penguins.test <- penguins.split %>%
rsample::testing()
penguins.cv <- penguins.train %>%
rsample::vfold_cv(v = 10)
building a model
In one of my first real introductions to tidymodels
(which was also with the palmer penguins data
set), the author made a large effort to use logistic regression. That is the wrong way to go,
because a logistic is expecting two factors for the response. Linear regression is open to a greater
range of response variables. Multi-class regression is done here with the engine ranger
which is
improved over randomForest
.
The ‘r set_engine’ is not a necessity here. Depending on the parsnip chosen, it automatically sets to rangr. I imagine that for the complicated models, it will be necessary to specify the engine.
The package tune
brings a handy fit_resample
function. As the name suggests, it adds to the ‘fit’
function by adding resampling capabilities. Specifically, it measures metrics over multiple
resamples (folds) for tuning the model.
model.fit.prep <- parsnip::rand_forest(mode = "classification") %>%
parsnip::set_engine("ranger") %>%
tune::fit_resamples(species ~ bill_length_mm + bill_depth_mm + flipper_length_mm + body_mass_g,
resamples = penguins.cv)
model.fit.prep
## # Resampling results
## # 10-fold cross-validation
## # A tibble: 10 x 4
## splits id .metrics .notes
## <list> <chr> <list> <list>
## 1 <split [201/23]> Fold01 <tibble [2 × 3]> <tibble [0 × 1]>
## 2 <split [201/23]> Fold02 <tibble [2 × 3]> <tibble [0 × 1]>
## 3 <split [201/23]> Fold03 <tibble [2 × 3]> <tibble [0 × 1]>
## 4 <split [201/23]> Fold04 <tibble [2 × 3]> <tibble [0 × 1]>
## 5 <split [202/22]> Fold05 <tibble [2 × 3]> <tibble [0 × 1]>
## 6 <split [202/22]> Fold06 <tibble [2 × 3]> <tibble [0 × 1]>
## 7 <split [202/22]> Fold07 <tibble [2 × 3]> <tibble [0 × 1]>
## 8 <split [202/22]> Fold08 <tibble [2 × 3]> <tibble [0 × 1]>
## 9 <split [202/22]> Fold09 <tibble [2 × 3]> <tibble [0 × 1]>
## 10 <split [202/22]> Fold10 <tibble [2 × 3]> <tibble [0 × 1]>
Now that we have a basic model, lets build a training example.
model.fit <- parsnip::rand_forest(mode = "classification") %>%
parsnip::set_engine("ranger") %>%
fit(species ~ bill_length_mm + bill_depth_mm + flipper_length_mm + body_mass_g,
data = penguins.train)
model.fit
## parsnip model object
##
## Fit time: 43ms
## Ranger result
##
## Call:
## ranger::ranger(formula = species ~ bill_length_mm + bill_depth_mm + flipper_length_mm + body_mass_g, data = data, num.threads = 1, verbose = FALSE, seed = sample.int(10^5, 1), probability = TRUE)
##
## Type: Probability estimation
## Number of trees: 500
## Sample size: 224
## Number of independent variables: 4
## Mtry: 2
## Target node size: 10
## Variable importance mode: none
## Splitrule: gini
## OOB prediction error (Brier s.): 0.03363747
evaluating classification model
Here, yardstick
comes into play.
penguins.fit.pred <- model.fit %>%
predict(penguins.test) %>%
bind_cols(penguins.test)
Looking at the metrics, the model is acceptable. Other metrics could be observed through ‘metric_set’ which I will not elaborate here. I’m satisfied by looking at kappa and accuracy.
penguins.fit.pred %>%
metrics(species, estimate = .pred_class)
## # A tibble: 2 x 3
## .metric .estimator .estimate
## <chr> <chr> <dbl>
## 1 accuracy multiclass 0.975
## 2 kap multiclass 0.960
Region-Over-Curves (ROC) are a popular measure of success. The smaller the ROC, the greater the ability to identify species. ROC curves of this dataset suggests a very high ability to identity species with few type 1 or 2 errors. Interestingly, I thought that chinstrap would be difficult to identify as it always overlapped with one of the other species. This seems to have been a benefit in identification.
penguins.met <- model.fit %>%
predict(penguins.test, type = "prob") %>%
bind_cols(penguins.test) %>%
yardstick::gain_curve(species,
.pred_Adelie,
.pred_Chinstrap,
.pred_Gentoo)
penguins.met %>%
autoplot() +
ggpubr::theme_pubclean() +
theme(strip.background = element_blank())
summary
As mentioned in the beginning, it is easy to modify the ml approach on the fly. I have enjoyed
working with tidy models, and plan to continue with it for a while. The only regret I have about the
tidymodels
is a near lack of documentation. The website is definitely rich in articles, but I am
having a tough time finding the minutia about function details. That is to be expected for the time
being while the package develops further. For those who are not happy with tidymodels
, there is the
alternative ml wrapper ‘mlr3’. It is as capable as the tidymodels
and does not follow the tidy
approach, which some may find attractive. Plus, I’ve noticed after writing this article that mlr3
has a bigger following than I thought. I plan to eventually post about my experiences with mlr3 vs
tidymodels
.


This Screen is used to track the information of the package related to specific Location and displays the package details like date on which the package is delivered, customer name to whom it is delivered and the signature of the customer after receiving the packages.
Figure 7.1
This page accepts Order Number or Tracking Number as input accordingly selected from the list as in Figure 7.2 and populates the Header and Package Information as in Figure 7.4.
If any error occurs in this page the relevant message is displayed. This page displays the information for Shipped Orders only.
Input Validations:
1. Order Number should not be Null and should be only Numeric values.
2. Tracking Number should not be null and accepts any value.
Figure 7.2
The Location will be disabled. The shipping user can select a name from the list and date from the calendar (Figure 7.3)on which the order is Shipped now select name and click on GO button as shown in Figure 7.4. Header and Package information will be populated as in Figure 7.5
Figure 7.3
Figure 7.4
The details will be displayed as below shown in Figure 7.5
Figure 7.5
When the Waybill button is clicked the new page populates with brief information of the package like Tracking Number, Status and Delivery Information as in Figure 7.6
Figure 7.6
When the Tracking Number button is clicked the detail information of the package is populated as in Figure 7.7.
Figure 7.7
This screen Figure 7.7 shows the complete information of the package like: date on which package is delivered, customer name, address, weight of the package and the type of the service used.
POD:
As the customer receives the packages the customer signature becomes the proof of delivery. On Clicking POD Button the customer information and the customer signature is populated on the right side of the page as Figure 7.8.
Figure 7.8


---
title: log
sidebar: function_reference_sidebar
permalink: quaternion.log.html
folder: function_reference
toc: false
---
the logarithmic map that translates a rotation into a rotation vector
Syntax
{% highlight matlab %}
v = log(q) % rotation vector with reference to the identical rotation
v = log(q,q_ref) % rotation vector with reference q_ref
{% endhighlight %}
{% highlight matlab %}
v = loq(q,'right')
v = loq(q,'left')
{% endhighlight %}
Input
Output
See also
quaternion.logm vector3d.exp spinTensor.spinTensor


Running Benchmarks and Algorithms
| Name | Graph | Creation Method | R |
|---|
Running Metrics
| Name | Graph | Creation Method | R |
|---|
Running Cooperation Simulation
| Status | Name | R |
|---|
Running Centrality Calculations and Simulations
| Name | Graph | Creation Method | R |
|---|


VIDEO: Our N1.1trn budget of olimpotic meristemasis is driven by complex metaphysical realism —Gov Ayade
by Agency ReportThe Cross River State Governor, Prof. Ben Ayade, has said that his N1.1trn 2020 budget estimate tagged, ‘Budget of olimpotic meristemasis,’ is “driven by complex metaphysical realism.”
Ayade, who has a PhD in Environmental Microbiology, notes that “The budget will catalyse into existence a great opportunity for us all to put our hands and legs on the pedal and hit it off.”
Suffice to say, in tissue biology, olympic meristematic (rather than ‘olimpotic meristemasis’ as Ayade calls it) cells give rise to various organs of a plant and are responsible for growth.
“Meristematic tissue consists of actively dividing cells, and leads to increase in length and thickness of the plant,” according to Wikipedia.
See the video:


Bitcoin 90-Day Volume Breaks Bullish Above Peak 2017 Levels
by Christine VasilevaThe Bitcoin market has recently surpassed the activity levels of late 2017, when BTC started its historical rally that shot prices to an all-time high above $19,900.
Futures Markets Take Over Spot Trading
This time, the BTC market has made a complete recovery from the low volumes in January. The biggest boost comes from the newly launched futures markets, and the heightened activity on BitMEX. The leading futures exchange saw open interest exceed $1 billion several times in the second half of the year.
According to a market commentator, Charles Edwards, Bitcoin’s 90-day volume recently broke as high as 40% above 2017’s peak levels. This, he says, ‘has powerful implications’.
The BTC spot market, which was a source of price discovery for years, has been supplanted by the larger, more agile futures market. Price discovery is happening both on BitMEX and Binance futures markets, as well as CME and Bakkt, which are attracting mainstream investors with cash-settled and physical delivered bitcoin futures.
Bitcoin and Tether Flows Signal End of Rally
The BTC market is also dependent on the interplay between exchange inflows and outflows. BTC movements to exchanges signal selling pressures. Tether (USDT) inflows however, are usually followed by a rally.
But the most effective combination for a bullish price move would be an outflow of BTC, while USDT liquidity grows. The “hodling” behavior on the side of whales and new funds to boost buying BTC positions.
In the past few months, the flow profile signals a potential price slide, due to diminishing sources of new liquidity. The attitude to BTC is signalling a return to pessimism, with the Bitcoin fear and greed index moving down to “fear” territory at 39 points.
For BTC, almost all factors that determine a rally period can shift within days or even hours. BTC has shown that even at depressed prices, a rally to a higher valuation can take just a few weeks to set up. The activity of USDT shifts quickly. Right now, each USDT coin moves about four times through the exchange ecosystem in a day. During peak activity periods, each coin moves more than 11 times per day, giving the 4.1 billion coins a magnified effect.
BTC turned down at the end of last week, abandoning the levels above $9,000. On Monday, the coin touched lows very near $8,700, without breaking down under that level. On Tuesday, the price started sinking again, to $8,775.59. The BTC market cap dominance shrank to 66.1%, as BTC erased its positions, while altcoins for now are holding onto relatively stable prices.
What do you think about the BTC market activity? Share your thoughts in the comments section below!
Images via Shutterstock, Twitter @Caprioleio @Ceterispar1bus
The Rundown


What is Verified by BitGo?
It’s a publicly accessible audit service that BitGo offers to provide transparency and security to end users of other services. It gives the public assurance of the total bitcoin-denominated funds held by a company while allowing users to cryptographically verify their own account balances have been included in the company’s stated liabilities. By combining these two features, Verified by BitGo provides reassurance that a company is not operating a fractional reserve scheme.
Why does BitGo offer this service?
Demonstrating 100% reserves, meaning that a company’s bitcoin assets are greater than or equal to its liabilities, has posed a significant challenge to the bitcoin industry. Many businesses offer no way for customers to verify that their funds are indeed held by the company.
Other businesses have taken steps to demonstrate their ownership of assets and associated keys through a number of other methods, including periodically moving coins or signing messages. These steps are often cumbersome and pose substantial security and privacy risks. Our new Verified by BitGo pages instead offer this transparency as a service, allowing any business to easily show its assets and cryptographically prove that it is operating in a solvent state.
Where did the idea for Proof of Reserves come from?
Other members of the community, in particular Greg Maxwell and Olivier Lalonde, have proposed and designed a series of implementations that can allow companies to prove they are operating in a solvent state. Several of these concepts are integral to BitGo’s Proof of Reserves service. We build on Lalonde’s work on our server and in our client, and our API produces proof trees which are compatible with his proof of liability tools.
What are the limitations of the service?
No system of audit is perfect. While BitGo is able to directly verify and attest to Bitcoin assets held in its multi-sig wallets, it cannot directly verify the liability information provided by the company. BitGo must rely on the end users to verify the liability information by checking that their own balances are correct and have been included in the overall totals. If the company excludes certain inactive customers from the stated liabilities, BitGo cannot detect that unless those users attempt to verify their balances.
BitGo also has no knowledge of the state of the company’s overall balance sheet. It cannot verify that the company has not borrowed bitcoin temporarily in order to show an apparent state of solvency.
The audit is not renewed in real time. In the time intervals since the last audit, the company may have entered a state of insolvency. Users should always consider the recency of the audit data.
While the crowd-sourced audit of liabilities can be applied to fiat currencies, BitGo has no way to directly verify any fiat assets.
Does BitGo have access to my bitcoin?
No, BitGo do not hold your bitcoin or the company’s bitcoin and cannot transact independently. We act as a co-signer to the company’s pool of bitcoin in order to provide enhanced security and trasparency.


{% load static %}
{% if user.is_authenticated and request.get_full_path == "/" %} {% endif %}Smart City Service.
"Dustbin IoT is an intelligent system for managing the refuse collection of a Smart City."
This project was conceived as a thesis for the biennial ITS course: "Superior Technician for Smart City & Clean Energy Management".>
Stock footage provided by Videvo, downloaded from www.videvo.net


- emptyColorRow(int) - Static method in class presentation.view.components.ColorRow
-
Return a array of empty pegs.
- EqualRolesException - Exception in domain.model.exceptions
-
Treats Equal Roles exception Interruption
Happens when the system detects a new game is being created with two users of the same role.
- EqualRolesException() - Constructor for exception domain.model.exceptions.EqualRolesException
-
Instantiates a new Command interrupt exception.
- equals(Object) - Method in class domain.model.row.ColorRow.ColorPeg
-
Checks if two ColorRow are equals, have the same value
- equals(Object) - Method in class domain.model.row.ControlRow
-
Checks if two ControlRows are equals, have the same value
- error - Variable in class presentation.utils.TerminalMenuBuilder
-
- ERROR_INPUT - Static variable in class resources.strings.TerminalConstants
-
The Main exit title.
- ERROR_USER_NOT_FOUND - Static variable in class resources.strings.TerminalConstants
-
The Error user not found.
- errorLine(String) - Method in class presentation.utils.TerminalUtils
-
Error line.
- evolvePopulation(int, int) - Method in class domain.model.player.computer.GeneticComputer
-
- execute() - Method in class presentation.utils.TerminalMenuBuilder
-
Execute.
- executeCommands(String) - Method in class presentation.utils.TerminalUtils
-
Execute the in-game commands.
- exists(String) - Method in interface domain.controller.data.DataController
-
Check if key value exists in the dataSet
- exists(String) - Method in class persistence.model.DataModel
-
Checks if the key exists
- existsUser(String) - Method in class domain.controller.UserController
-
Checks if the user already exists
- EXIT - Static variable in class resources.strings.TerminalConstants
-
The Main exit title.


AmplitudeAnalyzer
The AmplitudeAnalyzer measures (or "tracks") the amplitude of its input
in 128-sample windows.
It sends this measurement directly to a VSS client, or via a
message group other actors.
To use the AmplitudeAnalyzer implementation, load amplAnalyzer.so
and create an actor of type AmplitudeAnalyzer.
-
To send amplitude measurements directly to a client, the client should
send the message "Analyze" to the AmplitudeAnalyzer (typically from a
.aud file via AUDupdate), and then immediately call actorGetReplyData()
which returns a const char * string. This string can be parsed with
sscanf(..., "%f", ...) or atof() to get the floating-point measurement value.
Analyses will occur once for every "Analyze" message received (unless
a measurement is already in progress).
-
To send amplitude measurements to another actor,
send a SetMessageGroup message and then
send the SetRate message with a nonzero argument indicating how many times
per second the message group should be called.
The message group will then be called that often, with an argument array
consisting of one element, the measurement value.
AmplitudeAnalyzer
In addition to the messages understood by all Generator actors,
the AmplitudeAnalyzer understands the following messages:
- SetRate hActor rate
- Set rate for subsequent analyses (default is zero).
AmplitudeAnalyzer handler messages
In addition to the messages understood by all handlers, the handler for the
AmplitudeAnalyzer algorithm understands the following messages. If the optional
parameter time is specified in any of the following messages,
then the corresponding parameter is linearly modulated from the old value
to the specified value over the duration of time seconds after receipt
of the command.
- SetMessageGroup hActor name
- Specify which message group the analyses are to be regularly sent to.
The format of the analysis is an array with a single element, which
is the maximum amplitude measured in the most recent window of 128 samples
(i.e., *0).
- SetRate hActor rate
- Set the rate in Hz at which the message group should be called with
a new analysis result. Defaults to zero, which means that the message
group is not called at all.
- Analyze hActor
- Measure amplitude right now. The VSS client application should
immediately call actorGetReplyData() after this message is sent,
to get the result of the analysis.


Run the reactor with the given configuration. Subclasses should probably call this from C{postApplication}. @see: L{runReactorWithLogging}


With a background in horticulture from Boise State University and operating her families Nursery in Idaho from a young age; Betsy brings a wealth of knowledge to the Green Thumb team. Idaho is known for rough agricultural climates and uncertain extreme changes in the weather. She is the first to show up for class and the last to leave and somehow never skips a beat. She sometimes gets a little out of hand when working with dirt and has been known to start dirt fights with the support staff. If you have a question, she most likely has an answer for you when it comes to gardening. A human index of gardening knowledge and is always willing to listen and help. In her spare time, Betsy loves to volunteer her time at community gardens and helping high-school level educators teach their students the importance of gardening.
Technical Instructor
Josiah Carter
With his welcoming disposition and willingness to help others succeed, Josiah spent his youth backpacking the country. Every company has an employee that thinks it should be Hawaiian shirt day every day and Green Thumb's version is Josiah. The office prankster and the one you must look out for if he is quiet. When he is not pranking the GT staff, he is teaching all levels of the classes that Green Thumb offers with a base knowledge of each subject. Years of backpacking has made him a sponge when it comes to picking up new things and being able to excel at them. While only a member of the team for a couple of years, Josiah has assumed most the responsibilities of a veteran instructor. With his innovative outlook on gardening, he is known for teaching the other teachers something new every day.
Technical Instructor
Naomi Nguyen
Naomi is the Green Thumb Academy Index and Reference Library. She just seems to have knowledge of every aspect of gardening and loves to teach others. Students usually enjoy her classes because of just how informative she is. With her gift for gardening, she loves to can her own harvest and is hoping to one day open her own line of craft salsa's and pickle's. Naomi usually teaches the advanced classes at Green Thumb but sometimes jumps in to cover the "green" students as a mentor. What staff does not need a mentor to mentor their own staff, Naomi handles that as well in stride. She is responsible for the Green Thumb Academy learning structure and curriculum. In her spare time Naomi like to memorize the various Latin names of plants and volunteers her time at the Zoo helping the staff cultivate exotic foods for the animals.


Question Answering¶
Knowledge Base Question Answering¶
Answer Selection¶
Note: For Answer Selection, sequence length is an important parameter to be tuned.
| paper | authors | year | TrecQA (TRAIN-ALL): MAP | TrecQA (TRAIN-ALL): MRR | TrecQA (clean): MAP | TrecQA (clean): MRR | TrecQA (TRAIN): MAP | TrecQA (TRAIN): MRR | WikiQA: MAP | WikiQA: MRR | InsuranceQA-test1: P@1 | InsuranceQA-test2: P@1 | SemEval-cQA: MAP | SemEval-cQA: MRR | code |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering | Di Wang and Eric Nyberg | 2015 | 0.7134 | 0.7913 | |||||||||||
| Learning to rank short text pairs with convolutional deep neural networks | Aliaksei Severyn | 2015 | 0.7459 | 0.8078 | 0.7329 | 0.7962 | |||||||||
| Wikiqa:Achallenge dataset for open-domain question answering | Yi Yang | 2015 | 0.652 | 0.6652 | |||||||||||
| Abcnn: Attention-based convolutional neural network for modeling sentence pairs | Wenpeng Yin, Hinrich Sch¨ utze | 2016 | 0.6921 | 0.7108 | https://github.com/yinwenpeng/Answer_Selection | ||||||||||
| aNMM : Ranking Short Answer Texts with Attention-Based Neural Matching Model | Liu Yang1 Qingyao Ai1 Jiafeng Guo2 W. Bruce Croft | 2016 | 0.7495 | 0.8109 | 0.7417 | 0.8102 | |||||||||
| Attentive pooling networks | Cicero dos Santos | 2016 | 0.753 | 0.8511 | 0.6886 | 0.6957 | 0.717 | 0.664 | |||||||
| Convolutional Neural Networks vs . Convolution Kernels : Feature Engineering for Answer Sentence Reranking | Kateryna Tymoshenko† and Daniele Bonadiman† and Alessandro Moschitti | 2016 | 0.7518 | 0.8553 | 0.7417 | 0.7588 | |||||||||
| Employing External Rich Knowledge for Machine Comprehension | BingningWang, Shangmin Guo, Kang Liu, Shizhu He, Jun Zhao | 2016 | 0.6936 | 0.7094 | |||||||||||
| Improved Representation Learning for Question Answer Matching | Ming Tan, Cicero dos Santos, Bing Xiang & Bowen Zhou | 2016 | 0.753 | 0.83 | 0.69 | 0.648 | |||||||||
| Inner attention based recurrent neural networks for answer selection | BingningWang, Kang Liu, Jun Zhao | 2016 | 0.7369 | 0.8208 | 0.7341 | 0.7418 | 0.7011 | 0.6514 | |||||||
| LSTM-based Deep Learning Models for non-factoid answer selection | Ming Tan, Cicero dos Santos, Bing Xiang & Bowen Zhou | 2016 | 0.7279 | 0.824 | 0.681 | 0.633 | |||||||||
| Modeling relational information in question-answer pairs with convolutional neural networks | Aliaksei Severyn | 2016 | 0.7654 | 0.8186 | 0.7325 | 0.8018 | 0.6951 | 0.7107 | |||||||
| Neural variational inference for text processing | Yishu Miao | 2016 | 0.6886 | 0.7069 | |||||||||||
| Noise-contrastive estimation for answer selection with deep neural networks | Jinfeng Rao1 , Hua He1 , and Jimmy Lin | 2016 | 0.78 | 0.834 | 0.801 | 0.877 | 0.709 | 0.723 | https://github.com/Jeffyrao/pairwise-neural-network | ||||||
| Pairwise word interaction modeling with deep neural networks for semantic similarity measurement | Hua He1 and Jimmy Lin | 2016 | 0.7588 | 0.8219 | 0.709 | 0.7234 | |||||||||
| Sentence similarity learning by lexical decomposition and composition | ZhiguoWang and Haitao Mi and Abraham Ittycheriah | 2016 | 0.7058 | 0.7226 | |||||||||||
| A compare aggregate model for matching text sequences | ShuohangWang | 2017 | 0.7433 | 0.7545 | 0.756 | 0.734 | https://github.com/shuohangwang/SeqMatchSeq | ||||||||
| A compare aggregate model with dynamic-clip attention for answer selection | Weijie Bian, Si Li, Zhao Yang, Guang Chen, Zhiqing Lin | 2017 | 0.821 | 0.899 | 0.754 | 0.764 | https://github.com/wjbianjason/Dynamic-Clip-Attention | ||||||||
| A Hybrid Framework for Text Modeling with Convolutional RNN | ChenglongWang, Feijun Jiang, Hongxia Yang | 2017 | 0.7427 | 0.7504 | 0.714 | 0.683 | |||||||||
| Bilateral multi-perspective matching for natural language sentences | ZhiguoWang,Wael Hamza, Radu Florian | 2017 | 0.802 | 0.875 | 0.718 | 0.731 | https://github.com/zhiguowang/BiMPM | ||||||||
| Enhancing Recurrent Neural Networks with Positional Attention for Question Answering | Qin Chen1, Qinmin Hu1, Jimmy Xiangji Huang2, Liang He1,3 andWeijie An | 2017 | 0.7814 | 0.8513 | 0.7212 | 0.7312 | |||||||||
| Inter-weighted alignment network for sentence pair modeling | Gehui Shen Yunlun Yang Zhi-Hong Deng | 2017 | 0.822 | 0.889 | 0.733 | 0.75 | |||||||||
| Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2017 | 0.7499 | 0.8153 | 0.752 | 0.8146 | |||||||||
| On the Benefit of Incorporating External Features in a Neural Architecture for Answer Sentence Selection | Ruey-Cheng Chen, Evi Yulianti, Mark Sanderson,W. Bruce Croſt | 2017 | 0.782 | 0.837 | 0.701 | 0.718 | |||||||||
| Ranking Kernels for Structures and Embeddings : A Hybrid Preference and Classification Model | Kateryna Tymoshenko† and Daniele Bonadiman† and Alessandro Moschitti | 2017 | 0.7219 | 0.7408 | 0.771 | 0.8345 | https://github.com/iKernels/RelTextRank | ||||||||
| A Multi-View Fusion Neural Network for Answer Selection | Lei Sha,∗ Xiaodong Zhang,∗ Feng Qian, Baobao Chang, Zhifang Sui | 2018 | 0.7462 | 0.7576 | 0.8005 | 0.8718 | |||||||||
| CA-RNN : Using Context-Aligned Recurrent Neural Networks for Modeling Sentence Similarity | Qin Chen,1 Qinmin Hu,1 Jimmy Xiangji Huang,2 Liang He | 2018 | 0.8227 | 0.8886 | 0.7358 | 0.745 | |||||||||
| CAN : Enhancing Sentence Similarity Modeling with Collaborative and Adversarial Network | Qin Chen1, Qinmin Hu, Jimmy Xiangji Huang3 and Liang He | 2018 | 0.841 | 0.9168 | 0.7303 | 0.7431 | |||||||||
| Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.854 | 0.935 | |||||||||||
| Context-Aware Answer Sentence Selection With Hierarchical Gated Recurrent Neural Networks | Chuanqi Tan , FuruWei, Qingyu Zhou, Nan Yang, Bowen Du ,Weifeng Lv, and Ming Zhou | 2018 | 0.7638 | 0.7825 | |||||||||||
| Cross Temporal Recurrent Networks for Ranking Question Answer Pairs | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.7712 | 0.8384 | 0.7582 | 0.8233 | |||||||||
| End-to-End Quantum-like Language Models with Application to Question Answering | Peng Zhang1,∗, Jiabin Niu1, Zhan Su1, BenyouWang2, Liqun Ma3, Dawei Song | 2018 | 0.7589 | 0.8254 | 0.6496 | 0.6594 | |||||||||
| Hermitian Co-Attention Networks for Text Matching in Asymmetrical Domains | Yi Tay1, Anh Tuan Luu2, Siu Cheung Hui | 2018 | 0.784 | 0.895 | 0.743 | 0.756 | |||||||||
| Hyperbolic representation learning for fast and efficient neural question answering | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.77 | 0.825 | 0.784 | 0.865 | 0.712 | 0.727 | 0.795 | null | https://github.com/vanzytay/WSDM2018_HyperQA | ||||
| Knowledge as A Bridge: Improving Cross-domain Answer Selection with External Knowledge | Yang Deng1, Ying Shen1,∗, Min Yang2, Yaliang Li3, Nan Du3,Wei Fan3, Kai Lei1 | 2018 | 0.797 | 0.85 | |||||||||||
| Knowledge-aware Attentive Neural Network for Ranking Question Answer Pairs | Ying Shen1, Yang Deng1, Min Yang2, Yaliang Li3, Nan Du3,Wei Fan3, Kai Lei | 2018 | 0.7921 | 0.8444 | 0.8038 | 0.8846 | 0.7323 | 0.7494 | |||||||
| Multihop Attention Networks for Question Answer Matching | Nam Khanh Tran, Claudia Niederée | 2018 | 0.813 | 0.893 | 0.722 | 0.738 | 0.705 | 0.669 | https://github.com/namkhanhtran/nn4nqa | ||||||
| Recurrently Controlled Recurrent Networks | Yi Tay1, Luu Anh Tuan2, and Siu Cheung Hui3 | 2018 | 0.779 | 0.882 | 0.724 | 0.737 | https://github.com/vanzytay/NIPS2018_RCRN | ||||||||
| Self-Training for Jointly Learning to Ask and Answer Questions | Mrinmaya Sachan | 2018 | 0.798 | 0.854 | 0.754 | 0.753 | |||||||||
| Semantic Linking in Convolutional Neural Networks for Answer Sentence Selection | Massimo Nicosia∗ and Alessandro Moschitti† | 2018 | 0.7793 | 0.8489 | 0.7224 | 0.7391 |


Table Name: tabFrappe Job
| Sr | Fieldname | Type | Label | Options |
|---|---|---|---|---|
| 1 | job_title |
Data | Job Title | |
| 2 | company_name |
Data | Company Name | |
| 3 | status |
Select | Status |
Open Withdrawn Expired Completed Assigned |
| 4 | need_it_by |
Date | Need It By | |
| 5 | country |
Select | Country |
Afghanistan Albania Algeria Andorra Angola Antigua & Deps Argentina Armenia Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bhutan Bolivia Bosnia Herzegovina Botswana Brazil Brunei Bulgaria Burkina Burundi Cambodia Cameroon Canada Cape Verde Central African Rep Chad Chile China Colombia Comoros Congo Congo {Democratic Rep} Costa Rica Croatia Cuba Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic East Timor Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Fiji Finland France Gabon Gambia Georgia Germany Ghana Greece Grenada Guatemala Guinea Guinea-Bissau Guyana Haiti Honduras Hungary Iceland India Indonesia Iran Iraq Ireland {Republic} Israel Italy Ivory Coast Jamaica Japan Jordan Kazakhstan Kenya Kiribati Korea North Korea South Kosovo Kuwait Kyrgyzstan Laos Latvia Lebanon Lesotho Liberia Libya Liechtenstein Lithuania Luxembourg Macedonia Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Mauritania Mauritius Mexico Micronesia Moldova Monaco Mongolia Montenegro Morocco Mozambique Myanmar, {Burma} Namibia Nauru Nepal Netherlands New Zealand Nicaragua Niger Nigeria Norway Oman Pakistan Palau Panama Papua New Guinea Paraguay Peru Philippines Poland Portugal Qatar Romania Russian Federation Rwanda St Kitts & Nevis St Lucia Saint Vincent & the Grenadines Samoa San Marino Sao Tome & Principe Saudi Arabia Senegal Serbia Seychelles Sierra Leone Singapore Slovakia Slovenia Solomon Islands Somalia South Africa South Sudan Spain Sri Lanka Sudan Suriname Swaziland Sweden Switzerland Syria Taiwan Tajikistan Tanzania Thailand Togo Tonga Trinidad & Tobago Tunisia Turkey Turkmenistan Tuvalu Uganda Ukraine United Arab Emirates United Kingdom United States Uruguay Uzbekistan Vanuatu Vatican City Venezuela Vietnam Yemen Zambia Zimbabwe |
| 6 | job_detail |
Text | Job Detail | |
| 7 | delivery |
Select | Delivery |
Online In Person |
| 8 | service |
Select | Job Type |
Customization ERP Implementation Integration App Development Website Development Core Development Other |
| 9 | budget |
Int | Approx Budget (US$) | |
| 10 | show_in_website |
Check | Show in Website | |
| 11 | frappe_partner |
Link | Frappe Partner | Frappe Partner |
| 12 | rating |
Int | Partner Rating | |
| 13 | feedback |
Text | Partner Feedback | |
| 14 | page_name |
Data | Page Name | |
| 15 | parent_website_route |
Data | Parent Website Route | |
| 16 | amended_from |
Link | Amended From | Frappe Job |
Inherits from frappe.website.website_generator.WebsiteGenerator
Public API
/api/method/community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.bid
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.bid (job)
No docs
Public API
/api/method/community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.close
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.close (job)
No docs
Public API
/api/method/community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.complete
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.complete (job, feedback, rating)
No docs
Public API
/api/method/community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.delete
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.delete (job)
No docs
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.expire_jobs ()
No docs
community_erpnext_com.erpnext_community_portal.doctype.frappe_job.frappe_job.weekly_digest ()
No docs


--- date: 2010-03-01T00:00:00Z published: true title: tweethook. twitter data. pushed to you. url: /2010/03/01/tweethook-twitter-data-pushed-to-you-/ ---
TweetHook makes receiving Twitter Search data simple and automatic. Using the power of webhooks, setting up a continuous search on Twitter is a snap.  Learn More »


from typing import Any, Dict, Type # NOQA
from easydict import EasyDict
from kedro.utils import load_obj
import numpy as np
import pandas as pd
import sklearn # NOQA
from sklearn.metrics import (
accuracy_score,
confusion_matrix,
f1_score,
precision_score,
recall_score,
roc_auc_score,
)
[docs]def get_cols_features(
df,
non_feature_cols=[
"Treatment",
"Outcome",
"TransformedOutcome",
"Propensity",
"Recommendation",
],
):
return [column for column in df.columns if column not in non_feature_cols]
[docs]def concat_train_test(args, train, test):
r"""
Concatenate train and test series.
Use series.xs('train') or series.xs('test') to split
"""
series = pd.concat(
[pd.Series(train), pd.Series(test)],
keys=["train", "test"],
names=[args.partition_name, args.index_name],
)
return series
[docs]def concat_train_test_df(args, train, test):
r"""
Concatenate train and test data frames.
Use df.xs('train') or df.xs('test') to split.
"""
df = pd.concat(
[train, test],
keys=["train", "test"],
names=[args.partition_name, args.index_name],
)
return df
[docs]def len_t(df, treatment=1.0, col_treatment="Treatment"):
return df.query("{}=={}".format(col_treatment, treatment)).shape[0]
[docs]def len_o(df, outcome=1.0, col_outcome="Outcome"):
return df.query("{}=={}".format(col_outcome, outcome)).shape[0]
[docs]def len_to(
df, treatment=1.0, outcome=1.0, col_treatment="Treatment", col_outcome="Outcome"
):
len_ = df.query(
"{}=={} & {}=={}".format(col_treatment, treatment, col_outcome, outcome)
).shape[0]
return len_
[docs]def treatment_fraction_(df, col_treatment="Treatment"):
return len_t(df, col_treatment=col_treatment) / len(df)
[docs]def treatment_fractions_(
args, # type: Dict[str, Any]
df, # type: Type[pd.DataFrame]
):
# type: (...) -> Type[EasyDict]
col_treatment = args.col_treatment
treatment_fractions = {
"train": treatment_fraction_(df.xs("train"), col_treatment=col_treatment),
"test": treatment_fraction_(df.xs("test"), col_treatment=col_treatment),
}
return EasyDict(treatment_fractions)
[docs]def outcome_fraction_(df, col_outcome="Outcome"):
return len_o(df, col_outcome=col_outcome) / len(df)
[docs]def overall_uplift_gain_(
df, treatment=1.0, outcome=1.0, col_treatment="Treatment", col_outcome="Outcome"
):
overall_uplift_gain = (
len_to(df, col_treatment=col_treatment, col_outcome=col_outcome)
/ len_t(df, col_treatment=col_treatment)
) - (
len_to(df, 0, 1, col_treatment=col_treatment, col_outcome=col_outcome)
/ len_t(df, 0, col_treatment=col_treatment)
)
return overall_uplift_gain
[docs]def gain_tuple(df_, r_):
treatment_fraction = treatment_fraction_(df_)
outcome_fraction = outcome_fraction_(df_)
overall_uplift_gain = overall_uplift_gain_(df_)
cgain = np.interp(treatment_fraction, r_.cgains_x, r_.cgains_y)
cgain_base = overall_uplift_gain * treatment_fraction
cgain_factor = cgain / cgain_base
return (
treatment_fraction,
outcome_fraction,
overall_uplift_gain,
cgain,
cgain_base,
cgain_factor,
r_.Q_cgains,
r_.q1_cgains,
r_.q2_cgains,
)
[docs]def score_df(y_train, y_test, y_pred_train, y_pred_test, average="binary"):
if len(y_train) != len(y_pred_train):
raise Exception("Lengths of true and predicted for train do not match.")
if len(y_pred_test) != len(y_pred_test):
raise Exception("Lengths of true and predicted for test do not match.")
num_classes = pd.Series(y_train).nunique()
score_2darray = [
[
len(y_),
pd.Series(y_).nunique(),
accuracy_score(y_, y_pred_),
precision_score(y_, y_pred_, average=average),
recall_score(y_, y_pred_, average=average),
f1_score(y_, y_pred_, average=average),
]
+ (
[
roc_auc_score(y_, y_pred_),
pd.Series(y_).mean(),
pd.Series(y_pred_).mean(),
]
if num_classes == 2
else []
)
for (y_, y_pred_) in [(y_train, y_pred_train), (y_test, y_pred_test)]
]
score_df = pd.DataFrame(
score_2darray,
index=["train", "test"],
columns=["# samples", "# classes", "accuracy", "precision", "recall", "f1"]
+ (
["roc_auc", "observed conversion rate", "predicted conversion rate"]
if num_classes == 2
else []
),
)
return score_df
[docs]def conf_mat_df(y_true, y_pred):
conf_mat = confusion_matrix(y_true, y_pred)
num_class = len(conf_mat)
true_labels = ["True_{}".format(i) for i in range(num_class)]
pred_labels = ["Pred_{}".format(i) for i in range(num_class)]
conf_mat_df = pd.DataFrame(conf_mat, index=true_labels, columns=pred_labels)
return conf_mat_df
[docs]def bundle_train_and_test_data(args, train_df, test_df):
assert isinstance(train_df, pd.DataFrame)
assert isinstance(test_df, pd.DataFrame)
assert set(train_df.columns) == set(test_df.columns)
assert all([isinstance(col_name, str) for col_name in train_df.columns])
index_name = args.index_name
if index_name is not None:
train_df = train_df.reset_index(drop=True).copy()
train_df.index.name = index_name
test_df = test_df.reset_index(drop=True).copy()
test_df.index.name = index_name
else:
assert train_df.index.name == test_df.index.name
df = concat_train_test_df(args, train_df, test_df)
return df
[docs]def impute_cols_features(args, df):
non_feature_cols = [
args.col_treatment,
args.col_outcome,
args.col_propensity,
args.col_cate,
args.col_recommendation,
]
args.cols_features = args.cols_features or get_cols_features(
df, non_feature_cols=non_feature_cols
)
return args
[docs]def compute_cate(proba_treated, proba_untreated):
cate_estimated = proba_treated - proba_untreated
return cate_estimated
[docs]def add_cate_to_df(args, df, cate_estimated):
df.loc[:, args.col_cate] = cate_estimated.values
return df
[docs]def recommend_by_cate(args, df, treatment_fractions):
cate_series = df[args.col_cate]
def recommendation(cate_series, treatment_fraction):
rank_series = cate_series.rank(method="first", ascending=False, pct=True)
r = np.where(rank_series <= treatment_fraction, 1.0, 0.0)
return r
recommendation_train = recommendation(
cate_series.xs("train"), treatment_fractions.train
)
recommendation_test = recommendation(
cate_series.xs("test"), treatment_fractions.test
)
df.loc[:, args.col_recommendation] = concat_train_test(
args, recommendation_train, recommendation_test
)
return df
[docs]def estimate_effect(sim_treated_df, sim_untreated_df):
estimated_effect_df = pd.DataFrame()
estimated_effect_df["# samples"] = (
sim_treated_df["# samples chosen without uplift model"]
+ sim_untreated_df["# samples chosen without uplift model"]
)
## Original (without uplift model)
estimated_effect_df["observed conversion rate without uplift model"] = (
sim_treated_df["# samples chosen without uplift model"]
* sim_treated_df["observed conversion rate without uplift model"]
+ sim_untreated_df["# samples chosen without uplift model"]
* sim_untreated_df["observed conversion rate without uplift model"]
) / (
sim_treated_df["# samples chosen without uplift model"]
+ sim_untreated_df["# samples chosen without uplift model"]
)
## Recommended (with uplift model)
estimated_effect_df["predicted conversion rate using uplift model"] = (
sim_treated_df["# samples recommended by uplift model"]
* sim_treated_df["predicted conversion rate using uplift model"]
+ sim_untreated_df["# samples recommended by uplift model"]
* sim_untreated_df["predicted conversion rate using uplift model"]
) / (
sim_treated_df["# samples recommended by uplift model"]
+ sim_untreated_df["# samples recommended by uplift model"]
)
estimated_effect_df["predicted improvement rate"] = (
estimated_effect_df["predicted conversion rate using uplift model"]
/ estimated_effect_df["observed conversion rate without uplift model"]
)
return estimated_effect_df
[docs]def initialize_model(
args, # type: Type[EasyDict]
model_key="uplift_model_params", # type: str
default_estimator="sklearn.linear_model.LogisticRegression", # type: str
):
# type: (...) -> Type[sklearn.base.BaseEstimator]
if not isinstance(args[model_key], dict):
model = args[model_key]
return model
model_params = args[model_key].copy()
if not model_params.get("estimator"):
model_params["estimator"] = default_estimator
estimator_str = model_params.pop("estimator")
estimator_obj = load_obj(estimator_str)
const_params = (
(model_params.pop("const_params") or dict())
if "const_params" in model_params
else dict()
)
if not model_params.get("search_cv"):
const_params.update(model_params)
model = estimator_obj(**const_params)
return model
search_cv_str = model_params.pop("search_cv")
search_cv_obj = load_obj(search_cv_str)
model_params["estimator"] = estimator_obj(**const_params)
model = search_cv_obj(**model_params)
return model
# class CausalLiftParamError(Exception):


An occupational change occurs when one gets promoted to a higher-level occupation. An Elven Wizard may advance to a
Spellsinger
or an
Elemental Summoner
. Which occupation do you want to know about?
Spellsinger
Elemental Summoner


--- layout: default title: About Us description: About Baremetallics ---
Embedded Systems are cool and fun. And more than anything else, easy.
But it's a sad fact that most of the content/information out there is not easy to process.
Specially for a beginner.
We are a community that wants to spread knowledge.
Our goal is to make our content as easy as possible to understand.
We don't want you to decipher tutorials, guides or books to learn.
Let people know about this proyect.
And how can they support us.
We use Amazon Associates Program to support our website.
Check our booklist. You can find there a great collection to go to the next level.
You can also write a post or tutorial. There are three ways to do it:
1. Send us an email:
Write a post in Markdown format and send us by mail to<EMAIL_ADDRESS>You can find the template with instructions HERE.
2. Create a pull request on Github.
The instructions are in HERE.
You will find the README file with instructions at the bottom.
3. Write it in any text format
We don't recommend this. Because it will take some time to convert it into our format.
But if you want to collaborate. Then write the post and send us by mail to<EMAIL_ADDRESS>Right now, there are no web-developers or graphic/UI-designers in our team.
Some of us have software development and marketing experience.
We try our best to give you the best experience.
If you have ideas to improve this site.
We want to hear from you.
Don't be shy. Take Part
We are sure that the comunnity will really appreciate it ;).
If you still don't know how you can help. No problem.
Send us a mail to<EMAIL_ADDRESS>And let's us know your interest! :)


The power of Open Source is the power of the people, The people rule.
This project is launched with in an open source platform.
| Junior Full Stack Programmer. |
|---|
| One years of experience in analysis, design, and development of Software. |
| Deadline orientated and having the ability to organize and present complex solutions clearly and accurately. |
| Effective in working independently and collaboratively in a team. |
| Solution/Software Architect |
|
Python
Primary 86%
|
|---|
|
JavaScript
Secondary 20%
|
| Degree | Major | University |
|---|---|---|
| PHD | - | - |
| Master's degree | - | - |
| Bachelor's degree | Software Engineering | Islamic Azad University |
| Diploma | Computer Science | Islamic Azad University |
| Audio Mixing and Mastering | Vivid Sound Austrailia |
| SQL Database Structure
70%
|
|---|
|
NoSql Database Structure
60%
|
|
Sqlite3
Mostly Used For Django Practices 40%
|
|
MongoDB
15%
|
|
MariaDB
26%
|
|
Navicat
Database Management Platform 40%
|
| Django
Model, View, Routing, Template Engine, i18 55%
|
|---|
|
Flask
SQLAlchemy, Login, Jinja 25%
|
|
HTML5
55%
|
|
CSS3
30%
|
|
JQuery
10%
|
|
Bootstrap
50%
|
Capital is a free financial assistant that helps you spend smarter and save more. It applies artificial intelligence to learn your spending patterns, to predict your purchases, and to recommend ways to get more for your money. Capital is trained to analyze your past spending habits, to predict your future purchases, and to offer unique insights on how you can stretch your money - all through a familiar, intuitive text messaging interface.


FamilyGuy Season15 Episode01
FamilyGuy Season15 Episode02
FamilyGuy Season15 Episode03
FamilyGuy Season15 Episode04
FamilyGuy Season15 Episode05
FamilyGuy Season15 Episode06
FamilyGuy Season15 Episode07
FamilyGuy Season15 Episode08
FamilyGuy Season15 Episode09


Doug Moore
About
Portfolio
Contact
Entering your info below is one way to contact me
Contact
Name
Email
Message
Submit
Contact Form submission
×
This isn't functional, yet. It is just for visual effect for now.
You can reach me instead using the GitHub or LinkedIn buttons below.
^


salaryCalculator
You want to know how much you worth?
Start now!!
How old are you?
Less than 22
23 to 35
36 to 45
45 and above
Education
PH.d and Masters
BSc.
Secondary School
Illiterate
Next >>
Back
Job Experience
No Experience at all
3 to 5 years
6 to 10 years
Professional
Performance in class
Top 10
11 to 25
26 to 40
Below Average
Next >>
Previous
Marital Status
Single
Married
Divorced/Seperated
Confused
Sex
Male
Female
Both
Next >>
Previous
How many N5 stamps are there in a score?
4
20
2
I don't know
Your car breaks down in the middle of nowhere, what would you do?
Cry
Curser your bad luck
Hitch a ride
I don't own a car
Next >>
Previous
What's your job?
Doctor
Nurse
Old
I don't know
Education
Highest Level
Secondary School
Illiterate
I don't know
Finish!!
Previous
You want to know how much you worth?
Start now!!


A quick, one-hour introduction to Android app development
Course source materials at on github.
Taught by Audrey Troutt / @auditty
You should..
Set aside an hour to complete these tasks (a lot of waiting for downloads)
HINT: Press Down Arrow ↓ to navigate to the sub-sections below.
If you don't have it already, download and install Android Studio.
This is the editor we will use to write our apps.
Once Android Studio is installed, go to the Android SDK Manager inside Android Studio (Tools -> Android -> SDK Manager) and check two things:
Download the code for this class:
In Android Studio, choose to "Open an existing Android Studio project" (Or go to File -> New -> Import Project)
Choose the AndroidBeginnersFirstApp folder on your computer
Android Studio will prompt you to install a lot of components in order to be able to build the app. Follow all of the suggestions.
If you have an an Android phone enable developer options so that you can install and run your own apps.
An Android phone is not required for this course.
If you *don't* have an an Android phone:
If you just can't wait to get started, here's some resources to start learning now:
HINT: This is the end of the section, press Right Arrow → to navigate to the next section.
HINT: Press ESC to enter the slide overview.
Additional lessons and projects based on this same app are available for you to complete at home:


| Changelist/Build | Owner | Time Started | State | Metrics | Build | Unit Test | Func. Test |
|---|
| computer line_style {{record.id}} | {{record.owner}} | {{record.timeStartedDate | date: 'MM/dd/yyyy HH:mm'}} | {{record.state}} | {{getIcon(record.state)}} label | {{getIcon(record.state)}} label | {{getIcon(record.state)}} label | {{getIcon(record.state)}} label |
{{recordDetail.testArrow}}{{recordDetail.testScore}} Test |
{{recordDetail.maintainabilityArrow}}{{recordDetail.maintainabilityScore}} Maintainability |
{{recordDetail.securityArrow}}{{recordDetail.securityScore}} Security |
{{recordDetail.workmanshipArrow}}{{recordDetail.workmanshipScore}} Workmanship |
computer Debug |
computer Release |
{{recordDetail.timeEndedDate | date: 'MM/dd/yyyy HH:mm'}} |
| Test Coverage: {{recordDetail.funcTestCoverage}}% |
| Tests Passed: {{ recordDetail.funcTestSuccess | number : '1.2-2'}}% |


This guide documents the Ruby on Rails API documentation guidelines.
After reading this guide, you will know:
The Rails API documentation is generated with RDoc. Please consult the documentation for help with the markup, and also take into account these additional directives.
Write simple, declarative sentences. Brevity is a plus: get to the point.
Write in present tense: “Returns a hash that…”, rather than “Returned a hash that…” or “Will return a hash that…”.
Start comments in upper case. Follow regular punctuation rules:
# Declares an attribute reader backed by an internally-named # instance variable. def attr_internal_reader(*attrs) ... end
Communicate to the reader the current way of doing things, both explicitly and implicitly. Use the idioms recommended in edge. Reorder sections to emphasize favored approaches if needed, etc. The documentation should be a model for best practices and canonical, modern Rails usage.
Documentation has to be concise but comprehensive. Explore and document edge cases. What happens if a module is anonymous? What if a collection is empty? What if an argument is nil?
The proper names of Rails components have a space in between the words, like “Active Support”. ActiveRecord
is a Ruby module, whereas Active Record is an ORM. All Rails documentation should consistently refer to Rails components by their proper name, and if in your next blog post or presentation you remember this tidbit and take it into account that’d be phenomenal.
Spell names correctly: Arel, Test::Unit, RSpec, HTML, MySQL, JavaScript, ERB. When in doubt, please have a look at some authoritative source like their official documentation.
Use the article “an” for “SQL”, as in “an SQL statement”. Also “an SQLite database”.
Please use American English (color, center, modularize, etc).. See a list of American and British English spelling differences here.
Choose meaningful examples that depict and cover the basics as well as interesting points or gotchas.
Use two spaces to indent chunks of code–that is, for markup purposes, two spaces with respect to the left margin. The examples themselves should use Rails coding conventions.
Short docs do not need an explicit “Examples” label to introduce snippets; they just follow paragraphs:
# Converts a collection of elements into a formatted string by # calling +to_s+ on all elements and joining them. # # Blog.all.to_formatted_s # => "First PostSecond PostThird Post"
On the other hand, big chunks of structured documentation may have a separate “Examples” section:
# ==== Examples # # Person.exists?(5) # Person.exists?('5') # Person.exists?(name: "David") # Person.exists?(['name LIKE ?', "%#{query}%"])
The results of expressions follow them and are introduced by “# => “, vertically aligned:
# For checking if a fixnum is even or odd. # # 1.even? # => false # 1.odd? # => true # 2.even? # => true # 2.odd? # => false
If a line is too long, the comment may be placed on the next line:
# label(:post, :title) # # => Title # # label(:post, :title, "A short title") # # => A short title # # label(:post, :title, "A short title", class: "title_label") # # => A short title
Avoid using any printing methods like puts
or p
for that purpose.
On the other hand, regular comments do not use an arrow:
# polymorphic_url(record) # same as comment_url(record)
As a rule of thumb, use filenames relative to the application root:
config/routes.rb # YES
routes.rb # NO
RAILS_ROOT/config/routes.rb # NO
Use fixed-width fonts for:
nil
, false
, true
, self
.class Array # Calls +to_param+ on all its elements and joins the result with # slashes. This is used by +url_for+ in Action Pack. def to_param collect { |e| e.to_param }.join '/' end end
WARNING: Using a pair of +...+
for fixed-width font only works with words; that is: anything matching \A\w+\z
. For anything else use <tt>...</tt>
, notably symbols, setters, inline snippets, etc.
When “true” and “false” are English words rather than Ruby keywords use a regular font:
# Runs all the validations within the specified context. # Returns true if no errors are found, false otherwise. # # If the argument is false (default is +nil+), the context is # set to <tt>:create</tt> if <tt>new_record?</tt> is true, # and to <tt>:update</tt> if it is not. # # Validations with no <tt>:on</tt> option will run no # matter the context. Validations with # some <tt>:on</tt> # option will only run in the specified context. def valid?(context = nil) ... end
In lists of options, parameters, etc. use a hyphen between the item and its description (reads better than a colon because normally options are symbols):
# * <tt>:allow_nil</tt> - Skip validation if attribute is +nil+.
The description starts in upper case and ends with a full stop—it’s standard English.
Methods created with (module|class)_eval(STRING)
have a comment by their side with an instance of the generated code. That comment is 2 spaces away from the template:
for severity in Severity.constants class_eval <<-EOT, __FILE__, __LINE__ def #{severity.downcase}(message = nil, progname = nil, &block) # def debug(message = nil, progname = nil, &block) add(#{severity}, message, progname, &block) # add(DEBUG, message, progname, &block) end # end # def #{severity.downcase}? # def debug? #{severity} >= @level # DEBUG >= @level end # end EOT end
If the resulting lines are too wide, say 200 columns or more, put the comment above the call:
# def self.find_by_login_and_activated(*args) # options = args.extract_options! # ... # end self.class_eval %{ def self.#{method_id}(*args) options = args.extract_options! ... end }