Welcome to Chapter 2 of the "Implementing a language with LLVM in Objective Caml" tutorial. This chapter shows you how to use the lexer, built in Chapter 1, to build a full parser for our Kaleidoscope language. Once we have a parser, we'll define and build an Abstract Syntax Tree (AST).
The parser we will build uses a combination of Recursive Descent Parsing and Operator-Precedence Parsing to parse the Kaleidoscope language (the latter for binary expressions and the former for everything else). Before we get to parsing though, lets talk about the output of the parser: the Abstract Syntax Tree.
The AST for a program captures its behavior in such a way that it is easy for later stages of the compiler (e.g. code generation) to interpret. We basically want one object for each construct in the language, and the AST should closely model the language. In Kaleidoscope, we have expressions, a prototype, and a function object. We'll start with expressions first:
(* expr - Base type for all expression nodes. *) type expr = (* variant for numeric literals like "1.0". *) | Number of float
The code above shows the definition of the base ExprAST class and one subclass which we use for numeric literals. The important thing to note about this code is that the Number variant captures the numeric value of the literal as an instance variable. This allows later phases of the compiler to know what the stored numeric value is.
Right now we only create the AST, so there are no useful functions on them. It would be very easy to add a function to pretty print the code, for example. Here are the other expression AST node definitions that we'll use in the basic form of the Kaleidoscope language:
(* variant for referencing a variable, like "a". *) | Variable of string (* variant for a binary operator. *) | Binary of char * expr * expr (* variant for function calls. *) | Call of string * expr array
This is all (intentionally) rather straight-forward: variables capture the variable name, binary operators capture their opcode (e.g. '+'), and calls capture a function name as well as a list of any argument expressions. One thing that is nice about our AST is that it captures the language features without talking about the syntax of the language. Note that there is no discussion about precedence of binary operators, lexical structure, etc.
For our basic language, these are all of the expression nodes we'll define. Because it doesn't have conditional control flow, it isn't Turing-complete; we'll fix that in a later installment. The two things we need next are a way to talk about the interface to a function, and a way to talk about functions themselves:
(* proto - This type represents the "prototype" for a function, which captures * its name, and its argument names (thus implicitly the number of arguments the * function takes). *) type proto = Prototype of string * string array (* func - This type represents a function definition itself. *) type func = Function of proto * expr
In Kaleidoscope, functions are typed with just a count of their arguments. Since all values are double precision floating point, the type of each argument doesn't need to be stored anywhere. In a more aggressive and realistic language, the "expr" variants would probably have a type field.
With this scaffolding, we can now talk about parsing expressions and function bodies in Kaleidoscope.
Now that we have an AST to build, we need to define the parser code to build it. The idea here is that we want to parse something like "x+y" (which is returned as three tokens by the lexer) into an AST that could be generated with calls like this:
let x = Variable "x" in let y = Variable "y" in let result = Binary ('+', x, y) in ...
The error handling routines make use of the builtin Stream.Failure and Stream.Errors. Stream.Failure is raised when the parser is unable to find any matching token in the first position of a pattern. Stream.Error is raised when the first token matches, but the rest do not. The error recovery in our parser will not be the best and is not particular user-friendly, but it will be enough for our tutorial. These exceptions make it easier to handle errors in routines that have various return types.
With these basic types and exceptions, we can implement the first piece of our grammar: numeric literals.
We start with numeric literals, because they are the simplest to process. For each production in our grammar, we'll define a function which parses that production. We call this class of expressions "primary" expressions, for reasons that will become more clear later in the tutorial. In order to parse an arbitrary primary expression, we need to determine what sort of expression it is. For numeric literals, we have:
(* primary * ::= identifier * ::= numberexpr * ::= parenexpr *) parse_primary = parser (* numberexpr ::= number *) | [< 'Token.Number n >] -> Ast.Number n
This routine is very simple: it expects to be called when the current token is a Token.Number token. It takes the current number value, creates a Ast.Number node, advances the lexer to the next token, and finally returns.
There are some interesting aspects to this. The most important one is that this routine eats all of the tokens that correspond to the production and returns the lexer buffer with the next token (which is not part of the grammar production) ready to go. This is a fairly standard way to go for recursive descent parsers. For a better example, the parenthesis operator is defined like this:
(* parenexpr ::= '(' expression ')' *) | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e
This function illustrates a number of interesting things about the parser:
1) It shows how we use the Stream.Error exception. When called, this function expects that the current token is a '(' token, but after parsing the subexpression, it is possible that there is no ')' waiting. For example, if the user types in "(4 x" instead of "(4)", the parser should emit an error. Because errors can occur, the parser needs a way to indicate that they happened. In our parser, we use the camlp4 shortcut syntax token ?? "parse error", where if the token before the ?? does not match, then Stream.Error "parse error" will be raised.
2) Another interesting aspect of this function is that it uses recursion by calling Parser.parse_primary (we will soon see that Parser.parse_primary can call Parser.parse_primary). This is powerful because it allows us to handle recursive grammars, and keeps each production very simple. Note that parentheses do not cause construction of AST nodes themselves. While we could do it this way, the most important role of parentheses are to guide the parser and provide grouping. Once the parser constructs the AST, parentheses are not needed.
The next simple production is for handling variable references and function calls:
(* identifierexpr * ::= identifier * ::= identifier '(' argumentexpr ')' *) | [< 'Token.Ident id; stream >] -> let rec parse_args accumulator = parser | [< e=parse_expr; stream >] -> begin parser | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e | [< >] -> e :: accumulator end stream | [< >] -> accumulator in let rec parse_ident id = parser (* Call. *) | [< 'Token.Kwd '('; args=parse_args []; 'Token.Kwd ')' ?? "expected ')'">] -> Ast.Call (id, Array.of_list (List.rev args)) (* Simple variable ref. *) | [< >] -> Ast.Variable id in parse_ident id stream
This routine follows the same style as the other routines. (It expects to be called if the current token is a Token.Ident token). It also has recursion and error handling. One interesting aspect of this is that it uses look-ahead to determine if the current identifier is a stand alone variable reference or if it is a function call expression. It handles this by checking to see if the token after the identifier is a '(' token, constructing either a Ast.Variable or Ast.Call node as appropriate.
We finish up by raising an exception if we received a token we didn't expect:
| [< >] -> raise (Stream.Error "unknown token when expecting an expression.")
Now that basic expressions are handled, we need to handle binary expressions. They are a bit more complex.
Binary expressions are significantly harder to parse because they are often ambiguous. For example, when given the string "x+y*z", the parser can choose to parse it as either "(x+y)*z" or "x+(y*z)". With common definitions from mathematics, we expect the later parse, because "*" (multiplication) has higher precedence than "+" (addition).
There are many ways to handle this, but an elegant and efficient way is to use Operator-Precedence Parsing. This parsing technique uses the precedence of binary operators to guide recursion. To start with, we need a table of precedences:
(* binop_precedence - This holds the precedence for each binary operator that is * defined *) let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10 (* precedence - Get the precedence of the pending binary operator token. *) let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1 ... let main () = (* Install standard binary operators. * 1 is the lowest precedence. *) Hashtbl.add Parser.binop_precedence '<' 10; Hashtbl.add Parser.binop_precedence '+' 20; Hashtbl.add Parser.binop_precedence '-' 20; Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *) ...
For the basic form of Kaleidoscope, we will only support 4 binary operators (this can obviously be extended by you, our brave and intrepid reader). The Parser.precedence function returns the precedence for the current token, or -1 if the token is not a binary operator. Having a Hashtbl.t makes it easy to add new operators and makes it clear that the algorithm doesn't depend on the specific operators involved, but it would be easy enough to eliminate the Hashtbl.t and do the comparisons in the Parser.precedence function. (Or just use a fixed-size array).
With the helper above defined, we can now start parsing binary expressions. The basic idea of operator precedence parsing is to break down an expression with potentially ambiguous binary operators into pieces. Consider ,for example, the expression "a+b+(c+d)*e*f+g". Operator precedence parsing considers this as a stream of primary expressions separated by binary operators. As such, it will first parse the leading primary expression "a", then it will see the pairs [+, b] [+, (c+d)] [*, e] [*, f] and [+, g]. Note that because parentheses are primary expressions, the binary expression parser doesn't need to worry about nested subexpressions like (c+d) at all.
To start, an expression is a primary expression potentially followed by a sequence of [binop,primaryexpr] pairs:
(* expression * ::= primary binoprhs *) and parse_expr = parser | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream
Parser.parse_bin_rhs is the function that parses the sequence of pairs for us. It takes a precedence and a pointer to an expression for the part that has been parsed so far. Note that "x" is a perfectly valid expression: As such, "binoprhs" is allowed to be empty, in which case it returns the expression that is passed into it. In our example above, the code passes the expression for "a" into Parser.parse_bin_rhs and the current token is "+".
The precedence value passed into Parser.parse_bin_rhs indicates the minimal operator precedence that the function is allowed to eat. For example, if the current pair stream is [+, x] and Parser.parse_bin_rhs is passed in a precedence of 40, it will not consume any tokens (because the precedence of '+' is only 20). With this in mind, Parser.parse_bin_rhs starts with:
(* binoprhs * ::= ('+' primary)* *) and parse_bin_rhs expr_prec lhs stream = match Stream.peek stream with (* If this is a binop, find its precedence. *) | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c -> let token_prec = precedence c in (* If this is a binop that binds at least as tightly as the current binop, * consume it, otherwise we are done. *) if token_prec < expr_prec then lhs else begin
This code gets the precedence of the current token and checks to see if if is too low. Because we defined invalid tokens to have a precedence of -1, this check implicitly knows that the pair-stream ends when the token stream runs out of binary operators. If this check succeeds, we know that the token is a binary operator and that it will be included in this expression:
(* Eat the binop. *) Stream.junk stream; (* Okay, we know this is a binop. *) let rhs = match Stream.peek stream with | Some (Token.Kwd c2) ->
As such, this code eats (and remembers) the binary operator and then parses the primary expression that follows. This builds up the whole pair, the first of which is [+, b] for the running example.
Now that we parsed the left-hand side of an expression and one pair of the RHS sequence, we have to decide which way the expression associates. In particular, we could have "(a+b) binop unparsed" or "a + (b binop unparsed)". To determine this, we look ahead at "binop" to determine its precedence and compare it to BinOp's precedence (which is '+' in this case):
(* If BinOp binds less tightly with rhs than the operator after * rhs, let the pending operator take rhs as its lhs. *) let next_prec = precedence c2 in if token_prec < next_prec
If the precedence of the binop to the right of "RHS" is lower or equal to the precedence of our current operator, then we know that the parentheses associate as "(a+b) binop ...". In our example, the current operator is "+" and the next operator is "+", we know that they have the same precedence. In this case we'll create the AST node for "a+b", and then continue parsing:
... if body omitted ... in (* Merge lhs/rhs. *) let lhs = Ast.Binary (c, lhs, rhs) in parse_bin_rhs expr_prec lhs stream end
In our example above, this will turn "a+b+" into "(a+b)" and execute the next iteration of the loop, with "+" as the current token. The code above will eat, remember, and parse "(c+d)" as the primary expression, which makes the current pair equal to [+, (c+d)]. It will then evaluate the 'if' conditional above with "*" as the binop to the right of the primary. In this case, the precedence of "*" is higher than the precedence of "+" so the if condition will be entered.
The critical question left here is "how can the if condition parse the right hand side in full"? In particular, to build the AST correctly for our example, it needs to get all of "(c+d)*e*f" as the RHS expression variable. The code to do this is surprisingly simple (code from the above two blocks duplicated for context):
match Stream.peek stream with | Some (Token.Kwd c2) -> (* If BinOp binds less tightly with rhs than the operator after * rhs, let the pending operator take rhs as its lhs. *) if token_prec < precedence c2 then parse_bin_rhs (token_prec + 1) rhs stream else rhs | _ -> rhs in (* Merge lhs/rhs. *) let lhs = Ast.Binary (c, lhs, rhs) in parse_bin_rhs expr_prec lhs stream end
At this point, we know that the binary operator to the RHS of our primary has higher precedence than the binop we are currently parsing. As such, we know that any sequence of pairs whose operators are all higher precedence than "+" should be parsed together and returned as "RHS". To do this, we recursively invoke the Parser.parse_bin_rhs function specifying "token_prec+1" as the minimum precedence required for it to continue. In our example above, this will cause it to return the AST node for "(c+d)*e*f" as RHS, which is then set as the RHS of the '+' expression.
Finally, on the next iteration of the while loop, the "+g" piece is parsed and added to the AST. With this little bit of code (14 non-trivial lines), we correctly handle fully general binary expression parsing in a very elegant way. This was a whirlwind tour of this code, and it is somewhat subtle. I recommend running through it with a few tough examples to see how it works.
This wraps up handling of expressions. At this point, we can point the parser at an arbitrary token stream and build an expression from it, stopping at the first token that is not part of the expression. Next up we need to handle function definitions, etc.
The next thing missing is handling of function prototypes. In Kaleidoscope, these are used both for 'extern' function declarations as well as function body definitions. The code to do this is straight-forward and not very interesting (once you've survived expressions):
(* prototype * ::= id '(' id* ')' *) let parse_prototype = let rec parse_args accumulator = parser | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e | [< >] -> accumulator in parser | [< 'Token.Ident id; 'Token.Kwd '(' ?? "expected '(' in prototype"; args=parse_args []; 'Token.Kwd ')' ?? "expected ')' in prototype" >] -> (* success. *) Ast.Prototype (id, Array.of_list (List.rev args)) | [< >] -> raise (Stream.Error "expected function name in prototype")
Given this, a function definition is very simple, just a prototype plus an expression to implement the body:
(* definition ::= 'def' prototype expression *) let parse_definition = parser | [< 'Token.Def; p=parse_prototype; e=parse_expr >] -> Ast.Function (p, e)
In addition, we support 'extern' to declare functions like 'sin' and 'cos' as well as to support forward declaration of user functions. These 'extern's are just prototypes with no body:
(* external ::= 'extern' prototype *) let parse_extern = parser | [< 'Token.Extern; e=parse_prototype >] -> e
Finally, we'll also let the user type in arbitrary top-level expressions and evaluate them on the fly. We will handle this by defining anonymous nullary (zero argument) functions for them:
(* toplevelexpr ::= expression *) let parse_toplevel = parser | [< e=parse_expr >] -> (* Make an anonymous proto. *) Ast.Function (Ast.Prototype ("", [||]), e)
Now that we have all the pieces, let's build a little driver that will let us actually execute this code we've built!
The driver for this simply invokes all of the parsing pieces with a top-level dispatch loop. There isn't much interesting here, so I'll just include the top-level loop. See below for full code in the "Top-Level Parsing" section.
(* top ::= definition | external | expression | ';' *) let rec main_loop stream = match Stream.peek stream with | None -> () (* ignore top-level semicolons. *) | Some (Token.Kwd ';') -> Stream.junk stream; main_loop stream | Some token -> begin try match token with | Token.Def -> ignore(Parser.parse_definition stream); print_endline "parsed a function definition."; | Token.Extern -> ignore(Parser.parse_extern stream); print_endline "parsed an extern."; | _ -> (* Evaluate a top-level expression into an anonymous function. *) ignore(Parser.parse_toplevel stream); print_endline "parsed a top-level expr"; with Stream.Error s -> (* Skip token for error recovery. *) Stream.junk stream; print_endline s; end; print_string "ready> "; flush stdout; main_loop stream
The most interesting part of this is that we ignore top-level semicolons. Why is this, you ask? The basic reason is that if you type "4 + 5" at the command line, the parser doesn't know whether that is the end of what you will type or not. For example, on the next line you could type "def foo..." in which case 4+5 is the end of a top-level expression. Alternatively you could type "* 6", which would continue the expression. Having top-level semicolons allows you to type "4+5;", and the parser will know you are done.
With just under 300 lines of commented code (240 lines of non-comment, non-blank code), we fully defined our minimal language, including a lexer, parser, and AST builder. With this done, the executable will validate Kaleidoscope code and tell us if it is grammatically invalid. For example, here is a sample interaction:
$ ./toy.byte ready> def foo(x y) x+foo(y, 4.0); Parsed a function definition. ready> def foo(x y) x+y y; Parsed a function definition. Parsed a top-level expr ready> def foo(x y) x+y ); Parsed a function definition. Error: unknown token when expecting an expression ready> extern sin(a); ready> Parsed an extern ready> ^D $
There is a lot of room for extension here. You can define new AST nodes, extend the language in many ways, etc. In the next installment, we will describe how to generate LLVM Intermediate Representation (IR) from the AST.
Here is the complete code listing for this and the previous chapter. Note that it is fully self-contained: you don't need LLVM or any external libraries at all for this. (Besides the ocaml standard libraries, of course.) To build this, just compile with:
# Compile ocamlbuild toy.byte # Run ./toy.byte
Here is the code:
<{lexer,parser}.ml>: use_camlp4, pp(camlp4of)
(*===----------------------------------------------------------------------=== * Lexer Tokens *===----------------------------------------------------------------------===*) (* The lexer returns these 'Kwd' if it is an unknown character, otherwise one of * these others for known things. *) type token = (* commands *) | Def | Extern (* primary *) | Ident of string | Number of float (* unknown *) | Kwd of char
(*===----------------------------------------------------------------------=== * Lexer *===----------------------------------------------------------------------===*) let rec lex = parser (* Skip any whitespace. *) | [< ' (' ' | '\n' | '\r' | '\t'); stream >] -> lex stream (* identifier: [a-zA-Z][a-zA-Z0-9] *) | [< ' ('A' .. 'Z' | 'a' .. 'z' as c); stream >] -> let buffer = Buffer.create 1 in Buffer.add_char buffer c; lex_ident buffer stream (* number: [0-9.]+ *) | [< ' ('0' .. '9' as c); stream >] -> let buffer = Buffer.create 1 in Buffer.add_char buffer c; lex_number buffer stream (* Comment until end of line. *) | [< ' ('#'); stream >] -> lex_comment stream (* Otherwise, just return the character as its ascii value. *) | [< 'c; stream >] -> [< 'Token.Kwd c; lex stream >] (* end of stream. *) | [< >] -> [< >] and lex_number buffer = parser | [< ' ('0' .. '9' | '.' as c); stream >] -> Buffer.add_char buffer c; lex_number buffer stream | [< stream=lex >] -> [< 'Token.Number (float_of_string (Buffer.contents buffer)); stream >] and lex_ident buffer = parser | [< ' ('A' .. 'Z' | 'a' .. 'z' | '0' .. '9' as c); stream >] -> Buffer.add_char buffer c; lex_ident buffer stream | [< stream=lex >] -> match Buffer.contents buffer with | "def" -> [< 'Token.Def; stream >] | "extern" -> [< 'Token.Extern; stream >] | id -> [< 'Token.Ident id; stream >] and lex_comment = parser | [< ' ('\n'); stream=lex >] -> stream | [< 'c; e=lex_comment >] -> e | [< >] -> [< >]
(*===----------------------------------------------------------------------=== * Abstract Syntax Tree (aka Parse Tree) *===----------------------------------------------------------------------===*) (* expr - Base type for all expression nodes. *) type expr = (* variant for numeric literals like "1.0". *) | Number of float (* variant for referencing a variable, like "a". *) | Variable of string (* variant for a binary operator. *) | Binary of char * expr * expr (* variant for function calls. *) | Call of string * expr array (* proto - This type represents the "prototype" for a function, which captures * its name, and its argument names (thus implicitly the number of arguments the * function takes). *) type proto = Prototype of string * string array (* func - This type represents a function definition itself. *) type func = Function of proto * expr
(*===---------------------------------------------------------------------=== * Parser *===---------------------------------------------------------------------===*) (* binop_precedence - This holds the precedence for each binary operator that is * defined *) let binop_precedence:(char, int) Hashtbl.t = Hashtbl.create 10 (* precedence - Get the precedence of the pending binary operator token. *) let precedence c = try Hashtbl.find binop_precedence c with Not_found -> -1 (* primary * ::= identifier * ::= numberexpr * ::= parenexpr *) let rec parse_primary = parser (* numberexpr ::= number *) | [< 'Token.Number n >] -> Ast.Number n (* parenexpr ::= '(' expression ')' *) | [< 'Token.Kwd '('; e=parse_expr; 'Token.Kwd ')' ?? "expected ')'" >] -> e (* identifierexpr * ::= identifier * ::= identifier '(' argumentexpr ')' *) | [< 'Token.Ident id; stream >] -> let rec parse_args accumulator = parser | [< e=parse_expr; stream >] -> begin parser | [< 'Token.Kwd ','; e=parse_args (e :: accumulator) >] -> e | [< >] -> e :: accumulator end stream | [< >] -> accumulator in let rec parse_ident id = parser (* Call. *) | [< 'Token.Kwd '('; args=parse_args []; 'Token.Kwd ')' ?? "expected ')'">] -> Ast.Call (id, Array.of_list (List.rev args)) (* Simple variable ref. *) | [< >] -> Ast.Variable id in parse_ident id stream | [< >] -> raise (Stream.Error "unknown token when expecting an expression.") (* binoprhs * ::= ('+' primary)* *) and parse_bin_rhs expr_prec lhs stream = match Stream.peek stream with (* If this is a binop, find its precedence. *) | Some (Token.Kwd c) when Hashtbl.mem binop_precedence c -> let token_prec = precedence c in (* If this is a binop that binds at least as tightly as the current binop, * consume it, otherwise we are done. *) if token_prec < expr_prec then lhs else begin (* Eat the binop. *) Stream.junk stream; (* Parse the primary expression after the binary operator. *) let rhs = parse_primary stream in (* Okay, we know this is a binop. *) let rhs = match Stream.peek stream with | Some (Token.Kwd c2) -> (* If BinOp binds less tightly with rhs than the operator after * rhs, let the pending operator take rhs as its lhs. *) let next_prec = precedence c2 in if token_prec < next_prec then parse_bin_rhs (token_prec + 1) rhs stream else rhs | _ -> rhs in (* Merge lhs/rhs. *) let lhs = Ast.Binary (c, lhs, rhs) in parse_bin_rhs expr_prec lhs stream end | _ -> lhs (* expression * ::= primary binoprhs *) and parse_expr = parser | [< lhs=parse_primary; stream >] -> parse_bin_rhs 0 lhs stream (* prototype * ::= id '(' id* ')' *) let parse_prototype = let rec parse_args accumulator = parser | [< 'Token.Ident id; e=parse_args (id::accumulator) >] -> e | [< >] -> accumulator in parser | [< 'Token.Ident id; 'Token.Kwd '(' ?? "expected '(' in prototype"; args=parse_args []; 'Token.Kwd ')' ?? "expected ')' in prototype" >] -> (* success. *) Ast.Prototype (id, Array.of_list (List.rev args)) | [< >] -> raise (Stream.Error "expected function name in prototype") (* definition ::= 'def' prototype expression *) let parse_definition = parser | [< 'Token.Def; p=parse_prototype; e=parse_expr >] -> Ast.Function (p, e) (* toplevelexpr ::= expression *) let parse_toplevel = parser | [< e=parse_expr >] -> (* Make an anonymous proto. *) Ast.Function (Ast.Prototype ("", [||]), e) (* external ::= 'extern' prototype *) let parse_extern = parser | [< 'Token.Extern; e=parse_prototype >] -> e
(*===----------------------------------------------------------------------=== * Top-Level parsing and JIT Driver *===----------------------------------------------------------------------===*) (* top ::= definition | external | expression | ';' *) let rec main_loop stream = match Stream.peek stream with | None -> () (* ignore top-level semicolons. *) | Some (Token.Kwd ';') -> Stream.junk stream; main_loop stream | Some token -> begin try match token with | Token.Def -> ignore(Parser.parse_definition stream); print_endline "parsed a function definition."; | Token.Extern -> ignore(Parser.parse_extern stream); print_endline "parsed an extern."; | _ -> (* Evaluate a top-level expression into an anonymous function. *) ignore(Parser.parse_toplevel stream); print_endline "parsed a top-level expr"; with Stream.Error s -> (* Skip token for error recovery. *) Stream.junk stream; print_endline s; end; print_string "ready> "; flush stdout; main_loop stream
(*===----------------------------------------------------------------------=== * Main driver code. *===----------------------------------------------------------------------===*) let main () = (* Install standard binary operators. * 1 is the lowest precedence. *) Hashtbl.add Parser.binop_precedence '<' 10; Hashtbl.add Parser.binop_precedence '+' 20; Hashtbl.add Parser.binop_precedence '-' 20; Hashtbl.add Parser.binop_precedence '*' 40; (* highest. *) (* Prime the first token. *) print_string "ready> "; flush stdout; let stream = Lexer.lex (Stream.of_channel stdin) in (* Run the main "interpreter loop" now. *) Toplevel.main_loop stream; ;; main ()


westro presents an exhaustive collection of exclusive gifts and greeting cards to help you express your deepest feelings for your loved ones. In today’s increasingly hurried times, we get so engrossed in making a living that we almost forget to appreciate the real blessings in our lives - our people. Whether we talk about our moms who brave countless sleepless nights after we are born or that best friend who is always by our side through thick and thin, it would be safe to say that our existence wouldn’t be the same if it weren’t for our loved ones. Now is the time to let them know how you feel, without saying a word, through a range of exciting online gift ideas put together by our team of experts. Every gift listed with us not only qualifies the stringent quality benchmarks that have been synonymous with the Archies brand, but also encapsulates all the emotions you may need to convey your feelings, just the way you want.
Westro Online is a one-stop-shop to buy the perfect gifts for the ones you hold close to your heart. Whether you are looking for a birthday gift for an old friend or a Valentine’s Day token for the love of your life, we have got you covered. Our team of experts understand the variance of human emotions and spend countless hours in putting together thoughtful gifts that help you express your feelings to your loved ones. We truly believe that every human relationship is unique, with every person in our life holding an indispensable place that calls for a unique appreciation, and we, at Archies, are here to make that happen. Committed to serving as the go-to destination to buy and send gifts online, we carry a comprehensive range of gifts and greetings covering every aspect of human emotion worth cherishing.
offers you an endearing and endless variety of gift items including greeting cards, perfumes, chocolates, stationery essentials, mugs, soft toys and much more, for both him and her. Over the years, we have brought to India some of the world’s most famous gifting brands, such as Garfield, Smiley World, Cow Parade and Hallmark. Explore Archies’ extensive collection of online greetings and gifts and send your love to your loved ones anywhere in the world. Here’s a glimpse of our exclusive gifts collection:


In this heart-thumping nuclear power plant manager, ensure you don’t lose your power output while finding the balance of your hopeless employees. Drag and drop your flailing employees to repair broken rooms, fling irradiated waste out of the window, and stop mayhem from unraveling the power plant!
Download the game here: ajweeks.itch.io/ultimate-efficiency
Made in 48 hours for Global Game Jam 2021 by:
Made using Godot 3.2
Jam page: globalgamejam.org/2021/games/ultimate-efficiency-9
Source code available here: github.com/Game-Smashers/GGJ-2021


Protocols
The following protocols are available globally.
-
Instances of conforming types represent a document object model node that can be searched by an XPath or CSS selector and also provide the node’s underlying data in a number of ways.
In the HTML/XML DOM (Document Object Model), everything is a node:
The document itself is a document node;
All HTML/XML elements are element nodes;
All HTML/XML attributes are attribute nodes;
Text inside HTML elements are text nodes;
Comments are comment nodes.
Declaration
Swift
public protocol Node: Searchable, CustomStringConvertible
-
Instances of conforming types can use XPath or CSS selectors to form a search request.
See moreDeclaration
Swift
public protocol Searchable


|
CellularHelper
|
Things that return a simple string, like the manufacturer string, use this. More...
#include <CellularHelper.h>
Public Member Functions | |
| virtual int | parse (int type, const char *buf, int len) |
| Method to parse the output from the modem. More... | |
| Public Member Functions inherited from CellularHelperCommonResponse | |
| void | logCellularDebug (int type, const char *buf, int len) const |
| Used when enableDebug is true to log the Cellular.command callback data using Log.info. More... | |
Data Fields | |
| String | string |
| Returned string is stored here. | |
| Data Fields inherited from CellularHelperCommonResponse | |
| int | resp = RESP_ERROR |
| Response code from Cellular.command. More... | |
| bool | enableDebug = false |
| Enables debug mode (default: false) More... | |
Things that return a simple string, like the manufacturer string, use this.
Since it inherits from CellularHelperCommonResponse you can check resp == RESP_OK to make sure the call succeeded.
|
virtual |
Method to parse the output from the modem.
| type | one of 13 different enumerated AT command response types. |
| buf | a pointer to the character array containing the AT command response. |
| len | length of the AT command response buf. |
This is called from responseCallback, which is the callback to Cellular.command.
This class just appends all TYPE_UNKNOWN data into string.
Implements CellularHelperCommonResponse.


--- layout: default title: Understand JavaScript Apps og_image: http://www.fromjs.com/img/fromjs-chrome-extension.png og_description: Find the source of every HTML character in a JavaScript app. ---
FromJS observes JavaScript data flow at runtime, and helps you analyze it.
The first version of FromJS was too flaky to be useful in practice. FromJS 2 will actually work!
FromJS is a developer tool that helps you understand how a JavaScript app works.
You can select an HTML element to view where its content came from. The screenshot shows that the string "Hello" originally came from localStorage.
Other possible string sources include string literals in the app's JS code, as well as HTML served on the initial page load.
React TodoMVC Demo
Nope, most websites will break if you try to inspect them with FromJS.
FromJS uses Babel to add tracking to strings, but this results in a lot of flakiness.
A better approach would be to build the tracking into the JavaScript engine.
Non-trivial apps also take a long time to load and use up large amounts of memory.
Install the FromJS Chrome Extension to try FromJS yourself. The Github readme lists some example sites where FromJS works well.


"Life" marks yet another unexpected career move by Eddie Murphy, who's apparently been a bit more willing to take chances in the past few years after spending most of the late 1980s and early 1990s running his smoother-than-thou persona into the ground with such shaky projects as "Harlem Nights" and "Beverly Hills Cop 3."
Last year's "Holy Man" found Murphy trying to play it semi-straight; had the film been better written, it might have worked. "Life" continues in the same vein, attempting to find a middle ground between the sassy Murphy of days gone by and the slightly gentler contemporary model. The movie doesn't entirely come off, but it's still a noble stab at something different.
Though the advertising for "Life" plays up the picture's comic content, this is by no means a flat-out laugh riot. Perhaps it's Murphy and co-star Martin Lawrence's roundabout way of testing the water to see whether audiences are willing to accept them as actors, rather than simply as funnymen.
Robert Ramsey and Matthew Stone's script follows New Yorkers Ray Gibson (Murphy) and Claude Banks (Lawrence) from 1932 - when they are reluctantly pressed into service as bootleggers for crooked club owner Spanky (Rick James) - into their septuagenarian years.
Sentenced to "the long ride" on a trumped-up murder charge during a run through Mississippi, Claude and Ray find themselves locked up in the dusty Camp 8, which, though not exactly a resort, does have its little amenities; certainly this must be the only prison in the country that allows inmates to drink and carouse openly with women or men during their time off. When times are tough, Ray enthralls his fellow convicts with "Kiss of theSpider Woman"-style stories about the mythical Manhattan hot spot he'll be running when he gets out. But as years turn into decades, it becomes clear to Ray and Claude the only other address they're going to have will be in the cemetery.
Although that's a fairly sobering thought for a movie that starts off on a free-wheeling, fairly lighthearted note, director Ted Demme capably handles the transition. So does Murphy, who proves he can play a senior citizen without immediately resorting to tics. Lawrence's performance is less steady, although he does have a fairly powerful moment when he catches a reflected glimpse of his withered face in a car window.
"Life" has about an even number of pros and cons. On the positive side, Rick Baker's makeup jobs on the stars are terrific, the rhythmic score by Wyclef Jean is effective, and there are no duds in the supporting cast.
However, like "Hoodlum," the 1997 Lawrence Fishburne crime drama set during the Depression, "Life" has no real sense of period style, throwing in scores of distracting anachronisms in both attitudes and dialogue. And though this script is considerably less homophobic than some other Murphy epics, isn't it time to permanently retire the stereotype of the squealing queen with the flapping wrists?
"Life" is ultimately less satisfying as a comedy than it is as a story about men trying to deal with a future they can't change. Fans might be surprised and even a bit disappointed, but it's intriguing to see Lawrence and Murphy challenging themselves. Hopefully, the next time out will be even better.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


Where To Go
International Destinations
Customizable
8-9 Days
From RM 1000All Inclusive
From RM 2000All Inclusive
From RM 2500All Inclusive
Hiking Adventures
Exclusive Hiking and Tour Trips
Free This Weekend?
Local Destinations
Free & Easy
2-3 Days
From RM 250All Inclusive
4-5 Days
From RM 300All Inclusive
3-4 Days
From RM 350All Inclusive
Culture & Nature
From RM 500All Inclusive
Find Out More
Contact Us


@* * Copyright 2010-2013 Artima, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. *@ @(pagename: String)(body: => Html) @topImage = {
Just Released - ScalaTest 1.9.2-RC1 and 2.0.M6. The epic ScalaTest 2.0 effort is now "feature complete." Once we finish the Scaladoc documentation, give it a thorough once over, and perform any needed cleanup, we will release 2.0-RC1. For a glimpse of what's coming in ScalaTest 2.0, check out the 2.0.M6 release notes.
} @main(pagename, topImage){@body}


The House Organ
The House Organ is a DIY label for audio and other artefacts from the No-Audience Underground. Primarily sounding across a spectrum of improvisation and noise, it has taken a critical stance to digital-self-publishing whilst providing alternatives to dominant streaming and sharing models. For deep-thinkers who don’t take themselves too seriously.
Initially it shunned Bandcamp, focussing upon Crypto, Bitcoin, Torrents, Tor networks and p2p. Whilst still privacy conscious it has learned from the alienation these approaches create within the DIY music community. For more on these approaches, read An (Opinionated) Critical Guide to DIY Digital Publishing (kindly published by TQ).
The current version is exploring online streaming platforms and more user-friendly approaches with the aim of self-publishing and close-collaboration.


wiki / xorg / x11-forwarding Edit this page
Edited (8a55f77) at 2021-07-02 by Dilyn Corner
X11-FORWARDING [0]
________________________________________________________________________________
X11-Forwarding is a secure shell feature, which allows one to forward/tunnel
X11 connections through an existing SSH session. This is used to run X11
programs on a server while the ssh-client displays the graphical window through
the user's X11-server.
Dependencies
________________________________________________________________________________
In most cases, you will already have the required dependencies.
All that is required are a few modifications to configuration files that exist
on your remote X server:
+------------------------------------------------------------------------------+
| |
| $ echo "XauthLocation /usr/bin/xauth" >> /etc/ssh/sshd_config |
| $ echo "X11Fordwarding yes" >> /etc/ssh/sshd_config |
| |
+------------------------------------------------------------------------------+
At this point you are ready to test your server!
Client Configuration
________________________________________________________________________________
In order to connect to your remote server, you will need an SSH client that
supports X11-Forwarding, as well as an X server running on the same client. Some
popular cross-platform options include the following:
* vcxsrv (recommended, server only) [1]
* MobaXterm (both SSH client and X server, for Windows only) [2]
* Xming (server only) [3]
* X410 (server only) [4]
From the client side, connect to the server via SSH through your favorite
terminal application while passing the "-X" switch. Pay attention to any
errors that may occur on connection. More verbose output can be achieved by
passing the "-v" switch:
+------------------------------------------------------------------------------+
| |
| $ ssh -X -v user@localhost |
| |
+------------------------------------------------------------------------------+
You can now start any X program on the remote server, the output will be
forwarded to your local session:
+------------------------------------------------------------------------------+
| |
| $ xclock |
| |
+------------------------------------------------------------------------------+
This should create a new window with the xclock application on your client side
X server.
Use an "&" at the end of the command to prevent tying up the terminal in
question:
+------------------------------------------------------------------------------+
| |
| $ xclock & |
| |
+------------------------------------------------------------------------------+
Tips and Tricks
________________________________________________________________________________
* If your connection is slow, try enabling SSH compression by passing the "-C"
switch.
+----------------------------------------------------------------------------+
| |
| $ ssh -X -C user@localhost |
| |
+----------------------------------------------------------------------------+
* You can further improve your connection speed by using a cypher to connect to
the remove server. This can be passed as an argument using the "-c" switch
at the initialization of a new SSH connection. [5]
+----------------------------------------------------------------------------+
| |
| $ ssh -X -C -c aes256-ctr user@localhost |
| |
+----------------------------------------------------------------------------+
* Your remote system most likely has many cypher options already available for
you to choose from (es128-ctr, aes192-ctr, aes256-ctr, arcfour256, arcfour128,
aes128-cbc, 3des-cbc, blowfish-cbc, cast128-cbc, aes192-cbc, aes256-cbc,
arcfour, etc.) and each will vary in performance and security. Check out
websites that benchmark the various security cyphers and choose the one that
works best for you. [6]
* Can you forward an entire desktop session? Why yes, you can! Instructions for
doing so vary per client, server configuration, and platform [7]. If you chose
vcxsrv as your client on a Windows host, then I would recommend checking out
the following youtube video:
"Linux and Windows | X11 Forwarding with SSH | VNC Alternative" by knary
https://www.youtube.com/watch?v=UWlsS6Jaibc
References
________________________________________________________________________________
[0] https://wiki.archlinux.org/index.php/OpenSSH#X11_forwarding
[1] https://sourceforge.net/projects/vcxsrv/
[2] https://mobaxterm.mobatek.net/
[3] http://straightrunning.com/XmingNotes/
[4] https://x410.dev
[5] https://cyberciti.biz/faq/speeding-up-ssh-x11-forwarding-with-unix-osx-linux-bsd/
[6] https://blog.famzah.net/2010/06/11/openssh-ciphers-performance-benchmark/
[7] https://blog.warbel.net/index.php/2018/02/21/using-xnest-or-putty-vcxsrv-to-start-a-full-remote-session/
________________________________________________________________________________
Dylan Araps (C) 2019-2020
kiss-community (C) 2020-2021
Dilyn Corner (C) 2021
Linux(R) is the registered trademark of Linus Torvalds in the U.S. and
other countries.


Changelog
Public alpha - feature incomplete
Fixes
-
all art assets finished and in place
-
increased overall game speed
-
increased bird rotation when falling
-
fixed the skybox bug where you could fly over obstacles
-
added sound assets (may still have issues on mozilla)
To do list
-
fix collision, it's a little wonky
-
minify everything to increase performance
-
work out why the cookies aren't saving scores correctly
-
add social network sharing
-
add database integration for hi-scores


The key to understanding recursion is to begin by understanding recursion. The rest is easy. -- Koenig/Moo, Accelerated C++ [source]


Like Home Made
Sure to make everyone enjoy vegetarian food.
We Are
Food Galaxy managed by Bappa's Kitchen with the simple aim of providing consumers with great tasting, nutritious, ready-to-eat meals.
Our Services
Our service includes catering to the food requirements of small gatherings, of weddings and receptions, tea parties, Shashtipoorthi celebrations, birthday parties, office get-together's etc. We have perfect blend of experience and creativity to make your occasion a special success. We offer best quality, best quantity and best affordable price to our valuable Customers.
About Us
We were founded in 2016, we are now a well known and established catering service provider in Pune. Where food and style is the purpose, Food Galaxy will provide you with the best.We prepare special feasts for various occasions like Marriages, Family Gatherings, Corporate Meets etc. We also have carved out a position in this business by our dedication, promptness and quality.
Gallery
Send Inquiry
Contact Us
We are incredibly responsive to your requests and value your questions.


Version 1.0
Simple Cookie Consent Layer.
Currently only technically necessary cookies supported!
Include in head:
<link rel="stylesheet" type="text/css" href="novaconsent.css" />
Include at the end of the body:

You can use your own configuration. Just include a javascript variable with the name novaConsentConfig
and overwride the values as your wish. It's not necessary to overwrite every value.


Look at this beardGlorious!
I nearly robbed
the world of its majesty
In 2012 I averaged 70 hours per week on the computer
And it wasn’t unusual to pass 90.
My long hours were a point of pride for me
I landed a huge contract:
A high-profile, Black Friday marketing site for a Fortune 500 company — charging agency rates and rush fees.
We planned it all out...
...then reality happened
I worked over 100 hours that week
The Business Result:
- The client loved it
- The project made the client a bunch of money
- I got paid a bunch of money
- The project won an Addy Award
The Personal Cost:
- I missed Thanksgiving with my family
- I slept less than 4 hours a night
- I ate fast food delivery for every. single. meal.

Losing my beard
was the final straw.
Poor & healthy
is better than successful & sick
I cut back on everything:
- I only accepted calls during business hours
- I turned off ALL notifications
- I turned on Do Not Disturb mode from 6pm to 8am
- I only checked email twice a day
- I set up activities with my friends...
- ...and left my phone in the car while I was there
I was ready to lose business if I could have my life back
…but that’s not what happened
What I expected:
my clients would leave
I’d miss big opportunities
I’d accomplish less
What happened:
no one noticed
my business grew
I got more done
“How is it possible that you got more done by working fewer hours?”
Working 60+ hours per week makes us less productive than working 40 hours1
Long hours destroy passion and engagement2
Overworking increases employee turnover3
Creativity suffers if we don’t take time to go outside and be people4
Lack of downtime
hurts productivity
and creativity5
We lose 20% of our time per task while multitasking6
Sleeping less than 7 hours impairs us as much as being too drunk to drive7
“You haven’t answered the question, nerd.”
5 Strategies to
Boost Productivity
- Track your time (passively)
- Time-box tasks
- Front-load important work
- Prioritize aggressively
- DO NOT multitask
Strategy #1:
Track Your Time
The simple act of acknowledging how we spend our time encourages us
to use it more wisely.
If I have a long day...
...I know to take a day off
Strategy #2:
Time-Box Tasks
You’re never more productive than when you’re up against a deadline with
13% battery left.
Find your rhythm.
- Decide how long you want to focus for
- Close and turn off everything unrelated
- Start a timer
- Unleash your brilliance upon the world
- When the timer goes off, take a break
- Repeat
1. We’re not distracted
2. We get into a flow state
3. We’re taking breaks
4. We’re still available
Also: science
We all have something called an “ultradian rhythm”, which regulates our
energy in 90-minute-ish intervals.8
Strategy #3:
Front-Load Important Work
Motivation and self-control are resources that deplete as the day goes on.
Don’t waste them.
We’re the best versions of ourselves early in our day
- energy is higher
- focus is improved
- self-control is at its peak
Morning Me
Evening Me
Strategy #4:
Prioritize Aggressively
If everything is The Most Important Thing™, nothing is.
Use the Highlander Technique
Put tasks into a tournament bracket...
...and then start choosing.
Strategy #5:
DO NOT Multitask
Ever. Just stop it.
Multitasking makes us
feel productive.
Single-tasking makes us actually productive.
“How do I know that any of this junk really works?”
My 2016 RescueTime report
Average: 37 hours per week
My 2017 RescueTime report
Average: 39 hours per week
I’ve been able to manage:
- My IBM workload
- My blogs and side projects
- Multiple conference sessions and workshops
- A reasonably full social calendar
- 3× weekly workouts
- 8 hours average sleep
If we work too much, we’re:
- worse at our jobs
- more likely to burn out
- less likely to feel satisfied at our job
- less able to enjoy our lives
If we work a reasonable schedule, we’re:
- more creative
- more productive
- happier (both in and out of work)
- healthier
We can be more productive in less time by:
- paying attention to how we spend our time
- time-boxing tasks to create urgency & limit distractions
- front-loading demanding tasks while our energy is high
- aggressively prioritizing work to enable better focus
- never, ever, ever multitasking again


Web Development
Building websites don’t only look great, they also perform.
Technologies: JavaScript with React, NodeJS with Express Hapi and more.
App Development
Building experience that taps into your audience and grows your business online.
Technologies: React Native, Ionic, Cordova and more.
Hardware Development
Building custom prototypes and creating beautiful POCs.
Technologies: Arduino, RaspberryPi and more.


--- layout: default title: Articles at BitByteYum ---
{{ p.title }} {{p.date | date: "%b %Y" }} on {{p.date | date: "%B %d, %Y" }}


Privacy
Introduction
This Privacy Policy relates to tripBuddy's collection and use of your data when you use or access the tripBuddy's service, websites, servers, software applications, or content therefrom (together, the "tripBuddy Service" or "Service"). This Privacy Policy is incorporated by reference into tripBuddy's Terms and Conditions. Capitalized terms not defined in this Privacy Policy shall have the meanings ascribed to them in the Terms and Conditions.
Collection and Use of Personally Identifiable Information
Personally identifiable information ("PPI") is information that can be used to identify an individual person. Examples of PII include a person's name, email address, or phone number. We may receive and store any PII that you provide on our Service, send to us in an email or voluntarily provide to us in any other way. As you use, access, or register for the Service, you may be asked to provide certain PII.
HERE'S THE PERSONAL INFORMATION WE COLLECT
The information we gather from users enables us to personalize and improve our Services, and allows our users to set up a user account and profile that can be used to personalize their experience on the Site. We collect the following types of information ("Personal Information") from our users:
Personal Information You Provide to Us:
We collect and store the following types of information on our servers only when you optionally choose to provide it to us by filling out a form or taking some other explicit action. You can choose not to provide us with any of the information below, but you may not be able to take advantage of certain features without creating an account or providing the appropriate information.
HERE'S HOW WE USE YOUR INFORMATION:
In addition to the specific uses discussed above, information you submit to us may be used in the following ways:
HERE'S HOW WE DISCLOSE AGGREGATE INFORMATION
We may combine your Personal Information and information about your preferences and activities, with those of all or a particular group of our users to prepare collective profiles of our users and their activities for our internal use. For example, we may track the number of our users who use certain features or compile the total dollar amount of trips conducted through the app in a particular geographic region or track purchasing trends. Such information, and other information we derive from individual users' information, allows us to continue to grow and provide you with quality information through more accurate marketing and advertising of the app.
HERE'S WHAT WE DO TO KEEP YOUR INFORMATION SECURE:
We take commercially reasonable security measures to protect against unauthorized access to, or unauthorized alteration,
disclosure or destruction of your Personal Information that you share and we collect and store.
These security measures include read-only replicas of our databases, high durability storage, full daily backups,
secure encryption using Amazon S3 (which is designed for 99.99% availability and 99.999999999% durability),
SSL encryption, data storage across multiple facilities that exceed industry standards, internal reviews of our
data collection practices and platforms, as well as physical security measures to guard against
unauthorized access to systems where we store your information. The technology we have employed to
deliver our Services to you protects your information using both server authentication and data encryption,
ensuring that your data is safe, secure, and available only to registered users in your organization.
Unfortunately, no data transmission over the Internet or any wireless network can be guaranteed to be 100% secure.
As a result, while we strive to protect your personal information: (a) there are security and privacy
limitations of the Internet which are beyond our control; (b) the security, integrity and privacy of
any and all information and data exchanged between you and us through this Site cannot be guaranteed
and you transmit such information at your own risk; and (c) any such information and data may be
viewed or tampered with in transit by a third party.
HERE'S WHAT YOU CAN DO TO PROTECT YOUR PERSONAL INFORMATION:
While we are committed to taking all reasonable precautions to protect your Personal Information, there are steps you can take as well; such as:
HERE'S HOW YOU CAN UPDATE, DELETE, OR CORRECT YOUR PERSONAL INFORMATION
We believe you should have the ability to access, edit, and delete the Personal Information that you have provided to us and encourage you to promptly update your Personal information if it changes. You may change any of your Personal Information in your Account on the app at any time. If you are concerned about the accuracy of Personal Information maintained by us or wish to review, access, amend or correct it, or would like your Account deleted or your Personal Information removed from our records or otherwise deactivated, you may contacting us with your request at<EMAIL_ADDRESS>We may decline requests that are unreasonably repetitive, require disproportionate technical effort, jeopardize the privacy of others, or are extremely impractical. In some cases, we may not be able to remove your Personal Information, in which case we will let you know if we are unable to do so and why.
HERE'S WHAT WILL HAPPEN IF WE CHANGE THIS POLICY:
We may amend this Policy from time to time. Use of information we collect now is subject to the Policy in effect at the time such information is used. If we make changes that, in our sole discretion, we consider to be material, we will notify you by delivering an announcement via our Service to the email address associated with your account or by means of a notice on this Site prior to the change becoming effective. It is your responsibility to maintain a valid e-mail address as a registered user. If you opt out of communications from us, you may not receive these notifications, however they will still govern your use of the Service, and you are responsible for checking for any changes. If you continue to use the app or the Service after changes become effective, you agree to abide by and be bound by the modified privacy policy
A SPECIAL NOTE ABOUT CHILDREN
Our app is not designed or intended for use by children under 18. If you are under 18, you should use the Site and Services only with involvement of a parent or guardian. Children under 18 may not submit any personally identifiable information to us, and if we discover that we have inadvertently gathered any such information from a child under 13, we will take appropriate steps to delete it. If you are the parent or guardian of a person under the age of 18 who has provided personally identifiable information to us, please inform us by contacting us at<EMAIL_ADDRESS>and we will remove such information from our database.
HERE'S HOW YOU CAN CONTACT US:
If you have any questions about this Privacy Policy, the practices of this Site, or your dealings with this Site, please contact us by email at<EMAIL_ADDRESS>or by sending a letter to:


Like many things in Twisted, Minerva supports producers/consumers for efficient high-volume streaming. [1]. In Twisted, pressure information from consumers controls the creation of bytes. In Minerva, it controls the creation of frames, not bytes.
In Minerva, a producer can be attached to the ServerStream. Usually, an IStringProtocol will perform this attachment.
In general, TCP pressure from the TCP connection of the primary transport directly affects the producer attached to ServerStream. Also, if the producer is a push producer, the producer is paused while there are no Minerva transports attached to the ServerStream.
The implementation is complicated because Minerva transports may frequently attach and detach from the ServerStream. Producers/consumers technical details describes what really happens. However, it does "just work".
| [1] | http://twistedmatrix.com/projects/core/documentation/howto/producers.html |
Skip this section unless you are trying to understand the producer/consumer code in minerva.mserver.
"Type of producer" is pull, or push. [2]
This is the object chain, "upstream" objects are at the top. Objects on adjacent lines usually know about each other (have references).
Producer attachment goes downstream, pressure information goes upstream.
When a client connects, (Twisted) causes *Transport creation, which causes ServerStream creation, which causes IStringProtocol creation. This might not happen instantly, because *Transport must be authenticated first. At this time, there are no producers in the system.
At any time, a pull or push producer can be registered with ServerStream. The producer can be unregistered at any time. Usually, an IStringProtocol will do the registration and unregistration.
ServerStream's goal is to register the same type of producer with every primary transport that attaches to it, even if the primary transport isn't attached yet (or not yet primary). ServerStream must also unregister producers from transports that are no longer primary transports.
If type of producer is push, ServerStream must also call pauseProducing on IStringProtocol whenever there is no primary transport. It must also call resumeProducing when this situation ends.
*Transport's job is simple, it just registers itself as the correct type of producer with (Twisted). One edge case: it must remember if (Twisted) paused it, and if so, pauseProducing newly-attached push producers.
During normal operation for a registered pull producer, these conditions result in resumeProducing calls on IStringProtocol:
During normal operation for a registered push producer, these conditions result in pauseProducing or resumeProducing calls on IStringProtocol:
| [2] | http://twistedmatrix.com/projects/core/documentation/howto/producers.html |
Minerva client supports registering a special pull producer that will be pulled right before Minerva client makes any HTTP request. This is useful if Minerva client is using HTTP as primary transport, and client application wants to occasionally upload data without incurring the cost of a C2S HTTP request. If Minerva is using HTTP as primary transport, the pull producer will be pulled around every NN seconds.


Beta This is a test version of the layout of this page. Take the survey to help us improve it
Statutory guidance
The standards that school and childcare providers must meet for the learning, development and care of children under 5.
The framework:
This statutory framework is for:
It relates to:
Statutory guidance is issued by law; you must follow it unless there’s a good reason not to.
The EYFS framework published in March 2014 remains in force until 2 April 2017. The EYFS framework published on 3 March 2017 will come into force on 3 April 2017.
Supporting material is available on the Foundation Years website.


{% if title %}
{{ msg }}
Trail Route Analytics: Copyright 2015 Nekomatayama Studio. powered by GPX.py licensed under the Apache License, Version 2.0Development Blog


In the Gregorc Mind Styles model, I'm an ABSTRACT SEQUENTIAL thinker.
Not by a small margin, either - I fall very squarely into this camp, and the profile it gives of me seems pretty accurate. I found myself agreeing with most of the descriptors of my personality type and learning experiences.
I did some research into the Gregorc model and stumbled upon an academic paper, however, that made me question what this model says about me.
About how firmly I fell into one camp, I encountered this critique of the Gregorc model:
[t]he structure of the Delineator's protocol makes it extremely easy for the individual completing the test to "decipher" how the test works. This greatly enhances the probability that, after an individual rates one or two sets of words, a conscious or unconscious effort will be made to rate the rest of the word sets consistently, creating spuriously high Alpha coefficients.
Come to think of it, I remember pretty clearly taking this test and after a few questions starting to sense a certain pattern to the answer options. I certainly felt a certain amount of pressure to test consistently - times when I felt hung up by an answer, I always chose the one that I felt best fit with the pattern of answers I had already given.
So much for the solidity of my type! I had almost certainly done that myself, out of a desire to feel like a consistent human being.
What of how accurate the descriptions seemed of my personality seemed? Next I found this bit of criticism:
This procedure for validating an assessment tool is much like reading your horoscope in the newspaper at the end of the day. In most instances you can recall at least one situation or event which occurred during the day wtiich corresponds to the prediction made. In fact one could probably mix-up the predictions assigned to the various astrological signs and still get a high level of agreement.
To answer this, I went back to the four learning types and asked myself how well the other three seemed to fit with me. And man, I have to say, a whole lot of it seemed to describe me perfectly!
I don't mean to totally discredit this line of analysis, but I think it's important to take this kind of thing with a very big grain of salt. Being aware of myself and my emotions as I plunge into a very intense learning environment is tremendously important. But I think I can do that without necessarily buying completely into a somewhat discredited (or at least seriously challenged) theory of learning types.


--- layout: page title: "About" description: "" header-img: "img/about-bg.jpg" ---
Hi, I'm William.guo, a Develop Engineer. On this blog, I intend to write about my projects and side-projects.


The AES (CBC) encryption algorithm (also known as Rijndael) works with a variable block length e.g. 128 bits, and a variable key length e.g. 128, 192 and 256 bits.
CrypTool uses Rijndael in CBC mode with zero initialization vector and 01-00 padding.
The key is entered in the Key entry dialog.
CrypTool contains a brute-force attack on the Rijndael encryption algorithm.
The Rijndael Algorithm has been proposed for FIPS standardisation. Therefore it will be the successor of the antiquated Data Encryption Standard (DES) algorithm (see AES winner).
Remark:
The funny cartoon "A Stick Figure Guide to the Advanced Encryption Standard (AES)" by Jeff Moser,
published in September 2009, explains the AES algorithm in 4 acts:
http://www.moserware.com/2009/09/stick-figure-guide-to-advanced.html


Running a Distributed Simulation
When distributed simulation starts working, a simulation on each machine will start, and you can check the list of sensors allocated to the machine in Sensor Visualizers.
Requirements top#
Distributed simulation needs multiple computers connected to the same router. The operating system of each computer can be either Linux or Windows 10. In this document, we'll use:
- Two Linux PCs (Master, Client PC)
- Apollo 5 on Master PC
Instructions top#
Install Apollo 5 top#
If you already installed and built Apollo 5, jump to launching Apollo.
docker pull lgsvl/apollo-5.0
git clone --recurse-submodules https://github.com/lgsvl/apollo-5.0.git ~/apollo-5.0
Building Apollo and bridge top#
cd ~/apollo-5.0
./docker/scripts/dev_start.sh
./docker/scripts/dev_into.sh
./apollo.sh build_gpu
Launching Apollo 5 top#
bootstrap.sh
bridge.sh
You can check the network address of your bridge. This network address should be used in bridge setup in the web user interface.
Launching SVL Simulator and Setting Cluster top#
In Master Simulator,
-
Run Simulator.
-
Click LINK TO CLOUD.
-
In the Create new cluster tab, give cluster name into New Cluster and click Create cluster.
In Client Simulator,
-
Run Simulator.
-
Click LINK TO CLOUD.
-
In the Add to Existing cluster tab, choose cluster name that master simulator made and click Create cluster.
-
After this, Clusters should show one cluster which has two computer name together. If this setup doesn't work, you should remove your pre-existed cluster.
-
Make sure that you now have a cluster which has each machine listed together, like the following:
Adding a Map top#
If you already added a map you want to use, jump to Adding a vehicle.
- In the left tab, under Store, click on Maps and enter a map's name. Here we're gonna use BorregasAve map. Click on + icon. You can check the message in the left bottom like the following: "BorregasAve" has been successfully added to your library.
Adding a Vehicle top#
If you already added a vehicle you want to use, jump to Creating simulations.
- In the left tab, under Store, click on Vehicles and enter a vehicle's name. Here we're gonna use Lincoln2017MKZ. Click on + icon. You can check the message in the left bottom like the following: "Lincoln2017MKZ" has been successfully added to your library.
Adding sensors top#
-
In the left tab, click Vehicles under Library.
-
Choose Lincoln2017MKZ.
-
Click Apollo 5.0 under Sensor Configurations.
- If you see exclamation mark, click Add to Library button.
Creating Simulations in Web User Interface top#
Below is how to make a new simulation with a distributed simulation cluster setup.
-
In the left side, click Simulations.
-
Click Add New.
-
Give name in the Simulation Name.
-
Select cluster name in the Select Cluster.
-
Choose other setups like test report, headless mode, interactive mode as you wish and click next.
- Select Random Traffic in Runtime Template.
- Choose map and vehicle essentially and others optionally and click next.
- Choose Apollo 5.0 in the Autopilot and enter exact your IP address plus port 9090 where Apollo 5 is running and click next.
- Click Publish.
Running Simulation top#
Click Run Simulation.
Make sure the simulation is started on both the master and client machines.
Checking Bridge Status top#
- Click a power cord shape icon in the bottom and make sure that master and client simulator have connected to bridge in the left tab.
Visualizing Sensors top#
-
Click the an eye shape icon and make sure master and client simulators have own sensor lists automatically allocated by distributed simulation setup.
-
Make sure that master simulator has light-computational sensors like CAN bus, IMU, etc. and the client has heavy-computational sensors like LiDARs, Cameras.


Pan the map to select a new area
LNG
LAT
RESET
GO
Source Code
Built on top of
ClayGL
by
@pissang
Vector tile data is from
nextzen
.


Package net.glowstone.util
package net.glowstone.util
-
InterfaceDescriptionA map whose values are of variable types known to those invoking the getters.Non-boxing version of
{@link java.util.function.Consumer}<Float>
.Non-boxing version of{@link java.util.function.Consumer}<Short>
. -
ClassDescriptionA small utility class that allow to maintain a
BlockState
's list in order to capture a chain of modified blocks and update all the states in once, or never (ie: event cancelled).Utility methods for dealing with entities.Container for the game rule map for worlds.An implementation ofHelpMap
.ACachedServerIcon
implementation.Deprecated.An array of nibbles (4-bit values) stored efficiently as a byte array of half the size.Identity methods used to prevent inlining of constant.A static class housing position-related utilities and constants.Utility class for performing encrypted authentication.Thread started on shutdown that monitors for and kills rogue non-daemon threads.A class that encapsulates relevant data for playing sounds (volume and pitch).A collection of utility methods to play sounds.A container for statistic tracking.Simple container for chat message structures until more advanced chat formatting is implemented.Utility class to ease conversion from standard time units to in-game ticks.Utility methods for dealing with UUIDs. -
Enum ClassDescriptionCompatibility bundles are bundles of libraries that other servers include in their servers but Glowstone does not.


5.2. nlp¶
Some notes on natural language processing, focused on modern improvements based on deep learning.
5.2.1. nlp basics¶
basics come from book “Speech and Language Processing”
language models - assign probabilities to sequences of words
ex. n-gram model - assigns probs to shorts sequences of words, known as n-grams
for full sentence, use markov assumption
eval: perplexity (PP) - inverse probability of the test set, normalized by the number of words (want to minimize it)
\(PP(W_{test}) = P(w_1, ..., w_N)^{-1/N}\)
can think of this as the weighted average branching factor of a language
should only be compared across models w/ same vocab
vocabulary
sometimes closed, otherwise have unkown words, which we assign its own symbol
can fix training vocab, or just choose the top words and have the rest be unkown
topic models (e.g. LDA) - apply unsupervised learning on large sets of text to learn sets of associated words
embeddings - vectors for representing words
ex. tf-idf - defined as counts of nearby words (big + sparse)
pointwise mutual info - instead of counts, consider whether 2 words co-occur more than we would have expected by chance
ex. word2vec - short, dense vectors
intuition: train classifier on binary prediction: is word \(w\) likely to show up near this word? (algorithm also called skip-gram)
the weights are the embeddings
also GloVe, which is based on ratios of word co-occurrence probs
some tasks
tokenization
pos tagging
named entity recognition
nested entity recognition - not just names (but also Jacob’s brother type entity)
sentiment classification
language modeling (i.e. text generation)
machine translation
hardest: coreference resolution
question answering
natural language inference - does one sentence entail another?
most popular datasets
(by far) WSJ
then twitter
then Wikipedia
eli5 has nice text highlighting for interp
5.2.2. dl for nlp¶
some recent topics based on this blog
rnns
when training rnn, accumulate gradients over sequence and then update all at once
stacked rnns have outputs of rnns feed into another rnn
bidirectional rnn - one rnn left to right and another right to left (can concatenate, add, etc.)
standard seq2seq
encoder reads input and outputs context vector (the hidden state)
decoder (rnn) takes this context vector and generates a sequence
misc papers
Deal or No Deal? End-to-End Learning for Negotiation Dialogues - controversial FB paper where agents “make up their own language”
<IP_ADDRESS>. attention / transformers¶
self-attention layer implementation and mathematics
**self-attention ** - layer that lets word learn its relation to other layers
for each word, want score telling how much importance to place on each other word (queries \(\cdot\) keys)
we get an encoding for each word
the encoding of each word returns a weighted sum of the values of the words (the current word gets the highest weight)
softmax this and use it to do weighted sum of values
(optional) implementation details
multi-headed attention - just like having many filters, get many encodings for each word
each one can take input as the embedding from the previous attention layer
position vector - add this into the embedding of each word (so words know how far apart they are) - usually use sin/cos rather than actual position number
padding mask - add zeros to the end of the sequence
look-ahead mask - might want to mask to only use previous words (e.g. if our final task is decoding)
residual + normalize - after self-attention layer, often have residual connection to previous input, which gets added then normalized
decoder - each word only allowed to attend to previous positions
3 components
queries
keys
values
attention
encoder reads input and ouputs context vector after each word
decoder at each step uses a different weighted combination of these context vectors
specifically, at each step, decoder concatenates its hidden state w/ the attention vector (the weighted combination of the context vectors)
this is fed to a feedforward net to output a word
at a high level we have \(Q, K, V\) and compute \(softmax(QK^T)V\)
instead could simplify it and do \(softmax(XX^T)V\) - this would then be based on kernel
transformer
uses many self-attention layers
many stacked layers in encoder + decoder (not rnn: self-attention + feed forward)
details
initial encoding: each word -> vector
each layer takes a list of fixed size (hyperparameter e.g. length of longest sentence) and outputs a list of that same fixed size (so one output for each word)
can easily train with a masked word to predict the word at the predicted position in the encoding
multi-headed attention has several of each of these (then just concat them)
recent papers
attention is all you need paper - proposes transformer
Semi-supervised Sequence Learning (by Andrew Dai and Quoc Le)
ELMo (by Matthew Peters and researchers from AI2 and UW CSE) - no word embeddings - train embeddings w/ bidirectional lstm (on language modelling)
context vector is weighted sum of context vector at each word
ULMFiT (by fast.ai founder Jeremy Howard and Sebastian Ruder), the
OpenAI transformer (by OpenAI researchers Radford, Narasimhan, Salimans, and Sutskever)
BERT - semi-supervised learning (predict masked word - this is bidirectional) + supervised finetuning
GPT-2 (small released model, full trained model, even larger model from Nvidia)
these ideas are starting to be applied to vision cnns


{% extends 'core/wizard-step-base.html' %}
{% block head_title %}ERP - great.gov.uk{% endblock %}
{% block step_title %}What were your global sales revenues (in GBP) for products related to these goods before Brexit?{% endblock %}
{% block caption %}
This helps us to draw a benchmark. The more data you give, the better we can assess the impact. Enter 0 where there's no data.
For trade associations, give a cumulative figure
{% endblock %}


In the last notebook, we built a Multilayer Perceptron for recognizing hand-written digits from the MNIST data-set. The best achieved accuracy on testing data was about 97%, but modern implementations of Convolutional Neural Networks whould surpass that mark.
In this notebook, we will build, train and optimize in TensorFlow one of the early Convolutional Neural Networks, LeNet-5, and push it beyond 99% accuracy.
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.contrib.layers import flatten
from __future__ import print_function
from numpy import array
import numpy as np
import time
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
X_train, y_train = mnist.train.images, mnist.train.labels
X_validation, y_validation = mnist.validation.images, mnist.validation.labels
X_test, y_test = mnist.test.images, mnist.test.labels
print("Image Shape: {}".format(X_train[0].shape[0]))
print("Training Set: {} samples".format(len(X_train)))
print("Validation Set: {} samples".format(len(X_validation)))
print("Test Set: {} samples".format(len(X_test)))
Before starting with CNN, let's train and test in TensorFlow a simple example : y=softmax(Wx+b)
This model should reach an accuracy of about 92 %.
#GRAPH DEFINITION
# Parameters
learning_rate = 0.01
training_epochs = 100
batch_size = 128
display_step = 1
logs_path = 'log_files/' # useful for tensorboard
# tf Graph Input: mnist data image of shape 28*28=784
x = tf.placeholder(tf.float32, [None, 784], name='InputData')
# 0-9 digits recognition, 10 classes
y = tf.placeholder(tf.float32, [None, 10], name='LabelData')
# Set model weights
W = tf.Variable(tf.zeros([784, 10]), name='Weights')
b = tf.Variable(tf.zeros([10]), name='Bias')
# Construct model and encapsulating all ops into scopes,
# making Tensorboard's Graph visualization more convenient
with tf.name_scope('Model'):
# Model
pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax
with tf.name_scope('Loss'):
# Minimize error using cross entropy
cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1))
with tf.name_scope('SGD'):
# Gradient Descent
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
with tf.name_scope('Accuracy'):
# Accuracy
acc = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
acc = tf.reduce_mean(tf.cast(acc, tf.float32))
# Initializing the variables
init = tf.global_variables_initializer()
# Create a summary to monitor cost tensor
tf.summary.scalar("TrainingLoss", cost)
# Create a summary to monitor accuracy tensor
tf.summary.scalar("TrainingAccuracy", acc)
# Merge all summaries into a single op
merged_summary_op = tf.summary.merge_all()
#TRAINING
# Launch the graph for training
with tf.Session() as sess:
sess.run(init)
# op to write logs to Tensorboard
summary_writer = tf.summary.FileWriter(logs_path, graph=tf.get_default_graph())
# Training cycle
for epoch in range(training_epochs):
avg_cost = 0.
total_batch = int(mnist.train.num_examples/batch_size)
# Loop over all batches
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size, shuffle=True)
# Run optimization op (backprop), cost op (to get loss value)
# and summary nodes
_, c, summary = sess.run([optimizer, cost, merged_summary_op],
feed_dict={x: batch_xs, y: batch_ys})
# Write logs at every iteration
summary_writer.add_summary(summary, epoch * total_batch + i)
# Compute average loss
avg_cost += c / total_batch
# Display logs per epoch step
if (epoch+1) % display_step == 0:
print("Epoch: ", '%02d' % (epoch+1), " =====> Loss=",
"{:.9f}".format(avg_cost))
print("Optimization Finished!")
# Test model
# Calculate accuracy
print("Accuracy:", acc.eval({x: mnist.test.images, y: mnist.test.labels}))
Using Tensorboard, we can now visualize the created graph, giving us an overview of the architecture and how all of the major components are connected. You can also visalize and analyse the learning curves.
In order to launch tensorBoard we follow these steps:

Now that we are familiar with familar with tensorFlow and tensorBoard, we are going to build, train and test the baseline LeNet-5 model for the MNIST digits recognition problem.
Further ahead we will make some optimizations to surpass 99% of accuracy. The best model so far achieved over 99.7% accuracy (List of Results)

The LeNet architecture accepts a 32x32xC image as input, where C is the number of color channels. Since MNIST images are grayscale, C is 1 in this case.
Layer 2: Convolutional. The output shape should be 10x10x16. Activation. sigmoid Pooling. The output shape should be 5x5x16.
Flatten. Flatten the output shape of the final pooling layer such that it's 1D instead of 3D. You may need to use *flatten from tensorflow.contrib.layers import flatten
Layer 3: Fully Connected. This should have 120 outputs. Activation. sigmoid
Layer 4: Fully Connected. This should have 84 outputs. Activation. sigmoid
Layer 5: Fully Connected. This should have 10 outputs. Activation. softmax
The implementation draws classes and functions from the Tensorflow API.
# LeNet5 variables init
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.01)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
# LeNet5 convolutional and max pool layers
def conv2d(x, W,pad='SAME'):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding=pad)
def max_pool_2x2(x,pad='VALID'):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1], padding=pad)
def LeNet5_Model(data, keep_pr=1, activFunc=tf.nn.sigmoid):
input_data = tf.reshape(data,[-1,28,28,1])
# -------------------------------------------------
# --------------------VARIABLES--------------------
# -------------------------------------------------
# Convolutional layer 1 variables
W_conv1 = weight_variable([5,5,1,6])
b_conv1 = bias_variable([6])
# Convolutional layer 2 variables
W_conv2 = weight_variable([5,5,6,16])
b_conv2 = bias_variable([16])
# Fully connected layer 1 param
W_fc1 = weight_variable([400, 120])
b_fc1 = bias_variable([120])
# Fully connected layer 2 param
W_fc2 = weight_variable([120, 84])
b_fc2 = bias_variable([84])
# Fully connected layer 3 param
W_fc3 = weight_variable([84, 10])
b_fc3 = bias_variable([10])
# ----------------------------------------------------
# --------------------COMPUTATIONS--------------------
# ----------------------------------------------------
# Convolutional layer 1 & max pooling
h_conv1 = activFunc(conv2d(input_data,W_conv1)+ b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
# Convolutional layer 2 & max pooling
h_conv2 = activFunc(conv2d(h_pool1,W_conv2, pad='VALID')+ b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
# Flattening
h_pool2_flat = tf.contrib.layers.flatten(h_pool2)
# Fully connected layer 1, sigmoid activation
h_fc1 = activFunc(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
drop_fc1 = tf.nn.dropout(h_fc1, keep_pr)
# Fully connected layer 2, sigmoid activation
h_fc2 = activFunc(tf.matmul(drop_fc1, W_fc2) + b_fc2)
drop_fc2 = tf.nn.dropout(h_fc2, keep_pr)
# Fully connected layer 3, softmax activation
predicted = tf.nn.softmax(tf.matmul(drop_fc2, W_fc3) + b_fc3)
return predicted
NParameters_LeNet5 = \
2*(5*5*1*6) + \
2*(5*5*6*16) + \
400*120 + \
120 + \
120*84 + \
84 + \
84*10 + \
10
print('Mumber of parameters in LeNet5: %d'%NParameters_LeNet5)
The initial training will be using the parameters cited below:
Learning rate =0.1
Loss Function : Cross entropy
Optimisateur: SGD
Number of training iterations= 10000
The batch size =128
def CNNet ( modelName,
learning_rate = 0.1,
training_epochs = 100,
batch_size = 128,
display_step = 1,
keep_p = 1,
activationFunc = tf.nn.sigmoid,
optimFunc = tf.train.GradientDescentOptimizer,
X_train=mnist.train.images,
y_train=mnist.train.labels,
X_val=mnist.validation.images,
y_val=mnist.validation.labels,
X_test= mnist.test.images,
y_test=mnist.test.labels,
loadModel=None
):
# ---------- DESCRIPTION OF DATASET ----------
InputSize = X_train[0].shape[0]
OutputSize = y_train[0].shape[0]
TrainingSetSize = len(X_train)
ValidationSetSize = len(X_validation)
TestSetSize = len(X_test)
# ---------- OUTPUT FOLDERS ----------
logsFolder = 'log_files/' # useful for tensorboard
saveFolder = 'Models/' # useful to restore the model
# ---------- RESET GRAPH ----------
tf.reset_default_graph()
# ---------- DEFINE VARIABLES ----------
# tf Graph Input: mnist data image of shape 28*28*1
x = tf.placeholder(tf.float32, [None,InputSize], name='InputData')
# 0-9 digits recognition, 10 classes
y = tf.placeholder(tf.float32, [None,OutputSize], name='LabelData')
# Dropout
keep_prob = tf.placeholder(tf.float32, name='DropoutKeepProbability')
# ---------- DEFINE GRAPH NODES ----------
with tf.name_scope('Model'):
# Model
model = LeNet5_Model(x, keep_prob, activFunc=activationFunc)
with tf.name_scope('Loss'):
# Minimize error using cross entropy
cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(model+1e-9), reduction_indices=1))
#cost = tf.nn.softmax_cross_entropy_with_logits(model, tf.one_hot(y, 10))
with tf.name_scope('Optimizer'):
#Optimization, using cost reduction
optimizer = optimFunc(learning_rate).minimize(cost)
with tf.name_scope('Accuracy'):
# Accuracies
acc = CNNetAccuracy(model, y)
# ---------- INITIALIZE VARIABLES ----------
init = tf.global_variables_initializer()
# ---------- TRACK BATCH LOSS AND ACCURACY ----------
# Create a summary to monitor cost tensor
tf.summary.scalar("BatchLoss", cost)
# Create a summary to monitor batch accuracy tensor
tf.summary.scalar("BatchAccuracy", acc)
# Merge all summaries into a single op
merged_summary_op = tf.summary.merge_all()
# ---------- TRAIN MODEL ----------
CNNtrain(model, cost, optimizer, acc,
x, y, keep_prob, TrainingSetSize,
X_train, y_train, X_val, y_val, X_test, y_test,
init, merged_summary_op,
modelName, saveFolder,logsFolder,
learning_rate, training_epochs, batch_size, display_step , keep_p,
loadModel
)
Here we implement the evaluation function for accuracy computation:
def CNNetAccuracy(model, y):
accuracy = tf.reduce_mean(
tf.cast( tf.equal( tf.argmax(model, 1),
tf.argmax(y, 1)), tf.float32))
return accuracy
Here we implement training pipeline and run the training data through it to train the model. Other steps to consider are:
def read_my_file_format(filename_queue):
reader = tf.SomeReader()
key, record_string = reader.read(filename_queue)
example, label = tf.some_decoder(record_string)
processed_example = some_processing(example)
return processed_example, label
def input_pipeline(filenames, batch_size, num_epochs=None):
filename_queue = tf.train.string_input_producer(
filenames, num_epochs=num_epochs, shuffle=True)
example, label = read_my_file_format(filename_queue)
# min_after_dequeue defines how big a buffer we will randomly sample
# from -- bigger means better shuffling but slower start up and more
# memory used.
# capacity must be larger than min_after_dequeue and the amount larger
# determines the maximum we will prefetch. Recommendation:
# min_after_dequeue + (num_threads + a small safety margin) * batch_size
min_after_dequeue = 10000
capacity = min_after_dequeue + 3 * batch_size
example_batch, label_batch = tf.train.shuffle_batch(
[example, label], batch_size=batch_size, capacity=capacity,
min_after_dequeue=min_after_dequeue)
return example_batch, label_batch
def CNNtrain(model, cost, optimizer, acc,
x, y, keep_prob, TrainingSetSize,
X_train, y_train, X_val, y_val, X_test, y_test,
init, merged_summary_op,
modelName, saveFolder, logsFolder,
learning_rate, training_epochs, batch_size, display_step, keep_p,
loadModel, dataFileName=None
):
# Initial model print
print("*Model [", modelName,"] {l_r: %.4f; n_iter: %d; batch: %d}"%\
(learning_rate, training_epochs, batch_size))
# Start a tensorflow session
with tf.Session() as sess:
print (" Start Training!")
sess.run(init)
# Load model if the parameter loadModel is not empty
saver = tf.train.Saver()
if(loadModel):
saver.restore(sess=sess,save_path='Models/'+loadModel)
# op to write logs to Tensorboard
summary_writer = tf.summary.FileWriter(logsFolder,
graph=tf.get_default_graph())
# Training cycle
t0 = time.time()
for epoch in range(training_epochs):
avg_cost = 0.
total_batch = int(mnist.train.num_examples/batch_size)
# Loop over all batches
for i in range(total_batch):
batch_xs, batch_ys = mnist.train.next_batch(batch_size, shuffle=True)
# batch_xs,batch_ys = input_pipeline(filenames=dataFileName,
batch_size=batch_size,
num_epochs=total_batch)
# Run optimization op (backprop), cost op (to get loss value)
# and summary nodes
_, c, summary = sess.run([optimizer, cost, merged_summary_op],
feed_dict={x: batch_xs,
y: batch_ys,
keep_prob: keep_p})
# Write logs at every iteration
summary_writer.add_summary(summary, epoch * total_batch + i)
# Compute average loss
avg_cost += c / total_batch
# Display logs per epoch step
if (epoch+1) % display_step == 0:
tr_acc = acc.eval({x: X_train, y: y_train, keep_prob: 1})
vl_acc = acc.eval({x: X_val, y: y_val, keep_prob: 1})
print(" Epoch: %02d | Loss=%.9f | TrainAcc=%.3f %% | ValAcc=%.3f %%"%
(epoch+1, avg_cost, tr_acc*100, vl_acc*100));
print (" Training Finished in %.1f seconds."%(time.time()-t0))
# Evaluating model with the accuracies
print (" Final accuracies:")
print (" ~ TrainAcc: %.3f %%"%(100*acc.eval({x: X_train, y: y_train, keep_prob: 1})))
print (" ~ ValAcc: %.3f %%"%(100*acc.eval({x: X_val, y: y_val, keep_prob: 1})))
print (" ~ TestAcc: %.3f %%"%(100*acc.eval({x: X_test, y: y_test, keep_prob: 1})))
# Saving Model
saver.save(sess=sess,save_path=saveFolder+modelName)
print (" Saving model in file: %s"%(saveFolder+modelName))
# Training our first model!
CNNet ('lenet5-model',
learning_rate = 0.1,
training_epochs = 100,
batch_size = 128,
)
We use tensorBoard to visualise and save the LeNet5 Graph and all learning curves. The data is then converted into CSV using the GUI drom tensorboard and is then plotted using Excel. The resulting figures are:

We change the sigmoid function to a Relu and perform the next steps:
| Optimizer | Gradient Descent | AdamOptimizer |
|---|---|---|
| Validation Accuracy | 98.760 % | 99.080 % |
| Testing Accuracy | 98.670 % | 99.110 % |
| Training Time | 8048 s | 2670 s |
Try with different learning rates for each Optimizer (0.0001 and 0.001 ) and different Batch sizes (50 and 128) for 10000 Epochs.
For each optimizer, plot (on the same curve) the testing accuracies function to (learning rate, batch size)

# Trying parameters with SGD
setOfParams = [ [0.001, 250, 50, 'lenet5-model_relu_lr0001_bs50'],
[0.001, 100, 128, 'lenet5-model_relu_lr0001_bs128'],
[0.0001, 250, 50, 'lenet5-model_relu_lr00001_bs50'],
[0.0001, 100, 128, 'lenet5-model_relu_lr00001_bs128'],
]
for p in setOfParams:
CNNet (modelName=p[3],
learning_rate = p[0],
training_epochs = p[1],
batch_size = p[2],
activationFunc=tf.nn.relu,
)
# Trying parameters with Adam Optimizer
setOfParams = [ [0.001, 250, 50, 'lenet5-model_adam_relu_lr0001_bs50'],
[0.001, 100, 128, 'lenet5-model_adam_relu_lr0001_bs128'],
[0.0001, 250, 50, 'lenet5-model_adam_relu_lr00001_bs50'],
[0.0001, 100, 128, 'lenet5-model_adam_relu_lr00001_bs128'],
]
for p in setOfParams:
CNNet (modelName=p[3],
learning_rate = p[0],
training_epochs = p[1],
batch_size = p[2],
activationFunc=tf.nn.relu,
optimFunc=tf.train.AdamOptimizer
)
# Trying parameters with Adam Optimizer - AFTER CHANGING THE OPTIMIZER!
# (Weight init & safe softmax)
setOfParams = [ [0.001, 100, 50, 'lenet5-model_adam_relu_lr0001_bs50'],
[0.001, 100, 128, 'lenet5-model_adam_relu_lr0001_bs128'],
[0.0001, 100, 50, 'lenet5-model_adam_relu_lr00001_bs50'],
[0.0001, 100, 128, 'lenet5-model_adam_relu_lr00001_bs128'],
]
for p in setOfParams:
CNNet (modelName=p[3],
learning_rate = p[0],
training_epochs = p[1],
batch_size = p[2],
activationFunc=tf.nn.relu,
optimFunc=tf.train.AdamOptimizer
)
# Ending the training of the best optimizer
CNNet (modelName='lenet5-model_best',
learning_rate = 0.001,
training_epochs = 30,
batch_size = 128,
activationFunc=tf.nn.relu,
optimFunc=tf.train.AdamOptimizer,
loadModel='lenet5-model_adam_relu_lr0001_bs128'
)
What about applying a dropout layer on the Fully conntected layer and then retraining the model with the best Optimizer and parameters(Learning rate and Batsh size) obtained in the previous section? (probability to keep units=0.75). For this stage we ensure that the keep prob is set to 1.0 to evaluate the performance of the network including all nodes.
CNNet ('lenet5-model',
learning_rate = 0.001,
training_epochs = 100,
batch_size = 128,
activationFunc = tf.nn.relu,
optimFunc=tf.train.AdamOptimizer,
keep_p = 0.75
)


Class: Rumale::FeatureExtraction::FeatureHasher
- Inherits:
-
Base::Estimator
- Object
- Base::Estimator
- Rumale::FeatureExtraction::FeatureHasher
- Includes:
- Base::Transformer
- Defined in:
- rumale-feature_extraction/lib/rumale/feature_extraction/feature_hasher.rb
Overview
Encode array of feature-value hash to vectors with feature hashing (hashing trick). This encoder turns array of mappings (Array<Hash>) with pairs of feature names and values into Numo::NArray. This encoder employs signed 32-bit Murmurhash3 as the hash function.
Instance Attribute Summary
Attributes inherited from Base::Estimator
Instance Method Summary collapse
-
#fit(x) ⇒ FeatureHasher
This method does not do anything.
-
#fit_transform(x) ⇒ Numo::DFloat
Encode given the array of feature-value hash.
-
#initialize(n_features: 1024, alternate_sign: true) ⇒ FeatureHasher
constructor
Create a new encoder for converting array of hash consisting of feature names and values to vectors with feature hashing algorith.
-
#transform(x) ⇒ Numo::DFloat
Encode given the array of feature-value hash.
Constructor Details
#initialize(n_features: 1024, alternate_sign: true) ⇒ FeatureHasher
Create a new encoder for converting array of hash consisting of feature names and values to vectors with feature hashing algorith.
35 36 37 38 39 40 41 |
# File 'rumale-feature_extraction/lib/rumale/feature_extraction/feature_hasher.rb', line 35 def initialize(n_features: 1024, alternate_sign: true) super() @params = { n_features: n_features, alternate_sign: alternate_sign } end |
Instance Method Details
#fit(x) ⇒ FeatureHasher
This method does not do anything. The encoder does not require training.
48 49 50 |
# File 'rumale-feature_extraction/lib/rumale/feature_extraction/feature_hasher.rb', line 48 def fit(_x = nil, _y = nil) self end |
#fit_transform(x) ⇒ Numo::DFloat
Encode given the array of feature-value hash. This method has the same output as the transform method because the encoder does not require training.
59 60 61 |
# File 'rumale-feature_extraction/lib/rumale/feature_extraction/feature_hasher.rb', line 59 def fit_transform(x, _y = nil) fit(x).transform(x) end |
#transform(x) ⇒ Numo::DFloat
Encode given the array of feature-value hash.
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
# File 'rumale-feature_extraction/lib/rumale/feature_extraction/feature_hasher.rb', line 67 def transform(x) x = [x] unless x.is_a?(Array) n_samples = x.size z = Numo::DFloat.zeros(n_samples, n_features) x.each_with_index do |f, i| f.each do |k, v| k = "#{k}=#{v}" if v.is_a?(String) val = v.is_a?(String) ? 1 : v next if val.zero? h = Mmh3.hash32(k) fid = h.abs % n_features val *= h >= 0 ? 1 : -1 if alternate_sign? z[i, fid] = val end end z end |


Kelsi Proulx
WHO I AM: I am a fullstack web developer with a passion for frontend development. I work well on
a team, have
a
strong work ethic, and the ability to
complete any task assigned to me. I have advanced research and people skills from my background in
library work.
PASSION: I am passionate about front-end web
development and design. Any projects that allow me to
express my creativity!
STACK: I enjoy working with CSS, React, and
Vanilla JS.
WHY I'M HERE: I chose web development because I
want a career that will allow me to continually learn,
explore new things, and always be excited about what I
do.