--- layout: example categories: example/v1.0.0 version: v1.0.0 title: Clusters with custom cluster icons description: Show the number of markers in a cluster with custom simplestyle markers tags: - cluster ---


Fishing Guild Member:
On Devil's Isle, if you hunt Zaken's Archers, Mardians, Pirate Zombies, Tainted Ogres, Fiend Archers, Zaken's Guards, Pirate Zombie Captains, Veil Masters and Unpleasant Hummings, you'll get the sweet fluid that Water Spirits like.
Return


Help with studying
Studying in something a lot of students struggle with. This is most likely due to lack of motivation and organisation. This website will identify all the working tips that will help with studying. It is important to understand that studying is an important skill that is needed in DP and university. This is why we start studying properly at a young age.
  It also helps free up your brain. It stops your brain from doing unnecessary thinking. When you are trying to remember something, there are three parts of your brain that work. Working memory, short term memory and long term memory. If you don't have a place to write stuff down, your brain goes into stress mode because it's trying to hold onto all of the information.  If you do have an area to write stuff down, your brain focuses on the learning rather than the remembering. This erases your stress level.


Results Screen
There are two different results views:
The Default view may be changed in the Options.Standard View
-
Radius
The Radius that was entered previously. -
Entrance Length
The Entrance Length that was entered previously. -
Budget
Total Number of Bricks calculated for the Budget size (40 × 25 × 15cm). -
Superior
Total Number of Bricks calculated for the Superior size (35 × 25 × 20cm). -
Luxury
Total Number of Bricks calculated for the Luxury size (30 × 20 × 20cm). -
Save
Saves the current results. Click here for details. -
Advanced View
Switches the results screen to Advanced View. -
Help
Displays the current Web Help. -
Close
Closes the program. -
Back
Returns to the previous screen to allow for new values to be entered.
Note: This is disabled when loading from a previously saved file.
Advanced View
-
Same as in Standard View
Refer to Standard View. -
Budget / Superior / Luxury
Switches between displaying values for the Budget, Superior and Luxury brick sizes respectively. -
Entrance Tunnel Length
Length of the entrance tunnel in bricks. -
Total No. of Rows
Total Number of Rows in the main dome. -
Total No. of Bricks for Entrance
Total Number of Bricks required to make the entrance tunnel. -
Total No. of Bricks for Igloo Dome
Total Number of Bricks required to make the main igloo dome. -
No. of Bricks per Row
Number of Bricks required for each row of the main dome.
Note: Numbers marked with an * signify that bricks have been removed to allow for an entrance hole. -
Save
Saves the current results. Click here for details. -
Standard View
Switches the results screen to Standar View. -
Help
Displays the current Web Help. -
Close
Closes the program. -
Back
Returns to the previous screen to allow for new values to be entered.
Note: This is disabled when loading from a previously saved file.


Issue: LOOP-INITFORM-ENVIRONMENTForum: Editorial
References: LOOP (pages 8-85,
Draft 8.81
Category: CLARIFICATION
Edit history: 05-Mar-91, Version 1 by Pitman
15-Mar-91, Version 2 by Pitman
Status: For X3J13 consideration
Problem Description:
The Symbolics implementation of X3J13's LOOP spec surprised some users
by being incompatible with the old Symbolics Genera LOOP on the
following test case:
(loop for list = list then (cdr list)
The Symbolics Genera implementation prints nothing, but old Symbolics
Genera LOOP prints 123. In double-checking the implementation against the
working draft specification to determine if the implementation was in
error, more than one Symbolics implementors did not feel this behavior
was clearly enough spelled out and worried that implementations might not
agree.
Further investigation showed that Lucid Common Lisp had the same behavior.
The following are the only references which appeared relevant:
From page 8-85:
The LOOP macro translates the given <form> into code and returns the
expanded <form>. The expanded <form> is one or more <lambda
expressions> for the local <binding> of loop variables and a block
and a tagbody that express a looping control structure. The variables
established in LOOP are bound as if by LET or LAMBDA.
Implementations can interleave the setting of initial values with
the bindings. However, the assignment of the initial values is
always calculated in othe order specified by the user. A variable is
thus sometimes bound to a meaningless value of the correct type
and then later in the prologue is set to the true initial value by
using SETQ.
Later, on page 8-86:
The FOR and AS constructs provide iteration control clauses that
establish a variable to be initialized. FOR and AS clauses can be
combined with the LOOP keyword AND to get parallel initialization and
stepping. Otherwise, the initialization and stepping are sequential.
The need for this information is so fundamental that it should be very
plainly stated. These extremely vague phrases don't really say much
about the environment, and since they don't say what goes into the LET
or the LAMBDA, or even how many LET or LAMBDA forms are involved, they
don't really say much at all.
Further, the vague statement on p8-85 about how LET+SETQ might be used to
implement binding leaves an unusually large amount of latitude to
implementations.
Proposal (LOOP-INITFORM-ENVIRONMENT:INITFORM-BEFORE-BINDING)
Clarify that the initforms in a FOR-AS clause
(each being variously called the <form1>, the <hash-table>, or the
<package> in the `bnf' on pages 8-82..8-83)
are all evaluated (in left to right order) prior to establishing
the binding for any of the the <vars> in the same clause.
Clarify that the initforms in a WITH clause (each being
called a <form1> in the `bnf' on pages 8-82..8-83)
are all evaluated (in left to right order) prior to establishing
the binding for any <varN> in the same clause.
Test Case:
(loop for list = list then (cdr list)
=> (1 2 3)
Rationale:
This is what most users who make an analogy to LET or DO will expect.
Current Practice:
Symbolics Genera's FUTURE-COMMON-LISP:LOOP (which purports to conform
to the draft specification) returns NIL.
Symbolics Genera's LISP:LOOP (which purports to conform to CLtL)
returns (1 2 3) for the test case.
Cost to Implementors:
Hopefully small.
Cost to Users:
Hopefully most users will consider this the status quo.
Cost of Non-Adoption:
The specification might be ambiguous on an important point.
Benefits:
Cost of non-adoption is avoided.
Aesthetics:
Clarity improves aesthetics.
Discussion:
Pitman and Moon support this proposal.
JonL opposes this proposal.
A long-winded discussion ensued, excerpts of which follow.
Moon writes:
``It's intentional that implementations should have latitude in
the expansion of LOOP, but not to the extent that the meaning of
the program changes! The problem is that the only thing this
version of the LOOP specification says about the scope of LOOP
variables is that they are lexical unless proclaimed special and
their scope is the loop. This isn't specific enough.
``Also in CLtL2 p.722 there is an example expansion which could be
taken as an argument against the INITFORM-BEFORE-BINDING proposal.
However I don't think that example was intended to talk about
variable scoping. This example does not appear in the draft ANSI
CL spec, which is good in my opinion.''
JonL writes:
``I don't see this as a clarification but as a change; in particular,
it isn't consistent with the section of CLtL/II that you cite:
[... page 8-85 ...]
since it forbids the aforementioned "interleaving". As Moon said,
it is "intentional that implementations should have latitude in
the expansion of LOOP, ...". So the question is just how
importantant is the case of loop variables "shadowing" local
lexical variables? That is, I agree that the current spec is
ambiguous on just when the binding of 'list' occurs in
(let ((list '(1 2 3)))
(loop for list = list then (cdr list)
. . . ))
but I'm more inclined to say SO WHAT? Is the world going into
terminal meltdown just because this case isn't completely portable?
is LOOP completely unusable *** just because of this lexical
shadowing ambiguity?
``The point I'm trying to make is not that this case isn't a problem,
but that it isn't a *big* problem; and even if it remained
ambiguous (as was the intention for the draft proposal -- reasons
cited below), it is nowhere near the magnitude of problem that we
face in other areas.
``... Lucid's LOOP (unchanged for many years now) was modeled after
GSB's MIT code, and it has the same behaviour as Genera. Even if
the "original designers" intended something here, I think Glenn's
execution of it might have led people to think otherwise. ...''
Moon writes:
``I don't see this particular issue as some technical issue of angels
dancing on the heads of pins that is only of real interest to
specialists. It's not uncommon for user programs to use the same
variable name more than once. I hear that users very often find it
frustrating to try to use LOOP because there are implementations around
that do unintuitive things and documentation around that they can't
decode. Those problems are so unnecessary.
``... I just now loaded up the MIT LOOP, unchanged for nearly five
years (unless someone's been changing it without updating the edit
history at the front) from REAGAN.AI.MIT.EDU, and tried the test case
from the cleanup issue. It prints 123, so it conforms to the cleanup.
This makes me suspect that the MIT LOOP has always had the intuitive
behavior, but programmers at Lucid and also at Symbolics broke it in the
process of improving it. I don't think it's true that Glenn implemented
it differently from what the cleanup says.
``So then I tried the CMU implementation referenced in Scott Fahlman's
recent message. It, too, conforms to the cleanup, although it seems to
have its share of bugs (several compiler warnings and errors while
loading that I fixed up and proceeded past).
``... We can always specify a language feature as doing something
unpredictable, but is that really necessary?''
JonL writes:
``I claim this ambiguity is purposeful -- the "interleaving" of binding
and initialization clauses, which in some implementations has
performance consequences -- and that the case that brought this
ambiguity to public attention is not of great consequence to the
language.
``Of course, I agree that we must specify *exactly* the behaviour
of LET when local variables are being shadowed, such as in:
because renaming of bound variables is an important concept (not just
"pinheads dancing on angels"). But LOOP bindings are not documented to
be *exactly* like LET, and they aren't the primariy means to achieve
lexical "shadows"; thus I strongly disagree that seeking out and "fixing"
documented ambiguities in the binding-order for LOOP "shadows" is of high
priority.''


The divergence theorem states that the surface integral of the normal component of a vector point function “F” over a closed surface “S” is equal to the volume integral of the divergence of F, taken over the volume “V” enclosed by the surface S.
\[\iint_S F.n \,dS = \iiint_v divF\,dV\]
Let S be an oriented smooth surface that is bounded by a simple, closed, smooth boundary curve C with positive orientation. Also let F be a vector field then,
\[\int_C F\,dr = \iint_S curlF\,dS\]
Let C be a positively oriented, piecewise smooth, simple, closed curve and let D be the region enclosed by the curve. If P and Q have continuous first order partial derivatives on D then,
\[\int_C {P\,dx + Q\,dy} = \iint_S ({\frac {∂Q}{∂x}-\frac{∂P}{∂y}})\,dA \]


This function expects a data frame produced by an salic estimation function (
est_part
, est_recruit
, est_churn
).
It returns a data frame with additional formatting that allows stacking all results
into a single table.
format_result(df, timeframe, group, rename_input = c("category", "year", "value"))
| df | data frame: Input table (3 variables) with estimated metrics |
|---|---|
| timeframe | character: value to store in the 'timeframe' variable of the output (e.g, 'full-year', 'mid-year') |
| group | character: value to sore in the 'group' variable of the output (e.g., 'all_sports', 'fish', 'hunt') |
| rename_input | character: generic names for input variables as they will appear in the output |
Salic Function Reference: salic
Other dashboard functions: check_threshold
,
est_churn
, est_part
,
scaleup_part
library(dplyr) data(metrics) # format a table metrics$participants$res#> # A tibble: 22 x 3 #> res year participants #> <fct> <int> <int> #> 1 Resident 2008 4768 #> 2 Resident 2009 5264 #> 3 Resident 2010 5223 #> 4 Resident 2011 5171 #> 5 Resident 2012 5341 #> 6 Resident 2013 5394 #> 7 Resident 2014 5522 #> 8 Resident 2015 5460 #> 9 Resident 2016 5332 #> 10 Resident 2017 5289 #> # ... with 12 more rowsx <- format_result(metrics$participants$res, "full-year", "all_sports") x#> # A tibble: 22 x 7 #> timeframe group segment year category metric value #> <chr> <chr> <chr> <int> <chr> <chr> <int> #> 1 full-year all_sports Residency 2008 Resident participants 4768 #> 2 full-year all_sports Residency 2009 Resident participants 5264 #> 3 full-year all_sports Residency 2010 Resident participants 5223 #> 4 full-year all_sports Residency 2011 Resident participants 5171 #> 5 full-year all_sports Residency 2012 Resident participants 5341 #> 6 full-year all_sports Residency 2013 Resident participants 5394 #> 7 full-year all_sports Residency 2014 Resident participants 5522 #> 8 full-year all_sports Residency 2015 Resident participants 5460 #> 9 full-year all_sports Residency 2016 Resident participants 5332 #> 10 full-year all_sports Residency 2017 Resident participants 5289 #> # ... with 12 more rows# combine formatted tables y <- format_result(metrics$participants$tot, "full-year", "all_sports") bind_rows(y, x)#> # A tibble: 33 x 7 #> timeframe group segment year category metric value #> <chr> <chr> <chr> <int> <chr> <chr> <int> #> 1 full-year all_sports All 2008 All participants 5834 #> 2 full-year all_sports All 2009 All participants 6775 #> 3 full-year all_sports All 2010 All participants 6820 #> 4 full-year all_sports All 2011 All participants 6734 #> 5 full-year all_sports All 2012 All participants 6964 #> 6 full-year all_sports All 2013 All participants 6801 #> 7 full-year all_sports All 2014 All participants 7253 #> 8 full-year all_sports All 2015 All participants 7153 #> 9 full-year all_sports All 2016 All participants 7095 #> 10 full-year all_sports All 2017 All participants 7045 #> # ... with 23 more rows# apply formatting across all segments x <- lapply(metrics$participants, function(x) format_result(x, "full-year", "sports")) bind_rows(x)#> # A tibble: 110 x 7 #> timeframe group segment year category metric value #> <chr> <chr> <chr> <int> <chr> <chr> <int> #> 1 full-year sports All 2008 All participants 5834 #> 2 full-year sports All 2009 All participants 6775 #> 3 full-year sports All 2010 All participants 6820 #> 4 full-year sports All 2011 All participants 6734 #> 5 full-year sports All 2012 All participants 6964 #> 6 full-year sports All 2013 All participants 6801 #> 7 full-year sports All 2014 All participants 7253 #> 8 full-year sports All 2015 All participants 7153 #> 9 full-year sports All 2016 All participants 7095 #> 10 full-year sports All 2017 All participants 7045 #> # ... with 100 more rows# apply across all metrics & segments bind_rows( lapply(metrics$participants, function(x) format_result(x, "full-year", "sports")), lapply(metrics$recruits, function(x) format_result(x, "full-year", "sports")), lapply(metrics$churn, function(x) format_result(x, "full-year", "sports")) )#> # A tibble: 270 x 7 #> timeframe group segment year category metric value #> <chr> <chr> <chr> <dbl> <chr> <chr> <dbl> #> 1 full-year sports All 2008 All participants 5834 #> 2 full-year sports All 2009 All participants 6775 #> 3 full-year sports All 2010 All participants 6820 #> 4 full-year sports All 2011 All participants 6734 #> 5 full-year sports All 2012 All participants 6964 #> 6 full-year sports All 2013 All participants 6801 #> 7 full-year sports All 2014 All participants 7253 #> 8 full-year sports All 2015 All participants 7153 #> 9 full-year sports All 2016 All participants 7095 #> 10 full-year sports All 2017 All participants 7045 #> # ... with 260 more rows


http://learn.canterbury.ac.nz/mod/resource/view.php?id=326926the page via file:// -->
Freelancer is a free bootstrap theme created by Start Bootstrap. The download includes the complete source files including HTML, CSS, and JavaScript as well as optional LESS stylesheets for easy customization.
Whether you're a student looking to showcase your work, a professional looking to attract clients, or a graphic artist looking to share your projects, this template is the perfect starting point!


You can apply to vote absentee as early as one year before the election. Ballots are available 45 days before most elections. The deadline for receipt of your application to vote absentee by mail is 5:00 PM the Tuesday before the election. You can contact the State Board of Elections at<PHONE_NUMBER> or<EMAIL_ADDRESS>Your absentee ballot application has been emailed to the {{REGISTRAR_LOCALITY}} registrar, at {{REGISTRAR_EMAIL}}. No further action is required on your part.


FlightSurety
Status
Passenger
Passenger Address
Amount (ether)
Flight
ND1309 04:00AM
CF1309 11:00AM
ND1310 04:00PM
CF1310 11:00PM
Buy
Oracles
ND1309 04:00AM
CF1309 11:00AM
ND1310 04:00PM
CF1310 11:00PM
Submit to Oracles
Claim Insurance
Passenger Address
Show Credit
Claim Insurance
Log


First 50 Days of a New Job: A Marketing Specialist’s View
This isn’t the first blog that I have written for The Virtual Training Team webpage, however it is the first blog that is about me and the professional practice methods that I incorporate into a workday. Keep in mind that when we start looking at practice and productivity techniques, these are the things that work well for me; the best methods for you may be different, it feels strange that today is my fiftieth day.
While time has flown by, I have also learnt so much; from awkwardly saying hello to my manager on day one (I’m looking at you, Abi) to being able to start work today with a to-do list knowing what ‘ad for Train the Trainer for Facilitators’ means and how to do it.
Check out our Proactivity Workshop Today!
I should probably start by explaining a bit about me and what I do here at VTT.
So, hi! My name is Amy and I write some of the content you see here on the website (not just the blogs), across our social media channels, emails and more! I am studying towards my MSc in Digital Marketing & Analytics and in my (very sparse) free time I like reading, running and drinking a lot of coffee (but not at the same time!).
Onboarding
Whenever we start a new job, whether it is the same role in a different company, a promotion or we have changed directions completely, it can be tough to navigate your first few months. You want to effortlessly integrate into the team, but you don’t necessarily have the full picture of the company or processes that are already in place that you should be following.
That is where the onboarding process comes in.
Getting to know the people you work with every day and talking about roles and responsibilities can transform your broader perspective. This is what has had one of the biggest impacts on my ability to pick up tasks and approach them.
I am so thankful for the opportunities that I have when it comes to onboarding and the time that people have taken to share their experience and expertise with me, especially when everyone is already busy with their own tasks.
Good Practice and Productivity Techniques
Keeping an eye on tasks (outstanding and completed), progress that you are making and any unexpected situations that crop up are essential when approaching work. Here are the top three techniques I have been using over the last 50 days as I have adjusted to working with VTT!
1. To-Do List (Prioritising!)
I cannot manage without a to-do list!
While you may enjoy keeping a note of tasks digitally (maybe you have a document or a post-it app), I work best with handwritten to do lists so that everything I need to do can stay within my line-of-sight during the day.
Tending to have two lists with me, one for my tasks for the week and one for tasks for the day, means that it is easier to keep sight of everything that must be done but also that approaching a new day doesn’t become overwhelming.
A second thing that can be done, especially if a to-do list is extremely long, is to give tasks numbers depending on what must be completed first. This can help you to prioritise your to-do lists as well as work out when you can fit in new tasks.
Not only can to-do lists help you manage your time and keep sight of deadlines, but it can also give a visual representation of your success.
We have all had those days at work where it feels like you haven’t managed to get much done; but if you look back at your to-do list, chances are you’ve made more progress than you thought!
2. ‘Two-Minute Tasks’
If you haven’t heard of this technique it is designed to be a method that reduces the likelihood of procrastination when approaching tasks that take less than two minutes (it takes a ‘stop planning / start doing’ approach).
In the last 50 days this technique has proven extremely successful even if I have been flexible with the definition of two minutes (anything that is going to take me 60 minutes or less sometimes). So, if I’ve just had a meeting or opened an email that highlights a new task for my to-do list that will take concentration but will only require (up to) 60 minutes of undivided attention that is what I will do.
That means that I can check another thing off, and it doesn’t invade my headspace as I am getting on with tasks that require more time or creative flow. There is nothing worse than trying to get into the swing of a task but have something in the back of your mind going ‘but it won’t take very long / but it needs to get done’.
3. Time Management
While ‘two-minute tasks’ are a small percentage of time management, I want to reinforce how much time management can influence your productivity.
There are occasions where you will underestimate the time you have or overestimate the speed you can complete a task; we all have days where the email that would normally take five minutes to write ends up taking half an hour.
The important thing to remember is that if you are having a day where it is taking longer to get things done that you don’t feel guilty. Instead accept that it is happening and try to regulate your day by the time that you spend on a task, rather than forcefully working on something until it is finished.
One way that I circumvent this is that when I start work in the mornings, I’ll check in with myself and see how I am feeling.
Knowing how you are doing and knowing your capacity for the day (and limits!) can feed into how you manage your time for the day and ensure that you are being productive but are avoiding putting yourself under immense stress and pressure.
If you start of the day with a to-do list and you only manage to get halfway through one task you set out to do because other things cropped up, be assured that it is ok!
You have prioritised more important things or you’ve ended up spending the day in meetings and supporting/communicating with colleagues.
Still Learning, Still Adapting
Over the last 50 days I have learnt so much about the company and my profession, but I’ve also learnt so much about myself (although that’s a blog for a different day!).
But my journey is not yet over!
Over the upcoming 50 days, I am striving to keep learning and developing, positively contributing to the workplace and continuing to do what I love, finding inspiration as I go.
I suppose that’s what it’s really all about, continuing to employ the mindset and methodology that helps you thrive.


+++ title = "Jaza's law of washing" slug = "jazas-law-of-washing" date = 2007-03-30T14:00:00+10:00 tags = ["just-for-fun"] locations = ["canada/vancouver"] draft = false summary = """
When I was at home back in Australia, I had a massive arsenal of clothing, and I generally wore things for quite a short time. However, since I've been on the road, I've had a much more modest arsenal at my disposal, and this has forced me to wear things for (even) longer than I usually would. The net result of this is that I end up washing my clothes at about the same frequency as I did previously. In my case, this is about once every 2-3 weeks. Hence, we have the core of what I shall now dub Jaza's law of washing: the amount of clothes and the time spent wearing them may vary, but the period between each wash remains constant.
""" [[comments]] title = "laundry" date = 2007-04-03T13:58:08+10:00 author_name = "Sandra Epstein" content = """This comment was put in to totally confuse me (your mum) so that I won't know (and at this point it's probably better that I don't know) for how many days you are wearing things before they start to grow things! Hopefully you will meet lots of nice girls and then the length of time between laundry will decrease!
""" +++When I was at home back in Australia, I had a massive arsenal of clothing, and I generally wore things for quite a short time. However, since I've been on the road, I've had a much more modest arsenal at my disposal, and this has forced me to wear things for (even) longer than I usually would. The net result of this is that I end up washing my clothes at about the same frequency as I did previously. In my case, this is about once every 2-3 weeks. Hence, we have the core of what I shall now dub Jaza's law of washing: the amount of clothes and the time spent wearing them may vary, but the period between each wash remains constant.
Jaza's law of washing can be expressed algebraically as follows:
p = st
Where:
Additionally:
Now, I know I'm not the best mathematician in the world — I did famously drop maths in my final year of high school — but I think I've hit this baby pretty much right on the head. And if I haven't, and it is actually complete bollocks from a mathematical POV, then hopefully nobody will work that out for some time, until my stroke of genius has earned me at least a few million of them dolares gringos.
I never thought I'd say this, but Mr Stevenson (my positively crap-diddly-app-not-to-mention-evil-sadistic-bastard Year 11 maths teacher), thank you. For everything.
 :P


We're sorry but frontend doesn't work properly without JavaScript enabled. Please enable it to continue.


I use Autoscaling Groups in AWS for all of my systems. The main benefit for me here was to make sure that when a node died in AWS, the Autoscaling Group policy made sure that the node was replaced. I wanted to get some visibility of when the Autoscaling Group was launching and terminating nodes and decided that posting notifications to Slack would be a good way of getting this. With Terraform and AWS Lambda, I was able to make this happen.
This post assumes that you are already setup and running with Terraform
Create an IAM Role that allows access to AWS Lambda:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
Create a lambda function as follows:
1 2 3 4 5 6 |
|
We assume here, that you have already created a Slack Integration. The hook URL from that integration is required for the lambda contents.
The filename slackNotify.zip
is a zip of a file called slackNotify.js
. The contents of that js file are available
Terraform currently does not support hooking AWS Lambda up to SNS Event Sources. Therefore, unfortunately, there is a manual step required to configure the Lambda to talk to the SNS Topic. There is a PR in Terraform to allow this to be automated as well.
In the AWS Console, go to Lambda and then chose the Lambda function.
Go to the Event Sources
tab:
Click on Add Event Source
and then choose SNS
from the dropdown and then make sure you chose the correct SNS Topic name:
We then use another Terraform resource to attach the Autoscale Groups to the Lambda as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
As we have configured notifications for autoscaling:TEST_NOTIFICATION, when you apply this configuration with Terraform, you will see something similar to the following in Slack:
In the current infrastructure I manage, there are 27 Autoscale groups. I don't really want to add 27 hardcoded groupnames in the awsautoscaling_notifcation in Terraform.
I wanted to take advantage of a Terraform module. In a nutshell, the module does a lookup of all the Autoscaling Groups in a region and then passes that list into the Terraform resource.
The output of the module looks as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
I then pass this into the Terraform resource as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
When anything happens within an Autoscaling Group, I now get notifications as follows:


The Explrer
Type something about the website here. It is a description.
Welcome back explorer. Login to start exploring.
Create an account for a better experience.
Continue exploring.


SDK and SDK Licensing details for the app
PyJWT
This app uses the PyJWT module, which is licensed under the MIT License (MIT), Copyright (c) Jose Padilla.
Setting up Trend Micro Apex One
This app requires an API Key to access the Trend Micro Apex One environment.
Please refer to the documentation "Obtain an Application ID
and API Key" on Apex Central to learn how to create and access such a key.
Phantom Trend Micro Apex One Asset
When creating an asset for Trend Micro Apex One app, place the Apex Server URL (eg.
https://myapexenv.manage.trendmicro.com) into the Apex Server URL field. Next, enter
the Application ID and API Key pair into their respective input fields. After saving the asset, run the Test
Connectivity action under Asset Settings to ensure the asset has been configured correctly.


Other Projects
Brief descriptions of other projects in Samsung Techwin.
My Role
Network camera main developer
Project Detail
I had worked as a Product Engineering team member several years. Tasks of Product Engineering team were to handle additional requirements on Samsung Techwin’s products from other branch corporations. I was able to get many experiences in differnt system, frameworks and devices.
TB-EYE project
- Work on various type OSD
- Implement image based charecter OSD module
- Handle raw image format(eg. yuv420, yuv422)
- Implement C# window application
Military Important facilities project
- Implement communication module
- Design communication protocol
- Communication module receives cgi command from central management system
- Communication module controls PTZ housing via RS-485
- Communication module controls sensors and other external devices via RS-485
SBER bank project
- Design communication protocol
- Implement communication module via RS-232
Transportation projects
- Implement input jpeg logo type OSD function
- Implement input multi lines OSD function
- Implement changing OSD size function
- Handle raw image format(eg. yuv420, yuv422)
Austria OEBB project
- Implement motion detection function by index
German Bonn project
- Train braking system overheating monitoring
- Implement request to check overheating alarm and area to the camera encoder via RS-485
IVIDEON project
- Implement camera LED contorl module via RESTful API
Buthan Dam project
- Implement defog function
Noemis project
- Improve PTZ control via serial communication
Agricultural bank project
- Implement control module connected with bill counter via RS-485
Add on board POC
- Implment video fusion between camera and thermal camera on raspberry pi
Chilsung engineering project:
- Implement camera control routine via RS485, AUX


hi, i'm ryan. i'm an engineer and designer currently working on making access to therapy easier at pace
previously i worked on partnerships at rideOS (acq), amundsen at lyft, and iOS ad products at facebook
i studied computer science at the university of michigan, while i was there i helped run shift and worked on a bunch of side projects
some of my personal inspirations are: jane jacobs, fan ho, jake phelps, ray & charles eames, lucie rie, douglas engelbart, and ai weiwei
outside of work, i shoot photos, run, make ceramics, run, and play guitar. i'm currently based in NYC
links to some of my stuff: twitter, instagram, linkedin, email, resume


Prof. Roddam Narasimha, FRS, Honorary Professor, Engineering Mechanics Unit, JNCASR, Bengaluru
Prof. Anurag Kumar, Director, IISC, Bengaluru
Prof. Satyajith Mayor, Director, NCBS, Bengaluru
Prof. Karisiddappa, Vice Chancellor, VTU, Belagavi
Dr. Vijay Chandru, Chairman & CEO, Stand Life Sciences, Kirloskar Business Park,Bengaluru
Prof. V. Nagaraja, ,Professor, IISC and President, JNCASR, Bengaluru
Prof. B.G. Mulimani,Former Vice Chancellor, Gulbarga University, kalaburgi
Shri Ramaji Raghavan, Chairman, Agastya International Foundation, Bengaluru
Prof. M.R.S. Rao, Molecular Biology and Genetics Unit (MBGU), JNCASR, Bengaluru
Dr. Rudra Pratap, CNSE, IISC, Bengaluru
Shri Jeetendra Singh, K.A.S, Advisor, Dept. of IT, BT and S & T, Bengaluru
Dr. H. Honnegowda, Special Director Technical, MD, KSTePS, Bengaluru
Dr. Vijay Chandru, Chairman & CEO, Stand Life Sciences, Kirloskar Business Park,Bengaluru
Prof. V. Nagaraja, ,Professor, IISC and President, JNCASR, Bengaluru
Formulation of Policy and Programmes of Science and Technology.
Promotion and Development of Science and Technology in the state.
To catalyze, facilitate, promote by supporting Science and Technology promotional programs.
Strengthening institutions with Science and Technology facilities for imparting quality teaching and research.
Prepare Science and Technology infrastructure in terms of quality manpower and other resources to meet the future needs of the state in the state in the Science and Technology sector.
To evolve a vision and to develop an approach for Science and Technology developments through identification of thrust areas.
To recommend strategies for developing high quality Science & Technology human resources.
To suggest means of catalyzing Industry-Academia collaboration.
To work-out a new vision for effective science education.
To create knowledge base on science and disseminating to general public.
Science Advisory Council to the Prime Minister, National Research Professor, and Honorary President, Jawaharlal Nehru Centre for Advance Scientific Research, Jakkur Post, Bengaluru-560 064
Ph :<PHONE_NUMBER>1
Email<EMAIL_ADDRESS>I.A.S.
Principal Secretary to Govt., Dept. of IT, BT, and S & T, Govt. of Karnataka,
6th floor, 5th stage, M.S Building, Ambedkar Road, Bengaluru-560001
Ph :<PHONE_NUMBER>2
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Vision Group on Science and Technology
Room No.702, 7th Floor, 4th Gate, M.S. Building,
Ambedkar Veedhi, Bengaluru-560001,
Mobile :<PHONE_NUMBER>
Phone :080-2204 2013
<EMAIL_ADDRESS>Honorary Professor,
Engineering Mechanics Unit, Jawaharlal Nehru Centre for Advanced Scientific Research,
Jakkur Post, Bengaluru-560 064
Ph :<PHONE_NUMBER>9
Email<EMAIL_ADDRESS>Director,
National Centre for Biological Science,
G.K.V.K. Campus
Bengaluru-560065
Ph :<PHONE_NUMBER>1
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Director,
Indian Institute of Science,
Bengaluru-560012
Ph : 080-2360 0690
Email<EMAIL_ADDRESS>Vice Chancellor,
Vishvesvaraya Technological University,
Jnana Sangama, Santibastawad Road,
Machhe, Belagavi-590014
Ph : 0831-240 5458
Email<EMAIL_ADDRESS>Former Vice Chancellor,
Gulbarga University & BLDE
(Deemed to be University), Vijayapura
Res: Chandra, Plot.No.154,
Rani Chennamma Nagar,
Dharwad-580 001
Email<EMAIL_ADDRESS>Chief Executive Officer,
Azim Premji Foundation,
134 - Doddakannelli, Next to WIPRO corporate office,
Sarjapur road, Bengaluru-560035
Tel : 91-80-6614 4996, Fax:080-6614 4903
Email<EMAIL_ADDRESS>Chief Executive Officer,
Strand Life Sciences,
Sir. C.V. Raman Avenue,
Rajmahal Vilas, Sadashivanagar,
Bengaluru-560080
Ph : 2361 8992-94
Email<EMAIL_ADDRESS>Government of Karnataka
Vidhana Soudha
Ph: 2225 3565, 2203 3382
Fax: 22281021
Email<EMAIL_ADDRESS>Department of Higher Education
Room No.645, 6th floor, Gate No.2, M.S. Building
Bengaluru-01
Phone: 080 2225 2437, 2203 2494
Email<EMAIL_ADDRESS>Education Department (Primary & Secondary Education)
Room: 641, 6th floor, Gate No. 2, M.S. Building,
Bengaluru -01
Phone: 080 2225 7334, 2203 2560
<EMAIL_ADDRESS>Special Director(Technical)
MD, KSTePS
Dept. Science and Technology, GoK
#24/2, vijnana Bhavan, 3rd Floor,
21st main road, Banashankari 2nd stage
Bengaluru -560 070
Phone: 26711166, 26711177
Email<EMAIL_ADDRESS>Prof. Roddam Narasimha, F.R.S., (Chairman Sub-committee - ARP, K-FIST, RGS/F and FDP-ENGG programmes), Honorary Professor, Engineering Mechanics Unit, JNCASR, Jakkur Campus, Bengaluru 560 064
Prof. K. VijayRaghavan, F.R.S., (Member, Sub-Committee), Director, National Centre for Biological Science, G.K.V.K. Campus, Bengaluru-560065
Prof. Karisiddappa (Member, VGST) Vice Chancellor, Visvesvaraya Technological University (VTU), Jnana Sangama, Belgaum 590018
Prof. K.J. Rao, (Member VGST), Emeritus Professor, SSCU, Indian Institute of Science, Bengaluru -560012
Prof. K. Satyamoorthy, (Member VGST), Director, Manipal Life Science Centre Manipal University, Manipal 576104
Prof. B.G. Mulimani (Member VGST), Former Vice Chancellor, Gulbarga University, Kalaburgi & BLDE (Deemed to be University), Vijayapura, Res:Chandra, Plot.No.154, Rani Chennamma Nagar, Dharwad-580001
Dr. P.S. Shankar, (Member VGST), Emeritus Professor, Rajiv Gandhi University of Health Science & Dean of Academy of Respiratory Medicine, Deepti, Behind District Court, Kalaburgi -585102
Prof. G. Umesh, (Member VGST), HOD, Physics & Electronics National Institute of Technology, Karnataka (NITK), Suratkal, Srinivasnagar, Dakshina Kannada District, Mangaluru 575025
Dr. Vijay Chandru, (VGST Member), CEO, Strand Life Science, Kirloskar Business park 5th floor, Bellary Road, Hebbal, Bengaluru 560024
Prof. K.N. Ganeshaiah, (Member VGST), Professor, Department of Genetics and Plant Breeding UAS, GKVK, Bengaluru -560065
Prof. H. P. Puttaraju, (Member VGST), Professor of Sericulture/Life Science and Coordinator, Dept. of Integrated Biological Sciences, School of Natural Science, Jnana Bharati Campus, Bengaluru University, Bengaluru-560 056
Dr. S. G. Sreekanteshwaraswamy, Consultant, Vision Group on Science and Technology, Dept. of IT, BT and S & T, 5th floor, 5th stage, M.S. Building, Dr. Ambedkar Veedhi, Bengaluru 560001
The VGST recommended programs to a total plan size of Rs. 25,824.60 lakhs to take up the programs in phased manner


Under normal circumstances, alcohol-related arrests and charges would be in the double digits. The last year has been anything but normal, though, and the number of underage drinking cases has decreased dramatically.
Ralph Harvey, captain of the Athens Police Department, said between Jan. 1 and April 5, only nine people were charged with underage drinking. The number of cases is down slightly from 17 people in that same time frame in 2020 and down significantly from 59 people during the same time frame in 2019.
“There are many factors that can play a role in this, however, the pandemic is the most likely, even if it is indirectly,” Harvey said in an email. “Similar to other crimes; less people in town, less activity, bars closing earlier, other businesses that sell alcohol such as restaurants doing less business overall, all play a role in this.”
The Ohio Investigative Unit, or OIU, has also seen a decline in alcohol-related citations while COVID-19 health order violations have increased, Eric Wolf, enforcement commander with the unit, said.
In many cases, catching people drinking underage is not even the primary reason for APD’s presence.
“Most of our arrests for underage (drinking) are in conjunction to another arrest or behavior that caused our intervention,” Harvey said in an email. “For example, we get called to a disturbance at a bar, it is two intoxicated persons and one or both are found to be underage. This can be anywhere in Athens, but most are in the uptown area.”
OIU’s work, however, is directly related to complaints it receives from the community or local law enforcement partners, Wolf said.
“Typically, we will receive complaints on various alcohol violations, be it underage sales, sales to intoxicated persons, any kind of illegal activity that's taking place on a liquor permitted establishment in Ohio,” Wolf said. “And then we go and make visits. So, our agents will visit these locations again in a plainclothes capacity, basically observe the actions that are taking place, and if there's any type of illegal activity taking place, be it criminal or administrative, then we take the appropriate actions.”
Wolf said action can also be taken against underage individuals who are drinking at house parties, in mall parking lots or at state or federal parks, among other locations.
For individuals who are found to be consuming alcohol underage, the charges associated with an arrest can span a wide gamut, depending on actions associated with alcohol consumption.
Mary Berger | Art Director
“Disorderly conduct by intoxication is the most common, but it can be almost anything,” Harvey said in an email. “If it is in conjunction with a more serious crime, such as assault, the person will usually be charged with the more serious crime and not underage (drinking).”
Similar to the specific charges given to individuals who drink underage, Wolf said the length of jail time for underage drinking offenders can vary widely.
Underage drinking violations are usually first-degree misdemeanors, Wolf said. First-time offenders are usually able to opt in to a diversion program or face a fine. Additionally, many individuals do not get jail time for their first offense but are instead given a citation and a day to appear in court. At a maximum, the penalty for a first degree misdemeanor in Ohio is six months of jail and/or a fine of up to $1,000.
In the eyes of Athens Mayor Steve Patterson, there are plenty of reasons for students not to drink underage, including a basic adherence to the law.
“Number one, you shouldn't (drink underage) because it's against the law. Number two is that we are under a pandemic,” Patterson said. “Think about the social inhibitions that occur by not being intoxicated. Those inhibitions go away when you are intoxicated, and you run the greater risk of being exposed, of coming in contact with someone else who may have COVID-19.”
Patterson also touched on a more dire consequence of drinking underage: serious injury or death.
“A good example to give you is Collin’s Law, that hopefully Collin’s Law is going to go through, due to the unfortunate death of one of our OU students due to hazing. With hazing … often excessive drinking is taking place, which takes lives,” Patterson said. “The law’s in place for a reason.”


A results driven Full-Stack Developer excited about building solutions. Transiting into programming after being impassioned by leading a local chapter of Girl Develop It. Combines past experience of event execution, marketing, collaborating with cross-functional teams to create and maintain dynamic websites, applications and databases that are user-experience driven in design and functionality.
HTML5 and CSS3
JavaScript and jQuery
Git and Github
MySQL and MongoDB
React and React Redux
Express
MERN Application This a full stack development project utilizing MongoDB, Express, Node, and React. This is my first full stack application. It searches The New York Times based on the users search parameters and then the user can choose to save those articles to the Mongo Database.
This project is an online store using JavaScript, Node.JS, and Mysql. This application runs through terminal.
This project accesses Giphy's API. This application allow the user to search for gifs based on the keyword used and it saves each search.
This project uses node and can access spotify, twitter, and movie information with three very specific commands. I really enjoyed programming this application.


fast_ml fast_ml-2013 fast_ml-2013-41 knowledge-graph by maker-knowledge-mining
Source: html
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology.  Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
sentIndex sentText sentNum sentScore
1 This post was inspired by a very good article about big data by Chris Stucchio (linked below). [sent-2, score-0.589]
2 Big data is hype Everybody talks about big data; nobody knows exactly what it is. [sent-5, score-0.9]
3 Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). [sent-7, score-0.195]
4 Now, to put things in context: Big data is right there (or maybe not quite yet? [sent-8, score-0.2]
5 In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. [sent-11, score-0.922]
6 What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. [sent-13, score-1.052]
7 Before cloud computing it was data warehousing or ‘software as a service’. [sent-14, score-0.349]
8 There’s a new buzzword every two years and the computer analysts come out with these things so that they will have something to consult about. [sent-15, score-0.223]
9 uk] Also see Most data isn’t big , and businesses are wasting money pretending it is and a paper from Microsoft: Nobody ever got fired for buying a cluster . [sent-18, score-0.595]
10 Another way to say it: big data is like a teenage sex… You already know this meme, don’t you? [sent-19, score-0.473]
11 Big data is technical difficulty Big data can be defined in terms of technical difficulty it causes. [sent-20, score-0.773]
12 For example, when deciding if your data is big, you could draw a line on whether it fits comfortably* into memory. [sent-21, score-0.332]
13 The point is, as data grows larger it becomes more difficult to process it. [sent-23, score-0.216]
14 The author is talking about Map Reduce: The only reason to put on this straightjacket is that by doing so, you can scale up to extremely large data sets. [sent-25, score-0.407]
15 Big data is effective If it’s hype and a source of difficulties, why bother? [sent-30, score-0.454]
16 In machine learning, particularly, more examples usually is better, especially when data dimensionality is high. [sent-35, score-0.193]
17 Which brings us to the last point… Big data is spying Consider a task the big guys like Google, Facebook etc. [sent-40, score-0.473]
18 are dealing with: they have visitor data from hundreds of thousands or millions or n sites. [sent-41, score-0.204]
19 Apple published a transparency report, notable for its, let’s say, tagline: our business does not depend on collecting personal data . [sent-52, score-0.334]
20 For a company not interested in personal data they’re pretty nosy, aren’t they? [sent-54, score-0.273]
wordName wordTfidf (topN-words)
[('big', 0.346), ('corpus', 0.244), ('apple', 0.22), ('hype', 0.22), ('cloud', 0.161), ('analysts', 0.146), ('brown', 0.146), ('nobody', 0.146), ('nsa', 0.146), ('personal', 0.146), ('sites', 0.146), ('straightjacket', 0.146), ('stucchio', 0.146), ('term', 0.134), ('google', 0.132), ('data', 0.127), ('ram', 0.122), ('chris', 0.122), ('difficulty', 0.122), ('article', 0.116), ('errors', 0.107), ('comfortably', 0.107), ('technical', 0.107), ('effective', 0.107), ('web', 0.107), ('norvig', 0.097), ('larger', 0.089), ('whether', 0.083), ('computer', 0.077), ('dealing', 0.077), ('put', 0.073), ('examples', 0.066), ('computing', 0.061), ('got', 0.061), ('fits', 0.061), ('money', 0.061), ('draw', 0.061), ('quiet', 0.061), ('extremely', 0.061), ('overview', 0.061), ('talks', 0.061), ('aren', 0.061), ('linked', 0.061), ('media', 0.061), ('mining', 0.061), ('social', 0.061), ('announced', 0.061), ('business', 0.061), ('coming', 0.061), ('defined', 0.061)]
simIndex simValue blogId blogTitle
same-blog 1 1.0000005 41 fast ml-2013-10-09-Big data made easy
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology.  Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
2 0.13892294 37 fast ml-2013-09-03-Our followers and who else they follow
Introduction: Recently we hit 400 followers mark on Twitter. To celebrate we decided to do some data mining on you , specifically to discover who our followers are and who else they follow. For your viewing pleasure we packaged the results nicely with Bootstrap. Here’s some data science in action. Our followers This table show our 20 most popular followers as measeared by their follower count. The occasional question marks stand for non-ASCII characters. Each link opens a new window. Followers Screen name Name Description 8685 pankaj Pankaj Gupta I lead the Personalization and Recommender Systems group at Twitter. Founded two startups in the past. 5070 ogrisel Olivier Grisel Datageek, contributor to scikit-learn, works with Python / Java / Clojure / Pig, interested in Machine Learning, NLProc, {Big|Linked|Open} Data and braaains! 4582 thuske thuske & 4442 ram Ram Ravichandran So
3 0.099074133 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
Introduction: On May 15th Yann LeCun answered “ask me anything” questions on Reddit . We hand-picked some of his thoughts and grouped them by topic for your enjoyment. Toronto, Montreal and New York All three groups are strong and complementary. Geoff (who spends more time at Google than in Toronto now) and Russ Salakhutdinov like RBMs and deep Boltzmann machines. I like the idea of Boltzmann machines (it’s a beautifully simple concept) but it doesn’t scale well. Also, I totally hate sampling. Yoshua and his colleagues have focused a lot on various unsupervised learning, including denoising auto-encoders, contracting auto-encoders. They are not allergic to sampling like I am. On the application side, they have worked on text, not so much on images. In our lab at NYU (Rob Fergus, David Sontag, me and our students and postdocs), we have been focusing on sparse auto-encoders for unsupervised learning. They have the advantage of scaling well. We have also worked on applications, mostly to v
4 0.093386173 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
Introduction: A/B testing is a way to optimize a web page. Half of visitors see one version, the other half another, so you can tell which version is more conducive to your goal - for example selling something. Since June 2013 A/B testing can be conveniently done with Google Analytics. Here’s how. This article is not quite about machine learning. If you’re not interested in testing, scroll down to the bayesian bandits section . Google Content Experiments We remember Google Website Optimizer from a few years ago. It wasn’t exactly user friendly or slick, but it felt solid and did the job. Unfortunately, at one point in time Google pulled the plug, leaving Genetify as a sole free (and open source) tool for multivariate testing. Multivariate means testing a few elements on a page simultanously. At that time they launched Content Experiments in Google Analytics, but it was a giant step backward. Content experiments were very primitive and only allowed rudimentary A/B split testing. It i
5 0.090366691 19 fast ml-2013-02-07-The secret of the big guys
Introduction: Are you interested in linear models, or K-means clustering? Probably not much. These are very basic techniques with fancier alternatives. But here’s the bomb: when you combine those two methods for supervised learning, you can get better results than from a random forest. And maybe even faster. We have already written about Vowpal Wabbit , a fast linear learner from Yahoo/Microsoft. Google’s response (or at least, a Google’s guy response) seems to be Sofia-ML . The software consists of two parts: a linear learner and K-means clustering. We found Sofia a while ago and wondered about K-means: who needs K-means? Here’s a clue: This package can be used for learning cluster centers (…) and for mapping a given data set onto a new feature space based on the learned cluster centers. Our eyes only opened when we read a certain paper, namely An Analysis of Single-Layer Networks in Unsupervised Feature Learning ( PDF ). The paper, by Coates , Lee and Ng, is about object recogni
6 0.087565042 5 fast ml-2012-09-19-Best Buy mobile contest - big data
7 0.085809767 15 fast ml-2013-01-07-Machine learning courses online
8 0.073488533 29 fast ml-2013-05-25-More on sparse filtering and the Black Box competition
9 0.068714879 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
10 0.068446122 27 fast ml-2013-05-01-Deep learning made easy
11 0.065416977 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
12 0.056780953 42 fast ml-2013-10-28-How much data is enough?
13 0.054281265 20 fast ml-2013-02-18-Predicting advertised salaries
14 0.052937668 57 fast ml-2014-04-01-Exclusive Geoff Hinton interview
15 0.05271196 7 fast ml-2012-10-05-Predicting closed questions on Stack Overflow
16 0.052089911 32 fast ml-2013-07-05-Processing large files, line by line
17 0.051545501 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
18 0.049779098 2 fast ml-2012-08-27-Kaggle job recommendation challenge
19 0.049383242 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
20 0.046895109 60 fast ml-2014-04-30-Converting categorical data into numbers with Pandas and Scikit-learn
topicId topicWeight
[(0, 0.215), (1, 0.049), (2, 0.151), (3, -0.118), (4, 0.124), (5, 0.12), (6, -0.04), (7, 0.13), (8, 0.272), (9, 0.027), (10, 0.008), (11, 0.202), (12, -0.062), (13, -0.126), (14, 0.107), (15, -0.015), (16, -0.006), (17, 0.034), (18, 0.202), (19, 0.16), (20, -0.137), (21, -0.174), (22, -0.324), (23, 0.077), (24, -0.014), (25, 0.12), (26, -0.075), (27, -0.225), (28, 0.171), (29, -0.059), (30, -0.129), (31, -0.079), (32, -0.076), (33, 0.158), (34, -0.034), (35, 0.163), (36, -0.046), (37, 0.01), (38, -0.148), (39, -0.056), (40, -0.021), (41, -0.253), (42, 0.363), (43, -0.053), (44, 0.036), (45, 0.102), (46, 0.059), (47, 0.027), (48, 0.026), (49, -0.131)]
simIndex simValue blogId blogTitle
same-blog 1 0.98026085 41 fast ml-2013-10-09-Big data made easy
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology.  Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
2 0.36318693 37 fast ml-2013-09-03-Our followers and who else they follow
Introduction: Recently we hit 400 followers mark on Twitter. To celebrate we decided to do some data mining on you , specifically to discover who our followers are and who else they follow. For your viewing pleasure we packaged the results nicely with Bootstrap. Here’s some data science in action. Our followers This table show our 20 most popular followers as measeared by their follower count. The occasional question marks stand for non-ASCII characters. Each link opens a new window. Followers Screen name Name Description 8685 pankaj Pankaj Gupta I lead the Personalization and Recommender Systems group at Twitter. Founded two startups in the past. 5070 ogrisel Olivier Grisel Datageek, contributor to scikit-learn, works with Python / Java / Clojure / Pig, interested in Machine Learning, NLProc, {Big|Linked|Open} Data and braaains! 4582 thuske thuske & 4442 ram Ram Ravichandran So
3 0.20904961 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA
Introduction: On May 15th Yann LeCun answered “ask me anything” questions on Reddit . We hand-picked some of his thoughts and grouped them by topic for your enjoyment. Toronto, Montreal and New York All three groups are strong and complementary. Geoff (who spends more time at Google than in Toronto now) and Russ Salakhutdinov like RBMs and deep Boltzmann machines. I like the idea of Boltzmann machines (it’s a beautifully simple concept) but it doesn’t scale well. Also, I totally hate sampling. Yoshua and his colleagues have focused a lot on various unsupervised learning, including denoising auto-encoders, contracting auto-encoders. They are not allergic to sampling like I am. On the application side, they have worked on text, not so much on images. In our lab at NYU (Rob Fergus, David Sontag, me and our students and postdocs), we have been focusing on sparse auto-encoders for unsupervised learning. They have the advantage of scaling well. We have also worked on applications, mostly to v
4 0.2015253 19 fast ml-2013-02-07-The secret of the big guys
Introduction: Are you interested in linear models, or K-means clustering? Probably not much. These are very basic techniques with fancier alternatives. But here’s the bomb: when you combine those two methods for supervised learning, you can get better results than from a random forest. And maybe even faster. We have already written about Vowpal Wabbit , a fast linear learner from Yahoo/Microsoft. Google’s response (or at least, a Google’s guy response) seems to be Sofia-ML . The software consists of two parts: a linear learner and K-means clustering. We found Sofia a while ago and wondered about K-means: who needs K-means? Here’s a clue: This package can be used for learning cluster centers (…) and for mapping a given data set onto a new feature space based on the learned cluster centers. Our eyes only opened when we read a certain paper, namely An Analysis of Single-Layer Networks in Unsupervised Feature Learning ( PDF ). The paper, by Coates , Lee and Ng, is about object recogni
5 0.18042299 5 fast ml-2012-09-19-Best Buy mobile contest - big data
Introduction: Last time we talked about the small data branch of Best Buy contest . Now it’s time to tackle the big boy . It is positioned as “cloud computing sized problem”, because there is 7GB of unpacked data, vs. younger brother’s 20MB. This is reflected in “cloud computing” and “cluster” and “Oracle” talk in the forum , and also in small number of participating teams: so far, only six contestants managed to beat the benchmark. But don’t be scared. Most of data mass is in XML product information. Training and test sets together are 378MB. Good news. The really interesting thing is that we can take the script we’ve used for small data and apply it to this contest, obtaining 0.355 in a few minutes (benchmark is 0.304). Not impressed? With simple extension you can up the score to 0.55. Read below for details. This is the very same script , with one difference. In this challenge, benchmark recommendations differ from line to line, so we can’t just hard-code five item IDs like before
6 0.16719963 21 fast ml-2013-02-27-Dimensionality reduction for sparse binary data
7 0.15322889 20 fast ml-2013-02-18-Predicting advertised salaries
8 0.14178911 47 fast ml-2013-12-15-A-B testing with bayesian bandits in Google Analytics
9 0.13989198 59 fast ml-2014-04-21-Predicting happiness from demographics and poll answers
10 0.13874939 2 fast ml-2012-08-27-Kaggle job recommendation challenge
11 0.13741782 60 fast ml-2014-04-30-Converting categorical data into numbers with Pandas and Scikit-learn
12 0.13390027 29 fast ml-2013-05-25-More on sparse filtering and the Black Box competition
13 0.1318763 27 fast ml-2013-05-01-Deep learning made easy
14 0.12595797 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
15 0.121056 40 fast ml-2013-10-06-Pylearn2 in practice
16 0.11975475 24 fast ml-2013-03-25-Dimensionality reduction for sparse binary data - an overview
17 0.11971775 38 fast ml-2013-09-09-Predicting solar energy from weather forecasts plus a NetCDF4 tutorial
18 0.1195166 42 fast ml-2013-10-28-How much data is enough?
19 0.11668549 15 fast ml-2013-01-07-Machine learning courses online
20 0.11421274 25 fast ml-2013-04-10-Gender discrimination
topicId topicWeight
[(26, 0.032), (31, 0.052), (35, 0.034), (55, 0.016), (58, 0.022), (69, 0.098), (71, 0.018), (73, 0.015), (78, 0.03), (81, 0.012), (84, 0.034), (92, 0.485), (99, 0.072)]
simIndex simValue blogId blogTitle
same-blog 1 0.88875175 41 fast ml-2013-10-09-Big data made easy
Introduction: An overview of key points about big data. This post was inspired by a very good article about big data by Chris Stucchio (linked below). The article is about hype and technology.  Big data is hype Everybody talks about big data; nobody knows exactly what it is. That’s pretty much the definition of hype. Google Trends suggest that the term took off at the beginning of 2011 (and the searches are coming mainly from Asia, curiously). Now, to put things in context: Big data is right there (or maybe not quite yet?) with other slogans like web 2.0 , cloud computing and social media . In effect, big data is a generic term for: data science machine learning data mining predictive analytics and so on. Don’t believe us? What about James Goodnight, the CEO of SAS : The term big data is being used today because computer analysts and journalists got tired of writing about cloud computing. Before cloud computing it was data warehousing or
2 0.24872841 19 fast ml-2013-02-07-The secret of the big guys
Introduction: Are you interested in linear models, or K-means clustering? Probably not much. These are very basic techniques with fancier alternatives. But here’s the bomb: when you combine those two methods for supervised learning, you can get better results than from a random forest. And maybe even faster. We have already written about Vowpal Wabbit , a fast linear learner from Yahoo/Microsoft. Google’s response (or at least, a Google’s guy response) seems to be Sofia-ML . The software consists of two parts: a linear learner and K-means clustering. We found Sofia a while ago and wondered about K-means: who needs K-means? Here’s a clue: This package can be used for learning cluster centers (…) and for mapping a given data set onto a new feature space based on the learned cluster centers. Our eyes only opened when we read a certain paper, namely An Analysis of Single-Layer Networks in Unsupervised Feature Learning ( PDF ). The paper, by Coates , Lee and Ng, is about object recogni
3 0.24214117 9 fast ml-2012-10-25-So you want to work for Facebook
Introduction: Good news, everyone! There’s a new contest on Kaggle - Facebook is looking for talent . They won’t pay, but just might interview. This post is in a way a bonus for active readers because most visitors of fastml.com originally come from Kaggle forums. For this competition the forums are disabled to encourage own work . To honor this, we won’t publish any code. But own work doesn’t mean original work , and we wouldn’t want to reinvent the wheel, would we? The contest differs substantially from a Kaggle stereotype, if there is such a thing, in three major ways: there’s no money prizes, as mentioned above it’s not a real world problem, but rather an assignment to screen job candidates (this has important consequences, described below) it’s not a typical machine learning project, but rather a broader AI exercise You are given a graph of the internet, actually a snapshot of the graph for each of 15 time steps. You are also given a bunch of paths in this graph, which a
4 0.23838973 48 fast ml-2013-12-28-Regularizing neural networks with dropout and with DropConnect
Introduction: We continue with CIFAR-10-based competition at Kaggle to get to know DropConnect. It’s supposed to be an improvement over dropout. And dropout is certainly one of the bigger steps forward in neural network development. Is DropConnect really better than dropout? TL;DR DropConnect seems to offer results similiar to dropout. State of the art scores reported in the paper come from model ensembling. Dropout Dropout , by Hinton et al., is perhaps a biggest invention in the field of neural networks in recent years. It adresses the main problem in machine learning, that is overfitting. It does so by “dropping out” some unit activations in a given layer, that is setting them to zero. Thus it prevents co-adaptation of units and can also be seen as a method of ensembling many networks sharing the same weights. For each training example a different set of units to drop is randomly chosen. The idea has a biological inspiration . When a child is conceived, it receives half its genes f
5 0.23540984 16 fast ml-2013-01-12-Intro to random forests
Introduction: Let’s step back from forays into cutting edge topics and look at a random forest, one of the most popular machine learning techniques today. Why is it so attractive? First of all, decision tree ensembles have been found by Caruana et al. as the best overall approach for a variety of problems. Random forests, specifically, perform well both in low dimensional and high dimensional tasks. There are basically two kinds of tree ensembles: bagged trees and boosted trees. Bagging means that when building each subsequent tree, we don’t look at the earlier trees, while in boosting we consider the earlier trees and strive to compensate for their weaknesses (which may lead to overfitting). Random forest is an example of the bagging approach, less prone to overfit. Gradient boosted trees (notably GBM package in R) represent the other one. Both are very successful in many applications. Trees are also relatively fast to train, compared to some more involved methods. Besides effectivnes
6 0.2251019 27 fast ml-2013-05-01-Deep learning made easy
7 0.22497991 18 fast ml-2013-01-17-A very fast denoising autoencoder
8 0.22157489 23 fast ml-2013-03-18-Large scale L1 feature selection with Vowpal Wabbit
9 0.22127399 12 fast ml-2012-12-21-Tuning hyperparams automatically with Spearmint
10 0.22114109 54 fast ml-2014-03-06-PyBrain - a simple neural networks library in Python
11 0.21993391 43 fast ml-2013-11-02-Maxing out the digits
12 0.21906021 40 fast ml-2013-10-06-Pylearn2 in practice
13 0.21681106 55 fast ml-2014-03-20-Good representations, distance, metric learning and supervised dimensionality reduction
14 0.21661294 13 fast ml-2012-12-27-Spearmint with a random forest
15 0.21612206 4 fast ml-2012-09-17-Best Buy mobile contest
16 0.21470615 17 fast ml-2013-01-14-Feature selection in practice
17 0.21447825 61 fast ml-2014-05-08-Impute missing values with Amelia
18 0.21331151 25 fast ml-2013-04-10-Gender discrimination
19 0.2132885 45 fast ml-2013-11-27-Object recognition in images with cuda-convnet
20 0.2125206 62 fast ml-2014-05-26-Yann LeCun's answers from the Reddit AMA


NOTE!
- Click on the [ Add Rows ] button to add two new rows.
- The "addRows" method was added in version 2.0.16, and is not part of the original plugin.
- This example could have used the "update" method to add rows, but to remove rows only the "update" method works.
- This method is better used for the pager plugin when not all of the rows are displayed.
Demo
| First Name | Last Name | Age | Total | Discount | Date |
|---|---|---|---|---|---|
| Peter | Parker | 28 | $9.99 | 20% | Jul 6, 2006 8:14 AM |
| John | Hood | 33 | $19.99 | 25% | Dec 10, 2002 5:14 AM |
| Clark | Kent | 18 | $15.89 | 44% | Jan 12, 2003 11:14 AM |
| Bruce | Almighty | 45 | $153.19 | 44% | Jan 18, 2001 9:12 AM |
| Bruce | Evans | 22 | $13.19 | 11% | Jan 18, 2007 9:12 AM |


« Tutorial 6: Setup Table :: Contents :: Tutorial 8: Execute Other Steps in a Step »
| Goal: | Use result tables to simplify comparison of an expected dataset. |
|---|
The usage of result tables come in variations. It often depends what you want to compare. These variations in the test automation layer are:
- ordered dataset comparison
- unordered dataset comparison
- ordered subset comparison (result table contains subset)
- unordered subset comparison (result table contains subset)
Hint
The FIT test framework provides similar concepts via Fixtures. An extension of FIT, the FitLibrary, provides even more advanced fixtures classes/tables.
| Dataset | Unordered comparison | Ordered Comparison |
|---|---|---|
| Subset | fitlibrary.SubsetFixture, fit.RowFixture (with table args) | fitlibrary.ArrayFixture (variant) |
| Complete | fit.RowFixture, fitlibrary.SetFixture | fitlibrary.ArrayFixture |
Besides other descriptions of these Fixtures, the Fixture Gallery project provides examples for these fixture in several languages.
Both, unordered dataset comparison and unordered subset comparison are used in this tutorial in two different scenarios.
# file:features/tutorial07_step_result_table.feature
Feature: Step Result Table (tutorial07)
Scenario: Unordered Result Table Comparison (RowFixture Table)
Given a set of specific users:
| name | department |
| Alice | Beer Cans |
| Bob | Beer Cans |
| Charly | Silly Walks |
| Dodo | Silly Walks |
Then we will have the following people in "Silly Walks":
| name |
| Charly |
| Dodo |
And we will have the following people in "Beer Cans":
| name |
| Bob |
| Alice |
Scenario: Subset Result Table Comparison
Given a set of specific users:
| name | department |
| Alice | Super-sonic Cars |
| Bob | Super-sonic Cars |
Then we will have at least the following people in "Super-sonic Cars":
| name |
| Alice |
# file:features/steps/step_tutorial07.py
# ----------------------------------------------------------------------------
# STEPS:
# ----------------------------------------------------------------------------
from behave import given, when, then
from hamcrest import assert_that, has_items
from hamcrest.library.collection.issequence_containinginanyorder \
import contains_inanyorder
@then('we will have the following people in "{department}"')
def step_impl(context, department):
"""
Compares expected with actual persons in a department.
NOTE: Unordered comparison (ordering is not important).
"""
department_ = context.model.departments.get(department, None)
if not department_:
assert_that(False, "Department %s is unknown" % department)
# -- NORMAl-CASE:
expected_persons = [ row["name"] for row in context.table ]
actual_persons = department_.members
# -- UNORDERED TABLE-COMPARISON (using: pyhamcrest)
assert_that(contains_inanyorder(*expected_persons), actual_persons)
@then('we will have at least the following people in "{department}"')
def step_impl(context, department):
"""
Compares subset of persons with actual persons in a department.
NOTE: Unordered subset comparison.
"""
department_ = context.model.departments.get(department, None)
if not department_:
assert_that(False, "Department %s is unknown" % department)
# -- NORMAl-CASE:
expected_persons = [ row["name"] for row in context.table ]
actual_persons = department_.members
# -- TABLE-SUBSET-COMPARISON (using: pyhamcrest)
assert_that(has_items(*expected_persons), actual_persons)
# @mark.more_steps
# ----------------------------------------------------------------------------
# MORE STEPS: step_tutorial06.py
# ----------------------------------------------------------------------------
# @given('a set of specific users')
# def step_impl(context):
# model = getattr(context, "model", None)
# if not model:
# context.model = CompanyModel()
# for row in context.table:
# context.model.add_user(row["name"], deparment=row["department"])
#
# @when('we count the number of people in each department')
# def step_impl(context):
# context.model.count_persons_per_department()
# file:features/steps/company_model.py
# -----------------------------------------------------------------------------
# DOMAIN-MODEL:
# -----------------------------------------------------------------------------
class Department(object):
def __init__(self, name, members=None):
if not members:
members = []
self.name = name
self.members = members
def add_member(self, name):
assert name not in self.members
self.members.append(name)
@property
def count(self):
return len(self.members)
def __len__(self):
return self.count
class CompanyModel(object):
def __init__(self):
self.users = []
self.departments = {}
def add_user(self, name, deparment):
assert name not in self.users
if not self.departments.has_key(deparment):
self.departments[deparment] = Department(deparment)
self.departments[deparment].add_member(name)
def count_persons_per_department(self):
pass
def get_headcount_for(self, department):
return self.departments[department].count
When you run the feature file from above:
$ behave ../features/tutorial07_step_result_table.feature Feature: Step Result Table (tutorial07) # ../features/tutorial07_step_result_table.feature:1 Scenario: Unordered Result Table Comparison (RowFixture Table) # ../features/tutorial07_step_result_table.feature:3 Given a set of specific users # ../features/steps/step_tutorial06.py:28 | name | department | | Alice | Beer Cans | | Bob | Beer Cans | | Charly | Silly Walks | | Dodo | Silly Walks | Then we will have the following people in "Silly Walks" # ../features/steps/step_tutorial07.py:46 | name | | Charly | | Dodo | And we will have the following people in "Beer Cans" # ../features/steps/step_tutorial07.py:46 | name | | Bob | | Alice | Scenario: Subset Result Table Comparison # ../features/tutorial07_step_result_table.feature:20 Given a set of specific users # ../features/steps/step_tutorial06.py:28 | name | department | | Alice | Super-sonic Cars | | Bob | Super-sonic Cars | Then we will have at least the following people in "Super-sonic Cars" # ../features/steps/step_tutorial07.py:62 | name | | Alice | 1 feature passed, 0 failed, 0 skipped 2 scenarios passed, 0 failed, 0 skipped 5 steps passed, 0 failed, 0 skipped, 0 undefined Took 0m0.001s


Multiple Consoles
In this tutorial we’ll create sub consoles for each part of our game window and Blit them to the root console.
Sub Consoles
Our game is more than a map. It will also have areas for the inventory, player statistics, and a message log. We could just draw all these directly to the root console but it will be better to draw each to their own console and Blit them to the root in their correct positions.
Defining Console Dimensions
First we need to define the width and height of each of our sub-consoles and declare each RLConsole. At the top of Game.cs
add the following code:
public static class Game
{
// The screen height and width are in number of tiles
private static readonly int _screenWidth = 100;
private static readonly int _screenHeight = 70;
private static RLRootConsole _rootConsole;
// The map console takes up most of the screen and is where the map will be drawn
private static readonly int _mapWidth = 80;
private static readonly int _mapHeight = 48;
private static RLConsole _mapConsole;
// Below the map console is the message console which displays attack rolls and other information
private static readonly int _messageWidth = 80;
private static readonly int _messageHeight = 11;
private static RLConsole _messageConsole;
// The stat console is to the right of the map and display player and monster stats
private static readonly int _statWidth = 20;
private static readonly int _statHeight = 70;
private static RLConsole _statConsole;
// Above the map is the inventory console which shows the players equipment, abilities, and items
private static readonly int _inventoryWidth = 80;
private static readonly int _inventoryHeight = 11;
private static RLConsole _inventoryConsole;
// ... additional code omitted
}
Instantiate the Consoles
Now we need to new up our four consoles with the widths and heights that we defined. In our Main()
just after the line where we make a new RLRootConsole add the following code:
public static void Main()
{
// ... previous code omitted
// After the line that starts _rootConsole = new RLRootConsole( ...
// Initialize the sub consoles that we will Blit to the root console
_mapConsole = new RLConsole( _mapWidth, _mapHeight );
_messageConsole = new RLConsole( _messageWidth, _messageHeight );
_statConsole = new RLConsole( _statWidth, _statHeight );
_inventoryConsole = new RLConsole( _inventoryWidth, _inventoryHeight );
// ... additional code omitted
}
Set Background Color and Label
Next we’ll want to set the background color of each console and put a label on them so that we can verify that they are in the correct positions when we Blit them to the Root console. In OnRootConsoleUpdate()
replace _rootConsole.Print( 10, 10, “It worked!”, RLColor.White );
with the following code:
private static void OnRootConsoleUpdate( object sender, UpdateEventArgs e )
{
// Set background color and text for each console
// so that we can verify they are in the correct positions
_mapConsole.SetBackColor( 0, 0, _mapWidth, _mapHeight, RLColor.Black );
_mapConsole.Print( 1, 1, "Map", RLColor.White );
_messageConsole.SetBackColor( 0, 0, _messageWidth, _messageHeight, RLColor.Gray );
_messageConsole.Print( 1, 1, "Messages", RLColor.White );
_statConsole.SetBackColor( 0, 0, _statWidth, _statHeight, RLColor.Brown );
_statConsole.Print( 1, 1, "Stats", RLColor.White );
_inventoryConsole.SetBackColor( 0, 0, _inventoryWidth, _inventoryHeight, RLColor.Cyan );
_inventoryConsole.Print( 1, 1, "Inventory", RLColor.White );
}
Blit and Render
The final step we’ll need to take is to update OnRootConsoleRender()
to Blit each of our consoles to the root console in the correct positions. The RLNET method we will use is RLConsole.Blit which takes several parameters.
A source console
- The X and Y position of the top left corner of a rectangular area from the source
- The width and height of the rectangular area from the source that we will be Blitting
- A destination console to Blit to
- The X and Y position of the top left corner of where we will Blit to in the destination console
private static void OnRootConsoleRender( object sender, UpdateEventArgs e )
{
// Blit the sub consoles to the root console in the correct locations
RLConsole.Blit( _mapConsole, 0, 0, _mapWidth, _mapHeight,
_rootConsole, 0, _inventoryHeight );
RLConsole.Blit( _statConsole, 0, 0, _statWidth, _statHeight,
_rootConsole, _mapWidth, 0 );
RLConsole.Blit( _messageConsole, 0, 0, _messageWidth, _messageHeight,
_rootConsole, 0, _screenHeight - _messageHeight );
RLConsole.Blit( _inventoryConsole, 0, 0, _inventoryWidth, _inventoryHeight,
_rootConsole, 0, 0 );
// Tell RLNET to draw the console that we set
_rootConsole.Draw();
}
And that’s it! If you run the program now you should see output similar to the screenshot at the beginning of this post.
Code on GitHub
As always the code for the tutorial series so far can be found on GitHub:
Bored waiting for the next tutorial? The complete tutorial project is already finished and the source code is available on Github:
- Sample Roguelike game using RogueSharp and RLNet console
- Sample Roguelike game using RogueSharp and SadConsole
Closing thoughts
So far we haven’t used RogueSharp at all. I think it is necessary to do some of these basic setup tasks to pave the way for the future. Don’t worry though we’ll be getting into RogueSharp functionality shortly.


- Installing on Amazon Elastic Compute Cloud (Amazon EC2)
- Appendix
- Appliance Console Command-Line Interface (CLI)
Installing on Amazon Elastic Compute Cloud (Amazon EC2)
Installing ManageIQ
Installing ManageIQ consists of the following steps:
-
Downloading the appliance for your environment as a virtual machine snapshot template.
-
Setting up a virtual machine based on the appliance.
-
Configuring the ManageIQ appliance.
After you have completed all the procedures in this guide, you will have a working environment on which additional customizations and configurations can be performed.
Obtaining the appliance
Requirements
Below are the two sets of requirements for installing ManageIQ on Amazon EC2.
ManageIQ Requirements
-
44 GB of space on the chosen datastore.
-
12 GB RAM.
-
4 vCPUs.
Amazon EC2 Requirements
-
An Amazon S3 bucket to store the disk image that will be imported to AWS as a snapshot.
-
A VM import service role (IAM role) named
vmimport
.
For information on creating an Amazon S3 bucket and a VM Import Service Role, see the Amazon EC2 documentation.
Uploading the Appliance to an Amazon S3 Bucket
From your local file system, you can now upload the ManageIQ appliance VHD image obtained in Obtaining the appliance to the Amazon S3 bucket, using your choice of tool, such as AWS client or directly upload the Appliance image from your S3 bucket on AWS Management Console.
Configuring Amazon EC2 to Import the Appliance
Important:
These are the procedural steps as of the time of writing. For the latest information on importing a virtual machine as an image, see the Amazon EC2 documentation.
-
Install the AWS client on the computer you want to interact with the AWS API from.
$ pip install awscli
-
Configure and download your AWS secret/access key by following the steps in the Managing Access Keys for Your AWS Account documentation.
-
Configure the AWS client with your access/secret key. For example:
$ aws configure AWS Access Key ID [******]: ACCESS_KEY AWS Secret Access Key [******]: SECRET_KEY Default region name [None]: Default output format [None]:
-
Create the
trust-policy.json
file for the vmimport role. For example:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "vmie.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals":{ "sts:Externalid": "vmimport" } } } ] }
-
Create the vmimport role using the
trust-policy.json
file that you just created.$ aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json
Note:
This user must have permissions to create and modify IAM roles.
-
Create the
role-policy.json
file. Be sure to use the exact S3 bucket name. For example:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListAllMyBuckets" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "s3:CreateBucket", "s3:DeleteBucket", "s3:DeleteObject", "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket", "s3:PutObject" ], "Resource": ["arn:aws:s3:::BUCKET_TO_UPLOAD_IMAGE","arn:aws:s3:::BUCKET_TO_UPLOAD_IMAGE/*"] }, { "Effect": "Allow", "Action": [ "iam:CreateRole", "iam:PutRolePolicy" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:CancelConversionTask", "ec2:CancelExportTask", "ec2:CreateImage", "ec2:CreateInstanceExportTask", "ec2:CreateTags", "ec2:DeleteTags", "ec2:DescribeConversionTasks", "ec2:DescribeExportTasks", "ec2:DescribeExportImageTasks", "ec2:DescribeInstanceAttribute", "ec2:DescribeInstanceStatus", "ec2:DescribeInstances", "ec2:DescribeTags", "ec2:ExportImage", "ec2:ImportInstance", "ec2:ImportVolume", "ec2:StartInstances", "ec2:StopInstances", "ec2:TerminateInstances", "ec2:ImportImage", "ec2:ImportSnapshot", "ec2:DescribeImportImageTasks", "ec2:DescribeImportSnapshotTasks", "ec2:CancelImportTask" ], "Resource": "*" } ] }
-
Apply the vmimport role to the ManageIQ appliance image you uploaded to the S3 bucket.
$ aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file://role-policy.json
Importing the Appliance to Amazon Elastic Compute Cloud (Amazon EC2)
To import the appliance:
-
Create a containers.json file:
{ "Description": "NAME OF IMPORTED SNAPSHOT IN AWS", "Format": "vhd", "UserBucket": { "S3Bucket": "BUCKET WITH UPLOADED .VHD IMAGE", "S3Key": "PATH OF .VHD IMAGE" } }
See the AWS documentation on VM import and export requirements, such as image formats, instances, volume and file system types, and using regions.
-
Use the AWS-CLI tools to import a disk as a snapshot. See the AWS documentation on using VM Import/Export to import a disk as a snapshot.
Note:
You can either specify a region, or if not, ensure that the S3 bucket is in the same region where you want to import the snapshot.
$ aws ec2 import-snapshot --disk-container file://containers.json
-
Check the progress of your snapshot import by running the following command:
$ aws ec2 describe-import-snapshot-tasks --import-task-ids SNAPSHOT_ID_GOT_FROM_RESPONSE
-
Create an AMI from the snapshot. See the AWS documentation on using options with the following command to create and register a Linux AMI from a snapshot.
$ aws ec2 register-image
Configuring ManageIQ
After installing ManageIQ and running it for the first time, you must perform some basic configuration. To configure ManageIQ, you must at a minimum:
-
Add a disk to the infrastructure hosting your appliance.
-
Configure the database.
Configure the ManageIQ appliance using the internal appliance console.
Accessing the Appliance Console
-
Start the appliance and open a terminal console.
-
Enter the
appliance_console
command. The ManageIQ appliance summary screen displays. -
Press
Enter
to manually configure settings. -
Press the number for the item you want to change, and press
Enter
. The options for your selection are displayed. -
Follow the prompts to make the changes.
-
Press
Enter
to accept a setting where applicable.
Configuring a Database
ManageIQ uses a database to store information about the environment. Before using ManageIQ, configure the database options for it; ManageIQ provides the following two options for database configuration:
-
Install an internal PostgreSQL database to the appliance
-
Configure the appliance to use an external PostgreSQL database
Configuring an Internal Database
-
Start the appliance and open a terminal console.
-
Enter the
appliance_console
command. The ManageIQ appliance summary screen displays. -
Press Enter to manually configure settings.
-
Select Configure Database from the menu.
-
You are prompted to create or fetch an encryption key.
-
If this is the first ManageIQ appliance, choose Create key.
-
If this is not the first ManageIQ appliance, choose Fetch key from remote machine to fetch the key from the first appliance. For worker and multi-region setups, use this option to copy key from another appliance.
All ManageIQ appliances in a multi-region deployment must use the same key.
-
-
Choose Create Internal Database for the database location.
-
Choose a disk for the database. This can be either a disk you attached previously, or a partition on the current disk.
Red Hat recommends using a separate disk for the database.If there is an unpartitioned disk attached to the virtual machine, the dialog will show options similar to the following:
1) /dev/vdb: 20480 2) Don't partition the disk
-
Enter 1 to choose
/dev/vdb
for the database location. This option creates a logical volume using this device and mounts the volume to the appliance in a location appropriate for storing the database. The default location is/var/lib/pgsql
, which can be found in the environment variable$APPLIANCE_PG_MOUNT_POINT
. -
Enter 2 to continue without partitioning the disk. A second prompt will confirm this choice. Selecting this option results in using the root filesystem for the data directory (not advised in most cases).
-
-
Enter Y or N for Should this appliance run as a standalone database server?
-
Select Y to configure the appliance as a database-only appliance. As a result, the appliance is configured as a basic PostgreSQL server, without a user interface.
-
Select N to configure the appliance with the full administrative user interface.
-
-
When prompted, enter a unique number to create a new region.
Creating a new region destroys any existing data on the chosen database. -
Create and confirm a password for the database.
ManageIQ then configures the internal database. This takes a few minutes. After the database is created and initialized, you can log in to ManageIQ.
Configuring an External Database
Based on your setup, you will choose to configure the appliance to use an external PostgreSQL database. For example, we can only have one database in a single region. However, a region can be segmented into multiple zones, such as database zone, user interface zone, and reporting zone, where each zone provides a specific function. The appliances in these zones must be configured to use an external database.
The postgresql.conf
file used with ManageIQ databases requires
specific settings for correct operation. For example, it must correctly
reclaim table space, control session timeouts, and format the PostgreSQL
server log for improved system support. Due to these requirements, Red
Hat recommends that external ManageIQ databases use a
postgresql.conf
file based on the standard file used by the
ManageIQ appliance.
Ensure you configure the settings in the postgresql.conf
to suit your
system. For example, customize the shared_buffers
setting according to
the amount of real storage available in the external system hosting the
PostgreSQL instance. In addition, depending on the aggregate number of
appliances expected to connect to the PostgreSQL instance, it may be
necessary to alter the max_connections
setting.
-
Start the appliance and open a terminal console.
-
Enter the
appliance_console
command. The ManageIQ appliance summary screen displays. -
Press Enter to manually configure settings.
-
Select Configure Database from the menu.
-
You are prompted to create or fetch a security key.
-
If this is the first ManageIQ appliance, choose Create key.
-
If this is not the first ManageIQ appliance, choose Fetch key from remote machine to fetch the key from the first appliance.
All ManageIQ appliances in a multi-region deployment must use the same key.
-
-
Choose Create Region in External Database for the database location.
-
Enter the database hostname or IP address when prompted.
-
Enter the database name or leave blank for the default (
vmdb_production
). -
Enter the database username or leave blank for the default (
root
). -
Enter the chosen database user’s password.
-
Confirm the configuration if prompted.
ManageIQ will then configure the external database.
Configuring a Worker Appliance
You can use multiple appliances to facilitate horizontal scaling, as well as for dividing up work by roles. Accordingly, configure an appliance to handle work for one or many roles, with workers within the appliance carrying out the duties for which they are configured. You can configure a worker appliance through the terminal. The following steps demonstrate how to join a worker appliance to an appliance that already has a region configured with a database.
-
Start the appliance and open a terminal console.
-
Enter the
appliance_console
command. The ManageIQ appliance summary screen displays. -
Press Enter to manually configure settings.
-
Select Configure Database from the menu.
-
You are prompted to create or fetch a security key. Since this is not the first ManageIQ appliance, choose 2) Fetch key from remote machine. For worker and multi-region setups, use this option to copy the security key from another appliance.
All ManageIQ appliances in a multi-region deployment must use the same key. -
Choose Join Region in External Database for the database location.
-
Enter the database hostname or IP address when prompted.
-
Enter the port number or leave blank for the default (
5432
). -
Enter the database name or leave blank for the default (
vmdb_production
). -
Enter the database username or leave blank for the default (
root
). -
Enter the chosen database user’s password.
-
Confirm the configuration if prompted.
Once ManageIQ is installed, you can log in and perform administration tasks.
Log in to ManageIQ for the first time after installing by:
-
Navigate to the URL for the login screen. (https://xx.xx.xx.xx on the virtual machine instance)
-
Enter the default credentials (Username: admin | Password: smartvm) for the initial login.
-
Click Login.
Appendix
Appliance Console Command-Line Interface (CLI)
Currently, the appliance_console_cli
feature is a subset of the full functionality of the appliance_console
itself, and covers functions most likely to be scripted by using the command-line interface (CLI).
-
After starting the ManageIQ appliance, log in with a user name of
root
and the default password ofsmartvm
. This displays the Bash prompt for the root user. -
Enter the
appliance_console_cli
orappliance_console_cli --help
command to see a list of options available with the command, or simply enterappliance_console_cli --option <argument>
directly to use a specific option.
Database Configuration Options
| Option | Description |
| –region (-r) | region number (create a new region in the database - requires database credentials passed) |
| –internal (-i) | internal database (create a database on the current appliance) |
| –dbdisk | database disk device path (for configuring an internal database) |
| –hostname (-h) | database hostname |
| –port | database port (defaults to 5432 ) |
| –username (-U) | database username (defaults to root ) |
| –password (-p) | database password |
| –dbname (-d) | database name (defaults to vmdb_production ) |
v2_key Options
| Option | Description |
| –key (-k) | create a new v2_key |
| –fetch-key (-K) | fetch the v2_key from the given host |
| –force-key (-f) | create or fetch the key even if one exists |
| –sshlogin | ssh username for fetching the v2_key (defaults to root ) |
| –sshpassword | ssh password for fetching the v2_key |
IPA Server Options
| Option | Description |
| –host (-H) | set the appliance hostname to the given name |
| –ipaserver (-e) | IPA server FQDN |
| –ipaprincipal (-n) | IPA server principal (default: admin ) |
| –ipapassword (-w) | IPA server password |
| –ipadomain (-o) | IPA server domain (optional). Will be based on the appliance domain name if not specified. |
| –iparealm (-l) | IPA server realm (optional). Will be based on the domain name of the ipaserver if not specified. |
| –uninstall-ipa (-u) | uninstall IPA client |
Note:
-
In order to configure authentication through an IPA server, in addition to using Configure External Authentication (httpd) in the
appliance_console
, external authentication can be optionally configured via theappliance_console_cli
(command-line interface). -
Specifying –host will update the hostname of the appliance. If this step was already performed via the
appliance_console
and the necessary updates that are made to/etc/hosts
if DNS is not properly configured, the –host option can be omitted.
Certificate Options
| Option | Description |
| –ca (-c) | CA name used for certmonger (default: ipa ) |
| –postgres-client-cert (-g) | install certs for postgres client |
| –postgres-server-cert | install certs for postgres server |
| –http-cert | install certs for http server (to create certs/httpd* values for a unique key) |
| –extauth-opts (-x) | external authentication options |
Note: The certificate options augment the functionality of the certmonger
tool and enable creating a certificate signing request (CSR), and specifying certmonger
the directories to store the keys.
Other Options
| Option | Description |
| –logdisk (-l) | log disk path |
| –tmpdisk | initialize the given device for temp storage (volume mounted at /var/www/miq_tmp ) |
| –verbose (-v) | print more debugging info |
Example Usage.
$ ssh<EMAIL_ADDRESS>To create a new database locally on the server by using /dev/sdb
:
# appliance_console_cli --internal --dbdisk /dev/sdb --region 0 --password smartvm
To copy the v2_key from a host some.example.com to local machine:
# appliance_console_cli --fetch-key some.example.com --sshlogin root --sshpassword smartvm
You could combine the two to join a region where db.example.com is the appliance hosting the database:
# appliance_console_cli --fetch-key db.example.com --sshlogin root --sshpassword smartvm --hostname db.example.com --password mydatabasepassword
To configure external authentication:
# appliance_console_cli --host appliance.test.company.com
--ipaserver ipaserver.test.company.com
--ipadomain test.company.com
--iparealm TEST.COMPANY.COM
--ipaprincipal admin
--ipapassword smartvm1
To uninstall external authentication:
# appliance_console_cli --uninstall-ipa