7 Data exploration
Already enrolled? Watch the full video: Practice Exams + Lessons
Exploration is about making discoveries and you should have a curious mind and strong technical skills to make sense of the data. This chapter will give you the tools to take dirty, unprocessed data, clean it up, discover interesting trends using graphs and summary statistics, and communicate your results to a business audience.
About 10 - 15% of your grade will be based on data exploration. Every version of Exam PA, such as December 2018, June 2019, and December 2019, have all had questions about data exploration. Putting in extra practice in this area is guaranteed to give you a better score because it will free up time that you can use elsewhere.
The syllabus divides this into three learning objectives, which can make the topic intimidating. Data Visualization, Data Types and Exploration, and Data Issues and Resolutions, but you will always be doing these things together, and so this chapter will cover all three at once.
Exploratory Data Analysis (EDA) is usually the first phase of an analytics project. This is as much of an art as it is a science because everyone has their style. If you ask two actuaries to perform EDA on the same data set, they will likely use different methods. These can be broken down into phases.
From the language on the December 2020 Syllabus, these are
Import:
- Evaluate the quality of appropriate data sources for a problem.
- Identify the regulations, standards, and ethics surrounding predictive modeling and data
Tidy:
- Identify structured, unstructured, and semi-structured data.
- Identify the types of variables and terminology used in predictive modeling.
- Understand basic methods of handling missing data.
- Implement effective data design with respect to time frame, sampling, and granularity.
Transform:
- Identify opportunities to create features from the primary data that may add value.
- Identify outliers and other data issues.
- Handle non-linear relationships via transformations.
Visualize:
- Apply univariate and bivariate data exploration techniques.
- Understand the fundamental principles of constructing graphs.
- Create a variety of graphs using the ggplot2 package.
Model:
- Fit and interpret models
Communicate:
- Write the report
7.1 How to make graphs in R?
Let us create a histogram of the claims. The first step is to create a blank canvas that holds the columns that are needed. The library to make this is called ggplot2.
The aesthetic
argument, aes
, means that the variable shown will the the claims.
The sample_frac
means that only 20% of the data is used. This makes the visuzlization less cluttered.
library(ExamPAData)
<- readmission %>% sample_frac(0.05)
df <- df %>%
p ggplot(aes(HCC.Riskscore))
If we look at p
, we see that it is nothing but white space with axis for count
and income
.
p
7.1.1 Add a plot
We add a histogram
+ geom_histogram() p
Different plots are called “geoms” for “geometric objects.” Geometry = Geo (space) + meter (measure), and graphs measure data. For instance, instead of creating a histogram, we can draw a gamma distribution with stat_density
.
+ stat_density() p
Create an xy plot by adding and x
and a y
argument to aesthetic
.
%>%
df ggplot(aes(x = HCC.Riskscore, y = Age)) +
geom_point()
7.2 The different graph types
Only four types of graphs are used for data exploration. You will only need to understand how to interpret them. The SOA will provide you with the code needed to create them. There will not be enough time for you to make your graphs.
7.2.1 Histogram
The histogram is used when you want to look at the probability distribution of a continuous variable.
The template code in your .Rmd file will look like this. Just change “variable” to the name without quotes of the variable and then copy and paste.
7.2.2 Box plot
The boxplot compactly displays the distribution of a continuous variable. It visualizes five summary statistics (the median, two hinges, and two whiskers) and all “outlying” observations individually.
7.2.3 Scatterplot
The point geom is used to create scatterplots. The scatterplot is most useful for displaying the relationship between two continuous variables. It can be used to compare one continuous and one categorical variable or two categorical variables.
7.2.4 Bar charts
There are two types of bar charts: geom_bar() (Univariate) and geom_col() (Bivatiate). geom_bar() makes the height of the bar proportional to the number of cases in each group (or if the weight aesthetic is supplied, the sum of the weights). If you want the heights of the bars to represent values in the data, use geom_col() instead. geom_bar() uses stat_count() by default: it counts the number of cases at each x position.
7.3 How to save time with dplyr?
You may have noticed that writing code for data manipulation can be slow. Fortunately, there is a faster, 100%-legal way of doing data manipulation that has worked for hundreds of exam candidates (the author included) who have taken Exam PA.
Up to this point, we have been using old R libraries. By making use of newer R libraries, we can save ourselves time. These will all be provided for you at Prometric within the tidyverse library.
7.3.1 Data manipulation chaining
Pipes allow for data manipulations to be chained with visualizations. The possibilities are nearly limitless.
library(tidyverse)
%>%
iris select_if(is.numeric) %>%
gather(feature,value) %>%
ggplot(aes(value)) +
geom_histogram() +
facet_wrap(vars(feature))
Suggested reading of R for Data Science (https://r4ds.had.co.nz/index.html):
| Chapter | Topic |
|---|---|
| 9 | Introduction |
| 10 | Tibbles |
| 12 | Tidy data |
| 15 | Factors |
| 17 | Introduction |
| 18 | Pipes |
| 19 | Functions |
| 20 | Vectors |
7.4 How to explore the data?
Let us look at the health insurance data set that contains information on patients along with their annual health care costs.
The descriptions of the columns are below.
age
: Age of the individualsex
: Sexbmi
: Body Mass Indexchildren
: Number of childrensmoker
: Is this person a smoker?region
: Regioncharges
: Annual health care costs.
head()
shows the top n rows. head(20)
shows the top 20 rows.
head(health_insurance)
## # A tibble: 6 x 7
## age sex bmi children smoker region charges
## <dbl> <chr> <dbl> <dbl> <chr> <chr> <dbl>
## 1 19 female 27.9 0 yes southwest 16885.
## 2 18 male 33.8 1 no southeast 1726.
## 3 28 male 33 3 no southeast 4449.
## 4 33 male 22.7 0 no northwest 21984.
## 5 32 male 28.9 0 no northwest 3867.
## 6 31 female 25.7 0 no southeast 3757.
Using a pipe is an alternative way of doing this.
%>% head() health_insurance
Shortcut: Use CTRL
+ SHFT
+ M
to create pipes %>%
The glimpse
function is similar to str()
.
%>% glimpse() health_insurance
## Rows: 1,338
## Columns: 7
## $ age <dbl> 19, 18, 28, 33, 32, 31, 46, 37, 37, 60, 25, 62, 23, 56, 27, 19, 52, 23, 56, 30, 60, 30, 18, 34, 37, 59, 63, 55, 23, 31, 22, 18, 19, 63, 28,…
## $ sex <chr> "female", "male", "male", "male", "male", "female", "female", "female", "male", "female", "male", "female", "male", "female", "male", "male…
## $ bmi <dbl> 27.900, 33.770, 33.000, 22.705, 28.880, 25.740, 33.440, 27.740, 29.830, 25.840, 26.220, 26.290, 34.400, 39.820, 42.130, 24.600, 30.780, 23.…
## $ children <dbl> 0, 1, 3, 0, 0, 0, 1, 3, 2, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 2, 3, 0, 2, 1, 2, 0, 0, 5, 0, 1, 0, 3, 0, 1, 0, 0, 2, 1, 2, 1, 0, 2…
## $ smoker <chr> "yes", "no", "no", "no", "no", "no", "no", "no", "no", "no", "no", "yes", "no", "no", "yes", "no", "no", "no", "no", "yes", "no", "no", "no…
## $ region <chr> "southwest", "southeast", "southeast", "northwest", "northwest", "southeast", "southeast", "northwest", "northeast", "northwest", "northeas…
## $ charges <dbl> 16884.924, 1725.552, 4449.462, 21984.471, 3866.855, 3756.622, 8240.590, 7281.506, 6406.411, 28923.137, 2721.321, 27808.725, 1826.843, 11090…
One of the most useful data science tools is counting things. The function count()
gives the number of observations by a categorical feature.
%>% dplyr::count(children) health_insurance
## # A tibble: 6 x 2
## children n
## <dbl> <int>
## 1 0 574
## 2 1 324
## 3 2 240
## 4 3 157
## 5 4 25
## 6 5 18
Two categories can be counted at once. This creates a table with all combinations of region
and sex
and shows the number ofobservations in each category.
%>% count(region, sex) health_insurance
## # A tibble: 8 x 3
## region sex n
## <chr> <chr> <int>
## 1 northeast female 161
## 2 northeast male 163
## 3 northwest female 164
## 4 northwest male 161
## 5 southeast female 175
## 6 southeast male 189
## 7 southwest female 162
## 8 southwest male 163
The summary()
function is shows a statistical summary. One caveat is that each column needs to be in its appropriate type. For example, smoker
, region
, and sex
are all listed as characters when if they were factors, summary
would give you count info.
With incorrect data types
%>% summary() health_insurance
## age sex bmi children smoker region charges
## Min. :18.00 Length:1338 Min. :15.96 Min. :0.000 Length:1338 Length:1338 Min. : 1122
## 1st Qu.:27.00 Class :character 1st Qu.:26.30 1st Qu.:0.000 Class :character Class :character 1st Qu.: 4740
## Median :39.00 Mode :character Median :30.40 Median :1.000 Mode :character Mode :character Median : 9382
## Mean :39.21 Mean :30.66 Mean :1.095 Mean :13270
## 3rd Qu.:51.00 3rd Qu.:34.69 3rd Qu.:2.000 3rd Qu.:16640
## Max. :64.00 Max. :53.13 Max. :5.000 Max. :63770
With correct data types
This tells you that there are 324 patients in the northeast, 325 in the northwest, 364 in the southeast, etc.
<- health_insurance %>%
health_insurance mutate_if(is.character, as.factor)
%>%
health_insurance summary()
## age sex bmi children smoker region charges
## Min. :18.00 female:662 Min. :15.96 Min. :0.000 no :1064 northeast:324 Min. : 1122
## 1st Qu.:27.00 male :676 1st Qu.:26.30 1st Qu.:0.000 yes: 274 northwest:325 1st Qu.: 4740
## Median :39.00 Median :30.40 Median :1.000 southeast:364 Median : 9382
## Mean :39.21 Mean :30.66 Mean :1.095 southwest:325 Mean :13270
## 3rd Qu.:51.00 3rd Qu.:34.69 3rd Qu.:2.000 3rd Qu.:16640
## Max. :64.00 Max. :53.13 Max. :5.000 Max. :63770
Shortcut: This “mutate_if” trick can save you time from having to copy and paste from the template code of SOA and convert each variable to a factor individually
7.5 How to transform the data?
Transforming, manipulating, querying, and wrangling are synonyms in data terminology.
R syntax is designed to be similar to SQL. They begin with a SELECT
, use GROUP BY
to aggregate, and have a WHERE
to remove observations. Unlike SQL, the ordering of these does not matter. SELECT
can come after a WHERE
.
R to SQL translation
select() -> SELECT
mutate() -> user-defined columns
summarize() -> aggregated columns
left_join() -> LEFT JOIN
filter() -> WHERE
group_by() -> GROUP BY
filter() -> HAVING
arrange() -> ORDER BY
%>%
health_insurance select(age, region) %>%
head()
## # A tibble: 6 x 2
## age region
## <dbl> <fct>
## 1 19 southwest
## 2 18 southeast
## 3 28 southeast
## 4 33 northwest
## 5 32 northwest
## 6 31 southeast
Let us look at only those in the southeast region. Instead of WHERE
, use filter
.
%>%
health_insurance filter(region == "southeast") %>%
select(age, region) %>%
head()
## # A tibble: 6 x 2
## age region
## <dbl> <fct>
## 1 18 southeast
## 2 28 southeast
## 3 31 southeast
## 4 46 southeast
## 5 62 southeast
## 6 56 southeast
The SQL translation is
SELECT age, region
FROM health_insurance
WHERE region = 'southeast'
Instead of ORDER BY
, use arrange
. Unlike SQL, the order does not matter and ORDER BY
doesn’t need to be last.
%>%
health_insurance arrange(age) %>%
select(age, region) %>%
head()
## # A tibble: 6 x 2
## age region
## <dbl> <fct>
## 1 18 southeast
## 2 18 southeast
## 3 18 northeast
## 4 18 northeast
## 5 18 northeast
## 6 18 southeast
The group_by
comes before the aggregation, unlike in SQL where the GROUP BY
comes last.
%>%
health_insurance group_by(region) %>%
summarise(avg_age = mean(age))
## # A tibble: 4 x 2
## region avg_age
## <fct> <dbl>
## 1 northeast 39.3
## 2 northwest 39.2
## 3 southeast 38.9
## 4 southwest 39.5
In SQL, this would be
SELECT region,
AVG(age) as avg_age
FROM health_insurance
GROUP BY region
Just like in SQL, many different aggregate functions can be used such as SUM
, MEAN
, MIN
, MAX
, and so forth.
%>%
health_insurance group_by(region) %>%
summarise(avg_age = mean(age),
max_age = max(age),
median_charges = median(charges),
bmi_std_dev = sd(bmi))
## # A tibble: 4 x 5
## region avg_age max_age median_charges bmi_std_dev
## <fct> <dbl> <dbl> <dbl> <dbl>
## 1 northeast 39.3 64 10058. 5.94
## 2 northwest 39.2 64 8966. 5.14
## 3 southeast 38.9 64 9294. 6.48
## 4 southwest 39.5 64 8799. 5.69
To create new columns, the mutate
function is used. For example, if we wanted a column of the annual charges of a person divided by their age
%>%
health_insurance mutate(charges_over_age = charges/age) %>%
select(age, charges, charges_over_age) %>%
head(5)
## # A tibble: 5 x 3
## age charges charges_over_age
## <dbl> <dbl> <dbl>
## 1 19 16885. 889.
## 2 18 1726. 95.9
## 3 28 4449. 159.
## 4 33 21984. 666.
## 5 32 3867. 121.
 For example, we can create a column which is 0
when age < 50
, 1
when 50 <= age <= 70
, and 2
when age > 70
.
%>%
health_insurance mutate(age_bucket = case_when(age < 50 ~ 0,
<= 70 ~ 1,
age > 70 ~ 2)) %>%
age select(age, age_bucket)
## # A tibble: 1,338 x 2
## age age_bucket
## <dbl> <dbl>
## 1 19 0
## 2 18 0
## 3 28 0
## 4 33 0
## 5 32 0
## 6 31 0
## 7 46 0
## 8 37 0
## 9 37 0
## 10 60 1
## # … with 1,328 more rows
SQL translation:
SELECT CASE WHEN AGE < 50 THEN 0
ELSE WHEN AGE <= 70 THEN 1
ELSE 2
FROM health_insurance
7.6 Missing values
The most recent PA exams have had questions related to missing values. You need to perform checks before moving on to the model-building phases. In real life, man-made data always have missing observations.
In R, there are two ways that data can be missing. The first is the NA
value. If you read in a csv file that has blank values, R will translate them into NAs.
<- tibble(x = c(1,2,NA), y = c(NA, NA, 2)) df
You can check for NA
values with summary
or sum(is.na(df))
.
summary(df)
## x y
## Min. :1.00 Min. :2
## 1st Qu.:1.25 1st Qu.:2
## Median :1.50 Median :2
## Mean :1.50 Mean :2
## 3rd Qu.:1.75 3rd Qu.:2
## Max. :2.00 Max. :2
## NA's :1 NA's :2
sum(is.na(df))
## [1] 3
The second type is for “Not A Number” (NaN
) or infinity Inf
values. Dividing by zero produces Inf
and taking the square root of a negative number produces NaN
. Notice that NaN
counts as NA
but Inf
does not. R says that there are only two missing values in this table even though there are two values which are infinity.
<- tibble(x = c(1/0, 4/0, 4, 1), y = c(sqrt(-1), NA, 2, 4)) df
## Warning in sqrt(-1): NaNs produced
sum(is.na(df))
## [1] 2
summary(df)
## x y
## Min. :1.00 Min. :2.0
## 1st Qu.:3.25 1st Qu.:2.5
## Median : Inf Median :3.0
## Mean : Inf Mean :3.0
## 3rd Qu.: Inf 3rd Qu.:3.5
## Max. : Inf Max. :4.0
## NA's :2
7.6.1 Types of Missing Values
Read the Data Dictionary from the Project Statement and check for these three types of missingness.
- Missing at random (MAR): There is no pattern between the missingness and the value of missing variable.
- Missing not at random (MNAR): The value of the missing variable is related to the reason it is missing. Example: A survey concerning illegal drug use where respondents are reluctant to admit that they have broken the law.
- Hidden missingness: When a value is coded incorrectly, such as when a numeric variable with “999” or “0” or a factor does not match the Data Dictionary.
7.6.2 Missing Value Resolutions:
You can use any of these methods but make sure that you describe your reasoning in the report.
- Remove observations: Use when there are only a few missing values or when they are MAR.
- Create new level “missing:” Use when there are more values or when they are MNAR.
- Impute with the mean: Use when there are numeric values MAR.
- Remove variable altogether: Use when most observations are missing (≥ 50% of observations), and there is no way to repair the data.
While you are reviewing the data for missingness, you may also decide to remove some variables altogether. This could be because of racial or ethical concerns, limitations of future availability, instability of the data over time, or inexplicability.
7.7 Example: SOA PA 12/12/19, Task 1
This chapter is based on the first task of the December 2019 Exam PA.
Already enrolled? Watch the full video: Practice Exams | Practice Exams + Lessons
TASK 1 (12 points)
Examine each variable and make appropriate adjustments.
Examine each predictor variable other than cap_gain both on its own and for value_flag. Make appropriate adjustments. Do not make any adjustments to the cap_gain variable at this time.
There should be no further variable adjustments unless specifically requested.
As a reminder, all data for this book can be accessed from the package ExamPAData
. In the actual exam, you will read the file from the Prometric computer.
Because the data is already loaded, use the below code to access the data.
library(ExamPAData)
To save keystrokes, give the data a short name such as df
for “data frame.”
<- customer_value %>% mutate_if(is.character, as.factor) df
7.7.1 Garbage in; garbage out 🗑
This is a common saying when working with predictive models. No matter how complex the model, the actual result will be wrong if the data is terrible. For this exam, master the art of data manipulation, and everything becomes easier!
Begin by looking at a summary.
summary(df)
## age education_num marital_status occupation cap_gain hours_per_week score value_flag
## Min. :17.00 Min. : 1.00 Divorced : 6633 Group 1 : 7237 Min. : 0 Min. : 1.00 Min. :43.94 High:11687
## 1st Qu.:28.00 1st Qu.: 9.00 Married-AF-spouse : 37 Group 2 :10123 1st Qu.: 0 1st Qu.:40.00 1st Qu.:57.50 Low :37155
## Median :37.00 Median :10.00 Married-civ-spouse :22379 Group 3 :13971 Median : 0 Median :40.00 Median :60.24
## Mean :38.64 Mean :10.08 Married-spouse-absent: 628 Group 4 : 2444 Mean : 1079 Mean :40.42 Mean :60.23
## 3rd Qu.:48.00 3rd Qu.:12.00 Never-married :16117 Group 5 :12258 3rd Qu.: 0 3rd Qu.:45.00 3rd Qu.:62.95
## Max. :90.00 Max. :16.00 Separated : 1530 Group NA: 2809 Max. :99999 Max. :99.00 Max. :76.53
## Widowed : 1518
str(df)
## spec_tbl_df [48,842 × 8] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
## $ age : num [1:48842] 39 50 38 53 28 37 49 52 31 42 ...
## $ education_num : num [1:48842] 13 13 9 7 13 14 5 9 14 13 ...
## $ marital_status: Factor w/ 7 levels "Divorced","Married-AF-spouse",..: 5 3 1 3 3 3 4 3 5 3 ...
## $ occupation : Factor w/ 6 levels "Group 1","Group 2",..: 2 5 1 1 5 5 1 5 5 5 ...
## $ cap_gain : num [1:48842] 2174 0 0 0 0 ...
## $ hours_per_week: num [1:48842] 40 13 40 40 40 40 16 45 50 40 ...
## $ score : num [1:48842] 59 55.8 62.8 60.1 53.3 ...
## $ value_flag : Factor w/ 2 levels "High","Low": 2 2 2 2 2 2 2 1 1 1 ...
## - attr(*, "spec")=
## .. cols(
## .. age = col_double(),
## .. education_num = col_double(),
## .. marital_status = col_character(),
## .. occupation = col_character(),
## .. cap_gain = col_double(),
## .. hours_per_week = col_double(),
## .. score = col_double(),
## .. value_flag = col_character()
## .. )
7.7.2 Be a detective 🔍
Sherlock Holmes is famous for saying, “You see, Watson, but you do not observe!”
Just like detectives, actuaries need to collect data and make observations. Each exam has a few “anomalies” in the data, which they expect candidates to mention. These could be
- Any value that does not match the Data Dictionary in the Project Statement
- Variables that have
NA
values - Variables that have many factor levels
- Incorrect data types
- Factors that are read as characters or vice versa
- Numeric variables that are factor/character
- Extreme values (Numeric values are too high or low)
You will get very good at spotting these with practice. Just from looking at the above summary
, we can observe the following:
Observations
- The data consists of 48,842 obs. of 8 variables.
- The lowest
age
is 17 but the Project Statement says to only include observations withage
>= 25. - The
cap_gain
distribution is right-skewed because the median (0) is less than the mean (1079), but the Project Statement said not to transform this. Otherwise, I would apply a log transform. education_num
takes integer values between 1 and 16. There are many values that are low.- There are missing values when
occupation
isgroup NA
, which means that the occupation of the person is unknown. - The amount that people work per work,
hours_per_week
, varies by a lot. The lowest is 1 hour, and the highest is 99. Most people work 40 hours per week.
Study Tip: Never apply a log transform to the target variable. Only the predictor variables get logs taken. At this stage, we note if it is right-skewed or not.
The solution of SOA recommends leaving comments in your .Rmd file. This helps to give you partial credit on questions that you may answer incorrectly.
Good comments
#I observe that there are no missing values other than those indicated by Group NA for occupation.
#I removed the code provided by my assistant. It is embedded in later chunks as needed.
#I excluded people under the age of 25
<- df[df$age >= 25,]
df
#I convert the target to 0-1.
$value_flag <- ifelse(df$value_flag == "High",1,0) df
Useless comments
#this is a comment
#this loads a library called ggplot2
library(ggplot2)
7.7.3 A picture is worth a thousand words 📷
What is your favorite type of graph? Mine is a radar chart. This is a graphical method of displaying multivariate data in the form of a two-dimensional chart of three or more quantitative variables represented on axes starting from the same point.
That is pretty, right?
Do not waste time trying to make the graphs perfect!
Only change code that you need to change. If you need assistance to do it faster then SOA can help you save time by giving you templates.
This is the code template that they give you. You only need to change the “variable” names.
# This code makes a histogram for a continuous variable.
ggplot(df, aes(x = variable)) +
geom_histogram(bins = 30) +
labs(x = "variable") +
theme(axis.text.x = element_text(angle = 90, hjust = 1))
# This code makes a bar chart for a factor variable.
ggplot(df, aes(x = variable)) +
geom_bar() +
labs(x = "variable") +
theme(axis.text.x = element_text(angle = 90, hjust = 1))
Okay, that was not 100% true. It is common for the code to require small changes. changes. For instance, the histogram that they give you has bins that are too narrow for age
, which causes these spikes to show up in the graph.
The best candidates altered the code to create more appropriate plots and summary tables.
# This code makes a histogram for a continuous variable.
ggplot(df, aes(x = age)) +
geom_histogram(bins = 30) + #not the right number of bins
labs(x = "age") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
ggtitle("Bad: Default histogram with spikes")
#Histograms of continuous variables
ggplot(df, aes(x = age)) +
geom_histogram(breaks = seq(24.5,99.5, by = 5)) + #make bins wider and set range
labs(x = "Age") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
ggtitle("Good: After adjusting the breaks")
How do you know when a variable should a factor and when it should be numeric?
7.7.4 Factor or numeric ❓
Which variables should be converted to factors and which should be numeric?
Questions of this sort have come up twice. On Hospital Readmissions, a Length of Stay variable was numeric but had only a few values, so some candidates treated it as a factor. The education_num
variable here is also numeric but only has 16 unique values. So should this be a numeric or a factor?
table(df$education_num)
##
## 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
## 68 231 446 877<PHONE_NUMBER> 382 13320 7812 1862 1411 7298 2621 834 593
Ask yourself this question: is there a way of comparing two values of the variable together?
- If yes, then use numeric
- If no, then use a factor
For exmaple, we can say that education_num = 2
is less than education_num = 4
, which means there is a natural order. This is also known as an ordinal.
If the factor is say, color
, which can be red
, blue
, or green
, then there is no way of comparing values together. Is red
greater than blue
? This question has no meaning.
ggplot(df, aes(x = education_num)) +
geom_histogram(bins = 30) +
labs(x = "Education") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
ggtitle("Default number of bins (30)")
ggplot(df, aes(x = education_num)) +
geom_histogram(bins = 16) +
labs(x = "Education") +
theme(axis.text.x = element_text(angle = 90, hjust = 1)) +
ggtitle("Set number of bins to number of factor levels (16) ")
# Set bins equal to number of levels, could have made bar chart.
We could also use a bar plot.
ggplot(df, aes(x = education_num)) +
geom_bar(stat = "count") +
ggtitle("Bar plot")
Lastly, read the Project Statement carefully and only do what it tells you to do.
Candidates were not required to make a plot for cap_gain. This solution has the plot made here rather than in Task 6.
7.7.5 73.6% of statistics are false 😲
Really? No, but statistics can help you see patterns that data visualization by itself can miss. Along with the ggplot codes, there will be code to look at summary statistics. Here is a refresher on what these statistics mean (no pun intended).
- Mean: The average. This gets skewed by outliers easily. If the mean is greater than the median, then the distribution is right-skewed.
- Median: The “middle” value. This is an average that reduces the impact of outliers.
- Variance: The amount by which each observation differs from the mean.
- Standard Deviation: The square root of the variance.
- n(): The number of observations. Always take note of groups that do not have many observations.
# This code provides, for each level of a factor variable, the number for which value_flag is zero, the number for which it is one, the total number of observations, and the proportion of ones.
# Note that the variable name should not be enclosed in quotation marks.
%>%
df group_by(variable) %>%
::summarise(
dplyrzeros = sum(value_flag == 0),
ones = sum(value_flag == 1),
n = n(),
proportion = mean(value_flag)
)
Factors levels should be simplified. If a group has only a few observations, then there will be problems with the model. In our data, take a look at the marital_status
column. Do you observe anything unusual?
table(df$marital_status)
##
## Divorced Married-AF-spouse Married-civ-spouse Married-spouse-absent Never-married Separated Widowed
## 6498 31 21661 573 8697 1438 1512
Only 31 observations have Married-AF-spouse
. This is because the sample size n = 31
is too small. In modeling jargon, this is “statistical insignificant” and will cause the p-value on marital_status
to be large. You can fix this in a few different ways
- Delete these observations (Not recommended)
- Group these observations together with
Married-spouse
(Simplest method)
Let us use the second method.
First, look at the levels of the factor variable.
levels(df$marital_status)
## [1] "Divorced" "Married-AF-spouse" "Married-civ-spouse" "Married-spouse-absent" "Never-married" "Separated"
## [7] "Widowed"
Now look at the profitability across marital status. For Married-AF-spouse
and Married-civ-spouse
the proportion of high profit customers is high, but for Married-spouse-absent
it is low. Even though these are all “married,” it would be a bad idea to combine them because the profitability is so different.
#Proportion of ones by category of factor variable
%>%
df group_by(marital_status) %>%
::summarise(
dplyrzeros = sum(value_flag == 0),
ones = sum(value_flag == 1),
n = n(),
proportion = mean(value_flag)
)
## # A tibble: 7 x 5
## marital_status zeros ones n proportion
## <fct> <int> <int> <int> <dbl>
## 1 Divorced 5829 669 6498 0.103
## 2 Married-AF-spouse 19 12 31 0.387
## 3 Married-civ-spouse 11727 9934 21661 0.459
## 4 Married-spouse-absent 515 58 573 0.101
## 5 Never-married 8000 697 8697 0.0801
## 6 Separated 1342 96 1438 0.0668
## 7 Widowed 1384 128 1512 0.0847
Then create a vector that has the simpler levels that you want. The order needs to be the same.
<- c("Divorced", "Married-spouse", "Married-spouse", "Married-spuouse-absent", "Neber-married", "Separated", "Widowed") simple_levels
The function mapvalues
takes in three arguments. You can read about this by typing ?mapvalues
into the console.
x
: the factor or vector to modify
from
: a vector of the items to replace
to
: a vector of replacement values
Then map the old values to the simpler values.
#Combine the two marital status levels
<- levels(df$marital_status)
var.levels $marital_status <- mapvalues(x = df$marital_status,
dffrom = var.levels,
to = simple_levels)
Now, when you look at the marital_status
levels, you will see the simpler levels.
levels(df$marital_status)
## [1] "Divorced" "Married-spouse" "Married-spuouse-absent" "Neber-married" "Separated" "Widowed"
You can also check that the number of observations is what you expect.
table(df$marital_status)
##
## Divorced Married-spouse Married-spuouse-absent Neber-married Separated Widowed
## 6498 21692 573 8697 1438 1512
7.8 Exercises
7.8.2 Dplyr Practice
Run this code on your computer to answer these exercises.
The data actuary_salaries
contains the salaries of actuaries collected from the DW Simpson survey. Use this data to answer the exercises below.
%>% glimpse() actuary_salaries
## Rows: 138
## Columns: 6
## $ industry <chr> "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Casualty", "Cas…
## $ exams <chr> "1 Exam", "2 Exams", "3 Exams", "4 Exams", "1 Exam", "2 Exams", "3 Exams", "4 Exams", "5 Exams", "6 Exams", "7 Exams", "ACAS", "1 Exam",…
## $ experience <dbl> 1, 1, 1, 1, 3, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5, 5, 5, 5, 5, 7, 7, 7, 7, 7, 7, 7, 7, 10, 10, 10, 10, 10, 10, 10, 10, 15, 15, 15, 15, 1…
## $ salary <chr> "48 - 65", "50 - 71", "54 - 77", "58 - 82", "54 - 72", "57 - 81", "62 - 87", "63 - 91", "65 - 95", "70 - 97", "72 - 99", "85 - 112", "55…
## $ salary_low <dbl> 48, 50, 54, 58, 54, 57, 62, 63, 65, 70, 72, 85, 55, 58, 66, 71, 73, 76, 85, 90, 11, 60, 79, 82, 84, 88, 94, 98, 116, 60, 79, 82, 84, 88,…
## $ salary_high <chr> "65", "71", "77", "82", "72", "81", "87", "91", "95", "97", "99", "112", "80", "83", "100", "07", "11", "16", "18", "38", "66", "92", "0…
- How many industries are represented?
- The
salary_high
column is a character type when it should be numeric. Change this column to numeric. - What are the highest and lowest salaries for an actuary in Health with 5 exams passed?
- Create a new column called
salary_mid
which has the middle of thesalary_low
andsalary_high
columns. - When grouping by industry, what is the highest
salary_mid
? What aboutsalary_high
? What is the lowestsalary_low
? - There is a mistake when
salary_low == 11
. Find and fix this mistake, and then rerun the code from the previous task. - Create a new column, called
n_exams
, which is an integer. Use 7 for ASA/ACAS and 10 for FSA/FCAS. Use the code below as a starting point and fill in the_
spaces - Create a column called
social_life
, which is equal ton_exams
/experience
. What is the average (mean)social_life
by industry? Bonus question: what is wrong with using this as a statistical measure?
<- actuary_salaries %>%
actuary_salaries mutate(n_exams = case_when(exams == "FSA" ~ _,
== "ASA" ~ _,
exams == "FCAS" ~ _,
exams == "ACAS" ~ _,
exams TRUE ~ as.numeric(substr(exams,_,_))))
- Create a column called
social_life
, which is equal ton_exams
/experience
. What is the average (mean)social_life
by industry? Bonus question: what is wrong with using this as a statistical measure?
7.9 Answers to exercises
Answers to these exercises, along with a video tutorial, are available at ExamPA.net.


You are participating in an open review of this book. During this open review you can read the book and help make it better. You can offer suggestions by make annotations using hypothes.is, an open source annotation system. Further, the website will be collecting implicit feedback, such as the traffic and abandonment rate of each section of the book.
Open review will take place while this book's publisher is conducting peer review. The feedback from open review and peer review will be used to create a revised manuscript. The open review period will end when the final manuscript is submitted to the publisher.
You get to read the manuscript and help make it better.
The open review process will benefit both authors and publishers, even if they have no interest in increasing access to knowledge. The process will lead to higher manuscript quality through the explicit and implicit feedback. Further, the open review process will provide valuable data that can be used during that marketing of the book.
We will do a full blog post about that in the future. For now, you can read more about our code.
I'm sure that they have. Here are some somewhat similar projects that I've heard about:
Please read our privacy and consent policy.
The AAUP recently published a report on best practices for peer review.
Sure. Check out the code section of this website for more about how we did it.


We live in a world where constant change presents both opportunities and challenges for branding. Google and Facebook, two companies that have redefined our culture and the business world, did not exist before 1996 and 2004, respectively. However, today the Internet browser has become the platform for communications, sharing and services. I know of many professionals and entrepreneurs who were slow to rebrand for the opportunities afforded by this shift in technology and business model, and they suffered the consequences.
The world is dynamic. Your brand needs to change with the times. According to the Bureau of Labor Statistics, an average worker may have multiple careers and hold more than ten different jobs during his or her lifetime. Here are six key changes that may trigger a need to refresh your brand:
•New job search or career change<
•Change in company ownership, leadership, your assignment or your boss
•Change in cultural values (customer-centric, one team, innovation, etc.)
•Shift in desired leadership traits (cross-functional, global, collaborative, flexible, etc.)
Strategically managing your personal brand will help you to be proactive and in charge of your career. The BrandingPays™ System has concepts and tools to help you refresh your brand throughout your lifetime.


Inês Almeida
Profile
When I’m not working, I write Python scripts and design websites for fun. When I’m working, I try to do the same. I see myself as a versatile hard-worker that is passionate about learning new skills and to be useful in the work environment.
Experience
Perspectum Diagnostics
Oxford, UK
Software Engineer
Jan 2020 - Present
- Implemented peer-reviewed features for a Django web portal
- Created, documented and released Python packages to be used by other teams
- Communicated with stakeholders, gathered requirements and divided work into digesteable tasks to get new features out
- Built CI pipelines in Jenkins and Bitbucket Pipelines
- Implemented REST APIs
- Wrote easily testable code and thorough unit tests
- Python
- Django
- OOP
- Javascript
- HTML
- CSS
- Agile
Software Tester
Mar 2019 - Jan 2020
- Designed multiple BDD automated tests for a web portal
- Wrote code to run hundreads of load tests in the web portal concurrently
- Improved the already existing test framework to run tests faster
- Refactored and restructured test functions/methods in Python using OOP principles
- Python
- BDD
- Selenium
- OOP
- Agile
Instituto Gulbenkian de Ciência
Lisbon, PT
Flow Cytometry Resource Specialist
Jan 2018 - Mar 2019
- Automatized laborous report creation and data conversion tasks
- Unified data from thousands of excel spreadsheets for statistical analysis
- Rehabilitated 2 of the institute’s old websites
- Built a game in Unity/C# to introduce the general public to flow cytometry
- Python
- HTML
- CSS
- Javascript
- C#/Unity
Institute of Neural Engineering
Graz, AT
Master's Thesis Research Intern
Oct 2016 - Jun 2017
- Designed and developed an experimental paradigm, acquired and analysed data
- Published: Reinmar J. Kobler & Inês Almeida, Andreea I. Sburlea, Gernot R. Müller-Putz (2020). Using machine learning to reveal the population vector from EEG signals
- Python
- MATLAB
- Data Processing
- Paradigm Design
Education
BSc & MSc in Biomedical Engineering and Biophysics
University of Lisbon
Sep 2012 - Nov 2017
MSc grade 17/20 (ECTS equiv. A)
Master’s thesis 19/20 (ECTS equiv. A)
Master’s thesis 19/20 (ECTS equiv. A)
- +44<PHONE_NUMBER>
-<EMAIL_ADDRESS>- ines-almeida.com
- in/ines-de-almeida
- Portugal
Skills
-
-
Programming
- Python
- HTML (including Jinja)
- Javascript
- CSS
- SASS
- SQL
- C#
- Bash
- MATLAB
-
Frameworks
- Django
- Flask
- Hugo
- React
-
Developer Tools
- Git
- Docker
- Jenkins
- AWS
-
Misc
- Adobe Tools
- LaTeX
Languages
-
PortugueseNative
-
EnglishC2
-
SpanishB1
-
GermanB1
Diplomas
- ISTQB Tester Certification (2019)
- Museum Instructor (2016)
- English Cambridge CAE (2009)
Interests
- Bouldering
- Cooking
- Windsurfing
- Bossa Nova
- Design
- Languages


Note: If this is your first time playing, I highly recommend getting aquainted with the ship first. You should know how to navigate from the spawn to the engine room without much difficulty before attempting the full Easter Egg.
When you spawn, you will find yourself on the front end (bow/Forecastle) of the ship. Your first goal will be to unlock doors until you reach the other end (stern/Poop Deck). Once there, you will see a glowing object suspended above a catwalk. Climb onto the catwalk and interact with the Artifact to open additional areas of the ship and begin the Easter Egg.
Four terminals will appear across the ship.
The first of these terminals will be located near where you grabbed the artifact. Interact with the terminal to unlock it. Once you have unlocked all four terminals, the Pack-a-Punch will appear in place of the last activated terminal. This location is not static, and will move from round to round to one of the four terminal locations.You can determine the current location of the Pack-a-Punch by walking up to a terminal and reading the text that appears on screen. Each of the locations also correlates with an elemental symbol (see Clocks). Once learned, they can be used to determine the location from a distance. The active terminal's element will be lit up.
Once you have an elemental Kraken, you may unlock the Pack-a-Punch in all locations. If you do not yet have one, you can follow the Kraken guide and Distillery guide.
In the engine room, use the Kraken to shoot the four moving pistons in the center of the room. They will begin smoking and stop moving.
Above the pistons, there will be several red wheels with matching lights further above them (3 per piston, 12 total). Looking up at the lights, turn the wheels that have a flashing light. This light can be red or white. If a wheel does not turn right away, holding the interact key (F on PC) will work. You must turn each wheel so the handle-bar is facing away from the light. The bars below the non-flashing lights must be turned toward the light.
There should be six flashing lights. Once all six wheels have been turned, step into the Turbine Room and look to your right. Against the wall there will be a lever. Pull it to complete the upgrade.
The new Pack-a-Punch will spawn in all locations, cost 5000 points to upgrade, but will also refill the ammo of your weapon. This effect will last one round, after which the Pack-a-Punch will return to its normal rotation.
When you reach the engine room and cargo hold, they will be flooded. You can drain these areas easily by using a well-lit wall valve located somewhere under water.
Once the Pipes Easter Egg is completed, these sections will re-flood, but can be drained again using the same valves.
The outlets will spawn in 4 locations, chosen randomly from a pool of 6 possible locations. The outlet will always be present in the location, but will not always be "active." You will know an outlet is active because it will occasionally spew a certain element from the list above. See the Help section for more information.
For this next section, you will need a Kraken upgraded with the Acid element. See the Distillery section for more info.
Once you have the restored Artifact, you will need to go to the Turbine Room. There will be 9 blue pipes with one or two leaks in them. Initially, these leaks will be spewing steam only. But, once you hit them with an Acid Kraken, they will also begin leaking small droplets of water. After all nine have been hit, the room will re-flood.
Next, you will need to wait until the Pack-a-Punch is moved back into the turbine room. You will then be able to Pack-a-Punch the Artifact, completing this step of the Easter Egg.
Click and drag a planet between the two lists to keep track of their order. The planets are initially sorted in alphabetical order for convenience. The sun always comes last. Use the reset button above to reset the order.


This is the Owin Framework itself. The Owin Framework consists of:
- A set of interfaces that define how middleware from different authors can work togther.
- A set of configurable mocks for the interfaces to make it easy to write unit tests.
- A pipeline builder that resolves dependencies between middleware and builds a middleware pipeline.
Further reading
Overview of Owin Framework concepts
The Owin Framework repository
The Owin Framework project


Welcome to 2d-imaging!¶
2d-imaging allows training and testing classification models for 2d images.
The source code for 2d-imaging can be found here.
Getting Started¶
- Activate environment: source activate chxr
- Train a model: run python train.py -h for information about available options.
- Open TensorBoard:
- While training, launch TensorBoard: tensorboard –logdir=logs –port=5678
- Port forward: ssh -N -f -L localhost:1234:localhost:5678 <SUNET>@bootcamp
- View in browser: http://localhost:1234/


Access control systems should have a "deny-all" default setting.
Access control systems should have a "deny-all" default to ensure that all access is explicitly authorized.
Use the following steps to verify that all the access control system implementations have a "deny-all" default:
Verify the ACL defaults on UNIX-type systems. Issue the "umask" command for each authorized user or check the login scripts for each user. The last two digits of the number returned by the "umask" command or supplied as an argument to the umask command in the login scripts should be 77.
Verify the ACL defaults on Windows systems. On Windows, access is denied by default unless explicitly granted. Windows does grant access privileges generously because objects inherit permissions from their containing folder or drive. It is prudent to make sure that the inherited permissions do not exceed the defined privileges. To verify the access control settings on Windows: right click on each drive, select "Properties", go to the "Security" tab, and verify that only the needed access privileges are granted.
Verify the ACL defaults on other system-types. Most operating systems implement either the UNIX or the Windows type ACL system. Some network devices may have to be configured using a web interface. Use intuition or follow vendor's instructions to check the ACL defaults on such devices.
Perform the following tasks to enable the "deny-all" default setting on access control systems:
Enable the "deny-all" ACL default setting on UNIX-type systems. add the "umask 077" command to the login scripts for all users.
Enable the "deny-all" ACL default setting on Windows systems. Delete any excessive permissions by using the controls in the “Security” tab: right click on each drive, select "Properties", go to the "Security" tab, and delete all access permissions that are not explitictly authorized.
Enable the "deny-all" ACL default settings on other system-types. Most operating systems implement either the UNIX or the Windows type ACL system. Some network devices may have to be configured using a web interface. Follow vendor's instructions to configure ACL defaults on such devices.


EBD - Easton Bible DIctionary
Naphish
NAPHISH
"NAPHISH "refresher, one of the sons of Ishmael (
Genesis 25:15
;
1 Chronicles 1:31
). He was the father of an Arab tribe.
"NAPHISH "refresher, one of the sons of Ishmael (Genesis 25:15; 1 Chronicles 1:31). He was the father of an Arab tribe.


Forensic scientist
Forensic scientists prepare traces of physical evidence for use in courts of law.
1. How to become a forensic science
To get into forensic science, you’ll usually need either:
- a degree or postgraduate award in forensic science
- a science degree like chemistry, biology, physics, life sciences, applied sciences or medical sciences
- a higher national certificate or diploma (HNC/D) in a science with relevant work experience
For a degree you’ll need at least 5 GCSEs at grades 9 to 4 (A* to C) and 2 A levels or equivalent level 3 qualifications.
There’s a lot of competition for posts. A postgraduate qualification or relevant laboratory experience could help you to get in.
UCAS has information on higher education courses and their entry requirements.
If you want to specialise in electronic casework (recovering data from computers and mobile phones for example), you may need experience and qualifications in computing, electrical engineering, electronics or physics.
To start as a forensics lab support assistant, you’ll need A levels, a BTEC or an HND in science. You could then study for higher qualifications to become a forensic scientist.
You’ll also need colour-normal vision.
You may be able to get financial help to go to university or college.
The Chartered Society of Forensic Sciences (CSFS) has more information about careers in forensic science Not all forensic science degrees teach you the skills and knowledge you need so it’s important to check for accreditation.
2. Skills required
You’ll need:
- a logical and analytical approach
- patience and concentration
- highly developed observation and scientific skills
- a high degree of accuracy and attention to detail
- strong written and spoken communication skills
3. What you'll do
Forensic scientists use a range of techniques to find and study evidence from a variety of sources, like blood and other body fluids, hairs, textile fibres, glass fragments and tyre marks.
Your main role will be looking for evidence to link a suspect with a crime scene, but you may also specialise.
Your day-to-day tasks may include:
- blood grouping and DNA profiling
- analysing fluid and tissue samples for traces of drugs and poisons
- examining splash patterns and the distribution of particles
- analysing handwriting, signatures, ink and paper
- providing expert advice on explosives, firearms and ballistics
- researching and developing new technologies
- recovering data from computers, mobile phones and other electronic equipment
- attending crime scenes, such as a murder or fire
- giving impartial scientific evidence in court
4. Salary
Starter: £20,000
Experienced: £25,000 to £35,000
Highly Experienced: £45,000 or more
These figures are a guide.
5. Working hours, patterns and environment
You’ll usually work 37 hours a week, Monday to Friday. Your employer may use a shift or on-call system for dealing with high priority work.
You’ll be based mainly in a laboratory, but may also visit crime scenes, which could be distressing
You’ll wear special clothing to prevent contamination and protect you from hazardous substances.
6. Career path and progression
With experience, you could move into management and direct other forensics staff as a forensics manager or casework examiner.
You could also work as a reporting scientist, acting as an expert witness in court.
Related careers
You may be interested in:
7. Current opportunities
Apprenticeships
There are no apprenticeships for forensic scientist available at the moment.
Explore other apprenticeshipsTraining courses
There no courses for forensic scientist available at the moment.
Explore other coursesLast updated: 23 March 2016


Before Change
super().__init__(inplanes, planes, stride, downsample)
mid_planes = cardinality * planes // 32
self.conv1 = nn.Conv3d(inplanes, mid_planes, kernel_size=1, bias=False)
self.bn1 = nn.BatchNorm3d(mid_planes)
self.conv2 = nn.Conv3d(
mid_planes,
mid_planes,
kernel_size=3,
stride=stride,
padding=1,
groups=cardinality,
bias=False)
self.bn2 = nn.BatchNorm3d(mid_planes)
self.conv3 = nn.Conv3d(
mid_planes, planes * self.expansion, kernel_size=1, bias=False)
class ResNeXt(resnet.ResNet):
After Change
groups=cardinality,
bias=False)
self.bn2 = nn.BatchNorm3d(mid_planes)
self.conv3 = conv1x1x1(mid_planes, planes * self.expansion)
class ResNeXt(ResNet):


Plot a PRISMA <http://prisma-statement.org/> flow chart describing the identification, screening, eligibility and inclusion or studies in systematic reviews. PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.
| Version: | 1.0.4 |
| Imports: | DiagrammeR |
| Suggests: | DiagrammeRsvg, knitr, rmarkdown, rsvg, testthat, utils |
| Published: | 2018-08-08 |
| Author: | Jack O. Wasey [aut, cre, cph] |
| Maintainer: | Jack O. Wasey <jack at jackwasey.com> |
| BugReports: | https://github.com/jackwasey/PRISMAstatement/issues |
| License: | GPL-3 |
| URL: | https://github.com/jackwasey/PRISMAstatement |
| NeedsCompilation: | no |
| Materials: | README NEWS |
| CRAN checks: | PRISMAstatement results |
| Reference manual: | PRISMAstatement.pdf |
| Vignettes: |
Make a PRISMA flow chart |
| Package source: | PRISMAstatement_1.0.4.tar.gz |
| Windows binaries: | r-devel: PRISMAstatement_1.0.4.zip, r-release: PRISMAstatement_1.0.4.zip, r-oldrel: PRISMAstatement_1.0.4.zip |
| OS X binaries: | r-release: PRISMAstatement_1.0.4.tgz, r-oldrel: PRISMAstatement_1.0.4.tgz |
| Old sources: | PRISMAstatement archive |
Please use the canonical form https://CRAN.R-project.org/package=PRISMAstatement to link to this page.


Email or Username This field is required! Min. {{ identifierErrors.minlength.requiredLength }} chars are required! account_circle
Password This field is required! Min. {{ passwordErrors.minlength.requiredLength }} chars are required! {{ isPasswordVisible ? "visibility" : "visibility_off" }}
Login lock


const runtime/internal/sys.SpAlign
3 uses
runtime/internal/sys (current package)
stubs.go#L11: const SpAlign = 1*(1-GoarchArm64) + 16*GoarchArm64 // SP alignment: 1 normally, 16 for ARM64
runtime
proc.go#L4034: totalSize += -totalSize & (sys.SpAlign - 1) // align to spAlign
stack.go#L1248: minsize = sys.SpAlign
The pages are generated with Golds v0.2.7-preview. (GOOS=darwin GOARCH=amd64)


--- layout: post title: Interaction links date: 2006-11-13 00:00:00 category: interfaces-&-interaction ---
The third leg of my dumpy link stool - interaction-related stuff:


A few screenshots are provided here to give you a flavour, but it’s easier to download the software and run the demos (from the demos menus in each view) to see the variety of stimuli that can be generated.
PsychoPy is one of very few packages that allows a choice of interface. Use the Coder view, for those that like to program (or just use your own editor)
and the Builder view for those that don’t:
PsychoPy can handle every type of stimulus you can imagine...
Images and movies of most formats:
Random dots and element arrays, drawn in realtime:
Many text options and dialog boxes:
For more ideas about PsychoPy’s massive range of stimuli, install it, go to the Coder view and run some of the demo scripts (there’s a whole demos menu).


|
My Project
|
#include <SMap.hpp>
Public Member Functions | |
| void | calculateSpawn () |
| _object * | getSpaw () |
| bool | useSpawn (_object *spawn) |
| Public Member Functions inherited from Map | |
| Map () | |
| virtual bool | save_map (std::string pathname) |
| virtual bool | load_map (std::string pathname) |
| virtual int | serialize (char *buffer) |
| virtual void | deserialize (char *buffer) |
| _object * | getObjects () |
| ~Map () | |
Additional Inherited Members | |
| Protected Attributes inherited from Map | |
| int8_t | type |
| char | name [50] |
| _object * | objects |
| int16_t | length |
| int16_t | width |
| int16_t | max_objects |
| int16_t | n_object |
| void SMap::calculateSpawn | ( | ) |
| _object * SMap::getSpaw | ( | ) |
| bool SMap::useSpawn | ( | _object * | spawn | ) |


--- layout: default title: On Generative Algorithms - Differential Mesh ---
I've already mentioned Jared Tarbell in this text, but I will mention him once more still. One of the first pieces of generative art I ever saw, and took notice of, was Tarbell's Substrate. It's a genius algorithm in all it's simplicity: Lines grow in a particular direction until they reach the boundary of the domain, or until they collide with another line. When a line stops, at least one new line is spawned perpendicularly off of one of the existing lines at an arbitrary position.
Recreating the city-like structures of Substrate is an interesting exercise in itself. However I wanted to combine (some of) the behaviour of Substrate with other features of the previously mentioned leaf venation algorithm. The goal wast to mimic the fracturing patterns that occur in brittle materials like glass or porcelain. You might have seen this yourself if you been unfortunate enough to drop your mobile phone, or if you own some old tableware. When related to (among other things) paintings these patterns are referred to as Craquelure.
Similar to the method used in the leaf venation algorithm we distribute a number of sources into the domain. We then start any number of seed fractures. The following growth steps of these fractures are controlled in the following way:
In addition to controlling the growth of the fractures, we need to add new fractures. The strategy used in the images displayed here is to randomly spawn new fractures from the tip of the fractures that are currently active (i.e. have not yet collied with anything). Yet again the frequency of new fractures will have a major impact on the final result.
If have ever seen a broken phone screen you might have noticed that the fractures are rarely uniformly distributed: the density of fractures tends to be greater near the point of impact. In order to fake this effect you can introduce a fracturing speed. As a fracture grows the speed is gradually reduced. When a fracture is spawn it inherits a fraction (e.g. 0.8) of it's parent's current speed. In this case the fracturing speed is not used to control the rate of growth, but rather the rate of spawned new fractures. The effect looks like in the following image.
I published a few images that display the variation of the fracturing algorithm on my Ello account.
This little text is still in progress, and I might add a new section soon. In the meantime I regularly post things I'm working on at @inconvergent.Previous: Differential Mesh 3D


The basis for a series of two dimensional curves. These may or may not be normalised This class also can be used to represent a straight line with gradient one, through the origin - basically your classic straight line
@constructor
pointOnCurve - Return the point on this curve given u (or x)
y - Given x give us y
stepForU - given U return the step distance
xForStep - given d step along the curve give us x
pointDistance
sqrt(d^2 / 2) = a
tangentOnCurve
tangentDistance
The basis for any parametric curve. We cache values for distance and u so we can step along the curve using distance or normalized values TODO - Rename this as CurveP - for parametric
Our step size along the curve to generate points
Cache points for speed as oppose to memory TODO - Might change to instance variables later
@constructor
create a table of positions as a cache
pointOnCurve - Return the point on this curve given u (or x)
stepForU - given U return the step distance
uForStep - given a euclidean distance what is the step
pointDistance
If we have a cache table step through it and find u
tangentOnCurve - get a tangent from the normalised co-ordinate
tangentDistance - get a tangent from the distance along the curve
A Quadratic, bezier curve with four vectors determining the shape A 3D curve as oppose to 2D
@constructor
pointOnCurve
f - The Focus as a Vec2, a - parameter of the line equation (or 0 if x directrix), b - parameter of the line equation (or 0 if y directrix), and c - parameter of the line equation
TODO This formula actually works IF you remmeber there are two answers It DOESNT work for lines that are parallel to the x or y axis We need to revert to the directrix equation for that - this happens if either a or b is 0 Takes the line in the form ax + by + c = 0 This is nothing to do with the song "Parabola" by the band Tool; at least I don't think it is :S
constructor
sample
Three ways to sample - arbitrary line, x axis or y axis
x axis aligned (x-h)^2 = 4p(y-k)
y axis aligned - use the quadratic formula as we will have two answers
lineCrossing - figure out where a line crosses this parabola given the equation in the form ax + by + c = 0
TODO - line crossing for edges that are parallel to a major axis!
TODO - as we are are not keeping a count of the points, we cant really alter the patch on the fly :(
@constructor
Create the matrices we need for the output points
sample - Sample the patch with a vec2 (u,w) to gain the required point
TODO - eventually use the proper matrix http://www.cubic.org/docs/hermite.htm
@constructor
pointOnCurve - Return the point on this curve given u (or x)
A collection of curves forming a Catmull-rom spline.
TODO - as we are are not keeping a count of the points, we cant really alter the spline on the fly :( @constructor
Build from a set of CubicHermiteSplines
pointOnCurve - Return the point on this curve given u (or x)
Decide where in spline we are given the numbers we have
http://www.cs.cmu.edu/~fp/courses/graphics/asst5/cameraMovement.pdf essentially step along via a distance, passing in previous values and setting new ones
@constructor
reset - reset the slide starting from distance d, with a starting normal
slide - slide along the curve from the starting point by d distance
NURB Class TODO - Most of it - there is still a fair bit to go


WITNESS LIST Human Services Committee February 28, 2005-12:00A CPS Reform For: Bresnen, Steve (Federation of Texas Psychiatry) Hebeler, M.D., Joan (Texas Medical Association;Texas Pediatric Society) Lambert, Tim (Texas Home School Coalition) Santos, M.D., George (Self and Federation of Texas Psychiatry) Specia, Judge John (225th District Court, Bexar County) On: Bauman, Jr., Clyde (Self) Cantu, Cruz (Self and AARP) DeWitt, Lauren (The Citizens Commission on Human Rights) Ferland, Megan (Texas CASA) Hawkins, Albert (Health and Human Services Commission) Heiligenstein, Anne (Health and Human Services Commission) James, Joyce (Department of Family and Protective Services) La Vallo, Richard (Advocacy, Incorporated) Rodriguez, Carolyne (Casey Family Programs) Shon, M.D., Steven (Department of State Health Services- Behavioral Health) Spiller, Lee (Citizens Commission on Human Rights) Steiner, Dwight (Self) Registering, but not testifying: On: Cockerell, Carey (Department of Family and Protective Services)


NSSWITCH.CONF
Section: Linux Programmer's Manual (5)Updated: 1999-01-17
Index Return to Main Contents
NAME
nsswitch.conf - System Databases and Name Service Switch configuration fileDESCRIPTION
Various functions in the C Library need to be configured to work correctly in the local environment. Traditionally, this was done by using files (e.g., /etc/passwd), but other nameservices (like the Network Information Service (NIS) and the Domain Name Service (DNS)) became popular, and were hacked into the C library, usually with a fixed search order.The Linux libc5 with NYS support and the GNU C Library 2.x (libc.so.6) contain a cleaner solution of this problem. It is designed after a method used by Sun Microsystems in the C library of Solaris 2. We follow their name and call this scheme "Name Service Switch" (NSS). The sources for the "databases" and their lookup order are specified in the /etc/nsswitch.conf file.
The following databases are available in the NSS:
- aliases
- Mail aliases, provides a system-wide mechanism to redirect mail for local recipients. Used by mail transfer agents such as Postfix or sendmail(8). Note: On Linux, not like on other Unices, Sendmail uses its own aliases resolution system independent on `/etc/nsswitch.conf'.
- ethers
- Ethernet numbers.
- group
- Groups of users, used by getgrent(3) functions.
- hosts
- Host names and numbers, used by gethostbyname(3) and similar functions.
- netgroup
- Network wide list of hosts and users, used for access rules. C libraries before glibc 2.1 only support netgroups over NIS.
- networks
- Network names and numbers, used by getnetent(3) functions.
- passwd
- User passwords, used by getpwent(3) functions.
- protocols
- Network protocols, used by getprotoent(3) functions.
- publickey
- Public and secret keys for Secure_RPC used by NFS and NIS+.
- rpc
- Remote procedure call names and numbers, used by getrpcbyname(3) and similar functions.
- services
- Network services, used by getservent(3) functions.
- shadow
- Shadow user passwords, used by getspnam(3).
An example /etc/nsswitch.conf (namely, the default used when /etc/nsswitch.conf is missing):
- passwd:
- compat
- group:
- compat
- shadow:
-
compat
- hosts:
- dns [!UNAVAIL=return] files
- networks:
- nis [NOTFOUND=return] files
- ethers:
- nis [NOTFOUND=return] files
- protocols:
- nis [NOTFOUND=return] files
- rpc:
- nis [NOTFOUND=return] files
- services:
- nis [NOTFOUND=return] files
The first column is the database. The rest of the line specifies how the lookup process works. You can specify the way it works for each database individually.
The configuration specification for each database can contain two different items:
- * The service specification like `files', `db', or `nis'.
- * The reaction on lookup result like `[NOTFOUND=return]'.
For libc5 with NYS, the allowed service specifications are `files', `nis', and `nisplus'. For hosts, you could specify `dns' as extra service, for passwd and group `compat', but not for shadow.
For glibc, you must have a file called /lib/libnss_SERVICE.so.X for every SERVICE you are using. On a standard installation, you could use `files', `db', `nis', and `nisplus'. For hosts, you could specify `dns' as extra service, for passwd, group, and shadow `compat'. These services will not be used by libc5 with NYS. The version number X is 1 for glibc 2.0 and 2 for glibc 2.1.
The second item in the specification gives the user much finer control on the lookup process. Action items are placed between two service names and are written within brackets. The general form is
`[' ( `!'? STATUS `=' ACTION )+ `]'
where
- STATUS => success | notfound | unavail | tryagain
- ACTION => return | continue
The case of the keywords is insignificant. The STATUS values are the results of a call to a lookup function of a specific service. They mean:
- success
- No error occurred and the wanted entry is returned. The default action for this is `return'.
- notfound
- The lookup process works ok but the needed value was not found. The default action is `continue'.
- unavail
- The service is permanently unavailable. This can either mean the needed file is not available, or, for DNS, the server is not available or does not allow queries. The default action is `continue'.
- tryagain
- The service is temporarily unavailable. This could mean a file is locked or a server currently cannot accept more connections. The default action is `continue'.
Interaction with +/- syntax (compat mode)
Linux libc5 without NYS does not have the name service switch but does allow the user some policy control. In /etc/passwd you could have entries of the form +user or +@netgroup (include the specified user from the NIS passwd map), -user or -@netgroup (exclude the specified user), and + (include every user, except the excluded ones, from the NIS passwd map). Since most people only put a + at the end of /etc/passwd to include everything from NIS, the switch provides a faster alternative for this case (`passwd: files nis') which doesn't require the single + entry in /etc/passwd, /etc/group, and /etc/shadow. If this is not sufficient, the NSS `compat' service provides full +/- semantics. By default, the source is `nis', but this may be overridden by specifying `nisplus' as source for the pseudo-databases passwd_compat, group_compat and shadow_compat. These pseudo-databases are only available in GNU C Library.FILES
A service named SERVICE is implemented by a shared object library named libnss_SERVICE.so.X that resides in /lib.- /etc/nsswitch.conf configuration file
- /lib/libnss_compat.so.X
- implements `compat' source for glibc2
- /lib/libnss_db.so.X
- implements `db' source for glibc2
- /lib/libnss_dns.so.X
- implements `dns' source for glibc2
- /lib/libnss_files.so.X
- implements `files' source for glibc2
- /lib/libnss_hesiod.so.X
- implements `hesiod' source for glibc2
- /lib/libnss_nis.so.X
- implements `nis' source for glibc2
- /lib/libnss_nisplus.so.2
- implements `nisplus' source for glibc 2.1
NOTES
Within each process that uses nsswitch.conf, the entire file is read only once; if the file is later changed, the process will continue using the old configuration.With Solaris, it isn't possible to link programs using the NSS Service statically. With Linux, this is no problem.
COLOPHON
This page is part of release 3.22 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/.
Index
This document was created by man2html, using the manual pages.
Time: 05:34:00 GMT, December 24, 2015


ABOUT
{{this.about}}
{{this.about}}
Here we highlight all the major projects the Python Nigeria community are a part of and supports. Please note that members who take part in these projects do so voluntarily without any remuneration. The community as a whole benefits and are proud of all the collective efforts and man hours put into such projects to make it a success fostering growth.
ViewThe Nigerian Python community is made up of members from around Nigeria with very diverse set of skills, personalities, cultures and experiences. It is through these differences that our community has experienced many successes and continued growth. When you're working with members of the community, we encourage you to follow these guidelines which help steer our interactions and strive to keep Python a positive, successful, and growing community. A member of the Nigeria Python community is:...
ViewThe Python Nigeria Community is governed by constitution. The details are as follows....
View


EXCLUSIVE: Quando Rondo Recounts Going on the Run at 13 After Cutting Ankle Monitor
by VladTVIn this clip, Quando Rondo detailed his history within the criminal justice system. After recounting going to juvenile hall on three separate occasions within the span of three months, Quando Rondo talked about going on the run at 13. This was the result of being placed on house arrest and Rondo deciding to cut off the ankle monitor.


6 Years
26 houses designed
21 spaces designed
Start by asking for an interior design consultation, and we will shape your vision according to your budget and requirements.
Want specific adjustments with your interior décor? Get detailed designs for every room and adapt them to your taste.
Inspect your plan and decisions about interior design, make improvements the way you want and visualize how it looks in 3D.
A team of professional craftsmen will start shaping your dream when you're pleased with our designs and our deals.
We make sure that everything is within your budget while ensuring best quality and long-lasting durability.
Enjoy your new home!
We know how to make your house into a home with an enjoyable experience. We know how to provide your home with a complete fusion of elegance, luxury and comfort to suit your taste and style of life.
We specialize in the construction of spaces for work, from office space to meeting facilities. We are experienced in architecture and turn-key solutions for the commercial space, whether it is Shops, Restaurants, Hotels, Clubs, or Spas.
Pixilo Design considers every project as a new partnership and goes that extra mile to ensure 100% client satisfaction. Our experience and passion is evident in our designs that are Smart, Polished, Graceful and Relaxed.
With over six years of experience in the field of commercial and residential interior design, Pixilo is suitably equipped to handle a wide array of projects.
We work only with the best, at Pixilo you can avail the services from the best brands in the Interior Space between the most reasonable pricing.
On Time – On Budget – On Quality Project is our promise.
Pixilo Design is a one-stop-shop for every commercial or residential interior design. Pixilo is turnkey. This means that we are a complete interior design solution provider, for both design and execution.
Ankush and his team were extremely responsive throughout the project, their creativity coupled with utmost professionalism, responsibility and integrity reflects in the beautiful home they've designed for us. They help you maintain your house and are most accessible at any point. We look forward to working with them again.
Concepts are very neat, clean and functional, and most of all, beautifully asthetic and up to date. Their attention to detail and quality, and their extremely professional work ethic are what stood out in their execution and made it very easy for us. Overall, we had an extermely positive experience, and we look forward to working with Pixilo for all our future interior design requirements.


Gateway
BOT
04-Jul-18 09:47 AM
[dscrdeql] <jellysmash> and then coconut milk added
[dscrdeql] <jellysmash> potatoes in it where perfect
[dscrdeql] <AndronicusBass-Founder> Slow cooked is awesome. I have a tefal cook for me pressure cooker it does well but then it's good to transfer to a pot and reduce the liquid for a bit
[dscrdeql] <jellysmash> yea, that sounds pretty decent
[dscrdeql] <jellysmash> we've cooked for hours in a cast iron casserole dish.
[dscrdeql] <AndronicusBass-Founder> You should try yuca we had it in Puerto Rico I like it better than potato.  I remember when @!!! Thorth !!! was going up to level 12 it was 1260 points and now I can see going from level 8 to level 9 is also 1260
[dscrdeql] <AndronicusBass-Founder> markets maybe
[dscrdeql] <jellysmash> yea, maybe the markets
[dscrdeql] <jellysmash> can't really get to the markets
[dscrdeql] <AndronicusBass-Founder> Not sure, i dont think the xp has changed, I can check.
[dscrdeql] <!!! Thorth !!!> aaa???? what si with me ???
[dscrdeql] <!!! Thorth !!!>
[dscrdeql] <jellysmash> just wondering if the number of xp required for each level has gone up
[dscrdeql] <jellysmash> when you were going to level 12 I think it was around 1260 xp, same as for me going from level 8 to level 9
[dscrdeql] <jellysmash> I could be wrong though.
[dscrdeql] <!!! Thorth !!!> well no ideea :-?? never ask myself
[dscrdeql] <jellysmash> not that it matters.....just wondering
[dscrdeql] <KQweci> What are the benefits of XP
[dscrdeql] <jellysmash> go up levels with more XP
[dscrdeql] <KQweci> And benefits of levels up
[dscrdeql] <KQweci> Good job
[dscrdeql] <x2maano> yeah
[dscrdeql] <Cypher - Founder>
[dscrdeql] <Cypher - Founder> we do our best
[dscrdeql] <cypherzero> good day!
[dscrdeql] <Cypher - Founder> hows it going
[dscrdeql] <KQweci> :EQL:
[dscrdeql] <Cypher - Founder> oh
[dscrdeql] <Cypher - Founder> is there any emoji you want for the server
[dscrdeql] <Cypher - Founder> btw
[dscrdeql] <!!! Thorth !!!> for all !!!
[dscrdeql] <!!! Thorth !!!>
[dscrdeql] <Cypher - Founder> indeed
[dscrdeql] <jellysmash> thx for hot beverage
[dscrdeql] <Cypher - Founder> omg i was soo tired instead of adding sugar cos i need it today i added 2 coffee spoons
[dscrdeql] <Cypher - Founder> sooo stronk
[dscrdeql] <Cypher - Founder> O.o
[dscrdeql] <!!! Thorth !!!> uuuu
[dscrdeql] <!!! Thorth !!!> nice
[dscrdeql] <!!! Thorth !!!> give it to me
[dscrdeql] <jellysmash> niiiiice
[dscrdeql] <Cypher - Founder> i love good coffeee
[dscrdeql] <Cypher - Founder> ask @AndronicusBass-Founder
[dscrdeql] <Cypher - Founder> we both fdo
[dscrdeql] <Cypher - Founder> but feels like an expresso
[dscrdeql] <Cypher - Founder> lol
[dscrdeql] <!!! Thorth !!!> black, strong coffee
[dscrdeql] <!!! Thorth !!!>
[dscrdeql] <!!! Thorth !!!>
[dscrdeql] <!!! Thorth !!!>
[dscrdeql] <AndronicusBass-Founder> oh extra strong coffee is my fav
[dscrdeql] <Cypher - Founder> -gif coffee
[dscrdeql] <AndronicusBass-Founder> greek coffee or turkish coffee i believe is thick strong
[dscrdeql] <jellysmash> looks like she'd serve a bit more than just coffee
[dscrdeql] <Cypher - Founder> turkish coffee is unique
[dscrdeql] <Cypher - Founder> i love my teas too
[dscrdeql] <x2maano> hahha
[dscrdeql] <jellysmash> i have green tea in afternoons sometimes
[dscrdeql] <jellysmash> I can't have coffee too late in afternoon
[dscrdeql] <jellysmash> otherwise I can't sleep
[dscrdeql] <AndronicusBass-Founder> @Cypher - Founder for a second i read the word 'teas' as 'tears' LMAO
[dscrdeql] <AndronicusBass-Founder> I think my dislexia is getting worse hahahaha
[dscrdeql] <Kitty> hi guys
[dscrdeql] <Kitty> have a nice day~
[dscrdeql] <AndronicusBass-Founder> How are you?
[dscrdeql] <cypherzero> nice progress!
[dscrdeql] <AndronicusBass-Founder> And yeah this is a good channel to chat to gain XP to be able to receive tipsy rain
[dscrdeql] <Kitty> just off from work ...hha
[dscrdeql] <Kitty> so tried
[dscrdeql] <Kitty>
[dscrdeql] <AndronicusBass-Founder> ave you heard of a tea called yerba mate?
[dscrdeql] <x2maano> what a pleasent wheather
[dscrdeql] <AndronicusBass-Founder> Its cold for me lol
[dscrdeql] <AndronicusBass-Founder> I'm in Adelaide Australia and its winter
[dscrdeql] <x2maano> oh we have summer
[dscrdeql] <x2maano> on its peak
[dscrdeql] <Kitty> how cold lol
[dscrdeql] <AndronicusBass-Founder> Another 2 weeks and we will be at the peak of winter
[dscrdeql] <x2maano> ausome
[dscrdeql] <AndronicusBass-Founder> But it does get very hot here. we are the driest state in all of australia.
[dscrdeql] <x2maano> oopss
[dscrdeql] <x2maano> then enjoy the winter completely
[dscrdeql] <AndronicusBass-Founder> 'Ausome' I like that. its a merge of Awesome and Australia words lol
[dscrdeql] <Kitty> hahaha... anyway have anice day
[dscrdeql] <AndronicusBass-Founder> ok rest well
[dscrdeql] <x2maano> thats why i use it
[dscrdeql] <AndronicusBass-Founder> Come back when you are less tired as we have trivia sometimes too
[dscrdeql] <AndronicusBass-Founder> @x2maano i havent heard it before, its Awesome
[dscrdeql] <x2maano> hhaahahaa
[dscrdeql] <x2maano> good joke
[dscrdeql] <x2maano> i just want a smile on ur face
[dscrdeql] <AndronicusBass-Founder> Most of the time I'm happy lol
[dscrdeql] <AndronicusBass-Founder> thanks, heres a smile hahaha


--- layout: page title: Contact permalink: /contact/ social_image: /assets/share.jpg type: page rel: contact css: contact.css ---


{% extends "base.html" %} {% block title %}{{ project.title }} Project{% endblock %} {% block heading %}{{ project.title }} ({{ project.id }}){% endblock %} {% block content %} {% include "project-info-snippet.html" %}
| Assignments Completed | Number of tasks |
| {{ assignment.completed_assignments }} | {{ assignment.howmany }} |
|---|
Links to completed but unmerged tasks look like this and links to merged tasks are like this.


Evaluation Protocol¶
Versioning: The evaluations below are for v1.5. If you are here for the competition, refer to the AICrowd page for results. If you are here for the publication, we are in the process of aggregating experiment results for v1.5.1 and generating figures.
Training is performed on a pool of N_TRAIN_MAPS procedurally generated maps simulated for TRAIN_HORIZON timesteps per episode. Evaluation is performed on EVAL_MAPS maps never seen during training (GENERALIZE=False) for EVALUATION_HORIZON timesteps. Approaches that modify these parameters, alter the reward function, or substantially modify the environment configuration are not directly comparable to the baselines below.
Publications including new models should clearly indicate training scale, domain knowledge assumptions, and any other significant changes to the default configuration. Ideally, they should also include a discussion of agent behavior and supporting training/evaluation plots, overlay figures, and/or other evidence specific to the associated work.
To reproduce the tables and figures below, run the associated command. Run train or evaluate instead of visualize to verify results from scratch or from the pretrained model.
Small Maps¶
This setting includes 256 80x80 maps (60x60 excluding lava) and supports up to 128 agents/32 NPCs. It is intended for training and evaluation horizons of 1000 timesteps, or 10 minutes in real time. Evaluation takes only a few minutes.
Neural Baseline¶
Our baseline is trained using an RTX 3080 + 64 cores for ~(100k maps/1000 epochs/4 days). The first skills to emerge are survival/foraging/exploration, followed by combat, followed much later by attacking NPCs for equipment. Training for an extra ~500 epochs did not result in any substantial improvements.
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
18.00 |
57.00 |
45.95 |
4.09 |
Lifetime |
0.00 |
1000.00 |
46.49 |
110.78 |
Skilling |
10.00 |
50.50 |
14.06 |
5.92 |
Combat |
3.00 |
28.00 |
4.64 |
3.06 |
Equipment |
0.00 |
18.00 |
0.22 |
1.36 |
Exploration |
0.00 |
73.00 |
8.23 |
6.34 |
Untrained¶
A randomly initialized copy of the trained model architecture
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
1.00 |
17.00 |
8.29 |
2.37 |
Lifetime |
0.00 |
480.00 |
8.86 |
14.85 |
Skilling |
10.00 |
36.50 |
10.41 |
1.19 |
Combat |
3.00 |
5.00 |
3.87 |
0.34 |
Equipment |
0.00 |
4.00 |
0.00 |
0.07 |
Exploration |
0.00 |
26.00 |
2.22 |
2.72 |
Zero-Shot Transfer¶
Evaluation of the large-map model on the SmallMaps domain
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
4.00 |
69.00 |
50.60 |
5.90 |
Lifetime |
0.00 |
999.00 |
50.90 |
85.25 |
Skilling |
10.00 |
48.50 |
14.43 |
6.24 |
Combat |
3.00 |
12.00 |
3.93 |
0.99 |
Equipment |
0.00 |
6.00 |
0.02 |
0.24 |
Exploration |
0.00 |
102.00 |
11.59 |
13.54 |
Scripted Foraging¶
A scripted model that forages for food and only fights back when attacked by NPCs.
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
40.00 |
102.00 |
80.70 |
9.71 |
Lifetime |
0.00 |
995.00 |
81.01 |
123.84 |
Skilling |
10.00 |
49.00 |
16.70 |
6.74 |
Combat |
3.00 |
5.00 |
3.09 |
0.29 |
Equipment |
0.00 |
0.00 |
0.00 |
0.00 |
Exploration |
0.00 |
111.00 |
17.50 |
13.12 |
Scripted Foraging + Combat¶
A scripted model that forages for food and actively fights other scripted agents and NPCs
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
27.00 |
62.00 |
49.50 |
4.43 |
Lifetime |
0.00 |
994.00 |
50.92 |
74.27 |
Skilling |
10.00 |
53.00 |
15.04 |
5.54 |
Combat |
3.00 |
33.00 |
4.35 |
2.77 |
Equipment |
0.00 |
26.00 |
0.10 |
1.04 |
Exploration |
0.00 |
101.00 |
14.94 |
10.80 |
Large Maps¶
This setting includes 256 1024x1024 maps (1004x1004 excluding lava) and supports up to 1024 agents/1024 NPCs. It is intended for training and evaluation horizons of 6000-12000+ timesteps, or 1-2 hours in real time. Evaluation takes several hours. Reasonable policies are trainable with 4 CPU cores and a single GPU in a few days. Our baseline is trained using an RTX 3080 + 16 cores for ~5 days. The bounds of scaling with additional compute are unknown – note that this setting is typically RAM bound.
Neural Baseline¶
Our baseline is trained using an RTX 3080 + 16 cores for ~(7.5k maps/100 epochs/1 day). The first skill to emerge is capable foraging and exploration, followed by combat and some equipment farming. Training for an extra ~740 epochs results in improved combat and equipment farming but greatly decreased lifetime/foraging/exploration.
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
56.00 |
904.00 |
680.67 |
170.47 |
Lifetime |
0.00 |
10000.00 |
232.52 |
833.39 |
Skilling |
10.00 |
78.50 |
19.51 |
10.30 |
Combat |
3.00 |
27.00 |
4.28 |
2.29 |
Equipment |
0.00 |
15.00 |
0.00 |
0.13 |
Exploration |
0.00 |
580.00 |
30.50 |
44.57 |
Untrained¶
A randomly initialized copy of the trained model architecture
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
60.00 |
103.00 |
84.49 |
4.44 |
Lifetime |
0.00 |
441.00 |
32.74 |
11.11 |
Skilling |
10.00 |
37.00 |
10.90 |
1.58 |
Combat |
3.00 |
7.00 |
3.02 |
0.16 |
Equipment |
0.00 |
5.00 |
0.00 |
0.02 |
Exploration |
0.00 |
42.00 |
6.25 |
2.80 |
Zero-Shot Transfer¶
Evaluation of the small-map model on the LargeMaps domain
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
41.00 |
260.00 |
197.00 |
33.20 |
Lifetime |
0.00 |
10000.00 |
75.53 |
412.76 |
Skilling |
10.00 |
80.00 |
13.57 |
6.37 |
Combat |
3.00 |
20.00 |
4.81 |
1.90 |
Equipment |
0.00 |
22.00 |
0.01 |
0.33 |
Exploration |
0.00 |
431.00 |
8.98 |
10.13 |
Scripted Foraging¶
A scripted model that forages for food and only fights back when attacked by NPCs.
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
95.00 |
1024.00 |
989.19 |
128.00 |
Lifetime |
0.00 |
9995.00 |
995.88 |
994.33 |
Skilling |
10.00 |
76.00 |
39.84 |
14.51 |
Combat |
3.00 |
3.00 |
3.00 |
0.00 |
Equipment |
0.00 |
0.00 |
0.00 |
0.00 |
Exploration |
0.00 |
570.00 |
222.63 |
144.32 |
Scripted Combat¶
Metric |
Min |
Max |
Mean |
Std |
|---|---|---|---|---|
Population |
55.00 |
648.00 |
548.28 |
58.87 |
Lifetime |
0.00 |
9996.00 |
194.07 |
587.94 |
Skilling |
10.00 |
76.00 |
20.46 |
10.07 |
Combat |
3.00 |
36.00 |
5.02 |
2.94 |
Equipment |
0.00 |
29.00 |
0.01 |
0.40 |
Exploration |
0.00 |
532.00 |
49.50 |
61.95 |


Giadi1833.github.io
View and Download Remington AQ7 manual online. AQ7 Electric Shaver pdf manual download. REMINGTON AQ7 MANUAL Pdf Download. View and Download Remington F3800 user manual online. DUAL-X Foil Shaver. F3800 Electric Shaver pdf manual download. Also for: F5800, F7800.
REMINGTON F3800 USER MANUAL Pdf Download. Remington F4790 Bedienungsanleitung.


Script Provided Below.. _calm_escript:
Calm: EScript and Task Library¶
Overview¶
In the Calm: Linux Workloads and Calm: Windows Workloads labs you explored how Calm can utilize Bash and PowerShell scripts to automate application deployments. While shell scripts can be both powerful and versatile, they require the deployment of an endpoint VM on which to copy and execute the script locally.
There are use cases that would be better served to execute code directly within Calm, such as making API calls to other RESTful services like Nutanix Era, GitHub, IFTTT, etc.
To fill this need, Calm offers a script type called EScript. Short for Epsilon Script (Epsilon is the orchestration engine that drives Calm), EScript is a sandboxed Python interpreter. It contains many commonly used modules for scripting and automation. In particular, it contains the requests module as urlreq, used to create external API calls.
Note
The interpreter is sandboxed because the script runs directly within the Calm engine, which means it runs directly within Prism Central. Allowing the user to import unknown and unvetted modules into Prism Central is a security concern.
In this lab you will leverage EScript to perform API calls against an existing web service, your Prism Central VM. You will also leverage Set Variable to set the data returned from an API call to a Calm macro, and the Task Library to understand how common code can be used across multiple blueprints or projects.
Lab Setup¶
This lab assumes basic familiarity with Nutanix Calm.
It does NOT require the Task Manager application deployed as part of other Calm labs. Instead, you will create a new, blank Blueprint and add a credential used to authenticate to Prism Central.
In Prism Central, select > Services > Calm > Blueprints.
Click + Create Blueprint > Multi VM/Pod Blueprint.
Fill out the following fields and click Proceed:
- Name - Initials-EScript
- Description - My First EScript Blueprint
- Project - Initials-Calm
From the toolbar along the top of the blueprint, click Credentials.
Click Credentials and fill out the following fields:
- Credential Name - PC_Creds
- Username - admin
- Secret Type - Password
- Password - Your Prism Central admin Password
Click Save and then Back, ensuring no errors or warnings appear.
Using Existing Machine Services¶
In Application Overview > Services, click to add a new Service.
Under the VM tab, fill in the following fields:
Service Name PC Name PrismCentral Cloud Existing Machine Operating System Linux IP Address localhost Check log-in upon create Unselect Credential Leave default Address Leave default Connection Type Leave default Connection Port Leave default Delay Leave default There are several new and interesting items in the above configuration:
- Cloud - Instead of creating a new VM on Nutanix or a public cloud provider, we’re instead choosing to automate against an already existing machine. All that’s required for us to enter is an IP Address of the machine, which in our case is Prism Central. Depending on your use case, you may specify something like Ansible Tower, or an Era Server, as an existing machine.
- IP Address - Since we’re going to be making API Calls against Prism Central, and Calm runs directly in Prism Central, we’re just entering localhost as an IP. If you were going to be automating against something like Ansible Tower or Era, you would need to put your Ansible Tower or Era Server IP addresses in this field, rather than localhost. IP Address could also be defined by a variable.
- Check log-in upon create - This is the first time we’ve seen this box unselected. Since EScript tasks run directly within Calm, there’s no need to SSH in to the Service in question. Instead we’ll use credentials directly within the EScript code to authenticate our REST API call.
Click Save, and ensure no errors or warnings appear.
RESTList Custom Action¶
In this exercise, we’re going to be creating a custom action for our application to make a REST API call against Prism Central. Specifically, it will be a POST /list call, where the entity (kind) to be listed (e.g. apps, hosts, clusters, roles, etc.) will be defined by a variable at runtime. The results of this call will then be output.
In the Application Overview > Application Profile section, expand the Default Application Profile.
Select next to Actions to add a new, custom action.
On the Configuration Pane to the right, name the action RESTList, and add a single variable:
- Name - kind
- Data Type - String
- Value - apps
- Select Runtime by toggling the running man icon in the upper right to blue
When running the custom action later, Calm will prompt the user for input. Apps will be pre-filled default value, but it can be changed prior to executing the script action.
Click the + Task button to add a task to the RESTList custom action. Fill in the following fields:
- Task Name - RuntimePost
- Type - Execute
- Script Type - EScript
- Script - Script Provided Below
# Set the credentials pc_user = '@@{PC_Creds.username}@@' pc_pass = '@@{PC_Creds.secret}@@' # Set the headers, url, and payload headers = {'Content-Type': 'application/json', 'Accept': 'application/json'} url = "https://@@{address}@@:9440/api/nutanix/v3/@@{kind}@@/list" payload = {} # Make the request resp = urlreq(url, verb='POST', auth='BASIC', user=pc_user, passwd=pc_pass, params=json.dumps(payload), headers=headers) # If the request went through correctly, print it out. Otherwise error out, and print the response. if resp.ok: print json.dumps(json.loads(resp.content), indent=4) exit(0) else: print "Post request failed", resp.content exit(1)
There are some new and interesting features of this task:
Note how there is not a Credential dropdown within the Calm UI, and instead we’re setting Python variables equal to our PC_Creds username and password specified earlier. Other APIs may not require authentication, or require an API key to be provided as part of the URL.
We also see the urlreq module being used, which is the exact line that our API call is made. If the response returns as expected, the JSON response will be formatted and printed, otherwise the corresponding error message will be printed.
Click Save, and ensure no errors or warnings appear.
GetDefaultSubnet Custom Action¶
In this exercise, we’re going to create an additional custom action to make a different REST API call. The call will return the list of Projects on this Prism Central instance. We’ll then parse the output of that API call to get the UUID of the default subnet configured for the project that the running application belongs to. This UUID will be set as a Calm variable, allowing for re-use elsewhere in the blueprint. We’ll then do another Rest API call, a GET on the default subnet (utilizing this newly set variable).
Select the PC service. In the Configuration Pane, select the Service tab. Add a variable named SUBNET, leaving all other fields blank.
In the Application Overview > Application Profile > Default, section, select next to Actions to add a new, custom action.
Name the action GetDefaultSubnet.
Click the + Task button to add a task to the GetDefaultSubnet custom action. Fill in the following fields:
- Task Name - GetSubnetUUID
- Type - Set Variable
- Script Type - EScript
- Script - Script Provided Below
- Output - SUBNET
# Get the JWT jwt = '@@{calm_jwt}@@' # Set the headers, url, and payload headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'Authorization': 'Bearer {}'.format(jwt)} url = "https://@@{address}@@:9440/api/nutanix/v3/projects/list" payload = {} # Make the request resp = urlreq(url, verb='POST', params=json.dumps(payload), headers=headers, verify=False) # If the request went through correctly if resp.ok: # Cycle through the project "entities", and check if its name matches the current project for project in json.loads(resp.content)['entities']: if project['spec']['name'] == '@@{calm_project_name}@@': # If there's a default subnet reference, print UUID to set variable and exit success, otherwise error out if 'uuid' in project['status']['resources']['default_subnet_reference']: print "SUBNET={0}".format(project['status']['resources']['default_subnet_reference']['uuid']) exit (0) else: print "The '@@{calm_project_name}@@' project does not have a default subnet set." exit(1) # If we've reached this point in the code, none of our projects matched the calm_project_name macro print "The '@@{calm_project_name}@@' project does not match any of our /projects/list api call." print json.dumps(json.loads(resp.content), indent=4) exit(0) # In case the request returns an error else: print "Post clusters/list request failed", resp.content exit(1)
There are two key differences between the RESTList and GetDefaultSubnet tasks. The first difference is the use of the Set Variable task type. Take note of the print “SUBNET={0}” line: Calm will parse output in the format of variable=value, and set the variable equal to the value. In this example, we’re printing the variable called SUBNET is equal to the UUID of the “default_subnet_reference” field in the initial API call response. In the Output field below the Script body, we must paste in the variable name for Calm to set the variable appropriately. The variable must already be defined in the Calm blueprint, whether globally, or in this case, as a variable local to the PC service.
The second difference is that the PC_Cred credential was not used to authorize the API call against Prism Central. Instead, we’re using a JSON Web Token provided by the built-in calm_jwt macro.
Click the + Task button again to add a second task to the GetDefaultSubnet custom action. Fill in the following fields:
- Task Name - GetSubnetInfo
- Type - Execute
- Script Type - EScript
- Script - Script Provided Below
# Get the JWT jwt = '@@{calm_jwt}@@' # Set the headers, url, and payload headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'Authorization': 'Bearer {}'.format(jwt)} url = "https://@@{address}@@:9440/api/nutanix/v3/subnets/@@{SUBNET}@@" payload = {} # Make the request resp = urlreq(url, verb='GET', params=json.dumps(payload), headers=headers, verify=False) # If the request went through correctly, print it out. Otherwise error out, and print the response. if resp.ok: print json.dumps(json.loads(resp.content), indent=4) exit(0) else: print "Get request failed", resp.content exit(1)
In this task we’re dynamically returning details about the default subnet using a GET API call and the SUBNET UUID variable returned by the previous task.
Click Save, and ensure no errors or warnings appear.
Running the Custom Actions¶
Launch the blueprint. Name the application Initials-RestCalls, and then click Create.
The Create task should complete quickly, as no VMs are being provisioned or Package Install scripts being run.
Once the application reaches Running status, select the Manage tab.
Next, run the RESTList action by clicking its icon. A new window appears displaying the kind variable and default apps value. Click Run.
In the output on the right pane, maximize the RuntimePost task, and view the API output. The output pane can be toggled by clicking the icon. Maximize the output/script window to make viewing easier. As expected, the script returns a JSON body with an array describing each launched application in Calm.
Run the RESTList action again, altering the value to another Prism Central API entity, such as images, clusters, hosts, or vms.
Finally, run the GetDefaultSubnet action. Expand both the GetSubnetUUID and GetSubnetInfo tasks, reviewing the output for each task. What is the name and VLAN id of your default subnet?
Publishing to the Task Library¶
Tasks such as common API calls, package installations for common services, domain joins, etc. can be broadly applicable to multiple blueprints. These tasks can be used without leveraging third party tools or manually copying and pasting scripts by instead publishing into the Task Library, Calm’s central repository for code re-use.
Open your Initials-EScript blueprint in the Blueprint Editor.
In the Application Overview > Application Profile pane, select the RESTList action.
Select the RuntimeList task to open the task in the Configuration Pane.
Click Publish to Library.
In the Publish Task window, make the following changes:
- Name - Initials Prism Central Runtime List
- Replace address with Prism_Central_IP
Click Apply and note that the original address macro was replaced with Prism_Central_IP in the script window. Replacing macro names allows you to be more generic or descriptive to increase task portability.
Click Publish.
Open the Task Library in the sidebar. Select your published task. By default, the task will be available to the project from which it was originally published, but you can specify additional projects with which to share the task.
Takeaways¶
What are the key things you should know about Nutanix Calm?
- The task library allows commonly used operations to be written once and reused over and over again. As time goes on more objects will be integrated into the task library, from Nutanix-provided common tasks to entire service objects
- Calm 2.7 introduced the HTTP task, allowing the most common use of Escript to be more easily implemented (sending API calls)
- In addition to being able to use Bash and Powershell scripts, Nutanix Calm can use EScript, which is a sandboxed Python interpreter, to provide application lifecycle management.
- EScript tasks are run directly within the Calm engine, rather than being executed on the remote machine.
- Shell, Powershell, and EScript tasks can all be utilized to set a variable based on script output. That variable can then be used in other portions of the blueprint.
- The Task Library allows for publishing of commonly used tasks into a central repository, giving the ability to share these scripts across Projects and Blueprints.


Making the Shift
So you've seen the statistics and watched as the jobs in your market slowly disappear. You want to move more to the "business side," but you don't quite know how to do it. As I'll describe next, making the shift can be done on-the-job by gaining more responsibility, polishing up your problem-solving skills, and using creativity in your work.
I began my journey into systems analysis and design by accepting more responsibilities throughout the project I was on when things proved too overwhelming for my superiors. I gradually accepted more of the project management and business analysis responsibilities when the opportunity presented itself. For example, I would walk to Suzy in accounting and work out a new enhancement with her one-on-one rather than wait for my manager to do so. Over time, as my manager's confidence in my abilities grew, these responsibilities became a part of my job. It wasn't long before I became the Programmer Analyst, and ultimately the Project Manager, as new positions were created to fulfill demand for our work.
When the need arises, I recommend walking to the end user yourself and working with her one-on-one. Your manager will be relieved when he discovers that you are capable of communicating with his end-users, identifying their issues, and resolving those issues before they are brought up in the weekly manager's meeting. Even the best IT managers need a subordinate who is visible to the users who they can trust to get the job done. If a manager is slowly factoring himself away from the day-to-day workings of the project, welcome it. The higher visibility that you are obtaining can be translated into higher value—and that can result in a promotion. Over time, your increased interactions with more business-oriented people will make you more sensitive to business concerns.
A good subordinate has to be open-minded and creative. When solving problems, one has to always believe that there is a way to accomplish something, even if it's never been done before. Sometimes, just listening to the user will produce an idea. A lot of issues may come down to the business process that the system is attempting to replicate. I have had users actually solve a business problem for me just by listening to what they had to say!
Whether you're open-minded and creative or not, you can still work towards more business-oriented positions. After all, business systems analysts and project managers are only a small subset of the many positions opening up each year to address the issues of complexity through simplicity. If you love programming, you don't have to necessarily give it up.