10 Data transformation with dplyr
This section focusses on transforming rectangular datasets.
The dplyr
verbs and concepts covered in this chapter are also covered in this video by Garrett Grolemund (a co-author of R for Data Science with Hadley Wickham).
10.1 Set up
Load your packages first. This chapter just uses the packages contained in the tidyverse
:
The sa3_income
dataset will be used for all key examples in this chapter.10 It is a long dataset from the ABS that contains the average income and number of workers by Statistical Area 3, occupation and sex between 2011 and 2016.
If you haven’t already, download the sa3_income.csv
file to your own data
folder:
download.file(url = "https://raw.githubusercontent.com/grattan/R_at_Grattan/master/data/sa3_income.csv",
destfile = "data/sa3_income.csv")
Then read it using the read_csv
function:
## Parsed with column specification:
## cols(
## sa3 = col_double(),
## sa3_name = col_character(),
## sa3_sqkm = col_double(),
## sa3_income_percentile = col_double(),
## sa4_name = col_character(),
## gcc_name = col_character(),
## state = col_character(),
## occupation = col_character(),
## occ_short = col_character(),
## prof = col_character(),
## gender = col_character(),
## year = col_double(),
## median_income = col_double(),
## average_income = col_double(),
## total_income = col_double(),
## workers = col_double()
## )
## # A tibble: 6 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774
## 2 2012 Belconnen ACT Men 56724. 69435
## 3 2013 Belconnen ACT Men 58918. 69697
## 4 2014 Belconnen ACT Men 60525. 68613
## 5 2015 Belconnen ACT Men 60964. 63428
## 6 2016 Belconnen ACT Men 63389. 69828
10.2 The pipe: %>%
You will almost always want to perform more than one of the operations described below on your dataset. One way to perform multiple operations, one after the other, is to ‘nest’ them inside. This nesting will be painfully familiar to Excel users.
Consider an example of baking and eating a cake.11 You take the ingredients, combine them, then mix, then bake, and then eat them. In a nested formula, this process looks like:
In a nested formula, you need to start in the middle and work your way out. This means anyone reading your code – including you in the future! – needs to start in the middle and work their way out. But because we’re used to left-right reading, we’re not particularly good at naturally interpreting nested functions like this one.
This is where the ‘pipe’ can help. The pipe operator %>%
(keyboard shortcut: cmd + shift + m
) takes an argument on the left and ‘pipes’ it into the function on the right. Each time you see %>%
, you can read it as ‘and then’.
So the you could express the baking example as:
ingredients %>% # and then
combine() %>% # and then
mix() %>% # and then
bake() %>% # and then
eat() # yum!
Which reads as:
take the
ingredients
, thencombine
, thenmix
, thenbake
, theneat
them.
This does the same thing as eat(bake(mix(combine(ingredients))))
. But it’s much nicer and more natural to read, and to write.
Another example: the function paste
takes arguments and combines them together into a single string. So you could use the pipe to:
## [1] "hello dear reader"
which is the same as
## [1] "hello dear reader"
Or you could define a vector of numbers and pass12 them to the sum()
function:
## [1] 32
Or you could skip the intermediate step altogether:
## [1] 32
This is the same as:
## [1] 32
The benefits of piping become more clear when you want to perform a few sequential operations on a dataset. For example, you might want to filter
the observations in the sa3_income
data to only NSW
, before you group_by
gender
and summarise
the income
of these grops (these functions are explained in detail below). All of these functions take ‘data’ as the first argument, and are designed to be used with pipes.
Like the income differential it shows, writing this process as a nested function is outrageous and hard to read:
## # A tibble: 2 x 2
## gender av_mean_income
## <chr> <dbl>
## 1 Men 58202.
## 2 Women 41662.
The original common way to avoid this unseemly nesting in R
was to assign each ‘step’ its own object, which is definitely clearer:
data1 <- filter(sa3_income, state == "NSW")
data2 <- group_by(data1, gender)
data3 <- summarise(data2, av_mean_income = mean(income))
data3
## # A tibble: 2 x 2
## gender av_mean_income
## <chr> <dbl>
## 1 Men 58202.
## 2 Women 41662.
And using pipes make the steps clearer still:
- take the
sa3_income
data, and then %>% filter
it to only NSW, and then %>%group
it by gender, and then %>%summarise
it
sa3_income %>% # and then
filter(state == "NSW") %>% # and then
group_by(gender) %>% # and then
summarise(av_mean_income = mean(income))
## # A tibble: 2 x 2
## gender av_mean_income
## <chr> <dbl>
## 1 Men 58202.
## 2 Women 41662.
10.3 Select variables with select()
The select
function takes a dataset and keeps or drops variables (columns) that are specified.
For example, look at the variables that are in the sa3_income
dataset (using the names()
function):
## [1] "year" "sa3_name" "state" "gender" "income" "workers"
If you wanted to keep just the state
and income
variables, you could take the sa3_income
dataset and select just those variables:
## # A tibble: 4,019 x 2
## state income
## <chr> <dbl>
## 1 ACT 54105.
## 2 ACT 56724.
## 3 ACT 58918.
## 4 ACT 60525.
## 5 ACT 60964.
## 6 ACT 63389.
## 7 ACT 53139.
## 8 ACT 54515.
## 9 ACT 58132.
## 10 ACT 56247.
## # … with 4,009 more rows
Or you could use -
(minus) to remove the state
and sa3_name
variables:13
## # A tibble: 4,019 x 4
## year gender income workers
## <dbl> <chr> <dbl> <dbl>
## 1 2011 Men 54105. 67774
## 2 2012 Men 56724. 69435
## 3 2013 Men 58918. 69697
## 4 2014 Men 60525. 68613
## 5 2015 Men 60964. 63428
## 6 2016 Men 63389. 69828
## 7 2011 Men 53139. 666
## 8 2012 Men 54515. 647
## 9 2013 Men 58132. 641
## 10 2014 Men 56247. 561
## # … with 4,009 more rows
10.3.1 Selecting groups of variables
Sometimes it can be useful to keep or drop variables with names that have a certain characteristic; they begin with some text string, or end with one, or contain one, or have some other pattern altogether.
You can use patterns and ‘select helpers’14 from the Tidyverse to help deal with these sets of variables.
For example, if you want to keep just the SA3 and state variables – ie the variables that start with "s"
– you could:
## # A tibble: 4,019 x 2
## sa3_name state
## <chr> <chr>
## 1 Belconnen ACT
## 2 Belconnen ACT
## 3 Belconnen ACT
## 4 Belconnen ACT
## 5 Belconnen ACT
## 6 Belconnen ACT
## 7 Canberra East ACT
## 8 Canberra East ACT
## 9 Canberra East ACT
## 10 Canberra East ACT
## # … with 4,009 more rows
Or, instead, if you wanted to keep just the variables that contain "er"
, you could:
## # A tibble: 4,019 x 2
## gender workers
## <chr> <dbl>
## 1 Men 67774
## 2 Men 69435
## 3 Men 69697
## 4 Men 68613
## 5 Men 63428
## 6 Men 69828
## 7 Men 666
## 8 Men 647
## 9 Men 641
## 10 Men 561
## # … with 4,009 more rows
And if you wanted to keep both the "s"
variables and the "er"
variables, you could:
## # A tibble: 4,019 x 4
## sa3_name state gender workers
## <chr> <chr> <chr> <dbl>
## 1 Belconnen ACT Men 67774
## 2 Belconnen ACT Men 69435
## 3 Belconnen ACT Men 69697
## 4 Belconnen ACT Men 68613
## 5 Belconnen ACT Men 63428
## 6 Belconnen ACT Men 69828
## 7 Canberra East ACT Men 666
## 8 Canberra East ACT Men 647
## 9 Canberra East ACT Men 641
## 10 Canberra East ACT Men 561
## # … with 4,009 more rows
The full list of these handy select functions are provided with the ?tidyselect::select_helpers
documentation, listed below:
starts_with()
: Starts with a prefix.ends_with()
: Ends with a suffix.contains()
: Contains a literal string.matches()
: Matches a regular expression.num_range()
: Matches a numerical range like x01, x02, x03.one_of()
: Matches variable names in a character vector.everything()
: Matches all variables.last_col()
: Select last variable, possibly with an offset.
10.4 Filter with filter()
The filter
function takes a dataset and keeps observations (rows) that meet the conditions.
filter
has one required first argument – the data – and then as many ‘conditions’ as you want to provide.
10.4.1 Conditions; logical operations; TRUE
or FALSE
The conditions are logical operations, meaning they are a statement that return either TRUE
or FALSE
in the computer’s mind.15
We know, for instance, that 12
is equal to 12
and that 1 + 2
does not equal 12
. Which means if we type 12 == 12
or 1 + 2 == 12
into the console it should give FALSE
:
## [1] TRUE
## [1] FALSE
Or, we can see if 1 + 2
is equal 5
or 9
or 3
by providing a vector of those numbers:
## [1] FALSE FALSE TRUE
This works for character strings, too:
## [1] FALSE TRUE FALSE
A lot of what we do in ‘data science’ is based on these TRUE
and FALSE
conditions.
10.4.2 Filtering data with filter
Turning back to the sa3_income
data, if you just wanted to see observations people in NT
:
## # A tibble: 123 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2011 Alice Springs NT Men 52602. 23663
## 2 2012 Alice Springs NT Men 55050. 24065
## 3 2013 Alice Springs NT Men 57251. 24218
## 4 2014 Alice Springs NT Men 58403. 24566
## 5 2015 Alice Springs NT Men 60084. 24562
## 6 2016 Alice Springs NT Men 64330. 22048
## 7 2011 Barkly NT Men 50517. 2272
## 8 2012 Barkly NT Men 52474. 2321
## 9 2013 Barkly NT Men 55006. 2364
## 10 2014 Barkly NT Men 56543. 2234
## # … with 113 more rows
Or you might just want to look at high-income (income > 70,000
) areas from Victoria in 2015:
## # A tibble: 3 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2015 Bayside Vic Men 77175. 62460
## 2 2015 Stonnington - East Vic Men 70652. 27922
## 3 2015 Stonnington - West Vic Men 70234. 47597
Each of the commas in the filter
function represent an ‘and’ &
. So you can read the steps above as:
take the
sa3_income
data and filter to keep only the observations that are from Victoria,
and that have a average income above 70k,
and are from the year 2015.
Sometimes you might want to relax a little, keeping observations from one category or another. You can do this with the or symbol: |
16
sa3_income %>%
filter(state == "Vic" | state == "Tas",
income > 100000,
year == 2015 | year == 2016)
## # A tibble: 0 x 6
## # … with 6 variables: year <dbl>, sa3_name <chr>, state <chr>, gender <chr>,
## # income <dbl>, workers <dbl>
Which reads:
take the
sa3_income
data and filter to keep only the observations that are from Victoria OR NSW, and that have a average income above 100k, and are from the year 2015 OR 2016.
10.4.3 Grouped filtering with group_by()
The group_by
function groups a dataset by given variables. This effectively generates one dataset per group within your main dataset. Any function you then apply – like filter()
– will be applied to each of the grouped datasets.
For example, you could filter the sa3_income
dataset to keep just the observation with the highest average income:
## # A tibble: 1 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2015 West Pilbara WA Men 107844. 22928
To keep the observations that have the highest average incomes in each state, you can group_by
state, then filter
:17
## # A tibble: 8 x 6
## # Groups: state [8]
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2013 Molonglo ACT Men 92947. 227
## 2 2016 North Sydney - Mosman NSW Men 90668. 74702
## 3 2016 Christmas Island NT Men 84474. 621
## 4 2015 Gladstone Qld Men 97282. 48026
## 5 2015 Outback - North and East SA Men 71791. 15849
## 6 2016 West Coast Tas Men 58116. 11117
## 7 2016 Bayside Vic Men 78624. 64541
## 8 2015 West Pilbara WA Men 107844. 22928
From the description of the tibble above, you can learn that your data has 8 unique groups of state:
## # Groups: state [8]
Or you could keep the observations with the lowest average incomes in each state and year:18
## # A tibble: 48 x 6
## # Groups: state, year [48]
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2014 Cocos (Keeling) Islands NT Men 32652. 45
## 2 2011 Belconnen ACT Women 43235 22708
## 3 2014 Belconnen ACT Women 48399. 22750
## 4 2015 Belconnen ACT Women 48814. 20577
## 5 2016 Belconnen ACT Women 50756. 22982
## 6 2012 Gungahlin ACT Women 45241 13647
## 7 2013 North Canberra ACT Women 45844. 11965
## 8 2012 Great Lakes NSW Women 32590 4730
## 9 2015 Lord Howe Island NSW Women 34173. 75
## 10 2011 Lower Murray NSW Women 30800. 2122
## # … with 38 more rows
The dataset remains grouped after your function(s). To explicitly ‘ungroup’ your data, add the ungroup
function to your chain (the ‘Groups’ note has disappeared in the below):
## # A tibble: 48 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2014 Cocos (Keeling) Islands NT Men 32652. 45
## 2 2011 Belconnen ACT Women 43235 22708
## 3 2014 Belconnen ACT Women 48399. 22750
## 4 2015 Belconnen ACT Women 48814. 20577
## 5 2016 Belconnen ACT Women 50756. 22982
## 6 2012 Gungahlin ACT Women 45241 13647
## 7 2013 North Canberra ACT Women 45844. 11965
## 8 2012 Great Lakes NSW Women 32590 4730
## 9 2015 Lord Howe Island NSW Women 34173. 75
## 10 2011 Lower Murray NSW Women 30800. 2122
## # … with 38 more rows
10.5 Edit and add new variables with mutate()
To add new variables to your dataset, use the mutate
function. Like all dplyr
verbs, the first argument to mutate
is your data. Then define variables using a new_variable_name = x
format, where x
can be a single number or character string, or simple operation or function using current variables.
To add a new variable to the sa3_income
dataset that shows the log the number of workers:
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers log_workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774 11.1
## 2 2012 Belconnen ACT Men 56724. 69435 11.1
## 3 2013 Belconnen ACT Men 58918. 69697 11.2
## 4 2014 Belconnen ACT Men 60525. 68613 11.1
## 5 2015 Belconnen ACT Men 60964. 63428 11.1
## 6 2016 Belconnen ACT Men 63389. 69828 11.2
## 7 2011 Canberra East ACT Men 53139. 666 6.50
## 8 2012 Canberra East ACT Men 54515. 647 6.47
## 9 2013 Canberra East ACT Men 58132. 641 6.46
## 10 2014 Canberra East ACT Men 56247. 561 6.33
## # … with 4,009 more rows
To edit a variable, redefine it in mutate
. For example, if you wanted to take the last two digits of year:
## # A tibble: 4,019 x 6
## year sa3_name state gender income workers
## <int> <chr> <chr> <chr> <dbl> <dbl>
## 1 11 Belconnen ACT Men 54105. 67774
## 2 12 Belconnen ACT Men 56724. 69435
## 3 13 Belconnen ACT Men 58918. 69697
## 4 14 Belconnen ACT Men 60525. 68613
## 5 15 Belconnen ACT Men 60964. 63428
## 6 16 Belconnen ACT Men 63389. 69828
## 7 11 Canberra East ACT Men 53139. 666
## 8 12 Canberra East ACT Men 54515. 647
## 9 13 Canberra East ACT Men 58132. 641
## 10 14 Canberra East ACT Men 56247. 561
## # … with 4,009 more rows
10.5.1 Using if_else()
or case_when()
Sometimes you want to create a new variable based on some sort of condition. Like, if the number of workers in an sa3
is more than 2,000
, set the new many_workers
variable to TRUE
, and set it to FALSE
otherwise.
This kind of operation can be thought of as if_else
: if
(some condition), do this, otherwise do that.
That’s what the if_else()
function does. It takes three arguments: a condition, a value if that condition is true, and a value if that condition is false.
You can use the if_else()
function when you are creating new variables in a mutate
command:
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers many_workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 Many workers
## 2 2012 Belconnen ACT Men 56724. 69435 Many workers
## 3 2013 Belconnen ACT Men 58918. 69697 Many workers
## 4 2014 Belconnen ACT Men 60525. 68613 Many workers
## 5 2015 Belconnen ACT Men 60964. 63428 Many workers
## 6 2016 Belconnen ACT Men 63389. 69828 Many workers
## 7 2011 Canberra East ACT Men 53139. 666 Not many workers
## 8 2012 Canberra East ACT Men 54515. 647 Not many workers
## 9 2013 Canberra East ACT Men 58132. 641 Not many workers
## 10 2014 Canberra East ACT Men 56247. 561 Not many workers
## # … with 4,009 more rows
Which reads:
Take the
sa3_income
dataset, and then add a variable that says ‘Many workers’ if there are more than 2,000 workers, and ‘Not many workers’ if there are fewer-or-equal than 2,000 workers.
With the if_else
function, you take one conditional statement and return something based on that. But often you don’t want to be so binary; you want to do this if this is true, that if that is true, and the other if the other is true, etc.
This could be done by nesting if_else
statements:
sa3_income %>%
mutate(worker_group = if_else(workers > 2000, "More than 2000 workers",
if_else(workers > 1000, "1000-2000 workers",
if_else(workers > 500, "500-1000 workers",
"500 workers or less"))))
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers worker_group
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 More than 2000 workers
## 2 2012 Belconnen ACT Men 56724. 69435 More than 2000 workers
## 3 2013 Belconnen ACT Men 58918. 69697 More than 2000 workers
## 4 2014 Belconnen ACT Men 60525. 68613 More than 2000 workers
## 5 2015 Belconnen ACT Men 60964. 63428 More than 2000 workers
## 6 2016 Belconnen ACT Men 63389. 69828 More than 2000 workers
## 7 2011 Canberra East ACT Men 53139.<PHONE_NUMBER> workers
## 8 2012 Canberra East ACT Men 54515.<PHONE_NUMBER> workers
## 9 2013 Canberra East ACT Men 58132.<PHONE_NUMBER> workers
## 10 2014 Canberra East ACT Men 56247.<PHONE_NUMBER> workers
## # … with 4,009 more rows
But that syntax can be a bit difficult to read. You can do this in a clearer way using case_when
:
sa3_income %>%
mutate(worker_group = case_when(
workers > 20000 ~ "More than 20,000 workers",
workers > 10000 ~ "More than 10,000 workers",
workers > 5000 ~ "More than 5,000 workers",
workers <= 5000 ~ "5,000 or fewer workers"
))
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers worker_group
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 More than 20,000 workers
## 2 2012 Belconnen ACT Men 56724. 69435 More than 20,000 workers
## 3 2013 Belconnen ACT Men 58918. 69697 More than 20,000 workers
## 4 2014 Belconnen ACT Men 60525. 68613 More than 20,000 workers
## 5 2015 Belconnen ACT Men 60964. 63428 More than 20,000 workers
## 6 2016 Belconnen ACT Men 63389. 69828 More than 20,000 workers
## 7 2011 Canberra East ACT Men 53139. 666 5,000 or fewer workers
## 8 2012 Canberra East ACT Men 54515. 647 5,000 or fewer workers
## 9 2013 Canberra East ACT Men 58132. 641 5,000 or fewer workers
## 10 2014 Canberra East ACT Men 56247. 561 5,000 or fewer workers
## # … with 4,009 more rows
The case_when
function takes the first condition (LHS) and applies some value (RHS) if it is true. It then moves to the next condition, and so on. Once an observation has been classified – eg an observation has more than 20,000 workers – it is ignored in proceeding conditions.
Ending a case_when
statement with TRUE ~ [some value]
is a catch all, and will apply the RHS [some value]
to any observations that did not meet an explicit condition. For example, you could end the worker classification with:
sa3_income %>%
mutate(worker_group = case_when(
workers > 20000 ~ "More than 20,000 workers",
workers > 10000 ~ "More than 10,000 workers",
workers > 5000 ~ "More than 5,000 workers",
TRUE ~ "5,000 or fewer workers"
))
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers worker_group
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 More than 20,000 workers
## 2 2012 Belconnen ACT Men 56724. 69435 More than 20,000 workers
## 3 2013 Belconnen ACT Men 58918. 69697 More than 20,000 workers
## 4 2014 Belconnen ACT Men 60525. 68613 More than 20,000 workers
## 5 2015 Belconnen ACT Men 60964. 63428 More than 20,000 workers
## 6 2016 Belconnen ACT Men 63389. 69828 More than 20,000 workers
## 7 2011 Canberra East ACT Men 53139. 666 5,000 or fewer workers
## 8 2012 Canberra East ACT Men 54515. 647 5,000 or fewer workers
## 9 2013 Canberra East ACT Men 58132. 641 5,000 or fewer workers
## 10 2014 Canberra East ACT Men 56247. 561 5,000 or fewer workers
## # … with 4,009 more rows
Meaning, for any observation that did not have workers more than 20,000 or more than 10,000 or more than 5,000, assign the value "5,000 or fewer workers"
.
Observations that do not meet a condition will be set to NA
:
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers worker_group
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 <NA>
## 2 2012 Belconnen ACT Men 56724. 69435 <NA>
## 3 2013 Belconnen ACT Men 58918. 69697 <NA>
## 4 2014 Belconnen ACT Men 60525. 68613 <NA>
## 5 2015 Belconnen ACT Men 60964. 63428 <NA>
## 6 2016 Belconnen ACT Men 63389. 69828 <NA>
## 7 2011 Canberra East ACT Men 53139. 666 <NA>
## 8 2012 Canberra East ACT Men 54515. 647 <NA>
## 9 2013 Canberra East ACT Men 58132. 641 <NA>
## 10 2014 Canberra East ACT Men 56247. 561 <NA>
## # … with 4,009 more rows
Like any if
or if_else
, you can provide more than one condition to your conditional statement:
sa3_income %>%
mutate(women_group = case_when(
gender == "Women" & workers > 20000 ~ "More than 20,000 women",
gender == "Women" & workers > 10000 ~ "More than 10,000 women",
gender == "Women" & workers > 5000 ~ "More than 5,000 women",
gender == "Women" ~ "5,000 or fewer women",
TRUE ~ "Men"
))
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers women_group
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <chr>
## 1 2011 Belconnen ACT Men 54105. 67774 Men
## 2 2012 Belconnen ACT Men 56724. 69435 Men
## 3 2013 Belconnen ACT Men 58918. 69697 Men
## 4 2014 Belconnen ACT Men 60525. 68613 Men
## 5 2015 Belconnen ACT Men 60964. 63428 Men
## 6 2016 Belconnen ACT Men 63389. 69828 Men
## 7 2011 Canberra East ACT Men 53139. 666 Men
## 8 2012 Canberra East ACT Men 54515. 647 Men
## 9 2013 Canberra East ACT Men 58132. 641 Men
## 10 2014 Canberra East ACT Men 56247. 561 Men
## # … with 4,009 more rows
10.5.2 Grouped mutates with group_by()
Like filtering, you can add or edit variables on grouped data. For example, you could get the average number of workers in each SA3 over the 6 years:
## # A tibble: 4,019 x 7
## # Groups: sa3_name, gender [672]
## year sa3_name state gender income workers av_workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774 68129.
## 2 2012 Belconnen ACT Men 56724. 69435 68129.
## 3 2013 Belconnen ACT Men 58918. 69697 68129.
## 4 2014 Belconnen ACT Men 60525. 68613 68129.
## 5 2015 Belconnen ACT Men 60964. 63428 68129.
## 6 2016 Belconnen ACT Men 63389. 69828 68129.
## 7 2011 Canberra East ACT Men 53139. 666 641.
## 8 2012 Canberra East ACT Men 54515. 647 641.
## 9 2013 Canberra East ACT Men 58132. 641 641.
## 10 2014 Canberra East ACT Men 56247. 561 641.
## # … with 4,009 more rows
Above, the mean()
function is applied separately to each unique group of sa3_name
and gender
, taking one average for women in Queanbeyan, one average for men in Queanbeyan, and so on.
Grouping a dataset does not prohibit operations that don’t utilise the grouping. For example, you could get each year’s workers relative to the SA3/gender average in the same call to mutate
:
sa3_income %>%
group_by(sa3_name, gender) %>%
mutate(av_workers = mean(workers),
worker_diff = workers / av_workers)
## # A tibble: 4,019 x 8
## # Groups: sa3_name, gender [672]
## year sa3_name state gender income workers av_workers worker_diff
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774 68129. 0.995
## 2 2012 Belconnen ACT Men 56724. 69435 68129. 1.02
## 3 2013 Belconnen ACT Men 58918. 69697 68129. 1.02
## 4 2014 Belconnen ACT Men 60525. 68613 68129. 1.01
## 5 2015 Belconnen ACT Men 60964. 63428 68129. 0.931
## 6 2016 Belconnen ACT Men 63389. 69828 68129. 1.02
## 7 2011 Canberra East ACT Men 53139. 666 641. 1.04
## 8 2012 Canberra East ACT Men 54515. 647 641. 1.01
## 9 2013 Canberra East ACT Men 58132. 641 641. 1.00
## 10 2014 Canberra East ACT Men 56247. 561 641. 0.875
## # … with 4,009 more rows
See that the data remains grouped after the mutate
. You can explicitly ungroup()
afterwards:
sa3_income %>%
group_by(sa3_name, gender) %>%
mutate(av_workers = mean(workers),
worker_diff = workers / av_workers) %>%
ungroup()
## # A tibble: 4,019 x 8
## year sa3_name state gender income workers av_workers worker_diff
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774 68129. 0.995
## 2 2012 Belconnen ACT Men 56724. 69435 68129. 1.02
## 3 2013 Belconnen ACT Men 58918. 69697 68129. 1.02
## 4 2014 Belconnen ACT Men 60525. 68613 68129. 1.01
## 5 2015 Belconnen ACT Men 60964. 63428 68129. 0.931
## 6 2016 Belconnen ACT Men 63389. 69828 68129. 1.02
## 7 2011 Canberra East ACT Men 53139. 666 641. 1.04
## 8 2012 Canberra East ACT Men 54515. 647 641. 1.01
## 9 2013 Canberra East ACT Men 58132. 641 641. 1.00
## 10 2014 Canberra East ACT Men 56247. 561 641. 0.875
## # … with 4,009 more rows
10.6 Summarise data with summarise()
Summarising is a useful way to assess and present data. The summarise
function collapses your data down into a single row, performing the operation(s) you provide:
sa3_income %>%
summarise(mean_income = mean(income),
total_workers = sum(workers)) # this is a silly statistic
## # A tibble: 1 x 2
## mean_income total_workers
## <dbl> <dbl>
## 1 50272. 117002608
Summarising is usually only useful when combined with group_by
.
10.6.1 Grouped summaries with group_by()
Grouped summaries can help change the detail of your data. In the original sa3_income
data, there is a unique workers
observation for each year, SA3 and gender. If you wanted to aggregate that information up see the total number of workers for each year and SA3:
## # A tibble: 2,010 x 3
## # Groups: year [6]
## year sa3_name workers
## <dbl> <chr> <dbl>
## 1 2011 Adelaide City 18048
## 2 2011 Adelaide Hills 59794
## 3 2011 Albany 43811
## 4 2011 Albury 50490
## 5 2011 Alice Springs 31563
## 6 2011 Armadale 56088
## 7 2011 Armidale 27957
## 8 2011 Auburn 57298
## 9 2011 Augusta - Margaret River - Busselton 35852
## 10 2011 Bald Hills - Everton Park 36273
## # … with 2,000 more rows
After the summarise
function, the dataset grouping remains but is reduced by one – so the right-hand-side grouping is lost. This enables a common combination to find a proportion of a group. For example, if you
Common functions to use with summarise
Grouped summaries generate summary statistics for grouped data. It uses the same summarise
function, but is preceded with a group_by
. For example, if you want to find the average income for women and men:
## # A tibble: 2 x 2
## gender mean_income
## <chr> <dbl>
## 1 Men 58780.
## 2 Women 41760.
Or the total workers in each year and state by gender:
## # A tibble: 96 x 4
## # Groups: year, state [48]
## year state gender workers
## <dbl> <chr> <chr> <dbl>
## 1 2011 ACT Men 265281
## 2 2011 ACT Women 88632
## 3 2011 NSW Men 4438272
## 4 2011 NSW Women 1415914
## 5 2011 NT Men 140946
## 6 2011 NT Women 44413
## 7 2011 Qld Men 2859150
## 8 2011 Qld Women 918841
## 9 2011 SA Men 997160
## 10 2011 SA Women 325980
## # … with 86 more rows
10.7 Arrange with arrange()
‘doesn’t add or subtract to your data’
Sorting data in one way or another can be useful. Use the arrange
function to sort data by the provided variable(s). Like with select
, you can use the minus sign -
to reverse the order.
For example, to find the areas in 2016 with the least workers:
## # A tibble: 670 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2016 Lord Howe Island NSW Women 37944 74
## 2 2016 Urriarra - Namadgi ACT Women 86672. 90
## 3 2016 Christmas Island NT Women 57640 141
## 4 2016 Canberra East ACT Women 52091. 182
## 5 2016 Lord Howe Island NSW Men 40292 255
## 6 2016 Urriarra - Namadgi ACT Men 86747. 296
## 7 2016 Christmas Island NT Men 84474. 621
## 8 2016 Barkly NT Women 52552. 704
## 9 2016 Canberra East ACT Men 58035. 711
## 10 2016 Daly - Tiwi - West Arnhem NT Women 50096. 1075
## # … with 660 more rows
You can provide more than one variable. To sort the data first by state
and, within each state, by the most workers (ie sorting by negative workers):
## # A tibble: 670 x 6
## year sa3_name state gender income workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl>
## 1 2016 Belconnen ACT Men 63389. 69828
## 2 2016 Tuggeranong ACT Men 66921. 65248
## 3 2016 Gungahlin ACT Men 66714. 55176
## 4 2016 North Canberra ACT Men 62258. 37481
## 5 2016 Woden Valley ACT Men 66853. 24690
## 6 2016 Belconnen ACT Women 50756. 22982
## 7 2016 Tuggeranong ACT Women 52058. 21949
## 8 2016 South Canberra ACT Men 72437. 20998
## 9 2016 Gungahlin ACT Women 50908. 18134
## 10 2016 Weston Creek ACT Men 67242. 15500
## # … with 660 more rows
10.7.1 lead
and lag
functions with arrange
Having your data arranged in the way you want lets you use the lead
(looking forward) and lag
(looking backward) functions.
Both the lead
and lag
functions take a variable as their only requried argument. The default number of lags or leads is 1
, and this can be changed with the second argument. For example:
## # A tibble: 4,019 x 7
## year sa3_name state gender income workers last_workers
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 2011 Belconnen ACT Men 54105. 67774 NA
## 2 2012 Belconnen ACT Men 56724. 69435 67774
## 3 2013 Belconnen ACT Men 58918. 69697 69435
## 4 2014 Belconnen ACT Men 60525. 68613 69697
## 5 2015 Belconnen ACT Men 60964. 63428 68613
## 6 2016 Belconnen ACT Men 63389. 69828 63428
## 7 2011 Canberra East ACT Men 53139. 666 69828
## 8 2012 Canberra East ACT Men 54515. 647 666
## 9 2013 Canberra East ACT Men 58132. 641 647
## 10 2014 Canberra East ACT Men 56247. 561 641
## # … with 4,009 more rows
If you wanted to see the growth rate of income over time, you could arrange
then group_by
your data before creating an income_growth
variable that is income / lag(income)
.
sa3_income %>%
arrange(sa3_name, gender, year) %>%
group_by(sa3_name, gender) %>%
mutate(income_growth = income / lag(income) - 1)
## # A tibble: 4,019 x 7
## # Groups: sa3_name, gender [672]
## year sa3_name state gender income workers income_growth
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl>
## 1 2011 Adelaide City SA Men 48760. 13737 NA
## 2 2012 Adelaide City SA Men 49974. 13730 0.0249
## 3 2013 Adelaide City SA Men 52975. 13955 0.0601
## 4 2014 Adelaide City SA Men 54818. 13782 0.0348
## 5 2015 Adelaide City SA Men 54185. 13930 -0.0115
## 6 2016 Adelaide City SA Men 56689. 15300 0.0462
## 7 2011 Adelaide City SA Women 38359. 4311 NA
## 8 2012 Adelaide City SA Women 40409. 4219 0.0534
## 9 2013 Adelaide City SA Women 41287. 4281 0.0217
## 10 2014 Adelaide City SA Women 42872 4200 0.0384
## # … with 4,009 more rows
10.8 Putting it all together
You will often use a combination of the above dplyr
functions to get your data into shape.
For example, say you want to get the total workers and total income in each state and year by gender. You could start with the sa3_income
dataset, and then filter to year 2016, then create a new variable equal to workers * income
, then group by year, state and gender before you summarise to get the statistics you want. With pipes, it could look something like:
sa3_income %>%
filter(year == 2016) %>%
mutate(total_income = workers * income) %>%
group_by(year, state, gender) %>%
summarise(total_workers = sum(workers),
mean_income = mean(income),
total_income = sum(total_income))
## # A tibble: 16 x 6
## # Groups: year, state [8]
## year state gender total_workers mean_income total_income
## <dbl> <chr> <chr> <dbl> <dbl> <dbl>
## 1 2016 ACT Men<PHONE_NUMBER>1.<PHONE_NUMBER>7.
## 2 2016 ACT Women 97565 58222.<PHONE_NUMBER>.
## 3 2016 NSW Men 4952353 62207.<PHONE_NUMBER>37
## 4 2016 NSW Women 1575308 45003.<PHONE_NUMBER>9.
## 5 2016 NT Men<PHONE_NUMBER>1.<PHONE_NUMBER>1.
## 6 2016 NT Women 48107 54143.<PHONE_NUMBER>.
## 7 2016 Qld Men 3110067 61794.<PHONE_NUMBER>12.
## 8 2016 Qld Women<PHONE_NUMBER>1.<PHONE_NUMBER>6.
## 9 2016 SA Men 1041747 58602.<PHONE_NUMBER>1.
## 10 2016 SA Women<PHONE_NUMBER>4.<PHONE_NUMBER>2
## 11 2016 Tas Men<PHONE_NUMBER>7.<PHONE_NUMBER>0.
## 12 2016 Tas Women<PHONE_NUMBER>5.<PHONE_NUMBER>.
## 13 2016 Vic Men 3926751 59814.<PHONE_NUMBER>49.
## 14 2016 Vic Women 1264225 42816.<PHONE_NUMBER>3.
## 15 2016 WA Men 1756314 72582.<PHONE_NUMBER>29.
## 16 2016 WA Women<PHONE_NUMBER>7.<PHONE_NUMBER>0.
Or say you want to see the annual growth rate of female workers in the SA3 with the highest female income. You could filter to keep women, and then group by SA3, then get the highest income for each of SA3, then ungroup and filter to keep only the SA3 with the highest income, then arrange by year and get the annual worker growth:
sa3_income %>%
filter(gender == "Women") %>%
group_by(sa3_name) %>%
mutate(highest_income = max(income)) %>%
ungroup() %>%
filter(highest_income == max(highest_income)) %>%
arrange(year) %>%
mutate(worker_growth = workers / lag(workers) - 1)
## # A tibble: 6 x 8
## year sa3_name state gender income workers highest_income worker_growth
## <dbl> <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 2011 Urriarra - Nam… ACT Women 48525. 84 86672. NA
## 2 2012 Urriarra - Nam… ACT Women 51648. 96 86672. 0.143
## 3 2013 Urriarra - Nam… ACT Women 61858. 124 86672. 0.292
## 4 2014 Urriarra - Nam… ACT Women 72980. 99 86672. -0.202
## 5 2015 Urriarra - Nam… ACT Women 68534. 72 86672. -0.273
## 6 2016 Urriarra - Nam… ACT Women 86672. 90 86672. 0.25
10.9 Joining datasets with left_join()
Joining one dataset with another is incredibly useful and can be a difficult concept to grasp. The concept of joining one dataset to another is well introduced in Chapter 13 of R for Data Science:
It’s rare that a data analysis involves only a single table of data. Typically you have many tables of data, and you must combine them to answer the questions that you’re interested in. Collectively, multiple tables of data are called relational data because it is the relations, not just the individual datasets, that are important.
The dplyr
package ‘Join two tbls together’ page provides a comprehensive summary of all join types. We will explore the key use of joins in our line of work – left_join
– below.
A ‘left’ join takes your main dataset, and adds variables from a new dataset based on a matching condition that’s unhelpful, fix. If an observation in the new dataset is not found in the main dataset, it is ignored.
It is probably easier to show this with an example. Say that we had the income percentiles of each SA3 in each year from a different data source:
## Parsed with column specification:
## cols(
## sa3_name = col_character(),
## year = col_double(),
## sa3_income_percentile = col_double()
## )
This is a tidied version of the ABS Employee income by occupation and sex, 2010-11 to 2016-16 dataset.↩
XXX cannot remember the source for this example; probably Hadley? Jenny Bryan? Maybe somenone else?↩
‘pass’ can also be used to mean ‘pipe’.↩
This is the same as keeping everything except the
state
andsa3_name
variables.↩Explained in useful detail by the Tidyverse people at https://tidyselect.r-lib.org/reference/select_helpers.html↩
Computers’ mind, more likely.↩
On the keyboard:
shift
+backslash
↩Wow they are all men!↩
Wow they are all women!↩


|
|
| Type (chstr name) |
| |
|
| Type (chstr name, unsigned int value) |
| |
|
Type | next () const |
| |
|
Type | previous () const |
| |
|
bool | operator< (const Type &other) const |
| |
|
bool | operator> (const Type &other) const |
| |
|
bool | operator<= (const Type &other) const |
| |
|
bool | operator>= (const Type &other) const |
| |
|
bool | operator== (const Type &other) const |
| |
|
bool | operator!= (const Type &other) const |
| |
|
|
static int | getCount () |
| |
|
static hltypes::Array< Type > | getValues () |
| |
|
static hstr | getEnumName () |
| |
|
static hltypes::Map< unsigned int, hstr > | getAsMap () |
| |
|
static Type | fromInt (int value) |
| |
|
static Type | fromUint (unsigned int value) |
| |
|
static Type | fromName (chstr name) |
| |
|
static bool | hasValue (chstr name) |
| |
|
static bool | hasValueFor (int value) |
| |
|
static bool | hasValueFor (unsigned int value) |
| |
|
|
static const Type | Escape |
| |
|
static const Type | Font |
| |
|
static const Type | Icon |
| |
|
static const Type | Color |
| |
|
static const Type | Scale |
| |
|
static const Type | NoEffect |
| |
|
static const Type | Shadow |
| |
|
static const Type | Border |
| |
|
static const Type | StrikeThrough |
| |
|
static const Type | Underline |
| |
|
static const Type | Italic |
| |
|
static const Type | Hide |
| |
|
static const Type | IgnoreFormatting |
| |
|
static const Type | Close |
| |
|
static const Type | CloseConsume |
| |
|
|
| Type (unsigned int value) |
| |
|
hltypes::Map< unsigned int, hstr > & | _getInstances () const |
| |
The documentation for this class was generated from the following file:


1847 Census Occupation Data Prototypes
Toggle between the 3 prototypes:
Prototype 1
Prototype 2
Prototype 3
Tabulated Data
ID
Last Name
First Name
Residence Street
Male Occupation 1
Female Occupation 1
Ward


Under Construction
Our engineer is still working hard to
deliver the best space experience.
Please come back later.
Proudly Hosted by DomaiNesia


Welcome to the page!!!
I hope you enjoy
My name is Nasif and i am 15 years old
I am learning HTML
it's not hard


--- layout: post permalink: alessi.html Tags: Array NameOfCoupon: Alessi Cash Back Summary: See our Alessi Coupon Code, Alessi Rebate, Alessi Promo Code, Alessi Discount, Alessi Coupon Cash Back Rebate here. MetaKeywords: Alessi Coupon Code, Alessi Rebate, Alessi Promo Code, Alessi Cash Back Rebate, Alessi Discount, Alessi Coupon Image: LinkURL: http://click.linksynergy.com/fs-bin/click?id=6ZrgceZgf7E&offerid=168480.10000011&type=3 affiliate_url_link: http://www.icouponblog.com/jane.php?id=4366 thesis_readmore: See the Alessi Coupon thesis_thumb: http://www.icouponblog.com/tn.php?merchantURL=alessi-shop.com Title: Save 4.5% - Alessi Coupon Code for alessi-shop.com. Alessi Rebate & Discount thesis_description: Our Alessi Discount Coupon Code for alessi-shop.com. We have the best Alessi Rebate anywhere. thesis_keywords: Alessi, Alessi Coupon Code, Alessi Discount, Alessi Rebate product_name: Alessi hop_url: http://www.icouponblog.com/hop.php?id=4366 ---
www.alessi-shop.com is an amazing website to shop from. They carry over 3000 products created by internationally renowned designers. Alessi is an icon of Italian design world over. You can buy beautiful, practical, attractive, useful products for your kitchen and bathroom. You can also purchase home accessories, electrical appliances and costume jewelry. Buy designer items at affordable prices. Each item is unique and beautiful. Use our discount coupons to get a great price on all items. At www.alessi-shop.com you can buy many products for your kitchen. You can get coffee makers, cutlery sets, kettles, and espresso machines, electric kettles, made by designers from around the world. You can also buy children’s cutlery sets that are especially designed for children’s safety and comfort. There are beautiful glass sets, clocks and home accessories including mirrors, candle sets, photo frames and lots of other items. They are made exclusively for Alessi by designers from around the world.
www.alessi-shop.com is an exciting place to buy items that you use in the kitchen or bathroom. All items are unique and beautiful; they are designed exclusively for Alessi by Italian and other designers around the world. You can also buy costume jewelry, cell phone charms, key rings, bracelets, pendants, banknotes pins with Alessi design. Get the best prices on the best products. Save money by using our discount coupons. Check it out today. www.alessi-shop.com user friendly website will help you find everything you need and more.


W͒⋮Aͣ
Register&Authn
Credentials
Tools
Response Decoder
Settings
Option Setting
v{{ app_version }}
WebAuthn practice
WebAuthn/FIDO2 Testing Platform
×
×
Registration
Username
Display Name
Register with WebAuthn
Authentication
Username
Authenticate with WebAuthn
Log
Registered Users
check_box_outline_blank
#
User Name
Display Name
Cred ID
arrow_back
Back to Users List
User Detail
#
User Name
Display Name
UKey
Credential ID
Count
RP ID
Icon URL
Public Key
Creation Option
Authenticator Response
Authenticator Response (Decoded)
arrow_back
Back to Users List
Response Decoder
Authenticator Response
Decode as Attestation Response
Decode as Assertion Response
Result
Save changes
Options Setting
Attestation
Assertion
Attestation Conveyance Preference
(not set)
direct
indirect
none
User Verification
(not set)
required
preferred
discouraged
Require ResidentKey
(not set)
true
false
Authenticator Attachment
(not set)
platform
cross-platform
Enable ExcludeCredentials
clear
ExcludeCredentials UserName(id)
clear
ExcludeCredentials transports
usb
nfc
ble
internal
Extentions for Attestation
Format is "Name=Value". Each option is per row.
Enable AllowCredentials
clear
AllowCredentials UserName(id)
clear
AllowCredentials transports
usb
nfc
ble
internal
Extentions for Assertion
Format is "Name=Value". Each option is per row.
Save changes


{% extends "layout.html"%} {% block body%}
Engine:{{redemo["engine_cc"][0]}}
No of Cylinders:{{redemo["no_of_cylinders"][0]}}
Power:{{redemo["power"][0]}}
Fuel tank:{{redemo["fuel_tank_capacity"][0]}}
Rs.{{redemo["price"][0]}}
Flow rate:{{redemo["flow_rate"][0]}}
Availablity: In Stock


Calculations example with flask
Resize this responsive page to see the effect!
Available calculations
Area calculation
Square calculation
Cube calculation


I build and maintain the technology needed to power the components which enable the user-facing side of the website to exist. The back end code adds utility to everything the front-end developer creates. I'm responsible for creating, maintaining, testing, and debugging the entire back end. This includes the core application logic, databases, data and application integration, API and other back end processes.
Digital Hub.
AWS backend developer.
September 2017 – November 2019
Database department.
Head of database department.
September 2017 – August 2019
Web development department.
Web developer.
November 2016 – September 2017
Innovation and systems coordination.
Database administrator.
January 2014 – November 2016
Systems department.
Application developer and database administrator.
January 2008 – December 2013


Drake launches cannabis brand
Ottawa - Rapper Drake has partnered with Canopy Growth on a new cannabis venture, the company announced on Thursday, expanding the Canadian pot giant's list of celebrity backers.
He joins Seth Rogen, Evan Goldberg, Martha Stewart and Snoop Dogg in partnering with Canopy Growth.
The new company, More Life Growth Company, will be based in Drake's hometown of Toronto.
Drake will own 60 percent of the company, while Canopy will retain a 40-percent stake and distribution rights, and run its day-to-day operations.
"Drake's perspective as a culture leader and entrepreneur combined with Canopy Growth's breadth of cannabis knowledge will allow our new company to bring an unmatched cannabis experience to global markets," Canopy chief executive Mark Zekulin said in a statement.
For his part, Drake called the business venture "really exciting".
"The idea of being able to build something special in an industry that is ever growing has been inspiring. More Life and More Blessing," he said.
Comedians Rogen and Goldberg (Superbad, Pineapple Express and This is the End) launched Houseplant in March with Canopy Growth.
Rapper Snoop Dogg partnered with the Smith Falls, Ontario company in 2016 to create the brand Leafs by Snoop or LBS, while his friend and lifestyle guru Martha Stewart plays an advisory role in the development of its CBD or cannabidiol products.


{% extends "base.html" %} {% block title %}Philip Kiely: Sitemap{% endblock %} {% block content %}
Maybe you were looking for one of these:


Bit 31 Sign Bit 0: + 1: -
Bits 30 - 23 Exponent Field Decimal value of exponent field and exponent - 127 =
Bits 22 - 0 Significand Decimal value of the significand
Bit 63 Sign Bit 0: + 1: -
Bits 62 - 52 Exponent Field Decimal value of exponent field and exponent - 1023 =
Bits 51 - 0 Significand Decimal value of the significand


Models: Getting Started
- Introduction
- Retrieving Models
- Retrieving Single Models
- Creating & Updating Models
- Restoring Deleted Models
- Attributes
- Deleting Models
- Comparing Models
- Query Scopes
- Events
- Serialization
Introduction
The LdapRecord ORM provides a beautiful and simple ActiveRecord implementation for working with your LDAP server. Each "Model" represents a type of LDAP object that resides in your directory.
Models allow you query your directory as well as create, update, and delete records.
Before getting started, ensure you've added at least one connection to the container.
By default, there are models included with LdapRecord for popular LDAP directories (namely Active Directory & OpenLDAP) so you can get up and running as fast as possible. More on this below.
Defining Models
To get started, you must create a new class that represents the LDAP object you would like to query.
For example, let's create a model that represents Active Directory users:
<?php
use LdapRecord\Models\Model;
class User extends Model
{
public static $objectClasses = [
'top',
'person',
'organizationalperson',
'user',
];
}
As you can see above, we must add a public static property that contains the object classes of the LDAP record.
These object classes are used to locate the proper objects in your LDAP directory.
If you do not provide any object classes, global directory searches will be performed when retrieving models.
Predefined Models
LdapRecord comes with many predefined models that allow you to get started right away.
You may extend these built-in models and add your own methods & functionality, as well as override built-in functionality.
Entry Model
Use the LdapRecord\Models\Entry
model for retrieving all objects from your directory - regardless of type.
Active Directory Models
Each below model references a type of object in Active Directory.
| Model |
|---|
LdapRecord\Models\ActiveDirectory\Entry |
LdapRecord\Models\ActiveDirectory\User |
LdapRecord\Models\ActiveDirectory\Group |
LdapRecord\Models\ActiveDirectory\Computer |
LdapRecord\Models\ActiveDirectory\Contact |
LdapRecord\Models\ActiveDirectory\Container |
LdapRecord\Models\ActiveDirectory\OrganizationalUnit |
LdapRecord\Models\ActiveDirectory\Printer |
LdapRecord\Models\ActiveDirectory\ForeignSecurityPrincipal |
OpenLDAP Models
| Model |
|---|
LdapRecord\Models\OpenLdap\Entry |
LdapRecord\Models\OpenLdap\User |
LdapRecord\Models\OpenLdap\Group |
LdapRecord\Models\OpenLdap\OrganizationalUnit |
Don't see a model for the LDAP server you're using? Create a pull request!
Connections
By default, all models you create will try to use your default
LDAP connection that resides in the connection
container. To set your model to use an alternate connection,
define a $connection
property equal to the name of your other connection.
<?php
use LdapRecord\Models\Model;
class User extends Model
{
protected $connection = 'domain-b';
}
Distinguished Names
To get an objects full distinguished name call the getDn
method:
$user = User::find('cn=user,dc=local,dc=com');
// Returns 'cn=user,dc=local,dc=com'
$user->getDn();
To get an objects relative distinguished name, call the getRdn
method:
$user = User::find('cn=user,dc=local,dc=com');
// Returns 'cn=user'
$user->getRdn();
To get an objects parent distinguished name, call the getParentDn
method:
$user = User::find('cn=user,dc=local,dc=com');
// Returns 'dc=local,dc=com'
$user->getParentDn();
To get an objects name, call the getName
method:
$user = User::find('cn=user,dc=local,dc=com');
// Returns 'user'
$user->getName();
Object GUIDs
To retrieve a models Object GUID (globally unique identifier) call the getConvertedGuid
method.
This method will return the string variant of your models GUID. Some LDAP directories (namely Active Directory) use hexadecimal byte arrays to store these, so conversion is necessary.
$user = User::find('cn=user,dc=local,dc=com');
$user->getConvertedGuid();
To retrieve the raw GUID value, use the getObjectGuid
method.
By default, LdapRecord models will use the objectguid
attribute in the
above methods. If your directory stores GUIDs in a different attribute,
define a $guidKey
attribute inside of your model:
class User extends Model
{
protected $guidKey = 'entryuuid';
}
Default Attribute Values
If you would like to define the default values for some of your model's attributes,
you may define an $attributes
property on your model. This helps you to assign
static default values when creating objects in your directory:
Due to LDAP's multi-valued nature, each attribute value you define must be an array, regardless if it is single-valued or or multi-valued.
class User extends Model
{
protected $attributes = [
'company' => ['Acme'],
'description' => ['User Account'],
'manager' => ['cn=John Doe,dc=acme,dc=org']
];
}
Retrieving Models
Once you've created an LdapRecord model you're ready to start retrieving data from your directory. If you've used Laravel's Eloquent ORM, you'll feel right at home.
You can think of a model as a powerful query builder allowing you to query your directory for objects fluently and easily.
<?php
$users = User::get();
foreach ($users as $user) {
$user->getFirstAttribute('cn');
}
Adding Constraints
Each model serves as a query builder for the object classes you've defined inside.
You can add constraints to your queries and then call get()
to retrieve the
results.
<?php
$users = User::whereStartsWith('cn', 'John')
->whereEndsWith('sn', 'Doe')
->limit(10)
->get();
Since models are query builders, it's a good idea to review the query builder methods so you can utilize them to their full potential.
Model Constraints
Models come with some built in constraint methods that you may find useful.
The below constraints will only retrieve the models that are equal to the type you have retrieved. For example, retrieving the descendants of an organizational unit will only return organizational units that are direct descendants.
If you would like to avoid this, use the default
LdapRecord\Models\Entry
model, which provides noobjectclass
constraints on queries.
Ancestors
To retrieve the direct ancestors of a model, call the ancestors()
constraint on a retrieved model:
$ou = OrganizationalUnit::find('ou=Accountants,ou=Users,dc=local,dc=com');
$ancestors = $ou->ancestors()->get();
The above example will execute a listing
on your LDAP directory in
the distinguished name dc=local,dc=com
. This effectively pulls
the ancestors of the model.
Siblings
To retrieve the siblings of a model, call the siblings()
constraint on a retrieved model:
$ou = OrganizationalUnit::find('ou=Accountants,ou=Users,dc=local,dc=com');
$siblings = $ou->siblings()->get();
The above example will execute a listing
on your LDAP directory in
the distinguished name ou=Users,dc=local,dc=com
. This effectively
pulls the siblings of the model. The current model will also be
included in the resulting collection.
Descendants
To retrieve the descendants of a model, call the descendants()
constraint on a retrieved model:
$ou = OrganizationalUnit::find('ou=Accountants,ou=Users,dc=local,dc=com');
$descendants = $ou->descendants()->get();
The above example will execute a listing
on your LDAP directory in
the distinguished name ou=Accountants,ou=Users,dc=local,dc=com
. This effectively
pulls the descendants of the model.
Refreshing Models
To re-retrieve a new model from your directory, call the fresh()
method.
Doing so will not affect the existing instance you already have:
$user = User::where('cn', '=', 'jdoe')->first();
$freshUser = $user->fresh();
If you would like to re-retrieve the existing model, call the synchronize()
method.
This will re-retrieve the models attributes from the directory:
$user = User::where('cn', '=', 'jdoe')->first();
$user->synchronize();
Collections
When you query your models, returned results will be contained inside of a
LdapRecord\Query\Collection
. The Collection
class directly extends
Laravel's collection. Be sure to check out its documentation
for all of the available helpful methods.
<?php
$users = User::get();
$usersWithEmail = $users->filter(function (User $user) {
return $user->hasAttribute('mail');
});
Retrieving Single Models
If you would like to retrieve a single model from your directory, you can utilize a variety of methods. Here is a list and usage of each:
| Method |
|---|
first() |
find($distinguishedName) |
findBy($attributeName, $attributeValue) |
findByAnr($attributeValue) |
findByGuid($objectGuid) |
// Retrieve the first model of a global LDAP search...
$user = User::first();
// Retrieve a model by its distinguished name...
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Retrieve the first model that matches the attribute...
$user = User::findBy('cn', 'John Doe');
// Retrieve the first model that matches an array of ANR attributes...
$user = User::findByAnr('John Doe');
// Retrieve a model by its object guid...
$user = User::findByGuid('bf9679e7-0de6-11d0-a285-00aa003049e2');
Not Found Exceptions
Occasionally you may want to throw an exception if a specific record you're looking
for cannot be found on your directory. You can substitute the above methods
with OrFail()
variants:
| Method |
|---|
firstOrFail() |
findOrFail($distinguishedName) |
findByOrFail($attributeName, $attributeValue) |
findByAnrOrFail($attributeValue) |
findByGuidOrFail($objectGuid) |
try {
// Retrieve the first model of a global LDAP search or fail...
$user = User::firstOrFail();
// Retrieve a model by its distinguished name or fail...
$user = User::findOrFail('cn=John Doe,dc=acme,dc=org');
// Retrieve the first model that matches the attribute or fail...
$user = User::findByOrFail('cn', 'John Doe');
// Retrieve the first model that matches an array of ANR attributes or fail...
$user = User::findByAnrOrFail('John Doe');
// Retrieve a model by its object guid or fail...
$user = User::findByGuidOrFail('bf9679e7-0de6-11d0-a285-00aa003049e2');
} catch (\LdapRecord\Models\ModelNotFoundException $e) {
// One of the models could not be located...
}
Creating & Updating Models
Creating
Before we begin it is paramount to know that LDAP objects require a Distinguished Name to be
created successfully in your LDAP directory. LdapRecord will always attempt to generate a
Distinguished Name for models that do not have one upon save
. In addition, some LDAP
objects require more attributes to be set for successful creation.
For example, to create a User
object in Active Directory, the cn
(Common Name) attribute is required.
If you do not set this attribute, an exception will be thrown upon saving your LDAP model. For another
example, OrganizationlUnit
's must have the ou
attribute set.
LdapRecord cannot validate this for you as LDAP objects differ widely in their attribute requirements.
To create a new record in your directory, create a new model instance and call the save()
method.
Upon calling save()
, if no Distinguished Name is set on a new model, one will be generated
based on your configured base_dn
that you have set inside your connections configuration:
$conn = new Connection([
// ...
'base_dn' => 'dc=local,dc=com',
]);
$conn->connect();
$user = new User();
$user->cn = 'John Doe';
// User will be saved with the DN: 'cn=John Doe,dc=local,dc=com
$user->save();
Dynamic Distinguished Name Generation
LdapRecord generates a models distinguished name via the model method getCreatableRdn
.
This method is responsible for generating the "Relative Distinguished Name" which is
the true name of the object inside of your LDAP directory that does not include
your base Distinguished Name.
Since most LDAP objects require a Common Name (cn
) this is defaulted to:
/**
* Get a creatable RDN for the model.
*
* @return string
*/
public function getCreatableRdn()
{
$name = $this->escape($this->getFirstAttribute('cn'))->dn();
return "cn=$name";
}
As you can see above, the attribute is escaped before being passed into the RDN string. You must do this, otherwise if commas or other reserved characters are inside the attribute you are using, it will generate a malformed distinguished name.
You may override this method to allow your models Distinguished Name's to be dynamically generated
rather than creating them yourself manually. For example, here is how we would set the Relative
Distinguished Name (RDN) for an Active Directory OrganizationalUnit
model:
public function getCreatableRdn()
{
$name = $this->escape($this->getFirstAttribute('ou'))->dn();
return "ou=$name";
}
This then gets prepended onto your connections configured base_dn
, for a resulting "Full" Distinguished Name:
ou=MyOrganizationalUnitName,dc=local,dc=com
You may set the base DN of where you would like the object to be created inside by
using the inside()
method, rather than your base_dn
from your configuration:
$user = new User(['cn' => 'John Doe']);
$user->inside('ou=Users,dc=local,dc=com');
$user->save();
You may also pass in an LdapRecord Model
instance. This is convenient so you
know the container / organizational unit distinguished name is valid:
$ou = OrganizationalUnit::findOrFail('ou=Users,dc=acme,dc=org');
$user = new User(['cn' => 'John Doe']);
$user->inside($ou)->save();
The above examples will save the user inside the Users
OU resulting in the full distinguished name:
cn=John Doe,ou=Users,dc=acme,dc=org
Setting A Distinguished Name
To set the models distinguished name, call the setDn()
method on your model and populate it
with any organization unit or container that you would like it to be created inside:
$user = new User();
$user->cn = 'John Doe';
$user->setDn('cn=John Doe,ou=Users,dc=acme,dc=org');
$user->save();
Updating
Updating models is as easy as creating them. When you have a model returned from a query,
set its attributes as you would for creating and call the save()
method:
$user = User::first();
$user->company = 'My Company';
$user->samaccountname = 'jdoe';
$user->department = 'Accounting';
$user->displayname = 'Johnathan Doe';
$user->save();
Moving
To move existing models into Organizational Units or Containers, call the move()
method:
When moving a model is successful, the users distinguished name will be automatically updated to reflect its new location in your directory, so you may continue to run operations on it during the same request.
$user = User::find('cn=Steve Bauman,dc=local,dc=com');
$ou = OrganizationalUnit::find('ou=Office Users,dc=local,dc=com');
$user->move($ou);
// Displays 'cn=Steve Bauman,ou=Office Users,dc=local,d=com'
echo $user->getDn();
Renaming
To rename existing models, call the rename()
method and supply the new objects RDN (relative distinguished name):
When renaming is successful, the users distinguished name is automatically updated to reflect its new name in the directory, so you may run further operations on it during the same request.
$user = User::find('cn=John Doe,dc=local,dc=com');
$user->rename('cn=Jane Doe');
// Displays 'cn=Jane Doe,dc=local,dc=com'
echo $user->getDn();
Restoring Deleted Models
Important: This feature is only possible when connecting to an Active Directory server.
To restore a deleted object, we must first query the directory
for deleted objects by using the whereDeleted
method:
use LdapRecord\LdapRecordException;
use LdapRecord\Models\ActiveDirectory\User;
$user = User::whereDeleted()
->where('mail', '='<EMAIL_ADDRESS>->first();
try {
$user->restore();
// Successfully restored user.
} catch (LdapRecordException $ex) {
// Failed restoring user.
}
If you're including deleted results in your queries using the withDeleted
clause,
you can call the isDeleted
method to check if an object has been deleted:
$users = User::withDeleted()->get();
foreach ($users as $user) {
if ($user->isDeleted()) {
$user->restore();
}
}
If you call restore
on a non-deleted object, it will simply return false
:
$user = User::where('cn', '=', 'Steve Bauman')->first();
$result = $user->restore();
// Displays bool(false)
var_dump($result);
Attributes
Methods
There are several built-in methods on models you may like to utilize:
Model::getAttributes()
The getAttributes
method returns all of the values on the model:
$user = User::first();
$attributes = $user->getAttributes();
foreach ($attributes as $name => $values) {
//
}
In the above example,
$values
will always be an array.
Model::getAttribute($name)
The getAttribute
method returns all of the values inside the given key. This will return an array
if the attribute exists:
$group = Group::first();
$members = $group->getAttribute('member');
if ($members) {
foreach ($members as $member) {
echo $member;
}
}
Model::getFirstAttribute($name)
The getFirstAttribute
method returns the first value of the given key. This will always return null
or string
:
$group = Group::first();
$firstMember = $group->getFirstAttribute('member');
Model::hasAttribute()
The hasAttribute
method determines whether the model contains the key in the models attributes:
$user = User::first();
if ($user->hasAttribute('company')) {
//
}
Model::addAttributeValue($name, $value)
To add a value to an attribute without clearing it, use the addAttributeValue
method:
$user = User::first();
$user->addAttributeValue('proxyaddresses', 'SMTP:sbauman@local.com');
Model::countAttributes()
The countAttributes
method returns the number of attributes the model contains:
$user = User::first();
echo $user->countAttributes();
Array Conversion
Attributes you retrieve from an LdapRecord model will always return and array. This is due to LDAP's multi-valued nature.
For example, if you would like to retrieve the users mail
attribute, you must request the first key from it:
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Get the users email address.
echo $user->mail[0];
However, the above will cause an exception if the attribute does not exist.
To work around this, we can use the getFirstAttribute()
method:
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Get the users email address.
echo $user->getFirstAttribute('mail');
When setting attributes on models, they will automatically be converted to an array for you if you do not provide one.
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Both approaches will set the attribute identically:
$user->mail =<EMAIL_ADDRESS>$user->mail =<EMAIL_ADDRESS>Similarly, you can use the setFirstAttribute()
method to
set the attributes first value in its array, even if
it does not currently exist on the model:
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Set the users email address.
$user->setFirstAttribute('mail', 'jdoe@acme.org');
Determining Attribute Existence
To check if a model has an attribute, you can use the hasAttribute()
method:
if ($user->hasAttribute('mail')) {
// This user has an email address.
}
As with all other attribute methods, this check is case-insensitive. You may pass any type of casing of the attribute you are looking for:
// Both will return 'true':
$user->hasAttribute('samaccountname');
$user->hasAttribute('sAMAccountname');
Casing & Hyphens
Attribute Casing
LdapRecord automatically normalizes all attribute keys to lowercase. This means when setting or getting model attributes, you can use alternate casing and the same attribute will be set or retrieved.
This is extremely handy so you do not have to look up the casing of each attribute every time you want to set or retrieve one. This also means you can use your own attribute convention:
$user = new User();
// Each will set the same attribute:
$user->samaccountname = 'John Doe';
$user->sAMAccountName = 'John Doe';
$user->samAccountName = 'John Doe';
Attribute Hyphens
Since LDAP does not support underscores in LDAP attributes but does support using hyphens, anytime you would like to set an attribute that contains a hypen, set it using an underscore instead. LdapRecord will automatically convert the underscore to a hyphen dynamically:
$user = new User();
$user->some_attribute = 'Value';
// Returns 'Value'
echo $user->some_attribute[0];
echo $user->getAttribute('some-attribute')[0];
Similarly, when retrieving attributes that contain a hyphen, use an underscore instead:
$user = User::find('cn=John Doe,dc=acme,dc=org');
// Both will act identically:
echo $user->apple_user_homeurl[0];
echo $user->getAttribute('apple-user-homeurl')[0];
Deleting Models
To delete a record from your directory, call the delete()
method on a model you have retrieved:
<?php
$user = User::first();
$user->delete();
The account you have configured to bind to your LDAP server must have permission to delete the record you have retrieved. If it does not, you will receive an exception upon deletion.
Deleting Models By Distinguished Name
In the example above, we are retrieving the record from the directory prior to deletion. However,
if you'd like to simply delete a model by its distinguished name, call the destroy()
method.
The number of deleted models will be returned from this method:
<?php
// Deleting single object...
$deleted = User::destroy('cn=John Doe,dc=acme,dc=org');
// Deleting multiple objects...
$deleted = User::destroy([
'cn=John Doe,dc=acme,dc=org',
'cn=Jane Doe,dc=acme,dc=org',
]);
You may also pass in
true
into the second parameter to recursively delete leaf entries if a record is located by the distinguished name you have given.
Recursive Deleting
Sometimes you will be working with containers or organizational units that contain nested records
inside of them. Calling delete()
on these records will generate an exception without first
deleting the records inside. If you would like to delete all records contained inside of
another model, pass in true
in the first parameter of the delete()
method:
<?php
$ou = OrganizationalUnit::find('ou=Users,dc=acme,dc=org');
$ou->delete($recursive = true);
Comparing Models
If you ever need to compare to models to see if they are the same, call the the is()
method.
This method will determine if the models have the same distinguished name and connection:
if ($user->is($anotherUser)) {
//
}
To see if a model is contained inside an organizational unit or another type of
container, call the isDescendantOf()
method:
$ou = OrganizationalUnit::find('ou=User Accounts,dc=acme,dc=org');
$user = User::find('cn=John Doe,ou=User Accounts,dc=acme,dc=org');
if ($user->isDescendantOf($ou)) {
// This user is contained inside this organizational unit.
}
You may also want to know whether a model is an ancestor of another. To
do so, call the isAncestorOf()
method:
$user = User::find('cn=John Doe,ou=User Accounts,dc=acme,dc=org');
$ou = OrganizationalUnit::find('ou=User Accounts,dc=acme,dc=org');
if ($ou->isAncestorOf($user)) {
// This OU is an ancestor of this user.
}
Calling
isDescendantOf()
orisAncestorOf()
performs recursive checks. If a model is contained in a nested OU / container of the one you are checking, the methods will returntrue
.
$ou = OrganizationalUnit::find('ou=User Accounts,dc=acme,dc=org');
$user = User::find('cn=John Doe,ou=Accounting,ou=User Accounts,dc=acme,dc=org');
// This will return true:
if ($user->isDescendantOf($ou)) {
//
}
// This will return true:
if ($ou->isAncestorOf($user)) {
//
}
To perform non-recursive checks, such as checking if a model is a
direct child of another model, call the isChildOf
method:
$ou = OrganizationalUnit::find('ou=User Accounts,dc=acme,dc=org');
$user = User::find('cn=John Doe,ou=User Accounts,dc=acme,dc=org');
if ($user->isChildOf($ou)) {
//
}
To perform the opposite, such as checking if a model is a parent of
another, call the isParentOf
method:
$officeOu = OrganizationalUnit::find('ou=Office,ou=User Accounts,dc=acme,dc=org');
$userAccountsOu = OrganizationalUnit::find('ou=User Accounts,dc=acme,dc=org');
if ($userAccountsOu->isParentOf($officeOu)) {
//
}
Query Scopes
Global Scopes
Global scopes allow you to add constraints to all LDAP queries that are created on a particular model. Writing a query scope allows you to be certain that a particular filter is always applied, rather than adding constraints every time you query the model.
Creating a global scope
To create a global query scope, create a class in your application that implements
the LdapRecord\Models\Scope
interface. This interface will require you to add
an apply
method. The apply
method accepts the query Builder
in the first
parameter, and the Model
in second parameter.
For an example, let's say our application must only retrieve user accounts whom
are employees of a particular company. We will create a file in our application
in the directory app/Ldap/Scopes
with the file name CompanyScope
:
<?php
namespace App\Ldap\Scopes;
use LdapRecord\Models\Scope;
use LdapRecord\Models\Model;
use LdapRecord\Query\Model\Builder;
class CompanyScope implements Scope
{
/**
* Apply the scope to the query.
*
* @param Builder $builder
* @param Model $model
*/
public function apply(Builder $builder, Model $model)
{
$builder->where('company', '=', 'Acme Company');
}
}
If you are looking to select additional attributes in your scope using the
select
method, useaddSelect
instead so your query selects are not overwritten.
Apply the global scope
Now that we've written our global scope, we can apply it to our users model.
We will do this by adding an override to the models boot
method and using the addGlobalScope
method:
<?php
namespace App\Ldap;
use LdapRecord\Models\Model;
use App\Ldap\Scopes\CompanyScope;
class User extends Model
{
/**
* The "booting" method of the model.
*
* @return void
*/
protected static function boot()
{
parent::boot();
static::addGlobalScope(new CompanyScope);
}
}
After adding the scope, queries ran on the User
model will always add the filter:
(company=\41\63\6d\65\20\43\6f\6d\70\61\6e\79)
Which your LDAP server will read as:
(company=Acme Company)
This is due to all values being automatically escaped using the LdapRecord query builder.
Anonymous global scopes
Instead of creating a class scope, you may also define global scopes using Closures. This is useful for simple scopes that do not warrant a separate class:
<?php
namespace App\Ldap;
use LdapRecord\Models\Model;
use LdapRecord\Query\Model\Builder;
class User extends Model
{
/**
* The "booting" method of the model.
*
* @return void
*/
protected static function boot()
{
parent::boot();
static::addGlobalScope('manager', function (Builder $builder) {
$builder->where('manager', '=', 'cn=John Doe,dc=local,dc=com');
});
}
}
Removing Global Scopes
If you would like to remove a global scope for a particular query, you may use
the withoutGlobalScope
method. The method accepts the class name of the
global scope as its argument:
User::withoutGlobalScope(CompanyScope::class)->get();
Or, if you've defined a global scope using a Closure:
User::withoutGlobalScope('manager')->get();
If you would like to remove several or even all of the global
scopes, you may use the withoutGlobalScopes
method:
// Remove all of the global scopes:
User::withoutGlobalScopes()->get();
// Remove some of the global scopes:
User::withoutGlobalScopes([
CompanyScope::class, 'manager'
])->get();
Events
LdapRecord models fire several different event during the creation, updating and deletion. Here is a list of all the events you can listen for:
| Event |
|---|
LdapRecord\Models\Events\Creating |
LdapRecord\Models\Events\Created |
LdapRecord\Models\Events\Updating |
LdapRecord\Models\Events\Updated |
LdapRecord\Models\Events\Saving |
LdapRecord\Models\Events\Saved |
LdapRecord\Models\Events\Deleting |
LdapRecord\Models\Events\Deleted |
To listen for these events, call the getEventDispatcher()
on the LdapRecord\Container
to retrieve the dispatcher, then call listen()
on the returned dispatcher:
<?php
$dispatcher = \LdapRecord\Container::getEventDispatcher();
$dispatcher->listen(\LdapRecord\Models\Events\Creating::class, function ($event) {
$model = $event->getModel();
});
You will want to setup any listeners prior to making changes to models, otherwise your listener will not be executed due to them not existing yet.
Serialization
All model instances can be converted to an array for JSON serialization. To serialize
a model instance, simply pass the model into json_encode()
. This calls
jsonSerialize()
on the model to retrieve is serializable data:
<?php
$user = User::first();
echo json_encode($user);
Hiding Attributes
You may want to exclude certain attributes from being included in the
serialization of your model, such as userPassword
for OpenLDAP.
To do this, add a $hides
property to your model:
use LdapRecord\Models\Model;
class User extends Model
{
protected $hides = ['userPassword'];
}
Now when you json_encode($model)
, all attributes will be included except the userPassword
attribute.
If you'd prefer a white-list of attributes, you can add a $visible
property instead,
which will ensure only the attributes specified will be included in serialization:
use LdapRecord\Models\Model;
class User extends Model
{
protected $visible = ['cn', 'mail', 'sn'];
}
Converting Attributes to JSON
Depending on the type of LDAP directory and model you are working with, you may need
to convert some attributes to a string before it can be properly serialized.
For example, if you your model is from Active Directory, you will need to
convert the objectguid
property to a string since it is in binary,
otherwise json_encode()
will throw an exception.
This can be done by adding a convertAttributesForJson()
method to your model:
By default, the
objectguid
andobjectsid
attributes are converted for you when using the built-in Active Directory models.
<?php
use LdapRecord\Models\Model;
class User extends Model
{
protected function convertAttributesForJson(array $attributes = [])
{
if ($this->hasAttribute('objectguid')) {
// If the model has a GUID set, we need to convert it due to it being in
// binary. Otherwise we will receive a JSON serialization exception.
return array_replace($attributes, [
'objectguid' => [$this->getConvertedGuid()]
]);
}
return $attributes;
}
}


You are viewing an historical archive of past issues. Please report new issues to the appropriate project issue tracker on GitHub.
Issues
| # | Summary | Product | Status | Kind | When |
|---|---|---|---|---|---|
| 137 |
Some user pictures appear shrunken inside the menu |
WinSwitch | closed | bug | Created , updated |
| 136 |
Icons inside menu are too widely spaced |
WinSwitch | closed | bug | Created , updated |
| 135 |
REQUEST: "Do not show again" checkbox should default to OFF |
WinSwitch | closed | feature request | Created , updated |
| 133 |
REQUEST: Limited access to preferences/features for non-admin users |
WinSwitch | closed | feature request | Created , updated |
| 132 |
REQUEST: Localize "Switch-In Items" and "Switch-Out Items" folders |
WinSwitch | closed | feature request | Created , updated |
| 131 |
Lengthy hangs without an internet connection |
Synergy | closed | bug | Created , updated |
| 130 |
REQUEST: EyeTV support |
Synergy Advance | open | feature request | Created , updated |
| 129 |
Add "Switch User..." button to ScreenSaver security window |
WinSwitch | closed | bug | |
| 128 |
REQUEST: Anti-alias album covers in the floater |
Synergy | open | feature request | Created , updated |
| 127 |
REQUEST: Options for when/how Growl notifications are sent |
Synergy Advance | closed | feature request | Created , updated |


JSON (or JavaScript Object Notation) is a highly portable data interchange format. While its structure is recognized natively by Javascript (as it is Javascript), its formatting conventions are easily recognized by other C-like languages.
JSON is built on two structures:
var jason = {
"age" : "28",
"hometown" : "Cairo, Egypt",
"gender" : "male"
};
The object created in the example could be accessed like any other object.
var family = [{
"name" : "Jason",
"age" : "24",
"gender" : "male"
},
{
"name" : "Kyle",
"age" : "21",
"gender" : "male"
}];
This array could be accessed like any array of objects.
Since JSON and XML are both used for data exchange, you may find yourself in the middle of deciding which format to use.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
namespace JSON_Demo1
{
public partial class Page1 : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
Employee oEmployee1 = new Employee { Name = "Emp1", ID = "11", Age = "30" };
Employee oEmployee2 = new Employee { Name = "Emp2", ID = "22", Age = "31" };
Employee oEmployee3 = new Employee { Name = "Emp3", ID = "33", Age = "20" };
List oList = new List() { oEmployee1, oEmployee2, oEmployee3 };
System.Web.Script.Serialization.JavaScriptSerializer oSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();
string sJSON = oSerializer.Serialize(oList);
Response.Write(sJSON);
}
}
public class Employee
{
public string Name { get; set; }
public string Age { get; set; }
public string ID { get; set; }
}
}
When you run the above application, you will see the JSON array on the response like so:
Now let’s change this example a little bit so we can make Ajax call to get the JSON data and parse it using javaScript. Change the code-behind to look like:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.Services;
namespace JSON_Demo1
{
public partial class Page1 : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
Response.Write(GetJSONData());
}
[WebMethod()]
public static string GetJSONData()
{
Employee oEmployee1 = new Employee { Name = "Emp1", ID = "11", Age = "30" };
Employee oEmployee2 = new Employee { Name = "Emp2", ID = "22", Age = "31" };
Employee oEmployee3 = new Employee { Name = "Emp3", ID = "33", Age = "20" };
List oList = new List() { oEmployee1, oEmployee2, oEmployee3 };
System.Web.Script.Serialization.JavaScriptSerializer oSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();
return oSerializer.Serialize(oList);
}
}
public class Employee
{
public string Name { get; set; }
public string Age { get; set; }
public string ID { get; set; }
}
}
now change the .aspx page to look like:
<%@ Page Language="C#" AutoEventWireup="true" CodeBehind="Page1.aspx.cs" Inherits="JSON_Demo1.Page1" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<title></title>

Now let’s run the application and see the output:
In the previous example, we created a WebMethod, then call it from jQuery and pass the output of it to JavaScript method BuildTable(msg). This method convert the string msg into a JSON array of objects and then loop on that array to build the shown simple data table. We also rendered the JSON data to the browser directly from server-side code, just to show the original data and how it transformed using JSON.
In this post we gave a brief background about JSON, when use it and when use XML to send data, how .Net provides native serializer for JSON, and how to deserialize it using JavaScript. We may look into more advanced topics regarding JSON in future posts.
Good Luck


3 references to RunspaceStateInfo
System.Management.Automation (3)
engine\hostifaces\Connection.cs (1)
244
:
this
(state, null)
engine\hostifaces\ConnectionBase.cs (1)
708
_runspaceStateInfo = new
RunspaceStateInfo
(state, reason);
engine\remoting\client\remoterunspace.cs (1)
1401
_runspaceStateInfo = new
RunspaceStateInfo
(state, reason);


Paul Suda
-
I've been developing web applications since 1998. I'm an experienced team
leader, data modeler, back-end coder, front-end browser troubleshooter,
server administrator, mentor, and perpetual learner.
- Currently a software team lead working at Civis Analytics.
- My new daily Three.js experiments


Which of the following is NOT relevant to the sorting problem lower bounds proof?
\Theta(n^2)
n
records
is n!
n
nodes has a depth of at least
\log n
The proof uses decision trees to model sorting algorithms.
The number of permutations affects the size of the decision tree.
Given a tree of a certain size, we can compute its minimum depth.
The cost of Bubble Sort does not affect the cost of other sorts.


Cedar Grove Christian Academy is a Christ-centered, mission-minded school. Students have the privilege of serving our community through our missions program using Acts 1:8 as our guideline. Throughout the year, each class will go out and serve in their mission field (Jerusalem and Judea). Also, in their missions classes, they will learn about, pray for, communicate with, and contribute to other missionaries serving elsewhere (the uttermost part of the earth).
Our goal is the development of students prepared academically and spiritually to serve Jesus Christ and to impact the world through their character and leadership.
Read Our VisionSince man was created in the image of God (Genesis 1:26-27), we are divine image bearers. As divine image bearers, we were created to glorify God, and to know Him.
Read Our School PhilosophyWe believe in the one true God (Deuteronomy 6:4), manifest in three distinct persons: Father, Son, and Holy Spirit (Matthew 28:19).
Read Our Statement of FaithAll constituents of the CGCA community are expected to manifest the highest Christian virtues in judgment, dignity, respect, and living as long as they are associated with CGCA.
Read Our Code of ConductLearning is something we do every day
We love to hear from our people, so please contact us for any questions or comments.
Contact Us


It's often said that the world feels like a dystopia at the moment. In MONTAG's Today's Dystopia series, our writers take a sideways look at fictional dystopias, compares them to reality, and ask: how close are we to living in tomorrow's dystopias today?
Pop culture allegories don’t get much more “pop” than They Live.  There’s a lengthy, seemingly gratuitous and unintentionally hilarious six minute street fight - over a pair of sunglasses - in the middle of it all.
And yet with each year that passes since its 1988 release, They Live reveals itself more and more to be a stiletto-sharp appraisal of the consumerism, capitalism, and inequality that saturates society – and how it is slowly, silently strangling us, right under our noses.
They Live is as much of a horror movie as director John Carpenter’s other more famous films, The Thing and Halloween. But instead of supernatural shocks and spills – and despite the bad guys being aliens – the film's allegory is so crystal clear that viewers will be in shock as they experience creeping dread.
Because, the viewer might reasonably conclude: in They Live, the horrorshow is real, and not only are we living in it every day, but we’re helping it consume us.
The premise is as stupendously silly as it is simple: a homeless, jobless man played by wrestler “Rowdy” Roddy Piper searches for employment in LA. He’s at the bottom of the heap, as his surname, Nada – meaning “Nothing” in Spanish – makes clear, and his struggle to find work is juxtaposed with the wealth and rampant consumerism of Reagan’s 1980s.
Having broken into a closed church to find a place to stay (here sounds the first of many ALLEGORY KLAXONS), John finds an unmarked cardboard box full of sunglasses. In the glaring Los Angeles sunshine, he slips a pair on – and to his horror, they reveal the real world to him for the very first time.
A gigantic wealth gap between the gilded top 10% and a vast population of people who don’t know when they’ll be paid next? A bloated middle class who believe that ultra-wealth is just within their grasp if only they keep working themselves into the ground?
Bombardment of multimedia messaging seducing us to part with cash for products that’ll get us that bit closer to the elite? A corrupt ruling class that promises us everything is better than ever?
Anything sound familiar to you?
Slip on your sunglasses as we peer into the cold heart of 2017 and compare how close today is to the dystopia of They Live. We’ll mark each scenario out of five sunglasses (🕶), in honour of John Nada’s iconic shades.
Hyper-commerciality and persuasion to conform and spend: 🕶 🕶 🕶 🕶
The glasses in They Live reveal the real message of billboards: the stark command OBEY. Ads featuring bikini-clad babes are a front for MARRY AND REPRODUCE; lifestyle magazine covers reveal CONFORM; radio transmitters drone the message SLEEP…. SLEEP… 24 hours a day.
John Carpenter said that They Live sprung from a realisation that the apparently bubbling economy and positive messaging from politicians had a distantly-connected flip side: he was constantly being pitched products.
"I began watching TV again. I quickly realized that everything we see is designed to sell us something... It's all about wanting us to buy something. The only thing they want to do is take our money.”
Nearly 30 years after They Live was filmed, what do 2017’s creative minds think about the persuasively commercial nature of media today?
Edgar Wright, director of hit movies Baby Driver and Shaun of the Dead, tweeted, “If you remade They Live now, the twist would be that you don’t even need the glasses.”
If you remade 'They Live' now, the twist could be that you don't even need the sunglasses.
— edgarwright (@edgarwright) February 23, 2017
But why? Maybe it's because the tools we use to interact with the world are so blatantly manipulative and the people that use them are so blatant in their manipulation.
Facebook’s “Data God” was Jeffrey Hammerbacher. He figured out how to slice up all the data they have on you, whih then helped the social media giant make more money than all the gods combined.
He left Facebook because he was tired of seeing, in part, creativity drained from the most gifted people. His off-the-cuff observation - “the best minds of my generation… are thinking about how to make people click ads. That sucks.” - has become a defining comment on our digital age.
There's an icky amorality to the work of these brightest minds, working at social media apps, dating apps, video apps, indeed, any app that values your time; which is to say, all of them.
By eagerly copying the technology that keeps punters gambling on video slot machines, they literally want to make you addicted to their app.
Why? To see more adverts, mainly.
 Modern art is a disaster area. Never in the field of human history has so much been used by so many to say so little.”
OBEY, indeed.
A super-elite using technology to enslave humanity: 🕶 🕶 🕶
“They've Recruited the rich and the powerful - they’re running the whole show!”
Every time They Live’s hero pops on his magic sunglasses, he can see that the wealthy people around him are actually aliens who have secretly assimilated society, assumed wealthy positions of authority, and put in place a scheme to control the other 90%.
This majority – the humans – are told what to do, and what to ignore, through media, advertising and branding.
So far, so preposterous – although these out-there ideas are fairly well-worn IRL, too.
Ex-Coventry City goalkeeper, ex-BBC broadcaster, self-proclaimed “son of God” and conspiracy theorist supremo David Icke believes that the world’s elite - notably, Queen Elizabeth II – are part of a shape-shifting reptilian race who enslave the rest of us.
Except – wake up sheeple: it’s actually our celebs who are in charge, and apparently it’s mainly female music stars. Beyonce is Satan; Rihanna, Miley Cyrus, Katy Perry, Nicki Minaj and Shakira are lizards; meanwhile, there must be a lot of evenings put aside for world-domination planning in the Knowles-Carter household, as Jay-Z is also an Illuminati puppet.
OK, this is all ludicrous.
But if the powerful, wealthy, and hyper-influential cabal of celebrity family empires like the Kardashians or the Hadids could be considered to be our social “super elite,” then it’s not a wild leap to consider that their mastery of social media is – in a small way - “controlling” the hundreds of millions that elect to follow their idols.
And whenever one of these elite convinces a follower to buy the now-ubiquitous fitness teas, waist trainers and teeth whiteners, they trap one more puny, working-class human into a cycle of financial dependance.
OBEY.
Magic glasses that reveal the truth of the world around you: 🕶 🕶 🕶
The magic glasses part of They Live is the biggest reality-leap of all, and in 2017 the most insight a pair of glasses will give you about the ultra-elite is an indication of how much Silicon Valley’s tech zillionaires are splashing on their shades (SPOILER: it’s at least $150).
However, there is already technology to rival the most iconic of They Live’s props.
When the viewer finds out that billboards are duping our hero, the most striking realisation – after taking on board the huge plot twist – is an understanding of just how surrounded we are by adverts, billboards, images, and signage imploring us to buy... stuff.
If you slip on an augmented reality headset created by Brand Killer, you will experience the same revelation. The Brand Killer team trained their software to recognise any brand logos that stray into the wearer’s field of vision – and then blur them out.
Brand Killer is ad-blocker for real life, and it answers an utterly compelling question: what would your world be like if it was completely unsullied by brand logos and adverts? If you got access to Brand Killer’s tech, you could have your own John Nada moment: pop 'em on and find out for yourself.
However - and here, please do not adjust your aluminium foil hat - Brand Killer suggests how our tech overlords could grab a little more control over us.
Pretty soon, AR glasses will be everyday tech. So what if clever code like Brand Killer was used nefariously, and instead of spotting ads and blocking them, it replaced them with ads tailored specifically to us instead? You couldn't escape. And why would you want to?
OBEY.
In total: 3.5 sunglasses.
In summation: They do, indeed, Live! (Kind of.)
OK, maybe there aren’t aliens enslaving us (sorry, David Icke) but we certainly do seem to be our own worst enemies with regards to allowing our intentions to be "influenced" by those we follow online.
Maybe, then, the 2017 version of popping on John Nada’s glasses is to voluntarily switch on flight mode, and step away from the internet for a few days: only then can we truly OBEY ourselves.
In the meantime, enjoy the fight for acceptance of your real-reality, as allegorically thumped home in They Live's preposterous fight scene...


Why did the programmer quit his job?
He didn't get arrays...
Describe what you did in this exercise. Did you understand the steps that you were making? Can you describe them in simple terms?u>
Today I connected my HTML and CSS files for my blog to each other. This is so that they can work together to make the content and style of my blog. The HTML is the content and the CSS tells it how to look.
Did you achieve the learning competencies and keep within the timebox?
I think all up i spent 45 minutes on this setion so that is pretty darn close to the suggested timebox. I've ticked off all the competencies and feel like I have a pretty good grasp of them.
What resources did you use during this challenge? Why did you choose those ones?
I chose to use the Udacity resourses for this challange. I really liked the format of diagrams and videos they used when we leant about version control and thought that I would stick with something I knew would work for me.
What obstacles did you come up against? How did you overcome them?
I struggled with mananging my time around the kids this week. I found catching up late at night was really helpful and I feel like I work quite well at night, when everyone is in bed and the house is quiet.
What are of you proud of from during this challenge?
I am really proud of not giving up when I realised there was so much I didnt know even after going through the Udacity courses. Having to research and teach myself how to make certain things was quite daunting but extreamly rewarding once I figured things out.
Did you keep within the timebox suggestions?
I did. I feel like they were much more generous this sprint.
How much faster could you do it if you started from the beginning again, right now?
I think I would definatly be faster, I know where to look for help now. I think I would solve problems a little faster.
Write a small story about HTML and CSS and their relationship. You could personify elements or speak from your own personal experience. The story should capture insight about the nature of HTML and CSS.
I need to come back to this question
What obstacles did you come up against? How did you overcome them?
I guess like every week the obsticle is trying to organise study around my family. Planning helps a lot but there are always things that pop up unexpectidly.
Any moments of surprise or joy?
Getting my links to work was really satisfying.Ithought I was going to struggle with that but it was surprisingly easy.
If HTML and CSS were animals, which animals do you think they would be? Why?
HTML would be a cat, they like boxes. CSS, probably something fussy and really pretty like a Lilac-breasted roller.


Needed if you want to run snapshot tests outside of the usual testthat framework For expert use only.
local_snapshotter(snap_dir = "_snaps", cleanup = FALSE, .env = parent.frame())


--- layout: page title: About header-img: "img/about-bg.png" ---
I spend my days learning about various subjects through either the news or youtube videos. Lately I've been focusing on the financial world and would like to pursue a career towards this. My outlook on the future is positive and I would love to someday give back to my community, family and friends if my abundance allows me to.


Psalm 148
1 Praise Yahweh.
Praise Yahweh, you in the heavens;
praise him, you in the heights.
2 Praise him, all his angels;
praise him, all his angel armies.
3 Praise him, sun and moon;
praise him, all you shining stars.
4 Praise him, highest heaven
and you waters that are above the sky.
5 Let them praise the name of Yahweh,
for he gave the command, and they were created.
6 He has also established them forever and ever;
he issued a decree that will never change.
7 Praise Yahweh from the earth,
you sea creatures and all ocean depths,
8 fire and hail, snow and clouds,
stormy wind fulfilling his word,
9 mountains and all hills,
fruit trees and all cedars,
10 animals wild and tame,
creatures that crawl and birds,
11 kings of the earth and all nations,
princes and all who govern on the earth,
12 both young men and young women,
elderly and children.
13 Let them all praise the name of Yahweh,
for his name alone is exalted
and his glory extends over the earth and the heavens.
14 He has lifted up the horn of his people
for praise from all his faithful ones,
the Israelites, the people near to him.
Praise Yahweh.