Chapter 1 Geographic Information
1.1 Learning outcomes
By the end of this practical you should be able to:
- Describe and explain GIS data formats and databases
- Source and pre-process spatial data
- Load and undertaken some basic manipulation of spatial data in QGIS and R
- Evaluate the (dis)advantages of each GIS you have used
1.2 Homework
Outside of our scheduled sessions you should be doing around 12 hours of extra study per week. Feel free to follow your own GIS interests, but good places to start include the following:
Exam
Each week we will provide a short task to test your knowledge, these should be used to guide your study for the final exam.
The task is to join some non spatial data to some spatial data.
Go to the New Zealand spatial data portal and download the file Territorial Authority 2018 (generalised), these are city or district councils. Make sure it’s the Territorial Authority 2018 data not SA1.
Go to the Stats NZ website and download the Statistical area 1 dataset for 2018 for the whole of New Zealand. Download the excel file this week, not the
.csv
.Unzip the downloaded census file and open 2018-SA1-dataset-individual-part-3a-total-NZ_updated_16-7-20, you will see a tab for Territorial authority. Join the 2018 paid employee field to the spatial data and make a basic map. Hint, you may need to make a new
.csv
file from the data.
Reading
Each week i’ll provide a core chapter to support the skills taught here. However, you are free to read whatever interests you and Adam and I strongly encourage this, so please don’t be constrained to this recommendation. Consult the reading list, practical and lecture each week for more ideas.
This week: - The Value of GIS & Spatial Data Science in Achieving the Sustainable Development Goals by Cobus van Rooyen
Chapter 2 “Geographic data in R” from Geocomputation with R by Lovelace, Nowosad and Muenchow (2020).
Opening practice: supporting reproducibility and critical spatial data science by Brunsdon and Comber (2020)
Watching
- What is Spatial Data Science from some of our friends at CARTO
1.3 Recommended listening 🎧
Some of these practicals are long, take regular breaks and have a listen to some of our fav tunes each week.
Andy. Week 1. No other choice than Vampire Weekend. The band formed during college (University in the USA) and produced their first album of their own whilst working full time jobs! Incredible. The lead vocalist and guitarist Ezra Koenig was a school teacher in Brooklyn, New York. I’ve seen them a few times at Glastonbuty and in London during which they gave the people what they wanted and took requests from their entire catalogue, really amazing musicians. Note some of the guest appearances in this album as they also might make an apperance later in the term!
Adam OK, it’s week 1, so I’m going in HARD. This week it’s the raver’s raver, the DJ’s DJ. This man is a style icon and so drum & bass it hurts - it’s Voltage! Here he is with his new album, Balance Over Symmetry. This will surprise a lot of people as it’s an eclectic mix of DnB bangers through to some Burial-inspired minimal dubstep. Follow him on Instagram and get your ears around his brilliant.
1.4 The Basics of Geographic Information
Geographic data, geospatial data or geographic information is data that identifies the location of features on Earth. There are two main types of data which are used in GIS applications to represent the real world. Vectors that are composed of points, lines and polygons and rasters that are grids of cells with individual values…
In the above example the features in the real world (e.g. lake, forest, marsh and grassland) have been represented by points, lines and polygons (vector) or discrete grid cells (raster) of a certain size (e.g. 1 x 1m) specifying land cover type.
1.4.1 Data types in statistics
Before we go any further let’s just quick go over the different types of data you might encounter
Continuous data can be measured on some sort of scale and can have any value on it such as height and weight.
Discrete data have finite values. It can be numeric such as number of shoes or categorial such as positive or negative.
Foot length would be continuous data but shoe size would be discrete data.
Nominal data has labels without any quantitative value such as hair colour or type of animal.
Ordinal data has an order or scale, for example if you have ever seen the chilli rating system on food labels or filled a happiness survey with a range between 1 and 10 — that’s ordinal. Here the order matters, but not the difference between them.
Binary data is that that can have only two possible outcomes, yes and no or shark and not shark.
1.4.2 Important GIS data formats
There are a number of commonly used geographic data formats that store vector and raster data that you will come across during this course and it’s important to understand what they are, how they represent data and how you can use them.
<IP_ADDRESS> Shapefiles
Perhaps the most commonly used GIS data format is the shapefile. Shapefiles were developed by ESRI, one of the first and now certainly the largest commercial GIS company in the world. Despite being developed by a commercial company, they are mostly an open format and can be used (read and written) by a host of GIS Software applications.
A shapefile is actually a collection of files —- at least three of which are needed for the shapefile to be displayed by GIS software. They are:
.shp
- the file which contains the feature geometry.shx
- an index file which stores the position of the feature IDs in the.shp
file.dbf
- the file that stores all of the attribute information associated with the coordinates – this might be the name of the shape or some other information associated with the feature.prj
- the file which contains all of the coordinate system information (the location of the shape on Earth’s surface). Data can be displayed without a projection, but the.prj
file allows software to display the data correctly where data with different projections might be being used
On Twitter and want to see the love for shapefiles….have a look at the shapefile account
<IP_ADDRESS> GeoJSON
GeoJSON Geospatial Data Interchange format for JavaScript Object Notation is becoming an increasingly popular spatial data format, particularly for web-based mapping as it is based on JavaScript Object Notation. Unlike a shapefile in a GeoJSON, the attributes, boundaries and projection information are all contained in the same file.
<IP_ADDRESS> Shapefile and GeoJSON
We’re now going to explore a shapefile (.shp
) and GeoJSON (.geojson
) in action.
Go to: http://geojson.io/#map=16/51.5247/-0.1339
- Using the drawing tools to the right of the map window, create 3 objects: a point, line and a polygon as I have done above. Click on your polygon and colour it red and colour your point green
- Using the ‘Save’ option at the top of the map, save two copies of your new data – one in
.geojson
format and one in.shp
format - Open your two newly saved files in a text editor such as notepad or notepad++. For the shapefile you might have to unzip the folder then open each file individually. What do you notice about the similarities or differences between the two ways that the data are encoded?
<IP_ADDRESS> Raster data
Most raster data is now provided in GeoTIFF (.tiff
) format, which stands for Geostarionary Earth Orbit Tagged Image File. The GeoTIFF data format was created by NASA and is a standard public domain format. All necesary information to establish the location of the data on Earth’s surface is embedded into the image. This includes: map projection, coordinate system, ellipsoid and datum type.
<IP_ADDRESS> Other data formats
Aforementioned data types and formats are likely to be the ones you predominately encounter. However there are several more used within spatial analysis. These include:
Vector
- GML (Geography Markup Language —- gave birth to Keyhold Markup Language (KML))
Raster
- Band SeQuential (BSQ) - technically a method for encoding data but commonly referred to as BSQ.
- Hierarchical Data Format (HDF)
- Arc Grid
There are normally valid reasons for storing data in one of these other formats. For example, BSQ are raster data with a separate text header file (.hdr
) providing geographic spatial reference information. Earth observation data often monitors the electromagnetic spectrum in bands. Humans see in the visible range of the spectrum and our vision is composed of red, green and blue wavelengths. If we wanted to analyse just the red wavelength the BSQ format would let us read in only that data. In comparison a GeoTIFF might come with all the data ‘packaged’ in one file and when doing analysis over thousands of images would significantly slow things down. That said you can now often find GeoTIFFs separated in a similar format to BSQ and it’s fairly straightforward to convert between raster formats.
<IP_ADDRESS> Geodatabase
A geodatabase is a collection of geographic data held within a database. Geodatabases were developed by ESRI to overcome some of the limitations of shapefiles. They come in two main types: Personal (up to 1 TB) and File (limited to 250 - 500 MB), with Personal Geodatabases storing everything in a Microsoft Access database (.mdb
) file and File Geodatabases offering more flexibility, storing everything as a series of folders in a file system. In the example below we can see that the FCC_Geodatabase (left hand pane) holds multiple points, lines, polygons, tables and raster layers in the contents tab.
<IP_ADDRESS> GeoPackage
A GeoPackage is an open, standards-based, platform-independent, portable, self-describing, compact format for transferring geospatial data. It stores spatial data layers (vector and raster) as a single file, and is based upon an SQLite database, a widely used relational database management system, permitting code based, reproducible and transparent workflows. As it stores data in a single file it is very easy to share, copy or move.
<IP_ADDRESS> SpatiaLite
SpatialLite is an open-source library that extends SQLite core. Support is fairly limited and most software that supports SpatiaLite also supports GeoPackage, as they both build upon SQLite. It doesn’t have any clear advantage over GeoPackage, however it is unable to support raster data.
<IP_ADDRESS> PostGIS
PostGIS is an opensource database extender for PostrgeSQL. Essentially PostgreSQL is a database and PostGIS is an add on which permits spatial functions. The advantages of using PostGIS over a GeoPackage are that it allows users to access the data at the same time, can handle large data more efficiently and reduces processing time. In this example calculating the number of bars per neighbourhood in Leon, Mexico the processing time reduced from 1.443 seconds (SQLite) to 0.08 seconds in PostGIS. However, data stored in PostGIS is much harder to share, move or copy.
<IP_ADDRESS> What will I use
The variety of data formats can see a bit overwhelming. I still have to check how to load some of these data formats that I don’t use frequently. But don’t worry, most of the time you’ll be using shapefiles, GeoPackages or raster data.
1.5 General data flow
As Grolemund and Wickham state in R for Data Science…

However, a nice place to start is looking at the typical workflow of a data science (or GIS) project which you will see throuhgout these practicals, which is summarised nicely in this diagram produced by Dr. Julia Lowndes adpted from Grolemund and Wickham.
To begin you have to import your data (not necessarily environmental) into R or someother kind of GIS to acatually be able to do any kind of analysis on it.
Once imported you might need to tidy the data. This really depends on what kind of data it is and we cover this later on in the course. However, putting all of your data into a consistent structure will be very beneficial when you have to do analysis on it — as you can treat it all in the same way. Grolemund and Wickham state that data is tidy when “each column is a variable, and each row is an observation”, we cover this more in next week in the Tidying data section.
When you have (or haven’t) tidied data you then will most likely want to transform it. Grolemund and Wickham define this as “narrowing in on observations of interest (like all people in one city, or all data from the last year), creating new variables that are functions of existing variables (like computing speed from distance and time), and calculating a set of summary statistics (like counts or means)”. However, from a GIS point of view I would also include putting all of your data into a similar projection, covered next week in Changing projections and any other basic process you might do before the core analysis. Arguably these processes could include things such as: clipping (cookie cutting your study area), buffering (making areas within a distance of a point) and intersecting (where two datasets overlap).
Tidying and transfrom = data wrangling. Remember from the introduction this could be 50-80% of a data science job!
After you have transformed the data the next best thing to do is visualise it — even with some basic summary statistics. This simple step will often let you look at your data in a different way and select more appropraite analysis.
Next up is modelling. Personally within GIS i’d say a better term is processing as the very data itself is usually a computer model of reality. The modelling or processing section is where you conduct the core analysis (more than the basic analysis already mentioned) and try to provide an answer to your research question.
Fianlly you have to communicate your study and outputs, it doesn’t matter how good your data wrangling, modelling or processing is, if your intended audience can’t interpret it, well, it’s pretty much useless.
In a few weeks come back and revisit this data flow section to see how what you have learnt fits the framework presented.
1.6 Data
The volume of geographic information which is freely available for use in the UK is increasing exponentially and spatially referenced data can often be found in many different places. In this practical we’re going to use data from the London data store — a free and open data-sharing portal provided by the Greater London Authority (GLA), also known as City Hall that is the devolved regional governance body of London.
We are going to get spatial data of the London boroughs and join flytipping (the illegal deposit of waste, commonly on road verges) data that is provided as a .csv
file. .csv
stands for comma-separated values (CSV) — it uses a comma to separate each value.
At the end of this document I’ll also run through some common sources of data that will stand you in good stead (be advantageous) for the rest of the course.
1.6.1 File paths
In your N drive: create a new folder called GIS and within this a sub folder called wk1. It is up to you how you organise your files. Make sure you change the file paths where appropriate to your own.
1.6.2 Data download
Firstly we need to get a spatial outline of the London boroughs. The geographic boundaries that are used in the UK are a complex, often inter-related, but ever changing mass of areas. For anyone new to the UK (or indeed not a trained quantitative geographer), it can be quite a daunting task to attempt to understand all of the boundaries that are in use. Fortunately the Office for National Statistics (ONS) has an online beginners guide to UK geography. If you need more information on the vast array of different UK geographies, go and explore these resources:
- A Beginners Guide to UK Geography (2019) v1.0
- https://geoportal.statistics.gov.uk/datasets/9c04ff58854040d09a5a7ce146ab59b4
Let’s download some data..
- Spatial Data
To get the data go to the London data store
Search for Statistical GIS Boundary Files for London
Download the statistical-gis-boundaries-london.zip
Unzip the data and save it to your wk1 folder.
- CSV data
On the same website search for fly-tipping incidents
Download the
.csv
file
1.6.3 Data pre-processing
Question Open the .csv
in Excel, what do you notice about how the data is stored?
Answer The year is a column and for each area the values are repeated for different years. In our analysis it is easier to have the different years as a column and populated for each area. So, we want to go from this…
To this…
In previous years there have errors in this .csv
, for example there used to be year value of 2017-2018 that was incorrect. Whilst these have been resolved, it’s always important to check your data.
As we are going to use this dataset in QGIS and R I’ve done it in Excel using a pivot table. In future we’ll use R to automate tasks like this.
Go to Insert > PivotTable
Select the original table and create a PivotTable in a new worksheet
The PivotTable Field box will appear, experiment with the different fields in each of the areas
I’ve used the following:
Note how I’ve altered the total_action_taken to the sum of… as the original was displaying incorrectly, to do so:
- Click on drop down button for total_action_taken > Value Field Settings > select Sum
It’s important to think about what data we actually need in the next step and it’s good practice to avoid data redundancy where possible.
Spoiler The spatial data we have downloaded already contains borough name, so we don’t need it twice. However, we do need a field to link the two datasets on. You could use borough name, but when using text fields sometimes input variations can affect joins. For example, if you had the University of Manchester in one dataset and Manchester University in another the join would fail. Consequently it’s usually best to join datasets on a code field.
Now save the Excel sheet that contains the pivot table as a new .csv
. Make sure that the first row of data holds the column titles. Remove all empty rows.
When saving the file also avoid any special characters (e.g. -) and spaces, use an underscore instead of spaces.
Warning Spatial software does not like file names with spaces or special characters.
Now it’s time to load, inspect and do some basic manipulation of this data. As mentioned in the lecture there are several GIS software ‘types’, here we will repeat the same process across QGIS and R.
1.6.4 QGIS
<IP_ADDRESS> Load data
- Search for and open QGIS
- Click on the open data source manager. Just above the word browser in the top left of the screen
- Navigate to the
London_Borough_Excluding_MHW.shp
and add it, you then have to close the data source manager.
You can right click on the layer to view the attribute table, however the .csv
file must be loaded nto QGIS in order to join it to a shapefile.
- Open the data source manager and select Delimited Text
- Navigate to our
.csv
file and provide a suitable layer name - Under Record and Fields Options make sure the number of header lines to discard is 0 and the First record has field names box is selected (this is assuming you left a title for each column in your
.csv
) - Under Geometry Definition select No geometry (attribute table only)
Does the sample data seem right?
- If so, click add then close
Note Your .csv
might have a extra header present (a row before Row Labels). You can either: (a) close this and go and delete this in the .csv
now and reload it or (b) change the number of header lines to discard to 1 in the Record and Fields Options in the screenshot above.
<IP_ADDRESS> Join data
- Right click on the London boroughs layer > Properties > Joins
- Click the plus button at the bottom of the box
- Complete the dialogue box
<IP_ADDRESS> Export data
Now let’s export to a GeoPackage.
- Right click on the London boroughs layer > Export > Save Feature As
- Select the GeoPackage format and complete the File name (the saved file name for the GeoPackage) and the Layer name (the name for this layer within the GeoPackage). Recall that a GeoPackage can store many data layers as a single file
- The new layer will be added to the map, so you can remove the old one (Right click on the layer > Remove Layer). Make sure you remove the right one!
We’ve now made a GeoPackage that we can connect our map project to.
- Under Browser, Right click on GeoPackage > New Connection > Navigate to your GeoPackage
- Click the down arrow left to GeoPackage and you should see the one you just navigated to. Click the down arrow on the database and you can see your layer.
- Now we are going to import our
.csv
into our GeoPackage. To do so go: Database > DB Manager - Select your GeoPackage in the left hand pane
- Import layer/file then select the
.csv
. Click Ok.
Again, remember that the .csv
in the Layers tab (bottom left) is the original. Remove it, then from the GeoPackage just click, hold and drag the .csv
into the layer pane.
Now let’s make a quick thematic map…
- Right click on your London boroughs layer > Properties > Symbology
- Select categorised and choose a data column and color ramp > click Classify
You could also select graduated, to do so, simply change the number of classes.
- Save your QGIS project
You should have produced something like this:
This a very basic map and we’ve just used raw counts to show how to join and map data. You should (almost) never do this, as counts can be misleading. This is because the counts are collected over areas of unequal size. It’s best to divide the values by some sort of denominator (e.g. area of borough, population) to normalise the data giving something like occurrence of flytipping per km2 enabling comparison between the boroughs. We cover this in more detail before reading week.
We haven’t talked about the Coordiante Reference System (CRS) (or Spatial Reference System (SRS)) of our map document
A coordinate reference system is a series of parameters that define the coordinate system. Within GIS we use geographic or projected coordinate systems. The former uses a three-dimensional spherical surface to define locations of Earth, whereas the latter is defined on a flat, two-dimensional surface giving it constant lengths, angles and areas. We cover this in much more detail later on in subsequent weeks.
QGIS defaults to the Coordinate Reference System (CRS) WGS 84, or known by its European Petroleum Survey Group (EPSG) code 4326. However, when you add your first layer it will default the CRS of the layer. We’ll go into the background of EPSG next time.
You can change the CRS by going Project > Properties and selecting CRS in the left hand pane.
You can also access the project CRS by just clicking on the EPSG code in the bottom right of your QGIS window, in the screenshot above click EPSG:27700.
Note As we are going to open the same files in difference GIS systems it is important to close the software before moving on. If you don’t then files can be locked and unreadable as they are still considered to be in use.
1.6.5 R
<IP_ADDRESS> Introduction
R is both a programming language and software environment, originally designed for statistical computing and graphics. R’s great strength is that it is open-source, can be used on any computer operating system and free for anyone to use and contribute to. Because of this, it is rapidly becoming the statistical language of choice for many academics and has a huge user community with people constantly contributing new packages to carry out all manner of statistical, graphical and importantly for us, geographical tasks.
The purpose of this practical is just to demonstrate data loading and manipulation in different software. The next practical will provide much more detail on R, so don’t worry about that.
In RStudio go:
- File > New File > R Script
You should be able to see these quadrants (without the code):
Below are bits of code, to start we will work using the console. So just copy the bits of code into the console window, changing the file names to where your data is stored. Then at the end of this section I’ll show you how to make a script.
R works on packages that are collections of functions and data. Packages are bits of code that extend R beyond the basic statistical functionality it was originally designed for. For spatial data, they enable R to process spatial data formats, carry out analysis tasks and create some of the maps that follow. Most pacakges are easily recognisable through a hexagon (hex) logo, or sticker. Mitchell O’Hara-Wild has even produced some code to make your own!
Bascially, without packages, R would be very limited. With packages, you can do virtually anything! One of the issues you will come across is that packages are being continually developed and updated and unless you keep your version of R updated and your packages updated, there may be some functions and options not available to you. This can be a problem, particularly with University installations which (at best) may only get updated once a year. Therefore, apologies in advance if things don’t work as intended, but there is often a work around.
For this practical we will need the ones listed in the code chunk below. Whilst we’ve installed them (with the code below), we haven’t yet loaded them. It’s best practice to do all this at the start of your code, however, for demonstration purposes I’ll load each one as we need it.
install.packages(c("sf", "tmap", "tmaptools", "RSQLite", "tidyverse"),
repos = "https://www.stats.bris.ac.uk/R/")
- We can also install and load packages through RStudio. If you can see the package listed in the package tab it is installed and a tick means it is loaded.
Packages we’ve installed with our code:
sf
: simple features, standard way to encode spatial vector datatmap
: layer-based and easy approach to make thematic mapstmaptools
: set of tools for reading and processing spatial dataRSQLite
: embeds the SQLite database engine in R
Here, repos
stands for repository where we will download the packages from. Once you have set it, you shouldn’t need to specify it again. Whilst we’ve installed the packages we haven’t loaded them…this is done through using library()
function. Thanks to Dianne Cook for this image…
If you are using a Mac you might also need rgdal
… this is because a lot of packages are dependent on it and it might not properly install…
install.packages("rgdal")
Then load rgdal
…
library(rgdal)
If you still have issues the check out the Mac R issues section
<IP_ADDRESS> Load data
Load the sf
package so we can read our shapefile in — remember to change to filepath to your shapefile.
For this first practical we’ll just leave the data where it is, next week we’ll show you how to use projects to make calling data much easier.
Note by default in R, the file path should be defined with
/
but on a windows file system it is defined with\
. Using\\
instead allows R to read the path correctly – alternatively, just use/
library(sf)
# change this to your file path!!!
<- st_read("C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/statistical-gis-boundaries-london/ESRI/London_Borough_Excluding_MHW.shp") shape
## Reading layer `London_Borough_Excluding_MHW' from data source
## `C:\Users\Andy\OneDrive - University College London\Teaching\CASA0005\2020_2021\CASA0005repo\Prac1_data\statistical-gis-boundaries-london\ESRI\London_Borough_Excluding_MHW.shp'
## using driver `ESRI Shapefile'
## Simple feature collection with 33 features and 7 fields
## Geometry type: MULTIPOLYGON
## Dimension: XY
## Bounding box: xmin: 503568.2 ymin: 155850.8 xmax: 561957.5 ymax: 200933.9
## Projected CRS: OSGB 1936 / British National Grid
Note If you are working on the UCL RStudio Server and have a shapefile (along with all the other relevant files required) you will just need to use…
<- st_read("London_Borough_Excluding_MHW.shp") shape
As RStudio cloud knows you have setup a project and to look for files within it…we cover this very soon for RStudio desktop.
To get a summary of the data held within the shapefile data (attribute table) enter the following:
summary(shape)
## NAME GSS_CODE HECTARES NONLD_AREA
## Length:33 Length:33 Min. : 314.9 Min. : 0.00
## Class :character Class :character 1st Qu.: 2724.9 1st Qu.: 0.00
## Mode :character Mode :character Median : 3857.8 Median : 2.30
## Mean : 4832.4 Mean : 64.22
## 3rd Qu.: 5658.5 3rd Qu.: 95.60
## Max. :15013.5 Max. :370.62
## ONS_INNER SUB_2009 SUB_2006 geometry
## Length:33 Length:33 Length:33 MULTIPOLYGON :33
## Class :character Class :character Class :character epsg:NA : 0
## Mode :character Mode :character Mode :character +proj=tmer...: 0
##
##
##
To have a quick look what the shapefile looks like enter the following:
plot(shape)
That plots everything in the shapefile (all the attitbues) if you just wanted the geometry (outline of the shape) you could use…
library(sf)
%>%
shape st_geometry() %>%
plot()
From what we did in QGIS this should look familiar.
We now need to load our .csv
file:
library(tidyverse)
#this needs to be your file path again
<- read_csv("C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/fly_tipping_borough_edit.csv") mycsv
##
## -- Column specification --------------------------------------------------------
## cols(
## `Row Labels` = col_character(),
## `2011_12` = col_double(),
## `2012_13` = col_double(),
## `2013_14` = col_double(),
## `2014_15` = col_double(),
## `2015_16` = col_double(),
## `2016_17` = col_double(),
## `2017_18` = col_double(),
## Grand_Total = col_double()
## )
When you load tidyverse
you might get a list of conflicts, this basically means that several packages have functions named the same thing. For example there is a function named filter()
in the packages dplyr
and stats
. If you wanted to use the dplyr
version you would use dplyr::filter()
or for the stats
version it would be stats::filter()
…
Note Your .csv
might have a extra header present (a row before Row Labels). You can either: (a) stop and go and delete this in the .csv
now and reload it or (b) change the code to… mycsv <- read_csv("Prac1_data/fly_tipping_borough_edit.csv", skip = 1)
To view the data just input:
mycsv
## # A tibble: 34 x 9
## `Row Labels` `2011_12` `2012_13` `2013_14` `2014_15` `2015_16` `2016_17`
## <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 E09000001 563 1492 433<PHONE_NUMBER>
## 2 E09000002 2687 2122 1399 3100 2672 1130
## 3 E09000003 2356 1828 890 219 615 130
## 4 E09000004 505 627 732 399 262 439
## 5 E09000005 6713 2232 3189 3926 3980 4366
## 6 E09000006 306 479 653 462 326 228
## 7 E09000007 5541 5962 8281 4837 4719 4656
## 8 E09000008 31 108<PHONE_NUMBER> 637
## 9 E09000009 6727 5679 6543 5521 6067 12688
## 10 E09000010 7262 3595 3269 4937 5292 10894
## # ... with 24 more rows, and 2 more variables: 2017_18 <dbl>, Grand_Total <dbl>
<IP_ADDRESS> Join data
In R we’ve given our London boroughs shapefile the name shape
and our flytipping .csv
the name mycsv
. If you look in the Environment quadrant you should see them both listed.
Join the .csv
to the shapefile. Here, replace Row Labels with whatever your GSS_CODE is called in the .csv
:
<- shape%>%
shape merge(.,
mycsv,by.x="GSS_CODE",
by.y="Row Labels")
Let’s break this down a bit. We just created a tibble of mycsv
(this is a new form of a dataframe) where each column has a variable and each row contains a set of values — so basically a normal table. We did this simply by using read_csv()
to read the data in. We then matched our csv
to our shape
based on the GSS_CODE
values in both.
Check the merge was successful, this is just going to show the top 10 rows:
%>%
shapehead(., n=10)
## Simple feature collection with 10 features and 15 fields
## Geometry type: MULTIPOLYGON
## Dimension: XY
## Bounding box: xmin: 509702.4 ymin: 155850.8 xmax: 554089.2 ymax: 200933.9
## Projected CRS: OSGB 1936 / British National Grid
## GSS_CODE NAME HECTARES NONLD_AREA ONS_INNER SUB_2009
## 1 E09000001 City of London 314.942 24.546 T <NA>
## 2 E09000002 Barking and Dagenham 3779.934 169.150 F <NA>
## 3 E09000003 Barnet 8674.837 0.000 F <NA>
## 4 E09000004 Bexley 6428.649 370.619 F <NA>
## 5 E09000005 Brent 4323.270 0.000 F <NA>
## 6 E09000006 Bromley 15013.487 0.000 F <NA>
## 7 E09000007 Camden 2178.932 0.000 T <NA>
## 8 E09000008 Croydon 8649.441 0.000 F <NA>
## 9 E09000009 Ealing 5554.428 0.000 F <NA>
## 10 E09000010 Enfield 8220.025 0.000 F <NA>
## SUB_2006 2011_12 2012_13 2013_14 2014_15 2015_16 2016_17 2017_18 Grand_Total
## 1 <NA> 563 1492 433<PHONE_NUMBER> 2579 7945
## 2 <NA> 2687 2122 1399 3100 2672 1130 1066 14176
## 3 <NA> 2356 1828 890 219 615<PHONE_NUMBER>
## 4 <NA> 505 627 732 399 262<PHONE_NUMBER>
## 5 <NA> 6713 2232 3189 3926 3980 4366 7483 31889
## 6 <NA> 306 479 653 462 326<PHONE_NUMBER>
## 7 <NA> 5541 5962 8281 4837 4719 4656 12671 46667
## 8 <NA> 31 108<PHONE_NUMBER> 637 2684 6339
## 9 <NA> 6727 5679 6543 5521 6067 12688 5471 48696
## 10 <NA> 7262 3595 3269 4937 5292 10894 5855 41104
## geometry
## 1 MULTIPOLYGON (((531145.1 18...
## 2 MULTIPOLYGON (((543905.4 18...
## 3 MULTIPOLYGON (((524579.9 19...
## 4 MULTIPOLYGON (((547226.2 18...
## 5 MULTIPOLYGON (((525201 1825...
## 6 MULTIPOLYGON (((540373.6 15...
## 7 MULTIPOLYGON (((528840.2 18...
## 8 MULTIPOLYGON (((535009.2 15...
## 9 MULTIPOLYGON (((510253.5 18...
## 10 MULTIPOLYGON (((531023.5 20...
Now, let’s make a quick thematic map (or a qtm) using the package tmap
. I’ve made mine for flytipping between 2011 and 2012 (column 2011_12). But check what your coloumn name is from the adove code head(shape, n=10)
it might be slightly different like 2011-2012 or x2011_2012….
library(tmap)
tmap_mode("plot")
# change the fill to your column name if different
%>%
shape qtm(.,fill = "2011_12")
1.6.6 Export data
Finally write shape to a new GeoPackage (.gpkg
) giving it the layer name of your choice:
%>%
shape st_write(.,"C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/Rwk1.gpkg",
"london_boroughs_fly_tipping",
delete_layer=TRUE)
## Deleting layer `london_boroughs_fly_tipping' using driver `GPKG'
## Writing layer `london_boroughs_fly_tipping' to data source
## `C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/Rwk1.gpkg' using driver `GPKG'
## Writing 33 features with 15 fields and geometry type Multi Polygon.
So here, we are saying the shape is the object we want to save, then to the GeoPackage file path, with the layer name of london_boroughs_fly_tipping
. I’ve set delete_layer
to true so I could overwrite mine when I developed this practical. Changing it to false would generate an error message if you ever tried to re-run the code.
Let’s also add the .csv
as we did in QGIS. This is a bit more complicated as we have to use the SQLite database package. Firstly, connect to the .gpkg
we just made:
library(readr)
library(RSQLite)
<- dbConnect(RSQLite::SQLite(),dbname="C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/Rwk1.gpkg") con
Now examine what is in the .gpkg
…you can see that i’ve already got my original_csv
stored within the .gpkg
as when i developed this practical i made sure it was working!
%>%
con dbListTables()
## [1] "gpkg_contents"
## [2] "gpkg_extensions"
## [3] "gpkg_geometry_columns"
## [4] "gpkg_metadata"
## [5] "gpkg_metadata_reference"
## [6] "gpkg_ogr_contents"
## [7] "gpkg_spatial_ref_sys"
## [8] "gpkg_tile_matrix"
## [9] "gpkg_tile_matrix_set"
## [10] "london_boroughs_fly_tipping"
## [11] "original_csv"
## [12] "rtree_london_boroughs_fly_tipping_geom"
## [13] "rtree_london_boroughs_fly_tipping_geom_node"
## [14] "rtree_london_boroughs_fly_tipping_geom_parent"
## [15] "rtree_london_boroughs_fly_tipping_geom_rowid"
## [16] "sqlite_sequence"
Add your .csv
and disconnect from the .gpkg
:
%>%
con dbWriteTable(.,
"original_csv",
mycsv,overwrite=TRUE)
%>%
con dbDisconnect()
Here overwrite
let’s you well..overwrite the exisiting file…if this wasn’t specified as true you would get an error saying the file already existed if you tried to run this code again.
<IP_ADDRESS> Making a script
To convert this bit of analysis into a script that we could save and run again in future, I would write the following in the script quadrant:
library(sf)
library(tmap)
library(tmaptools)
library(RSQLite)
library(tidyverse)
#read in the shapefile
<- st_read(
shape "C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/statistical-gis-boundaries-london/ESRI/London_Borough_Excluding_MHW.shp")
# read in the csv
<- read_csv("C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/fly_tipping_borough_edit.csv")
mycsv # merge csv and shapefile
<- shape%>%
shape merge(.,
mycsv,by.x="GSS_CODE",
by.y="Row Labels")
# set tmap to plot
tmap_mode("plot")
# have a look at the map
qtm(shape, fill = "2011_12")
# write to a .gpkg
%>%
shape st_write(.,"C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/Rwk1.gpkg",
"london_boroughs_fly_tipping",
delete_layer=TRUE)
# connect to the .gpkg
<- dbConnect(SQLite(),dbname="C:/Users/Andy/OneDrive - University College London/Teaching/CASA0005/2020_2021/CASA0005repo/Prac1_data/Rwk1.gpkg")
con # list what is in it
%>%
con dbListTables()
# add the original .csv
%>%
con dbWriteTable(.,
"original_csv",
mycsv,overwrite=TRUE)
# disconnect from it
%>%
con dbDisconnect()
You can then save your script through File > Save As.
If you are on a Mac you might also need to load rgdal
…just add the following at the top of the script.
library(rgdal)
1.6.7 What will I use
Well… it depends. If you wanted to quickly open a dataset to explore its contents then I’d use QGIS. However, if you had 100 raster images that you wanted to clip to your study area, I’d automate it in R. There are also specific packages developed for each type of software that might dictate what you use, for example I recently made use of the Urban Multi-scale Environmental Predictor (UMEP) plugin in QGIS. That said, as I needed to match different hourly meteorological variables over a three year period I automated the first part of the analysis in R and loaded a .csv
into QGIS. Recent advancments have also closed the gap somewhat between Graphic User Interface (GUI) GIS software (such as QGIS) and programming languages (such as R) with ‘bridges’ that allow you to control GUI software through code. To learn more about these bridges read Lovelace et al. (2019) chapter 9.
1.7 Data sources and task
Below I’ve listed a few good data sources. Explore these and any others you can find and get an interesting dataset (e.g. in this practical our flytipping .csv
) that you could join to some spatial data (e.g. in this practical the London boroughs .shp
). This could be for any location in the world.
1.7.1 UK Data Service
The UK Data Service geography service (https://census.edina.ac.uk/) has a library of hundreds of current and former boundary datasets for which attribute data are produced in the UK.
1.7.2 ONS
The Office for National Statistics (ONS) are the national statistical agency for England and Wales and have recently started to provide access to boundary data for the statistics they produce for various geographic areas.
Many of the boundaries on the ONS Geoportal are also available from the Edina Census Geography website in a more flexible fashion, however the ONS website provides very quick access to bulk-downloads — something which can be very useful when reading data directly from the web using computer software. From the ONS website you can also extract the URL at which the data is stored to use directly within your future code…
1.7.3 nomis
Nomis is provided by the ONS giving free access to UK labour market statistics. You can also bulk download census data from the site too!
1.7.4 OS
The Ordnance Survey (OS) are the national mapping agency for the UK. A few years ago, they opened up a number of their data products for public use including greenspace, OS Open Map and OS Terrain.
For the full range see: https://www.ordnancesurvey.co.uk/business-and-government/products/finder.html?Licensed%20for=OpenData%20(Free)&withdrawn=on
1.7.5 Edina Digimap
Before the Ordnance Survey opened up much of its data for public use, academics and students in the UK could access OS data using the Edina Digimap Service —- this service is still available today and provides access to a number of products in addition to those available from OS Open Data.
Perhaps the most exciting of the additional OS data products available from Digimap is OS MasterMap. MasterMap is a framework for all OS data and contains layers of data that include details of real world objects such as buildings, roads, paths, rivers, physical structures and land parcels, as well as the complete UK transport network. Whilst we still are required to go through Edina OS have recently announced plans to make this dataset free in the near future under the new Geospatial Commission.
1.7.6 OSM
Open Street Map (OSM) is a fantastic resource —- as the name suggests, all data contained in Open Street Map are open and free for anyone to use. Much like Wikipedia, anyone can contribute content to OSM and this brings with it its own benefits (frequent updates, very large user-base) and problems (data quality and patch coverage). OSM is a very good example of Volunteered Geographic Information (VGI).
It’s possible to download OSM data straight from the website, although the interface can be a little unreliable (it works better for small areas). There are, however, a number of websites that allow OSM data to be downloaded more easily and are directly linked to from the ‘Export’ option in OSM. Geofabrik (https://www.geofabrik.de/data/download.html) allows you to download frequently updated Shapefiles for various global subdivisions.
1.7.7 DEFRA
The Department for Environment and Rural Affairs (DEFRA) have recently created the Data Services Platform to openly distribute environmental data. See: https://environment.data.gov.uk/
1.7.8 Data lists
Another good place to start searching for data are data lists. They simply provide a comprehensive overview of all available data conveniently categorised by discipline and country.
I normally use this one: https://freegisdata.rtwilson.com/
1.8 Summary
Within this practical we have explored the different types, formats and software used to store, analyse and manipulate spatial data. In reflecting upon this practical you should consider the (dis)advantages of each, where and when they might be appropriate and the overall practicality. Next week we will delve further into R and RStudio.
1.9 Feedback
Was anything that we explained unclear this week or was something really clear…let us know using the feedback form. It’s anonymous and we’ll use the responses to clear any issues up in the future / adapt the material.
1.10 Mac R issues
If you have a Mac and run into issues using R, try this…
It seems that a lot of the packages are dependent on rgdal, which might not properly install…
Order of operations
Install java JDK (to respond to error Unable to find any JVMs matching version “(null)”, which occurred when we tried to load
OpenStreetMap
package in RStudio)Download and install the appropriate
.dmg
from Java SE Development Kit 11.0.1Run sudo R CMD javareconf in Terminal as described in this tutorial(though don’t follow this tutorial exactly, just up to that terminal command).
(Maybe) restart your computer (not really sure when this is necessary).
Install GDAL
GDAL, the Geospatial Data Abstraction Library, appears to be a massively important package / library / whatever that a lot of the spatial tools we use depends on. This is where things might be sticky — we already installed GDAL for QGIS, but obviously RStudio can’t find that install … dunno how to update the pointers, but we found that installing GDAL (again?) solved it. We HOPE that this didn’t ruin or break any other software we’ve installed, only time will tell :grimacing:
- To install GDAL on a mac, first (if you haven’t) install homebrew, the mac package manager
This can be done by executing /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
in Terminal, inputting password etc.
- Now we’re ready to brew install gdal (also in Terminal), which will run through and install a massive list of dependencies etc.
Set up RStudio
Restart RStudio (if the following doesn’t work on first go…)
Run through all the
install.packages()
commands one by one in RStudio and check to make sure that they install.Check if
rgdal
package is working.Check the other packages
OpenStreetMap
,tmap
, etc.


{% extends "base.html" %} {% block title %}Event Frames{% endblock %} {% block page_content %}
Navigate to and select an event frame template to use for overlaying.


Hi,
Greetings from the Symphony team!
A new tag $releaseVersion has been created for $repository. Your following PR(s) are a part of it:
Thanks,
The Symphony Team


deSEC is a free DNS hosting service, designed with security in mind. Running on open-source software and supported by SSE, deSEC is free for everyone to use.
deSEC requires JavaScript for accessing the control panel for DNS management. Please enable JavaScript for this site to proceed.
Hint: Accessing the API documentation does not require JavaScript.


# node-uuid Benchmarks
### Results
To see the results of our benchmarks visit https://github.com/broofa/node-uuid/wiki/Benchmark
### Run them yourself
node-uuid comes with some benchmarks to measure performance of generating UUIDs. These can be run using node.js. node-uuid is being benchmarked against some other uuid modules, that are available through npm namely uuid and uuid-js.
To prepare and run the benchmark issue;
`
npm install uuid uuid-js
node benchmark/benchmark.js
`
You’ll see an output like this one:
``` # v4 nodeuuid.v4(): 854700 uuids/second nodeuuid.v4(‘binary’): 788643 uuids/second nodeuuid.v4(‘binary’, buffer): 1336898 uuids/second uuid(): 479386 uuids/second uuid(‘binary’): 582072 uuids/second uuidjs.create(4): 312304 uuids/second
# v1 nodeuuid.v1(): 938086 uuids/second nodeuuid.v1(‘binary’): 683060 uuids/second nodeuuid.v1(‘binary’, buffer): 1644736 uuids/second uuidjs.create(1): 190621 uuids/second ```
- The uuid() entries are for Nikhil Marathe’s [uuid module](https://bitbucket.org/nikhilm/uuidjs) which is a wrapper around the native libuuid library.
- The uuidjs() entries are for Patrick Negri’s [uuid-js module](https://github.com/pnegri/uuid-js) which is a pure javascript implementation based on [UUID.js](https://github.com/LiosK/UUID.js) by LiosK.
If you want to get more reliable results you can run the benchmark multiple times and write the output into a log file:
`
for i in {0..9}; do node benchmark/benchmark.js >> benchmark/bench_0.4.12.log; done;
`
If you’re interested in how performance varies between different node versions, you can issue the above command multiple times.
You can then use the shell script bench.sh provided in this directory to calculate the averages over all benchmark runs and draw a nice plot:
`
(cd benchmark/ && ./bench.sh)
`
This assumes you have [gnuplot](http://www.gnuplot.info/) and [ImageMagick](http://www.imagemagick.org/) installed. You’ll find a nice bench.png graph in the benchmark/ directory then.


The REFMODE record stores the Reference Style option from the Options dialog box, General tab.
Record Data
Offset
Name
Size
Contents
4
fRefA1
2
Reference mode: =1 for A1 mode =0 for R1C1 mode


Enter the following details for your product:
Product Name
Price
Warranty
Type
Processor
Add another product
Search for a product
Go back to Home Page


class RockPaperScissorsGame
create master class for rpsg
Public Class Methods
continue(str1,str2,str3)
click to toggle source
add continue method for asking the user if they want to play rock paper scissors
# File lib/Main.rb, line 12 def continue(str1,str2,str3) puts str1 print str2 gets puts str3 end
new()
click to toggle source
initialize variables
# File lib/Main.rb, line 24 def initialize # initialize variables and set all equal to zero @player_score = @computer_score = @ties = 0 end
Public Instance Methods
play(winning_score)
click to toggle source
define play method, this will be the main function for playing rock paper scissors
# File lib/Main.rb, line 30 def play(winning_score) # make while loop while @player_score < winning_score && @computer_score < winning_score puts "\nPlayer score: #{@player_score}, " + "Computer score: #{@computer_score}, Ties<EMAIL_ADDRESS>player = PrivateMethods.player_choice computer = ProtectedConstants::COMPUTER_CHOICES.sample # chooses a random option puts "\nPlayer chooses #{player.to_s.downcase}." puts "Computer chooses #{computer.to_s.downcase}.\n" case PrivateMethods.player_outcome [player, computer] when :WIN puts "\n#{player.to_s.capitalize} beats #{computer.to_s.downcase}, player wins the round.\n" @player_score += 1 when :LOSE puts "\n#{computer.to_s.capitalize} beats #{player.to_s.downcase}, computer wins the round.\n" @computer_score += 1 else puts "\nTie, choose again\n" @ties += 1 end end puts "\nFinal score: player: #{@player_score}, " + "computer: #{@computer_score} (ties: #{@ties}).\n" case PrivateMethods.final_outcome(@player_score, @computer_score) when :WIN puts "\nPlayer wins!\n" when :LOSE puts "\nComputer wins!\n" else puts "\nIt's a tie!\n" end print "\n[press the enter/return key to exit game]" gets end


About
You could use this space to talk a bit about yourself, your research group, whoever you need to thank, etc. You could also put a link to the project’s GitHub repo, OSF page, research group website, and guide the reader where to look to learn more about the project and so on.
© Design:
d-asnaghi
and
HTML5 UP
. Adapted by
Daniel Roelfs
Code
This page serves to document the code associated with your paper. I’d
recommend not to copy-paste the code from your scripts verbatim in here,
but instead to use this page to document the organization of the
different scripts.
For instance, here’s an R script that does some stuff:
here’s an R
script
And for good measure, here’s a Python script:
here’s a Python
script
I recommend that you use simple markdown to write this document. It is
the simplest and most consistent way. Add your scripts to the
files/scripts/
folder in the website root directory. Link the scripts
with the code above and add a description to inform the reader about the
purpose of each script. Obviously, it’s still important to have
well-commented code within your scripts, but this script serves to guide
the reader through the pipeline, and to document how the scripts link
together.
Because I know this functionality will be appreciated by some, I also
added the possibility to knit Rmarkdown documents to markdown (see the
code.Rmd
file in the content/
directory). You can go into any text
editor you use to edit the other .md
files. Example will follow later.
Now let’s see what we can do when we knit in Rmarkdown. Here’s some code
loading a package:
library(tidyverse)
## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.0 ──
## ✔ ggplot2 3.3.2 ✔ purrr 0.3.4
## ✔ tibble 3.0.4 ✔ dplyr 1.0.2
## ✔ tidyr 1.1.2 ✔ stringr 1.4.0
## ✔ readr 1.4.0 ✔ forcats 0.5.0
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
And here’s a random plot:
ggplot(mtcars, aes(x = wt, y = mpg)) +
geom_point() +
geom_smooth() +
labs(x = "Weight (x 1000lbs)",
y = "Miles per Gallon") +
theme_minimal()
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'
The .md
file will show something like
this:
![](code_files/figure-markdown_strict/example-plot-1.png)
Here also counts:
“I didn’t have time to write a short letter, so I wrote a long one
instead.”
- Mark Twain
Or: anything is better than nothing
Data
Here is where we can store the data from the publication. Ideally, you’d link all files necessary to rerun the analyses using the scripts from the
code
section of this website. Be aware that the file limit for GitHub respositories is 100MB, so if you want to upload files larger than that (e.g. GWAS summary statistics) you’d need to link it to somewhere else, for instance
Open Science Framework
(file size limit of 50GB) or
Academic Torrents
(no limit). If you want to keep all your files together in the source directory of this site anyway, you can add the files larger than 100MB to the .gitignore
file to tell Git to not push these files to your repository.
Here’s an example of how to link to files in this page:
For instance:
here’s a csv file
And
here’s a plain text file
This link is generated like this:
[here's a csv file](files/mtcars.csv)
Here’s how you can link to for instance a file on OSF:
[Here's a file on OSF](https://osf.io/6tewj/)
Which will look like this:
Here’s a file on OSF
Internal links to for instance the
code
section are generated as follows:
[code](#code) (Note the `#` before the page name)
Populate this page with links and short descriptions of what is linked and a short meaning of the variables. Concise, readable, descriptions are better than really long and complex descriptions, but anything is better than nothing!
Paper and Preprint
The home page has direct links to the paper and the preprint, so this page exists just to offer the same link after you’ve left the landing page.
I’d recommend to make sections for the different versions of the manuscript online. Obviously, there are the final paper and the preprint, but perhaps you’ve also pre-registered your study, in which case you could link it here too to collect all versions and stages of the manuscript in one page. If appropriate, you could also add reviewer’s comments to the different stages to collect a full history of the submission/revision cycle!
Links work like they do for any other page. Normal Markdown syntax applies in the format:
[<label>](<url to the paper/preprint/etc.>)
You can use reverse chronological ordering to create a make-do timeline of the manuscript from preregistration to final publication. You could also add possible publications from the press office, media mentions or Twitter threads or whatever to document all the public communications about the project in one page.
Paper
The final paper was published in
NeuroImage
on the 1st of September 2020. We wrote a new
Twitter thread
when the final paper was published.
Press Release
The Press Office of the university released an item about this project on the 1st of February 2020, click
here
to read it.
Preprint
The preprint of the manuscript was published on
bioRxiv
on the 1st of January 2020. We also wrote a
Twitter thread
when the preprint came online.
Pre-registration
We submitted the preregistration for this project to
OSF
on the 1st of September 2019.
And so on…
Anything is better than nothing.
Summary
Popular scientific summaries are very important, and not often appreciated enough. Personally, I’ll always write one because I can’t be bothered to explain to my parents, grandparents, or friends not in science what the hell I’ve been doing for the past few months. Summarizing your project quickly and in a manner that is understandable for every lay person is worth the effort, believe me!
Here, you can use figures, tables, etc., to explain to Joe and Jane Carpenter what you did, what you found, and why they should care.
Don’t make it too long, don’t make it too short. Figures are always appreciated, illustrations even more! Also, don’t be too technical, don’t fuzz in details, mention only the main take-aways, and write with your grandmother in mind! Here’s some links to help you get started with writing a popular scientific summary with more tips:


You are one of the first to be using out client portal! Currently it will only be used to allow you to download
mailing lists which have been generated at your module_namerequest. the table includes a direct download link the your available
lists.
If you want to request a new listing then please select "List Builder" from your left menu and proceed from there.


JavaScript is disabled on your browser.
Skip navigation links
Package
Class
Use
Tree
Deprecated
Index
Help
Prev Letter
Next Letter
Frames
No Frames
All Classes
B
C
G
I
M
O
P
R
S
T
P
player
- Variable in class com.hero.time.
Timehero
playerBan(String)
- Method in class com.hero.time.
Timehero
playerFly()
- Method in class com.hero.time.
Timehero
playerFreeze()
- Method in class com.hero.time.
Timehero
playerFreezeAll()
- Method in class com.hero.time.
Timehero
playerGodmode()
- Method in class com.hero.time.
Timehero
playerHeal()
- Method in class com.hero.time.
Timehero
playerHide(boolean)
- Method in class com.hero.time.
Timehero
playerKick(String)
- Method in class com.hero.time.
Timehero
playerKill()
- Method in class com.hero.time.
Timehero
playerLocation()
- Method in class com.hero.time.
Timehero
playerMessage(String)
- Method in class com.hero.time.
Timehero
playerMute()
- Method in class com.hero.time.
Timehero
playerMuteAll()
- Method in class com.hero.time.
Timehero
playerName()
- Method in class com.hero.time.
Timehero
playerShow()
- Method in class com.hero.time.
Timehero
playerTP(double, double, double, World)
- Method in class com.hero.time.
Timehero
playerUnfly()
- Method in class com.hero.time.
Timehero
playerUnfreezeAll()
- Method in class com.hero.time.
Timehero
playerUngodmode()
- Method in class com.hero.time.
Timehero
playerUnmute()
- Method in class com.hero.time.
Timehero
playerUnmuteAll()
- Method in class com.hero.time.
Timehero
playerUnreeze()
- Method in class com.hero.time.
Timehero
playerWorld()
- Method in class com.hero.time.
Timehero
playerWorldName()
- Method in class com.hero.time.
Timehero
playerX()
- Method in class com.hero.time.
Timehero
playerY()
- Method in class com.hero.time.
Timehero
playerZ()
- Method in class com.hero.time.
Timehero
B
C
G
I
M
O
P
R
S
T
Skip navigation links
Package
Class
Use
Tree
Deprecated
Index
Help
Prev Letter
Next Letter
Frames
No Frames
All Classes


13 June 2018
Grid based design is a design format used to lay content out to work with the pattern our eyes naturally follow when viewing content. Grid based design is commonly used in print media like newspapers and magazines as well as websites.
Creating content with responsive design is an important part of making sure your webpage is viewed in the not only a legible way but a way that is user friendly and lays out in a way that works with the pattern our eyes naturally follow. Responsive design seems more important in this current age not only as hardware technology develops and there are more devices to accomodate but also as websites have gone from being static pages to having more and more interactive features.
Semantic structure describes a web page laid out a way where the sections have a hirachy and tags that describe them in a meaningful way. Having semantic structure by using semantic tags is important not only for search engine optimisation but for web accessibility reasons.


There are multiple button designs that are related with different scenarios. We use CTA buttons with a blue solid background to guide the user along the product, remarking with them the principal action of each screen. Only must be one CTA per screen.
Secondary buttons are used for non-principle actions and could live together with other secondary buttons although is not recommended if possible.
Third buttons are usually used for cancel actions on modals although could be used for other purposes.
Critical buttons are used only to remark actions that, dua its criticism, could create serious problems (E.g. delete actions or critical configuration changes).
Each of these four types works with main actions on a screen but on different levels of importance. Ideally, only one of each at most will exist in the same place. Besides, all these types of buttons could be accompanied by an icon after or before the text.
A five type of button exists. This button, unlike the others, is designed to be used multiple times on the same screen (E.g. in a toolbar). These are only-icon buttons.
Solid buttons look heavy on purpose. They direct the user’s attention to the primary action the application is suggesting that the user take.
Outline buttons (and lightgray solid ones) provide a lighter weight button style. They are used to indicate a secondary action visually that compliments a primary action or to reduce visual noise when there are many action of equal importance on the page.
Below are the good practices that should guide the uses of the component to maintain coherence in the design system and the spirit with which they were sustained.


About
Home
Product
HOW TO COMMAND A SEVEN-FIGURE INCOME AS AN AUTHOR
"discover the secret to becoming a bestselling author that makes a killing off his/her writing craft..."
- by Dafe Owolabi
Learn the master strategy that helped me sell a million copies of my book before it was launched.
Understand how I built a billion-dollar enterprise out of writing and how you too can.
Take advantage of the open doors that a well developed book affords you.
Learn how to launch a lucrative speaking career and a coaching business from writing a good book.
SEND MY FREE E-BOOK NOW!
" type ="video/ogg">
Your browser does not support this video format


| Option name | Type | Default |
|---|---|---|
| align | string | right |
| Aligment for the buttons | ||
| backgroundColor | color/td> | See below |
| Background color or gradient for the buttons backgroundColor=>array( linearGradient => array(0, 0, 0, 20), stops=>array( array(0.4, '#F7F7F7'), array(0.6, '#E3E3E3') ) ) | ||
| borderColor | color | #B0B0B0 |
| The border color of the buttons. | ||
| borderRadius | number | 3 |
| The border corner radius of the buttons | ||
| borderWidth | number | 1 |
| The border width of the buttons | ||
| enabled | boolean | true |
| Whether to enable buttons | ||
| height | number | 20 |
| Pixel height of the buttons | ||
| hoverBorderColor | string | #909090 |
| Color of the button border on hover. | ||
| hoverSymbolFill | string | #768F3E. |
| See navigation.buttonOptions => hoverSymbolFill. | ||
| hoverSymbolStroke | string | #4572A5. |
| Stroke (line) color for the symbol within the button on hover | ||
| onclick | function (javascript) | |
| A click handler callback to use on the button directly instead of the popup men | ||
| menuItems | array | |
| A collection of config options for the menu items. Each options object consists of a text option which is a string to show in the menu item, as well as an onclick parameter which is a callback function to run on click.
By default, there is one menu item for each of the available export types. Menu items can be customized by defining a new array of items and assigning null to unwanted positions (see override example below). |
||
| symbol | string | exportIcon |
| The symbol for the button. Points to a definition function in the Highcharts.Renderer.symbols collection. The default exportIcon function is part of the exporting module | ||
| symbolFill | string | #A8BF77. |
| See navigation.buttonOptions => symbolFill. | ||
| symbolSize | number | 12 |
| The pixel size of the symbol on the button | ||
| symbolStroke | string | #A0A0A0 |
| The color of the symbol's stroke or line | ||
| symbolStrokeWidth | number | 1 |
| The pixel stroke width of the symbol on the button | ||
| symbolX | number | 11.5 |
| The x position of the center of the symbol inside the button. | ||
| symbolY | number | 10.5 |
| The y position of the center of the symbol inside the button. | ||
| verticalAlign | string | top |
| The vertical alignment of the buttons. Can be one of "top", "middle" or "bottom". | ||
| width | number | 24 |
| The pixel width of the button | ||
| y | number | 10 |
| The vertical offset of the button's position relative to its verticalAlign. | ||
| x | number | 10 |
| The horizontal positioin of the button relative to the align option. |


<link href="https://acei.us/files/markupcss/latest.css" rel="stylesheet" type="text/css" />
in the <head>
tags of your documentclass=
atribute on elements as needed (i.e. using the <span>
element)Example HTML file:
(a.k.a. Quick Setup but longer and more detailed
Step 1
Before you use the stylesheet, you have to link it. You can link the minified latest sheet from my site, using this code snippet: <link href="https://acei.us/files/markupcss/latest.css" rel="stylesheet" type="text/css" />
Or, you can download the latest release from the downloads section of this page, or you can grab the latest pre-release (not reccomended) from GitHub.
Once you have downloaded the stylesheet, use the <link>
element in your document <head>
.
Step 2
Now that you have the stylesheet is linked, you can start applying mk-
classes to elements on a page. For example, on headers you can use the mk-title
class on header elements, to center and bold them.
Some more ideas you could do are using the mk-bold
and mk-itallic
classes for bold and itallic instead of <b>
and <i>
. The <b>
and <i>
elements are not screenreader friendly. You should not use them, and instead use css for decorative emphasis.
If you need to 'redact' text, you could use the mk-highlight-blackout
class to have a blackout appearance. Of course, you should never use this for actual redacting, as the text can be revealed by turning off css.
Step 3
If you need more help, you can ask for help in my discord server, or look at the complete reference avalible below.
ok, to be honest im using this as an excuse to add a 3rd step.
Basic Formatting
| Class | Demo | Added |
|---|---|---|
mk-bold |
Lorem Ipsum | 1.0.0 |
mk-bolder |
Lorem Ipsum | 1.0.0 |
mk-itallic |
Lorem Ipsum | 1.0.0 |
mk-underline |
Lorem Ipsum | 1.0.0 |
mk-strikethrough |
Lorem Ipsum | 1.0.0 |
Text Alignment
| Class | Demo | Added |
|---|---|---|
mk-align-center |
Lorem Ipsum | 2.2.0 |
mk-align-right |
Lorem Ipsum | 2.2.0 |
mk-align-left |
Lorem Ipsum | 2.2.0 |
Font Sizing
| Class | Demo | Added |
|---|---|---|
mk-superscript |
Lorem Ipsum | 1.0.0 |
mk-exponent |
Lorem Ipsum | 1.0.0 |
mk-subscript |
Lorem Ipsum | 1.0.0 |
mk-large |
Lorem Ipsum | 2.0.1 |
mk-larger |
Lorem Ipsum | 2.1.1 |
mk-small |
Lorem Ipsum | 2.0.1 |
mk-smaller |
Lorem Ipsum | 2.1.1 |
mk-title |
Lorem Ipsum | 2.0.1 |
Highlights
| Class | Demo | Added |
|---|---|---|
mk-highlight-red |
Lorem Ipsum | 2.0.0 |
mk-highlight-orange |
Lorem Ipsum | 2.0.0 |
mk-highlight-yellow |
Lorem Ipsum | 1.0.0 |
mk-highlight-green |
Lorem Ipsum | 1.0.0 |
mk-highlight-blue |
Lorem Ipsum | 1.0.0 |
mk-highlight-purple |
Lorem Ipsum | 2.0.0 |
mk-highlight-blackout |
Lorem Ipsum | 2.0.0 |
Coloring (this section bolded for readibility1)
| Class | Demo | Added |
|---|---|---|
mk-color-red |
Lorem Ipsum | 2.0.0 |
mk-color-orange |
Lorem Ipsum | 2.0.0 |
mk-color-yellow |
Lorem Ipsum | 2.0.0 |
mk-color-green |
Lorem Ipsum | 2.0.0 |
mk-color-blue |
Lorem Ipsum | 2.0.0 |
mk-color-purple |
Lorem Ipsum | 2.0.0 |
mk-color-black |
Lorem Ipsum | 2.0.0 |
Common Fonts (Fonts generally websafe, may not display on all sites)2
| Class | Demo | Added |
|---|---|---|
mk-font-arial |
Lorem Ipsum | 2.0.0 |
mk-font-calibri |
Lorem Ipsum | 2.0.0 |
mk-color-helvetica |
Lorem Ipsum | 2.0.0 |
mk-font-mono |
Lorem Ipsum | 2.0.0 |
mk-font-sans |
Lorem Ipsum | 2.0.0 |
mk-font-serif |
Lorem Ipsum | 2.0.0 |
Download on GitHub
GitHubVersion 2.2.0
See this version on GitHub
Version 2.1.1
See this version on GitHub
Version 2.0.0
See this version on GitHub
Version 1.0.0
mk-color
classes DO NOT automatically embolden text.


#include <math.h>
#include "copyright.h"
#include "system.h"
#include "machine.h"
#include "interrupt.h"
#include "mmu.h"
#include "tlb.h"
#include "thread.h"
#include "mipssim.h"
This code has been adapted from Ousterhout's MIPSSIM package. Byte ordering is little-endian, so we can be compatible with DEC RISC systems.


--- layout: post title: something was changed in my work date: 2012-09-29 01:11:09.000000000 +09:30 type: post published: true status: publish categories: - MyLife tags: - work meta: _edit_last: '1' posturl_add_url: 'yes' views: '18886' ---
Something was changed in my work these days. I think it is useful and important.
1. adjust some variables in mysql, it looks good for database.
2. adjust the huge page, turn off
3. reboot / restart
4. the slave was stop, because :MySQL Replication Fatal Error 1236 . Fix it by google.
Maybe there are many mistake in the passage. I just try to write in English. English is so important for me.
---
These day I got to sleep to late, It is very terrible.
Must change from now on.


The sscache library is a simple library that emulates memcache functions using HTML5 sessionStorage, so that you can store items with an expiration date. These demos show its current functionality - set, get, remove.
Set memcache entries (most with expiration of 2 minutes):
(Check "Storage->Session Storage" in Chrome Dev Tools to see them stored.)
Retrieve sscache entries:
Delete sscache entries and try to retrieve them:


Logan Paul confirms appeal to KSI loss has been filed
The YouTuber is furious with the result of the highly-anticipated bout after he lost by split decision earlier this month in Los Angeles
by Irishmirror.ieLogan Paul and his team have confirmed that they will file an official complaint following the the 24-year-old's defeat to YouTube rival KSI earlier this month.
Brit KSI- real name Olajide Williams Olatunji- beat Paul by split decision on November 9 with a final score of 56-55, 57-54 55-56 in the six-rounds fight.
Logan, however, had two points deducted in round four for hitting KSI while he was down, meaning without the penalizations he could have won.
Speaking on his YouTube channel, Paul said: "My team and I have decided to file an official appeal for the fight, because we just don't believe the outcome was right.
"I truthfully 100 per cent believe I did not lose this fight. My stomach just started turning over the fact I didn't believe the outcome was correct on this, and if I just stood by and didn't do anything about it, or at least try, then I'd be way disappointed with myself.
"This is a plea to the California State Athletic Commission, I would just hope that you give this the weight that it deserves.
"Because I fear that we aren't real p rofessional fighters, it may be overlooked, but I know this means a lot to me, my team, my friends and most importantly my fans.
"Even if the outcome doesn't come back in my favour, please treat it as a normal boxing appeal."
Paul was initially planning to give KSI the 'benefit of the doubt', but claimed he changed his mind after fans begged him to attempt an appeal.
"My initial reaction was to fight it, to contest it, to try my hardest to make it right," Paul continued.
"But I decided to let it go and give JJ the benefit of the doubt and go hands off, give the man his win he deserved it.
"So I took the backseat and let life works itself out.
"But then I get bombarded by thousands of people, fans, boxing fanatics, friends, random people on the street telling me the outcome of the fight just wasn't right."


|
Bullet Collision Detection & Physics Library
|
#include "btGjkPairDetector.h"
#include "BulletCollision/CollisionShapes/btConvexShape.h"
#include "BulletCollision/NarrowPhaseCollision/btSimplexSolverInterface.h"
#include "BulletCollision/NarrowPhaseCollision/btConvexPenetrationDepthSolver.h"
Go to the source code of this file.
Classes | |
| struct | btSupportVector |
| struct | btSimplex |
Macros | |
| #define | REL_ERROR2 btScalar(1.0e-6) |
Variables | |
| btScalar | gGjkEpaPenetrationTolerance = 0.001 |
| static btVector3 | ccd_vec3_origin (0, 0, 0) |
| #define REL_ERROR2 btScalar(1.0e-6) |
Definition at line 35 of file btGjkPairDetector.cpp.
|
static |
Definition at line 80 of file btGjkPairDetector.cpp.
Definition at line 665 of file btGjkPairDetector.cpp.
Definition at line 414 of file btGjkPairDetector.cpp.
Definition at line 460 of file btGjkPairDetector.cpp.
Definition at line 569 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 235 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 118 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 128 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 242 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 247 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 123 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 133 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 187 of file btGjkPairDetector.cpp.
Definition at line 138 of file btGjkPairDetector.cpp.
Definition at line 180 of file btGjkPairDetector.cpp.
Definition at line 158 of file btGjkPairDetector.cpp.
Definition at line 230 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 270 of file btGjkPairDetector.cpp.
| btScalar btVec3PointTriDist2 | ( | const btVector3 * | P, |
| const btVector3 * | x0, | ||
| const btVector3 * | B, | ||
| const btVector3 * | C, | ||
| btVector3 * | witness | ||
| ) |
Definition at line 335 of file btGjkPairDetector.cpp.
Definition at line 173 of file btGjkPairDetector.cpp.
Definition at line 154 of file btGjkPairDetector.cpp.
Definition at line 195 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 257 of file btGjkPairDetector.cpp.
|
inline |
Definition at line 252 of file btGjkPairDetector.cpp.
Definition at line 143 of file btGjkPairDetector.cpp.
Definition at line 166 of file btGjkPairDetector.cpp.
Definition at line 150 of file btGjkPairDetector.cpp.
Definition at line 216 of file btGjkPairDetector.cpp.
Definition at line 221 of file btGjkPairDetector.cpp.
Definition at line 226 of file btGjkPairDetector.cpp.
|
static |
| btScalar gGjkEpaPenetrationTolerance = 0.001 |
Definition at line 36 of file btGjkPairDetector.cpp.


Spaghetti sort is a linear-time, analog algorithm for sorting a sequence of items, introduced by Alexander Dewdney in his Scientific American column. This algorithm sorts a sequence of items requiring O(n) stack space in a stable manner. It requires a parallel processor
Case 1) O(n) (Worst case): Preparing the n rods of spaghetti takes linear time. Lowering the rods on the table takes constant time, O(1). This is possible because the hand, the spaghetti rods and the table work as a fully parallel computing device. There are then n rods to remove so, assuming each contact-and-removal operation takes constant time, the worst-case time complexity of the algorithm is O(n). Case 2) O(n) (Average case) Case 2) O(n) (Average case)
Spaghetti sort is not meant to be a practical sorting algorithm. It's a theoretical model for linear-time sorting given a parallel machinery for detecting the tallest spaghetti in a bunch in constant time. You can "simulate" this process in any number of ways using any programming language, but this result will be simulated-spaghetti-sort, not the "real" spaghetti sort. At any rate, this isn't what you would use to sort a trillion numbers. There's no single reasonable answer for "how to sort a trillion random integers"; it depends on context, purpose, and specific details.
The algorithm for Spaghetti sort is shown as:
For simplicity, assume we are sorting a list of natural numbers. The sorting method is illustrated using uncooked rods of
spaghetti:
1.For each number x in the list, obtain a rod of length x. (One practical way of choosing the unit is to let the largest number m
in the list correspond to one full rod of spaghetti. In this case, the full rod equals m spaghetti units. To get a rod of
length x, break a rod in two so that one piece is of length x units; discard the other piece.)
2.Once you have all your spaghetti rods, take them loosely in your fist and lower them to the table, so that they all stand
upright, resting on the table surface. Now, for each rod, lower your other hand from above until it meets with a rod—this one
is clearly the longest. Remove this rod and insert it into the front of the (initially empty) output list (or equivalently,
place it in the last unused slot of the output array). Repeat until all rods have been removed.


News feed
{% include feedurl %}Notifications
{% if friend_requests_num %}{{ friend_requests_num }} new friend request{{ friend_requests_num|pluralize }}
{% endif %} {% if notification_num %}{{notification_num}} new course recommendation{{ notification_num|pluralize }}
{% endif %} {% if not notification_num and not friend_requests_num %}No notifications
{% endif %}


BMLTSession
class BMLTSession : NSObject, URLSessionDataDelegate, BMLTCommunicatorDataSourceProtocol
This manages an interactive session connection with the server.
For whatever reason, Cocoa/Swift won’t let me derive from NSURLSession, so this class aggregates it, as opposed to extends it (not a big deal, probably better design in the long run anyway).
The semantic administration requires a constant session. Authentication is done once during the session, then the server maintains the authentication for the duration of the session. For this reason, we need to maintain a consistent session throughout our administration duties. This class is instantiated once by the App Delegate, and holds the session open.
This class maintains a dictionary of completion blocks that it uses to forward server responses to the users of this class. User instances need to define a completion block/callback, in the RequestCompletionBlock format. When they call a URL with this class, they provide this completion block. When the class instance gets the response, it forwards the data as an NSData instance to the completion block.
-
This class allows us to
See moretag
tasks with a completion block, allowing better autonomy.Declaration
Swift
class BMLTSessionTaskData
-
This is our session.
Declaration
Swift
var mySession: URLSession!
-
This will be set to true if the URL call is expected to fail (suppresses error report).
Declaration
Swift
var suppressErrors: Bool
-
This is true if this is an SSL session
Declaration
Swift
var isSSL: Bool
-
We are strictly linear. No concurrent I/O. We have just one job, and we do it through completion.
Declaration
Swift
var myCurrentTask: BMLTSession.BMLTSessionTaskData!
-
Make sure we clean up after ourselves.
Declaration
Swift
deinit
-
Pretty much exactly what it says on the tin.
Declaration
Swift
func disconnectSession()
-
Declares the connection status.
Declaration
Swift
func isSessionConnected() -> Bool
Return Value
a Bool, true if the session is currently connected.
-
Called when a session throws a nutty.
Declaration
Swift
func urlSession(_ inSession: URLSession, didBecomeInvalidWithError inError: Error?)
Parameters
inSession
The NSURLSession that controls this task.
error
The error returned.
-
Called as the task receives data.
Declaration
Swift
func urlSession(_ inSession: URLSession, dataTask: URLSessionDataTask, didReceive: Data)
Parameters
inSession
The NSURLSession that controls this task.
dataTask
The task that triggered this callback.
didReceive
the received data piece.
-
Called when a task has completed. We use this call to extract a JSON object from the response, then pass that object to the next handler.
Declaration
Swift
func urlSession(_ inSession: URLSession, task: URLSessionTask, didCompleteWithError error: Error?)
Parameters
inSession
The NSURLSession that controls this task.
task
The task that triggered this callback.
error
Any error that occurred.
-
Calls a URL
Declaration
Swift
func callURI(_ inCommunicator: BMLTCommunicator, inURIAsAString: String!, inCompletionBlock: BMLTCommunicator.RequestCompletionBlock!) -> Bool
Parameters
inCommunicator
The communicator instance calling this.
inURIAsAString
This contains a string, with the URI.
inCompletionBlock
This is the completion block supplied by the caller. It is to be called upon receipt of data.
inIsTest
If true, then we don’t report task errors (the call is expected to fail).
Return Value
a Bool. True, if the call was successfully initiated.
-
This is a call that is made to validate an SSL security challenge (session variant).
Declaration
Swift
func urlSession(_ session: URLSession, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void)
Parameters
session
The session thatt’s running the connection being challenged.
challenge
The challenge type
completionHandler
The handler we need to call with our response.
-
This is a call that is made to validate an SSL security challenge (task variant).
Declaration
Swift
func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void)
Parameters
session
The session thatt’s running the connection being challenged.
task
The actual URL task that resulted in this challenge.
challenge
The challenge type
completionHandler
The handler we need to call with our response.


Note
Click here to download the full example code
Finding ‘Oscillations’ With Filters¶
Examining the results of filtering aperiodic signals.
Filtering Signals¶
A common component of many analyses of neural time series is to apply filters, typically to try to extract information from frequency bands of interest.
However, one thing to keep in mind is that signals with aperiodic activity will always contain power at all frequencies. One of the corollaries of thinking of neural signals as containing aperiodic activity, is that there is always power within any arbitrarily defined frequency range. This power does not necessarily entail any periodic activity, but it can look like periodic activity when applying transforms such as narrow-band filters.
In this notebook we will simulate purely aperiodic signals, and apply filters to them, to explore these ideas.
# Import numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt
# Import the Bands object, for managing frequency band definitions
from fooof.bands import Bands
# Imports from NeuroDSP to simulate & plot time series
from neurodsp.sim import sim_powerlaw
from neurodsp.filt import filter_signal
from neurodsp.plts import plot_time_series
from neurodsp.utils import create_times, set_random_seed
Simulating Data¶
We will use simulated data for this example, to create some example aperiodic signals, that we can then apply filters to. First, let’s simulate some data.
# Simulation settings
s_rate = 1000
n_seconds = 4
times = create_times(n_seconds, s_rate)
# Set random seed, for consistency generating simulated data
set_random_seed(21)
# Simulate a signal of aperiodic activity: pink noise
sig = sim_powerlaw(n_seconds, s_rate, exponent=-1)
# Plot our simulated time series
plot_time_series(times, sig)
Filtering Aperiodic Signals¶
Now that we have a simulated signal, let’s filter it into each of our frequency bands.
To do so, we will loop across our band definitions, and plot the filtered version of the signal.
# Apply band-by-band filtering of our signal into each defined frequency band
_, axes = plt.subplots(len(bands), 1, figsize=(12, 15))
for ax, (label, f_range) in zip(axes, bands):
# Filter the signal to the current band definition
band_sig = filter_signal(sig, s_rate, 'bandpass', f_range)
# Plot the time series of the current band, and adjust plot aesthetics
plot_time_series(times, band_sig, title=label + ' ' + str(f_range), ax=ax)
ax.set_xlim(0, n_seconds); ax.set_ylim(-1, 1); ax.set_xlabel('');
As we can see, filtering a signal with aperiodic activity into arbitrary frequency ranges returns filtered signals that look like rhythmic activity.
Also, because our simulated signal has some random variation, the filtered components also exhibit some fluctuations.
Overall, we can see from filtering this signal that:
narrow band filters return rhythmic looking outputs
filtering a signal with aperiodic activity will always return non-zero outputs
there can be dynamics in the filtered results, due to variations of the aperiodic properties of the input signal
In this case, recall that our simulated signal contains no periodic activity. Altogether, this can be taken as example that just because time series can be represented as and decomposed into sinusoids, this does not indicate that these signals, or resulting decompositions, reflect rhythmic activity.
Observing Changes in Filtered Signals¶
Next, let’s consider what it looks like if you filter a signal that contains changes in the aperiodic activity.
For this example, we will simulate a signal, with an abrupt change in the aperiodic activity.
We will then filter this signal into narrow-band frequency ranges, to observe how changes in aperiodic activity appear in filtered data.
# Simulate a two signals with different aperiodic activity
sig_comp1 = sim_powerlaw(n_seconds/2, s_rate, exponent=-1.5, f_range=(None, 150))
sig_comp2 = sim_powerlaw(n_seconds/2, s_rate, exponent=-1, f_range=(None, 150))
# Combine each component signal to create a signal with a shift in aperiodic activity
sig_delta_ap = np.hstack([sig_comp1, sig_comp2])
# Plot our time series, with a shift in aperiodic activity
plot_time_series(times, sig_delta_ap)
Let’s first filter this signal in a low-frequency range that is typically examined for oscillatory activity, using the beta band as an example.
# Filter the signal to the current band definition
band_sig = filter_signal(sig_delta_ap, s_rate, 'bandpass', bands.beta)
# Plot the filtered time series
plot_time_series(times, band_sig)
plt.xlim(0, n_seconds); plt.ylim(-1, 1);
Out:
(-1, 1)
In the above, we can see that this shift in the aperiodic component of the data exhibits as what looks to be change in beta band activity.
We can also examine what this kind of shift looks like in high frequency regions that are sometimes analyzed, like our ‘high-gamma’ frequency band.
# Filter the signal to the current band definition
band_sig = filter_signal(sig_delta_ap, s_rate, 'bandpass', bands.high_gamma)
# Plot the filtered time series
plot_time_series(times, band_sig)
plt.xlim(0, n_seconds); plt.ylim(-1, 1);
Out:
(-1, 1)
Collectively, we can see that changes in aperiodic properties, that affect all frequencies, can look like band-specific changes when time series are analyzed using narrow-band filters.
If individual bands are filtered and analyzed in isolation, without comparison to either aperiodic measures, or other frequency bands, this kind of analysis could mis-interpret broadband aperiodic changes as oscillatory changes.
Note that in real data, to what extent such aperiodic shifts occur is something of an open question. Within subject changes in aperiodic activity has been observed, and so this remains a possibility that should be considered.
Conclusions¶
Here we have seen that filtering signals to narrow band signals can return results that reflect periodic activity and dynamics. We therefore suggest that narrow band filtered signals should not be presumed to necessarily reflect periodic activity. In order to ascertain whether narrow band frequency regions reflect periodic and/or aperiodic activity and which aspects are changing in the data, additional analyses, such as parameterizing neural power spectra, are recommended.
Total running time of the script: ( 0 minutes 5.648 seconds)


{% from 'macros.html' import include_src %} {% extends "base.html" %} {% block title %}mpld3PointLabel{% endblock %} {% block content %} {% filter markdown %} # mpld3PointLabel Testing the PointLabelTooltip plugin of [mpld3](http://mpld3.github.io/). The plot is created with [matplotlib](http://matplotlib.org/) and rendered in [d3.js](http://d3js.org/) using [mpld3](http://mpld3.github.io/). If you look at the source code for this template, you will also see that this documentation is a block of `markdown` embedded in the HTML. {% endfilter %}


Embedding Items with Square Market — Masonry test
Try embedding our items with popular layout plug-ins such as the fabulous
Masonry
.
Black Keychain on Square Market
Chicken Salad from Souvla on Square Market
Theodore Roosevelt - Pillow Doll on Square Market
Bonnie / Clyde - Cherry on Square Market
L'Orchidée Terrible on Square Market
Geranium Patchouli on Square Market
Halcyon & Mila on Square Market
Bitter Orange Soap on Square Market
Patty Melt Sandwich from Wise Sons at 24th St on Square Market
Back to Basics - Triangles on Square Market
Au Lait from Happy Bones on Square Market
Apples & Oranges: Best Butch Graphic Design on Square Market
Red Cardinal Gourd Ornament (Set of 2) on Square Market
PHRENOLOGY HEAD on Square Market
Billykirk: Medium Carryall (Olive) from TRADE Men's Wares on Square Market
Large Blue Artichoke on Square Market
Creme de la Earl Grey from Miansai on Square Market
Bolga Shopper on Square Market
Jamais deux sans trois orange turquoise on Square Market
I (718) Cufflinks from Soluna Soluna on Square Market
Oak & Porcelain Lamp on Square Market
Latte from Lost Weekend NYC on Square Market
Vintage Style Robot Pocket Watch from Miss Li Chun on Square Market
Cappuccino from Birch Coffee on Square Market
Gaba Cali Lunch Box from Gaba Sushi on Square Market


SVG 2D To 3D With JSModeler
https://github.com/jsgao0/3DFont-Feeder
https://github.com/kovacsv/JSModeler
Loading...


Name That Triangle!
Give us the length of Side One:
Give us the length of Side Two:
And the length of Side Three:
Triangulate!
Your triangle is:


Custom Page
ZAP can accommodate the definition of various non-standard error handling conditions.
Each Context may include multiple Custom Page definitions, with the following elements:
- Enabled > Whether the definition is enabled or not.
- Content > The String or Regex which defines the URL or response content to be matched.
- Content Location > Whether the "Content" should be matched against a URL or response "Content".
- Is Regex? > Indicating whether or not "Content" is a regular expression or not.
- Custom Page Type > Specifying what type of Custom Page is being defined:
- Error Page > For '500 - Internal Server Error' type pages.
- Not Found > For '404 - Not Found' responses.
- Ok > For '200 - Ok' definitions
- Other > To facilitate use of Custom Pages in scripts or other usages that have not yet been foreseen.
Configuration example
A configuration example showing how to fully configure a webapp that returns a 200 - Ok response with the message
"Sorry, we can't seem to find what you were looking for" is seen below:
- Set up a context for the web application
- Make sure your browser proxies everything through ZAP and browse to a page or endpoint which doesn't exist
- Go to ZAP and identify the request
- Set up the Custom Page definition:
- Highlight "Sorry, we can't seem to find what you were looking for" in the response pane, Right click and 'Flag as Context... Custom Page Indicator'
- A dialog will be opened already containing the highlighted response string.
- Accept the addition of the Custom Page definition.
Configured via