See: https://git.wiki.kernel.org/index.php/Aliases#Advanced
A shell alias is defined in a Drush configuration file called drushrc.php. See `drush topic docs-configuration`. There are two kinds of shell aliases: an alias whose value begins with a '!' will execute the rest of the line as bash commands. Aliases that do not start with a '!' will be interpreted as Drush commands.
$options['shell-aliases']['pull'] = '!git pull'; $options['shell-aliases']['noncore'] = 'pm-list --no-core';With the above two aliases defined, `drush pull` will then be equivalent to `git pull`, and `drush noncore` will be equivalent to `drush pm-list --no-core`.
For example, given the following site alias:
$aliases['dev'] = array ( 'root' => '/path/to/drupal', 'uri' => 'mysite.org', '#live' =><EMAIL_ADDRESS>);The alias below can be used for all your projects to fetch the database and files from the client's live site via `drush @dev pull-data`. Note that these aliases assume that the alias used defines an item named '#live' (as shown in the above alias).
$options['shell-aliases']['pull-data'] = '!drush sql-sync {{#live}} {{@target}} && drush rsync {{#live}}:%files {{@target}}:%files';If the user does not use these shell aliases with any site alias, then an error will be returned and the script will not run. These aliases with replacements can be used to quickly run combinations of drush sql-sync and rsync commands on the "standard" source or target site, reducing the risk of typos that might send information in the wrong direction or to the wrong site.


{% load staticfiles %}
{% load staticfiles %} Your activity "{{ activity_name }}" from {{timestamp}} has been processed.Distances:
Original on Strava : {{distance}} m
Original in gpx : {{distance_old}} m ({{change_old}}%)
Smoothened activity: {{distance_new}} m ({{change_new}}%)You will lose any already received comments, kudos, attached photos etc. At the moment only location data is saved, so also heartrate data etc. will be lost. See the github page for more information. See the original gpx See the smoothened gpx {% if graph != '' %} {% endif %}


Add the specified member to the Set value at key
-
string
$key: Key name
-
mixed
$member: Member
Add member to sorted set
-
string
$key: Key name
-
mixed
$member: Member
-
number
$score: Score of member
Append value to a end of string key
-
$key
$key: Key name
-
$value
$value: Value
Append value to the end of List
-
string
$key: Key name
-
mixed
$value: Element value
Decrement the number value of key by integer
-
string
$key: Key name
-
integer[optional]
$amount: Amount to decrement. One for default
Delete a key or keys
-
string|array
$keyOrKeys: Key name or array of key names
Delete field from hash
-
string
$key: Key name
-
mixed
$field: Field
Delete element from list by member at key
-
$key
$key: Key name
-
$value
$value: Element value
-
$count[optional]
$count: Limit of deleted items. For default no limit.
Remove the specified member from the Set value at key
-
string
$key: Key name
-
mixed
$member: Member
Delete the specified member from the sorted set by value
-
string
$key: Key name
-
mixed
$member: Member
Remove all elements in the sorted set at key with rank between start and end
-
string
$key: Key name
-
numeric
$start: Start position
-
numeric
$end: End position
Remove all the elements in the sorted set at key with a score between min and max (including elements with score equal to min or max).
-
string
$key: Key name
-
numeric
$min: Min value
-
numeric
$max: Max value
Return the difference between the Set stored at key1 and all the Sets key2, ..., keyN
-
array
$keys: Array of key names
-
string[optional]
$storeKey: Store to set with key name
Discard transaction
boolean
discard
()
Execute transaction
array
execute
()
Test if field is present in hash
-
string
$key: Key name
-
mixed
$field: Field
Test if the specified value is a member of the Set at key
-
string
$key: Key value
-
mixed
$member: Member
Set a time to live in seconds or timestamp on a key
Rediska_Transaction
expire
(
string $key,
integer $secondsOrTimestamp, [
$isTimestamp =
false],
boolean $isTimestamp[optional])
-
string
$key: Key name
-
integer
$secondsOrTimestamp: Time in seconds or timestamp
-
boolean
$isTimestamp[optional]: Time is timestamp. For default is false.
-
$isTimestamp
Remove all the keys of the currently selected DB
-
boolean[optional]
$all: Remove from all Db. For default is false.
Get value of key or array of values by array of keys
-
string|array
$keyOrKeys: Key name or array of names
Get value from hash field or fields
-
string
$key: Key name
-
string|array
$fieldOrFields: Field or fields
Return element of List by index at key
-
string
$key: Key name
-
integer
$index: Index
Get members from sorted set by min and max score
Rediska_Transaction
getFromSortedSetByScore
(
string $key,
number $min,
number $max, [
boolean[optional] $withScores =
false], [
integer[optional] $limit =
null], [
integer[optional] $offset =
null])
-
string
$key: Key name
-
number
$min: Min score
-
number
$max: Max score
-
boolean[optional]
$withScores: Get with scores. For default is false
-
integer[optional]
$limit: Limit. For default is no limit
-
integer[optional]
$offset: Offset. For default is no offset
Get hash fields and values
Return the number of fields in hash
Returns all the keys matching the glob-style pattern
Glob style patterns examples: h?llo will match hello hallo hhllo h*llo will match hllo heeeello h[ae]llo will match hello and hallo, but not hillo
Return the UNIX time stamp of the last successfully saving of the dataset on disk
Get List by key
Rediska_Transaction
getList
(
string $key, [
$start =
0], [
$end =
-1],
integer $start[optional],
integer $end[optional])
-
string
$key: Key name
-
integer
$start[optional]: Start index. For default is begin of list
-
integer
$end[optional]: End index. For default is end of list
-
$start
-
$end
Return the length of the List value at key
Get random element from the Set value at key
-
string
$key: Key name
-
boolean[optional]
$pop: If true - pop value from the set. For default is false
Return a random key from the key space
Get rank of member from sorted set
Rediska_Transaction
getRankFromSortedSet
(
string $key,
integer $member, [
$revert =
false],
boolean $revert[optional])
-
string
$key: Key name
-
integer
$member: Member value
-
boolean
$revert[optional]: Revert elements (not used in sorting). For default is false
-
$revert
Get member score from Sorted Set
-
string
$key: Key name
-
mixed
$member: Member value
Return all the members of the Set value at key
Return the number of elements (the cardinality) of the Set at key
Get all the members of the Sorted Set value at key
Rediska_Transaction
getSortedSet
(
string $key, [
$withScores =
false], [
$start =
0], [
$end =
-1], [
$revert =
false],
integer $withScores[optional],
integer $start[optional],
integer $end[optional],
boolean $revert[optional])
-
string
$key: Key name
-
integer
$withScores[optional]: Return values with scores. For default is false.
-
integer
$start[optional]: Start index. For default is begin of set.
-
integer
$end[optional]: End index. For default is end of set.
-
boolean
$revert[optional]: Revert elements (not used in sorting). For default is false
-
$withScores
-
$start
-
$end
-
$revert
Increment the number value of key by integer
-
string
$key: Key name
-
integer[optional]
$amount: Amount to increment. One for default
Increment field value in hash
Rediska_Transaction
incrementInHash
(
string $key,
mixed $field, [
$amount =
1],
number $amount[optional])
-
string
$key: Key name
-
mixed
$field: Field
-
number
$amount[optional]: Increment amount. One for default
-
$amount
Increment score of sorted set element
Rediska_Transaction
incrementScoreInSortedSet
(
string $key,
mixed $value,
integer|float $score)
-
string
$key: Key name
-
mixed
$value: Member
-
integer|float
$score: Score to increment
Provide information and statistics about the server
Return the intersection between the Sets stored at key1, key2, ..., keyN
-
array
$keys: Array of key names
-
string[optional]
$storeKey: Store to set with key name
Store to key intersection between sorted sets
Rediska_Transaction
intersectSortedSets
(
$keys,
string $storeKey, [
string $aggregation =
Rediska_Command_IntersectSortedSets::SUM])
-
array
$keys: Array of key names or associative array with weights
-
string
$storeKey: Result sorted set key name
-
string
$aggregation: Aggregation method: SUM (for default), MIN, MAX.
Is transaction started
boolean
isStarted
()
Move the key from the currently selected DB to the DB having as index dbindex
-
string
$key: Key name
-
integer
$dbIndex: Redis DB index
Move the specified member from one Set to another atomically
-
string
$fromKey: From key name
-
string
$toKey: To key name
-
mixed
$member: Member
Return and remove the last element of the List at key
Rediska_Transaction
popFromList
(
$key, [
$pushToKey =
null],
string $name,
string[optional] $pushToName)
-
string
$name: Key name
-
string[optional]
$pushToName: If not null - push value to another key.
-
$key
-
$pushToKey
Return and remove the last element of the List at key and block if list is empty or not exists
Rediska_Transaction
popFromListBlocking
(
string|array $keyOrKeys, [
$timeout =
0],
integer $timeout[optional])
-
string|array
$keyOrKeys: Key name or array of names
-
integer
$timeout[optional]: Timeout. Disable for default.
-
$timeout
Append value to the head of List
-
string
$key: Key name
-
mixed
$member: Member
Publish message to pubsub channel
-
array|string
$channelOrChannels: Channel or array of channels
-
mixed
$message: Message
Ask the server to silently close the connection.
Rename the old key in the new one
Rediska_Transaction
rename
(
string $oldKey,
string $newKey, [
boolean[optional] $overwrite =
true])
-
string
$oldKey: Old key name
-
string
$newKey: New key name
-
boolean[optional]
$overwrite: Overwrite the new name key if it already exists. For default is false.
Rewrite the Append Only File in background when it gets too big
Save the DB on disk
-
boolean[optional]
$background: Save asynchronously. For default is false
Select the DB having the specified index
Set value to a key or muliple values to multiple keys
Rediska_Transaction
set
(
string|array $keyOrData, [
$valueOrOverwrite =
null], [
$overwrite =
true],
mixed $valueOrOverwrite[optional],
boolean $overwrite[optional])
-
string|array
$keyOrData: Key name or array with key => value.
-
mixed
$valueOrOverwrite[optional]: Value or overwrite property for array of values. For default true.
-
boolean
$overwrite[optional]: Overwrite for single value (if false don't set and return false if key already exist). For default true.
-
$valueOrOverwrite
-
$overwrite
Set + Expire atomic command
-
string
$key: Key name
-
mixed
$value: Value
-
integer
$time: Expire time
Atomic set value and return old
-
string
$key: Key name
-
mixed
$value: Value
Set value to a hash field or fields
Rediska_Transaction
setToHash
(
string $key,
array|string $fieldOrData, [
mixed $value =
null], [
boolean $overwrite =
true])
-
string
$key: Key name
-
array|string
$fieldOrData: Field or array of many fields and values: field => value
-
mixed
$value: Value for single field
-
boolean
$overwrite: Overwrite for single field (if false don't set and return false if key already exist). For default true.
Set a new value as the element at index position of the List at key
-
string
$key: Key name
-
mixed
$value: Value
-
integer
$index: Index
-
$member
Return and remove the first element of the List at key
Return and remove the first element of the List at key and block if list is empty or not exists
-
string
$keyOrKeys: Key name or array of names
-
string
$timeout: Blocking timeout in seconds. Timeout disabled for default.
Stop all the clients, save the DB, then quit the server
Change the replication settings of a slave on the fly
-
string|Rediska_Connection|false
$aliasOrConnection: Server alias, Rediska_Connection object or false if not slave
Get sorted elements contained in the List, Set, or Sorted Set value at key.
-
string
$key: Key name
-
string|array
$value: Options or SORT query string (http://code.google.com/p/redis/wiki/SortCommand). Important notes for SORT query string:
- If you set Rediska namespace option don't forget add it to key names.
- If you use more then one connection to Redis servers, it will choose by key name,
and key by you pattern's may not present on it.
-
$options
Start transaction
boolean
start
()
Return a subset of the string from offset start to offset end (both offsets are inclusive)
-
string
$key: Key name
-
integer
$start: Start
-
integer[optional]
$end: End. If end is omitted, the substring starting from $start until the end of the string will be returned. For default end of string
Trim the list at key to the specified range of elements
-
string
$key: Key name
-
integer
$start: Start index
-
integer
$end: End index
Return the union between the Sets stored at key1, key2, ..., keyN
-
array
$keys: Array of key names
-
string[optional]
$storeKey: Store to set with key name
Store to key union between the sorted sets
Rediska_Transaction
unionSortedSets
(
$keys,
string $storeKey, [
string $aggregation =
Rediska_Command_UnionSortedSets::SUM])
-
array
$keys: Array of key names or associative array with weights
-
string
$storeKey: Result sorted set key name
-
string
$aggregation: Aggregation method: SUM (for default), MIN, MAX.
Add command to transaction
-
string
$name: Command name
-
array
$args: Arguments
Throw if transaction not supported by Redis
void
_throwIfNotSupported
()
Magic method for adding command to transaction
Magic method for execute
array
__invoke
()


There's a few spots still available for the GZGC17
Overwatch Tournament!
There's a few spot still available for our 3v3 Overwatch
tournament next weekend. Get together with two other
teammates and sign up at https://goo.gl/uc9Yei
for a chance to win a cash prize!
|
|
|
|
|


Time: Christmas and New Year, 230pm - 730pm.
Time: Saturday Jan 7, 14, 21, 28, 530pm - 930pm
Format: Round Robin
Top two teams in each group advance to Final.
Time: Saturday Feb 4, 530 - 930p, Sunday Feb 5, 130 - 530p
Format: * Double Elimination
Teams that have not qualified for finals can register at one of the two Lost Pearl events. Two Winners of the events qualify for final. Two runner ups of the events get consolation prize and are the replacement teams for top 16 teams that cannot make the final.
* Double E Format is tentative. It may be changed to group RR followed by SE depends on the number of entries.
Time: Saturday Feb 11, 230 - 730p
Format: Single Elimination
Top 16 teams will fight for the U3700 Champion. Semi-finals and Finals will be lively broadcasted with commentators.


Work simultaneously on different panels. Share the work with teammates.
Help us spreading the word. Tell your friends with just one-click
Choose settings depending on the criteria you value the most
We would love to hear from you!
Sign up to our newsletter for the latest news
© Copyright 2017 Ac interactive media design - All rights reserved.


Hi my name is Martin, i ’m a developer working in Javascript, HTML, and CSS. Before this I was a parking attendant working out of Burbank which required me to be organized and cool under pressure of anxious and nervous people. Before that I was an Ultrasound field engineer where I used to debug, scan, clean, and perform routine maintenance on Ultrasound machines and their peripherals. I am Currently working on a Game of Thrones themed Quiz application using the MERN stack. Technical skills React js, Mongo DB, Express, Node js, HTML, JavaScript, CSS, Flexbox, jQuery, Github, and Git
This is my version of Connect Four. The Technologies I used were HTML5, CSS3, and JavaScript.
The hardest part of this project for me was refactoring my win conditions which I am still working
on at the moment. It was really fun to make and I enjoyed the styling and theme I had chosen. This app us hosted on GitHub.
Try the game here
This is Catch a Fade. This idea came from my barber who needed something easier for his customers to
book him for a haircut. At the time I was learning Ruby on Rails which is the language I chose to use for
this project. The reason for it looking so rugged is because its my Barbers style. I will restyle this and
make it a bit more presentable and hand it off to my Barber for his use. This app is hosted on Heroku
Try the app here
This is Gobbler. This was a group project. We got the idea for making this application from Twitter.
We made twitter ... but with an Edit button. He hit the Twitter API to get the users tweets. We used
HTML5, CSS3, Javascript, Node js, Express, Mongoose, and MongoDB. I was in charge of Styling for this
project I used a mix of Materialize and Bootstrap, the reason for that honestly was for experimentation
purposes. (I like the Material design a bit better). This app is hosted on Heroku.
Try the app here
Valar Dohaeris, This project was inspired by Game of Thrones. I wanted to make a trivia/quiz game that
prepared people for the final season of Game of Thrones. the technologies I used were HTML5, CSS3,
Mongo DB, Express, Node js, and React. I did have a token based authentication for this app but I had decided
to remove it so users do not have to deal with the annoyance of having to sign up just to play the game.
This was my first React project, This was the Language I really enjoyed
using because of its Flexability and modularity. This app is Hosted on Heroku.
Try the app here


Instances
Factories
Handlers
Instance Name
State
Factory
Provided Services
Specifications
State
Service Id
Service Properties
Required Services
Specification
State
Filter
Binding Policy
Optional
Aggregate
Matching Services
Used Services
Architecture
Instances
Factories
Handlers


Last updated: 08-March-2019
TheBluePlanet.net ("us", "we", or "our") operates http:/www.theblueplanet.net (the "Site"). This page informs you of our policies regarding the collection, use and disclosure of Personal Information we receive from users of the Site. We use your Personal Information only for providing and improving the Site. By using the Site, you agree to the collection and use of information in accordance with this policy.
While using our Site, we may ask you to provide us with certain personally identifiable information that can be used to contact or identify you. Personally identifiable information may include, but is not limited to your name, email address ("Personal Information").
Like many site operators, we collect information that your browser sends whenever you visit our Site ("Log data"). This Log data may include information such as your computer's Internet Protocol ("IP") address, browser type, browser version, the pages of our Site that you visit, the time and date of your visit, the time spent on those pages and other statistics. In addition, we may in the future use third party services such as Google Analytics that collect, monitor and analyse this.
We may use your Personal Information to contact you with newsletters, marketing or promotional materials and other information that you provide us with.
Cookies are files with small amount of data, which may include an anonymous unique identifier. Cookies are sent to your browser from a web site and stored on your computer's hard drive.
Like many sites, we use "cookies" to collect information. You can instruct your browser to refuse all cookies or to indicate when a cookie is being sent. However, if you do not accept cookies, you may not be able to use some portions of our Site. By using this site you agree to our use of cookies.
The security of your Personal Information is important to us, but remember that no method of transmission over the Internet, or method of electronic storage, is 100% secure. While we strive to use commercially acceptable means to protect your Personal Information, we cannot guarantee its absolute security.
This Privacy Policy is effective as of 08-March-2019 and will remain in effect except with respect to any changes in its provisions in the future, which will be in effect immediately after being posted on this page. We reserve the right to update or change our Privacy Policy at any time and you should check this Privacy Policy periodically. Your continued use of the Service after we post any modifications to the Privacy Policy on this page will constitute your acknowledgment of the modifications and your consent to abide and be bound by the modified Privacy Policy. If we make any material changes to this Privacy Policy, we will notify you either through the email address you have provided us, or by placing a prominent notice on our website.
If you have any questions about this Privacy Policy, please contact us.


This is a simple Cordova application that implements an alternate alert capability that is compatible with Windows Universal Apps.


Captain America is a fictional superhero appearing in American comic books published by Marvel Comics. Created by cartoonists Joe Simon and Jack Kirby, the character first appeared in Captain America Comics #1 (cover dated March 1941) from Timely Comics, a predecessor of Marvel Comics. Captain America was designed as a patriotic supersoldier who often fought the Axis powers of World War II and was Timely Comics' most popular character during the wartime period. The popularity of superheroes waned following the war and the Captain America comic book was discontinued in 1950, with a short-lived revival in 1953. Since Marvel Comics revived the character in 1964, Captain America has remained in publication.
The character wears a costume bearing an American flag motif, and he utilizes a nearly indestructible shield which he throws as a projectile. Captain America is the alter ego of Steve Rogers, a frail young man enhanced to the peak of human perfection by an experimental serum to aid the United States government's efforts in World War II. Near the end of the war, he was trapped in ice and survived in suspended animation until he was revived in the present day. Although Captain America often struggles to maintain his ideals as a man out of his time with its modern realities, he remains a highly respected figure in his community which includes becoming the long-time leader of the Avengers.
Iron Man is a fictional superhero appearing in American comic books published by Marvel Comics. The character was co-created by writer and editor Stan Lee, developed by scripter Larry Lieber, and designed by artists Don Heck and Jack Kirby. The character made his first appearance in Tales of Suspense #39 (cover dated March 1963), and received his own title in Iron Man #1 (May 1968).
A wealthy American business magnate, playboy, and ingenious scientist, Anthony Edward "Tony" Stark suffers a severe chest injury during a kidnapping. When his captors attempt to force him to build a weapon of mass destruction, he instead creates a powered suit of armor to save his life and escape captivity. Later, Stark develops his suit, adding weapons and other technological devices he designed through his company, Stark Industries. He uses the suit and successive versions to protect the world as Iron Man. Although at first concealing his true identity, Stark eventually declared that he was, in fact, Iron Man in a public announcement.
In Germanic mythology, Thor (/θɔːr/; from Old Norse: Þórr) is a hammer-wielding god associated with thunder, lightning, storms, oak trees, strength, the protection of mankind and also hallowing and fertility. Besides Old Norse Þórr, extensions of the god occur in Old English as Þunor and in Old High German as Donar (runic þonar ᚦᛟᚾᚨᚱ). All forms of the deity stem from a Common Germanic *Þunraz (meaning thunder).
Thor is a prominently mentioned god throughout the recorded history of the Germanic peoples, from the Roman occupation of regions of Germania, to the tribal expansions of the Migration Period, to his high popularity during the Viking Age, when, in the face of the process of the Christianization of Scandinavia, emblems of his hammer, Mjölnir, were worn and Norse pagan personal names containing the name of the god bear witness to his popularity.
The Hulk is a fictional superhero appearing in publications by the American publisher Marvel Comics. Created by writer Stan Lee and artist Jack Kirby, the character first appeared in the debut issue of The Incredible Hulk (May 1962). In his comic book appearances, the character is both the Hulk, a green-skinned, hulking and muscular humanoid possessing a vast degree of physical strength, and his alter ego Dr. Robert Bruce Banner, a physically weak, socially withdrawn, and emotionally reserved physicist, the two existing as independent personalities and resenting of the other.
The Hulk once unleashed all of his power and destroyed a planet with a single punch.
He has fought everyone in the Marvel Universe and most of the characters in the DC Universe as well.
He has been married 3 times and has 4 children.
Berserker Wolverine once gutted Hulk until he was sure that Hulk was dead. As Wolverine started to walk away, Hulk slowly got up, already healing from the extreme wounds.
In one of the most depressing comic book stories, Dr. Bruce Banner is the sole survivor of a nuclear war. His mutation not only allows him to survive the death of humanity, but it also curses him to wander a dead Earth forever.
Following his accidental exposure to gamma rays during the detonation of an experimental bomb, Banner is physically transformed into the Hulk when subjected to emotional stress, at or against his will, often leading to destructive rampages and conflicts that complicate Banner's civilian life. The Hulk's level of strength is normally conveyed as proportionate to his level of anger. Commonly portrayed as a raging savage, the Hulk has been represented with other personalities based on Banner's fractured psyche, from a mindless, destructive force, to a brilliant warrior, or genius scientist in his own right. Despite both Hulk and Banner's desire for solitude, the character has a large supporting cast, including Banner's lover Betty Ross, his friend Rick Jones, his cousin She-Hulk, sons Hiro-Kala and Skaar, and his co-founders of the superhero team the Avengers. However, his uncontrollable power has brought him into conflict with his fellow heroes and others.
Black Panther is a fictional superhero appearing in American comic books published by Marvel Comics. The character was created by writer-editor Stan Lee and writer-artist Jack Kirby, first appearing in Fantastic Four #52 (cover-dated July 1966) in the Silver Age of Comic Books. Black Panther's real name is T'Challa, king and protector of the fictional African nation of Wakanda. Along with possessing enhanced abilities achieved through ancient Wakandan rituals of drinking the essence of the heart-shaped herb, T'Challa also relies on his proficiency in science, rigorous physical training, hand-to-hand combat skills, and access to wealth and advanced Wakandan technology to combat his enemies.
Black Panther is the first superhero of African descent in mainstream American comics, having debuted years before early black superheroes such as Marvel Comics' the Falcon (1969), Luke Cage (1972) and Blade (1973) or DC Comics' John Stewart in the role of Green Lantern (1971). In one comic book storyline, the Black Panther mantle is handled by Kasper Cole, a multiracial New York City police officer. Beginning as an impersonator, Cole would later take on the moniker of White Tiger and become an ally to T'Challa. The role of Black Panther and leadership of Wakanda is also given to T'Challa's sister Shuri for a short time.
Ant-Man is the name of several fictional superheroes appearing in books published by Marvel Comics. Created by Stan Lee, Larry Lieber and Jack Kirby, Ant-Man's first appearance was in Tales to Astonish #35 (September 1962). The persona was originally the brilliant scientist Hank Pym's superhero alias after inventing a substance that can change size, but Scott Lang and Eric O'Grady also took on the mantle after the original changed his superhero identity to various other aliases, such as Giant-Man, Goliath, and Yellowjacket.
5 Things You Didn't Know About Ant Man
- Control All Insects (Not Just Ants)
- Quantum Telepathy
- Dimension Hopping
- Bio-energy Projection
- Create An Infinite Amount of Space


How are you? Need help?Have something to share?
At Firebeta, we value exceptional communication above all else! Every single one of our Customer Success agents has a sole goal of Making You Successful! That is why our dedicated support team works 24/7/365 to help you make your webmastering journey as smooth as possible!
Time is money! We value our customers and respect their time, that’s why our team always does their best to find out and fix your problem ASAP!
We have several offices around the world full of people, working to provide you an amazing experience with Hostinger.


Webmaking made easy
Mozilla's Thimble App makes it ridiculously simple to create and share your own web pages. Write and edit HTML and CSS right in your browser. Instantly preview your work. Then host and share your finished projects with a single click.


EssaySampler
Contact Us
Please, leave your message here and we will write you back!
Your message
Your email
send
Thank you!
Lorem ipsum


The method receiver, just like the arguments, is always passed by value. This means that when
the method reflect
is called, the Point is actually copied, then the copy is
transformed and thrown away.
If your want a method to modify the content of a struct, then you must declare a pointer receiver type. Try it here.
A useful guide to decide whether to declare a method receiver as a pointer type or not is in the FAQ: Should I define methods on values or pointers?.


I recently had alot of troubles with my Javascript and read alot of the questions wrong in a lot of manners alot of which caused me to have Syntax errors. Which meant i ahd to closely follow my faults and take breaks and come back to the questions again after gathering my thoughts and brainstorming the idea's.
In my own way i found it nice to before mountain biking look over the city of Upper Hutt and all the forestry and Reflect,
this helped me pass through alot of obstacles and prevent me 'most of the time from being blocked'.
I always think its important for me to Reflect on what it is I've doen that day and figure out how my Code has functioned
and maybe think of other ways i could have interpreted the scenario and work on it.


class: center, middle, inverse, title-slide # Getting tidy
### Rockefeller University, Bioinformatics Resource Centre ###
http://rockefelleruniversity.github.io/Intro_To_R_1Day/
--- --- # BRC - Bioinformatic Resource Center We are your local friendly bioinformaticians. We want to help with your problematic data. New or Old. Drop by and say Hi! We are currently in the IT pavillion. # Introduction to Data Wrangling with Tidy --- ## A problem we face when dealing with data regularly. Every dataset is different. Sometimes very different. There are many ways to do things. Everyone has their favorite syntax. The issue: Many fundamental data processing functions exist in *Base R* and beyond. Sometimes they can be inconsistent or unnecessarily complex. Espeically when dealing with non-standard dataframes. The result is code that is confusing and doesn't flow i.e. nested functions --- ## What does it mean to be tidy? Tidyverse is most importantly a philosophy for data analysis that more often then not makes wrangling data easier. The tidyverse community have built what they describe as an "opinionated" group of packages. These packages readily talk to one another. * More efficient code * Easier to remember syntax * Easier to read syntax Their manifesto: https://cran.r-project.org/web/packages/tidyverse/vignettes/manifesto.html --- ## What does it actually mean to be tidy? * A defined vision for coding style in R * A defined vision for data formats in R * A defined vision for package design in R * Unified set of community pushing in a cohesive direction * Critical mass of people to influence the way the whole R community evolves --- ## What are the main tools in the tidyverse? * ggplot2 – making pretty graphs * readr – reading data into R * dplyr – manipulating data * tibble - working with tibbles * tidyr – miscellaneous tools for tidying data * purrr - iterating over data * stringr – working with strings * forcats - working with factors Other tools have now been made for the tidy community. This community also overlaps with bioconductor. But the packages above are the linchpins that hold it together. --- ## What are we doing today Workflow Image for working with data. --- ## What we won't be doing today We already have a course online for plotting, including ggplot. https://rockefelleruniversity.github.io/Plotting_In_R/ # Lets get tidy! --- ## First step lets load in the data we are using today ```r load(file='../dataset/my_tidy.Rdata') ``` --- ## Are all data frames equal? ```r head(df1) ``` ``` ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## 1 35032 Chinook salmon yearling 147 41.26006 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35037 Steelhead juvenile 220 42.70981 ## 5 35038 Steelhead juvenile 152 NA ## 6 35033 Chinook salmon mixed age juvenile 444 62.11528 ``` ```r head(df2) ``` ``` ## salmon_id common_name age_classbylength variable value ## 1 35032 Chinook salmon yearling length_mm 147.00000 ## 2 35032 Chinook salmon yearling IGF1_ng_ml 41.26006 ## 3 35033 Chinook salmon mixed age juvenile length_mm 444.00000 ## 4 35033 Chinook salmon mixed age juvenile IGF1_ng_ml 62.11528 ## 5 35034 Sockeye salmon juvenile length_mm 139.00000 ## 6 35034 Sockeye salmon juvenile IGF1_ng_ml NA ``` ```r head(df3a) ``` ``` ## salmon_id common_name age_classbylength variable value ## 1 35032 Chinook salmon yearling length_mm 147 ## 2 35033 Chinook salmon mixed age juvenile length_mm 444 ## 3 35034 Sockeye salmon juvenile length_mm 139 ## 4 35035 Sockeye salmon juvenile length_mm 121 ## 5 35036 Sockeye salmon juvenile length_mm 112 ## 6 35037 Steelhead juvenile length_mm 220 ``` ```r head(df3b) ``` ``` ## salmon_id common_name age_classbylength variable value ## 1 35032 Chinook salmon yearling IGF1_ng_ml 41.26006 ## 2 35033 Chinook salmon mixed age juvenile IGF1_ng_ml 62.11528 ## 3 35034 Sockeye salmon juvenile IGF1_ng_ml NA ## 4 35035 Sockeye salmon juvenile IGF1_ng_ml NA ## 5 35036 Sockeye salmon juvenile IGF1_ng_ml NA ## 6 35037 Steelhead juvenile IGF1_ng_ml 42.70981 ``` --- ## What is a tidy dataset? A tidy dataset is a data frame (or table) for which the following are true: * Each variable has its own column * Each observation has its own row * Each value has its own cell Our first dataframe is tidy --- ## Why bother? Consistent dataframe layouts help to ensure that all values are present and that relationships between data points are clear. R is a vectorized programming language. R builds data frames from vectors, and R works best when its operation are vectorized. Tidy data utilizes of both of these aspects of R. => Precise and Fast --- ## Lets load in the tidyverse ```r library(tidyverse) ``` ``` ## Registered S3 methods overwritten by 'ggplot2': ## method from ## [.quosures rlang ## c.quosures rlang ## print.quosures rlang ``` ``` ## Registered S3 method overwritten by 'rvest': ## method from ## read_xml.response xml2 ``` ``` ## ── Attaching packages ────────────────────────────────── tidyverse 1.2.1 ── ``` ``` ## ✔ ggplot2 3.1.1 ✔ purrr 0.3.2 ## ✔ tibble 2.1.1 ✔ dplyr 0.8.3 ## ✔ tidyr 1.0.0 ✔ stringr 1.4.0 ## ✔ readr 1.3.1 ✔ forcats 0.4.0 ``` ``` ## ── Conflicts ───────────────────────────────────── tidyverse_conflicts() ── ## ✖ dplyr::filter() masks stats::filter() ## ✖ dplyr::lag() masks stats::lag() ``` --- ## These tools have same logic -> insert graphic here https://github.com/trinker/tidyr_in_a_nutshell # dplyr: A tool to access and manipulate data in a dataframe --- ## Select Select allows you to make a vector or dataframe from a specific variable or variables ```r # Select one variable (common_name) select(df1, common_name) ``` ``` ## # A tibble: 97 x 1 ## common_name ## <chr> ## 1 Chinook salmon ## 2 Sockeye salmon ## 3 Sockeye salmon ## 4 Steelhead ## 5 Steelhead ## 6 Chinook salmon ## 7 Sockeye salmon ## 8 Steelhead ## 9 Steelhead ## 10 Steelhead ## # … with 87 more rows ``` ```r # Select two variables (age_classbylength and common_name) select(df1, age_classbylength, common_name) ``` ``` ## # A tibble: 97 x 2 ## age_classbylength common_name ## <chr> <chr> ## 1 yearling Chinook salmon ## 2 juvenile Sockeye salmon ## 3 juvenile Sockeye salmon ## 4 juvenile Steelhead ## 5 juvenile Steelhead ## 6 mixed age juvenile Chinook salmon ## 7 juvenile Sockeye salmon ## 8 juvenile Steelhead ## 9 juvenile Steelhead ## 10 juvenile Steelhead ## # … with 87 more rows ``` ```r # Select all but one variable (length_mm) select(df1,-length_mm) ``` ``` ## # A tibble: 97 x 4 ## salmon_id common_name age_classbylength IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> ## 1 35032 Chinook salmon yearling 41.3 ## 2 35035 Sockeye salmon juvenile NA ## 3 35036 Sockeye salmon juvenile NA ## 4 35037 Steelhead juvenile 42.7 ## 5 35038 Steelhead juvenile NA ## 6 35033 Chinook salmon mixed age juvenile 62.1 ## 7 35034 Sockeye salmon juvenile NA ## 8 35048 Steelhead juvenile 24.2 ## 9 35049 Steelhead juvenile NA ## 10 35050 Steelhead juvenile 63.5 ## # … with 87 more rows ``` ```r # Select all a range of contiguous varibles (common_name:length_mm) select(df1, common_name:length_mm) ``` ``` ## # A tibble: 97 x 3 ## common_name age_classbylength length_mm ## <chr> <chr> <dbl> ## 1 Chinook salmon yearling 147 ## 2 Sockeye salmon juvenile 121 ## 3 Sockeye salmon juvenile 112 ## 4 Steelhead juvenile 220 ## 5 Steelhead juvenile 152 ## 6 Chinook salmon mixed age juvenile 444 ## 7 Sockeye salmon juvenile 139 ## 8 Steelhead juvenile 288 ## 9 Steelhead juvenile 190 ## 10 Steelhead juvenile 283 ## # … with 87 more rows ``` --- ## Filter Filter allows you to access observations based on specific criteria ```r # Filter all observation where the variable common_name is Sockeye salmon filter(df1, common_name == 'Sockeye salmon') ``` ``` ## # A tibble: 11 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35035 Sockeye salmon juvenile 121 NA ## 2 35036 Sockeye salmon juvenile 112 NA ## 3 35034 Sockeye salmon juvenile 139 NA ## 4 35144 Sockeye salmon juvenile 140 NA ## 5 35147 Sockeye salmon juvenile 115 NA ## 6 35096 Sockeye salmon juvenile 115 NA ## 7 35097 Sockeye salmon juvenile 110 NA ## 8 35098 Sockeye salmon juvenile 112 NA ## 9 35099 Sockeye salmon juvenile 111 NA ## 10 35100 Sockeye salmon juvenile 118 NA ## 11 35119 Sockeye salmon juvenile 122 NA ``` ```r # Filter all observations where the variable common_name is either Sockeye salmon or Chinook Salmon filter(df1, common_name %in% c('Sockeye salmon', 'Chinook salmon')) ``` ``` ## # A tibble: 57 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35033 Chinook salmon mixed age juvenile 444 62.1 ## 5 35034 Sockeye salmon juvenile 139 NA ## 6 35142 Chinook salmon yearling 149 66.5 ## 7 35143 Chinook salmon yearling 204 80.9 ## 8 35144 Sockeye salmon juvenile 140 NA ## 9 35145 Chinook salmon yearling 130 23.4 ## 10 35146 Chinook salmon mixed age juvenile 422 101. ## # … with 47 more rows ``` ```r # Filter all observations where the variable common_name ends with 'salmon'. To do this we use stringr function str_ends recognise strings that end with 'salmon'. filter(df1, str_ends(common_name, 'salmon')) ``` ``` ## # A tibble: 59 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35033 Chinook salmon mixed age juvenile 444 62.1 ## 5 35034 Sockeye salmon juvenile 139 NA ## 6 35142 Chinook salmon yearling 149 66.5 ## 7 35143 Chinook salmon yearling 204 80.9 ## 8 35144 Sockeye salmon juvenile 140 NA ## 9 35145 Chinook salmon yearling 130 23.4 ## 10 35146 Chinook salmon mixed age juvenile 422 101. ## # … with 49 more rows ``` ```r # Filter all observations where the variable length_mm is greater than 200 or less than 120 filter(df1, length_mm > 200 | length_mm < 120) ``` ``` ## # A tibble: 36 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35036 Sockeye salmon juvenile 112 NA ## 2 35037 Steelhead juvenile 220 42.7 ## 3 35033 Chinook salmon mixed age juvenile 444 62.1 ## 4 35048 Steelhead juvenile 288 24.2 ## 5 35050 Steelhead juvenile 283 63.5 ## 6 35051 Steelhead juvenile 279 61.2 ## 7 35052 Steelhead juvenile 235 30.6 ## 8 35053 Steelhead juvenile 230 49.4 ## 9 35056 Steelhead juvenile 208 57.4 ## 10 35057 Steelhead juvenile 240 20.2 ## # … with 26 more rows ``` --- ## CHALLENGE * Create a dataframe of the variables *age* and *IGF* of only the *Steelhead* fish * Create a dataframe of all variables but the *IGF* values, for all fish that begin with *S* --- ## ANSWERS ```r # Create a dataframe of the variables *age* and *IGF* of only the *Steelhead* fish df1_A <- filter(df1, common_name == 'Steelhead') select(df1_A, age_classbylength, IGF1_ng_ml) ``` ``` ## # A tibble: 38 x 2 ## age_classbylength IGF1_ng_ml ## <chr> <dbl> ## 1 juvenile 42.7 ## 2 juvenile NA ## 3 juvenile 24.2 ## 4 juvenile NA ## 5 juvenile 63.5 ## 6 juvenile 61.2 ## 7 juvenile 30.6 ## 8 juvenile 49.4 ## 9 juvenile 56.3 ## 10 juvenile 55.7 ## # … with 28 more rows ``` ```r # Create a dataframe of all variables but the *IGF* values, for all fish that begin with *S* df1_A <- filter(df1, str_starts(common_name,'S')) select(df1_A, -IGF1_ng_ml) ``` ``` ## # A tibble: 49 x 4 ## salmon_id common_name age_classbylength length_mm ## <dbl> <chr> <chr> <dbl> ## 1 35035 Sockeye salmon juvenile 121 ## 2 35036 Sockeye salmon juvenile 112 ## 3 35037 Steelhead juvenile 220 ## 4 35038 Steelhead juvenile 152 ## 5 35034 Sockeye salmon juvenile 139 ## 6 35048 Steelhead juvenile 288 ## 7 35049 Steelhead juvenile 190 ## 8 35050 Steelhead juvenile 283 ## 9 35051 Steelhead juvenile 279 ## 10 35052 Steelhead juvenile 235 ## # … with 39 more rows ``` --- ## Arrange Arrange sorts the dataframe based on a specific variable or variables ```r # Arrange the data based on the variable length_mm arrange(df1, length_mm) ``` ``` ## # A tibble: 97 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35095 Chinook salmon subyearling 90 NA ## 2 35097 Sockeye salmon juvenile 110 NA ## 3 35099 Sockeye salmon juvenile 111 NA ## 4 35036 Sockeye salmon juvenile 112 NA ## 5 35098 Sockeye salmon juvenile 112 NA ## 6 35147 Sockeye salmon juvenile 115 NA ## 7 35096 Sockeye salmon juvenile 115 NA ## 8 35100 Sockeye salmon juvenile 118 NA ## 9 35035 Sockeye salmon juvenile 121 NA ## 10 35119 Sockeye salmon juvenile 122 NA ## # … with 87 more rows ``` ```r # Arrange the data first based on the variable common_name, then secondly based on length_mm in a descending order. arrange(df1, common_name, desc(length_mm)) ``` ``` ## # A tibble: 97 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35033 Chinook salmon mixed age juvenile 444 62.1 ## 2 35146 Chinook salmon mixed age juvenile 422 101. ## 3 35110 Chinook salmon mixed age juvenile 275 81.5 ## 4 35129 Chinook salmon yearling 225 72.7 ## 5 35103 Chinook salmon yearling 216 81.2 ## 6 35115 Chinook salmon yearling 215 53.5 ## 7 35112 Chinook salmon yearling 205 90.5 ## 8 35143 Chinook salmon yearling 204 80.9 ## 9 35079 Chinook salmon yearling 199 53.2 ## 10 35081 Chinook salmon yearling 196 5.56 ## # … with 87 more rows ``` --- ## Mutate Mutate creates a new variable based on some form of computation ```r # A new variable is created based on the calculation of the z-score of the variable IGF1_ng_ml using scale() mutate(df1, scale(IGF1_ng_ml)) ``` ``` ## # A tibble: 97 x 6 ## salmon_id common_name age_classbyleng… length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook sa… yearling 147 41.3 ## 2 35035 Sockeye sa… juvenile 121 NA ## 3 35036 Sockeye sa… juvenile 112 NA ## 4 35037 Steelhead juvenile 220 42.7 ## 5 35038 Steelhead juvenile 152 NA ## 6 35033 Chinook sa… mixed age juven… 444 62.1 ## 7 35034 Sockeye sa… juvenile 139 NA ## 8 35048 Steelhead juvenile 288 24.2 ## 9 35049 Steelhead juvenile 190 NA ## 10 35050 Steelhead juvenile 283 63.5 ## # … with 87 more rows, and 1 more variable: `scale(IGF1_ng_ml)`[,1] <dbl> ``` ```r # A new variable is created called IGFngml_zscore, based on the caluclation of the z-score of the variable IGF1_ng_ml using scale() mutate(df1, IGFngml_zscore = scale(IGF1_ng_ml)) ``` ``` ## # A tibble: 97 x 6 ## salmon_id common_name age_classbyleng… length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook sa… yearling 147 41.3 ## 2 35035 Sockeye sa… juvenile 121 NA ## 3 35036 Sockeye sa… juvenile 112 NA ## 4 35037 Steelhead juvenile 220 42.7 ## 5 35038 Steelhead juvenile 152 NA ## 6 35033 Chinook sa… mixed age juven… 444 62.1 ## 7 35034 Sockeye sa… juvenile 139 NA ## 8 35048 Steelhead juvenile 288 24.2 ## 9 35049 Steelhead juvenile 190 NA ## 10 35050 Steelhead juvenile 283 63.5 ## # … with 87 more rows, and 1 more variable: IGFngml_zscore[,1] <dbl> ``` --- ## CHALLENGE * Create a dataframe with a new variable that contains the rank of the *length* variable. Arrange this new data frame by *IGF* (lowest to highest). * Create a dataframe with a new variable that is *IGF*/*length*. Arrange by this new variable (highest to lowest). --- ## ANSWERS ```r # Create a dataframe with a new variable that contains the rank of the *length* variable. Arrange this new data frame by *IGF* (lowest to highest). df1_A <- mutate(df1, length_rank=rank(length_mm)) arrange(df1_A, IGF1_ng_ml) ``` ``` ## # A tibble: 97 x 6 ## salmon_id common_name age_classbyleng… length_mm IGF1_ng_ml length_rank ## <dbl> <chr> <chr> <dbl> <dbl> <dbl> ## 1 35045 Steelhead juvenile 177 4.46 49 ## 2 35081 Chinook sal… yearling 196 5.56 63.5 ## 3 35062 Steelhead juvenile 198 5.90 65.5 ## 4 35138 Chinook sal… yearling 142 9.17 18 ## 5 35039 Steelhead juvenile 201 9.92 70 ## 6 35135 Chinook sal… yearling 181 11.5 52.5 ## 7 35113 Chinook sal… yearling 162 14.2 38 ## 8 35064 Steelhead juvenile 225 16.0 80.5 ## 9 35082 Chinook sal… yearling 150 17.1 29.5 ## 10 35057 Steelhead juvenile 240 20.2 86 ## # … with 87 more rows ``` ```r #Create a dataframe with a new variable that is *IGF*/*length*. Arrange by this new variable (highest to lowest). df1_A <- mutate(df1, IGF_perlength=IGF1_ng_ml/length_mm) arrange(df1_A, -IGF_perlength) ``` ``` ## # A tibble: 97 x 6 ## salmon_id common_name age_classbyleng… length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35055 Steelhead juvenile 123 55.7 ## 2 35087 Coho salmon yearling 164 73.6 ## 3 35142 Chinook sa… yearling 149 66.5 ## 4 35112 Chinook sa… yearling 205 90.5 ## 5 35092 Chinook sa… yearling 192 81.8 ## 6 35117 Chinook sa… yearling 147 61.2 ## 7 35090 Chinook sa… yearling 167 67.3 ## 8 35143 Chinook sa… yearling 204 80.9 ## 9 35059 Steelhead juvenile 185 73.1 ## 10 35101 Steelhead juvenile 167 65.4 ## # … with 87 more rows, and 1 more variable: IGF_perlength <dbl> ``` --- ## Summarize Summarize applies aggregating or summary function to a group ```r # First we define the common_name as a group. df1_byname <- group_by(df1, common_name) # Summarise is used to count over the grouped common_names summarise(df1_byname, count = n()) ``` ``` ## # A tibble: 4 x 2 ## common_name count ## <chr> <int> ## 1 Chinook salmon 46 ## 2 Coho salmon 2 ## 3 Sockeye salmon 11 ## 4 Steelhead 38 ``` ```r # Summarise is used to calculate mean IGF1_ng_ml over the grouped common_names summarise(df1_byname, IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm = T)) ``` ``` ## # A tibble: 4 x 2 ## common_name IGF1_ng_ml_ave ## <chr> <dbl> ## 1 Chinook salmon 46.8 ## 2 Coho salmon 73.6 ## 3 Sockeye salmon NaN ## 4 Steelhead 46.1 ``` --- ## Group Grouping can also help ask questions with other functions ```r # Filter obsevrations with the 2 smallest length_mm for each grouped common_names filter(df1_byname, rank(length_mm) <= 2) ``` ``` ## # A tibble: 8 x 5 ## # Groups: common_name [4] ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35038 Steelhead juvenile 152 NA ## 2 35055 Steelhead juvenile 123 55.7 ## 3 35145 Chinook salmon yearling 130 23.4 ## 4 35085 Coho salmon yearling 140 NA ## 5 35087 Coho salmon yearling 164 73.6 ## 6 35095 Chinook salmon subyearling 90 NA ## 7 35097 Sockeye salmon juvenile 110 NA ## 8 35099 Sockeye salmon juvenile 111 NA ``` ```r # Filter obsevrations with at least 5 for each grouped common_names filter(df1_byname, n() > 5) ``` ``` ## # A tibble: 95 x 5 ## # Groups: common_name [3] ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35037 Steelhead juvenile 220 42.7 ## 5 35038 Steelhead juvenile 152 NA ## 6 35033 Chinook salmon mixed age juvenile 444 62.1 ## 7 35034 Sockeye salmon juvenile 139 NA ## 8 35048 Steelhead juvenile 288 24.2 ## 9 35049 Steelhead juvenile 190 NA ## 10 35050 Steelhead juvenile 283 63.5 ## # … with 85 more rows ``` ```r # A new variable is created using z-score within the grouped common_names mutate(df1_byname, IGFngml_zscore = scale(IGF1_ng_ml)) ``` ``` ## # A tibble: 97 x 6 ## # Groups: common_name [4] ## salmon_id common_name age_classbyleng… length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook sa… yearling 147 41.3 ## 2 35035 Sockeye sa… juvenile 121 NA ## 3 35036 Sockeye sa… juvenile 112 NA ## 4 35037 Steelhead juvenile 220 42.7 ## 5 35038 Steelhead juvenile 152 NA ## 6 35033 Chinook sa… mixed age juven… 444 62.1 ## 7 35034 Sockeye sa… juvenile 139 NA ## 8 35048 Steelhead juvenile 288 24.2 ## 9 35049 Steelhead juvenile 190 NA ## 10 35050 Steelhead juvenile 283 63.5 ## # … with 87 more rows, and 1 more variable: IGFngml_zscore <dbl> ``` --- ## CHALLENGE 1. Group df1 by the variable *age_class*. 2. Filter to get the biggest 5 by the variable *length* in each group. 3. Summarise this data frame over the variable *length* by calculating the mean. --- ## ANSWERS ```r # Group df1 by the variable *age_class*. df1_A <- group_by(df1, age_classbylength) # Filter to get the biggest 5 by the variable *length* in each group. df1_A <- filter(df1_A, rank(-length_mm) <= 5) # Summarise this data frame over the variable *length* by calculating the mean. summarise(df1_A, mean_length_mm = mean(length_mm, na.rm = T)) ``` ``` ## # A tibble: 4 x 2 ## age_classbylength mean_length_mm ## <chr> <dbl> ## 1 juvenile 274 ## 2 mixed age juvenile 380. ## 3 subyearling 90 ## 4 yearling 213 ``` # Piping from magrittR (%>%): A way to string functions together Piping allows you to pass the result from one expression directly into another. -> same graphic as before , but extend https://github.com/trinker/tidyr_in_a_nutshell --- ## Piping versus not piping ```r # Without pipe df1_byname <- group_by(df1, common_name) summarise(df1_byname, IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm=T)) ``` ``` ## # A tibble: 4 x 2 ## common_name IGF1_ng_ml_ave ## <chr> <dbl> ## 1 Chinook salmon 46.8 ## 2 Coho salmon 73.6 ## 3 Sockeye salmon NaN ## 4 Steelhead 46.1 ``` ```r # With pipe group_by(df1, common_name) %>% summarise(IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm=T)) ``` ``` ## # A tibble: 4 x 2 ## common_name IGF1_ng_ml_ave ## <chr> <dbl> ## 1 Chinook salmon 46.8 ## 2 Coho salmon 73.6 ## 3 Sockeye salmon NaN ## 4 Steelhead 46.1 ``` --- ## Building pipes together ```r # (1) Group by common_name # (2) Filter to all those that have length bigger then 200 # (3) Summarise is used to calculate mean IGF1_ng_ml over the grouped common_names for these larger fish group_by(df1, common_name) %>% filter(length_mm > 200) %>% summarise(IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm = T)) ``` ``` ## # A tibble: 2 x 2 ## common_name IGF1_ng_ml_ave ## <chr> <dbl> ## 1 Chinook salmon 77.9 ## 2 Steelhead 45.3 ``` ```r # (1) Create new variable that is discrete label depending on size of the fish # (2) Group by common_name and size # (3) Summarise is used to calculate mean IGF1_ng_ml over the grouped common_names and sizes mutate(df1, size = if_else(length_mm > 200, 'big_fish', 'small_fish')) %>% group_by(common_name, size) %>% summarise(IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm = T)) ``` ``` ## # A tibble: 6 x 3 ## # Groups: common_name [4] ## common_name size IGF1_ng_ml_ave ## <chr> <chr> <dbl> ## 1 Chinook salmon big_fish 77.9 ## 2 Chinook salmon small_fish 39.5 ## 3 Coho salmon small_fish 73.6 ## 4 Sockeye salmon small_fish NaN ## 5 Steelhead big_fish 45.3 ## 6 Steelhead small_fish 47.3 ``` ```r # (1) Create new variable that is discrete label depending on size of the fish # (2) Group by common_name and size # (3) Summarise is used to calculate mean IGF1_ng_ml over the grouped common_names and sizes # (4) Filter out Coho and Sockeye salmon mutate(df1, size = if_else(length_mm > 200, 'big_fish', 'small_fish')) %>% group_by(common_name, size) %>% summarise(IGF1_ng_ml_ave = mean(IGF1_ng_ml, na.rm = T)) %>% filter(common_name != 'Coho salmon') %>% filter(common_name !='Sockeye salmon') ``` ``` ## # A tibble: 4 x 3 ## # Groups: common_name [2] ## common_name size IGF1_ng_ml_ave ## <chr> <chr> <dbl> ## 1 Chinook salmon big_fish 77.9 ## 2 Chinook salmon small_fish 39.5 ## 3 Steelhead big_fish 45.3 ## 4 Steelhead small_fish 47.3 ``` --- ## You can put these pipes straight into plotting ```r p <- mutate(df1,size=if_else(length_mm>200, 'big_fish', 'small_fish')) %>% group_by(common_name, size) %>% summarize(IGF1_ng_ml_ave=mean(IGF1_ng_ml, na.rm=T)) %>% filter(common_name != 'Coho salmon') %>% filter(common_name != 'Sockeye salmon') %>% ggplot( aes(x = common_name, y = IGF1_ng_ml_ave, group = size, fill = size)) + geom_bar(stat = "identity", position = "dodge") + theme(axis.text.x = element_text(angle = 90)) + scale_fill_brewer(palette = "Paired") p ``` ![](20191009_tidyr_files/figure-html/unnamed-chunk-15-1.png)<!-- --> --- ## CHALLENGE 1. Filter to only look at *juvenile* in variable 2. Group by common_name 3. Create new variable that is z-score of *length* across groups 4. Create boxplot of grouped length zscores --- ## ANSWER ```r df1 %>% filter(age_classbylength == 'yearling') %>% group_by(common_name) %>% mutate(length_zscore = scale(length_mm)) %>% ggplot(aes(x = common_name, y = length_zscore)) + geom_boxplot(fill=c('gold','darkorange')) ``` ![](20191009_tidyr_files/figure-html/unnamed-chunk-16-1.png)<!-- --> # Readr: Reading data into R So we blasted through what being tidy can give you. Now lets start from the beginning and tidy some data. First step is to read in data. ReadR: * read_csv(): comma separated (CSV) files * read_tsv(): tab separated files * read_delim(): general delimited files * read_fwf(): fixed width files * read_table(): tabular files where columns are separated by white-space * read_log(): web log files --- ## ReadR vs Base ```r # Base gives you everything untidy_counts_base <- read.csv("../dataset/hemato_rnaseq_counts.csv") head(untidy_counts_base) ``` ``` ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ``` ```r # ReadR gives you a tibble. # Note the rownames are now their own column. untidy_counts <- read_csv("../dataset/hemato_rnaseq_counts.csv") ``` ``` ## Parsed with column specification: ## cols( ## ENTREZ = col_double(), ## CD34_1 = col_double(), ## ORTHO_1 = col_double(), ## CD34_2 = col_double(), ## ORTHO_2 = col_double() ## ) ``` ```r untidy_counts ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <dbl> <dbl> <dbl> <dbl> <dbl> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` ```r # Tibbles carry and display extra information. While reading in it is easy to specify data type. untidy_counts <- read_csv("../dataset/hemato_rnaseq_counts.csv", col_types = cols( ENTREZ = col_character(), CD34_1 = col_integer(), ORTHO_1 = col_integer(), CD34_2 = col_integer(), ORTHO_2 = col_integer() )) untidy_counts ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <chr> <int> <int> <int> <int> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` # Tibbles --- ## Subsetting tibbles ```r #Can use the same way as base to interact with the tibble dataframe untidy_counts[,1] ``` ``` ## # A tibble: 100 x 1 ## ENTREZ ## <chr> ## 1 350 ## 2 351 ## 3 353 ## 4 354 ## 5 355 ## 6 356 ## 7 357 ## 8 358 ## 9 359 ## 10 360 ## # … with 90 more rows ``` ```r untidy_counts[1,] ``` ``` ## # A tibble: 1 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <chr> <int> <int> <int> <int> ## 1 350 204 0 103 0 ``` ```r #Can also not specify which dimension you pull from. This will default to grabbing the column untidy_counts[1] ``` ``` ## # A tibble: 100 x 1 ## ENTREZ ## <chr> ## 1 350 ## 2 351 ## 3 353 ## 4 354 ## 5 355 ## 6 356 ## 7 357 ## 8 358 ## 9 359 ## 10 360 ## # … with 90 more rows ``` ```r #All the prior outputs have been outputting another tibble. If double brackets are used a vector is returned untidy_counts[[1]] ``` ``` ## [1] "350" "351" "353" "354" "355" "356" "357" "358" "359" "360" "361" ## [12] "362" "363" "364" "366" "367" "368" "369" "372" "373" "374" "375" ## [23] "377" "378" "379" "381" "382" "383" "384" "387" "388" "389" "390" ## [34] "391" "392" "393" "394" "395" "396" "397" "398" "399" "400" "401" ## [45] "402" "403" "405" "406" "407" "408" "409" "410" "411" "412" "414" ## [56] "415" "416" "417" "419" "420" "421" "427" "429" "430" "432" "433" ## [67] "434" "435" "440" "443" "444" "445" "460" "462" "463" "466" "467" ## [78] "468" "471" "472" "473" "474" "475" "476" "477" "478" "479" "480" ## [89] "481" "482" "483" "486" "487" "488" "489" "490" "491" "492" "493" ## [100] "495" ``` ```r #This is also the case if you use the dollar and colname to access a column untidy_counts$ENTREZ ``` ``` ## [1] "350" "351" "353" "354" "355" "356" "357" "358" "359" "360" "361" ## [12] "362" "363" "364" "366" "367" "368" "369" "372" "373" "374" "375" ## [23] "377" "378" "379" "381" "382" "383" "384" "387" "388" "389" "390" ## [34] "391" "392" "393" "394" "395" "396" "397" "398" "399" "400" "401" ## [45] "402" "403" "405" "406" "407" "408" "409" "410" "411" "412" "414" ## [56] "415" "416" "417" "419" "420" "421" "427" "429" "430" "432" "433" ## [67] "434" "435" "440" "443" "444" "445" "460" "462" "463" "466" "467" ## [78] "468" "471" "472" "473" "474" "475" "476" "477" "478" "479" "480" ## [89] "481" "482" "483" "486" "487" "488" "489" "490" "491" "492" "493" ## [100] "495" ``` --- ## CHALLENGE * Read in count data as different data types. What happens when we extract the vector of these counts? --- ## ANSWER ```r untidy_counts_test <- read_csv("../dataset/hemato_rnaseq_counts.csv", col_types = cols( ENTREZ = col_character(), CD34_1 = col_integer(), ORTHO_1 = col_character(), CD34_2 = col_factor(), ORTHO_2 = col_logical() )) ``` ``` ## Warning: 62 parsing failures. ## row col expected actual file ## 2 ORTHO_2 1/0/T/F/TRUE/FALSE 39 '../dataset/hemato_rnaseq_counts.csv' ## 3 ORTHO_2 1/0/T/F/TRUE/FALSE 86 '../dataset/hemato_rnaseq_counts.csv' ## 5 ORTHO_2 1/0/T/F/TRUE/FALSE 16 '../dataset/hemato_rnaseq_counts.csv' ## 8 ORTHO_2 1/0/T/F/TRUE/FALSE 4051 '../dataset/hemato_rnaseq_counts.csv' ## 10 ORTHO_2 1/0/T/F/TRUE/FALSE 1114 '../dataset/hemato_rnaseq_counts.csv' ## ... ....... .................. ...... ..................................... ## See problems(...) for more details. ``` ```r untidy_counts_test$CD34_1 %>% head(n=8) ``` ``` ## [1] 204 15586 842 0 123 1 380 572 ``` ```r untidy_counts_test$ORTHO_1 %>% head(n=8) ``` ``` ## [1] "0" "479" "355" "0" "291" "1" "3" "2225" ``` ```r untidy_counts_test$CD34_2 %>% head(n=8) ``` ``` ## [1] 103 10476 1188 0 139 0 177 597 ## 82 Levels: 103 10476 1188 0 139 177 597 1 46 15 4 51 24 833 2741 ... 3053 ``` ```r untidy_counts_test$ORTHO_2 %>% head(n=8) ``` ``` ## [1] FALSE NA NA FALSE NA FALSE FALSE NA ``` --- ## Converting Tibbles - Back and Forth ```r # Can convert base dataframes into tibbles as_tibble(untidy_counts_base) ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <int> <int> <int> <int> <int> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` ```r # Once it is a tibble it is straight forward to modify the datatype untidy_counts_base <- as_tibble(untidy_counts_base) %>% mutate_at(vars(ENTREZ), as.character) untidy_counts_base ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <chr> <int> <int> <int> <int> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` ```r # Some tools are not tibble friendly. Calling as.data.frame is sufficient to convert it back to a base data frame as.data.frame(untidy_counts_base) %>% head(n=12) ``` ``` ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## 11 361 0 1 0 0 ## 12 362 3 1 15 0 ``` --- ## Make your own tibble. Lets grab some metadata to do this. ```r # Lets load in some packages library(org.Hs.eg.db) library(TxDb.Hsapiens.UCSC.hg19.knownGene) # Lets use the ENTREZ ID as a key keys <- untidy_counts$ENTREZ # We can use the ENTREZ ID to look up Gene Symbol symbols <- select(org.Hs.eg.db, keys = keys,columns = "SYMBOL", keytype = "ENTREZID") # We can use the ENTREZ ID to look up the chormosome the gene resides on chrs <- select(TxDb.Hsapiens.UCSC.hg19.knownGene, keys = keys, columns = "TXCHROM", keytype = "GENEID") # We can use the ENTREZ ID to get a list of genes with grange of their exons geneExons <- exonsBy(TxDb.Hsapiens.UCSC.hg19.knownGene,by = "gene")[keys] # We will then use an apply to get the transcript length from each gene in the list. The transcript length is calculated by first flattening overalpping exons with reduce(), then calculating the length of each exon with width(), then summing upthe total exon length to get our transcript length. txsLength <- sapply(geneExons, function(x){ x %>% reduce() %>% width() %>% sum() }) # Finally we have all this metadata. Lets put it together into a tibble. counts_metadata <- tibble(ID = symbols$ENTREZID, SYMBOL = symbols$SYMBOL, CHR = chrs$TXCHROM, LENGTH = txsLength) ``` --- ## Tidying data up What is wrong with the count dataframe from a tidy viewpoint? *Remember*. These are the rules: * Each variable has its own column * Each observation has its own row * Each value has its own cell ```r untidy_counts ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <chr> <int> <int> <int> <int> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` A single variable with multiple columns --- ## How do we get tidy? - Pivot tools (formerly known as gather/spread ) ```r # Pivot longer allows you to collapse variables single varibles that are spread over multiple columns tidier_counts <- pivot_longer(untidy_counts, cols = c(-ENTREZ), names_to = c("Sample"), values_to = "counts") tidier_counts ``` ``` ## # A tibble: 400 x 3 ## ENTREZ Sample counts ## <chr> <chr> <int> ## 1 350 CD34_1 204 ## 2 350 ORTHO_1 0 ## 3 350 CD34_2 103 ## 4 350 ORTHO_2 0 ## 5 351 CD34_1 15586 ## 6 351 ORTHO_1 479 ## 7 351 CD34_2 10476 ## 8 351 ORTHO_2 39 ## 9 353 CD34_1 842 ## 10 353 ORTHO_1 355 ## # … with 390 more rows ``` ```r # Pivot wider allows you to spread single variables over multiple columns pivot_wider(tidier_counts, names_from = c("Sample"), values_from = "counts") ``` ``` ## # A tibble: 100 x 5 ## ENTREZ CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## <chr> <int> <int> <int> <int> ## 1 350 204 0 103 0 ## 2 351 15586 479 10476 39 ## 3 353<PHONE_NUMBER> 86 ## 4 354 0 0 0 0 ## 5 355 123 291 139 16 ## 6 356 1 1 0 0 ## 7 357 380 3 177 0 ## 8<PHONE_NUMBER> 597 4051 ## 9 359 0 12 1 0 ## 10 360 320 502 46 1114 ## # … with 90 more rows ``` --- ## CHALLENGE * tidy up df2 --- ## ANSWER ```r df2 %>% pivot_wider(values_from = value, names_from = variable) ``` ``` ## # A tibble: 97 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35033 Chinook salmon mixed age juvenile 444 62.1 ## 3 35034 Sockeye salmon juvenile 139 NA ## 4 35035 Sockeye salmon juvenile 121 NA ## 5 35036 Sockeye salmon juvenile 112 NA ## 6 35037 Steelhead juvenile 220 42.7 ## 7 35038 Steelhead juvenile 152 NA ## 8 35039 Steelhead juvenile 201 9.92 ## 9 35040 Steelhead juvenile 200 38.1 ## 10 35042 Steelhead juvenile 247 46.4 ## # … with 87 more rows ``` --- ## What next? ```r tidier_counts ``` ``` ## # A tibble: 400 x 3 ## ENTREZ Sample counts ## <chr> <chr> <int> ## 1 350 CD34_1 204 ## 2 350 ORTHO_1 0 ## 3 350 CD34_2 103 ## 4 350 ORTHO_2 0 ## 5 351 CD34_1 15586 ## 6 351 ORTHO_1 479 ## 7 351 CD34_2 10476 ## 8 351 ORTHO_2 39 ## 9 353 CD34_1 842 ## 10 353 ORTHO_1 355 ## # … with 390 more rows ``` Multiple variables in a single column --- ## Dealing with multiple variables in a single column ```r #Seperate allows you to break a strings in a vraible by a seperator. In this case the cell type and replicate number are broken by underscore tidier_counts <- separate(tidier_counts, Sample, sep = "_", into=c("CellType","Rep"), remove=TRUE) tidier_counts ``` ``` ## # A tibble: 400 x 4 ## ENTREZ CellType Rep counts ## <chr> <chr> <chr> <int> ## 1 350 CD34 1 204 ## 2 350 ORTHO 1 0 ## 3 350 CD34 2 103 ## 4 350 ORTHO 2 0 ## 5 351 CD34 1 15586 ## 6 351 ORTHO 1 479 ## 7 351 CD34 2 10476 ## 8 351 ORTHO 2 39 ## 9 353 CD34 1 842 ## 10 353 ORTHO 1 355 ## # … with 390 more rows ``` ```r #Unite can go the other way if you want to generate a key. We can use this to make sure we have a Key. unite(tidier_counts, Sample, CellType, Rep, remove=FALSE) ``` ``` ## # A tibble: 400 x 5 ## ENTREZ Sample CellType Rep counts ## <chr> <chr> <chr> <chr> <int> ## 1 350 CD34_1 CD34 1 204 ## 2 350 ORTHO_1 ORTHO 1 0 ## 3 350 CD34_2 CD34 2 103 ## 4 350 ORTHO_2 ORTHO 2 0 ## 5 351 CD34_1 CD34 1 15586 ## 6 351 ORTHO_1 ORTHO 1 479 ## 7 351 CD34_2 CD34 2 10476 ## 8 351 ORTHO_2 ORTHO 2 39 ## 9 353 CD34_1 CD34 1 842 ## 10 353 ORTHO_1 ORTHO 1 355 ## # … with 390 more rows ``` ```r #Remember you can always pipe eveyrthing together into a single expression tidy_counts <- untidy_counts %>% pivot_longer(cols = c(-ENTREZ), names_to = c("Sample"), values_to = "counts") %>% separate(Sample, sep = "_", into = c("CellType","Rep"), remove=FALSE) ``` # Joining: We have two related data frames. Can we link them together? --- ## Data frames can be joined on a shared variable *a.k.a.* a key You need a key. We want this to be unique i.e. ENTREZ ID. ```r tidy_counts ``` ``` ## # A tibble: 400 x 5 ## ENTREZ Sample CellType Rep counts ## <chr> <chr> <chr> <chr> <int> ## 1 350 CD34_1 CD34 1 204 ## 2 350 ORTHO_1 ORTHO 1 0 ## 3 350 CD34_2 CD34 2 103 ## 4 350 ORTHO_2 ORTHO 2 0 ## 5 351 CD34_1 CD34 1 15586 ## 6 351 ORTHO_1 ORTHO 1 479 ## 7 351 CD34_2 CD34 2 10476 ## 8 351 ORTHO_2 ORTHO 2 39 ## 9 353 CD34_1 CD34 1 842 ## 10 353 ORTHO_1 ORTHO 1 355 ## # … with 390 more rows ``` ```r counts_metadata ``` ``` ## # A tibble: 100 x 4 ## ID SYMBOL CHR LENGTH ## <chr> <chr> <chr> <int> ## 1 350 APOH chr17 1201 ## 2 351 APP chr21 4480 ## 3 353 APRT chr16 807 ## 4 354 KLK3 chr19 1906 ## 5 355 FAS chr10 6691 ## 6 356 FASLG chr1 1859 ## 7 357 SHROOM2 chrX 8206 ## 8 358 AQP1 chr7 3786 ## 9 359 AQP2 chr12 4179 ## 10 360 AQP3 chr9 2950 ## # … with 90 more rows ``` ```r inner_join(tidy_counts, counts_metadata, by = c("ENTREZ" = "ID")) ``` ``` ## # A tibble: 400 x 8 ## ENTREZ Sample CellType Rep counts SYMBOL CHR LENGTH ## <chr> <chr> <chr> <chr> <int> <chr> <chr> <int> ## 1 350 CD34_1 CD34 1 204 APOH chr17 1201 ## 2 350 ORTHO_1 ORTHO 1 0 APOH chr17 1201 ## 3 350 CD34_2 CD34 2 103 APOH chr17 1201 ## 4 350 ORTHO_2 ORTHO 2 0 APOH chr17 1201 ## 5 351 CD34_1 CD34 1 15586 APP chr21 4480 ## 6 351 ORTHO_1 ORTHO 1 479 APP chr21 4480 ## 7 351 CD34_2 CD34 2 10476 APP chr21 4480 ## 8 351 ORTHO_2 ORTHO 2 39 APP chr21 4480 ## 9 353 CD34_1 CD34 1 842 APRT chr16 807 ## 10 353 ORTHO_1 ORTHO 1 355 APRT chr16 807 ## # … with 390 more rows ``` --- ## There are many ways to join things Inner Join * Keeps all observations in x and y with matching keys Outer Join * A left join keeps all observations in x and those in y with matching keys. * A right join keeps all observations in y and those in x with matching keys. * A full join keeps all observations in x and y --- ## Example - I only want to look at expressed genes ```r # In this pipe I group by gene, summarise the data based on the sum of counts, and filter for anything that has a count greater than 0. expressed_genes <- tidy_counts %>% group_by(ENTREZ) %>% summarise(count_total=sum(counts)) %>% filter(count_total>0) # Left join shows all genes as my full data frame tidy_counts is used as the backbone. The filtered expressed genes is secondary, and has missing values (unexpressed genes) which are filled with NA left_join(tidy_counts, expressed_genes, by = c("ENTREZ" = "ENTREZ")) %>% print(n=20) ``` ``` ## # A tibble: 400 x 6 ## ENTREZ Sample CellType Rep counts count_total ## <chr> <chr> <chr> <chr> <int> <int> ## 1 350 CD34_1 CD34 1 204 307 ## 2 350 ORTHO_1 ORTHO 1 0 307 ## 3 350 CD34_2 CD34 2 103 307 ## 4 350 ORTHO_2 ORTHO 2 0 307 ## 5 351 CD34_1 CD34 1 15586 26580 ## 6 351 ORTHO_1 ORTHO 1 479 26580 ## 7 351 CD34_2 CD34 2 10476 26580 ## 8 351 ORTHO_2 ORTHO 2 39 26580 ## 9 353 CD34_1 CD34 1 842 2471 ## 10 353 ORTHO_1 ORTHO 1 355 2471 ## 11 353 CD34_2 CD34 2 1188 2471 ## 12 353 ORTHO_2 ORTHO 2 86 2471 ## 13 354 CD34_1 CD34 1 0 NA ## 14 354 ORTHO_1 ORTHO 1 0 NA ## 15 354 CD34_2 CD34 2 0 NA ## 16 354 ORTHO_2 ORTHO 2 0 NA ## 17 355 CD34_1 CD34 1 123 569 ## 18 355 ORTHO_1 ORTHO 1 291 569 ## 19 355 CD34_2 CD34 2 139 569 ## 20 355 ORTHO_2 ORTHO 2 16 569 ## # … with 380 more rows ``` ```r # Right join shows only genes that survived filtering as it is using the second dataframe as the backbone for the new dataframe. tidy_counts_expressed <- right_join(tidy_counts,expressed_genes, by = c("ENTREZ" = "ENTREZ")) tidy_counts_expressed %>% print(n=20) ``` ``` ## # A tibble: 376 x 6 ## ENTREZ Sample CellType Rep counts count_total ## <chr> <chr> <chr> <chr> <int> <int> ## 1 350 CD34_1 CD34 1 204 307 ## 2 350 ORTHO_1 ORTHO 1 0 307 ## 3 350 CD34_2 CD34 2 103 307 ## 4 350 ORTHO_2 ORTHO 2 0 307 ## 5 351 CD34_1 CD34 1 15586 26580 ## 6 351 ORTHO_1 ORTHO 1 479 26580 ## 7 351 CD34_2 CD34 2 10476 26580 ## 8 351 ORTHO_2 ORTHO 2 39 26580 ## 9 353 CD34_1 CD34 1 842 2471 ## 10 353 ORTHO_1 ORTHO 1 355 2471 ## 11 353 CD34_2 CD34 2 1188 2471 ## 12 353 ORTHO_2 ORTHO 2 86 2471 ## 13 355 CD34_1 CD34 1 123 569 ## 14 355 ORTHO_1 ORTHO 1 291 569 ## 15 355 CD34_2 CD34 2 139 569 ## 16 355 ORTHO_2 ORTHO 2 16 569 ## 17 356 CD34_1 CD34 1 1 2 ## 18 356 ORTHO_1 ORTHO 1 1 2 ## 19 356 CD34_2 CD34 2 0 2 ## 20 356 ORTHO_2 ORTHO 2 0 2 ## # … with 356 more rows ``` --- ## Filtering joins. ```r #Semi join only keeps observations in x that are matched in y. y is only used as a reference and is not in output semi_join(tidy_counts, expressed_genes) ``` ``` ## Joining, by = "ENTREZ" ``` ``` ## # A tibble: 376 x 5 ## ENTREZ Sample CellType Rep counts ## <chr> <chr> <chr> <chr> <int> ## 1 350 CD34_1 CD34 1 204 ## 2 350 ORTHO_1 ORTHO 1 0 ## 3 350 CD34_2 CD34 2 103 ## 4 350 ORTHO_2 ORTHO 2 0 ## 5 351 CD34_1 CD34 1 15586 ## 6 351 ORTHO_1 ORTHO 1 479 ## 7 351 CD34_2 CD34 2 10476 ## 8 351 ORTHO_2 ORTHO 2 39 ## 9 353 CD34_1 CD34 1 842 ## 10 353 ORTHO_1 ORTHO 1 355 ## # … with 366 more rows ``` ```r #Anti join only keeps observations in x that are not matched in y. y is only used as a reference and is not in output anti_join(tidy_counts, expressed_genes) ``` ``` ## Joining, by = "ENTREZ" ``` ``` ## # A tibble: 24 x 5 ## ENTREZ Sample CellType Rep counts ## <chr> <chr> <chr> <chr> <int> ## 1 354 CD34_1 CD34 1 0 ## 2 354 ORTHO_1 ORTHO 1 0 ## 3 354 CD34_2 CD34 2 0 ## 4 354 ORTHO_2 ORTHO 2 0 ## 5 398 CD34_1 CD34 1 0 ## 6 398 ORTHO_1 ORTHO 1 0 ## 7 398 CD34_2 CD34 2 0 ## 8 398 ORTHO_2 ORTHO 2 0 ## 9 401 CD34_1 CD34 1 0 ## 10 401 ORTHO_1 ORTHO 1 0 ## # … with 14 more rows ``` --- ## CHALLENGE * Calculate CPMs and TPMs -> Show diagram of the math --- ## ANSWER ```r # Use Group to focus computation on each sample. Use mutate to make new variable that is the CPM tidy_counts_expressed <- tidy_counts_expressed %>% group_by(Sample) %>% mutate(CPM=(counts/sum(counts))*1000000) # Join our tidy data to the metadata, then make new variable that is the TPM tidy_counts_expressed <- tidy_counts_expressed %>% inner_join(counts_metadata, by = c("ENTREZ" = "ID")) %>% mutate(TPM=(counts/(LENGTH/1000))/(sum(counts)/1000000)) # Simple X-Y plot comparing TPM and CPM. p <- tidy_counts_expressed %>% ggplot(aes(x=CPM, y=TPM)) + geom_point() + scale_x_continuous(name="log2(CPM)",trans='log2') + scale_y_continuous(name="log2(TPM)",trans='log2') p ``` ![](20191009_tidyr_files/figure-html/unnamed-chunk-30-1.png)<!-- --> # Readr again: Writing your lovely new tibble to file ```r # A key difference compared to base is that it does not write out row names. Tibbles generally don't have rownames. # Theres a wide range of writing options. Can specify the delmiter directly or use a specific function write_delim(tidy_counts_expressed, '../expressed_genes_output.csv', delim =',') write_csv(tidy_counts_expressed, '../expressed_genes_output.csv') ``` --- ## At this point we have covered or touched on the most essential facets of tidy * ~~ggplot2 – making pretty graphs~~ * ~~readr – reading data into R~~ * ~~dplyr – manipulating data~~ * ~~tibble - working with tibbles~~ * ~~tidyr – miscellaneous tools for tidying data~~ * purrr - iterating over data * stringr – working with strings * forcats - working with factors # purrr - Functional programming * Applying functions to datasets * Base people use for loops or apply * Big advantage is that purrr readily handles nested dataframes and has standard outputs --- ## map - tidy way to iterate over a dataset ```r # Map is the tidy equivalent to apply. Here we take our untidy counts, trim of IDs, and then calculate means for each column. By default the output is a list untidy_counts %>% dplyr::select(-ENTREZ) %>% map(mean) ``` ``` ## $CD34_1 ## [1] 1497.67 ## ## $ORTHO_1 ## [1] 822.33 ## ## $CD34_2 ## [1] 1056.85 ## ## $ORTHO_2 ## [1] 329.05 ``` ```r # Same as the above line, but using map_dbl specifies the outputs is going to be a double untidy_counts %>% dplyr::select(-ENTREZ) %>% map_dbl(mean) ``` ``` ## CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## 1497.67 822.33 1056.85 329.05 ``` ```r # Summary also works in this context tidy_counts %>% group_by(Sample) %>% summarize(mean_counts = mean(counts)) ``` ``` ## # A tibble: 4 x 2 ## Sample mean_counts ## <chr> <dbl> ## 1 CD34_1 1498. ## 2 CD34_2 1057. ## 3 ORTHO_1 822. ## 4 ORTHO_2 329. ``` ```r # This is an alternative method for doing this with an tidied frame tidy_counts %>% split(.$Sample) %>% map_dbl(~mean(.$counts)) ``` ``` ## CD34_1 CD34_2 ORTHO_1 ORTHO_2 ## 1497.67 1056.85 822.33 329.05 ``` ```r # pmap is a map variant for dealing with multiple inputs. This can be used to apply a function on a row by row basis list(untidy_counts$ORTHO_1, untidy_counts$ORTHO_2) %>% pmap_dbl(mean) ``` ``` ## [1] 0 479 355 0 291 1 3 2225 12 502 1 ## [12] 1 6 0 0 0 0 1034 1864 728 5 6697 ## [23] 314 4845 1 574 2265 1632 10977 2457 171 193 0 ## [34] 577 304 138 464 67 702 410 0 11 147 0 ## [45] 218 25 824 175 1 111 1819 197 217 45 15 ## [56] 0 1 0 5 1136 255 11889 0 13 0 0 ## [67] 1 151 2520 0 151 15 0 0 118 480 402 ## [78] 6132 771 628 783 0 83 3453 3 121 0 5 ## [89] 18 1392 1454 0 33 1118 153 1263 1 0 3585 ## [100] 0 ``` --- ## Nest - simplifying your dataframe by making it more complex ```r # Nest all the data by sample tidy_counts_nest <- tidy_counts_expressed %>% group_by(Sample) %>% nest() #Looking at tibble it is a new datatype that appears simplified tidy_counts_nest ``` ``` ## # A tibble: 4 x 2 ## # Groups: Sample [4] ## Sample data ## <chr> <S3: vctrs_list_of> ## 1 CD34_1 350 , 351 , 353 , 355 … ## 2 ORTHO_1 350 , 351 , 353 , 355 … ## 3 CD34_2 350 , 351 , 353 , 355 … ## 4 ORTHO_2 350 , 351 , 353 , 355 … ``` ```r tidy_counts_nest$data %>% is() ``` ``` ## [1] "vctrs_list_of" ``` ```r # The data is still there, nested within one of the variables tidy_counts_nest$data[[1]] ``` ``` ## # A tibble: 94 x 10 ## ENTREZ CellType Rep counts count_total CPM SYMBOL CHR LENGTH ## <chr> <chr> <chr> <int> <int> <dbl> <chr> <chr> <int> ## 1 350 CD34 1 204 307 1.36e3 APOH chr17 1201 ## 2 351 CD34 1 15586 26580 1.04e5 APP chr21 4480 ## 3 353 CD34 1 842 2471 5.62e3 APRT chr16 807 ## 4 355 CD34 1 123 569 8.21e2 FAS chr10 6691 ## 5 356 CD34 1 1 2 6.68e0 FASLG chr1 1859 ## 6 357 CD34 1 380 560 2.54e3 SHROO… chrX 8206 ## 7 358 CD34 1 572 7445 3.82e3 AQP1 chr7 3786 ## 8 359 CD34 1 0 13 0. AQP2 chr12 4179 ## 9 360 CD34 1 320 1982 2.14e3 AQP3 chr9 2950 ## 10 361 CD34 1 0 1 0. AQP4 chr18 5217 ## # … with 84 more rows, and 1 more variable: TPM <dbl> ``` ```r # Map can be used to apply functions across nested dataframes. Here we calculate a linear model. This is also saved in the tibble. tidy_counts_nest <- tidy_counts_nest %>% mutate(my_model = map(data, ~lm(CPM ~ TPM, data = .))) tidy_counts_nest ``` ``` ## # A tibble: 4 x 3 ## # Groups: Sample [4] ## Sample data my_model ## <chr> <S3: vctrs_list_of> <list> ## 1 CD34_1 350 , 351 , 353 , 3… <S3: lm> ## 2 ORTHO_1 350 , 351 , 353 , 3… <S3: lm> ## 3 CD34_2 350 , 351 , 353 , 3… <S3: lm> ## 4 ORTHO_2 350 , 351 , 353 , 3… <S3: lm> ``` ```r tidy_counts_nest$my_model %>% is() ``` ``` ## [1] "list" "vector" "AssayData" ## [4] "list_OR_List" "vector_OR_factor" ``` ```r tidy_counts_nest$my_model[[1]] ``` ``` ## ## Call: ## lm(formula = CPM ~ TPM, data = .) ## ## Coefficients: ## (Intercept) TPM ## 3864.980 1.723 ``` ```r # Tidy also has the ability to "tidy" up outputs from common statisitical packages, using broom. library(broom) tidy_counts_nest <- tidy_counts_nest %>% mutate(my_tidy_model = map(my_model, broom::tidy)) tidy_counts_nest ``` ``` ## # A tibble: 4 x 4 ## # Groups: Sample [4] ## Sample data my_model my_tidy_model ## <chr> <S3: vctrs_list_of> <list> <list> ## 1 CD34_1 350 , 351 , 353 … <S3: lm> <tibble [2 × … ## 2 ORTHO_1 350 , 351 , 353 … <S3: lm> <tibble [2 × … ## 3 CD34_2 350 , 351 , 353 … <S3: lm> <tibble [2 × … ## 4 ORTHO_2 350 , 351 , 353 … <S3: lm> <tibble [2 × … ``` ```r tidy_counts_nest$my_tidy_model[[1]] ``` ``` ## # A tibble: 2 x 5 ## term estimate std.error statistic p.value ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 (Intercept) 3865. 1101. 3.51 6.94e- 4 ## 2 TPM 1.72 0.108 16.0 2.75e-28 ``` --- ## Unnest - Expand out dataframes ```r #Unnesting to get everything back into a dataframe is very straightforward tidy_counts_nest %>% unnest(my_tidy_model) ``` ``` ## # A tibble: 8 x 8 ## # Groups: Sample [4] ## Sample data my_model term estimate std.error statistic p.value ## <chr> <S3: vctrs_> <list> <chr> <dbl> <dbl> <dbl> <dbl> ## 1 CD34_1 350 … <S3: lm> (Int… 3865. 1101. 3.51 6.94e- 4 ## 2 CD34_1 350 … <S3: lm> TPM 1.72 0.108 16.0 2.75e-28 ## 3 ORTHO_1 350 … <S3: lm> (Int… 1522. 784. 1.94 5.53e- 2 ## 4 ORTHO_1 350 … <S3: lm> TPM 2.20 0.0696 31.6 3.83e-51 ## 5 CD34_2 350 … <S3: lm> (Int… 4083. 1085. 3.76 2.94e- 4 ## 6 CD34_2 350 … <S3: lm> TPM 1.54 0.0944 16.3 7.03e-29 ## 7 ORTHO_2 350 … <S3: lm> (Int… 1787. 937. 1.91 5.96e- 2 ## 8 ORTHO_2 350 … <S3: lm> TPM 2.21 0.0884 25.0 8.45e-43 ``` ```r #Unnesting can be done sequentially to keep adding to master dataframe tidy_counts_nest %>% unnest(my_tidy_model) %>% unnest(data) ``` ``` ## # A tibble: 752 x 17 ## # Groups: Sample [4] ## Sample ENTREZ CellType Rep counts count_total CPM SYMBOL CHR ## <chr> <chr> <chr> <chr> <int> <int> <dbl> <chr> <chr> ## 1 CD34_1 350 CD34 1 204 307 1.36e3 APOH chr17 ## 2 CD34_1 351 CD34 1 15586 26580 1.04e5 APP chr21 ## 3 CD34_1 353 CD34 1 842 2471 5.62e3 APRT chr16 ## 4 CD34_1 355 CD34 1 123 569 8.21e2 FAS chr10 ## 5 CD34_1 356 CD34 1 1 2 6.68e0 FASLG chr1 ## 6 CD34_1 357 CD34 1 380 560 2.54e3 SHROO… chrX ## 7 CD34_1 358 CD34 1 572 7445 3.82e3 AQP1 chr7 ## 8 CD34_1 359 CD34 1 0 13 0. AQP2 chr12 ## 9 CD34_1 360 CD34 1 320 1982 2.14e3 AQP3 chr9 ## 10 CD34_1 361 CD34 1 0 1 0. AQP4 chr18 ## # … with 742 more rows, and 8 more variables: LENGTH <int>, TPM <dbl>, ## # my_model <list>, term <chr>, estimate <dbl>, std.error <dbl>, ## # statistic <dbl>, p.value <dbl> ``` --- ## CHALLENGE --- ## ANSWER # stringr - working with strings If the data you are working with involves characters from data entry often there will be errors i.e. clinical study metadata or a hand-typed list of genes of interest. Tidying data also means fixing these problems. stringr helps make this easy. * Access and manipulate characters * Deal with whitspace * Pattern Recognition Though stringr is pretty comprehensive and covers most of what you will need, there is a sister package called stringi with even more functionality. --- ## Basic functions to interact with strings Many overlapping functions with base for combining, subsetting, converting and finding strings ```r brc <- c("Tom", "Ji-Dung", "Matt") # Extract substrings from a range. Here the 1st to 3rd character brc %>% str_sub( 1, 3) ``` ``` ## [1] "Tom" "Ji-" "Mat" ``` ```r brc ``` ``` ## [1] "Tom" "Ji-Dung" "Matt" ``` ```r # Extract substrings from a range. Here the 2nd to 2nd to last character brc %>% str_sub( 2, -2) ``` ``` ## [1] "o" "i-Dun" "at" ``` ```r # Assign values back to substrings. Here the 2nd to 2nd to last character is replaced with X. str_sub(brc, 2, -2)<-'X' brc ``` ``` ## [1] "TXm" "JXg" "MXt" ``` --- ## Stripping whitespace ```r brc2 <- c("Tom ", " Ji -Dung", "Matt ") #Trim whitespace from strings brc2 <- str_trim(brc2) brc2 ``` ``` ## [1] "Tom" "Ji -Dung" "Matt" ``` ```r #Can add whitespace to strings to get consistent length. Here all are 10 characters str_pad(brc2, width=10, side='left') ``` ``` ## [1] " Tom" " Ji -Dung" " Matt" ``` --- ## Converting strings - Capitalization ```r #Lets reuse our counts tibble. pull from dplyr can be used to grab a tibble column and make it into a vector tidy_counts_expressed %>% pull(SYMBOL) %>% head() ``` ``` ## [1] "APOH" "APOH" "APOH" "APOH" "APP" "APP" ``` ```r #Here we pull our gene symbols from our tibble into a vector, and then convert them into title style capitalization tidy_counts_expressed %>% pull(SYMBOL) %>% str_to_title() %>% head() ``` ``` ## [1] "Apoh" "Apoh" "Apoh" "Apoh" "App" "App" ``` ```r #String manipulation functions can be used on tibbles using mutate. Here we convert gene symbols to title style capitalization tidy_counts_expressed %>% mutate(SYMBOL = str_to_title(SYMBOL)) ``` ``` ## # A tibble: 376 x 11 ## # Groups: Sample [4] ## ENTREZ Sample CellType Rep counts count_total CPM SYMBOL CHR ## <chr> <chr> <chr> <chr> <int> <int> <dbl> <chr> <chr> ## 1 350 CD34_1 CD34 1 204 307 1.36e3 Apoh chr17 ## 2 350 ORTHO… ORTHO 1 0 307 0. Apoh chr17 ## 3 350 CD34_2 CD34 2 103 307 9.75e2 Apoh chr17 ## 4 350 ORTHO… ORTHO 2 0 307 0. Apoh chr17 ## 5 351 CD34_1 CD34 1 15586 26580 1.04e5 App chr21 ## 6 351 ORTHO… ORTHO 1 479 26580 5.82e3 App chr21 ## 7 351 CD34_2 CD34 2 10476 26580 9.91e4 App chr21 ## 8 351 ORTHO… ORTHO 2 39 26580 1.19e3 App chr21 ## 9 353 CD34_1 CD34 1 842 2471 5.62e3 Aprt chr16 ## 10 353 ORTHO… ORTHO 1 355 2471 4.32e3 Aprt chr16 ## # … with 366 more rows, and 2 more variables: LENGTH <int>, TPM <dbl> ``` ```r # Here we convert CHR annotation to capitals tidy_counts_expressed %>% mutate(CHR = str_to_upper(CHR)) ``` ``` ## # A tibble: 376 x 11 ## # Groups: Sample [4] ## ENTREZ Sample CellType Rep counts count_total CPM SYMBOL CHR ## <chr> <chr> <chr> <chr> <int> <int> <dbl> <chr> <chr> ## 1 350 CD34_1 CD34 1 204 307 1.36e3 APOH CHR17 ## 2 350 ORTHO… ORTHO 1 0 307 0. APOH CHR17 ## 3 350 CD34_2 CD34 2 103 307 9.75e2 APOH CHR17 ## 4 350 ORTHO… ORTHO 2 0 307 0. APOH CHR17 ## 5 351 CD34_1 CD34 1 15586 26580 1.04e5 APP CHR21 ## 6 351 ORTHO… ORTHO 1 479 26580 5.82e3 APP CHR21 ## 7 351 CD34_2 CD34 2 10476 26580 9.91e4 APP CHR21 ## 8 351 ORTHO… ORTHO 2 39 26580 1.19e3 APP CHR21 ## 9 353 CD34_1 CD34 1 842 2471 5.62e3 APRT CHR16 ## 10 353 ORTHO… ORTHO 1 355 2471 4.32e3 APRT CHR16 ## # … with 366 more rows, and 2 more variables: LENGTH <int>, TPM <dbl> ``` --- ## Finding patterns ```r #Find patterns in different ways #Detect gives a T/F whether the pattern 'salmon' is present in vector df1 %>% pull(common_name) %>% str_detect('salmon') ``` ``` ## [1] TRUE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE ## [12] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [23] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE ## [34] TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE ## [45] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## [56] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE FALSE TRUE ## [67] FALSE FALSE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## [78] TRUE FALSE FALSE FALSE FALSE TRUE TRUE TRUE TRUE TRUE TRUE ## [89] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ``` ```r #Subset returns the match if the pattern 'salmon' is present in vector df1 %>% dplyr::pull(common_name) %>% str_subset('salmon') ``` ``` ## [1] "Chinook salmon" "Sockeye salmon" "Sockeye salmon" "Chinook salmon" ## [5] "Sockeye salmon" "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [9] "Chinook salmon" "Chinook salmon" "Sockeye salmon" "Chinook salmon" ## [13] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [17] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Coho salmon" ## [21] "Coho salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [25] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [29] "Sockeye salmon" "Sockeye salmon" "Sockeye salmon" "Sockeye salmon" ## [33] "Sockeye salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [37] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [41] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [45] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [49] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [53] "Chinook salmon" "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [57] "Chinook salmon" "Chinook salmon" "Chinook salmon" ``` ```r #Ends is similar to detect as it gives gives a T/F whether the pattern 'salmon' is present in vector, but the pattern has to be at the end. df1 %>% dplyr::pull(common_name) %>% str_ends('salmon') ``` ``` ## [1] TRUE TRUE TRUE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE ## [12] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE ## [23] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE TRUE ## [34] TRUE TRUE TRUE FALSE FALSE FALSE FALSE FALSE FALSE TRUE TRUE ## [45] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## [56] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE FALSE TRUE ## [67] FALSE FALSE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ## [78] TRUE FALSE FALSE FALSE FALSE TRUE TRUE TRUE TRUE TRUE TRUE ## [89] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE ``` ```r df1 %>% filter(str_ends(common_name,'salmon')) ``` ``` ## # A tibble: 59 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35033 Chinook salmon mixed age juvenile 444 62.1 ## 5 35034 Sockeye salmon juvenile 139 NA ## 6 35142 Chinook salmon yearling 149 66.5 ## 7 35143 Chinook salmon yearling 204 80.9 ## 8 35144 Sockeye salmon juvenile 140 NA ## 9 35145 Chinook salmon yearling 130 23.4 ## 10 35146 Chinook salmon mixed age juvenile 422 101. ## # … with 49 more rows ``` ```r #Count gives you the total number of times your pattern appears in each chracter in the vector df1 %>% dplyr::pull(common_name) %>% str_count('salmon') ``` ``` ## [1] 1 1 1 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 ## [36] 1 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 ## [71] 1 1 1 1 1 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ``` ```r df1 %>% dplyr::pull(common_name) %>% str_count('o') ``` ``` ## [1] 3 2 2 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 3 2 3 3 ## [36] 2 0 0 0 0 0 0 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 2 2 2 2 0 3 0 0 3 3 ## [71] 3 3 3 3 3 3 3 2 0 0 0 0 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 ``` ```r #Replace df1 %>% dplyr::pull(common_name) %>% str_replace_all('Steelhead','Steelhead trout' ) ``` ``` ## [1] "Chinook salmon" "Sockeye salmon" "Sockeye salmon" ## [4] "Steelhead trout" "Steelhead trout" "Chinook salmon" ## [7] "Sockeye salmon" "Steelhead trout" "Steelhead trout" ## [10] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [13] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [16] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [19] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [22] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [25] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [28] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [31] "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [34] "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [37] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [40] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [43] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [46] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [49] "Chinook salmon" "Chinook salmon" "Coho salmon" ## [52] "Coho salmon" "Chinook salmon" "Chinook salmon" ## [55] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [58] "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [61] "Sockeye salmon" "Sockeye salmon" "Sockeye salmon" ## [64] "Sockeye salmon" "Steelhead trout" "Chinook salmon" ## [67] "Steelhead trout" "Steelhead trout" "Chinook salmon" ## [70] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [73] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [76] "Chinook salmon" "Chinook salmon" "Sockeye salmon" ## [79] "Steelhead trout" "Steelhead trout" "Steelhead trout" ## [82] "Steelhead trout" "Chinook salmon" "Chinook salmon" ## [85] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [88] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [91] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [94] "Chinook salmon" "Chinook salmon" "Chinook salmon" ## [97] "Chinook salmon" ``` ```r df1 %>% mutate(common_name = str_replace_all(common_name,'Steelhead','Steelhead trout' )) ``` ``` ## # A tibble: 97 x 5 ## salmon_id common_name age_classbylength length_mm IGF1_ng_ml ## <dbl> <chr> <chr> <dbl> <dbl> ## 1 35032 Chinook salmon yearling 147 41.3 ## 2 35035 Sockeye salmon juvenile 121 NA ## 3 35036 Sockeye salmon juvenile 112 NA ## 4 35037 Steelhead trout juvenile 220 42.7 ## 5 35038 Steelhead trout juvenile 152 NA ## 6 35033 Chinook salmon mixed age juvenile 444 62.1 ## 7 35034 Sockeye salmon juvenile 139 NA ## 8 35048 Steelhead trout juvenile 288 24.2 ## 9 35049 Steelhead trout juvenile 190 NA ## 10 35050 Steelhead trout juvenile 283 63.5 ## # … with 87 more rows ``` --- ## CHALLENGE --- ## ANSWER # forcats - Handling factors Factors are a data type that R uses to handle fixed categorical variables that have a known set of possible values. Factors are ordered, allowing hierachy to be presevred in relatively simple vectors. --- ## Making a factor - This is all base ```r # Vectors are easy to turn into factors with factor() tidy_counts_expressed_samples <- tidy_counts_expressed %>% pull(Sample) %>% factor() tidy_counts_expressed_samples %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 CD34_2 ORTHO_2 CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## [9] CD34_1 ORTHO_1 ## Levels: CD34_1 CD34_2 ORTHO_1 ORTHO_2 ``` ```r # Can also modify the data type of a tibble column with as_facotr, in an approach we have used before. tidy_counts_expressed %>% ungroup() %>% mutate_at(vars(Sample), as_factor) ``` ``` ## # A tibble: 376 x 11 ## ENTREZ Sample CellType Rep counts count_total CPM SYMBOL CHR ## <chr> <fct> <chr> <chr> <int> <int> <dbl> <chr> <chr> ## 1 350 CD34_1 CD34 1 204 307 1.36e3 APOH chr17 ## 2 350 ORTHO… ORTHO 1 0 307 0. APOH chr17 ## 3 350 CD34_2 CD34 2 103 307 9.75e2 APOH chr17 ## 4 350 ORTHO… ORTHO 2 0 307 0. APOH chr17 ## 5 351 CD34_1 CD34 1 15586 26580 1.04e5 APP chr21 ## 6 351 ORTHO… ORTHO 1 479 26580 5.82e3 APP chr21 ## 7 351 CD34_2 CD34 2 10476 26580 9.91e4 APP chr21 ## 8 351 ORTHO… ORTHO 2 39 26580 1.19e3 APP chr21 ## 9 353 CD34_1 CD34 1 842 2471 5.62e3 APRT chr16 ## 10 353 ORTHO… ORTHO 1 355 2471 4.32e3 APRT chr16 ## # … with 366 more rows, and 2 more variables: LENGTH <int>, TPM <dbl> ``` ```r tidy_counts_expressed %>% ungroup() %>% mutate_at(vars(Sample), as_factor) %>% pull(Sample) %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 CD34_2 ORTHO_2 CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## [9] CD34_1 ORTHO_1 ## Levels: CD34_1 ORTHO_1 CD34_2 ORTHO_2 ``` ```r # When you factorize you can use a vector to determine the order my_levels1<-c('ORTHO_1','ORTHO_2','CD34_1','CD34_2') tidy_counts_expressed %>% pull(Sample) %>% factor(levels = my_levels1 ) %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 CD34_2 ORTHO_2 CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## [9] CD34_1 ORTHO_1 ## Levels: ORTHO_1 ORTHO_2 CD34_1 CD34_2 ``` ```r # When you factorize anything not in the given levels is turned to NA my_levels2<-c('ORTHO_1','CD34_1') tidy_counts_expressed %>% pull(Sample) %>% factor(levels = my_levels2 ) %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 <NA> <NA> CD34_1 ORTHO_1 <NA> <NA> ## [9] CD34_1 ORTHO_1 ## Levels: ORTHO_1 CD34_1 ``` ```r # Its straightforward to grab the levels from the factor tidy_counts_expressed_samples %>% levels() ``` ``` ## [1] "CD34_1" "CD34_2" "ORTHO_1" "ORTHO_2" ``` --- ## Why do we factorize? For order: ```r tidy_counts_expressed %>% ungroup() %>% mutate_at(vars(Sample), as_factor) %>% group_by(Sample) %>% summarize(mean_count=mean(counts)) %>% ggplot(aes(x=Sample, y= mean_count)) + geom_bar(stat = "identity") + theme(axis.text.x = element_text(angle = 90)) ``` ![](20191009_tidyr_files/figure-html/unnamed-chunk-42-1.png)<!-- --> --- ## Changing the order ```r #fct_relevel - reorder manually tidy_counts_expressed %>% ungroup() %>% mutate_at(vars(Sample), as_factor) %>% mutate(Sample = fct_relevel(Sample, my_levels1)) %>% pull(Sample) %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 CD34_2 ORTHO_2 CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## [9] CD34_1 ORTHO_1 ## Levels: ORTHO_1 ORTHO_2 CD34_1 CD34_2 ``` ```r #fct_relevel - reorder manually tidy_counts_expressed %>% ungroup() %>% mutate_at(vars(Sample), as_factor) %>% mutate(Sample = fct_relevel(Sample, my_levels2)) %>% pull(Sample) %>% head(n=10) ``` ``` ## [1] CD34_1 ORTHO_1 CD34_2 ORTHO_2 CD34_1 ORTHO_1 CD34_2 ORTHO_2 ## [9] CD34_1 ORTHO_1 ## Levels: ORTHO_1 CD34_1 CD34_2 ORTHO_2 ``` ```r #fct_reorder - reorder based on the data.  Hadley wrote a paper about why he thinks tidy data is best: www.jstatsoft.org/v59/i10/paper. There is a lot of support for all things tidy at: https://www.tidyverse.org/ (This includes really great cheat sheets for each tool) --- ## Tidy packages to check out: plyranges: dplyr equivalent for working with ranges i.e. Granges readxl: This package is very useful when you want to import Excel sheets in R googledrive: Interact with your googledrive through R lubridate and hms: Allow managin of calendar and time formats magrittr: piping tools broom: helps tidy up standard base function i.e. lm or t.test tidymodels: A collection of tools for preparing for and validating model functions --- ## Other Good Resources ggplot https://rockefelleruniversity.github.io/Plotting_In_R/ R for Data Science text book https://r4ds.had.co.nz/ Data Science with R text book http://garrettgman.github.io/ Cancer Research UK - Intermediate R course https://bioinformatics-core-shared-training.github.io/r-intermediate/ # Thanks *BRC* IT Pavillion Tom Carroll<EMAIL_ADDRESS>Ji-Dung Luo<EMAIL_ADDRESS>Matt Paul<EMAIL_ADDRESS>*Data Science Research Group* Eli: Doug: Greg:


Last updated:
N/A
Highlight same area path
View by PBIs
on
All
AreaPath
AssignedTo
Title
Parent Title
State
Theme:
Classic
Colorful
Solid
Please wait while loading data ...


Track coach Alberto Salazar files doping case appeal at CAS
LAUSANNE, Switzerland (AP) - The Court of Arbitration for Sport says it has registered an appeal by track coach Alberto Salazar against his ban for doping violations, though a hearing will take several months to prepare.
CAS says Salazar and Dr. Jeffrey Brown appealed against their four-year bans by the United States Anti-Doping Agency.
After a multi-year USADA investigation, Salazar and Brown were found guilty of doping violations linked to the Nike Oregon Project training camp. USADA said Salazar ran experiments with supplements and testosterone, and possessed and trafficked the banned substance.
The case also related to falsified and incomplete medical records that disguised the work.
CAS says Salazar and Brown asked for more time to file “written submissions and evidence,” adding the hearing is “unlikely to take place before March.”
Verdicts typically take at least a further several weeks.
More AP sports: https://apnews.com/apf-sports and https://twitter.com/AP_Sports
Sign up for Daily Newsletters
Copyright © 2019 The Washington Times, LLC.
Please read our comment policy before commenting.