The Bird's Christmas Carol
by Kate Douglas Wiggin
In The Bird's Christmas Carol, the arrival of Carol into the Bird family brings joy and sadness throughout the years, but her presence reminds everyone of the real meaning of Christmas.
Readability levels for passages on Lit2Go are reported as Flesch-Kincaid grade levels which are roughly equivalent to U.S. grade levels.
In The Bird's Christmas Carol, the arrival of Carol into the Bird family brings joy and sadness throughout the years, but her presence reminds everyone of the real meaning of Christmas.
The Castle of Otranto is considered the first "gothic" novel, a genre that loves melodrama, mystery, hidden places, ancestral curses, and fainting heroines. Its roots are the "romance," which was a tale of heroism (not love as it is now known), and the Romantic movement in literature, which focused on emotion and the sublimity of nature. When The Castle of Otranto was first published, it was said to be a translation of a lost medieval transcript, and received positive attention. But when it was next published, the truth was revealed--that the story was quite modern and written by a priviledged author. Critics then panned it, but it survives today as the seminal Gothic literary novel.
The Little Lame Prince and his Travelling Cloak (often published under its shorter title The Little Lame Prince) is a story for children written by Dinah Maria Mulock Craik and first published in 1875. In the story, a young prince whose legs are paralysed due to a childhood trauma is given a magical travelling cloak by his fairy godmother; he uses this cloak to go on various adventures, and develops great wisdom and empathy in the process.
The Sea-Wolf is a psychological adventure novel by American novelist Jack London about a literary critic and other survivors of an ocean collision who come under the dominance of Wolf Larsen, the powerful and amoral sea captain who rescues them.
Tik-Tok of Oz is the eighth Land of Oz book written by L. Frank Baum. Published in 1914, the book actually has little to do with Tik-Tok and is primarily the quest of the Shaggy Man (introduced in The Road to Oz) to rescue his brother, and his resulting conflict with the Nome King.
The Tin Woodman of Oz is the twelfth Land of Oz book written by L. Frank Baum and was originally published in 1918. The Tin Woodman is unexpectedly reunited with his Munchkin sweetheart Nimmie Amee from the days when he was flesh and blood. This was a backstory from The Wizard of Oz.
Treasure Island is an adventure novel by Robert Louis Stevenson, narrating a tale of “buccaneers and buried gold.” Traditionally considered a coming of age story, its influence on popular lore about pirates can not be overestimated.
The Call of the Wild is a novel by American writer Jack London. The plot concerns a previously domesticated happy dog named Buck, whose primordial instincts return after a series of events leads to his serving as a sled dog in the Yukon during the 19th-century Klondike Gold Rush, in which sled dogs were bought at generous prices.
Published in 1903, The Call of the Wild is London's most-read book, and it is generally considered his best, the masterpiece of his so-called "early period". Because the protagonist is a dog, it is sometimes classified as a juvenile novel, suitable for children, but it is dark in tone and contains numerous scenes of cruelty and violence.
London followed the book in 1906 with White Fang, a companion novel with many similar plot elements and themes as Call of the Wild, although following a mirror image plot in which a wild wolf becomes civilized by a mining expert from San Francisco named Weedon Scott.

Babbitt is a novel by Sinclair Lewis. Largely a satire of American culture, society, and behavior, it critiques the vacuity of middle-class American life and its pressure on individuals toward conformity.
Behind the Scenes Or, Thirty Years a Slave and Four Years in the White House is an autobiographical narrative by Elizabeth Keckley. In it she tells the story of her life as a slave and her time as a seamstress for Mrs. Lincoln in the White House.
Anne of Green Gables is a coming-of-age novel about Anne Shirley, the Cuthbert's, and the community of Avonlea on Prince Edward Island.
David Copperfield is a novel by Charles Dickens. Like most of his works, it originally appeared in serial form a year earlier. Many elements within the novel follow events in Dickens' own life, and it is probably the most autobiographical of all of his novels.
The Invisible Man is a suspense novel by H.G. Wells, narrating the tale of "Griffin", a scientist who undergoes an irreversible procedure, the results of which eventually drive him insane.
Peter Pan (also known as the Boy Who Wouldn't Grow Up or Peter and Wendy) is the story of a mischievous little boy who can fly, and his adventures on the island of Neverland with Wendy Darling and her brothers, the fairy Tinker Bell, the Lost Boys, the Indian princess Tiger Lily, and the pirate Captain Hook.
Little Women concerns the lives and loves of four sisters growing up during the American Civil War. It was based on Alcott's own experiences as a child in Germantown, Pennsylvania with her three sisters, Anna, May, and Elizabeth.
The Lion wakes as a mouse runs across his face. He caught the mouse who begs to be let go with a promise to help the Lion someday. Later, the Lion is captured by hunters. The mouse gnaws the rope and sets the Lion free.
A Man and a Lion talked about the strength of men and lions. Each believed his species was stronger. The Man showed the Lion a statue of Hercules overcoming the Lion. The Lion believed this proved nothing because a man made the statue.
A Man with an axe begged the Trees for a branch. The good-natured Trees gave him one of their branches. The man fixed it into the axe and began cutting down trees.
A short story about the first days of spring.
A short story about a caterpillar’s transformation into a butterfly.
A short story about the rain and the first few days of March/Spring.
Each of three sons goes out to see the world. The first two are cruel to two beggars and birds and end up killed by an ogress. The third brother is kind to the beggars and the birds, and they help him win out over the ogress and earn many rewards.
Poe’s character rationalizes his thinking to convince the reader of his sanity and regrets.
Von Kempelen discovers how to turn lead into gold.
Fortunato pays the price for insulting the Montressor family name.
A sailor recounts his experience with the Maelstrom, a dangerous whirlpool off the coast of Norway.
This is a tale of an experiment to hypnotize a man on his death bed. The experiment works—but with horrifying results!
The narrator discovers a long-lost Oriental text called Tellmenow Isitsoornot, and is shocked to find the real story of the grand vizier's daughter Scheherazade in Arabian Nights. Then the narrator summarizes what he finds in Isitsoornot.
In this story, a demon recounts the curses he placed on the beauty that surrounded him.
Poe’s character rationalizes his thinking to convince the reader of his sanity and regrets.
An unnamed narrator tells how a Parisian detective, Auguste Dupin, solves a case of a “purloined letter.” The letter belonged to the Queen, and the man who took it had switched it with a plain letter, and was using the information contained in the stolen letter to blackmail the Queen. The police Prefect wants Dupin to figure out how to catch the man, and Dupin reasons his way through the case, eventually nabbing the thief by using his own technique against him—switching letters back.
The author discovers and enchanted island of the Fays and witnessess the demise of a Fay.
About the torments endured by a prisoner of the revived Spanish Inquisition.
The plague is sweeping the countryside, so a prince takes many friends into his castle to save them. They live grandly and have a masked ball, but Death comes to the ball and kills everyone anyway.
An Irishman meets an enchanted princess and has to overcome obstacles to finally marry her.


Welcome to the homepage of the PsychoAcoustics Lab at ASU! Our group consists of (pictured left-to-right)
Sid Bacon, Christopher Brown, Kumiko Boike, Kang Li, Nicole Scherrer, Erica Williams, Kate Helms-Tillery,
Farris Walling, and Bill Yost. We are located in the Lattie F. Coor building, on the ASU campus in
Tempe, Arizona.


new CanvasHelper(parent)
Manages the Annotator's canvas, passing user input to the tools and performing all rendering functionality.
Parameters:
| Name | Type | Description |
|---|---|---|
parent |
Annotator | The Annotator using this helper instance. |
- Source:
Members
canvas :Canvas
The HTML canvas object
Type:
- Canvas
- Source:
curScale :Number
The current scaling level
Type:
- Number
- Source:
defScale :Number
The default scaling level for the current image (see CanvasHelper#calcZoom)
Type:
- Number
- Source:
g :GraphicsContext
The canvas' attached graphics context, used for drawing
Type:
- GraphicsContext
- Source:
hlt :SuperShape
Annotation to highlight independently of the currently selected annotation
Type:
- Source:
xOffs :Number
The current x offset (pan)
Type:
- Number
- Source:
yOffs :Number
The current y offset (pan)
Type:
- Number
- Source:
Methods
calcZoom()
Calculates the correct default zoom level, taking into account aspect ratio to keep the image fully in view at the furthest zoom level.
- Source:
doPan(x, y)
Pans the view of the image and annotations.
Parameters:
| Name | Type | Description |
|---|---|---|
x |
Number | Distance to pan horizontally |
y |
Number | Distance to pan vertically |
- Source:
drawAnn(ann, fInd)
Draws an annotation to the canvas.
Parameters:
| Name | Type | Description |
|---|---|---|
ann |
SuperShape | Annotation to draw. |
fInd |
Number | The index of the feature the annotation belongs to. |
- Source:
drawPt(pt)
Draws a point as a small circle to the canvas.
Parameters:
| Name | Type | Description |
|---|---|---|
pt |
Object |
- Source:
imgLoaded(img)
Finalizes image load, registering the image's true width and height. Calls CanvasHelper#calcZoom.
This is called automatically when a new image finishes loading.
Parameters:
| Name | Type | Description |
|---|---|---|
img |
Image |
- Source:
ptToImg(xin, yin) → {Object}
Utility function which transforms a point from screen to image space according to the current pan and zoom settings.
Parameters:
| Name | Type | Description |
|---|---|---|
xin |
Number | |
yin |
Number |
- Source:
Returns:
Transformed point
- Type
- Object
repaint()
Re-draws the canvas - updates the view of the image and its annotations.
- Source:
reset(w, h)
Updates the canvas to new dimensions and resets the pan and zoom to their default levels.
Parameters:
| Name | Type | Description |
|---|---|---|
w |
Number | New canvas width |
h |
Number | New canvas height |
- Source:
scaleDist(dist) → {Number}
Utility function which transforms a distance from screen to image space.
Parameters:
| Name | Type | Description |
|---|---|---|
dist |
Number |
- Source:
Returns:
Scaled distance
- Type
- Number
setHlt(ann)
Sets the annotation which is to be highlighted.
Parameters:
| Name | Type | Description |
|---|---|---|
ann |
SuperShape |
- Source:
zoom(scale)
Zooms in or out of the image and annotations.
Parameters:
| Name | Type | Description |
|---|---|---|
scale |
Number | Scaling factor, e.g. 2 to zoom in by 2x |
- Source:


Neighborhood Eats: 'While in Kathmandu'
by Lauren GlassbergRIDGEWOOD, Queens (WABC) -- 'While in Kathmandu' is a taste of Nepal in Queens.
Kathmandu is the capital of Nepal, and the owner of the restaurant is paying homage to his roots, which pull strongly even though he has lived in New York half his life.
The restaurant is located at 758 Seneca Ave in Ridgewood.
Pork Momo
Ingredients :
1 lb Minced pork
Chopped Cabbage
1 yellow onion, chopped
2 tsp chopped cilantro
2 scallions, chopped
2 tsp Ground Cumin
2 tsp Ground Cilantro
2 tsp Turmeric
Salt (to taste)
1 tsp Black pepper
Dumpling/momo wrappers (available in Southeast Asian markets)
Directions:
Mix all ingredients (except for the momo wrappers) in a bowl.
Take dumpling wrapper, lightly wet it with water and then spoon 1 tsp of pork filling onto wrapper, and crimp closed
Put dumplings in steamer and steam for 15 minutes.
Can be served over johl sauce. Or with your favorite chutney.


Ashley Benson & Gal Pal Pick Up Her Pups From The Vet
Ashley Benson carries one of her cute dogs out of the veterinary hospital on Friday (November 22) in Los Angeles.
The 29-year-old actress was joined by her pal Theresa Picciallo.
PHOTOS: Check out the latest pics of Ashley Benson
The night before, Ashley and girlfriend Cara Delevingne stepped out to catch a performance of Jagged Little Pill on Broadway in New York City.
Ashley recently shared a cute selfie showing off her makeup look and praised the makeup artist for her look.
“When my girl kills it,” she captioned the pic. Check it out below!


The pipeline performs the following main steps:
Visit http://newsfeed.ijs.si/visual_demo/ for a real-time visualization of the news stream.
We currently offer two streams:
Each stream of articles gets xml-formatted and segmented by time into .gz
files, each a few MB in size.
The streams are password-protected but free for research use. To obtain a username, please contact Mitja Trampus and/or Blaz Novak at<EMAIL_ADDRESS>Note: Due to the streaming nature of the pipeline, the articles in the gzipped files are only approximately chronological; they are sorted in the order in which they were processed rather than published.
The stream is accessible at http://newsfeed.ijs.si/stream/.
The URL accepts an optional ?after=TIMESTAMP
parameter, where TIMESTAMP
takes the ISO format yyyy-mm-ddThh:mm:ssZ
(Z
denotes GMT timezone). The server will return
the oldest gzip created later than TIMESTAMP
.
HTTP headers (Content-disposition: attachment; filename="..."
) will contain the new gzip's filename which
you can use to generate the next query, and so on. If the after
parameter is too recent (no newer gzips available),
HTTP 404 is returned. If no after
is provided, the oldest available gzip is returned; we will attempt
to maintain a month's worth of backlog.
You can also download our python script which uses the simple API described above to poll the server at one-minute intervals and copies new .gz files to the local disk as they are made available on the server.
A sample call for downloading the public stream (the default) into folder ijs_news:
./http2fs.py -o ijs_news
This will download the whole available history and continue to follow the real-time stream. To follow the X-Like stream (only available to X-Like project partners), we provide an additinal stream URL besides the default one:
./http2fs.py -o ijs_news -f http://newsfeed.ijs.si/stream/ -f http://newsfeed.ijs.si/stream/xlike
To have a quick look at the real stream data, click either of the links in the API description above.
Each .gz file contains a single XML tree. The root element, <article-set>
, contains zero or more articles in the following format:
<article id="internal article ID; consistent across streams">
<source>
<uri> URL from which the article was discovered; typically the RSS feed </uri>
<title> Title for the source </title>
<type> MAINSTREAM_NEWS </type>
<location?>
<longitude> publisher longitude in degrees </longitude>
<latitude> publisher latitude in degrees </latitude>
<city?> publisher city </city>
<country?> publisher country </country>
</location>
<tags?> RSS-provided tags; the tag vocabulary is not controlled
<tag> some_tag </tag>
<tag> another_tag </tag>
</tags>
</source>
<uri> URL from which the article was downloaded </uri>
<publish-date?> The publication time and date.</publish-date>
<retrieve-date> The retrieval time and date.</retrieve-date>
<lang> 3-letter ISO 639-2 language code </lang>
<location?>
<longitude> story content longitude in degrees </longitude>
<latitude> story content latitude in degrees </latitude>
<city?> story city </city>
<country?> story country </country>
</location>
<tags?> RSS-provided tags; the tag vocabulary is not controlled
<tag> some_tag </tag>
<tag> another_tag </tag>
</tags>
<img?> The URL of a related image, usually a thumbnail. </img>
<title> Title. Can be empty if we fail to identify it. </title>
<body-cleartext>
Clear text body of the article, formatted only with <p> tags
</body-cleartext>
<body-rych?; only English, Slovene>
Enriched article body; an XML subtree as returned by Enrycher.
</body-rych>
<body-xlike?; only English, Spanish, Catalan>
Enriched article body; an XML subtree as returned by iSOCO; experimental.
</body-xlike>
</article>
Elements marked with ? are omitted if the data is missing.
All times are in UTC and take the format yyyy-mm-ddThh:mm:ssZ
.
The contents of Enrycher stream are being index and can be queried using a web service. The service is
accessible at /query/news-search
(JSon output) or /query/news-search-xml
(XML output), and returns the latest (by time of arrival) 100 articles. The service accepts any combination of the following parameters:
q=slovenia hockey
— retrieve all articles with the words "slovenia" and "hockey" in the title or the bodyqt=slovenia hockey
— retrieve all articles with "slovenia" and "hockey" in the titleqb=slovenia hockey
— retrieve all articles with "slovenia" and "hockey" in the bodycu=http://en.wikipedia.org/wiki/Slovenia
— retrieve all annotatied with "Slovenia" wikipedia pagecl=Slovenia
— with "Slovenia" named entitylang=eng
— retrieve all English articlesdate=2012-06-06
— retrieve all articles published on 16th of March, 2012offset=3
— used to retrieve more than first 100 articles; for example, offset 3 returns articles from 301 to 400Example 1: /query/news-search?q=slovenia&lang=eng&date=2012-06-06
Example 2: /query/news-search-xml?q=slovenia&lang=eng&date=2012-06-06
Example 3: /query/news-search?c=http://en.wikipedia.org/wiki/Slovenia&lang=eng&date=2012-06-06
Try querying the pipeline and browsing retrieved articles using newsfeed search.
The pipeline has been developed and is being maintained by the Artificial Intelligence Laboratory at Jozef Stefan Institute, Slovenia. In case of questions, contact Mitja Trampus and/or Blaz Novak at<EMAIL_ADDRESS>Referencing: If you use newsfeed data, please reference it with the following paper:
The development was supported in part by the RENDER, X-Like, PlanetData and MetaNet EU FP7 projects.


ipcrm
Delete IPC (Inter-process Communication) resources.
More information: home page
- Delete a shared memory segment by ID:
ipcrm --shmem-id shmem_id
- Delete a shared memory segment by key:
ipcrm --shmem-key shmem_key
- Delete an IPC queue by ID:
ipcrm --queue-id ipc_queue_id
- Delete an IPC queue by key:
ipcrm --queue-key ipc_queue_key
- Delete a semaphore by ID:
ipcrm --semaphore-id semaphore_id
- Delete a semaphore by key:
ipcrm --semaphore-key semaphore_key
- Delete all IPC resources:
ipcrm --all


I am an aspiring and greatly motivated software developer in the making.
This was done with the intention to help car-owners easily
find car-washes in their environ.
It utilizes HTML5,CSS.
This was done to provide potential customers with information
on the available menu at Annita's Kitchen.
It utilizes
HTML,CSS,JavaScript.
I study at Moringa School and live in Nairobi.
My niche in society
seems to be at the junction between creativity and logic. All projects
are problems to be solved, whether it has to do with code or design;
only the tools change.
My focus is on mobile-first responsive
websites with HTML, CSS, JavaScript.
I am very curious and always
eager to learn.


It is a long established fact that a reader will be distracted by the readable content of a page when looking at its layout.
The point of using Lorem Ipsum is that it has a more-or-less normal distribution of letters
Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots
Popular belief, Lorem Ipsum is not simply random text. It has roots


Order Details
Order ID
Customer ID
Freight
Ship Country
Argentina
Austria
Belgium
Brazil
Canada
Denmark
Finland
France
Germany
Ireland
Italy
Mexico
Norway
Poland
Portugal
Spain
Sweden
Switzerland
UK
USA
Venezuela
Ship City
Ship Name
Ship Address
{{: ShipAddress}}
Frozen Columns
Frozen Rows
Edit Mode
Normal
Dialog
Inline
External


Rich Glor, Character Evolution
- Weakness of ancestral state reconstruction
- Cunningham 1999
- Losos 1999
- Testing hypotheses with ancestral reconstruction
- Parsimony
- Farris method. Downward/upward pass. Gives a maximum parsimony reconstruction. May miss other ones.
- Apparently this Parsimony analysis can be done for continuous traits as well, needs a measure of distance
- Likelihood
- Pagel 1994
- Schluter 1997
R tutorial
(see anolis_cheat.R)
- Discrete trait ancestral state reconstruction
- Wouldn’t it be nice to have functions that applied Rich’s awesome plots to any data? Segway to writing functions
- Wouldn’t a zoomable updatable plot interface be nice? update plot fn and cairo pdf.
- Testing for phylogenetic signal
- Lambda: rescale all internal branches by a factor of lambda.
- Compare likelihood of output under rescaled and original trees.
- The model for these is a single constant rate of transition between all states. Wondering what would happen if we didn’t refit the model to each tree but simply compared the likelihood of the model fit to the original tree when analyzed with the dataset on the modified tree.
Carl Boettiger: useful software
- I gave a 20 minute introduction to subversion and Mendeley.
- Gave a demonstration on using subversion with MrBayes. Useful example, but should have emphasized the generality. Also didn’t have a chance to demonstrate the rapidsvn GUI.
- I’ve added information about using subversion to the Bodega wiki.
- Settled on rapidsvn as the suggested GUI, even though I’m not totally happy with it. The kdesvn gui seems a better choice for linux, but rapidsvn works on all platforms and has an excellent guide.
Justin Whittall: cts character traits ————————————-
- Check out the Charlie Nunn Papers.
- Compare incoperates uncertainty in the species means.
- Are contrasts standardized by branch length? Absolute value of contrast independent of standard deviation of contrast? If not, transform data..
My Thoughts and Explorations for the Day —————————————-
- Many analyses are not explicit about the model used and the assumptions it involves!
- Not happy with the independence assumption on columns of the alignment in DNA data. We can quantify the predictive capacity of that information…
- Rich Glor’s likelihood test under Pagel’s lambda. seems a strange thing to do, what does it actually mean?
- Need to explore the later applications of independent contrasts further.
Research Progress
- Figuring out strange behavior in ouch. the value kept at model@alpha is actually the Cholesky decomposition of the alpha matrix, just as the values of sigma are the Cholesky decomposition of the σ2 matrix. This seems like a strange convention, surely the user would rather specify the matrices as full matrices and let the software do the decomposition? Because this decomposes matrix M into M = *L**L*T, for single trait value alpha is a scalar and this is the square root. sym.par() function in ouch is often called in the source to assemble M from the decomposition…


Filter and manipulate annotated files
SnpSift is a toolbox that allows you to filter and manipulate annotated files.
For older version of this page, see: Manual page for SnpSift version 4.0
Once your genomic variants have been annotated, you need to filter them out in order to find the "interesting / relevant variants". Given the large data files, this is not a trivial task (e.g. you cannot load all the variants into XLS spreadsheet). SnpSift helps to perform this VCF file manipulation and filtering required at this stage in data processing pipelines.
SnpSift is part of SnpEff main distribution, so please click on here and follow the instructions on how to download and install SnpEff.

Some examples of what you can do:
| Filter | You can filter using arbitrary expressions, for instance "(QUAL > 30) | (exists INDEL) | ( countHet() > 2 )". The actual expressions can be quite complex, so it allows for a lot of flexibility. |
| Annotate | You can add 'ID' and INFO fields from another "VCF database" (e.g. typically dbSnp database in VCF format). |
| CaseControl | You can compare how many variants are in 'case' and in 'control' groups. Also calculates p-values (Fisher exact test). |
| Intervals | Filter variants that intersect with intervals. |
| Intervals (intidx) | Filter variants that intersect with intervals. Index the VCF file using memory mapped I/O to speed up the search. This is intended for huge VCF files and a small number of intervals to retrieve. |
| Join | Join by generic genomic regions (intersecting or closest). |
| RmRefGen | Remove reference genotype (i.e. replace '0/0' genotypes by '.') |
| TsTv | Calculate transition to transversion ratio. |
| Extract fields | Extract fields from a VCF file to a TXT (tab separated) format. |
| Variant type | Adds SNP/MNP/INS/DEL to info field. It also adds "HOM/HET" if there is only one sample. |
| GWAS Catalog | Annotate using GWAS Catalog. |
| DbNSFP | Annotate using dbNSFP: The dbNSFP is an integrated database of functional predictions from multiple algorithms (SIFT, Polyphen2, LRT and MutationTaster, PhyloP and GERP++, etc.) |
| SplitChr | Split a VCF file by chromosome |
In order to cite SnpSift, please use the following reference:
"Using Drosophila melanogaster as a model for genotoxic chemical mutational studies with a new program, SnpSift", Cingolani, P., et. al., Frontiers in Genetics, 3, 2012.BibTex entry:
@article{cingolani2012using, title={Using Drosophila melanogaster as a model for genotoxic chemical mutational studies with a new program, SnpSift}, author={Cingolani, P. and Patel, V.M. and Coon, M. and Nguyen, T. and Land, S.J. and Ruden, D.M. and Lu, X.}, journal={Frontiers in Genetics}, volume={3}, year={2012}, publisher={Frontiers Media SA} }
The project is hosted at SourceForge.
Here is the SVN command to check out the development version of the code:svn co https://snpeff.svn.sourceforge.net/svnroot/snpeff/SnpSift/trunk
SnpSift filter is one of the most useful SnpSift commands. Using SnpSift filter you can filter VCF files using arbitrary expressions, for instance "(QUAL > 30) | (exists INDEL) | ( countHet() > 2 )". The actual expressions can be quite complex, so it allows for a lot of flexibility.
Some examples for the impatient:
cat variants.vcf | java -jar SnpSift.jar filter " ( QUAL >= 30 )" > filtered.vcf
cat variants.vcf | java -jar SnpSift.jar filter "(( exists INDEL ) & (QUAL >= 20)) | (QUAL >= 30 )" > filtered.vcf
cat variants.vcf | java -jar SnpSift.jar filter "(countHom() > 3) | (( exists INDEL ) & (QUAL >= 20)) | (QUAL >= 30 )" > filtered.vcf
cat variants.vcf | java -jar SnpSift.jar filter "((countHet() > 0) && (DP >= 25)) | (countHom() > 3) | (( exists INDEL ) & (QUAL >= 20)) | (QUAL >= 30 )" > filtered.vcf
cat variants.vcf | java -jar SnpSift.jar filter "isHom( GEN[0] ) & isVariant( GEN[0] ) & isRef( GEN[1] )" > filtered.vcf
cat variants.vcf | java -jar SnpSift.jar filter --set my_rs.txt "ID in SET[0]" > filtered.vcfand the file my_rs.txt has one string per line, e.g.:
rs58108140 rs71262674 rs71262673
cat variants.vcf | java -jar SnpSift.jar filter '((AN+0.0) / AO) > 1.0'Note that we used 'AN+0.0' to convert the integer number to float (otherwise integer division would be used)
Usage: java -jar SnpSift.jar filter [options] 'expression' [input.vcf] Options: -a|--addFilter: Add a string to FILTER VCF field if 'expression' is true. Default: '' (none) -e|--exprFile : Read expression from a file -f|--file : VCF input file. Default: STDIN -i|--filterId : ID for this filter (##FILTER tag in header and FILTER VCF field). Default: 'SnpSift' -n|--inverse : Inverse. Show lines that do not match filter expression -p|--pass : Use 'PASS' field instead of filtering out VCF entries -r|--rmFilter : Remove a string from FILTER VCF field if 'expression' is true (and 'str' is in the field). Default: '' (none) -s|--set : Create a SET using 'file' --errMissing : Error is a field is missing. Default: false --format : SnpEff format version: {2, 3}. Default: Auto --galaxy : Used from Galaxy (expressions have been sanitized).
"( CHROM = 'chr1' )"
"( POS > 123456 ) & ( POS < 654321 )"
"(exists ID) & ( ID =~ 'rs' )"
"( REF = 'A' )"
"( ALT = 'T' )"
"( QUAL > 30 )"
"( na FILTER ) | (FILTER = 'PASS')"
( DP > 10 ) & ( AF1 = 0 )
"( CI95[0] > 0.1 ) & (CI95[1] <= 0.3)"
ANY or *: If you use 'ANY' as index, the expression will be true if any field satisfies the expression.
So, for instance, the following expressions
"( CI95[ANY] > 0.1 )"or
"( CI95[*] > 0.1 )"are equivalent to (in this case, there are only two values in the array)
"( CI95[0] > 0.1 ) | ( CI95[1] > 0.1 )"
ALL or ?: If you use 'ALL' as index, the expression will be true if all field satisfy the expression.
So, for instance, the following expressions
"( CI95[ALL] > 0.1 )"
"( CI95[?] > 0.1 )"are equivalent to (in this case, there are only two values in the array)
"( CI95[0] > 0.1 ) & ( CI95[1] > 0.1 )"
"( GEN[0].GQ > 60 ) & ( GEN[1].GQ > 90 )"You may use an asterisk to represent 'ANY' field
"( GEN[*].GQ > 60 )"
"( GEN[0].PL[2] = 0 )"You may use an asterisk to represent 'ANY' field
"( GEN[0].PL[*] = 0 )"...or even
"( GEN[*].PL[*] = 0 )"
$ java -jar SnpSift.jar filter "( GEN[HG00096].DS > 0.2 ) & ( GEN[HG00097].DS > 0.5 )" examples/1kg.head_chr1.vcf.gz
"( ID in SET[2] )"
ANN
sub-fields are (for details, take a look at the specification):
missense_variant
variant:
has
operator is better than using equality operator (=
).
For instance 'missense_variant&splice_region_variant' = 'missense_variant'
is false, whereas 'missense_variant&splice_region_variant' has 'missense_variant'
is true.
$ java -jar SnpSift.jar filter "ANN[0].EFFECT has 'missense_variant'" examples/test.chr22.ann.vcf > test.chr22.ann.filter_missense_first.vcf # Output example (edited for readibility) $ cat test.chr22.ann.filter_missense_first.vcf 22 17072035 . C T . . ANN=T|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.1406G>A|p.Gly469Glu|1666/2034|1406/1674|469/557||,T|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397G>A|||||3944| 22 17072258 . C A . . ANN=A|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.1183G>T|p.Gly395Cys|1443/2034|1183/1674|395/557||,A|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397G>T|||||3721| 22 17072674 . G A . . ANN=A|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.767C>T|p.Pro256Leu|1027/2034|767/1674|256/557||,A|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397C>T|||||3305| 22 17072747 . T C . . ANN=C|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.694A>G|p.Met232Val|954/2034|694/1674|232/557||,C|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397A>G|||||3232| 22 17073043 . C T . . ANN=T|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.398G>A|p.Arg133Gln|658/2034|398/1674|133/557||,T|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397G>A|||||2936| 22 17073119 . C T . . ANN=T|missense_variant|MODERATE|CCT8L2|ENSG00000198445|transcript|ENST00000359963|protein_coding|1/1|c.322G>A|p.Val108Met|582/2034|322/1674|108/557||,T|downstream_gene_variant|MODIFIER|FABP5P11|ENSG00000240122|transcript|ENST00000430910|processed_pseudogene||n.*397G>A|||||2860|...but this probably doesn't make much sense. What you may really want are lines where ANY effect to be
missense_variant
:
$ java -jar SnpSift.jar filter "ANN[*].EFFECT has 'missense_variant'" examples/test.chr22.ann.vcf > test.chr22.ann.filter_missense_any.vcf # Output example (edited for readibility) $ cat test.chr22.ann.filter_missense_any.vcf ... 22 24891462 . G A . . ANN=A|stop_gained|HIGH|UPB1|ENSG00000100024|transcript|ENST00000413389|protein_coding|2/10|c.59G>A|p.Trp20*|1652/3418|59/951|20/316|| ,A|missense_variant|MODERATE|UPB1|ENSG00000100024|transcript|ENST00000326010|protein_coding|1/10|c.91G>A|p.Gly31Ser|435/2290|91/1155|31/384|| ,A|missense_variant|MODERATE|UPB1|ENSG00000100024|transcript|ENST00000382760|protein_coding|1/4|c.91G>A|p.Gly31Ser|253/1928|91/561|31/186|| 22 24896158 . A T . . ANN=T|missense_variant|MODERATE|UPB1|ENSG00000100024|transcript|ENST00000326010|protein_coding|2/10|c.188A>T|p.Glu63Val|532/2290|188/1155|63/384|| ,T|missense_variant|MODERATE|UPB1|ENSG00000100024|transcript|ENST00000382760|protein_coding|2/4|c.188A>T|p.Glu63Val|350/1928|188/561|63/186||May be you want only the ones that affect gene 'TRMT2A'
$ java -jar SnpSift.jar filter "(ANN[*].EFFECT has 'missense_variant') && (ANN[*].GENE = 'TRMT2A')" examples/test.chr22.ann.vcf > test.chr22.ann.filter_missense_any_TRMT2A.vcf $ cat test.chr22.ann.filter_missense_any_TRMT2A.vcf 22 20103915 . C T . . ANN=T|stop_gained|HIGH|RANBP1|ENSG00000099901|transcript|ENST00000432879|protein_coding|1/3|c.208C>T|p.Arg70*|455/744|208/497|70/164||WARNING_TRANSCRIPT_INCOMPLETE ,T|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000439169|protein_coding|2/12|c.245G>A|p.Arg82His|561/2473|245/1932|82/643|| ,T|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000252136|protein_coding|2/12|c.245G>A|p.Arg82His|634/2964|245/1878|82/625|| ,T|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000403707|protein_coding|3/13|c.245G>A|p.Arg82His|607/2928|245/1878|82/625|| ,T|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000404751|protein_coding|2/12|c.245G>A|p.Arg82His|584/2498|245/1689|82/562|| ,T|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000445045|protein_coding|2/2|c.209G>A|p.Arg70His|432/582|209/359|70/118||WARNING_TRANSCRIPT_INCOMPLETE ,... 22 20103925 . T C . . ANN=C|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000439169|protein_coding|2/12|c.235A>G|p.Asn79Asp|551/2473|235/1932|79/643|| ,C|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000252136|protein_coding|2/12|c.235A>G|p.Asn79Asp|624/2964|235/1878|79/625|| ,C|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000403707|protein_coding|3/13|c.235A>G|p.Asn79Asp|597/2928|235/1878|79/625|| ,C|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000404751|protein_coding|2/12|c.235A>G|p.Asn79Asp|574/2498|235/1689|79/562|| ,C|missense_variant|MODERATE|RANBP1|ENSG00000099901|transcript|ENST00000432879|protein_coding|1/3|c.218T>C|p.Phe73Ser|465/744|218/497|73/164||WARNING_TRANSCRIPT_INCOMPLETE ,C|missense_variant|MODERATE|TRMT2A|ENSG00000099899|transcript|ENST00000445045|protein_coding|2/2|c.199A>G|p.Asn67Asp|422/582|199/359|67/118||WARNING_TRANSCRIPT_INCOMPLETE ,C|splice_region_variant&intron_variant|LOW|RANBP1|ENSG00000099901|transcript|ENST00000430524|protein_coding|1/5|c.-374+7T>C|||||| ,...
-classic
or -formatEff
command line options.
SnpEff annotations are parsed, so you can access individual sub-fields:
EFF
sub-fields are:
"( EFF[0].EFFECT = 'NON_SYNONYMOUS_CODING' )"...but this probably doesn't make much sense. What you may really want are lines where ANY effect is NON_SYNONYMOUS:
"( EFF[*].EFFECT = 'NON_SYNONYMOUS_CODING' )"May be you want only the ones that affect gene 'TCF7L2'
"( EFF[*].EFFECT = 'NON_SYNONYMOUS_CODING' ) & ( EFF[*].GENE = 'TCF7L2' )"
LOF
and NMD
sub-fields are available:
$cat test.snpeff.vcf | java -Xmx1G -jar SnpSift.jar filter "(exists LOF[*].PERC) & (LOF[*].PERC > 0.9)"We assume that 'test.snpeff.vcf' was annotated with SnpEff using '-lof' command line option.
The following operators and functions are interpreted by SnpSift filter
:
| Operand | Description | Data type | Example |
|---|---|---|---|
| = | Equality test | FLOAT, INT or STRING | (REF = 'A') |
| > | Greater than | FLOAT or INT | (DP > 20) |
| ≥ | Greater or equal than | FLOAT or INT | (DP ≥ 20) |
| < | Less than | FLOAT or INT | (DP < 20) |
| ≤ | Less or equal than | FLOAT or INT | (DP ≤ 20) |
| =~ | Match regular expression | STRING | (REL =~ 'AC') |
| !~ | Does not match regular expression | STRING | (REL !~ 'AC') |
| & | AND operator | Boolean | (DP > 20) & (REF = 'A') |
| | | OR operator | Boolean | (DP > 20) | (REF = 'A') |
| ! | NOT operator | Boolean | ! (DP > 20) |
| exists | The variable exists (not missing) | Any | (exists INDEL) |
| has | The right hand side expression is equalt to any of the items in a list consisting of separating the left hand side expression using delimiters: '&', '+', ';', ',', ':', '(', ')' ,'[', ']'
Example: If the expression is: ANN[*].EFFECT has 'missense_variant' |
Any | (ANN[*].EFFECT has 'missense_variant') |
| Function | Description | Data type | Example |
| countHom() | Count number of homozygous genotypes | No arguments | (countHom() > 0) |
| countHet() | Count number of heterozygous genotypes | No arguments | (countHet() > 2) |
| countVariant() | Count number of genotypes that are variants (i.e. not reference 0/0) | No arguments | (countVariant() > 5) |
| countRef() | Count number of genotypes that are NOT variants (i.e. reference 0/0) | No arguments | (countRef() < 1) |
| Genotype Function | Description | Data type | Example |
| isHom | Is homozygous genotype? | Genotype | isHom( GEN[0] ) |
| isHet | Is heterozygous genotype? | Genotype | isHet( GEN[0] ) |
| isVariant | Is genotype a variant? (i.e. not reference 0/0) | Genotype | isVariant( GEN[0] ) |
| isRef | Is genotype a reference? (i.e. 0/0) | Genotype | isRef( GEN[0] ) |
$ cat cancer.vcf | java -jar SnpSift.jar filter "GEN[Somatic].GT = '2/1'" #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Germline Somatic 1 69091 . A C,G . PASS AC=1 GT 1/0 2/1Note that we used
GEN[Somatic]
instead of GEN[1]
.
Annotate using fields from another VCF file (e.g. dbSnp, 1000 Genomes projects, ClinVar, ExAC, etc.).
This is typically used to annotate IDs and INFO fields from a 'database' VCF file (e.g. dbSnp). Here is an example:java -jar SnpSift.jar annotate dbSnp132.vcf variants.vcf > variants_annotated.vcf
Important: SnpSift annotate
command has different strategies depending on the input VCF file:
Note:
-info
command line option if you only want select only a subset of fields from db.vcf file.
-id
command line option if you only want to add ID fields (no INFO fields will be added).
-exists
command line option, you can annotate entries that exists in the 'database' file.
Example 1: Annotating ID
from dbSnp:
$ cat test.chr22.vcf #CHROM POS ID REF ALT QUAL FILTER INFO 22 16157571 . T G 0.0 FAIL NS=53 22 16346045 . T C 0.0 FAIL NS=244 22 16350245 . C A 0.0 FAIL NS=192 22 17054103 . G A 0.0 PASS NS=404 22 17071906 . A T 0.0 PASS NS=464 22 17072347 . C T 0.0 PASS NS=464 22 17072394 . C G 0.0 PASS NS=463 22 17072411 . G T 0.0 PASS NS=464 $ java -jar SnpSift.jar annotate -id db/dbSnp/dbSnp137.20120616.vcf test.chr22.vcf #CHROM POS ID REF ALT QUAL FILTER INFO 22 16157571 . T G 0.0 FAIL NS=53 22 16346045 rs56234788 T C 0.0 FAIL NS=244 22 16350245 rs2905295 C A 0.0 FAIL NS=192 22 17054103 rs4008588 G A 0.0 PASS NS=404 22 17071906 . A T 0.0 PASS NS=464 22 17072347 rs139948519 C T 0.0 PASS NS=464 22 17072394 . C G 0.0 PASS NS=463 22 17072411 rs41277596 G T 0.0 PASS NS=464Example 2: Annotatating
ID
and all INFO
fields from dbSnp (VCF headers not shown for brevity):
$ cat test.chr22.vcf #CHROM POS ID REF ALT QUAL FILTER INFO 22 16157571 . T G 0.0 FAIL NS=53 22 16346045 . T C 0.0 FAIL NS=244 22 16350245 . C A 0.0 FAIL NS=192 22 17054103 . G A 0.0 PASS NS=404 22 17071906 . A T 0.0 PASS NS=464 22 17072347 . C T 0.0 PASS NS=464 22 17072394 . C G 0.0 PASS NS=463 22 17072411 . G T 0.0 PASS NS=464 $ java -jar SnpSift.jar annotate db/dbSnp/dbSnp137.20120616.vcf test.chr22.vcf #CHROM POS ID REF ALT QUAL FILTER INFO 22 16157571 . T G 0.0 FAIL NS=53 22 16346045 rs56234788 T C 0.0 FAIL NS=244;RSPOS=16346045;GMAF=0.162248628884826;dbSNPBuildID=129;SSR=0;SAO=0;VP=050100000000000100000100;WGT=0;VC=SNV;SLO;GNO 22 16350245 rs2905295 C A 0.0 FAIL NS=192;RSPOS=16350245;GMAF=0.230804387568556;dbSNPBuildID=101;SSR=1;SAO=0;VP=050000000000000100000140;WGT=0;VC=SNV;GNO 22 17054103 rs4008588 G A 0.0 PASS NS=404;RSPOS=17054103;GMAF=0.123400365630713;dbSNPBuildID=108;SSR=0;SAO=0;VP=050100000000070010000100;WGT=0;VC=SNV;SLO;VLD;G5A;G5;KGPilot123 22 17071906 . A T 0.0 PASS NS=464 22 17072347 rs139948519 C T 0.0 PASS NS=464;RSPOS=17072347;dbSNPBuildID=134;SSR=0;SAO=0;VP=050200000004040010000100;WGT=0;VC=SNV;S3D;ASP;VLD;KGPilot123 22 17072394 . C G 0.0 PASS NS=463 22 17072411 rs41277596 G T 0.0 PASS NS=464;RSPOS=17072411;GMAF=0.00411334552102377;dbSNPBuildID=127;SSR=0;SAO=0;VP=050200000008040010000100;GENEINFO=CCT8L2:150160;WGT=0;VC=SNV;S3D;CFL;VLD;KGPilot123
Allows you to count how many samples are in 'case' and 'control' groups.
This command counts the number of 'homozygous', 'heterozygous' and 'total' variants in a case and control groups and performs some basic pValue calculation using Fisher exact test and Cochran-Armitage test.
Case and Control groups can be defined either by a command line string or a TFAM file (see PLINK's documentation).
Case/Control command line string containing plus and minus symbols {'+', '-', '0'} where '+' is case, '-' is control and '0' is neutral (ignored).
E.g. We have ten samples, which means ten genotype columns in the VCF file.
The first four are 'cases', the fifth one is 'neutral', and the last five are 'control'.
So the description string would be "++++0-----" (note that the following output has been edited, only counts are shown, no pValues):
$ java -jar SnpSift.jar caseControl "++++0-----" cc.vcf #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Sample_01 Sample_02 Sample_03 Sample_04 Sample_05 Sample_06 Sample_07 Sample_08 Sample_09 Sample_10 1 69496 . G A . PASS AF=0.01;Cases=1,2,4;Controls=2,2,6 GT 0/1 1/1 1/0 0/0 0/0 0/1 1/1 1/1 1/0 0/0Cases genotypes are samples 1 to 4 : 0/1, 1/1, 1/0 and 0/0. So there are 1 homozygous, 2 heterozygous, and a total of 4 variants (2 * 1 + 1 * 2 = 4). Thus the annotation is
Cases=1,2,4
Controls=2,2,6
-tfam
command line option to specify a TFAM file.
Case, control from are read from phenotype field of a TFAM file (6th column).
Phenotype order in TFAM files do not need to match VCF sample order (sample IDs are used).
Phenotype column should be coded as {0,1,2} meaning {Missing, Control, Case} respectively.
See PLINK's reference for details about TFAM file format.
-name nameString
command line option to add name to the INFO tags.
This can be used to count different case/control groups in the same dataset (e.g. multiple phenotypes)
$ java -jar SnpSift.jar caseControl -name "_MY_GROUP" "++++0-----" cc.vcf \ | java -jar SnpSift.jar caseControl -name "_ANOTHER_GROUP" "+-+-+-+-+-" - #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Sample_01 Sample_02 Sample_03 Sample_04 Sample_05 Sample_06 Sample_07 Sample_08 Sample_09 Sample_10 1 69496 . G A . PASS AF=0.01;Cases_MY_GROUP=1,2,4;Controls_MY_GROUP=2,2,6;Cases_ANOTHER_GROUP=1,3,5;Controls_ANOTHER_GROUP=2,1,5 GT 0/1 1/1 1/0 0/0 0/0 0/1 1/1 1/1 1/0 0/0
CC_DOM
): A 2 by 2 contingency table is created
| Alt (A/a + a/a) | Ref (A/A) | |
|---|---|---|
| Cases | N11 | N12 |
| Controls | N21 | N22 |
CC_REC
): A 2 by 2 contingency table is created
| Alt (a/a) | Ref + Het (A/A + A/a) | |
|---|---|---|
| Cases | N11 | N12 |
| Controls | N21 | N22 |
CC_ALL
): A 2 by 2 contingency table is created
| Variants | References | |
|---|---|---|
| Cases | N11 | N12 |
| Controls | N21 | N22 |
CC_GENO
): A 2 by 3 contingency table is created
| A/A | a/A | a/a | |
|---|---|---|---|
| Cases | N11 | N12 | N13 |
| Controls | N21 | N22 | N23 |
CC_TREND
): A 2 by 3 contingency table is created
| A/A | a/A | a/a | |
|---|---|---|---|
| Cases | N11 | N12 | N13 |
| Controls | N21 | N22 | N23 |
| Weight | 0.0 | 1.0 | 2.0 |
This is used to extract variants that intersect any interval.
You must provide intervals as BED files.
Command line options:
E.g.:
cat variants.vcf | java -jar SnpSift.jar intervals my_intervals.bed > variants_intersecting_intervals.vcf
This is used to extract variants that intersect any interval.
This is similar to "SnpSift intervals", but intended for huge VCF files, and relatively small number of intervals.
This command indexes the VCF file, thus is optimized for huge VCF files.
You must provide intervals as BED files.
BED format is tab separated zero-based coordinates "chr \t start \t end " (for this application, all other fields in the BED file are ignored).
You can use command line option '-if 1' if you want one-based coordinates.
E.g.:
java -jar SnpSift.jar intidx variants.vcf my_intervals.bed > variants_intersecting_intervals.vcfYou can also have genomic coordinate in the command line. Note that in this case, coordinates are assumed to be one-based (instead of zero-based, like in BED files):
java -jar SnpSift.jar intidx -c variants.vcf chr1:12345-23456 chr2:3456789-4567890 > variants_intersecting_intervals.vcf
Join files by genomic regions (i.e. chr:start-end).
Files can be generic TXT (tab separated), VCF or BED.Usage: java -jar SnpSift.jar join [options] file1 file2 Note: It is assumed that both files fit in memory. Options: -if1 <num> : Offset for file1 (e.g. 1 if coordinates are one-based. Default: 1 -if2 <num> : Offset for file2 (e.g. 2 if coordinates are one-based. Default: 1 -cols1 <colDef> : Column definition for file 1. Format: chrCol,startCol,endCol (e.g. '1,2,3'). Shortcuts 'bed' or 'vcf' are allowed. Default: 'vcf -cols2 <colDef> : Column definition for file 2. Format: chrCol,startCol,endCol (e.g. '1,2,3'). Shortcuts 'bed' or 'vcf' are allowed. Default: 'vcf -all : For each interval, show all intersecting. Default: show only one (the largest intersection) -closest : Show closest intervals in file2 if none intersect. Default: off -empty : Show intervals in file1 even if they do not intersect with any other interval. Default: offExample: Join two bed files, showing intersecting or closest intervals
java -Xmx2G -jar SnpSift.jar join -v -cols1 bed -cols2 bed -closest file1.bed file2.bedExample: Join one bed file and another file having chr:start-end in columns 7,8 and 9 respectively. Showing intervals form file1 that do not intersect any interval from file2
java -Xmx2G -jar SnpSift.jar join -v -cols1 bed -cols2 7,8,9 -empty file.bed my_weird_file.txt
Remove reference genotypes.
Replaces genotype information for non-variant samples.E.g. If you have this file:
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT M1 M2 X1 X2 2L 426906 . C G 53.30 . DP=169 GT:PL:GQ 0/1:7,0,255:4 0/1:7,0,255:4 0/0:0,0,0:6 0/0:0,30,255:35 2L 601171 . C A 999.00 . DP=154 GT:PL:GQ 0/1:81,0,141:78 0/1:42,0,251:39 0/0:0,0,0:4 0/0:0,33,255:36 2L 648611 . A T 999.00 . DP=225 GT:PL:GQ 0/1:52,0,42:47 1/1:75,21,0:14 0/0:0,0,0:3 0/0:0,60,255:61 2L 807373 . A G 106.00 . DP=349 GT:PL:GQ 0/1:14,0,65:12 0/1:60,0,42:50 0/0:0,0,0:4 0/0:0,69,255:72 2L 816766 . G T 999.00 . DP=411 GT:PL:GQ 0/1:108,0,45:53 0/1:7,0,255:6 0/0:0,0,0:4 0/0:0,57,255:59You can run:
cat file.vcf | java -jar SnpSift.jar rmRefGen > file_noref.vcfand you get this (notice the last two columns, that had '0/0' genotype):
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT M1 M2 X1 X2 2L 426906 . C G 53.30 . DP=169 GT:PL:GQ 0/1:7,0,255:4 0/1:7,0,255:4 . . 2L 601171 . C A 999.00 . DP=154 GT:PL:GQ 0/1:81,0,141:78 0/1:42,0,251:39 . . 2L 648611 . A T 999.00 . DP=225 GT:PL:GQ 0/1:52,0,42:47 1/1:75,21,0:14 . . 2L 807373 . A G 106.00 . DP=349 GT:PL:GQ 0/1:14,0,65:12 0/1:60,0,42:50 . . 2L 816766 . G T 999.00 . DP=411 GT:PL:GQ 0/1:108,0,45:53 0/1:7,0,255:6 . .
Calculate transition vs transversion ratios for each sample.
Usage example:$ java -jar SnpSift.jar tstv hom s.vcf Sample : 1 2 3 4 5 6 7 8 9 10 11 12 Total Transitions :<PHONE_NUMBER>64<PHONE_NUMBER>74<PHONE_NUMBER>56<PHONE_NUMBER>14<PHONE_NUMBER>76<PHONE_NUMBER>68 1847670 Transversions : 70878 70358 73688 72434 70828 76150 72030 71958 72960 69348 70180 71688 862500 Ts/Tv : 2.123 2.139 2.154 2.163 2.159 2.106 2.114 2.158 2.145 2.153 2.154 2.141 2.142
Extract fields from a VCF file to a TXT, tab separated format, that you can easily load in R, XLS, etc.
You can also use sub-fields and genotype fields / sub-fields such as:
vcfEffOnePerLine.pl
script with SnpSift extractFields
if you want to have each effect in a seprarate line.
-s
to specify multiple field separator and -e
to specify how to represent empty fields.
Example 1: Extracting chromosome, position, ID and allele frequency from a VCF file:
$ java -jar SnpSift.jar extractFields s.vcf CHROM POS ID AF | head #CHROM POS ID AF 1 69134 0.086 1 69496 rs150690004 0.001 1 69511 rs75062661 0.983 1 69569 0.538 1 721559 0.001 1 721757 0.011 1 846854 rs111957712 0.003 1 865584 rs148711625 0.001 1 865625 rs146327803 0.001
Example 2: Extracting genotype fields
$ java -jar SnpSift.jar extractFields file.vcf CHROM" "POS" "ID" "THETA" "GEN[0].GL[1]" "GEN[1].GL" "GEN[3].GL[*]" "GEN[*].GT"This means to extract:
#CHROM POS ID THETA GEN[0].GL[1] GEN[1].GL GEN[3].GL[*] GEN[*].GT 1 10583 rs58108140 0.0046 -0.47 -0.24,-0.44,-1.16 -0.48 -0.48 -0.48 0|0 0|0 0|0 0|1 0|0 0|1 0|0 0|0 0|1 1 10611 rs189107123 0.0077 -0.48 -0.24,-0.44,-1.16 -0.48 -0.48 -0.48 0|0 0|1 0|0 0|0 0|0 0|0 0|0 0|0 0|0 1 13302 rs180734498 0.0048 -0.58 -2.45,-0.00,-5.00 -0.48 -0.48 -0.48 0|0 0|1 0|0 0|0 0|0 1|0 0|0 0|1 0|0 1 13327 rs144762171 0.0204 -1.11 -1.97,-0.01,-2.51 -0.48 -0.48 -0.48 0|0 0|1 0|0 0|0 0|0 1|0 0|0 0|0 0|0 1 13957 rs201747181 0.0100 0 0,0,0 0 0 0 0|0 0|1 0|0 0|0 0|0 0|0 0|0 0|0 0|0 1 13980 rs151276478 0.0139 -0.48 -0.48,-0.48,-0.48 -0.48 -0.48 -0.48 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 1 30923 rs140337953 0.0162 -0.61 -0.10,-0.69,-2.81 -0.48 -0.48 -0.48 1|1 0|0 0|0 1|1 1|0 0|0 1|1 1|0 1|1 1 46402 rs199681827 0.0121 0 0,0,0 0 0 0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 1 47190 rs200430748 0.0153 0 0,0,0 0 0 0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0 0|0
-s
to specify multiple field separator and -e
to specify how to represent empty fields.
$ java -jar SnpSift.jar extractFields -s "," -e "." test.chr22.ann.vcf CHROM POS REF ALT "EFF[*].EFFECT" "EFF[*].AA"
-s ","
, and we use "." for empty fiels (option -e "."
).
The results is:
$ java -jar SnpSift.jar extractFields -s "," -e "." examples/test.chr22.ann.vcf CHROM POS REF ALT "ANN[*].EFFECT" "ANN[*].HGVS_P" #CHROM POS REF ALT ANN[*].EFFECT ANN[*].HGVS_P 22 17071756 T C 3_prime_UTR_variant,downstream_gene_variant .,. 22 17072035 C T missense_variant,downstream_gene_variant p.Gly469Glu,. 22 17072258 C A missense_variant,downstream_gene_variant p.Gly395Cys,. 22 17072674 G A missense_variant,downstream_gene_variant p.Pro256Leu,. 22 17072747 T C missense_variant,downstream_gene_variant p.Met232Val,. 22 17072781 C T synonymous_variant,downstream_gene_variant p.Pro220Pro,. 22 17073043 C T missense_variant,downstream_gene_variant p.Arg133Gln,. 22 17073066 A G synonymous_variant,downstream_gene_variant p.Ala125Ala,. 22 17073119 C T missense_variant,downstream_gene_variant p.Val108Met,.
$ java -jar SnpSift.jar extractFields examples/test.chr22.ann.vcf CHROM POS REF ALT "ANN[*].EFFECT" #CHROM POS REF ALT ANN[*].EFFECT 22 17071756 T C 3_prime_UTR_variant downstream_gene_variant 22 17072035 C T missense_variant downstream_gene_variant 22 17072258 C A missense_variant downstream_gene_variant 22 17072674 G A missense_variant downstream_gene_variant 22 17072747 T C missense_variant downstream_gene_variant 22 17072781 C T synonymous_variant downstream_gene_variant 22 17073043 C T missense_variant downstream_gene_variant 22 17073066 A G synonymous_variant downstream_gene_variant 22 17073119 C T missense_variant downstream_gene_variantNote that since some variant have more than one effect, there can be more than one "EFFECT" column.
vcfEffOnePerLine.pl
provided with SnpEff distribution
$ cat examples/test.chr22.ann.vcf \ | ./scripts/vcfEffOnePerLine.pl \ | java -jar SnpSift.jar extractFields - CHROM POS REF ALT "ANN[*].EFFECT" \ #CHROM POS REF ALT ANN[*].EFFECT 22 17071756 T C 3_prime_UTR_variant 22 17071756 T C downstream_gene_variant 22 17072035 C T missense_variant 22 17072035 C T downstream_gene_variant 22 17072258 C A missense_variant 22 17072258 C A downstream_gene_variant 22 17072674 G A missense_variant 22 17072674 G A downstream_gene_variant 22 17072747 T C missense_variantNow we obtain one effect per line, while all other parameters in the line are repeated across mutiple lines (e.g. there are two chr22:17071756 lines, one for each variant annotation).
-
as input file name, which denotes STDIN.
$ java -jar SnpSift.jar extractFields examples/1kg.head_chr1.vcf.gz CHROM POS REF ALT "GEN[HG00096].DS" "GEN[HG00097].DS" #CHROM POS REF ALT GEN[HG00096].DS GEN[HG00097].DS 1 10583 G A 0.2 0.15 1 10611 C G 0.05 0.75 1 13302 C T 0.05 1.0 1 13327 G C 0.0 0.95 1 13957 TC T 0.05 0.65 1 13980 T C 0.05 0.6 1 30923 G T 1.75 0.35 1 46402 C CTGT 0.05 0.15 1 47190 G GA 0.15 0.0
SnpSift extractFields
can get confused if the VCF field has non-alphanumeric charaters in the name (e.g. dbNSFP_GERP++_RS
has two "+" signs).
A quick fix, it so is to change the field names in the VCF file.
Here is an example:
# Change field names in VCF $ cat kath.vcf | sed "s/dbNSFP_GERP++/dbNSFP_GERP/g" > kath.gerp.vcf # Use new names to extract fields $ java -jar SnpSift.jar extractFields kath.gerp.vcf CHROM POS REF ALT dbNSFP_GERP_RS dbNSFP_GERP_NS #CHROM POS REF ALT dbNSFP_GERP_RS 1 142827044 G A 2 132914561 G A 7 151933217 C A 7 151933251 T C 7 151933302 T C 7 151945101 G C -0.892 7 151945167 G T 7 151962176 T A 7 151970672 A T 7 151970856 T A 3.71 18 14183638 G C 18 14183710 A G 18 14542909 G A 18 14543039 T C -0.942
Adds an INFO field denoting variant type.
It adds "SNP/MNP/INS/DEL/MIXED" in the INFO field. It also adds "HOM/HET", but this last one works if there is only one sample (otherwise it doesn't make any sense).
$ java -jar SnpSift.jar varType test.vcf | grep -v "^#" | head 20 10469 . C G 100.0 PASS SNP;HOM GT:AP 0|0:0.075,0.060 20 10492 . C T 100.0 PASS SNP;HET GT:AP 0|1:0.180,0.345 20 10575 . C CG 100.0 PASS DEL;HET GT:AP 0|1:0.000,0.000 20 10611 . CG C 100.0 PASS INS;HET GT:AP 0|1:0.000,0.010 20 10618 . GT TA 100.0 PASS MNP;HET GT:AP 0|1:0.020,0.030
Annotate using GWAS catalog
$ java -jar SnpSift.jar gwasCat gwascatalog.txt test.vcf | tee test.gwas.vcf 1 1005806 rs3934834 C T . PASS AF=0.091;GWASCAT=Body_mass_index 1 2069172 rs425277 C T . PASS AF=0.400;GWASCAT=Height 1 2069681 rs3753242 C T . PASS AF=0.211;GWASCAT=Reasoning 1 2392648 rs2477686 G C . PASS AF=0.745;GWASCAT=Non_obstructive_azoospermia 1 2513216 rs734999 C T . PASS AF=0.547;GWASCAT=Ulcerative_colitis 1 2526746 rs3748816 A G . PASS AF=0.489;GWASCAT=Celiac_disease 1 3083712 rs2651899 T C . PASS AF=0.467;GWASCAT=Migraine 1 3280253 rs6658356 G A . PASS AF=0.070;GWASCAT=Response_to_statin_therapy 1 4315204 rs966321 G T . PASS AF=0.522;GWASCAT=Factor_VII 1 5170712 rs7513590 A G . PASS AF=0.256;GWASCAT=Anthropometric_traits 1 6279370 rs846111 G C . PASS AF=0.153;GWASCAT=QT_interval,QT_interval 1 6631431 rs11587438 C T . PASS AF=0.906;GWASCAT=White_blood_cell_types 1 7879063 rs2797685 C T . PASS AF=0.186;GWASCAT=Crohn_s_disease 1 8021973 rs35675666 G T . PASS AF=0.093;GWASCAT=Ulcerative_colitis 1 8046672 rs12727642 C A . PASS AF=0.101;GWASCAT=Celiac_disease 1 8422676 rs2252865 T C . PASS AF=0.771;GWASCAT=Schizophrenia 1 8526142 rs4908760 G A . PASS AF=0.630;GWASCAT=Vitiligo
The dbNSFP is an integrated database of functional predictions from multiple algorithms (SIFT, Polyphen2, LRT and MutationTaster, PhyloP and GERP++, etc.).
One of the main advantages is that you can annotate using multiple prediction tools with just one command. This allows for faster annotations. Here is the link to dbNSFP database website for more details.
Database: In order to annotate using dbNSFP, you need to download the dbNSFP database and the index file.
dbNSFP is large (several GB) so it might take a while to download it.
The database is compressed (block-gzip) and tabix indexed, so two files are required (the data *.gz file and the *.gz.tbi index file).
You can download the files from SnpEff's site (remember that you need both the databse and the index file):
Here is a full example how to perform annotations
# Annotate using dbNSFP # Note that the fisr time you run the command, it will attempt to download the dbNSFP database. java -jar SnpSift.jar dbnsfp -v myFile.vcf > myFile.annotated.vcf
java -jar SnpSift.jar dbNSFP
-db
command line option:
# Annotate using dbNSFP java -jar SnpSift.jar dbnsfp -v -db path/to/my/dbNSFP2.9.txt.gz myFile.vcf > myFile.annotated.vcf
# Download dbNSFP database $ wget http://dbnsfp.houstonbioinformatics.org/dbNSFPzip/dbNSFP2.9.zip # Uncompress $ unzip dbNSFP2.9.zip # Create a single file version $ (head -n 1 dbNSFP2.9_variant.chr1 ; cat dbNSFP2.9_variant.chr* | grep -v "^#" ) > dbNSFP2.9.txt # Compress using block-gzip algorithm bgzip dbNSFP2.9.txt # Create tabix index tabix -s 1 -b 2 -e 2 dbNSFP2.9.txt.gz
Building dbNSFP for hg19/GRCh37 using dbNSFP 3.X:
Latest dbNSFP versions (3.X) are based on GRCh38/hg38 genomic coordinates.
In order to use the latest dbNSFP databses with GRCh37/hg19 genome versions you need to create a new dbNSFP file with the right coordinates.
Fortunately, dbNSFP 3.X provides GRCh37/hg19 coordinates, so we only need to swap coordinates and sort by genomic position.
You can easily do this by using the dbNSFP_sort.pl
script (you can find it here) by running something like the following command lines:
# Set to your downloaded dbNSFP version version="3.2a" # Replace coordinates by columns 7 and 8 (hg19 coordinates) and sort by those coordinates cat dbNSFP${version}_variant.chr* \ | $HOME/snpEff/scripts_build/dbNSFP_sort.pl 7 8 \ > dbNSFP${version}_hg19.txt # Compress and index bgzip dbNSFP${version}_hg19.txt tabix -s 1 -b 2 -e 2 dbNSFP${version}_hg19.txt.gz
Simply split (or join) VCF files. Allows to create one file per chromosome or one file every N lines.
A typical usage for this command is to:SnpSift split huge.vcf
SnpSift split -j huge.000.vcf huge.001.vcf huge.002.vcf ... > huge.out.vcf
.
java -jar SnpSift.jar split myHugeVcf.vcf.gz
Will create files myHugeVcf.1.vcf, myHugeVcf.2.vcf, ... , myHugeVcf.22.vcf, myHugeVcf.X.vcf, myHugeVcf.Y.vcf
You can also specify '-l' command line option to split the file every N lines
E.g.: Split a VCF file every 10,000 lines
java -jar SnpSift.jar split -l 10000 myHugeVcf.vcf.gz
Will create files myHugeVcf.001.vcf, myHugeVcf.002.vcf, ...
VCF header will be added to each file, so resulting files will be more than 10,000 lines.-j
(join) command line option to join a set of VCF files.
java -jar SnpSift.jar split -j huge.000.vcf huge.001.vcf huge.002.vcf ... > huge.out.vcf
Annotate using PhastCons conservation scores.
You must download PhastCons files here
You also need a chromosome size file, which can be created using samtools faidx
, or you can download it from here
Full example.
Most of the example deals with downloading and installing PhastCons database, which is done only once.
The real annotation process is done in the last line.
# Create a dir for PhastCons database cd ~/snpEff mkdir -p db/phastCons/ # Download all PhastCons files cd db/phastCons/ wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr1.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr2.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr3.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr4.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr5.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr6.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr7.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr8.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr9.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr10.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr11.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr12.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr13.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr14.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr15.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr16.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr17.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr18.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr19.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr20.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr21.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chr22.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chrM.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chrX.phastCons100way.wigFix.gz wget http://hgdownload.soe.ucsc.edu/goldenPath/hg19/phastCons100way/hg19.100way.phastCons/chrY.phastCons100way.wigFix.gz # Create a chromosome size file and name it "genome.fai" samtools faidx path/to/genome/hg19.fa.gz cp path/to/genome/hg19.fa.gz.fai ./genome.fai # Now we are ready to annotate java -Xmx4g -jar SnpSift.jar phastCons ~/snpEff/db/phastCons file.vcf > file.phastCons.vcf
-bed
command line option.
In the output BED formatted intervals, the score column (fifth column), is the average conservation score of all bases within the interval.
-minScore score
and -extract len
command line options.
For instance, the following command:
java -jar SnpSift.jar phastCons -minScore 0.8 -extract 10 -bed path/to/phastCons/dir input.bedextracts all subintervals from each line in
input.bed
, that has at least 10
bases length and a conservation score of 0.8
Calculate concordance between two VCF files.
This is typically used when you want to calculate concordance between a genotyping experiment and a sequencing experiment.
For instance, you sequenced several samples and, as part of a related experiment or just as quality control, you also genotype the same samples using a genotyping array.
Now you want to compare the two experiments.
Ideally there would be no difference between the variants from genotyping and sequencing, but this is hardly the case in real world.
You can use SnpSift concordance
to measure the differences between the two experiments.
It is assumed that both VCF files are sorted by chromosome and position.
Sample names are defined in '#CHROM' line of the header section.
Concordance is calculated only if sample label matches in both files.
Example:
$ java -Xmx1g -jar SnpSift.jar concordance -v genotype.vcf sequencing.vcf > concordance.txt 00:00:00.000 Indexing file 'genotype.vcf' index: MT 460030998 index: 1 19705 1 / 2 45170805 / 45174315 2 / 3 77052081 / 77055591 3 / 4 104065531 / 104069041 4 / 5 124098372 / 124101881 5 / 6 146535292 / 146538802 6 / 7 184793526 / 184797035 7 / 8 206156508 / 206160018 8 / 9 223072816 / 223076326 9 / 10 242315995 / 242319505 10 / 11 261053789 / 261057299 11 / 12 290190553 / 290194063 12 / 13 312869636 / 312873146 13 / 14 321966539 / 321970049 14 / 15 336131317 / 336134827 15 / 16 350871669 / 350875179 16 / 17 368900523 / 368904032 17 / 18 391305860 / 391309369 18 / 19 398932237 / 398935747 19 / 20 425219198 / 425222708 20 / 21 437022008 / 437025517 21 / 22 442563678 / 442567188 22 / X 451783418 / 451786927 X / Y 459553691 / 459557200 Y / MT 459588787 / 459592296 00:00:01.137 Open VCF file 'genotype.vcf' 00:00:01.141 Open VCF file 'sequencing.vcf' 00:00:01.176 Chromosome: '1' 00:00:02.127 1:1550992 1:1528859 00:00:02.739 1:2426313 1:2389636 ... 00:02:13.780 1:248487058 1:248471945
SnpSif's concordance output is written to STDOUT and two files.
For instance the command java -jar SnpSift.jar concordance -v genotype.vcf sequencing.vcf
will write:
concordance_genotyping_sequencing.by_sample.txt
concordance_genotyping_sequencing.summary.txt
This sections is a table showing concordance details for every entry (chr:position) that both files have in common. E.g.:
chr pos ref alt change_0_0 change_0_1 change_0_2 change_1_0 change_1_1 change_1_2 change_2_0 change_2_1 change_2_2 missing_genotype_genotype missing_genotype_sequencing 1 865584 G A 508 0 0 0 2 0 0 0 0 0 5 1 865625 G A 512 0 0 0 1 0 0 0 0 0 1 1 865628 G A 511 0 0 0 2 0 0 0 0 0 1 1 865665 G A 495 0 0 0 4 0 0 0 0 0 17 1 865694 C T 428 0 0 0 82 0 0 0 4 0 0
This section shows details in the same format as the previous section. Here, concordance metrics are shown aggregated for each sample. E.g.:
# Totals by sample sample change_0_0 change_0_1 change_0_2 change_1_0 change_1_1 change_1_2 change_2_0 change_2_1 change_2_2 missing_genotype_genotype missing_genotype_sequencing ID_003 79 0 0 1 8 0 0 0 2 1 1 ID_004 83 0 0 1 2 0 0 0 5 0 1 ID_005 80 0 0 0 7 0 0 0 4 1 0 ID_006 79 0 0 0 5 0 0 0 6 0 2 ID_008 81 0 0 0 4 0 0 0 4 0 3 ID_009 80 0 0 0 7 0 0 0 3 0 2 ID_012 74 0 0 0 10 0 0 0 1 0 7 ID_013 79 1 0 0 4 0 0 0 5 0 3 ID_018 84 0 0 0 5 0 0 0 3 0 0 ...Summary
Summary file contains overall information and errors. Here is an example of a summary file:
$ cat concordance_genotyping_sequencing.summary.txt Number of samples: 929 File genotype.vcf 583 File sequencing.vcf 514 Both files Errors: ALT field does not match 19The header indicates that one file ('genotype.vcf') has 929 samples, the other file has 583 and there are 514 matching sample IDs in both files.
-v
).
Annotate if a variant is private to a family.
A Private=Family_ID
is added to a variant's INFO field, if the variant is only found in one family.
A TFAM file (see PLINK's documentation) specifies a mapping from sample IDs to family IDs.
E.g.:
$ java -jar SnpSift.jar private pheotypes.tfam imp.ann.vcf > imp.ann.private.vcf
1 1005806 rs3934834 C T . PASS AF=0.091;..;Private=Family_47This indicates that the variant is only found in members of
Family_47
, according to the definitions in pheotypes.tfam
.
Convert from VCF to PLINK's TPED file format.
The vcf2tped
command uses a VCF and a TFAP file as input, creating a TPED and a consolidated TFAM as outputs.
Command line options are:
$ java -jar SnpSift.jar vcf2tped SnpSift version 1.9d (build 2013-04-26), by Pablo Cingolani Usage: java -jar SnpSift.jar vcf2tped [options] file.tfam file.vcf outputName Options: -f : Force. Overwrite new files if they exist. Default: false -onlySnp : Use only SNPs when converting VCF to TPED. Default: false -onlyBiAllelic : Use only bi-allelic variants. Default: false -useMissing : Use entries with missing genotypes (otherwise they are filtered out). Default: false -useMissingRef : Use entries with missing genotypes marking them as 'reference' instead of 'missing'. Default: false Parameters: file.tfam : File with genotypes and groups information (in PLINK's TFAM format) file.vcf : A VCF file (variants and genotype data) outputName : Base name for the new TPED and TFAM files.
vcf2tped
command supports the following features:
-onlyBiAllelic
option filters out non bi-allelic variants.
-onlySnp
option filters out non SNP variants.
-useMissing
uses missing variants in TPED file.
-useMissingRef
Converts missing variants to reference genotype.
This command intersects several intervals files (e.g. BED, BigBed, TXT) and produces a result of all intersections.
A typical usage example is to create a consensus of peaks from several Chip-Seq experiments.
Algorithm: This command creates one interval forest for each input file.
For every interval in all input files, finds all intervals that intersect at least minOverlap
bases (default 1 base).
If there are at least cluster
number of intersecting intervals it creates a consensus interval from the intersections (or union
) of all intervals found.
The consensus interval, if any, is shown as result.
Command line options:
$ java -jar SnpSift.jar intersect SnpSift version 1.9d (build 2013-04-26), by Pablo Cingolani Usage: java -jar SnpSift.jar [options] file_1.bed file_2.bed ... file_N.bed Options: -minOverlap: Minimum number of bases that two intervals have to overlap. Default : 0 -cluster : An interval has to intersect at least 'num' intervals (from other files) to be considered. Default: 0 -intersect : Report the intersection of all intervals. Default: false -union : Report the union of all intervals. Default: true
This command removes INFO fields from a VCF file (i.e. removes annotations)
Removing INFO fields is usually done because you want to re-annotate the VCF file, thus removing old INFO fields in order to add new ones later.
SnpEff & SnpSift only add annotations and do not change current ones.
So, in order to re-annotate a file, you should first remove the old annotations and then re-annotate.
The reason for this behavior is simply because replacing annotation values is considered a bad practice.
Imagine that you have a VCF entry in your re-annotated file having the value "AA=1": How do you know if this is from the old annotations or from the new ones?
This confusion often leads to problems in downstream steps of your pipelines, so it's better to avoid the problem by first removing all the previous annotations and then adding the new ones.
Usage example:
$ cat test.snpeff.vcf #CHROM POS ID REF ALT QUAL FILTER INFO 1<PHONE_NUMBER> G A . s50 AC=348;EFF=DOWNSTREAM(MODIFIER|||||RP11-206L10.8|processed_transcript|NON_CODING|ENST00000447500||1),INTRON(MODIFIER|||||RP11-206L10.6|processed_transcript|NON_CODING|ENST00000429505|1|1) $ java -jar SnpSift.jar rmInfo test.snpeff.vcf EFF #CHROM POS ID REF ALT QUAL FILTER INFO 1<PHONE_NUMBER> G A . s50 AC=348
Annotating GeneSets, such as Gene Ontology (GO), KEGG, Reactome, etc.; can be quite useful to find significant variants.
Gene set annotations can be added to a SnpEff annotated file using SnpSift geneSets
command.
The VCF file must be annotated using SnpEff
before perforimg Gene Sets annotations.
This is because we must know which gene the variant affects).
You can download MSigDb from Broad Institute
Usage example:
$ java -jar SnpSift.jar geneSets -v db/msigDb/msigdb.v3.1.symbols.gmt test.ann.vcf > test.eff.geneSets.vcf 00:00:00.000 Reading MSigDb from file: 'db/msigDb/msigdb.v3.1.symbols.gmt' 00:00:01.168 Done. Total: 8513 gene sets 31847 genes 00:00:01.168 Annotating variants from: 'test.ann.vcf' 00:00:01.298 Done. # Summary # gene_set gene_set_size variants # ACEVEDO_METHYLATED_IN_LIVER_CANCER_DN 940 8 # CHR1P36 504 281 # KEGG_OLFACTORY_TRANSDUCTION 389 8 # REACTOME_GPCR_DOWNSTREAM_SIGNALING 805 8 # REACTOME_OLFACTORY_SIGNALING_PATHWAY 328 8 ... # REACTOME_SIGNALING_BY_GPCR 920 8 $ cat test.eff.geneSets.vcf ##INFO=<ID=MSigDb,Number=.,Type=String,Description="Gene set from MSigDB database (GSEA)"> 1 69849 . G A 454.73 PASS AC=33;EFF=STOP_GAINED(HIGH|NONSENSE|tgG/tgA|W253*|305|OR4F5|protein_coding|CODING|ENST00000335137|1|1);MSigDb=ACEVEDO_METHYLATED_IN_LIVER_CANCER_DN,CHR1P36,KEGG_OLFACTORY_TRANSDUCTION,REACTOME_GPCR_DOWNSTREAM_SIGNALING,REACTOME_OLFACTORY_SIGNALING_PATHWAY,REACTOME_SIGNALING_BY_GPCR
Compress genotype calls, reducing the overall size of the VCF file.
This is intended for compressing very large VCF in very large sequencing projects (e.g. thousands of samples).
For instance, we've reduced 1Tb (1,000 Gb) VCF file to roughly 1Gb in a project that has over 10,000 samples.
The underlying idea is quite simple.
In large re-sequenceing projects most of the variants are singletons.
This means that most variants are present in only one of the samples.
For those variants, you have thousands of samples that are homozygous reference (i.e. genotype entry is "0/0") and one that is a variant (e.g. '0/1' or '1/1').
A trivial way to compress these VCF entries is just to state which sample has non-reference information.
Intuitively, this is similar to the way used to represent sparse matrices (only store non-zero elements).
SnpSift gt
creates three INFO fields. These three files are composed of comma separated indexes of samples having:
-u
command line option to uncompress.
$ cat test.vcf #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Sample_1 Sample_2 Sample_3 Sample_4 Sample_5 Sample_6 Sample_7 Sample_8 Sample_9 Sample_10 Sample_11 Sample_12 Sample_13 Sample_14 Sample_15 1 861276 . A G . PASS AC=1 GT 0/0 1/1 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 #--- # Compress genotypes #--- $ java -jar SnpSift.jar gt test.vcf | tee test.gt.vcf ##INFO=<ID=HO,Number=.,Type=Integer,Description="List of sample indexes having homozygous ALT genotypes"> ##INFO=<ID=HE,Number=.,Type=Integer,Description="List of sample indexes having heterozygous ALT genotypes"> ##INFO=<ID=NA,Number=.,Type=Integer,Description="List of sample indexes having missing genotypes"> #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Sample_1 Sample_2 Sample_3 Sample_4 Sample_5 Sample_6 Sample_7 Sample_8 Sample_9 Sample_10 Sample_11 Sample_12 Sample_13 Sample_14 Sample_15 1 861276 . A G . PASS AC=1;HO=1 #--- # Uncompress genotypes (command line option '-u') #--- $ java -jar SnpSift.jar gt -u test.gt.vcf #CHROM POS ID REF ALT QUAL FILTER INFO FORMAT Sample_1 Sample_2 Sample_3 Sample_4 Sample_5 Sample_6 Sample_7 Sample_8 Sample_9 Sample_10 Sample_11 Sample_12 Sample_13 Sample_14 Sample_15 1 861276 . A G . PASS AC=1 GT 0/0 1/1 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0
Perform some basic check ups on VCF files to spot common problems.
SnpSift vcfCheck
checks for some common problems where VCF files are not following the specification.
Given that many common VCF problems cause analysis tools and pipelines to behave unexpectedly, this command is intended as a simple debugging tool.
E.g.:
$ java -jar SnpSift.jar vcfCheck bad.vcf WARNING: Malformed VCF entryfile 'bad.vcf', line 7: Entry : 3 148885779 . A ATT,AT 999.0 PASS UK10KWES_AC=0,0;MDV=94 Errors : INFO filed 'UK10KWES_AC' has 'Number=1' in header, but it contains '2' elements. Cannot find header for INFO field 'MDV' WARNING: Malformed VCF entryfile 'bad.vcf', line 14: Entry : 3 148890104 . TCA T . . . Errors : File is not sorted: Position '3:148890104' after position '3:148890105'


@bazel/concatjs
Concatjs is a JavaScript bundler, in a trivial sense: the UNIX cat
command is a basic implementation:
$ cat one.js two.js > bundle.js
Clearly self-evident is that this bundler is super-fast and simple. A performant implementation adds some in-memory caching, and for developer ergonomics you add a simple IIFE wrapper around each file so that the Chrome DevTools shows the files in the tree as if they had been independently loaded.
However at its core, concatjs requires a big tradeoff of a migration cost to buy-in, to get this incredible performance. The path of the JavaScript files is lost in the bundling process, so they must contain their module ID internally.
Named AMD/UMD modules and goog.module
are the two JS module formats that are compatible with concatjs.
Most packages do not ship with this format, so in order to use concatjs tooling, you have to shim your code and dependencies. See the Compatibility section below.
This is at the core of how Google does JavaScript development.
So Bazel rules that originated in Google’s codebase have affordances for concatjs.
For example ts_library
produces named AMD modules in its “devmode” output, and
karma_web_test
expects to bundle inputs using concatjs.
Compatibility
To make it easier to produce a UMD version of a third-party npm package, we automatically generate a target that uses Browserify to build one, using the main
entry from the package’s package.json
.
In most cases this will make the package loadable under concatjs.
This target has a __umd
suffix. For example, if your library is at @npm//foo
then the UMD target is @npm//foo:foo__umd
.
An example where this fixes a users issue: https://github.com/bazelbuild/rules_nodejs/issues/2317#issuecomment-735921318
In some cases, the generated UMD bundle is not sufficient, and in others it fails to build because it requires some special Browserify configuration.
You can always write your own shim that grabs a symbol from a package you use, and exposes it in an AMD/require.js-compatible way.
For example, even though RxJS ships with a UMD bundle, it contains multiple entry points and uses anonymous modules, not named modules. So our Angular/concatjs example has a rxjs_shims.js
file that exposes some RxJS operators, then at https://github.com/bazelbuild/rules_nodejs/blob/2.3.1/examples/angular/src/BUILD.bazel#L65-L71 this is combined in a filegroup
with the rxjs.umd.js
file. Now we use this filegroup target when depending on RxJS in a concatjs_*
rule.
Ultimately by using concatjs, you’re signing up for at least a superficial understanding of these shims and may need to update them when you change your dependencies.
Serving JS in development mode under Bazel
There are two choices for development mode:
- Use the
concatjs_devserver
rule to bring up our simple, fast development server. This is intentionally very simple, to help you get started quickly. However, since there are many development servers available, we do not want to mirror their features in yet another server we maintain. - Teach your real frontend server to serve files from Bazel’s output directory.
This is not yet documented. Choose this option if you have an existing server
used in development mode, or if your requirements exceed what the
concatjs_devserver
supports. Be careful that your development round-trip stays fast (should be under two seconds).
To use concatjs_devserver
, you simply load
the rule, and call it with deps
that
point to your ts_library
target(s):
load("@npm//@bazel/typescript:index.bzl", "concatjs_devserver", "ts_library")
ts_library(
name = "app",
srcs = ["app.ts"],
)
concatjs_devserver(
name = "devserver",
# We'll collect all the devmode JS sources from these TypeScript libraries
deps = [":app"],
# This is the path we'll request from the browser, see index.html
serving_path = "/bundle.js",
# The devserver can serve our static files too
static_files = ["index.html"],
)
The index.html
should be the same one you use for production, and it should
load the JavaScript bundle from the path indicated in serving_path
.
If you don’t have an index.html file, a simple one will be generated by the
concatjs_devserver
.
See examples/app
in this repository for a working example. To run the
devserver, we recommend you use ibazel:
$ ibazel run examples/app:devserver
ibazel
will keep the devserver program running, and provides a LiveReload
server so the browser refreshes the application automatically when each build
finishes.
Testing with Karma
The karma_web_test
rule runs karma tests with Bazel.
It depends on rules_webtesting, so you need to add this to your WORKSPACE
if you use the web testing rules in @bazel/concatjs
:
# Fetch transitive Bazel dependencies of karma_web_test
http_archive(
name = "io_bazel_rules_webtesting",
sha256 = "9bb461d5ef08e850025480bab185fd269242d4e533bca75bfb748001ceb343c3",
urls = ["https://github.com/bazelbuild/rules_webtesting/releases/download/0.3.3/rules_webtesting.tar.gz"],
)
# Set up web testing, choose browsers we can test on
load("@io_bazel_rules_webtesting//web:repositories.bzl", "web_test_repositories")
web_test_repositories()
load("@io_bazel_rules_webtesting//web/versioned:browsers-0.3.2.bzl", "browser_repositories")
browser_repositories(
chromium = True,
firefox = True,
)
Installing with user-managed dependencies
If you didn’t use the yarn_install
or npm_install
rule to create an npm
workspace, you’ll have to declare a rule in your root BUILD.bazel
file to execute karma:
# Create a karma rule to use in karma_web_test_suite karma
# attribute when using user-managed dependencies
nodejs_binary(
name = "karma/karma",
entry_point = "//:node_modules/karma/bin/karma",
# Point bazel to your node_modules to find the entry point
data = ["//:node_modules"],
)
concatjs_devserver
USAGE
concatjs_devserver(name, additional_root_paths, bootstrap, deps, devserver, devserver_host, entry_module, port, scripts, serving_path, static_files)
concatjs_devserver is a simple development server intended for a quick “getting started” experience.
Additional documentation here
ATTRIBUTES
name
(Name, mandatory): A unique name for this target.
additional_root_paths
(List of strings): Additional root paths to serve static_files
from.
Paths should include the workspace name such as ["__main__/resources"]
Defaults to []
bootstrap
(List of labels): Scripts to include in the JS bundle before the module loader (require.js)
Defaults to []
deps
(List of labels): Targets that produce JavaScript, such as ts_library
Defaults to []
devserver
(Label): Go based devserver executable.
With cross-platform RBE for OSX & Windows ctx.executable.devserver will be linux as --cpu and
--host_cpu must be overridden to k8. However, we still want to be able to run the devserver on the host
machine so we need to include the host devserver binary, which is ctx.executable.devserver_host, in the
runfiles. For non-RBE and for RBE with a linux host, ctx.executable.devserver & ctx.executable.devserver_host
will be the same binary.
Defaults to precompiled go binary setup by @bazel/typescript npm package
Defaults to @npm//@bazel/devserver:devserver
devserver_host
(Label): Go based devserver executable for the host platform. Defaults to precompiled go binary setup by @bazel/typescript npm package
Defaults to @npm//@bazel/devserver:devserver_darwin_amd64
entry_module
(String): The entry_module
should be the AMD module name of the entry module such as "__main__/src/index".
concatjs_devserver
concats the following snippet after the bundle to load the application:
require(["entry_module"]);
Defaults to ""
port
(Integer): The port that the devserver will listen on.
Defaults to 5432
scripts
(List of labels): User scripts to include in the JS bundle before the application sources
Defaults to []
serving_path
(String): The path you can request from the client HTML which serves the JavaScript bundle. If you don’t specify one, the JavaScript can be loaded at /_/ts_scripts.js
Defaults to "/_/ts_scripts.js"
static_files
(List of labels): Arbitrary files which to be served, such as index.html. They are served relative to the package where this rule is declared.
Defaults to []
karma_web_test
USAGE
karma_web_test(srcs, deps, data, configuration_env_vars, bootstrap, runtime_deps, static_files, config_file, tags, peer_deps, kwargs)
Runs unit tests in a browser with Karma.
When executed under bazel test
, this uses a headless browser for speed.
This is also because bazel test
allows multiple targets to be tested together,
and we don’t want to open a Chrome window on your machine for each one. Also,
under bazel test
the test will execute and immediately terminate.
Running under ibazel test
gives you a “watch mode” for your tests. The rule is
optimized for this case - the test runner server will stay running and just
re-serve the up-to-date JavaScript source bundle.
To debug a single test target, run it with bazel run
instead. This will open a
browser window on your computer. Also you can use any other browser by opening
the URL printed when the test starts up. The test will remain running until you
cancel the bazel run
command.
This rule will use your system Chrome by default. In the default case, your
environment must specify CHROME_BIN so that the rule will know which Chrome binary to run.
Other browsers
and customLaunchers
may be set using the a base Karma configuration
specified in the config_file
attribute.
By default we open a headless Chrome. To use a real Chrome browser window, you can pass
--define DISPLAY=true
to Bazel, along with configuration_env_vars = ["DISPLAY"]
on
karma_web_test
.
PARAMETERS
srcs
A list of JavaScript test files
Defaults to []
deps
Other targets which produce JavaScript such as ts_library
Defaults to []
data
Runtime dependencies
Defaults to []
configuration_env_vars
Pass these configuration environment variables to the resulting binary. Chooses a subset of the configuration environment variables (taken from ctx.var), which also includes anything specified via the –define flag. Note, this can lead to different outputs produced by this rule.
Defaults to []
bootstrap
JavaScript files to include before the module loader (require.js). For example, you can include Reflect,js for TypeScript decorator metadata reflection, or UMD bundles for third-party libraries.
Defaults to []
runtime_deps
Dependencies which should be loaded after the module loader but before the srcs and deps.
These should be a list of targets which produce JavaScript such as ts_library
.
The files will be loaded in the same order they are declared by that rule.
Defaults to []
static_files
Arbitrary files which are available to be served on request.
Files are served at:
/base/<WORKSPACE_NAME>/<path-to-file>
, e.g.
/base/npm_bazel_typescript/examples/testing/static_script.js
Defaults to []
config_file
User supplied Karma configuration file. Bazel will override certain attributes of this configuration file. Attributes that are overridden will be outputted to the test log.
Defaults to None
tags
Standard Bazel tags, this macro adds tags for ibazel support
Defaults to []
peer_deps
list of peer npm deps required by karma_web_test
Defaults to ["@npm//karma", "@npm//karma-chrome-launcher", "@npm//karma-firefox-launcher", "@npm//karma-jasmine", "@npm//karma-requirejs", "@npm//karma-sourcemap-loader", "@npm//requirejs"]
kwargs
Passed through to karma_web_test
karma_web_test_suite
USAGE
karma_web_test_suite(name, browsers, web_test_data, wrapped_test_tags, kwargs)
Defines a test_suite of web_test targets that wrap a karma_web_test target.
This macro accepts all parameters in karma_web_test and adds additional parameters for the suite. See karma_web_test docs for all karma_web_test.
The wrapping macro is web_test_suite
which comes from rules_websting:
https://github.com/bazelbuild/rules_webtesting/blob/master/web/web.bzl.
PARAMETERS
name
The base name of the test
browsers
A sequence of labels specifying the browsers to use.
Defaults to None
web_test_data
Data dependencies for the wrapper web_test targets.
Defaults to []
wrapped_test_tags
A list of test tag strings to use for the wrapped karma_web_test target.
Defaults to ["manual", "noci"]
kwargs
Arguments for the wrapped karma_web_test target.


{% extends 'registration/base.html' %} {% block content %}
We've emailed you instructions to reset your password.If an account exists with the email you entered ,you should receive them shortly.
If you don’t receive an email, please make sure you’ve entered the correct email address you registered with, and check your spam folder.
{% endblock content %}


{% extends "base.html" %} {% block title %}About{% endblock %} {% block content %}
Stay up to date about news from GAFAM without account.


Browse to a collection or element using the sidebar to view its documentation.
- The frames are available frame types to use in Lookit studies - like instructions pages or looking-time trials. The documentation will tell you how each frame works and what parameters you need to give it.
- The randomizers are used to set up counterbalancing or condition assignment. The randomizers described here correspond to values for 'sampler' when you define a frame with type 'choice'. The documentation will tell you how each randomizer works and what parameters you need to give it.
- The mixins and services are not relevant unless you are creating your own frames.
- These are the docs for particular frames - they are autogenerated from comments in the source code. For more general (and readable) information on how to define and test your Lookit study, please see the Lookit docs.


--- layout: post title: change default directory listing index date: type: post published: false status: private categories: [] tags: [] meta: _edit_last: '1' author: login: satyadeep email<EMAIL_ADDRESS>display_name: Satyadeep first_name: Satyadeep last_name: K ---
Place the following in httpd.conf or apache.conf in your apache installation folder.
<Directory /var/www/>
# enable ssi directives
Options +Includes
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
# avoid duplicated <html><body> and no column sorting
# place folders first
# IndexOptions FoldersFirst
# define absolute path to the header file from server home
HeaderName /header.html
ReadmeName /readme.html
</Directory>
then restart apache.
Ref: http://www.redips.net/apache/styling-directory-listings/


{% extends "post_sequence.html" %}
{% block page_name %}@{{ user_id }}{% endblock page_name %}
{% block sequence_header %}Posts by @{{ user_id }}{% endblock sequence_header %}
{% block sequence_empty %}There are no posts by @{{ user_id }} yet!{% endblock sequence_empty %}