--- layout: page title: "Learning GNU Emacs, 3rd Edition" prev: OEBPS/apd.html next: OEBPS/co01.html book_path: books/learning-gnu-emacs--_oeb/ --- {% include JB/setup %} {% raw %}
This quick reference is arranged topically, in roughly the same order as the commands were treated in the text. Unfortunately, it's impossible to be both "quick" and thorough, particularly with an editor as large and comprehensive as GNU Emacs. We've tried to take a middle road between completeness and quickness; we'll confess that, if we've erred, we've erred on the side of quickness.
Table E-1. File-handling commands (Chapter 1)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x C-f File → Open File |
find-file |
Find file and read it in a new buffer. |
|
C-x C-v |
find-alternate-file |
Read an alternate file, replacing the one read with C-x C-f. |
|
C-x i File → Insert File |
insert-file |
Insert file at cursor position. |
|
C-x C-s File → Save (current buffer) |
save-buffer |
Save file. |
|
C-x C-w File → Save Buffer As |
write-file |
Write buffer contents to file. |
|
C-x C-c File → Exit Emacs |
save-buffers-kill-emacs |
Exit Emacs. |
Table E-2. Cursor movement commands (Chapter 2)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-f |
forward-char |
Move forward one character (right). |
|
C-b |
backward-char |
Move backward one character (left). |
|
C-p |
previous-line |
Move to previous line (up). |
|
C-n |
next-line |
Move to next line (down). |
|
M-f |
forward-word |
Move one word forward. |
|
M-b |
backward-word |
Move one word backward. |
|
C-a |
beginning-of-line |
Move to beginning of line. |
|
C-e |
end-of-line |
Move to end of line. |
|
M-e |
forward-sentence |
Move forward one sentence. |
|
M-a |
backward-sentence |
Move backward one sentence. |
|
C-v |
scroll-up |
Move forward one screen. |
|
M-v |
scroll-down |
Move backward one screen. |
|
M-< |
beginning-of-buffer |
Move to beginning of file. |
|
M-> |
end-of-buffer |
Move to end of file. |
|
(none) |
goto-line |
Go to line n of file. |
|
(none) |
goto-char |
Go to character n of file. |
|
M- n |
digit-argument |
Repeat the next command n times. |
|
C-u n |
universal-argument |
Repeat the next command n times (four times if you omit n). |
Table E-3. Deleting, yanking, region, and clipboard commands(Chapter 2)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-d |
delete-char |
Delete character under cursor. |
|
Del |
delete-backward-char |
Delete previous character. |
|
M-d |
kill-word |
Delete next word. |
|
M-Del |
backward-kill-word |
Delete previous word. |
|
C-y Edit → Paste Most Recent |
yank |
Restore what you've deleted. |
|
C-w Edit → Cut |
kill-region |
Delete a marked region (see next section). |
|
C-@ or C- Space |
set-mark-command |
Mark the beginning (or end) of a region. |
|
C-x C-x |
exchange-point-and-mark |
Exchange location of cursor and mark. |
|
C-w |
kill-region |
Delete the region. |
|
C-y |
yank |
Paste most recently killed or copied text. |
|
M-w |
kill-ring-save |
Copy the region (so it can be pasted with C-y). |
|
C-x h |
mark-whole-buffer |
Mark buffer. |
|
M-y |
yank-pop |
After C-y, pastes earlier deletion. |
|
(none) |
clipboard-kill-region |
Cut region and place both in kill ring and on system clipboard. |
|
(none) |
clipboard-yank |
Paste text from clipboard. |
|
(none) |
clipboard-kill-ring-save |
Copy text to clipboard. |
Table E-4. Text filling and reformatting commands (Chapter 2)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
(none) Options → Word Wrap in Text Modes |
auto-fill-mode |
Toggle auto-fill mode, in which Emacs formats paragraphs as you type them. |
|
M-q |
fill-paragraph |
Reformat paragraph. |
|
(none) Edit → Fill |
fill-region |
Reformat individual paragraphs within a region. |
Table E-5. Stopping and undoing commands (Chapter 2)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-g |
keyboard-quit |
Abort current command. |
|
C-x u |
advertised-undo |
Undo last edit (can be done repeatedly). |
|
C-_ Edit → Undo |
undo |
Undo last edit. |
|
(none) |
revert-buffer |
Restore buffer to the state it was in when the file was last saved (or auto-saved). |
Table E-6. Search and replace commands (Chapter 3)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-s Edit → Search → Incremental Search → Forward String |
isearch-forward |
Start incremental search forward; follow by search string. Also, find next occurrence (forward) of search string. |
|
C-r Edit → Search → Incremental Search → Backward String |
isearch-backward |
Start incremental search backward; follow by search string. Also, find next occurrence (backward) of search string. |
Table E-7. Regular expression search commands (Chapter 3)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-M-s Enter Edit → Search → Regexp Forward |
re-search-forward |
Search for a regular expression forward. |
|
C-M-r Enter Edit → Search → Regexp Backwards |
re-search-backward |
Search for a regular expression backward. |
|
C-M-s Edit → Search → Incremental Search → Forward Regexp |
isearch-forward-regexp |
Search incrementally forward for a regular expression. |
|
C-M-r Edit → Search → Incremental Search → Backward Regexp |
isearch-backward-regexp |
Search incrementally backward for a regular expression. |
|
C-M-% Edit → Replace → Replace Regexp |
query-replace-regexp |
Query-replace a regular expression. |
Table E-8. Spell-checking commands (Chapter 3)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
(none) Tools → Spell Checking → Spell-Check Buffer |
ispell-buffer |
Check spelling of the buffer. |
|
(none) Tools → Spell Checking → Spell-Check Comments |
ispell-comments-and-strings |
Checks spelling of comments and strings in a program. |
|
(none) Tools → Spell Checking → Automatic Spell-Checking (Flyspell) |
flyspell-mode |
Enter the flyspell minor mode, in which incorrectly spelled words are highlighted. |
|
(none) |
flyspell-buffer |
Spell-check the current buffer, underlining all misspelled words. Use middle mouse button to correct. |
Table E-9. Buffer commands (Chapter 4)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x b Buffers → Select Named Buffer |
switch-to-buffer |
Move to the buffer specified. |
|
C-x → Buffers → Next Buffer |
next-buffer |
Move to the next buffer in the buffer list. |
|
C-x Buffers → Previous Buffer |
previous-buffer |
Move to the previous buffer in the buffer list. |
|
C-x C-b Buffers → List All Buffers |
list-buffers |
Display the buffer list. |
|
C-x k |
kill-buffer |
Delete the buffer specified. |
|
(none) |
kill-some-buffers |
Ask about deleting each buffer. |
Table E-10. Windows and frames (Chapter 4)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x 2 File → Split Window |
split-window-vertically |
Divide current window into two windows, one above the other. |
|
C-x 3 |
split-window-horizontally |
Divide current window into two side-by-side windows. |
|
C-x o |
other-window |
Move to the other window; if there are several, move to the next window. |
|
C-x 0 |
delete-window |
Delete the current window. |
|
C-x 1 File → Unsplit Windows |
delete-other-windows |
Delete all windows but this one. |
|
C-x 4 f |
find-file-other-window |
Find a file in the other window. |
|
C-x 4 b |
switch-to-buffer-other-window |
Select a buffer in the other window. |
|
(none) Tools → Compare (Ediff) → This Window and Next Window |
compare-windows |
Compare this window with the next window and show the first difference. |
|
C-x 5 o Buffers → Frames |
other-frame |
Move to other frame. |
|
C-x 5 0 File → Delete Frame |
delete-frame |
Delete current frame. |
|
C-x 5 2 File → New Frame |
make-frame |
Create a new frame on the current buffer. |
|
C-x 5 f |
find-file-other-frame |
Find file in a new frame. |
|
C-x 5 r |
find-file-read-only-other-frame |
Finds a file in a new frame, but it is read-only. |
|
C-x 5 b |
switch-to-buffer-other-frame |
Make frame and display other buffer in it. |
Table E-11. Shell mode commands (Chapter 5)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
(none) |
shell |
Enter shell mode. |
|
C-c C-c Signals → BREAK |
comint-interrupt-subjob |
Interrupt current job; equivalent to C-c. |
|
C-c C-z Signals → STOP |
comint-stop-subjob |
Suspend or stop a job; C-z in Unix shells. |
|
M-p In/Out → Previous Input |
comint-previous-input |
Retrieve previous commands (can be repeated to find earlier commands). |
|
M-n In/Out → Next Input |
comint-next-input |
Retrieve subsequent commands (can be repeated to find more recent commands). |
|
Enter |
comint-send-input |
Send input on current line. |
|
Tab |
comint-dynamic-complete |
Complete current command, filename, or variable name. |
Table E-12. Dired commands (Chapter 5)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x d File → Open Directory |
dired |
Start Dired. |
|
C Operate → Copy to |
dired-do-copy |
Copy file. |
|
d Mark → Flag |
dired-flag-file-deletion |
Flag for deletion. |
|
D Operate → Delete |
dired-do-delete |
Query for immediate deletion. |
|
f |
dired-advertised-find-file |
Find (so you can edit). |
|
g Immediate → Refresh |
revert-buffer |
Reread the directory from disk. |
|
m or * m Mark → Mark |
dired-mark |
Mark with *. |
|
Q Operate → Query Replace in Files |
dired-do-query-replace |
Query replace string in marked files. |
|
R Operate → Rename to |
dired-do-rename |
Rename file. |
|
s |
dired-sort-toggle-or-edit |
Sort the Dired display by date or by filename (toggles between these). |
|
t Mark → Toggle Marks |
dired-toggle-marks |
Toggle marks on files and directories; pressing t once marks all unmarked files and directories; pressing t again restores original marks. |
|
u Mark → Unmark |
dired-unmark |
Remove mark. |
|
+ Immediate → Create Directory |
dired-create-directory |
Create a directory. |
|
* ! or M-Del Mark → Unmark All |
dired-unmark-all-files |
Remove all marks from all files. |
Table E-13. Macro commands (Chapter 6)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x ( |
kmacro-start-macro |
Start macro definition. |
|
F3 |
kmacro-start-macro-or-insert-counter |
Start macro definition. If pressed while defining a macro, insert a counter. |
|
C-x ) |
kmacro-end-macro |
End macro definition. |
|
F4 |
kmacro-end-or-call-macro |
End macro definition (if definition is in progress) or invoke last keyboard macro. |
|
C-x e |
kmacro-end-and-call-macro |
Execute last keyboard macro defined. Can type e to repeat macro. |
|
C-x C-k n |
name-last-kbd-macro |
Name the last macro you created (before saving it). |
|
C-x C-k e |
edit-kbd-macro |
Edit a keyboard macro by typing C-x e for the last keyboard macro defined, M-x for a named macro, C-h l for lossage, or keystrokes for a macro bound to a key. |
|
C-x C-k Enter |
kmacro-edit-macro |
Edit the last keyboard macro. |
Table E-14. Outline mode commands (Chapter 7)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
(none) |
outline-mode |
Toggle outline mode |
|
C-c C-t Hide → Hide Body |
hide-body |
Hide all body lines. |
|
C-c C-a Show → Show All |
show-all |
Show everything that's hidden. |
Table E-15. Compilation mode commands (Chapter 9)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x ` |
next-error |
Move to the next error message and visit the corresponding source code. |
|
M-n |
compilation-next-error |
Move to the next error message. |
|
M-p |
compilation-previous-error |
Move to the previous error message. |
|
C-c C-c |
compilation-goto-error |
Visit the source code for the current error message. |
Table E-16. Basic indentation commands (Chapter 7 and Chapter 9)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-M-\ |
indent-region |
Indent each line between the cursor and mark. |
|
M-m |
back-to-indentation |
Move to the first nonblank character on the line. |
|
M-^ |
delete-indentation |
Join this line to the previous one. |
Table E-17. C motion commands (Chapter 9)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
M-a |
c-beginning-of-statement |
Move to the beginning of the current statement. |
|
M-e |
c-end-of-statement |
Move to the end of the current statement. |
|
M-q |
c-fill-paragraph |
If in comment, fill the paragraph, preserving indentations and decorations. |
|
C-M-a |
beginning-of-defun |
Move to the beginning of the body of the function surrounding the point. |
|
C-M-e |
end-of-defun |
Move to the end of the function. |
|
C-M-h |
c-mark-function |
Put the cursor at the beginning of the function, the mark at the end. |
|
C-c C-q |
c-indent-defun |
Indent the entire function according to indentation style. |
|
C-c C-u |
c-up-conditional |
Move to the beginning of the current preprocessor conditional. |
|
C-c C-p |
c-backward-conditional |
Move to the previous preprocessor conditional. |
|
C-c C-n |
c-forward-conditional |
Move to the next preprocessor conditional. |
Table E-18. SQL mode commands (Chapter 9)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-c C-c |
sql-send-paragraph |
Send the paragraph the cursor is on. A paragraph is defined by the particular database client. For the sql-mysql process, for example, a paragraph begins with a statement like select or update and ends with a semicolon. Any number of lines can intervene. |
|
C-c C-r |
sql-send-region |
Send the marked region. |
|
C-c C-b |
sql-send-buffer |
Send the entire buffer. |
Table E-19. Lisp commands (Chapter 9)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-M-b |
backward-sexp |
Move backward by one S-expression. |
|
C-M-f |
forward-sexp |
Move forward by one S-expression. |
|
C-M-t |
transpose-sexps |
Transpose the two S-expressions around the cursor. |
|
C-M-@ |
mark-sexp |
Set mark to the end of the current S-expression; set the cursor to the beginning. |
|
C-M-k |
kill-sexp |
Delete the S-expression following the cursor. |
|
(none) |
backward-kill-sexp |
Delete the S-expression preceding the cursor. |
|
C-M-n |
forward-list |
Move forward by one list. |
|
C-M-p |
backward-list |
Move backward by one list. |
|
C-M-d |
down-list |
Move forward and down one parenthesis level. |
|
(none) |
up-list |
Move forward out of one parenthesis level. |
|
C-M-u |
backward-up-list |
Move backward out of one parenthesis level. |
|
C-M-a |
beginning-of-defun |
Move to the beginning of the current function. |
|
C-M-e |
end-of-defun |
Move to the end of the current function. |
|
C-M-h |
mark-defun |
Put the cursor at the beginning of the function, put the mark at the end. |
Table E-20. VC commands (Chapter 12)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-x v v |
vc-next-action |
Go to the next logical version control state. |
|
C-x v d |
vc-directory |
Show all registered files beneath a directory. |
|
C-x v = |
vc-diff |
Generate a version difference report. |
|
C-x v u |
vc-revert-buffer |
Throw away changes since the last checked-in revision. |
|
C-x v ~ |
vc-version-other-window |
Retrieve a given revision in another window. |
|
C-x v l |
vc-print-log |
Display a file's change comments and history. |
|
C-x v i |
vc-register |
Register a file for version control. |
|
C-x v h |
vc-insert-headers |
Insert version control headers in a file. |
|
C-x v r |
vc-retrieve-snapshot |
Check out a named project snapshot. |
|
C-x v s |
vc-create-snapshot |
Create a named project snapshot. |
|
C-x v c |
vc-cancel-version |
Throw away a saved revision. |
|
C-x v a |
vc-update-change-log |
Update a GNU-style ChangeLog file. |
Table E-21. Ediff commands (Chapter 12)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
Space or n |
ediff-next-difference |
Move to the next difference between the files. |
|
Del or p |
ediff-previous-difference |
Move to the preceding difference between the files. |
|
j |
ediff-jump-to-difference |
Go to the difference specified as a numeric prefix argument. |
|
a |
ediff-copy-A-to-B |
Copy the version of the current difference found in buffer A to buffer B. |
|
b |
ediff-copy-B-to-A |
Copy the version of the current difference found in buffer B to buffer A. |
|
r a or r b |
ediff-restore-diff |
Restore the current difference in buffer A (or B) to the way it was before copying from the other buffer. |
|
A or B |
ediff-toggle-read-only |
Switch the specified buffer into (or out of) read-only mode. |
|
g a or g b |
ediff-jump-to-difference-at-point |
Recenter the comparison buffers on the difference nearest to your current location (point) in the specified buffer. |
|
! |
ediff-update-diffs |
Recalculate and redisplay the highlighted regions; useful if you've manually made extensive changes to a buffer. |
|
w a or w b |
ediff-save-buffer |
Save the specified buffer to disk. |
|
z |
ediff-suspend |
Close the Ediff control window, but leave the session active so you can resume it later. |
|
q |
ediff-quit |
Close the Ediff window and end this comparison session. |
Table E-22. CUA mode commands (Chapter 13)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-c C-x C-x |
cua-exchange-point-and-mark |
Exchange location of cursor and mark. |
|
C-x or C-w or S-Delete |
kill-region |
Delete the region. |
|
C-v or C-y or S-Insert |
cua-paste |
Paste most recently killed or copied text. |
|
C-c |
copy-region-as-kill |
Copy the region. |
|
M-v |
cua-repeat-replace-region |
After highlighting and replacing a string, find the next string and replace it the same way. |
|
M-y |
cua-paste-pop |
After C-v, pastes earlier deletion. |
|
C-z or C-x u |
cua-undo |
Undoes the last change. |
|
C-x C-z |
iconify-frame |
Minimize the current frame (what C-z does outside CUA mode). |
Table E-23. Help commands (Chapter 14)
|
Keystrokes |
Command name |
Question answered |
|---|---|---|
|
C-h k Help → Describe → Describe Key |
describe-key |
What command does this keystroke sequence run, and what does it do? |
|
C-h f Help → Describe → Describe Function |
describe-function |
What does this function do? |
|
C-h v Help → Describe → Describe Variable |
describe-variable |
What does this variable mean, and what is its value? |
|
C-h m Help → Describe → Describe Buffer Modes |
describe-mode |
Tell me about the modes the current buffer is in. |
|
C-h b Help → Describe → List Key Bindings |
describe-bindings |
What are all the key bindings for this buffer? |
|
C-h a Help → Search Documentation → Find Commands by Name |
apropos-command |
What commands include this word? |
|
(none) Help → Search Documentation → Find Options by Name |
apropos-variable |
What variables include this regular expression? |
|
(none) Help → Search Documentation → Find Any Object by Name |
apropos |
What functions and variables involve this regular expression? |
Table E-24. Documentation help commands (Chapter 14)
|
Keystrokes |
Command name |
Action |
|---|---|---|
|
C-h t Help → Emacs Tutorial |
help-with-tutorial |
Run the Emacs tutorial. |
|
C-h i |
info |
Start the Info documentation reader. If prefaced with C-u, reads an Info file of your choice. |
|
C-h r Help → Read the Emacs Manual |
info-emacs-manual |
Open the Emacs manual. |
|
C-h K Help → More Manuals → Find Key in Manual |
Info-goto-emacs-key-command-node |
Start Info documentation reader at the node that discusses this key sequence. |
|
(none) Help → Search Documentation → Look Up Subject in User Manual |
emacs-index-search |
Search the index of the Emacs user manual. |
|
C-h p |
finder-by-keyword |
Invoke a menu that lets you get information about Emacs Lisp packages available on your system. |
Table E-25. Important modes
|
Mode |
Function |
|---|---|
|
fundamental-mode |
The default mode |
|
text-mode |
Major mode for writing text (Chapter 2) |
|
outline-mode |
Major mode for writing outlines (Chapter 7) |
|
picture-mode |
Major mode for creating ASCII drawings using the keyboard (Chapter 7) |
|
html-mode |
Major mode for writing HTML (Chapter 8) |
|
sgml-mode |
Major mode for writing SGML and XML (Chapter 8) |
|
latex-mode |
Major mode for formatting files for TEX and LATEX (Chapter 8) |
|
c-mode, c++-mode |
Major mode for writing C and C++, and Java programs (Chapter 9) |
|
java-mode |
Major mode for writing Java programs (Chapter 9) |
|
perl-mode, cperl-mode |
Major modes for writing Perl programs (Chapter 9) |
|
sql-mode |
Major mode for interacting with databases using SQL (Chapter 9) |
|
emacs-lisp-mode |
Major mode for writing Emacs Lisp functions (Chapter 9 and Chapter 11) |
|
lisp-mode |
Major mode for writing Lisp programs (Chapter 9 and Chapter 11) |
|
lisp-interaction-mode |
Major mode for writing and evaluating Lisp expressions ( Chapter 9 andChapter 11) |
|
auto-fill-mode |
Minor mode that enables word wrap (Chapter 2) |
|
overwrite-mode |
Minor mode that replaces characters as you type instead of inserting them (Chapter 2) |
|
flyspell-mode |
Minor mode for flyspell spell-checker (Chapter 3) |
|
flyspell-prog-mode |
Minor mode for spell-checking programs with flyspell (Chapter 3) |
|
abbrev-mode |
Minor mode for word abbreviations (Chapter 3) |
|
artist-mode |
Minor mode for creating ASCII drawings using the mouse (Chapter 7) |
|
font-lock-mode |
Minor mode for highlighting text in colors and fonts (Chapter 9) |
|
vc-mode |
Minor mode for using version control systems (Chapter 12) |


Car Wash
For Requester
For Provider
Create Car Wash Request
Title
Name
Email
Phone
Choose Location
Area
Full Address
Short Address
Geolocation
Remarks
Submit Request
Car Wash Provider
Register
Sign In
Car Wash Provider Sign In
Email
Password
Car Wash Provider Registration
Email
Password
Full Name
IC
Phone
Area of coverage
Duno what else
Logout
Dr
Khamis Bin Jumaat
01115134217
<EMAIL_ADDRESS>123, JLN Mati, Pulau Pinang
This 1 owez ffk
Apply


Brian W Portfolio
Education
Highschool
College
Projects
AboutMe
Salmon Cookies
Bus Mall
HomeFit
Work History
IT
About Me:
This will eventually be some random stuff about me.


Command-line Tricks
Mort Yao(Note: This page contains tips and tricks of most commonly used command-line utilities on Unix-like OSes. For usages related to the shell itself, see Shell tricks instead.)
Common usage
See how a Unicode string is encoded internally
$ echo hello | hexdump -C
See how it is encoded in UTF-16: (Assume UTF-8 is the default encoding)
$ echo hello | iconv -f utf-8 -t utf-16 | hexdump -C
Show Unix epoch time
$ date +%s
Show timestamp in RFC 3339 format
$ date --iso-8601=seconds
Show the MIME type of a file
$ file -0 --mime-type 'filename' | cut -d $'\0' -f2 | cut -d' ' -f2
List files recursively, etc.
$ find .
List files by extension:
$ find . -name "*.py"
To calculate the total number of lines of code, use:
$ find . -name "*.py" | xargs grep -v "^$" | wc -l
To delete all files with some extension, use:
$ find . -name "*.o" -delete
Get the size of a directory
In terms of kilobytes:
$ du -hs foo/bar | cut -f1
Or: (in human-readable format)
$ du -s foo/bar | cut -f1
List sub-directories in descending order of size
$ du -hs * | sort -hr
Archive (and compress) a directory
As ZIP format:
$ zip -r src.zip src
As gzip format:
$ tar -czf src.tar.gz src
To archive some files under a certain directory without including the pathname in the resulting archive, change the directory with -C
:
$ tar -czf files.tar.gz -C src files
Login to a remote host
e.g., login to <IP_ADDRESS>
as user root
, via port 22
:
$ ssh -p 22<EMAIL_ADDRESS>Copy files to a remote host
e.g., copy some files
to <IP_ADDRESS>
as user root
, via port 22
:
$ scp -P 22 files root@<IP_ADDRESS>:/home/root/bak
System maintenance
View ACPI information (battery status, etc.)
$ acpi -i
View SMART information
# smartctl -a /dev/sda
Wipe out a disk
Data removed by rm -rf
can be easily recovered by a software tool like TestDisk. To wipe out all data on a device (e.g., /dev/sda
) reliably, use:
# dd if=/dev/urandom of=/dev/sda bs=4k
(Warning: Do NOT try the above command until you need it!)
Networking
Show all active Internet connections
Show only TCP connections:
$ netstat -ntlp
To include UDP connections:
$ netstat -ntulp
Port scanning
Check for a port number (e.g., 8000
):
$ netcat -v -w 3 -z scanme.nmap.org 8000
Check for a range of ports (e.g., 20
- 80
):
$ netcat -v -w 3 -z scanme.nmap.org 20-80
Scan all open ports with Nmap:
$ nmap -v -A scanme.nmap.org
Query a specific DNS server (e.g., Google Public DNS)
$ dig @<IP_ADDRESS> www.example.com
Find out your IP address on the public Internet
$ dig +short myip.opendns.com @resolver1.opendns.com
Look for the geolocation of a host
$ geoiplookup google.com
Or: (IPv6)
$ geoiplookup6 google.com
Start an HTTP server for development
$ python3 -m http.server 4000
Web scraping
Check unread Gmail
Replace “abcdefghijklmnop
” with your 16-digit app password (not your login password!):
$ curl -u foo.bar@gmail.com:abcdefghijklmnop \
-s "https://mail.google.com/mail/feed/atom" | tr -d '\n' | \
awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | \
sed -n "s/<title>\(.*\)<\/title.*name>\(.*\)<\/name>.*/\2 - \1/p"
Source: http://www.commandlinefu.com/commands/view/3386/check-your-unread-gmail-from-the-command-line
Check an Atom feed for the newest entry
$ curl -s https://github.com/blog.atom | sed -e "s/xmlns/ignore/" - | \
xmllint --xpath "/feed/entry[1]/title/text()" -
Harvest email addresses from a web page
$ curl -s 'https://pgp.mit.edu/pks/lookup?search=0x07DA00CB78203251' | \
grep -o '[[:alnum:].]*@[[:alnum:].]*'


Below are tables that show the sequence of calls between Java and C when the PtinyOSDirector is used in simulation mode in Viptos. The PtinyOSDirector uses a Java PtinyOSLoader class for each instance of TOSSIM to keep the namespace separate for each TinyOS node in the model. unique_name is used in native method names to maintain separate namespaces. All calls that cross the Java / C (and C / Java) boundary are JNI calls.
A JNI call from TOSSIM to the Java PtinyOSLoader class (_loader) refers to a call from the C code in TOSSIM to the Java PtinyOSLoader class associated with that instance of TOSSIM. The same is true for a JNI call from the Java PtinyOSLoader class to TOSSIM.
A Java call from PtinyOSDirector to the Java PtinyOSLoader class refers to a call from the Java code in PtinyOSDirector to the Java PtinyOSLoader class associated with that instance of PtinyOSDirector. The same is true for a Java call from the Java PtinyOSLoader class to the PtinyOSDirector.
See:
$PTII/ptolemy/domains/ptinyos/kernel/PtinyOSDirector.java (Start here) $PTII/ptolemy/domains/ptinyos/kernel/PtinyOSLoader.javaand
$TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/adc_model.c $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/event_queue.c $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/external_comm.c $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/external_comm.h $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/Nido.nc $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/nido.h $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/ptII.c (Start here) $TOSROOT/contrib/ptII/ptinyos/tos/platform/ptII/rfm_model.c $TOSROOT/contrib/ptII/ptinyos/beta/TOSSIM-packet/Nido.nc $TOSROOT/contrib/ptII/ptinyos/beta/TOSSIM-packet/nido.h $TOSROOT/contrib/ptII/ptinyos/beta/TOSSIM-packet/packet_sim.c $TOSROOT/contrib/ptII/ptinyos/tos/types/dbg.h
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
PtinyOSDirector.preinitialize()
| - | - |
PtinyOSDirector.initialize()
| - | - |
| - | _loader.load() loads the TOSSIM shared object into memory, using Java System.load() | - |
PtinyOSDirector.initialize(), continued
| - | - |
| - | _loader.main() calls JNI main_unique_name() native method. | - |
| - | - | JNI main_unique_name() calls TOSSIM main(), which contains an initialization routine:
|
| - | _loader.startThreads() creates Java threads and calls Java Thread.start() on each thread. | - |
| - | - | TOSSIM main() initialization routine, continued
|
| - | _loader.enqueueEvent() calls Java PtinyOSDirector.enqueueEvent() | - |
PtinyOSDirector.enqueueEvent()
| - | - |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
PtinyOSDirector.fire()
| - | - |
| - | _loader.processEvent() calls JNI processEvent_unique_name() native method. | - |
| - | - | JNI processEvent_unique_name() calls TOSSIM ptII_fire(), which invokes a single iteration of the TOSSIM scheduler. The TOSSIM scheduler processes a single TOSSIM event and runs tasks in the task queue. Note that processing a TOSSIM event may cause events with the same time stamp to be created, and for additional TinyOS tasks to be created. All events created in TOSSIM use the same event call sequence described above for the bootup event (see ptII_queue_insert_event()). |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
| - | - | TOSSIM requests sensor value in TOSSIM generic_adc_read() by calling ptII_get_adc_value(), which calls JNI _loader.getCharParameterValue(). |
| - | _loader.getCharParameterValue() calls Java PtinyOSDirector.getCharParameterValue(). | - |
| PtinyOSDirector.getCharParameterValue() gets a double token from the requested PortParameter and converts it to a Java char and returns the value. | - | - |
| - | _loader.getCharParameterValue() returns the Java char value. | - |
| - | - | JNI _loader.getCharParameterValue() returns a JNI jchar value. TOSSIM ptII_get_adc_value() converts the JNI jchar into a C unsigned short and returns the value to TOSSIM generic_adc_read() for use as the sensor (ADC) value. |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
| - | - | TOSSIM generates an LED or packet value and calls TOSSIM ptII_updateLeds() or TOSSIM ptII_send_packet(), both of which call JNI _loader.sendToPort() with the port name and value as string arguments. |
| - | _loader.sendToPort() calls Java PtinyOSDirector.sendToPort(). | - |
| PtinyOSDirector.sendToPort() performs type conversion from a Java string to the appropriate Token type (depending on the type of the requested port), and sends the token to the requested port using the Java Ptolemy II Port.send() method. | - | - |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
PtinyOSDirector.receivePacket() called (e.g., by the fire() method of the container, which should be a PtinyOSCompositeActor, in response to a packet arriving at the packetIn port).
| - | - |
| - | _loader.receivePacket() calls JNI receivePacket_unique_name() native method with the packet as a Java String argument. | - |
| - | - | JNI receivePacket_unique_name() receives the packet as a JNI jstring argument and converts it into a C const char array. It then calls TOSSIM ptII_receive_packet(), which copies the C const char array into a TOS_Msg data structure. TOSSIM ptII_receive_packet() then calls TOSSIM ptII_insert_packet_event(), which creates a TOSSIM packet event. |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
| - | - | TOSSIM generates a debug message in dbg() or dbg_clear(), both of which call TOSSIM ptII_dbg(), which calls JNI _loader.tosDebug(). |
| - | _loader.tosDebug() calls Java PtinyOSDirector.tosDebug(). | - |
| PtinyOSDirector.tosDebug() prints the debug message using the Java Ptolemy II _debug() method. | - | - |
| Java (PtinyOSDirector) | Java (PtinyOSLoader) | C (TOSSIM) |
|---|---|---|
PtinyOSDirector.wrapup()
| - | - |
| - | _loader.wrapup() calls JNI wrapup_unique_name() native method. | - |
| - | - | JNI wrapup_unique_name() calls TOSSIM shutdownSockets(), which:
|
| - | _loader.joinThreads() calls Java Thread.join() on all threads created during initialization. Returns Java int value of 0 upon success. | - |
| - | - | JNI _loader.joinThreads() returns jint value of 0 upon success, which is converted into a C int and returned by ptII_joinThreads(). If successful, TOSSIM execution stops. |
TOSSIM also contains JNI wrapper calls to Java socket routines in the Java PtinyOSLoader and in the Java PtinyOSDirector. We omit detailed discussion of these call sequence for these socket routines above, since they are not necessary for understanding the basics of the PtinyOS domain. The wrappers for the socket routines are similar for the wrappers in the Java PtinyOSLoader as discussed above.
Detailed list of socket routines:


Why Skewness and Kurtosis Don't Matter
Contents
The Sharpe Ratio is one of the most common performance metrics for both individual stocks and portfolios. However, a more natural choice for a portfolio performance metric is the ratio of the expected return to some Value-at-Risk of the return distribution, which can be characterized as a Generalized Sharpe Ratio i.e., $$ GSR \equiv \lim_{\alpha \to 0} \frac{\mu}{V_\alpha(\mu=0,\sigma,s,k,…)} $$ where \(V_{\alpha}()\) is the Value-at-Risk, \(\alpha\) is the probability threshold and \(\mu\), \(\sigma\), \(s\), etc. are the 1st (mean), 2nd (standard deviation), 3rd (skewness), etc. moments of of the return distribution. \(\mu=0\) indicates that Value-at-Risk calculations assume 0 expected return.
For a standard normal distribution $$V_{0.1587}(\mu=0,\sigma,s,k,…) = V_{0.1587}(\mu=0,\sigma) = \sigma$$ $$V_{0.0250}(\mu=0,\sigma,s,k,…) = V_{0.0250}(\mu=0,\sigma) = 1.96\sigma$$ $$V_{0.0100}(\mu=0,\sigma,s,k,…) = V_{0.0100}(\mu=0,\sigma) = 2.33\sigma$$
Question: If the GSR is our portfolio’s performance metric, what is the decision criteria for including an individual asset in that portfolio?
Now if the portfolio’s performance metric is the standard Sharpe Ratio, then the decision criteria (called a Marginal Sharpe Ratio) is i.e., $$MSR_a = SR_a - \rho SR_m$$
If the MSR is greater than 0, then an asset’s portfolio weight is greater than 0. Is it possible to derive a formula that characterizes a Marginal Generalized Sharpe Ratio (MGSR)?
This is a complicated question mathematically, but it is possible to show that $$GMSR_a = SR_a - (\rho-\gamma) SR_m$$ where \(\gamma\) is related to the second derivative of the asset’s return with respect to the portfolio’s return i.e., $$ \gamma \propto \frac{\partial^2 f(r_m)}{\partial r_m^2} $$ where $$r_a = f(r_m) + \epsilon$$ for some non-linear function f() and error term \(\epsilon\).
(To the best of my knowledge, the proof of this isn’t published. If you’d like to see a proof of this, send me an email.)
This is a pretty surprising result for a few reasons:
- The ordinary Sharpe Ratio reappears even though it isn’t in our definition of the MGSR
- Higher order moments of the asset’s distribution and the portfolio’s distribution appear nowhere in this equation
The latter point implies that an individual asset’s skewness in itself doesn’t impact its portfolio weight. The 2nd order factor that is relevant is \(\gamma\).
On further consideration, this is actually just a consequence of the Central Limit Theorem. Imagine a portfolio with an infinite number of uncorrelated assets with arbitrary return distributions. By the Central Limit Theorem, the portfolio’s returns will be normallly distributed!
\(\gamma\) is also related to \(\Gamma\) in traditional optional pricing theory. The “fair” price for an asset corresponds to the discount rate where the MGSR is 0 i.e., $$GMSR_a = SR_a - (\rho-\gamma) SR_m \equiv 0$$ $$SR_a = (\rho-\gamma) SR_m$$ $$\mu_a = (\rho-\gamma) SR_m \sigma_a$$ $$\mu_a = (\rho-\gamma) \mu_m \frac{\sigma_a}{\sigma_m}$$ $$\mu_a = \left(\beta -\gamma\frac{\sigma_a}{\sigma_m}\right)\mu_m$$ which is the classical CAPM formula with an addtional “convexity” term.
While estimating \(\gamma\) isn’t trivial, the qualitative implication is profound: when pricing assets, convexity matters, not skewness.
Author David Rashty
LastMod 2020-09-04


Larencia Bright understands that Home Buying is a very serious investment, let me assist you in the process by giving you best in class service!
Find Out MoreI believe that the customer’s needs and concerns are my number one priority. My objective is to be available and readily accessible to my clients and to always work with their best interest in mind. I have chosen to align my business with one of the Southeast’s leading real estate brokerages, PalmerHouse Properties.
As an agent representing one of the most dynamic real estate brands in the country, you can expect the utmost professionalism and dedication to all of your real estate needs.
From our very first meeting to the day of closing, I’m here to provide you with best-in-class service to help you buy or sell a home. Equipped with the finest tools and technology, I’ll guide you through every step of the real estate transaction.
Please don’t hesitate to reach out to me to learn more or to schedule a meeting or consultation Larencia Bright Realty in Atlanta!
Get Started!


Droplet Autofill
You already have JSON data associated with your extension. If you need to update the file, the paste the contents in the textfield below and click on update button.


Places You've Been
Please input a place
Destination
Landmark
Dates
Notes
Add a trip
Trips you've been on
Trip Dates:
Landmarks:
Notes:


What is Exploratory Data Analysis(EDA)?
- It’s often the first step in data analysis, implemented before any other techniques. It is an approach for summarizing, visualizing, and becoming intimately familiar with the important characteristics of a data set.
- It consists techniques that are used to gain some insights on the dataset before doing any modeling.
- EDA, is essentially a type of storytelling for statisticians. It allows us to uncover patterns and insights, often with visual methods, within data.
- EDA is not a set of rules to follow, it is our state of mind.
Purpose of EDA:
- It is basically used to filter data from redundancies.
- It helps us to identify the faulty points in data and to understand the relationships between variables.
- Gain maximum insight into the data set and its underlying structure.
- Identify the most influential variables
- EDA is used to gain some statistics and perform visualizations for better understanding of data, and finds some information about the tendencies of the data, its quality and have some assumptions for the hypothesis test.
- EDA is NOT about making fancy visualizations or even aesthetically pleasing ones; the goal is to try and answer questions with data.
- We should be able to create a figure which should be undersandable to others in a couple of seconds
- EDA plays an important role in data analysis. We should always check the quality of data even if it provided in platter.
- Data Cleaning is an application of EDA where we can clarify our doubt if our data reaches the expectations or not.
Steps involved in EDA:
Understand the data
Clean the data
Analysis of relationship between variables
Methods for exploratory data analysis:
It is good to explore the data through various EDA techniques and compare them. Once the data set is fully understood, it is quite possible that data scientist will have to go back to data and cleansing phases in order to transform the data set according to the desired outcomes.
Methods for EDA are:
- Univariate visualization :- provides summary statistics for each field in the raw data set
- Bivariate visualization :- is performed to find the relationship between each variable in the dataset and the target variable of interest
- Multivariate visualization :- is performed to understand interactions between different fields in the dataset
- Dimensionality reduction :- helps to understand the fields in the data that account for the most variance between observations and allow for the processing of a reduced volume of data.
Through these methods, the data scientist validates assumptions and identifies patterns that will allow for the understanding of the problem and model selection and validates that the data has been generated in the way it was expected to.
Is skipping Exploratory Data Analysis a good idea?
NO,In a hurry to get to the machine learning stage or simply impress business stakeholders very fast, data scientists tend to either entirely skip the exploratory process or do a very shallow work. It is a very serious and, sadly, common mistake of amateur data science consulting “professionals”.
Such behaviour can lead to skewed data, with outliers and too many missing values and, therefore, some sad outcomes for the project:
- generating inaccurate models;
- generating accurate models on the wrong data;
- choosing the wrong variables for the model;
- inefficient use of the resources, including the rebuilding of the model.
There are a lot of ways to reach these goals as follows:
- Import the data
- Get a feel of the data ,describe the data,look at a sample of data like first and last rows
- Take a deeper look into the data by querying or indexing the data
- Identify features of interest
- Recognise the challenges posed by data - missing values, outliers
- Discover patterns in the data
One of the important things about EDA is Data profiling.
Data profiling is concerned with summarizing your dataset through descriptive statistics. The goal of data profiling is to have a understanding of the data so that you can start querying and visualizing the data in various ways. Depending on the result of the data profiling, you might decide to correct, discard or handle your data differently.
Visualizing data
- Data visualization is the graphical representation of data. It contains statistical graphics, plots, information graphics and many more to explore the data easily.
- Your goal should be to be able to create a figure which someone can look at in a couple of seconds and understand what is going on.
- If not, the visualization is too complicated (or fancy) and something similar should be used.
- EDA is also very iterative since we first make assumptions based on our first exploratory visualizations, then build some models.
- We then make visualizations of the model results and tune our models.
Summary Statistics:
• Summary statistics are measurements meant to describe data. Examples of summary/ descriptive statistics for a single numerical variable is the mean, median, mode, max, min, range, quartiles/percentiles, variance, standard deviation, coefficient of determination, skewness and kurtosis. • Summary statistics for categorical variables is the number of distinct counts.The most basic summary statistic for text data is term frequency and inverse document frequency.For bivariate data, the summary statistics is the linear correlation, chi-squared, or the p value based z-test, t-test or analysis of variance.
- In addition to summary statistics, visualizations can be used to explore and describe data.
- Examples of visualizations for numeric data are line charts with error bars, histograms, box and whisker plots, for categorical data bar charts and waffle charts, and for bivariate data are scatter charts or combination charts.
Some examples of plots used for EDA are:
- Histograms
- Scatter plots
- Pair plots
- Box plots
- Violin plots
- Distribution Plots
Histograms:
- It is used in univariate analysis, they give a rough sense of density of underlying distribution of data more precisely probability distribution of data.
- The plotting of the histogram depends upon ‘bins’ i.e dividing the entire range into series of interval then based upon the number of values present inside a range of a bin the height of the bar of that bin is determined.
Scatter plot:
- A scatter plot is a type of plot which uses X and Y coordinates on a 2d plane to display points.
- It is a well known plotting technique to study the interdependence of one variable over other.
PairPlot:
- This is a extension to scatter plots and histogram and its PDF representation.
- This is primarily used when we want to study the behavior of all variables with every other variables when the data is more than 2-Dimension.
Boxplot:
- These are fairly complex types of plots ,these are used to represent data according to their quartiles.
- They also have lines extending vertically from the boxes indicating the variance outside the upper and lower quartile .
- The Space between different parts of the box indicate the variance(Spread) of the data.
- It also helps in detection of outliers
Voilon Plot:
- It is a extension of box plots in this the kernel density plot is also plotted with box plots .
- A violin plot is better than a plain box plot as a box plot only shows summary statistics such as mean/median and interquartile ranges, the violin plot shows the full distribution of the data.
Distribution plots:
- These plots are used to know distribution of attributes whether they are normally distributed or skewed distributed.
- You can select multiple feature distributions and compare them side by side that is the beauty of plotly very interactive.
You can learn more about this topic fromhere.


← Back
Home →
🆎 CRO Tools →
🏎 F1 Calendar, Teams & Drivers →
🎒 Travel Gear →
📘 Meditations →
📕 Discourses →


List of devices the server recived from domoticz. Room and Nicknames is added from configuration. NOTE: If you don't see any device check your connection to domoticz.


{% extends "base.html" %}
{% comment %}
Copyright (c) 2011 Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
 IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
{% endcomment %}
{% block title %}Attribute Mapper{% endblock title %}
{% block content %}
| Map Attributes for {{ meta.category }} |
View Table
Accept All
Reset
Save
|
{% if attr_list %}
{% for attr in attr_list %}
| {{ attr.name }} |
|
{% if attr.match.is_mapping %}
{{ attr.match.name }}
{% else %}
{{ attr.match.name }}
{% endif %}
|
V |
|
|
Yes
No
Reset
Details...
|
{% if not attr.match.is_mapping %}
unmapped |
{% else %}
mapped |
{% endif %}
{% endfor %}
{% else %}
Nothing to do!
{% endif %}
{% endblock content %}
{% block jquery %}
{% endblock jquery %}


Most ways that isobaric labeling data are analyzed hinder statistical testing and data visualization. This notebook will analyze data from (Ref-1) to demonstrate a two-condition (3 controls and 4 treatments) comparison done with 10-plex TMT reagents using the SPS MS3 method (Ref-2) on a Thermo Fusion instrument. The notebook will show how to:
The KUR1502 data were mouse bone marrow cell cultures where the "media" samples are controls (one of the 4 did not label) and there are leukemia exosome-dosed cells (4 "exosome" samples). 10-plex TMT was used on a Thermo Fusion using the SPS MS3 method. The PAW pipeline (Ref-3) is our in-house processing using the Comet search engine (Ref-4). The pipeline converts the RAW files into text files using MSConvert from the Proteowizard toolkit (Ref-5).
Data from the MS2 scans are extracted for the Comet searches and the reporter peak heights are extracted from the MS3 scans. The pipeline uses the target/decoy method to make score histograms and determine score filtering thresholds. Accurate mass is used to create conditional score histograms where target/decoy delta mass histograms are used to set the accurate mass windows. Basic parsimony principles are used for protein inference and 2 peptides per protein were required. An additional protein grouping step was used to combine nearly identical peptide sets (often these are housekeeping genes).
1. Huan, J., Hornick, N.I., Goloviznina, N.A., Kamimae-Lanning, A.N., David, L.L., Wilmarth, P.A., Mori, T., Chevillet, J.R., Narla, A., Roberts Jr, C.T. and Loriaux, M.M., 2015. Coordinate regulation of residual bone marrow function by paracrine trafficking of AML exosomes. Leukemia, 29(12), p.2285.
2. McAlister, G.C., Nusinow, D.P., Jedrychowski, M.P., Wühr, M., Huttlin, E.L., Erickson, B.K., Rad, R., Haas, W. and Gygi, S.P., 2014. MultiNotch MS3 enables accurate, sensitive, and multiplexed detection of differential expression across cancer cell line proteomes. Analytical chemistry, 86(14), pp.7150-7158.
3. Wilmarth, P.A., Riviere, M.A. and David, L.L., 2009. Techniques for accurate protein identification in shotgun proteomic studies of human, mouse, bovine, and chicken lenses. Journal of ocular biology, diseases, and informatics, 2(4), pp.223-234.
4. Eng, J.K., Jahan, T.A. and Hoopmann, M.R., 2013. Comet: an open‐source MS/MS sequence database search tool. Proteomics, 13(1), pp.22-24.
5. Chambers, M.C., Maclean, B., Burke, R., Amodei, D., Ruderman, D.L., Neumann, S., Gatto, L., Fischer, B., Pratt, B., Egertson, J. and Hoff, K., 2012. A cross-platform toolkit for mass spectrometry and proteomics. Nature biotechnology, 30(10), p.918
# load libraries
library("tidyverse")
library("psych")
library("gridExtra")
library("scales")
library("limma")
library("edgeR")
# read the grouped protein summary file
paw_raw <- read_tsv("grouped_protein_summary_TMT_8_sorted.txt", skip = 4,
n_max = 5427, guess_max = 5427)
# extract protein accession column and TMT data
# need to exclude any rows with an entry in "Filter" column
# there are also 4 unused channels (of 11-plex slots)
paw_tmt <- paw_raw %>%
filter(., is.na(Filter)) %>%
select(Accession, starts_with("TotInt_")) %>%
select(-contains("127N"), -contains("128N"), -contains("130N"), -contains("131C"))
# separate accessions from the data
accession <- paw_tmt$Accession
paw_tmt <- paw_tmt %>% select(-Accession)
# rename the columns and gather the conditions
colnames(paw_tmt) <- c("Media_2.1", "Exo_2.1", "Exo_2.2", "Media_2.2",
"Exo_3.1", "Exo_3.2", "Media_3.2")
paw_tmt <- paw_tmt %>% select(contains("Media"), contains("Exo"))
head(paw_tmt)
nrow(paw_tmt)
Zero replacement is always a controversial topic. How it applies to TMT data is also not simple. How I approached this problem in the PAW pipeline is described here. There is another complication at the moment. With high resolution instruments, reporter ions can be extracted for all 11 m/z windows whether or not an 11-plex kit was used. This can be handy to see what the true background levels are like in known unused channels. The script that adds reporter ion intensities to the peptide and protein reports, does a trimmed average test to zero out reporter ions near the lower detection limits. This dataset had a lot of PSMs rejected despite a low minimum trimmed intensity of cutoff of 500.
| Total PSMs | Rejected PSMs | % Rejected |
|---|---|---|
| 86,781 | 28,052 | 32% |
The first-generation Fusion instruments are not as sensitive as the Lumos models. The extra round of isolation in the SPS MS3 method reduces signal levels. Sample loads and separations need to be appropriately scaled up for Fusion instruments.
With so many rejected PSMs, there might be significant numbers of proteins that had zero reporter ion sums. Those were replaced by a value of 150 in the add_TMT_intensites.py
script. Checking for zeros after aggregating to the final protein values, where many PSMs might be combined, helps reduce missing data, but we should never take things for granted. After we have dropped the 4 unused channels (of the 11 TMT slots), we can see how many proteins had different numbers of zero replacements.
counts <- 1:nrow(paw_tmt) # create vector of appropriate length
for(i in 1:nrow(paw_tmt)){
# TRUE will be coerced to 1 for the summing
counts[i] <- sum(paw_tmt[i, ] == 150)
}
table(counts) # create the summary
The vast majority of the protein have no missing values or just one. There are only 17 proteins that more than one missing value. We can keep an eye out for those in the final results sheet after the analysis. We really only need to worry about them if they get flagged as significant differential expression (DE) candidates.
We will use box plots, a common way to summarize distributions, to visualize the Log base 10 of the intensities. The 3 media samples are in red color and the 4 exosome-dosed samples are in blue color.
# let's see what the starting data look like
color = c(rep("red", 3), rep("blue", 4))
boxplot(log10(paw_tmt), col = color, notch = TRUE, main = "Starting PAW data")
The boxplots are not very well horizontally aligned. A global scaling seems in order. Each channel is supposed to be the same total amount of protein digest that was labeled. We would, therefore, expect the total sum of reporter ion intensities to be the same in each channel.
We can make a function that takes the column sums, computes the average sum, and scales each column so that its new sum will be the average.
SL_Norm <- function(df, color = NULL, plot = TRUE) {
# This makes each channel sum to the average grand total
# df - data frame of TMT intensities
# returns a new data frame with normalized values
# compute scaling factors to make colsums match the average sum
norm_facs <- mean(c(colSums(df))) / colSums(df)
cat("SL Factors:\n", sprintf("%-5s -> %f\n", colnames(df), norm_facs))
df_sl <- sweep(df, 2, norm_facs, FUN = "*")
# visualize results and return data frame
if(plot == TRUE) {
boxplot(log10(df_sl), col = color, notch = TRUE, main = "SL Normalized data")
}
df_sl
}
# SL norm the tmt data
paw_tmt_sl <- SL_Norm(paw_tmt, color)
The normalization brought the boxplots into horizontal alignment. We can double check the computations by seeing if the column sums are equal after normalization.
# compute column sums before and after normalization
print("Before:")
format(round(colSums(paw_tmt), digits = 0), big.mark = ",")
print("After:")
format(round(colSums(paw_tmt_sl), digits = 0), big.mark = ",")
The basic starting steps in any analysis are to make sure the data was read in correctly. Proteomics summary files usually have lots of information in addition to the quantitative data. Some work to extract the correct quantitative columns will be needed (this is often called the data wrangling part of an analysis). This can be done in Excel before reading into R (this could be a possible source of errors) or can be done in R (a different set of possible errors). Filtering of rows that should be excluded from the statistical testing is also necessary. Those include common contaminants, any decoy proteins, and proteins that might have too much missing data to be reliable.
The extracted data may need some further formatting depending on the downstream statistical testing requirements. Here, we are saving the protein accession information as a vector and the actual TMT reporter ions as a 2D table of numbers.
Some exploratory statistics on the starting data are always a good idea. R has many functions to do far more than we have done here. We did some very basic checks to see what the data looked like before and after one simple normalization method. The basic features of the data look okay, so we will move on to the statistical testing.
In the dark ages before proteomics, mass spectrometers were made of stone (sector magnets and chemical ionization) and the t-test was developed to test beer (an essential ingredient of proteomics from the very beginning). Then came the not-so-golden ratio age (Penning ion traps, Q-TOFs, ICAT, and SILAC) where quantitative proteomics became synonymous with ratios and fold-changes. Battle lines were drawn between complicated standard formats with lengthy command line processing pipelines and the enduring allure of mysterious "black box" commercial products.
Ironically, progress on the instrument side seems to have accelerated while advances on the processing side seems to be in an ice age glacial state. We have seen an increase in the number of standard formats and their complexity. They seemed to have found the sweet spot of being quite hard for both humans and computers to read. We now have shiny "open source" black boxes to compete with more faded commercial black boxes. Statistical testing has evolved from one ratio (and no need for statistics) to boatloads of ratios as multiplexing capacity has increased. This leads to a literal "log2" jam that might involve beer consumption because it always ends up with a t-test.
Can we actually do quantitative proteomics without going down this ancient, overgrown path to failure? The answer is yes. Genomics had a head start on proteomics and has been doing the larger sample number, complicated experimental designs that biological studies require for many years. Modern instruments and methods are producing quantitative proteomics datasets that are much more similar to genomics data than they were in the past. If we do a little research (are we not scientists?) into genomic tools and data formats, we can learn how to prepare proteomics data to leverage these more mature statistical packages and stop trying to invent square wheels and making sacrifices to the Volcano plot Gods.
Joking aside, proteomics needs to recognize the similarity between proteomics and genomics data and abandon outdated thinking that was never all that useful. Computers and data science methods have changed just as much as new instruments like the timsTOF have changed compared to a sector magnet mass specs.
This is not the place for a review of hypothesis testing. It is, however, a good place to get the gist of statistical testing in this context and define some concepts for the rest of the notebook. There are many types of proteomics experiments, and they do not fit into just one pigeon hole. A common scenario is testing if a protein is expressed at the same or different levels in a background of many other proteins.
Complex protein mixtures from two or more biological conditions (such as control and treatment, or normal and disease) are compared by making measurements of relative abundances. There are some hidden constraints in these experimental designs. The total amount of protein is usually fixed at the same value across all samples (an individual biological replicate) and there is a dynamic range within the proteomes. High abundance proteins are constrained by the total protein amount and do not have the same freedom for abundance differences that proteins that are smaller fractions of the total have. Higher abundance proteins must, by experimental design, have moderated differences in means and reduced variances. Lower abundance proteins have more freedom and will not really appear to be constrained.
The constrained nature of these experimental designs (both genomics and proteomics) also has implications in normalization algorithms. An increased expression in abundant proteins will "push" all other proteins down (their abundance total has to remain the same). Grand total normalization like we use above can be too simplistic. The Bioconductor (Ref-6) differential expression testing package edgeR (Ref-7) includes a trimmed mean of M values (TMM) normalization step (Ref-8) designed for these types of samples.
6. Gentleman, R.C., Carey, V.J., Bates, D.M., Bolstad, B., Dettling, M., Dudoit, S., Ellis, B., Gautier, L., Ge, Y., Gentry, J. and Hornik, K., 2004. Bioconductor: open software development for computational biology and bioinformatics. Genome biology, 5(10), p.R80.
7. Robinson, M.D., McCarthy, D.J. and Smyth, G.K., 2010. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics, 26(1), pp.139-140.
8. Robinson, M.D. and Oshlack, A., 2010. A scaling normalization method for differential expression analysis of RNA-seq data. Genome biology, 11(3), p.R25.
Bioconductor uses its own data classes. The main data table is organized by each gene or protein in a separate row, and each biological sample in a separate column. Genes/proteins can have rich annotations and the samples can have metadata. Think of this like a table of information about the rows and a table of information about the columns. These are linked tables. If you change the data rows, you have to update the row table, etc.
EdgeR has extensive documentation and you can read more about its data classes in the user's guide. We need to get the reporter ion intensities into a DGEList object. In addition to the data table, we can add the protein accessions as row labels, and the group membership (plain media or exosome dosed) information. We will use variable names similar to what is in the user's guide to make learning more about edgeR easier.
# load data into DGEList object
group <- c(rep("media", 3), rep("exosome", 4))
y <- DGEList(counts = paw_tmt, group = group, genes = accession)
y$samples
EdgeR normalization is actually done in two steps. The first, called a library size adjustment, is like the SL normalization we did above. This gets rid of the big differences between samples so that the TMM algorithm has better starting data.
# run the TMM normalization
y <- calcNormFactors(y)
# check what happened to the normalization factors
y$samples
EdgeR does not change the counts table. The "samples" list slot does get new norm.factor values. EdgeR uses the normalization factors as offsets in its modeling. We will want to check the TMM step and visualize the data results, so we need to get a table of normalized data values. A more detailed discussion of edgeR normalization can be found here. We will use a function that returns a data frame of normalized data from a DGEList object.
apply_tmm_factors <- function(y, color = NULL, plot = TRUE) {
# computes the tmm normalized data from the DGEList object
# y - DGEList object
# returns a dataframe with normalized intensities
# compute grand total (library size) scalings
lib_facs <- mean(y$samples$lib.size) / y$samples$lib.size
# the TMM factors are library adjustment factors (so divide by them)
norm_facs <- lib_facs / y$samples$norm.factors
cat("Overall Factors (lib.size+TMM):\n", sprintf("%-5s -> %f\n",
colnames(y$counts), norm_facs))
# compute the normalized data as a new data frame
tmt_tmm <- as.data.frame(sweep(y$counts, 2, norm_facs, FUN = "*"))
colnames(tmt_tmm) <- str_c(colnames(y$counts), "_tmm")
# visualize results and return data frame
if(plot == TRUE) {
boxplot(log10(tmt_tmm), col = color, notch = TRUE, main = "TMM Normalized data")
}
tmt_tmm
}
# get the normalized data values
paw_tmt_tmm <- apply_tmm_factors(y, color)
# check table
head(paw_tmt_tmm)
# check column sums after TMM
print("After TMM:")
format(round(colSums(paw_tmt_tmm), digits = 0), big.mark = ",")
The point of TMM is that changes in the abundance of more abundant proteins distort the grand totals. That is the trimmed part of the algorithm. With freedom to change the abundances of the more abundant proteins, the column totals will no longer stay constrained. Even though the column totals are not the same, the boxplots are nicely aligned.
If we have large adjustment factors, it means we have some compositional differences between samples. That has to be reconciled with what is known about the system under study. Generally, some compositional differences between conditions could make sense. Compositional differences between samples within the same condition could mean contaminating tissue in a dissection, for example.
It is critical to remember that most normalization methods assume large numbers of proteins with unchanged expression are present in the samples. These methods are for samples in conditions that are mostly similar in their proteomes. When that is not the case, much more care has to go into picking a normalization method.
Now that we have the table of TMM normalized data, we can do a few more QC type visualizations. We should have smaller CV values when normalization has worked well. If we do not get relatively tight CV distributions with nice median CV values, there are two likely reasons. There is significant inherent biological variability between samples (humans), or the sample preparation is too variable (need to optimize more).
# define variables for the columns in each condition
M <- 1:3
E <- 4:7
CV <- function(df) {
# Computes CVs of data frame rows
# df - data frame,
# returns vector of CVs (%)
ave <- rowMeans(df) # compute averages
sd <- apply(df, 1, sd) # compute standard deviations
cv <- 100 * sd / ave # compute CVs in percent (last thing gets returned)
}
labeled_boxplot <- function(df, ylim, title) {
# Makes a box plot with the median value labeled
# df - data frame with data to compute CVs of
# ylim - upper limit for y-axis
# title - plot title
cv = CV(df)
boxplot(cv, ylim = c(0, ylim), notch = TRUE, main = title)
text(x = 0.65, y = boxplot.stats(cv)$stats[3],
labels = round(boxplot.stats(cv)$stats[3], 1))
}
# compare CV distributions
par(mfrow = c(2, 2))
labeled_boxplot(paw_tmt[M], ylim = 150, title = "Starting Media CVs")
labeled_boxplot(paw_tmt[E], ylim = 150, title = "Starting Exosome CVs")
labeled_boxplot(paw_tmt_tmm[M], ylim = 75, title = "Media CVs after TMM")
labeled_boxplot(paw_tmt_tmm[E], ylim = 75, title = "Exosome CVs after TMM")
par(mfrow = c(1, 1))
This is a two-condition comparison. One of the 4 media only samples did not label. The cultures were done in two batches ("2.x" and "3.x" in the labels). We expect to see the media only samples cluster together and be separated from a cluster of the exosome dosed samples.
# see how things cluster after we have normalized data
plotMDS(log2(paw_tmt_tmm), col = c(rep("red", 3), rep("blue", 4)), main = "PAW TMM")
Media only clearly separates from exosome dosed in the leading dimension. These MDS plots are like principal component analysis. The x-axis is the major dimension of differences between samples, the y-axis is the second most important dimension of difference. The samples are separating left-to-right by condition. We probably also have a culture prep batch effect between batch 2 and 3. We should have more samples in each condition before attempting things like batch corrections, so we will focus on the differences between media and exosome conditions.
Multi-panel scatter plot grids are another exploratory data analysis technique that is well supported in R. We can see how similar biological replicates are to each other within the median only condition or in the exosome dosed condition. We can also see if the samples between conditions seem different. We would expect to see larger differences (more scatter) between groups that within groups.
# scatter plots within groups and betwen groups
pairs.panels(log10(paw_tmt_tmm), lm = TRUE, main = "After TMM")
Correlation coefficients are larger within conditions (0.98-ish) than between conditions (0.94-ish), but the differences are not large. Correlation coefficients are not a great metric. Media only samples are a little more similar between batch 2 and 3 than are the exosome dosed samples. Samples within the same condition and within the same batch are pretty similar to each other.
The typical analysis goes something like this:
We have already done the first 3 steps.
# we need to get dispersion estimates
y <- estimateDisp(y)
plotBCV(y, main = "Dispersion trends")
There are commonly abundance-dependent dispersion effects in many measurement methods. Higher abundance signals have better signal-to-noise (or are the result of more averaging), so they can appear more precise than lower abundance signals. We see that the blue line increases as the protein intensity decreases. The statistical testing will use trended variances (estimated from multiple proteins) as a function of protein intensity, which spans several orders of magnitude.
If we had summarized the data using ratios, we would have collapsed the range of protein values into a concentrated blob centered at 1:1 ratios. We would lose all ability to have trended variance estimates. The trended variance is how the test statistics are moderated and is why limma and edgeR outperform t-tests in typical study designs where replicate numbers can be small. I do not know how many replicates in each condition are needed before the trended variance stops improving results (10, 20, 50?). I think most proteomics studies are still considerably short of replicate numbers where the t-test will work well. Avoiding ratios is very important in getting the best results.
We will use the exact test in edgeR for this simple two-state comparison. EdgeR has general linear model extensions that allow for more complicated experimental designs. They are a bit more work to set up and run. We will avoid that here, but the user's guide has many examples.
If you read up on edgeR or know a bit about statistics, you may wonder why we are using statistical testing built on a negative binomial distribution for counting experiments. The short answer is because edgeR is easier to setup and run, and the modeling seems okay given the data. The longer story is that the variance is modeled by two terms: one for Poisson numbers and an over-dispersion term. As counts get larger (a few hundred or so), the Poisson term becomes small and the variance is handled by the over-dispersion term. The values of reporter ion peak heights, particularly when summed into protein totals, are large enough numbers. One could argue that limma would be a better choice. However, limma is a little more complicated to set up and run than edgeR.
Notebooks are a great framework for trying different analysis steps. In the previous version of this data set analysis, edgeR testing was compared to a two-sample t-test. Running different tests in R is not hard, what is hard is comparing different analysis methods and trying to define a metric to measure performance.
collect_results <- function(df, tt, x, xlab, y, ylab) {
# Computes new columns and extracts some columns to make results frame
# df - data in data.frame
# tt - top tags from edgeR test
# x - columns for first condition
# xlab - label for x
# y - columns for second condition
# ylab - label for y
# returns a new dataframe
# condition average vectors
ave_x <- rowMeans(df[x])
ave_y <- rowMeans(df[y])
# FC, direction, candidates
fc <- ifelse(ave_y > ave_x, (ave_y / ave_x), (-1 * ave_x / ave_y))
direction <- ifelse(ave_y > ave_x, "up", "down")
candidate <- cut(tt$FDR, breaks = c(-Inf, 0.01, 0.05, 0.10, 1.0),
labels = c("high", "med", "low", "no"))
# make data frame
temp <- cbind(df[c(x, y)], data.frame(logFC = tt$logFC, FC = fc,
PValue = tt$PValue, FDR = tt$FDR,
ave_x = ave_x, ave_y = ave_y,
direction = direction, candidate = candidate,
Acc = tt$genes))
# fix column headers for averages
names(temp)[names(temp) %in% c("ave_x", "ave_y")] <- str_c("ave_", c(xlab, ylab))
temp # return the data frame
}
# compute the exact test models, p-values, FC, etc.
et <- exactTest(y, pair = c("media", "exosome"))
# make the results table
tt <- topTags(et, n = Inf, sort.by = "none")$table
med_exo <- collect_results(paw_tmt_tmm, tt, M, "med", E, "exo")
The exactTest
function does the modeling for the specified pair. More than one pair can be present in DGEList objects to improve the dispersion estimates. The topTags
function does the multiple testing corrections and returns more concise summaries of the testing results. The collect_results
function combines some of the normalized data and test results into a handy results data frame.
It is important to see if the modeling looks reasonable. Our general assumptions are that we have a large fraction of the proteins that are not differentially expressed. Those will have a uniform (flat) p-value distribution from 0.0 to 1.0. We also expect (hopefully) some true differential expression candidates. Those should have very small p-values and have a sharper distribution at low p-values.
pvalue_plots <- function(results, ylim, title) {
# Makes p-value distribution plots
# results - results data frame
# ylim - ymax for expanded view
# title - plot title
p_plot <- ggplot(results, aes(PValue)) +
geom_histogram(bins = 100, fill = "white", color = "black") +
geom_hline(yintercept = mean(hist(results$PValue, breaks = 100,
plot = FALSE)$counts[26:100]))
# we will need an expanded plot
p1 <- p_plot + ggtitle(str_c(title, " p-value distribution"))
p2 <- p_plot + coord_cartesian(xlim = c(0, 1.0), ylim = c(0, ylim)) +
ggtitle("p-values expanded")
grid.arrange(p1, p2, nrow = 2) # from gridExtra package
}
# check the p-value distrubution
pvalue_plots(med_exo, 100, "Media vs Exosome-dosed")
We have the two distributions of p-values, so the testing seems reasonable. We can also see how the up-regulated protein number compares to the down-regulated number. We can use the topTags
function to see which proteins have the smallest p-values.
# see how many up and down candidates (10% FDR)
summary(decideTests(et, p.value = 0.10))
# see which proteins have the smallest p-values
topTags(et)$table
In many discovery experiments, a Benjamini-Hochberg adjusted p-value (FDR) cutoff of 0.05 (5%) might be pretty strict. We use a 10% cutoff to distinguish DE from non-DE candidates. We define three cuts on the FDR: 10% to 5% are "low" significance, 5% to 1% are medium significance, and less than 1% are more "highly" significant. Cut values can be adjusted depending on the experimental situation.
We can look at expression ratio distributions as a function of candidate category. If variance is not too variable protein-to-protein, then we would expect larger mean differences to be associated with lower FDR values. Faceted plotting in ggplot2 is a nice method for showing such things.
# see how many candidates are in each category
med_exo %>% count(candidate)
log2FC_plots <- function(results, range, title) {
# Makes faceted log2FC plots by candidate
# results - results data frame
# range - plus/minus log2 x-axis limits
# title - plot title
ggplot(results, aes(x = logFC, fill = candidate)) +
geom_histogram(binwidth=0.1, color = "black") +
facet_wrap(~candidate) +
ggtitle(title) +
coord_cartesian(xlim = c(-range, range))
}
# can look at log2FC distributions as a check
log2FC_plots(med_exo, 3, "LogFC by candidate for Media vs Exosome-dosed")
transform <- function(results, x, y) {
# Make data frame with some transformed columns
# results - results data frame
# x - columns for x condition
# y - columns for y condition
# return new data frame
df <- data.frame(log10((results[x] + results[y])/2),
log2(results[y] / results[x]),
results$candidate,
-log10(results$FDR))
colnames(df) <- c("A", "M", "candidate", "P")
df # return the data frame
}
MA_plots <- function(results, x, y, title, make_facet = TRUE) {
# makes MA-plot DE candidate ggplots
# results - data frame with edgeR results and some condition average columns
# x - string for x-axis column
# y - string for y-axis column
# title - title string to use in plots
# make_facet - flag to plot facet views
# returns a list of plots
# uses transformed data
temp <- transform(results, x, y)
# 2-fold change lines
ma_lines <- list(geom_hline(yintercept = 0.0, color = "black"),
geom_hline(yintercept = 1.0, color = "black", linetype = "dotted"),
geom_hline(yintercept = -1.0, color = "black", linetype = "dotted"))
# make main MA plot
ma <- ggplot(temp, aes(x = A, y = M)) +
geom_point(aes(color = candidate, shape = candidate)) +
scale_y_continuous(paste0("logFC (", y, "/", x, ")")) +
scale_x_continuous("Ave_intensity") +
ggtitle(title) +
ma_lines
# make separate MA plots
if (make_facet == TRUE) {
ma_facet <- ggplot(temp, aes(x = A, y = M)) +
geom_point(aes(color = candidate, shape = candidate)) +
scale_y_continuous(paste0("log2 FC (", y, "/", x, ")")) +
scale_x_continuous("log10 Ave_intensity") +
ma_lines +
facet_wrap(~ candidate) +
ggtitle(str_c(title, " (separated)"))
}
# make the plots visible
print(ma)
if (make_facet == TRUE) {
print(ma_facet)
}
}
# MA plots of DE candidates
MA_plots(med_exo, "ave_med", "ave_exo", "Media versus exosome-dosed")
The solid diagonal line is 1:1, the dotted lines are 2-fold changes. The axes are in log scale.
scatter_plots <- function(results, x, y, title, make_facet = TRUE) {
# makes scatter-plot DE candidate ggplots
# results - data frame with edgeR results and some condition average columns
# x - string for x-axis column
# y - string for y-axis column
# title - title string to use in plots
# make_facet - flag to plot facet views
# returns a list of plots
# 2-fold change lines
scatter_lines <- list(geom_abline(intercept = 0.0, slope = 1.0, color = "black"),
geom_abline(intercept = 0.301, slope = 1.0, color = "black", linetype = "dotted"),
geom_abline(intercept = -0.301, slope = 1.0, color = "black", linetype = "dotted"),
scale_y_log10(),
scale_x_log10())
# make main scatter plot
scatter <- ggplot(results, aes_string(x, y)) +
geom_point(aes(color = candidate, shape = candidate)) +
ggtitle(title) +
scatter_lines
# make separate scatter plots
if (make_facet == TRUE) {
scatter_facet <- ggplot(results, aes_string(x, y)) +
geom_point(aes(color = candidate, shape = candidate)) +
scatter_lines +
facet_wrap(~ candidate) +
ggtitle(str_c(title, " (separated)"))
}
# make the plots visible
print(scatter)
if (make_facet == TRUE) {
print(scatter_facet)
}
}
scatter_plots(med_exo, "ave_med", "ave_exo", "Media versus exosome-dosed")
Volcano plots are another common way to visualize DE candidates.
volcano_plot <- function(results, x, y, title, ymax) {
# makes a volcano plot
# results - a data frame with edgeR results
# x - string for the x-axis column
# y - string for y-axis column
# ymax - upper limit for y-axis
# title - plot title string
# uses transformed data
temp <- transform(results, x, y)
# build the plot
ggplot(temp, aes(x = M, y = P)) +
geom_point(aes(color = candidate, shape = candidate)) +
xlab("log2 FC") +
ylab("-log10 FDR") +
coord_cartesian(xlim = c(-5, 5), ylim = c(0, ymax)) +
ggtitle(str_c(title, " Volcano Plot"))
}
# finally, a volcano plot
volcano_plot(med_exo, "ave_med", "ave_exo", "Media versus exosome-dosed", 50)
We can see how the intensities of the individual samples compare for the top 20 up- and down-regulated candidate proteins.
# function to extract the identifier part of the accesssion
get_identifier <- function(accession) {
identifier <- str_split(accession, "\\|", simplify = TRUE)
identifier[,3]
}
set_plot_dimensions <- function(width_choice, height_choice) {
options(repr.plot.width=width_choice, repr.plot.height=height_choice)
}
plot_top_tags <- function(results, nleft, nright, top_tags) {
# results should have data first, then test results (two condition summary table)
# nleft, nright are number of data points in each condition
# top_tags is number of up and number of down top DE candidates to plot
# get top up-regulated
up <- results %>%
filter(logFC >= 0) %>%
arrange(FDR)
up <- up[1:top_tags, ]
# get top down-regulated
down <- results %>%
filter(logFC < 0) %>%
arrange(FDR)
down <- down[1:top_tags, ]
# pack them
proteins <- rbind(up, down)
color = c(rep("red", nleft), rep("blue", nright))
for (row_num in 1:nrow(proteins)) {
row <- proteins[row_num, ]
vec <- as.vector(unlist(row[1:(nleft + nright)]))
names(vec) <- colnames(row[1:(nleft + nright)])
title <- str_c(get_identifier(row$Acc), ", int: ", scientific(mean(vec), 2),
", p-val: ", scientific(row$FDR, digits = 3),
", FC: ", round(row$FC, digits = 1))
barplot(vec, col = color, main = title)
}
}
# plot the top 20 up and 20 down proteins
set_plot_dimensions(7, 4)
plot_top_tags(med_exo, 3, 4, 20)
set_plot_dimensions(7, 7)
EdgeR normalizations and testing seemed reasonable for this data.
There were a large number of significant candidates. There are options in edgeR (if using the glm extensions) to add a minimum fold-change to the testing. That can be used to reduce the number of candidates. Stricter FDR cutoffs could also be used. Ranking candidates for follow-up study is not trivial. Statistical values like fold-changes and p-values may not be sufficient to determine biologically relevant rankings. Biological information for this system should be an important consideration.
There is a very uneven distribution of where effort is directed in these workflows. Label reagents, instrument methods, search engines, and PSM validation have gotten the most attention. Normalizations and statistical models have received some attention. Visualizing results and making biological sense of things are critical steps that have been neglected for too long. This is the critical bottleneck.
One step towards more meaningful results summaries are getting the main proteomics results and the statistical results combined into a single table. If we are careful with how we bring the data into R and how we take it back out, we can keep tables in order so merging results are easier. There are several strategies that can work well. If the data coming into R and the data coming out of R are not easy to align correctly, then edit the notebooks to make it work right. More data prep can occur before importing into R, for example. We kept the accessions column as we worked with the data. We can be really robust and use merge functions to weave together starting tables with final results tables.
We should always end notebooks with information about what packages and versions were used in the analysis.
# save the testing results
write.table(med_exo, file = "KUR1502_results.txt", sep = "\t",
row.names = FALSE, na = " ")
# log the session
sessionInfo()


Program
With Alerts
U.S. Department of Veterans Affairs | eScreening Health Assessment (Ver 1.0)
Dashboard
Alerts
Help
Search
No Alerts
Assessments Nearing Completion (Top 5)
Charts
List
Welcome Last Name, First Name | Logout
* This wireframe provide the workflow, the structure, and the interaction between some specific pages and sections in the eScreening project and it's covered all the interaction between the application features and behaviours.
* Some of the interaction between pages may require using a specific rules and steps in order to demonstrate.
* This wireframe still in progress.
IMPORTANT NOTE - Click to Expand
Time
10%
20%
30%
40%
50%
60%
70%
80%
90%
Veteran 1
2122
Veteran 2
2992
Veteran 3
2002
Veteran 4
2929
Veteran 5
1234
100%
Slow Moving Incomplete Assessments (Bottom 5)
10
20
30
40
50
60
70
80
90
Veteran 1
1234
Veteran 2
0000
Veteran 3
1111
Veteran 4
2239
Veteran 5
2929
With Alerts
No Alerts
No Alert
Total Veteran:1
CSRA Required
Total Veteran:1
Imminent Risk
Total Veteran:1
DX/TX Present
Total Veteran:1


Seems your browser doesn't support Javascript! Websocket relies on Javascript being enabled. Please enable Javascript and reload this page!
Websocket connection disconnected!
Please try connecting again.
×
Falcon Chat
Message:
Send over WS
Message Subscription
Connect
Disconnect
Messages:
#
Chat Id
Message
Sent from Host


 We observe that there is some pattern we follow while defining these structures:
setoid
) of objects| Structure | Totality | Associativity | Identity | Invertibility | Commutativity |
|---|---|---|---|---|---|
| Magma | ★ | ||||
| Semigroupoid | ★ | ||||
| Small Category | ★ | ★ | |||
| Groupoid | ★ | ★ | ★ | ||
| Semigroup | ★ | ★ | |||
| Inverse Semigroup | ★ | ★ | ★ | ||
| Monoid | ★ | ★ | ★ | ||
| Commutative monoid | ★ | ★ | ★ | ★ | |
| Group | ★ | ★ | ★ | ★ | |
| Abelian group | ★ | ★ | ★ | ★ | ★ |
It turns out there is a set of basic properties of a large number of mathematical structures, one can build tools on top of these properties such that they’re applicable in a wide area of mathematics.
Category Theory is the set of tools to describe and work with generalized aspects of mathematical structures. A systematic study of category theory then allows us to prove general results about any of these types of mathematical structures. The stanford encyclopedia of philosophy describes it as:
Roughly, it is a general mathematical theory of structures and of systems of structures. At minimum, it is a powerful language, or conceptual framework, allowing us to see the universal components of a family of structures of a given kind, and how structures of different kinds are interrelated.
Different areas of math share common patterns/trends/structures. For mathematicians, this becomes extraordinarily useful when you want to solve a problem in one realm (say, topology) but don’t have the right tools at your disposal. By transporting the problem to a different realm (say, algebra), you can see the problem in a different light and perhaps discover new tools, and the solution may become much easier. The bridges between realms are also provided by category theory.
Category Theory has grown tremendously over the last few decades and has a large number of applications including:
monad
that represents side-effects, especially IO. A mixture of structures from abstract algebras and category theory are used to form a class of programming patterns called “functional programming”.


White House blocked me from Twitter, claims former Trump advisor
White House spokeswoman Stephanie Grisham, however, says John Bolton simply doesn’t know how to use the social media platform.
by AFPNational Security Adviser John Bolton stands alongside US President Donald Trump as he speaks about the prospects for the summit with North Korea going ahead -- a meeting put in doubt when Pyongyang bristled at comments made by Bolton. AFP/SAUL LOEB
Former Trump National Security Advisor John Bolton said the White House denied him access to his Twitter account after his dismissal in early September, in a series of tweets that signal his return to the social media site.
“Since resigning as National Security Advisor, the White House has refused to return access to my personal Twitter account. Out of fear of what I may say?” Bolton wrote on Friday.
Prior to Friday’s series of tweets, nothing had been published by Bolton since September 10, the day the US president announced his dismissal.
“To those who speculated I went into hiding, I’m sorry to disappoint!” he added.
The White House swept aside the charges, with spokeswoman Stephanie Grisham suggesting Bolton simply didn’t know how to use the platform.
“Sometimes, I’ll use my father as an example,” she told a Fox Business Network discussion program.
“Somebody who is of an advanced age may not understand all you have to do is contact Twitter and reset your password if you’ve forgotten it.”
Since being dismissed, 71-year-old John Bolton has disagreed with the US president’s strategy on North Korea.
Early on Friday, the former diplomat announced his return to Twitter with an enigmatic message: “Glad to be back on Twitter after more than two months. For the backstory, stay tuned…”.
His final tweet in the series read: “Thank you to Twitter for standing by their community standards and rightfully returning control of my account.”


Next: Grune and Jacobs 1990, Previous: Dominus 2005, Up: Annotated bibliography [Contents][Index]
Of Jay Earley’s papers on his general parsing algorithm, the most readily available is “An efficient context-free parsing algorithm”, Communications of the Association for Computing Machinery, 13:2:94-102, 1970.
Ordinarily, I’d not bother pointing out 35-year old nits in a brilliant and historically important article. But more than a few people treat this article as not just the first word in Earley parsing, but the last as well. Many implementations of Earley’s algorithm come, directly and unaltered, from his paper. These implementers and their users need to be aware of two issues.
First, the recognition engine itself, as described, has a serious bug. There’s an easy fix, but one that greatly slows down an algorithm whose main problem, in its original form, was speed. This issue is well laid out by Aycock and Horspool in their article. See Aycock and Horspool 2002.
Second, according to Tomita there is a mistake in the parse tree representation. See page 153 of Grune and Jacobs 1990, page 210 of Grune and Jacobs 2008, and the bibliography entry for Earley 1970 in Grune and Jacobs 2008. In the printed edition of the 2008 bibliography, the entry is on page 578, and on the web (ftp://ftp.cs.vu.nl/pub/dick/PTAPG_2nd_Edition/CompleteList.pdf), it’s on pp. 583-584. My methods for producing parse results from Earley sets do not come from Earley 1970, so I am taking Tomita’s word on this one.


Mathematical researcher, who thinks and lives algorithms.
I’m the founder of Mathscapes Research and co-founder of M56 Design Lab with two of my friends. Most of my work lies in the union of mathematics and computer science. I’m mostly self-taught in the areas of my research. My research explores mathematical boundaries in the context of computer science and speculating possibilities in future by pushing these boundaries, which is also the basis of the Mathscapes vision.
After completing my degree in Computer Science & Engineering, I studied Innovation and Experience Design in my masters. Later, I spent a year as a research assistant at Art in Transit project and then started working as a full-time faculty member at Srishti Institute of Art, Design & Technology, Bangalore between 2016-Present. At Srishti, I facilitate studios related to Algorithms, Computation, and HCI. My time at Srishti has been both memorable and eye-opening – and informed a lot of decisions that shaped who I am today.
Our world is unavoidably complex. And to design is not to avoid complexity but to embrace complexity. The process of design deals with this complexity and translates it into an illusion of simplicity.


You have 60 seconds to complete 3 questions. Click reset at the end if you want to take the quiz again.


We're sorry but moon_ui doesn't work properly without JavaScript enabled. Please enable it to continue.


--- layout: post esbma_id: 1680 title: This was NOT my fault date: 2007-07-02 14:38:34 keywords: laughs ---
I use Haloscan to manage my blog comments. One of the nicer features is that every time somebody posts a comment, Haloscan sends me a copy of that comment by email.
But sometime in the last few months, they started truncating those emails, presumably to save bandwidth. No problem, but sometimes the point of truncation lands in a rather unfortunate spot.
Lots of people have been commenting on last week's post about women in our comic. This morning I posted a comment of my own, responding to someone who asked if I was being serious or just fishing for publicity. I said:
"Actually, it is a serious question. Although I am comfortable with the comic, I've heard some complaints from women I respect, so I thought I would solicit a broader set of perspectives."
And I kid you not, when Haloscan sent me the email, this is what I received:
"Actually, it is a serious question. Although I am comfortable with the comic, I've heard some complaints from women I respect, so I thought I would solicit a broad"
:-)


If you want more such products and images please message us on our Whatsapp Number-86055897751. Or see below
If you have any query feel free to contact or mail us. We will reply you as soon as possible.
NTC, Tapadiya Estate Hingoli-431513, Maharashtra


Presidential Election Results
By State | 2000–2016
Republican
Democrat
2000
Zach Postone, 2017. Data: U.S. National Archives and Records Administration


|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Receiver: This interface represents a recipient of XML tree-walking events. It is based on SAX2's ContentHandler, but adapted to handle additional events, and to use Saxon's name pool. Namespaces and Attributes are handled by separate events following the startElement event. Schema types can be defined for elements and attributes. This interface is designed as an eventual replacement for the old Emitter class, as well as the Outputter interface. The name change from "Emitter" to "Receiver" reflects that the essence of the interface is that all implementations receive notification of these events; they do very different things with the events once recieved.
| Field Summary |
| Fields inherited from interface javax.xml.transform.Result |
PI_DISABLE_OUTPUT_ESCAPING, PI_ENABLE_OUTPUT_ESCAPING |
| Method Summary | |
void |
attribute(int nameCode,
int typeCode,
java.lang.CharSequence value,
int properties)
Notify an attribute. |
void |
characters(java.lang.CharSequence chars,
int properties)
Notify character data. |
void |
comment(java.lang.CharSequence content,
int properties)
Notify a comment. |
void |
endDocument()
Notify the end of the document |
void |
endElement()
Notify the end of an element. |
Configuration |
getConfiguration()
Get the configuration |
void |
namespace(int namespaceCode,
int properties)
Notify a namespace. |
void |
processingInstruction(java.lang.String name,
java.lang.CharSequence data,
int properties)
Output a processing instruction |
void |
setConfiguration(Configuration config)
Set the configuration |
void |
setDocumentLocator(org.xml.sax.Locator locator)
Set locator, to identify position in the document. |
void |
setSystemId(java.lang.String systemId)
Set the System ID of the destination tree |
void |
setUnparsedEntity(java.lang.String name,
java.lang.String systemID,
java.lang.String publicID)
Notify an unparsed entity URI. |
void |
startContent()
Notify the start of the content, that is, the completion of all attributes and namespaces. |
void |
startDocument()
Notify the start of the document |
void |
startElement(int nameCode,
int typeCode,
int properties)
Notify the start of an element |
| Methods inherited from interface javax.xml.transform.Result |
getSystemId |
| Method Detail |
public void setConfiguration(Configuration config)
public Configuration getConfiguration()
public void setSystemId(java.lang.String systemId)
setSystemId
in interface javax.xml.transform.Result
public void setDocumentLocator(org.xml.sax.Locator locator) throws javax.xml.transform.TransformerException
javax.xml.transform.TransformerException
public void startDocument() throws javax.xml.transform.TransformerException
javax.xml.transform.TransformerException
public void setUnparsedEntity(java.lang.String name, java.lang.String systemID, java.lang.String publicID) throws javax.xml.transform.TransformerException
name
- The name of the unparsed entitysystemID
- The system identifier of the unparsed entitypublicID
- The public identifier of the unparsed entity
javax.xml.transform.TransformerException
public void startElement(int nameCode, int typeCode, int properties) throws javax.xml.transform.TransformerException
nameCode
- integer code identifying the name of the element within the name pool.typeCode
- integer code identifying the element's type within the name pool.
javax.xml.transform.TransformerException
public void namespace(int namespaceCode, int properties) throws javax.xml.transform.TransformerException
namespaceCode
- an integer: the top half is a prefix code, the bottom half a URI code.
These may be translated into an actual prefix and URI using the name pool. A prefix code of
zero represents the empty prefix (that is, the default namespace). A URI code of zero represents
a URI of "", that is, a namespace undeclaration.
IllegalStateException:
- attempt to output a namespace when there is no open element
start tag
javax.xml.transform.TransformerException
public void attribute(int nameCode, int typeCode, java.lang.CharSequence value, int properties) throws javax.xml.transform.TransformerException
nameCode
- The name of the attribute, as held in the name pooltypeCode
- The type of the attribute, as held in the name poolproperties
- Bit significant value. The following bits are defined:
IllegalStateException:
- attempt to output an attribute when there is no open element
start tag
javax.xml.transform.TransformerException
public void startContent() throws javax.xml.transform.TransformerException
javax.xml.transform.TransformerException
public void endElement() throws javax.xml.transform.TransformerException
javax.xml.transform.TransformerException
public void characters(java.lang.CharSequence chars, int properties) throws javax.xml.transform.TransformerException
chars
- The charactersproperties
- Bit significant value. The following bits are defined:
javax.xml.transform.TransformerException
public void processingInstruction(java.lang.String name, java.lang.CharSequence data, int properties) throws javax.xml.transform.TransformerException
name
- The PI name. This must be a legal name (it will not be checked).data
- The data portion of the processing instructionproperties
- Additional information about the PI. The following bits are
defined:
IllegalArgumentException:
- the content is invalid for an XML processing instruction
javax.xml.transform.TransformerException
public void comment(java.lang.CharSequence content, int properties) throws javax.xml.transform.TransformerException
content
- The content of the commentproperties
- Additional information about the comment. The following bits are
defined:
IllegalArgumentException:
- the content is invalid for an XML comment
javax.xml.transform.TransformerException
public void endDocument() throws javax.xml.transform.TransformerException
javax.xml.transform.TransformerException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |


4 references to s_sizeDict
System.Management.Automation (4)
engine\runtime\MutableTuple.cs (4)
284
lock (
s_sizeDict
) if (
s_sizeDict
.TryGetValue(tupleType, out count)) return count;
306
lock (
s_sizeDict
)
s_sizeDict
[tupleType] = count;


Valley Trader Kitzka:
You selected
Will of Antharas
. Here is is. If you obtain the Sealed Blood Crystal later, ask the Separated Soul to combine it for you. If you do, you will realize the true value of this item. Good luck!