I pity the fool...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
Get to the chopper...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
What's a horse doing on a spaceship...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
I pity the fool...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
Get to the chopper...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
What's a horse doing on a spaceship...
What is happening with you two? Ma we both went fishing at the stream, while I was busy dipping the net into the stream to check for fishes, he was busy playing, he never cared to give me a helping hand even as I called him, now my hook has caught this big fish and he now said we must share the fish equally.
My First Bootstrap Page
Resize this responsive page to see the effect!


Please make sure all users are logged out of your account, and all sync processes are disabled while using utilities in this area. They are *very* resource intensive and can cause outages if concurrent use is attempted.
| Resets/Rebuilds Product Index, clears out deleted items, and source pid from zoovy:related_products, and zoovy:accessory_products. Removes unicode characters. | |
| Removes orphan inventory records which are not associated with a sku, creates records w/0 qty for pids which don't have inv. records. | |
| Removes orphaned inventory reservations / verifies reserves. | |
| Image Reindex | |
| Elastic Search Rebuild (note: will be offline/unavailable during rebuild) | |
|
Goes through all products which have customer reviews and updates zoovy:prod_salesrank
NOTE: sometimes design uses prod_salesrank to store other data like dates, doing this will destroy that data. Verify with customer this field is intended to store sales ranking data based on reviews prior to execution. |
| Ends all active listings on eBay. |
| Rebuilds all KPIBI data for the last year |
| A report of inventory events. | |
| A report of unprocessed user events - including inventory, crm, product, order events - create, paid, ship, cancel |
JT can build these these
but probably not this one


Split Earth
Splits the earth under enemies. Deals damage and stuns for a short duration.
| Mana Cost: | 100/125/140/160 |
| Cooldown: | 9 |
| ABILITY: | Point Target |
| AFFECTS: | Enemy Units |
| DAMAGE TYPE: | Magical |
| PIERCES SPELL IMMUNITY: | No |
| DAMAGE: | 120 / 180 / 240 / 300 |
| STUN DELAY: | 0.35 |
| RADIUS: | 150 / 175 / 200 / 225 |
| STUN DURATION: | 2 |
Twisting nature to his vile will, the shifting earth consumes those unlucky enough to cross Leshrac's path.
Diabolic Edict
Saturates the area around Leshrac with magical explosions that deal physical damage to enemy units and structures. The fewer units available to attack, the more damage those units will take. Deals 40% more damage to towers. Lasts 10 seconds.
| Mana Cost: | 95/120/135/155 |
| Cooldown: | 22 |
| ABILITY: | No Target |
| AFFECTS: | Enemy Units |
| DAMAGE TYPE: | Physical |
| PIERCES SPELL IMMUNITY: | Yes |
| DAMAGE: | 9 / 18 / 27 / 36 |
| EXPLOSIONS: | 40 |
| RADIUS: | 500 |
Chronoptic energy bursts from one plane to the other, evaporating anything it touches.
Lightning Storm
Summons a lightning storm that blasts the target enemy unit, then strikes any nearby enemy units. Struck enemies are slowed.
| Mana Cost: | 90/100/110/120 |
| Cooldown: | 4 |
| ABILITY: | Unit Target |
| AFFECTS: | Enemy Units |
| DAMAGE TYPE: | Magical |
| PIERCES SPELL IMMUNITY: | No |
| DAMAGE: | 80 / 140 / 200 / 260 |
| LIGHTNING STRIKES: | 4 / 5 / 6 / 7 |
| MOVEMENT SLOW: | 75% |
| SLOW DURATION: | 0.7 / 0.8 / 0.9 / 1 |
The Tormented Soul's mastery of the elements is evident in the massive storms that strike down armies before him.
Pulse Nova
Creates waves of damaging energy around Leshrac, one per second, to damage nearby enemy units. Upgradable by Aghanim's Scepter.
| Mana Cost: | 70/90/110 |
| Cooldown: | 1 |
| ABILITY: | No Target, Toggle |
| AFFECTS: | Enemy Units |
| DAMAGE TYPE: | Magical |
| PIERCES SPELL IMMUNITY: | No |
| MANA/SEC: | 20 / 40 / 60 |
| RADIUS: | 450 |
| DAMAGE: | 100 / 130 / 160 |
| SCEPTER DAMAGE: | 160 / 190 / 220 |
If necessary, the Tormented Soul can manipulate space time itself, ravaging lesser beings.


- Change left_command to lazy left_control
- Change right_command to lazy right_command
-
Change spacebar to left_shift if pressed with other keys (Post spacebar when
pressed alone)
- Change command+control+f to command+shift+[
- Change command+control+j to command+shift+]
- Change command+control+g to command+[
- Change command+control+h to command+]
- Change control+1 to home
- Change control+2 to page_down
- Change control+3 to page_up
- Change control+4 to end
- Change option+- to ----------
- Change option+= to ==========
- Change fn+page_down to display_brightness_decrement_decrement
- Change fn+page_up to display_brightness_decrement_increment
- Change page_down to volume_decrement
- Change page_up to volume_increment
- Miscellaneous configuration for apps
- Change simultaneous escape+1 to display_brightness_decrement
- Change simultaneous escape+2 to display_brightness_increment


True if this and the other Collection have value equality, as defined
by Immutable.is()
.
Note: This is equivalent to Immutable.is(this, other)
, but provided to
allow for chained expressions.
Computes and returns the hashed identity for this Collection.
The hashCode
of a Collection is used to determine potential equality,
and is used when adding this to a Set
or as a key in a Map
, enabling
lookup via a different instance.
const { List, Set } = require('immutable');
const a = List([ 1, 2, 3 ]);
const b = List([ 1, 2, 3 ]);
assert.notStrictEqual(a, b); // different instances
const set = Set([ a ]);
assert.equal(set.has(b), true);
Note: hashCode() MUST return a Uint32 number. The easiest way to
guarantee this is to return myHash | 0
from a custom implementation.
If two values have the same hashCode
, they are not guaranteed
to be equal. If two values have different hashCode
s,
they must not be equal.
Note: hashCode()
is not guaranteed to always be called before
equals()
. Most but not all Immutable.js collections use hash codes to
organize their internal data structures, while all Immutable.js
collections use equality during lookups.
Generated using TypeDoc
The interface to fulfill to qualify as a Value Object.


Administration Guide
This document provides information on using the administration pages for Php Image Host to manage your image hosting site, its members, and the images they upload. There are eight separate sections in the admin area:
- Admin Home: This page just displays a quick link to and brief description of all the other pages.
- Images: Allows you to view and delete all uploaded images.
- Users: Search for, list, view, edit, add, upgrade and downgrade all registered members.
- Payments: List and view details of all account upgrade payments processed by the PayPal IPN script.
- Site Settings: Main control panel for site options and editing templates for any email messages automatically sent by the site, for example when a new member signs up.
- Plan Details: Configure the different features for the free and the paid image hosting account types.
- Banners: Manage the banner ads that are rotated at the top and bottom of each page of the site.
Images Admin
The images page lets you view a list of all images uploaded by your members. Each listing includes a thumbnail version of the image that displays the full image in a new window when clicked.
The size of the image (in pixels and kbytes) together with the format (jpeg, gif or png), the date it was uploaded and the member who uploaded it are also displayed next to each thumbnail. Clicking on the name of the member lists all the images uploaded by that member. A "View" link next to the member's username takes you to the member details page for that member.
You can filter the list of images using the search box on the left of the page. The list can be filtered by the following options: image name, dates uploaded between, member, format and filesize. Images can be listed in ascending or descending order of name, date uploaded, format and filesize.
Images can be deleted in bulk (for example if a member is abusing the service) with just a couple of clicks, by selecting the checkboxes next to the images to delete and clicking the "Delete Selected Images" button.
 This page lists the following information about each user: username, name, email address, date joined (with I.P. address), the date they last logged in (with I.P. address), the number of images they have uploaded (as a link to the images themselves), their account status, and their account type (free or paid).
You can search for members by username, name and email, using * as a wildcard (for example, *sam matches all users whose name ends in sam). You can also search by the date that the users last logged in, or the date that they joined, in addition to their account type (free, paid or all) and account status (unconfirmed email address, confirmed, and suspended).
Users can be listed in ascending or descending order of name, username, date joined, last login date, email address, account status, account type, and number of images uploaded.
There are four bulk-edit options available for managing users. To bulk edit users you check the checkboxes next to the users you want to edit and click one of the buttons at the bottom of the list.
The options available are:
"Upgrade Selected Users For n Days" which makes gives the selected users "Paid Account" status for an additional n days
"Downgrade Selected Users" which makes all selected users' accounts "Free".
"Change Selected Users Status To" either unconfirmed (which means that they have to click a link in an email sent to their email address to confirm it is real before they can access their account, confirmed, which gives them access to their account, or suspended which stops them from accessing their account without actually deleting the account and the images in it - useful if you are looking into reports of abuse of your terms and conditions.
"Delete Selected Users" deletes all the selected users from the database, and also deletes all images they have uploaded from your server. This option is irreversible so should only be used as a last resort. If you just need to stop a user from accessing their account while you check if they are abusing your terms & conditions, please change the selected users' status to suspended instead.
You can also edit the account details for a single user by clicking on their name. This displays a form where you can edit their username, name, email address, password and account status. You can manually add a new user by clicking the "Add A New User" link on the bottom left of the page.
Payments Admin
The payments page lists all payments that have been made via PayPal for account upgrades that have been logged by the included PayPal Instant Payment Notification script. The following information is displayed for each payment: PayPal transaction id#, the payer's PayPal email address, their username, account email address, the date of the transaction and the amount. Full PayPal details of the transaction can be viewed in a pop-up window by clicking the transaction id#.
The payments list can be filtered by username, user email address, PayPal email address, PayPal transaction id#, and date received, and can be displayed in ascending or descending order of date, transaction id#, amount, PayPal email address, user email address and username.
Payment records can be deleted from the database by checking the checkboxes next to the payments and clicking the "Delete Selected Payments" button. IMPORTANT: this does not refund, or affect in any way, the actual payments made or the balances of the PayPal accounts. It just removes the payment history from your site's database.
Site Settings Page
This page is used to setup and configure most of your image hosting site's settings, and is divided into 7 sections:
Admin Settings include the username and password you use to login to the admin area (please change these from the default "admin" / "password" ASAP), your email address (that messages submitted via the contact form are sent to), and a checkbox that can be used to turn on or off the ability for new members to sign up for an account.
General Site Settings include the name for your site (modifying this here changes it on all pages of the site), the full url of your site (which should be in the format http://www.yoursite.com/), and an option to temporarily disable all image uploads (for example if you need time to investigate abuse of your terms & conditions).
Image Settings lets you set the maximum width and height for the thumbnail images automatically generated when a user uploads an image, as well as specifiying the urls and path to the folders on your servers where uploaded images and generated thumbnails are stored. These must be set correctly for your site to function properly.
Image Branding Options allow you to customize the branding watermark that can be added to images uploaded by your users (normally only used on free accounts). You can specify the text, size, color, background color and transparency, as well as set a minimum width and height for images that the branding will be added to.
Payment Settings lets you specify your PayPal email address (where payments are made to when users upgrade their accounts), as well as the subject and message for the confirmation email which is automatically sent to them if PayPal Instant Payment Notification is turned on for your account. The message can contain certain "tags" which are replaced with the relevant values (for example, {username} is replaced by the member's username). See the settings page itself for a full list of these tags.
Email Images Settings lets you customize the email subject and message that is sent when a member emails links to their uploaded images to their friends from within their control panel. You can add anything you want to this email message template, including your own advertising; just make sure you remember to include the {imagelinks} tag to provide the links back to their images.
Member Email Settings allows you to customize the email subject and message for emails sent to members when they sign up for a free account or request a password reminder. A full description of the tags that can be used in these messages is displayed on the settings page below the forms to edit them.
Plan Details
The Plan Details page lets you specify the limits and featured for the free and paid account types that your member's can sign up for. In addition to setting the name for each type of account and the price for a paid account for 30 days, you can set the following options for each account type: total allowed images, maximum upload filesize, maximum image width and height (and whether to automatically resize oversized images, or reject them), as well as whether to automatically convert all uploads to jpegs (reducing your bandwidth usage) and the jpeg quality level to save them as (higher = better quality, larger file size).
Additionally, the simultaneous uploads number refers to the number of images a member can upload in one go, "thumbnails per page" refers to the number of their images that are listed on each page of their account page (less per page = more page impressions for you), and the "email images to friends" option specifies how many email addresses they can send links to their images to at one time.
The final option gives you the choice of adding your own branding watermark to every image uploaded for each account type. For more information, see the Image Branding Options section in the settings admin above.
Page Content Admin
The page content admin area lets you edit most of the text that appears on the pages of your images hosting site. You can select which bit of content to edit by selecting it from the drop down box at the top of the page. You will then be able to edit the html content in the textarea. Remember to click the "Update Content" button to save your changes when you have finished editing them.
The option to create new page content only applies if you have added extra pages to your site and want to use this page to edit the content on them. For an example of displaying content edited here see the source code in the faq.php script.
When editing the page content, remember that it will be displayed directly as html, so you need to include the relevant html tags such as <br /> for a line break, etc.
Banners Admin
Your image hosting site contains code to rotate banner adverts at the top and bottom of every page of the site. You can add and edit these banners in the "Banners" section of the site admin area.
To allow for the most flexibility in the type of banners displayed, you must enter the html to display the banner directly (this means that if you are displaying banners with your own images, you have to manually upload them to your site).
This does mean that the banners displayed can be in any format (image, flash, or even plain text or html, like google adsense). You can view banners, set them to active or inactive, and delete them in this section.


--- layout: post title: "A funny song on the MVC paradigm" date: 2008-05-02 comments: false categories: - geek fun music ---


Before Change
]
logging.info("raw diffs lengths: %s" % str([len(row) for row in raw_diffs]))
sums = [reduce(th.add, param) for param in raw_diffs]
logging.info("sums shapes: %s" % str([sum.shape for sum in sums]))
diff_avg = [th.div(param, len(diffs)) for param in sums]
logging.info("diff_avg shapes: %s" % str([d.shape for d in diff_avg]))
// apply avg diff!
_updated_model_params = [
After Change
// check if the uploaded avg plan is iterative or not
iterative_plan = server_config.get("iterative_plan", False)
if iterative_plan:
diff_avg = raw_diffs[0]
for i, diff in enumerate(raw_diffs[1:]):
diff_avg = avg_plan(diff_avg, diff, i + 1)
else:
diff_avg = avg_plan(raw_diffs)
logging.info("diff_avg shapes: %s" % str([d.shape for d in diff_avg]))
// apply avg diff!
_updated_model_params = [


As the sea monsters started moving into the city they started to destroy the land making room for them "stop said cody what are you guys doing" "we're moving in duh" said the sea monsters general "but this wasn't part of the treaty you promised to put the past behind you" said cody "oh i'm sorry your highness said the general he held out his hand with his fingers crossed "But crosses doesn't count" "You sick.."said cody "ah ah ah" said the general "trash talk,us sea monsters will never forgive you merpeople for kicking us out of the kingdom" said the sea monster captain "i wanted the merfolk and sea monsters to live in peace now i realize that taking you guys down is the only way to end this war"said cody "so be it"said the captain "attack" the captain said to his troops "counterattack"cody said to the merpeople and so to make a long story short,the merfolk and sea monsters got into a roughhousing battle,it ended when the king of merdor returned to his kingdom and he blasted the monsters out of the kingdom with his trident after that situation Cody told hai father that he was his lost son the king was overjoyed over his son's return he threw a party to celebrate our heroes homecoming,but one thing on cody's mind was if he should stay with the merpeople of should he go back to the surface ,well that's another story. Man,it seems like the merfolk and sea monsters will never get along,but will the war ever end? well if you go back to page 11 and go down a different path there might be a way for the war to end permanently,but until then i have to say
THE END


A web application that takes a user's birthday and calculates the day of the week they were born. Then depending on their gender outputs their Akan Name.
Akan names are derived from Ghanian culture. Frequently in Ghana, children are given their first name as a 'day name' which corresponds to the day in the week they were born. Here are Ghanian day names.
Here are some Ghanaian Names:


| Github Repository |
PyPI |
|
The base rated repository is Flask with {{flask.github_info.watchers}} stars
and {{flask.github_info.forks}} forks, resulting in a Github score of 100.00
|
The base rated PyPI package is Flask with {{flask.pypi_info.download_num}} total downloads
|
| These are the references for the score, marking the popularity of 100% |
|
Now, the repository for {{package.name}} has got {{package.github_info.watchers}} watchers and {{package.github_info.forks}} forks
|
Now, the package {{package.name}} has got {{package.pypi_info.download_num}} total downloads
|
|
Therefore, the relative popularity percentage can be calculated for {{package.name}}
|
|
{{package.github_info.watchers}} / {{flask.github_info.watchers}} * 100 * 45% +
{{package.github_info.forks}} / {{flask.github_info.forks}} * 100 * 55%
|
= {{ ((package.github_info.watchers / flask.github_info.watchers * 45 + package.github_info.forks / flask.github_info.forks * 55) / 2) | round(3) }}
|
| 2 * 100 |
|
|
{{package.pypi_info.download_num}} / {{flask.pypi_info.download_num}} * 100
|
= {{ (package.pypi_info.download_num / flask.pypi_info.download_num * 100) | round(3) }}
|
| 100 |
|
|
The average of those two values results in the score: {{package.score}}
|


Produces a rendered view that must be connected to a UIShell, UIframe or UIpanel to make it visible. The view has a small amount of interactivity but will make a new object fit the scene although it is translated by the same amount as the previous object. The Top Object in the scene can be manipulated with the mouse (rotate, scale, xy translate or z translate). The object can also be Reset, Normalized and Centred.
Input Ports
For the "View3D_TrackObj"
Name Type Description Top child_obj A renderable 3D object parent UIconnection Connects the rendered view to a UI object so it can be displayed. mode int Connects to the UI, values vary from 0-3
0 is rotation
1 is scale
2 is xytranslate
3 is ztranslate
trigger prim When this value changes, the object in the viewer is Reset, Normalized and Centred. Connect this to the UI. For the "View2D_TrackObj"
Name Type Description Top child_obj A renderable 2D object parent UIconnection Connects the rendered view to a UI object so it can be displayed. mode int Connects to the UI, values vary from 0-3
0 is rotation
1 is scale
2 is xytranslate
3 is ztranslate
trigger prim When this value changes, the object in the viewer is Reset, Normalized and Centred. Connect this to the UI. For the "View_TrackObj"
Name Type Description Top child_obj A renderable 3D object Top2D child_obj A renderable 2D object parent UIconnection Connects the rendered view to a UI object so it can be displayed. mode int Connects to the UI, values vary from 0-3
0 is rotation
1 is scale
2 is xytranslate
3 is ztranslate
trigger prim When this value changes, the object in the viewer is Reset, Normalized and Centred. Connect this to the UI. Description
Reducing the object count of an application is the thing most likely to speed up the loading and running of an application. This is one of a library of utility macros designed to reduce the object count.
These are all AVS/Express macros which as such require no compilation. Most applications are produced with AVS/Express Developers Edition and these macros also require AVS/Express Developers Edition. They contain no UI so that the developer has as much freedom as possible.
Any Viewer has a high object count because it has a high level of functionality. If the functionality is reduced to the minimum requirement then the object count is kept as low as possible.
This is a viewer with just a small amount of interactivity. The Top object in the scene can be manipulated with the mouse (rotate, scale, xy translate or z translate). The Type of manipulation can be selected by attaching a cmdlist to the mode port, mode is 0 for rotate, mode is 1 for scale, mode is 2 for xy translate and mode is 3 for z translate. The type of transformation matrix used for this manipulation can be altered by altering values in the GDtrack_edit module.
The object can be Reset, Normalized and Centred by adding a UItoggle or UIbutton to the trigger port.
The settings of the viewer may be customised by hand before it is used.
The size of the scene is altered by altering the values width and height in the subobject UIrenderView, this can be done with absolute values or relative values written in v.
Input Ports
As aboveUtility Macros
This macro can be connected into the AVS/Express standard macros DataObject, DataObjectNoTexture and DataObjectLite as well as the other altered DataObject given in this utility library.Example
An example application EgViewLite is provided, which shows two viewers that allow this type in interactivity.
This example also demonstrates the use of DataObjects altered to have a set Datamap and how ViewersLite can be used.
Files
xp_comm_proj/viewlite/vl_macs.v contains all of the viewers (View3D_TrackObj, View2D_TrackObj, View_TrackObj) as well as all the DataObject macros (eg DO_GreyScale), DO_Dmap_Switch, DO_NT_Dmap_Switch, DO_LITE_Dmap_Switch, View3DMinimal, View3D_TrackAll3, View2DMinimal, View2D_TrackAll3, ViewMinimal, View_TrackAll3 and an example EgViewLite.Other Notes
The macros can be acessed through the XP_COMM_PROJ library in the Templates library or through one of four libraries DataObjects_apps, Views3DLite, Views2DLite and UViewsLite all of which are in the Misc sub-library IAC library area.
The example EgViewLite can also be acessed through the IAC library but is located in the Examples sub-library.
Authors
J Leng Manchester Visualization CentreModifications
Paul G. LeverSee Also
DataObject, the DataObjects "DO_", "DO_NT_" and "DO_LITE_", the DataObjects "_Dmap_Switch", and the viewers lite 3D, 2D and Universal "Minimal", "_TrackObj" and "_TrackAll3".


Methods with too much statement nesting may be confusing, and are a good sign that refactoring may be necessary.
Use the Maximum nesting depth field to specify the maximum acceptable nesting depth a method might have.


Darren Randolph fit for Euro 2020 qualifier showdown with Denmark
by Will O'CallaghanDarren Randolph is fit for the Republic of Ireland's must-win clash with Denmark on Monday.
The Middlesbrough 'keeper trained with the squad in Dublin today after shaking off a thigh issue.
"I said we'd wrap Darren Randolph in cotton wool ahead of the Denmark match and we’ve almost been true to my word," manager Mick McCarthy told FAI.ie after training.
"He wasn’t involved for the New Zealand win on Thursday night and he’s done most of his work in the gym this week so it was great to see him working on the pitch today.
"Like everyone else in the squad, Darren is fit and raring to go and he can’t wait for the game now."
Denmark arrive in Ireland on the back of a 6-0 win over Gibraltar last night while Switzerland picked up a hard-fought 1-0 victory at home to Georgia. It means the Republic of Ireland need to a victory against the Danes to secure their passage to the finals.
"The results on Friday were exactly as expected but with Switzerland-Georgia being such a close game," McCarthy added.
"I did cheer that late Switzerland goal as it means a one-goal win will do us on Monday.
"Denmark will be happy with their result but we knew before their game that we'd have to beat them and that’s still the case. Nothing has changed."


Salary Calculator
Add Employee:
Submit
Employees
First Name
Last Name
ID
Title
Annual Salary
Delete
Person
Name
1234
Example
$80,000
Delete
*Monthly budget should not exceed $20000
Total Monthly:


Train to teach mindfulness with the NHS-based Sussex Mindfulness Centre
Train to teach Mindfulness Based Cognitive Therapy (MBCT) and Mindfulness Based Stress Reduction (MBSR) with the Sussex Mindfulness Centre. You'll be learning at one of the leading mindfulness centres in the UK and one of the few mindfulness teacher training and research centres embedded in the NHS.
The teacher training is open to everybody whether or not you have a clinical background. And if you work in the NHS there are opportunities to apply for NHS Trust funded places.
Alternatively we offer training in delivering Adapted Mindfulness-Based Interventions for people who wish to teach adapted mindfulness based approaches safely and effectively, but who don't need to complete the extensive training required to deliver Mindfulness Based Cognitive Therapy and Mindfulness Based Stress Reduction.
These two NHS-based training courses are part-time and run from February to December. The applications for the 2024 courses are open now.
Meet the teachers and find out more
Attend one of our teacher training information sessions. The next one/s are publicised here:
- Monday 3 July 10 - 11am
Register to attend one of these sessions by emailing<EMAIL_ADDRESS>You can also find out more about the MBCT/MBSR teacher training here and the adapted training here.
Why train to teach with us?
The Sussex Mindfulness Centre is one of the leading UK mindfulness centres and have been running mindfulness teacher training since 2011. We have 60 members of staff trained in mindfulness-based approaches and a thriving team of trainers and researchers working together. We are also part of the national training centres collaboration, which with our partners at Exeter, Nottingham and Tees, Esk and Wear Valleys NHS Foundation Trust, delivers MBCT training in IAPT services. Sussex Mindfulness Centre is a member of the British Association of Mindfulness-Based Approaches (BAMBA) which recognises training programmes that meet agreed national minimum standards.
For further information about training, please contact<EMAIL_ADDRESS>or call our Sussex Mindfulness Centre Administrator on 0300 304 0689. You can also stay up to date with our training courses by signing up to our newsletter.


category create
category delete
testcase create
testcase delete
category create
category delete
testcase create
testcase delete


Introduction
I am a self-taught Developer, with almost 3 years experience developing. I am currently focusing on Front-End Development and aspiring to be a Full-Stack Developer. This website contains information about me, my resume, portfolio, stories, and ways to stay in touch or contact me.
important_devices
Web
My love for Web Development started when I was working at the Co-op Group. I have been hooked since then and I love adding to the World Wide Web.
palette
Design
I have been picking up some design skills. I am skilled at using Adobe Photoshop, XD, Draw and Acrobat, and I create most of the assets for my projects.
android
Android
I graduated from Android Basic Nanodegree by Google via Udacity in July 2017, thanks to a Google scholarship I received in January 2017.


|
QuantLib: a free/open-source library for quantitative finance
Fully annotated sources - version 1.30
|
Generic multifactor latent variable model. More...
#include <ql/experimental/math/latentmodel.hpp>
Classes | |
| class | FactorSampler |
| class | IntegrationFactory |
Public Member Functions | |
Latent model direct integration facility. | |
| void | update () override |
Latent model random factor number generator facility. | |
| Size | size () const |
| Size | numFactors () const |
| Number of systemic factors. More... | |
| Size | numTotalFactors () const |
| Number of total free random factors; systemic and idiosyncratic. More... | |
| LatentModel (const std::vector< std::vector< Real > > &factorsWeights, const typename copulaType::initTraits &ini=copulaType::initTraits()) | |
| LatentModel (const std::vector< Real > &factorsWeight, const typename copulaType::initTraits &ini=copulaType::initTraits()) | |
| LatentModel (Real correlSqr, Size nVariables, const typename copulaType::initTraits &ini=copulaType::initTraits()) | |
| LatentModel (const Handle< Quote > &singleFactorCorrel, Size nVariables, const typename copulaType::initTraits &ini=copulaType::initTraits()) | |
| const std::vector< std::vector< Real > > & | factorWeights () const |
| Provides values of the factors \( a_{i,k} \). More... | |
| const std::vector< Real > & | idiosyncFctrs () const |
| Provides values of the normalized idiosyncratic factors \( Z_i \). More... | |
| Real | latentVariableCorrel (Size iVar1, Size iVar2) const |
| Latent variable correlations: More... | |
| Public Member Functions inherited from Observer | |
| Observer ()=default | |
| Observer (const Observer &) | |
| Observer & | operator= (const Observer &) |
| virtual | ~Observer () |
| std::pair< iterator, bool > | registerWith (const ext::shared_ptr< Observable > &) |
| void | registerWithObservables (const ext::shared_ptr< Observer > &) |
| Size | unregisterWith (const ext::shared_ptr< Observable > &) |
| void | unregisterWithAll () |
| virtual void | update ()=0 |
| virtual void | deepUpdate () |
| Public Member Functions inherited from Observable | |
| Observable () | |
| Observable (const Observable &) | |
| Observable & | operator= (const Observable &) |
| virtual | ~Observable ()=default |
| void | notifyObservers () |
Copula interface. | |
| typedef copulaPolicyImpl | copulaType |
| Probability | cumulativeY (Real val, Size iVariable) const |
| Probability | cumulativeZ (Real z) const |
| Cumulative distribution of Z, the idiosyncratic/error factors. More... | |
| Probability | density (const std::vector< Real > &m) const |
| Density function of M, the market/systemic factors. More... | |
| Real | inverseCumulativeDensity (Probability p, Size iFactor) const |
| Inverse cumulative distribution of the systemic factor iFactor. More... | |
| Real | inverseCumulativeY (Probability p, Size iVariable) const |
| Real | inverseCumulativeZ (Probability p) const |
| std::vector< Real > | allFactorCumulInverter (const std::vector< Real > &probs) const |
| Real | latentVarValue (const std::vector< Real > &allFactors, Size iVar) const |
| const copulaType & | copula () const |
Integration facility interface | |
| std::vector< std::vector< Real > > | factorWeights_ |
| Handle< Quote > | cachedMktFactor_ |
| std::vector< Real > | idiosyncFctrs_ |
| Size | nFactors_ |
| Number of systemic factors. More... | |
| Size | nVariables_ |
| Number of latent model variables, idiosyncratic terms or model dim. More... | |
| copulaType | copula_ |
| Real | integratedExpectedValue (const ext::function< Real(const std::vector< Real > &v1)> &f) const |
| std::vector< Real > | integratedExpectedValueV (const ext::function< std::vector< Real >(const std::vector< Real > &v1)> &f) const |
| virtual const ext::shared_ptr< LMIntegration > & | integration () const |
Additional Inherited Members | |
| Public Types inherited from Observer | |
| typedef set_type::iterator | iterator |
Generic multifactor latent variable model.
\[ \begin{array}{ccccc} Y_1 & = & \sum_k M_k a_{1,k} & + \sqrt{1-\sum_k a_{1,k}^2} Z_1 & \sim \Phi_{Y_1}\nonumber \\ ... & = & ... & ... & \nonumber \\ Y_i & = & \sum_k M_k a_{i,k} & + \sqrt{1-\sum_k a_{i,k}^2} Z_i & \sim \Phi_{Y_i}\nonumber \\ ... & = & ... & ... & \nonumber \\ Y_N & = & \sum_k M_k a_{N,k} & + \sqrt{1-\sum_k a_{N,k}^2} Z_N & \sim \Phi_{Y_N} \end{array} \]
where the systemic \( M_k \) and idiosyncratic \( Z_i \) (this last one known as error term in some contexts) random variables have independent zero-mean unit-variance distributions. A restriction of the model implemented here is that the N idiosyncratic variables all follow the same probability law \( \Phi_Z(z)\) (but they are still independent random variables) Also the model is normalized so that: \(-1\leq a_{i,k} \leq 1\) (technically the \(Y_i\) are convex linear combinations). The correlation between \(Y_i\) and \(Y_j\) is then \(\sum_k a_{i,k} a_{j,k}\). \(\Phi_{Y_i}\) denotes the cumulative distribution function of \(Y_i\) which in general differs for each latent variable.\[ \begin{array}{ccc} F_i(Y_i) & = & F_i(\sum_k M_k a_{i,k} + \sqrt{1-\sum_k a_{i,k}^2} Z_i )\nonumber \\ & = & F_i(M_1,..., M_k, ..., M_K, Z_i) \end{array} \]
The transfer function can have a more generic form: \(F_i(Y_1,....,Y_N)\) but here the model is restricted to a one to one relation between the latent variables and the modelled ones. Also it is assumed that \(F_i(y_i; \tau)\) is monotonic in \(y_i\); it can then be inverted and the relation of the cumulative probability of \(F_i\) and \(Y_i\) is simple:\[ \int_{\infty}^b \phi_{F_i} df = \int_{\infty}^{F_i^{-1}(b)} \phi_{Y_i} dy \]
If \(t\) is some value of the functional or modelled variable, \(y\) is mapped to \(t\) such that percentiles match, i.e. \(F_Y(y)=Q_i(t)\) or \(y=F_Y^{-1}(Q_i(t))\). The class provides an integration facility of arbitrary functions dependent on the model states. It also provides random number generation interfaces for usage of the model in monte carlo simulations.\[ Prob \,(Y_i < y|M_k) = \Phi_Z \left( \frac{y-\sum_k a_{i,k}\,M_k} {\sqrt{1-\sum_k a_{i,k}^2}}\right) \qquad \mbox{or} \qquad Prob \,(t_i < t|M) = \Phi_Z \left( \frac {F_{Y_{i}}^{-1}(Q_i(t))-\sum_k a_{i,k}\,M_k} {\sqrt{1-\sum_k a_{i,k}^2}} \right) \]
The distribution functions of \( M_k, Z_i \) are specified in specific copula template classes. The distribution function of \( Y_i \) is then given by the convolution\[ F_{Y_{i}}(y) = Prob\,(Y_i<y) = \int_{-\infty}^\infty\,\cdots\,\int_{-\infty}^{\infty}\: D_Z(z)\,\prod_k D_{M_{k}}(m_k) \quad \Theta \left(y - \sum_k a_{i,k}m_k - \sqrt{1-\sum_k a_{i,k}^2}\,z\right)\,d\bar{m}\,dz, \qquad \Theta (x) = \left\{ \begin{array}{ll} 1 & x \geq 0 \\ 0 & x < 0 \end{array}\right. \]
where \( D_Z(z) \) and \( D_M(m) \) are the probability densities of \( Z\) and \( M, \) respectively.\[ F_{Y_{i}}(y) = Prob \,(Y_i < y) = \int_{-\infty}^\infty\,\cdots\,\int_{-\infty}^{\infty} D_{M_{k}}(m_k)\,dm_k\: \int_{-\infty}^{g(y,\vec{a},\vec{m})} D_Z(z)\,dz, \qquad g(y,\vec{a},\vec{m}) = \frac{y - \sum_k a_{i,k}m_k} {\sqrt{1-\sum_k a_{i,k}^2}}, \qquad \sum_k a_{i,k}^2 < 1 \]
In general, \( F_{Y_{i}}(y) \) needs to be computed numerically.To make direct integration and simulation time efficient virtual functions have been avoided in accessing methods in the copula policy and in the sampling of the random factors
Definition at line 284 of file latentmodel.hpp.
| typedef copulaPolicyImpl copulaType |
Definition at line 291 of file latentmodel.hpp.
|
explicit |
Constructs a LM with an arbitrary number of latent variables and factors given by the dimensions of the passed matrix.
| factorsWeights | Ordering is factorWeights_[iVar][iFactor] |
| ini | Initialization variables. Trait type from the copula policy to allow for static policies (this solution needs to be revised, possibly drop the static policy and create a policy member in LatentModel) |
|
explicit |
Constructs a LM with an arbitrary number of latent variables depending only on one random factor but contributing to each latent variable through different weights.
| factorsWeight | Ordering is factorWeights_[iVariable] |
| ini | Initialization variables. Trait type from the copula policy to allow for static policies (this solution needs to be revised, possibly drop the static policy and create a policy member in LatentModel) |
|
explicit |
Constructs a LM with an arbitrary number of latent variables depending only on one random factor with the same weight for all latent variables.
correlSqr is the weight, same for all.
ini is a trait type from the copula policy, to allow for static policies (this solution needs to be revised, possibly drop the static policy and create a policy member in LatentModel)
|
explicit |
Constructs a LM with an arbitrary number of latent variables depending only on one random factor with the same weight for all latent variables. The weight is observed and this constructor is intended to be used when the model relates to a market value.
singleFactorCorrel is the weight/mkt-factor, same for all.
ini is a trait type from the copula policy, to allow for static policies (this solution needs to be revised, possibly drop the static policy and create a policy member in LatentModel)
|
overridevirtual |
This method must be implemented in derived classes. An instance of Observer does not call this method directly: instead, it will be called by the observables the instance registered with when they need to notify any changes.
Implements Observer.
Definition at line 716 of file latentmodel.hpp.
| Probability cumulativeY | ( | Real | val, |
| Size | iVariable | ||
| ) | const |
Cumulative probability of the \( Y_i \) modelled latent random variable to take a given value.
Definition at line 295 of file latentmodel.hpp.
| Probability cumulativeZ | ( | Real | z | ) | const |
Cumulative distribution of Z, the idiosyncratic/error factors.
Definition at line 299 of file latentmodel.hpp.
| Probability density | ( | const std::vector< Real > & | m | ) | const |
Density function of M, the market/systemic factors.
Definition at line 303 of file latentmodel.hpp.
| Real inverseCumulativeDensity | ( | Probability | p, |
| Size | iFactor | ||
| ) | const |
Inverse cumulative distribution of the systemic factor iFactor.
Definition at line 311 of file latentmodel.hpp.
| Real inverseCumulativeY | ( | Probability | p, |
| Size | iVariable | ||
| ) | const |
Inverse cumulative value of the i-th random latent variable with a given probability.
Definition at line 316 of file latentmodel.hpp.
| Real inverseCumulativeZ | ( | Probability | p | ) | const |
Inverse cumulative value of the idiosyncratic variable with a given probability.
Definition at line 321 of file latentmodel.hpp.
All factor cumulative inversion. Used in integrations and sampling. Inverts all the cumulative random factors probabilities in the model. These are all the systemic factors plus all the idiosyncratic ones, so the size of the inversion is the number of systemic factors plus the number of latent modelled variables
Definition at line 329 of file latentmodel.hpp.
The value of the latent variable Y_i conditional to (given) a set of values of the factors.
The passed allFactors vector contains values for all the independent factors in the model (systemic and idiosyncratic, in that order). A full sample is required, i.e. all the idiosyncratic values are expected to be present even if only the relevant one is used.
Definition at line 343 of file latentmodel.hpp.
| const copulaType & copula | ( | ) | const |
Definition at line 354 of file latentmodel.hpp.
| Size size | ( | ) | const |
Definition at line 504 of file latentmodel.hpp.
| Size numFactors | ( | ) | const |
Number of systemic factors.
Definition at line 506 of file latentmodel.hpp.
| Size numTotalFactors | ( | ) | const |
Number of total free random factors; systemic and idiosyncratic.
Definition at line 508 of file latentmodel.hpp.
| const std::vector< std::vector< Real > > & factorWeights | ( | ) | const |
Provides values of the factors \( a_{i,k} \).
Definition at line 566 of file latentmodel.hpp.
| const std::vector< Real > & idiosyncFctrs | ( | ) | const |
Provides values of the normalized idiosyncratic factors \( Z_i \).
Definition at line 570 of file latentmodel.hpp.
Latent variable correlations:
Definition at line 573 of file latentmodel.hpp.
Integrates an arbitrary scalar function over the density domain(i.e. computes its expected value).
Definition at line 586 of file latentmodel.hpp.
| std::vector< Real > integratedExpectedValueV | ( | const ext::function< std::vector< Real >(const std::vector< Real > &v1)> & | f | ) | const |
Integrates an arbitrary vector function over the density domain(i.e. computes its expected value).
Definition at line 596 of file latentmodel.hpp.
|
protectedvirtual |
Reimplemented in DefaultLatentModel< copulaPolicy >, and SpotRecoveryLatentModel< copulaPolicy >.
Definition at line 608 of file latentmodel.hpp.
|
mutableprotected |
Definition at line 614 of file latentmodel.hpp.
Definition at line 628 of file latentmodel.hpp.
|
mutableprotected |
Definition at line 633 of file latentmodel.hpp.
|
mutableprotected |
Number of systemic factors.
Definition at line 635 of file latentmodel.hpp.
|
mutableprotected |
Number of latent model variables, idiosyncratic terms or model dim.
Definition at line 637 of file latentmodel.hpp.
|
mutableprotected |
Definition at line 639 of file latentmodel.hpp.


Sand Carving is a technique that uses high pressure abrasive to etch glass, crystal, marble stone or other surfaces. The process is also more commonly called sandblasting. We use a sand-like material called aluminum oxide as our abrasive in class. The process is relatively simple to follow. First you will create a image file using either CorelDraw or Illustrator. This is then printed onto special sheets of AccuBlack sheets to create the template. This is then exposed using a sheet of RapidMask film in the Lectralite unit. This will transfer what you have printed on the AccuBlack sheet onto the RapidMask. this usually takes around 2 1/2 minutes of exposure time. You then need to apply the mask to the object being sandcarved. Once the mask is applied, you will need to place the object into the CrystalBlast Elite cabinet to apply the abbrasive to the object.
Using the machines and knowledge you gained from your DigFab course, design a prototype to be sold in the business we visited on Tuesday. The item needs to be within the price range set out.
1. Design Challenge
2. Sand Carving
3. Product Creation Challenge $50


Datasets¶
The datasets can be downloaded from the following links.
Training and Validation¶
Sub-Task Training Validation Other 1 None 2 Ubuntu_st2_train Ubuntu_st2_validation Candidate_pool 3 Advising_st3_train Advising_st3_validation None 4 None 5 Same as subtask 1 Same as subtask 1
Additionally, for the Advising data, we are providing a form of the data with the original dialogs and their paraphrases before remixing. This can be used for training in any subtask, and can be downloaded here. The global candidate pool for the sub-task 2, should be shared across training, validation and test datasets for sub-task 2.
Test¶
Sub-Task Test 1 2 Ubuntu_st2_test 3 4 5 Same as subtask 1
Ground truth for test datasets¶
Sub-Task Test 1 2 Ubuntu_st2_ground_truth 3 4 5 Same as subtask 1


Brandon Tyrone Marshall is an American football wide receiver for the New York Giants of the National Football League (NFL).
He played college football at UCF, and was drafted by the Denver Broncos in the fourth round of the 2006 NFL Draft. Marshall has also played for the Miami Dolphins,
Chicago Bears and New York Jets. Marshall is known for his ability to break and dodge tackles. He led all NFL wide receivers in yards after first contact
for the 2007 NFL season. Regarding Marshall's breakaway ability, cornerback Brandon Flowers said, "Brandon Marshall is a defensive lineman playing wide
receiver. He wants to inflict punishment on you.
Cornerback Nnamdi Asomugha said Marshall is 'the toughest guy to bring down, one-on-one.'


Welcome to PyMethylProcess’s documentation!¶
https://github.com/Christensen-Lab-Dartmouth/PyMethylProcess
To get started, download pymethylprocess using Docker (joshualevy44/pymethylprocess) or PIP (pymethylprocess) and run pymethyl-install_r_dependencies.
There is both an API and CLI available for use. Examples for CLI usage can be found in ./example_scripts.
PreProcessDataTypes.py¶
Contains datatypes core to downloading IDATs, preprocessing IDATs and samplesheets.
-
class
pymethylprocess.PreProcessDataTypes.
PreProcessIDAT
(idat_dir, minfi=None, enmix=None, base=None, meffil=None)[source]¶ Class that will preprocess IDATs using R pipelines.
- idat_dir
- Location of idats or samplesheet csv.
- minfi
- Rpy2 importr minfi library, default to None will load through rpy2
- enmix
- Rpy2 importr enmix library, default to None will load through rpy2
- base
- Rpy2 importr base library, default to None will load through rpy2
- meffil
- Rpy2 importr meffil library, default to None will load through rpy2
-
export_csv
(output_dir)[source]¶ Export pheno and beta dataframes to CSVs
- output_dir
- Where to store csvs.
-
export_pickle
(output_pickle, disease='')[source]¶ Export pheno and beta dataframes to pickle, stored in python dict that can be loaded into MethylationArray
- output_pickle
- Where to store MethylationArray.
- disease
- Custom naming scheme for data.
-
export_sql
(output_db, disease='')[source]¶ Export pheno and beta dataframes to SQL
- output_db
- Where to store data, sqlite db.
- disease
- Custom naming scheme for data.
-
extract_pheno_data
(methylset=False)[source]¶ Extract pheno data from MSet or RGSet, minfi.
- methylset
- If MSet has beenn created, set to True, else extract from original RGSet.
-
output_pheno_beta
(meffil=False)[source]¶ Get pheno and beta dataframe objects stored as attributes for input to MethylationArray object.
- meffil
- True if ran meffil pipeline.
-
plot_original_qc
(output_dir)[source]¶ Plot QC results from ENmix pipeline and possible minfi. Still experimental.
- output_dir
- Where to store plots.
-
plot_qc_metrics
(output_dir)[source]¶ Plot QC results from ENmix pipeline and possible minfi. Still experimental.
- output_dir
- Where to store plots.
-
preprocessENmix
(n_cores=6)[source]¶ Run ENmix preprocessing pipeline.
- n_cores
- Number of CPUs to use.
-
preprocessMeffil
(n_cores=6, n_pcs=4, qc_report_fname='qc/report.html', normalization_report_fname='norm/report.html', pc_plot_fname='qc/pc_plot.pdf', useCache=True, qc_only=True, qc_parameters={'p.beadnum.cpgs': 0.1, 'p.beadnum.samples': 0.1, 'p.detection.cpgs': 0.1, 'p.detection.samples': 0.1}, rm_sex=False)[source]¶ Run meffil preprocessing pipeline with functional normalization.
- n_cores
- Number of CPUs to use.
- n_pcs
- Number of principal components to use for functional normalization, set to -1 to autoselect via kneedle algorithm.
- qc_report_fname
- HTML filename to store QC report.
- normalization_report_fname
- HTML filename to store normalization report
- pc_plot_fname
- PDF file to store principal components plot.
- useCache
- Use saved QC objects instead of running through QC again.
- qc_only
- Perform QC, then save and quit before normalization.
- qc_parameters
- Python dictionary with parameters for qc.
- rm_sex
- Remove non-autosomal cpgs?
-
preprocess_enmix_pipeline
(n_cores=6, pipeline='enmix', noob=False, qc_only=False, use_cache=False)[source]¶ Run complete ENmix or minfi preprocessing pipeline.
- n_cores
- Number CPUs.
- pipeline
- Run enmix or minfi
- noob
- Noob norm or RAW if minfi running.
- qc_only
- Save and quit after only running QC?
- use_cache
- Load preexisting RGSet instead of running QC again.
-
class
pymethylprocess.PreProcessDataTypes.
PreProcessPhenoData
(pheno_sheet, idat_dir, header_line=0)[source]¶ Class that will manipute phenotype samplesheet before preprocessing of IDATs.
- pheno_sheet
- Location of clinical info csv.
- idat_dir
- Location of idats
- header_line
- Where to start reading clinical csv
-
concat
(other_formatted_sheet)[source]¶ Concat multiple PreProcessPhenoData objects, concat their dataframes to accept more than one smaplesheet/dataset.
- other_formatted_sheet
- Other PreProcessPhenoData to concat.
-
export
(output_sheet_name)[source]¶ Export pheno data to csv after done with manipulation.
- output_sheet_name
- Output csv name.
-
format_custom
(basename_col, disease_class_column, include_columns={})[source]¶ Custom format clinical sheet if user supplied idats.
- basename_col
- Column name of sample names.
- disease_class_column
- Disease column of clinical info csv.
- include_columns
- Dictionary specifying other columns to include, and new names to assign them to.
-
format_geo
(disease_class_column='methylation class:ch1', include_columns={})[source]¶ Format clinical sheets if downloaded geo idats.
- disease_class_column
- Disease column of clinical info csv.
- include_columns
- Dictionary specifying other columns to include, and new names to assign them to.
-
format_tcga
(mapping_file='idat_filename_case.txt')[source]¶ Format clinical sheets if downloaded tcga idats.
- mapping_file
- Maps uuids to proper tcga sample names, should be downloaded with tcga clinical information.
-
get_categorical_distribution
(key, disease_only=False, subtype_delimiter=', ')[source]¶ Print categorical distribution, counts for each unique value in phenotype column.
- key
- Phenotype Column.
- disease_only
- Whether to split phenotype column entries by delimiter.
- subtype_delimiter
- Subtype delimiter to split on.
-
merge
(other_formatted_sheet, use_second_sheet_disease=True, no_disease_merge=False)[source]¶ Merge multiple PreProcessPhenoData objects, merge their dataframes to accept more than one saplesheet/dataset or add more pheno info.
- other_formatted_sheet
- Other PreProcessPhenoData to merge.
- use_second_sheet_disease
- Change disease column to that of second sheet instead of first.
- no_disease_merge
- Keep both disease columns from both sheets.
-
remove_diseases
(exclude_disease_list, low_count, disease_only, subtype_delimiter)[source]¶ Remove samples with certain diseases from disease column.
- exclude_disease_list
- List containing diseases to remove.
- low_count
- Remove samples that have less than x disease occurances in column.
- disease_only
- Whether to split phenotype column entries by delimiter.
- subtype_delimiter
- Subtype delimiter to split on.
-
class
pymethylprocess.PreProcessDataTypes.
TCGADownloader
[source]¶ Downloads TCGA and GEO IDAT and clinical data
-
download_clinical
(output_dir)[source]¶ Download TCGA Clinical Data.
- output_dir
- Where to output clinical data csv.
-
MethylationDataTypes.py¶
Contains datatypes core to storing beta and phenotype methylation data, and imputation.
-
class
pymethylprocess.MethylationDataTypes.
ImputerObject
(solver, method, opts={})[source]¶ Class that stores and accesses different types of imputers. Construct sklearn-like imputer given certain input arguments.
- solver
- Library for imputation, eg. sklearn, fancyimpute.
- method
- Imputation method in library, named.
- opts
- Additional options to assign to imputer.
-
class
pymethylprocess.MethylationDataTypes.
MethylationArray
(pheno_df, beta_df, name='')[source]¶ Stores beta and phenotype information and performs various operations. Initialize MethylationArray object by inputting dataframe of phenotypes and dataframe of beta values with samples as index.
- pheno_df
- Phenotype dataframe (samples x covariates)
- beta_df
- Beta Values Dataframe (samples x cpgs)
-
bin_column
(col, n_bins)[source]¶ Turn continuous variable/covariate into categorical bins. Returns name of new column and updates phenotype matrix to reflect this change.
- col
- Continuous column of phenotype array to bin.
- n_bins
- Number of bins to create.
-
categorical_breakdown
(key)[source]¶ Print categorical distribution, counts for each unique value in phenotype column.
- key
- Phenotype Column.
-
feature_select
(n_top_cpgs, feature_selection_method='mad', metric='correlation', nn=10)[source]¶ Perform unsupervised feature selection on MethylationArray.
- n_top_cpgs
- Number of CpGs to retain.
- feature_selection_method
- Method to perform selection.
- metric
- If considering structural feature selection like SPEC, use this distance metric.
- nn
- Number of nearest neighbors.
-
classmethod
from_pickle
(input_pickle)[source]¶ Load MethylationArray stored in pickle.
Usage: MethylationArray.from_pickle([input_pickle])
- input_pickle
- Stored MethylationArray pickle.
-
groupby
(key)[source]¶ Groupby for Methylation Array. Returns generator of methylation arrays grouped by key.
- preprocess_sample_df
- New phenotype dataframe.
-
impute
(imputer)[source]¶ Perform imputation on NaN beta vaues. Input imputater returned from ImputerObject.
- imputer
- Type of imputer object, in sklearn type interface.
-
merge_preprocess_sheet
(preprocess_sample_df)[source]¶ Feed in another phenotype dataframe that will be merged with existing phenotype array.
- preprocess_sample_df
- New phenotype dataframe.
-
overwrite_pheno_data
(preprocess_sample_df)[source]¶ Feed in another phenotype dataframe that will overwrite overlapping keys of existing phenotype array.
- preprocess_sample_df
- New phenotype dataframe.
-
remove_missingness
(cpg_threshold=None, sample_threshold=None)[source]¶ Remove samples and CpGs with certain level of missingness..
- cpg_threshold
- If more than fraction of Samples for this CpG are missing, remove cpg.
- sample_threshold
- If more than fraction of CpGs for this sample are missing, remove sample.
-
remove_na_samples
(outcome_cols)[source]¶ Remove samples of MethylationArray who have missing values in phenotype column.
- outcome_cols
- Phenotype columns, if any rows contain missing values, samples are removed.
-
split_by_subtype
(disease_only, subtype_delimiter)[source]¶ Split MethylationArray into generator of MethylationArrays by phenotype column. Much akin to groupby. Only splits from disease column.
- disease_only
- Consider disease superclass.
- subtype_delimiter
- How to break up disease column if using disease_only.
-
split_key
(key, subtype_delimiter)[source]¶ Manipulate an entire phenotype column, splitting each element up by some delimiter.
- key
- Phenotype column.
- subtype_delimiter
- How to break up strings in columns. S1,s2 -> S1 for instance.
-
split_train_test
(train_p=0.8, stratified=True, disease_only=False, key='disease', subtype_delimiter=', ', val_p=0.0)[source]¶ Split MethylationArray into training and test sets, with option to stratify by categorical covariate.
- train_p
- Fraction of methylation array to use as training set.
- stratified
- Whether to stratify by categorical variable.
- disease_only
- Consider disease superclass by some delimiter. For instance if disease is S1,s2, superclass would be S1.
- key
- Column to stratify on.
- subtype_delimiter
- How to split disease column into super/subclass.
- val_p
- If set greater than 0, will create additional validation set, fraction of which is broken off from training set.
-
subsample
(key='disease', n_samples=None, frac=None, categorical=False)[source]¶ Subsample MethylationArray, make the set randomly smaller.
- key
- If stratifying, use this column of pheno array.
- n_samples
- Number of samples to consider overall, or per stratum.
- frac
- Alternative to n_samples, where x frac of array or stratum is considered.
- categorical
- Whether to stratify by column.
-
subset_cpgs
(cpgs)[source]¶ Subset beta matrix by list of Cpgs. Parameters ———- cpgs
Cpgs to subset by.
-
write_csvs
(output_dir)[source]¶ Write phenotype data and beta values to csvs.
- output_dir
- Directory to output csv files.
-
class
pymethylprocess.MethylationDataTypes.
MethylationArrays
(list_methylation_arrays)[source]¶ Literally a list of methylation arrays, with methods operate on these arrays that is memory efficient. Initialize with list of methylation arrays. Can optionally leave list empty or with one element.
- list_methylation_arrays
- List of methylation arrays.
-
combine
(array_generator=None)[source]¶ Combine the list of methylation arrays into one array via concatenation of beta matrices and phenotype arrays.
- array_generator
- Generator of additional methylation arrays for computational memory minimization.
-
pymethylprocess.MethylationDataTypes.
extract_pheno_beta_df_from_folder
(folder)[source]¶ Return phenotype and beta dataframes from specified folder with csv.
- folder
- Input folder.
meffil_functions.py¶
Contains a few R functions that interact with meffil and minfi.
-
pymethylprocess.meffil_functions.
est_cell_counts_IDOL
(rgset, library)[source]¶ Given RGSet object, estimate cell counts for 450k/850k using reference approach via IDOL library.
- rgset
- RGSet object stored in python via rpy2
- library
- What type of CpG library to use.
-
pymethylprocess.meffil_functions.
est_cell_counts_meffil
(qc_list, cell_type_reference)[source]¶ Given QCObject list R object, estimate cell counts using reference approach via meffil.
- qc_list
- R list containing qc objects.
- cell_type_reference
- Reference blood/tissue set.
-
pymethylprocess.meffil_functions.
est_cell_counts_minfi
(rgset)[source]¶ Given RGSet object, estimate cell counts using reference approach via minfi.
- rgset
- RGSet object stored in python via rpy2
-
pymethylprocess.meffil_functions.
load_detection_p_values_beadnum
(qc_list, n_cores)[source]¶ Return list of detection p-value matrix and bead number matrix.
- qc_list
- R list containing qc objects.
- n_cores
- Number of cores to use in computation.
-
pymethylprocess.meffil_functions.
r_autosomal_cpgs
(array_type='450k')[source]¶ Return list of autosomal cpg probes per platform.
- array_type
- 450k/850k array?
-
pymethylprocess.meffil_functions.
r_snp_cpgs
(array_type='450k')[source]¶ Return list of SNP cpg probes per platform.
- array_type
- 450k/850k array?
-
pymethylprocess.meffil_functions.
remove_sex
(beta, array_type='450k')[source]¶ Remove non-autosomal cpgs from beta matrix.
- array_type
- 450k/850k array?
-
pymethylprocess.meffil_functions.
set_missing
(beta, pval_beadnum, detection_val=1e-06)[source]¶ Set missing beta values to NA, taking into account detection values and bead number thesholds.
- pval_beadnum
- Detection pvalues and number of beads per cpg/samples
- detection_val
- If threshold to set site to missingness based on p-value detection.
general_machine_learning.py¶
Contains a machine learning class to perform scikit-learn like operations, along with held-out hyperparameter grid search.
-
class
pymethylprocess.general_machine_learning.
MachineLearning
(model, options, grid={}, labelencode=False, n_eval=0)[source]¶ Machine learning class to run sklearn-like pipeline on MethylationArray data. Initialize object with scikit-learn model, and optionally supply a hyperparameter search grid.
- model
- Scikit-learn-like model, classification, regression, dimensionality reduction, clustering etc.
- options
- Options to supply model in form of dictionary.
- grid
- Alternatively, supply search grid to search for bets hyperparameters.
- labelencode
- T/F encode string labels.
- n_eval
- Number of evaluations for randomized grid search, if set to 0, perform exhaustive grid search
-
assign_results_to_pheno_col
(methyl_array, new_col, output_pkl)[source]¶ Assign results to new phenotype column.
- methyl_array
- MethylationArray.
- new_col
- New column name.
- output_pkl
- Output pickle to dump MethylationArray to.
-
fit
(train_methyl_array, val_methyl_array=None, outcome_cols=None)[source]¶ Fit data to model.
- train_methyl_array
- Training MethylationArray.
- val_methyl_array
- Validation MethylationArray. Can set to None.
- outcome_cols
- Set to none if not needed, but phenotype column to train on, can be multiple.
-
fit_predict
(train_methyl_array, outcome_cols=None)[source]¶ Fit and predict training data.
- train_methyl_array
- Training MethylationArray.
- outcome_cols
- Set to none if not needed, but phenotype column to train on, can be multiple.
-
fit_transform
(train_methyl_array, outcome_cols=None)[source]¶ Fit and transform to training data.
- train_methyl_array
- Training MethylationArray.
- outcome_cols
- Set to none if not needed, but phenotype column to train on, can be multiple.
-
predict
(test_methyl_array)[source]¶ Make new predictions on test methylation array.
- test_methyl_array
- Testing MethylationArray.
-
return_outcome_metric
(methyl_array, outcome_cols, metric, run_bootstrap=False)[source]¶ Supply metric to evaluate results.
- methyl_array
- MethylationArray to evaluate.
- outcome_cols
- Outcome phenotype columns.
- metric
- Sklearn evaluation metric.
- run_bootstrap
- Make 95% CI from 1k bootstraps.
-
store_results
(output_pkl, results_dict={})[source]¶ Store results in pickle file.
- output_pkl
- Output pickle to dump results to.
- results_dict
- Supply own results dict to be dumped.
pymethyl-install¶
pymethyl-install [OPTIONS] COMMAND [ARGS]...
Options
-
--version
¶
Show the version and exit.
change_gcc_path¶
Change GCC and G++ paths if don’t have version 7.2.0. [Experimental]
pymethyl-install change_gcc_path [OPTIONS]
install_custom¶
Installs bioconductor packages.
pymethyl-install install_custom [OPTIONS]
Options
-
-p
,
--package
<package>
¶ Custom packages. [default: ENmix]
-
-m
,
--manager
¶
Use BiocManager (recommended).
install_minfi_others¶
Installs minfi and other dependencies.
pymethyl-install install_minfi_others [OPTIONS]
install_r_packages¶
Installs r packages.
pymethyl-install install_r_packages [OPTIONS]
Options
-
-p
,
--package
<package>
¶ Custom packages. [default: ]
install_some_deps¶
Installs bioconductor, minfi, enmix, tcga biolinks, and meffil.
pymethyl-install install_some_deps [OPTIONS]
pymethyl-visualize¶
pymethyl-visualize [OPTIONS] COMMAND [ARGS]...
Options
-
--version
¶
Show the version and exit.
plot_cell_type_results¶
Plot csv containing cell type results into side by side boxplots.
pymethyl-visualize plot_cell_type_results [OPTIONS]
Options
-
-i
,
--input_csv
<input_csv>
¶ Input csv. [default: cell_type_estimates.csv]
-
-o
,
--outfilename
<outfilename>
¶ Output png. [default: visualizations/cell_type_results.png]
-
-cols
,
--plot_cols
<plot_cols>
¶ Plot columns. [default: Gran, CD4T, CD8T, Bcell, Mono, NK, gMDSC]
-
-fs
,
--font_scale
<font_scale>
¶ Font scaling [default: 1.0]
plot_heatmap¶
Plot heatmap from CSV file.
pymethyl-visualize plot_heatmap [OPTIONS]
Options
-
-i
,
--input_csv
<input_csv>
¶ Input csv. [default: ]
-
-o
,
--outfilename
<outfilename>
¶ Output png. [default: output.png]
-
-idx
,
--index_col
<index_col>
¶ Index load dataframe [default: 0]
-
-fs
,
--font_scale
<font_scale>
¶ Font scaling [default: 1.0]
-
-min
,
--min_val
<min_val>
¶ Min heat val [default: 0.0]
-
-max
,
--max_val
<max_val>
¶ Max heat val, if -1, defaults to None [default: 1.0]
-
-a
,
--annot
¶
Annotate heatmap [default: False]
-
-n
,
--norm
¶
Normalize matrix data [default: False]
-
-c
,
--cluster
¶
Cluster matrix data [default: False]
-
-m
,
--matrix_type
<matrix_type>
¶ Type of matrix supplied [default: none]
-
-x
,
--xticks
¶
Show x ticks [default: False]
-
-y
,
--yticks
¶
Show y ticks [default: False]
-
-t
,
--transpose
¶
Transpose matrix data [default: False]
-
-col
,
--color_column
<color_column>
¶ Color column. [default: color]
transform_plot¶
Dimensionality reduce VAE or original beta values using UMAP and plot using plotly.
pymethyl-visualize transform_plot [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-c
,
--column_of_interest
<column_of_interest>
¶ Column extract from phenotype data. [default: disease]
-
-o
,
--output_file
<output_file>
¶ Output visualization. [default: ./visualization.html]
-
-nn
,
--n_neighbors
<n_neighbors>
¶ Number of neighbors UMAP. [default: 5]
-
-a
,
--axes_off
¶
Whether to turn axes on or off.
-
-s
,
--supervised
¶
Supervise umap embedding.
-
-d
,
--min_dist
<min_dist>
¶ UMAP min distance. [default: 0.1]
-
-m
,
--metric
<metric>
¶ Reduction metric. [default: euclidean]
-
-cc
,
--case_control_override
¶
Add controls from case_control column and override current disease for classification tasks. [default: False]
pymethyl-preprocess¶
pymethyl-preprocess [OPTIONS] COMMAND [ARGS]...
Options
-
--version
¶
Show the version and exit.
batch_deploy_preprocess¶
Deploy multiple preprocessing jobs in series or parallel.
pymethyl-preprocess batch_deploy_preprocess [OPTIONS]
Options
-
-n
,
--n_cores
<n_cores>
¶ Number cores to use for preprocessing. [default: 6]
-
-i
,
--subtype_output_dir
<subtype_output_dir>
¶ Output subtypes pheno csv. [default: ./preprocess_outputs/]
-
-m
,
--meffil
¶
Preprocess using meffil.
-
-t
,
--torque
¶
Job submission torque.
-
-r
,
--run
¶
Actually run local job or just print out command.
-
-s
,
--series
¶
Run commands in series.
-
-p
,
--pc_qc_parameters_csv
<pc_qc_parameters_csv>
¶ For meffil, qc parameters and pcs for final qc and functional normalization. [default: ./preprocess_outputs/pc_qc_parameters.csv]
-
-u
,
--use_cache
¶
If this is selected, loads qc results rather than running qc again. Only works for meffil selection.
-
-qc
,
--qc_only
¶
Only perform QC for meffil pipeline, caches results into rds file for loading again, only works if use_cache is false.
-
-c
,
--chunk_size
<chunk_size>
¶ If not series, chunk up and run these number of commands at once.. -1 means all commands at once.
combine_methylation_arrays¶
If split MethylationArrays by subtype for either preprocessing or imputation, can use to recombine data for downstream step.
pymethyl-preprocess combine_methylation_arrays [OPTIONS]
Options
-
-i
,
--input_pkls
<input_pkls>
¶ Input pickles for beta and phenotype data. [default: ./preprocess_outputs/methyl_array.pkl]
-
-d
,
--optional_input_pkl_dir
<optional_input_pkl_dir>
¶ Auto grab input pkls. [default: ]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./combined_outputs/methyl_array.pkl]
-
-e
,
--exclude
<exclude>
¶ If -d selected, these diseases will be excluded from study. [default: ]
concat_sample_sheets¶
Concat two sample files for more fields for minfi+ input, adds more samples.
pymethyl-preprocess concat_sample_sheets [OPTIONS]
Options
-
-s1
,
--sample_sheet1
<sample_sheet1>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/clinical_info1.csv]
-
-s2
,
--sample_sheet2
<sample_sheet2>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/clinical_info2.csv]
-
-os
,
--output_sample_sheet
<output_sample_sheet>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
create_sample_sheet¶
Create sample sheet for input to minfi, meffil, or enmix.
pymethyl-preprocess create_sample_sheet [OPTIONS]
Options
-
-is
,
--input_sample_sheet
<input_sample_sheet>
¶ Clinical information downloaded from tcga/geo/custom. [default: ./tcga_idats/clinical_info.csv]
-
-s
,
--source_type
<source_type>
¶ Source type of data. [default: tcga]
-
-i
,
--idat_dir
<idat_dir>
¶ Idat directory. [default: ./tcga_idats/]
-
-os
,
--output_sample_sheet
<output_sample_sheet>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
-
-m
,
--mapping_file
<mapping_file>
¶ Mapping file from uuid to TCGA barcode. Downloaded using download_tcga. [default: ./idat_filename_case.txt]
-
-l
,
--header_line
<header_line>
¶ Line to begin reading csv/xlsx. [default: 0]
-
-d
,
--disease_class_column
<disease_class_column>
¶ Disease classification column, for custom and geo datasets. [default: methylation class:ch1]
-
-b
,
--basename_col
<basename_col>
¶ Basename classification column, for custom datasets. [default: Sentrix ID (.idat)]
-
-c
,
--include_columns_file
<include_columns_file>
¶ Custom columns file containing columns to keep, separated by n. Add a tab for each line if you wish to rename columns: original_name t new_column_name [default: ]
download_clinical¶
Download all TCGA 450k clinical info.
pymethyl-preprocess download_clinical [OPTIONS]
Options
-
-o
,
--output_dir
<output_dir>
¶ Output directory for exported idats. [default: ./tcga_idats/]
download_geo¶
Download geo methylation study idats and clinical info.
pymethyl-preprocess download_geo [OPTIONS]
Options
-
-g
,
--geo_query
<geo_query>
¶ GEO study to query. [default: ]
-
-o
,
--output_dir
<output_dir>
¶ Output directory for exported idats. [default: ./geo_idats/]
download_tcga¶
Download all tcga 450k data.
pymethyl-preprocess download_tcga [OPTIONS]
Options
-
-o
,
--output_dir
<output_dir>
¶ Output directory for exported idats. [default: ./tcga_idats/]
feature_select¶
Filter CpGs by taking x top CpGs with highest mean absolute deviation scores or via spectral feature selection.
pymethyl-preprocess feature_select [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./imputed_outputs/methyl_array.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-n
,
--n_top_cpgs
<n_top_cpgs>
¶ Number cpgs to include with highest variance across population. [default: 300000]
-
-f
,
--feature_selection_method
<feature_selection_method>
¶
-
-mm
,
--metric
<metric>
¶
-
-nn
,
--n_neighbors
<n_neighbors>
¶ Number neighbors for feature selection, default enacts rbf kernel. [default: 0]
-
-m
,
--mad_top_cpgs
<mad_top_cpgs>
¶ Number cpgs to apply mad filtering first before more sophisticated feature selection. If 0 or primary feature selection is mad, no mad pre-filtering. [default: 0]
get_categorical_distribution¶
Get categorical distribution of columns of sample sheet.
pymethyl-preprocess get_categorical_distribution [OPTIONS]
Options
-
-is
,
--formatted_sample_sheet
<formatted_sample_sheet>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/minfiSheet.csv]
-
-k
,
--key
<key>
¶ Column of csv to print statistics for. [default: disease]
-
-d
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]
imputation_pipeline¶
Imputation of subtype or no subtype using various imputation methods.
pymethyl-preprocess imputation_pipeline [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./combined_outputs/methyl_array.pkl]
-
-ss
,
--split_by_subtype
¶
Imputes CpGs by subtype before combining again.
-
-m
,
--method
<method>
¶ Method of imputation. [default: KNN]
-
-s
,
--solver
<solver>
¶ Imputation library. [default: fancyimpute]
-
-k
,
--n_neighbors
<n_neighbors>
¶ Number neighbors for imputation if using KNN. [default: 5]
-
-r
,
--orientation
<orientation>
¶ Impute CpGs or samples. [default: Samples]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./imputed_outputs/methyl_array.pkl]
-
-n
,
--n_top_cpgs
<n_top_cpgs>
¶ Number cpgs to include with highest variance across population. Greater than 0 allows for mad filtering during imputation to skip mad step. [default: 0]
-
-f
,
--feature_selection_method
<feature_selection_method>
¶
-
-mm
,
--metric
<metric>
¶
-
-nfs
,
--n_neighbors_fs
<n_neighbors_fs>
¶ Number neighbors for feature selection, default enacts rbf kernel. [default: 0]
-
-d
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]
-
-st
,
--sample_threshold
<sample_threshold>
¶ Value between 0 and 1 for NaN removal. If samples has sample_threshold proportion of cpgs missing, then remove sample. Set to -1 to not remove samples. [default: -1.0]
-
-ct
,
--cpg_threshold
<cpg_threshold>
¶ Value between 0 and 1 for NaN removal. If cpgs has cpg_threshold proportion of samples missing, then remove cpg. Set to -1 to not remove samples. [default: -1.0]
meffil_encode¶
Reformat file for meffil input.
pymethyl-preprocess meffil_encode [OPTIONS]
Options
-
-is
,
--input_sample_sheet
<input_sample_sheet>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
-
-os
,
--output_sample_sheet
<output_sample_sheet>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
merge_sample_sheets¶
Merge two sample files for more fields for minfi+ input.
pymethyl-preprocess merge_sample_sheets [OPTIONS]
Options
-
-s1
,
--sample_sheet1
<sample_sheet1>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/clinical_info1.csv]
-
-s2
,
--sample_sheet2
<sample_sheet2>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/clinical_info2.csv]
-
-os
,
--output_sample_sheet
<output_sample_sheet>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
-
-d
,
--second_sheet_disease
¶
Use second sheet’s disease column.
-
-nd
,
--no_disease_merge
¶
Don’t merge disease columns.
na_report¶
Print proportion of missing values throughout dataset.
pymethyl-preprocess na_report [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./preprocess_outputs/methyl_array.pkl]
-
-o
,
--output_dir
<output_dir>
¶ Output database for na report. [default: ./na_report/]
-
-r
,
--head_directory
¶
-i option becomes directory, and searches there for multiple input pickles.
preprocess_pipeline¶
Perform preprocessing of idats using enmix or meffil.
pymethyl-preprocess preprocess_pipeline [OPTIONS]
Options
-
-i
,
--idat_dir
<idat_dir>
¶ Idat dir for one sample sheet, alternatively can be your phenotype sample sheet. [default: ./tcga_idats/]
-
-n
,
--n_cores
<n_cores>
¶ Number cores to use for preprocessing. [default: 6]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./preprocess_outputs/methyl_array.pkl]
-
-m
,
--meffil
¶
Preprocess using meffil.
-
-pc
,
--n_pcs
<n_pcs>
¶ For meffil, number of principal components for functional normalization. If set to -1, then PCs are selected using elbow method. [default: -1]
-
-p
,
--pipeline
<pipeline>
¶ If not meffil, preprocess using minfi or enmix. [default: enmix]
-
-noob
,
--noob_norm
¶
Run noob normalization of minfi selected.
-
-u
,
--use_cache
¶
If this is selected, loads qc results rather than running qc again and update with new qc parameters. Only works for meffil selection. Minfi and enmix just loads RG Set.
-
-qc
,
--qc_only
¶
Only perform QC for meffil pipeline, caches results into rds file for loading again, only works if use_cache is false. Minfi and enmix just saves the RGSet before preprocessing.
-
-bns
,
--p_beadnum_samples
<p_beadnum_samples>
¶ From meffil documentation, “fraction of probes that failed the threshold of 3 beads”. [default: 0.05]
-
-pds
,
--p_detection_samples
<p_detection_samples>
¶ From meffil documentation, “fraction of probes that failed a detection.pvalue threshold of 0.01”. [default: 0.05]
-
-bnc
,
--p_beadnum_cpgs
<p_beadnum_cpgs>
¶ From meffil documentation, “fraction of samples that failed the threshold of 3 beads”. [default: 0.05]
-
-pdc
,
--p_detection_cpgs
<p_detection_cpgs>
¶ From meffil documentation, “fraction of samples that failed a detection.pvalue threshold of 0.01”. [default: 0.05]
-
-sc
,
--sex_cutoff
<sex_cutoff>
¶ From meffil documentation, “difference of total median intensity for Y chromosome probes and X chromosome probes”. [default: -2]
-
-sd
,
--sex_sd
<sex_sd>
¶ From meffil documentation, “sex detection outliers if outside this range”. [default: 5]
remove_diseases¶
Exclude diseases from study by count number or exclusion list.
pymethyl-preprocess remove_diseases [OPTIONS]
Options
-
-is
,
--formatted_sample_sheet
<formatted_sample_sheet>
¶ Clinical information downloaded from tcga/geo/custom, formatted using create_sample_sheet. [default: ./tcga_idats/clinical_info.csv]
-
-e
,
--exclude_disease_list
<exclude_disease_list>
¶ List of conditions to exclude, from disease column, comma delimited. [default: ]
-
-os
,
--output_sheet_name
<output_sheet_name>
¶ CSV for minfi input. [default: ./tcga_idats/minfiSheet.csv]
-
-l
,
--low_count
<low_count>
¶ Remove diseases if they are below a certain count, default this is not used. [default: 0]
-
-d
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]
split_preprocess_input_by_subtype¶
Split preprocess input samplesheet by disease subtype.
pymethyl-preprocess split_preprocess_input_by_subtype [OPTIONS]
Options
-
-i
,
--idat_csv
<idat_csv>
¶ Idat csv for one sample sheet, alternatively can be your phenotype sample sheet. [default: ./tcga_idats/minfiSheet.csv]
-
-d
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]
-
-o
,
--subtype_output_dir
<subtype_output_dir>
¶ Output subtypes pheno csv. [default: ./preprocess_outputs/]
pymethyl-utils¶
pymethyl-utils [OPTIONS] COMMAND [ARGS]...
Options
-
--version
¶
Show the version and exit.
backup_pkl¶
Copy methylarray pickle to new location to backup.
pymethyl-utils backup_pkl [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./backup/methyl_array.pkl]
bin_column¶
Convert continuous phenotype column into categorical by binning.
pymethyl-utils bin_column [OPTIONS]
Options
-
-t
,
--test_pkl
<test_pkl>
¶ Pickle containing testing set. [default: ./train_val_test_sets/test_methyl_array.pkl]
-
-c
,
--col
<col>
¶ Column to turn into bins. [default: age]
-
-n
,
--n_bins
<n_bins>
¶ Number of bins. [default: 10]
-
-ot
,
--output_test_pkl
<output_test_pkl>
¶ Binned shap pickle for further testing. [default: ./train_val_test_sets/test_methyl_array_shap_binned.pkl]
concat_csv¶
Concatenate two csv files together.
pymethyl-utils concat_csv [OPTIONS]
Options
-
-i1
,
--input_csv
<input_csv>
¶ Beta csv. [default: ./beta1.csv]
-
-i2
,
--input_csv2
<input_csv2>
¶ Beta/other csv 2. [default: ./cell_estimates.csv]
-
-o
,
--output_csv
<output_csv>
¶ Output csv. [default: ./beta.concat.csv]
-
-a
,
--axis
<axis>
¶ Axis to merge on. Columns are 0, rows are 1. [default: 1]
counts¶
Return categorical breakdown of phenotype column.
pymethyl-utils counts [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-k
,
--key
<key>
¶ Key to split on. [default: disease]
create_external_validation_set¶
Create external validation set containing same CpGs as training set.
pymethyl-utils create_external_validation_set [OPTIONS]
Options
-
-t
,
--train_pkl
<train_pkl>
¶ Input methyl array. [default: ./train_val_test_sets/train_methyl_array.pkl]
-
-q
,
--query_pkl
<query_pkl>
¶ Input methylation array to add/subtract cpgs to. [default: ./final_preprocessed/methyl_array.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output methyl array external validation. [default: ./external_validation/methyl_array.pkl]
-
-c
,
--cpg_replace_method
<cpg_replace_method>
¶ What to do for missing CpGs. [default: mid]
feature_select_train_val_test¶
Filter CpGs by taking x top CpGs with highest mean absolute deviation scores or via spectral feature selection.
pymethyl-utils feature_select_train_val_test [OPTIONS]
Options
-
-i
,
--input_pkl_dir
<input_pkl_dir>
¶ Input database for beta and phenotype data. [default: ./train_val_test_sets/]
-
-o
,
--output_dir
<output_dir>
¶ Output database for beta and phenotype data. [default: ./train_val_test_sets_fs/]
-
-n
,
--n_top_cpgs
<n_top_cpgs>
¶ Number cpgs to include with highest variance across population. [default: 300000]
-
-f
,
--feature_selection_method
<feature_selection_method>
¶
-
-mm
,
--metric
<metric>
¶
-
-nn
,
--n_neighbors
<n_neighbors>
¶ Number neighbors for feature selection, default enacts rbf kernel. [default: 0]
-
-m
,
--mad_top_cpgs
<mad_top_cpgs>
¶ Number cpgs to apply mad filtering first before more sophisticated feature selection. If 0 or primary feature selection is mad, no mad pre-filtering. [default: 0]
fix_key¶
Format certain column of phenotype array in MethylationArray.
pymethyl-utils fix_key [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-k
,
--key
<key>
¶ Key to split on. [default: disease]
-
-d
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]
-
-o
,
--output_pkl
<output_pkl>
¶ Input database for beta and phenotype data. [default: ./fixed_preprocessed/methyl_array.pkl]
modify_pheno_data¶
Use another spreadsheet to add more descriptive data to methylarray.
pymethyl-utils modify_pheno_data [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-is
,
--input_formatted_sample_sheet
<input_formatted_sample_sheet>
¶ Information passed through function create_sample_sheet, has Basename and disease fields. [default: ./tcga_idats/minfi_sheet.csv]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./modified_processed/methyl_array.pkl]
move_jpg¶
Move preprocessing jpegs to preprocessing output directory.
pymethyl-utils move_jpg [OPTIONS]
Options
-
-i
,
--input_dir
<input_dir>
¶ Directory containing jpg. [default: ./]
-
-o
,
--output_dir
<output_dir>
¶ Output directory for images. [default: ./preprocess_output_images/]
overwrite_pheno_data¶
Use another spreadsheet to add more descriptive data to methylarray.
pymethyl-utils overwrite_pheno_data [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-is
,
--input_formatted_sample_sheet
<input_formatted_sample_sheet>
¶ Information passed through function create_sample_sheet, has Basename and disease fields. [default: ./tcga_idats/minfi_sheet.csv]
-
-o
,
--output_pkl
<output_pkl>
¶ Output database for beta and phenotype data. [default: ./modified_processed/methyl_array.pkl]
-
-c
,
--index_col
<index_col>
¶ Index col when reading csv. [default: 0]
pkl_to_csv¶
Output methylarray pickle to csv.
pymethyl-utils pkl_to_csv [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-o
,
--output_dir
<output_dir>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/]
-
-c
,
--col
<col>
¶ Column to color. [default: ]
print_number_sex_cpgs¶
Print number of non-autosomal CpGs.
pymethyl-utils print_number_sex_cpgs [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-a
,
--array_type
<array_type>
¶ Array Type. [default: 450k]
print_shape¶
Print dimensions of beta matrix.
pymethyl-utils print_shape [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
ref_estimate_cell_counts¶
Reference based cell type estimates.
pymethyl-utils ref_estimate_cell_counts [OPTIONS]
Options
-
-ro
,
--input_r_object_dir
<input_r_object_dir>
¶ Input directory containing qc data. [default: ./preprocess_outputs/]
-
-a
,
--algorithm
<algorithm>
¶ Algorithm to run cell type. [default: meffil]
-
-ref
,
--reference
<reference>
¶ Cell Type Reference. [default: cord blood gse68456]
-
-l
,
--library
<library>
¶ IDOL Library. [default: IDOLOptimizedCpGs450klegacy]
-
-o
,
--output_csv
<output_csv>
¶ Output cell type estimates. [default: ./added_cell_counts/cell_type_estimates.csv]
remove_sex¶
Remove non-autosomal CpGs.
pymethyl-utils remove_sex [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./preprocess_outputs/methyl_array.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output methyl array autosomal. [default: ./autosomal/methyl_array.pkl]
-
-a
,
--array_type
<array_type>
¶ Array Type. [default: 450k]
remove_snps¶
Remove SNPs from methylation array.
pymethyl-utils remove_snps [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./autosomal/methyl_array.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output methyl array autosomal. [default: ./no_snp/methyl_array.pkl]
-
-a
,
--array_type
<array_type>
¶ Array Type. [default: 450k]
set_part_array_background¶
Set subset of CpGs from beta matrix to background values.
pymethyl-utils set_part_array_background [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input methyl array. [default: ./final_preprocessed/methyl_array.pkl]
-
-c
,
--cpg_pkl
<cpg_pkl>
¶ Pickled numpy array for subsetting. [default: ./subset_cpgs.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output methyl array external validation. [default: ./removal/methyl_array.pkl]
stratify¶
Split methylation array by key and store.
pymethyl-utils stratify [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-k
,
--key
<key>
¶ Key to split on. [default: disease]
-
-o
,
--output_dir
<output_dir>
¶ Output directory for stratified. [default: ./stratified/]
subset_array¶
Only retain certain number of CpGs from methylation array.
pymethyl-utils subset_array [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input methyl array. [default: ./final_preprocessed/methyl_array.pkl]
-
-c
,
--cpg_pkl
<cpg_pkl>
¶ Pickled numpy array for subsetting. [default: ./subset_cpgs.pkl]
-
-o
,
--output_pkl
<output_pkl>
¶ Output methyl array external validation. [default: ./subset/methyl_array.pkl]
train_test_val_split¶
Split methylation array into train, test, val.
pymethyl-utils train_test_val_split [OPTIONS]
Options
-
-i
,
--input_pkl
<input_pkl>
¶ Input database for beta and phenotype data. [default: ./final_preprocessed/methyl_array.pkl]
-
-o
,
--output_dir
<output_dir>
¶ Output directory for training, testing, and validation sets. [default: ./train_val_test_sets/]
-
-tp
,
--train_percent
<train_percent>
¶ Percent data training on. [default: 0.8]
-
-vp
,
--val_percent
<val_percent>
¶ Percent of training data that comprises validation set. [default: 0.1]
-
-cat
,
--categorical
¶
Multi-class prediction. [default: False]
-
-do
,
--disease_only
¶
Only look at disease, or text before subtype_delimiter.
-
-k
,
--key
<key>
¶ Key to split on. [default: disease]
-
-sd
,
--subtype_delimiter
<subtype_delimiter>
¶ Delimiter for disease extraction. [default: ,]


ExtExec allows you to execute Linux shell commands in the powerful Linux environment provided by the free app called Termux.
Note: This function is a premium feature. Please consider subscribing to Premium to use this feature and support DroidScript in its further development.
The Termux app runs as a service and there's almost no limit to what you can achieve with a full blown Linux machine running on your Android device!
How about installing git, or nodejs or using ssh or gzip.
Try typing 'busybox' at the terminal prompt to see what built-in commands are already available to you.
Enable Termux
To enable termux support you have to install Termux and the Tasker Plugin from PlayStore first.
Then give termux sdcard permissions in the Android settings under apps/termux.
After that create a symbolic link from the tasker plugin folder to your home directory:
mkdir -p .termux /sdcard/termux/scripts
ln -s /sdcard/termux/scripts .termux/tasker
Finally you can run a termux shell script from inside DroidScript:
Example - Open Termux and pass arguments
function OnStart()
{
app.WriteFile( "/sdcard/termux/scripts/hello.sh", "echo Hello $1 and $2" );
var err = app.ExtExec( "termux", "hello.sh", 'World "' + app.GetUser() + '"', "" );
if( err ) app.Alert( "Termux Error:" + err );
}
Example - Execute in Background and retreive output
var itv, lock = "/sdcard/.termuxlock";
var script = `
{ # this is a comment
echo This is a message # prints to stdout
sleep 1 # waits 1 second
echo This is an error 1>&2 # prints to stderr
} 1>/sdcard/out.txt 2>/sdcard/err.txt; # forward stdout and stderr to files
rm ` + lock;
function OnStart()
{
app.WriteFile( "/sdcard/termux/scripts/hello.sh", script );
app.WriteFile( lock, "" );
app.ShowProgress( "Script is running" );
var err = app.ExtExec( "termux", "hello.sh", "", "hide" );
if(err) app.Alert(err);
lay = app.CreateLayout( "linear", "VCenter,FillXY" );
txt = app.CreateText( "", .8, .8, "monospace,multiline,left" );
lay.AddChild( txt );
app.AddLayout( lay );
itv = setInterval( CheckReady, 100 );
}
function CheckReady()
{
if( !app.FileExists( lock ))
{
app.HideProgress();
clearInterval(itv);
var stdout = app.ReadFile( "/sdcard/out.txt" );
var stderr = app.ReadFile( "/sdcard/err.txt" );
var text =
"stdout:\n" + stdout + "\n\n" +
"stderr:\n" + stderr;
txt.SetText( text );
}
}


With the rapid development of technologies to capture and transmit human, sensor and
machine-generated data, the opportunities for data scientists are growing rapidly.
Companies and public institutions are hiring data scientists to find insights, to help on
decision-making or to provide novel services.
Universities and research institutes need data
scientists to process and analyze large amounts of data in fields as diverse as biology,
medicine, astronomy, climate or environmental sciences. However, data science is an
interdisciplinary discipline that encompasses theories and methods within broad areas such
as computer science, information science, mathematics and statistics, which, until recently,
were mostly treated as independent knowledge domains.
This course is for beginners in data science who are interested in understanding the main
concepts, methods and tools behind big data analytics and its applications. The aim is to
cover the entire data science pipeline and to provide an integrated view on databases, data
mining, machine learning and data visualization. The program includes lectures and practice
problems using real-world data and case studies.


Wireless networks should be using strong encryption and authentication protocols, such as WPA or WPA2, to secure communications.
Strong encryption and authentication protocols for wireless communications prevent both unauthorized access to the network and eavesdropping on wireless communications. An earlier implementation of wireless encryption and authentication, known as WEP, has proven to be insecure.
Examine the configurations of wireless access points to make sure that strong encryption and authentication protocols, such as WPA or WPA2, are being used. Examine a wireless client to make sure that strong encryption and authentication protocols are actually being used.
Configure the wireless access point(s) to use strong encryption and authentication protocols, such as WPA or WPA2. The detailed instructions for enabling strong encryption and authentication vary by wireless access points and network designs. Please refer to the vendor's documentation or technical support for specific instructions on enabling strong encryption and authentication protocols on your wireless network(s).