Computer Vision Framework 4 - Visual Recognition
Visual Recognition is another broad category of image processing, which is simply the idea of identifying things in an image and typically labeling them as such. There is a large variety of things that computer vision can be used to recognized, but for this project we focus on categorizing an image based on colors in the image, as well as classifying pixels in a particular image.
Image Classification
For Image classification, we used the concept of creating histograms of color intensities for each color channel for the whole image. We had 3 different categories to train for: "coast", "forest", and "inside city". I tried two different approaches. First I created a list of histograms for each training image. Then use the label of the nearest neighbor to the test image for classification. Next, I created an average histogram for each label based on the average of the training image. I actually got better accuracy and faster classification for this method. I also tried various bin sizes. The best seems to be around 7-10 bins with the average model, but for individual there isn't a huge difference from 2-15 bins.
Here are the actual classification results on my average model with 8 bins per color channel, in red and green highlights with the training and test images for reference, as well as a accuracy summary below:
| Label | Accuracy |
|---|---|
| coast | 1.00 |
| forest | 1.00 |
| inside_city | 0.75 |
| OVERALL | 0.92 |
The one image, "insidecity_test2", lacks any significant amount of red, unlike the inside city training images, while it does have more grays and slight blues so it was misclassified as coast. We can see a downside of this classification method is that all structural information of the location of these colors is ignored. However it is very fast to calculate.
Pixel Classification
For Pixel classification we start with 1 training image and 4 testing images. The task was to first manually find the given the ground truth of sky pixels in the training image(by manual image masking techniques), then use k-means to find 10 "visual words" each for both "sky" pixels and "not sky" pixels. These visual words are essentially just pixel clusters, found in a similar way as "K-means color segmentation" of our previous framework. based on my past experience with slow sequential code when using k-means, I use as much vectorized numpy code as I could to keep it fast. Then I simply find the closest center for each pixel in a test image to classify. Here are the results:
Training image:
Results:
This method works fairly well overall, but of course fails whenever there are non-sky pixels that are a close color to the sky or clouds. So the bluish flowers in test3 and white lines on the road on test 2 are identified as sky as well. You can also see some of the darker sky pixels in test4 are misclassified due to the training image only having brighter skys. A couple of ways to improve this would be to include spacial information for where sky pixels typically are, or to add extra logic to identify sky boundaries, such as assuming the sky is the largest contiguous section of sky pixels and ignoring other smaller sections.


Finally a Way to Batch-Rename Files on iOS
by Andrew OrrEvery since I bought an iPad Pro I’ve been able to conduct most of my work and personal life on my iPad. But I still need my MacBook Pro for a couple things, like using Automator to batch-rename files (renaming multiple files simultaneously). This isn’t possible to do on iOS, until now. A shortcut I found on RoutineHub lets you do this. It lets you rename your files in custom formats, like alphabetically, prepending text, sorting by date modified, etc.
Check It Out: Finally a Way to Batch-Rename Files on iOS


The JS9 preferences plugin allows you to set JS9 user preferences for JS9-enabled Web pages in the current domain. By and large, these are the same preferences that can be found in the JS9 site preferences file (js9Prefs.json), and of course you can set defaults in that file. However, the preferences plugin allows you to set these preferences in cases where you do not have access to this file, e.g. public sites such as js9.si.edu.
Due to cross-browser security limitations, preferences are browser-specific: your Firefox preferences will be maintained separately from your Chrome and Safari preferences.
The following types of user preferences can be set:
You can apply your preference choices to the current session using the Apply button. To apply and save these preferences for future sessions, use the Save button. The Show Saved and Delete Saved buttons allow you to view and clear the currently saved values, respectively.


This series of posts/post has origins on the last Pentaho Community Meeting that took place in London, November last year, PCM 2015. You can follow up on
Twitter.
The first evening, there was a hackathon, where me Miguel Cunhal and Sam … (sorry Sam, can’t remenber your surname), received a Raspberry Pi, coutesy of Ivy IS as a prize. We really hadn’t the time to complete the project, so I felt that I needed to contribute with something else.
Given that the Rapesberry Pi is largely used on IoT (Internet Of Things) projects, I will share what we I am calling a Streaming Dashboard, using Pentaho technology.
Quick introduction
When talking about IoT and Social Innovation (I feel like I should write a new blog post, about my point of view about this terminologies), there are some use cases were we need to display “near real-time” or even “real-time” data, coming from all kind of “things”.
For the title of the blog post, I’ve used the name “Streaming” and not “Real-time”, that’s because don’t want to get into a discussion Electronic Engineers, where real-time is done in milli seconds. Not that we can’t do it really fast at the software level, but ir ight not be fair to use the same terminology. Let’s keep in mind that displaying information on the browser and comming from any place in the world, will probably take more than milli seconds.
In this project I have used a Rapesberry Pi sending sensor data, but I might have used a mobile phone, a twitter account, a car and any other “thing”. For you to be able to test it and for the samples I am provideing, I’ve creates a kettle transformation were I am generating some random data emulating sensors.
Considerations
There are a huge demand for real-time dashboards, and there are a couple of questions you may need to consider before starting with this kind of projects:
-
How often is it important to have data refreshed in the dashboard? Usually, after making this question you might realize that you wont need a high refresh rate as you first might though about. A refresh rate of one, two, …, five seconds or more might be more than enough. Even if you have your device is gathering data faster, it might make sense to display it at that refresh rate and you might want to consider to do some aggregation, apply some signal processing algorithms, etc. Of course that always depends on the use case.
-
Can the users really understand what you are displaying when data is being refreshed too fast? Well I don’t believe so unless, of course some users might be able to do it, but does it makes sense? Usually the user will get lost just when trying to interpret results faster than he is able to. His mind might be so focused about not missing anything that his mind might lost focus on interpreting the values/results. Your end-user should always be able to easily understand what it’s being presented. Once again don’t do it more times than necessary.
-
Another question that was already slightly touched is. Should we keep the same speed the device is using to sent new information? Not always, specially when the refresh rate is too high. We might want to do some aggregations, display just some part of the values, just keep in mind to process all the values. Let’s suppose you are getting sensor values from an engine and you want to get warnings on values below/above a certain threshold. You can’t simple discard the values, before checking if it’s above or below the thresholds but you might be able to do it after. Another way is to implement that login on your back-end. The dashboard just need to receive notifications about the breaking the thresholds.
Putting pieces together
There are many ways to bring live data to a dashboard, and I have developed one sample, were I have considered the following requirements:
- Messages should be sent to the dashboard, but the dashboard should also be able to send messages/instructions to the device.
- Use Pentaho Data Integration (PDI) to process data, and it should be done as soon as messages arrives from the sensors;
- Present live data in the dashboard as fast as possible, just for performance measurements on the front-end and back-end.
- Needs be presented on a CTools dashboard, in a way to complement an existing dashboard
- Save as much bandwidth as possible.
- Should be able to work on mobile devices.
Even if I am going to get into details in latter posts, let me inform you ahead about the technologies that were used:
- Pentaho BA Server, CTools & Pentaho Data Integration;
- Pentaho BA Server (version 6.0);
- Ctools (last stable);
- Pentaho DI (version 6.0);
- MQTT protocol and libraries MQTT, as the protocol to exchange messages:
- Mosquitto Mosquitto, a publish/subscribe MQTT Message Broker;
- I have used it in a Docker container.
Small video
As this a live/real-time solution and I don’t have the infrastructure to make it available for you to see it working, I have made a video that shows the dashboard working with live data.
Future work
I can’t cover all in this post, so I will write a couple more posts under this subject, providing details about the architecture, the back-end and front-end as well as performance metrics like messages per second, memory/processor consumption, etc. If you want to see some specific details, please comment bellow.
For sure you will find details about the architecture on how to scale the solution. This is a solution based on message-centric concepts but I also want to bring some posts on it would work on a data-centric proof-of-concept, of course the architecture and technology will change a bit.
I already started to think how I can make it work out-of-the-box in CDA/CDE, but that will take much more time to develop it. If there are some folks that wnat’s to help, just send a message and let’s work together.


Cognitive Science Related Links
Cognitive Science at Indiana University
Indiana University Cognitive Science Program
Undergraduate Program in Cognitive Science
Links to Other Undergraduate Cognitive Science Programs
Brown University
Cognitive and Linguistic Sciences Department
Hampshire College
School of Cognitive Science and Culture Studies
Johns Hopkins University
Department of Cognitive Science
Rice University
Undergraduate Cognitive Science Program
Stanford University
University of California at San Diego
University of Rochester
Department of Brain and Cognitive
Sciences
Cognitive Science Program
Vassar College
Cognitive Science on the Web
Mind Brain Resources (from Virginia Tech)
Cognitive and Psychological Sciences on the Internet
This page was lasted edited on September 27, 1998.
Please address questions and comments to: iusocs@indiana.edu.


Based on information available, we appear to experiencing a potential application outage.
{STATUS_PAGE_TEXT}
NO TREND DETECTED
At this time we are not noticing a trend of similiar issues.
Error Information
The application reported that this is a {EXCEPTION_TYPE} error,
with the reason listed as {EXCEPTION_MESSAGE}. This information may be useful
to your support team, so please provide this information if asked for it.
While trying to load the page you requested, our systems ran into a problem. It's never fun
to see an error message, and we definitly want to work to ensure you don't continue to see them.


 The syndrome shows itself via a number of symptoms ranging from dry eyes, blurred vision, fatigue, headaches, difficulty in concentrating, and so on. Unfortunately, many individuals are heavily tied to various device screens for work, school, or other important reasons and do not have the option to detach from these screens in order to reduce symptoms. Furthermore, many individuals are not even aware that their symptoms are caused by a combination of their environment (lighting, font size, etc.) and device screens, thus do not even know where to begin in alleviating their symptoms. Over time, these symptoms may worsen and put an individual’s eyes at serious health risk as well as make it nearly impossible to work at a screen for prolonged periods of time.
Lensy, is a wearable pair of glasses that contain several key functionalities in order to help reduce digital eye strain symptoms. They block blue HEV light that projects off screens and adjusts the lenses to the perfect shading depending on an individual’s environment. As an individual works at various devices throughout the day, the glasses remind the individual to take breaks via customizable vibrating or pop-up reminders. The glasses are highly customizable and can be configured to not disturb during certain hours of the day, allowing them to work with a variety of schedules. Combined with sensors that detect certain symptoms like dry eyes and fatigue, individuals can monitor their progress over time via the analytics button on the side of the glasses. Resultantly, individuals can work at various device screens for long periods of time without experiencing symptoms, and assure that their eye health is in good hands.
Lensy analyzes your surroundings and devices in real time to give feedback on screen text size, viewing angle, and more.
When reduced usage of devices isn’t an option, studies show following the 20-20-20 rule is crucial to reduce DES. Lensy sends reminders to look away every 20 minutes at something 20 feet away for 20 seconds.
Track your personal eye strain symptoms across usage of your devices in real time. See Analytics of your screen time, rest taken, eye health, and environment lighting.
Connect your devices with Lensy so that brightness can be auto adjusted to reduce eye strain no matter your environment.
Biochemistry
Computer Science
Computer Science
Computer Science


SpiceJet promoter Ajay Singh pledges 2.78% shares to secure credit facility
With the latest pledge of shares, a total of 26.32 per cent shareholding of Singh is now encumbered.
by Press Trust of IndiaSpiceJet promoter and chief Ajay Singh has pledged around 2.78 per cent shares towards securing a credit facility for the airline.
A total of 1.67 crore shares were pledged in favour of IDFC First Bank Ltd on October 24, according to a filing submitted by the carrier to the BSE on Saturday.
These shares, which account for 2.78 per cent shareholding, have been pledged to "secure a credit facility obtained by SpiceJet", it added.
With the latest pledge of shares, a total of 26.32 per cent shareholding of Singh is now encumbered.
Individually, he has a stake of 50.75 per cent in the company, as per the filing.


Create a Customized 'Groups' Table Definition from an SQL Query
With the query in the top right panel we are going to generate a base table definition to expand upon later in this lab.
Steps:
- Run the query. You should see a result set appear after the query has been run.
- View the table definition XML for this query by clicking on 'Generate table definition XML' in the upper right corner of the result set. This will open a new tab to the right of this text in which the XML will be displayed
- Save the table definition XML. On Firefox, this would be by right-clicking in the XML document and then selecting 'This Frame' -> 'Save Frame As...'
- Save the file as 'db2authCustomGroups.xml' in the folder '[your_TE_folder]/tableDefinitions/db2authLab'


{% extends "layouts/main.html" %} {% import "ui/forms.html" as forms %} {% block content_page %}
You can export your account at any time into a JSON file, containing all of your user info (except passwords), projects, channels, and hooks. This file can be imported into a self-hosted Notifico instance.
Export


Anand Mahindra takes a trip down memory lane with 'Bijlee', Mahindra's EV from 1999
by BusinessToday.InBack in the day, Mahindra group had partnered with Delhi government in 1998-99 to conceptualise and launch the 10-seater electric three-wheeler Bijlee
As India hopes to see electric mobility become the norm in coming decades, Anand Mahindra reminisced about an electric vehicle from the last millennium. The Chairman of Mahindra group posted a tweet on Thursday about Bijlee, an electric three-wheeler developed by Mahindra back in 1999.
The tweet read: "Mahesh Babu of @MahindraElctrc fished this out from the archives. Very few know we launched an electric 3wheeler called 'Bijlee' back in 1999. Conceptualised by S.V.Nagarkar,here in the pic. Didn't work out financially but we learned a lot & renewed our commitment to promote EVs!"
Back in the day, Mahindra group had partnered with Delhi government in 1998-99 to conceptualise and launch the 10-seater electric three-wheeler Bijlee. Mahindra undertook two pilot projects to test the viability of Bijlee, one in Delhi in August 1999 and another in Mumbai in September 2000. It was officially launched in Delhi in 2002 and was known to be popular among traders in Connaught Place.
ALSO READ: Around 75% EV charging stations to be in public; rest will be at homes: Fortum India MD
The Bijlee came with a range of 80-90 kilometers on a single charge. The electric vehicle was priced at Rs 3.32 lakh (ex-showroom Delhi) at the time of launch. Mahindra even exported Bijlees to Nepal. The running cost for Bijlee was very low, with reports claiming that the life-time cost per passenger came out to be a mere 48 paise.
In commercial space, Mahindra currently has a five-seater e-rickshaw called e-ALFA Mini. Mahindra also offers e20 Plus and eVerito in the electric passenger vehicle segment.
ALSO READ: Ratan Tata's heart-warming post on Instagram traces his Cornell days
ALSO READ: 'Just isn't our style': Anand Mahindra gets Perak clip taking dig at Royal Enfield removed
ALSO READ: Delhi pollution: Supreme Court proposes hydrogen fuel vehicles; industry disposes


Falling In Love With Computers
The first computer I ever used was an Apple IIe. My elementary school had a computer lab and if you went to school early, you could play with computers instead of the playground. Some of the cool kids would go to the computer room, so I followed. We learned to use LOGO and BASIC along with playing video games like Karateka.
These computers, which booted on 5 1/4 floppy disks, were amazing to me and the other kids. I've read about other's early experiences with computers and how they wanted to command them to do their bidding. I was no different. I programmed and entire copy of a Choose Your Own Adventure book using BASIC and GOTO statements.
When I was in 3rd grade, I remember playing some game one of the 5th graders had written. He programmed a space game that procedurally generated a map using the at ("@") symbol. You would use the arrow keys to navigate a space ship, which looked an awful lot like the letter "I", between a ravine. To this day, I still think how amazing it was that a 5th grader wrote that program.
Around the same time my father was working at California State University at Sacramento helping professors and other educators connect to the PLATO network. In order to understand more about the technology our family purchased our first computer, an Apple Macintosh Classic.
This computer, with it's black and white, monitor wasn't too amazing to me. I couldn't program on it the way I could at school. But, when my dad showed me PLATO my mind was blown. PLATO was an education system and had teaching games you could play. These games would teach all kinds of different things, but my dad decided to show me a math game.
A space ship would fly around the monitor and then face three doors. Upon choosing one of the doors, the game would present a simple math problem, such as 14 minus 6. I would count on my fingers to solve the problem and then enter an answer. The space ship would advance to the next door and present another problem.
After answering the third arithmetic problem, the game said, "stop counting on your fingers." I was mesmerized. How did this computer know I was counting on my fingers? I now understand it was using a simple timer to see how long it took me to enter the answer, but to an 8-year old kid the computer was watching me count on my fingers. This incident led me to learning as much about computers as I could.
A few years went by and Pentium computers were being released. My frugal family weren't early adopters and couldn't justify the need for a new computer. I would often go to another kids house after school and watch him use DOS commands to install Wolfenstein. The game didn't interest me, but his use of the command line did. After two years of telling my parents stories of him using the command line, my mother sympathized and brought home a 286 which was thrown away from her office. Although, it was ancient, even at that time, I was in love. I quickly destroyed OS/2 and installed DOS 6.22 on the 20MB hard drive and started learning the command line. I tried to install Windows 3.1 on it, but the computer was so old it didn't support it.
My mother also brought home a 14.4 modem, even though 56k modems were already available, I was excited. I quickly went to the grocery store to grab a copy of California Computer News. In the back of CCN was a list of BBS's. Armed with Procomm Plus, I became involved in the BBS community, even though I was mostly interested in playing LORD.
DOOM soon was released and again my computer wouldn't support it. The lack of my computer supporting all of these games is probably why I am not as hard core of a gamer as others my age. Friends of mine would constantly call, This BBS sucks to try to hopefully play 3 others in a networked version of DOOM.
Since my computer didn't have video games to play, and I had the summer at home by myself, I started delving more into BBS's. Then I discovered the HPVAC scene. An entire underground of BBS's not listed in CCN. Each one had a list of more BBS's. I started calling every local BBS I could find and reading about the hacking, phreaking, virii, anarchy, and cracks scene.
My entire love of computers and information security can be traced to having an old, command-line only, computer that couldn't play video games.


| Home | Trees | Index | Help |
|
|---|
| Package openid :: Package store :: Module filestore :: Class FileOpenIDStore |
|
object
--+ |OpenIDStore
--+ | FileOpenIDStore
This is a filesystem-based store for OpenID associations and nonces. This store should be safe for use in concurrent systems on both windows and unix (excluding NFS filesystems). There are a couple race conditions in the system, but those failure cases have been set up in such a way that the worst-case behavior is someone having to try to log in a second time.
Most of the methods of this class are implementation details. People
wishing to just use this store need only pay attention to the
method.__init__
| Method Summary | |
|---|---|
Initializes a new FileOpenIDStore. | |
Remove expired entries from the database. | |
| int |
Remove expired associations from the store. |
| int |
Remove expired nonces from the store. |
Retrieve an association. | |
Create a unique filename for a given server url and handle. | |
Remove an association if it exists. | |
Store an association in the association directory. | |
Return whether this nonce is valid. | |
Inherited from object :
__delattr__ ,
__getattribute__ ,
__hash__ ,
__new__ ,
__reduce__ ,
__reduce_ex__ ,
__repr__ ,
__setattr__ ,
__str__
|
| Method Details |
|---|
__init__(self,
directory)
Initializes a new FileOpenIDStore. This initializes the nonce and
association directories, which are subdirectories of the directory
passed in.
|
cleanup(self)Remove expired entries from the database. This is potentially expensive, so only run when it is acceptable to take time. () -> NoneType
|
cleanupAssociations(self)Remove expired associations from the store. This method is not called in the normal operation of the library. It provides a way for store admins to keep their storage from filling up with expired data.
|
cleanupNonces(self)Remove expired nonces from the store. Discards any nonce from storage that is old enough that its
timestamp would not pass
|
getAssociation(self, server_url, handle=None)Retrieve an association. If no handle is specified, return the association with the latest expiration. (str, str or NoneType) -> Association or NoneType |
getAssociationFilename(self, server_url, handle)Create a unique filename for a given server url and handle. This implementation does not assume anything about the format of the handle. The filename that is returned will contain the domain name from the server URL for ease of human inspection of the data directory. (str, str) -> str |
removeAssociation(self, server_url, handle)Remove an association if it exists. Do nothing if it does not. (str, str) -> bool |
storeAssociation(self, server_url, association)Store an association in the association directory. (str, Association) -> NoneType |
useNonce(self, server_url, timestamp, salt)Return whether this nonce is valid. str -> bool
|
| Home | Trees | Index | Help |
|
|---|
| Generated by Epydoc 2.1 on Fri Dec 14 16:10:53 2007 | http://epydoc.sf.net |


{% extends "_post.html" %} {%hyde title: "Grumpy, Sneezy, Sleepy, ..." created: 2005-06-21 20:41:33 %} {% block article %} {%article%}
Several people have asked for a little standalone app that uses ipod-sharp, since not everyone uses Muine and keeps all of their music on their PC. So this weekend I wrote Dopi.
You simply DnD folders/files onto the song list there to add stuff. The delete key deletes (surprise!). To try it, you’ll need the latest ipod-sharp. It has been moved into Mono’s SVN, so if you have the stuff from baz, that is out-of-date. Also, ipod-sharp depends on libipoddevice in GNOME CVS, so you’ll need that too. And lastly, Dopi itself is in my baz archive, here.
{%endarticle%} {% endblock %}


#firstworldproblem
Typing this tweet on my computer because I didn't get an iPad for Christmas.


Toggle navigation
Menu
Home
Stock Inventory
Customer
Stock Display
User Actions
{{InvoiceStaff}}
Logout


This visualisation shows the energy usage and Co2 emissions of buildings across campus.
Click on a circle to view information about that building
Name:
Location:
Energy Usage:
Co2 Emissions:
Energy Cost:
0kWh
0kg CO2
0£


Hiring just the right housekeeper can be life-altering for the busy family. Now this may seem to be a little over-the-top, but I am
serious.
Think about your life-whether single, a couple or a family-you have no time. No time to keep your home as organized or
clean as you would like.
No time to do the smaller tidying jobs that would make a huge difference...
BY ADMIN
“I am still learning about my home. If you can’t do it yourself, find good service people and treat them well.”  Often, our earliest memories of figuring out how to tidy up and take care of our homes are with our moms—they taught us how to do laundry, how to do the dishes, and how to make our beds. So it should come as no surprise that when we asked our moms for their best homekeeping advice, they came back with some great tips.
Most agreed on a few major themes: it’s important to make sure everything has a home, that you put things back when you take them out, and that your home is an inviting, relaxing place to come home to—and invite others to.
Hiring just the right housekeeper can be life-altering for the busy family. Now this may seem to be a little over-the-top, but I am serious. Think about your life-whether single, a couple or a family-you have no time. No time to keep your home as organized or clean as you would like. No time to do the smaller tidying jobs that would make a huge difference..
BY ADMIN
Do you ever casually go about your day, popping into the kitchen for a little peanut butter toast and then the next thing you know you’re turning over the toaster oven because it’s been so long and you just can’t take the crumbs anymore?
BY ADMIN


Version 2.2.0 Beta
First Beta Release
Changes
+ phpBB
+ concrete5
+ osTicket
+ General
+ Better Menu
+ Sorting
+ Design Changes
CMS-Installer
Fast Installing
Easy to use
Efficient Working
First Beta Release
Changes
+ phpBB
+ concrete5
+ osTicket
+ General
+ Better Menu
+ Sorting
+ Design Changes
Important Bugfixes! Please download this version!
Updated Shopware (5.2.24 -> 5.2.26)
Updated softwares:
Wordpress 4.7.6 -> 4.8
Typo3 7.6.18 -> 7.6.19
EspoCRM 4.7.0 -> 4.7.2
Owncloud 10.0.1 -> 10.0.2


Trees
Indices
Help
[
hide private
]
[
frames
] |
no frames
]
Identifier Index
[
A
B
C
D
E
F
G
H
I
J K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y Z
_
]
A
Address
(in
Arrest_test
)
Also_Known_As
(in
Arrest_test
)
Arrest_Type
(in
Arrest_test
)
ADMINS
(in
odnc_police.settings
)
application
(in
odnc_police.wsgi
)
Arrestee_Phone
(in
Arrest_test
)
Age
(in
Arrest_test
)
Arrest_Number
(in
Arrest_test
)
Arrestee_Signature
(in
Arrest_test
)
Agency_Name
(in
Arrest_test
)
Arrest_test
(in
odnc_police.models
)
Arresting_Officer_Signature
(in
Arrest_test
)
ALLOWED_HOSTS
(in
odnc_police.settings
)
Arrest_Tract
(in
Arrest_test
)
Assisting_Officer
(in
Arrest_test
)
B
Bond_Amount
(in
Arrest_test
)
C
Case_Disposition
(in
Arrest_test
)
Charge2_Statute_Number
(in
Arrest_test
)
Committing_Magistrate
(in
Arrest_test
)
Case_Status
(in
Arrest_test
)
Charge2_Type
(in
Arrest_test
)
Comp
(in
Arrest_test
)
Charge1
(in
Arrest_test
)
Charge2_Warrant_Date
(in
Arrest_test
)
Comp_Address
(in
Arrest_test
)
Charge1_Counts
(in
Arrest_test
)
Charge3
(in
Arrest_test
)
Comp_Name
(in
Arrest_test
)
Charge1_DCI_Code
(in
Arrest_test
)
Charge3_Counts
(in
Arrest_test
)
Comp_Phone
(in
Arrest_test
)
Charge1_Offense_Jurisdiction
(in
Arrest_test
)
Charge3_DCI_Code
(in
Arrest_test
)
Consumed_Drug_Alcohol
(in
Arrest_test
)
Charge1_Statute_Number
(in
Arrest_test
)
Charge3_Offense_Jurisdiction
(in
Arrest_test
)
Country_of_Citizenship
(in
Arrest_test
)
Charge1_Type
(in
Arrest_test
)
Charge3_Statute_Number
(in
Arrest_test
)
Court_of_Trial
(in
Arrest_test
)
Charge1_Warrant_Date
(in
Arrest_test
)
Charge3_Type
(in
Arrest_test
)
createLineIndex()
(in
xmlScraper
)
Charge2
(in
Arrest_test
)
Charge3_Warrant_Date
(in
Arrest_test
)
createPDFDict()
(in
xmlScraper
)
Charge2_Counts
(in
Arrest_test
)
city
(in
Arrest_test
)
createPDFList()
(in
xmlScraper
)
Charge2_DCI_Code
(in
Arrest_test
)
City_of_Court
(in
Arrest_test
)
currentOCA
(in
xmlScraper
)
Charge2_Offense_Jurisdiction
(in
Arrest_test
)
Color
(in
Arrest_test
)
D
DATABASES
(in
odnc_police.settings
)
detail()
(in
odnc_police.views
)
Drug1_Suspected_Type
(in
Arrest_test
)
Date_Arrested
(in
Arrest_test
)
DOB
(in
Arrest_test
)
Drug2_Activities
(in
Arrest_test
)
Date_Submitted
(in
Arrest_test
)
Drug1_Activities
(in
Arrest_test
)
Drug2_DCI
(in
Arrest_test
)
Datetime_Confined
(in
Arrest_test
)
Drug1_DCI
(in
Arrest_test
)
Drug2_Quantity
(in
Arrest_test
)
Datetime_Released
(in
Arrest_test
)
Drug1_Quantity
(in
Arrest_test
)
Drug2_Status
(in
Arrest_test
)
DEBUG
(in
odnc_police.settings
)
Drug1_Status
(in
Arrest_test
)
Drug2_Suspected_Type
(in
Arrest_test
)
E
Employer_Address
(in
Arrest_test
)
Employer_Phone
(in
Arrest_test
)
Employer_Name
(in
Arrest_test
)
Eyes
(in
Arrest_test
)
F
FEMALE
(in
Arrest_test
)
Fingerprint_Card_Number_CKN
(in
Arrest_test
)
G
getFileLinesList()
(in
xmlScraper
)
getTextBetweenDelimiters()
(in
xmlScraper
)
getSinglePDFObj()
(in
xmlScraper
)
grabRelevantDataAndFields()
(in
xmlScraper
)
H
Hair
(in
Arrest_test
)
Hgt
(in
Arrest_test
)
I
If_Armed_Type_of_Weapon
(in
Arrest_test
)
INSTALLED_APPS
(in
odnc_police.settings
)
L
LANGUAGE_CODE
(in
odnc_police.settings
)
LOGGING
(in
odnc_police.settings
)
M
Make
(in
Arrest_test
)
MEDIA_ROOT
(in
odnc_police.settings
)
Model
(in
Arrest_test
)
MALE
(in
Arrest_test
)
MEDIA_URL
(in
odnc_police.settings
)
models
(in
odnc_police
)
MANAGERS
(in
odnc_police.settings
)
MIDDLEWARE_CLASSES
(in
odnc_police.settings
)
markAppropriateFieldsNull()
(in
xmlScraper
)
Misc_Number_and_Type
(in
Arrest_test
)
N
Name
(in
Arrest_test
)
Nearest_Relative_Address
(in
Arrest_test
)
Nearest_Relative_Phone
(in
Arrest_test
)
Narrative
(in
Arrest_test
)
Nearest_Relative_Name
(in
Arrest_test
)
O
OCA
(in
Arrest_test
)
odnc_police
ORI
(in
Arrest_test
)
Occupation
(in
Arrest_test
)
OLN_and_State
(in
Arrest_test
)
P
pairCheckmarkData()
(in
xmlScraper
)
Photos_Taken
(in
Arrest_test
)
Plate_No_and_State
(in
Arrest_test
)
pairFieldandData()
(in
xmlScraper
)
Place_Confined
(in
Arrest_test
)
printPDFDict()
(in
xmlScraper
)
pdf
(in
Arrest_test
)
Place_of_Arrest
(in
Arrest_test
)
Prints_Taken
(in
Arrest_test
)
pdfObjectList
(in
test
)
Place_of_Birth
(in
Arrest_test
)
processLists()
(in
xmlScraper
)
Q
qsort()
(in
xmlScraper
)
R
Race
(in
Arrest_test
)
removeMultipleFromFieldList()
(in
xmlScraper
)
Residency_Status
(in
Arrest_test
)
Released_By
(in
Arrest_test
)
renamingScript()
(in
xmlScraper
)
result()
(in
odnc_police.views
)
removeFromFieldList()
(in
xmlScraper
)
Residence_Tract
(in
Arrest_test
)
ROOT_URLCONF
(in
odnc_police.settings
)
S
Scars_Marks_Tattoos
(in
Arrest_test
)
SEX_CHOICES
(in
Arrest_test
)
STATIC_ROOT
(in
odnc_police.settings
)
search()
(in
odnc_police.views
)
SITE_ID
(in
odnc_police.settings
)
STATIC_URL
(in
odnc_police.settings
)
SECRET_KEY
(in
odnc_police.settings
)
Skin_Tone
(in
Arrest_test
)
STATICFILES_DIRS
(in
odnc_police.settings
)
seeChargesTrue
(in
xmlScraper
)
Social_Security
(in
Arrest_test
)
STATICFILES_FINDERS
(in
odnc_police.settings
)
settings
(in
odnc_police
)
sortByKey()
(in
xmlScraper
)
Style
(in
Arrest_test
)
Sex
(in
Arrest_test
)
state
(in
Arrest_test
)
Supervisor_Signature
(in
Arrest_test
)
T
TEMPLATE_DEBUG
(in
odnc_police.settings
)
test
TIME_ZONE
(in
odnc_police.settings
)
TEMPLATE_DIRS
(in
odnc_police.settings
)
Time_Arrested
(in
Arrest_test
)
Trial_Date
(in
Arrest_test
)
TEMPLATE_LOADERS
(in
odnc_police.settings
)
Time_Submitted
(in
Arrest_test
)
Type_Bond
(in
Arrest_test
)
U
UNSPEC
(in
Arrest_test
)
urls
(in
odnc_police
)
USE_L10N
(in
odnc_police.settings
)
urlpatterns
(in
odnc_police.urls
)
USE_I18N
(in
odnc_police.settings
)
USE_TZ
(in
odnc_police.settings
)
V
Vehicle_Status
(in
Arrest_test
)
views
(in
odnc_police
)
VYR
(in
Arrest_test
)
Vehicle_Status_Datetime
(in
Arrest_test
)
VIN
(in
Arrest_test
)
W
Wgt
(in
Arrest_test
)
wsgi
(in
odnc_police
)
WSGI_APPLICATION
(in
odnc_police.settings
)
X
xmlScraper
_
__str__()
(in
Arrest_test
)
Trees
Indices
Help
Generated by Epydoc 3.0.1 on Sat Dec 6 07:51:16 2014
http://epydoc.sourceforge.net


--- layout: post status: publish published: true title: "开平行，出发" author: Leask author_login: leask author_email<EMAIL_ADDRESS>author_url: https://leaskh.com wordpress_id: 548 wordpress_url: http://leaskh.wordpress.com/2006/05/26/%e5%bc%80%e5%b9%b3%e8%a1%8c%ef%bc%8c%e5%87%ba%e5%8f%91 date: '2006-05-26 07:38:15 +0800' date_gmt: '2006-05-26 07:38:15 +0800' categories: tags: [] comments: [] ---


Hello World!
Home
Profile
Messages
This page does not look very good. You should put some content here!!


Born in 1993. The year of Rooster.
It's not my favourite meat, though.
Hong Kong
It's in Hong Kong where I born
And after I've spent 21/24 of my life there... and wondering maybe I shall have some changes?
Stockholm
Ok, I made a decision, and now I am living in Stockholm.
Another side of the world, Another page for my world!
More in Detail!
Master´s Programme in Digital Narration: Game and Cultural Heritage
Marketing Study - New Media Study - Game Study
Master´s Programme in Fashion Studies
Fashion History - Gender Study - Philosophy
Exchange Study
Industrial Design - Animation - 3D Design
Bachelor of Arts (Honours) Scheme in Fashion and Textiles
Fashion Technology - Textile Engineering - Marketing - Fashion Design
Studio Heijne, Stockholm
Technical Drawing for New Collection - Showroom Preparation - Daily Support for designer
Hong Kong Polytechnic University, Hong Kong
Data Mining - Experiment Preparation - Data Analysis
Assessment of Fabric Handle
HSBC, Hong Kong
Customer Service - Marketing Support - Cash Handling
Lead Marketer
'A hobby a day keeps the doldrums away.'
- Phyllis McGinley


15th SoCal PLS Program
9:00 am
Breakfast
10:00 am
Parallelizing Seemingly-Sequential
Computations
Important applications, such as machine
learning and log processing, involve iterating over a large data set
with loop-carried dependences across iterations. As such, these
computations are not embarrassingly parallel. In this talk, I will
describe symbolic parallelism, a general methodology for
parallelizing dependent computations. The basic idea is to break
dependences using efficient symbolic reasoning. This work is
primarily motivated by the need to extract parallelism from data
processing queries that combine standard relational operations (such
as map, reduce, filter) with non-relational application-specific
code. By applying symbolic parallelism to such code and exposing the
resulting parallelism to the underlying query optimizer, we obtain
2-4 orders of magnitude performance improvements over Hadoop and SQL
Server.
10:30 am
OpenMP is Not as Easy as It Appears
This paper aims to show that knowing the
core concepts related to a given parallel architecture is
necessary to write correct code, regardless of the parallel
programming paradigm used. Programmers unaware of architecture
concepts, such as beginners and students, often write parallel
code that is slower than their sequential versions. It is also
easy to write code that produces incorrect answers under
specific conditions, which are hard to detect and correct. The
increasing popularization of multi-core architectures
motivates the implementation of parallel programming
frameworks and tools, such as OpenMP, that aim to lower the
difficulty of parallel programming. OpenMP uses compilation
directives, or pragmas, to reduce the number of lines that the
programmer needs to write. However, the programmer still has
to know when and how to use each of these directives. The
documentation and available tutorials for OpenMP give the idea
that using compilation directives for parallel programming is
easy. In this paper we show that this is not always the case
by analysing a set of corrections of OpenMP programs made by
students of a graduate course in Parallel and Distributed
Computing, at University of São Paulo. Several
incorrect examples of OpenMP pragmas were found in tutorials
and official documents available in the Internet. The idea
that OpenMP is easy to use can lead to superficial efforts in
teaching fundamental parallel programming concepts. This can
in its turn lead to code that does not develop the full
potential of OpenMP, and could also crash inexplicably due to
very specific and hard-to-detect conditions. Our main
contribution is showing how important it is to teach core
architecture and parallel programming concepts properly, even
when you have powerful tools such as OpenMP available.
11:00 am
Break
11:15 am
Data-Driven Precondition Inference with Learned
Features
We present a new black-box approach for likely
precondition inference. We infer preconditions for a code snippet by
using a form of machine learning over a set of test executions. Like
prior data-driven approaches, we employ a feature based learning --
features are boolean predicates over inputs. But prior approaches
are limited by a fixed set of features or a fixed structure for the
predicates. Our approach overcomes these limitations by employing a
generic on-demand feature learning using a program synthesizer, and
producing a CNF over features without any predefined structure. We
also show that, given access to the source code, and an SMT solver
which is able to handle the theories used in the source code, we can
infer verified loop invariants. Furthermore, our approach is
agnostic to the solver, program synthesizer and boolean function
learner used. We have implemented our approach as a tool, PIE
(Precondition Inference Engine) and we demonstrate its capabilities
by inferring preconditions for OCaml library code and verified
invariants for C++ code.
11:45 am
Supporting Reasoning at the Hardware
Level
Modern machines are complicated, even in
their interface. Previous works have tried to formalize
hardware ISAs to aid in verification and reasoning, but the
inherently global, imperative, and stateful nature of existing
machines severely hamstrings these efforts. We take a
different approach: rather than attempting to bridge the gap
between proof and machine, we attempt to shrink it with a new
hardware architecture that presents a purely functional,
lambda calculus-like ISA. We present a working prototype,
hardware usage figures, compiler, and initial proofs of
correctness of some binary programs, and will touch on future
uses and research directions of the machine.
12:15 pm
Lunch
1:30 pm
The Sourcerer 2011 Dataset
We present Sourcerer 2011, a publicly
available dataset of software projects written in the Java
programming language. This dataset contains over 74,000 open
source projects collected from Google Code, SourceForge and
Apache, along with over 119,000 jar files found in those
projects. The dataset comprises over 630M source lines of
code. Because of its large size, processing this many projects
is a time-consuming task. As such, one characteristic of this
dataset is the provision of three different representations
that support different kinds of research work. The different
representations correspond to stages of processing in a tool
chain, from the downloading of the projects over the Internet
to the creation of a relational database of code entities and
their relations for open-ended querying. We believe this
dataset will help a wide variety of research involving source
code analysis, by avoiding many necessary but time-consuming
tasks.
2:00 pm
Abstractor: Embedding Imperative Programs Into
Linear Temporal Logic for Hybrid Systems Reasoning
As part of the VeriDrone hybrid-systems
verification project, we need to reason formally about the behavior
of physical systems that contain controllers running imperative
code. I'll describe how we are able to embed programs (in a mostly
language-indepedent way) into the variant of LTL that we used to
reason about physical systems, including a sound accounting for
floating-point computation errors incurred by the controller. The
talk will touch on programming-language semantics, temporal logic,
verification, and interval-based floating-point reasoning.
2:30 pm
Break
3:00 pm
Fuzzing for SMT Solvers
SMT solvers are vitally important tools which
are employed in a variety of domains, including automated testing,
synthesis, and verification. While their correctness is often
paramount to the task at hand, they are still susceptible to
bugs. To help find these bugs before they become problems, we employ
black-box language fuzzing techniques in order to to automatically
test SMT solvers. To this end, we instantiate existing general
fuzzing techniques specifically for SMT solvers. In addition, we
employ a novel fuzzing technique which generates edge cases that
stress corners of a provided language standard. Another major
contribution of this work is an empirical evaluation of the
bug-finding effectiveness of fuzzers implemented according to four
orthogonal dimensions: syntax-guided versus semantics-guided,
unrestricted versus restricted to a subset of an input grammar
(i.e. Swarm Testing), random versus bounded depth-first, and the
generation of big versus small test cases. Such an extensive
evaluation of different fuzzing approaches has not been performed
before, and the community at large is heavily focused on only a
small portion of this space. While this work is still relatively
young, we report our experiences with a semantics-guided and
unrestricted fuzzer which generates small inputs according to a
bounded depth-first search strategy. Although common wisdom
suggests that this point in the fuzzer state space should be
relatively uninteresting, this fuzzer alone has already found over a
dozen bugs across multiple popular SMT solvers, including Z3, CVC4,
MathSAT 5, and Boolector. More often than not, these bugs have
exposed correctness problems with the underlying solver under test,
and developers have usually been quick to fix them.
3:30 pm
Meta-FRP for livecoding audio
Functional Reactive Programming provides
an interesting programming paradigm. Applying it to the
process of livecoding musical composition results in a novel
outlook---the program is the signal, as input to the
compiler/runtime system---and also invites a notion of
*meta*-FRP---we can reify the time-based program editing
process as a meta-program. Indeed, we can take another step,
and reify edits to the meta-program as a
meta-meta-program.
4:00 pm
Discussion and "quick ideas"
4:30 pm
Business meeting


arkstock module
import sys sys.dont_write_bytecode = True from .data import * from .diversify import * from .optimize import * from .pipelines import PortfolioGeneratorFactory from .exceptions import * __all__ = [ # Actual classes 'TimeInterval', 'AbstractStockDatabaseClient', 'AbstractStockMetadataClient', 'AbstractStatefulStockDatabaseClient', 'yahoo_finance_gateway', 'YahooTimeInterval', 'YahooFinanceMetadataClient', 'HelperUtils', 'YahooFinanceClient', 'YahooFinanceStatefulClient', 'helper', 'diversify', 'optimize', 'PortfolioGeneratorFactory', 'ArkstockException', # Test classes # '_YahooFinanceGatewayTest', '_YahooFinanceMetadataClientTest', '_YahooFinanceClientTest', # '_HelperUtilsTest', # '_DiversificationHelperTest', '_DiversifyTest', # '_OptimizeTest' ] testclasses = [ # Test classes '_YahooFinanceGatewayTest', '_YahooFinanceMetadataClientTest', '_YahooFinanceClientTest', '_HelperUtilsTest', '_DiversificationHelperTest', '_DiversifyTest', '_OptimizeTest' ]
Classes
class AbstractStatefulStockDatabaseClient
Basically a stateful proxy of AbstractStockDatabaseClient
class AbstractStatefulStockDatabaseClient(AbstractStockDatabaseClient): ''' Basically a stateful proxy of AbstractStockDatabaseClient ''' @abstractmethod def set_state(self, start_date: date, end_date: date, time_interval: TimeInterval): ''' This method sets the class wide execution state for stateful database clients Args: * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > None ''' pass
Ancestors (in MRO)
- AbstractStatefulStockDatabaseClient
- AbstractStockDatabaseClient
- builtins.object
Static methods
def get_available_exchanges(
self)
This method returns a list of stock exchanges for which this client can return data
@abstractmethod def get_available_exchanges(self) -> List[str]: ''' This method returns a list of stock exchanges for which this client can return data ''' pass
def get_historical_data_for_stocks(
self, symbols, start_date, end_date, time_interval)
Inheritance:
AbstractStockDatabaseClient
.get_historical_data_for_stocks
This method returns historical data for all the provided ticker symbols in USD
Args:
symbols
- the ticker symbols for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
This method will skip all ticker symbols that are invalid for some reason or another
@abstractmethod def get_historical_data_for_stocks(self, symbols: List[str], start_date: date, end_date: date, time_interval: TimeInterval) -> Dict[str, List[float]]: ''' This method returns historical data for all the provided ticker symbols in USD Args: * `symbols` - the ticker symbols for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > This method will skip all ticker symbols that are invalid for some reason or another ''' pass
def get_historical_data_for_stocks_in_exchange(
self, exchange_name, start_date, end_date, time_interval, threshold_mcap=True, mcap_limit_in_millions=300)
This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit
Args:
exchange_name
- string repr of the exchange for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time rangethreshold_mcap
- whether or not the stocks in the exchange should be filtered based on market capitalizationmcap_limit_in_millions
- the market cap threshold
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
The returned data is for the ticker symbols found in the provided exchange.
@abstractmethod def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, start_date: date, end_date: date, time_interval: TimeInterval, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: ''' This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit Args: * `exchange_name` - string repr of the exchange for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range * `threshold_mcap` - whether or not the stocks in the exchange should be filtered based on market capitalization * `mcap_limit_in_millions` - the market cap threshold Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > The returned data is for the ticker symbols found in the provided exchange. ''' pass
def get_historical_stock_prices(
self, symbol, start_date, end_date, time_interval)
This method retrieves historical price data for a ticker symbol in USD.
Args:
symbol
- the ticker symbol for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A list of the historical stock price data in USD. The most recent price should be at index 0.
Note:
In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred
@abstractmethod def get_historical_stock_prices(self, symbol: str, start_date: date, end_date: date, time_interval: TimeInterval) -> List[float]: ''' This method retrieves historical price data for a ticker symbol in USD. Args: * `symbol` - the ticker symbol for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A list of the historical stock price data in USD. The most recent price should be at index 0. Note: > In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred ''' pass
def set_state(
self, start_date, end_date, time_interval)
This method sets the class wide execution state for stateful database clients
Args:
start_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
None
@abstractmethod def set_state(self, start_date: date, end_date: date, time_interval: TimeInterval): ''' This method sets the class wide execution state for stateful database clients Args: * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > None ''' pass
class AbstractStockDatabaseClient
An abstract stock database client that defines the methods that need to be implemented for interoperability with the rest of the code
class AbstractStockDatabaseClient(metaclass=ABCMeta): ''' An abstract stock database client that defines the methods that need to be implemented for interoperability with the rest of the code ''' @abstractmethod def get_historical_stock_prices(self, symbol: str, start_date: date, end_date: date, time_interval: TimeInterval) -> List[float]: ''' This method retrieves historical price data for a ticker symbol in USD. Args: * `symbol` - the ticker symbol for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A list of the historical stock price data in USD. The most recent price should be at index 0. Note: > In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred ''' pass @abstractmethod def get_historical_data_for_stocks(self, symbols: List[str], start_date: date, end_date: date, time_interval: TimeInterval) -> Dict[str, List[float]]: ''' This method returns historical data for all the provided ticker symbols in USD Args: * `symbols` - the ticker symbols for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > This method will skip all ticker symbols that are invalid for some reason or another ''' pass @abstractmethod def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, start_date: date, end_date: date, time_interval: TimeInterval, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: ''' This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit Args: * `exchange_name` - string repr of the exchange for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range * `threshold_mcap` - whether or not the stocks in the exchange should be filtered based on market capitalization * `mcap_limit_in_millions` - the market cap threshold Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > The returned data is for the ticker symbols found in the provided exchange. ''' pass @abstractmethod def get_available_exchanges(self) -> List[str]: ''' This method returns a list of stock exchanges for which this client can return data ''' pass
Ancestors (in MRO)
- AbstractStockDatabaseClient
- builtins.object
Static methods
def get_available_exchanges(
self)
This method returns a list of stock exchanges for which this client can return data
@abstractmethod def get_available_exchanges(self) -> List[str]: ''' This method returns a list of stock exchanges for which this client can return data ''' pass
def get_historical_data_for_stocks(
self, symbols, start_date, end_date, time_interval)
This method returns historical data for all the provided ticker symbols in USD
Args:
symbols
- the ticker symbols for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
This method will skip all ticker symbols that are invalid for some reason or another
@abstractmethod def get_historical_data_for_stocks(self, symbols: List[str], start_date: date, end_date: date, time_interval: TimeInterval) -> Dict[str, List[float]]: ''' This method returns historical data for all the provided ticker symbols in USD Args: * `symbols` - the ticker symbols for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > This method will skip all ticker symbols that are invalid for some reason or another ''' pass
def get_historical_data_for_stocks_in_exchange(
self, exchange_name, start_date, end_date, time_interval, threshold_mcap=True, mcap_limit_in_millions=300)
This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit
Args:
exchange_name
- string repr of the exchange for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time rangethreshold_mcap
- whether or not the stocks in the exchange should be filtered based on market capitalizationmcap_limit_in_millions
- the market cap threshold
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
The returned data is for the ticker symbols found in the provided exchange.
@abstractmethod def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, start_date: date, end_date: date, time_interval: TimeInterval, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: ''' This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit Args: * `exchange_name` - string repr of the exchange for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range * `threshold_mcap` - whether or not the stocks in the exchange should be filtered based on market capitalization * `mcap_limit_in_millions` - the market cap threshold Returns: > A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0. Note: > The returned data is for the ticker symbols found in the provided exchange. ''' pass
def get_historical_stock_prices(
self, symbol, start_date, end_date, time_interval)
This method retrieves historical price data for a ticker symbol in USD.
Args:
symbol
- the ticker symbol for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A list of the historical stock price data in USD. The most recent price should be at index 0.
Note:
In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred
@abstractmethod def get_historical_stock_prices(self, symbol: str, start_date: date, end_date: date, time_interval: TimeInterval) -> List[float]: ''' This method retrieves historical price data for a ticker symbol in USD. Args: * `symbol` - the ticker symbol for which you wish to retrieve data * `start_date` - the start of the time range for which you would like to get data * `end_date` - the end of the time range for which you would like to get data * `time_interval` - the time interval at which you would like to get data for the given time range Returns: > A list of the historical stock price data in USD. The most recent price should be at index 0. Note: > In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred ''' pass
class AbstractStockMetadataClient
An abstract stock metadata client that defines the methods that need to be implemented for interoperability with the rest of the code
class AbstractStockMetadataClient(metaclass=ABCMeta): ''' An abstract stock metadata client that defines the methods that need to be implemented for interoperability with the rest of the code ''' @abstractmethod def get_stocks_in_exchange(self, exchange_name: str) -> List[str]: ''' This method returns a list of ticker symbols that correspond to a provided `exchange_name` Args: * `exchange_name` - string repr of the exchange for which you wish to retrieve data Returns: > A list of ticker symbols found in that exchange ''' pass
Ancestors (in MRO)
- AbstractStockMetadataClient
- builtins.object
Static methods
def get_stocks_in_exchange(
self, exchange_name)
This method returns a list of ticker symbols that correspond to a provided exchange_name
Args:
exchange_name
- string repr of the exchange for which you wish to retrieve data
Returns:
A list of ticker symbols found in that exchange
@abstractmethod def get_stocks_in_exchange(self, exchange_name: str) -> List[str]: ''' This method returns a list of ticker symbols that correspond to a provided `exchange_name` Args: * `exchange_name` - string repr of the exchange for which you wish to retrieve data Returns: > A list of ticker symbols found in that exchange ''' pass
class ArkstockException
Base exception class from which all other package exceptions inherit. This allows the end user to simply catch ArkstockException and handle exceptions from the arkstock package in a consistent way.
class ArkstockException(Exception): ''' Base exception class from which all other package exceptions inherit. This allows the end user to simply catch ArkstockException and handle exceptions from the arkstock package in a consistent way. ''' pass
Ancestors (in MRO)
- ArkstockException
- builtins.Exception
- builtins.BaseException
- builtins.object
Class variables
var args
class HelperUtils
class HelperUtils(object): @staticmethod def chunker(iterable: Iterable[Any], chunksize: int, fillvalue: Any =None) -> Iterable[Any]: ''' Collect data into fixed-length chunks or blocks Args: * `iterable` - a data type is iterable * `chunksize` - the size of the blocks you wish to extract from the iterable * `fillvalue` - the default value to fill for the last block if there arent enough elements Returns: > A generator/iterable wherein each element is a block of size `chunksize` ''' # chunker('ABCDEFG', 3, 'x') --> [('A', 'B', 'C'), ('D', 'E', 'F'), ('G', 'x', 'x')] args = [iter(iterable)] * chunksize return zip_longest(*args, fillvalue=fillvalue) @staticmethod def is_valid_ticker(ticker: str) -> bool: ''' Check to ensure that a given ticker doesn't contain any weird symbols Args: * `ticker` - the string repr of the ticker symbol to check Returns: > True if the ticker is valid, false otherwise ''' if '@' in ticker: return False if '&' in ticker: return False return True @staticmethod def convert_TimeInterval_to_YahooTimeInterval(time_interval: TimeInterval) -> YahooTimeInterval: ''' A utility method to convert from the generic TimeInterval to the specific YahooTimeInterval ''' mapping = { TimeInterval.weeks : YahooTimeInterval.weeks } return mapping.get(time_interval, time_interval)
Ancestors (in MRO)
- HelperUtils
- builtins.object
Static methods
def chunker(
iterable, chunksize, fillvalue=None)
Collect data into fixed-length chunks or blocks
Args:
iterable
- a data type is iterablechunksize
- the size of the blocks you wish to extract from the iterablefillvalue
- the default value to fill for the last block if there arent enough elements
Returns:
A generator/iterable wherein each element is a block of size
chunksize
@staticmethod def chunker(iterable: Iterable[Any], chunksize: int, fillvalue: Any =None) -> Iterable[Any]: ''' Collect data into fixed-length chunks or blocks Args: * `iterable` - a data type is iterable * `chunksize` - the size of the blocks you wish to extract from the iterable * `fillvalue` - the default value to fill for the last block if there arent enough elements Returns: > A generator/iterable wherein each element is a block of size `chunksize` ''' # chunker('ABCDEFG', 3, 'x') --> [('A', 'B', 'C'), ('D', 'E', 'F'), ('G', 'x', 'x')] args = [iter(iterable)] * chunksize return zip_longest(*args, fillvalue=fillvalue)
def convert_TimeInterval_to_YahooTimeInterval(
time_interval)
A utility method to convert from the generic TimeInterval to the specific YahooTimeInterval
@staticmethod def convert_TimeInterval_to_YahooTimeInterval(time_interval: TimeInterval) -> YahooTimeInterval: ''' A utility method to convert from the generic TimeInterval to the specific YahooTimeInterval ''' mapping = { TimeInterval.weeks : YahooTimeInterval.weeks } return mapping.get(time_interval, time_interval)
def is_valid_ticker(
ticker)
Check to ensure that a given ticker doesn't contain any weird symbols
Args:
ticker
- the string repr of the ticker symbol to check
Returns:
True if the ticker is valid, false otherwise
@staticmethod def is_valid_ticker(ticker: str) -> bool: ''' Check to ensure that a given ticker doesn't contain any weird symbols Args: * `ticker` - the string repr of the ticker symbol to check Returns: > True if the ticker is valid, false otherwise ''' if '@' in ticker: return False if '&' in ticker: return False return True
class PortfolioGeneratorFactory
This class encapsulates the logic to create a data-pipeline that returns a diversified and optimized portfolio.
class PortfolioGeneratorFactory(object): ''' This class encapsulates the logic to create a data-pipeline that returns a diversified and optimized portfolio. ''' @staticmethod def get_default(exchange_filter: Callable[[str], bool] =lambda x: True): ''' This returns the default pipeline that can be used to diversify and optimize portfolios Args: * `exchange_filter` - a function that returns True or False depending on whether a stock exchange should be used to diversify/optimize a portfolio. Returns: > A function that allows one to pass in a list of preferred stocks and output a diversified portfolio Note: > Simple ad-hoc testing shows that the first call to the output pipeline takes 60 seconds, whereas subsequent calls only take 0.5 seconds. ''' db_client = YahooFinanceStatefulClient() db_client = default_cached_db_client(db_client) stock_exchange_data_source = default_stateful_stock_exchange_data_source(db_client, exchange_filter) transformed_stock_exch_data = default_data_transformator(stock_exchange_data_source) diversifier = default_diversifier(transformed_stock_exch_data) optimizer = default_optimizer(diversifier) return optimizer
Ancestors (in MRO)
- PortfolioGeneratorFactory
- builtins.object
Static methods
def get_default(
exchange_filter=<function PortfolioGeneratorFactory.<lambda> at 0x000001B6E68860D0>)
This returns the default pipeline that can be used to diversify and optimize portfolios
Args:
exchange_filter
- a function that returns True or False depending on whether a stock exchange should be used to diversify/optimize a portfolio.
Returns:
A function that allows one to pass in a list of preferred stocks and output a diversified portfolio
Note:
Simple ad-hoc testing shows that the first call to the output pipeline takes 60 seconds, whereas subsequent calls only take 0.5 seconds.
@staticmethod def get_default(exchange_filter: Callable[[str], bool] =lambda x: True): ''' This returns the default pipeline that can be used to diversify and optimize portfolios Args: * `exchange_filter` - a function that returns True or False depending on whether a stock exchange should be used to diversify/optimize a portfolio. Returns: > A function that allows one to pass in a list of preferred stocks and output a diversified portfolio Note: > Simple ad-hoc testing shows that the first call to the output pipeline takes 60 seconds, whereas subsequent calls only take 0.5 seconds. ''' db_client = YahooFinanceStatefulClient() db_client = default_cached_db_client(db_client) stock_exchange_data_source = default_stateful_stock_exchange_data_source(db_client, exchange_filter) transformed_stock_exch_data = default_data_transformator(stock_exchange_data_source) diversifier = default_diversifier(transformed_stock_exch_data) optimizer = default_optimizer(diversifier) return optimizer
class TimeInterval
An enum to encapsulate package specific time intervals
class TimeInterval(Enum): ''' An enum to encapsulate package specific time intervals ''' weeks = ('weeks', 7) '''TimeInterval enum value to denote a time interval of 1 week i.e. 7 days''' def __init__(self, symbol, days): self.symbol = symbol # string specific to yahoo finance that represents that time range self.days = days # interval length in days
Ancestors (in MRO)
- TimeInterval
- enum.Enum
- builtins.object
Class variables
var name
var value
var weeks
TimeInterval enum value to denote a time interval of 1 week i.e. 7 days
class YahooFinanceClient
An concrete stock database client that implementes the methods that need to be implemented for interoperability with the rest of the code. This client is based on the yahoo finance API.
This client is essentially stateless and is hence threadsafe.
class YahooFinanceClient(AbstractStockDatabaseClient): ''' An concrete stock database client that implementes the methods that need to be implemented for interoperability with the rest of the code. This client is based on the yahoo finance API. This client is essentially stateless and is hence threadsafe. ''' def __init__(self, yfgateway =yahoo_finance_gateway, yfmetadata: AbstractStockMetadataClient =None, global_exchange_list: List[str] =['ASE', 'NCM', 'NGM', 'NMS', 'NYQ', 'OBB', 'TOR', 'VAN']): self.yfgateway = yfgateway '''A pointer to the YahooFinanceGateway module''' if yfmetadata is None: yfmetadata = YahooFinanceMetadataClient() self.yfmetadata = yfmetadata '''A yahoo finance metadata client''' self.global_exchange_list = global_exchange_list '''Stock exchanges for which data is available''' def filter_based_on_mcap_limit(self, symbols: Iterable[str], mcap_limit_in_millions: int) -> List[str]: ''' Given a list of stocks, only return stocks that have a market capitalization over the provided threshold Args: * `symbols` - the ticker symbols for which you wish to retrieve data * `mcap_limit_in_millions` - the mcap threshold Returns: > A list of symbols above the threshold ''' if mcap_limit_in_millions == 0: return symbols valid_symbols = [] chunked_stocks = HelperUtils.chunker(symbols, 500) for chunk in chunked_stocks: _stocks = [x for x in chunk if x is not None] _mcap = self.yfgateway.get_mcap(_stocks) _stocks_above_threshold = [ symbol for symbol, mcap in _mcap if self.yfgateway.is_mcap_over_limit(mcap) ] valid_symbols += _stocks_above_threshold return valid_symbols @lru_cache(maxsize=50) def _cached_get_exchange_rate(self, curr: str, start_date: date, end_date: date, time_interval: YahooTimeInterval) -> List[float]: ''' This is a cached wrapper around `yfgateway.get_historical_exch_rate_to_USD` > See `yfgateway.get_historical_exch_rate_to_USD` ''' return self.yfgateway.get_historical_exch_rate_to_USD(curr, start_date, end_date, time_interval) def get_historical_stock_prices(self, symbol: str, start_date: date, end_date: date, time_interval: TimeInterval) -> List[float]: if not HelperUtils.is_valid_ticker(symbol): raise IllegalArgument('%s is not a valid ticker symbol' % symbol) yahoo_time_interval = HelperUtils.convert_TimeInterval_to_YahooTimeInterval(time_interval) stock_native_curr = self.yfgateway.get_native_currency_for_ticker(symbol) exch_rate = self._cached_get_exchange_rate(stock_native_curr, start_date, end_date, yahoo_time_interval) historical_prices = self.yfgateway.get_historical_stock_prices(symbol, start_date, end_date, yahoo_time_interval) return [ rate * price for rate, price in zip(exch_rate, historical_prices) ] def get_historical_data_for_stocks(self, symbols: List[str], start_date: date, end_date: date, time_interval: TimeInterval) -> Dict[str, List[float]]: symbols = [x for x in symbols if HelperUtils.is_valid_ticker(x)] historical_prices = {} args = [(symbol, start_date, end_date, time_interval) for symbol in symbols] with ThreadPoolExecutor() as executor: future_to_stock = { executor.submit(self.get_historical_stock_prices, *arg): arg[0] for arg in args } for future in as_completed(future_to_stock): symbol = future_to_stock[future] try: price_in_usd = future.result() except Exception as exc: pass else: historical_prices[symbol] = price_in_usd return historical_prices def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, start_date: date, end_date: date, time_interval: TimeInterval, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: stocks_in_exch = [ x for x in self.yfmetadata.get_stocks_in_exchange(exchange_name) if HelperUtils.is_valid_ticker(x) ] if threshold_mcap: stocks_in_exch = self.filter_based_on_mcap_limit(stocks_in_exch, mcap_limit_in_millions) yahoo_time_interval = HelperUtils.convert_TimeInterval_to_YahooTimeInterval(time_interval) if len(stocks_in_exch) == 0: return {} stock_exch_native_currency = self.yfgateway.get_native_currency_for_ticker(stocks_in_exch[0]) exchange_rate = self._cached_get_exchange_rate(stock_exch_native_currency, start_date, end_date, yahoo_time_interval) historical_prices = {} args = [(symbol, start_date, end_date, yahoo_time_interval) for symbol in stocks_in_exch] with ThreadPoolExecutor() as executor: future_to_stock = { executor.submit(self.yfgateway.get_historical_stock_prices, *arg): arg[0] for arg in args } for future in as_completed(future_to_stock): symbol = future_to_stock[future] try: price_in_native_currency = future.result() except Exception as exc: pass else: historical_prices[symbol] = [ rate * price for rate, price in zip(exchange_rate, price_in_native_currency) ] return historical_prices def get_available_exchanges(self) -> List[str]: return self.global_exchange_list
Ancestors (in MRO)
- YahooFinanceClient
- AbstractStockDatabaseClient
- builtins.object
Static methods
def __init__(
self, yfgateway=<module '__pdoc_file_module__.data.yahoo_finance_gateway' from 'C:\\Users\\abhinav\\Documents\\BitBucket\\stockanalysis\\arkstock\\data\\yahoo_finance_gateway.py'>, yfmetadata=None, global_exchange_list=['ASE', 'NCM', 'NGM', 'NMS', 'NYQ', 'OBB', 'TOR', 'VAN'])
Initialize self. See help(type(self)) for accurate signature.
def __init__(self, yfgateway =yahoo_finance_gateway, yfmetadata: AbstractStockMetadataClient =None, global_exchange_list: List[str] =['ASE', 'NCM', 'NGM', 'NMS', 'NYQ', 'OBB', 'TOR', 'VAN']): self.yfgateway = yfgateway '''A pointer to the YahooFinanceGateway module''' if yfmetadata is None: yfmetadata = YahooFinanceMetadataClient() self.yfmetadata = yfmetadata '''A yahoo finance metadata client''' self.global_exchange_list = global_exchange_list '''Stock exchanges for which data is available'''
def filter_based_on_mcap_limit(
self, symbols, mcap_limit_in_millions)
Given a list of stocks, only return stocks that have a market capitalization over the provided threshold
Args:
symbols
- the ticker symbols for which you wish to retrieve datamcap_limit_in_millions
- the mcap threshold
Returns:
A list of symbols above the threshold
def filter_based_on_mcap_limit(self, symbols: Iterable[str], mcap_limit_in_millions: int) -> List[str]: ''' Given a list of stocks, only return stocks that have a market capitalization over the provided threshold Args: * `symbols` - the ticker symbols for which you wish to retrieve data * `mcap_limit_in_millions` - the mcap threshold Returns: > A list of symbols above the threshold ''' if mcap_limit_in_millions == 0: return symbols valid_symbols = [] chunked_stocks = HelperUtils.chunker(symbols, 500) for chunk in chunked_stocks: _stocks = [x for x in chunk if x is not None] _mcap = self.yfgateway.get_mcap(_stocks) _stocks_above_threshold = [ symbol for symbol, mcap in _mcap if self.yfgateway.is_mcap_over_limit(mcap) ] valid_symbols += _stocks_above_threshold return valid_symbols
def get_available_exchanges(
self)
This method returns a list of stock exchanges for which this client can return data
def get_available_exchanges(self) -> List[str]: return self.global_exchange_list
def get_historical_data_for_stocks(
self, symbols, start_date, end_date, time_interval)
This method returns historical data for all the provided ticker symbols in USD
Args:
symbols
- the ticker symbols for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
This method will skip all ticker symbols that are invalid for some reason or another
def get_historical_data_for_stocks(self, symbols: List[str], start_date: date, end_date: date, time_interval: TimeInterval) -> Dict[str, List[float]]: symbols = [x for x in symbols if HelperUtils.is_valid_ticker(x)] historical_prices = {} args = [(symbol, start_date, end_date, time_interval) for symbol in symbols] with ThreadPoolExecutor() as executor: future_to_stock = { executor.submit(self.get_historical_stock_prices, *arg): arg[0] for arg in args } for future in as_completed(future_to_stock): symbol = future_to_stock[future] try: price_in_usd = future.result() except Exception as exc: pass else: historical_prices[symbol] = price_in_usd return historical_prices
def get_historical_data_for_stocks_in_exchange(
self, exchange_name, start_date, end_date, time_interval, threshold_mcap=True, mcap_limit_in_millions=300)
This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit
Args:
exchange_name
- string repr of the exchange for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time rangethreshold_mcap
- whether or not the stocks in the exchange should be filtered based on market capitalizationmcap_limit_in_millions
- the market cap threshold
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
The returned data is for the ticker symbols found in the provided exchange.
def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, start_date: date, end_date: date, time_interval: TimeInterval, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: stocks_in_exch = [ x for x in self.yfmetadata.get_stocks_in_exchange(exchange_name) if HelperUtils.is_valid_ticker(x) ] if threshold_mcap: stocks_in_exch = self.filter_based_on_mcap_limit(stocks_in_exch, mcap_limit_in_millions) yahoo_time_interval = HelperUtils.convert_TimeInterval_to_YahooTimeInterval(time_interval) if len(stocks_in_exch) == 0: return {} stock_exch_native_currency = self.yfgateway.get_native_currency_for_ticker(stocks_in_exch[0]) exchange_rate = self._cached_get_exchange_rate(stock_exch_native_currency, start_date, end_date, yahoo_time_interval) historical_prices = {} args = [(symbol, start_date, end_date, yahoo_time_interval) for symbol in stocks_in_exch] with ThreadPoolExecutor() as executor: future_to_stock = { executor.submit(self.yfgateway.get_historical_stock_prices, *arg): arg[0] for arg in args } for future in as_completed(future_to_stock): symbol = future_to_stock[future] try: price_in_native_currency = future.result() except Exception as exc: pass else: historical_prices[symbol] = [ rate * price for rate, price in zip(exchange_rate, price_in_native_currency) ] return historical_prices
def get_historical_stock_prices(
self, symbol, start_date, end_date, time_interval)
This method retrieves historical price data for a ticker symbol in USD.
Args:
symbol
- the ticker symbol for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A list of the historical stock price data in USD. The most recent price should be at index 0.
Note:
In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred
def get_historical_stock_prices(self, symbol: str, start_date: date, end_date: date, time_interval: TimeInterval) -> List[float]: if not HelperUtils.is_valid_ticker(symbol): raise IllegalArgument('%s is not a valid ticker symbol' % symbol) yahoo_time_interval = HelperUtils.convert_TimeInterval_to_YahooTimeInterval(time_interval) stock_native_curr = self.yfgateway.get_native_currency_for_ticker(symbol) exch_rate = self._cached_get_exchange_rate(stock_native_curr, start_date, end_date, yahoo_time_interval) historical_prices = self.yfgateway.get_historical_stock_prices(symbol, start_date, end_date, yahoo_time_interval) return [ rate * price for rate, price in zip(exch_rate, historical_prices) ]
Instance variables
var global_exchange_list
Stock exchanges for which data is available
var yfgateway
A pointer to the YahooFinanceGateway module
var yfmetadata
A yahoo finance metadata client
class YahooFinanceMetadataClient
A StockMetadataClient that retrieves metadata for most stocks and exchanges in YahooFinance
class YahooFinanceMetadataClient(AbstractStockMetadataClient): """ A StockMetadataClient that retrieves metadata for most stocks and exchanges in YahooFinance """ file_name = abs_file_path '''The absolute path to the file where the metadata is stored''' def _get_all_stock_metadata(self) -> List[Dict[str, str]]: ''' Return all stock metadata mappings in the classes file path ''' stock_md = [] with open(self.file_name, 'r') as metadata_file: reader = csv.DictReader(metadata_file) for row in reader: stock_md.append(row) return stock_md def _retrieve_stock_metadata(self, filter_func: Callable[[Dict[str, str]], bool], transformator: Callable[[Dict[str, str]], Any]) -> Iterable[Any]: """ Return a generator that applies a transformation to every valid stock metadata object Args: * `filter_func` - the function that determines if a given row in a data file should be used * `transformator` - the transformation that should be applied to every valid stock metadata object Returns: > A generator with the transformed stock metadata object Note: > Throws IOException """ filtered_stock_md = [md_obj for md_obj in self._get_all_stock_metadata() if filter_func(md_obj)] return [transformator(md_obj) for md_obj in filtered_stock_md] @staticmethod def _filter_for_exchange(exchange: str) -> Callable[[Dict[str, str]], bool]: return lambda stock_md: stock_md['Exchange'] == exchange @staticmethod def _retrieve_ticker() -> Callable[[Dict[str, str]], str]: return lambda stock_md: stock_md['Ticker'] def get_stocks_in_exchange(self, exchange: str) -> Iterable[str]: """ Retrieve all ticker symbols that correspond to an exchange Args: * `exchange` - a string repr of the exchange for which the tickers should be retrieved Returns: > A generator over the tickers present in the exchange """ return self._retrieve_stock_metadata( self._filter_for_exchange(exchange), self._retrieve_ticker())
Ancestors (in MRO)
- YahooFinanceMetadataClient
- AbstractStockMetadataClient
- builtins.object
Class variables
var file_name
The absolute path to the file where the metadata is stored
Static methods
def get_stocks_in_exchange(
self, exchange)
Inheritance:
AbstractStockMetadataClient
.get_stocks_in_exchange
Retrieve all ticker symbols that correspond to an exchange
Args:
exchange
- a string repr of the exchange for which the tickers should be retrieved
Returns:
A generator over the tickers present in the exchange
def get_stocks_in_exchange(self, exchange: str) -> Iterable[str]: """ Retrieve all ticker symbols that correspond to an exchange Args: * `exchange` - a string repr of the exchange for which the tickers should be retrieved Returns: > A generator over the tickers present in the exchange """ return self._retrieve_stock_metadata( self._filter_for_exchange(exchange), self._retrieve_ticker())
class YahooFinanceStatefulClient
An concrete stock database client that implementes the methods that need to be implemented for interoperability with the rest of the code. This client is based on the yahoo finance API.
This client is stateful and is as such not threadsafe.
class YahooFinanceStatefulClient(AbstractStatefulStockDatabaseClient): ''' An concrete stock database client that implementes the methods that need to be implemented for interoperability with the rest of the code. This client is based on the yahoo finance API. This client is stateful and is as such not threadsafe. ''' def __init__(self): self.yfclient = YahooFinanceClient() time_interval = TimeInterval.weeks end_date = date.today() - timedelta(days=date.today().weekday()) start_date = end_date - timedelta(weeks=77) self.set_state(start_date, end_date, time_interval) def set_state(self, start_date: date, end_date: date, time_interval: TimeInterval): self.start_date = start_date self.end_date = end_date self.time_interval = time_interval def get_historical_stock_prices(self, symbol: str) -> List[float]: return self.yfclient.get_historical_stock_prices(symbol, self.start_date, self.end_date, self.time_interval) def get_historical_data_for_stocks(self, symbols: List[str]) -> Dict[str, List[float]]: return self.yfclient.get_historical_data_for_stocks(symbols, self.start_date, self.end_date, self.time_interval) def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: return self.yfclient.get_historical_data_for_stocks_in_exchange(exchange_name, self.start_date, self.end_date, self.time_interval, threshold_mcap=threshold_mcap, mcap_limit_in_millions=mcap_limit_in_millions) def get_available_exchanges(self) -> List[str]: return self.yfclient.get_available_exchanges()
Ancestors (in MRO)
- YahooFinanceStatefulClient
- AbstractStatefulStockDatabaseClient
- AbstractStockDatabaseClient
- builtins.object
Static methods
def __init__(
self)
Initialize self. See help(type(self)) for accurate signature.
def __init__(self): self.yfclient = YahooFinanceClient() time_interval = TimeInterval.weeks end_date = date.today() - timedelta(days=date.today().weekday()) start_date = end_date - timedelta(weeks=77) self.set_state(start_date, end_date, time_interval)
def get_available_exchanges(
self)
This method returns a list of stock exchanges for which this client can return data
def get_available_exchanges(self) -> List[str]: return self.yfclient.get_available_exchanges()
def get_historical_data_for_stocks(
self, symbols)
This method returns historical data for all the provided ticker symbols in USD
Args:
symbols
- the ticker symbols for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
This method will skip all ticker symbols that are invalid for some reason or another
def get_historical_data_for_stocks(self, symbols: List[str]) -> Dict[str, List[float]]: return self.yfclient.get_historical_data_for_stocks(symbols, self.start_date, self.end_date, self.time_interval)
def get_historical_data_for_stocks_in_exchange(
self, exchange_name, threshold_mcap=True, mcap_limit_in_millions=300)
This method returns historical data for all ticker symbols in an exchange in USD that meet a given mcap_limit
Args:
exchange_name
- string repr of the exchange for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time rangethreshold_mcap
- whether or not the stocks in the exchange should be filtered based on market capitalizationmcap_limit_in_millions
- the market cap threshold
Returns:
A dict mapping a ticker symbol to a list of historical stock price data in USD. The most recent price should be at index 0.
Note:
The returned data is for the ticker symbols found in the provided exchange.
def get_historical_data_for_stocks_in_exchange(self, exchange_name: str, threshold_mcap: bool =True, mcap_limit_in_millions: int =300) -> Dict[str, List[float]]: return self.yfclient.get_historical_data_for_stocks_in_exchange(exchange_name, self.start_date, self.end_date, self.time_interval, threshold_mcap=threshold_mcap, mcap_limit_in_millions=mcap_limit_in_millions)
def get_historical_stock_prices(
self, symbol)
This method retrieves historical price data for a ticker symbol in USD.
Args:
symbol
- the ticker symbol for which you wish to retrieve datastart_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
A list of the historical stock price data in USD. The most recent price should be at index 0.
Note:
In theory, the default currency does not need to be USD, it only needs to be consistent within the application. In practice, USD is by far the most commonly used and hence is preferred
def get_historical_stock_prices(self, symbol: str) -> List[float]: return self.yfclient.get_historical_stock_prices(symbol, self.start_date, self.end_date, self.time_interval)
def set_state(
self, start_date, end_date, time_interval)
This method sets the class wide execution state for stateful database clients
Args:
start_date
- the start of the time range for which you would like to get dataend_date
- the end of the time range for which you would like to get datatime_interval
- the time interval at which you would like to get data for the given time range
Returns:
None
def set_state(self, start_date: date, end_date: date, time_interval: TimeInterval): self.start_date = start_date self.end_date = end_date self.time_interval = time_interval
Instance variables
var yfclient
class YahooTimeInterval
An enum to encapsulate yahoo finance specific time interval
class YahooTimeInterval(Enum): ''' An enum to encapsulate yahoo finance specific time interval ''' weeks = ('w', 7) '''TimeInterval enum value to denote a time interval of 1 week i.e. 7 days''' def __init__(self, symbol, days): self.symbol = symbol # string specific to yahoo finance that represents that time range self.days = days # interval length in days
Ancestors (in MRO)
- YahooTimeInterval
- enum.Enum
- builtins.object
Class variables
var name
var value
var weeks
TimeInterval enum value to denote a time interval of 1 week i.e. 7 days
Sub-modules
This is a module that defines functions that can be used to diversify a portfolio given a set of required stocks and the much larger set of viable stocks.
This module provides helper functions that extract information from stocks and diversify a portfolio
This is a module that defines functions that can be used to optimize the weights in a diversified portfolio
arkstock.yahoo_finance_gateway
A module that defines a set of method used to retrieve data from yahoo finance.
Due to the singleton nature of modules in python, all methods must inherently be functional.


Because Microsoft has no longer provided support and updates for Internet explorer 11 since January 12, 2016. That is, without browser itself security updates, your PC can be vulnerable to attack by harmful viruses, spyware, malware that will steal or damage your important data and information. Further information, please refer to Microsoft's support for earlier versions of Internet Explorer. Note: Support for Windows XP and earlier versions of Internet Explorer are ceased.
More advanced browsers
The following browsers are recommended. If your computer has any of them, you can use it directly .
Why does this page appear?
If you don't know how to upgrade your browser, ask your computer-savvy friends. If you are a browser user of 360, browser, QQ browser, sogou, this will tell you to update to the latest version.
Boycott IE6, IE7, IE8, IE9, IE10
In order to compatible with this once dominant browser, web designers need to do a lot of code work. For the average user, the low versions of Internet Explorer is a security risk, and several major Trojan viruses in the history of Windows have been exploited by using Internet explorer security vulnerabilities. So please join us in resisting the earlier versions of IE!


Daesung required to pay 1.2 billion KRW (approximately $1.025 million) in taxes on controversial building
by sl278Big Bang's Daesung will have to pay more than 1.2 billion KRW (~1.025 million USD) worth in taxes due to the operation of illegal adult businesses without a license.
Channel A News reported that the tax authorities sent a bill to Daesung to pay the tax of 1.2 billion KRW. This includes the acquisition tax, property tax, and additional local tax which adds up to about 1.2 billion KRW. This tax was levied because the tax authority discovered that Daesung was running unlicensed adult businesses when they had previously believed there were normal businesses in the building.
The popular star was previously under controversy after it was revealed that illegal activities were happening in his building, although Daesung has stated that he had no knowledge of the businesses. Investigations are also being conducted on the entertainment venues in the building under suspicion of individual consumption tax evasion by the Seoul Regional Tax Service.
Channel A also reported that Daesung borrowed 5 billion KRW (~4.27 million USD) from a loan company and used the building as collateral. When Daesung originally purchased the building in 2017, he purchased it for 31 Billion KRW ($26.5 million) with a loan from the bank. It is believed Daesung took out the 5 billion KRW loan to pay back the original 2017 loan.
What do you think of this situation?


As of June 2017, component inheritance is not supported by Hibernate (this may change in the future):
More information:
• Hibernate ORM: HHH-1152
• Hibernate ORM: HHH-1910
• Hibernate ORM: HHH-3455
• Stackoverflow question
• Stackoverflow question
• Stackoverflow question
What happens when an embeddable class subclasses another embeddable class? Only the direct fields of the exact subclass declared in the entity that contains the component (the embeddable object) will be persisted. Also, the component creation may silently fail when loading the entity that contains the component (Hibernate will set it to null).


--- layout: post title: Io, Day 1 date: 2011-05-19 22:08:00.000000000 +02:00 categories: [] tags: - Io - Seven Languages in Seven Weeks status: publish type: post published: true meta: _edit_last: '1' _wp_old_slug: io-day-one author: login: admin email<EMAIL_ADDRESS>display_name: Philippe first_name: Philippe last_name: Vlérick excerpt: !ruby/object:Hpricot::Doc options: {} ---
First day on Io, which I never heard about…
Io is a dynamic prototype-based programming language.
Apart from JavaScript, I never heard any other prototype-based language. And even in JavaScript, I don’t know exactly what this implies, so my expectation of learning new things while studying Io is quite high, which is good!
By the way, Io is the name of a nymph in Greek mythology and one of Jupiter’s satellite. Believe it or not, but I knew that before encountering the language, and still wonder if there is any relation whatsoever, but didn’t find anything about the language name’s origin…
(to me it was the perfect excuse to put a picture of a sexy girl on my blog and get more traffic!)
Anyway, first task, get Io to run on my box, namely a Windows 7 x64 machine. After downloading the sources and trying to build them according to the readme.txt, lots of errors…
Finally, I found that the best solution is to download the binaries which work fine on my machine. That’ll do for now.
I also needed some reference to lookup to when in doubt, and that’s all available here.
So, on day one, no real assignments, or at least nothing that can’t be found in 30 seconds on the official website, so I’m not going to replicate this here.