Ok, I couldn’t resist anymore after this post on Techcrunch. It talks about the new app that Jajah has just released which is a kind of plugin for your mobile phone, or for my Nokia N70 at least. It simply reroutes my calls via Jajah, making international calling a lot cheaper. I already have a phone flatrate but this only applies for germany. I will have to turn it off while being in Germany and calling people in Germany. But to EU countries and the USA it costs me 69 cents per minute with Base and 15 cents per minute with Jajah. Now that’s nice.
Dear people of Jajah. It would be nice if I would be able to configure the plugin to where it applies. All of +49 should be off limits for the app because Base is just cheaper for me.
Update: Just had a chat with Frederik Hermann, who is currently at DEMO in the US and Marketing Manager for Jajah. When he called it was close to 1am his. After a short chat about the people we are both connected to on OpenBC, and general talk about Jajah, he quickly explained to me how the Jajah client behaves.
The thing is that all my worries are settled. In the main settings you can set your own international prefix, 49 for me. This means that for all numbers that are +49, the Jajah plugin will not do anything. Then there is a kind of include list in the settings. This routing table holds "+" and "00" as default settings, meaning that it will handle all calls (ex. those with country code from your own international settings) that look like the above. Very nice indeed. So if I would just like Jajah to handle my calls to certain numbers, I could do that.
Oh the mobile carriers will get pissed at this. But Jajah seems to be developing nicely, and have their development in Israel, who seem to be doing a nice job. As I heard now the founders are not really living in the US… mainly because they don’t live anywhere due to the amount of traveling to exotic countries :) Looking forward to seeing more from them. Thanks again to Frederik for taking the time late at night, from a conference none the less.
Update 2: Seem like Sarik Weber is going into a similar space. He is leaving OpenBC, having been their first employee, and is starting something in the mobile space around cheaper calls. See more here.
Update 3: Sarik Weber is starting a company called cellity, a least cost router for your mobile phone. Registered.
Technorati Tags: Base, Free Calls, Jajah, Mobility


We have recently made changes on our price deals for you to relax and enjoy a low-cost vacation.
Let us help you find the right place for you to spend time with family or yourslef.
By looking through all of our images, we hope you find it easier to look for the perfect place.
Feel free to call us<PHONE_NUMBER>). Our online operators are vulnerable at any time for your service.


 Abuse this in lane. If
you're not able to get last hits because her animation blows, at least make life annoying for whoever is
in your lane.
- Don't forget to turn off Frost Arrows if you have Lifesteal. The orbs don't stack.
- The knockback on Gust doesn't break channels, so don't bother using it to cancel a TP.
- However, the knockback does move enemies, making it cancel Templar Assassin's Meld and Shadow Amulet.
- You have to activate Precision Aura if you want ranged summons like Visage's Familiars or
Enigma's Eidolons to benefit from the Aura.
- Post-Agh, all split arrows will have whatever on-hit effects the main arrow has. All 3 arrows will independently roll for crit/bash/lightning and all 3 arrows will have the Skadi/Deso/lifesteal effect.
 There is nothing Drows hate more than someone in melee range of her. If a hero is within
400 range of her, she loses the agility bonus from Marksmanship, so try closing the distance even if you're
a ranged hero.


Add
Kodify Restaurants
Options
Search
Are you sure?
Yes
No
Rate this restaurant
Food
Service
Speed
Submit
Login
Register
Logout
Please sign in
Username:
Password:
Sign in
register
Send next url to your friend:
Registration form
Username (only letters and numbers, minimun 5 chars):
Password (minium 5 chars):
Repeat your password:
Passwords do not match
Email:
Submit
Your current address
Max distance:
Tags:
Submit
Add new restaurant
Name:
Address:
Website:
Price:
Description:
Tags:
Submit


This page will contain the full process that we use every time we update our verifiers. We have a total of 9 verifiers in the cycle: ⛵ Argo 746, ⛵ Argo 752, 😱 😱 Killr 😱 😱, and 6 that we have not disclosed. We update at least one verifier with all new code before release, and we update the rest of our verifiers shortly after release.
For us, the update process is largely manual. We have various automatic processes that we used before public release and continue to use for convenience in testnets, but we do not feel that the convenience of an automatic process for updating our verifiers on the production blockchain would justify the risk of something going wrong. Regardless of whether you update your verifiers manually or through an automatic process, the only responsible way to update verifiers is serially. If you update all of your verifiers at once, you are risking the stability of your sentinel and leaving your verifiers vulnerable to removal from the cycle if something goes wrong.
Even in our testnet update scripts, we always update serially. One verifier is updated, and then the script queries the verifier until it is confirmed to be tracking the blockchain and producing blocks again. In a production environment, some element of stochasticism is desirable, as well, so that timing of updates does not make ownership of verifiers too obvious. For critical updates, we spread out updating of our 9 verifiers over at least several hours. For less time-sensitive updates, we typically spread out updating over several days.
In these notes, we will describe our manual process. Part or all of this could easily be automated.
This page is not yet complete.


Chapter 3 Risk of Violent Recidivism
3.1 Setup
## Loading required package: pacman
pacman::p_load(
tidyverse, # tidyverse packages
conflicted, # an alternative conflict resolution strategy
ggthemes, # other themes for ggplot2
patchwork, # arranging ggplots
scales, # rescaling
survival, # survival analysis
broom, # for modeling
here, # reproducibility
glue # pasting strings and objects
)
# To avoid conflicts
conflict_prefer("filter", "dplyr")
## [conflicted] Will prefer dplyr::filter over any other package
## [conflicted] Will prefer dplyr::select over any other package
3.2 Load data
## Warning: Duplicated column names deduplicated: 'decile_score' =>
## 'decile_score_1' [40], 'priors_count' => 'priors_count_1' [49], 'two_year_recid'
## => 'two_year_recid_1' [54]
##
## ── Column specification ─────────────────────────
## cols(
## .default = col_double(),
## name = col_character(),
## first = col_character(),
## last = col_character(),
## compas_screening_date = col_date(format = ""),
## sex = col_character(),
## dob = col_date(format = ""),
## age_cat = col_character(),
## race = col_character(),
## c_jail_in = col_datetime(format = ""),
## c_jail_out = col_datetime(format = ""),
## c_case_number = col_character(),
## c_offense_date = col_date(format = ""),
## c_arrest_date = col_date(format = ""),
## c_charge_degree = col_character(),
## c_charge_desc = col_character(),
## r_case_number = col_character(),
## r_charge_degree = col_character(),
## r_offense_date = col_date(format = ""),
## r_charge_desc = col_character(),
## r_jail_in = col_date(format = "")
## # ... with 14 more columns
## )
## ℹ Use `spec()` for the full column specifications.
glue("N of observations (rows): {nrow(two_years_violent)}
N of variables (columns): {ncol(two_years_violent)}")
## N of observations (rows): 4743
## N of variables (columns): 54
3.3 Wrangling
3.3.1 Create a function
wrangle_data <- function(data){
df <- data %>%
# Select variables
select(age, c_charge_degree, race, age_cat, v_score_text, sex, priors_count,
days_b_screening_arrest, v_decile_score, is_recid, two_year_recid) %>%
# Filter rows
filter(days_b_screening_arrest <= 30,
days_b_screening_arrest >= -30,
is_recid != -1,
c_charge_degree != "O",
v_score_text != 'N/A') %>%
# Mutate variables
mutate(c_charge_degree = factor(c_charge_degree),
age_cat = factor(age_cat),
race = factor(race, levels = c("Caucasian","African-American","Hispanic","Other","Asian","Native American")),
sex = factor(sex, levels = c("Male","Female")),
v_score_text = factor(v_score_text, levels = c("Low", "Medium", "High")),
# I added this new variable to test whether measuring the DV as a binary or continuous var makes a difference
score_num = as.numeric(v_score_text)) %>%
# Rename variables
rename(crime = c_charge_degree,
gender = sex,
score = v_score_text)
return(df)}
3.3.2 Apply the function to the data
## [1] "age" "crime"
## [3] "race" "age_cat"
## [5] "score" "gender"
## [7] "priors_count" "days_b_screening_arrest"
## [9] "v_decile_score" "is_recid"
## [11] "two_year_recid" "score_num"
## # A tibble: 5 x 12
## age crime race age_cat score gender priors_count days_b_screenin…
## <dbl> <fct> <fct> <fct> <fct> <fct> <dbl> <dbl>
## 1 69 F Other Greate… Low Male 0 -1
## 2 34 F Afri… 25 - 45 Low Male 0 -1
## 3 44 M Other 25 - 45 Low Male 0 0
## 4 43 F Other 25 - 45 Low Male 3 -1
## 5 39 M Cauc… 25 - 45 Low Female 0 -1
## # … with 4 more variables: v_decile_score <dbl>, is_recid <dbl>,
## # two_year_recid <dbl>, score_num <dbl>
3.3.3 Descriptive analysis
Score distribution
df %>%
group_by(score) %>%
count() %>%
ggplot(aes(x = score, y = n)) +
geom_col() +
labs(x = "Score",
y = "Count",
title = "Score distribution")
Score distribution by race
3.3.4 Modeling
After filtering out bad rows, our first question is whether there is a significant difference in COMPAS scores between races. To do so we need to change some variables into factors, and run a logistic regression, comparing low scores to high scores.
model_data <- function(data){
# Logistic regression model
lr_model <- glm(score ~ gender + age_cat + race + priors_count + crime + two_year_recid,
family = "binomial", data = data)
# OLS
ols_model1 <- lm(score_num ~ gender + age_cat + race + priors_count + crime + two_year_recid,
data = data)
ols_model2 <- lm(v_decile_score ~ gender + age_cat + race + priors_count + crime + two_year_recid,
data = data)
# Extract model outcomes with confidence intervals
lr_est <- lr_model %>%
tidy(conf.int = TRUE)
ols_est1 <- ols_model1 %>%
tidy(conf.int = TRUE)
ols_est2 <- ols_model2 %>%
tidy(conf.int = TRUE)
# AIC scores
lr_AIC <- AIC(lr_model)
ols_AIC1 <- AIC(ols_model1)
ols_AIC2 <- AIC(ols_model2)
list(lr_est, ols_est1, ols_est2, lr_AIC, ols_AIC1, ols_AIC2)
}
<IP_ADDRESS> Model comparisons
glue("AIC score of logistic regression: {model_data(df)[4]}
AIC score of OLS regression (with categorical DV): {model_data(df)[5]}
AIC score of OLS regression (with continuous DV): {model_data(df)[6]}")
## AIC score of logistic regression: 3022.77943765996
## AIC score of OLS regression (with categorical DV): 5414.49127581608
## AIC score of OLS regression (with continuous DV): 15458.3861723106
<IP_ADDRESS> Logistic regression model
lr_model <- model_data(df)[1] %>%
data.frame()
lr_model %>%
filter(term != "(Intercept)") %>%
mutate(term = gsub("race|age_cat|gender","", term)) %>%
ggplot(aes(x = fct_reorder(term, estimate), y = estimate, ymax = conf.high, ymin = conf.low)) +
geom_pointrange() +
coord_flip() +
labs(y = "Estimate", x = "",
title = "Logistic regression") +
geom_hline(yintercept = 0, linetype = "dashed")
interpret_estimate <- function(model){
# Control
intercept <- model$estimate[model$term == "(Intercept)"]
control <- exp(intercept) / (1 + exp(intercept))
# Likelihood
model <- model %>% filter(term != "(Intercept)")
model$likelihood <- (exp(model$estimate) / (1 - control + (control * exp(model$estimate))))
return(model)
}
interpret_estimate(lr_model) %>%
mutate(term = gsub("race|age_cat|gender","", term)) %>%
ggplot(aes(x = fct_reorder(term, likelihood), y = likelihood)) +
geom_point(size = 3) +
coord_flip() +
labs(y = "Likelihood", x = "",
title ="Logistic regression") +
scale_y_continuous(labels = scales::percent_format(accuracy = 1)) +
geom_hline(yintercept = 1, linetype = "dashed")


{% include head.html %}
{% include header.html %}
IKATS
Innovative Toolkit for Analysing Time Series
{{ content }}
{% include footer.html %}


Natural Language Processing, usually shortened as NLP, is a branch of artificial intelligence that deals with the interaction between computers and humans using the natural language.
The ultimate objective of NLP is to read, decipher, understand, and make sense of the human languages in a manner that is valuable.
Giving the MID Server the proper tools to process the information
node.js
natural.js
express.js
brain.js
Using flow designer, visual task boards, reporting and task tables to bring an idea togehter. Tying together different parts of the platform to create a valuable process.
Talking to systems in and out of ServiceNow
Sending updates from ServiceNow to MS Teams via webhooks, Outlook via the notifications.
Updating Visual Task Boards with important information at the right time.
Copyright © Sense IT 2019


 Since it took an hour and 45 minutes to get between my hotel in Madrid and my workplace in the suburbs of Toledo — via foot, Metro, Light Rail, and car — I figured the weight difference vs a MacBook (2 pounds plus the weight difference of the p/s) would pay for any hit I would take on purchasing it and reselling it.
Since then I've hung onto it, but considered selling it over and over, something I've never really done with a computer. Beautiful machine, it's light, bright screen, great keyboard... but the thing was damn slow. I experienced the CPU dropout watching video and especially with the external monitor attached. I experimented with CoolBook to see if it would help (it did, a lot, but then the machine would crash once a week or so - fine for a windows machine but not what I'm used to with OSX). When an update came out from Apple that was supposed to fix it, I was really happy.
It does fix one of the cores dropping out and the CPU going insane on the remaining core, but Flash was unusable. Or more specifically videos were unwatchable. I can't believe that in 2009 watching videos on a 1.6 GHz Core 2 Duo would be so bad. I was still thinking about selling it (same model refurbished is $1099).
Today I installed Flash 10 and then checked out a couple of tough videos. Here, watch this one, a really interesting bouldering sequence. This ones really good, bouldering in New York State, it's awesome: The Season. A couple of little hiccups, but watchable.
Hmm... +1 climbing fix, maybe this little box can stay.
About this entry
You"re currently reading "Today The MacBook Air doesn't suck half as bad," an entry on a crick in the net
- Published:
- 01.19.09 / 10pm
- Category:
- Gadget-o-rama
My climbing laptop is an Asus S5N:
Intel Pentium M 1.50GHz
768 MB Memory
Ubuntu 8.04 (Hardy)
2.8 lbs and 12.1″ Screen. Perfect climbing laptop.
I browse using Firefox 3.0.5 and flash video chuggs. It really pisses me off. I'm currently taking your advice and I am installing the Flash 10 .deb package. I'll let you know if it makes any difference.


{pos ?= {}}
Expressions can have filters, to alter the output quoting. In example 1, Functions can also have function filters (similar to Python decorators). In example 2, a function named Note that the * is picking up Example 3 is a function expression, which is usually preferable to a function filter where both could be used. Function filters are better used for behaviors like caching that change how a function behaves.
{builder::as_code('examples/filters.html')}{greeting|trim}
will output the greeting after trimming the excess spaces.
fn
is defined that will also remove extra whitespaceGenerally the builtin filters will support being used as function filters or expression filters.
name
as a parameter -- free variables
(eg, not a param
target, assigment, or loop iteration variable) will be
added automatically.


This module allows you to search various types of documentation stored
on your system. Just enter the search terms into the Search for
field and select the kinds of documentation using the checkboxes next
to Search in. If you enter more than one word to search for,
select the Match all option to find documents that contain all
the words, or Match any to find documents that contain any of
the words.
The Match option determines whether Webmin looks at just file names
for the search terms, or the file contents as well. Searching contents can
return more results, but will take longer.
When you are ready, click the Search button. A page listing all
the documentation that matches will be displayed, from which you can click
on a specific file or page to display it.


Second opinion, when patient goes to see another specialist in the same field, is a growing phenomenon. More and more major institutions and 2nd opinion consulting agencies are offering this service to patients. There was even a study published recently in the Journal of Evaluation in Clinical Practice conducting at Mayo clinic indicating that as many as 88% of patients who came for a 2nd opinion, ended up with an alternative diagnosis, and with this new diagnosis their plan of care was changed and so were their lives. Only astonishing 12% of patients that sought 2nd opinion learned that their initial diagnosis was indeed correct.
Neurology with its complexity and confounding factors like polypharmacy, chronic medical problems, occupational injuries, and psychiatric issues, is a field where rate of misdiagnosis is likely much higher. While in the Mayo study, 1 out of 5 patients was misdiagnosed by an internal medicine physician, in neurology, I would estimate, it is about 1 out 3 patients that end up with a completely different diagnosis by the end of the 2nd opinion visit.
So what is the reason for this degree of medical error in a standard neurological practice? The reasons are many, but one major factor is increasing shortage of neurologists. Because of this many primary care doctors end up treating neurological diseases and by the time patient ends up seeing a neurologist, the disease may have progressed and valuable time was lost. The neurologist that ends up taking over the care of some of these patients does not have the early signs that are helpful to make a much more specific diagnosis, and can end up re-prescribing the same medication. I call this sort of practice “maintaining status quo care” or “do not rock the boat” treatment plan with neurologist keeping the same plan originally started by a primary care physician that is inappropriate for the disease process.
Even if a patient is lucky to get evaluated by the neurologist at the very beginning of the symptom onset, it is still not a guarantee this patient will end up with the correct diagnosis. The reason for this may have to do with the degree of burnout experienced by neurologists.
According to 2017 study by the American Academy of Neurology, 60% of neurologists in the United States are experiencing some form of burnout. The implications of the burned out physician are that the quality of care suffers. Burned out neurologists lack compassion and empathy, feeling detached from the humanity aspect of medicine, and presence of compassion and empathy are vital to understanding the root cause of patient’s neurological complaints. Burned out neurologists simply cannot practice medicine to the same degree of quality as those that are not burned out, and that is only 40% of the neurologist in the US. Understandably, patients’ misdiagnosis and mistreatment occurs more commonly for those patients who end up in the hands of a burned out specialist who is “checked out” and is just going through the motions, treating patients like objects just to get to the end of the day.
But this gets worse. With the unappealing statistics of neurologists being one of the most burned out specialists while being the least compensated for their services and having the worst work to life balance comparing to other specialists, fewer medical students end up pursuing neurology residency thus less neurologists are available to handle the neuro issues. At the same time, as baby boomers age, people living longer developing neuro issues, health problems becoming more complex, the need for appropriate knowledge and management of neurological diseases is only increasing. Thus we end up with a vicious cycle where neurologists are even more overwhelmed and overworked and primary care physicians end up being the ones managing the majority of neurological diseases, despite not being trained at neurological pattern recognition.
Another factor is that some neurologists, particularly the ones practicing neurology in the private practice setting, are at the mercy of government reimbursement for the services. There have been cuts to neurological procedures of up to 40% and some private practice neurologists can no longer afford to stay in practice and compete with large institutions and hospitals. Thus neurologists end up getting employed by these hospitals that operate on RVU model, i.e. productivity incentivised model. This leads to the doctor being like a hamster on a treadmill, on average having only 20 minutes to spend with the follow-up patient, 45 min with the new patient, of which 80% is spent staring and typing on the laptop. I have been to many hospital employed neurology practices and doctors there are quite stressed out. They don’t even get a proper lunch break, gobbling down sandwiches while answering patient’s emails or phone calls, or attending to some urgent administrative matter. These neurologists see their last patient at 4:30 pm but still go home only after 6 or 7 pm due to having to finish those notes in their EMR systems, making sure all of the quality measures have been met, so that government does not fine them later on.
Some of my patients tell me that neurologists in those practices spent with them about 10 minutes, and sometimes as little as 2 minutes. This is usually in a practice where a nurse practitioner, medical assistant or a physician assistant is employed for the sole purpose of cutting down on the physician’s time with patient. The midlevel provider asks the questions and spends majority of the time with the patient, followed by physician walking in, quickly examining the patient and saying good buy after just a few minutes of interaction. The consequence of such a limited time spent by the neurologist evaluating the patient can be erroneous diagnosis with wrongful choice of treatment.
The correctness of neurological diagnosis depends highly on the thoroughness of the neurologist obtaining history of present illness. Good history taking sometimes makes ancillary testing unnecessary. The types of questions asked of a patient depend on the interviewer’s deep understanding of human brain, musculoskeletal and nervous system. If most of the questions are asked by a midlevel provider or a PCP, then the quality of the obtained information may be compromised.
So is it surprising why so many of neurological patients, with whom 2nd opinion experts spend three times longer than a typical neurologist, end up with a new diagnosis and a revised treatment plan?
Let me share a couple of stories highlighting the danger of misdiagnosis in neurology.
I had a patient that came to me for a 2nd opinion, who was heartbroken when was diagnosed with a neurodegenerative condition, called Parkinson’s disease based on head tremor. After thorough examination, it was found that the actual diagnosis was dystonic tremor of the neck from the neck muscles pulling head forward (anterocollis). I now treat that patient successfully every 3 months with botulinum toxin injection and as a result, the tremor and the neck pain have fully resolved. This patient was relieved that dire diagnosis of Parkinson’s was incorrect. Fortunately this patient came for a 2nd opinion before having been started on parkinson specific medication which would have exacerbated the tremor and caused the neck to spasm more. The patient also may have gotten an unnecessary brain PET scan which would have cost a significant amount of money.
What about the big name universities, don’t they guarantee to provide a correct diagnosis?
Academic establishments are now offering 2nd opinion consults remotely, via teleservices and it might be a great option for some. However, here are a couple of examples that speak to the notion that big name does not necessarily translate to the highest quality of neurological care. I saw a patient who ended up with an invasive deep brain stimulator implanted device for something that could have been handled with cognitive behavioral therapy and an anti-depressant. This patient had somatic symptom disorder where psychiatric symptoms manifest in physical symptoms, in this case, muscle spasms. The original misdiagnosis of refractory generalized dystonia came from the highly reputable university that had arranged for the patient to get deep brain stimulation, which only made the symptoms worse post surgery.
There was another patient I saw for a 2nd opinion who went to the center of excellence at another big university with multidisciplinary specialists.The patient paid out of pocket hundreds of dollars. The neurologist there spent 20 minutes just to tell the patient he did not know what the diagnosis was, did not do a thoroughly exam, did not look at the tongue. When the patient came to me, I spent an hour and my neurological examination revealed tongue and muscle fasciculations and atrophy with EMG later confirming the diagnosis of ALS (Lou Gehrig's disease). This patient is now receiving treatment for ALS that became available this year.
To conclude, with more awareness of the fact that neurology is a field with high chance of a wrong diagnosis, my hope is to spare patients from unnecessary burden of wrong medications, numerous tests and imaging and increase patient’s quality of life one correct diagnosis at a time.
By Galina Nikolskaya, MD Board Certified Neurologist


NetData is one of the most popular and most powerful monitoring tools that provide a web interface. It is a system for distributed real-time performance and health monitoring. It provides unparalleled insights, in real-time, of everything happening on the system it runs (including applications such as web and database servers), using modern interactive web dashboards.
NetData is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.
Captain provides a simple interface to enable NetData on your captain instance. Currently, Captain only supports installing NetData on your leader node, i.e., the node where Captain instance resides. This limitation is due to a limitation in Docker interface, see here .
For more details regarding NetData, visit their GitHub page .
NetData offers multiple ways for you to receive notifications if something is going wrong with your server resource usage. All notification options are completely OPTIONAL.


| Quest | Type | Recommended to Complete? | Notes |
| A Fowl Shortage | Daily Quest Cooking | Yes | |
| A Round for the Guards | Daily Quest Cooking | Yes | |
| Can't Get Enough Spice Bread | Daily Quest Cooking | Yes | |
| Cold Water Fishing | Daily Quest Fishing | Yes | |
| Fish fer Squrky | Daily Quest Fishing | Yes | |
| I Need to Cask a Favor | Daily Quest Cooking | Yes | |
| Keepin' the Haggis Flowin' | Daily Quest Cooking | Yes | |
| Live Bait | Daily Quest Fishing | Yes | |
| One fer the Ages | Daily Quest Fishing | Yes | |
| The Darkmoon Faire | Quest | Yes | |
| The Gnomish Bait-o-Matic | Daily Quest Fishing | Yes |


About Us
Locations
Hours
Specials
Offers
Contact Us
Cookies made into the shape of a salmon, and actually made with salmon
Locationwise Sales Projection Report
Pike Place Center
SeaTac Airport
Seattle Center
Capitol Hill
Alki


Tuesday — November 5, 2019
latest news >
- Napier López
- • 30 minutes ago
- Napier López
- • 3 hours ago
- Ivan Mehta
- • 5 hours ago
- TNW Deals
- • 7 hours ago
- Napier López
- • 9 hours ago
- Ravie Lakshmanan
- • 35 minutes ago
- Napier López
- • 10 minutes ago
- Satoshi Nakaboto
- • 9 hours ago
Residently
INSHUR
ReadMe.io
Koru Kids
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well
Apps >
Nice: Firefox 69 now blocks cryptominers and tracking cookies by default
- Chinese face-swapping app goes viral, invites criticism over privacy clause
- How to disable Chrome notifications and finally browse in peace
- YouTube settles with FTC over child privacy violations
- How to see what ads Instagram thinks you'll care about
- YouTube's adding more age gates to YouTube Kids -- this should go well


L-Systems-Tree-Modelling.test_GUI
¶
Module Contents¶
Classes¶
A class whose instances are single test cases. |
Attributes¶
- L-Systems-Tree-Modelling.test_GUI.app¶
- class L-Systems-Tree-Modelling.test_GUI.GUITest(methodName='runTest')¶
Bases:
unittest.TestCase
A class whose instances are single test cases.
By default, the test code itself should be placed in a method named ‘runTest’.
If the fixture may be used for many test cases, create as many test methods as are needed. When instantiating such a TestCase subclass, specify in the constructor arguments the name of the test method that the instance is to execute.
Test authors should subclass TestCase for their own tests. Construction and deconstruction of the test’s environment (‘fixture’) can be implemented by overriding the ‘setUp’ and ‘tearDown’ methods respectively.
If it is necessary to override the __init__ method, the base class __init__ method must always be called. It is important that subclasses should not change the signature of their __init__ method, since instances of the classes are instantiated automatically by parts of the framework in order to be run.
When subclassing TestCase, you can set these attributes: * failureException: determines which exception will be raised when
the instance’s assertion methods fail; test methods raising this exception will be deemed to have ‘failed’ rather than ‘errored’.
- longMessage: determines whether long messages (including repr of
objects used in assert methods) will be printed on failure in addition to any explicit message passed.
- maxDiff: sets the maximum length of a diff in failure messages
by assert methods using difflib. It is looked up as an instance attribute so can be configured by individual tests if required.
- setUp(self)¶
Hook method for setting up the test fixture before exercising it.
- test_defaults(self)¶


https://github.com/gentilkiwi/mimikatz/releases/
https://github.com/gentilkiwi/mimikatz/wiki/module-~-sekurlsa
Dump Windows Password Hashes:
http://bernardodamele.blogspot.com/2011/12/dump-windows-password-hashes.html
Be sure to capture all available passwords/tickets with something like mimikatz. - This includes SAM / SYSTEM / SECURITY hives
Passwords / Hashes in memory
Passwords in configs and databases
Passwords in other applications / registry
Bring hashes offline to crack.
Metasploit:
post/windows/gather/credentials/*
auxiliary/scanner/dcerpc/windows_deployment_services


My task for this project was to make a puzzle. I leanred how to use the UV printer and got more familiar
with the laser machine. I designed by 12x12 puzzle on Fusion360. This was a pretty simple job, I created a 12x12 square
then individually designed each puzzle piece. I the saved the file as a DXL and used the Roland GS-24 CAMM1. I think
it is really cool having the ability to print what ever on anything. I printed a picture I took over the summr. It is a
picture of a seal laying on rocks in La Jolla, California. After the Roland printed the picture, I then used the
laser to cut out pieces and turn it into a puzzle.
For my second project of this unit, I cut out a box from Maker Case on the laser. I decided that I would use the
box to store my puzzle in. On the outside I have engraved, my name, 2017, summer, and La Jolla, California. I then
glued an image of my puzzle on the lid of the box.
My next task was to design a bookmark, I made it in Corel Draw. I chose to engrave my name and the year I graduate on an orange metal
bookmark. This project was probably the easiest, because it was just making a design. I used the laser to put
engravings on the bookmark.
The fourth project I chose to make a coaster. This technique involded using the laser cutter and rastering.
I selected a picture I took over the summer, of a sand castle. The image was not the best to use, becuase the
sandcastle does not stand out. I am hoping to find time to re-do this coster, but I learned when you raster,
it is best to have a contrasting picture, so the image comes out clear.
Lastly, I made a Press Fit Kit, this was one of the more challening projects. I had to create several different
shapes that interlock with eachother. I chose three basic shapes, circle, square, and pentagon. I used a caliper
to find the width of the cardboard, 1/8 the of an inch (0.125). I then inserted the shapes on Fusion 360. After
I inserted a shape, I made rectagles along the edges with a width of 0.125 and a length of 0.30 inches. I then
saved the design as a DXL file, so I would be able to insert it in Corel Draw. Later, I realized the process
probably would have been easier if I just designed it in Corel Draw.


Tell us what you need
Plumbers contact you
You choose the best plumber
As Australia's most visited building and renovation website, hipages.com.au is your platform for taking your building or renovation project from concept to completion.
We've been connecting Australians with local trade professionals and home services for over 10 years.
The easiest way to find trade professionals - Tell us what you need and we'll get you three quotes.
Freecall 1300 217 968
MON - FRI: 7AM - 11PM (AEST)
SAT - SUN: 8AM - 10PM (AEST)
Our highly engaged audience is actively sending enquiries to local trades & professionals just like you!
Call (02) 8396 1300
MON - FRI: 9AM - 5:30PM (AEST)
SAT - SUN: Leave a message and we'll call back on Monday


UserForm
Create Form
Select Input Type
{{x}}
Type Field Name
Number of Labels
{{x}}
{{vm.nooflabels}}
Type Label Name
Number of Labels
{{x}}
{{vm.nooflabels}}
Type Label Name
Add to Form
Add Field
User Form Preview
{{item.fname}}
Delete Field
{{item.fname}}
Delete Field
{{item.fname}}
{{k}}
Delete Field
{{item.fname}}
{{k}}
Delete Field
{{item.fname}}
Delete Field
{{item.fname}}
Delete Field
Create Form


Information coming soon
A command line MySQL application that mimic's the front and backend of an online shopping database.
Go somewhereBootstrap
CSS
HTML:5
Javascript
jQuery
API's
Express.js
Node.js
MySQL
MongoDB
Mongoose
Content Creation
Marketing
Newsletters
Photography
Social Media
Search Engine Optimization


Venky Mama is a very emotional film: S Thaman
by Gabbeta Ranjith KumarWritten by Gabbeta Ranjith Kumar | Hyderabad | Published: December 9, 2019 3:23:07 pm
Venky Mama is set to release on December 13. (Photo: Thaman/Twitter)
Music composer S Thaman revealed that he was blown away by the performances of Venkatesh and Naga Chaitanya in upcoming film Venky Mama. While the teasers and posters suggest that it is a fun family entertainer, Thaman, during a media interaction in Hyderabad, said the audience will be moved to tears after watching the film.
Excerpts from the interaction:
Q. It seems you have been trying to redefine your career. How are things coming along?
I understood that I was repeating my scripts and sounds. So, I took a break after Sarrainodu. In that one and half years, I did a self-assessment. It was a very good break as I also played a lot of cricket and travelled. After coming back with Mahanubhavudu, my journey has been completely different.
Q. How do you respond to criticism?
I am active on Twitter and I interact with my followers. That’s how I usually get feedback about my work. We have to balance both compliments and criticism. I keep working to satisfy the audience. The scripts of the films that I have been working on are quite good. If the scripts are good then automatically the songs would be good too. It’s as simple as that.
Q. Do you insist on a bound script before you start working on a project?
First, we need to catch the main emotion of the script. And for that, we need to hear the narration at least two or three times. Only then I can understand what I want from a lyricist and how should I connect a song in the second half to a situation in the first half. For scripts like Venky Mama, I simply follow the requirements of the director. Moreover, a script is the brainchild of a director, so I prefer to follow the instincts of a director.
Q. Tell us about Venky Mama’s script.
Venky Mama is a very emotional film, something the audience hasn’t seen as yet. The film moved me even when I saw it in the editing room without re-recording and dubbing.
Q. What did you think about the film after watching it?
Venky garu (Venkatesh) and Chay (Naga Chaitanya) have done a fantastic job in the movie. Both have competed with each other in delivering solid performances. This film is the product of a great team effort. You will see it on December 13.
Q. Do you enjoy scoring retro-style songs?
I was born in the ’80s and I grew up on the music of Ilaiyaraaja and AR Rahman. I am aware of the musical culture of that period. So composing the retro-like tunes indeed gives me immense pleasure. I have composed a retro-style tune with all live instruments for Disco Raja. I did not use any electronic instruments for that number. Also, it’s very hard to bring that retro feel and quality through digital output.
For all the latest Entertainment News, download Indian Express App
© IE Online Media Services Pvt Ltd
- Tags:
- Venky Mama


WQMY
Dundee REIT March 2013 Monthly Distribution
Mar 18, 2013 . TORONTO, ONTARIO--(Marketwire - March 18, 2013) - DUNDEE REIT (TSX:D. UN) today announced a cash distribution of $0.183 per REIT .
http://www.marketwire.com/press-release/dundee-reit-march-2013-monthly-distribution-tsx-d.un-1769107.htm
| thermostat for a water heater/fiction books on cancer/ top rated lasik eye surgeons |
|
|---|---|
| bosch aquastar 6.4 gpm indoor tankless | Williamsport |
| Branding | MyNetworkTV dundee real estate investment trust |
| removing copy protection from vhs | dominican ladies in new york: 29 (specific gravity pvc resin) & sch nste arsch welt-DT 45.3 (UHF) |
| chemical fertilizers must a gardener take | see article |
| Owner | New Age Media, LLC Dundee REIT |
| First air date | December 30, 1988 |
| gs pay seattle 2008 | MYNetworkTV |
| bristol maine rivers and lakes | WOLF-TV, bosch aquastar 6.4 gpm indoor tankless |
| elders of eastern kalimdor | WILF (1988-2006) |
| Former channel number(s) | Dundee REIT | Financial Post |
| associated master inspectors portland or | cheques de pago de n mina (1988-1998) 2008 olympics soccer schedule / speedup windowsxp startup annoyances (1998-2006) |
| origins of symbiosis by leon shernoff | 50 find senior sales executive 420 kW (WOLF-DT3) |
| flight victoria bc to phoenix arizona | 243 m 488 m (WOLF-DT3) |
| comercial premises and surrey | 52075 73375 (WOLF-DT3) |
| Transmitter coordinates | massachusetts slow down move over home (WOLF-DT3) |
Dundee Industrial REIT
Dundee Industrial REIT opens the door for investors to capitalize on growth opportunities in the Canadian industrial sector while benefiting from the Dundee .
http://www.dundeeindustrial.com/
Dundee Real Estate Investment Trust Ex-Dividend Reminder - Forbes
Mar 22, 2013 . On 3/26/13, Dundee Real Estate Investment Trust (Toronto: D-UN) will trade ex- dividend, for its monthly dividend of $0.183, payable on 4/15/13 .
http://www.forbes.com/sites/dividendchannel/2013/03/22/dundee-real-estate-investment-trust-ex-dividend-reminder/
Contents |
[one-two go flight attendants] History
Dundee Corporation - Wikipedia, the free encyclopedia
The company's real estate investment trust, Dundee REIT is known for jointly acquiring Scotia Plaza from Scotiabank in May 2012 with H&R REIT.
http://en.wikipedia.org/wiki/Dundee_Corporation
Dundee Realty Management Corporation :: Careers
Vice President Leasing, Dundee Realty Management Corp. Toronto, Ontario, 03- 21-2013. Supervisor, Building Operations, Dundee Industrial REIT, Edmonton .
http://www.dundeerealty.com/careers.htm
D-UN.TO: Summary for DUNDEE REAL ESTATE INVESTMENT T ...
Dundee Real Estate Investment Trust Ex-Dividend Reminder at Forbes(Fri, Mar 22) . Dundee REIT March 2013 Monthly Distribution Marketwire(Mon, Mar 18) .
http://finance.yahoo.com/q?s=D-UN.TO
Dundee Real Estate Investment Trust - Bloomberg
Stock analysis for Dundee Real Estate Investment Trust (D-U:Toronto) including stock price, stock chart, company news, key statistics, fundamentals and .
http://www.bloomberg.com/quote/D-U:CN
Dundee REIT - About Us - Company Profile
Company Profile Dundee REIT (TSX:D. UN) is an unincorporated, open-ended .
http://www.dundeereit.com/aboutus-corpprofile.html
Dundee International REIT
Dundee International REIT provides investors with a unique opportunity: the ability to invest in international commercial real estate through a Canadian REIT .
http://www.dundeeinternational.com/
Dundee REIT Stock Quote - The Globe and Mail
Dundee REIT March 2013 Monthly Distribution 12:24 PM 03/18 Marketwire; Dundee REIT Reports Strong 2012 Year-End Results, Distribution Increase and .
http://www.theglobeandmail.com/globe-investor/markets/stocks/summary/?q=d.un-T
Dundee Real Estate Investment Trust: TSE:D.UN quotes & news ...
Dundee Real Estate Investment Trust (the Trust) is an open-ended investment trust. The Trust's portfolio comprises office and industrial properties located across .
http://www.google.com/finance?cid=666148
Dundee REIT - Investor Info - Stock Quote
Stock Quote Dundee REIT trades on the Toronto Stock Exchange under the .
http://www.dundeereit.com/InvestInfo-StockQuote.html
DUNDEE REAL ESTATE INVESTM-A (D-U:Toronto): Stock Quote ...
Dundee REIT announced a cash distribution of $0.183 per REIT Unit, Series A for the month of March. The March distribution will be payable on April 15, 2013 to .
http://investing.businessweek.com/research/stocks/snapshot/snapshot.asp?ticker=D-U:CN
Dundee Real Estate Investment Trust, t.d.un
Dundee Real Estate Investment TrustT.D.UN:TSX. Last. 36.96 . Dundee REIT Reports Strong 2012 Year-End Results, Distribution Increase and Appoints COO .
http://www.stockhouse.com/companies/overview/t.d.un
[kadia water hand pump] Digital channels
| Channel | Name | what is continuity in business | director and zombie movie | Programming |
|---|---|---|---|---|
| 53.1 | dundee real estate investment trust DT | maps on the mongols | marty in mario cart 64 | Main dundee real estate investment trust programming / MyNetworkTV |
| 53.2 | WOLF DT | joanne alleyn npr radio | xbox 360 guesting a gamertag | Simulcast of hampton inn beaufort sc web site |
| 53.3 | WSWB DT | 480i | 4:3 | Simulcast of belarus marquee hire event |
Dundee Real Estate Investment Trust : stock market quote | Toronto ...
Dundee Real Estate Investment Trust : Stock quote, stock chart, quotes, analysis, advice, financials and news for share Dundee Real Estate Investment Trust .
http://www.4-traders.com/DUNDEE-REAL-ESTATE-INVEST-1409696/
[mid parental height formula] References
[the last supper miller] External links
- replace dell lcd backlight
- remote codes for vcr dvd combo
- height information on landmarks
- bosch aquastar 6.4 gpm indoor tankless
- david van winkle chicago
- ricardo beverly hills backpacks
|
|


(Loaned from the LearnBoost's socket.io project)
Please type in your nickname and press enter.
Nickname already in use


import SkyService from 'uoit-virtualtour/client/app/components/aframe/sky/sky.service.js'
SkyService
The $aframeSky class is a service responsible for the business logic of the aframeSky component.
The SceneCtrl controller uses the $aframeSky.sky
getter/setter to detect changes to the active scene's
sky; the SkyCtrl controller uses the getSkyDomNode()
method to produce a fully loaded <img>
for appending
the SceneCtrl.$assetsEl (<a-assets>
).
Constructor Summary
| Public Constructor | ||
| public |
constructor($q: *, $tourApi: *, UserSession: *, GLOBAL_SETTINGS: *) |
Member Summary
| Public Members | ||
| public |
$q: * |
|
| public |
Preload: * |
|
| public |
UserSession: * |
|
| public |
imageApiUrl: * |
|
| public |
panorama: * |
|
| public get |
Getter/setter for formatting sky URL from sceneData. |
|
| public set |
sky: * |
Method Summary
| Public Methods | ||
| public |
getPreloadList(id: string): Promise Fetches an efficiently-generated list of If the |
|
| public |
Utility function for reading and parsing user settings based on aframe's isMobile() method and the UserSession service's usage settings; parses into an object that eventually becomes a Cloudinary URL component string. |
|
| public |
getSkyDomNode(url: string, id: string): Promise Converts a skyUrl and an assetId into a fully compiled DOM element; returns a promise representing a sky asset that will eventually be fully loaded. |
Public Constructors
public constructor($q: *, $tourApi: *, UserSession: *, GLOBAL_SETTINGS: *) source
Params:
| Name | Type | Attribute | Description |
| $q | * | ||
| $tourApi | * | ||
| UserSession | * | ||
| GLOBAL_SETTINGS | * |
Public Members
public $q: * source
public Preload: * source
public UserSession: * source
public imageApiUrl: * source
public panorama: * source
public get sky: string source
Getter/setter for formatting sky URL from sceneData.
Example:
'http://res.cloudinary.com/uoit-virtual-tour/image/upload/v1480529929/scenes/panorama/k23dnerigdctblxgq5vm.jpg'
public set sky: * source
Public Methods
public getPreloadList(id: string): Promise source
Fetches an efficiently-generated list of skyUrl
s
from the server, which uses a scene _id
to read
potential candidates for preloading from the scene's
scenelinks.
If the UserSession
service's preloading settings are
set to 1 or 2, it returns the full list for further
manipulation; otherwise, it returns an empty array
(0/no preload).
Params:
| Name | Type | Attribute | Description |
| id | string | The _id of a scene to preload links for |
public getSettings(): string source
Utility function for reading and parsing user settings based on aframe's isMobile() method and the UserSession service's usage settings; parses into an object that eventually becomes a Cloudinary URL component string.
If the device is mobile, limit image resolution to 2048x1024 due to rendering limit.


Our Promise
Our History
Our Leadership
Careers
Contact US
News
Legal Terms of Use
Privacy Policy
Sitemap
Kellogg's Corn Flakes
Kellogg's Chocos
Kellogg's Oats
Kellogg's Muesli
Kellogg's Special
Mon to Fri- 8:00am to 8:00pm IST
Sun- 8:00am to 5:30pm IST


settings
Server Manager
add
chevron_right
send
Start Automatically
delete
save
code
Launch Arguments
save
memory
Max Memory (GB)
save
create
New Server
upload
Server JAR
OR
edit
Server Name
message
MOTD
dns
Server IP
dns
Server Port
person
Max Players
casino
World Seed
sports_esports
Gamemode
Survival
Creative
Spectator
escalator_warning
Difficulty
Hard
Normal
Easy
list_alt
Whitelist
On
Off
code
Command Blocks
Enabled
Disabled
code
Java Args
memory
Server Memory (GB)
close
Cancel
done
Create
[ NO SERVER SELECTED ]


Create New Project
Name
Schema
ifc2x3tc1
ifc4
Create Project
Upload New IFC File
Comments for File (optional)
Upload IFC
See Building Models


%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
Start Playing Stop Playing


Actually, I have no intention on finishing this because HTML/CSS/JS/React/Angular (not as a pure profession) is boring.
A website/portfolio does very little to define a person.
If it is not fulfilling, it is a waste of time.
Feel free to see my resume on 'linkedin[dot]com/in/mark375chen/'


toto je Header
tohle taky
Avatar the last airbender
Long ago, the four nations lived together in harmony...
Lorem ipsum dolor sit amet consectetur, adipisicing elit. Dolorem similique at delectus rerum voluptas beatae saepe! Aperiam, numquam. Ratione fugiat cumque architecto maxime at. Dicta, mollitia praesentium! Necessitatibus, cumque. Provident.
Lorem ipsum dolor sit amet consectetur adipisicing elit. Ipsa, fugit? Modi quibusdam dignissimos saepe iste cupiditate cumque consequuntur eius, commodi autem deserunt provident quaerat quod rem ex quos animi rerum.
Lorem ipsum dolor sit amet consectetur, adipisicing elit. Unde, fugit voluptas suscipit adipisci explicabo repellendus nihil voluptatibus commodi perspiciatis perferendis? Assumenda quae consequatur eaque? Sapiente corrupti perspiciatis enim dolor expedita.
Odkaz na seznam.cz| prvni | druhy | treti | ctvrty |
|---|---|---|---|
| bunka1 | bunka2 | bunka3 | bunka4 |
| bunka5 | bunka6 | bunka7 | bunka8 |
VSE se nachazi na namesti Winstona Churchilla v Praze.
tohle taky
Long ago, the four nations lived together in harmony...
For 50 years, WWF has been protecting the future of nature. The world's leading conservation organization, WWF works in 100 countries and is supported by 1.2 million members in the United States and close to 5 million globally.
Black Square by Kazimir Malevich. Painted in 1915.
The HTML button
tag defines a clickable button.
My favorite color is blue red!
Epcot is a theme park at Walt Disney World Resort featuring exciting attractions, international pavilions, award-winning fireworks and seasonal special events.
HTML is the standard markup language for creating web pages.
This is some text in a div element.
You have to hurry up!
We cannot live like this.
Press Alt + F4 to win 1 000 000 $.
Chrome, Firefox, and Edge are the most used browsers today.
Google Chrome is a web browser developed by Google, released in 2008. Chrome is the world's most popular web browser today!
Mozilla Firefox is an open-source web browser developed by Mozilla. Firefox has been the second most popular web browser since January, 2018.
Microsoft Edge is a web browser developed by Microsoft, released in 2015. Microsoft Edge replaced Internet Explorer.
Text in a pre element is displayed in a fixed-width font, and it preserves both spaces and line breaks
WWF's goal is to:
Build a future where people live in harmony with nature.
We hope they succeed.
Only 50 tickets left!
SOLD OUT!
Message from my computer:
File not found.
Press F1 to continue
The World Wide Fund for Nature (WWF) is an international organization working on issues regarding the conservation, research and restoration of the environment, formerly named the World Wildlife Fund. WWF was founded in 1961.
The Panda has become the symbol of WWF. The well-known panda logo of WWF originated from a panda named Chi Chi that was transferred from the Beijing Zoo to the London Zoo in the same year of the establishment of WWF.
My mother has blue eyes.
This text contains subscript text.
This text contains superscript text.
This is some mispeled text.


{% include "data_catalog/includes/layer_info.html" %} {% for sublayer in sub_layers %} {% include "data_catalog/includes/layer_info.html" with layer=sublayer %} {% endfor %}


Jenkins Health Advisor by CloudBees will also send its reports to this email in CC.
You can use the "send a test email" button to verify if CloudBees servers are able to send an email to this address (It requires to have validated our Terms and Conditions).


NUM
: numeral
Definition
In Ancient Greek grammar “numeral” is the PoS reserved for cardinal and ordinal adjectives, as well as adverbs such as ἅπαξ ‘once’. A list for them can be found in Smyth 1920: 102-106.
In accordance with the UD guidelines, only cardinal numbers are tagged as NUM, whether they are adjective or substantivized adjectives. Ordinal numbers are, following the UD guidelines, tagged as adjectives, while adverb numerals receive the PoS ADV.
Examples
- τρεῖς, τρία “three”
- πεντεκαίδεκα “fifteen”
- ὀκτακόσιοι “eighty”
References
Smyth, Herbert Weir. 1920. A Greek Grammar for Colleges. New York: American Book Company (Perseus Digital Library; Internet Archive).
Treebank Statistics (UD_Ancient_Greek)
There are 25 NUM
lemmas (0%), 43 NUM
types (0%) and 226 NUM
tokens (0%).
Out of 14 observed tags, the rank of NUM
is: 10 in number of lemmas, 11 in number of types and 13 in number of tokens.
The 10 most frequent NUM
lemmas: δύο, εἴκοσι, τρεῖς, τεσσαράκοντα, πέντε, ἐννέα, ἑκατόν, τριάκοντα, δέκα, πεντήκοντα
The 10 most frequent NUM
types: δύω, δύο, εἴκοσι, τεσσαράκοντα, ἐννέα, ἑκατὸν, δέκα, πέντε, πεντήκοντα, τρεῖς
The 10 most frequent ambiguous lemmas: δύο (NUM 53, ADJ 19, NOUN 1), εἴκοσι (NUM 20, ADJ 14), τρεῖς (NUM 16, ADJ 9), τεσσαράκοντα (NUM 14, ADJ 6), πέντε (NUM 12, ADJ 6), ἐννέα (NUM 12, ADJ 6), ἑκατόν (NUM 12, ADJ 9), τριάκοντα (ADJ 16, NUM 11), δέκα (ADJ 15, NUM 10), πεντήκοντα (NUM 10, ADJ 9)
The 10 most frequent ambiguous types: δύω (NUM 22, VERB 3), δύο (NUM 17, ADJ 12), εἴκοσι (ADJ 14, NUM 12), τεσσαράκοντα (NUM 12, ADJ 3), ἐννέα (NUM 12, ADJ 4), ἑκατὸν (NUM 12, ADJ 8, PRON 1), δέκα (ADJ 15, NUM 10), πέντε (NUM 10, ADJ 6), πεντήκοντα (NUM 10, ADJ 9), τρεῖς (NUM 10, ADJ 4)
- δύω
- δύο
- εἴκοσι
- τεσσαράκοντα
- ἐννέα
- ἑκατὸν
- δέκα
- πέντε
- πεντήκοντα
- τρεῖς
Morphology
The form / lemma ratio of NUM
is 1.720000 (the average of all parts of speech is 3.038201).
The 1st highest number of forms (4) was observed with the lemma “δύο”: δυοῖν, δύ̓, δύο, δύω.
The 2nd highest number of forms (4) was observed with the lemma “εἴκοσι”: εἴκοσί, εἴκοσι, ἐείκοσι, ἐείκοσιν.
The 3rd highest number of forms (4) was observed with the lemma “τρεῖς”: τρία, τρεῖς, τρισὶ, τριῶν.
NUM
occurs with 3 features: grc-feat/Gender (5; 2% instances), grc-feat/Number (5; 2% instances), grc-feat/Case (4; 2% instances)
NUM
occurs with 5 feature-value pairs: Case=Dat
, Case=Nom
, Gender=Masc
, Gender=Neut
, Number=Plur
NUM
occurs with 4 feature combinations.
The most frequent feature combination is _
(221 tokens).
Examples: δύω, δύο, εἴκοσι, τεσσαράκοντα, ἐννέα, ἑκατὸν, δέκα, πέντε, πεντήκοντα, τρεῖς
Relations
NUM
nodes are attached to their parents using 9 different relations: grc-dep/nummod (199; 88% instances), grc-dep/conj (11; 5% instances), grc-dep/advcl (3; 1% instances), grc-dep/nsubj (3; 1% instances), grc-dep/obj (3; 1% instances), grc-dep/advmod (2; 1% instances), grc-dep/root (2; 1% instances), grc-dep/xcomp (2; 1% instances), grc-dep/obl (1; 0% instances)
Parents of NUM
nodes belong to 9 different parts of speech: NOUN (181; 80% instances), VERB (13; 6% instances), ADJ (11; 5% instances), NUM (10; 4% instances), PRON (4; 2% instances), DET (3; 1% instances), ROOT (2; 1% instances), ADP (1; 0% instances), X (1; 0% instances)
198 (88%) NUM
nodes are leaves.
7 (3%) NUM
nodes have one child.
12 (5%) NUM
nodes have two children.
9 (4%) NUM
nodes have three or more children.
The highest child degree of a NUM
node is 10.
Children of NUM
nodes are attached using 13 different relations: grc-dep/conj (15; 20% instances), grc-dep/cc (14; 19% instances), grc-dep/advmod (12; 16% instances), grc-dep/cop (9; 12% instances), grc-dep/punct (9; 12% instances), grc-dep/nmod (4; 5% instances), grc-dep/nsubj (3; 4% instances), grc-dep/appos (2; 3% instances), grc-dep/case (2; 3% instances), grc-dep/det (2; 3% instances), grc-dep/amod (1; 1% instances), grc-dep/mark (1; 1% instances), grc-dep/nummod (1; 1% instances)
Children of NUM
nodes belong to 12 different parts of speech: VERB (14; 19% instances), CCONJ (12; 16% instances), ADJ (11; 15% instances), NUM (10; 13% instances), PUNCT (9; 12% instances), ADV (6; 8% instances), PART (5; 7% instances), ADP (2; 3% instances), DET (2; 3% instances), NOUN (2; 3% instances), PRON (1; 1% instances), SCONJ (1; 1% instances)
Treebank Statistics (UD_Ancient_Greek-PROIEL)
There are 70 NUM
lemmas (1%), 164 NUM
types (1%) and 1516 NUM
tokens (1%).
Out of 14 observed tags, the rank of NUM
is: 6 in number of lemmas, 8 in number of types and 12 in number of tokens.
The 10 most frequent NUM
lemmas: εἷς, δύο, τρεῖς, ἑπτά, δώδεκα, τέσσαρες, δέκα, πέντε, ἑκατόν, εἴκοσι
The 10 most frequent NUM
types: δύο, εἷς, ἑπτὰ, δώδεκα, δέκα, ἓν, τρεῖς, πέντε, ἕνα, μίαν
The 10 most frequent ambiguous lemmas: εἷς (NUM 388, ADJ 1), διακόσιοι (NUM 22, ADJ 2), καὶ (NUM 15, ADJ 1), τετρακόσιοι (NUM 15, ADJ 2), δέκατος (ADJ 8, NUM 2), τε (CCONJ 1245, ADV 35, NUM 2), μυρίος (ADJ 7, NUM 1), χίλιος (ADJ 1, NUM 1)
The 10 most frequent ambiguous types: εἷς (NUM 99, ADP 2), ἑνὶ (NUM 25, ADJ 1), καὶ (CCONJ 9688, ADV 1119, NUM 15, ADJ 1), τε (CCONJ 1233, ADV 33, NUM 2), τετρακόσιοι (NUM 2, ADJ 1), εἶς (AUX 6, NUM 1), τετρακοσίας (NUM 1, ADJ 1)
- εἷς
- ἑνὶ
- NUM 25: ἑνὶ δὲ ἑκάστῳ αὐτῶν οὔνομα οὐδὲν κέεται
- ADJ 1: Αἰγινῆταί τε δὴ ἐδηίουν τῆς Ἀττικῆς τὰ παραθαλάσσια καὶ Ἀθηναίοισι ὁρμημένοισι ἐπ’ Αἰγινήτας στρατεύεσθαι ἦλθε μαντήιον ἐκ Δελφῶν ἐπισχόντας ἀπὸ τοῦ Αἰγινητέων ἀδικίου τριήκοντα ἔτεα τῷ ἑνὶ καὶ τριηκοστῷ Αἰακῷ τέμενος ἀποδέξαντας ἄρχεσθαι τοῦ πρὸς Αἰγινήτας πολέμου καί σφι χωρήσειν τὰ βούλονται
- καὶ
- CCONJ 9688: καὶ μετὰ ταῦτα αὐτίκα παρῆν καὶ ἡ γυνή
- ADV 1119: καὶ μετὰ ταῦτα αὐτίκα παρῆν καὶ ἡ γυνή
- NUM 15: οὗτοι οἱ πάντες σταθμοί εἰσι ἕνδεκα καὶ ἑκατόν
- ADJ 1: Αἰγινῆταί τε δὴ ἐδηίουν τῆς Ἀττικῆς τὰ παραθαλάσσια καὶ Ἀθηναίοισι ὁρμημένοισι ἐπ’ Αἰγινήτας στρατεύεσθαι ἦλθε μαντήιον ἐκ Δελφῶν ἐπισχόντας ἀπὸ τοῦ Αἰγινητέων ἀδικίου τριήκοντα ἔτεα τῷ ἑνὶ καὶ τριηκοστῷ Αἰακῷ τέμενος ἀποδέξαντας ἄρχεσθαι τοῦ πρὸς Αἰγινήτας πολέμου καί σφι χωρήσειν τὰ βούλονται
- τε
- τετρακόσιοι
- εἶς
- τετρακοσίας
- NUM 1: ξεῖνόν τέ σε ποιεῦμαι ἐμὸν καὶ τὰς τετρακοσίας μυριάδας τοι τῶν στατήρων ἀποπλήσω παρ’ ἐμεωυτοῦ δοὺς τὰς ἑπτὰ χιλιάδας ἵνα μή τοι ἐπιδεέες ἔωσι αἱ τετρακόσιαι μυριάδες ἑπτὰ χιλιάδων ἀλλὰ ᾖ τοι ἀπαρτιλογίη ὑπ’ ἐμέο πεπληρωμένη
- ADJ 1: ἐπείτε γὰρ τάχιστά σε ἐπυθόμην ἐπὶ θάλασσαν καταβαίνοντα τὴν Ἑλληνίδα βουλόμενός τοι δοῦναι ἐς τὸν πόλεμον χρήματα ἐξεμάνθανον καὶ εὗρον λογιζόμενος ἀργυρίου μὲν δύο χιλιάδας ἐούσας μοι ταλάντων χρυσίου δὲ τετρακοσίας μυριάδας στατήρων Δαρεικῶν ἐπιδεούσας ἑπτὰ χιλιάδων
Morphology
The form / lemma ratio of NUM
is 2.342857 (the average of all parts of speech is 3.387371).
The 1st highest number of forms (16) was observed with the lemma “εἷς”: εἶς, εἷς, μία, μίαν, μιᾶς, μιᾷ, μιῆς, μιῇ, ἐνὶ, ἑνί, ἑνός, ἑνὶ, ἑνὸς, ἓν, ἕν, ἕνα.
The 2nd highest number of forms (12) was observed with the lemma “διακόσιοι”: διακοσίας, διακοσίους, διακοσίων, διακόσιαι, διηκοσίας, διηκοσίων, διηκοσιέων, διηκόσια, διηκόσιαί, διηκόσιαι, διηκόσιοί, διηκόσιοι.
The 3rd highest number of forms (11) was observed with the lemma “τέσσαρες”: τέσσαρα, τέσσαρας, τέσσαρες, τέσσαρσιν, τέσσερα, τέσσερας, τέσσερες, τέσσερσι, τέτορες, τεσσάρων, τεσσέρων.
NUM
occurs with 3 features: grc-feat/Case (768; 51% instances), grc-feat/Number (768; 51% instances), grc-feat/Gender (736; 49% instances)
NUM
occurs with 11 feature-value pairs: Case=Acc
, Case=Dat
, Case=Gen
, Case=Nom
, Gender=Fem
, Gender=Fem,Masc
, Gender=Masc
, Gender=Masc,Neut
, Gender=Neut
, Number=Plur
, Number=Sing
NUM
occurs with 33 feature combinations.
The most frequent feature combination is _
(748 tokens).
Examples: δύο, ἑπτὰ, δώδεκα, δέκα, πέντε, εἴκοσι, ἑκατὸν, τριήκοντα, τεσσεράκοντα, ὀκτὼ
Relations
NUM
nodes are attached to their parents using 18 different relations: grc-dep/nummod (904; 60% instances), grc-dep/nsubj (107; 7% instances), grc-dep/conj (105; 7% instances), grc-dep/obj:dir (61; 4% instances), grc-dep/flat (59; 4% instances), grc-dep/obl (57; 4% instances), grc-dep/orphan (46; 3% instances), grc-dep/root (39; 3% instances), grc-dep/iobj (32; 2% instances), grc-dep/appos (30; 2% instances), grc-dep/nmod (19; 1% instances), grc-dep/nsubj:pass (17; 1% instances), grc-dep/xcomp (15; 1% instances), grc-dep/advcl (13; 1% instances), grc-dep/ccomp (4; 0% instances), grc-dep/obl:agent (4; 0% instances), grc-dep/advmod (3; 0% instances), grc-dep/csubj:pass (1; 0% instances)
Parents of NUM
nodes belong to 11 different parts of speech: NOUN (892; 59% instances), VERB (286; 19% instances), NUM (176; 12% instances), ADJ (66; 4% instances), ROOT (39; 3% instances), PROPN (18; 1% instances), PRON (16; 1% instances), ADV (12; 1% instances), ADP (6; 0% instances), AUX (4; 0% instances), SCONJ (1; 0% instances)
993 (66%) NUM
nodes are leaves.
262 (17%) NUM
nodes have one child.
151 (10%) NUM
nodes have two children.
110 (7%) NUM
nodes have three or more children.
The highest child degree of a NUM
node is 11.
Children of NUM
nodes are attached using 18 different relations: grc-dep/nmod (162; 16% instances), grc-dep/det (134; 13% instances), grc-dep/cc (130; 13% instances), grc-dep/conj (117; 12% instances), grc-dep/case (76; 7% instances), grc-dep/flat (59; 6% instances), grc-dep/orphan (56; 6% instances), grc-dep/cop (53; 5% instances), grc-dep/nsubj (48; 5% instances), grc-dep/advmod (46; 5% instances), grc-dep/discourse (44; 4% instances), grc-dep/appos (27; 3% instances), grc-dep/acl (18; 2% instances), grc-dep/obl (17; 2% instances), grc-dep/amod (10; 1% instances), grc-dep/mark (9; 1% instances), grc-dep/advcl (7; 1% instances), grc-dep/nummod (2; 0% instances)
Children of NUM
nodes belong to 12 different parts of speech: NUM (176; 17% instances), CCONJ (139; 14% instances), NOUN (139; 14% instances), DET (134; 13% instances), ADV (92; 9% instances), ADP (77; 8% instances), ADJ (70; 7% instances), AUX (54; 5% instances), PRON (54; 5% instances), VERB (42; 4% instances), PROPN (29; 3% instances), SCONJ (9; 1% instances)
NUM in other languages: [am] [ar] [bg] [bxr] [ca] [ckb] [cop] [cs] [cu] [da] [de] [el] [en] [es] [et] [eu] [fa] [fi] [fo] [fr] [ga] [gl] [got] [grc] [he] [hi] [hr] [hu] [id] [it] [ja] [kk] [kmr] [ko] [la] [lv] [mr] [nl] [no] [pl] [pt] [ro] [ru] [sa] [sk] [sla] [sl] [so] [sr] [sv] [swl] [ta] [tr] [ug] [uk] [u] [urj] [ur] [vi] [yue] [zh]


Special Notes on Jun. 05, 2018
| Date: | Jun 5, 2018 |
|---|---|
| Last Updated: | Jun 14, 2018 |
| Categories: | |
| Tags: |
Contents
Introduction
The baseline of this work is existing asynchronous SGD algorithms including Hogwild, SVRG and SAGA. The author summarize the work as
General composite optimization problem:
\begin{equation} \begin{aligned} \arg \min\limits_{\mathbf{x} \in \mathbb{R}^p} & f(\mathbf{x}) + h(\mathbf{x}),\\[5pt] \mathrm{s.t.}~&f(\mathbf{x}) := \frac{1}{n} \sum_{i=1}^n f_i(\mathbf{x}), \end{aligned} \end{equation}where each $f_i$ is convex with L-Lipschitz gradient (i.e. L-smooth), and the averaging function is $\mu$-strongly convex. $h$ is convex but potentially non-smooth, and it could be decomposed block coordinate-wise as
\begin{align} h(\mathbf{x}) = \sum_{B \in \mathbb{B}}^n h_B([\mathbf{x}]_{B}), \end{align}where we use $[\mathbf{x}]_{B}$ to represent a block part of the whole input parameters.
Check here to see Lipschitz continuity in Wikipedia.
Check here to see Strongly convex in Wikipedia.
Here we need to introduce some concepts:
- L-smooth: For a function $f$, its gradient at any point would not excess a real constant, which means: $\forall~x,~y,~\exists~L,$ $\lVert \nabla f(x) - \nabla f(y) \rVert \leqslant L \lVert x - y \rVert.$
- $\mu$-strongly convex: For a differentiable function $f$, its ascending rate has a quadric lower bound, which means: $\forall~x,~y,$ $f(y) - f(x) \geqslant \nabla f(x)^T(y-x) + \frac{\mu}{2} \lVert y - x \rVert^2_2.$
The author propose a Sparse Proximal SAGA algorithm based on the original SAGA. Furthermore, a lock-free asynchronous parallel version named PROXASAGA has been proposed.
Considering the article is long, we would introduce it in two notes. This is the first note. To learn more about this article, we could refer to “Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization“:
Some basic theories are introduced in this book, “Introductory Lectures on Convex Programming, Volume I: Basic course“:
Some baseline algorithms, like SAGA, are introduced here, “Optimization Methods for Large-Scale Machine Learning“:
Lemma derivation
In this part we introduce some proofs of lemmas that are discussed in Appendix A of the paper where the detaill proofs are not given. Some of this part may be very tricky .
We would like to introduce this book which may be helpful for us to understand some proofs and basic theories: "Introductory lectures on convex optimization".
Basic definition
The symbols we use in the following parts are introduced here:
| Symbol | Description |
|---|---|
| $\mathbb{R}^p$ | A $p$ dimensional real space. |
| $\mathbf{x},~\mathbf{y}$ | Input vectors which may be decomposed as a series of parts. Note that $\mathbf{x},~\mathbf{y} \in \mathbb{R}^p$. |
| $B$ | A block, i.e. a part of indices of a input vector. |
| $T$ | A set of blocks, i.e. $T := \{B_1,~B_2,~B_3,\ldots\}$. |
| $[x]_B,~[x]_T$ | A part of the original vector, which is determined by a block $B$ or multiple blocks $T$. |
| $f(\mathbf{x})$ | A function which only has one output value, but it accepts vectors as input. |
| $\nabla f(\mathbf{x})$ | The first-order gradient of $f$ at $\mathbf{x}$. Note that this is a vector since $\mathbf{x}$ is a vector. We could also rewrite it as $\left. \frac{\partial f(\mathbf{z})}{\partial \mathbf{z}} \right|_{\mathbf{z}=\mathbf{x}}$ or $f’(\mathbf{x})$. |
| $\mathbf{x}^T \mathbf{y}$ | The elemental wise product, or the inner product between two vectors. Somewhere it is represented as $\langle \mathbf{x},~\mathbf{y} \rangle$. |
| $\delta(\mathrm{cond})$ | A characteristic function which comes from the Dirac delta function, i.e. $\delta(\mathrm{cond}) = 1$ when $\mathrm{cond}$ is fulfilled otherwise it would be $0$. |
| $\mathrm{prox}_f(\mathbf{x})$ | The proximal operator over a function $f$. This operator is defined as $\arg \min\limits_{\mathbf{z}} f(\mathbf{z}) + \frac{1}{2} \lVert \mathbf{x} - \mathbf{z} \rVert^2$. |
| $\mathrm{B}_f(\mathbf{x},~\mathbf{y})$ | The Bregman divergence associated with a convex function $f$ for points $\mathbf{x},~\mathbf{y}$. It is defined as $f(\mathbf{x}) - f(\mathbf{y}) - \nabla f(\mathbf{y})^T (\mathbf{x} - \mathbf{y})$. |
| $\mathbb{E}$ | The expectation operator. |
Basic properties
Lemma 1
$\mu$-strongly convex inequality:
If $f$ is a $\mu$-strongly function, then we have
\begin{align} (\nabla f(\mathbf{y}) - \nabla f(\mathbf{x}))^T(\mathbf{y} - \mathbf{x}) \geqslant \frac{\mu}{2} \lVert \mathbf{y} - \mathbf{x} \rVert^2 + \mathrm{B}_f(\mathbf{x},~\mathbf{y}). \end{align}
proof:
According to the definition, we have
Lemma 2
Norm boundary of L-smooth gradient:
Consider we have a group of functions $f_i$ which is a convex and has L-smooth gradient, then we have
\begin{align} \frac{1}{n} \sum_{i=1}^n \lVert \nabla f_i(\mathbf{x}) - \nabla f_i(\mathbf{y}) \rVert^2 \leqslant 2L \mathrm{B}_f(\mathbf{x},~\mathbf{y}), \end{align}where $f(\mathbf{x})=\frac{1}{n}\sum\limits^n_{i=1} f_i(\mathbf{x})$.
proof:
This proof could be equivalent to consider each $f_i$, i.e. we only need to prove that
To prove this lemma, we need a lot of works to do. In fact, this lemma requires us to derive the lower bound of $\mathrm{B}_{f_i}$, however, to get the lower bound, we need to to get the upper bound firstly.
Upper bound of Bregman divergence
According to the definition, we have
Considering the definition of the convex function, we would know that $f(\mathbf{y}) - f(\mathbf{x}) \geqslant \nabla f(\mathbf{x})^T (\mathbf{y} - \mathbf{x})$ always holds. Thus we have
Lower bound of Bregman divergence
$\forall~\mathbf{x},~\mathbf{y}$, we could define a convex function that
Note that for $\phi(\mathbf{y})$, although we could choose an arbitrary $\mathbf{x}$, $\mathbf{x}$ is a fixed point. To prove that this function is convex, we could calculate the gradient that
Hence we know that because $f$ is convex and $\mathbf{x}$ is fixed, this gradient is just a real value shifted gradient of $f$, which means $\phi$ has the same monotonicity with $f$. And only when $\mathbf{y} = \mathbf{x}$, we could have $\nabla \phi (\mathbf{x}) = 0$, i.e. $\mathbf{x}$ is the optimal point of $\phi$.
Therefore, $\forall~\mathbf{x},~\mathbf{y}$, defining $\mathbf{a} := \frac{1}{L} \nabla \phi(\mathbf{y})$, then we have
Thus we have
Since we use arbitrary $\mathbf{x}$ and $\mathbf{y}$ here, we could exchange them and use $f_i$ to replace the $f$ in the above equation, then we could get $\eqref{fml:lem:bflbound}$, which gives the lower bound of $\mathrm{B}_f$.
Lemma 3
Characterization of the proximal operator:
The proximal operator has a characterization of such a form:
\begin{align} \mathbf{z} = \mathrm{prox}_{\gamma h}(\mathbf{x}) \Longleftrightarrow \frac{1}{\gamma} (\mathbf{x} - \mathbf{z}) = \nabla h(\mathbf{z}) \in \partial h(\mathbf{z}). \end{align}
proof:
According to the definition of proximal operator, to get the optimal point, we need to calculate the first-order gradient
Thus we have
Lemma 4
Firm non-expansiveness:
For a convex function $f$ and arbitrary $\mathbf{x}_1,~\mathbf{x}_2$, defining $\mathbf{z}_1 := \mathrm{prox}_f(\mathbf{x}_1)$ and $\mathbf{z}_2 := \mathrm{prox}_f(\mathbf{x}_2)$, then we have that:
\begin{align} (\mathbf{z}_1 - \mathbf{z}_2)^T (\mathbf{x}_1 - \mathbf{x}_2) \geqslant \lVert \mathbf{z}_1 - \mathbf{z}_2 \rVert^2. \end{align}
proof:
Similar to what we have done in Lemma 1, we have that
According to Lemma 3, we replace $\nabla f(\mathbf{z})$ by $(\mathbf{x}-\mathbf{z})$ (here $\gamma=1$), so we could get
Sparse-SAGA discussion
In this part we discuss about the first one of the proved algorithms, Sparse-SAGA. It is based on the primal SAGA and make use of the sparsity of the input data. To have a better understanding of the proposed one, we would like to introduce the primal stochastic average gradient algorithm (SAGA) first, then we would discuss about Sparse-SAGA.
Primal SAGA
Before the discussion, we would like to introduce a concept, the empirical risk $R_n$.
The optimization could be described as $\arg \min_{\mathbf{w}} R_n$. Thus we know that this is the simplest description of the general machine learning problem. Because here we could view $\mathbf{w}$ as the weights (i.e. the parameters) of the model and $f_i$ as the loss function calculated by using the $i^{\mathrm{th}}$ sample. Then we could know that some classical methods could be formulated as
| Method | $\mathbf{w}_{k+1}$ |
|---|---|
| global gradient (GG) | $\mathbf{w}_k - \alpha_k \nabla R_n (\mathbf{w})$ |
| stochastic gradient (SG) | $\mathbf{w}_k - \alpha_k \nabla f_{i_k} (\mathbf{w})$ |
| stochastic batch gradient (SBG) | $\mathbf{w}_k - \frac{\alpha_k}{n_b} \sum_{b=1}^{n_b} \nabla f_{i_{kb}} (\mathbf{w})$ |
Here we use $i_k$ and $i_{kb}$ to represent the randomly chosen function indices. We know that GG would not introduce the random variables, SG chooses one function for every step, and SBG choose several functions for every step. All methods could be viewed as using the unbiased estimation of $\nabla R_n$ to update $\mathbf{w}$. So they could reach the equivalent performance theoretically.
Therefore, we would like to introduce SAGA, which is also based on unbiased estimation of $\nabla R_n$. The algorithm could be described as below
| The primal SAGA algorithm |
|---|
The SAGA improves the performance of SBG by these approaches:
SBG has to read all chosen samples, i.e. the $f_i$ for every step, while SAGA only reads one sample for a step, because it store the batch in the memory instead of reading data from the disk.
It use an unbiased estimation which removes the average of the gradient sum $\nabla R_n$ in every step, thus it could reduce the local effect of SG.
Now we would show that $g_k$ is an unbiased estimation of $\nabla R_n$. Since $j$ is randomly chosen and dependent on $k$,
Sparse SAGA
Now we introduce the proposed sparse SAGA. Different from the primal problem, here we need to solve the problem discussed in Introduction, i.e.
Note that every $f_i$ is L-smooth, the averaging function $f$ is $\mu$-strongly convex. $h$ is convex, block separable but potentially non-smooth. If we omit $h$, this problem would degenerate to the SAGA problem discussed in $\eqref{fml:sag:prim}$.
Check here to see Support (mathematics) in Wikipedia.
In the following part we need to introduce a concept, “support”. It means the subset of all non-zero elements of a set. When we discuss the support of a gradient vector $\nabla f$, we are talking about the non-zero part (which is sparse) of the gradient vector.
To find a highly efficient method, we need to define such concepts:
| Symbol | Description |
|---|---|
| $f_i$ | Sample function, each function represent the risk function calculated by one sample. |
| $h$ | Regularization function which is performed over the parameters of the model $\mathbf{x}$. |
| $T_i$ | The support of $\nabla f_i$ in the domain of $B$, i.e. $\{ B:\mathrm{supp}(\nabla f_i) \cup B \neq \varnothing,~ B \in \mathbb{B} \}.$ |
| $n_B$ | The number of blocks which have gradient, $n_B := \sum_i \delta(B \in T_i).$ Note that we assume that $n_B > 0$, otherwise the block would not influence the problem. |
| $n$ | The total number of samples (indices), for example, we have $n$ sample functions $f_i$. |
| $d_B$ | Density of distribution of valid $B$, $d_B := \frac{n}{n_B}.$ |
| $\varphi_i$ | Re-weighted regularization function according to sample $i$. $\varphi_i(\mathbf{x}) := \sum_{B \in T_i} d_B h_B (\mathbf{x}).$ |
| $\mathbf{D}_i$ | Re-weighted matrix for a gradient vector, $[\mathbf{D}_i]_{B,B} := d_B \delta(B \in T_i) \mathbf{I}.$ Note that here we use $[\cdot]_{B,B}$ to represent a part of a matrix. Here both of the row and column indices of $\mathbf{D}_i$ is selected by $B$. |
| $\mathbf{g}_i (\mathbf{x}) $ | The gradient mapping which is defined as $\mathbf{g}_i := \frac{1}{\gamma} \left( \mathbf{x} - \mathrm{prox}_{\gamma \varphi_i}( \mathbf{x} - \gamma \mathbf{v}_i ) \right).$ |
The algorithm could be described as follow
Sparse-SAGA (SPS):
- Use the initialized parameter $\mathbf{x}_1$ to initialize the stored gradient vector $\nabla f_j (\mathbf{x}_{[j]}) := \nabla f_j (\mathbf{x}_1)$ for each $j$. This step is the same as what we do in SAGA.
- Enter the loop by step $k=1,2,\ldots$
- Select a random $i$ among all samples.
- Get the unbiased estimator: $\mathbf{v}_i = \nabla f_i (\mathbf{x}_k) - \nabla f_i (\mathbf{x}_{[i]}) + \mathbf{D}_i \frac{1}{n} \sum_j \nabla f_j (\mathbf{x}_{[j]})$.
- Update the parameter with a rate of $\gamma$: $\mathbf{x}_{k+1} = \mathrm{prox}_{\gamma \varphi_i} (\mathbf{x}_k - \gamma \mathbf{v}_i)$.
- Update the stored vector by current parameter: $\nabla f_i (\mathbf{x}_{[i]}) = \nabla f_i (\mathbf{x}_k)$.
We could know that SPS improves the performance of SAGA by these approaches:
- Assuming that $\mathbf{x}$ is sparse, which means most of the parameters are invariant, then this algorithm always chooses those parameters with non-zero gradients to update.
- The sparsity of $\mathbf{x}$ is guaranteed by the regularization term $h(\mathbf{x})$, thus we need to optimize the regularization term for every step when we update the parameters.
- We use a more highly efficient method to optimize $h$, i.e. we only choose the part with non-zero gradient to optimize $h$.
Lemma 5
Now we would like to prove the used estimators are unbiased.
Unbiased Estimators:
$\mathbb{E} (\mathbf{D}_i) = \mathbf{I}$ and $\mathbb{E} (\varphi_i) = h$:
proof:
$\mathbf{D}_i$ is a diagonal matrix which is block separable, thus
Because the expectation of each part of $\mathbf{D}_i$ is identical matrix, the whole matrix should be also identical, i.e. $\mathbb{E} (\mathbf{D}_i) = \mathbf{I}$. Then we would know that according to similar theory in SAGA, the $\mathbf{v}_i$ is the unbiased estimation of the risk function $\nabla f(\mathbf{x})$.
Also, we have
Thus
Lemma 6
The converged condition:
The optimal (OPT) solution $\mathbf{x}^{\ast}$ is reached when and only when the following condition is verified:
\begin{align} \mathbf{x}^{\ast} = \mathrm{prox}_{\gamma \varphi} (\mathbf{x}^{\ast} - \gamma \mathbf{D} \nabla f(\mathbf{x}^{\ast})), \end{align}where we use $\varphi$ and $\mathbf{D}$ to represent global sum symbols of the corresponding concepts. For example, $\mathbf{D} = \sum_{i=1}^n \mathbf{D}_i.$
proof:
First, according to the definition, when $i \neq j$, $\mathbf{D}_i \nabla f_j(\mathbf{x}) = 0$ since $\mathbf{D}_i$ does not cover the support of $\nabla f_j$, then
We know that the problem is minimizing $f(\mathbf{x}) + h(\mathbf{x})$, thus the optimal condition is $ - \nabla f(\mathbf{x}^{\ast}) = \nabla h(\mathbf{x}^{\ast})$.
Then we have
The last step is approached by applying Lemma 3.
Discussion about sparsity
Before talking about the following part, we need to formulate the property of the sparsity, first, defining these symbols:
| Symbol | Description |
|---|---|
| $\mathbf{x}_{(i)}$ | A local part of the vector, i.e. $\mathbf{x}_{(i)} := \sum_{B \in T_i} [\mathbf{x}]_B.$ |
| $(\mathbf{x}^T\mathbf{y})_{(i)}$ | The inner product from a local part, i.e. $(\mathbf{x}^T\mathbf{y})_{(i)} := \sum_{B \in T_i} ([\mathbf{x}]_B^T [\mathbf{y}]_B).$ |
| $\lVert \mathbf{x} \rVert_{(i)}$ | The norm from a local part, i.e. $\lVert \mathbf{x} \rVert_{(i)} := (\mathbf{x}^T\mathbf{x})_{(i)}.$ |
Sparsity
A vector is sparse means that we could use the local part to replace the whole part, i.e. $\mathbf{a} = \mathbf{a}_{(i)}$.
An example is, the gradient mapping $\mathbf{g}_i(\mathbf{x})$ is sparse. To prove this, we need to specify two facts:
- The unbiased estimator $\mathbf{v}_i$ has two parts. The first part, i.e. $\nabla f_i (\mathbf{x}_k) - \nabla f_i (\mathbf{x}_{[i]})$ is only related to $f_i$; The second part, i.e. $\mathbf{D}_i \frac{1}{n} \sum_j \nabla f_j (\mathbf{x}_{[j]})$ is constrained by the diagonal matrix $\mathbf{D}_i$. Therefore, $\forall~B \notin T_i$, $[\mathbf{v}_i]_B = \mathbf{0}$.
- The local part of the proximal operator over $\gamma \varphi_i$ has two case. When $B \in T_i$, we have $[\mathrm{prox}_{\gamma \varphi_i}(\mathbf{x})]_B = [\mathrm{prox}_{\gamma d_B h_B}(\mathbf{x})]_B$; When $B \notin T_i$, $[\mathrm{prox}_{\gamma \varphi_i}(\mathbf{x})]_B = [\mathbf{x}]_B$. This property is due to the definition of $\varphi_i$.
Hence, we know that
Sparse inner product
Considering the inner product of two block separable vectors. If one element of the inner product is sparse over set $T_i$, the inner product from the whole inputs is equivalent to that from the local parts. Assuming that $\mathbf{a}$ is sparse over $T_i$, and $\mathbf{b}$ could be arbitrary, then
Sparse norm
Similar to what we derive in the case of inner product. If a vector $\mathbf{a}$ is sparse over $T_i$, its norm is equivalent to that from a local part, i.e.
Lemma 7
Gradient mapping inequality:
Denoting $\mathbf{x}^{\ast}$ as the solution, the gradient mapping with an arbitrary input vector $\mathbf{x}$ has such an inequality:
\begin{align} \mathbf{g}_i^T(\mathbf{x} - \mathbf{x}^{\ast}) \geqslant -\frac{\gamma}{2} (\beta - 2) \lVert \mathbf{g}_i \rVert^2 - \frac{\gamma}{2\beta} \lVert \mathbf{v}_i - \mathbf{D}_i \nabla f(\mathbf{x}^{\ast}) \rVert^2 + \left( \mathbf{v}_i - \mathbf{D}_i \nabla f(\mathbf{x}^{\ast}) \right)^T\left( \mathbf{x} - \mathbf{x}^{\ast} \right). \end{align}
proof:
Denote $\mathbf{x}^+ := \mathrm{prox}_{\gamma \varphi_i}(\mathbf{x} - \gamma \mathbf{v}_i)$ as the updated $\mathbf{x}$ of the next step. And from Lemma 6 we know that $\mathbf{x}^{\ast} = \mathrm{prox}_{\gamma \varphi_i}(\mathbf{x}^{\ast} - \gamma \mathbf{D} \nabla f(\mathbf{x}^{\ast})).$ According to Lemma 3, we have
According to the sparsity of $\mathbf{g}_i$, we could derive that
Here we apply a theorem that
Young's inequality:
For arbitrary $\beta > 0$ and two vectors $\mathbf{a},~\mathbf{b}$, we have
\begin{align} 2 \mathbf{a}^T \mathbf{b} \leqslant \frac{\lVert \mathbf{a} \rVert^2}{\beta} + \beta \lVert \mathbf{b} \rVert^2. \end{align}
Then
By dividing $\gamma$ on both sides, we could prove this issue. Note that we claim the equality beween $\lVert \mathbf{v}_i - \mathbf{D} \nabla f(\mathbf{x}^{\ast}) \rVert^2_{(i)}$ and $\lVert \mathbf{v}_i - \mathbf{D}_i \nabla f(\mathbf{x}^{\ast}) \rVert^2$. If we want to prove the equality, we need to expand the local part norm over $T_i$, which would not be discussed here .
Lemma 8
Here we introduced a symbol $\boldsymbol{\alpha}$, which would be an abbreviation of what we discuss in the algorithm.
| Symbol | Description |
|---|---|
| $\mathbf{x},~\mathbf{x}^+$ | An arbitrary parameter $\mathbf{x}$ and its updated version. |
| $\boldsymbol{\alpha}_i,~\boldsymbol{\alpha}_i^+$ | The abbreviation of $\nabla f_i (\mathbf{x}_i).$ The $\boldsymbol{\alpha}_i^+$ is the updated $\boldsymbol{\alpha}_i$. |
| $\bar{\boldsymbol{\alpha}}$ | The average value of $\boldsymbol{\alpha}_i$ over all $i$. |
| $\lVert \mathbf{x} \rVert^2_{\mathbf{D}}$ | An associated distance ($\langle \mathbf{x},~\mathbf{D}\mathbf{x} \rangle$ or $\mathbf{x}^T\mathbf{D}\mathbf{x}$) which is defined by a semi-definite matrix $\mathbf{D}$. |
Upper bound of the gradient estimator variance:
We still use $\mathbf{x}^{\ast}$ to represent the optimal parameters. Then for any parameter $\mathbf{x}$, we have:
\begin{align} \mathbb{E} \lVert \mathbf{v}_i - \mathbf{D}_i \nabla f(\mathbf{x}^{\ast}) \rVert^2 \leqslant 4L \mathrm{B}_f (\mathbf{x},~\mathbf{x}^{\ast}) + 2 \mathbb{E} \lVert \boldsymbol{\alpha}_i - \nabla f_i(\mathbf{x}^{\ast}) \rVert^2. \end{align}
proof:
Considering the inequality that $\lVert \mathbf{a} + \mathbf{b} \rVert^2 \leqslant 2\lVert \mathbf{a} \rVert^2 + 2\lVert \mathbf{b} \rVert^2$, then we have
where
and
By introducing Lemma 2, we have
Lemma 9
Lyapunov inequality:
Denote $\mathcal{L}$ as a function with a parameter $c$,
\begin{align} \mathcal{L}(\mathbf{x},~\boldsymbol{\alpha}) := \lVert \mathbf{x} - \mathbf{x}^{\ast} \rVert^2 + \frac{c}{n} \sum_{i=1}^n \lVert \boldsymbol{\alpha}_i - \nabla f_i (\mathbf{x}^{\ast}) \rVert^2. \end{align}Then we have such an equality that:\begin{equation} \begin{aligned} \mathbb{E}\left[\mathcal{L}(\mathbf{x}^+,~\boldsymbol{\alpha}^+)\right] - \mathcal{L}(\mathbf{x},~\boldsymbol{\alpha}) \leqslant & -\gamma \mu \lVert \mathbf{x} - \mathbf{x}^{\ast} \rVert^2 + \left(4L\gamma^2 - 2\gamma + 2L\frac{c}{n}\right)\mathrm{B}_f(\mathbf{x},~\mathbf{x}^{\ast}) \\[5pt] &+ \left( 2\gamma^2 - \frac{c}{n} \right) \mathbb{E} \lVert \boldsymbol{\alpha}_i - \nabla f_i (\mathbf{x}^{\ast}) \rVert^2. \end{aligned} \end{equation}
proof:
By applying Lemma 7 with $\beta=1$, we have
By applying Lemma 5, we could replace the unbiased estimator. Then we use the Lemma 1 to acquire the Bregman divergence.
Then we use Lemma 8 to convert the second term
Consider the last term, $\alpha_i^+$ is a collection of $\alpha_i$ with only one of it replaced by $\nabla f_i(\mathbf{x})$. Thus we could use the averaging policy and Lemma 2 to estimate the expectation:
Then we could integrate the results
Theorem 1 (convergence)
Convergence of SPS:
For any $a \leqslant 1$, let $\gamma = \frac{a}{5L}$. Denote $\kappa$ as $\frac{L}{\mu}$, then we could define a rate factor $\rho = \frac{1}{5} \min \{ \frac{1}{n},~a\frac{1}{\kappa} \}$ with which the SPS converges geometrically in expectation, i.e.
\begin{equation} \begin{aligned} \mathbb{E} &\lVert \mathbf{x}_t - \mathbf{x}^{\ast} \rVert^2 \leqslant (1-\rho)^t C_0, \\[5pt] \mathrm{s.t.}~& C_0 := \lVert \mathbf{x}_0 - \mathbf{x}^{\ast} \rVert^2 + \frac{1}{5L^2} \sum_{i=1}^n \lVert \nabla f_i (\mathbf{x}_0) - \nabla f_i (\mathbf{x}^{\ast}) \rVert^2, \end{aligned} \end{equation}where we use $\mathbf{x}_t$ to represent the parameter iterated by $t$ steps.
proof:
Denote $\bar{H}$ as $\frac{1}{n} \sum_i \lVert \boldsymbol{\alpha}_i - \nabla f_i (\mathbf{x}^{\ast}) \rVert^2$, then we could rewrite Lemma 9 and expand the $\mathcal{L}_t$ term as
Then we set the constraint that $\rho \leqslant \frac{1}{3n}$ , thus we could rewrite the last term as
The next step is, choosing $\frac{c}{n} = 3 \gamma^2$ so that the last term could be removed,
Since we have known that $\gamma = \frac{a}{5L}$, then we know that $10L\gamma^2 - 2\gamma \leqslant 0$ could be verified. That is because of the boundaries of $\rho$ and $c$ that we have set before. Thus, we could remove the second term.
Thus we know, when $\rho \leqslant \frac{a}{5\kappa}$, we could verify that $\mathbb{E}\left[\mathcal{L}_{t+1}\right] \leqslant (1-\rho)\mathcal{L}_t$.
The inequality above is not relevant to the step $t$, thus we could use the mathematical induction to verify that:


(Webdev) -- This morning, Mr. Brion, our class moderator, read us a poem to reflect on. The peom was entitled "Buti pa sila". I was struck with epiphany as every line was read to us. Some lines made me laugh while others made me feel guilty. The last line was the line that shook me the most.
I remember my class moderator say the author's name, but neither
one of us remembers. There are many versions in the internet, but this
was the closest I found.
"Buti pa ang Projector Screen tinitignan, pero ang nasa taas nito ay hindi." our moderator added. He related it to God. He was right. We marvel at the things we have, the people we know, but not the only Creator. He left us with that thought to think of.