This page disables CSP2, but uses a CSP2 manifest directive. It should log an informative message to the console: not an error, showing that the directive is parsed correctly but experimental.


--- layout: post title: Four Practical Ways to Avoid Packing on the Pounds date: 2012-12-02 08:56:58.000000000 -08:00 type: post parent_id: '0' published: true pd: '' status: publish categories: - Classroom Talk - Guest Posts tags: - college 15 - College Prep - Freshman 15 - health author: login: k email: k@k display_name: mcary first_name: '' last_name: '' ---
Guest Post By Melissa Miller
We've all heard the horror stories regarding weight gain in college. Through Facebook, I've seen one too many pictures of high school jocks and homecoming queens going off to college only to return looking much more filled out. It's a sad tale, and it happens more than it should. Even though it's a highly publicized trend, thousands of freshman students still manage to gain unwanted weight each and every year. What gives? Well, aside from the fact that freshman are being introduced to a whole new world of late-night study sessions and intensive, stressful courses, they often find themselves reaching for one-too-many midnight snacks, skipping their daily exercise sessions, and drowning their sorrows and stresses in alcohol. If you're preparing to venture off to university for the first time or just want to integrate some healthier habits into your routine, here are four ways to avoid packing on the pounds in college.
Find a Devoted Workout Buddy
Working out is the best way to avoid weight gain. Unfortunately, it's ever so easy to neglect your normal workout routine in college, especially when you get overloaded and busy. Although you might want to attend every party possible and cram in as much last-minute studying as you can, you should never skimp on workouts. If you realize you're not hitting the gym as much as you should, the best thing to do is find an inspiring, reliable workout buddy. As long as they drag you to the gym as opposed to ice cream socials, then you and your partner should be able to stick to a solid, effective routine.
Keep a Food Diary
This is a tedious tip, but it's a helpful one. Sometimes we deny how much we've been eating when we notice we've gained weight. Especially when you're having a good time, the last thing you want to admit is that you've been overindulging or living unhealthy. Eating sinful foods every once in a while can't hurt, but when it becomes a habit, you're only doing harm to your body. To prevent unexpected calories from creeping up on your waistline, document every single thing you eat in a journal. By making yourself acknowledge every morsel and crumb that enters your mouth, you'll be able to hold yourself accountable for any weight gain.
Join Weight Watchers
One of the best ways to solve a problem is to prevent it from happening in the first place. Weight Watchers isn't just a source for people losing weight; it's also a helpful outlet for those trying to maintain weight. By joining Weight Watchers, you'll preemptively strike and prevent weight gain from happening in the first place. The Weight Watchers system is a highly successful, highly trusted resource, so if you're really serious about avoiding any weight gain whatsoever, join Weight Watchers and start tracking your daily food intake.
Weigh Yourself Every Day
Scales can either be your friend or your foe; it's up to you. If you make time for stepping on the scale every morning, you'll be able to recognize and acknowledge any weight problems as they come. The longer you delay it, however, the more dreadful your weigh-ins will become. I've had friends who say they limit their weigh-ins to once a week, as weight does tend to fluctuate from day to day. Yet I think it's a much better idea to weigh every day, since once a week is much easier to skip out on. So as much as you might hate it, go ahead and step on the scale every morning. Once you see your weight remember it and then repeat the routine again the next morning.  Heck, it might even become something you look forward to.
Weight gain is not a pleasant topic to talk about, yet it's something almost everybody struggles with from time to time. If you're about to venture off to college, try integrating these four habits into your routine to prevent gaining any weight.
Melissa Miller is health blogger and freelance writer for associatedegreeonline.com. She is interested in education and health trends and writes to help students navigate the challenging world of academics and nutrition. Email any questions to melissamiller831@gmail.com


Monkey patches dm-serialization to_json method to add ruby_class: YourClass to all serialized objects
Methods
Public Instance methods
Serialize a Resource to JavaScript Object Notation (JSON; RFC 4627)
@return <String> a JSON representation of the Resource
[ show source ]
# File lib/restfulx/datamapper_foo.rb, line 16 16: def to_json(*args) 17: options = args.first || {} 18: result = '{ ' 19: fields = [] 20: 21: propset = properties_to_serialize(options) 22: 23: fields += propset.map do |property| 24: "#{property.name.to_json}: #{send(property.getter).to_json}" 25: end 26: 27: fields << "\"ruby_class\": #{self.class.to_json}" 28: 29: # add methods 30: (options[:methods] || []).each do |meth| 31: if self.respond_to?(meth) 32: fields << "#{meth.to_json}: #{send(meth).to_json}" 33: end 34: end 35: 36: # Note: if you want to include a whole other model via relation, use :methods 37: # comments.to_json(:relationships=>{:user=>{:include=>[:first_name],:methods=>[:age]}}) 38: # add relationships 39: # TODO: This needs tests and also needs to be ported to #to_xml and #to_yaml 40: (options[:relationships] || {}).each do |rel,opts| 41: if self.respond_to?(rel) 42: fields << "#{rel.to_json}: #{send(rel).to_json(opts)}" 43: end 44: end 45: 46: result << fields.join(', ') 47: result << ' }' 48: result 49: end


Demonstrates the use of Bing layers.
This example demonstrates the ability to create layers using tiles from Bing maps.


The Classic Spectator
Athletically Inspired Casual Clothes: Delivered
Toggle Big Images
Remove my email address from semi-hourly offer mailings
home
Men's Clothes
Women's Clothes
Unisex Clothes
Clasic Memorabilia
Sporting Goods


Follow @MehdiChch
on Twitter for help and suggestions! Or add a new issue on
GitHub
.
From
To
Display notifications feature
Display portfolio feature
Refresh delay
seconde(s)
Icon color change
Portfolio conversion in badge text
Round badge text
Animate badge text
Sound
Mute
Pop
Click-Clack
Clack-Click
Click-Off
Pling
Ting
Save changes


A required minimum amount of time has been reached to unlock activities in the course.
Would you like to refresh this page to unlock any restricted activities or resources? Refreshing the page may result in the loss of unsaved work.


Want to get in touch with me? If you wanna talk about anything, be it to request some information about myself, some tips for coding, psychology, random sudoku puzzle, a book or some evolutionary question, feel free to message me.
I promise to try to reply asap.
Noida, India
mohitagarora@gmail.com


Ratings are counted as an input and can be used as such.
Rating: {{basicRating}}Ratings can take on different icons to use for the rating system.
Rating: {{starRating}} Rating: {{heartRating}}Ratings can also come in different sizes.
A set of components that will build out the rating system.
| Usage | Description | |
|---|---|---|
| .ui.rating |

 | The rating wrapper that will build out the rating icons and create the input for that element. |
Settings that can be added to the elements to change the functionality of the rating component.
| Default | Description | |
|---|---|---|
| [data-rating] | [rating] | 4 | The amount of rating icons the rating object will have. |
| [clearable] | true | Clearable if the current rating is selected twice. |


{{template "header.html" .}}
{{template "titlebar.html" .}}
Compare Results
Vertical to compare:
arch
config
os
scale
Compare
Use latest results up to:


A big, big change!
The Monospace conference just got bigger! First of all, we are moving the location to Boston. Second, the date will now be in July because, well, Boston gets a bit cold in October. Now, the reason for this change is simple: all the Mono engineers will be in Boston during this time and will be available during the conference. To us, this is big. We wanted to revive Monospace to give everyone a chance to learn and participate in discussions and we believe that there will be no better way to do that than to have all the Mono team present, giving talks and mingling with the attendees. This is going to be a huge opportunity for the community!
We are aware that July will be on us soon and we are furiously working towards securing a location and an exact date. We will let you know as soon as possible!
Thank you!


TL;DR: YinChannel helps new content creators figure out what they're doing and attract new followers.
YinChannel is a reboot of the original /YinChannel/ board from the chooperchannel site created in 2011 that was originally an imageboard designed to share user-created content within a small network. The new format reduces the facilitation of individual pieces of user-generated content in favor of aggregating each individual's independent content collections. The goal is to help our content creators more easily reach out to their userbases while remaining independent, while supporting those who want to learn how to become content creators by helping promote new content creators. Support provided by fellow content creators is, after all, the best way to bind a community!
Get in touch with us! We're always looking for new content creators to join our channel and contribute to our network, so feel free to hop in, join the community, and leverage our awesome network of creative geniuses for whatever endeavors you may be undertaking!
You can email our site admin at<EMAIL_ADDRESS>for more information at the moment while we're rebuilding our network!
Probably not. We're working every day to make our portal more and more responsive, and part of that effort involves dropping our old #chan style imageboards. Keep an eye out in the future for more ways to interact with our site and community as we build out our network though!
That's not ruling out a return of our imageboards sometime in the near future though. As we evaluate the different methods available to us with which to interact with our community, we may determine a return of the imageboards in some capacity to be the best solution!
You can join us on our Discord server at the following link to stay in touch!


Hot Restaurant
Current Reservations and Waiting List
Make Reservation
Home
Reservations
Waiting List
Control Panel
Clear Reservation Table |
View Reservations


{{template "base" .}} {{define "title"}}Home{{end}} {{define "body"}}
There's nothing to see here yet!


Alberto Salazar weight-shaming affected athletes' mental health - Steve Magness
Banned coach Alberto Salazar's "obsession" with controlling weight led to mental health problems in athletes, his former assistant has claimed.
Steve Magness was reacting to American athlete Mary Cain's account of ill effects she suffered under Salazar, who received a four-year ban for doping violations.
"I've witnessed the harm and damage that such a culture creates," Magness posted on social media on Thursday.
"It's lasting mental health issues."
In an interview with the New York Times, Cain claimed Salazar's methods at the Nike Oregon Project (NOP) training set-up resulted in her losing her period for three years and broken bones. She also stated she had "suicidal thoughts" and began to cut herself.
Salazar told the publication he "denied many of Cain's claims and had supported her health and welfare". The BBC has also approached the 61-year-old American about Cain's allegations and those made by Magness.
Nike told BBC Sport that Cain's allegations are "completely inconsistent" with its values. However, it added that it previously had not been made aware of the issues and that the athlete was "seeking to rejoin the Oregon Project and Alberto's team as recently as April of this year". It also said it would launch an investigation to hear from former NOP athletes.
Earlier in November, the World Anti-Doping Agency (Wada) announced it would investigate those who trained with Salazar.
He was found guilty of doping violations after a four-year investigation by the US Anti-Doping Agency (Usada) and has said he will appeal against the ruling.
'Salazar was obsessed with weight'
Magness spent 18 months as Salazar's assistant coach, leaving just before the London 2012 Olympics. He later emailed Usada to ask them to look into Salazar and the NOP.
He eventually chose to blow the whistle publicly on the BBC's Panorama programme in 2015, which is what prompted the Usada investigation.
In a series of social media posts on Friday, he said: "Salazar was obsessed with weight. He'd 'joke' about using liposuction or removing your appendix for weight loss. He'd try to get athletes to take shady diet supplements.
"There were other instances. Comments about using thyroid meds to speed up metabolism. Comments to runners about their weight/size of body parts during/after workouts. Encouragement to leave meals hungry and admonishing athletes for eating things like a burger."
Salazar was also quoted to have said that Jenny Simpson's "butt was big", a year after the United States athlete had won 1500m gold at the World Championships in Daegu in 2011.
American Amy Yoder Begley said she was kicked out of the NOP, having finished sixth in the 10,000m at the 2011 national championships.
 This [Cain's interview] brings those painful memories back."
If you or someone you know has been affected by a mental health issue, help and support is available at bbc.co.uk/actionline


I merged yesterday a series of fairly large changes that where sitting in the Development branch of emoncms into the master branch. The master branch version number is now v8.2.1
This release features:
- Improved feed engine status logging (as in INFO, WARN type logs rather than data logging)
- npoints meta data separation for increased robustness
- improved timezone support
Meta data npoints
The count of the number of datapoints in a phpfina, phpfiwa, phptimestore feed was stored in one meta data file (per feed) along with other meta such as the feed start time and feed interval. In the event of a system crash mid-write to a meta file there is a risk that several of these meta properties can get corrupted. Its only the npoints property that needs to be updated regularly the feeds start time and interval only gets written to once at feed creation. By separating the meta data that gets written at the start and the meta data that gets written to regularly into two different files we reduce the risk of corruption as the npoints count can be recovered independently from the data file size divided by 4 bytes per datapoint. The start time and interval are harder to recover from if you dont have a backup.
As of v8.2.1 there are now two meta files per feed one for the meta data that does not get changed and another for the npoints value.
Improved timezone support
Daily data types now use the PHPTimeSeries engine rather than PHPFina and support a midnight reset time set by the user timezone setting under user account. This is a significant change as there was previously minimal support for different timezones if the system time was different from the user timezone.
Testing and upgrading
I've been running this version on my own system for almost two weeks and I havent had any problems, to upgrade to it is just a matter of pulling in the latest changes and creating a fresh copy of the settings.php file. I would recommend doing a backup of your data before doing an upgrade via approach 2 here: http://emoncms.org/site/docs/backup
A restart to the rfm12piphp script or oem_gateway will probably be required.
I'd appreciate feedback on the work to improve timezone support. To make use of it you will unfortunatly need to create new daily feeds using the PHPTimeSeries feed engine which is now the default engine for this feed type (as it will allow hour changes for summer time) (or if your still using mysql thats also fine). You could initially do a parallel test with new PHPTimeSeries daily feeds to check that they give the correct results. You will also need to select your timezone in the user > account page.
I havent yet upgraded emoncms.org, Il do that early next week.
Thanks a lot,
Trystan
Re: emoncms v 8.2.1
Just updated to v8.2.1 and My Electric page has stopped working. I've made a new settings.php, cleared the browser cache and rebooted emoncms.
Please see screenshot below, there is no data, and the wrench does not open the config settings.
Developer tools show the error;
Uncaught TypeError: Cannot read property '1' of undefined myelectric?apikey=503hq2wgeqhj21e4qeqc241b5740b5e:283
Paul
Re: emoncms v 8.2.1
Hello Paul, thanks for testing. Can you check if the data is visible via the normal feeds vis? and do you know which line that error relates too?
Re: emoncms v 8.2.1
For additional debugging it might also be useful to use the new logging tool (thanks to Dave McCraw):
sudo pear channel-discover pear.apache.org/log4php
sudo pear install log4php/Apache_log4php
You may want to change line 102 in settings.php to the location of your emoncms installation if its not via the apt method
ie from:
$log4php_configPath = '/etc/emoncms/emoncms_log4j.xml';
to
$log4php_configPath = '/var/www/emoncms/logconfig.xml';
Create a blank file for the logger to use in the location given in logconfig.xml which by default is the same directory as the emoncms installation and then ensure that log file has write permissions for www-data, pi and root.
sudo chmod 660 emoncms.log
Re: emoncms v 8.2.1
I only have 2 PHPFINA feeds (both power to kwh/d), one of which feeds Myelectric. Both of these feeds are now frozen with no feeds vis, and showing SIZE: 0.0kb. - (They were OK before the update).
Everything else seems OK, all other charts display OK.
There is no line number or other info in the console.
Paul
Re: emoncms v 8.2.1
As for the log... I get the following error logged, repeated 94 times.
WARN - 2014-5-18 09:49:30, PHPFina:get_meta meta file npoints () does not match filesize npoints (9) id=125
WARN - 2014-5-18 09:49:30, PHPFina:post failed to fetch meta id=125
WARN - 2014-5-18 09:49:31, PHPFina:get_meta meta file npoints () does not match filesize npoints (44) id=122
WARN - 2014-5-18 09:49:31, PHPFina:post failed to fetch meta id=122
WARN - 2014-5-18 09:49:38, PHPFina:get_meta meta file npoints () does not match filesize npoints (9) id=125
WARN - 2014-5-18 09:49:38, PHPFina:post failed to fetch meta id=125
WARN - 2014-5-18 09:49:38, PHPFina:get_meta meta file npoints () does not match filesize npoints (44) id=122
WARN - 2014-5-18 09:49:38, PHPFina:post failed to fetch meta id=122
WARN - 2014-5-18 09:49:38, PHPFina:get_meta meta file npoints () does not match filesize npoints (9) id=125
WARN - 2014-5-18 09:49:38, PHPFina:post failed to fetch meta id=125
Paul
Re: emoncms v 8.2.1
Could you try pulling the latest changes, I spotted one error that might be it.
Re: emoncms v 8.2.1
Yes, that's sorted it.
Everything appears ok now.
Thanks
Paul
Re: emoncms v 8.2.1
Brilliant!
Re: emoncms v 8.2.1
Version with fix has now been tagged 8.2.3 https://github.com/emoncms/emoncms/releases/tag/8.2.3
Re: emoncms v 8.2.1
Hmmm,
since updating to V8.2.4 i regularly get feed data that is 'shifted' in time. From 4 feeds shown in a multigraph one starts to be shifted by some hours or days into the future, then another one and at some time all four feeds 'begin' at the current time (?).
I lost all my data due to this some days ago when updating (feeds were overwritten with new data from the start) and recreated all feeds on the weekend. Now I have the same thing again.
Why is the feed meta file touched at all? I thought that with the new changes the meta file is only written once when creating the feed?
BR, Jörg.
Re: emoncms v 8.2.1
Hello Jorg, O dear that doesn't sound good. There must be something going wrong at this point:
https://github.com/emoncms/emoncms/blob/master/Modules/feed/engine/PHPFi...
Looking at it again its not very robust, in the event that the npoints meta file gets set to 0 it will set the starttime to now. I will extend it with a check that will only create the meta file if the start time is also 0 which should only be the case when the feed is first created.
if ($meta->npoints[0] == 0 && $meta->start_time==0) { }
Re: emoncms v 8.2.1
There we are commit done:
https://github.com/emoncms/emoncms/commit/715f47bf90c61066f5484df3a8d192...
version is 8.2.5
https://github.com/emoncms/emoncms/releases/tag/8.2.5
Before upgrading I would suggest everyone do a backup of their phpfiwa, phpfina and phptimestore data directories following approach 2 here: http://emoncms.org/site/docs/backup.
Make sure you disable oem_gateway/emonhub or raspberrypi_run and any posting to the http api's (stop apache) before copying the data files so that when you make the copy the data is in a state where its not being written to. (I've added this as a note to the backup procedure)
Re: emoncms v 8.2.1
Hi Trystan,
good you found something. I will try that immediately.
BR, Jörg.
PS: it was my own fault not to backup the data files before updating emoncms. Everything on my server is automatically backed up every day ........... with the exception of the 'emoncmsdata' directory. Don't ask why! (I think I thought that the size of the data files would be too big! Must have been very late again that day .....)
Re: emoncms v 8.2.1
Hi
Running import_full.php script I get the following errors:-
PHP Notice: Use of undefined constant _ENABLE_PASSWORD_RESET_ - assumed '_ENABL E_PASSWORD_RESET_' in /var/www/emoncms/settings.php on line 56
PHP Notice: Undefined variable: timestore_adminkey in /var/www/emoncms/scripts/ usefulscripts/replication/import_full.php on line 15
Notice: Undefined variable: timestore_adminkey in /var/www/emoncms/scripts/usefu lscripts/replication/import_full.php on line 15
PHP Fatal error: Class 'EmonLogger' not found in /var/www/emoncms/Modules/feed/ feed_model.php on line 30
Fatal error: Class 'EmonLogger' not found in /var/www/emoncms/Modules/feed/feed_ model.php on line 30
I am not using timestore on the server I am backing up as I changed all feeds to the PHP versions.
Regards
Ian
Re: emoncms v 8.2.1
Hello Ian, I think I need to upgrade the backup scripts I will take a look later today.


Contents
Short Descriptions
|
is a FLTK-based block elimination game. |
|
|
is a FLTK-based version of the game of checkers. |
|
|
is a utility script that can be used to get information about the current version of FLTK that is installed on the system. |
|
|
is an interactive GUI designer for FLTK. |
|
|
is an implementation of the popular Sudoku game. |
|
|
contains functions that provide an API to implement graphical user interfaces. |


Defines a simple implementation of linked lists and functions to manipulate it.
More...
Go to the source code of this file.
|
| struct | llist_node_t |
| | Defines a node of the linked list. Holds a void pointer. More...
|
| |
| struct | llist_t |
| | Defines a linked list. If it is empty, head == tail == NULL. Pushing or popping from both the head or tail is always O(1). More...
|
| |
|
| void | llist_init (llist_t *list) |
| | Initialised a linked list. More...
|
| |
| void | llist_push_head (llist_t *list, llist_node_t *node) |
| | Pushes a node to the head of the list. More...
|
| |
| void | llist_push_tail (llist_t *list, llist_node_t *node) |
| | Pushes a node to the tail of the list. More...
|
| |
| llist_node_t * | llist_pop_head (llist_t *list) |
| | Pops a node from the head of the list. More...
|
| |
| llist_node_t * | llist_pop_tail (llist_t *list) |
| | Pops a node from the tail of the list. More...
|
| |
| llist_node_t * | llist_peek_head (llist_t *list) |
| | Peeks the element at the head of the list. More...
|
| |
| llist_node_t * | llist_peek_tail (llist_t *list) |
| | Peeks the element at the tail of the list. More...
|
| |
| bool | llist_remove (llist_t *list, llist_node_t *node) |
| | Removes a node from the queue. More...
|
| |
| llist_t | llist_remove_while (llist_t *list, closure_t *should_remove) |
| | Splits a list in two. The rupture point is determined by the supplied closure. More...
|
| |
| void | llist_insert_at (llist_t *list, llist_node_t *node, closure_t *should_insert) |
| | Scans a list until it finds a suitable spot to insert the provided node. More...
|
| |
Defines a simple implementation of linked lists and functions to manipulate it.
◆ llist_init()
| void llist_init |
( |
llist_t * |
list | ) |
|
Initialised a linked list.
- Parameters
-
| list | The list to be initialised. It will be empty after initialisation. |
◆ llist_insert_at()
| void llist_insert_at |
( |
llist_t * |
list, |
|
|
llist_node_t * |
node, |
|
|
closure_t * |
should_insert |
|
) |
| |
Scans a list until it finds a suitable spot to insert the provided node.
This function iterates the linked list invoking the supplied closure until it returns true. The supplied node is then inserted at this position.
The should_insert
closure will be invoked for each pair of node addresses in the range [NULL, &NODE_1, &NODE_2, ..., &NODE_N, NULL] as parameter. When it finds a suitable position, it must return true.
- Parameters
-
| list | The list to insert the node at |
| node | The node to be inserted |
| should_insert | A closure that returns whether the node ought to be inserted at the current position. The closure function must unpack the closure parameters to a llist_node_t **[2] and return a boolean. |
◆ llist_peek_head()
| llist_node_t* llist_peek_head |
( |
llist_t * |
list | ) |
|
Peeks the element at the head of the list.
- Parameters
-
| list | The list from where the node will be peeked |
- Returns
- node A pointer to the peeked node if it exists. Otherwise, NULL.
◆ llist_peek_tail()
| llist_node_t* llist_peek_tail |
( |
llist_t * |
list | ) |
|
Peeks the element at the tail of the list.
- Parameters
-
| list | The list from where the node will be peeked |
- Returns
- node A pointer to the peeked node if it exists. Otherwise, NULL.
◆ llist_pop_head()
| llist_node_t* llist_pop_head |
( |
llist_t * |
list | ) |
|
Pops a node from the head of the list.
- Parameters
-
| list | The list from where the node will be popped |
- Returns
- node A pointer to the popped node if it exists. Otherwise, NULL.
◆ llist_pop_tail()
| llist_node_t* llist_pop_tail |
( |
llist_t * |
list | ) |
|
Pops a node from the tail of the list.
- Parameters
-
| list | The list from where the node will be popped |
- Returns
- node A pointer to the popped node if it exists. Otherwise, NULL.
◆ llist_push_head()
| void llist_push_head |
( |
llist_t * |
list, |
|
|
llist_node_t * |
node |
|
) |
| |
Pushes a node to the head of the list.
- Parameters
-
| list | The list into which to insert the node |
| node | The node to be inserted. |
◆ llist_push_tail()
| void llist_push_tail |
( |
llist_t * |
list, |
|
|
llist_node_t * |
node |
|
) |
| |
Pushes a node to the tail of the list.
- Parameters
-
| list | The list into which to insert the node |
| node | The node to be inserted. |
◆ llist_remove()
| bool llist_remove |
( |
llist_t * |
list, |
|
|
llist_node_t * |
node |
|
) |
| |
Removes a node from the queue.
- Parameters
-
| list | The list from where the node will be removed |
| node | Address of the node being removed |
- Returns
- Whether the node was found and removed
◆ llist_remove_while()
| llist_t llist_remove_while |
( |
llist_t * |
list, |
|
|
closure_t * |
should_remove |
|
) |
| |
Splits a list in two. The rupture point is determined by the supplied closure.
This function iterates the linked list invoking the provided closure for each node in it. While the closure returns true, nodes are popped from the list.
- Parameters
-
| list | The list from where to pop nodes |
| should_remove | A closure that will be invoked taking each node as parameter. The closure should return a boolean indicating whether the node should be removed or not. |
- Returns
- A new list containing the removed nodes, in their original order.


| Total net profit | -17.03 | Gross profit | 42.83 | Gross loss | -59.86 |
| Profit factor | 0.72 | Expected payoff | -0.26 | ||
| Absolute drawdown | -24.18 | Maximal drawdown (%) | 35.2 (319.42%) | ||
| Total trades | 65 | Profit trades (% of total) | 35 (53.85%) | Loss trades (% of total) | 30 (46.15%) |
| Long positions (won %) | 37 (56.76%) | Short positions (won %) | 28 (50.0%) | ||
| Largest | profit trade | 6.29 | loss trade | -2.98 | |
| Average | profit trade | 1.22 | loss trade | -2.0 | |
| Minimum | profit trade | 0.02 | loss trade | -0.89 | |
| Maximum | take profit in pips | 86.8 | stop loss in pips | -50.0 | |
| Average | take profit in pips | 20.3 | stop loss in pips | -31.1 | |
| Minimum | take profit in pips | -12.3 | stop loss in pips | -13.3 | |
| Maximum | Reward-Risk | 2.8 | Duration in minutes | 543 | |
| Average | Reward-Risk | 0.7 | Duration in minutes | 95.0 | |
| Minimum | Reward-Risk | 0.0 | Duration in minutes | 3 | |
| Maximum | consecutive wins (profit in money) | 6.0 (1.97) | consecutive losses (loss in money) | 4.0 (-8.97) | |
| Average | consecutive wins | 2.0 | consecutive losses | 2.0 |
| Total net profit | -4.24 | Gross profit | 29.21 | Gross loss | -33.45 |
| Profit factor | 0.87 | Expected payoff | -0.11 | ||
| Absolute drawdown | -10.51 | Maximal drawdown (%) | 21.53 (195.37%) | ||
| Total trades | 37 | Profit trades (% of total) | 21 (56.76%) | Loss trades (% of total) | 16 (43.24%) |
| Largest | profit trade | 6.29 | loss trade | -2.98 | |
| Average | profit trade | 1.39 | loss trade | -2.09 | |
| Minimum | profit trade | 0.02 | loss trade | -0.89 | |
| Maximum | take profit in pips | 86.8 | stop loss in pips | -50.0 | |
| Average | take profit in pips | 17.8 | stop loss in pips | -33.1 | |
| Minimum | take profit in pips | -12.3 | stop loss in pips | -13.7 | |
| Maximum | Reward-Risk | 2.7 | Duration in minutes | 543.0 | |
| Average | Reward-Risk | 0.6 | Duration in minutes | 105.6 | |
| Minimum | Reward-Risk | 0.0 | Duration in minutes | 3.0 | |
| Maximum | consecutive wins (profit in money) | 4.0 (0.97) | consecutive losses (loss in money) | 3.0 (-7.96) | |
| Average | consecutive wins | 2.0 | consecutive losses | 1.0 |
| Total net profit | -12.79 | Gross profit | 13.62 | Gross loss | -26.41 |
| Profit factor | 0.52 | Expected payoff | -0.46 | ||
| Absolute drawdown | -14.65 | Maximal drawdown (%) | 12.59 (-611.17%) | ||
| Total trades | 28 | Profit trades (% of total) | 14 (50.0%) | Loss trades (% of total) | 14 (50.0%) |
| Largest | profit trade | 2.32 | loss trade | -2.46 | |
| Average | profit trade | 0.97 | loss trade | -1.89 | |
| Minimum | profit trade | 0.39 | loss trade | -0.98 | |
| Maximum | take profit in pips | 81.7 | stop loss in pips | -39.1 | |
| Average | take profit in pips | 23.7 | stop loss in pips | -28.4 | |
| Minimum | take profit in pips | 5.2 | stop loss in pips | -13.3 | |
| Maximum | Reward-Risk | 2.8 | Duration in minutes | 330.0 | |
| Average | Reward-Risk | 0.9 | Duration in minutes | 81.0 | |
| Minimum | Reward-Risk | Duration in minutes | 3.0 | ||
| Maximum | consecutive wins (profit in money) | 5.0 (4.25) | consecutive losses (loss in money) | 4.0 (-8.97) | |
| Average | consecutive wins | 2.0 | consecutive losses | 2.0 |


---
layout: default
title: Best Dentist Hackensack NJ - Highly Rated Dentist near me. Gentle and Painless Dentist
description: What makes our dental office unique? Read 100's of five star reviews. Check out our Yelp and Facebook pages. Why patient choose Dr. Samina Kathawala to be their trusted dentist in NJ. Read reviews on google.
---
I have been seeing Dr. Kathawala for five years now. She is highly skilled and I know I can trust her to do things correctly. Other dentists have cut corners with the way they have taken care of my oral health, but Dr. Kathawala explains to me what needs to be done and why. She has a gentle touch. I am never in pain after seeing her. I am 100% satisfied with her service! - Kristine Aguinaldo
Facebook Reviewer
Amazingly friendly and helpful dentist. Always willing to answer any questions.  Superb.. thank you Dr Samina. - Ajay Kamal
Health Grades Reviewer
Doctor Samina is very friendly and a very nice person. She does a very good job and professional. She is very patient to explain everything to you. Her staff's all are as sweet as her.
- Asha
Google Reviewer
I fell way behind in my dental cleanings and had not visited any dentist for a number of years. I let Dr. Kathawala know at the start of my appointment, and she was very helpful in setting me at ease. Throughout the exam and cleaning, she explained each step and was very aware of my comfort levels and overall experience. - Carl Sagar
Facebook Reviewer
I have been to a dentist but no one explained me the way Dr kathawala
Explained my mouth condition
I always have the gum problem ,she explained me what it is and how it can be treated, since that time onwards on every appointment she use to check my oral hygiene and explain again if I am not following what she explained.
My gums have improved a lot.
I have never seen a dentist who takes care of patient so much.
I would highly recommend her who really wants to take care of their teeth. And wants the right treatment - Shaikh Abdul Rehman
Health Grades Reviewer
Dr. Samina Kathawala is professional and very thorough. She is always warm & welcoming, very attentive, and has a way with calming the nerves (no pun intended!) :) I would highly recommend her. - Collingswood
Google Reviewer
I have been taking my eleven year old to see Dr. Kathawala for over a year now. My kid has had very bad experiences with dentists before her. One dentist scraped her gums with a power cleaning tool during a cleaning (putting her in a lot of pain), another dentist would not give her enough pain killer for an extraction (causing her even more pain). So when we met Dr. Kathawala, my kid was nervous. Dr. Kathawala was very patient and took the time to calm my kid down and she actually respected her hand signals to stop drilling when it was too painful. I highly recommend this doctor. I started seeing her for myself after my child's visit. She is extremely professional and kind. - Calise Hawkins
Facebook Reviewer
Dr. Samina Kathawala, has been my dentist for a long time. Since first appointment, she has been very nice, respectful, friendly, concerned, and above all, very professional and human, on her job, about the treatments and care of my dental health. She always takes her time and dedication to do her best, and I, personally, appreciate that, because it tells a lot, how professional and devoted a doctor is, with his/her patients. Thanks a lot, Dr. Samina, and thanks also, to the the Clinics personnel and medical staff. God bless you all..!! - Alex M. Gonzalez
Health Grades Reviewer
Love Dr. Samina, she is so attentive, answers all questions upfront, explains and always shows the end result, you know you are in good hands when in her chair. No pain at all, she is very gentle. I'm a happy patient, would definitely recommend!. - Mardy Roman
Google Reviewer
Dr Kathawala is awesome. She has a great personality as well as a professional excellence about her. She explains everything to you while she is doing it and makes you feel comfortable and at ease. I would recommend her to anyone that needs a great dentist. I am lucky that she is mine - Ray Manley
Facebook Reviewer
I have been receiving dental care from Dr. Kathawala for a few years now. What I like about her is the professionalism, the care, the way I am kept informed about what is best for me and what the next step in my care. Dr Kathawala is warm, informative, and makes me feel at ease because of my dreaded fear of needles.
I would highly recommend her service to anyone that is looking for a dentist. - Ollie S Donaldson
Health Grades Reviewer
I have been coming to her for a little over 2 years now. Dr. Kathawala is the first dentist I have gotten treated from that does not make me want to run screaming. She is very sweet and super nice. Listens to you to ensure the least amount of pain during those pesky little cavity fill-in of mine. I wouldn't change her for the world. If you're looking for a new dentist I highly recommend her! - Katherine Bultron
Google Reviewer
Dr. Kathawala is THE BEST DENTIST! I became her patient 5 years ago and from my very first appointment with her she has always catered to my needs, she always greets me with a smile and answers any questions I have. Dr. Kathawala is an amazing dentist and makes it her mission to leave me satisfied with her services everytime I have an appointment. I appreciate her so much that I travel from Queens to Bayonne every 6 months just for my dentist appointment. Dr. Kathawala is simply and TRULY the best! - C Spann
Facebook Reviewer
I have been coming to her for a little over 2 years now. Dr. Kathawala is the first dentist I have gotten treated from that does not make me want to run screaming. She is very sweet and super nice. Listens to you to ensure the least amount of pain during those pesky little cavity fill-in of mine. I wouldn't change her for the world. If you're looking for a new dentist I highly recommend her! - Katy P Withluv
Health Grades Reviewer
Dr Samina Kathawala is excellent!! I highly recommend her. She listens to you very patiently and explains everything in detail. - Sabarish
Google Reviewer
My parents and I have been going to Dr.Samina for the past few years. Dr.Samina is a very caring dentist.She makes sure that you are comfortable during your treatment. She is also always willing to answer any questions you have regarding your treatment. I would recommend her to anyone who is looking for a great dentist! - Summen Mushtaq
Facebook Reviewer
Dr. Samina Kathawala, has a distinctive professional approach with her patients. Her kindness and personality makes you feel comfortable as she communicates with you.
Most people do not like visiting the dentist, but Dr. Kathawala has a way to make you feel at ease and comfortable upon your visit.
Dr. Kathawala let's you know exactly what's going on with your teeth, and the best professional solutions to resolve your problems.
I strongly recommend her professional services to anyone. - Richard Tank Greene
Health Grades Reviewer
Dr. Kathawala is a wonderful dentist! She's been taking good care of my teeth for years! I greatly appreciate her professionalism, she's the best! I highly recommend her to anyone that's looking for a GREAT dentist! - Reginald Lanham
Google Reviewer
Dr. Samina is very professional, friendly and approachable. She explains all facts very well. I also take my 3.5 year old daughter to her and my daughter did not feel a thing! I highly recommend her for adults and kids. Happy to have being treated by such a friendly doctor. - Geetanjali Sukhrani
Facebook Reviewer
Where do I begin. She's awesome! She's the best dentist ever. She's a professional at her craft. I never had an issue. I wish there were more like her. I had a liver transplant a couple of years ago...what I liked the most about her she wanted to make sure I was cleared by my doctors before she began any work on me. I was so impressed.
I would highly recommend her to anyone that needs to see a dentist! - Reynaldo Bear
Health Grades Reviewer
Excellent dentist who has performed multiple procedures on me. Very caring and excellent way of explaining procedures. - Chandrasekhar
Google Reviewer
I have been going to Dr Kathawala for the last five years. She is a sweet, competent dentist.
She was the first dentist who explained to me what will be done, how it will be done and once the procedure was done she showed me in a mirror what she had done. Amazing.

I will continue to go to her. Prices as very reasonable compared to any dentist in the area.
Thank you. - Fin Yun
Facebook Reviewer
I have been taking my eleven year old to see Dr. Kathawala for over a year now. My kid has had very bad experiences with dentists before her. One dentist scraped her gums with a power cleaning tool during a cleaning (putting her in a lot of pain), another dentist would not give her enough pain killer for an extraction (causing her even more pain). So when we met Dr. Kathawala, my kid was nervous. Dr. Kathawala was very patient and took the time to calm my kid down and she actually respected her hand signals to stop drilling when it was too painful. I highly recommend this doctor. I started seeing her for myself after my child's visit. She's extremely professional and kind and takes the time to clearly explain any problems you're having with your dental care. - Calise Hawkins
Health Grades Reviewer
I have been searching years for a quality dentist and always would fall for one that had no idea what they were doing. Once I met Dr Kathawala , that search was over. She is very welcoming and best of all gives you her honest advice as to what truly needs to be done. She has been my dentist for over 3 years and will never go anywhere else because she is true gem ! If you want a amazing dentist that gets the job done ,she's your dentist !! You will not regret it ! . - Elvia Gallardo
Google Reviewer
Dr Kathawala has been the best dentist I have ever met in my life. I have been going to her going more than 3 years and I feel so lucky to have her as my dentist. She is a honest ,professional and kind at heart. You will never experience your typical disconnected cold handed dentist once you meet her. I truly recommend her to everyone and trust me you will not regret it . Your search for a quality and amazing dentist will be over ,once you meet her . Guaranteed !!! - Elvia Gallardo
Facebook Reviewer
Love Dr. Samina, she is so attentive, answers all questions upfront, explains and always shows the end result, you know you are in good hands when in her chair. No pain at all, she is very gentle. I'm a happy patient, would definitely recommend! - Tatjana Sinkevica
Health Grades Reviewer
She is kind, sincere, loving. I get excited to go to she her & my kids do as well - she is the best! - Gabrielle
Don't believe us, click the links below to see what others are saying about us...


PoyPet’s harness gives dog owners peace of mind
by Brooke HallNEWS – PoyPet’s dog harness is well designed and easy to clip into place, even with an anxious little pup like mine. Plus, it discourages your canine companion from pulling without choking them.
While hiking in Red River Gorge, Kentucky, my high-energy lab mix got spooked and wiggled herself right out of her harness. It was terrifying and dangerous. Thankfully, no one got hurt. But that harness fail immediately sent me shopping for a stronger, better fitting replacement.
PoyPet’s harness comes in five sizes, and there’s a detailed sizing chart on Amazon to help you get the perfect fit. After taking her measurements, I was surprised that my pup needed a small, whereas I’d typically get her a medium.
The quick-release buttons make it easy to get the harness on and off, with a red button around the neck, and clips on either side of the body.
My favorite feature is the padded handle, which really helps to steer, control and reposition my anxious rescue dog during walks, runs, hikes and even car rides.
It comes in 16 colors and patterns.
Prices range from $19.99 (XS) to $21.99 (XL) on Amazon. Right now, the average rating on Amazon is 4.3 out of 5, with more than 1,800 reviews.


Doctree node classes added by Sphinx¶
Nodes for domain-specific object descriptions¶
-
class
sphinx.addnodes.
desc
(rawsource=’‘, *children, **attributes)[source]¶ Node for object descriptions.
This node is similar to a “definition list” with one definition. It contains one or more
desc_signature
and adesc_content
.
-
class
sphinx.addnodes.
desc_signature
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for object signatures.
The “term” part of the custom Sphinx definition list.
As default the signature is a single line signature, but set
is_multiline = True
to describe a multi-line signature. In that case all child nodes must bedesc_signature_line
nodes.
-
class
sphinx.addnodes.
desc_signature_line
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for a line in a multi-line object signatures.
It should only be used in a
desc_signature
withis_multiline
set. Setadd_permalink = True
for the line that should get the permalink.
-
class
sphinx.addnodes.
desc_addname
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for additional name parts (module name, class name).
-
class
sphinx.addnodes.
desc_type
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for return types or object type names.
-
class
sphinx.addnodes.
desc_returns
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for a “returns” annotation (a la -> in Python).
-
class
sphinx.addnodes.
desc_name
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for the main object name.
-
class
sphinx.addnodes.
desc_parameterlist
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for a general parameter list.
-
class
sphinx.addnodes.
desc_parameter
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for a single parameter.
-
class
sphinx.addnodes.
desc_optional
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for marking optional parts of the parameter list.
New admonition-like constructs¶
Other paragraph-level nodes¶
New inline nodes¶
-
class
sphinx.addnodes.
index
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node for index entries.
This node is created by the
index
directive and has one attribute,entries
. Its value is a list of 5-tuples of(entrytype, entryname, target, ignored, key)
.entrytype is one of “single”, “pair”, “double”, “triple”.
key is categolziation characters (usually it is single character) for general index page. For the detail of this, please see also:
glossary
and issue #2320.
-
class
sphinx.addnodes.
pending_xref
(rawsource=’‘, *children, **attributes)[source]¶ Node for cross-references that cannot be resolved without complete information about all documents.
These nodes are resolved before writing output, in BuildEnvironment.resolve_references.
-
class
sphinx.addnodes.
literal_emphasis
(rawsource=’‘, text=’‘, *children, **attributes)[source]¶ Node that behaves like emphasis, but further text processors are not applied (e.g. smartypants for HTML output).
Special nodes¶
-
class
sphinx.addnodes.
only
(rawsource=’‘, *children, **attributes)[source]¶ Node for “only” directives (conditional inclusion based on tags).
-
class
sphinx.addnodes.
meta
(rawsource=’‘, *children, **attributes)[source]¶ Node for meta directive – same as docutils’ standard meta node, but pickleable.
-
class
sphinx.addnodes.
highlightlang
(rawsource=’‘, *children, **attributes)[source]¶ Inserted to set the highlight language and line number options for subsequent code blocks.
You should not need to generate the nodes below in extensions.
-
class
sphinx.addnodes.
glossary
(rawsource=’‘, *children, **attributes)[source]¶ Node to insert a glossary.
-
class
sphinx.addnodes.
toctree
(rawsource=’‘, *children, **attributes)[source]¶ Node for inserting a “TOC tree”.
-
class
sphinx.addnodes.
start_of_file
(rawsource=’‘, *children, **attributes)[source]¶ Node to mark start of a new file, used in the LaTeX builder only.


What type of Programming Language is best for you?
What colors are you attracted to?
Neutrals
Pastels
Primary
What's your favorite type of candy?
Chocolate
Licorice
Taffy
Who's your favorite Parks and Rec character?
Donna Miegal
Leslie Knope
April Ludgate-Dwyer
Where's your favorite City?
New York City
Seattle
Chicago
What's your favorite house pet?
Cat
Dog
Bird
Find out!
Oh snap! you got Ruby. That's dope.
Oh snap! you got JavaScript. That's dope.
Oh snap! you got HTML. That's dope.


What happens when you ship two dope balls of fluff into the middle of nowhere (a.k.a. Ähtäri Zoo)? Magic happens, brother. Magic.
This site is your chance to see what happens behind the bamboo leaves. Who are we? Why are we here? Where do we come from? What do we do? When did we become so freakin’ famous?
Welcome to the brotherhood.


Welcome to SnackBot. This is an open-source effort to build a robust robotics control platform using the Java programming language and robust programming techniques.


public interface AccelerationStructure
| Modifier and Type | Method and Description |
|---|---|
void |
build(PrimitiveList primitives)
Construct an acceleration structure for the specified primitive list.
|
void |
intersect(Ray r,
IntersectionState istate)
Intersect the specified ray with the geometry in local space.
|
void build(PrimitiveList primitives)
primitives
- void intersect(Ray r, IntersectionState istate)
r
- ray in local spaceistate
- state to store the intersection into


Israeli students win award for making honey without bees
The Technion team’s synthetic honey is produced by the bacterium Bacillus subtilis, which ‘learns’ to make honey following reprogramming in a lab.
by Brian BlumWith the global population of bees in decline, honey could become a rare commodity on supermarket shelves.
A team of 12 students from the Technion – Israel Institute of Technology has been working for the past year on the development of a bee-free honey. It’s produced by the bacterium Bacillus subtilis, which “learns” to make honey following reprogramming in a lab.
Their efforts have paid off sweetly: The team won a gold medal at the recent iGEM (International Genetically Engineered Machine) competition held in Boston.Some 300 teams from universities around the world took part.
This is the sixth gold medal Technion students have won at iGEM since it was established in 2004 by the Massachusetts Institute of Technology to give students the opportunity to experiment in synthetic biology.
In addition to developing the science, an equally important component of iGEM involves creating a real business enterprise. The Technion team created its own website and a catchy company name: BeeFree.
“Our vision is to create a sustainable BeeFree honey using engineered bacteria, which will process a nectar-like solution using secreted enzymes that mimic the honey stomach environment,” the website states.
The team collaborated with two Israeli technology firms: Bio-castle, which makes a product that biodegrades contaminants in water (they usedit to collect the proteins needed to make synthetic honey), and Tsuf Globus, which produces herbal “medical honey.”
Students from six different disciplines – biomedical engineering, medicine, biotechnology and food engineering, industrial management and engineering, chemical engineering, and aerospace engineering – came together to create BeeFree.
The BeeFree team consists of Asaf Licht, Lior Haim, Zeinat Awwad, Nir Litver, Mai Dror, Ofri Warsha, Ilan Brajzblat, Oriyet Tibi, Yehonatan Zur, Dor Ben Meir, Shira Levi, and Lidya Tannenzapf.
One previous iGEM team from Technionpublished an article in the journal ACS Biomaterials Science and Engineering describing how engineered bacteria can be used to detect and measure harmful substances in food and water. Another team patented a device to prevent baldness based on the activity of bacteria in the body.
“The winnings are definitely exciting, but equally important is the intellectual property created around the project,” said Prof. Roee Amit, who heads the Technion’s Synthetic Biology Laboratory for the Decipherment of Genomic Codes.
Participation in iGEM also requires a social element. This year, the Technion group held a hackathon for 10th to 12th graders in the Haifa area on environmental issues and sustainability. The winning group proposed an application that would allow supermarkets to offer lower prices on products about to expire.
For honey-lovers, continued availability of the sweet stuff will be its own reward – at any price.
More News
-
TytoHome medical exam kit now sold across US at Best Buy
By ISRAEL21c Staff -
Israelis won’t let rockets from Gaza ruin weddings
By Abigail Klein Leichman -
TabTale games accompany Charlie’s Angels and Jumanji films
By ISRAEL21c Staff -
Israeli eatery named one of best 25 restaurants in world
By ISRAEL21c Staff -
Teeth carry clues to mysterious early human Aurignacians
By Daniel Ben Tal -
New route to blocking children’s bone cancer
By ISRAEL21c Staff -
CN Traveler readers vote Israel one of 20 top countries
By Abigail Klein Leichman -
Israeli machine converts air into water for Colombians
By Abigail Klein Leichman -
Firms gather in Israel to share ideas for doing good
By Abigail Klein Leichman -
Movies can be early detection system for autism
By Daniel Ben Tal -
OurCrowd and Toyota Tsusho sign tech scouting agreement
By ISRAEL21c Staff -
Israel’s SpaceIL inspires fun escape room touring America
By Abigail Klein Leichman -
Israel’s REE gets behind wheel with Japan’s Hino Motors
By ISRAEL21c Staff -
Israeli tech sector sees biggest growth in investment and deals since 2013
By Brian Blum -
Israeli antiradiation vest to be tested in outer space
By Abigail Klein Leichman -
12,000 foreigners are pursuing higher education in Israel
By Abigail Klein Leichman -
Young Israeli menswear designer wins Lagerfeld Prize
By Abigail Klein Leichman -
Microsoft technology brings Ben-Gurion’s writings alive
By ISRAEL21c Staff -
Dream Doctors bring healing magic to hurricane-hit Bahamas
By Abigail Klein Leichman


AnimationController QML Type
Enables manual control of animations More...
| Import Statement: | import QtQuick 2.7 |
Properties
Methods
Detailed Description
Normally animations are driven by an internal timer, but the AnimationController allows the given animation to be driven by a progress value explicitly.
Property Documentation
[default] animation : Animation |
This property holds the animation to be controlled by the AnimationController.
Note:An animation controlled by AnimationController will always have its running
and paused
properties set to true. It can not be manually started or stopped (much like an animation in a Behavior can not be manually started or stopped).
This property holds the animation progress value.
The valid progress
value is 0.0 to 1.0, setting values less than 0 will be converted to 0, setting values great than 1 will be converted to 1.
Method Documentation
Finishes running the controlled animation in a backwards direction.
After calling this method, the animation runs normally from the current progress point in a backwards direction to the beginning state.
The animation controller's progress value will be automatically updated while the animation is running.
See also completeToEnd() and progress.
Finishes running the controlled animation in a forwards direction.
After calling this method, the animation runs normally from the current progress point in a forwards direction to the end state.
The animation controller's progress value will be automatically updated while the animation is running.
See also completeToBeginning() and progress.
Reloads the animation properties
If the animation properties changed, calling this method to reload the animation definations.


Subversion Security
If you discover a security vulnerability in Subversion, please
email:
<EMAIL_ADDRESS>It is safe to send sensitive reports to this address. List
membership is controlled, and the archives are not publicly
accessible.
Our usual procedure is to
- Make a fix for the vulnerability.
- Discreetly distribute the fix to a few large sites that run
Subversion servers and are trusted to be discreet themselves.
- Release a new version of Subversion (containing just that fix)
and publicly announce the vulnerability on the same day.
This procedure may vary depending on the nature of the
vulnerability and the degree of pre-existing public awareness, of
course.
Please do not reproduce the above email
address on other web pages or in public postings. Due to
the need for responsiveness, the security list is unmoderated, which
makes it particularly vulnerably to spammers. Furthermore, we cannot
easily change its address, even if the list were to start receiving
spam, because it's too important to have a consistent, dependable
place to report security holes.
On this page, the address has been encoded in various ways to
reduce the likelihood of a spam harvester noticing it. But if the
address starts appearing in other places on the Internet, then the
harvesters will inevitably pick it up, and we'll be stuck wading
through ever-increasing amounts of spam, trying not to lose important
vulnerability reports in the noise.


What to expect from virtual reality in 2016
The 10 edgiest innovation ideas of 2016
7 of the biggest innovations that made headlines in 2015


This page helps you to fill out the document for the public consultation on the review of the EU copyright rules.
Just follow these steps:
In the interests of transparency, organisations (including, for example, NGOs, trade associations and commercial enterprises) are invited to provide the public with relevant information about themselves by registering in the Interest Representative Register and subscribing to its Code of Conduct.
If you are a registered organisation, please indicate your Register ID number below. Your contribution will then be considered as representing the views of your organisation.
If your organisation is not registered, you have the opportunity to register now. Responses from organisations not registered will be published separately.


President Buhari Mourns Former Minister, Alex Akinyele
by Enioluwa AdeniyiPresident Muhammadu Buhari has sympathised with the family of Alexandre Opeyemi Akinyele, the former Minister of Information.
Naija News had earlier reported that the former Minister of Information died on Thursday after a brief illness.
The spokesperson of President Buhari, Mr Femi Adesina, said, “The President commiserates with the government and people of Ondo State, friends and professional colleagues of the departed, urging them to take solace in the lasting legacies of the former colourful minister, who also served as Chairman of National Sports Commission and Chairman of National Reconciliation Committee.”
“As a trained teacher and public relations specialist, President Buhari affirms that Chief Akinyele’s scrupulousness, discipline and diligence were always obvious to those who worked closely with him, and his love for the nation, which was zealously demonstrated through providing exemplary leadership.”
“The President prays that the Almighty God will grant the soul of the former minister eternal rest, and comfort all members of his family.”


On the left pane are listed all the chat rooms you have joined. Above My rooms you can:
On any chat room, you can:
Inside a chat room, on any participant, you can retrieve his/her details by hovering the mouse on


Report Parameters
Azure Billing Report
Billing Trend
Billing by Category (Dollars)
$44
$35
$16
Billing by Service
Overall Billing for Azure Usage and IG Services
| Azure Usage | $68.98 |
|---|
Azure VMs In-Scope for IG Services
| Project | Virtual Machine | Monitoring | Patching | Remediation |
|---|---|---|---|---|
| Total Count | Update software | $24.58 | $24.40 |


--- layout: post title: Embed Gists into a Markdown File date: '2012-10-22T13:35:00-05:00' tags: - gist - github - code - markdown permalink: /post/34105834146/embed-gists-into-a-markdown-file ---
Have you ever wanted to convert all the code blocks in a markdown file to embedded gists?
Well now you can! Embed gists into markdown by copying this script into a new file, and run it with the name of the markdown file you want to convert:
Useful for formatting lots of code on a blog if you don’t have prettify.js.
Enjoy!


Event Handling with jQuery - Short hand helpers
hover()
click()
dblclick()
resize()
Hover over this block to change its background clor. Click, Double click and resize the main window


At the end of the lab, upload to your student google drive:
the slicing file from Prusa Slicer (.3mf file format)
the 3D printable file as exported from Prusa Slicer (.gcode file format)
2-3 photos (.jpg or .png) of your 3D printed lamp, at least one photo where both parts are assembled
(1) Rotoscope
(2) Photos
(3) Video
(4) Slides
Use the slide template called 'Final Presentation' in your team's google drive folder. Feel free to change the layout.
(5) Grading
While the weekly milestones will only check if you did something that was appropriate for the weekly work required by each student (i.e., did you do the work or not), the midterm presentation will be graded by the quality of the work you accomplished so far and how much progress you have made towards the overall project goal. We will look at what you set out to do in your technical proposal, and ask ourselves how much of the described work and at what quality you have delivered results so far? The midterm happens when around 2/3rds of the time for the project is already over, so we expect that you either already have a minimum viable product or are very close to it. In addition, we will consider the presentation quality in this grade portion as well.
20% Final Presentation The final presentation is graded similar to the midterm presentation, i.e. we will look at what you set out to do in your technical proposal, and ask ourselves how much of the described work and at what quality you have delivered in your final presentation. In addition, we will consider the presentation quality in this grade portion as well.
Deliverables
At the end of the lab, upload to your student google drive:
the slicing file from Prusa Slicer (.3mf file format)
the 3D printable file as exported from Prusa Slicer (.gcode file format)
2-3 photos (.jpg or .png) of your 3D printed lamp, at least one photo where both parts are assembled


Tome of Mastering
Download
Tools
Graph
About
Green Dragon
Subset of
Chromatic Dragons
Part of
Type: Dragon
Speaker of
Language: Common
Speaker of
Language: Draconic
Maker of
Treachery
Found in
Forest
Protector of
Treasures
Hunter of
Elves
Master of
Bugbear
Master of
Goblin
Master of
Hobgoblin
Master of
Ettercap
Master of
Ettin
Master of
Kobold
Master of
Orc
Master of
Yuan-ti
Protector of
Emeralds


Dash
board
Courses
Grades
Upcoming events
Log out
Welcome to your Dashboard
Don't forget to register for the Hackathon!


Guidance
Information for organisations thinking about applying to the register of apprentice assessment organisations.
PDF , 246KB , 13 pages
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
PDF , 506KB , 23 pages
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
PDF , 479KB , 11 pages
If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email<EMAIL_ADDRESS>. Please tell us what format you need. It will help us if you say what assistive technology you use.
If you are interested in delivering the knowledge, skills and behaviours elements of the apprenticeship do not apply to the Register of Apprentice Assessment Organisations.
You need to apply to the register of apprenticeship training providers .
Before you start an application you must read the ‘pre-application guidance’. It contains important information about the application process and the type of organisation that we are seeking for the register.
We have been working closely with the Education and Training Foundation to extend their successful Future Apprenticeships programme to support the end-point assessment process.
The support package, which includes a one-day workshop and follow-up activity with action planning, looks at the detail, the issues and opportunities for organisations considering delivering end-point assessment. The workshop in particular will help understand end-point-assessment processes, possible organisational models and the Register of Apprenticeship Assessment Organisations.
Organisations considering applying to the register may want to take advantage of this development activity being offered through the Future Apprenticeships programme .
Published: 16 October 2016
Updated: 7 February 2017


--- title: "OpenAI Gym's FrozenLake: Converging on the true Q-values" author: "Gertjan Verhoeven" date: '2021-03-07' summary: This blog post concerns a famous toy problem in Reinforcement Learning, the [FrozenLake environment](https://gym.openai.com/envs/FrozenLake-v0/). We compare solving an environment with RL by reaching **maximum performance** versus obtaining the **true state-action values** $Q_{s,a}$. slug: frozenlake-qlearning-convergence draft: no categories: - Machine Learning tags: - Reinforcement learning - Python - OpenAI Gym - AI baseurl: https://gsverhoeven.github.io header: image: headers/ryan-fishel-xm9NIbVD-8E-unsplash_ed.png preview: no ---
(Photo by Ryan Fishel on Unsplash)
This blog post concerns a famous “toy” problem in Reinforcement Learning, the FrozenLake environment. We compare solving an environment with RL by reaching maximum performance versus obtaining the true state-action values \(Q_{s,a}\). In doing so I learned a lot about RL as well as about Python (such as the existence of a ggplot
clone for Python, plotnine
, see this blog post for some cool examples).
FrozenLake was created by OpenAI in 2016 as part of their Gym python package for Reinforcement Learning. Nowadays, the interwebs is full of tutorials how to “solve” FrozenLake. Most of them focus on performance in terms of episodic reward. As soon as this maxes out the algorithm is often said to have converged. For example, in this question on Cross-Validated about Convergence and Q-learning:
In practice, a reinforcement learning algorithm is considered to converge when the learning curve gets flat and no longer increases.
Now, for Q-learning it has been proven that, under certain conditions, the Q-values convergence to their true values. But does this happens when the performance maxes out? In this blog we’ll see that this is not generally the case.
We start with obtaining the true, exact state-action values. For this we use Dynamic Programming (DP). Having implemented Dynamic Programming (DP) for the FrozenLake environment as an exercise notebook already (created by Udacity, go check them out), this was a convenient starting point.
We need various Python packages and modules.
# data science packages
import numpy as np
import pandas as pd
import plotnine as p9
import matplotlib.pyplot as plt
%matplotlib inline
# RL algorithms
from qlearning import *
from dp import *
# utils
from helpers import *
from plot_utils import plot_values
import copy
import dill
from frozenlake import FrozenLakeEnv
env = FrozenLakeEnv(is_slippery = True)
Winter is here. You and your friends were tossing around a frisbee at the park when you made a wild throw that left the frisbee out in the middle of the lake (G). The water is mostly frozen (F), but there are a few holes (H) where the ice has melted.
If you step into one of those holes, you’ll fall into the freezing water. At this time, there’s an international frisbee shortage, so it’s absolutely imperative that you navigate across the lake and retrieve the disc. However, the ice is slippery, so you won’t always move in the direction you intend.
The episode ends when you reach the goal or fall in a hole. You receive a reward of 1 if you reach the goal, and zero otherwise.
The agent moves through a \(4 \times 4\) gridworld, with states numbered as follows:
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]
[12 13 14 15]]
and the agent has 4 potential actions:
LEFT = 0
DOWN = 1
RIGHT = 2
UP = 3
The FrozenLake Gym environment has been amended to make the one-step dynamics accessible to the agent. For example, if we are in State s = 1
and we perform action a = 0
, the probabilities of ending up in new states, including the associated rewards are contained in the \(P_{s,a}\) array:
env.P[1][0]
[(0.3333333333333333, 1, 0.0, False),
(0.3333333333333333, 0, 0.0, False),
(0.3333333333333333, 5, 0.0, True)]
The FrozenLake environment is highly stochastic, with a very sparse reward: only when the agent reaches the goal, a reward of +1
is obtained. This means that if we do not set a discount rate, the agent can keep on wandering around without receiving a learning “signal” that can be propagated back through the preceding state-actions (since falling into the holes does not result in a negative reward) and thus learning very slowly. We will focus on the discounting case (gamma = 0.95
) for this reason (less computation needed for convergence), but compare also with the undiscounted case.
Let us solve FrozenLake first for the no discounting case (gamma = 1
).
The Q-value for the first state will then tell us the average episodic reward, which for FrozenLake translates into the percentage of episodes in which the Agent succesfully reaches its goal.
policy_pi, V_pi = policy_iteration(env, gamma = 1, theta=1e-9, \
verbose = False)
plot_values(V_pi)
The Q-value for state s = 0
is 0.824
. This means that for gamma = 1
and following an optimal policy \(\pi^*\), 82.4% of all episodes ends in succes.
As already mentioned above, for computational reasons we will apply Q-learning to the environment with gamma = 0.95
. So… lets solve FrozenLake for gamma = 0.95
as well:
# obtain the optimal policy and optimal state-value function
policy_pi_dc, V_pi_dc = policy_iteration(env, gamma = 0.95, theta=1e-9, \
verbose = False)
Q_perfect = create_q_dict_from_v(env, V_pi_dc, gamma = 0.95)
df_true = convert_vals_to_pd(V_pi_dc)
plot_values(V_pi_dc)
Now, comparing the optimal policies for gamma = 0.95
and for gamma = 1
(not shown here) we find that they are not the same. Therefore, 82.4% succes rate is likely an upper bound for gamma = 0.95
, since introducing discounting in this stochastic environment can (intuitively) either have no effect on the optimal policy, or favor more risky (lower succes rate) but faster (less discounting) policies, leading to a lower overall succes rate.
For example, for the undiscounted case, the Agent is indifferent in choosing direction in the first state. If it ends up going right , it can choose UP and wander around at the top row at no cost until it reaches the starting state again. As soon as this wandering around gets a cost by discounting we see (not shown) that the Agent is no longer indifferent, and does NOT want to end up wandering in the top row, but chooses to play LEFT in state s = 0
instead.
Now that we have the \(Q_{s,a}\) values corresponding to the optimal policy given that gamma = 0.95
, we can check its performance. To do so, we use brute force and simulate the average reward under the optimal policy for a large number of episodes.
To do so, I wrote a function test_performance()
by taking the q_learning()
function, removing the learning (Q-updating) part and setting epsilon to zero when selecting an action (i.e. a pure greedy policy based on a given Q-table).
Playing around with the binomial density in R
(summary(rbinom(1e5, 1e5, prob = 0.8)/1e5
) tells me that choosing 100.000 episodes gives a precision of around three decimals in estimating the probability of succes. This is good enough for me.
# Obtain Q for Gamma = 0.95 and convert to defaultdict
Q_perfect = create_q_dict_from_v(env, V_pi_dc, gamma = 0.95)
fullrun = 0
if fullrun == 1:
d = []
runs = 100
for i in range(runs):
avg_scores = test_performance(Q_perfect, env, num_episodes = 1000, \
plot_every = 1000, verbose = False)
d.append({'run': i,
'avg_score': np.mean(avg_scores)})
print("\ri {}/{}".format(i, runs), end="")
sys.stdout.flush()
d_perfect = pd.DataFrame(d)
d_perfect.to_pickle('cached/scores_perfect_0.95.pkl')
else:
d_perfect = pd.read_pickle('cached/scores_perfect_0.95.pkl')
round(np.mean(d_perfect.avg_score), 3)
0.781
Thus, we find that with the true optimal policy for gamma = 0.95
, in the long run 78% of all episodes is succesful. This is therefore the expected plateau value for the learning curve in Q-learning, provided that the exploration rate has become sufficiently small.
Let’s move on to Q-learning and convergence.
The typical RL tutorial approach to solve a simple MDP as FrozenLake is to choose a constant learning rate, not too high, not too low, say \(\alpha = 0.1\). Then, the exploration parameter \(\epsilon\) starts at 1 and is gradually reduced to a floor value of say \(\epsilon = 0.0001\).
Lets solve FrozenLake this way, monitoring the learning curve (average reward per episode) as it learns, and compare the learned Q-values with the true Q-values found using DP.
I wrote Python functions that generate a decay schedule, a 1D numpy array of \(\epsilon\) values, with length equal to the total number of episode the Q-learning algorithm is to run. This array is passed on to the Q-learning algorithm, and used during learning.
It is helpful to visualize the decay schedule of \(\epsilon\) to check that it is reasonable before we start to use them with our Q-learning algorithm. I played around with the decay rate until the “elbow” of the curve is around 20% of the number of episodes, and reaches the desired minimal end value (\(\epsilon = 0.0001\)).
def create_epsilon_schedule(num_episodes, \
epsilon_start=1.0, epsilon_decay=.9999, epsilon_min=1e-4):
x = np.arange(num_episodes)+1
y = np.full(num_episodes, epsilon_start)
y = np.maximum((epsilon_decay**x)*epsilon_start, epsilon_min)
return y
epsilon_schedule = create_epsilon_schedule(100_000)
plot_schedule(epsilon_schedule, ylab = 'Epsilon')
My version of the Q-learning algorithm has slowly evolved to include more and more lists of things to monitor during the execution of the algorithm. Every 100 episodes, a copy of \(Q^{*}_{s}\) and of \(Q_{s, a}\) is stored in a list, as well as the average reward over this episode, as well as the final \(Q_{s,a}\) table, and \(N_{s,a}\) that kept track of how often each state-action was chosen. The dill
package is used to store these datastructures on disk to avoid the need to rerun the algorithm every time the notebook is run.
# K / N + K decay learning rate schedule
# fully random policy
plot_every = 100
n_episodes = len(epsilon_schedule)
fullrun = 0
random.seed(123)
np.random.seed(123)
if fullrun == 1:
Q_sarsamax, N_sarsamax, avg_scores, Qlist, Qtable_list = q_learning(env, num_episodes = n_episodes, \
eps_schedule = epsilon_schedule,\
alpha = 0.1, \
gamma = 0.95, \
plot_every = plot_every, \
verbose = True, log_full = True)
with open('cached/es1_Q_sarsamax.pkl', 'wb') as f:
dill.dump(Q_sarsamax, f)
with open('cached/es1_avg_scores.pkl', 'wb') as f:
dill.dump(avg_scores, f)
with open('cached/es1_Qlist.pkl', 'wb') as f:
dill.dump(Qlist, f)
with open('cached/es1_Qtable_list.pkl', 'wb') as f:
dill.dump(Qtable_list, f)
else:
with open('cached/es1_Q_sarsamax.pkl', 'rb') as f:
Q_sarsamax = dill.load(f)
with open('cached/es1_avg_scores.pkl', 'rb') as f:
avg_scores = dill.load(f)
with open('cached/es1_Qlist.pkl', 'rb') as f:
Qlist = dill.load(f)
with open('cached/es1_Qtable_list.pkl', 'rb') as f:
Qtable_list = dill.load(f)
Q_es1_lasttable = [np.max(Q_sarsamax[key]) if key in Q_sarsamax \
else 0 for key in np.arange(env.nS)]
Lets plot the learning curve. Here plotnine
, a ggplot
clone in Python comes in handy.
As we can see below, the “recipe” for solving FrozenLake has worked really well. The displayed red line gives the theoretical optimal performance for gamma = 0.95
, I used a loess
smoother so we can more easily compare the Q-learning results with the theoretical optimum.
We can see that the Q-learning algorithm has indeed found a policy that performs optimal. This appears to happen at around 60.000 episodes. We return to this later.
# plot performance
df_scores = pd.DataFrame(
{'episode_nr': np.linspace(0,n_episodes,len(avg_scores),endpoint=False),
'avg_score': np.asarray(avg_scores)})
(p9.ggplot(data = df_scores.loc[df_scores.episode_nr > -1], mapping = p9.aes(x = 'episode_nr', y = 'avg_score'))
+ p9.geom_point(colour = 'gray')
+ p9.geom_smooth(method = 'loess')
+ p9.geom_hline(yintercept = 0.781, colour = "red")
+ p9.geom_vline(xintercept = 60_000, color = "blue"))
But now comes the big question: did the \(Q^{*}_{a}\) estimates converge onto the true values as well?
q_showdown = pd.DataFrame(
{'Q_es1_lasttable': Q_es1_lasttable,
'q_true': V_pi_dc})
q_showdown['state'] = range(16)
q_showdown = pd.melt(q_showdown, ['state'])
(p9.ggplot(data = q_showdown, mapping = p9.aes(x = 'state', y = 'value', color = 'factor(variable)'))
+ p9.geom_point(size = 5, shape = 'x'))
Nope, they did not. Ok, most learned Q-values are close to their true values, but they clearly did not converge exactly to their true value.
To really understand what is going on, I found it helpful to plot the learning curves for all the 16 x 4 = 64 state-action values at the same time. Here plotnine
really shines. I leave out the actual estimates, and only plot a loess
smoothed curve for each of the state-action values over time.
# convert to pandas df
dfm = list_of_tables_to_df(Qtable_list)
#10 s
(p9.ggplot(data = dfm.loc[(dfm.episode > -1)], mapping = p9.aes(x = 'episode', y = 'value', group = 'action', color = 'factor(action)'))
#+ p9.geom_point(shape = 'x')
+ p9.geom_smooth(method = 'loess')
+ p9.geom_hline(data = df_true, mapping = p9.aes(yintercept = 'value'), color = 'red')
+ p9.geom_vline(xintercept = 600, color = 'blue')
+ p9.facet_wrap("variable", ncol = 4)
+ p9.theme(subplots_adjust={'wspace':0.4}) # fix a displaying issue
)
First the main question: what about convergence of \(Q^{*}_{a}\)? For this we need only to look, for each state, at the action with the highest value, and compare that to the true value (red horizontal lines). Most of the values appear to have converged to a value close to the true value, but Q3 is clearly still way off. Note that we using smoothing here to average out the fluctuations around the true value.
We noted earlier that at around episode 60.000, the optimal policy emerges and the learning curve becomes flat. Now, the most obvious reason why performance increases is because the value of \(\epsilon\) is decaying, so the deleterious effects of exploration should go down, and performance should go up.
Another reason that performance goes up could be that the greedy policy is improving. It is interesting to examine whether at this point, meaningfull changes in the greedy policy still occur. Meaningfull changes in policy are caused by changes in the estimated state-action values. For example, we might expect two or more state-action value lines crossing, with the “right” action becoming dominant over the “wrong” action. Is this indeed the case?
Indeed, from the plot above, with the change point at around episode 600 (x 100),a change occurs in Q6, where the actions 0 and 2 cross. However, from the true Q-value table (not shown) we see that both actions are equally rewarding, so the crossing has no effect on performance. Note that only one of the two converges to the true value, because the exploration rate becomes so low that learning for the other action almost completely stops in the end.
lets zoom in then at the states that have low expected reward, Q0, Q1, Q2, and Q3. These are difficult to examine in the plot above, and have actions with expected rewards that are similar and therefore more difficult to resolve (BUT: since the difference in expected reward is small, the effect of resolving them on the overall performance is small as well). For these we plot the actual state-actions values:
(p9.ggplot(data = dfm[dfm.variable.isin(['Q0', 'Q1','Q2', 'Q3'])], mapping = p9.aes(x = 'episode', y = 'value', group = 'action', color = 'factor(action)'))
+ p9.geom_point(shape = 'x')
#+ p9.geom_smooth(method = 'loess')
+ p9.geom_hline(data = df_true[df_true.variable.isin(['Q0', 'Q1','Q2', 'Q3'])], mapping = p9.aes(yintercept = 'value'), color = 'red')
+ p9.geom_vline(xintercept = 600, color = 'blue')
+ p9.facet_wrap("variable", scales = 'free_y')
+ p9.theme(subplots_adjust={'wspace':0.15})
)
From Q1, Q2 and Q3, we can see that exploration really goes down at around episode 500 (x 100) (\(\epsilon\) at this point is 0.007
), and with the optimal action standing out already long before reaching this point.
Only with Q2 there is a portion of the learning curve where action 1 has the highest value, and is chosen for quite some time before switching back to action 0 again at around episode 60.000. Let’s compare with the true values for Q2:
Q_perfect[2]
array([0.15347714, 0.14684971, 0.14644451, 0.13958106])
Indeed, the difference in expected reward between the actions in state 2 is really small, and because of the increasingly greedy action selection, only action 0 converges to its true values, with the other values “frozen” in place because of the low exploration rate.
Now, after analyzing what happens if we apply the “cookbook” approach to solving problems using RL, let’s change our attention to getting convergence for preferably ALL the state-action values.
According to our RL bible (Sutton & Barto), to obtain exact convergence, we need two conditions to hold.
The nice thing of Q-learning is that it is an off-policy learning algorithm. This means that no matter what the actual policy is used to explore the states, the Q-values we learn correspond to the expected reward when following the optimal policy. Which is quite miraculous if you ask me.
Now, our (admittedly, a bit academic) goal is getting all of the learned state values as close as possible to the true values. An epsilon-greedy policy with a low \(epsilon\) would spent a lot of time by choosing state-actions that are on the optimal path between start and goal state, and would only rarely visit low value states, or choose low value state-actions.
Because we can choose any policy we like, I chose a completely random policy. This way, the Agent is more likely to end up in low value states and estimate the Q-values of those state-actions accurately as well.
Note that since theFrozen Lake enviroment has a lot of inherent randomness already because of the ice being slippery, a policy with a low \(\epsilon\) (most of the time exploiting and only rarely exploring) will still bring the agent in low values states, but this would require much more episodes.
For a particular constant learning rate, provided that it is not too high, the estimated Q-values will converge to a situation where they start to fluctuate around their true values. If we subsequently lower the learning rate, the scale of the fluctuations (their variance) will decrease. If the learning rate is gradually decayed to zero, the estimates will converge. According to Sutton & Barto (2018), the decay schedule of \(\alpha\) must obey two constraints to assure convergence: (p 33).
On the interwebs, I found two formulas that are often used to decay hyperparameters of Reinforcement Learning algorithms:
\(\alpha_n = K / (K + n)\) (Eq. 1)
\(\alpha_n = \Sigma^{\infty}_{n = 1} \delta^n \alpha_0\) (Eq. 2)
Again, just as with the decay schedule for \(\epsilon\), it is helpful to visualize the decay schedules to check that they are reasonable before we start to use them with our Q-learning algorithm.
# Eq 1
def create_alpha_schedule(num_episodes, \
alpha_K = 100, alpha_min = 1e-3):
x = np.arange(num_episodes)+1
y = np.maximum(alpha_K/(x + alpha_K), alpha_min)
return y
# Eq 2
def create_alpha_schedule2(num_episodes, \
alpha_start=1.0, alpha_decay=.999, alpha_min=1e-3):
x = np.arange(num_episodes)+1
y = np.full(num_episodes, alpha_start)
y = np.maximum((alpha_decay**x)*alpha_start, alpha_min)
return y
I played a bit with the shape parameter to get a curve with the “elbow” around 20% of the episodes.
alpha_schedule = create_alpha_schedule(num_episodes = 500_000, \
alpha_K = 5_000)
plot_schedule(alpha_schedule, ylab = 'Alpha')
round(min(alpha_schedule), 3)
0.01
This curve decays \(\alpha\) in 500.000 episodes to 0.01
.
The second formula (Equation 2) decays \(\alpha\) even further, to 0.001
:
alpha_schedule2 = create_alpha_schedule2(num_episodes = 500_000, \
alpha_decay = 0.99997)
plot_schedule(alpha_schedule2, ylab = 'Alpha')
min(alpha_schedule2)
0.001
We start with the decay function that follows Equation 1. To get a full random policy, we set \(\epsilon = 1\). Note that this gives awful performance where the learning curve suggests it is hardly learning anything at all. However, wait until we try out a fully exploiting policy on the Q-value table learned during this run!
# K / N + K decay learning rate schedule
# fully random policy
plot_every = 100
alpha_schedule = create_alpha_schedule(num_episodes = 500_000, \
alpha_K = 5_000)
n_episodes = len(alpha_schedule)
fullrun = 0
if fullrun == 1:
Q_sarsamax, N_sarsamax, avg_scores, Qlist = q_learning(env, num_episodes = n_episodes, \
eps = 1,\
alpha_schedule = alpha_schedule, \
gamma = 0.95, \
plot_every = plot_every, \
verbose = True)
with open('cached/as1_Q_sarsamax.pkl', 'wb') as f:
dill.dump(Q_sarsamax, f)
with open('cached/as1_avg_scores.pkl', 'wb') as f:
dill.dump(avg_scores, f)
with open('cached/as1_Qlist.pkl', 'wb') as f:
dill.dump(Qlist, f)
else:
with open('cached/as1_Q_sarsamax.pkl', 'rb') as f:
Q_sarsamax = dill.load(f)
with open('cached/as1_avg_scores.pkl', 'rb') as f:
avg_scores = dill.load(f)
with open('cached/as1_Qlist.pkl', 'rb') as f:
Qlist = dill.load(f)
Q_as1_lasttable = [np.max(Q_sarsamax[key]) if key in Q_sarsamax \
else 0 for key in np.arange(env.nS)]
Here is the learning curve for this run of Q-learning:
# plot performance
plt.plot(np.linspace(0,n_episodes,len(avg_scores),endpoint=False), np.asarray(avg_scores))
plt.xlabel('Episode Number')
plt.ylabel('Average Reward (Over Next %d Episodes)' % plot_every)
plt.show()
print(('Best Average Reward over %d Episodes: ' % plot_every), np.max(avg_scores))
Best Average Reward over 100 Episodes: 0.08
Pretty awfull huh? Now let us check out the performance of the learned Q-table:
fullrun = 0
runs = 100
if fullrun == 1:
d_q = []
for i in range(runs):
avg_scores = test_performance(Q_sarsamax, env, num_episodes = 1000, \
plot_every = 1000, verbose = False)
d_q.append({'run': i,
'avg_score': np.mean(avg_scores)})
print("\ri = {}/{}".format(i+1, runs), end="")
sys.stdout.flush()
d_qlearn = pd.DataFrame(d_q)
d_qlearn.to_pickle('cached/scores_qlearn_0.95.pkl')
else:
d_qlearn = pd.read_pickle('cached/scores_qlearn_0.95.pkl')
round(np.mean(d_qlearn.avg_score), 3)
0.778
Bam! Equal to the performance of the optimal policy found using Dynamic programming (sampling error (2x SD) is +/- 0.008). The random policy has actual learned Q-values that for a greedy policy result in optimal performance!
Ok, so Q-learning found an optimal policy. But did it converge?
Our q_learning()
function made a list of Q-tables while learning, adding a new table every 100 episodes. This gives us 5.000 datapoints for each Q-value, which we can plot to visually check for convergence.
As with the list of state-action tables above, It takes some datawrangling to get the list of Q-tables in a nice long pandas DataFrame suitable for plotting. This is hidden away in the list_to_df()
function.
# 13s
dfm = list_to_df(Qlist)
#10 s
(p9.ggplot(data = dfm.loc[(dfm.episode > -1)], mapping = p9.aes(x = 'episode', y = 'value'))
+ p9.geom_point()
+ p9.geom_smooth(method = "loess", color = "yellow")
+ p9.geom_hline(data = df_true, mapping = p9.aes(yintercept = 'value'), color = 'red')
+ p9.facet_wrap("variable", scales = 'free_y')
+ p9.theme(subplots_adjust={'wspace':0.4}) # fix plotting issue
)
This looks good, lets zoom in at one of the more noisier Q-values, Q14
.
In the learning schedule used, the lowest value for \(\alpha\) is 0.01
.
At this value of the learning rate, there is still considerable variation around the true value.
(p9.ggplot(data = dfm.loc[(dfm.variable == 'Q10') & (dfm.episode > -1)], mapping = p9.aes(x = 'episode', y = 'value'))
+ p9.geom_point()
+ p9.geom_smooth(method = "loess", color = "yellow")
+ p9.geom_hline(data = df_true[df_true.variable == 'Q10'], mapping = p9.aes(yintercept = 'value'), color = 'red')
)
Suppose we did not know the true value of Q(S = 14), and wanted to estimate it using Q-learning. From the plot above, an obvious strategy is to average all the values in the tail of the learning rate schedule, say after episode 400.000.
# drop burn-in, then average Q-vals by variable
Q_est = (dfm.loc[dfm.episode > 4000]
.groupby(['variable'])
.mean()
)
# convert to a 1D array sorted by Q-value
Q_est['sort_order'] = [int(str.replace(x, 'Q', '')) \
for x in Q_est.index.values]
Q_est = Q_est.sort_values(by=['sort_order'])
Q_est_as1 = Q_est['value'].values
Lets compare these with the final estimated values, and with true values:
Q_as1_lasttable = [np.max(Q_sarsamax[key]) if key in Q_sarsamax \
else 0 for key in np.arange(env.nS)]
q_showdown = pd.DataFrame(
{'q_est_as1': Q_est_as1,
'q_est_as1_lasttable': Q_as1_lasttable,
'q_true': V_pi_dc})
q_showdown['state'] = range(16)
q_showdown = pd.melt(q_showdown, ['state'])
(p9.ggplot(data = q_showdown, mapping = p9.aes(x = 'state', y = 'value', color = 'factor(variable)'))
+ p9.geom_point(size = 5, shape = 'x'))
Pretty close! So here we see Q-learning finally delivering on its convergence promise.
And what about final values vs averaging? Averaging seems to have improved the estimates a bit. Note that we had to choose an averaging window based on eyeballing the learning curves for the separate Q-values.
Can we do even better? A learning rate schedule where alpha is lowered further would diminish the fluctuations around the true values, but at the risk of lowering it too fast and effectively freezing (or very slowly evolving) the Q-values at non-equilibrium values.
The second formula decays \(\alpha\) to 0.001
, ten times lower than the previous decay schedule:
# fully random policy
plot_every = 100
alpha_schedule2 = create_alpha_schedule2(num_episodes = 500_000, \
alpha_decay = 0.99997)
n_episodes = len(alpha_schedule2)
fullrun = 0
if fullrun == 1:
Q_sarsamax, N_sarsamax, avg_scores, Qlist = q_learning(env, num_episodes = n_episodes, \
eps = 1,\
alpha_schedule = alpha_schedule2, \
gamma = 0.95, \
plot_every = plot_every, \
verbose = True)
with open('cached/as2_Q_sarsamax.pkl', 'wb') as f:
dill.dump(Q_sarsamax, f)
with open('cached/as2_avg_scores.pkl', 'wb') as f:
dill.dump(avg_scores, f)
with open('cached/as2_Qlist.pkl', 'wb') as f:
dill.dump(Qlist, f)
else:
with open('cached/as2_Q_sarsamax.pkl', 'rb') as f:
Q_sarsamax = dill.load(f)
with open('cached/as2_avg_scores.pkl', 'rb') as f:
avg_scores = dill.load(f)
with open('cached/as2_Qlist.pkl', 'rb') as f:
Qlist = dill.load(f)
dfm = list_to_df(Qlist)
(p9.ggplot(data = dfm.loc[(dfm.episode > -1)], mapping = p9.aes(x = 'episode', y = 'value'))
+ p9.geom_point()
+ p9.geom_smooth(method = "loess", color = "yellow")
+ p9.geom_hline(data = df_true, mapping = p9.aes(yintercept = 'value'), color = 'red')
+ p9.facet_wrap("variable", scales = 'free_y')
+ p9.theme(subplots_adjust={'wspace':0.4})
)
Cleary, the fluctuations are reduced compared to the previous schedule. AND all Q-values still fluctuate around their true values, so it seems that this schedule is better with respect to finding the true values.
Let’s see if the accuracy of the estimated Q-values is indeed higher:
# drop burn-in, then average Q-vals by variable
Q_est = (dfm.loc[dfm.episode > 2000]
.groupby(['variable'])
.mean()
)
# convert to 1d array sorted by state nr
Q_est['sort_order'] = [int(str.replace(x, 'Q', '')) \
for x in Q_est.index.values]
Q_est = Q_est.sort_values(by=['sort_order'])
Q_est_as2 = Q_est['value'].values
Q_as2_lasttable = [np.max(Q_sarsamax[key]) if key in Q_sarsamax \
else 0 for key in np.arange(env.nS)]
q_showdown = pd.DataFrame(
{'Q_as2_lasttable': Q_as2_lasttable,
'q_est_as2': Q_est_as2,
'q_true': V_pi_dc})
q_showdown['state'] = range(16)
q_showdown = pd.melt(q_showdown, ['state'])
(p9.ggplot(data = q_showdown, mapping = p9.aes(x = 'state', y = 'value', color = 'factor(variable)'))
+ p9.geom_point(size = 5, shape = 'x'))
Boom! Now our \(Q^{*}_{a}\) estimates are really getting close to the true values. Clearly, the second learning rate schedule is able to learn the true Q-values compared to the first rate schedule, given the fixed amount of computation, in this case 500.000 episodes each.
Averaging out does not do much anymore, except for states 10 and 14, where it improves the estimates a tiny bit.
In conclusion, we have seen that the common approach of using Q-learning with a constant learning rate and gradually decreasing the exploration rate, given sensible values and rates, will indeed find the optimal policy. However, this approach does not necessary converge to the true state-values. We have to tune the algorithm exactly the other way around: keep the exploration rate constant and sufficiently high, and decay the learning rate. For sufficiently low learning rates, averaging out the fluctuations does not meaningfully increase accuracy of the learned Q-values.


Pitcher
656 alburtus placePittsfield, MA 923874
62 Years Experience
389 alburt rdChicago Heights, IL 923874
54 Years Experience
99 alby stOakland, CA 923874
63 Years Experience


I've put up a copy of Commentary in front of the Python 2.4 documentation at http://pythonpaste.org/comment/python24/
This is kind of a test, to see whether the particulars of Commentary can fit the needs for the Python documentation. People have been asking for the Python documentation to have comments for a long time. So, here's some a system that allows that, see if you think it feels right. But if you leave long and involved comments, it's possible no one will see them. (Well, if there really are good comments and I shut this off, I'll try to pass those comments upstream.)
Feedback welcome, preferred as trac tickets.
I made a note on http://pythonpaste.org/comment/python24/lib/module-logging.html Had to click the ? to figure out how, but the interface was groovy thereafter.
Wow Ian, it looks really good, but I do have my doubt about the per paragraph commenting. Won't more than 4 or 5 comments on a single paragraph completely ruin the layout of the page?
great work!
It can be difficult for the layout. They should just extend downward, but it gets bad when there's lots of comments on one paragraph, and then a comment on the next paragraph. That's ugly. The folded view might be better, or maybe some overflow thing, or otherwise combining comments. Really long comments (which can be very useful) will also cause these problems.
I also want to add a better interface for commenting on the entire page, probably presented as a simple list of comments at the end of the page. Or maybe threaded.
# Ian Bicking
Great idea!
I started looking through the Tutorial, and came a cropper on this page: http://pythonpaste.org/comment/python24/tut/node5.html The long and colorful traceback ends with UnicodeEncodeError: 'ascii' codec can't encode characters in position 18864-18866: ordinal not in range(128)
Yeah, someone (perhaps you) submitted a ticket on that too. I thought I was decoding stuff properly (the Python docs are ISO-8859-1), but apparently I put the configuration in for that without actually implementing it.
# Ian Bicking
Ian, you rock! I didn't check the source code, but at least the visual appeal is incredible!
Thanks!
Indeed, it is amazing! I think this will be most helpful for adding examples in the library section -which I find very dry-. Thanks for this excellent work! I hope it gets "promoted" to the official docs.
# Ismael
Great site, keep up the good work!
asesoe


No.
Name
Address
Salary
Action
No.
Name
Address
Salary
Action
1
Here is primary section
USMAN
Here is primary section
Inike
Here is primary section
25000
Here is primary section
Edit
Here is primary section
No.
Name:
Address:
Salary
Select Section
Nursery
Reception
Primary
Secondary
Select Class
1
2
1A
1B
1C
1D
1E
2A
2B
2C
2D
2E
3A
3B
3C
3D
3E
4A
4B
4C
4D
4E
5A
5B
5C
5D
5E
JSS1A
JSS1B
JSS1C
JSS2A
JSS2B
JSS2C
JSS3A
JSS3B
JSS3C
SS1A
SS1B
SS1C
SS2A
SSBB
SS2C
SS3A
SS3B
SS3C
Select Sex
Male
Female
Select Age
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Enter your data here to update previous record above
Id:
Name:
Address:
Salary
Form
3
Nomber
Name
Address
Salary
Action
Enter your data here to update previous record above
No.
Name:
Address:
Salary


Description:
Cognitive robotics is a new approach to robot programming based on high level primitives for perception and action. These primitives draw inspiration from ideas in cognitive science combined with state of the art robotics algorithms. Students will experiment with these primitives and help develop new ones using the Tekkotsu software framework on the Calliope robot, which includes a 5 degree-of-freedom arm with gripper, a Kinect camera on a pan/tilt mount, and Ubuntu Linux on a dual-core on-board netbook. Prior robotics experience is not necessary, but strong programming skills are required.


Image from Wenatchee Valley College
Model from Sara Carena, Sketchfab
Modified from R. Harwood [http://profharwood.x10host.com/GEOL101/Labs/Sediment/]


import time, os, json
from textops import cat
from manafa.services.service import Service
import re
from manafa.utils.Utils import execute_shell_command, get_results_dir
from manafa.utils.Logger import log
RESULTS_DIR = get_results_dir()
[docs]class AppConsumptionStats(object):
"""Class that stores method traces of apps registered with hunter.
This class parses and batterystats history events from files parsed using parseFile function. It starts by load
information contained in the device power profile file and also the current state values known by the profiler,
stored in definitions attribute.
Attributes:
app_traces (dict): method traces.
results_dir (int): results' directory.
"""
def __init__(self, results_dir=os.path.join(RESULTS_DIR, "consumptions")):
self.results_dir = results_dir
self.app_traces = {}
[docs] def clean(self):
"""clears app_traces attribute.
"""
self.app_traces.clear()
[docs] def get_output_filepath(self, test_id):
"""gets path of output file where the consumption info will be written.
Returns:
str: filepath
"""
filename = f"consumptions-{test_id}.log"
return os.path.join(self.results_dir, filename)
[docs] @staticmethod
def write_consumptions(filename, consumption, function=None):
"""writes consumption info to file.
Args:
filename(str): path of output file.
consumption(float): consumption value.
function: option function name if the consumption is from a function.
"""
if function is not None:
line = "%s : %f Joules\n" % (function, consumption)
else:
line = "\nTOTAL: %f Joules" % consumption
with open(filename, 'a') as filehandle:
filehandle.write(line)
filehandle.close()
[docs] def save_function_info(self, filename, filter_zeros=False):
"""save method traces contained in app_traces to file.
Args:
filename: name of output file.
filter_zeros: filter zeros
Returns:
str: path of the output file
"""
filepath = os.path.join(self.results_dir, filename)
with open(filepath, 'w') as jj:
json.dump(self.app_traces, jj, indent=1)
return filepath