|
FlatBuffers
|
Assuming you have written a schema using the above language in say mygame.fbs
(FlatBuffer Schema, though the extension doesn't matter), you've generated a C++ header called mygame_generated.h
using the compiler (e.g. flatc -c mygame.fbs
), you can now start using this in your program by including the header. As noted, this header relies on flatbuffers/flatbuffers.h
, which should be in your include path.
To start creating a buffer, create an instance of FlatBufferBuilder
which will contain the buffer as it grows:
FlatBufferBuilder fbb;
Before we serialize a Monster, we need to first serialize any objects that are contained there-in, i.e. we serialize the data tree using depth first, pre-order traversal. This is generally easy to do on any tree structures. For example:
auto name = fbb.CreateString("MyMonster"); unsigned char inv[] = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }; auto inventory = fbb.CreateVector(inv, 10);
CreateString
and CreateVector
serialize these two built-in datatypes, and return offsets into the serialized data indicating where they are stored, such that Monster
below can refer to them.
CreateString
can also take an std::string
, or a const char *
with an explicit length, and is suitable for holding UTF-8 and binary data if needed.
CreateVector
can also take an std::vector
. The offset it returns is typed, i.e. can only be used to set fields of the correct type below. To create a vector of struct objects (which will be stored as contiguous memory in the buffer, use CreateVectorOfStructs
instead.
Vec3 vec(1, 2, 3);
Vec3
is the first example of code from our generated header. Structs (unlike tables) translate to simple structs in C++, so we can construct them in a familiar way.
We have now serialized the non-scalar components of of the monster example, so we could create the monster something like this:
auto mloc = CreateMonster(fbb, &vec, 150, 80, name, inventory, Color_Red, Offset<void>(0), Any_NONE);
Note that we're passing 150
for the mana
field, which happens to be the default value: this means the field will not actually be written to the buffer, since we'll get that value anyway when we query it. This is a nice space savings, since it is very common for fields to be at their default. It means we also don't need to be scared to add fields only used in a minority of cases, since they won't bloat up the buffer sizes if they're not actually used.
We do something similarly for the union field test
by specifying a 0
offset and the NONE
enum value (part of every union) to indicate we don't actually want to write this field.
Tables (like Monster
) give you full flexibility on what fields you write (unlike Vec3
, which always has all fields set because it is a struct
). If you want even more control over this (i.e. skip fields even when they are not default), instead of the convenient CreateMonster
call we can also build the object field-by-field manually:
MonsterBuilder mb(fbb); mb.add_pos(&vec); mb.add_hp(80); mb.add_name(name); mb.add_inventory(inventory); auto mloc = mb.Finish();
We start with a temporary helper class MonsterBuilder
(which is defined in our generated code also), then call the various add_
methods to set fields, and Finish
to complete the object. This is pretty much the same code as you find inside CreateMonster
, except we're leaving out a few fields. Fields may also be added in any order, though orderings with fields of the same size adjacent to each other most efficient in size, due to alignment. You should not nest these Builder classes (serialize your data in pre-order).
Regardless of whether you used CreateMonster
or MonsterBuilder
, you now have an offset to the root of your data, and you can finish the buffer using:
fbb.Finish(mloc);
The buffer is now ready to be stored somewhere, sent over the network, be compressed, or whatever you'd like to do with it. You can access the start of the buffer with fbb.GetBufferPointer()
, and it's size from fbb.GetSize()
.
samples/sample_binary.cpp
is a complete code sample similar to the code above, that also includes the reading code below.
If you've received a buffer from somewhere (disk, network, etc.) you can directly start traversing it using:
auto monster = GetMonster(buffer_pointer);
monster
is of type Monster *
, and points to somewhere inside your buffer. If you look in your generated header, you'll see it has convenient accessors for all fields, e.g.
assert(monster->hp() == 80); assert(monster->mana() == 150); // default assert(strcmp(monster->name()->c_str(), "MyMonster") == 0);
These should all be true. Note that we never stored a mana
value, so it will return the default.
To access sub-objects, in this case the Vec3
:
auto pos = monster->pos(); assert(pos); assert(pos->z() == 3);
If we had not set the pos
field during serialization, it would be NULL
.
Similarly, we can access elements of the inventory array:
auto inv = monster->inventory(); assert(inv); assert(inv->Get(9) == 9);
As you can see from the above examples, all elements in a buffer are accessed through generated accessors. This is because everything is stored in little endian format on all platforms (the accessor performs a swap operation on big endian machines), and also because the layout of things is generally not known to the user.
For structs, layout is deterministic and guaranteed to be the same accross platforms (scalars are aligned to their own size, and structs themselves to their largest member), and you are allowed to access this memory directly by using sizeof()
and memcpy
on the pointer to a struct, or even an array of structs.
To compute offsets to sub-elements of a struct, make sure they are a structs themselves, as then you can use the pointers to figure out the offset without having to hardcode it. This is handy for use of arrays of structs with calls like glVertexAttribPointer
in OpenGL or similar APIs.
It is important to note is that structs are still little endian on all machines, so only use tricks like this if you can guarantee you're not shipping on a big endian machine (an assert(FLATBUFFERS_LITTLEENDIAN)
would be wise).
Using binary buffers with the generated header provides a super low overhead use of FlatBuffer data. There are, however, times when you want to use text formats, for example because it interacts better with source control, or you want to give your users easy access to data.
Another reason might be that you already have a lot of data in JSON format, or a tool that generates JSON, and if you can write a schema for it, this will provide you an easy way to use that data directly.
There are two ways to use text formats:
This is the preferred path, as it doesn't require you to add any new code to your program, and is maximally efficient since you can ship with binary data. The disadvantage is that it is an extra step for your users/developers to perform, though you might be able to automate it.
flatc -b myschema.fbs mydata.json
This will generate the binary file mydata_wire.bin
which can be loaded as before.
This gives you maximum flexibility. You could even opt to support both, i.e. check for both files, and regenerate the binary from text when required, otherwise just load the binary.
This option is currently only available for C++, or Java through JNI.
As mentioned in the section "Building" above, this technique requires you to link a few more files into your program, and you'll want to include flatbuffers/idl.h
.
Load text (either a schema or json) into an in-memory buffer (there is a convenient LoadFile()
utility function in flatbuffers/util.h
if you wish). Construct a parser:
flatbuffers::Parser parser;
Now you can parse any number of text files in sequence:
parser.Parse(text_file.c_str());
This works similarly to how the command-line compiler works: a sequence of files parsed by the same Parser
object allow later files to reference definitions in earlier files. Typically this means you first load a schema file (which populates Parser
with definitions), followed by one or more JSON files.
If there were any parsing errors, Parse
will return false
, and Parser::err
contains a human readable error string with a line number etc, which you should present to the creator of that file.
After each JSON file, the Parser::fbb
member variable is the FlatBufferBuilder
that contains the binary buffer version of that file, that you can access as described above.
samples/sample_text.cpp
is a code sample showing the above operations.
None of the code is thread-safe, by design. That said, since currently a FlatBuffer is read-only and entirely const
, reading by multiple threads is possible.


File.Copy renames a file or directory named by the srcPathName parameter to the destName parameter. The destName parameter must be a name only. In other words File.Rename can't move a file between different directories.
Details
If the File.Rename call fails, then Error.Last will return a non-zero value indicating the reason for the failure. Error.LastMsg will return a string which contains the textual version of the error.
Example
This program renames the file "/usr/west/example" to "testcase"
File.Rename ("/usr/west/example", "testcase")
if Error.Last = eNoError then
put "File renamed"
else
put "Did not rename the file."
put "Error: ", Error.LastMsg
end if
Execute
An example program is available that creates a file in the current
directory, renames it to a different name using File.Rename,
checks the existence of both files using File.Exists and then
deletes the file using File.Delete.
Status
Exported qualified.
This means that you can only call the function by calling File.Rename, not by calling Rename.


The curve is constructed as follows:
On the segments P0P1, P1P2 and P2P3 the points Q0,Q1 and Q2 are drawn such that P0Q0/P0P1=P1Q1/P1P2=P2Q2/P2P3=t (t in [0,1]).
On the segments Q0Q1 and Q1Q2 the points R0 and R1 are drawn such that
Q0R0/Q0Q1=Q1R1/Q1Q2=t for the same value of t.
On the segment R0R1 the point B is drawn such that R0B/R0R1=t for the same value of t.
The Bézier curve defined by the points P0, P1, P2, P3 is the locus of B as Q0 takes all possible positions on the segment P0P1. (Please note that for all points the value of t is the same.)
In the applet to the right you can drag the points P0, P1, P2 and P3 to see what the Bézier curve (green curve) defined by those points looks like. You can also drag the point Q0 along the segment P0P1.
From the construction it is clear that the Bézier curve will be tangent to the segments P0P1 in P0 and P2P3 in P3.
A cubic Bézier curve with P0=(1,0), P1=(1,v), P2=(v,1) and P3=(0,1) is used to approximate a quarter circle.
The value v0 is chosen such that the area enclosed by the lines OP0, OP3 and the curve is equal to π/4 (the area of the quarter circle).
By how many percent does the length of the curve differ from the length of the quarter circle?
That is, if L is the length of the curve, calculate 100*(L-π/2)/(π/2).
Give your answer rounded to 10 digits behind the decimal point.


--- layout: post title: A short guide on unit testing Angular JS Controllers date: 2014-06-01 10:32:25.000000000 +05:00 tags: - AngularJS status: publish type: post published: true meta: geo_public: '0' _edit_last: '65775760' _publicize_pending: '1' author: login: mohammadumairkh email<EMAIL_ADDRESS>display_name: mohammadumairkh first_name: 'Mohammad Umair' last_name: 'Khan' ---
Unit testing becomes a challenging task when one is not used to doing it. I faced a similar scenario writing my first application in Angular JS. I took more 3x more time to write unit tests than writing the actual controllers, services etc. I was not sure what and how to unit test. In this post I will go through of what I think is a comprehensive unit test for a controller in Angular JS and for that I will be using Jasmine as my testing framework.
** One thing I believe everyone should concentrate on is that it is a “unit” test and everything else this “unit” is dependent upon must be mocked out. (e.g. services, APIs, route resolves etc)
Controllers in Angular JS are only responsible to interacting with the view via the scope and almost any other operation is done via the Angular Services. So, the controllers in my application (and probably yours too) will contain:
So a typical controller would look like this:
{% gist c0dd8534075e809387b6 %}
Lets go ahead and analyze what unit tests would completely test this controller …
I often do a little exercise to visualize the controller as a tree and chalk out all the possible unit tests. For example, unit tests for the above controller would be something like this…
Map for unit testing an Angular Controller[/caption]
Once our analysis of unit tests is complete we will then set up the controller’s test file and it looks like this …
** Setting up the test environment is out of the scope of this document
{% gist c1acd6536f37af45f970 %}
In the above code:
let’s go ahead and start writing unit tests for each node in the above tree from left to right..
Testing initialize values and types
In the following unit tests, I just made sure that all the initialize values and needed functions are present in the controller.
{% gist 5cf595eaa3c0b5597dd0 %}
Pretty simple!
Testing onDeleteAdmin function
The following unit tests, will test the onDeleteAdmin function with all its possibilities.
{% gist 805554ea52614926f355 %}
*** Note, since we are resolving and rejecting promises outside the Angular cycle, we need to let Angular know when we resolved or rejected a promise. It is accomplished via scope.$apply() method.
Testing onViewPortals function
The following unit tests will test the onViewPortals function with all its possibilities
{% gist 6336c48e757ee5e052b9 %}
With this, we complete unit testing our sample controller. You can find the source for the controller and its unit tests on the following links:
Controller is here
Unit test is here
Conclusion
Controllers if written with the aim of just to interact with the view are pretty simple to test. Apart from how to test controllers, I believe it is important to create either a mind-map or a small diagram which outlines all the possibilities for the tests. This technique applies to all tests, irrespective of language or framework and helps one stay focused. I recommend that the readers should at least give it a try.
If you have any questions, comments please feel free to ask.


This Website is an on-going development process. Thanks to DBC. Contents will change from time to time as I get more ideas, suggestions, and experiences. The main purpose of this site is to practice and solidify various concepts as I go through my program at DBC. Navigate to My Blog posts to get an idea of some the concepts I'm learning. Feel free to connect with me on social media to leave comments and suggestions.
Now go forth, and be awesome!


--- title: Cluster demo sidebar: documentation_sidebar permalink: ClusterDemo.html folder: documentation toc: false ---
This code demonstrates how the clustering algorithm can be used to assign vector3d and crystal directions to groups by proximity.
define an octohedral crystal symmetry
{% highlight matlab %} cs = crystalSymmetry('432'); % define an ODF with two radial peaks ori = orientation.byEuler([10 40]*degree,[30 50]*degree,[50 70]*degree,cs) odf = unimodalODF(ori,'halfwidth',5*degree); % view the odf plotPDF(odf,Miller(1,0,0,odf.CS),'contour','linewidth',2); % generate 10k orientations from this randomly defined ODF function ori = odf.discreteSample(10000); % convert the orientations to vector3D r = ori * Miller(1,0,0,odf.CS); {% endhighlight %} {% highlight plaintext %} ori = orientation (432 → xyz) size: 1 x 2 Bunge Euler angles in degree phi1 Phi phi2 10 30 50 40 50 70 {% endhighlight %}assign each vector3d to one of twelve clusters, and calculate the vector3D located at the center of each cluster
{% highlight matlab %} [cId,center] = calcCluster(r,'numCluster',12); % plot the clusters, sorted by colour figure; plot(r,ind2color(cId)) % annotate all the cluster centers, on all figures. annotate(center,'add2all'); {% endhighlight %}Note that the upper and lower hemisphere plots are versions of each other, reflected horizontally plus vertically. This means that the underlying data has antipodal symmetry, contributing equally to both hemispheres. Let's include that in the cluster sorting.
repeat the calculation after changing all the vector3d to be antipodal
{% highlight matlab %} r.antipodal = true; % repeat the calculation assigning vector3D to clusters. Due to the % increase in symmetry, there are only six clusters now. [cId,center] = calcCluster(r,'numCluster',6); % plot the vectors. Note that we no longer get an upper and lower hemisphere plot; the antipodal symmetry tells MTEX they are equivilent and so one sufficient to represent the data. figure;plot(r,ind2color(cId)) % annotate the cluster centers. annotate(center,'add2all') {% endhighlight %}pick a vector3d, and use that to convert the 10k random orientations previously generated into crystal directions.
{% highlight matlab %} h = ori \ vector3d(1,1,0); % assign the crystal directions to two clusters [cId,center] = calcCluster(h,'numCluster',2); % plot the crystal symmetry data on appropiate fundamental sector plot(h.project2FundamentalRegion,ind2color(cId),'fundamentalSector') % annote the cluster centers annotate(center,'add2all') {% endhighlight %}just as we calculated clusters for vector3D and crystal directions, we're now going to do so for orientations
{% highlight matlab %} [cId,center] = calcCluster(ori,'numCluster',2,'method','hierarchical'); % create a pole figure of the orientations coloured by the cluster they % belong to. plotPDF(ori,ind2color(cId),Miller(1,0,0,cs),'all') {% endhighlight %}If you have the statistics toolbox, you can make some calculations about the spread of points assigned to each cluster.
{% highlight matlab %} % compute the full distance matrix between all combinations of vector3D d = angle_outer(r,r); % convert all small values to zero to simplify later calculations d(d<0.01) = 0; %d = d(triu(true(size(d)),1)); % use the statistic toolbox try d = squareform(d); z = linkage(d,'ward'); %cId = cluster(z,'cutoff',30*degree); cId = cluster(z,'maxclust',6); plotCluster(r,cId) catch warning('Statistics Toolbox not installed!') end {% endhighlight %} {% highlight plaintext %} Warning: ward's linkage specified with non-Euclidean dissimilarity matrix. {% endhighlight %} {% highlight matlab %} function plotCluster(r,cId,varargin) scatter(r(cId==1),'MarkerFaceColor',ind2color(1),varargin{:}) hold on for i = 2:max(cId) scatter(r(cId==i),'add2all','MarkerFaceColor',ind2color(i),varargin{:}) end hold off end {% endhighlight %}


Home
/
Job Search
/
Run Name
Job Id
Job Status
ALL
COMPLETED
RUNNING
Active In Last
Search
Job Id
Run Name
Start Time
End Time
Status
Remarks
Delete Jobs
7
10
20
30
40


Preparation time
less than 30 mins
Cooking time
over 2 hours
Serves
Serves 6-8
Preheat the oven to 180C/350F/Gas 4. For the roast leg of lamb, place the leg of lamb on a clean work surface or chopping board.
With a long, thin, sharp knife, make lots of incisions all over the meat.
Insert the strips of ham and rosemary together deep into the incisions leaving a little out.
Rub the olive oil all over the lamb and season with salt and pepper, rubbing well in.
In a bowl, mix together the parmesan or pecorino and breadcrumbs.
Carefully sprinkle over the top and side of the lamb and press well with your hands so the meat is well covered and the crumbs don't fall off - if some do, just gather them up and place onto the meat.
Carefully place the meat in a roasting tin which will just fit the lamb. Pour in the wine and water on the side of the tin (not on the meat).
Cover with foil and place in the preheated oven for about two and a half hours, which will give you a medium- to well-cooked lamb. Cook for less time if you prefer your lamb pink.
During cooking, check from time to time the liquid content - if necessary add more water. For the last 10 minutes of cooking time, remove the foil and increase the heat to 200C/390F/Gas 6 and roast until the topping goes golden and crispy, but be careful not to burn.
Remove from the oven, slice and serve. Don't forget to pour the cooking juices over the lamb slices if you desire. Serve with small roast Jersey potatoes with the skins on, and the peas with pancetta.
For the fresh peas with onion and pancetta, heat the olive oil in a saucepan, add the onion and pancetta and sauté.
Stir in the peas and stir fry for a couple of minutes. Add the stock, bring to the boil, then reduce the heat, cover and simmer for approximately 20 minutes, stirring now and again.
After this time, the stock should almost have evaporated.
Serve immediately or make in advance and re-heat when required.


source file: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/concurrent/futures/_base.py
file stats: 321 lines, 95 executed: 29.6% covered
1. # Copyright 2009 Brian Quinlan. All Rights Reserved. 2. # Licensed to PSF under a Contributor Agreement. 3. 4. import collections 5. import logging 6. import threading 7. import itertools 8. import time 9. 10. __author__ = 'Brian Quinlan (brian@sweetapp.com)' 11. 12. FIRST_COMPLETED = 'FIRST_COMPLETED' 13. FIRST_EXCEPTION = 'FIRST_EXCEPTION' 14. ALL_COMPLETED = 'ALL_COMPLETED' 15. _AS_COMPLETED = '_AS_COMPLETED' 16. 17. # Possible future states (for internal use by the futures package). 18. PENDING = 'PENDING' 19. RUNNING = 'RUNNING' 20. # The future was cancelled by the user... 21. CANCELLED = 'CANCELLED' 22. # ...and _Waiter.add_cancelled() was called by a worker. 23. CANCELLED_AND_NOTIFIED = 'CANCELLED_AND_NOTIFIED' 24. FINISHED = 'FINISHED' 25. 26. _FUTURE_STATES = [ 27. PENDING, 28. RUNNING, 29. CANCELLED, 30. CANCELLED_AND_NOTIFIED, 31. FINISHED 32. ] 33. 34. _STATE_TO_DESCRIPTION_MAP = { 35. PENDING: "pending", 36. RUNNING: "running", 37. CANCELLED: "cancelled", 38. CANCELLED_AND_NOTIFIED: "cancelled", 39. FINISHED: "finished" 40. } 41. 42. # Logger for internal use by the futures package. 43. LOGGER = logging.getLogger("concurrent.futures") 44. 45. class Error(Exception): 46. """Base class for all future-related exceptions.""" 47. pass 48. 49. class CancelledError(Error): 50. """The Future was cancelled.""" 51. pass 52. 53. class TimeoutError(Error): 54. """The operation exceeded the given deadline.""" 55. pass 56. 57. class _Waiter(object): 58. """Provides the event that wait() and as_completed() block on.""" 59. def __init__(self): 60. self.event = threading.Event() 61. self.finished_futures = [] 62. 63. def add_result(self, future): 64. self.finished_futures.append(future) 65. 66. def add_exception(self, future): 67. self.finished_futures.append(future) 68. 69. def add_cancelled(self, future): 70. self.finished_futures.append(future) 71. 72. class _AsCompletedWaiter(_Waiter): 73. """Used by as_completed().""" 74. 75. def __init__(self): 76. super(_AsCompletedWaiter, self).__init__() 77. self.lock = threading.Lock() 78. 79. def add_result(self, future): 80. with self.lock: 81. super(_AsCompletedWaiter, self).add_result(future) 82. self.event.set() 83. 84. def add_exception(self, future): 85. with self.lock: 86. super(_AsCompletedWaiter, self).add_exception(future) 87. self.event.set() 88. 89. def add_cancelled(self, future): 90. with self.lock: 91. super(_AsCompletedWaiter, self).add_cancelled(future) 92. self.event.set() 93. 94. class _FirstCompletedWaiter(_Waiter): 95. """Used by wait(return_when=FIRST_COMPLETED).""" 96. 97. def add_result(self, future): 98. super(_FirstCompletedWaiter, self).add_result(future) 99. self.event.set() 100. 101. def add_exception(self, future): 102. super(_FirstCompletedWaiter, self).add_exception(future) 103. self.event.set() 104. 105. def add_cancelled(self, future): 106. super(_FirstCompletedWaiter, self).add_cancelled(future) 107. self.event.set() 108. 109. class _AllCompletedWaiter(_Waiter): 110. """Used by wait(return_when=FIRST_EXCEPTION and ALL_COMPLETED).""" 111. 112. def __init__(self, num_pending_calls, stop_on_exception): 113. self.num_pending_calls = num_pending_calls 114. self.stop_on_exception = stop_on_exception 115. self.lock = threading.Lock() 116. super(_AllCompletedWaiter, self).__init__() 117. 118. def _decrement_pending_calls(self): 119. with self.lock: 120. self.num_pending_calls -= 1 121. if not self.num_pending_calls: 122. self.event.set() 123. 124. def add_result(self, future): 125. super(_AllCompletedWaiter, self).add_result(future) 126. self._decrement_pending_calls() 127. 128. def add_exception(self, future): 129. super(_AllCompletedWaiter, self).add_exception(future) 130. if self.stop_on_exception: 131. self.event.set() 132. else: 133. self._decrement_pending_calls() 134. 135. def add_cancelled(self, future): 136. super(_AllCompletedWaiter, self).add_cancelled(future) 137. self._decrement_pending_calls() 138. 139. class _AcquireFutures(object): 140. """A context manager that does an ordered acquire of Future conditions.""" 141. 142. def __init__(self, futures): 143. self.futures = sorted(futures, key=id) 144. 145. def __enter__(self): 146. for future in self.futures: 147. future._condition.acquire() 148. 149. def __exit__(self, *args): 150. for future in self.futures: 151. future._condition.release() 152. 153. def _create_and_install_waiters(fs, return_when): 154. if return_when == _AS_COMPLETED: 155. waiter = _AsCompletedWaiter() 156. elif return_when == FIRST_COMPLETED: 157. waiter = _FirstCompletedWaiter() 158. else: 159. pending_count = sum( 160. f._state not in [CANCELLED_AND_NOTIFIED, FINISHED] for f in fs) 161. 162. if return_when == FIRST_EXCEPTION: 163. waiter = _AllCompletedWaiter(pending_count, stop_on_exception=True) 164. elif return_when == ALL_COMPLETED: 165. waiter = _AllCompletedWaiter(pending_count, stop_on_exception=False) 166. else: 167. raise ValueError("Invalid return condition: %r" % return_when) 168. 169. for f in fs: 170. f._waiters.append(waiter) 171. 172. return waiter 173. 174. def as_completed(fs, timeout=None): 175. """An iterator over the given futures that yields each as it completes. 176. 177. Args: 178. fs: The sequence of Futures (possibly created by different Executors) to 179. iterate over. 180. timeout: The maximum number of seconds to wait. If None, then there 181. is no limit on the wait time. 182. 183. Returns: 184. An iterator that yields the given Futures as they complete (finished or 185. cancelled). If any given Futures are duplicated, they will be returned 186. once. 187. 188. Raises: 189. TimeoutError: If the entire result iterator could not be generated 190. before the given timeout. 191. """ 192. if timeout is not None: 193. end_time = timeout + time.time() 194. 195. fs = set(fs) 196. with _AcquireFutures(fs): 197. finished = set( 198. f for f in fs 199. if f._state in [CANCELLED_AND_NOTIFIED, FINISHED]) 200. pending = fs - finished 201. waiter = _create_and_install_waiters(fs, _AS_COMPLETED) 202. 203. try: 204. for future in finished: 205. yield future 206. 207. while pending: 208. if timeout is None: 209. wait_timeout = None 210. else: 211. wait_timeout = end_time - time.time() 212. if wait_timeout < 0: 213. raise TimeoutError( 214. '%d (of %d) futures unfinished' % ( 215. len(pending), len(fs))) 216. 217. waiter.event.wait(wait_timeout) 218. 219. with waiter.lock: 220. finished = waiter.finished_futures 221. waiter.finished_futures = [] 222. waiter.event.clear() 223. 224. for future in finished: 225. yield future 226. pending.remove(future) 227. 228. finally: 229. for f in fs: 230. with f._condition: 231. f._waiters.remove(waiter) 232. 233. DoneAndNotDoneFutures = collections.namedtuple( 234. 'DoneAndNotDoneFutures', 'done not_done') 235. def wait(fs, timeout=None, return_when=ALL_COMPLETED): 236. """Wait for the futures in the given sequence to complete. 237. 238. Args: 239. fs: The sequence of Futures (possibly created by different Executors) to 240. wait upon. 241. timeout: The maximum number of seconds to wait. If None, then there 242. is no limit on the wait time. 243. return_when: Indicates when this function should return. The options 244. are: 245. 246. FIRST_COMPLETED - Return when any future finishes or is 247. cancelled. 248. FIRST_EXCEPTION - Return when any future finishes by raising an 249. exception. If no future raises an exception 250. then it is equivalent to ALL_COMPLETED. 251. ALL_COMPLETED - Return when all futures finish or are cancelled. 252. 253. Returns: 254. A named 2-tuple of sets. The first set, named 'done', contains the 255. futures that completed (is finished or cancelled) before the wait 256. completed. The second set, named 'not_done', contains uncompleted 257. futures. 258. """ 259. with _AcquireFutures(fs): 260. done = set(f for f in fs 261. if f._state in [CANCELLED_AND_NOTIFIED, FINISHED]) 262. not_done = set(fs) - done 263. 264. if (return_when == FIRST_COMPLETED) and done: 265. return DoneAndNotDoneFutures(done, not_done) 266. elif (return_when == FIRST_EXCEPTION) and done: 267. if any(f for f in done 268. if not f.cancelled() and f.exception() is not None): 269. return DoneAndNotDoneFutures(done, not_done) 270. 271. if len(done) == len(fs): 272. return DoneAndNotDoneFutures(done, not_done) 273. 274. waiter = _create_and_install_waiters(fs, return_when) 275. 276. waiter.event.wait(timeout) 277. for f in fs: 278. with f._condition: 279. f._waiters.remove(waiter) 280. 281. done.update(waiter.finished_futures) 282. return DoneAndNotDoneFutures(done, set(fs) - done) 283. 284. class Future(object): 285. """Represents the result of an asynchronous computation.""" 286. 287. def __init__(self): 288. """Initializes the future. Should not be called by clients.""" 289. self._condition = threading.Condition() 290. self._state = PENDING 291. self._result = None 292. self._exception = None 293. self._traceback = None 294. self._waiters = [] 295. self._done_callbacks = [] 296. 297. def _invoke_callbacks(self): 298. for callback in self._done_callbacks: 299. try: 300. callback(self) 301. except Exception: 302. LOGGER.exception('exception calling callback for %r', self) 303. 304. def __repr__(self): 305. with self._condition: 306. if self._state == FINISHED: 307. if self._exception: 308. return '<Future at %s state=%s raised %s>' % ( 309. hex(id(self)), 310. _STATE_TO_DESCRIPTION_MAP[self._state], 311. self._exception.__class__.__name__) 312. else: 313. return '<Future at %s state=%s returned %s>' % ( 314. hex(id(self)), 315. _STATE_TO_DESCRIPTION_MAP[self._state], 316. self._result.__class__.__name__) 317. return '<Future at %s state=%s>' % ( 318. hex(id(self)), 319. _STATE_TO_DESCRIPTION_MAP[self._state]) 320. 321. def cancel(self): 322. """Cancel the future if possible. 323. 324. Returns True if the future was cancelled, False otherwise. A future 325. cannot be cancelled if it is running or has already completed. 326. """ 327. with self._condition: 328. if self._state in [RUNNING, FINISHED]: 329. return False 330. 331. if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: 332. return True 333. 334. self._state = CANCELLED 335. self._condition.notify_all() 336. 337. self._invoke_callbacks() 338. return True 339. 340. def cancelled(self): 341. """Return True if the future has cancelled.""" 342. with self._condition: 343. return self._state in [CANCELLED, CANCELLED_AND_NOTIFIED] 344. 345. def running(self): 346. """Return True if the future is currently executing.""" 347. with self._condition: 348. return self._state == RUNNING 349. 350. def done(self): 351. """Return True of the future was cancelled or finished executing.""" 352. with self._condition: 353. return self._state in [CANCELLED, CANCELLED_AND_NOTIFIED, FINISHED] 354. 355. def __get_result(self): 356. if self._exception: 357. raise type(self._exception), self._exception, self._traceback 358. else: 359. return self._result 360. 361. def add_done_callback(self, fn): 362. """Attaches a callable that will be called when the future finishes. 363. 364. Args: 365. fn: A callable that will be called with this future as its only 366. argument when the future completes or is cancelled. The callable 367. will always be called by a thread in the same process in which 368. it was added. If the future has already completed or been 369. cancelled then the callable will be called immediately. These 370. callables are called in the order that they were added. 371. """ 372. with self._condition: 373. if self._state not in [CANCELLED, CANCELLED_AND_NOTIFIED, FINISHED]: 374. self._done_callbacks.append(fn) 375. return 376. fn(self) 377. 378. def result(self, timeout=None): 379. """Return the result of the call that the future represents. 380. 381. Args: 382. timeout: The number of seconds to wait for the result if the future 383. isn't done. If None, then there is no limit on the wait time. 384. 385. Returns: 386. The result of the call that the future represents. 387. 388. Raises: 389. CancelledError: If the future was cancelled. 390. TimeoutError: If the future didn't finish executing before the given 391. timeout. 392. Exception: If the call raised then that exception will be raised. 393. """ 394. with self._condition: 395. if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: 396. raise CancelledError() 397. elif self._state == FINISHED: 398. return self.__get_result() 399. 400. self._condition.wait(timeout) 401. 402. if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: 403. raise CancelledError() 404. elif self._state == FINISHED: 405. return self.__get_result() 406. else: 407. raise TimeoutError() 408. 409. def exception_info(self, timeout=None): 410. """Return a tuple of (exception, traceback) raised by the call that the 411. future represents. 412. 413. Args: 414. timeout: The number of seconds to wait for the exception if the 415. future isn't done. If None, then there is no limit on the wait 416. time. 417. 418. Returns: 419. The exception raised by the call that the future represents or None 420. if the call completed without raising. 421. 422. Raises: 423. CancelledError: If the future was cancelled. 424. TimeoutError: If the future didn't finish executing before the given 425. timeout. 426. """ 427. with self._condition: 428. if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: 429. raise CancelledError() 430. elif self._state == FINISHED: 431. return self._exception, self._traceback 432. 433. self._condition.wait(timeout) 434. 435. if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: 436. raise CancelledError() 437. elif self._state == FINISHED: 438. return self._exception, self._traceback 439. else: 440. raise TimeoutError() 441. 442. def exception(self, timeout=None): 443. """Return the exception raised by the call that the future represents. 444. 445. Args: 446. timeout: The number of seconds to wait for the exception if the 447. future isn't done. If None, then there is no limit on the wait 448. time. 449. 450. Returns: 451. The exception raised by the call that the future represents or None 452. if the call completed without raising. 453. 454. Raises: 455. CancelledError: If the future was cancelled. 456. TimeoutError: If the future didn't finish executing before the given 457. timeout. 458. """ 459. return self.exception_info(timeout)[0] 460. 461. # The following methods should only be used by Executors and in tests. 462. def set_running_or_notify_cancel(self): 463. """Mark the future as running or process any cancel notifications. 464. 465. Should only be used by Executor implementations and unit tests. 466. 467. If the future has been cancelled (cancel() was called and returned 468. True) then any threads waiting on the future completing (though calls 469. to as_completed() or wait()) are notified and False is returned. 470. 471. If the future was not cancelled then it is put in the running state 472. (future calls to running() will return True) and True is returned. 473. 474. This method should be called by Executor implementations before 475. executing the work associated with this future. If this method returns 476. False then the work should not be executed. 477. 478. Returns: 479. False if the Future was cancelled, True otherwise. 480. 481. Raises: 482. RuntimeError: if this method was already called or if set_result() 483. or set_exception() was called. 484. """ 485. with self._condition: 486. if self._state == CANCELLED: 487. self._state = CANCELLED_AND_NOTIFIED 488. for waiter in self._waiters: 489. waiter.add_cancelled(self) 490. # self._condition.notify_all() is not necessary because 491. # self.cancel() triggers a notification. 492. return False 493. elif self._state == PENDING: 494. self._state = RUNNING 495. return True 496. else: 497. LOGGER.critical('Future %s in unexpected state: %s', 498. id(self), 499. self._state) 500. raise RuntimeError('Future in unexpected state') 501. 502. def set_result(self, result): 503. """Sets the return value of work associated with the future. 504. 505. Should only be used by Executor implementations and unit tests. 506. """ 507. with self._condition: 508. self._result = result 509. self._state = FINISHED 510. for waiter in self._waiters: 511. waiter.add_result(self) 512. self._condition.notify_all() 513. self._invoke_callbacks() 514. 515. def set_exception_info(self, exception, traceback): 516. """Sets the result of the future as being the given exception 517. and traceback. 518. 519. Should only be used by Executor implementations and unit tests. 520. """ 521. with self._condition: 522. self._exception = exception 523. self._traceback = traceback 524. self._state = FINISHED 525. for waiter in self._waiters: 526. waiter.add_exception(self) 527. self._condition.notify_all() 528. self._invoke_callbacks() 529. 530. def set_exception(self, exception): 531. """Sets the result of the future as being the given exception. 532. 533. Should only be used by Executor implementations and unit tests. 534. """ 535. self.set_exception_info(exception, None) 536. 537. class Executor(object): 538. """This is an abstract base class for concrete asynchronous executors.""" 539. 540. def submit(self, fn, *args, **kwargs): 541. """Submits a callable to be executed with the given arguments. 542. 543. Schedules the callable to be executed as fn(*args, **kwargs) and returns 544. a Future instance representing the execution of the callable. 545. 546. Returns: 547. A Future representing the given call. 548. """ 549. raise NotImplementedError() 550. 551. def map(self, fn, *iterables, **kwargs): 552. """Returns a iterator equivalent to map(fn, iter). 553. 554. Args: 555. fn: A callable that will take as many arguments as there are 556. passed iterables. 557. timeout: The maximum number of seconds to wait. If None, then there 558. is no limit on the wait time. 559. 560. Returns: 561. An iterator equivalent to: map(func, *iterables) but the calls may 562. be evaluated out-of-order. 563. 564. Raises: 565. TimeoutError: If the entire result iterator could not be generated 566. before the given timeout. 567. Exception: If fn(*args) raises for any values. 568. """ 569. timeout = kwargs.get('timeout') 570. if timeout is not None: 571. end_time = timeout + time.time() 572. 573. fs = [self.submit(fn, *args) for args in itertools.izip(*iterables)] 574. 575. # Yield must be hidden in closure so that the futures are submitted 576. # before the first iterator value is required. 577. def result_iterator(): 578. try: 579. for future in fs: 580. if timeout is None: 581. yield future.result() 582. else: 583. yield future.result(end_time - time.time()) 584. finally: 585. for future in fs: 586. future.cancel() 587. return result_iterator() 588. 589. def shutdown(self, wait=True): 590. """Clean-up the resources associated with the Executor. 591. 592. It is safe to call this method several times. Otherwise, no other 593. methods can be called after this one. 594. 595. Args: 596. wait: If True then shutdown will not return until all running 597. futures have finished executing and the resources used by the 598. executor have been reclaimed. 599. """ 600. pass 601. 602. def __enter__(self): 603. return self 604. 605. def __exit__(self, exc_type, exc_val, exc_tb): 606. self.shutdown(wait=True) 607. return False


|
Noise++
|
#include <NoiseOutStream.h>
Public Member Functions | |
| MemoryOutStream () | |
| void | clear () |
| char * | getBuffer () |
| size_t | getBufferSize () const |
| template<class T > | |
| void | write (T t) |
| virtual | ~MemoryOutStream () |
| virtual void | write (const void *buffer, size_t len) |
| virtual size_t | tell () |
| virtual void | seek (size_t pos) |
| void | writeInt (int t) |
| void | writeDouble (double t) |
|
virtual |
Jumps to the specified position.
Implements noisepp::utils::OutStream.
|
virtual |
Returns the current position.
Implements noisepp::utils::OutStream.
|
inline |
Write len bytes from buffer.
|
virtual |
Write len bytes from buffer.
Implements noisepp::utils::OutStream.


Status
So, the great thing is, a large portion of the process is no longer a mystery. Developing this website is no longer a mystery of services and hooking together interfaces and running random programs--it's about code! Any app I could want to create, I have the infrastructure to create it in a combination of back-end Python and front-end Javascript. I can't stress enough how great that is. Front-end development can be more about programming than about installing programs (which, if you've been following along--it's mostly been installing programs so far.)
The one missing link is socket communication. I can make a multiplayer game, if it's turn-based. I can even make a chatroom, maybe by polling for new messages repeatedly. But for a fast multiplayer game, or anything requiring real-time updates--I need to figure out sockets. I'm curious about socket.io for now, but either Gunicorn or Werkzeug or Flask or whatever should also have some sort of back-end support? I don't know how it works. I think both the back and the front end need some kind of framework for handling sockets.
But then there are other fun things I can do now with these newfound powers. I think it should be very straightforward to make a simple multiplayer game now, like a card game. I can put my security knowledge to the test and try building out a custom authentication system. I can even focus on the comment system, make it nicer, add features, fix bugs, and build the next Disqus. Or, I could focus on workflow--improve the amateurish CMS I have right now, start using/building modules, switch to Typescript, write some documentation--things that service bigger, more complex projects.
My inclination is to stay away from bigger complex projects for now, because then I will move away from "learning web dev" and enter "spend every day fixing bugs in a massive programming project", and every post every day will be "I continued working on my game." So I think the natural next step is to figure out sockets. Today's been yet another hectic day, so we'll make it a reading/information gathering day.
First point of investigation: what are web sockets? Web sockets are communication channels over a TCP connection. Whereas HTML request are one-way (one side sends a request, the other sends a response), sockets facilitate two-way communication.
Socket.io is... I defy you to read the Socket.io website and find an actual description of what it is. "Socket.IO enables real-time bidirectional event-based communication." That's what I just said WebSockets in general does. Does it require node? Does it run on the server or on the client? These questions might be obvious, but you'd think I could figure it out from the site.
Google informs me Socket.IO has a client-side and a server-side library. The server-side library uses node.js. I'm wondering to myself--do I need to use this at all? Why is Socket.IO the go-to library whenever people discuss WebSockets? Is it just a sign that node.js has taken over the internet?
Python has a websockets library. Requires Python 3.4. That's a little scary. It may be unnecessary.
WebSockets seem easy enough to use on the client-side with vanilla JS. Hard to imagine why anyone would want to use Socket.IO, it's that easy. Python also has a socket module, distinct from the websockets module. Seems slightly harder to use. This tutorial seems useful, though. We'll try following through it tomorrow.
Comment:


Douglas Boyce
About
Portfolio
Contact
Portfolio
Word Guess
RPG Game
Trivia Game
Rutger's Info Widget
Rock Paper Scissors


Warning: You are heading into developer territory!
Our APIs are designed for programmers. If our APIs were a ski trail, they would be a double black diamond with plenty of moguls and littered with trees. Scary trees.
Only move forward if you are comfortable working with APIs, or have a nearby friend that is.


Hi there! Please Sign in
Login
Forgot password?
Forgot Password?
Lorem ipsum dolor fringilla enim feugiat commodo sed ac lacus.
Reset password
Back to login


Argos is selling half Christmas trees to keep naughty pets at bay
It comes in two different designs
by Entertainment DailyArgos 'half' Christmas trees are an absolute must for anyone with a naughty cat or dog who loves nothing more than to play with your brand new festive tree as soon as you get it through the front door.
The high street chain's genius faux trees are on sale for less than £35, and you can choose from two designs.
The trees are 6ft but the branches only start at 93cm, which means baubles, tinsel and fairy lights will be well out of the reach of pets and curious toddlers.
The stand is included and the product weighs 6kg in total.
Read more: Sprouts may have to be imported this Christmas as British shoppers 'face shortage'
Two designs are available, including one that looks like it's been sprinkled with snow and a traditional green one.
Oh, and you're in luck because they've both just been reduced, with the white tree now costing £33.75 (it was £45) and the green tree is now priced at £30.00.
They've gone down a treat with Argos customers who have left more than 200 reviews raving about the festive product.
Read more: Marks & Spencer releases cheese advent calendar – and it's on sale now for just £15!
One commented: "It is perfect! Good quality, easy to put up and the dog and cats can't ruin it. Doesn't take up much room which is a huge bonus plus it looks amazing. My husband was sceptical and thought it was pointless but has said "it was worth every penny".
A second said it's been brilliant at keeping their pet dogs at bay.
"We are really pleased with this tree. Didn't expect it to be as good. It looks great and so far both of our dogs have left it alone!"
A third commented: "Gorgeous tree. Looks like an expensive designer one. Definitely buying another next year.. Don’t hesitate- buy it."
If you're sold, you can snap up one here.
Head to our Facebook page @EntertainmentDailyFix and let us know what you think of our story.


Midterm Examination
The midterm examination was scheduled as follows:
- Date: Friday, September 28, 2018.
- Time: 8:30 am - 10:00 am
- Venue: LH-1, Department of Mathematics, IISc
Distribution of scores in the midterm
Answer scripts, queries etc.
If you have not seen your answer script or want to meet me (Siddhartha Gadgil) about any related issue, I will be available on Tuesday, October 9, 2019
between 8:30 am and 11:20 am. in my office N-15, Department of Mathematics.
Syllabus etc
Please note the rules for the examination.
The syllabus is the material covered in the lectures up to Wednesday, September 19, 2018.
Topics for the midterm with (roughly corresponding) chapters in the notes are as follows
- Discrete Probability spaces (Chapters 2 and 3).
- Laws of Probability (Chapter 6).
- Combinations of Events (Chapters 8 and 9).
- Independence and Conditional Probabilities (Chapters 9 to 12).
- Discrete and Continuous distributions (Chapters 13 to 16).
- Some important distributions: Bernoulli, Binomial, Geometric, Exponential, Cauchy.
- Random variables: Distributions, Independence, Expectation and Variance (Chapters 18, 20, 21).


Travel Gear
An unsorted list of travel gear I pack in my backpack. Inspired by Matt Mullenweg's "What's in my bag" post, Polle de Maagt's travel notebook.
I like to travel with only the minimal essentials. Everything that I'm not using regularly, I simply don't want to take it with me. IMHO there is no need to pack your backpack for all the possible "What if" moments. Here are 23 other reasons why you should travel with just one carry-on backpack.
What are my preferred backpacks?
- Goruck Echo
- My 16L daily EDC backpack that I use for commuting to work when I only carry my essentials. And still some space available for an extra jacket or some groceries.
- Goruck GR1
- This is my other daily EDC backpack that I use when I need to take some extra stuff with me. For example, when I go to the gym before or after work. It always contains the same EDC gear as the Echo. It has a volume of 26L. This daypack is also big enough for a weekend or midweek away. It even worked perfectly for my 10 day trip to Montevideo and Buenos Aires. While my brother used the GR2.
- Goruck GR2
- The GR2 is my workhorse for longer trips. I used it to travel around Japan for 3 weeks, and Mexico for 2 weeks. And the best thing is, it is still a TSA carry-on compliant 40L backpack, so a great travel backpack.
For me, the pros of the Goruck backpacks are first of all the aesthetics. It has a low profile, it is not obvious that you are a backpacker or tourist and thus carry value stuff. Plus it looks good when traveling for work. The backpack opens flat due to its clamshell design., which makes (un)packing so much easier. It has a separate compartment for my laptop, which you can easily access, but still safe because it is hidden and close to your back.
A big con is definitely the steep price. (Be patient and wait for a discount moment like the 4th of July, Veterans Day, or Black Friday). But I also think it will pay itself back because of the robust build of the bag. It feels like it will last a lifetime. Another thing is the outside front pocket.  Unfortunately, it is not really a quick access pocket. For that purpose, I use the small pocket inside the bag. And the front pocket to put away small flat items and I irregularly need.
Reviews by others:
Size comparison of the Goruck backpacks
Centimeters and Kilograms are converted from Inches and Pounds mentioned on Goruck's website.
| |
Goruck Echo |
Goruck GR1 |
Goruck GR2 |
| Volume |
16 Liter |
26 Liter |
40 Liter |
| Height (Centimeter) |
41.91 cm |
50.8 cm |
55.88 cm |
| Width (Centimeter) |
29.21 cm |
30.48 cm |
31.75 cm |
| Depth (Centimeter) |
10.79 cm |
17.15 cm |
22.86 cm |
| Height (Inch) |
16.5″ |
20″ |
22″ |
| Width (Inch) |
11.5″ |
12″ |
12.5″ |
| Depth (Inch) |
4.25″ |
6.75″ |
9″ |
| Weight (Kilogram) |
0.88 kg |
1.45 kg |
2.15 kg |
| Weight (Pounds) |
1.95 lbs |
3.20 lbs |
4.75 lbs |
| Laptop Compartment |
Up to 13″ |
Up to 17″ |
Up to 17″ |
| Price |
Discontinued |
$295 |
$495 |
| Warranty & Customization |
What travel gear is in my backpack?
- Goruck Field Pocket
- 3L or 4L field pocket that carries all sorts of small EDC and valuable tech items. Like all the Apple adapters, cables, and power plugs. I have multiple field pockets that I can use if I'm packing more or less depending on the trip.
- Sea to Summit See Pouch (medium)
- This is a handy and light 2L Dopp kit for my toiletries. Because you can see through it, this makes it very easy to go through security at an airport when traveling with only a carry-on backpack.
- Peak Design packing cubes & shoe pouch
- Light and strong packing cubes to hold my clothing. I have the packing cubes in both sizes, small and medium. They are compressible, so they can almost hold double the amount of a normal non-compressible packing cube. And there is a separate compartment in the cube to hold dirty clothing. Besides the packing cubes, I also have to shoe pouch to hold an extra pair of shoes and flip-flops. This way I keep the inside of my backpack clean from any sand and odors.
- Goruck American Waterproof Jacket
- A super straightforward jacket to block out wind and water when cycling through Amsterdam. It has enough big pockets to put a lot of stuff in. It is the perfect travel winter jacket in combination with my Patagonia Nano Puff.
- Patagonia Nano Puff
- A warm lightweight jacket that can be worn underneath my wind and water jacket. Together they make an excellent winter jacket. And it is very packable so it doesn't take up a lot of space in my backpack.
- Apple MacBook Pro 14”
- With charger + cable.
- Apple World Travel Adapter Kit
- Handy kit with all the adapters that you may need. They work for the laptop and iPad charger. I always take an iPad charger and cable with me for charging my iPhone. And only the adapters that I need to take for the specific trip.
- Apple Keyboard & Mouse
- Both are small and lightweight. Ideal for traveling.
- Roost laptop stand
- A laptop stand that can raise your laptop to the ideal height for your back and neck. Making remote working just as good as working from the office. It is super lightweight and packs super small, so it fits easily in that small spot somewhere in my Goruck.
- Apple AirPods Pro
- Maybe the best gadget purchase I have done in the last couple of years. I don't miss untangling the cables. And I feel totally comfortable wearing them while biking through Amsterdam or rushing to an airport gate.
- Bose QuietComfort 25 Noise Canceling headphones
- I can't imagine flying intercontinental without noise-canceling headphones anymore. To me, they are essential for a comfortable flight.
- Anker Powerbank (20.100 mAh)
- ABC! Always Be Charging. To charge my iPhone and MacBook a couple of times. I noticed it can charge my iPhone at least 5 times.
- Dopper water bottle
- Stay hydrated. Drink water. Alcohol doesn't count. Reduce plastic waste. But make sure it's empty before you go through security at the airport. Fill it up after security for free and you will easily save €5,- every trip.
- Secrid wallet
- I actually have 2 Secrid wallets. One is in my pocket with the most important cards that I use daily. And a second one with less important cards and some emergency cash.
- Other small items
- Lip balm, tissues, pen (to fill out forms at the airport border control), eye mask, earplugs, nail clipper, painkillers, sleep aids, multivitamins, protein bars (for a healthy snack), and energy gum (to battle jetlag and hangovers).
My favorite apps when traveling
- Google Maps
- IMHO the best app to find my way around in a city, because it now also include public transport information.
- Foursquare & Swarm
- To find hidden local gems, and keep a logbook of all the places I've been.
- Google Translate
- Before I go on a trip I download the local language in the app. Live translations to English by the camera even worked for Japanese.
- Uber
- Get around a city quickly. Especially handy for traveling to and from an airport. When I'm tired or a bit stressful because of catching my flights. I don't want to deal with obnoxious cab drivers, and even negotiate about prices.
- Reisapp Buitenlandse Zaken
- The app from the Dutch Ministry of Foreign Affairs that provides information around travel advise, safety and contact details for each country. Always handy if you get in to an emergency situation.
- Spotify
- For podcasts and my favorite music playlists (already downloaded at home!) to lift me up when feeling down or to get to sleep in the airplane.
- Notes
- Yes, the native note-taking app from Apple. I use it to make a simple itinerary. Which I can share easily with family and friends and always works even when I don't have an internet connection.
- Things
- Simply the best to-do list app for iOS and Mac.
Basic words to use when traveling
| English |
Dutch |
German |
French |
Spanish |
Italian |
| Hi! |
Hoi! |
Hallo! |
Bonjour! |
Hola! |
Ciao! |
| Two beers, please. |
Twee biertjes alstublieft. |
Zwei Bier bitte. |
Deux bières, s'il vous plaît. |
Dos cervezas, por favor. |
Due birre, per favore. |
| Thanks |
Dankjewel |
Danke schön! |
Merci beaucoup! |
¡Muchas gracias! |
Grazie mille! |
| Bye! |
Tot ziens! |
Auf Wiedersehen! |
Au revoir! |
¡Adiós! |
Arrivederci! |


We are happy to announce that FIRSTNAME LASTNAME has decided to join the AlethiCorp family. FIRSTNAME will be starting as a Junior Information Management Consultant.
We look forward to working with FIRSTNAME, and hope everyone will do their best to ensure that PRONOUN finds POSSESSIVE place in our global community.


mortality climate excess change adaptation heatwave-related temperature Data future Writing heatwaves review editing curation population [], data Health scenarios daily deaths increase Research heat health heatwave http://orcid.org/--- different series {border-style gas greenhouse University px;} #ffffff;} Environmental Table period using Health, Public expected projections National number RCP., projected study Guo average impact impacts mean results used Appendix Department changes communities global hypothetical relative Medical United historical model related time School University, countries countries/regions days estimated heatwave–mortality higher including models mortality. temperatures variant, Council Environ Gasparrini Institute Moldova RCP. associated community. high-variant observed percentile physiological risk supported Climate analysis behavioural community community-specific countries/regions, effect effects estimates findings high human influence interventions levels project px; variability warming waves {font-family Fig However, Perspect. Project Sciences States Taiwan adaptation, adaptation. associations change, change. change; empirical font-size increases information percent policy public regions smaller subtropical tropical weather (e.g., Assessment Atmospheric China Concentration European Heatwaves Japan Medicine, Model Population RCP, ambient based calculated changing city consider considered double; emissions, future. heatwaves. http://dx.doi.org/./ intensity large method modelled people present range risks scenario small summer threshold ./ehp. Armstrong Bell Finland GCMs Global Health. Heat Impact Int Intercomparison Lavigne Oulu, Representative Thus, Tobias adapt affected assumption assumptions border-color border-width defined developed developing distribution emission emissions heatwaves, hot include indirect limited low lower mitigation overall physical populations potential regional research scenario, scenarios, scenarios. significant solid; sources temperature, uncertainties uncertainty year {font-style {font-weight (Fig Australia Brazil Change Earth Funding ISIMIP Korea Medicine Methodology Pathway. Paulo, Science Spain São Tawatsupa Tong ability accounting annual applied assessment association assumed attributable bold cause characterised clear collection compared confidence country courier; current day definition demographic differ diseases, em; estimate frequency future, improvement interventions, low-variant median multicountry normal original outcomes possible previous provided pt;} quality rate reduce rise variant) variation wave


{% extends "layout.html" %} {% block content %} Time limit exceded It took more than {{timeout}} seconds for {{error_report}} to respond. {% endblock %}
It took more than {{timeout}} seconds for {{error_report}} to respond.


--- layout: post title: Cloud doodle date: '2017-02-23T10:30:31+01:00' categories: - Sketches picture_url: https://film-Storyboards.tumblr.com/post/157614887465 ---


My name is Calvin Yu, I'm a Technologist living in Atlanta, Georgia, where I'm a developer for Scoutmob.


[gecode-users] Creating a function to solve CP sub-problem instances
Christian Schulte
cschulte at kth.se
Mon Sep 15 11:49:07 CEST 2014
Hi,
The setup is that a script must be a sub-class of Gecode::Space. However,
you can pass these objects around, so I do not really see where the problem
is, sorry.
Cheers
Christian
--
Christian Schulte, Professor of Computer Science, KTH,
www.gecode.org/~schulte/
From: users-bounces at gecode.org [mailto:users-bounces at gecode.org] On Behalf
Of Nathan Horvath
Sent: Tuesday, September 09, 2014 8:45 PM
To: users at gecode.org
Subject: [gecode-users] Creating a function to solve CP sub-problem
instances
Hi,
I'm trying to implement a tabu search algorithm that involves solving small
constraint programming sub-problems. Firstly, I used the examples found in
the Modelling and Programming Guide to develop a script to solve an instance
of a sub-problem. Now, I'm looking to expand upon this by creating a
function that consumes data to solve a sub-problem, solve it using Gecode,
and then return an optimal solution to the sub-problem in a some data
structure.
I'm having problems trying to turn my script into a callable function. Given
from what I've seen in the guide, it seems like it is not possible, or it's
taken as something trivial and is not glossed over, as I haven't found any
similar examples after searching on-line. What I started to do was play
around with the print function (same layout as in the SEND MOST MONEY
examples in the guide) and try replacing it with a function that interprets
the solution, but I've had no luck.
One could classify my experience with Gecode and C++ as very beginner (I
picked up C++ so I could use Gecode's capabilities), so I apologize if this
question comes off as ignorant or out of scope. Any help would be greatly
appreciated.
Cheers,
Nathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gecode.org/pipermail/users/attachments/20140915/7316cabc/attachment.html>
More information about the users
mailing list


Using the Python library¶
Installation¶
If you have installed SimulaQron using pip, the cqc interface for python should already be installed. If needed you can also only install the CQC interface in Python using pip by typing:
pip3 install cqc
Usage¶
The python library provides a way to program a protocol on a network where the nodes listen to instructions through the classical-quantum combiner (CQC) interface. In the following examples the network is simulated by SimulaQron. But the same examples could be executed on a network with real quantum hardware which allows for instructions through the CQC interface, which is the aim for the 2020 quantum internet demonstrator.
To use the Python library you first need instantiate an object from the class cqc.pythonLib.CQCConnection
. This should be done in a context, that is using a with
-statement as follows:
with CQCConnection("Alice") as alice:
# your program
This is to make sure the CQCConnection is correctly closed by the end of the program and that the qubits in the backend are released, even if an error occurs in your program.
Note
It is still possible to initialize a CQCConnection
in the old way, i.e. without with
, however you will receive a warning message everytime you create a qubit.
Let’s look at an extremely trivial example were we have the node Alice allocate a qubit, perform a Hadamard gate and measure the qubit:
with CQCConnection("Alice") as alice:
q = qubit(Alice)
q.H()
m = q.measure()
print(m)
A object from the qubit
-class is created with the CQCConnection
as argument, such that whenever an operation is applied to the qubit a CQC message will be sent to the simulation backend to actually perform this operation on the simulated qubit.
For more examples using the Python library see Getting started and Examples using the python library.
We give some more detailed information below on how the classical communication between nodes in the application layer can be realized and also provide some useful commands to program a protocol using the Python library.


Manifesto Software Craftsmanship
Not only working software, but also well-crafted software
Not only responding to change, but also steadily adding value
Not only individuals and interactions, but also a community of professionals
Not only customer collaboration, but also productive partnerships


Heading 1
This is a paragraph
This is another paragraph
first name
last name
Byron
Gilbreath
Heading
2
JMU Website
Coffee
Tea
Black tea
Green tea
Milk


WWI Challenge
A New, Interesting and Instructive Game for Young and Old
Games
Rules
1. Each game consists of 12 Questions and Answers - The questions appear at the top of the page, and the answers appear at the bottom.
2. Read the questions at the top and choose the answer which you think is correct. Drag and drop your answer card over to the corresponding question card.
3. If you make a mistake or change your mind, press the Reset button to start the game again
4. After you drag and drop the last card, your score will be revealed.
5. Have fun!


Toggle navigation
Brooklyn Outdoor film Festival
Home
About
Schedule
Reservations
Contact
Contact us
Brooklyn Outdoor Film Festival
180 S 4th St,
Brooklyn, NY 11211
P:
(321) 965-3486
BrooklynFilmFestival
<EMAIL_ADDRESS>Write to Us
Name
Email
Phone
Subject
General inquiry
Admission
Registration
Press inquiries
Group booking
Comment
Send Me Notifications
Submit


Lists By Sagar
Can you make a list of types of foods (Meat, Vegetables, E.T.C.)
with example lists inside the other list? E.G.
Chicken
Beans
Beef
Brocoli
Now try it on your own!
Add to these lists
An ordered List
pizza
avacado
An un-ordered list
Spain
Brazil
The following are all form elements (add to them/finish them): Put in your Full name.
First Name: Pick 2 or more that apply to you
Im Super
Im very easily BAMBOOZLED
Im I dont know
I need help
I do not have depression Pick the one best describing you
Im not very mentally powered
Im decent
Im Smart


Hmm, we couldn't find your address or ward. Please try again.
{% endif %} {% if ward %}Get more information about this ward in Wazimap
#} {% for clinic in clinics.records %} {% if clinic > 0 %}Name: {{ clinic.Name }}
Ward: ({{ clinic.wardid }})
Physical Address: {{ clinic.Physical_Address }}
Trading Hours: ({{ clinic.Trading_Hours }})
Contact Details: {{ clinic.Contact_Details }}
District: ({{ clinic.District }})
No Hospitals or clinics found, Please try again.
{% endfor %}An Open Data Durban widget.


Class AbstractUdpLink
Abstract base class for non-reliable UDP/IP links.
Inheritance
System.Object
AbstractUdpLink
Implements
System.IDisposable
Inherited Members
System.Object.Equals(System.Object)
System.Object.Equals(System.Object, System.Object)
System.Object.GetHashCode()
System.Object.GetType()
System.Object.MemberwiseClone()
System.Object.ReferenceEquals(System.Object, System.Object)
System.Object.ToString()
Namespace: x2net
Assembly: x2net.dll
Syntax
public abstract class AbstractUdpLink : Link, IDisposable, ICase
Constructors
|
Improve this Doc
View Source
AbstractUdpLink(String)
Initializes a new instance of the AbsractUdpLink class.
Declaration
protected AbstractUdpLink(string name)
Parameters
| Type |
Name |
Description |
| System.String |
name |
|
Fields
|
Improve this Doc
View Source
map
Declaration
protected SortedList<int, EndPoint> map
Field Value
| Type |
Description |
| System.Collections.Generic.SortedList<System.Int32, System.Net.EndPoint> |
|
|
Improve this Doc
View Source
reverseMap
Declaration
protected Dictionary<EndPoint, int> reverseMap
Field Value
| Type |
Description |
| System.Collections.Generic.Dictionary<System.Net.EndPoint, System.Int32> |
|
|
Improve this Doc
View Source
rwlock
Declaration
protected ReaderWriterLockSlim rwlock
Field Value
| Type |
Description |
| System.Threading.ReaderWriterLockSlim |
|
|
Improve this Doc
View Source
rxBuffer
Declaration
protected Buffer rxBuffer
Field Value
|
Improve this Doc
View Source
socket
Declaration
Field Value
| Type |
Description |
| System.Net.Sockets.Socket |
|
|
Improve this Doc
View Source
syncRoot
Declaration
protected object syncRoot
Field Value
| Type |
Description |
| System.Object |
|
|
Improve this Doc
View Source
txBuffer
Declaration
protected Buffer txBuffer
Field Value
|
Improve this Doc
View Source
txFlag
Declaration
Field Value
| Type |
Description |
| System.Boolean |
|
|
Improve this Doc
View Source
txQueue
Declaration
protected Queue<Event> txQueue
Field Value
| Type |
Description |
| System.Collections.Generic.Queue<Event> |
|
Properties
|
Improve this Doc
View Source
Gets or sets the BufferTransform for this link.
Declaration
public IBufferTransform BufferTransform { get; set; }
Property Value
Methods
|
Improve this Doc
View Source
AddEndPoint(EndPoint)
Adds the specified remote end point to the known peers, and returns
the handle associated with it.
Declaration
public int AddEndPoint(EndPoint endPoint)
Parameters
| Type |
Name |
Description |
| System.Net.EndPoint |
endPoint |
|
Returns
| Type |
Description |
| System.Int32 |
|
|
Improve this Doc
View Source
BeginReceiveFrom()
Declaration
protected void BeginReceiveFrom()
|
Improve this Doc
View Source
Bind(Int32)
Binds the underlying socket with the specified local port.
Declaration
public AbstractUdpLink Bind(int localPort)
Parameters
| Type |
Name |
Description |
| System.Int32 |
localPort |
|
Returns
|
Improve this Doc
View Source
Dispose(Boolean)
Declaration
protected override void Dispose(bool disposing)
Parameters
| Type |
Name |
Description |
| System.Boolean |
disposing |
|
Overrides
|
Improve this Doc
View Source
GetEndPoint(Int32)
Gets the remote end point of the known peer identified by the
specified handle.
Declaration
public EndPoint GetEndPoint(int handle)
Parameters
| Type |
Name |
Description |
| System.Int32 |
handle |
|
Returns
| Type |
Description |
| System.Net.EndPoint |
|
|
Improve this Doc
View Source
GetHandle(EndPoint)
Gets the handle of the known peer associated with the specified
remote end point.
Declaration
public int GetHandle(EndPoint endPoint)
Parameters
| Type |
Name |
Description |
| System.Net.EndPoint |
endPoint |
|
Returns
| Type |
Description |
| System.Int32 |
|
|
Improve this Doc
View Source
Listen()
Begin receiving incoming events on this link.
Declaration
|
Improve this Doc
View Source
OnReceiveFromInternal(Int32, EndPoint)
Declaration
protected void OnReceiveFromInternal(int bytesTransferred, EndPoint endPoint)
Parameters
| Type |
Name |
Description |
| System.Int32 |
bytesTransferred |
|
| System.Net.EndPoint |
endPoint |
|
|
Improve this Doc
View Source
OnSendToInternal(Int32)
Declaration
protected void OnSendToInternal(int bytesTransferred)
Parameters
| Type |
Name |
Description |
| System.Int32 |
bytesTransferred |
|
|
Improve this Doc
View Source
ReceiveFromInternal()
Declaration
protected abstract void ReceiveFromInternal()
|
Improve this Doc
View Source
RemoveEndPoint(EndPoint)
Removes the specified remote end point from the known peers.
Declaration
public void RemoveEndPoint(EndPoint endPoint)
Parameters
| Type |
Name |
Description |
| System.Net.EndPoint |
endPoint |
|
|
Improve this Doc
View Source
RemoveHandle(Int32)
Removes the specified handle from the known peers.
Declaration
public void RemoveHandle(int handle)
Parameters
| Type |
Name |
Description |
| System.Int32 |
handle |
|
|
Improve this Doc
View Source
Send(Event)
Sends out the specified event to a known peer.
Declaration
public override void Send(Event e)
Parameters
| Type |
Name |
Description |
| Event |
e |
|
Overrides
|
Improve this Doc
View Source
SendToInternal(EndPoint)
Declaration
protected abstract void SendToInternal(EndPoint endPoint)
Parameters
| Type |
Name |
Description |
| System.Net.EndPoint |
endPoint |
|
Implements
System.IDisposable
Extension Methods


[1]
hotel has its own private car park which from memory was about 10 euro and about 10 euro a day .
[2]
singles pushed together as the norm in spain -lrb-/-lrb- an absolute pet hate of mine -rrb-/-rrb- .


This data is about patients taking an appointment from the doctors. We are going to predict the ‘no-show’ event which occurs when they have an appointment but never showed up. The original data is on Kaggle.
library(ggplot2)
library(data.table)
library(magrittr)
data <- fread("data/noshowdata/no-show-data.csv")
# some data cleaning
data[, c("PatientId", "AppointmentID", "Neighbourhood") := NULL]
setnames(data,
c("No-show",
"Age",
"Gender",
"ScheduledDay",
"AppointmentDay",
"Scholarship",
"Hipertension",
"Diabetes",
"Alcoholism",
"Handcap",
"SMS_received"),
c("no_show",
"age",
"gender",
"scheduled_day",
"appointment_day",
"scholarship",
"hypertension",
"diabetes",
"alcoholism",
"handicap",
"sms_received"))
# for binary prediction, the target variable must be a factor
data[, no_show := factor(no_show, levels = c("Yes", "No"))]
data[, handicap := ifelse(handicap > 0, 1, 0)]
# create new variables
data[, gender := factor(gender)]
data[, scholarship := factor(scholarship)]
data[, hypertension := factor(hypertension)]
data[, alcoholism := factor(alcoholism)]
data[, handicap := factor(handicap)]
data[, scheduled_day := as.Date(scheduled_day)]
data[, appointment_day := as.Date(appointment_day)]
data[, days_since_scheduled := as.integer(appointment_day - scheduled_day)]
# clean up a little bit
data <- data[age %between% c(0, 95)]
data <- data[days_since_scheduled > -1]
data[, c("scheduled_day", "appointment_day", "sms_received") := NULL]
library(h2o)
h2o.init()
take a look to remember how the data looked like
data <- data.table(data)
skimr::skim(data)
turn it into a h2o dataframe
h2o_data <- as.h2o(data) #
str(h2o_data)
Create train / validation / test sets, cutting the data into 5% - 45% - 50% parts.
#3 parts
splitted_data <- h2o.splitFrame(h2o_data,
ratios = c(0.05, 0.45), #3 parts
seed = 123)
data_train <- splitted_data[[1]]
data_valid <- splitted_data[[2]]
data_test <- splitted_data[[3]]
#just cross checking the number of rows
nrow(data_train) #5508 rows
nrow(data_valid) #49793 rows
nrow(data_test) #55174 rows
nrow(data_test) + nrow(data_train) + nrow(data_valid)
Train a benchmark model of your choice using h2o (such as random forest, gbm or glm) and evaluate it on the validation set.
Lets get a benchmark of RF:
y <- "no_show"
X <- setdiff(names(h2o_data), y) #take the set difference. i dont drop any other variables
print(X) #cross check if variables are right
benchmark <- h2o.randomForest(X, y, #
training_frame = data_train, #specify training frame
ntrees = 200,#
nfolds = 5, #5 folds
seed = 123)
check out the AUC alone, coming from cv
print(h2o.auc(benchmark, xval = TRUE))
## [1] 0.6881471
now, evaluate it on the validation set.
validation_performances <- list(
"rf" = h2o.auc(h2o.performance(benchmark, newdata = data_valid))) #we could collect more models under a list like this... We will do it during deep learning below
validation_performances
## $rf
## [1] 0.7004903
Experiment with parameter settings
For all models, supply the validation_frame and use AUC as a stopping metric. Present different model versions and evaluate them on the validation set. Which one performs the best?
dl_1 <- h2o.deeplearning(x = X,
y = y,
training_frame = data_train,
validation_frame = data_valid, #early stop to prevent overfitting
stopping_metric = 'AUC',
reproducible = TRUE, # makes training slower but makes it reproducible
seed = 123)
First we play a bit with topology: varying number of layers and nodes within layers…
The default: two hidden layers with 200-200 neurons. Makes sense to experiment with shallower but more neuron or with deeper and less neurons per layer architectures.
dl_2 <- h2o.deeplearning(x = X, #shallow_small_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, # makes training slower but makes it reproducible
hidden = c(50), #this is only 1 hidden layer. 50 node in it.
seed = 123)
dl_3 <- h2o.deeplearning(x = X, #shallow_large_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, # makes training slower but makes it reproducible
hidden = c(500), #shallow but large number of neurons.
seed = 123)
dl_4 <- h2o.deeplearning(x = X, #deep_small_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, # makes training slower but makes it reproducible
hidden = c(30, 30, 30, 30, 30, 30, 30),#7 layers, 30 nodes in
seed = 123)
keeping in mind, there is no one single recipe for good results, try, iterate and see…
dl_5 <- h2o.deeplearning(x = X, #deep_large_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, # makes training slower but makes it reproducible
hidden = c(100, 100, 100, 100, 100),
seed = 123)
Now experiment with activation function
…the nonlinear transformative function used. Its defaultis Rectifier.
dl_6 <- h2o.deeplearning(x = X, #tanh_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, #
hidden = c(30, 30, 30, 30, 30),
activation = "Tanh", #similar to the sigmoid. transformed variant of sigmoid
seed = 123)
Trying with epochs..
How many times will all training datapoints be used to adjust the model in the course of the optimization (note: early stopping is used by default so there is no guarantee that all epochs will be used).
The algorithm is stachoastic gradient descent. It is a computationally optimum variant. Takes all the points and they are used one by one, once.
dl_7 <- h2o.deeplearning(x = X, #more_epochs_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = TRUE, #
hidden = c(100, 100, 100),
activation = "Tanh",
epochs = 20, #typically increase this and make it stop earlier.
seed = 123)
Now experiment with dropout (both hidden and input layers)
with how large probability will neurons be left out of the model at a step (defaults to 0.5). Have to use “WithDropout” activation to use dropout.
Basically at each update we are shutting down some nodes, randomly. this way we will have a large variaty of networks. A bit similar to random forest idea.
dl_8 <- h2o.deeplearning(x = X, #dropout_model
y = y,
reproducible = T,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
hidden = c(100, 100, 100),
activation = "RectifierWithDropout", #withDropout!
hidden_dropout_ratios = c(0.05, 0.05, 0.1), #for each hidden layer give a probability. leave out 10% of the neurons. defaults to 0.5 if you dont specify.
seed = 123)
input dropout ratio to drop some input features randomly similar to the one above, this time for imputs..
dl_9 <- h2o.deeplearning(x = X, #input_dropout_model
y = y,
reproducible = T,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
hidden = c(100, 100, 100),
input_dropout_ratio = 0.3,
activation = "RectifierWithDropout", #withDropout!
hidden_dropout_ratios = c(0.05, 0.05, 0.1), #for each hidden layer give a probability. leave out 10% of the neurons. defaults to 0.5 if you dont specify.
seed = 123)
Experiment with lasso, ridge regularization l1
, l2
, penalty terms. This is like in the penalized models we have done before.
dl_10 <- h2o.deeplearning(x = X, #regularized_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = 'AUC',
reproducible = T,
hidden = c(100, 100, 100),
activation = "RectifierWithDropout", #withDropout!
hidden_dropout_ratios = c(0.05, 0.05, 0.1), #for each hidden layer give a probability.
l1 = 0.001,
l2 = 0.001,
seed = 123)
Now, early stopping (changing stopping rounds, tolerance) and add also number of epochs
We have our metric (AUC). A moving average is calculated for AUC. Question is how much is enough so that we continue the optimzation. The answer is stopping tolerance. 0.01 is needed in order to continue training in our example below. If it was 0 then any increase would be fine and it would take a long long time.
validation frame. point is performing an optimization ultimately we want our data to perform out of sample. we check this out continusly at each step. then we see if it worths to continue training this.
dl_11 <- h2o.deeplearning(x = X, #early_stopping_model
y = y,
training_frame = data_train,
validation_frame = data_valid,
stopping_metric = "AUC",
reproducible = T,
hidden = c(100, 100, 100),
epochs = 100,
stopping_rounds = 2,
activation = "RectifierWithDropout", #withDropout!
hidden_dropout_ratios = c(0.05, 0.05, 0.1), #for each hidden layer give a probability.
stopping_tolerance = 0.01,
seed = 123)
validation_performances <- list(
"benchmark" = h2o.auc(h2o.performance(benchmark, newdata = data_valid)),
"dl_01" = h2o.auc(h2o.performance(dl_1, newdata = data_valid)),
"dl_02" = h2o.auc(h2o.performance(dl_2, newdata = data_valid)),
"dl_03" = h2o.auc(h2o.performance(dl_3, newdata = data_valid)),
"dl_04" = h2o.auc(h2o.performance(dl_4, newdata = data_valid)),
"dl_05" = h2o.auc(h2o.performance(dl_5, newdata = data_valid)),
"dl_06" = h2o.auc(h2o.performance(dl_6, newdata = data_valid)),
"dl_07" = h2o.auc(h2o.performance(dl_7, newdata = data_valid)),
"dl_08" = h2o.auc(h2o.performance(dl_8, newdata = data_valid)),
"dl_09" = h2o.auc(h2o.performance(dl_9, newdata = data_valid)),
"dl_10" = h2o.auc(h2o.performance(dl_10, newdata = data_valid)),
"dl_11" = h2o.auc(h2o.performance(dl_11, newdata = data_valid))
)
Make a data frame and present the results
library(tidyr)
library(dplyr)
df <- data.frame(validation_performances) %>%
gather("model", "AUC") %>%
arrange(desc(AUC))
df %>%
head() %>%
pander::pander()
| model | AUC |
|---|---|
| dl_04 | 0.7096 |
| dl_01 | 0.7083 |
| dl_08 | 0.7079 |
| dl_06 | 0.7068 |
| dl_07 | 0.7056 |
| dl_05 | 0.7051 |
library(ggplot2)
library(ggthemes)
mycolours <- c("TRUE" = "red", "FALSE" = "grey50")
df %>%
filter(AUC >= 0.69) %>% #sometimes I got models less than 69%, i dont need them
mutate(maxAUC = if_else(AUC == max(AUC), TRUE, FALSE)) %>%
ggplot(aes(model,AUC, group = 1)) +
geom_line() +
geom_point(size = 3, aes(colour = maxAUC)) +
scale_color_manual("Winner", values = mycolours) +
scale_y_continuous(labels=scales::percent) +
ggtitle("The Box of Pandora") +
theme_fivethirtyeight()
This was a back and forth process. I was tuning and plotting different possibilities.
Let me save this as a function and use in my future analysis. Because it feels like this kind of comparison will repeat a lot. The function is called “pandora”. It takes a list of model performances, makes a dataframe and gives out visualization. And actually Pandora is a bad name. A better name can be something like: make_performance_comparison_plot_validset(). I just like using the word Pandora from the book Elements of Statistical Learning. There the autor uses this word when he is comparing different models: “Now it is time to open the box of Pandora and see the results…”
Evaluate the model that performs best based on the validation set on the test set.
print(h2o.auc(h2o.performance(dl_4, newdata = data_test)))
## [1] 0.7094104
We got almost the same result as validation set…
Next task is to apply Stacking. Take the same problem and data splits.
We will build 4 models of different families using cross validation, keeping cross validated predictions. We will evaluate validation set performance of each model.We will also see how large are the correlations of predicted scores of the validation set produced by the base learners.
glm_model <- h2o.glm(
X, y,
training_frame = data_train,
family = "binomial",
alpha = 1,
lambda_search = TRUE,
seed = 123,
nfolds = 5,
keep_cross_validation_predictions = TRUE
)
gbm_model <- h2o.gbm(
X, y,
training_frame = data_train,
ntrees = 200,
max_depth = 10,
learn_rate = 0.1,
seed = 123,
nfolds = 5,
keep_cross_validation_predictions = TRUE
)
deeplearning_model <- h2o.deeplearning(
X, y,
training_frame = data_train,
hidden = c(32, 8),
seed = 123,
nfolds = 5,
keep_cross_validation_predictions = TRUE
)
rf_model <- h2o.randomForest(
X, y,
training_frame = data_train,
ntrees = 200,
seed = 123,
nfolds = 5,
keep_cross_validation_predictions = TRUE
)
ensemble_model <- h2o.stackedEnsemble(
X, y,
training_frame = data_train,
base_models = list(glm_model,
gbm_model,
deeplearning_model,
rf_model))
# inspect validation set correlations of scores
predictions <- data.table(
"glm" = as.data.frame(h2o.predict(glm_model, newdata = data_valid)$Y)$Y,
"gbm" = as.data.frame(h2o.predict(gbm_model, newdata = data_valid)$Y)$Y,
"dl" = as.data.frame(h2o.predict(deeplearning_model, newdata = data_valid)$Y),
"rf" = as.data.frame(h2o.predict(rf_model, newdata = data_valid)$Y)$Y
)
GGally::ggcorr(predictions, label = TRUE, label_round = 2)
We can see very clear correlations. Highly correlated ones are moving together. It means when RF is saying “yes”, most of the time GBM is also saying “yes”. They have similarity in predicted probability scores. This kind of relationship is not as much strong between GBM and GLM.
The more uncorrelated predictions are, the more room there is to improve individual models.
valid_performances <- list(
"glm" = h2o.auc(h2o.performance(glm_model, newdata = data_valid)),
"gbm" = h2o.auc(h2o.performance(gbm_model, newdata = data_valid)),
"dl" = h2o.auc(h2o.performance(deeplearning_model, newdata = data_valid)),
"rf" = h2o.auc(h2o.performance(rf_model, newdata = data_valid))
)
#lets use the function we created before: pandora!
pandora(valid_performances)
The bests are around 69.6% - 70.04%
# for the ensemble model
print(h2o.auc(h2o.performance(ensemble_model, newdata = data_valid)))
## [1] 0.7116316
an improvement! better then the individuals.
The baseline meta-learner is a glm model. We can try GBM also:
ensemble_model_gbm <- h2o.stackedEnsemble(
X, y,
training_frame = data_train,
metalearner_algorithm = "gbm",
base_models = list(glm_model,
gbm_model,
deeplearning_model,
rf_model))
print(h2o.auc(h2o.performance(ensemble_model_gbm, newdata = data_valid)))
## [1] 0.6961582
But it gave a lower score. We can maybe give deep learning a chance:
ensemble_model_dl <- h2o.stackedEnsemble(
X, y,
training_frame = data_train,
metalearner_algorithm = "deeplearning",
base_models = list(glm_model,
gbm_model,
deeplearning_model,
rf_model))
print(h2o.auc(h2o.performance(ensemble_model_dl, newdata = data_valid)))
## [1] 0.6976598
The default glm metalearner is giving better result than deeplearning and gbm here. So let’s apply it to or test set:
print(h2o.auc(h2o.performance(ensemble_model, newdata = data_test)))
## [1] 0.710706
It is really very closed to the validation set performance of 71.2%!
Take the “Fashion MNIST dataset” where images of fashion items are to be classified in a similar manner to what we saw with handwritten digits. Images are in exactly the same format as we saw digits: 28x28 pixel grayscale images. The task is to build deep neural net models to predict image classes. The goal is to have as accurate classifier as possible: we are using accuracy as a measure of predictive power.
library(keras)
fashion_mnist <- dataset_fashion_mnist()
x_train <- fashion_mnist$train$x
y_train <- fashion_mnist$train$y
x_test <- fashion_mnist$test$x
y_test <- fashion_mnist$test$y
show_mnist_image <- function(x) {
image(1:28, 1:28, t(x)[,nrow(x):1],col=gray((0:255)/255))
}
show_mnist_image(x_train[18, , ])
show_mnist_image(x_train[35, , ])
show_mnist_image(x_train[100, , ])
We have 10 classes again just like the classic MNIST data. This time the classes are items like T-shirt, Bag, Dress, etc.
# reshape
x_train <- array_reshape(x_train, c(dim(x_train)[1], 784))
x_test <- array_reshape(x_test, c(dim(x_test)[1], 784))
# rescale
x_train <- x_train / 255 #between zero and one
x_test <- x_test / 255
# one-hot encoding of the target variable
y_train <- to_categorical(y_train, 10)
y_test <- to_categorical(y_test, 10)
model <- keras_model_sequential()
model %>%
layer_dense(units = 128, activation = 'relu', input_shape = c(784)) %>% #layers of neurons, dense means its a dense network. dropout is the probability of dropping. activation .
layer_dropout(rate = 0.3) %>%
layer_dense(units = 10, activation = 'softmax') #units of 10 because we want 10 cat. multiclass classification is done by softmax. input shape is important to give.
summary(model)
## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## dense_1 (Dense) (None, 128) 100480
## ___________________________________________________________________________
## dropout_1 (Dropout) (None, 128) 0
## ___________________________________________________________________________
## dense_2 (Dense) (None, 10) 1290
## ===========================================================================
## Total params: 101,770
## Trainable params: 101,770
## Non-trainable params: 0
## ___________________________________________________________________________
model %>% compile(
loss = 'categorical_crossentropy',
optimizer = optimizer_rmsprop(),
metrics = c('accuracy')
)
After I tried 25, 30, 60 and 35 epochs, I ended up using 5 epochs. It started to flatten out after 3 epochs already
history <- model %>% fit(
x_train, y_train, #they are 2 separete objects! train train
epochs = 5, batch_size = 128, #after how many gradient to be computed
validation_split = 0.2
)
plot(history)
model %>% evaluate(x_test, y_test)
Test set results are acc 85.5% and loss 39%.
It was 86% after 5 epochs. Test set performance is worse. Overall this is not a nice result. I should have played with the parameters better. This dataset is better than classic MNIST.
It makes use of the 2d structure of the original input data, applying filters exploiting the 2d images. In h2o
there is no option to use such models by default.
mnist <- dataset_mnist()
x_train <- mnist$train$x
y_train <- mnist$train$y
x_test <- mnist$test$x
y_test <- mnist$test$y
x_train <- array_reshape(x_train, c(nrow(x_train), 28, 28, 1)) #extra dim of 1. at each node we are not having real nubers but bew mageis are buint jeeking 2 dimen features. new artificial features and they are 2 dims.
x_test <- array_reshape(x_test, c(nrow(x_test), 28, 28, 1))
# rescale
x_train <- x_train / 255
x_test <- x_test / 255
# one-hot encoding of the target variable
y_train <- to_categorical(y_train, 10)
y_test <- to_categorical(y_test, 10)
it creates the neurons differently. keeps the original image.
cnn_model <- keras_model_sequential()
cnn_model %>%
layer_conv_2d(filters = 32, #
kernel_size = c(3, 3), #28 28 looks up 3 by 3 pixels and applys filters.
activation = 'relu',
input_shape = c(28, 28, 1)) %>%
layer_max_pooling_2d(pool_size = c(2, 2)) %>%
layer_dropout(rate = 0.25) %>%
layer_flatten() %>% #data transformation vectorization of 2 dim into 1.
layer_dense(units = 32, activation = 'relu') %>%
layer_dense(units = 10, activation = 'softmax')
summary(cnn_model)
## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## conv2d_1 (Conv2D) (None, 26, 26, 32) 320
## ___________________________________________________________________________
## max_pooling2d_1 (MaxPooling2D) (None, 13, 13, 32) 0
## ___________________________________________________________________________
## dropout_2 (Dropout) (None, 13, 13, 32) 0
## ___________________________________________________________________________
## flatten_1 (Flatten) (None, 5408) 0
## ___________________________________________________________________________
## dense_3 (Dense) (None, 32) 173088
## ___________________________________________________________________________
## dense_4 (Dense) (None, 10) 330
## ===========================================================================
## Total params: 173,738
## Trainable params: 173,738
## Non-trainable params: 0
## ___________________________________________________________________________
cnn_model %>% compile(
loss = 'categorical_crossentropy',
optimizer = optimizer_rmsprop(),
metrics = c('accuracy')
)
history <- cnn_model %>% fit(
x_train, y_train,
epochs = 5,
batch_size = 128,
validation_split = 0.2
)
after 5 epochs: val_loss: 0.0571 - val_acc: 0.9840
plot(history)
cnn_model %>% evaluate(x_test, y_test)
CNN acc is 98.13% and loss is 5.7%. This is a much better result than deep neural net! A clear victory CNN in image classification task.


[% page.title = "Perl 5.20.0 Release Announcement" %]
We are delighted to announce perl v5.20.0, the first stable release of version 20 of Perl 5.
You will soon be able to download Perl 5.20.0 from your favorite CPAN mirror or find it at:
https://metacpan.org/release/RJBS/perl-5.20.0/
SHA1 digests for this release are:
e925e4fc36e90eace19a1ca850f912618ba6788f perl-5.20.0.tar.bz2 52087047fd35e1020387f7c492f594a397be4e12 perl-5.20.0.tar.gz
You can find a full list of changes in the file "perldelta.pod" located in the "pod" directory inside the release and on the web.
Perl 5.20.0 represents approximately 12 months of development since Perl 5.18.0 and contains approximately 470,000 lines of changes across 2,900 files from 124 authors.
Excluding auto-generated files, documentation and release tools, there were approximately 280,000 lines of changes to 1,800 .pm, .t, .c and .h files.
Perl continues to flourish into its third decade thanks to a vibrant community of users and developers. The following people are known to have contributed the improvements that became Perl 5.20.0:
Aaron Crane, Abhijit Menon-Sen, Abigail, Abir Viqar, Alan Haggai Alavi, Alan Hourihane, Alexander Voronov, Alexandr Ciornii, Andy Dougherty, Anno Siegel, Aristotle Pagaltzis, Arthur Axel 'fREW' Schmidt, Brad Gilbert, Brendan Byrd, Brian Childs, Brian Fraser, Brian Gottreu, Chris 'BinGOs' Williams, Christian Millour, Colin Kuskie, Craig A. Berry, Dabrien 'Dabe' Murphy, Dagfinn Ilmari Mannsåker, Daniel Dragan, Darin McBride, David Golden, David Leadbeater, David Mitchell, David Nicol, David Steinbrunner, Dennis Kaarsemaker, Dominic Hargreaves, Ed Avis, Eric Brine, Evan Zacks, Father Chrysostomos, Florian Ragwitz, François Perrad, Gavin Shelley, Gideon Israel Dsouza, Gisle Aas, Graham Knop, H.Merijn Brand, Hauke D, Heiko Eissfeldt, Hiroo Hayashi, Hojung Youn, James E Keenan, Jarkko Hietaniemi, Jerry D. Hedden, Jess Robinson, Jesse Luehrs, Johan Vromans, John Gardiner Myers, John Goodyear, John P. Linderman, John Peacock, kafka, Kang-min Liu, Karen Etheridge, Karl Williamson, Keedi Kim, Kent Fredric, kevin dawson, Kevin Falcone, Kevin Ryde, Leon Timmermans, Lukas Mai, Marc Simpson, Marcel Grünauer, Marco Peereboom, Marcus Holland-Moritz, Mark Jason Dominus, Martin McGrath, Matthew Horsfall, Max Maischein, Mike Doherty, Moritz Lenz, Nathan Glenn, Nathan Trapuzzano, Neil Bowers, Neil Williams, Nicholas Clark, Niels Thykier, Niko Tyni, Olivier Mengué, Owain G. Ainsworth, Paul Green, Paul Johnson, Peter John Acklam, Peter Martini, Peter Rabbitson, Petr Písař, Philip Boulain, Philip Guenther, Piotr Roszatycki, Rafael Garcia-Suarez, Reini Urban, Reuben Thomas, Ricardo Signes, Ruslan Zakirov, Sergey Alekseev, Shirakata Kentaro, Shlomi Fish, Slaven Rezic, Smylers, Steffen Müller, Steve Hay, Sullivan Beck, Thomas Sibley, Tobias Leich, Toby Inkster, Tokuhiro Matsuno, Tom Christiansen, Tom Hukins, Tony Cook, Victor Efimov, Viktor Turskyi, Vladimir Timofeev, YAMASHINA Hio, Yves Orton, Zefram, Zsbán Ambrus, Ævar Arnfjörð Bjarmason.
The list above is almost certainly incomplete as it is automatically generated from version control history. In particular, it does not include the names of the (very much appreciated) contributors who reported issues to the Perl bug tracker.
Many of the changes included in this version originated in the CPAN modules included in Perl's core. We're grateful to the entire CPAN community for helping Perl to flourish.
For a more complete list of all of Perl's historical contributors, please see the AUTHORS file in the Perl source distribution.
The next major stable release of Perl 5, version 22.0, should appear in May 2015. It will be the conclusion of the v5.21.x development series, which will begin today and produce monthly releases on the 20th of each month.


January 1
A Happy New Year
For whosoever shall call upon the name of the Lord shall be saved.
Romans 10:13
Today people everywhere are gathering together to celebrate the first bright moments of the New Year.
Yet, for thousands of others, this day will be the toughest 24 hours of their lives. For those people, New Year's will serve only as a dim reminder of loneliness and loss. It will simply mark the beginning of another year of failure.
What about you? What's this year really going to be like for you?
You may appear to be happy enough. You may be passing out seasonal smiles and holiday greetings just like everybody else. But inside, you may hurt. You may be disappointed. You may even feel like you can't go on.
If so, I want you to know something. All of that can change in an instant. You can start your life over today and make this New Year's Day the most joyful day of your life!
How many times have you said to yourself, "If I could just start over, I'd do it all differently"? That doesn't have to be simply an idle wish. Jesus Christ has actually made it possible. That's His Christmas present to you. He paid the price for all your sins. He paid the penalty for all your mistakes.
That's the reason He came to earth. That's the reason He was born--so you could start over!
You may look at your life and say, "But I've made some terrible mistakes. I've done some despicable things." That doesn't matter. Jesus paid the price for them all!
How can you make a new start? Romans 10:9 says that "if thou shalt confess with thy mouth the Lord Jesus, and shalt believe in thine heart that God hath raised him from the dead, thou shalt be saved." It's as easy as saying, "Jesus, I'm turning my life over to You. From this day forward I'm Yours."
What better time to turn your life around than on this first day of the New Year. Right now, wherever you are, just give your life to Jesus. Then jump up and down and holler, "Praise God, I'm starting all over!" And find out what it really means to have a Happy New Year!
Scripture Reading: Romans 10:1-13


Abstract
Underwriters currently analyze the risk using Customer provided data, Medical reports etc. "e-RAS" system provides additional risk analyzing data from Social Media. This system not a replacement of existing methods, instead this is complimentry system.
Outcome
In addition to traditional inputs such as Customer provided data and medical info etc. The underwriter can make use of Social media in analyzing the risk.