Deployment
This section introduces production-ready deployment solution. Generally speaking, we are deploying a resilient RocketMQ cluster having no single point of failure.
Prerequisite
Before starting this section, make sure you have read Quick Start section, and are farmiliar with the core concepts and components of RocketMQ.
Production-ready Deployment
Name Server
To ensure the cluster can still function when one instance crashes, two or more name server instances are recommended. As long as there is one name server instance alive, the whole cluster remains in service.
Name server follows the share-nothing design paradigm. Brokers send heartbeat data to all name servers. Producers and consumers can query meta data from any of name servers available while sending / consuming messages.
Broker
Brokers can be divided into two categories according to their roles: master and slave. Master brokers provide RW access while slave brokers only accept read access.
To deploy a high-availability RocketMQ cluster with no single point of failure, a series of broker sets should be deployed. A broker set contains one master with brokerId set to 0 and several slaves with non-zero brokerIDs. All of the brokers in one set have the same brokerName. In serious scenarios, we should have at least two brokers in one broker set. Each topic resides in two or more brokers.
Configuration
When deploying a RocketMQ cluster, recommended configuration is listed below:
Broker configuration
| Property Name | Default value | Details |
|---|---|---|
| listenPort | 10911 | listen port for client |
| namesrvAddr | null | name server address |
| brokerIP1 | InetAddress for network interface | Should be configured if having multiple addresses |
| brokerName | null | broker name |
| brokerClusterName | DefaultCluster | this broker belongs to which cluster |
| brokerId | 0 | broker id, 0 means master, positive integers mean slave |
| storePathCommitLog | $HOME/store/commitlog/ | file path for commit log |
| storePathConsumerQueue | $HOME/store/consumequeue/ | file path for consume queue |
| mapedFileSizeCommitLog | 1024 * 1024 * 1024(1G) | mapped file size for commit log |
| deleteWhen | 04 | When to delete the commitlog which is out of the reserve time |
| fileReserverdTime | 72 | The number of hours to keep a commitlog before deleting it |
| brokerRole | ASYNC_MASTER | SYNC_MASTER/ASYNC_MASTER/SLAVE |
| flushDiskType | ASYNC_FLUSH | {SYNC_FLUSH/ASYNC_FLUSH}. Broker of SYNC_FLUSH mode flushes each message onto disk before acknowledging producer. Broker of ASYNC_FLUSH mode, on the other hand, takes advantage of group-committing, achieving better performance. |
CLI Admin Tool
RocketMQ provides a CLI(command-line interface) admin tool belt to query, manage and diagnose various issues.
How To Get
The admin tool is shipped along with RocketMQ. Either you download a pre-built binary version or build from source by yourself, you already have it.
In case you need the source code, the rocketmq-tools module contains its source code.
How to use
The Admin Tool is very easy to use. Here, for demonstration purpose, *nix environment is assumed.
Change directory to ${PACKAGE}/bin, command bash mqadmin, you should see the following help menu:
The most commonly used mqadmin commands are:
updateTopic Update or create topic
deleteTopic Delete topic from broker and NameServer
updateSubGroup Update or create subscription group
deleteSubGroup Delete subscription group from broker
updateBrokerConfig Update broker's config
updateTopicPerm Update topic perm
topicRoute Examine topic route info
topicStatus Examine topic Status info
topicClusterList get cluster info for topic
brokerStatus Fetch broker runtime status data
queryMsgById Query Message by Id
queryMsgByKey Query Message by Key
queryMsgByUniqueKey Query Message by Unique key
queryMsgByOffset Query Message by offset
queryMsgByUniqueKey Query Message by Unique key
printMsg Print Message Detail
sendMsgStatus Send msg to broker
brokerConsumeStats Fetch broker consume stats data
producerConnection Query producer's socket connection and client version
consumerConnection Query consumer's socket connection, client version and subscription
consumerProgress Query consumers's progress, speed
consumerStatus Query consumer's internal data structure
cloneGroupOffset Clone offset from other group
clusterList List all of clusters
topicList Fetch all topic list from name server
updateKvConfig Create or update KV config
deleteKvConfig Delete KV config
wipeWritePerm Wipe write perm of broker in all name server
resetOffsetByTime Reset consumer offset by timestamp(without client restart)
updateOrderConf Create or update or delete order conf
cleanExpiredCQ Clean expired ConsumeQueue on broker.
cleanUnusedTopic Clean unused topic on broker
startMonitoring Start Monitoring
statsAll Topic and Consumer tps stats
syncDocs Synchronize wiki and issue to github.com
allocateMQ Allocate MQ
checkMsgSendRT Check message send response time
clusterRT List All clusters Message Send RT
See ‘mqadmin help
usage: mqadmin clusterList [-h] [-i <arg>] [-m] [-n <arg>]
-h,--help Print help
-i,--interval <arg> specify intervals numbers, it is in seconds
-m,--moreStats Print more stats
-n,--namesrvAddr <arg> Name server address list, eg: <IP_ADDRESS>:9876;<IP_ADDRESS>:9876
Replication mode
To make sure that no successfully published message will be lost, RocketMQ provides a Replication mode to gain stronger durability and higher availability with two replication ways: Sync & Async.
Replication: Sync / Async Broker
Like many replication system, sync brokers wait until commit log is replicated to the slave before acknowledging. Async brokers, instead, return immediately after messages are processed on master.
How to configure
There are three pre-built configurations shipped with the distribution of RocketMQ under conf folder for your reference:
2m-2s-sync
2m-2s-async
2m-noslave
Note: all configurations uses ASYNC_FLUSH.
Deployment
Take the deployment of 2m-2s-sync as example. First, start up two name servers as is shown in the Quick Start section. Assume their IPs are <IP_ADDRESS> and <IP_ADDRESS>.
Then start the brokers(Assume binary RocketMQ is at /home/rocketmq/dist)
>cd /home/rocketmq/dist/bin
>bash mqbroker -c ../conf/2m-2s-sync/broker-a.properties -n <IP_ADDRESS>:9876,<IP_ADDRESS>:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-a-s.properties -n <IP_ADDRESS>:9876,<IP_ADDRESS>:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b.properties -n <IP_ADDRESS>:9876,<IP_ADDRESS>:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b-s.properties -n <IP_ADDRESS>:9876,<IP_ADDRESS>:9876
How to verify
Execute the following command to verify according to the CLI section:
> bash mqadmin clusterlist
Leave a Comment


Zen is a general-purpose programming language designed to build simple, reliable and efficient programs. When I say simple, reliable, and efficient I mean it. The syntax of Zen is designed to be clear, which means the code you write speaks for itself. Further, the architecture of the runtime is designed to run your programs reliably, regardless of the platform, with efficiency. You can develop Zen applications in nearly any environment and deploy with little to no performance loss irrespective of the platform.
As of this writing, Zen is currently under development. The alpha version of the compiler and virtual machine will be available shortly. Meanwhile, you can browse the source code at https://github.com/itssamuelrowe/zen.
I decided to create my own programming language because of my interest in compiler design since 9th grade. Also, I did not find any existing language that met all of my needs. In this article, I will explain the concepts that I feel are necessary for a dynamically typed programming language that is both enjoyable and efficient. My goal is not to build a toy language but to create a full-blown programming language. Why? Because I think it is really cool to create a programming language that everybody loves.
Simple
The syntax of Zen was intentionally designed to restrict the number of ways a certain task can be achieved. For example, unlike other dynamically typed programming languages methods and attributes cannot be introduced to an object at runtime. As a Zen programmer, you will focus more on the logic of your program rather than dealing with the convoluted state of your objects. The tokens required to write your code are minimal. With this in mind, tokens such as semicolons to terminate simple statements, braces to enclose blocks, etc. which are common in C-like programming languages were removed.
Zen offers good readability and clear syntax with an emphasis on natural language. At first glance, code written in Zen looks almost like pseudo-code. You can write code and execute it quickly to test your ideas.
For newcomers and beginners, Zen is incredibly easy to learn and use.  Zen automatically takes care of things like garbage collection, releasing resources, etc. for you.
Are you still not convinced Zen is simple? In that case, here is an example of the binary search algorithm written in Zen.
function search(values, key)
var start = 0
var end = values.size - 1
var result = null
for var i in range(0, end)
var middle = (end - start) / 2
if values[i] == key
result = i
break
else if values[i] > key
end = middle - 1
else
end = middle + 1
return result
function main(...arguments)
var values = array(53, 118, 519, 1216)
var result = search(values, 27)
if result != null
print('Found key 27 at index ' + result + '.\n')
else
print('Could not find key 27.')
Dynamically Typed
Zen is dynamically typed, which helps you avoid writing support code. However, it was designed to detect many programming errors at compile-time, where other dynamically typed languages report these errors at runtime. This provides the compiler with opportunities for optimizations. Thus, you can enjoy both the increased runtime efficiency and the developer experience.
In fact, the premise on which I started Zen was to create a programming language that was inspired by both Python and Java. The simple and clear syntax from Python was adopted to improve the developer experience. Similarly, the runtime architecture was inspired by Java to increase the runtime efficiency.
The hypothesis is that with certain restrictions on the language, both productivity and efficiency can be achieved. The restrictions imposed by the language allows the compiler to take advantage of static information available in the source code. For example, in Zen attributes and methods cannot be introduced dynamically to objects. Further, if static methods cannot be invoked via objects (like C#), the compiler can emit a single instruction to invoke static functions. Similarly, when methods invoke other methods in the same class, the compiler can simply emit an instruction to invoke the target method via the virtual method table. The same technique can be applied to attributes. Do you see what I am trying to say?
Portable: Write Once, Run Anywhere
The programs you write in Zen are compiled to bytecodes that are encoded in a special binary format known as Binary Entity Format (FEB). When you run your program, the virtual machine interprets these bytecodes. For each platform, a specific implementation of the virtual machine is required. However, once the implementation is done, your programs can run on the virtual machine seamlessly.
Lack of Implicit Idioms
Undefined values, truthy/falsy values, and many other implicit idioms are unavailable in Zen. The idea is to limit the number of ways you can express something. For example, Boolean values are always derived from relational and equality expressions. There is no implicit conversion of objects to Boolean values. This design choice, inspired by Java and C#, allows you to write readable and safe code. Personally, I believe that more bugs arise from implicit idioms than other types of errors. Additionally, such bugs are hard to track down. For example, when a programming language treats dissimilar objects as if they were the same, it leads to unexpected behaviors when the programmer attempts to use them.
Here is an example from JavaScript. Given, JavaScript did not allow default parameters before ES6 was released, it was a common pattern to simulate default parameters as shown below.
function example(value) {
value = value || 1;
}
In this example, the function updates value
to 1 when no argument is passed. However, if you tried to pass 0 as an argument, the truthy test in JavaScript would fail causing your variable to hold 1 instead of 0. Obviously, there is a better way of writing this. But we would not have this problem, to begin with, if the language did not have truthy/falsy values.
Concurrency
In CPython and MRuby, the reference implementations of Python and Ruby, respectively, a global interpreter lock (GIL) exists. The GIL protects the internals of the interpreters from race conditions that could corrupt data. In a multithreaded context, this lock allows only one thread to execute code at any given time. For example, if you have 4 threads working on a 4-core machine, only one thread and one core will be active at any given time.
In Zen, the virtual machine implements threads as first-class citizens. The various internal components (including garbage collector, JIT compiler, class loader) are thread-safe. Therefore, the global interpreter lock does not exist. This means your programs can take complete advantage of multicore systems.
Fast Compile Time
The simplistic design of Zen, both in terms of syntax and semantics, allows you to compile your source codes quickly. This stands true even though the compiler makes multiple passes. Further, the dependency management in Zen is elegant. The compiler processes only classes that are directly imported. The programs you write in Zen are compiled to a compact binary format known as Binary Entity Format (FEB). When the compiled classes are imported from your source code, the compiler makes use of a simple symbol loader which discards unnecessary information and loads only the required symbols.
Conclusion
I understand that this post was rather long. However, it presents a broad overview of some of the most important features of the Zen programming language. Many of the features discussed here are not available in Zen right now. However, this article describes the overall vision for Zen.
I believe creating Zen was worth giving a shot. At worst, I would have enjoyed three years learning to write a compiler and a virtual machine. On the other end, I would have created a programming language others appreciate. Finally, my deepest hope is that after reading this article, you will be motivated to explore Zen and find out how it might help with your own projects.
It should be noted that I am not a compiler design expert. I understand that the arguments I put forward in this post are easily beatable. In fact, I may be wrong about a few things that I mentioned. If you feel like getting in touch or would like to contribute, here is my email address<EMAIL_ADDRESS>Thank you for reading.


Looks like my troubles actually stem from a bug in the code. I summarize the problem here, just as I posted in my query to R sig phylo. Essentially the program erroneously squares alpha at the moment, explaining the pattern I found yesterday. Giving it root alpha preemptively should be a good work around.
~~ {.de1} data(bimac) tree <- with(bimac,ouchtree(node,ancestor,time/max(time),species)) print(h2 <- hansen(log(bimac[‘size’]),tree,bimac[‘OU.1’],alpha=1,sigma=1)) ~~
The print command says alpha = 0.1921554 The call h2@alpha says alpha = .4383554
which is the sqrt of the other. Certainly these should all agree. The summary command agrees with the print command:
smry = summary(h2)
smry$alpha
A little exploration seems to show that the print command is the correct one. If we want to simulate a model with a particular alpha value, we first have to generate a fitted model as above, then update the alpha value and call simulate on the fitted model. I believe ouch is accidentally squaring the value of alpha we give it before running the simulation. For example:
> h2@alpha = 5
> replicates <- as.data.frame(simulate(h2, 1000))
</syntax>
I find the expected variance by averaging the variance across the replicates,
<syntaxhighlight>
> mean(sapply(1:1000, function(i){var(replicates[i], na.rm=T)} ) )
[1] 0.0009774488
So did it use 5 or 25 as the alpha value? Since either value is large compared to the length of the tree, the tip variance should be stationary independent of tree structure so we can check it against the analytic predicted variance (sigma^2/2 alpha):
> (h2@sigma)^2/(2*h2@alpha)
[1] 0.004836469
> (h2@sigma)^2/(2*(h2@alpha)^2)
[1] 0.0009672938
As the second equation agrees with the simulation, it believe that ouch has erroneously squared the value of alpha in the simulation.
Limits to Power in Trees
Now that I’m sure I can compare apples to apples (matching stationary distributions), I can finally perform my intended comparison.
- First, confirm that the distributions generated by the models on the star tree are indistinguishable:
- Next, the same models are distinguishable on the Felsenstein Tree:
Here α = 1 and σ = 3
Intuition at the moment: The longer the two branches before the polytome in the Felsenstein tree, the more powerful the tree will be. More precisely, we can ask what is the smallest value of alpha that can be reliably distinguished from BM on this particular tree? That answer should be proportional to how close the variances of two branches can get to their stationary values, 1 − e^−\ 2αs^ where s is the branch length, 0.9 in this case, giving us 83% of the way to the equilibrium.
Compare moving the branch points from t=0.9 to t=0.45 vs halving alpha:
Both show an roughly equal reduction on the discriminatory power, and the ability to discriminate between models is lost.
- On the flip size, we can increase the power by doubling alpha from 1 to 2,
- Still need to explore the effect of simply increasing the length of time in the entire tree, doesn’t match doubling alpha. Increasing alpha also tightens the distribution, so reaching the stationary distribution along the branch isn’t enough.
Computational Comments ———————-
Have started using google style guidelines for R code. Will also be implementing google style for C.
SVN log
r214 | cboettig | 2010-03-03 18:43:28 -0800 (Wed, 03 Mar 2010) | 2 lines
final version of the day. continues tomorrow…
r213 | cboettig | 2010-03-03 17:48:33 -0800 (Wed, 03 Mar 2010) | 2 lines
figure three: weaker
r212 | cboettig | 2010-03-03 17:29:29 -0800 (Wed, 03 Mar 2010) | 2 lines
code for felsenstein_tree.R as produced the first two plots in today’s entry, about to modify for plots 3 and 4
r211 | cboettig | 2010-03-03 15:52:37 -0800 (Wed, 03 Mar 2010) | 2 lines
resolved problem discussed in yesterday’s entry, see todays entry
r210 | cboettig | 2010-03-03 01:23:14 -0800 (Wed, 03 Mar 2010) | 2 lines
Commiting version that appears in March 2nd notes


--- layout: post title: Echo java system properties date: 2011-01-02 00:24:46.000000000 +00:00 type: post parent_id: '0' published: true password: '' status: publish categories: - java tags: [] meta: _edit_last: '1' _wp_old_slug: '' author: Noel permalink: "/echo-java-system-properties/" ---
Since I'm always forgetting these two...
System.getProperty("java.library.path");
System.getProperty("java.class.path");


What a responsive site is, and why responsiveness is important.a responsive site is a website that reacts to the size of the users screen and also changes the layout of the website to ajust to make everything fit better, this is important because people have different sized screens and all websites to work on all sized screens What mobile first design is, and why it's important.i think its a website that is first designed to fit a mobile , so everything is stacked on on top of another (because of the narrow screen) its important becuse its using the idea that mobile sites are harder to create, so if you do it first, it will make everything easier after. What frameworks are, and their pros and cons.from works have pre built code and structure to make a website being built easier. pros are its quicker and a con is that it can be hard to make it unique and is a shortcut .. What a wireframe is and why we use it. is a mock up or bones of a website to help the user to design a website before its actually made. The aspects of your wireframes you found difficult to implement, and why.i initially thought the website was going to be about whatever i wanted , not a remake of my old site, so that was strange


WITNESS LIST
Economic Competitiveness, Select Committee
December 5, 2017 - 9:00 AM
Economic Competitiveness
On:
Acevedo, Art (Houston Police Department)
Allen, Chuck (North Texas Commission)
Bass, James (TxDOT)
Cagle-Hinojosa, Laura (Texas Comptroller of Public Accounts)
Daniel, Bryan (Office of the Governor)
Eastman, Anna (Self)
Gilcrease, Buck (Alvin ISD)
Guenther, Roger (Port of Houston Authority)
Keller, Andy (Meadows Mental Health Policy Institute)
LaMantia, Linda (Texas Cultural Trust)
Lloyd, Brian (Public Utility Commission)
Marek, Stan (Self; TxSIP)
Margo, Dee (City of El Paso)
McCall, Brian (The Texas State University System)
McRaven, William (The University of Texas System)
Misso, Robb (Self)
Montford, John T. (Self; JTM Consulting LLC)
Reeser, Mike (Texas State Technical College System)
Rhodes, Richard (Austin Community College)
Salinas, Gilberto (Greater Brownsville Incentives Corp.)
Sharp, John (Texas A&M University System)
Shaw, Bryan (TCEQ)
Shellebarger, Jeff (Greater Houston Partnership)
Staples, Todd (Texas Oil and Gas Association)
Vasey, Iain (Self; Corpus Christi Regional EDC)
Walker, DeAnn (Public Utility Commission)
Weber, Julie (Southwest Airlines)
Zachry, David (Zachry Corporation)
Registering, but not testifying:
On:
Cahn, Adam (Cahnman's Musings)
Eastland, Bill (Self)
McClean, Lilyanne (Greater Houston Partnership)
Rolland, Blake (Self)
1


While all organizations should be ethical and transparent in their conduct, Christian organizations should hold themselves to an even higher standard. In Titus 2:7-8 (ESV) Paul urges Titus, a Christian leader, to live this way:
Show yourself in all respects to be a model of good works, and in your teaching show integrity, dignity, and sound speech that cannot be condemned, so that an opponent may be put to shame, having nothing evil to say about us.
This is what drives us to see Christian organizations improve. We want to be above reproach not to avoid injury, but because we represent Christ.
What We Believe
We believe that the entire, canonical Bible is the actual words of God and that it is completely sufficient guidance for the Christian to live a life that honors God.
We believe that humanity was created directly by God for God's glory.
Our Goals
We want Christian organizations to be open, transparent, honest, and above reproach in their practices and partnerships. We want employees of these organizations to act in a way that reflects the character of Christ who is our model for all things. Organizations are composed of individuals, and no organization is good in and of itself. It is good when it is composed of good people.
While we call on leaders of Christian organizations to act Biblically, our primary mission is to educate all members of these organizations so that change can be effected from within.
Contract Transparency
All employees should be given standard contracts for their job description. A copy of each type of contract should be publicly available for review by employees. Ideally, salaries should also be public to ensure that employers are not taking advantage of certain workers.
Clear Employee Handbooks
Employees should know what is expected of them. Handbooks should include job descriptions for all positions, including administrative.


Kinyu Demo¶
Introduction¶
This is a repository that contains interesting demos that can be applied in the financial industry.
There are also some modules that assist the demo.
Below are some demos:
Python is an extremely dynamic and flexible language. In this demo, the goal is to show how we can use python’s path_hooks mechanism to change imports from using local filesystem to a remote source.
As a bonus, this article also demonstrates creating necessary AWS resources via Cloudformation and how to remove them with a single command. There will be some basic performance analysis of the each remote source.
Demonstrate the dependency graph heavily used in Goldman Sache (SecDB), JPMorgan (Athena), Bank of America (Quartz) and Beacon.io.
For a single swap and a portfolio, show how we could display a risk ladder. The tenors would be based on market data used to build the curves. Two curves are involved in this demo, a projection curve and a funding curve.
Kinyu Modules¶
Below are the modules used for the demos.


Links
Swagger UI:
https://{{services.swaggerUI}}
Jenkins:
https://{{jenkinsUrl}}/
PCF:
https://{{devOpsUrl}}/
VeniceGeo GitHub:
https://github.com/venicegeo
VeniceGeo Docs:
https://{{services.docs}}
Running Services
Logger:
https://{{services.loggerHost}}
Workflow:
https://{{services.workflowHost}}
Search:
https://{{services.searchHost}}
Service Controller:
https://{{services.serviceControllerHost}}
Gateway:
https://{{services.gatewayHost}}


--- layout: post title: Test Driven Developement C++ with Eclipse categories: - C++ tags: - C++ - Eclipse - Google Test Framework - GTest - TDD - Test Driven Development status: publish type: post published: true meta: _edit_last: '1' --- The first thing to do is create a new project for testing. I have a project called MyProject, and then I create a new project called MyProjectTesting. I have all the source code in a folder called src. Download the Google C++ Testing Framework at these link : http://code.google.com/p/googletest/downloads/detail?name=gtest-1.6.0.zip&can=2&q= . Make a folder into your project and name it testing_lib, and then extract the Google Testing framework with the following command (from the Google Test folder): */scripts/fuse_gtest_files.py ./testing_lib* Now the most important part, create a folder and name it imported_src. While you're creating it select advanced, and then Link to alternate location, and select the *src* folder in MyProject, like in these screenshot.
Now you've a mirror to your src folder. Select the file containing the main, and right-click. Select Exclude from build, and select all the build profiles. Now we need to tell the compiler how to link the test library with the source file. First of all create a folder in your project and name it test; then right-click on test ->properties->C/C++ Build, and then add to the Includes->Incude Path two folders: the first is the folder with the test library, that we call testing_lib, and the other id the original source file folder, called imported_src, as showed in these screenshot. Make the same thing with the testing folder, but these time link the folder with itself. These is an important step, if you don't do this, the compiler'll complain. Now in the test folder add a file called MainTest.cpp and paste the following code:include "gtest/gtest.h" int main(int argc, char **argv) { ::testing::InitGoogleTest(&argc, argv); return RUN_ALL_TESTS(); }This will launch all the tests (these step is not necessary because the testing framework has an auto-main option). After that write a first test case, let's assume that I have a class called Sin that implements a simple sin function. Now I write a simple test case using GoogleTest rules, and launch it, the result is showed here: You've done! Now you can write all the test case that you want, and Google Test Framework'll execute all the tests on your source code without modifying it.


Memory Consumption
Optimizing Memory Consumption in Deep Learning
Over the last ten years, a constant trend in deep learning is towards deeper and larger networks. Despite rapid advances in hardware performance, cutting-edge deep learning models continue to push the limits of GPU RAM. So even today, it's always desirable to find ways to train larger models while consuming less memory. Doing so enables us to train faster, using larger batch sizes, and consequently achieving a higher GPU utilization rate.
In this document, we explore techniques for optimizing memory allocation for deep neural networks. We discuss a few candidate solutions. While our proposals are by no means exhaustive, these solutions are instructive and allow us to introduce the major design issues at play.
Computation Graph
First, let's revisit the idea of the computation graph. A computation graph describes the (data flow) dependencies between the operations in the deep network. The operations performed in the graph can be either fine-grained or coarse-grained. The following figure shows two examples of computation graphs.
The concept of a computation graph is explicitly encoded in packages like Theano and CGT. In other libraries, computation graphs appear implicitly as network configuration files. The major difference in these libraries comes down to how they calculate gradients. There are mainly two ways: performing back-propagation on the same graph or explicitly representing a backwards path to calculate the required gradients.
Libraries like Caffe, CXXNet, and Torch take the former approach, performing back-prop on the original graph. Libraries like Theano and CGT take the latter approach, explicitly representing the backward path. In this discussion, we adopt the explicit backward path approach because it has several advantages for optimization.
However, we should emphasize that choosing the explicit backward path approach doesn't restrict us
to symbolic libraries, such as Theano and CGT. We can also use the explicit backward path for gradient calculation of
layer-based (which ties forward and backward together) libraries. The following graph shows how to do this.
Basically, we introduce a backward node that links to the forward node of the graph and calls the layer.backward
in the backward operations.
This discussion applies to almost all existing deep learning libraries. (There are differences between libraries, e.g., higher-order differentiation, which is beyond the scope of this topic.)
Why is the explicit backward path better? Let's explain it with two examples.
The first reason is that the explicit backward path
clearly describes the dependency between computations.
Consider the following case, where we want to get
the gradient of A and B. As we can see clearly from the graph,
the computation of the d(C)
gradient doesn't depend on F.
This means that we can free the memory of F
right after the forward computation is done.
Similarly, the memory of C
can be recycled.
Another advantage of the explicit backward path is the ability to have a different backward path, instead of a mirror of forward one. A common example is the split connection case, as shown in the following figure.
In this example, the output of B is referenced by two operations. If we want to do the gradient calculation in the same network, we need to introduce an explicit split layer. This means we need to do the split for the forward pass, too. In this figure, the forward pass doesn't contain a split layer, but the graph will automatically insert a gradient aggregation node before passing the gradient back to B. This helps us to save the memory cost of allocating the output of the split layer, and the operation cost of replicating the data in the forward pass.
If we adopt the explicit backward approach, there's no difference between the forward pass and the backward pass. We simply step through the computation graph in chronological order and carry out computations. This makes the explicit backward approach easy to analyze. We just need to answer the question: how do we allocate memory for each output node of a computation graph?
What Can Be Optimized?
As you can see, the computation graph is a useful way to discuss memory allocation optimization techniques. Already, we've shown how you can save some memory by using the explicit backward graph. Now let's explore further optimizations, and see how we might determine reasonable baselines for benchmarking.
Assume that we want to build a neural network with n
layers.
Typically, when implementing a neural network,
we need to allocate node space for both the output of each layer
and the gradient values used during back-propagation.
This means we need roughly 2 n
memory cells.
We face the same requirement when using the explicit backward graph approach
because the number of nodes in a backward pass
is roughly the same as in a forward pass.
In-place Operations
One of the simplest techniques we can employ is in-place memory sharing across operations. For neural networks, we can usually apply this technique for the operations corresponding to activation functions. Consider the following case, where we want to compute the value of three chained sigmoid functions.
Because we can compute sigmoid in-place
,
using the same memory for input and output,
we can compute an arbitrary-length chain
of sigmoid functions using constant memory.
Note: it's easy to make mistakes when implementing in-place optimization. Consider the following case, where the value of B is used not only by C, but also by F.
We can't perform in-place optimization because the value of B
is still needed after C=sigmoid(B)
is computed.
An algorithm that simply does in-place optimization
for every sigmoid operation might fall into such trap,
so we need to be careful about when we can use it.
Standard Memory Sharing
In-place operations are not the only places where we can share memory. In the following example, because the value of B is no longer needed after we compute E, we can reuse B's memory to hold the result of E.
Memory sharing doesn't necessarily require the same data shape.
Note that in the preceding example, the shapes of B
and E
can differ.
To handle such a situation, we can allocate a memory region
of size equal to the maximum of that required by B
and E
and share it between them.
Example of Real Neural Network Allocation
Of course, these are only toy examples and they address only the computation of the forward pass. But the same ideas apply to real neural networks. The following figure shows an allocation plan for a two-layer perceptron.
In this example:
- In-place optimization is applied when computing
act1
,d(fc1)
,out
andd(fc2)
. - Memory is shared between
d(act1)
andd(A)
.
Memory Allocation Algorithm
So far, we've discussed general techniques for optimizing memory allocation. We've seen that there are traps to avoid, as demonstrated in the case of in-place memory optimization. So, how can we allocate memory correctly? This is not a new problem. For example, it is very similar to the problem with register allocation in compilers. There might be techniques that we can borrow.
The key problem is that we need to place resources
so that they don't conflict with each other.
More specifically, each variable has a life time
between the time it gets computed until the last time it is used.
In the case of the multi-layer perceptron,
the life time of fc1
ends after act1
get computed.
The principle is to allow memory sharing only between variables whose lifetimes don't overlap.
There are multiple ways to do this.
You can construct the conflicting graph
with each variable as a node and link the edge
between variables with overlapping lifespans,
and then run a graph-coloring algorithm.
This likely has $O(n^2)$
complexity,
where n
is the number of nodes in the graph.
This might be too costly.
Let's consider another simple heuristic. The idea is to simulate the procedure of traversing the graph, and keep a count of future operations that depends on the node.
- An in-place optimization can be performed when only the current operation depends on the source (i.e.,
count==1
). - Memory can be recycled into the box on the upper right corner when the
count
goes to 0. - When we need new memory, we can either get it from the box or allocate a new one.
Note: During the simulation, no memory is allocated. Instead, we keep a record of how much memory each node needs, and allocate the maximum of the shared parts in the final memory plan.
Static vs. Dynamic Allocation
The preceding strategy exactly simulates
the dynamic memory allocation procedure
in imperative languages, such as Python.
The count
is the reference counter for each memory object,
and the object gets garbage collected
when the reference counter goes to 0.
In that sense,
we are simulating dynamic memory allocation once
to create a static allocation plan.
Can we simply use an imperative language
that dynamically allocates and deallocates memory?
The major difference is that static allocation is only done once, so we can afford to use more complicated algorithms. For example, we can search for memory sizes that are similar to the required memory block. The Allocation can also be made graph aware. We'll talk about that in the next section. Dynamic allocation puts more pressure on fast memory allocation and garbage collection.
There is also one takeaway for users who want to rely on dynamic memory allocations: do not unnecessarily reference objects. For example, if we organize all of the nodes in a list and store then in a Net object, these nodes will never get dereferenced, and we gain no space. Unfortunately, this is a common way to organize code.
Memory Allocation for Parallel Operations
In the previous section, we discussed how we can simulate running the procedure for a computation graph to get a static allocation plan. However, optimizing for parallel computation presents other challenges because resource sharing and parallelization are on the two ends of a balance. Let's look at the following two allocation plans for the same graph:
Both allocation plans are valid
if we run the computation serially,
from A[1]
to A[8]
.
However, the allocation plan on the left
introduces additional dependencies,
which means we can't run computation of A[2]
and A[5]
in parallel.
The plan on the right can.
To parallelize computation, we need to take greater care.
Be Correct and Safe First
Being correct is our first principle. This means to execute in a way that takes implicit dependency memory sharing into consideration. You can do this by adding the implicit dependency edge to the execution graph. Or, even simpler, if the execution engine is mutation aware, as described in our discussion of dependency engine design, push the operation in sequence and write to the same variable tag that represents the same memory region.
Always produce a safe memory allocation plan. This means never allocate the same memory to nodes that can be parallelized. This might not be ideal when memory reduction is more desirable, and we don't gain too much when we can get benefit from multiple computing streams simultaneously executing on the same GPU.
Try to Allow More Parallelization
Now we can safely perform some optimizations.
The general idea is to try and encourage memory sharing between nodes that can't be parallelized.
You can do this by creating an ancestor relationship
graph and querying it during allocation,
which costs approximately $O(n^2)$
in time to construct.
We can also use a heuristic here,
for example, color the path in the graph.
As shown in the following figure,
when you try to find the longest paths in the graph,
color them the same color and continue.
After you get the color of the node,
you allow sharing (or encourage sharing)
only between nodes of the same color.
This is a stricter version of the ancestor relationship,
but it costs only $O(n)$
of time
if you search for only the first k
path.
This is by no means the only solution. More sophisticated approaches might exist:
How Much Can you Save?
We've discussed the techniques and algorithms you can use to squeeze memory usage for deep learning. How much can you really save by using these techniques?
On coarse-grained operation graphs that are already optimized for big operations, you can reduce memory consumption roughly by half. You can reduce memory usage even more if you are optimizing a fine-grained computation network used by symbolic libraries, such as Theano. Most of the ideas in this article inspired the design of MXNet.
Also, you will notice that memory cost, for forward pass only execution, is extremely low compared to running both forward and backward pass. This is simply because there's more memory reuse if you run only the forward pass.
So here are two takeaways:
- Use a computation graph to allocate memory.
- For deep learning models, prediction consumes much less memory than training.


'br}
'br}
{\
ptar - a tar-like program written in perl
ptar is a small, tar look-alike program that uses the perl module Archive::Tar to extract, create and list tar archives.
ptar -c [-v] [-z] [-C] [-f ARCHIVE_FILE | -] FILE FILE ... ptar -c [-v] [-z] [-C] [-T index | -] [-f ARCHIVE_FILE | -] ptar -x [-v] [-z] [-f ARCHIVE_FILE | -] ptar -t [-z] [-f ARCHIVE_FILE | -] ptar -h
c Create ARCHIVE_FILE or STDOUT (-) from FILE x Extract from ARCHIVE_FILE or STDIN (-) t List the contents of ARCHIVE_FILE or STDIN (-) f Name of the ARCHIVE_FILE to use. Default is *(Aq./default.tar*(Aq z Read/Write zlib compressed ARCHIVE_FILE (not always available) v Print filenames as they are added or extracted from ARCHIVE_FILE h Prints this help message C CPAN mode - drop 022 from permissions T get names to create from file
tar(1), L<Archive::Tar>.


Immersive Experiences Lab, HP Inc.
Summer 2016
Questionnaire, interview, emergent coding, thematic analysis
Understand the pen-based digital workflows of creative professionals in order to design a disruptive product.
The Immersive Experiences Lab (IXL), led by Dr. Mirjana Spasojevic, studies and anticipates the ways in which people will engage with technologies that don’t yet exist. Using a people-first approach, IXL conducts frequent rounds of requirements gathering, prototyping and usability testing to deliver insights in areas as diverse as blended reality, authentic user experiences, and shaping future interactions with smart devices.
During a 3-month internship at IXL at HP Inc's Palo Alto headquarters, I worked with Senior Manager Dr. Alex Thayer on an exploration of the workflows and requirements of design professionals in architecture, industrial design, illustration, graphic design, and user experience design.
IXL was approached by an HP Inc. business unit which was exploring applications for a new technology that could improve the working lives of designers and creative thinkers. I performed a review of their recent developments and past research in the area, compiled research questions that would help their exploration, and devised a two-stage research plan.
20 respondents + 4 semi-structured follow-up interviews
In the first stage, I deployed an online questionnaire to internal HP Inc. employees in designer and designer-adjacent roles. Of the ~20 respondents, I identified notable participants and approached them for follow-up interviews. Four semi-structured interviews were conducted over video conferencing, yielding additional rich details on design workflows and barriers in a corporate setting. One of the key respondents was a manager of designers at the director level, and their contribution was invaluable in understanding the specific challenges faced by larger design teams.
11 external design professionals
To understand independent designers who were not part of a larger organization, in the second stage of my research I sought out freelancers, students and part-time designers across disciplines. I conducted 11 semi-structured interviews with design professionals in a neutral space in downtown San Francisco. They were asked to bring samples of their work and the design tools they used the most.
After interviews were transcribed, I performed emergent coding on a subset, yielding a candidate list of themes touched on by our respondents. The rest of the interviews were then analyzed using thematic analysis. Over 15 high-level themes emerged and developed in the course of the analysis. Finally, I compiled a report with the most relevant insights. I presented my findings to the IXL team, and was asked to repeat the presentation for 2 other business teams and their executive stakeholders.
Regrettably, detailed results from this study are under a Non-Disclosure Agreement.


Section 1 - Essential Scripts¶
Section 1.1 - Argparse Example¶
Example for setting up arguments for your command line utility.
Example Usage:
$ python argparse.py
References:
Argparse configuration¶
This function shows an example of creating an argparse instance
with required and optional parameters. Further, it demonstrates
how to set default values and boolean arguments. the argparse
module has many more features documented at
https://docs.python.org/3/library/argparse.html
def setup_argparse():
# Setup a parser instance with common fields including a
# description and epilog. The `formatter_class` instructs
# argparse to show default values set for parameters.
parser = argparse.ArgumentParser(
description='Sample Argparse',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
epilog=f"Built by {__author__}, v.{__date__}"
)
# The simplest form of adding an argument, the name of the
# parameter and a description of its form.
parser.add_argument('INPUT_FILE', help="Input file to parse")
parser.add_argument('OUTPUT_FOLDER',
help="Folder to store output")
# An optional argument with multiple methods of specifying
# the parameter. Includes a default value
parser.add_argument('-l', '--log', help="Path to log file",
default=os.path.abspath(os.path.join(
PurePath(__file__).parent,
PurePath(__file__).name.rsplit('.', 1)[0] + '.log'))
)
# An optional argument which does not accept a value, instead
# just modifies functionality.
parser.add_argument('-v', '--verbose', action='store_true',
help='Include debug log messages')
# Once we've specified our arguments we can parse them for
# reference
args = parser.parse_args()
# Returning our parsed arguments for further use.
return args
Section 1.2 - Logging Example¶
Example for writing logging information to the console and a log file.
Example Usage:
$ python logging_example.py
References:
Logging configuration¶
This function shows an example of creating a logging instance that writes messages to both STDERR and a file, allowing your script to write content to STDOUT uninterrupted. Additionally, you can set different logging levels for the two handlers - generally you keep debugging information in the log file while writing more critical messages to the console in STDERR.
def setup_logging():
"""Function to setup logging configuration and test it."""
# Allow us to modify the `logger` variable within a function
global logger
# Set logger object, uses module's name
logger = logging.getLogger(name=__name__)
# Set default logger level to DEBUG. You can change this later
logger.setLevel(logging.DEBUG)
# Logging formatter. Best to keep consistent for most usecases
log_format = logging.Formatter(
'%(asctime)s %(filename)s %(levelname)s %(module)s '
'%(funcName)s %(lineno)d %(message)s')
# Setup STDERR logging, allowing you uninterrupted
# STDOUT redirection
stderr_handle = logging.StreamHandler(stream=sys.stderr)
stderr_handle.setLevel(logging.INFO)
stderr_handle.setFormatter(log_format)
# Setup file logging
file_handle = logging.FileHandler('sample.log', 'a')
file_handle.setLevel(logging.DEBUG)
file_handle.setFormatter(log_format)
# Add handles
logger.addHandler(stderr_handle)
logger.addHandler(file_handle)
# Sample log messages
logger.debug("This is a debug message")
logger.info("This is an info message")
logger.warning("This is a warning message")
logger.error("This is a error message")
logger.critical("This is a critical message")
def sample_function():
"""Sample function to demonstrate logging formatting."""
logger.info("Called from a function")
sample_function()
Section 1.3 - Open Files¶
Example for reading data from encoded text files.
Demonstrates how to handle setting the proper encoding for UTF-8, UTF-16-LE, and UTF-16-BE with the ability to easily expand to support checking other file magic values/signatures.
Example Usage:
$ python open_files.py
References:
Open files with proper encoding¶
This first function shows an example of opening a file after checking for a byte-order mark (BOM). While this method could be expanded to check for a file’s magic value/file signature, this low-tech method will help with parsing a collection of files that may be UTF-8, UTF-16-LE, and UTF-16-BE, three very common text file encodings. Feel free to build and share on this.
def open_file(input_file):
"""Opens an encoded text file and prints the contents
Arguments:
input_file (str): Path to file to open
"""
test_encoding = open(input_file, 'rb')
bom = test_encoding.read(2)
file_encoding = 'utf-8'
if bom == b'FEFF':
file_encoding = 'utf-16-le'
elif bom == b'FFFE':
file_encoding = 'utf-16-be'
with open(input_file, 'r', encoding=file_encoding) as fopen:
for raw_line in fopen:
line = raw_line.strip()
print(line)
Section 1.4 - CSV Example¶
Example for writing datasets into CSV files.
Demonstrates source datasets comprised of lists of dictionaries and lists of lists as separate functions. Example data is provided in line and will generate two identical CSVs as output.
Example Usage:
$ python csv_example.py
References:
List of dictionaries to CSV¶
Example data
variable:
[
{'name': 'apple', 'quantity': 10, 'location': 'VT'},
{'name': 'orange', 'quantity': 5, 'location': 'FL'}
]
This first function shows an example of writing a list containing multiple dictionaries to a CSV file. You can optionally provide an ordered list of headers to filter what rows to show, or let the function use the keys of the first dictionary in the list to generate the header information. The latter option may produce a new order each iteration and is not prefered if you can determine the headers in advance.
def write_csv_dicts(outfile, data, headers=None):
"""Writes a list of dictionaries to a CSV file.
Arguments:
outfile (str): Path to output file
data (list): List of dictionaries to write to file
headers (list): Header row to use. If empty, will use the
first dictionary in the `data` list.
"""
if not headers:
# Use the first line of data
headers = [str(x) for x in data[0].keys()]
with open(outfile, 'w', newline="") as open_file:
# Write only provided headers, ignore others
csvfile = csv.DictWriter(open_file, headers,
extrasaction='ignore')
csvfile.writeheader()
csvfile.writerows(data)
List of ordered lists to CSV¶
Example data
variable:
[
['name', 'quantity', 'location'],
['apple', 10, 'VT'],
['orange', 5, 'FL']
]
This function shows an example of writing a list containing multiple lists to a CSV file. You can optionally provide an ordered list of headers, or let the function use the values of the first element in the list to generate the header information. Unlike the dictionary option, you cannot filter column data by adjusting the provided headers, you must write all columns to the CSV.
def write_csv_lists(outfile, data, headers=None):
"""Writes a list of lists to a CSV file.
Arguments:
outfile (str): Path to output file
data (list): List of lists to write to file
headers (list): Header row to use. If empty, will use the
first list in the `data` list.
"""
with open(outfile, 'w', newline="") as open_file:
# Write only provided headers, ignore others
csvfile = csv.writer(open_file)
for count, entry in enumerate(data):
if count == 0:
# If headers are defined, write them, otherwise
# continue as they will be written anyways
if headers:
csvfile.writerow(headers)
csvfile.writerow(entry)
Docstring References¶
-
write_csv_dicts
(outfile, data, headers=None)¶ Writes a list of dictionaries to a CSV file.
- Parameters
outfile (str) – Path to output file
data (list) – List of dictionaries to write to file
headers (list) – Header row to use. If empty, will use the first dictionary in the data list.
-
write_csv_lists
(outfile, data, headers=None)¶ Writes a list of lists to a CSV file.
- Parameters
outfile (str) – Path to output file
data (list) – List of lists to write to file
headers (list) – Header row to use. If empty, will use the first list in the data list.
Section 1.5 - Directory Recursion¶
File recursion example.
Demonstration of iterating through a directory to interact with files.
Example Usage:
$ python recursion_example.py
References:
List a directory¶
This function shows an example of displaying all files and folders within a single directory. From here you can further interact with individual files and folders or iterate recursively by calling the function on identified subdirectories.
def list_directory(path):
"""List all file and folder entries in `path`."""
print(f"Files and folders in '{os.path.abspath(path)}':")
# Quick and easy method for listing items within a single
# folder.
for entry in os.listdir(path):
# Print all entry names
print(f"\t{entry}")
List a directory recursively¶
This function shows an example of displaying all files and
folders within a all directories. You don’t need to worry about
additional function calls as the os.walk()
method handles
the recursion on subdirectories and your logic can focus on
handling the processing of files. This sample shows a method of
counting the number of files, subdirectories, and files ending in
“.py” as an example.
def iterate_files(path):
# Though `os.walk()` exposes a list of directories in the
# current `root`, it is rarely used since we are generally
# interested in the files found within the subdirectories.
# For this reason, it is common to see `dirs` named `_`.
# DO NOT NAME `dirs` as `dir` since `dir` is a reserved word!
for root, dirs, files in os.walk(os.path.abspath(path)):
# Both `dirs` and `files` are lists containing all entries
# at the current `root`.
for fentry in files:
# To effectively reference a file, you should include
# the below line which creates a full path reference
# to the specific file, regardless of how nested it is
file_entry = os.path.join(root, fentry)
# We can then hand `file_entry` off to other functions.
yield file_entry
-
list_directory
(path)¶ List all file and folder entries in path.


I plan to make the blog more visually appealing, but clearly defining the header, footer, content, and sidebar. This week I further my knowledge of absolute, relative, and static position which will help with this layout. I'll also seek to incorporate a responsive design for this template. Below is my mockup of the site


Modern application architecture and design practices recommend the concept of "composition" (of existing modules and components) in the assembly of an overall application, to maximize the resuse of previously built functionality. However, it is still generally necessary to write original code that "glues" the components together. It would be useful if individuals familiar with the business requirements of their systems could create "scripts" to represent the linkages between components, rather than requiring Java code. The Workflow proposal intends to create support for just such a capability.
Scripted workflows is an idea that can be used in many different application environments and use cases. In particular, a highly desired use is in managing the interaction with web application users, where the transaction being performed requires more than one HTML page to be displayed. However, the Workflow implementation will be strictly layered so that the underlying scripting engine can be used without any reliance on web layer APIs
The Workflow project shall create and maintain an engine to manage and execute scripts, codified in XML. The basic engine shall be environment independent, but specialized implementations of the basic engine can adapt the engine to that environment.
One specialized implementation will be included in the Workflow package itself - an adapter to integrate the workflow engine into the web layer, by mapping the Workflow concept of bean scopes into the scopes provided by the Servlet API (request, session, and application). This implementation will be independent of any particular web application framework - it is expected that frameworks will further specialize the web layer implementation in order to integrate the framework's own capabilities and features.
Scripts for the workflow engine will be codified in XML. It shall be possible for specialized implementations to register their own sets of custom Step implementations, under a specific XML namespace, to transparently extend the set of built in Step implementations that is available to workflow engine users.
Digester relies on:
This package represents a new approach to the workflow management process, so no initial codebase has been identified.
The proposed package name for the new component is
org.apache.commons.workflow
, with subpackages for implementations
of the basic interfaces.
workflow
in the
jakarta-commons
SVN repository.The initial committers on the Workflow component shall be:


Any questions or concerns?
We'd love to hear from you! You can contact us through our website, or on social media, or at our email.
Lettuce know if you have any problems
You can contact us with anything related to FoodSpan. We'll get in touch with you as soon as possible.


Stereotype Threat
July 26, 2015
Stereotype threat is the expectation of judgment from others that leads people from a group that suffers from negative stereotypes to perform below their ability.
Our self-image is molded by the way others perceive us, so judgments and expectations placed upon us by others can be internalized. When these judgments are negative, they can undermine us, causing us to subconsciously sabotage our performance even when we're capable of more.
In trying to recall a personal experience of stereotype threat, I'm remembering several instances in the past when I've been in spaces where men were dominating the discussion. The desire to speak up began to recede as I subconsciously conformed to the stereotype that women are quiet and have nothing to say. At that point, even if I wanted to correct a factual inaccuracy, I might second-guess myself and keep quiet.
To reduce stereotype threat, a safe space must be maintained. People should feel respected as individuals and evaluated according to their abilities. One article on stereotype threat mentioned that for one group of college students, weekly discussions among students of different races, from different backgrounds, helped to ease feelings of stereotype threat. When members of one group could hear members of another expressing the same kind of anxieties they had, their concerns at being profiled according to a stereotype were lessened.
The discussion sessions at Dev Bootcamp will probably have a similar function. Already, the experience of seeing other students write about their anxieties on an email thread has let us see that everyone is worrying about the same things. Encouraging everyone to speak up and feel like their perspective is valued will be an important part of building trust in each other in order to learn at our maximum potential.


Welcome! These are some bonus puzzles to accompany the Intro to JavaScript class. Please take a look at the puzzles, and work on one that looks challenging but not completely overwhelming. You can always reference the slides if you get stuck. Commit to spend at least 20 minutes trying to figure out a problem before you look at the answers. It helps you learn!
Class 1
The Fortune Teller
Why pay a fortune teller when you can just program your fortune yourself?
- Store the following into variables: number of children, partner's name, geographic location, job title.
- Output your fortune to the screen like so: "You will be a X in Y, and married to Z with N kids."
var numKids = 5; var partner = 'David Beckham'; var geolocation = 'Costa Rica'; var jobTitle = 'web developer'; var future = 'You will be a ' + jobTitle + ' in ' + geolocation + ', and married to ' + partner + ' with ' + numKids + ' kids.'; console.log(future);
The Age Calculator
Forgot how old someone is? Calculate it!
- Store the current year in a variable.
- Store their birth year in a variable.
- Calculate their 2 possible ages based on the stored values.
- Output them to the screen like so: "They are either NN or NN", substituting the values.
var birthYear = 1984; var currentYear = 2016; var age = currentYear - birthYear; console.log('They are either ' + age + ' or ' + (age - 1));
The Temperature Converter
Let's make a program to convert celsius tempatures to fahrenheit and vice versa, so we can complain about the weather with our friends oversees.
Reminder: To convert celsius to fahrenheit, multiply by 9, then divide by 5, then add 32. To convert fahrenheit to celsius, deduct 32, then multiply by 5, then divide by 9.
Unicode Characters: To print the degrees symbol in JavaScript, we need to use the write \u00B0
to represent the unicode character for the degress symbol.
- Store a celsius temperature into a variable.
- Convert it to fahrenheit and output "NN°C is NN°F".
- Now store a fahrenheit temperature into a variable.
- Convert it to celsius and output "NN°F is NN°C."
var inputCelsius = 25; var outputFahrenheit = (inputCelsius * 9)/5 + 32; console.log(inputCelsius + '\u00B0 Celsious is ' + outputFahrenheit + '\u00B0 Fahrenheit'); var inputFahrenheit = 73; var outputCelsius = (inputFahrenheit - 32)/9 * 5; console.log(inputFahrenheit + '\u00B0 Fahrenheit is ' + outputCelsius + '\u00B0 Celsious.');
Challenge Question: Using Math functions
JavaScript has a built-in tool that can generate a random number between 0 and 1. This tool is called a method. We'll talk about methods more later in the class. For now, know if you type Math.random()
, you'll get a random number between 0 and 1.
Using this tool, update your fahrenheit to celsius tempature conversion program to do the following:
- Instead of inputing a value for the Fahrenheit tempature, use Math.random() to generate a random tempature between 0 and 100
- Have to program output: "It is NN°F today. That's NN°C."
var inputFahrenheit = Math.random()*100; var outputCelsius = (inputFahrenheit - 32)/9 * 5; console.log('It is ' + inputFahrenheit + '\u00B0 Fahrenheit today. That\'s ' + outputCelsius + '\u00B0 Celsius.');
Class 2
The Fortune Teller: With Functions!
Let's turn one of the Class 1 Exercises into a function.
- Write a function named
tellFortune
that:- takes 4 arguments: number of children, partner's name, geographic location, job title.
- outputs your fortune to the screen like so: "You will be a X in Y, and married to Z with N kids."
- Call that function 3 times with 3 different values for the arguments.
function tellFortune(jobTitle, geolocation, partner, numKids) { var future = 'You will be a ' + jobTitle + ' in ' + geolocation + ' and married to ' + partner + ' with ' + numKids + ' kids.'; console.log(future); } tellFortune('astronaut', 'Spain', 'Shaq', 3); tellFortune('stunt double', 'Japan', 'Bruce Wayne', 9); tellFortune('Elvis impersonator', 'Russia', 'Elon Musk', 0);
What number is bigger?
Write a function that compares two numbers and returns the larger one. Be sure to try it out with some different numbers. Bonus: add error messages if the numbers are equal or cannot be compared.
Don't forget to test it!
function greaterNum(num1, num2) { if (num1 === num2) { console.log('Those numbers are equal'); } else if (num1 > num2) { return num1; } else if (num2 > num1) { return num2; } else { console.log('Those are simply incomparable!'); return undefined; } } console.log(greaterNum(3, 3)); console.log(greaterNum(7, -2)); console.log(greaterNum(5, 9)); console.log(greaterNum(5, 'dog'));
Pluralize
Write a function pluralize
that takes in two arguments, a number and a word, and returns the plural. For example:
pluralize(5, 'cat'): '5 cats' pluralize(7, 'turtle'): '7 turtles'
Bonus: Make it handle a few collective nouns like "sheep" and "geese".
function pluralize(number, noun) { if (number != 1 && noun != 'sheep' && noun != 'geese') { return number + ' ' + noun + 's'; } else { return number + ' ' + noun; } } console.log('I have ' + pluralize(0, 'cat')); console.log('I have ' + pluralize(1, 'cat')); console.log('I have ' + pluralize(2, 'cat')); console.log('I have ' + pluralize(3, 'geese'));
The Calculator
- Write a function called
squareNumber
that will take one argument (a number), square that number, and return the result. It should also log a string like "The result of squaring the number 3 is 9." - Write a function called
halfNumber
that will take one argument (a number), divide it by 2, and return the result. It should also log a string like "Half of 5 is 2.5.". - Write a function called
percentOf
that will take two numbers, figure out what percent the first number represents of the second number, and return the result. It should also log a string like "2 is 50% of 4." - Write a function called
areaOfCircle
that will take one argument (the radius), calculate the area based on that, and return the result. It should also log a string like "The area for a circle with radius 2 is 12.566370614359172."- Bonus: Round the result so there are only two digits after the decimal.
- Write a function that will take one argument (a number) and perform the following operations, using the functions
you wrote earlier:
- Take half of the number and store the result.
- Square the result of #1 and store that result.
- Calculate the area of a circle with the result of #2 as the radius.
- Calculate what percentage that area is of the squared result (#3).
function squareNumber(num) { var squaredNumber = num * num; console.log('The result of squaring the number ' + num + ' is ' + squaredNumber); return squaredNumber; } squareNumber(3); function halfOf(num) { var half = num / 2; console.log('Half of ' + num + ' is ' + half); return half; } halfOf(5); function percentOf(num1, num2) { var percent = (num1/num2) * 100; console.log(num1 + ' is ' + percent + '% of ' + num2); return percent; } percentOf(5, 10); function areaOfCircle(radius) { var area = Math.PI * squareNumber(radius); console.log('The area of circle with radius ' + radius + ' is ' + area); return area; } areaOfCircle(2); function doCrazyStuff(num) { var half = halfOf(num); var squared = squareNumber(half); var area = areaOfCircle(squared); var result = percentOf(squared, area); } doCrazyStuff(5);
Challenge Question: String Manipulation
If you feel comfortable with the other exercises, it's time to expand your knowledge! These puzzles involve manipulating strings; to try them out, you'll need to use some of the built-in JavaScript methods for strings. Methods are pre-written functions that are built into the language.
These questions are not as straightforward as the others. They challenge you to really think like a programmer. Really try to solve them before you peek at the answer.
MixUp
Create a function called mixUp
. It should take in two strings, and return the concatenation of the two strings (separated by a space) slicing out and swapping the first 2 characters of each. You can assume that the strings are at least 2 characters long. For example:
mixUp('mix', 'pod'): 'pox mid' mixUp('dog', 'dinner'): 'dig donner'
//This function uses the slice() method. It extracts a part of a string and returns a new string function mixUp(string1, string2) { return string2.slice(0, 2) + string1.slice(2) + " " + string1.slice(0, 2) + string2.slice(2); } console.log(mixUp('mix', 'pod')); console.log(mixUp('dog', 'dinner'));
FixStart
Create a function called fixStart
. It should take a single argument, a string, and return a version where all occurrences of its first character have been replaced with '*', except for the first character itself. You can assume that the string is at least one character long. For example:
fixStart('babble'): 'ba**le' fixStart('turtle'): 'tur*le'
//This function uses a few new methods and regular expressions function fixStart(inputString) { var firstChar = inputString.charAt(0); return firstChar + inputString.slice(1).replace(new RegExp(firstChar, 'g'), '*'); } console.log(fixStart('babble')); console.log(fixStart('turtle'));
Class 3
Even/Odd Counter
Write a for loop that will iterate from 0 to 20. For each iteration, it will check if the current number is even or odd, and report that to the screen (e.g. "2 is even")
for (var i = 0; i <= 20; i++) { if (i % 2 === 0) { console.log(i + ' is even'); } else { console.log(i + ' is odd'); } }
Top Choice
Create an array to hold your top choices (colors, presidents, whatever). For each choice, log to the screen a string like: "My #1 choice is blue."
Bonus: Change it to log "My 1st choice, "My 2nd choice", "My 3rd choice", picking the right suffix for the number based on what it is. The method slice might be helpful here.
//Simple code var choices = ['red', 'orange', 'pink', 'yellow']; for (var i = 0; i < choices.length; i++) { console.log('My #' + (i + 1) + ' choice is ' + choices[i]); } //Bonus solution var choices = ['red', 'orange', 'pink', 'yellow']; function findLast (inputNum) { tmpSlice = String(inputNum); var lastDigit = (tmpSlice.slice(-1)); return lastDigit; } for (var i = 0; i < choices.length; i++) { var choiceNum = i + 1; var choiceNumSuffix = 0; var choiceNumLast = findLast(choiceNum); if (choiceNumLast == 1) { choiceNumSuffix = 'st'; } else if (choiceNumLast == 2) { choiceNumSuffix = 'nd'; } else if (choiceNumLast == 3) { choiceNumSuffix = 'rd'; } else { choiceNumSuffix = 'th'; } console.log('My ' + choiceNum + choiceNumSuffix + ' choice is ' + choices[i]); }
The Reading List
Keep track of which books you read and which books you want to read!
- Create an array of objects, where each object describes a book and has properties for the title (a string), author (a string), and alreadyRead (a boolean indicating if you read it yet).
- Iterate through the array of books. For each book, log the book title and book author like so: "The Hobbit by J.R.R. Tolkien".
- Now use an if/else statement to change the output depending on whether you read it yet or not. If you read it, log a string like 'You already read "The Hobbit" by J.R.R. Tolkien', and if not, log a string like 'You still need to read "The Lord of the Rings" by J.R.R. Tolkien.'
var books = [ { title: 'You Don'\t Know JS', author: 'Kyle Simpson', alreadyRead: false }, { title: 'Eloquent JavaScript', author: 'Marijn Haverbeke', alreadyRead: true } ]; for (var i = 0; i < books.length; i++) { var book = books[i]; var bookInfo = book.title + '" by ' + book.author; if (book.alreadyRead) { console.log('You already read "' + bookInfo); } else { console.log('You still need to read "' + bookInfo); } }
Class 4
Logo Hijack
- Open up www.google.com in Chrome or Firefox, and open up the console.
- Find the Google logo and store it in a variable.
- Modify the source of the logo IMG so that it's a Yahoo logo instead. (http://www.logostage.com/logos/yahoo.GIF)
var img = document.getElementById('hplogo'); img.width = 400; img.src = 'http://www.logostage.com/logos/yahoo.GIF';
About Me
Start with this HTML and save it as "aboutme.html":
<!DOCTYPE html>   <meta charset="utf-8"/> <title>About Me</title>
About Me

Nickname:

Favorites:

Hometown:

- Add a script tag to the bottom.
- (In JavaScript) Change the body tag's style so it has a font-family of "Arial, sans-serif".
- (In JavaScript) Replace the content of each of the spans (nickname, favorites, hometown) with your own information.
- Iterate through each li and add a class of "listitem". Add a style tag that sets a rule for "listitem" to make the color red.
- Create a new img element and set its src attribute to a picture of you. Append that element to the body.
<!DOCTYPE html>   <meta charset="utf-8"/> <title>About Me</title>
About Me

Nickname:

Favorites:

Hometown:

The Reading List Part II
- Create a webpage with an
h1
of "My Book List". - Add a script tag to the bottom of the page, where all your JavaScript will go.
- Copy the array of books from the previous exercise.
- Iterate through the array of books. For each book, create a
p
element with the book title and author and append it to the page. - Bonus: Use a
ul
andli
to display the books. - Bonus: add a property to each book with the URL of the book cover, and add an
img
element for each book on the page. - Bonus: Change the style of the book depending on whether you have read it or not.
<!DOCTYPE html>   <meta charset="utf-8"/> <title>Book List</title>
My Book List

Challenge Question: The Counter
Write a function that takes a certain type of tag and counts the number of elements with that tag. The function should return "There a X tags of type y on the page. For example:
countTags('p'): 'There are 3 tags of type p on the page'
function countTags (tagType) { var tagArray = document.getElementsByTagName(tagType); console.log('There are ' + tagArray.length + ' tags of type ' + tagType + ' on the page.'); }


?'
ˇ
17
The
New
York
copyright
reserved
botanical
Garden
PéRE
DUSS,
HERBIER
DE
LA
MARTINIQUE
clu^nb"wuA^>
fal-lpcu/,
nro-A.
York
Botanical
Garden
VELVA
E.
RUDD.
U.
S.
National
Museum
lbl5754
1615754
Legend -
Level of confidence that token is an accurately-transcribed word
extremely low
very low
low
undetermined
medium
high
very high


My first published research looked at bovine tubeculosis in cattle and widlife in New Zealand. Take a look at it here.
We used whole genome sequencing to trace the transmission of Mycobacterium bovis, a bacteria that causes bovine tuberculosis, between cattle and wildlife.
In New Zealand, the brushtail possum (see my impression below 😳) is a pest species - originally imported for the fur industry. There are 30 million possums in New Zealand and they can be infected by the M. bovis bacteria. Their infection can spread into cattle, especially where cattle live close to the bush (dense forested areas), where possums live.
Our research showed that the M. bovis bacteria was frequently being transmitted between cattle and wildlife populations in New Zealand. Given that there were less than 50 infected cattle herds in New Zealand at the time, this suggested that the possum population were acting as the main source of infection.
For me, this research was an amazing opportunity to work some incredible researchers in New Zealand - as well as meet them in person! Marian Price-Carter led this research project and continues to amazing work in the bovine tuberculosis program in New Zealand.
Have a look at the research here and let me know what you think.


Patch number: 1
Delete character coordinates: 2,24
Insert character coordinates: 2,40
p>start
¶

new·line
¶

line·with
Patch number: 2
Delete character coordinates: 40,16
Insert character coordinates: 40,25
NOTE: Patch coordinates are based on patch 1 being applied
th·some·
modified·
content<
Patch number: 3
Delete character coordinates: 83,32
Insert character coordinates: 83,53
NOTE: Patch coordinates are based on patch 2 being applied
ome·more·content
»that·has·a·tab·added

¶

line·wit
Patch number: 4
Delete character coordinates: 135,17
Insert character coordinates: 135,16
NOTE: Patch coordinates are based on patch 3 being applied
NOTE: Patch contains whitespace only differences
th·some·
»
whitespa
Patch number: 5
Delete character coordinates: 145,17
Insert character coordinates: 145,16
NOTE: Patch coordinates are based on patch 4 being applied
NOTE: Patch contains whitespace only differences
itespace
»
·content
Patch number: 6
Delete character coordinates: 187,28
Insert character coordinates: 187,8
NOTE: Patch coordinates are based on patch 5 being applied
/p>¶

removed·line
¶

e


This graph shows mean daily temperature and average relative humidity in {{ city_name }}.
{{ city_climate }}
{% if session['logged_in'] %}
Want to add more data?

Please login first.
{% endif %}
Back to the main page


This projet analyzes how weather changes in cities that are closer or further from the equator. The purpose of the anaylysis was to prove that cities closer to the equator wound generally have warmer temperatures. Although this may be quite obvious- the data in this study "proves" this statement. The project compares different weather factors- temperature, humidity, cloudiness and wind speed- in over 500 randomly selected cities.
This analysis was performed using data from the OpenWeatherMap API. After assesmbling the dataset, Matplotlib was used to plot the different weather factors against the latitude. This site provides the source data and visualizations created as part of the analysis, as well as explanations and descriptions of any trends and correlations witnessed.


TinyPilot is currently using MJPEG H.264 compression to stream the video output of your target device.
You can configure the streaming mode in the “System” > “Video Settings” dialog.
TinyPilot failed to stream video output from your target device using H.264 compression.
See “System” > “Logs” for diagnostic details.
Refreshing the page or restarting your device may fix the issue.


Interested in sponsoring an autonomous soapbox car and supporting STEM education? Email<EMAIL_ADDRESS>5 November 2019
Brighten Academy Dominates Field at this Year's Georgia Gravity Games
DOUGLASVILLE, Georgia, Nov. 5, 2019 /Douglas County Sentinel/ --
More than 65 student-led teams from schools across the state competed Saturday in the 6th annual Georgia Gravity Games, a gravity-only car race competition sponsored by Google, Georgia Tech, and the City of Douglasville...
Read More
all work © Mahdi Al-Husseini 2014 - 2019 • all rights reserved • to license images, get in touch


The Government had sold the 'Beatrice' to the Commercial Rowing Club (Brisbane Courier 27/07/1896:4). The 'Beatrice' foundered at its moorings at the Commercial Rowing Club in late July 1896 (Queenslander 01/08/1896:234). The vessel had been moored close to the north bank of the river, just below the bridge (Brisbane Courier 27/07/1896:4). The Marine Department later advertised in November 1896 for tenders from persons willing to undertake the breaking up and removal of the hulk of the 'Beatrice' as it lay below the Victoria Bridge (Brisbane Courier 18/11/1896:8). According to Stone (2006:90) the vessel was raised and broken up.


Preparation time
overnight
Cooking time
30 mins to 1 hour
Serves
Serves 12
Dietary
Warm buttery brioche, laced with homemade fudge and butterscotch, topped with creamy frosting.
For the brioche, heat the milk to 37C/99F, pour it into a measuring jug and add the sachet of yeast. Cover with cling film and set aside for 10 minutes.
Beat the eggs and add the sugar and vanilla paste. Sieve the flour into a large bowl and add the salt and nutmeg. Make a well in the flour and pour in the milk mixture. Use your hands to bring the dough together - it will be very sticky and wet. Attach a dough hook to a freestanding mixer, add the dough and begin to knead.
In a separate bowl beat the cinnamon into the butter until combined, then add to the dough mix as it is being kneaded. When all the butter has been incorporated into the dough, flour the work surface and knead the dough by hand until it is smooth, soft and elastic. Transfer to an oiled bowl, cover with cling film and refrigerate overnight.
For the butterscotch, put a pan over a medium heat. Add the sugar, cream, vanilla, cream of tartar and 75ml/2½fl oz water. Stir and cook the butterscotch until it reaches 114C (use a sugar thermometer for this), add the butter and keep checking the temperature. When the syrup reaches 143C pour it into the prepared tin and set aside to cool and harden.
For the fudge, grease and line a second small tin. In a pan set over a medium heat combine the milk, butter, vanilla, sugar and salt. Stir until the sugar dissolves. Cover with a tightly fitting lid and cook for a further two minutes. Remove the lid and continue to cook at a rolling boil for 10 minutes until it reaches 114C (use a sugar thermometer), then transfer the pan to a bowl filled with cold water. Leave the pan in a cool place until it is lukewarm (or 43C) - do not stir. When the fudge has cooled, use a wooden spoon to beat it until it is thick, smooth and creamy. Pour into the prepared tin and set aside in a cool place to harden.
For the cream cheese frosting, whisk the butter and cream cheese until smooth and light. Stir in the icing sugar until incorporated, then briefly whisk again until light and smooth (take care not to over whisk the frosting as it will become runny). Set aside until ready to serve.
Preheat the oven to 200C/400F/Gas 6 and grease and line a small square baking tin. When the butterscotch and fudge have hardened break them into small pieces.
Remove the dough from the fridge, knead the fudge and butterscotch pieces into the dough, and divide into 12 equally-sized balls.
Grease a 30cm/12in high-sided round baking tin with butter and dust with sugar. Place the dough balls in neatly and set aside for two hours to rise. When the brioche has risen, mix together the egg yolk and milk and brush over the loaf gently with a pastry brush.
Put the brioche in the preheated oven for five minutes, then reduce the temperature to 180C/350F/Gas 4 and bake for a further 25 minutes, or until it is golden-brown and sounds hollow when tapped underneath. Transfer to a cooling rack.
When the loaf is cool, remove from the tin and spread the cream cheese icing on the top of each bun. Allow the icing to set a little before serving.


Support for the Microsoft SQL Server database.
The MSSQL dialect will work with three different available drivers:
Drivers are loaded in the order listed above based on availability.
If you need to load a specific driver pass module_name when creating the engine:
engine = create_engine('mssql://dsn', module_name='pymssql')
module_name currently accepts: pyodbc, pymssql, and adodbapi.
Currently the pyodbc driver offers the greatest level of compatibility.
Connecting with create_engine() uses the standard URL approach of mssql://user:pass@host/dbname[?key=value&key=value...].
If the database name is present, the tokens are converted to a connection string with the specified values. If the database is not present, then the host token is taken directly as the DSN name.
Examples of pyodbc connection string URLs:
mssql://mydsn - connects using the specified DSN named mydsn. The connection string that is created will appear like:
dsn=mydsn;TrustedConnection=Yes
mssql://user:pass@mydsn - connects using the DSN named mydsn passing in the UID and PWD information. The connection string that is created will appear like:
dsn=mydsn;UID=user;PWD=pass
mssql://user:pass@mydsn/?LANGUAGE=us_english - connects using the DSN named mydsn passing in the UID and PWD information, plus the additional connection configuration option LANGUAGE. The connection string that is created will appear like:
dsn=mydsn;UID=user;PWD=pass;LANGUAGE=us_english
mssql://user:pass@host/db - connects using a connection string dynamically created that would appear like:
DRIVER={SQL Server};Server=host;Database=db;UID=user;PWD=pass
mssql://user:pass@host:123/db - connects using a connection string that is dynamically created, which also includes the port information using the comma syntax. If your connection string requires the port information to be passed as a port keyword see the next example. This will create the following connection string:
DRIVER={SQL Server};Server=host,123;Database=db;UID=user;PWD=pass
mssql://user:pass@host/db?port=123 - connects using a connection string that is dynamically created that includes the port information as a separate port keyword. This will create the following connection string:
DRIVER={SQL Server};Server=host;Database=db;UID=user;PWD=pass;port=123
If you require a connection string that is outside the options presented above, use the odbc_connect keyword to pass in a urlencoded connection string. What gets passed in will be urldecoded and passed directly.
For example:
mssql:///?odbc_connect=dsn%3Dmydsn%3BDatabase%3Ddb
would create the following connection string:
dsn=mydsn;Database=db
Encoding your connection string can be easily accomplished through the python shell. For example:
>>> import urllib
>>> urllib.quote_plus('dsn=mydsn;Database=db')
'dsn%3Dmydsn%3BDatabase%3Ddb'
Additional arguments which may be specified either as query string arguments on the URL, or as keyword argument to create_engine() are:
IDENTITY columns are supported by using SQLAlchemy schema.Sequence() objects. In other words:
Table('test', mss_engine,
Column('id', Integer,
Sequence('blah',100,10), primary_key=True),
Column('name', String(20))
).create()
would yield:
CREATE TABLE test (
id INTEGER NOT NULL IDENTITY(100,10) PRIMARY KEY,
name VARCHAR(20) NULL,
)
Note that the start and increment values for sequences are optional and will default to 1,1.
MSSQL specific string types support a collation parameter that creates a column-level specific collation for the column. The collation parameter accepts a Windows Collation Name or a SQL Collation Name. Supported types are MSChar, MSNChar, MSString, MSNVarchar, MSText, and MSNText. For example:
Column('login', String(32, collation='Latin1_General_CI_AS'))
will yield:
login VARCHAR(32) COLLATE Latin1_General_CI_AS NULL
MSSQL has no support for the LIMIT or OFFSET keysowrds. LIMIT is supported directly through the TOP Transact SQL keyword:
select.limit
will yield:
SELECT TOP n
If the has_window_funcs flag is set then LIMIT with OFFSET support is available through the ROW_NUMBER OVER construct. This construct requires an ORDER BY to be specified as well and is only available on MSSQL 2005 and later.
MSSQL has support for three levels of column nullability. The default nullability allows nulls and is explicit in the CREATE TABLE construct:
name VARCHAR(20) NULL
If nullable=None is specified then no specification is made. In other words the database’s configured default is used. This will render:
name VARCHAR(20)
If nullable is True or False then the column will be NULL` or ``NOT NULL respectively.
For MSSQL versions that support the DATE and TIME types (MSSQL 2008+) the data type is used. For versions that do not support the DATE and TIME types a DATETIME type is used instead and the MSSQL dialect handles converting the results properly. This means Date() and Time() are fully supported on all versions of MSSQL. If you do not desire this behavior then do not use the Date() or Time() types.
MSSQL supports the notion of setting compatibility levels at the database level. This allows, for instance, to run a database that is compatibile with SQL2000 while running on a SQL2005 database server. server_version_info will always retrun the database server version information (in this case SQL2005) and not the compatibiility level information. Because of this, if running under a backwards compatibility mode SQAlchemy may attempt to use T-SQL statements that are unable to be parsed by the database server.


Everyone has a certain social identity. We all belong to certain groups and social categories, such as our age, sex, race, religion, nationality, etc. Along with the social identity, comes stereotype threat that can hurt your academic or professional performance. I am male, of Asian descent, and affected by the stereotype threat in the tech industry. Asian-Americans make up a large portion of the technology workforce. Thus, stereotypes that Asians are good at math and belong in the tech field are reiterated. This negatively impacts me, because I've identified that some people may make assumptions about my skill set based on a generic identification with my race. I've encountered situations where others have identified that I can excel only at a specific set of tasks because of my racial identity. In these situations I've been expected to perform at a higher level at some tasks and never been given the opportunity to solve other problems that are not associated with those tasks because of my race. If I were to feel a stereotype threat, I think the best course of action for me is to just disregard those threats because I am self-driven and I can't have stereotypes create a road block or slow me down. I've learned that success in life is not determined by the initial assessment strangers may develop about me. But, about my resilience to continually develop and challenge myself to improve.


___
Judging from amounts, it almost seems as if a partial fill alice turned into a bob for the remainder
ph03nix24-Jun-18 11:51 PM
g.1 f.1
jl777c25-Jun-18 05:03 AM
how far did that trade progress?
CrisF25-Jun-18 08:54 AM
Hi, Is it normal for many of these messages in terminal when using BarterDEX-cli?
<<<<<<<<<<< A bitcoind_RPC: LTC post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: LTC post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: LTC post_process_bitcoind_RPC.getinfo
<<<<<<<<<<< A bitcoind_RPC: ZEC post_process_bitcoind_RPC.getinfo
<<<<<<<<<<< A bitcoind_RPC: BCH post_process_bitcoind_RPC.getinfo
<<<<<<<<<<< A bitcoind_RPC: BCH post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: BCH post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: BCH post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: BCH post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: ZEC post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: ZEC post_process_bitcoind_RPC.validateaddress
<<<<<<<<<<< A bitcoind_RPC: ZEC post_process_bitcoind_RPC.validateaddress
jl777c25-Jun-18 09:20 AM
@ComputerGenie ok, I think I really fixed the spontaneaous bob issue. internally now there are separate sets of prices for alice vs bob, so if you are not doing bob things like autoprice there wont be any bob price to ordermatch
before there was a single set of prices so if you did an alice order, it ended up with a bob price for the reverse for a short period of time and if an incoming alice request came in, it could have started the ordermatch process. But I think it would not have completed as there wasnt a pending alice order i that direction.
now it wont even start
1
jl777 branch
jl777c25-Jun-18 10:29 AM
added persistence to gtc orders, but it is untested. if it works or doesnt, let me know. basically it will restore any pending GTC order, but it is quite inefficient for now if you have a lot of orders
SHossain25-Jun-18 11:05 AM
my first try got me time expired for alice request. using electrum KMD and BTC
BTC/KMD ignore as qprice 3797.24912360 vs myprice 3885.11223442
bob ignored it as the price was too low
so it expired
this is expected behavior
to "fix" this would require forcing bob to accept whatever price alice offers and while alice would be quite happy to have all her orders filled (regardless of price) it is possible some bobs will complain
SHossain25-Jun-18 11:13 AM
i increased it to 4000. now received a quote
>>>>>>>>>> received quote BTC/KMD 3613.15766178
ComputerGenie25-Jun-18 11:36 AM
anyone know what the fix is for the BarterDEX GUI when it imports the coinsdb list but not the images?
SHossain25-Jun-18 11:37 AM
restart BarterDEX GUI
that should fix it. after adding any new coins using the manage coin function, you need to restart the GUI for the changes to take effect
ComputerGenie25-Jun-18 11:39 AM
I tried that 4 or 5 times, it didn't dl any of the icons
SHossain25-Jun-18 11:40 AM
which coins are missing?
might as well try restarting your computer
ComputerGenie25-Jun-18 11:41 AM
pretty much all of them are gone and restarting is not an option
using dev branch, btw
SHossain25-Jun-18 11:48 AM
strange! i don't see this issue in Ubuntu
just tested
ComputerGenie25-Jun-18 11:52 AM
I'm going to dump and try clean, may be an issue updating from 1.06 to 1.1
GatewayBOT25-Jun-18 11:54 AM
[matrix] <shossain:matrix.org> i also used git reset --hard then updated and used my locally built fresh mm. testing the gtc and fill orders.
ComputerGenie25-Jun-18 12:00 PM
also, anyone know where the db read timer is? the console output is making me dizzy
cipi25-Jun-18 12:45 PM
removing ~/.BarterDEX and restarting might help
ComputerGenie25-Jun-18 12:50 PM
That made it worse, although deling CoinsDB did get etomic this time
cipi25-Jun-18 12:54 PM
there is also a reset coinsdb-button somewhere on the coins-management page...
ComputerGenie25-Jun-18 12:59 PM
ya, I found that and it works if I add each manually, but I couldn't get it to add all (I think it just hates me)
cipi25-Jun-18 01:01 PM
all? you mean all coins? that will not work since there are more then 255 coins and the limit is 255... that is why all this was made, to prevent having more then 255 coins in the coins-file
ComputerGenie25-Jun-18 01:01 PM
"all" meaning any default set larger than just KMD, BTC, and etomic
or is that the intention (just 3 and manually add anything else)?
cipi25-Jun-18 01:03 PM
yes, there is a default collection, i think it is BTC, KMD and ETOMIC and every other coin must be added manually to your collection... and you have to restart BarterDEX after adding coins
ComputerGenie25-Jun-18 01:04 PM
fek me, I shoulda backed up my folder before doing a pull
cipi25-Jun-18 01:06 PM
and when you add coins with the coins-management-page they are only added to coins.json and the icons and stuff is downloaded... you will have to also enable them as usualk after restarting BarterDEX
ComputerGenie25-Jun-18 01:08 PM
So, we took a nice, ordered list (that was conveniently separated into 3 categories) and turned it into an alphabetical free-for-all guessing game
goes to grab a coffee and start working on his stratum
CHMEX25-Jun-18 01:31 PM
is it technically necessary for the orderbook to have the corresponding coin enabled? (does it scan the chain for certain info) or would it be possible to show the orderbooks even the coin is not enabled (like it's now possible for price without an enabled coin)
jl777c25-Jun-18 01:32 PM
it is possible
CHMEX25-Jun-18 01:34 PM
that would be great than I could than give blocfolio all orderbooks currently it would be limited to the the ones with electrum support
jl777c25-Jun-18 01:35 PM
its there in jl777
plz test, as i didnt have time to test it
CHMEX25-Jun-18 01:37 PM
wow thanks you program faster than I type here
I will test it
jl777c25-Jun-18 01:38 PM
it took me twice as long as I had a pending syntax error from a different code
CHMEX25-Jun-18 01:39 PM
always a joy
grewalsatinder25-Jun-18 01:43 PM
we should have video tutorial of coinsdb feature to clear on it's needs and how to use it.
ComputerGenie25-Jun-18 01:46 PM
imo, if you need a video tutorial in order to explain a required basic component, then'it's likely too complex
grewalsatinder25-Jun-18 01:47 PM
that's all I could do.
cipi25-Jun-18 01:49 PM
i don't think it is too complex... it is a tool for editing configs and after you edit a config you have to restart the application for the changes to take effect... that's all... it is not a mars rocket
ComputerGenie25-Jun-18 01:50 PM
it's ya'll's boat; I'm just along for the ride
CHMEX25-Jun-18 02:05 PM
orderbooks without enabled coin works thanks
jl777c25-Jun-18 02:06 PM
it was a lot of work, but worth it!
i might have broken the CHIPS, it seems the miners dont like the tx in the mempool
no, I mean it was found after I fired the stratum up; my miners hadn't even come off the failover yet
CHMEX25-Jun-18 02:17 PM
ow I see
jl777c25-Jun-18 02:24 PM
hopefully the diff will come back down to where CPU mining can be done again
ComputerGenie25-Jun-18 02:25 PM
a long way to go before it gets there
jl777c25-Jun-18 04:02 PM
pushed initial version of mpnet support. this should make the networking nearly 100% reliable, even in windows, at least to the point where the ordermatch is done. I dont have mpnet active for the actual canbind:1 direct comms yet, wanted to make sure the basics work first
i use CHIPS mempool as the network, mempool network mpnet
it also goes into the blocks so we can see the gtc orders that are recently placed
you just need to have CHIPS enabled and some small amounts
so even for win10, if it can sync CHIPS, it will get a reliable connection
siu - Chainmakers25-Jun-18 04:38 PM
for any other trading pair that is not chips?
jl777c25-Jun-18 04:39 PM
yes, CHIPS is just the network
grewalsatinder25-Jun-18 04:39 PM
so to have more reliable mm on win10 user should have CHIPS enabled in any mode?
jl777c25-Jun-18 04:40 PM
i only tested native, but for alice SPV might be enough
grewalsatinder25-Jun-18 04:41 PM
so do I get this correct that barterdex p2p netwok which is individual, is now either partially dependent or more dependent on CHIPS's p2p network in case if it's windows ?
jl777c25-Jun-18 04:42 PM
it acts as a backup comm link
grewalsatinder25-Jun-18 04:42 PM
okay
jl777c25-Jun-18 04:42 PM
and any OS can use this
even without any CHIPS if it is enabled you can see the packets from the other nodes
grewalsatinder25-Jun-18 04:43 PM
nice!
jl777c25-Jun-18 04:44 PM
I have seen occasional lost packets via nanomsg and this will fix those cases, i think the ordermatch will have almost no networking issues after this is debugged
it will print MPNET.(json ) if it gets a CHIPS packet
and you will see an opreturn withdraw when you send normal ordermatch packets (if CHIPS enabled and funded)
grewalsatinder25-Jun-18 04:46 PM
any recommendations for BarterDEX GUI if I should have CHIPS autoenabled like KMD/BTC/ETOMIC in app ?
jl777c25-Jun-18 04:47 PM
if it is in the system, probably a good idea to enable it
grewalsatinder25-Jun-18 04:48 PM
okay will add that there in default coins list.
jl777c25-Jun-18 04:49 PM
if SPV it shouldnt be automatic though
it could slow things down, not sure
grewalsatinder25-Jun-18 04:50 PM
clear
then I'll have it added to default coins list which shows on first startup like on fresh install and let user decide which mode they want to have it setup to autoenable it to.
GatewayBOT25-Jun-18 04:54 PM
[matrix] <siu:matrix.org> how do i cancel a gtc order?